Title
Visualization Generation with Large Language Models: An Evaluation Guozheng Li Xinyu Wang Gerile Aodeng Shunyuan Zheng Yu Zhang Chuangxin Ou Song Wang Chi Harold Liu 29 27 0 20 Jan 2024
Knowledge Fusion of Large Language Models Fanqi Wan Xinting Huang Deng Cai Xiaojun Quan Wei Bi Shuming Shi MoMe 40 63 0 19 Jan 2024
JumpCoder: Go Beyond Autoregressive Coder via Online Modification Mouxiang Chen Hao Tian Zhongxi Liu Xiaoxue Ren Jianling Sun SyDa KELM 43 2 0 15 Jan 2024
PythonSaga: Redefining the Benchmark to Evaluate Code Generating LLMs Ankit Yadav Himanshu Beniwal Mayank Singh LRM ALM 33 12 0 08 Jan 2024
Instruction Fusion: Advancing Prompt Evolution through Hybridization Weidong Guo Jiuding Yang Kaitong Yang Xiangyang Li Zhuwei Rao Yu-Syuan Xu Di Niu 15 5 0 25 Dec 2023
ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code Xiangru Tang Yuliang Liu Zefan Cai Yan Shao Junjie Lu ... Yujia Qin Wangchunshu Zhou Yilun Zhao Arman Cohan Mark B. Gerstein ELM LLMAG 46 18 0 16 Nov 2023
CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation Weixiang Yan Haitian Liu Yunkun Wang Yunzhe Li Qian Chen ... Tingyu Lin Weishan Zhao Li Zhu Hari Sundaram Shuiguang Deng ELM LRM 34 35 0 14 Nov 2023
Data Augmentation for Code Translation with Comparable Corpora and Multiple References Yiqing Xie Atharva Naik Daniel Fried Carolyn Rose 54 6 0 01 Nov 2023
SWE-bench: Can Language Models Resolve Real-World GitHub Issues? Carlos E. Jimenez John Yang Alexander Wettig Shunyu Yao Kexin Pei Ofir Press Karthik Narasimhan ELM 34 469 0 10 Oct 2023
Code Llama: Open Foundation Models for Code Baptiste Rozière Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat ... Hugo Touvron Louis Martin Nicolas Usunier Thomas Scialom Gabriel Synnaeve ELM ALM 63 1,906 0 24 Aug 2023
Benchmarking Causal Study to Interpret Large Language Models for Source Code Daniel Rodríguez-Cárdenas David Nader-Palacio Dipin Khati Henry Burke Denys Poshyvanyk CML ELM 30 15 0 23 Aug 2023
Reflexion: Language Agents with Verbal Reinforcement Learning Noah Shinn Federico Cassano Beck Labash A. Gopinath Karthik Narasimhan Shunyu Yao LLMAG KELM 11 1,119 0 20 Mar 2023
xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval Mohammad Abdullah Matin Khan M Saiful Bari Xuan Long Do Weishi Wang Md. Rizwan Parvez Chenyu You ALM ELM 34 14 0 06 Mar 2023
Large Language Models for Code: Security Hardening and Adversarial Testing Jingxuan He Martin Vechev ELM AAML 23 108 0 10 Feb 2023
Multi-lingual Evaluation of Code Generation Models Ben Athiwaratkun Sanjay Krishna Gouda Zijian Wang Xiaopeng Li Yuchen Tian ... Baishakhi Ray Parminder Bhatia Sudipta Sengupta Dan Roth Bing Xiang ELM 120 161 0 26 Oct 2022
Productivity Assessment of Neural Code Completion Albert Ziegler Eirini Kalliamvakou Shawn Simister Ganesh Sittampalam Alice Li Andrew Rice Devon Rifkin E. Aftandilian 102 177 0 13 May 2022
A Systematic Evaluation of Large Language Models of Code Frank F. Xu Uri Alon Graham Neubig Vincent J. Hellendoorn ELM ALM 204 631 0 26 Feb 2022
Measuring Coding Challenge Competence With APPS Dan Hendrycks Steven Basart Saurav Kadavath Mantas Mazeika Akul Arora ... Collin Burns Samir Puranik Horace He D. Song Jacob Steinhardt ELM AIMat ALM 208 627 0 20 May 2021
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation Shuai Lu Daya Guo Shuo Ren Junjie Huang Alexey Svyatkovskiy ... Nan Duan Neel Sundaresan Shao Kun Deng Shengyu Fu Shujie Liu ELM 204 853 0 09 Feb 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 282 1,996 0 31 Dec 2020