RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems

5 June 2023

Papers citing "RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems"

50 / 110 papers shown

Title
DataSculpt: Crafting Data Landscapes for Long-Context LLMs through Multi-Objective Partitioning Keer Lu Xiaonan Nie Zheng Liang Zhuoran Zhang Da Pan ... Weipeng Chen Zenan Zhou Guosheng Dong Bin Cui Wentao Zhang 39 0 0 02 Sep 2024
Statically Contextualizing Large Language Models with Typed Holes Andrew Blinn Xiang Li June Hyung Kim Cyrus Omar 50 2 0 02 Sep 2024
Self-evolving Agents with reflective and memory-augmented abilities Xuechen Liang Yangfan He Yinghui Xia Xinyuan Song Jianhui Wang ... Keqin Li Jiaqi Chen Jinsong Yang Siyuan Chen Tianyu Shi LLMAG KELM CLL 46 2 0 01 Sep 2024
SWE-bench-java: A GitHub Issue Resolving Benchmark for Java Daoguang Zan Zhirong Huang Ailun Yu Shaoxin Lin Yifan Shi ... Bei Guan Pengjie Huang Tao Xie Yongji Wang Qianxiang Wang 33 10 0 26 Aug 2024
CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding? Yuwei Zhao Ziyang Luo Yuchen Tian Hongzhan Lin Weixiang Yan Annan Li Jing Ma ELM ALM LRM 50 8 0 20 Aug 2024
Retrieval-augmented code completion for local projects using large language models Marko Hostnik Marko Robnik-Sikonja RALM 35 0 0 09 Aug 2024
CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases Xiangyan Liu Bo Lan Zhiyuan Hu Yang Liu Zhicheng Zhang Fei Wang Michael Shieh Ang Wang 44 16 0 07 Aug 2024
NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time Yilong Chen Guoxia Wang Junyuan Shang Shiyao Cui Zhenyu Zhang Tingwen Liu Shuohuan Wang Yu Sun Dianhai Yu Hua Wu 32 15 0 07 Aug 2024
Making Long-Context Language Models Better Multi-Hop Reasoners Yanyang Li Shuo Liang M. Lyu Liwei Wang LLMAG LRM 30 11 0 06 Aug 2024
Palu: Compressing KV-Cache with Low-Rank Projection Chi-Chih Chang Wei-Cheng Lin Chien-Yu Lin Chong-Yan Chen Yu-Fang Hu Pei-Shuo Wang N. Huang Luis Ceze Kai-Chiang Wu 59 0 0 30 Jul 2024
Scaling Granite Code Models to 128K Context Matt Stallone Vaibhav Saxena Leonid Karlinsky Bridget McGinn Tim Bula ... Rogerio Feris Nirmit Desai David D. Cox Ruchir Puri Yikang Shen 45 4 0 18 Jul 2024
On Mitigating Code LLM Hallucinations with API Documentation Nihal Jain Robert Kwiatkowski Baishakhi Ray M. K. Ramanathan Varun Kumar 41 7 0 13 Jul 2024
CodeUpdateArena: Benchmarking Knowledge Editing on API Updates Zeyu Leo Liu Shrey Pandit Xi Ye Eunsol Choi Greg Durrett KELM ALM 81 4 0 08 Jul 2024
Let the Code LLM Edit Itself When You Edit the Code Zhenyu He Jun Zhang Shengjie Luo Jingjing Xu Z. Zhang Di He KELM 39 0 0 03 Jul 2024
Agentless: Demystifying LLM-based Software Engineering Agents Chunqiu Steven Xia Yinlin Deng Soren Dunn Lingming Zhang LLMAG 43 86 0 01 Jul 2024
Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLP Omer Goldman Alon Jacovi Aviv Slobodkin Aviya Maimon Ido Dagan Reut Tsarfaty 69 11 0 29 Jun 2024
Hierarchical Context Pruning: Optimizing Real-World Code Completion with Repository-Level Pretrained Code LLMs Lei Zhang Yunshui Li Jiaming Li Xiaobo Xia Jiaxi Yang Run Luo Minzheng Wang Longze Chen Junhao Liu Min Yang 40 2 0 26 Jun 2024
Qiskit HumanEval: An Evaluation Benchmark For Quantum Code Generative Models Sanjay Vishwakarma Francis Harkins Siddharth Golecha Vishal Sharathchandra Bajpe Nicolas Dupuis Luca Buratti David Kremer Ismael Faro Ruchir Puri Juan Cruz-Benito ELM 55 3 0 20 Jun 2024
DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence DeepSeek-AI Qihao Zhu Daya Guo Zhihong Shao Dejian Yang ... Jiashi Li Chenggang Zhao Chong Ruan Fuli Luo Wenfeng Liang MoE LRM ELM VLM 48 170 0 17 Jun 2024
AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodology Minh Huynh Nguyen Thang Phan Chau Phong X. Nguyen Nghi D. Q. Bui 37 12 0 16 Jun 2024
Benchmarking Generative Models on Computational Thinking Tests in Elementary Visual Programming Victor-Alexandru Pădurean Adish Singla ELM 56 3 0 14 Jun 2024
DCA-Bench: A Benchmark for Dataset Curation Agents Benhao Huang Yingzhuo Yu Jin Huang Xingjian Zhang Jiaqi Ma 36 1 0 11 Jun 2024
RepoQA: Evaluating Long Context Code Understanding Jiawei Liu Jia Le Tian Vijay Daita Yuxiang Wei Yifeng Ding Yuhan Katherine Wang Jun Yang Lingming Zhang LLMAG 44 18 0 10 Jun 2024
Enhancing Repository-Level Code Generation with Integrated Contextual Information Zhiyuan Pan Xing Hu Xin Xia Xiaohu Yang 34 4 0 05 Jun 2024
Chain of Agents: Large Language Models Collaborating on Long-Context Tasks Yusen Zhang Ruoxi Sun Yanfei Chen Tomas Pfister Rui Zhang Sercan Ö. Arik RALM AI4CE LLMAG 59 30 0 04 Jun 2024
PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling Zefan Cai Yichi Zhang Bofei Gao Yuliang Liu Yong Li ... Wayne Xiong Yue Dong Baobao Chang Junjie Hu Wen Xiao 75 86 0 04 Jun 2024
A Survey on Large Language Models for Code Generation Juyong Jiang Fan Wang Jiasi Shen Sungju Kim Sunghun Kim 56 169 0 01 Jun 2024
Divide-and-Conquer Meets Consensus: Unleashing the Power of Functions in Code Generation Jingchang Chen Hongxuan Tang Zheng Chu Qianglong Chen Zekun Wang Ming Liu Bing Qin 55 4 0 30 May 2024
DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories Jia Li Ge Li Yunfei Zhao Yongming Li Huanyu Liu ... Yihong Dong Zhi Jin Binhua Li Fei Huang Yongbin Li ALM 35 26 0 30 May 2024
Dataflow-Guided Retrieval Augmentation for Repository-Level Code Completion Wei Cheng Yuhan Wu Wei Hu 38 11 0 30 May 2024
RTL-Repo: A Benchmark for Evaluating LLMs on Large-Scale RTL Design Projects Ahmed Allam Mohamed Shalan 43 15 0 27 May 2024
MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generation Jianbo Dai Jianqiao Lu Yunlong Feng Rongju Ruan Ming Cheng Haochen Tan Zhijiang Guo ELM LRM 44 12 0 19 May 2024
Granite Code Models: A Family of Open Foundation Models for Code Intelligence Mayank Mishra Matt Stallone Gaoyuan Zhang Songlin Yang Aditya Prasad ... Amith Singhee Nirmit Desai David D. Cox Ruchir Puri Yikang Shen AI4TS 63 58 0 07 May 2024
On the Limitations of Embedding Based Methods for Measuring Functional Correctness for Code Generation Atharva Naik 48 2 0 26 Apr 2024
CORM: Cache Optimization with Recent Message for Large Language Model Inference Jincheng Dai Zhuowei Huang Haiyun Jiang Chen Chen Deng Cai Wei Bi Shuming Shi 38 3 0 24 Apr 2024
Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks Chonghua Wang Haodong Duan Songyang Zhang Dahua Lin Kai-xiang Chen ELM 31 17 0 09 Apr 2024
EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories Jia Li Ge Li Xuanming Zhang Yihong Dong Zhi Jin 34 33 0 31 Mar 2024
Iterative Refinement of Project-Level Code Context for Precise Code Generation with Compiler Feedback Zhangqian Bi Yao Wan Zheng Wang Hongyu Zhang Batu Guan Fangxin Lu Zili Zhang Yulei Sui Hai Jin Xuanhua Shi 37 14 0 25 Mar 2024
CodeS: Natural Language to Code Repository via Multi-Layer Sketch Daoguang Zan Ailun Yu Wei Liu Dong Chen Bo Shen ... Bei Guan Zhiguang Yang Yongji Wang Qianxiang Wang Li-zhen Cui 35 14 0 25 Mar 2024
DevBench: A Comprehensive Benchmark for Software Development Bowen Li Wenhan Wu Ziwei Tang Lin Shi John Yang ... He Du Ping Yang Dahua Lin Chao Peng Kai Chen 99 9 0 13 Mar 2024
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code Naman Jain King Han Alex Gu Wen-Ding Li Fanjia Yan Tianjun Zhang Sida I. Wang Armando Solar-Lezama Koushik Sen Ion Stoica ELM 36 306 0 12 Mar 2024
InfiBench: Evaluating the Question-Answering Capabilities of Code Large Language Models Linyi Li Shijie Geng Zhenwen Li Yibo He Hao Yu Ziyue Hua Guanghan Ning Siwei Wang Tao Xie Hongxia Yang ELM 37 2 0 11 Mar 2024
RepoHyper: Better Context Retrieval Is All You Need for Repository-Level Code Completion Huy N. Phan Hoang N. Phan Tien N. Nguyen Nghi D. Q. Bui 50 3 0 10 Mar 2024
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks Linyuan Gong Sida Wang Mostafa Elhoushi Alvin Cheung 32 15 0 07 Mar 2024
LongWanjuan: Towards Systematic Measurement for Long Text Quality Kai Lv Xiaoran Liu Qipeng Guo Hang Yan Conghui He Xipeng Qiu Dahua Lin 33 4 0 21 Feb 2024
EffiBench: Benchmarking the Efficiency of Automatically Generated Code Dong Huang Yuhao Qing Weiyi Shang Heming Cui Jie M. Zhang 87 32 0 03 Feb 2024
Extending LLMs' Context Window with 100 Samples Yikai Zhang Junlong Li Pengfei Liu 37 11 0 13 Jan 2024
ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code Xiangru Tang Yuliang Liu Zefan Cai Yan Shao Junjie Lu ... Yujia Qin Wangchunshu Zhou Yilun Zhao Arman Cohan Mark B. Gerstein ELM LLMAG 46 18 0 16 Nov 2023
Prompt Cache: Modular Attention Reuse for Low-Latency Inference In Gim Guojun Chen Seung-seob Lee Nikhil Sarda Anurag Khandelwal Lin Zhong 42 77 0 07 Nov 2023
Bias Testing and Mitigation in LLM-based Code Generation Dong Huang Qingwen Bu Jie M. Zhang Xiaofei Xie Junjie Chen Heming Cui 51 21 0 03 Sep 2023