Bugs in Large Language Models Generated Code: An Empirical Study

13 March 2024

Florian Tambon

Arghavan Moradi Dakhel

Papers citing "Bugs in Large Language Models Generated Code: An Empirical Study"

24 / 24 papers shown

Title
Assessing LLM code generation quality through path planning tasks Wanyi Chen Meng-Wen Su Mary L. Cummings ELM 53 0 0 30 Apr 2025
Hallucination by Code Generation LLMs: Taxonomy, Benchmarks, Mitigation, and Challenges Yunseo Lee John Youngeun Song Dongsun Kim Jindae Kim Mijung Kim Jaechang Nam HILM LRM 42 0 0 29 Apr 2025
On Developers' Self-Declaration of AI-Generated Code: An Analysis of Practices Syed Mohammad Kashif Peng Liang Amjed Tahir 27 0 0 23 Apr 2025
Code Copycat Conundrum: Demystifying Repetition in LLM-based Code Generation Mingwei Liu Juntao Li Ying Wang Xueying Du Zuoyu Ou ... Zhao Wei Yanwei Xu Fangming Zou Xin Peng Yiling Lou 38 0 0 17 Apr 2025
CodeIF-Bench: Evaluating Instruction-Following Capabilities of Large Language Models in Interactive Code Generation Peiding Wang L. Zhang Fang Liu Lin Shi Minxiao Li Bo Shen An Fu ELM LRM 151 0 0 05 Mar 2025
Deep-Bench: Deep Learning Benchmark Dataset for Code Generation Alireza Daghighfarsoodeh Chung-Yu Wang Hamed Taherkhani Melika Sepidband Mohammad Abdollahi Hadi Hemmati Hung Viet Pham ALM ELM 96 0 0 26 Feb 2025
Turbulence: Systematically and Automatically Testing Instruction-Tuned Large Language Models for Code Shahin Honarvar Mark van der Wilk Alastair Donaldson 80 6 0 28 Jan 2025
Model Editing for LLMs4Code: How Far are We? Xiaopeng Li Shangwen Wang Shasha Li Jun Ma Jie Yu Xiaodong Liu Jing Wang Bin Ji Weimin Zhang KELM 39 2 0 11 Nov 2024
MdEval: Massively Multilingual Code Debugging Shukai Liu Linzheng Chai Jian Yang Jiajun Shi He Zhu ... Yu Hao Liqun Yang Guanglin Niu Ge Zhang Zheng Li LRM ELM 70 6 0 04 Nov 2024
A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why? QiHong Chen Jiawei Li Jiecheng Deng Jiachen Yu Justin Tian Jin Chen Iftekhar Ahmed 56 0 0 03 Nov 2024
One Step at a Time: Combining LLMs and Static Analysis to Generate Next-Step Hints for Programming Tasks Anastasiia Birillo Elizaveta Artser Anna Potriasaeva Ilya Vlasov Katsiaryna Dzialets Yaroslav Golubev Igor Gerasimov Hieke Keuning T. Bryksin 41 3 0 11 Oct 2024
RAMBO: Enhancing RAG-based Repository-Level Method Body Completion Tuan-Dung Bui Duc-Thieu Luu-Van Thanh-Phat Nguyen Thu-Trang Nguyen Son Nguyen H. Vo 34 4 0 23 Sep 2024
Understanding Defects in Generated Codes by Language Models Ali Mohammadi Esfahani N. Kahani S. Ajila 25 1 0 23 Aug 2024
DeepCodeProbe: Towards Understanding What Models Trained on Code Learn Vahid Majdinasab Amin Nikanjam Foutse Khomh 40 1 0 11 Jul 2024
What's Wrong with Your Code Generated by Large Language Models? An Extensive Study Shihan Dou Haoxiang Jia Shenxi Wu Huiyuan Zheng Weikang Zhou ... Xunliang Cai Tao Gui Xipeng Qiu Qi Zhang Xuanjing Huang 31 32 0 08 Jul 2024
Chain of Targeted Verification Questions to Improve the Reliability of Code Generated by LLMs Sylvain Kouemo Ngassom Arghavan Moradi Dakhel Florian Tambon Foutse Khomh 32 2 0 22 May 2024
When LLMs Meet Cybersecurity: A Systematic Literature Review Jie Zhang Haoyu Bu Hui Wen Yu Chen Lun Li Hongsong Zhu 42 36 0 06 May 2024
Enhancing Trust in LLM-Generated Code Summaries with Calibrated Confidence Scores Yuvraj Virk Prem Devanbu Toufique Ahmed 62 10 0 30 Apr 2024
Rethinking Software Engineering in the Foundation Model Era: From Task-Driven AI Copilots to Goal-Driven AI Pair Programmers Ahmed E. Hassan G. Oliva Dayi Lin Boyuan Chen Zhen Ming Jiang Jiang 33 14 0 16 Apr 2024
Exploring and Evaluating Hallucinations in LLM-Powered Code Generation Fang Liu Yang Liu Lin Shi Houkun Huang Ruifeng Wang Zhen Yang Li Zhang Zhongqi Li Yuchi Ma 52 108 0 01 Apr 2024
EffiBench: Benchmarking the Efficiency of Automatically Generated Code Dong Huang Yuhao Qing Weiyi Shang Heming Cui Jie M. Zhang 82 30 0 03 Feb 2024
Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation Jiawei Liu Chun Xia Yuyao Wang Lingming Zhang ELM ALM 183 791 0 02 May 2023
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 367 8,495 0 28 Jan 2022
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation Shuai Lu Daya Guo Shuo Ren Junjie Huang Alexey Svyatkovskiy ... Nan Duan Neel Sundaresan Shao Kun Deng Shengyu Fu Shujie Liu ELM 198 1,105 0 09 Feb 2021