How Efficient is LLM-Generated Code? A Rigorous & High-Standard Benchmark

20 February 2025

Papers citing "How Efficient is LLM-Generated Code? A Rigorous & High-Standard Benchmark"

47 / 47 papers shown

Title
Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization Mingzhe Du Luu Tuan Tuan Yue Liu Yuhao Qing Dong Huang Xinyi He Qian Liu Zejun Ma See-Kiong Ng 39 0 0 29 May 2025
Breaking Silos: Adaptive Model Fusion Unlocks Better Time Series Forecasting Zhining Liu Ze Yang Xiao Lin Ruizhong Qiu Tianxin Wei Yada Zhu Hendrik Hamann Jingrui He Hanghang Tong AI4TS 82 0 0 24 May 2025
Ask, and it shall be given: On the Turing completeness of prompting Ruizhong Qiu Zhe Xu Wenxuan Bao Hanghang Tong ReLM LRM AI4CE 129 1 0 24 Feb 2025
LLM4EFFI: Leveraging Large Language Models to Enhance Code Efficiency and Correctness Tong Ye Weigang Huang Xinsong Zhang Tengfei Ma Peiyu Liu Yuxiang Cai Wenhai Wang LLMAG 103 3 0 17 Feb 2025
LLM Program Optimization via Retrieval Augmented Search Sagnik Anupam Alexander Shypula Osbert Bastani 190 1 0 31 Jan 2025
How Should We Build A Benchmark? Revisiting 274 Code-Related Benchmarks For LLMs Jialun Cao Yuk-Kit Chan Zixuan Ling Wenxuan Wang Shuqing Li ... Pinjia He Shuai Wang Zibin Zheng Michael R. Lyu Shing-Chi Cheung ALM 142 2 0 18 Jan 2025
CKGFuzzer: LLM-Based Fuzz Driver Generation Enhanced By Code Knowledge Graph HanXiang Xu Wei Ma Ting Zhou Yanjie Zhao Kai Chen Q. Hu Yang Liu Haoyu Wang 104 0 0 18 Nov 2024
WAPITI: A Watermark for Finetuned Open-Source LLMs Lingjie Chen Ruizhong Qiu Siyu Yuan Zhining Liu Tianxin Wei Hyunsik Yoo Zhichen Zeng Deqing Yang Hanghang Tong WaLM 85 6 0 09 Oct 2024
BACKTIME: Backdoor Attacks on Multivariate Time Series Forecasting Xiao Lin Zhining Liu Dongqi Fu Ruizhong Qiu Hanghang Tong AAML AI4TS 73 11 0 03 Oct 2024
Fair Anomaly Detection For Imbalanced Groups Ziwei Wu Lecheng Zheng Yuancheng Yu Ruizhong Qiu John Birge Jingrui He FaML 45 6 0 17 Sep 2024
AIM: Attributing, Interpreting, Mitigating Data Unfairness Zhining Liu Ruizhong Qiu Zhichen Zeng Yada Zhu Hendrik Hamann Hanghang Tong FaML 67 6 0 13 Jun 2024
Gradient Compressed Sensing: A Query-Efficient Gradient Estimator for High-Dimensional Zeroth-Order Optimization Ruizhong Qiu Hanghang Tong 77 5 0 27 May 2024
Discrete-state Continuous-time Diffusion for Graph Generation Zhe Xu Ruizhong Qiu Yuzhong Chen Huiyuan Chen Xiran Fan Menghai Pan Zhichen Zeng Mahashweta Das Hanghang Tong 105 20 0 19 May 2024
Logic Query of Thoughts: Guiding Large Language Models to Answer Complex Logic Queries with Knowledge Graphs Lihui Liu Zihao Wang Ruizhong Qiu Yikun Ban Eunice Chan Yangqiu Song Jingrui He Hanghang Tong LRM 40 8 0 17 Mar 2024
Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and Beyond Tianxin Wei Bowen Jin Ruirui Li Hansi Zeng Zhengyang Wang ... Qingyu Yin Hanqing Lu Suhang Wang Jingrui He Xianfeng Tang 83 22 0 15 Mar 2024
DevBench: A Comprehensive Benchmark for Software Development Bowen Li Wenhan Wu Ziwei Tang Lin Shi John Yang ... He Du Ping Yang Dahua Lin Chao Peng Kai Chen 120 10 0 13 Mar 2024
EffiBench: Benchmarking the Efficiency of Automatically Generated Code Dong Huang Yuhao Qing Weiyi Shang Heming Cui Jie M. Zhang 111 37 0 03 Feb 2024
Can Large Language Models Write Parallel Code? Daniel Nichols Joshua H. Davis Zhaojun Xie Arjun Rajaram A. Bhatele LRM ELM ALM 51 21 0 23 Jan 2024
Mixtral of Experts Albert Q. Jiang Alexandre Sablayrolles Antoine Roux A. Mensch Blanche Savary ... Théophile Gervet Thibaut Lavril Thomas Wang Timothée Lacroix William El Sayed MoE LLMAG 151 1,085 0 08 Jan 2024
A Metadata-Driven Approach to Understand Graph Neural Networks Tinghong Li Qiaozhu Mei Jiaqi Ma AI4CE 76 5 0 30 Oct 2023
$$\mathcal{B}$-Coder: Value-Based Deep Reinforcement Learning for Program Synthesis$ $\mathcal{B}$ -Coder: Value-Based Deep Reinforcement Learning for Program Synthesis Zishun Yu Yunzhe Tao Liyu Chen Tao Sun Hongxia Yang 54 11 0 04 Oct 2023
Ensuring User-side Fairness in Dynamic Recommender Systems Hyunsik Yoo Zhichen Zeng Jian Kang Ruizhong Qiu David Zhou Zhining Liu Fei Wang Charlie Xu Eunice Chan Hanghang Tong 71 16 0 29 Aug 2023
Class-Imbalanced Graph Learning without Class Rebalancing Zhining Liu Ruizhong Qiu Zhichen Zeng Hyunsik Yoo David Zhou Zhe Xu Yada Zhu Kommy Weldemariam Jingrui He Hanghang Tong AI4CE 65 12 0 27 Aug 2023
Code Llama: Open Foundation Models for Code Baptiste Rozière Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat ... Hugo Touvron Louis Martin Nicolas Usunier Thomas Scialom Gabriel Synnaeve ELM ALM 108 2,037 0 24 Aug 2023
Graph Neural Bandits Yunzhe Qi Yikun Ban Jingrui He 55 20 0 21 Aug 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 299 11,894 0 18 Jul 2023
BeMap: Balanced Message Passing for Fair Graph Neural Network Xiao Lin Jian Kang Weilin Cong Hanghang Tong MoE 54 8 0 07 Jun 2023
Reconstructing Graph Diffusion History from a Single Snapshot Ruizhong Qiu Dingsu Wang Lei Ying H. Vincent Poor Yifang Zhang Hanghang Tong DiffM 90 7 0 01 Jun 2023
Networked Time Series Imputation via Position-aware Graph Enhanced Variational Autoencoders Dingsu Wang Yuchen Yan Ruizhong Qiu Yada Zhu Kaiyu Guan A. Margenot Hanghang Tong AI4TS 79 32 0 29 May 2023
CodeT5+: Open Code Large Language Models for Code Understanding and Generation Yue Wang Hung Le Akhilesh Deepak Gotmare Nghi D. Q. Bui Junnan Li Steven C. H. Hoi ALM 62 477 0 13 May 2023
StarCoder: may the source be with you! Raymond Li Loubna Ben Allal Yangtian Zi Niklas Muennighoff Denis Kocetkov ... Sean M. Hughes Thomas Wolf Arjun Guha Leandro von Werra H. D. Vries 104 768 0 09 May 2023
Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation Jiawei Liu Chun Xia Yuyao Wang Lingming Zhang ELM ALM 246 921 0 02 May 2023
Self-Refine: Iterative Refinement with Self-Feedback Aman Madaan Niket Tandon Prakhar Gupta Skyler Hallinan Luyu Gao ... Bodhisattwa Prasad Majumder Katherine Hermann Sean Welleck Amir Yazdanbakhsh Peter Clark ReLM LRM DiffM 147 1,638 0 30 Mar 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.4K 14,359 0 15 Mar 2023
DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation Yuhang Lai Chengxi Li Yiming Wang Tianyi Zhang Ruiqi Zhong Luke Zettlemoyer Scott Yih Daniel Fried Si-yi Wang Tao Yu ELM ALM 89 334 0 18 Nov 2022
DIMES: A Differentiable Meta Solver for Combinatorial Optimization Problems Ruizhong Qiu Zhiqing Sun Yiming Yang 109 87 0 08 Oct 2022
MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generation Federico Cassano John Gouwar Daniel Nguyen S. Nguyen Luna Phipps-Costin ... Carolyn Jane Anderson Molly Q. Feldman Arjun Guha Michael Greenberg Abhinav Jangda ELM 96 90 0 17 Aug 2022
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis Erik Nijkamp Bo Pang Hiroaki Hayashi Lifu Tu Haiquan Wang Yingbo Zhou Silvio Savarese Caiming Xiong ELM 150 1,023 0 25 Mar 2022
Competition-Level Code Generation with AlphaCode Yujia Li David Choi Junyoung Chung Nate Kushman Julian Schrittwieser ... Esme Sutherland Robson Pushmeet Kohli Nando de Koray Kavukcuoglu Oriol Vinyals 132 1,386 0 08 Feb 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 814 9,387 0 28 Jan 2022
EE-Net: Exploitation-Exploration Neural Networks in Contextual Bandits Yikun Ban Yuchen Yan A. Banerjee Jingrui He OffRL 63 40 0 07 Oct 2021
Program Synthesis with Large Language Models Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski ... Ellen Jiang Carrie J. Cai Michael Terry Quoc V. Le Charles Sutton ELM AIMat ReCod ALM 195 1,954 0 16 Aug 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 231 5,539 0 07 Jul 2021
Measuring Coding Challenge Competence With APPS Dan Hendrycks Steven Basart Saurav Kadavath Mantas Mazeika Akul Arora ... Collin Burns Samir Puranik Horace He D. Song Jacob Steinhardt ELM AIMat ALM 251 681 0 20 May 2021
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 795 42,055 0 28 May 2020
Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task Tao Yu Rui Zhang Kai-Chou Yang Michihiro Yasunaga Dongxu Wang ... Irene Li Qingning Yao Shanelle Roman Zilin Zhang Dragomir R. Radev RALM 95 1,233 0 24 Sep 2018
Neural-Guided Deductive Search for Real-Time Program Synthesis from Examples Ashwin J. Vijayakumar Abhishek Mohta Oleksandr Polozov Dhruv Batra Prateek Jain Sumit Gulwani NAI 97 166 0 03 Apr 2018