Evaluating Instruction-Tuned Large Language Models on Code Comprehension and Generation

2 August 2023

Xin Peng

Papers citing "Evaluating Instruction-Tuned Large Language Models on Code Comprehension and Generation"

27 / 27 papers shown

Title
Transferable text data distillation by trajectory matching Rong Yao Hailin Hu Yifei Fu Hanting Chen Wenyi Fang Fanyi Du Kai Han Yunhe Wang 54 0 0 14 Apr 2025
Turbulence: Systematically and Automatically Testing Instruction-Tuned Large Language Models for Code Shahin Honarvar Mark van der Wilk Alastair Donaldson 158 8 0 28 Jan 2025
MdEval: Massively Multilingual Code Debugging Shukai Liu Linzheng Chai Jian Yang Jiajun Shi He Zhu ... Yu Hao Liqun Yang Guanglin Niu Ge Zhang Zheng Li LRM ELM 115 7 0 04 Nov 2024
Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges Aman Singh Thakur Kartik Choudhary Venkat Srinik Ramayapally Sankaran Vaidyanathan Dieuwke Hupkes ELM ALM 119 64 0 18 Jun 2024
WizardCoder: Empowering Code Large Language Models with Evol-Instruct Ziyang Luo Can Xu Pu Zhao Qingfeng Sun Xiubo Geng Wenxiang Hu Chongyang Tao Jing Ma Qingwei Lin Daxin Jiang ELM SyDa ALM 79 674 0 14 Jun 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 312 4,253 0 09 Jun 2023
The False Promise of Imitating Proprietary LLMs Arnav Gudibande Eric Wallace Charles Burton Snell Xinyang Geng Hao Liu Pieter Abbeel Sergey Levine Dawn Song ALM 99 204 0 25 May 2023
AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback Yann Dubois Xuechen Li Rohan Taori Tianyi Zhang Ishaan Gulrajani Jimmy Ba Carlos Guestrin Percy Liang Tatsunori B. Hashimoto ALM 108 590 0 22 May 2023
Automating Code-Related Tasks Through Transformers: The Impact of Pre-training Rosalia Tufano L. Pascarella Gabriele Bavota 51 20 0 08 Feb 2023
Talking About Large Language Models Murray Shanahan AI4CE 86 266 0 07 Dec 2022
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model BigScience Workshop : Teven Le Scao Angela Fan Christopher Akiki ... Zhongli Xie Zifan Ye M. Bras Younes Belkada Thomas Wolf VLM 353 2,377 0 09 Nov 2022
Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writing Tuhin Chakrabarty Vishakh Padmakumar Hengxing He 49 77 0 25 Oct 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 167 3,110 0 20 Oct 2022
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng Zhang Yuxiao Dong Jie Tang BDL LRM 342 1,090 0 05 Oct 2022
No More Fine-Tuning? An Experimental Evaluation of Prompt Tuning in Code Intelligence Chaozheng Wang Yuanhang Yang Cuiyun Gao Yun Peng Hongyu Zhang Michael R. Lyu AAML 83 136 0 24 Jul 2022
Deep Learning Meets Software Engineering: A Survey on Pre-Trained Models of Source Code Changan Niu Chuanyi Li Bin Luo Vincent Ng SyDa VLM 88 50 0 24 May 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 420 6,202 0 05 Apr 2022
Training Compute-Optimal Large Language Models Jordan Hoffmann Sebastian Borgeaud A. Mensch Elena Buchatskaya Trevor Cai ... Karen Simonyan Erich Elsen Jack W. Rae Oriol Vinyals Laurent Sifre AI4TS 180 1,941 0 29 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 738 9,267 0 28 Jan 2022
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 116 3,723 0 03 Sep 2021
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation Yue Wang Weishi Wang Shafiq Joty Guosheng Lin 280 1,552 0 02 Sep 2021
Program Synthesis with Large Language Models Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski ... Ellen Jiang Carrie J. Cai Michael Terry Quoc V. Le Charles Sutton ELM AIMat ReCod ALM 171 1,925 0 16 Aug 2021
SynCoBERT: Syntax-Guided Multi-Modal Contrastive Pre-Training for Code Representation Xin Wang Yasheng Wang Fei Mi Pingyi Zhou Yao Wan Xiao Liu Li Li Hao Wu Jin Liu Xin Jiang 95 114 0 10 Aug 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 203 5,454 0 07 Jul 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 509 4,021 0 18 Apr 2021
Unified Pre-training for Program Understanding and Generation Wasi Uddin Ahmad Saikat Chakraborty Baishakhi Ray Kai-Wei Chang 122 764 0 10 Mar 2021
CodeBERT: A Pre-Trained Model for Programming and Natural Languages Zhangyin Feng Daya Guo Duyu Tang Nan Duan Xiaocheng Feng ... Linjun Shou Bing Qin Ting Liu Daxin Jiang Ming Zhou 140 2,613 0 19 Feb 2020