Unsupervised Evaluation of Code LLMs with Round-Trip Correctness

Unsupervised Evaluation of Code LLMs with Round-Trip Correctness

13 February 2024

Miltiadis Allamanis

Sheena Panthaplackel

Papers citing "Unsupervised Evaluation of Code LLMs with Round-Trip Correctness"

12 / 12 papers shown

Title
Natural Language Outlines for Code: Literate Programming in the LLM Era Kensen Shi Deniz Altınbüken Saswat Anand Mihai Christodorescu Katja Grünwedel ... Tobias Welp Pengcheng Yin Manzil Zaheer Satish Chandra Charles Sutton 83 6 0 09 Aug 2024
CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion Yangruibo Ding Zijian Wang Wasi Uddin Ahmad Hantian Ding Ming Tan ... M. K. Ramanathan Ramesh Nallapati Parminder Bhatia Dan Roth Bing Xiang ELM 72 125 0 17 Oct 2023
OctoPack: Instruction Tuning Code Large Language Models Niklas Muennighoff Qian Liu A. Zebaze Qinkai Zheng Binyuan Hui Terry Yue Zhuo Swayam Singh Xiangru Tang Leandro von Werra Shayne Longpre VLM ALM 93 125 0 14 Aug 2023
Faithfulness Tests for Natural Language Explanations Pepa Atanasova Oana-Maria Camburu Christina Lioma Thomas Lukasiewicz J. Simonsen Isabelle Augenstein FAtt 54 64 0 29 May 2023
Natural Language to Code Generation in Interactive Data Science Notebooks Pengcheng Yin Wen-Ding Li Kefan Xiao Abhishek Rao Yeming Wen ... Paige Bailey Michele Catasta Henryk Michalewski Oleksandr Polozov Charles Sutton 44 60 0 19 Dec 2022
Competition-Level Code Generation with AlphaCode Yujia Li David Choi Junyoung Chung Nate Kushman Julian Schrittwieser ... Esme Sutherland Robson Pushmeet Kohli Nando de Koray Kavukcuoglu Oriol Vinyals 56 1,352 0 08 Feb 2022
Program Synthesis with Large Language Models Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski ... Ellen Jiang Carrie J. Cai Michael Terry Quoc V. Le Charles Sutton ELM AIMat ReCod ALM 97 1,893 0 16 Aug 2021
On Multi-Modal Learning of Editing Source Code Saikat Chakraborty Baishakhi Ray KELM 42 59 0 15 Aug 2021
Measuring Coding Challenge Competence With APPS Dan Hendrycks Steven Basart Saurav Kadavath Mantas Mazeika Akul Arora ... Collin Burns Samir Puranik Horace He D. Song Jacob Steinhardt ELM AIMat ALM 229 657 0 20 May 2021
CodeBLEU: a Method for Automatic Evaluation of Code Synthesis Shuo Ren Daya Guo Shuai Lu Long Zhou Shujie Liu Duyu Tang Neel Sundaresan M. Zhou Ambrosio Blanco Shuai Ma ELM 86 517 0 22 Sep 2020
JuICe: A Large Scale Distantly Supervised Dataset for Open Domain Context-based Code Generation R. Agashe R. Campello Arthur Zimek 57 83 0 05 Oct 2019
Improving Neural Machine Translation Models with Monolingual Data Rico Sennrich Barry Haddow Alexandra Birch 216 2,710 0 20 Nov 2015