CodeBLEU: a Method for Automatic Evaluation of Code Synthesis

22 September 2020

Papers citing "CodeBLEU: a Method for Automatic Evaluation of Code Synthesis"

45 / 245 papers shown

Title
ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages Yekun Chai Shuohuan Wang Chao Pang Yu Sun Hao Tian Hua Wu 30 35 0 13 Dec 2022
Who Evaluates the Evaluators? On Automatic Metrics for Assessing AI-based Offensive Code Generators Pietro Liguori Cristina Improta R. Natella B. Cukic Domenico Cotroneo ELM 36 16 0 12 Dec 2022
Parameter-Efficient Finetuning of Transformers for Source Code Shamil Ayupov Nadezhda Chirkova 22 17 0 12 Dec 2022
A Survey on Natural Language Processing for Programming Qingfu Zhu Xianzhen Luo Fang Liu Cuiyun Gao Wanxiang Che 25 1 0 12 Dec 2022
DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation Yuhang Lai Chengxi Li Yiming Wang Tianyi Zhang Ruiqi Zhong Luke Zettlemoyer Scott Yih Daniel Fried Si-yi Wang Tao Yu ELM ALM 32 309 0 18 Nov 2022
Execution-based Evaluation for Data Science Code Generation Models Junjie Huang Chenglong Wang Jipeng Zhang Cong Yan Haotian Cui J. Inala Colin B. Clement Nan Duan Jianfeng Gao ELM 41 35 0 17 Nov 2022
CodePAD: Sequence-based Code Generation with Pushdown Automaton Yihong Dong Xue Jiang Yuchen Liu Ge Li Zhi Jin 28 6 0 02 Nov 2022
Aligning Offline Metrics and Human Judgments of Value for Code Generation Models Victor C. Dibia Adam Fourney Gagan Bansal Forough Poursabzi-Sangdeh Han Liu Saleema Amershi ALM OffRL 46 12 0 29 Oct 2022
Searching for Better Database Queries in the Outputs of Semantic Parsers A. Osokin Irina Saparina R. B. Yarullin 29 0 0 13 Oct 2022
SimSCOOD: Systematic Analysis of Out-of-Distribution Generalization in Fine-tuned Source Code Models Hossein Hajipour Ning Yu Cristian-Alexandru Staicu Mario Fritz OODD 27 4 0 10 Oct 2022
Exploring Code Style Transfer with Neural Networks Karl Munson Anish Savla Chih-Kai Ting Serenity Wade Kiran Kate Kavitha Srinivas CLIP 16 0 0 13 Sep 2022
FOLIO: Natural Language Reasoning with First-Order Logic Simeng Han Hailey Schoelkopf Yilun Zhao Zhenting Qi Martin Riddell ... Yingbo Zhou Caiming Xiong Rex Ying Arman Cohan Dragomir R. Radev ReLM LRM 39 93 0 02 Sep 2022
MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generation Federico Cassano John Gouwar Daniel Nguyen S. Nguyen Luna Phipps-Costin ... Carolyn Jane Anderson Molly Q. Feldman Arjun Guha Michael Greenberg Abhinav Jangda ELM 24 81 0 17 Aug 2022
Out of the BLEU: how should we assess quality of the Code Generation models? Mikhail Evtikhiev Egor Bogomolov Yaroslav Sokolov T. Bryksin ALM 34 88 0 05 Aug 2022
No More Fine-Tuning? An Experimental Evaluation of Prompt Tuning in Code Intelligence Chaozheng Wang Yuanhang Yang Cuiyun Gao Yun Peng Hongyu Zhang Michael R. Lyu AAML 67 134 0 24 Jul 2022
PanGu-Coder: Program Synthesis with Function-Level Language Modeling Fenia Christopoulou Gerasimos Lampouras Milan Gritta Guchun Zhang Yinpeng Guo ... Guangtai Liang Jia Wei Xin Jiang Qianxiang Wang Qun Liu ELM SyDa ALM 45 74 0 22 Jul 2022
CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning Hung Le Yue Wang Akhilesh Deepak Gotmare Silvio Savarese Guosheng Lin SyDa ALM 132 240 0 05 Jul 2022
GitHub Copilot AI pair programmer: Asset or Liability? Arghavan Moradi Dakhel Vahid Majdinasab Amin Nikanjam Foutse Khomh Michel C. Desmarais Zhen Ming Z. Jiang 31 332 0 30 Jun 2022
DeepPERF: A Deep Learning-Based Approach For Improving Software Performance Spandan Garg Roshanak Zilouchian Moghaddam Colin B. Clement Neel Sundaresan Chen Henry Wu 40 7 0 27 Jun 2022
XLCoST: A Benchmark Dataset for Cross-lingual Code Intelligence Ming Zhu Aneesh Jain Karthik Suresh Roshan Ravindran Sindhu Tipirneni Chandan K. Reddy 27 69 0 16 Jun 2022
FixEval: Execution-based Evaluation of Program Fixes for Programming Problems Md. Mahim Anjum Haque W. Ahmad Ismini Lourentzou Chris Brown 26 16 0 15 Jun 2022
NatGen: Generative pre-training by "Naturalizing" source code Saikat Chakraborty Toufique Ahmed Yangruibo Ding Prem Devanbu Baishakhi Ray AI4CE 57 116 0 15 Jun 2022
StructCoder: Structure-Aware Transformer for Code Generation Sindhu Tipirneni Ming Zhu Chandan K. Reddy 30 55 0 10 Jun 2022
CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Models Akshita Jha Chandan K. Reddy SILM ELM AAML 27 59 0 31 May 2022
Deep Learning Meets Software Engineering: A Survey on Pre-Trained Models of Source Code Changan Niu Chuanyi Li Bin Luo Vincent Ng SyDa VLM 55 48 0 24 May 2022
Summarize and Generate to Back-translate: Unsupervised Translation of Programming Languages Wasi Uddin Ahmad Saikat Chakraborty Baishakhi Ray Kai-Wei Chang 47 27 0 23 May 2022
A Survey of Deep Learning Models for Structural Code Understanding Ruoting Wu Yuxin Zhang Qibiao Peng Liang Chen Zibin Zheng 24 6 0 03 May 2022
GPT-based Open-Ended Knowledge Tracing Naiming Liu Zichao Wang Richard G. Baraniuk Andrew S. Lan AI4Ed 32 3 0 21 Feb 2022
A Survey on Artificial Intelligence for Source Code: A Dialogue Systems Perspective Erfan Al-Hossami Samira Shaikh 32 6 0 10 Feb 2022
Competition-Level Code Generation with AlphaCode Yujia Li David Choi Junyoung Chung Nate Kushman Julian Schrittwieser ... Esme Sutherland Robson Pushmeet Kohli Nando de Koray Kavukcuoglu Oriol Vinyals 26 1,299 0 08 Feb 2022
Enabling Automatic Repair of Source Code Vulnerabilities Using Data-Driven Methods Anastasiia Grishina 47 7 0 07 Feb 2022
Training and Evaluating a Jupyter Notebook Data Science Assistant Shubham Chandel Colin B. Clement Guillermo Serrato Neel Sundaresan 40 43 0 30 Jan 2022
GAP-Gen: Guided Automatic Python Code Generation Junchen Zhao Yurun Song Junlin Wang Ian G. Harris 16 6 0 19 Jan 2022
Many Heads but One Brain: Fusion Brain -- a Competition and a Single Multimodal Multitask Architecture Daria Bakshandaeva Denis Dimitrov V.Ya. Arkhipkin Alex Shonenkov M. Potanin ... Mikhail Martynov Anton Voronov Vera Davydova E. Tutubalina Aleksandr Petiushko 33 0 0 22 Nov 2021
Using Document Similarity Methods to create Parallel Datasets for Code Translation Mayank Agarwal Kartik Talamadupula Fernando Martinez Stephanie Houde Michael J. Muller John T. Richards Steven I. Ross Justin D. Weisz SyDa 34 8 0 11 Oct 2021
Jointly Learning to Repair Code and Generate Commit Message Jiaqi Bai Long Zhou Ambrosio Blanco Shujie Liu Furu Wei Ming Zhou Zhoujun Li 19 4 0 25 Sep 2021
Towards A Measure Of General Machine Intelligence Gautham Venkatasubramanian Sibesh Kar Abhimanyu Singh Shubham Mishra Dushyant Yadav Shreyansh Chandak ALM ELM 13 1 0 24 Sep 2021
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation Yue Wang Weishi Wang Chenyu You Guosheng Lin 243 1,492 0 02 Sep 2021
Retrieval Augmented Code Generation and Summarization Md. Rizwan Parvez W. Ahmad Saikat Chakraborty Baishakhi Ray Kai-Wei Chang 26 183 0 26 Aug 2021
AVATAR: A Parallel Corpus for Java-Python Program Translation W. Ahmad Md Golam Rahman Tushar Saikat Chakraborty Kai-Wei Chang 38 79 0 26 Aug 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 78 5,082 0 07 Jul 2021
Programming Puzzles Tal Schuster Ashwin Kalyan Oleksandr Polozov Adam Tauman Kalai ELM 17 32 0 10 Jun 2021
Measuring Coding Challenge Competence With APPS Dan Hendrycks Steven Basart Saurav Kadavath Mantas Mazeika Akul Arora ... Collin Burns Samir Puranik Horace He D. Song Jacob Steinhardt ELM AIMat ALM 208 627 0 20 May 2021
CoTexT: Multi-task Learning with Code-Text Transformer Long Phan H. Tran Daniel Le Hieu Duy Nguyen J. Anibal Alec Peltekian Yanfang Ye 19 135 0 18 May 2021
Unified Pre-training for Program Understanding and Generation Wasi Uddin Ahmad Saikat Chakraborty Baishakhi Ray Kai-Wei Chang 23 748 0 10 Mar 2021