CodeBLEU: a Method for Automatic Evaluation of Code Synthesis

22 September 2020

Papers citing "CodeBLEU: a Method for Automatic Evaluation of Code Synthesis"

50 / 245 papers shown

Title
Evolutionary Large Language Models for Hardware Security: A Comparative Survey Mohammad Akyash Hadi M Kamali 36 11 0 25 Apr 2024
Peer-aided Repairer: Empowering Large Language Models to Repair Advanced Student Assignments Qianhui Zhao Fang Liu Li Zhang Yang Liu Zhen Yan Zhenghao Chen Yufei Zhou Jing Jiang Ge Li 38 3 0 02 Apr 2024
Exploring the Impact of the Output Format on the Evaluation of Large Language Models for Code Translation Marcos Macedo Yuan Tian F. Côgo Bram Adams 46 12 0 25 Mar 2024
Concerned with Data Contamination? Assessing Countermeasures in Code Language Model Jialun Cao Wuqi Zhang Shing-Chi Cheung 19 16 0 25 Mar 2024
CodeS: Natural Language to Code Repository via Multi-Layer Sketch Daoguang Zan Ailun Yu Wei Liu Dong Chen Bo Shen ... Bei Guan Zhiguang Yang Yongji Wang Qianxiang Wang Li-zhen Cui 33 14 0 25 Mar 2024
Genetic Auto-prompt Learning for Pre-trained Code Intelligence Language Models Chengzhe Feng Yanan Sun Ke Li Pan Zhou Jiancheng Lv Aojun Lu 51 1 0 20 Mar 2024
Repoformer: Selective Retrieval for Repository-Level Code Completion Di Wu W. Ahmad Dejiao Zhang M. K. Ramanathan Xiaofei Ma 25 26 0 15 Mar 2024
RepoHyper: Better Context Retrieval Is All You Need for Repository-Level Code Completion Huy N. Phan Hoang N. Phan Tien N. Nguyen Nghi D. Q. Bui 43 12 0 10 Mar 2024
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks Linyuan Gong Sida Wang Mostafa Elhoushi Alvin Cheung 32 15 0 07 Mar 2024
CatCode: A Comprehensive Evaluation Framework for LLMs On the Mixture of Code and Text Zhenru Lin Yiqun Yao Yang Yuan ELM 25 0 0 04 Mar 2024
Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented Generation Shicheng Xu Liang Pang Mo Yu Fandong Meng Huawei Shen Xueqi Cheng Jie Zhou RALM 38 10 0 28 Feb 2024
Exploring Advanced Methodologies in Security Evaluation for LLMs Junming Huang Jiawei Zhang Qi Wang Weihong Han Yanchun Zhang 45 0 0 28 Feb 2024
OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Web Raghav Kapoor Y. Butala M. Russak Jing Yu Koh Kiran Kamble Waseem Alshikh Ruslan Salakhutdinov LLMAG 51 44 0 27 Feb 2024
Studying LLM Performance on Closed- and Open-source Data Toufique Ahmed Christian Bird Prem Devanbu Saikat Chakraborty 72 9 0 23 Feb 2024
Copilot Evaluation Harness: Evaluating LLM-Guided Software Programming Anisha Agarwal Aaron Chan Shubham Chandel Jinu Jang Shaun Miller Roshanak Zilouchian Moghaddam Yevhen Mohylevskyy Neel Sundaresan Michele Tufano ELM 26 13 0 22 Feb 2024
From Text to CQL: Bridging Natural Language and Corpus Search Engine Luming Lu Jiyuan An Yujie Wang Liner Yang Cunliang Kong ... Shuo Wang Haozhe Lin Mingwei Fang Yaping Huang Erhong Yang 45 1 0 21 Feb 2024
Analyzing Prompt Influence on Automated Method Generation: An Empirical Study with Copilot Ionut Daniel Fagadau Leonardo Mariani D. Micucci O. Riganelli 29 7 0 13 Feb 2024
Unsupervised Evaluation of Code LLMs with Round-Trip Correctness Miltiadis Allamanis Sheena Panthaplackel Pengcheng Yin ALM OffRL LRM 43 9 0 13 Feb 2024
Mercury: A Code Efficiency Benchmark for Code Large Language Models Mingzhe Du A. Luu Bin Ji Qian Liu See-Kiong Ng ALM ELM OffRL 24 6 0 12 Feb 2024
Text-to-Code Generation with Modality-relative Pre-training Fenia Christopoulou Guchun Zhang Gerasimos Lampouras AI4TS 23 1 0 08 Feb 2024
Do Large Code Models Understand Programming Concepts? Counterfactual Analysis for Code Predicates Ashish Hooda Mihai Christodorescu Miltos Allamanis Aaron Wilson Kassem Fawaz Somesh Jha ELM 38 7 0 08 Feb 2024
Enhancing LLM-Based Coding Tools through Native Integration of IDE-Derived Static Context Yichen Li Yun Peng Yintong Huo Michael R. Lyu 11 11 0 06 Feb 2024
User Centric Evaluation of Code Generation Tools Tanha Miah Hong Zhu ELM 25 3 0 05 Feb 2024
NoFunEval: Funny How Code LMs Falter on Requirements Beyond Functional Correctness Manav Singhal Tushar Aggarwal Abhijeet Awasthi Nagarajan Natarajan Aditya Kanade 32 12 0 29 Jan 2024
Structured Code Representations Enable Data-Efficient Adaptation of Code Language Models Mayank Agarwal Songlin Yang Bailin Wang Yoon Kim Jie Chen 47 5 0 19 Jan 2024
A Novel Approach for Automatic Program Repair using Round-Trip Translation with Large Language Models Fernando Vallecillos Ruiz Anastasiia Grishina Max Hort Leon Moonen LRM 47 4 0 15 Jan 2024
Code Security Vulnerability Repair Using Reinforcement Learning with Large Language Models Nafis Tanveer Islam Mohammad Bahrami Karkevandi Peyman Najafirad 30 6 0 13 Jan 2024
OOP: Object-Oriented Programming Evaluation Benchmark for Large Language Models Shuai Wang Liang Ding Li Shen Yong Luo Bo Du Dacheng Tao ELM ALM 45 2 0 12 Jan 2024
Rewriting the Code: A Simple Method for Large Language Model Augmented Code Search Haochen Li Xin Zhou Zhiqi Shen 34 10 0 09 Jan 2024
AST-T5: Structure-Aware Pretraining for Code Generation and Understanding Linyuan Gong Mostafa Elhoushi Alvin Cheung 29 11 0 05 Jan 2024
If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents Ke Yang Jiateng Liu John Wu Chaoqi Yang Yi R. Fung ... Xu Cao Xingyao Wang Yiquan Wang Chenhui Xu Chengxiang Zhai LLMAG ELM 26 75 0 01 Jan 2024
A Comprehensive Evaluation of Parameter-Efficient Fine-Tuning on Software Engineering Tasks Wentao Zou Qi Li Jidong Ge Chuanyi Li Xiaoyu Shen LiGuo Huang Bin Luo 32 5 0 25 Dec 2023
Xpert: Empowering Incident Management with Query Recommendations via Large Language Models Yuxuan Jiang Chaoyun Zhang Shilin He Zhihao Yang Ming-Jie Ma ... Yu Kang Yingnong Dang Saravan Rajmohan Qingwei Lin Dongmei Zhang 45 17 0 19 Dec 2023
Towards Trustworthy AI Software Development Assistance Daniel Maninger Krishna Narasimhan Mira Mezini 35 3 0 14 Dec 2023
STraceBERT: Source Code Retrieval using Semantic Application Traces Claudio Spiess 29 0 0 07 Dec 2023
A Machine Learning Approach Towards SKILL Code Autocompletion Enrique Dehaerne Bappaditya Dey Wannes Meert 26 0 0 04 Dec 2023
CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation Weixiang Yan Haitian Liu Yunkun Wang Yunzhe Li Qian Chen ... Tingyu Lin Weishan Zhao Li Zhu Hari Sundaram Shuiguang Deng ELM LRM 34 35 0 14 Nov 2023
Data Augmentation for Code Translation with Comparable Corpora and Multiple References Yiqing Xie Atharva Naik Daniel Fried Carolyn Rose 54 6 0 01 Nov 2023
Automating the Correctness Assessment of AI-generated Code for Security Contexts Domenico Cotroneo Alessio Foggia Cristina Improta Pietro Liguori R. Natella 26 8 0 28 Oct 2023
Pitfalls in Language Models for Code Intelligence: A Taxonomy and Survey Xinyu She Yue Liu Yanjie Zhao Yiling He Li Li C. Tantithamthavorn Zhan Qin Haoyu Wang ELM 35 13 0 27 Oct 2023
Enhancing Large Language Models for Secure Code Generation: A Dataset-driven Study on Vulnerability Mitigation Jiexin Wang Liuwen Cao Xitong Luo Zhiping Zhou Jiayuan Xie Adam Jatowt Yi Cai 44 10 0 25 Oct 2023
SteloCoder: a Decoder-Only LLM for Multi-Language to Python Code Translation Jialing Pan Adrien Sadé Jin Kim Eric Soriano Guillem Sole Sylvain Flamant SyDa 13 16 0 24 Oct 2023
SUT: Active Defects Probing for Transcompiler Models Mengnan Qi Yufan Huang Maoquan Wang Yongqiang Yao Zihan Liu Bin Gu Colin B. Clement Neel Sundaresan 25 2 0 22 Oct 2023
Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChain Marcus J. Min Yangruibo Ding Luca Buratti Saurabh Pujar Gail E. Kaiser Suman Jana Baishakhi Ray LRM HILM 27 18 0 21 Oct 2023
Program Translation via Code Distillation Yufan Huang Mengnan Qi Yongqiang Yao Maoquan Wang Bin Gu Colin B. Clement Neel Sundaresan 23 5 0 17 Oct 2023
Large Language Model-Aware In-Context Learning for Code Generation Jia Li Ge Li Chongyang Tao Jia Li Huangzhao Zhang Fang Liu Zhi Jin 51 28 0 15 Oct 2023
Assessing Evaluation Metrics for Neural Test Oracle Generation Jiho Shin Hadi Hemmati Moshi Wei Song Wang ELM 26 7 0 11 Oct 2023
Benchmarking and Explaining Large Language Model-based Code Generation: A Causality-Centric Approach Zhenlan Ji Pingchuan Ma Zongjie Li Shuai Wang 29 21 0 10 Oct 2023
Optimizing Large Language Models to Expedite the Development of Smart Contracts Nii Osae Osae Dade Margaret Lartey-Quaye Emmanuel Teye-Kofi Odonkor Paul Ammah 29 4 0 08 Oct 2023
CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation Weixiang Yan Yuchen Tian Yunzhe Li Qian Chen Wen Wang 34 35 0 08 Oct 2023