CodeBLEU: a Method for Automatic Evaluation of Code Synthesis

22 September 2020

Papers citing "CodeBLEU: a Method for Automatic Evaluation of Code Synthesis"

50 / 245 papers shown

Title
A Preliminary Study of Multilingual Code Language Models for Code Generation Task Using Translated Benchmarks Rohit Dandamudi Gema Rodríguez-Pérez ELM 79 0 0 23 Nov 2024
StackEval: Benchmarking LLMs in Coding Assistance Nidhish Shah Zulkuf Genc Dogu Araci ELM 66 0 0 21 Nov 2024
Schemato -- An LLM for Netlist-to-Schematic Conversion Ryoga Matsuo Stefan Uhlich Arun Venkitaraman Andrea Bonetti Chia-Yu Hsieh Ali Momeni Lukas Mauch Augusto Capone Eisaku Ohbuchi Lorenzo Servadei 74 0 0 21 Nov 2024
Parameter-Efficient Fine-Tuning of Large Language Models for Unit Test Generation: An Empirical Study André Storhaug Jingyue Li ALM 53 1 0 04 Nov 2024
Fixing Security Vulnerabilities with AI in OSS-Fuzz Yuntong Zhang Jiawei Wang Dominic Berzin Martin Mirchev Dongge Liu Abhishek Arya Oliver Chang Abhik Roychoudhury 37 1 0 03 Nov 2024
Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvement Yingwei Ma Rongyu Cao Yongchang Cao Wenjie Qu J. Chen Yibo Liu Yuchen Liu Binhua Li Fei Huang Yongbin Li 63 5 0 01 Nov 2024
Metamorphic Malware Evolution: The Potential and Peril of Large Language Models Pooria Madani 42 5 0 31 Oct 2024
Can Language Models Replace Programmers? REPOCOD Says Ñot Yet' Shanchao Liang Yiran Hu Nan Jiang Lin Tan ALM ELM 29 2 0 29 Oct 2024
M2rc-Eval: Massively Multilingual Repository-level Code Completion Evaluation Jiaheng Liu Ken Deng Congnan Liu Jian Yang Shukai Liu ... Zekun Wang Guoan Zhang Bangyu Xiang Wenbo Su Jian Xu 75 4 0 28 Oct 2024
CodeRosetta: Pushing the Boundaries of Unsupervised Code Translation for Parallel Programming Ali TehraniJamsaz Arijit Bhattacharjee Le Chen Nesreen Ahmed Amir Yazdanbakhsh Ali Jannesari 34 5 0 27 Oct 2024
Building A Coding Assistant via the Retrieval-Augmented Language Model Xinze Li Hanbin Wang Zhenghao Liu S. Yu Shuo Wang Yukun Yan Yukai Fu Yu Gu Ge Yu 3DV RALM 23 2 0 21 Oct 2024
SecCodePLT: A Unified Platform for Evaluating the Security of Code GenAI Yu Yang Yuzhou Nie Zhun Wang Yuheng Tang Wenbo Guo Bo Li D. Song ELM 38 6 0 14 Oct 2024
Generating Driving Simulations via Conversation Rimvydas Rubavicius Antonio Valerio Miceli Barone A. Lascarides S. Ramamoorthy 23 0 0 13 Oct 2024
Autonomous Evaluation of LLMs for Truth Maintenance and Reasoning Tasks Rushang Karia Daniel Bramblett D. Dobhal Siddharth Srivastava ELM LRM 35 0 0 11 Oct 2024
An evaluation of LLM code generation capabilities through graded exercises Álvaro Barbero Jiménez ELM 36 1 0 06 Oct 2024
Text2Chart31: Instruction Tuning for Chart Generation with Automatic Feedback Fatemeh Pesaran Zadeh Juyeon Kim Jin-Hwa Kim Gunhee Kim ALM 51 2 0 05 Oct 2024
Generating Equivalent Representations of Code By A Self-Reflection Approach Jia Li Ge Li Lecheng Wang Hao Zhu Zhi Jin 34 1 0 04 Oct 2024
Showing LLM-Generated Code Selectively Based on Confidence of LLMs Jia Li Yuqi Zhu Yongmin Li Ge Li Zhi Jin 33 0 0 04 Oct 2024
CodeJudge: Evaluating Code Generation with Large Language Models Weixi Tong Tianyi Zhang ELM ALM 39 8 0 03 Oct 2024
Codev-Bench: How Do LLMs Understand Developer-Centric Code Completion? Zhenyu Pan Rongyu Cao Yongchang Cao Yingwei Ma Binhua Li Fei Huang Han Liu Yongbin Li 50 4 0 02 Oct 2024
TRANSAGENT: An LLM-Based Multi-Agent System for Code Translation Zhiqiang Yuan Weitong Chen Hanlin Wang Kai Yu Xin Peng Yiling Lou LLMAG 28 10 0 30 Sep 2024
CRScore: Grounding Automated Evaluation of Code Review Comments in Code Claims and Smells Atharva Naik Marcus Alenius Daniel Fried Carolyn Rose 36 0 0 29 Sep 2024
Test Case-Informed Knowledge Tracing for Open-ended Coding Tasks Zhangqi Duan Nigel Fernandez Alexander Hicks Andrew S. Lan AI4Ed 38 2 0 28 Sep 2024
CodeInsight: A Curated Dataset of Practical Coding Solutions from Stack Overflow Nathanael Beau Benoît Crabbé 42 1 0 25 Sep 2024
Multi-objective Evolution of Heuristic Using Large Language Model Shunyu Yao Fei Liu Xi Lin Zhichao Lu Zhenkun Wang Qingfu Zhang 26 6 0 25 Sep 2024
RAMBO: Enhancing RAG-based Repository-Level Method Body Completion Tuan-Dung Bui Duc-Thieu Luu-Van Thanh-Phat Nguyen Thu-Trang Nguyen Son Nguyen H. Vo 37 4 0 23 Sep 2024
VulnLLMEval: A Framework for Evaluating Large Language Models in Software Vulnerability Detection and Patching Arastoo Zibaeirad Marco Vieira 29 7 0 16 Sep 2024
Enhancing Source Code Security with LLMs: Demystifying The Challenges and Generating Reliable Repairs Nafis Tanveer Islam Joseph Khoury Andrew Seong E. Bou-Harb Peyman Najafirad AAML 38 3 0 01 Sep 2024
CSAD: Unsupervised Component Segmentation for Logical Anomaly Detection Yu-Hsuan Hsieh Shang-Hong Lai 37 3 0 28 Aug 2024
Understanding Defects in Generated Codes by Language Models Ali Mohammadi Esfahani N. Kahani S. Ajila 25 1 0 23 Aug 2024
Enhancing Automated Program Repair with Solution Design Jiuang Zhao Donghao Yang Li Zhang Xiaoli Lian Zitian Yang Fang Liu 29 4 0 22 Aug 2024
SimBench: A Rule-Based Multi-Turn Interaction Benchmark for Evaluating an LLM's Ability to Generate Digital Twins Jingquan Wang Harry Zhang H. Unjhawala Peter Negrut Shu Wang Khailanii Slaton R. Serban Jin-Long Wu Dan Negrut 62 0 0 21 Aug 2024
What can Large Language Models Capture about Code Functional Equivalence? Nickil Maveli Antonio Vergari Shay B. Cohen 44 2 0 20 Aug 2024
A Disguised Wolf Is More Harmful Than a Toothless Tiger: Adaptive Malicious Code Injection Backdoor Attack Leveraging User Behavior as Triggers Shangxi Wu Jitao Sang SILM AAML 37 1 0 19 Aug 2024
Generating Unseen Code Tests In Infinitum Marcel Zalmanovici Orna Raz E. Farchi Iftach Freund 41 0 0 29 Jul 2024
Enhancing Code Translation in Language Models with Few-Shot Learning via Retrieval-Augmented Generation Manish Bhattarai Javier E. Santos Shawn Jones Ayan Biswas Boian Alexandrov Dan O’Malley 42 9 0 29 Jul 2024
Empowering Agile-Based Generative Software Development through Human-AI Teamwork Sai Zhang Zhenchang Xing Ronghui Guo Fangzhou Xu Lei Chen Zhaoyuan Zhang Xiaowang Zhang Zhiyong Feng Zhiqiang Zhuang 52 2 0 22 Jul 2024
Towards More Trustworthy and Interpretable LLMs for Code through Syntax-Grounded Explanations David Nader-Palacio Daniel Rodríguez-Cárdenas Alejandro Velasco Dipin Khati Kevin Moran Denys Poshyvanyk 58 6 0 12 Jul 2024
Defending Code Language Models against Backdoor Attacks with Deceptive Cross-Entropy Loss Guang Yang Yu Zhou Xiang Chen Xiangyu Zhang Terry Yue Zhuo David Lo Taolue Chen AAML 57 4 0 12 Jul 2024
FuncEvalGMN: Evaluating Functional Correctness of SQL via Graph Matching Network Yi Zhan Yang Sun Han Weng Longjie Cui Guifeng Wang Jiajun Xie Yu Tian Xiaoming Yin Boyi Liu Dongchi Huang 34 0 0 09 Jul 2024
Code Hallucination Mirza Masfiqur Rahman Ashish Kundu LRM HILM 36 2 0 05 Jul 2024
Applying RLAIF for Code Generation with API-usage in Lightweight LLMs Sujan Dutta Sayantan Mahinder R. Anantha Bortik Bandyopadhyay ALM 39 4 0 28 Jun 2024
Automatically Generating UI Code from Screenshot: A Divide-and-Conquer-Based Approach Yuxuan Wan Chaozheng Wang Yi Dong Wenxuan Wang Shuqing Li Yintong Huo M. Lyu 3DV 76 10 0 24 Jun 2024
FVEL: Interactive Formal Verification Environment with Large Language Models via Theorem Proving Xiaohan Lin Qingxing Cao Yinya Huang Haiming Wang Jianqiao Lu Zhengying Liu Linqi Song Xiaodan Liang LRM 44 4 0 20 Jun 2024
Can LLMs Reason in the Wild with Programs? Yuan Yang Siheng Xiong Ali Payani Ehsan Shareghi Faramarz Fekri LRM 40 13 0 19 Jun 2024
Benchmarks and Metrics for Evaluations of Code Generation: A Critical Review Debalina Ghosh Paul Hong Zhu Ian Bayley ALM ELM 39 9 0 18 Jun 2024
ScenEval: A Benchmark for Scenario-Based Evaluation of Code Generation Debalina Ghosh Paul Hong Zhu Ian Bayley 35 2 0 18 Jun 2024
A Survey on Large Language Models for Code Generation Juyong Jiang Fan Wang Jiasi Shen Sungju Kim Sunghun Kim 53 166 0 01 Jun 2024
Interpreting Latent Student Knowledge Representations in Programming Assignments Nigel Fernandez Andrew S. Lan 40 2 0 13 May 2024
On the Limitations of Embedding Based Methods for Measuring Functional Correctness for Code Generation Atharva Naik 46 2 0 26 Apr 2024