CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution

23 August 2024

Papers citing "CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution"

30 / 30 papers shown

Title
DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence DeepSeek-AI Qihao Zhu Daya Guo Zhihong Shao Dejian Yang ... Jiashi Li Chenggang Zhao Chong Ruan Fuli Luo Wenfeng Liang MoE LRM ELM VLM 86 197 0 17 Jun 2024
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone Marah Abdin Sam Ade Jacobs A. A. Awan J. Aneja Ahmed Hassan Awadallah ... Li Zhang Yi Zhang Yue Zhang Yunan Zhang Xiren Zhou LRM ALM 123 1,224 0 22 Apr 2024
Concerned with Data Contamination? Assessing Countermeasures in Code Language Model Jialun Cao Wuqi Zhang Shing-Chi Cheung 53 20 0 25 Mar 2024
Reasoning Runtime Behavior of a Program with LLM: How Far Are We? Junkai Chen Zhiyuan Pan Xing Hu Zhenhao Li Ge Li Xin Xia LRM 75 26 0 25 Mar 2024
StarCoder 2 and The Stack v2: The Next Generation Anton Lozhkov Raymond Li Loubna Ben Allal Federico Cassano J. Lamy-Poirier ... Sean M. Hughes Thomas Wolf Arjun Guha Leandro von Werra H. D. Vries OSLM ELM 62 340 0 29 Feb 2024
DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence Daya Guo Qihao Zhu Dejian Yang Zhenda Xie Kai Dong ... Yu-Huan Wu Yiming Li Fuli Luo Yingfei Xiong W. Liang ELM 120 762 0 25 Jan 2024
CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution Alex Gu Baptiste Rozière Hugh Leather Armando Solar-Lezama Gabriel Synnaeve Sida I. Wang ELM ALM LRM 46 110 0 05 Jan 2024
SWE-bench: Can Language Models Resolve Real-World GitHub Issues? Carlos E. Jimenez John Yang Alexander Wettig Shunyu Yao Kexin Pei Ofir Press Karthik Narasimhan ELM 90 591 0 10 Oct 2023
CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation Weixiang Yan Yuchen Tian Yunzhe Li Qian Chen Wen Wang 69 39 0 08 Oct 2023
Large Language Models for Test-Free Fault Localization Aidan Z. H. Yang Ruben Martins Claire Le Goues Vincent J. Hellendoorn LRM 57 96 0 03 Oct 2023
Qwen Technical Report Jinze Bai Shuai Bai Yunfei Chu Zeyu Cui Kai Dang ... Zhenru Zhang Chang Zhou Jingren Zhou Xiaohuan Zhou Tianhang Zhu OSLM 259 1,816 0 28 Sep 2023
Textbooks Are All You Need II: phi-1.5 technical report Yuan-Fang Li Sébastien Bubeck Ronen Eldan Allison Del Giorno Suriya Gunasekar Yin Tat Lee ALM LRM 154 474 0 11 Sep 2023
Code Llama: Open Foundation Models for Code Baptiste Rozière Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat ... Hugo Touvron Louis Martin Nicolas Usunier Thomas Scialom Gabriel Synnaeve ELM ALM 108 2,037 0 24 Aug 2023
WizardCoder: Empowering Code Large Language Models with Evol-Instruct Ziyang Luo Can Xu Pu Zhao Qingfeng Sun Xiubo Geng Wenxiang Hu Chongyang Tao Jing Ma Qingwei Lin Daxin Jiang ELM SyDa ALM 87 678 0 14 Jun 2023
StarCoder: may the source be with you! Raymond Li Loubna Ben Allal Yangtian Zi Niklas Muennighoff Denis Kocetkov ... Sean M. Hughes Thomas Wolf Arjun Guha Leandro von Werra H. D. Vries 104 768 0 09 May 2023
Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation Jiawei Liu Chun Xia Yuyao Wang Lingming Zhang ELM ALM 243 921 0 02 May 2023
Teaching Large Language Models to Self-Debug Xinyun Chen Maxwell Lin Nathanael Scharli Denny Zhou LRM 112 693 0 11 Apr 2023
Keep the Conversation Going: Fixing 162 out of 337 bugs for $0.42 each using ChatGPT$ Chun Xia Lingming Zhang KELM LRM 69 121 0 01 Apr 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.4K 14,359 0 15 Mar 2023
Multi-lingual Evaluation of Code Generation Models Ben Athiwaratkun Sanjay Krishna Gouda Zijian Wang Xiaopeng Li Yuchen Tian ... Baishakhi Ray Parminder Bhatia Sudipta Sengupta Dan Roth Bing Xiang ELM 143 172 0 26 Oct 2022
XLCoST: A Benchmark Dataset for Cross-lingual Code Intelligence Ming Zhu Aneesh Jain Karthik Suresh Roshan Ravindran Sindhu Tipirneni Chandan K. Reddy 75 72 0 16 Jun 2022
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis Erik Nijkamp Bo Pang Hiroaki Hayashi Lifu Tu Haiquan Wang Yingbo Zhou Silvio Savarese Caiming Xiong ELM 150 1,023 0 25 Mar 2022
Leveraging Automated Unit Tests for Unsupervised Code Translation Baptiste Roziere Jie M. Zhang François Charton Mark Harman Gabriel Synnaeve Guillaume Lample 76 115 0 13 Oct 2021
AVATAR: A Parallel Corpus for Java-Python Program Translation W. Ahmad Md Golam Rahman Tushar Saikat Chakraborty Kai-Wei Chang 80 81 0 26 Aug 2021
Program Synthesis with Large Language Models Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski ... Ellen Jiang Carrie J. Cai Michael Terry Quoc V. Le Charles Sutton ELM AIMat ReCod ALM 195 1,954 0 16 Aug 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 229 5,539 0 07 Jul 2021
CodeTrans: Towards Cracking the Language of Silicon's Code Through Self-Supervised Deep Learning and High Performance Computing Ahmed Elnaggar Wei Ding Llion Jones Tom Gibbs Tamas B. Fehér Christoph Angerer Silvia Severini Florian Matthes B. Rost 48 72 0 06 Apr 2021
Unsupervised Translation of Programming Languages Marie-Anne Lachaux Baptiste Roziere L. Chanussot Guillaume Lample 96 418 0 05 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 762 42,055 0 28 May 2020
SPoC: Search-based Pseudocode to Code Sumith Kulal Panupong Pasupat Kartik Chandra Mina Lee Oded Padon A. Aiken Percy Liang 62 225 0 12 Jun 2019