NExT: Teaching Large Language Models to Reason about Code Execution

23 April 2024

Ansong Ni

Miltiadis Allamanis

Arman Cohan

Yinlin Deng

Papers citing "NExT: Teaching Large Language Models to Reason about Code Execution"

30 / 30 papers shown

Title
CodeCrash: Stress Testing LLM Reasoning under Structural and Semantic Perturbations Man Ho Lam Chaozheng Wang Jen-tse Huang Michael R. Lyu LRM 80 1 0 19 Apr 2025
The Hitchhiker's Guide to Program Analysis, Part II: Deep Thoughts by LLMs Haonan Li Hang Zhang Kexin Pei Zhiyun Qian 118 1 0 16 Apr 2025
Why Stop at One Error? Benchmarking LLMs as Data Science Code Debuggers for Multi-Hop and Multi-Bug Errors Zhiyu Yang Shuo Wang Yukun Yan Yang Deng 88 0 0 28 Mar 2025
L0-Reasoning Bench: Evaluating Procedural Correctness in Language Models via Simple Program Execution Simeng Sun Cheng-Ping Hsieh Faisal Ladhak Erik Arakelyan Santiago Akle Serano Boris Ginsburg ReLM ELM LRM 484 0 0 28 Mar 2025
Learning to Generate Unit Tests for Automated Debugging Archiki Prasad Elias Stengel-Eskin Justin Chih-Yao Chen Zaid Khan Joey Tianyi Zhou ELM 158 4 0 03 Feb 2025
Mixtral of Experts Albert Q. Jiang Alexandre Sablayrolles Antoine Roux A. Mensch Blanche Savary ... Théophile Gervet Thibaut Lavril Thomas Wang Timothée Lacroix William El Sayed MoE LLMAG 164 1,123 0 08 Jan 2024
Orca 2: Teaching Small Language Models How to Reason Arindam Mitra Luciano Del Corro Shweti Mahajan Andres Codas Clarisse Simoes ... Hamid Palangi Guoqing Zheng Corby Rosset Hamed Khanpour Ahmed Hassan Awadallah ReLM LRM 89 144 0 18 Nov 2023
OctoPack: Instruction Tuning Code Large Language Models Niklas Muennighoff Qian Liu A. Zebaze Qinkai Zheng Binyuan Hui Terry Yue Zhuo Swayam Singh Xiangru Tang Leandro von Werra Shayne Longpre VLM ALM 127 137 0 14 Aug 2023
LEVER: Learning to Verify Language-to-Code Generation with Execution Ansong Ni Srini Iyer Dragomir R. Radev Ves Stoyanov Wen-tau Yih Sida I. Wang Xi Lin 103 226 0 16 Feb 2023
Natural Language to Code Generation in Interactive Data Science Notebooks Pengcheng Yin Wen-Ding Li Kefan Xiao Abhishek Rao Yeming Wen ... Paige Bailey Michele Catasta Henryk Michalewski Oleksandr Polozov Charles Sutton 61 65 0 19 Dec 2022
Synthesizing Research on Programmers' Mental Models of Programs, Tasks and Concepts -- a Systematic Literature Review Ava Heinonen Bettina Lehtelä Arto Hellas Fabian Fagerholm 41 13 0 15 Dec 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 234 3,158 0 20 Oct 2022
ReAct: Synergizing Reasoning and Acting in Language Models Shunyu Yao Jeffrey Zhao Dian Yu Nan Du Izhak Shafran Karthik Narasimhan Yuan Cao LLMAG ReLM LRM 450 2,982 0 06 Oct 2022
Learning Math Reasoning from Self-Sampled Correct and Partially-Correct Solutions Ansong Ni J. Inala Chenglong Wang Oleksandr Polozov Christopher Meek Dragomir R. Radev Jianfeng Gao ReLM AIMat LRM 89 44 0 28 May 2022
STaR: Bootstrapping Reasoning With Reasoning E. Zelikman Yuhuai Wu Jesse Mu Noah D. Goodman ReLM LRM 144 511 0 28 Mar 2022
Automating Code Review Activities by Large-Scale Pre-training Zhiyu Li Shuai Lu Daya Guo Nan Duan Shailesh Jannu ... Deep Majumder Jared Green Alexey Svyatkovskiy Shengyu Fu Neel Sundaresan VLM 89 152 0 17 Mar 2022
Static Prediction of Runtime Errors by Learning to Execute Programs with External Resource Descriptions David Bieber Rishab Goel Daniel Zheng Hugo Larochelle Daniel Tarlow 53 15 0 07 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 853 9,714 0 28 Jan 2022
Show Your Work: Scratchpads for Intermediate Computation with Language Models Maxwell Nye Anders Andreassen Guy Gur-Ari Henryk Michalewski Jacob Austin ... Aitor Lewkowycz Maarten Bosma D. Luan Charles Sutton Augustus Odena ReLM LRM 183 756 0 30 Nov 2021
Counterfactual Explanations for Models of Code Jürgen Cito Işıl Dillig V. Murali S. Chandra AAML LRM 68 51 0 10 Nov 2021
Program Synthesis with Large Language Models Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski ... Ellen Jiang Carrie J. Cai Michael Terry Quoc V. Le Charles Sutton ELM AIMat ReCod ALM 216 2,009 0 16 Aug 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 238 5,675 0 07 Jul 2021
Synthesize, Execute and Debug: Learning to Repair for Neural Program Synthesis Kavi Gupta Peter Ebert Christensen Xinyun Chen Basel Alomair NAI 70 51 0 16 Jul 2020
Unsupervised Commonsense Question Answering with Self-Talk Vered Shwartz Peter West Ronan Le Bras Chandra Bhagavatula Yejin Choi ReLM SSL AI4MH LRM 65 263 0 11 Apr 2020
SPoC: Search-based Pseudocode to Code Sumith Kulal Panupong Pasupat Kartik Chandra Mina Lee Oded Padon A. Aiken Percy Liang 74 228 0 12 Jun 2019
Explain Yourself! Leveraging Language Models for Commonsense Reasoning Nazneen Rajani Bryan McCann Caiming Xiong R. Socher ReLM LRM 99 566 0 06 Jun 2019
SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair Zimin Chen Steve Kommrusch Michele Tufano L. Pouchet Denys Poshyvanyk Monperrus Martin KELM 77 445 0 24 Dec 2018
Thinking Fast and Slow with Deep Learning and Tree Search Thomas W. Anthony Zheng Tian David Barber 133 402 0 23 May 2017
Neural Turing Machines Alex Graves Greg Wayne Ivo Danihelka 111 2,333 0 20 Oct 2014
Learning to Execute Wojciech Zaremba Ilya Sutskever ODL 97 560 0 17 Oct 2014