Exploring the Robustness of Large Language Models for Solving Programming Problems

26 June 2023

Papers citing "Exploring the Robustness of Large Language Models for Solving Programming Problems"

37 / 37 papers shown

Title
RobuNFR: Evaluating the Robustness of Large Language Models on Non-Functional Requirements Aware Code Generation Feng Lin Dong Jae Kim Zhiyu Li Jinqiu Yang Tse-Husn Chen AAML 101 1 0 28 Mar 2025
Fully Autonomous Programming using Iterative Multi-Agent Debugging with Large Language Models Anastasiia Grishina Vadim Liventsev Aki Härmä Leon Moonen ELM 153 0 0 10 Mar 2025
Enhancing LLM Character-Level Manipulation via Divide and Conquer Zhen Xiong Yujun Cai Bryan Hooi Nanyun Peng Kai-Wei Chang Zhecheng Li 142 0 0 12 Feb 2025
Turbulence: Systematically and Automatically Testing Instruction-Tuned Large Language Models for Code Shahin Honarvar Mark van der Wilk Alastair Donaldson 200 8 0 28 Jan 2025
Codex Hacks HackerRank: Memorization Issues and a Framework for Code Synthesis Evaluation Anjan Karmakar Julian Aron Prenner Marco DÁmbros Romain Robbes ELM 67 17 0 06 Dec 2022
Repair Is Nearly Generation: Multilingual Program Repair with LLMs Harshit Joshi J. Cambronero Sumit Gulwani Vu Le Ivan Radicek Gust Verbruggen LRM 68 135 0 24 Aug 2022
CodeT: Code Generation with Generated Tests Bei Chen Fengji Zhang A. Nguyen Daoguang Zan Zeqi Lin Jian-Guang Lou Weizhu Chen 103 344 0 21 Jul 2022
DocPrompting: Generating Code by Retrieving the Docs Shuyan Zhou Uri Alon Frank F. Xu Zhiruo Wang Zhengbao Jiang Graham Neubig LLMAG 92 140 0 13 Jul 2022
Automatic Generation of Programming Exercises and Code Explanations using Large Language Models Sami Sarsa Paul Denny Arto Hellas Juho Leinonen ELM 170 360 0 03 Jun 2022
Are Large Pre-Trained Language Models Leaking Your Personal Information? Jie Huang Hanyin Shao Kevin Chen-Chuan Chang PILM 100 201 0 25 May 2022
InCoder: A Generative Model for Code Infilling and Synthesis Daniel Fried Armen Aghajanyan Jessy Lin Sida I. Wang Eric Wallace Freda Shi Ruiqi Zhong Wen-tau Yih Luke Zettlemoyer M. Lewis SyDa 89 652 0 12 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 537 6,301 0 05 Apr 2022
Less is More: Summary of Long Instructions is Better for Program Synthesis Kirby Kuznia Swaroop Mishra Mihir Parmar Chitta Baral AIMat 81 24 0 16 Mar 2022
Probing Pretrained Models of Source Code Sergey Troshin Nadezhda Chirkova ELM 98 38 0 16 Feb 2022
Code Generation for Unknown Libraries via Reading API Documentations Koki Washio Yusuke Miyao 46 3 0 16 Feb 2022
Competition-Level Code Generation with AlphaCode Yujia Li David Choi Junyoung Chung Nate Kushman Julian Schrittwieser ... Esme Sutherland Robson Pushmeet Kohli Nando de Koray Kavukcuoglu Oriol Vinyals 148 1,425 0 08 Feb 2022
A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Level Iddo Drori Sarah J. Zhang Reece Shuttleworth Leonard Tang Albert Lu ... J. Lynch A. Shporer Nakul Verma Eugene Wu G. Strang AIMat ReLM 126 152 0 31 Dec 2021
Leveraging Automated Unit Tests for Unsupervised Code Translation Baptiste Roziere Jie M. Zhang François Charton Mark Harman Gabriel Synnaeve Guillaume Lample 94 115 0 13 Oct 2021
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation Yue Wang Weishi Wang Shafiq Joty Guosheng Lin 299 1,592 0 02 Sep 2021
What do pre-trained code models know about code? Anjan Karmakar Romain Robbes ELM 73 91 0 25 Aug 2021
Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions Hammond Pearce Baleegh Ahmad Benjamin Tan Brendan Dolan-Gavitt Ramesh Karri SILM 100 424 0 20 Aug 2021
SynCoBERT: Syntax-Guided Multi-Modal Contrastive Pre-Training for Code Representation Xin Wang Yasheng Wang Fei Mi Pingyi Zhou Yao Wan Xiao Liu Li Li Hao Wu Jin Liu Xin Jiang 122 115 0 10 Aug 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 238 5,675 0 07 Jul 2021
CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks Ruchi Puri David S. Kung G. Janssen Wei Zhang Giacomo Domeniconi ... Saurabh Pujar Shyam Ramji Ulrich Finkler Susan Malaika Frederick Reiss 91 244 0 25 May 2021
Measuring Coding Challenge Competence With APPS Dan Hendrycks Steven Basart Saurav Kadavath Mantas Mazeika Akul Arora ... Collin Burns Samir Puranik Horace He Basel Alomair Jacob Steinhardt ELM AIMat ALM 274 707 0 20 May 2021
CoTexT: Multi-task Learning with Code-Text Transformer Long Phan H. Tran Daniel Le Hieu Duy Nguyen J. Anibal Alec Peltekian Yanfang Ye 70 136 0 18 May 2021
Unified Pre-training for Program Understanding and Generation Wasi Uddin Ahmad Saikat Chakraborty Baishakhi Ray Kai-Wei Chang 140 771 0 10 Mar 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 476 2,123 0 31 Dec 2020
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown Basel Alomair Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 517 1,956 0 14 Dec 2020
CodeBLEU: a Method for Automatic Evaluation of Code Synthesis Shuo Ren Daya Guo Shuai Lu Long Zhou Shujie Liu Duyu Tang Neel Sundaresan M. Zhou Ambrosio Blanco Shuai Ma ELM 126 543 0 22 Sep 2020
GraphCodeBERT: Pre-training Code Representations with Data Flow Daya Guo Shuo Ren Shuai Lu Zhangyin Feng Duyu Tang ... Dawn Drain Neel Sundaresan Jian Yin Daxin Jiang M. Zhou 167 1,152 0 17 Sep 2020
CodeBERT: A Pre-Trained Model for Programming and Natural Languages Zhangyin Feng Daya Guo Duyu Tang Nan Duan Xiaocheng Feng ... Linjun Shou Bing Qin Ting Liu Daxin Jiang Ming Zhou 178 2,672 0 19 Feb 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 506 20,376 0 23 Oct 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 700 24,572 0 26 Jul 2019
SPoC: Search-based Pseudocode to Code Sumith Kulal Panupong Pasupat Kartik Chandra Mina Lee Oded Padon A. Aiken Percy Liang 74 228 0 12 Jun 2019
DeepCoder: Learning to Write Programs Matej Balog Alexander L. Gaunt Marc Brockschmidt Sebastian Nowozin Daniel Tarlow AIMat NAI 103 575 0 07 Nov 2016
Improving Neural Machine Translation Models with Monolingual Data Rico Sennrich Barry Haddow Alexandra Birch 261 2,724 0 20 Nov 2015