Is Next Token Prediction Sufficient for GPT? Exploration on Code Logic
Comprehension

Is Next Token Prediction Sufficient for GPT? Exploration on Code Logic Comprehension

13 April 2024

Neel Sundaresan

ArXiv (abs)PDF HTML

Papers citing "Is Next Token Prediction Sufficient for GPT? Exploration on Code Logic Comprehension"

10 / 10 papers shown

Title
WizardCoder: Empowering Code Large Language Models with Evol-Instruct Ziyang Luo Can Xu Pu Zhao Qingfeng Sun Xiubo Geng Wenxiang Hu Chongyang Tao Jing Ma Qingwei Lin Daxin Jiang ELM SyDa ALM 95 679 0 14 Jun 2023
Efficient Training of Language Models to Fill in the Middle Mohammad Bavarian Heewoo Jun Nikolas Tezak John Schulman C. McLeavey Jerry Tworek Mark Chen 73 195 0 28 Jul 2022
GPT-NeoX-20B: An Open-Source Autoregressive Language Model Sid Black Stella Biderman Eric Hallahan Quentin G. Anthony Leo Gao ... Shivanshu Purohit Laria Reynolds J. Tow Benqi Wang Samuel Weinbach 176 835 0 14 Apr 2022
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation Yue Wang Weishi Wang Shafiq Joty Guosheng Lin 289 1,560 0 02 Sep 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 233 5,635 0 07 Jul 2021
CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks Ruchi Puri David S. Kung G. Janssen Wei Zhang Giacomo Domeniconi ... Saurabh Pujar Shyam Ramji Ulrich Finkler Susan Malaika Frederick Reiss 79 239 0 25 May 2021
GLM: General Language Model Pretraining with Autoregressive Blank Infilling Zhengxiao Du Yujie Qian Xiao Liu Ming Ding J. Qiu Zhilin Yang Jie Tang BDL AI4CE 137 1,550 0 18 Mar 2021
Unified Pre-training for Program Understanding and Generation Wasi Uddin Ahmad Saikat Chakraborty Baishakhi Ray Kai-Wei Chang 135 769 0 10 Mar 2021
CodeBERT: A Pre-Trained Model for Programming and Natural Languages Zhangyin Feng Daya Guo Duyu Tang Nan Duan Xiaocheng Feng ... Linjun Shou Bing Qin Ting Liu Daxin Jiang Ming Zhou 165 2,637 0 19 Feb 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 445 20,298 0 23 Oct 2019