Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale

17 February 2025

Papers citing "Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale"

25 / 75 papers shown

Title
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling Stella Biderman Hailey Schoelkopf Quentin G. Anthony Herbie Bradley Kyle O'Brien ... USVSN Sai Prashanth Edward Raff Aviya Skowron Lintang Sutawika Oskar van der Wal 107 1,307 0 03 Apr 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.5K 14,748 0 15 Mar 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.5K 13,472 0 27 Feb 2023
Data Selection for Language Models via Importance Resampling Sang Michael Xie Shibani Santurkar Tengyu Ma Percy Liang 118 195 0 06 Feb 2023
Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning Pan Lu Liang Qiu Kai-Wei Chang Ying Nian Wu Song-Chun Zhu Tanmay Rajpurohit Peter Clark Ashwin Kalyan ReLM LRM 171 296 0 29 Sep 2022
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge Linxi Fan Guanzhi Wang Yunfan Jiang Ajay Mandlekar Yuncong Yang Haoyi Zhu Andrew Tang De-An Huang Yuke Zhu Anima Anandkumar LM&Ro 131 386 0 17 Jun 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 532 4,077 0 24 May 2022
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model Shaden Smith M. Patwary Brandon Norick P. LeGresley Samyam Rajbhandari ... Mohammad Shoeybi Yuxiong He Michael Houston Saurabh Tiwary Bryan Catanzaro MoE 155 742 0 28 Jan 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 847 9,683 0 28 Jan 2022
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Jack W. Rae Sebastian Borgeaud Trevor Cai Katie Millican Jordan Hoffmann ... Jeff Stanway L. Bennett Demis Hassabis Koray Kavukcuoglu G. Irving 136 1,325 0 08 Dec 2021
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 350 4,596 0 27 Oct 2021
A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers Shen-Yun Miao Chao-Chun Liang Keh-Yih Su 73 342 0 30 Jun 2021
Are NLP Models really able to Solve Simple Math Word Problems? Arkil Patel S. Bhattamishra Navin Goyal ReLM LRM 104 845 0 12 Mar 2021
Measuring Mathematical Problem Solving With the MATH Dataset Dan Hendrycks Collin Burns Saurav Kadavath Akul Arora Steven Basart Eric Tang Basel Alomair Jacob Steinhardt ReLM FaML 183 2,405 0 05 Mar 2021
LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning Jian Liu Leyang Cui Hanmeng Liu Dandan Huang Yile Wang Yue Zhang RALM 109 379 0 16 Jul 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 611 4,921 0 23 Jan 2020
PIQA: Reasoning about Physical Commonsense in Natural Language Yonatan Bisk Rowan Zellers Ronan Le Bras Jianfeng Gao Yejin Choi OOD LRM 178 1,847 0 26 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 488 20,342 0 23 Oct 2019
MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms Aida Amini Saadia Gabriel Shanchuan Lin Rik Koncel-Kedziorski Yejin Choi Hannaneh Hajishirzi AIMat ReLM AI4CE 127 581 0 30 May 2019
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions Christopher Clark Kenton Lee Ming-Wei Chang Tom Kwiatkowski Michael Collins Kristina Toutanova 244 1,560 0 24 May 2019
HellaSwag: Can a Machine Really Finish Your Sentence? Rowan Zellers Ari Holtzman Yonatan Bisk Ali Farhadi Yejin Choi 182 2,532 0 19 May 2019
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge Alon Talmor Jonathan Herzig Nicholas Lourie Jonathan Berant RALM 144 1,752 0 02 Nov 2018
Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering Todor Mihaylov Peter Clark Tushar Khot Ashish Sabharwal 119 1,570 0 08 Sep 2018
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge Peter Clark Isaac Cowhey Oren Etzioni Tushar Khot Ashish Sabharwal Carissa Schoenick Oyvind Tafjord ELM RALM LRM 172 2,670 0 14 Mar 2018
Crowdsourcing Multiple Choice Science Questions Johannes Welbl Nelson F. Liu Matt Gardner AI4Ed 106 522 0 19 Jul 2017