Title
Transformers Learn Shortcuts to Automata Bingbin Liu Jordan T. Ash Surbhi Goel A. Krishnamurthy Cyril Zhang OffRL LRM 118 174 0 19 Oct 2022
A Logic for Expressing Log-Precision Transformers William Merrill Ashish Sabharwal ReLM NAI LRM 77 52 0 06 Oct 2022
Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought Abulhair Saparov He He ELM LRM ReLM 210 303 0 03 Oct 2022
Faithful Reasoning Using Large Language Models Antonia Creswell Murray Shanahan ReLM LRM 60 124 0 30 Aug 2022
Induced Natural Language Rationales and Interleaved Markup Tokens Enable Extrapolation in Large Language Models M. Bueno Carlos Gemmel Jeffrey Stephen Dalton R. Lotufo Rodrigo Nogueira LRM 74 12 0 24 Aug 2022
Exploring Length Generalization in Large Language Models Cem Anil Yuhuai Wu Anders Andreassen Aitor Lewkowycz Vedant Misra V. Ramasesh Ambrose Slone Guy Gur-Ari Ethan Dyer Behnam Neyshabur ReLM LRM 78 168 0 11 Jul 2022
The Parallelism Tradeoff: Limitations of Log-Precision Transformers William Merrill Ashish Sabharwal 54 112 0 02 Jul 2022
PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change Karthik Valmeekam Matthew Marquez Alberto Olmo S. Sreedharan Subbarao Kambhampati ReLM LRM 85 227 0 21 Jun 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Percy Liang J. Dean W. Fedus ELM ReLM LRM 267 2,468 0 15 Jun 2022
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models Aarohi Srivastava Abhinav Rastogi Abhishek Rao Abu Awal Md Shoeb Abubakar Abid ... Zhuoye Zhao Zijian Wang Zijie J. Wang Zirui Wang Ziyi Wu ELM 166 1,749 0 09 Jun 2022
Unveiling Transformers with LEGO: a synthetic reasoning task Yi Zhang A. Backurs Sébastien Bubeck Ronen Eldan Suriya Gunasekar Tal Wagner LRM 78 89 0 09 Jun 2022
NaturalProver: Grounded Mathematical Proof Generation with Language Models Sean Welleck Jiacheng Liu Ximing Lu Hannaneh Hajishirzi Yejin Choi AIMat LRM 66 72 0 25 May 2022
Towards Understanding Grokking: An Effective Theory of Representation Learning Ziming Liu O. Kitouni Niklas Nolte Eric J. Michaud Max Tegmark Mike Williams AI4CE 72 152 0 20 May 2022
Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning Antonia Creswell Murray Shanahan I. Higgins ReLM LRM 87 360 0 19 May 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 446 6,222 0 05 Apr 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 763 9,330 0 28 Jan 2022
LaMDA: Language Models for Dialog Applications R. Thoppilan Daniel De Freitas Jamie Hall Noam M. Shazeer Apoorv Kulshreshtha ... Blaise Aguera-Arcas Claire Cui M. Croak Ed H. Chi Quoc Le ALM 126 1,593 0 20 Jan 2022
Unobserved Local Structures Make Compositional Generalization Hard Ben Bogin Shivanshu Gupta Jonathan Berant CoGe 58 33 0 15 Jan 2022
Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets Alethea Power Yuri Burda Harrison Edwards Igor Babuschkin Vedant Misra 75 356 0 06 Jan 2022
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Jack W. Rae Sebastian Borgeaud Trevor Cai Katie Millican Jordan Hoffmann ... Jeff Stanway L. Bennett Demis Hassabis Koray Kavukcuoglu G. Irving 109 1,311 0 08 Dec 2021
Show Your Work: Scratchpads for Intermediate Computation with Language Models Maxwell Nye Anders Andreassen Guy Gur-Ari Henryk Michalewski Jacob Austin ... Aitor Lewkowycz Maarten Bosma D. Luan Charles Sutton Augustus Odena ReLM LRM 166 741 0 30 Nov 2021
Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematics Sean Welleck Peter West Jize Cao Yejin Choi 91 30 0 28 Sep 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 303 754 0 27 Aug 2021
Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks Avi Schwarzschild Eitan Borgnia Arjun Gupta Furong Huang U. Vishkin Micah Goldblum Tom Goldstein 59 75 0 08 Jun 2021
Neural Algorithmic Reasoning Petar Velickovic Charles Blundell NAI OOD 29 100 0 06 May 2021
ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language Oyvind Tafjord Bhavana Dalvi Peter Clark 68 273 0 24 Dec 2020
On the Practical Ability of Recurrent Neural Networks to Recognize Hierarchical Languages S. Bhattamishra Kabir Ahuja Navin Goyal ReLM 44 12 0 08 Nov 2020
Scaling Laws for Autoregressive Generative Modeling T. Henighan Jared Kaplan Mor Katz Mark Chen Christopher Hesse ... Nick Ryder Daniel M. Ziegler John Schulman Dario Amodei Sam McCandlish 90 422 0 28 Oct 2020
Limitations of Autoregressive Models and Their Alternatives Chu-cheng Lin Aaron Jaech Xin Li Matthew R. Gormley Jason Eisner 45 61 0 22 Oct 2020
The EOS Decision and Length Extrapolation Benjamin Newman John Hewitt Percy Liang Christopher D. Manning 52 48 0 14 Oct 2020
Critical Thinking for Language Models Gregor Betz Christian Voigt Kyle Richardson SyDa ReLM LRM AI4CE 72 35 0 15 Sep 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 731 41,894 0 28 May 2020
Shortcut Learning in Deep Neural Networks Robert Geirhos J. Jacobsen Claudio Michaelis R. Zemel Wieland Brendel Matthias Bethge Felix Wichmann 201 2,049 0 16 Apr 2020
Transformers as Soft Reasoners over Language Peter Clark Oyvind Tafjord Kyle Richardson ReLM OffRL LRM 96 358 0 14 Feb 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 557 4,797 0 23 Jan 2020
Location Attention for Extrapolation to Longer Sequences Yann Dubois Gautier Dagan Dieuwke Hupkes Elia Bruni 44 43 0 10 Nov 2019
Theoretical Limitations of Self-Attention in Neural Sequence Models Michael Hahn 53 271 0 16 Jun 2019
Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems Wang Ling Dani Yogatama Chris Dyer Phil Blunsom AIMat 79 727 0 11 May 2017