Achieving and Understanding Out-of-Distribution Generalization in
Systematic Reasoning in Small-Scale Transformers

Achieving and Understanding Out-of-Distribution Generalization in Systematic Reasoning in Small-Scale Transformers

7 October 2022

Trevor C. Maxfield

James L. McClelland

Papers citing "Achieving and Understanding Out-of-Distribution Generalization in Systematic Reasoning in Small-Scale Transformers"

13 / 13 papers shown

Title
Systematic Generalization and Emergent Structures in Transformers Trained on Structured Tasks Yuxuan Li James L. McClelland 80 19 0 02 Oct 2022
Solving Quantitative Reasoning Problems with Language Models Aitor Lewkowycz Anders Andreassen David Dohan Ethan Dyer Henryk Michalewski ... Theo Gutman-Solo Yuhuai Wu Behnam Neyshabur Guy Gur-Ari Vedant Misra ReLM ELM LRM 138 827 0 29 Jun 2022
Transformer Language Models without Positional Encodings Still Learn Positional Information Adi Haviv Ori Ram Ofir Press Peter Izsak Omer Levy 71 123 0 30 Mar 2022
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? Sewon Min Xinxi Lyu Ari Holtzman Mikel Artetxe M. Lewis Hannaneh Hajishirzi Luke Zettlemoyer LLMAG LRM 142 1,471 0 25 Feb 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 738 9,267 0 28 Jan 2022
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 225 4,354 0 27 Oct 2021
Do Prompt-Based Models Really Understand the Meaning of their Prompts? Albert Webson Ellie Pavlick LRM 101 368 0 02 Sep 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 297 749 0 27 Aug 2021
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 624 41,736 0 28 May 2020
Relational inductive biases, deep learning, and graph networks Peter W. Battaglia Jessica B. Hamrick V. Bapst Alvaro Sanchez-Gonzalez V. Zambaldi ... Pushmeet Kohli M. Botvinick Oriol Vinyals Yujia Li Razvan Pascanu AI4CE NAI 619 3,112 0 04 Jun 2018
Recurrent Relational Networks Rasmus Berg Palm Ulrich Paquet Ole Winther GNN ReLM NAI 89 139 0 21 Nov 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 628 130,942 0 12 Jun 2017
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.4K 149,842 0 22 Dec 2014