A Theory of Emergent In-Context Learning as Implicit Structure Induction

14 March 2023

Papers citing "A Theory of Emergent In-Context Learning as Implicit Structure Induction"

18 / 18 papers shown

Title
Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers Alireza Amiri Xinting Huang Mark Rofin Michael Hahn LRM 522 1 0 04 Feb 2025
Out-of-distribution generalization via composition: a lens through induction heads in Transformers Jiajun Song Zhuoyan Xu Yiqiao Zhong 141 10 0 31 Dec 2024
On the Training Convergence of Transformers for In-Context Classification of Gaussian Mixtures Wei Shen Ruida Zhou Jing Yang Cong Shen 64 3 0 15 Oct 2024
Representing Rule-based Chatbots with Transformers Dan Friedman Abhishek Panigrahi Danqi Chen 124 1 0 15 Jul 2024
Demystifying Chains, Trees, and Graphs of Thoughts Maciej Besta Florim Memedi Zhenyu Zhang Robert Gerstenberger Guangyuan Piao ... Aleš Kubíček H. Niewiadomski Aidan O'Mahony Onur Mutlu Torsten Hoefler AI4CE LRM 252 30 0 25 Jan 2024
Unsupervised Discontinuous Constituency Parsing with Mildly Context-Sensitive Grammars Aaron Courville R. Levy Yoon Kim 52 5 0 18 Dec 2022
Transformers learn in-context by gradient descent J. Oswald Eyvind Niklasson E. Randazzo João Sacramento A. Mordvintsev A. Zhmoginov Max Vladymyrov MLT 110 489 0 15 Dec 2022
Demystifying Prompts in Language Models via Perplexity Estimation Hila Gonen Srini Iyer Terra Blevins Noah A. Smith Luke Zettlemoyer LRM 122 210 0 08 Dec 2022
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes Shivam Garg Dimitris Tsipras Percy Liang Gregory Valiant 139 506 0 01 Aug 2022
Data Distributional Properties Drive Emergent In-Context Learning in Transformers Stephanie C. Y. Chan Adam Santoro Andrew Kyle Lampinen Jane X. Wang Aaditya K. Singh Pierre Harvey Richemond J. Mcclelland Felix Hill 134 263 0 22 Apr 2022
Iteratively Prompt Pre-trained Language Models for Chain of Thought Boshi Wang Xiang Deng Huan Sun KELM ReLM LRM 105 101 0 16 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 817 9,576 0 28 Jan 2022
Show Your Work: Scratchpads for Intermediate Computation with Language Models Maxwell Nye Anders Andreassen Guy Gur-Ari Henryk Michalewski Jacob Austin ... Aitor Lewkowycz Maarten Bosma D. Luan Charles Sutton Augustus Odena ReLM LRM 179 746 0 30 Nov 2021
An Explanation of In-context Learning as Implicit Bayesian Inference Sang Michael Xie Aditi Raghunathan Percy Liang Tengyu Ma ReLM BDL VPVLM LRM 204 751 0 03 Nov 2021
COGS: A Compositional Generalization Challenge Based on Semantic Interpretation Najoung Kim Tal Linzen CoGe 48 280 0 12 Oct 2020
Visually Grounded Compound PCFGs Yanpeng Zhao Ivan Titov 68 45 0 25 Sep 2020
Information-Theoretic Probing with Minimum Description Length Elena Voita Ivan Titov 85 275 0 27 Mar 2020
Designing and Interpreting Probes with Control Tasks John Hewitt Percy Liang 76 537 0 08 Sep 2019