Title
Hyena Hierarchy: Towards Larger Convolutional Language Models Michael Poli Stefano Massaroli Eric Q. Nguyen Daniel Y. Fu Tri Dao S. Baccus Yoshua Bengio Stefano Ermon Christopher Ré VLM 174 314 0 21 Feb 2023
Simple Hardware-Efficient Long Convolutions for Sequence Modeling Daniel Y. Fu Elliot L. Epstein Eric N. D. Nguyen A. Thomas Michael Zhang Tri Dao Atri Rudra Christopher Ré 64 55 0 13 Feb 2023
A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operations Bilal Chughtai Lawrence Chan Neel Nanda 108 103 0 06 Feb 2023
Conditioning Predictive Models: Risks and Strategies Evan Hubinger Adam Jermyn Johannes Treutlein Rubi Hudson Kate Woolverton 73 5 0 02 Feb 2023
Learning Functional Transduction Mathieu Chalvidal Thomas Serre Rufin VanRullen AI4CE 104 2 0 01 Feb 2023
Evaluating Neuron Interpretation Methods of NLP Models Yimin Fan Fahim Dalvi Nadir Durrani Hassan Sajjad 82 8 0 30 Jan 2023
Transformers as Algorithms: Generalization and Stability in In-context Learning Yingcong Li M. E. Ildiz Dimitris Papailiopoulos Samet Oymak 112 174 0 17 Jan 2023
Progress measures for grokking via mechanistic interpretability Neel Nanda Lawrence Chan Tom Lieberum Jess Smith Jacob Steinhardt 100 450 0 12 Jan 2023
Tracr: Compiled Transformers as a Laboratory for Interpretability David Lindner János Kramár Sebastian Farquhar Matthew Rahtz Tom McGrath Vladimir Mikulik 130 75 0 12 Jan 2023
A Survey on In-context Learning Qingxiu Dong Lei Li Damai Dai Ce Zheng Jingyuan Ma ... Zhiyong Wu Baobao Chang Xu Sun Lei Li Zhifang Sui ReLM AIMat 152 546 0 31 Dec 2022
Black-box language model explanation by context length probing Ondřej Cífka Antoine Liutkus MILM LRM 124 6 0 30 Dec 2022
Hungry Hungry Hippos: Towards Language Modeling with State Space Models Daniel Y. Fu Tri Dao Khaled Kamal Saab A. Thomas Atri Rudra Christopher Ré 157 404 0 28 Dec 2022
Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers Damai Dai Yutao Sun Li Dong Y. Hao Shuming Ma Zhifang Sui Furu Wei LRM 103 169 0 20 Dec 2022
Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations Xinxi Lyu Sewon Min Iz Beltagy Luke Zettlemoyer Hannaneh Hajishirzi VLM 73 68 0 19 Dec 2022
Training Trajectories of Language Models Across Scales Mengzhou Xia Mikel Artetxe Chunting Zhou Xi Lin Ramakanth Pasunuru Danqi Chen Luke Zettlemoyer Ves Stoyanov AIFin LRM 98 64 0 19 Dec 2022
Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale Hritik Bansal Karthik Gopalakrishnan Saket Dingliwal S. Bodapati Katrin Kirchhoff Dan Roth LRM 87 51 0 18 Dec 2022
Transformers learn in-context by gradient descent J. Oswald Eyvind Niklasson E. Randazzo João Sacramento A. Mordvintsev A. Zhmoginov Max Vladymyrov MLT 148 496 0 15 Dec 2022
Talking About Large Language Models Murray Shanahan AI4CE 124 275 0 07 Dec 2022
What learning algorithm is in-context learning? Investigations with linear models Ekin Akyürek Dale Schuurmans Jacob Andreas Tengyu Ma Denny Zhou 125 493 0 28 Nov 2022
GPT-Neo for commonsense reasoning -- a theoretical and practical lens Rohan Kashyap Vivek Kashyap Narendra C.P ReLM ELM LRM 84 7 0 28 Nov 2022
Explainability Via Causal Self-Talk Nicholas A. Roy Junkyung Kim Neil C. Rabinowitz CML 87 7 0 17 Nov 2022
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small Kevin Wang Alexandre Variengien Arthur Conmy Buck Shlegeris Jacob Steinhardt 320 563 0 01 Nov 2022
Characterizing Verbatim Short-Term Memory in Neural Language Models K. Armeni C. Honey Tal Linzen KELM RALM 85 6 0 24 Oct 2022
Language Models Understand Us, Poorly Jared Moore LRM 50 4 0 19 Oct 2022
The Debate Over Understanding in AI's Large Language Models Melanie Mitchell D. Krakauer ELM 155 222 0 14 Oct 2022
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes Shivam Garg Dimitris Tsipras Percy Liang Gregory Valiant 160 514 0 01 Aug 2022
Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks Tilman Raukur A. Ho Stephen Casper Dylan Hadfield-Menell AAML AI4CE 128 134 0 27 Jul 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Percy Liang J. Dean W. Fedus ELM ReLM LRM 322 2,524 0 15 Jun 2022
Scaling Laws and Interpretability of Learning from Repeated Data Danny Hernandez Tom B. Brown Tom Conerly Nova Dassarma Dawn Drain ... Catherine Olsson Dario Amodei Nicholas Joseph Jared Kaplan Sam McCandlish 88 118 0 21 May 2022
Towards Understanding Grokking: An Effective Theory of Representation Learning Ziming Liu O. Kitouni Niklas Nolte Eric J. Michaud Max Tegmark Mike Williams AI4CE 101 154 0 20 May 2022
STaR: Bootstrapping Reasoning With Reasoning E. Zelikman Yuhuai Wu Jesse Mu Noah D. Goodman ReLM LRM 157 512 0 28 Mar 2022
Acquisition of Chess Knowledge in AlphaZero Thomas McGrath A. Kapishnikov Nenad Tomašev Adam Pearce Demis Hassabis Been Kim Ulrich Paquet Vladimir Kramnik 77 168 0 17 Nov 2021
Inductive Biases and Variable Creation in Self-Attention Mechanisms Benjamin L. Edelman Surbhi Goel Sham Kakade Cyril Zhang 102 125 0 19 Oct 2021
Systematic human learning and generalization from a brief tutorial with explanatory feedback A. Nam James L. McClelland 38 1 0 10 Jul 2021