Title
Neural Networks Learn Statistics of Increasing Complexity Nora Belrose Quintin Pope Lucia Quirke Alex Troy Mallen Xiaoli Z. Fern 68 11 0 06 Feb 2024
A phase transition between positional and semantic learning in a solvable model of dot-product attention Hugo Cui Freya Behrens Florent Krzakala Lenka Zdeborová MLT 98 16 0 06 Feb 2024
Challenges in Mechanistically Interpreting Model Representations Satvik Golechha James Dao 71 3 0 06 Feb 2024
On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigm Zhanpeng Zhou Zijun Chen Yilan Chen Bo Zhang Junchi Yan MoMe 102 11 0 06 Feb 2024
An introduction to graphical tensor notation for mechanistic interpretability Jordan K. Taylor 67 3 0 02 Feb 2024
Repeat After Me: Transformers are Better than State Space Models at Copying Samy Jelassi David Brandfonbrener Sham Kakade Eran Malach 176 95 0 01 Feb 2024
LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling law Toni J. B. Liu Nicolas Boullé Raphaël Sarfati Christopher Earls AI4TS 106 18 0 01 Feb 2024
Assertion Detection Large Language Model In-context Learning LoRA Fine-tuning Yuelyu Ji Zeshui Yu Yanshan Wang 72 0 0 31 Jan 2024
Rethinking Interpretability in the Era of Large Language Models Chandan Singh J. Inala Michel Galley Rich Caruana Jianfeng Gao LRM AI4CE 126 71 0 30 Jan 2024
In-Context Language Learning: Architectures and Algorithms Ekin Akyürek Bailin Wang Yoon Kim Jacob Andreas LRM ReLM 131 56 0 23 Jan 2024
Universal Neurons in GPT2 Language Models Wes Gurnee Theo Horsley Zifan Carl Guo Tara Rezaei Kheirkhah Qinyi Sun Will Hathaway Neel Nanda Dimitris Bertsimas MILM 158 47 0 22 Jan 2024
Understanding Video Transformers via Universal Concept Discovery M. Kowal Achal Dave Rares Andrei Ambrus Adrien Gaidon Konstantinos G. Derpanis P. Tokmakov ViT 115 12 0 19 Jan 2024
Anchor function: a type of benchmark functions for studying language models Zhongwang Zhang Zhiwei Wang Junjie Yao Zhangchen Zhou Xiaolong Li E. Weinan Z. Xu 118 7 0 16 Jan 2024
Carrying over algorithm in transformers J. Kruthoff 92 0 0 15 Jan 2024
AI-as-exploration: Navigating intelligence space Dimitri Coelho Mollo 86 1 0 15 Jan 2024
Batch-ICL: Effective, Efficient, and Order-Agnostic In-Context Learning Kaiyi Zhang Ang Lv Yuhan Chen Hansen Ha Tao Xu Rui Yan 101 21 0 12 Jan 2024
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems Tianyu Cui Yanling Wang Chuanpu Fu Yong Xiao Sijia Li ... Junwu Xiong Xinyu Kong ZuJie Wen Ke Xu Qi Li 165 64 0 11 Jan 2024
MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts Maciej Pióro Kamil Ciebiera Krystian Król Jan Ludziejewski Michał Krutul Jakub Krajewski Szymon Antoniak Piotr Miłoś Marek Cygan Sebastian Jaszczur MoE Mamba 84 56 0 08 Jan 2024
Beyond Output Matching: Bidirectional Alignment for Enhanced In-Context Learning Chengwei Qin Wenhan Xia Fangkai Jiao Chen Chen Yuchen Hu Bosheng Ding R. Chen Shafiq Joty 106 7 0 28 Dec 2023
Forbidden Facts: An Investigation of Competing Objectives in Llama-2 Tony T. Wang Miles Wang Kaivu Hariharan Nir Shavit 69 2 0 14 Dec 2023
SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention Róbert Csordás Piotr Piekos Kazuki Irie Jürgen Schmidhuber MoE 55 16 0 13 Dec 2023
Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context Xiang Cheng Yuxin Chen S. Sra 111 41 0 11 Dec 2023
TCNCA: Temporal Convolution Network with Chunked Attention for Scalable Sequence Processing Aleksandar Terzić Michael Hersche G. Karunaratne Zixiao Huang Abu Sebastian Abbas Rahimi AI4TS 57 1 0 09 Dec 2023
Emergence and Function of Abstract Representations in Self-Supervised Transformers Quentin RV. Ferry Joshua Ching Takashi Kawai 78 3 0 08 Dec 2023
Interpretability Illusions in the Generalization of Simplified Models Dan Friedman Andrew Kyle Lampinen Lucas Dixon Danqi Chen Asma Ghandeharioun 113 15 0 06 Dec 2023
FlexModel: A Framework for Interpretability of Distributed Large Language Models Matthew Choi Muhammad Adil Asif John Willes David Emerson AI4CE ALM 64 1 0 05 Dec 2023
Transformers are uninterpretable with myopic methods: a case study with bounded Dyck grammars Kaiyue Wen Yuchen Li Bing Liu Andrej Risteski 88 24 0 03 Dec 2023
Honesty Is the Best Policy: Defining and Mitigating AI Deception Francis Rhys Ward Francesco Belardinelli Francesca Toni Tom Everitt 179 31 0 03 Dec 2023
MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning Chaoyi Zhang Kevin Qinghong Lin Zhengyuan Yang Jianfeng Wang Linjie Li Chung-Ching Lin Zicheng Liu Lijuan Wang VGen 109 32 0 29 Nov 2023
One Fits All: Universal Time Series Analysis by Pretrained LM and Specially Designed Adaptors Tian Zhou Peisong Niu Xue Wang Liang Sun Rong Jin AI4TS 101 2 0 24 Nov 2023
Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks Rahul Ramesh Ekdeep Singh Lubana Mikail Khona Robert P. Dick Hidenori Tanaka CoGe 87 12 0 21 Nov 2023
Looped Transformers are Better at Learning Learning Algorithms Liu Yang Kangwook Lee Robert D. Nowak Dimitris Papailiopoulos 97 26 0 21 Nov 2023
Flexible Model Interpretability through Natural Language Model Editing Karel DÓosterlinck Thomas Demeester Chris Develder Christopher Potts MILM KELM 54 0 0 17 Nov 2023
Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarks Ting-Yun Chang Jesse Thomason Robin Jia 81 19 0 15 Nov 2023
The Transient Nature of Emergent In-Context Learning in Transformers Aaditya K. Singh Stephanie C. Y. Chan Ted Moskovitz Erin Grant Andrew M. Saxe Felix Hill 144 44 0 14 Nov 2023
In-context Learning and Gradient Descent Revisited Gilad Deutch Nadav Magar Tomer Bar Natan Guy Dar 82 16 0 13 Nov 2023
Finding and Editing Multi-Modal Neurons in Pre-Trained Transformers Haowen Pan Yixin Cao Xiaozhi Wang Xun Yang Meng Wang KELM 106 27 0 13 Nov 2023
Towards Interpretable Sequence Continuation: Analyzing Shared Circuits in Large Language Models Michael Lan Phillip H. S. Torr Fazl Barez LRM 60 3 0 07 Nov 2023
Uncovering Intermediate Variables in Transformers using Circuit Probing Michael A. Lepori Thomas Serre Ellie Pavlick 161 7 0 07 Nov 2023
Training Dynamics of Contextual N-Grams in Language Models Lucia Quirke Lovis Heindrich Wes Gurnee Neel Nanda 75 5 0 01 Nov 2023
The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysis Yuxiang Zhou Jiazheng Li Yanzheng Xiang Hanqi Yan Lin Gui Yulan He 91 19 0 01 Nov 2023
What Algorithms can Transformers Learn? A Study in Length Generalization Hattie Zhou Arwen Bradley Etai Littwin Noam Razin Omid Saremi Josh Susskind Samy Bengio Preetum Nakkiran 105 125 0 24 Oct 2023
How Much Context Does My Attention-Based ASR System Need? Robert Flynn Anton Ragni 63 2 0 24 Oct 2023
When Language Models Fall in Love: Animacy Processing in Transformer Language Models Michael Hanna Yonatan Belinkov Sandro Pezzelle 53 13 0 23 Oct 2023
Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problems David T. Hoffmann Simon Schrodi Jelena Bratulić Nadine Behrmann Volker Fischer Thomas Brox 116 8 0 19 Oct 2023
Instilling Inductive Biases with Subnetworks Enyan Zhang Michael A. Lepori Ellie Pavlick AI4CE 78 5 0 17 Oct 2023
How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations Tianyu Guo Wei Hu Song Mei Huan Wang Caiming Xiong Silvio Savarese Yu Bai 107 60 0 16 Oct 2023
Circuit Component Reuse Across Tasks in Transformer Language Models Jack Merullo Carsten Eickhoff Ellie Pavlick 84 71 0 12 Oct 2023
Do pretrained Transformers Learn In-Context by Gradient Descent? Lingfeng Shen Aayush Mishra Daniel Khashabi 135 9 0 12 Oct 2023
Large Language Models Are Zero-Shot Time Series Forecasters Nate Gruver Marc Finzi Shikai Qiu Andrew Gordon Wilson AI4TS 97 375 0 11 Oct 2023