Title
Structural Inference: Interpreting Small Language Models with Susceptibilities Garrett Baker George Wang Jesse Hoogland Daniel Murfet AAML 157 1 0 25 Apr 2025
In-Context Learning can distort the relationship between sequence likelihoods and biological fitness Pranav Kantroo Günter P. Wagner Benjamin B. Machta 131 0 0 23 Apr 2025
Understanding the Skill Gap in Recurrent Language Models: The Role of the Gather-and-Aggregate Mechanism Aviv Bick Eric P. Xing Albert Gu RALM 146 1 0 22 Apr 2025
Bigram Subnetworks: Mapping to Next Tokens in Transformer Language Models Tyler A. Chang Benjamin Bergen 143 0 0 21 Apr 2025
The Geometry of Self-Verification in a Task-Specific Reasoning Model Andrew Lee Lihao Sun Chris Wendler Fernanda Viégas Martin Wattenberg LRM 170 1 0 19 Apr 2025
Emergence of Computational Structure in a Neural Network Physics Simulator Rohan Hitchcock Gary W. Delaney J. Manton Richard Scalzo Jingge Zhu 63 0 0 16 Apr 2025
Steering off Course: Reliability Challenges in Steering Language Models Patrick Queiroz Da Silva Hari Sethuraman Dheeraj Rajagopal Hannaneh Hajishirzi Sachin Kumar LLMSV 100 2 0 06 Apr 2025
Page Classification for Print Imaging Pipeline Shaoyuan Xu Cheng Lu Mark Shaw Peter Bauer J. Allebach VLM 98 1 0 03 Apr 2025
Almost Bayesian: The Fractal Dynamics of Stochastic Gradient Descent Max Hennick Stijn De Baerdemacker 82 0 0 28 Mar 2025
Why Representation Engineering Works: A Theoretical and Empirical Study in Vision-Language Models Bowei Tian Xuntao Lyu Meng Liu Hongyi Wang Ang Li 94 0 0 25 Mar 2025
Efficient but Vulnerable: Benchmarking and Defending LLM Batch Prompting Attack Murong Yue Ziyu Yao SILM AAML 101 0 0 18 Mar 2025
Combining Causal Models for More Accurate Abstractions of Neural Networks Theodora-Mara Pîslar Sara Magliacane Atticus Geiger AI4CE 87 0 0 14 Mar 2025
Taming Knowledge Conflicts in Language Models Gaotang Li Yuzhong Chen Hanghang Tong KELM 86 2 0 14 Mar 2025
PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs Oskar van der Wal Pietro Lesci Max Muller-Eberstein Naomi Saphra Hailey Schoelkopf Willem H. Zuidema Stella Biderman LRM 108 2 0 12 Mar 2025
Identity Lock: Locking API Fine-tuned LLMs With Identity-based Wake Words Hongyu Su Yifeng Gao Yifan Ding Jie Zhang 105 0 0 10 Mar 2025
Strategy Coopetition Explains the Emergence and Transience of In-Context Learning Aaditya K. Singh Ted Moskovitz Sara Dragutinovic Felix Hill Stephanie C. Y. Chan Andrew Saxe 441 5 0 07 Mar 2025
Biases in Large Language Model-Elicited Text: A Case Study in Natural Language Inference Grace Proebsting Adam Poliak 101 0 0 06 Mar 2025
(How) Do Language Models Track State? Belinda Z. Li Zifan Carl Guo Jacob Andreas LRM 115 3 0 04 Mar 2025
Triple Phase Transitions: Understanding the Learning Dynamics of Large Language Models from a Neuroscience Perspective Yuko Nakagi Keigo Tada Sota Yoshino Shinji Nishimoto Yu Takagi LRM 128 0 0 28 Feb 2025
Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models Yukang Yang Declan Campbell Kaixuan Huang Mengdi Wang Jonathan D. Cohen Taylor Webb LRM 193 5 0 27 Feb 2025
Language Models Grow Less Humanlike beyond Phase Transition Tatsuya Aoyama Ethan Wilcox 86 1 0 26 Feb 2025
Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activations Lucy Farnik Tim Lawson Conor Houghton Laurence Aitchison 107 1 0 25 Feb 2025
On the Robustness of Transformers against Context Hijacking for Linear Classification Tianle Li Chenyang Zhang Xingwu Chen Yuan Cao Difan Zou 126 2 0 24 Feb 2025
Representation Engineering for Large-Language Models: Survey and Research Challenges Lukasz Bartoszcze Sarthak Munshi Bryan Sukidi Jennifer Yen Zejia Yang David Williams-King Linh Le Kosi Asuzu Carsten Maple 176 0 0 24 Feb 2025
SAE-V: Interpreting Multimodal Models for Enhanced Alignment Hantao Lou Changye Li Yalan Qin Yaodong Yang 122 1 0 22 Feb 2025
CoT-ICL Lab: A Synthetic Framework for Studying Chain-of-Thought Learning from In-Context Demonstrations Vignesh Kothapalli Hamed Firooz Maziar Sanjabi 130 0 0 21 Feb 2025
A Close Look at Decomposition-based XAI-Methods for Transformer Language Models L. Arras Bruno Puri Patrick Kahardipraja Sebastian Lapuschkin Wojciech Samek 98 3 0 21 Feb 2025
Looped ReLU MLPs May Be All You Need as Practical Programmable Computers Yingyu Liang Zhizhou Sha Zhenmei Shi Zhao Song Yufa Zhou 179 19 0 21 Feb 2025
LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformers Anton Razzhigaev Matvey Mikhalchuk Temurbek Rahmatullaev Elizaveta Goncharova Polina Druzhinina Ivan Oseledets Andrey Kuznetsov 125 5 0 20 Feb 2025
An explainable transformer circuit for compositional generalization Cheng Tang Brenden Lake Mehrdad Jazayeri LRM 149 3 0 19 Feb 2025
Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis Xiang Wang Yan Hu Wenyu Du Reynold Cheng Benyou Wang Difan Zou 149 3 0 17 Feb 2025
Hyper-SET: Designing Transformers via Hyperspherical Energy Minimization Yunzhe Hu Difan Zou Dong Xu 157 1 0 17 Feb 2025
Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers Alireza Amiri Xinting Huang Mark Rofin Michael Hahn LRM 575 3 0 04 Feb 2025
It's Not Just a Phase: On Investigating Phase Transitions in Deep Learning-based Side-channel Analysis Sengim Karayalçin Marina Krček Stjepan Picek AAML 157 0 0 01 Feb 2025
Training Dynamics of In-Context Learning in Linear Attention Yedi Zhang Aaditya K. Singh Peter E. Latham Andrew Saxe MLT 130 5 0 27 Jan 2025
Physics of Skill Learning Ziming Liu Yizhou Liu Eric J. Michaud Jeff Gore Max Tegmark 119 2 0 21 Jan 2025
Unlocking In-Context Learning for Natural Datasets Beyond Language Modelling Jelena Bratulić Sudhanshu Mittal David T. Hoffmann Samuel Böhm R. Schirrmeister T. Ball Christian Rupprecht Thomas Brox 90 1 0 09 Jan 2025
In-Context Learning with Iterative Demonstration Selection Chengwei Qin Aston Zhang Chong Chen Anirudh Dagar Wenming Ye LRM 188 55 0 31 Dec 2024
Out-of-distribution generalization via composition: a lens through induction heads in Transformers Jiajun Song Zhuoyan Xu Yiqiao Zhong 158 10 0 31 Dec 2024
Unveiling Visual Perception in Language Models: An Attention Head Analysis Approach Jing Bi Junjia Guo Yunlong Tang Lianggong Wen Zhang Liu Chenliang Xu 52 6 0 24 Dec 2024
Tracking the Feature Dynamics in LLM Training: A Mechanistic Study Yang Xu Yansen Wang Hao Wang Hao Wang 410 4 0 23 Dec 2024
Understanding Knowledge Hijack Mechanism in In-context Learning through Associative Memory Shuo Wang Issei Sato 183 0 0 16 Dec 2024
Concept Learning in the Wild: Towards Algorithmic Understanding of Neural Networks Elad Shohama Hadar Cohena Khalil Wattada Havana Rikab Dan Vilenchik 115 1 0 15 Dec 2024
PLD+: Accelerating LLM inference by leveraging Language Model Artifacts Shwetha Somasundaram Anirudh Phukan Apoorv Saxena 186 3 0 02 Dec 2024
Enhancing Few-Shot Vision-Language Classification with Large Multimodal Model Features Chancharik Mitra Brandon Huang Tianning Chai Zhiqiu Lin Assaf Arbelle Rogerio Feris Leonid Karlinsky Trevor Darrell Deva Ramanan Roei Herzig VLM 391 4 0 28 Nov 2024
Predicting Emergent Capabilities by Finetuning Charlie Snell Eric Wallace Dan Klein Sergey Levine ELM LRM 142 6 0 25 Nov 2024
Selective Attention: Enhancing Transformer through Principled Context Control Xuechen Zhang Xiangyu Chang Mingchen Li Amit K. Roy-Chowdhury Jiasi Chen Samet Oymak 129 3 0 19 Nov 2024
Transformer verbatim in-context retrieval across time and scale Kristijan Armeni Marko Pranjic Senja Pollak 54 1 0 11 Nov 2024
Can Custom Models Learn In-Context? An Exploration of Hybrid Architecture Performance on In-Context Learning Tasks Ryan Campbell Nelson Lojo Kesava Viswanadha Christoffer Grondal Tryggestad Derrick Han Sun Sriteja Vijapurapu August Rolfsen Anant Sahai 58 0 0 06 Nov 2024
A Implies B: Circuit Analysis in LLMs for Propositional Logical Reasoning Guan Zhe Hong Nishanth Dikkala Enming Luo Cyrus Rashtchian Xin Wang Rina Panigrahy OffRL LRM NAI 96 0 0 06 Nov 2024