Title
From Concepts to Components: Concept-Agnostic Attention Module Discovery in Transformers Jingtong Su Julia Kempe Karen Ullrich 14 0 0 20 Jun 2025
Latent Concept Disentanglement in Transformer-based Language Models Guan Zhe Hong Bhavya Vasudeva Vatsal Sharan Cyrus Rashtchian Prabhakar Raghavan Rina Panigrahy ReLM LRM 15 0 0 20 Jun 2025
Hidden Breakthroughs in Language Model Training Sara Kangaslahti Elan Rosenfeld Naomi Saphra 26 0 0 18 Jun 2025
Emergence of Primacy and Recency Effect in Mamba: A Mechanistic Point of View Muhammad Cendekia Airlangga Hilal AlQuabeh Munachiso Nwadike Kentaro Inui KELM 12 0 0 18 Jun 2025
Brewing Knowledge in Context: Distillation Perspectives on In-Context Learning Chengye Li Haiyun Liu Yuanxi Li 15 0 0 13 Jun 2025
Interpreting learned search: finding a transition model and value function in an RNN that plays Sokoban Mohammad Taufeeque Aaron David Tucker Adam Gleave Adrià Garriga-Alonso 37 0 0 11 Jun 2025
Query-Focused Retrieval Heads Improve Long-Context Reasoning and Re-ranking Wuwei Zhang Fangcong Yin Howard Yen Danqi Chen Xi Ye LRM 79 0 0 11 Jun 2025
Did I Faithfully Say What I Thought? Bridging the Gap Between Neural Activity and Self-Explanations in Large Language Models Milan Bhan Jean-Noel Vittaut Nicolas Chesneau Sarath Chandar Marie-Jeanne Lesot LRM 31 0 0 10 Jun 2025
On Finetuning Tabular Foundation Models Ivan Rubachev Akim Kotelnikov Nikolay Kartashev Artem Babenko 29 0 0 10 Jun 2025
Overclocking LLM Reasoning: Monitoring and Controlling Thinking Path Lengths in LLMs Roy Eisenstadt Itamar Zimerman Lior Wolf LRM 15 0 0 08 Jun 2025
Neural Network Reprogrammability: A Unified Theme on Model Reprogramming, Prompt Tuning, and Prompt Instruction Zesheng Ye C. Cai Ruijiang Dong Jianzhong Qi Lei Feng Pin-Yu Chen Feng Liu 208 0 0 05 Jun 2025
Sample Complexity and Representation Ability of Test-time Scaling Paradigms Baihe Huang Shanda Li Tianhao Wu Yiming Yang Ameet Talwalkar Kannan Ramchandran Michael I. Jordan Jiantao Jiao LRM 109 0 0 05 Jun 2025
A Statistical Physics of Language Model Reasoning Jack David Carson Amir Reisizadeh LRM AI4CE 78 0 0 04 Jun 2025
SkipGPT: Dynamic Layer Pruning Reinvented with Token Awareness and Module Decoupling Anhao Zhao Fanghua Ye Yingqi Fan Junlong Tong Zhiwei Fei Hui Su Xiaoyu Shen 66 0 0 04 Jun 2025
Bridging Neural ODE and ResNet: A Formal Error Bound for Safety Verification Abdelrahman Sayed Sayed Pierre-Jean Meyer Mohamed Ghazel 29 0 0 03 Jun 2025
The Unified Cognitive Consciousness Theory for Language Models: Anchoring Semantics, Thresholds of Activation, and Emergent Reasoning Edward Y. Chang LRM 20 0 0 02 Jun 2025
Attention Retrieves, MLP Memorizes: Disentangling Trainable Components in the Transformer Yihe Dong Lorenzo Noci Mikhail Khodak Mufan Li 50 0 0 01 Jun 2025
Circuit Stability Characterizes Language Model Generalization Alan Sun LRM 22 0 0 30 May 2025
Characterising the Inductive Biases of Neural Networks on Boolean Data Chris Mingard Lukas Seier Niclas Goring Andrei-Vlad Badelita Charles London Ard A. Louis AI4CE 37 0 0 29 May 2025
LoLA: Low-Rank Linear Attention With Sparse Caching Luke McDermott Robert W. Heath Jr. Rahul Parhi RALM 53 0 0 29 May 2025
Geometric Hyena Networks for Large-scale Equivariant Learning Artem Moskalev Mangal Prakash Junjie Xu Tianyu Cui Rui Liao Tommaso Mansi 45 1 0 28 May 2025
Born a Transformer -- Always a Transformer? Yana Veitsman Mayank Jobanputra Yash Sarrof Aleksandra Bakalova Vera Demberg Ellie Pavlick Michael Hahn 61 0 0 27 May 2025
Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective Junnan Liu Hongwei Liu Linchen Xiao Shudong Liu Taolin Zhang Zihan Ma Songyang Zhang Kai Chen LRM 120 0 0 26 May 2025
How Syntax Specialization Emerges in Language Models Xufeng Duan Zhaoqian Yao Yunhao Zhang Shaonan Wang Zhenguang G. Cai MILM LRM 71 0 0 26 May 2025
The Coverage Principle: A Framework for Understanding Compositional Generalization Hoyeon Chang Jinho Park Hanseul Cho Sohee Yang Miyoung Ko Hyeonbin Hwang Seungpil Won Dohaeng Lee Youbin Ahn Minjoon Seo 59 0 0 26 May 2025
The Birth of Knowledge: Emergent Features across Time, Space, and Scale in Large Language Models Shashata Sawmya Micah Adler Nir Shavit MILM 31 0 0 26 May 2025
Understanding Transformer from the Perspective of Associative Memory Shu Zhong Mingyu Xu Tenglong Ao Guang Shi 47 1 0 26 May 2025
Dynamic Manifold Evolution Theory: Modeling and Stability Analysis of Latent Representations in Large Language Models Yukun Zhang Qi Dong AI4CE 23 0 0 24 May 2025
TRACE for Tracking the Emergence of Semantic Representations in Transformers Nura Aljaafari Danilo S. Carvalho André Freitas 92 0 0 23 May 2025
The emergence of sparse attention: impact of data distribution and benefits of repetition Nicolas Zucchet Francesco dÁngelo Andrew Kyle Lampinen Stephanie C. Y. Chan 214 1 0 23 May 2025
Inference-Time Decomposition of Activations (ITDA): A Scalable Approach to Interpreting Large Language Models Patrick Leask Neel Nanda Noura Al Moubayed 87 1 0 23 May 2025
Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence Gouki Minegishi Hiroki Furuta Shohei Taniguchi Yusuke Iwasawa Yutaka Matsuo 81 0 0 22 May 2025
On the creation of narrow AI: hierarchy and nonlocality of neural network skills Eric J. Michaud Asher Parker-Sartori Max Tegmark 109 0 0 21 May 2025
The Strawberry Problem: Emergence of Character-level Understanding in Tokenized Language Models Adrian Cosma Stefan Ruseti Emilian Radoi Mihai Dascalu LRM 75 0 0 20 May 2025
Do Language Models Use Their Depth Efficiently? Róbert Csordás Christopher D. Manning Christopher Potts 208 2 0 20 May 2025
Soft Prompts for Evaluation: Measuring Conditional Distance of Capabilities Ross Nordby 58 0 0 20 May 2025
Language Models use Lookbacks to Track Beliefs Nikhil Prakash Natalie Shapira Arnab Sen Sharma Christoph Riedl Yonatan Belinkov Tamar Rott Shaham David Bau Atticus Geiger KELM 82 1 0 20 May 2025
Internal Chain-of-Thought: Empirical Evidence for Layer-wise Subtask Scheduling in LLMs Zhipeng Yang Junzhuo Li Siyu Xia Xuming Hu AIFin LRM 105 0 0 20 May 2025
AutoMathKG: The automated mathematical knowledge graph based on LLM and vector database Rong Bian Yu Geng Zijian Yang Bing Cheng 124 0 0 19 May 2025
Panda: A pretrained forecast model for universal representation of chaotic dynamics Jeffrey Lai Anthony Bao William Gilpin AI4TS AI4CE 101 0 0 19 May 2025
Understanding Task Representations in Neural Networks via Bayesian Ablation Andrew Nam Declan Campbell Thomas Griffiths Jonathan Cohen Sarah-Jane Leslie BDL 40 0 0 19 May 2025
Induction Head Toxicity Mechanistically Explains Repetition Curse in Large Language Models Shuxun Wang Qingyu Yin Chak Tou Leong Qiang Zhang Linyi Yang 58 0 0 17 May 2025
Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviors Jing Huang Junyi Tao Thomas Icard Diyi Yang Christopher Potts OODD 87 0 0 17 May 2025
Delta Attention: Fast and Accurate Sparse Attention Inference by Delta Correction Jeffrey Willette Heejun Lee Sung Ju Hwang 73 0 0 16 May 2025
Illusion or Algorithm? Investigating Memorization, Emergence, and Symbolic Processing in In-Context Learning Jingcheng Niu Subhabrata Dutta Ahmed Elshabrawy Harish Tayyar Madabushi Iryna Gurevych 148 1 0 16 May 2025
Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMs Jingcheng Niu Xingdi Yuan Tong Wang Hamidreza Saghir Amir H. Abdi 73 0 0 14 May 2025
AttentionInfluence: Adopting Attention Head Influence for Weak-to-Strong Pretraining Data Selection Kai Hua Steven Wu Ge Zhang Ke Shen LRM 83 0 0 12 May 2025
Task Reconstruction and Extrapolation for $π_0$ using Text Latent Quanyi Li 103 0 0 06 May 2025
Quiet Feature Learning in Algorithmic Tasks Prudhviraj Naidu Zixian Wang Leon Bergen R. Paturi VLM 113 0 0 06 May 2025
On the generalization of language models from in-context learning and finetuning: a controlled study Andrew Kyle Lampinen Arslan Chaudhry Stephanie Chan Cody Wild Diane Wan Alex Ku Jorg Bornschein Razvan Pascanu Murray Shanahan James L. McClelland 169 5 0 01 May 2025