Let's Think Dot by Dot: Hidden Computation in Transformer Language Models

24 April 2024

Papers citing "Let's Think Dot by Dot: Hidden Computation in Transformer Language Models"

22 / 22 papers shown

Title
Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought Hanlin Zhu Shibo Hao Zhiting Hu Jiantao Jiao Stuart Russell Yuandong Tian OffRL LRM 14 0 0 18 May 2025
Lost in Transmission: When and Why LLMs Fail to Reason Globally Tobias Schnabel Kiran Tomlinson Adith Swaminathan Jennifer Neville LRM 35 0 0 13 May 2025
Chain-of-Thought Tokens are Computer Program Variables Fangwei Zhu Peiyi Wang Zhifang Sui LRM 44 0 0 08 May 2025
Accurate and Diverse LLM Mathematical Reasoning via Automated PRM-Guided GFlowNets Adam Younsi Abdalgader Abubaker M. Seddik Hakim Hacid Salem Lahlou LRM 59 0 0 28 Apr 2025
RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning Zihan Wang Kaidi Wang Q. Wang Pingyue Zhang Linjie Li ... Jiajun Wu L. Fei-Fei Lijuan Wang Yejin Choi Manling Li 92 6 0 24 Apr 2025
Efficient Reasoning Models: A Survey Sicheng Feng Gongfan Fang Xinyin Ma Xinchao Wang ReLM LRM 172 4 0 15 Apr 2025
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models Yang Sui Yu-Neng Chuang Guanchu Wang Jiamu Zhang Tianyi Zhang ... Hongyi Liu Andrew Wen Shaochen Zhong Hanjie Chen OffRL ReLM LRM 86 35 0 20 Mar 2025
Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More More Arvid Frydenlund LRM 58 0 0 13 Mar 2025
CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation Zhenyi Shen Hanqi Yan Linhai Zhang Zhanghao Hu Yali Du Yulan He LRM 72 9 0 28 Feb 2025
Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers Alireza Amiri Xinting Huang Mark Rofin Michael Hahn LRM 233 0 0 04 Feb 2025
Padding Tone: A Mechanistic Analysis of Padding Tokens in T2I Models Michael Toker Ido Galil Hadas Orgad Rinon Gal Yoad Tewel Gal Chechik Yonatan Belinkov DiffM 56 2 0 12 Jan 2025
Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoning Md Rifat Arefin G. Subbaraj Nicolas Angelard-Gontier Yann LeCun Irina Rish Ravid Shwartz-Ziv C. Pal LRM 207 0 0 04 Nov 2024
Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA Sangmin Bae Adam Fisch Hrayr Harutyunyan Ziwei Ji Seungyeon Kim Tal Schuster KELM 84 5 0 28 Oct 2024
Take Caution in Using LLMs as Human Surrogates: Scylla Ex Machina Yuan Gao Dokyun Lee Gordon Burtch Sina Fazelpour LRM 56 7 0 25 Oct 2024
System 2 Reasoning Capabilities Are Nigh Scott C. Lowe VLM LRM 51 0 0 04 Oct 2024
Reasoning in Large Language Models: A Geometric Perspective Romain Cosentino Sarath Shekkizhar LRM 46 2 0 02 Jul 2024
Eliminating Position Bias of Language Models: A Mechanistic Approach Ziqi Wang Hanlin Zhang Xiner Li Kuan-Hao Huang Chi Han Shuiwang Ji Sham Kakade Hao Peng Heng Ji 62 12 0 01 Jul 2024
PORT: Preference Optimization on Reasoning Traces Salem Lahlou Abdalgader Abubaker Hakim Hacid LRM 41 2 0 23 Jun 2024
Break the Chain: Large Language Models Can be Shortcut Reasoners Mengru Ding Hanmeng Liu Zhizhang Fu Jian Song Wenbo Xie Yue Zhang KELM LRM 36 7 0 04 Jun 2024
Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Models Marianna Nezhurina Lucia Cipolina-Kun Mehdi Cherti J. Jitsev LLMAG LRM ELM ReLM 58 28 0 04 Jun 2024
A Logic for Expressing Log-Precision Transformers William Merrill Ashish Sabharwal ReLM NAI LRM 53 47 0 06 Oct 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 416 8,650 0 28 Jan 2022