Enhancing Auto-regressive Chain-of-Thought through Loop-Aligned Reasoning

12 February 2025

Papers citing "Enhancing Auto-regressive Chain-of-Thought through Loop-Aligned Reasoning"

34 / 34 papers shown

Title
Efficient Reasoning Models: A Survey Sicheng Feng Gongfan Fang Xinyin Ma Xinchao Wang ReLM LRM 286 10 0 15 Apr 2025
CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation Zhenyi Shen Hanqi Yan Linhai Zhang Zhanghao Hu Yali Du Yulan He LRM 118 19 0 28 Feb 2025
LIFT: Improving Long Context Understanding Through Long Input Fine-Tuning Yansheng Mao Jiaqi Li Fanxu Meng Jing Xiong Zilong Zheng Muhan Zhang LLMAG RALM 126 1 0 18 Dec 2024
Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descent Bo Chen Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao Song 117 20 0 15 Oct 2024
Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning? Khashayar Gatmiry Nikunj Saunshi Sashank J. Reddi Stefanie Jegelka Sanjiv Kumar 97 20 0 10 Oct 2024
On Expressive Power of Looped Transformers: Theoretical Analysis and Enhancement via Timestep Encoding Kevin Xu Issei Sato 79 4 0 02 Oct 2024
Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning Debjit Paul Robert West Antoine Bosselut Boi Faltings ReLM LRM 77 25 0 21 Feb 2024
V-STaR: Training Verifiers for Self-Taught Reasoners Arian Hosseini Xingdi Yuan Nikolay Malkin Rameswar Panda Alessandro Sordoni Rishabh Agarwal ReLM LRM 71 119 0 09 Feb 2024
LESS: Selecting Influential Data for Targeted Instruction Tuning Mengzhou Xia Sadhika Malladi Suchin Gururangan Sanjeev Arora Danqi Chen 121 210 0 06 Feb 2024
Simulation of Graph Algorithms with Looped Transformers George Giapitzakis Kimon Fountoulakis 73 15 0 02 Feb 2024
The Impact of Reasoning Step Length on Large Language Models Mingyu Jin Qinkai Yu Dong Shu Haiyan Zhao Wenyue Hua Yanda Meng Yongfeng Zhang Jundong Li ReLM LRM 82 96 0 10 Jan 2024
Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models Avi Singh John D. Co-Reyes Rishabh Agarwal Ankesh Anand Piyush Patil ... Yamini Bansal Ethan Dyer Behnam Neyshabur Jascha Narain Sohl-Dickstein Noah Fiedel ALM LRM ReLM SyDa 179 171 0 11 Dec 2023
Conditions for Length Generalization in Learning Reasoning Skills Changnan Xiao Bing Liu LRM 59 7 0 22 Nov 2023
Looped Transformers are Better at Learning Learning Algorithms Liu Yang Kangwook Lee Robert D. Nowak Dimitris Papailiopoulos 69 25 0 21 Nov 2023
The Expressive Power of Transformers with Chain of Thought William Merrill Ashish Sabharwal LRM AI4CE ReLM 46 41 0 11 Oct 2023
PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training Dawei Zhu Nan Yang Liang Wang Yifan Song Wenhao Wu Furu Wei Sujian Li 105 85 0 19 Sep 2023
Scaling Relationship on Learning Mathematical Reasoning with Large Language Models Zheng Yuan Hongyi Yuan Cheng Li Guanting Dong Keming Lu Chuanqi Tan Chang Zhou Jingren Zhou LRM ALM 64 184 0 03 Aug 2023
Extending Context Window of Large Language Models via Positional Interpolation Shouyuan Chen Sherman Wong Liangjian Chen Yuandong Tian 138 513 0 27 Jun 2023
Let's Verify Step by Step Hunter Lightman V. Kosaraju Yura Burda Harrison Edwards Bowen Baker Teddy Lee Jan Leike John Schulman Ilya Sutskever K. Cobbe ALM OffRL LRM 122 1,044 0 31 May 2023
Randomized Positional Encodings Boost Length Generalization of Transformers Anian Ruoss Grégoire Delétang Tim Genewein Jordi Grau-Moya Róbert Csordás Mehdi Abbana Bennani Shane Legg J. Veness LLMAG 57 102 0 26 May 2023
Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective Guhao Feng Bohang Zhang Yuntian Gu Haotian Ye Di He Liwei Wang LRM 77 236 0 24 May 2023
Looped Transformers as Programmable Computers Angeliki Giannou Shashank Rajput Jy-yong Sohn Kangwook Lee Jason D. Lee Dimitris Papailiopoulos 69 105 0 30 Jan 2023
A Length-Extrapolatable Transformer Yutao Sun Li Dong Barun Patra Shuming Ma Shaohan Huang Alon Benhaim Vishrav Chaudhary Xia Song Furu Wei 68 120 0 20 Dec 2022
Dissecting Transformer Length Extrapolation via the Lens of Receptive Field Analysis Ta-Chung Chi Ting-Han Fan Alexander I. Rudnicky Peter J. Ramadge 40 40 0 20 Dec 2022
Decomposed Prompting: A Modular Approach for Solving Complex Tasks Tushar Khot H. Trivedi Matthew Finlayson Yao Fu Kyle Richardson Peter Clark Ashish Sabharwal ReLM LRM 93 437 0 05 Oct 2022
KERPLE: Kernelized Relative Positional Embedding for Length Extrapolation Ta-Chung Chi Ting-Han Fan Peter J. Ramadge Alexander I. Rudnicky 72 65 0 20 May 2022
STaR: Bootstrapping Reasoning With Reasoning E. Zelikman Yuhuai Wu Jesse Mu Noah D. Goodman ReLM LRM 83 468 0 28 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 616 9,009 0 28 Jan 2022
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 297 728 0 27 Aug 2021
RoFormer: Enhanced Transformer with Rotary Position Embedding Jianlin Su Yu Lu Shengfeng Pan Ahmed Murtadha Bo Wen Yunfeng Liu 172 2,307 0 20 Apr 2021
Measuring Mathematical Problem Solving With the MATH Dataset Dan Hendrycks Collin Burns Saurav Kadavath Akul Arora Steven Basart Eric Tang D. Song Jacob Steinhardt ReLM FaML 129 2,109 0 05 Mar 2021
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 335 19,824 0 23 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 290 6,420 0 26 Sep 2019
Universal Transformers Mostafa Dehghani Stephan Gouws Oriol Vinyals Jakob Uszkoreit Lukasz Kaiser 72 752 0 10 Jul 2018