Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens

23 May 2025

Papers citing "Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens"

40 / 40 papers shown

Title
Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMs Jinyan Su Jennifer Healey Preslav Nakov Claire Cardie LRM 179 9 0 30 Apr 2025
Dynamic Early Exit in Reasoning Models Chenxu Yang Qingyi Si Yongjie Duan Zheliang Zhu Chenyu Zhu Zheng Lin Zheng Lin Li Cao Weiping Wang ReLM LRM 69 14 0 22 Apr 2025
Reasoning Models Can Be Effective Without Thinking Wenjie Ma Jingxuan He Charlie Snell Tyler Griggs Sewon Min Matei A. Zaharia ReLM LRM 75 36 1 14 Apr 2025
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models Yang Sui Yu-Neng Chuang Guanchu Wang Jiamu Zhang Tianyi Zhang ... Hongyi Liu Andrew Wen Shaochen Zhong Hanjie Chen OffRL ReLM LRM 111 71 0 20 Mar 2025
Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning Yuxiao Qu Matthew Y. R. Yang Amrith Rajagopal Setlur Lewis Tunstall E. Beeching Ruslan Salakhutdinov Aviral Kumar OffRL 94 28 0 10 Mar 2025
DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models Yi Shen Jing Zhang Jieyun Huang Shuming Shi Wenjing Zhang Jiangze Yan Rongjia Du Ning Wang Kai Wang LRM 85 39 0 06 Mar 2025
Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning Wenkai Yang Shuming Ma Yankai Lin Furu Wei LRM 57 35 0 25 Feb 2025
The Relationship Between Reasoning and Performance in Large Language Models -- o3 (mini) Thinks Harder, Not Longer Marthe Ballon Andres Algaba Vincent Ginis LRM ReLM 59 13 0 24 Feb 2025
Entropy-Lens: The Information Signature of Transformer Computations Riccardo Ali Francesco Caso Christopher Irwin Pietro Lio 69 3 0 23 Feb 2025
CoT-Valve: Length-Compressible Chain-of-Thought Tuning Xinyin Ma Guangnian Wan Runpeng Yu Gongfan Fang Xinchao Wang LRM 119 37 0 13 Feb 2025
When More is Less: Understanding Chain-of-Thought Length in LLMs Yuyang Wu Yifei Wang Tianqi Du Stefanie Jegelka Yisen Wang Yisen Wang LRM 71 34 0 11 Feb 2025
BOLT: Bootstrap Long Chain-of-Thought in Language Models without Distillation Bo Pang Hanze Dong Jiacheng Xu Siyang Song Yingbo Zhou Caiming Xiong KELM LRM 94 8 0 06 Feb 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI Daya Guo Dejian Yang Haowei Zhang Junxiao Song ... Shiyu Wang S. Yu Shunfeng Zhou Shuting Pan S.S. Li ReLM VLM OffRL AI4TS LRM 128 1,368 0 22 Jan 2025
Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs Xingyu Chen Jiahao Xu Tian Liang Zhiwei He Jianhui Pang ... Zizhuo Zhang Rui Wang Zhaopeng Tu Haitao Mi Dong Yu LRM ReLM 110 150 0 30 Dec 2024
Understanding Chain-of-Thought in LLMs through Information Theory Jean-François Ton Muhammad Faaiz Taufiq Yang Liu LRM 73 9 0 18 Nov 2024
Learning How Hard to Think: Input-Adaptive Allocation of LM Computation Mehul Damani Idan Shenfeld Andi Peng Andreea Bobu Jacob Andreas 47 22 0 07 Oct 2024
To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning Zayne Sprague Fangcong Yin Juan Diego Rodriguez Dongwei Jiang Manya Wadhwa Prasann Singhal Xinyu Zhao Xi Ye Kyle Mahowald Greg Durrett ReLM LRM 147 101 0 18 Sep 2024
Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoning Xinglin Wang Shaoxiong Feng Yiwei Li Peiwen Yuan Y. Zhang Boyuan Pan Heda Wang Yao Hu Kan Li LRM 66 24 0 24 Aug 2024
Prover-Verifier Games improve legibility of LLM outputs Jan Hendrik Kirchner Yining Chen Harri Edwards Jan Leike Nat McAleese Yuri Burda LRM AAML 30 28 0 18 Jul 2024
DynaThink: Fast or Slow? A Dynamic Decision-Making Framework for Large Language Models Jiabao Pan Yan Zhang Chen Zhang Zuozhu Liu Hongwei Wang Haizhou Li LRM 37 6 0 01 Jul 2024
Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs Jannik Kossen Jiatong Han Muhammed Razzak Lisa Schut Shreshth A. Malik Yarin Gal HILM 79 42 0 22 Jun 2024
MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark Yubo Wang Xueguang Ma Ge Zhang Yuansheng Ni Abhranil Chandra ... Kai Wang Alex Zhuang Rongqi Fan Xiang Yue Wenhu Chen LRM ELM 66 376 0 03 Jun 2024
ReFT: Reasoning with Reinforced Fine-Tuning Trung Quoc Luong Xinbo Zhang Zhanming Jie Peng Sun Xiaoran Jin Hang Li OffRL LRM ReLM 48 108 0 17 Jan 2024
The Impact of Reasoning Step Length on Large Language Models Mingyu Jin Qinkai Yu Dong Shu Haiyan Zhao Wenyue Hua Yanda Meng Yongfeng Zhang Jundong Li ReLM LRM 72 96 0 10 Jan 2024
MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoning Zayne Sprague Xi Ye Kaj Bostrom Swarat Chaudhuri Greg Durrett LRM 18 68 0 24 Oct 2023
Efficient Memory Management for Large Language Model Serving with PagedAttention Woosuk Kwon Zhuohan Li Siyuan Zhuang Ying Sheng Lianmin Zheng Cody Hao Yu Joseph E. Gonzalez Haotong Zhang Ion Stoica VLM 84 2,049 0 12 Sep 2023
Reasoning with Language Model is Planning with World Model Shibo Hao Yi Gu Haodi Ma Joshua Jiahua Hong Zhen Wang D. Wang Zhiting Hu ReLM LRM LLMAG 78 539 0 24 May 2023
Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought Abulhair Saparov He He ELM LRM ReLM 138 297 0 03 Oct 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 484 9,009 0 28 Jan 2022
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 165 4,175 0 27 Oct 2021
Noisy Channel Language Model Prompting for Few-Shot Text Classification Sewon Min Michael Lewis Hannaneh Hajishirzi Luke Zettlemoyer VLM 47 219 0 09 Aug 2021
Contrastive Reinforcement Learning of Symbolic Reasoning Domains Gabriel Poesia Wenxin Dong Noah D. Goodman 68 19 0 16 Jun 2021
Measuring Massive Multitask Language Understanding Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika D. Song Jacob Steinhardt ELM RALM 105 4,222 0 07 Sep 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 261 41,106 0 28 May 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 315 4,662 0 23 Jan 2020
BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle Peter West Ari Holtzman Jan Buys Yejin Choi SSL 32 60 0 16 Sep 2019
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge Alon Talmor Jonathan Herzig Nicholas Lourie Jonathan Berant RALM 98 1,677 0 02 Nov 2018
Neural-Symbolic Learning and Reasoning: A Survey and Interpretation Tarek R. Besold Artur Garcez Sebastian Bader Howard L. Bowman Pedro M. Domingos ... P. Lima L. Penning Gadi Pinkas Hoifung Poon Gerson Zaverucha LRM AI4CE 37 335 0 10 Nov 2017
Information-theoretic analysis of generalization capability of learning algorithms Aolin Xu Maxim Raginsky 38 442 0 22 May 2017
Understanding ACT-R - an Outsider's Perspective Jacob Whitehill LRM 35 22 0 01 Jun 2013