SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought

30 May 2025

Papers citing "SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought"

40 / 40 papers shown

Title
Reasoning with Latent Thoughts: On the Power of Looped Transformers Nikunj Saunshi Nishanth Dikkala Zhiyuan Li Sanjiv Kumar Sashank J. Reddi OffRL LRM AI4CE 89 18 0 24 Feb 2025
Distillation Scaling Laws Dan Busbridge Amitis Shidani Floris Weers Jason Ramapuram Etai Littwin Russ Webb 63 7 0 12 Feb 2025
Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA Sangmin Bae Adam Fisch Hrayr Harutyunyan Ziwei Ji Seungyeon Kim Tal Schuster KELM 93 6 0 28 Oct 2024
On the Inductive Bias of Stacking Towards Improving Reasoning Nikunj Saunshi Stefani Karp Shankar Krishnan Sobhan Miryoosefi Sashank J. Reddi Sanjiv Kumar LRM AI4CE 66 5 0 27 Sep 2024
Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models Taiqiang Wu Chaofan Tao Jiahao Wang Zhe Zhao Ngai Wong ALM 64 15 0 03 Apr 2024
LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models Yaowei Zheng Richong Zhang Junhao Zhang Yanhan Ye Zheyan Luo Zhangchi Feng Yongqiang Ma 94 479 0 20 Mar 2024
Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking E. Zelikman Georges Harik Yijia Shao Varuna Jayasiri Nick Haber Noah D. Goodman LLMAG ReLM LRM 82 131 0 14 Mar 2024
Teaching Large Language Models to Reason with Reinforcement Learning Alex Havrilla Yuqing Du Sharath Chandra Raparthy Christoforos Nalmpantis Jane Dwivedi-Yu Maksym Zhuravinskyi Eric Hambro Sainbayar Sukhbaatar Roberta Raileanu ReLM LRM 54 82 0 07 Mar 2024
SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling Dahyun Kim Chanjun Park Sanghoon Kim Wonsung Lee Wonho Song ... Hyunbyung Park Gyoungjin Gim Mikyoung Cha Hwalsuk Lee Sunghun Kim ALM ELM 65 141 0 23 Dec 2023
Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations Peiyi Wang Lei Li Zhihong Shao R. X. Xu Damai Dai Yifei Li Deli Chen Y.Wu Zhifang Sui AIMat LRM ALM 66 316 0 14 Dec 2023
Implicit Chain of Thought Reasoning via Knowledge Distillation Yuntian Deng Kiran Prasad Roland Fernandez P. Smolensky Vishrav Chaudhary Stuart M. Shieber ReLM LRM 34 51 0 02 Nov 2023
Sparse Universal Transformer Shawn Tan Songlin Yang Zhenfang Chen Aaron Courville Chuang Gan MoE 53 13 0 11 Oct 2023
MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models L. Yu Weisen Jiang Han Shi Jincheng Yu Zhengying Liu Yu Zhang James T. Kwok Zheng Li Adrian Weller Weiyang Liu OSLM LRM 66 363 0 21 Sep 2023
MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning Xiang Yue Xingwei Qu Ge Zhang Yao Fu Wenhao Huang Huan Sun Yu-Chuan Su Wenhu Chen AIMat LRM 95 391 0 11 Sep 2023
Baby Llama: knowledge distillation from an ensemble of teachers trained on a small dataset with no performance penalty I. Timiryasov J. Tastet 47 50 0 03 Aug 2023
Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective Guhao Feng Bohang Zhang Yuntian Gu Haotian Ye Di He Liwei Wang LRM 75 236 0 24 May 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 189 603 0 06 Apr 2023
Active Prompting with Chain-of-Thought for Large Language Models Shizhe Diao Pengcheng Wang Yong Lin Tong Zhang ReLM KELM LLMAG LRM 61 123 0 23 Feb 2023
Looped Transformers as Programmable Computers Angeliki Giannou Shashank Rajput Jy-yong Sohn Kangwook Lee Jason D. Lee Dimitris Papailiopoulos 59 105 0 30 Jan 2023
Automatic Chain of Thought Prompting in Large Language Models Zhuosheng Zhang Aston Zhang Mu Li Alexander J. Smola ReLM LRM 110 603 0 07 Oct 2022
Decomposed Prompting: A Modular Approach for Solving Complex Tasks Tushar Khot H. Trivedi Matthew Finlayson Yao Fu Kyle Richardson Peter Clark Ashish Sabharwal ReLM LRM 86 437 0 05 Oct 2022
Less is More: Task-aware Layer-wise Distillation for Language Model Compression Chen Liang Simiao Zuo Qingru Zhang Pengcheng He Weizhu Chen Tuo Zhao VLM 59 70 0 04 Oct 2022
Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango Aman Madaan Amir Yazdanbakhsh LRM 173 119 0 16 Sep 2022
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models Denny Zhou Nathanael Scharli Le Hou Jason W. Wei Nathan Scales ... Dale Schuurmans Claire Cui Olivier Bousquet Quoc Le Ed H. Chi RALM LRM AI4CE 47 1,085 0 21 May 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 570 9,009 0 28 Jan 2022
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 186 4,175 0 27 Oct 2021
TruthfulQA: Measuring How Models Mimic Human Falsehoods Stephanie C. Lin Jacob Hilton Owain Evans HILM 87 1,825 0 08 Sep 2021
Program Synthesis with Large Language Models Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski ... Ellen Jiang Carrie J. Cai Michael Terry Quoc V. Le Charles Sutton ELM AIMat ReCod ALM 88 1,846 0 16 Aug 2021
Distilling Knowledge via Knowledge Review Pengguang Chen Shu Liu Hengshuang Zhao Jiaya Jia 177 429 0 19 Apr 2021
Measuring Massive Multitask Language Understanding Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika D. Song Jacob Steinhardt ELM RALM 135 4,222 0 07 Sep 2020
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 48 2,907 0 09 Jun 2020
On the Efficacy of Knowledge Distillation Ligang He Rui Mao 77 603 0 03 Oct 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 121 7,437 0 02 Oct 2019
Improved Knowledge Distillation via Teacher Assistant Seyed Iman Mirzadeh Mehrdad Farajtabar Ang Li Nir Levine Akihiro Matsukawa H. Ghasemzadeh 79 1,073 0 09 Feb 2019
Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering Todor Mihaylov Peter Clark Tushar Khot Ashish Sabharwal 69 1,475 0 08 Sep 2018
CoQA: A Conversational Question Answering Challenge Siva Reddy Danqi Chen Christopher D. Manning RALM HAI 82 1,197 0 21 Aug 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 644 7,080 0 20 Apr 2018
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge Peter Clark Isaac Cowhey Oren Etzioni Tushar Khot Ashish Sabharwal Carissa Schoenick Oyvind Tafjord ELM RALM LRM 69 2,474 0 14 Mar 2018
Sequence-Level Knowledge Distillation Yoon Kim Alexander M. Rush 84 1,109 0 25 Jun 2016
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 236 19,523 0 09 Mar 2015