Specializing Smaller Language Models towards Multi-Step Reasoning

30 January 2023

Papers citing "Specializing Smaller Language Models towards Multi-Step Reasoning"

42 / 42 papers shown

Title
Who Taught You That? Tracing Teachers in Model Distillation Somin Wadhwa Chantal Shaib Silvio Amir Byron C. Wallace 74 1 0 21 May 2025
SmallPlan: Leverage Small Language Models for Sequential Path Planning with Simulation-Powered, LLM-Guided Distillation Quang P.M. Pham Khoi T.N. Nguyen Nhi H. Doan Cuong Pham Kentaro Inui Dezhen Song 65 0 0 01 May 2025
Honey, I Shrunk the Language Model: Impact of Knowledge Distillation Methods on Performance and Explainability Daniel Hendriks Philipp Spitzer Niklas Kühl G. Satzger 27 1 0 22 Apr 2025
Cross-Tokenizer Distillation via Approximate Likelihood Matching Benjamin Minixhofer Ivan Vulić E. Ponti 151 0 0 25 Mar 2025
Efficient Algorithms for Verifying Kruskal Rank in Sparse Linear Regression and Related Applications Fengqin Zhou 58 3 0 06 Mar 2025
Efficient Jailbreaking of Large Models by Freeze Training: Lower Layers Exhibit Greater Sensitivity to Harmful Content Hongyuan Shen Min Zheng Jincheng Wang Yang Zhao 44 0 0 28 Feb 2025
Towards Reasoning Ability of Small Language Models Gaurav Srivastava Shuxiang Cao Xuan Wang ReLM LRM 60 4 0 17 Feb 2025
Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents Weiwei Sun Lingyong Yan Xinyu Ma Shuaiqiang Wang Pengjie Ren Zhumin Chen Dawei Yin Z. Ren RALM ALM ELM LRM LM&MA 76 285 0 31 Dec 2024
In-Context Learning with Iterative Demonstration Selection Chengwei Qin Aston Zhang Cheng Chen Anirudh Dagar Wenming Ye LRM 68 38 0 31 Dec 2024
Boosting LLM Translation Skills without General Ability Loss via Rationale Distillation Junhong Wu Yang Zhao Yangyifan Xu Bing Liu Chengqing Zong CLL 40 1 0 17 Oct 2024
DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedback Zaid Khan Elias Stengel-Eskin Jaemin Cho Joey Tianyi Zhou VGen 43 1 0 08 Oct 2024
Neural-Symbolic Collaborative Distillation: Advancing Small Language Models for Complex Reasoning Tasks Huanxuan Liao Shizhu He Yao Xu Yuanzhe Zhang Kang Liu Jun Zhao LRM 53 3 0 20 Sep 2024
Efficient Knowledge Distillation: Empowering Small Language Models with Teacher Model Insights Mohamad Ballout U. Krumnack Gunther Heidemann Kai-Uwe Kühnberger 35 2 0 19 Sep 2024
What is the Role of Small Models in the LLM Era: A Survey Lihu Chen Gaël Varoquaux ALM 63 23 0 10 Sep 2024
Path-Consistency: Prefix Enhancement for Efficient Inference in LLM Jiace Zhu Yingtao Shen Jie Zhao An Zou LLMAG LRM 27 4 0 25 Aug 2024
Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoning Xinglin Wang Shaoxiong Feng Yiwei Li Peiwen Yuan Y. Zhang Boyuan Pan Heda Wang Yao Hu Kan Li LRM 40 17 0 24 Aug 2024
Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systems Tamer Abuelsaad Deepak Akkil Prasenjit Dey Ashish Jagmohan Aditya Vempaty Ravi Kokku 46 23 0 17 Jul 2024
Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models Jinliang Lu Ziliang Pang Min Xiao Yaochen Zhu Rui Xia Jiajun Zhang MoMe 49 18 0 08 Jul 2024
Learn Beyond The Answer: Training Language Models with Reflection for Mathematical Reasoning Zhihan Zhang Zhenwen Liang Wenhao Yu Dian Yu Mengzhao Jia Dong Yu Meng Jiang AIMat RALM LRM ReLM 30 12 0 17 Jun 2024
Improve Student's Reasoning Generalizability through Cascading Decomposed CoTs Distillation Chengwei Dai Kun Li Wei Zhou Song Hu LRM 49 3 0 30 May 2024
Enhancing Reinforcement Learning with Label-Sensitive Reward for Natural Language Understanding Kuo Liao Shuang Li Meng Zhao Liqun Liu Mengge Xue Zhenyu Hu Honglin Han Chengguo Yin 40 1 0 30 May 2024
Beyond Imitation: Learning Key Reasoning Steps from Dual Chain-of-Thoughts in Reasoning Distillation Chengwei Dai Kun Li Wei Zhou Song Hu LRM 43 5 0 30 May 2024
Distilling Reasoning Ability from Large Language Models with Adaptive Thinking Xiao Chen Sihang Zhou K. Liang Xinwang Liu ReLM LRM 37 2 0 14 Apr 2024
The Unreasonable Ineffectiveness of the Deeper Layers Andrey Gromov Kushal Tirumala Hassan Shapourian Paolo Glorioso Daniel A. Roberts 52 79 0 26 Mar 2024
Meta-Cognitive Analysis: Evaluating Declarative and Procedural Knowledge in Datasets and Large Language Models Zhuoqun Li Hongyu Lin Yaojie Lu Hao Xiang Xianpei Han Le Sun 38 1 0 14 Mar 2024
UrbanKGent: A Unified Large Language Model Agent Framework for Urban Knowledge Graph Construction Yansong NING Hao Liu LLMAG 29 2 0 10 Feb 2024
Knowledge Fusion of Large Language Models Fanqi Wan Xinting Huang Deng Cai Xiaojun Quan Wei Bi Shuming Shi MoMe 37 61 0 19 Jan 2024
Mixed Distillation Helps Smaller Language Model Better Reasoning Chenglin Li Qianglong Chen Liangyue Li Wang Caiyu Yicheng Li Zhang Yin Yin Zhang LRM 41 11 0 17 Dec 2023
ADaPT: As-Needed Decomposition and Planning with Language Models Archiki Prasad Alexander Koller Mareike Hartmann Peter Clark Ashish Sabharwal Mohit Bansal Tushar Khot LM&Ro 29 76 0 08 Nov 2023
DialCoT Meets PPO: Decomposing and Exploring Reasoning Paths in Smaller Language Models Chengcheng Han Xiaowei Du Che Zhang Yixin Lian Xiang Li Ming Gao Baoyuan Wang LRM 37 14 0 08 Oct 2023
Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning Jiasheng Ye Zaixiang Zheng Yu Bao Lihua Qian Quanquan Gu DiffM 54 14 0 23 Aug 2023
Enhancing Small Medical Learners with Privacy-preserving Contextual Prompting Xinlu Zhang Shiyang Li Xianjun Yang Chenxin Tian Yao Qin Linda R. Petzold 24 9 0 22 May 2023
LeTI: Learning to Generate from Textual Interactions Xingyao Wang Hao Peng Reyhaneh Jabbarvand Heng Ji 35 30 0 17 May 2023
MoT: Memory-of-Thought Enables ChatGPT to Self-Improve Xiaonan Li Xipeng Qiu ReLM KELM LRM AI4MH 26 32 0 09 May 2023
A Systematic Study of Knowledge Distillation for Natural Language Generation with Pseudo-Target Training Nitay Calderon Subhabrata Mukherjee Roi Reichart Amir Kantor 33 17 0 03 May 2023
From Zero to Hero: Examining the Power of Symbolic Tasks in Instruction Tuning Qian Liu Fan Zhou Zhengbao Jiang Longxu Dou Min-Bin Lin 18 17 0 17 Apr 2023
Explicit Planning Helps Language Models in Logical Reasoning Hongyu Zhao Kangrui Wang Mo Yu Hongyuan Mei LRM ReLM 40 15 0 28 Mar 2023
Reasoning with Language Model Prompting: A Survey Shuofei Qiao Yixin Ou Ningyu Zhang Xiang Chen Yunzhi Yao Shumin Deng Chuanqi Tan Fei Huang Huajun Chen ReLM ELM LRM 71 311 0 19 Dec 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 314 3,248 0 21 Mar 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 319 11,953 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 379 8,495 0 28 Jan 2022
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 258 4,489 0 23 Jan 2020