Teaching Small Language Models to Reason

16 December 2022

Lucie Charlotte Magister

Papers citing "Teaching Small Language Models to Reason"

50 / 191 papers shown

Title
Recall with Reasoning: Chain-of-Thought Distillation for Mamba's Long-Context Memory and Extrapolation Junyu Ma Tianqing Fang Z. Zhang Hongming Zhang Haitao Mi Dong Yu ReLM RALM LRM 137 0 0 06 May 2025
KETCHUP: K-Step Return Estimation for Sequential Knowledge Distillation Jiabin Fan Guoqing Luo Michael Bowling Lili Mou OffRL 63 0 0 26 Apr 2025
ConTextual: Improving Clinical Text Summarization in LLMs with Context-preserving Token Filtering and Knowledge Graphs Fahmida Liza Piya Rahmatollah Beheshti 131 0 0 23 Apr 2025
Honey, I Shrunk the Language Model: Impact of Knowledge Distillation Methods on Performance and Explainability Daniel Hendriks Philipp Spitzer Niklas Kühl G. Satzger 27 1 0 22 Apr 2025
Efficient Reasoning Models: A Survey Sicheng Feng Gongfan Fang Xinyin Ma Xinchao Wang ReLM LRM 145 0 0 15 Apr 2025
A Short Survey on Small Reasoning Models: Training, Inference, Applications and Research Directions Chengyu Wang Taolin Zhang Richang Hong Jun Huang ReLM LRM 42 1 0 12 Apr 2025
UNDO: Understanding Distillation as Optimization Kushal Kumar Jain Piyushi Goyal Kumar Shridhar 36 0 0 03 Apr 2025
OpenCodeReasoning: Advancing Data Distillation for Competitive Coding Wasi Uddin Ahmad Sean Narenthiran Somshubra Majumdar Aleksander Ficek Siddhartha Jain Jocelyn Huang Vahid Noroozi Boris Ginsburg LRM 50 3 0 02 Apr 2025
A Survey of Scaling in Large Language Model Reasoning Zihan Chen Song Wang Zhen Tan Xingbo Fu Zhenyu Lei Peng Wang Huan Liu Cong Shen Jundong Li LRM 88 0 0 02 Apr 2025
FeRG-LLM : Feature Engineering by Reason Generation Large Language Models Jeonghyun Ko Gyeongyun Park Donghoon Lee Kyunam Lee LRM 52 0 0 30 Mar 2025
LEMMA: Learning from Errors for MatheMatical Advancement in LLMs Zhuoshi Pan Yu-Hu Li Honglin Lin Qizhi Pei Zinan Tang Wei Yu Wu Chenlin Ming H. V. Zhao Conghui He Lijun Wu LRM 59 0 0 21 Mar 2025
VITED: Video Temporal Evidence Distillation Yujie Lu Yale Song William Yang Wang Lorenzo Torresani Tushar Nagarajan 156 0 0 17 Mar 2025
Rule-Guided Feedback: Enhancing Reasoning by Enforcing Rule Adherence in Large Language Models Aissatou Diallo Antonis Bikakis Luke Dickens Anthony Hunter Rob Miller LRM 43 0 0 14 Mar 2025
Efficient Algorithms for Verifying Kruskal Rank in Sparse Linear Regression and Related Applications Fengqin Zhou 55 0 0 06 Mar 2025
Scientific Reasoning: Assessment of Multimodal Generative LLMs Florian Dreyer Ekaterina Kolos Daria Matiash ReLM LRM 65 0 0 03 Mar 2025
Distill Not Only Data but Also Rewards: Can Smaller Language Models Surpass Larger Ones? Yudi Zhang Lu Wang Meng Fang Yali Du Chenghua Huang ... Qingwei Lin Mykola Pechenizkiy Dongmei Zhang Saravan Rajmohan Qi Zhang ALM 78 0 0 26 Feb 2025
Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning Xinghao Chen Zhijing Sun Wenjin Guo Miaoran Zhang Yanjun Chen ... Hui Su Yijie Pan Dietrich Klakow Wenjie Li Xiaoyu Shen LRM 56 5 0 25 Feb 2025
MemeIntel: Explainable Detection of Propagandistic and Hateful Memes Mohamed Bayan Kmainasi A. Hasnat Md. Arid Hasan Ali Ezzat Shahroor Firoj Alam VLM 45 0 0 23 Feb 2025
Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignment Zhili Liu Yunhao Gou Kai Chen Lanqing Hong Jiahui Gao ... Yu Zhang Zhenguo Li Xin Jiang Q. Liu James T. Kwok MoE 96 9 0 20 Feb 2025
Towards Reasoning Ability of Small Language Models Gaurav Srivastava Shuxiang Cao Xuan Wang ReLM LRM 54 4 0 17 Feb 2025
Rationalization Models for Text-to-SQL Gaetano Rossiello Nhan Pham Michael R. Glass Junkyu Lee Shankar Subramanian ReLM LRM 50 0 0 10 Feb 2025
Who Taught You That? Tracing Teachers in Model Distillation Somin Wadhwa Chantal Shaib Silvio Amir Byron C. Wallace 72 1 0 10 Feb 2025
Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents Weiwei Sun Lingyong Yan Xinyu Ma Shuaiqiang Wang Pengjie Ren Zhumin Chen Dawei Yin Z. Ren RALM ALM ELM LRM LM&MA 76 285 0 31 Dec 2024
Towards Efficient and Explainable Hate Speech Detection via Model Distillation Paloma Piot Javier Parapar 83 173 0 18 Dec 2024
Enhancing the Reasoning Capabilities of Small Language Models via Solution Guidance Fine-Tuning Jing Bi Yuting Wu Weiwei Xing Zhenjie Wei ReLM OffRL LRM 86 4 0 13 Dec 2024
Learning to Reason via Self-Iterative Process Feedback for Small Language Models Kaiyuan Chen Jin Wang Xuejie Zhang LRM ReLM 85 2 0 11 Dec 2024
Neuro-Symbolic Data Generation for Math Reasoning Zenan Li Zhi-Hua Zhou Yuan Yao Yu Li Chun Cao Fan Yang Xian Zhang Xiaoxing Ma OffRL LRM 76 7 0 06 Dec 2024
Dynamic Self-Distillation via Previous Mini-batches for Fine-tuning Small Language Models Y. Fu Yin Yu Xiaotian Han Runchao Li Xianxuan Long Haotian Yu Pan Li SyDa 62 0 0 25 Nov 2024
What Should Baby Models Read? Exploring Sample-Efficient Data Composition on Model Performance Hong Meng Yam Nathan J Paek 41 1 0 11 Nov 2024
Clustering Algorithms and RAG Enhancing Semi-Supervised Text Classification with Large LLMs Shan Zhong Jiahao Zeng Yongxin Yu Bohong Lin 34 1 0 09 Nov 2024
LLMs for Domain Generation Algorithm Detection Reynier Leyva La O Carlos A. Catania Tatiana Parlanti 25 0 0 05 Nov 2024
SIKeD: Self-guided Iterative Knowledge Distillation for mathematical reasoning Shivam Adarsh Kumar Shridhar Caglar Gulcehre Nicholas Monath Mrinmaya Sachan LRM 29 2 0 24 Oct 2024
Optimizing Chain-of-Thought Reasoning: Tackling Arranging Bottleneck via Plan Augmentation Yuli Qiu Jiashu Yao Heyan Huang Yuhang Guo LRM 29 0 0 22 Oct 2024
TextMaster: Universal Controllable Text Edit Aoqiang Wang Yufei Guo Zhenyu Yan Wenxiang Shang Ran Lin Zhao Zhang DiffM 28 2 0 13 Oct 2024
Mentor-KD: Making Small Language Models Better Multi-step Reasoners Hojae Lee Junho Kim SangKeun Lee LRM 32 1 0 11 Oct 2024
LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints Thomas Palmeira Ferraz Kartik Mehta Yu-Hsiang Lin Haw-Shiuan Chang Shereen Oraby Sijia Liu Vivek Subramanian Tagyoung Chung Mohit Bansal Nanyun Peng 53 7 0 09 Oct 2024
Chain-of-Thoughts for Molecular Understanding Yunhui Jang Jaehyung Kim Sungsoo Ahn LRM AI4CE 33 1 0 08 Oct 2024
DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedback Zaid Khan Elias Stengel-Eskin Jaemin Cho Joey Tianyi Zhou VGen 43 1 0 08 Oct 2024
Towards an Improved Metric for Evaluating Disentangled Representations Sahib Julka Yashu Wang Michael Granitzer 29 0 0 04 Oct 2024
Make Compound Sentences Simple to Analyze: Learning to Split Sentences for Aspect-based Sentiment Analysis Yongsik Seo Sungwon Song Ryang Heo Jieyong Kim Dongha Lee CoGe 21 0 0 03 Oct 2024
Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compression Jingcun Wang Yu-Guang Chen Ing-Chao Lin Bing Li Grace Li Zhang 35 4 0 02 Oct 2024
Mixed Chain-of-Psychotherapies for Emotional Support Chatbot Siyuan Chen Cong Ming Zhiling Zhang Yanyi Chen Kenny Q. Zhu Mengyue Wu AI4MH 29 0 0 29 Sep 2024
Small Language Models: Survey, Measurements, and Insights Zhenyan Lu Xiang Li Dongqi Cai Rongjie Yi Fangming Liu Xiwen Zhang Nicholas D. Lane Mengwei Xu ObjD LRM 55 36 0 24 Sep 2024
$$\textit{SKIntern}$: Internalizing Symbolic Knowledge for Distilling Better CoT Capabilities into Small Language Models$ $\textit{SKIntern}$ : Internalizing Symbolic Knowledge for Distilling Better CoT Capabilities into Small Language Models Huanxuan Liao Shizhu He Yupu Hao Xiang Li Yuanzhe Zhang Kang Liu Jun Zhao LRM 41 0 0 20 Sep 2024
Neural-Symbolic Collaborative Distillation: Advancing Small Language Models for Complex Reasoning Tasks Huanxuan Liao Shizhu He Yao Xu Yuanzhe Zhang Kang Liu Jun Zhao LRM 53 3 0 20 Sep 2024
JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images Zhecan Wang Junzhang Liu Chia-Wei Tang Hani Alomari Anushka Sivakumar ... Haoxuan You A. Ishmam Kai-Wei Chang Shih-Fu Chang Chris Thomas CoGe VLM 61 2 0 19 Sep 2024
Efficient Knowledge Distillation: Empowering Small Language Models with Teacher Model Insights Mohamad Ballout U. Krumnack Gunther Heidemann Kai-Uwe Kühnberger 35 2 0 19 Sep 2024
Hackphyr: A Local Fine-Tuned LLM Agent for Network Security Environments M. Rigaki C. Catania Sebastian Garcia LLMAG 34 3 0 17 Sep 2024
What is the Role of Small Models in the LLM Era: A Survey Lihu Chen Gaël Varoquaux ALM 63 23 0 10 Sep 2024
Chatting Up Attachment: Using LLMs to Predict Adult Bonds Paulo Soares Sean McCurdy Andrew J. Gerber Peter Fonagy 25 1 0 31 Aug 2024