v1v2 (latest)

Distilling Reasoning Capabilities into Smaller Language Models

1 December 2022

ArXiv (abs)PDF HTML Github (7★)

Papers citing "Distilling Reasoning Capabilities into Smaller Language Models"

32 / 32 papers shown

Title
R3: Robust Rubric-Agnostic Reward Models David Anugraha Zilu Tang Lester James V. Miranda Hanyang Zhao Mohammad Rifqi Farhansyah Garry Kuwanto Derry Wijaya Genta Indra Winata 190 1 0 19 May 2025
Critique-Guided Distillation: Improving Supervised Fine-tuning via Better Distillation Berkcan Kapusuzoglu Supriyo Chakraborty Chia-Hsuan Lee Sambit Sahu 116 0 0 16 May 2025
Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning Xinghao Chen Zhijing Sun Wenjin Guo Miaoran Zhang Yanjun Chen ... Hui Su Yijie Pan Dietrich Klakow Wenjie Li Xiaoyu Shen LRM 166 8 0 25 Feb 2025
Pub-Guard-LLM: Detecting Fraudulent Biomedical Articles with Reliable Explanations Lihu Chen Shuojie Fu Gabriel Freedman Cemre Zor Guy Martin James Kinross Uddhav Vaghela Ovidiu Serban Francesca Toni DeLMO 134 0 0 21 Feb 2025
Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking Junda Zhu Lingyong Yan Shuaiqiang Wang Dawei Yin Lei Sha AAML LRM 96 6 0 18 Feb 2025
Who Taught You That? Tracing Teachers in Model Distillation Somin Wadhwa Chantal Shaib Silvio Amir Byron C. Wallace 246 2 0 10 Feb 2025
DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedback Zaid Khan Elias Stengel-Eskin Jaemin Cho Joey Tianyi Zhou VGen 147 3 0 08 Oct 2024
Automated Knowledge Concept Annotation and Question Representation Learning for Knowledge Tracing Yilmazcan Ozyurt Stefan Feuerriegel Mrinmaya Sachan AI4Ed 117 2 0 02 Oct 2024
What is the Role of Small Models in the LLM Era: A Survey Lihu Chen Gaël Varoquaux ALM 219 32 0 10 Sep 2024
LUK: Empowering Log Understanding with Expert Knowledge from Large Language Models Lipeng Ma Weidong Yang Sihang Jiang Ben Fei Mingjie Zhou Shuhao Li Bo Xu Bo Xu Yanghua Xiao 141 0 0 03 Sep 2024
Socratic Reasoning Improves Positive Text Rewriting Anmol Goel Nico Daheim Iryna Gurevych Iryna Gurevych LRM 117 4 0 05 Mar 2024
World Models for Math Story Problems Andreas Opedal Niklas Stoehr Abulhair Saparov Mrinmaya Sachan ReLM 107 13 0 07 Jun 2023
OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization Srinivasan Iyer Xi Lin Ramakanth Pasunuru Todor Mihaylov Daniel Simig ... Jeff Wang Christopher Dewan Asli Celikyilmaz Luke Zettlemoyer Veselin Stoyanov ALM 161 268 0 22 Dec 2022
Automatic Generation of Socratic Subquestions for Teaching Math Word Problems Kumar Shridhar Jakub Macina Mennatallah El-Assady Tanmay Sinha Manu Kapur Mrinmaya Sachan AIMat 87 48 0 23 Nov 2022
A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models Alessandro Stolfo Zhijing Jin Kumar Shridhar Bernhard Schölkopf Mrinmaya Sachan ELM OOD LRM 121 66 0 21 Oct 2022
Large Language Models Can Self-Improve Jiaxin Huang S. Gu Le Hou Yuexin Wu Xuezhi Wang Hongkun Yu Jiawei Han ReLM AI4MH LRM 209 616 0 20 Oct 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 248 3,165 0 20 Oct 2022
Learning by Distilling Context Charles Burton Snell Dan Klein Ruiqi Zhong ReLM LRM 233 48 0 30 Sep 2022
Solving Quantitative Reasoning Problems with Language Models Aitor Lewkowycz Anders Andreassen David Dohan Ethan Dyer Henryk Michalewski ... Theo Gutman-Solo Yuhuai Wu Behnam Neyshabur Guy Gur-Ari Vedant Misra ReLM ELM LRM 185 860 0 29 Jun 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Percy Liang J. Dean W. Fedus ELM ReLM LRM 295 2,521 0 15 Jun 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 540 6,304 0 05 Apr 2022
Training Compute-Optimal Large Language Models Jordan Hoffmann Sebastian Borgeaud A. Mensch Elena Buchatskaya Trevor Cai ... Karen Simonyan Erich Elsen Jack W. Rae Oriol Vinyals Laurent Sifre AI4TS 211 1,988 0 29 Mar 2022
Show Your Work: Scratchpads for Intermediate Computation with Language Models Maxwell Nye Anders Andreassen Guy Gur-Ari Henryk Michalewski Jacob Austin ... Aitor Lewkowycz Maarten Bosma D. Luan Charles Sutton Augustus Odena ReLM LRM 186 756 0 30 Nov 2021
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 377 4,598 0 27 Oct 2021
A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers Shen-Yun Miao Chao-Chun Liang Keh-Yih Su 76 343 0 30 Jun 2021
Unsupervised Commonsense Question Answering with Self-Talk Vered Shwartz Peter West Ronan Le Bras Chandra Bhagavatula Yejin Choi ReLM SSL AI4MH LRM 70 263 0 11 Apr 2020
Learning to Answer by Learning to Ask: Getting the Best of GPT-2 and BERT Worlds T. Klein Moin Nabi 76 63 0 06 Nov 2019
Explain Yourself! Leveraging Language Models for Commonsense Reasoning Nazneen Rajani Bryan McCann Caiming Xiong R. Socher ReLM LRM 102 566 0 06 Jun 2019
BERTScore: Evaluating Text Generation with BERT Tianyi Zhang Varsha Kishore Felix Wu Kilian Q. Weinberger Yoav Artzi 397 5,878 0 21 Apr 2019
A Call for Clarity in Reporting BLEU Scores Matt Post 187 3,000 0 23 Apr 2018
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation Kyunghyun Cho B. V. Merrienboer Çağlar Gülçehre Dzmitry Bahdanau Fethi Bougares Holger Schwenk Yoshua Bengio AIMat 1.1K 23,414 0 03 Jun 2014
Do Deep Nets Really Need to be Deep? Lei Jimmy Ba R. Caruana 188 2,120 0 21 Dec 2013