Reinforcement Learning vs. Distillation: Understanding Accuracy and Capability in LLM Reasoning

Reinforcement Learning vs. Distillation: Understanding Accuracy and Capability in LLM Reasoning

20 May 2025

Anubhav Shrestha

Author Contacts:

keithwross@nyu.edu

ArXiv (abs)PDF HTML

Papers citing "Reinforcement Learning vs. Distillation: Understanding Accuracy and Capability in LLM Reasoning"

14 / 14 papers shown

Title
Phi-4-Mini-Reasoning: Exploring the Limits of Small Reasoning Language Models in Math Haoran Xu Baolin Peng Hany Awadalla DongDong Chen Yen-Chun Chen ... Yelong Shen Shuaiqiang Wang Weijian Xu Jianfeng Gao Weizhu Chen ReLM LRM 135 5 0 30 Apr 2025
Reinforcement Learning for Reasoning in Large Language Models with One Training Example Yiping Wang Qing Yang Zhiyuan Zeng Liliang Ren Liu Liu ... Jianfeng Gao Weizhu Chen Shuaiqiang Wang Simon Shaolei Du Yelong Shen OffRL ReLM LRM 278 34 0 29 Apr 2025
Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? Yang Yue Zhiqi Chen Rui Lu Andrew Zhao Zhaokai Wang Yang Yue Shiji Song Gao Huang ReLM LRM 184 93 0 18 Apr 2025
Climbing the Ladder of Reasoning: What LLMs Can-and Still Can't-Solve after SFT? Yiyou Sun Georgia Zhou Haoran Wang Dexun Li Nouha Dziri Dawn Song ReLM ALM ELM LRM 116 5 1 16 Apr 2025
A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility Andreas Hochlehnert Hardik Bhatnagar Vishaal Udandarao Samuel Albanie Ameya Prabhu Matthias Bethge ReLM ALM LRM 191 20 0 09 Apr 2025
Understanding R1-Zero-Like Training: A Critical Perspective Zichen Liu Changyu Chen Wenjun Li Penghui Qi Tianyu Pang Chao Du Wee Sun Lee Min Lin OffRL LRM 190 141 0 26 Mar 2025
SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild Weihao Zeng Yuzhen Huang Qian Liu Wei Liu Keqing He Zejun Ma Junxian He OffRL ReLM LRM 160 109 0 24 Mar 2025
Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs Kanishk Gandhi Ayush Chakravarthy Anikait Singh Nathan Lile Noah D. Goodman ReLM LRM 149 84 0 03 Mar 2025
Demystifying Long Chain-of-Thought Reasoning in LLMs Edward Yeo Yuxuan Tong Morry Niu Graham Neubig Xiang Yue OffRL LRM 165 121 0 05 Feb 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI Daya Guo Dejian Yang Haowei Zhang Junxiao Song ... Shiyu Wang S. Yu Shunfeng Zhou Shuting Pan S.S. Li ReLM VLM OffRL AI4TS LRM 370 1,692 0 22 Jan 2025
Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Though Violet Xiang Charlie Snell Kanishk Gandhi Alon Albalak Anikait Singh ... Dakota Mahan Louis Castricato Jan-Philipp Fränken Nick Haber Chelsea Finn LRM 108 51 0 08 Jan 2025
O1 Replication Journey: A Strategic Progress Report -- Part 1 Yiwei Qin Xuefeng Li Haoyang Zou Yixiu Liu Shijie Xia ... Yixin Ye Weizhe Yuan Hector Liu Yuezun Li Pengfei Liu VLM 87 88 0 08 Oct 2024
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models Zhihong Shao Peiyi Wang Qihao Zhu Runxin Xu Jun-Mei Song ... Haowei Zhang Mingchuan Zhang Yiming Li Yu-Huan Wu Daya Guo ReLM LRM 138 1,119 0 05 Feb 2024
Reinforced Self-Training (ReST) for Language Modeling Çağlar Gülçehre T. Paine S. Srinivasan Ksenia Konyushkova L. Weerts ... Chenjie Gu Wolfgang Macherey Arnaud Doucet Orhan Firat Nando de Freitas OffRL 119 305 0 17 Aug 2023

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.