EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models

27 May 2025

Papers citing "EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models"

14 / 14 papers shown

Title
DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models Chengyu Wang Junbing Yan Yuanhao Yue Jun Huang 40 1 0 21 Apr 2025
Training Small Reasoning LLMs with Cognitive Preference Alignment Wenrui Cai Chengyu Wang Junbing Yan Jun Huang Xiangzhong Fang LRM 55 2 0 14 Apr 2025
Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning Wenkai Yang Shuming Ma Yankai Lin Furu Wei LRM 86 43 0 25 Feb 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI Daya Guo Dejian Yang Haowei Zhang Junxiao Song ... Shiyu Wang S. Yu Shunfeng Zhou Shuting Pan S.S. Li ReLM VLM OffRL AI4TS LRM 359 1,641 0 22 Jan 2025
Building a Family of Data Augmentation Models for Low-cost LLM Fine-tuning on the Cloud Yuanhao Yue Chengyu Wang Jun Huang Peng Wang 76 2 0 06 Dec 2024
Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planning Yuanhao Yue Chengyu Wang Jun Huang Peng Wang ALM 41 8 0 22 May 2024
Best Practices and Lessons Learned on Synthetic Data for Language Models Ruibo Liu Jerry W. Wei Fangyu Liu Chenglei Si Yanzhe Zhang ... Steven Zheng Daiyi Peng Diyi Yang Denny Zhou Andrew M. Dai SyDa EgoV 104 95 0 11 Apr 2024
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models Zhihong Shao Peiyi Wang Qihao Zhu Runxin Xu Jun-Mei Song ... Haowei Zhang Mingchuan Zhang Yiming Li Yu-Huan Wu Daya Guo ReLM LRM 115 1,107 0 05 Feb 2024
ReFT: Reasoning with Reinforced Fine-Tuning Trung Quoc Luong Xinbo Zhang Zhanming Jie Peng Sun Xiaoran Jin Hang Li OffRL LRM ReLM 58 116 0 17 Jan 2024
Zephyr: Direct Distillation of LM Alignment Lewis Tunstall E. Beeching Nathan Lambert Nazneen Rajani Kashif Rasul ... Nathan Habib Nathan Sarrazin Omar Sanseviero Alexander M. Rush Thomas Wolf ALM 97 388 0 25 Oct 2023
Efficient Memory Management for Large Language Model Serving with PagedAttention Woosuk Kwon Zhuohan Li Siyuan Zhuang Ying Sheng Lianmin Zheng Cody Hao Yu Joseph E. Gonzalez Haotong Zhang Ion Stoica VLM 158 2,196 0 12 Sep 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 353 3,935 0 29 May 2023
Constitutional AI: Harmlessness from AI Feedback Yuntao Bai Saurav Kadavath Sandipan Kundu Amanda Askell John Kernion ... Dario Amodei Nicholas Joseph Sam McCandlish Tom B. Brown Jared Kaplan SyDa MoMe 182 1,614 0 15 Dec 2022
Proximal Policy Optimization Algorithms John Schulman Filip Wolski Prafulla Dhariwal Alec Radford Oleg Klimov OffRL 473 19,006 0 20 Jul 2017