TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural
Language Processing

TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural Language Processing

28 February 2020

Papers citing "TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural Language Processing"

13 / 13 papers shown

Title
GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Model Shicheng Tan Weng Lam Tam Yuanchun Wang Wenwen Gong Yang Yang ... Jiahao Liu Jingang Wang Shuo Zhao Peng Zhang Jie Tang ALM MoE 33 11 0 11 Jun 2023
MiniRBT: A Two-stage Distilled Small Chinese Pre-trained Model Xin Yao Ziqing Yang Yiming Cui Shijin Wang 31 3 0 03 Apr 2023
In-context Learning Distillation: Transferring Few-shot Learning Ability of Pre-trained Language Models Yukun Huang Yanda Chen Zhou Yu Kathleen McKeown 27 30 0 20 Dec 2022
Progressive Learning without Forgetting Tao Feng Hangjie Yuan Mang Wang Ziyuan Huang Ang Bian Jianzhou Zhang CLL KELM 44 4 0 28 Nov 2022
ExPUNations: Augmenting Puns with Keywords and Explanations Jiao Sun Anjali Narayan-Chen Shereen Oraby Alessandra Cervone Tagyoung Chung Jing Huang Yang Liu Nanyun Peng 19 10 0 24 Oct 2022
Knowledge Distillation of Transformer-based Language Models Revisited Chengqiang Lu Jianwei Zhang Yunfei Chu Zhengyu Chen Jingren Zhou Fei Wu Haiqing Chen Hongxia Yang VLM 27 10 0 29 Jun 2022
Chemical transformer compression for accelerating both training and inference of molecular modeling Yi Yu K. Börjesson 27 0 0 16 May 2022
TextPruner: A Model Pruning Toolkit for Pre-Trained Language Models Ziqing Yang Yiming Cui Zhigang Chen SyDa VLM 31 12 0 30 Mar 2022
A Survey of Pretraining on Graphs: Taxonomy, Methods, and Applications Jun Xia Yanqiao Zhu Yuanqi Du Stan Z. Li VLM 30 41 0 16 Feb 2022
Information Theoretic Representation Distillation Roy Miles Adrian Lopez-Rodriguez K. Mikolajczyk MQ 13 21 0 01 Dec 2021
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 19 2,851 0 09 Jun 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 243 1,452 0 18 Mar 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 299 6,984 0 20 Apr 2018