Distilling BERT into Simple Neural Networks with Unlabeled Transfer Data

4 October 2019

Papers citing "Distilling BERT into Simple Neural Networks with Unlabeled Transfer Data"

13 / 13 papers shown

Title
Logits of API-Protected LLMs Leak Proprietary Information Matthew Finlayson Xiang Ren Swabha Swayamdipta PILM 29 23 0 14 Mar 2024
How to Fine-Tune Vision Models with SGD Ananya Kumar Ruoqi Shen Sébastien Bubeck Suriya Gunasekar VLM 14 29 0 17 Nov 2022
Surgical Fine-Tuning Improves Adaptation to Distribution Shifts Yoonho Lee Annie S. Chen Fahim Tajwar Ananya Kumar Huaxiu Yao Percy Liang Chelsea Finn OOD 61 198 0 20 Oct 2022
Sound Natural: Content Rephrasing in Dialog Systems Arash Einolghozati Anchit Gupta K. Diedrick S. Gupta 23 6 0 03 Nov 2020
MixKD: Towards Efficient Distillation of Large-scale Language Models Kevin J Liang Weituo Hao Dinghan Shen Yufan Zhou Weizhu Chen Changyou Chen Lawrence Carin 13 73 0 01 Nov 2020
Weight Squeezing: Reparameterization for Knowledge Transfer and Model Compression Artem Chumachenko Daniil Gavrilov Nikita Balagansky Pavel Kalaidin 13 0 0 14 Oct 2020
BERT-EMD: Many-to-Many Layer Mapping for BERT Compression with Earth Mover's Distance Jianquan Li Xiaokang Liu Honghong Zhao Ruifeng Xu Min Yang Yaohong Jin 17 54 0 13 Oct 2020
Adversarial Self-Supervised Data-Free Distillation for Text Classification Xinyin Ma Yongliang Shen Gongfan Fang Chen Chen Chenghao Jia Weiming Lu 30 24 0 10 Oct 2020
Deep Learning Meets Projective Clustering Alaa Maalouf Harry Lang Daniela Rus Dan Feldman 24 9 0 08 Oct 2020
Compressed Deep Networks: Goodbye SVD, Hello Robust Low-Rank Approximation M. Tukan Alaa Maalouf Matan Weksler Dan Feldman 23 9 0 11 Sep 2020
Extracurricular Learning: Knowledge Transfer Beyond Empirical Distribution Hadi Pouransari Mojan Javaheripi Vinay Sharma Oncel Tuzel 14 5 0 30 Jun 2020
Compressing Large-Scale Transformer-Based Models: A Case Study on BERT Prakhar Ganesh Yao Chen Xin Lou Mohammad Ali Khan Yifan Yang Hassan Sajjad Preslav Nakov Deming Chen Marianne Winslett AI4CE 21 197 0 27 Feb 2020
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,748 0 26 Sep 2016