To Distill or Not to Distill? On the Robustness of Robust Knowledge
Distillation

To Distill or Not to Distill? On the Robustness of Robust Knowledge Distillation

6 June 2024

Muhammad Abdul-Mageed

Papers citing "To Distill or Not to Distill? On the Robustness of Robust Knowledge Distillation"

10 / 10 papers shown

Title
Fast Inference from Transformers via Speculative Decoding Yaniv Leviathan Matan Kalman Yossi Matias LRM 54 663 0 30 Nov 2022
FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech Alexis Conneau Min Ma Simran Khanuja Yu Zhang Vera Axelrod Siddharth Dalmia Jason Riesa Clara E. Rivera Ankur Bapna VLM 95 306 0 25 May 2022
Self-supervised Learning with Random-projection Quantizer for Speech Recognition Chung-Cheng Chiu James Qin Yu Zhang Jiahui Yu Yonghui Wu SSL 47 167 0 03 Feb 2022
XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale Arun Babu Changhan Wang Andros Tjandra Kushal Lakhotia Qiantong Xu ... Yatharth Saraf J. Pino Alexei Baevski Alexis Conneau Michael Auli SSL 59 678 0 17 Nov 2021
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units Wei-Ning Hsu Benjamin Bolte Yao-Hung Hubert Tsai Kushal Lakhotia Ruslan Salakhutdinov Abdel-rahman Mohamed SSL 108 2,879 0 14 Jun 2021
CompRess: Self-Supervised Learning by Compressing Representations Soroush Abbasi Koohpayegani Ajinkya Tejankar Hamed Pirsiavash SSL 35 89 0 28 Oct 2020
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations Alexei Baevski Henry Zhou Abdel-rahman Mohamed Michael Auli SSL 40 5,677 0 20 Jun 2020
Quantifying the Carbon Emissions of Machine Learning Alexandre Lacoste A. Luccioni Victor Schmidt Thomas Dandres 54 688 0 21 Oct 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 56 7,386 0 02 Oct 2019
Speech Recognition Challenge in the Wild: Arabic MGB-3 Ahmed M. Ali S. Vogel Steve Renals 25 103 0 21 Sep 2017