uDistil-Whisper: Label-Free Data Filtering for Knowledge Distillation in Low-Data Regimes

v1v2v3v4v5 (latest)

uDistil-Whisper: Label-Free Data Filtering for Knowledge Distillation in Low-Data Regimes

1 July 2024

Muhammad Abdul-Mageed

ArXiv (abs)PDF HTML

Papers citing "uDistil-Whisper: Label-Free Data Filtering for Knowledge Distillation in Low-Data Regimes"

13 / 13 papers shown

Title
To Distill or Not to Distill? On the Robustness of Robust Knowledge Distillation Abdul Waheed Karima Kadaoui Muhammad Abdul-Mageed VLM 63 3 0 06 Jun 2024
Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling Sanchit Gandhi Patrick von Platen Alexander M. Rush VLM 65 62 0 01 Nov 2023
SONAR: Sentence-Level Multimodal and Language-Agnostic Representations Paul-Ambroise Duquenne Holger Schwenk Benoît Sagot AI4TS VLM 107 70 0 22 Aug 2023
Robust Speech Recognition via Large-Scale Weak Supervision Alec Radford Jong Wook Kim Tao Xu Greg Brockman C. McLeavey Ilya Sutskever OffRL 215 3,750 0 06 Dec 2022
FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech Alexis Conneau Min Ma Simran Khanuja Yu Zhang Vera Axelrod Siddharth Dalmia Jason Riesa Clara E. Rivera Ankur Bapna VLM 147 329 0 25 May 2022
XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale Arun Babu Changhan Wang Andros Tjandra Kushal Lakhotia Qiantong Xu ... Yatharth Saraf J. Pino Alexei Baevski Alexis Conneau Michael Auli SSL 110 708 0 17 Nov 2021
DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT Heng-Jui Chang Shu-Wen Yang Hung-yi Lee SSL 121 175 0 05 Oct 2021
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units Wei-Ning Hsu Benjamin Bolte Yao-Hung Hubert Tsai Kushal Lakhotia Ruslan Salakhutdinov Abdel-rahman Mohamed SSL 184 3,003 0 14 Jun 2021
ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic Muhammad Abdul-Mageed AbdelRahim Elmadany El Moatez Billah Nagoudi VLM 112 464 0 27 Dec 2020
Common Voice: A Massively-Multilingual Speech Corpus Rosana Ardila Megan Branson Kelly Davis Michael Henretty M. Kohler Josh Meyer Reuben Morais Lindsay Saunders Francis M. Tyers Gregor Weber VLM 93 1,620 0 13 Dec 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 255 7,554 0 02 Oct 2019
Data-Free Knowledge Distillation for Deep Neural Networks Raphael Gontijo-Lopes Stefano Fenu Thad Starner 62 273 0 19 Oct 2017
Speech Recognition Challenge in the Wild: Arabic MGB-3 Ahmed M. Ali S. Vogel Steve Renals 56 104 0 21 Sep 2017

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.