Comparison of Soft and Hard Target RNN-T Distillation for Large-scale
ASR

Comparison of Soft and Hard Target RNN-T Distillation for Large-scale ASR

11 October 2022

Trevor Strohman

Papers citing "Comparison of Soft and Hard Target RNN-T Distillation for Large-scale ASR"

12 / 12 papers shown

Title
Replay-Based Continual Learning with Dual-Layered Distillation and a Streamlined U-Net for Efficient Text-to-Image Generation Md. Naimur Asif Borno Md Sakib Hossain Shovon Asmaa Soliman Al-Moisheer Mohammad Ali Moni 36 0 0 11 May 2025
Knowledge Distillation Using Frontier Open-source LLMs: Generalizability and the Role of Synthetic Data Anup Shirgaonkar Nikhil Pandey Nazmiye Ceren Abay Tolga Aktas Vijay Aski ALM SyDa 36 0 0 24 Oct 2024
Fine-Tuning Automatic Speech Recognition for People with Parkinson's: An Effective Strategy for Enhancing Speech Technology Accessibility Xiuwen Zheng Bornali Phukon Mark Hasegawa-Johnson 29 1 0 29 Sep 2024
DETECLAP: Enhancing Audio-Visual Representation Learning with Object Information Shota Nakada Taichi Nishimura Hokuto Munakata Masayoshi Kondo Tatsuya Komatsu CLIP VLM 36 0 0 18 Sep 2024
Optimizing Vision Transformers with Data-Free Knowledge Transfer Gousia Habib Damandeep Singh I. Malik Brejesh Lall 43 1 0 12 Aug 2024
Generation, Distillation and Evaluation of Motivational Interviewing-Style Reflections with a Foundational Language Model Andrew Brown Jiading Zhu Mohamed Abdelwahab Alec Dong Cindy Wang Jonathan Rose LLMAG 11 3 0 01 Feb 2024
Revisiting the Entropy Semiring for Neural Speech Recognition Oscar Chang DongSeon Hwang Olivier Siohan 32 2 0 13 Dec 2023
O-1: Self-training with Oracle and 1-best Hypothesis M. Baskar Andrew Rosenberg Bhuvana Ramabhadran Kartik Audhkhasi VLM 22 0 0 14 Aug 2023
Robust Knowledge Distillation from RNN-T Models With Noisy Training Labels Using Full-Sum Loss Mohammad Zeineldeen Kartik Audhkhasi M. Baskar Bhuvana Ramabhadran 24 2 0 10 Mar 2023
Sample-Efficient Unsupervised Domain Adaptation of Speech Recognition Systems A case study for Modern Greek Georgios Paraskevopoulos Theodoros Kouzelis Georgios Rouvalis Athanasios Katsamanis Vassilis Katsouros Alexandros Potamianos VLM 30 7 0 31 Dec 2022
Primer: Searching for Efficient Transformers for Language Modeling David R. So Wojciech Mañke Hanxiao Liu Zihang Dai Noam M. Shazeer Quoc V. Le VLM 91 153 0 17 Sep 2021
Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition Yu Zhang James Qin Daniel S. Park Wei Han Chung-Cheng Chiu Ruoming Pang Quoc V. Le Yonghui Wu VLM SSL 148 308 0 20 Oct 2020