Efficient Knowledge Distillation for RNN-Transducer Models

Efficient Knowledge Distillation for RNN-Transducer Models

11 November 2020

S. Panchapagesan

Chung-Cheng Chiu

Papers citing "Efficient Knowledge Distillation for RNN-Transducer Models"

13 / 13 papers shown

Title
Knowledge Distillation from Non-streaming to Streaming ASR Encoder using Auxiliary Non-streaming Layer Kyuhong Shim Jinkyu Lee Simyoung Chang Kyuwoong Hwang 40 2 0 31 Aug 2023
Reducing the gap between streaming and non-streaming Transducer-based ASR by adaptive two-stage knowledge distillation Haitao Tang Yu Fu Lei Sun Jiabin Xue Dan Liu ... Zhiqiang Ma Minghui Wu Jia Pan Genshun Wan Ming’En Zhao 21 2 0 27 Jun 2023
Accelerator-Aware Training for Transducer-Based Speech Recognition Suhaila M. Shakiah R. Swaminathan Hieu Duy Nguyen Raviteja Chinta Tariq Afzal Nathan Susanj Athanasios Mouchtaris Grant P. Strimel Ariya Rastrow 19 1 0 12 May 2023
Practical Conformer: Optimizing size, speed and flops of Conformer for on-Device and cloud ASR Rami Botros Anmol Gulati Tara N. Sainath K. Choromanski Ruoming Pang Trevor Strohman Weiran Wang Jiahui Yu MQ 26 3 0 31 Mar 2023
Robust Knowledge Distillation from RNN-T Models With Noisy Training Labels Using Full-Sum Loss Mohammad Zeineldeen Kartik Audhkhasi M. Baskar Bhuvana Ramabhadran 24 2 0 10 Mar 2023
Practical Knowledge Distillation: Using DNNs to Beat DNNs Chungman Lee Pavlos Anastasios Apostolopulos Igor L. Markov FedML 24 1 0 23 Feb 2023
Sample-Efficient Unsupervised Domain Adaptation of Speech Recognition Systems A case study for Modern Greek Georgios Paraskevopoulos Theodoros Kouzelis Georgios Rouvalis Athanasios Katsamanis V. Katsouros Alexandros Potamianos VLM 25 7 0 31 Dec 2022
Predicting Multi-Codebook Vector Quantization Indexes for Knowledge Distillation Liyong Guo Xiaoyu Yang Quandong Wang Yuxiang Kong Zengwei Yao ... Wei Kang Long Lin Mingshuang Luo Piotr Żelasko Daniel Povey VLM 31 7 0 31 Oct 2022
Comparison of Soft and Hard Target RNN-T Distillation for Large-scale ASR DongSeon Hwang K. Sim Yu Zhang Trevor Strohman 14 10 0 11 Oct 2022
Multi-mode Transformer Transducer with Stochastic Future Context Kwangyoun Kim Felix Wu Prashant Sridhar Kyu Jeong Han Shinji Watanabe 30 9 0 17 Jun 2021
Collaborative Training of Acoustic Encoders for Speech Recognition Varun K. Nagaraja Yangyang Shi Ganesh Venkatesh Ozlem Kalinli M. Seltzer Vikas Chandra 43 11 0 16 Jun 2021
HMM-Free Encoder Pre-Training for Streaming RNN Transducer Lu Huang J. Sun Yu Tang Junfeng Hou Jinkun Chen Jun Zhang Zejun Ma 25 3 0 02 Apr 2021
Dual-mode ASR: Unify and Improve Streaming ASR with Full-context Modeling Jiahui Yu Wei Han Anmol Gulati Chung-Cheng Chiu Bo-wen Li Tara N. Sainath Yonghui Wu Ruoming Pang 27 18 0 12 Oct 2020