Distilling the Knowledge of BERT for Sequence-to-Sequence ASR

Distilling the Knowledge of BERT for Sequence-to-Sequence ASR

9 August 2020

Tatsuya Kawahara

Papers citing "Distilling the Knowledge of BERT for Sequence-to-Sequence ASR"

10 / 10 papers shown

Title
It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition Chen Chen Ruizhe Li Yuchen Hu Sabato Marco Siniscalchi Pin-Yu Chen Ensiong Chng Chao-Han Huck Yang 28 19 0 08 Feb 2024
Multiple Representation Transfer from Large Language Models to End-to-End ASR Systems Takuma Udagawa Masayuki Suzuki Gakuto Kurata Masayasu Muraoka G. Saon 32 2 0 07 Sep 2023
Towards domain generalisation in ASR with elitist sampling and ensemble knowledge distillation Rehan Ahmad Md. Asif Jalal Muhammad Umar Farooq A. Ollerenshaw Thomas Hain 18 2 0 01 Mar 2023
Topics in Contextualised Attention Embeddings Mozhgan Talebpour A. G. S. D. Herrera Shoaib Jameel 26 2 0 11 Jan 2023
A context-aware knowledge transferring strategy for CTC-based ASR Keda Lu Kuan-Yu Chen 15 14 0 12 Oct 2022
Effect and Analysis of Large-scale Language Model Rescoring on Competitive ASR Systems Takuma Udagawa Masayuki Suzuki Gakuto Kurata N. Itoh G. Saon 34 23 0 01 Apr 2022
Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decoding Peter Sullivan Toshiko Shibano Muhammad Abdul-Mageed 34 11 0 10 Feb 2022
Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classification Yiding Jiang Bidisha Sharma Maulik C. Madhavi Haizhou Li 20 25 0 05 Aug 2021
Hierarchical Transformer-based Large-Context End-to-end ASR with Large-Context Knowledge Distillation Ryo Masumura Naoki Makishima Mana Ihori Akihiko Takashima Tomohiro Tanaka Shota Orihashi 28 29 0 16 Feb 2021
Speech Recognition by Simply Fine-tuning BERT Wen-Chin Huang Chia-Hua Wu Shang-Bao Luo Kuan-Yu Chen Hsin-Min Wang T. Toda 70 28 0 30 Jan 2021