Improving Transformer-based Speech Recognition Using Unsupervised Pre-training

22 October 2019

Wei Zou

Xiangang Li

ArXiv PDF HTML

Papers citing "Improving Transformer-based Speech Recognition Using Unsupervised Pre-training"

24 / 24 papers shown

Title
Homophone Disambiguation Reveals Patterns of Context Mixing in Speech Transformers Hosein Mohebbi Grzegorz Chrupała Willem H. Zuidema A. Alishahi 28 12 0 15 Oct 2023
Indonesian Automatic Speech Recognition with XLSR-53 Panji Arisaputra Amalia Zahra 21 6 0 20 Aug 2023
On-Device Constrained Self-Supervised Speech Representation Learning for Keyword Spotting via Knowledge Distillation Gene-Ping Yang Yue Gu Qingming Tang Dongsu Du Yuzong Liu 20 5 0 06 Jul 2023
PulseImpute: A Novel Benchmark Task for Pulsative Physiological Signal Imputation Maxwell A. Xu Alexander Moreno Supriya Nagesh V. Aydemir D. Wetter Santosh Kumar James M. Rehg AI4TS 22 7 0 14 Dec 2022
MelHuBERT: A simplified HuBERT on Mel spectrograms Tzu-Quan Lin Hung-yi Lee Hao Tang SSL 32 13 0 17 Nov 2022
SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learning Tzu-hsun Feng Annie Dong Ching-Feng Yeh Shu-Wen Yang Tzu-Quan Lin ... Xuankai Chang Shinji Watanabe Abdel-rahman Mohamed Shang-Wen Li Hung-yi Lee ELM SSL 26 33 0 16 Oct 2022
Transfer Learning of wav2vec 2.0 for Automatic Lyric Transcription Longshen Ou Xiangming Gu Ye Wang 25 21 0 20 Jul 2022
Speaker Identification using Speech Recognition Syeda Rabia Arshad Syed Mujtaba Haider Abdul Basit Mughal 19 1 0 29 May 2022
Self-Supervised Speech Representation Learning: A Review Abdel-rahman Mohamed Hung-yi Lee Lasse Borgholt Jakob Drachmann Havtorn Joakim Edin ... Shang-Wen Li Karen Livescu Lars Maaløe Tara N. Sainath Shinji Watanabe SSL AI4TS 128 349 0 21 May 2022
Audio Self-supervised Learning: A Survey Shuo Liu Adria Mallol-Ragolta Emilia Parada-Cabeleiro Kun Qian Xingshuo Jing Alexander Kathan Bin Hu Bjoern W. Schuller SSL 35 106 0 02 Mar 2022
A Brief Overview of Unsupervised Neural Speech Representation Learning Lasse Borgholt Jakob Drachmann Havtorn Joakim Edin Lars Maaløe Christian Igel BDL AI4TS SSL 19 11 0 01 Mar 2022
Lacuna Reconstruction: Self-supervised Pre-training for Low-Resource Historical Document Transcription Nikolai Vogler J. Allen M. Miller Taylor Berg-Kirkpatrick 21 5 0 16 Dec 2021
Bi-APC: Bidirectional Autoregressive Predictive Coding for Unsupervised Pre-training and Its Application to Children's ASR Ruchao Fan Amber Afshan Abeer Alwan 21 14 0 12 Feb 2021
Efficiently Fusing Pretrained Acoustic and Linguistic Encoders for Low-resource Speech Recognition Cheng Yi Shiyu Zhou Bo Xu 49 40 0 17 Jan 2021
Exploring wav2vec 2.0 on speaker verification and language identification Zhiyun Fan Meng Li Shiyu Zhou Bo Xu 103 202 0 11 Dec 2020
Towards Semi-Supervised Semantics Understanding from Speech Cheng-I Jeff Lai Jin Cao S. Bodapati Shang-Wen Li SSL 14 7 0 11 Nov 2020
Learning Contextual Tag Embeddings for Cross-Modal Alignment of Audio and Tags Xavier Favory K. Drossos Tuomas Virtanen Xavier Serra 24 15 0 27 Oct 2020
Similarity Analysis of Self-Supervised Speech Representations Yu-An Chung Yonatan Belinkov James R. Glass SSL 33 36 0 22 Oct 2020
Representation Learning for Sequence Data with Deep Autoencoding Predictive Components Junwen Bai Weiran Wang Yingbo Zhou Caiming Xiong SSL AI4TS 23 12 0 07 Oct 2020
Transformer with Bidirectional Decoder for Speech Recognition Xi Chen Songyang Zhang Dandan Song P. Ouyang Shouyi Yin 16 13 0 11 Aug 2020
Transformer based unsupervised pre-training for acoustic representation learning Ruixiong Zhang Haiwei Wu Wubo Li Dongwei Jiang Wei Zou Xiangang Li SSL ViT 20 27 0 29 Jul 2020
TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech Andy T. Liu Shang-Wen Li Hung-yi Lee SSL 53 356 0 12 Jul 2020
Unsupervised Cross-lingual Representation Learning for Speech Recognition Alexis Conneau Alexei Baevski R. Collobert Abdel-rahman Mohamed Michael Auli SSL 41 754 0 24 Jun 2020
Input-independent Attention Weights Are Expressive Enough: A Study of Attention in Self-supervised Audio Transformers Tsung-Han Wu Chun-Chen Hsieh Yen-Hao Chen Po-Han Chi Hung-yi Lee 18 1 0 09 Jun 2020