Injecting Text in Self-Supervised Speech Pretraining

27 August 2021

Zhehuai Chen

Yu Zhang

Andrew Rosenberg

Bhuvana Ramabhadran

Papers citing "Injecting Text in Self-Supervised Speech Pretraining"

39 / 39 papers shown

Title
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units Wei-Ning Hsu Benjamin Bolte Yao-Hung Hubert Tsai Kushal Lakhotia Ruslan Salakhutdinov Abdel-rahman Mohamed SSL 149 2,949 0 14 Jun 2021
SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network William Chan Daniel S. Park Chris A. Lee Yu Zhang Quoc V. Le Mohammad Norouzi AI4TS 70 138 0 05 Apr 2021
Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training Wei-Ning Hsu Anuroop Sriram Alexei Baevski Tatiana Likhomanenko Qiantong Xu ... Jacob Kahn Ann Lee R. Collobert Gabriel Synnaeve Michael Auli SSL 67 240 0 02 Apr 2021
Self-supervised Text-independent Speaker Verification using Prototypical Momentum Contrastive Learning Wei Xia Chunlei Zhang Chao Weng Meng Yu Dong Yu SSL 53 79 0 13 Dec 2020
Joint Masked CPC and CTC Training for ASR Chaitanya Talnikar Tatiana Likhomanenko R. Collobert Gabriel Synnaeve SSL 88 27 0 30 Oct 2020
Speech SIMCLR: Combining Contrastive and Reconstruction Objective for Self-supervised Speech Representation Learning Dongwei Jiang Wubo Li Miao Cao Wei Zou Xiangang Li SSL 50 65 0 27 Oct 2020
Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition Yu Zhang James Qin Daniel S. Park Wei Han Chung-Cheng Chiu Ruoming Pang Quoc V. Le Yonghui Wu VLM SSL 179 309 0 20 Oct 2020
Improving Tail Performance of a Deliberation E2E ASR Model Using a Large Text Corpus Cal Peyser S. Mavandadi Tara N. Sainath J. Apfel Ruoming Pang Shankar Kumar 53 46 0 24 Aug 2020
Semi-Supervised Learning with Data Augmentation for End-to-End ASR F. Weninger F. Mana R. Gemello Jesús Andrés-Ferrer P. Zhan 58 30 0 27 Jul 2020
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations Alexei Baevski Henry Zhou Abdel-rahman Mohamed Michael Auli SSL 245 5,774 0 20 Jun 2020
Improved Noisy Student Training for Automatic Speech Recognition Daniel S. Park Yu Zhang Ye Jia Wei Han Chung-Cheng Chiu Yue Liu Yonghui Wu Quoc V. Le 92 242 0 19 May 2020
Conformer: Convolution-augmented Transformer for Speech Recognition Anmol Gulati James Qin Chung-Cheng Chiu Niki Parmar Yu Zhang ... Wei Han Shibo Wang Zhengdong Zhang Yonghui Wu Ruoming Pang 212 3,130 0 16 May 2020
Hybrid Autoregressive Transducer (hat) Ehsan Variani David Rybach Cyril Allauzen Michael Riley 53 160 0 12 Mar 2020
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence Kihyuk Sohn David Berthelot Chun-Liang Li Zizhao Zhang Nicholas Carlini E. D. Cubuk Alexey Kurakin Han Zhang Colin Raffel AAML 153 3,545 0 21 Jan 2020
Libri-Light: A Benchmark for ASR with Limited or No Supervision Jacob Kahn M. Rivière Weiyi Zheng Evgeny Kharitonov Qiantong Xu ... Tatiana Likhomanenko Gabriel Synnaeve Armand Joulin Abdel-rahman Mohamed Emmanuel Dupoux AuLLM 55 670 0 17 Dec 2019
Self-training with Noisy Student improves ImageNet classification Qizhe Xie Minh-Thang Luong Eduard H. Hovy Quoc V. Le NoLa 300 2,387 0 11 Nov 2019
Generative Pre-Training for Speech with Autoregressive Predictive Coding Yu-An Chung James R. Glass SSL 52 174 0 23 Oct 2019
Speech Recognition with Augmented Synthesized Speech Andrew Rosenberg Yu Zhang Bhuvana Ramabhadran Ye Jia Pedro J. Moreno Yonghui Wu Zelin Wu 61 127 0 25 Sep 2019
Almost Unsupervised Text to Speech and Automatic Speech Recognition Yi Ren Xu Tan Tao Qin Sheng Zhao Zhou Zhao Tie-Yan Liu 69 102 0 13 May 2019
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition Daniel S. Park William Chan Yu Zhang Chung-Cheng Chiu Barret Zoph E. D. Cubuk Quoc V. Le VLM 167 3,455 0 18 Apr 2019
Parrotron: An End-to-End Speech-to-Speech Conversion Model and its Applications to Hearing-Impaired Speech and Speech Separation Fadi Biadsy Ron J. Weiss Pedro J. Moreno D. Kanvesky Ye Jia 62 115 0 08 Apr 2019
LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech Heiga Zen Viet Dang R. Clark Yu Zhang Ron J. Weiss Ye Jia Zhiwen Chen Yonghui Wu 96 951 0 05 Apr 2019
Adversarial Training of End-to-end Speech Recognition Using a Criticizing Language Model Alexander H. Liu Hung-yi Lee Lin-Shan Lee AuLLM 48 47 0 02 Nov 2018
Cycle-consistency training for end-to-end speech recognition Takaaki Hori Ramón Fernández Astudillo Tomoki Hayashi Yu Zhang Shinji Watanabe Jonathan Le Roux 65 87 0 02 Nov 2018
Training Neural Speech Recognition Systems with Synthetic Speech Augmentation Jason Chun Lok Li R. Gadde Boris Ginsburg Vitaly Lavrukhin 44 55 0 02 Nov 2018
Hierarchical Generative Modeling for Controllable Speech Synthesis Wei-Ning Hsu Yu Zhang Ron J. Weiss Heiga Zen Yonghui Wu ... Ye Jia Zhiwen Chen Jonathan Shen Patrick Nguyen Ruoming Pang BDL 66 275 0 16 Oct 2018
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing Taku Kudo John Richardson 180 3,518 0 19 Aug 2018
Back-Translation-Style Data Augmentation for End-to-End ASR Tomoki Hayashi Shinji Watanabe Yu Zhang Tomoki Toda Takaaki Hori Ramón Fernández Astudillo K. Takeda 70 103 0 28 Jul 2018
Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis Ye Jia Yu Zhang Ron J. Weiss Quan Wang Jonathan Shen ... Zhiwen Chen Patrick Nguyen Ruoming Pang Ignacio López Moreno Yonghui Wu 251 830 0 12 Jun 2018
Adafactor: Adaptive Learning Rates with Sublinear Memory Cost Noam M. Shazeer Mitchell Stern ODL 72 1,045 0 11 Apr 2018
Machine Speech Chain with One-shot Speaker Adaptation Andros Tjandra S. Sakti Satoshi Nakamura 60 55 0 28 Mar 2018
Cold Fusion: Training Seq2Seq Models Together with Language Models Anuroop Sriram Heewoo Jun S. Satheesh Adam Coates VLM 75 281 0 21 Aug 2017
Listening while Speaking: Speech Chain by Deep Learning Andros Tjandra S. Sakti Satoshi Nakamura AuLLM 147 167 0 16 Jul 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 656 131,414 0 12 Jun 2017
Dual Learning for Machine Translation Yingce Xia Di He Tao Qin Liwei Wang Nenghai Yu Tie-Yan Liu Wei-Ying Ma AI4CE 80 849 0 01 Nov 2016
Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning Suyoun Kim Takaaki Hori Shinji Watanabe 74 928 0 21 Sep 2016
On Using Monolingual Corpora in Neural Machine Translation Çağlar Gülçehre Orhan Firat Kelvin Xu Kyunghyun Cho Loïc Barrault Huei-Chi Lin Fethi Bougares Holger Schwenk Yoshua Bengio 119 561 0 11 Mar 2015
Speech Recognition with Deep Recurrent Neural Networks Alex Graves Abdel-rahman Mohamed Geoffrey E. Hinton 212 8,513 0 22 Mar 2013
Sequence Transduction with Recurrent Neural Networks Alex Graves 183 1,868 0 14 Nov 2012