PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition

10 June 2021

Kaizhi Qian

Papers citing "PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition"

35 / 35 papers shown

Title
Low-Rank and Sparse Model Merging for Multi-Lingual Speech Recognition and Translation Qiuming Zhao Guangzhi Sun Chao Zhang Mingxing Xu Thomas Fang Zheng MoMe VLM 160 0 0 24 Feb 2025
Efficient Training of Self-Supervised Speech Foundation Models on a Compute Budget Andy T. Liu Yi-Cheng Lin Haibin Wu Stefan Winkler Hung-yi Lee 31 1 0 09 Sep 2024
Tailored Design of Audio-Visual Speech Recognition Models using Branchformers David Gimeno-Gómez Carlos David Martínez Hinarejos 91 2 0 09 Jul 2024
Automatic speech recognition for the Nepali language using CNN, bidirectional LSTM and ResNet Manish Dhakal Arman Chhetri Aman Kumar Gupta Prabin B. Lamichhane S. Pandey S. Shakya AI4TS 35 10 0 25 Jun 2024
Outlier Reduction with Gated Attention for Improved Post-training Quantization in Large Sequence-to-sequence Speech Foundation Models Dominik Wagner Ilja Baumann K. Riedhammer Tobias Bocklet MQ 30 1 0 16 Jun 2024
Sustainable self-supervised learning for speech representations Luis Lugo Valentin Vielzeuf 31 2 0 11 Jun 2024
DAISY: Data Adaptive Self-Supervised Early Exit for Speech Representation Models T. Lin Hung-yi Lee Hao Tang 40 1 0 08 Jun 2024
OWSM-CTC: An Open Encoder-Only Speech Foundation Model for Speech Recognition, Translation, and Language Identification Yifan Peng Yui Sudo Muhammad Shakeel Shinji Watanabe VLM 37 17 0 20 Feb 2024
Efficiency-oriented approaches for self-supervised speech representation learning Luis Lugo Valentin Vielzeuf SSL 26 1 0 18 Dec 2023
USM-Lite: Quantization and Sparsity Aware Fine-tuning for Speech Recognition with Universal Speech Models Shaojin Ding David Qiu David Rim Yanzhang He Oleg Rybakov ... Tara N. Sainath Zhonglin Han Jian Li Amir Yazdanbakhsh Shivani Agrawal MQ 26 9 0 13 Dec 2023
One-Step Knowledge Distillation and Fine-Tuning in Using Large Pre-Trained Self-Supervised Learning Models for Speaker Verification Ju-Sung Heo Chan-yeong Lim Ju-ho Kim Hyun-Seo Shin Ha-Jin Yu 23 2 0 27 May 2023
I3D: Transformer architectures with input-dependent dynamic depth for speech recognition Yifan Peng Jaesong Lee Shinji Watanabe 27 19 0 14 Mar 2023
Compressing Transformer-based self-supervised models for speech processing Tzu-Quan Lin Tsung-Huan Yang Chun-Yao Chang Kuang-Ming Chen Tzu-hsun Feng Hung-yi Lee Hao Tang 40 6 0 17 Nov 2022
$RedApt: An Adaptor for wav2vec 2 Encoding \\ Faster and Smaller Speech Translation without Quality Compromise$ RedApt: An Adaptor for wav2vec 2 Encoding \\ Faster and Smaller Speech Translation without Quality Compromise Jinming Zhao Haomiao Yang Gholamreza Haffari Ehsan Shareghi VLM 16 2 0 16 Oct 2022
Self-Supervised Speech Representation Learning: A Review Abdel-rahman Mohamed Hung-yi Lee Lasse Borgholt Jakob Drachmann Havtorn Joakim Edin ... Shang-Wen Li Karen Livescu Lars Maaløe Tara N. Sainath Shinji Watanabe SSL AI4TS 128 349 0 21 May 2022
Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages Felix Wu Kwangyoun Kim Shinji Watanabe Kyu Jeong Han Ryan T. McDonald Kilian Q. Weinberger Yoav Artzi SyDa 45 37 0 02 May 2022
PADA: Pruning Assisted Domain Adaptation for Self-Supervised Speech Representations L. D. Prasad Sreyan Ghosh S. Umesh 25 12 0 31 Mar 2022
SpeechPrompt: An Exploration of Prompt Tuning on Generative Spoken Language Model for Speech Processing Tasks Kai-Wei Chang Wei-Cheng Tseng Shang-Wen Li Hung-yi Lee 24 22 0 31 Mar 2022
LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT Rui Wang Qibing Bai Junyi Ao Long Zhou Zhixiang Xiong Zhihua Wei Yu Zhang Tom Ko Haizhou Li 34 61 0 29 Mar 2022
Don't speak too fast: The impact of data bias on self-supervised speech models Yen Meng Yi-Hui Chou Andy T. Liu Hung-yi Lee 34 25 0 15 Oct 2021
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing Junyi Ao Rui Wang Long Zhou Chengyi Wang Shuo Ren ... Yu Zhang Zhihua Wei Yao Qian Jinyu Li Furu Wei 118 193 0 14 Oct 2021
Mandarin-English Code-switching Speech Recognition with Self-supervised Speech Representation Models Liang-Hsuan Tseng Yu-Kuan Fu Heng-Jui Chang Hung-yi Lee SSL 28 14 0 07 Oct 2021
DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT Heng-Jui Chang Shu-Wen Yang Hung-yi Lee SSL 34 163 0 05 Oct 2021
On the Interplay Between Sparsity, Naturalness, Intelligibility, and Prosody in Speech Synthesis Cheng-I Jeff Lai Erica Cooper Yang Zhang Shiyu Chang Kaizhi Qian ... Yung-Sung Chuang Alexander H. Liu Junichi Yamagishi David D. Cox James R. Glass 26 6 0 04 Oct 2021
Simple and Effective Zero-shot Cross-lingual Phoneme Recognition Qiantong Xu Alexei Baevski Michael Auli VLM 29 77 0 23 Sep 2021
Speech Representation Learning Combining Conformer CPC with Deep Cluster for the ZeroSpeech Challenge 2021 Takashi Maekaku Xuankai Chang Yuya Fujita Li-Wei Chen Shinji Watanabe Alexander I. Rudnicky 109 13 0 13 Jul 2021
Generative Spoken Language Modeling from Raw Audio Kushal Lakhotia Evgeny Kharitonov Wei-Ning Hsu Yossi Adi Adam Polyak ... Tu Nguyen Jade Copet Alexei Baevski A. Mohamed Emmanuel Dupoux AuLLM 191 337 0 01 Feb 2021
Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognition Yangyang Shi Yongqiang Wang Chunyang Wu Ching-Feng Yeh Julian Chan Frank Zhang Duc Le M. Seltzer 56 168 0 21 Oct 2020
Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition Yu Zhang James Qin Daniel S. Park Wei Han Chung-Cheng Chiu Ruoming Pang Quoc V. Le Yonghui Wu VLM SSL 146 308 0 20 Oct 2020
The Lottery Ticket Hypothesis for Pre-trained BERT Networks Tianlong Chen Jonathan Frankle Shiyu Chang Sijia Liu Yang Zhang Zhangyang Wang Michael Carbin 156 345 0 23 Jul 2020
Similarity Analysis of Contextual Word Representation Models John M. Wu Yonatan Belinkov Hassan Sajjad Nadir Durrani Fahim Dalvi James R. Glass 51 73 0 03 May 2020
What is the State of Neural Network Pruning? Davis W. Blalock Jose Javier Gonzalez Ortiz Jonathan Frankle John Guttag 191 1,027 0 06 Mar 2020
Comparing Rewinding and Fine-tuning in Neural Network Pruning Alex Renda Jonathan Frankle Michael Carbin 224 383 0 05 Mar 2020
Scaling Up Online Speech Recognition Using ConvNets Vineel Pratap Qiantong Xu Jacob Kahn Gilad Avidov Tatiana Likhomanenko Awni Y. Hannun Vitaliy Liptchinsky Gabriel Synnaeve R. Collobert 154 38 0 27 Jan 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018