Structured Pruning of Self-Supervised Pre-trained Models for Speech Recognition and Understanding

27 February 2023

Kwangyoun Kim

Papers citing "Structured Pruning of Self-Supervised Pre-trained Models for Speech Recognition and Understanding"

22 / 22 papers shown

Title
StableQuant: Layer Adaptive Post-Training Quantization for Speech Foundation Models Yeona Hong Hyewon Han Woo-Jin Chung Hong-Goo Kang MQ 28 0 0 21 Apr 2025
How Redundant Is the Transformer Stack in Speech Representation Models? Teresa Dorszewski Albert Kjøller Jacobsen Lenka Tětková Lars Kai Hansen 107 0 0 20 Jan 2025
Deploying Foundation Model Powered Agent Services: A Survey Wenchao Xu Jinyu Chen Peirong Zheng Xiaoquan Yi Tianyi Tian ... Quan Wan Yining Qi Yunfeng Fan Qinliang Su Xuemin Shen AI4CE 119 1 0 18 Dec 2024
Efficient Training of Self-Supervised Speech Foundation Models on a Compute Budget Andy T. Liu Yi-Cheng Lin Haibin Wu Stefan Winkler Hung-yi Lee 31 1 0 09 Sep 2024
Convexity-based Pruning of Speech Representation Models Teresa Dorszewski Lenka Tětková Lars Kai Hansen 25 2 0 16 Aug 2024
Dynamic Encoder Size Based on Data-Driven Layer-wise Pruning for Speech Recognition Jingjing Xu Wei Zhou Zijian Yang Eugen Beck Ralf Schlueter 34 1 0 10 Jul 2024
Finding Task-specific Subnetworks in Multi-task Spoken Language Understanding Model Hayato Futami Siddhant Arora Yosuke Kashiwagi E. Tsunoo Shinji Watanabe 34 0 0 18 Jun 2024
One-pass Multiple Conformer and Foundation Speech Systems Compression and Quantization Using An All-in-one Neural Model Zhaoqing Li Haoning Xu Tianzi Wang Shoukang Hu Zengrui Jin Shujie Hu Jiajun Deng Mingyu Cui Mengzhe Geng Xunying Liu MQ 37 1 0 14 Jun 2024
Sustainable self-supervised learning for speech representations Luis Lugo Valentin Vielzeuf 31 2 0 11 Jun 2024
DAISY: Data Adaptive Self-Supervised Early Exit for Speech Representation Models T. Lin Hung-yi Lee Hao Tang 40 1 0 08 Jun 2024
SKILL: Similarity-aware Knowledge distILLation for Speech Self-Supervised Learning Luca Zampierin G. B. Hacene Bac Nguyen Mirco Ravanelli 38 2 0 26 Feb 2024
OWSM-CTC: An Open Encoder-Only Speech Foundation Model for Speech Recognition, Translation, and Language Identification Yifan Peng Yui Sudo Muhammad Shakeel Shinji Watanabe VLM 37 17 0 20 Feb 2024
OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformer Yifan Peng Jinchuan Tian William Chen Siddhant Arora Brian Yan ... Kwanghee Choi Jiatong Shi Xuankai Chang Jee-weon Jung Shinji Watanabe VLM OSLM 34 40 0 30 Jan 2024
Efficiency-oriented approaches for self-supervised speech representation learning Luis Lugo Valentin Vielzeuf SSL 26 1 0 18 Dec 2023
Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Data Yifan Peng Jinchuan Tian Brian Yan Dan Berrebbi Xuankai Chang ... Yui Sudo Muhammad Shakeel Jee-weon Jung Soumi Maiti Shinji Watanabe VLM 39 35 0 25 Sep 2023
CoLLD: Contrastive Layer-to-layer Distillation for Compressing Multilingual Pre-trained Speech Encoders Heng-Jui Chang Ning Dong Ruslan Mavlyutov Sravya Popuri Yu-An Chung 42 6 0 14 Sep 2023
Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Compute William Chen Xuankai Chang Yifan Peng Zhaoheng Ni Soumi Maiti Shinji Watanabe SSL 24 25 0 11 Jun 2023
Task-Agnostic Structured Pruning of Speech Representation Models Haoyu Wang Siyuan Wang Weiqiang Zhang Hongbin Suo Yulong Wan VLM 22 14 0 02 Jun 2023
DPHuBERT: Joint Distillation and Pruning of Self-Supervised Speech Models Yifan Peng Yui Sudo Muhammad Shakeel Shinji Watanabe 24 37 0 28 May 2023
Compressing Transformer-based self-supervised models for speech processing Tzu-Quan Lin Tsung-Huan Yang Chun-Yao Chang Kuang-Ming Chen Tzu-hsun Feng Hung-yi Lee Hao Tang 40 6 0 17 Nov 2022
MelHuBERT: A simplified HuBERT on Mel spectrograms Tzu-Quan Lin Hung-yi Lee Hao Tang SSL 32 13 0 17 Nov 2022
Self-Supervised Speech Representation Learning: A Review Abdel-rahman Mohamed Hung-yi Lee Lasse Borgholt Jakob Drachmann Havtorn Joakim Edin ... Shang-Wen Li Karen Livescu Lars Maaløe Tara N. Sainath Shinji Watanabe SSL AI4TS 128 349 0 21 May 2022