DPHuBERT: Joint Distillation and Pruning of Self-Supervised Speech Models

28 May 2023

Papers citing "DPHuBERT: Joint Distillation and Pruning of Self-Supervised Speech Models"

23 / 23 papers shown

Title
On The Landscape of Spoken Language Models: A Comprehensive Survey Siddhant Arora Kai-Wei Chang Chung-Ming Chien Yifan Peng Haibin Wu Yossi Adi Emmanuel Dupoux Hung-yi Lee Karen Livescu Shinji Watanabe 52 3 0 11 Apr 2025
How Redundant Is the Transformer Stack in Speech Representation Models? Teresa Dorszewski Albert Kjøller Jacobsen Lenka Tětková Lars Kai Hansen 107 0 0 20 Jan 2025
Deploying Foundation Model Powered Agent Services: A Survey Wenchao Xu Jinyu Chen Peirong Zheng Xiaoquan Yi Tianyi Tian ... Quan Wan Yining Qi Yunfeng Fan Qinliang Su Xuemin Shen AI4CE 119 1 0 18 Dec 2024
SSDM: Scalable Speech Dysfluency Modeling Jiachen Lian Xuanru Zhou Z. Ezzes Jet M J Vonk Brittany Morin D. Baquirin Zachary Mille M. G. Tempini Gopala Anumanchipalli AuLLM 32 1 0 29 Aug 2024
One-pass Multiple Conformer and Foundation Speech Systems Compression and Quantization Using An All-in-one Neural Model Zhaoqing Li Haoning Xu Tianzi Wang Shoukang Hu Zengrui Jin Shujie Hu Jiajun Deng Mingyu Cui Mengzhe Geng Xunying Liu MQ 37 1 0 14 Jun 2024
AdaPTwin: Low-Cost Adaptive Compression of Product Twins in Transformers Emil Biju Anirudh Sriram Mert Pilanci 54 0 0 13 Jun 2024
GenDistiller: Distilling Pre-trained Language Models based on an Autoregressive Generative Model Yingying Gao Shilei Zhang Chao Deng Junlan Feng 32 0 0 12 Jun 2024
Sustainable self-supervised learning for speech representations Luis Lugo Valentin Vielzeuf 31 2 0 11 Jun 2024
Active Learning with Task Adaptation Pre-training for Speech Emotion Recognition Dongyuan Li Ying Zhang Yusong Wang Funakoshi Kataro Manabu Okumura 32 1 0 01 May 2024
A Large-Scale Evaluation of Speech Foundation Models Shu-Wen Yang Heng-Jui Chang Zili Huang Andy T. Liu Cheng-I Jeff Lai ... Kushal Lakhotia Shang-Wen Li Abdelrahman Mohamed Shinji Watanabe Hung-yi Lee 38 19 0 15 Apr 2024
An Efficient End-to-End Approach to Noise Invariant Speech Features via Multi-Task Learning Heitor R. Guimarães Arthur Pimentel Anderson R. Avila Mehdi Rezagholizadeh Boxing Chen Tiago H. Falk 67 1 0 13 Mar 2024
SKILL: Similarity-aware Knowledge distILLation for Speech Self-Supervised Learning Luca Zampierin G. B. Hacene Bac Nguyen Mirco Ravanelli 46 2 0 26 Feb 2024
OWSM-CTC: An Open Encoder-Only Speech Foundation Model for Speech Recognition, Translation, and Language Identification Yifan Peng Yui Sudo Muhammad Shakeel Shinji Watanabe VLM 46 17 0 20 Feb 2024
OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformer Yifan Peng Jinchuan Tian William Chen Siddhant Arora Brian Yan ... Kwanghee Choi Jiatong Shi Xuankai Chang Jee-weon Jung Shinji Watanabe VLM OSLM 34 40 0 30 Jan 2024
Pheme: Efficient and Conversational Speech Generation Paweł Budzianowski Taras Sereda Tomasz Cichy Ivan Vulić 32 7 0 05 Jan 2024
Efficiency-oriented approaches for self-supervised speech representation learning Luis Lugo Valentin Vielzeuf SSL 31 1 0 18 Dec 2023
STaR: Distilling Speech Temporal Relation for Lightweight Speech Self-Supervised Learning Models Kangwook Jang Sungnyun Kim Hoi-Rim Kim 36 1 0 14 Dec 2023
USM-Lite: Quantization and Sparsity Aware Fine-tuning for Speech Recognition with Universal Speech Models Shaojin Ding David Qiu David Rim Yanzhang He Oleg Rybakov ... Tara N. Sainath Zhonglin Han Jian Li Amir Yazdanbakhsh Shivani Agrawal MQ 34 9 0 13 Dec 2023
EFFUSE: Efficient Self-Supervised Feature Fusion for E2E ASR in Low Resource and Multilingual Scenarios Tejes Srivastava Jiatong Shi William Chen Shinji Watanabe 32 1 0 05 Oct 2023
Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Data Yifan Peng Jinchuan Tian Brian Yan Dan Berrebbi Xuankai Chang ... Yui Sudo Muhammad Shakeel Jee-weon Jung Soumi Maiti Shinji Watanabe VLM 39 35 0 25 Sep 2023
CoLLD: Contrastive Layer-to-layer Distillation for Compressing Multilingual Pre-trained Speech Encoders Heng-Jui Chang Ning Dong Ruslan Mavlyutov Sravya Popuri Yu-An Chung 42 6 0 14 Sep 2023
MelHuBERT: A simplified HuBERT on Mel spectrograms Tzu-Quan Lin Hung-yi Lee Hao Tang SSL 32 13 0 17 Nov 2022
Self-Supervised Speech Representation Learning: A Review Abdel-rahman Mohamed Hung-yi Lee Lasse Borgholt Jakob Drachmann Havtorn Joakim Edin ... Shang-Wen Li Karen Livescu Lars Maaløe Tara N. Sainath Shinji Watanabe SSL AI4TS 137 352 0 21 May 2022