Comparative layer-wise analysis of self-supervised speech models

8 November 2022

Papers citing "Comparative layer-wise analysis of self-supervised speech models"

37 / 87 papers shown

Title
Speech foundation models in healthcare: Effect of layer selection on pathological speech feature prediction D. Wiepert Rene L. Utianski Joseph R. Duffy John L. Stricker L. Barnard David T. Jones Hugo Botha 25 3 0 02 Feb 2024
Revisiting speech segmentation and lexicon learning with better features Herman Kamper Benjamin van Niekerk VLM SSL 24 1 0 31 Jan 2024
What Do Self-Supervised Speech and Speaker Models Learn? New Findings From a Cross Model Layer-Wise Analysis Takanori Ashihara Marc Delcroix Takafumi Moriya Kohei Matsuura Taichi Asami Yusuke Ijima SSL 21 7 0 31 Jan 2024
Revisiting Self-supervised Learning of Speech Representation from a Mutual Information Perspective Alexander H. Liu Sung-Lin Yeh James R. Glass SSL 19 3 0 16 Jan 2024
Noise robust distillation of self-supervised speech models via correlation metrics Fabian Ritter Gutierrez Kuan-Po Huang Dianwen Ng Jeremy H. M. Wong Hung-yi Lee Chng Eng Siong Nancy F. Chen 21 1 0 19 Dec 2023
Efficiency-oriented approaches for self-supervised speech representation learning Luis Lugo Valentin Vielzeuf SSL 26 1 0 18 Dec 2023
Understanding Probe Behaviors through Variational Bounds of Mutual Information Kwanghee Choi Jee-weon Jung Shinji Watanabe SSL 22 4 0 15 Dec 2023
STaR: Distilling Speech Temporal Relation for Lightweight Speech Self-Supervised Learning Models Kangwook Jang Sungnyun Kim Hoi-Rim Kim 33 1 0 14 Dec 2023
A Quantitative Approach to Understand Self-Supervised Models as Cross-lingual Feature Extractors Shuyue Stella Li Beining Xu Xiangyu Zhang Hexin Liu Wen-Han Chao Leibny Paola García SSL 31 4 0 27 Nov 2023
Self-Supervised Models of Speech Infer Universal Articulatory Kinematics Cheol Jun Cho Abdelrahman Mohamed Alan W. Black Gopala K. Anumanchipalli SSL 14 10 0 16 Oct 2023
Toward Joint Language Modeling for Speech Units and Text Ju-Chieh Chou Chung-Ming Chien Wei-Ning Hsu Karen Livescu Arun Babu Alexis Conneau Alexei Baevski Michael Auli VLM 26 20 0 12 Oct 2023
MeSa: Masked, Geometric, and Supervised Pre-training for Monocular Depth Estimation Muhammad Osama Khan Junbang Liang Chun-Kai Wang Shan Yang Yu Lou MDE 49 4 0 06 Oct 2023
Transferring speech-generic and depression-specific knowledge for Alzheimer's disease detection Ziyun Cui Wen Wu Wei-Qiang Zhang Ji Wu Chao Zhang 23 2 0 06 Oct 2023
Cross-Modal Multi-Tasking for Speech-to-Text Translation via Hard Parameter Sharing B. Grimstad Xuankai Chang Antonios Anastasopoulos Yuya Fujita Shinji Watanabe 26 2 0 27 Sep 2023
Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Study Xuankai Chang Brian Yan Kwanghee Choi Jee-weon Jung Yichen Lu ... Pengcheng Guo Yao-Fei Cheng Pavel Denisov Kohei Saijo Hsiu-Hsuan Wang 28 36 0 27 Sep 2023
Direct Text to Speech Translation System using Acoustic Units Victoria Mingote Pablo Gimeno Luis Vicente Sameer Khurana Antoine Laurent J. Duret 23 3 0 14 Sep 2023
LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speech Titouan Parcollet H. Nguyen Solène Evain Marcely Zanon Boito Adrien Pupier ... François Portet Solange Rossato F. Ringeval D. Schwab Laurent Besacier 40 15 0 11 Sep 2023
Self-Supervised Video Transformers for Isolated Sign Language Recognition Marcelo Sandoval-Castaneda Yanhong Li D. Brentari Karen Livescu Gregory Shakhnarovich SLR 23 2 0 02 Sep 2023
Speech Self-Supervised Representations Benchmarking: a Case for Larger Probing Heads Salah Zaiem Youcef Kemiche Titouan Parcollet S. Essid Mirco Ravanelli SSL 27 11 0 28 Aug 2023
Decoding Emotions: A comprehensive Multilingual Study of Speech Models for Speech Emotion Recognition Anant Singh Akshat Gupta 26 4 0 17 Aug 2023
AKVSR: Audio Knowledge Empowered Visual Speech Recognition by Compressing Audio Knowledge of a Pretrained Model Jeong Hun Yeo Minsu Kim J. Choi Dae Hoe Kim Y. Ro 26 18 0 15 Aug 2023
Non Intrusive Intelligibility Predictor for Hearing Impaired Individuals using Self Supervised Speech Representations George Close Thomas Hain Stefan Goetze 21 4 0 25 Jul 2023
On-Device Constrained Self-Supervised Speech Representation Learning for Keyword Spotting via Knowledge Distillation Gene-Ping Yang Yue Gu Qingming Tang Dongsu Du Yuzong Liu 22 5 0 06 Jul 2023
What Do Self-Supervised Speech Models Know About Words? Ankita Pasad C. Chien Shane Settle Karen Livescu SSL 33 26 0 30 Jun 2023
SpeechGLUE: How Well Can Self-Supervised Speech Models Capture Linguistic Knowledge? Takanori Ashihara Takafumi Moriya Kohei Matsuura Tomohiro Tanaka Yusuke Ijima Taichi Asami Marc Delcroix Yukinori Honma SSL ELM 27 11 0 14 Jun 2023
Probing self-supervised speech models for phonetic and phonemic information: a case study in aspiration Kinan Martin Jon Gauthier Canaan Breiss R. Levy SSL 21 14 0 09 Jun 2023
Investigating Pre-trained Audio Encoders in the Low-Resource Condition Haomiao Yang Jinming Zhao Gholamreza Haffari Ehsan Shareghi 19 6 0 28 May 2023
Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Model Puyuan Peng Shang-Wen Li Okko Rasanen Abdel-rahman Mohamed David Harwath SSL VLM 26 7 0 19 May 2023
ML-SUPERB: Multilingual Speech Universal PERformance Benchmark Jiatong Shi Dan Berrebbi William Chen Ho-Lam Chung En-Pei Hu ... Xuankai Chang Shang-Wen Li Abdel-rahman Mohamed Hung-yi Lee Shinji Watanabe ELM 55 58 0 18 May 2023
MelHuBERT: A simplified HuBERT on Mel spectrograms Tzu-Quan Lin Hung-yi Lee Hao Tang SSL 32 13 0 17 Nov 2022
Analyzing Acoustic Word Embeddings from Pre-trained Self-supervised Speech Models Ramon Sanabria Hao Tang Sharon Goldwater SSL 38 18 0 28 Oct 2022
Opening the Black Box of wav2vec Feature Encoder Kwanghee Choi E. Yeo SSL 38 15 0 27 Oct 2022
Exploration of A Self-Supervised Speech Model: A Study on Emotional Corpora Yuanchao Li Yumnah Mohamied P. Bell Catherine Lai SSL 37 45 0 05 Oct 2022
Predicting within and across language phoneme recognition performance of self-supervised learning speech pre-trained models Han Ji T. Patel O. Scharenborg 36 7 0 24 Jun 2022
Self-Supervised Speech Representation Learning: A Review Abdel-rahman Mohamed Hung-yi Lee Lasse Borgholt Jakob Drachmann Havtorn Joakim Edin ... Shang-Wen Li Karen Livescu Lars Maaløe Tara N. Sainath Shinji Watanabe SSL AI4TS 128 349 0 21 May 2022
What do End-to-End Speech Models Learn about Speaker, Language and Channel Information? A Layer-wise and Neuron-level Analysis Shammur A. Chowdhury Nadir Durrani Ahmed M. Ali 36 12 0 01 Jul 2021
Exploring wav2vec 2.0 on speaker verification and language identification Zhiyun Fan Meng Li Shiyu Zhou Bo Xu 117 202 0 11 Dec 2020