HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units

14 June 2021

Papers citing "HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units"

50 / 149 papers shown

Title
Fast and High-Quality Auto-Regressive Speech Synthesis via Speculative Decoding Bohan Li Hankun Wang Situo Zhang Yiwei Guo Kai Yu 90 8 0 29 Oct 2024
VoiceTextBlender: Augmenting Large Language Models with Speech Capabilities via Single-Stage Joint Speech-Text Supervised Fine-Tuning Yifan Peng Krishna Puvvada Zhehuai Chen Piotr .Zelasko He Huang Kunal Dhawan Ke Hu Shinji Watanabe Jagadeesh Balam Boris Ginsburg 113 5 0 23 Oct 2024
LSCodec: Low-Bitrate and Speaker-Decoupled Discrete Speech Codec Yiwei Guo Zhihan Li Chenpeng Du Hankun Wang Xie Chen Kai Yu 78 3 0 21 Oct 2024
Improving Pronunciation and Accent Conversion through Knowledge Distillation And Synthetic Ground-Truth from Native TTS T. Nguyen Seymanur Akti Ngoc-Quan Pham A. Waibel 73 1 0 19 Oct 2024
DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework for Talking Head Video Generation Hanbo Cheng Limin Lin Chenyu Liu Pengcheng Xia Pengfei Hu Jiefeng Ma Jun Du Jia Pan DiffM VGen 395 0 0 17 Oct 2024
Beyond Fixed Topologies: Unregistered Training and Comprehensive Evaluation Metrics for 3D Talking Heads Federico Nocentini T. Besnier Claudio Ferrari Sylvain Arguillere Stefano Berretti Mohamed Daoudi 92 1 0 14 Oct 2024
Improving Semantic Understanding in Speech Language Models via Brain-tuning Omer Moussa Dietrich Klakow Mariya Toneva 73 8 0 11 Oct 2024
The First VoicePrivacy Attacker Challenge Evaluation Plan N. Tomashenko Xiaoxiao Miao Emmanuel Vincent Junichi Yamagishi 204 3 0 09 Oct 2024
SCOREQ: Speech Quality Assessment with Contrastive Regression Alessandro Ragano Jan Skoglund Andrew Hines 93 11 0 09 Oct 2024
Joint Fine-tuning and Conversion of Pretrained Speech and Language Models towards Linear Complexity Mutian He Philip N. Garner 162 0 0 09 Oct 2024
F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching Yushen Chen Zhikang Niu Ziyang Ma Keqi Deng Chunhui Wang Jian Zhao Kai Yu Xie Chen 118 85 0 09 Oct 2024
EmoGene: Audio-Driven Emotional 3D Talking-Head Generation Wenqing Wang Yun Fu VGen 103 0 0 07 Oct 2024
Audio-Agent: Leveraging LLMs For Audio Generation, Editing and Composition Zixuan Wang Chi-Keung Tang Chi-Keung Tang DiffM VGen LLMAG 91 4 0 04 Oct 2024
Recent Advances in Speech Language Models: A Survey Wenqian Cui Dianzhi Yu Xiaoqi Jiao Ziqiao Meng Guangyan Zhang Qichao Wang Yiwen Guo Irwin King AuLLM 144 23 0 01 Oct 2024
SSR: Alignment-Aware Modality Connector for Speech Language Models Weiting Tan Hirofumi Inaguma Ning Dong Paden Tomasello Xutai Ma 96 6 0 30 Sep 2024
MIO: A Foundation Model on Multimodal Tokens Zekun Wang King Zhu Chunpu Xu Wangchunshu Zhou Jiaheng Liu ... Yuanxing Zhang Ge Zhang Ke Xu Jie Fu Wenhao Huang MLLM AuLLM 114 11 0 26 Sep 2024
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions Kai Chen Yunhao Gou Runhui Huang Zhili Liu Daxin Tan ... Qun Liu Jun Yao Lu Hou Hang Xu Hang Xu AuLLM MLLM VLM 118 28 0 26 Sep 2024
MT2KD: Towards A General-Purpose Encoder for Speech, Speaker, and Audio Events Xiaoyu Yang Qiujia Li Chao Zhang P. Woodland 65 1 0 25 Sep 2024
Semi-Supervised Cognitive State Classification from Speech with Multi-View Pseudo-Labeling Yuanchao Li Zixing Zhang Jing Han P. Bell Catherine Lai 92 1 0 25 Sep 2024
TCSinger: Zero-Shot Singing Voice Synthesis with Style Transfer and Multi-Level Style Control Yu Zhang Ziyue Jiang Ruiqi Li Changhao Pan Jinzheng He Rongjie Huang Chuxin Wang Zhou Zhao DiffM VLM 107 7 0 24 Sep 2024
A Comprehensive Survey with Critical Analysis for Deepfake Speech Detection Lam Pham Phat Lam Dat Tran Hieu Tang Tin Nguyen Alexander Schindler Canh Vu Alexander Polonsky Canh Vu 93 5 0 23 Sep 2024
Unsupervised Word Discovery: Boundary Detection with Clustering vs. Dynamic Programming Simon Malan Benjamin van Niekerk Herman Kamper 66 0 0 22 Sep 2024
LC-Protonets: Multi-Label Few-Shot Learning for World Music Audio Tagging Charilaos Papaioannou Emmanouil Benetos Alexandros Potamianos 156 0 0 17 Sep 2024
Speaker-IPL: Unsupervised Learning of Speaker Characteristics with i-Vector based Pseudo-Labels Zakaria Aldeneh Takuya Higuchi Jee-weon Jung Li-Wei Chen Stephen Shum Ahmed Hussen Abdelaziz Shinji Watanabe Tatiana Likhomanenko B. Theobald VLM SSL 82 0 0 16 Sep 2024
Exploring Prediction Targets in Masked Pre-Training for Speech Foundation Models Li-Wei Chen Takuya Higuchi He Bai Ahmed Hussen Abdelaziz Alexander Rudnicky Shinji Watanabe Tatiana Likhomanenko B. Theobald Zakaria Aldeneh 86 0 0 16 Sep 2024
Towards Automatic Assessment of Self-Supervised Speech Models using Rank Zakaria Aldeneh Vimal Thilak Takuya Higuchi B. Theobald Tatiana Likhomanenko SSL 119 1 0 16 Sep 2024
Self-supervised Learning for Acoustic Few-Shot Classification Jingyong Liang Bernd Meyer Isaac Ning Lee Thanh-Toan Do SSL 96 0 0 15 Sep 2024
MacST: Multi-Accent Speech Synthesis via Text Transliteration for Accent Conversion Sho Inoue Shuai Wang Wanxing Wang Pengcheng Zhu Mengxiao Bi Haizhou Li 98 2 0 14 Sep 2024
ProbTalk3D: Non-Deterministic Emotion Controllable Speech-Driven 3D Facial Animation Synthesis Using VQ-VAE Sichun Wu Kazi Injamamul Haque Zerrin Yumak VGen 70 2 0 12 Sep 2024
Salmon: A Suite for Acoustic Language Model Evaluation Gallil Maimon Amit Roth Yossi Adi ELM AuLLM 95 6 0 11 Sep 2024
MoWE-Audio: Multitask AudioLLMs with Mixture of Weak Encoders Wentao Zhang Shuo Sun Bin Wang Xunlong Zou Zhuohan Liu Yingxu He Geyu Lin Nancy F. Chen Ai Ti Aw AuLLM 84 1 0 10 Sep 2024
Efficient Training of Self-Supervised Speech Foundation Models on a Compute Budget Andy T. Liu Yi-Cheng Lin Haibin Wu Stefan Winkler Hung-yi Lee 88 2 0 09 Sep 2024
FireRedTTS: A Foundation Text-To-Speech Framework for Industry-Level Generative Speech Applications Hao-Han Guo Kun Liu Fei-Yu Shen Yi-Chen Wu Xu Tang Kun Xie Kai-Tuo Xu Kun Xie Kai-Tuo Xu 75 26 0 05 Sep 2024
SelectTTS: Synthesizing Anyone's Voice via Discrete Unit-Based Frame Selection Ismail Rasim Ulgen Shreeram Suresh Chandra Junchen Lu Berrak Sisman 410 1 0 30 Aug 2024
Audio xLSTMs: Learning Self-Supervised Audio Representations with xLSTMs Sarthak Yadav Sergios Theodoridis Zheng-Hua Tan 82 3 0 29 Aug 2024
WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling Shengpeng Ji Ziyue Jiang Xize Cheng Yifu Chen Minghui Fang ... Rongjie Huang Yidi Jiang Qian Chen Zhou Zhao Zhou Zhao VLM 103 42 0 29 Aug 2024
Adapting General Disentanglement-Based Speaker Anonymization for Enhanced Emotion Preservation Xiaoxiao Miao Yuxiang Zhang Xin Wang N. Tomashenko D. Soh Ian Mcloughlin 69 2 0 12 Aug 2024
VQ-CTAP: Cross-Modal Fine-Grained Sequence Representation Learning for Speech Processing Chunyu Qiang Wang Geng Yi Zhao Ruibo Fu Tao Wang ... Chen Zhang Hao Che L. Wang Jianwu Dang J. Tao AI4TS 80 0 0 11 Aug 2024
dMel: Speech Tokenization made Simple Richard He Bai Tatiana Likhomanenko Ruixiang Zhang Zijin Gu Zakaria Aldeneh Navdeep Jaitly 83 6 0 22 Jul 2024
Autoregressive Speech Synthesis without Vector Quantization Lingwei Meng Long Zhou Shujie Liu Sanyuan Chen Bing Han ... Jinyu Li Sheng Zhao Xixin Wu Helen M. Meng Furu Wei 124 40 0 11 Jul 2024
Mobile Edge Intelligence for Large Language Models: A Contemporary Survey Guanqiao Qu Qiyuan Chen Wei Wei Zheng Lin Xianhao Chen Kaibin Huang 110 53 0 09 Jul 2024
GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinement Yifan Yang Zheshu Song Jianheng Zhuo Mingyu Cui Jinpeng Li ... Shuai Fan Kai Yu Wei Zhang Guoguo Chen Xie Chen 109 11 0 17 Jun 2024
PianoMotion10M: Dataset and Benchmark for Hand Motion Generation in Piano Performance Qijun Gan Song Wang Shengtao Wu Jianke Zhu 232 1 0 13 Jun 2024
SSAMBA: Self-Supervised Audio Representation Learning with Mamba State Space Model Siavash Shams Sukru Samet Dindar Xilin Jiang N. Mesgarani Mamba 95 21 0 20 May 2024
DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation Jie Xu Karthikeyan P. Saravanan Rogier van Dalen Haaris Mehmood David Tuckey Mete Ozay 136 6 0 10 May 2024
MAD Speech: Measures of Acoustic Diversity of Speech Matthieu Futeral A. Agostinelli Marco Tagliasacchi Neil Zeghidour Eugene Kharitonov 102 1 0 16 Apr 2024
Masked LoGoNet: Fast and Accurate 3D Image Analysis for Medical Domain Amin Karimi Monsefi Payam Karisani Mengxi Zhou Stacey S. Choi Nathan Doble Heng Ji Srinivasan Parthasarathy R. Ramnath 78 5 0 09 Feb 2024
Streaming Sequence Transduction through Dynamic Compression Weiting Tan Yunmo Chen Tongfei Chen Guanghui Qin Haoran Xu Heidi C. Zhang Benjamin Van Durme Philipp Koehn 120 2 0 02 Feb 2024
GMTalker: Gaussian Mixture-based Audio-Driven Emotional Talking Video Portraits Yibo Xia Lizhen Wang Xiang Deng Xiaoyan Luo Yunhong Wang Yebin Liu VGen 83 1 0 12 Dec 2023
On the Effectiveness of ASR Representations in Real-world Noisy Speech Emotion Recognition Xiaohan Shi Jiajun He Xingfeng Li Tomoki Toda 62 4 0 13 Nov 2023