Title
Voila: Voice-Language Foundation Models for Real-Time Autonomous Interaction and Voice Role-Play Yemin Shi Yu Shu Siwei Dong Guangyi Liu Jaward Sesay Jingwen Li Zhiting Hu AuLLM VLM 50 0 0 05 May 2025
fastabx: A library for efficient computation of ABX discriminability Maxime Poli Emmanuel Chemla Emmanuel Dupoux 34 0 0 05 May 2025
SIFT-50M: A Large-Scale Multilingual Dataset for Speech Instruction Fine-Tuning Prabhat Pandey R. Swaminathan K V Vijay Girish Arunasish Sen Jian Xie Grant P. Strimel Andreas Schwarz 140 0 0 12 Apr 2025
Universal Speech Token Learning via Low-Bitrate Neural Codec and Pretrained Representations Xue Jiang Xiulian Peng Yuan Zhang Yan-Heng Lu SSL 83 0 0 15 Mar 2025
MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation Sihyun Yu Meera Hahn Dan Kondratyuk Jinwoo Shin Agrim Gupta José Lezama Irfan Essa David A. Ross Jonathan Huang DiffM VGen 77 0 0 18 Feb 2025
Discrete Speech Unit Extraction via Independent Component Analysis Tomohiko Nakamura Kwanghee Choi Keigo Hojo Yoshiaki Bando Satoru Fukayama Shinji Watanabe 43 0 0 11 Jan 2025
VoiceTextBlender: Augmenting Large Language Models with Speech Capabilities via Single-Stage Joint Speech-Text Supervised Fine-Tuning Yifan Peng Krishna C. Puvvada Zhehuai Chen Piotr .Zelasko He Huang Kunal Dhawan Ke Hu Shinji Watanabe Jagadeesh Balam Boris Ginsburg 58 2 0 23 Oct 2024
Ichigo: Mixed-Modal Early-Fusion Realtime Voice Assistant Alan Dao Dinh Bach Vu Huy Hoang Ha AuLLM VLM 70 3 0 20 Oct 2024
Sylber: Syllabic Embedding Representation of Speech from Raw Audio Cheol Jun Cho Nicholas Lee Akshat Gupta Dhruv Agarwal Ethan Chen Alan W Black Gopala K. Anumanchipalli 34 0 0 09 Oct 2024
Recent Advances in Speech Language Models: A Survey Wenqian Cui Dianzhi Yu Xiaoqi Jiao Ziqiao Meng Guangyan Zhang Qichao Wang Yiwen Guo Irwin King AuLLM 59 14 0 01 Oct 2024
SSR: Alignment-Aware Modality Connector for Speech Language Models Weiting Tan H. Inaguma Ning Dong Paden Tomasello Xutai Ma 32 3 0 30 Sep 2024
Learning Source Disentanglement in Neural Audio Codec Xiaoyu Bie Xubo Liu Gaël Richard 29 1 0 17 Sep 2024
Salmon: A Suite for Acoustic Language Model Evaluation Gallil Maimon Amit Roth Yossi Adi ELM AuLLM 51 5 0 11 Sep 2024
Sample-Efficient Diffusion for Text-To-Speech Synthesis Justin Lovelace Soham Ray Kwangyoun Kim Kilian Q. Weinberger Felix Wu 36 2 0 01 Sep 2024
Language Model Can Listen While Speaking Ziyang Ma Yakun Song Chenpeng Du Jian Cong Zhuo Chen Yuping Wang Yixuan Wang Xie Chen AuLLM 37 23 0 05 Aug 2024
J-CHAT: Japanese Large-scale Spoken Dialogue Corpus for Spoken Dialogue Language Modeling Wataru Nakata Kentaro Seki Hitomi Yanaka Yuki Saito Shinnosuke Takamichi Hiroshi Saruwatari AuLLM 43 0 0 22 Jul 2024
A Benchmark for Multi-speaker Anonymization Xiaoxiao Miao Ruijie Tao Chang Zeng Xin Wang 44 1 0 08 Jul 2024
DiscreteSLU: A Large Language Model with Self-Supervised Discrete Speech Units for Spoken Language Understanding Suwon Shon Kwangyoun Kim Yi-Te Hsu Prashant Sridhar Shinji Watanabe Karen Livescu AuLLM 46 2 0 13 Jun 2024
The Interspeech 2024 Challenge on Speech Processing Using Discrete Units Xuankai Chang Jiatong Shi Jinchuan Tian Yuning Wu Yuxun Tang Yihan Wu Shinji Watanabe Yossi Adi Xie Chen Qin Jin 45 15 0 11 Jun 2024
Learning Fine-Grained Controllability on Speech Generation via Efficient Fine-Tuning Chung-Ming Chien Andros Tjandra Apoorv Vyas Matt Le Bowen Shi Wei-Ning Hsu 32 0 0 10 Jun 2024
MAD Speech: Measures of Acoustic Diversity of Speech Matthieu Futeral A. Agostinelli Marco Tagliasacchi Neil Zeghidour Eugene Kharitonov 51 1 0 16 Apr 2024
Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations Guan-Ting Lin Cheng-Han Chiang Hung-yi Lee 34 22 0 20 Feb 2024
VALL-T: Decoder-Only Generative Transducer for Robust and Decoding-Controllable Text-to-Speech Chenpeng Du Yiwei Guo Hankun Wang Yifan Yang Zhikang Niu Shuai Wang Hui Zhang Xie Chen Kai Yu VLM 22 25 0 25 Jan 2024
GSQA: An End-to-End Model for Generative Spoken Question Answering Min-Han Shih Ho-Lam Chung Yu-Chi Pai Ming-Hao Hsu Guan-Ting Lin Shang-Wen Li Hung-yi Lee ELM AuLLM 33 2 0 15 Dec 2023
SD-HuBERT: Sentence-Level Self-Distillation Induces Syllabic Organization in HuBERT Cheol Jun Cho Abdelrahman Mohamed Shang-Wen Li Alan W. Black Gopala K. Anumanchipalli 31 8 0 16 Oct 2023
Toward Joint Language Modeling for Speech Units and Text Ju-Chieh Chou Chung-Ming Chien Wei-Ning Hsu Karen Livescu Arun Babu Alexis Conneau Alexei Baevski Michael Auli VLM 26 20 0 12 Oct 2023
Few-Shot Spoken Language Understanding via Joint Speech-Text Models Chung-Ming Chien Mingjiamei Zhang Ju-Chieh Chou Karen Livescu 28 3 0 09 Oct 2023
Lip Reading for Low-resource Languages by Learning and Combining General Speech Knowledge and Language-specific Knowledge Minsu Kim Jeong Hun Yeo J. Choi Y. Ro 34 16 0 18 Aug 2023
When to Use Efficient Self Attention? Profiling Text, Speech and Image Transformer Variants Anuj Diwan Eunsol Choi David Harwath 41 0 0 14 Jun 2023
The Effects of Input Type and Pronunciation Dictionary Usage in Transfer Learning for Low-Resource Text-to-Speech P. Do Matt Coler J. Dijkstra E. Klabbers OffRL 16 0 0 01 Jun 2023
Speaker anonymization using orthogonal Householder neural network Xiaoxiao Miao Xin Wang Erica Cooper Junichi Yamagishi N. Tomashenko BDL 24 18 0 30 May 2023
Improving Textless Spoken Language Understanding with Discrete Units as Intermediate Target Guanyong Wu Guan-Ting Lin Shang-Wen Li Hung-yi Lee 26 5 0 29 May 2023
AudioToken: Adaptation of Text-Conditioned Diffusion Models for Audio-to-Image Generation Guy Yariv Itai Gat Lior Wolf Yossi Adi Idan Schwartz DiffM 20 20 0 22 May 2023
Textually Pretrained Speech Language Models Michael Hassid Tal Remez Tu Nguyen Itai Gat Alexis Conneau ... Alexandre Défossez Gabriel Synnaeve Emmanuel Dupoux Roy Schwartz Yossi Adi VLM SyDa 29 53 0 22 May 2023
Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Model Puyuan Peng Shang-Wen Li Okko Rasanen Abdel-rahman Mohamed David Harwath SSL VLM 26 7 0 19 May 2023
DUB: Discrete Unit Back-translation for Speech Translation Dong Zhang Rong Ye Tom Ko Mingxuan Wang Yaqian Zhou 15 23 0 19 May 2023
SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities Dong Zhang Shimin Li Xin Zhang Jun Zhan Pengyu Wang Yaqian Zhou Xipeng Qiu AuLLM MLLM 50 293 0 18 May 2023
SoundStorm: Efficient Parallel Audio Generation Zalan Borsos Matthew Sharifi Damien Vincent Eugene Kharitonov Neil Zeghidour Marco Tagliasacchi 23 97 0 16 May 2023
Back Translation for Speech-to-text Translation Without Transcripts Qingkai Fang Yang Feng 30 13 0 15 May 2023
FoundationTTS: Text-to-Speech for ASR Customization with Generative Language Model Rui Xue Yanqing Liu Lei He Xuejiao Tan Linquan Liu Ed Lin Sheng Zhao 28 7 0 06 Mar 2023
Evaluating Parameter-Efficient Transfer Learning Approaches on SURE Benchmark for Speech Understanding Yingting Li Ambuj Mehrish Shuaijiang Zhao Rishabh Bhardwaj Amir Zadeh Navonil Majumder Rada Mihalcea Soujanya Poria AAML 23 16 0 02 Mar 2023
Speech Corpora Divergence Based Unsupervised Data Selection for ASR Changfeng Gao Gaofeng Cheng Pengyuan Zhang Yonghong Yan 11 0 0 26 Feb 2023
ACE-VC: Adaptive and Controllable Voice Conversion using Explicitly Disentangled Self-supervised Speech Representations Shehzeen Samarah Hussain Paarth Neekhara Jocelyn Huang Jason Chun Lok Li Boris Ginsburg 13 21 0 16 Feb 2023
Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers Chengyi Wang Sanyuan Chen Yu-Huan Wu Zi-Hua Zhang Long Zhou ... Huaming Wang Jinyu Li Lei He Sheng Zhao Furu Wei 48 641 0 05 Jan 2023
ReVISE: Self-Supervised Speech Resynthesis with Visual Input for Universal and Generalized Speech Enhancement Wei-Ning Hsu Tal Remez Bowen Shi Jacob Donley Yossi Adi DiffM 27 12 0 21 Dec 2022
Speaking Style Conversion in the Waveform Domain Using Discrete Self-Supervised Units Gallil Maimon Yossi Adi 26 13 0 19 Dec 2022
Audio Language Modeling using Perceptually-Guided Discrete Representations Felix Kreuk Yaniv Taigman Adam Polyak Jade Copet Gabriel Synnaeve Alexandre Défossez Yossi Adi 29 4 0 02 Nov 2022
Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challenge Ewan Dunbar Nicolas Hamilakis Emmanuel Dupoux SSL 32 30 0 27 Oct 2022
Bootstrapping meaning through listening: Unsupervised learning of spoken sentence embeddings Jian Zhu Zuoyu Tian Yadong Liu Cong Zhang Chia-wen Lo SSL 32 2 0 23 Oct 2022
GAN You Hear Me? Reclaiming Unconditional Speech Synthesis from Diffusion Models Matthew Baas Herman Kamper DiffM 37 8 0 11 Oct 2022