Title
Analyzing Mitigation Strategies for Catastrophic Forgetting in End-to-End Training of Spoken Language Models Chi-Yuan Hsiao Ke-Han Lu Kai-Wei Chang Chih-Kai Yang Wei-Chih Chen Hung-yi Lee CLL MoMe 112 0 0 23 May 2025
CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training Zhihao Du Changfeng Gao Yuxuan Wang Fan Yu Tianyu Zhao ... Mengzhe Chen Yafeng Chen Shiliang Zhang Wen Wang Jieping Ye AuLLM 75 0 0 23 May 2025
Large Language Models Implicitly Learn to See and Hear Just By Reading Prateek Verma Mert Pilanci 93 0 0 20 May 2025
SepALM: Audio Language Models Are Error Correctors for Robust Speech Separation Zhaoxi Mu Xinyu Yang Gang Wang AuLLM KELM VLM 105 0 0 06 May 2025
EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Prompting Guanrou Yang Chen Yang Qian Chen Ziyang Ma Wenxi Chen ... Fan Yu Zhihao Du Zhifu Gao Shiliang Zhang Xie Chen AuLLM 109 0 0 17 Apr 2025
TASTE: Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling Liang-Hsuan Tseng Yi-Chang Chen Kuan-Yi Lee Da-shan Shiu Hung-yi Lee AuLLM 94 0 0 09 Apr 2025
F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimization Xiaohui Sun Ruitong Xiao Jianye Mo Bowen Wu Qun Yu Baoxun Wang 71 2 0 03 Apr 2025
SupertonicTTS: Towards Highly Scalable and Efficient Text-to-Speech System Hyeongju Kim Jinhyeok Yang Yechan Yu Seunghun Ji Jacob Morton Frederik Bous Joon Byun Juheon Lee 87 0 0 29 Mar 2025
Universal Speech Token Learning via Low-Bitrate Neural Codec and Pretrained Representations Xue Jiang Xiulian Peng Yuan Zhang Yan Lu SSL 106 1 0 15 Mar 2025
Slamming: Training a Speech Language Model on One GPU in a Day Gallil Maimon Avishai Elmakies Yossi Adi 56 3 0 19 Feb 2025
Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free Guidance Shehzeen Samarah Hussain Paarth Neekhara Xuesong Yang Edresson Casanova Subhankar Ghosh Mikyas T. Desta Roy Fejgin Rafael Valle Jason Chun Lok Li 88 4 0 07 Feb 2025
AudioMiXR: Spatial Audio Object Manipulation with 6DoF for Sound Design in Augmented Reality Brandon Woodard Margarita Geleta Joseph J. LaViola Jr. Andrea Fanelli Rhonda Wilson 87 4 0 05 Feb 2025
Everyone-Can-Sing: Zero-Shot Singing Voice Synthesis and Conversion with Speech Reference Shuqi Dai Yunyun Wang Roger B. Dannenberg Zeyu Jin DiffM 88 0 0 23 Jan 2025
Generative Data Augmentation Challenge: Zero-Shot Speech Synthesis for Personalized Speech Enhancement Jae-Sung Bae Anastasia Kuznetsova Dinesh Manocha John Hershey Trausti Kristjansson Minje Kim 91 0 0 23 Jan 2025
SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words Junyi Ao Yuancheng Wang Xiaohai Tian Dekun Chen Jing Zhang Lu Lu Yansen Wang Haizhou Li Zhikai Wu AuLLM 111 21 0 17 Jan 2025
Towards Lightweight and Stable Zero-shot TTS with Self-distilled Representation Disentanglement Qianniu Chen Xiaoyang Hao Yangqiu Song Yunxing Liu Li Lu 60 0 0 15 Jan 2025
AccentBox: Towards High-Fidelity Zero-Shot Accent Generation Jinzuomu Zhong Korin Richmond Zhiba Su Siqi Sun 93 6 0 10 Jan 2025
ZSVC: Zero-shot Style Voice Conversion with Disentangled Latent Diffusion Models and Adversarial Training Xinfa Zhu Lei He Yujia Xiao Xi Wang Xu Tan Sheng Zhao Lei Xie DiffM 59 2 0 08 Jan 2025
SSR-Speech: Towards Stable, Safe and Robust Zero-shot Text-based Speech Editing and Synthesis Helin Wang Meng Yu Jiarui Hai Chen Chen Yuchen Hu Rilin Chen Najim Dehak Dong Yu 99 6 0 03 Jan 2025
In-Context Learning with Iterative Demonstration Selection Chengwei Qin Aston Zhang Chong Chen Anirudh Dagar Wenming Ye LRM 121 48 0 31 Dec 2024
SongEditor: Adapting Zero-Shot Song Generation Language Model as a Multi-Task Editor Chenyu Yang Shuai Wang Hangting Chen Jianwei Yu Wei Tan Rongzhi Gu Yongjun Xu Yizhi Zhou Haina Zhu Haoyang Li KELM 322 1 0 18 Dec 2024
Fast and High-Quality Auto-Regressive Speech Synthesis via Speculative Decoding Bohan Li Hankun Wang Situo Zhang Yiwei Guo Kai Yu 68 8 0 29 Oct 2024
ARLON: Boosting Diffusion Transformers with Autoregressive Models for Long Video Generation Zongyi Li Shujie Hu Shujie Liu Long Zhou Jeongsoo Choi Lingwei Meng Xun Guo Jiajian Li H. Ling Furu Wei VGen DiffM 105 5 0 27 Oct 2024
Continuous Speech Tokenizer in Text To Speech Yixing Li Ruobing Xie Xingwu Sun Yu Cheng Zhanhui Kang AuLLM CLL 76 2 0 22 Oct 2024
LSCodec: Low-Bitrate and Speaker-Decoupled Discrete Speech Codec Yiwei Guo Zhihan Li Chenpeng Du Hankun Wang Xie Chen Kai Yu 52 2 0 21 Oct 2024
Ichigo: Mixed-Modal Early-Fusion Realtime Voice Assistant Alan Dao Dinh Bach Vu Huy Hoang Ha AuLLM VLM 90 5 0 20 Oct 2024
SF-Speech: Straightened Flow for Zero-Shot Voice Clone Xuyuan Li Zengqiang Shang Hua Hua Peiyang Shi Chen Yang Li Wang Pengyuan Zhang 100 2 0 16 Oct 2024
F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching Yushen Chen Zhikang Niu Ziyang Ma Keqi Deng Chunhui Wang Jian Zhao Kai Yu Xie Chen 78 73 0 09 Oct 2024
Audio-Agent: Leveraging LLMs For Audio Generation, Editing and Composition Zixuan Wang Chi-Keung Tang Chi-Keung Tang DiffM VGen LLMAG 69 4 0 04 Oct 2024
Recent Advances in Speech Language Models: A Survey Wenqian Cui Dianzhi Yu Xiaoqi Jiao Ziqiao Meng Guangyan Zhang Qichao Wang Yiwen Guo Irwin King AuLLM 84 22 0 01 Oct 2024
MIO: A Foundation Model on Multimodal Tokens Zekun Wang King Zhu Chunpu Xu Wangchunshu Zhou Jiaheng Liu ... Yuanxing Zhang Ge Zhang Ke Xu Jie Fu Wenhao Huang MLLM AuLLM 83 11 0 26 Sep 2024
Emotional Dimension Control in Language Model-Based Text-to-Speech: Spanning a Broad Spectrum of Human Emotions Kun Zhou You Zhang Shengkui Zhao Hao Wang Zexu Pan ... Chongjia Ni Yukun Ma Trung Hieu Nguyen J. Yip Bin Ma 83 6 0 25 Sep 2024
SpoofCeleb: Speech Deepfake Detection and SASV In The Wild Jee-weon Jung Yihan Wu Xin Wang Ji-Hoon Kim Soumi Maiti ... Joon Son Chung Wangyou Zhang Seyun Um Shinnosuke Takamichi Shinji Watanabe 80 2 0 18 Sep 2024
Improving Robustness of Diffusion-Based Zero-Shot Speech Synthesis via Stable Formant Generation C. Han Seokgi Lee Gyuhyeon Nam Gyeongsu Chae DiffM 382 0 0 14 Sep 2024
SelectTTS: Synthesizing Anyone's Voice via Discrete Unit-Based Frame Selection Ismail Rasim Ulgen Shreeram Suresh Chandra Junchen Lu Berrak Sisman 364 1 0 30 Aug 2024
WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling Shengpeng Ji Ziyue Jiang Xize Cheng Yifu Chen Minghui Fang ... Rongjie Huang Yidi Jiang Qian Chen Zhou Zhao Zhou Zhao VLM 75 40 0 29 Aug 2024
VQ-CTAP: Cross-Modal Fine-Grained Sequence Representation Learning for Speech Processing Chunyu Qiang Wang Geng Yi Zhao Ruibo Fu Tao Wang ... Chen Zhang Hao Che L. Wang Jianwu Dang J. Tao AI4TS 60 0 0 11 Aug 2024
dMel: Speech Tokenization made Simple Richard He Bai Tatiana Likhomanenko Ruixiang Zhang Zijin Gu Zakaria Aldeneh Navdeep Jaitly 66 6 0 22 Jul 2024
Autoregressive Speech Synthesis without Vector Quantization Lingwei Meng Long Zhou Shujie Liu Sanyuan Chen Bing Han ... Jinyu Li Sheng Zhao Xixin Wu Helen M. Meng Furu Wei 91 40 0 11 Jul 2024
VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers Sanyuan Chen Shujie Liu Long Zhou Yanqing Liu Xu Tan Jinyu Li Sheng Zhao Yao Qian Furu Wei VLM 61 76 0 08 Jun 2024
VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling Zeyue Tian Zhaoyang Liu Ruibin Yuan Jiahao Pan Xiaoqiang Huang Xu Tan Xu Tan Qifeng Chen Yu Guo VGen 167 16 0 06 Jun 2024
MAD Speech: Measures of Acoustic Diversity of Speech Matthieu Futeral A. Agostinelli Marco Tagliasacchi Neil Zeghidour Eugene Kharitonov 78 1 0 16 Apr 2024
Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt Yongqi Wang Ruofan Hu Rongjie Huang Zhiqing Hong Ruiqi Li Wenrui Liu Fuming You Tao Jin Zhou Zhao 54 12 0 18 Mar 2024
NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models Zeqian Ju Yuancheng Wang Kai Shen Xu Tan Detai Xin ... Shikun Zhang Jiang Bian Lei He Jinyu Li Sheng Zhao DiffM 61 164 0 05 Mar 2024
PITCH: AI-assisted Tagging of Deepfake Audio Calls using Challenge-Response Govind Mittal Arthur Jakobsson Kelly O. Marshall Chinmay Hegde Nasir Memon 65 0 0 28 Feb 2024
BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data Mateusz Lajszczak Guillermo Cámbara Yang Li Fatih Beyhan Arent van Korlaar ... Bartosz Putrycz Soledad López Gambino Kayeon Yoo Elena Sokolova Thomas Drugman LM&MA 50 84 0 12 Feb 2024
CREMA: Generalizable and Efficient Video-Language Reasoning via Multimodal Modular Fusion Shoubin Yu Jaehong Yoon Mohit Bansal 102 5 0 08 Feb 2024
ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reordering Ya-Zhen Song Zhuo Chen Xiaofei Wang Ziyang Ma Xie Chen AuLLM 89 42 0 14 Jan 2024
Audiobox: Unified Audio Generation with Natural Language Prompts Apoorv Vyas Bowen Shi Matt Le Andros Tjandra Yi-Chiao Wu ... Chris Summers Carleigh Wood Joshua Lane Mary Williamson Wei-Ning Hsu 72 82 0 25 Dec 2023
SD-HuBERT: Sentence-Level Self-Distillation Induces Syllabic Organization in HuBERT Cheol Jun Cho Abdelrahman Mohamed Shang-Wen Li Alan W. Black Gopala K. Anumanchipalli 64 8 0 16 Oct 2023