UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocoding

13 June 2023

Chenpeng Du

Yiwei Guo

Feiyu Shen

Zhijun Liu

Zheng Liang

Xie Chen

Shuai Wang

Papers citing "UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocoding"

38 / 38 papers shown

Title
OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching Hieu-Nghia Huynh-Nguyen Ngoc Son Nguyen Huynh Nguyen Dang Thieu Vo Truong-Son Hy Van Nguyen 9 0 0 19 May 2025
SongEditor: Adapting Zero-Shot Song Generation Language Model as a Multi-Task Editor Chenyu Yang Shuai Wang Hangting Chen Jianwei Yu Wei Tan Rongzhi Gu Yongjun Xu Yizhi Zhou Haina Zhu Yiming Li KELM 200 1 0 18 Dec 2024
Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook Florinel-Alin Croitoru Andrei Iulian Hiji Vlad Hondru Nicolae-Cătălin Ristea Paul Irofti Marius Popescu Cristian Rusu Radu Tudor Ionescu Fahad Shahbaz Khan Mubarak Shah 89 3 0 29 Nov 2024
Fast and High-Quality Auto-Regressive Speech Synthesis via Speculative Decoding Bohan Li Hankun Wang Situo Zhang Yiwei Guo Kai Yu 42 5 0 29 Oct 2024
LSCodec: Low-Bitrate and Speaker-Decoupled Discrete Speech Codec Yiwei Guo Zhihan Li Chenpeng Du Hankun Wang Xie Chen Kai Yu 36 1 0 21 Oct 2024
Takin-ADA: Emotion Controllable Audio-Driven Animation with Canonical and Landmark Loss Optimization Bin Lin Yanzhen Yu Jianhao Ye Ruitao Lv Yuqing Yang Ruoye Xie Pan Yu Hongbin Zhou VGen 35 1 0 18 Oct 2024
Recent Advances in Speech Language Models: A Survey Wenqian Cui Dianzhi Yu Xiaoqi Jiao Ziqiao Meng Guangyan Zhang Qichao Wang Yiwen Guo Irwin King AuLLM 61 17 0 01 Oct 2024
E1 TTS: Simple and Fast Non-Autoregressive TTS Zhijun Liu Shuai Wang Pengcheng Zhu Mengxiao Bi Haizhou Li VLM DiffM 38 3 0 14 Sep 2024
Estimating the Completeness of Discrete Speech Units Sung-Lin Yeh Hao Tang 36 1 0 09 Sep 2024
SongCreator: Lyrics-based Universal Song Generation Shun Lei Yixuan Zhou Boshi Tang Max W. Y. Lam Feng Liu Hangyu Liu Jingcheng Wu Shiyin Kang Zhiyong Wu Helen Meng 52 5 0 09 Sep 2024
vec2wav 2.0: Advancing Voice Conversion via Discrete Token Vocoders Yiwei Guo Zhihan Li Junjie Li Chenpeng Du Hankun Wang Shuai Wang Xie Chen Kai Yu 35 0 0 03 Sep 2024
MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformer Yuancheng Wang Haoyue Zhan Liwei Liu Ruihong Zeng Haotian Guo Jiachen Zheng Qiang Zhang Shunsi Zhang Shunsi Zhang Zhizheng Wu 36 42 0 01 Sep 2024
SelectTTS: Synthesizing Anyone's Voice via Discrete Unit-Based Frame Selection Ismail Rasim Ulgen Shreeram Suresh Chandra Junchen Lu Berrak Sisman 192 0 0 30 Aug 2024
Improvement Speaker Similarity for Zero-Shot Any-to-Any Voice Conversion of Whispered and Regular Speech Anastasia Avdeeva Aleksei Gusev 24 0 0 21 Aug 2024
Language Model Can Listen While Speaking Ziyang Ma Yakun Song Chenpeng Du Jian Cong Zhuo Chen Yuping Wang Yansen Wang Xie Chen AuLLM 47 23 0 05 Aug 2024
Overview of Speaker Modeling and Its Applications: From the Lens of Deep Speaker Representation Learning Shuai Wang Zheng-Shou Chen Kong Aik Lee Yan-min Qian Haizhou Li 39 4 0 21 Jul 2024
On the Effectiveness of Acoustic BPE in Decoder-Only TTS Bohan Li Feiyu Shen Yiwei Guo Shuai Wang Xie Chen Kai Yu 39 2 0 04 Jul 2024
Autoregressive Diffusion Transformer for Text-to-Speech Synthesis Zhijun Liu Shuai Wang Sho Inoue Qibing Bai Haizhou Li DiffM 50 15 0 08 Jun 2024
VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers Sanyuan Chen Shujie Liu Long Zhou Yanqing Liu Xu Tan Jinyu Li Sheng Zhao Yao Qian Furu Wei VLM 47 67 0 08 Jun 2024
Seed-TTS: A Family of High-Quality Versatile Speech Generation Models Philip Anastassiou Jiawei Chen J. Chen Yuanzhe Chen Zhuo Chen ... Wenjie Zhang Wenjie Qu Zilin Zhao Dejian Zhong Xiaobin Zhuang 49 80 0 04 Jun 2024
AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encoding Tao Liu Feilong Chen Shuai Fan Chenpeng Du Qi Chen Xie Chen Kai Yu DiffM PINN 36 25 0 06 May 2024
Attention-Constrained Inference for Robust Decoder-Only Text-to-Speech Hankun Wang Chenpeng Du Yiwei Guo Shuai Wang Xie Chen Kai Yu 40 1 0 30 Apr 2024
The X-LANCE Technical Report for Interspeech 2024 Speech Processing Using Discrete Speech Unit Challenge Yiwei Guo Chenrun Wang Yifan Yang Hankun Wang Ziyang Ma ... Hanzheng Li Shuai Fan Hui Zhang Xie Chen Kai Yu 38 1 0 09 Apr 2024
VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild Puyuan Peng Po-Yao (Bernie) Huang Daniel Li Abdelrahman Mohamed David Harwath 74 59 0 25 Mar 2024
NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models Zeqian Ju Yuancheng Wang Kai Shen Xu Tan Detai Xin ... Shikun Zhang Jiang Bian Lei He Jinyu Li Sheng Zhao DiffM 49 145 0 05 Mar 2024
MobileSpeech: A Fast and High-Fidelity Framework for Mobile Zero-Shot Text-to-Speech Shengpeng Ji Ziyue Jiang Hanting Wang Jia-li Zuo Zhou Zhao 40 10 0 14 Feb 2024
VALL-T: Decoder-Only Generative Transducer for Robust and Decoding-Controllable Text-to-Speech Chenpeng Du Yiwei Guo Hankun Wang Yifan Yang Zhikang Niu Shuai Wang Hui Zhang Xie Chen Kai Yu VLM 35 25 0 25 Jan 2024
MM-TTS: Multi-modal Prompt based Style Transfer for Expressive Text-to-Speech Synthesis Wenhao Guan Yishuang Li Tao Li Hukai Huang Feng Wang Jiayan Lin Lingyan Huang Lin Li Q. Hong 36 8 0 17 Dec 2023
Efficient and Scalable Graph Generation through Iterative Local Expansion Andreas Bergmeister Karolis Martinkus Nathanael Perraudin Roger Wattenhofer 25 12 0 14 Dec 2023
SEF-VC: Speaker Embedding Free Zero-Shot Voice Conversion with Cross Attention Junjie Li Yiwei Guo Xie Chen Kai Yu 45 13 0 14 Dec 2023
Acoustic BPE for Speech Generation with Discrete Tokens Feiyu Shen Yiwei Guo Chenpeng Du Xie Chen Kai Yu 20 9 0 23 Oct 2023
Towards Universal Speech Discrete Tokens: A Case Study for ASR and TTS Yifan Yang Feiyu Shen Chenpeng Du Ziyang Ma K. Yu Daniel Povey Xie Chen 35 24 0 14 Sep 2023
VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matching Yiwei Guo Chenpeng Du Ziyang Ma Xie Chen K. Yu DiffM 35 37 0 10 Sep 2023
MuLanTTS: The Microsoft Speech Synthesis System for Blizzard Challenge 2023 Zhihang Xu Shaofei Zhang Xi Wang Jiajun Zhang Wenning Wei Lei He Sheng Zhao 23 2 0 06 Sep 2023
Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers Chengyi Wang Sanyuan Chen Yu-Huan Wu Zi-Hua Zhang Long Zhou ... Huaming Wang Jinyu Li Lei He Sheng Zhao Furu Wei 48 648 0 05 Jan 2023
YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone Edresson Casanova Julian Weber C. Shulby Arnaldo Cândido Júnior Eren Golge M. Ponti 185 379 0 04 Dec 2021
EdiTTS: Score-based Editing for Controllable Text-to-Speech Jaesung Tae Hyeongju Kim Taesu Kim DiffM 173 39 0 06 Oct 2021
Generative Spoken Language Modeling from Raw Audio Kushal Lakhotia Evgeny Kharitonov Wei-Ning Hsu Yossi Adi Adam Polyak ... Tu Nguyen Jade Copet Alexei Baevski A. Mohamed Emmanuel Dupoux AuLLM 191 337 0 01 Feb 2021