SoundStorm: Efficient Parallel Audio Generation

16 May 2023

Papers citing "SoundStorm: Efficient Parallel Audio Generation"

50 / 74 papers shown

Title
SepALM: Audio Language Models Are Error Correctors for Robust Speech Separation Zhaoxi Mu Xinyu Yang Gang Wang AuLLM KELM VLM 55 0 0 06 May 2025
Voice Cloning: Comprehensive Survey Hussam Azzuni Abdulmotaleb El Saddik VLM 39 0 0 01 May 2025
ALMTokenizer: A Low-bitrate and Semantic-rich Audio Codec Tokenizer for Audio Language Modeling Dongchao Yang Songxiang Liu Haohan Guo Jiankun Zhao Yuanyuan Wang ... Xubo Liu Xueyuan Chen Xu Tan Xixin Wu H. Meng 105 0 0 14 Apr 2025
On The Landscape of Spoken Language Models: A Comprehensive Survey Siddhant Arora Kai-Wei Chang Chung-Ming Chien Yifan Peng Haibin Wu Yossi Adi Emmanuel Dupoux Hung-yi Lee Karen Livescu Shinji Watanabe 49 2 0 11 Apr 2025
LoopGen: Training-Free Loopable Music Generation Davide Marincione Giorgio Strano Donato Crisostomi Roberto Ribuoli Emanuele Rodolà MGen 51 0 0 06 Apr 2025
AudioMiXR: Spatial Audio Object Manipulation with 6DoF for Sound Design in Augmented Reality Brandon Woodard Margarita Geleta Joseph J. LaViola Jr. Andrea Fanelli Rhonda Wilson 55 2 0 05 Feb 2025
Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation Haorui He Zengqiang Shang Chaoren Wang Xuyuan Li Yicheng Gu ... Peiyang Shi Y. Wang Kai Chen Pengyuan Zhang Z. Wu AuLLM 54 4 0 28 Jan 2025
SoundSpring: Loss-Resilient Audio Transceiver with Dual-Functional Masked Language Modeling Shengshi Yao Jincheng Dai Xiaoqi Qin Sixian Wang Siye Wang K. Niu Ping Zhang 31 0 0 22 Jan 2025
ZSVC: Zero-shot Style Voice Conversion with Disentangled Latent Diffusion Models and Adversarial Training Xinfa Zhu Lei He Yujia Xiao Xi Wang Xu Tan Sheng Zhao Lei Xie DiffM 38 0 0 08 Jan 2025
SSR-Speech: Towards Stable, Safe and Robust Zero-shot Text-based Speech Editing and Synthesis Helin Wang Meng Yu Jiarui Hai Chen Chen Yuchen Hu Rilin Chen Najim Dehak Dong Yu 82 3 0 03 Jan 2025
SOLAMI: Social Vision-Language-Action Modeling for Immersive Interaction with 3D Autonomous Characters Jianping Jiang Weiye Xiao Zhengyu Lin H. Zhang Tianxiang Ren Yang Gao Zhiqian Lin Zhongang Cai Lei Yang Ziwei Liu 86 3 0 29 Nov 2024
Accelerating Codec-based Speech Synthesis with Multi-Token Prediction and Speculative Decoding Tan Dat Nguyen Ji-Hoon Kim Jeongsoo Choi Shukjae Choi Jinseok Park Younglo Lee Joon Son Chung 26 0 0 17 Oct 2024
Code Drift: Towards Idempotent Neural Audio Codecs P. O'Reilly Prem Seetharaman Jiaqi Su Zeyu Jin Bryan Pardo 105 0 0 14 Oct 2024
CTC-GMM: CTC guided modality matching for fast and accurate streaming speech translation Rui Zhao Jinyu Li Ruchao Fan Matt Post 36 1 0 07 Oct 2024
MDSGen: Fast and Efficient Masked Diffusion Temporal-Aware Transformers for Open-Domain Sound Generation T. Pham Tri Ton Chang D. Yoo 36 3 0 03 Oct 2024
From Vision to Audio and Beyond: A Unified Model for Audio-Visual Representation and Generation Kun Su Xiulong Liu Eli Shlizerman VGen 30 6 0 27 Sep 2024
Codec-SUPERB @ SLT 2024: A lightweight benchmark for neural audio codec models Haibin Wu Xuanjun Chen Yi-Cheng Lin Kaiwei Chang Jiawei Du ... Yi-Chiao Wu Xu Tan James Glass Shinji Watanabe Hung-yi Lee 29 6 0 21 Sep 2024
Preference Alignment Improves Language Model-Based TTS Jinchuan Tian Chunlei Zhang Jiatong Shi Hao Zhang Jianwei Yu Shinji Watanabe Dong Yu 30 7 0 19 Sep 2024
Takin: A Cohort of Superior Quality Zero-shot Speech Generation Models Sijing Chen Yuan Feng Laipeng He Tianwei He Wendi He ... Huimin Zhang Xiang Zhang Guangcheng Zhao Hongbin Zhou Pengpeng Zou 30 4 0 18 Sep 2024
SpMis: An Investigation of Synthetic Spoken Misinformation Detection Peizhuo Liu Li Wang Renqiang He Haorui He Lei Wang Huadi Zheng Jie Shi Tong Xiao Zhizheng Wu 32 1 0 17 Sep 2024
Investigating Disentanglement in a Phoneme-level Speech Codec for Prosody Modeling Sotirios Karapiperis Nikolaos Ellinas Alexandra Vioni Junkwang Oh Gunu Jho Inchul Hwang S. Raptis 31 0 0 13 Sep 2024
VMAS: Video-to-Music Generation via Semantic Alignment in Web Music Videos Yan-Bo Lin Yu Tian L. Yang Gedas Bertasius Heng Wang VGen 34 7 0 11 Sep 2024
An End-to-End Approach for Chord-Conditioned Song Generation Shuochen Gao Shun Lei Fan Zhuo Hangyu Liu Feng Liu Boshi Tang Qiaochu Huang Shiyin Kang Zhiyong Wu 26 2 0 10 Sep 2024
Investigating Neural Audio Codecs for Speech Language Model-Based Speech Generation Jiaqi Li Dongmei Wang Xiaofei Wang Yao Qian Long Zhou ... Junkun Chen Sheng Zhao Jinyu Li Zhizheng Wu Michael Zeng AuLLM 30 2 0 06 Sep 2024
MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformer Yuancheng Wang Haoyue Zhan Liwei Liu Ruihong Zeng Haotian Guo Jiachen Zheng Qiang Zhang Shunsi Zhang Shunsi Zhang Zhizheng Wu 34 38 0 01 Sep 2024
SSDM: Scalable Speech Dysfluency Modeling Jiachen Lian Xuanru Zhou Z. Ezzes Jet M J Vonk Brittany Morin D. Baquirin Zachary Mille M. G. Tempini Gopala Anumanchipalli AuLLM 32 1 0 29 Aug 2024
WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling Shengpeng Ji Ziyue Jiang Xize Cheng Yifu Chen Minghui Fang ... Rongjie Huang Yidi Jiang Qian Chen Zhou Zhao Zhou Zhao VLM 52 33 0 29 Aug 2024
Bailing-TTS: Chinese Dialectal Speech Synthesis Towards Human-like Spontaneous Representation Xinhan Di Jiahao Lu Yunming Liang Junjie Zheng Yihua Wang Chaofan Ding ALM 33 1 0 01 Aug 2024
Masked Generative Video-to-Audio Transformers with Enhanced Synchronicity Santiago Pascual Chunghsin Yeh Ioannis Tsiamas Joan Serra DiffM VGen 39 15 0 15 Jul 2024
Autoregressive Speech Synthesis without Vector Quantization Lingwei Meng Long Zhou Shujie Liu Sanyuan Chen Bing Han ... Jinyu Li Sheng Zhao Xixin Wu Helen Meng Furu Wei 46 30 0 11 Jul 2024
Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generation Haorui He Zengqiang Shang Chaoren Wang Xuyuan Li Yicheng Gu ... Peiyang Shi Yuancheng Wang Kai Chen Pengyuan Zhang Zhizheng Wu 33 34 0 07 Jul 2024
Codec-ASR: Training Performant Automatic Speech Recognition Systems with Discrete Speech Representations Kunal Dhawan Nithin Rao Koluguri Ante Jukić Ryan Langman Jagadeesh Balam Boris Ginsburg 39 1 0 03 Jul 2024
MMM: Multi-Layer Multi-Residual Multi-Stream Discrete Speech Representation from Self-supervised Learning Model Jiatong Shi Xutai Ma Hirofumi Inaguma Anna Y. Sun Shinji Watanabe 50 7 0 14 Jun 2024
An Investigation of Noise Robustness for Flow-Matching-Based Zero-Shot TTS Xiaofei Wang Sefik Emre Eskimez Manthan Thakker Hemin Yang Zirun Zhu ... Yufei Xia Jinzhu Li Sheng Zhao Jinyu Li Naoyuki Kanda 40 3 0 09 Jun 2024
Exploring the Benefits of Tokenization of Discrete Acoustic Units Avihu Dekel Raul Fernandez 36 2 0 08 Jun 2024
VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers Sanyuan Chen Shujie Liu Long Zhou Yanqing Liu Xu Tan Jinyu Li Sheng Zhao Yao Qian Furu Wei VLM 45 64 0 08 Jun 2024
Small-E: Small Language Model with Linear Attention for Efficient Speech Synthesis Théodor Lemerle Nicolas Obin Axel Roebel 29 6 0 06 Jun 2024
Improving Audio Codec-based Zero-Shot Text-to-Speech Synthesis with Multi-Modal Context and Large Language Model Jinlong Xue Yayue Deng Yicheng Han Yingming Gao Ya Li 40 4 0 06 Jun 2024
SimpleSpeech: Towards Simple and Efficient Text-to-Speech with Scalar Latent Transformer Diffusion Models Dongchao Yang Dingdong Wang Haohan Guo Xueyuan Chen Xixin Wu Helen M. Meng 57 25 0 04 Jun 2024
MaskSR: Masked Language Model for Full-band Speech Restoration Xu Li Qirui Wang Xiaoyu Liu 39 8 0 04 Jun 2024
ControlSpeech: Towards Simultaneous Zero-shot Speaker Cloning and Zero-shot Language Style Control With Decoupled Codec Shengpeng Ji Jia-li Zuo Minghui Fang Siqi Zheng Qian Chen ... Ziyue Jiang Hai Huang Xize Cheng Rongjie Huang Zhou Zhao 45 8 0 03 Jun 2024
Zipper: A Multi-Tower Decoder Architecture for Fusing Modalities Vicky Zayats Peter Chen Melissa Ferrari Dirk Padfield AI4CE 30 0 0 29 May 2024
CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversations Leying Zhang Yao Qian Long Zhou Shujie Liu Dongmei Wang ... Yanmin Qian Jinyu Li Lei He Sheng Zhao Michael Zeng 26 1 0 10 Apr 2024
CLaM-TTS: Improving Neural Codec Language Model for Zero-Shot Text-to-Speech Jaehyeon Kim Keon Lee Seungjun Chung Jaewoong Cho 65 39 0 03 Apr 2024
PromptCodec: High-Fidelity Neural Speech Codec using Disentangled Representation Learning based Adaptive Feature-aware Prompt Encoders Yu Pan Lei Ma Jianjun Zhao 32 4 0 03 Apr 2024
BAMM: Bidirectional Autoregressive Motion Model Ekkasit Pinyoanuntapong Muhammad Usama Saleem Pu Wang Minwoo Lee Srijan Das C. L. P. Chen VGen 27 22 0 28 Mar 2024
VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild Puyuan Peng Po-Yao (Bernie) Huang Daniel Li Abdelrahman Mohamed David F. Harwath 66 57 0 25 Mar 2024
UTDUSS: UTokyo-SaruLab System for Interspeech2024 Speech Processing Using Discrete Speech Unit Challenge Wataru Nakata Kazuki Yamauchi Dong Yang Hiroaki Hyodo Yuki Saito 22 0 0 20 Mar 2024
NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models Zeqian Ju Yuancheng Wang Kai Shen Xu Tan Detai Xin ... Shikun Zhang Jiang Bian Lei He Jinyu Li Sheng Zhao DiffM 38 143 0 05 Mar 2024
VoxGenesis: Unsupervised Discovery of Latent Speaker Manifold for Speech Synthesis Wei-wei Lin Chenhang He Man-Wai Mak Jiachen Lian Kong Aik Lee DiffM 39 0 0 01 Mar 2024