SoundStream: An End-to-End Neural Audio Codec

7 July 2021

Papers citing "SoundStream: An End-to-End Neural Audio Codec"

50 / 109 papers shown

Title
PolySpeech: Exploring Unified Multitask Speech Models for Competitiveness with Single-task Models Runyan Yang Huibao Yang Xiqing Zhang Tiantian Ye Ying Liu Yingying Gao Shilei Zhang Chao Deng Junlan Feng 34 0 0 12 Jun 2024
AudioMarkBench: Benchmarking Robustness of Audio Watermarking Hongbin Liu Moyang Guo Zhengyuan Jiang Lun Wang Neil Zhenqiang Gong 39 6 0 11 Jun 2024
VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling Zeyue Tian Zhaoyang Liu Ruibin Yuan Jiahao Pan Xiaoqiang Huang Xu Tan Xu Tan Qifeng Chen Y. Guo VGen 102 16 0 06 Jun 2024
LiveSpeech: Low-Latency Zero-shot Text-to-Speech via Autoregressive Modeling of Audio Discrete Codes Trung D. Q. Dang David Aponte Dung Tran K. Koishida 38 3 0 05 Jun 2024
Discrete Multimodal Transformers with a Pretrained Large Language Model for Mixed-Supervision Speech Processing V. Trinh Rosy Southwell Yiwen Guan Xinlu He Zhiyong Wang Jacob Whitehill OffRL 36 2 0 04 Jun 2024
MAD Speech: Measures of Acoustic Diversity of Speech Matthieu Futeral A. Agostinelli Marco Tagliasacchi Neil Zeghidour Eugene Kharitonov 51 1 0 16 Apr 2024
VoiceShop: A Unified Speech-to-Speech Framework for Identity-Preserving Zero-Shot Voice Editing Philip Anastassiou Zhenyu Tang Kainan Peng Dongya Jia Jiaxin Li Ming Tu Yuping Wang Yuxuan Wang Mingbo Ma 42 4 0 10 Apr 2024
HyperVQ: MLR-based Vector Quantization in Hyperbolic Space Nabarun Goswami Yusuke Mukuta Tatsuya Harada 40 3 0 18 Mar 2024
Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations Guan-Ting Lin Cheng-Han Chiang Hung-yi Lee 34 22 0 20 Feb 2024
Proactive Detection of Voice Cloning with Localized Watermarking Robin San Roman Pierre Fernandez Alexandre Défossez Teddy Furon Tuan Tran Hady ElSahar 49 40 0 30 Jan 2024
StreamVoice: Streamable Context-Aware Language Modeling for Real-time Zero-Shot Voice Conversion Zhichao Wang Yuan-Jui Chen Xinsheng Wang Lei Xie Yuping Wang 22 6 0 19 Jan 2024
Learning to Act without Actions Dominik Schmidt Minqi Jiang OffRL 26 30 0 17 Dec 2023
4M: Massively Multimodal Masked Modeling David Mizrahi Roman Bachmann Ouguzhan Fatih Kar Teresa Yeo Mingfei Gao Afshin Dehghan Amir Zamir MLLM 44 63 0 11 Dec 2023
MoMask: Generative Masked Modeling of 3D Human Motions Chuan Guo Yuxuan Mu Muhammad Gohar Javed Sen Wang Li Cheng VGen 25 120 0 29 Nov 2023
Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models Yunfei Chu Jin Xu Xiaohuan Zhou Qian Yang Shiliang Zhang Zhijie Yan Chang Zhou Jingren Zhou AuLLM 33 267 0 14 Nov 2023
InstrumentGen: Generating Sample-Based Musical Instruments From Text S. Nercessian Johannes Imort 27 2 0 07 Nov 2023
Content-based Controls For Music Large Language Modeling Liwei Lin Gus Xia Junyan Jiang Yixiao Zhang 18 14 0 26 Oct 2023
AdaMesh: Personalized Facial Expressions and Head Poses for Adaptive Speech-Driven 3D Facial Animation Liyang Chen Weihong Bao Shunwei Lei Boshi Tang Zhiyong Wu Shiyin Kang Haozhi Huang Helen M. Meng 40 1 0 11 Oct 2023
VaSAB: The variable size adaptive information bottleneck for disentanglement on speech and singing voice F. Bous Axel Roebel 16 0 0 05 Oct 2023
Discrete Audio Representation as an Alternative to Mel-Spectrograms for Speaker and Speech Recognition Krishna C. Puvvada Nithin Rao Koluguri Kunal Dhawan Jagadeesh Balam Boris Ginsburg 24 12 0 19 Sep 2023
FunCodec: A Fundamental, Reproducible and Integrable Open-source Toolkit for Neural Speech Codec Zhihao Du Shiliang Zhang Kai Hu Siqi Zheng 34 54 0 14 Sep 2023
Self-Supervised Disentanglement of Harmonic and Rhythmic Features in Music Audio Signals Yiming Wu CoGe DRL 29 0 0 06 Sep 2023
AudioFormer: Audio Transformer learns audio feature representations from discrete acoustic codes Zhaohui Li Haitao Wang Xinghua Jiang 40 1 0 14 Aug 2023
AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining Haohe Liu Yiitan Yuan Xubo Liu Xinhao Mei Qiuqiang Kong Qiao Tian Yuping Wang Wenwu Wang Yuxuan Wang Mark D. Plumbley DiffM 25 222 0 10 Aug 2023
JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Models Peike Li Bo-Yu Chen Yao Yao Yikai Wang Allen Wang Alex Jinpeng Wang MGen VLM DiffM 60 37 0 09 Aug 2023
Minimally-Supervised Speech Synthesis with Conditional Diffusion Model and Language Model: A Comparative Study of Semantic Coding Chunyu Qiang Hao Li Hao Ni He Qu Ruibo Fu Tao Wang Longbiao Wang J. Dang DiffM 30 8 0 28 Jul 2023
Incrementally-Computable Neural Networks: Efficient Inference for Dynamic Inputs Or Sharir Anima Anandkumar 27 0 0 27 Jul 2023
SoundStorm: Efficient Parallel Audio Generation Zalan Borsos Matthew Sharifi Damien Vincent Eugene Kharitonov Neil Zeghidour Marco Tagliasacchi 23 97 0 16 May 2023
Leveraging Neural Representations for Audio Manipulation Scott H. Hawley C. Steinmetz 33 2 0 10 Apr 2023
Native Multi-Band Audio Coding within Hyper-Autoencoded Reconstruction Propagation Networks Darius Petermann Inseon Jang Minje Kim 16 1 0 14 Mar 2023
FoundationTTS: Text-to-Speech for ASR Customization with Generative Language Model Rui Xue Yanqing Liu Lei He Xuejiao Tan Linquan Liu Ed Lin Sheng Zhao 26 7 0 06 Mar 2023
Hypernetworks build Implicit Neural Representations of Sounds Filip Szatkowski Karol J. Piczak Przemtslaw Spurek Jacek Tabor Tomasz Trzciñski 24 11 0 09 Feb 2023
InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Prompt Dongchao Yang Songxiang Liu Rongjie Huang Chao Weng H. Meng DiffM VLM 31 85 0 31 Jan 2023
SingSong: Generating musical accompaniments from singing Chris Donahue Antoine Caillon Adam Roberts Ethan Manilow P. Esling ... Mauro Verzetti Ian Simon Olivier Pietquin Neil Zeghidour Jesse Engel 32 52 0 30 Jan 2023
Learning from Mistakes: Self-Regularizing Hierarchical Representations in Point Cloud Semantic Segmentation Elena Camuffo Umberto Michieli Simone Milani 3DPC 24 4 0 26 Jan 2023
Mesostructures: Beyond Spectrogram Loss in Differentiable Time-Frequency Analysis Cyrus Vahidi Han Han Changhong Wang Mathieu Lagrange Gyorgy Fazekas Vincent Lostanlen 16 8 0 24 Jan 2023
Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers Chengyi Wang Sanyuan Chen Yu-Huan Wu Zi-Hua Zhang Long Zhou ... Huaming Wang Jinyu Li Lei He Sheng Zhao Furu Wei 43 641 0 05 Jan 2023
High Quality Audio Coding with MDCTNet G. Davidson M. Vinton P. Ekstrand Cong Zhou Lars Villemoes Lie Lu MedIm 18 8 0 08 Dec 2022
Autovocoder: Fast Waveform Generation from a Learned Speech Representation using Differentiable Digital Signal Processing J. Webber Cassia Valentini-Botinhao Evelyn Williams G. Henter Simon King 11 9 0 13 Nov 2022
Neural Feature Predictor and Discriminative Residual Coding for Low-Bitrate Speech Coding Haici Yang Wootaek Lim Minje Kim 24 9 0 04 Nov 2022
HyperSound: Generating Implicit Neural Representations of Audio Signals with Hypernetworks Filip Szatkowski Karol J. Piczak P. Spurek Jacek Tabor Tomasz Trzciñski 23 12 0 03 Nov 2022
Audio Language Modeling using Perceptually-Guided Discrete Representations Felix Kreuk Yaniv Taigman Adam Polyak Jade Copet Gabriel Synnaeve Alexandre Défossez Yossi Adi 27 4 0 02 Nov 2022
WaveFit: An Iterative and Non-autoregressive Neural Vocoder based on Fixed-Point Iteration Yuma Koizumi Kohei Yatabe Heiga Zen M. Bacchiani DiffM 42 29 0 03 Oct 2022
AudioGen: Textually Guided Audio Generation Felix Kreuk Gabriel Synnaeve Adam Polyak Uriel Singer Alexandre Défossez Jade Copet Devi Parikh Yaniv Taigman Yossi Adi DiffM 27 289 0 30 Sep 2022
AudioLM: a Language Modeling Approach to Audio Generation Zalan Borsos Raphaël Marinier Damien Vincent Eugene Kharitonov Olivier Pietquin ... Dominik Roblek O. Teboul David Grangier Marco Tagliasacchi Neil Zeghidour AuLLM 41 567 0 07 Sep 2022
Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications Deniz Gunduz Zhijin Qin Iñaki Estella Aguerri Harpreet S. Dhillon Zhaohui Yang Aylin Yener Kai‐Kit Wong C. Chae 27 432 0 19 Jul 2022
Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Models Alex Lamb Riashat Islam Yonathan Efroni Aniket Didolkar Dipendra Kumar Misra Dylan J. Foster Lekan Molu Rajan Chari A. Krishnamurthy John Langford 41 24 0 17 Jul 2022
NESC: Robust Neural End-2-End Speech Coding with GANs N. Pia Kishan Gupta Srikanth Korse M. Multrus Guillaume Fuchs 33 15 0 07 Jul 2022
Cross-Scale Vector Quantization for Scalable Neural Speech Coding Xue Jiang Xiulian Peng Huaying Xue Yuan Zhang Yan Lu MQ 39 9 0 07 Jul 2022
Lossy Compression with Gaussian Diffusion Lucas Theis Tim Salimans Matthew D. Hoffman Fabian Mentzer DiffM 28 77 0 17 Jun 2022