Title
Towards General-Purpose Text-Instruction-Guided Voice Conversion Chun-Yi Kuan Chen An Li Tsung-Yuan Hsu T. Lin Ho-Lam Chung Kai-Wei Chang Shuo-yiin Chang Hung-yi Lee 20 5 0 25 Sep 2023
Speaker anonymization using neural audio codec language models Michele Panariello Francesco Nespoli Massimiliano Todisco Nicholas W. D. Evans 22 17 0 25 Sep 2023
AutoPrep: An Automatic Preprocessing Framework for In-the-Wild Speech Data Jianwei Yu Hangting Chen Yanyao Bian Xiang Li Yimin Luo Jinchuan Tian Mengyang Liu Jiayi Jiang Shuai Wang VLM 15 12 0 25 Sep 2023
VoiceLDM: Text-to-Speech with Environmental Context Yeong-Won Lee In-won Yeon Juhan Nam Joon Son Chung VLM DiffM 22 10 0 24 Sep 2023
Improving Language Model-Based Zero-Shot Text-to-Speech Synthesis with Multi-Scale Acoustic Prompts Shunwei Lei Yixuan Zhou Liyang Chen Dan Luo Zhiyong Wu ... Shiyin Kang Tao Jiang Yahui Zhou Yuxing Han Helen M. Meng VLM 38 2 0 21 Sep 2023
Speak While You Think: Streaming Speech Synthesis During Text Generation Avihu Dekel Slava Shechtman Raul Fernandez David Haws Zvi Kons R. Hoory 24 8 0 20 Sep 2023
Towards Joint Modeling of Dialogue Response and Speech Synthesis based on Large Language Model Xinyu Zhou Delong Chen Yudong Chen AuLLM 40 0 0 20 Sep 2023
Discrete Audio Representation as an Alternative to Mel-Spectrograms for Speaker and Speech Recognition Krishna C. Puvvada Nithin Rao Koluguri Kunal Dhawan Jagadeesh Balam Boris Ginsburg 32 12 0 19 Sep 2023
MelodyGLM: Multi-task Pre-training for Symbolic Melody Generation Xinda Wu Zhijie Huang Kejun Zhang Jiaxing Yu Xu Tan Tieyao Zhang Zihao Wang Lingyun Sun 24 5 0 19 Sep 2023
FoleyGen: Visually-Guided Audio Generation Xinhao Mei Varun K. Nagaraja Gaël Le Lan Zhaoheng Ni Ernie Chang Yangyang Shi Vikas Chandra VGen 23 21 0 19 Sep 2023
Do learned speech symbols follow Zipf's law? Shinnosuke Takamichi Hiroki Maeda Joonyong Park Daisuke Saito Hiroshi Saruwatari 17 1 0 18 Sep 2023
Unifying Robustness and Fidelity: A Comprehensive Study of Pretrained Generative Methods for Speech Enhancement in Adverse Conditions Heming Wang Meng Yu Huan Zhang Chunlei Zhang Zhongweiyang Xu Muqiao Yang Yixuan Zhang Dong Yu 37 3 0 16 Sep 2023
Enhance audio generation controllability through representation similarity regularization Yangyang Shi Gaël Le Lan Varun K. Nagaraja Zhaoheng Ni Xinhao Mei Ernie Chang Forrest N. Iandola Yang Liu Vikas Chandra 42 1 0 15 Sep 2023
Speech-to-Speech Translation with Discrete-Unit-Based Style Transfer Yongqiang Wang Jionghao Bai Rongjie Huang Ruiqi Li Zhiqing Hong Zhou Zhao 19 3 0 14 Sep 2023
Masked Generative Modeling with Enhanced Sampling Scheme Daesoo Lee Erlend Aune Sara Malacarne DiffM 13 3 0 14 Sep 2023
Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation tasks Soumi Maiti Yifan Peng Shukjae Choi Jee-weon Jung Xuankai Chang Shinji Watanabe VLM AuLLM 29 57 0 14 Sep 2023
Towards Universal Speech Discrete Tokens: A Case Study for ASR and TTS Yifan Yang Feiyu Shen Chenpeng Du Ziyang Ma K. Yu Daniel Povey Xie Chen 35 24 0 14 Sep 2023
UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletons Sicheng Yang Zehao Wang Zhiyong Wu Minglei Li Zhensong Zhang ... Lei Hao Songcen Xu Xiaofei Wu Changpeng Yang Zonghong Dai DiffM 49 14 0 13 Sep 2023
MAGMA: Music Aligned Generative Motion Autodecoder Sohan Anisetty Amit Raj James Hays 26 0 0 03 Sep 2023
RepCodec: A Speech Representation Codec for Speech Tokenization Zhichao Huang Chutong Meng Tom Ko 22 23 0 31 Aug 2023
SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models Xin Zhang Dong Zhang Shimin Li Yaqian Zhou Xipeng Qiu 36 64 0 31 Aug 2023
Spoken Language Intelligence of Large Language Models for Language Learning Linkai Peng Baorian Nuchged Yingming Gao ELM 64 4 0 28 Aug 2023
Sparks of Large Audio Models: A Survey and Outlook S. Latif Moazzam Shoukat Fahad Shamshad Muhammad Usama Yi Ren ... Wenwu Wang Xulong Zhang Roberto Togneri Min Zhang Björn W. Schuller LM&MA AuLLM 33 38 0 24 Aug 2023
Example-Based Framework for Perceptually Guided Audio Texture Generation Purnima Kamath Chitralekha Gupta L. Wyse Suranga Nanayakkara 24 4 0 23 Aug 2023
TokenSplit: Using Discrete Speech Representations for Direct, Refined, and Transcript-Conditioned Speech Separation and Recognition Hakan Erdogan Scott Wisdom Xuankai Chang Zalan Borsos Marco Tagliasacchi Neil Zeghidour J. Hershey 18 9 0 21 Aug 2023
SpeechX: Neural Codec Language Model as a Versatile Speech Transformer Xiaofei Wang Manthan Thakker Zhuo Chen Naoyuki Kanda Sefik Emre Eskimez Sanyuan Chen M. Tang Shujie Liu Jinyu Li Takuya Yoshioka 26 80 0 14 Aug 2023
ModelScope Text-to-Video Technical Report Jiuniu Wang Hangjie Yuan Dayou Chen Yingya Zhang Xiang Wang Shiwei Zhang VGen DiffM 33 392 0 12 Aug 2023
Improving Joint Speech-Text Representations Without Alignment Cal Peyser Zhong Meng Ke Hu Rohit Prabhavalkar Andrew Rosenberg Tara N. Sainath M. Picheny Kyunghyun Cho VLM 31 4 0 11 Aug 2023
AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining Haohe Liu Yiitan Yuan Xubo Liu Xinhao Mei Qiuqiang Kong Qiao Tian Yuping Wang Wenwu Wang Yuxuan Wang Mark D. Plumbley DiffM 42 224 0 10 Aug 2023
EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis Tu Nguyen Wei-Ning Hsu Antony DÁvirro Bowen Shi Itai Gat ... Gabriel Synnaeve Michael Hassid Felix Kreuk Yossi Adi Emmanuel Dupoux 24 55 0 10 Aug 2023
JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Models Peike Li Bo-Yu Chen Yao Yao Yikai Wang Allen Wang Alex Jinpeng Wang MGen VLM DiffM 72 37 0 09 Aug 2023
DiffProsody: Diffusion-based Latent Prosody Generation for Expressive Speech Synthesis with Prosody Conditional Adversarial Training H. Oh Sang-Hoon Lee Seong-Whan Lee DiffM 23 14 0 31 Jul 2023
Minimally-Supervised Speech Synthesis with Conditional Diffusion Model and Language Model: A Comparative Study of Semantic Coding Chunyu Qiang Hao Li Hao Ni He Qu Ruibo Fu Tao Wang Longbiao Wang J. Dang DiffM 30 8 0 28 Jul 2023
Towards Generalist Biomedical AI Tao Tu Shekoofeh Azizi Danny Driess M. Schaekermann Mohamed Amin ... Yossi Matias K. Singhal Peter R. Florence Alan Karthikesalingam Vivek Natarajan LM&MA MedIm AI4MH 49 243 0 26 Jul 2023
Brain2Music: Reconstructing Music from Human Brain Activity Timo I. Denk Yu Takagi Takuya Matsuyama A. Agostinelli Tomoya Nakai Christian Frank Shinji Nishimoto 31 13 0 20 Jul 2023
VampNet: Music Generation via Masked Acoustic Token Modeling Hugo Flores Garcia Prem Seetharaman Rithesh Kumar Bryan Pardo MGen 45 64 0 10 Jul 2023
ChatGPT in the Age of Generative AI and Large Language Models: A Concise Survey S. Mohamadi G. Mujtaba Ngan Le Gianfranco Doretto Don Adjeroh LM&MA AI4MH 31 21 0 09 Jul 2023
Self-Consuming Generative Models Go MAD Sina Alemohammad Josue Casco-Rodriguez Lorenzo Luzi Ahmed Imtiaz Humayun Hossein Babaei Daniel LeJeune Ali Siahkoohi Richard G. Baraniuk WIGM 21 141 0 04 Jul 2023
SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs Lijun Yu Yong Cheng Zhiruo Wang Vivek Kumar Wolfgang Macherey ... Yonatan Bisk Ming Yang Kevin Patrick Murphy Alexander G. Hauptmann Lu Jiang MLLM 22 49 0 30 Jun 2023
Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale Matt Le Apoorv Vyas Bowen Shi Brian Karrer Leda Sari ... Mary Williamson Vimal Manohar Yossi Adi Jay Mahadeokar Wei-Ning Hsu AuLLM 28 265 0 23 Jun 2023
Impacts and Risk of Generative AI Technology on Cyber Defense Subash Neupane Ivan A. Fernandez Sudip Mittal Shahram Rahimi 29 17 0 22 Jun 2023
AudioPaLM: A Large Language Model That Can Speak and Listen Paul Kishan Rubenstein Chulayuth Asawaroengchai D. Nguyen Ankur Bapna Zalan Borsos ... Neil Zeghidour Yu Zhang Zhishuai Zhang Lukás Zilka Christian Frank LM&MA AuLLM VLM 39 264 0 22 Jun 2023
LM-VC: Zero-shot Voice Conversion via Speech Generation based on Language Models Zhichao Wang Yuan-Jui Chen Linfu Xie Qiao Tian Yuping Wang 72 30 0 18 Jun 2023
Text-Driven Foley Sound Generation With Latent Diffusion Model Yiitan Yuan Haohe Liu Xubo Liu Xiyuan Kang Peipei Wu Mark D.Plumbley Wenwu Wang DiffM 41 10 0 17 Jun 2023
FALL-E: A Foley Sound Synthesis Model and Strategies Minsung Kang Sangshin Oh Hyeongi Moon Kyungyun Lee Ben Sangbae Chon 25 4 0 16 Jun 2023
UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocoding Chenpeng Du Yiwei Guo Feiyu Shen Zhijun Liu Zheng Liang Xie Chen Shuai Wang Hui Zhang K. Yu DiffM 23 42 0 13 Jun 2023
High-Fidelity Audio Compression with Improved RVQGAN Rithesh Kumar Prem Seetharaman Alejandro Luebs I. Kumar Kundan Kumar 56 288 0 11 Jun 2023
Speech-to-Text Adapter and Speech-to-Entity Retriever Augmented LLMs for Speech Understanding Mingqiu Wang Izhak Shafran H. Soltau Wei Han Yuan Cao Dian Yu Laurent El Shafey RALM AuLLM 21 9 0 08 Jun 2023
Privately generating tabular data using language models Alexandre Sablayrolles Yue Wang Brian Karrer LMTD 26 4 0 07 Jun 2023
Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias Ziyue Jiang Yi Ren Zhe Ye Jinglin Liu Chen Zhang ... Rongjie Huang Chunfeng Wang Xiang Yin Zejun Ma Zhou Zhao DiffM 32 73 0 06 Jun 2023