Title
Pictures Of MIDI: Controlled Music Generation via Graphical Prompts for Image-Based Diffusion Inpainting Scott H. Hawley 38 2 0 01 Jul 2024
FLY-TTS: Fast, Lightweight and High-Quality End-to-End Text-to-Speech Synthesis Yinlin Guo Yening Lv Jinqiao Dou Yan Zhang Yuehai Wang 29 0 0 30 Jun 2024
From Efficient Multimodal Models to World Models: A Survey Xinji Mai Zeng Tao Junxiong Lin Haoran Wang Yang Chang Yanlan Kang Yan Wang Wenqiang Zhang 32 5 0 27 Jun 2024
Improving Robustness of LLM-based Speech Synthesis by Learning Monotonic Alignment Paarth Neekhara Shehzeen Samarah Hussain Subhankar Ghosh Jason Chun Lok Li Rafael Valle Rohan Badlani Boris Ginsburg 58 11 0 25 Jun 2024
SpecMaskGIT: Masked Generative Modeling of Audio Spectrograms for Efficient Audio Synthesis and Beyond Marco Comunità Zhi-Wei Zhong Akira Takahashi Shiqi Yang Mengjie Zhao Koichi Saito Yukara Ikemiya Takashi Shibuya Shusuke Takahashi Yuki Mitsufuji 71 2 0 25 Jun 2024
TacoLM: GaTed Attention Equipped Codec Language Model are Efficient Zero-Shot Text to Speech Synthesizers Yakun Song Zhuo Chen Xiaofei Wang Ziyang Ma Guanrou Yang Xie Chen AuLLM 46 3 0 22 Jun 2024
Are Language Models Actually Useful for Time Series Forecasting? Mingtian Tan Mike A. Merrill Vinayak Gupta Tim Althoff Thomas Hartvigsen AI4TS 46 45 0 22 Jun 2024
DASB -- Discrete Audio and Speech Benchmark Pooneh Mousavi Luca Della Libera J. Duret Artem Ploujnikov Cem Subakan Mirco Ravanelli 35 12 0 20 Jun 2024
NAST: Noise Aware Speech Tokenization for Speech Language Models Shoval Messica Yossi Adi 30 6 0 16 Jun 2024
Joint Audio and Symbolic Conditioning for Temporally Controlled Text-to-Music Generation Or Tal Alon Ziv Itai Gat Felix Kreuk Yossi Adi 55 13 0 16 Jun 2024
How Should We Extract Discrete Audio Tokens from Self-Supervised Models? Pooneh Mousavi J. Duret Salah Zaiem Luca Della Libera Artem Ploujnikov Cem Subakan Mirco Ravanelli 42 9 0 15 Jun 2024
UniAudio 1.5: Large Language Model-driven Audio Codec is A Few-shot Audio Task Learner Dongchao Yang Haohan Guo Yuanyuan Wang Rongjie Huang Xiang Li Xu Tan Xixin Wu Helen Meng AuLLM 47 15 0 14 Jun 2024
ToneUnit: A Speech Discretization Approach for Tonal Language Speech Synthesis Dehua Tao Daxin Tan Y. Yeung Xiao Chen Tan Lee 35 3 0 13 Jun 2024
SingOMD: Singing Oriented Multi-resolution Discrete Representation Construction from Speech Models Yuxun Tang Yuning Wu Jiatong Shi Qin Jin 60 5 0 13 Jun 2024
Self-Supervised Speech Representations are More Phonetic than Semantic Kwanghee Choi Ankita Pasad Tomohiko Nakamura Satoru Fukayama Karen Livescu Shinji Watanabe 39 14 0 12 Jun 2024
CoLM-DSR: Leveraging Neural Codec Language Modeling for Multi-Modal Dysarthric Speech Reconstruction Xueyuan Chen Dongchao Yang Dingdong Wang Xixin Wu Zhiyong Wu Helen Meng 51 1 0 12 Jun 2024
Codecfake: An Initial Dataset for Detecting LLM-based Deepfake Audio Yi Lu Yuankun Xie Ruibo Fu Zhengqi Wen Jianhua Tao ... Xuefei Liu Yongwei Li Yukun Liu Xiaopeng Wang Shuchen Shi 51 1 0 12 Jun 2024
Let's Go Real Talk: Spoken Dialogue Model for Face-to-Face Conversation Se Jin Park Chae Won Kim Hyeongseop Rha Minsu Kim Joanna Hong Jeong Hun Yeo Yong Man Ro CVBM AuLLM 48 6 0 12 Jun 2024
VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignment Bing Han Long Zhou Shujie Liu Sanyuan Chen Lingwei Meng Yanming Qian Yanqing Liu Sheng Zhao Jinyu Li Furu Wei 49 14 0 12 Jun 2024
PolySpeech: Exploring Unified Multitask Speech Models for Competitiveness with Single-task Models Runyan Yang Huibao Yang Xiqing Zhang Tiantian Ye Ying Liu Yingying Gao Shilei Zhang Chao Deng Junlan Feng 36 0 0 12 Jun 2024
CodecFake: Enhancing Anti-Spoofing Models Against Deepfake Audios from Codec-Based Speech Synthesis Systems Haibin Wu Yuan Tseng Hung-yi Lee AuLLM 35 6 0 11 Jun 2024
Learning Fine-Grained Controllability on Speech Generation via Efficient Fine-Tuning Chung-Ming Chien Andros Tjandra Apoorv Vyas Matt Le Bowen Shi Wei-Ning Hsu 32 0 0 10 Jun 2024
Towards Expressive Zero-Shot Speech Synthesis with Hierarchical Prosody Modeling Yuepeng Jiang Tao Li Fengyu Yang Lei Xie Meng Meng Yujun Wang 40 2 0 09 Jun 2024
Autoregressive Diffusion Transformer for Text-to-Speech Synthesis Zhijun Liu Shuai Wang Sho Inoue Qibing Bai Haizhou Li DiffM 47 15 0 08 Jun 2024
Exploring the Benefits of Tokenization of Discrete Acoustic Units Avihu Dekel Raul Fernandez 49 2 0 08 Jun 2024
VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers Sanyuan Chen Shujie Liu Long Zhou Yanqing Liu Xu Tan Jinyu Li Sheng Zhao Yao Qian Furu Wei VLM 47 67 0 08 Jun 2024
SEE-2-SOUND: Zero-Shot Spatial Environment-to-Spatial Sound Rishit Dagli Shivesh Prakash Robert Wu H. Khosravani 39 3 0 06 Jun 2024
Retrieval Augmented Generation in Prompt-based Text-to-Speech Synthesis with Context-Aware Contrastive Language-Audio Pretraining Jinlong Xue Yayue Deng Yingming Gao Ya Li RALM VLM 42 4 0 06 Jun 2024
Improving Audio Codec-based Zero-Shot Text-to-Speech Synthesis with Multi-Modal Context and Large Language Model Jinlong Xue Yayue Deng Yicheng Han Yingming Gao Ya Li 40 4 0 06 Jun 2024
LiveSpeech: Low-Latency Zero-shot Text-to-Speech via Autoregressive Modeling of Audio Discrete Codes Trung D. Q. Dang David Aponte Dung Tran K. Koishida 38 3 0 05 Jun 2024
Discrete Multimodal Transformers with a Pretrained Large Language Model for Mixed-Supervision Speech Processing V. Trinh Rosy Southwell Yiwen Guan Xinlu He Zhiyong Wang Jacob Whitehill OffRL 36 2 0 04 Jun 2024
SimpleSpeech: Towards Simple and Efficient Text-to-Speech with Scalar Latent Transformer Diffusion Models Dongchao Yang Dingdong Wang Haohan Guo Xueyuan Chen Xixin Wu Helen M. Meng 67 26 0 04 Jun 2024
An Independence-promoting Loss for Music Generation with Language Models Jean-Marie Lemercier Simon Rouard Jade Copet Yossi Adi Alexandre Défossez 30 1 0 04 Jun 2024
Phonetic Enhanced Language Modeling for Text-to-Speech Synthesis Kun Zhou Shengkui Zhao Yukun Ma Chong Zhang Hao Wang Dianwen Ng Chongjia Ni Nguyen Trung Hieu J. Yip Bin Ma 35 5 0 04 Jun 2024
Enhancing Zero-shot Text-to-Speech Synthesis with Human Feedback Chen Chen Yuchen Hu Wen Wu Helin Wang Chng Eng Siong Chao Zhang 40 10 0 02 Jun 2024
Wav2Prompt: End-to-End Speech Prompt Generation and Tuning For LLM in Zero and Few-shot Learning Keqi Deng Guangzhi Sun Phil Woodland VLM 33 4 0 01 Jun 2024
Creative Text-to-Audio Generation via Synthesizer Programming Manuel Cherep Nikhil Singh Jessica Shand 27 3 0 01 Jun 2024
A Survey of Deep Learning Audio Generation Methods Matej Bozic Marko Horvat VLM MedIm 58 0 0 31 May 2024
Zipper: A Multi-Tower Decoder Architecture for Fusing Modalities Vicky Zayats Peter Chen Melissa Ferrari Dirk Padfield AI4CE 38 0 0 29 May 2024
TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservation Chenyang Le Yao Qian Dongmei Wang Long Zhou Shujie Liu ... Midia Yousefi Yanmin Qian Jinyu Li Sheng Zhao Michael Zeng 47 3 0 28 May 2024
"Pass the butter": A study on desktop-classic multitasking robotic arm based on advanced YOLOv7 and BERT Haohua Que Wenbin Pan Jie Xu Hao Luo Pei Wang Li Zhang 39 1 0 27 May 2024
iVideoGPT: Interactive VideoGPTs are Scalable World Models Jialong Wu Shaofeng Yin Ningya Feng Xu He Dong Li Haifeng Zhang Mingsheng Long VGen 49 26 0 24 May 2024
FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher information Dongseong Hwang ODL 37 4 0 21 May 2024
Multi-speaker Text-to-speech Training with Speaker Anonymized Data Wen-Chin Huang Yi-Chiao Wu T. Toda 40 1 0 20 May 2024
Evaluating Text-to-Speech Synthesis from a Large Discrete Token-based Speech Language Model Siyang Wang Éva Székely 47 4 0 16 May 2024
FastSAG: Towards Fast Non-Autoregressive Singing Accompaniment Generation Jianyi Chen Wei Xue Xu Tan Zhen Ye Qi-fei Liu Yi-Ting Guo 50 2 0 13 May 2024
The Codecfake Dataset and Countermeasures for the Universally Detection of Deepfake Audio Yuankun Xie Yi Lu Ruibo Fu Zhengqi Wen Zhiyong Wang ... Xiaopeng Wang Yukun Liu Haonan Cheng Long Ye Yi Sun 47 15 0 08 May 2024
SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound Haohe Liu Xuenan Xu Yiitan Yuan Mengyue Wu Wenwu Wang Mark D. Plumbley 37 18 0 30 Apr 2024
Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesis Shivam Mehta Anna Deichler Jim O'Regan Birger Moëll Jonas Beskow G. Henter Simon Alexanderson 46 4 0 30 Apr 2024
ESC: Efficient Speech Coding with Cross-Scale Residual Vector Quantized Transformers Yuzhe Gu Enmao Diao 37 4 0 30 Apr 2024