Title
SpeechPrompt: Prompting Speech Language Models for Speech Processing Tasks Kai-Wei Chang Haibin Wu Yu-Kai Wang Yuan-Kuei Wu Hua Shen Wei-Cheng Tseng Iu-thing Kang Shang-Wen Li Hung-yi Lee 93 3 0 23 Aug 2024
Does Current Deepfake Audio Detection Model Effectively Detect ALM-based Deepfake Audio? Yuankun Xie Chenxu Xiong Xiaopeng Wang Zhiyong Wang Yi Lu ... Yukun Liu Zhengqi Wen Jianhua Tao Guanjun Li Long Ye AuLLM 119 1 0 20 Aug 2024
Adversarial training of Keyword Spotting to Minimize TTS Data Overfitting Hyun Jin Park Dhruuv Agarwal Neng Chen Rentao Sun Kurt Partridge ... Jacob Bartel Kyle Kastner Gary Wang Andrew Rosenberg Quan Wang 61 2 0 20 Aug 2024
Convert and Speak: Zero-shot Accent Conversion with Minimum Supervision Zhijun Jia Huaying Xue Xiulian Peng Yan Lu 152 3 0 19 Aug 2024
PRESENT: Zero-Shot Text-to-Prosody Control Perry Lam Huayun Zhang Nancy F. Chen Berrak Sisman Dorien Herremans 92 0 0 13 Aug 2024
FLEURS-R: A Restored Multilingual Speech Corpus for Generation Tasks Min Ma Yuma Koizumi Shigeki Karita Heiga Zen Jason Riesa Haruko Ishikawa M. Bacchiani VLM 97 5 0 12 Aug 2024
VQ-CTAP: Cross-Modal Fine-Grained Sequence Representation Learning for Speech Processing Chunyu Qiang Wang Geng Yi Zhao Ruibo Fu Tao Wang ... Chen Zhang Hao Che L. Wang Jianwu Dang J. Tao AI4TS 105 0 0 11 Aug 2024
Simulating Articulatory Trajectories with Phonological Feature Interpolation Angelo Ortiz Tandazo Thomas Schatz Thomas Hueber Emmanuel Dupoux 69 0 0 08 Aug 2024
Language Model Can Listen While Speaking Ziyang Ma Yakun Song Chenpeng Du Jian Cong Zhuo Chen Yuping Wang Yansen Wang Xie Chen AuLLM 103 28 0 05 Aug 2024
SynesLM: A Unified Approach for Audio-visual Speech Recognition and Translation via Language Model and Synthetic Data Yichen Lu Álvaro Huertas-García Xuankai Chang Hengwei Bian Soumi Maiti Shinji Watanabe 95 2 0 01 Aug 2024
Bailing-TTS: Chinese Dialectal Speech Synthesis Towards Human-like Spontaneous Representation Xinhan Di Jiahao Lu Yunming Liang Junjie Zheng Yihua Wang Chaofan Ding ALM 98 1 0 01 Aug 2024
Enhancing Anti-spoofing Countermeasures Robustness through Joint Optimization and Transfer Learning Yikang Wang Xingming Wang Hiromitsu Nishizaki Ming Li AAML 61 1 0 29 Jul 2024
Utilizing TTS Synthesized Data for Efficient Development of Keyword Spotting Model Hyun Jin Park Dhruuv Agarwal Neng Chen Rentao Sun Kurt Partridge ... Jacob Bartel Kyle Kastner Gary Wang Andrew Rosenberg Quan Wang 61 4 0 26 Jul 2024
Generating Sample-Based Musical Instruments Using Neural Audio Codec Language Models S. Nercessian Johannes Imort Ninon Devis Frederik Blang 73 1 0 22 Jul 2024
Chronologically Accurate Retrieval for Temporal Grounding of Motion-Language Models Kent Fujiwara Mikihiro Tanaka Qing Yu 92 2 0 22 Jul 2024
dMel: Speech Tokenization made Simple Richard He Bai Tatiana Likhomanenko Ruixiang Zhang Zijin Gu Zakaria Aldeneh Navdeep Jaitly 113 6 0 22 Jul 2024
Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Models Amir Mohammad Karimi Mamaghan Samuele Papa Karl Henrik Johansson Stefan Bauer Andrea Dittadi OCL 178 9 0 22 Jul 2024
On Pre-training of Multimodal Language Models Customized for Chart Understanding Wan-Cyuan Fan Yen-Chun Chen Mengchen Liu Lu Yuan Leonid Sigal 103 7 0 19 Jul 2024
Spontaneous Style Text-to-Speech Synthesis with Controllable Spontaneous Behaviors Based on Language Models Weiqin Li Pei-Yin Yang Yicheng Zhong Yixuan Zhou Zhisheng Wang Zhiyong Wu Xixin Wu Helen M. Meng 149 3 0 18 Jul 2024
A Language Modeling Approach to Diacritic-Free Hebrew TTS Amit Roth A. Turetzky Yossi Adi 89 3 0 16 Jul 2024
Masked Generative Video-to-Audio Transformers with Enhanced Synchronicity Santiago Pascual Chunghsin Yeh Ioannis Tsiamas Joan Serrà DiffM VGen 95 16 0 15 Jul 2024
Autoregressive Speech Synthesis without Vector Quantization Lingwei Meng Long Zhou Shujie Liu Sanyuan Chen Bing Han ... Jinyu Li Sheng Zhao Xixin Wu Helen M. Meng Furu Wei 174 43 0 11 Jul 2024
Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generation Haorui He Zengqiang Shang Chaoren Wang Xuyuan Li Yicheng Gu ... Peiyang Shi Yuancheng Wang Kai Chen Pengyuan Zhang Zhizheng Wu 101 54 0 07 Jul 2024
PAGURI: a user experience study of creative interaction with text-to-music models Francesca Ronchini Luca Comanducci Gabriele Perego Fabio Antonacci 103 3 0 05 Jul 2024
On the Effectiveness of Acoustic BPE in Decoder-Only TTS Bohan Li Feiyu Shen Yiwei Guo Shuai Wang Xie Chen Kai Yu 102 2 0 04 Jul 2024
Robust Zero-Shot Text-to-Speech Synthesis with Reverse Inference Optimization Yuchen Hu Chen Chen Siyin Wang Eng Siong Chng C. Zhang 91 4 0 02 Jul 2024
Accompanied Singing Voice Synthesis with Fully Text-controlled Melody Ruiqi Li Zhiqing Hong Yongqi Wang Lichao Zhang Rongjie Huang Siqi Zheng Zhou Zhao 114 8 0 02 Jul 2024
Towards the Next Frontier in Speech Representation Learning Using Disentanglement Varun Krishna Sriram Ganapathy SSL 64 1 0 02 Jul 2024
Investigating the Effects of Large-Scale Pseudo-Stereo Data and Different Speech Foundation Model on Dialogue Generative Spoken Language Model Yu-Kuan Fu Cheng-Kuang Lee Hsiu-Hsuan Wang Hung-yi Lee 54 0 0 02 Jul 2024
Pictures Of MIDI: Controlled Music Generation via Graphical Prompts for Image-Based Diffusion Inpainting Scott H. Hawley 86 2 0 01 Jul 2024
FLY-TTS: Fast, Lightweight and High-Quality End-to-End Text-to-Speech Synthesis Yinlin Guo Yening Lv Jinqiao Dou Yan Zhang Yuehai Wang 78 0 0 30 Jun 2024
From Efficient Multimodal Models to World Models: A Survey Xinji Mai Zeng Tao Junxiong Lin Haoran Wang Yang Chang Yanlan Kang Yan Wang Wenqiang Zhang 100 6 0 27 Jun 2024
Improving Robustness of LLM-based Speech Synthesis by Learning Monotonic Alignment Paarth Neekhara Shehzeen Samarah Hussain Subhankar Ghosh Jason Chun Lok Li Rafael Valle Rohan Badlani Boris Ginsburg 83 14 0 25 Jun 2024
SpecMaskGIT: Masked Generative Modeling of Audio Spectrograms for Efficient Audio Synthesis and Beyond Marco Comunità Zhi-Wei Zhong Akira Takahashi Shiqi Yang Mengjie Zhao Koichi Saito Yukara Ikemiya Takashi Shibuya Shusuke Takahashi Yuki Mitsufuji 116 6 0 25 Jun 2024
TacoLM: GaTed Attention Equipped Codec Language Model are Efficient Zero-Shot Text to Speech Synthesizers Yakun Song Zhuo Chen Xiaofei Wang Ziyang Ma Guanrou Yang Xie Chen AuLLM 55 4 0 22 Jun 2024
Are Language Models Actually Useful for Time Series Forecasting? Mingtian Tan Mike A. Merrill Vinayak Gupta Tim Althoff Thomas Hartvigsen AI4TS 126 68 0 22 Jun 2024
DASB -- Discrete Audio and Speech Benchmark Pooneh Mousavi Luca Della Libera J. Duret Artem Ploujnikov Cem Subakan Mirco Ravanelli 103 21 0 20 Jun 2024
NAST: Noise Aware Speech Tokenization for Speech Language Models Shoval Messica Yossi Adi 91 7 0 16 Jun 2024
Joint Audio and Symbolic Conditioning for Temporally Controlled Text-to-Music Generation Or Tal Alon Ziv Itai Gat Felix Kreuk Yossi Adi 88 17 0 16 Jun 2024
How Should We Extract Discrete Audio Tokens from Self-Supervised Models? Pooneh Mousavi J. Duret Salah Zaiem Luca Della Libera Artem Ploujnikov Cem Subakan Mirco Ravanelli 99 15 0 15 Jun 2024
UniAudio 1.5: Large Language Model-driven Audio Codec is A Few-shot Audio Task Learner Dongchao Yang Haohan Guo Yuanyuan Wang Rongjie Huang Xiang Li Xu Tan Xixin Wu Helen Meng AuLLM 93 17 0 14 Jun 2024
ToneUnit: A Speech Discretization Approach for Tonal Language Speech Synthesis Dehua Tao Daxin Tan Y. Yeung Xiao Chen Tan Lee 84 3 0 13 Jun 2024
SingOMD: Singing Oriented Multi-resolution Discrete Representation Construction from Speech Models Yuxun Tang Yuning Wu Jiatong Shi Qin Jin 102 5 0 13 Jun 2024
Self-Supervised Speech Representations are More Phonetic than Semantic Kwanghee Choi Ankita Pasad Tomohiko Nakamura Satoru Fukayama Karen Livescu Shinji Watanabe 100 24 0 12 Jun 2024
CoLM-DSR: Leveraging Neural Codec Language Modeling for Multi-Modal Dysarthric Speech Reconstruction Xueyuan Chen Dongchao Yang Dingdong Wang Xixin Wu Zhiyong Wu Helen Meng 75 2 0 12 Jun 2024
Codecfake: An Initial Dataset for Detecting LLM-based Deepfake Audio Yi Lu Yuankun Xie Ruibo Fu Zhengqi Wen Jianhua Tao ... Xuefei Liu Yongwei Li Yukun Liu Xiaopeng Wang Shuchen Shi 73 1 0 12 Jun 2024
Let's Go Real Talk: Spoken Dialogue Model for Face-to-Face Conversation Se Jin Park Chae Won Kim Hyeongseop Rha Minsu Kim Joanna Hong Jeong Hun Yeo Yong Man Ro CVBM AuLLM 97 14 0 12 Jun 2024
VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignment Bing Han Long Zhou Shujie Liu Sanyuan Chen Lingwei Meng Yanming Qian Yanqing Liu Sheng Zhao Jinyu Li Furu Wei 112 24 0 12 Jun 2024
PolySpeech: Exploring Unified Multitask Speech Models for Competitiveness with Single-task Models Runyan Yang Huibao Yang Xiqing Zhang Tiantian Ye Ying Liu Yingying Gao Shilei Zhang Chao Deng Junlan Feng 93 0 0 12 Jun 2024
CodecFake: Enhancing Anti-Spoofing Models Against Deepfake Audios from Codec-Based Speech Synthesis Systems Haibin Wu Yuan Tseng Hung-yi Lee AuLLM 65 11 0 11 Jun 2024