Title
Spatial Information Integration in Small Language Models for Document Layout Generation and Classification Pablo Melendez Clemens Havas 36 0 0 09 Jan 2025
ZSVC: Zero-shot Style Voice Conversion with Disentangled Latent Diffusion Models and Adversarial Training Xinfa Zhu Lei He Yujia Xiao Xi Wang Xu Tan Sheng Zhao Lei Xie DiffM 40 0 0 08 Jan 2025
Unsupervised Speech Segmentation: A General Approach Using Speech Language Models Avishai Elmakies Omri Abend Yossi Adi 33 1 0 08 Jan 2025
SongEditor: Adapting Zero-Shot Song Generation Language Model as a Multi-Task Editor Chenyu Yang Shuai Wang Hangting Chen Jianwei Yu Wei Tan Rongzhi Gu Yongjun Xu Yizhi Zhou Haina Zhu Yiming Li KELM 197 1 0 18 Dec 2024
SOLAMI: Social Vision-Language-Action Modeling for Immersive Interaction with 3D Autonomous Characters Jianping Jiang Weiye Xiao Zhengyu Lin H. Zhang Tianxiang Ren Yang Gao Zhiqian Lin Zhongang Cai Lei Yang Ziwei Liu 86 3 0 29 Nov 2024
Scaling Transformers for Low-Bitrate High-Quality Speech Coding Julian Parker Anton Smirnov Jordi Pons CJ Carr Zack Zukowski Zach Evans Xubo Liu 77 9 0 29 Nov 2024
Tiny-Align: Bridging Automatic Speech Recognition and Large Language Model on the Edge Ruiyang Qin Dancheng Liu Gelei Xu Zheyu Yan Chenhui Xu Yuting Hu Xiaolin Hu Jinjun Xiong Yiyu Shi AuLLM 115 1 0 21 Nov 2024
ESTVocoder: An Excitation-Spectral-Transformed Neural Vocoder Conditioned on Mel Spectrogram Xiao-Hang Jiang Hui-Peng Du Yang Ai Ye-Xin Lu Zhen-Hua Ling 30 0 0 18 Nov 2024
Mamba-based Decoder-Only Approach with Bidirectional Speech Modeling for Speech Recognition Yoshiki Masuyama Koichi Miyazaki Masato Murata Mamba 43 0 0 11 Nov 2024
Long-Form Text-to-Music Generation with Adaptive Prompts: A Case of Study in Tabletop Role-Playing Games Soundtracks Felipe Marra Lucas N. Ferreira 31 0 0 06 Nov 2024
Addressing Representation Collapse in Vector Quantized Models with One Linear Layer Yongxin Zhu B. Li Yifei Xin Linli Xu 41 10 0 04 Nov 2024
Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback Guan-Ting Lin Prashanth Gurunath Shivakumar Aditya Gourav Yile Gu Ankur Gandhe Hung-yi Lee I. Bulyko 34 8 0 04 Nov 2024
Sing-On-Your-Beat: Simple Text-Controllable Accompaniment Generations Quoc-Huy Trinh Minh-Van Nguyen Trong-Hieu Nguyen-Mau Khoa Tran Thanh Do 35 0 0 03 Nov 2024
MDCTCodec: A Lightweight MDCT-based Neural Audio Codec towards High Sampling Rate and Low Bitrate Scenarios Xiao-Hang Jiang Yang Ai Rui Zheng Hui-Peng Du Ye-Xin Lu Zhen-Hua Ling 53 2 0 01 Nov 2024
DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Models Heng-Jui Chang Hongyu Gong Changhan Wang James R. Glass Yu-An Chung 28 0 0 31 Oct 2024
An Empirical Analysis of Speech Self-Supervised Learning at Multiple Resolutions Theo Clark Benedetta Cevoli Eloy de Jong Timofey Abramski Jamie Dougherty SSL 41 0 0 31 Oct 2024
APCodec+: A Spectrum-Coding-Based High-Fidelity and High-Compression-Rate Neural Audio Codec with Staged Training Paradigm Hui-Peng Du Yang Ai Rui Zheng Zhen-Hua Ling 40 0 0 30 Oct 2024
A Closer Look at Neural Codec Resynthesis: Bridging the Gap between Codec and Waveform Generation Alexander H. Liu Qirui Wang Yuan Gong James Glass 33 0 0 29 Oct 2024
Survey of User Interface Design and Interaction Techniques in Generative AI Applications Reuben Luera Ryan Rossi Alexa F. Siu Franck Dernoncourt Tong Yu ... Hanieh Salehy Jian Zhao Samyadeep Basu Puneet Mathur Nedim Lipka AI4TS 65 1 0 28 Oct 2024
Get Large Language Models Ready to Speak: A Late-fusion Approach for Speech Generation Maohao Shen Shun Zhang Jilong Wu Zhiping Xiu Ehab AlBadawy Yiting Lu M. Seltzer Qing He 36 2 0 27 Oct 2024
GE2E-KWS: Generalized End-to-End Training and Evaluation for Zero-shot Keyword Spotting Pai Zhu Jacob Bartel Dhruuv Agarwal Kurt Partridge Hyun-jin Park Quan Wang 26 0 0 22 Oct 2024
Continuous Speech Tokenizer in Text To Speech Yixing Li Ruobing Xie Xingchen Sun Yu Cheng Zhanhui Kang AuLLM CLL 63 2 0 22 Oct 2024
Ichigo: Mixed-Modal Early-Fusion Realtime Voice Assistant Alan Dao Dinh Bach Vu Huy Hoang Ha AuLLM VLM 73 3 0 20 Oct 2024
DM-Codec: Distilling Multimodal Representations for Speech Tokenization Md Mubtasim Ahasan Md Fahim Tasnim Mohiuddin A K M Mahbubur Rahman Aman Chadha Tariq Iqbal M. A. Amin Md. Mofijul Islam Amin Ahsan Ali 30 0 0 19 Oct 2024
SNAC: Multi-Scale Neural Audio Codec Hubert Siuzdak Florian Grötschla Luca A. Lanzendörfer 21 10 0 18 Oct 2024
Sound Check: Auditing Audio Datasets William Agnew Julia Barnett Annie Chu Rachel Hong Michael Feffer Robin Netzorg Harry H. Jiang Ezra Awumey Sauvik Das 44 1 0 17 Oct 2024
JOOCI: a Framework for Learning Comprehensive Speech Representations Hemant Yadav R. Shah Sunayana Sitaram 28 0 0 14 Oct 2024
Code Drift: Towards Idempotent Neural Audio Codecs P. O'Reilly Prem Seetharaman Jiaqi Su Zeyu Jin Bryan Pardo 158 0 0 14 Oct 2024
LS-EEND: Long-Form Streaming End-to-End Neural Diarization with Online Attractor Extraction Di Liang Xiaofei Li 24 0 0 09 Oct 2024
Can DeepFake Speech be Reliably Detected? Hongbin Liu Youzheng Chen Arun Narayanan Athula Balachandran Pedro J. Moreno Lun Wang AAML 35 1 0 09 Oct 2024
IntrinsicVoice: Empowering LLMs with Intrinsic Real-time Voice Interaction Abilities Xin Zhang Xiang Lyu Zhihao Du Qian Chen Dong Zhang ... Yuxuan Wang Bin Zhang Heng Lu Yaqian Zhou Xipeng Qiu AuLLM 41 6 0 09 Oct 2024
Sylber: Syllabic Embedding Representation of Speech from Raw Audio Cheol Jun Cho Nicholas Lee Akshat Gupta Dhruv Agarwal Ethan Chen Alan W Black Gopala K. Anumanchipalli 34 0 0 09 Oct 2024
Diversity-Rewarded CFG Distillation Geoffrey Cideron A. Agostinelli Johan Ferret Sertan Girgin Romuald Elie Olivier Bachem Sarah Perrin Alexandre Ramé 49 2 0 08 Oct 2024
Computational design of target-specific linear peptide binders with TransformerBeta Haowen Zhao Francesco A. Aprile Barbara Bravi 31 0 0 07 Oct 2024
HALL-E: Hierarchical Neural Codec Language Model for Minute-Long Zero-Shot Text-to-Speech Synthesis Yuto Nishimura Takumi Hirose Masanari Ohi Hideki Nakayama Nakamasa Inoue VLM 29 1 0 06 Oct 2024
Audio-Agent: Leveraging LLMs For Audio Generation, Editing and Composition Zixuan Wang Chi-Keung Tang Chi-Keung Tang DiffM VGen LLMAG 49 4 0 04 Oct 2024
MultiVerse: Efficient and Expressive Zero-Shot Multi-Task Text-to-Speech Taejun Bak Youngsik Eom SeungJae Choi Young-Sun Joo 35 0 0 04 Oct 2024
FedMAC: Tackling Partial-Modality Missing in Federated Learning with Cross-Modal Aggregation and Contrastive Regularization Manh Duong Nguyen Trung Thanh Nguyen Huy Hieu Pham Trong Nghia Hoang Phi Le Nguyen T. T. Huynh 28 1 0 04 Oct 2024
Zero-Shot Text-to-Speech from Continuous Text Streams Trung D. Q. Dang David Aponte Dung Tran Tianyi Chen K. Koishida AuLLM VLM 37 3 0 01 Oct 2024
Recent Advances in Speech Language Models: A Survey Wenqian Cui Dianzhi Yu Xiaoqi Jiao Ziqiao Meng Guangyan Zhang Qichao Wang Yiwen Guo Irwin King AuLLM 61 14 0 01 Oct 2024
SSR: Alignment-Aware Modality Connector for Speech Language Models Weiting Tan Hirofumi Inaguma Ning Dong Paden Tomasello Xutai Ma 32 3 0 30 Sep 2024
MIO: A Foundation Model on Multimodal Tokens Zekun Wang King Zhu Chunpu Xu Wangchunshu Zhou Jiaheng Liu ... Yuanxing Zhang Ge Zhang Ke Xu Jie Fu Wenhao Huang MLLM AuLLM 60 11 0 26 Sep 2024
Codec-SUPERB @ SLT 2024: A lightweight benchmark for neural audio codec models Haibin Wu Xuanjun Chen Yi-Cheng Lin Kaiwei Chang Jiawei Du ... Yi-Chiao Wu Xu Tan James Glass Shinji Watanabe Hung-yi Lee 34 6 0 21 Sep 2024
Preference Alignment Improves Language Model-Based TTS Jinchuan Tian Chunlei Zhang Jiatong Shi Hao Zhang Jianwei Yu Shinji Watanabe Dong Yu 32 7 0 19 Sep 2024
Low Frame-rate Speech Codec: a Codec Designed for Fast High-quality Speech LLM Training and Inference Edresson Casanova Ryan Langman Paarth Neekhara Shehzeen Samarah Hussain Jason Chun Lok Li Subhankar Ghosh Ante Jukić Sang-gil Lee AuLLM 37 2 0 18 Sep 2024
Adaptive Large Language Models By Layerwise Attention Shortcuts Prateek Verma Mert Pilanci KELM OffRL 58 0 0 17 Sep 2024
Learning Source Disentanglement in Neural Audio Codec Xiaoyu Bie Xubo Liu Gaël Richard 29 1 0 17 Sep 2024
Self-Supervised Syllable Discovery Based on Speaker-Disentangled HuBERT Ryota Komatsu Takahiro Shinozaki SSL 39 1 0 16 Sep 2024
SafeEar: Content Privacy-Preserving Audio Deepfake Detection Xinfeng Li Kai Li Yifan Zheng Chen Yan Xiaoyu Ji Wenyuan Xu 33 13 0 14 Sep 2024
Investigating Disentanglement in a Phoneme-level Speech Codec for Prosody Modeling Sotirios Karapiperis Nikolaos Ellinas Alexandra Vioni Junkwang Oh Gunu Jho Inchul Hwang S. Raptis 36 0 0 13 Sep 2024