Title
LLaMA-Omni2: LLM-based Real-time Spoken Chatbot with Autoregressive Streaming Speech Synthesis Qingkai Fang Yan Zhou Shoutao Guo Shaolei Zhang Yang Feng AuLLM 56 1 0 05 May 2025
Spatial Speech Translation: Translating Across Space With Binaural Hearables Tuochao Chen Qirui Wang Runlin He Shyam Gollakota 31 0 0 25 Apr 2025
SIFT-50M: A Large-Scale Multilingual Dataset for Speech Instruction Fine-Tuning Prabhat Pandey R. Swaminathan K V Vijay Girish Arunasish Sen Jian Xie Grant P. Strimel Andreas Schwarz 182 0 0 12 Apr 2025
Speculative End-Turn Detector for Efficient Speech Chatbot Assistant Hyunjong Ok Suho Yoo Jaeho Lee 34 0 0 30 Mar 2025
QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions Siyin Wang Wenyi Yu Xianzhao Chen Xiaohai Tian Jingyang Zhang Lu Lu Yu Tsao Junichi Yamagishi Yansen Wang Chao Zhang AuLLM 83 0 0 26 Mar 2025
ImF: Implicit Fingerprint for Large Language Models Wu jiaxuan Peng Wanli Fu hang Xue Yiming Wen juan 41 0 0 25 Mar 2025
Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey Yansen Wang Shengqiong Wu Yujie Zhang William Yang Wang Ziwei Liu Jiebo Luo Hao Fei LRM 92 9 0 16 Mar 2025
Universal Speech Token Learning via Low-Bitrate Neural Codec and Pretrained Representations Xue Jiang Xiulian Peng Yuan Zhang Yan-Heng Lu SSL 85 0 0 15 Mar 2025
Low-Rank and Sparse Model Merging for Multi-Lingual Speech Recognition and Translation Qiuming Zhao Guangzhi Sun Chao Zhang Mingxing Xu Thomas Fang Zheng MoMe VLM 202 0 0 24 Feb 2025
Retrieval-Augmented Speech Recognition Approach for Domain Challenges Peng Shen Xugang Lu Hisashi Kawai RALM 60 0 0 24 Feb 2025
DuplexMamba: Enhancing Real-time Speech Conversations with Duplex and Streaming Capabilities Xiangyu Lu Wang Xu Haoyu Wang Hongyun Zhou Haiyan Zhao Conghui Zhu T. Zhao M. Yang Mamba AuLLM 66 0 0 16 Feb 2025
FireRedASR: Open-Source Industrial-Grade Mandarin Speech Recognition Models from Encoder-Decoder to LLM Integration Kai-Tuo Xu Feng-Long Xie Xu Tang Yao Hu 74 4 0 24 Jan 2025
A Comprehensive Survey of Foundation Models in Medicine Wasif Khan Seowung Leem Kyle B. See Joshua K. Wong Shaoting Zhang R. Fang AI4CE LM&MA VLM 105 18 0 17 Jan 2025
Discrete Speech Unit Extraction via Independent Component Analysis Tomohiko Nakamura Kwanghee Choi Keigo Hojo Yoshiaki Bando Satoru Fukayama Shinji Watanabe 48 0 0 11 Jan 2025
AMPS: ASR with Multimodal Paraphrase Supervision Amruta Parulekar Abhishek Gupta Sameep Chattopadhyay P. Jyothi 75 0 0 27 Nov 2024
VoiceTextBlender: Augmenting Large Language Models with Speech Capabilities via Single-Stage Joint Speech-Text Supervised Fine-Tuning Yifan Peng Krishna C. Puvvada Zhehuai Chen Piotr .Zelasko He Huang Kunal Dhawan Ke Hu Shinji Watanabe Jagadeesh Balam Boris Ginsburg 64 2 0 23 Oct 2024
Continuous Speech Tokenizer in Text To Speech Yixing Li Ruobing Xie Xingchen Sun Yu Cheng Zhanhui Kang AuLLM CLL 63 2 0 22 Oct 2024
Ichigo: Mixed-Modal Early-Fusion Realtime Voice Assistant Alan Dao Dinh Bach Vu Huy Hoang Ha AuLLM VLM 73 3 0 20 Oct 2024
Thought2Text: Text Generation from EEG Signal using Large Language Models (LLMs) Abhijit Mishra Shreya Shukla Jose Torres Jacek Gwizdka Shounak Roychowdhury 51 4 0 10 Oct 2024
SPARTUN3D: Situated Spatial Understanding of 3D World in Large Language Models Yue Zhang Zhiyang Xu Ying Shen Parisa Kordjamshidi Lifu Huang 34 6 0 04 Oct 2024
Frozen Large Language Models Can Perceive Paralinguistic Aspects of Speech Wonjune Kang Junteng Jia Chunyang Wu Wei Zhou Egor Lakomkin ... Leda Sari Suyoun Kim Ke Li Jay Mahadeokar Ozlem Kalinli AuLLM 46 2 0 02 Oct 2024
Recent Advances in Speech Language Models: A Survey Wenqian Cui Dianzhi Yu Xiaoqi Jiao Ziqiao Meng Guangyan Zhang Qichao Wang Yiwen Guo Irwin King AuLLM 61 14 0 01 Oct 2024
SSR: Alignment-Aware Modality Connector for Speech Language Models Weiting Tan Hirofumi Inaguma Ning Dong Paden Tomasello Xutai Ma 32 3 0 30 Sep 2024
MIO: A Foundation Model on Multimodal Tokens Zekun Wang King Zhu Chunpu Xu Wangchunshu Zhou Jiaheng Liu ... Yuanxing Zhang Ge Zhang Ke Xu Jie Fu Wenhao Huang MLLM AuLLM 60 11 0 26 Sep 2024
Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation Siyin Wang Wenyi Yu Yudong Yang Changli Tang Yixuan Li ... Jun Zhang Guangzhi Sun Lu Lu Yuxuan Wang Chao Zhang AuLLM LM&MA 69 5 0 25 Sep 2024
Salmon: A Suite for Acoustic Language Model Evaluation Gallil Maimon Amit Roth Yossi Adi ELM AuLLM 66 5 0 11 Sep 2024
MoWE-Audio: Multitask AudioLLMs with Mixture of Weak Encoders Feiyu Xiong Shuo Sun Bin Wang Xunlong Zou Zhuohan Liu Yingxu He Geyu Lin Nancy F. Chen Ai Ti Aw AuLLM 67 1 0 10 Sep 2024
FLEURS-R: A Restored Multilingual Speech Corpus for Generation Tasks Min Ma Yuma Koizumi Shigeki Karita Heiga Zen Jason Riesa Haruko Ishikawa M. Bacchiani VLM 35 4 0 12 Aug 2024
dMel: Speech Tokenization made Simple Richard He Bai Tatiana Likhomanenko Ruixiang Zhang Zijin Gu Zakaria Aldeneh Navdeep Jaitly 43 4 0 22 Jul 2024
Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognition Ye Bai Jingping Chen Jitong Chen Wei Chen Zhuo Chen ... Wanyi Zhang Yang Zhang Yawei Zhang Yijie Zheng Ming Zou AuLLM 52 19 0 05 Jul 2024
Robust Zero-Shot Text-to-Speech Synthesis with Reverse Inference Optimization Yuchen Hu Chen Chen Siyin Wang Eng Siong Chng C. Zhang 43 3 0 02 Jul 2024
AudioBench: A Universal Benchmark for Audio Large Language Models Bin Wang Xunlong Zou Geyu Lin Shri Kiran Srinivasan Zhuohan Liu Wenyu Zhang Zhengyuan Liu AiTi Aw Nancy F. Chen AuLLM ELM LM&MA 92 21 0 23 Jun 2024
Large Language Models for Dysfluency Detection in Stuttered Speech Dominik Wagner Sebastian P. Bayerl Ilja Baumann Korbinian Riedhammer Elmar Nöth Tobias Bocklet 45 4 0 16 Jun 2024
How Should We Extract Discrete Audio Tokens from Self-Supervised Models? Pooneh Mousavi J. Duret Salah Zaiem Luca Della Libera Artem Ploujnikov Cem Subakan Mirco Ravanelli 42 9 0 15 Jun 2024
DiscreteSLU: A Large Language Model with Self-Supervised Discrete Speech Units for Spoken Language Understanding Suwon Shon Kwangyoun Kim Yi-Te Hsu Prashant Sridhar Shinji Watanabe Karen Livescu AuLLM 46 2 0 13 Jun 2024
Can Large Language Models Understand Spatial Audio? Changli Tang Wenyi Yu Guangzhi Sun Xianzhao Chen Tian Tan ... Jun Zhang Lu Lu Zejun Ma Yuxuan Wang Chao Zhang 49 4 0 12 Jun 2024
PolySpeech: Exploring Unified Multitask Speech Models for Competitiveness with Single-task Models Runyan Yang Huibao Yang Xiqing Zhang Tiantian Ye Ying Liu Yingying Gao Shilei Zhang Chao Deng Junlan Feng 36 0 0 12 Jun 2024
The Interspeech 2024 Challenge on Speech Processing Using Discrete Units Xuankai Chang Jiatong Shi Jinchuan Tian Yuning Wu Yuxun Tang Yihan Wu Shinji Watanabe Yossi Adi Xie Chen Qin Jin 47 15 0 11 Jun 2024
Discrete Multimodal Transformers with a Pretrained Large Language Model for Mixed-Supervision Speech Processing V. Trinh Rosy Southwell Yiwen Guan Xinlu He Zhiyong Wang Jacob Whitehill OffRL 36 2 0 04 Jun 2024
SeamlessExpressiveLM: Speech Language Model for Expressive Speech-to-Speech Translation with Chain-of-Thought Hongyu Gong Bandhav Veluri 61 0 0 30 May 2024
SpeechVerse: A Large-scale Generalizable Audio Language Model Nilaksh Das Saket Dingliwal S. Ronanki Rohit Paturi David Huang ... Monica Sunkara S. Srinivasan Kyu J. Han Katrin Kirchhoff Katrin Kirchhoff 41 37 0 14 May 2024
MAD Speech: Measures of Acoustic Diversity of Speech Matthieu Futeral A. Agostinelli Marco Tagliasacchi Neil Zeghidour Eugene Kharitonov 54 1 0 16 Apr 2024
WavLLM: Towards Robust and Adaptive Speech Large Language Model Shujie Hu Long Zhou Shujie Liu Sanyuan Chen Hongkun Hao ... Xunying Liu Jinyu Li S. Sivasankaran Linquan Liu Furu Wei AuLLM 23 45 0 31 Mar 2024
Direct Punjabi to English speech translation using discrete units Prabhjot Kaur L. A. M. Bush Weisong Shi 34 0 0 25 Feb 2024
Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations Guan-Ting Lin Cheng-Han Chiang Hung-yi Lee 34 24 0 20 Feb 2024
OWSM-CTC: An Open Encoder-Only Speech Foundation Model for Speech Recognition, Translation, and Language Identification Yifan Peng Yui Sudo Muhammad Shakeel Shinji Watanabe VLM 39 17 0 20 Feb 2024
InkSight: Offline-to-Online Handwriting Conversion by Learning to Read and Write B. Mitrevski Arina Rak Julian Schnitzler Chengkun Li Andrii Maksai Jesse Berent C. Musat DiffM 34 0 0 08 Feb 2024
VideoPoet: A Large Language Model for Zero-Shot Video Generation Dan Kondratyuk Lijun Yu Xiuye Gu José Lezama Jonathan Huang ... Irfan Essa Huisheng Wang David A. Ross Bryan Seybold Lu Jiang VGen 20 241 0 21 Dec 2023
Unconstrained Dysfluency Modeling for Dysfluent Speech Transcription and Detection Jiachen Lian Carly Feng Naasir Farooqi Steve Li Anshul Kashyap Cheol Jun Cho Peter Wu Robin Netzorg Tingle Li Gopala Krishna Anumanchipalli 40 13 0 20 Dec 2023
ADriver-I: A General World Model for Autonomous Driving Fan Jia Weixin Mao Yingfei Liu Yucheng Zhao Yuqing Wen Chi Zhang Xiangyu Zhang Tiancai Wang 48 63 0 22 Nov 2023