SALMONN-omni: A Standalone Speech LLM without Codec Injection for Full-duplex Conversation

17 May 2025

Papers citing "SALMONN-omni: A Standalone Speech LLM without Codec Injection for Full-duplex Conversation"

42 / 42 papers shown

Title
Kimi-Audio Technical Report KimiTeam Ding Ding Zeqian Ju Yichong Leng Shixuan Liu ... Zhiyong Yang Aoxiong Yin Ruibin Yuan Yanzhe Zhang Zaida Zhou AuLLM VLM 144 7 0 25 Apr 2025
Qwen2.5-Omni Technical Report Jin Xu Zhifang Guo Jinzheng He Hangrui Hu Ting He ... K. Dang Bin Zhang Xinyu Wang Yunfei Chu Junyang Lin VGen AuLLM 116 31 0 26 Mar 2025
Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics Siddhant Arora Zhiyun Lu Chung-Cheng Chiu Ruoming Pang Shinji Watanabe 66 5 0 03 Mar 2025
Baichuan-Audio: A Unified Framework for End-to-End Speech Interaction Tianpeng Li Qingbin Liu Tao Zhang Yuanbo Fang Zheng Liang ... Bin Cui Jianhua Xu Haoze Sun Guosheng Dong Xin Wu AuLLM 73 6 0 24 Feb 2025
Step-Audio: Unified Understanding and Generation in Intelligent Speech Interaction Ailin Huang Boyong Wu Bruce Wang Chao Yan Chen Hu ... Tianyu Wang Wenjin Deng Wuxun Xie Weipeng Ming Wenqing He AuLLM 99 12 0 17 Feb 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI Daya Guo Dejian Yang Haowei Zhang Junxiao Song ... Shiyu Wang S. Yu Shunfeng Zhou Shuting Pan S.S. Li ReLM VLM OffRL AI4TS LRM 251 1,503 0 22 Jan 2025
MinMo: A Multimodal Large Language Model for Seamless Voice Interaction Qian Chen Yafeng Chen Yanni Chen Mengzhe Chen Yuxiao Chen ... Shiliang Zhang Nan Zhao Pei Zhang Chuxu Zhang Jinren Zhou AuLLM MLLM 63 20 0 10 Jan 2025
Speech Recognition With LLMs Adapted to Disordered Speech Using Reinforcement Learning Chirag Nagpal Subhashini Venugopalan Jimmy Tobin Marilyn A. Ladewig Katherine Heller Katrin Tomanek 30 1 0 03 Jan 2025
Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLM Xiong Wang Yangze Li Chaoyou Fu Yunhang Shen Lei Xie Ke Li Xing Sun Long Ma AuLLM MLLM 66 35 0 01 Nov 2024
OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation Qinglin Zhang Luyao Cheng Chong Deng Qian Chen Wen Wang ... Jiaqing Liu Hai Yu Chaohong Tan Zhihao Du Shiliang Zhang SyDa BDL AuLLM VLM 75 18 0 23 Oct 2024
VoiceBench: Benchmarking LLM-Based Voice Assistants Yiming Chen Xianghu Yue Chen Zhang Xiaoxue Gao R. Tan Haoyang Li ELM AuLLM 56 24 0 22 Oct 2024
MT2KD: Towards A General-Purpose Encoder for Speech, Speaker, and Audio Events Xiaoyu Yang Qiujia Li Chao Zhang P. Woodland 52 1 0 25 Sep 2024
Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agents Bandhav Veluri Benjamin Peloquin Bokai Yu Hongyu Gong Shyamnath Gollakota AuLLM OffRL 70 17 0 23 Sep 2024
Moshi: a speech-text foundation model for real-time dialogue Alexandre Défossez Laurent Mazaré Manu Orsini Amélie Royer P. Pérez Hervé Jégou Edouard Grave Neil Zeghidour AuLLM 81 122 0 17 Sep 2024
LLaMA-Omni: Seamless Speech Interaction with Large Language Models Qingkai Fang Shoutao Guo Yan Zhou Zhengrui Ma Shaolei Zhang Yang Feng AuLLM 60 44 0 10 Sep 2024
Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming Zhifei Xie Changqiao Wu AuLLM VGen VLM SyDa LRM 45 65 0 29 Aug 2024
Qwen2-Audio Technical Report Yunfei Chu Jin Xu Qian Yang Haojie Wei Xipin Wei ... Yuanjun Lv Jinzheng He Junyang Lin Chang Zhou Jingren Zhou AuLLM VLM 50 129 0 15 Jul 2024
Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognition Ye Bai Jingping Chen Jitong Chen Wei Chen Zhuo Chen ... Wanyi Zhang Yang Zhang Yawei Zhang Yijie Zheng Ming Zou AuLLM 83 24 0 05 Jul 2024
Seed-TTS: A Family of High-Quality Versatile Speech Generation Models Philip Anastassiou Jiawei Chen Jingshu Chen Yuanzhe Chen Zhuo Chen ... Wenjie Zhang Yanzhe Zhang Zilin Zhao Dejian Zhong Xiaobin Zhuang 75 90 0 04 Jun 2024
Enhancing Zero-shot Text-to-Speech Synthesis with Human Feedback Chen Chen Yuchen Hu Wen Wu Helin Wang Chng Eng Siong Chao Zhang 58 12 0 02 Jun 2024
SpeechAlign: Aligning Speech Generation to Human Preferences Dong Zhang Zhaowei Li Shimin Li Xin Zhang Pengyu Wang Yaqian Zhou Xipeng Qiu ALM AuLLM 58 16 0 08 Apr 2024
SpiRit-LM: Interleaved Spoken and Written Language Model Tu Nguyen Benjamin Muller Bokai Yu Marta R. Costa-jussá Maha Elbayad ... Itai Gat Gabriel Synnaeve Juan Pino Benoît Sagot Emmanuel Dupoux AuLLM VLM 71 42 0 08 Feb 2024
Boosting Large Language Model for Speech Synthesis: An Empirical Study Hong-ping Hao Long Zhou Shujie Liu Jinyu Li Shujie Hu Rui Wang Furu Wei 85 19 0 30 Dec 2023
Mamba: Linear-Time Sequence Modeling with Selective State Spaces Albert Gu Tri Dao Mamba 97 2,552 0 01 Dec 2023
SALMONN: Towards Generic Hearing Abilities for Large Language Models Changli Tang Wenyi Yu Guangzhi Sun Xianzhao Chen Tian Tan Wei Li Lu Lu Zejun Ma Chao Zhang LM&MA AuLLM 56 232 0 20 Oct 2023
Libriheavy: a 50,000 hours ASR corpus with punctuation casing and context Wei Kang Xiaoyu Yang Zengwei Yao Fangjun Kuang Yifan Yang Liyong Guo Long Lin Daniel Povey 52 50 0 15 Sep 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 293 3,712 0 29 May 2023
Spoken Question Answering and Speech Continuation Using Spectrogram-Powered LLM Eliya Nachmani Alon Levkovitch Roy Hirsch Julián Salazar Chulayutsh Asawaroengchai Soroosh Mariooryad Ehud Rivlin RJ Skerry-Ryan Michelle Tadmor Ramanovich AuLLM 54 38 0 24 May 2023
SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities Dong Zhang Shimin Li Xin Zhang Jun Zhan Pengyu Wang Yaqian Zhou Xipeng Qiu AuLLM MLLM 89 320 0 18 May 2023
Listen, Think, and Understand Yuan Gong Hongyin Luo Alexander H. Liu Leonid Karlinsky James R. Glass ELM MLLM LRM 76 147 0 18 May 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 721 13,788 0 15 Mar 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 871 12,840 0 27 Feb 2023
Robust Speech Recognition via Large-Scale Weak Supervision Alec Radford Jong Wook Kim Tao Xu Greg Brockman C. McLeavey Ilya Sutskever OffRL 113 3,515 0 06 Dec 2022
Generative Spoken Dialogue Language Modeling Tu Nguyen Eugene Kharitonov Jade Copet Yossi Adi Wei-Ning Hsu ... Paden Tomasello Robin Algayres Benoît Sagot Abdel-rahman Mohamed Emmanuel Dupoux AuLLM 75 83 0 30 Mar 2022
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 268 10,099 0 17 Jun 2021
GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio Guoguo Chen Shuzhou Chai Guan-Bo Wang Jiayu Du Weiqiang Zhang ... Xuchen Yao Yongqing Wang Yujun Wang Zhao You Zhiyong Yan 86 360 0 13 Jun 2021
Proximal Policy Optimization Algorithms John Schulman Filip Wolski Prafulla Dhariwal Alec Radford Oleg Klimov OffRL 245 18,685 0 20 Jul 2017
Optimizing expected word error rate via sampling for speech recognition Matt Shannon 63 56 0 08 Jun 2017
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension Mandar Joshi Eunsol Choi Daniel S. Weld Luke Zettlemoyer RALM 173 2,610 0 09 May 2017
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 170 8,067 0 16 Jun 2016
Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models Iulian Serban Alessandro Sordoni Yoshua Bengio Aaron Courville Joelle Pineau AILaw 118 1,752 0 17 Jul 2015
A Neural Conversational Model Oriol Vinyals Quoc V. Le BDL 96 1,768 0 19 Jun 2015