LiSTEN: Learning Soft Token Embeddings for Neural Audio LLMs

24 May 2025

Papers citing "LiSTEN: Learning Soft Token Embeddings for Neural Audio LLMs"

31 / 31 papers shown

Title
LAST SToP For Modeling Asynchronous Time Series Shubham Gupta Thibaut Durand Graham Taylor Lilian W. Białokozowicz AI4TS 86 1 0 04 Feb 2025
Chain-of-Thought Prompting for Speech Translation Ke Hu Zhehuai Chen Chao-Han Huck Yang Piotr Żelasko Oleksii Hrinchuk Vitaly Lavrukhin Jagadeesh Balam Boris Ginsburg LRM 140 9 0 17 Sep 2024
Language Model Can Listen While Speaking Ziyang Ma Yakun Song Chenpeng Du Jian Cong Zhuo Chen Yuping Wang Yansen Wang Xie Chen AuLLM 91 28 0 05 Aug 2024
Qwen2-Audio Technical Report Yunfei Chu Jin Xu Qian Yang Haojie Wei Xipin Wei ... Yuanjun Lv Jinzheng He Junyang Lin Chang Zhou Jingren Zhou AuLLM VLM 64 152 0 15 Jul 2024
DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignment Ke-Han Lu Zhehuai Chen Szu-Wei Fu He Huang Boris Ginsburg Yu-Chiang Frank Wang Hung-yi Lee VLM AuLLM 78 19 0 27 Jun 2024
AudioBench: A Universal Benchmark for Audio Large Language Models Bin Wang Xunlong Zou Geyu Lin Siyang Song Zhuohan Liu Wenyu Zhang Zhengyuan Liu AiTi Aw Nancy F. Chen AuLLM ELM LM&MA 140 34 0 23 Jun 2024
WavLLM: Towards Robust and Adaptive Speech Large Language Model Shujie Hu Long Zhou Shujie Liu Sanyuan Chen Hongkun Hao ... Xunying Liu Jinyu Li S. Sivasankaran Linquan Liu Furu Wei AuLLM 84 62 0 31 Mar 2024
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling Jun Zhan Junqi Dai Jiasheng Ye Yunhua Zhou Dong Zhang ... Jie Fu Tao Gui Tianxiang Sun Yugang Jiang Xipeng Qiu MLLM 67 132 0 19 Feb 2024
Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities Zhifeng Kong Arushi Goel Rohan Badlani Ming-Yu Liu Rafael Valle Bryan Catanzaro AuLLM LM&MA MLLM 113 89 0 02 Feb 2024
SALMONN: Towards Generic Hearing Abilities for Large Language Models Changli Tang Wenyi Yu Guangzhi Sun Xianzhao Chen Tian Tan Wei Li Lu Lu Zejun Ma Chao Zhang LM&MA AuLLM 86 254 0 20 Oct 2023
Instruction Tuning for Large Language Models: A Survey Shengyu Zhang Linfeng Dong Xiaoya Li Sen Zhang Xiaofei Sun ... Jiwei Li Runyi Hu Tianwei Zhang Leilei Gan Guoyin Wang LM&MA 83 597 0 21 Aug 2023
AudioPaLM: A Large Language Model That Can Speak and Listen Paul Kishan Rubenstein Chulayuth Asawaroengchai D. Nguyen Ankur Bapna Zalan Borsos ... Neil Zeghidour Yu Zhang Zhishuai Zhang Lukás Zilka Christian Frank LM&MA AuLLM VLM 96 288 0 22 Jun 2023
SpeechGen: Unlocking the Generative Power of Speech Language Models with Prompts Haibin Wu Kai-Wei Chang Yuan-Kuei Wu Hung-yi Lee 93 23 0 03 Jun 2023
QLoRA: Efficient Finetuning of Quantized LLMs Tim Dettmers Artidoro Pagnoni Ari Holtzman Luke Zettlemoyer ALM 147 2,555 0 23 May 2023
Pengi: An Audio Language Model for Audio Tasks Soham Deshmukh Benjamin Elizalde Rita Singh Huaming Wang MLLM AuLLM 77 180 0 19 May 2023
Listen, Think, and Understand Yuan Gong Hongyin Luo Alexander H. Liu Leonid Karlinsky James R. Glass ELM MLLM LRM 100 157 0 18 May 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 226 613 0 06 Apr 2023
SpeechPrompt v2: Prompt Tuning for Speech Classification Tasks Kai-Wei Chang Yu-Kai Wang Hua Shen Iu-thing Kang Wei-Cheng Tseng Shang-Wen Li Hung-yi Lee VLM 67 45 0 01 Mar 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.5K 13,247 0 27 Feb 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 426 4,563 0 30 Jan 2023
BEATs: Audio Pre-Training with Acoustic Tokenizers Sanyuan Chen Yu-Huan Wu Chengyi Wang Shujie Liu Daniel C. Tompkins Zhuo Chen Furu Wei 103 288 0 18 Dec 2022
Robust Speech Recognition via Large-Scale Weak Supervision Alec Radford Jong Wook Kim Tao Xu Greg Brockman C. McLeavey Ilya Sutskever OffRL 191 3,684 0 06 Dec 2022
DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning Zifeng Wang Zizhao Zhang Sayna Ebrahimi Ruoxi Sun Han Zhang ... Xiaoqi Ren Guolong Su Vincent Perot Jennifer Dy Tomas Pfister CLL VLM VPVLM 117 490 0 10 Apr 2022
SpeechPrompt: An Exploration of Prompt Tuning on Generative Spoken Language Model for Speech Processing Tasks Kai-Wei Chang Wei-Cheng Tseng Shang-Wen Li Hung-yi Lee 78 23 0 31 Mar 2022
Learning to Prompt for Continual Learning Zifeng Wang Zizhao Zhang Chen-Yu Lee Han Zhang Ruoxi Sun Xiaoqi Ren Guolong Su Vincent Perot Jennifer Dy Tomas Pfister CLL VPVLM KELM VLM 101 775 0 16 Dec 2021
SoundStream: An End-to-End Neural Audio Codec Neil Zeghidour Alejandro Luebs Ahmed Omran Jan Skoglund Marco Tagliasacchi AI4TS 110 791 0 07 Jul 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 466 10,367 0 17 Jun 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 570 4,047 0 18 Apr 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 242 4,261 0 01 Jan 2021
Common Voice: A Massively-Multilingual Speech Corpus Rosana Ardila Megan Branson Kelly Davis Michael Henretty M. Kohler Josh Meyer Reuben Morais Lindsay Saunders Francis M. Tyers Gregor Weber VLM 91 1,600 0 13 Dec 2019
VoxCeleb: a large-scale speaker identification dataset Arsha Nagrani Joon Son Chung Andrew Zisserman 125 2,274 0 26 Jun 2017