Joint Audio and Speech Understanding

25 September 2023

Papers citing "Joint Audio and Speech Understanding"

21 / 21 papers shown

Title
Analyzing Mitigation Strategies for Catastrophic Forgetting in End-to-End Training of Spoken Language Models Chi-Yuan Hsiao Ke-Han Lu Kai-Wei Chang Chih-Kai Yang Wei-Chih Chen Hung-yi Lee CLL MoMe 104 0 0 23 May 2025
Large Language Models Implicitly Learn to See and Hear Just By Reading Prateek Verma Mert Pilanci 72 0 0 20 May 2025
DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data Ke-Han Lu Zhehuai Chen Szu-Wei Fu Chao-Han Huck Yang Jagadeesh Balam Boris Ginsburg Yu-Te Wang Hung-yi Lee AuLLM SyDa 128 13 0 28 Jan 2025
SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words Junyi Ao Yuancheng Wang Xiaohai Tian Dekun Chen Jing Zhang Lu Lu Yansen Wang Haizhou Li Zhikai Wu AuLLM 111 21 0 17 Jan 2025
Audio-Language Datasets of Scenes and Events: A Survey Gijs Wijngaard Elia Formisano Michele Esposito M. Dumontier 111 2 0 10 Jan 2025
Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoning Chun-Yi Kuan Hung-yi Lee AuLLM LRM 94 5 0 03 Jan 2025
The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalities Zhaofeng Wu Xinyan Velocity Yu Dani Yogatama Jiasen Lu Yoon Kim AIFin 68 17 0 07 Nov 2024
Frozen Large Language Models Can Perceive Paralinguistic Aspects of Speech Wonjune Kang Junteng Jia Chunyang Wu Wei Zhou Egor Lakomkin ... Leda Sari Suyoun Kim Ke Li Jay Mahadeokar Ozlem Kalinli AuLLM 67 4 0 02 Oct 2024
What Are They Doing? Joint Audio-Speech Co-Reasoning Yingzhi Wang Pooneh Mousavi Artem Ploujnikov Mirco Ravanelli AuLLM 56 2 0 22 Sep 2024
Chain-of-Thought Prompting for Speech Translation Ke Hu Zhehuai Chen Chao-Han Huck Yang Piotr Żelasko Oleksii Hrinchuk Vitaly Lavrukhin Jagadeesh Balam Boris Ginsburg LRM 90 6 0 17 Sep 2024
Enhancing Low-Resource Language and Instruction Following Capabilities of Audio Language Models Potsawee Manakul Guangzhi Sun Warit Sirichotedumrong Kasima Tharnpipitchai Kunat Pipatanakul AuLLM 56 6 0 17 Sep 2024
MoWE-Audio: Multitask AudioLLMs with Mixture of Weak Encoders Wentao Zhang Shuo Sun Bin Wang Xunlong Zou Zhuohan Liu Yingxu He Geyu Lin Nancy F. Chen Ai Ti Aw AuLLM 76 1 0 10 Sep 2024
PEFT-SER: On the Use of Parameter Efficient Transfer Learning Approaches For Speech Emotion Recognition Using Pre-trained Speech Models Tiantian Feng Shrikanth Narayanan 56 29 0 08 Jun 2023
WAVPROMPT: Towards Few-Shot Spoken Language Understanding with Frozen Language Models Heting Gao Junrui Ni Kaizhi Qian Yang Zhang Shiyu Chang M. Hasegawa-Johnson VLM 129 31 0 29 Mar 2022
HEAR: Holistic Evaluation of Audio Representations Joseph P. Turian Jordie Shier H. Khan Bhiksha Raj Björn W. Schuller ... P. Esling Pranay Manocha Shinji Watanabe Zeyu Jin Yonatan Bisk 58 103 0 06 Mar 2022
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing Sanyuan Chen Chengyi Wang Zhengyang Chen Yu-Huan Wu Shujie Liu ... Yao Qian Jian Wu Micheal Zeng Xiangzhan Yu Furu Wei SSL 173 1,794 0 26 Oct 2021
BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition Yu Zhang Daniel S. Park Wei Han James Qin Anmol Gulati ... Zhifeng Chen Quoc V. Le Chung-Cheng Chiu Ruoming Pang Yonghui Wu SSL 47 174 0 27 Sep 2021
AudioCLIP: Extending CLIP to Image, Text and Audio A. Guzhov Federico Raue Jörn Hees Andreas Dengel CLIP VLM 57 362 0 24 Jun 2021
PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation Yuan Gong Yu-An Chung James R. Glass VLM 142 144 0 02 Feb 2021
LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech Heiga Zen Viet Dang R. Clark Yu Zhang Ron J. Weiss Ye Jia Zhiwen Chen Yonghui Wu 68 933 0 05 Apr 2019
Bottom-up Broadcast Neural Network For Music Genre Classification Caifeng Liu Lin Feng Guochao Liu Huibing Wang Sheng-lan Liu 33 108 0 24 Jan 2019