Pengi: An Audio Language Model for Audio Tasks

19 May 2023

Papers citing "Pengi: An Audio Language Model for Audio Tasks"

50 / 122 papers shown

Title
HAKES: Scalable Vector Database for Embedding Search Service Guoyu Hu Shaofeng Cai Tien Tuan Anh Dinh Zhongle Xie Cong Yue Gang Chen Beng Chin Ooi 2 0 0 18 May 2025
Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM? Andrew Rouditchenko Saurabhchand Bhati Edson Araujo Samuel Thomas Hilde Kuehne Rogerio Feris James R. Glass AuLLM VLM 44 0 0 14 May 2025
Multi-Domain Audio Question Answering Toward Acoustic Content Reasoning in The DCASE 2025 Challenge Chao-Han Huck Yang Sreyan Ghosh Qing Wang Jaeyeon Kim Hengyi Hong ... Tianyi Zhou Gunhee Kim Jun Du Rafael Valle Bryan Catanzaro 36 0 0 12 May 2025
CaReAQA: A Cardiac and Respiratory Audio Question Answering Model for Open-Ended Diagnostic Reasoning Tsai-Ning Wang Lin-Lin Chen Neil Zeghidour Aaqib Saeed AuLLM LM&MA 173 0 0 02 May 2025
A Survey of Interactive Generative Video Jiwen Yu Yiran Qin Haoxuan Che Quande Liu Xinyu Wang Pengfei Wan Di Zhang Kun Gai Hao Chen Xihui Liu VGen 65 0 0 30 Apr 2025
Enhancing Non-Core Language Instruction-Following in Speech LLMs via Semi-Implicit Cross-Lingual CoT Reasoning Hongfei Xue Yufeng Tang Hexin Liu Jun Zhang Xuelong Geng Lei Xie LRM 57 0 0 29 Apr 2025
Transformation of audio embeddings into interpretable, concept-based representations Alice Zhang Edison Thomaz Lie Lu 29 0 0 18 Apr 2025
Make Some Noise: Towards LLM audio reasoning and generation using sound tokens Shivam Mehta Nebojsa Jojic Hannes Gamper 31 0 0 28 Mar 2025
Qwen2.5-Omni Technical Report Jin Xu Zhifang Guo Jinzheng He Hangrui Hu Ting He ... K. Dang Bin Zhang Xinyu Wang Yunfei Chu Junyang Lin VGen AuLLM 96 16 0 26 Mar 2025
Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector Xiao Guo Xiufeng Song Yue Zhang Xiaohong Liu X. Liu 63 1 0 26 Mar 2025
Position: Interactive Generative Video as Next-Generation Game Engine Jiwen Yu Yiran Qin Haoxuan Che Quande Liu Xintao Wang Pengfei Wan Di Zhang Xihui Liu VGen 45 1 0 21 Mar 2025
Quality Over Quantity? LLM-Based Curation for a Data-Efficient Audio-Video Foundation Model Ali Vosoughi Dimitra Emmanouilidou H. Gamper 55 0 0 12 Mar 2025
Mellow: a small audio language model for reasoning Soham Deshmukh Satvik Dixit Rita Singh Bhiksha Raj AuLLM ReLM LRM 78 2 0 11 Mar 2025
CL-MoE: Enhancing Multimodal Large Language Model with Dual Momentum Mixture-of-Experts for Continual Visual Question Answering Tianyu Huai Jie Zhou Xingjiao Wu Qin Chen Qingchun Bai Ze Zhou Liang He MoE 38 2 0 01 Mar 2025
Baichuan-Audio: A Unified Framework for End-to-End Speech Interaction Tianpeng Li Jiaheng Liu Tao Zhang Yuanbo Fang Zhuoran Zhang ... Guosheng Dong Jianhua Xu Haoze Sun Zenan Zhou Xin Wu AuLLM 61 3 0 24 Feb 2025
Soundwave: Less is More for Speech-Text Alignment in LLMs Yunke Zhang Zhiheng Liu Fan Bu Ruiyu Zhang Benyou Wang Yiming Li AuLLM SyDa VLM 107 0 0 18 Feb 2025
From No to Know: Taxonomy, Challenges, and Opportunities for Negation Understanding in Multimodal Foundation Models Mayank Vatsa Aparna Bharati S. Mittal Richa Singh 58 0 0 10 Feb 2025
Unbiased Sliced Wasserstein Kernels for High-Quality Audio Captioning Manh Luong Khai Nguyen Dinh Q. Phung Gholamreza Haffari Lizhen Qu 47 0 0 08 Feb 2025
Audio-Language Models for Audio-Centric Tasks: A survey Yi Su Jisheng Bai Qisheng Xu Kele Xu Yong Dou AuLLM 99 2 0 28 Jan 2025
AudioBERT: Audio Knowledge Augmented Language Model Hyunjong Ok Suho Yoo Jaeho Lee AuLLM RALM VLM 53 0 0 17 Jan 2025
Audio-Language Datasets of Scenes and Events: A Survey Gijs Wijngaard Elia Formisano Michele Esposito M. Dumontier 81 2 0 10 Jan 2025
OneLLM: One Framework to Align All Modalities with Language Jiaming Han Kaixiong Gong Yiyuan Zhang Jiaqi Wang Kaipeng Zhang Dahua Lin Yu Qiao Peng Gao Xiangyu Yue MLLM 106 109 0 10 Jan 2025
"Yeah Right!" -- Do LLMs Exhibit Multimodal Feature Transfer? Benjamin Z. Reichman Kartik Talamadupula 41 0 0 07 Jan 2025
Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoning Chun-Yi Kuan Hung-yi Lee AuLLM LRM 72 1 0 03 Jan 2025
Instruction-Guided Scene Text Recognition Yongkun Du Z. Chen Yuchen Su Caiyan Jia Yu-Gang Jiang 75 3 0 03 Jan 2025
Multiple Consistency-guided Test-Time Adaptation for Contrastive Audio-Language Models with Unlabeled Audio Gongyu Chen Haomin Zhang Chaofan Ding Zihao Chen Xinhan Di 37 0 0 23 Dec 2024
Empowering LLMs to Understand and Generate Complex Vector Graphics Ximing Xing Juncheng Hu Guotao Liang Jing Zhang Dong Xu Qian Yu 94 7 0 15 Dec 2024
MotionLLaMA: A Unified Framework for Motion Synthesis and Comprehension Zeyu Ling Bo Han Shiyang Li H. Shen Jikang Cheng Changqing Zou 81 1 0 26 Nov 2024
State-Space Large Audio Language Models Saurabhchand Bhati Yuan Gong Leonid Karlinsky Hilde Kuehne Rogerio Feris James Glass 99 0 0 24 Nov 2024
MACE: Leveraging Audio for Evaluating Audio Captioning Systems Satvik Dixit Soham Deshmukh Bhiksha Raj 35 60 0 01 Nov 2024
MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark S. Sakshi Utkarsh Tyagi Sonal Kumar Ashish Seth Ramaneswaran Selvakumar Oriol Nieto R. Duraiswami Sreyan Ghosh Dinesh Manocha AuLLM ELM 75 23 0 24 Oct 2024
AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Models Kim Sung-Bin Oh Hyun-Bin JungMok Lee Arda Senocak Joon Son Chung Tae-Hyun Oh MLLM VLM 48 3 0 23 Oct 2024
Generative AI Agents in Autonomous Machines: A Safety Perspective Jason J. Jabbour Vijay Janapa Reddi AI4CE 43 4 0 20 Oct 2024
Ichigo: Mixed-Modal Early-Fusion Realtime Voice Assistant Alan Dao Dinh Bach Vu Huy Hoang Ha AuLLM VLM 73 3 0 20 Oct 2024
Roadmap towards Superhuman Speech Understanding using Large Language Models Fan Bu Yuhao Zhang Xuben Wang Benyou Wang Qiang Liu Yiming Li LM&MA ELM AuLLM 165 1 0 17 Oct 2024
An Eye for an Ear: Zero-shot Audio Description Leveraging an Image Captioner using Audiovisual Distribution Alignment Hugo Malard Michel Olvera Stéphane Lathuilière S. Essid VLM 34 0 0 08 Oct 2024
MINER: Mining the Underlying Pattern of Modality-Specific Neurons in Multimodal Large Language Models Kaichen Huang Jiahao Huo Yibo Yan Kun Wang Yutao Yue Xuming Hu 39 2 0 07 Oct 2024
Distilling an End-to-End Voice Assistant Without Instruction Training Data William B. Held Ella Li Michael Joseph Ryan Weiyan Shi Yanzhe Zhang Diyi Yang AuLLM 47 8 0 03 Oct 2024
PALM: Few-Shot Prompt Learning for Audio Language Models Asif Hanif M. Agro Mohammad Areeb Qazi Hanan Aldarmaki VLM 21 1 0 29 Sep 2024
Beyond Single-Audio: Advancing Multi-Audio Processing in Audio Large Language Models Yiming Chen Xianghu Yue Xiaoxue Gao Chen Zhang L. F. D’Haro R. Tan Haizhou Li AuLLM 32 0 0 27 Sep 2024
Semi-intrusive audio evaluation: Casting non-intrusive assessment as a multi-modal text prediction task Jozef Coldenhoff Milos Cernak 41 0 0 21 Sep 2024
Large Language Models are Strong Audio-Visual Speech Recognition Learners Umberto Cappellazzo Minsu Kim Honglie Chen Pingchuan Ma Stavros Petridis Daniele Falavigna Alessio Brutti Maja Pantic 36 9 0 18 Sep 2024
Integrating Audio Narrations to Strengthen Domain Generalization in Multimodal First-Person Action Recognition Cagri Gungor Adriana Kovashka EgoV 33 0 0 15 Sep 2024
Towards Diverse and Efficient Audio Captioning via Diffusion Models Manjie Xu Chenxing Li Xinyi Tu Yong Ren Ruibo Fu Wei Liang Dong Yu DiffM 49 1 0 14 Sep 2024
ReCLAP: Improving Zero Shot Audio Classification by Describing Sounds Sreyan Ghosh Sonal Kumar Chandra Kiran Reddy Evuru Oriol Nieto R. Duraiswami Dinesh Manocha VLM 37 3 0 13 Sep 2024
TSELM: Target Speaker Extraction using Discrete Tokens and Language Models Beilong Tang Bang Zeng Ming Li 35 2 0 12 Sep 2024
Enhancing Temporal Understanding in Audio Question Answering for Large Audio Language Models A. Sridhar Yinyi Guo Erik M. Visser AuLLM 27 0 0 10 Sep 2024
LLaMA-Omni: Seamless Speech Interaction with Large Language Models Qingkai Fang Shoutao Guo Yan Zhou Zhengrui Ma Shaolei Zhang Yang Feng AuLLM 33 30 0 10 Sep 2024
MoWE-Audio: Multitask AudioLLMs with Mixture of Weak Encoders Feiyu Xiong Shuo Sun Bin Wang Xunlong Zou Zhuohan Liu Yingxu He Geyu Lin Nancy F. Chen A. Aw AuLLM 67 1 0 10 Sep 2024
EnCLAP++: Analyzing the EnCLAP Framework for Optimizing Automated Audio Captioning Performance Jaeyeon Kim Minjeon Jeon Jaeyoon Jung Sang Hoon Woo Jinjoo Lee 34 2 0 02 Sep 2024