v1v2 (latest)

Audio-Language Datasets of Scenes and Events: A Survey

10 January 2025

Papers citing "Audio-Language Datasets of Scenes and Events: A Survey"

50 / 179 papers shown

Title
UniAudio: An Audio Foundation Model Toward Universal Audio Generation Dongchao Yang Jinchuan Tian Xuejiao Tan Rongjie Huang Songxiang Liu ... Jiang Bian Xixin Wu Zhou Zhao Shinji Watanabe Helen M. Meng CVBM AuLLM 84 127 0 01 Oct 2023
Improving Audio Captioning Models with Fine-grained Audio Features, Text Embedding Supervision, and LLM Mix-up Augmentation Shih-Lun Wu Xuankai Chang Gordon Wichern Jee-weon Jung Franccois G. Germain Jonathan Le Roux Shinji Watanabe 59 20 0 29 Sep 2023
Joint Audio and Speech Understanding Yuan Gong Alexander H. Liu Hongyin Luo Leonid Karlinsky James R. Glass AuLLM 72 80 0 25 Sep 2023
Weakly-supervised Automated Audio Captioning via text only training Theodoros Kouzelis Vassilis Katsouros CLIP 75 7 0 21 Sep 2023
A Large-scale Dataset for Audio-Language Representation Learning Luoyi Sun Xuenan Xu Mengyue Wu Weidi Xie 55 24 0 20 Sep 2023
ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillation Yatong Bai Trung D. Q. Dang Dung N. Tran K. Koishida Somayeh Sojoudi DiffM 98 23 0 19 Sep 2023
RECAP: Retrieval-Augmented Audio Captioning Sreyan Ghosh Sonal Kumar Chandra Kiran Reddy Evuru R. Duraiswami Tianyi Zhou VLM 90 20 0 18 Sep 2023
Contrastive Latent Space Reconstruction Learning for Audio-Text Retrieval Kaiyi Luo Xulong Zhang Jianzong Wang Huaxiong Li Ning Cheng Jing Xiao 100 2 0 16 Sep 2023
Audio Difference Learning for Audio Captioning Tatsuya Komatsu Yusuke Fujita K. Takeda Tomoki Toda 69 4 0 15 Sep 2023
Multilingual Audio Captioning using machine translated data Matéo Cousin Etienne Labbé Thomas Pellegrini 92 4 0 14 Sep 2023
Natural Language Supervision for General-Purpose Audio Representations Benjamin Elizalde Soham Deshmukh Huaming Wang AuLLM AI4TS 48 56 0 11 Sep 2023
Zero-Shot Audio Captioning via Audibility Guidance Tal Shaharabany Ariel Shaulov Lior Wolf 48 4 0 07 Sep 2023
CoNeTTE: An efficient Audio Captioning system leveraging multiple datasets with Task Embedding Etienne Labbé Thomas Pellegrini J. Pinquier 51 13 0 01 Sep 2023
Sparks of Large Audio Models: A Survey and Outlook S. Latif Moazzam Shoukat Fahad Shamshad Muhammad Usama Yi Ren ... Wenwu Wang Xulong Zhang Roberto Togneri Min Zhang Björn W. Schuller LM&MA AuLLM 113 39 0 24 Aug 2023
Audio Generation with Multiple Conditional Diffusion Model Zhifang Guo Jianguo Mao Ruijie Tao Long Yan Kazushige Ouchi Hong Liu Xiangdong Wang DiffM 53 14 0 23 Aug 2023
Audio Difference Captioning Utilizing Similarity-Discrepancy Disentanglement Daiki Takeuchi Yasunori Ohishi Daisuke Niizumi Noboru Harada K. Kashino 68 7 0 23 Aug 2023
Bridging High-Quality Audio and Video via Language for Sound Effects Retrieval from Visual Queries J. Wilkins Justin Salamon Magdalena Fuentes J. P. Bello Oriol Nieto CLIP 44 5 0 17 Aug 2023
AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining Haohe Liu Yiitan Yuan Xubo Liu Xinhao Mei Qiuqiang Kong Qiao Tian Yuping Wang Wenwu Wang Yuxuan Wang Mark D. Plumbley DiffM 87 246 0 10 Aug 2023
Separate Anything You Describe Xubo Liu Qiuqiang Kong Yan Zhao Haohe Liu Yiitan Yuan Yuzhuo Liu Rui Xia Yuxuan Wang Mark D. Plumbley Wenwu Wang VLM 67 51 0 09 Aug 2023
Transferable Models for Bioacoustics with Human Language Supervision David Robinson Adelaide Robinson Lily Akrapongpisak 60 8 0 09 Aug 2023
Advancing Natural-Language Based Audio Retrieval with PaSST and Large Audio-Caption Data Sets Paul Primus Khaled Koutini Gerhard Widmer 53 13 0 08 Aug 2023
From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusion Robin San Roman Yossi Adi Antoine Deleforge Romain Serizel Gabriel Synnaeve Alexandre Défossez DiffM 56 24 0 02 Aug 2023
WavJourney: Compositional Audio Creation with Large Language Models Xubo Liu Zhongkai Zhu Haohe Liu Yiitan Yuan Meng Cui ... Jinhua Liang Yin Cao Qiuqiang Kong Mark D. Plumbley Wenwu Wang AuLLM 49 25 0 26 Jul 2023
BuboGPT: Enabling Visual Grounding in Multi-Modal LLMs Yang Zhao Zhijie Lin Daquan Zhou Zilong Huang Jiashi Feng Bingyi Kang MLLM 73 111 0 17 Jul 2023
A Comprehensive Overview of Large Language Models Humza Naveed Asad Ullah Khan Shi Qiu Muhammad Saqib Saeed Anwar Muhammad Usman Naveed Akhtar Nick Barnes Ajmal Mian OffRL 101 595 0 12 Jul 2023
A Survey on Multimodal Large Language Models Shukang Yin Chaoyou Fu Sirui Zhao Ke Li Xing Sun Tong Xu Enhong Chen MLLM LRM 117 591 0 23 Jun 2023
Crowdsourcing and Evaluating Text-Based Audio Retrieval Relevances Huang Xie Khazar Khorrami Okko Räsänen Tuomas Virtanen 45 4 0 16 Jun 2023
FALL-E: A Foley Sound Synthesis Model and Strategies Minsung Kang Sangshin Oh Hyeongi Moon Kyungyun Lee Ben Sangbae Chon 30 4 0 16 Jun 2023
Valley: Video Assistant with Large Language model Enhanced abilitY Ruipu Luo Ziwang Zhao Min Yang Junwei Dong Da Li Pengcheng Lu Tao Wang Linmei Hu Ming-Hui Qiu MLLM 106 205 0 12 Jun 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 359 4,388 0 09 Jun 2023
Attention-Based Methods For Audio Question Answering Parthasaarathy Sudarsanam Tuomas Virtanen 56 3 0 31 May 2023
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset Sihan Chen Handong Li Qunbo Wang Zijia Zhao Ming-Ting Sun Xinxin Zhu Qingbin Liu 140 105 0 29 May 2023
Multi-Scale Attention for Audio Question Answering Guangyao Li Yixin Xu Di Hu 32 16 0 29 May 2023
CAPTDURE: Captioned Sound Dataset of Single Sources Yuki Okamoto Kanta Shimonishi Keisuke Imoto Kota Dohi Shota Horiguchi Yohei Kawaguchi 45 1 0 28 May 2023
ChatBridge: Bridging Modalities with Large Language Model as a Language Catalyst Zijia Zhao Longteng Guo Tongtian Yue Si-Qing Chen Shuai Shao Xinxin Zhu Zehuan Yuan Jing Liu MLLM 80 58 0 25 May 2023
Any-to-Any Generation via Composable Diffusion Zineng Tang Ziyi Yang Chenguang Zhu Michael Zeng Joey Tianyi Zhou VGen DiffM 93 184 0 19 May 2023
Pengi: An Audio Language Model for Audio Tasks Soham Deshmukh Benjamin Elizalde Rita Singh Huaming Wang MLLM AuLLM 77 180 0 19 May 2023
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities Peng Wang Shijie Wang Junyang Lin Shuai Bai Xiaohuan Zhou Jingren Zhou Xinggang Wang Chang Zhou VLM MLLM ObjD 99 122 0 18 May 2023
Listen, Think, and Understand Yuan Gong Hongyin Luo Alexander H. Liu Leonid Karlinsky James R. Glass ELM MLLM LRM 100 157 0 18 May 2023
PaLM 2 Technical Report Rohan Anil Andrew M. Dai Orhan Firat Melvin Johnson Dmitry Lepikhin ... Ce Zheng Wei Zhou Denny Zhou Slav Petrov Yonghui Wu ReLM LRM 187 1,192 0 17 May 2023
A Whisper transformer for audio captioning trained with synthetic captions and transfer learning Marek Kadlcík Adam Hájek Jürgen Kieslich Radoslaw Winiecki VLM 53 11 0 15 May 2023
VideoChat: Chat-Centric Video Understanding Kunchang Li Yinan He Yi Wang Yizhuo Li Wen Wang Ping Luo Yali Wang Limin Wang Yu Qiao MLLM 92 578 0 10 May 2023
Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model Deepanway Ghosal Navonil Majumder Ambuj Mehrish Soujanya Poria 197 151 0 24 Apr 2023
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models Deyao Zhu Jun Chen Xiaoqian Shen Xiang Li Mohamed Elhoseiny VLM MLLM 153 2,033 0 20 Apr 2023
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset Sihan Chen Xingjian He Longteng Guo Xinxin Zhu Weining Wang Jinhui Tang Jinhui Tang VLM 76 110 0 17 Apr 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 226 613 0 06 Apr 2023
Prefix tuning for automated audio captioning Minkyu Kim Kim Sung-Bin Tae-Hyun Oh 66 45 0 30 Mar 2023
WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research Xinhao Mei Chutong Meng Haohe Liu Qiuqiang Kong Tom Ko Chengqi Zhao Mark D. Plumbley Yuexian Zou Wenwu Wang 117 211 0 30 Mar 2023
Fine-grained Audible Video Description Xuyang Shen Dong Li Jinxing Zhou Zhen Qin Bowen He ... Yuchao Dai Lingpeng Kong Meng Wang Yu Qiao Yiran Zhong VGen 75 11 0 27 Mar 2023
Audio-Text Models Do Not Yet Leverage Natural Language Ho-Hsiang Wu Oriol Nieto J. P. Bello Justin Salamon VLM 55 33 0 19 Mar 2023