Quality Over Quantity? LLM-Based Curation for a Data-Efficient Audio-Video Foundation Model

12 March 2025

Papers citing "Quality Over Quantity? LLM-Based Curation for a Data-Efficient Audio-Video Foundation Model"

39 / 39 papers shown

Title
From Vision to Audio and Beyond: A Unified Model for Audio-Visual Representation and Generation Kun Su Xiulong Liu Eli Shlizerman VGen 145 7 0 27 Sep 2024
EAGLE: Egocentric AGgregated Language-video Engine Jing Bi Yunlong Tang Luchuan Song Ali Vosoughi Nguyen Nguyen Chenliang Xu 88 10 0 26 Sep 2024
Meerkat: Audio-Visual Large Language Model for Grounding in Space and Time Sanjoy Chowdhury Sayan Nag Subhrajyoti Dasgupta Jun Chen Mohamed Elhoseiny Ruohan Gao Dinesh Manocha VLM MLLM 78 12 0 01 Jul 2024
video-SALMONN: Speech-Enhanced Audio-Visual Large Language Models Guangzhi Sun Wenyi Yu Changli Tang Xianzhao Chen Tian Tan Wei Li Lu Lu Zejun Ma Yuxuan Wang Chao Zhang 78 30 0 22 Jun 2024
Separating the "Chirp" from the "Chat": Self-supervised Visual Grounding of Sound and Language Mark Hamilton Andrew Zisserman John R. Hershey William T. Freeman VLM 110 7 0 09 Jun 2024
AVicuna: Audio-Visual LLM with Interleaver and Context-Boundary Alignment for Temporal Referential Dialogue Yunlong Tang Daiki Shimada Jing Bi Chenliang Xu VGen 49 17 0 24 Mar 2024
Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Aligners Yazhou Xing Yin-Yin He Zeyue Tian Xintao Wang Qifeng Chen 80 57 0 27 Feb 2024
OSCaR: Object State Captioning and State Change Representation Nguyen Nguyen Jing Bi Ali Vosoughi Yapeng Tian Pooyan Fazli Chenliang Xu 98 10 0 27 Feb 2024
Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities Zhifeng Kong Arushi Goel Rohan Badlani Ming-Yu Liu Rafael Valle Bryan Catanzaro AuLLM LM&MA MLLM 113 89 0 02 Feb 2024
Learning Audio Concepts from Counterfactual Natural Language Ali Vosoughi Luca Bondi Ho-Hsiang Wu Chenliang Xu CML 82 5 0 10 Jan 2024
Video Understanding with Large Language Models: A Survey Yunlong Tang Jing Bi Siting Xu Luchuan Song Susan Liang ... Feng Zheng Jianguo Zhang Chenliang Xu Jiebo Luo Chenliang Xu VLM 117 95 0 29 Dec 2023
Fusion of Audio and Visual Embeddings for Sound Event Localization and Detection Davide Berghi Peipei Wu Jinzheng Zhao Wenwu Wang Philip J. B. Jackson 73 10 0 14 Dec 2023
VTimeLLM: Empower LLM to Grasp Video Moments Bin Huang Xin Wang Hong Chen Zihan Song Wenwu Zhu MLLM 127 126 0 30 Nov 2023
SALMONN: Towards Generic Hearing Abilities for Large Language Models Changli Tang Wenyi Yu Guangzhi Sun Xianzhao Chen Tian Tan Wei Li Lu Lu Zejun Ma Chao Zhang LM&MA AuLLM 86 254 0 20 Oct 2023
Separating Invisible Sounds Toward Universal Audiovisual Scene-Aware Sound Separation Yiyang Su Ali Vosoughi Shijian Deng Yapeng Tian Chenliang Xu 48 4 0 18 Oct 2023
LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment Bin Zhu Bin Lin Munan Ning Yang Yan Jiaxi Cui ... Zongwei Li Wancai Zhang Zhifeng Li Wei Liu Liejie Yuan VLM MLLM 77 227 0 03 Oct 2023
Joint Audio and Speech Understanding Yuan Gong Alexander H. Liu Hongyin Luo Leonid Karlinsky James R. Glass AuLLM 61 80 0 25 Sep 2023
Whisper-AT: Noise-Robust Automatic Speech Recognizers are Also Strong General Audio Event Taggers Yuan Gong Sameer Khurana Leonid Karlinsky James R. Glass 63 71 0 06 Jul 2023
What Makes for Good Visual Tokenizers for Large Language Models? Guangzhi Wang Yixiao Ge Xiaohan Ding Mohan S. Kankanhalli Ying Shan MLLM VLM 64 39 0 20 May 2023
Pengi: An Audio Language Model for Audio Tasks Soham Deshmukh Benjamin Elizalde Rita Singh Huaming Wang MLLM AuLLM 77 180 0 19 May 2023
Listen, Think, and Understand Yuan Gong Hongyin Luo Alexander H. Liu Leonid Karlinsky James R. Glass ELM MLLM LRM 100 157 0 18 May 2023
ImageBind: One Embedding Space To Bind Them All Rohit Girdhar Alaaeldin El-Nouby Zhuang Liu Mannat Singh Kalyan Vasudev Alwala Armand Joulin Ishan Misra VLM 141 927 0 09 May 2023
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset Sihan Chen Xingjian He Longteng Guo Xinxin Zhu Weining Wang Jinhui Tang Jinhui Tang VLM 70 110 0 17 Apr 2023
DINOv2: Learning Robust Visual Features without Supervision Maxime Oquab Timothée Darcet Théo Moutakanni Huy Q. Vo Marc Szafraniec ... Hervé Jégou Julien Mairal Patrick Labatut Armand Joulin Piotr Bojanowski VLM CLIP SSL 325 3,410 0 14 Apr 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 426 4,563 0 30 Jan 2023
Robust Speech Recognition via Large-Scale Weak Supervision Alec Radford Jong Wook Kim Tao Xu Greg Brockman C. McLeavey Ilya Sutskever OffRL 191 3,684 0 06 Dec 2022
Beyond neural scaling laws: beating power law scaling via data pruning Ben Sorscher Robert Geirhos Shashank Shekhar Surya Ganguli Ari S. Morcos 95 441 0 29 Jun 2022
Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction Bowen Shi Wei-Ning Hsu Kushal Lakhotia Abdel-rahman Mohamed SSL 93 316 0 05 Jan 2022
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions Hongwei Xue Tiankai Hang Yanhong Zeng Yuchong Sun Bei Liu Huan Yang Jianlong Fu B. Guo AI4TS VLM 70 193 0 19 Nov 2021
Wav2CLIP: Learning Robust Audio Representations From CLIP Ho-Hsiang Wu Prem Seetharaman Kundan Kumar J. P. Bello CLIP VLM 145 269 0 21 Oct 2021
AudioCLIP: Extending CLIP to Image, Text and Audio A. Guzhov Federico Raue Jörn Hees Andreas Dengel CLIP VLM 119 366 0 24 Jun 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 929 29,436 0 26 Feb 2021
Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing Yapeng Tian Dingzeyu Li Chenliang Xu 97 184 0 21 Jul 2020
The EPIC-KITCHENS Dataset: Collection, Challenges and Baselines Dima Damen Hazel Doughty G. Farinella Sanja Fidler Antonino Furnari ... Davide Moltisanti Jonathan Munro Toby Perrett Will Price Michael Wray EgoV 68 232 0 29 Apr 2020
VGGSound: A Large-scale Audio-Visual Dataset Honglie Chen Weidi Xie Andrea Vedaldi Andrew Zisserman 89 577 0 29 Apr 2020
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips Antoine Miech Dimitri Zhukov Jean-Baptiste Alayrac Makarand Tapaswi Ivan Laptev Josef Sivic VGen 110 1,200 0 07 Jun 2019
The Sound of Motions Hang Zhao Chuang Gan Wei-Chiu Ma Antonio Torralba 80 254 0 11 Apr 2019
Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Input David Harwath Adrià Recasens Dídac Surís Galen Chuang Antonio Torralba James R. Glass 72 201 0 04 Apr 2018
Audio-Visual Event Localization in Unconstrained Videos Yapeng Tian Jing Shi Bochen Li Zhiyao Duan Chenliang Xu 99 435 0 23 Mar 2018