v1v2 (latest)

VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset

29 May 2023

ArXiv (abs)PDF HTML Github (277★)

Papers citing "VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset"

50 / 79 papers shown

Title
MEXA: Towards General Multimodal Reasoning with Dynamic Multi-Expert Aggregation Shoubin Yu Yue Zhang Ziyang Wang Jaehong Yoon Mohit Bansal MoE LRM 15 0 0 20 Jun 2025
Q2E: Query-to-Event Decomposition for Zero-Shot Multilingual Text-to-Video Retrieval Shubhashis Roy Dipta Francis Ferraro 53 0 0 11 Jun 2025
DiscoVLA: Discrepancy Reduction in Vision, Language, and Alignment for Parameter-Efficient Video-Text Retrieval Leqi Shen Guoqiang Gong Tianxiang Hao Tao He Yifeng Zhang Pengzhang Liu Sicheng Zhao Jungong Han Guiguang Ding 24 0 0 10 Jun 2025
MAGNET: A Multi-agent Framework for Finding Audio-Visual Needles by Reasoning over Multi-Video Haystacks Sanjoy Chowdhury Mohamed Elmoghany Yohan Abeysinghe Junjie Fei Sayan Nag Salman Khan Mohamed Elhoseiny Dinesh Manocha 35 0 0 08 Jun 2025
CLaMR: Contextualized Late-Interaction for Multimodal Content Retrieval David Wan Han Wang Elias Stengel-Eskin Jaemin Cho Mohit Bansal VLM Presented at ResearchTrend Connect \| VLM on 02 Jul 2025 56 0 0 06 Jun 2025
Leveraging Auxiliary Information in Text-to-Video Retrieval: A Review A. Fragomeni Dima Damen Michael Wray 27 0 0 29 May 2025
HuMoCon: Concept Discovery for Human Motion Understanding Qihang Fang Chengcheng Tang Bugra Tekin Shugao Ma Yanchao Yang 43 0 0 27 May 2025
RTV-Bench: Benchmarking MLLM Continuous Perception, Understanding and Reasoning through Real-Time Video Shuhang Xun Sicheng Tao Jiajun Li Yibo Shi Zhixin Lin ... Shikang Wang Yang Liu Hao Zhang Ying Ma Xuming Hu VLM LRM 100 1 0 04 May 2025
VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models Weiye Xu Jun Wang Weiyun Wang Zhe Chen Wengang Zhou ... Xiaohua Wang Xizhou Zhu Wenhai Wang Jifeng Dai Jinguo Zhu VLM LRM 186 7 0 21 Apr 2025
A Survey on Cross-Modal Interaction Between Music and Multimodal Data Sifei Li Mining Tan Feier Shen Minyan Luo Zijiao Yin Fan Tang W. Dong Changsheng Xu 120 1 0 17 Apr 2025
LVC: A Lightweight Compression Framework for Enhancing VLMs in Long Video Understanding Ziyi Wang Haoran Wu Yiming Rong Deyang Jiang Yixin Zhang Yue Zhao Shuang Xu Bo Xu VLM 99 0 0 09 Apr 2025
Safety Modulation: Enhancing Safety in Reinforcement Learning through Cost-Modulated Rewards Hanping Zhang Yuhong Guo OffRL 118 0 0 03 Apr 2025
Leveraging Modality Tags for Enhanced Cross-Modal Video Retrieval A. Fragomeni Dima Damen Michael Wray 110 0 0 02 Apr 2025
WikiVideo: Article Generation from Multiple Videos Alexander Martin Reno Kriz William Walden Kate Sanders Hannah Recknor Eugene Yang Francis Ferraro Benjamin Van Durme DiffM VGen 154 2 0 01 Apr 2025
Chapter-Llama: Efficient Chaptering in Hour-Long Videos with LLMs Lucas Ventura Antoine Yang Cordelia Schmid Gül Varol 101 0 0 31 Mar 2025
Aurelia: Test-time Reasoning Distillation in Audio-Visual LLMs Sanjoy Chowdhury Hanan Gani Nishit Anand Sayan Nag Ruohan Gao Mohamed Elhoseiny Salman Khan Dinesh Manocha LRM 184 1 0 29 Mar 2025
MMMORRF: Multimodal Multilingual Modularized Reciprocal Rank Fusion Saron Samuel Dan DeGenaro Jimena Guallar-Blasco Kate Sanders Oluwaseun Eisape ... David Etter Efsun Kayi Matthew Wiesner Kenton W. Murray Reno Kriz 185 0 0 26 Mar 2025
PAVE: Patching and Adapting Video Large Language Models Zhuoming Liu Yiquan Li Khoi Duc Nguyen Yiwu Zhong Yin Li KELM LRM 134 1 0 25 Mar 2025
ACVUBench: Audio-Centric Video Understanding Benchmark Yue Yang Jimin Zhuang Guangzhi Sun Changli Tang Yongqian Li P. Li Yifan Jiang W. Li Zejun Ma Chao Zhang AuLLM CoGe 116 0 0 25 Mar 2025
Unbiasing through Textual Descriptions: Mitigating Representation Bias in Video Benchmarks Nina Shvetsova Arsha Nagrani Bernt Schiele Hilde Kuehne Christian Rupprecht 87 1 0 24 Mar 2025
Video-ColBERT: Contextualized Late Interaction for Text-to-Video Retrieval Arun V. Reddy Alexander Martin Eugene Yang Andrew Yates Kate Sanders Kenton W. Murray Reno Kriz Celso M. De Melo Benjamin Van Durme Rama Chellappa 121 1 0 24 Mar 2025
Quantum EigenGame for excited state calculation David Quiroga Jason Han Anastasios Kyrillidis 116 0 0 17 Mar 2025
DAVE: Diagnostic benchmark for Audio Visual Evaluation Gorjan Radevski Teodora Popordanoska Matthew B. Blaschko Tinne Tuytelaars 92 0 0 12 Mar 2025
HarmonySet: A Comprehensive Dataset for Understanding Video-Music Semantic Alignment and Temporal Synchronization Zitang Zhou Ke Mei Yu Lu Tianyi Wang Fengyun Rao 137 2 0 03 Mar 2025
Pretrained Image-Text Models are Secretly Video Captioners Chunhui Zhang Yiren Jian Z. Ouyang Soroush Vosoughi VLM 142 8 0 20 Feb 2025
video-SALMONN-o1: Reasoning-enhanced Audio-visual Large Language Model Guangzhi Sun Yudong Yang Jimin Zhuang Changli Tang Yongqian Li W. Li Zejun Ma Chao Zhang LRM MLLM VLM 128 5 0 17 Feb 2025
Learning Human Skill Generators at Key-Step Levels Yilu Wu Chenhui Zhu Shuai Wang Hanlin Wang Jing Wang Zhaoxiang Zhang Limin Wang VGen 217 0 0 12 Feb 2025
A Comprehensive Survey of Foundation Models in Medicine Wasif Khan Seowung Leem Kyle B. See Joshua K. Wong Shaoting Zhang R. Fang AI4CE LM&MA VLM 288 27 0 17 Jan 2025
Audio-Language Datasets of Scenes and Events: A Survey Gijs Wijngaard Elia Formisano Michele Esposito M. Dumontier 189 3 0 10 Jan 2025
OneLLM: One Framework to Align All Modalities with Language Jiaming Han Kaixiong Gong Yiyuan Zhang Jiaqi Wang Kaipeng Zhang Dahua Lin Yu Qiao Peng Gao Xiangyu Yue MLLM 254 134 0 10 Jan 2025
A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future Shilin Sun Wenbin An Feng Tian Fang Nan Qidong Liu Jing Liu N. Shah Ping Chen 157 6 0 18 Dec 2024
Do Language Models Understand Time? Xi Ding Lei Wang 335 2 0 18 Dec 2024
Gramian Multimodal Representation Learning and Alignment Giordano Cicchetti Eleonora Grassucci Luigi Sigillo Danilo Comminiello 204 4 0 16 Dec 2024
Multi-Modal interpretable automatic video captioning Antoine Hanna-Asaad Decky Aspandi Titus Zaharia 65 0 0 11 Nov 2024
Sensor2Text: Enabling Natural Language Interactions for Daily Activity Tracking Using Wearable Sensors Wenqiang Chen Jiaxuan Cheng Leyao Wang Wei Zhao Wojciech Matusik 126 2 0 26 Oct 2024
AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Models Kim Sung-Bin Oh Hyun-Bin JungMok Lee Arda Senocak Joon Son Chung Tae-Hyun Oh MLLM VLM 160 8 0 23 Oct 2024
OMCAT: Omni Context Aware Transformer Arushi Goel Karan Sapra Matthieu Le Rafael Valle Andrew Tao Bryan Catanzaro MLLM VLM 79 1 0 15 Oct 2024
LocoMotion: Learning Motion-Focused Video-Language Representations Hazel Doughty Fida Mohammad Thoker Cees G. M. Snoek 119 2 0 15 Oct 2024
MultiVENT 2.0: A Massive Multilingual Benchmark for Event-Centric Video Retrieval Reno Kriz Kate Sanders David Etter Kenton W. Murray Cameron Carpenter ... Alexander Martin Ronald Colaianni Nolan King Eugene Yang Benjamin Van Durme VGen 160 3 0 15 Oct 2024
TRACE: Temporal Grounding Video LLM via Causal Event Modeling Yongxin Guo Jingyu Liu Mingda Li Xiaoying Tang Qingbin Liu Xiaoying Tang 119 19 0 08 Oct 2024
Video DataFlywheel: Resolving the Impossible Data Trinity in Video-Language Understanding Xiao Wang Jianlong Wu Zijia Lin Fuzheng Zhang Di Zhang Liqiang Nie VGen 71 3 0 29 Sep 2024
Language-based Audio Moment Retrieval Hokuto Munakata Taichi Nishimura Shota Nakada Tatsuya Komatsu 131 2 0 24 Sep 2024
From Linguistic Giants to Sensory Maestros: A Survey on Cross-Modal Reasoning with Large Language Models Shengsheng Qian Zuyi Zhou Dizhan Xue Bing Wang Changsheng Xu LRM 148 2 0 19 Sep 2024
TempMe: Video Temporal Token Merging for Efficient Text-Video Retrieval Leqi Shen Tianxiang Hao Tao He Sicheng Zhao Pengzhang Liu Yongjun Bao Guiguang Ding Guiguang Ding 264 15 0 02 Sep 2024
Dissecting Temporal Understanding in Text-to-Audio Retrieval Andreea-Maria Oncescu João F. Henriques A. Sophia Koepke 91 2 0 01 Sep 2024
Estimated Audio-Caption Correspondences Improve Language-Based Audio Retrieval Paul Primus Florian Schmid Gerhard Widmer 83 2 0 21 Aug 2024
ExpertAF: Expert Actionable Feedback from Video Kumar Ashutosh Tushar Nagarajan Georgios Pavlakos Kris Kitani Kristen Grauman VGen 160 3 0 01 Aug 2024
EgoCVR: An Egocentric Benchmark for Fine-Grained Composed Video Retrieval Thomas Hummel Shyamgopal Karthik Mariana-Iuliana Georgescu Zeynep Akata EgoV 154 7 0 23 Jul 2024
MERLIN: Multimodal Embedding Refinement via LLM-based Iterative Navigation for Text-Video Retrieval-Rerank Pipeline D. Han Eunhwan Park Gisang Lee Adam Lee Nojun Kwak 122 4 0 17 Jul 2024
Meerkat: Audio-Visual Large Language Model for Grounding in Space and Time Sanjoy Chowdhury Sayan Nag Subhrajyoti Dasgupta Jun Chen Mohamed Elhoseiny Ruohan Gao Dinesh Manocha VLM MLLM 98 15 0 01 Jul 2024