v1v2v3 (latest)

Towards Open-Vocabulary Audio-Visual Event Localization

18 November 2024

Papers citing "Towards Open-Vocabulary Audio-Visual Event Localization"

50 / 55 papers shown

Title
PreFM: Online Audio-Visual Event Parsing via Predictive Future Modeling X. Yu Yan Fang Xiaojie Jin Yao Zhao Yunchao Wei 41 0 0 29 May 2025
Reading to Listen at the Cocktail Party: Multi-Modal Speech Separation Akam Rahimi Triantafyllos Afouras Andrew Zisserman 119 29 0 02 Jan 2025
Tri-Ergon: Fine-grained Video-to-Audio Generation with Multi-modal Conditions and LUFS Control Bingliang Li Fengyu Yang Yuxin Mao Qingwen Ye Hongkai Chen Yiran Zhong VGen DiffM 80 4 0 31 Dec 2024
Multimodal Class-aware Semantic Enhancement Network for Audio-Visual Video Parsing Pengcheng Zhao Jinxing Zhou Yang Zhao Dan Guo Yanxiang Chen 124 3 0 15 Dec 2024
Patch-level Sounding Object Tracking for Audio-Visual Question Answering Zhangbin Li Jinxing Zhou Jing Zhang Shengeng Tang Kun Li Dan Guo 110 6 0 14 Dec 2024
CACE-Net: Co-guidance Attention and Contrastive Enhancement for Effective Audio-Visual Event Localization Xiang He Xiangxi Liu Yang Li Dongcheng Zhao Guobin Shen Qingqun Kong Xin Yang Yi Zeng 100 5 0 04 Aug 2024
Open-Vocabulary Audio-Visual Semantic Segmentation Zhenghao Zhang Junchao Liao Dantong Niu Yanyu Qi Menghao Li Ji Shi Bowei Xing Xianghua Ying VOS VLM 91 8 0 31 Jul 2024
RAVSS: Robust Audio-Visual Speech Separation in Multi-Speaker Scenarios with Missing Visual Cues Tianrui Pan Jie Liu Bohan Wang Jie Tang Gangshan Wu 73 2 0 27 Jul 2024
Audio-visual Generalized Zero-shot Learning the Easy Way Shentong Mo Pedro Morgado 54 5 0 18 Jul 2024
Label-anticipated Event Disentanglement for Audio-Visual Video Parsing Jinxing Zhou Dan Guo Yuxin Mao Yiran Zhong Xiaojun Chang Meng Wang 89 13 0 11 Jul 2024
Advancing Weakly-Supervised Audio-Visual Video Parsing via Segment-wise Pseudo Labeling Jinxing Zhou Dan Guo Yiran Zhong Meng Wang VLM 78 19 0 03 Jun 2024
TAVGBench: Benchmarking Text to Audible-Video Generation Yuxin Mao Xuyang Shen Jing Zhang Zhen Qin Jinxing Zhou Mochu Xiang Yiran Zhong Yuchao Dai 65 12 0 22 Apr 2024
T-VSL: Text-Guided Visual Sound Source Localization in Mixtures Tanvir Mahmud Yapeng Tian Diana Marculescu 60 8 0 02 Apr 2024
Unsupervised Audio-Visual Segmentation with Modality Alignment Swapnil Bhosale Haosen Yang Diptesh Kanojia Jiangkang Deng Xiatian Zhu VOS 69 6 0 21 Mar 2024
A Study of Dropout-Induced Modality Bias on Robustness to Missing Video Frames for Audio-Visual Speech Recognition Yusheng Dai Hang Chen Jun Du Ruoyu Wang Shihao Chen Jie Ma Haotian Wang Chin-Hui Lee 81 4 0 07 Mar 2024
Object-aware Adaptive-Positivity Learning for Audio-Visual Question Answering Zhangbin Li Dan Guo Jinxing Zhou Jing Zhang Meng Wang 85 14 0 20 Dec 2023
Weakly-Supervised Audio-Visual Segmentation Shentong Mo Bhiksha Raj VOS 81 13 0 25 Nov 2023
PG-Video-LLaVA: Pixel Grounding Large Video-Language Models Shehan Munasinghe Rusiru Thushara Muhammad Maaz H. Rasheed Salman Khan Mubarak Shah Fahad Khan VLM MLLM 67 36 0 22 Nov 2023
Audio-Visual Instance Segmentation Ruohao Guo Yaru Chen Yanyu Qi Wenzhen Yue Dantong Niu ... Wenzhen Yue Ji Shi Qixun Wang Peiliang Zhang Buwen Liang VLM VOS 96 2 0 28 Oct 2023
Multimodal Variational Auto-encoder based Audio-Visual Segmentation Yuxin Mao Jing Zhang Mochu Xiang Yiran Zhong Yuchao Dai 52 35 0 12 Oct 2023
CM-PIE: Cross-modal perception for interactive-enhanced audio-visual video parsing Yaru Chen Ruohao Guo Xubo Liu Peipei Wu Guangyao Li Zhenbo Li Wenwu Wang 83 8 0 11 Oct 2023
Improving Audio-Visual Segmentation with Bidirectional Generation Dawei Hao Yuxin Mao Bowen He Xiaodong Han Yuchao Dai Yiran Zhong VOS VGen 69 34 0 16 Aug 2023
Progressive Spatio-temporal Perception for Audio-Visual Question Answering Guangyao Li Wenxuan Hou Di Hu 63 31 0 10 Aug 2023
Contrastive Conditional Latent Diffusion for Audio-visual Segmentation Yuxin Mao Jing Zhang Mochu Xiang Yun-Qiu Lv Yiran Zhong Yuchao Dai DiffM 89 29 0 31 Jul 2023
Modality-Independent Teachers Meet Weakly-Supervised Audio-Visual Event Parser Yun-hsuan Lai Yen-Chun Chen Y. Wang 43 12 0 27 May 2023
ImageBind: One Embedding Space To Bind Them All Rohit Girdhar Alaaeldin El-Nouby Zhuang Liu Mannat Singh Kalyan Vasudev Alwala Armand Joulin Ishan Misra VLM 161 941 0 09 May 2023
Fine-grained Audible Video Description Xuyang Shen Dong Li Jinxing Zhou Zhen Qin Bowen He ... Yuchao Dai Lingpeng Kong Meng Wang Yu Qiao Yiran Zhong VGen 85 11 0 27 Mar 2023
Improving Audio-Visual Video Parsing with Pseudo Visual Labels Jinxing Zhou Dan Guo Yiran Zhong Meng Wang VLM 79 14 0 04 Mar 2023
Audio-Visual Segmentation with Semantics Jinxing Zhou Xuyang Shen Jianyuan Wang Jiayi Zhang Weixuan Sun ... Stan Birchfield Dan Guo Lingpeng Kong Meng Wang Yiran Zhong VOS 91 42 0 30 Jan 2023
Contrastive Positive Sample Propagation along the Audio-Visual Event Line Jinxing Zhou Dan Guo Meng Wang 114 54 0 18 Nov 2022
Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation Yusong Wu Kai Chen Tianyu Zhang Yuchen Hui Marianna Nezhurina Taylor Berg-Kirkpatrick Shlomo Dubnov CLIP 131 540 0 12 Nov 2022
AVE-CLIP: AudioCLIP-based Multi-window Temporal Transformer for Audio Visual Event Localization Tanvir Mahmud Diana Marculescu CLIP 61 33 0 11 Oct 2022
Temporal and cross-modal attention for audio-visual zero-shot learning Otniel-Bogdan Mercea Thomas Hummel A. Sophia Koepke Zeynep Akata 83 26 0 20 Jul 2022
Audio-Visual Segmentation Jinxing Zhou Jianyuan Wang Jing Zhang Weixuan Sun Jing Zhang Stan Birchfield Dan Guo Lingpeng Kong Meng Wang Yiran Zhong VOS 87 116 0 11 Jul 2022
Learning to Answer Questions in Dynamic Audio-Visual Scenarios Guangyao Li Yake Wei Yapeng Tian Chenliang Xu Ji-Rong Wen Di Hu 123 153 0 26 Mar 2022
Localizing Visual Sounds the Easy Way Shentong Mo Pedro Morgado 140 81 0 17 Mar 2022
Audio-visual Generalised Zero-shot Learning with Cross-modal Attention and Language Otniel-Bogdan Mercea Lukas Riesch A. Sophia Koepke Zeynep Akata 59 50 0 07 Mar 2022
MM-Pyramid: Multimodal Pyramid Attentional Network for Audio-Visual Event Localization and Video Parsing Jiashuo Yu Ying Cheng Ruiwei Zhao Rui Feng Yuejie Zhang 86 58 0 24 Nov 2021
$Pano-AVQA: Grounded Audio-Visual Question Answering on 360$^\circ$ Videos$ Pano-AVQA: Grounded Audio-Visual Question Answering on 360 $^\circ$ Videos Heeseung Yun Youngjae Yu Wonsuk Yang Kangil Lee Gunhee Kim 96 86 0 11 Oct 2021
Localizing Visual Sounds the Hard Way Honglie Chen Weidi Xie Triantafyllos Afouras Arsha Nagrani Andrea Vedaldi Andrew Zisserman ObjD 88 191 0 06 Apr 2021
Positive Sample Propagation along the Audio-Visual Event Line Jinxing Zhou Liang Zheng Yiran Zhong Shijie Hao Meng Wang 85 102 0 01 Apr 2021
Looking into Your Speech: Learning Cross-modal Affinity for Audio-visual Speech Separation Jiyoung Lee Soo-Whan Chung Sunok Kim Hong-Goo Kang Kwanghoon Sohn 46 51 0 25 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 990 29,871 0 26 Feb 2021
Audio-Visual Event Localization via Recursive Fusion by Joint Co-Attention Bin Duan Hao Tang Wei Wang Ziliang Zong Guowei Yang Yan Yan 80 59 0 14 Aug 2020
Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing Yapeng Tian Dingzeyu Li Chenliang Xu 108 184 0 21 Jul 2020
Multiple Sound Sources Localization from Coarse to Fine Rui Qian Di Hu Heinrich Dinkel Mengyue Wu N. Xu Weiyao Lin 64 157 0 13 Jul 2020
AVGZSLNet: Audio-Visual Generalized Zero-Shot Learning by Reconstructing Label Features from Multi-Modal Embeddings Pratik Mazumder Pravendra Singh Kranti K. Parida Vinay P. Namboodiri 60 35 0 27 May 2020
VGGSound: A Large-scale Audio-Visual Dataset Honglie Chen Weidi Xie Andrea Vedaldi Andrew Zisserman 89 583 0 29 Apr 2020
Coordinated Joint Multimodal Embeddings for Generalized Audio-Visual Zeroshot Classification and Retrieval of Videos Kranti K. Parida Neeraj Matiyali T. Guha Gaurav Sharma VLM 64 42 0 19 Oct 2019
Dual-modality seq2seq network for audio-visual event localization Yan-Bo Lin Yu-Jhe Li Y. Wang 64 128 0 20 Feb 2019