Audio-Visual Scene Analysis with Self-Supervised Multisensory Features

10 April 2018

Papers citing "Audio-Visual Scene Analysis with Self-Supervised Multisensory Features"

50 / 169 papers shown

Title
CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment Edson Araujo Andrew Rouditchenko Yuan Gong Saurabhchand Bhati Samuel Thomas Brian Kingsbury Leonid Karlinsky Rogerio Feris James Glass 39 0 0 02 May 2025
SoundVista: Novel-View Ambient Sound Synthesis via Visual-Acoustic Binding Mingfei Chen I. D. Gebru Ishwarya Ananthabhotla Christian Richardt Dejan Marković Jake Sandakly Steven Krenn Todd Keebler Eli Shlizerman Alexander Richard 24 0 0 08 Apr 2025
Reading to Listen at the Cocktail Party: Multi-Modal Speech Separation Akam Rahimi Triantafyllos Afouras Andrew Zisserman 40 28 0 02 Jan 2025
The Sound of Water: Inferring Physical Properties from Pouring Liquids Piyush Bagad Makarand Tapaswi Cees G. M. Snoek Andrew Zisserman 45 0 0 18 Nov 2024
A Critical Assessment of Visual Sound Source Localization Models Including Negative Audio Xavier Juanola Gloria Haro Magdalena Fuentes 31 2 0 01 Oct 2024
Sequential Contrastive Audio-Visual Learning Ioannis Tsiamas Santiago Pascual Chunghsin Yeh Joan Serra 38 2 0 08 Jul 2024
Images that Sound: Composing Images and Sounds on a Single Canvas Ziyang Chen Daniel Geng Andrew Owens DiffM 48 9 0 20 May 2024
Audio-Visual Generalized Zero-Shot Learning using Pre-Trained Large Multi-Modal Models David Kurzendörfer Otniel-Bogdan Mercea A. Sophia Koepke Zeynep Akata VLM CLIP 28 2 0 09 Apr 2024
Siamese Vision Transformers are Scalable Audio-visual Learners Yan-Bo Lin Gedas Bertasius 37 5 0 28 Mar 2024
Robust Active Speaker Detection in Noisy Environments Siva Sai Nagender Vasireddy Chenxu Zhang Xiaohu Guo Yapeng Tian 32 0 0 27 Mar 2024
Audio-Visual Segmentation via Unlabeled Frame Exploitation Jinxiang Liu Yikun Liu Fei Zhang Chen Ju Ya-Qin Zhang Yanfeng Wang 39 10 0 17 Mar 2024
Multimodal Transformer With a Low-Computational-Cost Guarantee Sungjin Park Edward Choi 49 1 0 23 Feb 2024
BBA: Bi-Modal Behavioral Alignment for Reasoning with Large Vision-Language Models Xueliang Zhao Xinting Huang Tingchen Fu Qintong Li Shansan Gong Lemao Liu Wei Bi Lingpeng Kong LRM 37 1 0 21 Feb 2024
Multimodal Action Quality Assessment Ling-an Zeng Wei-Shi Zheng 43 13 0 31 Jan 2024
Synchformer: Efficient Synchronization from Sparse Cues Vladimir E. Iashin Weidi Xie Esa Rahtu Andrew Zisserman 11 11 0 29 Jan 2024
POP-3D: Open-Vocabulary 3D Occupancy Prediction from Images Antonín Vobecký Oriane Siméoni David Hurych Spyros Gidaris Andrei Bursuc Patrick Pérez Josef Sivic 40 33 0 17 Jan 2024
CAD -- Contextual Multi-modal Alignment for Dynamic AVQA Asmar Nadeem Adrian Hilton R. Dawes Graham A. Thomas A. Mustafa 21 9 0 25 Oct 2023
Sound Source Localization is All about Cross-Modal Alignment Arda Senocak H. Ryu Junsik Kim Tae-Hyun Oh Hanspeter Pfister Joon Son Chung 26 18 0 19 Sep 2023
Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lectures Kun Yuan V. Srivastav Tong Yu Joël L. Lavanchy Pietro Mascagni Pietro Mascagni N. Padoy Nicolas Padoy 27 20 0 27 Jul 2023
Video-to-Music Recommendation using Temporal Alignment of Segments Laure Prétet G. Richard Clement Souchier Geoffroy Peeters AI4TS 29 13 0 12 Jun 2023
Learning Fine-grained View-Invariant Representations from Unpaired Ego-Exo Videos via Temporal Alignment Zihui Xue Kristen Grauman EgoV 33 31 0 08 Jun 2023
How does Contrastive Learning Organize Images? Yunzhe Zhang Yao Lu Qi Xuan SSL 26 0 0 17 May 2023
Looking Similar, Sounding Different: Leveraging Counterfactual Cross-Modal Pairs for Audiovisual Representation Learning Nikhil Singh Chih-Wei Wu Iroro Orife Mahdi M. Kalayeh 23 2 0 12 Apr 2023
Egocentric Auditory Attention Localization in Conversations Fiona Ryan Hao Jiang Abhinav Shukla James M. Rehg V. Ithapu EgoV 29 16 0 28 Mar 2023
Machine Learning for Brain Disorders: Transformers and Visual Transformers Robin Courant Maika Edberg Nicolas Dufour Vicky Kalogeiton MedIm ViT 27 1 0 21 Mar 2023
Transformers in Speech Processing: A Survey S. Latif Aun Zaidi Heriberto Cuayáhuitl Fahad Shamshad Moazzam Shoukat Junaid Qadir 42 47 0 21 Mar 2023
Neural Target Speech Extraction: An Overview Kateřina Žmolíková Marc Delcroix Tsubasa Ochiai K. Kinoshita JanHonza'' vCernocký Dong Yu 21 84 0 31 Jan 2023
Skeleton-based Action Recognition through Contrasting Two-Stream Spatial-Temporal Networks Chen Pang Xuequan Lu Lei Lyu 30 20 0 27 Jan 2023
LoCoNet: Long-Short Context Network for Active Speaker Detection Xizi Wang Feng Cheng Gedas Bertasius David J. Crandall 24 15 0 19 Jan 2023
EXIF as Language: Learning Cross-Modal Associations Between Images and Camera Metadata Chenhao Zheng Ayush Shrivastava Andrew Owens VLM 28 11 0 11 Jan 2023
CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos Hao-Wen Dong Naoya Takahashi Yuki Mitsufuji Julian McAuley Taylor Berg-Kirkpatrick VLM CLIP 25 24 0 14 Dec 2022
Motion and Context-Aware Audio-Visual Conditioned Video Prediction Yating Xu Conghui Hu G. Lee VGen 40 0 0 09 Dec 2022
See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation Hao Li Yizhi Zhang Junzhe Zhu Shaoxiong Wang Michelle A. Lee Huazhe Xu Edward H. Adelson Li Fei-Fei Ruohan Gao Jiajun Wu 27 58 0 07 Dec 2022
Audio-Visual Activity Guided Cross-Modal Identity Association for Active Speaker Detection Rahul Sharma Shrikanth Narayanan 35 8 0 01 Dec 2022
Mix and Localize: Localizing Sound Sources in Mixtures Xixi Hu Ziyang Chen Andrew Owens 23 51 0 28 Nov 2022
Unifying Tracking and Image-Video Object Detection Peirong Liu Rui Wang Pengchuan Zhang Omid Poursaeed Yipin Zhou Xuefei Cao Sreya . Dutta Roy Ashish Shah Ser-Nam Lim 13 0 0 20 Nov 2022
PMR: Prototypical Modal Rebalance for Multimodal Learning Yunfeng Fan Wenchao Xu Haozhao Wang Junxiao Wang Song Guo 23 60 0 14 Nov 2022
Multimodal Transformer Distillation for Audio-Visual Synchronization Xuan-Bo Chen Haibin Wu Chung-Che Wang Hung-yi Lee J. Jang 24 3 0 27 Oct 2022
Leveraging the Video-level Semantic Consistency of Event for Audio-visual Event Localization Yuanyuan Jiang Jianqin Yin Yonghao Dang 35 5 0 11 Oct 2022
Contrastive Audio-Visual Masked Autoencoder Yuan Gong Andrew Rouditchenko Alexander H. Liu David F. Harwath Leonid Karlinsky Hilde Kuehne James R. Glass 32 120 0 02 Oct 2022
TVLT: Textless Vision-Language Transformer Zineng Tang Jaemin Cho Yixin Nie Mohit Bansal VLM 51 28 0 28 Sep 2022
Learning State-Aware Visual Representations from Audible Interactions Himangi Mittal Pedro Morgado Unnat Jain Abhinav Gupta 72 22 0 27 Sep 2022
A Closer Look at Weakly-Supervised Audio-Visual Source Localization Shentong Mo Pedro Morgado 81 64 0 30 Aug 2022
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey Yanbei Chen Massimiliano Mancini Xiatian Zhu Zeynep Akata 36 113 0 24 Aug 2022
Impact Makes a Sound and Sound Makes an Impact: Sound Guides Representations and Explorations Xufeng Zhao C. Weber Muhammad Burhan Hafez S. Wermter 18 8 0 04 Aug 2022
Benchmarking Visual-Inertial Deep Multimodal Fusion for Relative Pose Regression and Odometry-aided Absolute Pose Regression Felix Ott N. Raichur David Rügamer Tobias Feigl Heiko Neumann Bernd Bischl Christopher Mutschler 20 1 0 01 Aug 2022
AudioScopeV2: Audio-Visual Attention Architectures for Calibrated Open-Domain On-Screen Sound Separation Efthymios Tzinis Scott Wisdom Tal Remez J. Hershey 33 29 0 20 Jul 2022
Is an Object-Centric Video Representation Beneficial for Transfer? Chuhan Zhang Ankush Gupta Andrew Zisserman ViT 31 26 0 20 Jul 2022
Temporal and cross-modal attention for audio-visual zero-shot learning Otniel-Bogdan Mercea Thomas Hummel A. Sophia Koepke Zeynep Akata 32 25 0 20 Jul 2022
SVGraph: Learning Semantic Graphs from Instructional Videos Madeline Chantry Schiappa Y. S. Rawat 17 4 0 16 Jul 2022