Audio-Visual Instance Discrimination with Cross-Modal Agreement

27 April 2020

Papers citing "Audio-Visual Instance Discrimination with Cross-Modal Agreement"

50 / 71 papers shown

Title
CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment Edson Araujo Andrew Rouditchenko Yuan Gong Saurabhchand Bhati Samuel Thomas Brian Kingsbury Leonid Karlinsky Rogerio Feris James Glass 44 0 0 02 May 2025
The Sound of Water: Inferring Physical Properties from Pouring Liquids Piyush Bagad Makarand Tapaswi Cees G. M. Snoek Andrew Zisserman 45 0 0 18 Nov 2024
Aligning Sight and Sound: Advanced Sound Source Localization Through Audio-Visual Alignment Arda Senocak H. Ryu Junsik Kim Tae-Hyun Oh Hanspeter Pfister Joon Son Chung 38 3 0 18 Jul 2024
Sequential Contrastive Audio-Visual Learning Ioannis Tsiamas Santiago Pascual Chunghsin Yeh Joan Serrà 44 2 0 08 Jul 2024
Images that Sound: Composing Images and Sounds on a Single Canvas Ziyang Chen Daniel Geng Andrew Owens DiffM 50 9 0 20 May 2024
Siamese Vision Transformers are Scalable Audio-visual Learners Yan-Bo Lin Gedas Bertasius 37 5 0 28 Mar 2024
Cross-Modal Coordination Across a Diverse Set of Input Modalities Jorge Sánchez Rodrigo Laguna VLM 44 0 0 29 Jan 2024
CAD -- Contextual Multi-modal Alignment for Dynamic AVQA Asmar Nadeem Adrian Hilton R. Dawes Graham A. Thomas A. Mustafa 30 9 0 25 Oct 2023
Sound Source Localization is All about Cross-Modal Alignment Arda Senocak H. Ryu Junsik Kim Tae-Hyun Oh Hanspeter Pfister Joon Son Chung 36 18 0 19 Sep 2023
Quantifying Representation Reliability in Self-Supervised Learning Models Young-Jin Park Hao Wang Shervin Ardeshir Navid Azizan SSL UQCV 34 3 0 31 May 2023
Looking Similar, Sounding Different: Leveraging Counterfactual Cross-Modal Pairs for Audiovisual Representation Learning Nikhil Singh Chih-Wei Wu Iroro Orife Mahdi M. Kalayeh 25 2 0 12 Apr 2023
Procedure-Aware Pretraining for Instructional Video Understanding Honglu Zhou Roberto Martín-Martín Mubbasir Kapadia Silvio Savarese Juan Carlos Niebles 31 39 0 31 Mar 2023
Audio-Visual Contrastive Learning with Temporal Self-Supervision Simon Jenni Alexander Black John Collomosse SSL 31 15 0 15 Feb 2023
SemanticAC: Semantics-Assisted Framework for Audio Classification Yicheng Xiao Yue Ma Shuyan Li Hantao Zhou Ran Liao Xiu Li 13 8 0 12 Feb 2023
Bridging the Emotional Semantic Gap via Multimodal Relevance Estimation Chuan Zhang Daoxin Zhang Ruixiu Zhang Jiawei Li Jianke Zhu 33 1 0 03 Feb 2023
STEPs: Self-Supervised Key Step Extraction and Localization from Unlabeled Procedural Videos Anshul B. Shah Benjamin Lundell H. Sawhney Ramalingam Chellappa SSL 16 8 0 02 Jan 2023
Jointly Learning Visual and Auditory Speech Representations from Raw Data A. Haliassos Pingchuan Ma Rodrigo Mira Stavros Petridis M. Pantic SSL 45 48 0 12 Dec 2022
Motion and Context-Aware Audio-Visual Conditioned Video Prediction Yating Xu Conghui Hu G. Lee VGen 43 0 0 09 Dec 2022
FakeOut: Leveraging Out-of-domain Self-supervision for Multi-modal Video Deepfake Detection Gil Knafo Ohad Fried 23 5 0 01 Dec 2022
Unifying Tracking and Image-Video Object Detection Peirong Liu Rui Wang Pengchuan Zhang Omid Poursaeed Yipin Zhou Xuefei Cao Sreya . Dutta Roy Ashish Shah Ser-Nam Lim 21 0 0 20 Nov 2022
VTC: Improving Video-Text Retrieval with User Comments Laura Hanu James Thewlis Yuki M. Asano Christian Rupprecht VGen 32 7 0 19 Oct 2022
Towards Effective Image Manipulation Detection with Proposal Contrastive Learning Yuyuan Zeng Bowen Zhao Shanzhao Qiu Tao Dai Shutao Xia 34 25 0 16 Oct 2022
Contrastive Audio-Visual Masked Autoencoder Yuan Gong Andrew Rouditchenko Alexander H. Liu David Harwath Leonid Karlinsky Hilde Kuehne James R. Glass 35 120 0 02 Oct 2022
Learning State-Aware Visual Representations from Audible Interactions Himangi Mittal Pedro Morgado Unnat Jain Abhinav Gupta 78 23 0 27 Sep 2022
Federated Transfer Learning with Multimodal Data Yulian Sun FedML 21 4 0 05 Sep 2022
A Closer Look at Weakly-Supervised Audio-Visual Source Localization Shentong Mo Pedro Morgado 83 64 0 30 Aug 2022
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey Yanbei Chen Massimiliano Mancini Xiatian Zhu Zeynep Akata 45 113 0 24 Aug 2022
Learning in Audio-visual Context: A Review, Analysis, and New Perspective Yake Wei Di Hu Yapeng Tian Xuelong Li 46 55 0 20 Aug 2022
RenyiCL: Contrastive Representation Learning with Skew Renyi Divergence Kyungmin Lee Jinwoo Shin SSL DRL 31 10 0 12 Aug 2022
LAVA: Language Audio Vision Alignment for Contrastive Video Pre-Training Sumanth Gurram An Fang David M. Chan John F. Canny VLM AI4TS 38 1 0 16 Jul 2022
Modality-Aware Contrastive Instance Learning with Self-Distillation for Weakly-Supervised Audio-Visual Violence Detection Jiashuo Yu Jin-Yuan Liu Ying Cheng Rui Feng Yuejie Zhang 21 35 0 12 Jul 2022
Learning Music-Dance Representations through Explicit-Implicit Rhythm Synchronization Jiashuo Yu Junfu Pu Ying Cheng Rui Feng Ying Shan 21 5 0 07 Jul 2022
A Comprehensive Survey on Video Saliency Detection with Auditory Information: the Audio-visual Consistency Perceptual is the Key! Chenglizhao Chen Mengke Song Wenfeng Song Li Guo Muwei Jian 35 26 0 20 Jun 2022
Self-Supervised Learning for Videos: A Survey Madeline Chantry Schiappa Yogesh S Rawat M. Shah SSL 36 131 0 18 Jun 2022
OmniMAE: Single Model Masked Pretraining on Images and Videos Rohit Girdhar Alaaeldin El-Nouby Mannat Singh Kalyan Vasudev Alwala Armand Joulin Ishan Misra ViT 37 97 0 16 Jun 2022
Multimodal Conversational AI: A Survey of Datasets and Approaches Anirudh S. Sundar Larry Heck 38 29 0 13 May 2022
On Negative Sampling for Audio-Visual Contrastive Learning from Movies Mahdi M. Kalayeh Shervin Ardeshir Lingyi Liu Nagendra Kamath Ashok Chandrashekar SSL 32 3 0 29 Apr 2022
Self-supervised Contrastive Learning for Audio-Visual Action Recognition Yang Liu Y. Tan Haoyu Lan SSL 47 6 0 28 Apr 2022
Robust Cross-Modal Representation Learning with Progressive Self-Distillation A. Andonian Shixing Chen Raffay Hamid VLM 29 54 0 10 Apr 2022
ECLIPSE: Efficient Long-range Video Retrieval using Sight and Sound Yan-Bo Lin Jie Lei Joey Tianyi Zhou Gedas Bertasius 46 39 0 06 Apr 2022
Audio Self-supervised Learning: A Survey Shuo Liu Adria Mallol-Ragolta Emilia Parada-Cabeleiro Kun Qian Xingshuo Jing Alexander Kathan Bin Hu Bjoern W. Schuller SSL 35 106 0 02 Mar 2022
CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understanding Mohamed Afham Isuru Dissanayake Dinithi Dissanayake Amaya Dharmasiri Kanchana Thilakarathna Ranga Rodrigo 3DPC 16 251 0 01 Mar 2022
Learning Contextually Fused Audio-visual Representations for Audio-visual Speech Recognition Zitian Zhang Jie Zhang Jian-Shu Zhang Ming Wu Xin Fang Lirong Dai SSL 41 10 0 15 Feb 2022
Self-supervised Video Representation Learning with Cascade Positive Retrieval Cheng-En Wu Farley Lai Yujie Hu Asim Kadav SSL AI4TS 30 3 0 20 Jan 2022
Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection A. Haliassos Rodrigo Mira Stavros Petridis M. Pantic CVBM 40 126 0 18 Jan 2022
Robust Contrastive Learning against Noisy Views Ching-Yao Chuang R. Devon Hjelm Xin Wang Vibhav Vineet Neel Joshi Antonio Torralba Stefanie Jegelka Ya-heng Song NoLa 13 68 0 12 Jan 2022
Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction Bowen Shi Wei-Ning Hsu Kushal Lakhotia Abdel-rahman Mohamed SSL 46 305 0 05 Jan 2022
Sound and Visual Representation Learning with Multiple Pretraining Tasks A. Vasudevan Dengxin Dai Luc Van Gool SSL 33 6 0 04 Jan 2022
Beyond Mono to Binaural: Generating Binaural Audio from Mono Audio with Depth and Cross Modal Attention Kranti K. Parida Siddharth Srivastava Gaurav Sharma MDE 36 20 0 15 Nov 2021
Latent Structure Mining with Contrastive Modality Fusion for Multimedia Recommendation Jinghao Zhang Yanqiao Zhu Qiang Liu Mengqi Zhang Shu Wu Liang Wang 22 34 0 01 Nov 2021