Self-Supervised Learning by Cross-Modal Audio-Video Clustering

28 November 2019

Papers citing "Self-Supervised Learning by Cross-Modal Audio-Video Clustering"

50 / 111 papers shown

Title
Multimodal Emotion Recognition using Audio-Video Transformer Fusion with Cross Attention Joe Dhanith Shravan Venkatraman Modigari Narendra Vigya Sharma Santhosh Malarvannan 84 0 0 20 Feb 2025
The Sound of Water: Inferring Physical Properties from Pouring Liquids Piyush Bagad Makarand Tapaswi Cees G. M. Snoek Andrew Zisserman 45 0 0 18 Nov 2024
What to align in multimodal contrastive learning? Benoit Dufumier J. Castillo-Navarro D. Tuia Jean-Philippe Thiran 29 3 0 11 Sep 2024
Audio-Visual Generalized Zero-Shot Learning using Pre-Trained Large Multi-Modal Models David Kurzendörfer Otniel-Bogdan Mercea A. Sophia Koepke Zeynep Akata VLM CLIP 33 2 0 09 Apr 2024
Siamese Vision Transformers are Scalable Audio-visual Learners Yan-Bo Lin Gedas Bertasius 37 5 0 28 Mar 2024
Unsupervised Audio-Visual Segmentation with Modality Alignment Swapnil Bhosale Haosen Yang Diptesh Kanojia Jiangkang Deng Xiatian Zhu VOS 43 5 0 21 Mar 2024
Self Supervised Correlation-based Permutations for Multi-View Clustering Ran Eisenberg Jonathan Svirsky Ofir Lindenbaum 32 2 0 26 Feb 2024
POP-3D: Open-Vocabulary 3D Occupancy Prediction from Images Antonín Vobecký Oriane Siméoni David Hurych Spyros Gidaris Andrei Bursuc Patrick Pérez Josef Sivic 40 33 0 17 Jan 2024
Collaboratively Self-supervised Video Representation Learning for Action Recognition Jie Zhang Zhifan Wan Lanqing Hu Stephen Lin Shuzhe Wu Shiguang Shan TTA 67 1 0 15 Jan 2024
Modality Mixer Exploiting Complementary Information for Multi-modal Action Recognition Sumin Lee Sangmin Woo Muhammad Adi Nugroho Changick Kim 30 0 0 21 Nov 2023
Towards Generalized Multi-stage Clustering: Multi-view Self-distillation Jiatai Wang Zhiwei Xu Xin Wang Tao Li 21 1 0 29 Oct 2023
Deep Neural Networks in Video Human Action Recognition: A Review Zihan Wang Yang Yang Zhi Liu Y. Zheng 56 4 0 25 May 2023
Looking Similar, Sounding Different: Leveraging Counterfactual Cross-Modal Pairs for Audiovisual Representation Learning Nikhil Singh Chih-Wei Wu Iroro Orife Mahdi M. Kalayeh 25 2 0 12 Apr 2023
Procedure-Aware Pretraining for Instructional Video Understanding Honglu Zhou Roberto Martín-Martín Mubbasir Kapadia Silvio Savarese Juan Carlos Niebles 31 39 0 31 Mar 2023
Egocentric Auditory Attention Localization in Conversations Fiona Ryan Hao Jiang Abhinav Shukla James M. Rehg V. Ithapu EgoV 29 16 0 28 Mar 2023
Audio-Visual Contrastive Learning with Temporal Self-Supervision Simon Jenni Alexander Black John Collomosse SSL 31 15 0 15 Feb 2023
SemanticAC: Semantics-Assisted Framework for Audio Classification Yicheng Xiao Yue Ma Shuyan Li Hantao Zhou Ran Liao Xiu Li 13 8 0 12 Feb 2023
Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models Zhiqiu Lin Samuel Yu Zhiyi Kuang Deepak Pathak Deva Ramana VLM 20 100 0 16 Jan 2023
What You Say Is What You Show: Visual Narration Detection in Instructional Videos Kumar Ashutosh Rohit Girdhar Lorenzo Torresani Kristen Grauman 24 4 0 05 Jan 2023
Look, Listen, and Attack: Backdoor Attacks Against Video Action Recognition Hasan Hammoud Shuming Liu Mohammad Alkhrashi Fahad Albalawi Guohao Li AAML 32 8 0 03 Jan 2023
Similarity Contrastive Estimation for Image and Video Soft Contrastive Self-Supervised Learning J. Denize Jaonary Rabarisoa Astrid Orcesi Romain Hérault SSL 19 6 0 21 Dec 2022
C2F-TCN: A Framework for Semi and Fully Supervised Temporal Action Segmentation Dipika Singhania R. Rahaman Angela Yao 14 28 0 20 Dec 2022
Jointly Learning Visual and Auditory Speech Representations from Raw Data A. Haliassos Pingchuan Ma Rodrigo Mira Stavros Petridis M. Pantic SSL 45 48 0 12 Dec 2022
Audiovisual Masked Autoencoders Mariana-Iuliana Georgescu Eduardo Fonseca Radu Tudor Ionescu Mario Lucic Cordelia Schmid Anurag Arnab SSL 37 43 0 09 Dec 2022
Complete Cross-triplet Loss in Label Space for Audio-visual Cross-modal Retrieval Donghuo Zeng Yanan Wang Jianming Wu K. Ikeda 27 4 0 07 Nov 2022
VTC: Improving Video-Text Retrieval with User Comments Laura Hanu James Thewlis Yuki M. Asano Christian Rupprecht VGen 29 7 0 19 Oct 2022
Self-supervised Video Representation Learning with Motion-Aware Masked Autoencoders Haosen Yang Deng Huang Bin Wen Jiannan Wu H. Yao Yi-Xin Jiang Xiatian Zhu Zehuan Yuan 37 19 0 09 Oct 2022
Learning State-Aware Visual Representations from Audible Interactions Himangi Mittal Pedro Morgado Unnat Jain Abhinav Gupta 78 23 0 27 Sep 2022
Non-Linguistic Supervision for Contrastive Learning of Sentence Embeddings Yiren Jian Chongyang Gao Soroush Vosoughi SSL 31 15 0 20 Sep 2022
ImageArg: A Multi-modal Tweet Dataset for Image Persuasiveness Mining Zhexiong Liu M. Guo Y. Dai Diane Litman 29 15 0 14 Sep 2022
Modality Mixer for Multi-modal Action Recognition Sumin Lee Sangmin Woo Yeonju Park Muhammad Adi Nugroho Changick Kim 26 10 0 24 Aug 2022
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey Yanbei Chen Massimiliano Mancini Xiatian Zhu Zeynep Akata 45 113 0 24 Aug 2022
Learning in Audio-visual Context: A Review, Analysis, and New Perspective Yake Wei Di Hu Yapeng Tian Xuelong Li 46 55 0 20 Aug 2022
ModSelect: Automatic Modality Selection for Synthetic-to-Real Domain Generalization Zdravko Marinov Alina Roitberg David Schneider Rainer Stiefelhagen 24 4 0 19 Aug 2022
Temporal and cross-modal attention for audio-visual zero-shot learning Otniel-Bogdan Mercea Thomas Hummel A. Sophia Koepke Zeynep Akata 38 25 0 20 Jul 2022
SVGraph: Learning Semantic Graphs from Instructional Videos Madeline Chantry Schiappa Yogesh S Rawat 17 4 0 16 Jul 2022
Learning Music-Dance Representations through Explicit-Implicit Rhythm Synchronization Jiashuo Yu Junfu Pu Ying Cheng Rui Feng Ying Shan 21 5 0 07 Jul 2022
A Comprehensive Survey on Video Saliency Detection with Auditory Information: the Audio-visual Consistency Perceptual is the Key! Chenglizhao Chen Mengke Song Wenfeng Song Li Guo Muwei Jian 35 26 0 20 Jun 2022
Self-Supervised Learning for Videos: A Survey Madeline Chantry Schiappa Yogesh S Rawat M. Shah SSL 36 131 0 18 Jun 2022
SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learning Changan Chen Carl Schissler Sanchit Garg Philip Kobernik Alexander Clegg P. Calamia Dhruv Batra Philip Robinson Kristen Grauman 3DGS 36 80 0 16 Jun 2022
Multimodal Conversational AI: A Survey of Datasets and Approaches Anirudh S. Sundar Larry Heck 38 29 0 13 May 2022
TransRank: Self-supervised Video Representation Learning via Ranking-based Transformation Recognition Haodong Duan Nanxuan Zhao Kai-xiang Chen Dahua Lin ViT AI4TS 33 19 0 04 May 2022
On Negative Sampling for Audio-Visual Contrastive Learning from Movies Mahdi M. Kalayeh Shervin Ardeshir Lingyi Liu Nagendra Kamath Ashok Chandrashekar SSL 32 3 0 29 Apr 2022
MILES: Visual BERT Pre-training with Injected Language Semantics for Video-text Retrieval Yuying Ge Yixiao Ge Xihui Liu Alex Jinpeng Wang Jianping Wu Ying Shan Xiaohu Qie Ping Luo VLM 18 44 0 26 Apr 2022
Contrastive Language-Action Pre-training for Temporal Localization Mengmeng Xu Erhan Gundogdu ⋆⋆ Maksim Guohao Li M. Donoser Loris Bazzani 38 27 0 26 Apr 2022
SETTI: A Self-supervised Adversarial Malware Detection Architecture in an IoT Environment Marjan Golmaryami R. Taheri Zahra Pooranian Mohammad Shojafar Pei Xiao 33 12 0 16 Apr 2022
Frequency Selective Augmentation for Video Representation Learning Jinhyung Kim Taeoh Kim Minho Shim Dongyoon Han Dongyoon Wee Junmo Kim AI4TS 49 3 0 08 Apr 2022
ECLIPSE: Efficient Long-range Video Retrieval using Sight and Sound Yan-Bo Lin Jie Lei Joey Tianyi Zhou Gedas Bertasius 46 39 0 06 Apr 2022
Single-Stream Multi-Level Alignment for Vision-Language Pretraining Zaid Khan B. Vijaykumar Xiang Yu S. Schulter Manmohan Chandraker Y. Fu CLIP VLM 25 16 0 27 Mar 2022
Reshaping Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformers A. Bucker Luis F. C. Figueredo Sami Haddadin Ashish Kapoor Shuang Ma Rogerio Bonatti LM&Ro 38 49 0 25 Mar 2022