Labelling unlabelled videos from scratch with multi-modal self-supervision

24 June 2020

Yuki M. Asano

Mandela Patrick

Christian Rupprecht

Andrea Vedaldi

SSL

ArXiv PDF HTML

Papers citing "Labelling unlabelled videos from scratch with multi-modal self-supervision"

39 / 39 papers shown

Title
The Sound of Water: Inferring Physical Properties from Pouring Liquids Piyush Bagad Makarand Tapaswi Cees G. M. Snoek Andrew Zisserman 45 0 0 18 Nov 2024
What to align in multimodal contrastive learning? Benoit Dufumier J. Castillo-Navarro D. Tuia Jean-Philippe Thiran 29 3 0 11 Sep 2024
Images that Sound: Composing Images and Sounds on a Single Canvas Ziyang Chen Daniel Geng Andrew Owens DiffM 50 9 0 20 May 2024
Audio-Visual Generalized Zero-Shot Learning using Pre-Trained Large Multi-Modal Models David Kurzendörfer Otniel-Bogdan Mercea A. Sophia Koepke Zeynep Akata VLM CLIP 33 2 0 09 Apr 2024
Siamese Vision Transformers are Scalable Audio-visual Learners Yan-Bo Lin Gedas Bertasius 37 5 0 28 Mar 2024
Collaboratively Self-supervised Video Representation Learning for Action Recognition Jie Zhang Zhifan Wan Lanqing Hu Stephen Lin Shuzhe Wu Shiguang Shan TTA 67 1 0 15 Jan 2024
Test-Time Training on Video Streams Renhao Wang Yu Sun Yossi Gandelsman Xinlei Chen Alexei A. Efros Alexei A. Efros Xiaolong Wang TTA ViT 3DGS 37 16 0 11 Jul 2023
Align and Attend: Multimodal Summarization with Dual Contrastive Losses Bo He Jun Wang Jielin Qiu Trung Bui Abhinav Shrivastava Zhaowen Wang 22 65 0 13 Mar 2023
Day2Dark: Pseudo-Supervised Activity Recognition beyond Silent Daylight Yunhua Zhang Hazel Doughty Cees G. M. Snoek VLM 40 0 0 05 Dec 2022
Mix and Localize: Localizing Sound Sources in Mixtures Xixi Hu Ziyang Chen Andrew Owens 23 51 0 28 Nov 2022
VTC: Improving Video-Text Retrieval with User Comments Laura Hanu James Thewlis Yuki M. Asano Christian Rupprecht VGen 26 7 0 19 Oct 2022
Learning to Discover and Detect Objects V. Fomenko Ismail Elezi Deva Ramanan Laura Leal-Taixé Aljosa Osep ObjD 31 10 0 19 Oct 2022
Self-supervised Video Representation Learning with Motion-Aware Masked Autoencoders Haosen Yang Deng Huang Bin Wen Jiannan Wu H. Yao Yi-Xin Jiang Xiatian Zhu Zehuan Yuan 37 19 0 09 Oct 2022
TVLT: Textless Vision-Language Transformer Zineng Tang Jaemin Cho Yixin Nie Joey Tianyi Zhou VLM 51 28 0 28 Sep 2022
Learning State-Aware Visual Representations from Audible Interactions Himangi Mittal Pedro Morgado Unnat Jain Abhinav Gupta 75 22 0 27 Sep 2022
Learning in Audio-visual Context: A Review, Analysis, and New Perspective Yake Wei Di Hu Yapeng Tian Xuelong Li 46 55 0 20 Aug 2022
Temporal and cross-modal attention for audio-visual zero-shot learning Otniel-Bogdan Mercea Thomas Hummel A. Sophia Koepke Zeynep Akata 38 25 0 20 Jul 2022
Towards Realistic Semi-Supervised Learning Mamshad Nayeem Rizve Navid Kardan M. Shah 47 44 0 05 Jul 2022
Self-Supervised Learning for Videos: A Survey Madeline Chantry Schiappa Y. S. Rawat M. Shah SSL 36 131 0 18 Jun 2022
Discovering Object Masks with Transformers for Unsupervised Semantic Segmentation Wouter Van Gansbeke Simon Vandenhende Luc Van Gool 39 55 0 13 Jun 2022
SelfD: Self-Learning Large-Scale Driving Policies From the Web Jimuyang Zhang Ruizhao Zhu Eshed Ohn-Bar 11 20 0 21 Apr 2022
ECLIPSE: Efficient Long-range Video Retrieval using Sight and Sound Yan-Bo Lin Jie Lei Joey Tianyi Zhou Gedas Bertasius 41 39 0 06 Apr 2022
Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection A. Haliassos Rodrigo Mira Stavros Petridis M. Pantic CVBM 32 126 0 18 Jan 2022
Robust Contrastive Learning against Noisy Views Ching-Yao Chuang R. Devon Hjelm Xin Wang Vibhav Vineet Neel Joshi Antonio Torralba Stefanie Jegelka Ya-heng Song NoLa 13 68 0 12 Jan 2022
Class-aware Sounding Objects Localization via Audiovisual Correspondence Di Hu Yake Wei Rui Qian Weiyao Lin Ruihua Song Ji-Rong Wen 24 41 0 22 Dec 2021
Exploring Temporal Granularity in Self-Supervised Video Representation Learning Rui Qian Yeqing Li Liangzhe Yuan Boqing Gong Ting Liu Matthew A. Brown Serge J. Belongie Ming-Hsuan Yang Hartwig Adam Huayu Chen AI4TS 59 6 0 08 Dec 2021
Revitalizing CNN Attentions via Transformers in Self-Supervised Visual Representation Learning Chongjian Ge Youwei Liang Yibing Song Jianbo Jiao Jue Wang Ping Luo ViT 21 36 0 11 Oct 2021
Enhancing Self-supervised Video Representation Learning via Multi-level Feature Optimization Rui Qian Yuxi Li Huabin Liu John See Shuangrui Ding Xian Liu Dian Li Weiyao Lin 35 42 0 04 Aug 2021
Multi-level Feature Learning for Contrastive Multi-view Clustering Jie Xu Huayi Tang Yazhou Ren Liang Peng Xiao-lan Zhu Lifang He 26 161 0 21 Jun 2021
Self-supervised Video Representation Learning with Cross-Stream Prototypical Contrasting Martine Toering Ioannis Gatopoulos M. Stol Vincent Tao Hu SSL 40 11 0 18 Jun 2021
Joint Representation Learning and Novel Category Discovery on Single- and Multi-modal Data Xu Jia Kai Han Yukun Zhu Bradley Green 147 57 0 26 Apr 2021
Object Priors for Classifying and Localizing Unseen Actions Pascal Mettes William Thong Cees G. M. Snoek 24 20 0 10 Apr 2021
Multiview Pseudo-Labeling for Semi-supervised Learning from Video Bo Xiong Haoqi Fan Kristen Grauman Christoph Feichtenhofer SSL 22 49 0 01 Apr 2021
Unsupervised Sound Localization via Iterative Contrastive Learning Yan-Bo Lin Hung-Yu Tseng Hsin-Ying Lee Yen-Yu Lin Ming-Hsuan Yang SSL 19 34 0 01 Apr 2021
Space-Time Crop & Attend: Improving Cross-modal Video Representation Learning Mandela Patrick Yuki M. Asano Bernie Huang Ishan Misra Florian Metze Joao Henriques Andrea Vedaldi AI4TS 29 33 0 18 Mar 2021
Hard Negative Mixing for Contrastive Learning Yannis Kalantidis Mert Bulent Sariyildiz Noé Pion Philippe Weinzaepfel Diane Larlus SSL 32 628 0 02 Oct 2020
Delving into Inter-Image Invariance for Unsupervised Visual Representations Jiahao Xie Xiaohang Zhan Ziwei Liu Yew-Soon Ong Chen Change Loy SSL VLM 21 58 0 26 Aug 2020
Spatiotemporal Contrastive Video Representation Learning Rui Qian Tianjian Meng Boqing Gong Ming-Hsuan Yang Haoran Wang Serge J. Belongie Huayu Chen SSL AI4TS 36 491 0 09 Aug 2020
Audiovisual SlowFast Networks for Video Recognition Fanyi Xiao Yong Jae Lee Kristen Grauman Jitendra Malik Christoph Feichtenhofer 197 206 0 23 Jan 2020