ECLIPSE: Efficient Long-range Video Retrieval using Sight and Sound

6 April 2022

Yan-Bo Lin

Jie Lei

Joey Tianyi Zhou

Gedas Bertasius

ArXiv PDF HTML

Papers citing "ECLIPSE: Efficient Long-range Video Retrieval using Sight and Sound"

39 / 39 papers shown

Title
Learning Audio-guided Video Representation with Gated Attention for Video-Text Retrieval Boseung Jeong Jicheol Park Sungyeon Kim Suha Kwak 36 0 0 03 Apr 2025
Leveraging Modality Tags for Enhanced Cross-Modal Video Retrieval A. Fragomeni Dima Damen Michael Wray 33 0 0 02 Apr 2025
NeighborRetr: Balancing Hub Centrality in Cross-Modal Retrieval Zengrong Lin Zheng Wang Tianwen Qian Pan Mu Sixian Chan Cong Bai 55 0 0 13 Mar 2025
Masked Differential Privacy David Schneider Sina Sajadmanesh Vikash Sehwag Saquib Sarfraz Rainer Stiefelhagen Lingjuan Lyu Vivek Sharma 33 0 0 22 Oct 2024
Learning Video Context as Interleaved Multimodal Sequences S. Shao Pengchuan Zhang Y. Li Xide Xia A. Meso Ziteng Gao Jinheng Xie N. Holliman Mike Zheng Shou 46 5 0 31 Jul 2024
Meta-optimized Angular Margin Contrastive Framework for Video-Language Representation Learning Thong Nguyen Yi Bin Xiaobao Wu Xinshuai Dong Zhiyuan Hu Khoi M. Le Cong-Duy Nguyen See-Kiong Ng Luu Anh Tuan 39 6 0 04 Jul 2024
Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectives Thong Nguyen Yi Bin Junbin Xiao Leigang Qu Yicong Li Jay Zhangjie Wu Cong-Duy Nguyen See-Kiong Ng Luu Anh Tuan VLM 56 10 1 09 Jun 2024
Encoding and Controlling Global Semantics for Long-form Video Question Answering Thong Nguyen Zhiyuan Hu Xiaobao Wu Cong-Duy Nguyen See-Kiong Ng A. Luu 43 3 0 30 May 2024
Siamese Vision Transformers are Scalable Audio-visual Learners Yan-Bo Lin Gedas Bertasius 37 5 0 28 Mar 2024
Text Is MASS: Modeling as Stochastic Embedding for Text-Video Retrieval Jiamian Wang Guohao Sun Pichao Wang Dongfang Liu S. Dianat Majid Rabbani Raghuveer M. Rao Zhiqiang Tao VGen 59 20 0 26 Mar 2024
Retrieval-Augmented Egocentric Video Captioning Jilan Xu Yifei Huang Junlin Hou Guo Chen Yue Zhang Rui Feng Weidi Xie EgoV 51 29 0 01 Jan 2024
MoVQA: A Benchmark of Versatile Question-Answering for Long-Form Movie Understanding Hongjie Zhang Yi Liu Lu Dong Yifei Huang Z. Ling Yali Wang Limin Wang Yu Qiao 23 25 0 08 Dec 2023
CAST: Cross-Attention in Space and Time for Video Action Recognition Dongho Lee Jongseo Lee Jinwoo Choi EgoV 35 12 0 30 Nov 2023
ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Models .Ilker Kesen Andrea Pedrotti Mustafa Dogan Michele Cafagna Emre Can Acikgoz ... Iacer Calixto Anette Frank Albert Gatt Aykut Erdem Erkut Erdem 41 15 0 13 Nov 2023
Query-aware Long Video Localization and Relation Discrimination for Deep Video Understanding Yuanxing Xu Yuting Wei Bin Wu 33 0 0 19 Oct 2023
Unified Coarse-to-Fine Alignment for Video-Text Retrieval Ziyang Wang Yi-Lin Sung Feng Cheng Gedas Bertasius Joey Tianyi Zhou 101 44 0 18 Sep 2023
Multimodal Fish Feeding Intensity Assessment in Aquaculture Meng Cui Xubo Liu Haohe Liu Zhuangzhuang Du Tao Chen Guoping Lian Daoliang Li Wenwu Wang 28 5 0 10 Sep 2023
Audio-Enhanced Text-to-Video Retrieval using Text-Conditioned Feature Alignment Sarah Ibrahimi Xiaohang Sun Pichao Wang Amanmeet Garg Ashutosh Sanan Mohamed Omar 46 14 0 24 Jul 2023
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset Sihan Chen Handong Li Qunbo Wang Zijia Zhao Ming-Ting Sun Xinxin Zhu Jiaheng Liu 32 97 0 29 May 2023
SViTT: Temporal Learning of Sparse Video-Text Transformers Yi Li Kyle Min Subarna Tripathi Nuno Vasconcelos 28 12 0 18 Apr 2023
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset Sihan Chen Xingjian He Longteng Guo Xinxin Zhu Weining Wang Jinhui Tang Jinhui Tang VLM 31 102 0 17 Apr 2023
Multiscale Audio Spectrogram Transformer for Efficient Audio Classification Wenjie Zhu M. Omar 37 22 0 19 Mar 2023
MIST: Multi-modal Iterative Spatial-Temporal Transformer for Long-form Video Question Answering Difei Gao Luowei Zhou Lei Ji Linchao Zhu Yezhou Yang Mike Zheng Shou 44 60 0 19 Dec 2022
Vision Transformers are Parameter-Efficient Audio-Visual Learners Yan-Bo Lin Yi-Lin Sung Jie Lei Joey Tianyi Zhou Gedas Bertasius 28 73 0 15 Dec 2022
VindLU: A Recipe for Effective Video-and-Language Pretraining Feng Cheng Xizi Wang Jie Lei David J. Crandall Joey Tianyi Zhou Gedas Bertasius VLM 32 78 0 09 Dec 2022
TVLT: Textless Vision-Language Transformer Zineng Tang Jaemin Cho Yixin Nie Joey Tianyi Zhou VLM 51 28 0 28 Sep 2022
CONE: An Efficient COarse-to-fiNE Alignment Framework for Long Video Temporal Grounding Zhijian Hou Wanjun Zhong Lei Ji Difei Gao Kun Yan W. Chan Chong-Wah Ngo Zheng Shou Nan Duan AI4TS 36 24 0 22 Sep 2022
MuMUR : Multilingual Multimodal Universal Retrieval Avinash Madasu Estelle Aflalo Gabriela Ben-Melech Stan Shachar Rosenman Shao-Yen Tseng Gedas Bertasius Vasudev Lal 44 3 0 24 Aug 2022
Revealing Single Frame Bias for Video-and-Language Learning Jie Lei Tamara L. Berg Joey Tianyi Zhou 24 110 0 07 Jun 2022
A CLIP-Hitchhiker's Guide to Long Video Retrieval Max Bain Arsha Nagrani Gül Varol Andrew Zisserman CLIP 129 62 0 17 May 2022
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Dmytro Okhonko Armen Aghajanyan Florian Metze Luke Zettlemoyer Florian Metze Luke Zettlemoyer Christoph Feichtenhofer CLIP VLM 259 558 0 28 Sep 2021
Exploiting Audio-Visual Consistency with Partial Supervision for Spatial Audio Generation Yan-Bo Lin Y. Wang 48 21 0 03 May 2021
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text Hassan Akbari Liangzhe Yuan Rui Qian Wei-Hong Chuang Shih-Fu Chang Huayu Chen Boqing Gong ViT 251 577 0 22 Apr 2021
T2VLAD: Global-Local Sequence Alignment for Text-Video Retrieval Xiaohan Wang Linchao Zhu Yi Yang 170 170 0 20 Apr 2021
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval Huaishao Luo Lei Ji Ming Zhong Yang Chen Wen Lei Nan Duan Tianrui Li CLIP VLM 317 780 0 18 Apr 2021
A Straightforward Framework For Video Retrieval Using CLIP Jesús Andrés Portillo-Quintero J. C. Ortíz-Bayliss Hugo Terashima-Marín CLIP 324 117 0 24 Feb 2021
Is Space-Time Attention All You Need for Video Understanding? Gedas Bertasius Heng Wang Lorenzo Torresani ViT 280 1,982 0 09 Feb 2021
PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation Yuan Gong Yu-An Chung James R. Glass VLM 104 144 0 02 Feb 2021
Multi-modal Transformer for Video Retrieval Valentin Gabeur Chen Sun Alahari Karteek Cordelia Schmid ViT 424 596 0 21 Jul 2020