Title
Ask2Loc: Learning to Locate Instructional Visual Answers by Asking Questions Chang Zong Bin Li Shoujun Zhou Jian Wan Lei Zhang 450 0 0 22 Apr 2025
FocusedAD: Character-centric Movie Audio Description Xiaojun Ye C. Wang Yiren Song Sheng Zhou Liangcheng Li Jiajun Bu VGen 95 0 0 16 Apr 2025
Hoi2Anomaly: An Explainable Anomaly Detection Approach Guided by Human-Object Interaction Yuhan Wang Cheng Liu Daou Zhang Weichao Wu 92 0 0 13 Mar 2025
List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMs An Yan Zhengyuan Yang Junda Wu Wanrong Zhu Jianwei Yang ... Kevin Qinghong Lin Jianfeng Wang Julian McAuley Jianfeng Gao Lijuan Wang LRM 85 12 0 25 Apr 2024
Contextual AD Narration with Interleaved Multimodal Sequence Hanlin Wang Zhan Tong Kecheng Zheng Yujun Shen Limin Wang VGen 92 4 0 19 Mar 2024
Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf Yuzhuang Xu Shuo Wang Peng Li Ziyue Wang Xiaolong Wang Weidong Liu Yang Liu LLMAG 45 208 0 09 Sep 2023
Building Cooperative Embodied Agents Modularly with Large Language Models Hongxin Zhang Weihua Du Jiaming Shan Qinhong Zhou Yilun Du J. Tenenbaum Tianmin Shu Chuang Gan LLMAG LM&Ro 121 175 0 05 Jul 2023
Valley: Video Assistant with Large Language model Enhanced abilitY Ruipu Luo Ziwang Zhao Min Yang Junwei Dong Da Li Pengcheng Lu Tao Wang Linmei Hu Ming-Hui Qiu MLLM 108 208 0 12 Jun 2023
AutoAD: Movie Description in Context Tengda Han Max Bain Arsha Nagrani Gül Varol Weidi Xie Andrew Zisserman VGen 57 35 0 29 Mar 2023
MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action Zhengyuan Yang Linjie Li Jianfeng Wang Kevin Qinghong Lin E. Azarnasab Faisal Ahmed Zicheng Liu Ce Liu Michael Zeng Lijuan Wang ReLM KELM LRM 103 394 0 20 Mar 2023
Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos Bowen Baker Ilge Akkaya Peter Zhokhov Joost Huizinga Jie Tang Adrien Ecoffet Brandon Houghton Raul Sampedro Jeff Clune OffRL 128 303 0 23 Jun 2022
Multimodal Learning with Transformers: A Survey Peng Xu Xiatian Zhu David Clifton ViT 186 567 0 13 Jun 2022
Egocentric Video-Language Pretraining Kevin Qinghong Lin Alex Jinpeng Wang Mattia Soldan Michael Wray Rui Yan ... Hongfa Wang Dima Damen Guohao Li Wei Liu Mike Zheng Shou VLM EgoV 75 205 0 03 Jun 2022
i-Code: An Integrative and Composable Multimodal Learning Framework Ziyi Yang Yuwei Fang Chenguang Zhu Reid Pryzant DongDong Chen ... Bin Xiao Yuanxun Lu Takuya Yoshioka Michael Zeng Xuedong Huang 69 48 0 03 May 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 418 3,585 0 29 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 515 6,279 0 05 Apr 2022
All in One: Exploring Unified Video-Language Pre-training Alex Jinpeng Wang Yixiao Ge Rui Yan Yuying Ge Xudong Lin Guanyu Cai Jianping Wu Ying Shan Xiaohu Qie Mike Zheng Shou 90 202 0 14 Mar 2022
SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning Kevin Qinghong Lin Linjie Li Chung-Ching Lin Faisal Ahmed Zhe Gan Zicheng Liu Yumao Lu Lijuan Wang ViT 83 244 0 25 Nov 2021
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling Tsu-Jui Fu Linjie Li Zhe Gan Kevin Qinghong Lin Wenjie Wang Lijuan Wang Zicheng Liu VLM 103 221 0 24 Nov 2021
Florence: A New Foundation Model for Computer Vision Lu Yuan Dongdong Chen Yi-Ling Chen Noel Codella Xiyang Dai ... Zhen Xiao Jianwei Yang Michael Zeng Luowei Zhou Pengchuan Zhang VLM 135 908 0 22 Nov 2021
Ego4D: Around the World in 3,000 Hours of Egocentric Video Kristen Grauman Andrew Westbury Eugene Byrne Zachary Chavis Antonino Furnari ... Mike Zheng Shou Antonio Torralba Lorenzo Torresani Mingfei Yan Jitendra Malik EgoV 399 1,107 0 13 Oct 2021
Video Swin Transformer Ze Liu Jia Ning Yue Cao Yixuan Wei Zheng Zhang Stephen Lin Han Hu ViT 109 1,487 0 24 Jun 2021
Towards Long-Form Video Understanding Chaoxia Wu Philipp Krahenbuhl VLM ViT 117 170 0 21 Jun 2021
VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation Linjie Li Jie Lei Zhe Gan Licheng Yu Yen-Chun Chen ... Tamara L. Berg Joey Tianyi Zhou Jingjing Liu Lijuan Wang Zicheng Liu VLM 94 101 0 08 Jun 2021
Multiscale Vision Transformers Haoqi Fan Bo Xiong K. Mangalam Yanghao Li Zhicheng Yan Jitendra Malik Christoph Feichtenhofer ViT 135 1,265 0 22 Apr 2021
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval Max Bain Arsha Nagrani Gül Varol Andrew Zisserman VGen 153 1,186 0 01 Apr 2021
ViViT: A Video Vision Transformer Anurag Arnab Mostafa Dehghani G. Heigold Chen Sun Mario Lucic Cordelia Schmid ViT 225 2,163 0 29 Mar 2021
Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling Jie Lei Linjie Li Luowei Zhou Zhe Gan Tamara L. Berg Joey Tianyi Zhou Jingjing Liu CLIP 127 664 0 11 Feb 2021
Is Space-Time Attention All You Need for Video Understanding? Gedas Bertasius Heng Wang Lorenzo Torresani ViT 389 2,061 0 09 Feb 2021
Continuous Speech Separation with Conformer Sanyuan Chen Yu-Huan Wu Zhuo Chen Jian Wu Jinyu Li Takuya Yoshioka Chengyi Wang Shujie Liu M. Zhou 56 130 0 13 Aug 2020
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training Linjie Li Yen-Chun Chen Yu Cheng Zhe Gan Licheng Yu Jingjing Liu MLLM VLM OffRL AI4TS 118 504 0 01 May 2020
Learning Interactions and Relationships between Movie Characters Anna Kukleva Makarand Tapaswi Ivan Laptev 74 51 0 29 Mar 2020
TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval Jie Lei Licheng Yu Tamara L. Berg Joey Tianyi Zhou 197 286 0 24 Jan 2020
A Short Note on the Kinetics-700 Human Action Dataset João Carreira Eric Noland Chloe Hillier Andrew Zisserman 76 455 0 15 Jul 2019
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips Antoine Miech Dimitri Zhukov Jean-Baptiste Alayrac Makarand Tapaswi Ivan Laptev Josef Sivic VGen 118 1,207 0 07 Jun 2019
VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research Xin Eric Wang Jiawei Wu Junkun Chen Lei Li Yuan-fang Wang William Yang Wang 101 555 0 06 Apr 2019
Spatiotemporal Pyramid Network for Video Action Recognition Yunbo Wang Mingsheng Long Jianmin Wang Philip S. Yu 91 229 0 04 Mar 2019
SlowFast Networks for Video Recognition Christoph Feichtenhofer Haoqi Fan Jitendra Malik Kaiming He 166 3,282 0 10 Dec 2018
VoxCeleb: a large-scale speaker identification dataset Arsha Nagrani Joon Son Chung Andrew Zisserman 127 2,279 0 26 Jun 2017
Multimodal Machine Learning: A Survey and Taxonomy T. Baltrušaitis Chaitanya Ahuja Louis-Philippe Morency 106 2,936 0 26 May 2017
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset João Carreira Andrew Zisserman 235 8,037 0 22 May 2017
A Dataset for Movie Description Anna Rohrbach Marcus Rohrbach Niket Tandon Bernt Schiele VGen 119 502 0 12 Jan 2015
UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild K. Soomro Amir Zamir M. Shah CLIP VGen 160 6,162 0 03 Dec 2012