Learning Fine-Grained Visual Understanding for Video Question Answering via Decoupling Spatial-Temporal Modeling

8 October 2022

Hsin-Ying Lee

Hung-Ting Su

Papers citing "Learning Fine-Grained Visual Understanding for Video Question Answering via Decoupling Spatial-Temporal Modeling"

46 / 46 papers shown

Title
Zero-Shot Video Question Answering via Frozen Bidirectional Language Models Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid 127 236 0 16 Jun 2022
Revealing Single Frame Bias for Video-and-Language Learning Jie Lei Tamara L. Berg Joey Tianyi Zhou 51 113 0 07 Jun 2022
Learning to Answer Visual Questions from Web Videos Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid ViT 64 34 0 10 May 2022
AGQA 2.0: An Updated Benchmark for Compositional Spatio-Temporal Reasoning Madeleine Grunde-McLaughlin Ranjay Krishna Maneesh Agrawala CoGe 46 14 0 12 Apr 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 501 4,324 0 28 Jan 2022
End-to-end Generative Pretraining for Multimodal Video Captioning Paul Hongsuck Seo Arsha Nagrani Anurag Arnab Cordelia Schmid 66 168 0 20 Jan 2022
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling Tsu-Jui Fu Linjie Li Zhe Gan Kevin Qinghong Lin Wenjie Wang Lijuan Wang Zicheng Liu VLM 87 220 0 24 Nov 2021
Perceiver IO: A General Architecture for Structured Inputs & Outputs Andrew Jaegle Sebastian Borgeaud Jean-Baptiste Alayrac Carl Doersch Catalin Ionescu ... Olivier J. Hénaff M. Botvinick Andrew Zisserman Oriol Vinyals João Carreira MLLM VLM GNN 52 579 0 30 Jul 2021
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation Junnan Li Ramprasaath R. Selvaraju Akhilesh Deepak Gotmare Shafiq Joty Caiming Xiong Guosheng Lin FaML 167 1,951 0 16 Jul 2021
Video Swin Transformer Ze Liu Jia Ning Yue Cao Yixuan Wei Zheng Zhang Stephen Lin Han Hu ViT 94 1,481 0 24 Jun 2021
Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answering Jungin Park Jiyoung Lee Kwanghoon Sohn 219 100 0 29 Apr 2021
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval Max Bain Arsha Nagrani Gül Varol Andrew Zisserman VGen 133 1,172 0 01 Apr 2021
AGQA: A Benchmark for Compositional Spatio-Temporal Reasoning Madeleine Grunde-McLaughlin Ranjay Krishna Maneesh Agrawala CoGe 74 117 0 30 Mar 2021
Perceiver: General Perception with Iterative Attention Andrew Jaegle Felix Gimeno Andrew Brock Andrew Zisserman Oriol Vinyals João Carreira VLM ViT MDE 168 1,014 0 04 Mar 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 426 1,125 0 17 Feb 2021
Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling Jie Lei Linjie Li Luowei Zhou Zhe Gan Tamara L. Berg Joey Tianyi Zhou Jingjing Liu CLIP 114 661 0 11 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 422 3,826 0 11 Feb 2021
Look Before you Speak: Visually Contextualized Utterances Paul Hongsuck Seo Arsha Nagrani Cordelia Schmid 43 67 0 10 Dec 2020
ActBERT: Learning Global-Local Video-Text Representations Linchao Zhu Yi Yang ViT 115 420 0 14 Nov 2020
COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs Jena D. Hwang Chandra Bhagavatula Ronan Le Bras Jeff Da Keisuke Sakaguchi Antoine Bosselut Yejin Choi 73 409 0 12 Oct 2020
Location-aware Graph Convolutional Networks for Video Question Answering Deng Huang Peihao Chen Runhao Zeng Qing Du Mingkui Tan Chuang Gan GNN BDL 83 175 0 07 Aug 2020
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training Linjie Li Yen-Chun Chen Yu Cheng Zhe Gan Licheng Yu Jingjing Liu MLLM VLM OffRL AI4TS 101 503 0 01 May 2020
CLEVRER: CoLlision Events for Video REpresentation and Reasoning Kexin Yi Yuta Saito Yunzhu Li Pushmeet Kohli Jiajun Wu Antonio Torralba J. Tenenbaum NAI 104 473 0 03 Oct 2019
VL-BERT: Pre-training of Generic Visual-Linguistic Representations Weijie Su Xizhou Zhu Yue Cao Bin Li Lewei Lu Furu Wei Jifeng Dai VLM MLLM SSL 145 1,663 0 22 Aug 2019
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Hao Hao Tan Joey Tianyi Zhou VLM MLLM 227 2,474 0 20 Aug 2019
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks Jiasen Lu Dhruv Batra Devi Parikh Stefan Lee SSL VLM 217 3,674 0 06 Aug 2019
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips Antoine Miech Dimitri Zhukov Jean-Baptiste Alayrac Makarand Tapaswi Ivan Laptev Josef Sivic VGen 105 1,199 0 07 Jun 2019
ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering Zhou Yu D. Xu Jun-chen Yu Ting Yu Zhou Zhao Yueting Zhuang Dacheng Tao 101 461 0 06 Jun 2019
AssembleNet: Searching for Multi-Stream Neural Connectivity in Video Architectures Michael S. Ryoo A. Piergiovanni Mingxing Tan A. Angelova 50 102 0 30 May 2019
Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering Chenyou Fan Xiaofan Zhang Shu Zhang Wensheng Wang Chi Zhang Heng-Chiao Huang 44 278 0 08 Apr 2019
VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research Xin Eric Wang Jiawei Wu Junkun Chen Lei Li Yuan-fang Wang William Yang Wang 93 549 0 06 Apr 2019
SlowFast Networks for Video Recognition Christoph Feichtenhofer Haoqi Fan Jitendra Malik Kaiming He 162 3,262 0 10 Dec 2018
A Joint Sequence Fusion Model for Video Question Answering and Retrieval Youngjae Yu Jongseok Kim Gunhee Kim 73 345 0 07 Aug 2018
Motion-Appearance Co-Memory Networks for Video Question Answering J. Gao Runzhou Ge Kan Chen Ram Nevatia 111 241 0 29 Mar 2018
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering Peter Anderson Xiaodong He Chris Buehler Damien Teney Mark Johnson Stephen Gould Lei Zhang AIMat 113 4,214 0 25 Jul 2017
Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics Alex Kendall Y. Gal R. Cipolla 3DH 266 3,122 0 19 May 2017
Dense-Captioning Events in Videos Ranjay Krishna Kenji Hata F. Ren Li Fei-Fei Juan Carlos Niebles 134 1,242 0 02 May 2017
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering Yash Goyal Tejas Khot D. Summers-Stay Dhruv Batra Devi Parikh CoGe 322 3,224 0 02 Dec 2016
Leveraging Video Descriptions to Learn Video Question Answering Kuo-Hao Zeng Tseng-Hung Chen Ching-Yao Chuang Yuan-Hong Liao Juan Carlos Niebles Min Sun 81 179 0 12 Nov 2016
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition Limin Wang Yuanjun Xiong Zhe Wang Yu Qiao Dahua Lin Xiaoou Tang Luc Van Gool ViT 98 3,831 0 02 Aug 2016
Sort Story: Sorting Jumbled Images and Captions into Stories Harsh Agrawal Arjun Chandrasekaran Dhruv Batra Devi Parikh Joey Tianyi Zhou 47 60 0 23 Jun 2016
Convolutional Two-Stream Network Fusion for Video Action Recognition Christoph Feichtenhofer A. Pinz Andrew Zisserman 156 2,610 0 22 Apr 2016
Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding Gunnar Sigurdsson Gül Varol Xinyu Wang Ali Farhadi Ivan Laptev Abhinav Gupta VGen 92 1,245 0 06 Apr 2016
Microsoft COCO Captions: Data Collection and Evaluation Server Xinlei Chen Hao Fang Nayeon Lee Ramakrishna Vedantam Saurabh Gupta Piotr Dollar C. L. Zitnick 203 2,475 0 01 Apr 2015
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention Ke Xu Jimmy Ba Ryan Kiros Kyunghyun Cho Aaron Courville Ruslan Salakhutdinov R. Zemel Yoshua Bengio DiffM 328 10,050 0 10 Feb 2015
Two-Stream Convolutional Networks for Action Recognition in Videos Karen Simonyan Andrew Zisserman 237 7,526 0 09 Jun 2014