End-to-End Multimodal Representation Learning for Video Dialog

26 October 2022

Papers citing "End-to-End Multimodal Representation Learning for Video Dialog"

38 / 38 papers shown

Title
Mamba Fusion: Learning Actions Through Questioning Zhikang Dong Apoorva Beedu Jason Sheinkopf Irfan Essa Mamba 109 2 0 17 Sep 2024
Audio Visual Scene-Aware Dialog Generation with Transformer-based Video Representations Yoshihiro Yamazaki Shota Orihashi Ryo Masumura Mihiro Uchida Akihiko Takashima 36 8 0 21 Feb 2022
Audio-Visual Scene-Aware Dialog and Reasoning using Audio-Visual Transformers with Joint Student-Teacher Learning Ankit Parag Shah Shijie Geng Peng Gao A. Cherian Takaaki Hori Tim K. Marks Jonathan Le Roux Chiori Hori 57 22 0 13 Oct 2021
$C^3$ : Compositional Counterfactual Contrastive Learning for Video-grounded Dialogues Hung Le Nancy F. Chen Guosheng Lin 46 2 0 16 Jun 2021
Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answering Jungin Park Jiyoung Lee Kwanghoon Sohn 236 100 0 29 Apr 2021
DSTC8-AVSD: Multimodal Semantic Transformer Network with Retrieval Style Word Generator Hwanhee Lee Seunghyun Yoon Franck Dernoncourt Doo Soon Kim Trung Bui Kyomin Jung 51 15 0 01 Apr 2020
UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation Huaishao Luo Lei Ji Botian Shi Haoyang Huang Nan Duan Tianrui Li Jason Li Xilin Chen Ming Zhou VLM 85 439 0 15 Feb 2020
Bridging Text and Video: A Universal Multimodal Transformer for Video-Audio Scene-Aware Dialog Zekang Li Zongjia Li Jinchao Zhang Yang Feng Cheng Niu Jie Zhou 119 37 0 01 Feb 2020
Multi-step Joint-Modality Attention Network for Scene-Aware Dialogue System Yun-Wei Chu Kuan-Yen Lin Chao-Chun Hsu Lun-Wei Ku 100 22 0 17 Jan 2020
Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline Vishvak Murahari Dhruv Batra Devi Parikh Abhishek Das VLM 78 116 0 05 Dec 2019
PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable Siqi Bao H. He Fan Wang Hua Wu Haifeng Wang 54 269 0 17 Oct 2019
VL-BERT: Pre-training of Generic Visual-Linguistic Representations Weijie Su Xizhou Zhu Yue Cao Bin Li Lewei Lu Furu Wei Jifeng Dai VLM MLLM SSL 153 1,663 0 22 Aug 2019
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Hao Hao Tan Joey Tianyi Zhou VLM MLLM 237 2,479 0 20 Aug 2019
Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training Gen Li Nan Duan Yuejian Fang Ming Gong Daxin Jiang Ming Zhou SSL VLM MLLM 202 902 0 16 Aug 2019
VisualBERT: A Simple and Performant Baseline for Vision and Language Liunian Harold Li Mark Yatskar Da Yin Cho-Jui Hsieh Kai-Wei Chang VLM 138 1,951 0 09 Aug 2019
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks Jiasen Lu Dhruv Batra Devi Parikh Stefan Lee SSL VLM 226 3,678 0 06 Aug 2019
Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems Hung Le Doyen Sahoo Nancy F. Chen Guosheng Lin 63 111 0 02 Jul 2019
Learning Video Representations using Contrastive Bidirectional Transformer Chen Sun Fabien Baradel Kevin Patrick Murphy Cordelia Schmid SSL ViT 109 134 0 13 Jun 2019
TVQA+: Spatio-Temporal Grounding for Video Question Answering Jie Lei Licheng Yu Tamara L. Berg Joey Tianyi Zhou 71 229 0 25 Apr 2019
A Simple Baseline for Audio-Visual Scene-Aware Dialog Idan Schwartz Alex Schwing Tamir Hazan 61 70 0 11 Apr 2019
VideoBERT: A Joint Model for Video and Language Representation Learning Chen Sun Austin Myers Carl Vondrick Kevin Patrick Murphy Cordelia Schmid VLM SSL 77 1,246 0 03 Apr 2019
Audio-Visual Scene-Aware Dialog Huda AlAmri Vincent Cartillier Abhishek Das Jue Wang A. Cherian ... Tim K. Marks Chiori Hori Peter Anderson Stefan Lee Devi Parikh VGen 52 192 0 25 Jan 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.7K 94,770 0 11 Oct 2018
TVQA: Localized, Compositional Video Question Answering Muhammad Abdul Wahab Licheng Yu Mounir Nasr Allah Tamara L. Berg 87 638 0 05 Sep 2018
Textual Explanations for Self-Driving Vehicles Jinkyu Kim Anna Rohrbach Trevor Darrell John F. Canny Zeynep Akata 52 341 0 30 Jul 2018
End-to-End Audio Visual Scene-Aware Dialog using Multimodal Attention-Based Video Features Chiori Hori Huda AlAmri Jue Wang Gordon Wichern Takaaki Hori ... Raphael Gontijo-Lopes Abhishek Das Irfan Essa Dhruv Batra Devi Parikh VGen 64 125 0 21 Jun 2018
End-to-End Dense Video Captioning with Masked Transformer Luowei Zhou Yingbo Zhou Jason J. Corso R. Socher Caiming Xiong 92 528 0 03 Apr 2018
Reconstruction Network for Video Captioning Bairui Wang Lin Ma Wei Zhang Wen Liu 116 318 0 30 Mar 2018
Deep contextualized word representations Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee Luke Zettlemoyer NAI 206 11,549 0 15 Feb 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 690 131,526 0 12 Jun 2017
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset João Carreira Andrew Zisserman 232 8,015 0 22 May 2017
Visual Dialog Abhishek Das Satwik Kottur Khushi Gupta Avi Singh Deshraj Yadav José M. F. Moura Devi Parikh Dhruv Batra 142 997 0 26 Nov 2016
CNN Architectures for Large-Scale Audio Classification Shawn Hershey Sourish Chaudhuri D. Ellis J. Gemmeke A. Jansen ... Rif A. Saurous Bryan Seybold M. Slaney Ron J. Weiss K. Wilson 120 2,498 0 29 Sep 2016
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Zhiwen Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 894 6,788 0 26 Sep 2016
Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding Gunnar Sigurdsson Gül Varol Xinyu Wang Ali Farhadi Ivan Laptev Abhinav Gupta VGen 98 1,245 0 06 Apr 2016
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.8K 150,039 0 22 Dec 2014
CIDEr: Consensus-based Image Description Evaluation Ramakrishna Vedantam C. L. Zitnick Devi Parikh 286 4,484 0 20 Nov 2014
Going Deeper with Convolutions Christian Szegedy Wei Liu Yangqing Jia P. Sermanet Scott E. Reed Dragomir Anguelov D. Erhan Vincent Vanhoucke Andrew Rabinovich 460 43,649 0 17 Sep 2014