Multi-Modal interpretable automatic video captioning

11 November 2024

Papers citing "Multi-Modal interpretable automatic video captioning"

28 / 28 papers shown

Title
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset Sihan Chen Handong Li Qunbo Wang Zijia Zhao Ming-Ting Sun Xinxin Zhu Qingbin Liu 124 104 0 29 May 2023
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset Sihan Chen Xingjian He Longteng Guo Xinxin Zhu Weining Wang Jinhui Tang Jinhui Tang VLM 60 110 0 17 Apr 2023
SEM-POS: Grammatically and Semantically Correct Video Captioning Asmar Nadeem A. Hilton R. Dawes Graham A. Thomas A. Mustafa 41 8 0 26 Mar 2023
GIT: A Generative Image-to-text Transformer for Vision and Language Jianfeng Wang Zhengyuan Yang Xiaowei Hu Linjie Li Kevin Qinghong Lin Zhe Gan Zicheng Liu Ce Liu Lijuan Wang VLM 123 546 0 27 May 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 490 4,324 0 28 Jan 2022
SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning Kevin Qinghong Lin Linjie Li Chung-Ching Lin Faisal Ahmed Zhe Gan Zicheng Liu Yumao Lu Lijuan Wang ViT 73 242 0 25 Nov 2021
Hierarchical Modular Network for Video Captioning Hanhua Ye Guorong Li Yuankai Qi Shuhui Wang Qingming Huang Ming-Hsuan Yang 61 67 0 24 Nov 2021
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text Hassan Akbari Liangzhe Yuan Rui Qian Wei-Hong Chuang Shih-Fu Chang Huayu Chen Boqing Gong ViT 300 587 0 22 Apr 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 812 29,167 0 26 Feb 2021
ActBERT: Learning Global-Local Video-Text Representations Linchao Zhu Yi Yang ViT 115 420 0 14 Nov 2020
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training Linjie Li Yen-Chun Chen Yu Cheng Zhe Gan Licheng Yu Jingjing Liu MLLM VLM OffRL AI4TS 101 503 0 01 May 2020
Multi-modal Dense Video Captioning Vladimir E. Iashin Esa Rahtu 45 169 0 17 Mar 2020
Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou Hamid Palangi Lei Zhang Houdong Hu Jason J. Corso Jianfeng Gao MLLM VLM 337 937 0 24 Sep 2019
Controllable Video Captioning with POS Sequence Guidance Based on Gated Fusion Network Bairui Wang Lin Ma Wei Zhang Wenhao Jiang Jingwen Wang Wei Liu 85 163 0 27 Aug 2019
VL-BERT: Pre-training of Generic Visual-Linguistic Representations Weijie Su Xizhou Zhu Yue Cao Bin Li Lewei Lu Furu Wei Jifeng Dai VLM MLLM SSL 142 1,661 0 22 Aug 2019
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Hao Hao Tan Joey Tianyi Zhou VLM MLLM 227 2,474 0 20 Aug 2019
VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research Xin Eric Wang Jiawei Wu Junkun Chen Lei Li Yuan-fang Wang William Yang Wang 91 549 0 06 Apr 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.5K 94,511 0 11 Oct 2018
End-to-End Dense Video Captioning with Masked Transformer Luowei Zhou Yingbo Zhou Jason J. Corso R. Socher Caiming Xiong 88 528 0 03 Apr 2018
Axiomatic Attribution for Deep Networks Mukund Sundararajan Ankur Taly Qiqi Yan OOD FAtt 175 5,968 0 04 Mar 2017
Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning Jiasen Lu Caiming Xiong Devi Parikh R. Socher 115 1,450 0 06 Dec 2016
SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning Long Chen Hanwang Zhang Jun Xiao Liqiang Nie Jian Shao Wei Liu Tat-Seng Chua 63 1,655 0 17 Nov 2016
Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization Ramprasaath R. Selvaraju Michael Cogswell Abhishek Das Ramakrishna Vedantam Devi Parikh Dhruv Batra FAtt 256 19,929 0 07 Oct 2016
Multimodal Convolutional Neural Networks for Matching Image and Sentence Lin Ma Zhengdong Lu Lifeng Shang Hang Li 97 337 0 23 Apr 2015
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention Ke Xu Jimmy Ba Ryan Kiros Kyunghyun Cho Aaron Courville Ruslan Salakhutdinov R. Zemel Yoshua Bengio DiffM 316 10,050 0 10 Feb 2015
Translating Videos to Natural Language Using Deep Recurrent Neural Networks Subhashini Venugopalan Huijuan Xu Jeff Donahue Marcus Rohrbach Raymond J. Mooney Kate Saenko 114 952 0 15 Dec 2014
CIDEr: Consensus-based Image Description Evaluation Ramakrishna Vedantam C. L. Zitnick Devi Parikh 256 4,471 0 20 Nov 2014
Video In Sentences Out Andrei Barbu Alexander Bridge Zachary Burchill D. Coroian Sven J. Dickinson ... Jarrell W. Waggoner Song Wang Jinlian Wei Yifan Yin Zhiqi Zhang 62 156 0 09 Aug 2014