HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training

1 May 2020

Papers citing "HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training"

50 / 175 papers shown

Title
YTCommentQA: Video Question Answerability in Instructional Videos Saelyne Yang Sunghyun Park Yunseok Jang Moontae Lee 31 3 0 30 Jan 2024
Audio-Visual LLM for Video Understanding Fangxun Shu Lei Zhang Hao Jiang Cihang Xie VLM MLLM 27 38 0 11 Dec 2023
MM-VID: Advancing Video Understanding with GPT-4V(ision) Kevin Qinghong Lin Faisal Ahmed Linjie Li Chung-Ching Lin E. Azarnasab ... Lin Liang Zicheng Liu Yumao Lu Ce Liu Lijuan Wang MLLM 33 63 0 30 Oct 2023
CAD -- Contextual Multi-modal Alignment for Dynamic AVQA Asmar Nadeem Adrian Hilton R. Dawes Graham A. Thomas A. Mustafa 33 9 0 25 Oct 2023
SCANet: Scene Complexity Aware Network for Weakly-Supervised Video Moment Retrieval Sunjae Yoon Gwanhyeong Koo Dahyun Kim Changdong Yoo 29 12 0 08 Oct 2023
AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model Avamarie Brueggeman Andrea Madotto Zhaojiang Lin Tushar Nagarajan Matt Smith ... Peyman Heidari Yue Liu Kavya Srinet Babak Damavandi Anuj Kumar MLLM 39 93 0 27 Sep 2023
Zero-Shot and Few-Shot Video Question Answering with Multi-Modal Prompts Bipin Rajendran Bashir M. Al-Hashimi MLLM VLM 32 2 0 27 Sep 2023
Dual-Modal Attention-Enhanced Text-Video Retrieval with Triplet Partial Margin Contrastive Learning Chen Jiang Hong Liu Xuzheng Yu Qing Wang Yuan Cheng ... Zhongyi Liu Qingpei Guo Wei Chu Ming Yang Yuan Qi 33 10 0 20 Sep 2023
EgoPCA: A New Framework for Egocentric Hand-Object Interaction Understanding Yue Xu Yong-Lu Li Zhemin Huang Michael Xu Liu Cewu Lu Yu-Wing Tai Chi-Keung Tang EgoV 33 9 0 05 Sep 2023
Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions Weizhen He Yihe Deng Shixiang Tang Qihao Chen Qingsong Xie ... Feng Zhu Rui Zhao Wanli Ouyang Donglian Qi Yunfeng Yan 80 19 0 13 Jun 2023
VLAB: Enhancing Video Language Pre-training by Feature Adapting and Blending Xingjian He Sihan Chen Fan Ma Zhicheng Huang Xiaojie Jin Zikang Liu Dongmei Fu Yi Yang Qingbin Liu Jiashi Feng VLM CLIP 23 17 0 22 May 2023
Self-Chained Image-Language Model for Video Localization and Question Answering Shoubin Yu Jaemin Cho Prateek Yadav Joey Tianyi Zhou 58 130 0 11 May 2023
Making the Most of What You Have: Adapting Pre-trained Visual Language Models in the Low-data Regime Chuhan Zhang Antoine Miech Jiajun Shen Jean-Baptiste Alayrac Pauline Luc VLM VPVLM 47 2 0 03 May 2023
SViTT: Temporal Learning of Sparse Video-Text Transformers Yi Li Kyle Min Subarna Tripathi Nuno Vasconcelos 31 12 0 18 Apr 2023
Video ChatCaptioner: Towards Enriched Spatiotemporal Descriptions Jun Chen Deyao Zhu Kilichbek Haydarov Xiang Li Mohamed Elhoseiny 36 37 0 09 Apr 2023
Personality-aware Human-centric Multimodal Reasoning: A New Task, Dataset and Baselines Yaochen Zhu Xiangqing Shen Rui Xia 26 5 0 05 Apr 2023
Procedure-Aware Pretraining for Instructional Video Understanding Honglu Zhou Roberto Martín-Martín Mubbasir Kapadia Silvio Savarese Juan Carlos Niebles 43 39 0 31 Mar 2023
Hierarchical Video-Moment Retrieval and Step-Captioning Abhaysinh Zala Jaemin Cho Satwik Kottur Xilun Chen Barlas Ouguz Yasher Mehdad Joey Tianyi Zhou 3DV 20 51 0 29 Mar 2023
VideoXum: Cross-modal Visual and Textural Summarization of Videos Jingyang Lin Hang Hua Ming Chen Yikang Li Jenhao Hsiao C. Ho Jiebo Luo 36 30 0 21 Mar 2023
Accommodating Audio Modality in CLIP for Multimodal Processing Ludan Ruan Anwen Hu Yuqing Song Liang Zhang S. Zheng Qin Jin VLM 29 10 0 12 Mar 2023
Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning Antoine Yang Arsha Nagrani Paul Hongsuck Seo Antoine Miech Jordi Pont-Tuset Ivan Laptev Josef Sivic Cordelia Schmid AI4TS VLM 39 221 0 27 Feb 2023
STOA-VLP: Spatial-Temporal Modeling of Object and Action for Video-Language Pre-training Weihong Zhong Mao Zheng Duyu Tang Xuan Luo Heng Gong Xiaocheng Feng Bing Qin 32 8 0 20 Feb 2023
Interactive Video Corpus Moment Retrieval using Reinforcement Learning Zhixin Ma Chong-Wah Ngo 35 3 0 19 Feb 2023
ViewCo: Discovering Text-Supervised Segmentation Masks via Multi-View Semantic Consistency Pengzhen Ren Changlin Li Hang Xu Yi Zhu Guangrun Wang Jian-zhuo Liu Xiaojun Chang Xiaodan Liang 47 43 0 31 Jan 2023
Temporal Perceiving Video-Language Pre-training Fan Ma Xiaojie Jin Heng Wang Jingjia Huang Linchao Zhu Jiashi Feng Yi Yang VLM 32 15 0 18 Jan 2023
In Defense of Structural Symbolic Representation for Video Event-Relation Prediction Andrew Lu Xudong Lin Yulei Niu Shih-Fu Chang 32 2 0 06 Jan 2023
HierVL: Learning Hierarchical Video-Language Embeddings Kumar Ashutosh Rohit Girdhar Lorenzo Torresani Kristen Grauman VLM AI4TS 33 53 0 05 Jan 2023
What You Say Is What You Show: Visual Narration Detection in Instructional Videos Kumar Ashutosh Rohit Girdhar Lorenzo Torresani Kristen Grauman 29 4 0 05 Jan 2023
NaQ: Leveraging Narrations as Queries to Supervise Episodic Memory Santhosh Kumar Ramakrishnan Ziad Al-Halah Kristen Grauman 119 39 0 02 Jan 2023
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training Qinghao Ye Guohai Xu Ming Yan Haiyang Xu Qi Qian Ji Zhang Fei Huang VLM AI4TS 188 70 0 30 Dec 2022
Masked Contrastive Pre-Training for Efficient Video-Text Retrieval Fangxun Shu Biaolong Chen Yue Liao Shuwen Xiao Wenyu Sun Xiaobo Li Yousong Zhu Jinqiao Wang Si Liu CLIP 35 11 0 02 Dec 2022
Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention Zineng Tang Jaemin Cho Jie Lei Joey Tianyi Zhou VLM 24 9 0 21 Nov 2022
SMAUG: Sparse Masked Autoencoder for Efficient Video-Language Pre-training Yuanze Lin Chen Wei Huiyu Wang Alan Yuille Cihang Xie 3DGS 36 15 0 21 Nov 2022
Grafting Pre-trained Models for Multimodal Headline Generation Lingfeng Qiao Chen Wu Ye Liu Haoyuan Peng Di Yin Bo Ren 46 5 0 14 Nov 2022
Watching the News: Towards VideoQA Models that can Read Soumya Jahagirdar Minesh Mathew Dimosthenis Karatzas C. V. Jawahar 32 18 0 10 Nov 2022
Modal-specific Pseudo Query Generation for Video Corpus Moment Retrieval Minjoon Jung Seongho Choi Joo-Kyung Kim Jin-Hwa Kim Byoung-Tak Zhang 40 7 0 23 Oct 2022
VTC: Improving Video-Text Retrieval with User Comments Laura Hanu James Thewlis Yuki M. Asano Christian Rupprecht VGen 41 7 0 19 Oct 2022
Temporal Action Segmentation: An Analysis of Modern Techniques Guodong Ding Fadime Sener Angela Yao 52 76 0 19 Oct 2022
Dense but Efficient VideoQA for Intricate Compositional Reasoning Jihyeon Janel Lee Wooyoung Kang Eun-Sol Kim CoGe 24 3 0 19 Oct 2022
Selective Query-guided Debiasing for Video Corpus Moment Retrieval Sunjae Yoon Jiajing Hong Eunseop Yoon Dahyun Kim Junyeong Kim Hee Suk Yoon Changdong Yoo 46 21 0 17 Oct 2022
Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning Yuchong Sun Hongwei Xue Ruihua Song Bei Liu Huan Yang Jianlong Fu AI4TS VLM 20 68 0 12 Oct 2022
Contrastive Video-Language Learning with Fine-grained Frame Sampling Zixu Wang Yujie Zhong Yishu Miao Lin Ma Lucia Specia 59 11 0 10 Oct 2022
Fighting FIRe with FIRE: Assessing the Validity of Text-to-Video Retrieval Benchmarks Pedro Rodriguez Mahmoud Azab Becka Silvert Renato Sanchez Linzy Labson Hardik Shah Seungwhan Moon 50 1 0 10 Oct 2022
Hierarchical3D Adapters for Long Video-to-text Summarization Pinelopi Papalampidi Mirella Lapata VGen 36 12 0 10 Oct 2022
Music-to-Text Synaesthesia: Generating Descriptive Text from Music Recordings Zhihuan Kuang Shi Zong Jianbing Zhang Jiajun Chen Hongfu Liu 30 4 0 02 Oct 2022
Learning Transferable Spatiotemporal Representations from Natural Script Knowledge Ziyun Zeng Yuying Ge Xihui Liu Bin Chen Ping Luo Shutao Xia Yixiao Ge AI4TS 37 8 0 30 Sep 2022
TVLT: Textless Vision-Language Transformer Zineng Tang Jaemin Cho Yixin Nie Joey Tianyi Zhou VLM 54 28 0 28 Sep 2022
Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval Xiang Fang Daizong Liu Pan Zhou Yuchong Hu 82 39 0 23 Sep 2022
An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modeling Tsu-Jui Fu Linjie Li Zhe Gan Kevin Qinghong Lin William Yang Wang Lijuan Wang Zicheng Liu VLM 35 64 0 04 Sep 2022
Hierarchical Local-Global Transformer for Temporal Sentence Grounding Xiang Fang Daizong Liu Pan Zhou Zichuan Xu Rui Li 38 28 0 31 Aug 2022