VLP: A Survey on Vision-Language Pre-training

18 February 2022

Minglun Han

Papers citing "VLP: A Survey on Vision-Language Pre-training"

50 / 140 papers shown

Title
Reasoning Segmentation for Images and Videos: A Survey Yiqing Shen Chenjia Li Fei Xiong Jeong-O Jeong Tianpeng Wang Michael Latman Mathias Unberath VOS 148 0 0 24 May 2025
AS3D: 2D-Assisted Cross-Modal Understanding with Semantic-Spatial Scene Graphs for 3D Visual Grounding Feng Xiao Hongbin Xu Guocan Zhao Wenxiong Kang 142 0 0 07 May 2025
CLIP-PING: Boosting Lightweight Vision-Language Models with Proximus Intrinsic Neighbors Guidance Chu Myaet Thwal Ye Lin Tun Minh N. H. Nguyen Eui-nam Huh Choong Seon Hong VLM 107 0 0 05 Dec 2024
Libra: Leveraging Temporal Images for Biomedical Radiology Analysis Xi Zhang Zaiqiao Meng Jake Lever Edmond S. L. Ho MedIm 150 1 0 28 Nov 2024
MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Models Shengkang Wang Hongzhan Lin Ziyang Luo Zhen Ye Guang Chen Jing Ma 86 3 0 17 Jun 2024
One Perturbation is Enough: On Generating Universal Adversarial Perturbations against Vision-Language Pre-training Models Hao Fang Jiawei Kong Wenbo Yu Bin Chen Jiawei Li Hao Wu Ke Xu Ke Xu AAML VLM 81 13 0 08 Jun 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 210 49 0 23 May 2024
RankCLIP: Ranking-Consistent Language-Image Pretraining Yiming Zhang Zhuokai Zhao Zhaorun Chen Zhili Feng Zenghui Ding Yining Sun SSL VLM 80 7 0 15 Apr 2024
GET: Unlocking the Multi-modal Potential of CLIP for Generalized Category Discovery Enguang Wang Zhimao Peng Zhengyuan Xie Fei Yang Xialei Liu Ming-Ming Cheng 88 3 0 15 Mar 2024
Enhancing Vision-Language Pre-training with Rich Supervisions Yuan Gao Kunyu Shi Pengkai Zhu Edouard Belval Oren Nuriel Srikar Appalaraju Shabnam Ghadar Vijay Mahadevan Zhuowen Tu Stefano Soatto VLM CLIP 94 12 0 05 Mar 2024
Advances in Embodied Navigation Using Large Language Models: A Survey Jinzhou Lin Han Gao Xuxiang Feng Rongtao Xu Changwei Wang Man Zhang Li Guo Shibiao Xu LM&Ro LLMAG 110 9 0 01 Nov 2023
Diffusion-LM Improves Controllable Text Generation Xiang Lisa Li John Thickstun Ishaan Gulrajani Percy Liang Tatsunori B. Hashimoto AI4CE 216 802 0 27 May 2022
HiVLP: Hierarchical Vision-Language Pre-Training for Fast Image-Text Retrieval Feilong Chen Xiuyi Chen Jiaxin Shi Duzhen Zhang Jianlong Chang Qi Tian VLM CLIP 41 6 0 24 May 2022
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding Chitwan Saharia William Chan Saurabh Saxena Lala Li Jay Whang ... Raphael Gontijo-Lopes Tim Salimans Jonathan Ho David J Fleet Mohammad Norouzi VLM 308 5,904 0 23 May 2022
Improving Cross-Modal Understanding in Visual Dialog via Contrastive Learning Feilong Chen Xiuyi Chen Shuang Xu Bo Xu VLM 51 18 0 15 Apr 2022
Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions Jing Gu Eliana Stefani Qi Wu Jesse Thomason Xinze Wang LM&Ro 93 105 0 22 Mar 2022
CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailment Haoyu Song Li Dong Weinan Zhang Ting Liu Furu Wei VLM CLIP 67 138 0 14 Mar 2022
Recent Advances and New Frontiers in Spiking Neural Networks Duzhen Zhang Shuncheng Jia Qingyu Wang AAML AI4CE 30 23 0 12 Mar 2022
Vision-Language Pre-Training with Triple Contrastive Learning Jinyu Yang Jiali Duan Son N. Tran Yi Xu Sampath Chanda Liqun Chen Belinda Zeng Trishul Chilimbi Junzhou Huang VLM 94 292 0 21 Feb 2022
MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound Rowan Zellers Jiasen Lu Ximing Lu Youngjae Yu Yanpeng Zhao Mohammadreza Salehi Aditya Kusupati Jack Hessel Ali Farhadi Yejin Choi 77 209 0 07 Jan 2022
Video-Text Pre-training with Learned Regions Rui Yan Mike Zheng Shou Yixiao Ge Alex Jinpeng Wang Xudong Lin Guanyu Cai Jinhui Tang 54 24 0 02 Dec 2021
Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts Yan Zeng Xinsong Zhang Hang Li VLM CLIP 48 302 0 16 Nov 2021
Palette: Image-to-Image Diffusion Models Chitwan Saharia William Chan Huiwen Chang Chris A. Lee Jonathan Ho Tim Salimans David J. Fleet Mohammad Norouzi DiffM VLM 458 1,617 0 10 Nov 2021
An Empirical Study of Training End-to-End Vision-and-Language Transformers Zi-Yi Dou Yichong Xu Zhe Gan Jianfeng Wang Shuohang Wang ... Pengchuan Zhang Lu Yuan Nanyun Peng Zicheng Liu Michael Zeng VLM 58 371 0 03 Nov 2021
VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts Hangbo Bao Wenhui Wang Li Dong Qiang Liu Owais Khan Mohammed Kriti Aggarwal Subhojit Som Furu Wei VLM MLLM MoE 63 543 0 03 Nov 2021
Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm Yangguang Li Feng Liang Lichen Zhao Yufeng Cui Wanli Ouyang Jing Shao F. Yu Junjie Yan VLM CLIP 137 450 0 11 Oct 2021
Multimodal Incremental Transformer with Visual Grounding for Visual Dialogue Generation Feilong Chen Fandong Meng Xiuyi Chen Peng Li Jie Zhou 68 23 0 17 Sep 2021
GoG: Relation-aware Graph-over-Graph Network for Visual Dialog Feilong Chen Xiuyi Chen Fandong Meng Peng Li Jie Zhou 91 35 0 17 Sep 2021
MURAL: Multimodal, Multitask Retrieval Across Languages Aashi Jain Mandy Guo Krishna Srinivasan Ting-Li Chen Sneha Kudugunta Chao Jia Yinfei Yang Jason Baldridge VLM 135 52 0 10 Sep 2021
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision Zirui Wang Jiahui Yu Adams Wei Yu Zihang Dai Yulia Tsvetkov Yuan Cao VLM MLLM 108 789 0 24 Aug 2021
Adaptive Hierarchical Graph Reasoning with Semantic Coherence for Video-and-Language Inference Juncheng Li Siliang Tang Linchao Zhu Haochen Shi Xuanwen Huang Leilei Gan Yi Yang Yueting Zhuang 54 28 0 26 Jul 2021
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation Junnan Li Ramprasaath R. Selvaraju Akhilesh Deepak Gotmare Shafiq Joty Caiming Xiong Guosheng Lin FaML 161 1,915 0 16 Jul 2021
How Much Can CLIP Benefit Vision-and-Language Tasks? Sheng Shen Liunian Harold Li Hao Tan Joey Tianyi Zhou Anna Rohrbach Kai-Wei Chang Z. Yao Kurt Keutzer CLIP VLM MLLM 248 407 0 13 Jul 2021
OPT: Omni-Perception Pre-Trainer for Cross-Modal Understanding and Generation Jing Liu Xinxin Zhu Fei Liu Longteng Guo Zijia Zhao ... Weining Wang Hanqing Lu Shiyu Zhou Jiajun Zhang Jinqiao Wang 49 37 0 01 Jul 2021
Multimodal Few-Shot Learning with Frozen Language Models Maria Tsimpoukelli Jacob Menick Serkan Cabi S. M. Ali Eslami Oriol Vinyals Felix Hill MLLM 137 766 0 25 Jun 2021
Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training Hongwei Xue Yupan Huang Bei Liu Houwen Peng Jianlong Fu Houqiang Li Jiebo Luo 59 89 0 25 Jun 2021
AudioCLIP: Extending CLIP to Image, Text and Audio A. Guzhov Federico Raue Jörn Hees Andreas Dengel CLIP VLM 78 362 0 24 Jun 2021
CLIP2Video: Mastering Video-Text Retrieval via Image CLIP Han Fang Pengfei Xiong Luhui Xu Yu Chen CLIP VLM 77 294 0 21 Jun 2021
Population-coding and Dynamic-neurons improved Spiking Actor Network for Reinforcement Learning Duzhen Zhang Tielin Zhang Shuncheng Jia Xiang Cheng Bo Xu AI4CE 31 1 0 15 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 123 836 0 14 Jun 2021
E2E-VLP: End-to-End Vision-Language Pre-training Enhanced by Visual Learning Haiyang Xu Ming Yan Chenliang Li Bin Bi Songfang Huang Wenming Xiao Fei Huang VLM 46 118 0 03 Jun 2021
You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detection Yuxin Fang Bencheng Liao Xinggang Wang Jiemin Fang Jiyang Qi Rui Wu Jianwei Niu Wenyu Liu ViT 59 321 0 01 Jun 2021
Segmenter: Transformer for Semantic Segmentation Robin Strudel Ricardo Garcia Pinel Ivan Laptev Cordelia Schmid ViT 157 1,442 0 12 May 2021
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges M. Bronstein Joan Bruna Taco S. Cohen Petar Velivcković GNN 309 1,137 0 27 Apr 2021
MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding Aishwarya Kamath Mannat Singh Yann LeCun Gabriel Synnaeve Ishan Misra Nicolas Carion ObjD VLM 161 876 0 26 Apr 2021
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text Hassan Akbari Liangzhe Yuan Rui Qian Wei-Hong Chuang Shih-Fu Chang Huayu Chen Boqing Gong ViT 289 581 0 22 Apr 2021
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval Huaishao Luo Lei Ji Ming Zhong Yang Chen Wen Lei Nan Duan Tianrui Li CLIP VLM 381 796 0 18 Apr 2021
Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning Zhicheng Huang Zhaoyang Zeng Yupan Huang Bei Liu Dongmei Fu Jianlong Fu VLM ViT 125 271 0 07 Apr 2021
Compressing Visual-linguistic Model via Knowledge Distillation Zhiyuan Fang Jianfeng Wang Xiaowei Hu Lijuan Wang Yezhou Yang Zicheng Liu VLM 80 99 0 05 Apr 2021
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval Max Bain Arsha Nagrani Gül Varol Andrew Zisserman VGen 128 1,154 0 01 Apr 2021