PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models

23 May 2022

Yuan Yao

Qi-An Chen

Ao Zhang

Wei Ji

Zhiyuan Liu

Maosong Sun

Papers citing "PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models"

40 / 40 papers shown

Title
Pose-Aware Weakly-Supervised Action Segmentation Seth Z. Zhao Reza Ghoddoosian Isht Dwivedi Nakul Agarwal Behzad Dariush 34 0 0 08 Apr 2025
Visual Position Prompt for MLLM based Visual Grounding Wei Tang Yanpeng Sun Qinying Gu Zechao Li VLM 47 0 0 19 Mar 2025
ComAlign: Compositional Alignment in Vision-Language Models Ali Abdollah Amirmohammad Izadi Armin Saghafian Reza Vahidimajd Mohammad Mozafari Amirreza Mirzaei Mohammadmahdi Samiei M. Baghshah CoGe VLM 30 0 0 12 Sep 2024
EventLens: Leveraging Event-Aware Pretraining and Cross-modal Linking Enhances Visual Commonsense Reasoning Mingjie Ma Zhihuan Yu Yichao Ma Guohui Li LRM 33 1 0 22 Apr 2024
From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models Rongjie Li Songyang Zhang Dahua Lin Kai-xiang Chen Xuming He VLM 37 14 0 01 Apr 2024
Voila-A: Aligning Vision-Language Models with User's Gaze Attention Kun Yan Lei Ji Zeyu Wang Yuntao Wang Nan Duan Shuai Ma 50 7 0 22 Dec 2023
Lenna: Language Enhanced Reasoning Detection Assistant Fei Wei Xinyu Zhang Ailing Zhang Bo-Wen Zhang Xiangxiang Chu MLLM LRM 27 23 0 05 Dec 2023
DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback Yangyi Chen Karan Sikka Michael Cogswell Heng Ji Ajay Divakaran 24 58 0 16 Nov 2023
NExT-Chat: An LMM for Chat, Detection and Segmentation Ao Zhang Yuan Yao Wei Ji Zhiyuan Liu Tat-Seng Chua MLLM VLM 40 51 0 08 Nov 2023
Meta-Adapter: An Online Few-shot Learner for Vision-Language Model Cheng Cheng Lin Song Ruoyi Xue Hang Wang Hongbin Sun Yixiao Ge Ying Shan VLM ObjD 27 18 0 07 Nov 2023
Augment the Pairs: Semantics-Preserving Image-Caption Pair Augmentation for Grounding-Based Vision and Language Models Jingru Yi Burak Uzkent Oana Ignat Zili Li Amanmeet Garg Xiang Yu Linda Liu VLM 25 1 0 05 Nov 2023
Domain-wise Invariant Learning for Panoptic Scene Graph Generation Li Li Youxuan Qin Wei Ji Yuxiao Zhou Roger Zimmermann 27 4 0 09 Oct 2023
Pre-training on Synthetic Driving Data for Trajectory Prediction Yiheng Li Seth Z. Zhao Chenfeng Xu Chen Tang Chenran Li Mingyu Ding M. Tomizuka Wei Zhan 32 11 0 18 Sep 2023
Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models Yangyi Chen Karan Sikka Michael Cogswell Heng Ji Ajay Divakaran LRM 34 24 0 08 Sep 2023
Position-Enhanced Visual Instruction Tuning for Multimodal Large Language Models Chi Chen Ruoyu Qin Fuwen Luo Xiaoyue Mi Peng Li Maosong Sun Yang Liu MLLM VLM 14 45 0 25 Aug 2023
SCoRD: Subject-Conditional Relation Detection with Text-Augmented Data Ziyan Yang Kushal Kafle Zhe-nan Lin Scott D. Cohen Zhihong Ding Vicente Ordonez 21 1 0 24 Aug 2023
RegionBLIP: A Unified Multi-modal Pre-training Framework for Holistic and Regional Comprehension Qiang-feng Zhou Chaohui Yu Shaofeng Zhang Sitong Wu Zhibin Wang Fan Wang 34 27 0 03 Aug 2023
Panoptic Scene Graph Generation with Semantics-Prototype Learning Li Li Wei Ji Yiming Wu Meng Li Youxuan Qin Lina Wei Roger Zimmermann 26 35 0 28 Jul 2023
GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest Shilong Zhang Pei Sun Shoufa Chen Min Xiao Wenqi Shao Wenwei Zhang Yu Liu Kai-xiang Chen Ping Luo VLM MLLM 83 224 0 07 Jul 2023
Kosmos-2: Grounding Multimodal Large Language Models to the World Zhiliang Peng Wenhui Wang Li Dong Y. Hao Shaohan Huang Shuming Ma Furu Wei MLLM ObjD VLM 38 694 0 26 Jun 2023
Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining Emanuele Bugliarello Aida Nematzadeh Lisa Anne Hendricks SSL 22 5 0 23 May 2023
TreePrompt: Learning to Compose Tree Prompts for Explainable Visual Grounding Chenchi Zhang Jun Xiao Lei Chen Jian Shao Long Chen VLM LRM 22 2 0 19 May 2023
Measuring Progress in Fine-grained Vision-and-Language Understanding Emanuele Bugliarello Laurent Sartran Aishwarya Agrawal Lisa Anne Hendricks Aida Nematzadeh VLM 28 22 0 12 May 2023
Unified Visual Relationship Detection with Vision and Language Models Long Zhao Liangzhe Yuan Boqing Gong Yin Cui Florian Schroff Ming Yang Hartwig Adam Ting Liu ObjD 27 9 0 16 Mar 2023
ViM: Vision Middleware for Unified Downstream Transferring Yutong Feng Biao Gong Jianwen Jiang Yiliang Lv Yujun Shen Deli Zhao Jingren Zhou 32 1 0 13 Mar 2023
Learning to Agree on Vision Attention for Visual Commonsense Reasoning Zhenyang Li Yangyang Guo Ke-Jyun Wang Fan Liu Liqiang Nie Mohan S. Kankanhalli 32 10 0 04 Feb 2023
Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Models Cheng Ma Yang Liu Jiankang Deng Lingxi Xie Weiming Dong Changsheng Xu VLM VPVLM 26 43 0 04 Nov 2022
Prompt Tuning for Generative Multimodal Pretrained Models Han Yang Junyang Lin An Yang Peng Wang Chang Zhou Hongxia Yang VLM LRM VPVLM 37 30 0 04 Aug 2022
Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks Jiasen Lu Christopher Clark Rowan Zellers Roozbeh Mottaghi Aniruddha Kembhavi ObjD VLM MLLM 51 392 0 17 Jun 2022
Fine-Grained Scene Graph Generation with Data Transfer Ao Zhang Yuan Yao Qián Chen Wei Ji Zhiyuan Liu Maosong Sun Tat-Seng Chua 19 89 0 22 Mar 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 305 7,434 0 11 Nov 2021
CLIP-Adapter: Better Vision-Language Models with Feature Adapters Peng Gao Shijie Geng Renrui Zhang Teli Ma Rongyao Fang Yongfeng Zhang Hongsheng Li Yu Qiao VLM CLIP 60 982 0 09 Oct 2021
CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models Yuan Yao Ao Zhang Zhengyan Zhang Zhiyuan Liu Tat-Seng Chua Maosong Sun MLLM VPVLM VLM 202 221 0 24 Sep 2021
Pix2seq: A Language Modeling Framework for Object Detection Ting-Li Chen Saurabh Saxena Lala Li David J. Fleet Geoffrey E. Hinton MLLM ViT VLM 235 344 0 22 Sep 2021
ActionCLIP: A New Paradigm for Video Action Recognition Mengmeng Wang Jiazheng Xing Yong Liu VLM 149 362 0 17 Sep 2021
Open-vocabulary Object Detection via Vision and Language Knowledge Distillation Xiuye Gu Tsung-Yi Lin Weicheng Kuo Yin Cui VLM ObjD 223 898 0 28 Apr 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 298 3,693 0 11 Feb 2021
Unifying Vision-and-Language Tasks via Text Generation Jaemin Cho Jie Lei Hao Tan Mohit Bansal MLLM 253 525 0 04 Feb 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,918 0 31 Dec 2020
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 413 2,584 0 03 Sep 2019