BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

30 January 2023

Silvio Savarese

Papers citing "BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models"

50 / 822 papers shown

Title
UP-DP: Unsupervised Prompt Learning for Data Pre-Selection with Vision-Language Models Xin Li Sima Behpour T. Doan Wenbin He Liangke Gou Liu Ren VLM 50 3 0 20 Jul 2023
Identifying Interpretable Subspaces in Image Representations Neha Kalibhat S. Bhardwaj Bayan Bruss Hamed Firooz Maziar Sanjabi S. Feizi FAtt 42 26 0 20 Jul 2023
AnyDoor: Zero-shot Object-level Image Customization Xi Chen Lianghua Huang Yu Liu Yujun Shen Deli Zhao Hengshuang Zhao DiffM 46 257 0 18 Jul 2023
BuboGPT: Enabling Visual Grounding in Multi-Modal LLMs Yang Zhao Zhijie Lin Daquan Zhou Zilong Huang Jiashi Feng Bingyi Kang MLLM 44 107 0 17 Jul 2023
Image Captions are Natural Prompts for Text-to-Image Models Shiye Lei Hao Chen Senyang Zhang Bo Zhao Dacheng Tao VLM 32 19 0 17 Jul 2023
Vision Language Transformers: A Survey Clayton Fields C. Kennington VLM 28 5 0 06 Jul 2023
JourneyDB: A Benchmark for Generative Image Understanding Keqiang Sun Junting Pan Yuying Ge Hao Li Haodong Duan ... Yi Wang Jifeng Dai Yu Qiao Limin Wang Hongsheng Li 54 103 0 03 Jul 2023
Image Background Serves as Good Proxy for Out-of-distribution Data Sen Pei 41 1 0 02 Jul 2023
VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution S. Hall F. G. Abrantes Hanwen Zhu Grace A. Sodunke Aleksandar Shtedritski Hannah Rose Kirk CoGe 25 39 0 21 Jun 2023
Improving Image Captioning Descriptiveness by Ranking and LLM-based Fusion Simone Bianco Luigi Celona Marco Donzella Paolo Napoletano 34 18 0 20 Jun 2023
Investigating Prompting Techniques for Zero- and Few-Shot Visual Question Answering Rabiul Awal Le Zhang Aishwarya Agrawal LRM 46 12 0 16 Jun 2023
Towards AGI in Computer Vision: Lessons Learned from GPT and Large Language Models Lingxi Xie Longhui Wei Xiaopeng Zhang Kaifeng Bi Xiaotao Gu Jianlong Chang Qi Tian 33 7 0 14 Jun 2023
Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions Weizhen He Yihe Deng Shixiang Tang Qihao Chen Qingsong Xie ... Feng Zhu Rui Zhao Wanli Ouyang Donglian Qi Yunfeng Yan 77 19 0 13 Jun 2023
Sticker820K: Empowering Interactive Retrieval with Stickers Sijie Zhao Yixiao Ge Zhongang Qi Lin Song Xiaohan Ding Zehua Xie Ying Shan 34 6 0 12 Jun 2023
Valley: Video Assistant with Large Language model Enhanced abilitY Ruipu Luo Ziwang Zhao Min Yang Junwei Dong Da Li Pengcheng Lu Tao Wang Linmei Hu Ming-Hui Qiu MLLM 51 190 0 12 Jun 2023
AircraftVerse: A Large-Scale Multimodal Dataset of Aerial Vehicle Designs Adam D. Cobb Anirban Roy Daniel Elenius F. M. Heim Brian Swenson ... Theodore Bapty Joseph Hite K. Ramani Christopher McComb Susmit Jha 20 7 0 08 Jun 2023
LRVS-Fashion: Extending Visual Search with Referring Instructions Simon Lepage Jérémie Mary David Picard 25 1 0 05 Jun 2023
Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models Shuo Chen Jindong Gu Zhen Han Yunpu Ma Philip Torr Volker Tresp VPVLM VLM 34 17 0 03 Jun 2023
PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigation Jialu Li Joey Tianyi Zhou DiffM 31 49 0 30 May 2023
Learning without Forgetting for Vision-Language Models Da-Wei Zhou Yuanhan Zhang Jingyi Ning Jingyi Ning De-Chuan Zhan De-Chuan Zhan Ziwei Liu VLM CLL 74 37 0 30 May 2023
Contextual Object Detection with Multimodal Large Language Models Yuhang Zang Wei Li Jun Han Kaiyang Zhou Chen Change Loy ObjD VLM MLLM 32 78 0 29 May 2023
InstructEdit: Improving Automatic Masks for Diffusion-based Image Editing With User Instructions Qian Wang Biao Zhang Michael Birsak Peter Wonka DiffM 30 31 0 29 May 2023
Towards Consistent Video Editing with Text-to-Image Diffusion Models Zicheng Zhang Bonan li Xuecheng Nie Congying Han Tiande Guo Luoqi Liu DiffM 20 24 0 27 May 2023
NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models Gengze Zhou Yicong Hong Qi Wu ELM LM&Ro LLMAG LRM 25 142 0 26 May 2023
A Reminder of its Brittleness: Language Reward Shaping May Hinder Learning for Instruction Following Agents Sukai Huang N. Lipovetzky Trevor Cohn 35 2 0 26 May 2023
ChatBridge: Bridging Modalities with Large Language Model as a Language Catalyst Zijia Zhao Longteng Guo Tongtian Yue Si-Qing Chen Shuai Shao Xinxin Zhu Zehuan Yuan Jing Liu MLLM 40 52 0 25 May 2023
Visual Programming for Text-to-Image Generation and Evaluation Jaemin Cho Abhaysinh Zala Joey Tianyi Zhou MLLM 29 50 0 24 May 2023
IdealGPT: Iteratively Decomposing Vision and Language Reasoning via Large Language Models Haoxuan You Rui Sun Zhecan Wang Long Chen Gengyu Wang Hammad A. Ayyubi Kai-Wei Chang Shih-Fu Chang VLM MLLM LRM 52 43 0 24 May 2023
Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining Emanuele Bugliarello Aida Nematzadeh Lisa Anne Hendricks SSL 30 5 0 23 May 2023
LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models Long Lian Boyi Li Adam Yala Trevor Darrell 43 152 0 23 May 2023
VLAB: Enhancing Video Language Pre-training by Feature Adapting and Blending Xingjian He Sihan Chen Fan Ma Zhicheng Huang Xiaojie Jin Zikang Liu Dongmei Fu Yi Yang Jiaheng Liu Jiashi Feng VLM CLIP 23 17 0 22 May 2023
AudioToken: Adaptation of Text-Conditioned Diffusion Models for Audio-to-Image Generation Guy Yariv Itai Gat Lior Wolf Yossi Adi Idan Schwartz DiffM 23 21 0 22 May 2023
ViT-TTS: Visual Text-to-Speech with Scalable Diffusion Transformer Huadai Liu Rongjie Huang Xuan Lin Wenqiang Xu Maozong Zheng Hong Chen Jinzheng He Zhou Zhao DiffM 47 20 0 22 May 2023
Enhancing Vision-Language Pre-Training with Jointly Learned Questioner and Dense Captioner Zikang Liu Sihan Chen Longteng Guo Handong Li Xingjian He Jiaheng Liu 15 1 0 19 May 2023
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities Peng Wang Shijie Wang Junyang Lin Shuai Bai Xiaohuan Zhou Jingren Zhou Xinggang Wang Chang Zhou VLM MLLM ObjD 48 115 0 18 May 2023
MedBLIP: Bootstrapping Language-Image Pre-training from 3D Medical Images and Texts Qiuhui Chen Xinyue Hu Zirui Wang Yi Hong LM&MA MedIm 22 34 0 18 May 2023
What You See is What You Read? Improving Text-Image Alignment Evaluation Michal Yarom Yonatan Bitton Soravit Changpinyo Roee Aharoni Jonathan Herzig Oran Lang E. Ofek Idan Szpektor EGVM 59 74 0 17 May 2023
Evaluating Object Hallucination in Large Vision-Language Models Yifan Li Yifan Du Kun Zhou Jinpeng Wang Wayne Xin Zhao Ji-Rong Wen MLLM LRM 122 702 0 17 May 2023
Make-A-Protagonist: Generic Video Editing with An Ensemble of Experts Yuyang Zhao Enze Xie Lanqing Hong Zhenguo Li G. Lee DiffM VGen 35 32 0 15 May 2023
Bridging the Domain Gap: Self-Supervised 3D Scene Understanding with Foundation Models Zhimin Chen Longlong Jing Yingwei Li Bing Li 29 31 0 15 May 2023
ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding Le Xue Ning Yu Shu Zhen Zhang Artemis Panagopoulou Junnan Li ... Jiajun Wu Caiming Xiong Ran Xu Juan Carlos Niebles Silvio Savarese 24 115 0 14 May 2023
Self-Chained Image-Language Model for Video Localization and Question Answering Shoubin Yu Jaemin Cho Prateek Yadav Joey Tianyi Zhou 54 130 0 11 May 2023
VPGTrans: Transfer Visual Prompt Generator across LLMs Ao Zhang Hao Fei Yuan Yao Wei Ji Li Li Zhiyuan Liu Tat-Seng Chua MLLM VLM 32 85 0 02 May 2023
MVP-SEG: Multi-View Prompt Learning for Open-Vocabulary Semantic Segmentation Jie Guo Qimeng Wang Yan Gao Xiaolong Jiang Xu Tang Yao Hu Baochang Zhang VLM 37 11 0 14 Apr 2023
On the Opportunities and Challenges of Foundation Models for Geospatial Artificial Intelligence Gengchen Mai Weiming Huang Jin Sun Suhang Song Deepak Mishra ... Yingjie Hu Chris Cundy Ziyuan Li Rui Zhu Ni Lao AI4CE 32 123 0 13 Apr 2023
HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Models Eslam Mohamed Bakr Pengzhan Sun Xiaoqian Shen Faizan Farooq Khan Li Erran Li Mohamed Elhoseiny VLM 24 76 0 11 Apr 2023
Video ChatCaptioner: Towards Enriched Spatiotemporal Descriptions Jun Chen Deyao Zhu Kilichbek Haydarov Xiang Li Mohamed Elhoseiny 25 37 0 09 Apr 2023
MaMMUT: A Simple Architecture for Joint Learning for MultiModal Tasks Weicheng Kuo A. Piergiovanni Dahun Kim Xiyang Luo Benjamin Caine ... Luowei Zhou Andrew M. Dai Zhifeng Chen Claire Cui A. Angelova MLLM VLM 29 23 0 29 Mar 2023
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention Renrui Zhang Jiaming Han Chris Liu Peng Gao Aojun Zhou Xiangfei Hu Shilin Yan Pan Lu Hongsheng Li Yu Qiao MLLM 71 745 0 28 Mar 2023
EVA-CLIP: Improved Training Techniques for CLIP at Scale Quan-Sen Sun Yuxin Fang Ledell Yu Wu Xinlong Wang Yue Cao CLIP VLM 78 470 0 27 Mar 2023