BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

30 January 2023

Silvio Savarese

Papers citing "BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models"

50 / 833 papers shown

Title
What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Instruction Tuning Yifan Du Hangyu Guo Kun Zhou Wayne Xin Zhao Jinpeng Wang Chuyuan Wang Mingchen Cai Ruihua Song Ji-Rong Wen VLM MLLM LRM 75 22 0 02 Nov 2023
MM-VID: Advancing Video Understanding with GPT-4V(ision) Kevin Qinghong Lin Faisal Ahmed Linjie Li Chung-Ching Lin E. Azarnasab ... Lin Liang Zicheng Liu Yumao Lu Ce Liu Lijuan Wang MLLM 28 63 0 30 Oct 2023
Audio-Visual Instance Segmentation Ruohao Guo Yaru Chen Yanyu Qi Wenzhen Yue Dantong Niu ... Wenzhen Yue Ji Shi Qixun Wang Peiliang Zhang Buwen Liang VLM VOS 34 2 0 28 Oct 2023
Image Clustering Conditioned on Text Criteria Sehyun Kwon Jaeseung Park Minkyu Kim Jaewoong Cho Ernest K. Ryu Kangwook Lee VLM 42 11 0 27 Oct 2023
Evaluating Bias and Fairness in Gender-Neutral Pretrained Vision-and-Language Models Laura Cabello Emanuele Bugliarello Stephanie Brandl Desmond Elliott 23 7 0 26 Oct 2023
Leveraging Image-Text Similarity and Caption Modification for the DataComp Challenge: Filtering Track and BYOD Track Shuhei Yokoo Peifei Zhu Yuchi Ishikawa Mikihiro Tanaka Masayoshi Kondo Hirokatsu Kataoka 24 0 0 23 Oct 2023
Semantic and Expressive Variation in Image Captions Across Languages Andre Ye Sebastin Santy Jena D. Hwang Amy X. Zhang Ranjay Krishna VLM 61 3 0 22 Oct 2023
SALMONN: Towards Generic Hearing Abilities for Large Language Models Changli Tang Wenyi Yu Guangzhi Sun Xianzhao Chen Tian Tan Wei Li Lu Lu Zejun Ma Chao Zhang LM&MA AuLLM 39 206 0 20 Oct 2023
Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook Ming Jin Qingsong Wen Keli Zhang Chaoli Zhang Siqiao Xue ... Shirui Pan Vincent S. Tseng Yu Zheng Lei Chen Hui Xiong AI4TS SyDa 35 118 0 16 Oct 2023
VLIS: Unimodal Language Models Guide Multimodal Language Generation Jiwan Chung Youngjae Yu VLM 30 1 0 15 Oct 2023
Beyond Segmentation: Road Network Generation with Multi-Modal LLMs Sumedh Rasal Sanjay K. Boddhu 35 5 0 15 Oct 2023
PaLI-3 Vision Language Models: Smaller, Faster, Stronger Xi Chen Xiao Wang Lucas Beyer Alexander Kolesnikov Jialin Wu ... Keran Rong Tianli Yu Daniel Keysers Xiao-Qi Zhai Radu Soricut MLLM VLM 41 94 0 13 Oct 2023
Towards Robust Multi-Modal Reasoning via Model Selection Xiangyan Liu Rongxue Li Wei Ji Tao Lin LLMAG LRM 37 3 0 12 Oct 2023
Improving Compositional Text-to-image Generation with Large Vision-Language Models Song Wen Guian Fang Renrui Zhang Peng Gao Hao Dong Dimitris N. Metaxas 25 17 0 10 Oct 2023
Fine-grained Audio-Visual Joint Representations for Multimodal Large Language Models Guangzhi Sun Wenyi Yu Changli Tang Xianzhao Chen Tian Tan Wei Li Lu Lu Zejun Ma Chao Zhang 33 12 0 09 Oct 2023
Sentence-level Prompts Benefit Composed Image Retrieval Yang Bai Xinxing Xu Yong-Jin Liu Salman Khan Fahad Khan Wangmeng Zuo Rick Siow Mong Goh Chun-Mei Feng 38 26 0 09 Oct 2023
Improving Discriminative Multi-Modal Learning with Large-Scale Pre-Trained Models Chenzhuang Du Yue Zhao Chonghua Liao Jiacheng You Jie Fu Hang Zhao 44 2 0 08 Oct 2023
Improving Automatic VQA Evaluation Using Large Language Models Oscar Manas Benno Krojer Aishwarya Agrawal 27 21 0 04 Oct 2023
Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Driving Tushar Choudhary Vikrant Dewangan Shivam Chandhok Shubham Priyadarshan Anushka Jain A. K. Singh Siddharth Srivastava Krishna Murthy Jatavallabhula K. M. Krishna 50 59 0 03 Oct 2023
Towards reporting bias in visual-language datasets: bimodal augmentation by decoupling object-attribute association Qiyu Wu Mengjie Zhao Yutong He Lang Huang Junya Ono Hiromi Wakaki Yuki Mitsufuji 33 4 0 02 Oct 2023
Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning Mustafa Shukor Alexandre Ramé Corentin Dancette Matthieu Cord LRM MLLM 46 20 0 01 Oct 2023
Data Filtering Networks Alex Fang Albin Madappally Jose Amit Jain Ludwig Schmidt Alexander Toshev Vaishaal Shankar CLIP 46 125 0 29 Sep 2023
AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model Avamarie Brueggeman Andrea Madotto Zhaojiang Lin Tushar Nagarajan Matt Smith ... Peyman Heidari Yue Liu Kavya Srinet Babak Damavandi Anuj Kumar MLLM 34 93 0 27 Sep 2023
Jointly Training Large Autoregressive Multimodal Models Emanuele Aiello L. Yu Yixin Nie Armen Aghajanyan Barlas Oğuz 19 29 0 27 Sep 2023
Tackling VQA with Pretrained Foundation Models without Further Training Alvin De Jun Tan Bingquan Shen MLLM 37 1 0 27 Sep 2023
InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition Pan Zhang Xiaoyi Wang Bin Wang Yuhang Cao Chao Xu ... Conghui He Xingcheng Zhang Yu Qiao Da Lin Jiaqi Wang MLLM 80 225 0 26 Sep 2023
BLIP-Adapter: Parameter-Efficient Transfer Learning for Mobile Screenshot Captioning Ching-Yu Chiang I-Hua Chang Shih-Wei Liao 53 1 0 26 Sep 2023
Species196: A One-Million Semi-supervised Dataset for Fine-grained Species Recognition W. He Kai Han Ying Nie Chengcheng Wang Yunhe Wang VLM 48 6 0 25 Sep 2023
ContextRef: Evaluating Referenceless Metrics For Image Description Generation Elisa Kreiss E. Zelikman Christopher Potts Nick Haber 29 5 0 21 Sep 2023
PatFig: Generating Short and Long Captions for Patent Figures Dana Aubakirova Kim Gerdes Lufei Liu 17 9 0 15 Sep 2023
Sight Beyond Text: Multi-Modal Training Enhances LLMs in Truthfulness and Ethics Haoqin Tu Bingchen Zhao Chen Wei Cihang Xie MLLM 39 14 0 13 Sep 2023
DreamStyler: Paint by Style Inversion with Text-to-Image Diffusion Models Namhyuk Ahn Junsoo Lee Chunggi Lee Kunhee Kim Daesik Kim Seung-Hun Nam Kibeom Hong DiffM 18 21 0 13 Sep 2023
Cognitive Mirage: A Review of Hallucinations in Large Language Models Hongbin Ye Tong Liu Aijia Zhang Wei Hua Weiqiang Jia HILM 48 77 0 13 Sep 2023
Can you text what is happening? Integrating pre-trained language encoders into trajectory prediction models for autonomous driving Ali Keysan Andreas Look Eitan Kosman Gonca Gürsun Jörg Wagner Yu Yao Barbara Rakitsch 27 29 0 11 Sep 2023
Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models Yangyi Chen Karan Sikka Michael Cogswell Heng Ji Ajay Divakaran LRM 36 25 0 08 Sep 2023
ImageBind-LLM: Multi-modality Instruction Tuning Jiaming Han Renrui Zhang Wenqi Shao Peng Gao Peng Xu ... Yafei Wen Xiaoxin Chen Xiangyu Yue Hongsheng Li Yu Qiao MLLM 49 117 0 07 Sep 2023
Reuse and Diffuse: Iterative Denoising for Text-to-Video Generation Jiaxi Gu Shicong Wang Haoyu Zhao Tianyi Lu Xing Zhang Zuxuan Wu Songcen Xu Wei Zhang Yu-Gang Jiang Hang Xu DiffM VGen 39 44 0 07 Sep 2023
DetermiNet: A Large-Scale Diagnostic Dataset for Complex Visually-Grounded Referencing using Determiners Clarence Lee M Ganesh Kumar Cheston Tan 28 3 0 07 Sep 2023
CIEM: Contrastive Instruction Evaluation Method for Better Instruction Tuning Hongyu Hu Jiyuan Zhang Minyi Zhao Zhenbang Sun MLLM 25 42 0 05 Sep 2023
OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentation Zhening Huang Xiaoyang Wu Xi Chen Hengshuang Zhao Lei Zhu Joan Lasenby ISeg 3DPC VLM 55 46 0 01 Sep 2023
VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation Xin Li Wenqing Chu Ye Wu Weihang Yuan Fanglong Liu Qi Zhang Fu Li Haocheng Feng Errui Ding Jingdong Wang VGen 45 52 0 01 Sep 2023
Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Models Yupan Huang Zaiqiao Meng Fangyu Liu Yixuan Su Nigel Collier Yutong Lu MLLM 41 22 0 31 Aug 2023
AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models Zhaopeng Gu Bingke Zhu Guibo Zhu Yingying Chen Ming Tang Jinqiao Wang VLM MLLM 34 102 0 29 Aug 2023
Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization Tao Yang Rongyuan Wu Peiran Ren Xuansong Xie Lei Zhang DiffM 50 138 0 28 Aug 2023
InstructionGPT-4: A 200-Instruction Paradigm for Fine-Tuning MiniGPT-4 Lai Wei Zihao Jiang Weiran Huang Lichao Sun VLM MLLM 24 56 0 23 Aug 2023
ViCo: Engaging Video Comment Generation with Human Preference Rewards Yuchong Sun Bei Liu Xu Chen Ruihua Song Jianlong Fu VGen 22 2 0 22 Aug 2023
An Examination of the Compositionality of Large Generative Vision-Language Models Teli Ma Rong Li Junwei Liang CoGe 34 2 0 21 Aug 2023
UniDoc: A Universal Large Multimodal Model for Simultaneous Text Detection, Recognition, Spotting and Understanding Hao Feng Zijian Wang Jingqun Tang Jinghui Lu Wen-gang Zhou Houqiang Li Can Huang MLLM VLM 45 47 0 19 Aug 2023
Causal Intersectionality and Dual Form of Gradient Descent for Multimodal Analysis: a Case Study on Hateful Memes Yosuke Miyanishi M. Nguyen 34 2 0 19 Aug 2023
Text-Only Training for Visual Storytelling Yuechen Wang Wen-gang Zhou Zhenbo Lu Houqiang Li DiffM 28 2 0 17 Aug 2023