v1v2v3 (latest)

BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

30 January 2023

Silvio Savarese

Papers citing "BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models"

50 / 2,338 papers shown

Title
AutoEval-Video: An Automatic Benchmark for Assessing Large Vision Language Models in Open-Ended Video Question Answering Xiuyuan Chen Yuan Lin Yuchen Zhang Weiran Huang ELM MLLM 98 27 0 25 Nov 2023
Griffon: Spelling out All Object Locations at Any Granularity with Large Language Models Yufei Zhan Yousong Zhu Zhiyang Chen Fan Yang E. Goles Jinqiao Wang ObjD 114 17 0 24 Nov 2023
Boosting3D: High-Fidelity Image-to-3D by Boosting 2D Diffusion Prior to 3D Prior with Progressive Learning Kai Yu Jinlin Liu Mengyang Feng Miaomiao Cui Xuansong Xie 101 6 0 22 Nov 2023
Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents Zhuosheng Zhang Yao Yao Aston Zhang Xiangru Tang Xinbei Ma ... Yiming Wang Mark B. Gerstein Rui Wang Gongshen Liu Hai Zhao LLMAG LM&Ro LRM 151 61 0 20 Nov 2023
Towards Open-Ended Visual Recognition with Large Language Model Qihang Yu Xiaohui Shen Liang-Chieh Chen VLM 74 8 0 14 Nov 2023
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks Bin Xiao Haiping Wu Weijian Xu Xiyang Dai Houdong Hu Yumao Lu Michael Zeng Ce Liu Lu Yuan VLM 114 174 0 10 Nov 2023
FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts Yichen Gong Delong Ran Jinyuan Liu Conglei Wang Tianshuo Cong Anyu Wang Sisi Duan Xiaoyun Wang MLLM 235 160 0 09 Nov 2023
LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents Shilong Liu Hao Cheng Haotian Liu Hao Zhang Feng Li ... Hang Su Jun Zhu Lei Zhang Jianfeng Gao Chun-yue Li MLLM VLM 113 126 0 09 Nov 2023
Chain of Images for Intuitively Reasoning Fanxu Meng Haotong Yang Yiding Wang Muhan Zhang LRM 78 10 0 09 Nov 2023
A Data Perspective on Enhanced Identity Preservation for Diffusion Personalization Xingzhe He Zhiwen Cao Nicholas I. Kolkin Lantao Yu Kun Wan Helge Rhodin Ratheesh Kalarot 93 14 0 07 Nov 2023
Sculpting Holistic 3D Representation in Contrastive Language-Image-3D Pre-training Yipeng Gao Zeyu Wang Wei-Shi Zheng Cihang Xie Yuyin Zhou 3DPC 150 10 0 03 Nov 2023
LLM4Drive: A Survey of Large Language Models for Autonomous Driving Zhenjie Yang Xiaosong Jia Hongyang Li Junchi Yan ELM 142 121 0 02 Nov 2023
What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Instruction Tuning Yifan Du Hangyu Guo Kun Zhou Wayne Xin Zhao Jinpeng Wang Chuyuan Wang Mingchen Cai Ruihua Song Ji-Rong Wen VLM MLLM LRM 185 23 0 02 Nov 2023
De-Diffusion Makes Text a Strong Cross-Modal Interface Chen Wei Chenxi Liu Siyuan Qiao Zhishuai Zhang Alan Yuille Jiahui Yu VLM DiffM 103 11 0 01 Nov 2023
CoLLM: Integrating Collaborative Embeddings into Large Language Models for Recommendation Yang Zhang Fuli Feng Jizhi Zhang Keqin Bao Qifan Wang Xiangnan He 97 88 0 30 Oct 2023
Audio-Visual Instance Segmentation Ruohao Guo Yaru Chen Yanyu Qi Wenzhen Yue Dantong Niu ... Wenzhen Yue Ji Shi Qixun Wang Peiliang Zhang Buwen Liang VLM VOS 125 2 0 28 Oct 2023
Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Models Tsun-Hsuan Wang Alaa Maalouf Wei Xiao Yutong Ban Alexander Amini Guy Rosman S. Karaman Daniela Rus 73 46 0 26 Oct 2023
Woodpecker: Hallucination Correction for Multimodal Large Language Models Shukang Yin Chaoyou Fu Sirui Zhao Tong Xu Hao Wang Dianbo Sui Yunhang Shen Ke Li Xingguo Sun Enhong Chen VLM MLLM 108 133 0 24 Oct 2023
Large Language Models can Share Images, Too! Young-Jun Lee Dokyong Lee Joo Won Sung Jonghwan Hyeon Ho-Jin Choi MLLM 79 2 0 23 Oct 2023
Leveraging Image-Text Similarity and Caption Modification for the DataComp Challenge: Filtering Track and BYOD Track Shuhei Yokoo Peifei Zhu Yuchi Ishikawa Mikihiro Tanaka Masayoshi Kondo Hirokatsu Kataoka 26 1 0 23 Oct 2023
Towards image compression with perfect realism at ultra-low bitrates Marlene Careil Matthew Muckley Jakob Verbeek Stéphane Lathuilière DiffM 90 57 0 16 Oct 2023
Joint Music and Language Attention Models for Zero-shot Music Tagging Xingjian Du Zhesong Yu Jiaju Lin Bilei Zhu Qiuqiang Kong BDL VLM 65 9 0 16 Oct 2023
Beyond Segmentation: Road Network Generation with Multi-Modal LLMs Sumedh Rasal Sanjay K. Boddhu 71 5 0 15 Oct 2023
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning Jun Chen Deyao Zhu Xiaoqian Shen Xiang Li Zechun Liu Pengchuan Zhang Raghuraman Krishnamoorthi Vikas Chandra Yunyang Xiong Mohamed Elhoseiny MLLM 253 473 0 14 Oct 2023
Fine-grained Audio-Visual Joint Representations for Multimodal Large Language Models Guangzhi Sun Wenyi Yu Changli Tang Xianzhao Chen Tian Tan Wei Li Lu Lu Zejun Ma Chao Zhang 101 13 0 09 Oct 2023
Improving Discriminative Multi-Modal Learning with Large-Scale Pre-Trained Models Chenzhuang Du Yue Zhao Chonghua Liao Jiacheng You Jie Fu Hang Zhao 86 2 0 08 Oct 2023
Kosmos-G: Generating Images in Context with Multimodal Large Language Models Xichen Pan Li Dong Shaohan Huang Zhiliang Peng Wenhu Chen Furu Wei VLM 152 68 0 04 Oct 2023
ReForm-Eval: Evaluating Large Vision Language Models via Unified Re-Formulation of Task-Oriented Benchmarks Zejun Li Ye Wang Mengfei Du Qingwen Liu Binhao Wu ... Zhihao Fan Jie Fu Jingjing Chen Xuanjing Huang Zhongyu Wei 118 15 0 04 Oct 2023
Towards End-to-End Embodied Decision Making via Multi-modal Large Language Model: Explorations with GPT4-Vision and Beyond Liang Chen Yichi Zhang Shuhuai Ren Haozhe Zhao Zefan Cai Yuchi Wang Peiyi Wang Tianyu Liu Baobao Chang LM&Ro LLMAG 182 44 0 03 Oct 2023
Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving Long Chen Oleg Sinavski Jan Hünermann Alice Karnsund Andrew James Willmott Danny Birch Daniel Maund Jamie Shotton MLLM 127 210 0 03 Oct 2023
Towards reporting bias in visual-language datasets: bimodal augmentation by decoupling object-attribute association Qiyu Wu Mengjie Zhao Yutong He Lang Huang Junya Ono Hiromi Wakaki Yuki Mitsufuji 107 5 0 02 Oct 2023
Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning Mustafa Shukor Alexandre Ramé Corentin Dancette Matthieu Cord LRM MLLM 113 22 0 01 Oct 2023
Data Filtering Networks Alex Fang Albin Madappally Jose Amit Jain Ludwig Schmidt Alexander Toshev Vaishaal Shankar CLIP 134 144 0 29 Sep 2023
Jointly Training Large Autoregressive Multimodal Models Emanuele Aiello L. Yu Yixin Nie Armen Aghajanyan Barlas Oğuz 123 31 0 27 Sep 2023
InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition Pan Zhang Xiaoyi Wang Bin Wang Yuhang Cao Chao Xu ... Conghui He Xingcheng Zhang Yu Qiao Da Lin Jiaqi Wang MLLM 189 241 0 26 Sep 2023
Instruction-Following Speech Recognition Cheng-I Jeff Lai Zhiyun Lu Liangliang Cao Ruoming Pang AuLLM 75 6 0 18 Sep 2023
DreamStyler: Paint by Style Inversion with Text-to-Image Diffusion Models Namhyuk Ahn Junsoo Lee Chunggi Lee Kunhee Kim Daesik Kim Seung-Hun Nam Kibeom Hong DiffM 89 24 0 13 Sep 2023
Cognitive Mirage: A Review of Hallucinations in Large Language Models Hongbin Ye Tong Liu Aijia Zhang Wei Hua Weiqiang Jia HILM 120 81 0 13 Sep 2023
Can you text what is happening? Integrating pre-trained language encoders into trajectory prediction models for autonomous driving Ali Keysan Andreas Look Eitan Kosman Gonca Gürsun Jörg Wagner Yu Yao Barbara Rakitsch 90 31 0 11 Sep 2023
Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models Yangyi Chen Karan Sikka Michael Cogswell Heng Ji Ajay Divakaran LRM 99 27 0 08 Sep 2023
Reuse and Diffuse: Iterative Denoising for Text-to-Video Generation Jiaxi Gu Shicong Wang Haoyu Zhao Tianyi Lu Xing Zhang Zuxuan Wu Songcen Xu Wei Zhang Yu-Gang Jiang Hang Xu DiffM VGen 82 48 0 07 Sep 2023
Zero-Shot Recommendations with Pre-Trained Large Language Models for Multimodal Nudging Rachel M. Harrison Anton Dereventsov A. Bibin 76 9 0 02 Sep 2023
BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writing Chen Wang Minpeng Liao Zhongqiang Huang Jinliang Lu Junhong Wu Yuchen Liu Chengqing Zong Jiajun Zhang AuLLM 131 45 0 02 Sep 2023
Large Content And Behavior Models To Understand, Simulate, And Optimize Content And Behavior Ashmit Khandelwal Aditya Agrawal Aanisha Bhattacharyya Yaman Kumar Singla Somesh Singh ... Ishita Dasgupta Stefano Petrangeli R. Shah Changyou Chen Balaji Krishnamurthy 81 8 0 01 Sep 2023
CoVR: Learning Composed Video Retrieval from Web Video Captions Lucas Ventura Antoine Yang Cordelia Schmid Gül Varol 75 21 0 28 Aug 2023
ViCo: Engaging Video Comment Generation with Human Preference Rewards Yuchong Sun Bei Liu Xu Chen Ruihua Song Jianlong Fu VGen 55 2 0 22 Aug 2023
DUAW: Data-free Universal Adversarial Watermark against Stable Diffusion Customization Xiaoyu Ye Hao Huang Jiaqi An Yongtao Wang WIGM 85 23 0 19 Aug 2023
PUMGPT: A Large Vision-Language Model for Product Understanding Wei Xue Zongyi Guo Baoliang Cui Zengming Tang Weiwei Zhang Haihong Tang Shuhui Wu Weiming Lu VLM 72 2 0 18 Aug 2023
Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenes Zehan Wang Haifeng Huang Yang Zhao Ziang Zhang Zhou Zhao 118 73 0 17 Aug 2023
Large Language Models and Foundation Models in Smart Agriculture: Basics, Opportunities, and Challenges Jiajia Li Mingle Xu Lirong Xiang Dong Chen Weichao Zhuang Xunyuan Yin Zhao Li 119 3 0 13 Aug 2023