BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

30 January 2023

Silvio Savarese

Papers citing "BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models"

50 / 822 papers shown

Title
Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation Kun Yuan V. Srivastav Nassir Navab N. Padoy 46 7 0 30 Sep 2024
UniAff: A Unified Representation of Affordances for Tool Usage and Articulation with Vision-Language Models Qiaojun Yu Siyuan Huang Xibin Yuan Zhengkai Jiang Ce Hao ... Junbo Wang Liu Liu Hongsheng Li Peng Gao Cewu Lu 75 3 0 30 Sep 2024
SSR: Alignment-Aware Modality Connector for Speech Language Models Weiting Tan Hirofumi Inaguma Ning Dong Paden Tomasello Xutai Ma 32 3 0 30 Sep 2024
Resolving Multi-Condition Confusion for Finetuning-Free Personalized Image Generation Qihan Huang Siming Fu Jinlong Liu Hao Jiang Yipeng Yu Jie Song 39 5 0 26 Sep 2024
MIO: A Foundation Model on Multimodal Tokens Zekun Wang King Zhu Chunpu Xu Wangchunshu Zhou Jiaheng Liu ... Yuanxing Zhang Ge Zhang Ke Xu Jie Fu Wenhao Huang MLLM AuLLM 60 11 0 26 Sep 2024
Robotic Environmental State Recognition with Pre-Trained Vision-Language Models and Black-Box Optimization Kento Kawaharazuka Yoshiki Obinata Naoaki Kanazawa Kei Okada Masayuki Inaba LM&Ro 30 0 0 26 Sep 2024
Multi-View and Multi-Scale Alignment for Contrastive Language-Image Pre-training in Mammography Yuexi Du John Onofrey Nicha Dvornek VLM 53 1 0 26 Sep 2024
SECURE: Semantics-aware Embodied Conversation under Unawareness for Lifelong Robot Learning Rimvydas Rubavicius Peter David Fagan A. Lascarides Subramanian Ramamoorthy LM&Ro 169 0 0 26 Sep 2024
LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Chenming Zhu Tai Wang Wenwei Zhang Jiangmiao Pang Xihui Liu 134 32 0 26 Sep 2024
EventHallusion: Diagnosing Event Hallucinations in Video LLMs Jiacheng Zhang Yang Jiao Shaoxiang Chen Jingjing Chen Zhiyu Tan Hao Li Jingjing Chen MLLM 61 18 0 25 Sep 2024
Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation Siyin Wang Wenyi Yu Yudong Yang Changli Tang Yixuan Li ... Jun Zhang Guangzhi Sun Lu Lu Yuxuan Wang Chao Zhang AuLLM LM&MA 67 5 0 25 Sep 2024
OmniBench: Towards The Future of Universal Omni-Language Models Yizhi Li Ge Zhang Yinghao Ma Ruibin Yuan Kang Zhu ... Zhaoxiang Zhang Zachary Liu Emmanouil Benetos Wenhao Huang Chenghua Lin LRM 54 11 0 23 Sep 2024
ReVLA: Reverting Visual Domain Limitation of Robotic Foundation Models Sombit Dey Jan-Nico Zaech Nikolay Nikolov Luc Van Gool Danda Pani Paudel MoMe VLM 59 4 0 23 Sep 2024
What Are They Doing? Joint Audio-Speech Co-Reasoning Yingzhi Wang Pooneh Mousavi Artem Ploujnikov Mirco Ravanelli AuLLM 46 0 0 22 Sep 2024
Dormant: Defending against Pose-driven Human Image Animation Jiachen Zhou Mingsi Wang Tianlin Li Guozhu Meng Kai Chen 67 3 0 22 Sep 2024
JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images Zhecan Wang Junzhang Liu Chia-Wei Tang Hani Alomari Anushka Sivakumar ... Haoxuan You A. Ishmam Kai-Wei Chang Shih-Fu Chang Chris Thomas CoGe VLM 66 2 0 19 Sep 2024
End-to-end Open-vocabulary Video Visual Relationship Detection using Multi-modal Prompting Yongqi Wang Xinxiao Wu Shuo Yang Jiebo Luo 170 1 0 19 Sep 2024
TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation Junjie Wen Yichen Zhu Jinming Li Minjie Zhu Kun Wu ... Ran Cheng Chaomin Shen Yaxin Peng Feifei Feng Jian Tang LM&Ro 74 48 0 19 Sep 2024
Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolution Zuyan Liu Yuhao Dong Ziwei Liu Winston Hu Jiwen Lu Yongming Rao ObjD 86 55 0 19 Sep 2024
Large Language Models are Strong Audio-Visual Speech Recognition Learners Umberto Cappellazzo Minsu Kim Honglie Chen Pingchuan Ma Stavros Petridis Daniele Falavigna Alessio Brutti Maja Pantic 36 9 0 18 Sep 2024
Generative Semantic Communication via Textual Prompts: Latency Performance Tradeoffs Mengmeng Ren Li Qiao Long Yang Zhen Gao Jian Chen Mahdi Boloursaz Mashhadi Pei Xiao Rahim Tafazolli Mehdi Bennis VLM 101 5 0 15 Sep 2024
One missing piece in Vision and Language: A Survey on Comics Understanding Emanuele Vivoli Andrey Barsky Mohamed Ali Souibgui Artemis LLabres Marco Bertini Dimosthenis Karatzas 42 3 0 14 Sep 2024
What to align in multimodal contrastive learning? Benoit Dufumier J. Castillo-Navarro D. Tuia Jean-Philippe Thiran 29 3 0 11 Sep 2024
EDADepth: Enhanced Data Augmentation for Monocular Depth Estimation Nischal Khanal Shivanand Venkanna Sheshappanavar MDE 42 0 0 10 Sep 2024
Revisiting Prompt Pretraining of Vision-Language Models Zhenyuan Chen Lingfeng Yang Shuo Chen Zhaowei Chen Jiajun Liang Xiang Li MLLM VPVLM VLM 43 1 0 10 Sep 2024
Referring Expression Generation in Visually Grounded Dialogue with Discourse-aware Comprehension Guiding Bram Willemsen Gabriel Skantze 30 0 0 09 Sep 2024
Enhanced Generative Data Augmentation for Semantic Segmentation via Stronger Guidance Quang-Huy Che Duc-Tri Le Vinh-Tiep Nguyen D. Lam Vinh-Tiep Nguyen DiffM 44 1 0 09 Sep 2024
Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understanding Yunze Man Shuhong Zheng Zhipeng Bao M. Hebert Liang-Yan Gui Yu-xiong Wang 78 15 0 05 Sep 2024
The Compressor-Retriever Architecture for Language Model OS Yuan Yang Siheng Xiong Ehsan Shareghi Faramarz Fekri RALM KELM 32 1 0 02 Sep 2024
SAM4MLLM: Enhance Multi-Modal Large Language Model for Referring Expression Segmentation Yi-Chia Chen Wei-Hua Li Cheng Sun Yu-Chiang Frank Wang Chu-Song Chen VLM 42 11 0 01 Sep 2024
StimuVAR: Spatiotemporal Stimuli-aware Video Affective Reasoning with Multimodal Large Language Models Y. Guo Faizan Siddiqui Yang Zhao Rama Chellappa Shao-Yuan Lo LRM 44 2 0 31 Aug 2024
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders Min Shi Fuxiao Liu Shihao Wang Shijia Liao Subhashree Radhakrishnan ... Andrew Tao Andrew Tao Zhiding Yu Guilin Liu Guilin Liu MLLM 33 53 0 28 Aug 2024
FungiTastic: A multi-modal dataset and benchmark for image categorization Lukás Picek Klara Janouskova Milan Šulc Jirí Matas 77 1 0 24 Aug 2024
MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans? Yi-Fan Zhang Huanyu Zhang Haochen Tian Chaoyou Fu Shuangqing Zhang ... Qingsong Wen Zhang Zhang Liwen Wang Rong Jin Tieniu Tan OffRL 69 36 0 23 Aug 2024
Vocabulary-Free 3D Instance Segmentation with Vision and Language Assistant Guofeng Mei Luigi Riz Yiming Wang Fabio Poiesi ISeg VLM 69 3 0 20 Aug 2024
CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Driving Hidehisa Arai Keita Miwa Kento Sasaki Yu Yamaguchi Kohei Watanabe Shunsuke Aoki Issei Yamamoto 54 9 0 19 Aug 2024
Beyond the Hype: A dispassionate look at vision-language models in medical scenario Yang Nan Huichi Zhou Xiaodan Xing Guang Yang 54 3 0 16 Aug 2024
Visual Agents as Fast and Slow Thinkers Guangyan Sun Mingyu Jin Zhenting Wang Cheng-Long Wang Siqi Ma Qifan Wang Ying Nian Wu Ying Nian Wu Dongfang Liu Dongfang Liu LLMAG LRM 79 13 0 16 Aug 2024
End-to-end Semantic-centric Video-based Multimodal Affective Computing Ronghao Lin Ying Zeng Sijie Mai Haifeng Hu VGen 48 0 0 14 Aug 2024
Connecting Dreams with Visual Brainstorming Instruction Yasheng Sun Bohan Li Mingchen Zhuge Deng-Ping Fan Salman Khan Fahad Shahbaz Khan Hideki Koike DiffM 42 0 0 14 Aug 2024
CROME: Cross-Modal Adapters for Efficient Multimodal LLM Sayna Ebrahimi Sercan Ö. Arik Tejas Nama Tomas Pfister 44 1 0 13 Aug 2024
D2Styler: Advancing Arbitrary Style Transfer with Discrete Diffusion Methods Onkar Susladkar Gayatri S Deshmukh Sparsh Mittal Parth Shastri DiffM 44 3 0 07 Aug 2024
AgentsCoMerge: Large Language Model Empowered Collaborative Decision Making for Ramp Merging Senkang Hu Zhengru Fang Zihan Fang Yiqin Deng Xianhao Chen Yuguang Fang Sam Kwong 54 14 0 07 Aug 2024
Attacks and Defenses for Generative Diffusion Models: A Comprehensive Survey V. T. Truong Luan Ba Dang Long Bao Le DiffM MedIm 56 16 0 06 Aug 2024
AdvQDet: Detecting Query-Based Adversarial Attacks with Adversarial Contrastive Prompt Tuning Xin Wang Kai-xiang Chen Xingjun Ma Zhineng Chen Jingjing Chen Yu-Gang Jiang AAML 46 4 0 04 Aug 2024
Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models Fushuo Huo Wenchao Xu Zhong Zhang Yining Qi Zhicheng Chen Peilin Zhao VLM MLLM 66 19 0 04 Aug 2024
WAS: Dataset and Methods for Artistic Text Segmentation Xudong Xie Yuzhe Li Yang Liu Zhifei Zhang Zhaowen Wang Wei Xiong Xiang Bai DiffM 52 2 0 31 Jul 2024
Learning Video Context as Interleaved Multimodal Sequences S. Shao Pengchuan Zhang Y. Li Xide Xia A. Meso Ziteng Gao Jinheng Xie N. Holliman Mike Zheng Shou 49 5 0 31 Jul 2024
ControlMLLM: Training-Free Visual Prompt Learning for Multimodal Large Language Models Ming-Kuan Wu Xinyue Cai Jiayi Ji Jiale Li Oucheng Huang Gen Luo Hao Fei Xiaoshuai Sun Rongrong Ji MLLM 54 7 0 31 Jul 2024
Prompting Medical Large Vision-Language Models to Diagnose Pathologies by Visual Question Answering Danfeng Guo Sumitaka Honji LRM 79 0 0 31 Jul 2024