BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

30 January 2023

Silvio Savarese

Papers citing "BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models"

50 / 726 papers shown

Title
Controllable Image Colorization with Instance-aware Texts and Masks Yanru An Ling Gui Qiang Hu Chunlei Cai Tianxiao Ye Xiaoyun Zhang Yanfeng Wang DiffM 34 0 0 13 May 2025
OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning Zhaochen Su Linjie Li Mingyang Song Yunzhuo Hao Zhengyuan Yang ... Guanjie Chen Jiawei Gu Juntao Li Xiaoye Qu Yu Cheng OffRL LRM 31 0 0 13 May 2025
FauForensics: Boosting Audio-Visual Deepfake Detection with Facial Action Units Jian Wang Baoyuan Wu Li Liu Qingshan Liu AAML 24 0 0 13 May 2025
CLTP: Contrastive Language-Tactile Pre-training for 3D Contact Geometry Understanding Wenxuan Ma Xiaoge Cao Y. Zhang Chaofan Zhang Shaobo Yang Peng Hao Bin Fang Yinghao Cai Shaowei Cui Shuo Wang 33 0 0 13 May 2025
Visually Guided Decoding: Gradient-Free Hard Prompt Inversion with Language Models Donghoon Kim Minji Bae Kyuhong Shim B. Shim 36 0 0 13 May 2025
Extending Large Vision-Language Model for Diverse Interactive Tasks in Autonomous Driving Zongchuang Zhao Haoyu Fu Dingkang Liang Xin Zhou Dingyuan Zhang Hongwei Xie Bing Wang Xiang Bai MLLM VLM 49 0 0 13 May 2025
Visually Interpretable Subtask Reasoning for Visual Question Answering Yu Cheng A. Goel Hakan Bilen LRM 29 0 0 12 May 2025
DriveSOTIF: Advancing Perception SOTIF Through Multimodal Large Language Models Shucheng Huang Freda Shi Chen Sun Jiaming Zhong Minghao Ning Yufeng Yang Yukun Lu Hong Wang A. Khajepour 26 0 0 11 May 2025
Visual Instruction Tuning with Chain of Region-of-Interest Yixin Chen Shuai Zhang Boran Han Bernie Wang 26 0 0 11 May 2025
METOR: A Unified Framework for Mutual Enhancement of Objects and Relationships in Open-vocabulary Video Visual Relationship Detection Yongqi Wang Xinxiao Wu Shuo Yang ObjD 26 0 0 10 May 2025
Describe Anything in Medical Images Xi Xiao Yunbei Zhang Thanh-Huy Nguyen Ba Thinh Lam Janet Wang ... Xingjian Li X. U. Wang Hao Xu Tianming Liu Min Xu MedIm VLM 46 0 0 09 May 2025
Fine-Tuning Video-Text Contrastive Model for Primate Behavior Retrieval from Unlabeled Raw Videos Giulio Cesare Mastrocinque Santo Patrícia Izar Irene Delval Victor de Napole Gregolin Nina S. T. Hirata VGen 40 0 0 08 May 2025
X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIP Hanxun Huang Sarah Monazam Erfani Yige Li Xingjun Ma James Bailey AAML 44 0 0 08 May 2025
FG-CLIP: Fine-Grained Visual and Textual Alignment Chunyu Xie Bin Wang Fanjing Kong Jincheng Li Dawei Liang Gengshen Zhang Dawei Leng Yuhui Yin CLIP VLM 46 0 0 08 May 2025
PlaceIt3D: Language-Guided Object Placement in Real 3D Scenes Ahmed Abdelreheem Filippo Aleotti Jamie Watson Z. Qureshi Abdelrahman Eldesokey Peter Wonka Gabriel J. Brostow Sara Vicente Guillermo Garcia-Hernando DiffM 59 0 0 08 May 2025
StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistant Haibo Wang Bo Feng Zhengfeng Lai Mingze Xu Shiyu Li Weifeng Ge Afshin Dehghan Meng Cao Ping-Chia Huang OffRL 51 0 0 08 May 2025
Object-Shot Enhanced Grounding Network for Egocentric Video Yisen Feng Haoyu Zhang Meng Liu Weili Guan Liqiang Nie 38 0 0 07 May 2025
Robust Fairness Vision-Language Learning for Medical Image Analysis Sparsh Bansal Mingyang Wu Xin Wang S. Hu VLM 50 0 0 06 May 2025
DyGEnc: Encoding a Sequence of Textual Scene Graphs to Reason and Answer Questions in Dynamic Scenes S. Linok Vadim Semenov Anastasia Trunova Oleg Bulichev Dmitry A. Yudin 52 0 0 06 May 2025
RAVU: Retrieval Augmented Video Understanding with Compositional Reasoning over Graph Sameer Malik Moyuru Yamada Ayush Singh Dishank Aggarwal 138 0 0 06 May 2025
Reducing Annotation Burden in Physical Activity Research Using Vision-Language Models Abram Schonfeldt Benjamin Maylor Xiaofang Chen Ronald Clark Aiden Doherty 68 0 0 06 May 2025
MCCD: Multi-Agent Collaboration-based Compositional Diffusion for Complex Text-to-Image Generation Mingcheng Li Xiaolu Hou Ziyang Liu Dingkang Yang Ziyun Qian Jiawei Chen Jinjie Wei Y. Jiang Qingyao Xu L. Zhang DiffM 129 0 0 05 May 2025
VGLD: Visually-Guided Linguistic Disambiguation for Monocular Depth Scale Recovery Bojin Wu Jing Chen MDE 46 0 0 05 May 2025
Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunities X. Zhang Jintao Guo Shanshan Zhao Minghao Fu Lunhao Duan Guo-Hua Wang Qing-Guo Chen Zhao Xu Weihua Luo Kaifu Zhang DiffM 74 0 0 05 May 2025
Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generation Lu Ling C. Lin Tsung-Yi Lin Yifan Ding Y. Zeng Yichen Sheng Yunhao Ge Ming-Yu Liu Aniket Bera Zhaoshuo Li VGen 3DV 56 0 0 05 May 2025
Uncertainty-Weighted Image-Event Multimodal Fusion for Video Anomaly Detection SungHeon Jeong Jihong Park Mohsen Imani 59 0 0 05 May 2025
HapticVLM: VLM-Driven Texture Recognition Aimed at Intelligent Haptic Interaction Muhammad Haris Khan Miguel Altamirano Cabrera Dmitrii Iarchuk Yara Mahmoud Daria Trinitatova Issatay Tokmurziyev Dzmitry Tsetserukou VLM 48 0 0 05 May 2025
Interleave-VLA: Enhancing Robot Manipulation with Interleaved Image-Text Instructions Cunxin Fan Xiaosong Jia Yihang Sun Yixiao Wang Jianglan Wei ... Xiangyu Zhao M. Tomizuka Xue Yang Junchi Yan Mingyu Ding LM&Ro VLM 64 2 0 04 May 2025
Compositional Image-Text Matching and Retrieval by Grounding Entities Madhukar Reddy Vongala Saurabh Srivastava Jana Kosecka CLIP CoGe VLM 36 0 0 04 May 2025
RAGAR: Retrieval Augment Personalized Image Generation Guided by Recommendation Run Ling W. Wang Yuting Liu G. Guo Linying Jiang Xingwei Wang DiffM 54 0 0 03 May 2025
RESAnything: Attribute Prompting for Arbitrary Referring Segmentation Ruiqi Wang Hao Zhang VLM 56 0 0 03 May 2025
Enhancing the Learning Experience: Using Vision-Language Models to Generate Questions for Educational Videos Markos Stamatakis Joshua Berger Christian Wartena Ralph Ewerth Anett Hoppe AI4Ed 41 0 0 03 May 2025
Vision and Intention Boost Large Language Model in Long-Term Action Anticipation Congqi Cao Lanshu Hu Yating Yu Y. Zhang VLM 135 0 0 03 May 2025
Efficient Vocabulary-Free Fine-Grained Visual Recognition in the Age of Multimodal LLMs Hari Chandana Kuchibhotla Sai Srinivas Kancheti Abbavaram Gowtham Reddy Vineeth N. Balasubramanian 45 0 0 02 May 2025
Scalability Matters: Overcoming Challenges in InstructGLM with Similarity-Degree-Based Sampling Hyun Lee Chris Yi Maminur Islam B.D.S. Aritra 33 0 0 02 May 2025
JointDiT: Enhancing RGB-Depth Joint Modeling with Diffusion Transformers Kwon Byung-Ki Qi Dai Lee Hyoseok Chong Luo Tae-Hyun Oh 71 0 0 01 May 2025
SpatialLLM: A Compound 3D-Informed Design towards Spatially-Intelligent Large Multimodal Models Wufei Ma Luoxin Ye Nessa McWeeney Celso M de Melo A. Yuille Jieneng Chen LRM 65 1 0 01 May 2025
Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation Vaidehi Patil Yi-Lin Sung Peter Hase Jie Peng Tianlong Chen Mohit Bansal AAML MU 83 3 0 01 May 2025
Improving Routing in Sparse Mixture of Experts with Graph of Tokens Tam Minh Nguyen Ngoc N. Tran Khai Nguyen Richard G. Baraniuk MoE 59 0 0 01 May 2025
RoboGround: Robotic Manipulation with Grounded Vision-Language Priors Haifeng Huang Xinyi Chen Y. Chen H. Li Xiaoshen Han Z. Wang Tai Wang Jiangmiao Pang Zhou Zhao LM&Ro 80 0 0 30 Apr 2025
An Evaluation of a Visual Question Answering Strategy for Zero-shot Facial Expression Recognition in Still Images Modesto Castrillón-Santana Oliverio J. Santana David Freire-Obregón Daniel Hernández-Sosa J. Lorenzo-Navarro 52 0 0 30 Apr 2025
AGHI-QA: A Subjective-Aligned Dataset and Metric for AI-Generated Human Images Yunhao Li Sijing Wu Wei Sun Zhichao Zhang Yucheng Zhu Zicheng Zhang Huiyu Duan Xiongkuo Min Guangtao Zhai EGVM 90 0 0 30 Apr 2025
Rethinking Visual Layer Selection in Multimodal LLMs H. Chen Junyan Lin Xinhao Chen Yue Fan Xin Jin Hui Su Jianfeng Dong Jinlan Fu Xiaoyu Shen VLM 95 0 0 30 Apr 2025
Eye2Eye: A Simple Approach for Monocular-to-Stereo Video Synthesis Michal Geyer Omer Tov Linyi Jin Richard Tucker Inbar Mosseri Tali Dekel Noah Snavely DiffM VGen 100 0 0 30 Apr 2025
Synergy-CLIP: Extending CLIP with Multi-modal Integration for Robust Representation Learning Sangyeon Cho Jangyeong Jeon Mingi Kim Junyeong Kim CLIP VLM 76 0 0 30 Apr 2025
Multimodal Large Language Models for Medicine: A Comprehensive Survey Jiarui Ye Hao Tang LM&MA 89 0 0 29 Apr 2025
X-Fusion: Introducing New Modality to Frozen Large Language Models Sicheng Mo Thao Nguyen Xun Huang Siddharth Srinivasan Iyer Yijun Li ... Eli Shechtman Krishna Kumar Singh Yong Jae Lee Bolei Zhou Yuheng Li 77 0 0 29 Apr 2025
MemeBLIP2: A novel lightweight multimodal system to detect harmful memes Jiaqi Liu Ran Tong Aowei Shen Shuzheng Li Changlin Yang Lisha Xu VLM 77 0 0 29 Apr 2025
CMT: A Cascade MAR with Topology Predictor for Multimodal Conditional CAD Generation Jianyu Wu Yizhou Wang Xiangyu Yue Xinzhu Ma J. Guo Dongzhan Zhou Wanli Ouyang Shixiang Tang 66 0 0 29 Apr 2025
EcoWikiRS: Learning Ecological Representation of Satellite Images from Weak Supervision with Species Observations and Wikipedia Valerie Zermatten J. Castillo-Navarro Pallavi Jain D. Tuia Diego Marcos 62 0 0 28 Apr 2025