SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understanding

23 October 2023

Haoxiang Wang

Pavan Kumar Anasosalu Vasu

Papers citing "SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understanding"

29 / 29 papers shown

Title
Causal Prompt Calibration Guided Segment Anything Model for Open-Vocabulary Multi-Entity Segmentation Wenwen Qiang Jianqi Zhang Jingyao Wang Changwen Zheng VLM 37 0 0 10 May 2025
DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perception Junjie Wang Bin Chen Yulin Li Bin Kang Yulin Chen Zhuotao Tian VLM 38 0 0 07 May 2025
Show or Tell? A Benchmark To Evaluate Visual and Textual Prompts in Semantic Segmentation Gabriele Rosi Fabio Cermelli VLM 42 0 0 06 May 2025
RFMI: Estimating Mutual Information on Rectified Flow for Text-to-Image Alignment Chao Wang Giulio Franzese A. Finamore Pietro Michiardi 66 0 0 18 Mar 2025
Recovering Partially Corrupted Major Objects through Tri-modality Based Image Completion Yongle Zhang Yimin Liu Qiang Wu DiffM 35 0 0 10 Mar 2025
Bayesian Fields: Task-driven Open-Set Semantic Gaussian Splatting Dominic Maggio Luca Carlone 145 0 0 07 Mar 2025
MoFM: A Large-Scale Human Motion Foundation Model Mohammadreza Baharani Ghazal Alinezhad Noghre Armin Danesh Pazho Gabriel Maldonado Hamed Tabkhi AI4CE 150 1 0 08 Feb 2025
A Comprehensive Survey of Foundation Models in Medicine Wasif Khan Seowung Leem Kyle B. See Joshua K. Wong Shaoting Zhang R. Fang AI4CE LM&MA VLM 105 18 0 17 Jan 2025
Wearable Accelerometer Foundation Models for Health via Knowledge Distillation Salar Abbaspourazad Anshuman Mishra Joseph D. Futoma Andrew C. Miller Ian Shapiro 90 0 0 15 Dec 2024
Swiss Army Knife: Synergizing Biases in Knowledge from Vision Foundation Models for Multi-Task Learning Yuxiang Lu Shengcao Cao Yu-xiong Wang 49 1 0 18 Oct 2024
TG-LMM: Enhancing Medical Image Segmentation Accuracy through Text-Guided Large Multi-Modal Model Yihao Zhao Enhao Zhong Cuiyun Yuan Yang Li Man Zhao Chunxia Li Jun Hu Chenbin Liu VLM MedIm 45 0 0 05 Sep 2024
iSeg: An Iterative Refinement-based Framework for Training-free Segmentation Lin Sun Jiale Cao J. Xie F. Khan Yanwei Pang DiffM 43 1 0 05 Sep 2024
SAM-CP: Marrying SAM with Composable Prompts for Versatile Segmentation Pengfei Chen Lingxi Xie Xinyue Huo Xuehui Yu Xiaopeng Zhang Yingfei Sun Zhenjun Han Qi Tian VLM 68 1 0 23 Jul 2024
Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation Model Huan Ma Yan Zhu Changqing Zhang Peilin Zhao Baoyuan Wu Long-Kai Huang Qinghua Hu Bing Wu VLM 67 1 0 01 Mar 2024
Large Language Models and Foundation Models in Smart Agriculture: Basics, Opportunities, and Challenges Jiajia Li Mingle Xu Lirong Xiang Dong Chen Weichao Zhuang Xunyuan Yin Zhao Li 36 3 0 13 Aug 2023
What Do Self-Supervised Vision Transformers Learn? Namuk Park Wonjae Kim Byeongho Heo Taekyung Kim Sangdoo Yun SSL 67 76 1 01 May 2023
$π$ -Tuning: Transferring Multimodal Foundation Models with Optimal Multi-task Interpolation Chengyue Wu Teng Wang Yixiao Ge Zeyu Lu Rui-Zhi Zhou Ying Shan Ping Luo MoMe 80 35 0 27 Apr 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 270 4,244 0 30 Jan 2023
APE: Aligning Pretrained Encoders to Quickly Learn Aligned Multimodal Representations Elan Rosenfeld Preetum Nakkiran Hadi Pouransari Oncel Tuzel Fartash Faghri 68 6 0 08 Oct 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 252 314 0 11 Sep 2022
GroupViT: Semantic Segmentation Emerges from Text Supervision Jiarui Xu Shalini De Mello Sifei Liu Wonmin Byeon Thomas Breuel Jan Kautz Xinyu Wang ViT VLM 189 499 0 22 Feb 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 392 4,137 0 28 Jan 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 305 7,443 0 11 Nov 2021
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 213 1,657 0 15 Oct 2021
Efficiently Identifying Task Groupings for Multi-Task Learning Christopher Fifty Ehsan Amid Zhe Zhao Tianhe Yu Rohan Anil Chelsea Finn 206 238 1 10 Sep 2021
ImageNet-21K Pretraining for the Masses T. Ridnik Emanuel Ben-Baruch Asaf Noy Lihi Zelnik-Manor SSeg VLM CLIP 181 687 0 22 Apr 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 278 1,082 0 17 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 298 3,700 0 11 Feb 2021
Semantic Understanding of Scenes through the ADE20K Dataset Bolei Zhou Hang Zhao Xavier Puig Tete Xiao Sanja Fidler Adela Barriuso Antonio Torralba SSeg 253 1,828 0 18 Aug 2016