Scaling Language-Image Pre-training via Masking

1 December 2022

Yanghao Li

Haoqi Fan

Ronghang Hu

Christoph Feichtenhofer

Papers citing "Scaling Language-Image Pre-training via Masking"

50 / 249 papers shown

Title
GeoMM: On Geodesic Perspective for Multi-modal Learning Shibin Mei Hang Wang Bingbing Ni 22 0 0 16 May 2025
$Simple Semi-supervised Knowledge Distillation from Vision-Language Models via $\mathbf{\texttt{D}}$ual-$\mathbf{\texttt{H}}$ead $\mathbf{\texttt{O}}$ptimization$ Simple Semi-supervised Knowledge Distillation from Vision-Language Models via $\mathbf{\texttt{D}}$ ual- $\mathbf{\texttt{H}}$ ead $\mathbf{\texttt{O}}$ ptimization Seongjae Kang Dong Bok Lee Hyungjoon Jang Sung Ju Hwang VLM 58 0 0 12 May 2025
X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIP Hanxun Huang Sarah Monazam Erfani Yige Li Xingjun Ma James Bailey AAML 53 0 0 08 May 2025
DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perception Junjie Wang Bin Chen Yulin Li Bin Kang Yulin Chen Zhuotao Tian VLM 38 0 0 07 May 2025
OpenVision: A Fully-Open, Cost-Effective Family of Advanced Vision Encoders for Multimodal Learning Xianhang Li Y. Liu Haoqin Tu Hongru Zhu Cihang Xie VLM 204 0 0 07 May 2025
Decoupled Global-Local Alignment for Improving Compositional Understanding Xiaoxing Hu Kaicheng Yang Jianmin Wang Haoran Xu Ziyong Feng Yansen Wang VLM 165 0 0 23 Apr 2025
Perception Encoder: The best visual embeddings are not at the output of the network Daniel Bolya Po-Yao (Bernie) Huang Peize Sun Jang Hyun Cho Andrea Madotto ... Shiyu Dong Nikhila Ravi Daniel Li Piotr Dollár Christoph Feichtenhofer ObjD VOS 103 2 0 17 Apr 2025
Low-hallucination Synthetic Captions for Large-Scale Vision-Language Model Pre-training Xinsong Zhang Yarong Zeng Xinting Huang Hu Hu Runquan Xie Han Hu Zhanhui Kang MLLM VLM 55 0 0 17 Apr 2025
Towards Cardiac MRI Foundation Models: Comprehensive Visual-Tabular Representations for Whole-Heart Assessment and Beyond Yundi Zhang Paul Hager Che Liu Suprosanna Shit Chong Chen Daniel Rueckert Jiazhen Pan 46 0 0 17 Apr 2025
The Scalability of Simplicity: Empirical Analysis of Vision-Language Learning with a Single Transformer Weixian Lei Jiacong Wang Haochen Wang Xuelong Li Jun Hao Liew Jiashi Feng Zilong Huang 30 2 0 14 Apr 2025
Refining CLIP's Spatial Awareness: A Visual-Centric Perspective Congpei Qiu Yanhao Wu Wei Ke Xiuxiu Bai Tong Zhang VLM 52 0 0 03 Apr 2025
Seeing What Matters: Empowering CLIP with Patch Generation-to-Selection Gensheng Pei Tao Chen Yujia Wang Xinhao Cai Xiangbo Shu Tianfei Zhou Yazhou Yao VLM 55 1 0 21 Mar 2025
EgoDTM: Towards 3D-Aware Egocentric Video-Language Pretraining Boshen Xu Yuting Mei Xinbi Liu Sipeng Zheng Qin Jin VLM MDE 70 0 0 19 Mar 2025
Dynamic Relation Inference via Verb Embeddings Omri Suissa Muhiim Ali Ariana Azarbal Hui Shen Shekhar Pradhan 46 0 0 17 Mar 2025
Robustness Tokens: Towards Adversarial Robustness of Transformers Brian Pulfer Yury Belousov S. Voloshynovskiy AAML 45 0 0 13 Mar 2025
Task-Agnostic Attacks Against Vision Foundation Models Brian Pulfer Yury Belousov Vitaliy Kinakh Teddy Furon S. Voloshynovskiy AAML 77 0 0 05 Mar 2025
ELIP: Enhanced Visual-Language Foundation Models for Image Retrieval Guanqi Zhan Yuanpei Liu Kai Han Weidi Xie Andrew Zisserman VLM 219 0 0 21 Feb 2025
Myna: Masking-Based Contrastive Learning of Musical Representations Ori Yonay Tracy Hammond Tianbao Yang AAML 61 0 0 20 Feb 2025
Black Sheep in the Herd: Playing with Spuriously Correlated Attributes for Vision-Language Recognition Xinyu Tian Shu Zou Zhaoyuan Yang Mengqi He Jing Zhang VLM 48 0 0 19 Feb 2025
ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classification Jiangbo Shi Chen Li Tieliang Gong Yefeng Zheng Huazhu Fu VLM 65 7 0 12 Feb 2025
A Comprehensive Survey of Foundation Models in Medicine Wasif Khan Seowung Leem Kyle B. See Joshua K. Wong Shaoting Zhang R. Fang AI4CE LM&MA VLM 105 19 0 17 Jan 2025
VideoAuteur: Towards Long Narrative Video Generation Junfei Xiao Feng Cheng Lu Qi Liangke Gui Jiepeng Cen Zhibei Ma Alan Yuille Lu Jiang VGen 58 2 0 10 Jan 2025
BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs Sheng Zhang Yanbo Xu Naoto Usuyama Hanwen Xu J. Bagga ... Carlo Bifulco M. Lungren Tristan Naumann Sheng Wang Hoifung Poon LM&MA MedIm 154 208 0 10 Jan 2025
A Simple Recipe for Contrastively Pre-training Video-First Encoders Beyond 16 Frames Pinelopi Papalampidi Skanda Koppula Shreya Pathak Justin T Chiu Joseph Heyward Viorica Patraucean Jiajun Shen Antoine Miech Andrew Zisserman Aida Nematzdeh VLM 69 24 0 31 Dec 2024
HyperCLIP: Adapting Vision-Language models with Hypernetworks Victor Akinwande Mohammad Sadegh Norouzzadeh Devin Willmott Anna Bair Madan Ravi Ganesh J. Zico Kolter CLIP VLM 95 0 0 21 Dec 2024
I0T: Embedding Standardization Method Towards Zero Modality Gap Na Min An Eunki Kim James Thorne Hyunjung Shim VLM 72 1 0 18 Dec 2024
CLIP-PING: Boosting Lightweight Vision-Language Models with Proximus Intrinsic Neighbors Guidance Chu Myaet Thwal Ye Lin Tun Minh N. H. Nguyen Eui-nam Huh Choong Seon Hong VLM 74 0 0 05 Dec 2024
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training Sanghwan Kim Rui Xiao Mariana-Iuliana Georgescu Stephan Alaniz Zeynep Akata VLM 93 2 0 02 Dec 2024
Sparse Attention Vectors: Generative Multimodal Model Features Are Discriminative Vision-Language Classifiers Chancharik Mitra Brandon Huang Tianning Chai Zhiqiu Lin Assaf Arbelle Rogerio Feris Leonid Karlinsky Trevor Darrell Deva Ramanan Roei Herzig VLM 134 4 0 28 Nov 2024
RankByGene: Gene-Guided Histopathology Representation Learning Through Cross-Modal Ranking Consistency Wentao Huang Meilong Xu Xiaoling Hu Shahira Abousamra Aniruddha Ganguly ... Prateek Prasanna Tahsin M. Kurc Joel H. Saltz Michael L. Miller Chong Chen 81 0 0 22 Nov 2024
On the Surprising Effectiveness of Attention Transfer for Vision Transformers Alexander C. Li Yuandong Tian Bin Chen Deepak Pathak Xinlei Chen 43 0 0 14 Nov 2024
CROPS: A Deployable Crop Management System Over All Possible State Availabilities Jing Wu Zhixin Lai Shengjie Liu Suiyao Chen Ran Tao Pan Zhao Chuyuan Tao Yikun Cheng N. Hovakimyan OffRL 53 0 0 09 Nov 2024
Don't Look Twice: Faster Video Transformers with Run-Length Tokenization Rohan Choudhury Guanglei Zhu Sihan Liu Koichiro Niinuma Kris M. Kitani László A. Jeni 31 11 0 07 Nov 2024
Classification Done Right for Vision-Language Pre-Training Zilong Huang Qinghao Ye Bingyi Kang Jiashi Feng Haoqi Fan CLIP VLM 50 2 0 05 Nov 2024
Revisiting MAE pre-training for 3D medical image segmentation Tassilo Wald Constantin Ulrich Stanislav Lukyanenko Andrei Goncharov Alberto Paderno Leander Maerkisch Paul F. Jäger Paul F. Jäger Klaus Maier-Hein 50 2 0 30 Oct 2024
Accelerating Augmentation Invariance Pretraining Jinhong Lin Cheng-En Wu Yibing Wei Pedro Morgado ViT 33 1 0 27 Oct 2024
Probabilistic Language-Image Pre-Training Sanghyuk Chun Wonjae Kim Song Park Sangdoo Yun MLLM VLM CLIP 191 4 2 24 Oct 2024
TIPS: Text-Image Pretraining with Spatial awareness Kevis-Kokitsi Maninis Kaifeng Chen Soham Ghosh Arjun Karpur Koert Chen ... Jan Dlabal Dan Gnanapragasam Mojtaba Seyedhosseini Howard Zhou Andre Araujo VLM 37 3 0 21 Oct 2024
CtrlSynth: Controllable Image Text Synthesis for Data-Efficient Multimodal Learning Qingqing Cao Mahyar Najibi Sachin Mehta CLIP DiffM 35 1 0 15 Oct 2024
DRACO: A Denoising-Reconstruction Autoencoder for Cryo-EM Yingjun Shen Haizhao Dai Qihe Chen Yan Zeng Jiakai Zhang Yuan Pei Jingyi Yu 26 0 0 15 Oct 2024
Locality Alignment Improves Vision-Language Models Ian Covert Tony Sun James Zou Tatsunori Hashimoto VLM 72 4 0 14 Oct 2024
OneRef: Unified One-tower Expression Grounding and Segmentation with Mask Referring Modeling Linhui Xiao Xiaoshan Yang Fang Peng Yaowei Wang Changsheng Xu ObjD 38 5 0 10 Oct 2024
Enhancing Vision-Language Model Pre-training with Image-text Pair Pruning Based on Word Frequency Mingliang Liang Martha Larson VLM CLIP 26 0 0 09 Oct 2024
LoTLIP: Improving Language-Image Pre-training for Long Text Understanding Wei Wu Kecheng Zheng Shuailei Ma Fan Lu Yuxin Guo Yifei Zhang Wei Chen Qingpei Guo Yujun Shen Zheng-Jun Zha VLM 35 9 0 07 Oct 2024
Adaptive Masking Enhances Visual Grounding Sen Jia Lei Li 31 0 0 04 Oct 2024
SegEarth-OV: Towards Training-Free Open-Vocabulary Segmentation for Remote Sensing Images Kaiyu Li Ruixun Liu Xiangyong Cao Deyu Meng Zhi Wang Deyu Meng Zhi Wang 41 3 0 02 Oct 2024
Harnessing Shared Relations via Multimodal Mixup Contrastive Learning for Multimodal Classification Raja Kumar Raghav Singhal Pranamya Kulkarni Deval Mehta Kshitij Jadhav 26 0 0 26 Sep 2024
Stochastic Subsampling With Average Pooling Bum Jun Kim Sang Woo Kim 26 0 0 25 Sep 2024
BrainDreamer: Reasoning-Coherent and Controllable Image Generation from EEG Brain Signals via Language Guidance Ling Wang Chen Wu Lin Wang DiffM 36 0 0 21 Sep 2024
DetailCLIP: Detail-Oriented CLIP for Fine-Grained Tasks Amin Karimi Monsefi Kishore Prakash Sailaja Ali Alilooee Ser-Nam Lim R. Ramnath VLM 37 6 0 10 Sep 2024