Scaling Language-Image Pre-training via Masking

1 December 2022

Yanghao Li

Haoqi Fan

Ronghang Hu

Christoph Feichtenhofer

Papers citing "Scaling Language-Image Pre-training via Masking"

50 / 249 papers shown

Title
Expediting Contrastive Language-Image Pretraining via Self-distilled Encoders Bumsoo Kim Jinhyung Kim Yeonsik Jo S. Kim VLM 36 4 0 19 Dec 2023
Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model Shraman Pramanick Guangxing Han Rui Hou Sayan Nag Ser-Nam Lim Nicolas Ballas Qifan Wang Rama Chellappa Amjad Almahairi VLM MLLM 48 29 0 19 Dec 2023
Tokenize Anything via Prompting Ting Pan Lulu Tang Xinlong Wang Shiguang Shan VLM 31 22 0 14 Dec 2023
A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions Jack Urbanek Florian Bordes Pietro Astolfi Mary Williamson Vasu Sharma Adriana Romero Soriano CLIP 3DV 41 42 0 14 Dec 2023
Foundation Models in Robotics: Applications, Challenges, and the Future Roya Firoozi Johnathan Tucker Stephen Tian Anirudha Majumdar Jiankai Sun ... Brian Ichter Danny Driess Jiajun Wu Cewu Lu Mac Schwager LM&Ro AI4CE LRM VLM 37 143 0 13 Dec 2023
CLIP as RNN: Segment Countless Visual Concepts without Training Endeavor Shuyang Sun Runjia Li Philip Torr Xiuye Gu Siyang Li VLM CLIP 39 32 0 12 Dec 2023
Fast Training of Diffusion Transformer with Extreme Masking for 3D Point Clouds Generation Shentong Mo Enze Xie Yue Wu Junsong Chen Matthias Nießner Zhenguo Li 26 5 0 12 Dec 2023
Scaling Laws of Synthetic Images for Model Training ... for Now Lijie Fan Kaifeng Chen Dilip Krishnan Dina Katabi Phillip Isola Yonglong Tian CLIP VLM 49 62 0 07 Dec 2023
Alpha-CLIP: A CLIP Model Focusing on Wherever You Want Zeyi Sun Ye Fang Tong Wu Pan Zhang Yuhang Zang Shu Kong Yuanjun Xiong Dahua Lin Jiaqi Wang VLM CLIP 51 83 0 06 Dec 2023
APoLLo: Unified Adapter and Prompt Learning for Vision Language Models Sanjoy Chowdhury Sayan Nag Dinesh Manocha VLM 33 17 0 04 Dec 2023
Improve Supervised Representation Learning with Masked Image Modeling Kaifeng Chen Daniel M. Salz Huiwen Chang Kihyuk Sohn Dilip Krishnan Mojtaba Seyedhosseini SSL ViT 50 3 0 01 Dec 2023
MLLMs-Augmented Visual-Language Representation Learning Yanqing Liu Kai Wang Wenqi Shao Ping Luo Yu Qiao Mike Zheng Shou Kaipeng Zhang Yang You VLM 29 11 0 30 Nov 2023
MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training Pavan Kumar Anasosalu Vasu Hadi Pouransari Fartash Faghri Raviteja Vemulapalli Oncel Tuzel CLIP VLM 36 43 0 28 Nov 2023
Unified Medical Image Pre-training in Language-Guided Common Semantic Space Xiaoxuan He Yifan Yang Xinyang Jiang Xufang Luo Haoji Hu Siyun Zhao Dongsheng Li Yuqing Yang Lili Qiu 48 1 0 24 Nov 2023
Comparing Generalization in Learning with Limited Numbers of Exemplars: Transformer vs. RNN in Attractor Dynamics Rui Fukushima Jun Tani 11 0 0 15 Nov 2023
Pretrain like Your Inference: Masked Tuning Improves Zero-Shot Composed Image Retrieval Junyang Chen Hanjiang Lai VLM 45 15 0 13 Nov 2023
Rethinking Evaluation Metrics of Open-Vocabulary Segmentaion Hao Zhou Tiancheng Shen Xu Yang Hai Huang Xiangtai Li Lu Qi Ming-Hsuan Yang 97 12 0 06 Nov 2023
FLAP: Fast Language-Audio Pre-training Ching-Feng Yeh Po-Yao Huang Vasu Sharma Shang-Wen Li Gargi Ghosh CLIP VLM 44 8 0 02 Nov 2023
CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoders A. Fuller K. Millard James R. Green 29 60 0 01 Nov 2023
Harvest Video Foundation Models via Efficient Post-Pretraining Yizhuo Li Kunchang Li Yinan He Yi Wang Yali Wang Limin Wang Yu Qiao Ping Luo CLIP VLM VGen 54 2 0 30 Oct 2023
Bridging The Gaps Between Token Pruning and Full Pre-training via Masked Fine-tuning Fengyuan Shi Limin Wang ViT 38 0 0 26 Oct 2023
PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm Haoyi Zhu Honghui Yang Xiaoyang Wu Di Huang Sha Zhang ... Hengshuang Zhao Chunhua Shen Yu Qiao Tong He Wanli Ouyang SSL 77 43 0 12 Oct 2023
VeCLIP: Improving CLIP Training via Visual-enriched Captions Zhengfeng Lai Haotian Zhang Bowen Zhang Wentao Wu Haoping Bai ... Zhe Gan Jiulong Shan Chen-Nee Chuah Yinfei Yang Meng Cao CLIP VLM 37 28 0 11 Oct 2023
An HCI-Centric Survey and Taxonomy of Human-Generative-AI Interactions Jingyu Shi Rahul Jain Hyungjun Doh Ryo Suzuki Karthik Ramani 3DV 39 22 0 11 Oct 2023
Uni3D: Exploring Unified 3D Representation at Scale Junsheng Zhou Jinsheng Wang Baorui Ma Yu-Shen Liu Tiejun Huang Xinlong Wang 40 89 0 10 Oct 2023
LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment Bin Zhu Bin Lin Munan Ning Yang Yan Jiaxi Cui ... Zongwei Li Wancai Zhang Zhifeng Li Wei Liu Liejie Yuan VLM MLLM 32 207 0 03 Oct 2023
DST-Det: Simple Dynamic Self-Training for Open-Vocabulary Object Detection Shilin Xu Xiangtai Li Size Wu Wenwei Zhang Yunhai Tong Chen Change Loy ObjD VLM 34 0 0 02 Oct 2023
ELIP: Efficient Language-Image Pre-training with Fewer Vision Tokens Yangyang Guo Haoyu Zhang Yongkang Wong Liqiang Nie Mohan Kankanhalli VLM 30 3 0 28 Sep 2023
BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuning Ruyang Liu Chen Li Yixiao Ge Ying Shan Thomas H. Li Ge Li 25 29 0 27 Sep 2023
Survey of Social Bias in Vision-Language Models Nayeon Lee Yejin Bang Holy Lovenia Samuel Cahyawijaya Wenliang Dai Pascale Fung VLM 47 16 0 24 Sep 2023
TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance Kan Wu Houwen Peng Zhenghong Zhou Bin Xiao Mengchen Liu ... Xi Xi Chen Xinggang Wang Hongyang Chao Han Hu VLM OODD 29 54 0 21 Sep 2023
DreamLLM: Synergistic Multimodal Comprehension and Creation Runpei Dong Chunrui Han Yuang Peng Zekun Qi Zheng Ge ... Hao-Ran Wei Xiangwen Kong Xiangyu Zhang Kaisheng Ma Li Yi MLLM 47 176 0 20 Sep 2023
Disentangling Spatial and Temporal Learning for Efficient Image-to-Video Transfer Learning Zhiwu Qing Shiwei Zhang Ziyuan Huang Yingya Zhang Changxin Gao Deli Zhao Nong Sang 37 18 0 14 Sep 2023
Language Models as Black-Box Optimizers for Vision-Language Models Shihong Liu Zhiqiu Lin Samuel Yu Ryan Lee Tiffany Ling Deepak Pathak Deva Ramanan VLM 35 28 0 12 Sep 2023
DAT++: Spatially Dynamic Vision Transformer with Deformable Attention Zhuofan Xia Xuran Pan Shiji Song Li Erran Li Gao Huang ViT 39 25 0 04 Sep 2023
Contrastive Feature Masking Open-Vocabulary Vision Transformer Dahun Kim A. Angelova Weicheng Kuo ObjD VLM 25 27 0 02 Sep 2023
Cross-Modal Retrieval Meets Inference:Improving Zero-Shot Classification with Cross-Modal Retrieval Seong-Hoon Eom Namgyu Ho Jaehoon Oh Se-Young Yun CLIP VLM 38 0 0 29 Aug 2023
Prompting Visual-Language Models for Dynamic Facial Expression Recognition Zengqun Zhao Ioannis Patras VLM 13 33 0 25 Aug 2023
The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open World Weiyun Wang Min Shi Qingyun Li Wen Wang Zhenhang Huang ... Zhiguo Cao Yushi Chen Tong Lu Jifeng Dai Yu Qiao LRM MLLM 53 84 0 03 Aug 2023
A Survey on Generative Modeling with Limited Data, Few Shots, and Zero Shot Milad Abdollahzadeh Touba Malekzadeh Christopher T. H. Teo Keshigeyan Chandrasegaran Guimeng Liu Ngai-man Cheung VLM MedIm 47 21 0 26 Jul 2023
Foundational Models Defining a New Era in Vision: A Survey and Outlook Muhammad Awais Muzammal Naseer Salman Khan Rao Muhammad Anwer Hisham Cholakkal M. Shah Ming Yang Fahad Shahbaz Khan VLM 40 119 0 25 Jul 2023
Towards a Visual-Language Foundation Model for Computational Pathology Ming Y. Lu Bowen Chen Drew F. K. Williamson Richard J. Chen Ivy Liang ... Andrew Zhang L. Le Georg Gerber Anil V. Parwani Faisal Mahmood VLM MedIm 42 46 0 24 Jul 2023
CLIP-KD: An Empirical Study of CLIP Model Distillation Chuanguang Yang Zhulin An Libo Huang Junyu Bi Xinqiang Yu Hansheng Yang Boyu Diao Yongjun Xu VLM 29 27 0 24 Jul 2023
GEM: Boost Simple Network for Glass Surface Segmentation via Vision Foundation Models Jing Hao Xinyu Li Liang Gao Shumin Han VLM DiffM 30 2 0 22 Jul 2023
GIST: Generating Image-Specific Text for Fine-grained Object Classification Kathleen M. Lewis Emily Mu Adrian Dalca John Guttag VLM 29 7 0 21 Jul 2023
M-FLAG: Medical Vision-Language Pre-training with Frozen Language Models and Latent Space Geometry Optimization Che Liu Sibo Cheng Cen Chen Mengyun Qiao Weitong Zhang Anand Shah Wenjia Bai Rossella Arcucci VLM 30 56 0 17 Jul 2023
Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution Mostafa Dehghani Basil Mustafa Josip Djolonga Jonathan Heek Matthias Minderer ... Avital Oliver Piotr Padlewski A. Gritsenko Mario Luvcić N. Houlsby ViT 31 105 0 12 Jul 2023
EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbone Shraman Pramanick Yale Song Sayan Nag Kevin Qinghong Lin Hardik Shah Mike Zheng Shou Ramalingam Chellappa Pengchuan Zhang VLM 42 89 0 11 Jul 2023
Masked Vision and Language Pre-training with Unimodal and Multimodal Contrastive Losses for Medical Visual Question Answering Pengfei Li Gang Liu Jinlong He Zixu Zhao Shenjun Zhong 13 34 0 11 Jul 2023
Knowledge Graph Self-Supervised Rationalization for Recommendation Yuhao Yang Chao Huang Lianghao Xia Chunzhen Huang 32 88 0 06 Jul 2023