Data-Efficient Contrastive Language-Image Pretraining: Prioritizing Data
Quality over Quantity

v1v2 (latest)

Data-Efficient Contrastive Language-Image Pretraining: Prioritizing Data Quality over Quantity

18 March 2024

Siddharth Joshi

Ali Payani

Baharan Mirzasoleiman

ArXiv (abs)PDF HTML Github (8★)

Papers citing "Data-Efficient Contrastive Language-Image Pretraining: Prioritizing Data Quality over Quantity"

18 / 18 papers shown

Title
MM-GEN: Enhancing Task Performance Through Targeted Multimodal Data Curation S. Joshi Besmira Nushi Vidhisha Balachandran Varun Chandrasekaran Vibhav Vineet Neel Joshi Baharan Mirzasoleiman MLLM VLM 160 0 0 07 Jan 2025
Dataset Distillation via Knowledge Distillation: Towards Efficient Self-Supervised Pre-Training of Deep Networks S. Joshi Jiayi Ni Baharan Mirzasoleiman DD 169 2 0 03 Oct 2024
On the Generalization of Multi-modal Contrastive Learning Qi Zhang Yifei Wang Yisen Wang 68 26 0 07 Jun 2023
Towards Sustainable Learning: Coresets for Data-efficient Deep Learning Yu Yang Hao Kang Baharan Mirzasoleiman 81 35 0 02 Jun 2023
Understanding Multimodal Contrastive Learning and Incorporating Unpaired Data Ryumei Nakada Halil Ibrahim Gulluk Zhun Deng Wenlong Ji James Zou Linjun Zhang SSL VLM 89 41 0 13 Feb 2023
Adaptive Second Order Coresets for Data-efficient Machine Learning Omead Brandon Pooladzandi David Davini Baharan Mirzasoleiman 94 64 0 28 Jul 2022
SLIP: Self-supervision meets Language-Image Pre-training Norman Mu Alexander Kirillov David Wagner Saining Xie VLM CLIP 143 490 0 23 Dec 2021
Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm Yangguang Li Feng Liang Lichen Zhao Yufeng Cui Wanli Ouyang Jing Shao F. Yu Junjie Yan VLM CLIP 150 458 0 11 Oct 2021
The Power of Contrast for Feature Learning: A Theoretical Analysis Wenlong Ji Zhun Deng Ryumei Nakada James Zou Linjun Zhang SSL 98 53 0 06 Oct 2021
Deep Learning on a Data Diet: Finding Important Examples Early in Training Mansheej Paul Surya Ganguli Gintare Karolina Dziugaite 117 461 0 15 Jul 2021
Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss Jeff Z. HaoChen Colin Wei Adrien Gaidon Tengyu Ma SSL 77 321 0 08 Jun 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 447 1,127 0 17 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 459 3,893 0 11 Feb 2021
On Robustness and Transferability of Convolutional Neural Networks Josip Djolonga Jessica Yung Michael Tschannen Rob Romijnders Lucas Beyer ... D. Moldovan Sylvain Gelly N. Houlsby Xiaohua Zhai Mario Lucic OOD 84 156 0 16 Jul 2020
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization Dan Hendrycks Steven Basart Norman Mu Saurav Kadavath Frank Wang ... Samyak Parajuli Mike Guo Basel Alomair Jacob Steinhardt Justin Gilmer OOD 350 1,754 0 29 Jun 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 378 18,866 0 13 Feb 2020
Learning Robust Global Representations by Penalizing Local Predictive Power Haohan Wang Songwei Ge Eric Xing Zachary Chase Lipton OOD 122 966 0 29 May 2019
Do ImageNet Classifiers Generalize to ImageNet? Benjamin Recht Rebecca Roelofs Ludwig Schmidt Vaishaal Shankar OOD SSeg VLM 121 1,726 0 13 Feb 2019