v1v2 (latest)

Contrastive Learning of Medical Visual Representations from Paired Images and Text

2 October 2020

Yuhao Zhang

Hang Jiang

Yasuhide Miura

Christopher D. Manning

C. Langlotz

MedIm

ArXiv (abs)PDF HTML

Papers citing "Contrastive Learning of Medical Visual Representations from Paired Images and Text"

50 / 459 papers shown

Title
MXM-CLR: A Unified Framework for Contrastive Learning of Multifold Cross-Modal Representations Ye Wang Bo‐Shu Jiang C. Zou Rui Ma 104 5 0 20 Mar 2023
Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generation Mingjie Li Bingqian Lin Zicong Chen Haokun Lin Xiaodan Liang Xiaojun Chang MedIm 82 117 0 18 Mar 2023
Identifiability Results for Multimodal Contrastive Learning Imant Daunhawer Alice Bizeul Emanuele Palumbo Alexander Marx Julia E. Vogt 63 42 0 16 Mar 2023
Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning Qian Jiang Changyou Chen Han Zhao Liqun Chen Q. Ping S. D. Tran Yi Xu Belinda Zeng Trishul Chilimbi 97 43 0 10 Mar 2023
Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Language Philipp Seidl Andreu Vall Sepp Hochreiter Günter Klambauer 144 41 0 06 Mar 2023
ConTEXTual Net: A Multimodal Vision-Language Model for Segmentation of Pneumothorax Zachary Huemann Xin Tie Junjie Hu Tyler Bradshaw 59 17 0 02 Mar 2023
Knowledge-enhanced Visual-Language Pre-training on Chest Radiology Images Xiaoman Zhang Chaoyi Wu Ya Zhang Yanfeng Wang Weidi Xie MedIm 108 137 0 27 Feb 2023
Knowledge-infused Contrastive Learning for Urban Imagery-based Socioeconomic Prediction Yu Liu Xin Zhang Jingtao Ding Yanxin Xi Yong Li 99 36 0 25 Feb 2023
Test-Time Distribution Normalization for Contrastively Learned Vision-language Models Yi Zhou Juntao Ren Fengyu Li Ramin Zabih Ser-Nam Lim VLM 98 15 0 22 Feb 2023
Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts Zhihong Chen Shizhe Diao Benyou Wang Guanbin Li Xiang Wan MedIm 127 33 0 17 Feb 2023
Multi-modal Machine Learning in Engineering Design: A Review and Future Directions Binyang Song Ruilin Zhou Faez Ahmed AI4CE 144 46 0 14 Feb 2023
NYCU-TWO at Memotion 3: Good Foundation, Good Teacher, then you have Good Meme Analysis Yu-Chien Tang Kuang-Da Wang Ting-Yun Ou Wenjie Peng 32 2 0 13 Feb 2023
A Review of Predictive and Contrastive Self-supervised Learning for Medical Images Wei-Chien Wang Euijoon Ahn Da-wei Feng Jinman Kim MedIm 130 29 0 10 Feb 2023
Diagnosing and Rectifying Vision Models using Language Yuhui Zhang Jeff Z. HaoChen Shih-Cheng Huang Kuan-Chieh Wang James Zou Serena Yeung 89 48 0 08 Feb 2023
Effective Robustness against Natural Distribution Shifts for Models with Different Training Data Zhouxing Shi Nicholas Carlini Ananth Balashankar Ludwig Schmidt Cho-Jui Hsieh Alex Beutel Yao Qin OOD 64 10 0 02 Feb 2023
Advancing Radiograph Representation Learning with Masked Record Modeling Hong-Yu Zhou Chenyu Lian Lian-cheng Wang Yizhou Yu MedIm 112 59 0 30 Jan 2023
Exploring Image Augmentations for Siamese Representation Learning with Chest X-Rays Rogier van der Sluijs Nandita Bhaskhar D. Rubin C. Langlotz Akshay S. Chaudhari SSL 48 14 0 30 Jan 2023
Pre-text Representation Transfer for Deep Learning with Limited Imbalanced Data : Application to CT-based COVID-19 Detection F. Altaf Syed Mohammed Shamsul Islam N. Janjua Naveed Akhtar MedIm AI4TS 44 1 0 21 Jan 2023
RILS: Masked Visual Reconstruction in Language Semantic Space Shusheng Yang Yixiao Ge Kun Yi Dian Li Ying Shan Xiaohu Qie Xinggang Wang CLIP 95 11 0 17 Jan 2023
CLIP the Gap: A Single Domain Generalization Approach for Object Detection Vidit Vidit Martin Engilberge Mathieu Salzmann VLM ObjD 93 83 0 13 Jan 2023
Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study Mariya Hendriksen Svitlana Vakulenko E. Kuiper Maarten de Rijke 89 4 0 12 Jan 2023
EXIF as Language: Learning Cross-Modal Associations Between Images and Camera Metadata Chenhao Zheng Ayush Shrivastava Andrew Owens VLM 139 12 0 11 Jan 2023
Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing Shruthi Bannur Stephanie L. Hyland Qianchu Liu Fernando Pérez-García Maximilian Ilse ... Maria T. A. Wetscherek M. Lungren A. Nori Javier Alvarez-Valle Ozan Oktay 87 127 0 11 Jan 2023
CiT: Curation in Training for Effective Vision-Language Data Hu Xu Saining Xie Po-Yao (Bernie) Huang Licheng Yu Russ Howes Gargi Ghosh Luke Zettlemoyer Christoph Feichtenhofer VLM DiffM 64 26 0 05 Jan 2023
MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training in Radiology Chaoyi Wu Xiaoman Zhang Ya Zhang Yanfeng Wang Weidi Xie LM&MA VLM 118 120 0 05 Jan 2023
Contrastive Language-Vision AI Models Pretrained on Web-Scraped Multimodal Data Exhibit Sexual Objectification Bias Robert Wolfe Yiwei Yang Billy Howe Aylin Caliskan DiffM 131 57 0 21 Dec 2022
Significantly Improving Zero-Shot X-ray Pathology Classification via Fine-tuning Pre-trained Image-Text Encoders Jongseong Jang Daeun Kyung Seunghyeon Kim Honglak Lee Kyunghoon Bae Edward Choi LM&MA MedIm 70 11 0 14 Dec 2022
TIER: Text-Image Entropy Regularization for CLIP-style models Anil Palepu Andrew L. Beam MedIm 118 6 0 13 Dec 2022
Using Multiple Instance Learning to Build Multimodal Representations Peiqi Wang W. Wells Seth Berkowitz Steven Horng Polina Golland SSL 65 6 0 11 Dec 2022
Vision and Structured-Language Pretraining for Cross-Modal Food Retrieval Mustafa Shukor Nicolas Thome Matthieu Cord CLIP CoGe 95 9 0 08 Dec 2022
Generating and Weighting Semantically Consistent Sample Pairs for Ultrasound Contrastive Learning Yixiong Chen Chunhui Zhang C. Ding Li Liu 99 14 0 08 Dec 2022
Improving Zero-Shot Models with Label Distribution Priors Jonathan Kahana Niv Cohen Yedid Hoshen VLM 136 14 0 01 Dec 2022
Normalized Contrastive Learning for Text-Video Retrieval Yookoon Park Mahmoud Azab Bo Xiong Seungwhan Moon Florian Metze Gourab Kundu Kirmani Ahmed 75 12 0 30 Nov 2022
SuS-X: Training-Free Name-Only Transfer of Vision-Language Models Vishaal Udandarao Ankush Gupta Samuel Albanie VLM MLLM 105 103 0 28 Nov 2022
Can we Adopt Self-supervised Pretraining for Chest X-Rays? Arsh Verma Makarand Tapaswi SSL 49 3 0 23 Nov 2022
RoentGen: Vision-Language Foundation Model for Chest X-ray Generation Pierre J. Chambon Christian Blüthgen Jean-Benoit Delbrouck Rogier van der Sluijs M. Polacin Juan Manuel Zambrano Chaves Tanishq Mathew Abraham Shivanshu Purohit C. Langlotz Akshay S. Chaudhari LM&MA DiffM MedIm 92 102 0 23 Nov 2022
Multitask Vision-Language Prompt Tuning Sheng Shen Shijia Yang Tianjun Zhang Bohan Zhai Joseph E. Gonzalez Kurt Keutzer Trevor Darrell VLM VPVLM 115 53 0 21 Nov 2022
Delving StyleGAN Inversion for Image Editing: A Foundation Latent Space Viewpoint Hongyu Liu Yibing Song Qifeng Chen DiffM 96 21 0 21 Nov 2022
Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works Chao Tao Ji Qi Mingning Guo Qing Zhu Haifeng Li SSL 104 59 0 15 Nov 2022
Multilingual and Multimodal Topic Modelling with Pretrained Embeddings Elaine Zosa Lidia Pivovarova BDL 58 8 0 15 Nov 2022
The Role of Local Alignment and Uniformity in Image-Text Contrastive Learning on Medical Images Philip Muller Georgios Kaissis Daniel Rueckert MedIm 120 8 0 14 Nov 2022
ContextCLIP: Contextual Alignment of Image-Text pairs on CLIP visual representations Chanda Grover Indra Deep Mastan Debayan Gupta VLM CLIP 59 4 0 14 Nov 2022
MuMIC -- Multimodal Embedding for Multi-label Image Classification with Tempered Sigmoid Feng Wang Sarai Mizrachi Moran Beladev Guy Nadav Gil Amsalem Karen Lastmann Assaraf Hadas Harush Boker VLM 78 13 0 02 Nov 2022
Towards Reliable Zero Shot Classification in Self-Supervised Models with Conformal Prediction Bhawesh Kumar Anil Palepu Rudraksh Tuwani Andrew L. Beam 33 8 0 27 Oct 2022
Learning Joint Representation of Human Motion and Language Jihoon Kim Youngjae Yu Seungyoung Shin Taehyun Byun Sungjoon Choi 77 5 0 27 Oct 2022
FaD-VLP: Fashion Vision-and-Language Pre-training towards Unified Retrieval and Captioning Suvir Mirchandani Licheng Yu Mengjiao MJ Wang Animesh Sinha Wen-Jun Jiang Tao Xiang Ning Zhang 81 16 0 26 Oct 2022
MedCLIP: Contrastive Learning from Unpaired Medical Images and Text Zifeng Wang Zhenbang Wu Dinesh Agarwal Jimeng Sun CLIP VLM MedIm 129 436 0 18 Oct 2022
Improving Radiology Summarization with Radiograph and Anatomy Prompts Jinpeng Hu Zhihong Chen Yang Liu Xiang Wan Tsung-Hui Chang MedIm 88 8 0 15 Oct 2022
Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learning Fuying Wang Yuyin Zhou Shujun Wang V. Vardhanabhuti Lequan Yu 117 149 0 12 Oct 2022
HiCo: Hierarchical Contrastive Learning for Ultrasound Video Model Pretraining Chunhui Zhang Yixiong Chen Li Liu Qiong Liu Xiaoping Zhou VLM 108 9 0 10 Oct 2022