Contrastive Learning of Medical Visual Representations from Paired Images and Text

2 October 2020

Yuhao Zhang

Hang Jiang

Yasuhide Miura

Christopher D. Manning

C. Langlotz

MedIm

ArXiv PDF HTML

Papers citing "Contrastive Learning of Medical Visual Representations from Paired Images and Text"

50 / 445 papers shown

Title
Exploring Image Augmentations for Siamese Representation Learning with Chest X-Rays Rogier van der Sluijs Nandita Bhaskhar D. Rubin C. Langlotz Akshay S. Chaudhari SSL 37 13 0 30 Jan 2023
Pre-text Representation Transfer for Deep Learning with Limited Imbalanced Data : Application to CT-based COVID-19 Detection F. Altaf Syed Mohammed Shamsul Islam N. Janjua Naveed Akhtar MedIm AI4TS 27 1 0 21 Jan 2023
RILS: Masked Visual Reconstruction in Language Semantic Space Shusheng Yang Yixiao Ge Kun Yi Dian Li Ying Shan Xiaohu Qie Xinggang Wang CLIP 43 11 0 17 Jan 2023
CLIP the Gap: A Single Domain Generalization Approach for Object Detection Vidit Vidit Martin Engilberge Mathieu Salzmann VLM ObjD 27 75 0 13 Jan 2023
Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study Mariya Hendriksen Svitlana Vakulenko E. Kuiper Maarten de Rijke 34 5 0 12 Jan 2023
EXIF as Language: Learning Cross-Modal Associations Between Images and Camera Metadata Chenhao Zheng Ayush Shrivastava Andrew Owens VLM 33 11 0 11 Jan 2023
Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing Shruthi Bannur Stephanie L. Hyland Qianchu Liu Fernando Pérez-García Maximilian Ilse ... Maria T. A. Wetscherek M. Lungren A. Nori Javier Alvarez-Valle Ozan Oktay 36 113 0 11 Jan 2023
CiT: Curation in Training for Effective Vision-Language Data Hu Xu Saining Xie Po-Yao (Bernie) Huang Licheng Yu Russ Howes Gargi Ghosh Luke Zettlemoyer Christoph Feichtenhofer VLM DiffM 33 25 0 05 Jan 2023
MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training in Radiology Chaoyi Wu Xiaoman Zhang Ya Zhang Yanfeng Wang Weidi Xie LM&MA VLM 32 109 0 05 Jan 2023
Contrastive Language-Vision AI Models Pretrained on Web-Scraped Multimodal Data Exhibit Sexual Objectification Bias Robert Wolfe Yiwei Yang Billy Howe Aylin Caliskan DiffM 15 51 0 21 Dec 2022
Significantly Improving Zero-Shot X-ray Pathology Classification via Fine-tuning Pre-trained Image-Text Encoders Jongseong Jang Daeun Kyung Seunghyeon Kim Honglak Lee Kyunghoon Bae Edward Choi LM&MA MedIm 32 10 0 14 Dec 2022
TIER: Text-Image Entropy Regularization for CLIP-style models Anil Palepu Andrew L. Beam MedIm 26 6 0 13 Dec 2022
Using Multiple Instance Learning to Build Multimodal Representations Peiqi Wang W. Wells Seth Berkowitz Steven Horng Polina Golland SSL 24 6 0 11 Dec 2022
Vision and Structured-Language Pretraining for Cross-Modal Food Retrieval Mustafa Shukor Nicolas Thome Matthieu Cord CLIP CoGe 37 8 0 08 Dec 2022
Generating and Weighting Semantically Consistent Sample Pairs for Ultrasound Contrastive Learning Yixiong Chen Chunhui Zhang C. Ding Li Liu 34 14 0 08 Dec 2022
Improving Zero-Shot Models with Label Distribution Priors Jonathan Kahana Niv Cohen Yedid Hoshen VLM 14 14 0 01 Dec 2022
Normalized Contrastive Learning for Text-Video Retrieval Yookoon Park Mahmoud Azab Bo Xiong Seungwhan Moon Florian Metze Gourab Kundu Kirmani Ahmed 25 11 0 30 Nov 2022
SuS-X: Training-Free Name-Only Transfer of Vision-Language Models Vishaal Udandarao Ankush Gupta Samuel Albanie VLM MLLM 29 103 0 28 Nov 2022
Can we Adopt Self-supervised Pretraining for Chest X-Rays? Arsh Verma Makarand Tapaswi SSL 25 3 0 23 Nov 2022
RoentGen: Vision-Language Foundation Model for Chest X-ray Generation Pierre J. Chambon Christian Blüthgen Jean-Benoit Delbrouck Rogier van der Sluijs M. Polacin Juan Manuel Zambrano Chaves Tanishq Mathew Abraham Shivanshu Purohit C. Langlotz Akshay S. Chaudhari LM&MA DiffM MedIm 37 98 0 23 Nov 2022
Multitask Vision-Language Prompt Tuning Sheng Shen Shijia Yang Tianjun Zhang Bohan Zhai Joseph E. Gonzalez Kurt Keutzer Trevor Darrell VLM VPVLM 19 49 0 21 Nov 2022
Delving StyleGAN Inversion for Image Editing: A Foundation Latent Space Viewpoint Hongyu Liu Yibing Song Qifeng Chen DiffM 33 21 0 21 Nov 2022
Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works Chao Tao Ji Qi Mingning Guo Qing Zhu Haifeng Li SSL 24 56 0 15 Nov 2022
Multilingual and Multimodal Topic Modelling with Pretrained Embeddings Elaine Zosa Lidia Pivovarova BDL 13 8 0 15 Nov 2022
The Role of Local Alignment and Uniformity in Image-Text Contrastive Learning on Medical Images Philip Muller Georgios Kaissis Daniel Rueckert MedIm 24 7 0 14 Nov 2022
ContextCLIP: Contextual Alignment of Image-Text pairs on CLIP visual representations Chanda Grover Indra Deep Mastan Debayan Gupta VLM CLIP 24 4 0 14 Nov 2022
MuMIC -- Multimodal Embedding for Multi-label Image Classification with Tempered Sigmoid Feng Wang Sarai Mizrachi Moran Beladev Guy Nadav Gil Amsalem Karen Lastmann Assaraf Hadas Harush Boker VLM 22 13 0 02 Nov 2022
Towards Reliable Zero Shot Classification in Self-Supervised Models with Conformal Prediction Bhawesh Kumar Anil Palepu Rudraksh Tuwani Andrew L. Beam 28 8 0 27 Oct 2022
Learning Joint Representation of Human Motion and Language Jihoon Kim Youngjae Yu Seungyoung Shin Taehyun Byun Sungjoon Choi 31 5 0 27 Oct 2022
FaD-VLP: Fashion Vision-and-Language Pre-training towards Unified Retrieval and Captioning Suvir Mirchandani Licheng Yu Mengjiao MJ Wang Animesh Sinha Wen-Jun Jiang Tao Xiang Ning Zhang 35 16 0 26 Oct 2022
MedCLIP: Contrastive Learning from Unpaired Medical Images and Text Zifeng Wang Zhenbang Wu Dinesh Agarwal Jimeng Sun CLIP VLM MedIm 49 401 0 18 Oct 2022
Improving Radiology Summarization with Radiograph and Anatomy Prompts Jinpeng Hu Zhihong Chen Yang Liu Xiang Wan Tsung-Hui Chang MedIm 34 8 0 15 Oct 2022
Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learning Fuying Wang Yuyin Zhou Shujun Wang V. Vardhanabhuti Lequan Yu 31 137 0 12 Oct 2022
HiCo: Hierarchical Contrastive Learning for Ultrasound Video Model Pretraining Chunhui Zhang Yixiong Chen Li Liu Qiong Liu Xiaoping Zhou VLM 45 8 0 10 Oct 2022
Detailed Annotations of Chest X-Rays via CT Projection for Report Understanding C. Seibold Simon Reiß Saquib Sarfraz M. Fink Victoria L. Mayer Jan Sellner Moon S. Kim Klaus H. Maier-Hein Jens Kleesiek Rainer Stiefelhagen 37 18 0 07 Oct 2022
When and why vision-language models behave like bags-of-words, and what to do about it? Mert Yuksekgonul Federico Bianchi Pratyusha Kalluri Dan Jurafsky James Zou VLM CoGe 30 362 0 04 Oct 2022
Medical Image Understanding with Pretrained Vision Language Models: A Comprehensive Study Ziyuan Qin Huahui Yi Qicheng Lao Kang Li VLM 36 65 0 30 Sep 2022
Domain-aware Self-supervised Pre-training for Label-Efficient Meme Analysis Shivam Sharma Mohd Khizir Siddiqui Md. Shad Akhtar Tanmoy Chakraborty SSL 28 5 0 29 Sep 2022
A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective Chaoqi Chen Yushuang Wu Qiyuan Dai Hong-Yu Zhou Mutian Xu Sibei Yang Xiaoguang Han Yizhou Yu ViT MedIm AI4CE 27 73 0 27 Sep 2022
RepsNet: Combining Vision with Language for Automated Medical Reports A. Tanwani Joelle Barral Daniel Freedman MedIm 42 20 0 27 Sep 2022
Contrastive learning for unsupervised medical image clustering and reconstruction Matteo Ferrante T. Boccato Simeon E. Spasov A. Duggento N. Toschi SSL DRL 27 2 0 24 Sep 2022
FETA: Towards Specializing Foundation Models for Expert Task Applications Amit Alfassy Assaf Arbelle Oshri Halimi Sivan Harary Roei Herzig ... Christoph Auer Kate Saenko Peter W. J. Staar Rogerio Feris Leonid Karlinsky 23 19 0 08 Sep 2022
Real-Time Cattle Interaction Recognition via Triple-stream Network Yang Yang Mizuka Komatsu K. Oyama T. Ohkawa 25 3 0 06 Sep 2022
Disentangle and Remerge: Interventional Knowledge Distillation for Few-Shot Object Detection from A Conditional Causal Perspective Jiangmeng Li Yanan Zhang Jingyao Wang Hui Xiong Chengbo Jiao Xiaohui Hu Changwen Zheng Gang Hua CML 36 28 0 26 Aug 2022
CMSBERT-CLR: Context-driven Modality Shifting BERT with Contrastive Learning for linguistic, visual, acoustic Representations Junghun Kim Jihie Kim 25 2 0 21 Aug 2022
Dual Modality Prompt Tuning for Vision-Language Pre-Trained Model Yinghui Xing Qirui Wu De-Chun Cheng Shizhou Zhang Guoqiang Liang Peng Wang Yanning Zhang VLM VPVLM 56 51 0 17 Aug 2022
Towards Open-vocabulary Scene Graph Generation with Prompt-based Finetuning Tao He Lianli Gao Jingkuan Song Yuan-Fang Li VLM 34 50 0 17 Aug 2022
Self-supervised Multi-modal Training from Uncurated Image and Reports Enables Zero-shot Oversight Artificial Intelligence in Radiology Sangjoon Park Eunha Lee Kyung Sook Shin Jeonghyeon Lee Jong Chul Ye 33 2 0 10 Aug 2022
RadTex: Learning Efficient Radiograph Representations from Text Reports Keegan Quigley Miriam Cha Ruizhi Liao Geeticka Chauhan Steven Horng Seth Berkowitz Polina Golland MedIm 25 3 0 05 Aug 2022
NewsStories: Illustrating articles with visual summaries Reuben Tan Bryan A. Plummer Kate Saenko J. P. Lewis Avneesh Sud Thomas Leung VLM SSL 26 5 0 26 Jul 2022