v1v2 (latest)

Contrastive Learning of Medical Visual Representations from Paired Images and Text

2 October 2020

Yuhao Zhang

Hang Jiang

Yasuhide Miura

Christopher D. Manning

C. Langlotz

MedIm

ArXiv (abs)PDF HTML

Papers citing "Contrastive Learning of Medical Visual Representations from Paired Images and Text"

50 / 459 papers shown

Title
Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning Weixin Liang Yuhui Zhang Yongchan Kwon Serena Yeung James Zou VLM 149 430 0 03 Mar 2022
Provable Stochastic Optimization for Global Contrastive Learning: Small Batch Does Not Harm Performance Zhuoning Yuan Yuexin Wu Zi-qi Qiu Xianzhi Du Lijun Zhang Denny Zhou Tianbao Yang 133 30 0 24 Feb 2022
GroupViT: Semantic Segmentation Emerges from Text Supervision Jiarui Xu Shalini De Mello Sifei Liu Wonmin Byeon Thomas Breuel Jan Kautz Xinyu Wang ViT VLM 304 527 0 22 Feb 2022
A Survey on Visual Transfer Learning using Knowledge Graphs Sebastian Monka Lavdim Halilaj Achim Rettinger 118 23 0 27 Jan 2022
Contrastive and Selective Hidden Embeddings for Medical Image Segmentation Zhuowei Li Zihao Liu Zhiqiang Hu Qing Xia Ruiqin Xiong Shaoting Zhang Dimitris N. Metaxas Tingting Jiang SSL 65 8 0 21 Jan 2022
TriCoLo: Trimodal Contrastive Loss for Text to Shape Retrieval Yue Ruan Han-Hung Lee Yiming Zhang Ke Zhang Angel X. Chang 95 22 0 19 Jan 2022
Weakly Supervised Contrastive Learning for Better Severity Scoring of Lung Ultrasound Gautam Rajendrakumar Gare H. V. Tran Bennett P deBoisblanc R. Rodriguez J. Galeotti 83 5 0 18 Jan 2022
MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound Rowan Zellers Jiasen Lu Ximing Lu Youngjae Yu Yanpeng Zhao Mohammadreza Salehi Aditya Kusupati Jack Hessel Ali Farhadi Yejin Choi 113 215 0 07 Jan 2022
C2-CRS: Coarse-to-Fine Contrastive Learning for Conversational Recommender System Yuanhang Zhou Kun Zhou Wayne Xin Zhao Cheng Wang Peng Jiang He Hu 80 53 0 04 Jan 2022
Does CLIP Benefit Visual Question Answering in the Medical Domain as Much as it Does in the General Domain? Sedigheh Eslami Gerard de Melo Christoph Meinel CLIP MedIm 84 121 0 27 Dec 2021
A Fistful of Words: Learning Transferable Visual Models from Bag-of-Words Supervision Ajinkya Tejankar Maziar Sanjabi Bichen Wu Saining Xie Madian Khabsa Hamed Pirsiavash Hamed Firooz VLM 108 18 0 27 Dec 2021
Extending CLIP for Category-to-image Retrieval in E-commerce Mariya Hendriksen Maurits J. R. Bleeker Svitlana Vakulenko Nanne van Noord E. Kuiper Maarten de Rijke VLM 79 30 0 21 Dec 2021
Data Efficient Language-supervised Zero-shot Recognition with Optimal Transport Distillation Bichen Wu Rui Cheng Peizhao Zhang Tianren Gao Peter Vajda Joseph E. Gonzalez VLM 110 45 0 17 Dec 2021
Technical Language Supervision for Intelligent Fault Diagnosis in Process Industry Karl Lowenmark C. Taal S. Schnabel Marcus Liwicki Fredrik Sandin 52 7 0 11 Dec 2021
Multimodal Conditional Image Synthesis with Product-of-Experts GANs Xun Huang Arun Mallya Ting-Chun Wang Xuan Li DiffM 109 90 0 09 Dec 2021
CMA-CLIP: Cross-Modality Attention CLIP for Image-Text Classification Huidong Liu Shaoyuan Xu Jinmiao Fu Yang Liu Ning Xie Chien Wang Bryan Wang Yi Sun CLIP VLM 72 29 0 07 Dec 2021
Joint Learning of Localized Representations from Medical Images and Reports Philipp Muller Georgios Kaissis Cong Zou Daniel Munich 210 86 0 06 Dec 2021
Overcoming the Domain Gap in Neural Action Representations Semih Günel Florian Aymanns S. Honari Pavan Ramdya Pascal Fua 82 3 0 02 Dec 2021
Sound-Guided Semantic Image Manipulation Seung Hyun Lee Wonseok Roh Wonmin Byeon Sang Ho Yoon Chanyoung Kim Jinkyu Kim Sangpil Kim DiffM 105 43 0 30 Nov 2021
An implementation of the "Guess who?" game using CLIP Arnau Martí-Sarri V. Rodríguez-Fernández CLIP MLLM VLM 43 0 0 30 Nov 2021
Overcoming the Domain Gap in Contrastive Learning of Neural Action Representations Semih Günel Florian Aymanns S. Honari Pavan Ramdya Pascal Fua SSL 67 0 0 29 Nov 2021
LAFITE: Towards Language-Free Training for Text-to-Image Generation Yufan Zhou Ruiyi Zhang Changyou Chen Chunyuan Li Chris Tensmeyer Tong Yu Jiuxiang Gu Jinhui Xu Tong Sun VLM 86 168 0 27 Nov 2021
ContIG: Self-supervised Multimodal Contrastive Learning for Medical Imaging with Genetics Aiham Taleb Matthias Kirchler Remo Monti C. Lippert SSL MedIm 75 58 0 26 Nov 2021
Combined Scaling for Zero-shot Transfer Learning Hieu H. Pham Zihang Dai Golnaz Ghiasi Kenji Kawaguchi Hanxiao Liu ... Yi-Ting Chen Minh-Thang Luong Yonghui Wu Mingxing Tan Quoc V. Le VLM 118 202 0 19 Nov 2021
LiT: Zero-Shot Transfer with Locked-image text Tuning Xiaohua Zhai Tianlin Li Basil Mustafa Andreas Steiner Daniel Keysers Alexander Kolesnikov Lucas Beyer VLM 162 561 0 15 Nov 2021
Deep AUC Maximization for Medical Image Classification: Challenges and Opportunities Tianbao Yang 97 5 0 01 Nov 2021
Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm Yangguang Li Feng Liang Lichen Zhao Yufeng Cui Wanli Ouyang Jing Shao F. Yu Junjie Yan VLM CLIP 165 458 0 11 Oct 2021
Pre-trained Language Models in Biomedical Domain: A Systematic Survey Benyou Wang Qianqian Xie Jiahuan Pei Zhihong Chen Prayag Tiwari Zhao Li Jie Fu LM&MA AI4CE 154 172 0 11 Oct 2021
SCEHR: Supervised Contrastive Learning for Clinical Risk Prediction using Electronic Health Records Chengxi Zang Fei Wang 38 16 0 11 Oct 2021
Contrastive Video-Language Segmentation Chen Liang Yawei Luo Yu Wu Yi Yang VLM VOS 108 1 0 29 Sep 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 219 741 0 04 Sep 2021
Learning to Prompt for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VPVLM CLIP VLM 550 2,444 0 02 Sep 2021
Imposing Relation Structure in Language-Model Embeddings Using Contrastive Learning Christos Theodoropoulos James Henderson Andrei Catalin Coman Marie-Francine Moens 48 15 0 02 Sep 2021
LocTex: Learning Data-Efficient Visual Representations from Localized Textual Supervision Zhijian Liu Simon Stent Jie Li John Gideon Song Han VLM 106 10 0 26 Aug 2021
Unsupervised Local Discrimination for Medical Images Huai Chen Renzhen Wang Xiuying Wang Jieyu Li Qu Fang Hui Li Jianhao Bai Qing Peng Deyu Meng Lisheng Wang SSL 60 7 0 21 Aug 2021
Semi-weakly Supervised Contrastive Representation Learning for Retinal Fundus Images Boon Peng Yap B. Ng SSL 75 3 0 04 Aug 2021
Exploiting the relationship between visual and textual features in social networks for image classification with zero-shot deep learning Luis Lucas David Tomás Jose Garcia-Rodriguez VLM CLIP 21 5 0 08 Jul 2021
Challenges for machine learning in clinical translation of big data imaging studies Nicola K. Dinsdale Emma Bluemke V. Sundaresan M. Jenkinson Stephen Smith Ana I. L. Namburete AI4CE 122 44 0 07 Jul 2021
Poisoning and Backdooring Contrastive Learning Nicholas Carlini Andreas Terzis 68 166 0 17 Jun 2021
MERLOT: Multimodal Neural Script Knowledge Models Rowan Zellers Ximing Lu Jack Hessel Youngjae Yu J. S. Park Jize Cao Ali Farhadi Yejin Choi VLM LRM 104 384 0 04 Jun 2021
Recent advances and clinical applications of deep learning in medical image analysis Xuxin Chen Ximing Wang Kecheng Zhang K. Fung T. Thai K. Moore Robert S. Mannel Hong Liu B. Zheng Y. Qiu OOD 136 612 0 27 May 2021
AC-CovidNet: Attention Guided Contrastive CNN for Recognition of Covid-19 in Chest X-Ray Images Anirudh Ambati S. Dubey OOD 49 6 0 21 May 2021
Data-Efficient Language-Supervised Zero-Shot Learning with Self-Distillation Rui Cheng Bichen Wu Peizhao Zhang Peter Vajda Joseph E. Gonzalez CLIP VLM 83 33 0 18 Apr 2021
Exploring Visual Engagement Signals for Representation Learning Menglin Jia Zuxuan Wu A. Reiter Claire Cardie Serge Belongie Ser-Nam Lim 82 13 0 15 Apr 2021
Patient Contrastive Learning: a Performant, Expressive, and Practical Approach to ECG Modeling N. Diamant Erik Reinertsen S. Song Aaron D Aguirre Collin M. Stultz P. Batra 64 43 0 09 Apr 2021
Modern Hopfield Networks for Few- and Zero-Shot Reaction Template Prediction Philipp Seidl Philipp Renz N. Dyubankova Paulo Neves Jonas Verhoeven Marwin H. S. Segler J. Wegner Sepp Hochreiter Günter Klambauer 90 17 0 07 Apr 2021
Contrastive Learning of Single-Cell Phenotypic Representations for Treatment Classification A. Perakis A. Gorji Samriddhi Jain K. Chaitanya Simone Rizza E. Konukoglu SSL 82 34 0 30 Mar 2021
Self-supervised Image-text Pre-training With Mixed Data In Chest X-rays Xiaosong Wang Ziyue Xu Leo K. Tam Dong Yang Daguang Xu ViT MedIm 68 24 0 30 Mar 2021
Let Your Heart Speak in its Mother Tongue: Multilingual Captioning of Cardiac Signals Dani Kiyasseh T. Zhu David Clifton 122 0 0 19 Mar 2021
Multimodal Representation Learning via Maximization of Local Mutual Information Ruizhi Liao Daniel Moyer Miriam Cha Keegan Quigley Seth Berkowitz Steven Horng Polina Golland W. Wells SSL 94 43 0 08 Mar 2021