Contrastive Learning of Medical Visual Representations from Paired Images and Text

2 October 2020

Yuhao Zhang

Hang Jiang

Yasuhide Miura

Christopher D. Manning

C. Langlotz

MedIm

ArXiv PDF HTML

Papers citing "Contrastive Learning of Medical Visual Representations from Paired Images and Text"

50 / 445 papers shown

Title
Unimodal vs. Multimodal Siamese Networks for Outfit Completion Mariya Hendriksen Viggo Overes 8 1 0 21 Jul 2022
Contrastive Adapters for Foundation Model Group Robustness Michael Zhang Christopher Ré VLM 18 62 0 14 Jul 2022
Text-driven Emotional Style Control and Cross-speaker Style Transfer in Neural TTS Yookyung Shin Younggun Lee Suhee Jo Yeongtae Hwang Taesu Kim 17 14 0 13 Jul 2022
American == White in Multimodal Language-and-Image AI Robert Wolfe Aylin Caliskan VLM 29 46 0 01 Jul 2022
LViT: Language meets Vision Transformer in Medical Image Segmentation Zihan Li Yunxiang Li Qingde Li Puyang Wang Dazhou Guo Le Lu D. Jin You Zhang Qingqi Hong VLM MedIm 67 134 0 29 Jun 2022
Language-Based Audio Retrieval with Converging Tied Layers and Contrastive Loss Andrew Koh Chng Eng Siong 24 1 0 29 Jun 2022
Stain Based Contrastive Co-training for Histopathological Image Analysis Bodong Zhang Beatrice Knudsen Deepika Sirohi Alessandro Ferrero Tolga Tasdizen SSL 31 5 0 24 Jun 2022
Self-Supervision on Images and Text Reduces Reliance on Visual Shortcut Features Anil Palepu Andrew L. Beam OOD VLM 26 5 0 14 Jun 2022
Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts Basil Mustafa C. Riquelme J. Puigcerver Rodolphe Jenatton N. Houlsby VLM MoE 28 185 0 06 Jun 2022
Post-hoc Concept Bottleneck Models Mert Yuksekgonul Maggie Wang James Zou 145 185 0 31 May 2022
CyCLIP: Cyclic Contrastive Language-Image Pretraining Shashank Goel Hritik Bansal S. Bhatia Ryan A. Rossi Vishwa Vinay Aditya Grover CLIP VLM 184 133 0 28 May 2022
Many-Class Text Classification with Matching Yi-Fan Song Yuxian Gu Minlie Huang VLM 22 1 0 23 May 2022
Markedness in Visual Semantic AI Robert Wolfe Aylin Caliskan VLM 30 35 0 23 May 2022
Supporting Vision-Language Model Inference with Confounder-pruning Knowledge Prompt Jiangmeng Li Wenyi Mo Jingyao Wang Bing-Huang Su Changwen Zheng Hui Xiong Ji-Rong Wen VLM 16 0 0 23 May 2022
Global Contrast Masked Autoencoders Are Powerful Pathological Representation Learners Hao Quan Xingyu Li Weixing Chen Qun Bai Mingchen Zou Ruijie Yang Tingting Zheng R. Qi Xin Gao Xiaoyu Cui MedIm 31 19 0 18 May 2022
Breaking with Fixed Set Pathology Recognition through Report-Guided Contrastive Training C. Seibold Simon Reiß M. Sarfraz Rainer Stiefelhagen Jens Kleesiek 21 31 0 14 May 2022
Multimodal Conversational AI: A Survey of Datasets and Approaches Anirudh S. Sundar Larry Heck 38 29 0 13 May 2022
Anatomy-aware Self-supervised Learning for Anomaly Detection in Chest Radiographs Junya Sato Yuki Suzuki T. Wataya Daiki Nishigaki Kosuke Kita Kazuki Yamagata Noriyuki Tomiyama Shoji Kido 18 14 0 09 May 2022
Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP) Alex Fang Gabriel Ilharco Mitchell Wortsman Yu Wan Vaishaal Shankar Achal Dave Ludwig Schmidt VLM OOD 33 139 0 03 May 2022
iCAR: Bridging Image Classification and Image-text Alignment for Visual Recognition Yixuan Wei Yue Cao Zheng-Wei Zhang Zhuliang Yao Zhenda Xie Han Hu B. Guo VLM 29 11 0 22 Apr 2022
Making the Most of Text Semantics to Improve Biomedical Vision--Language Processing Benedikt Boecking Naoto Usuyama Shruthi Bannur Daniel Coelho De Castro Anton Schwaighofer ... Tristan Naumann A. Nori Javier Alvarez-Valle Hoifung Poon Ozan Oktay 21 232 0 21 Apr 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 104 6,660 0 13 Apr 2022
MuCoT: Multilingual Contrastive Training for Question-Answering in Low-resource Languages Gokul Karthik Kumar Abhishek Singh Gehlot Sahal Shaji Mullappilly Karthik Nandakumar 34 13 0 12 Apr 2022
Unified Contrastive Learning in Image-Text-Label Space Jianwei Yang Chunyuan Li Pengchuan Zhang Bin Xiao Ce Liu Lu Yuan Jianfeng Gao VLM SSL 39 221 0 07 Apr 2022
Multi-View Transformer for 3D Visual Grounding Shijia Huang Yilun Chen Jiaya Jia Liwei Wang 31 113 0 05 Apr 2022
Interactive Audio-text Representation for Automated Audio Captioning with Contrastive Learning Chen Chen Nana Hou Yuchen Hu Heqing Zou Xiaofeng Qi Chng Eng Siong VLM 26 21 0 29 Mar 2022
AUC Maximization in the Era of Big Data and AI: A Survey Tianbao Yang Yiming Ying 44 179 0 28 Mar 2022
Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation Guanglei Yang Enrico Fini Dan Xu Paolo Rota Mingli Ding Moin Nabi Xavier Alameda-Pineda Elisa Ricci CLL 40 64 0 26 Mar 2022
Domino: Discovering Systematic Errors with Cross-Modal Embeddings Sabri Eyuboglu M. Varma Khaled Kamal Saab Jean-Benoit Delbrouck Christopher Lee-Messer Jared A. Dunnmon James Zou Christopher Ré 38 143 0 24 Mar 2022
Multi-modal learning for predicting the genotype of glioma Yiran Wei Xi Chen Lei Zhu Lipei Zhang Carola-Bibiane Schönlieb S. Price Chong Li 9 24 0 21 Mar 2022
Leveraging Visual Knowledge in Language Tasks: An Empirical Study on Intermediate Pre-training for Cross-modal Knowledge Transfer Woojeong Jin Dong-Ho Lee Chenguang Zhu Jay Pujara Xiang Ren CLIP VLM 11 9 0 14 Mar 2022
Contrastive Visual Semantic Pretraining Magnifies the Semantics of Natural Language Representations Robert Wolfe Aylin Caliskan VLM 21 13 0 14 Mar 2022
Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation Wenliang Dai Lu Hou Lifeng Shang Xin Jiang Qun Liu Pascale Fung VLM 22 90 0 12 Mar 2022
Conditional Prompt Learning for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VLM CLIP VPVLM 35 1,288 0 10 Mar 2022
Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning Weixin Liang Yuhui Zhang Yongchan Kwon Serena Yeung James Zou VLM 45 391 0 03 Mar 2022
Provable Stochastic Optimization for Global Contrastive Learning: Small Batch Does Not Harm Performance Zhuoning Yuan Yuexin Wu Zi-qi Qiu Xianzhi Du Lijun Zhang Denny Zhou Tianbao Yang 34 26 0 24 Feb 2022
GroupViT: Semantic Segmentation Emerges from Text Supervision Jiarui Xu Shalini De Mello Sifei Liu Wonmin Byeon Thomas Breuel Jan Kautz Xinyu Wang ViT VLM 192 501 0 22 Feb 2022
A Survey on Visual Transfer Learning using Knowledge Graphs Sebastian Monka Lavdim Halilaj Achim Rettinger 28 23 0 27 Jan 2022
Contrastive and Selective Hidden Embeddings for Medical Image Segmentation Zhuowei Li Zihao Liu Zhiqiang Hu Qing Xia Ruiqin Xiong Shaoting Zhang Dimitris N. Metaxas Tingting Jiang SSL 35 8 0 21 Jan 2022
TriCoLo: Trimodal Contrastive Loss for Text to Shape Retrieval Yue Ruan Han-Hung Lee Yiming Zhang Ke Zhang Angel X. Chang 32 22 0 19 Jan 2022
Weakly Supervised Contrastive Learning for Better Severity Scoring of Lung Ultrasound Gautam Rajendrakumar Gare H. V. Tran Bennett P deBoisblanc R. Rodriguez J. Galeotti 22 5 0 18 Jan 2022
MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound Rowan Zellers Jiasen Lu Ximing Lu Youngjae Yu Yanpeng Zhao Mohammadreza Salehi Aditya Kusupati Jack Hessel Ali Farhadi Yejin Choi 36 207 0 07 Jan 2022
C2-CRS: Coarse-to-Fine Contrastive Learning for Conversational Recommender System Yuanhang Zhou Kun Zhou Wayne Xin Zhao Cheng Wang Peng Jiang He Hu 27 52 0 04 Jan 2022
Does CLIP Benefit Visual Question Answering in the Medical Domain as Much as it Does in the General Domain? Sedigheh Eslami Gerard de Melo Christoph Meinel CLIP MedIm 18 116 0 27 Dec 2021
A Fistful of Words: Learning Transferable Visual Models from Bag-of-Words Supervision Ajinkya Tejankar Maziar Sanjabi Bichen Wu Saining Xie Madian Khabsa Hamed Pirsiavash Hamed Firooz VLM 29 17 0 27 Dec 2021
Extending CLIP for Category-to-image Retrieval in E-commerce Mariya Hendriksen Maurits J. R. Bleeker Svitlana Vakulenko Nanne van Noord E. Kuiper Maarten de Rijke VLM 11 30 0 21 Dec 2021
Data Efficient Language-supervised Zero-shot Recognition with Optimal Transport Distillation Bichen Wu Rui Cheng Peizhao Zhang Tianren Gao Peter Vajda Joseph E. Gonzalez VLM 22 45 0 17 Dec 2021
Technical Language Supervision for Intelligent Fault Diagnosis in Process Industry Karl Lowenmark C. Taal S. Schnabel Marcus Liwicki Fredrik Sandin 27 7 0 11 Dec 2021
Multimodal Conditional Image Synthesis with Product-of-Experts GANs Xun Huang Arun Mallya Ting-Chun Wang Xuan Li DiffM 27 90 0 09 Dec 2021
CMA-CLIP: Cross-Modality Attention CLIP for Image-Text Classification Huidong Liu Shaoyuan Xu Jinmiao Fu Yang Liu Ning Xie Chien Wang Bryan Wang Yi Sun CLIP VLM 26 27 0 07 Dec 2021