Contrastive Learning of Medical Visual Representations from Paired Images and Text

2 October 2020

Yuhao Zhang

Hang Jiang

Yasuhide Miura

Christopher D. Manning

C. Langlotz

MedIm

ArXiv PDF HTML

Papers citing "Contrastive Learning of Medical Visual Representations from Paired Images and Text"

50 / 445 papers shown

Title
Dynamic Multi-Domain Knowledge Networks for Chest X-ray Report Generation Weihua Liu Youyuan Xue Chaochao Lin Said Boumaraf MedIm 15 0 0 08 Oct 2023
Continual Contrastive Spoken Language Understanding Umberto Cappellazzo Enrico Fini Muqiao Yang Daniele Falavigna A. Brutti Bhiksha Raj CLL 28 1 0 04 Oct 2023
Understanding Transferable Representation Learning and Zero-shot Transfer in CLIP Zixiang Chen Yihe Deng Yuanzhi Li Quanquan Gu VLM 28 11 0 02 Oct 2023
PRIME: Prioritizing Interpretability in Failure Mode Extraction Keivan Rezaei Mehrdad Saberi Mazda Moayeri S. Feizi 25 8 0 29 Sep 2023
CWCL: Cross-Modal Transfer with Continuously Weighted Contrastive Loss R. S. Srinivasa Jaejin Cho Chouchang Yang Yashas Malur Saidutta Ching Hua Lee Yilin Shen Hongxia Jin VLM 36 8 0 26 Sep 2023
Exploiting CLIP-based Multi-modal Approach for Artwork Classification and Retrieval Alberto Baldrati Marco Bertini Tiberio Uricchio A. Bimbo CLIP VLM 28 1 0 21 Sep 2023
MOSAIC: Learning Unified Multi-Sensory Object Property Representations for Robot Learning via Interactive Perception Gyan Tatiya Jonathan M Francis Ho-Hsiang Wu Yonatan Bisk Jivko Sinapov 31 1 0 15 Sep 2023
UniBrain: Universal Brain MRI Diagnosis with Hierarchical Knowledge-enhanced Pre-training Jiayu Lei Lisong Dai Haoyun Jiang Chaoyi Wu Xiaoman Zhang ... Weidi Xie Yanyong Zhang Yuehua Li Ya Zhang Yanfeng Wang 27 10 0 13 Sep 2023
Enhancing Representation in Radiography-Reports Foundation Model: A Granular Alignment Algorithm Using Masked Contrastive Learning Weijian Huang Cheng Li Hao Yang Jiarun Liu Shanshan Wang MedIm 31 20 0 12 Sep 2023
InstructDiffusion: A Generalist Modeling Interface for Vision Tasks Zigang Geng Binxin Yang Tiankai Hang Chen Li Shuyang Gu ... Jianmin Bao Zheng-Wei Zhang Han Hu Dongdong Chen Baining Guo DiffM VLM 53 93 0 07 Sep 2023
Distribution-Aware Prompt Tuning for Vision-Language Models Eulrang Cho Jooyeon Kim Hyunwoo J. Kim VPVLM VLM 38 21 0 06 Sep 2023
An Empirical Analysis for Zero-Shot Multi-Label Classification on COVID-19 CT Scans and Uncurated Reports Ethan Dack Lorenzo Brigato Matthew McMurray Matthias Fontanellaz Thomas Frauenfelder ... Thomas Geiser M. Funke-Chambour Andreas Christe L. Ebner S. Mougiakakou 25 2 0 04 Sep 2023
Unified Pre-training with Pseudo Texts for Text-To-Image Person Re-identification Zhiyin Shao Xinyu Zhang Changxing Ding Jian Wang Jingdong Wang 31 17 0 04 Sep 2023
Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following Ziyu Guo Renrui Zhang Xiangyang Zhu Yiwen Tang Xianzheng Ma ... Ke Chen Peng Gao Xianzhi Li Hongsheng Li Pheng-Ann Heng MLLM 35 126 0 01 Sep 2023
Learning multi-modal generative models with permutation-invariant encoders and tighter variational bounds Marcel Hirt Domenico Campolo Victoria Leong Juan-Pablo Ortega DRL 10 0 0 01 Sep 2023
Exploring the Transfer Learning Capabilities of CLIP in Domain Generalization for Diabetic Retinopathy Sanoojan Baliah F. Maani Santosh Sanjeev M. H. Khan 48 8 0 27 Aug 2023
PECon: Contrastive Pretraining to Enhance Feature Alignment between CT and EHR Data for Improved Pulmonary Embolism Diagnosis Santosh Sanjeev Salwa K. Al Khatib Mai A. Shaaban Ibrahim Almakky Vijay Ram Papineni Mohammad Yaqub 28 1 0 27 Aug 2023
ViLLA: Fine-Grained Vision-Language Representation Learning from Real-World Data M. Varma Jean-Benoit Delbrouck Sarah Hooper Akshay S. Chaudhari C. Langlotz VLM CoGe 40 5 0 22 Aug 2023
A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervision Julio Silva-Rodríguez H. Chakor Riadh Kobbi Jose Dolz Ismail Ben Ayed VLM MedIm 74 35 0 15 Aug 2023
Exploring Transfer Learning in Medical Image Segmentation using Vision-Language Models K. Poudel Manish Dhakal Prasiddha Bhandari Rabin Adhikari Safal Thapaliya Bishesh Khanal VLM 30 17 0 15 Aug 2023
Seeing in Flowing: Adapting CLIP for Action Recognition with Motion Prompts Learning Qianqian Wang Junlong Du Ke Yan Shouhong Ding VLM 38 17 0 09 Aug 2023
ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoders Shawn Xu L. Yang Christopher J. Kelly M. Sieniek Timo Kohlberger ... Shruthi Prabhakara Daniel Golden Rory Pilgrim Krish Eswaran Andrew Sellergren LM&MA MedIm 31 51 0 02 Aug 2023
Sat2Cap: Mapping Fine-Grained Textual Descriptions from Satellite Images A. Dhakal Adeel Ahmad Subash Khanal S. Sastry Hannah Kerner Nathan Jacobs 25 13 0 29 Jul 2023
Towards a Visual-Language Foundation Model for Computational Pathology Ming Y. Lu Bowen Chen Drew F. K. Williamson Richard J. Chen Ivy Liang ... Andrew Zhang L. Le Georg Gerber Anil V. Parwani Faisal Mahmood VLM MedIm 42 46 0 24 Jul 2023
PRIOR: Prototype Representation Joint Learning from Medical Images and Reports Pujin Cheng Li Lin Junyan Lyu Yijin Huang Wenhan Luo Xiaoying Tang MedIm 37 45 0 24 Jul 2023
Learning Navigational Visual Representations with Semantic Map Supervision Yicong Hong Yang Zhou Ruiyi Zhang Franck Dernoncourt Trung Bui Stephen Gould Hao Tan SSL 30 21 0 23 Jul 2023
Identifying Interpretable Subspaces in Image Representations Neha Kalibhat S. Bhardwaj Bayan Bruss Hamed Firooz Maziar Sanjabi S. Feizi FAtt 42 26 0 20 Jul 2023
M-FLAG: Medical Vision-Language Pre-training with Frozen Language Models and Latent Space Geometry Optimization Che Liu Sibo Cheng Chong Chen Mengyun Qiao Weitong Zhang Anand Shah Wenjia Bai Rossella Arcucci VLM 30 56 0 17 Jul 2023
A scoping review on multimodal deep learning in biomedical images and texts Zhaoyi Sun Mingquan Lin Qingqing Zhu Qianqian Xie Fei Wang Zhiyong Lu Yifan Peng 31 18 0 14 Jul 2023
Knowledge Boosting: Rethinking Medical Contrastive Vision-Language Pre-Training Xiaofei Chen Yuting He Cheng Xue Rongjun Ge Shuo Li Guanyu Yang VLM MedIm 29 13 0 14 Jul 2023
Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution Mostafa Dehghani Basil Mustafa Josip Djolonga Jonathan Heek Matthias Minderer ... Avital Oliver Piotr Padlewski A. Gritsenko Mario Luvcić N. Houlsby ViT 26 105 0 12 Jul 2023
Unified Medical Image-Text-Label Contrastive Learning With Continuous Prompt Yuhao Wang VLM MedIm 30 1 0 12 Jul 2023
FILM: How can Few-Shot Image Classification Benefit from Pre-Trained Language Models? Zihao Jiang Yunkai Dang Dong Pang Huishuai Zhang Weiran Huang VLM 44 2 0 09 Jul 2023
Dental CLAIRES: Contrastive LAnguage Image REtrieval Search for Dental Research Tanjida Kabir Luyao Chen Muhammad Walji Luca Giancardo Xiaoqian Jiang Shayan Shams 8 0 0 27 Jun 2023
Quilt-1M: One Million Image-Text Pairs for Histopathology Wisdom O. Ikezogwo M. S. Seyfioglu Fatemeh Ghezloo Dylan Stefan Chan Geva Fatwir Sheikh Mohammed Pavan Kumar Anand Ranjay Krishna Linda G. Shapiro CLIP VLM 141 114 0 20 Jun 2023
RemoteCLIP: A Vision Language Foundation Model for Remote Sensing F. Liu Delong Chen Zhan-Rong Guan Xiaocong Zhou Jiale Zhu Qiaolin Ye Liyong Fu Jun Zhou VLM 71 192 0 19 Jun 2023
Generation of Radiology Findings in Chest X-Ray by Leveraging Collaborative Knowledge Manuela Danu George Marica Sanjeev Kumar Karn Bogdan Georgescu Awais Mansoor ... Lucian Mihai Itu C. Suciu Sasa Grbic Oladimeji Farri Dorin Comaniciu MedIm 23 8 0 18 Jun 2023
M3PT: A Multi-Modal Model for POI Tagging Jingsong Yang Guanzhou Han Deqing Yang Jingping Liu Yanghua Xiao Xiang Xu Baohua Wu Shenghua Ni 50 3 0 16 Jun 2023
Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Images Ming Y. Lu Bowen Chen Andrew Zhang Drew F. K. Williamson Richard J. Chen Tong Ding L. Le Yung-Sung Chuang Faisal Mahmood VLM MedIm 30 100 0 13 Jun 2023
Retrieval-Enhanced Contrastive Vision-Text Models Ahmet Iscen Mathilde Caron Alireza Fathi Cordelia Schmid CLIP VLM 31 26 0 12 Jun 2023
Augmenting Zero-Shot Detection Training with Image Labels Katharina Kornmeier Ulla Scheler P. Herrmann ObjD VLM 19 1 0 12 Jun 2023
Multi-modal Pre-training for Medical Vision-language Understanding and Generation: An Empirical Study with A New Benchmark Li Xu Bo Liu Ameer Hamza Khan Lu Fan Xiao-Ming Wu LM&MA 35 9 0 10 Jun 2023
On the Challenges and Perspectives of Foundation Models for Medical Image Analysis Shaoting Zhang Dimitris N. Metaxas LM&MA VLM MedIm AI4CE 42 128 0 09 Jun 2023
Generative Text-Guided 3D Vision-Language Pretraining for Unified Medical Image Segmentation Yinda Chen Che Liu Wei Huang Sibo Cheng Rossella Arcucci Zhiwei Xiong VLM MedIm 34 48 0 07 Jun 2023
On the Generalization of Multi-modal Contrastive Learning Qi Zhang Yifei Wang Yisen Wang 22 24 0 07 Jun 2023
Exploring the Versatility of Zero-Shot CLIP for Interstitial Lung Disease Classification Cara Van Uden Christian Blüthgen Maayane Attias M. Polacin H. Guo N. Simha Rishi Raj C. Langlotz VLM 14 2 0 01 Jun 2023
Continual Learning for Abdominal Multi-Organ and Tumor Segmentation Yixiao Zhang Xinyi Li Huimiao Chen Alan Yuille Yaoyao Liu Zongwei Zhou CLL 24 33 0 01 Jun 2023
Additional Positive Enables Better Representation Learning for Medical Images Dewen Zeng Yawen Wu Xinrong Hu Xiaowei Xu Jingtong Hu Yiyu Shi SSL 19 1 0 31 May 2023
Med-UniC: Unifying Cross-Lingual Medical Vision-Language Pre-Training by Diminishing Bias Zhongwei Wan Che Liu Mi Zhang Jie Fu Benyou Wang Sibo Cheng Lei Ma César Quilodrán-Casas Rossella Arcucci 50 71 0 31 May 2023
BiomedGPT: A Unified and Generalist Biomedical Generative Pre-trained Transformer for Vision, Language, and Multimodal Tasks Kai Zhang Jun Yu Eashan Adhikarla Rong-Er Zhou Zhilin Yan ... Xun Chen Yong Chen Quanzheng Li Hongfang Liu Lichao Sun LM&MA MedIm 37 157 0 26 May 2023