v1v2 (latest)

Contrastive Learning of Medical Visual Representations from Paired Images and Text

2 October 2020

Yuhao Zhang

Hang Jiang

Yasuhide Miura

Christopher D. Manning

C. Langlotz

MedIm

ArXiv (abs)PDF HTML

Papers citing "Contrastive Learning of Medical Visual Representations from Paired Images and Text"

50 / 459 papers shown

Title
Detailed Annotations of Chest X-Rays via CT Projection for Report Understanding C. Seibold Simon Reiß Saquib Sarfraz M. Fink Victoria L. Mayer Jan Sellner Moon S. Kim Klaus H. Maier-Hein Jens Kleesiek Rainer Stiefelhagen 106 19 0 07 Oct 2022
When and why vision-language models behave like bags-of-words, and what to do about it? Mert Yuksekgonul Federico Bianchi Pratyusha Kalluri Dan Jurafsky James Zou VLM CoGe 152 394 0 04 Oct 2022
Medical Image Understanding with Pretrained Vision Language Models: A Comprehensive Study Ziyuan Qin Huahui Yi Qicheng Lao Kang Li VLM 103 71 0 30 Sep 2022
Domain-aware Self-supervised Pre-training for Label-Efficient Meme Analysis Shivam Sharma Mohd Khizir Siddiqui Md. Shad Akhtar Tanmoy Chakraborty SSL 38 5 0 29 Sep 2022
A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective Chaoqi Chen Yushuang Wu Qiyuan Dai Hong-Yu Zhou Mutian Xu Sibei Yang Xiaoguang Han Yizhou Yu ViT MedIm AI4CE 137 80 0 27 Sep 2022
RepsNet: Combining Vision with Language for Automated Medical Reports A. Tanwani Joelle Barral Daniel Freedman MedIm 85 23 0 27 Sep 2022
Contrastive learning for unsupervised medical image clustering and reconstruction Matteo Ferrante T. Boccato Simeon E. Spasov A. Duggento N. Toschi SSL DRL 68 2 0 24 Sep 2022
FETA: Towards Specializing Foundation Models for Expert Task Applications Amit Alfassy Assaf Arbelle Oshri Halimi Sivan Harary Roei Herzig ... Christoph Auer Kate Saenko Peter W. J. Staar Rogerio Feris Leonid Karlinsky 90 20 0 08 Sep 2022
Real-Time Cattle Interaction Recognition via Triple-stream Network Yang Yang Mizuka Komatsu K. Oyama T. Ohkawa 42 3 0 06 Sep 2022
Disentangle and Remerge: Interventional Knowledge Distillation for Few-Shot Object Detection from A Conditional Causal Perspective Jiangmeng Li Yanan Zhang Jingyao Wang Hui Xiong Chengbo Jiao Xiaohui Hu Changwen Zheng Gang Hua CML 112 30 0 26 Aug 2022
CMSBERT-CLR: Context-driven Modality Shifting BERT with Contrastive Learning for linguistic, visual, acoustic Representations Junghun Kim Jihie Kim 38 2 0 21 Aug 2022
Dual Modality Prompt Tuning for Vision-Language Pre-Trained Model Yinghui Xing Qirui Wu De Cheng Shizhou Zhang Guoqiang Liang Peng Wang Yanning Zhang VLM VPVLM 144 59 0 17 Aug 2022
Towards Open-vocabulary Scene Graph Generation with Prompt-based Finetuning Tao He Lianli Gao Jingkuan Song Yuan-Fang Li VLM 88 53 0 17 Aug 2022
Self-supervised Multi-modal Training from Uncurated Image and Reports Enables Zero-shot Oversight Artificial Intelligence in Radiology Sangjoon Park Eunha Lee Kyung Sook Shin Jeonghyeon Lee Jong Chul Ye 53 2 0 10 Aug 2022
RadTex: Learning Efficient Radiograph Representations from Text Reports Keegan Quigley Miriam Cha Ruizhi Liao Geeticka Chauhan Steven Horng Seth Berkowitz Polina Golland MedIm 52 3 0 05 Aug 2022
NewsStories: Illustrating articles with visual summaries Reuben Tan Bryan A. Plummer Kate Saenko J. P. Lewis Avneesh Sud Thomas Leung VLM SSL 133 5 0 26 Jul 2022
Unimodal vs. Multimodal Siamese Networks for Outfit Completion Mariya Hendriksen Viggo Overes 55 1 0 21 Jul 2022
Contrastive Adapters for Foundation Model Group Robustness Michael Zhang Christopher Ré VLM 54 64 0 14 Jul 2022
Text-driven Emotional Style Control and Cross-speaker Style Transfer in Neural TTS Yookyung Shin Younggun Lee Suhee Jo Yeongtae Hwang Taesu Kim 100 14 0 13 Jul 2022
American == White in Multimodal Language-and-Image AI Robert Wolfe Aylin Caliskan VLM 85 51 0 01 Jul 2022
LViT: Language meets Vision Transformer in Medical Image Segmentation Zihan Li Yunxiang Li Qingde Li Puyang Wang Dazhou Guo Le Lu D. Jin You Zhang Qingqi Hong VLM MedIm 115 141 0 29 Jun 2022
Language-Based Audio Retrieval with Converging Tied Layers and Contrastive Loss Andrew Koh Chng Eng Siong 144 1 0 29 Jun 2022
Stain Based Contrastive Co-training for Histopathological Image Analysis Bodong Zhang Beatrice Knudsen Deepika Sirohi Alessandro Ferrero Tolga Tasdizen SSL 92 5 0 24 Jun 2022
Self-Supervision on Images and Text Reduces Reliance on Visual Shortcut Features Anil Palepu Andrew L. Beam OOD VLM 51 5 0 14 Jun 2022
Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts Basil Mustafa C. Riquelme J. Puigcerver Rodolphe Jenatton N. Houlsby VLM MoE 170 205 0 06 Jun 2022
Post-hoc Concept Bottleneck Models Mert Yuksekgonul Maggie Wang James Zou 245 198 0 31 May 2022
CyCLIP: Cyclic Contrastive Language-Image Pretraining Shashank Goel Hritik Bansal S. Bhatia Ryan Rossi Vishwa Vinay Aditya Grover CLIP VLM 280 140 0 28 May 2022
Many-Class Text Classification with Matching Yi-Fan Song Yuxian Gu Minlie Huang VLM 29 1 0 23 May 2022
Markedness in Visual Semantic AI Robert Wolfe Aylin Caliskan VLM 107 36 0 23 May 2022
Supporting Vision-Language Model Inference with Confounder-pruning Knowledge Prompt Jiangmeng Li Wenyi Mo Jingyao Wang Fuchun Sun Changwen Zheng Hui Xiong Ji-Rong Wen VLM 86 0 0 23 May 2022
Global Contrast Masked Autoencoders Are Powerful Pathological Representation Learners Hao Quan Xingyu Li Weixing Chen Qun Bai Mingchen Zou Ruijie Yang Tingting Zheng R. Qi Xin Gao Xiaoyu Cui MedIm 110 21 0 18 May 2022
Breaking with Fixed Set Pathology Recognition through Report-Guided Contrastive Training C. Seibold Simon Reiß M. Sarfraz Rainer Stiefelhagen Jens Kleesiek 53 33 0 14 May 2022
Multimodal Conversational AI: A Survey of Datasets and Approaches Anirudh S. Sundar Larry Heck 102 30 0 13 May 2022
Anatomy-aware Self-supervised Learning for Anomaly Detection in Chest Radiographs Junya Sato Yuki Suzuki T. Wataya Daiki Nishigaki Kosuke Kita Kazuki Yamagata Noriyuki Tomiyama Shoji Kido 56 14 0 09 May 2022
Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP) Alex Fang Gabriel Ilharco Mitchell Wortsman Yu Wan Vaishaal Shankar Achal Dave Ludwig Schmidt VLM OOD 108 149 0 03 May 2022
iCAR: Bridging Image Classification and Image-text Alignment for Visual Recognition Yixuan Wei Yue Cao Zheng Zhang Zhuliang Yao Zhenda Xie Han Hu B. Guo VLM 59 11 0 22 Apr 2022
Making the Most of Text Semantics to Improve Biomedical Vision--Language Processing Benedikt Boecking Naoto Usuyama Shruthi Bannur Daniel Coelho De Castro Anton Schwaighofer ... Tristan Naumann A. Nori Javier Alvarez-Valle Hoifung Poon Ozan Oktay 89 247 0 21 Apr 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 514 6,941 0 13 Apr 2022
MuCoT: Multilingual Contrastive Training for Question-Answering in Low-resource Languages Gokul Karthik Kumar Abhishek Singh Gehlot Sahal Shaji Mullappilly Karthik Nandakumar 83 13 0 12 Apr 2022
Unified Contrastive Learning in Image-Text-Label Space Jianwei Yang Chunyuan Li Pengchuan Zhang Bin Xiao Ce Liu Lu Yuan Jianfeng Gao VLM SSL 148 227 0 07 Apr 2022
Multi-View Transformer for 3D Visual Grounding Shijia Huang Yilun Chen Jiaya Jia Liwei Wang 106 127 0 05 Apr 2022
Interactive Audio-text Representation for Automated Audio Captioning with Contrastive Learning Chen Chen Nana Hou Yuchen Hu Heqing Zou Xiaofeng Qi Chng Eng Siong VLM 84 21 0 29 Mar 2022
AUC Maximization in the Era of Big Data and AI: A Survey Tianbao Yang Yiming Ying 181 188 0 28 Mar 2022
Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation Guanglei Yang Enrico Fini Dan Xu Paolo Rota Mingli Ding Moin Nabi Xavier Alameda-Pineda Elisa Ricci CLL 82 67 0 26 Mar 2022
Domino: Discovering Systematic Errors with Cross-Modal Embeddings Sabri Eyuboglu M. Varma Khaled Kamal Saab Jean-Benoit Delbrouck Christopher Lee-Messer Jared A. Dunnmon James Zou Christopher Ré 115 148 0 24 Mar 2022
Multi-modal learning for predicting the genotype of glioma Yiran Wei Xi Chen Lei Zhu Lipei Zhang Carola-Bibiane Schönlieb S. Price Chong Li 67 26 0 21 Mar 2022
Leveraging Visual Knowledge in Language Tasks: An Empirical Study on Intermediate Pre-training for Cross-modal Knowledge Transfer Woojeong Jin Dong-Ho Lee Chenguang Zhu Jay Pujara Xiang Ren CLIP VLM 75 10 0 14 Mar 2022
Contrastive Visual Semantic Pretraining Magnifies the Semantics of Natural Language Representations Robert Wolfe Aylin Caliskan VLM 67 14 0 14 Mar 2022
Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation Wenliang Dai Lu Hou Lifeng Shang Xin Jiang Qun Liu Pascale Fung VLM 92 94 0 12 Mar 2022
Conditional Prompt Learning for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VLM CLIP VPVLM 161 1,362 0 10 Mar 2022