Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts

17 February 2023

Xiang Wan

Papers citing "Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts"

33 / 33 papers shown

Title
MedUnifier: Unifying Vision-and-Language Pre-training on Medical Data with Vision Generation Task using Discrete Visual Representations Ziyang Zhang Yang Yu Yucheng Chen Xulei Yang S. Yeo MedIm 56 1 0 02 Mar 2025
Enhanced Contrastive Learning with Multi-view Longitudinal Data for Chest X-ray Report Generation Kang Liu Zhuoqi Ma Xiaolu Kang Yunan Li Kun Xie Zhicheng Jiao Qiguang Miao 36 3 0 27 Feb 2025
GEMeX: A Large-Scale, Groundable, and Explainable Medical VQA Benchmark for Chest X-ray Diagnosis Bo Liu K. Zou Liming Zhan Zexin Lu Xiaoyu Dong Yidi Chen Chengqiang Xie Jiannong Cao Xiao-Ming Wu Huazhu Fu 120 0 0 25 Nov 2024
Panther: Illuminate the Sight of Multimodal LLMs with Instruction-Guided Visual Prompts Honglin Li Yuting Gao Chenglu Zhu Jingdong Chen M. Yang Lin Yang MLLM 89 0 0 21 Nov 2024
BenchX: A Unified Benchmark Framework for Medical Vision-Language Pretraining on Chest X-Rays Yang Zhou Tan Li Hui Faith Yanyu Xu Sicong Leng Xinxing Xu Yong Liu Rick Siow Mong Goh SSL VLM LM&MA MedIm 40 0 0 29 Oct 2024
CXPMRG-Bench: Pre-training and Benchmarking for X-ray Medical Report Generation on CheXpert Plus Dataset Xiao Wang Fuling Wang Yuehang Li Qingchuan Ma Shiao Wang Bo Jiang Chuanfu Li Jin Tang 37 2 0 01 Oct 2024
Advancing Medical Radiograph Representation Learning: A Hybrid Pre-training Paradigm with Multilevel Semantic Granularity Hanqi Jiang Xixuan Hao Yuzhou Huang Chong Ma Jiaxun Zhang Yi Pan Ruimao Zhang MedIm 37 0 0 01 Oct 2024
Has Multimodal Learning Delivered Universal Intelligence in Healthcare? A Comprehensive Survey Qika Lin Yifan Zhu Xin Mei Ling Huang Jingying Ma Kai He Zhen Peng Erik Cambria Mengling Feng 42 18 0 23 Aug 2024
UrFound: Towards Universal Retinal Foundation Models via Knowledge-Guided Masked Modeling Kai Yu Yang Zhou Yang Bai Z. Soh Xinxing Xu Rick Siow Mong Goh Ching-Yu Cheng Yong Liu VLM 28 1 0 10 Aug 2024
CAR-MFL: Cross-Modal Augmentation by Retrieval for Multimodal Federated Learning with Missing Modalities Pranav Poudel Prashant Shrestha Sanskar Amgain Yash Raj Shrestha P. Gyawali Binod Bhattarai 50 0 0 11 Jul 2024
Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation Pablo Messina René Vidal Denis Parra Álvaro Soto Vladimir Araujo MedIm 56 2 0 02 Jul 2024
Multimodal Data Integration for Precision Oncology: Challenges and Future Directions Huajun Zhou Fengtao Zhou Chenyu Zhao Yingxue Xu Luyang Luo Hao Chen 32 5 0 28 Jun 2024
Boosting Medical Image-based Cancer Detection via Text-guided Supervision from Reports Guangyu Guo Jiawen Yao Yingda Xia Tony C. W. Mok Zhilin Zheng Junwei Han Le Lu Dingwen Zhang Jian Zhou Ling Zhang 37 1 0 23 May 2024
Structural Entities Extraction and Patient Indications Incorporation for Chest X-ray Report Generation Kang Liu Zhuoqi Ma Xiaolu Kang Zhusi Zhong Zhicheng Jiao Grayson Baird Harrison X. Bai Qiguang Miao 41 4 0 23 May 2024
Factual Serialization Enhancement: A Key Innovation for Chest X-ray Report Generation Kang Liu Zhuoqi Ma Mengmeng Liu Zhicheng Jiao Xiaolu Kang Qiguang Miao Kun Xie MedIm 42 6 0 15 May 2024
Pre-training on High Definition X-ray Images: An Experimental Study Tianlin Li Yuehang Li Wentao Wu Jiandong Jin Yao Rong Bowei Jiang Chuanfu Li Jin Tang MedIm ViT LM&MA 41 3 0 27 Apr 2024
DeViDe: Faceted medical knowledge for improved medical vision-language pre-training Haozhe Luo Ziyu Zhou Corentin Royer Anjany Sekuboyina Bjoern H. Menze VLM ViT MedIm 48 7 0 04 Apr 2024
C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion Hee Suk Yoon Eunseop Yoon Joshua Tian Jin Tee M. Hasegawa-Johnson Yingzhen Li C. Yoo VLM 62 23 0 21 Mar 2024
UniDCP: Unifying Multiple Medical Vision-language Tasks via Dynamic Cross-modal Learnable Prompts Chenlu Zhan Yufei Zhang Yu Lin Gaoang Wang Hongwei Wang VLM MedIm 30 5 0 18 Dec 2023
BESTMVQA: A Benchmark Evaluation System for Medical Visual Question Answering Xiaojie Hong Zixin Song Liangzhi Li Xiaoli Wang Feiyan Liu 20 1 0 13 Dec 2023
Medical Vision Language Pretraining: A survey Prashant Shrestha Sanskar Amgain Bidur Khanal Cristian A. Linte Binod Bhattarai VLM 34 14 0 11 Dec 2023
Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4V Zhiling Yan Kai Zhang Rong-Er Zhou Lifang He Xiang Li Lichao Sun LM&MA 24 48 0 29 Oct 2023
Foundational Models in Medical Imaging: A Comprehensive Survey and Future Vision Bobby Azad Reza Azad Sania Eskandari Afshin Bozorgpour A. Kazerouni I. Rekik Dorit Merhof VLM MedIm 98 59 0 28 Oct 2023
Exploring Transfer Learning in Medical Image Segmentation using Vision-Language Models K. Poudel Manish Dhakal Prasiddha Bhandari Rabin Adhikari Safal Thapaliya Bishesh Khanal VLM 30 17 0 15 Aug 2023
Prompt Engineering for Healthcare: Methodologies and Applications Jiaqi Wang Enze Shi Sigang Yu Zihao Wu Chong Ma ... Dajiang Zhu Yixuan Yuan Dinggang Shen Tianming Liu Shu Zhang LM&MA 44 111 0 28 Apr 2023
Align, Reason and Learn: Enhancing Medical Vision-and-Language Pre-training with Knowledge Zhihong Chen Guanbin Li Xiang Wan 124 65 0 15 Sep 2022
Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Training Zhihong Chen Yu Du Jinpeng Hu Yang Liu Guanbin Li Xiang Wan Tsung-Hui Chang 86 111 0 15 Sep 2022
Joint Learning of Localized Representations from Medical Images and Reports Philipp Muller Georgios Kaissis Cong Zou Daniel Munich 137 81 0 06 Dec 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 298 3,700 0 11 Feb 2021
Unifying Vision-and-Language Tasks via Text Generation Jaemin Cho Jie Lei Hao Tan Joey Tianyi Zhou MLLM 256 525 0 04 Feb 2021
Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation Yasuhide Miura Yuhao Zhang Emily Bao Tsai C. Langlotz Dan Jurafsky MedIm 151 156 0 20 Oct 2020
Text Summarization with Pretrained Encoders Yang Liu Mirella Lapata MILM 258 1,432 0 22 Aug 2019
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,743 0 26 Sep 2016