Light-weight Fine-tuning Method for Defending Adversarial Noise in Pre-trained Medical Vision-Language Models

2 July 2024

Xuezhe Ma

Papers citing "Light-weight Fine-tuning Method for Defending Adversarial Noise in Pre-trained Medical Vision-Language Models"

39 / 39 papers shown

Title
BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs Sheng Zhang Yanbo Xu Naoto Usuyama Hanwen Xu J. Bagga ... Carlo Bifulco M. Lungren Tristan Naumann Sheng Wang Hoifung Poon LM&MA MedIm 182 222 0 10 Jan 2025
Is Synthetic Data all We Need? Benchmarking the Robustness of Models Trained with Synthetic Images Krishnakant Singh Thanush Navaratnam Jannik Holmer Simone Schaub-Meyer Stefan Roth DiffM 63 18 0 30 May 2024
ROCOv2: Radiology Objects in COntext Version 2, an Updated Multimodal Image Dataset Johannes Ruckert Louise Bloch Raphael Brüngel Ahmad Idrissi-Yaghir Henning Schafer ... A. G. S. D. Herrera Henning Müller Peter A. Horn F. Nensa Christoph M. Friedrich 57 30 0 16 May 2024
Revisiting the Adversarial Robustness of Vision Language Models: a Multimodal Perspective Wanqi Zhou Shuanghao Bai Qibin Zhao Badong Chen VLM AAML 80 9 0 30 Apr 2024
Understanding the Effect of Noise in LLM Training Data with Algorithmic Chains of Thought Alex Havrilla Maia Iyer 33 8 0 06 Feb 2024
Demonstration of an Adversarial Attack Against a Multimodal Vision Language Model for Pathology Imaging Poojitha Thota Jai Prakash Veerla Partha Sai Guttikonda M. Nasr Shirin Nilizadeh Jacob M. Luber AAML 51 8 0 04 Jan 2024
On the Robustness of Large Multimodal Models Against Image Adversarial Attacks Xuanimng Cui Alejandro Aparcedo Young Kyun Jang Ser-Nam Lim AAML VLM 41 43 0 06 Dec 2023
Generating Medical Prescriptions with Conditional Transformer Samuel Belkadi Nicolo Micheletti Lifeng Han Warren Del-Pinto Goran Nenadic MedIm 42 5 0 30 Oct 2023
An LLM can Fool Itself: A Prompt-Based Adversarial Attack Xilie Xu Keyi Kong Ning Liu Li-zhen Cui Di Wang Jingfeng Zhang Mohan Kankanhalli AAML SILM 74 82 0 20 Oct 2023
VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Models Ziyi Yin Muchao Ye Tianrong Zhang Tianyu Du Jinguo Zhu Han Liu Jinghui Chen Ting Wang Fenglong Ma AAML VLM CoGe 59 41 0 07 Oct 2023
Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks Hao Chen Jindong Wang Ankit Shah Ran Tao Hongxin Wei Berfin cSimcsek Masashi Sugiyama Bhiksha Raj 60 28 0 29 Sep 2023
LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day Chunyuan Li Cliff Wong Sheng Zhang Naoto Usuyama Haotian Liu Jianwei Yang Tristan Naumann Hoifung Poon Jianfeng Gao LM&MA MedIm 95 760 0 01 Jun 2023
Machine Learning for Synthetic Data Generation: A Review Ying-Cheng Lu Minjie Shen Huazheng Wang Xiao Wang Capucine Van Rechem Tianfan Fu Wenqi Wei SyDa 70 143 0 08 Feb 2023
Multimodal Deep Learning Cem Akkus Jiquan Ngiam Vladana Djakovic Steffen Jauch-Walser A. Khosla ... Jann Goschenhofer Honglak Lee A. Ng Daniel Schalk Matthias Aßenmacher 81 3,166 0 12 Jan 2023
Reproducible scaling laws for contrastive language-image learning Mehdi Cherti Romain Beaumont Ross Wightman Mitchell Wortsman Gabriel Ilharco Cade Gordon Christoph Schuhmann Ludwig Schmidt J. Jitsev VLM CLIP 107 794 0 14 Dec 2022
Unsupervised Domain Adaptation for Segmentation with Black-box Source Model Xiaofeng Liu Chaehwa Yoo Fangxu Xing C.-C. Jay Kuo Xiaofeng Liu Jonghye Woo 57 16 0 16 Aug 2022
Guided Diffusion Model for Adversarial Purification Jinyi Wang Zhaoyang Lyu Dahua Lin Bo Dai Hongfei Fu DiffM 224 86 0 30 May 2022
Robust Medical Image Classification from Noisy Labeled Data with Global and Local Representation Guided Co-training Cheng Xue Lequan Yu Pengfei Chen Qi Dou Pheng-Ann Heng NoLa 37 52 0 10 May 2022
NoisyTune: A Little Noise Can Help You Finetune Pretrained Language Models Better Chuhan Wu Fangzhao Wu Tao Qi Yongfeng Huang Xing Xie 32 60 0 24 Feb 2022
Tightening the Approximation Error of Adversarial Risk with Auto Loss Function Search Pengfei Xia Ziqiang Li Bin Li AAML 80 3 0 09 Nov 2021
An Empirical Study of Training End-to-End Vision-and-Language Transformers Zi-Yi Dou Yichong Xu Zhe Gan Jianfeng Wang Shuohang Wang ... Pengchuan Zhang Lu Yuan Nanyun Peng Zicheng Liu Michael Zeng VLM 61 376 0 03 Nov 2021
Multi-modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Training Jong Hak Moon HyunGyung Lee W. Shin Young-Hak Kim Edward Choi MedIm 47 157 0 24 May 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 786 29,167 0 26 Feb 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 349 4,918 0 24 Feb 2021
SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answering Bo Liu Li-Ming Zhan Li Xu Lin Ma Y. Yang Xiao-Ming Wu 68 259 0 18 Feb 2021
Recent Advances in Adversarial Training for Adversarial Robustness Tao Bai Jinqi Luo Jun Zhao Bihan Wen Qian Wang AAML 113 489 0 02 Feb 2021
To be Robust or to be Fair: Towards Fairness in Adversarial Training Han Xu Xiaorui Liu Yaxin Li Anil K. Jain Jiliang Tang 41 180 0 13 Oct 2020
MedICaT: A Dataset of Medical Images, Captions, and Textual References Sanjay Subramanian Lucy Lu Wang Sachin Mehta Ben Bogin Madeleine van Zuylen Sravanthi Parasa Sameer Singh Matt Gardner Hannaneh Hajishirzi MedIm 37 70 0 12 Oct 2020
Normalized Loss Functions for Deep Learning with Noisy Labels Xingjun Ma Hanxun Huang Yisen Wang Simone Romano S. Erfani James Bailey NoLa 59 439 0 24 Jun 2020
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Hao Hao Tan Joey Tianyi Zhou VLM MLLM 227 2,474 0 20 Aug 2019
Are Anchor Points Really Indispensable in Label-Noise Learning? Xiaobo Xia Tongliang Liu N. Wang Bo Han Chen Gong Gang Niu Masashi Sugiyama NoLa 62 377 0 01 Jun 2019
Adversarial Training for Free! Ali Shafahi Mahyar Najibi Amin Ghiasi Zheng Xu John P. Dickerson Christoph Studer L. Davis Gavin Taylor Tom Goldstein AAML 119 1,242 0 29 Apr 2019
Theoretically Principled Trade-off between Robustness and Accuracy Hongyang R. Zhang Yaodong Yu Jiantao Jiao Eric Xing L. Ghaoui Michael I. Jordan 118 2,541 0 24 Jan 2019
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison Jeremy Irvin Pranav Rajpurkar M. Ko Yifan Yu Silviana Ciurea-Ilcus ... D. Larson C. Langlotz Bhavik Patel M. Lungren A. Ng 110 2,583 0 21 Jan 2019
Hyperspectral Image Classification in the Presence of Noisy Labels Junjun Jiang Jiayi Ma Zheng Wang Chen Chen Xianming Liu NoLa 50 161 0 12 Sep 2018
Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise Dan Hendrycks Mantas Mazeika Duncan Wilson Kevin Gimpel NoLa 127 553 0 14 Feb 2018
Robust Loss Functions under Label Noise for Deep Neural Networks Aritra Ghosh Himanshu Kumar P. Sastry NoLa OOD 61 955 0 27 Dec 2017
Towards Deep Learning Models Resistant to Adversarial Attacks Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras Adrian Vladu SILM OOD 255 12,029 0 19 Jun 2017
ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases Xiaosong Wang Yifan Peng Le Lu Zhiyong Lu M. Bagheri Ronald M. Summers LM&MA 135 2,521 0 05 May 2017