Robust-LLaVA: On the Effectiveness of Large-Scale Robust Image Encoders for Multi-modal Large Language Models

3 February 2025

Papers citing "Robust-LLaVA: On the Effectiveness of Large-Scale Robust Image Encoders for Multi-modal Large Language Models"

34 / 34 papers shown

Title
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders Min Shi Fuxiao Liu Shihao Wang Shijia Liao Subhashree Radhakrishnan ... Andrew Tao Andrew Tao Zhiding Yu Guilin Liu Guilin Liu MLLM 96 65 0 28 Aug 2024
Hallucination of Multimodal Large Language Models: A Survey Zechen Bai Pichao Wang Tianjun Xiao Tong He Zongbo Han Zheng Zhang Mike Zheng Shou VLM LRM 151 177 0 29 Apr 2024
Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models Yifan Li Hangyu Guo Kun Zhou Wayne Xin Zhao Ji-Rong Wen 94 49 0 14 Mar 2024
Visual Hallucinations of Multi-modal Large Language Models Wen Huang Hongbin Liu Minxin Guo Neil Zhenqiang Gong MLLM VLM 64 32 0 22 Feb 2024
Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models Christian Schlarmann Naman D. Singh Francesco Croce Matthias Hein VLM AAML 73 46 0 19 Feb 2024
Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs Shengbang Tong Zhuang Liu Yuexiang Zhai Yi-An Ma Yann LeCun Saining Xie VLM MLLM 79 325 0 11 Jan 2024
FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts Yichen Gong Delong Ran Jinyuan Liu Conglei Wang Tianshuo Cong Anyu Wang Sisi Duan Xiaoyun Wang MLLM 205 147 0 09 Nov 2023
Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity Cunxiang Wang Xiaoze Liu Yuanhao Yue Xiangru Tang Tianhang Zhang ... Linyi Yang Jindong Wang Xing Xie Zheng Zhang Yue Zhang HILM KELM 90 197 0 11 Oct 2023
Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models Erfan Shayegani Yue Dong Nael B. Abu-Ghazaleh 85 145 0 26 Jul 2023
Foundational Models Defining a New Era in Vision: A Survey and Outlook Muhammad Awais Muzammal Naseer Salman Khan Rao Muhammad Anwer Hisham Cholakkal M. Shah Ming-Hsuan Yang Fahad Shahbaz Khan VLM 110 124 0 25 Jul 2023
Visual Adversarial Examples Jailbreak Aligned Large Language Models Xiangyu Qi Kaixuan Huang Ashwinee Panda Peter Henderson Mengdi Wang Prateek Mittal AAML 76 156 0 22 Jun 2023
On Evaluating Adversarial Robustness of Large Vision-Language Models Yunqing Zhao Tianyu Pang Chao Du Xiao Yang Chongxuan Li Ngai-Man Cheung Min Lin VLM AAML MLLM 90 175 0 26 May 2023
Text-To-Concept (and Back) via Cross-Model Alignment Mazda Moayeri Keivan Rezaei Maziar Sanjabi Soheil Feizi CLIP 48 44 0 10 May 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 441 4,715 0 17 Apr 2023
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 830 29,341 0 26 Feb 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 559 40,961 0 22 Oct 2020
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models Samuel Gehman Suchin Gururangan Maarten Sap Yejin Choi Noah A. Smith 133 1,194 0 24 Sep 2020
Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks Francesco Croce Matthias Hein AAML 213 1,842 0 03 Mar 2020
Towards VQA Models That Can Read Amanpreet Singh Vivek Natarajan Meet Shah Yu Jiang Xinlei Chen Dhruv Batra Devi Parikh Marcus Rohrbach EgoV 71 1,215 0 18 Apr 2019
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations Dan Hendrycks Thomas G. Dietterich OOD VLM 167 3,429 0 28 Mar 2019
Using Pre-Training Can Improve Model Robustness and Uncertainty Dan Hendrycks Kimin Lee Mantas Mazeika NoLa 69 727 0 28 Jan 2019
Disentangling Adversarial Robustness and Generalization David Stutz Matthias Hein Bernt Schiele AAML OOD 251 280 0 03 Dec 2018
Adversarially Robust Generalization Requires More Data Ludwig Schmidt Shibani Santurkar Dimitris Tsipras Kunal Talwar Aleksander Madry OOD AAML 131 790 0 30 Apr 2018
VizWiz Grand Challenge: Answering Visual Questions from Blind People Danna Gurari Qing Li Abigale Stangl Anhong Guo Chi Lin Kristen Grauman Jiebo Luo Jeffrey P. Bigham CoGe 82 847 0 22 Feb 2018
Towards Deep Learning Models Resistant to Adversarial Attacks Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras Adrian Vladu SILM OOD 287 12,060 0 19 Jun 2017
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering Yash Goyal Tejas Khot D. Summers-Stay Dhruv Batra Devi Parikh CoGe 322 3,235 0 02 Dec 2016
Towards Evaluating the Robustness of Neural Networks Nicholas Carlini D. Wagner OOD AAML 247 8,550 0 16 Aug 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 193,814 0 10 Dec 2015
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models Bryan A. Plummer Liwei Wang Christopher M. Cervantes Juan C. Caicedo Julia Hockenmaier Svetlana Lazebnik 193 2,053 0 19 May 2015
Explaining and Harnessing Adversarial Examples Ian Goodfellow Jonathon Shlens Christian Szegedy AAML GAN 254 19,045 0 20 Dec 2014
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.6K 39,509 0 01 Sep 2014
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 391 43,619 0 01 May 2014
Intriguing properties of neural networks Christian Szegedy Wojciech Zaremba Ilya Sutskever Joan Bruna D. Erhan Ian Goodfellow Rob Fergus AAML 257 14,912 1 21 Dec 2013
UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild K. Soomro Amir Zamir M. Shah CLIP VGen 141 6,145 0 03 Dec 2012