Defense-Prefix for Preventing Typographic Attacks on CLIP

10 April 2023

Papers citing "Defense-Prefix for Preventing Typographic Attacks on CLIP"

43 / 43 papers shown

Title
SCAM: A Real-World Typographic Robustness Evaluation for Multimodal Foundation Models Justus Westerhoff Erblina Purellku Jakob Hackstein Jonas Loos Leo Pinetzki Lorenz Hufe AAML 64 0 0 07 Apr 2025
Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks Maan Qraitem Nazia Tasnim Piotr Teterwak Kate Saenko Bryan A. Plummer AAML VLM 53 12 0 01 Feb 2024
Discriminative Class Tokens for Text-to-Image Diffusion Models Idan Schwartz Vésteinn Snaebjarnarson Hila Chefer Ryan Cotterell Serge Belongie Lior Wolf Sagie Benaim 55 10 0 30 Mar 2023
Multi-Concept Customization of Text-to-Image Diffusion Nupur Kumari Bin Zhang Richard Y. Zhang Eli Shechtman Jun-Yan Zhu 122 870 0 08 Dec 2022
Magic3D: High-Resolution Text-to-3D Content Creation Chen-Hsuan Lin Jun Gao Luming Tang Towaki Takikawa Fangyin Wei Xun Huang Karsten Kreis Sanja Fidler Ming-Yuan Liu Nayeon Lee 166 1,159 0 18 Nov 2022
MaPLe: Multi-modal Prompt Learning Muhammad Uzair Khattak H. Rasheed Muhammad Maaz Salman Khan Fahad Shahbaz Khan VPVLM VLM 251 565 0 06 Oct 2022
Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models Manli Shu Weili Nie De-An Huang Zhiding Yu Tom Goldstein Anima Anandkumar Chaowei Xiao VLM VPVLM 212 302 0 15 Sep 2022
DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation Nataniel Ruiz Yuanzhen Li Varun Jampani Yael Pritch Michael Rubinstein Kfir Aberman 256 2,851 0 25 Aug 2022
Patching open-vocabulary models by interpolating weights Gabriel Ilharco Mitchell Wortsman S. Gadre Shuran Song Hannaneh Hajishirzi Simon Kornblith Ali Farhadi Ludwig Schmidt VLM KELM 84 175 0 10 Aug 2022
Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification Renrui Zhang Zhang Wei Rongyao Fang Peng Gao Kunchang Li Jifeng Dai Yu Qiao Hongsheng Li VLM 87 314 0 19 Jul 2022
DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations Ximeng Sun Ping Hu Kate Saenko VLM 72 124 0 20 Jun 2022
Disentangling visual and written concepts in CLIP Joanna Materzyñska Antonio Torralba David Bau CoGe 49 51 0 15 Jun 2022
Prefix Conditioning Unifies Language and Label Supervision Kuniaki Saito Kihyuk Sohn Xinming Zhang Chun-Liang Li Chen-Yu Lee Kate Saenko Tomas Pfister VLM CLIP 54 16 0 02 Jun 2022
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding Chitwan Saharia William Chan Saurabh Saxena Lala Li Jay Whang ... Raphael Gontijo-Lopes Tim Salimans Jonathan Ho David J Fleet Mohammad Norouzi VLM 382 6,006 0 23 May 2022
CLIP-Art: Contrastive Pre-training for Fine-Grained Art Classification Marcos V. Conde Kerem Turgutlu CLIP VLM 67 98 0 29 Apr 2022
VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance Katherine Crowson Stella Biderman Daniel Kornis Dashiell Stander Eric Hallahan Louis Castricato Edward Raff CLIP 105 380 0 18 Apr 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 360 6,854 0 13 Apr 2022
PromptDet: Towards Open-vocabulary Detection using Uncurated Images Chengjian Feng Yujie Zhong Zequn Jie Xiangxiang Chu Haibing Ren Xiaolin K. Wei Weidi Xie Lin Ma VPVLM VLM 34 155 0 30 Mar 2022
Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Model Yu Du Fangyun Wei Zihe Zhang Miaojing Shi Yue Gao Guoqi Li VPVLM VLM 66 332 0 28 Mar 2022
Open-Vocabulary One-Stage Detection with Hierarchical Visual-Language Knowledge Distillation Zongyang Ma Guan Luo Jin Gao Liang Li Yuxin Chen Shaoru Wang Congxuan Zhang Weiming Hu VLM ObjD 108 83 0 20 Mar 2022
Conditional Prompt Learning for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VLM CLIP VPVLM 105 1,348 0 10 Mar 2022
RegionCLIP: Region-based Language-Image Pretraining Yiwu Zhong Jianwei Yang Pengchuan Zhang Chunyuan Li Noel Codella ... Luowei Zhou Xiyang Dai Lu Yuan Yin Li Jianfeng Gao VLM CLIP 130 575 0 16 Dec 2021
DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting Yongming Rao Wenliang Zhao Guangyi Chen Yansong Tang Zheng Zhu Guan Huang Jie Zhou Jiwen Lu VLM CLIP 195 574 0 02 Dec 2021
Extract Free Dense Labels from CLIP Chong Zhou Chen Change Loy Bo Dai VLM CLIP 121 477 0 02 Dec 2021
Blended Diffusion for Text-driven Editing of Natural Images Omri Avrahami Dani Lischinski Ohad Fried DiffM 101 947 0 29 Nov 2021
CLIP-Adapter: Better Vision-Language Models with Feature Adapters Peng Gao Shijie Geng Renrui Zhang Teli Ma Rongyao Fang Yongfeng Zhang Hongsheng Li Yu Qiao VLM CLIP 269 1,040 0 09 Oct 2021
Learning to Prompt for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VPVLM CLIP VLM 466 2,394 0 02 Sep 2021
BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning Jinyuan Jia Yupei Liu Neil Zhenqiang Gong SILM SSL 79 156 0 01 Aug 2021
Open-vocabulary Object Detection via Vision and Language Knowledge Distillation Xiuye Gu Nayeon Lee Weicheng Kuo Huayu Chen VLM ObjD 267 915 0 28 Apr 2021
Factual Probing Is [MASK]: Learning vs. Learning to Recall Zexuan Zhong Dan Friedman Danqi Chen 47 410 0 12 Apr 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 861 29,341 0 26 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 429 3,839 0 11 Feb 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 593 40,961 0 22 Oct 2020
How Can We Know What Language Models Know? Zhengbao Jiang Frank F. Xu Jun Araki Graham Neubig KELM 130 1,403 0 28 Nov 2019
LVIS: A Dataset for Large Vocabulary Instance Segmentation Agrim Gupta Piotr Dollár Ross B. Girshick ISeg VLM 100 1,367 0 08 Aug 2019
EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification P. Helber B. Bischke Andreas Dengel Damian Borth 130 1,815 0 31 Aug 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 670 131,414 0 12 Jun 2017
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 193,814 0 10 Dec 2015
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren Kaiming He Ross B. Girshick Jian Sun AIMat ObjD 493 62,243 0 04 Jun 2015
Microsoft COCO Captions: Data Collection and Evaluation Server Xinlei Chen Hao Fang Nayeon Lee Ramakrishna Vedantam Saurabh Gupta Piotr Dollar C. L. Zitnick 205 2,475 0 01 Apr 2015
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 398 43,619 0 01 May 2014
Describing Textures in the Wild Mircea Cimpoi Subhransu Maji Iasonas Kokkinos S. Mohamed Andrea Vedaldi 3DV 116 2,669 0 14 Nov 2013
Fine-Grained Visual Classification of Aircraft Subhransu Maji Esa Rahtu Arno Solin Matthew Blaschko Andrea Vedaldi 114 2,257 0 21 Jun 2013