Optimizing Relevance Maps of Vision Transformers Improves Robustness

2 June 2022

Lior Wolf

Papers citing "Optimizing Relevance Maps of Vision Transformers Improves Robustness"

27 / 27 papers shown

Title
A Meaningful Perturbation Metric for Evaluating Explainability Methods Danielle Cohen Hila Chefer Lior Wolf AAML 25 0 0 09 Apr 2025
SpiLiFormer: Enhancing Spiking Transformers with Lateral Inhibition Zeqi Zheng Yanchen Huang Yingchao Yu Zizheng Zhu Junfeng Tang Zhaofei Yu Yaochu Jin 39 0 0 20 Mar 2025
GenFormer -- Generated Images are All You Need to Improve Robustness of Transformers on Small Datasets Sven Oehri Nikolas Ebert Ahmed Abdullah Didier Stricker Oliver Wasenmüller ViT 26 5 0 26 Aug 2024
MePT: Multi-Representation Guided Prompt Tuning for Vision-Language Model Xinyang Wang Yi Yang Minfeng Zhu Kecheng Zheng Shi Liu Wei Chen VPVLM MLLM VLM 47 1 0 19 Aug 2024
MaskInversion: Localized Embeddings via Optimization of Explainability Maps Walid Bousselham Sofian Chaybouti Christian Rupprecht Vittorio Ferrari Hilde Kuehne 64 0 0 29 Jul 2024
Explanation Regularisation through the Lens of Attributions Pedro Ferreira Wilker Aziz Ivan Titov 36 1 0 23 Jul 2024
ACTRESS: Active Retraining for Semi-supervised Visual Grounding Weitai Kang Mengxue Qu Yunchao Wei Yan Yan 31 6 0 03 Jul 2024
Improving Accuracy-robustness Trade-off via Pixel Reweighted Adversarial Training Jiacheng Zhang Feng Liu Dawei Zhou Jingfeng Zhang Tongliang Liu AAML 38 2 0 02 Jun 2024
Tilt your Head: Activating the Hidden Spatial-Invariance of Classifiers Johann Schmidt Sebastian Stober 43 1 0 06 May 2024
Approximate Nullspace Augmented Finetuning for Robust Vision Transformers Haoyang Liu Aditya Singh Yijiang Li Haohan Wang AAML ViT 33 1 0 15 Mar 2024
The Hidden Attention of Mamba Models Ameen Ali Itamar Zimerman Lior Wolf Mamba 39 58 0 03 Mar 2024
Attention-aware Semantic Communications for Collaborative Inference Jiwoong Im Nayoung Kwon Taewoo Park Jiheon Woo Jaeho Lee Yongjune Kim 46 2 0 23 Feb 2024
Backward Lens: Projecting Language Model Gradients into the Vocabulary Space Shahar Katz Yonatan Belinkov Mor Geva Lior Wolf 55 10 1 20 Feb 2024
Channel Vision Transformers: An Image Is Worth 1 x 16 x 16 Words Yu Bao Srinivasan Sivanandan Theofanis Karaletsos ViT 19 23 0 28 Sep 2023
PV2TEA: Patching Visual Modality to Textual-Established Information Extraction Hejie Cui Rongmei Lin Nasser Zalmout Chenwei Zhang Jingbo Shang Carl Yang Xian Li VLM 13 2 0 01 Jun 2023
Contextual Vision Transformers for Robust Representation Learning Yu Bao Theofanis Karaletsos ViT 21 13 0 30 May 2023
Benchmarking Low-Shot Robustness to Natural Distribution Shifts Aaditya K. Singh Kartik Sarangmath Prithvijit Chattopadhyay Judy Hoffman OOD 36 1 0 21 Apr 2023
ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classification Mohammad Reza Taesiri Giang Nguyen Sarra Habchi C. Bezemer Anh Totti Nguyen VLM 32 20 0 11 Apr 2023
Neglected Free Lunch -- Learning Image Classifiers Using Annotation Byproducts Dongyoon Han Junsuk Choe Dante Chun John Joon Young Chung Minsuk Chang Sangdoo Yun Jean Y. Song Seong Joon Oh OOD 424 4 1 30 Mar 2023
Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models Hila Chefer Yuval Alaluf Yael Vinker Lior Wolf Daniel Cohen-Or DiffM 53 498 0 31 Jan 2023
A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Others Zhiheng Li Ivan Evtimov Albert Gordo C. Hazirbas Tal Hassner Cristian Canton Ferrer Chenliang Xu Mark Ibrahim 28 70 0 09 Dec 2022
On Background Bias in Deep Metric Learning Konstantin Kobs Andreas Hotho 14 1 0 04 Oct 2022
Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Models Huy Ha Shuran Song LM&Ro VLM 34 101 0 23 Jul 2022
Rethinking Attention-Model Explainability through Faithfulness Violation Test Y. Liu Haoliang Li Yangyang Guo Chen Kong Jing Li Shiqi Wang FAtt 116 42 0 28 Jan 2022
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 311 5,773 0 29 Apr 2021
Counterfactual Samples Synthesizing for Robust Visual Question Answering Long Chen Xin Yan Jun Xiao Hanwang Zhang Shiliang Pu Yueting Zhuang OOD AAML 149 290 0 14 Mar 2020
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 296 39,194 0 01 Sep 2014