Perceptual Score: What Data Modalities Does Your Model Perceive?

27 October 2021

Papers citing "Perceptual Score: What Data Modalities Does Your Model Perceive?"

12 / 12 papers shown

Title
Do Vision & Language Decoders use Images and Text equally? How Self-consistent are their Explanations? Letitia Parcalabescu Anette Frank MLLM CoGe VLM 84 3 0 29 Apr 2024
Impact of architecture on robustness and interpretability of multispectral deep neural networks Charles Godfrey Elise Bishoff Myles Mckay E. Byler 34 0 0 21 Sep 2023
Boosting Multi-modal Model Performance with Adaptive Gradient Modulation Hong Li Xingyu Li Pengbo Hu Yinuo Lei Chunxiao Li Yi Zhou 47 22 0 15 Aug 2023
Dissecting Multimodality in VideoQA Transformer Models by Impairing Modality Fusion Isha Rawal Alexander Matyasko Shantanu Jaiswal Basura Fernando Cheston Tan 26 2 0 15 Jun 2023
MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks Letitia Parcalabescu Anette Frank 37 22 0 15 Dec 2022
Understanding Cross-modal Interactions in V&L Models that Generate Scene Descriptions Michele Cafagna Kees van Deemter Albert Gatt CoGe 16 4 0 09 Nov 2022
Optimizing Relevance Maps of Vision Transformers Improves Robustness Hila Chefer Idan Schwartz Lior Wolf ViT 40 38 0 02 Jun 2022
SHAPE: An Unified Approach to Evaluate the Contribution and Cooperation of Individual Modalities Pengbo Hu Xingyu Li Yi Zhou 30 10 0 30 Apr 2022
ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic Yoad Tewel Yoav Shalev Idan Schwartz Lior Wolf VLM 34 192 0 29 Nov 2021
Removing Bias in Multi-modal Classifiers: Regularization by Maximizing Functional Entropies Itai Gat Idan Schwartz A. Schwing Tamir Hazan 60 90 0 21 Oct 2020
Counterfactual Samples Synthesizing for Robust Visual Question Answering Long Chen Xin Yan Jun Xiao Hanwang Zhang Shiliang Pu Yueting Zhuang OOD AAML 154 290 0 14 Mar 2020
Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Mor Geva Yoav Goldberg Jonathan Berant 242 320 0 21 Aug 2019