Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded

11 February 2019

Ramprasaath R. Selvaraju

Devi Parikh

Papers citing "Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded"

50 / 68 papers shown

Title
Overcoming Language Priors for Visual Question Answering Based on Knowledge Distillation Daowan Peng Wei Wei 211 0 0 10 Jan 2025
Benchmarking the Attribution Quality of Vision Models Robin Hesse Simone Schaub-Meyer Stefan Roth FAtt 39 3 0 16 Jul 2024
A look under the hood of the Interactive Deep Learning Enterprise (No-IDLE) Daniel Sonntag Michael Barz Thiago S. Gouvêa VLM 52 4 0 27 Jun 2024
Right on Time: Revising Time Series Models by Constraining their Explanations Maurice Kraus David Steinmann Antonia Wüst Andre Kokozinski Kristian Kersting AI4TS 40 4 0 20 Feb 2024
A Joint Study of Phrase Grounding and Task Performance in Vision and Language Models Noriyuki Kojima Hadar Averbuch-Elor Yoav Artzi 34 2 0 06 Sep 2023
Making the V in Text-VQA Matter Shamanthak Hegde Soumya Jahagirdar Shankar Gangisetty CoGe 39 4 0 01 Aug 2023
One Explanation Does Not Fit XIL Felix Friedrich David Steinmann Kristian Kersting LRM 37 2 0 14 Apr 2023
Divide and Conquer: Answering Questions with Object Factorization and Compositional Reasoning Shi Chen Qi Zhao 47 5 0 18 Mar 2023
ICICLE: Interpretable Class Incremental Continual Learning Dawid Rymarczyk Joost van de Weijer Bartosz Zieliñski Bartlomiej Twardowski CLL 34 28 0 14 Mar 2023
Use Perturbations when Learning from Explanations Juyeon Heo Vihari Piratla Matthew Wicker Adrian Weller AAML 40 1 0 11 Mar 2023
IFAN: An Explainability-Focused Interaction Framework for Humans and NLP Models Edoardo Mosca Daryna Dementieva Tohid Ebrahim Ajdari Maximilian Kummeth Kirill Gringauz Yutong Zhou Georg Groh 24 8 0 06 Mar 2023
ProtoSeg: Interpretable Semantic Segmentation with Prototypical Parts Mikolaj Sacha Dawid Rymarczyk Lukasz Struski Jacek Tabor Bartosz Zieliñski VLM 38 29 0 28 Jan 2023
Explaining Cross-Domain Recognition with Interpretable Deep Classifier Yiheng Zhang Ting Yao Zhaofan Qiu Tao Mei OOD 37 3 0 15 Nov 2022
Prophet Attention: Predicting Attention with Future Attention for Image Captioning Fenglin Liu Xuancheng Ren Xian Wu Wei Fan Yuexian Zou Xu Sun 29 46 0 19 Oct 2022
Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learning Q. Si Yuanxin Liu Fandong Meng Zheng Lin Peng Fu Yanan Cao Weiping Wang Jie Zhou 39 23 0 10 Oct 2022
Leveraging Explanations in Interactive Machine Learning: An Overview Stefano Teso Öznur Alkan Wolfgang Stammer Elizabeth M. Daly XAI FAtt LRM 26 62 0 29 Jul 2022
Visual Perturbation-aware Collaborative Learning for Overcoming the Language Prior Problem Yudong Han Liqiang Nie Jianhua Yin Jianlong Wu Yan Yan 26 13 0 24 Jul 2022
Predicting is not Understanding: Recognizing and Addressing Underspecification in Machine Learning Damien Teney Maxime Peyrard Ehsan Abbasnejad 38 29 0 06 Jul 2022
Multimodal Conversational AI: A Survey of Datasets and Approaches Anirudh S. Sundar Larry Heck 45 29 0 13 May 2022
Unsupervised Learning of Unbiased Visual Representations C. Barbano Enzo Tartaglione Marco Grangetto SSL CML OOD 39 1 0 26 Apr 2022
CNN Attention Guidance for Improved Orthopedics Radiographic Fracture Classification Zhibin Liao Kewen Liao Haifeng Shen M. F. van Boxel J. Prijs R. Jaarsma J. Doornberg Anton Van Den Hengel Johan Verjans 25 14 0 21 Mar 2022
Keyword localisation in untranscribed speech using visually grounded speech models Kayode Olaleye Dan Oneaţă Herman Kamper 32 7 0 02 Feb 2022
Right for the Right Latent Factors: Debiasing Generative Models via Disentanglement Xiaoting Shao Karl Stelzner Kristian Kersting CML DRL 29 3 0 01 Feb 2022
Debiased-CAM to mitigate systematic error with faithful visual explanations of machine learning Wencan Zhang Mariella Dimiccoli Brian Y. Lim FAtt 21 1 0 30 Jan 2022
Language bias in Visual Question Answering: A Survey and Taxonomy Desen Yuan 30 12 0 16 Nov 2021
Introspective Distillation for Robust Question Answering Yulei Niu Hanwang Zhang 27 59 0 01 Nov 2021
Counterfactual Samples Synthesizing and Training for Robust Visual Question Answering Long Chen Yuhang Zheng Yulei Niu Hanwang Zhang Jun Xiao AAML OOD 21 36 0 03 Oct 2021
Multimodal Integration of Human-Like Attention in Visual Question Answering Ekta Sood Fabian Kögel Philippe Muller Dominike Thomas Mihai Bâce Andreas Bulling 41 16 0 27 Sep 2021
Toward a Unified Framework for Debugging Concept-based Models A. Bontempelli Fausto Giunchiglia Andrea Passerini Stefano Teso 20 4 0 23 Sep 2021
Discovering the Unknown Knowns: Turning Implicit Knowledge in the Dataset into Explicit Training Examples for Visual Question Answering Jihyung Kil Cheng Zhang D. Xuan Wei-Lun Chao 61 20 0 13 Sep 2021
Panoptic Narrative Grounding Cristina González Nicolás Ayobi Isabela Hernández José Hernández Jordi Pont-Tuset Pablo Arbeláez 87 22 0 10 Sep 2021
On the Significance of Question Encoder Sequence Model in the Out-of-Distribution Performance in Visual Question Answering K. Gouthaman Anurag Mittal CML 45 0 0 28 Aug 2021
Leveraging Explainability for Comprehending Referring Expressions in the Real World Fethiye Irmak Dogan G. I. Melsión Iolanda Leite 39 8 0 12 Jul 2021
Explainable Machine Learning with Prior Knowledge: An Overview Katharina Beckh Sebastian Müller Matthias Jakobs Vanessa Toborek Hanxiao Tan Raphael Fischer Pascal Welke Sebastian Houben Laura von Rueden XAI 22 28 0 21 May 2021
A Review on Explainability in Multimodal Deep Neural Nets Gargi Joshi Rahee Walambe K. Kotecha 29 140 0 17 May 2021
Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models Mengnan Du Varun Manjunatha R. Jain Ruchi Deshpande Franck Dernoncourt Jiuxiang Gu Tong Sun Xia Hu 59 105 0 11 Mar 2021
Detecting Spurious Correlations with Sanity Tests for Artificial Intelligence Guided Radiology Systems U. Mahmood Robik Shrestha D. Bates L. Mannelli G. Corrias Y. Erdi Christopher Kanan 18 16 0 04 Mar 2021
EnD: Entangling and Disentangling deep representations for bias correction Enzo Tartaglione C. Barbano Marco Grangetto 26 123 0 02 Mar 2021
When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data Peter Hase Joey Tianyi Zhou XAI 25 87 0 03 Feb 2021
Answer Questions with Right Image Regions: A Visual Attention Regularization Approach Yebin Liu Yangyang Guo Jianhua Yin Xuemeng Song Weifeng Liu Liqiang Nie 29 28 0 03 Feb 2021
A Closer Look at the Robustness of Vision-and-Language Pre-trained Models Linjie Li Zhe Gan Jingjing Liu VLM 33 42 0 15 Dec 2020
Debiased-CAM to mitigate image perturbations with faithful visual explanations of machine learning Wencan Zhang Mariella Dimiccoli Brian Y. Lim FAtt 26 18 0 10 Dec 2020
CASTing Your Model: Learning to Localize Improves Self-Supervised Representations Ramprasaath R. Selvaraju Karan Desai Justin Johnson Nikhil Naik SSL 36 79 0 08 Dec 2020
ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity Discovery Dawid Rymarczyk Lukasz Struski Jacek Tabor Bartosz Zieliñski 19 111 0 29 Nov 2020
Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations Wolfgang Stammer P. Schramowski Kristian Kersting FAtt 14 107 0 25 Nov 2020
mForms : Multimodal Form-Filling with Question Answering Larry Heck S. Heck Anirudh S. Sundar 59 6 0 24 Nov 2020
Removing Bias in Multi-modal Classifiers: Regularization by Maximizing Functional Entropies Itai Gat Idan Schwartz Alex Schwing Tamir Hazan 60 90 0 21 Oct 2020
Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding Alexander Ku Peter Anderson Roma Patel Eugene Ie Jason Baldridge 43 302 0 15 Oct 2020
Remembering for the Right Reasons: Explanations Reduce Catastrophic Forgetting Sayna Ebrahimi Suzanne Petryk Akash Gokul William Gan Joseph E. Gonzalez Marcus Rohrbach Trevor Darrell CLL 37 45 0 04 Oct 2020
Trustworthy Convolutional Neural Networks: A Gradient Penalized-based Approach Nicholas F Halliwell Freddy Lecue FAtt 25 9 0 29 Sep 2020