e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks

8 May 2021

Papers citing "e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks"

50 / 79 papers shown

Title
ChartQA-X: Generating Explanations for Charts Shamanthak Hegde Pooyan Fazli H. Seifi 24 0 0 17 Apr 2025
Unlocking Open-Set Language Accessibility in Vision Models Fawaz Sammani Jonas Fischer Nikos Deligiannis VLM 55 0 0 14 Mar 2025
VIKSER: Visual Knowledge-Driven Self-Reinforcing Reasoning Framework Chunbai Zhang Chao Wang Yang Zhou Yan Peng LRM ReLM 60 0 0 02 Feb 2025
What Factors Affect Multi-Modal In-Context Learning? An In-Depth Exploration L. Qin Qiguang Chen Hao Fei Zhi Chen Min Li Wanxiang Che 39 5 0 27 Oct 2024
Fool Me Once? Contrasting Textual and Visual Explanations in a Clinical Decision-Support Setting Maxime Kayser Bayar I. Menzat Cornelius Emde Bogdan Bercean Alex Novak Abdala Espinosa B. Papież Susanne Gaube Thomas Lukasiewicz Oana-Maria Camburu 23 1 0 16 Oct 2024
Semantic Token Reweighting for Interpretable and Controllable Text Embeddings in CLIP Eunji Kim Kyuhong Shim Simyung Chang Sungroh Yoon CLIP 28 0 0 11 Oct 2024
LLaVA Needs More Knowledge: Retrieval Augmented Natural Language Generation with Knowledge Graph for Explaining Thoracic Pathologies Ameer Hamza Abdullah Yong Hyun Ahn Sungyoung Lee Seong Tae Kim 24 2 0 07 Oct 2024
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark Wenhao Chai Enxin Song Y. Du Chenlin Meng Vashisht Madhavan Omer Bar-Tal Jeng-Neng Hwang Saining Xie Christopher D. Manning 3DV 84 25 0 04 Oct 2024
Towards Efficient and Robust VQA-NLE Data Generation with Large Vision-Language Models Patrick Amadeus Irawan Genta Indra Winata Samuel Cahyawijaya Ayu Purwarianti 31 0 0 23 Sep 2024
Retrieval-Augmented Natural Language Reasoning for Explainable Visual Question Answering Su Hyeon Lim Minkuk Kim Hyeon Bae Kim Seong Tae Kim ReLM LRM 40 0 0 30 Aug 2024
CT-AGRG: Automated Abnormality-Guided Report Generation from 3D Chest CT Volumes Theo Di Piazza 26 0 0 21 Aug 2024
The Illusion of Competence: Evaluating the Effect of Explanations on Users' Mental Models of Visual Question Answering Systems Judith Sieker Simeon Junker R. Utescher Nazia Attari H. Wersing Hendrik Buschmeier Sina Zarrieß 30 1 0 27 Jun 2024
A look under the hood of the Interactive Deep Learning Enterprise (No-IDLE) Daniel Sonntag Michael Barz Thiago S. Gouvêa VLM 44 4 0 27 Jun 2024
A Survey of Multimodal Large Language Model from A Data-centric Perspective Tianyi Bai Hao Liang Binwang Wan Yanran Xu Xi Li ... Ping-Chia Huang Jiulong Shan Conghui He Binhang Yuan Wentao Zhang 47 36 0 26 May 2024
ChatGPT Rates Natural Language Explanation Quality Like Humans: But on Which Scales? Fan Huang Haewoon Kwak Kunwoo Park Jisun An ALM ELM AI4MH 34 12 0 26 Mar 2024
CoTBal: Comprehensive Task Balancing for Multi-Task Visual Instruction Tuning Yanqi Dai Dong Jing Nanyi Fei Zhiwu Lu Nanyi Fei Guoxing Yang Zhiwu Lu 55 3 0 07 Mar 2024
A Multimodal In-Context Tuning Approach for E-Commerce Product Description Generation Yunxin Li Baotian Hu Wenhan Luo Lin Ma Yuxin Ding Min-Ling Zhang 51 1 0 21 Feb 2024
Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarks Stefan Blücher Johanna Vielhaben Nils Strodthoff AAML 66 20 0 12 Jan 2024
Towards More Faithful Natural Language Explanation Using Multi-Level Contrastive Learning in VQA Chengen Lai Shengli Song Shiqi Meng Jingyang Li Sitong Yan Guangneng Hu 23 5 0 21 Dec 2023
From Wrong To Right: A Recursive Approach Towards Vision-Language Explanation Jiaxin Ge Sanjay Subramanian Trevor Darrell Boyi Li LRM 24 4 0 21 Nov 2023
Representing visual classification as a linear combination of words Shobhit Agarwal Yevgeniy R. Semenov William Lotter 40 1 0 18 Nov 2023
Using Natural Language Explanations to Improve Robustness of In-context Learning Xuanli He Yuxiang Wu Oana-Maria Camburu Pasquale Minervini Pontus Stenetorp AAML 31 1 0 13 Nov 2023
Characterizing Large Language Models as Rationalizers of Knowledge-intensive Tasks Aditi Mishra Sajjadur Rahman H. Kim Kushan Mitra Estevam R. Hruschka 13 6 0 09 Nov 2023
Zero-shot Translation of Attention Patterns in VQA Models to Natural Language Leonard Salewski A. Sophia Koepke Hendrik P. A. Lensch Zeynep Akata 31 2 0 08 Nov 2023
Active Reasoning in an Open-World Environment Manjie Xu Guangyuan Jiang Weihan Liang Chi Zhang Yixin Zhu LLMAG LRM 21 10 0 03 Nov 2023
Learning to Follow Object-Centric Image Editing Instructions Faithfully Tuhin Chakrabarty Kanishk Singh Arkadiy Saakyan Smaranda Muresan DiffM 25 6 0 29 Oct 2023
Large Language Models are Visual Reasoning Coordinators Liangyu Chen Bo Li Sheng Shen Jingkang Yang Chunyuan Li Kurt Keutzer Trevor Darrell Ziwei Liu VLM LRM 41 48 0 23 Oct 2023
Dataset Bias Mitigation in Multiple-Choice Visual Question Answering and Beyond Zhecan Wang Long Chen Haoxuan You Keyang Xu Yicheng He Wenhao Li Noal Codella Kai-Wei Chang Shih-Fu Chang 23 3 0 23 Oct 2023
REFER: An End-to-end Rationale Extraction Framework for Explanation Regularization Mohammad Reza Ghasemi Madani Pasquale Minervini 27 4 0 22 Oct 2023
Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning Mustafa Shukor Alexandre Ramé Corentin Dancette Matthieu Cord LRM MLLM 38 20 0 01 Oct 2023
Black-box Attacks on Image Activity Prediction and its Natural Language Explanations Alina Elena Baia Valentina Poggioni Andrea Cavallaro AAML 16 1 0 30 Sep 2023
S3C: Semi-Supervised VQA Natural Language Explanation via Self-Critical Learning Wei Suo Mengyang Sun Weisong Liu Yi-Meng Gao Peifeng Wang Yanning Zhang Qi Wu LRM 32 7 0 05 Sep 2023
DeViL: Decoding Vision features into Language Meghal Dani Isabel Rio-Torto Stephan Alaniz Zeynep Akata VLM 42 7 0 04 Sep 2023
Uni-NLX: Unifying Textual Explanations for Vision and Vision-Language Tasks Fawaz Sammani Nikos Deligiannis 13 5 0 17 Aug 2023
Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step Liunian Harold Li Jack Hessel Youngjae Yu Xiang Ren Kai-Wei Chang Yejin Choi LRM AI4CE ReLM 22 129 0 24 Jun 2023
M $^3$ IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning Lei Li Yuwei Yin Shicheng Li Liang Chen Peiyi Wang ... Yazheng Yang Jingjing Xu Xu Sun Lingpeng Kong Qi Liu MLLM VLM 27 115 0 07 Jun 2023
KNOW How to Make Up Your Mind! Adversarially Detecting and Alleviating Inconsistencies in Natural Language Explanations Myeongjun Jang Bodhisattwa Prasad Majumder Julian McAuley Thomas Lukasiewicz Oana-Maria Camburu AAML 12 4 0 05 Jun 2023
Faithfulness Tests for Natural Language Explanations Pepa Atanasova Oana-Maria Camburu Christina Lioma Thomas Lukasiewicz J. Simonsen Isabelle Augenstein FAtt 21 59 0 29 May 2023
SPARSEFIT: Few-shot Prompting with Sparse Fine-tuning for Jointly Generating Predictions and Natural Language Explanations Jesus Solano Oana-Maria Camburu Pasquale Minervini 11 1 0 22 May 2023
ZARA: Improving Few-Shot Self-Rationalization for Small Language Models Wei-Lin Chen An-Zi Yen Cheng-Kuang Wu Hen-Hsen Huang Hsin-Hsi Chen ReLM LRM 24 10 0 12 May 2023
Accountable Textual-Visual Chat Learns to Reject Human Instructions in Image Re-creation Zhiwei Zhang Yuliang Liu MLLM 22 0 0 10 Mar 2023
Interpretable Visual Question Answering Referring to Outside Knowledge He Zhu Ren Togo Takahiro Ogawa Miki Haseyama 16 0 0 08 Mar 2023
Benchmarks for Automated Commonsense Reasoning: A Survey E. Davis ELM LRM 24 57 0 09 Feb 2023
Towards a Unified Model for Generating Answers and Explanations in Visual Question Answering Chenxi Whitehouse Tillman Weyde Pranava Madhyastha LRM 41 3 0 25 Jan 2023
Harnessing the Power of Multi-Task Pretraining for Ground-Truth Level Natural Language Explanations Björn Plüster Jakob Ambsdorf Lukas Braach Jae Hee Lee S. Wermter 25 6 0 08 Dec 2022
Going Beyond XAI: A Systematic Survey for Explanation-Guided Learning Yuyang Gao Siyi Gu Junji Jiang S. Hong Dazhou Yu Liang Zhao 29 39 0 07 Dec 2022
Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works Chao Tao Ji Qi Mingning Guo Qing Zhu Haifeng Li SSL 24 56 0 15 Nov 2022
ExPUNations: Augmenting Puns with Keywords and Explanations Jiao Sun Anjali Narayan-Chen Shereen Oraby Alessandra Cervone Tagyoung Chung Jing Huang Yang Liu Nanyun Peng 19 10 0 24 Oct 2022
Challenges in Explanation Quality Evaluation Hendrik Schuff Heike Adel Peng Qi Ngoc Thang Vu XAI 33 3 0 13 Oct 2022
Explanations from Large Language Models Make Small Reasoners Better Shiyang Li Jianshu Chen Yelong Shen Zhiyu Zoey Chen Xinlu Zhang ... Jingu Qian Baolin Peng Yi Mao Wenhu Chen Xifeng Yan ReLM LRM 38 129 0 13 Oct 2022