Reassessing Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalization

24 May 2022

Papers citing "Reassessing Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalization"

43 / 43 papers shown

Title
Investigating Prompting Techniques for Zero- and Few-Shot Visual Question Answering Rabiul Awal Le Zhang Aishwarya Agrawal LRM 86 13 0 16 Jun 2023
Answer-Me: Multi-Task Open-Vocabulary Visual Question Answering A. Piergiovanni Wei Li Weicheng Kuo M. Saffar Fred Bertsch A. Angelova 25 16 0 02 May 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 320 3,515 0 29 Apr 2022
CARETS: A Consistency And Robustness Evaluative Test Suite for VQA Carlos E. Jimenez Olga Russakovsky Karthik Narasimhan CoGe 44 14 0 15 Mar 2022
CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailment Haoyu Song Li Dong Weinan Zhang Ting Liu Furu Wei VLM CLIP 67 138 0 14 Mar 2022
LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs Christoph Schuhmann Richard Vencu Romain Beaumont R. Kaczmarczyk Clayton Mullis Aarush Katta Theo Coombes J. Jitsev Aran Komatsuzaki VLM MLLM CLIP 204 1,422 0 03 Nov 2021
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision Zirui Wang Jiahui Yu Adams Wei Yu Zihang Dai Yulia Tsvetkov Yuan Cao VLM MLLM 112 792 0 24 Aug 2021
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation Junnan Li Ramprasaath R. Selvaraju Akhilesh Deepak Gotmare Shafiq Joty Caiming Xiong Guosheng Lin FaML 167 1,943 0 16 Jul 2021
Multimodal Few-Shot Learning with Frozen Language Models Maria Tsimpoukelli Jacob Menick Serkan Cabi S. M. Ali Eslami Oriol Vinyals Felix Hill MLLM 139 775 0 25 Jun 2021
Human-Adversarial Visual Question Answering Sasha Sheng Amanpreet Singh Vedanuj Goswami Jose Alberto Lopez Magana Wojciech Galuba Devi Parikh Douwe Kiela OOD EgoV AAML 36 61 0 04 Jun 2021
Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models Linjie Li Jie Lei Zhe Gan Jingjing Liu AAML VLM 57 73 0 01 Jun 2021
Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answering Corentin Dancette Rémi Cadène Damien Teney Matthieu Cord CML 44 77 0 07 Apr 2021
Domain-robust VQA with diverse datasets and methods but no target labels Ruotong Wang Tristan D. Maidment Ahmad Diab Adriana Kovashka R. Hwa OOD 107 23 0 29 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 810 29,167 0 26 Feb 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 426 1,120 0 17 Feb 2021
Unifying Vision-and-Language Tasks via Text Generation Jaemin Cho Jie Lei Hao Tan Joey Tianyi Zhou MLLM 310 535 0 04 Feb 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 343 6,731 0 23 Dec 2020
Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs Emanuele Bugliarello Ryan Cotterell Naoaki Okazaki Desmond Elliott 67 119 0 30 Nov 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 528 40,739 0 22 Oct 2020
MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering Tejas Gokhale Pratyay Banerjee Chitta Baral Yezhou Yang OOD 44 142 0 18 Sep 2020
Counterfactual VQA: A Cause-Effect Look at Language Bias Yulei Niu Kaihua Tang Hanwang Zhang Zhiwu Lu Xiansheng Hua Ji-Rong Wen CML 102 401 0 08 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 604 41,736 0 28 May 2020
On the Value of Out-of-Distribution Testing: An Example of Goodhart's Law Damien Teney Kushal Kafle Robik Shrestha Ehsan Abbasnejad Christopher Kanan Anton Van Den Hengel OODD OOD 66 146 0 19 May 2020
Pretrained Transformers Improve Out-of-Distribution Robustness Dan Hendrycks Xiaoyuan Liu Eric Wallace Adam Dziedzic R. Krishnan D. Song OOD 159 434 0 13 Apr 2020
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks Jiasen Lu Dhruv Batra Devi Parikh Stefan Lee SSL VLM 217 3,667 0 06 Aug 2019
Visual Entailment: A Novel Task for Fine-Grained Image Understanding Ning Xie Farley Lai Derek Doran Asim Kadav CoGe 95 322 0 20 Jan 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.4K 94,511 0 11 Oct 2018
Cross-Dataset Adaptation for Visual Question Answering Wei-Lun Chao Hexiang Hu Fei Sha OOD 61 49 0 10 Jun 2018
Deep Learning using Rectified Linear Units (ReLU) Abien Fred Agarap 56 3,212 0 22 Mar 2018
VizWiz Grand Challenge: Answering Visual Questions from Blind People Danna Gurari Qing Li Abigale Stangl Anhong Guo Chi Lin Kristen Grauman Jiebo Luo Jeffrey P. Bigham CoGe 80 844 0 22 Feb 2018
Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering Aishwarya Agrawal Dhruv Batra Devi Parikh Aniruddha Kembhavi OOD 136 585 0 01 Dec 2017
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering Peter Anderson Xiaodong He Chris Buehler Damien Teney Mark Johnson Stephen Gould Lei Zhang AIMat 111 4,208 0 25 Jul 2017
C-VQA: A Compositional Split of the Visual Question Answering (VQA) v1.0 Dataset Aishwarya Agrawal Aniruddha Kembhavi Dhruv Batra Devi Parikh CoGe 58 80 0 26 Apr 2017
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning Justin Johnson B. Hariharan Laurens van der Maaten Li Fei-Fei C. L. Zitnick Ross B. Girshick CoGe 287 2,367 0 20 Dec 2016
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering Yash Goyal Tejas Khot D. Summers-Stay Dhruv Batra Devi Parikh CoGe 320 3,224 0 02 Dec 2016
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Zhuowen Tu Kaiming He 476 10,305 0 16 Nov 2016
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Zhiwen Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 867 6,781 0 26 Sep 2016
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations Ranjay Krishna Yuke Zhu Oliver Groth Justin Johnson Kenji Hata ... Yannis Kalantidis Li Li David A. Shamma Michael S. Bernstein Fei-Fei Li 194 5,726 0 23 Feb 2016
Stacked Attention Networks for Image Question Answering Zichao Yang Xiaodong He Jianfeng Gao Li Deng Alex Smola BDL 103 1,877 0 07 Nov 2015
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren Kaiming He Ross B. Girshick Jian Sun AIMat ObjD 463 62,122 0 04 Jun 2015
VQA: Visual Question Answering Aishwarya Agrawal Jiasen Lu Stanislaw Antol Margaret Mitchell C. L. Zitnick Dhruv Batra Devi Parikh CoGe 170 5,452 0 03 May 2015
A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input Mateusz Malinowski Mario Fritz 191 696 0 01 Oct 2014
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 369 43,524 0 01 May 2014