Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Task

7 December 2022

Shailaja Keyur Sampat

Pratyay Banerjee

Yezhou Yang

Chitta Baral

ArXiv (abs)PDF HTML

Papers citing "Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Task"

27 / 27 papers shown

Title
Reasoning about Actions over Visual and Linguistic Modalities: A Survey Shailaja Keyur Sampat Maitreya Patel Subhasish Das Yezhou Yang Chitta Baral ReLM LM&Ro LRM 62 12 0 15 Jul 2022
Weakly Supervised Relative Spatial Reasoning for Visual Question Answering Pratyay Banerjee Tejas Gokhale Yezhou Yang Chitta Baral LRM 63 19 0 04 Sep 2021
Self-supervised Dialogue Learning for Spoken Conversational Question Answering Nuo Chen Chenyu You Yuexian Zou SSL 65 34 0 04 Jun 2021
CLEVR_HYP: A Challenge Dataset and Baselines for Visual Question Answering with Hypothetical Actions over Images Shailaja Keyur Sampat Akshay Kumar Yezhou Yang Chitta Baral 66 26 0 13 Apr 2021
Can Transformers Reason About Effects of Actions? Pratyay Banerjee Chitta Baral Man Luo Arindam Mitra Kuntal Kumar Pal Tran Cao Son Neeraj Varshney LRM AI4CE 57 10 0 17 Dec 2020
Language-Conditioned Imitation Learning for Robot Manipulation Tasks Simon Stepputtis Joseph Campbell Mariano Phielipp Stefan Lee Chitta Baral H. B. Amor LM&Ro 198 203 0 22 Oct 2020
SSCR: Iterative Language-Based Image Editing via Self-Supervised Counterfactual Reasoning Tsu-Jui Fu Xinze Wang Scott T. Grafton Miguel P. Eckstein William Yang Wang 72 41 0 21 Sep 2020
Graph Edit Distance Reward: Learning to Edit Scene Graph Lichang Chen Guosheng Lin Shijie Wang Qingyao Wu 40 19 0 15 Aug 2020
Large-Scale Adversarial Training for Vision-and-Language Representation Learning Zhe Gan Yen-Chun Chen Linjie Li Chen Zhu Yu Cheng Jingjing Liu ObjD VLM 76 498 0 11 Jun 2020
Following Instructions by Imagining and Reaching Visual Goals John D. Kanu Eadom Dessalene Xiaomin Lin Cornelia Fermuller Yiannis Aloimonos 39 7 0 25 Jan 2020
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks Mohit Shridhar Jesse Thomason Daniel Gordon Yonatan Bisk Winson Han Roozbeh Mottaghi Luke Zettlemoyer Dieter Fox LM&Ro 117 779 0 03 Dec 2019
WIQA: A dataset for "What if..." reasoning over procedural text Niket Tandon Bhavana Dalvi Keisuke Sakaguchi Antoine Bosselut Peter Clark 63 102 0 10 Sep 2019
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Hao Hao Tan Joey Tianyi Zhou VLM MLLM 250 2,488 0 20 Aug 2019
Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology Ran Zmigrod Sabrina J. Mielke Hanna M. Wallach Ryan Cotterell 74 283 0 11 Jun 2019
Composing Text and Image for Image Retrieval - An Empirical Odyssey Nam S. Vo Lu Jiang Chen Sun Kevin Patrick Murphy Li Li Li Fei-Fei James Hays CoGe 56 368 0 18 Dec 2018
Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Intervention Khanh Nguyen Debadeepta Dey Chris Brockett W. Dolan LM&Ro 76 131 0 10 Dec 2018
Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments Howard Chen Alane Suhr Dipendra Kumar Misra Noah Snavely Yoav Artzi 84 390 0 29 Nov 2018
Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding Kexin Yi Jiajun Wu Chuang Gan Antonio Torralba Pushmeet Kohli J. Tenenbaum NAI 84 611 0 04 Oct 2018
Counterfactual Fairness in Text Classification through Robustness Sahaj Garg Vincent Perot Nicole Limtiaco Ankur Taly Ed H. Chi Alex Beutel 99 261 0 27 Sep 2018
Answering Visual What-If Questions: From Actions to Predicted Scene Descriptions M. Wagner H. Basevi Rakshith Shetty Wenbin Li Mateusz Malinowski M. Fritz A. Leonardis 68 29 0 11 Sep 2018
ODSQA: Open-domain Spoken Question Answering Dataset Chia-Hsuan Lee Shang-Ming Wang Huan-Cheng Chang Hung-yi Lee RALM 61 54 0 07 Aug 2018
Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehension Chia-Hsuan Lee Szu-Lin Wu Chi-Liang Liu Hung-yi Lee 66 98 0 01 Apr 2018
Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments Peter Anderson Qi Wu Damien Teney Jake Bruce Mark Johnson Niko Sünderhauf Ian Reid Stephen Gould Anton Van Den Hengel LM&Ro 104 1,322 0 20 Nov 2017
Mask R-CNN Kaiming He Georgia Gkioxari Piotr Dollár Ross B. Girshick ObjD 360 27,244 0 20 Mar 2017
Counterfactual Fairness Matt J. Kusner Joshua R. Loftus Chris Russell Ricardo M. A. Silva FaML 224 1,586 0 20 Mar 2017
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning Justin Johnson B. Hariharan Laurens van der Maaten Li Fei-Fei C. L. Zitnick Ross B. Girshick CoGe 313 2,387 0 20 Dec 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,426 0 10 Dec 2015