Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction

4 September 2018

Papers citing "Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction"

50 / 133 papers shown

Title
Evaluating Multimodal Interactive Agents Josh Abramson Arun Ahuja Federico Carnevale Petko Georgiev Alex Goldin ... Adam Santoro Tamara von Glehn Greg Wayne Nathaniel Wong Chen Yan 31 3 0 26 May 2022
Aerial Vision-and-Dialog Navigation Yue Fan Winson X. Chen Tongzhou Jiang Chun-ni Zhou Yi Zhang Qing Guo 52 20 0 24 May 2022
GoalNet: Inferring Conjunctive Goal Predicates from Human Plan Demonstrations for Robot Instruction Following Shreya Sharma Jigyasa Gupta Shreshth Tuli Rohan Paul Mausam 11 3 0 14 May 2022
Moment-based Adversarial Training for Embodied Language Comprehension Shintaro Ishikawa K. Sugiura LM&Ro 46 8 0 02 Apr 2022
Counterfactual Cycle-Consistent Learning for Instruction Following and Generation in Vision-Language Navigation H. Wang Wei Liang Jianbing Shen Luc Van Gool Wenguan Wang 35 55 0 30 Mar 2022
EnvEdit: Environment Editing for Vision-and-Language Navigation Jialu Li Hao Tan Joey Tianyi Zhou 36 80 0 29 Mar 2022
Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions Jing Gu Eliana Stefani Qi Wu Jesse Thomason Qing Guo LM&Ro 30 105 0 22 Mar 2022
Object Manipulation via Visual Target Localization Kiana Ehsani Ali Farhadi Aniruddha Kembhavi Roozbeh Mottaghi LM&Ro 26 9 0 15 Mar 2022
One-Shot Learning from a Demonstration with Hierarchical Latent Language Nathaniel Weir Xingdi Yuan Marc-Alexandre Côté Matthew J. Hausknecht Romain Laroche Ida Momennejad H. V. Seijen Benjamin Van Durme BDL 27 6 0 09 Mar 2022
LISA: Learning Interpretable Skill Abstractions from Language Divyansh Garg Skanda Vaidyanath Kuno Kim Jiaming Song Stefano Ermon LM&Ro OffRL 156 29 0 28 Feb 2022
SGL: Symbolic Goal Learning in a Hybrid, Modular Framework for Human Instruction Following Ruinian Xu Hongyi Chen Yunzhi Lin Patricio A. Vela 27 6 0 25 Feb 2022
One Step at a Time: Long-Horizon Vision-and-Language Navigation with Milestones Chan Hee Song Jihyung Kil Tai-Yu Pan Brian M. Sadler Wei-Lun Chao Yu-Chuan Su LRM 22 33 0 14 Feb 2022
COIN: Counterfactual Image Generation for VQA Interpretation Zeyd Boukhers Timo Hartmann Jan Jurjens 21 7 0 10 Jan 2022
Talk-to-Resolve: Combining scene understanding and spatial dialogue to resolve granular task ambiguity for a collocated robot Pradip Pramanick Chayan Sarkar Snehasis Banerjee Brojeshwar Bhowmick 16 14 0 22 Nov 2021
A Framework for Learning to Request Rich and Contextually Useful Information from Humans Khanh Nguyen Yonatan Bisk Hal Daumé 49 16 0 14 Oct 2021
Are you doing what I say? On modalities alignment in ALFRED Ting-Rui Chiang Yi-Ting Yeh Ta-Chung Chi Yau-Shian Wang 27 1 0 12 Oct 2021
Waypoint Models for Instruction-guided Navigation in Continuous Environments Jacob Krantz Aaron Gokaslan Dhruv Batra Stefan Lee Oleksandr Maksymets LM&Ro 137 76 0 05 Oct 2021
Mapping Language to Programs using Multiple Reward Components with Inverse Reinforcement Learning Sayan Ghosh Shashank Srivastava 31 3 0 02 Oct 2021
TEACh: Task-driven Embodied Agents that Chat Aishwarya Padmakumar Jesse Thomason Ayush Shrivastava P. Lange Anjali Narayan-Chen Spandana Gella Robinson Piramithu Gokhan Tur Dilek Z. Hakkani-Tür LM&Ro 169 180 0 01 Oct 2021
CLIPort: What and Where Pathways for Robotic Manipulation Mohit Shridhar Lucas Manuelli Dieter Fox LM&Ro 65 632 0 24 Sep 2021
Procedures as Programs: Hierarchical Control of Situated Agents through Natural Language Shuyan Zhou Pengcheng Yin Graham Neubig LM&Ro 19 1 0 16 Sep 2021
Vision-Language Navigation: A Survey and Taxonomy Wansen Wu Tao Chang Xinmeng Li LM&Ro 17 19 0 26 Aug 2021
Communicative Learning with Natural Gestures for Embodied Navigation Agents with Human-in-the-Scene Qi Wu Cheng-Ju Wu Yixin Zhu Jungseock Joo 43 14 0 05 Aug 2021
Neural Abstructions: Abstractions that Support Construction for Grounded Language Learning Kaylee Burns Christopher D. Manning Li Fei-Fei 32 0 0 20 Jul 2021
A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution Valts Blukis Chris Paxton Dieter Fox Animesh Garg Yoav Artzi LM&Ro 214 135 0 12 Jul 2021
SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents Grgur Kovač Rémy Portelas Katja Hofmann Pierre-Yves Oudeyer ALM 27 6 0 02 Jul 2021
Habitat 2.0: Training Home Assistants to Rearrange their Habitat Andrew Szot Alexander Clegg Eric Undersander Erik Wijmans Yili Zhao ... Z. Kira V. Koltun Jitendra Malik Manolis Savva Dhruv Batra LM&Ro 39 501 0 28 Jun 2021
Draw Me a Flower: Processing and Grounding Abstraction in Natural Language R. Lachmy Valentina Pyatkin Avshalom Manevich Reut Tsarfaty 26 18 0 27 Jun 2021
Core Challenges in Embodied Vision-Language Planning Jonathan M Francis Nariaki Kitamura Felix Labelle Xiaopeng Lu Ingrid Navarro Jean Oh LM&Ro 49 45 0 26 Jun 2021
Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring Yichi Zhang J. Chai 25 78 0 07 Jun 2021
VSGM -- Enhance robot task understanding ability through visual semantic graph Cheng-Yu Tsai Mu-Chun Su 30 0 0 19 May 2021
Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigation Muhammad Zubair Irshad Chih-Yao Ma Z. Kira LM&Ro 30 49 0 21 Apr 2021
Improving Cross-Modal Alignment in Vision Language Navigation via Syntactic Information Jialu Li Hao Hao Tan Joey Tianyi Zhou 38 33 0 19 Apr 2021
Diagnosing Vision-and-Language Navigation: What Really Matters Wanrong Zhu Yuankai Qi P. Narayana Kazoo Sone Sugato Basu Qing Guo Qi Wu Miguel P. Eckstein Wei Wang LM&Ro 27 50 0 30 Mar 2021
Reinforcement Learning of Implicit and Explicit Control Flow in Instructions Ethan A. Brooks Janarthanan Rajendran Richard L. Lewis Satinder Singh 19 10 0 25 Feb 2021
A modular vision language navigation and manipulation framework for long horizon compositional tasks in indoor environment Homagni Saha Fateme Fotouhif Qisai Liu S. Sarkar LM&Ro 11 8 0 19 Jan 2021
Natural Language Specification of Reinforcement Learning Policies through Differentiable Decision Trees Pradyumna Tambwekar Andrew Silva N. Gopalan Matthew C. Gombolay 17 9 0 18 Jan 2021
Spatial Reasoning from Natural Language Instructions for Robot Manipulation S. Gubbi Anirban Biswas Raviteja Upadrashta V. Srinivasan Partha P. Talukdar B. Amrutur LM&Ro LRM 48 29 0 26 Dec 2020
Visually Grounding Language Instruction for History-Dependent Manipulation Hyemin Ahn Obin Kwon Kyungdo Kim Jaeyeon Jeong Howoong Jun Hongjung Lee Dongheui Lee Songhwai Oh LM&Ro 21 6 0 16 Dec 2020
Where Are You? Localization from Embodied Dialog Meera Hahn Jacob Krantz Dhruv Batra Devi Parikh James M. Rehg Stefan Lee Peter Anderson LM&Ro 22 27 0 16 Nov 2020
ArraMon: A Joint Navigation-Assembly Instruction Interpretation Task in Dynamic Environments Hyounghun Kim Abhaysinh Zala Graham Burri Hao Tan Joey Tianyi Zhou LM&Ro 27 16 0 15 Nov 2020
Few-shot Object Grounding and Mapping for Natural Language Robot Instruction Following Valts Blukis Ross A. Knepper Yoav Artzi LM&Ro 26 33 0 14 Nov 2020
Modular Networks for Compositional Instruction Following Rodolfo Corona Daniel Fried Coline Devin Dan Klein Trevor Darrell 30 5 0 24 Oct 2020
The RobotSlang Benchmark: Dialog-guided Robot Localization and Navigation Shurjo Banerjee Jesse Thomason Jason J. Corso LM&Ro 78 30 0 23 Oct 2020
Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration Xavier Puig Tianmin Shu Shuang Li Zilin Wang Yuan-Hong Liao J. Tenenbaum Sanja Fidler Antonio Torralba LM&Ro 104 123 0 19 Oct 2020
Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding Alexander Ku Peter Anderson Roma Patel Eugene Ie Jason Baldridge 43 302 0 15 Oct 2020
Safe Reinforcement Learning with Natural Language Constraints Tsung-Yen Yang Michael Y. Hu Yinlam Chow Peter J. Ramadge Karthik R. Narasimhan 25 29 0 11 Oct 2020
AllenAct: A Framework for Embodied AI Research Luca Weihs Jordi Salvador Klemen Kotar Unnat Jain Kuo-Hao Zeng Roozbeh Mottaghi Aniruddha Kembhavi LM&Ro AI4CE 27 71 0 28 Aug 2020
PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to Rewards Prasoon Goyal S. Niekum Raymond J. Mooney LM&Ro 32 53 0 30 Jul 2020
Active Visual Information Gathering for Vision-Language Navigation Hanqing Wang Wenguan Wang Tianmin Shu Wei Liang Jianbing Shen 35 73 0 15 Jul 2020