Executing Instructions in Situated Collaborative Interactions

8 October 2019

Papers citing "Executing Instructions in Situated Collaborative Interactions"

50 / 64 papers shown

Title
CrafText Benchmark: Advancing Instruction Following in Complex Multimodal Open-Ended World Zoya Volovikova G. Gorbov Petr Kuderov Aleksandr I. Panov A. Skrynnik 14 0 0 17 May 2025
Collaborating Action by Action: A Multi-agent LLM Framework for Embodied Reasoning Isadora White Kolby Nottingham Ayush Maniar Max Robinson Hansen Lillemark Mehul Maheshwari Lianhui Qin Prithviraj Ammanabrolu LLMAG LM&Ro 125 0 0 24 Apr 2025
SCOUT: A Situated and Multi-Modal Human-Robot Dialogue Corpus S. Lukin C. Bonial M. Marge Taylor Hudson C. Hayes ... Lucia Donatelli Anton Leuski S. Hill David Traum Clare R. Voss LM&Ro 87 0 0 19 Nov 2024
Infer Human's Intentions Before Following Natural Language Instructions Yanming Wan Yue Wu Yiping Wang Jiayuan Mao Natasha Jaques LM&Ro 33 3 0 26 Sep 2024
CPS-TaskForge: Generating Collaborative Problem Solving Environments for Diverse Communication Tasks Nikita Haduong Irene Wang Bo-Ru Lu Prithviraj Ammanabrolu Noah A. Smith 60 1 0 16 Aug 2024
Situated Instruction Following So Yeon Min Xavi Puig Devendra Singh Chaplot Tsung-Yen Yang Akshara Rai Priyam Parashar Ruslan Salakhutdinov Yonatan Bisk Roozbeh Mottaghi 43 1 0 15 Jul 2024
IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agents Shrestha Mohanty Negar Arabzadeh Andrea Tupini Yuxuan Sun Alexey Skrynnik Artem Zholus Marc-Alexandre Côté Julia Kiseleva 48 0 0 12 Jul 2024
Efficacy of Language Model Self-Play in Non-Zero-Sum Games Austen Liao Nicholas Tomlin Dan Klein 78 1 0 27 Jun 2024
Autonomous Workflow for Multimodal Fine-Grained Training Assistants Towards Mixed Reality Jiahuan Pei Irene Viola Haochen Huang Junxiao Wang Moonisa Ahsan ... Yao Sai Di Wang Zhumin Chen Pengjie Ren Pablo César LM&Ro LLMAG 49 7 0 16 May 2024
Vision-Language Navigation with Embodied Intelligence: A Survey Peng Gao Peng Wang Feng Gao Fei Wang Ruyue Yuan LM&Ro 50 2 0 22 Feb 2024
Improving Agent Interactions in Virtual Environments with Language Models Jack Zhang LLMAG 37 0 0 08 Feb 2024
"We Demand Justice!": Towards Social Context Grounding of Political Texts Rajkumar Pujari Chengfei Wu Dan Goldwasser AILaw 21 3 0 15 Nov 2023
Navigating to Success in Multi-Modal Human-Robot Collaboration: Analysis and Corpus Release S. Lukin Kimberly A. Pollard C. Bonial Taylor Hudson Ron Artstein Clare R. Voss David Traum 8 1 0 26 Oct 2023
Learning a Hierarchical Planner from Humans in Multiple Generations Leonardo Hernandez Cano Yewen Pu Robert D. Hawkins Josh Tenenbaum Armando Solar-Lezama 34 2 0 17 Oct 2023
HandMeThat: Human-Robot Communication in Physical and Social Environments Yanming Wan Jiayuan Mao J. Tenenbaum 44 16 0 05 Oct 2023
MindAgent: Emergent Gaming Interaction Ran Gong Qiuyuan Huang Xiaojian Ma Hoi Vo Zane Durante ... Zilong Zheng Song-Chun Zhu Demetri Terzopoulos Fei-Fei Li Jianfeng Gao LM&Ro 107 65 0 18 Sep 2023
Solving Dialogue Grounding Embodied Task in a Simulated Environment using Further Masked Language Modeling Weijie Zhang 40 0 0 21 Jun 2023
Decision-Oriented Dialogue for Human-AI Collaboration Jessy Lin Nicholas Tomlin Jacob Andreas J. Eisner LLMAG 37 27 0 31 May 2023
Yes, this Way! Learning to Ground Referring Expressions into Actions with Intra-episodic Feedback from Supportive Teachers P. Sadler Sherzod Hakimov David Schlangen 46 1 0 22 May 2023
Multimodal Contextualized Plan Prediction for Embodied Task Completion Mert Inan Aishwarya Padmakumar Spandana Gella P. Lange Dilek Z. Hakkani-Tür LM&Ro 57 0 0 10 May 2023
CB2: Collaborative Natural Language Interaction Research Platform Jacob Sharf Mustafa Omer Gul Yoav Artzi LLMAG 40 1 0 14 Mar 2023
Meta-Explore: Exploratory Hierarchical Vision-and-Language Navigation Using Scene Object Spectrum Grounding Minyoung Hwang Jaeyeon Jeong Minsoo Kim Yoonseon Oh Songhwai Oh 43 19 0 07 Mar 2023
Alexa Arena: A User-Centric Interactive Platform for Embodied AI Qiaozi Gao Govind Thattai Suhaila Shakiah Xiaofeng Gao Shreyas Pansare ... Michael Johnston R. Ghanadan Arindam Mandal Dilek Z. Hakkani-Tür Premkumar Natarajan 6 27 0 02 Mar 2023
Multimodal Speech Recognition for Language-Guided Embodied Agents Allen Chang Xiaoyuan Zhu Aarav Monga Seoho Ahn Tejas Srinivasan Jesse Thomason AuLLM 28 3 0 27 Feb 2023
Learning by Asking for Embodied Visual Navigation and Task Completion Ying Shen Ismini Lourentzou 39 2 0 09 Feb 2023
Continual Learning for Instruction Following from Realtime Feedback Alane Suhr Yoav Artzi 34 17 0 19 Dec 2022
Last-Mile Embodied Visual Navigation Justin Wasserman Karmesh Yadav Girish Chowdhary Abhi Gupta Unnat Jain 41 34 0 21 Nov 2022
Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approaches Daniel Fried Nicholas Tomlin Jennifer Hu Roma Patel Aida Nematzadeh 31 6 0 15 Nov 2022
ViLPAct: A Benchmark for Compositional Generalization on Multimodal Human Activities Terry Yue Zhuo Yaqing Liao Yuecheng Lei Lizhen Qu Gerard de Melo Xiaojun Chang Yazhou Ren Zenglin Xu 47 2 0 11 Oct 2022
Don't Copy the Teacher: Data and Model Challenges in Embodied Dialogue So Yeon Min Hao Zhu Ruslan Salakhutdinov Yonatan Bisk LM&Ro 66 12 0 10 Oct 2022
MUG: Interactive Multimodal Grounding on User Interfaces Tao Li Gang Li Jingjie Zheng Purple Wang Yang Li LLMAG 46 8 0 29 Sep 2022
A Computational Interface to Translate Strategic Intent from Unstructured Language in a Low-Data Setting Pradyumna Tambwekar Lakshita Dodeja Nathan Vaska Wei Xu Matthew C. Gombolay 35 0 0 17 Aug 2022
Few-shot Subgoal Planning with Language Models Lajanugen Logeswaran Yao Fu Moontae Lee Honglak Lee LRM 39 26 0 28 May 2022
Evaluating Multimodal Interactive Agents Josh Abramson Arun Ahuja Federico Carnevale Petko Georgiev Alex Goldin ... Adam Santoro Tamara von Glehn Greg Wayne Nathaniel Wong Chen Yan 36 3 0 26 May 2022
Workflow Discovery from Dialogues in the Low Data Regime Amine El Hattami Stefania Raimondo I. Laradji David Vazquez Pau Rodríguez López C. Pal 55 10 0 24 May 2022
Learning to Execute Actions or Ask Clarification Questions Zhengxiang Shi Yue Feng Aldo Lipani LM&Ro 25 44 0 18 Apr 2022
Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions Jing Gu Eliana Stefani Qi Wu Jesse Thomason Xinze Wang LM&Ro 35 105 0 22 Mar 2022
Towards Collaborative Question Answering: A Preliminary Study Xiangkun Hu Hang Yan Qipeng Guo Xipeng Qiu Weinan Zhang Zheng Zhang 25 0 0 24 Jan 2022
LEMON: Language-Based Environment Manipulation via Execution-Guided Pre-training Qi Shi Qian Liu Bei Chen Yu Zhang Ting Liu Jian-Guang Lou 37 9 0 20 Jan 2022
Less is More: Generating Grounded Navigation Instructions from Landmarks Su Wang Ceslee Montgomery Jordi Orbay Vighnesh Birodkar Aleksandra Faust Izzeddin Gur Natasha Jaques Austin Waters Jason Baldridge Peter Anderson 25 63 0 25 Nov 2021
TEACh: Task-driven Embodied Agents that Chat Aishwarya Padmakumar Jesse Thomason Ayush Shrivastava P. Lange Anjali Narayan-Chen Spandana Gella Robinson Piramithu Gokhan Tur Dilek Z. Hakkani-Tür LM&Ro 174 180 0 01 Oct 2021
Procedures as Programs: Hierarchical Control of Situated Agents through Natural Language Shuyan Zhou Pengcheng Yin Graham Neubig LM&Ro 19 1 0 16 Sep 2021
MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks Cristian-Paul Bara Sky CH-Wang J. Chai 67 63 0 13 Sep 2021
Reference-Centric Models for Grounded Collaborative Dialogue Daniel Fried Justin T. Chiu Dan Klein 52 21 0 10 Sep 2021
Analysis of Language Change in Collaborative Instruction Following Anna Effenberger Eva Yan Rhia Singh Alane Suhr Yoav Artzi 50 13 0 09 Sep 2021
Vision-Language Navigation: A Survey and Taxonomy Wansen Wu Tao Chang Xinmeng Li LM&Ro 17 19 0 26 Aug 2021
Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior Noriyuki Kojima Alane Suhr Yoav Artzi 30 24 0 10 Aug 2021
Neural Abstructions: Abstractions that Support Construction for Grounded Language Learning Kaylee Burns Christopher D. Manning Li Fei-Fei 32 0 0 20 Jul 2021
Learning to communicate about shared procedural abstractions William P. McCarthy Robert D. Hawkins Haoliang Wang Cameron Holdaway Judith E. Fan 18 24 0 30 Jun 2021
Draw Me a Flower: Processing and Grounding Abstraction in Natural Language R. Lachmy Valentina Pyatkin Avshalom Manevich Reut Tsarfaty 29 18 0 27 Jun 2021