Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring

7 June 2021

Papers citing "Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring"

50 / 58 papers shown

Title
Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Use Jiajun Xi Yinong He Jianing Yang Yinpei Dai Joyce Chai LM&Ro 24 2 0 31 Oct 2024
Discovering Robotic Interaction Modes with Discrete Representation Learning Liquan Wang Ankit Goyal Haoping Xu Animesh Garg 33 0 0 26 Oct 2024
SEAL: SEmantic-Augmented Imitation Learning via Language Model Chengyang Gu Yuxin Pan Haotian Bai Hui Xiong Yize Chen 27 0 0 03 Oct 2024
P-RAG: Progressive Retrieval Augmented Generation For Planning on Embodied Everyday Task Weiye Xu Min Wang Wengang Zhou Houqiang Li LM&Ro 24 3 0 17 Sep 2024
Enabling Uniform Computer Interaction Experience for Blind Users through Large Language Models Satwik Ram Kodandaram Utkucan Uçkun Xiaojun Bi I. Ramakrishnan V. Ashok 18 7 0 28 Jul 2024
HAPFI: History-Aware Planning based on Fused Information Sujin Jeon Suyeon Shin Byoung-Tak Zhang 29 0 0 23 Jul 2024
DISCO: Embodied Navigation and Interaction via Differentiable Scene Semantics and Dual-level Control Xinyu Xu Shengcheng Luo Yanchao Yang Yong-Lu Li Cewu Lu LM&Ro 42 1 0 20 Jul 2024
ET tu, CLIP? Addressing Common Object Errors for Unseen Environments Ye Won Byun Cathy Jiao Shahriar Noroozizadeh Jimin Sun Rosa Vitiello VLM 44 1 0 25 Jun 2024
Embodied Instruction Following in Unknown Environments Zhenyu Wu Ziwei Wang Xiuwei Xu Jiwen Lu Haibin Yan LM&Ro 28 4 0 17 Jun 2024
RoboCoder: Robotic Learning from Basic Skills to General Tasks with Large Language Models Jingyao Li Pengguang Chen Sitong Wu Chuanyang Zheng Hong Xu Jiaya Jia ELM 29 0 0 06 Jun 2024
HELPER-X: A Unified Instructable Embodied Agent to Tackle Four Interactive Vision-Language Domains with Memory-Augmented Language Models Gabriel H. Sarch Sahil Somani Raghav Kapoor Michael J. Tarr Katerina Fragkiadaki LM&Ro LLMAG 34 3 0 29 Apr 2024
Socratic Planner: Self-QA-Based Zero-Shot Planning for Embodied Instruction Following Suyeon Shin Sujin Jeon Junghyun Kim Gi-Cheon Kang Byoung-Tak Zhang LLMAG 36 0 0 21 Apr 2024
Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoning Rao Fu Jingyu Liu Xilun Chen Yixin Nie Wenhan Xiong LM&Ro LRM 47 50 0 18 Mar 2024
Less is more -- the Dispatcher/ Executor principle for multi-task Reinforcement Learning Martin Riedmiller Tim Hertweck Roland Hafner OffRL 13 1 0 14 Dec 2023
ThinkBot: Embodied Instruction Following with Thought Chain Reasoning Guanxing Lu Ziwei Wang Changliu Liu Jiwen Lu Yansong Tang LRM 25 8 0 12 Dec 2023
tagE: Enabling an Embodied Agent to Understand Human Instructions Chayan Sarkar Avik Mitra Pradip Pramanick Tapas Nayak LM&Ro 36 1 0 24 Oct 2023
LACMA: Language-Aligning Contrastive Learning with Meta-Actions for Embodied Instruction Following Cheng Yang Yen-Chun Chen Jianwei Yang Xiyang Dai Lu Yuan Yu-Chiang Frank Wang Kai-Wei Chang LM&Ro 12 9 0 18 Oct 2023
CoLadder: Supporting Programmers with Hierarchical Code Generation in Multi-Level Abstraction Ryan Yen Jiawen Zhu Sangho Suh Haijun Xia Jian Zhao 40 14 0 12 Oct 2023
Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions Yevgen Chebotar Q. Vuong A. Irpan Karol Hausman F. Xia ... Brianna Zitkovich Tomas Jackson Kanishka Rao Chelsea Finn Sergey Levine OffRL 126 81 0 18 Sep 2023
Multi-Level Compositional Reasoning for Interactive Instruction Following Suvaansh Bhambri Byeonghwi Kim Jonghyun Choi LM&Ro 33 11 0 18 Aug 2023
Context-Aware Planning and Environment-Aware Memory for Instruction Following Embodied Agents Byeonghwi Kim Jinyeon Kim Yuyeong Kim Cheol-Hui Min Jonghyun Choi LM&Ro 35 26 0 14 Aug 2023
Heterogeneous Embodied Multi-Agent Collaboration Xinzhu Liu Di Guo Huaping Liu 24 7 0 26 Jul 2023
MAEA: Multimodal Attribution for Embodied AI Vidhi Jain Jayant Sravan Tamarapalli Sahiti Yerramilli Yonatan Bisk 39 0 0 25 Jul 2023
DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignment Yanjiang Guo Yen-Jen Wang Lihan Zha Zheyuan Jiang Jianyu Chen LM&Ro 24 39 0 01 Jul 2023
Egocentric Planning for Scalable Embodied Task Achievement Xiaotian Liu Héctor Palacios Christian Muise LM&Ro 22 4 0 02 Jun 2023
Thought Cloning: Learning to Think while Acting by Imitating Human Thinking Shengran Hu Jeff Clune LM&Ro OffRL LRM AI4CE 35 27 0 01 Jun 2023
ANPL: Towards Natural Programming with Interactive Decomposition Di Huang Ziyuan Nan Xingui Hu Pengwei Jin Shaohui Peng ... Rui Zhang Zidong Du Qi Guo Yewen Pu Yunji Chen 25 8 0 29 May 2023
Instruction Tuned Models are Quick Learners Himanshu Gupta Saurabh Arjun Sawant Swaroop Mishra Mutsumi Nakamura Arindam Mitra Santosh Mashetty Chitta Baral 26 26 0 17 May 2023
ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenes Ran Gong Jiangyong Huang Yizhou Zhao Haoran Geng Xiaofeng Gao ... Ziheng Zhou D. Terzopoulos Song-Chun Zhu Baoxiong Jia Siyuan Huang LM&Ro 42 45 0 09 Apr 2023
Embodied Concept Learner: Self-supervised Learning of Concepts and Mapping through Instruction Following Mingyu Ding Yan Xu Zhenfang Chen David D. Cox Ping Luo J. Tenenbaum Chuang Gan LM&Ro 56 21 0 07 Apr 2023
Chain-of-Thought Predictive Control Zhiwei Jia Vineet Thumuluri Fangchen Liu Ling-Hao Chen Zhiao Huang H. Su LM&Ro 31 20 0 03 Apr 2023
Task and Motion Planning with Large Language Models for Object Rearrangement Yan Ding Xiaohan Zhang Chris Paxton Shiqi Zhang LM&Ro LRM 29 179 0 10 Mar 2023
PaLM-E: An Embodied Multimodal Language Model Danny Driess F. Xia Mehdi S. M. Sajjadi Corey Lynch Aakanksha Chowdhery ... Marc Toussaint Klaus Greff Andy Zeng Igor Mordatch Peter R. Florence LM&Ro 22 1,565 0 06 Mar 2023
InstructABSA: Instruction Learning for Aspect Based Sentiment Analysis Kevin Scaria Himanshu Gupta Siddharth Goyal Saurabh Arjun Sawant Swaroop Mishra Chitta Baral 26 25 0 16 Feb 2023
RT-1: Robotics Transformer for Real-World Control at Scale Anthony Brohan Noah Brown Justice Carbajal Yevgen Chebotar Joseph Dabis ... Ted Xiao Peng-Tao Xu Sichun Xu Tianhe Yu Brianna Zitkovich LM&Ro 28 1,021 0 13 Dec 2022
LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models Chan Hee Song Jiaman Wu Clay Washington Brian M. Sadler Wei-Lun Chao Yu-Chuan Su LLMAG LM&Ro 24 383 0 08 Dec 2022
DOROTHIE: Spoken Dialogue for Handling Unexpected Situations in Interactive Autonomous Driving Agents Ziqiao Ma B. VanDerPloeg Cristian-Paul Bara Yidong Huang Eui-In Kim Felix Gervits M. Marge J. Chai 55 7 0 22 Oct 2022
DANLI: Deliberative Agent for Following Natural Language Instructions Yichi Zhang Jianing Yang Jiayi Pan Shane Storks N. Devraj Ziqiao Ma Keunwoo Peter Yu Yuwei Bao J. Chai LM&Ro 52 16 0 22 Oct 2022
See, Plan, Predict: Language-guided Cognitive Planning with Video Prediction Maria Attarian Advaya Gupta Ziyi Zhou Wei Yu Igor Gilitschenski Animesh Garg LM&Ro 18 7 0 07 Oct 2022
Providers-Clients-Robots: Framework for spatial-semantic planning for shared understanding in human-robot interaction Tribhi Kathuria Yifan Xu Theodor Chakhachiro X. J. Yang Maani Ghaffari 16 3 0 21 Jun 2022
Few-shot Subgoal Planning with Language Models Lajanugen Logeswaran Yao Fu Moontae Lee Honglak Lee LRM 20 26 0 28 May 2022
Moment-based Adversarial Training for Embodied Language Comprehension Shintaro Ishikawa K. Sugiura LM&Ro 43 8 0 02 Apr 2022
LEBP -- Language Expectation & Binding Policy: A Two-Stream Framework for Embodied Vision-and-Language Interaction Task Learning Agents Hao Liu Yang Liu Hong He Hang Yang LM&Ro 8 21 0 09 Mar 2022
SGL: Symbolic Goal Learning in a Hybrid, Modular Framework for Human Instruction Following Ruinian Xu Hongyi Chen Yunzhi Lin Patricio A. Vela 19 6 0 25 Feb 2022
One Step at a Time: Long-Horizon Vision-and-Language Navigation with Milestones Chan Hee Song Jihyung Kil Tai-Yu Pan Brian M. Sadler Wei-Lun Chao Yu-Chuan Su LRM 22 33 0 14 Feb 2022
Pre-Trained Language Models for Interactive Decision-Making Shuang Li Xavier Puig Chris Paxton Yilun Du Clinton Jia Wang ... Anima Anandkumar Jacob Andreas Igor Mordatch Antonio Torralba Yuke Zhu LM&Ro 34 246 0 03 Feb 2022
Learning to Act with Affordance-Aware Multimodal Neural SLAM Zhiwei Jia Kaixiang Lin Yizhou Zhao Qiaozi Gao Govind Thattai Gaurav Sukhatme LM&Ro 31 15 0 24 Jan 2022
LUMINOUS: Indoor Scene Generation for Embodied AI Challenges Yizhou Zhao Kaixiang Lin Zhiwei Jia Qiaozi Gao Govind Thattai Jesse Thomason Gaurav Sukhatme 3DV LM&Ro 19 15 0 10 Nov 2021
FILM: Following Instructions in Language with Modular Methods So Yeon Min Devendra Singh Chaplot Pradeep Ravikumar Yonatan Bisk Ruslan Salakhutdinov LM&Ro 214 159 0 12 Oct 2021
Are you doing what I say? On modalities alignment in ALFRED Ting-Rui Chiang Yi-Ting Yeh Ta-Chung Chi Yau-Shian Wang 22 1 0 12 Oct 2021