v1v2v3 (latest)

DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignment

1 July 2023

Papers citing "DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignment"

34 / 34 papers shown

Title
RAIDER: Tool-Equipped Large Language Model Agent for Robotic Action Issue Detection, Explanation and Recovery Silvia Izquierdo-Badiola Carlos Rizzo Guillem Alenyà LLMAG LM&Ro 153 0 0 22 Mar 2025
Open-World Task and Motion Planning via Vision-Language Model Inferred Constraints Nishanth Kumar F. Ramos Dieter Fox Caelan Reed Garrett Tomás Lozano-Pérez Leslie Pack Kaelbling Caelan Reed Garrett LRM LM&Ro 112 5 0 13 Nov 2024
EMOS: Embodiment-aware Heterogeneous Multi-robot Operating System with LLM Agents Junting Chen Checheng Yu Xunzhe Zhou Tianqi Xu Yao Mu Mengkang Hu Wenqi Shao Yun Wang Ge Li Lin Shao 140 5 0 30 Oct 2024
Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modeling Jinghan Li Zhicheng Sun Fei Li 177 2 0 02 Oct 2024
Evaluating Uncertainty-based Failure Detection for Closed-Loop LLM Planners Zhi Zheng Qian Feng Hang Li Alois C. Knoll Jianxiang Feng 144 7 0 01 Jun 2024
Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V Peiyuan Zhi Zhiyuan Zhang Muzhi Han Zeyu Zhang Zhitian Li Ziyuan Jiao Ziyuan Jiao Siyuan Huang Siyuan Huang LRM LM&Ro 92 33 0 16 Apr 2024
Open-World Object Manipulation using Pre-trained Vision-Language Models Austin Stone Ted Xiao Yao Lu K. Gopalakrishnan Kuang-Huei Lee ... Sean Kirmani Brianna Zitkovich F. Xia Chelsea Finn Karol Hausman LM&Ro 259 155 0 02 Mar 2023
Interactive Language: Talking to Robots in Real Time Corey Lynch Ayzaan Wahid Jonathan Tompson Tianli Ding James Betker Robert Baruch Travis Armstrong Peter R. Florence LM&Ro 96 229 0 12 Oct 2022
ReAct: Synergizing Reasoning and Acting in Language Models Shunyu Yao Jeffrey Zhao Dian Yu Nan Du Izhak Shafran Karthik Narasimhan Yuan Cao LLMAG ReLM LRM 436 2,976 0 06 Oct 2022
Instruction-driven history-aware policies for robotic manipulations Pierre-Louis Guhur Shizhe Chen Ricardo Garcia Pinel Makarand Tapaswi Ivan Laptev Cordelia Schmid LM&Ro 166 107 0 11 Sep 2022
A Generalist Agent Scott E. Reed Konrad Zolna Emilio Parisotto Sergio Gomez Colmenarejo Alexander Novikov ... Yutian Chen R. Hadsell Oriol Vinyals Mahyar Bordbar Nando de Freitas LM&Ro LLMAG AI4CE 211 824 0 12 May 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 529 6,293 0 05 Apr 2022
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances Michael Ahn Anthony Brohan Noah Brown Yevgen Chebotar Omar Cortes ... Ted Xiao Peng Xu Sichun Xu Mengyuan Yan Andy Zeng LM&Ro 192 1,984 0 04 Apr 2022
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language Andy Zeng Maria Attarian Brian Ichter K. Choromanski Adrian S. Wong ... Michael S. Ryoo Vikas Sindhwani Johnny Lee Vincent Vanhoucke Peter R. Florence ReLM LRM 160 588 0 01 Apr 2022
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning Eric Jang A. Irpan Mohi Khansari Daniel Kappler F. Ebert Corey Lynch Sergey Levine Chelsea Finn LM&Ro 263 549 0 04 Feb 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 555 4,413 0 28 Jan 2022
Combining Learning-based Locomotion Policy with Model-based Manipulation for Legged Mobile Manipulators Yuntao Ma Farbod Farshidian Takahiro Miki Joonho Lee Marco Hutter 94 77 0 11 Jan 2022
Simple but Effective: CLIP Embeddings for Embodied AI Apoorv Khandelwal Luca Weihs Roozbeh Mottaghi Aniruddha Kembhavi VLM LM&Ro 87 230 0 18 Nov 2021
CLIPort: What and Where Pathways for Robotic Manipulation Mohit Shridhar Lucas Manuelli Dieter Fox LM&Ro 123 661 0 24 Sep 2021
Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation Suraj Nair E. Mitchell Kevin Chen Brian Ichter Silvio Savarese Chelsea Finn LM&Ro OffRL 117 158 0 02 Sep 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 490 10,496 0 17 Jun 2021
Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring Yichi Zhang J. Chai 44 79 0 07 Jun 2021
Open-vocabulary Object Detection via Vision and Language Knowledge Distillation Xiuye Gu Nayeon Lee Weicheng Kuo Huayu Chen VLM ObjD 293 920 0 28 Apr 2021
kPAM 2.0: Feedback Control for Category-Level Robotic Manipulation Wei Gao Russ Tedrake 148 71 0 11 Feb 2021
PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to Rewards Prasoon Goyal S. Niekum Raymond J. Mooney LM&Ro 62 54 0 30 Jul 2020
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks Mohit Shridhar Jesse Thomason Daniel Gordon Yonatan Bisk Winson Han Roozbeh Mottaghi Luke Zettlemoyer Dieter Fox LM&Ro 120 779 0 03 Dec 2019
Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou Hamid Palangi Lei Zhang Houdong Hu Jason J. Corso Jianfeng Gao MLLM VLM 357 942 0 24 Sep 2019
Language as an Abstraction for Hierarchical Deep Reinforcement Learning Yiding Jiang S. Gu Kevin Patrick Murphy Chelsea Finn OffRL 57 225 0 18 Jun 2019
VirtualHome: Simulating Household Activities via Programs Xavier Puig K. Ra Marko Boben Jiaman Li Tingwu Wang Sanja Fidler Antonio Torralba LM&Ro 100 500 0 19 Jun 2018
DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills Xue Bin Peng Pieter Abbeel Sergey Levine M. van de Panne AI4CE 246 499 0 08 Apr 2018
Gated-Attention Architectures for Task-Oriented Language Grounding Devendra Singh Chaplot Kanthashree Mysore Sathyendra Rama Kumar Pasumarthi Dheeraj Rajagopal Ruslan Salakhutdinov LM&Ro 63 279 0 22 Jun 2017
Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences Hongyuan Mei Joey Tianyi Zhou Matthew R. Walter LM&Ro 92 244 0 12 Jun 2015
VQA: Visual Question Answering Aishwarya Agrawal Jiasen Lu Stanislaw Antol Margaret Mitchell C. L. Zitnick Dhruv Batra Devi Parikh CoGe 226 5,509 0 03 May 2015
PDDL2.1: An Extension to PDDL for Expressing Temporal Planning Domains M. Fox D. Long 85 2,178 0 22 Jun 2011