Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents

18 January 2022

Wenlong Huang

Pieter Abbeel

Papers citing "Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents"

50 / 205 papers shown

Title
3D CAVLA: Leveraging Depth and 3D Context to Generalize Vision Language Action Models for Unseen Tasks V. Bhat Yu-Hsiang Lan P. Krishnamurthy Ramesh Karri Farshad Khorrami 52 0 0 09 May 2025
On-Device LLM for Context-Aware Wi-Fi Roaming Ju-Hyung Lee Yanqing Lu 23 0 0 07 May 2025
Prompt-responsive Object Retrieval with Memory-augmented Student-Teacher Learning Malte Mosbach Sven Behnke 31 0 0 04 May 2025
ReLI: A Language-Agnostic Approach to Human-Robot Interaction Linus Nwankwo Bjoern Ellensohn Ozan Özdenizci Elmar Rueckert LM&Ro 55 0 0 03 May 2025
LLM-Empowered Embodied Agent for Memory-Augmented Task Planning in Household Robotics Marc Glocker Peter Honig Matthias Hirschmanner Markus Vincze LM&Ro 83 1 0 30 Apr 2025
LLM-based Interactive Imitation Learning for Robotic Manipulation Jonas Werner Kun-Mo Chu C. Weber S. Wermter 86 0 0 30 Apr 2025
Generative AI in Embodied Systems: System-Level Analysis of Performance, Efficiency and Scalability Zishen Wan Jiayi Qian Yuhang Du Jason J. Jabbour Yilun Du Yang Katie Zhao A. Raychowdhury Tushar Krishna Vijay Janapa Reddi LM&Ro 91 0 0 26 Apr 2025
Chain-of-Modality: Learning Manipulation Programs from Multimodal Human Videos with Vision-Language-Models Chen Wang Fei Xia Wenhao Yu Tingnan Zhang Ruohan Zhang Ce Liu Li Fei-Fei Jie Tan Jacky Liang 33 0 0 17 Apr 2025
Exploring Expert Failures Improves LLM Agent Tuning Li-Cheng Lan Andrew Bai Minhao Cheng Ruochen Wang Cho-Jui Hsieh LRM 147 0 0 17 Apr 2025
Reimagining Urban Science: Scaling Causal Inference with Large Language Models Yutong Xia Ao Qu Yunhan Zheng Yihong Tang Dingyi Zhuang ... Cathy Wu R. Zimmermann Lijun Sun Roger Zimmermann Jinhua Zhao AI4CE 60 0 0 15 Apr 2025
Reward Generation via Large Vision-Language Model in Offline Reinforcement Learning Younghwan Lee T. Luu Donghoon Lee Chang-Dong Yoo 3DV VLM OffRL 41 0 0 03 Apr 2025
Visual Environment-Interactive Planning for Embodied Complex-Question Answering Ning Lan Baoshan Ou Xuemei Xie G. Shi LM&Ro 66 1 0 01 Apr 2025
Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks W. Zhang Mengna Wang Gangao Liu Xu Huixin Yiwei Jiang ... Hang Zhang Xin Li Weiming Lu Peng Li Y. Zhuang LM&Ro LRM 65 3 0 27 Mar 2025
Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning Nvidia A. Azzolini Hannah Brandon Prithvijit Chattopadhyay Huayu Chen ... Yao Xu X. Yang Zhuolin Yang Xiaohui Zeng Z. Zhang LM&Ro LRM AI4CE 54 5 0 18 Mar 2025
A Generalist Hanabi Agent Arjun Vaithilingam Sudhakar Hadi Nekoei Mathieu Reymond Miao Liu Janarthanan Rajendran Sarath Chandar 149 0 0 17 Mar 2025
Generative Artificial Intelligence in Robotic Manipulation: A Survey Kun Zhang Peng Yun Jun Cen Junhao Cai DiDi Zhu ... Qifeng Chen Jia Pan Wei K. Zhang Bo Yang Hua Chen 59 1 0 05 Mar 2025
Conversational Planning for Personal Plans Konstantina Christakopoulou Iris Qu John Canny Andrew Goodridge Cj Adams Minmin Chen Maja Matarić LLMAG LM&Ro 55 0 0 26 Feb 2025
Starjob: Dataset for LLM-Driven Job Shop Scheduling Henrik Abgaryan Tristan Cazenave Ararat Harutyunyan 36 0 0 26 Feb 2025
RobotIQ: Empowering Mobile Robots with Human-Level Planning for Real-World Execution Emmanuel K. Raptis Athanasios Ch. Kapoutsis Elias B. Kosmatopoulos LM&Ro 80 0 0 18 Feb 2025
SELP: Generating Safe and Efficient Task Plans for Robot Agents with Large Language Models Yi Wu Z. Xiong Yiran Hu Shreyash S. Iyengar Nan Jiang Aniket Bera Lin Tan Suresh Jagannathan LM&Ro LLMAG 84 4 0 17 Feb 2025
Logical forms complement probability in understanding language model (and human) performance Yixuan Wang Freda Shi ReLM LRM 73 2 0 13 Feb 2025
A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards Shivansh Patel Xinchen Yin Wenlong Huang Shubham Garg H. Nayyeri Li Fei-Fei Svetlana Lazebnik Y. Li 92 0 0 12 Feb 2025
TRAVEL: Training-Free Retrieval and Alignment for Vision-and-Language Navigation Navid Rajabi Jana Kosecka LM&Ro 3DV 58 0 0 11 Feb 2025
HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation Yi Li Yuquan Deng J. Zhang Joel Jang Marius Memme ... Fabio Ramos Dieter Fox Anqi Li Abhishek Gupta Ankit Goyal LM&Ro 99 8 0 08 Feb 2025
Large Language Models as Common-Sense Heuristics Andrey Borro Patricia J. Riddle Michael W Barley Michael Witbrock LRM LM&Ro 136 1 0 31 Jan 2025
Learn from the Past: Language-conditioned Object Rearrangement with Large Language Models Guanqun Cao Ryan Mckenna Erich Graf John Oyekan LM&Ro 118 0 0 30 Jan 2025
Episodic memory in AI agents poses risks that should be studied and mitigated Chad DeChant 57 2 0 20 Jan 2025
A Roadmap to Guide the Integration of LLMs in Hierarchical Planning Israel Puerta-Merino Carlos Núnez-Molina Pablo Mesejo Juan Fernández-Olivares 57 2 0 14 Jan 2025
RoboHorizon: An LLM-Assisted Multi-View World Model for Long-Horizon Robotic Manipulation Zixuan Chen Jing Huo Yangtao Chen Yang Gao 43 2 0 11 Jan 2025
Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web Hiroki Furuta Yutaka Matsuo Aleksandra Faust Izzeddin Gur CLL 87 13 0 03 Jan 2025
Lost-in-Distance: Impact of Contextual Proximity on LLM Performance in Graph Tasks Hamed Firooz Maziar Sanjabi Wenlong Jiang Xiaoling Zhai 65 3 0 03 Jan 2025
Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Models Yue Zhang Ziqiao Ma Jialu Li Yanyuan Qiao Zun Wang J. Chai Qi Wu Mohit Bansal Parisa Kordjamshidi LRM 57 18 0 31 Dec 2024
Multi-Agent Planning Using Visual Language Models Michele Brienza F. Argenziano Vincenzo Suriani D. Bloisi Daniele Nardi LM&Ro LLMAG 62 4 0 31 Dec 2024
LLM-Personalize: Aligning LLM Planners with Human Preferences via Reinforced Self-Training for Housekeeping Robots Dongge Han Trevor A. McInroe Adam Jelley Stefano V. Albrecht Peter Bell Amos Storkey 58 11 0 31 Dec 2024
RoboMatrix: A Skill-centric Hierarchical Framework for Scalable Robot Task Planning and Execution in Open-World Weixin Mao Weiheng Zhong Zhou Jiang Dong Fang Zhongyue Zhang ... Fan Jia Tiancai Wang Haoqiang Fan Osamu Yoshie Osamu Yoshie 116 4 0 29 Nov 2024
RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics Chan Hee Song Valts Blukis Jonathan Tremblay Stephen Tyree Yu-Chuan Su Stan Birchfield 94 5 0 25 Nov 2024
CROPS: A Deployable Crop Management System Over All Possible State Availabilities Jing Wu Zhixin Lai Shengjie Liu Suiyao Chen Ran Tao Pan Zhao Chuyuan Tao Yikun Cheng N. Hovakimyan OffRL 41 0 0 09 Nov 2024
CaPo: Cooperative Plan Optimization for Efficient Embodied Multi-Agent Cooperation Jie Liu Pan Zhou Yingjun Du Ah-Hwee Tan Cees G. M. Snoek J. Sonke E. Gavves LLMAG 29 2 0 07 Nov 2024
CLIP-RT: Learning Language-Conditioned Robotic Policies from Natural Language Supervision Gi-Cheon Kang Junghyun Kim Kyuhwan Shim Jun Ki Lee Byoung-Tak Zhang LM&Ro 97 1 1 01 Nov 2024
EMOS: Embodiment-aware Heterogeneous Multi-robot Operating System with LLM Agents Junting Chen Checheng Yu Xunzhe Zhou Tianqi Xu Yao Mu Mengkang Hu Wenqi Shao Y. Wang G. Li Lin Shao 66 4 0 30 Oct 2024
Local Policies Enable Zero-shot Long-horizon Manipulation Murtaza Dalal Min Liu Walter Talbott Chen Chen Deepak Pathak Jian Zhang Ruslan Salakhutdinov 38 3 0 29 Oct 2024
Guide-LLM: An Embodied LLM Agent and Text-Based Topological Map for Robotic Guidance of People with Visual Impairments Sangmim Song S. Kodagoda A. Gunatilake Marc G. Carmichael Karthick Thiyagarajan Jodi Martin LM&Ro 30 1 0 28 Oct 2024
Coherence-Driven Multimodal Safety Dialogue with Active Learning for Embodied Agents Sabit Hassan Hye-Young Chung Xiang Zhi Tan Malihe Alikhani 50 0 0 18 Oct 2024
In-Context Learning Enables Robot Action Prediction in LLMs Yida Yin Zekai Wang Yuvan Sharma Dantong Niu Trevor Darrell Roei Herzig LM&Ro 112 1 0 16 Oct 2024
Planning Anything with Rigor: General-Purpose Zero-Shot Planning with LLM-based Formalized Programming Yilun Hao Yang Zhang Chuchu Fan LLMAG 46 10 0 15 Oct 2024
ImagineNav: Prompting Vision-Language Models as Embodied Navigator through Scene Imagination Xinxin Zhao Wenzhe Cai Likun Tang Teng Wang LM&Ro 34 3 0 13 Oct 2024
Enabling Novel Mission Operations and Interactions with ROSA: The Robot Operating System Agent Rob Royce Marcel Kaufmann Jonathan Becktor Sangwoo Moon Kalind Carpenter Kai Pak Amanda Towler Rohan Thakker Shehryar Khattak LM&Ro 34 2 0 09 Oct 2024
AgentSquare: Automatic LLM Agent Search in Modular Design Space Yu Shang Yu Li Keyu Zhao Likai Ma J. Liu Fengli Xu Yong Li LLMAG 50 9 0 08 Oct 2024
SEAL: SEmantic-Augmented Imitation Learning via Language Model Chengyang Gu Yuxin Pan Haotian Bai Hui Xiong Yize Chen 27 0 0 03 Oct 2024
Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents Hanrong Zhang Jingyuan Huang Kai Mei Yifei Yao Zhenting Wang Chenlu Zhan Hongwei Wang Yongfeng Zhang AAML LLMAG ELM 51 18 0 03 Oct 2024