Large Language Models as Generalizable Policies for Embodied Tasks

26 October 2023

Papers citing "Large Language Models as Generalizable Policies for Embodied Tasks"

50 / 51 papers shown

Title
Sample Efficient Reinforcement Learning via Large Vision Language Model Distillation Donghoon Lee Tung M. Luu Younghwan Lee Chang D. Yoo OffRL VLM 9 0 0 16 May 2025
DSADF: Thinking Fast and Slow for Decision Making Alex Zhihao Dou Dongfei Cui Jun Yan Wei Wang Benteng Chen Haoming Wang Zeke Xie Shufei Zhang OffRL 41 0 0 13 May 2025
RobotxR1: Enabling Embodied Robotic Intelligence on Large Language Models through Closed-Loop Reinforcement Learning Liam Boyle Nicolas Baumann Paviththiren Sivasothilingam Michele Magno Luca Benini LM&Ro LRM 51 0 0 06 May 2025
VLM Q-Learning: Aligning Vision-Language Models for Interactive Decision-Making Jake Grigsby Yuke Zhu Michael S Ryoo Juan Carlos Niebles OffRL VLM 41 0 0 06 May 2025
Text-to-Decision Agent: Learning Generalist Policies from Natural Language Supervision Shilin Zhang Zican Hu Wenhao Wu Xinyi Xie Jianxiang Tang Chunlin Chen Daoyi Dong Yu Cheng Zhenhong Sun Zhi Wang OffRL 139 0 0 21 Apr 2025
Reward Generation via Large Vision-Language Model in Offline Reinforcement Learning Younghwan Lee Tung M. Luu Donghoon Lee Chang D. Yoo 3DV VLM OffRL 41 0 0 03 Apr 2025
Grounding Multimodal LLMs to Embodied Agents that Ask for Help with Reinforcement Learning Ram Ramrakhya Matthew Chang Xavier Puig Ruta Desai Z. Kira Roozbeh Mottaghi LLMAG LM&Ro 66 0 0 01 Apr 2025
SWEET-RL: Training Multi-Turn LLM Agents on Collaborative Reasoning Tasks Yifei Zhou Song Jiang Yuandong Tian Jason Weston Sergey Levine Sainbayar Sukhbaatar Xian Li LLMAG LRM 62 4 0 19 Mar 2025
World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning Siyin Wang Zhaoye Fei Qinyuan Cheng S. Zhang Panpan Cai Jinlan Fu Xipeng Qiu 48 1 0 13 Mar 2025
Enhancing Multi-Agent Systems via Reinforcement Learning with LLM-based Planner and Graph-based Policy Ziqi Jia Junjie Li Xiaoyang Qu Jianzong Wang LLMAG 56 0 0 13 Mar 2025
VLA Model-Expert Collaboration for Bi-directional Manipulation Learning Tian-Yu Xiang Ao-Qun Jin Xiao-Hu Zhou Mei-Jiang Gui Xiao-Liang Xie ... Shuang-Yi Wang Sheng-Bin Duang Si-Cheng Wang Zheng Lei Z. Hou 58 1 0 06 Mar 2025
Generative Artificial Intelligence in Robotic Manipulation: A Survey Anton van den Hengel Peng Yun Jun Cen Junhao Cai DiDi Zhu ... Qifeng Chen Jia Pan Wei Zhang Bo Yang Hua Chen 59 1 0 05 Mar 2025
NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks In Open Domains Wonje Choi Jinwoo Park Sanghyun Ahn Daehee Lee Honguk Woo 141 1 0 02 Mar 2025
Highly Parallelized Reinforcement Learning Training with Relaxed Assignment Dependencies Zhouyu He Peng Qiao Rongchun Li Yong Dou Yusong Tan OffRL 57 0 0 27 Feb 2025
Conversational Planning for Personal Plans Konstantina Christakopoulou Iris Qu John Canny Andrew Goodridge Cj Adams Minmin Chen Maja Matarić LLMAG LM&Ro 62 0 0 26 Feb 2025
The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learning Sheila Schoepp Masoud Jafaripour Yingyue Cao Tianpei Yang Fatemeh Abdollahi Shadan Golestan Zahin Sufiyan Osmar Zaiane Matthew E. Taylor OffRL LM&Ro 46 0 0 24 Feb 2025
FLAG-Trader: Fusion LLM-Agent with Gradient-based Reinforcement Learning for Financial Trading Guojun Xiong Zhiyang Deng Keyi Wang Yupeng Cao Haohang Li ... Kaleb Smith Xiao-Yang Liu J. Huang Sophia Ananiadou Qianqian Xie AIFin 163 0 0 20 Feb 2025
Visual Large Language Models for Generalized and Specialized Applications Yifan Li Zhixin Lai Wentao Bao Zhen Tan Anh Dao Kewei Sui Jiayi Shen Dong Liu Huan Liu Yu Kong VLM 88 11 0 06 Jan 2025
Multi-Modal Grounded Planning and Efficient Replanning For Learning Embodied Agents with A Few Examples Taewoong Kim Byeonghwi Kim Jonghyun Choi LLMAG LM&Ro 49 1 0 23 Dec 2024
I Can Tell What I am Doing: Toward Real-World Natural Language Grounding of Robot Experiences Zihan Wang Brian Liang Varad Dhat Zander Brumbaugh Nick Walker Ranjay Krishna Maya Cakmak 61 4 0 20 Nov 2024
Vision Language Models are In-Context Value Learners Yecheng Jason Ma Joey Hejna Ayzaan Wahid Chuyuan Fu Dhruv Shah ... Dinesh Jayaraman Wenhao Yu Tingnan Zhang Dorsa Sadigh Fei Xia 54 5 0 07 Nov 2024
Eurekaverse: Environment Curriculum Generation via Large Language Models William Liang Sam Wang Hung-Ju Wang Osbert Bastani Dinesh Jayaraman Yecheng Jason Ma SyDa 36 1 0 04 Nov 2024
Reinforcement Learning for Aligning Large Language Models Agents with Interactive Environments: Quantifying and Mitigating Prompt Overfitting Mohamed Salim Aissi Clément Romac Thomas Carta Sylvain Lamprier Pierre-Yves Oudeyer Olivier Sigaud Laure Soulier Nicolas Thome 24 2 0 25 Oct 2024
Agents Thinking Fast and Slow: A Talker-Reasoner Architecture Konstantina Christakopoulou Shibl Mourad Maja Matarić LLMAG 33 11 0 10 Oct 2024
PREDICT: Preference Reasoning by Evaluating Decomposed preferences Inferred from Candidate Trajectories Stephane Aroca-Ouellette Natalie Mackraz B. Theobald Katherine Metcalf 33 0 0 08 Oct 2024
On the Modeling Capabilities of Large Language Models for Sequential Decision Making Martin Klissarov Devon Hjelm Alexander Toshev Bogdan Mazoure LM&Ro ELM OffRL LRM 34 2 0 08 Oct 2024
ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videos Jr-Jen Chen Yu-Chien Liao Hsi-Che Lin Yu-Chu Yu Yen-Chun Chen Yu-Chiang Frank Wang 37 10 0 27 Jun 2024
Language-Driven Closed-Loop Grasping with Model-Predictive Trajectory Replanning Huy Hoang Nguyen Minh Nhat Vu F. Beck Gerald Ebmer Anh Nguyen Andreas Kugi 18 0 0 13 Jun 2024
Grounding Multimodal Large Language Models in Actions Andrew Szot Bogdan Mazoure Harsh Agrawal Devon Hjelm Z. Kira Alexander Toshev LM&Ro 35 10 0 12 Jun 2024
RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots Soroush Nasiriany Abhiram Maddukuri Lance Zhang Adeet Parikh Aaron Lo Abhishek Joshi Ajay Mandlekar Yuke Zhu LM&Ro 45 73 0 04 Jun 2024
DrEureka: Language Model Guided Sim-To-Real Transfer Yecheng Jason Ma William Liang Hung-Ju Wang Sam Wang Yuke Zhu Linxi Fan Osbert Bastani Dinesh Jayaraman 77 43 0 04 Jun 2024
Matryoshka Query Transformer for Large Vision-Language Models Wenbo Hu Zi-Yi Dou Liunian Harold Li Amita Kamath Nanyun Peng Kai-Wei Chang MLLM 36 8 0 29 May 2024
Benchmarking General-Purpose In-Context Learning Fan Wang Chuan Lin Yang Cao Yu Kang 35 1 0 27 May 2024
Meta-Control: Automatic Model-based Control Synthesis for Heterogeneous Robot Skills Tianhao Wei Liqian Ma Rui Chen Weiye Zhao Changliu Liu 45 3 0 18 May 2024
Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning Yuexiang Zhai Hao Bai Zipeng Lin Jiayi Pan Shengbang Tong ... Alane Suhr Saining Xie Yann LeCun Yi Ma Sergey Levine LLMAG LRM 47 59 0 16 May 2024
Knowledgeable Agents by Offline Reinforcement Learning from Large Language Model Rollouts Jing-Cheng Pang Si-Hang Yang Kaiyuan Li Jiaji Zhang Xiong-Hui Chen Nan Tang Yang Yu OffRL KELM LLMAG 41 4 0 14 Apr 2024
GeRM: A Generalist Robotic Model with Mixture-of-experts for Quadruped Robot Wenxuan Song Han Zhao Pengxiang Ding Can Cui Shangke Lyu Yaning Fan Donglin Wang OffRL 27 11 0 20 Mar 2024
Reflect-RL: Two-Player Online RL Fine-Tuning for LMs Runlong Zhou Simon S. Du Beibin Li OffRL 47 3 0 20 Feb 2024
BBSEA: An Exploration of Brain-Body Synchronization for Embodied Agents Sizhe Yang Qian Luo Anumpam Pani Yanchao Yang 34 2 0 13 Feb 2024
Mutual Enhancement of Large Language and Reinforcement Learning Models through Bi-Directional Feedback Mechanisms: A Planning Case Study Shangding Gu LLMAG 43 0 0 12 Jan 2024
LightHouse: A Survey of AGI Hallucination Feng Wang LRM HILM VLM 32 3 0 08 Jan 2024
QUAR-VLA: Vision-Language-Action Model for Quadruped Robots Pengxiang Ding Han Zhao Wenxuan Song Zhitao Wang Zhenyu Wei Shangke Lyu Ningxi Yang Donglin Wang 32 19 0 22 Dec 2023
Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis Yafei Hu Quanting Xie Vidhi Jain Jonathan M Francis Jay Patrikar ... Xiaolong Wang Sebastian A. Scherer Z. Kira Fei Xia Yonatan Bisk LM&Ro AI4CE 32 63 0 14 Dec 2023
Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agents Yu-Chih Chen So Yeon Min Chase Davis Ruslan Salakhutdinov A. Azaria Yuan-Fang Li Tom Michael Mitchell A. Bovik LM&Ro LLMAG 78 33 0 03 May 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 272 4,244 0 30 Jan 2023
Visual Language Maps for Robot Navigation Chen Huang Oier Mees Andy Zeng Wolfram Burgard LM&Ro 159 344 0 11 Oct 2022
Improving alignment of dialogue agents via targeted human judgements Amelia Glaese Nat McAleese Maja Trkebacz John Aslanides Vlad Firoiu ... John F. J. Mellor Demis Hassabis Koray Kavukcuoglu Lisa Anne Hendricks G. Irving ALM AAML 227 502 0 28 Sep 2022
Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation Mohit Shridhar Lucas Manuelli D. Fox LM&Ro 163 457 0 12 Sep 2022
LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action Dhruv Shah B. Osinski Brian Ichter Sergey Levine LM&Ro 158 436 0 10 Jul 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,953 0 04 Mar 2022