Title
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 82 43 0 23 May 2024
A Survey of Robotic Language Grounding: Tradeoffs between Symbols and Embeddings Vanya Cohen J. Liu Raymond J. Mooney Stefanie Tellex David Watkins LM&Ro 43 12 0 21 May 2024
Probing Multimodal LLMs as World Models for Driving Shiva Sreeram Tsun-Hsuan Wang Alaa Maalouf Guy Rosman S. Karaman Daniela Rus 32 7 0 09 May 2024
Towards Generalizable Agents in Text-Based Educational Environments: A Study of Integrating RL with LLMs Bahar Radmehr Adish Singla Tanja Kaser LLMAG AI4CE 40 6 0 29 Apr 2024
Action Contextualization: Adaptive Task Planning and Action Tuning using Large Language Models Sthithpragya Gupta Kunpeng Yao Loic Niederhauser A. Billard 34 1 0 19 Apr 2024
COMBO: Compositional World Models for Embodied Multi-Agent Cooperation Hongxin Zhang Zeyuan Wang Qiushi Lyu Zheyuan Zhang Sunli Chen Tianmin Shu Yilun Du Kwonjoon Lee Yilun Du Chuang Gan 48 12 0 16 Apr 2024
Do LLMs Play Dice? Exploring Probability Distribution Sampling in Large Language Models for Behavioral Simulation Jia Gu Liang Pang Huawei Shen Xueqi Cheng 40 5 0 13 Apr 2024
Enhancing Mobile "How-to" Queries with Automated Search Results Verification and Reranking Lei Ding Jeshwanth Bheemanpally Yi Zhang 34 1 0 13 Apr 2024
On the Importance of Uncertainty in Decision-Making with Large Language Models Nicolò Felicioni Lucas Maystre Sina Ghiassian K. Ciosek LLMAG 34 2 0 03 Apr 2024
A Survey on Large Language Model-Based Game Agents Sihao Hu Tiansheng Huang Gaowen Liu Ramana Rao Kompella Gaowen Liu Selim Furkan Tekin Yichang Xu Zachary Yahn Ling Liu LLMAG LM&Ro AI4CE LM&MA 71 51 0 02 Apr 2024
Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods Yuji Cao Huan Zhao Yuheng Cheng Ting Shu Guolong Liu Gaoqi Liang Junhua Zhao Yun Li LLMAG KELM OffRL LM&Ro 37 50 0 30 Mar 2024
MLDT: Multi-Level Decomposition for Complex Long-Horizon Robotic Task Planning with Open-Source Large Language Model Yike Wu Jiatao Zhang Nan Hu LanLing Tang Guilin Qi Jun Shao Jie Ren Wei Song 62 10 0 27 Mar 2024
Grounding Language Plans in Demonstrations Through Counterfactual Perturbations Yanwei Wang Tsun-Hsuan Wang Jiayuan Mao Michael Hagenow Julie Shah LLMAG LM&Ro LRM 43 9 0 25 Mar 2024
SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priors Chenyang Ma Kai Lu Ta-Ying Cheng Niki Trigoni Andrew Markham LRM 40 7 0 18 Mar 2024
LLM3:Large Language Model-based Task and Motion Planning with Motion Failure Reasoning Shu Wang Muzhi Han Ziyuan Jiao Zeyu Zhang Yingnian Wu Song-Chun Zhu Hangxin Liu LRM 33 33 0 18 Mar 2024
Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoning Rao Fu Jingyu Liu Xilun Chen Yixin Nie Wenhan Xiong LM&Ro LRM 52 55 0 18 Mar 2024
VANP: Learning Where to See for Navigation with Self-Supervised Vision-Action Pre-Training Mohammad Nazeri Junzhe Wang Amirreza Payandeh Xuesu Xiao SSL ViT 49 6 0 12 Mar 2024
Language Guided Exploration for RL Agents in Text Environments Hitesh Golchha Sahil Yerawar Dhruvesh Patel Soham Dan K. Murugesan 35 1 0 05 Mar 2024
How Can LLM Guide RL? A Value-Based Approach Shenao Zhang Sirui Zheng Shuqi Ke Zhihan Liu Wanxin Jin Jianbo Yuan Yingxiang Yang Hongxia Yang Zhaoran Wang 35 8 0 25 Feb 2024
Intelligent Director: An Automatic Framework for Dynamic Visual Composition using ChatGPT Sixiao Zheng Jingyang Huo Yu Wang Yanwei Fu VGen DiffM 44 1 0 24 Feb 2024
A Decision-Language Model (DLM) for Dynamic Restless Multi-Armed Bandit Tasks in Public Health Nikhil Behari Edwin Zhang Yunfan Zhao Aparna Taneja Dheeraj M. Nagaraj Milind Tambe 59 11 0 22 Feb 2024
On the Roles of LLMs in Planning: Embedding LLMs into Planning Graphs H. Zhuo Xin Chen Rong Pan 34 2 0 18 Feb 2024
LoTa-Bench: Benchmarking Language-oriented Task Planners for Embodied Agents Jae-Woo Choi Youngwoo Yoon Hyobin Ong Jaehong Kim Minsu Jang 19 13 0 13 Feb 2024
Efficient Non-Parametric Uncertainty Quantification for Black-Box Large Language Models and Decision Planning Yao-Hung Tsai Walter Talbott Jian Zhang LLMAG 18 4 0 01 Feb 2024
True Knowledge Comes from Practice: Aligning LLMs with Embodied Environments via Reinforcement Learning Weihao Tan Wentao Zhang Shanqi Liu Longtao Zheng Xinrun Wang Bo An OffRL 44 17 0 25 Jan 2024
Compositional Generative Inverse Design Tailin Wu Takashi Maruyama Long Wei Tao Zhang Yilun Du Gianluca Iaccarino J. Leskovec DiffM AI4CE 31 6 0 24 Jan 2024
HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environments Qinhong Zhou Sunli Chen Yisong Wang Haozhe Xu Weihua Du Hongxin Zhang Yilun Du Josh Tenenbaum Chuang Gan AI4CE 28 14 0 23 Jan 2024
MMToM-QA: Multimodal Theory of Mind Question Answering Chuanyang Jin Yutong Wu Jing Cao Jiannan Xiang Yen-Ling Kuo Zhiting Hu T. Ullman Antonio Torralba Joshua B. Tenenbaum Tianmin Shu 33 33 0 16 Jan 2024
Large Language Models for Robotics: Opportunities, Challenges, and Perspectives Jiaqi Wang Zihao Wu Yiwei Li Hanqi Jiang Peng Shu ... Lin Zhao Bao Ge Xiang Li Tianming Liu Shu Zhang LM&Ro 45 61 0 09 Jan 2024
LLMs for Robotic Object Disambiguation Connie Jiang Yiqing Xu David Hsu LM&Ro 30 1 0 07 Jan 2024
Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis Yafei Hu Quanting Xie Vidhi Jain Jonathan M Francis Jay Patrikar ... Xiaolong Wang Sebastian A. Scherer Z. Kira Fei Xia Yonatan Bisk LM&Ro AI4CE 43 63 0 14 Dec 2023
LDM $^2$ : A Large Decision Model Imitating Human Cognition with Dynamic Memory Enhancement Xingjin Wang Linjing Li D. Zeng 35 0 0 13 Dec 2023
Interactive Planning Using Large Language Models for Partially Observable Robotics Tasks Lingfeng Sun Devesh K. Jha Chiori Hori Siddarth Jain Radu Corcodel Xinghao Zhu Masayoshi Tomizuka Diego Romeres LM&Ro LLMAG 30 20 0 11 Dec 2023
DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physics Zhiao Huang Feng Chen Yewen Pu Chun-Tse Lin Hao Su Chuang Gan 30 4 0 11 Dec 2023
Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planning Zhiting Hu Tianmin Shu LLMAG LM&Ro LRM 104 34 0 08 Dec 2023
Self-Supervised Behavior Cloned Transformers are Path Crawlers for Text Games Ruoyao Wang Peter Alexander Jansen LRM 25 1 0 07 Dec 2023
FoMo Rewards: Can we cast foundation models as reward functions? Ekdeep Singh Lubana Johann Brehmer P. D. Haan Taco S. Cohen OffRL LRM 48 2 0 06 Dec 2023
Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty Cheng-Fu Yang Haoyang Xu Te-Lin Wu Xiaofeng Gao Kai-Wei Chang Feng Gao DiffM 38 8 0 02 Dec 2023
PALM: Predicting Actions through Language Models Sanghwan Kim Daoji Huang Yongqin Xian Otmar Hilliges Luc Van Gool Xi Wang VLM 24 10 0 29 Nov 2023
GPT4Motion: Scripting Physical Motions in Text-to-Video Generation via Blender-Oriented GPT Planning Jiaxi Lv Yi Huang Mingfu Yan Jiancheng Huang Jianzhuang Liu Yifan Liu Yafei Wen Xiaoxin Chen Shifeng Chen VGen DiffM 32 23 0 21 Nov 2023
Language Model-In-The-Loop: Data Optimal Approach to Learn-To-Recommend Actions in Text Games Arjun Vaithilingam Sudhakar Prasanna Parthasarathi Janarthanan Rajendran Sarath Chandar 23 3 0 13 Nov 2023
Get the Ball Rolling: Alerting Autonomous Robots When to Help to Close the Healthcare Loop Jiaxin Shen Yanyao Liu Ziming Wang Ziyuan Jiao Yufeng Chen Wenjuan Han 20 0 0 05 Nov 2023
Emergence of Abstract State Representations in Embodied Sequence Modeling Tian Yun Zilai Zeng Kunal Handa Ashish V. Thapliyal Bo Pang Ellie Pavlick Chen Sun LM&Ro 35 7 0 03 Nov 2023
A Statistical Guarantee for Representation Transfer in Multitask Imitation Learning Bryan Chan Karime Pereida James Bergstra 44 1 0 02 Nov 2023
Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learning Ruizhe Shi Yuyao Liu Yanjie Ze Simon S. Du Huazhe Xu OffRL RALM 34 18 0 31 Oct 2023
Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models Ziqiao Ma Jacob Sansom Run Peng Joyce Chai 47 17 0 30 Oct 2023
Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Models Tsun-Hsuan Wang Alaa Maalouf Wei Xiao Yutong Ban Alexander Amini Guy Rosman S. Karaman Daniela Rus 27 42 0 26 Oct 2023
Steve-Eye: Equipping LLM-based Embodied Agents with Visual Perception in Open Worlds Sipeng Zheng Jiazheng Liu Yicheng Feng Zongqing Lu 47 29 0 20 Oct 2023
Video Language Planning Yilun Du Mengjiao Yang Peter R. Florence Fei Xia Ayzaan Wahid ... Pieter Abbeel Josh Tenenbaum L. Kaelbling Andy Zeng Jonathan Tompson PINN LM&Ro 96 86 0 16 Oct 2023
Mastering Robot Manipulation with Multimodal Prompts through Pretraining and Multi-task Fine-tuning Jiachen Li Qiaozi Gao Michael Johnston Xiaofeng Gao Xuehai He Suhaila Shakiah Hangjie Shi R. Ghanadan William Y. Wang LM&Ro 27 12 0 14 Oct 2023