VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models

12 July 2023

Wenlong Huang

Chen Wang

Ruohan Zhang

Yunzhu Li

Jiajun Wu

Li Fei-Fei

LM&Ro

ArXiv PDF HTML

Papers citing "VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models"

50 / 94 papers shown

Title
Robotic Visual Instruction Yuchen Li Ziyang Gong Haoyang Li Xiaoqi Huang Haolan Kang Guangping Bai Xianzheng Ma LM&Ro 114 1 0 01 May 2025
Cracking the Code of Action: a Generative Approach to Affordances for Reinforcement Learning Lynn Cherif Flemming Kondrup David Venuto Ankit Anand Doina Precup Khimya Khetarpal LM&Ro 157 0 0 24 Apr 2025
HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model Jiaming Liu Hao Chen Pengju An Zhuoyang Liu Renrui Zhang ... Chengkai Hou Mengdi Zhao KC alex Zhou Pheng-Ann Heng Shanghang Zhang 122 16 0 13 Mar 2025
Efficient Alignment of Unconditioned Action Prior for Language-conditioned Pick and Place in Clutter Kechun Xu Xunlong Xia Kaixuan Wang Yifei Yang Yunxuan Mao Bing Deng R. Xiong Yansen Wang OffRL 136 0 0 12 Mar 2025
NVP-HRI: Zero Shot Natural Voice and Posture-based Human-Robot Interaction via Large Language Model Yuzhi Lai Shenghai Yuan Youssef Nassar Mingyu Fan T. Weber Matthias Rätsch LM&Ro 89 3 0 12 Mar 2025
EMMOE: A Comprehensive Benchmark for Embodied Mobile Manipulation in Open Environments Dongping Li Tielong Cai Tianci Tang Wenhao Chai Katherine Rose Driggs-Campbell Gaoang Wang LM&Ro 153 0 0 11 Mar 2025
A Data-Centric Revisit of Pre-Trained Vision Models for Robot Learning Xin Wen Bingchen Zhao Yilun Chen Jiangmiao Pang Xiaojuan Qi LM&Ro 164 0 0 10 Mar 2025
GAT-Grasp: Gesture-Driven Affordance Transfer for Task-Aware Robotic Grasping Ruixiang Wang Huayi Zhou Xinyue Yao Guiliang Liu Kui Jia 80 0 0 08 Mar 2025
Generative Artificial Intelligence in Robotic Manipulation: A Survey Kun Zhang Peng Yun Jun Cen Junhao Cai DiDi Zhu ... Qifeng Chen Jia Pan Wei Zhang Bo Yang Hua Chen 130 1 0 05 Mar 2025
A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards Shivansh Patel Xinchen Yin Wenlong Huang Shubham Garg H. Nayyeri Li Fei-Fei Svetlana Lazebnik Yongqian Li 127 1 0 12 Feb 2025
Bilevel Learning for Bilevel Planning Bowen Li Tom Silver Sebastian A. Scherer Alexander G. Gray 209 2 0 12 Feb 2025
HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation Yi Li Yuquan Deng Jing Zhang Joel Jang Marius Memme ... Fabio Ramos Dieter Fox Anqi Li Abhishek Gupta Ankit Goyal LM&Ro 133 14 0 08 Feb 2025
Anticipate & Act : Integrating LLMs and Classical Planning for Efficient Task Execution in Household Environments Raghav Arora Shivam Singh Karthik Swaminathan Ahana Datta Snehasis Banerjee Brojeshwar Bhowmick Krishna Murthy Jatavallabhula Mohan Sridharan M. Krishna LLMAG 87 11 0 04 Feb 2025
Learn from the Past: Language-conditioned Object Rearrangement with Large Language Models Guanqun Cao Ryan Mckenna Erich Graf John Oyekan LM&Ro 162 0 0 30 Jan 2025
RoboHorizon: An LLM-Assisted Multi-View World Model for Long-Horizon Robotic Manipulation Zixuan Chen Jing Huo Yangtao Chen Yang Gao 124 3 0 11 Jan 2025
CoDriveVLM: VLM-Enhanced Urban Cooperative Dispatching and Motion Planning for Future Autonomous Mobility on Demand Systems Haichao Liu Ruoyu Yao Wenru Liu Zhenmin Huang Shaojie Shen Jun Ma 58 2 0 10 Jan 2025
RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation Kun Wu Chengkai Hou Jiaming Liu Zhengping Che Xiaozhu Ju ... Zhenyu Wang Pengju An Siyuan Qian Shanghang Zhang Jian Tang LM&Ro 175 21 0 18 Dec 2024
AnyBimanual: Transferring Unimanual Policy for General Bimanual Manipulation Guanxing Lu Tengbo Yu Haoyuan Deng Season Si Chen Yansong Tang Ziwei Wang 115 3 0 09 Dec 2024
AdvDreamer Unveils: Are Vision-Language Models Truly Ready for Real-World 3D Variations? Shouwei Ruan Hanqin Liu Yao Huang Xiaoqi Wang Caixin Kang Hang Su Yinpeng Dong Xingxing Wei VGen 155 0 0 04 Dec 2024
LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferences Hongyan Zhi Peihao Chen Junyan Li Shuailei Ma Xinyu Sun Tianhang Xiang Yinjie Lei Mingkui Tan Chuang Gan 149 7 0 02 Dec 2024
RoboMatrix: A Skill-centric Hierarchical Framework for Scalable Robot Task Planning and Execution in Open-World Weixin Mao Weiheng Zhong Zhou Jiang Dong Fang Zhongyue Zhang ... Fan Jia Tiancai Wang Haoqiang Fan Osamu Yoshie Osamu Yoshie 167 7 0 29 Nov 2024
Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics Taowen Wang Dongfang Liu James Liang Wenhao Yang Qifan Wang Cheng Han Jiebo Luo Ruixiang Tang Ruixiang Tang AAML 141 6 0 18 Nov 2024
DART-LLM: Dependency-Aware Multi-Robot Task Decomposition and Execution using Large Language Models Yongdong Wang Runze Xiao Jun Younes Louhi Kasahara Ryosuke Yajima Keiji Nagatani Atsushi Yamashita Hajime Asama 75 5 0 13 Nov 2024
Open-World Task and Motion Planning via Vision-Language Model Inferred Constraints Nishanth Kumar F. Ramos Dieter Fox Caelan Reed Garrett Tomás Lozano-Pérez Leslie Pack Kaelbling Caelan Reed Garrett LRM LM&Ro 85 4 0 13 Nov 2024
Local Policies Enable Zero-shot Long-horizon Manipulation Murtaza Dalal Min Liu Walter Talbott Chen Chen Deepak Pathak Jian Zhang Ruslan Salakhutdinov 109 3 0 29 Oct 2024
On-Robot Reinforcement Learning with Goal-Contrastive Rewards Ondrej Biza Thomas Weng Lingfeng Sun Karl Schmeckpeper Tarik Kelestemur Yecheng Jason Ma Robert Platt Jan-Willem van de Meent Lawson L. S. Wong OffRL 72 0 0 25 Oct 2024
Semantically Safe Robot Manipulation: From Semantic Scene Understanding to Motion Safeguards Lukas Brunke Yanni Zhang Ralf Romer Jack Naimer Nikola Staykov Siqi Zhou Angela P. Schoellig 79 4 0 19 Oct 2024
In-Context Learning Enables Robot Action Prediction in LLMs Yida Yin Zekai Wang Yuvan Sharma Dantong Niu Trevor Darrell Roei Herzig LM&Ro 182 4 0 16 Oct 2024
Open-World Reinforcement Learning over Long Short-Term Imagination Jiajian Li Q. Wang Yunbo Wang Xin Jin Yang Li Wenjun Zeng Xiaokang Yang OCL VLM 115 3 0 04 Oct 2024
UniAff: A Unified Representation of Affordances for Tool Usage and Articulation with Vision-Language Models Qiaojun Yu Siyuan Huang Xibin Yuan Zhengkai Jiang Ce Hao ... Junbo Wang Liu Liu Hongsheng Li Peng Gao Cewu Lu 101 3 0 30 Sep 2024
GravMAD: Grounded Spatial Value Maps Guided Action Diffusion for Generalized 3D Manipulation Yangtao Chen Zixuan Chen Junhui Yin Jing Huo Pinzhuo Tian Jieqi Shi Yang Gao LM&Ro 86 3 0 30 Sep 2024
FoAM: Foresight-Augmented Multi-Task Imitation Policy for Robotic Manipulation Litao Liu Wentao Wang Yifan Han Zhuoli Xie Pengfei Yi Junyan Li Yi Qin Wenzhao Lian 66 2 0 29 Sep 2024
COHERENT: Collaboration of Heterogeneous Multi-Robot System with Large Language Models Kehui Liu Zixin Tang Dong Wang Ziyi Wang Bin Zhao Bin Zhao 80 13 0 23 Sep 2024
Points2Plans: From Point Clouds to Long-Horizon Plans with Composable Relational Dynamics Yixuan Huang Christopher Agia Jimmy Wu Tucker Hermans Jeannette Bohg 3DPC 102 2 0 27 Aug 2024
General-purpose Clothes Manipulation with Semantic Keypoints Yuhong Deng David Hsu 90 2 0 15 Aug 2024
LLaRA: Supercharging Robot Learning Data for Vision-Language Policy Xiang Li Cristina Mata J. Park Kumara Kahatapitiya Yoo Sung Jang ... Kanchana Ranasinghe R. Burgert Mu Cai Yong Jae Lee Michael S. Ryoo LM&Ro 99 28 0 28 Jun 2024
LiveScene: Language Embedding Interactive Radiance Fields for Physical Scene Rendering and Control Delin Qu Qizhi Chen Pingrui Zhang Xianqiang Gao Bin Zhao Bin Zhao Dong Wang Xuelong Li AI4CE 73 8 0 23 Jun 2024
3D-GRAND: A Million-Scale Dataset for 3D-LLMs with Better Grounding and Less Hallucination Jianing Yang Xuweiyi Chen Nikhil Madaan Madhavan Iyengar Shengyi Qian David Fouhey Joyce Chai 3DV 100 13 0 07 Jun 2024
Evaluating Uncertainty-based Failure Detection for Closed-Loop LLM Planners Zhi Zheng Qian Feng Hang Li Alois C. Knoll Jianxiang Feng 129 7 0 01 Jun 2024
Robo-Instruct: Simulator-Augmented Instruction Alignment For Finetuning Code LLMs Zichao Hu Junyi Jessy Li Arjun Guha Joydeep Biswas SyDa ALM 79 1 0 30 May 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 235 52 0 23 May 2024
Context-aware LLM-based Safe Control Against Latent Risks Quang Khanh Luu Xiyu Deng Anh Van Ho Yorie Nakahira 95 4 0 18 Mar 2024
ExploRLLM: Guiding Exploration in Reinforcement Learning with Large Language Models Runyu Ma Jelle Luijkx Zlatan Ajanović Jens Kober LM&Ro LRM 72 9 0 14 Mar 2024
Verifiably Following Complex Robot Instructions with Foundation Models Benedict Quartey Eric Rosen Stefanie Tellex George Konidaris LM&Ro 97 13 0 18 Feb 2024
Teaching Robots to Build Simulations of Themselves Yuhang Hu Jiong Lin Hod Lipson SSL 87 4 0 20 Nov 2023
Advances in Embodied Navigation Using Large Language Models: A Survey Jinzhou Lin Han Gao Xuxiang Feng Rongtao Xu Changwei Wang Man Zhang Li Guo Shibiao Xu LM&Ro LLMAG 116 9 0 01 Nov 2023
Learning Reward for Physical Skills using Large Language Model Yuwei Zeng Yiqing Xu 44 6 0 21 Oct 2023
LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving Hao Sha Yao Mu Yuxuan Jiang Li Chen Chenfeng Xu Ping Luo Shengbo Eben Li Masayoshi Tomizuka Wei Zhan Mingyu Ding 225 173 0 04 Oct 2023
Reinforcement Learning for Generative AI: A Survey Yuanjiang Cao Quan.Z Sheng Julian McAuley Lina Yao SyDa 135 13 0 28 Aug 2023
Language to Rewards for Robotic Skill Synthesis Wenhao Yu Nimrod Gileadi Chuyuan Fu Sean Kirmani Kuang-Huei Lee ... N. Heess Dorsa Sadigh Jie Tan Yuval Tassa F. Xia LM&Ro 79 278 0 14 Jun 2023