Mastering Robot Manipulation with Multimodal Prompts through Pretraining
and Multi-task Fine-tuning

Mastering Robot Manipulation with Multimodal Prompts through Pretraining and Multi-task Fine-tuning

14 October 2023

Michael Johnston

Suhaila Shakiah

William Yang Wang

Papers citing "Mastering Robot Manipulation with Multimodal Prompts through Pretraining and Multi-task Fine-tuning"

18 / 18 papers shown

Title
Few-Shot Vision-Language Action-Incremental Policy Learning Mingchen Song Xiang Deng Guoqiang Zhong Qi Lv Jia Wan Yinchuan Li Jianye Hao Weili Guan 41 0 0 22 Apr 2025
VIPER: Visual Perception and Explainable Reasoning for Sequential Decision-Making Mohamed Salim Aissi Clemence Grislain Mohamed Chetouani Olivier Sigaud Laure Soulier Nicolas Thome LRM 44 0 0 19 Mar 2025
Reinforcement Learning for Aligning Large Language Models Agents with Interactive Environments: Quantifying and Mitigating Prompt Overfitting Mohamed Salim Aissi Clément Romac Thomas Carta Sylvain Lamprier Pierre-Yves Oudeyer Olivier Sigaud Laure Soulier Nicolas Thome 24 2 0 25 Oct 2024
Semantic Skill Grounding for Embodied Instruction-Following in Cross-Domain Environments Sangwoo Shin Takehiro Matsuoka Youngsoo Jang Moontae Lee Kazuya Yoshida 38 0 0 02 Aug 2024
Investigating the Role of Instruction Variety and Task Difficulty in Robotic Manipulation Tasks Amit Parekh Nikolas Vitsakis Alessandro Suglia Ioannis Konstas AAML 38 5 0 04 Jul 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 82 42 0 23 May 2024
What Foundation Models can Bring for Robot Learning in Manipulation : A Survey Dingzhe Li Yixiang Jin A. Yong Hongze Yu Jun Shi Xiaoshuai Hao Peng Hao Huaping Liu Gang Hua Bin Fang AI4CE LM&Ro 72 13 0 28 Apr 2024
Natural Language as Policies: Reasoning for Coordinate-Level Embodied Control with LLMs Yusuke Mikami Andrew Melnik Jun Miura Ville Hautamaki LM&Ro LRM 60 4 0 20 Mar 2024
Reward Guided Latent Consistency Distillation Jiachen Li Weixi Feng Wenhu Chen William Yang Wang EGVM 25 11 0 16 Mar 2024
An Interactive Agent Foundation Model Zane Durante Bidipta Sarkar Ran Gong Rohan Taori Yusuke Noda ... Katsushi Ikeuchi Fei-Fei Li Jianfeng Gao Naoki Wake Qiuyuan Huang LM&Ro AI4CE LLMAG 91 16 0 08 Feb 2024
GPT-4V(ision) for Robotics: Multimodal Task Planning from Human Demonstration Naoki Wake Atsushi Kanehira Kazuhiro Sasabuchi Jun Takamatsu Katsushi Ikeuchi LM&Ro 21 61 0 20 Nov 2023
Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation Mohit Shridhar Lucas Manuelli D. Fox LM&Ro 161 457 0 12 Sep 2022
Instruction-driven history-aware policies for robotic manipulations Pierre-Louis Guhur Shizhe Chen Ricardo Garcia Pinel Makarand Tapaswi Ivan Laptev Cordelia Schmid LM&Ro 110 102 0 11 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,953 0 04 Mar 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 213 1,657 0 15 Oct 2021
Pix2seq: A Language Modeling Framework for Object Detection Ting-Li Chen Saurabh Saxena Lala Li David J. Fleet Geoffrey E. Hinton MLLM ViT VLM 241 344 0 22 Sep 2021
Unifying Vision-and-Language Tasks via Text Generation Jaemin Cho Jie Lei Hao Tan Joey Tianyi Zhou MLLM 256 525 0 04 Feb 2021
Transformer Networks for Trajectory Forecasting Francesco Giuliari Irtiza Hasan Marco Cristani Fabio Galasso 113 371 0 18 Mar 2020