In-Context Imitation Learning via Next-Token Prediction

28 August 2024

Letian Fu

Huang Huang

Gaurav Datta

Lawrence Yunliang Chen

William Chung-Ho Panitch

Ken Goldberg

Papers citing "In-Context Imitation Learning via Next-Token Prediction"

27 / 27 papers shown

Title
Grounding Bodily Awareness in Visual Representations for Efficient Policy Learning Junlin Wang Zhiyun Lin 1.2K 0 0 24 May 2025
OTTER: A Vision-Language-Action Model with Text-Aware Visual Feature Extraction Huang Huang Fangchen Liu Letian Fu Tingfan Wu Mustafa Mukadam Jitendra Malik Ken Goldberg Pieter Abbeel LM&Ro VLM 159 10 0 05 Mar 2025
Towards Synergistic, Generalized, and Efficient Dual-System for Robotic Manipulation Qingwen Bu Hongyang Li Li Chen Jisong Cai Jia Zeng Heming Cui Maoqing Yao Yu Qiao 121 11 0 10 Oct 2024
DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset Alexander Khazatsky Karl Pertsch Suraj Nair Ashwin Balakrishna Sudeep Dasari ... Thomas Kollar Sergey Levine Chelsea Finn Sergey Levine Chelsea Finn 226 221 0 19 Mar 2024
Rethinking Patch Dependence for Masked Autoencoders Letian Fu Long Lian Renhao Wang Baifeng Shi Xudong Wang Adam Yala Trevor Darrell Alexei A. Efros Ken Goldberg 105 16 0 25 Jan 2024
Sequential Modeling Enables Scalable Learning for Large Vision Models Yutong Bai Xinyang Geng K. Mangalam Amir Bar Alan Yuille Trevor Darrell Jitendra Malik Alexei A. Efros MLLM VLM 70 169 0 01 Dec 2023
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning Wenliang Dai Junnan Li Dongxu Li A. M. H. Tiong Junqi Zhao Weisheng Wang Boyang Albert Li Pascale Fung Steven C. H. Hoi MLLM VLM 134 2,095 0 11 May 2023
Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence? Arjun Majumdar Karmesh Yadav Sergio Arnaud Yecheng Jason Ma Claire Chen ... Dhruv Batra Yixin Lin Oleksandr Maksymets Aravind Rajeswaran Franziska Meier LM&Ro 63 184 0 31 Mar 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.5K 14,699 0 15 Mar 2023
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion Cheng Chi Zhenjia Xu S. Feng Eric A. Cousineau Yilun Du Benjamin Burchfiel Russ Tedrake Shuran Song 349 1,231 0 07 Mar 2023
Interactive Language: Talking to Robots in Real Time Corey Lynch Ayzaan Wahid Jonathan Tompson Tianli Ding James Betker Robert Baruch Travis Armstrong Peter R. Florence LM&Ro 91 229 0 12 Oct 2022
Real-World Robot Learning with Masked Visual Pre-training Ilija Radosavovic Tete Xiao Stephen James Pieter Abbeel Jitendra Malik Trevor Darrell SSL 230 253 0 06 Oct 2022
VIMA: General Robot Manipulation with Multimodal Prompts Yunfan Jiang Agrim Gupta Zichen Zhang Guanzhi Wang Yongqiang Dou Yanjun Chen Li Fei-Fei Anima Anandkumar Yuke Zhu Linxi Fan LM&Ro 101 355 0 06 Oct 2022
Prompting Decision Transformer for Few-Shot Policy Generalization Mengdi Xu Songlin Yang Shun Zhang Yuchen Lu Ding Zhao J. Tenenbaum Chuang Gan OffRL 76 148 0 27 Jun 2022
A Generalist Agent Scott E. Reed Konrad Zolna Emilio Parisotto Sergio Gomez Colmenarejo Alexander Novikov ... Yutian Chen R. Hadsell Oriol Vinyals Mahyar Bordbar Nando de Freitas LM&Ro LLMAG AI4CE 211 824 0 12 May 2022
Demonstrate Once, Imitate Immediately (DOME): Learning Visual Servoing for One-Shot Imitation Learning Eugene Valassakis Georgios Papagiannis Norman Di Palo Edward Johns 59 43 0 06 Apr 2022
R3M: A Universal Visual Representation for Robot Manipulation Suraj Nair Aravind Rajeswaran Vikash Kumar Chelsea Finn Abhi Gupta LM&Ro 101 584 0 23 Mar 2022
Masked Visual Pre-training for Motor Control Tete Xiao Ilija Radosavovic Trevor Darrell Jitendra Malik SSL 105 250 0 11 Mar 2022
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning Eric Jang A. Irpan Mohi Khansari Daniel Kappler F. Ebert Corey Lynch Sergey Levine Chelsea Finn LM&Ro 263 549 0 04 Feb 2022
Towards More Generalizable One-shot Visual Imitation Learning Zhao Mandi Fangchen Liu Kimin Lee Pieter Abbeel 66 61 0 26 Oct 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 490 10,496 0 17 Jun 2021
ACRONYM: A Large-Scale Grasp Dataset Based on Simulation Clemens Eppner Arsalan Mousavian Dieter Fox 96 210 0 18 Nov 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 670 41,369 0 22 Oct 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 853 42,332 0 28 May 2020
One-Shot Visual Imitation Learning via Meta-Learning Chelsea Finn Tianhe Yu Tianhao Zhang Pieter Abbeel Sergey Levine SSL 126 566 0 14 Sep 2017
One-Shot Imitation Learning Yan Duan Marcin Andrychowicz Bradly C. Stadie Jonathan Ho Jonas Schneider Ilya Sutskever Pieter Abbeel Wojciech Zaremba OffRL 82 688 0 21 Mar 2017
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 827 11,943 0 09 Mar 2017