DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning

7 November 2024

Papers citing "DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning"

46 / 46 papers shown

Title
Imagine Beyond! Distributionally Robust Auto-Encoding for State Space Coverage in Online Reinforcement Learning Nicolas Castanet Olivier Sigaud Sylvain Lamprier OffRL 83 0 0 23 May 2025
UniVLA: Learning to Act Anywhere with Task-centric Latent Actions Qingwen Bu Yanting Yang Jisong Cai Shenyuan Gao Guanghui Ren Maoqing Yao Ping Luo Hongyang Li 314 6 0 09 May 2025
Strengthening Generative Robot Policies through Predictive World Modeling Han Qi Haocheng Yin Aris Zhu Yilun Du Heng Yang 121 3 0 02 Feb 2025
Navigation World Models Amir Bar G. Zhou Danny Tran Trevor Darrell Yann LeCun VGen EgoV 122 24 0 04 Dec 2024
Robot Utility Models: General Policies for Zero-Shot Deployment in New Environments Haritheja Etukuru Norihito Naka Zijin Hu Seungjae Lee Julian Mehu Aaron Edsinger Chris Paxton Soumith Chintala Lerrel Pinto Nur Muhammad (Mahi) Shafiullah LM&Ro 74 24 0 09 Sep 2024
AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulation Kaifeng Zhang Baoyu Li Kris Hauser Yunzhu Li AI4CE 76 18 0 10 Jul 2024
BAKU: An Efficient Transformer for Multi-Task Policy Learning Siddhant Haldar Zhuoran Peng Lerrel Pinto OffRL 74 38 0 11 Jun 2024
Behavior Generation with Latent Actions Seungjae Lee Yibin Wang Haritheja Etukuru H. J. Kim Mahi Shafiullah Lerrel Pinto VGen OffRL 72 76 0 05 Mar 2024
Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models Yixin Liu Kai Zhang Yuan Li Zhiling Yan Chujie Gao ... Yue Huang Hanchi Sun Jianfeng Gao Lifang He Lichao Sun VLM VGen EGVM 101 288 0 27 Feb 2024
TD-MPC2: Scalable, Robust World Models for Continuous Control Nicklas Hansen Hao Su Xiaolong Wang MU 97 148 0 25 Oct 2023
Eureka: Human-Level Reward Design via Coding Large Language Models Yecheng Jason Ma William Liang Guanzhi Wang De-An Huang Osbert Bastani Dinesh Jayaraman Yuke Zhu Linxi Fan A. Anandkumar 63 314 0 19 Oct 2023
Learning to Act from Actionless Videos through Dense Correspondences Po-Chen Ko Jiayuan Mao Yilun Du Shao-Hua Sun Josh Tenenbaum 76 85 0 12 Oct 2023
Structured World Models from Human Videos Russell Mendonca Shikhar Bahl Deepak Pathak LM&Ro 85 94 0 21 Aug 2023
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control Anthony Brohan Noah Brown Justice Carbajal Yevgen Chebotar Xi Chen ... Ted Xiao Peng Xu Sichun Xu Tianhe Yu Brianna Zitkovich LM&Ro LRM 110 1,217 0 28 Jul 2023
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware Tony Zhao Vikash Kumar Sergey Levine Chelsea Finn 65 606 0 23 Apr 2023
Transformer-based World Models Are Happy With 100k Interactions Jan Robine Marc Höftmann Tobias Uelwer Stefan Harmeling OffRL 64 82 0 13 Mar 2023
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion Cheng Chi Zhenjia Xu S. Feng Eric A. Cousineau Yilun Du Benjamin Burchfiel Russ Tedrake Shuran Song 322 1,170 0 07 Mar 2023
ALAN: Autonomously Exploring Robotic Agents in the Real World Russell Mendonca Shikhar Bahl Deepak Pathak LM&Ro 54 21 0 13 Feb 2023
Learning Universal Policies via Text-Guided Video Generation Yilun Du Mengjiao Yang Bo Dai H. Dai Ofir Nachum J. Tenenbaum Dale Schuurmans Pieter Abbeel PINN LM&Ro 91 253 0 31 Jan 2023
Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture Mahmoud Assran Quentin Duval Ishan Misra Piotr Bojanowski Pascal Vincent Michael G. Rabbat Yann LeCun Nicolas Ballas SSL AI4TS MDE 66 346 0 19 Jan 2023
Mastering Diverse Domains through World Models Danijar Hafner J. Pašukonis Jimmy Ba Timothy Lillicrap 63 598 0 10 Jan 2023
RT-1: Robotics Transformer for Real-World Control at Scale Anthony Brohan Noah Brown Justice Carbajal Yevgen Chebotar Joseph Dabis ... Ted Xiao Peng Xu Sichun Xu Tianhe Yu Brianna Zitkovich LM&Ro 76 1,099 0 13 Dec 2022
Legged Locomotion in Challenging Terrains using Egocentric Vision Ananye Agarwal Ashish Kumar Jitendra Malik Deepak Pathak 68 216 0 14 Nov 2022
Transformers are Sample-Efficient World Models Vincent Micheli Eloi Alonso Franccois Fleuret VLM OffRL 107 179 0 01 Sep 2022
R3M: A Universal Visual Representation for Robot Manipulation Suraj Nair Aravind Rajeswaran Vikash Kumar Chelsea Finn Abhi Gupta LM&Ro 69 566 0 23 Mar 2022
Masked Visual Pre-training for Motor Control Tete Xiao Ilija Radosavovic Trevor Darrell Jitendra Malik SSL 77 246 0 11 Mar 2022
Temporal Difference Learning for Model Predictive Control Nicklas Hansen Xiaolong Wang H. Su PINN MU 79 241 0 09 Mar 2022
Discovering and Achieving Goals via World Models Russell Mendonca Oleh Rybkin Kostas Daniilidis Danijar Hafner Deepak Pathak 48 126 0 18 Oct 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 611 6,029 0 29 Apr 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 543 40,739 0 22 Oct 2020
Planning to Explore via Self-Supervised World Models Ramanan Sekar Oleh Rybkin Kostas Daniilidis Pieter Abbeel Danijar Hafner Deepak Pathak SSL 56 405 0 12 May 2020
D4RL: Datasets for Deep Data-Driven Reinforcement Learning Justin Fu Aviral Kumar Ofir Nachum George Tucker Sergey Levine GP OffRL 210 1,359 0 15 Apr 2020
Learning Predictive Representations for Deformable Objects Using Contrastive Estimation Wilson Yan Ashwin Vangipuram Pieter Abbeel Lerrel Pinto 75 190 0 11 Mar 2020
Dream to Control: Learning Behaviors by Latent Imagination Danijar Hafner Timothy Lillicrap Jimmy Ba Mohammad Norouzi VLM 108 1,349 0 03 Dec 2019
Learning to Manipulate Deformable Objects without Demonstrations Yilin Wu Wilson Yan Thanard Kurutach Lerrel Pinto Pieter Abbeel OffRL 65 201 0 29 Oct 2019
Deep Dynamics Models for Learning Dexterous Manipulation Anusha Nagabandi K. Konolige Sergey Levine Vikash Kumar 216 413 0 25 Sep 2019
Generating Diverse High-Fidelity Images with VQ-VAE-2 Ali Razavi Aaron van den Oord Oriol Vinyals DRL BDL 123 1,804 0 02 Jun 2019
Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control F. Ebert Chelsea Finn Sudeep Dasari Annie Xie Alex X. Lee Sergey Levine SSL 106 385 0 03 Dec 2018
Learning Latent Dynamics for Planning from Pixels Danijar Hafner Timothy Lillicrap Ian S. Fischer Ruben Villegas David R Ha Honglak Lee James Davidson BDL 84 1,430 0 12 Nov 2018
Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models Kurtland Chua Roberto Calandra R. McAllister Sergey Levine BDL 210 1,272 0 30 May 2018
Zero-Shot Visual Imitation Deepak Pathak Parsa Mahmoudieh Guanghao Luo Pulkit Agrawal Dian Chen Yide Shentu Evan Shelhamer Jitendra Malik Alexei A. Efros Trevor Darrell LM&Ro 105 299 0 23 Apr 2018
World Models David R Ha Jürgen Schmidhuber SyDa 113 1,079 0 27 Mar 2018
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric Richard Y. Zhang Phillip Isola Alexei A. Efros Eli Shechtman Oliver Wang EGVM 327 11,734 0 11 Jan 2018
Deep Visual Foresight for Planning Robot Motion Chelsea Finn Sergey Levine 111 783 0 03 Oct 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.1K 193,426 0 10 Dec 2015
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.6K 39,472 0 01 Sep 2014