Masked World Models for Visual Control

28 June 2022

Pieter Abbeel

Papers citing "Masked World Models for Visual Control"

50 / 117 papers shown

Title
DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation Guosheng Zhao Xiaofeng Wang Zheng Zhu Xinze Chen Guan Huang Xiaoyi Bao Xingang Wang VGen 40 14 0 11 Mar 2024
Spatiotemporal Predictive Pre-training for Robotic Motor Control Jiange Yang Bei Liu Jianlong Fu Bocheng Pan Gangshan Wu Limin Wang 42 10 0 08 Mar 2024
3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations Yanjie Ze Gu Zhang Kangning Zhang Chenyuan Hu Muhan Wang Huazhe Xu VGen 39 77 0 06 Mar 2024
DINOBot: Robot Manipulation via Retrieval and Alignment with Vision Foundation Models Norman Di Palo Edward Johns LM&Ro 40 25 0 20 Feb 2024
Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss Ruijie Zheng Yongyuan Liang Xiyao Wang Shuang Ma Hal Daumé Huazhe Xu John Langford Praveen Palanisamy Kalyan Shankar Basu Furong Huang 40 5 0 09 Feb 2024
WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens Xiaofeng Wang Zheng Zhu Guan Huang Boyuan Wang Xinze Chen Jiwen Lu VGen 34 32 0 18 Jan 2024
Unsupervised Salient Patch Selection for Data-Efficient Reinforcement Learning Zhaohui Jiang Paul Weng OffRL 25 0 0 10 Jan 2024
Gradient-based Planning with World Models V. JyothirS Siddhartha Jalagam Yann LeCun Vlad Sobal 34 4 0 28 Dec 2023
Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation Hongtao Wu Ya Jing Chi-Hou Cheang Guangzeng Chen Jiafeng Xu Xinghang Li Minghuan Liu Hang Li Tao Kong 27 94 0 20 Dec 2023
Value Explicit Pretraining for Learning Transferable Representations Kiran Lekkala Henghui Bao S. Sontakke Laurent Itti SSL 32 0 0 19 Dec 2023
ReCoRe: Regularized Contrastive Representation Learning of World Model Rudra P. K. Poudel Harit Pandya Stephan Liwicki Roberto Cipolla DRL OffRL 28 6 0 14 Dec 2023
FoMo Rewards: Can we cast foundation models as reward functions? Ekdeep Singh Lubana Johann Brehmer P. D. Haan Taco S. Cohen OffRL LRM 48 2 0 06 Dec 2023
LanGWM: Language Grounded World Model Rudra P. K. Poudel Harit Pandya Chao Zhang Roberto Cipolla 30 5 0 29 Nov 2023
ADriver-I: A General World Model for Autonomous Driving Fan Jia Weixin Mao Yingfei Liu Yucheng Zhao Yuqing Wen Chi Zhang Xiangyu Zhang Tiancai Wang 42 63 0 22 Nov 2023
Provable Representation with Efficient Planning for Partial Observable Reinforcement Learning Hongming Zhang Tongzheng Ren Chenjun Xiao Dale Schuurmans Bo Dai 45 3 0 20 Nov 2023
Imitation Bootstrapped Reinforcement Learning Hengyuan Hu Suvir Mirchandani Dorsa Sadigh 41 24 0 03 Nov 2023
DreamSmooth: Improving Model-based Reinforcement Learning via Reward Smoothing Vint Lee Pieter Abbeel Youngwoon Lee 24 5 0 02 Nov 2023
The Power of the Senses: Generalizable Manipulation from Vision and Touch through Masked Multimodal Learning Carmelo Sferrazza Younggyo Seo Hao Liu Youngwoon Lee Pieter Abbeel 46 15 0 02 Nov 2023
Towards Control-Centric Representations in Reinforcement Learning from Images Chen Liu Hongyu Zang Xin Li Yong Heng Yifei Wang Zhen Fang Yisen Wang Mingzhong Wang 20 0 0 25 Oct 2023
Robust Visual Imitation Learning with Inverse Dynamics Representations Siyuan Li Xun Wang Rongchang Zuo Kewu Sun Lingfei Cui Jishiyu Ding Peng Liu Zhe Ma 26 3 0 22 Oct 2023
HarmonyDream: Task Harmonization Inside World Models Haoyu Ma Jialong Wu Ningya Feng Chenjun Xiao Dong Li Jianye Hao Jianmin Wang Mingsheng Long 35 7 0 30 Sep 2023
Robotic Offline RL from Internet Videos via Value-Function Pre-Training Chethan Bhateja Derek Guo Dibya Ghosh Anika Singh Manan Tomar Q. Vuong Yevgen Chebotar Sergey Levine Aviral Kumar OffRL 36 20 0 22 Sep 2023
Sequential Action-Induced Invariant Representation for Reinforcement Learning Dayang Liang Qihang Chen Yunlong Liu 24 4 0 22 Sep 2023
Guide Your Agent with Adaptive Multimodal Rewards Changyeon Kim Younggyo Seo Hao Liu Lisa Lee Jinwoo Shin Honglak Lee Kimin Lee 23 9 0 19 Sep 2023
DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving Xiaofeng Wang Zheng Hua Zhu Guan Huang Xinze Chen Jiagang Zhu Jiwen Lu VGen 22 148 0 18 Sep 2023
RMP: A Random Mask Pretrain Framework for Motion Prediction Yi Yang Qingwen Zhang Thomas Gilles Nazre Batool John Folkesson 53 5 0 16 Sep 2023
Language Reward Modulation for Pretraining Reinforcement Learning Ademi Adeniji Amber Xie Carmelo Sferrazza Younggyo Seo Stephen James Pieter Abbeel 39 26 0 23 Aug 2023
Transformers in Reinforcement Learning: A Survey Pranav Agarwal A. Rahman P. St-Charles Simon J. D. Prince Samira Ebrahimi Kahou OffRL 24 18 0 12 Jul 2023
SpawnNet: Learning Generalizable Visuomotor Skills from Pre-trained Networks Xingyu Lin John So Sashwat Mahalingam Fangchen Liu Pieter Abbeel SSL 27 22 0 07 Jul 2023
Goal-Conditioned Predictive Coding for Offline Reinforcement Learning Zilai Zeng Ce Zhang Shijie Wang Chen Sun OffRL 29 5 0 07 Jul 2023
FOCUS: Object-Centric World Models for Robotics Manipulation Stefano Ferraro Pietro Mazzaglia Tim Verbelen Bart Dhoedt OCL LM&Ro 40 13 0 05 Jul 2023
Facing Off World Model Backbones: RNNs, Transformers, and S4 Fei Deng Junyeong Park Sungjin Ahn 32 24 0 05 Jul 2023
VIBR: Learning View-Invariant Value Functions for Robust Visual Control Tom Dupuis Jaonary Rabarisoa Q. C. Pham David Filliat 36 0 0 14 Jun 2023
Masked Autoencoders with Multi-Window Local-Global Attention Are Better Audio Learners Sarthak Yadav Sergios Theodoridis Lars Kai Hansen Zheng-Hua Tan 28 7 0 01 Jun 2023
Accelerating Reinforcement Learning with Value-Conditional State Entropy Exploration Dongyoung Kim Jinwoo Shin Pieter Abbeel Younggyo Seo 21 19 0 31 May 2023
Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning Jialong Wu Haoyu Ma Chao Deng Mingsheng Long OffRL 28 24 0 29 May 2023
Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitation David Brandfonbrener Ofir Nachum Joan Bruna AI4CE 26 21 0 26 May 2023
A Reminder of its Brittleness: Language Reward Shaping May Hinder Learning for Instruction Following Agents Sukai Huang N. Lipovetzky Trevor Cohn 30 2 0 26 May 2023
Emergent Agentic Transformer from Chain of Hindsight Experience Hao Liu Pieter Abbeel OffRL 32 25 0 26 May 2023
Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning Guozheng Ma Linrui Zhang Haoyu Wang Lu Li Zilin Wang Zhen Wang Li Shen Xueqian Wang Dacheng Tao 42 10 0 25 May 2023
Masked Path Modeling for Vision-and-Language Navigation Zi-Yi Dou Feng Gao Nanyun Peng LM&Ro 31 3 0 23 May 2023
MIMEx: Intrinsic Rewards from Masked Input Modeling Toru Lin Allan Jabri OffRL 23 6 0 15 May 2023
Masked Trajectory Models for Prediction, Representation, and Control Philipp Wu Arjun Majumdar Kevin Stone Yixin Lin Igor Mordatch Pieter Abbeel Aravind Rajeswaran OffRL 36 38 0 04 May 2023
Reinforcement Learning from Passive Data via Latent Intentions Dibya Ghosh Chethan Bhateja Sergey Levine OffRL 23 42 0 10 Apr 2023
UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learning Weikang Wan Haoran Geng Yun-Hai Liu Zikang Shan Yaodong Yang Li Yi He-Nan Wang 47 94 0 02 Apr 2023
PartManip: Learning Cross-Category Generalizable Part Manipulation Policy from Point Cloud Observations Haoran Geng Ziming Li Yiran Geng Jiayi Chen Hao Dong He-Nan Wang 3DPC 45 41 0 29 Mar 2023
Cross-Modal Causal Intervention for Medical Report Generation Weixing Chen Yang Liu Ce Wang Jiarui Zhu Shen Zhao Guanbin Li Cheng-Lin Liu Liang Lin 34 6 0 16 Mar 2023
OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNav Karmesh Yadav Arjun Majumdar Ram Ramrakhya Naoki Yokoyama Alexei Baevski Z. Kira Oleksandr Maksymets Dhruv Batra ViT 23 45 0 14 Mar 2023
Foundation Models for Decision Making: Problems, Methods, and Opportunities Sherry Yang Ofir Nachum Yilun Du Jason W. Wei Pieter Abbeel Dale Schuurmans LM&Ro OffRL LRM AI4CE 92 155 0 07 Mar 2023
Combining Reconstruction and Contrastive Methods for Multimodal Representations in RL P. Becker Sebastian Mossburger Fabian Otto Gerhard Neumann SSL 34 2 0 10 Feb 2023