Masked Generative Priors Improve World Models Sequence Modelling Capabilities

10 October 2024

Papers citing "Masked Generative Priors Improve World Models Sequence Modelling Capabilities"

50 / 55 papers shown

Title
$α$ -TCVAE: On the relationship between Disentanglement and Diversity Cristian Meo Louis Mahon Anirudh Goyal Justin Dauwels DRL 108 8 0 01 Nov 2024
iVideoGPT: Interactive VideoGPTs are Scalable World Models Jialong Wu Shaofeng Yin Ningya Feng Xu He Dong Li Haifeng Zhang Mingsheng Long VGen 85 34 0 24 May 2024
Extreme Precipitation Nowcasting using Transformer-based Generative Models Cristian Meo Ankush Roy Mircea Lica Junzhe Yin Zeineb Bou Che Yanbo Wang R. Imhoff R. Uijlenhoet Justin Dauwels 66 3 0 06 Mar 2024
VideoPoet: A Large Language Model for Zero-Shot Video Generation Dan Kondratyuk Lijun Yu Xiuye Gu José Lezama Jonathan Huang ... Irfan Essa Huisheng Wang David A. Ross Bryan Seybold Lu Jiang VGen 83 257 0 21 Dec 2023
4M: Massively Multimodal Masked Modeling David Mizrahi Roman Bachmann Ouguzhan Fatih Kar Teresa Yeo Mingfei Gao Afshin Dehghan Amir Zamir MLLM 73 67 0 11 Dec 2023
Reward Scale Robustness for Proximal Policy Optimization via DreamerV3 Tricks Ryan Sullivan Akarsh Kumar Shengyi Huang John P. Dickerson Joseph Suárez OffRL 33 5 0 26 Oct 2023
STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learning Weipu Zhang Gang Wang Jian Sun Yetian Yuan Gao Huang 69 42 0 14 Oct 2023
Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation Lijun Yu José Lezama N. B. Gundavarapu Luca Versari Kihyuk Sohn ... Boqing Gong Ming-Hsuan Yang Irfan Essa David A. Ross Lu Jiang 100 305 0 09 Oct 2023
GAIA-1: A Generative World Model for Autonomous Driving Masane Fuchi Lloyd Russell Hudson Yeo Zak Murez Hiroto Minami Alex Kendall Tomohiro Takagi Gianluca Corrado VGen 76 237 0 29 Sep 2023
Transformer-based World Models Are Happy With 100k Interactions Jan Robine Marc Höftmann Tobias Uelwer Stefan Harmeling OffRL 64 82 0 13 Mar 2023
Mastering Diverse Domains through World Models Danijar Hafner J. Pašukonis Jimmy Ba Timothy Lillicrap 63 598 0 10 Jan 2023
MAGVIT: Masked Generative Video Transformer Lijun Yu Yong Cheng Kihyuk Sohn José Lezama Han Zhang ... Alexander G. Hauptmann Ming-Hsuan Yang Yuan Hao Irfan Essa Lu Jiang DiffM VGen 54 239 0 10 Dec 2022
Stateful active facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learning Dianbo Liu Vedant Shah Oussama Boussif Cristian Meo Anirudh Goyal Tianmin Shu Michael C. Mozer N. Heess Yoshua Bengio 62 8 0 04 Oct 2022
Transformers are Sample-Efficient World Models Vincent Micheli Eloi Alonso Franccois Fleuret VLM OffRL 107 179 0 01 Sep 2022
Draft-and-Revise: Effective Image Generation with Contextual RQ-Transformer Doyup Lee Chiheon Kim Saehoon Kim Minsu Cho Wook-Shin Han 63 29 0 09 Jun 2022
TransDreamer: Reinforcement Learning with Transformer World Models Changgu Chen Yi-Fu Wu Jaesik Yoon Sungjin Ahn OffRL 76 94 0 19 Feb 2022
MaskGIT: Masked Generative Image Transformer Huiwen Chang Han Zhang Lu Jiang Ce Liu William T. Freeman ViT 112 672 0 08 Feb 2022
Survey of Hallucination in Natural Language Generation Ziwei Ji Nayeon Lee Rita Frieske Tiezheng Yu D. Su ... Delong Chen Wenliang Dai Ho Shu Chan Andrea Madotto Pascale Fung HILM LRM 186 2,356 0 08 Feb 2022
Controllable Animation of Fluid Elements in Still Images Aniruddha Mahapatra K. Kulkarni VGen 90 48 0 06 Dec 2021
Active Inference in Robotics and Artificial Agents: Survey and Challenges Pablo Lanillos Cristian Meo Corrado Pezzato A. Meera Mohamed Baioumy ... Alexander Tschantz Beren Millidge M. Wisse Christopher L. Buckley Jun Tani AI4CE 70 79 0 03 Dec 2021
NÜWA: Visual Synthesis Pre-training for Neural visUal World creAtion Chenfei Wu Jian Liang Lei Ji Fan Yang Yuejian Fang Daxin Jiang Nan Duan ViT VGen 59 294 0 24 Nov 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 427 7,705 0 11 Nov 2021
Benchmarking the Spectrum of Agent Capabilities Danijar Hafner ELM 62 136 0 14 Sep 2021
Deep Reinforcement Learning at the Edge of the Statistical Precipice Rishabh Agarwal Max Schwarzer Pablo Samuel Castro Aaron Courville Marc G. Bellemare OffRL 96 669 0 30 Aug 2021
Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning Denis Yarats Rob Fergus A. Lazaric Lerrel Pinto OffRL 75 346 0 20 Jul 2021
Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraft I. Kanitscheider Joost Huizinga David Farhi William H. Guss Brandon Houghton ... Bowen Baker Adrien Ecoffet Jie Tang Oleg Klimov Jeff Clune 50 21 0 28 Jun 2021
Offline Reinforcement Learning as One Big Sequence Modeling Problem Michael Janner Qiyang Li Sergey Levine OffRL 118 675 0 03 Jun 2021
Decision Transformer: Reinforcement Learning via Sequence Modeling Lili Chen Kevin Lu Aravind Rajeswaran Kimin Lee Aditya Grover Michael Laskin Pieter Abbeel A. Srinivas Igor Mordatch OffRL 107 1,635 0 02 Jun 2021
M6-UFC: Unifying Multi-Modal Controls for Conditional Image Synthesis via Non-Autoregressive Generative Transformers Zhu Zhang Jianxin Ma Chang Zhou Rui Men Zhikang Li Ming Ding Jie Tang Jingren Zhou Hongxia Yang 67 46 0 29 May 2021
VideoGPT: Video Generation using VQ-VAE and Transformers Wilson Yan Yunzhi Zhang Pieter Abbeel A. Srinivas ViT VGen 295 498 0 20 Apr 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 543 40,739 0 22 Oct 2020
Mastering Atari with Discrete World Models Danijar Hafner Timothy Lillicrap Mohammad Norouzi Jimmy Ba DRL 95 849 0 05 Oct 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 682 41,736 0 28 May 2020
Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels Ilya Kostrikov Denis Yarats Rob Fergus OffRL 92 789 0 28 Apr 2020
CURL: Contrastive Unsupervised Representations for Reinforcement Learning A. Srinivas Michael Laskin Pieter Abbeel SSL DRL OffRL 81 1,084 0 08 Apr 2020
Decision-Making with Auto-Encoding Variational Bayes Romain Lopez Pierre Boyeau Nir Yosef Michael I. Jordan Jeffrey Regier BDL 346 10,591 0 17 Feb 2020
Dream to Control: Learning Behaviors by Latent Imagination Danijar Hafner Timothy Lillicrap Jimmy Ba Mohammad Norouzi VLM 108 1,349 0 03 Dec 2019
Leveraging Procedural Generation to Benchmark Reinforcement Learning K. Cobbe Christopher Hesse Jacob Hilton John Schulman 72 554 0 03 Dec 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 379 20,053 0 23 Oct 2019
Model-Based Reinforcement Learning for Atari Lukasz Kaiser Mohammad Babaeizadeh Piotr Milos B. Osinski R. Campbell ... Sergey Levine Afroz Mohiuddin Ryan Sepassi George Tucker Henryk Michalewski OffRL 112 859 0 01 Mar 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 200 3,724 0 09 Jan 2019
Learning Latent Dynamics for Planning from Pixels Danijar Hafner Timothy Lillicrap Ian S. Fischer Ruben Villegas David R Ha Honglak Lee James Davidson BDL 84 1,430 0 12 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.6K 94,511 0 11 Oct 2018
Recurrent World Models Facilitate Policy Evolution David R Ha Jürgen Schmidhuber SyDa TPM 117 937 0 04 Sep 2018
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor Tuomas Haarnoja Aurick Zhou Pieter Abbeel Sergey Levine 284 8,313 0 04 Jan 2018
DeepMind Control Suite Yuval Tassa Yotam Doron Alistair Muldal Tom Erez Yazhe Li ... A. Abdolmaleki J. Merel Andrew Lefrancq Timothy Lillicrap Martin Riedmiller ELM LM&Ro BDL 127 1,133 0 02 Jan 2018
Neural Discrete Representation Learning Aaron van den Oord Oriol Vinyals Koray Kavukcuoglu BDL SSL OCL 208 4,989 0 02 Nov 2017
Proximal Policy Optimization Algorithms John Schulman Filip Wolski Prafulla Dhariwal Alec Radford Oleg Klimov OffRL 444 18,931 0 20 Jul 2017
The "something something" video database for learning and evaluating visual common sense Raghav Goyal Samira Ebrahimi Kahou Vincent Michalski Joanna Materzynska S. Westphal ... Moritz Mueller-Freitag F. Hoppe Christian Thurau Ingo Bax Roland Memisevic VLM 82 1,529 0 13 Jun 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 644 130,942 0 12 Jun 2017