Video as the New Language for Real-World Decision Making

27 February 2024

Pieter Abbeel

Papers citing "Video as the New Language for Real-World Decision Making"

27 / 27 papers shown

Title
Video-GPT via Next Clip Diffusion Shaobin Zhuang Zhipeng Huang Ying Zhang Fangyikang Wang Canmiao Fu Binxin Yang Chong Sun Chen Li Yali Wang DiffM VGen 127 0 0 18 May 2025
Solving New Tasks by Adapting Internet Video Knowledge Calvin Luo Zilai Zeng Yilun Du Chen Sun 60 2 0 21 Apr 2025
Grounding Video Models to Actions through Goal Conditioned Exploration Yunhao Luo Yilun Du LM&Ro VGen 99 2 0 11 Nov 2024
Latent Action Pretraining from Videos Seonghyeon Ye Joel Jang Byeongguk Jeon Sejune Joo Jianwei Yang ... Kimin Lee J. Gao Luke Zettlemoyer Dieter Fox Minjoon Seo 56 34 0 15 Oct 2024
VideoAgent: Self-Improving Video Generation Achint Soni Sreyas Venkataraman Abhranil Chandra Sebastian Fischmeister Percy Liang Bo Dai Sherry Yang LM&Ro VGen 65 8 0 14 Oct 2024
MarS: a Financial Market Simulation Engine Powered by Generative Foundation Model Junjie Li Yang Liu Weiqing Liu Shikai Fang Lewen Wang Chang Xu Jiang Bian VGen 63 4 0 04 Sep 2024
Video In-context Learning: Autoregressive Transformers are Zero-Shot Video Imitators Wentao Zhang Junliang Guo Tianyu He Li Zhao Linli Xu Jiang Bian 63 3 0 10 Jul 2024
ARDuP: Active Region Video Diffusion for Universal Policies Shuaiyi Huang Mara Levy Zhenyu Jiang Anima Anandkumar Yuke Zhu Linxi Fan De-An Huang Abhinav Shrivastava VGen 59 2 0 19 Jun 2024
Do's and Don'ts: Learning Desirable Skills with Instruction Videos Hyunseung Kim ByungKun Lee Hojoon Lee Dongyoon Hwang Donghu Kim Jaegul Choo 94 1 0 01 Jun 2024
GenHowTo: Learning to Generate Actions and State Transformations from Instructional Videos Tomávs Souvcek Dima Damen Michael Wray Ivan Laptev Josef Sivic VGen 33 20 0 12 Dec 2023
GAIA-1: A Generative World Model for Autonomous Driving Masane Fuchi Lloyd Russell Hudson Yeo Zak Murez Hiroto Minami Alex Kendall Tomohiro Takagi Gianluca Corrado VGen 70 227 0 29 Sep 2023
Images Speak in Images: A Generalist Painter for In-Context Visual Learning Xinlong Wang Wen Wang Yue Cao Chunhua Shen Tiejun Huang VLM MLLM 84 249 0 05 Dec 2022
Interactive Language: Talking to Robots in Real Time Corey Lynch Ayzaan Wahid Jonathan Tompson Tianli Ding James Betker Robert Baruch Travis Armstrong Peter R. Florence LM&Ro 56 218 0 12 Oct 2022
Classifier-Free Diffusion Guidance Jonathan Ho Tim Salimans FaML 80 3,786 0 26 Jul 2022
Video Diffusion Models Jonathan Ho Tim Salimans Alexey A. Gritsenko William Chan Mohammad Norouzi David J. Fleet DiffM VGen 138 1,563 0 07 Apr 2022
MaskGIT: Masked Generative Image Transformer Huiwen Chang Han Zhang Lu Jiang Ce Liu William T. Freeman ViT 83 656 0 08 Feb 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 570 9,009 0 28 Jan 2022
Mastering Atari with Discrete World Models Danijar Hafner Timothy Lillicrap Mohammad Norouzi Jimmy Ba DRL 71 834 0 05 Oct 2020
Learning to Simulate Dynamic Environments with GameGAN Seung Wook Kim Yuhao Zhou Jonah Philion Antonio Torralba Sanja Fidler GAN 41 102 0 25 May 2020
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 133 3,707 0 09 Jan 2019
Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation Niels Justesen R. Torrado Philip Bontrager Ahmed Khalifa Julian Togelius S. Risi 104 181 0 28 Jun 2018
Learning what you can do before doing anything Oleh Rybkin Karl Pertsch Konstantinos G. Derpanis Kostas Daniilidis Andrew Jaegle SSL 39 21 0 25 Jun 2018
Imitating Latent Policies from Observation Ashley D. Edwards Himanshu Sahni Yannick Schroecker Charles Isbell 53 137 0 21 May 2018
Neural Discrete Representation Learning Aaron van den Oord Oriol Vinyals Koray Kavukcuoglu BDL SSL OCL 154 4,928 0 02 Nov 2017
Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World Joshua Tobin Rachel Fong Alex Ray Jonas Schneider Wojciech Zaremba Pieter Abbeel 169 2,948 0 20 Mar 2017
The Predictron: End-To-End Learning and Planning David Silver H. V. Hasselt Matteo Hessel Tom Schaul A. Guez ... Gabriel Dulac-Arnold David P. Reichert Neil C. Rabinowitz André Barreto T. Degris 45 289 0 28 Dec 2016
The Arcade Learning Environment: An Evaluation Platform for General Agents Marc G. Bellemare Yavar Naddaf J. Veness Michael Bowling 70 2,992 0 19 Jul 2012