Learning Interactive Real-World Simulators

9 October 2023

Pieter Abbeel

Papers citing "Learning Interactive Real-World Simulators"

50 / 139 papers shown

Title
FlowDreamer: A RGB-D World Model with Flow-based Motion Representations for Robot Manipulation Jun Guo Xiaojian Ma Yikai Wang Min Yang Huaping Liu Qing Li VGen 32 0 0 15 May 2025
EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Models Hu Yue Siyuan Huang Yue Liao Shengcong Chen Pengfei Zhou Liliang Chen Maoqing Yao Guanghui Ren VGen 29 0 0 14 May 2025
EnerVerse-AC: Envisioning Embodied Environments with Action Condition Yuxin Jiang Shengcong Chen Siyuan Huang Liliang Chen Pengfei Zhou ... Xindong He Chiming Liu Hongsheng Li Maoqing Yao Guanghui Ren 21 0 0 14 May 2025
ReinboT: Amplifying Robot Visual-Language Manipulation with Reinforcement Learning Hongyin Zhang Zifeng Zhuang H. Zhao Pengxiang Ding Hongchao Lu Donglin Wang OffRL 44 0 0 12 May 2025
Learning 3D Persistent Embodied World Models Siyuan Zhou Yilun Du Yuncong Yang Lei Han Peihao Chen Dit-Yan Yeung Chuang Gan VGen 47 0 0 05 May 2025
CrayonRobo: Object-Centric Prompt-Driven Vision-Language-Action Model for Robotic Manipulation Xiaoqi Li Lingyun Xu M. Zhang Jiaming Liu Yan Shen ... Jiahui Xu Liang Heng Siyuan Huang S. Zhang Hao Dong LM&Ro 51 0 0 04 May 2025
A Survey of Interactive Generative Video Jiwen Yu Yiran Qin Haoxuan Che Quande Liu Xinyu Wang Pengfei Wan Di Zhang Kun Gai Hao Chen Xihui Liu VGen 65 0 0 30 Apr 2025
Latent Diffusion Planning for Imitation Learning Amber Xie Oleh Rybkin Dorsa Sadigh Chelsea Finn 35 1 0 23 Apr 2025
Solving New Tasks by Adapting Internet Video Knowledge Calvin Luo Zilai Zeng Yilun Du Chen Sun 25 0 0 21 Apr 2025
DRAWER: Digital Reconstruction and Articulation With Environment Realism Hongchi Xia Entong Su Marius Memmel Arhan Jain Raymond Yu Numfor Mbiziwo-Tiapo Ali Farhadi Abhishek Gupta Shenlong Wang Wei-Chiu Ma VGen 30 1 0 21 Apr 2025
MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft Junliang Guo Yang Ye Tianyu He Haoyu Wu Yushu Jiang Tim Pearce Jiang Bian VGen SyDa 56 2 0 11 Apr 2025
Diffusion Models for Robotic Manipulation: A Survey Rosa Wolf Yitian Shi Sheng Liu Rania Rayyes 51 1 0 11 Apr 2025
Real-is-Sim: Bridging the Sim-to-Real Gap with a Dynamic Digital Twin for Real-World Robot Policy Evaluation Jad Abou-Chakra Lingfeng Sun Krishan Rana Brandon B. May Karl Schmeckpeper M. Minniti Laura Herlant OffRL 137 0 0 04 Apr 2025
Exploration-Driven Generative Interactive Environments N. Savov Naser Kazemi Mohammad Mahdi Danda Pani Paudel Xi Wang Luc Van Gool VGen 3DV 43 0 0 03 Apr 2025
VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Prior Xindi Yang Baolu Li Yuhang Zhang Zhenfei Yin Lei Bai ... Zhiyong Wang Jianfei Cai Tien-Tsin Wong Huchuan Lu Xu Jia DiffM VGen 49 0 0 30 Mar 2025
Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environments Luke Rowe Roger Girgis Anthony Gosselin Liam Paull C. Pal Felix Heide DiffM VGen 38 1 0 28 Mar 2025
CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models Qingqing Zhao Yao Lu Moo Jin Kim Zipeng Fu Zhuoyang Zhang ... Ankur Handa Xuan Li Donglai Xiang Gordon Wetzstein Nayeon Lee LM&Ro LRM 43 11 0 27 Mar 2025
Exploring the Evolution of Physics Cognition in Video Generation: A Survey Minghui Lin Xiang Wang Yixuan Wang Shu Wang Fengqi Dai ... Cunxiang Wang Zhengrong Zuo Nong Sang Siteng Huang Donglin Wang EGVM VGen 87 3 0 27 Mar 2025
AdaWorld: Learning Adaptable World Models with Latent Actions Shenyuan Gao Siyuan Zhou Yilun Du Jun Zhang Chuang Gan VGen 62 3 0 24 Mar 2025
DyWA: Dynamics-adaptive World Action Model for Generalizable Non-prehensile Manipulation Jiangran Lyu Ziming Li Xuesong Shi Chaoyi Xu Yizhou Wang He Wang 49 0 0 21 Mar 2025
Position: Interactive Generative Video as Next-Generation Game Engine Jiwen Yu Yiran Qin Haoxuan Che Quande Liu Xintao Wang Pengfei Wan Di Zhang Xihui Liu VGen 45 1 0 21 Mar 2025
Missing Target-Relevant Information Prediction with World Model for Accurate Zero-Shot Composed Image Retrieval Yuanmin Tang Jing Yu Keke Gai Jiamin Zhuang Gang Xiong Gaopeng Gou Qi Wu VGen 49 1 0 21 Mar 2025
Fast Autoregressive Video Generation with Diagonal Decoding Yang Ye Junliang Guo Haoyu Wu Tianyu He Tim Pearce Tabish Rashid Katja Hofmann Jiang Bian DiffM VGen 78 1 0 18 Mar 2025
Being-0: A Humanoid Robotic Agent with Vision-Language Models and Modular Skills Haoqi Yuan Yu Bai Yuhui Fu Bohan Zhou Yicheng Feng Xinrun Xu Yi Zhan Börje F. Karlsson Zongqing Lu LM&Ro 88 0 0 16 Mar 2025
Diffusion Dynamics Models with Generative State Estimation for Cloth Manipulation Tongxuan Tian Haoyang Li Bo Ai Xiaodi Yuan Zhiao Huang H. Su DiffM AI4CE 68 3 0 15 Mar 2025
PISA Experiments: Exploring Physics Post-Training for Video Diffusion Models by Watching Stuff Drop Chenyu Li Oscar Michel Xichen Pan Sainan Liu Mike Roberts Saining Xie VGen 55 3 0 12 Mar 2025
LuciBot: Automated Robot Policy Learning from Generated Videos Xiaowen Qiu Yian Wang Jiting Cai Zhehuan Chen Chunru Lin Tsun-Hsuan Wang Chuang Gan LM&Ro VGen 72 0 0 12 Mar 2025
Object-Centric World Model for Language-Guided Manipulation Youngjoon Jeong Junha Chun S. Cha Taesup Kim OCL VGen 152 1 0 08 Mar 2025
Generative Artificial Intelligence in Robotic Manipulation: A Survey Anton van den Hengel Peng Yun Jun Cen Junhao Cai DiDi Zhu ... Qifeng Chen Jia Pan Wei Zhang Bo Yang Hua Chen 59 1 0 05 Mar 2025
What Makes a Good Diffusion Planner for Decision Making? Haofei Lu Dongqi Han Yifei Shen Dongsheng Li DiffM 38 3 0 01 Mar 2025
SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation Zekun Qi Wenyao Zhang Yufei Ding Runpei Dong Xinqiang Yu ... Xin Jin Kaisheng Ma Zhizheng Zhang He Wang Li Yi LM&Ro 131 4 0 18 Feb 2025
Learning Human Skill Generators at Key-Step Levels Yilu Wu Chenhui Zhu Shuai Wang Hanlin Wang Jing Wang Zhaoxiang Zhang Limin Wang VGen 119 0 0 12 Feb 2025
VILP: Imitation Learning with Latent Video Planning Zhengtong Xu Qiang Qiu Yu She VGen 75 1 0 03 Feb 2025
Strengthening Generative Robot Policies through Predictive World Modeling Han Qi Haocheng Yin Yilun Du Heng Yang 63 2 0 02 Feb 2025
RLS3: RL-Based Synthetic Sample Selection to Enhance Spatial Reasoning in Vision-Language Models for Indoor Autonomous Perception Joshua R. Waite Md Zahid Hasan Qisai Liu Zhanhong Jiang Chinmay Hegde S. Sarkar OffRL SyDa 175 1 0 31 Jan 2025
Environment Descriptions for Usability and Generalisation in Reinforcement Learning Dennis J. N. J. Soemers Spyridon Samothrakis Kurt Driessens M. Winands OffRL 82 1 0 22 Dec 2024
Learning from Massive Human Videos for Universal Humanoid Pose Control Jiageng Mao Siheng Zhao Siqi Song Tianheng Shi Junjie Ye Mingtong Zhang Haoran Geng Jitendra Malik Vitor Campagnolo Guizilini Yue Wang 93 5 0 18 Dec 2024
GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Control Mariam Hassan Sebastian Stapf Ahmad Rahimi Pedro M B Rezende Yasaman Haghighi ... Mathieu Salzmann Davide Scaramuzza Marc Pollefeys Paolo Favaro Alexandre Alahi VLM VGen 71 5 0 15 Dec 2024
ManiSkill-HAB: A Benchmark for Low-Level Manipulation in Home Rearrangement Tasks Arth Shukla Stone Tao Hao Su 91 6 0 09 Dec 2024
The Matrix: Infinite-Horizon World Generation with Real-Time Moving Control Ruili Feng Han Zhang Zhantao Yang Jie Xiao Zhilei Shu Zhiheng Liu Andy Zheng Yukun Huang Yu Liu H. Zhang VGen 92 9 0 04 Dec 2024
Navigation World Models Amir Bar G. Zhou Danny Tran Trevor Darrell Yann LeCun VGen EgoV 82 14 0 04 Dec 2024
PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generation Qiyao Xue Xiangyu Yin Boyuan Yang Wei Gao DiffM VGen 80 9 0 30 Nov 2024
EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Video Generation Xiaofeng Wang Kang Zhao F. Liu Jiayu Wang Guosheng Zhao Xiaoyi Bao Zheng Hua Zhu Yingya Zhang Xingang Wang VGen 56 6 0 13 Nov 2024
World Models: The Safety Perspective Zifan Zeng Chongzhe Zhang Feng Liu Joseph Sifakis Qunli Zhang Shiming Liu Peng Wang KELM LLMAG 42 1 0 12 Nov 2024
Grounding Video Models to Actions through Goal Conditioned Exploration Yunhao Luo Yilun Du LM&Ro VGen 85 1 0 11 Nov 2024
Scaling Laws for Pre-training Agents and World Models Tim Pearce Tabish Rashid Dave Bignell Raluca Georgescu Sam Devlin Katja Hofmann LM&Ro 40 6 0 07 Nov 2024
AutoVFX: Physically Realistic Video Editing from Natural Language Instructions Hao-Yu Hsu Zhi-Hao Lin Albert Zhai Hongchi Xia Shenlong Wang VGen 48 9 0 04 Nov 2024
GameGen-X: Interactive Open-world Game Video Generation Haoxuan Che Xuanhua He Quande Liu C. Jin Hao Chen VGen 62 17 0 01 Nov 2024
Bridging the Human to Robot Dexterity Gap through Object-Oriented Rewards Irmak Güzey Yinlong Dai Georgy Savva Raunaq M. Bhirangi Lerrel Pinto 43 7 0 30 Oct 2024
SlowFast-VGen: Slow-Fast Learning for Action-Driven Long Video Generation Yining Hong Beide Liu Maxine Wu Yuanhao Zhai Kai-Wei Chang ... Chung-Ching Lin Jianfeng Wang Z. Yang Yingnian Wu Lijuan Wang VGen 42 6 0 30 Oct 2024