Learning Interactive Real-World Simulators

9 October 2023

Pieter Abbeel

Papers citing "Learning Interactive Real-World Simulators"

39 / 139 papers shown

Title
Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search Nicola Dainese Matteo Merler Minttu Alakuijala Pekka Marttinen LLMAG 41 8 0 24 May 2024
iVideoGPT: Interactive VideoGPTs are Scalable World Models Jialong Wu Shaofeng Yin Ningya Feng Xu He Dong Li Jianye Hao Mingsheng Long VGen 46 22 0 24 May 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 82 42 0 23 May 2024
A Survey of Robotic Language Grounding: Tradeoffs between Symbols and Embeddings Vanya Cohen J. Liu Raymond J. Mooney Stefanie Tellex David Watkins LM&Ro 43 12 0 21 May 2024
From LLMs to Actions: Latent Codes as Bridges in Hierarchical Robot Control Yide Shentu Philipp Wu Aravind Rajeswaran Pieter Abbeel 37 9 0 08 May 2024
Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond Zheng Zhu Xiaofeng Wang Wangbo Zhao Chen Min Nianchen Deng ... Dawei Zhao Liang Xiao Jian-jun Zhao Jiwen Lu Guan Huang VGen LM&Ro 87 37 0 06 May 2024
Video Diffusion Models: A Survey Andrew Melnik Michal Ljubljanac Cong Lu Qi Yan Weiming Ren Helge J. Ritter VGen 71 12 0 06 May 2024
Track2Act: Predicting Point Tracks from Internet Videos enables Diverse Zero-shot Robot Manipulation Homanga Bharadhwaj Roozbeh Mottaghi Abhinav Gupta Shubham Tulsiani 3DPC 51 16 0 02 May 2024
LEGENT: Open Platform for Embodied Agents Zhili Cheng Zhitong Wang Jinyi Hu Shengding Hu An Liu Yuge Tu Pengkai Li Lei Shi Zhiyuan Liu Maosong Sun VLM 30 6 0 28 Apr 2024
WorldGPT: Empowering LLM as Multimodal World Model Zhiqi Ge Hongzhe Huang Mingze Zhou Juncheng Li Guoming Wang Siliang Tang Yueting Zhuang 40 26 0 28 Apr 2024
Ag2Manip: Learning Novel Manipulation Skills with Agent-Agnostic Visual and Action Representations Puhao Li Tengyu Liu Yuyang Li Muzhi Han Haoran Geng Shu Wang Yixin Zhu Song-Chun Zhu Siyuan Huang 39 16 0 26 Apr 2024
RoboDreamer: Learning Compositional World Models for Robot Imagination Siyuan Zhou Yilun Du Jiaben Chen Yandong Li Dit-Yan Yeung Chuang Gan VGen LM&Ro 82 29 0 18 Apr 2024
Automated Discovery of Functional Actual Causes in Complex Environments Caleb Chuck Sankaran Vaidyanathan Stephen Giguere Amy Zhang David Jensen S. Niekum CML 44 2 0 16 Apr 2024
COMBO: Compositional World Models for Embodied Multi-Agent Cooperation Hongxin Zhang Zeyuan Wang Qiushi Lyu Zheyuan Zhang Sunli Chen Tianmin Shu Yilun Du Kwonjoon Lee Yilun Du Chuang Gan 48 12 0 16 Apr 2024
Generalized Predictive Model for Autonomous Driving Jiazhi Yang Shenyuan Gao Yihang Qiu Li Chen Tianyu Li ... Ping Luo Jun Zhang Andreas Geiger Yu Qiao Hongyang Li VGen 73 57 0 14 Mar 2024
Scaling Instructable Agents Across Many Simulated Worlds Sima Team Maria Abi Raad Arun Ahuja Catarina Barros F. Besse ... Daan Wierstra Duncan Williams Nathaniel Wong Sarah York Nick Young LM&Ro 115 38 0 13 Mar 2024
Learning and Leveraging World Models in Visual Representation Learning Q. Garrido Mahmoud Assran Nicolas Ballas Adrien Bardes Laurent Najman Yann LeCun SSL 46 24 0 01 Mar 2024
Pushing the Limits of Cross-Embodiment Learning for Manipulation and Navigation Jonathan Yang Catherine Glossop Arjun Bhorkar Dhruv Shah Quan Vuong Chelsea Finn Dorsa Sadigh Sergey Levine 41 41 0 29 Feb 2024
Video as the New Language for Real-World Decision Making Sherry Yang Jacob Walker Jack Parker-Holder Yilun Du Jake Bruce Andre Barreto Pieter Abbeel Dale Schuurmans VGen 29 46 0 27 Feb 2024
Genie: Generative Interactive Environments Jake Bruce Michael Dennis Ashley D. Edwards Jack Parker-Holder Yuge Shi ... Konrad Zolna Jeff Clune Nando de Freitas Satinder Singh Tim Rocktaschel VGen VLM 74 144 0 23 Feb 2024
Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Following Brian Yang Huangyuan Su N. Gkanatsios Tsung-Wei Ke Ayush Jain Jeff Schneider Katerina Fragkiadaki DiffM 37 20 0 09 Feb 2024
Real-World Robot Applications of Foundation Models: A Review Kento Kawaharazuka T. Matsushima Andrew Gambardella Jiaxian Guo Chris Paxton Andy Zeng OffRL VLM LM&Ro 48 45 0 08 Feb 2024
Code as Reward: Empowering Reinforcement Learning with VLMs David Venuto Sami Nur Islam Martin Klissarov Doina Precup Sherry Yang Ankit Anand VLM 28 9 0 07 Feb 2024
"Task Success" is not Enough: Investigating the Use of Video-Language Models as Behavior Critics for Catching Undesirable Agent Behaviors L. Guan Yifan Zhou Denis Liu Yantian Zha H. B. Amor Subbarao Kambhampati LM&Ro 39 16 0 06 Feb 2024
Diffusion World Model: Future Modeling Beyond Step-by-Step Rollout for Offline Reinforcement Learning Zihan Ding Amy Zhang Yuandong Tian Qinqing Zheng OffRL 47 17 0 05 Feb 2024
EgoGen: An Egocentric Synthetic Data Generator Gen Li Kai Zhao Siwei Zhang X. Lyu Mihai Dusmanu Yan Zhang Marc Pollefeys Siyu Tang EgoV VGen 42 14 0 16 Jan 2024
Any-point Trajectory Modeling for Policy Learning Chuan Wen Xingyu Lin John So Kai-xiang Chen Qi Dou Yang Gao Pieter Abbeel PINN VGen 35 81 0 28 Dec 2023
Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis Yafei Hu Quanting Xie Vidhi Jain Jonathan M Francis Jay Patrikar ... Xiaolong Wang Sebastian A. Scherer Z. Kira Fei Xia Yonatan Bisk LM&Ro AI4CE 32 63 0 14 Dec 2023
World Models via Policy-Guided Trajectory Diffusion Marc Rigter Jun Yamada Ingmar Posner 34 19 0 13 Dec 2023
Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planning Zhiting Hu Tianmin Shu LLMAG LM&Ro LRM 102 34 0 08 Dec 2023
Robot Learning in the Era of Foundation Models: A Survey Xuan Xiao Jiahang Liu Zhipeng Wang Yanmin Zhou Yong Qi Qian Cheng Bin He Shuo Jiang AI4CE LM&Ro 26 27 0 24 Nov 2023
Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents Zhuosheng Zhang Yao Yao Aston Zhang Xiangru Tang Xinbei Ma ... Yiming Wang Mark B. Gerstein Rui Wang Gongshen Liu Hai Zhao LLMAG LM&Ro LRM 36 53 0 20 Nov 2023
Learning to Model the World with Language Jessy Lin Yuqing Du Olivia Watkins Danijar Hafner Pieter Abbeel Dan Klein Anca Dragan LM&Ro SyDa 28 51 0 31 Jul 2023
Learning Dynamic Attribute-factored World Models for Efficient Multi-object Reinforcement Learning Fan Feng Sara Magliacane OffRL OCL 36 11 0 18 Jul 2023
SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Models Ziyi Wu Nikita Dvornik Klaus Greff Thomas Kipf Animesh Garg OCL BDL 64 90 0 12 Oct 2022
Fairness and Bias in Robot Learning Laura Londoño Juana Valeria Hurtado Nora Hertz P. Kellmeyer S. Voeneky Abhinav Valada FaML 21 9 0 07 Jul 2022
Ego4D: Around the World in 3,000 Hours of Egocentric Video Kristen Grauman Andrew Westbury Eugene Byrne Zachary Chavis Antonino Furnari ... Mike Zheng Shou Antonio Torralba Lorenzo Torresani Mingfei Yan Jitendra Malik EgoV 232 1,024 0 13 Oct 2021
Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets F. Ebert Yanlai Yang Karl Schmeckpeper Bernadette Bucher G. Georgakis Kostas Daniilidis Chelsea Finn Sergey Levine 169 219 0 27 Sep 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 304 3,708 0 11 Feb 2021