MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge

17 June 2022

Linxi Fan

De-An Huang

Papers citing "MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge"

50 / 289 papers shown

Title
Brain in a Vat: On Missing Pieces Towards Artificial General Intelligence in Large Language Models Yuxi Ma Chi Zhang Song-Chun Zhu ELM ALM 40 8 0 07 Jul 2023
Goal Representations for Instruction Following: A Semi-Supervised Language Interface to Control Vivek Myers Andre Wang He Kuan Fang Homer Walke Philippe Hansen-Estruch Ching-An Cheng Mihai Jalobeanu Andrey Kolobov Anca Dragan Sergey Levine LM&Ro 35 29 0 30 Jun 2023
Large Sequence Models for Sequential Decision-Making: A Survey Muning Wen Runji Lin Hanjing Wang Yaodong Yang Ying Wen Luo Mai Jun Wang Haifeng Zhang Weinan Zhang LM&Ro LRM 39 35 0 24 Jun 2023
Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks Maxime Chevalier-Boisvert Bolun Dai Mark Towers Rodrigo de Lazcano Lucas Willems Salem Lahlou Suman Pal Pablo Samuel Castro Jordan Terry VGen 13 185 0 24 Jun 2023
SPRINT: Scalable Policy Pre-Training via Language Instruction Relabeling Jesse Zhang Karl Pertsch Jiahui Zhang Joseph J. Lim LM&Ro 45 17 0 20 Jun 2023
Semantic HELM: A Human-Readable Memory for Reinforcement Learning Fabian Paischer Thomas Adler M. Hofmarcher Sepp Hochreiter 31 10 0 15 Jun 2023
LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models Peng Xu Wenqi Shao Kaipeng Zhang Peng Gao Shuo Liu Meng Lei Fanqing Meng Siyuan Huang Yu Qiao Ping Luo ELM MLLM 41 159 0 15 Jun 2023
ChessGPT: Bridging Policy Learning and Language Modeling Xidong Feng Yicheng Luo Ziyan Wang Hongrui Tang Mengyue Yang Kun Shao D. Mguni Yali Du Jun Wang 29 39 0 15 Jun 2023
Behavioral Cloning via Search in Embedded Demonstration Dataset Federico Malato Florian Leopold Ville Hautamaki Andrew Melnik OffRL 29 3 0 15 Jun 2023
Toward Grounded Commonsense Reasoning Minae Kwon Hengyuan Hu Vivek Myers Siddharth Karamcheti Anca Dragan Dorsa Sadigh LM&Ro ReLM LRM 50 9 0 14 Jun 2023
Language to Rewards for Robotic Skill Synthesis Wenhao Yu Nimrod Gileadi Chuyuan Fu Sean Kirmani Kuang-Huei Lee ... N. Heess Dorsa Sadigh Jie Tan Yuval Tassa F. Xia LM&Ro 41 269 0 14 Jun 2023
Value function estimation using conditional diffusion models for control Bogdan Mazoure Walter A. Talbott Miguel Angel Bautista R. Devon Hjelm Alexander Toshev J. Susskind DiffM 40 4 0 09 Jun 2023
Embodied Executable Policy Learning with Language-based Scene Summarization Jielin Qiu Mengdi Xu William Jongwon Han Seungwhan Moon Ding Zhao LM&Ro 28 7 0 09 Jun 2023
OMNI: Open-endedness via Models of human Notions of Interestingness Jenny Zhang Joel Lehman Kenneth O. Stanley Jeff Clune LRM 23 31 0 02 Jun 2023
Heterogeneous Knowledge for Augmented Modular Reinforcement Learning Lorenz Wolf Mirco Musolesi OffRL 19 0 0 01 Jun 2023
Active Vision Reinforcement Learning under Limited Visual Observability Jinghuan Shang Michael S. Ryoo 40 0 0 01 Jun 2023
LIV: Language-Image Representations and Rewards for Robotic Control Yecheng Jason Ma William Liang Vaidehi Som Vikash Kumar Amy Zhang Osbert Bastani Dinesh Jayaraman LM&Ro 42 123 0 01 Jun 2023
STEVE-1: A Generative Model for Text-to-Behavior in Minecraft Shalev Lifshitz Keiran Paster Harris Chan Jimmy Ba Sheila A. McIlraith LM&Ro 37 68 0 01 Jun 2023
Thought Cloning: Learning to Think while Acting by Imitating Human Thinking Shengran Hu Jeff Clune LM&Ro OffRL LRM AI4CE 45 27 0 01 Jun 2023
Bigger, Better, Faster: Human-level Atari with human-level efficiency Max Schwarzer J. Obando-Ceron Rameswar Panda Marc G. Bellemare Rishabh Agarwal Pablo Samuel Castro OffRL 54 85 0 30 May 2023
ChatGPT-powered Conversational Drug Editing Using Retrieval and Domain Feedback Shengchao Liu Jiong Wang Yijin Yang Chengpeng Wang Ling Liu Hongyu Guo Chaowei Xiao LM&MA KELM AI4MH 36 39 0 29 May 2023
Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory Xizhou Zhu Yuntao Chen Hao Tian Chenxin Tao Weijie Su ... Lewei Lu Xiaogang Wang Yu Qiao Zhaoxiang Zhang Jifeng Dai LLMAG LM&Ro 44 215 0 25 May 2023
Voyager: An Open-Ended Embodied Agent with Large Language Models Guanzhi Wang Yuqi Xie Yunfan Jiang Ajay Mandlekar Chaowei Xiao Yuke Zhu Linxi Fan Anima Anandkumar LM&Ro SyDa 75 761 0 25 May 2023
Augmenting Autotelic Agents with Large Language Models Cédric Colas Laetitia Teodorescu Pierre-Yves Oudeyer Xingdi Yuan Marc-Alexandre Côté LLMAG LM&Ro 33 22 0 21 May 2023
Multimodal Web Navigation with Instruction-Finetuned Foundation Models Hiroki Furuta Kuang-Huei Lee Ofir Nachum Yutaka Matsuo Aleksandra Faust S. Gu Izzeddin Gur LM&Ro 38 93 0 19 May 2023
Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learning Wenhao Li Dan Qiao Baoxiang Wang Xiangfeng Wang Bo Jin H. Zha 46 5 0 18 May 2023
Learning Video-Conditioned Policies for Unseen Manipulation Tasks Elliot Chane-Sane Cordelia Schmid Ivan Laptev 27 18 0 10 May 2023
Games for Artificial Intelligence Research: A Review and Perspectives Chengpeng Hu Yunlong Zhao Ziqi Wang Haocheng Du Jialin Liu AI4CE 35 13 0 26 Apr 2023
Contrastive Language, Action, and State Pre-training for Robot Learning Krishan Rana Andrew Melnik Niko Sünderhauf 25 12 0 21 Apr 2023
Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedback Nikhil Mehta Milagro Teruel Patricio Figueroa Sanz Xinwei Deng Ahmed Hassan Awadallah Julia Kiseleva LLMAG 36 23 0 21 Apr 2023
LLM as A Robotic Brain: Unifying Egocentric Memory and Control Jinjie Mai Jun Chen Bing Li Guocheng Qian Mohamed Elhoseiny Guohao Li LM&Ro 22 33 0 19 Apr 2023
Think Before You Act: Unified Policy for Interleaving Language Reasoning with Actions Lina Mezghani Piotr Bojanowski Alahari Karteek Sainbayar Sukhbaatar LM&Ro OffRL LRM 23 8 0 18 Apr 2023
Grounding Classical Task Planners via Vision-Language Models Xiaohan Zhang Yan Ding S. Amiri Hao Yang Andy Kaminski Chad Esselink Shiqi Zhang 26 17 0 17 Apr 2023
Language Instructed Reinforcement Learning for Human-AI Coordination Hengyuan Hu Dorsa Sadigh LM&Ro 35 60 0 13 Apr 2023
Object-centric Inference for Language Conditioned Placement: A Foundation Model based Approach Zhi-Wei Xu Kechun Xu Yue Wang R. Xiong OCL 18 4 0 06 Apr 2023
Chain-of-Thought Predictive Control Zhiwei Jia Vineet Thumuluri Fangchen Liu Ling-Hao Chen Zhiao Huang H. Su LM&Ro 48 20 0 03 Apr 2023
Accelerating exploration and representation learning with offline pre-training Bogdan Mazoure Jake Bruce Doina Precup Rob Fergus Ankit Anand OffRL 41 5 0 31 Mar 2023
When Learning Is Out of Reach, Reset: Generalization in Autonomous Visuomotor Reinforcement Learning Zichen Zhang Luca Weihs OffRL 29 5 0 30 Mar 2023
Language Models can Solve Computer Tasks Geunwoo Kim Pierre Baldi Stephen Marcus McAleer LLMAG LM&Ro 45 344 0 30 Mar 2023
Skill Reinforcement Learning and Planning for Open-World Long-Horizon Tasks Haoqi Yuan Chi Zhang Hongchen Wang Feiyang Xie Penglin Cai Hao Dong Zongqing Lu LM&Ro LLMAG 28 19 0 29 Mar 2023
Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competition Stephanie Milani Anssi Kanervisto Karolis Ramanauskas Sander Schulhoff Brandon Houghton ... Vinicius G. Goecks Nicholas R. Waytowich David Watkins J. Miller Rohin Shah 37 16 0 23 Mar 2023
Building artificial neural circuits for domain-general cognition: a primer on brain-inspired systems-level architecture Jascha Achterberg Danyal Akarca Moataz Assem Moritz P. Heimbach D. Astle John Duncan AI4CE 42 4 0 21 Mar 2023
CLIP4MC: An RL-Friendly Vision-Language Model for Minecraft Ziluo Ding Hao Luo Ke Li Junpeng Yue Tiejun Huang Zongqing Lu VLM 28 9 0 19 Mar 2023
Vision-Language Models as Success Detectors Yuqing Du Ksenia Konyushkova Misha Denil A. Raju Jessica Landon Felix Hill Nando de Freitas Serkan Cabi MLLM LRM 91 79 0 13 Mar 2023
Foundation Models for Decision Making: Problems, Methods, and Opportunities Sherry Yang Ofir Nachum Yilun Du Jason W. Wei Pieter Abbeel Dale Schuurmans LM&Ro OffRL LRM AI4CE 98 156 0 07 Mar 2023
Cross-domain Random Pre-training with Prototypes for Reinforcement Learning Xin Liu Yaran Chen Haoran Li Boyu Li Dong Zhao SSL 71 10 0 11 Feb 2023
A Song of Ice and Fire: Analyzing Textual Autotelic Agents in ScienceWorld Laetitia Teodorescu Eric Yuan Marc-Alexandre Côté Pierre-Yves Oudeyer 32 1 0 10 Feb 2023
Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manuals Yue Wu Yewen Fan Paul Pu Liang A. Azaria Yuan-Fang Li Tom Michael Mitchell OffRL 31 47 0 09 Feb 2023
Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning Thomas Carta Clément Romac Thomas Wolf Sylvain Lamprier Olivier Sigaud Pierre-Yves Oudeyer LM&Ro LLMAG 25 183 0 06 Feb 2023
Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents Zihao Wang Shaofei Cai Guanzhou Chen Guy Van den Broeck Xiaojian Ma Yitao Liang LM&Ro LLMAG 60 318 0 03 Feb 2023