MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge

17 June 2022

Linxi Fan

De-An Huang

Papers citing "MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge"

37 / 287 papers shown

Title
Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling Kolby Nottingham Prithviraj Ammanabrolu Alane Suhr Yejin Choi Hannaneh Hajishirzi Sameer Singh Roy Fox LLMAG LM&Ro 50 77 0 28 Jan 2023
Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction Shaofei Cai Zihao Wang Xiaojian Ma Guy Van den Broeck Yitao Liang 50 40 0 21 Jan 2023
Human-Timescale Adaptation in an Open-Ended Task Space Adaptive Agent Team Jakob Bauer Kate Baumli Satinder Baveja Feryal M. P. Behbahani ... Jakub Sygnowski K. Tuyls Sarah York Alexander Zacherl Lei Zhang LM&Ro OffRL AI4CE LRM 38 109 0 18 Jan 2023
Mastering Diverse Domains through World Models Danijar Hafner J. Pašukonis Jimmy Ba Timothy Lillicrap 38 553 0 10 Jan 2023
A Survey on Transformers in Reinforcement Learning Wenzhe Li Hao Luo Zichuan Lin Chongjie Zhang Zongqing Lu Deheng Ye OffRL MU AI4CE 37 56 0 08 Jan 2023
On Transforming Reinforcement Learning by Transformer: The Development Trajectory Shengchao Hu Li Shen Ya Zhang Yixin Chen Dacheng Tao OffRL 27 25 0 29 Dec 2022
On Realization of Intelligent Decision-Making in the Real World: A Foundation Decision Model Perspective Ying Wen Bo Liu M. Zhou Shufang Hou Zhe Cao Chenyang Le Jingxiao Chen Zheng Tian Weinan Zhang Jun Wang AI4CE 26 10 0 24 Dec 2022
Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing Shengchao Liu Weili Nie Chengpeng Wang Jiarui Lu Zhuoran Qiao Ling Liu Jian Tang Chaowei Xiao Anima Anandkumar 48 153 0 21 Dec 2022
Powderworld: A Platform for Understanding Generalization via Rich Task Distributions Kevin Frans Phillip Isola OffRL 47 9 0 23 Nov 2022
Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Models Ted Xiao Harris Chan P. Sermanet Ayzaan Wahid Anthony Brohan Karol Hausman Sergey Levine Jonathan Tompson VLM LM&Ro 38 65 0 21 Nov 2022
General Intelligence Requires Rethinking Exploration Minqi Jiang Tim Rocktaschel Edward Grefenstette LRM 29 18 0 15 Nov 2022
Pretraining in Deep Reinforcement Learning: A Survey Zhihui Xie Zichuan Lin Junyou Li Shuai Li Deheng Ye OffRL OnRL AI4CE 26 23 0 08 Nov 2022
Dichotomy of Control: Separating What You Can Control from What You Cannot Mengjiao Yang Dale Schuurmans Pieter Abbeel Ofir Nachum OffRL 30 42 0 24 Oct 2022
Augmentative Topology Agents For Open-Ended Learning Muhammad Umair Nasir Michael Beukman Steven D. James C. Cleghorn 37 3 0 20 Oct 2022
WILD-SCAV: Benchmarking FPS Gaming AI on Unity3D-based Environments Xi Chen Tianyuan Shi Qing Zhao Yuchen Sun Yunfei Gao Xiangjun Wang 33 2 0 14 Oct 2022
VIMA: General Robot Manipulation with Multimodal Prompts Yunfan Jiang Agrim Gupta Zichen Zhang Guanzhi Wang Yongqiang Dou Yanjun Chen Li Fei-Fei Anima Anandkumar Yuke Zhu Linxi Fan LM&Ro 38 338 0 06 Oct 2022
Neural-Symbolic Recursive Machine for Systematic Generalization Qing Li Yixin Zhu Yitao Liang Ying Nian Wu Song-Chun Zhu Siyuan Huang NAI 40 9 0 04 Oct 2022
Pretraining the Vision Transformer using self-supervised methods for vision based Deep Reinforcement Learning Manuel Goulão Arlindo L. Oliveira ViT 45 6 0 22 Sep 2022
Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos Bowen Baker Ilge Akkaya Peter Zhokhov Joost Huizinga Jie Tang Adrien Ecoffet Brandon Houghton Raul Sampedro Jeff Clune OffRL 50 289 0 23 Jun 2022
NovelCraft: A Dataset for Novelty Detection and Discovery in Open Worlds Patrick Feeney Sarah Schneider Panagiotis Lymperopoulos Liping Liu matthias. scheutz Michael C. Hughes Dept. of Computer Science 41 5 0 23 Jun 2022
Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progress Rishabh Agarwal Max Schwarzer Pablo Samuel Castro Rameswar Panda Marc G. Bellemare OffRL OnRL 37 63 0 03 Jun 2022
Language and Culture Internalisation for Human-Like Autotelic AI Cédric Colas Tristan Karch Clément Moulin-Frier Pierre-Yves Oudeyer LM&Ro 41 25 0 02 Jun 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 328 4,139 0 24 May 2022
The Primacy Bias in Deep Reinforcement Learning Evgenii Nikishin Max Schwarzer P. DÓro Pierre-Luc Bacon Rameswar Panda OnRL 96 181 0 16 May 2022
Geodesic Multi-Modal Mixup for Robust Fine-Tuning Changdae Oh Junhyuk So Hoyoon Byun Yongtaek Lim Minchul Shin Jong-June Jeon Kyungwoo Song 33 26 0 08 Mar 2022
MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learned Anssi Kanervisto Stephanie Milani Karolis Ramanauskas Nicholay Topin Zichuan Lin ... Franccois Fleuret Alexander Nikulin Yury Belousov Oleg Svidchenko A. Shpilman OffRL 60 32 0 17 Feb 2022
Can Wikipedia Help Offline Reinforcement Learning? Machel Reid Yutaro Yamada S. Gu 3DV RALM OffRL 140 95 0 28 Jan 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 322 7,481 0 11 Nov 2021
Ego4D: Around the World in 3,000 Hours of Egocentric Video Kristen Grauman Andrew Westbury Eugene Byrne Zachary Chavis Antonino Furnari ... Mike Zheng Shou Antonio Torralba Lorenzo Torresani Mingfei Yan Jitendra Malik EgoV 275 1,026 0 13 Oct 2021
NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environment Julia Kiseleva Ziming Li Mohammad Aliannejadi Shrestha Mohanty Maartje ter Hoeve ... Arthur Szlam Yuxuan Sun Katja Hofmann Michel Galley Ahmed Hassan Awadallah LLMAG 70 15 0 13 Oct 2021
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Dmytro Okhonko Armen Aghajanyan Florian Metze Luke Zettlemoyer Florian Metze Luke Zettlemoyer Christoph Feichtenhofer CLIP VLM 259 561 0 28 Sep 2021
BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments S. Srivastava Chengshu Li Michael Lingelbach Roberto Martín-Martín Fei Xia ... Chenxi Liu Silvio Savarese H. Gweon Jiajun Wu Li Fei-Fei LM&Ro 151 158 0 06 Aug 2021
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text Hassan Akbari Liangzhe Yuan Rui Qian Wei-Hong Chuang Shih-Fu Chang Huayu Chen Boqing Gong ViT 251 577 0 22 Apr 2021
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval Huaishao Luo Lei Ji Ming Zhong Yang Chen Wen Lei Nan Duan Tianrui Li CLIP VLM 329 782 0 18 Apr 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 282 2,000 0 31 Dec 2020
LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding Yang Xu Yiheng Xu Tengchao Lv Lei Cui Furu Wei ... D. Florêncio Cha Zhang Wanxiang Che Min Zhang Lidong Zhou ViT MLLM 153 501 0 29 Dec 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,833 0 17 Sep 2019