Semantic Exploration from Language Abstractions and Pretrained Representations

8 April 2022

Nicholas A. Roy

Papers citing "Semantic Exploration from Language Abstractions and Pretrained Representations"

50 / 52 papers shown

Title
SySLLM: Generating Synthesized Policy Summaries for Reinforcement Learning Agents Using Large Language Models Sahar Admoni Omer Ben-Porat Ofra Amir LLMAG 49 0 0 13 Mar 2025
SYNERGAI: Perception Alignment for Human-Robot Collaboration Yixin Chen Guoxi Zhang Yaowei Zhang Hongming Xu Peiyuan Zhi Qing Li Siyuan Huang 37 0 0 24 Sep 2024
Efficient Exploration and Discriminative World Model Learning with an Object-Centric Abstraction Anthony GX-Chen Kenneth Marino Rob Fergus OCL 53 1 0 21 Aug 2024
Visual Grounding for Object-Level Generalization in Reinforcement Learning Haobin Jiang Zongqing Lu LM&Ro 34 2 0 04 Aug 2024
Improving Sample Efficiency of Reinforcement Learning with Background Knowledge from Large Language Models Fuxiang Zhang Junyou Li Yi-Chen Li Zongzhang Zhang Yang Yu Deheng Ye OffRL KELM 47 1 0 04 Jul 2024
ET tu, CLIP? Addressing Common Object Errors for Unseen Environments Ye Won Byun Cathy Jiao Shahriar Noroozizadeh Jimin Sun Rosa Vitiello VLM 44 1 0 25 Jun 2024
Policy Learning with a Language Bottleneck Megha Srivastava Cédric Colas Dorsa Sadigh Jacob Andreas 40 3 0 07 May 2024
Scaling Instructable Agents Across Many Simulated Worlds Sima Team Maria Abi Raad Arun Ahuja Catarina Barros F. Besse ... Daan Wierstra Duncan Williams Nathaniel Wong Sarah York Nick Young LM&Ro 115 38 0 13 Mar 2024
The Essential Role of Causality in Foundation World Models for Embodied AI Tarun Gupta Wenbo Gong Chao Ma Nick Pawlowski Agrin Hilmkil ... Jianfeng Gao Stefan Bauer Danica Kragic Bernhard Schölkopf Cheng Zhang 30 15 0 06 Feb 2024
Large Language Models for Robotics: Opportunities, Challenges, and Perspectives Jiaqi Wang Zihao Wu Yiwei Li Hanqi Jiang Peng Shu ... Lin Zhao Bao Ge Xiang Li Tianming Liu Shu Zhang LM&Ro 37 60 0 09 Jan 2024
Vision-Language Models as a Source of Rewards Kate Baumli Satinder Baveja Feryal M. P. Behbahani Harris Chan Gheorghe Comanici ... Yannick Schroecker Stephen Spencer Richie Steigerwald Luyu Wang Lei Zhang VLM LRM 42 26 0 14 Dec 2023
DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physics Zhiao Huang Feng Chen Yewen Pu Chun-Tse Lin Hao Su Chuang Gan 24 4 0 11 Dec 2023
Robot Learning in the Era of Foundation Models: A Survey Xuan Xiao Jiahang Liu Zhipeng Wang Yanmin Zhou Yong Qi Qian Cheng Bin He Shuo Jiang AI4CE LM&Ro 23 26 0 24 Nov 2023
Improve the efficiency of deep reinforcement learning through semantic exploration guided by natural language Zhourui Guo Meng Yao Yang Yu Qiyue Yin OnRL 28 1 0 21 Sep 2023
Developmental Scaffolding with Large Language Models Batuhan Celik Alper Ahmetoglu Emre Ugur Erhan Öztop LM&Ro LLMAG 28 3 0 02 Sep 2023
Learning to Model the World with Language Jessy Lin Yuqing Du Olivia Watkins Danijar Hafner Pieter Abbeel Dan Klein Anca Dragan LM&Ro SyDa 26 51 0 31 Jul 2023
Selective Perception: Optimizing State Descriptions with Reinforcement Learning for Language Model Actors Kolby Nottingham Yasaman Razeghi Kyungmin Kim JB Lanier Pierre Baldi Roy Fox Sameer Singh 28 9 0 21 Jul 2023
VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models Wenlong Huang Chen Wang Ruohan Zhang Yunzhu Li Jiajun Wu Li Fei-Fei LM&Ro 33 480 0 12 Jul 2023
Semantic HELM: A Human-Readable Memory for Reinforcement Learning Fabian Paischer Thomas Adler M. Hofmarcher Sepp Hochreiter 21 9 0 15 Jun 2023
Embodied Executable Policy Learning with Language-based Scene Summarization Jielin Qiu Mengdi Xu William Jongwon Han Seungwhan Moon Ding Zhao LM&Ro 24 7 0 09 Jun 2023
A Study of Global and Episodic Bonuses for Exploration in Contextual MDPs Mikael Henaff Minqi Jiang Roberta Raileanu 38 13 0 05 Jun 2023
LIV: Language-Image Representations and Rewards for Robotic Control Yecheng Jason Ma William Liang Vaidehi Som Vikash Kumar Amy Zhang Osbert Bastani Dinesh Jayaraman LM&Ro 33 121 0 01 Jun 2023
Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learning Wenhao Li Dan Qiao Baoxiang Wang Xiangfeng Wang Bo Jin H. Zha 35 5 0 18 May 2023
Language Instructed Reinforcement Learning for Human-AI Coordination Hengyuan Hu Dorsa Sadigh LM&Ro 35 60 0 13 Apr 2023
Training Language Models with Language Feedback at Scale Jérémy Scheurer Jon Ander Campos Tomasz Korbak Jun Shern Chan Angelica Chen Kyunghyun Cho Ethan Perez ALM 39 101 0 28 Mar 2023
Architext: Language-Driven Generative Architecture Design Theodoros Galanos Antonios Liapis Georgios N. Yannakakis VLM AI4CE 26 6 0 13 Mar 2023
Foundation Models for Decision Making: Problems, Methods, and Opportunities Sherry Yang Ofir Nachum Yilun Du Jason W. Wei Pieter Abbeel Dale Schuurmans LM&Ro OffRL LRM AI4CE 90 155 0 07 Mar 2023
Guiding Pretraining in Reinforcement Learning with Large Language Models Yuqing Du Olivia Watkins Zihan Wang Cédric Colas Trevor Darrell Pieter Abbeel Abhishek Gupta Jacob Andreas LM&Ro 21 174 0 13 Feb 2023
A Song of Ice and Fire: Analyzing Textual Autotelic Agents in ScienceWorld Laetitia Teodorescu Eric Yuan Marc-Alexandre Côté Pierre-Yves Oudeyer 24 1 0 10 Feb 2023
Distilling Internet-Scale Vision-Language Models into Embodied Agents T. Sumers Kenneth Marino Arun Ahuja Rob Fergus Ishita Dasgupta LM&Ro 28 24 0 29 Jan 2023
Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling Kolby Nottingham Prithviraj Ammanabrolu Alane Suhr Yejin Choi Hannaneh Hajishirzi Sameer Singh Roy Fox LLMAG LM&Ro 44 77 0 28 Jan 2023
Transfer Knowledge from Natural Language to Electrocardiography: Can We Detect Cardiovascular Disease Through Language Models? Jielin Qiu William Jongwon Han Jiacheng Zhu Mengdi Xu Michael A. Rosenberg Emerson Liu Douglas Weber Ding Zhao 29 21 0 21 Jan 2023
On the Effect of Pre-training for Transformer in Different Modality on Offline Reinforcement Learning S. Takagi OffRL 18 7 0 17 Nov 2022
What is Wrong with Language Models that Can Not Tell a Story? Ivan P. Yamshchikov Alexey Tikhonov 22 6 0 09 Nov 2022
Foundation Models for Semantic Novelty in Reinforcement Learning Tarun Gupta Peter Karkus Tong Che Danfei Xu Marco Pavone VLM OffRL LRM 39 7 0 09 Nov 2022
Pretraining in Deep Reinforcement Learning: A Survey Zhihui Xie Zichuan Lin Junyou Li Shuai Li Deheng Ye OffRL OnRL AI4CE 26 23 0 08 Nov 2022
Large Language Models can Implement Policy Iteration Ethan A. Brooks Logan Walls Richard L. Lewis Satinder Singh LM&Ro OffRL 134 21 0 07 Oct 2022
Improving Policy Learning via Language Dynamics Distillation Victor Zhong Jesse Mu Luke Zettlemoyer Edward Grefenstette Tim Rocktaschel OffRL 38 15 0 30 Sep 2022
TextWorldExpress: Simulating Text Games at One Million Steps Per Second Peter Alexander Jansen Marc-Alexandre Côté VLM LRM 29 6 0 01 Aug 2022
Automatic Exploration of Textual Environments with Language-Conditioned Autotelic Agents Laetitia Teodorescu Xingdi Yuan Marc-Alexandre Côté Pierre-Yves Oudeyer LLMAG 27 0 0 08 Jul 2022
EAGER: Asking and Answering Questions for Automatic Reward Shaping in Language-guided RL Thomas Carta Pierre-Yves Oudeyer Olivier Sigaud Sylvain Lamprier OffRL 23 24 0 20 Jun 2022
How to talk so AI will learn: Instructions, descriptions, and autonomy T. Sumers Robert D. Hawkins Mark K. Ho Thomas L. Griffiths Dylan Hadfield-Menell LM&Ro 32 20 0 16 Jun 2022
Language and Culture Internalisation for Human-Like Autotelic AI Cédric Colas Tristan Karch Clément Moulin-Frier Pierre-Yves Oudeyer LM&Ro 32 25 0 02 Jun 2022
History Compression via Language Models in Reinforcement Learning Fabian Paischer Thomas Adler Vihang Patil Angela Bitto-Nemling Markus Holzleitner Sebastian Lehner Hamid Eghbalzadeh Sepp Hochreiter OffRL AI4TS 18 42 0 24 May 2022
Training Language Models with Language Feedback Jérémy Scheurer Jon Ander Campos Jun Shern Chan Angelica Chen Kyunghyun Cho Ethan Perez ALM 38 47 0 29 Apr 2022
Improving Intrinsic Exploration with Language Abstractions Jesse Mu Victor Zhong Roberta Raileanu Minqi Jiang Noah D. Goodman Tim Rocktaschel Edward Grefenstette 103 63 0 17 Feb 2022
VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learning Che Wang Xufang Luo Keith Ross Dongsheng Li OffRL 26 49 0 17 Feb 2022
Can Wikipedia Help Offline Reinforcement Learning? Machel Reid Yutaro Yamada S. Gu 3DV RALM OffRL 137 95 0 28 Jan 2022
Tell me why! Explanations support learning relational and causal structure Andrew Kyle Lampinen Nicholas A. Roy Ishita Dasgupta Stephanie C. Y. Chan Allison C. Tam ... Chen Yan Adam Santoro Neil C. Rabinowitz Jane X. Wang Felix Hill 30 45 0 07 Dec 2021
Curious Representation Learning for Embodied Intelligence Yilun Du Chuang Gan Phillip Isola SSL LM&Ro 112 40 0 03 May 2021