CAREL: Instruction-guided reinforcement learning with cross-modal
auxiliary objectives

CAREL: Instruction-guided reinforcement learning with cross-modal auxiliary objectives

29 November 2024

Armin Saghafian

Amirmohammad Izadi

Negin Hashemi Dijujin

ArXiv (abs)PDF HTML

Papers citing "CAREL: Instruction-guided reinforcement learning with cross-modal auxiliary objectives"

18 / 18 papers shown

Title
Contrastive Learning as Goal-Conditioned Reinforcement Learning Benjamin Eysenbach Tianjun Zhang Ruslan Salakhutdinov Sergey Levine SSL OffRL 104 163 0 15 Jun 2022
CoCa: Contrastive Captioners are Image-Text Foundation Models Jiahui Yu Zirui Wang Vijay Vasudevan Legg Yeung Mojtaba Seyedhosseini Yonghui Wu VLM CLIP OffRL 184 1,309 0 04 May 2022
X-Pool: Cross-Modal Language-Video Attention for Text-Video Retrieval S. Gorti Noël Vouitsis Junwei Ma Keyvan Golestan M. Volkovs Animesh Garg Guangwei Yu 91 160 0 28 Mar 2022
Goal-Conditioned Reinforcement Learning: Problems and Solutions Minghuan Liu Menghui Zhu Weinan Zhang 97 143 0 20 Jan 2022
FILIP: Fine-grained Interactive Language-Image Pre-Training Lewei Yao Runhu Huang Lu Hou Guansong Lu Minzhe Niu Hang Xu Xiaodan Liang Zhenguo Li Xin Jiang Chunjing Xu VLM CLIP 111 643 0 09 Nov 2021
FILM: Following Instructions in Language with Modular Methods So Yeon Min Devendra Singh Chaplot Pradeep Ravikumar Yonatan Bisk Ruslan Salakhutdinov LM&Ro 266 163 0 12 Oct 2021
ELLA: Exploration through Learned Language Abstraction Suvir Mirchandani Siddharth Karamcheti Dorsa Sadigh LLMAG 55 58 0 10 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 1.0K 29,926 0 26 Feb 2021
Decoupling Representation Learning from Reinforcement Learning Adam Stooke Kimin Lee Pieter Abbeel Michael Laskin SSL DRL 367 345 0 14 Sep 2020
Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigation Zhiwei Deng Karthik Narasimhan Olga Russakovsky 74 88 0 11 Jul 2020
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks Mohit Shridhar Jesse Thomason Daniel Gordon Yonatan Bisk Winson Han Roozbeh Mottaghi Luke Zettlemoyer Dieter Fox LM&Ro 124 781 0 03 Dec 2019
Language as an Abstraction for Hierarchical Deep Reinforcement Learning Yiding Jiang S. Gu Kevin Patrick Murphy Chelsea Finn OffRL 67 225 0 18 Jun 2019
A Survey of Reinforcement Learning Informed by Natural Language Jelena Luketina Nantas Nardelli Gregory Farquhar Jakob N. Foerster Jacob Andreas Edward Grefenstette Shimon Whiteson Tim Rocktaschel LM&Ro KELM OffRL LRM 101 282 0 10 Jun 2019
Using Natural Language for Reward Shaping in Reinforcement Learning Prasoon Goyal S. Niekum Raymond J. Mooney LM&Ro 102 183 0 05 Mar 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 263 3,747 0 09 Jan 2019
BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning Maxime Chevalier-Boisvert Dzmitry Bahdanau Salem Lahlou Lucas Willems Chitwan Saharia Thien Huu Nguyen Yoshua Bengio ELM 106 241 0 18 Oct 2018
Proximal Policy Optimization Algorithms John Schulman Filip Wolski Prafulla Dhariwal Alec Radford Oleg Klimov OffRL 583 19,315 0 20 Jul 2017
Modular Multitask Reinforcement Learning with Policy Sketches Jacob Andreas Dan Klein Sergey Levine OffRL 179 463 0 06 Nov 2016