Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning

9 March 2023

Papers citing "Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning"

40 / 90 papers shown

Title
Diffusion World Model: Future Modeling Beyond Step-by-Step Rollout for Offline Reinforcement Learning Zihan Ding Amy Zhang Yuandong Tian Qinqing Zheng OffRL 55 17 0 05 Feb 2024
Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation Problem Maciej Wolczyk Bartłomiej Cupiał M. Ostaszewski Michal Bortkiewicz Michal Zajkac Razvan Pascanu Lukasz Kuciñski Piotr Milo's CLL 53 13 0 05 Feb 2024
Small Dataset, Big Gains: Enhancing Reinforcement Learning by Offline Pre-Training with Model Based Augmentation Girolamo Macaluso Alessandro Sestini Andrew D. Bagdanov OffRL OnRL 35 3 0 15 Dec 2023
Efficient Sparse-Reward Goal-Conditioned Reinforcement Learning with a High Replay Ratio and Regularization Takuya Hiraoka OffRL 34 1 0 10 Dec 2023
Model-Based Epistemic Variance of Values for Risk-Aware Policy Optimization Carlos E. Luis A. Bottero Julia Vinogradska Felix Berkenkamp Jan Peters OffRL 43 3 0 07 Dec 2023
Pearl: A Production-ready Reinforcement Learning Agent Zheqing Zhu Rodrigo de Salvo Braz Jalaj Bhandari Daniel Jiang Yi Wan ... D. Korenkevych Ürün Dogan Frank Cheng Zheng Wu Wanqiao Xu VLM OffRL OnRL 41 6 0 06 Dec 2023
LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models Marwa Abdulhai Isadora White Charles Burton Snell Charles Sun Joey Hong Yuexiang Zhai Kelvin Xu Sergey Levine LLMAG OffRL LRM 39 31 0 30 Nov 2023
RLIF: Interactive Imitation Learning as Reinforcement Learning Jianlan Luo Perry Dong Yuexiang Zhai Yi Ma Sergey Levine OffRL 35 14 0 21 Nov 2023
Offline Data Enhanced On-Policy Policy Gradient with Provable Guarantees Yifei Zhou Ayush Sekhari Yuda Song Wen Sun OffRL OnRL 32 8 0 14 Nov 2023
Accelerating Exploration with Unlabeled Prior Data Qiyang Li Jason Zhang Dibya Ghosh Amy Zhang Sergey Levine OffRL OnRL 54 9 0 09 Nov 2023
Uni-O4: Unifying Online and Offline Deep Reinforcement Learning with Multi-Step On-Policy Optimization Kun Lei Zhengmao He Chenhao Lu Kaizhe Hu Yang Gao Huazhe Xu OffRL OnRL 67 13 0 06 Nov 2023
TD-MPC2: Scalable, Robust World Models for Continuous Control Nicklas Hansen Hao Su Xiaolong Wang MU 37 128 0 25 Oct 2023
Finetuning Offline World Models in the Real World Yunhai Feng Nicklas Hansen Ziyan Xiong Chandramouli Rajagopalan Xiaolong Wang OffRL OnRL 27 20 0 24 Oct 2023
Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning Jingyun Yang Max Sobol Mark Brandon Vu Archit Sharma Jeannette Bohg Chelsea Finn OffRL OnRL 40 21 0 23 Oct 2023
Offline Retraining for Online RL: Decoupled Policy Learning to Mitigate Exploration Bias Max Sobol Mark Archit Sharma Fahim Tajwar Rafael Rafailov Sergey Levine Chelsea Finn OffRL OnRL 36 1 0 12 Oct 2023
Planning to Go Out-of-Distribution in Offline-to-Online Reinforcement Learning Trevor A. McInroe Adam Jelley Stefano V. Albrecht Amos Storkey OffRL OnRL 30 6 0 09 Oct 2023
Improving Offline-to-Online Reinforcement Learning with Q Conditioned State Entropy Exploration Ziqi Zhang Xiao Xiong Zifeng Zhuang Jinxin Liu Donglin Wang OffRL OnRL 50 0 0 07 Oct 2023
Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learning Daoce Wang Chi Jin OffRL DiffM 27 25 0 29 Sep 2023
Towards Robust Offline-to-Online Reinforcement Learning via Uncertainty and Smoothness Xiaoyu Wen Xudong Yu Rui Yang Chenjia Bai Zhen Wang OffRL OnRL 36 10 0 29 Sep 2023
Zero-Shot Reinforcement Learning from Low Quality Data Scott Jeen Tom Bewley Jonathan M. Cullen OffRL OnRL 43 1 0 26 Sep 2023
H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps Haoyi Niu Tianying Ji Bingqi Liu Haocheng Zhao Xiangyu Zhu Jianying Zheng Pengfei Huang Guyue Zhou Jianming Hu Xianyuan Zhan OffRL OnRL AI4CE 31 7 0 22 Sep 2023
Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration Jinning Li Xinyi Liu Banghua Zhu Jiantao Jiao Masayoshi Tomizuka Chen Tang Wei Zhan OffRL OnRL 74 10 0 18 Sep 2023
Autonomy 2.0: The Quest for Economies of Scale Shuang Wu Bo Yu Shaoshan Liu Yuhao Zhu 21 2 0 08 Jul 2023
A Simple Unified Uncertainty-Guided Framework for Offline-to-Online Reinforcement Learning Siyuan Guo Yanchao Sun Jifeng Hu Sili Huang Hechang Chen Haiyin Piao Lichao Sun Yi-Ju Chang OffRL OnRL 41 7 0 13 Jun 2023
Improving Offline-to-Online Reinforcement Learning with Q-Ensembles Kai-Wen Zhao Yi Ma Jianye Hao Jinyi Liu Yan Zheng Zhaopeng Meng OffRL OnRL 22 12 0 12 Jun 2023
PROTO: Iterative Policy Regularized Offline-to-Online Reinforcement Learning Jianxiong Li Xiao Hu Haoran Xu Jingjing Liu Xianyuan Zhan Ya Zhang OffRL OnRL 42 19 0 25 May 2023
Making Offline RL Online: Collaborative World Models for Offline Visual Reinforcement Learning Q. Wang Jun Yang Yunbo Wang Xin Jin Wenjun Zeng Xiaokang Yang OffRL OnRL 41 3 0 24 May 2023
Reward-agnostic Fine-tuning: Provable Statistical Benefits of Hybrid Reinforcement Learning Gen Li Wenhao Zhan Jason D. Lee Yuejie Chi Yuxin Chen OffRL OnRL 73 13 0 17 May 2023
Revisiting the Minimalist Approach to Offline Reinforcement Learning Denis Tarasov Vladislav Kurenkov Alexander Nikulin Sergey Kolesnikov OffRL 38 37 0 16 May 2023
IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies Philippe Hansen-Estruch Ilya Kostrikov Michael Janner J. Kuba Sergey Levine OffRL 34 130 0 20 Apr 2023
Learning to Control Autonomous Fleets from Observation via Offline Reinforcement Learning Carolin Schmidt Daniele Gammelli Francisco Câmara Pereira Filipe Rodrigues OffRL 21 4 0 28 Feb 2023
Dual RL: Unification and New Methods for Reinforcement and Imitation Learning Harshit S. Sikchi Qinqing Zheng Amy Zhang S. Niekum OffRL 38 19 0 16 Feb 2023
Off-the-Grid MARL: Datasets with Baselines for Offline Multi-Agent Reinforcement Learning Claude Formanek Asad Jeewa Jonathan P. Shock Arnu Pretorius OffRL 43 2 0 01 Feb 2023
Leveraging Offline Data in Online Reinforcement Learning Andrew Wagenmaker Aldo Pacchiano OffRL OnRL 35 38 0 09 Nov 2022
CORL: Research-oriented Deep Offline Reinforcement Learning Library Denis Tarasov Alexander Nikulin Dmitry Akimov Vladislav Kurenkov Sergey Kolesnikov OffRL 66 81 0 13 Oct 2022
Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trials Aviral Kumar Anika Singh F. Ebert Mitsuhiko Nakamoto Yanlai Yang Chelsea Finn Sergey Levine OffRL OnRL 131 66 0 11 Oct 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 322 7,503 0 11 Nov 2021
Offline Reinforcement Learning with Implicit Q-Learning Ilya Kostrikov Ashvin Nair Sergey Levine OffRL 214 852 0 12 Oct 2021
EMaQ: Expected-Max Q-Learning Operator for Simple Yet Effective Offline and Online RL Seyed Kamyar Seyed Ghasemipour Dale Schuurmans S. Gu OffRL 217 119 0 21 Jul 2020
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems Sergey Levine Aviral Kumar George Tucker Justin Fu OffRL GP 343 1,968 0 04 May 2020