Towards Instance-Optimal Offline Reinforcement Learning with Pessimism

17 October 2021

Ming Yin

Papers citing "Towards Instance-Optimal Offline Reinforcement Learning with Pessimism"

43 / 43 papers shown

Title
On The Statistical Complexity of Offline Decision-Making Thanh Nguyen-Tang R. Arora OffRL 205 1 0 10 Jan 2025
Selective Uncertainty Propagation in Offline RL Sanath Kumar Krishnamurthy Shrey Modi Tanmay Gangwani S. Katariya Branislav Kveton A. Rangi OffRL 191 0 0 01 Feb 2023
Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learning Andrea Zanette Martin J. Wainwright Emma Brunskill OffRL 95 119 0 19 Aug 2021
Bellman-consistent Pessimism for Offline Reinforcement Learning Tengyang Xie Ching-An Cheng Nan Jiang Paul Mineiro Alekh Agarwal OffRL LRM 158 278 0 13 Jun 2021
Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learning Tengyang Xie Nan Jiang Huan Wang Caiming Xiong Yu Bai OffRL OnRL 90 164 0 09 Jun 2021
Mitigating Covariate Shift in Imitation Learning via Offline Data Without Great Coverage Jonathan D. Chang Masatoshi Uehara Dhruv Sreenivas Rahul Kidambi Wen Sun OffRL 97 32 0 06 Jun 2021
Offline Reinforcement Learning as One Big Sequence Modeling Problem Michael Janner Qiyang Li Sergey Levine OffRL 158 685 0 03 Jun 2021
Optimal Uniform OPE and Model-based Offline Reinforcement Learning in Time-Homogeneous, Reward-Free and Task-Agnostic Settings Ming Yin Yu Wang OffRL 95 19 0 13 May 2021
On the Optimality of Batch Policy Optimization Algorithms Chenjun Xiao Yifan Wu Tor Lattimore Bo Dai Jincheng Mei Lihong Li Csaba Szepesvári Dale Schuurmans OffRL 70 33 0 06 Apr 2021
Benchmarks for Deep Off-Policy Evaluation Justin Fu Mohammad Norouzi Ofir Nachum George Tucker Ziyun Wang ... Yutian Chen Aviral Kumar Cosmin Paduraru Sergey Levine T. Paine ELM OffRL 78 104 0 30 Mar 2021
Nearly Horizon-Free Offline Reinforcement Learning Zhaolin Ren Jialian Li Bo Dai S. Du Sujay Sanghavi OffRL 69 49 0 25 Mar 2021
Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism Paria Rashidinejad Banghua Zhu Cong Ma Jiantao Jiao Stuart J. Russell OffRL 227 290 0 22 Mar 2021
Near-Optimal Offline Reinforcement Learning via Double Variance Reduction Ming Yin Yu Bai Yu Wang OffRL 85 68 0 02 Feb 2021
Is Pessimism Provably Efficient for Offline RL? Ying Jin Zhuoran Yang Zhaoran Wang OffRL 176 359 0 30 Dec 2020
Exponential Lower Bounds for Batch Reinforcement Learning: Batch RL can be Exponentially Harder than Online RL Andrea Zanette OffRL 153 71 0 14 Dec 2020
What are the Statistical Limits of Offline RL with Linear Function Approximation? Ruosong Wang Dean Phillips Foster Sham Kakade OffRL 162 163 0 22 Oct 2020
Is Reinforcement Learning More Difficult Than Bandits? A Near-optimal Algorithm Escaping the Curse of Horizon Zihan Zhang Xiangyang Ji S. Du OffRL 110 107 0 28 Sep 2020
The Importance of Pessimism in Fixed-Dataset Policy Optimization Jacob Buckman Carles Gelada Marc G. Bellemare OffRL 90 139 0 15 Sep 2020
Batch Value-function Approximation with Only Realizability Tengyang Xie Nan Jiang OffRL 384 121 0 11 Aug 2020
Provably Good Batch Reinforcement Learning Without Great Exploration Yao Liu Adith Swaminathan Alekh Agarwal Emma Brunskill OffRL 159 105 0 16 Jul 2020
Near-Optimal Provable Uniform Convergence in Offline Policy Evaluation for Reinforcement Learning Ming Yin Yu Bai Yu Wang OffRL 82 31 0 07 Jul 2020
RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learning Çağlar Gülçehre Ziyun Wang Alexander Novikov T. Paine Sergio Gomez Colmenarejo ... Matthew W. Hoffman Ofir Nachum George Tucker N. Heess Nando de Freitas OffRL 77 72 0 24 Jun 2020
Reinforcement Learning for Non-Stationary Markov Decision Processes: The Blessing of (More) Optimism Wang Chi Cheung D. Simchi-Levi Ruihao Zhu OffRL 73 96 0 24 Jun 2020
MOReL : Model-Based Offline Reinforcement Learning Rahul Kidambi Aravind Rajeswaran Praneeth Netrapalli Thorsten Joachims OffRL 98 676 0 12 May 2020
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems Sergey Levine Aviral Kumar George Tucker Justin Fu OffRL GP 576 2,044 0 04 May 2020
D4RL: Datasets for Deep Data-Driven Reinforcement Learning Justin Fu Aviral Kumar Ofir Nachum George Tucker Sergey Levine GP OffRL 229 1,381 0 15 Apr 2020
Is Temporal Difference Learning Optimal? An Instance-Dependent Analysis K. Khamaru A. Pananjady Feng Ruan Martin J. Wainwright Michael I. Jordan OffRL 71 49 0 16 Mar 2020
Q* Approximation Schemes for Batch Reinforcement Learning: A Theoretical Comparison Tengyang Xie Nan Jiang 134 35 0 09 Mar 2020
Minimax-Optimal Off-Policy Evaluation with Linear Function Approximation Yaqi Duan Mengdi Wang OffRL 139 152 0 21 Feb 2020
Asymptotically Efficient Off-Policy Evaluation for Tabular Reinforcement Learning Ming Yin Yu Wang OffRL 112 82 0 29 Jan 2020
Provably Efficient Exploration in Policy Optimization Qi Cai Zhuoran Yang Chi Jin Zhaoran Wang 66 283 0 12 Dec 2019
Problem Dependent Reinforcement Learning Bounds Which Can Identify Bandit Structure in MDPs Andrea Zanette Emma Brunskill 100 15 0 03 Nov 2019
Model-Based Reinforcement Learning with a Generative Model is Minimax Optimal Alekh Agarwal Sham Kakade Lin F. Yang OffRL 91 175 0 10 Jun 2019
Provably Efficient Q-Learning with Low Switching Cost Yu Bai Tengyang Xie Nan Jiang Yu Wang 73 94 0 30 May 2019
Information-Theoretic Considerations in Batch Reinforcement Learning Jinglin Chen Nan Jiang OOD OffRL 161 378 0 01 May 2019
Off-Policy Policy Gradient with State Distribution Correction Yao Liu Adith Swaminathan Alekh Agarwal Emma Brunskill OffRL 157 67 0 17 Apr 2019
Batch Policy Learning under Constraints Hoang Minh Le Cameron Voloshin Yisong Yue OffRL 60 335 0 20 Mar 2019
Tighter Problem-Dependent Regret Bounds in Reinforcement Learning without Domain Knowledge using Value Function Bounds Andrea Zanette Emma Brunskill OffRL 115 276 0 01 Jan 2019
Minimax Regret Bounds for Reinforcement Learning M. G. Azar Ian Osband Rémi Munos 92 778 0 16 Mar 2017
Contextual Decision Processes with Low Bellman Rank are PAC-Learnable Nan Jiang A. Krishnamurthy Alekh Agarwal John Langford Robert Schapire 154 421 0 29 Oct 2016
Doubly Robust Off-policy Value Evaluation for Reinforcement Learning Nan Jiang Lihong Li OffRL 217 624 0 11 Nov 2015
Gaussian Processes for Sample Efficient Reinforcement Learning with RMAX-like Exploration T. Jung Peter Stone GP OffRL 55 34 0 31 Jan 2012
Empirical Bernstein Bounds and Sample Variance Penalization Andreas Maurer Massimiliano Pontil 410 545 0 21 Jul 2009