Exponential Lower Bounds for Batch Reinforcement Learning: Batch RL can be Exponentially Harder than Online RL

14 December 2020

Papers citing "Exponential Lower Bounds for Batch Reinforcement Learning: Batch RL can be Exponentially Harder than Online RL"

22 / 22 papers shown

Title
When is Agnostic Reinforcement Learning Statistically Tractable? Zeyu Jia Gene Li Alexander Rakhlin Ayush Sekhari Nathan Srebro OffRL 27 5 0 09 Oct 2023
VIPeR: Provably Efficient Algorithm for Offline RL with Neural Function Approximation Thanh Nguyen-Tang R. Arora OffRL 46 5 0 24 Feb 2023
When is Realizability Sufficient for Off-Policy Reinforcement Learning? Andrea Zanette OffRL 16 14 0 10 Nov 2022
Distributionally Robust Offline Reinforcement Learning with Linear Function Approximation Xiaoteng Ma Zhipeng Liang Jose H. Blanchet MingWen Liu Li Xia Jiheng Zhang Qianchuan Zhao Zhengyuan Zhou OOD OffRL 35 22 0 14 Sep 2022
Offline Reinforcement Learning with Differential Privacy Dan Qiao Yu-Xiang Wang OffRL 36 23 0 02 Jun 2022
Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning Andrea Zanette Martin J. Wainwright OOD 34 5 0 01 Jun 2022
Pessimism in the Face of Confounders: Provably Efficient Offline Reinforcement Learning in Partially Observable Markov Decision Processes Miao Lu Yifei Min Zhaoran Wang Zhuoran Yang OffRL 51 22 0 26 May 2022
Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimism Ming Yin Yaqi Duan Mengdi Wang Yu-Xiang Wang OffRL 32 65 0 11 Mar 2022
A Complete Characterization of Linear Estimators for Offline Policy Evaluation Juan C. Perdomo A. Krishnamurthy Peter L. Bartlett Sham Kakade OffRL 27 3 0 08 Mar 2022
Why Should I Trust You, Bellman? The Bellman Error is a Poor Replacement for Value Error Scott Fujimoto D. Meger Doina Precup Ofir Nachum S. Gu 30 31 0 28 Jan 2022
Accelerated and instance-optimal policy evaluation with linear function approximation Tianjiao Li Guanghui Lan A. Pananjady OffRL 37 13 0 24 Dec 2021
DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization Aviral Kumar Rishabh Agarwal Tengyu Ma Aaron Courville George Tucker Sergey Levine OffRL 31 65 0 09 Dec 2021
Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximation Dylan J. Foster A. Krishnamurthy D. Simchi-Levi Yunzong Xu OffRL 11 62 0 21 Nov 2021
Exploiting Action Impact Regularity and Exogenous State Variables for Offline Reinforcement Learning Vincent Liu James Wright Martha White OffRL 31 1 0 15 Nov 2021
Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learning Andrea Zanette Martin J. Wainwright Emma Brunskill OffRL 29 111 0 19 Aug 2021
Offline RL Without Off-Policy Evaluation David Brandfonbrener William F. Whitney Rajesh Ranganath Joan Bruna OffRL 42 161 0 16 Jun 2021
Bellman-consistent Pessimism for Offline Reinforcement Learning Tengyang Xie Ching-An Cheng Nan Jiang Paul Mineiro Alekh Agarwal OffRL LRM 22 269 0 13 Jun 2021
Optimal Uniform OPE and Model-based Offline Reinforcement Learning in Time-Homogeneous, Reward-Free and Task-Agnostic Settings Ming Yin Yu-Xiang Wang OffRL 26 19 0 13 May 2021
Cautiously Optimistic Policy Optimization and Exploration with Linear Function Approximation Andrea Zanette Ching-An Cheng Alekh Agarwal 32 52 0 24 Mar 2021
An Exponential Lower Bound for Linearly-Realizable MDPs with Constant Suboptimality Gap Yuanhao Wang Ruosong Wang Sham Kakade OffRL 37 43 0 23 Mar 2021
Infinite-Horizon Offline Reinforcement Learning with Linear Function Approximation: Curse of Dimensionality and Algorithm Lin Chen B. Scherrer Peter L. Bartlett OffRL 67 16 0 17 Mar 2021
Instabilities of Offline RL with Pre-Trained Neural Representation Ruosong Wang Yifan Wu Ruslan Salakhutdinov Sham Kakade OffRL 15 42 0 08 Mar 2021