Accelerated and instance-optimal policy evaluation with linear function approximation

24 December 2021

Papers citing "Accelerated and instance-optimal policy evaluation with linear function approximation"

27 / 27 papers shown

Title
Optimal and instance-dependent guarantees for Markovian linear stochastic approximation Wenlong Mou A. Pananjady Martin J. Wainwright Peter L. Bartlett 48 23 0 23 Dec 2021
Optimal policy evaluation using kernel-based temporal difference methods Yaqi Duan Mengdi Wang Martin J. Wainwright OffRL 59 27 0 24 Sep 2021
Instance-optimality in optimal value estimation: Adaptivity via variance-reduced Q-learning K. Khamaru Eric Xia Martin J. Wainwright Michael I. Jordan OffRL 50 22 0 28 Jun 2021
Variance-Aware Off-Policy Evaluation with Linear Function Approximation Yifei Min Tianhao Wang Dongruo Zhou Quanquan Gu OffRL 75 38 0 22 Jun 2021
Is Q-Learning Minimax Optimal? A Tight Sample Complexity Analysis Gen Li Changxiao Cai Ee Yuting Wei Yuejie Chi OffRL 76 77 0 12 Feb 2021
A Lyapunov Theory for Finite-Sample Guarantees of Asynchronous Q-Learning and TD-Learning Variants Zaiwei Chen S. T. Maguluri Sanjay Shakkottai Karthikeyan Shanmugam OffRL 153 55 0 02 Feb 2021
On the Stability of Random Matrix Product with Markovian Noise: Application to Linear Stochastic Approximation and TD Learning Alain Durmus Eric Moulines A. Naumov S. Samsonov Hoi-To Wai 72 19 0 30 Jan 2021
Exponential Lower Bounds for Batch Reinforcement Learning: Batch RL can be Exponentially Harder than Online RL Andrea Zanette OffRL 149 71 0 14 Dec 2020
Optimal oracle inequalities for solving projected fixed-point equations Wenlong Mou A. Pananjady Martin J. Wainwright 55 14 0 09 Dec 2020
Simple and optimal methods for stochastic variational inequalities, II: Markovian noise and policy evaluation in reinforcement learning Georgios Kotsalis Guanghui Lan Tianjiao Li OffRL 41 31 0 15 Nov 2020
Simple and optimal methods for stochastic variational inequalities, I: operator extrapolation Georgios Kotsalis Guanghui Lan Tianjiao Li 37 61 0 05 Nov 2020
What are the Statistical Limits of Offline RL with Linear Function Approximation? Ruosong Wang Dean Phillips Foster Sham Kakade OffRL 160 163 0 22 Oct 2020
Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Model Gen Li Yuting Wei Yuejie Chi Yuxin Chen 99 129 0 26 May 2020
Is Temporal Difference Learning Optimal? An Instance-Dependent Analysis K. Khamaru A. Pananjady Feng Ruan Martin J. Wainwright Michael I. Jordan OffRL 64 48 0 16 Mar 2020
Reanalysis of Variance Reduced Temporal Difference Learning Tengyu Xu Zhe Wang Yi Zhou Yingbin Liang OffRL 75 38 0 07 Jan 2020
Variance-reduced $Q$ -learning is minimax optimal Martin J. Wainwright OffRL 59 92 0 11 Jun 2019
Finite-Time Error Bounds For Linear Stochastic Approximation and TD Learning R. Srikant Lei Ying 63 252 0 03 Feb 2019
Estimating the Mixing Time of Ergodic Markov Chains Geoffrey Wolfer A. Kontorovich 53 42 0 01 Feb 2019
Stochastic Variance-Reduced Policy Gradient Matteo Papini Damiano Binaghi Giuseppe Canonaco Matteo Pirotta Marcello Restelli 72 177 0 14 Jun 2018
A Finite Time Analysis of Temporal Difference Learning With Linear Function Approximation Jalaj Bhandari Daniel Russo Raghav Singal 106 339 0 06 Jun 2018
Multi-Agent Reinforcement Learning via Double Averaging Primal-Dual Optimization Hoi-To Wai Zhuoran Yang Zhaoran Wang Mingyi Hong 64 170 0 03 Jun 2018
Mixing time estimation in reversible Markov chains from a single sample path Daniel J. Hsu A. Kontorovich D. A. Levin Yuval Peres Csaba Szepesvári 67 83 0 24 Aug 2017
Stochastic Variance Reduction Methods for Policy Evaluation S. Du Jianshu Chen Lihong Li Lin Xiao Dengyong Zhou OffRL 39 157 0 25 Feb 2017
On TD(0) with function approximation: Concentration bounds and a centered variant with exponential convergence N. Korda Prashanth La 47 45 0 12 Nov 2014
SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives Aaron Defazio Francis R. Bach Simon Lacoste-Julien ODL 133 1,826 0 01 Jul 2014
A Proximal Stochastic Gradient Method with Progressive Variance Reduction Lin Xiao Tong Zhang ODL 156 738 0 19 Mar 2014
Minimizing Finite Sums with the Stochastic Average Gradient Mark Schmidt Nicolas Le Roux Francis R. Bach 321 1,248 0 10 Sep 2013