A Tale of Two-Timescale Reinforcement Learning with the Tightest Finite-Time Bound

20 November 2019

Gal Dalal

Papers citing "A Tale of Two-Timescale Reinforcement Learning with the Tightest Finite-Time Bound"

21 / 21 papers shown

Title
$O(1/k)$ Finite-Time Bound for Non-Linear Two-Time-Scale Stochastic Approximation Siddharth Chandak 36 0 0 27 Apr 2025
Gaussian Approximation and Multiplier Bootstrap for Polyak-Ruppert Averaged Linear Stochastic Approximation with Applications to TD Learning S. Samsonov Eric Moulines Qi-Man Shao Zhuo-Song Zhang Alexey Naumov 33 4 0 26 May 2024
Fast Two-Time-Scale Stochastic Gradient Method with Applications in Reinforcement Learning Sihan Zeng Thinh T. Doan 56 5 0 15 May 2024
Central Limit Theorem for Two-Timescale Stochastic Approximation with Markovian Noise: Theory and Applications Jie Hu Vishwaraj Doshi Do Young Eun 38 4 0 17 Jan 2024
Tight Finite Time Bounds of Two-Time-Scale Linear Stochastic Approximation with Markovian Noise Shaan ul Haque S. Khodadadian S. T. Maguluri 44 11 0 31 Dec 2023
Finite-Time Analysis of Whittle Index based Q-Learning for Restless Multi-Armed Bandits with Neural Network Function Approximation Guojun Xiong Jian Li 38 13 0 03 Oct 2023
High-probability sample complexities for policy evaluation with linear function approximation Gen Li Weichen Wu Yuejie Chi Cong Ma Alessandro Rinaldo Yuting Wei OffRL 38 7 0 30 May 2023
n-Step Temporal Difference Learning with Optimal n Lakshmi Mandal S. Bhatnagar 34 2 0 13 Mar 2023
Finite-Time Error Bounds for Greedy-GQ Yue Wang Yi Zhou Shaofeng Zou 34 1 0 06 Sep 2022
A Single-Timescale Analysis For Stochastic Approximation With Multiple Coupled Sequences Han Shen Tianyi Chen 54 15 0 21 Jun 2022
Convergence Rates of Two-Time-Scale Gradient Descent-Ascent Dynamics for Solving Nonconvex Min-Max Problems Thinh T. Doan 22 15 0 17 Dec 2021
Gradient Temporal Difference with Momentum: Stability and Convergence Rohan Deb S. Bhatnagar 19 5 0 22 Nov 2021
A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learning Sihan Zeng Thinh T. Doan Justin Romberg 67 22 0 29 Sep 2021
Online Robust Reinforcement Learning with Model Uncertainty Yue Wang Shaofeng Zou OOD OffRL 76 97 0 29 Sep 2021
Finite-Time Convergence Rates of Nonlinear Two-Time-Scale Stochastic Approximation under Markovian Noise Thinh T. Doan 21 15 0 04 Apr 2021
CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guarantee Tengyu Xu Yingbin Liang Guanghui Lan 52 122 0 11 Nov 2020
Nonlinear Two-Time-Scale Stochastic Approximation: Convergence and Finite-Time Performance Thinh T. Doan 14 45 0 03 Nov 2020
Online Algorithms for Estimating Change Rates of Web Pages Konstantin Avrachenkov Kishor P. Patil Gugan Thoppe 24 16 0 17 Sep 2020
A Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Critic Mingyi Hong Hoi-To Wai Zhaoran Wang Zhuoran Yang 18 135 0 10 Jul 2020
Non-asymptotic Convergence Analysis of Two Time-scale (Natural) Actor-Critic Algorithms Tengyu Xu Zhe Wang Yingbin Liang 26 57 0 07 May 2020
On the Sample Complexity of Actor-Critic Method for Reinforcement Learning with Function Approximation Harshat Kumar Alec Koppel Alejandro Ribeiro 104 80 0 18 Oct 2019