Finite Time Analysis of Linear Two-timescale Stochastic Approximation with Markovian Noise

4 February 2020

Papers citing "Finite Time Analysis of Linear Two-timescale Stochastic Approximation with Markovian Noise"

25 / 25 papers shown

Title
$O(1/k)$ Finite-Time Bound for Non-Linear Two-Time-Scale Stochastic Approximation Siddharth Chandak 38 0 0 27 Apr 2025
Non-Expansive Mappings in Two-Time-Scale Stochastic Approximation: Finite-Time Analysis Siddharth Chandak 52 1 0 18 Jan 2025
Central Limit Theorem for Two-Timescale Stochastic Approximation with Markovian Noise: Theory and Applications Jie Hu Vishwaraj Doshi Do Young Eun 40 4 0 17 Jan 2024
Tight Finite Time Bounds of Two-Time-Scale Linear Stochastic Approximation with Markovian Noise Shaan ul Haque S. Khodadadian S. T. Maguluri 46 11 0 31 Dec 2023
Finite-Time Analysis of Whittle Index based Q-Learning for Restless Multi-Armed Bandits with Neural Network Function Approximation Guojun Xiong Jian Li 38 13 0 03 Oct 2023
Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation Davoud Ataee Tarzanagh Mingchen Li Pranay Sharma Samet Oymak 36 0 0 02 Jun 2023
High-probability sample complexities for policy evaluation with linear function approximation Gen Li Weichen Wu Yuejie Chi Cong Ma Alessandro Rinaldo Yuting Wei OffRL 40 7 0 30 May 2023
Finite-Time Error Bounds for Greedy-GQ Yue Wang Yi Zhou Shaofeng Zou 34 1 0 06 Sep 2022
A Single-Timescale Analysis For Stochastic Approximation With Multiple Coupled Sequences Han Shen Tianyi Chen 57 15 0 21 Jun 2022
Convergence Rates of Two-Time-Scale Gradient Descent-Ascent Dynamics for Solving Nonconvex Min-Max Problems Thinh T. Doan 22 15 0 17 Dec 2021
Gradient Temporal Difference with Momentum: Stability and Convergence Rohan Deb S. Bhatnagar 19 5 0 22 Nov 2021
PER-ETD: A Polynomially Efficient Emphatic Temporal Difference Learning Method Ziwei Guan Tengyu Xu Yingbin Liang 26 4 0 13 Oct 2021
A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learning Sihan Zeng Thinh T. Doan Justin Romberg 67 22 0 29 Sep 2021
Online Robust Reinforcement Learning with Model Uncertainty Yue Wang Shaofeng Zou OOD OffRL 76 97 0 29 Sep 2021
Online Bootstrap Inference For Policy Evaluation in Reinforcement Learning Pratik Ramprasad Yuantong Li Zhuoran Yang Zhaoran Wang W. Sun Guang Cheng OffRL 52 27 0 08 Aug 2021
Analysis of a Target-Based Actor-Critic Algorithm with Linear Function Approximation Anas Barakat Pascal Bianchi Julien Lehmann 32 9 0 14 Jun 2021
Finite-Time Convergence Rates of Nonlinear Two-Time-Scale Stochastic Approximation under Markovian Noise Thinh T. Doan 24 15 0 04 Apr 2021
On the Stability of Random Matrix Product with Markovian Noise: Application to Linear Stochastic Approximation and TD Learning Alain Durmus Eric Moulines A. Naumov S. Samsonov Hoi-To Wai 38 19 0 30 Jan 2021
Nonlinear Two-Time-Scale Stochastic Approximation: Convergence and Finite-Time Performance Thinh T. Doan 14 45 0 03 Nov 2020
Variance-Reduced Off-Policy TDC Learning: Non-Asymptotic Convergence Analysis Shaocong Ma Yi Zhou Shaofeng Zou OffRL 22 14 0 26 Oct 2020
Online Algorithms for Estimating Change Rates of Web Pages Konstantin Avrachenkov Kishor P. Patil Gugan Thoppe 24 16 0 17 Sep 2020
A Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Critic Mingyi Hong Hoi-To Wai Zhaoran Wang Zhuoran Yang 18 135 0 10 Jul 2020
Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Model Gen Li Yuting Wei Yuejie Chi Yuxin Chen 39 125 0 26 May 2020
Non-asymptotic Convergence Analysis of Two Time-scale (Natural) Actor-Critic Algorithms Tengyu Xu Zhe Wang Yingbin Liang 26 57 0 07 May 2020
A Finite Time Analysis of Two Time-Scale Actor Critic Methods Yue Wu Weitong Zhang Pan Xu Quanquan Gu 92 146 0 04 May 2020