Sample Complexity of Asynchronous Q-Learning: Sharper Analysis and Variance Reduction

4 June 2020

Papers citing "Sample Complexity of Asynchronous Q-Learning: Sharper Analysis and Variance Reduction"

35 / 35 papers shown

Title
Mean-Field Sampling for Cooperative Multi-Agent Reinforcement Learning Emile Anand Ishani Karmarkar Guannan Qu 83 1 0 01 Dec 2024
Gap-Dependent Bounds for Q-Learning using Reference-Advantage Decomposition Zhong Zheng Haochen Zhang Lingzhou Xue OffRL 75 2 0 10 Oct 2024
The Bandit Whisperer: Communication Learning for Restless Bandits Yunfan Zhao Tonghan Wang Dheeraj M. Nagaraj Aparna Taneja Milind Tambe 49 5 0 11 Aug 2024
Reinforcement Learning with Intrinsically Motivated Feedback Graph for Lost-sales Inventory Control Zifan Liu Xinran Li Shibo Chen Gen Li Jiashuo Jiang Jun Zhang 37 0 0 26 Jun 2024
SwiftRL: Towards Efficient Reinforcement Learning on Real Processing-In-Memory Systems Kailash Gogineni Sai Santosh Dayapule Juan Gómez Luna Karthikeya Gogineni Peng Wei Tian-Shing Lan Mohammad Sadrosadati Onur Mutlu Guru Venkataramani 50 10 0 07 May 2024
Federated Offline Reinforcement Learning: Collaborative Single-Policy Coverage Suffices Jiin Woo Laixi Shi Gauri Joshi Yuejie Chi OffRL 29 3 0 08 Feb 2024
Settling the Sample Complexity of Online Reinforcement Learning Zihan Zhang Yuxin Chen Jason D. Lee S. Du OffRL 98 21 0 25 Jul 2023
Finite-Time Analysis of Minimax Q-Learning for Two-Player Zero-Sum Markov Games: Switching System Approach Dong-hwan Lee 21 2 0 09 Jun 2023
High-probability sample complexities for policy evaluation with linear function approximation Gen Li Weichen Wu Yuejie Chi Cong Ma Alessandro Rinaldo Yuting Wei OffRL 27 6 0 30 May 2023
Off-Policy RL Algorithms Can be Sample-Efficient for Continuous Control via Sample Multiple Reuse Jiafei Lyu Le Wan Zongqing Lu Xiu Li OffRL 34 9 0 29 May 2023
Regret-Optimal Model-Free Reinforcement Learning for Discounted MDPs with Short Burn-In Time Xiang Ji Gen Li OffRL 32 7 0 24 May 2023
Scalable Multi-Agent Reinforcement Learning with General Utilities Donghao Ying Yuhao Ding Alec Koppel Javad Lavaei 38 1 0 15 Feb 2023
Improved Regret Bounds for Linear Adversarial MDPs via Linear Optimization Fang-yuan Kong Xiangcheng Zhang Baoxiang Wang Shuai Li 26 12 0 14 Feb 2023
Finite-Time Analysis of Asynchronous Q-learning under Diminishing Step-Size from Control-Theoretic View Han-Dong Lim Dong-hwan Lee 24 1 0 25 Jul 2022
A Near-Optimal Primal-Dual Method for Off-Policy Learning in CMDP Fan Chen Junyu Zhang Zaiwen Wen OffRL 39 8 0 13 Jul 2022
Settling the Sample Complexity of Model-Based Offline Reinforcement Learning Gen Li Laixi Shi Yuxin Chen Yuejie Chi Yuting Wei OffRL 54 79 0 11 Apr 2022
The Efficacy of Pessimism in Asynchronous Q-Learning Yuling Yan Gen Li Yuxin Chen Jianqing Fan OffRL 78 40 0 14 Mar 2022
Target Network and Truncation Overcome The Deadly Triad in $Q$ -Learning Zaiwei Chen John-Paul Clarke S. T. Maguluri 18 19 0 05 Mar 2022
Instance-Dependent Confidence and Early Stopping for Reinforcement Learning K. Khamaru Eric Xia Martin J. Wainwright Michael I. Jordan 34 5 0 21 Jan 2022
A Statistical Analysis of Polyak-Ruppert Averaged Q-learning Xiang Li Wenhao Yang Jiadong Liang Zhihua Zhang Michael I. Jordan 40 15 0 29 Dec 2021
Finite-Sample Analysis of Decentralized Q-Learning for Stochastic Games Zuguang Gao Qianqian Ma Tamer Bacsar J. Birge OffRL 22 7 0 15 Dec 2021
Convergence Results For Q-Learning With Experience Replay Liran Szlak Ohad Shamir OffRL 29 5 0 08 Dec 2021
Breaking the Sample Complexity Barrier to Regret-Optimal Model-Free Reinforcement Learning Gen Li Laixi Shi Yuxin Chen Yuejie Chi OffRL 45 50 0 09 Oct 2021
Explaining Off-Policy Actor-Critic From A Bias-Variance Perspective Ting-Han Fan Peter J. Ramadge CML FAtt OffRL 21 2 0 06 Oct 2021
Online Robust Reinforcement Learning with Model Uncertainty Yue Wang Shaofeng Zou OOD OffRL 76 97 0 29 Sep 2021
Concentration of Contractive Stochastic Approximation and Reinforcement Learning Siddharth Chandak Vivek Borkar Parth Dodhia 43 17 0 27 Jun 2021
Gradient play in stochastic games: stationary points, convergence, and sample complexity Runyu Zhang Zhaolin Ren Na Li 26 43 0 01 Jun 2021
Sample-Efficient Reinforcement Learning Is Feasible for Linearly Realizable MDPs with Limited Revisiting Gen Li Yuxin Chen Yuejie Chi Yuantao Gu Yuting Wei OffRL 26 28 0 17 May 2021
Finite-Sample Analysis of Off-Policy Natural Actor-Critic Algorithm S. Khodadadian Zaiwei Chen S. T. Maguluri CML OffRL 71 26 0 18 Feb 2021
Is Q-Learning Minimax Optimal? A Tight Sample Complexity Analysis Gen Li Changxiao Cai Ee Yuting Wei Yuejie Chi OffRL 48 75 0 12 Feb 2021
A Lyapunov Theory for Finite-Sample Guarantees of Asynchronous Q-Learning and TD-Learning Variants Zaiwei Chen S. T. Maguluri Sanjay Shakkottai Karthikeyan Shanmugam OffRL 89 53 0 02 Feb 2021
Is Pessimism Provably Efficient for Offline RL? Ying Jin Zhuoran Yang Zhaoran Wang OffRL 27 346 0 30 Dec 2020
Finite-Time Analysis for Double Q-learning Huaqing Xiong Linna Zhao Yingbin Liang Wei Zhang 11 31 0 29 Sep 2020
Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Model Gen Li Yuting Wei Yuejie Chi Yuxin Chen 34 124 0 26 May 2020
Scalable Reinforcement Learning for Multi-Agent Networked Systems Guannan Qu Adam Wierman Na Li 16 33 0 05 Dec 2019