Neural Policy Gradient Methods: Global Optimality and Rates of Convergence

29 August 2019

Papers citing "Neural Policy Gradient Methods: Global Optimality and Rates of Convergence"

50 / 76 papers shown

Title
Global Optimality of Single-Timescale Actor-Critic under Continuous State-Action Space: A Study on Linear Quadratic Regulator Xuyang Chen Jingliang Duan Lin Zhao 62 1 0 02 May 2025
A learning-based approach to stochastic optimal control under reach-avoid constraint Tingting Ni Maryam Kamgarpour 85 0 0 21 Dec 2024
Asynchronous Federated Reinforcement Learning with Policy Gradient Updates: Algorithm Design and Convergence Analysis Guangchen Lan Dong-Jun Han Abolfazl Hashemi Vaneet Aggarwal Christopher G. Brinton 124 15 0 09 Apr 2024
Order-Optimal Regret with Novel Policy Gradient Approaches in Infinite-Horizon Average Reward MDPs Swetha Ganesh Washim Uddin Mondal Vaneet Aggarwal 49 3 0 02 Apr 2024
Global Convergence Guarantees for Federated Policy Gradient Methods with Adversaries Swetha Ganesh Jiayu Chen Gugan Thoppe Vaneet Aggarwal FedML 68 1 0 15 Mar 2024
Regularized Q-Learning with Linear Function Approximation Jiachen Xi Alfredo Garcia P. Momcilovic 38 2 0 26 Jan 2024
Neural Network Approximation for Pessimistic Offline Reinforcement Learning Di Wu Yuling Jiao Li Shen Haizhao Yang Xiliang Lu OffRL 29 1 0 19 Dec 2023
An Invitation to Deep Reinforcement Learning Bernhard Jaeger Andreas Geiger OffRL OOD 80 5 0 13 Dec 2023
A safe exploration approach to constrained Markov decision processes Tingting Ni Maryam Kamgarpour 41 3 0 01 Dec 2023
Regret Analysis of Policy Gradient Algorithm for Infinite Horizon Average Reward Markov Decision Processes Qinbo Bai Washim Uddin Mondal Vaneet Aggarwal 34 9 0 05 Sep 2023
Provably Robust Temporal Difference Learning for Heavy-Tailed Rewards Semih Cayci A. Eryilmaz 23 2 0 20 Jun 2023
Semi-Infinitely Constrained Markov Decision Processes and Efficient Reinforcement Learning Liangyu Zhang Yang Peng Wenhao Yang Zhihua Zhang 21 1 0 29 Apr 2023
Matryoshka Policy Gradient for Entropy-Regularized RL: Convergence and Global Optimality François Ged M. H. Veiga 33 0 0 22 Mar 2023
Policy Gradient Converges to the Globally Optimal Policy for Nearly Linear-Quadratic Regulators Yin-Huan Han Meisam Razaviyayn Renyuan Xu 27 5 0 15 Mar 2023
Reinforcement Learning with Function Approximation: From Linear to Nonlinear Jihao Long Jiequn Han 27 5 0 20 Feb 2023
Provably Efficient Offline Goal-Conditioned Reinforcement Learning with General Function Approximation and Single-Policy Concentrability Hanlin Zhu Amy Zhang OffRL 27 2 0 07 Feb 2023
Importance Weighted Actor-Critic for Optimal Conservative Offline Reinforcement Learning Hanlin Zhu Paria Rashidinejad Jiantao Jiao OffRL 42 15 0 30 Jan 2023
Beyond Exponentially Fast Mixing in Average-Reward Reinforcement Learning via Multi-Level Monte Carlo Actor-Critic Wesley A Suttle Amrit Singh Bedi Bhrij Patel Brian M Sadler Alec Koppel Dinesh Manocha 31 14 0 28 Jan 2023
On the Convergence of Discounted Policy Gradient Methods Chris Nota 21 0 0 28 Dec 2022
Examining Policy Entropy of Reinforcement Learning Agents for Personalization Tasks Anton Dereventsov Andrew Starnes Clayton Webster 26 4 0 21 Nov 2022
An Improved Analysis of (Variance-Reduced) Policy Gradient and Natural Policy Gradient Methods Yanli Liu Kaipeng Zhang Tamer Basar W. Yin 48 102 0 15 Nov 2022
DC-MRTA: Decentralized Multi-Robot Task Allocation and Navigation in Complex Environments Aakriti Agrawal Senthil Hariharan Amrit Singh Bedi Tianyi Zhou 33 12 0 07 Sep 2022
Sampling Through the Lens of Sequential Decision Making J. Dou Alvin Pan Runxue Bao Haiyi Mao Lei Luo Zhi-Hong Mao 26 19 0 17 Aug 2022
Achieving Zero Constraint Violation for Constrained Reinforcement Learning via Conservative Natural Policy Gradient Primal-Dual Algorithm Qinbo Bai Amrit Singh Bedi Vaneet Aggarwal 26 20 0 12 Jun 2022
Algorithm for Constrained Markov Decision Process with Linear Convergence E. Gladin Maksim Lavrik-Karmazin K. Zainullina Varvara Rudenko Alexander V. Gasnikov Martin Takáč 33 6 0 03 Jun 2022
Independent Natural Policy Gradient Methods for Potential Games: Finite-time Global Convergence with Entropy Regularization Shicong Cen Fan Chen Yuejie Chi 35 15 0 12 Apr 2022
Jump-Start Reinforcement Learning Ikechukwu Uchendu Ted Xiao Yao Lu Banghua Zhu Mengyuan Yan ... Chuyuan Fu Cong Ma Jiantao Jiao Sergey Levine Karol Hausman OffRL OnRL 44 109 0 05 Apr 2022
Linear convergence of a policy gradient method for some finite horizon continuous time control problems C. Reisinger Wolfgang Stockinger Yufei Zhang 21 5 0 22 Mar 2022
Policy Optimization for Stochastic Shortest Path Liyu Chen Haipeng Luo Aviv A. Rosenberg 19 12 0 07 Feb 2022
Convergence of Policy Gradient for Entropy Regularized MDPs with Neural Network Approximation in the Mean-Field Regime B. Kerimkulov J. Leahy David Siska Lukasz Szpruch 33 11 0 18 Jan 2022
Efficient Performance Bounds for Primal-Dual Reinforcement Learning from Demonstrations Angeliki Kamoutsi G. Banjac John Lygeros OffRL 26 7 0 28 Dec 2021
Nearly Optimal Policy Optimization with Stable at Any Time Guarantee Tianhao Wu Yunchang Yang Han Zhong Liwei Wang S. Du Jiantao Jiao 55 14 0 21 Dec 2021
Differentially Private Regret Minimization in Episodic Markov Decision Processes Sayak Ray Chowdhury Xingyu Zhou 26 21 0 20 Dec 2021
Recent Advances in Reinforcement Learning in Finance B. Hambly Renyuan Xu Huining Yang OffRL 29 167 0 08 Dec 2021
Global Optimality and Finite Sample Analysis of Softmax Off-Policy Actor Critic under State Distribution Mismatch Shangtong Zhang Rémi Tachet des Combes Romain Laroche 30 10 0 04 Nov 2021
Finite-Time Complexity of Online Primal-Dual Natural Actor-Critic Algorithm for Constrained Markov Decision Processes Sihan Zeng Thinh T. Doan Justin Romberg 102 17 0 21 Oct 2021
Variance Reduction based Experience Replay for Policy Optimization Hua Zheng Wei Xie M. Feng OffRL 39 2 0 17 Oct 2021
Online Robust Reinforcement Learning with Model Uncertainty Yue Wang Shaofeng Zou OOD OffRL 76 97 0 29 Sep 2021
Theoretical Guarantees of Fictitious Discount Algorithms for Episodic Reinforcement Learning and Global Convergence of Policy Gradient Methods Xin Guo Anran Hu Junzi Zhang OffRL 31 6 0 13 Sep 2021
On the Approximation of Cooperative Heterogeneous Multi-Agent Reinforcement Learning (MARL) using Mean Field Control (MFC) Washim Uddin Mondal Mridul Agarwal Vaneet Aggarwal S. Ukkusuri 33 43 0 09 Sep 2021
Sample and Communication-Efficient Decentralized Actor-Critic Algorithms with Finite-Time Analysis Ziyi Chen Yi Zhou Rongrong Chen Shaofeng Zou 19 24 0 08 Sep 2021
Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learning Andrea Zanette Martin J. Wainwright Emma Brunskill OffRL 29 115 0 19 Aug 2021
Mean-Field Multi-Agent Reinforcement Learning: A Decentralized Network Approach Haotian Gu Xin Guo Xiaoli Wei Renyuan Xu OOD 42 36 0 05 Aug 2021
Towards General Function Approximation in Zero-Sum Markov Games Baihe Huang Jason D. Lee Zhaoran Wang Zhuoran Yang 33 47 0 30 Jul 2021
A general sample complexity analysis of vanilla policy gradient Rui Yuan Robert Mansel Gower A. Lazaric 79 62 0 23 Jul 2021
On the Sample Complexity and Metastability of Heavy-tailed Policy Search in Continuous Control Amrit Singh Bedi Anjaly Parayil Junyu Zhang Mengdi Wang Alec Koppel 38 15 0 15 Jun 2021
Analysis of a Target-Based Actor-Critic Algorithm with Linear Function Approximation Anas Barakat Pascal Bianchi Julien Lehmann 32 9 0 14 Jun 2021
Finite-Sample Analysis of Off-Policy Natural Actor-Critic with Linear Function Approximation Zaiwei Chen S. Khodadadian S. T. Maguluri OffRL 63 29 0 26 May 2021
On the Linear convergence of Natural Policy Gradient Algorithm S. Khodadadian P. Jhunjhunwala Sushil Mahavir Varma S. T. Maguluri 40 56 0 04 May 2021
Doubly Robust Off-Policy Actor-Critic: Convergence and Optimality Tengyu Xu Zhuoran Yang Zhaoran Wang Yingbin Liang OffRL 47 24 0 23 Feb 2021