An Improved Analysis of (Variance-Reduced) Policy Gradient and Natural Policy Gradient Methods

15 November 2022

Papers citing "An Improved Analysis of (Variance-Reduced) Policy Gradient and Natural Policy Gradient Methods"

50 / 73 papers shown

Title
Enhancing PPO with Trajectory-Aware Hybrid Policies Qisai Liu Zhanhong Jiang Hsin-Jung Yang Mahsa Khosravi Joshua R. Waite S. Sarkar 49 0 0 21 Feb 2025
A learning-based approach to stochastic optimal control under reach-avoid constraint Tingting Ni Maryam Kamgarpour 82 0 0 21 Dec 2024
Last-Iterate Convergence of General Parameterized Policies in Constrained MDPs Washim Uddin Mondal Vaneet Aggarwal 41 1 0 21 Aug 2024
Momentum for the Win: Collaborative Federated Reinforcement Learning across Heterogeneous Environments Han Wang Sihong He Zhili Zhang Fei Miao James Anderson 51 3 0 29 May 2024
Fast Stochastic Policy Gradient: Negative Momentum for Reinforcement Learning Haobin Zhang Zhuang Yang 32 0 0 08 May 2024
Linear Convergence of Independent Natural Policy Gradient in Games with Entropy Regularization Youbang Sun Tao-Wen Liu P. R. Kumar Shahin Shahrampour 37 0 0 04 May 2024
Learning Optimal Deterministic Policies with Stochastic Policy Gradients Alessandro Montenegro Marco Mussi Alberto Maria Metelli Matteo Papini 48 2 0 03 May 2024
Asynchronous Federated Reinforcement Learning with Policy Gradient Updates: Algorithm Design and Convergence Analysis Guangchen Lan Dong-Jun Han Abolfazl Hashemi Vaneet Aggarwal Christopher G. Brinton 124 15 0 09 Apr 2024
Order-Optimal Regret with Novel Policy Gradient Approaches in Infinite-Horizon Average Reward MDPs Swetha Ganesh Washim Uddin Mondal Vaneet Aggarwal 49 3 0 02 Apr 2024
Towards Global Optimality for Practical Average Reward Reinforcement Learning without Mixing Time Oracles Bhrij Patel Wesley A Suttle Alec Koppel Vaneet Aggarwal Brian M Sadler Amrit Singh Bedi Dinesh Manocha 34 1 0 18 Mar 2024
Global Convergence Guarantees for Federated Policy Gradient Methods with Adversaries Swetha Ganesh Jiayu Chen Gugan Thoppe Vaneet Aggarwal FedML 64 1 0 15 Mar 2024
On the Stochastic (Variance-Reduced) Proximal Gradient Method for Regularized Expected Reward Optimization Ling Liang Haizhao Yang 14 1 0 23 Jan 2024
Global Convergence of Natural Policy Gradient with Hessian-aided Momentum Variance Reduction Jie Feng Ke Wei Jinchi Chen 36 1 0 02 Jan 2024
PPO-Clip Attains Global Optimality: Towards Deeper Understandings of Clipping Nai-Chieh Huang Ping-Chun Hsieh Kuo-Hao Ho I-Chen Wu 21 8 0 19 Dec 2023
A safe exploration approach to constrained Markov decision processes Tingting Ni Maryam Kamgarpour 36 3 0 01 Dec 2023
Improved Sample Complexity Analysis of Natural Policy Gradient Algorithm with General Parameterization for Infinite Horizon Discounted Reward Markov Decision Processes Washim Uddin Mondal Vaneet Aggarwal 30 9 0 18 Oct 2023
When is Agnostic Reinforcement Learning Statistically Tractable? Zeyu Jia Gene Li Alexander Rakhlin Ayush Sekhari Nathan Srebro OffRL 29 5 0 09 Oct 2023
Improved Communication Efficiency in Federated Natural Policy Gradient via ADMM-based Gradient Updates Guangchen Lan Han Wang James Anderson Christopher G. Brinton Vaneet Aggarwal FedML 29 27 0 09 Oct 2023
Accelerating Large Batch Training via Gradient Signal to Noise Ratio (GSNR) Guo-qing Jiang Jinlong Liu Zixiang Ding Lin Guo W. Lin AI4CE 24 1 0 24 Sep 2023
Oracle Complexity Reduction for Model-free LQR: A Stochastic Variance-Reduced Policy Gradient Approach Leonardo F. Toso Han Wang James Anderson 37 2 0 19 Sep 2023
Regret Analysis of Policy Gradient Algorithm for Infinite Horizon Average Reward Markov Decision Processes Qinbo Bai Washim Uddin Mondal Vaneet Aggarwal 34 9 0 05 Sep 2023
On the Global Convergence of Natural Actor-Critic with Two-layer Neural Network Parametrization Mudit Gaur Amrit Singh Bedi Di-di Wang Vaneet Aggarwal 40 3 0 18 Jun 2023
Reinforcement Learning with General Utilities: Simpler Variance Reduction and Large State-Action Space Anas Barakat Ilyas Fatkhullin Niao He 26 11 0 02 Jun 2023
Regret-Optimal Model-Free Reinforcement Learning for Discounted MDPs with Short Burn-In Time Xiang Ji Gen Li OffRL 32 7 0 24 May 2023
Deep Metric Tensor Regularized Policy Gradient Gang Chen Victoria Huang 28 0 0 18 May 2023
Policy Gradient Converges to the Globally Optimal Policy for Nearly Linear-Quadratic Regulators Yin-Huan Han Meisam Razaviyayn Renyuan Xu 27 5 0 15 Mar 2023
Optimal Convergence Rate for Exact Policy Mirror Descent in Discounted Markov Decision Processes Emmeran Johnson Ciara Pike-Burke Patrick Rebeschini 28 11 0 22 Feb 2023
Stochastic Policy Gradient Methods: Improved Sample Complexity for Fisher-non-degenerate Policies Ilyas Fatkhullin Anas Barakat Anastasia Kireeva Niao He 32 37 0 03 Feb 2023
A Novel Framework for Policy Mirror Descent with General Parameterization and Linear Convergence Carlo Alfano Rui Yuan Patrick Rebeschini 65 15 0 30 Jan 2023
Stochastic Dimension-reduced Second-order Methods for Policy Optimization Jinsong Liu Chen Xie Qinwen Deng Dongdong Ge Yi-Li Ye 29 1 0 28 Jan 2023
Mean-Field Control based Approximation of Multi-Agent Reinforcement Learning in Presence of a Non-decomposable Shared Global State Washim Uddin Mondal Vaneet Aggarwal S. Ukkusuri 33 8 0 13 Jan 2023
Coordinate Ascent for Off-Policy RL with Global Convergence Guarantees Hsin-En Su Yen-Ju Chen Ping-Chun Hsieh Xi Liu OffRL 23 0 0 10 Dec 2022
On the Global Convergence of Fitted Q-Iteration with Two-layer Neural Network Parametrization Mudit Gaur Vaneet Aggarwal Mridul Agarwal MLT 38 1 0 14 Nov 2022
Decentralized Policy Gradient for Nash Equilibria Learning of General-sum Stochastic Games Yan Chen Taoying Li 16 2 0 14 Oct 2022
SoftTreeMax: Policy Gradient with Tree Search Gal Dalal Assaf Hallak Shie Mannor Gal Chechik 24 1 0 28 Sep 2022
A Robust and Constrained Multi-Agent Reinforcement Learning Electric Vehicle Rebalancing Method in AMoD Systems Sihong He Yue Wang Shuo Han Shaofeng Zou Fei Miao 35 11 0 17 Sep 2022
On the Near-Optimality of Local Policies in Large Cooperative Multi-Agent Reinforcement Learning Washim Uddin Mondal Vaneet Aggarwal S. Ukkusuri 28 5 0 07 Sep 2022
An Approximate Policy Iteration Viewpoint of Actor-Critic Algorithms Zaiwei Chen S. T. Maguluri 26 0 0 05 Aug 2022
Achieving Zero Constraint Violation for Constrained Reinforcement Learning via Conservative Natural Policy Gradient Primal-Dual Algorithm Qinbo Bai Amrit Singh Bedi Vaneet Aggarwal 26 20 0 12 Jun 2022
Finite-Time Analysis of Fully Decentralized Single-Timescale Actor-Critic Qijun Luo Xiao Li 30 1 0 12 Jun 2022
Policy Optimization for Markov Games: Unified Framework and Faster Convergence Runyu Zhang Qinghua Liu Haiquan Wang Caiming Xiong Na Li Yu Bai 27 26 0 06 Jun 2022
Convergence and sample complexity of natural policy gradient primal-dual methods for constrained MDPs Dongsheng Ding Kaipeng Zhang Jiali Duan Tamer Bacsar Mihailo R. Jovanović 20 19 0 06 Jun 2022
Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function Saeed Masiha Saber Salehkaleybar Niao He Negar Kiyavash Patrick Thiran 87 18 0 25 May 2022
Momentum-Based Policy Gradient with Second-Order Information Saber Salehkaleybar Sadegh Khorasani Negar Kiyavash Niao He Patrick Thiran 31 9 0 17 May 2022
Independent Natural Policy Gradient Methods for Potential Games: Finite-time Global Convergence with Entropy Regularization Shicong Cen Fan Chen Yuejie Chi 33 15 0 12 Apr 2022
Deep Reinforcement Learning for Data-Driven Adaptive Scanning in Ptychography M. Schloz Johannes Müller T. Pekin W. V. D. Broek C. Koch 33 7 0 29 Mar 2022
Can Mean Field Control (MFC) Approximate Cooperative Multi Agent Reinforcement Learning (MARL) with Non-Uniform Interaction? Washim Uddin Mondal Vaneet Aggarwal S. Ukkusuri 23 9 0 28 Feb 2022
Understanding Curriculum Learning in Policy Optimization for Online Combinatorial Optimization Runlong Zhou Zelin He Yuandong Tian Yi Wu S. Du OffRL 23 3 0 11 Feb 2022
Single Time-scale Actor-critic Method to Solve the Linear Quadratic Regulator with Convergence Guarantees Mo Zhou Jianfeng Lu 26 13 0 31 Jan 2022
Recent Advances in Reinforcement Learning in Finance B. Hambly Renyuan Xu Huining Yang OffRL 27 166 0 08 Dec 2021