v1v2 (latest)

On the Convergence and Sample Efficiency of Variance-Reduced Policy Gradient Method

17 February 2021

Mengdi Wang

Papers citing "On the Convergence and Sample Efficiency of Variance-Reduced Policy Gradient Method"

38 / 38 papers shown

Title
Online Episodic Convex Reinforcement Learning B. Moreno Khaled Eldowa Pierre Gaillard Margaux Brégère Nadia Oudjane OffRL 184 0 0 12 May 2025
Small steps no more: Global convergence of stochastic gradient bandits for arbitrary learning rates Jincheng Mei Bo Dai Alekh Agarwal Sharan Vaswani Anant Raj Csaba Szepesvári Dale Schuurmans 114 0 0 11 Feb 2025
Policy Mirror Descent for Reinforcement Learning: Linear Convergence, New Sampling Complexity, and Generalized Problem Classes Guanghui Lan 177 143 0 30 Jan 2021
Sample Efficient Reinforcement Learning with REINFORCE Junzi Zhang Jongho Kim Brendan O'Donoghue Stephen P. Boyd 84 107 0 22 Oct 2020
Variational Policy Gradient Method for Reinforcement Learning with General Utilities Junyu Zhang Alec Koppel Amrit Singh Bedi Csaba Szepesvári Mengdi Wang 64 140 0 04 Jul 2020
On the Global Convergence Rates of Softmax Policy Gradient Methods Jincheng Mei Chenjun Xiao Csaba Szepesvári Dale Schuurmans 145 294 0 13 May 2020
A Hybrid Stochastic Policy Gradient Algorithm for Reinforcement Learning Nhan H. Pham Lam M. Nguyen Dzung Phan Phuong Ha Nguyen Marten van Dijk Quoc Tran-Dinh 59 25 0 01 Mar 2020
Cautious Reinforcement Learning via Distributional Risk in the Dual Domain Junyu Zhang Amrit Singh Bedi Mengdi Wang Alec Koppel 63 28 0 27 Feb 2020
Sample Efficient Policy Gradient Methods with Recursive Variance Reduction Pan Xu F. Gao Quanquan Gu 82 89 0 18 Sep 2019
Adaptive Trust Region Policy Optimization: Global Convergence and Faster Rates for Regularized MDPs Lior Shani Yonathan Efroni Shie Mannor 57 176 0 06 Sep 2019
Multi-Level Composite Stochastic Optimization via Nested Variance Reduction Junyu Zhang Lin Xiao 116 56 0 29 Aug 2019
On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shift Alekh Agarwal Sham Kakade Jason D. Lee G. Mahajan 72 321 0 01 Aug 2019
A Stochastic Composite Gradient Method with Incremental Variance Reduction Junyu Zhang Lin Xiao 51 68 0 24 Jun 2019
Reinforcement Learning with Convex Constraints Sobhan Miryoosefi Kianté Brantley Hal Daumé Miroslav Dudík Robert Schapire 51 92 0 21 Jun 2019
Global Convergence of Policy Gradient Methods to (Almost) Locally Optimal Policies Kai Zhang Alec Koppel Haoqi Zhu Tamer Basar 70 191 0 19 Jun 2019
Global Optimality Guarantees For Policy Gradient Methods Jalaj Bhandari Daniel Russo 82 194 0 05 Jun 2019
An Improved Convergence Analysis of Stochastic Variance-Reduced Policy Gradient Pan Xu F. Gao Quanquan Gu 69 97 0 29 May 2019
Provably Efficient Maximum Entropy Exploration Elad Hazan Sham Kakade Karan Singh A. V. Soest 81 304 0 06 Dec 2018
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator Cong Fang C. J. Li Zhouchen Lin Tong Zhang 95 580 0 04 Jul 2018
Stochastic Variance-Reduced Policy Gradient Matteo Papini Damiano Binaghi Giuseppe Canonaco Matteo Pirotta Marcello Restelli 81 178 0 14 Jun 2018
Scalable Bilinear $π$ Learning Using State and Action Features Yichen Chen Lihong Li Mengdi Wang 67 46 0 27 Apr 2018
Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator Maryam Fazel Rong Ge Sham Kakade M. Mesbahi 86 609 0 15 Jan 2018
Accelerated Method for Stochastic Composition Optimization with Nonsmooth Regularization Zhouyuan Huo Bin Gu Ji Liu Heng-Chiao Huang 87 51 0 10 Nov 2017
Variance Reduced Value Iteration and Faster Algorithms for Solving Markov Decision Processes Aaron Sidford Mengdi Wang X. Wu Yinyu Ye 63 128 0 27 Oct 2017
Proximal Policy Optimization Algorithms John Schulman Filip Wolski Prafulla Dhariwal Alec Radford Oleg Klimov OffRL 541 19,265 0 20 Jul 2017
SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient Lam M. Nguyen Jie Liu K. Scheinberg Martin Takáč ODL 171 607 0 01 Mar 2017
Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving Shai Shalev-Shwartz Shaked Shammah Amnon Shashua 111 840 0 11 Oct 2016
Accelerating Stochastic Composition Optimization Mengdi Wang Ji Liu Ethan X. Fang 70 148 0 25 Jul 2016
Stochastic Variance Reduction for Nonconvex Optimization Sashank J. Reddi Ahmed S. Hefny S. Sra Barnabás Póczós Alex Smola 106 604 0 19 Mar 2016
Fast Incremental Method for Nonconvex Optimization Sashank J. Reddi S. Sra Barnabás Póczós Alex Smola 81 44 0 19 Mar 2016
Variance Reduction for Faster Non-Convex Optimization Zeyuan Allen-Zhu Elad Hazan ODL 120 392 0 17 Mar 2016
Asynchronous Methods for Deep Reinforcement Learning Volodymyr Mnih Adria Puigdomenech Badia M. Berk Mirza Alex Graves Timothy Lillicrap Tim Harley David Silver Koray Kavukcuoglu 207 8,879 0 04 Feb 2016
Trust Region Policy Optimization John Schulman Sergey Levine Philipp Moritz Michael I. Jordan Pieter Abbeel 279 6,801 0 19 Feb 2015
Stochastic Compositional Gradient Descent: Algorithms for Minimizing Compositions of Expected-Value Functions Mengdi Wang Ethan X. Fang Han Liu 104 264 0 14 Nov 2014
SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives Aaron Defazio Francis R. Bach Simon Lacoste-Julien ODL 137 1,828 0 01 Jul 2014
Minimizing Finite Sums with the Stochastic Average Gradient Mark Schmidt Nicolas Le Roux Francis R. Bach 327 1,250 0 10 Sep 2013
On the Sample Complexity of Reinforcement Learning with a Generative Model M. G. Azar Rémi Munos H. Kappen 79 156 0 27 Jun 2012
Infinite-Horizon Policy-Gradient Estimation Jonathan Baxter Peter L. Bartlett 102 812 0 03 Jun 2011