Where Did My Optimum Go?: An Empirical Analysis of Gradient Descent Optimization in Policy Gradient Methods

5 October 2018

Papers citing "Where Did My Optimum Go?: An Empirical Analysis of Gradient Descent Optimization in Policy Gradient Methods"

23 / 23 papers shown

Title
Iterative Self-Incentivization Empowers Large Language Models as Agentic Searchers Zhengliang Shi Lingyong Yan Dawei Yin Suzan Verberne Maarten de Rijke Zhaochun Ren LRM 113 1 0 26 May 2025
Adam on Local Time: Addressing Nonstationarity in RL with Relative Adam Timesteps Benjamin Ellis Matthew Jackson Andrei Lupu Alexander David Goldie Mattie Fellows Shimon Whiteson Jakob Foerster 142 3 0 22 Dec 2024
$Segmenting Action-Value Functions Over Time-Scales in SARSA via TD($\Delta$)$ Segmenting Action-Value Functions Over Time-Scales in SARSA via TD( $\Delta$ ) Mahammad Humayoo 78 0 0 22 Nov 2024
Can Learned Optimization Make Reinforcement Learning Less Difficult? Alexander David Goldie Chris Xiaoxuan Lu Matthew Jackson Shimon Whiteson Jakob N. Foerster 130 5 0 09 Jul 2024
Task-optimal data-driven surrogate models for eNMPC via differentiable simulation and optimization Daniel Mayfrank Na Young Ahn Alexander Mitsos Manuel Dahmen 134 2 0 21 Mar 2024
Efficiently Quantifying Individual Agent Importance in Cooperative MARL Omayma Mahjoub Ruan de Kock Siddarth S. Singh Wiem Khlifi Abidine Vall Kale-ab Tessera Arnu Pretorius FAtt 78 2 0 13 Dec 2023
Aligning Language Models with Offline Learning from Human Feedback Jian Hu Li Tao J. Yang Chandler Zhou ALM OffRL 80 7 0 23 Aug 2023
Stepsize Learning for Policy Gradient Methods in Contextual Markov Decision Processes Luca Sabbioni Francesco Corda Marcello Restelli 47 0 0 13 Jun 2023
Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlo Haque Ishfaq Qingfeng Lan Pan Xu A. R. Mahmood Doina Precup Anima Anandkumar Kamyar Azizzadenesheli BDL OffRL 110 23 0 29 May 2023
Learning to Optimize for Reinforcement Learning Qingfeng Lan Rupam Mahmood Shuicheng Yan Zhongwen Xu OffRL 83 7 0 03 Feb 2023
Design and Planning of Flexible Mobile Micro-Grids Using Deep Reinforcement Learning Cesare Caputo Michel-Alexandre Cardin Pudong Ge Fei Teng A. Korre Ehecatl Antonio del Rio Chanona 39 18 0 08 Dec 2022
A Scalable Finite Difference Method for Deep Reinforcement Learning Matthew Allen John C. Raisbeck Hakho Lee 50 0 0 14 Oct 2022
Application of Monte Carlo Stochastic Optimization (MOST) to Deep Learning Sin-ichi Inage Hana Hebishima 46 10 0 02 Sep 2021
Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspective Florin Gogianu Tudor Berariu Mihaela Rosca Claudia Clopath L. Buşoniu Razvan Pascanu 86 56 0 11 May 2021
Parameter-free Gradient Temporal Difference Learning Andrew Jacobsen Alan Chan OffRL 59 2 0 10 May 2021
On Proximal Policy Optimization's Heavy-tailed Gradients Saurabh Garg Joshua Zhanson Emilio Parisotto Adarsh Prasad J. Zico Kolter Zachary Chase Lipton Sivaraman Balakrishnan Ruslan Salakhutdinov Pradeep Ravikumar 98 13 0 20 Feb 2021
TDprop: Does Jacobi Preconditioning Help Temporal Difference Learning? Joshua Romoff Peter Henderson David Kanaa Emmanuel Bengio Ahmed Touati Pierre-Luc Bacon Joelle Pineau 58 3 0 06 Jul 2020
Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO Logan Engstrom Andrew Ilyas Shibani Santurkar Dimitris Tsipras Firdaus Janoos L. Rudolph Aleksander Madry AAML 76 229 0 25 May 2020
Towards Combining On-Off-Policy Methods for Real-World Applications Kai-Chun Hu Chen-Huan Pi Ting Han Wei I-Chen Wu Stone Cheng Yi-Wei Dai Wei-Yuan Ye OffRL 30 2 0 24 Apr 2019
Fast Efficient Hyperparameter Tuning for Policy Gradients Supratik Paul Vitaly Kurin Shimon Whiteson 72 32 0 18 Feb 2019
Separating value functions across time-scales Joshua Romoff Peter Henderson Ahmed Touati Emma Brunskill Joelle Pineau Yann Ollivier 78 25 0 05 Feb 2019
Understanding the impact of entropy on policy optimization Zafarali Ahmed Nicolas Le Roux Mohammad Norouzi Dale Schuurmans 81 238 0 27 Nov 2018
A Closer Look at Deep Policy Gradients Andrew Ilyas Logan Engstrom Shibani Santurkar Dimitris Tsipras Firdaus Janoos Larry Rudolph Aleksander Madry 87 51 0 06 Nov 2018