A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learning

31 December 2021

Bo Liu

Papers citing "A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learning"

34 / 34 papers shown

Title
TorchOpt: An Efficient Library for Differentiable Optimization Jie Ren Xidong Feng Bo Liu Xuehai Pan Yao Fu Kai Zou Yaodong Yang 52 12 0 13 Nov 2022
An Investigation of the Bias-Variance Tradeoff in Meta-Gradients Risto Vuorio Jacob Beck Shimon Whiteson Jakob N. Foerster Gregory Farquhar 64 7 0 22 Sep 2022
One Step at a Time: Pros and Cons of Multi-Step Meta-Gradient Reinforcement Learning Clément Bonnet Paul Caron Thomas D. Barrett Ian Davies Alexandre Laterre 32 5 0 30 Oct 2021
Unifying Gradient Estimators for Meta-Reinforcement Learning via Off-Policy Evaluation Yunhao Tang Tadashi Kozuno Mark Rowland Rémi Munos Michal Valko OffRL 100 9 0 24 Jun 2021
Discovery of Options via Meta-Learned Subgoals Vivek Veeriah Tom Zahavy Matteo Hessel Zhongwen Xu Junhyuk Oh Iurii Kemaev H. V. Hasselt David Silver Satinder Singh 46 33 0 12 Feb 2021
Learning to Utilize Shaping Rewards: A New Approach of Reward Shaping Yujing Hu Weixun Wang Hangtian Jia Yixiang Wang Yingfeng Chen Jianye Hao Feng Wu Changjie Fan OffRL 57 176 0 05 Nov 2020
A Policy Gradient Algorithm for Learning to Learn in Multiagent Reinforcement Learning Dong-Ki Kim Miao Liu Matthew D Riemer Chuangchuang Sun Marwa Abdulhai Golnaz Habibi Sebastian Lopez-Cot Gerald Tesauro Jonathan P. How 46 56 0 31 Oct 2020
Towards Effective Context for Meta-Reinforcement Learning: an Approach based on Contrastive Learning Haotian Fu Hongyao Tang Jianye Hao Chong Chen Xidong Feng Dong Li Wulong Liu OffRL 58 50 0 29 Sep 2020
Discovering Reinforcement Learning Algorithms Junhyuk Oh Matteo Hessel Wojciech M. Czarnecki Zhongwen Xu H. V. Hasselt Satinder Singh David Silver 62 129 0 17 Jul 2020
Meta-Gradient Reinforcement Learning with an Objective Discovered Online Zhongwen Xu H. V. Hasselt Matteo Hessel Junhyuk Oh Satinder Singh David Silver 71 77 0 16 Jul 2020
Online Meta-Critic Learning for Off-Policy Actor-Critic Methods Wei Zhou Yiying Li Yongxin Yang Huaimin Wang Timothy M. Hospedales OffRL 55 47 0 11 Mar 2020
Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learning Kaiyi Ji Junjie Yang Yingbin Liang 75 50 0 18 Feb 2020
On the Convergence Theory of Debiased Model-Agnostic Meta-Reinforcement Learning Alireza Fallah Kristian Georgiev Aryan Mokhtari Asuman Ozdaglar 114 23 0 12 Feb 2020
What Can Learned Intrinsic Rewards Capture? Zeyu Zheng Junhyuk Oh Matteo Hessel Zhongwen Xu M. Kroiss H. V. Hasselt David Silver Satinder Singh 50 77 0 11 Dec 2019
Improving Generalization in Meta Reinforcement Learning using Learned Objectives Louis Kirsch Sjoerd van Steenkiste Jürgen Schmidhuber OffRL 69 119 0 09 Oct 2019
Loaded DiCE: Trading off Bias and Variance in Any-Order Score Function Estimators for Reinforcement Learning Gregory Farquhar Shimon Whiteson Jakob N. Foerster 56 17 0 23 Sep 2019
Discovery of Useful Questions as Auxiliary Tasks Vivek Veeriah Matteo Hessel Zhongwen Xu Richard L. Lewis Janarthanan Rajendran Junhyuk Oh H. V. Hasselt David Silver Satinder Singh LLMAG 61 85 0 10 Sep 2019
On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms Alireza Fallah Aryan Mokhtari Asuman Ozdaglar 78 225 0 27 Aug 2019
Meta-Learning via Learned Loss Sarah Bechtle Artem Molchanov Yevgen Chebotar Edward Grefenstette Ludovic Righetti Gaurav Sukhatme Franziska Meier 65 111 0 12 Jun 2019
Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables Kate Rakelly Aurick Zhou Deirdre Quillen Chelsea Finn Sergey Levine OffRL 78 653 0 19 Mar 2019
Stable Opponent Shaping in Differentiable Games Alistair Letcher Jakob N. Foerster David Balduzzi Tim Rocktaschel Shimon Whiteson 67 110 0 20 Nov 2018
ProMP: Proximal Meta-Policy Search Jonas Rothfuss Dennis Lee I. Clavera Tamim Asfour Pieter Abbeel 55 210 0 16 Oct 2018
Meta-Gradient Reinforcement Learning Zhongwen Xu H. V. Hasselt David Silver 104 324 0 24 May 2018
On Learning Intrinsic Rewards for Policy Gradient Methods Zeyu Zheng Junhyuk Oh Satinder Singh 57 205 0 17 Apr 2018
Meta-Reinforcement Learning of Structured Exploration Strategies Abhishek Gupta Russell Mendonca YuXuan Liu Pieter Abbeel Sergey Levine OffRL 84 342 0 20 Feb 2018
DiCE: The Infinitely Differentiable Monte-Carlo Estimator Jakob N. Foerster Gregory Farquhar Maruan Al-Shedivat Tim Rocktaschel Eric Xing Shimon Whiteson 44 97 0 14 Feb 2018
Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments Maruan Al-Shedivat Trapit Bansal Yuri Burda Ilya Sutskever Igor Mordatch Pieter Abbeel CLL 63 353 0 10 Oct 2017
Learning with Opponent-Learning Awareness Jakob N. Foerster Richard Y. Chen Maruan Al-Shedivat Shimon Whiteson Pieter Abbeel Igor Mordatch 89 538 0 13 Sep 2017
Online Learning Rate Adaptation with Hypergradient Descent A. G. Baydin R. Cornish David Martínez-Rubio Mark Schmidt Frank Wood ODL 61 246 0 14 Mar 2017
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 806 11,866 0 09 Mar 2017
Forward and Reverse Gradient-Based Hyperparameter Optimization Luca Franceschi Michele Donini P. Frasconi Massimiliano Pontil 207 416 0 06 Mar 2017
Learning to reinforcement learn Jane X. Wang Z. Kurth-Nelson Dhruva Tirumala Hubert Soyer Joel Z Leibo Rémi Munos Charles Blundell D. Kumaran M. Botvinick OffRL 95 977 0 17 Nov 2016
RL $^2$ : Fast Reinforcement Learning via Slow Reinforcement Learning Yan Duan John Schulman Xi Chen Peter L. Bartlett Ilya Sutskever Pieter Abbeel OffRL 76 1,015 0 09 Nov 2016
Asynchronous Methods for Deep Reinforcement Learning Volodymyr Mnih Adria Puigdomenech Badia M. Berk Mirza Alex Graves Timothy Lillicrap Tim Harley David Silver Koray Kavukcuoglu 189 8,833 0 04 Feb 2016