v1v2 (latest)

MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL

11 October 2024

C. Voelcker

Marcel Hussing

Eric Eaton

Amir-massoud Farahmand

Igor Gilitschenski

ArXiv (abs)PDF HTML

Papers citing "MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL"

50 / 74 papers shown

Title
FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Control Younggyo Seo Carmelo Sferrazza Haoran Geng Michal Nauman Zhao-Heng Yin Pieter Abbeel OffRL 75 0 0 28 May 2025
Calibrated Value-Aware Model Learning with Probabilistic Environment Models C. Voelcker Anastasiia Pedan Arash Ahmadian Romina Abachi Igor Gilitschenski Amir-massoud Farahmand 60 0 0 28 May 2025
Hyperspherical Normalization for Scalable Deep Reinforcement Learning Hojoon Lee Youngdo Lee Takuma Seno Donghu Kim Peter Stone Jaegul Choo 181 4 0 21 Feb 2025
Weight Clipping for Deep Continual and Reinforcement Learning Mohamed Elsayed Qingfeng Lan Clare Lyle A. Rupam Mahmood 91 12 0 01 Jul 2024
When does Self-Prediction help? Understanding Auxiliary Tasks in Reinforcement Learning C. Voelcker Tyler Kastner Igor Gilitschenski Amir-massoud Farahmand SSL 90 6 0 25 Jun 2024
On the consistency of hyper-parameter selection in value-based deep reinforcement learning J. Obando-Ceron J. G. Araújo Rameswar Panda Pablo Samuel Castro 120 9 0 25 Jun 2024
Bounding-Box Inference for Error-Aware Model-Based Reinforcement Learning Erin J. Talvitie Zilei Shao Huiying Li Jinghan Hu Jacob Boerma Rory Zhao Xintong Wang OffRL 63 1 0 23 Jun 2024
Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous control Michal Nauman M. Ostaszewski Krzysztof Jankowski Piotr Milo's Marek Cygan OffRL 116 36 0 25 May 2024
Dissecting Deep RL with High Update Ratios: Combatting Value Divergence Marcel Hussing C. Voelcker Igor Gilitschenski Amir-massoud Farahmand Eric Eaton 100 3 0 09 Mar 2024
Stop Regressing: Training Value Functions via Classification for Scalable Deep RL Jesse Farebrother Jordi Orbay Q. Vuong Adrien Ali Taïga Yevgen Chebotar ... Sergey Levine Pablo Samuel Castro Aleksandra Faust Aviral Kumar Rishabh Agarwal OffRL 105 66 0 06 Mar 2024
Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learning Michal Nauman Michal Bortkiewicz Piotr Milo's Tomasz Trzciñski M. Ostaszewski Marek Cygan OffRL 95 23 0 01 Mar 2024
Disentangling the Causes of Plasticity Loss in Neural Networks Clare Lyle Zeyu Zheng Khimya Khetarpal H. V. Hasselt Razvan Pascanu James Martens Will Dabney AI4CE 130 38 0 29 Feb 2024
Bridging State and History Representations: Understanding Self-Predictive RL Tianwei Ni Benjamin Eysenbach Erfan Seyedsalehi Michel Ma Clement Gehring Aditya Mahajan Pierre-Luc Bacon AI4TS AI4CE 92 29 0 17 Jan 2024
Replay across Experiments: A Natural Extension of Off-Policy RL Dhruva Tirumala Thomas Lampe José Enrique Chen Tuomas Haarnoja Sandy Huang ... Tim Hertweck Leonard Hasenclever Martin Riedmiller N. Heess Markus Wulfmeier OffRL 105 8 0 27 Nov 2023
DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization Guowei Xu Ruijie Zheng Yongyuan Liang Xiyao Wang Zhecheng Yuan ... Shuzhen Li Yanjie Ze Hal Daumé Furong Huang Huazhe Xu 131 31 0 30 Oct 2023
TD-MPC2: Scalable, Robust World Models for Continuous Control Nicklas Hansen Hao Su Xiaolong Wang MU 131 159 0 25 Oct 2023
A Unified View on Solving Objective Mismatch in Model-Based Reinforcement Learning Ran Wei Nathan Lambert Anthony D. McDonald Alfredo Garcia Roberto Calandra 109 7 0 10 Oct 2023
Variance Control for Distributional Reinforcement Learning Qi Kuang Zhoufan Zhu Liwen Zhang Fan Zhou OffRL 143 3 0 30 Jul 2023
Distributional Model Equivalence for Risk-Sensitive Reinforcement Learning Tyler Kastner Murat A. Erdogdu Amir-massoud Farahmand OffRL 98 4 0 04 Jul 2023
PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learning Hojoon Lee Hanseul Cho Hyunseung Kim Daehoon Gwak Joonkee Kim Jaegul Choo Se-Young Yun Chulhee Yun OffRL 157 30 0 19 Jun 2023
Simplified Temporal Consistency Reinforcement Learning Yi Zhao Wenshuai Zhao Rinu Boney Arno Solin Joni Pajarinen OffRL 72 13 0 15 Jun 2023
For SALE: State-Action Representation Learning for Deep Reinforcement Learning Scott Fujimoto Wei-Di Chang Edward James Smith S. Gu Doina Precup David Meger OffRL 95 55 0 04 Jun 2023
Bigger, Better, Faster: Human-level Atari with human-level efficiency Max Schwarzer J. Obando-Ceron Rameswar Panda Marc G. Bellemare Rishabh Agarwal Pablo Samuel Castro OffRL 122 102 0 30 May 2023
Deep Reinforcement Learning with Plasticity Injection Evgenii Nikishin Junhyuk Oh Georg Ostrovski Clare Lyle Razvan Pascanu Will Dabney André Barreto OffRL 64 52 0 24 May 2023
Replicable Reinforcement Learning Eric Eaton Marcel Hussing Michael Kearns Jessica Sorrell OffRL 92 13 0 24 May 2023
Proto-Value Networks: Scaling Representation Learning with Auxiliary Tasks Jesse Farebrother Joshua Greaves Rishabh Agarwal Charline Le Lan Ross Goroshin Pablo Samuel Castro Marc G. Bellemare 101 29 0 25 Apr 2023
Efficient Deep Reinforcement Learning Requires Regulating Overfitting Qiyang Li Aviral Kumar Ilya Kostrikov Sergey Levine OffRL 118 36 0 20 Apr 2023
Loss of Plasticity in Continual Deep Reinforcement Learning Zaheer Abbas Rosie Zhao Joseph Modayil Adam White Marlos C. Machado CLL OffRL 118 85 0 13 Mar 2023
Synthetic Experience Replay Cong Lu Philip J. Ball Yee Whye Teh Jack Parker-Holder OffRL 170 80 0 12 Mar 2023
Understanding plasticity in neural networks Clare Lyle Zeyu Zheng Evgenii Nikishin Bernardo Avila-Pires Razvan Pascanu Will Dabney AI4CE 121 105 0 02 Mar 2023
The Dormant Neuron Phenomenon in Deep Reinforcement Learning Ghada Sokar Rishabh Agarwal Pablo Samuel Castro Utku Evci CLL 108 99 0 24 Feb 2023
Efficient Online Reinforcement Learning with Offline Data Philip J. Ball Laura M. Smith Ilya Kostrikov Sergey Levine OffRL OnRL 145 184 0 06 Feb 2023
Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objective Raj Ghugare Homanga Bharadhwaj Benjamin Eysenbach Sergey Levine Ruslan Salakhutdinov OffRL 106 27 0 18 Sep 2022
Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progress Rishabh Agarwal Max Schwarzer Pablo Samuel Castro Rameswar Panda Marc G. Bellemare OffRL OnRL 126 66 0 03 Jun 2022
The Primacy Bias in Deep Reinforcement Learning Evgenii Nikishin Max Schwarzer P. DÓro Pierre-Luc Bacon Rameswar Panda OnRL 150 196 0 16 May 2022
Understanding and Preventing Capacity Loss in Reinforcement Learning Clare Lyle Mark Rowland Will Dabney CLL 105 115 0 20 Apr 2022
Value Gradient weighted Model-Based Reinforcement Learning C. Voelcker Victor Liao Animesh Garg Amir-massoud Farahmand 72 33 0 04 Apr 2022
Simplicial Embeddings in Self-Supervised Learning and Downstream Classification Samuel Lavoie Christos Tsirigotis Max Schwarzer Ankit Vani Michael Noukhovitch Kenji Kawaguchi Rameswar Panda SSL 54 18 0 01 Apr 2022
Temporal Difference Learning for Model Predictive Control Nicklas Hansen Xiaolong Wang H. Su PINN MU 99 255 0 09 Mar 2022
The Difficulty of Passive Learning in Deep Reinforcement Learning Georg Ostrovski Pablo Samuel Castro Will Dabney OffRL 74 59 0 26 Oct 2021
Dropout Q-Functions for Doubly Efficient Reinforcement Learning Takuya Hiraoka Takahisa Imagawa Taisei Hashimoto Takashi Onishi Yoshimasa Tsuruoka 90 113 0 05 Oct 2021
Deep Reinforcement Learning at the Edge of the Statistical Precipice Rishabh Agarwal Max Schwarzer Pablo Samuel Castro Aaron Courville Marc G. Bellemare OffRL 179 680 0 30 Aug 2021
Proper Value Equivalence Christopher Grimm André Barreto Gregory Farquhar David Silver Satinder Singh OffRL 75 35 0 18 Jun 2021
Tactical Optimism and Pessimism for Deep Reinforcement Learning Theodore H. Moskovitz Jack Parker-Holder Aldo Pacchiano Michael Arbel Michael I. Jordan 96 59 0 07 Feb 2021
Is Pessimism Provably Efficient for Offline RL? Ying Jin Zhuoran Yang Zhaoran Wang OffRL 193 360 0 30 Dec 2020
The Value Equivalence Principle for Model-Based Reinforcement Learning Christopher Grimm André Barreto Satinder Singh David Silver OffRL 65 86 0 06 Nov 2020
Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning Aviral Kumar Rishabh Agarwal Dibya Ghosh Sergey Levine OffRL 84 123 0 27 Oct 2020
Mastering Atari with Discrete World Models Danijar Hafner Timothy Lillicrap Mohammad Norouzi Jimmy Ba DRL 181 875 0 05 Oct 2020
Private Reinforcement Learning with PAC and Regret Guarantees G. Vietri Borja Balle A. Krishnamurthy Zhiwei Steven Wu 64 63 0 18 Sep 2020
On the model-based stochastic value gradient for continuous reinforcement learning Brandon Amos Samuel Stanton Denis Yarats A. Wilson 83 71 0 28 Aug 2020