IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies

20 April 2023

Philippe Hansen-Estruch

Papers citing "IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies"

40 / 40 papers shown

Title
Fine-tuning Diffusion Policies with Backpropagation Through Diffusion Timesteps Ningyuan Yang Jiaxuan Gao Feng Gao Yi Wu C. Yu 29 0 0 15 May 2025
Adaptive Diffusion Policy Optimization for Robotic Manipulation Huiyun Jiang Zhuang Yang 29 0 0 13 May 2025
CHD: Coupled Hierarchical Diffusion for Long-Horizon Tasks Ce Hao Anxing Xiao Zhiwei Xue Harold Soh 46 0 0 12 May 2025
What Matters for Batch Online Reinforcement Learning in Robotics? Perry Dong Suvir Mirchandani Dorsa Sadigh Chelsea Finn OffRL 28 0 0 12 May 2025
You Only Look One Step: Accelerating Backpropagation in Diffusion Sampling with Gradient Shortcuts Hongkun Dou Zeyu Li Xingyu Jiang H. Li Lijun Yang Wen Yao Yue Deng DiffM 38 0 0 12 May 2025
Efficient Robotic Policy Learning via Latent Space Backward Planning Dongxiu Liu Haoyi Niu Zhihao Wang Jinliang Zheng Yinan Zheng Zhonghong Ou Jianming Hu Jianxiong Li Xianyuan Zhan 28 0 0 11 May 2025
Wasserstein Convergence of Score-based Generative Models under Semiconvexity and Discontinuous Gradients Stefano Bruno Sotirios Sabanis DiffM 50 0 0 06 May 2025
Analytic Energy-Guided Policy Optimization for Offline Reinforcement Learning Jifeng Hu Sili Huang Z. Yang Shengchao Hu Li Shen H. Chen Lichao Sun Yi-Ju Chang Dacheng Tao OffRL 143 0 0 03 May 2025
Latent Diffusion Planning for Imitation Learning Amber Xie Oleh Rybkin Dorsa Sadigh Chelsea Finn 35 1 0 23 Apr 2025
An Optimal Discriminator Weighted Imitation Perspective for Reinforcement Learning Haoran Xu Shuozhe Li Harshit S. Sikchi S. Niekum Amy Zhang OffRL 25 0 0 17 Apr 2025
Offline Reinforcement Learning with Discrete Diffusion Skills Ruixi Qiao Jie Cheng Xingyuan Dai Yonglin Tian Yisheng Lv OffRL 84 0 0 26 Mar 2025
COLSON: Controllable Learning-Based Social Navigation via Diffusion-Based Reinforcement Learning Yuki Tomita Kohei Matsumoto Yuki Hyodo Ryo Kurazume 61 0 0 18 Mar 2025
Uncertainty Comes for Free: Human-in-the-Loop Policies with Diffusion Models Zhanpeng He Yifeng Cao M. Ciocarlie 57 0 0 26 Feb 2025
Hyperspherical Normalization for Scalable Deep Reinforcement Learning Hojoon Lee Youngdo Lee Takuma Seno Donghu Kim Peter Stone Jaegul Choo 63 1 0 24 Feb 2025
Score-Based Diffusion Policy Compatible with Reinforcement Learning via Optimal Transport Mingyang Sun Pengxiang Ding Weinan Zhang Donglin Wang OT 83 0 0 24 Feb 2025
Learning a Diffusion Model Policy from Rewards via Q-Score Matching Michael Psenka Alejandro Escontrela Pieter Abbeel Yi-An Ma DiffM 91 23 0 17 Feb 2025
Skill Expansion and Composition in Parameter Space Tenglong Liu J. Li Yinan Zheng Haoyi Niu Yixing Lan Xin Xu Xianyuan Zhan 58 4 0 09 Feb 2025
Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation Xuanlin Li Tong Zhao Xinghao Zhu Jiuguang Wang Tao Pang Kuan Fang 82 4 0 03 Dec 2024
Enhancing Exploration with Diffusion Policies in Hybrid Off-Policy RL: Application to Non-Prehensile Manipulation Huy Le Miroslav Gabriel Tai Hoang Gerhard Neumann Ngo Anh Vien 101 1 0 22 Nov 2024
Q-Distribution guided Q-learning for offline reinforcement learning: Uncertainty penalized Q-value via consistency model Jing Zhang Linjiajie Fang Kexin Shi Wenjia Wang Bing-Yi Jing OffRL 36 0 0 27 Oct 2024
Leveraging Skills from Unlabeled Prior Data for Efficient Online Exploration Max Wilcoxson Qiyang Li Kevin Frans Sergey Levine SSL OffRL OnRL 57 0 0 23 Oct 2024
On Diffusion Models for Multi-Agent Partial Observability: Shared Attractors, Error Bounds, and Composite Flow Tonghan Wang Heng Dong Yanchen Jiang David C. Parkes Milind Tambe DiffM 44 2 0 17 Oct 2024
Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance Mitsuhiko Nakamoto Oier Mees Aviral Kumar Sergey Levine OffRL 76 12 0 17 Oct 2024
DIAR: Diffusion-model-guided Implicit Q-learning with Adaptive Revaluation Jaehyun Park Yunho Kim Sejin Kim Byung-Jun Lee Sundong Kim OffRL 30 1 0 15 Oct 2024
Residual-MPPI: Online Policy Customization for Continuous Control Pengcheng Wang Chenran Li Catherine Weaver Kenta Kawamoto M. Tomizuka Chen Tang Wei Zhan OffRL 29 3 0 01 Jul 2024
Residual Learning and Context Encoding for Adaptive Offline-to-Online Reinforcement Learning Mohammadreza Nakhaei Aidan Scannell J. Pajarinen OffRL 49 1 0 12 Jun 2024
UDQL: Bridging The Gap between MSE Loss and The Optimal Value Function in Offline Reinforcement Learning Yu Zhang Rui Yu Zhipeng Yao Wenyuan Zhang Jun Wang Liming Zhang OffRL 50 0 0 05 Jun 2024
Amortizing intractable inference in diffusion models for vision, language, and control S. Venkatraman Moksh Jain Luca Scimeca Minsu Kim Marcin Sendera ... Alexandre Adam Jarrid Rector-Brooks Yoshua Bengio Glen Berseth Nikolay Malkin 68 24 0 31 May 2024
Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning Linjiajie Fang Ruoxue Liu Jing Zhang Wenjia Wang Bing-Yi Jing OffRL 53 1 0 31 May 2024
DNAct: Diffusion Guided Multi-Task 3D Policy Learning Ge Yan Yueh-hua Wu Xiaolong Wang VGen 37 20 0 07 Mar 2024
Boosting Continuous Control with Consistency Policy Yuhui Chen Haoran Li Dongbin Zhao OffRL 32 20 0 10 Oct 2023
H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps Haoyi Niu Tianying Ji Bingqi Liu Haocheng Zhao Xiangyu Zhu Jianying Zheng Pengfei Huang Guyue Zhou Jianming Hu Xianyuan Zhan OffRL OnRL AI4CE 25 6 0 22 Sep 2023
LLQL: Logistic Likelihood Q-Learning for Reinforcement Learning Outongyi Lv Bingxin Zhou OffRL 36 0 0 05 Jul 2023
Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning Mitsuhiko Nakamoto Yuexiang Zhai Anika Singh Max Sobol Mark Yi-An Ma Chelsea Finn Aviral Kumar Sergey Levine OffRL OnRL 109 108 0 09 Mar 2023
Offline Reinforcement Learning via High-Fidelity Generative Behavior Modeling Huayu Chen Cheng Lu Chengyang Ying Hang Su Jun Zhu DiffM OffRL 103 105 0 29 Sep 2022
Planning with Diffusion for Flexible Behavior Synthesis Michael Janner Yilun Du J. Tenenbaum Sergey Levine DiffM 202 629 0 20 May 2022
Offline Reinforcement Learning with Implicit Q-Learning Ilya Kostrikov Ashvin Nair Sergey Levine OffRL 214 839 0 12 Oct 2021
EMaQ: Expected-Max Q-Learning Operator for Simple Yet Effective Offline and Online RL Seyed Kamyar Seyed Ghasemipour Dale Schuurmans S. Gu OffRL 209 119 0 21 Jul 2020
Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Critics Arsenii Kuznetsov Pavel Shvechikov Alexander Grishin Dmitry Vetrov 136 185 0 08 May 2020
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems Sergey Levine Aviral Kumar George Tucker Justin Fu OffRL GP 337 1,951 0 04 May 2020