IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies

20 April 2023

Philippe Hansen-Estruch

Papers citing "IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies"

50 / 116 papers shown

Title
Beyond Local Views: Global State Inference with Diffusion Models for Cooperative Multi-Agent Reinforcement Learning Zhiwei Xu Hangyu Mao Nianmin Zhang Xin Xin Pengjie Ren ... Bin Zhang Guoliang Fan Zhumin Chen Changwei Wang Jiangjin Yin DiffM 22 1 0 18 Aug 2024
Diffusion-DICE: In-Sample Diffusion Guidance for Offline Reinforcement Learning Liyuan Mao Haoran Xu Weinan Zhang Xianyuan Zhan Amy Zhang OffRL 41 5 0 29 Jul 2024
From Imitation to Refinement -- Residual RL for Precise Visual Assembly Lars Ankile Anthony Simeonov Idan Shenfeld M. Torné Pulkit Agrawal OffRL 36 7 0 23 Jul 2024
Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Review Masatoshi Uehara Yulai Zhao Tommaso Biancalani Sergey Levine 66 22 0 18 Jul 2024
Aligning Diffusion Behaviors with Q-functions for Efficient Continuous Control Huayu Chen Kaiwen Zheng Hang Su Jun Zhu 51 1 0 12 Jul 2024
Residual-MPPI: Online Policy Customization for Continuous Control Pengcheng Wang Chenran Li Catherine Weaver Kenta Kawamoto Masayoshi Tomizuka Chen Tang Wei Zhan OffRL 37 3 0 01 Jul 2024
Provable Statistical Rates for Consistency Diffusion Models Zehao Dou Minshuo Chen Mengdi Wang Zhuoran Yang DiffM 37 3 0 23 Jun 2024
Diffusion Spectral Representation for Reinforcement Learning Dmitry Shribak Chen-Xiao Gao Yitong Li Chenjun Xiao Bo Dai DiffM 29 3 0 23 Jun 2024
CleanDiffuser: An Easy-to-use Modularized Library for Diffusion Models in Decision Making Zibin Dong Yifu Yuan Jianye Hao Fei Ni Yi Ma Pengyi Li Yan Zheng DiffM 58 9 0 13 Jun 2024
Is Value Learning Really the Main Bottleneck in Offline RL? Seohong Park Kevin Frans Sergey Levine Aviral Kumar OffRL 53 8 0 13 Jun 2024
DiffPoGAN: Diffusion Policies with Generative Adversarial Networks for Offline Reinforcement Learning Xuemin Hu Shen Li Yingfen Xu Bo Tang Long Chen 44 0 0 13 Jun 2024
Residual Learning and Context Encoding for Adaptive Offline-to-Online Reinforcement Learning Mohammadreza Nakhaei Aidan Scannell Joni Pajarinen OffRL 49 1 0 12 Jun 2024
UDQL: Bridging The Gap between MSE Loss and The Optimal Value Function in Offline Reinforcement Learning Yu Zhang Rui Yu Zhipeng Yao Wenyuan Zhang Jun Wang Liming Zhang OffRL 53 0 0 05 Jun 2024
Learning Multimodal Behaviors from Scratch with Diffusion Policy Gradient Zechu Li Rickmer Krohn Tao Chen Anurag Ajay Pulkit Agrawal Georgia Chalvatzaki DiffM 50 9 0 02 Jun 2024
Amortizing intractable inference in diffusion models for vision, language, and control S. Venkatraman Moksh Jain Luca Scimeca Minsu Kim Marcin Sendera ... Alexandre Adam Jarrid Rector-Brooks Yoshua Bengio Glen Berseth Nikolay Malkin 68 25 0 31 May 2024
Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning Linjiajie Fang Ruoxue Liu Jing Zhang Wenjia Wang Bing-Yi Jing OffRL 56 2 0 31 May 2024
Instruction-Guided Visual Masking Jinliang Zheng Jianxiong Li Si Cheng Yinan Zheng Jiaming Li Jihao Liu Yu Liu Jingjing Liu Xianyuan Zhan 53 5 0 30 May 2024
Diffusion Policies creating a Trust Region for Offline Reinforcement Learning Tianyu Chen Zhendong Wang Mingyuan Zhou OffRL 32 5 0 30 May 2024
Long-Horizon Rollout via Dynamics Diffusion for Offline Reinforcement Learning Hanye Zhao Xiaoshen Han Zhengbang Zhu Minghuan Liu Yong Yu Weinan Zhang OffRL 45 0 0 29 May 2024
Preferred-Action-Optimized Diffusion Policies for Offline Reinforcement Learning Tianle Zhang Jiayi Guan Lin Zhao Yihang Li Dongjiang Li ... Lei Sun Yue Chen Xuelong Wei Lusong Li Xiaodong He 43 1 0 29 May 2024
AlignIQL: Policy Alignment in Implicit Q-Learning through Constrained Optimization Longxiang He Li Shen Junbo Tan Xueqian Wang 49 1 0 28 May 2024
Diffusion-based Reinforcement Learning via Q-weighted Variational Policy Optimization Shutong Ding Ke Hu Zhenhao Zhang Kan Ren Weinan Zhang Jingyi Yu Jingya Wang Ye-ling Shi 40 8 0 25 May 2024
AIGB: Generative Auto-bidding via Diffusion Modeling Jiayan Guo Yusen Huo Zhilin Zhang Tianyu Wang Chuan Yu Jian Xu Yan Zhang Bo Zheng DiffM 38 1 0 25 May 2024
DIDI: Diffusion-Guided Diversity for Offline Behavioral Generation Jinxin Liu Xinghong Guo Zifeng Zhuang Donglin Wang DiffM OffRL 50 2 0 23 May 2024
Pre-trained Text-to-Image Diffusion Models Are Versatile Representation Learners for Control Gunshi Gupta Karmesh Yadav Y. Gal Dhruv Batra Z. Kira Cong Lu Tim G. J. Rudner 39 7 0 09 May 2024
AFU: Actor-Free critic Updates in off-policy RL for continuous control Nicolas Perrin-Gilbert OffRL 32 0 0 24 Apr 2024
An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization Minshuo Chen Song Mei Jianqing Fan Mengdi Wang VLM MedIm DiffM 37 48 0 11 Apr 2024
Regularized Conditional Diffusion Model for Multi-Task Preference Alignment Xudong Yu Chenjia Bai Haoran He Changhong Wang Xuelong Li 40 6 0 07 Apr 2024
A Contact Model based on Denoising Diffusion to Learn Variable Impedance Control for Contact-rich Manipulation Masashi Okada Mayumi Komatsu Tadahiro Taniguchi DiffM 40 0 0 20 Mar 2024
Stabilizing Policy Gradients for Stochastic Differential Equations via Consistency with Perturbation Process Xiangxin Zhou Liang Wang Yichi Zhou DiffM 32 4 0 07 Mar 2024
DNAct: Diffusion Guided Multi-Task 3D Policy Learning Ge Yan Yueh-hua Wu Xiaolong Wang VGen 37 20 0 07 Mar 2024
3D Diffuser Actor: Policy Diffusion with 3D Scene Representations Tsung-Wei Ke N. Gkanatsios Katerina Fragkiadaki VGen 39 108 0 16 Feb 2024
Entropy-regularized Diffusion Policy with Q-Ensembles for Offline Reinforcement Learning Ruoqing Zhang Ziwei Luo Jens Sjölund Thomas B. Schon Per Mattsson 20 7 0 06 Feb 2024
Diffusion World Model: Future Modeling Beyond Step-by-Step Rollout for Offline Reinforcement Learning Zihan Ding Amy Zhang Yuandong Tian Qinqing Zheng OffRL 47 17 0 05 Feb 2024
Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Model Yinan Zheng Jianxiong Li Dongjie Yu Yujie Yang Shengbo Eben Li Xianyuan Zhan Jingjing Liu OffRL 36 24 0 19 Jan 2024
Diffusion Reward: Learning Rewards via Conditional Video Diffusion Tao Huang Guangqi Jiang Yanjie Ze Huazhe Xu VGen 39 22 0 21 Dec 2023
World Models via Policy-Guided Trajectory Diffusion Marc Rigter Jun Yamada Ingmar Posner 34 19 0 13 Dec 2023
The Generalization Gap in Offline Reinforcement Learning Ishita Mediratta Qingfei You Minqi Jiang Roberta Raileanu OffRL 86 10 0 10 Dec 2023
PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Play Lili Chen Shikhar Bahl Deepak Pathak 22 41 0 07 Dec 2023
Diffusion Models for Reinforcement Learning: A Survey Zhengbang Zhu Hanye Zhao Haoran He Yichao Zhong Shenyu Zhang Haoquan Guo Tingting Chen Weinan Zhang 43 60 0 02 Nov 2023
Towards Robust Offline Reinforcement Learning under Diverse Data Corruption Rui Yang Han Zhong Jiawei Xu Amy Zhang Chong Zhang Lei Han Tong Zhang OffRL OnRL 41 15 0 19 Oct 2023
Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models Kevin Black Mitsuhiko Nakamoto P. Atreya Homer Walke Chelsea Finn Aviral Kumar Sergey Levine DiffM LM&Ro 35 132 0 16 Oct 2023
NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration A. Sridhar Dhruv Shah Catherine Glossop Sergey Levine 31 114 0 11 Oct 2023
Score Regularized Policy Optimization through Diffusion Behavior Huayu Chen Cheng Lu Zhengyi Wang Hang Su Jun Zhu 31 20 0 11 Oct 2023
Boosting Continuous Control with Consistency Policy Yuhui Chen Haoran Li Dongbin Zhao OffRL 41 20 0 10 Oct 2023
DiffCPS: Diffusion Model based Constrained Policy Search for Offline Reinforcement Learning Longxiang He Li Shen Linrui Zhang Junbo Tan Xueqian Wang OffRL 28 8 0 09 Oct 2023
Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learning Daoce Wang Chi Jin OffRL DiffM 27 25 0 29 Sep 2023
H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps Haoyi Niu Tianying Ji Bingqi Liu Haocheng Zhao Xiangyu Zhu Jianying Zheng Pengfei Huang Guyue Zhou Jianming Hu Xianyuan Zhan OffRL OnRL AI4CE 27 6 0 22 Sep 2023
BridgeData V2: A Dataset for Robot Learning at Scale Homer Walke Kevin Black Abraham Lee Moo Jin Kim Maximilian Du ... Andre Wang He Vivek Myers Kuan Fang Chelsea Finn Sergey Levine 32 207 0 24 Aug 2023
Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior Adam Block Ali Jadbabaie Daniel Pfrommer Max Simchowitz Russ Tedrake DiffM 47 22 0 27 Jul 2023