Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism

22 March 2021

Papers citing "Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism"

50 / 73 papers shown

Title
A Unified Theoretical Analysis of Private and Robust Offline Alignment: from RLHF to DPO Xingyu Zhou Yulian Wu Francesco Orabona OffRL 7 0 0 21 May 2025
Addressing Concept Mislabeling in Concept Bottleneck Models Through Preference Optimization Emiliano Penaloza Tianyue H. Zhan Laurent Charlin Mateo Espinosa Zarlenga 51 0 0 25 Apr 2025
Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF Shicong Cen Jincheng Mei Katayoon Goshvadi Hanjun Dai Tong Yang Sherry Yang Dale Schuurmans Yuejie Chi Bo Dai OffRL 67 24 0 20 Feb 2025
Nearly Optimal Sample Complexity of Offline KL-Regularized Contextual Bandits under Single-Policy Concentrability Qingyue Zhao Kaixuan Ji Heyang Zhao Tong Zhang Q. Gu OffRL 47 0 0 09 Feb 2025
On The Statistical Complexity of Offline Decision-Making Thanh Nguyen-Tang R. Arora OffRL 48 1 0 10 Jan 2025
The Central Role of the Loss Function in Reinforcement Learning Kaiwen Wang Nathan Kallus Wen Sun OffRL 62 7 0 19 Sep 2024
Leveraging Unlabeled Data Sharing through Kernel Function Approximation in Offline Reinforcement Learning Yen-Ru Lai Fu-Chieh Chang Pei-Yuan Wu OffRL 81 1 0 22 Aug 2024
Hokoff: Real Game Dataset from Honor of Kings and its Offline Reinforcement Learning Benchmarks Yun Qu Boyuan Wang Jianzhun Shao Yuhang Jiang Chen Chen ... Qiang Fu Wei Yang Guang Yang Lanxiao Huang Xiangyang Ji OffRL 54 9 0 20 Aug 2024
Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer Zhihan Liu Miao Lu Shenao Zhang Boyi Liu Hongyi Guo Yingxiang Yang Jose H. Blanchet Zhaoran Wang 50 43 0 26 May 2024
How to Leverage Diverse Demonstrations in Offline Imitation Learning Sheng Yue Jiani Liu Xingyuan Hua Ju Ren Sen Lin Junshan Zhang Yaoxue Zhang OffRL 34 3 0 24 May 2024
Offline RL via Feature-Occupancy Gradient Ascent Gergely Neu Nneka Okolo OffRL 34 0 0 22 May 2024
A Unified Linear Programming Framework for Offline Reward Learning from Human Demonstrations and Feedback Kihyun Kim Jiawei Zhang Asuman Ozdaglar P. Parrilo OffRL 41 1 0 20 May 2024
Towards Robust Policy: Enhancing Offline Reinforcement Learning with Adversarial Attacks and Defenses Thanh Nguyen Tung M. Luu Tri Ton Chang D. Yoo OffRL AAML 36 0 0 18 May 2024
Enhancing Q-Learning with Large Language Model Heuristics Xiefeng Wu LRM 32 0 0 06 May 2024
DPO Meets PPO: Reinforced Token Optimization for RLHF Han Zhong Zikang Shan Guhao Feng Li Zhao Xinle Cheng Jiang Bian Di He Jiang Bian Liwei Wang 60 57 0 29 Apr 2024
Federated Offline Reinforcement Learning: Collaborative Single-Policy Coverage Suffices Jiin Woo Laixi Shi Gauri Joshi Yuejie Chi OffRL 34 3 0 08 Feb 2024
Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF Banghua Zhu Michael I. Jordan Jiantao Jiao 31 25 0 29 Jan 2024
MoMA: Model-based Mirror Ascent for Offline Reinforcement Learning Mao Hong Zhiyue Zhang Yue Wu Yan Xu OffRL 50 0 0 21 Jan 2024
On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling, and Beyond Thanh Nguyen-Tang Raman Arora OffRL 35 3 0 06 Jan 2024
Neural Network Approximation for Pessimistic Offline Reinforcement Learning Di Wu Yuling Jiao Li Shen Haizhao Yang Xiliang Lu OffRL 29 1 0 19 Dec 2023
RLIF: Interactive Imitation Learning as Reinforcement Learning Jianlan Luo Perry Dong Yuexiang Zhai Yi Ma Sergey Levine OffRL 33 14 0 21 Nov 2023
Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining Licong Lin Yu Bai Song Mei OffRL 37 45 0 12 Oct 2023
When is Agnostic Reinforcement Learning Statistically Tractable? Zeyu Jia Gene Li Alexander Rakhlin Ayush Sekhari Nathan Srebro OffRL 32 5 0 09 Oct 2023
Beyond Uniform Sampling: Offline Reinforcement Learning with Imbalanced Datasets Zhang-Wei Hong Aviral Kumar Sathwik Karnik Abhishek Bhandwaldar Akash Srivastava Joni Pajarinen Romain Laroche Abhishek Gupta Pulkit Agrawal OffRL 38 19 0 06 Oct 2023
Stackelberg Batch Policy Learning Wenzhuo Zhou Annie Qu OffRL 35 1 0 28 Sep 2023
Settling the Sample Complexity of Online Reinforcement Learning Zihan Zhang Yuxin Chen Jason D. Lee S. Du OffRL 98 22 0 25 Jul 2023
Policy Finetuning in Reinforcement Learning via Design of Experiments using Offline Data Ruiqi Zhang Andrea Zanette OffRL OnRL 42 7 0 10 Jul 2023
High-probability sample complexities for policy evaluation with linear function approximation Gen Li Weichen Wu Yuejie Chi Cong Ma Alessandro Rinaldo Yuting Wei OffRL 30 7 0 30 May 2023
Reinforcement Learning with Human Feedback: Learning Dynamic Choices via Pessimism Zihao Li Zhuoran Yang Mengdi Wang OffRL 37 55 0 29 May 2023
Offline Primal-Dual Reinforcement Learning for Linear MDPs Germano Gabbianelli Gergely Neu Nneka Okolo Matteo Papini OffRL 29 7 0 22 May 2023
Double Pessimism is Provably Efficient for Distributionally Robust Offline Reinforcement Learning: Generic Algorithm and Robust Partial Coverage Jose H. Blanchet Miao Lu Tong Zhang Han Zhong OffRL 45 30 0 16 May 2023
Local Optimization Achieves Global Optimality in Multi-Agent Reinforcement Learning Yulai Zhao Zhuoran Yang Zhaoran Wang Jason D. Lee 45 3 0 08 May 2023
Learning Robot Manipulation from Cross-Morphology Demonstration G. Salhotra Isabella Liu Gaurav Sukhatme LM&Ro 25 9 0 07 Apr 2023
The Provable Benefits of Unsupervised Data Sharing for Offline Reinforcement Learning Haotian Hu Yiqin Yang Qianchuan Zhao Chongjie Zhang OffRL 11 5 0 27 Feb 2023
VIPeR: Provably Efficient Algorithm for Offline RL with Neural Function Approximation Thanh Nguyen-Tang R. Arora OffRL 46 5 0 24 Feb 2023
Provably Efficient Offline Goal-Conditioned Reinforcement Learning with General Function Approximation and Single-Policy Concentrability Hanlin Zhu Amy Zhang OffRL 27 2 0 07 Feb 2023
Offline Learning in Markov Games with General Function Approximation Yuheng Zhang Yunru Bai Nan Jiang OffRL 21 8 0 06 Feb 2023
Offline Minimax Soft-Q-learning Under Realizability and Partial Coverage Masatoshi Uehara Nathan Kallus Jason D. Lee Wen Sun OffRL 50 5 0 05 Feb 2023
Importance Weighted Actor-Critic for Optimal Conservative Offline Reinforcement Learning Hanlin Zhu Paria Rashidinejad Jiantao Jiao OffRL 42 15 0 30 Jan 2023
Principled Reinforcement Learning with Human Feedback from Pairwise or $K$ -wise Comparisons Banghua Zhu Jiantao Jiao Michael I. Jordan OffRL 42 183 0 26 Jan 2023
Risk-Averse MDPs under Reward Ambiguity Haolin Ruan Zhi Chen C. Ho 38 2 0 03 Jan 2023
Policy learning "without'' overlap: Pessimism and generalized empirical Bernstein's inequality Ying Jin Zhimei Ren Zhuoran Yang Zhaoran Wang OffRL 32 25 0 19 Dec 2022
A Review of Off-Policy Evaluation in Reinforcement Learning Masatoshi Uehara C. Shi Nathan Kallus OffRL 36 69 0 13 Dec 2022
Confidence-Conditioned Value Functions for Offline Reinforcement Learning Joey Hong Aviral Kumar Sergey Levine OffRL 39 20 0 08 Dec 2022
Transfer Learning for Contextual Multi-armed Bandits Changxiao Cai T. Tony Cai Hongzhe Li 39 16 0 22 Nov 2022
Offline Estimation of Controlled Markov Chains: Minimaxity and Sample Complexity Imon Banerjee Harsha Honnappa Vinayak A. Rao OffRL 11 0 0 14 Nov 2022
Distributionally Robust Offline Reinforcement Learning with Linear Function Approximation Xiaoteng Ma Zhipeng Liang Jose H. Blanchet MingWen Liu Li Xia Jiheng Zhang Qianchuan Zhao Zhengyuan Zhou OOD OffRL 41 22 0 14 Sep 2022
Strategic Decision-Making in the Presence of Information Asymmetry: Provably Efficient RL with Algorithmic Instruments Mengxin Yu Zhuoran Yang Jianqing Fan OffRL 21 8 0 23 Aug 2022
Distributionally Robust Model-Based Offline Reinforcement Learning with Near-Optimal Sample Complexity Laixi Shi Yuejie Chi OOD OffRL 31 61 0 11 Aug 2022
Learning Deformable Object Manipulation from Expert Demonstrations G. Salhotra Isabella Liu Marcus Dominguez-Kuhne Gaurav Sukhatme 34 27 0 20 Jul 2022