Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimism

11 March 2022

Ming Yin

Mengdi Wang

Papers citing "Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimism"

50 / 52 papers shown

Title
Towards Optimal Differentially Private Regret Bounds in Linear MDPs Sharan Sahu 105 0 0 12 Apr 2025
Leveraging Unlabeled Data Sharing through Kernel Function Approximation in Offline Reinforcement Learning Yen-Ru Lai Fu-Chieh Chang Pei-Yuan Wu OffRL 131 1 0 22 Aug 2024
Settling the Sample Complexity of Online Reinforcement Learning Zihan Zhang Yuxin Chen Jason D. Lee S. Du OffRL 194 25 0 25 Jul 2023
Pessimistic Q-Learning for Offline Reinforcement Learning: Towards Optimal Sample Complexity Laixi Shi Gen Li Yuting Wei Yuxin Chen Yuejie Chi OffRL 91 96 0 28 Feb 2022
First-Order Regret in Reinforcement Learning with Linear Function Approximation: A Robust Estimation Approach Andrew Wagenmaker Yifang Chen Max Simchowitz S. Du Kevin Jamieson 136 39 0 07 Dec 2021
Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximation Dylan J. Foster A. Krishnamurthy D. Simchi-Levi Yunzong Xu OffRL 149 63 0 21 Nov 2021
Towards Instance-Optimal Offline Reinforcement Learning with Pessimism Ming Yin Yu Wang OffRL 151 82 0 17 Oct 2021
Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learning Andrea Zanette Martin J. Wainwright Emma Brunskill OffRL 95 119 0 19 Aug 2021
Provably Efficient Generative Adversarial Imitation Learning for Online and Offline Setting with Linear Function Approximation Zhihan Liu Yufeng Zhang Zuyue Fu Zhuoran Yang Zhaoran Wang OffRL 57 6 0 19 Aug 2021
Variance-Aware Off-Policy Evaluation with Linear Function Approximation Yifei Min Tianhao Wang Dongruo Zhou Quanquan Gu OffRL 82 38 0 22 Jun 2021
Bellman-consistent Pessimism for Offline Reinforcement Learning Tengyang Xie Ching-An Cheng Nan Jiang Paul Mineiro Alekh Agarwal OffRL LRM 186 279 0 13 Jun 2021
Safe Reinforcement Learning with Linear Function Approximation Sanae Amani Christos Thrampoulidis Lin F. Yang 62 36 0 11 Jun 2021
Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learning Tengyang Xie Nan Jiang Huan Wang Caiming Xiong Yu Bai OffRL OnRL 99 164 0 09 Jun 2021
Decision Transformer: Reinforcement Learning via Sequence Modeling Lili Chen Kevin Lu Aravind Rajeswaran Kimin Lee Aditya Grover Michael Laskin Pieter Abbeel A. Srinivas Igor Mordatch OffRL 156 1,660 0 02 Jun 2021
Optimal Uniform OPE and Model-based Offline Reinforcement Learning in Time-Homogeneous, Reward-Free and Task-Agnostic Settings Ming Yin Yu Wang OffRL 97 19 0 13 May 2021
On the Optimality of Batch Policy Optimization Algorithms Chenjun Xiao Yifan Wu Tor Lattimore Bo Dai Jincheng Mei Lihong Li Csaba Szepesvári Dale Schuurmans OffRL 72 33 0 06 Apr 2021
Nearly Horizon-Free Offline Reinforcement Learning Zhaolin Ren Jialian Li Bo Dai S. Du Sujay Sanghavi OffRL 83 49 0 25 Mar 2021
Risk Bounds and Rademacher Complexity in Batch Reinforcement Learning Yaqi Duan Chi Jin Zhiyuan Li OffRL 93 48 0 25 Mar 2021
Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism Paria Rashidinejad Banghua Zhu Cong Ma Jiantao Jiao Stuart J. Russell OffRL 233 290 0 22 Mar 2021
Bilinear Classes: A Structural Framework for Provable Generalization in RL S. Du Sham Kakade Jason D. Lee Shachar Lovett G. Mahajan Wen Sun Ruosong Wang OffRL 184 191 0 19 Mar 2021
Bellman Eluder Dimension: New Rich Classes of RL Problems, and Sample-Efficient Algorithms Chi Jin Qinghua Liu Sobhan Miryoosefi OffRL 101 219 0 01 Feb 2021
A Provably Efficient Algorithm for Linear Markov Decision Process with Low Switching Cost Minbo Gao Tianle Xie S. Du Lin F. Yang 71 46 0 02 Jan 2021
Is Pessimism Provably Efficient for Offline RL? Ying Jin Zhuoran Yang Zhaoran Wang OffRL 187 360 0 30 Dec 2020
Nearly Minimax Optimal Reinforcement Learning for Linear Mixture Markov Decision Processes Dongruo Zhou Quanquan Gu Csaba Szepesvári 86 209 0 15 Dec 2020
Exponential Lower Bounds for Batch Reinforcement Learning: Batch RL can be Exponentially Harder than Online RL Andrea Zanette OffRL 177 71 0 14 Dec 2020
Logarithmic Regret for Reinforcement Learning with Linear Function Approximation Jiafan He Dongruo Zhou Quanquan Gu 53 95 0 23 Nov 2020
What are the Statistical Limits of Offline RL with Linear Function Approximation? Ruosong Wang Dean Phillips Foster Sham Kakade OffRL 169 163 0 22 Oct 2020
Provably Good Batch Reinforcement Learning Without Great Exploration Yao Liu Adith Swaminathan Alekh Agarwal Emma Brunskill OffRL 165 105 0 16 Jul 2020
Provably Efficient Reinforcement Learning for Discounted MDPs with Feature Mapping Dongruo Zhou Jiafan He Quanquan Gu 81 136 0 23 Jun 2020
On Reward-Free Reinforcement Learning with Linear Function Approximation Ruosong Wang S. Du Lin F. Yang Ruslan Salakhutdinov OffRL 75 107 0 19 Jun 2020
Conservative Q-Learning for Offline Reinforcement Learning Aviral Kumar Aurick Zhou George Tucker Sergey Levine OffRL OnRL 146 1,836 0 08 Jun 2020
Model-Based Reinforcement Learning with Value-Targeted Regression Alex Ayoub Zeyu Jia Csaba Szepesvári Mengdi Wang Lin F. Yang OffRL 96 305 0 01 Jun 2020
MOPO: Model-based Offline Policy Optimization Tianhe Yu G. Thomas Lantao Yu Stefano Ermon James Zou Sergey Levine Chelsea Finn Tengyu Ma OffRL 80 773 0 27 May 2020
MOReL : Model-Based Offline Reinforcement Learning Rahul Kidambi Aravind Rajeswaran Praneeth Netrapalli Thorsten Joachims OffRL 107 677 0 12 May 2020
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems Sergey Levine Aviral Kumar George Tucker Justin Fu OffRL GP 576 2,046 0 04 May 2020
Learning Near Optimal Policies with Low Inherent Bellman Error Andrea Zanette A. Lazaric Mykel Kochenderfer Emma Brunskill OffRL 86 222 0 29 Feb 2020
Minimax-Optimal Off-Policy Evaluation with Linear Function Approximation Yaqi Duan Mengdi Wang OffRL 150 152 0 21 Feb 2020
Asymptotically Efficient Off-Policy Evaluation for Tabular Reinforcement Learning Ming Yin Yu Wang OffRL 124 82 0 29 Jan 2020
Provably Efficient Exploration in Policy Optimization Qi Cai Zhuoran Yang Chi Jin Zhaoran Wang 85 283 0 12 Dec 2019
Optimism in Reinforcement Learning with Generalized Linear Function Approximation Yining Wang Ruosong Wang S. Du A. Krishnamurthy 186 137 0 09 Dec 2019
Behavior Regularized Offline Reinforcement Learning Yifan Wu George Tucker Ofir Nachum OffRL 97 690 0 26 Nov 2019
Sample Complexity of Reinforcement Learning using Linearly Combined Model Ensembles Aditya Modi Nan Jiang Ambuj Tewari Satinder Singh 70 132 0 23 Oct 2019
Provably Efficient Reinforcement Learning with Linear Function Approximation Chi Jin Zhuoran Yang Zhaoran Wang Michael I. Jordan 109 560 0 11 Jul 2019
Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction Aviral Kumar Justin Fu George Tucker Sergey Levine OffRL OnRL 140 1,067 0 03 Jun 2019
Reinforcement Learning in Feature Space: Matrix Bandit, Kernels, and Regret Bound Lin F. Yang Mengdi Wang OffRL GP 91 288 0 24 May 2019
Information-Theoretic Considerations in Batch Reinforcement Learning Jinglin Chen Nan Jiang OOD OffRL 165 378 0 01 May 2019
Batch Policy Learning under Constraints Hoang Minh Le Cameron Voloshin Yisong Yue OffRL 68 335 0 20 Mar 2019
Provably efficient RL with Rich Observations via Latent State Decoding S. Du A. Krishnamurthy Nan Jiang Alekh Agarwal Miroslav Dudík John Langford OffRL 74 230 0 25 Jan 2019
Tighter Problem-Dependent Regret Bounds in Reinforcement Learning without Domain Knowledge using Value Function Bounds Andrea Zanette Emma Brunskill OffRL 126 276 0 01 Jan 2019
Off-Policy Deep Reinforcement Learning without Exploration Scott Fujimoto David Meger Doina Precup OffRL BDL 253 1,625 0 07 Dec 2018