What are the Statistical Limits of Offline RL with Linear Function Approximation?

22 October 2020

Papers citing "What are the Statistical Limits of Offline RL with Linear Function Approximation?"

50 / 56 papers shown

Title
DARLR: Dual-Agent Offline Reinforcement Learning for Recommender Systems with Dynamic Reward Yi Zhang Ruihong Qiu Xuwei Xu Jiajun Liu Sen Wang OffRL 36 0 0 12 May 2025
Mitigating Preference Hacking in Policy Optimization with Pessimism Dhawal Gupta Adam Fisch Christoph Dann Alekh Agarwal 76 0 0 10 Mar 2025
Towards User-level Private Reinforcement Learning with Human Feedback Jingyang Zhang Mingxi Lei Meng Ding Mengdi Li Zihang Xiang Difei Xu Jinhui Xu Di Wang 52 0 0 22 Feb 2025
GNN-DT: Graph Neural Network Enhanced Decision Transformer for Efficient Optimization in Dynamic Environments Stavros Orfanoudakis Nanda Kishor Panda Peter Palensky Pedro P. Vergara AI4CE 66 0 0 03 Feb 2025
Counterfactually Fair Reinforcement Learning via Sequential Data Preprocessing Jitao Wang C. Shi John D. Piette Joshua R. Loftus Donglin Zeng Zhenke Wu OffRL 64 0 0 10 Jan 2025
Leveraging Unlabeled Data Sharing through Kernel Function Approximation in Offline Reinforcement Learning Yen-Ru Lai Fu-Chieh Chang Pei-Yuan Wu OffRL 81 1 0 22 Aug 2024
ROLeR: Effective Reward Shaping in Offline Reinforcement Learning for Recommender Systems Yi Zhang Ruihong Qiu Jiajun Liu Sen Wang OffRL 23 0 0 18 Jul 2024
On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling, and Beyond Thanh Nguyen-Tang Raman Arora OffRL 38 3 0 06 Jan 2024
Neural Network Approximation for Pessimistic Offline Reinforcement Learning Di Wu Yuling Jiao Li Shen Haizhao Yang Xiliang Lu OffRL 34 1 0 19 Dec 2023
When is Agnostic Reinforcement Learning Statistically Tractable? Zeyu Jia Gene Li Alexander Rakhlin Ayush Sekhari Nathan Srebro OffRL 34 5 0 09 Oct 2023
The Optimal Approximation Factors in Misspecified Off-Policy Value Function Estimation Philip Amortila Nan Jiang Csaba Szepesvári OffRL 31 3 0 25 Jul 2023
Policy Finetuning in Reinforcement Learning via Design of Experiments using Offline Data Ruiqi Zhang Andrea Zanette OffRL OnRL 42 7 0 10 Jul 2023
Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning Tongzhou Wang Antonio Torralba Phillip Isola Amy Zhang OffRL 39 34 0 03 Apr 2023
VIPeR: Provably Efficient Algorithm for Offline RL with Neural Function Approximation Thanh Nguyen-Tang R. Arora OffRL 53 5 0 24 Feb 2023
Why Target Networks Stabilise Temporal Difference Methods Matt Fellows Matthew Smith Shimon Whiteson OOD AAML 21 7 0 24 Feb 2023
A Strong Baseline for Batch Imitation Learning Matthew Smith Lucas Maystre Zhenwen Dai K. Ciosek OffRL 25 4 0 06 Feb 2023
A Review of Off-Policy Evaluation in Reinforcement Learning Masatoshi Uehara C. Shi Nathan Kallus OffRL 43 69 0 13 Dec 2022
On Instance-Dependent Bounds for Offline Reinforcement Learning with Linear Function Approximation Thanh Nguyen-Tang Ming Yin Sunil R. Gupta Svetha Venkatesh R. Arora OffRL 58 16 0 23 Nov 2022
When is Realizability Sufficient for Off-Policy Reinforcement Learning? Andrea Zanette OffRL 24 14 0 10 Nov 2022
Reliable Conditioning of Behavioral Cloning for Offline Reinforcement Learning Tung Nguyen Qinqing Zheng Aditya Grover OffRL 39 6 0 11 Oct 2022
Distributionally Robust Offline Reinforcement Learning with Linear Function Approximation Xiaoteng Ma Zhipeng Liang Jose H. Blanchet MingWen Liu Li Xia Jiheng Zhang Qianchuan Zhao Zhengyuan Zhou OOD OffRL 41 22 0 14 Sep 2022
Statistical Estimation of Confounded Linear MDPs: An Instrumental Variable Approach Miao Lu Wenhao Yang Liangyu Zhang Zhihua Zhang OffRL 40 1 0 12 Sep 2022
On the Statistical Efficiency of Reward-Free Exploration in Non-Linear RL Jinglin Chen Aditya Modi A. Krishnamurthy Nan Jiang Alekh Agarwal 38 25 0 21 Jun 2022
Sample-Efficient Reinforcement Learning in the Presence of Exogenous Information Yonathan Efroni Dylan J. Foster Dipendra Kumar Misra A. Krishnamurthy John Langford OffRL 31 25 0 09 Jun 2022
Offline Reinforcement Learning with Differential Privacy Dan Qiao Yu Wang OffRL 44 23 0 02 Jun 2022
Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning Andrea Zanette Martin J. Wainwright OOD 45 5 0 01 Jun 2022
Byzantine-Robust Online and Offline Distributed Reinforcement Learning Yiding Chen Xuezhou Zhang Kaipeng Zhang Mengdi Wang Xiaojin Zhu OffRL 26 16 0 01 Jun 2022
When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning? Aviral Kumar Joey Hong Anika Singh Sergey Levine OffRL 50 77 0 12 Apr 2022
Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimism Ming Yin Yaqi Duan Mengdi Wang Yu Wang OffRL 36 66 0 11 Mar 2022
A Complete Characterization of Linear Estimators for Offline Policy Evaluation Juan C. Perdomo A. Krishnamurthy Peter L. Bartlett Sham Kakade OffRL 29 3 0 08 Mar 2022
Off-Policy Fitted Q-Evaluation with Differentiable Function Approximators: Z-Estimation and Inference Theory Ruiqi Zhang Xuezhou Zhang Chengzhuo Ni Mengdi Wang OffRL 37 16 0 10 Feb 2022
Why Should I Trust You, Bellman? The Bellman Error is a Poor Replacement for Value Error Scott Fujimoto David Meger Doina Precup Ofir Nachum S. Gu 30 32 0 28 Jan 2022
Accelerated and instance-optimal policy evaluation with linear function approximation Tianjiao Li Guanghui Lan A. Pananjady OffRL 44 13 0 24 Dec 2021
DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization Aviral Kumar Rishabh Agarwal Tengyu Ma Aaron Courville George Tucker Sergey Levine OffRL 31 65 0 09 Dec 2021
The Impact of Data Distribution on Q-learning with Function Approximation Pedro P. Santos Diogo S. Carvalho Alberto Sardinha Francisco S. Melo OffRL 19 2 0 23 Nov 2021
Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximation Dylan J. Foster A. Krishnamurthy D. Simchi-Levi Yunzong Xu OffRL 21 62 0 21 Nov 2021
Exploiting Action Impact Regularity and Exogenous State Variables for Offline Reinforcement Learning Vincent Liu James Wright Martha White OffRL 33 1 0 15 Nov 2021
The Difficulty of Passive Learning in Deep Reinforcement Learning Georg Ostrovski Pablo Samuel Castro Will Dabney OffRL 24 57 0 26 Oct 2021
Representation Learning for Online and Offline RL in Low-rank MDPs Masatoshi Uehara Xuezhou Zhang Wen Sun OffRL 67 127 0 09 Oct 2021
Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learning Andrea Zanette Martin J. Wainwright Emma Brunskill OffRL 34 115 0 19 Aug 2021
Provably Efficient Representation Selection in Low-rank Markov Decision Processes: From Online to Offline RL Weitong Zhang Jiafan He Dongruo Zhou Amy Zhang Quanquan Gu OffRL 24 11 0 22 Jun 2021
Offline RL Without Off-Policy Evaluation David Brandfonbrener William F. Whitney Rajesh Ranganath Joan Bruna OffRL 42 162 0 16 Jun 2021
Bellman-consistent Pessimism for Offline Reinforcement Learning Tengyang Xie Ching-An Cheng Nan Jiang Paul Mineiro Alekh Agarwal OffRL LRM 27 271 0 13 Jun 2021
Optimal Uniform OPE and Model-based Offline Reinforcement Learning in Time-Homogeneous, Reward-Free and Task-Agnostic Settings Ming Yin Yu Wang OffRL 32 19 0 13 May 2021
Risk Bounds and Rademacher Complexity in Batch Reinforcement Learning Yaqi Duan Chi Jin Zhiyuan Li OffRL 36 48 0 25 Mar 2021
Cautiously Optimistic Policy Optimization and Exploration with Linear Function Approximation Andrea Zanette Ching-An Cheng Alekh Agarwal 34 53 0 24 Mar 2021
An Exponential Lower Bound for Linearly-Realizable MDPs with Constant Suboptimality Gap Yuanhao Wang Ruosong Wang Sham Kakade OffRL 41 43 0 23 Mar 2021
Infinite-Horizon Offline Reinforcement Learning with Linear Function Approximation: Curse of Dimensionality and Algorithm Lin Chen B. Scherrer Peter L. Bartlett OffRL 83 16 0 17 Mar 2021
Sample Complexity of Offline Reinforcement Learning with Deep ReLU Networks Thanh Nguyen-Tang Sunil R. Gupta Hung The Tran Svetha Venkatesh OffRL 70 7 0 11 Mar 2021
Instabilities of Offline RL with Pre-Trained Neural Representation Ruosong Wang Yifan Wu Ruslan Salakhutdinov Sham Kakade OffRL 22 42 0 08 Mar 2021