Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximation

21 November 2021

Papers citing "Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximation"

49 / 49 papers shown

Title
On The Statistical Complexity of Offline Decision-Making Thanh Nguyen-Tang R. Arora OffRL 43 1 0 10 Jan 2025
Primal-Dual Spectral Representation for Off-policy Evaluation Yang Hu Tianyi Chen Na Li Kai Wang Bo Dai OffRL 32 0 0 23 Oct 2024
The Central Role of the Loss Function in Reinforcement Learning Kaiwen Wang Nathan Kallus Wen Sun OffRL 59 7 0 19 Sep 2024
The Role of Inherent Bellman Error in Offline Reinforcement Learning with Linear Function Approximation Noah Golowich Ankur Moitra OffRL 34 2 0 17 Jun 2024
Trajectory Data Suffices for Statistically Efficient Learning in Offline RL with Linear $q^π$ -Realizability and Concentrability Volodymyr Tkachuk Gellert Weisz Csaba Szepesvári OffRL 30 0 0 27 May 2024
ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL Yifei Zhou Andrea Zanette Jiayi Pan Sergey Levine Aviral Kumar 65 50 0 29 Feb 2024
Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimization Philip Ndikum Serge Ndikum 52 1 0 27 Feb 2024
Mitigating Covariate Shift in Misspecified Regression with Applications to Reinforcement Learning P. Amortila Tongyi Cao Akshay Krishnamurthy OffRL OOD 46 2 0 22 Jan 2024
Offline Data Enhanced On-Policy Policy Gradient with Provable Guarantees Yifei Zhou Ayush Sekhari Yuda Song Wen Sun OffRL OnRL 30 8 0 14 Nov 2023
On the Theory of Risk-Aware Agents: Bridging Actor-Critic and Economics Michal Nauman Marek Cygan 35 1 0 30 Oct 2023
Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregression Adam Block Dylan J. Foster Akshay Krishnamurthy Max Simchowitz Cyril Zhang 30 4 0 17 Oct 2023
When is Agnostic Reinforcement Learning Statistically Tractable? Zeyu Jia Gene Li Alexander Rakhlin Ayush Sekhari Nathan Srebro OffRL 27 5 0 09 Oct 2023
The Optimal Approximation Factors in Misspecified Off-Policy Value Function Estimation P. Amortila Nan Jiang Csaba Szepesvári OffRL 29 3 0 25 Jul 2023
Provable Benefits of Policy Learning from Human Preferences in Contextual Bandit Problems Xiang Ji Huazheng Wang Minshuo Chen Tuo Zhao Mengdi Wang OffRL 34 6 0 24 Jul 2023
Policy Finetuning in Reinforcement Learning via Design of Experiments using Offline Data Ruiqi Zhang Andrea Zanette OffRL OnRL 40 7 0 10 Jul 2023
Active Coverage for PAC Reinforcement Learning Aymen Al Marjani Andrea Tirinzoni E. Kaufmann OffRL 21 4 0 23 Jun 2023
Harnessing Mixed Offline Reinforcement Learning Datasets via Trajectory Weighting Zhang-Wei Hong Pulkit Agrawal Rémi Tachet des Combes Romain Laroche OffRL 31 17 0 22 Jun 2023
A Primal-Dual-Critic Algorithm for Offline Constrained Reinforcement Learning Kihyuk Hong Yuhang Li Ambuj Tewari OffRL 18 7 0 13 Jun 2023
Unified Off-Policy Learning to Rank: a Reinforcement Learning Perspective Zeyu Zhang Yi-Hsun Su Hui Yuan Yiran Wu R. Balasubramanian Qingyun Wu Huazheng Wang Mengdi Wang OffRL CML 36 4 0 13 Jun 2023
Survival Instinct in Offline Reinforcement Learning Anqi Li Dipendra Kumar Misra Andrey Kolobov Ching-An Cheng OffRL 22 16 0 05 Jun 2023
The Benefits of Being Distributional: Small-Loss Bounds for Reinforcement Learning Kaiwen Wang Kevin Zhou Runzhe Wu Nathan Kallus Wen Sun OffRL 31 17 0 25 May 2023
Offline Reinforcement Learning with Additional Covering Distributions Chenjie Mao OffRL 25 0 0 22 May 2023
Finetuning from Offline Reinforcement Learning: Challenges, Trade-offs and Practical Solutions Yicheng Luo Jackie Kay Edward Grefenstette M. Deisenroth OffRL OnRL 13 15 0 30 Mar 2023
VIPeR: Provably Efficient Algorithm for Offline RL with Neural Function Approximation Thanh Nguyen-Tang R. Arora OffRL 46 5 0 24 Feb 2023
Minimax Instrumental Variable Regression and $L_2$ Convergence Guarantees without Identification or Closedness Andrew Bennett Nathan Kallus Xiaojie Mao Whitney Newey Vasilis Syrgkanis Masatoshi Uehara 30 14 0 10 Feb 2023
Selective Uncertainty Propagation in Offline RL Sanath Kumar Krishnamurthy Shrey Modi Tanmay Gangwani S. Katariya B. Kveton A. Rangi OffRL 61 0 0 01 Feb 2023
Offline Robot Reinforcement Learning with Uncertainty-Guided Human Expert Sampling Ashish Kumar Ilya Kuzovkin OffRL OnRL 34 1 0 16 Dec 2022
A Review of Off-Policy Evaluation in Reinforcement Learning Masatoshi Uehara C. Shi Nathan Kallus OffRL 36 47 0 13 Dec 2022
When is Realizability Sufficient for Off-Policy Reinforcement Learning? Andrea Zanette OffRL 16 14 0 10 Nov 2022
Oracle Inequalities for Model Selection in Offline Reinforcement Learning Jonathan Lee George Tucker Ofir Nachum Bo Dai Emma Brunskill OffRL 24 13 0 03 Nov 2022
Behavior Prior Representation learning for Offline Reinforcement Learning Hongyu Zang Xin Li Jie Yu Chen Liu Riashat Islam Rémi Tachet des Combes Romain Laroche OffRL OnRL 35 10 0 02 Nov 2022
Optimal Conservative Offline RL with General Function Approximation via Augmented Lagrangian Paria Rashidinejad Hanlin Zhu Kunhe Yang Stuart J. Russell Jiantao Jiao OffRL 38 26 0 01 Nov 2022
Agent-Controller Representations: Principled Offline RL with Rich Exogenous Information Riashat Islam Manan Tomar Alex Lamb Yonathan Efroni Hongyu Zang ... Dipendra Kumar Misra Xin-hui Li H. V. Seijen Rémi Tachet des Combes John Langford OffRL 24 6 0 31 Oct 2022
Hybrid RL: Using Both Offline and Online Data Can Make RL Efficient Yuda Song Yi Zhou Ayush Sekhari J. Andrew Bagnell A. Krishnamurthy Wen Sun OffRL OnRL 32 92 0 13 Oct 2022
Reliable Conditioning of Behavioral Cloning for Offline Reinforcement Learning Tung Nguyen Qinqing Zheng Aditya Grover OffRL 23 6 0 11 Oct 2022
The Role of Coverage in Online Reinforcement Learning Tengyang Xie Dylan J. Foster Yu Bai Nan Jiang Sham Kakade OffRL 32 57 0 09 Oct 2022
Offline Reinforcement Learning with Differentiable Function Approximation is Provably Efficient Ming Yin Mengdi Wang Yu-Xiang Wang OffRL 77 12 0 03 Oct 2022
Learning Bellman Complete Representations for Offline Policy Evaluation Jonathan D. Chang Kaiwen Wang Nathan Kallus Wen Sun OffRL 29 14 0 12 Jul 2022
On the Statistical Efficiency of Reward-Free Exploration in Non-Linear RL Jinglin Chen Aditya Modi A. Krishnamurthy Nan Jiang Alekh Agarwal 38 25 0 21 Jun 2022
Provably Efficient Offline Reinforcement Learning with Trajectory-Wise Reward Tengyu Xu Yue Wang Shaofeng Zou Yingbin Liang OffRL 30 13 0 13 Jun 2022
Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning Andrea Zanette Martin J. Wainwright OOD 36 5 0 01 Jun 2022
$Pessimism for Offline Linear Contextual Bandits using $\ell_p$ Confidence Sets$ Pessimism for Offline Linear Contextual Bandits using $\ell_p$ Confidence Sets Gen Li Cong Ma Nathan Srebro OffRL 30 11 0 21 May 2022
Offline Reinforcement Learning Under Value and Density-Ratio Realizability: The Power of Gaps Jinglin Chen Nan Jiang OffRL 21 33 0 25 Mar 2022
Bellman Residual Orthogonalization for Offline Reinforcement Learning Andrea Zanette Martin J. Wainwright OffRL 27 8 0 24 Mar 2022
Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimism Ming Yin Yaqi Duan Mengdi Wang Yu-Xiang Wang OffRL 34 65 0 11 Mar 2022
A Complete Characterization of Linear Estimators for Offline Policy Evaluation Juan C. Perdomo A. Krishnamurthy Peter L. Bartlett Sham Kakade OffRL 27 3 0 08 Mar 2022
Offline Reinforcement Learning with Realizability and Single-policy Concentrability Wenhao Zhan Baihe Huang Audrey Huang Nan Jiang Jason D. Lee OffRL 34 104 0 09 Feb 2022
Provably Efficient Representation Selection in Low-rank Markov Decision Processes: From Online to Offline RL Weitong Zhang Jiafan He Dongruo Zhou Amy Zhang Quanquan Gu OffRL 22 11 0 22 Jun 2021
Goodness-of-fit Tests for high-dimensional Gaussian linear models Nicolas Verzélen Fanny Villers 103 34 0 14 Nov 2007