Minimax-Optimal Off-Policy Evaluation with Linear Function Approximation

21 February 2020

Papers citing "Minimax-Optimal Off-Policy Evaluation with Linear Function Approximation"

43 / 43 papers shown

Title
Leveraging Unlabeled Data Sharing through Kernel Function Approximation in Offline Reinforcement Learning Yen-Ru Lai Fu-Chieh Chang Pei-Yuan Wu OffRL 81 1 0 22 Aug 2024
From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems Jianliang He Siyu Chen Fengzhuo Zhang Zhuoran Yang LM&Ro LLMAG 44 2 0 30 May 2024
Imitation Learning in Discounted Linear MDPs without exploration assumptions Luca Viano Stratis Skoulakis V. Cevher 30 3 0 03 May 2024
Multiple-policy Evaluation via Density Estimation Yilei Chen Aldo Pacchiano I. Paschalidis OffRL 32 0 0 29 Mar 2024
On the Curses of Future and History in Future-dependent Value Functions for Off-policy Evaluation Yuheng Zhang Nan Jiang OffRL 29 4 0 22 Feb 2024
On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling, and Beyond Thanh Nguyen-Tang Raman Arora OffRL 33 3 0 06 Jan 2024
Conservative Exploration for Policy Optimization via Off-Policy Policy Evaluation Paul Daoudi Mathias Formoso Othman Gaizi Achraf Azize Evrard Garcelon OffRL 26 0 0 24 Dec 2023
Stackelberg Batch Policy Learning Wenzhuo Zhou Annie Qu OffRL 35 1 0 28 Sep 2023
The Optimal Approximation Factors in Misspecified Off-Policy Value Function Estimation P. Amortila Nan Jiang Csaba Szepesvári OffRL 29 3 0 25 Jul 2023
High-probability sample complexities for policy evaluation with linear function approximation Gen Li Weichen Wu Yuejie Chi Cong Ma Alessandro Rinaldo Yuting Wei OffRL 30 7 0 30 May 2023
Reinforcement Learning with Human Feedback: Learning Dynamic Choices via Pessimism Zihao Li Zhuoran Yang Mengdi Wang OffRL 37 55 0 29 May 2023
A Review of Off-Policy Evaluation in Reinforcement Learning Masatoshi Uehara C. Shi Nathan Kallus OffRL 36 69 0 13 Dec 2022
Counterfactual Learning with General Data-generating Policies Yusuke Narita Kyohei Okumura Akihiro Shimizu Kohei Yata CML OffRL 19 0 0 04 Dec 2022
Offline Policy Evaluation and Optimization under Confounding Chinmaya Kausik Yangyi Lu Kevin Tan Maggie Makar Yixin Wang Ambuj Tewari OffRL 26 8 0 29 Nov 2022
On Instance-Dependent Bounds for Offline Reinforcement Learning with Linear Function Approximation Thanh Nguyen-Tang Ming Yin Sunil R. Gupta Svetha Venkatesh R. Arora OffRL 58 16 0 23 Nov 2022
When is Realizability Sufficient for Off-Policy Reinforcement Learning? Andrea Zanette OffRL 24 14 0 10 Nov 2022
Beyond the Return: Off-policy Function Estimation under User-specified Error-measuring Distributions Audrey Huang Nan Jiang OffRL 55 9 0 27 Oct 2022
Statistical Estimation of Confounded Linear MDPs: An Instrumental Variable Approach Miao Lu Wenhao Yang Liangyu Zhang Zhihua Zhang OffRL 40 1 0 12 Sep 2022
Strategic Decision-Making in the Presence of Information Asymmetry: Provably Efficient RL with Algorithmic Instruments Mengxin Yu Zhuoran Yang Jianqing Fan OffRL 21 8 0 23 Aug 2022
Provably Efficient Reinforcement Learning in Partially Observable Dynamical Systems Masatoshi Uehara Ayush Sekhari Jason D. Lee Nathan Kallus Wen Sun OffRL 51 32 0 24 Jun 2022
Federated Offline Reinforcement Learning D. Zhou Yufeng Zhang Aaron Sonabend-W Zhaoran Wang Junwei Lu Tianxi Cai OffRL 40 13 0 11 Jun 2022
Offline Stochastic Shortest Path: Learning, Evaluation and Towards Optimality Ming Yin Wenjing Chen Mengdi Wang Yu-Xiang Wang OffRL 30 4 0 10 Jun 2022
Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimism Ming Yin Yaqi Duan Mengdi Wang Yu-Xiang Wang OffRL 34 66 0 11 Mar 2022
A Complete Characterization of Linear Estimators for Offline Policy Evaluation Juan C. Perdomo A. Krishnamurthy Peter L. Bartlett Sham Kakade OffRL 27 3 0 08 Mar 2022
Off-Policy Fitted Q-Evaluation with Differentiable Function Approximators: Z-Estimation and Inference Theory Ruiqi Zhang Xuezhou Zhang Chengzhuo Ni Mengdi Wang OffRL 35 16 0 10 Feb 2022
Hyperparameter Selection Methods for Fitted Q-Evaluation with Error Guarantee Kohei Miyaguchi OffRL 43 1 0 07 Jan 2022
DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization Aviral Kumar Rishabh Agarwal Tengyu Ma Aaron Courville George Tucker Sergey Levine OffRL 31 65 0 09 Dec 2021
Offline Neural Contextual Bandits: Pessimism, Optimization and Generalization Thanh Nguyen-Tang Sunil R. Gupta A. Nguyen Svetha Venkatesh OffRL 31 29 0 27 Nov 2021
The Impact of Data Distribution on Q-learning with Function Approximation Pedro P. Santos Diogo S. Carvalho A. Sardinha Francisco S. Melo OffRL 19 2 0 23 Nov 2021
Explaining Off-Policy Actor-Critic From A Bias-Variance Perspective Ting-Han Fan Peter J. Ramadge CML FAtt OffRL 21 2 0 06 Oct 2021
Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learning Andrea Zanette Martin J. Wainwright Emma Brunskill OffRL 29 115 0 19 Aug 2021
Variance-Aware Off-Policy Evaluation with Linear Function Approximation Yifei Min Tianhao Wang Dongruo Zhou Quanquan Gu OffRL 37 38 0 22 Jun 2021
Offline RL Without Off-Policy Evaluation David Brandfonbrener William F. Whitney Rajesh Ranganath Joan Bruna OffRL 42 162 0 16 Jun 2021
Bellman-consistent Pessimism for Offline Reinforcement Learning Tengyang Xie Ching-An Cheng Nan Jiang Paul Mineiro Alekh Agarwal OffRL LRM 27 270 0 13 Jun 2021
Optimal Uniform OPE and Model-based Offline Reinforcement Learning in Time-Homogeneous, Reward-Free and Task-Agnostic Settings Ming Yin Yu-Xiang Wang OffRL 32 19 0 13 May 2021
Nearly Horizon-Free Offline Reinforcement Learning Tongzheng Ren Jialian Li Bo Dai S. Du Sujay Sanghavi OffRL 32 49 0 25 Mar 2021
Infinite-Horizon Offline Reinforcement Learning with Linear Function Approximation: Curse of Dimensionality and Algorithm Lin Chen B. Scherrer Peter L. Bartlett OffRL 83 16 0 17 Mar 2021
Instabilities of Offline RL with Pre-Trained Neural Representation Ruosong Wang Yifan Wu Ruslan Salakhutdinov Sham Kakade OffRL 20 42 0 08 Mar 2021
Fast Rates for the Regret of Offline Reinforcement Learning Yichun Hu Nathan Kallus Masatoshi Uehara OffRL 15 30 0 31 Jan 2021
Is Pessimism Provably Efficient for Offline RL? Ying Jin Zhuoran Yang Zhaoran Wang OffRL 27 349 0 30 Dec 2020
Exponential Lower Bounds for Batch Reinforcement Learning: Batch RL can be Exponentially Harder than Online RL Andrea Zanette OffRL 26 71 0 14 Dec 2020
Near-Optimal Provable Uniform Convergence in Offline Policy Evaluation for Reinforcement Learning Ming Yin Yu Bai Yu-Xiang Wang OffRL 41 31 0 07 Jul 2020
Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Model Gen Li Yuting Wei Yuejie Chi Yuxin Chen 34 125 0 26 May 2020