A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processes

12 November 2021

Papers citing "A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processes"

46 / 46 papers shown

Title
Counterfactually Fair Reinforcement Learning via Sequential Data Preprocessing Jitao Wang C. Shi John D. Piette Joshua R. Loftus Donglin Zeng Zhenke Wu OffRL 94 0 0 10 Jan 2025
Off-Policy Confidence Interval Estimation with Confounded Markov Decision Process C. Shi Jin Zhu Ye Shen Shuang Luo Hong Zhu R. Song OffRL 81 33 0 22 Feb 2022
Proximal Reinforcement Learning: Efficient Off-Policy Evaluation in Partially Observed Markov Decision Processes Andrew Bennett Nathan Kallus OffRL 37 43 0 28 Oct 2021
Off-Policy Evaluation in Partially Observed Markov Decision Processes under Sequential Ignorability Yupeng Tang Seung-seob Lee OffRL 81 25 0 24 Oct 2021
A Spectral Approach to Off-Policy Evaluation for POMDPs Yash Nair Nan Jiang OffRL 36 18 0 22 Sep 2021
Proximal Causal Inference for Complex Longitudinal Studies Andrew Ying Wang Miao Xu Shi E. T. Tchetgen 54 39 0 15 Sep 2021
Deep Proxy Causal Learning and its Application to Confounded Bandit Policy Evaluation Liyuan Xu Heishiro Kanagawa Arthur Gretton CML 33 36 0 07 Jun 2021
Deeply-Debiased Off-Policy Interval Estimation C. Shi Runzhe Wan Victor Chernozhukov R. Song OffRL 38 38 0 10 May 2021
Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restriction Afsaneh Mastouri Yuchen Zhu Limor Gultchin Anna Korba Ricardo M. A. Silva Matt J. Kusner Arthur Gretton Krikamol Muandet CML 27 61 0 10 May 2021
Autoregressive Dynamics Models for Offline Policy Evaluation and Optimization Michael Ruogu Zhang T. Paine Ofir Nachum Cosmin Paduraru George Tucker Ziyun Wang Mohammad Norouzi OffRL 61 46 0 28 Apr 2021
Minimax Kernel Machine Learning for a Class of Doubly Robust Functionals with Application to Proximal Causal Inference AmirEmad Ghassami Andrew Ying I. Shpitser E. T. Tchetgen 45 43 0 07 Apr 2021
Causal Inference Under Unmeasured Confounding With Negative Controls: A Minimax Learning Approach Nathan Kallus Xiaojie Mao Masatoshi Uehara CML 55 67 0 25 Mar 2021
Instrumental Variable Value Iteration for Causal Offline Reinforcement Learning Luofeng Liao Zuyue Fu Zhuoran Yang Yixin Wang Mladen Kolar Zhaoran Wang OffRL 50 35 0 19 Feb 2021
Finite Sample Analysis of Minimax Offline Reinforcement Learning: Completeness, Fast Rates and First-Order Efficiency Masatoshi Uehara Masaaki Imaizumi Nan Jiang Nathan Kallus Wen Sun Tengyang Xie OffRL 17 53 0 05 Feb 2021
Provably Efficient Reinforcement Learning with Linear Function Approximation Under Adaptivity Constraints Chi Jin Zhuoran Yang Zhaoran Wang OffRL 170 167 0 06 Jan 2021
Kernel Methods for Unobserved Confounding: Negative Controls, Proxies, and Instruments Rahul Singh CML 49 40 0 18 Dec 2020
The Variational Method of Moments Andrew Bennett Nathan Kallus 37 30 0 17 Dec 2020
Semiparametric proximal causal inference Yifan Cui Hongming Pu Xu Shi Wang Miao E. T. Tchetgen Tchetgen 31 102 0 17 Nov 2020
CoinDICE: Off-Policy Confidence Interval Estimation Bo Dai Ofir Nachum Yinlam Chow Lihong Li Csaba Szepesvári Dale Schuurmans OffRL 39 87 0 22 Oct 2020
Off-policy Evaluation in Infinite-Horizon Reinforcement Learning with Latent Confounders Andrew Bennett Nathan Kallus Lihong Li Ali Mousavi OffRL 47 43 0 27 Jul 2020
Batch Policy Learning in Average Reward Markov Decision Processes Peng Liao Zhengling Qi Runzhe Wan P. Klasnja Susan Murphy OffRL 70 84 0 23 Jul 2020
Off-Policy Evaluation via the Regularized Lagrangian Mengjiao Yang Ofir Nachum Bo Dai Lihong Li Dale Schuurmans OffRL 20 115 0 07 Jul 2020
Sample-Efficient Reinforcement Learning of Undercomplete POMDPs Chi Jin Sham Kakade A. Krishnamurthy Qinghua Liu 68 65 0 22 Jun 2020
Provably Efficient Causal Reinforcement Learning with Confounded Observational Data Lingxiao Wang Zhuoran Yang Zhaoran Wang OffRL 47 45 0 22 Jun 2020
Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weighting Ilja Kuzborskij Claire Vernade András Gyorgy Csaba Szepesvári OffRL 26 47 0 18 Jun 2020
Minimax Estimation of Conditional Moment Models Nishanth Dikkala Greg Lewis Lester W. Mackey Vasilis Syrgkanis 113 99 0 12 Jun 2020
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems Sergey Levine Aviral Kumar George Tucker Justin Fu OffRL GP 471 1,994 0 04 May 2020
Off-policy Policy Evaluation For Sequential Decisions Under Unobserved Confounding Hongseok Namkoong Ramtin Keramati Steve Yadlowsky Emma Brunskill OffRL 96 64 0 12 Mar 2020
Confounding-Robust Policy Evaluation in Infinite-Horizon Reinforcement Learning Nathan Kallus Angela Zhou OffRL 53 59 0 11 Feb 2020
Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Framework C. Shi Xiaoyu Wang Shuang Luo Hongtu Zhu Jieping Ye R. Song CML OffRL 44 37 0 05 Feb 2020
Asymptotically Efficient Off-Policy Evaluation for Tabular Reinforcement Learning Ming Yin Yu Wang OffRL 83 82 0 29 Jan 2020
POPCORN: Partially Observed Prediction COnstrained ReiNforcement Learning Joseph D. Futoma M. C. Hughes Finale Doshi-Velez OffRL 46 49 0 13 Jan 2020
Minimax Weight and Q-Function Learning for Off-Policy Evaluation Masatoshi Uehara Jiawei Huang Nan Jiang OffRL 85 186 0 28 Oct 2019
Off-Policy Evaluation in Partially Observable Environments Guy Tennenholtz Shie Mannor Uri Shalit OffRL 41 86 0 09 Sep 2019
Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes Nathan Kallus Masatoshi Uehara OffRL 66 185 0 22 Aug 2019
DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Corrections Ofir Nachum Yinlam Chow Bo Dai Lihong Li OffRL 81 332 0 10 Jun 2019
Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Sampling Tengyang Xie Yifei Ma Yu Wang OffRL 76 181 0 08 Jun 2019
A Kernel Loss for Solving the Bellman Equation Yihao Feng Lihong Li Qiang Liu 45 70 0 25 May 2019
Information-Theoretic Considerations in Batch Reinforcement Learning Jinglin Chen Nan Jiang OOD OffRL 65 373 0 01 May 2019
Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation Qiang Liu Lihong Li Ziyang Tang Dengyong Zhou OffRL 82 354 0 29 Oct 2018
OpenAI Gym Greg Brockman Vicki Cheung Ludwig Pettersson Jonas Schneider John Schulman Jie Tang Wojciech Zaremba OffRL ODL 166 5,048 0 05 Jun 2016
Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learning Philip S. Thomas Emma Brunskill OffRL 198 573 0 04 Apr 2016
Doubly Robust Off-policy Value Evaluation for Reinforcement Learning Nan Jiang Lihong Li OffRL 125 621 0 11 Nov 2015
Tensor decompositions for learning latent variable models Anima Anandkumar Rong Ge Daniel J. Hsu Sham Kakade Matus Telgarsky 240 1,142 0 29 Oct 2012
Closing the Learning-Planning Loop with Predictive State Representations Byron Boots S. Siddiqi Geoffrey J. Gordon 184 264 0 12 Dec 2009
A Spectral Algorithm for Learning Hidden Markov Models Daniel J. Hsu Sham Kakade Tong Zhang 111 309 0 26 Nov 2008