14
1

Super-Exponential Regret for UCT, AlphaGo and Variants

Abstract

We improve the proofs of the lower bounds of Coquelin and Munos (2007) that demonstrate that UCT can have exp(exp(1))\exp(\dots\exp(1)\dots) regret (with Ω(D)\Omega(D) exp terms) on the DD-chain environment, and that a `polynomial' UCT variant has exp2(exp2(DO(logD)))\exp_2(\exp_2(D - O(\log D))) regret on the same environment -- the original proofs contain an oversight for rewards bounded in [0,1][0, 1], which we fix in the present draft. We also adapt the proofs to AlphaGo's MCTS and its descendants (e.g., AlphaZero, Leela Zero) to also show exp2(exp2(DO(logD)))\exp_2(\exp_2(D - O(\log D))) regret.

View on arXiv
Comments on this paper