An index-based deterministic convergent optimal algorithm for constrained multi-armed bandit problems

초록

For the constrained multi-armed bandit model, we show that by construction there exists an indexbased deterministic "convergent optimal'' algorithm. The optimality is achieved by the convergence of the probability of choosing an optimal feasible arm to one over infinite horizon. The algorithm is built upon Locatelli et al. (2016) "anytime parameter-free thresholding'' algorithm under the assumption that the optimal value is known. We provide a finite-time lower bound of 1 - O(vertical bar A vertical bar Te-T) to the convergent optimality, where T is the horizon size and A is the set of the arms in the bandit. We then study a relaxed-version of the algorithm in a general form that estimates the optimal value and discuss the convergent optimality of the algorithm after a sufficiently large T. (C) 2021 Elsevier Ltd. All rights reserved.

키워드