Abstract
Multi-armed bandit problem is an important optimization game that requires an exploration-exploitation tradeoff to achieve optimal total reward. Motivated from industrial applications such as online advertising and clinical research, we consider a setting where the rewards of bandit machines are associated with covariates, and the accurate estimation of the corresponding mean reward functions plays an important role in the performance of allocation rules. Under a flexible problem setup, we establish asymptotic strong consistency and perform a finite-time regret analysis for a sequential randomized allocation strategy based on kernel estimation. In addition, since many nonparametric and parametric methods in supervised learning may be applied to estimating the mean reward functions but guidance on how to choose among them is generally unavailable, we propose a model combining allocation strategy for adaptive performance. Simulations and a real data evaluation are conducted to illustrate the performance of the proposed allocation strategy.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Publication Date
2016
Document Type
Article
Department, Program, or Center
School of Mathematical Sciences (COS)
Recommended Citation
W. Qian and Y. Yang. Kernel Estimation and Model Combination in a Bandit Problem with Covariates. Journal of Machine Learning Research, 17(149):1-37, 2016.
Campus
RIT – Main Campus
Comments
Originally published in the Journal of Machine Learning Research: http://jmlr.org/papers/v17/13-210.html