Logo Logo
Switch Language to German

Bengs, Viktor ORCID logoORCID: https://orcid.org/0000-0001-6988-6186 and Hüllermeier, Eyke ORCID logoORCID: https://orcid.org/0000-0002-9944-4108 (16. November 2022): Multi-armed bandits with censored consumption of resources. In: Machine Learning, Vol. 112, No. 2: pp. 217-240

Full text not available from 'Open Access LMU'.


We consider a resource-aware variant of the classical multi-armed bandit problem: In each round, the learner selects an arm and determines a resource limit. It then observes a corresponding (random) reward, provided the (random) amount of consumed resources remains below the limit. Otherwise, the observation is censored, i.e., no reward is obtained. For this problem setting, we introduce a measure of regret, which incorporates both the actual amount of consumed resources of each learning round and the optimality of realizable rewards as well as the risk of exceeding the allocated resource limit. Thus, to minimize regret, the learner needs to set a resource limit and choose an arm in such a way that the chance to realize a high reward within the predefined resource limit is high, while the resource limit itself should be kept as low as possible. We propose a UCB-inspired online learning algorithm, which we analyze theoretically in terms of its regret upper bound. In a simulation study, we show that our learning algorithm outperforms straightforward extensions of standard multi-armed bandit algorithms.

Actions (login required)

View Item View Item