Identification of the Generalized Condorcet Winner in Multi-dueling Bandits

Haddenhorst, Björn ; Bengs, Viktor und Hüllermeier, Eyke (7. Dezember 2021): Identification of the Generalized Condorcet Winner in Multi-dueling Bandits. Advances in Neural Information Processing Systems, Virtual, December 7 2021. Ranzato, M.; Beygelzimer, A.; Dauphin, Y.; Liang, P.S. und Vaughan, J. Wortman (Hrsg.): Bd. 34 Curran Associates, Inc.. S. 25904-25916

Abstract

The reliable identification of the “best” arm while keeping the sample complexity as low as possible is a common task in the field of multi-armed bandits. In the multi-dueling variant of multi-armed bandits, where feedback is provided in the form of a winning arm among as set of k chosen ones, a reasonable notion of best arm is the generalized Condorcet winner (GCW). The latter is an the arm that has the greatest probability of being the winner in each subset containing it. In this paper, we derive lower bounds on the sample complexity for the task of identifying the GCW under various assumptions. As a by-product, our lower bound results provide new insights for the special case of dueling bandits (k = 2). We propose the Dvoretzky–Kiefer–Wolfowitz tournament (DKWT) algorithm, which we prove to be nearly optimal. In a numerical study, we show that DKWT empirically outperforms current state-of-the-art algorithms, even in the special case of dueling bandits or under a Plackett-Luce assumption on the feedback mechanism.

• Dokumententyp: Konferenzbeitrag (Paper)
• Publikationsform: Publisher's Version
• Fakultät: Mathematik, Informatik und Statistik > Informatik > Künstliche Intelligenz und Maschinelles Lernen
• Themengebiete: 000 Informatik, Informationswissenschaft, allgemeine Werke > 000 Informatik, Wissen, Systeme