Exploratory and Inferential Analysis of Benchmark Experiments

Eugster, Manuel J. A.; Hothorn, Torsten und Leisch, Friedrich (30. Mai 2008): Exploratory and Inferential Analysis of Benchmark Experiments. Department of Statistics: Technical Reports, Nr. 30

Abstract

Benchmark experiments produce data in a very specific format. The observations are drawn from the performance distributions of the candidate algorithms on resampled data sets. In this paper we introduce a comprehensive toolbox of exploratory and inferential analysis methods for benchmark experiments based on one or more data sets. We present new visualization techniques, show how formal non-parametric and parametric test procedures can be used to evaluate the results, and, finally, how to sum up to a statistically correct overall order of the candidate algorithms.

• Dokumententyp: Paper
• Keywords: benchmark experiment, learning algorithm, visualisation, inference, ranking
• Fakultät: Mathematik, Informatik und Statistik > Statistik > Technische Reports
• Sprache: Englisch