Bench Plot and Mixed Effects Models: First steps toward a comprehensiv benchmark analysis toolbox

Eugster, Manuel J. A. and Leisch, Friedrich (8. April 2008): Bench Plot and Mixed Effects Models: First steps toward a comprehensiv benchmark analysis toolbox. Department of Statistics: Technical Reports, No.26 Compstat 2008-Proceedings in Computational Statistics

Abstract

Benchmark experiments produce data in a very specific format. The observations are drawn from the performance distributions of the candidate algorithms on resampled data sets. In this paper we introduce new visualisation techniques and show how formal test procedures can be used to evaluate the results. This is the first step towards a comprehensive toolbox of exploratory and inferential analysis methods for benchmark experiments.

• Item Type: Paper
• Keywords: benchmark experiments, visualisation, hypothesis tests
• Faculties: Mathematics, Computer Science and Statistics > Statistics > Technical Reports
• Language: English