Bench Plot and Mixed Effects Models: First steps toward a comprehensive benchmark analysis toolbox

Eugster, Manuel J. A. and Leisch, Friedrich (08. April 2008): Bench Plot and Mixed Effects Models: First steps toward a comprehensive benchmark analysis toolbox. Department of Statistics: Technical Reports, No.26

Abstract

Benchmark experiments produce data in a very specific format. The observations are drawn from the performance distributions of the candidate algorithms on resampled data sets. In this paper we introduce new visualisation techniques and show how formal test procedures can be used to evaluate the results. This is the first step towards a comprehensive toolbox of exploratory and inferential analysis methods for benchmark experiments.

• Item Type: Paper (Technical Report)
• Published in: Compstat 2008-Proceedings in Computational Statistics
• Keywords: benchmark experiments, visualisation, hypothesis tests
• Subjects: Mathematics, Computer Science and Statistics > Statistics > Technical Reports
• Language: English