MetaQuRe: Meta-learning from Model Quality and Resource Consumption

Fischer, Raphael ; Wever, Marcel ; Buschjäger, Sebastian und Liebig, Thomas (September 2024): MetaQuRe: Meta-learning from Model Quality and Resource Consumption. European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2024), Vilnius, Lithuania, 9. - 13. September 2024. In: Machine Learning and Knowledge Discovery in Databases Part VII, Proc. ECML PKDD 2024, Bd. 14947, Nr. 14947 Springer Cham. S. 209-226

Abstract

Automated machine learning (AutoML) allows for selecting, parametrizing, and composing learning algorithms for a given data set. While resources play a pivotal role in neural architecture search, it is less pronounced by classical AutoML approaches. In fact, they generally focus on only maximizing predictive quality and disregard the importance of finding resource-efficient solutions. To push resource awareness further, our work explicitly explores how measures such as running time or energy consumption can be better considered in AutoML. Firstly, we propose a novel method for algorithm selection that balances multiple performance aspects (including resource demand) as prioritized by the user with the help of compositional meta-learning. Secondly, to foster research on green meta-learning and AutoML, we release the MetaQuRe data set, which contains information on predictive (Qu)ality and (Re)source consumption of models evaluated across hundreds of data sets and four execution environments. We use this data to put our methodology into practice and conduct an in-depth analysis of how our approach and data set can help in making AutoML more resource-aware, which represents our third contribution. Lastly, we publish MetaQuRe alongside an extensive code base, allowing for reproducing all results, expanding our data with results from custom environments, and exploring MetaQuRe interactively. In short, our work demonstrates both the importance as well as benefits of rethinking AutoML and meta-learning in a resource-aware way, thus paving the path for making future ML solutions more sustainable.

• Dokumententyp: Konferenzbeitrag (Paper)
• Keywords: Meta-learning, AutoML, Resource-aware ML, Green AI, Sustainable AI, Algorithm Selection
• Fakultät: Mathematik, Informatik und Statistik > Informatik > Künstliche Intelligenz und Maschinelles Lernen
• Themengebiete: 000 Informatik, Informationswissenschaft, allgemeine Werke > 000 Informatik, Wissen, Systeme
• ISBN: 978-3-031-70367-6
• ISSN: 0302-9743
• Sprache: Englisch