Position: Why We Must Rethink Empirical Research in Machine Learning

authored by: Moritz Herrmann, F. Julian D. Lange, Katharina Eggensperger, Giuseppe Casalicchio, Marcel Wever, Matthias Feurer, David Rügamer, Eyke Hüllermeier, Anne-Laure Boulesteix, Bernd Bischl
Abstract: We warn against a common but incomplete understanding of empirical research in machine learning that leads to non-replicable results, makes findings unreliable, and threatens to undermine progress in the field. To overcome this alarming situation, we call for more awareness of the plurality of ways of gaining knowledge experimentally but also of some epistemic limitations. In particular, we argue most current empirical machine learning research is fashioned as confirmatory research while it should rather be considered exploratory.
Organisation(s): Machine Learning Section
Type: Contribution to book/anthology
Publication date: 07.2024
Publication status: E-pub ahead of print
Peer reviewed: Yes
Electronic version(s): https://doi.org/10.48550/arXiv.2405.02200 (Access: Unknown)

BibTeX

@inbook{ebdbce8eaa0d407e938b0688d929b989,
title = "Position: Why We Must Rethink Empirical Research in Machine Learning",
abstract = "We warn against a common but incomplete understanding of empirical research in machine learning that leads to non-replicable results, makes findings unreliable, and threatens to undermine progress in the field. To overcome this alarming situation, we call for more awareness of the plurality of ways of gaining knowledge experimentally but also of some epistemic limitations. In particular, we argue most current empirical machine learning research is fashioned as confirmatory research while it should rather be considered exploratory.",
author = "Moritz Herrmann and Lange, {F. Julian D.} and Katharina Eggensperger and Giuseppe Casalicchio and Marcel Wever and Matthias Feurer and David R{\"u}gamer and Eyke H{\"u}llermeier and Anne-Laure Boulesteix and Bernd Bischl",
note = "DBLP License: DBLP's bibliographic metadata records provided through http://dblp.org/ are distributed under a Creative Commons CC0 1.0 Universal Public Domain Dedication. Although the bibliographic metadata records are provided consistent with CC0 1.0 Dedication, the content described by the metadata records is not. Content may be subject to copyright, rights of privacy, rights of publicity and other restrictions.",
year = "2024",
month = jul,
doi = "10.48550/arXiv.2405.02200",
language = "English",
booktitle = "Proceedings of the international conference on machine learning",
}