Position: Why We Must Rethink Empirical Research in Machine Learning

MCML Authors

Moritz Herrmann

Dr.

Transfer Coordinator

→ Group Anne-Laure Boulesteix
 Biometry in Molecular Medicine

Giuseppe Casalicchio

Dr.

→ Group Bernd Bischl
 Statistical Learning and Data Science

Marcel Wever

Dr.

* Former Member

→ Group Eyke Hüllermeier
 Artificial Intelligence and Machine Learning

Matthias Feurer

Prof. Dr.

Thomas Bayes Fellow

* Former Thomas Bayes Fellow

David Rügamer

Prof. Dr.

Core PI

Statistics, Data Science and Machine Learning

Eyke Hüllermeier

Prof. Dr.

Core PI

Artificial Intelligence and Machine Learning

Anne-Laure Boulesteix

Prof. Dr.

Core PI

Biometry in Molecular Medicine

Bernd Bischl

Prof. Dr.

Director

Statistical Learning and Data Science

Abstract

We warn against a common but incomplete understanding of empirical research in machine learning (ML) that leads to non-replicable results, makes findings unreliable, and threatens to undermine progress in the field. To overcome this alarming situation, we call for more awareness of the plurality of ways of gaining knowledge experimentally but also of some epistemic limitations. In particular, we argue most current empirical ML research is fashioned as confirmatory research while it should rather be considered exploratory.

inproceedings HLE+24

---

ICML 2024

41st International Conference on Machine Learning. Vienna, Austria, Jul 21-27, 2024.

---

Authors

M. Herrmann • F. J. D. Lange • K. Eggensperger • G. Casalicchio • M. Wever • M. Feurer • D. Rügamer • E. Hüllermeier • A.-L. Boulesteix • B. Bischl

---

Links

URL

---

Research Areas

 A1 | Statistical Foundations & Explainability

 A3 | Computational Models