matbench is an ImageNet for materials science; a set of 13 (with more to come!) curated machine learning tasks for benchmarking and performance testing.

⚠️ Note on classification scores: ROC-AUC scores on the matbench_v0.1 classification leaderboards were computed on predictions thresholded at 0.5. They therefore equal balanced accuracy, not true ROC-AUC, and understate submissions that provided probabilities. This is fixed for future scoring; existing leaderboard scores have not been recomputed. See #137 and #181.

Tests	Release

• Website

  • Leaderboard
  • Per-task leaderboards
  • Documentation
  • Full benchmark data
  • Code reference

• Help/Support:

• Source

If you find matbench useful, please consider citing our paper:

Dunn, A., Wang, Q., Ganose, A., Dopp, D., Jain, A. Benchmarking Materials Property
Prediction Methods: The Matbench Test Set and Automatminer Reference Algorithm. npj 
Computational Materials 6, 138 (2020). https://doi.org/10.1038/s41524-020-00406-3

Matbench is pip installable! Install with pip install matbench or see the installation page for more details.

Python versions 3.8+ are supported.