load_classification_bake_off_2021_results

load_classification_bake_off_2021_results(num_resamples=30, as_array=False)

Pull down all the results of the 2021 multivariate bake off.

Basic utility function to recover legacy results from [1]. Loads results for 26 tsml data sets for classifiers used in the publication. Can load either the default train/test split, or the resampled results up to 30 resamples.

Parameters:

num_resamplesint or None, default=1

The number of data resamples to return scores for. The first resample is the default train/test split for the dataset. For 1, only the score for the default train/test split of the dataset is returned. For 2 or more, a np.ndarray of scores for all resamples up to num_resamples are returned. If None, the scores of all resamples are returned.

If as_array is true, the scores are averaged instead of being returned as a np.ndarray.

as_arraybool, default=False

If True, return the results as a tuple containing a np.ndarray of (averaged) scores for each classifier. Also returns a list of dataset names for each row of the np.ndarray, and classifier names for each column.

Returns:

results: dict or tuple

Dictionary with estimator name keys containing another dictionary. Sub-dictionary consists of dataset name keys and contains of scores for each dataset. If as_array is true, instead returns a tuple of: An array of scores. Each column is a results for a classifier, each row a dataset. A list of dataset names for each row. A list of classifier names for each column.

References

[1]

AP Ruiz, M Flynn, J Large, M Middlehurst, A Bagnall, “The great multivariate time series classification bake off: a review and experimental evaluation of recent algorithmic advances”, Data mining and knowledge discovery 35, 401-449, 2021.

Examples

>>> from aeon.benchmarking.published_results import (
...     load_classification_bake_off_2021_results
... )
>>> from aeon.visualisation import plot_critical_difference
>>> # Load the results
>>> results, data, cls = load_classification_bake_off_2023_results(
...     num_resamples=30, as_array=True
... )  
>>> # Plot the critical difference diagram
>>> plot = plot_critical_difference(results, cls)  
>>> plot.show()