load_classification_bake_off_2023_results

load_classification_bake_off_2023_results(num_resamples=30, as_array=False)

Pull down all the results of the 2023 univariate bake off.

Basic utility function to recover legacy results from [1]. Loads results for 112 UCR/tsml data sets for classifiers used in the publication. Can load either the default train/test split, or the resampled results up to 30 resamples.

Parameters:

num_resamplesint or None, default=1

The number of data resamples to return scores for. The first resample is the default train/test split for the dataset. For 1, only the score for the default train/test split of the dataset is returned. For 2 or more, a np.ndarray of scores for all resamples up to num_resamples are returned. If None, the scores of all resamples are returned.

If as_array is true, the scores are averaged instead of being returned as a np.ndarray.

as_arraybool, default=False

If True, return the results as a tuple containing a np.ndarray of (averaged) scores for each classifier. Also returns a list of dataset names for each row of the np.ndarray, and classifier names for each column.

Returns:

results: dict or tuple

Dictionary with estimator name keys containing another dictionary. Sub-dictionary consists of dataset name keys and contains of scores for each dataset. If as_array is true, instead returns a tuple of: An array of scores. Each column is a results for a classifier, each row a dataset. A list of dataset names for each row. A list of classifier names for each column.

References

[1]

M Middlehurst, P Schaefer, A Bagnall, “Bake off redux: a review and experimental evaluation of recent time series classification algorithms”, arXiv preprint arXiv:2304.13029, 2023.

Examples

>>> from aeon.benchmarking.published_results import (
...     load_classification_bake_off_2023_results
... )
>>> from aeon.visualisation import plot_critical_difference
>>> # Load the results
>>> results, data, cls = load_classification_bake_off_2023_results(
...     num_resamples=30, as_array=True
... )  
>>> # Select a subset of classifiers
>>> cls = ["HC2","MR-Hydra","InceptionT","FreshPRINCE","RDST"] 
>>> index = [cls.index(i) for i in cls] 
>>> selected = results[:,index]  
>>> # Plot the critical difference diagram
>>> plot = plot_critical_difference(selected, cls)  
>>> plot.show()