load_classification_bake_off_2017_results

load_classification_bake_off_2017_results(num_resamples=100, as_array=False, ignore_nan=False)

Fetch all the results of the 2017 univariate TSC bake off.

Basic utility function to recover legacy results from [1]. Loads results for 85 univariate UCR data sets for classifiers used in the publication. Can load either the default train/test split, or the resampled results up to 100 resamples.

Parameters:

num_resamplesint or None, default=1

The number of data resamples to return scores for. The first resample is the default train/test split for the dataset. For 1, only the score for the default train/test split of the dataset is returned. For 2 or more, a np.ndarray of scores for all resamples up to num_resamples are returned. If None, the scores of all resamples are returned.

If as_array is true, the scores are averaged instead of being returned as a np.ndarray.

as_arraybool, default=False

If True, return the results as a tuple containing a np.ndarray of (averaged) scores for each classifier. Also returns a list of dataset names for each row of the np.ndarray, and classifier names for each column.

ignore_nanbool, default=False

Ignore the error raised when NaN values are present in the results. Ignores NaN values when averaging when as_array is True.

Returns:

results: dict or tuple

Dictionary with estimator name keys containing another dictionary. Sub-dictionary consists of dataset name keys and contains of scores for each dataset. If as_array is true, instead returns a tuple of: An array of scores. Each column is a results for a classifier, each row a dataset. A list of dataset names for each row. A list of classifier names for each column.

References

[1]

A Bagnall, J Lines, A Bostrom, J Large, E Keogh, “The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances”, Data mining and knowledge discovery 31, 606-660, 2017.

Examples

>>> from aeon.benchmarking.published_results import (
...     load_classification_bake_off_2017_results
... )
>>> from aeon.visualisation import plot_critical_difference
>>> # Load the results
>>> results, data, cls = load_classification_bake_off_2023_results(
...     num_resamples=100, as_array=True
... )  
>>> # Select a subset of classifiers
>>> cls = ["MSM_1NN","TSF","DTW_F","EE","BOSS","ST","FlatCOTE"] 
>>> index = [cls.index(i) for i in cls] 
>>> selected = results[:,index]  
>>> # Plot the critical difference diagram
>>> plot = plot_critical_difference(selected, cls)  
>>> plot.show()