get_estimator_results_as_array¶

get_estimator_results_as_array(estimators: str | list[str], datasets: list[str] | None = None, num_resamples: int | None = 1, task: str = 'classification', measure: str = 'accuracy', remove_dataset_modifiers: bool = False, path: str = 'http://timeseriesclassification.com/results/ReferenceResults', include_missing: bool = False)[source]¶

Look for results for given estimators for a list of datasets.

This function loads or pulls down a CSV of results, scans it for datasets and returns any results found as an array. If a dataset is not present, it is ignored.

Parameters:

estimatorslist of str: Estimator name or list of estimator names to search for. See get_available_estimators, aeon.benchmarking.results_loading.NAME_ALIASES or the directory at path for valid options.
datasetslist of or None, default=1: List of problem names to search for. If None, all datasets the estimator has results for is returned. If the dataset is not present in any of the results, it is ignored unless include_missing is true.
num_resamplesint or None, default=None: The number of data resamples to average over for all scores. The first resample is the default train/test split for the dataset. For 1, only the score for the default train/test split of the dataset is returned. For 2 or more, the scores of all resamples up to num_resamples are averaged and returned. If None, the scores of all resamples are averaged and returned.
taskstr, default=”classification”: Should be one of aeon.benchmarking.results_loading.VALID_TASK_TYPES. i.e. “classification”, “clustering”, “regression”.
measurestr, default=”accuracy”: Should be one of aeon.benchmarking.results_loading.VALID_RESULT_MEASURES[task]. Dependent on the task, i.e. for classification, “accuracy”, “auroc”, “balacc”, and regression, “mse”, “mae”, “r2”.
remove_dataset_modifiers: bool, default=False: If True, will remove any dataset modifier (anything after the first underscore) from the dataset names in the loaded results file. i.e. a loaded result row for “Dataset_eq” will be converted to just “Dataset”.
pathstr, default=”https://timeseriesclassification.com/results/ReferenceResults/”: Path where to read results from. Defaults to timeseriesclassification.com.
include_missingbool, default=False: Whether to include datasets with missing results in the output. If False, the whole problem is ignored if any estimator is missing results it. If True, NaN is returned instead of a score in missing cases.

Returns:

results: 2D numpy array: Array of scores. Each column is a results for a classifier, each row a dataset.
names: list of str: List of dataset names that were retained.

Examples

>>> from aeon.benchmarking.results_loaders import get_estimator_results
>>> cls = ["HC2", "FreshPRINCE"] 
>>> data = ["Chinatown", "Adiac"] 
>>> get_estimator_results_as_array(estimators=cls, datasets=data) 
(array([[0.98250729, 0.98250729],
       [0.81074169, 0.84143223]]), ['Chinatown', 'Adiac'])