CompStats.measurements

CI(samples: ndarray, alpha=0.05)[source]

Compute the Confidence Interval of a statistic using bootstrap. :param samples: Bootstrap samples :type samples: np.ndarray :param alpha: \([\frac{\alpha}{2}, 1 - \frac{\alpha}{2}]\). :type alpha: float

>>> from CompStats import StatisticSamples, CI
>>> from sklearn.metrics import accuracy_score
>>> import numpy as np
>>> labels = np.r_[[0, 0, 0, 0, 0, 1, 1, 1, 1, 1]]
>>> pred   = np.r_[[0, 0, 1, 0, 0, 1, 1, 1, 0, 1]]
>>> bootstrap = StatisticSamples(statistic=accuracy_score)
>>> samples = bootstrap(labels, pred)
>>> CI(samples)
(0.6, 1.0)
SE(samples: ndarray)[source]

Compute the Standard Error of a statistic using bootstrap: the standard deviation of the bootstrap samples.

Parameters:

samples (numpy.ndarray or CompStats.bootstrap.StatisticSamples) – Bootstrap samples, or a StatisticSamples instance, in which case a dictionary with one value per system is returned.

Returns:

Standard error (one value per measure when the samples have several columns).

>>> from CompStats import StatisticSamples, SE
>>> from sklearn.metrics import accuracy_score
>>> import numpy as np
>>> labels = np.r_[[0, 0, 0, 0, 0, 1, 1, 1, 1, 1]]
>>> pred   = np.r_[[0, 0, 1, 0, 0, 1, 1, 1, 0, 1]]
>>> bootstrap = StatisticSamples(statistic=accuracy_score)
>>> samples = bootstrap(labels, pred)
>>> SE(samples)
0.1459
difference_p_value(samples: ndarray, BiB: bool = True)[source]

Estimate the p-value of the paired differences, as the fraction of bootstrap differences that are at least twice the mean difference (the bootstrap distribution is centred on the observed difference rather than on zero). With BiB false the comparison is reversed.

Parameters:
Returns:

Estimated p-value.