API Reference

Samplers for Evaluation

Classes

OASISSampler(alpha, predictions, scores, oracle)

Optimal Asymptotic Sequential Importance Sampling (OASIS) for estimation of the weighted F-measure.

ImportanceSampler(alpha, predictions, …[, …])

Importance sampling for estimation of the weighted F-measure

DruckSampler(alpha, predictions, scores, oracle)

Stratified sampling for estimation of the weighted F-measure

PassiveSampler(alpha, predictions, oracle[, …])

Passive sampling for estimation of the weighted F-measure

Stratification

Classes

stratification.Strata(allocations)

Represents a collection of strata and facilitates sampling from them

Functions

stratification.stratify_by_scores(scores[, …])

Stratify by binning the items based on their scores

stratification.stratify_by_features(…)

Stratify by clustering the items in feature space

stratification.auto_stratify(scores, **kwargs)

Generate Strata instance automatically