Physical Sciences and Mathematics Commons^™

Sampling With Confidence: Using K-Nn Confidence Measures In Active Learning, Rong Hu, Sarah Jane Delany, Brian Macnamee

Conference papers

Active learning is a process through which classifiers can be built from collections of unlabelled examples through the cooperation of a human oracle who can label a small number of examples selected as most informative. Typically the most informative examples are selected through uncertainty sampling based on classification scores. However, previous work has shown that, contrary to expectations, there is not a direct relationship between classification scores and classification confidence. Fortunately, there exists a collection of particularly effective techniques for building measures of classification confidence from the similarity information generated by k-NN classifiers. This paper investigates using these confidence measures …