This item is available under a Creative Commons License for non-commercial use only
Active learning is a process through which classiﬁers can be built from collections of unlabelled examples through the cooperation of a human oracle who can label a small number of examples selected as most informative. Typically the most informative examples are selected through uncertainty sampling based on classiﬁcation scores. However, previous work has shown that, contrary to expectations, there is not a direct relationship between classiﬁcation scores and classiﬁcation conﬁdence. Fortunately, there exists a collection of particularly eﬀective techniques for building measures of classiﬁcation conﬁdence from the similarity information generated by k-NN classiﬁers. This paper investigates using these conﬁdence measures in a new active learning sampling selection strategy, and shows how the performance of this strategy is better than one based on uncertainty sampling using classiﬁcation scores.
Hu, R., Delany, S.J., & Mac Namee, B. (2009) Sampling with Conﬁdence: Using k-NN Conﬁdence Measures in Active Learning, In: Proceedings of the UKDS Workshop at 8th International Conference on Case-based Reasoning (ICCBR 09) p.181-192. doi:10.21427/D7H90Z