Document Type
Conference Paper
Rights
Available under a Creative Commons Attribution Non-Commercial Share Alike 4.0 International Licence
Abstract
Producing estimates of classification confidence is surprisingly difficult. One might expect that classifiers that can produce numeric classification scores (e.g. k-Nearest Neighbour, Na¨ıve Bayes or Support Vector Machines) could readily produce confidence estimates based on thresholds. In fact, this proves not to be the case, probably because these are not probabilistic classifiers in the strict sense. The numeric scores coming from k-Nearest Neighbour, Na¨ıve Bayes and Support Vector Machine classifiers are not well correlated with classification confidence. In this paper we describe a case-based spam filtering application that would benefit significantly from an ability to attach confidence predictions to positive classifications (i.e. messages classified as spam). We show that ‘obvious’ confidence metrics for a case-based classifier are not effective. We propose an ensemble-like solution that aggregates a collection of confidence metrics and show that this offers an effective solution in this spam filtering domain.
DOI
https://doi.org/10.1007/11536406_16
Recommended Citation
Delany, S.J., Cunningham, P. & Zamolotskikh, A. (2005). Generating estimates of classification confidence for a case-based spam filter. Proceedings of the 6th. International Conference on Case-Based Reasoning ICCBR'05, Chicago, IL, USA, August 23-26. doi:10.1007/11536406_16