Combined classification error rate estimator for the Fisher linear classifier

Abstract

Classification error rate estimation is one of the most important issues in machine learning  and pattern recognition. This problem has been studied by many researchers and a number of error estimators have been proposed. However, theoretical analysis and empirical experiments show that most of these error estimation techniques are biased.  One way to correct this biasis to use a linear combination of two different error rate estimators. In this paper we propose a new combined classification error rate estimator designed specially for the Fisher linear classifier. Experiments with real world and synthetic data sets show that resubstitution, leave-one-out, repeated10-fold cross-validation, repeated 2-foldcross-validation, basic bootstrap, 0.632 bootstrap, zero bootstrap, D-method, DS-method and M-method are outperformed by the proposed combined error rate estimator (in terms of root-mean-square error).

DOI: http://dx.doi.org/10.5755/j01.itc.45.4.14268