Approximation of unbiased convex classification error rate estimator

  • Mindaugas Gvardinskas Department of System Analysis, Vytautas Magnus University, Vileikos St. 8, LT-44404 Kaunas, Lithuania
  • Minija Tamošiūnaitė Department for Computational Neuroscience, III Physikalisches Institut—Biophysik, Bernstein Center for Computational Neuroscience, Georg-August-Universitaet Goettingen, Goettingen D-37077, Germany
Keywords: Error estimation, Classification, Resubstitution, Cross-validation, Bootstrap

Abstract

Convex classification error rate estimator is described as weighted combination of the low-biased estimator and the high-biased estimator. If the underlying data model is known, the coefficients (weights)  can be optimized so that the bias and root-mean-square error of the estimator is minimized. However, in most situations,  data model is unknown. In this paper we propose a new error estimation method, based on approximation of unbiased convex error rate estimator. Experiments with real world and synthetic data sets show that common error estimation methods, such as resubstitution, repeated 10-foldcross-validation, leave-one-out and random subsampling are outperformed (in terms of root-mean-square error) by the proposed method.

DOI: http://dx.doi.org/10.5755/j01.itc.45.2.12052

Published
2016-06-27
Section
Regular Papers