PRTools contents

PRTools manual

testc

TESTC

Test classifier, error / performance estimation

    [E,C] = TESTC(A*W,TYPE)
    [E,C] = TESTC(A,W,TYPE)
     E = A*W*TESTC([],TYPE)

    [E,F] = TESTC(A*W,TYPE,LABEL)
    [E,F] = TESTC(A,W,TYPE,LABEL)
     E = A*W*TESTC([],TYPE,LABEL)

Input
 A Dataset
 W Trained classifier mapping
 TYPE Type of performance estimate, default: probability of error
 LABEL Target class, default: none

Output
 E Error / performance estimate
 C Number of erroneously classified objects per class.  They are sorted according to A.LABLIST
 F Error / performance estimate of the non-target classes

Description

This routine supplies several performance estimates for a trained  classifier W based on a test dataset A. It is possible to supply a cell  array of datasets {A*W}, an N x 1 cell array of datasets {A} and an N x M cell array of mappings {W}.

A should contain test objects for every class assigned by W.  Objects in A belonging to different classes than defined for W as well  as unlabeled objects are neglected. Note that this implies that TESTC applied to a rejecting classifier (e.g. REJECTC) estimates the  performance on the not rejected objects only. By    [E,C] = TESTC(A,W); E = (C./CLASSSIZES(A))*GETPRIOR(A)';
  the classification error with respect to all objects in A may be  computed. Use CONFMAT for an overview of the total class assignment  including the unlabeled (rejected) objects.

In case of missing classes in A, [E,C] = TESTC(A*W) returns in E a NaN  but in C still the number of erroneously classified objects per class.

If LABEL is given, the performance estimate relates just to that class as  target class. If LABEL is not given a class average is returned weighted  by the class priors.

The following performance measures are supported for TYPE

'crisp' Expected classification error based on error counting,  weighted by the class priors (default).
 'FN' E False negative  F False positive
 'TP' E True positive  F True negative
'soft' Expected classification error based on soft error  summation, i.e. a sum of the absolute difference between  classifier output and target, weighted by class priors.
 'F' Lissack and Fu error estimate
'mse' Expected mean square difference between classifier output  and target (based on soft labels), weighted by class  priors.
'auc' Area under the ROC curve (this is an error and not a  performance!). For multi class problems this is the  weigthed average (by class priors) of the one-against-rest  contributions of the classes.
'precision' E Fraction of true target objects among the objects  classified as target. The target class is defined by LABEL.  Priors are not used.  F Recall, fraction of correctly classified objects in the  target class. Priors are not used.
'sensitivity' E Fraction of correctly classified objects in the target  class (defined by LABEL). Priors are not used.  Sensitivity as used her is identical to recall.  F Specificity, fraction non target objects that are not  classified into the target class (defined by LABEL).  Priors are not used.

 EXAMPLES   See PREX_PLOTC.

See also

mappings, datasets, confmat, rejectc,

PRTools contents

PRTools manual