| PRTools contents |
TESTC
[E,C] = TESTC(A*W,TYPE)
[E,C] = TESTC(A,W,TYPE)
E = A*W*TESTC([],TYPE)
[E,F] = TESTC(A*W,TYPE,LABEL)
[E,F] = TESTC(A,W,TYPE,LABEL)
E = A*W*TESTC([],TYPE,LABEL)
| Input | |
| A | Dataset |
| W | Trained classifier mapping |
| TYPE | Type of performance estimate, default: probability of error |
| LABEL | Target class, default: none |
| Output | |
| E | Error / performance estimate |
| C | Number of erroneously classified objects per class. They are sorted according to A.LABLIST |
| F | Error / performance estimate of the non-target classes |
This routine supplies several performance estimates for a trained classifier W based on a test dataset A. It is possible to supply a cell array of datasets {A*W}, an N x 1 cell array of datasets {A} and an N x M cell array of mappings {W}.
A should contain test objects for every class assigned by W. Objects in A belonging to different classes than defined for W as well as unlabeled objects are neglected. Note that this implies that TESTC applied to a rejecting classifier (e.g. REJECTC) estimates the performance on the not rejected objects only. By [E,C] = TESTC(A,W); E = (C./CLASSSIZES(A))*GETPRIOR(A)';
the classification error with respect to all objects in A may be computed. Use CONFMAT for an overview of the total class assignment including the unlabeled (rejected) objects.
In case of missing classes in A, [E,C] = TESTC(A*W) returns in E a NaN but in C still the number of erroneously classified objects per class.
If LABEL is given, the performance estimate relates just to that class as target class. If LABEL is not given a class average is returned weighted by the class priors.
The following performance measures are supported for TYPE
| 'crisp' | Expected classification error based on error counting, weighted by the class priors (default). |
| 'FN' | E False negative F False positive |
| 'TP' | E True positive F True negative |
| 'soft' | Expected classification error based on soft error summation, i.e. a sum of the absolute difference between classifier output and target, weighted by class priors. |
| 'F' | Lissack and Fu error estimate |
| 'mse' | Expected mean square difference between classifier output and target (based on soft labels), weighted by class priors. |
| 'auc' | Area under the ROC curve (this is an error and not a performance!). For multi class problems this is the weigthed average (by class priors) of the one-against-rest contributions of the classes. |
| 'precision' | E Fraction of true target objects among the objects classified as target. The target class is defined by LABEL. Priors are not used. F Recall, fraction of correctly classified objects in the target class. Priors are not used. |
| 'sensitivity' | E Fraction of correctly classified objects in the target class (defined by LABEL). Priors are not used. Sensitivity as used her is identical to recall. F Specificity, fraction non target objects that are not classified into the target class (defined by LABEL). Priors are not used. |
| PRTools contents |