Visit neurosolutions.com | Download Evaluation Volume 7, No.2 This is a Neurosolutions Newsletter, which you are receiving because you requested to stay informed about new developments at NeuroDimension. You can also access current and past issues of Neurosolutions Newsletters online. Interpreting Outputs of a Classification Neural Network as Probabilities The most common way to solve a classification problem with a neural network is to unary encode the desired output. This is done automatically when you tag a desired output column as symbolic. What this does is create a column of 0's and 1's for each unique category you are trying to classify. For each row there will be one desired output column encoded with a 1 (indicating the desired category for the row) and the remaining columns will be encoded with a 0. After training a classification network, the network outputs will not be all 1's and 0's – they will be values in between. The most common approach to evaluating the performance of a classification network is to take the output column with the highest value from each row and assign it a “1” and then assign the remaining output columns in the row as “0”. This is then compared with the desired 0's and 1's to determine if the row was correctly classified or not. Note that the Confusion Matrix probe provides a good summary of the number of instances of each category that were classified correctly. There are applications where it is not only useful to determine which category the model maps the row to, but some estimate of probability that it is right. For example, let's say we have a network with three unary-encoded outputs representing three categories. Here is the desired output for the first two rows: D1 D2 D3 0 1 0 1 0 0 Here is the actual network output for those first two rows: O1 O2 O3 0.10 0.85 0.15 0. 45 0.20 0.50 By postprocessing the outputs to take the max value, we get: O1 O2 O3 0 1 0 0 0 1 We see that it classified one of the two correctly. Looking at the outputs for that first row, observe that the highest value in the second column is significantly higher than the other two output values. This is not the case for the second row. The highest value in the third column is only slightly higher than the value in the first column. Not knowing the desired outputs, it would be reasonable to assume that there would be a higher level of confidence in the results from the first row than those in the second row. The SoftMaxAxon in NeuroSolutions can be used as a postprocessor to translate the network outputs into probabilities. For the example we just gave, the outputs from a SoftMaxAxon would be as follows: O1 O2 O3 0.24 0.51 0.25 0. 35 0.28 0.37 Note that the outputs for each row sum to 1. The confidence level for the classification of the first row would be 51%, while the confidence level for the second row would be 37%. You cannot conclude from this that SoftMaxAxon outputs that produce a 0.51 will be correct 51% of the time. However, you can use these postprocessed outputs as a relative measure of confidence between classifications. To add a SoftMaxAxon to a classification network in NeuroSolutions, you simply need to: Display the Axon toolbar if it is not already visible (Tools -> Customize -> Axon Components) Stamp a SoftMaxAxon below the output Axon Left-click on the output Axon Right-click on the SoftMaxAxon and select “Connect To” from the context menu Stamp a DataWriter probe on the right side of the SoftMaxAxon Train the network Double click the DataWriter to display the probe window Run the Testing Wizard The probabilities should be displayed for each row in your test set.