Code Description

Click to jump to a particular section of this page.

Code Reference

      The code from Project 3: Neural Networks has been modified to be extra special and spiffy than it already was. Please see that project for more details on the history of the code.

Back to Top

• Attribute value representation
  
  
  • As before, all nominal (discrete) attributes are transformed into binomial/boolean attributes. For example:

    	att {value1, value2, value3}

    becomes

    	att=value1 {false, true}
    	att=value2 {false, true}

    The third value is implicitly represented by all the new attributes having a value of false.
  
  
  
  • The numbers {0, 1} used to represent the values in the previous Neural Net project have been changed to {0.1, 0.9} in this version of the code.
  
  
  
  • The reasoning behind this was that binomial attributes could not contribute to the answer produced by the neural network when their value is 0 since the weights are multiplied by the node output. Since many of the attributes in this data set are nominal and have many possibly values, a large number of the binomial attributes had a value of 0 for every instance. This representation does not allow the neural net to use this information.
  
  
  
  • It is interesting to note, when initializing weights to zero in the original project, one could see many weights that did not change. These were grouped by the input node associated with the weight. These were the nodes who's input was 0 most of the time. A similar experiment with the modified code did not exhibit this behavior.


• Normalization
  
  
  • Previously, all numeric attribute values were fed to the neural net as is. Since the numeric attributes in the data set varied a great deal in their range, it took many iterations of training for back propagation to effectively set weights to negate the extreme bias that attributes with exceptionally large numeric values had influencing the output so that the available information could be used to classify correctly.
  
  
  
  • All the numeric attribute values have been normalized between {0, 1}. this was done using the weka.filter.NormalizationFilter. It takes up to two data sets, tracks the range of values each numeric attribute can have and then normalizes all the values using the range specific to that attribute. This insures the same normalization procedure is used on both the training and test data.


• Reporting
  
  
  • The reporting capabilities of this Back Propagation system have been enhanced to display the error sum (and optionally, current network) every 100 iterations. If necessary, the number of iterations can be made a parameter.
  
  
  
  • Next to the input node weights, the corresponding attribute name is displayed.
  
  
  
  • Under the hidden node weights, the corresponding classification is displayed. The weights above a classification go from a hidden node to the output node for that classification.

Back to Top