|
Project 10.2: Neural Networks Revisted : Report
Code Description
Click to jump to a particular section of this page.
Code Reference
The code from
Project 3: Neural Networks
has been modified to be extra special and spiffy than it
already was. Please see that project for more details on
the history of the code.
Back to Top
Code Adaptation
- Attribute value representation
-
As before, all nominal (discrete) attributes are transformed
into binomial/boolean attributes. For example:
att {value1, value2, value3}
becomes att=value1 {false, true}
att=value2 {false, true}
The third value is implicitly represented by all the new
attributes having a value of false.
-
The numbers {0, 1} used to represent the values in the
previous Neural Net project have been changed to {0.1, 0.9} in this
version of the code.
-
The reasoning behind this was that binomial attributes could not
contribute to the answer produced by the neural network
when their value is 0 since the weights are multiplied
by the node output. Since many of the attributes in this
data set are nominal and have many possibly values, a large number
of the binomial attributes had a value of 0 for every instance. This
representation does not allow the neural net to use this
information.
-
It is interesting to note, when initializing weights to zero in the
original project, one could see many weights that did not change.
These were grouped by the input node associated with the weight.
These were the nodes who's input was 0 most of the time.
A similar experiment with the modified code did not exhibit this
behavior.
- Normalization
-
Previously, all numeric attribute values were fed to the neural
net as is. Since the numeric attributes in the data set varied
a great deal in their range, it took many iterations of training
for back propagation to effectively set weights to negate the extreme
bias that attributes with exceptionally large numeric values had
influencing the output so that the available
information could be used to classify correctly.
-
All the numeric attribute values have been normalized between {0, 1}.
this was done using the weka.filter.NormalizationFilter. It takes
up to two data sets, tracks the range of values each numeric attribute
can have and then normalizes all the values using the range specific
to that attribute. This insures the same normalization
procedure is used on both the training and test data.
- Reporting
-
The reporting capabilities of this Back Propagation system have
been enhanced
to display the error sum (and optionally, current network) every
100 iterations.
If necessary, the number of iterations can be made a parameter.
-
Next to the input node weights, the corresponding attribute
name is displayed.
-
Under the hidden node weights, the corresponding classification
is displayed. The weights above a classification go from a hidden
node to the output node for that classification.
Back to Top
|
|