Control Experiment Results

      All experiments here were done without pruning.

Click to jump to a particular section of this page.

No Missing Values

      This test was performed by generating the tree using the first 1000 instances of the training data set that did not contain any missing values. The tree for this can be found in test-1000-500-no-missing.txt. The resulting tree's performance was measured against the first 500 instances found in the test data set. A summary of these results is given below:

      Correctly Classified Instances         408               81.4371 %
      Incorrectly Classified Instances        93               18.5629 %
      Mean absolute error                      0.205 
      Root mean squared error                  0.4081
      Relative absolute error                 56.5203 %
      Root relative squared error             96.7046 %
      Total Number of Instances              501    
    

Back to Top

      This test was performed by generating the tree using only the attributes that had a discrete number of values. All instances in the training data set were used. The tree for this can be found in test-500-discrete-only.txt. The resulting tree's performance was measured against the first 500 instances found in the test data set. A summary of these results is given below:

      Correctly Classified Instances         427               81.9578 %
      Incorrectly Classified Instances        94               18.0422 %
      Mean absolute error                      0.2342
      Root mean squared error                  0.3648
      Relative absolute error                 65.2094 %
      Root relative squared error             86.8557 %
      Total Number of Instances              521
    

Back to Top

      The trees from the above tests were also tested against the full set of testing data. The trees described in each test's respective file are the same as those found above. The results are summarized and compared below.

No Missing Values:

      Correctly Classified Instances       13135               80.6769 %
      Incorrectly Classified Instances      3146               19.3231 %
      Mean absolute error                      0.2062
      Root mean squared error                  0.4052
      Relative absolute error                 56.4718 %
      Root relative squared error             95.3749 %
      Total Number of Instances            16281
    

Discrete Values Only

      Correctly Classified Instances       13410               82.3659 %
      Incorrectly Classified Instances      2871               17.6341 %
      Mean absolute error                      0.2301
      Root mean squared error                  0.3595
      Relative absolute error                 63.3294 %
      Root relative squared error             84.6192 %
      Total Number of Instances            16281
    

It would seem that using the first 500 instances for testing purposes yields an accurate measurement of these trees' performance.

Back to Top