[R] regression tree xerror

[Luis Torgo]

Sherri Miller wrote:

>I am running some models (for the first time) using rpart and am getting
>results I don't know how to interpret. I'm using cross-validation to prune
>the tree and the results look like:
>Root node error: 172.71/292 = 0.59148
>
>n= 292
>
>         CP nsplit rel error  xerror     xstd
>1  0.124662      0   1.00000 1.00731 0.093701
>2  0.064634      1   0.87534 1.08076 0.092337
>3  0.057300      2   0.81070 1.07684 0.095582
>4  0.038462      4   0.69610 0.99104 0.091659
>5  0.036200      5   0.65764 1.01596 0.094635
>6  0.029228      7   0.58524 1.00058 0.095440
>7  0.028779      8   0.55601 1.00704 0.093242
>8  0.024192      9   0.52724 0.97844 0.088936
>9  0.018038     11   0.47885 1.02749 0.092263
>10 0.016867     13   0.44278 1.08704 0.092112
>11 0.015465     14   0.42591 1.10805 0.097813
>12 0.015000     15   0.41044 1.11130 0.097881
>
>I do not understand why the rel error rate is going down, but the xerror
>generally goes up. For some of the runs, the xerror never goes down. Is
>result caused by something in my data structure? I have run some example
>datasets from the various help manuals and the xerror goes down, as one
>would expect. Any suggestions?
>
>Sherri
>
>Sherri L. Miller
>Wildlife Biologist
>Redwood Sciences Laboratory
>707.825.2949
>707.825.2901 (FAX)
>
>______________________________________________
>R-help at stat.math.ethz.ch mailing list
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>  
>
rel error is estimated with the training data (the sample used for 
obtaining the tree) and thus it decreases as the tree increases, because 
the tree becomes more and more adjusted to the data. This apparently 
better performance should not be taken for "real" when predicting for a 
new sample of data because larger trees do tend to overfit the traning 
sample and will hardly generalise well on new fresh data samples.

That's the motivation for the xerror (and xstd) estimates. These are 
more realistic estimates of the performance of the tree on new samples 
of data. They are obtained by the rpart function by an internal cross 
validation process. The function prune() can be used to select a subtree 
of the tree obtained with rpart() if you think (by looking at the xerror 
estimates) you would be better off with this subtree.

Hope this helps.

Luis Torgo

-- 
Luis Torgo
    FEP/LIACC, University of Porto   Phone : (+351) 22 339 20 93
    Machine Learning Group           Fax   : (+351) 22 339 20 99
    R. de Ceuta, 118, 6o             email : ltorgo at liacc.up.pt
    4050-190 PORTO - PORTUGAL        WWW   : http://www.liacc.up.pt/~ltorgo