[R] How to estimate whether overfitting?

[David Winsemius]

On May 9, 2010, at 9:20 AM, bbslover wrote:

>
> 1. is there some criterion to estimate overfitting?  e.g. R2 and Q2  
> in the
> training set, as well as R2 in the test set, when means  
> overfitting.   for
> example,  in my data, I have R2=0.94 for the training set and  for  
> the test
> set R2=0.70, is overfitting?
> 2. in this scatter, can one say this overfitting?
>
> 3. my result is obtained by svm, and the sample are 156 and 52 for the
> training and test sets, and predictors are 96,   In this case, can  
> svm be
> employed to perform prediction?   whether the number of the  
> predictors are
> too many ?
>

I think you need to buy a copy of Hastie, Tibshirani, and Friedman and  
do some self-study of chapters 7 and 12.


> 4.from this picture, can you give me some suggestion to improve model
> performance？ and is the picture bad?
>
>
> 5. the picture and data below.
> thank you!
>
>
> http://n4.nabble.com/file/n2164417/scatter.jpg scatter.jpg
>
> http://n4.nabble.com/file/n2164417/pkc-svm.txt pkc-svm.txt
> -- 
-- 

David Winsemius, MD
West Hartford, CT