[R] value of complexity parameter in ridge regression

[Simon Wood]

> What is the optimum range to look for a value of lambda while doing ridge
> regression. Can/ should lambda be greater than 1 ?
-- I think it's data dependent, but lambda can certainly be greater than one. 
For many ridge regression problems you can choose lambda `objectively' by 
generalized cross validation (GCV). package `mgcv' provides a routine `magic' 
for doing this (although it doesn't use the most efficient method if you only 
have one lambda/ridge penalty). Of course this won't be appropriate if the 
ridge penalty is only being used to stabilize the fit and you want the 
minimum lambda that e.g. makes X'X+ \lambda I +ve definite. 

> I have conflicting (or what appears conflicting to me) sources that use
> lambda >= 0, without any upper limit, but that makes the search space
> infinite.. right ??
>
> So, perhaps my question is: is there an upper limit to lambda.
I don't think so.
> Does the  value of lambda convey something about my data ?
Depends on the details of the model. For some models ridge penalties can be 
viewed as inverses of random effect covariance matrices, in which case lambda 
is related to the random effect variance. (see e.g. section 6.2.6. of 2006 
book referenced in ?gam from mgcv package).

best,
Simon

>
> Thanks a lot,
> Sikander
>
>
> ---------------------------------
>
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html and provide commented, minimal,
> self-contained, reproducible code.

-- 
> Simon Wood, Mathematical Sciences, University of Bath, Bath, BA2 7AY UK
> +44 1225 386603  www.maths.bath.ac.uk/~sw283