[R] value of complexity parameter in ridge regression
Simon Wood
s.wood at bath.ac.uk
Wed Apr 18 10:45:16 CEST 2007
> What is the optimum range to look for a value of lambda while doing ridge
> regression. Can/ should lambda be greater than 1 ?
-- I think it's data dependent, but lambda can certainly be greater than one.
For many ridge regression problems you can choose lambda `objectively' by
generalized cross validation (GCV). package `mgcv' provides a routine `magic'
for doing this (although it doesn't use the most efficient method if you only
have one lambda/ridge penalty). Of course this won't be appropriate if the
ridge penalty is only being used to stabilize the fit and you want the
minimum lambda that e.g. makes X'X+ \lambda I +ve definite.
> I have conflicting (or what appears conflicting to me) sources that use
> lambda >= 0, without any upper limit, but that makes the search space
> infinite.. right ??
>
> So, perhaps my question is: is there an upper limit to lambda.
I don't think so.
> Does the value of lambda convey something about my data ?
Depends on the details of the model. For some models ridge penalties can be
viewed as inverses of random effect covariance matrices, in which case lambda
is related to the random effect variance. (see e.g. section 6.2.6. of 2006
book referenced in ?gam from mgcv package).
best,
Simon
>
> Thanks a lot,
> Sikander
>
>
> ---------------------------------
>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html and provide commented, minimal,
> self-contained, reproducible code.
--
> Simon Wood, Mathematical Sciences, University of Bath, Bath, BA2 7AY UK
> +44 1225 386603 www.maths.bath.ac.uk/~sw283
More information about the R-help
mailing list