[R] Ridge regression
Ravi Varadhan
RVaradhan at jhmi.edu
Wed Dec 2 19:22:40 CET 2009
The help page clearly states that ans$coef is "not on the original scale and
are for use by the coef method". You also see that ans$scales gives you the
scales used in the computation of ans$coef.
So, to get coefficients on the original scale, you can either use coef(ans)
or you can divide ans$coef by ans$scales.
X1 <- runif(20)
X2 <- runif(20)
Y <- 2 * X1 - 2 * X2 + rnorm(20, sd=0.1)
lam <- 10
ans1 <- lm.ridge(Y ~ X1 + X2, lambda = lam)
all.equal(ans1$coef / ans1$scales, coef(ans1)[2:3] )
Hope this helps,
Ravi.
----------------------------------------------------------------------------
-------
Ravi Varadhan, Ph.D.
Assistant Professor, The Center on Aging and Health
Division of Geriatric Medicine and Gerontology
Johns Hopkins University
Ph: (410) 502-2619
Fax: (410) 614-9625
Email: rvaradhan at jhmi.edu
Webpage:
http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h
tml
----------------------------------------------------------------------------
--------
-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On
Behalf Of Ravi Varadhan
Sent: Wednesday, December 02, 2009 12:25 PM
To: 'David Winsemius'; 'Eleni Christodoulou'
Cc: r-help at r-project.org
Subject: Re: [R] Ridge regression
You are right that the ans$coef and coef(ans) are different in ridge
regression, where `ans' is the object from lm.ridge. It is the coef(ans)
that yields the coefficients on the original scale. ans$coef is the
coefficient of "X-scaled" and "Y-centered" version.
Here is an example that illustrates the workings of ridge regression.
First let us create some data:
X1 <- runif(20)
X2 <- runif(20)
Y <- 2 * X1 - 2 * X2 + rnorm(20, sd=0.1)
lam <- 10
ans1 <- lm.ridge(Y ~ X1 + X2, lambda = lam)
ans1$coef
coef(ans1)
# Note that these two are different
# Now Let us scale the variables X1 and X2 and center Y
#
cY <- scale(Y, scale=FALSE)
n <- length(Y)
sX1 <- scale(X1) * sqrt(n/(n-1))
sX2 <- scale(X2) * sqrt(n/(n-1))
require(MASS)
lam <- 10
ans2 <- lm.ridge(cY ~ sX1 + sX2, lambda = lam)
ans2$coef
coef(ans2)
# Now, see that the coefficients of sX1 and sX2 are the same
# This is the connection!
# Armed with this insight, we now compare the ans1$coef with scaled
coefficients
#
ans1$coef
c(coef(ans1)[2] * sd(X1), coef(ans1)[3] * sd(X2)) * sqrt((n-1)/n)
# Now they are the same!
I hope this is clear.
Best,
Ravi.
----------------------------------------------------------------------------
-------
Ravi Varadhan, Ph.D.
Assistant Professor, The Center on Aging and Health
Division of Geriatric Medicine and Gerontology
Johns Hopkins University
Ph: (410) 502-2619
Fax: (410) 614-9625
Email: rvaradhan at jhmi.edu
Webpage:
http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h
tml
----------------------------------------------------------------------------
--------
-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On
Behalf Of David Winsemius
Sent: Wednesday, December 02, 2009 11:04 AM
To: Eleni Christodoulou
Cc: r-help at r-project.org
Subject: Re: [R] Ridge regression
On Dec 2, 2009, at 10:42 AM, Eleni Christodoulou wrote:
> Dear list,
>
> I have a couple of questions concerning ridge regression. I am using
> the
> lm.ridge(...) function in order to fit a model to my microarray data.
> Thus *model=lm.ridge(...)*
> I retrieve some coefficients and some scales for each gene. First of
> all, I
> would like to ask: the real coefficients of the model are not
> included in
> the first argument of the output but in the result of coef(model),
> am I
> right?
Not exactly. coef(model) extracts the coefficients from the model but
the coefficients do in the example instance I created following the
help page happen to be in the first element of the model.
eg:
> long.rr$coef
GNP Unemployed Armed.Forces Population Year
Employed
25.3615288 3.3009416 0.7520553 -11.6992718 -6.5403380
0.7864825
> long.rr[[1]]
GNP Unemployed Armed.Forces Population Year
Employed
25.3615288 3.3009416 0.7520553 -11.6992718 -6.5403380
0.7864825
> Moreover, what does the scale argument represent? Which is its
> connection with the coefficients? The R help file os not very
> informative
> for me...
A plausible response to such a question might be that the help page is
a sketchy substitute for the MASS book. However, I cannot find ridge
regression in the table of contents or in the index of my copy, but I
only have ed. 2 and the current edition is the 4th. So we will both
need to wait for more knowledgeable (or with more recent editions of
MASS) persons to answer that question.
(And "scales" is not an argument, rather it's a returned value.)
>
> Thank you very much in advance,
> Eleni Christodoulou
>
David Winsemius, MD
Heritage Laboratories
West Hartford, CT
______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
More information about the R-help
mailing list