[R] building a formula for glm() with 30,000 independent variables
Thomas Lumley
tlumley at u.washington.edu
Sun Nov 10 22:42:47 CET 2002
On Sun, 10 Nov 2002, Ben Liblit wrote:
> I would like to use R to perform a logistic regression with about
> 30,000 independent variables. That's right, thirty thousand. Most
> will be irrelevant: the intent is to use the regression to identify
> the few that actually matter.
>
> Among other things, this calls for giving glm() a colossal "y ~ ..."
> formula with thirty thousand summed terms on its right hand side. I
> build up the formula as a string and then call as.formula() to convert
> it. Unfortunately, the conversion fails. The parser reports that it
> has overflowed its stack. :-(
>
> Is there any way to pull this off in R? Can anyone suggest
> alternatives to glm() or to R itself that might be capable of handling
> a problem of this size? Or am I insane to even be considering an
> analysis like this?
You could call glm.fit directly. It takes a design matrix rather than a
formula.
You'll still have some trouble with inverting the 30,000x30,000
information matrix, and of course it will be singular if you don't have
30,000 observations.
Since there are 2^30000 possible submodels, finding the good ones isn't
going to be easy either. My guess is that you want to do something else,
but I don't know what.
-thomas
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
More information about the R-help
mailing list