[R] memory problems for fixed effect models

[Daniel Malter]

Are y and s continuous or is one of them a factor/dummy variable? From the
mle specification I grasp that s is the unit of observation and is
factor-coded. If that is so, then estimating lm(o~y+s) includes a lot of
dummy/factor variables (lots of columns of 0/1 in the X matrix), and then
there could be a problem as you describe it. To avoid this problem you may
want to use the plm library. This library conducts fixed-effects analyses
without including dummy variables to denote the unit of observation.
Instead, it demeans the data for each unit of observation and readjusts the
standard errors to account for the reduction in the degrees of freedom due
to the fixed effects.

HTH,
Daniel

-------------------------
cuncta stricte discussurus
-------------------------

-----Ursprüngliche Nachricht-----
Von: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] Im
Auftrag von Jörg Schaber
Gesendet: Wednesday, September 23, 2009 10:06 AM
An: r-help at stat.math.ethz.ch
Betreff: [R] memory problems for fixed effect models

Hi,

I am trying to fit a simple two-way fixed effect linear model (o ~ y + s
- 1). However, my problem is large (length(o)=79333).
I am already using slm.fit with a dense design matrix (ddm), but still:

fit <- slm.fit(ddm,o)
Error: cannot allocate vector of size 9.1 Gb

Is there a way to redefine the model or any other trick such that I do not
run into these memory problems?

Running a mixed effect model, like
lme(o ~ y - 1 ,random = ~ 1 | s, method="REML",
contrasts=list(s=("contr.sum")),na.action=na.exclude)
poses no problem, but I want to have both effects fixed.

Thanks,

joerg

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.