[R] analysis of large data set

[Greg Snow]

On behalf of those of us trapped in 32-bit windows worlds.  Thanks to
Brian and the others that work on the windows port of R for the great
tool and for pushing the boundaries of microsoft.  Know that your
efforts are appreciated even if we don't say Thank You often enough.

-- 
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
greg.snow at intermountainmail.org
(801) 408-8111
 
 

> -----Original Message-----
> From: r-help-bounces at r-project.org 
> [mailto:r-help-bounces at r-project.org] On Behalf Of Prof Brian Ripley
> Sent: Friday, November 16, 2007 11:47 PM
> To: sj
> Cc: r-help
> Subject: Re: [R] analysis of large data set
> 
> On Fri, 16 Nov 2007, sj wrote:
> 
> > All,
> >
> > I am working with a large data set (~ 450,000 rows by 34 
> columns) I am 
> > trying to fit a regression model (I have tried to use several 
> > procedures psm (Design package) lm, glm). However whenever I try to 
> > fit the model I get the following error:
> >
> >
> > Error: cannot allocate vector of size 1.1 Gb
> >
> > Here are the specs of the machine and version of R I am using
> >
> > Windows Server 2003 R2 Enterprise x64 Service Pack 2
> >
> > Intel Pentium D 3.00 Ghz
> > 3.93 GB Ram
> >
> > R 2.6.0
> >
> > when I type the command
> >
> > memory.limit()
> > I get:
> > 3583.875
> >
> > I assume that means that I have about 3.5 GB at my disposal 
>  so I am 
> > confused why I can't allocate a vector of 1.1 GB. Any 
> suggestions on 
> > what to do.
> 
> See ?"Memory-limits", which explains this in detail.
> 
> What is strange is that no one has ever thanked us for 
> finding out (despite most Microsoft documentation) that you 
> can get up to 3.5Gb in a 32-bit process on certain 64-bit 
> versions of Windows and enabling you to use it in recent 
> versions of R.
> 
> -- 
> Brian D. Ripley,                  ripley at stats.ox.ac.uk
> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
> University of Oxford,             Tel:  +44 1865 272861 (self)
> 1 South Parks Road,                     +44 1865 272866 (PA)
> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide 
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>