[R] Odp: Error: cannot allocate vector of size xxx Mb

[Petr PIKAL]

Hi

r-help-bounces at r-project.org napsal dne 05.08.2010 09:53:21:

> I am dealing with very large data frames, artificially created with
> the following code, that are combined using rbind.
> 
> 
> a <- rnorm(5000000)
> b <- rnorm(5000000)
> c <- rnorm(5000000)
> d <- rnorm(5000000)
> first <- data.frame(one=a, two=b, three=c, four=d)
> second <- data.frame(one=d, two=c, three=b, four=a)

Up to this point there is no error on my system

> version
               _  
platform       i386-pc-mingw32  
arch           i386  
os             mingw32  
system         i386, mingw32  
status         Under development (unstable)  
major          2  
minor          12.0  
year           2010  
month          05  
day            31  
svn rev        52164  
language       R  
version.string R version 2.12.0 Under development (unstable) (2010-05-31 
r52164)

> sessionInfo()
R version 2.12.0 Under development (unstable) (2010-05-31 r52164)
Platform: i386-pc-mingw32/i386 (32-bit)

attached base packages:
[1] stats     grDevices datasets  utils     graphics  methods   base 

other attached packages:
[1] lattice_0.18-8 fun_1.0 

loaded via a namespace (and not attached):
[1] grid_2.12.0  tools_2.12.0

> rbind(first, second)

Although size of first and second is only roughly 160 MB their 
concatenation probably consumes all remaining memory space as you already 
have a-d first and second in memory.

Regards
Petr

> 
> which results in the following error for each of the statements:
> 
> > a <- rnorm(5000000)
> Error: cannot allocate vector of size 38.1 Mb
> > b <- rnorm(5000000)
> Error: cannot allocate vector of size 38.1 Mb
> > c <- rnorm(5000000)
> Error: cannot allocate vector of size 38.1 Mb
> > d <- rnorm(5000000)
> Error: cannot allocate vector of size 38.1 Mb
> > first <- data.frame(one=a, two=b, three=c, four=d)
> Error: cannot allocate vector of size 38.1 Mb
> > second <- data.frame(one=d, two=c, three=b, four=a)
> Error: cannot allocate vector of size 38.1 Mb
> > rbind(first, second)
> 
> When running memory.limit() I am getting this:
> 
> memory.limit()
> [1] 2047
> 
> Which shows me that I have 2 GB of memory available. What is wrong?
> Shouldn't 38 MB be very feasible?
> 
> Best,
> Ralf
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide 
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.