[R] speed issues? read R_inferno by Patrick Burns: & a memory query

[Uwe Ligges]

Actually the issue is not the size of memory that is consumed, but that 
memory allocation takes place and the object is copied in each iteration 
of the "bad" loop you have given below. This is not required for the 
second loop, where R can allocate the memory at once and does not need 
to copy the object around.

Uwe Ligges


On 23.12.2010 14:13, maddox wrote:
>
> Hi,
>
> I'm just starting out with R and came across R_inferno.pdf by Patrick Burns
> just yesterday - I recommend it!
>
> His description of how 'growing' objects (e.g. obj<- c(obj,
> additionalValue) eats up memory prompted me to rewrite a function (which
> made such calls ~210 times) so that it used indexing into a dimensioned
> object instead (i.e. obj[i, ]<- additionalValue).
>
> This transformed the process from
> old version:
>     user  system elapsed
> 133.436  14.257 155.807
>
> new version:
>     user  system elapsed
>   16.041   1.180  18.535
>
> To say I'm delighted is understatement. Thanks for putting the Inferno
> together,  Patrick.
>
> However I'm misunderstanding the effect this has on memory use, (or
> misunderstanding the code I've highjacked to look at memory use). To look at
> virtual memory use I'm using  the code below from this forum:
> cmd<- paste("ps -o vsz", Sys.getpid())
> cat("\nVirtual size: ", system(cmd, intern = TRUE) [2], "\n", sep = "")
>
> I did three runs of the old version, and three with the new, preceding each
> with gc()&  got the outputs below. In summary, the runs of old method
> required 17712, 17744&  17744&  runs of new method required 13788, 15140&
> 13656.
>
> Two questions:
> 1. why does each run of the same process not make the same demand on memory?
> They're doing exactly the same work&  creating exactly the same new objects.
> 2. is the modest decrease in memory consumed by new method expected? (having
> read R_Inferno I was, perhaps naively, expecting more of an improvement)
>
> ? or am I missing something (more than likely! )
>
> Thanks
>
> M
>
>
>
>
>> gc()
>           used (Mb) gc trigger (Mb) max used (Mb)
> Ncells 786300 21.0    1265230 33.8  1166886 31.2
> Vcells 948412  7.3    3244126 24.8  3766604 28.8
>
>> cat("old version")
>
> Virtual size before call: 881692
>     user  system elapsed
> 131.872  14.417 159.653
>
> Virtual size after call: 899404
>> 899404-881692
> [1] 17712
>
> ##################
>
>> gc()
>           used (Mb) gc trigger (Mb) max used (Mb)
> Ncells 786294 21.0    1265230 33.8  1166886 31.2
> Vcells 948407  7.3    3244126 24.8  3766604 28.8
>
>> cat("old version")
>
> Virtual size before call: 881660
>     user  system elapsed
> 133.281  14.473 159.661
>
> Virtual size after call: 899440
>> 899440-881660
> [1] 17780
>
> ##################
>
>> gc()
>           used (Mb) gc trigger (Mb) max used (Mb)
> Ncells 786294 21.0    1265230 33.8  1166886 31.2
> Vcells 948407  7.3    3244126 24.8  3766604 28.8
>
>> cat("old version")
>
> Virtual size before call: 881696
>     user  system elapsed
> 133.436  14.257 155.807
>
>
> Virtual size after call: 899440
>> 899440-881696
> [1] 17744
>
> ################## ##################
>
>> gc()
>           used (Mb) gc trigger (Mb) max used (Mb)
> Ncells 786413 21.0    1265230 33.8  1166886 31.2
> Vcells 948460  7.3    3244126 24.8  3766604 28.8
>
>> cat("new version")
>
> Virtual size before call: 881696
>
>     user  system elapsed
>   16.041   1.180  18.535
>
>
> Virtual size after call: 895484
>> 895484-881696
> [1] 13788
>
> ##################
>
>> gc()
>           used (Mb) gc trigger (Mb) max used (Mb)
> Ncells 786441 21.1    1265230 33.8  1166886 31.2
> Vcells 948480  7.3    3244126 24.8  3766604 28.8
>
>> cat("new version")
>
> Virtual size before call: 882648
>     user  system elapsed
>   16.321   1.068  18.136
>
> Virtual size after call: 897788
>> 897788- 882648
> [1] 15140
>
> ##################
>
>> gc()
>           used (Mb) gc trigger (Mb) max used (Mb)
> Ncells 786441 21.1    1265230 33.8  1166886 31.2
> Vcells 948480  7.3    3244126 24.8  3766604 28.8
>
>> cat("new version")
>
> Virtual size before call: 882648
>
>     user  system elapsed
>   16.581   0.992  19.351
>
> Virtual size after call: 896304
>> 896304-882648
> [1] 13656
>
>
>
>
>