[R] memory error with 64-bit R in linux

Thu Jul 19 06:09:45 CEST 2007

The output from gc() indicates that you had a maximum usage of
476MB+119MB=~600MB.  If you look at the output of ps you will notice
that the process size is 523MB (or about 500MB if you want to be
exact).  So you are using about 25% of the 2GB that you have
available.

mem.limit just shows the current value of the parameters, and as the
help file says:

"Value
mem.limits() returns an integer vector giving the current settings of
the maxima, possibly NA."

On 7/18/07, zhihua li <lzhtom at hotmail.com> wrote:
> Thanks for replying!
> i don't think i'm paging. i tried to use a smaller version of my matrix and
> do all the checkings as suggested by jim. The smaller matrix caused another
> problem, for which I've opened another thread. But i've found something
> about memory that I don't understand.
> > gc()
>          used (Mb) gc trigger  (Mb) max used  (Mb)
> Ncells  269577 14.4    5570995 297.6  8919855 476.4
> Vcells 3353395 25.6    9493567  72.5 15666095 119.6
>
> Does this mean the maximum memory I can use for variables is only 120 M?
> However, when I tried to check the memory limits:
> > mem.limits()
> nsize vsize
>   NA    NA
>
> Here it seems the maximum memory is not limited?
>
> When there is no R function is being executed, I checked the system process
> by:
> ps u
>
> PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> 7821  0.0  0.1  10048  2336 pts/0    Ss   Jul18   0:00 -bash
> 8076  2.9 24.5 523088 504004 pts/0   S+   Jul18   2:46 /usr/lib64/R/bi
> 8918  1.5  0.1   9912  2328 pts/1    Ss   00:44   0:00 -bash
> 8962  0.0  0.0   3808   868 pts/1    R+   00:45   0:00 ps u
>
> Does this mean R is using 25% of my memory? But my RAM is 2 GB and the
> objects in R only occupy 40 MB from gc().
>
> Did I interpret it wrong?
>
> Thanks a lot!
>
>
>
> >From: "jim holtman" <jholtman at gmail.com>
> >To: "zhihua li" <lzhtom at hotmail.com>
> >CC: r-help at stat.math.ethz.ch
> >Subject: Re: [R] memory error with 64-bit R in linux
> >Date: Wed, 18 Jul 2007 17:50:31 -0500
> >
> >Are you paging?  That might explain the long run times. How much
> >space
> >are your other objects taking up?  The matrix by itself should only
> >require about 13MB if it is numeric.  I would guess it is some of
> >the
> >other objects that you have in your working space.  Put some gc() in
> >your loop to see how much space is being used.  Run it with a subset
> >of the data and see how long it takes.  This might give you an
> >estimate of the time, and space, that might be needed for the entire
> >dataset.
> >
> >Do a 'ps' to see how much memory your process is using.  Do one
> >every
> >couple of minutes to see if it is growing.  You can alway use
> >Rprof()
> >to get an idea of where time is being spent (use it on a small
> >subset).
> >
> >On 7/18/07, zhihua li <lzhtom at hotmail.com> wrote:
> >>Hi netters,
> >>
> >>I'm using the 64-bit R-2.5.0 on a x86-64 cpu, with an RAM of 2 GB.
> >>The
> >>operating system is SUSE 10.
> >>The system information is:
> >>-uname -a
> >>Linux someone 2.6.13-15.15-smp #1 SMP Mon Feb 26 14:11:33 UTC 2007
> >>x86_64
> >>x86_64 x86_64 GNU/Linux
> >>
> >>I used heatmap to process a matrix of the dim [16000,100].  After 3
> >>hours
> >>of desperating waiting, R told me:
> >>cannot allocate vector of size 896 MB.
> >>
> >>I know the matrix is very big, but since I have 2 GB of RAM and in
> >>a 64-bit
> >>system, there should be no problem to deal with a vector smaller
> >>than 1 GB?
> >>(I was not running any other applications in my system)
> >>
> >>Does anyone know what's going on?  Is there a hardware limit where
> >>I have
> >>to add more RAM, or is there some way to resolve it softwarely?
> >>Also is it
> >>possible to speed up the computing (I don't wanna wait another 3
> >>hours to
> >>know I get another error message)
> >>
> >>Thank you in advance!
> >>
> >>_________________________________________________________________
> >>ÏíÓÃÊÀ½çÉÏ×î´óµÄµç×ÓÓÊ¼þÏµÍ³¡ª MSN Hotmail¡£
> >>http://www.hotmail.com
> >>
> >>
> >>______________________________________________
> >>R-help at stat.math.ethz.ch mailing list
> >>https://stat.ethz.ch/mailman/listinfo/r-help
> >>PLEASE do read the posting guide
> >>http://www.R-project.org/posting-guide.html
> >>and provide commented, minimal, self-contained, reproducible code.
> >>
> >>
> >
> >
> >--
> >Jim Holtman
> >Cincinnati, OH
> >+1 513 646 9390
> >
> >What is the problem you are trying to solve?
>
> _________________________________________________________________
> ÓëÁª»úµÄÅóÓÑ½øÐÐ½»Á÷£¬ÇëÊ¹ÓÃ MSN Messenger:  http://messenger.msn.com/cn
>
>

-- 
Jim Holtman
Cincinnati, OH
+1 513 646 9390

What is the problem you are trying to solve?