[R] memory management

[William Dunlap]

> I do a lot of strsplit, unlist, & subsetting, so I could imagine why
> the RSS is triple the total size of my data if all the intermediate
> results are not released.

I can only give some generalities about that.  Using lots of
small chunks of memory (like short strings) may cause fragmentation
(wasted space between blocks of memory).  Depending on your operating
system, calling free(pointerToMemoryBlock) may or may not reduce the
virtual memory size of the process, so something like '/bin/ps -o vsize,size'
or Process Explorer may only show the high water mark of memory usage.

Another way to gauge the total size of the visible data and the
environments associated with it is to call save(list=objects(all=TRUE),
compress=FALSE,file="someFile") and look at the size of the file.
Headers probably have a different size in the file than in the process,
but it can give some hints about how much hidden environments are
adding to things.

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com 

> -----Original Message-----
> From: Sam Steingold [mailto:sam.steingold at gmail.com] On Behalf Of Sam Steingold
> Sent: Wednesday, February 29, 2012 8:42 AM
> To: William Dunlap
> Cc: r-help at r-project.org
> Subject: Re: memory management
> 
> > * William Dunlap <jqhaync at gvopb.pbz> [2012-02-28 23:06:54 +0000]:
> >
> > You need to walk through the objects, checking for environments on
> > each component or attribute of an object.
> 
> so why doesn't object.size do that?
> 
> >   > f <- function(n) {
> >   +   d <- data.frame(y = rnorm(n), x = rnorm(n))
> >   +   lm(y ~ poly(x, 4), data=d)
> >   + }
> 
> I am not doing any modeling. No "~". No formulas.
> The whole thing is just a bunch of data frames.
> I do a lot of strsplit, unlist, & subsetting, so I could imagine why
> the RSS is triple the total size of my data if all the intermediate
> results are not released.
> 
> --
> Sam Steingold (http://sds.podval.org/) on Ubuntu 11.10 (oneiric) X 11.0.11004000
> http://www.childpsy.net/ http://honestreporting.com http://memri.org
> http://jihadwatch.org http://pmw.org.il http://camera.org http://ffii.org
> To be popular with ladies one has to be smart, handsome & rich. Or to be a cat.