[R] memory tops out at 1.84gb on OS X 10.4 machine w/ 5GB ram

[Prof Brian Ripley]

The issue here is not the per-process addess space, which is 4Gb on all 
32-bit OSes (more or less by definition), but the user address space of 
the process.  That will be less, and how much less depends on the OS.
For most it seems to be 3Gb for the user, 1Gb for the system, but Windows 
by default has 2Gb for each.  I've never seen an authorative figure for 
MacOS X.

That's one issue.  Another is that with a 3Gb address space, it is likely 
that there will be no large contiguous blocks once more than half of it is 
in use, so memory allocation can fail even if there is memory available 
(as if is fragmented).

See ?"Memory-limits" and the R-admin manual (on 32- vs 64-bit versions) 
for further discussion.

Note that although Apple claims to have a 64-bit OS, the reality is
that they have made some small steps in that direction:

http://arstechnica.com/reviews/os/macosx-10.4.ars/4

AFAIK the precompiled version of R is 32-bit only rather than the 'fat 
binary' discussed there, and in any case GUI applications are restricted 
to 32-bit.

Note that we don't even know if 'Jerk Alert' was talking about MacOS, as 
he did not actually say so.  But only MacOS users seems to refer to a 
operating system using two Tens (one Roman).


On Thu, 15 Dec 2005, Sean Davis wrote:

> On 12/15/05 10:36 AM, "Albert Vilella" <avilella at gmail.com> wrote:
>
>> El dj 15 de 12 del 2005 a les 08:17 -0500, en/na Roger D. Peng va
>> escriure:
>>> I'm not completely sure, but I don't think OS X is at the point yet where it
>>> can
>>> access > 2GB of memory (like, for example, Linux on Opteron).  More
>>> specifically, I'm not sure a single process image can access > 2GB of memory,
>>> but I'd welcome any corrections to that statement.  To be sure, this problem
>>> is
>>> not an issue with R because R has regularly been reported to access u> 4GB of
>>> memory when the OS allows it.
>>
>> I may seen somewhere that OSX has a "per process" limit of ~1.5GB max
>> RAM,
>
> I don't think this is true.  See here for Apple's answer:
>
> http://developer.apple.com/documentation/Performance/Conceptual/ManagingMemo
> ry/index.html
>
> Note that receiving a malloc error means that malloc failed.  Therefore,
> what you see in ActivityMonitor is the memory consumption BEFORE a large
> memory block is allocated, if I understand things correctly.  Correct me if
> I'm wrong here, please.


-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595