[R] Memory limit on Linux?

[Stackpole, Chris]

> From: Jack Challen [mailto:jack.challen at ocsl.co.uk] 
> Sent: Wednesday, August 14, 2013 10:45 AM
> Subject: RE: Memory limit on Linux?
>
> (I'm replying from a horrific WebMail UI. I've attempted to maintain
> what I think is sensible quoting. Hopefully it reads ok).
[snip]
> If all users are able to allocate that much RAM in a single process
> (e.g. "top" shows the process taking 20 GBytes) then it's very unlikely
> to be an OS- or user-specific restriction (there are exceptions to that if
>  e.g. the R job is submitted in a different shell environment [e.g. batch
>  queuing]). Your ulimits look sensible to me.

That is what I thought as well. I was just looking for maybe some cgroup limitation or something similar that might be stirring problems. However, I don't see anything like that.

> > The only differences I have found between the two boxes that really 
> > stands out is that the system that works runs RHEL proper and has R 
> > compiled but the one that doesn't allocate all of the memory was installed
> > via EPEL RPM on CentOS. Compiling R on the CentOS system is on the
> > try-this list, but before I spend that time trying to compile I thought I
> > would ask a few questions.
> 
> I would look there first. It seems (from the first quoted bit) that your
> problem is specific to that version of R on that machine as Matlab can
> gobble up RAM happily (I do have a very simple bit of C kicking about
> here which is specifically for testing the memory allocation limit of a
> system which you could have if you really wanted).

Thanks for the offer. I may take you up on that. I am downloading the latest and greatest version of R right now for compiling purposes. If that doesn't work, then I may try your program just to see what results it has.

[snip]

> > 2) When I compile from source to test this, is there a specific option I should pass to ensure max usage?
>
> Absolutely no idea, I'm afraid. There is an --enable-memory-profiling
> option, but I doubt that''ll solve your problem and it'll probably just
> slow R down. I'd simply give compiling it a go.

I will report back to the list after I get R compiled.

Thanks!