[R] Memory(RAM) issues

David Winsemius dwinsemius at comcast.net
Thu Jun 23 18:04:34 CEST 2011 

On Jun 23, 2011, at 11:57 AM, Abhisek Saha wrote:

> Hi Lui, Anupam and my other R-user friends,
> Thanks  for your previous suggestions. As for my issue, it is clearly
> RAM problem as my code is running perfectly as long as my input data
> size is small and code has been refined a number of times to increase
> efficiency [ by using matrix more in the context of manipulations and
> reducing order of time complexity wherever possible]. Yet  it has
> gotta run 2000 iterations and in each of the iterations it repeatedly
> modifies  few [0.5 Mn * 15] tables
>
> After few basic investigation it seems that the problem MAY be
> resolved if 64 bit version of R is installed in 64 bit OS as it claims
> to be able use potentially a  8TB RAM ( for my case I think 4 GB RAM
> may serve the purpose but 2 GB is not enough) if available.I don't
> have any domain knowledge in IT since I hail from a statistics
> background.  I work in a company as a statistical analyst. The IT
> folks of my company are ready to deploy any free solution but I need
> to tell them what software needs to be installed and what all OS
> configurations would be required. They normally run all of the
> applications on unix servors.
>
> So I need to know if any free 64 bit of R version can be installed in
> unix servor or if not unix, may be on other servor. So hereby I
> request my R-user friends to please let me know if any free 64 bit
> version of R is available

Yes.

> that can be installed on any unix servor. If
> that is not available is there any other FREE  solution and if
> available, how to get that and what all configuration is required.

There are a bunch of "UNIX-alike" platforms. Read the FAQ:

http://cran.r-project.org/doc/FAQ/R-FAQ.html#What-machines-does-R-run-on_003f

--  
David.
>
> Awaiting your replies,
> Regards,
> Abhisek
>
> P.S. Please forward this mail to any other R-mailing list if you deem
> it fit for any of them.
>
> On Sat, Jun 11, 2011 at 4:38 PM, Lui ##  
> <lui.r.project at googlemail.com> wrote:
>> Hello Abhisek,
>>
>> maybe you wanna try it on just a bigger machine (I guess you are
>> working at a university, so I hope you do have access to them). In
>> case getting computing time or the like is a major issue, you may
>> wanna try Amazon AWS: For a few rupees (about 50-100 per hour) you  
>> can
>> "rent" pretty fast computers (20 Ghz, 8BG of RAM). You may want to  
>> try
>> out the Windows version (little bit more expensive) which is easily
>> accessible via remote desktop. Installing Revolution (which is free
>> for academic purposes) (64 Bit Version) might give you a good start.
>> Maybe its not a viable option in a long term view (pricing), but it
>> might help to get a clue whether the problem can be solved on a  
>> bigger
>> machine and just trying it out without a bigger hastle...
>>
>> Good luck!
>>
>> Lui
>>
>> On Sat, Jun 11, 2011 at 7:47 AM, Abhisek Saha <tading at gmail.com>  
>> wrote:
>>> Thanks Anupam for your inputs. I believe there are two ways to
>>> circumvent the issue...1> making the code more efficient 1>  
>>> Increasing
>>> the memory in some way.The reasons why I did not paste the code  
>>> are 1>
>>> It is long and using quite a number of functions that  I created 2>
>>> Secondly my intention is not to make the code more efficient if  
>>> that's
>>> possible. Here landing into a memory problem with 2 GB RAM is  
>>> natural
>>> as my analysis entails 1500 simulations from huge multivariate
>>> distributions that change after every simulation and tomorrow I may
>>> have to do similar analysis with 10 million observations * 20  
>>> columns.
>>>
>>> In view of above I shall be needing more memory sometime later and  
>>> my
>>> IT friends are ready to support me for that(probably with a sandbox)
>>> but I am not sure whether I can install probably a servor version  
>>> of R
>>> that can be capable of working with 8GB or so RAM. So it is more of
>>> technical help I need and I have no clue regarding the  
>>> plausibility of
>>> the solution mentioned( i.e. a servor version of R that is capable  
>>> of
>>> more memory).
>>>
>>> Regards,
>>> Abhisek
>>>
>>> On Sat, Jun 11, 2011 at 10:10 AM, Anupam <anupamtg at gmail.com> wrote:
>>>>
>>>> It will be helpful on this forum to use metric measures: 12 Lakh  
>>>> is 1.2
>>>> million, thus your data is 1.2 million observations x 15  
>>>> variables. I do not
>>>> know the intricacies of R. You may have to wait for someone with  
>>>> that
>>>> knowledge to respond.
>>>>
>>>> Including some relevant portions of error messages and code in  
>>>> your query
>>>> can also help someone to respond to your message.
>>>>
>>>> Anupam.
>>>> -----Original Message-----
>>>> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org 
>>>> ] On
>>>> Behalf Of Abhisek Saha
>>>> Sent: Saturday, June 11, 2011 6:25 AM
>>>> To: r-help at r-project.org
>>>> Subject: [R] Memory(RAM) issues
>>>>
>>>> Dear All,
>>>> I have been working with R(desktop version) in VISTA. I have the  
>>>> latest
>>>> version R 2.13.0. I have been working with few data-sets of size  
>>>> 12Lakh * 15
>>>> and my code is quite computing intensive ( applying MCMC gibbs  
>>>> sampler on a
>>>> posterior of 144 variables) that often runs into a memory issue  
>>>> such as
>>>> memory can not allocate the vector ,full size(shows to have reached
>>>> something like 1.5 GB) reached or something to this effect. I  
>>>> have a RAM of
>>>> 2GB.  I checked with the option like memory.size and it says a 64  
>>>> bit R if
>>>> sat on 64 bit windows then max memory capability is 8TB.
>>>>
>>>> Now I don't have  background to understand the definitions and  
>>>> differences
>>>> between 32 and 64 bit machines and other technical requirements  
>>>> like servor
>>>> etc but it would be of great help if anyone can let me have a  
>>>> feel of it.
>>>> Could any of you tell me whether some servor version of R would  
>>>> resolve my
>>>> issue or not (I am not sure now what kind of servor my company  
>>>> would allow R
>>>> to be installed at this point ,may be linux type) and if that's  
>>>> the case
>>>> could any of you guide me about how to go about installing that  
>>>> onto a
>>>> sevor.
>>>>
>>>> Thank you,
>>>> Abhisek
>>>>
>>>>        [[alternative HTML version deleted]]
>>>>
>>>> ______________________________________________
>>>> R-help at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>>> and provide commented, minimal, self-contained, reproducible code.
>>>>
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

David Winsemius, MD
West Hartford, CT

More information about the R-help mailing list