[R] Memory question

[Prof Brian Ripley]

On Thu, 14 Jul 2005, Kenneth Cabrera wrote:

> Thank you Dr. Spencer Graves for your answer.
>
> What kind of matrices? They come form an image of about 3000x5000, and
> I need to generate arround 1024 matrices of the same size, they are not 
> sparse
> matrices.
>
> What function can I use to, once generated one matrix, I could save into disk
> and then use the same space for the following, and so on.

You can use either save or .saveRDS/serialize followed by rm() and gc(). 
You cannot use the same space, but you can free up the space.

Then when you need the data again, load/.readRDS/unserialize can pull the 
object back.  (If you arrange this right the object will only go into a 
temporary frame and so only be needed one at a time.)

>
> Thank you very much for your help
>
> Kenneth
>
> Spencer Graves wrote:
>
>> 	  What kinds of matrices?  There are facilities in the Matrix and 
>> SparseM packages that might help for sparse matrices.  If they are N x k 
>> where N is large and k is not, can you compute something like the QR 
>> decomposition and get away with keeping only the R part for most of your 
>> matrices?
>> 
>> 	  One could potentially define a class of matrices that are only kept 
>> in memory only when needed;  I think S-Plus may do that.  It would take a 
>> lot of work to make that work generally, but you might be able to 
>> accomplish what you need with a much smaller effort.
>> 
>> 	  spencer graves
>> 
>> Kenneth Roy Cabrera Torres wrote:
>> 
>> 
>>> Hi R users and developers:
>>> 
>>> I want to know how can I save memory in R
>>> for example:
>>>  - saving on disk a matrix.
>>>  - using again the matrix (changing their values)
>>>  - saving again the matrix on disk in a different file.
>>> 
>>> The idea is that I have a process that generate several
>>> matrices, but if I keep them all in memory it will overflow.
>>> 
>>> How can I save them in different files, so I use the same
>>> amount of memory for each processed matrix?
>>> 
>>> Thank you for your help.
>>> 
>>> 
>> 
>> 
>

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595