[R] RAM usage
Vaidotas Zemlys
mpiktas at delfi.lt
Fri Oct 18 12:27:26 CEST 2002
Hi,
I'm having problems while working with large data sets with R 1.5.1 in
windows 2000. Given a integer matrix size of 30 columns and 15000 rows
my function should return a boolean matrix size of about 5000 rows and
15000 columns.
First of all I tried to run this function on computer with 256 MB of
RAM. I increased memory limit of R with memory.limit() up to 512 MB. I
was inspecting memory and processor usage through Windows task manager.
At first R was using 100% of processor and memory usage was constantly
increasing. When there were no physical memory left, R began using
virtual memory. Then the processor usage dropped, but there was
instensive work with hard drive. Of course that slowed down
calculations. Yet the memory used by R always changed, and that was I
think the sign, that R was calculating. But after a while the task
manager showed that R uses constant size of memory. The Rgui was not
responding, so I assumed that R crashed.
So I tried to run the calculations on another win2k box with 1024 MB of
RAM with the same R version 1.5.1. This time virtual memory was not
used, yet still R froze. The memory usage grew to about 450 MB and then
R stopped. Memory usage was not changing, Rgui did not respond, yet
processor was used 100%. Task manager showed that peak memory usage was
about 760 MB.
On smaller data sets there were no problems, memory usage was constantly
increasing and processor was used 100%. My function does not use fancy
functions. Basically it just sums, finds minimum and maximum, uses
subseting of a matrix, and calculates correlation matrix between 10
columns of a given matrix.
So I would like to ask can R at all perform such calculations where a
lot of memory must be used? And if R can do such calculations, what are
specific problems, topics or tips which should be known before letting R
to do these calculations?
Thanks in advance, for any help and special thanks for reading such a
long letter.
Vaidotas Zemlys
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
More information about the R-help
mailing list