[R] can the matrix size limit be increased?

[Robert Leach]

Hi there,

I'm brand new to R, so let me know if this question is not  
appropriate for this list.  I've been reading through the  
documentation and have tried a number of things, but am pretty much  
stuck so far.  Here's the session info:

 > sessionInfo()
R version 2.6.2 (2008-02-08)
i386-apple-darwin8.10.1

locale:
C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

loaded via a namespace (and not attached):
[1] rcompgen_0.1-17


So I seem to be hitting a limit on matrix size.  First I read in my  
data into a list and it's OK:

 > mz <- scan("data.column3.txt", list(0))
Read 158991 records
 >mz
 > mz
[[1]]
     [1] 0.000000e+00 0.000000e+00 1.003393e+01 3.651888e+00 0.000000e 
+00
     [6] 0.000000e+00 3.067042e+00 1.277249e+00 1.984366e+00 3.644203e 
+01
    [11] 1.172925e+02 1.933753e+02 2.020940e+02 1.570501e+02 8.990829e 
+01
...

But when I try to put it into a matrix like this, I don't get an  
error, but the matrix appears empty...

 > MZ=matrix(mz[[1]],nrow=1)
 > MZ
      [,1] [,2]     [,3]     [,4] [,5] [,6]     [,7]     [,8]     [, 
9]    [,10]
         [,11]    [,12]    [,13]    [,14]   [,15]    [,16]    [, 
17]    [,18]
         [,19]    [,20]    [,21]    [,22]    [,23]    [,24]    [, 
25]    [,26]
...

When I did a subset of my data, it was fine.  I did a manual binary  
search and determined the cutoff to be 100000 elements.  So if I do  
just 99,999 elements, it looks as I would expect:

 > mz <- scan("data.column3.txt", list(0), 99999)
Read 99999 records
 > tst <- matrix(mz[[1]],nrow=1)
 > tst
      [,1] [,2]     [,3]     [,4] [,5] [,6]     [,7]     [,8]     [, 
9]    [,10]
[1,]    0    0 10.03393 3.651888    0    0 3.067042 1.277249 1.984366  
36.44203
         [,11]    [,12]    [,13]    [,14]   [,15]    [,16]    [, 
17]    [,18]
[1,] 117.2925 193.3753 202.0940 157.0501 89.9083 26.44127 17.05373  
53.40315
         [,19]    [,20]    [,21]    [,22]    [,23]    [,24]    [, 
25]    [,26]
[1,] 65.20086 37.33463 17.71247 27.37268 41.83289 48.46916 58.94969  
76.05099
...

If I do 100,000, I get the same empty appearance.  I've assumed that  
there must be a limitation on the number of elements in a matrix.  Is  
that right?  If so, how do I increase the maximum number of  
elements?  I tried another machine's installation of R and it  
apparently doesn't have a 99,999 element limit.  I've tried using:

R --max-mem-size=2G
R --max-vsize=200000
R --max-nsize=200000
R --max-vsize=200000 --max-nsize=200000 --max-ppsize=200000
R --max-vsize=10M

I still end up with the empty-looking matrix when I try these.  How  
do I get my installation to work like the installation on another  
computer I tried where I was able to have larger matrices?

Oh yeah, I also tried this, just to rule out problems with my data:

 > tst <- matrix(seq(1,158991),nrow=1,ncol=158991)
 > tst
      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12]  
[,13] [,14]
      [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [, 
25] [,26]
      [,27] [,28] [,29] [,30] [,31] [,32] [,33] [,34] [,35] [,36] [, 
37] [,38]
...

Thanks,
Rob

Robert W. Leach
Scientific Programmer
Center for Computational Research
Center of Excellence in Bioinformatics
University at Buffalo
http://www.ccr.buffalo.edu/