[R] can the matrix size limit be increased?

[jim holtman]

You only have 1 row in your matrix, so what you are getting printed
out is not an empty matrix, but the header.  If you print the
transpose you get:

> head(t(tst))
     [,1]
[1,]    1
[2,]    2
[3,]    3
[4,]    4
[5,]    5
[6,]    6
>

The default is to only print out 100000 values.  Your data is there,
you just have to wait for the headers to print out, or at least make a
matrix with more than one row.

On Fri, Feb 29, 2008 at 2:25 PM, Robert Leach <rwleach at ccr.buffalo.edu> wrote:
> Hi there,
>
> I'm brand new to R, so let me know if this question is not
> appropriate for this list.  I've been reading through the
> documentation and have tried a number of things, but am pretty much
> stuck so far.  Here's the session info:
>
>  > sessionInfo()
> R version 2.6.2 (2008-02-08)
> i386-apple-darwin8.10.1
>
> locale:
> C
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base
>
> loaded via a namespace (and not attached):
> [1] rcompgen_0.1-17
>
>
> So I seem to be hitting a limit on matrix size.  First I read in my
> data into a list and it's OK:
>
>  > mz <- scan("data.column3.txt", list(0))
> Read 158991 records
>  >mz
>  > mz
> [[1]]
>     [1] 0.000000e+00 0.000000e+00 1.003393e+01 3.651888e+00 0.000000e
> +00
>     [6] 0.000000e+00 3.067042e+00 1.277249e+00 1.984366e+00 3.644203e
> +01
>    [11] 1.172925e+02 1.933753e+02 2.020940e+02 1.570501e+02 8.990829e
> +01
> ...
>
> But when I try to put it into a matrix like this, I don't get an
> error, but the matrix appears empty...
>
>  > MZ=matrix(mz[[1]],nrow=1)
>  > MZ
>      [,1] [,2]     [,3]     [,4] [,5] [,6]     [,7]     [,8]     [,
> 9]    [,10]
>         [,11]    [,12]    [,13]    [,14]   [,15]    [,16]    [,
> 17]    [,18]
>         [,19]    [,20]    [,21]    [,22]    [,23]    [,24]    [,
> 25]    [,26]
> ...
>
> When I did a subset of my data, it was fine.  I did a manual binary
> search and determined the cutoff to be 100000 elements.  So if I do
> just 99,999 elements, it looks as I would expect:
>
>  > mz <- scan("data.column3.txt", list(0), 99999)
> Read 99999 records
>  > tst <- matrix(mz[[1]],nrow=1)
>  > tst
>      [,1] [,2]     [,3]     [,4] [,5] [,6]     [,7]     [,8]     [,
> 9]    [,10]
> [1,]    0    0 10.03393 3.651888    0    0 3.067042 1.277249 1.984366
> 36.44203
>         [,11]    [,12]    [,13]    [,14]   [,15]    [,16]    [,
> 17]    [,18]
> [1,] 117.2925 193.3753 202.0940 157.0501 89.9083 26.44127 17.05373
> 53.40315
>         [,19]    [,20]    [,21]    [,22]    [,23]    [,24]    [,
> 25]    [,26]
> [1,] 65.20086 37.33463 17.71247 27.37268 41.83289 48.46916 58.94969
> 76.05099
> ...
>
> If I do 100,000, I get the same empty appearance.  I've assumed that
> there must be a limitation on the number of elements in a matrix.  Is
> that right?  If so, how do I increase the maximum number of
> elements?  I tried another machine's installation of R and it
> apparently doesn't have a 99,999 element limit.  I've tried using:
>
> R --max-mem-size=2G
> R --max-vsize=200000
> R --max-nsize=200000
> R --max-vsize=200000 --max-nsize=200000 --max-ppsize=200000
> R --max-vsize=10M
>
> I still end up with the empty-looking matrix when I try these.  How
> do I get my installation to work like the installation on another
> computer I tried where I was able to have larger matrices?
>
> Oh yeah, I also tried this, just to rule out problems with my data:
>
>  > tst <- matrix(seq(1,158991),nrow=1,ncol=158991)
>  > tst
>      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12]
> [,13] [,14]
>      [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,
> 25] [,26]
>      [,27] [,28] [,29] [,30] [,31] [,32] [,33] [,34] [,35] [,36] [,
> 37] [,38]
> ...
>
> Thanks,
> Rob
>
> Robert W. Leach
> Scientific Programmer
> Center for Computational Research
> Center of Excellence in Bioinformatics
> University at Buffalo
> http://www.ccr.buffalo.edu/
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



-- 
Jim Holtman
Cincinnati, OH
+1 513 646 9390

What is the problem you are trying to solve?  Tell me what you want to
do, not how you want to do it.