[R] Matrix Size

[paul s]

On 07/15/2010 05:07 AM, Peter Dalgaard wrote:
> paul s wrote:
>>
>> On 07/14/2010 06:15 PM, Peter Dalgaard wrote:
>>> A quick calculation reveals that a matrix of that size requires about
>>> 2.7 TERAbytes of storage, so I'm a bit confused as to how you might
>>> expect to fit it into 16GB of RAM...
>>>
>>> However, even with terabytes of memory, you would be running into the
>>> (current) limitation that a single vector in R can have at most 2^31-1 =
>>> ca. 2 trillion elements.
>>
>> thank you for also confirming what Douglas had written.
>
> (except that it should of course be billions not trillions; I got the
> DK/UK difference of whether a billion is a thousand or a million
> millions bass ackward there.)
>
>>> Yes, you could be doing it wrong, but what is "it"?
>>
>> we are trying to create a hedonic index: http://tinyurl.com/2fnl3jf
>
> That link is to a description of Linear Regression. I'm sure you didn't
> intend to instruct an audience of statisticioan about that....
>

definitely not instructing and do not mean to offend this audience... 
just trying to understand all of the information that is coming at me 
and keep it together...

the link i sent was my original intent. i am calculating the beta 
coefficients using a linear regression and the professor keeps 
addressing it as a hedonic index. so i guessing i am still trying to 
grasp what it is i am still doing.

>> Hmm, do you really have only 2 nonzero entries per row? If so, then a

no there will be just a few more, switched on and off, but more then two 
and less then a hundred would be a conservative guess..

>> sparse matrix representation would involve "only" 4 million entries and
>> it's beginning to look like the matrices involved in mixed-effects
>> models with crossed random factors, so that techniques like minimal

i will try to understand this.

>> fill-in Choleski decomposition would apply.

i will read about this.

cheers
paul