[R] looping through 3D array

[arun]

Hi Alex,

Not sure if this is what you wanted.
length(res) #from the previous 'example' using ##indx <- combn(dim(results)[1],2)
#[1] 45

mat1 <- matrix(0,10,10)
 mat1[lower.tri(mat1)] <- res
 mat1[upper.tri(mat1)] <- res

A.K.









On Saturday, January 11, 2014 12:22 AM, alex padron <alexpadron1082 at gmail.com> wrote:

Thanks for these. 

I am using 

library(multicore)
res <- unlist(mclapply(seq_len(ncol(indx)), function(i) { x1 <- indx[,i];emd2d(results2[x1[1],,],results2[x1[2],,])}, mc.cores=48))

How can I write the output of res as a matrix that has rows and columns for each of the calculated distances? So if there is a total of say 100 distances calculated the output matrix should have 10 rows and 10 columns. 


-Alex


On Thu, Jan 9, 2014 at 2:05 PM, arun <smartpink111 at yahoo.com> wrote:

Also,
>Check these links:
>
>http://stackoverflow.com/questions/1358003/tricks-to-manage-the-available-memory-in-an-r-session
>
>http://www.bigmemory.org/
>A.K.
>
>
>
>On Thursday, January 9, 2014 4:58 PM, arun <smartpink111 at yahoo.com> wrote:
>Hi Alex,
>
>Regarding the code:
>
>indx <- combn(dim(results)[1],2) #shouldn't be hard
>
>dim(results)[1]
>#[1] 10
>
>seq(dim(results)[1])
># [1]  1  2  3  4  5  6  7  8  9 10
> combn(seq(dim(results)[1]),2) #gives the same result
>
>l1 <- lapply(seq_len(ncol(indx)),function(i) {x1 <- indx[,i]}) ##check the results of this one
>#Here, I am looping through the columns of the 'indx' matrix. i.e. Each element of 'l1' is a vector of length 2.
>
>l1[[1]]
>#[1] 1 2
> is.vector(l1[[1]])
>#[1] TRUE
>
>
>#The idea is to apply the function emd2d() on each of the vector elements, that act as indx for the array "results"
>#For example:
> lapply(l1,function(x1) results[x1[1],,])
> lapply(l1,function(x1) results[x1[2],,])
>#When we combine that:
>
> lapply(l1,function(x1) emd2d(results[x1[1],,],results[x1[2],,]))
>sapply(l1,function(x1) emd2d(results[x1[1],,],results[x1[2],,])) #gets the result as vector.
>
>For the second part, you are using combn() on a vector of 13,000.  If the memory holds for getting the results of "indx".  Then, you could split the indx into smaller matrices and try to run.
>Eg.
>indx1 <- indx[,1:20]  #in your big dataset, change accordingly
>indx2 <- indx[,21:35]
>
>etc..
>A.K.
>
>
>
>
>
>
>
>
>
>On Thursday, January 9, 2014 4:34 PM, alex padron <alexpadron1082 at gmail.com> wrote:
>
>I don't fully understand your code because it is beyond me. Could you explain it a bit and also when I run it on my real data set R crashes because it runs out of memory. Any way around this?
>On Jan 9, 2014 1:15 PM, "alex padron" <alexpadron1082 at gmail.com> wrote:
>
>You are awesome. 
>>
>>
>>-Alex
>>
>>
>>On Thu, Jan 9, 2014 at 1:10 PM, arun <smartpink111 at yahoo.com> wrote:
>>
>>Hi,
>>>No problem.
>>>You can use ?lower.tri() or ?upper.tri()
>>>
>>>res[lower.tri(res)]
>>>res[lower.tri(res,diag=TRUE)]
>>>#Other way would be to use:
>>>?combn
>>>indx <- combn(dim(results)[1],m=2)
>>>
>>>
>>>res2 <- sapply(seq_len(ncol(indx)),function(i) {x1 <- indx[,i]; emd2d(results[x1[1],,],results[x1[2],,]) })
>>> identical(res[lower.tri(res)], res2)
>>>#[1] TRUE
>>>A.K.
>>>
>>>
>>>
>>>
>>>
>>>On Thursday, January 9, 2014 4:03 PM, alex padron <alexpadron1082 at gmail.com> wrote:
>>>
>>>Thanks. This works. I just noticed that half of the matrix repeats. For example res[1,2] is the same as res[2,1]. any way to get half of the matrix output (notice the diagonal 0 across the output matrix)?
>>>
>>>
>>>
>>>-Alex
>>>
>>>
>>>On Thu, Jan 9, 2014 at 12:57 PM, arun <smartpink111 at yahoo.com> wrote:
>>>
>>>#or
>>>>you can use ?expand.grid() and then loop over:
>>>>indx <- expand.grid(rep(list(seq(dim(results)[1])),2))
>>>>res1 <- matrix(sapply(seq_len(nrow(indx)),function(i) {x1 <- indx[i,]; emd2d(results[x1[,1],,],results[x1[,2],,]) }),ncol=10)
>>>>identical(res,res1)
>>>>#[1] TRUE
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>On Thursday, January 9, 2014 3:46 PM, arun <smartpink111 at yahoo.com> wrote:
>>>>Hi,
>>>>Try:
>>>>library(emdist)
>>>>
>>>>set.seed(435)
>>>>results<- array(sample(1:400,120,replace=TRUE),dim=c(10,3,4))
>>>>res <- sapply(seq(dim(results)[1]),function(i) {x1 <- results[i,,]; x2 <- results; sapply(seq(dim(x2)[1]),function(i) emd2d(x1,x2[i,,]))})
>>>>dim(res)
>>>>#[1] 10 10
>>>>A.K.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>On Thursday, January 9, 2014 3:25 PM, alex padron <alexpadron1082 at gmail.com> wrote:
>>>>
>>>>I'll try to be clearer. in your example we have: results<- array(1:120,dim=c(10,3,4)) 
>>>>
>>>>I want to do the following: compare results[1,,] with every matrix inside results. I then want to jump to results[2,,] and compare it to all of the other 10 matrices inside results and so on. so emd2d from the emdist package outputs a single value when comparing matrices and since your example has 10 matrices who are all being compared, the output should be 100 values. 
>>>>
>>>>Does that make sense?
>>>>
>>>>
>>>>-Alex
>>>>
>>>
>>
>