[R] progressive slowdown during script execution? [Broadcast ]

[Liaw, Andy]

You should try to figure out how large x and y need to be, and allocate them
before the computation.  "Growing" objects like that in R is very, very,
very bad, as you've found out.

Andy

From: Tim Alcon
> 
> I'm an R novice, so I hope my question is a valid one.  I'm trying to 
> run the following script in the current version of R.
> 
> for (i in 1:1640){for (j in (i+1):1641){
> if (i == 1 && j == 2){x <- 
> cor(sage[i,],sage[j,],method="spearman"); y 
> <- cor(frie[i,],frie[j,],method="spearman")}
> if (i != 1 || j != 2){x <- 
> c(x,cor(sage[i,],sage[j,],method="spearman")); y <- 
> c(y,cor(frie[i,],frie[j,],method="spearman"))}}}
> 
> It basically just finds all pairwise correlations of the rows in a 
> matrix for each of two matrices and stores the results for 
> each matrix 
> in a vector.  The problem I seem to be running into is that 
> it seems to 
> slow way down during execution somehow.  When I first tried 
> running it I 
> stopped execution to see how fast it was running, before trying to 
> compute the whole job (the two matrices each have 1641 rows). 
>  Based on 
> what I saw, I figured it would easily finish overnight.  
> Instead, it was 
> still running almost 24 hours later.  To quantify this a 
> little better I 
> checked it after running for 5 minutes, at which point it had added 
> 79120 correlations to each of the x and y vectors.  Since 
> there should 
> be a total of (1641*1640)/2 = 1345620 pairwise correlations in each 
> vector when it finishes running, I worked out that it should take 
> (1345620/79120)*5 = 85 minutes to run the whole job.  However, when I 
> checked it after running for 2 hours, it had added only 341870 
> correlations to each vector.
> 
> Any ideas what I'm doing wrong, or why it would run more slowly the 
> longer it runs?  Thanks for any help or advice.
> 
> Tim
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! 
> http://www.R-project.org/posting-guide.html
> 
>