[R] slowness when I use a list comprehension

[@vi@e@gross m@iii@g oii gm@ii@com]

Laurent,

Thank you for introducing me to a package I did not know existed as I use features like list comprehension in python all the time and could see using it in R now that I know it is available.

As to why you see your example as slow, I see you used a fairly complex and nested expression and wonder if it was a better way to go. As you are dealing with an interpreter doing delayed evaluation, I can imagine reasons it can be slow. But note the package comprehenr may not be designed to be more efficient than loops or of the more built-in functional methods that can be faster. The package is there perhaps more as a compatibility helper that allows you to write closer to the python style and perhaps re-shapes what you wrote into a set of instructions in more native R.

Just for comparison, in python, things like comprehensions for list or dictionaries or tuples often are syntactic sugar and the interpreter may simply rewrite them more like the first program you typed and evaluates that. The comprehensions are more designed for users who can think another way and write things more compactly as one-liners. Depending on implementations, they may be faster or slower on some examples.

I am not saying there is nothing else that is slowing it down for you. I am suggesting that using the feature as currently implemented may not be an advantage except in your thought process. It may be it could be improved, such as by replacing more functionality out of R and into compiled languages as has been done for many packages.

Avi

-----Original Message-----
From: R-help <r-help-bounces using r-project.org> On Behalf Of Laurent Rhelp
Sent: Sunday, June 16, 2024 11:28 AM
To: r-help using r-project.org
Subject: [R] slowness when I use a list comprehension

Dear RHelp-list,

    I try to use the package comprehenr to replace a for loop by a list 
comprehension.

  I wrote the code but I certainly miss something because it is very 
slower compared to the for loops. May you please explain to me why the 
list comprehension is slower in my case.

Here is my example. I do the calculation of the square difference 
between the values of two vectors vec1 and vec2, the ratio sampling 
between vec1 and vec2 is equal to ratio_sampling. I have to use only the 
500th value of the first serie before doing the difference with the 
value of the second serie (vec2).

Thank you

Best regards

Laurent

library(tictoc)
library(comprehenr)

ratio_sampling <- 500
## size of the first serie
N1 <- 70000
## size of the second serie
N2 <- 100
## mock data
set.seed(123)
vec1 <- rnorm(N1)
vec2 <- runif(N2)


## 1. with the "for" loops

## the square differences will be stored in a vector
S_diff2 <- numeric((N1-(N2-1)*ratio_sampling))
tic()
for( j in 1:length(S_diff2)){
   sum_squares <- 0
   for( i in 1:length(vec2)){
     sum_squares = sum_squares + ((vec1[(i-1)*ratio_sampling+j] - 
vec2[i])**2)
   }
   S_diff2[j] <- sum_squares
}
toc()
## 0.22 sec elapsed
which.max(S_diff2)
## 7857

## 2. with the lists comprehension
tic()
S_diff2 <- to_vec(for( j in 1:length(S_diff2)) sum(to_vec(for( i in 
1:length(vec2)) ((vec1[(i-1)*ratio_sampling+j] - vec2[i])**2))))
toc()
## 25.09 sec elapsed
which.max(S_diff2)
## 7857

______________________________________________
R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.