[R] foreach not recognizing functions in memory

[Adrienne Wootten]

Roger,

I think I know why you are having a problem.  Your foreach is using
%dopar% and judging from your output you have active parallel
processing.  When running something in paralell, the master is the
processor used by the Rgui, and the slaves are in the background.  My
guess is the the function you made is the memory of the master, but
not in the memory for the slaves.  You need to pass the function
"TestFun" to your slaves.  If the slave don't know about it, using
foreach with %dopar% will fail since the function doesn't exist on the
slave processors.  Without knowing what package you used to do your
parallel processing I can't tell you how to pass the function you
created to your slave processors.

Try running the foreach command with %do% in place of %dopar%.  If the
command works, then it is probably a problem that function isn't in
memory on the slave processors.

Adrienne

On Tue, Jul 12, 2011 at 5:28 PM, Bos, Roger <roger.bos at rothschild.com> wrote:
> All,
>
> I am not understanding the scoping used in foreach when it is used
> inside a function.  I keep getting "could not find function" errors for
> functions that are in memory when I try to use foreach within a function
> call.  I have a simple example below.  "testFun" is in memory and works
> when called by foreach directly, but when I place foreach in a function
> called "helper" and then call helper, foreach cannot find "testFun".
> Can anyone provide a fix?  I hope its just a simple mistake on my part
> and not a limitation of foreach.
>
>> source(.trPaths[5], echo=TRUE, max.deparse.length=150)
>
>> "testFun" <- function(m) {
> +     out <- m*m
> + }
>
>> out <- foreach(m=1:10, .combine=rbind) %dopar% testFun(m)
>
>> "helper" <- function(i) {
> +
> +     out <- foreach(m=1:i, .combine=rbind) %dopar% testFun(m)
> + }
>
>> out <- helper(10)
> Error in testFun(m) : task 1 failed - "could not find function
> "testFun""
>>
>
> Thanks,
>
> Roger
>
> ***************************************************************
>
> This message is for the named person's use only. It may\...{{dropped:20}}
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



-- 
Adrienne Wootten
Graduate Research Assistant
State Climate Office of North Carolina
Department of Marine, Earth and Atmospheric Sciences
North Carolina State University