[R] makeCluster() hangs infinitely

[Henrik Bengtsson]

On Mon, Sep 17, 2018 at 12:56 PM Zhihao Huang <zhhhwang using umich.edu> wrote:
>
> Hi Henrik,
>
> Thanks for the suggestions! I tried your approach, and obtained the following output, which is pretty similar to the previous ones.
>
> > cl <- future::makeClusterPSOCK(1, outfile = NULL, verbose = TRUE)
> Workers: [n = 1] ‘localhost’
> Base port: 11214
> Creating node 1 of 1 ...
> - setting up node
> Starting worker #1 on ‘localhost’: '/Library/Frameworks/R.framework/Resources/bin/Rscript' --default-packages=datasets,utils,grDevices,graphics,stats,methods -e 'parallel:::.slaveRSOCK()' MASTER=localhost PORT=11214 OUT= TIMEOUT=2592000 XDR=TRUE
> Waiting for worker #1 on ‘localhost’ to connect back
> starting worker pid=13731 on localhost:11214 at 15:48:41.991
>
> I guess this is a connection problem. I am not sure what these numbers mean. Do you have any further idea on this? I very much appreciate it!

Yes, it looks similar with the important difference of displaying:

"starting worker pid=13731 on localhost:11214 at 15:48:41.991"

That tells us that the background worker (separate R session running
parallel:::.slaveRSOCK()) was successfully launched, which is good.

BTW, you should see something similar with:

    cl <- parallel::makeCluster(1, outfile = NULL)

which helps others help you (in case they say "oh, it might be a
problem with the future package - as the maintainer").

Yes, it looks like a "connection problem" - this could be a firewall
issue or similar.  I'm not on macOS, so I cannot help you there, but
maybe others can pitch in.

/Henrik

>
> Thanks,
> Zhihao
> --
> Zhihao (Daniel) Huang
> Graduate Student
> Department of Statistics,
> University of Michigan, Ann Arbor
> Email: zhhhwang using umich.edu
>
>
>
> On Mon, Sep 17, 2018 at 12:38 AM Henrik Bengtsson <henrik.bengtsson using gmail.com> wrote:
>>
>> Hi,
>>
>> did you see my answer on StackOverflow? Specifically, if you set
>> argument 'outfile = NULL' to either of those two functions, you'll get
>> a little bit more information that *might* provide some clues.
>>
>> /Henrik
>>
>>
>> On Sun, Sep 16, 2018 at 5:38 PM Zhihao Huang <zhhhwang using umich.edu> wrote:
>> >
>> > Hi all,
>> >
>> > The function makeCluster() of parallel does not work on my laptop. It hangs
>> > infinitely.
>> >
>> > *1. Problem Summary:*
>> >
>> > > # Loading parallel packages
>> >
>> > > library(parallel)
>> >
>> > > cl <- makeCluster(2) # It hangs at this line of code.
>> > It hangs at the second line of the code.
>> >
>> > *2. Potential Reason*
>> > I also tried to see the details of what it does internally by using the
>> > following code.
>> >
>> > > library(future)
>> >
>> > > cl <- future::makeClusterPSOCK(1L, verbose = TRUE) # It hangs at this
>> > line of code.
>> > And it returns the following descriptions and hangs.
>> >
>> > *Workers: [n = 1] ‘localhost’*
>> >
>> > *Base port: 11214*
>> >
>> > *Creating node 1 of 1 ...*
>> >
>> > *- setting up node*
>> >
>> > *Starting worker #1 on ‘localhost’:
>> > '/Library/Frameworks/R.framework/Resources/bin/Rscript'
>> > --default-packages=datasets,utils,grDevices,graphics,stats,methods -e
>> > 'parallel:::.slaveRSOCK()' MASTER=localhost PORT=11214 OUT=/dev/null
>> > TIMEOUT=2592000 XDR=TRUE*
>> >
>> > *Waiting for worker #1 on ‘localhost’ to connect back*
>> > So the problem is that the "worker #1 on 'local host'" never connects back,
>> > and that's why it hangs forever. I have no idea what causes this.
>> >
>> > *3. my sessionInfo():*
>> >
>> > R version 3.5.1 (2018-07-02)
>> >
>> > Platform: x86_64-apple-darwin15.6.0 (64-bit)
>> >
>> > Running under: macOS High Sierra 10.13.6
>> >
>> >
>> > Matrix products: default
>> >
>> > BLAS:
>> > /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
>> >
>> > LAPACK:
>> > /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
>> >
>> >
>> > locale:
>> >
>> > [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
>> >
>> >
>> > attached base packages:
>> >
>> > [1] stats     graphics  grDevices utils     datasets  methods   base
>> >
>> >
>> > loaded via a namespace (and not attached):
>> >
>> > [1] compiler_3.5.1
>> >
>> > I spent hours searching for the solutions but failed. It looks like some
>> > other people met similar problem here
>> > <http://r.789695.n4.nabble.com/makeCluster-hangs-td4748238.html>. Also, I
>> > posted this question online here
>> > <https://stackoverflow.com/questions/52264460/r-parallel-makecluster-hangs-infinitely-on-mac/52284709#52284709>
>> > a
>> > week ago.
>> >
>> > Any suggestion would be appreciated. Thanks a lot!
>> >
>> > Thanks,
>> > Zhihao
>> > --
>> > Zhihao (Daniel) Huang
>> > Graduate Student
>> > Department of Statistics,
>> > University of Michigan, Ann Arbor
>> > Email: zhhhwang using umich.edu
>> >
>> > --
>> > 黄 之昊
>> > Zhihao Huang
>> >
>> > Graduate Student
>> > Department of Statistics,
>> > University of Michigan, Ann Arbor
>> > Email: zhhhwang using umich.edu
>> >
>> >         [[alternative HTML version deleted]]
>> >
>> > ______________________________________________
>> > R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> > https://stat.ethz.ch/mailman/listinfo/r-help
>> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> > and provide commented, minimal, self-contained, reproducible code.