[Rd] Rmpi, openMPI editions.

[Paul Johnson]

Greetings.

I see a warning message while compiling OpenMPI and would appreciate
it if you tell me what it means.

This warning happens with any OpenMPI > 1.6.5.  Even before starting a
cluster, just "sessionInfo" triggers this warning.

I'm pasting in the message from R-3.3.2 (this is MRO).

Do the R parallel package cluster functions violate the warnings described here?

> library("Rmpi")
> sessionInfo()
--------------------------------------------------------------------------
An MPI process has executed an operation involving a call to the
"fork()" system call to create a child process.  Open MPI is currently
operating in a condition that could result in memory corruption or
other system errors; your MPI job may hang, crash, or produce silent
data corruption.  The use of fork() (or system() or other calls that
create child processes) is strongly discouraged.

The process that invoked fork was:

  Local host:          n401 (PID 114242)
  MPI_COMM_WORLD rank: 0

If you are *absolutely sure* that your application will successfully
and correctly survive a call to fork(), you may disable this warning
by setting the mpi_warn_on_fork MCA parameter to 0.
--------------------------------------------------------------------------
R version 3.3.2 (2016-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux Server release 6.4 (Santiago)

locale:
[1] C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] Rmpi_0.6-6           RevoUtilsMath_10.0.0

loaded via a namespace (and not attached):
[1] RevoUtils_10.0.2 parallel_3.3.2   tools_3.3.2
>

What I think this means is that we need to never run any multicore
functions and we need to be very careful that MKL or such does not
launch threads.  Is that right? Is it worse than that?

Why am I chasing this one today?

I've been on an adventure compiling R in a RedHat 6 cluster again. The
cluster admins here like the Microsoft R, and they had both 3.3 and
3.4 installed. However, we found some packaging flaws in 3.4 and so
that MRO was removed. I'm interested in building R-3.4, but it is a
pretty big job on the old RedHat.  I want to get this correct.

I've run into the problem I'd forgotten about OpenMPI. If OpenMPI >=
2, then Rmpi will compile, but jobs hang with "stopCluster".  With
OpenMPI-1.6.5, we get a clean build and no warnings, and clusters do
start and stop cleanly.  With newer version 1 editions of OpenMPI,
such as 1.10 or 1.12 (I suspect any versions (> 1.6.5), the Rmpi
generates an intimidating warning, but the cluster will stop when
asked.


-- 
Paul E. Johnson   http://pj.freefaculty.org
Director, Center for Research Methods and Data Analysis http://crmda.ku.edu

To write to me directly, please address me at pauljohn at ku.edu.