[R] snow/Rmpi without MPI.spawn?

Martin Morgan mtmorgan at fhcrc.org
Thu Sep 4 02:07:36 CEST 2014

On 09/03/2014 03:25 PM, Jim Leek wrote:
> I'm a programmer at a high-performance computing center.  I'm not very
> familiar with R, but I have used MPI from C, C++, and Python.  I have to run
> an R code provided by a guy who knows R, but not MPI.  So, this fellow used
> the R snow library to parallelize his R code (theoretically, I'm not
> actually sure what he did.)  I need to get this code running on our
> machines.
> However, Rmpi and snow seem to require mpi spawn, which our computing center
> doesn't support.  I even tried building Rmpi with MPICH1 instead of 2,
> because Rmpi has that option, but it still tries to use spawn.
> I can launch plenty of processes, but I have to launch them all at once at
> the beginning. Is there any way to convince Rmpi to just use those processes
> rather than trying to spawn its own?  I haven't found any documentation on
> this issue, although I would've thought it would be quite common.

This script

# salloc -n 12 orterun -n 1 R -f spawn.R
## Recent Rmpi bug -- should be mpi.universe.size()
nWorkers <- mpi.universe.size()
mpiRank <- function(i)
   c(i=i, rank=mpi.comm.rank())
mpi.parSapply(seq_len(2*nWorkers), mpiRank)

can be run like the comment suggests

    salloc -n 12 orterun -n 1 R -f spawn.R

uses slurm (or whatever job manager) to allocate resources for 12 tasks and 
spawn within that allocation. Maybe that's 'good enough' -- spawning within the 
assigned allocation? Likely this requires minimal modification of the current code.

More extensive is to revise the manager/worker-style code to something more like 
single instruction, multiple data

## salloc -n 4 orterun R --slave -f simd.R
sink("/dev/null") # don't capture output -- more care needed here

.comm = 0L

## shared `work', here just determine rank and host
work = c(rank=mpi.comm.rank(.comm),
          host=system("hostname", intern=TRUE))

if (mpi.comm.rank(.comm) == 0) {
     ## manager
     nWorkers = mpi.comm.size(.comm)
     res = list(nWorkers)
     for (i in seq_len(nWorkers - 1L)) {
         res[[i]] <- mpi.recv.Robj(mpi.any.source(), TAGS$FROM_WORKER,
     res[[nWorkers]] = work
     sink() # start capturing output
     print(do.call(rbind, res))
} else {
     ## worker
     mpi.send.Robj(work, 0L, TAGS$FROM_WORKER, comm=.comm)

but this likely requires some serious code revision; if going this route then 
http://r-pbd.org/ might be helpful (and from a similar HPC environment).

It's always worth asking whether the code is written to be efficient in R -- a 
typical 'mistake' is to write R-level explicit 'for' loops that 
"copy-and-append" results, along the lines of

    len <- 100000
    result <- NULL
    for (i in seq_len(len))
        ## some complicated calculation, then...
        result <- c(result, sqrt(i))

whereas it's much better to "pre-allocate and fill"

     result <- integer(len)
     for (i in seq_len(len))
         result[[i]] = sqrt(i)


     lapply(seq_len(len), sqrt)

and very much better still to 'vectorize'

     result <- sqrt(seq_len(len))

(timing for me are about 1 minute for "copy-and-append", .2 s for "pre-allocate 
and fill", and .002s for "vectorize").

Pushing back on the guy providing the code (grep for "for" loops, and look for 
that copy-and-append pattern) might save you from having to use parallel 
evaluation at all.


> Thanks,
> Jim
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

Computational Biology / Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N.
PO Box 19024 Seattle, WA 98109

Location: Arnold Building M1 B861
Phone: (206) 667-2793

More information about the R-help mailing list