[R] Use BatchJobs package to submit Rmpi job

Feng Li fli at teraproc.com
Thu Mar 19 20:24:55 CET 2015

BatchJobs package is popular to submit R batch jobs on various job scheduler. Now we are facing the question is to submit a Rmpi jobs. So we want to know how we can leverage BatchJobs to do so. After a bit investigation/enhancement, we tested below approach on OpenLava job scheduler. we want to get some inputs from the group to see how this makes sense to you. thanks,

1. R script that submit a Rmpi job using BatchJobs:
conf = BatchJobs:::getBatchJobsConf()
conf$cluster.functions = makeClusterFunctionsOpenLava("/home/clusteradmin/rmpi.tmpl")      <-- specify job template file that includes "mpirun" to start R job.
reg = makeRegistry(id = "RMPIJobsExample")
f = function(x) {                                                <-- RMPI algorithm function. This is the normal Rmpi R script.
                mpi.remote.exec(paste("I am",mpi.comm.rank(),"of",mpi.comm.size()))
batchMap(reg, f, 1)                                       <-- specify 1 Rmpi job
submitJobs(reg, np=2)                                  <-- specify number of processors to use

2. the job template: rmpi.tmpl
                                            #BSUB -q mpi_job_queue
                                            /opt/openlava/bin/openmpi-mpirun -np 1 R CMD BATCH "<%= rscript %>"

Please let us know your comments,

Feng Li
Teraproc Inc.

	[[alternative HTML version deleted]]

More information about the R-help mailing list