[R] Re: R-1.1.0 is released : GUI

[Douglas Bates]

Peter Dalgaard BSA <p.dalgaard at biostat.ku.dk> writes:

> "Yves Gauvreau" <cyg at sympatico.ca> writes:
> 
> 
> > How much time and effort would
> > be required to improve R capabilities by a factor of 2 say? I think it would
> > be lots and lots. How much work would it be to give R parallelism I think it
> > could be a lot less. All together I think asking the core team to improve
> > the performance of R knowing that probably only very few would really
> > benefit is asking to much. I would be curious to know what it would involve
> > to provide R with parallelism enlisting PVM or other similar package?
> 
> There are many levels of parallelism, not all of which are equally
> easy to exploit. At the coarsest level, each machine does essentially
> its own thing in a separate process. This is very easy to implement
> and quite useful for e.g. simulations. For a somewhat more elaborate
> version of this there is the Condor project at Wisconsin which allows
> processes to be moved around between different CPUs - R has been tested
> on this but I don't know how far they've got. 

The Condor project to which Peter refers is described at
                    http://www.cs.wisc.edu/condor/
It allows groups of computers to be used as a batch processing pool.  
The scheduling software is relatively sophisticated.  The user can
specify the resources required for the job, such as physical memory
requirements, and queue up a series of jobs that can be run anywhere
in the pool.

Condor supports different "universe"s.  The simplest, called
"vanilla", uses the regularly compiled version of the program but does
not allow checkpointing of the program.  If your program, say a long
simulation, has been running on a workstation for a day or two then
someone starts using that workstation interactively, the scheduler
must either suspend your job with the hope of later restarting it on
the same machine or throw away the accumulated work and restart the
job elsewhere.

By relinking the program against special versions of system libraries
one can run the program in condor's "standard" universe.  That allows
checkpointing and migration of a partially job from one machine to
another. 

Using R in the vanilla universe is trivial.  I have done some
experiments trying to link R-1.1.0 for the standard universe and met
with some success.  I did have a few problems with the regular
expressions library and have not had time to continue the experiment.

Condor is (apparently) freely available but not Open Source or GPL'd.
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._