[R] Suggestions for poor man's parallel processing

[A.J. Rossini]

Yes, RPVM is annoying to startup, however, there are higher level
functions that are coming (parallel apply), that might help justify
the effort.  Better yet, they are wrappers that hopefully will be
sitting on top of RPVM, or RMPI (via the LAM MPI implementaation), and
incorporate a prarllel pRNG for assurance of minimum quality.
(RSPRNG, a wrapper to the SPRNG library)

Coming soon (no set release date planned yet) to a CRAN archive near
you (the package is called SNOW, by Luke Tierney).

>>>>> "david" == David Kane <David Kane" <a296180 at mica.fmr.com> writes:

    david> Almost all of the heavy crunching I do in R is like:
    >> for(i in long.list){
    david> + do.something(i)
    david> + }
    >> collect.results()

    david> Since all the invocations of do.something are independent of one another, there
    david> is no reason that I can't run them in parallel. Since my machine has four
    david> processors, a natural way to do this is to divide up long.list into 4 pieces
    david> and then start 4 jobs, each of which would process 1/4 of the items. I could
    david> then wait for the four jobs to finish (waiting for tag files and the like),
    david> collect the results, and go on my happy way. I might do this all within R
    david> (using system calls to fork off other R processes?) or by using Perl as a
    david> wrapper.

    david> But surely there are others that have faced and solved this problem already! I
    david> do not *think* that I want to go into the details of RPVM since my needs are so
    david> limitted. Does anyone have any advice for me? Various postings to R-help have
    david> hinted at ideas, but I couldn't find anything definitive. I will summarize for
    david> the list.

    david> To the extent that it matters:

    >> R.version
    david>          _                   
    david> platform sparc-sun-solaris2.6
    david> arch     sparc               
    david> os       solaris2.6          
    david> system   sparc, solaris2.6   
    david> status                       
    david> major    1                   
    david> minor    5.0                 
    david> year     2002                
    david> month    04                  
    david> day      29                  
    david> language R                   


    david> Regards,

    david> Dave Kane
    david> -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
    david> r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
    david> Send "info", "help", or "[un]subscribe"
    david> (in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
    david> _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._


-- 
A.J. Rossini				Rsrch. Asst. Prof. of Biostatistics
U. of Washington Biostatistics		rossini at u.washington.edu	
FHCRC/SCHARP/HIV Vaccine Trials Net	rossini at scharp.org
-------------- http://software.biostat.washington.edu/ ----------------
FHCRC: M-W: 206-667-7025 (fax=4812)|Voicemail is pretty sketchy/use Email
UW:   Th: 206-543-1044 (fax=3286)|Change last 4 digits of phone to FAX
(my friday location is usually completely unpredictable.)


-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._