[R] Rsge: recursive parallelization

Peter Danenberg pcd at roxygen.org
Mon Apr 12 18:11:00 CEST 2010


Quoth Martin Morgan on Boomtime, the 29th of Discord:
> It seems like it would be hard to think about the tasks that are
> being executed, how many processes there are, how load balancing
> works, etc.

A flat representation of our nested data is, alas, too large to
contain in memory; and dynamically re-establishing context for an
arbitrary subset of it is expensive.

Parallel recursion isn't unheard of [1]; and I was hoping SGE would
worry about the load balancing, etc. for me.

Barring that, though, we might have to get creatively iterative.


Footnotes: 
[1]  http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=44263



More information about the R-help mailing list