[R] Rsge: recursive parallelization
Peter Danenberg
pcd at roxygen.org
Fri Apr 9 17:52:21 CEST 2010
In principle, I'd like to be able to do something like this:
sge.parLapply(seq(10), function(x) parLapply(seq(x), function(x) x^2))
In practice, however, I have to resort to acrobatics like this:
sge.options(sge.remove.files=FALSE)
sge.options(sge.qsub.options='-cwd -V')
sge.parLapply(seq(10),
function(x) {
sge.options(sge.save.global=TRUE)
sge.options(sge.remove.files=FALSE)
sge.parLapply(seq(x),
function(x) x^2,
cluster=TRUE,
debug=FALSE,
trace=FALSE,
file.prefix='Rsge_data',
global.savelist=NULL,
packages=NULL)
},
function.savelist=c('sge.parLapply', 'sge.parParApply',
'sge.options', 'sge.taskPrep'),
global.savelist=c('sge.parParApply', 'sge.globalPrep',
'global.savelist', 'sge.taskPrep', 'sge.checkNotNow',
'sge.get.jobid', 'sge.get.result', 'docall',
'enquote'),
packages=NULL)
and I still get bizarre behavior: half of the results will be NULL,
for instance; the other half, incomplete.
Would non-trivial changes to Rsge be required to make something like
this possible?
More information about the R-help
mailing list