-
Notifications
You must be signed in to change notification settings - Fork 19
Description
since fork is unstable in RStudio, I came to another solution, first using multi-session to create a future from main R session, and use multicore to fork in the future expr body(I've modified future to make supportsMulticore return TRUE from none-interactive R), so future_lapply still use fork
plan(multisession)
res = future({
plan(multicore)
options(mc.cores = 4)
options(future.fork.enable=T)
future_lapply(1:100, function(i){
return(length(BIG_GLOBAL_VAR))
})
})
lapply.res = value(res)
I hope such setup should makes big global variable access faster ( should be copied only once) and worker cheaper. with some simple test seems it do complete faster than directly call future_lapply with multisession
my question is : does such setup have some implicit problem ? does future/future.apply expected to work correctly in such setup ? I tried to use such setup with progressr, found that the progressor object can not work, for example
plan(multisession)
with_progress({
p = progressor(100)
future({
plan(multicore)
options(mc.cores = 4)
options(future.fork.enable=T)
future_lapply(1:100, function(i){
p("")
})
})
})is this progressr limitation, or future/future.apply should not be used in such way ? Thanks for advise