[Rd] Questions/suggestions about new staged installation

[Tomas Kalibera]

On 4/25/19 3:11 AM, Pages, Herve wrote:
> Hi,
>
> I was playing around with inotifywait (great tool!) to see the new
> staged installation of source packages in action. In one terminal I'm
> monitoring the create/delete/move events of the installation library with:
>
>     inotifywait -m --timefmt '%F %T' --format '%T -- %w %e %f' -e create
> -e delete -e move path/to/R/library/
>
> While in another terminal I install CRAN package abc with:
>
>     R CMD INSTALL abc_2.1.tar.gz
>
> All deps are already installed.
>
> ## With R 3.5
>
> When installing abc the first time with R 3.5 inotifywait reports the
> following events:
>
> 2019-04-24 16:57:42 -- /home/hpages/R/R-3.5.r75051/library/ CREATE,ISDIR
> 00LOCK-abc
> 2019-04-24 16:57:42 -- /home/hpages/R/R-3.5.r75051/library/ CREATE,ISDIR abc
> 2019-04-24 16:57:45 -- /home/hpages/R/R-3.5.r75051/library/ DELETE,ISDIR
> 00LOCK-
>
> Then on subsequent times:
>
> 2019-04-24 16:58:14 -- /home/hpages/R/R-3.5.r75051/library/ CREATE,ISDIR
> 00LOCK-abc
> 2019-04-24 16:58:14 -- /home/hpages/R/R-3.5.r75051/library/
> MOVED_FROM,ISDIR abc
> 2019-04-24 16:58:14 -- /home/hpages/R/R-3.5.r75051/library/ CREATE,ISDIR abc
> 2019-04-24 16:58:17 -- /home/hpages/R/R-3.5.r75051/library/ DELETE,ISDIR
> 00LOCK-abc
>
> IIUC the additional 'MOVED_FROM,ISDIR abc'  event on subsequent installs
> is triggered by the backing up of the earlier installation in case the
> new installation fails.
>
> ## With R 3.6
>
> First installation:
>
> 2019-04-24 17:09:04 -- /home/hpages/R/R-3.6.r76385/library/ CREATE,ISDIR
> 00LOCK-abc
> 2019-04-24 17:09:04 -- /home/hpages/R/R-3.6.r76385/library/ CREATE,ISDIR abc
> 2019-04-24 17:09:08 -- /home/hpages/R/R-3.6.r76385/library/
> MOVED_TO,ISDIR abc
> 2019-04-24 17:09:09 -- /home/hpages/R/R-3.6.r76385/library/ DELETE,ISDIR
> 00LOCK-abc
>
> Subsequent installations:
>
> 2019-04-24 17:09:59 -- /home/hpages/R/R-3.6.r76385/library/ CREATE,ISDIR
> 00LOCK-abc
> 2019-04-24 17:09:59 -- /home/hpages/R/R-3.6.r76385/library/
> MOVED_FROM,ISDIR abc
> 2019-04-24 17:09:59 -- /home/hpages/R/R-3.6.r76385/library/ CREATE,ISDIR abc
> 2019-04-24 17:10:02 -- /home/hpages/R/R-3.6.r76385/library/
> MOVED_TO,ISDIR abc
> 2019-04-24 17:10:03 -- /home/hpages/R/R-3.6.r76385/library/ DELETE,ISDIR
> 00LOCK-abc
>
> IIUC the new 'MOVED_TO,ISDIR abc' event is triggered by the package
> installation folder being moved from the temporary location to the final
> location.
>
> However I wonder about the CREATE,ISDIR abc event. It looks like even
> for a staged installation tools:::.install_packages() still creates the
> empty abc dir. This should no longer be necessary because this empty
> folder is replaced later by the 'MOVED_TO,ISDIR abc' event. Am I missing
> somethin

Yes, the final directory location gets created before the installation. 
I did this to minimize the added cognitive complexity to the code. I 
could explore if this could be changed, if the installation could work 
without that directory (which would mean implementing and testing on all 
CRAN/BIOC, installing in different modes, so certainly some effort).  Is 
the current implementation causing any problem?

> Another thing: the times indicate that the 'MOVED_TO,ISDIR abc' event
> (move from temp to final location) happens about 3-4 seconds after the
> 'MOVED_FROM,ISDIR abc' event (backing up). In this interval of time, the
> abc package is missing. Wouldn't it be safer to leave the earlier
> installation where it is and to replace it when the new installation
> folder is moved from the temporary location to the final location? Said
> otherwise, the backing up step no longer seems necessary with staged
> installs. So inotifywait would report something like this:
>
> 2019-04-24 17:09:59 -- /home/hpages/R/R-3.6.r76385/library/ CREATE,ISDIR
> 00LOCK-abc
> 2019-04-24 17:10:02 -- /home/hpages/R/R-3.6.r76385/library/
> MOVED_TO,ISDIR abc
> 2019-04-24 17:10:03 -- /home/hpages/R/R-3.6.r76385/library/ DELETE,ISDIR
> 00LOCK-abc
>
> That is, only 3 events. On first and subsequent installation.

I don't think that would be possible. The package is test-loaded also 
from the final installation directory, and if that fails, restored from 
a back-up. This test-loading is useful as it can detect some problems 
with hard-coded directories possibly not detected before on test-loading 
from the temporary installation directory.

Best,
Tomas

>
> Cheers,
>
> H.
>
>
>   > sessionInfo()
> R version 3.6.0 beta (2019-04-12 r76385)
> Platform: x86_64-pc-linux-gnu (64-bit)
> Running under: Ubuntu 16.04.5 LTS
>
> Matrix products: default
> BLAS:   /home/hpages/R/R-3.6.r76385/lib/libRblas.so
> LAPACK: /home/hpages/R/R-3.6.r76385/lib/libRlapack.so
>
> locale:
>    [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
>    [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
>    [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
>    [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
>    [9] LC_ADDRESS=C               LC_TELEPHONE=C
> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods base
>
> loaded via a namespace (and not attached):
> [1] compiler_3.6.0
>
>