[R] Vector memory exhausted (limit reached?)

[David Winsemius]

On 10/28/19 2:17 PM, varin sacha via R-help wrote:
> Dear R-experts,
>
> My reproducible example here below is not working because of an error message : Erreur : vecteurs de mémoire épuisés (limite atteinte ?)
> My code perfectly works when n=3000 or n=5000 but as soon as n=10000 my code does not work anymore. By the way, my code takes a very long time to run.
> How can I solve my 2 problems :
> - Is there a way to make my code run much faster ?
> - Is there a way to make my code work for n=10000 ?

Improve your algorithm?

Buy more memory?

Switch to a cloud-based server?


-- 

David

>
>
> Here is my sessionInfo
>
> sessionInfo()
> R version 3.6.1 (2019-07-05)
> Platform: x86_64-apple-darwin15.6.0 (64-bit)
> Running under: macOS Sierra 10.12.6
>
> Matrix products: default
> BLAS:   /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRblas.0.dylib
> LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib
>
> Random number generation:
>   RNG:     Mersenne-Twister
>   Normal:  Inversion
>   Sample:  Rounding
>   
> locale:
> [1] fr_CH.UTF-8/fr_CH.UTF-8/fr_CH.UTF-8/C/fr_CH.UTF-8/fr_CH.UTF-8
>
> attached base packages:
> [1] splines   stats     graphics  grDevices utils     datasets  methods   base
>
> other attached packages:
>   [1] remotes_2.1.0     RobStatTM_1.0.1   fit.models_0.5-14 hbrfit_0.02       Rfit_0.23.0       RobPer_1.2.2      rgenoud_5.8-3.0
>   [8] BB_2019.10-1      quantreg_5.51     SparseM_1.77      MASS_7.3-51.4     robustbase_0.93-5
>
> loaded via a namespace (and not attached):
> [1] quadprog_1.5-7     lattice_0.20-38    grid_3.6.1         MatrixModels_0.4-1 curl_4.0           Matrix_1.2-17      tools_3.6.1
> [8] DEoptimR_1.0-8     compiler_3.6.1
>
>
> #  #  #  #  #  #  #   #  #  #  #  #  #
> install.packages( "robustbase",dependencies=TRUE )
> install.packages( "MASS" ,dependencies=TRUE )
> install.packages( "quantreg" ,dependencies=TRUE )
> install.packages( "RobPer",dependencies=TRUE  )
> install.packages("remotes") remotes::install_github("kloke/hbrfit")
> install.packages( "RobStatTM",dependencies=TRUE  )
>
> library(robustbase)
> library(MASS)
> library(quantreg)
> library(RobPer)
> library(hbrfit)
> library(RobStatTM)
> library("remotes")
>   
>
> my.experiment <- function() {
>
> n<-10000
>
> b<-runif(n, 0, 5)
>
> z <- rnorm(n, 2, 3)
>
> a <- runif(n, 0, 5)
>
>   
> y_model<- 0.1*b - 0.5 * z - a + 10
>
> y_obs <- y_model +c( rnorm(n*0.9, 0, 0.1), rnorm(n*0.1, 0, 0.5) )
>
> HBR<-hbrfit(y_obs ~ b+z+a)
>
> x<-model.matrix(~b+z+a)
> y<-y_obs
> fastTau <- FastTau(x=x, y=y)
> w<-as.vector(x %*% fastTau$beta)
>
> MSE_fastTau<-mean((w - y_model)^2)
> MSE_HBR<-mean((HBR$fitted.values - y_model)^2)
>
> return( c(MSE_fastTau,MSE_HBR) )
>
> }
>
> my.data = t(replicate( 10, my.experiment() ))
> colnames(my.data) <- c("MSE_fastTau","MSE_HBR")
> summary(my.data)
>
> #  #  #  #  #  #  #   #  #  #  #  #  #
>
> ______________________________________________
> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.