User Tools

Site Tools


r

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
r [2020/09/21 21:13]
root created
r [2022/12/07 16:17] (current)
root
Line 1: Line 1:
-==== R ====+=== R ===
  
-The "R" statistical package is installed on the Razor cluster.  We recommend the 3.3.version. +The "R" statistical/programming package is installed on the clusters.  The greatest number of add-on packages are installed in the 4.0.version.  Also included are older saved versions, newer 4.1.0 and 4.2.2 and slightly modified Microsoft R Open 4.0.2 [[https://web.archive.org/web/20200808203121/https://analyttica.com/microsoft-r-or-open-source-r-which-suits-you-the-best/]].
- +
-Here we have a quick benchmark for an older installed version of that should be using MKL blasIt takes 413 elapsed seconds and is actually running the slow reference blas.+
  
 <code> <code>
-$ module purge; module load gcc/4.6.mkl/13.1.R/3.1.2-mkl +module load gcc/9.3.mkl/19.0.R/4.2.2
-$ R -f bench2.r +
-R version 3.1.2 (2014-10-31) -- "Pumpkin Helmet" +
-... +
-> set.seed(123) +
-> X <- Matrix(rnorm(9e6), 3000) +
-> print(system.time(for(i in 1:25) solve(X))) +
-   user  system elapsed  +
-411.935   0.382 413.278 +
 </code> </code>
- +"gcc" module may be omitted unless you are adding packages (see below).  Substitute R/4.1.0 or MR/4.0.2 as desired for those versions.  You can also add rstudio if you want the gui.  This is best run through portal (via interactive-apps Desktop or Rstudio GUI).
- +
-A newer version of with the MKL blas properly linked is about 9 times faster in elapsed time, while still using only single core.+
 <code> <code>
-$ export MKL_NUM_THREADS=1 +module load gcc/9.3.1 mkl/19.0.5 R/4.2.2 rstudio
-$ R -f bench2.r +
-R version 3.3.1 (2016-06-21) -- "Bug in Your Hair" +
-... +
-> set.seed(123) +
-> X <- Matrix(rnorm(9e6), 3000) +
-> print(system.time(for(i in 1:25) solve(X))) +
-   user  system elapsed  +
- 47.278   0.405  48.181 +
 </code> </code>
  
-With maximum threading it is another factor of 6 times faster.+If you need a new R package, you can ask hpc-support to add it to the central installation, or you can add your own packages as demonstrated below for the package "broman"is not as prone to conflicts as Python, so most packages can be added to the central installation without issues.  Another way to create a custom installation is using [[python|conda]], which would be better if you see conflicts with standard ''R''. It will, however, install many thousands of files in your home. 
 + 
 +For this simple installation, before starting, create files ~/.Rprofile and ~/.Renviron as shown, plus  the R library directories (changing full version 4.2.2 to 4.2 in this example) and any of the other R environment variables you may want to set [[https://rstats.wtf/r-startup.html|rstats.wtf]]   
 <code> <code>
-export MKL_NUM_THREADS=16 +pinnacle-l4:feynman:cat .Rprofile 
-$ R -f bench2.r +options(repos = c(CRAN = "https://mirrors.nics.utk.edu/cran")) 
-R version 3.3.1 (2016-06-21) -- "Bug in Your Hair" +pinnacle-l4:feynman:$ cat .Renviron 
-... +R_LIBS_USER=~/R/%p/%v 
-> set.seed(123) +pinnacle-l4:feynman:$ cd;mkdir -p R 
-> X <Matrix(rnorm(9e6), 3000) +pinnacle-l4:feynman:$ mkdir -p R/x86_64-pc-linux-gnu 
-> print(system.time(for(i in 1:25) solve(X))) +pinnacle-l4:feynman:$ mkdir -p R/x86_64-pc-linux-gnu/4.2
-   user  system elapsed  +
-132.876   2.945   8.536 +
 </code> </code>
  
-The program is very simple and actual programs will not show such dramatic speedups, but it is still very worthwhile to have the right version and threading for the computer you are using.+With those files in place you can install any R package yourselfSome redundant text output below has been omitted.  
 <code> <code>
-cat bench2.r +pinnacle-l4:feynman:module load gcc/9.3.1 mkl/19.0.5 R/4.2.2 
-require(Matrix) +pinnacle-l4:feynman:$ R 
-set.seed(123) +R version 4.2.(2022-10-31-- "Innocent and Trusting" 
-X <Matrix(rnorm(9e6), 3000+Copyright (C) 2022 The R Foundation for Statistical Computing 
-print(system.time(for(i in 1:25) solve(X))+Platformx86_64-pc-linux-gnu (64-bit)
-</code>+
  
-Installation of recent R versions on Centos-6 is involved, as multiple packages need to be updated first, and the procedure is mostly undocumented, so the installation needs to be derived from the error messages of a failed installation. +[Previously saved workspace restored]
-<code> +
-yum install libXt-devel libssh2-devel openssl-devel +
-/share/apps/zlib/zlib-1.25 already installed +
-cd temp-install-directory +
-wget http://tukaani.org/xz/xz-5.2.2.tar.gz +
-tar zxf xz-5*gz +
-cd xz-5.2.2 +
-./configure --prefix=/share/apps/xz/5.2.2 +
-make && make install+
  
-cd .+> install.packages("broman") 
-wget https://sourceforge.net/projects/pcre/files/latest/download?source=files +Installing package into ‘/scrfs/storage/feynman/home/R/x86_64-pc-linux-gnu/4.0’ 
-mv down*es pcre-8.39.tar.bz2 +(as ‘lib’ is unspecified) 
-tar jxf pcre*2 +trying URL 'https://mirrors.nics.utk.edu/cran/src/contrib/broman_0.76.tar.gz' 
-cd pcre-8.39 +Content type 'application/x-gzip' length 79834 bytes (77 KB) 
-./configure --prefix=/share/apps/pcre/8.39 --enable-utf +================================================== 
-make && make install+downloaded 77 KB
  
-cd .. +* installing *source* package ‘broman’ ... 
-wget https://curl.haxx.se/download/curl-7.50.0.tar.gz +** package ‘broman’ successfully unpacked and MD5 sums checked 
-tar zxf curl*gz +** using staged installation 
-cd curl-7.50.0 +** libs 
-./configure --prefix=/share/apps/curl/7.50.0 --with-zlib=/share/apps/zlib/zlib-1.25 --enable-libcurl-option --with-ssl=/usr/include/ssl --enable-http --with-libssh2 +gcc -I"/share/apps/R/R-4.0.2/lib64/R/include" -DNDEBUG   -I/usr/local/include   -fpic  -g -O2  -c R_init.c -o R_init.o 
-make && make install+gcc -I"/share/apps/R/R-4.0.2/lib64/R/include" -DNDEBUG   -I/usr/local/include   -fpic  -g -O2  -c compare_rows.c -o compare_rows.o 
 +gcc -I"/share/apps/R/R-4.0.2/lib64/R/include" -DNDEBUG   -I/usr/local/include   -fpic  -g -O2  -c count_close.c -o count_close.o 
 +gcc -I"/share/apps/R/R-4.0.2/lib64/R/include" -DNDEBUG   -I/usr/local/include   -fpic  -g -O2  -c normalize.c -o normalize.o 
 +gcc -I"/share/apps/R/R-4.0.2/lib64/R/include" -DNDEBUG   -I/usr/local/include   -fpic  --O2  -c runningmean.c -o runningmean.o 
 +gcc -shared -L/share/apps/R/R-4.0.2/lib64/R/lib -L/usr/local/lib64 -o broman.so R_init.o compare_rows.o count_close.o normalize.o runningmean.o -L/share/apps/R/R-4.0.2/lib64/R/lib -lR 
 +installing to /scrfs/storage/feynman/home/R/x86_64-pc-linux-gnu/4.0/00LOCK-broman/00new/broman/libs 
 +** R 
 +** data 
 +*** moving datasets to lazyload DB 
 +** byte-compile and prepare package for lazy loading 
 +** help 
 +*** installing help indices 
 +** building package indices 
 +** testing if installed package can be loaded from temporary location 
 +** checking absolute paths in shared objects and dynamic libraries 
 +** testing if installed package can be loaded from final location 
 +** testing if installed package keeps a record of temporary installation path 
 +* DONE (broman)
  
-cd .. +The downloaded source packages are in 
-wget http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz + /tmp/RtmpRuxu78/downloaded_packages’ 
-tar zxf bzip*gz +> quit() 
-cd bzip2-1.0.6 +Save workspace image? [y/n/c]: n 
-vi Makefile (add -fPIC to cflags+pinnacle-l4:feynman:$
-make install PREFIX=/share/apps/bzip2/1.0.6+
  
-cd .. 
-wget https://cran.r-project.org/src/base/R-3/R-3.3.1.tar.gz 
-tar zxf R-3.3.1.tar.gz 
-cd R-3.3.1 
-module load intel/14.0.3 mkl/14.0.3 
-IOPT="-O3 -xsse4.2 -axavx" 
-FC=ifort CC=icc CXX=icpc FCFLAGS="$IOPT" CFLAGS="-I/share/apps/zlib/zlib-1.2.5/include -I/share/apps/bzip2/1.0.6/include -I/share/apps/xz/5.2.2/include -I/share/apps/pcre/8.39/include -I/share/apps/curl/7.50.0/include  $IOPT" CXXFLAGS="$IOPT" ./configure --prefix=/share/apps/R/R-3.3.1/intel-14.0.3 --with-blas=no --with-lapack=no --enable-R-shlib --with-x --with-recommended-packages --with-tcltk CFLAGS="-I/share/apps/zlib/zlib-1.2.5/include -I/share/apps/bzip2/1.0.6/include -I/share/apps/xz/5.2.2/include -I/share/apps/pcre/8.39/include -I/share/apps/curl/7.50.0/include  $IOPT" LDFLAGS="-L/share/apps/zlib/1.2.5/lib -L/share/apps/bzip2/1.0.6/lib -L/share/apps/xz/5.2.2/lib -L/share/apps/pcre/8.39/lib -L/share/apps/curl/7.50.0/lib -Wl,-rpath-link,/share/apps/pcre/8.39/lib -Wl,-rpath-link,/share/apps/xz/5.2.2/lib -L$MKLROOT/intel64 -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lpthread -liomp5" 
-make && make install 
 </code> </code>
r.1600722801.txt.gz · Last modified: 2020/09/21 21:13 by root