vasp
For use by authorized users only.
Local copy 5.4.4 (for Intel) is modified from distribution as follows:
installs$ diff -r VASP-REFERENCE/vasp.5.4.4/src VASP/5.4.4/vasp.5.4.4/build/std | grep -v "\.o$" | grep -v "\.f90$" | grep -v "\.mod$" Only in VASP-REFERENCE/vasp.5.4.4/src: CUDA Only in VASP-REFERENCE/vasp.5.4.4/src: fftlib Only in VASP/5.4.4/vasp.5.4.4/build/std/lib: libdmy.a Only in VASP/5.4.4/vasp.5.4.4/build/std/lib: makefile.include diff -r VASP-REFERENCE/vasp.5.4.4/src/makefile VASP/5.4.4/vasp.5.4.4/build/std/makefile 18c18 < LLIB=-Llib -ldmy --- > LLIB=lib/getshmem.o lib/linpack_double.o -Llib -ldmy 131c131 < #OBJCTS_f90=$(filter-out getshmem.o, $(OBJCTS)) --- > OBJCTS_f90=$(filter-out getshmem.o, $(OBJCTS)) Only in VASP/5.4.4/vasp.5.4.4/build/std: makefile.include Only in VASP/5.4.4/vasp.5.4.4/build/std/parser: libparser.a Only in VASP/5.4.4/vasp.5.4.4/build/std/parser: makefile.include Only in VASP-REFERENCE/vasp.5.4.4/src: README Only in VASP-REFERENCE/vasp.5.4.4/src: vasp.cfg Only in VASP/5.4.4/vasp.5.4.4/build/std: vasp.O2.smkl Only in VASP/5.4.4/vasp.5.4.4/build/std: vasp.O3.pmkl Only in VASP/5.4.4/vasp.5.4.4/build/std: vasp.O3.smkl installs$
getshmem.o and linpackdouble.o are made in build/lib, and modules and makefile.include'' are:
<code>
installs$ module load intel/19.0.5 mkl/19.0.5 impi/19.0.5
installs$ cat VASP/5.4.4/vasp.5.4.4/build/std/makefile.include
# Precompiler options
CPPOPTIONS= -DHOST=\“LinuxIFC\”\
-DMPI -DMPIBLOCK=8000 \
-Dusecollective \
-DscaLAPACK \
-DCACHESIZE=4000 \
-Davoidalloc \
-Dusebsete \
-Dtbdyn \
-Duseshmem
CPP = fpp -fcom=no -free -w0 $$(FUFFIX) $$(SUFFIX) $(CPPOPTIONS)
FC = mpiifort
FCL = mpiifort -mkl=sequential -lstdc++
FREE = -free -names lowercase
FFLAGS = -assume byterecl -w
OFLAG = -O2 -xHOST -qopenmp
OFLAG = -O3 -xsse3 -axsse4.2,AVX,COREAVX512 -qopenmp
OFLAG = -O2 -xsse3 -axsse4.2,AVX,COREAVX512 -qopenmp
OFLAG_IN = $(OFLAG)
DEBUG = -O0
MKLPATH = $(MKLROOT)/lib/intel64
BLAS =
LAPACK =
BLACS = -lmklblacsintelmpilp64
SCALAPACK = $(MKLPATH)/libmklscalapack_lp64.a $(BLACS)
OBJECTS = fftmpiw.o fftmpi_map.o fft3dlib.o fftw3d.o
INCS =-I$(MKLROOT)/include/fftw
LLIBS = $(SCALAPACK) $(LAPACK) $(BLAS)
OBJECTSO1 += fftw3d.o fftmpi.o fftmpiw.o
OBJECTSO2 += fft3dlib.o
# For what used to be vasp.5.lib
CPPLIB = $(CPP)
FCLIB = $(FC)
CCLIB = icc
CFLAGSLIB = -O
FFLAGSLIB = -O1
FREELIB = $(FREE)
OBJECTSLIB= linpackdouble.o getshmem.o
# For the parser library
CXX_PARS = icpc
LIBS += parser
LLIBS += -Lparser -lparser -lstdc++
# Normally no need to change this
SRCDIR = ../../src
BINDIR = ../../bin
#================================================
# GPU Stuff
#CPPGPU = -DCUDAGPU -DRPROMUCPROJOVERLAP -DUSEPINNEDMEMORY -DCUFFT_MIN=28 -UscaLAPACK
#OBJECTSGPU = fftmpiw.o fftmpimap.o fft3dlib.o fftw3dgpu.o fftmpiwgpu.o
#CC = icc
#CXX = icpc
#CFLAGS = -fPIC -DADD_ -Wall -openmp -DMAGMAWITHMKL -DMAGMASETAFFINITY -DGPUSHMEM=300 -DHAVECUBLAS
#CUDAROOT ?= /usr/local/cuda/
#NVCC := $(CUDAROOT)/bin/nvcc -ccbin=icc
#CUDALIB := -L$(CUDAROOT)/lib64 -lnvToolsExt -lcudart -lcuda -lcufft -lcublas
#GENCODEARCH := -gencode=arch=compute30,code=\“sm30,compute30\” \
-gencode=arch=compute35,code=\“sm35,compute35\” \
-gencode=arch=compute60,code=\“sm60,compute60\”
MPIINC = $(IMPIROOT)/intel64/include/
installs$
</code>
Some run examples with an OpenMP, MKL sequential build:
<code>
module load intel/19.0.5 mkl/19.0.5 impi/19.0.5
mpirun -np 16 -genv OMPNUMTHREADS=2 /scrfs/apps/vasp/vasp.5.4.4/build/std/vasp.O2.smkl
took 75 seconds on Trestles, 35 seconds on Razor 16-core (with OMPNUM_THREADS=1), 27.3 seconds on Pinnacle.
vasp.o2.smkl took 26.1 seconds on Pinnacle.
</code>
For performance MPI threads x OpenMP threads should not exceed physical cores.
