User Guide

Getting Started

To get started with LEAP you need to have a modern Fortran compiler that supports Fortran 2018 features, such as Gfortran 13x and higher, and follow the instructions below to get third-party libraries and compile the code.

Getting Third-Party Libraries

The following libraries are required to run LEAP:

An MPI distribution (recommended OpenMPI): This library handles the distributed memory aspects and allows the code to run in parallel on large computing systems.
HDF5: This library provides a file format (hdf5) that is widely used for scientific data. It is designed to be scalable, and optimizes IO on distributed systems. There is a nice little tool called HDF View that provides a GUI to look into the meta data in the file. A command line tool, h5dump, can give more information. Data produced with HDF5 can also be viewed in VisIt/ParaVeiw.
H5HUT: this is a wrapper around HDF5 that provides a high level API to make using HDF5 easier with block and Lagrangian data.
SZIP: A compression library from the HDF group used by HDF5.
ZLIB: Another compression library used by HDF5.
SILO: A file format for point, structured, and unstructured meshes developed by the Lawrence Livermore National Laboratory.
HYPRE: Scalable linear solvers and multigrid methods developed by the Lawrence Livermore National Laboratory

You may download these libraries from their developer websites and follow their compilation instructions. In general, it is best to use the latest version available.

Configuring a Makefile.in file

In order to compile LEAP, you need to create a file called Makefile.in and place it under path/to/leap/src/. This file is machine dependent and the parameters will depend on the compilers you use. Below are examples to follow for the following computers

MacOS with GNU compilers
SOL and PHX with GNU compilers
BRIDGES-2 with INTEL compilers

MacOS with GNU compilers

HOMEDIR = $(shell pwd | sed -e 's/\/src.*//')
LDIR  = $(HOMEDIR)/lib
MDIR  = $(HOMEDIR)/mod
ODIR  = $(HOMEDIR)/obj
BDIR  = $(HOMEDIR)/bin

# Compiler and archiver
F90 = mpifort
LD  = mpifort
AR  = ar rcv 
RL  = ranlib

# Flags
LDFLAGS  =
INCFLAGS = -I$(MDIR)
MODFLAGS = -J$(MDIR)
PPFLAGS  = -cpp
DBGFLAGS = -g -fcheck=all -fbacktrace -Wall -Wunderflow -ffpe-trap=invalid -fbounds-check -finit-local-zero -ffree-line-length-none -fallow-invalid-boz -fallow-argument-mismatch
OPTFLAGS = -O3 -march=native -ffree-line-length-none -fallow-invalid-boz -fallow-argument-mismatch

#HDF5
HDF5P_DIR = /path/to/thirdparty/hdf5
HDF5P_INC = -I$(HDF5P_DIR)/include
HDF5P_LIB = -Wl,-rpath -Wl,$(HDF5P_DIR)/lib -L$(HDF5P_DIR)/lib \
            -lhdf5 -lhdf5_fortran
#H5HUT  
H5HUT_DIR = /path/to/thirdparty/h5hut
H5HUT_INC = -I$(H5HUT_DIR)/include
H5HUT_LIB = -Wl,-rpath -Wl,$(H5HUT_DIR)/lib -L$(H5HUT_DIR)/lib \
            -lH5hutF -lH5hut
#HYPRE
HYPRE_DIR = /path/to/thirdparty/hypre
HYPRE_INC = -I$(HYPRE_DIR)/include
HYPRE_LIB = -Wl,-rpath -Wl,$(HYPRE_DIR)/lib -L$(HYPRE_DIR)/lib \
            -lHYPRE
#SZIP
SZIP_DIR = /path/to/thirdparty/szip
SZIP_LIB = -Wl,-rpath -Wl,$(SZIP_DIR)/lib -L$(SZIP_DIR)/lib    \
           -lsz
#ZLIB
ZLIB_DIR = /path/to/thirdparty/zlib
ZLIB_LIB = -Wl,-rpath -Wl,$(ZLIB_DIR)/lib -L$(ZLIB_DIR)/lib    \
           -lz
#SILO
SILO_DIR = /path/to/thirdparty/silo
SILO_INC = -I$(SILO_DIR)/include
SILO_LIB = -Wl,-rpath -Wl,$(SILO_DIR)/lib -L$(SILO_DIR)/lib    \
           -lsiloh5 $(HDF5P_LIB) $(SZIP_LIB) $(ZLIB_LIB)       \
           -L/usr/local/lib -lstdc++

SOL and PHX with GNU compilers

HOMEDIR = $(shell pwd | sed -e 's/\\/src.*//')
LDIR  = $(HOMEDIR)/lib
MDIR  = $(HOMEDIR)/mod
ODIR  = $(HOMEDIR)/obj
BDIR  = $(HOMEDIR)/bin

# Compiler and archiver
F90 = mpifort
LD  = mpifort
AR  = ar rcv
RL  = ranlib

# Flags
LDFLAGS  =
INCFLAGS = -I$(MDIR)
MODFLAGS = -J$(MDIR)
PPFLAGS  = -cpp
DBGFLAGS = -g -fcheck=all -fbacktrace  -Wall  -Wunderflow  -ffpe-trap=invalid  -fbounds-check -ffree-line-length-none -fallow-invalid-boz -fallow-argument-mismatch
OPTFLAGS = -O3 -march=native -ffree-line-length-none -fPIC -fallow-invalid-boz -fallow-argument-mismatch

#HDF5
HDF5P_DIR = /path/to/thirdparty/hdf5
HDF5P_INC = -I$(HDF5P_DIR)/include
HDF5P_LIB = -Wl,-rpath -Wl,$(HDF5P_DIR)/lib -L$(HDF5P_DIR)/lib \
            -lhdf5 -lhdf5_fortran
#H5HUT  
H5HUT_DIR = /path/to/thirdparty/h5hut
H5HUT_INC = -I$(H5HUT_DIR)/include
H5HUT_LIB = -Wl,-rpath -Wl,$(H5HUT_DIR)/lib -L$(H5HUT_DIR)/lib \
            -lH5hutF -lH5hut
#HYPRE
HYPRE_DIR = /path/to/thirdparty/hypre
HYPRE_INC = -I$(HYPRE_DIR)/include
HYPRE_LIB = -Wl,-rpath -Wl,$(HYPRE_DIR)/lib -L$(HYPRE_DIR)/lib \
            -lHYPRE
#SZIP
SZIP_DIR = /path/to/thirdparty/szip
SZIP_LIB = -Wl,-rpath -Wl,$(SZIP_DIR)/lib -L$(SZIP_DIR)/lib    \
           -lsz
#ZLIB
ZLIB_DIR = /path/to/thirdparty/zlib
ZLIB_LIB = -Wl,-rpath -Wl,$(ZLIB_DIR)/lib -L$(ZLIB_DIR)/lib    \
           -lz
#SILO
SILO_DIR = /path/to/thirdparty/silo
SILO_INC = -I$(SILO_DIR)/include
SILO_LIB = -Wl,-rpath -Wl,$(SILO_DIR)/lib -L$(SILO_DIR)/lib    \
           -lsiloh5 $(HDF5P_LIB) $(SZIP_LIB) $(ZLIB_LIB)       \
           -L/usr/local/lib -lstdc++

BRIDGES-2 with INTEL compilers

HOMEDIR = $(shell pwd | sed -e 's/\\/src.*//')
LDIR  = $(HOMEDIR)/lib
MDIR  = $(HOMEDIR)/mod
ODIR  = $(HOMEDIR)/obj
BDIR  = $(HOMEDIR)/bin

# Compiler and archiver
F90 = mpif90
LD  = mpif90
AR  = ar rcv
RL  = ranlib

# Flags
FLAGS   =
F90FLAGS =
F77FLAGS =
LDFLAGS  = -march=core-avx2
INCFLAGS = -I$(MDIR)
MODFLAGS = -module $(MDIR)
DBGFLAGS = -g -CB -CS -traceback -debug all -ftrapuv -check all -check noarg_temp_created -WB
OPTFLAGS = -O3 -march=core-avx2 -ip

#HDF5
HDF5P_DIR = /path/to/thirdparty/hdf5
HDF5P_INC = -I$(HDF5P_DIR)/include
HDF5P_LIB = -Wl,-rpath -Wl,$(HDF5P_DIR)/lib -L$(HDF5P_DIR)/lib \
            -lhdf5 -lhdf5_fortran
#H5HUT  
H5HUT_DIR = /path/to/thirdparty/h5hut
H5HUT_INC = -I$(H5HUT_DIR)/include
H5HUT_LIB = -Wl,-rpath -Wl,$(H5HUT_DIR)/lib -L$(H5HUT_DIR)/lib \
            -lH5hutF -lH5hut
#HYPRE
HYPRE_DIR = /path/to/thirdparty/hypre
HYPRE_INC = -I$(HYPRE_DIR)/include
HYPRE_LIB = -Wl,-rpath -Wl,$(HYPRE_DIR)/lib -L$(HYPRE_DIR)/lib \
            -lHYPRE
#SZIP
SZIP_DIR = /path/to/thirdparty/szip
SZIP_LIB = -Wl,-rpath -Wl,$(SZIP_DIR)/lib -L$(SZIP_DIR)/lib    \
           -lsz
#ZLIB
ZLIB_DIR = /path/to/thirdparty/zlib
ZLIB_LIB = -Wl,-rpath -Wl,$(ZLIB_DIR)/lib -L$(ZLIB_DIR)/lib    \
           -lz
#SILO
SILO_DIR = /path/to/thirdparty/silo
SILO_INC = -I$(SILO_DIR)/include
SILO_LIB = -Wl,-rpath -Wl,$(SILO_DIR)/lib -L$(SILO_DIR)/lib    \
           -lsiloh5 $(HDF5P_LIB) $(SZIP_LIB) $(ZLIB_LIB)       \
           -L/usr/local/lib -lstdc++

Compiling LEAP

After getting the third-party libraries and configuring Makefile.in, you may now compile LEAP. Go to /path/to/leap/src and run the following commands:

cd /path/to/leap/src
make debug

This will compile the code with debug flags. When coding for LEAP, it's best to compile in debug mode, since the compilation is much faster, and the compiler will pick up on any potential mistakes in the new code.

For production runs, compile in opt(imized) mode

cd /path/to/leap/src
make opt

This will instruct the compiler to turn on optimization flags. The compilation will take longer than in debug mode, but the code will run significantly faster!

When running make with debug and opt flags, the compiler will build the LEAP library and compile the main program in /path/to/leap/src/leap.f90. The binaries will be placed in /path/to/leap/bin. The library and mod files will be placed in /path/to/leap/lib and /path/to/leap/mod.

If for any reason you need to restart from a clean slate, you can delete all compilation files with the following

cd /path/to/leap/src
make clean

LEAP also has a limited number of tests. If you would like to compile those too run the following

make tests