Running The Solver

  • Mohamed Houssem Kasbaoui

Running The Solver

There are 3 steps to performing simulations with LEAP. First, you need to setup a case. This is explained in the section: Setting Up a New Case. Next, you need to generate initial conditions. Lastly, you integrate the solution forward in time.

For this case, we will use the predefined case for lid-driven cavity flow.

cd /path/to/leap/cases/cdifs_lid_driven_cavity

In this directory, you will find source files for a program that will generate initial conditions for this case. Go into the “src” folder, add a properly configured Makefile.in, and compile the program.

# Go to the src directory
cd /path/to/leap/cases/cdifs_lid_driven_cavity/src

# Add a Makefile.in (update paths and compilers to suit your machine)
cat << 'EOF' > Makefile.in
#!bash

HOMEDIR = $(shell pwd | sed -e 's/\/src.*//')
MDIR  = $(HOMEDIR)/mod
ODIR  = $(HOMEDIR)/obj
BDIR  = $(HOMEDIR)/bin

# Compiler and archiver
F90 = mpif90
LD  = mpif90
AR  = ar rcv 
RL  = ranlib

# Flags (using GNU compilers)
LDFLAGS  =
INCFLAGS = -I$(MDIR)
MODFLAGS = -J$(MDIR)
PPFLAGS  = -cpp
DBGFLAGS = -g  -fcheck=all -fbacktrace  -Wall  -Wunderflow  -ffpe-trap=invalid  -fbounds-check -ffree-line-length-none
OPTFLAGS = -O3 -march=native -ffree-line-length-none

#LEAP
LEAP_DIR  = /path/to/leap
LEAP_INC  = -I$(LEAP_DIR)/mod
LEAP_LIB  = -Wl,-rpath -Wl,$(LEAP_DIR)/lib -L$(LEAP_DIR)/lib \
            -lleap
#HF5P
HDF5P_DIR = /path/to/ThirdParty/hdf5
HDF5P_INC = -I$(HDF5P_DIR)/include
HDF5P_LIB = -Wl,-rpath -Wl,$(HDF5P_DIR)/lib -L$(HDF5P_DIR)/lib \
            -lhdf5 -lhdf5_fortran
#H5HUT
H5HUT_DIR = /path/to/ThirdParty/h5hut
H5HUT_INC = -I$(H5HUT_DIR)/include
H5HUT_LIB = -Wl,-rpath -Wl,$(H5HUT_DIR)/lib -L$(H5HUT_DIR)/lib \
            -lH5hutF -lH5hut

#HYPRE
HYPRE_DIR = /path/to/ThirdParty/hypre
HYPRE_INC = -I$(HYPRE_DIR)/include
HYPRE_LIB = -Wl,-rpath -Wl,$(HYPRE_DIR)/lib -L$(HYPRE_DIR)/lib \
            -lHYPRE

#SZIP
SZIP_DIR = /path/to/ThirdParty/szip
SZIP_LIB = -Wl,-rpath -Wl,$(SZIP_DIR)/lib -L$(SZIP_DIR)/lib    \
           -lsz

#ZLIB
ZLIB_DIR = /path/to/ThirdParty/zlib
ZLIB_LIB = -Wl,-rpath -Wl,$(ZLIB_DIR)/lib -L$(ZLIB_DIR)/lib    \
           -lz

#SILO
SILO_DIR = /path/to/ThirdParty/silo
SILO_INC = -I$(SILO_DIR)/include
SILO_LIB = -Wl,-rpath -Wl,$(SILO_DIR)/lib -L$(SILO_DIR)/lib    \
           -lsiloh5  $(SZIP_LIB) $(ZLIB_LIB)                   \
           -L/usr/local/lib -lstdc++
EOF

# Compile the source code
make debug

Upon compilation, a program called lid_dirven_cavity will be created under /path/to/leap/cases/cdifs_lid_driven_cavity/bin.

Next, we are going to run this program with the input file provided in this case folder. The input file contains various parameters for this case and can be modified to vary things like density, viscosity, lid velocity, grid, etc. Note that the input file calls for 2x2x1 domain partition (this can be changed too). Thus, this run requires 4 cores by default.

To generate the initial conditions for this case, run the case program as follows:

mpiexec -n 4 /path/to/leap/cases/cdifs_lid_driven_cavity/bin -i input

Once this completes successfully, we can proceed with the integration by calling LEAP:

mpiexec -n 4 /path/to/leap/bin/leap -i input