Compiling FVCOM: A Handy Guide to Solving Common Issues

The Finite Volume Community Ocean Model (FVCOM) is making waves in the academic and research world, with over 6,000 publications referencing it. 

You can easily run FVCOM simulations using the Inductiva API. Here’s how to send a FVCOM simulation to a machine hosted on Google Cloud (GCP), which you can adapt to your own case to get started:

"""FVCOM example"""
import inductiva

# Allocate Google cloud machine
cloud_machine = inductiva.resources.MachineGroup( \
provider="GCP",
machine_type="c3d-standard-180")

# Initialize the Simulator
fvcom = inductiva.simulators.FVCOM()

# Run simulation with config files in the input directory
task = fvcom.run( \
input_dir="path/to/my/fvcom/files",
working_dir="run/",
case_name="my_case_name",
on=cloud_machine)

# Wait for the simulation to finish and download the results
task.wait()
cloud_machine.terminate()

task.download_outputs()

On the other hand, if you’ve landed here, you probably already know that compiling FVCOM can be a challenge. Even experienced programmers often run into difficulties during the compilation process.
We’ve been there ourselves. During our attempt to compile FVCOM, we encountered several roadblocks and errors that weren’t covered in online forums or existing guides. So we decided to make our own toolbox!

We compiled FVCOM 5.1.0 on a machine powered by a 12-core 12th Gen Intel i5 processor, documenting every step along the way. This guide is structured to walk you through the process step by step, helping you build, troubleshoot, and apply the fixes we discovered.

Now, let’s dive right in!

Preparing the Prerequisites for FVCOM Compilation 

Before compiling FVCOM, make sure you have all the necessary software and compilers ready. Below, we’ve listed everything you’ll need, along with a few tips to help you avoid common issues.

Required Compilers

Make sure you have these compilers installed:

• GFortran: We used GNU Fortran version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04)
• GCC: We paired GFortran with GCC version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04)

Additional Software

Beyond compilers, several additional software packages are necessary:

• OpenMPI: Ensure you have OpenMPI version 4.1.2 installed. This is essential for parallel processing, which FVCOM relies on.
• Metis: Metis is crucial for mesh partitioning in FVCOM. We installed it via apt-get using the command apt-get install libmetis-dev. The version we used was 5.1.0.dfsg-7build2, and it worked without issues.
• NetCDF: NetCDF is used for managing data files in FVCOM. We recommend installing it through apt-get with the command `apt-get install libnetcdff-dev libnetcdf-dev` The versions we used were libnetcdff-dev 4.5.4+ds-1 and libnetcdf-dev 1:4.8.1-1. It’s important to note that although NetCDF is available directly from the repository, we encountered issues when compiling FVCOM. Installing via apt-get proved to be the most reliable method.
• Julian, Proj4, and Fortran-Proj4: These three components are provided directly in the FVCOM repository.
• PETSc: PETSc is another critical software package that FVCOM depends on. To avoid potential issues, we recommend cloning and building PETSc directly from its repository rather than using apt-get. The reason is that the apt-get version often lacks essential files that FVCOM requires, such as those found in /lib/petsc/conf/variables. You can check the version details in the /petsc/include/petscversion.h file to ensure compatibility.

#define PETSC_VERSION_RELEASE    1
#define PETSC_VERSION_MAJOR      3
#define PETSC_VERSION_MINOR      21
#define PETSC_VERSION_SUBMINOR   4
#define PETSC_RELEASE_DATE       "Mar 29, 2024"
#define PETSC_VERSION_DATE       "unknown"

Configuring FVCOM

Now that we’ve prepared all the prerequisites for FVCOM, it’s time to download the source code from FVCOM’s GitHub and begin the compilation process.

Step 1: Download FVCOM and Define Environment Variables

Clone the repository by running the following command:

git clone --branch v5.1.0 https://github.com/FVCOM-GitHub/FVCOM.git FVCOM

This should download FVCOM version 5.1.0 into a directory named FVCOM. Then, to compile FVCOM successfully, you’ll need to set up several environment variables. These variables tell the compiler where to find the necessary libraries and tools. Here’s what you need to set:

• PETSC_DIR: This should point to the directory where PETSc is installed.
• PETSC_ARCH: This specifies the architecture configuration for PETSc. Make sure it matches the setup on your machine.
• INCLUDEPATH: This variable includes the paths to various necessary libraries and header files. Adjust the paths if your installations are in different location

export PETSC_DIR=/petsc
export PETSC_ARCH=arch-linux-c-debug
export INCLUDEPATH=/usr/lib/x86_64-linux-gnu/fortran/gfortran-mod-15/openmpi:\
/FVCOM/src/libs/install/include/:\
/usr/include/

Step 2: Build PETSc

Next step is building the PETSc library. Here’s how you can do it:

1 – Clone the PETSc Repository:

git clone -b release https://gitlab.com/petsc/petsc.git petsc

2 – Navigate to the PETSc Directory: 

cd $PETSC_DIR

3- Configure PETSc with Hypre:

./configure --download-hypre --with-hypre

4- Build PETSc:

make PETSC_DIR=/petsc PETSC_ARCH=arch-linux-c-debug all

Common Errors

A common issue you might run into during the PETSc setup is forgetting to add the --download-hypre and --with-hypre flags. Skipping these flags during configuration usually leads to the following error:

/usr/bin/ld: mod_petsc.o: in function `__mod_petsc_MOD_petsc_set':
mod_petsc.f90:(.text+0x199e): undefined reference to `pchypresettype_'
collect2: error: ld returned 1 exit status
make: *** [makefile:135: fvcom] Error 1

When we ran into the pchypresettype_ error, we fixed it by searching for the function inside the PETSc directory using this command:

grep -Rnw '/petsc' -e 'pchypresettype_'

The search returned results indicating that this function is located in a folder named “hypre”:

/petsc/src/ksp/pc/impls/hypre/ftn-custom/zhypref.c:5:  
#define pchypresettype_                     PCHYPRESETTYPE

/petsc/src/ksp/pc/impls/hypre/ftn-custom/zhypref.c:10:  
#define pchypresettype_                     pchypresettype

/petsc/src/ksp/pc/impls/hypre/ftn-custom/zhypref.c:16:  
PETSC_EXTERN void pchypresettype_(PC *pc, char *name, 
                                  PetscErrorCode *ierr, 
                                  PETSC_FORTRAN_CHARLEN_T len)

We also discovered that the PETSc configuration includes several flags related to Hypre, which you can list using:

./configure -h | grep hypre

The output will include options such as:

--with-hypre=
     Indicate if you wish to test for hypre  current: 0
--download-hypre=<no,yes,filename,url>
     Download and install hypre  current: no
--with-hypre-include=
     Indicate the directory of the hypre include files
--with-hypre-lib=
     Indicate the hypre libraries
...

This error pops up because the Hypre library isn’t linked properly during the build. To avoid this, make sure to include the --download-hypre and --with-hypre flags when configuring PETSc. This will tell PETSc to download and set up Hypre correctly, saving you from running into this issue.

Step 3: Configure FVCOM

To get FVCOM up and running, you’ll need to tweak the make.inc file found in the /FVCOM/src directory. This file contains different settings (called flags) that you can adjust to customize FVCOM’s behavior. We made a few key adjustments to the configuration files to ensure a smooth compilation process:

1- Define the TOPDIR Value: Start by defining the TOPDIR variable, which should point to the source directory. In our setup, this will be /FVCOM/src.

2- Configure PETSc: Next, configure the file to use PETSc version 3.18.5. The include path can vary depending on the PETSc version, so it’s crucial to set this correctly. For version 3.18.5, your PETSc-related configuration should look like this:

#       FOR VERSION 2.0   TO 2.3.2
#            FLAG_41 = -DPETSC_A
#            include ${PETSC_DIR}/bmake/common/variables
#
#       FOR VERSION 2.3.3
#            FLAG_41 = -DPETSC_B
#            include ${PETSC_DIR}/bmake/common/variables
#
#       FOR VERSION 3.18.5
            FLAG_41 = -DPETSC_C
            include ${PETSC_DIR}/lib/petsc/conf/variables

3- Modify Compiler Definitions 

We made minimal changes to the “BEGIN USER DEFINITION SECTION,” mainly focusing on the compiler definitions. We chose to comment out the “Intel/MPI Compiler Definitions (SMAST)” section and instead, activated the “gfortran defs” section. Here’s how your updated configuration should look:

#--------------------------------------------------------------------------
#  Intel/MPI Compiler Definitions (SMAST)
#--------------------------------------------------------------------------
#         CPP      = /usr/bin/cpp
#         COMPILER = -DIFORT
#         CC       = mpicc
#         CXX      = mpicxx
#         CFLAGS   = -O3
#         FC       = mpif90
#         DEBFLGS  = -check all -traceback
#         OPT      = -O0 -g
#         OPT      = -axN -xN
#         OPT      = -O3

#--------------------------------------------------------------------------
#  gfortran defs
#--------------------------------------------------------------------------
         CPP      = /usr/bin/cpp
         COMPILER = -DGFORTRAN
         FC       = gfortran
         DEBFLGS  = -O3
         OPT      = -ffixed-line-length-none -ffree-line-length-0 -fallow-argument-mismatch
         CLIB     =

As you can see, we made a few key adjustments during the compilation process to avoid errors. 

First, we moved the -O3 optimization flag to DEBFLGS because keeping it in the FC variable with gfortran -O3 was causing compilation errors. Next, we added the -ffixed-line-length-none and -ffree-line-length-0 flags to remove the default line length limit for both fixed and free-format Fortran source files, which is crucial to prevent syntax errors from truncated lines. Lastly, we included the -fallow-argument-mismatch flag, which converts argument mismatch errors into warnings. Since we trust the source code, we treat these mismatches as non-critical.

4- Link the Julian Library

Next, we needed to make sure the compiler could properly locate and link the Julian library during the build. We ran into some issues during the compilation process, but we fixed them by modifying the DTLIBS variable in the make.inc file. This variable tells the compiler where to find the required libraries. In our case, we had to ensure that it pointed to the correct path where the Julian library was compiled.

To do this, we adjusted the DTLIBS line to look like this:

#--------------------------------------------------------------------------
#       STANDARD LIBRARIES FOR DATA AND TIME IN FVCOM:
#
            DTLIBS      = -L/FVCOM/src/libs/install/lib -ljulian
            DTINCS      =

Step 4: Compile the Libraries

Before we could get the libraries to compile smoothly, we ran into a few configuration issues. We had to make some tweaks to fix errors caused by missing paths and incorrect settings in the Makefile. Here’s a rundown of the edits we made to the Makefile in the /FVCOM/src/libs/ directory to make sure everything is properly set up before running the build.

1- Ensure All Necessary Folders Are Included: Make sure that all required libraries are listed in the PACKAGES variable.

Your PACKAGES line should look like this:

PACKAGES = proj fproj julian metis netcdf

2- Update the Makefile: Remove any configuration and build instructions for libraries that are already installed via apt-get. This prevents conflicts with libraries that are already present on your system. Then, Update the CFLAGS for fproj to include the following flags: -O3 -Df2cFortran.

After these changes, your Makefile should look like this:

all:
    for item in $(PACKAGES); do (./untar.sh $$item ) || exit 1; done
    cd proj && ./configure CC=$(CC) CFLAGS=-O3 CXX=$(CC) CXXFLAGS=-O3 F77=$(FC) FFLAGS=-O3 --prefix=$(MYINSTALLDIR)
    cd proj && make install
    cd fproj && ./configure CPPFLAGS='$(COMPILER)' CC=$(CC) CFLAGS="-O3 -Df2cFortran" CXX=$(CXX) CXXFLAGS=-O3 FC=$(FC) FFLAGS=-O3 --with-proj4=$(MYINSTALLDIR) --prefix=$(MYINSTALLDIR)
    cd fproj && make install
    cd julian && make install

With the make.inc file updated, it’s time to compile the libraries! With the configurations in place, compiling the required libraries should be straightforward. Just run these commands in the /FVCOM/src/libs/ directory:

cd /FVCOM/src/libs
make

After this, all the libraries should be installed in the /FVCOM/src/libs/install directory.

Compiling FVCOM

You’re almost ready to compile FVCOM, but if you try to compile it now, you may run into this error:

mod_station_timeseries.f90:1130:25:

 1130 |          WRITE(10,'(2I10,<MX_NBR_ELEM_GL+1>I10)') I,NTVEGL(I),(NBVEGL(I,J),J=1,NTVEGL(I))
      |                         1
Error: Unexpected element '<' in format string at (1)
make: *** [makefile:45: mod_station_timeseries.o] Error 1

This issue was fixed in a later commit to the repository (commit 476e820). To resolve it, you need to apply the changes from that commit to the relevant files. You can do this by running the following command in the /FVCOM/src directory:

sed -i "s/WRITE(10,'(2I10,<MX_NBR_ELEM_GL+1>I10)')/WRITE(10,'(2I10,\"<MX_NBR_ELEM_GL+1>\"I10)')/g" \
    mod_station_timeseries.F && \
sed -i "s/EXFLUXA=-UNA*((1.0+SIGN(REAL(1.0,SP),UNA))*FIJ2A(:,:,IB)+(1.0-SIGN(REAL(1.0,SP),UNA))*FIJ1A(:,:,IA))*0.5/\
EXFLUXA=-UNA*((1.0+SIGN(REAL(1.0,SP),REAL(UNA,SP)))*FIJ2A(:,:,IB)+(1.0-SIGN(REAL(1.0,SP),REAL(UNA,SP)))*FIJ1A(:,:,IA))*0.5/g" \
    mod_action_ex.F && \
sed -i "s/EXFLUXB=-UNBB*((1.0+SIGN(REAL(1.0,SP),UNBB))*FIJ2B(:,:,IB)+(1.0-SIGN(REAL(1.0,SP),UNBB))*FIJ1B(:,:,IA))*0.5/\
EXFLUXB=-UNBB*((1.0+SIGN(REAL(1.0,SP),REAL(UNBB,SP)))*FIJ2B(:,:,IB)+(1.0-SIGN(REAL(1.0,SP),REAL(UNBB,SP)))*FIJ1B(:,:,IA))*0.5/g" \
    mod_action_ex.F && \
sed -i "s/IF(ELTYPE == 'KEY' .AND. KEYWIS==.TRUE.) ELTYPE = 'USED'/\
IF((ELTYPE == 'KEY') .AND. (KEYWIS .EQV. .TRUE.)) ELTYPE = 'USED'/g" \
    ocpcre.F

After applying these fixes, you can run make, and FVCOM should compile smoothly without any further issues!

Final Thoughts: Navigating the FVCOM Compilation Process

Getting FVCOM to compile might be a challenge, but it’s also an exciting step toward unlocking powerful coastal dynamic simulations! While we can’t guarantee a perfect build, we hope that by sharing our experience, we’ve helped the FVCOM community.

Remember, different systems and configurations might require slight adjustments, so always check your environment settings and be ready to solve any problems. 

If you want to save time, you can download our ready-to-use files, like the pre-configured make.inc file, to skip some of the manual tweaking.

FVCOM is one of our many available simulators, all of which are integrated within our API for scalable cloud simulations. Keep an eye out for updates, and check out our other built-in simulators, like OpenFOAM, OpenFAST, Reef3D, SWAN, SWASH, and XBeach.

Want to try out Inductiva’s API to scale up your simulations? Log in to our console and try it for free