OpenFOAM® v1612+: Build process

OpenFOAM® v1612+: Build process

23/12/2016

Updated build process

Executive summary

  • Centralized/harmonized configuration
  • Command-line tool for making common adjustments.
  • Use the Allwmake with the -log option from the top-level OpenFOAM directory to capture the output for later diagnosis.
  • Extensive build information in ThirdParty BUILD.md

Robustness improvements

  • Ensure matches in label/scalar sizes between third-party components and what OpenFOAM expects.
  • Rationalized library names lib/ vs lib64/ for third-party components to ensure that the names are consistent between its newest and previous versions.
  • Cleaning configuration clutter by reducing the number of variables where possible and avoiding hidden logic. For example, now 3 instead of 5 variables for configuring ParaView.

Consistency improvements

  • The definition of the software versions for ThirdParty packages has now been centralized under the OpenFOAM etc/config.sh/ directory.
  • For most of the build scripts, the default software version is provided by an appropriate OpenFOAM etc/config.sh/... entry. This approach avoids duplicate entries for the default versions and ensures the best overall consistency between the OpenFOAM installation and its corresponding third-party installation.

Updated path configuration

Reconfiguring OpenFOAM to use different third-party software versions can be rather daunting, with many individual files to be edited. The bin/tools/foamConfigurePaths script provides a simple command-line interface to change the config.sh file contents.

This is an important step to automate installations and consistently package binaries. Obviously, not all parts of OpenFOAM can be easily configured with a single script, but foamConfigurePaths aims to tackle the most common variants. The build script in the Community repository further simplifies the procedure and includes scripting for the foamConfigurePaths script!

The available configuration options:

usage: foamConfigurePaths
  -foamInstall dir        specify installation directory (eg, /opt)
  -foamVersion ver        specify project version (eg, 1612)
  -projectName name       specify project directory name (eg, openfoam1612)
  -archOption 32|64       specify 'WM_ARCH_OPTION' architecture option
  -int32  | -int64        specify 'WM_LABEL_SIZE'
  -SP     | -DP           specify 'WM_PRECISION_OPTION'
  -system name            specify 'system' compiler to be used
  -third  name            specify 'ThirdParty' compiler to be used
  -boost ver              specify 'boost_version'
  -boost-path dir         specify 'BOOST_ARCH_PATH'
  -cgal ver               specify 'cgal_version'
  -cgal-path dir          specify 'CGAL_ARCH_PATH'
  -clang ver              specify 'clang_version' for ThirdParty Clang
  -cmake ver              specify 'cmake_version'
  -fftw ver               specify 'fffw_version'
  -fftw-path dir          specify 'FFTW_ARCH_PATH'
  -metis ver              specify 'METIS_VERSION'
  -metis-path dir         specify 'METIS_ARCH_PATH'
  -paraview ver           specify 'ParaView_VERSION' (eg, 5.0.1)
  -paraview-path dir      specify 'ParaView_DIR' (eg, /opt/paraviewopenfoam3120)
  -openmpi ver            specify ThirdParty openmpi version for 'FOAM_MPI'
  -openmpi-system         activate system openmpi
  -openmpi-third          activate ThirdParty openmpi (using default version)
  -scotch ver             specify 'SCOTCH_VERSION' (eg, scotch_6.0.4)
  -scotch-path dir        specify 'SCOTCH_ARCH_PATH' (eg, /opt/OpenFOAM-scotch_6.0.4)
  -vtk  ver               specify 'vtk_version' (eg, VTK-7.1.0)
  -mesa ver               specify 'mesa_version' (eg, mesa-13.0.1)
  -sigfpe | -no-sigfpe    activate/deactivate FOAM_SIGFPE handling
  gmp-VERSION             for ThirdParty gcc (gmp-system for system library)
  mpfr-VERSION            for ThirdParty gcc (mpfr-system for system library)
  mpc-VERSION             for ThirdParty gcc (mpc-system for system library)

* Adjust hardcoded versions and installation paths (for bash, POSIX shell).

Thirdy party component locations

In addition to specifying which version of a third-party software to use, we also need to handle situations where:

  • the third-party software is already available as part of the operating system, or
  • not provided by the operating system but provided in a central location, e.g. the modules package is a common approach used at data centers, or
  • the software is either not available or simply not wanted.

To handle these different situations, the specification of the software receives two variables: a version variable to specify the software name/version, and an arch-path (architecture path) variable for its installation location.

The version variable generally only exists within the confines of own config.sh file, but is also used by the ThirdParty build scripts to obtain information about the configured version. In contrast, the arch-path variable is exported as a shell variable for use in other parts of the build system.

To understand how these combinations work, let’s examine potential configurations for FFTW.

foamConfigurePaths
  -fftw ver               specify 'fffw_version'
  -fftw-path dir          specify 'FFTW_ARCH_PATH'

The corresponding configuration file happens to be the etc/config.sh/FFTW file.

A ThirdParty configuration of FFTW, as package name fftw-3.3.5

fftw_version=fftw-3.3.5
export FFTW_ARCH_PATH=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER/$fftw_version

Explanation
The makeFFTW script interrogates the config script to obtain the version. It tries to build the corresponding source directory fftw-3.3.5 located under ThirdParty, placing the resulting installation under the directory specified as FFTW_ARCH_PATH.

Later when building things that need FFTW, the FFTW_ARCH_PATH will contain include/ and lib/ (or lib64/) sub-directories.

FFTW from the operating system, e.g. as apt or rpm package

fftw_version=fftw-system
export FFTW_ARCH_PATH=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER/$fftw_version

or

export FFTW_ARCH_PATH=/dev/null/$fftw_version
Explanation
The makeFFTW script interrogates the config script to obtain the version. Since the version is ’...-system’, it recognises that it shouldn’t build anything.

Later when building things that need FFTW, the FFTW_ARCH_PATH is checked for an include/ directory that has an fftw3.h header. Since this fails, it checks if the FFTW_ARCH_PATH is "...-system" and then verifies that the /usr/include/fftw3.h system header exists.

Even although the FFTW_ARCH_PATH is invalid, the compile and link is successful since the standard paths for include and library do contain the required values.

FFTW from the central location, e.g. loaded via modules. Here, the environment values depend heavily on the system, but could resemble:

fftw_version=fftw-system
export FFTW_ARCH_PATH=$FFTW_HOME

where the FFTW_HOME is coming from the modules system in this particular example.

Explanation
As before, the makeFFTW script interrogates the config script and finds that it provides a ’...-system’ version and will not try to build anything.

Later when building things with FFTW, the FFTW_ARCH_PATH contains the needed include/ and library directories.

The final example is the easiest

fftw_version=fftw-none
export FFTW_ARCH_PATH=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER/$fftw_version

or

export FFTW_ARCH_PATH=/dev/null/$fftw_version
Explanation
This time the makeFFTW script finds that it provides a ’...-none’ version, which tells it not to build anything.

Later when building things with FFTW, the FFTW_ARCH_PATH is invalid, does not end with "...-system", so fftw will not be used.

usage: foamConfigurePaths
  -foamInstall dir        specify installation directory (eg, /opt)
  -foamVersion ver        specify project version (eg, 1612)
  -projectName name       specify project directory name (eg, openfoam1612)
  -archOption 32|64       specify 'WM_ARCH_OPTION' architecture option
  -int32  | -int64        specify 'WM_LABEL_SIZE'
  -SP     | -DP           specify 'WM_PRECISION_OPTION'
  -system name            specify 'system' compiler to be used
  -third  name            specify 'ThirdParty' compiler to be used
  -boost ver              specify 'boost_version'
  -boost-path dir         specify 'BOOST_ARCH_PATH'
  -cgal ver               specify 'cgal_version'
  -cgal-path dir          specify 'CGAL_ARCH_PATH'
  -clang ver              specify 'clang_version' for ThirdParty Clang
  -cmake ver              specify 'cmake_version'
  -fftw ver               specify 'fffw_version'
  -fftw-path dir          specify 'FFTW_ARCH_PATH'
  -metis ver              specify 'METIS_VERSION'
  -metis-path dir         specify 'METIS_ARCH_PATH'
  -paraview ver           specify 'ParaView_VERSION' (eg, 5.0.1)
  -paraview-path dir      specify 'ParaView_DIR' (eg, /opt/paraviewopenfoam3120)
  -openmpi ver            specify ThirdParty openmpi version for 'FOAM_MPI'
  -openmpi-system         activate system openmpi
  -openmpi-third          activate ThirdParty openmpi (using default version)
  -scotch ver             specify 'SCOTCH_VERSION' (eg, scotch_6.0.4)
  -scotch-path dir        specify 'SCOTCH_ARCH_PATH' (eg, /opt/OpenFOAM-scotch_6.0.4)
  -vtk  ver               specify 'vtk_version' (eg, VTK-7.1.0)
  -mesa ver               specify 'mesa_version' (eg, mesa-13.0.1)
  -sigfpe | -no-sigfpe    activate/deactivate FOAM_SIGFPE handling
  gmp-VERSION             for ThirdParty gcc (gmp-system for system library)
  mpfr-VERSION            for ThirdParty gcc (mpfr-system for system library)
  mpc-VERSION             for ThirdParty gcc (mpc-system for system library)

* Adjust hardcoded versions and installation paths (for bash, POSIX shell).