OpenFOAM® v3.0+: New Parallel Functionality

A new option for all parallel utilities

In all earlier versions of OpenFOAM, applications requiring decomposition information read the data from the decomposeParDict dictionary located within each case system directory. This made it difficult to execute different phases of the CFD work flow using different numbers of processors or different partitioning methods.

All parallel-aware utilities and solvers now accept a -decomposeParDict option, similar to the -dict option, to specify an alternative decomposition dictionary location, which can take the form of an alternative file or directory.

A possible workflow could look like:

Decomposing and reconstructing cases in parallel

The redistributePar utility can now be used to decompose and reconstruct models in parallel:

• all decomposition constraints, e.g. preserveFaceZones, are now integrated into the decomposition libraries, and therefore available to all applications;
• redistributePar will behave as decomposePar when run on an un-decomposed case (or if the -decompose command-line option is provided) and decompose the mesh and fields for a single time step;
• redistributePar will behave as reconstructParMesh and reconstructPar if the number of domains in the decomposeParDict dictionary is set to 1 or if the -reconstruct command-line option is provided. Similar to reconstructPar it will reconstruct all time steps; if neither the -decompose nor -reconstruct options are provided, it will operate in redistribution mode and re-distribute a single time step;
• it decomposes,redistributes and reconstructs mesh, volume fields, surface fields, and volume internal fields, e.g. source terms in lagrangian simulations, and lagrangian data.

Usage examples:

• To decompose the case in parallel
  

  mpirun -np 2 redistributePar -decompose -parallel
  
• To change the partitioning to the new decomposeParDict (number of processors or type of partitioning)
  

  mpirun -np 2 redistributePar -parallel
  
• To reconstruct the case in parallel
  

  mpirun -np 2 redistributePar -reconstruct -parallel
  

Note: This utility can only be run in parallel. Existing applications decomposePar and reconstructPar/reconstructParMesh are still available for serial operation.

Examples

cavity: $FOAM_TUTORIALS/mesh/parallel/cavity
filter: $FOAM_TUTORIALS/mesh/parallel/filter

Distributed geometry across processors

At the start of the meshing process, snappyHexMesh typically loads the complete STL file on every processor. Depending on the size of the STL file, this may saturate the memory available causing performance issues. surfaceRedistributePar was created in previous OpenFOAM versions to distribute the geometry across different processors, reducing the redundancy and hence the memory used. In this version, it has been improved for use in production.

Usage:

where mode can be:

• follow: attempts to decompose the surface according to the mesh, keeping the maximum amount of geometric queries local to the processor.
• independent: uses the current decomposition method to decompose the surface into equally-sized parts.

The independent method is generally preferable.

After decomposition of the geometry, the different files can be found under $FOAM_CASE/processor*/constant/triSurface

Once the geometry has been decomposed, it can be used in snappyHexMesh, using the syntax type distributedTriSurfaceMesh; instead of type triSurfaceMesh;, as follows: