v2106: New and improved parallel operation

New AMI support for local worlds

TOP

Multiple-world coupling has been extended to support non-conformal interfaces, i.e. for Arbitrary Mesh Interface (AMI) coupled patches.

Usage

The set-up requires the sampleMode to be set to nearestPatchFaceAMI instead of nearestPatchFace:

type                mappedWall;
sampleMode          nearestPatchFaceAMI;
sampleWorld         solid;
sampleRegion        bottomSolid;
samplePatch         top;

In the figure blow, the left boundary condition is set to a value of 1, the right 0, and it is solving a laplacian equation:

Tutorials:

Source code

Improved start-up time

TOP

Handling of the traditional OpenFOAM uncollated processor directories has been updated to improve startup times on large systems. Slow startup times are not necessarily apparent at lower processor counts, but can become a significant problem with larger numbers of processors.

The revised handling yields approximately a 70x speed-up of initial startup times at higher processor counts, as shown in the table below.

Configuration 3072 procs 6144 procs 12288 procs 24576 procs
old (uncollated) 212s 729s 2441s timeout
updated (uncollated) - - 33s 80s

These improvements in startup times should apply for various types of filing systems.

Acknowledgements

• This completes the changes initiated for OpenFOAM-v2012, which were based on the analyses and patches kindly provided by Akira AZAMI (RIST).

Improved redistributePar performance

TOP

redistributePar is a utility to decompose, redistribute and reconstruct cases (mesh and fields) in parallel. It can be used as a replacement for decomposePar, reconstructParMesh, and reconstructPar. The underlying algorithm to assemble the mesh and fields has been updated to a single step algorithm, yielding significant improvements when reconstructing large cases. The following table shows the timing results for a 14M cell case, running on 56 cores, timing in minutes:

Performance comparison
Application v2012 v2106
reconstructParMesh 4.32 2.58
redistributePar -reconstruct 11.40 3.34

Testing redistributePar -reconstruct only on a different case:

Scalability
Number of cells Number of processors v2012 v2106
1000 8 2s 5s
1000000 8 15s 13s
100000000 8 28m 23m
100000000 80 1h15m 19m

Notes

• the new algorithm does not require an additional point merging step so the -mergeTol optional argument is no longer required.
• this also means that potentially the mesh can have a slightly different point-connectivity in extreme circumstances. Note that it now strictly enforces the rule that a point connected to another point through a processor (face) patch is topologically the same point. So if both points end up on the same processor they will get merged automatically.
• support for the old algorithm is maintained in reconstructParMesh: reconstructParMesh -procMatch . This will likely be deprecated in a future release.

Tutorials

Source code

Improved decomposePar utility

TOP

Additional -dry-run options have been added to decomposePar: with -domains and -method. This enables quick exploration of decomposition behaviour and statistics without actually performing a decomposition. When -dry-run is used in combination with -cellDist, a VTK file is generated for visualization purposes.

This Avoids adding any OpenFOAM fields and is usually faster to load. It is also easier to rename than, e.g. a volScalarField, when exploring multiple decompositions.