Version of masterUncollatedFileOperation that collates regIOobjects into a container in the processors/ subdirectory. More...

Inheritance diagram for collatedFileOperation:

[legend]

Collaboration diagram for collatedFileOperation:

[legend]

Public Member Functions
	TypeName ("collated")
	Runtime type information. More...

	collatedFileOperation (bool verbose)
	Default construct. More...

	collatedFileOperation (const label comm, const labelList &ioRanks, const word &typeName, bool verbose)
	Construct from user communicator. More...

virtual	~collatedFileOperation ()
	Destructor. More...

virtual fileName	objectPath (const IOobject &io, const word &typeName) const
	Generate disk file name for object. Opposite of filePath. More...

virtual bool	writeObject (const regIOobject &, IOstreamOption streamOpt=IOstreamOption(), const bool valid=true) const
	Writes a regIOobject (so header, contents and divider). More...

virtual void	flush () const
	Forcibly wait until all output done. Flush any cached data. More...

virtual word	processorsDir (const IOobject &) const
	Actual name of processors dir. More...

virtual word	processorsDir (const fileName &) const
	Actual name of processors dir. More...

virtual void	setNProcs (const label nProcs)

Public Member Functions inherited from masterUncollatedFileOperation
	TypeName ("masterUncollated")
	Runtime type information. More...

	masterUncollatedFileOperation (bool verbose)
	Default construct. More...

	masterUncollatedFileOperation (const label comm, bool verbose)
	Construct from communicator. More...

virtual	~masterUncollatedFileOperation ()
	Destructor. More...

virtual bool	mkDir (const fileName &, mode_t=0777) const
	Make directory. More...

virtual bool	chMod (const fileName &, const mode_t) const
	Set the file mode. More...

virtual mode_t	mode (const fileName &, const bool followLink=true) const
	Return the file mode. More...

virtual fileName::Type	type (const fileName &, const bool followLink=true) const
	Return the file type: DIRECTORY, FILE or LINK. More...

virtual bool	exists (const fileName &, const bool checkGzip=true, const bool followLink=true) const
	Does the name exist (as DIRECTORY or FILE) in the file system? More...

virtual bool	isDir (const fileName &, const bool followLink=true) const
	Does the name exist as a DIRECTORY in the file system? More...

virtual bool	isFile (const fileName &, const bool checkGzip=true, const bool followLink=true) const
	Does the name exist as a FILE in the file system? More...

virtual off_t	fileSize (const fileName &, const bool followLink=true) const
	Return size of file. More...

virtual time_t	lastModified (const fileName &, const bool followLink=true) const
	Return time of last file modification. More...

virtual double	highResLastModified (const fileName &, const bool followLink=true) const
	Return time of last file modification. More...

virtual fileNameList	readDir (const fileName &, const fileName::Type=fileName::FILE, const bool filtergz=true, const bool followLink=true) const
	Read a directory and return the entries as a string list. More...

virtual bool	cp (const fileName &src, const fileName &dst, const bool followLink=true) const
	Copy, recursively if necessary, the source to the destination. More...

virtual bool	ln (const fileName &src, const fileName &dst) const
	Create a softlink. dst should not exist. Returns true if. More...

virtual bool	mv (const fileName &src, const fileName &dst, const bool followLink=false) const
	Rename src to dst. More...

virtual bool	mvBak (const fileName &, const std::string &ext="bak") const
	Rename to a corresponding backup file. More...

virtual bool	rm (const fileName &) const
	Remove a file, returning true if successful otherwise false. More...

virtual bool	rmDir (const fileName &dir, const bool silent=false) const
	Remove a directory and its contents. More...

virtual fileName	filePath (const bool checkGlobal, const IOobject &io, const word &typeName, const bool search) const
	Search for an object. checkGlobal : also check undecomposed case. More...

virtual fileName	dirPath (const bool checkGlobal, const IOobject &io, const bool search) const
	Search for a directory. checkGlobal : also check undecomposed. More...

virtual fileNameList	readObjects (const objectRegistry &db, const fileName &instance, const fileName &local, word &newInstance) const
	Search directory for objects. Used in IOobjectList. More...

virtual bool	readHeader (IOobject &, const fileName &, const word &typeName) const
	Read object header from supplied file. More...

virtual autoPtr< ISstream >	readStream (regIOobject &, const fileName &, const word &typeName, const bool valid=true) const
	Reads header for regIOobject and returns an ISstream. More...

virtual bool	read (regIOobject &, const bool masterOnly, const IOstreamOption::streamFormat format, const word &typeName) const
	Top-level read. More...

virtual autoPtr< ISstream >	NewIFstream (const fileName &) const
	Generate an ISstream that reads a file. More...

virtual autoPtr< OSstream >	NewOFstream (const fileName &pathname, IOstreamOption streamOpt=IOstreamOption(), const bool valid=true) const
	Generate an OSstream that writes a file. More...

virtual label	addWatch (const fileName &) const
	Add watching of a file. Returns handle. More...

virtual bool	removeWatch (const label) const
	Remove watch on a file (using handle) More...

virtual label	findWatch (const labelList &watchIndices, const fileName &) const
	Find index (or -1) of file in list of handles. More...

virtual void	addWatches (regIOobject &, const fileNameList &) const
	Helper: add watches for list of regIOobjects. More...

virtual fileName	getFile (const label) const
	Get name of file being watched (using handle) More...

virtual void	updateStates (const bool masterOnly, const bool syncPar) const
	Update state of all files. More...

virtual fileMonitor::fileState	getState (const label) const
	Get current state of file (using handle) More...

virtual void	setUnmodified (const label) const
	Set current state of file (using handle) to unmodified. More...

virtual instantList	findTimes (const fileName &, const word &) const
	Get sorted list of times. More...

virtual IOobject	findInstance (const IOobject &io, const scalar startValue, const word &stopInstance) const
	Find instance where IOobject is. Fails if cannot be found. More...

virtual void	setTime (const Time &) const
	Callback for time change. More...

const HashPtrTable< instantList > &	times () const
	Return cached times. More...

Public Member Functions inherited from fileOperation
	TypeName ("fileOperation")
	Runtime type information. More...

	fileOperation (const label comm, const bool distributedRoots=false)
	Construct from communicator, optionally with distributed roots. More...

	declareRunTimeSelectionTable (autoPtr, fileOperation, word,(bool verbose),(verbose))

virtual	~fileOperation ()=default
	Destructor. More...

bool	distributed () const noexcept
	Distributed roots (parallel run) More...

bool	distributed (bool on) const noexcept
	Set distributed roots on/off (mutable) More...

virtual fileName	filePath (const fileName &) const
	Search for a file or directory. Use IOobject version in. More...

virtual label	nProcs (const fileName &dir, const fileName &local="") const
	Get number of processor directories/results. Used for e.g. More...

fileName	processorsCasePath (const IOobject &, const word &procDir) const
	Generate path (like io.path) from root+casename with any. More...

fileName	processorsPath (const IOobject &, const word &instance, const word &procDir) const
	Generate path (like io.path) with provided instance and any. More...

fileName	processorsPath (const fileName &, const word &procDir) const
	Operating on fileName: replace processorXXX with procDir. More...

Static Public Attributes
static float	maxThreadFileBufferSize
	Max size of thread buffer size. This is the overall size of. More...

Static Public Attributes inherited from masterUncollatedFileOperation
static float	maxMasterFileBufferSize
	Max size of parallel communications. Switches from non-blocking. More...

Static Public Attributes inherited from fileOperation
static const Enum< pathType >	pathTypeNames_

static word	processorsBaseDir = "processors"
	Return the processors directory name (usually "processors") More...

static word	defaultFileHandler
	Name of the default fileHandler. More...

static autoPtr< fileOperation >	fileHandlerPtr_
	Static fileOperation. More...

Protected Member Functions
void	printBanner (const bool printRanks=false) const
	Print banner information, optionally with io ranks. More...

bool	isMasterRank (const label proci) const

bool	appendObject (const regIOobject &io, const fileName &pathName, IOstreamOption streamOpt) const
	Append to processorsNN/ file. More...

Protected Member Functions inherited from masterUncollatedFileOperation
template<class Type >
Type	scatterList (const UList< Type > &, const int, const label comm) const

template<class Type , class fileOp >
Type	masterOp (const fileName &, const fileOp &fop, const int tag, const label comm) const

template<class Type , class fileOp >
Type	masterOp (const fileName &, const fileName &, const fileOp &fop, const int tag, const label comm) const

virtual fileName	filePathInfo (const bool checkGlobal, const bool isFile, const IOobject &, const bool search, pathType &searchType, word &processorsDir, word &instance) const
	Search (locally!) for object; return info on how it was found. More...

fileName	localObjectPath (const IOobject &, const pathType &searchType, const word &processorsDir, const word &instancePath) const
	Construct filePath. More...

bool	exists (const dirIndexList &, IOobject &io) const
	Helper: check IO for local existence. Like filePathInfo but. More...

Protected Member Functions inherited from fileOperation
fileMonitor &	monitor () const
	Get or create fileMonitor singleton. More...

refPtr< dirIndexList >	lookupAndCacheProcessorsPath (const fileName &objectPath, const bool syncPar) const
	Lookup name of processorsDDD using cache. More...

virtual refPtr< dirIndexList >	lookupProcessorsPath (const fileName &objectPath) const
	Lookup name of processorsDDD using cache. More...

bool	exists (IOobject &io) const

Protected Attributes
const label	myComm_
	Any communicator allocated by me. More...

OFstreamCollator	writer_
	Threaded writer. More...

label	nProcs_
	Number of processors (overall) More...

const labelList	ioRanks_
	Ranks of IO handlers. More...

Protected Attributes inherited from masterUncollatedFileOperation
const label	myComm_
	Any communicator allocated by me. More...

HashPtrTable< instantList >	times_
	Cached times for a given directory. More...

Protected Attributes inherited from fileOperation
const label	comm_
	Communicator to use. More...

bool	distributed_
	Distributed roots (parallel run) More...

HashTable< dirIndexList >	procsDirs_
	Detected processors directories. More...

autoPtr< fileMonitor >	monitorPtr_
	File-change monitor for all registered files. More...

Additional Inherited Members
Public Types inherited from fileOperation
enum	pathType : int { NOTFOUND = 0, ABSOLUTE, OBJECT, WRITEOBJECT, PROCUNCOLLATED, PROCBASEOBJECT = PROCUNCOLLATED + 1, PROCOBJECT = PROCBASEOBJECT + 1, PARENTOBJECT, FINDINSTANCE, PROCUNCOLLATEDINSTANCE, PROCBASEINSTANCE, PROCINSTANCE }
	Enumeration for the location of an IOobject. More...

typedef Tuple2< fileName, Tuple2< pathType, int > >	dirIndex
	Augment fileName with pathType and local offset. More...

typedef List< dirIndex >	dirIndexList

typedef IntRange< int >	procRangeType

Static Public Member Functions inherited from masterUncollatedFileOperation
static bool	uniformFile (const fileNameList &)
	Same file? More...

Static Public Member Functions inherited from fileOperation
static autoPtr< fileOperation >	NewUncollated ()
	Static construct the commonly used uncollatedFileOperation. More...

static autoPtr< fileOperation >	New (const word &handlerType, bool verbose=false)
	Select fileHandler-type. More...

static instantList	sortTimes (const fileNameList &dirEntries, const word &constantName="constant")
	Sort directory entries according to time value,. More...

static label	splitProcessorPath (const fileName &objectPath, fileName &path, fileName &procDir, fileName &local, procRangeType &group, label &nProcs)
	Split objectPath into part before 'processor' and part after. More...

static label	detectProcessorPath (const fileName &objPath)
	Detect processor number from '/aa/bb/processorDDD/cc'. More...

Static Protected Member Functions inherited from masterUncollatedFileOperation
static labelList	subRanks (const label n)
	Get the list of processors that are part of this communicator. More...

static word	findInstancePath (const instantList &timeDirs, const instant &t)
	Equivalent of Time::findInstance. More...

static void	readAndSend (const fileName &filePath, const labelUList &procs, PstreamBuffers &pBufs)
	Read file contents and send to processors. More...

static autoPtr< ISstream >	read (IOobject &io, const label comm, const bool uniform, const fileNameList &filePaths, const boolList &procValid)
	Read files on comms master. More...

Static Protected Member Functions inherited from fileOperation
static labelList	ioRanks ()
	Retrieve list of IO ranks from FOAM_IORANKS env variable. More...

static void	mergeTimes (const instantList &extraTimes, const word &constantName, instantList &times)
	Merge two times. More...

static bool	isFileOrDir (const bool isFile, const fileName &)
	Helper: check for file (isFile) or directory (!isFile) More...

Detailed Description

Version of masterUncollatedFileOperation that collates regIOobjects into a container in the processors/ subdirectory.

Uses threading if maxThreadFileBufferSize != 0. > 0 : Can use mpi inside thread to collect data if buffer is not large enough. Does need full thread support inside MPI.

< 0 : special : -maxThreadFileBufferSize is guaranteed large enough for all writing. Initialises MPI without thread support.

See also: masterUncollatedFileOperation

Source files

Definition at line 65 of file collatedFileOperation.H.

Constructor & Destructor Documentation

◆ collatedFileOperation() [1/2]

collatedFileOperation ( bool verbose )

explicit

Default construct.

Definition at line 259 of file collatedFileOperation.C.

References Foam::infoDetailLevel.

◆ collatedFileOperation() [2/2]

collatedFileOperation	(	const label	comm,
		const labelList &	ioRanks,
		const word &	typeName,
		bool	verbose
	)

Construct from user communicator.

Definition at line 289 of file collatedFileOperation.C.

References Foam::infoDetailLevel.

◆ ~collatedFileOperation()

~collatedFileOperation ( )

virtual

Destructor.

Definition at line 311 of file collatedFileOperation.C.

References collatedFileOperation::flush(), UPstream::freeCommunicator(), collatedFileOperation::myComm_, and UPstream::worldComm.

Here is the call graph for this function:

Member Function Documentation

◆ printBanner()

void printBanner ( const bool printRanks = false ) const

protected

Print banner information, optionally with io ranks.

Definition at line 80 of file collatedFileOperation.C.

References DynamicList< T, SizeMin >::append(), DetailInfo, stdFoam::end(), Foam::endl(), forAll, Foam::constant::atomic::group, Foam::hostName(), and Foam::nl.

Here is the call graph for this function:

◆ isMasterRank()

bool isMasterRank ( const label proci ) const

protected

Is proci master of communicator (in parallel) or master of the io ranks (non-parallel)

Definition at line 165 of file collatedFileOperation.C.

◆ appendObject()

bool appendObject	(	const regIOobject &	io,
		const fileName &	pathName,
		IOstreamOption	streamOpt
	)		const

protected

Append to processorsNN/ file.

Definition at line 188 of file collatedFileOperation.C.

References Foam::expressions::patchExpr::debug, Foam::endl(), Foam::exit(), Foam::FatalError, FatalErrorInFunction, Foam::FatalIOError, FatalIOErrorInFunction, IOobject::name(), IOobject::objectPath(), os(), Foam::Pout, IOstreamOption::version(), and Foam::writeHeader().

Here is the call graph for this function:

◆ TypeName()

TypeName ( "collated" )

Runtime type information.

◆ objectPath()

Foam::fileName objectPath	(	const IOobject &	io,
		const word &	typeName
	)		const

virtual

Generate disk file name for object. Opposite of filePath.

Reimplemented from fileOperation.

Definition at line 326 of file collatedFileOperation.C.

References IOobject::instance(), masterUncollatedFileOperation::localObjectPath(), word::null, fileOperation::OBJECT, TimePaths::processorCase(), fileOperation::PROCOBJECT, and IOobject::time().

Here is the call graph for this function:

◆ writeObject()

bool writeObject	(	const regIOobject &	io,
		IOstreamOption	streamOpt = `IOstreamOption()`,
		const bool	valid = `true`
	)		const

virtual

Writes a regIOobject (so header, contents and divider).

Returns success state.

Reimplemented from masterUncollatedFileOperation.

Definition at line 356 of file collatedFileOperation.C.

References IOobject::bannerEnabled(), Foam::expressions::patchExpr::debug, dict, Foam::endl(), regIOobject::global(), IOobject::instance(), fileName::isAbsolute(), UPstream::master(), Foam::mkDir(), IOobject::name(), IOobject::objectPath(), os(), UPstream::parRun(), path(), IOobject::path(), Foam::Pout, TimePaths::processorCase(), IOobject::time(), regIOobject::writeData(), IOobject::writeEndDivider(), decomposedBlockData::writeExtraHeaderContent(), and IOobject::writeHeader().

Here is the call graph for this function:

◆ flush()

void flush ( ) const

virtual

Forcibly wait until all output done. Flush any cached data.

Reimplemented from masterUncollatedFileOperation.

Definition at line 519 of file collatedFileOperation.C.

References Foam::expressions::patchExpr::debug, Foam::endl(), masterUncollatedFileOperation::flush(), and Foam::Pout.

Referenced by collatedFileOperation::~collatedFileOperation().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ processorsDir() [1/2]

Foam::word processorsDir ( const IOobject & io ) const

virtual

Actual name of processors dir.

Reimplemented from fileOperation.

Definition at line 597 of file collatedFileOperation.C.

References IOobject::objectPath().

Here is the call graph for this function:

◆ processorsDir() [2/2]

Foam::word processorsDir ( const fileName & fName ) const

virtual

Actual name of processors dir.

Reimplemented from fileOperation.

Definition at line 533 of file collatedFileOperation.C.

References Foam::name(), UPstream::nProcs(), UPstream::parRun(), and UPstream::procID().

Here is the call graph for this function:

◆ setNProcs()

void setNProcs ( const label nProcs )

virtual

Set number of processor directories/results. Only used in decomposePar

Reimplemented from fileOperation.

Definition at line 605 of file collatedFileOperation.C.

References Foam::expressions::patchExpr::debug, Foam::endl(), and Foam::Pout.

Here is the call graph for this function:

Member Data Documentation

◆ myComm_

const label myComm_

protected

Any communicator allocated by me.

Definition at line 74 of file collatedFileOperation.H.

Referenced by collatedFileOperation::~collatedFileOperation().

◆ writer_

OFstreamCollator writer_

mutableprotected

Threaded writer.

Definition at line 77 of file collatedFileOperation.H.

◆ nProcs_

label nProcs_

protected

Number of processors (overall)

Definition at line 82 of file collatedFileOperation.H.

◆ ioRanks_

const labelList ioRanks_

protected

Ranks of IO handlers.

Definition at line 85 of file collatedFileOperation.H.

◆ maxThreadFileBufferSize

float maxThreadFileBufferSize

static

Max size of thread buffer size. This is the overall size of.

all files to be written. Starts blocking if not enough size. Read as float to enable easy specification of large sizes.

Definition at line 117 of file collatedFileOperation.H.

Referenced by collatedFileOperationInitialise::needsThreading().

The documentation for this class was generated from the following files:

src/OpenFOAM/global/fileOperations/collatedFileOperation/collatedFileOperation.H
src/OpenFOAM/global/fileOperations/collatedFileOperation/collatedFileOperation.C

Public Member Functions

Static Public Attributes

Protected Member Functions

Protected Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ collatedFileOperation() [1/2]

◆ collatedFileOperation() [2/2]

◆ ~collatedFileOperation()

Member Function Documentation

◆ printBanner()

◆ isMasterRank()

◆ appendObject()

◆ TypeName()

◆ objectPath()

◆ writeObject()

◆ flush()

◆ processorsDir() [1/2]

◆ processorsDir() [2/2]

◆ setNProcs()

Member Data Documentation

◆ myComm_

◆ writer_

◆ nProcs_

◆ ioRanks_

◆ maxThreadFileBufferSize