Version of masterUncollatedFileOperation that collates regIOobjects into a container in the processors/ subdirectory. More...
Public Member Functions | |
TypeName ("collated") | |
Runtime type information. More... | |
collatedFileOperation (bool verbose) | |
Default construct. More... | |
collatedFileOperation (const label comm, const labelList &ioRanks, const word &typeName, bool verbose) | |
Construct from user communicator. More... | |
virtual | ~collatedFileOperation () |
Destructor. More... | |
virtual fileName | objectPath (const IOobject &io, const word &typeName) const |
Generate disk file name for object. Opposite of filePath. More... | |
virtual bool | writeObject (const regIOobject &, IOstreamOption streamOpt=IOstreamOption(), const bool valid=true) const |
Writes a regIOobject (so header, contents and divider). More... | |
virtual void | flush () const |
Forcibly wait until all output done. Flush any cached data. More... | |
virtual word | processorsDir (const IOobject &) const |
Actual name of processors dir. More... | |
virtual word | processorsDir (const fileName &) const |
Actual name of processors dir. More... | |
virtual void | setNProcs (const label nProcs) |
Public Member Functions inherited from masterUncollatedFileOperation | |
TypeName ("masterUncollated") | |
Runtime type information. More... | |
masterUncollatedFileOperation (bool verbose) | |
Default construct. More... | |
masterUncollatedFileOperation (const label comm, bool verbose) | |
Construct from communicator. More... | |
virtual | ~masterUncollatedFileOperation () |
Destructor. More... | |
virtual bool | mkDir (const fileName &, mode_t=0777) const |
Make directory. More... | |
virtual bool | chMod (const fileName &, const mode_t) const |
Set the file mode. More... | |
virtual mode_t | mode (const fileName &, const bool followLink=true) const |
Return the file mode. More... | |
virtual fileName::Type | type (const fileName &, const bool followLink=true) const |
Return the file type: DIRECTORY, FILE or SYMLINK. More... | |
virtual bool | exists (const fileName &, const bool checkGzip=true, const bool followLink=true) const |
Does the name exist (as DIRECTORY or FILE) in the file system? More... | |
virtual bool | isDir (const fileName &, const bool followLink=true) const |
Does the name exist as a DIRECTORY in the file system? More... | |
virtual bool | isFile (const fileName &, const bool checkGzip=true, const bool followLink=true) const |
Does the name exist as a FILE in the file system? More... | |
virtual off_t | fileSize (const fileName &, const bool followLink=true) const |
Return size of file. More... | |
virtual time_t | lastModified (const fileName &, const bool followLink=true) const |
Return time of last file modification. More... | |
virtual double | highResLastModified (const fileName &, const bool followLink=true) const |
Return time of last file modification. More... | |
virtual fileNameList | readDir (const fileName &, const fileName::Type=fileName::FILE, const bool filtergz=true, const bool followLink=true) const |
Read a directory and return the entries as a string list. More... | |
virtual bool | cp (const fileName &src, const fileName &dst, const bool followLink=true) const |
Copy, recursively if necessary, the source to the destination. More... | |
virtual bool | ln (const fileName &src, const fileName &dst) const |
Create a softlink. dst should not exist. Returns true if. More... | |
virtual bool | mv (const fileName &src, const fileName &dst, const bool followLink=false) const |
Rename src to dst. More... | |
virtual bool | mvBak (const fileName &, const std::string &ext="bak") const |
Rename to a corresponding backup file. More... | |
virtual bool | rm (const fileName &) const |
Remove a file, returning true if successful otherwise false. More... | |
virtual bool | rmDir (const fileName &dir, const bool silent=false) const |
Remove a directory and its contents. More... | |
virtual fileName | filePath (const bool checkGlobal, const IOobject &io, const word &typeName, const bool search) const |
Search for an object. checkGlobal : also check undecomposed case. More... | |
virtual fileName | dirPath (const bool checkGlobal, const IOobject &io, const bool search) const |
Search for a directory. checkGlobal : also check undecomposed. More... | |
virtual fileNameList | readObjects (const objectRegistry &db, const fileName &instance, const fileName &local, word &newInstance) const |
Search directory for objects. Used in IOobjectList. More... | |
virtual bool | readHeader (IOobject &, const fileName &, const word &typeName) const |
Read object header from supplied file. More... | |
virtual autoPtr< ISstream > | readStream (regIOobject &, const fileName &, const word &typeName, const bool valid=true) const |
Reads header for regIOobject and returns an ISstream. More... | |
virtual bool | read (regIOobject &, const bool masterOnly, const IOstreamOption::streamFormat format, const word &typeName) const |
Top-level read. More... | |
virtual bool | writeObject (const regIOobject &io, IOstreamOption streamOpt=IOstreamOption(), const bool valid=true) const |
Writes a regIOobject (so header, contents and divider). More... | |
virtual autoPtr< ISstream > | NewIFstream (const fileName &) const |
Generate an ISstream that reads a file. More... | |
virtual autoPtr< OSstream > | NewOFstream (const fileName &pathname, IOstreamOption streamOpt=IOstreamOption(), const bool valid=true) const |
Generate an OSstream that writes a file. More... | |
virtual label | addWatch (const fileName &) const |
Add watching of a file. Returns handle. More... | |
virtual bool | removeWatch (const label) const |
Remove watch on a file (using handle) More... | |
virtual label | findWatch (const labelList &watchIndices, const fileName &) const |
Find index (or -1) of file in list of handles. More... | |
virtual void | addWatches (regIOobject &, const fileNameList &) const |
Helper: add watches for list of regIOobjects. More... | |
virtual fileName | getFile (const label) const |
Get name of file being watched (using handle) More... | |
virtual void | updateStates (const bool masterOnly, const bool syncPar) const |
Update state of all files. More... | |
virtual fileMonitor::fileState | getState (const label) const |
Get current state of file (using handle) More... | |
virtual void | setUnmodified (const label) const |
Set current state of file (using handle) to unmodified. More... | |
virtual instantList | findTimes (const fileName &, const word &) const |
Get sorted list of times. More... | |
virtual IOobject | findInstance (const IOobject &io, const scalar startValue, const word &stopInstance) const |
Find instance where IOobject is. More... | |
virtual void | setTime (const Time &) const |
Callback for time change. More... | |
virtual void | flush () const |
Forcibly wait until all output done. Flush any cached data. More... | |
const HashPtrTable< DynamicList< instant > > & | times () const noexcept |
Return cached times. More... | |
Public Member Functions inherited from fileOperation | |
TypeName ("fileOperation") | |
Runtime type information. More... | |
fileOperation (const label comm, const bool distributedRoots=false) | |
Construct from communicator, optionally with distributed roots. More... | |
declareRunTimeSelectionTable (autoPtr, fileOperation, word,(bool verbose),(verbose)) | |
virtual | ~fileOperation ()=default |
Destructor. More... | |
bool | distributed () const noexcept |
Distributed roots (parallel run) More... | |
bool | distributed (bool on) const noexcept |
Set distributed roots on/off (mutable) More... | |
virtual bool | mkDir (const fileName &, mode_t=0777) const =0 |
Make directory. More... | |
virtual bool | chMod (const fileName &, const mode_t) const =0 |
Set the file mode. More... | |
virtual mode_t | mode (const fileName &, const bool followLink=true) const =0 |
Return the file mode. More... | |
virtual fileName::Type | type (const fileName &, const bool followLink=true) const =0 |
Return the file type: DIRECTORY, FILE or SYMLINK. More... | |
virtual bool | exists (const fileName &, const bool checkGzip=true, const bool followLink=true) const =0 |
Does the name exist (as DIRECTORY or FILE) in the file system? More... | |
virtual bool | isDir (const fileName &, const bool followLink=true) const =0 |
Does the name exist as a DIRECTORY in the file system? More... | |
virtual bool | isFile (const fileName &, const bool checkGzip=true, const bool followLink=true) const =0 |
Does the name exist as a FILE in the file system? More... | |
virtual off_t | fileSize (const fileName &, const bool followLink=true) const =0 |
Return size of file. More... | |
virtual time_t | lastModified (const fileName &, const bool followLink=true) const =0 |
Return time of last file modification. More... | |
virtual double | highResLastModified (const fileName &, const bool followLink=true) const =0 |
Return time of last file modification. More... | |
virtual fileNameList | readDir (const fileName &, const fileName::Type=fileName::FILE, const bool filtergz=true, const bool followLink=true) const =0 |
Read a directory and return the entries as a string list. More... | |
virtual bool | cp (const fileName &src, const fileName &dst, const bool followLink=true) const =0 |
Copy, recursively if necessary, the source to the destination. More... | |
virtual bool | ln (const fileName &src, const fileName &dst) const =0 |
Create a softlink. dst should not exist. Returns true if. More... | |
virtual bool | mv (const fileName &src, const fileName &dst, const bool followLink=false) const =0 |
Rename src to dst. More... | |
virtual bool | mvBak (const fileName &, const std::string &ext="bak") const =0 |
Rename to a corresponding backup file. More... | |
virtual bool | rm (const fileName &) const =0 |
Remove a file, returning true if successful otherwise false. More... | |
virtual bool | rmDir (const fileName &dir, const bool silent=false) const =0 |
Remove a directory and its contents. More... | |
virtual fileName | objectPath (const IOobject &io, const word &typeName) const |
Generate disk file name for object. Opposite of filePath. More... | |
virtual fileName | filePath (const bool checkGlobal, const IOobject &, const word &typeName, const bool search=true) const =0 |
Search for an object. checkGlobal : also check undecomposed case. More... | |
virtual fileName | dirPath (const bool checkGlobal, const IOobject &io, const bool search=true) const =0 |
Search for a directory. checkGlobal : also check undecomposed. More... | |
virtual fileNameList | readObjects (const objectRegistry &db, const fileName &instance, const fileName &local, word &newInstance) const |
Search directory for objects. Used in IOobjectList. More... | |
virtual bool | readHeader (IOobject &, const fileName &, const word &typeName) const =0 |
Read object header from supplied file. More... | |
virtual autoPtr< ISstream > | readStream (regIOobject &, const fileName &, const word &typeName, const bool valid=true) const =0 |
Reads header for regIOobject and returns an ISstream. More... | |
virtual bool | read (regIOobject &, const bool masterOnly, const IOstreamOption::streamFormat format, const word &typeName) const =0 |
Top-level read. More... | |
virtual bool | writeObject (const regIOobject &io, IOstreamOption streamOpt=IOstreamOption(), const bool valid=true) const |
Writes a regIOobject (so header, contents and divider). More... | |
virtual fileName | filePath (const fileName &) const |
Search for a file or directory. Use IOobject version in. More... | |
virtual autoPtr< ISstream > | NewIFstream (const fileName &) const =0 |
Generate an ISstream that reads a file. More... | |
virtual autoPtr< OSstream > | NewOFstream (const fileName &pathname, IOstreamOption streamOpt=IOstreamOption(), const bool valid=true) const =0 |
Generate an OSstream that writes a file. More... | |
virtual label | addWatch (const fileName &) const |
Add watching of a file. Returns handle. More... | |
virtual bool | removeWatch (const label) const |
Remove watch on a file (using handle) More... | |
virtual label | findWatch (const labelList &watchIndices, const fileName &) const |
Find index (or -1) of file in list of handles. More... | |
virtual void | addWatches (regIOobject &, const fileNameList &) const |
Helper: add watches for list of regIOobjects. More... | |
virtual fileName | getFile (const label) const |
Get name of file being watched (using handle) More... | |
virtual void | updateStates (const bool masterOnly, const bool syncPar) const |
Update state of all files. More... | |
virtual fileMonitor::fileState | getState (const label) const |
Get current state of file (using handle) More... | |
virtual void | setUnmodified (const label) const |
Set current state of file (using handle) to unmodified. More... | |
virtual word | processorsDir (const IOobject &io) const |
Actual name of processors dir (for use in mode PROCOBJECT,. More... | |
virtual word | processorsDir (const fileName &) const |
Actual name of processors dir (for use in mode PROCOBJECT,. More... | |
virtual void | setNProcs (const label nProcs) |
Set number of processor directories/results. Only used in. More... | |
virtual label | nProcs (const fileName &dir, const fileName &local="") const |
Get number of processor directories/results. Used for e.g. More... | |
virtual instantList | findTimes (const fileName &, const word &) const |
Get sorted list of times. More... | |
virtual IOobject | findInstance (const IOobject &io, const scalar startValue, const word &stopInstance) const |
Find instance where IOobject is. More... | |
virtual void | setTime (const Time &) const |
Callback for time change. More... | |
virtual void | flush () const |
Forcibly wait until all output done. Flush any cached data. More... | |
fileName | processorsCasePath (const IOobject &, const word &procDir) const |
Generate path (like io.path) from root+casename with any. More... | |
fileName | processorsPath (const IOobject &, const word &instance, const word &procDir) const |
Generate path (like io.path) with provided instance and any. More... | |
fileName | processorsPath (const fileName &, const word &procDir) const |
Operating on fileName: replace processorXXX with procDir. More... | |
Static Public Attributes | |
static float | maxThreadFileBufferSize |
Max size of thread buffer size. This is the overall size of. More... | |
Static Public Attributes inherited from masterUncollatedFileOperation | |
static float | maxMasterFileBufferSize |
Max size of parallel communications. Switches from non-blocking. More... | |
Static Public Attributes inherited from fileOperation | |
static const Enum< pathType > | pathTypeNames_ |
static word | processorsBaseDir = "processors" |
Return the processors directory name (usually "processors") More... | |
static word | defaultFileHandler |
Name of the default fileHandler. More... | |
static autoPtr< fileOperation > | fileHandlerPtr_ |
Static fileOperation. More... | |
Protected Member Functions | |
void | printBanner (const bool printRanks=false) const |
Print banner information, optionally with io ranks. More... | |
bool | isMasterRank (const label proci) const |
bool | appendObject (const regIOobject &io, const fileName &pathName, IOstreamOption streamOpt) const |
Append to processorsNN/ file. More... | |
Protected Member Functions inherited from masterUncollatedFileOperation | |
template<class Type > | |
Type | scatterList (const UList< Type > &, const int, const label comm) const |
template<class Type , class FileOp > | |
Type | masterOp (const fileName &fName, const FileOp &fop, const int tag, const label comm) const |
template<class Type , class FileOp > | |
Type | masterOp (const fileName &src, const fileName &dest, const FileOp &fop, const int tag, const label comm) const |
virtual fileName | filePathInfo (const bool checkGlobal, const bool isFile, const IOobject &, const bool search, pathType &searchType, word &processorsDir, word &instance) const |
Search (locally!) for object; return info on how it was found. More... | |
fileName | localObjectPath (const IOobject &, const pathType &searchType, const word &processorsDir, const word &instancePath) const |
Construct filePath. More... | |
bool | exists (const dirIndexList &, IOobject &io) const |
Helper: check IO for local existence. Like filePathInfo but. More... | |
Protected Member Functions inherited from fileOperation | |
fileMonitor & | monitor () const |
Get or create fileMonitor singleton. More... | |
refPtr< dirIndexList > | lookupAndCacheProcessorsPath (const fileName &objectPath, const bool syncPar) const |
Lookup name of processorsDDD using cache. More... | |
virtual refPtr< dirIndexList > | lookupProcessorsPath (const fileName &objectPath) const |
Lookup name of processorsDDD using cache. More... | |
bool | exists (IOobject &io) const |
Protected Attributes | |
const label | myComm_ |
Any communicator allocated by me. More... | |
OFstreamCollator | writer_ |
Threaded writer. More... | |
label | nProcs_ |
Number of processors (overall) More... | |
const labelList | ioRanks_ |
Ranks of IO handlers. More... | |
Protected Attributes inherited from masterUncollatedFileOperation | |
const label | myComm_ |
Any communicator allocated by me. More... | |
HashPtrTable< DynamicList< instant > > | times_ |
Cached times for a given directory. More... | |
Protected Attributes inherited from fileOperation | |
const label | comm_ |
Communicator to use. More... | |
bool | distributed_ |
Distributed roots (parallel run) More... | |
HashTable< dirIndexList > | procsDirs_ |
Detected processors directories. More... | |
autoPtr< fileMonitor > | monitorPtr_ |
File-change monitor for all registered files. More... | |
Additional Inherited Members | |
Public Types inherited from fileOperation | |
enum | pathType : int { NOTFOUND = 0 , ABSOLUTE , OBJECT , WRITEOBJECT , PROCUNCOLLATED , PROCBASEOBJECT = PROCUNCOLLATED + 1 , PROCOBJECT = PROCBASEOBJECT + 1 , PARENTOBJECT , FINDINSTANCE , PROCUNCOLLATEDINSTANCE , PROCBASEINSTANCE , PROCINSTANCE } |
Enumeration for the location of an IOobject. More... | |
typedef Tuple2< fileName, Tuple2< pathType, int > > | dirIndex |
Augment fileName with pathType and local offset. More... | |
typedef List< dirIndex > | dirIndexList |
typedef IntRange< int > | procRangeType |
Static Public Member Functions inherited from masterUncollatedFileOperation | |
static bool | uniformFile (const fileNameList &) |
Same file? More... | |
Static Public Member Functions inherited from fileOperation | |
static autoPtr< fileOperation > | NewUncollated () |
Static construct the commonly used uncollatedFileOperation. More... | |
static autoPtr< fileOperation > | New (const word &handlerType, bool verbose=false) |
Select fileHandler-type. More... | |
static instantList | sortTimes (const fileNameList &dirEntries, const word &constantName="constant") |
Sort directory entries according to time value,. More... | |
static label | splitProcessorPath (const fileName &objectPath, fileName &path, fileName &procDir, fileName &local, procRangeType &group, label &nProcs) |
Split objectPath into part before 'processor' and part after. More... | |
static label | detectProcessorPath (const fileName &objPath) |
Detect processor number from '/aa/bb/processorDDD/cc'. More... | |
Static Protected Member Functions inherited from masterUncollatedFileOperation | |
static labelList | subRanks (const label n) |
Get the list of processors that are part of this communicator. More... | |
static word | findInstancePath (const instantList &timeDirs, const instant &t) |
Equivalent of Time::findInstance. More... | |
static void | readAndSend (const fileName &filePath, const labelUList &procs, PstreamBuffers &pBufs) |
Read file contents and send to processors. More... | |
static autoPtr< ISstream > | read (IOobject &io, const label comm, const bool uniform, const fileNameList &filePaths, const boolList &procValid) |
Read files on comms master. More... | |
Static Protected Member Functions inherited from fileOperation | |
static labelList | ioRanks () |
Retrieve list of IO ranks from FOAM_IORANKS env variable. More... | |
static void | mergeTimes (const instantList &extraTimes, const word &constantName, instantList ×) |
Merge two times. More... | |
static bool | isFileOrDir (const bool isFile, const fileName &) |
Helper: check for file (isFile) or directory (!isFile) More... | |
Version of masterUncollatedFileOperation that collates regIOobjects into a container in the processors/ subdirectory.
Uses threading if maxThreadFileBufferSize != 0. > 0 : Can use mpi inside thread to collect data if buffer is not large enough. Does need full thread support inside MPI.
< 0 : special : -maxThreadFileBufferSize is guaranteed large enough for all writing. Initialises MPI without thread support.
Definition at line 65 of file collatedFileOperation.H.
|
explicit |
Default construct.
Definition at line 256 of file collatedFileOperation.C.
References Foam::infoDetailLevel, collatedFileOperation::ioRanks_, collatedFileOperation::printBanner(), and UList< T >::size().
collatedFileOperation | ( | const label | comm, |
const labelList & | ioRanks, | ||
const word & | typeName, | ||
bool | verbose | ||
) |
Construct from user communicator.
Definition at line 286 of file collatedFileOperation.C.
References Foam::infoDetailLevel, collatedFileOperation::ioRanks_, collatedFileOperation::printBanner(), and UList< T >::size().
|
virtual |
Destructor.
Definition at line 309 of file collatedFileOperation.C.
References Foam::flush(), UPstream::freeCommunicator(), and UPstream::worldComm.
|
protected |
Print banner information, optionally with io ranks.
Definition at line 79 of file collatedFileOperation.C.
References DynamicList< T, SizeMin >::append(), fileOperation::comm_, DetailInfo, Foam::endl(), forAll, Pstream::gatherList(), Foam::hostName(), splitCell::master(), collatedFileOperation::maxThreadFileBufferSize, processorFaPatch::myProcNo(), Foam::nl, PstreamBuffers::nProcs(), UList< T >::size(), and masterUncollatedFileOperation::type().
Referenced by collatedFileOperation::collatedFileOperation(), and hostCollatedFileOperation::hostCollatedFileOperation().
|
protected |
Is proci master of communicator (in parallel) or master of the io ranks (non-parallel)
Definition at line 162 of file collatedFileOperation.C.
References splitCell::master(), and UPstream::parRun().
|
protected |
Append to processorsNN/ file.
Definition at line 185 of file collatedFileOperation.C.
References STLCore::BINARY, Foam::endl(), Foam::exit(), Foam::FatalError, FatalErrorInFunction, Foam::FatalIOError, FatalIOErrorInFunction, io(), os(), Foam::Pout, IOstreamOption::version(), and decomposedBlockData::writeBlockEntry().
TypeName | ( | "collated" | ) |
Runtime type information.
|
virtual |
Generate disk file name for object. Opposite of filePath.
Reimplemented from fileOperation.
Definition at line 323 of file collatedFileOperation.C.
References io(), masterUncollatedFileOperation::localObjectPath(), fileOperation::OBJECT, and fileOperation::PROCOBJECT.
|
virtual |
Writes a regIOobject (so header, contents and divider).
Returns success state.
Reimplemented from masterUncollatedFileOperation.
Definition at line 353 of file collatedFileOperation.C.
References IOobject::bannerEnabled(), dict, Foam::endl(), io(), fileName::isAbsolute(), splitCell::master(), Foam::mkDir(), os(), UPstream::parRun(), path(), Foam::Pout, TimePaths::processorCase(), IOobject::writeEndDivider(), and decomposedBlockData::writeExtraHeaderContent().
|
virtual |
Forcibly wait until all output done. Flush any cached data.
Reimplemented from masterUncollatedFileOperation.
Definition at line 517 of file collatedFileOperation.C.
References Foam::endl(), masterUncollatedFileOperation::flush(), and Foam::Pout.
|
virtual |
Actual name of processors dir.
Reimplemented from fileOperation.
Definition at line 594 of file collatedFileOperation.C.
References io().
|
virtual |
Actual name of processors dir.
Reimplemented from fileOperation.
Definition at line 530 of file collatedFileOperation.C.
References UList< T >::first(), UList< T >::last(), Foam::name(), PstreamBuffers::nProcs(), UPstream::parRun(), and UList< T >::size().
|
virtual |
Set number of processor directories/results. Only used in decomposePar
Reimplemented from fileOperation.
Definition at line 603 of file collatedFileOperation.C.
References Foam::endl(), and Foam::Pout.
|
protected |
Any communicator allocated by me.
Definition at line 74 of file collatedFileOperation.H.
|
mutableprotected |
Threaded writer.
Definition at line 77 of file collatedFileOperation.H.
|
protected |
Number of processors (overall)
Definition at line 82 of file collatedFileOperation.H.
|
protected |
Ranks of IO handlers.
Definition at line 85 of file collatedFileOperation.H.
Referenced by collatedFileOperation::collatedFileOperation(), and hostCollatedFileOperation::hostCollatedFileOperation().
|
static |
Max size of thread buffer size. This is the overall size of.
all files to be written. Starts blocking if not enough size. Read as float to enable easy specification of large sizes.
Definition at line 117 of file collatedFileOperation.H.
Referenced by collatedFileOperationInitialise::needsThreading(), and collatedFileOperation::printBanner().