This is a templated class library. Thus there is no library to compile or link with. Just include the appropriate header files in your application code when you compile.
When the Lagrangian solvers have a solid mesh, the boundary of the mesh is communicated to the Eulerian solvers and the fluid pressures at the boundary nodes are communicated back to the Lagrangian solvers. The elc::LagrangianCommBoundary and elc::EulerianCommBoundary classes implement point-to-point communication between the solid and fluid.
When the Lagrangian solvers have a shell, the shell is sent to the Eulerian solvers and the pressure differences across the shell faces are sent back. The elc::LagrangianCommShell and elc::EulerianCommShell classes implement the point-to-point communication in this case.
At each time step, the fluid needs the position and velocity of the solid boundary. This is used in the Ghost Fluid Method (GFM) to enforce boundary conditions at the solid/fluid interface. In return, the solid solvers need the fluid pressure at the boundary nodes.
The solid mesh is distributed. Below we show a distribution of the boundary over 9 processors.
The distributed solid boundary.
The gather-send-broadcast algorithm.
The gather-send-broadcast algorithm communicates the solid boundary to the fluid in three steps:
Each fluid processor can then select the portion of the assembled solid boundary that is relevant to its domain and use the closest point transform (CPT) to build the level set description of the boundary.
Communicating the fluid pressures to the solid is done with analogous operations:
The gather-send-broadcast algorithm has a number of advantages:
The gather-send-brodcast is simple and flexible. However, for large problems it may have poor performance due to the need to assemble, store and communicate the global solid boundary. For the solid processors, assembling the global solid boundary may be costly (execution time) or infeasible (memory requirements) for large meshes. Note that each fluid processor receives the global boundary, though it only needs the portion that intersects its domain. Storing the global boundary may be infeasible; extracting the relevant portion of the boundary may be costly.
Each solid processor makes a bounding box around its portion of the boundary. Each fluid processor makes a bounding box that contains its region of interest. Any portion of the solid boundary in the region of interest could influence the fluid grid. These are illustrated below.
The solid and fluid bounding boxes.
The solid processors perform a gather operation to collect the bounding box information to the root. The fluid processors do the same.
Gather the solid and fluid bounding boxes.
Next the two root processors exchang the collected bounding boxes.
Exchange the collected solid and fluid bounding boxes.
Then the solid processors broadcast the fluid regions of interest and the fluid processors broadcast the solid boundary bounding boxes.
Broadcast the collected solid and fluid bounding boxes.
Now each solid processor has all the fluid regions of interest and each fluid processor has all the solid bounding boxes. Each solid processor finds the fluid regions of interest which overlap its bounding box. This determines the fluid processors to which it will send its portion of the boundary. Analogously, each fluid processor finds the solid bounding boxes which overlap its region of interest. This determines the solid processors from which it will receive portions of the boundary. The point-to-point communication from the solid to the fluid is illustrated below.
The point-to-point communication pattern.
Like the gather-send-broadcast approach, the point-to-point communication scheme does not depend on any special distribution of the data. For reasonable data distributions, it has a number of advantages:
elc::LagrangianComm<3,Number> lagrangianComm(MPI_COMM_WORLD, comm, numEulerian, eulerianRoot, elc::LocalIndices);
Number is probably
comm is the Lagrangian communicator,
numEulerian is the number of Eulerian processors, and
eulerianRoot is the rank of the root Eulerian processor in the world communicator. For the last argument one can either pass
elc::GlobalIdentifiers. This indicates how one will specify the mesh in sendMesh(). The connectivities array may either contain indices of the local mesh, or global node identifiers.
To send the mesh to the Eulerian processors, do the following.
lagrangianComm.sendMesh(numNodes, identifiers, positions, velocities, numFaces, connectivities); lagrangianComm.waitForMesh();
connectivities are arrays of data that define the mesh. One can perform computations between
waitForMesh(). Upon completion of
waitForMesh(), mesh array data has been copied into communication buffers, and thus the data may be modified.
To receive the pressure data from the Eulerian processors, use
waitForPressure(). Below we receive the pressure defined at the nodes.
lagrangianComm.receivePressure(numNodes, pressures); lagrangianComm.waitForPressure();
If the Lagrangian mesh is a shell, then the pressure is defined at the centroids of the faces.
lagrangianComm.receivePressure(numFaces, pressures); lagrangianComm.waitForPressure();
One can perform computations between
waitForPressure(). The former initiates non-blocking receives, so it is best to call this function before the Eulerian processors start sending the pressure.
elc::EulerianCommBoundary<3,Number> eulerianComm(MPI_COMM_WORLD, comm, numLagrangian, lagrangianRoot, elc::LocalIndices);
For coupling with a shell mesh, use:
elc::EulerianCommShell<3,Number> eulerianComm(MPI_COMM_WORLD, comm, numLagrangian, lagrangianRoot, elc::LocalIndices);
comm is the Eulerian communicator,
numLagrangian is the number of Lagrangian processors, and
lagrangianRoot is the rank of the root Lagrangian processor in the world communicator. For the last argument one can either pass
elc::GlobalIdentifiers. This indicates how the mesh patches are specified in the Lagrangian processors' call to sendMesh(). There, the connectivities array may either contain indices of the local mesh, or global node identifiers.
To receive the mesh from the Lagrangian processors, do the following.
region is an array of 6 floating point numbers that define the bounding box containing all of the processors' grids. One can perform computations between
After setting the pressure with the
getPressures() or the
getPressuresData() member functions, send the pressure to the relevant Lagrangian processors.