inciter::DG class

DG Charm++ chare array used to advance PDEs in time with DG+RK.

Public static functions

static void registerReducers()
Configure Charm++ reduction types for concatenating BC nodelists.

Constructors, destructors, conversion operators

DG(CkMigrateMessage* msg) explicit
Migrate constructor.

Public functions

auto DG(const CProxy_Discretization& disc, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>&, const std::vector<std::size_t>& triinpoel) -> DG_SDAG_CODE explicit
Constructor.
void ResumeFromSync() override
Return from migration.
void resizeComm()
Start sizing communication buffers and setting up ghost data.
void comfac(int fromch, const tk::UnsMesh::FaceSet& infaces)
Receive unique set of faces we potentially share with/from another chare.
void comGhost(int fromch, const GhostData& ghost)
Receive ghost data on chare boundaries from fellow chare.
void reqGhost()
Receive requests for ghost data.
void sendGhost()
Send all of our ghost data to fellow chares.
void nodeNeighSetup()
Setup node-neighborhood (esup)
void comEsup(int fromch, const std::unordered_map<std::size_t, std::vector<std::size_t>>& bndEsup, const std::unordered_map<std::size_t, std::vector<tk::real>>& nodeBndCells)
Receive element-surr-points data on chare boundaries from fellow chare.
void setup()
Setup: query boundary conditions, output mesh, etc.
void box(tk::real v)
Receive total box IC volume and set conditions in box.
void evalLB(int nrestart)
void start()
Start time stepping.
void next()
Continue to next time step.
void comlim(int fromch, const std::vector<std::size_t>& tetid, const std::vector<std::vector<tk::real>>& u, const std::vector<std::vector<tk::real>>& prim, const std::vector<std::size_t>& ndof)
Receive chare-boundary limiter function data from neighboring chares.
void comreco(int fromch, const std::vector<std::size_t>& tetid, const std::vector<std::vector<tk::real>>& u, const std::vector<std::vector<tk::real>>& prim, const std::vector<std::size_t>& ndof)
Receive chare-boundary reconstructed data from neighboring chares.
void comsol(int fromch, std::size_t fromstage, const std::vector<std::size_t>& tetid, const std::vector<std::vector<tk::real>>& u, const std::vector<std::vector<tk::real>>& prim, const std::vector<std::size_t>& ndof)
Receive chare-boundary ghost data from neighboring chares.
void comnodalExtrema(const std::vector<std::size_t>& gid, const std::vector<std::vector<tk::real>>& G1, const std::vector<std::vector<tk::real>>& G2)
Receive contributions to nodal gradients on chare-boundaries.
void resizeNodalExtremac()
Initialize the vector of nodal extrema.
void evalNodalExtrm(const std::size_t ncomp, const std::size_t nprim, const std::size_t ndof_NodalExtrm, const std::vector<std::size_t>& bndel, const std::vector<std::size_t>& inpoel, const tk::UnsMesh::Coords& coord, const std::vector<std::size_t>& gid, const std::unordered_map<std::size_t, std::size_t>& bid, const tk::Fields& U, const tk::Fields& P, std::vector<std::vector<tk::real>>& uNodalExtrm, std::vector<std::vector<tk::real>>& pNodalExtrm)
Compute the nodal extrema for chare-boundary nodes.
void comnodeout(const std::vector<std::size_t>& gid, const std::vector<std::size_t>& nesup, const std::vector<std::vector<tk::real>>& L)
Receive nodal solution (ofor field output) contributions from neighboring chares.
void refine(const std::vector<tk::real>& l2res)
Optionally refine/derefine mesh.
void resizePostAMR(const std::vector<std::size_t>&, const tk::UnsMesh::Chunk& chunk, const tk::UnsMesh::Coords& coord, const std::unordered_map<std::size_t, tk::UnsMesh::Edge>&, const std::unordered_map<std::size_t, std::size_t>& addedTets, const std::set<std::size_t>&, const tk::NodeCommMap& nodeCommMap, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>&, const std::vector<std::size_t>& triinpoel)
Receive new mesh from Refiner.
void extractFieldOutput(const std::vector<std::size_t>&, const tk::UnsMesh::Chunk& chunk, const tk::UnsMesh::Coords& coord, const std::unordered_map<std::size_t, tk::UnsMesh::Edge>&, const std::unordered_map<std::size_t, std::size_t>& addedTets, const tk::NodeCommMap& nodeCommMap, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>&, const std::vector<std::size_t>& triinpoel, CkCallback c)
Extract field output going to file.
auto solution() const -> const tk::Fields&
void lhs()
Compute left hand side.
void resized()
Unused in DG.
void solve(tk::real newdt)
Compute right hand side and solve system.
void step()
Evaluate whether to continue with next time step.

Charm++ pack/unpack serializer member functions

using FaceMap = std::unordered_map<tk::UnsMesh::Face, std::array<std::size_t, 2>, tk::UnsMesh::Hash<3>, tk::UnsMesh::Eq<3>>
CProxy_Discretization m_disc
Discretization proxy.
std::size_t m_ndof_NodalExtrm
Degree of freedom for nodal extrema vector. When DGP1 is applied, there is one degree of freedom for cell average variable. When DGP2 is applied, the degree of freedom is 4 which refers to cell average and gradients in three directions.
std::size_t m_ncomfac
Counter for face adjacency communication map.
std::size_t m_nadj
Counter signaling that all ghost data have been received.
std::size_t m_ncomEsup
Counter for element-surr-node adjacency communication map.
std::size_t m_nsol
Counter signaling that we have received all our solution ghost data.
std::size_t m_ninitsol
Counter signaling that we have received all our solution ghost data during setup.
std::size_t m_nlim
Counter signaling that we have received all our limiter function ghost data.
std::size_t m_nnod
Counter signaling that we have received all our node solution contributions.
std::size_t m_nreco
Counter signaling that we have received all our reconstructed ghost data.
std::size_t m_nnodalExtrema
Counter signaling that we have received all our nodal extrema from ghost chare partitions.
std::vector<std::size_t> m_inpoel
Mesh connectivity extended.
tk::UnsMesh::Coords m_coord
Node coordinates extended.
FaceData m_fd
Face data.
tk::Fields m_u
Vector of unknown/solution average over each mesh element.
tk::Fields m_un
Vector of unknown at previous time-step.
tk::Fields m_p
Vector of primitive quantities over each mesh element.
tk::Fields m_geoFace
Face geometry.
tk::Fields m_geoElem
Element geometry.
tk::Fields m_lhs
Left-hand side mass-matrix which is a diagonal matrix.
tk::Fields m_rhs
Vector of right-hand side.
std::vector<std::vector<tk::real>> m_uNodalExtrm
Vector of nodal extrema for conservative variables.
std::vector<std::vector<tk::real>> m_pNodalExtrm
Vector of nodal extrema for primitive variables.
std::unordered_map<std::size_t, std::vector<tk::real>> m_uNodalExtrmc
Buffer for vector of nodal extrema for conservative variables.
std::unordered_map<std::size_t, std::vector<tk::real>> m_pNodalExtrmc
Buffer for vector of nodal extrema for primitive variables.
std::size_t m_nfac
Counter for number of faces on this chare (including chare boundaries)
std::size_t m_nunk
Counter for number of unknowns on this chare (including ghosts)
std::size_t m_npoin
Counter for number of nodes on this chare excluding ghosts.
tk::UnsMesh::FaceSet m_ipface
Internal + physical boundary faces (inverse of inpofa)
std::unordered_map<int, FaceMap> m_bndFace
std::unordered_map<int, GhostData> m_ghostData
Ghost data associated to chare IDs we communicate with.
std::unordered_map<int, std::unordered_set<std::size_t>> m_sendGhost
Elements which are ghosts for other chares associated to those chare IDs.
std::size_t m_ghostReq
Number of chares requesting ghost data.
std::unordered_map<int, std::unordered_map<std::size_t, std::size_t>> m_ghost
std::set<std::size_t> m_exptGhost
Expected ghost tet ids (used only in DEBUG)
std::set<std::size_t> m_recvGhost
Received ghost tet ids (used only in DEBUG)
ElemDiagnostics m_diag
Diagnostics object.
std::size_t m_stage
Runge-Kutta stage counter.
std::vector<std::size_t> m_ndof
Vector of local number of degrees of freedom for each element.
std::vector<std::size_t> m_numEqDof
Vector of number of degrees of freedom for each PDE equation/component.
std::unordered_map<std::size_t, std::size_t> m_bid
Map local ghost tet ids (value) and zero-based boundary ids (key)
std::array<std::vector<std::vector<tk::real>>, 3> m_uc
Solution receive buffers for ghosts only.
std::array<std::vector<std::vector<tk::real>>, 3> m_pc
Primitive-variable receive buffers for ghosts only.
std::array<std::vector<std::size_t>, 3> m_ndofc
Number of degrees of freedom (for p-adaptive) receive buffers for ghosts only.
std::size_t m_initial
1 if starting time stepping, 0 if during time stepping
tk::UnsMesh::FaceSet m_expChBndFace
Unique set of chare-boundary faces this chare is expected to receive.
std::unordered_map<int, tk::UnsMesh::FaceSet> m_infaces
Incoming communication buffer during chare-boundary face communication.
std::map<std::size_t, std::vector<std::size_t>> m_esup
Elements (value) surrounding point (key) data-structure.
std::map<std::size_t, std::vector<std::size_t>> m_esupc
Communication buffer for esup data-structure.
std::vector<std::vector<tk::real>> m_elemfields
Elem output fields.
std::vector<std::vector<tk::real>> m_nodefields
Node output fields.
std::unordered_map<std::size_t, std::pair<std::vector<tk::real>, std::size_t>> m_nodefieldsc
OutMesh m_outmesh
Storage for refined mesh used for field output.
std::vector<std::unordered_set<std::size_t>> m_boxelems
Element ids at which box ICs are defined by user (multiple boxes)
void pup(PUP::er& p) override
Pack/Unpack serialize member function.
void operator|(PUP::er& p, DG& i)
Pack/Unpack serialize operator|.

Typedef documentation

using inciter::DG::FaceMap = std::unordered_map<tk::UnsMesh::Face, std::array<std::size_t, 2>, tk::UnsMesh::Hash<3>, tk::UnsMesh::Eq<3>>

Local face & tet IDs associated to 3 global node IDs

This map stores tetrahedron cell faces (map key) and their associated local face ID and inner local tet id adjacent to the face (map value). A face is given by 3 global node IDs.

Function documentation

static void inciter::DG::registerReducers()

Configure Charm++ reduction types for concatenating BC nodelists.

Since this is a [initnode] routine, the runtime system executes the routine exactly once on every logical node early on in the Charm++ init sequence. Must be static as it is called without an object. See also: Section "Initializations at Program Startup" at in the Charm++ manual http://charm.cs.illinois.edu/manuals/html/charm++/manual.html.

DG_SDAG_CODE inciter::DG::DG(const CProxy_Discretization& disc, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>&, const std::vector<std::size_t>& triinpoel) explicit

Constructor.

Parameters
disc in Discretization proxy
bface in Boundary-faces mapped to side set ids
triinpoel in Boundary-face connectivity

void inciter::DG::ResumeFromSync() override

Return from migration.

This is called when load balancing (LB) completes. The presence of this function does not affect whether or not we block on LB.

void inciter::DG::comfac(int fromch, const tk::UnsMesh::FaceSet& infaces)

Receive unique set of faces we potentially share with/from another chare.

Parameters
fromch in Sender chare id
infaces in Unique set of faces we potentially share with fromch

void inciter::DG::comGhost(int fromch, const GhostData& ghost)

Receive ghost data on chare boundaries from fellow chare.

Parameters
fromch in Caller chare ID
ghost in Ghost data, see Inciter/FaceData.h for the type

void inciter::DG::nodeNeighSetup()

Setup node-neighborhood (esup)

At this point the face-ghost communication map has been established on this chare. This function begins generating the node-ghost comm map.

void inciter::DG::comEsup(int fromch, const std::unordered_map<std::size_t, std::vector<std::size_t>>& bndEsup, const std::unordered_map<std::size_t, std::vector<tk::real>>& nodeBndCells)

Receive element-surr-points data on chare boundaries from fellow chare.

Parameters
fromch in Sender chare id
bndEsup in Elements-surrounding-points data-structure from fromch
nodeBndCells in Map containing element geometries associated with remote element IDs in the esup

Receive elements-surrounding-points data-structure for points on.

void inciter::DG::box(tk::real v)

Receive total box IC volume and set conditions in box.

Parameters
in Total volume within user-specified box

void inciter::DG::evalLB(int nrestart)

Parameters
nrestart in Number of times restarted

void inciter::DG::comlim(int fromch, const std::vector<std::size_t>& tetid, const std::vector<std::vector<tk::real>>& u, const std::vector<std::vector<tk::real>>& prim, const std::vector<std::size_t>& ndof)

Receive chare-boundary limiter function data from neighboring chares.

Parameters
fromch in Sender chare id
tetid in Ghost tet ids we receive solution data for
in Limited high-order solution
prim in Limited high-order primitive quantities
ndof in Number of degrees of freedom for chare-boundary elements

This function receives contributions to the limited solution from fellow chares.

void inciter::DG::comreco(int fromch, const std::vector<std::size_t>& tetid, const std::vector<std::vector<tk::real>>& u, const std::vector<std::vector<tk::real>>& prim, const std::vector<std::size_t>& ndof)

Receive chare-boundary reconstructed data from neighboring chares.

Parameters
fromch in Sender chare id
tetid in Ghost tet ids we receive solution data for
in Reconstructed high-order solution
prim in Limited high-order primitive quantities
ndof in Number of degrees of freedom for chare-boundary elements

This function receives contributions to the reconstructed solution from fellow chares.

void inciter::DG::comsol(int fromch, std::size_t fromstage, const std::vector<std::size_t>& tetid, const std::vector<std::vector<tk::real>>& u, const std::vector<std::vector<tk::real>>& prim, const std::vector<std::size_t>& ndof)

Receive chare-boundary ghost data from neighboring chares.

Parameters
fromch in Sender chare id
fromstage in Sender chare time step stage
tetid in Ghost tet ids we receive solution data for
in Solution ghost data
prim in Primitive variables in ghost cells
ndof in Number of degrees of freedom for chare-boundary elements

This function receives contributions to the unlimited solution from fellow chares.

void inciter::DG::comnodalExtrema(const std::vector<std::size_t>& gid, const std::vector<std::vector<tk::real>>& G1, const std::vector<std::vector<tk::real>>& G2)

Receive contributions to nodal gradients on chare-boundaries.

Parameters
gid in Global mesh node IDs at which we receive grad contributions
G1 in Partial contributions of extrema for conservative variables to chare-boundary nodes
G2 in Partial contributions of extrema for primitive variables to chare-boundary nodes

This function receives contributions to m_uNodalExtrm/m_pNodalExtrm , which stores nodal extrems at mesh chare-boundary nodes. While m_uNodalExtrm/m_pNodalExtrm stores own contributions, m_uNodalExtrmc /m_pNodalExtrmc collects the neighbor chare contributions during communication.

void inciter::DG::evalNodalExtrm(const std::size_t ncomp, const std::size_t nprim, const std::size_t ndof_NodalExtrm, const std::vector<std::size_t>& bndel, const std::vector<std::size_t>& inpoel, const tk::UnsMesh::Coords& coord, const std::vector<std::size_t>& gid, const std::unordered_map<std::size_t, std::size_t>& bid, const tk::Fields& U, const tk::Fields& P, std::vector<std::vector<tk::real>>& uNodalExtrm, std::vector<std::vector<tk::real>>& pNodalExtrm)

Compute the nodal extrema for chare-boundary nodes.

Parameters
ncomp in Number of conservative variables
nprim in Number of primitive variables
ndof_NodalExtrm in Degree of freedom for nodal extrema
bndel in List of elements contributing to chare-boundary nodes
inpoel in Element-node connectivity for element e
coord in Array of nodal coordinates
gid in Local->global node id map
bid in Local chare-boundary node ids (value) associated to global node ids (key)
in Vector of conservative variables
in Vector of primitive variables
uNodalExtrm in/out Chare-boundary nodal extrema for conservative variables
pNodalExtrm in/out Chare-boundary nodal extrema for primitive variables

void inciter::DG::comnodeout(const std::vector<std::size_t>& gid, const std::vector<std::size_t>& nesup, const std::vector<std::vector<tk::real>>& L)

Receive nodal solution (ofor field output) contributions from neighboring chares.

Parameters
gid in Global mesh node IDs at which we receive contributions
nesup in Number of elements surrounding points
in Partial contributions of node fields to chare-boundary nodes

void inciter::DG::refine(const std::vector<tk::real>& l2res)

Optionally refine/derefine mesh.

Parameters
l2res in L2-norms of the residual for each scalar component computed across the whole problem

void inciter::DG::resizePostAMR(const std::vector<std::size_t>&, const tk::UnsMesh::Chunk& chunk, const tk::UnsMesh::Coords& coord, const std::unordered_map<std::size_t, tk::UnsMesh::Edge>&, const std::unordered_map<std::size_t, std::size_t>& addedTets, const std::set<std::size_t>&, const tk::NodeCommMap& nodeCommMap, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>&, const std::vector<std::size_t>& triinpoel)

Receive new mesh from Refiner.

Parameters
chunk in New mesh chunk (connectivity and global<->local id maps)
coord in New mesh node coordinates
addedTets in Newly added mesh cells and their parents (local ids)
nodeCommMap in New node communication map
bface in Boundary-faces mapped to side set ids
triinpoel in Boundary-face connectivity

void inciter::DG::extractFieldOutput(const std::vector<std::size_t>&, const tk::UnsMesh::Chunk& chunk, const tk::UnsMesh::Coords& coord, const std::unordered_map<std::size_t, tk::UnsMesh::Edge>&, const std::unordered_map<std::size_t, std::size_t>& addedTets, const tk::NodeCommMap& nodeCommMap, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>&, const std::vector<std::size_t>& triinpoel, CkCallback c)

Extract field output going to file.

Parameters
chunk in Field-output mesh chunk (connectivity and global<->local id maps)
coord in Field-output mesh node coordinates
addedTets in Field-output mesh cells and their parents (local ids)
nodeCommMap in Field-output mesh node communication map
bface in Field-output meshndary-faces mapped to side set ids
triinpoel in Field-output mesh boundary-face connectivity
in Function to continue with after the write

const tk::Fields& inciter::DG::solution() const

Returns Const-ref to current solution

Const-ref access to current solution

void inciter::DG::solve(tk::real newdt)

Compute right hand side and solve system.

Parameters
newdt in Size of this new time step

void inciter::DG::pup(PUP::er& p) override

Pack/Unpack serialize member function.

Parameters
in/out Charm++'s PUP::er serializer object reference

void inciter::DG::operator|(PUP::er& p, DG& i)

Pack/Unpack serialize operator|.

Parameters
in/out Charm++'s PUP::er serializer object reference
in/out DG object reference

Variable documentation

std::unordered_map<int, FaceMap> inciter::DG::m_bndFace

Face & tet IDs associated to global node IDs of the face for each chare

This map stores not only the unique faces associated to fellow chares, but also a newly assigned local face ID and adjacent local tet ID.

std::unordered_map<int, std::unordered_map<std::size_t, std::size_t>> inciter::DG::m_ghost

Local element id associated to ghost remote id charewise

This map associates the local element id (inner map value) to the (remote) element id of the ghost (inner map key) based on the chare id (outer map key) this remote element lies in.

std::unordered_map<std::size_t, std::pair<std::vector<tk::real>, std::size_t>> inciter::DG::m_nodefieldsc

Receive buffer for communication of node output fields

Key: global node id, value: output fields and number of elements surrounding the node