inciter::DG class

DG Charm++ chare array used to advance PDEs in time with DG+RK.

Reference

Public static functions

static void registerReducers(): Configure Charm++ reduction types for concatenating BC nodelists.

Constructors, destructors, conversion operators

DG(CkMigrateMessage* msg) explicit: Migrate constructor.

Public functions

auto DG(const CProxy_Discretization& disc, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>&, const std::vector<std::size_t>& triinpoel) -> DG_SDAG_CODE explicit: Constructor.
void ResumeFromSync() override: Return from migration.
void resizeComm(): Start sizing communication buffers and setting up ghost data.
void comfac(int fromch, const tk::UnsMesh::FaceSet& infaces): Receive unique set of faces we potentially share with/from another chare.
void comGhost(int fromch, const GhostData& ghost): Receive ghost data on chare boundaries from fellow chare.
void reqGhost(): Receive requests for ghost data.
void sendGhost(): Send all of our ghost data to fellow chares.
void nodeNeighSetup(): Setup node-neighborhood (esup)
void comEsup(int fromch, const std::unordered_map<std::size_t, std::vector<std::size_t>>& bndEsup, const std::unordered_map<std::size_t, std::vector<tk::real>>& nodeBndCells): Receive element-surr-points data on chare boundaries from fellow chare.
void setup(): Setup: query boundary conditions, output mesh, etc.
void box(tk::real v): Receive total box IC volume and set conditions in box.
void evalLB(int nrestart)
void start(): Start time stepping.
void next(): Continue to next time step.
void comlim(int fromch, const std::vector<std::size_t>& tetid, const std::vector<std::vector<tk::real>>& u, const std::vector<std::vector<tk::real>>& prim, const std::vector<std::size_t>& ndof): Receive chare-boundary limiter function data from neighboring chares.
void comreco(int fromch, const std::vector<std::size_t>& tetid, const std::vector<std::vector<tk::real>>& u, const std::vector<std::vector<tk::real>>& prim, const std::vector<std::size_t>& ndof): Receive chare-boundary reconstructed data from neighboring chares.
void comsol(int fromch, std::size_t fromstage, const std::vector<std::size_t>& tetid, const std::vector<std::vector<tk::real>>& u, const std::vector<std::vector<tk::real>>& prim, const std::vector<std::size_t>& ndof): Receive chare-boundary ghost data from neighboring chares.
void comnodalExtrema(const std::vector<std::size_t>& gid, const std::vector<std::vector<tk::real>>& G1, const std::vector<std::vector<tk::real>>& G2): Receive contributions to nodal gradients on chare-boundaries.
void resizeNodalExtremac(): Initialize the vector of nodal extrema.
void evalNodalExtrm(const std::size_t ncomp, const std::size_t nprim, const std::size_t ndof_NodalExtrm, const std::vector<std::size_t>& bndel, const std::vector<std::size_t>& inpoel, const tk::UnsMesh::Coords& coord, const std::vector<std::size_t>& gid, const std::unordered_map<std::size_t, std::size_t>& bid, const tk::Fields& U, const tk::Fields& P, std::vector<std::vector<tk::real>>& uNodalExtrm, std::vector<std::vector<tk::real>>& pNodalExtrm): Compute the nodal extrema for chare-boundary nodes.
void comnodeout(const std::vector<std::size_t>& gid, const std::vector<std::size_t>& nesup, const std::vector<std::vector<tk::real>>& L): Receive nodal solution (ofor field output) contributions from neighboring chares.
void refine(const std::vector<tk::real>& l2res): Optionally refine/derefine mesh.
void resizePostAMR(const std::vector<std::size_t>&, const tk::UnsMesh::Chunk& chunk, const tk::UnsMesh::Coords& coord, const std::unordered_map<std::size_t, tk::UnsMesh::Edge>&, const std::unordered_map<std::size_t, std::size_t>& addedTets, const std::set<std::size_t>&, const tk::NodeCommMap& nodeCommMap, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>&, const std::vector<std::size_t>& triinpoel): Receive new mesh from Refiner.
void extractFieldOutput(const std::vector<std::size_t>&, const tk::UnsMesh::Chunk& chunk, const tk::UnsMesh::Coords& coord, const std::unordered_map<std::size_t, tk::UnsMesh::Edge>&, const std::unordered_map<std::size_t, std::size_t>& addedTets, const tk::NodeCommMap& nodeCommMap, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>&, const std::vector<std::size_t>& triinpoel, CkCallback c): Extract field output going to file.
auto solution() const -> const tk::Fields&
void lhs(): Compute left hand side.
void resized(): Unused in DG.
void solve(tk::real newdt): Compute right hand side and solve system.
void step(): Evaluate whether to continue with next time step.

Charm++ pack/unpack serializer member functions

using FaceMap = std::unordered_map<tk::UnsMesh::Face, std::array<std::size_t, 2>, tk::UnsMesh::Hash<3>, tk::UnsMesh::Eq<3>>
CProxy_Discretization m_disc: Discretization proxy.
std::size_t m_ndof_NodalExtrm: Degree of freedom for nodal extrema vector. When DGP1 is applied, there is one degree of freedom for cell average variable. When DGP2 is applied, the degree of freedom is 4 which refers to cell average and gradients in three directions.
std::size_t m_ncomfac: Counter for face adjacency communication map.
std::size_t m_nadj: Counter signaling that all ghost data have been received.
std::size_t m_ncomEsup: Counter for element-surr-node adjacency communication map.
std::size_t m_nsol: Counter signaling that we have received all our solution ghost data.
std::size_t m_ninitsol: Counter signaling that we have received all our solution ghost data during setup.
std::size_t m_nlim: Counter signaling that we have received all our limiter function ghost data.
std::size_t m_nnod: Counter signaling that we have received all our node solution contributions.
std::size_t m_nreco: Counter signaling that we have received all our reconstructed ghost data.
std::size_t m_nnodalExtrema: Counter signaling that we have received all our nodal extrema from ghost chare partitions.
std::vector<std::size_t> m_inpoel: Mesh connectivity extended.
tk::UnsMesh::Coords m_coord: Node coordinates extended.
FaceData m_fd: Face data.
tk::Fields m_u: Vector of unknown/solution average over each mesh element.
tk::Fields m_un: Vector of unknown at previous time-step.
tk::Fields m_p: Vector of primitive quantities over each mesh element.
tk::Fields m_geoFace: Face geometry.
tk::Fields m_geoElem: Element geometry.
tk::Fields m_lhs: Left-hand side mass-matrix which is a diagonal matrix.
tk::Fields m_rhs: Vector of right-hand side.
std::vector<std::vector<tk::real>> m_uNodalExtrm: Vector of nodal extrema for conservative variables.
std::vector<std::vector<tk::real>> m_pNodalExtrm: Vector of nodal extrema for primitive variables.
std::unordered_map<std::size_t, std::vector<tk::real>> m_uNodalExtrmc: Buffer for vector of nodal extrema for conservative variables.
std::unordered_map<std::size_t, std::vector<tk::real>> m_pNodalExtrmc: Buffer for vector of nodal extrema for primitive variables.
std::size_t m_nfac: Counter for number of faces on this chare (including chare boundaries)
std::size_t m_nunk: Counter for number of unknowns on this chare (including ghosts)
std::size_t m_npoin: Counter for number of nodes on this chare excluding ghosts.
tk::UnsMesh::FaceSet m_ipface: Internal + physical boundary faces (inverse of inpofa)
std::unordered_map<int, FaceMap> m_bndFace
std::unordered_map<int, GhostData> m_ghostData: Ghost data associated to chare IDs we communicate with.
std::unordered_map<int, std::unordered_set<std::size_t>> m_sendGhost: Elements which are ghosts for other chares associated to those chare IDs.
std::size_t m_ghostReq: Number of chares requesting ghost data.
std::unordered_map<int, std::unordered_map<std::size_t, std::size_t>> m_ghost
std::set<std::size_t> m_exptGhost: Expected ghost tet ids (used only in DEBUG)
std::set<std::size_t> m_recvGhost: Received ghost tet ids (used only in DEBUG)
ElemDiagnostics m_diag: Diagnostics object.
std::size_t m_stage: Runge-Kutta stage counter.
std::vector<std::size_t> m_ndof: Vector of local number of degrees of freedom for each element.
std::vector<std::size_t> m_numEqDof: Vector of number of degrees of freedom for each PDE equation/component.
std::unordered_map<std::size_t, std::size_t> m_bid: Map local ghost tet ids (value) and zero-based boundary ids (key)
std::array<std::vector<std::vector<tk::real>>, 3> m_uc: Solution receive buffers for ghosts only.
std::array<std::vector<std::vector<tk::real>>, 3> m_pc: Primitive-variable receive buffers for ghosts only.
std::array<std::vector<std::size_t>, 3> m_ndofc: Number of degrees of freedom (for p-adaptive) receive buffers for ghosts only.
std::size_t m_initial: 1 if starting time stepping, 0 if during time stepping
tk::UnsMesh::FaceSet m_expChBndFace: Unique set of chare-boundary faces this chare is expected to receive.
std::unordered_map<int, tk::UnsMesh::FaceSet> m_infaces: Incoming communication buffer during chare-boundary face communication.
std::map<std::size_t, std::vector<std::size_t>> m_esup: Elements (value) surrounding point (key) data-structure.
std::map<std::size_t, std::vector<std::size_t>> m_esupc: Communication buffer for esup data-structure.
std::vector<std::vector<tk::real>> m_elemfields: Elem output fields.
std::vector<std::vector<tk::real>> m_nodefields: Node output fields.
std::unordered_map<std::size_t, std::pair<std::vector<tk::real>, std::size_t>> m_nodefieldsc
OutMesh m_outmesh: Storage for refined mesh used for field output.
std::vector<std::unordered_set<std::size_t>> m_boxelems: Element ids at which box ICs are defined by user (multiple boxes)
void pup(PUP::er& p) override: Pack/Unpack serialize member function.
void operator|(PUP::er& p, DG& i): Pack/Unpack serialize operator|.

Typedef documentation

using inciter::DG::FaceMap = std::unordered_map<tk::UnsMesh::Face, std::array<std::size_t, 2>, tk::UnsMesh::Hash<3>, tk::UnsMesh::Eq<3>>

Local face & tet IDs associated to 3 global node IDs

This map stores tetrahedron cell faces (map key) and their associated local face ID and inner local tet id adjacent to the face (map value). A face is given by 3 global node IDs.

Function documentation

static void inciter::DG::registerReducers()

Configure Charm++ reduction types for concatenating BC nodelists.

Since this is a [initnode] routine, the runtime system executes the routine exactly once on every logical node early on in the Charm++ init sequence. Must be static as it is called without an object. See also: Section "Initializations at Program Startup" at in the Charm++ manual http://charm.cs.illinois.edu/manuals/html/charm++/manual.html.

DG_SDAG_CODE inciter::DG::DG(const CProxy_Discretization& disc, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>&, const std::vector<std::size_t>& triinpoel) explicit

Constructor.

Parameters
disc in	Discretization proxy
bface in	Boundary-faces mapped to side set ids
triinpoel in	Boundary-face connectivity

void inciter::DG::ResumeFromSync() override

Return from migration.

This is called when load balancing (LB) completes. The presence of this function does not affect whether or not we block on LB.

void inciter::DG::comfac(int fromch, const tk::UnsMesh::FaceSet& infaces)

Receive unique set of faces we potentially share with/from another chare.

Parameters
fromch in	Sender chare id
infaces in	Unique set of faces we potentially share with fromch

void inciter::DG::comGhost(int fromch, const GhostData& ghost)

Receive ghost data on chare boundaries from fellow chare.

Parameters
fromch in	Caller chare ID
ghost in	Ghost data, see Inciter/FaceData.h for the type

void inciter::DG::nodeNeighSetup()

Setup node-neighborhood (esup)

At this point the face-ghost communication map has been established on this chare. This function begins generating the node-ghost comm map.

void inciter::DG::comEsup(int fromch, const std::unordered_map<std::size_t, std::vector<std::size_t>>& bndEsup, const std::unordered_map<std::size_t, std::vector<tk::real>>& nodeBndCells)

Receive element-surr-points data on chare boundaries from fellow chare.

Parameters
fromch in	Sender chare id
bndEsup in	Elements-surrounding-points data-structure from fromch
nodeBndCells in	Map containing element geometries associated with remote element IDs in the esup

Receive elements-surrounding-points data-structure for points on.

void inciter::DG::box(tk::real v)

Receive total box IC volume and set conditions in box.

Parameters
v in	Total volume within user-specified box

void inciter::DG::evalLB(int nrestart)

Parameters
nrestart in	Number of times restarted

void inciter::DG::comlim(int fromch, const std::vector<std::size_t>& tetid, const std::vector<std::vector<tk::real>>& u, const std::vector<std::vector<tk::real>>& prim, const std::vector<std::size_t>& ndof)

Receive chare-boundary limiter function data from neighboring chares.

Parameters
fromch in	Sender chare id
tetid in	Ghost tet ids we receive solution data for
u in	Limited high-order solution
prim in	Limited high-order primitive quantities
ndof in	Number of degrees of freedom for chare-boundary elements

This function receives contributions to the limited solution from fellow chares.

void inciter::DG::comreco(int fromch, const std::vector<std::size_t>& tetid, const std::vector<std::vector<tk::real>>& u, const std::vector<std::vector<tk::real>>& prim, const std::vector<std::size_t>& ndof)

Receive chare-boundary reconstructed data from neighboring chares.

Parameters
fromch in	Sender chare id
tetid in	Ghost tet ids we receive solution data for
u in	Reconstructed high-order solution
prim in	Limited high-order primitive quantities
ndof in	Number of degrees of freedom for chare-boundary elements

This function receives contributions to the reconstructed solution from fellow chares.

void inciter::DG::comsol(int fromch, std::size_t fromstage, const std::vector<std::size_t>& tetid, const std::vector<std::vector<tk::real>>& u, const std::vector<std::vector<tk::real>>& prim, const std::vector<std::size_t>& ndof)

Receive chare-boundary ghost data from neighboring chares.

Parameters
fromch in	Sender chare id
fromstage in	Sender chare time step stage
tetid in	Ghost tet ids we receive solution data for
u in	Solution ghost data
prim in	Primitive variables in ghost cells
ndof in	Number of degrees of freedom for chare-boundary elements

This function receives contributions to the unlimited solution from fellow chares.

void inciter::DG::comnodalExtrema(const std::vector<std::size_t>& gid, const std::vector<std::vector<tk::real>>& G1, const std::vector<std::vector<tk::real>>& G2)

Receive contributions to nodal gradients on chare-boundaries.

Parameters
gid in	Global mesh node IDs at which we receive grad contributions
G1 in	Partial contributions of extrema for conservative variables to chare-boundary nodes
G2 in	Partial contributions of extrema for primitive variables to chare-boundary nodes

This function receives contributions to m_uNodalExtrm/m_pNodalExtrm , which stores nodal extrems at mesh chare-boundary nodes. While m_uNodalExtrm/m_pNodalExtrm stores own contributions, m_uNodalExtrmc /m_pNodalExtrmc collects the neighbor chare contributions during communication.

void inciter::DG::evalNodalExtrm(const std::size_t ncomp, const std::size_t nprim, const std::size_t ndof_NodalExtrm, const std::vector<std::size_t>& bndel, const std::vector<std::size_t>& inpoel, const tk::UnsMesh::Coords& coord, const std::vector<std::size_t>& gid, const std::unordered_map<std::size_t, std::size_t>& bid, const tk::Fields& U, const tk::Fields& P, std::vector<std::vector<tk::real>>& uNodalExtrm, std::vector<std::vector<tk::real>>& pNodalExtrm)

Compute the nodal extrema for chare-boundary nodes.

Parameters
ncomp in	Number of conservative variables
nprim in	Number of primitive variables
ndof_NodalExtrm in	Degree of freedom for nodal extrema
bndel in	List of elements contributing to chare-boundary nodes
inpoel in	Element-node connectivity for element e
coord in	Array of nodal coordinates
gid in	Local->global node id map
bid in	Local chare-boundary node ids (value) associated to global node ids (key)
U in	Vector of conservative variables
P in	Vector of primitive variables
uNodalExtrm in/out	Chare-boundary nodal extrema for conservative variables
pNodalExtrm in/out	Chare-boundary nodal extrema for primitive variables

void inciter::DG::comnodeout(const std::vector<std::size_t>& gid, const std::vector<std::size_t>& nesup, const std::vector<std::vector<tk::real>>& L)

Receive nodal solution (ofor field output) contributions from neighboring chares.

Parameters
gid in	Global mesh node IDs at which we receive contributions
nesup in	Number of elements surrounding points
L in	Partial contributions of node fields to chare-boundary nodes

void inciter::DG::refine(const std::vector<tk::real>& l2res)

Optionally refine/derefine mesh.

Parameters
l2res in	L2-norms of the residual for each scalar component computed across the whole problem

void inciter::DG::resizePostAMR(const std::vector<std::size_t>&, const tk::UnsMesh::Chunk& chunk, const tk::UnsMesh::Coords& coord, const std::unordered_map<std::size_t, tk::UnsMesh::Edge>&, const std::unordered_map<std::size_t, std::size_t>& addedTets, const std::set<std::size_t>&, const tk::NodeCommMap& nodeCommMap, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>&, const std::vector<std::size_t>& triinpoel)

Receive new mesh from Refiner.

Parameters
chunk in	New mesh chunk (connectivity and global<->local id maps)
coord in	New mesh node coordinates
addedTets in	Newly added mesh cells and their parents (local ids)
nodeCommMap in	New node communication map
bface in	Boundary-faces mapped to side set ids
triinpoel in	Boundary-face connectivity

void inciter::DG::extractFieldOutput(const std::vector<std::size_t>&, const tk::UnsMesh::Chunk& chunk, const tk::UnsMesh::Coords& coord, const std::unordered_map<std::size_t, tk::UnsMesh::Edge>&, const std::unordered_map<std::size_t, std::size_t>& addedTets, const tk::NodeCommMap& nodeCommMap, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>&, const std::vector<std::size_t>& triinpoel, CkCallback c)

Extract field output going to file.

Parameters
chunk in	Field-output mesh chunk (connectivity and global<->local id maps)
coord in	Field-output mesh node coordinates
addedTets in	Field-output mesh cells and their parents (local ids)
nodeCommMap in	Field-output mesh node communication map
bface in	Field-output meshndary-faces mapped to side set ids
triinpoel in	Field-output mesh boundary-face connectivity
c in	Function to continue with after the write

const tk::Fields& inciter::DG::solution() const

Returns	Const-ref to current solution

Const-ref access to current solution

void inciter::DG::solve(tk::real newdt)

Compute right hand side and solve system.

Parameters
newdt in	Size of this new time step

void inciter::DG::pup(PUP::er& p) override

Pack/Unpack serialize member function.

Parameters
p in/out	Charm++'s PUP::er serializer object reference

void inciter::DG::operator|(PUP::er& p, DG& i)

Pack/Unpack serialize operator|.

Parameters
p in/out	Charm++'s PUP::er serializer object reference
i in/out	DG object reference

Variable documentation

std::unordered_map<int, FaceMap> inciter::DG::m_bndFace

Face & tet IDs associated to global node IDs of the face for each chare

This map stores not only the unique faces associated to fellow chares, but also a newly assigned local face ID and adjacent local tet ID.

std::unordered_map<int, std::unordered_map<std::size_t, std::size_t>> inciter::DG::m_ghost

Local element id associated to ghost remote id charewise

This map associates the local element id (inner map value) to the (remote) element id of the ghost (inner map key) based on the chare id (outer map key) this remote element lies in.

std::unordered_map<std::size_t, std::pair<std::vector<tk::real>, std::size_t>> inciter::DG::m_nodefieldsc

Receive buffer for communication of node output fields

Key: global node id, value: output fields and number of elements surrounding the node