class
TransporterTransporter drives the time integration of transport equations.
Contents
Constructors, destructors, conversion operators
 Transporter(CkMigrateMessage* m) explicit
 Migrate constructor: returning from a checkpoint.
Public functions
 auto Transporter() > Transporter_SDAG_CODE explicit
 Constructor.
 void load(std::size_t meshid, std::size_t nelem)
 Reduction target: the mesh has been read from file on all PEs.
 void partitioned(std::size_t meshid)
 Reduction target: a mesh has been partitioned.
 void partition()
 Reduction target: all Solver (PEs) have computed the number of chares they will recieve contributions from during linear solution.
 void distributed(std::size_t meshid)
 Reduction target: all compute nodes have distrbuted their mesh after partitioning.
 void queriedRef(std::size_t meshid)
 Reduction target: all Refiner chares have queried their boundary edges.
 void respondedRef(std::size_t meshid)
 Reduction target: all Refiner chares have setup their boundary edges.
 void refinserted(std::size_t meshid, std::size_t error)
 Reduction target: all compute nodes have created the mesh refiners.
 void discinserted(std::size_t meshid)
 Reduction target: all Discretization chares have been inserted.
 void disccreated(std::size_t summeshid, std::size_t npoin)
 Reduction target: all Discretization constructors have been called.
 void workinserted(std::size_t meshid)
 Reduction target: all worker (derived discretization) chares have been inserted.
 void compatibility(std::size_t meshid, std::size_t modified)
 Reduction target: all Refiner chares have received a round of edges, and have run their compatibility algorithm.
 void matched(std::size_t summeshid, std::size_t nextra, std::size_t nref, std::size_t nderef, std::size_t initial)
 Reduction target: all Refiner chares have matched/corrected the tagging of chareboundary edges, all chares are ready to perform refinement.

void bndint(tk::
real sx, tk::real sy, tk::real sz, tk::real cb, tk::real summeshid)  Compute surface integral across the whole problem and perform leaktest.
 void refined(std::size_t meshid, std::size_t nelem, std::size_t npoin)
 Reduction target: all chares have refined their mesh.
 void resized(std::size_t meshid)
 Reduction target: all worker chares have resized their own data after mesh refinement.
 void startEsup(std::size_t meshid)
 Reduction target: all worker chares have generated their own esup.
 void queried(std::size_t meshid)
 Reduction target: all Sorter chares have queried their boundary edges.
 void responded(std::size_t meshid)
 Reduction target: all Sorter chares have setup their boundary edges.
 void pepartitioned()
 Nonreduction target for receiving progress report on partitioning mesh.
 void pedistributed()
 Nonreduction target for receiving progress report on distributing mesh.
 void chbnd()
 Nonreduction target for receiving progress report on finding bnd nodes.
 void chcomm()
 Nonreduction target for receiving progress report on node ID comm map.
 void chmask()
 Nonreduction target for receiving progress report on node ID mask.
 void chreordered()
 Nonreduction target for receiving progress report on reordering mesh.
 void chcreated()
 Nonreduction target for receiving progress report on creating workers.
 void chbndface()
 Nonreduction target for receiving progress report on finding bnd faces.
 void chcomfac()
 Nonreduction target for receiving progress report on face communication.
 void chghost()
 Nonreduction target for receiving progress report on sending ghost data.
 void chadj()
 Nonreduction target for receiving progress report on face adjacency.
 void comfinal(std::size_t initial, std::size_t summeshid)
 Reduction target indicating that the communication maps have been setup.

void totalvol(tk::
real v, tk::real initial, tk::real summeshid)  Reduction target summing total mesh volume.

void minstat(tk::
real d0, tk::real d1, tk::real d2, tk::real rmeshid)  Reduction target yielding the minimum mesh statistics across all workers.

void maxstat(tk::
real d0, tk::real d1, tk::real d2, tk::real rmeshid)  Reduction target yielding the maximum mesh statistics across all workers.

void sumstat(tk::
real d0, tk::real d1, tk::real d2, tk::real d3, tk::real d4, tk::real d5, tk::real summeshid)  Reduction target yielding the sum of mesh statistics across all workers.
 void pdfstat(CkReductionMsg* msg)
 Reduction target yielding PDF of mesh statistics across all workers.

void boxvol(tk::
real v, tk::real rmeshid)  Reduction target computing total volume of IC box.
 void diagnostics(CkReductionMsg* msg)
 Reduction target optionally collecting diagnostics, e.g., residuals, from all worker chares.
 void resume()
 Resume execution from checkpoint/restart files.
 void checkpoint(std::size_t finished, std::size_t meshid)
 Save checkpoint/restart files.
 void finish(std::size_t meshid = 0)
 Normal finish of time stepping.
Charm++ pack/unpack serializer member functions
 enum class TimerTag { MESH_READ =0 }
 Timer tags.
 std::vector<std::string> m_input
 List of mesh files to be used for potentially multiple solvers.
 std::vector<int> m_nchare
 Number of worker chares (one per mesh)
 std::unordered_map<std::size_t, std::size_t> m_meshid
 Sum of mesh ids (across all chares, key) for each meshid (value)
 std::vector<std::size_t> m_ncit
 Number of mesh ref corr iter (one per mesh)
 std::size_t m_nload
 Number of meshes loaded.
 std::size_t m_npart
 Number of meshes partitioned.
 std::size_t m_nstat
 Number of mesh statistics computed.
 std::size_t m_ndisc
 Number of Discretization arrays created.
 std::size_t m_nchk
 Number of worker arrays checkpointed.
 std::size_t m_ncom
 Number of worker arrays have finished setting up their comm maps.
 std::vector<std::size_t> m_nt0refit
 Number of t0ref mesh ref iters (one per mesh)
 std::vector<std::size_t> m_ndtrefit
 Number of dtref mesh ref iters (one per mesh)
 std::vector<std::size_t> m_noutrefit
 Number of outref mesh ref iters (one per mesh)
 std::vector<std::size_t> m_noutderefit
 Number of outderef mesh ref iters (one per mesh)
 std::vector<Scheme> m_scheme
 Discretization scheme (one per mesh)
 std::vector<CProxy_Partitioner> m_partitioner
 Partitioner nodegroup proxies (one per mesh)
 std::vector<CProxy_Refiner> m_refiner
 Mesh refiner array proxies (one per mesh)
 std::vector<tk::CProxy_MeshWriter> m_meshwriter
 Mesh writer nodegroup proxies (one per mesh)
 std::vector<CProxy_Sorter> m_sorter
 Mesh sorter array proxy (one per mesh)
 std::vector<std::size_t> m_nelem
 Number of mesh points (per mesh)
 std::vector<std::size_t> m_npoin
 Nonzero if finished with timestepping (one per mesh)
 std::vector<tk::
real > m_meshvol  Total mesh volume (one per mesh)
 std::vector<std::array<tk::
real , 3>> m_minstat  Minimum mesh statistics (one per mesh)
 std::vector<std::array<tk::
real , 3>> m_maxstat  Maximum mesh statistics (one per mesh)
 std::vector<std::array<tk::
real , 3>> m_avgstat  Average mesh statistics (one per mesh)
 std::map<TimerTag, tk::
Timer > m_timer  Timers.
 tk::
Progress <7> m_progMesh  Progress object for preparing mesh.
 tk::
Progress <5> m_progWork  Progress object for preparing workers.
 void pup(PUP::er& p) override
 Pack/Unpack serialize member function.
 void operator(PUP::er& p, Transporter& t)
 Pack/Unpack serialize operator.
Function documentation
void inciter::Transporter:: load(std::size_t meshid,
std::size_t nelem)
Reduction target: the mesh has been read from file on all PEs.
Parameters  

meshid in  Mesh id (summed accross all compute nodes) 
nelem in  Number of mesh elements per mesh (summed across all compute nodes) 
void inciter::Transporter:: partitioned(std::size_t meshid)
Reduction target: a mesh has been partitioned.
Parameters  

meshid in  Mesh id 
void inciter::Transporter:: distributed(std::size_t meshid)
Reduction target: all compute nodes have distrbuted their mesh after partitioning.
Parameters  

meshid in  Mesh id 
void inciter::Transporter:: queriedRef(std::size_t meshid)
Reduction target: all Refiner chares have queried their boundary edges.
Parameters  

meshid in  Mesh id 
void inciter::Transporter:: respondedRef(std::size_t meshid)
Reduction target: all Refiner chares have setup their boundary edges.
Parameters  

meshid in  Mesh id 
void inciter::Transporter:: refinserted(std::size_t meshid,
std::size_t error)
Reduction target: all compute nodes have created the mesh refiners.
Parameters  

meshid in  Mesh id (aggregated across all compute nodes with operator max) 
error in  Error code (aggregated across all compute nodes with operator max) 
void inciter::Transporter:: discinserted(std::size_t meshid)
Reduction target: all Discretization chares have been inserted.
Parameters  

meshid in  Mesh id 
void inciter::Transporter:: disccreated(std::size_t summeshid,
std::size_t npoin)
Reduction target: all Discretization constructors have been called.
Parameters  

summeshid in  Mesh id (summed accross all chares) 
npoin in  Total number of mesh points (summed across all chares) Note that as opposed to npoin in refined(), this npoin is not multicounted, and thus should be correct in parallel. 
void inciter::Transporter:: workinserted(std::size_t meshid)
Reduction target: all worker (derived discretization) chares have been inserted.
Parameters  

meshid in  Mesh id 
void inciter::Transporter:: compatibility(std::size_t meshid,
std::size_t modified)
Reduction target: all Refiner chares have received a round of edges, and have run their compatibility algorithm.
Parameters  

meshid in  Mesh id (aggregated across all chares using operator max) 
modified in  Modified flag, aggregated across all chares using operator max), if nonzero, mesh is modified This is called iteratively, until convergence by Refiner. At this point all Refiner chares have received a round of edge data (tags whether an edge needs to be refined, etc.), and applied the compatibility algorithm independent of other Refiner chares. We keep going until the mesh is no longer modified by the compatibility algorithm, based on a new round of edge data communication started in Refiner:: 
void inciter::Transporter:: matched(std::size_t summeshid,
std::size_t nextra,
std::size_t nref,
std::size_t nderef,
std::size_t initial)
Reduction target: all Refiner chares have matched/corrected the tagging of chareboundary edges, all chares are ready to perform refinement.
Parameters  

summeshid in  Mesh id (summed across all chares) 
nextra in  Sum (across all chares) of the number of edges on each chare that need correction along chare boundaries 
nref in  Sum of number of refined tetrahedra across all chares. 
nderef in  Sum of number of derefined tetrahedra across all chares. 
initial 
void inciter::Transporter:: bndint(tk::real sx,
tk::real sy,
tk::real sz,
tk::real cb,
tk::real summeshid)
Compute surface integral across the whole problem and perform leaktest.
Parameters  

sx in  X component of vector summed 
sy in  Y component of vector summed 
sz in  Z component of vector summed 
cb in  Invoke callback if positive 
summeshid in  Mesh id (summed accross all chares) This function aggregates partial surface integrals across the boundary faces of the whole problem. After this global sum a nonzero vector result indicates a leak, e.g., a hole in the boundary, which indicates an error in the boundary face data structures used to compute the partial surface integrals. 
void inciter::Transporter:: refined(std::size_t meshid,
std::size_t nelem,
std::size_t npoin)
Reduction target: all chares have refined their mesh.
Parameters  

meshid  
nelem in  Total number of elements in mesh summed across the distributed mesh 
npoin in  Total number of mesh points summed across the distributed mesh. Note that in parallel this is larger than the number of points in the mesh, because the boundary nodes are multicounted. But we only need an equal or larger than npoin for Sorter:: 
void inciter::Transporter:: resized(std::size_t meshid)
Reduction target: all worker chares have resized their own data after mesh refinement.
Parameters  

meshid in  Mesh id 
void inciter::Transporter:: startEsup(std::size_t meshid)
Reduction target: all worker chares have generated their own esup.
Parameters  

meshid in  Mesh id 
void inciter::Transporter:: comfinal(std::size_t initial,
std::size_t summeshid)
Reduction target indicating that the communication maps have been setup.
Parameters  

initial in  Sum of contributions from all chares. If larger than zero, we are during time stepping and if zero we are during setup. 
summeshid in  Mesh id (summed accross the distributed mesh) 
void inciter::Transporter:: totalvol(tk::real v,
tk::real initial,
tk::real summeshid)
Reduction target summing total mesh volume.
Parameters  

v in  Mesh volume summed across the distributed mesh 
initial in  Sum of contributions from all chares. If larger than zero, we are during setup, if zero, during time stepping. 
summeshid in  Mesh id (summed accross the distributed mesh) 
void inciter::Transporter:: minstat(tk::real d0,
tk::real d1,
tk::real d2,
tk::real rmeshid)
Reduction target yielding the minimum mesh statistics across all workers.
Parameters  

d0 in  Minimum mesh statistics collected over all chares 
d1 in  Minimum mesh statistics collected over all chares 
d2 in  Minimum mesh statistics collected over all chares 
rmeshid in  Mesh id as a real 
void inciter::Transporter:: maxstat(tk::real d0,
tk::real d1,
tk::real d2,
tk::real rmeshid)
Reduction target yielding the maximum mesh statistics across all workers.
Parameters  

d0 in  Maximum mesh statistics collected over all chares 
d1 in  Maximum mesh statistics collected over all chares 
d2 in  Maximum mesh statistics collected over all chares 
rmeshid in  Mesh id as a real 
void inciter::Transporter:: sumstat(tk::real d0,
tk::real d1,
tk::real d2,
tk::real d3,
tk::real d4,
tk::real d5,
tk::real summeshid)
Reduction target yielding the sum of mesh statistics across all workers.
Parameters  

d0 in  Sum mesh statistics collected over all chares 
d1 in  Sum mesh statistics collected over all chares 
d2 in  Sum mesh statistics collected over all chares 
d3 in  Sum mesh statistics collected over all chares 
d4 in  Sum mesh statistics collected over all chares 
d5 in  Sum mesh statistics collected over all chares 
summeshid in  Mesh id (summed accross the distributed mesh) 
void inciter::Transporter:: pdfstat(CkReductionMsg* msg)
Reduction target yielding PDF of mesh statistics across all workers.
Parameters  

msg in  Serialized PDF 
void inciter::Transporter:: diagnostics(CkReductionMsg* msg)
Reduction target optionally collecting diagnostics, e.g., residuals, from all worker chares.
Parameters  

msg in  Serialized diagnostics vector aggregated across all PEs 
void inciter::Transporter:: resume()
Resume execution from checkpoint/restart files.
This is invoked by Charm++ after the checkpoint is done, as well as when the restart (returning from a checkpoint) is complete
void inciter::Transporter:: checkpoint(std::size_t finished,
std::size_t meshid)
Save checkpoint/restart files.
Parameters  

finished in  Nonzero if finished with time stepping 
meshid in  Mesh id 
void inciter::Transporter:: finish(std::size_t meshid = 0)
Normal finish of time stepping.
Parameters  

meshid in  Mesh id 
void inciter::Transporter:: pup(PUP::er& p) override
Pack/Unpack serialize member function.
Parameters  

p in/out  Charm++'s PUP::er serializer object reference 
void inciter::Transporter:: operator(PUP::er& p,
Transporter& t)
Pack/Unpack serialize operator.
Parameters  

p in/out  Charm++'s PUP::er serializer object reference 
t in/out  Transporter object reference 
Variable documentation
std::vector<CProxy_Sorter> inciter::Transporter::m_sorter
Mesh sorter array proxy (one per mesh)
Number of mesh elements (per mesh)