View
38
Download
4
Embed Size (px)
Citation preview
Tobias Fuchs
Ludwig-Maximilians Universität (LMU) München
DASH Locality Hierarchies
An Information System for Hierarchical Locality
Motivation
Motivation:Portable efficiency of ported applications like LULESH and graph applications for heterogeneoussystems
hwloc static hierarchical hardware topology
DASH locality as view from variable run-time configuration
… extended to full distributed topology… supporting heterogeneous systems, esp. Intel MIC… designed for use cases like hierarchical graph partitioning
DASH Locality Hierarchies
Trees vs. Reality
We do not model hardware locality as trees
… because many represented systems are anything but trees
DASH Locality Hierarchies
http://frankdenneman.nl/2016/07/11/numa-deep-dive-part-3-cache-coherency/
Related Concepts
Combining existing abstractions
- Hierarchical Place Treesabstraction of locality scopes and process mapping
- hwlocnode-level machine topology and hardware capacityfocuses at topology discovery (once, at startup) and provides queries on topology data
Locality Hierarchies
- are specifically designed for user-specified views and modifications of the topology representation
- are specifically designed for hierarchical process structures (teams)
DASH Locality Hierarchies
Examples: Xeon PHI
DASH Locality Hierarchies
Compute Node with Xeon Phi Accelerators as seen by hwloc
Examples: Xeon PHI
Compute Node with Xeon Phi Accelerators as seen by DASH
DASH Locality Hierarchies
Examples: Xeon PHI
DASH Locality Hierarchies
Examples: Xeon PHI
DASH Locality Hierarchies
Key Functionalities
Why we need locality hierarchies in DART/DASH:
- Locality-optimized grouping of processes into teams
- Load-balancing, requires: topology information
… obviously, to find suitable processes for balancing hardware capacities
… like number of cores/threads and (shared) memory capacitiesavailable to processes
- Dynamic distance measures: variable at run time instead of staticdistance matrix
DASH Locality Hierarchies
Locality Domains
DASH Locality Hierarchies
Splitting / Grouping
DASH Locality Hierarchies
Heterogeneous min_element
DASH Locality Hierarchies
Some fundamental operations on locality hierarchies
Basic principle of usage:( (filter/select) (group/split) )*
C API, bindings for Python in development, Fortran bindings feasible
C API
DASH Locality Hierarchies
Usage of C++ API
// split into num_groups teams at NUMA locality scopeauto & new_team = dash::Team::All().locality_split(
dash::util::Locality::Scope::NUMA,// optional, defaults to one team per locality scopenum_groups);
// split into two teams: leader team and workersauto & new_team = dash::Team::All().leader_split(
dash::util::Locality::Scope::NODE);
// split into teams by predicateauto & new_team = dash::Team::All().specific_split(
[](dash::util::LocalityDomain ld) {// ...return new_team_id_for_ld;
});
DASH Locality Hierarchies
Leader Groups
DASH Locality Hierarchies
Logical vs. physicaltopology
Domain Aliasing
DASH Locality Hierarchies
Splitting / Grouping
DASH Locality Hierarchies
Splitting / Grouping
DASH Locality Hierarchies
Basic Distance Model
DASH Locality Hierarchies
Links
DASH Locality Hierarchies
www.dash-project.org
www.github.com/dash-project
Stand-alone library of DASH locality hierarchies (libdhloc)available in ~4 weeks