Embed Size (px)
Citation preview
New Developments for PAPI 5.6+Anthony Danalis, Heike Jagode, and Jack Dongarra
University of Tennessee, KnoxvilleVince Weaver and Yan Liu
University of Maine
INTRODUCTION
PART 1: PAPI for Power‐Aware Computing
PART 2:
PAPI’s Counter Inspection Toolkit (CIT)
PART 3:
Modernizing PAPI Infrastructure
SUPPORTED ARCHITECTURES
3RD PARTY TOOLS APPLYING PAPI
• PAPI provides a consistent interface (and methodology) for hardware performance counters, found across a compute system:� i. e., CPUs, GPUs, on- and off-chip memory, interconnects, I/O system, file system, energy/power, etc.
• PAPI enables software engineers to see, in near real time, the relationship between �software performance and hardware events across the entire compute system.
Intel Knights Landing: FLAT mode• Entire MCDRAM is used as addressable memory� ➔ memory allocations are treated similarly to DDR4 memory allocations
PAPI for power-aware computing• We use PAPI’s latest powercap component for
measurement, control, and performance analysis.
• In the past, PAPI power components supported only reading power information.
• New component exposes RAPL functionality to allow users to read and write power.
• Study numerical building blocks of varying computational intensity.
• Use PAPI powercap component to detect power optimization opportunities.
• Cap the power on the architecture to reduce power usage while keeping the execution time constant ➔ energy savings.
Key ConceptsGoal:
• Create a set of micro-benchmarks for illustrating details in hardware events and how they relate to the behavior of the micro-architecture
Target audience:
• Performance-conscious application developers
• PAPI developers working on new architectures (think preset events)
• Developers interested in validating hardware-event counters
QUESTION
How many branch instructions will these codes execute per iteration?
The expectation is that both codes will execute 2.
The measured answer is 2 for the first and 2.5 for the second!
Can you guess why?
Improved PAPI Test Infrastructure• The existing PAPI testsuite is used to test the correctness of
PAPI before release.
• The hardware and operating systems used by PAPI are always changing, and some of the existing tests were outdated or gave false negatives.
• Existing tests were checked to ensure accurate results on modern hardware.
• New counter validation tests were created which should provide a sanity check when bringing up support for a new processor architecture.
Low-Overhead PAPI_read() support• Traditionally PAPI_read() counter reads went through the
standard Linux read() system call, which can be slow (around 1,000 cycles).
• x86 hardware supports a userspace rdpmc() instruction that bypasses the kernel and requires 200 cycles (a 5× speedup).
• Various bugs in the Linux kernel around this interface were found and fixed so that rdpmc() can be enabled by default.
Enhanced Sampling Interface• PAPI currently has a limited counter sampling interface that
only allows gathering the instruction pointer at regular intervals
• Modern processors support much richer sampling information, including the cause of cache misses, where in the cache hierarchy the miss happened, and the cycles taken.
• We extend the PAPI sampling interface to provide this additional sampling information.
Heike Jagode, Azzam Haidar, Phil Vaccaro, Asim YarKhan, and Stan Tomov
ACKNOWLEDGMENTS This material is based upon work supported in part by the National Science Foundation NSF under awards No. 1450429. A portion of this research was supported by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration.
Vince Weaver and Yan LiuAnthony Danalis, Heike Jagode, and Hanumanthrayappa
Power awareness for Level-3 BLAS: DGEMM68 cores KNL, Peak DP = 2,662 Gflop/s Bandwidth MCDRAM ~425 GB/s DDR4 ~90 GB/s
• DGEMM is run repeatedly for a fixed matrix size of 18K per iteration.
• For each iteration, we dropped the power in 10 or 20 Watts steps.
• Powercap achieves its power limiting though DVFS scaling. See frequency value, which — as it drops — corresponds to a decrease in power.
Power awareness for Level-3 BLAS: DGEMV68 cores KNL, Peak DP = 2,662 Gflop/s Bandwidth MCDRAM ~425 GB/s DDR4 ~90 GB/s
• DGEMV is run repeatedly for a fixed matrix size of 18K per iteration.• For each iteration, we dropped the power in 10 or 20 Watts steps.• Frequency is not affected until the power caps kick in.
Power awareness for applications: Jacobi• Solving Helmholtz equation with Jacobi Iterative Method (grid size 12,800 x 12,800, 2D 5-point stencil)➔ requires multiple memory accesses per update➔ high-memory bandwidth and low computational intensity
Lesson for DGEMM-type operations (compute intensive):• Both data allocation schemes provides roughly the same performance. This is caused by the high ratio of FLOPs to
bytes in the compute-intensive routines, which allows for as much cache reuse as possible while hiding the data transfer between memory and cache levels.
• Performance drops with decreasing power caps, which is expected for compute-intensive kernels.
Lesson for DGEMV-type operations (memory bound):• Performance varies between the two storage options: Data allocations in MCDRAM results in much higher performance
for memory-bound kernels (BW for MCDRAM is 425GB/s vs. 90 GB/s for DDR4).• Dgemv is bound by the reading of matrix A. Depending on its storage location, performance and power may be affected.
• We see lower package-power-usage but higher DDR4-power-usage.• Capping at 140 W improves energy efficiency by ~10%.• Capping at 120 W results in a 17% energy savings with some performance degradation (8%).
• Capping at 180 Watts renders in ~15% of energy saving with a negligible performance penalty (of less than 5%).
Lesson for Jacobi iteration:• Computation is about 3.5X faster when the data is allocated in MCDRAM compared to DDR4.
• MCDRAM: Capping at 170 Watts improves energy efficiency by ~14% without any loss in time to solution.
• DDR4: capping at 135 Watts improves energy efficiency by ~25% without any loss in time to solution.
Power awareness for applications: Lattice-Boltzman• Lattice-Boltzmann simulation of computational fluid dynamics (from SPEC 2006 benchmark)➔ high-memory bandwidth and low computational intensity
Lesson for Lattice-Boltzmann:• Computation is about 3.6X faster when the data is allocated in MCDRAM compared to DDR4.
• For MCDRAM, capping at 190 results in energy saving of ~6% without loss of time to solution.
• For DDR4, capping at 130 Watts improves energy efficiency by ~12%.
Native Event Characterization
Highlighting non-obvious behavior
Cortex A8, A9, A15, ARM64 Gemini and Aries interconnect, power Blue Gene Series, Q: 5-D Torus,I/0 System, EMON power, energy
Tesla, Kepler: CUDA support formultiple GPUs; PC Sampling NVML Virtual Environment Virtual Environment
Power Series Westmore, Sandy/Ivy Bridge, Haswell,Broadwell, Skylake(-X), Kaby Lake
RAPL (power/energy),power capping
PaRSECUTK
http://icl.utk.edu/parsec/
HPCToolkitRice University
http://hpctoolkit.org/
TAUUniversity of Oregon
http://www.cs.uoregon.edu/research/tau/
ScalascaFZ Juelich, TU Darmstadt
http://scalasca.org/
VampirTU Dresden
http://www.vampir.eu/
PerfSuite NCSA
http://perfsuite.ncsa.uiuc.edu/
Open|SpeedshopSGI
http://oss.sgi.com/projects/openspeedshop/
SvPabloRENCI at UNC
http://www.renci.org/research/pablo/
ompPUTK
http://www.ompp-tool.com/
Score-P http://score-p.org/
KNC, Knights Landingincluding power/energy
0
20
40
60
80
100
16 64 256 1024 4096 16384 65536
Aver
age
coun
t per
100
acc
esse
s
Buffer size in KB
L1 HITL2 HITL3 HIT
0
20
40
60
80
100
16 64 256 1024 4096 16384 65536
Aver
age
coun
t per
100
acc
esse
s
Buffer size in KB
L1 MISSL2 MISSL3 MISS
0
10
20
30
40
50
60
70
80
90
100
16 32 64 128 256 512 1024 2048 4096
Aver
age
coun
t per
100
acc
esse
s
Buffer size in KB
unit size= 64Bunit size=128Bunit size=256B
0
10
20
30
40
50
60
70
80
90
100
256 768 1280 1792 2304 2816 3328 3840
Aver
age
coun
t per
100
acc
esse
s
Buffer size in KB
ppb=4ppb=8
ppb=16ppb=32ppb=33ppb=34ppb=35
TDP = 215W
Boxplot showing read latency for various versions of PAPI and the large improvement by using rdpmc
Comparison of historical performance counter interfaces (perfmon2, perfctr) showing that perf_event rdpmc matches even the best historical interface.
Time (sec)0 10 20 30 40 50 60 70 80 90 100 110
Ave
rage
pow
er (W
atts
)
0 20 40 60 80 100120140160180200220240260280300320
1991
8.22
1383
1901
8.36
1341
1785
8.58
1314
1615
8.57
1325
1340
8.02
1419
1154
7.36
1562
971
6.66
1715
785
5.81
1982
575
4.64
2489
Performancein Gflop/s
Gflops/WattsJoules
Accelerator Power Usage (PACKAGE)Memory Power Usage (DDR4)Max power limit set
MH
z
0 200 400 600 800 100012001400160018002000
Frequency
Time (sec)0 10 20 30 40 50 60 70 80 90 100 110
Ave
rage
pow
er (W
atts
)
0 20 40 60 80 100120140160180200220240260280300320
1997
8.82
1303
1904
8.92
1279
1741
8.95
1267
1589
9.04
1253
1328
8.47
1345
1137
7.73
1480
956
6.95
1661
773
6.03
1904
560
4.72
2443
Performancein Gflop/s
Gflops/WattsJoules
Accelerator Power Usage (PACKAGE)Memory Power Usage (DDR4)Max power limit set
MH
z
0 200 400 600 800 100012001400160018002000
Frequency
DDR4 MCDRAM
MCDRAM
DDR4
MCDRAM
Time (sec)0 11 22 33 44 55 66 77 88 99 110 121 132 143 154
Ave
rage
pow
er (W
atts
)
0 20 40 60 80 100120140160180200220240260280300320
2185
0.12
2588
2185
0.12
2623
2185
0.12
2639
2185
0.12
2664
2185
0.12
2661
2184
0.13
2558
2182
0.13
2432
2082
0.14
2305
1978
0.14
2240
Performancein Gflop/sAchievedBandwidth GB/sGflops/WattsJoules
Accelerator Power Usage (PACKAGE)Memory Power Usage (DDR4)Max power limit set
MH
z
0 200 400 600 800 100012001400160018002000
Frequency
Time (sec)0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64
Ave
rage
pow
er (W
atts
)
0 20 40 60 80 100120140160180200220240260280300320
8433
50.
3981
5
8333
10.
3980
5
8232
80.
4271
779
317
0.45
686
6526
10.
4274
5
5622
50.
3882
2
4718
80.
3490
9
3815
00.
2910
77
2811
10.
2413
61
Performancein Gflop/sAchievedBandwidth GB/sGflops/WattsJoules
Accelerator Power Usage (PACKAGE)Memory Power Usage (DDR4)Max power limit set
MH
z
0 200 400 600 800 100012001400160018002000
Frequency
DDR4
MCDRAMDDR4DDR4 MCDRAM
Time (sec)0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32
Ave
rage
pow
er (W
atts
)
0 20 40 60 80 100120140160180200220240260280300
2770 joules2878 joules2689 joules2438 joules2137 joules2731 joules
DDR_215WattsDDR_200WattsDDR_180WattsDDR_160WattsDDR_140WattsDDR_120Watts
Time (sec)0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Ave
rage
pow
er (W
atts
)
0 20 40 60 80 100120140160180200220240260280300
826 joules842 joules724 joules803 joules1066 joules1865 joules
MCDRAM_215WattsMCDRAM_200WattsMCDRAM_180WattsMCDRAM_160WattsMCDRAM_140WattsMCDRAM_120Watts
Time (sec)0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68
Ave
rage
pow
er (W
atts
)
0 20 40 60 80 100120140160180200220240260280300
9064 joules9196 joules9194 joules9178 joules9005 joules8084 joules8467 joules
DDR_270wattsDDR_215wattsDDR_200wattsDDR_180wattsDDR_160wattsDDR_140wattsDDR_120watts
Time (sec)0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54
Ave
rage
pow
er (W
atts
)
0 20 40 60 80 100120140160180200220240260280300
3050 joules
2981 joules
2872 joules
3043 joules3383 joules
4033 joules5734 joules
MCDRAM_270wattsMCDRAM_215wattsMCDRAM_200wattsMCDRAM_180wattsMCDRAM_160wattsMCDRAM_140wattsMCDRAM_120watts
