Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 1
May 31, 2018Shunichi KikuchiFujitsu Limited
System Packaging Solution for Future High Performance Computing
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 2
• Future HPC spec. from the transition of TOP500• Possible System Packaging for the future
Outline
For the advent of Exascale Computing‐ A prediction from TOP500 history ‐
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 3
1.0E‐01
1.0E+00
1.0E+01
1.0E+02
1.0E+03
1.0E+04
1.0E+05
1.0E+06
1.0E+07
1.0E+08
1.0E+09
1.0E+10SUM
TOP
#500
1PF
1EF
1TF
“TOP”s have been recorded with 3.5‐4 year delay from “SUM”s. 1EF will be recorded in 2021‐2022.
ʼ97
ʻ08
Max
imal
LIN
PACK
Per
form
ance
(Fl
ops)
6 times4 times
845PF
The number of CPUs and Accelerators (TOP10)
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 4
1
10
100
1,000
10,000
100,000
1,000,000
Minimum
Maximum
On averageBG/L
ASCI Red
BG/LBG/Q
K
The number of them seems to have saturated since 2007.
Trend of the number of cores in system (TOP10)
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 5
Performance improvement needs to increase the number of cores.
1
10
100
1,000
10,000
100,000
1,000,000
10,000,000
100,000,000
Minimum
Maximum
On average
Multicore CPU6 out of TOP10
BG/L
Tianhe-2TaihuLight
# of CPUs and Accelerators : 10,000 – 100,000+ pcsPerformance per each : 100,000 – 10,000 GFlops[Exascale era]
(2core CPU)
(12core CPU)
(260core CPU)
(57core ACC)
Performance per power
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 6
25‐50 GF/W will be a key factor to implement 1EF.
Linpack Top1
TOP10 feature @ Nov. 2017
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 7
CPU Accelerator(bootable) CPU+Accelerator
JCAHPC
NSCC
JAMSTEC
ORL
LANL
NERSCLLNL
NRCPC
6.1GF/W by 260core CPU
14.2GF/W by 1984coreAcc.
8.6GF/W by GPU & CPU8.6GF/W by GPU & CPU
TaihuLight
Sequoia
K computer
Shoubu
TitanOak‐forest PACS
Cori
TrinityTianhe‐2
Piz Daint
No.1
No.6
No.10
No.2
No.3 CSCS
No.4
No.5
No.7
No.8
No.9
There are three types of configurations.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Other
Intel
Cray
IBM
InfiniBand
Ethernet
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Other
Intel
Cray
IBM
InfiniBand
Ethernet
I/O interconnects in system
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 8
Top500
Top20
Still major
Custom
Major
Both InfiniBand and Ethernet are popular as a whole.
Communication links for short‐distance
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 9
ACSI Purple ‐3K links @2.5G in 2005Roadrunner ‐40K links @5G in 2008Power 775 ‐60K fibers/rack @10G in 2011Sequoia ‐620K links @10G in 2011
Transition of interconnects in system
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 10
Optical interconnects have been applied between racks and between shelfs.They will be near to CPUs and Accelerators, depending on system requirements, technology evolution (including costs).
Speedconnected -10Gbps 25Gbps 50Gbps-
Distance ComponentExampleSystems
K computerFX10 FX100 Future
systemBetween
racks Elec Opto Opto - 100m Cable
Between shelfs Elec Opto Opto - 3m Cable
Between boards in
shelf
-- Opto Opto- 1m
Cable-- -- Opto Cable
Elec -- Opto Board, Back plane
On board
Elec -- Elec
- 0.3m
Board-- -- Opto Board(Fiber sheet, Waveguide)
-- Elec/Opto Elec/Opto Board&cable
-- -- Elec Cable-- -- Opto Cable
Interconnects on board
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 11
Altera, Avago, 2012
PETRA/NEDO, 2016
Cu cableOptical cables,Fiber sheet orpolymer waveguides
Cu patternCu pattern
board
CPU/ACC package Fiber sheet orpolymer waveguides
Related technologies in the past ECTCs
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 12
Compass EOS@ECTC2014
336ch, 64Gb/s/mm2
(168*8Gb/s=1.344Tb/s)348ch, 15Gb/s/mm2
IBM@ECTC2016 AIST/PETRA@ECTC2016
96ch, 2.4Tb/s
CPU/Accelerator chip trend (in general)
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 13
0.1
1.0
10.0
1995 2000 2005 2010 2015 2020Year
Clock Frequency (GHz) Die size (mm2)
200
300
400
500
600
700
800
900
1995 2000 2005 2010 2015 2020Year Year
0
50
100
150
200
250
300
350
400
450
500
1995 2000 2005 2010 2015 2020
Heat dissipation (Watts)
There seems no significant growth in clock frequency since 2007.Die size has been reached to the manufacturing limit.Heat dissipation may be controlled toward green computing.
Power delivery trend
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 14
Source: Daimler
Exascaler Inc. @SC17 nVIDIA
Mounted DDCs(ex. 48V1V)
Creating electrical power from
At Package/Module Level
At Datacenter LevelHydrogenNatural gas etc.
Possible Processor Package and Module
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 15
High density package Standard I/F module
Package substrate
Silicon‐Photonics
LD
Processor stacking(optional)
Transition of package substrate technology
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 16
Technology progress and design optimization may be able to mitigate the current patterning and cost issues.
2010 20202015
1μm
100μm
10μm
0.1μm
year
Patternpitch
Chip(Semiconductor)
Interposer
Substrate
Future prospect of system packaging
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 17
High density rack mountingHigh density system packaging
CPU/MEM package
Acceleration package
socket
High performance card with standard interface Highly dense rack mounting
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 18
Double sided cooling unit(CPU/MEM/DDC)
CPU
GPU
DLU
FPGA
DAU
FPGA
Rackmountable
CDU
Cooling platejackets
Future prospect of system packaging
Quick connecting/disconnecting plug
Cooling technology
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 19
A wide range of choices should be prepared in order to match with customers’ requirements :
Coolant availabilityLimitation of ambient temperatureLimitation of power consumptionLimitation of installation space
Liquid immersion cooling・Submersion of all parts・Natural convection・Forced circulation・Boiling
Liquid circulation cooling・Hot water・Chilled water・Micro-channel・Impinging
IBM
Air-forced cooling・Heat sink・Heat sink w/ heat pipe・Vapor and liquid loop・Vapor chamber・Thermo-siphon
airradiator
pump
Coolingunit
liquid circulation, and liquid immersionAir‐forced,
Cooling from the viewpoint of system packaging
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 20
A lot of PCB units
Liquid immersion
High density main board
Liquid circulation
17GF/W@2017
1.E-04
1.E-02
1.E+00
1.E+02
1.E+04
1.E+06
1990 2000 2010 2020 2030 2040 2050 2060
1.85x106 GF/W
2039 2055
BG/L
Earth SimlRoadrunner
Shoubu
NWT0.012GF/W@2002
Toward Big‐data era and the beyond
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 21
Source: FORTUNE: July 10, 2017
Human Brain37 PF (*1)/20W=1.85x106 GF/W
200PF@2018, 10times/4yrs, 200*10^(21/4)=36ZF@203920MW /20W = 1Million Processors, 37PF*1M=37ZF@2039
If technology is continuously evolved, we may reach…
1 billion times as fast as Earth Simulator(36TF) in 2002‐2004
Performan
ce/Pow
er
(*1) Source: 36.8PF, 3.2PBytesBest of H+ Magazine 2008‐2010 Volume 1
Tera Peta Exa Zeta
17GF/W@2017
1.E-04
1.E-02
1.E+00
1.E+02
1.E+04
1.E+06
1990 2000 2010 2020 2030 2040 2050 2060
1.85x106 GF/W
2039 2055
BG/L
Earth SimlRoadrunner
Shoubu
NWT0.012GF/W@2002
Toward Big‐data era and the beyond
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 22
Source: FORTUNE: July 10, 2017
Human Brain
Or else?Quantum Computing?Digital Annealing?
37 PF (*1)/20W=1.85x106 GF/W
200PF@2018, 10times/4yrs, 200*10^(21/4)=36ZF@203920MW /20W = 1Million Processors, 37PF*1M=37ZF@2039
If technology is continuously evolved, we may reach…
Which type of computing can be near to human brains?
Quantum Annealing?
1 billion times as fast as Earth Simulator(36TF) in 2002‐2004
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
Performan
ce/Pow
er
(*1) Source: 36.8PF, 3.2PBytesBest of H+ Magazine 2008‐2010 Volume 1
Conventional HPC?
Tera Peta Exa Zeta
Various device configurations for different issues
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 23
Health care Traffic Relaxation Sustainable Food and Agriculture
Reproducible Energy Education for everyone Sustainable Environment
It is difficult to solve and find a better goal by using one type of HPC. A suitable system packaging solution can be chosen in the coming several years for each.
Fujitsu’s High‐end HPC Development
2018 IEEE 68th Electronic Components and Technology Conference │ San Diego, California │ May 29 – June 1, 2018 24
Fujitsu has provided HPC systems with original technologies, developed for over 40 years, to accelerate advanced research
© RIKEN
The K computer continues to be competitive in various fields; from advanced research to manufacturing
K computer
RIKEN and Fujitsu are developing the Post‐K to achieve superior application performance
Post‐K computer
HPCGNo.1(2017)
Graph500No.1(2017)
Gordon Bell Prize Finalist
(2016)
PRIMEHPC FX10
PRIMEHPC FX100
24