Upload
jay-etchings
View
320
Download
0
Embed Size (px)
Citation preview
Executive Summary
ASU Senior Leadership has charged the Complex Adaptive Systems Network team with articulating the cyberinfrastructure necessary to successfully execute its portfolio of projects. It is the assessment of the team that a Next Generation Cyber Capability (NGCC) is required to meet the unique needs of CASN projects. Conceptually, the NGCC is composed of standing capabilities and dynamic, on-demand, virtual resources that grow and contract as CASN efforts require. The NGCC is a synergistic, systems whole that uniquely synthesizes physical and logical infrastructure which is fulfilled through dedicated staff resources working as a team. Each infrastructure component is itself a collection of diverse capabilities executed through staff with specialized skills. While components of the NGCC reside in various isolated niches throughout the ASU community, to successfully meet the needs of CASN projects requires a dedicated enterprise.
Dell Enterprise is a strategic partner of Dr. Buetow and Arizona State University
Scratch Space
HPC Parallel HPC SMABig Data
ContextOntologies
Data ElementsInformation Models
Middleware
AnalyticGeneral Purpose
GenomicBig Data
Data ResourcesFile SystemRelationalKey/Value
TransactClinical Research
Life Science researchQualitative Research
Capacity
TransCORE Framework Knowledge Engine
Transactional
Data Reservoir
TransactionalHPCC BIG DATA
Exp
an
sio
n to
Pu
blic
Clo
ud
Exp
an
sio
n to
Pu
blic
Clo
ud
Ne
xt G
en
era
tion
Co
mp
ute
Clu
ste
r
V.1
.5a
/ Pa
ge
1N
GC
C G
lob
al T
op
olo
gy D
esig
n0
9/0
6/2
01
3Ja
y E
tch
ing
s, S
olu
tion
s A
rch
itect, D
ell E
nte
rpris
e
TRANSACTIONAL CLUSTER COMPONENT
IS BUILT FROM A 20 NODE CLUSTER
OFFERING 240 CORES, 640GB OF DDR3
AND 40-200GB SOLID STATE DRIVES IN
AN M1000E CHASSIS WITH 40GBE
UPLINK CONNECTS, CONVERGED
NETWORK ADAPTERS FOR ISCSI
OFFLOAD.
HA STORAGE
40Gbe VLT Fabric
BIG DATA CLUSTER CONFIGURATION
CONSISTING OF 40 -10GBE
INTERCONNECTED NODES WITH DUAL 8
CORE SERVERS WITH 128GB DDR3
MEMORY, DUAL 10GBE INTERCONNECTS,
24-2TB DRIVES AND DUAL PSUS. HIGH SPEED 40GBE CONNECTIONS TO THE
SUPPORTING ENVIRONMENT ARE
PROVIDED.
100 NODE / 800 CORE CLUSTER WITH
32GB MEMORY PER NODE CONNECTED
VIA INFINIBANC QDR INTERCONNECTS.
CLUSTER HAS 250TB OF FAST SCRATCH
STORAGE SPACE IN THE LUSTRE FILE
SYSTEM. A HIGH SPEED 40GBE
INTERCONNECT TO THE SUPPORTING
ENVIRONMENT IS PROVIDED.
ContextOntologies
Data ElementsInformation Models
Middleware
AnalyticGeneral Purpose
GenomicBig Data
Data Resources
File SystemRelationalKey/Value
0202
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
0202
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
HIGHLY AVAILABLE COMPELLENT
STORAGE WITH CLUSTERED NAS/NFS/
CIFS CAPABILITIES, 30 ENCLOSURES
WITH 12-3TB (360) 7.2K DRIVE, 1PB
REPLICATED TO 2PB CAPACITY ARRAY VIA
FCIP FABRIC EXTENSION BRIDGE.
1080TB
TransactClinical Research
Life Science researchQualitative Research
Capacity
0202
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
0202
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
HIGHLY AVAILABLE COMPELLENT
STORAGE CONSISTING OF 21
ENCLOSURES WITH 12- 4TB (252) 7.2K
DRIVES. REPLICATION TARGET FOR
PRIMARY ETL INGESTION ARRAY.
1008TB
FCIP Replication via Fabric Extension
TransCORE Framework Knowledge Engine
Transactional
Scratch SpaceBig Data
HPC SMAHPC Parallel
Data Reservoir
High Speed Connectivity
16GB FC SAN
Connection(s)
Jay Etchings, Solutions Architect, Dell Enterprise
Big Data
New models of computing intimately combine processing, large data stores, and data transfer through a unique file system (Hadoop/MapReduce) that provides novel computational strategies.
Standing Big Data Capacity would consist of 10GE Hadoop 40 node cluster composed of dual 8 core servers each with 72GB DDR3 memory, 2 X 10GE, 24 x 2TB storage, with dual PSUs. To facilitate potential expansion the servers would be configured over multiple rack with appropriate switches added as growth might dictate. High speed L3 network connection to the transactional infrastructure is required.
Hadoop is the popular Open Source implementation of MapReduce, a powerful tool designed for deep analysis and transformation of very large data sets.
Big Data
Next Generation Compute Cluster -HPCC
V.1.5a / Page 3 Server Roles Definitions 09/06/2013 Jay Etchings, Solutions Architect, Dell Enterprise
PowerEdge R720 X 4 Units
Input Pow er: 545w atts 1859.6btu/h *
Maximum Potential Pow er:* 750.9w atts 2562.2btu/h *
Input Current: 2.5amps
Sound Pow er Level: 6.1bels *
Airf low Rate: 18.5l/s 39.1CFM
Weight: 29.5kg 65lbs
Air Temperature Rise: 25.1C 45.1F
PowerEdge R720XD X 40
Units
Input Pow er: 551w atts 1880.1btu/h *
Maximum Potential Pow er:* 789.5w atts 2694btu/h *
Input Current: 2.5amps
Sound Pow er Level: 6.3bels *
Airf low Rate: 17l/s 35.9CFM
Weight: 29.2kg 64.4lbs
Air Temperature Rise: 27.6C 49.7F
Workload: Computational
Pow erEdge R720xd Pow erEdge R720xd
Processor
Intel Xeon E5-2670 2.60GHz, 20M Cache,
8.0GT/s QPI, Turbo, 8C, 115W, Max Mem 1600MHz
Additional Processor
Intel Xeon E5-2670 2.60GHz, 20M Cache,
8.0GT/s QPI, Turbo, 8C, 115W
Memory Configuration Type Performance Optimized
Memory DIMM Type and Speed 1600 MHz RDIMMS
Memory Capacity
8 x 16GB RDIMM, 1600MT/s, Low Volt, Dual Rank,
x4 Data Width
Operating System No Operating System
OS Media Kits No Media Required
Chassis Configuration Chassis w ith up to 24, 2.5" Hard Drives
RAID Configuration No RAID for H310 (1-16 HDDs)
RAID Controller
PERC H710P Integrated RAID Controller, 1GB NV
Cache
24 x 1TB 7.2K RPM SATA 3Gbps 2.5in Hot-plug
Hard Drive
2 x 500GB 7.2K RPM SATA 3Gbps 2.5in Hot-plug
Hard Drive
Pow er Supply
Dual, Hot-plug, Redundant Pow er Supply (1+1),
750W
2 x NEMA 5-15P to C13 Wall Plug, 125 Volt, 15 AMP,
10 Feet (3m), Pow er Cord
No Pow er Cord
Pow er Management BIOS Settings Pow er Saving Dell Active Pow er Controller
Embedded Systems Management iDRAC7 Enterprise w ith Vflash, 16GB SD Card
Add-in Netw ork Adapter
Broadcom 57810 DP 10Gb DA/SFP+ Converged
Netw ork Adapter
Select Netw ork Adapter
Broadcom 57800 2x10Gb SR/SFP+ + 2x1Gb BT
Netw ork Daughter Card, w ith SR Optics
Rack Rails
ReadyRails Sliding Rails With Cable Management
Arm
Pow er Cords
Hard Drives
Workload: Computational
Pow erEdge R720 Pow erEdge R720
Processor
Intel Xeon E5-2670 2.60GHz, 20M Cache,
8.0GT/s QPI, Turbo, 8C, 115W, Max Mem 1600MHz
Additional Processor
Intel Xeon E5-2670 2.60GHz, 20M Cache,
8.0GT/s QPI, Turbo, 8C, 115W
Memory Configuration Type Performance Optimized
Memory DIMM Type and Speed 1333 MHz LRDIMMs
Memory Capacity
16 x 32GB LRDIMM, 1333 MT/s, Low Volt, Quad
Rank, x4 Data Width
OS Media Kits No Media Required
Chassis Configuration 2.5" Chassis w ith up to 16 Hard Drives
RAID Configuration RAID 1 for H710P/H710/H310 (2 HDDs)
RAID Controller
PERC H710P Integrated RAID Controller, 1GB NV
Cache
Hard Drives
10 x 300GB 15K RPM SAS 6Gbps 2.5in Hot-plug
Hard Drive
Pow er Supply
Dual, Hot-plug, Redundant Pow er Supply (1+1),
750W
Pow er Cords
2 x NEMA 5-15P to C13 Wall Plug, 125 Volt, 15 AMP,
10 Feet (3m), Pow er Cord
Pow er Management BIOS Settings Pow er Saving Dell Active Pow er Controller
Embedded Systems Management iDRAC7 Enterprise w ith Vflash, 8GB SD Card
Select Netw ork Adapter
Broadcom 57800 2x10Gb SR/SFP+ + 2x1Gb BT
Netw ork Daughter Card, w ith SR Optics
Add-in Netw ork Adapter
2 x Broadcom 57810 DP 10Gb SR/SFP+ Converged
Netw ork Adapter, w ith SR Optics
Rack Rails
ReadyRails Sliding Rails With Cable Management
Arm
Ha
do
op
Po
we
r Co
olin
g S
pe
cific
atio
ns
HD
FS
Hadoop H-Rack-ANGCC
Clu
stere
d N
od
e N
etw
ork
RS
-232
PSU0SYS ALM
LOC
IPMI ET
H
RESET PSU1Master
STACK ID USB-A
USB-B
SSD
60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3
124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67
Force10 Z9000
RS
-232
PSU0SYS ALM
LOC
IPMI ET
H
RESET PSU1Master
STACK ID USB-A
USB-B
SSD
60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3
124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67
Force10 Z9000
Sla
ve
No
de
sM
an
ag
em
en
tM
aste
rN
etw
ork
H-Rack-A10 Slave Nodes
R720XD2-Xeon E5-2670
128GB DDR324-2TB 7.2k
4-10Gb QFSP+640C/1280T
1.920PB
ManagementExpansion
FPM 185RM LED console
4020 Rack Frame2-Managed PDU
C13/L6-30A
Master/HA Master3 R720
2-Xeon E5-2670512GB DDR3
10-300GB 15k SAS6-10Gbe QSFP+
Rack Aware TOR2-S4810 10GB
40Gbe HA Stack
Ne
xt G
en
era
tion
Co
mp
ute
Clu
ste
r -Ha
do
op
/ Big
Da
ta
V.1
.5a
/ Pa
ge
5H
-Ra
ck-A
09
/06
/20
13
Ja
y E
tch
ing
s, S
olu
tion
s A
rch
itect, D
ell E
nte
rpris
e
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
AC OK
100-240V ~ 6A50 60Hz
100-240V ~ 6A50 60Hz
AC OK
PSU0
PSU1
AC OK
100-240V ~ 6A50 60Hz
100-240V ~ 6A50 60Hz
AC OK
PSU0
PSU1
2
VLT Domain 1
COREVLT Domain 2
10G
be
SF
P+
2
Quad 40Gbe Trunk Links
A & B Managed Power28C13 / 2-L6-30A
Distributed Data ProcessingJob TrackerMap Reduce
Secondary Name Node
7.8kW Peak6.1kW Typ
CB
1C
B2
CB
5C
B6
CB
3C
B4
CB
1C
B2
L1
CB
3C
B4
L2
CB
5C
B6
L3
ON
OFF
ON
OFF
ON
OFF
ON
OFF
ON
OFF
ON
OFF
2
2
2
2
2
2
2
2
2
2
40GB LAG
10GB Link
10GB iSCSI
10GB Trunk
1GB iSCSI
16GB FC
1Gbe Mgmt.
KMM FPM185
Jay Etchings, Solutions Architect, Dell Enterprise H-RACK-A Power Peak Pwr Typical BTU Peak BTU Typ Max Pot.
Totals 7865 6156 26819.65 20991.96 8661
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
FDX
/HDX
LNK
/ACT
DIAG
STACK ID
G1 G2 G4G3
LNK/ACT FDX/HDX
MASTER RPS PWR
FAN TEMP
RESET
SELECT
LNK
/ACT
LNK
/ACT
STACKING
PORT
1 2 3 4
5 6 7 8
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
FDX
/HDX
LNK
/ACT
DIAG
STACK ID
G1 G2 G4G3
LNK/ACT FDX/HDX
MASTER RPS PWR
FAN TEMP
RESET
SELECT
LNK
/ACT
LNK
/ACT
STACKING
PORT
1 2 3 4
5 6 7 8
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Hadoop H-Rack-BNGCC
Master
Ne
two
rkM
an
ag
em
en
tS
lav
e N
od
es
H-R
ac
kA
H-R
ac
kC
H-R
ac
kE
Ne
xt G
en
era
tion
Co
mp
ute
Clu
ste
r -Ha
do
op
/ Big
Da
ta
V.1
.5a
/ Pa
ge
6H
-Ra
ck-B
08
/26
/20
13
Ja
y E
tch
ing
s, S
olu
tion
s A
rch
itect, D
ell E
nte
rpris
e
0202
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
AC OK
100-240V ~ 6A50 60Hz
100-240V ~ 6A50 60Hz
AC OK
PSU0
PSU1
AC OK
100-240V ~ 6A50 60Hz
100-240V ~ 6A50 60Hz
AC OK
PSU0
PSU1
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
RS
-232
PSU0SYS ALM
LOC
IPMI ET
H
RESET PSU1Master
STACK ID USB-A
USB-B
SSD
60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3
124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67
Force10 Z9000
RS
-232
PSU0SYS ALM
LOC
IPMI ET
H
RESET PSU1Master
STACK ID USB-A
USB-B
SSD
60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3
124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67
Force10 Z9000
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
iDRAC1 2 3 4
ST
21
2
1
3750W 750W
5
4 6
1
1 2 3 4
iDRAC
1 2750W 750W
1
1 2 3 4
iDRAC
1 2750W 750W
H-Rack-B10 Slave Nodes
R720XD2-Xeon E5-2670
128GB DDR324-2TB 7.2k
4-10Gb QFSP+640C/1280T
1.920PB
ManagementExpansion
2-R620 ASM-HA4020 Rack Frame2-Managed PDU
C13/L6-30A
Rack Aware TOR2-S4810 10GB
40Gbe HA Stack2-Z9000 10/40Gbe
HA CLOS Fabric Core 50-m QFSP+
HA Name Node Map Reduce
HD
FS
VLT Domain 1
VLT Domain 1CORE
A & B Managed Power28C13 / 2-L6-30A
9.1kW Peak7.2kW Typ
CB
1C
B2
CB
5C
B6
CB
3C
B4
CB
1C
B2
L1
CB
3C
B4
L2
CB
5C
B6
L3
ON
OFF
ON
OFF
ON
OFF
ON
OFF
ON
OFF
ON
OFF
2
2
2
2
2
2
2
2
2
2
ManagementVLAN
2
2
2
2
Quad 40Gbe Trunk Links
10G
be
SF
P+
2
PCT 3548
40GB LAG
10GB Link
10GB iSCSI
10GB Trunk
1GB iSCSI
16GB FC
1Gbe Mgmt.
Jay Etchings, Solutions Architect, Dell Enterprise
H-RACK-B Power Peak Pwr Typical BTU Peak BTU Typ Max Pot.
Totals 9185 7228 31302.85 24647.48 9128
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
FDX
/HDX
LNK
/ACT
DIAG
STACK ID
G1 G2 G4G3
LNK/ACT FDX/HDX
MASTER RPS PWR
FAN TEMP
RESET
SELECT
LNK
/ACT
LNK
/ACT
STACKING
PORT
1 2 3 4
5 6 7 8
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
FDX
/HDX
LNK
/ACT
DIAG
STACK ID
G1 G2 G4G3
LNK/ACT FDX/HDX
MASTER RPS PWR
FAN TEMP
RESET
SELECT
LNK
/ACT
LNK
/ACT
STACKING
PORT
1 2 3 4
5 6 7 8
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Hadoop H-Rack-CNGCC
HD
FS
Clu
stere
d N
od
e N
etw
ork
RS
-232
PSU0SYS ALM
LOC
IPMI ET
H
RESET PSU1Master
STACK ID USB-A
USB-B
SSD
60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3
124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67
Force10 Z9000
RS
-232
PSU0SYS ALM
LOC
IPMI ET
H
RESET PSU1Master
STACK ID USB-A
USB-B
SSD
60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3
124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67
Force10 Z9000
Sla
ve
No
de
sM
an
ag
em
en
tN
etw
ork
H-Rack-C10 Slave Nodes
R720XD2-Xeon E5-2670
128GB DDR324-2TB 7.2k
4-10Gb QFSP+640C/1280T
1.920PB
ManagementExpansion
4322DS KVM 32 Port4020 Rack Frame2-Managed PDU
C13/L6-30A
Rack Aware TOR2-S4810 10GB
40Gbe HA Stack
Ne
xt G
en
era
tion
Co
mp
ute
Clu
ste
r -Ha
do
op
/ Big
Da
ta
V.1
.5a
/ Pa
ge
6H
-Ra
ck-C
09
/06
/20
13
Ja
y E
tch
ing
s, S
olu
tion
s A
rch
itect, D
ell E
nte
rpris
e
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
AC OK
100-240V ~ 6A50 60Hz
100-240V ~ 6A50 60Hz
AC OK
PSU0
PSU1
AC OK
100-240V ~ 6A50 60Hz
100-240V ~ 6A50 60Hz
AC OK
PSU0
PSU1
VLT Domain 1
COREVLT Domain 2
A & B Managed Power28C13 / 2-L6-30A
6.2kW Peak4.8kW Typ
CB
1C
B2
CB
5C
B6
CB
3C
B4
CB
1C
B2
L1
CB
3C
B4
L2
CB
5C
B6
L3
ON
OFF
ON
OFF
ON
OFF
ON
OFF
ON
OFF
ON
OFF
2
2
2
2
2
2
2
2
2
2
232
31
To remove power completely, both AC line cords must be disconnected.
B
A
A
100-240V~, 1.25A , 50/60 Hz LAN 2 PDU 1 SETUP
MODEM
LAN 1 PDU 22
1
B
To remove power completely, both AC line cords must be disconnected.
100-240V~, 1.25A , 50/60 Hz
ManagementVLAN
10G
be
SF
P+
2
Quad 40Gbe Trunk Links
2 PCT 3548Management Switch
ACL VLAN
40GB LAG
10GB Link
10GB iSCSI
10GB Trunk
1GB iSCSI
16GB FC
1Gbe Mgmt.
Jay Etchings, Solutions Architect, Dell Enterprise H-RACK-C Power Peak Pwr Typical BTU Peak BTU Typ Max Pot.
Totals 6237 4875 21268.17 16623.75 7890
Hadoop H-Rack-ENGCC
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
FDX
/HDX
LNK
/ACT
DIAG
STACK ID
G1 G2 G4G3
LNK/ACT FDX/HDX
MASTER RPS PWR
FAN TEMP
RESET
SELECT
LNK
/ACT
LNK
/ACT
STACKING
PORT
1 2 3 4
5 6 7 8
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
FDX
/HDX
LNK
/ACT
DIAG
STACK ID
G1 G2 G4G3
LNK/ACT FDX/HDX
MASTER RPS PWR
FAN TEMP
RESET
SELECT
LNK
/ACT
LNK
/ACT
STACKING
PORT
1 2 3 4
5 6 7 8
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
HD
FS
Clu
stere
d N
od
e N
etw
ork
RS
-232
PSU0SYS ALM
LOC
IPMI ET
H
RESET PSU1Master
STACK ID USB-A
USB-B
SSD
60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3
124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67
Force10 Z9000
RS
-232
PSU0SYS ALM
LOC
IPMI ET
H
RESET PSU1Master
STACK ID USB-A
USB-B
SSD
60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3
124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67
Force10 Z9000
Sla
ve
No
de
sM
an
ag
em
en
tN
etw
ork
H-Rack-E10 Slave Nodes
R720XD2-Xeon E5-2670
128GB DDR324-2TB 7.2k
4-10Gb QFSP+640C/1280T
1.920PB
ManagementExpansion
4322DS KVM 32 Port2-PCT 3548
96 Mgmt. Ports4020 Rack Frame2-Managed PDU
C13/L6-30A
Rack Aware TOR2-S4810 10GB
40Gbe HA Stack
Ne
xt G
en
era
tion
Co
mp
ute
Clu
ste
r -Ha
do
op
/ Big
Da
ta
V.1
.5a
/ Pa
ge
7H
-Ra
ck-C
09
/06
/20
13
Ja
y E
tch
ing
s, S
olu
tion
s A
rch
itect, D
ell E
nte
rpris
e
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
AC OK
100-240V ~ 6A50 60Hz
100-240V ~ 6A50 60Hz
AC OK
PSU0
PSU1
AC OK
100-240V ~ 6A50 60Hz
100-240V ~ 6A50 60Hz
AC OK
PSU0
PSU1
VLT Domain 1
COREVLT Domain 2
A & B Managed Power28C13 / 2-L6-30A
6.2kW Peak4.9kW Typ
CB
1C
B2
CB
5C
B6
CB
3C
B4
CB
1C
B2
L1
CB
3C
B4
L2
CB
5C
B6
L3
ON
OFF
ON
OFF
ON
OFF
ON
OFF
ON
OFF
ON
OFF
2
2
2
2
2
2
2
2
2
2
232
31
To remove power completely, both AC line cords must be disconnected.
B
A
A
100-240V~, 1.25A , 50/60 Hz LAN 2 PDU 1 SETUP
MODEM
LAN 1 PDU 22
1
B
To remove power completely, both AC line cords must be disconnected.
100-240V~, 1.25A , 50/60 Hz
10G
be
SF
P+
2
Quad 40Gbe Trunk Links
PowerConnect 3548Management Switch
ACL VLAN
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
FDX
/HDX
LNK
/ACT
DIAG
STACK ID
G1 G2 G4G3
LNK/ACT FDX/HDX
MASTER RPS PWR
FAN TEMP
RESET
SELECT
LNK
/ACT
LNK
/ACT
STACKING
PORT
1 2 3 4
5 6 7 8
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
FDX
/HDX
LNK
/ACT
DIAG
STACK ID
G1 G2 G4G3
LNK/ACT FDX/HDX
MASTER RPS PWR
FAN TEMP
RESET
SELECT
LNK
/ACT
LNK
/ACT
STACKING
PORT
1 2 3 4
5 6 7 8
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
ManagementVLAN
40GB LAG
10GB Link
10GB iSCSI
10GB Trunk
1GB iSCSI
16GB FC
1Gbe Mgmt.
Jay Etchings, Solutions Architect, Dell Enterprise H-RACK-E Power Peak Pwr Typical BTU Peak BTU Typ Max Pot.
Totals 6277 4915 21404.57 16760.15 7890
VLT Layer 2 Network
Transactional Shared Storage
0202
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
0202
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
VLAN-000 0.0.0.0/0
VLAN-000 0.0.0.0/0HPCC
VLAN-531 / 10.128.42.0/23 VLAN-230 10.128.230.0/24
Hadoop Cluster
32
31
To remove power completely, both AC line cords must be disconnected.
B
A
A
100-240V~, 1.25A , 50/60 Hz LAN 2 PDU 1 SETUP
MODEM
LAN 1 PDU 22
1
B
To remove power completely, both AC line cords must be disconnected.
100-240V~, 1.25A , 50/60 Hz
E S T
PowerEdge R620 6
7
5
3
4
2
1
0
E S T
PowerEdge R620 6
7
5
3
4
2
1
0
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
FDX
/HDX
LNK
/ACT
DIAG
STACK ID
G1 G2 G4G3
LNK/ACT FDX/HDX
MASTER RPS PWR
FAN TEMP
RESET
SELECT
LNK
/ACT
LNK
/ACT
STACKING
PORT
1 2 3 4
5 6 7 8
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
FDX
/HDX
LNK
/ACT
DIAG
STACK ID
G1 G2 G4G3
LNK/ACT FDX/HDX
MASTER RPS PWR
FAN TEMP
RESET
SELECT
LNK
/ACT
LNK
/ACT
STACKING
PORT
1 2 3 4
5 6 7 8
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
QSFP+
MASTER
SYSFANPSU
RS-232
ETHERNET LNKACT 46 44 42 40 38 36 34 32 30 28 26 24 22 20 18 16 14 12 10 8 6 4 2 04856
5260
SFP+
Force10 S4810P
QSFP+
MASTER
SYSFANPSU
RS-232
ETHERNET LNKACT 46 44 42 40 38 36 34 32 30 28 26 24 22 20 18 16 14 12 10 8 6 4 2 04856
5260
SFP+
Force10 S4810P
Ne
xt G
en
era
tion
Co
mp
ute
Clu
ste
r -Ha
do
op
/ Big
Da
ta
V.1
.5a
/ Pa
ge
8H
ad
oo
p N
etw
ork
To
po
log
y / C
om
ple
te
09
/06
/20
13
Ja
y E
tch
ing
s, S
olu
tion
s A
rch
itect, D
ell E
nte
rpris
e
QSFP+
MASTER
SYSFANPSU
RS-232
ETHERNET LNKACT 46 44 42 40 38 36 34 32 30 28 26 24 22 20 18 16 14 12 10 8 6 4 2 04856
5260
SFP+
Force10 S4810P
QSFP+
MASTER
SYSFANPSU
RS-232
ETHERNET LNKACT 46 44 42 40 38 36 34 32 30 28 26 24 22 20 18 16 14 12 10 8 6 4 2 04856
5260
SFP+
Force10 S4810P
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
RS
-232
PSU0SYS ALM
LOC
IPMI ET
H
RESET PSU1Master
STACK ID USB-A
USB-B
SSD
60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3
124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67
Force10 Z9000
RS
-232
PSU0SYS ALM
LOC
IPMI ET
H
RESET PSU1Master
STACK ID USB-A
USB-B
SSD
60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3
124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67
Force10 Z9000
VLT 4VLT 2
ASM
PowerEdge M1000e
9 101 2
11 123 4
13 145 6
15 167 8
0
1
00
1
0 0
1
00
1
0 0
1
00
1
0 0
1
00
1
0
0
1
00
1
0 0
1
00
1
0 0
1
00
1
0 0
1
00
1
0
EST
EST
PowerEdge M1000e
9 101 2
11 123 4
13 145 6
15 167 8
0
1
00
1
0 0
1
00
1
0 0
1
00
1
0 0
1
00
1
0
0
1
00
1
0 0
1
00
1
0 0
1
00
1
0 0
1
00
1
0
EST
EST
RACK ERACK CRACK BRACK A
40GB LAG
10GB Link
10GB iSCSI
10GB Trunk
1GB iSCSI
16GB FC
1Gbe Mgmt.
40G 40G 40G 40G
Management
QSFP+
MAST
ER
SYS
FAN
PSU
RS-232
ETHERNET
LNK
ACT
46
44
42
40
38
36
34
32
30
28
26
24
22
20
18
16
14
12
10
8
6
4
2
0
48
56
52
60
SFP+
Force10 S4810P
VLT 5
0202
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
0202
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
40G
VLAN-000 0.0.0.0/0
PowerEdge M1000e
9 101 2
11 123 4
13 145 6
15 167 8
0
1
00
1
0 0
1
00
1
0 0
1
00
1
0 0
1
00
1
0
0
1
00
1
0 0
1
00
1
0 0
1
00
1
0 0
1
00
1
0
40G40G
Public Cloud Space(INTERNET 2 READY)
Router
IO DataCenterPrimary Storage
CONSOLE
33-3637-40
LNKACT
LNKACT
Force10 MXL 10/40GbE
41-48
49-56
CONSOLE
33-3637-40
LNKACT
LNKACT
Force10 MXL 10/40GbE
41-48
49-56
VLT 6
Ja
y E
tch
ing
s, S
olu
tio
ns A
rch
ite
ct, D
ell
En
terp
rise
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
VLT 1 VLT 3
VLT 5
High Performance
High Performance systems are necessary to address computationally intensive tasks. These can be broken into two subclasses tasks which large number of computations and those that require large memory stores. Each requires a different physical infrastructure. The first configuration supports embarrassingly parallel tasks that are either naturally parallel or have specialized code to utilize parallel computing. This configuration needs a large collection of compute node/cores connected via high speed internal connections with fast access storage space to read/write information and to store intermediate information.
The standing infrastructure to support this need would be 100 node/ 800 core cluster with 32GB RAM/node connected via Infinaband QDR interconnects. The cluster would have 250TB of fast scratch storage space in a Lustre file system. High speed network connection to the transactional infrastructure is required.
The second High Performance configuration requires large shared memory. This environment processes jobs with high complexity/interdependencies requiring processing of a common data space, or code that isnt or cant be straightforwardly parallelized.
The infrastructure to support this component is composed of a 32 core (4 x 8 core) symmetric multiprocessor system with 2TB RAM, and 2 x 10GE with iSCSI offload. High speed network connection to the transactional infrastructure is required.
Reserved Space
HSS Lustre Expansion
10U
Reserved Space for NSS NFS
Appliance Expansion 6U
100V-240VAC, 1.6A
LOCATOR
RPS
DC IN 12V,15A
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
100V-240VAC, 1.5A
LOCATOR
RPS
DC IN 12V,15A
AC OK
100-240V ~ 6A50 60Hz
100-240V ~ 6A50 60Hz
AC OK
PSU0
PSU1
LOCATOR
100V-240VAC, 8A
MPS
DC IN 48V,12.5A
8
9
10
4
5
6
12
13
14
(1) R910 SMP 2TB RAM
PowerEdge M1000e
9 101 2
11 123 4
13 145 6
15 167 8
0
1
00
1
0 0
1
00
1
0 0
1
00
1
0 0
1
00
1
0
0
1
00
1
0 0
1
00
1
0 0
1
00
1
0 0
1
00
1
0
PowerEdge M1000e
9 101 2
11 123 4
13 145 6
15 167 8
0
1
00
1
0 0
1
00
1
0 0
1
00
1
0 0
1
00
1
0
0
1
00
1
0 0
1
00
1
0 0
1
00
1
0 0
1
00
1
0
100V-240VAC, 1.6A
LOCATOR
RPS
DC IN 12V,15A
100V-240VAC, 1.6A
LOCATOR
RPS
DC IN 12V,15A
0
21
3
54
6
87
9
1110
Front
0
21
3
54
6
87
9
1110
Front
0
21
3
54
6
87
9
1110
Front
0
21
3
54
6
87
9
1110
Front
0
21
3
54
6
87
9
1110
Front
0 1 2 3 4 5 6 7 8 9 10 11
0 1 2 3 4 5 6 7 8 9 10 11
0 1 2 3 4 5 6 7 8 9 10 11
0 1 2 3 4 5 6 7 8 9 10 11
0 1 2 3 4 5 6 7 8 9 10 11
0
21
3
54
6
87
9
1110
Front
0
21
3
54
6
87
9
1110
Front
0
21
3
54
6
87
9
1110
Front
0
21
3
54
6
87
9
1110
Front
0
21
3
54
6
87
9
1110
Front
0 1 2 3 4 5 6 7 8 9 10 11
0 1 2 3 4 5 6 7 8 9 10 11
0 1 2 3 4 5 6 7 8 9 10 11
0 1 2 3 4 5 6 7 8 9 10 11
0 1 2 3 4 5 6 7 8 9 10 11
0
21
3
54
6
87
9
1110
Front
0
21
3
54
6
87
9
1110
Front
0
21
3
54
6
87
9
1110
Front
0
21
3
54
6
87
9
1110
Front
0
21
3
54
6
87
9
1110
Front
0 1 2 3 4 5 6 7 8 9 10 11
0 1 2 3 4 5 6 7 8 9 10 11
0 1 2 3 4 5 6 7 8 9 10 11
0 1 2 3 4 5 6 7 8 9 10 11
0 1 2 3 4 5 6 7 8 9 10 11
LOCATOR
100V-240VAC, 8A
MPS
DC IN 48V,12.5A
PowerEdge M1000e
9 101 2
11 123 4
13 145 6
15 167 8
0
1
00
1
0 0
1
00
1
0 0
1
00
1
0 0
1
00
1
0
0
1
00
1
0 0
1
00
1
0 0
1
00
1
0 0
1
00
1
0
PowerEdge M1000e
9 101 2
11 123 4
13 145 6
15 167 8
0
1
00
1
0 0
1
00
1
0 0
1
00
1
0 0
1
00
1
0
0
1
00
1
0 0
1
00
1
0 0
1
00
1
0 0
1
00
1
0
Ne
xt G
en
era
tion
Co
mp
ute
Clu
ste
r -HP
CC
V.1
.5a
/ Pa
ge
10
HP
CC
Ra
ck D
esig
n0
9/0
6/2
01
3T
om
Ra
iso
r, HP
C S
olu
tion
s A
rch
itect, D
ell R
ese
arc
h C
om
pu
ting
(2) R620 + MD3260 NSS 4.5 (1) R210 TERASCALA HSS Mgmt. Node(1) PCT 5548 HSS4.5 Mgmt. Switch(1) R320 CIFS Gateway(3) R320 ISB Storage Bridge ServersIntelligent Storage Bridge(1) R620 TERASCALA MetaData Servers MD3220 StorageOSS Pair / MD32XX (HSS Expansion)(1) R620 TERASCALA OSS MD3260 Storage(2) R720 GPU Node w/ 2x NVIDIA K20(2) R620 Head Nodes(1) R620 Login Node/ KMM(4) M1000e(16) M620 Compute Nodes(256 cores/Chassis)(2) Force10 MXL Switch(1) M4001T IB(2) CMC
AC OK
100-240V ~ 6A50 60Hz
100-240V ~ 6A50 60Hz
AC OK
PSU0
PSU1
Reserved Space
NSS NFS Appliance 6U
Expansion
Reserved Space
HSS Lustre Expansion
10U
Ne
xt G
en
era
tion
Co
mp
ute
Clu
ste
r -HP
CC
V.1
.5a
/ Pa
ge
10
HP
CC
Ra
ck D
esig
n0
9/0
6/2
01
3T
om
Ra
iso
r, HP
C S
olu
tion
s A
rch
itect, D
ell R
ese
arc
h C
om
pu
ting
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
B2C2 A2B1 C1A1
4
1
7
5
2
8
6
3
9
CMC2CMC1 KVM
1 2 3 4 5 6
GbGb 21
CMC
iKVM
GbGb 21
CMC
B1 C1A1 B2C2 A2
B2C2 A2B1 C1A1
4
1
7
5
2
8
6
3
9
CMC2CMC1 KVM
1 2 3 4 5 6
GbGb 21
CMC
iKVM
GbGb 21
CMC
B1 C1A1 B2C2 A2
iDRAC
iDRAC
iDRAC
CONSOLE
33-3637-40
LNKACT
LNKACT
Force10 MXL 10/40GbE
41-48
49-56
CONSOLE
33-3637-40
LNKACT
LNKACT
Force10 MXL 10/40GbE
41-48
49-56
LNK ACT
Stacking HDMI1 2 SFP+1 2
LNK ACT LNK ACT
Reset
LNK ACT
Stack No.
M RPSFan
PW
R
Status
26 28 30 32 34 36 38 40 42 44 46 48
25 27 29 31 33 35 37 39 41 43 45 47
2 4 6 8 10 12 14 16 18 20 22 24
1 3 5 7 9 11 13 15 17 19 21 23
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
6 5 4 3 2 1
Gb 1
Gb 2
Gb 3
Gb 4
10Gb
10Gb
6 5 4 3 2 1
7
4
3
2
1
B2C2 A2B1 C1A1
4
1
7
5
2
8
6
3
9
CMC2CMC1 KVM
1 2 3 4 5 6
GbGb 21
CMC
iKVM
GbGb 21
CMC
B1 C1A1 B2C2 A2
CONSOLE
33-3637-40
LNKACT
LNKACT
Force10 MXL 10/40GbE
41-48
49-56
B2C2 A2B1 C1A1
4
1
7
5
2
8
6
3
9
CMC2CMC1 KVM
1 2 3 4 5 6
GbGb 21
CMC
iKVM
GbGb 21
CMC
B1 C1A1 B2C2 A2
CONSOLE
33-3637-40
LNKACT
LNKACT
Force10 MXL 10/40GbE
41-48
49-56
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
STATUS
PSU 1
PSU 2
FAN
RST
3433
3231
3635
2827
2625
3029
2221
2019
2423
1615
1413
1817
109
87
1211
43
21
65
IS 5025CONSOLE
STATUS
PSU 1
PSU 2
FAN
RST
3433
3231
3635
2827
2625
3029
2221
2019
2423
1615
1413
1817
109
87
1211
43
21
65
IS 5025CONSOLE
STATUS
PSU 1
PSU 2
FAN
RST
3433
3231
3635
2827
2625
3029
2221
2019
2423
1615
1413
1817
109
87
1211
43
21
65
IS 5025CONSOLE
LNK ACT
Stacking HDMI1 2 SFP+1 2
LNK ACT LNK ACT
Reset
LNK ACT
Stack No.
M RPSFan
PW
R
Status
2 4 6 8 10 12 14 16 18 20 22 24
1 3 5 7 9 11 13 15 17 19 21 23
LNK ACT
Stacking HDMI1 2 SFP+1 2
LNK ACT LNK ACT
Reset
LNK ACT
Stack No.
M RPSFan
PW
R
Status
26 28 30 32 34 36 38 40 42 44 46 48
25 27 29 31 33 35 37 39 41 43 45 47
2 4 6 8 10 12 14 16 18 20 22 24
1 3 5 7 9 11 13 15 17 19 21 23
2
1
2
1
DC AC
DC AC
PSU A
PSU B
SBB A
SBB B
SBB B
SBB A
PSU B
PSU A PSU A
PSU B
SBB A
SBB B
SBB B
SBB A
PSU B
PSU A
SAS IN 32
SAS IN 10 SAS OUT
SAS IN 32
SAS IN 10 SAS OUT
2
1
2
1
DC AC
DC AC
PSU A
PSU B
SBB A
SBB B
SBB B
SBB A
PSU B
PSU A PSU A
PSU B
SBB A
SBB B
SBB B
SBB A
PSU B
PSU A
SAS IN 32
SAS IN 10 SAS OUT
SAS IN 32
SAS IN 10 SAS OUT
iDRAC
iDRAC
iDRAC
iDRAC
iDRAC
Gb 2Gb 1eSATA
MEST
iDRAC
iDRAC
2
1
2
1
DC AC
DC AC
PSU A
PSU B
SBB A
SBB B
SBB B
SBB A
PSU B
PSU A PSU A
PSU B
SBB A
SBB B
SBB B
SBB A
PSU B
PSU A
SAS IN 32
SAS IN 10 SAS OUT
SAS IN 32
SAS IN 10 SAS OUT
MD32 Series
PPID
SAS OUT
SAS IN0 1
SAS IN2 3
MD32 Series
PPID
SAS OUT
SAS IN0 1
SAS IN2 3
MGMT
MGMT
7.1kW Peak5.7kW Typ
15.1kW Peak12.1kW Typ
13.2kW Peak10.5kW Typ
(2) Force10 S4810 10Gbe Core(3) Mellanox FDR Switches(2) Spine (1) Storage(1) PCT 5548 1Gbe/IPMI/Mgmt.(8) Managed PDU L6-30
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
iDRAC
iDRAC
iDRAC
Public Cloud Space
HPC Shared Memory
HPCC Topology
(1) PCT 5548
Gbe / IPMI / Mgt
18 initial, 30 for expansion
APC Mgd PDU (2)
NSS 4.5 (2) R620 + MD3260
(1) Login Node - R620
(2) Head Nodes R620
HPC Parallel
FDR IB Spine
(2) Mellanox 36 port
40 ports initially
populated,
32 left for expansion
(4 more blade chassis)
(1) FDR IB Leaf Switch
Storage and Admin
18 ports initially populated
18 free for expansion
Ne
xt G
en
era
tion
Co
mp
ute
Clu
ste
r -HP
CC
V.1
.5a
/ Pa
ge
11
HP
CC
/ NG
CC
To
po
log
y0
9/0
6/2
01
3T
om
Ra
iso
r, HP
C S
olu
tion
s A
rch
itect, D
ell R
ese
arc
h C
om
pu
ting LNK ACT
Stacking HDMI1 2 SFP+1 2
LNK ACT LNK ACT
Reset
LNK ACT
Stack No.
M RPSFan
PW
R
Status
2 4 6 8 10 12 14 16 18 20 22 24
1 3 5 7 9 11 13 15 17 19 21 23
4
LNK ACT
Stacking HDMI1 2 SFP+1 2
LNK ACT LNK ACT
Reset
LNK ACT
Stack No.
M RPSFan
PW
R
Status
26 28 30 32 34 36 38 40 42 44 46 48
25 27 29 31 33 35 37 39 41 43 45 47
2 4 6 8 10 12 14 16 18 20 22 24
1 3 5 7 9 11 13 15 17 19 21 23
B2C2 A2B1 C1A1
4
1
7
5
2
8
6
3
9
CMC2CMC1 KVM
1 2 3 4 5 6
GbGb 21
CMC
iKVM
GbGb 21
CMC
B1 C1A1 B2C2 A2
2
1
2
1
DC AC
DC AC
PSU A
PSU B
SBB A
SBB B
SBB B
SBB A
PSU B
PSU A PSU A
PSU B
SBB A
SBB B
SBB B
SBB A
PSU B
PSU A
SAS IN 32
SAS IN 10 SAS OUT
SAS IN 32
SAS IN 10 SAS OUT
2
1
2
1
DC AC
DC AC
PSU A
PSU B
SBB A
SBB B
SBB B
SBB A
PSU B
PSU A PSU A
PSU B
SBB A
SBB B
SBB B
SBB A
PSU B
PSU A
1 2
LnkLnkLnkLnk
ID/Diag
Lnk Lnk
1 2
LnkLnkLnkLnk
ID/Diag
Lnk Lnk
iDRAC
iDRAC
iDRAC
iDRAC
2
1
2
1
DC AC
DC AC
PSU A
PSU B
SBB A
SBB B
SBB B
SBB A
PSU B
PSU A PSU A
PSU B
SBB A
SBB B
SBB B
SBB A
PSU B
PSU A
SAS IN 32
SAS IN 10 SAS OUT
SAS IN 32
SAS IN 10 SAS OUT
1
1 2 3 4
iDRAC
1 2750W 750W
1
1 2 3 4
iDRAC
1 2750W 750W
1
1 2 3 4
iDRAC
1 2750W 750W
1
1 2 3 4
iDRAC
1 2750W 750W
1
1 2 3 4
iDRAC
1 2750W 750W
1
1 2 3 4
iDRAC
1 2750W 750W
MD32 Series
PPID
SAS OUT
SAS IN0 1
SAS IN2 3
MD32 Series
PPID
SAS OUT
SAS IN0 1
SAS IN2 3
MGMT
MGMT
LNK ACT
Stacking HDMI1 2 SFP+1 2
LNK ACT LNK ACT
Reset
LNK ACT
Stack No.
M RPSFan
PW
R
Status
26 28 30 32 34 36 38 40 42 44 46 48
25 27 29 31 33 35 37 39 41 43 45 47
2 4 6 8 10 12 14 16 18 20 22 24
1 3 5 7 9 11 13 15 17 19 21 23
STATUS
PSU 1
PSU 2
FAN
RST
3433
3231
3635
2827
2625
3029
2221
2019
2423
1615
1413
1817
109
87
1211
43
21
65
IS 5025CONSOLE
8
3 2
2
1
2
44
4
4
2
2
2
2
1
4
2
2
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
1
Gb 2Gb 1eSATA
MEST
2
CONSOLE
33-3637-40
LNKACT
LNKACT
Force10 MXL 10/40GbE
41-48
49-56
16
2
6 5 4 3 2 1
Gb 1
Gb 2
Gb 3
Gb 4
10Gb
10Gb
6 5 4 3 2 1
7
4
3
2
1
2
1 2 3 4
ST
21
iDRAC
2
1
3750W750W
5
4
7
6
2
1
2
STATUS
PSU 1
PSU 2
FAN
RST
3433
3231
3635
2827
2625
3029
2221
2019
2423
1615
1413
1817
109
87
1211
43
21
65
IS 5025CONSOLE
STATUS
PSU 1
PSU 2
FAN
RST
3433
3231
3635
2827
2625
3029
2221
2019
2423
1615
1413
1817
109
87
1211
43
21
65
IS 5025CONSOLE
8
1
1
(4) M1000E each with:
(16) m620 Compute Nodes
(1) Force10 MXL Ethernet Switches
(1) M4001F IB (8 uplinks per chassis)
(2) CMC
(2) R620
Terascala OSS and Storage
(4) R320
Terascala Appliance
CIFS Gateway/Intelligent Storage Bridge
Terascala HSS Mgt Node
(1) R210
(1) PCT 5524
NSS Storage Mgmt.
Switch / HA
(1) PCT 5548
HSS Storage Mgmt.
Switch / HA
(2) GPU Compute Nodes R720each w/2x NVIDIA K20
(2) R620
Terascala MetaData Servers + MD3220
(1) SMP 2TB RAM R910
(1) Force10 4810 10Gb Core
26 initial, 22 for expansion
VLT Layer 2 Network
iDRAC
iDRAC
iDRAC
iDRAC
40G
SW SM Option I2
Transactional
Transactional systems are the central component of the NGCC. They represent the general day-to-day utility platforms and are used for tasks such as routine computing, hosting services (including web servers, database servers, applications), hosting data, and supporting middleware.
The standing physical infrastructure to support utility computing would be comprised of a server cluster composed of 20 commodity duel quad core servers, each with 32 GB of memory. The Transactional systems would have 1 petabyte of RAID configured 10GE iSCSI Network Storage to support large data resources (replicated to 2 PB). Network gear would include VPN concentrator, gateway, switches, routers, and NICs. This component requires access to high speed internet (Internet 2).
Internet 2 Public Cloud VLT Layer 2 Network
SAS/SATA
PCIeSSD
SAS/SATA
PCIeSSD
SAS/SATA
PCIeSSD
SAS/SATA
PCIeSSD
0202
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
PowerEdge M1000e
9 101 2
11 123 4
13 145 6
15 167 8
M1000e Chassis20 M420 Blade Servers
Intel Xeon Processor E5-2430
(15M Cache, 2.20 GHz, 7.20 GT/s Intel QPI)
(40S, 240C, 640Thread)
32GB DDR3 Memory
2*200GB SSD RAID 0
2 16GB FC Controllers for SAN
Brocade 6505 FC Fabric 16G
2 Force 10 MXL 10/40Gbe
Dual Flex IO Modules (iSCSI)
64 Internal 10 Ports
Bright Cluster Manager
CMC/OM/iDRAC
ASM Integration
Dual SonicWall SM10K (Option I2)
(Unassigned Rack Location)
Transactional ClusterNGCC RS-23
2
PSU0SYS ALM
LOC
IPMI ET
H
RESET PSU1Master
STACK ID USB-A
USB-B
SSD
60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3
124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67
Force10 Z9000
RS
-232
PSU0SYS ALM
LOC
IPMI ET
H
RESET PSU1Master
STACK ID USB-A
USB-B
SSD
60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3
124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67
Force10 Z9000
Ne
xt G
en
era
tion
Co
mp
ute
Clu
ste
r
V.1
.5a
/ Pa
ge
13
NG
CC
Co
mp
uta
tion
al B
righ
t Co
mp
utin
g C
luste
r0
9/0
6/2
01
3Ja
y E
tch
ing
s, S
olu
tion
s A
rch
itect, D
ell E
nte
rpris
e
B2C2 A2B1 C1A1
4
1
7
5
2
8
6
3
9
CMC2CMC1 KVM
1 2 3 4 5 6
GbGb 21
CMC
iKVM
GbGb 21
CMC
B1 C1A1 B2C2 A2
CONSOLE
33-3637-40
LNKACT
LNKACT
Force10 MXL 10/40GbE
41-48
49-56
CONSOLE
33-3637-40
LNKACT
LNKACT
Force10 MXL 10/40GbE
41-48
49-56
0
23
22
21
20
19
18
17
8Gbps FC
Brocade M5424
0
23
22
21
20
19
18
17
8Gbps FC
Brocade M5424
Ma
na
ge
me
nt
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
FDX
/HDX
LNK
/ACT
DIAG
STACK ID
G1 G2 G4G3
LNK/ACT FDX/HDX
MASTER RPS PWR
FAN TEMP
RESET
SELECT
LNK
/ACT
LNK
/ACT
STACKING
PORT
1 2 3 4
5 6 7 8
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
FDX
/HDX
LNK
/ACT
DIAG
STACK ID
G1 G2 G4G3
LNK/ACT FDX/HDX
MASTER RPS PWR
FAN TEMP
RESET
SELECT
LNK
/ACT
LNK
/ACT
STACKING
PORT
1 2 3 4
5 6 7 8
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
0202
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
0202
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
QSFP+
MASTER
SYS FAN PSU
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
PO
RT
S
PCIe x8
8Gb FC
8 4 2
8 4 2
8 4 2
8 4 2
12
34
PO
RT
S
PCIe x8
8Gb FC
8 4 2
8 4 2
8 4 2
8 4 2
12
34
40GB LAG
10GB Link
10GB iSCSI
10GB Trunk
1GB iSCSI
16GB FC
1Gbe Mgmt.
Storage Zone
Management Zone
Fabric B
4-16GB F
C
Fabric A4-40GBe
Jay Etchings, Solutions Architect, Dell Enterprise
E S T
PowerEdge R620 6
7
5
3
4
2
1
0
E S T
PowerEdge R620 6
7
5
3
4
2
1
0
ASM
Replication Zone
01
23
45
67
89
1011
1213
1415
01
23
54
GE
FC
GE
1S
TA
T0
ST
AT
Bro
ca
de
78
00
01
23
45
67
89
1011
1213
1415
01
23
54
GE
FC
GE
1S
TA
T0
ST
AT
Bro
ca
de
78
00
FCIP Bridge
STORAGE FABRIC B
Tra
nsa
ctio
na
l
STORAGE FABRIC A
Du
al 6
GB
SA
S C
on
ne
cto
rs
Ne
xt G
en
era
tion
Co
mp
ute
Clu
ste
r
V.1
.5a
/ Pa
ge
14
Co
mp
elle
nt H
igh
Ava
ilab
ility S
tora
ge
with
Re
plic
atio
n0
9/0
6/2
01
3Ja
y E
tch
ing
s, S
olu
tion
s A
rch
itect, D
ell E
nte
rpris
e
0 1 2 34 5 6 7 8 9 10 11 12 13 14 15 0 1 2 3 54GEFC
GE
1ST AT
0ST ATBrocade 7800
0 1 2 34 5 6 7 8 9 10 11 12 13 14 15 0 1 2 3 54GEFC
GE
1ST AT
0ST ATBrocade 7800
Brocade 78F FCIP Bridge Brocade 78F FCIP Bridge
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22