Upload
others
View
9
Download
0
Embed Size (px)
Citation preview
HPE Apollo 8000 System User Guide
Abstract This guide provides user information for the HPE Apollo 8000 System.
Part Number: 747365-003R November 2015 Edition: 4
© Copyright 2015 Hewlett Packard Enterprise Development LP
The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.
Contents 3
Contents
Overview................................................................................................................................................ 6 Product overview ...................................................................................................................................................... 6 Example configurations ............................................................................................................................................ 6
Component identification ..................................................................................................................... 11 f8000 rack components .......................................................................................................................................... 11 iCDU components .................................................................................................................................................. 13
iCDU display components ........................................................................................................................... 14
Installation and configuration ............................................................................................................... 15 Installation included with service ............................................................................................................................ 15 Installation expectations ......................................................................................................................................... 15
Power components .............................................................................................................................. 17 About the power shelf ............................................................................................................................................. 17
Power shelf front components ..................................................................................................................... 18 Power shelf rear components ...................................................................................................................... 19 Installing the power shelf ............................................................................................................................. 21 Configuring the power shelf ......................................................................................................................... 22
Redundant power supplies ..................................................................................................................................... 23 Redundant power supply LED indicator ...................................................................................................... 24 Installing the redundant power supply ......................................................................................................... 24
Monitoring power usage ......................................................................................................................................... 24
Compute cell ........................................................................................................................................ 26 About the compute cell ........................................................................................................................................... 26
Compute cell components ........................................................................................................................... 27 Compute cell numbering.............................................................................................................................. 28 Management module connectors and LED indicators ................................................................................. 29 Fan numbering ............................................................................................................................................ 30
Installing the management module ........................................................................................................................ 35
Utility module ....................................................................................................................................... 37 About the utility module .......................................................................................................................................... 37
Utility module products ................................................................................................................................ 38 HPE Apollo 8000 UM1 ........................................................................................................................................... 39
Utility module front components .................................................................................................................. 39 Utility module rear components ................................................................................................................... 41
HPE Apollo 8000 UM2 ........................................................................................................................................... 42 Utility module front components .................................................................................................................. 43 Utility module rear components ................................................................................................................... 45
Installing the utility module ..................................................................................................................................... 46 Configuring the utility module ................................................................................................................................. 47
Thermal components ........................................................................................................................... 48 About the thermal components .............................................................................................................................. 48 HEX ........................................................................................................................................................................ 49
HEX components and connectors ............................................................................................................... 49 Thermal controller .................................................................................................................................................. 50
Thermal controller components and connectors ......................................................................................... 50 Updating the thermal controller firmware..................................................................................................... 50
Water walls ............................................................................................................................................................. 51 Water wall components and connectors...................................................................................................... 51
Rack door sensors and locks ................................................................................................................................. 52 Activating and deactivating door locks ........................................................................................................ 52
Contents 4
Infiniband switch .................................................................................................................................. 53 About the infiniband switch ..................................................................................................................................... 53
Infiniband switch front components and connectors.................................................................................... 54 Infiniband switch port numbering ................................................................................................................. 54 Infiniband switch LED indicators.................................................................................................................. 55 Infiniband switch rear components and connectors .................................................................................... 56
Installing the infiniband switch ................................................................................................................................ 56
Safety components .............................................................................................................................. 57 About the safety components ................................................................................................................................. 57 Leak detection sensors .......................................................................................................................................... 57 Temperature sensors and automatic door release mechanism ............................................................................. 57 (Optional) ARPO .................................................................................................................................................... 58
Cabling ................................................................................................................................................ 59 Cabling overview .................................................................................................................................................... 59 Cabling when the HPE Apollo 8000 UM1 is installed ............................................................................................. 59
Connecting the utility module to the power shelves .................................................................................... 59 Connecting the utility module to the management modules ........................................................................ 60
Cabling when the HPE Apollo 8000 UM2 is installed ............................................................................................. 61 Connecting power........................................................................................................................................ 61 Connecting communication ......................................................................................................................... 62 Connecting the f8000 rack to the iCDU rack ............................................................................................... 65
Connect the CMU solution hardware ..................................................................................................................... 67
iCDU .................................................................................................................................................... 69 About the iCDU ...................................................................................................................................................... 69 iCDU setup ............................................................................................................................................................. 69
Powering on the iCDU ................................................................................................................................. 69 Powering off the iCDU ................................................................................................................................. 74 BMS network communication ...................................................................................................................... 74
iCDU display ........................................................................................................................................................... 77 Password security level ............................................................................................................................... 77 Screen time-out ........................................................................................................................................... 77 Always available .......................................................................................................................................... 77 Always visible .............................................................................................................................................. 80
iCDU control ........................................................................................................................................................... 81 Control switch .............................................................................................................................................. 81 iCDU isolation valves................................................................................................................................... 82 Vacuum pressure ........................................................................................................................................ 83 Differential pressure .................................................................................................................................... 83 Pump and IT supply out water temperature ................................................................................................ 83
iCDU maintenance ................................................................................................................................................. 84 Water testing and maintenance schedule ................................................................................................... 84
Condensation management ................................................................................................................. 85 Condensation management ................................................................................................................................... 85 Frost damage ......................................................................................................................................................... 85
Software and configuration utilities ...................................................................................................... 86 HPE Insight Cluster Management Utility ................................................................................................................ 86 HPE Advanced Power Manager ............................................................................................................................. 86
Troubleshooting ................................................................................................................................... 87 HPE Apollo 8000 System Troubleshooting ............................................................................................................ 87
Rack and mechanical issues ....................................................................................................................... 87 Power issues ............................................................................................................................................... 88 Temperature and cooling issues ................................................................................................................. 90 Communication issues ................................................................................................................................ 91 iCDU issues ................................................................................................................................................. 93
iCDU alarm troubleshooting ................................................................................................................................... 93 Using HPE APM for sensor monitoring .................................................................................................................. 95
Contents 5
Specifications ...................................................................................................................................... 98 Physical specifications ........................................................................................................................................... 98 Electrical specifications .......................................................................................................................................... 98 Thermal and air flow performance ........................................................................................................................ 100 Environmental specifications ................................................................................................................................ 100
Safety considerations......................................................................................................................... 101 Important safety information ................................................................................................................................. 101 Symbols on equipment ......................................................................................................................................... 101 Warnings and cautions ......................................................................................................................................... 101
Support and other resources.............................................................................................................. 103 Accessing Hewlett Packard Enterprise Support ................................................................................................... 103
Information to collect ................................................................................................................................. 103 Accessing updates ............................................................................................................................................... 103 Websites ............................................................................................................................................................... 103 Customer Self Repair ........................................................................................................................................... 104 Remote support .................................................................................................................................................... 112
Warranty and regulatory information .................................................................................................. 113 Warranty information ............................................................................................................................................ 113 Regulatory information ......................................................................................................................................... 113
Safety and regulatory compliance ............................................................................................................. 113 Belarus Kazakhstan Russia marking ......................................................................................................... 113 Turkey RoHS material content declaration ................................................................................................ 114 Ukraine RoHS material content declaration .............................................................................................. 114
Acronyms and abbreviations .............................................................................................................. 115
Documentation feedback ................................................................................................................... 116
Index .................................................................................................................................................. 117
Overview 6
Overview
Product overview The HPE Apollo 8000 System is a high-density, energy-efficient, sustainable high-performance computing solution that uses an innovative warm-liquid cooling technology to fuel the future of supercomputing. It offers hundreds of teraflops per rack to accelerate results, with a modular, rack-based system that is easy to install, maintain, and monitor. The hot water "waste heat" can be recycled to heat the data center efficiently.
The Apollo 8000 System comprises the following components:
• One or more HPE 8000 iCDU Racks and their associated onboard iCDU control system
• One or more f8000 racks and their associated control system
• Primary plumbing interface assembly kit that connects the individual iCDUs to the facility water system
• Secondary plumbing system that connects the individual iCDUs to the f8000 racks. This secondary closed loop system provides cooling for the f8000 rack and isolates the f8000 rack from untreated, potentially incompatible primary water.
• Drain and leak detection system
The iCDU performs the following functions:
• Accepts water supplied from the facility water system
• Returns heated water back to the facility water system
• Supplies cooled water to the f8000 racks through the secondary plumbing system
• Accepts heated water returned from the f8000 racks
• Maintains the preset water temperature and pressure in the secondary plumbing system
• Monitors system health
• Isolates primary and secondary water in case of a leak
Example configurations The Apollo 8000 System is a self-contained IT product integrated with a closed-loop cooling system that offers two configuration options.
The following figures demonstrate examples of the f8000 rack (1) to iCDU (2) ratio. The doors are removed for clarity.
CAUTION: To reduce the risk of condensation and avoid damage to the equipment when operating with chilled water, keep the rack doors closed during operation.
Example single iCDU configuration (with up to four f8000 racks)
Overview 7
• Front view
Item Description 1 HPE Apollo f8000 Racks 2 HPE Apollo iCDU Racks
Overview 8
• Top view
Item Description 1 Facility water in 2 Facility water out 3 Secondary water out from f8000 rack 4 Secondary water in to f8000 rack 5 Heat exchanger
Example redundant iCDU configuration (with up to four f8000 racks)
Overview 9
• Front view
Item Description 1 Apollo f8000 Racks 2 Apollo iCDU Racks
Overview 10
• Top view
Item Description 1 Facility water in 2 Facility water out 3 Secondary water out of f8000 rack 4 Secondary water in to f8000 rack 5 Heat exchanger
The iCDU transfers heat to the facility water plant and has a secondary loop that provides cooling to the servers in the IT racks.
Component identification 11
Component identification
f8000 rack components The Apollo 8000 System is composed of several key components, described below, which work together to provide a modular, rack-based, cooling solution.
f8000 rack front components
Item Component Description 1 Servers Provides services for other computers and programs
within the network 2 Fan units Circulate the cool air from the HEX through the f8000 rack 3 Power shelves The top power shelf provides power to the components in
the top half of the f8000 rack, and the bottom power shelf provides power to the components in the bottom half of the f8000 rack.
4 Utility module Gathers and monitors data from the f8000 rack components to display in HPE APM
Component identification 12
Item Component Description 5 Door sensor Monitors the front door to detect if the door is open or
closed
The utility module might appear different based on the model installed. For more information on the different models available, see "Utility module products (on page 38)."
f8000 rack rear components
Item Component Description 1 Management module assembly Control center for 10U rack compute cell 2 Fan units Circulate the cool air from the HEX through the f8000 rack 3 Cable tray Routes cables from the front to the rear of the f8000 rack 4 Rack drain pan Collects any condensation from the HEX 5 Rack door magnet Gathers and adjusts temperature requirements based on
pre-configured setpoints 6 RTD sensors Measures water temperature going in and coming out of
the HEX
Component identification 13
iCDU components
Item Description 1 iCDU display 2 USB connector for firmware updates 3 Ethernet connector for network connection 4 DB9 connector for pLAN connection 5 Hand/Off/Auto switch 6 VFD control button 7 Power On/Off switch 8 Facility water actuating valve 9 Vacuum pump 10 Water level site glass 11 Water return from upper iCDU rack 12 Water return from f8000 rack 13 iCDU drain 14 Leak detection sensors in the iCDU drain pan 15 Auto/Manual water fill connection 16 Water supply to upper iCDU rack 17 Manual water fill button 18 Water supply to f8000 rack 19 Facility supply to iCDU 20 Facility return from iCDU
Component identification 14
iCDU display components
Item Description 1 Alarm button 2 Program button 3 Escape button 4 Up arrow button 5 Enter button 6 Down arrow button
Installation and configuration 15
Installation and configuration
Installation included with service The HPE Apollo 8000 System Startup Service purchased with your product includes the installation of Apollo f8000 racks and HPE Apollo 8000 iCDU racks, including all the associated HPE ProLiant servers and network switches. There are many benefits to the service agreement, including:
• Deployment management
• Service planning
• Site readiness verification checklist
• Installation and startup
• Installation verification tests (IVT)
• Customer orientation session
For more information on what is included in the service agreement, see the Apollo 8000 System Startup Service Data Sheet (http://www.hpe.com/support/hpesc).
Installation expectations Each installation has many variables, which dictate how the installation is performed and how long it might take, including:
• Size and configuration of the installation
• Site-specific variables, such as facility water and power
• Site-specific requirements, such as networking, safety, and security
A deployment manager will ensure all variables are accounted for prior to installation. For more information on what is included in the service agreement, see the Apollo 8000 System Startup Service Data Sheet (http://www.hpe.com/support/hpesc).
For the purpose of this explanation, a common installation includes:
• One iCDU rack
• Two f8000 racks
IMPORTANT: The following table is a general overview of what a customer can expect during the installation process, which is by the Apollo 8000 Startup Service. It is not intended to be used as a procedure or schedule but is only a sample.
Installation requirement
Description Responsible party
Complete site audit questionnaire
Determine a general baseline for site readiness. Hewlett Packard Enterprise Sales or Customer
Installation and configuration 16
Installation requirement
Description Responsible party
Site audit Determine any modifications required to ensure site readiness for delivery and installation based on specifications outlined in the Apollo 8000 System Site Preparation Guide.1 After the site audit is complete, Hewlett Packard Enterprise Services can provide a quote for custom services, if the Customer desires.
Hewlett Packard Enterprise Sales, Hewlett Packard Enterprise Services, and Customer
Site modifications Complete all required modifications to ensure site readiness for delivery and installation.
Customer, Third party, or Hewlett Packard Enterprise Services2
Install and test plumbing Varies depending on existing plumbing and modifications required.
Customer, Third party, or Hewlett Packard Enterprise Services2
Install rack solution Includes: • Deliver all racks.3 • Level and bolt the racks in place. • Make all facility connections, including power,
water, and networking.
Hewlett Packard Enterprise Services
Infiniband cabling between racks
Cabling all infiniband network components for the rack solution.
Customer, Third party, or Hewlett Packard Enterprise Services2
Complete water preparation
Includes: • Pressure testing. • Add water and required chemistry. • Test chemistry.
Hewlett Packard Enterprise Services
Power on the system Includes: • Power on the iCDU rack. • Power on the CMU solution. • Power on the f8000 racks.
Hewlett Packard Enterprise Services
Test the system Includes: • Run rack-level system tests. • Run system-level tests. • Clean up fabrics. • Verify all f8000 rack hardware components are
functioning. • Verify the APM is functioning and reporting data. • Verify the CMU solution is functioning and reporting
data.
Hewlett Packard Enterprise Services
Test the server functionality
Includes: • Verify all server nodes boot. • Run CT single-node tests. • Verify all nodes pass CT node tests.
Hewlett Packard Enterprise Services
Customer orientation session
Varies depending on customer agreement and requirements.
Hewlett Packard Enterprise Services
1To view the Apollo 8000 System Site Preparation Guide, see the Hewlett Packard Enterprise website (http://www.hpe.com/info/windows/docs). 2Based on a separate, custom scope of work to be performed by Hewlett Packard Enterprise Services. 3White Glove Delivery is standard for installing the rack solution at the Customer site. This includes equipment delivery, transportation off the truck and into the final destination, and then removing all trash from the site within a few days of delivery.
Power components 17
Power components
About the power shelf The power shelf takes power input from the facility and provides the output power to the f8000 rack components. The power meters installed within the power shelf are specific to meet the input voltage requirement based on location:
• North America—Power meters are configured for 480/277VAC; 3-wire plus Neutral plus ground; L22-30P connections (two per power shelf)
• International—Power meters are configured for 380/220–415/240VAC; 3-wire plus Neutral plus ground; Hubbell 532P6W connections (two per power shelf)
Power components 18
Two power shelves are installed in each f8000 rack, and each is responsible for providing power to half of the rack. They also provide redundant power to some of the major f8000 rack components, such as the HEX and utility module.
Power shelf front components
Power components 19
Item Component Description 1 High voltage regulator Four high voltage regulators convert the facility power into
accessible power for the f8000 rack components.
Power shelf LED indicators
Health status LED:
• Green—Normal operation
• Off—No power indicates an issue
Power shelf rear components
Item Component Description 1 Power shelf controller Powers the power shelf on and off and reports status out to
APM 2 High voltage DC output connectors Sends power out to the f8000 rack components 3 Communication pass-through module Provides a serial communication port to interface with the
APM 4 Input power modules Brings power in from the facility to be used by the f8000
rack components 5 Grounding post connection Connection point for grounding the power shelf
Power components 20
Power shelf rear connectors
Item Component Description 1 Appliance outlet Provides a breaker-protected outlet that is connected from
one phase to neutral. The Rong Feng RF-203P-HP-1.2 outlet is 277VAC for the 480VAC input meter and a standard IEC 60320-2-2F for the 380/220VAC to 415/240VAC input meter. Not used for the Apollo 8000 System
2 Appliance breaker switch Breaker switch that controls the appliance outlet
3 Main 3-phase, 5-wire input cord Main power input to the power shelf 4 Compute cell connection Provides power to the top, left compute cell within the half
rack 5 Breaker indicator posts Pops out to indicate that one of the two output connectors
on either side of the post has tripped 6 HEX connection Provides power to the HEX in the iCDU rack 7 Torx driver Attached driver is used for removing the input power
modules. 8 Utility module connection Provides power to the utility module 9 Compute cell connection Provides power to the top, right compute cell within the half
rack 10 Compute cell connection Provides power to the bottom, right compute cell within the
half rack 11 Discovery services connection Reserved for future use 12 EPO connection Provides an EPO port for the power shelf. Supports either
normally open or normally closed configurations. Not used for the Apollo 8000 System
13 Bottom HEX connection* Provides power to one of the bottom* water wall assemblies
14 Top HEX connection* Provides power to one of the top* water wall assemblies 15 APM data connection Connects to the utility module to provide power shelf data
and control to APM 16 Compute cell connection Provides power to the bottom, left compute cell within the
half rack
*Each power shelf provides power to the top and bottom HEX water wall assemblies, therefore creating redundant power in the event of one power shelf losing functionality. The exact connection to the top or bottom HEX varies based on the top and bottom power shelf locations.
Power components 21
Installing the power shelf 1. Align and slide the power shelf into its bay in the front of the rack until it is fully inserted. 2. Insert four P2 Phillips screws, two in each side, to secure the power shelf to the front of the rack.
3. Install two protective covers, one on each side, over the front ends of the power shelf. 4. Install two P2 Phillips screws, one in each protective cover, to secure it the power shelf.
5. With the release lever in the open position, align and slide the voltage regulator into the bay in the
front of the power shelf until the release lever begins to engage.
Power components 22
6. Close the release lever to lock it and fully seat the voltage regulator into the backplane of the power shelf.
7. Close the rack front door. 8. From the rear of the rack, install the power cables and serial control cables into the rear of the power
shelf. 9. Insert both input modules into the rear of the power shelf.
Configuring the power shelf The power shelf is pre-configured to meet the input voltage requirements, based on location:
• North America—Power meters are configured for 480/277VAC; 3-wire plus Neutral plus ground; L22-30P connections (two per power shelf)
• International—Power meters are configured for 380/220–415/240VAC; 3-wire plus Neutral plus ground; Hubbell 532P6W connections (two per power shelf)
There are no additional configuration steps required after installation.
Power components 23
Redundant power supplies The redundant power supplies exist in pairs throughout the rack for the purpose of providing redundant power in the event of power failure. They are hot-swappable and can be replaced without having to power down the f8000 rack.
There are 11 redundant power supply pairs throughout the f8000 rack, responsible for providing redundant power to rack components:
• One pair of redundant power supplies in the front, center of the top heat exchanger
• One pair of redundant power supplies in the front, center of the bottom heat exchanger
• One pair of redundant power supplies in each of the management module assemblies
• One pair of redundant power supplies in the front of the utility shelf
Power components 24
Redundant power supply LED indicator
Health status LED:
• Green—Normal operation
• Off—No power indicates an issue
Installing the redundant power supply 1. Align and slide the power supply into the power supply bay. 2. Push the power supply until it is fully seated and locks into place.
Monitoring power usage Power component status and usage is monitored through APM. Use the SHOW POWER command to retrieve the following details:
Power components 25
• Input AC power information
• AC voltage
• AC current
• AC power
• Power factor
• Total harmonic distortion
• High voltage DC output voltage
• High voltage DC output current
• High voltage DC output power
• Health status
To view instructions on retrieving power status and details, see the HPE Advanced Power Manager User Guide for Apollo 8000 Solution on the Hewlett Packard Enterprise website (http://www.hpe.com/support/Apollo8000_APM_UG_en).
Compute cell 26
Compute cell
About the compute cell One compute cell is installed in each 10U quadrant space within the f8000 rack. Each compute cell assembly is composed of the following items:
• The management module
• Five fan units
• One pair of redundant power supplies
There is a total of eight compute cell assemblies in the rear of each f8000 rack; four in the top of the rack and four in the bottom of the rack.
Compute cell 27
The compute cell is the control center for a 10U cell of servers. It is responsible for the following tasks:
• Providing an iLO aggregation point for all servers within the cell
• Providing a connection point to APM to communicate and access data including power and temperature information for the cell
• Controlling the power output and fan speeds for the cell, based on pre-determined setpoints within APM
Compute cell components
Item Component Description 1 Redundant power supplies Provides redundant power to the management module
assembly 2 Power distribution module assembly Distributes power from the redundant power supplies 3 Fan units Provides air flow throughout the rear of the rack 4 Ethernet ports Provides an Ethernet connection between the utility
module and the compute cell 5 Management module Provides a connection point for communication between
the APM and the 10U cell
Compute cell 28
Compute cell numbering
Compute cell 29
Management module connectors and LED indicators
Item Description 1 Display port connector to the APM utility shelf 2 Identification LED
• Solid blue = Management module is receiving power and communicating.
• Off = No power or communication
Compute cell 30
Fan numbering Front rack fan numbering
The front rack fan health status or failures are only reported after entering the show thermal status command in APM.
For example: Upper Rack Zone:
Sensor Index Status
------------------ ------ -------------
Leak Detector 0 no leaks
Leak Detector 1 no leaks
Leak Detector 2 no leaks
Door Sensor 0 opened ( front )
Compute cell 31
Door Sensor 1 opened ( rear )
Valve Sensor 0 not present
Valve Sensor 1 not present
Water Temp (RTD) 0 29.4613 degC ( supply )
Water Temp (RTD) 1 29.8675 degC ( intermediate )
Water Temp (RTD) 2 31.6488 degC ( return -bot )
Flow 0 2.21365 PSI (740.00 mV reading)
Temp-Humid -board 0 33.66 degC, 21.35 %RH
Dew Point Temp. 0 8.52 degC
Air Temp. 0 31.0000 degC (Dev 0:0 )
Air Temp. 6 36.2500 degC (Dev 2:0 )
DC/DC Power Supply 0 112.00 mV (PS 0 - present - OK)
DC/DC Power Supply 1 191.00 mV (PS 1 - present - OK)
Control Index Status
------------------ ------ -------------
Door Lock 0 closed ( front )
Door Lock 1 closed ( rear )
Valve Control 0 not present
Valve Control 1 not present
EPO 0 running
Fan State Health RPM R1 Status R2 Status Fan mV Fan%
---- -------- ----- ------- --------- ------ ----- -----
1 : Fan installed healthy 4828 healthy healthy 341 29
2 : Fan installed healthy 4828 healthy healthy 341 29
3 : Fan installed healthy 4828 healthy healthy 341 29
4 : Fan installed healthy 4828 healthy healthy 341 29
5 : Fan installed healthy 4828 healthy healthy 341 29
6 : Fan installed healthy 4828 healthy healthy 341 29
7 : Fan installed healthy 4828 healthy healthy 341 29
8 : Fan installed healthy 4828 healthy healthy 341 29
9 : Fan installed healthy 4828 healthy healthy 341 29
10 : Fan installed healthy 4828 healthy healthy 341 29
Cell Fan %
---- -----
5 050
6 050
Compute cell 32
7 050
8 050
Lower Rack Zone:
Sensor Index Status
------------------ ------ -------------
Leak Detector 0 no leaks
Leak Detector 1 no leaks
Leak Detector 2 no leaks
Door Sensor 0 (not present)
Door Sensor 1 (not present)
Valve Sensor 0 not present
Valve Sensor 1 not present
Water Temp (RTD) 0 29.4925 degC ( supply )
Water Temp (RTD) 1 29.8675 degC ( intermediate )
Water Temp (RTD) 2 31.8050 degC ( return -bot )
Flow 0 2.11198 PSI (722.00 mV reading)
Temp-Humid -board 0 33.75 degC, 21.19 %RH
Dew Point Temp. 0 8.49 degC
Air Temp. 0 30.7500 degC (Dev 0:0 )
Air Temp. 6 35.0000 degC (Dev 2:0 )
DC/DC Power Supply 0 146.00 mV (PS 0 - present - OK)
DC/DC Power Supply 1 148.00 mV (PS 1 - present - OK)
Control Index Status
------------------ ------ -------------
Door Lock 0 not present
Door Lock 1 not present
Valve Control 0 not present
Valve Control 1 not present
EPO 0 running
Fan State Health RPM R1 Status R2 Status Fan mV Fan%
---- -------- ----- ------- --------- ------ ----- -----
1 : Fan installed healthy 4828 healthy healthy 336 29
2 : Fan installed healthy 4828 healthy healthy 336 29
3 : Fan installed healthy 4828 healthy healthy 336 29
4 : Fan installed healthy 4828 healthy healthy 336 29
Compute cell 33
5 : Fan installed healthy 4828 healthy healthy 336 29
6 : Fan installed healthy 4828 healthy healthy 336 29
7 : Fan installed healthy 4828 healthy healthy 336 29
8 : Fan installed healthy 4828 healthy healthy 336 29
9 : Fan installed healthy 4828 healthy healthy 336 29
10 : Fan installed healthy 4828 healthy healthy 336 29
Cell Fan %
---- -----
1 050
2 050
3 050
4 050
Compute cell 34
Compute cell fan numbering
The compute cell fans health status or failures are reported after entering the show rack command in APM or by reviewing the Fan tab within iLO.
For example: 1: Dist. Module
2: Enclosure
Product Name :Apollo 8000
Product SKU :000000-000
Serial Number:0000000000
UID :off
Status :
1 : Compute node (active)
2 : (unused)
Compute cell 35
3 : Compute node (active)
4 : (unused)
5 : Compute node (active)
6 : (unused)
7 : Compute node (active)
8 : (unused)
9 : Compute node (active)
10 : (unused)
11 : Compute node (active)
12 : (unused)
13 : Compute node (active)
14 : (unused)
15 : Compute node (active)
16 : (unused)
17 : Compute node (active)
18 : (unused)
19 : Compute node (active)
20 : (unused)
Power supply slot 1 is occupied, status good
Power supply slot 2 is occupied, status good
Fan slot 1 is occupied, status good
Fan slot 2 is occupied, status good
Fan slot 3 is occupied, status good
Fan slot 4 is occupied, status good
Fan slot 5 is occupied, status good
Installing the management module 1. Align and slide the management module into its bay in the compute cell until the release lever begins
to engage.
Compute cell 36
2. Close the release lever until it locks under the release button. The module should now be fully seated in the tray infrastructure backplane of the compute cell.
3. Connect all cables, including the display port cable.
Utility module 37
Utility module
About the utility module The utility module serves as a central location for gathering and monitoring data for the Apollo 8000 system, including:
• Managing rack Ethernet infrastructure via an embedded 160 port 1GB switch
• Gathering data from the f8000 rack sensors
• Controlling power output and fan speed for each management module cell
• Updating component status within APM
• Maintaining alarm conditions within APM
Utility module 38
There is one utility module installed in each f8000 rack within the Apollo 8000 System.
The utility module might appear different based on the model installed. For more information on the different models available, see "Utility module products (on page 38)."
Utility module products There are two utility module products that can be installed in the Apollo 8000 System. Although they might appear different and have slightly different features, their basic functionality within the Apollo 8000 System is the same.
Utility module 39
For the purpose of clarity in this document, when removing and replacing spare parts, the components will be referred to as shown in the following illustration and table.
Item Description 1 HPE Apollo 8000 Utility Module v1 (Apollo 8000 UM1) 2 HPE Apollo 8000 Utility Module v2 (Apollo 8000 UM2)
HPE Apollo 8000 UM1
Utility module front components
Utility module 40
Item Component Description 1 iLO aggregation module A hot-pluggable switch module that aggregates all server
iLOs to one port to provide an interface for the System Manager server to APM
2 APM module The main management module that contains the processor monitoring rack health for APM
Utility module front connectors
Item Component Description 1 Expansion port/System management
port Connects to the CMU solution
2 iLO A port iLO aggregation port collects uplink information from all of the compute nodes.*
3 Reset button Used to reset the APM 4 Serial console RJ-45 serial interface to the APM management CLI 5 Service port RJ-45 serial console port for local service, excluding any
changes that require a password, such as configuration or set points
6 Ethernet connection Ethernet interface to access the APM remotely 7 Facility Redundant reset, On-Gen, and Aux signals
*If the iLO A port is used to provide IP addresses, do not connect the Ethernet cable to the management module. The two connections at the same time result in a network loop condition.
Utility module front LEDs
Item Component Behavior 1 UID LED Flashes to indicate:
• The firmware is being flashed. • Identification by use of the "Set UID on" or "Set UID
off" commands
Utility module 41
Item Component Behavior 2 Active LED Flashes to indicate Ethernet activity 3 UID LED Flashes to indicate:
• The firmware is being flashed. • Identification by use of the "Set UID on" or "Set UID
off" commands
4 Health LED Solid Green—No active faults logged Solid Red—One or more faults has been logged.
Utility module rear components
Item Component Description 1 Serial ports Reserved for future use. Set up using the "Connect port"
command. 2 Power distribution module ports for
power shelf Connects via serial interface to the internal power shelves
3 Management module connection ports
APM connection to the management module and thermal controller
4 Redundant power supplies Provide redundant power for the utility module 5 Power connectors High voltage DC power connection to the power shelves 6 Fan unit Circulates cool air through the utility module
Utility module 42
Utility module rear connectors
Item Component Description 1 Power supply board connector Connects the utility module to the redundant power
supplies 2 Serial port Reserved for future use 3 Serial port Reserved for future use 4 Power distribution port A Connects the utility module to the top power shelf 5 Power distribution port B Connects the utility module to the bottom power shelf 6 Management module and rack
controller display port connections Connects the utility module to each of the management modules and thermal controllers
7 Fan connector Connects to the fan module on the utility module
HPE Apollo 8000 UM2
Utility module 43
Utility module front components
Item Component Description 1 Reset button Resets the APM 2 Serial console RJ-45 serial interface to the APM management CLI 3 Service port RJ-45 serial console port for local service, excluding any
changes that require a password, such as configuration or set points
4 Ethernet connection Ethernet interface to access the APM remotely 5 Facility connection Redundant reset, On-Gen and Aux signals 6 Expansion port Connects to the CMU solution
7 iLO A port/ Ethernet switch configuration
iLO aggregation port collects uplink information from all of the compute nodes. Provides access to configure the Ethernet switch
8 Blank* Pre-installed blanks to ensure thermal consistency for internal utility module components
9 Redundant power supply Provides redundant power to the utility module 10 Redundant power supply Provides redundant power to the utility module 11 Ethernet switch Provides Ethernet network connectivity
*Blanks are pre-installed on the utility module to ensure that the thermal requirements for the internal utility module components are maintained. Blanks should remain installed at all times to reduce the risk of overheating the internal electronics.
Utility module 44
Utility module front LEDs
Item Component Behavior 1 Active LED Flashes to indicate Ethernet activity 2 UID LED Flashes to indicate:
• The firmware is being flashed. • Identification by use of the "Set UID on" or "Set UID
off" commands
3 Health LED Solid Green—No active faults logged Solid Red—One or more faults has been logged.
4 UID LED Flashes to indicate: • The firmware is being flashed. • Identification by use of the "Set UID on" or "Set UID
off" commands
5 Power health LED Solid Green—Normal operation Flashing Red—One or more faults has been logged.
6 Power health LED Solid Green—Normal operation Flashing Red—One or more faults has been logged.
7 Access panel sensor Not activated* 8 UID LED Flashes to indicate:
• The firmware is being flashed. • Identification by use of the "Set UID on" or "Set UID
off" commands
9 Downlink activity LED Off—No activity Flashing Green—Activity for port
10 Uplink activity LED Off—No activity Flashing Green—Activity for port
11 Health status LED Solid Green—Normal operation Solid Red—One or more faults has been logged.
12 Power LED Off—No power Solid Green—Normal power operation
*Reserved for future use
Utility module 45
Utility module rear components
Item Component Description 1 Affixed power input cables Connects to both power shelves to provide redundant
power to utility module 2 Mini DB-9 connector Connects to the internal power shelf (bottom) 3 Mini DB-9 connector Connects to the internal power shelf (top)
4 Mini DB-9 connector Reserved for future use 5 Mini DB-9 connector Reserved for future use
6 Mini SAS HD connector Connects to the compute cell management module (cells 1 and 2)
7 Mini SAS HD connector Connects to the compute cell management module (cells 3 and 4)
8 Mini SAS HD connector Connects to the compute cell management module (cells 5 and 6)
9 Mini SAS HD connector Connects to the compute cell management module (cells 7 and 8)
10 Display port connector Connects to the internal HEX (top f8000 rack) 11 Display port connector Connects to the internal HEX (bottom f8000 rack) 12 Display port connector Reserved for future use 13 Reset button Utility module reset
14 Ethernet switch management serial console
Management serial console
15 QSFP connector Q1 Ethernet Uplink 16 QSFP connector Q2 Ethernet Uplink 17 QSFP connector Q3 Ethernet Uplink 18 QSFP connector Q4 Ethernet Uplink 19 J15 and J16 connector Connects to the Ethernet ports of compute cell # 8 20 J13 and J14 connector Connects to the Ethernet ports of compute cell # 7 21 J11 and J12 connector Connects to the Ethernet ports of compute cell # 6 22 J9 and J10 connector Connects to the Ethernet ports of compute cell # 5 23 System fan Provides system cooling 24 System fan Provides system cooling
Utility module 46
Item Component Description 25 J7 and J8 connector Connects to the Ethernet ports of compute cell # 4 26 J5 and J6 connector Connects to the Ethernet ports of compute cell # 3 27 J3 and J4 connector Connects to the Ethernet ports of compute cell # 2 28 J1 and J2 connector Connects to the Ethernet ports of compute cell # 1
Utility module rear LEDs
Item Component Behavior 1 UID LED Flashes to indicate:
• The firmware is being flashed. • Identification by use of the "Set UID on" or "Set UID
off" commands
2 Health LED Solid Green—No active faults logged Solid Red—One or more faults has been logged.
3 Q1-Q4 Uplink LEDs Solid green indicates an uplink connection is established. 4 Q1-Q4 Activity LED Flashing green indicates link activity. 5 Power LED (fan) Illuminates to indicate power 6 Power LED (fan) Illuminates to indicate power
Installing the utility module 1. Align and slide the utility shelf into the rack until the front hinges can be rotated in place and the
captive thumbscrews are aligned with the front of the rack. 2. From the front of the f8000 rack, tighten the two thumbscrews, one on each side, to secure the utility
shelf the front of the rack.
Utility module 47
3. From the rear of the f8000 rack, tighten the two thumbscrews, one on each side, to secure the utility shelf to the inside of the rack.
The utility module might appear different based on the model installed. For more information on the different models available, see "Utility module products (on page 38)."
4. Connect all appropriate cables to the rear of the utility shelf.
Configuring the utility module For information on configuring the utility module, see the Advanced Power Manager for Apollo 8000 User Guide (http://www.hpe.com/support/hpesc).
Thermal components 48
Thermal components
About the thermal components
CAUTION: To reduce the risk of condensation and avoid damage to the equipment when operating with chilled water, keep the rack doors closed during operation.
The thermal components are responsible for providing cooling to the rack components and servers. The Apollo 8000 System uses both water cooling and air circulation to maintain thermal parameters for optimal server and IT health. The following components work together to maintain the thermal requirements:
• HEX with embedded thermal controller
• Water walls
• Fans
• Rack door sensors
There are thermal components located in the top and bottom half rack of the f8000 rack and the top half of the iCDU rack.
Item Component Description 1 Left water wall Provides cooling to the servers on the left side of the f8000
rack
Thermal components 49
Item Component Description 2 HEX Removes the heated air from within the f8000 rack 3 Right water wall Provides cooling to the servers on the right side of the
f8000 rack 4 Fans Provides rack-level air circulation for system cooling
Thermal gasketing is installed throughout the racks to ensure air flow maintains the pre-configured thermal parameters. The rack doors must remain closed during normal operation to ensure thermal parameters.
HEX The HEX removes the heated air from within the f8000 rack, and supplies rack-level air cooling that is circulated by the fans.
The HEX units are located in both the f8000 rack and the iCDU rack; one in the top of the f8000 rack, one in the bottom of the f8000 rack, and one in the top of the iCDU rack.
HEX components and connectors
Item Component Description 1 HEX Removes the heated air from within the f8000 rack 2 Fan unit Provides rack-level air circulation for system cooling 3 Redundant power supplies Provides redundant power to the fans and embedded
thermal controller on the HEX 4 Embedded thermal controller Adjusts temperature requirements for the HEX
Thermal components 50
Thermal controller The embedded thermal controller on each HEX gathers data and adjusts temperature requirements based on pre-configured setpoints within APM as follows:
• Gathers information for all of the sensors within the rack
• Sends data from the sensors and management module to APM
• Receives feedback from APM alarm conditions, and adjusts fan speeds to maintain pre-configured temperature setpoints
• Opens the rack doors during an over temperature event
There are three embedded thermal controllers; two in the f8000 rack and one in the iCDU rack.
Thermal controller components and connectors
Item Component Description 1 Display port connector Connects the thermal controller to APM 2 Ethernet connector Debug only 3 Serial port connector Debug only 4 Sensor cable connector 1 Connects to the cable harness that gathers all sensor data
for the thermal controller 5 Sensor cable connector 2 Connects to the cable harness that gathers all sensor data
for the thermal controller 6 Fan 6-10 management connector Connector for fan management signals 7 Fan 1-5 management connector Connector for fan management signals 8 Fan 1-5 power connector Connector to supply power to the fans 9 Fan 6-10 power connector Connector to supply power to the fans
Updating the thermal controller firmware 1. Log in to the APM CLI interface using a serial port, Telnet, or SSH connection. 2. Enter the CLI command: UPGRADE CONTROLLER RS485 <PORT#>, where <port#> is 9 or 10.
If the thermal controller is already running the latest firmware, the update will not start.
Thermal components 51
Following the firmware update, the APM is automatically updated and reset to run the new image.
For more information, see the Advanced Power Manager for Apollo 8000 User Guide (http://www.hpe.com/support/Apollo8000_APM_UG_en).
Water walls The water walls provide cooling to the servers in the Apollo 8000 System through the TBB interface. The water flows through the water wall, condensing the vapor within the cooling pipes of each server. Additionally, the water walls do the following:
• Link the HEX to the servers
• Supply filtered cooler water to the servers through the TBB interface
• Remove the heated water from the servers, returning it to the secondary plumbing
There are two water walls, a left and a right, on either side of each HEX. Each water wall provides cooling to the servers in that rack quadrant.
NOTE: The HEX in the iCDU rack does not have water walls or a TBB interface. The fans circulate cooled air throughout the iCDU rack, but there is no direct link between the HEX and any installed servers in that rack via TBB interface.
Water wall components and connectors
Item Component Description 1 Water wall vertical manifold Pathway for water supply and return 2 TBB interface Provides the cooler water to the servers, while removing
the heated water 3 TBB pull bar Provides a stop for server installation and engages the
TBB with the server
Thermal components 52
Rack door sensors and locks The rack door sensors help maintain the thermal conditions within the f8000 rack, by activating the door locks to keep the doors closed when the temperature is correct and deactivating the door locks to automatically open the rack doors when the temperature gets too high.
CAUTION: Do not force the rack door open when the door locks are activated. Instead, use the APM CLI to release the magnetized door locks, and then open the doors. Trying to force the door open while the door locks are activated can result in a bent or broken rack door.
There is one door sensor and lock on the rack front door, and one door sensors and three locks on the rack rear door. For more information on deactivating the magnetized door locks, see "Activating and deactivating door locks (on page 52)."
Activating and deactivating door locks The magnetic door locks should be activated and deactivated through the APM CLI: 1. Open VT reflections for Unix. 2. Select Connection>Connection Set-Up. 3. Select Secure Shell. 4. Log in to the f8000 rack APM, using the following settings:
o Host name—Rack IP address o User name—Administrator
5. Select Connect to enter your Administrator password and access the CLI. 6. Enter one of the following commands:
o SET DOOR OPEN FRONT—Disengages the magnetic door locks on the rack front door. o SET DOOR OPEN REAR—Disengages the magnetic door locks on the rack rear doors. o SET DOOR CLOSE FRONT—Engages the magnetic door locks and closes the rack front door. o SET DOOR CLOSE REAR—Engages the magnetic door locks and closes the rack rear doors.
Infiniband switch 53
Infiniband switch
About the infiniband switch The 1U infiniband switch communicates data between processors and I/O devices from the servers within its 10U cell. Data is transmitted in packets that together form a message, exchanging information across the network.
One infiniband switch is installed for each 10U compute cell. There are a total of eight infiniband switches in the f8000 rack; four in the top of the rack and four in the bottom of the rack.
NOTE: If an infiniband switch is not installed in a switch bay, install a switch blank to ensure thermal requirements are maintained.
Infiniband switch 54
Infiniband switch front components and connectors
NOTE: The orientation of the infiniband switch might appear different depending on which rack slot it is installed in.
Item Component Description 1 QSFP ports Connects the infiniband switch to the outer rack network
switches via optical cables 2 Power button Powers the infiniband switch on and off
Infiniband switch port numbering Infiniband switch port numbering is used to identify or map cables within the solution.
Infiniband switch 55
Infiniband switch LED indicators
Item Component Description 1 Status LED Off—No power to the switch
Flashing Green—Switch is powering up Solid Green—Switch has power and normal operation Solid Amber—Non fatal error, which requires troubleshooting, but does not require shutdown Solid Red—Fatal error
2 Blank Reserved for future use 3 Bad Port LED Off—All ports are in normal operation
Flashing Amber—One or more ports has a symbol error, such as a bad cable, connection, or connector
4 UID LED Flashes to indicate identification of infiniband switch 5 Link LED Flashing Green—Data activity. The flashing speed is proportional
to the data transfer speed. Solid Amber—Physical link up Flashing Amber—Fault or error with the physical link
6 Activity LED Flashing to indicate infiniband activity 7 Health LED Solid Green—No active faults logged
Flashing Amber or Red—One or more faults has been logged. 8 Power LED Green—Normal power to the infiniband switch
Amber—Auxiliary power to the infiniband switch
Infiniband switch 56
Infiniband switch rear components and connectors
Item Component Description 1 High voltage DC power connector Provides power input from the power shelf to the infiniband
switch 2 Management and Auxiliary power
connector Provides infrastructure management and auxiliary power from the compute cell to the infiniband switch
3 Infiniband server cartridge connector Connects the infiniband switch to 9 servers via the cable bundle
Installing the infiniband switch 1. Align the infiniband switch to the appropriate bay within the chassis. 2. Slide the infiniband switch into the bay, and secure it with the locking tab. 3. Connect cables to the appropriate connectors in the front and rear of the infiniband switch.
Safety components 57
Safety components
About the safety components There are many built-in safety components throughout the Apollo 8000 System. These components are designed to prevent major catastrophic damage to your equipment in the event of an emergency. Safety components include the following:
• Primary/Facility side leak detection sensors
• Secondary leak detection sensors
• Temperature sensors
• Automatic door opening release mechanism
• (Optional) Automatic Rapid Power Off (ARPO) system
Leak detection sensors The leak detection sensors are configured to detect extra moisture in the HEX drip pans. There are three leak detection sensors in each HEX drip pan, at the bottom each HEX unit. In order for the moisture to be considered a leak, two of the three sensors must indicate excess moisture.
If the sensors indicate a leak and ARPO is installed: 1. An alarm is triggered. 2. The rack with the leak is isolated. 3. The water valve is shut off, preventing further water flow to the HEX. 4. Power is automatically shut off to the rack and rack components.
If the sensors indicate a leak and ARPO is not installed: 1. An alarm is triggered. 2. The rack with the leak is isolated. 3. The water valve is shut off, preventing further water flow to the HEX.
Temperature sensors and automatic door release mechanism
The temperature sensors located throughout the Apollo 8000 System are configured to read a stream of temperatures and indicate when a sensor exceeds an over temperature threshold.
If an over temperature threshold is exceeded within a rack, the following events occur:
• The front and rear rack doors open automatically to enable additional air flow to the rack components.
• The system fans on the HEX and compute cells ramp up to provide extra cooling to the rack components.
Safety components 58
(Optional) ARPO ARPO is a unique safety option that can be installed in each rack (f8000 rack and iCDU rack) for the Apollo 8000 System that enables the immediate shutdown of power to the entire rack and rack components in the event of an emergency, such as a major water leak within a powered rack.
Cabling 59
Cabling
Cabling overview The cabling throughout the Apollo 8000 System is quite extensive and should be handled by an experienced service technician.
The images included in this chapter are designed to give an overview of the various cabling connections required throughout the Apollo 8000 System, not to designate specific instructions for cabling the product.
The cabling scenarios differ depending on which utility module you have installed. For more information on the different models available, see "Utility module products (on page 38)."
Cabling connections when the Apollo 8000 UM1 is installed include the following:
• Connecting the utility module to the power shelves (on page 59)
• Connecting the utility module to the management modules (on page 60)
Cabling connections when the Apollo 8000 UM2 is installed include the following:
• Connecting power (on page 61)
• Connecting communication (on page 62)
• Connecting the f8000 rack to the iCDU rack (on page 65)
• Connecting the CMU solution hardware ("Connect the CMU solution hardware" on page 67)
Cabling when the HPE Apollo 8000 UM1 is installed Due to the difference of the rear connections of the two utility module versions, the cabling is very different depending on which utility module is installed.
Cabling connections when the Apollo 8000 UM1 is installed include the following:
• Connecting the utility module to the power shelves (on page 59)
• Connecting the utility module to the management modules (on page 60)
Connecting the utility module to the power shelves There are two main connection points between the utility module and each of the power shelves:
• Blue—Indicates the power connection between the power shelves and the utility module (set up for redundant power via the dual power shelves)
Cabling 60
• Green—Indicates the communication connection via serial port between the power shelves and the utility module (transferring data from each power shelf to APM)
Connecting the utility module to the management modules The display port cables connecting the utility module to the compute cell management modules and rack thermal controllers must be connected to the correct ports on the rear of the utility module in order for accurate data to appear in APM.
Item Component Description 1 Management module number 1 Display port 1 on the utility module connects to the bottom,
right management module in the bottom of the rack 2 Management module number 2 Display port 2 on the utility module connects to the top,
right management module in the bottom of the rack
Cabling 61
Item Component Description 3 Management module number 3 Display port 3 on the utility module connects to the bottom,
left management module in the bottom of the rack 4 Management module number 4 Display port 4 on the utility module connects to the top, left
management module in the bottom of the rack 5 Management module number 5 Display port 5 on the utility module connects to the bottom,
right management module in the top of the rack 6 Management module number 6 Display port 6 on the utility module connects to the top,
right management module in the top of the rack 7 Management module number 7 Display port 7 on the utility module connects to the bottom,
left management module in the top of the rack 8 Management module number 8 Display port 8 on the utility module connects to the top, left
management module in the top of the rack 9 Embedded rack controller, top HEX* Connects the utility module to the rack thermal controller in
the top of the rack 10 Embedded rack controller, bottom
HEX* Connects the utility module to the rack thermal controller in the bottom of the rack
*The rack controller is not accessible to a user, and is accessed by Hewlett Packard Enterprise Service personnel only.
Cabling when the HPE Apollo 8000 UM2 is installed Due to the difference of the rear connections of the two utility module versions, the cabling is very different depending on which utility module is installed.
Cabling connections when the Apollo 8000 UM2 is installed include the following:
• Connecting power (on page 61)
• Connecting communication (on page 62)
• Connecting the f8000 rack to the iCDU rack (on page 65)
• Connecting the CMU solution hardware ("Connect the CMU solution hardware" on page 67)
Connecting power Input power from the facility comes into the power shelves and is then filtered out to the f8000 rack components.
In many instances, the dual power shelves provide redundant power to major components.
Connect power to the rack components The power shelves provide the power the to rack components within the f8000 rack.
• Blue—Indicates the connection between the power shelf and the compute cell management modules
• Green—Indicates the connection between the power shelf and the top and bottom HEX (set up for redundant power via the dual power shelves)
Cabling 62
• Red—Indicates the connection between the power shelf and the utility module (set up for redundant power via the dual power shelves)
Connecting communication The components in the f8000 rack all communicate with one another, and then send the information out via the APM utility module to the network. Therefore, many connections are required to ensure the components can accurately communicate data including the following connections:
• Connect the HEX and management modules to the utility module.
• Connect the power shelves to the utility module.
• Connect the compute cell Ethernet to the rack integrated Ethernet switch on the utility module.
Connect the HEX and management modules to the utility module The utility module serves as the central point of communication for many of the rack components, and therefore has many connection points to the various components. Each of these connections uses a display port cable that transfers the information from the component to the utility module.
Cabling 63
• Blue—Indicates the connection between the utility module and the top and bottom f8000 rack HEX
• Black—Indicates the connection between the utility module and the management module of compute cells 7 and 8
• Purple—Indicates the connection between the utility module and the management module of compute cells 5 and 6
• Green—Indicates the connection between the utility module and the management module of compute cells 1 and 2
• Red—Indicates the connection between the utility module and the management module of compute cells 3 and 4
Cabling 64
Connect the power shelves to the utility module The utility module collects power information from the two power shelves and displays the information in APM.
Cabling 65
Connect the compute cell management modules to the utility module The multi-port Ethernet cables connecting the utility module to the compute cell must be connected to the correct ports on the rear of the utility module for accurate data to appear in APM. Two cables from each port in the rear of the utility module connect to the two Ethernet ports on each compute cell.
Connecting the f8000 rack to the iCDU rack There are a few connections between the f8000 rack and the iCDU rack that include the following:
• Connect the iCDU HEX to the power shelves in the adjacent f8000 racks.
• Connect the iCDU rack management switch to the utility module in the f8000 rack.
Cabling 66
Connect the iCDU HEX to the power shelves The HEX in the top of the iCDU rack receives redundant power from the power shelves in the two adjacent f8000 racks.
Cabling 67
Connect the iCDU rack management switch to the f8000 utility module The iCDU rack management switch communicates data from the CMU solution and the iCDU to the f8000 rack utility module via a network connection.
Connect the CMU solution hardware The CMU solution requires several specific hardware components, including the Advanced Power Manager utility module, HPE DL360e Gen8 Server, and rack management switch. These hardware components are located within the top half of the iCDU rack, and there are many connection points between them.
• Green—Indicates the network connection between the APM and the switch
• Blue—Indicates the display port connection between port 9 of the APM and the HEX
Cabling 68
• Red—Indicates the network connection to the iCDU (for pcoWeb interface)
iCDU 69
iCDU
About the iCDU The iCDU controller evaluates, queries, and manages various measurements and warning and alarm messages from the 8000 iCDU Rack.
The controller analyzes measurements provided by the cooling unit, generates any necessary warning or alarm messages, and sends the messages to the iCDU display.
When a new warning or alarm is triggered, the following events occur:
• The warning and alarm messages appear on the iCDU display ("iCDU display components" on page 14).
• The red alarm LED lights on the iCDU appear for Warning and Critical alarms.
The iCDU monitors sensor values and utilizes PI control to modulate a VFD-controlled water pump and a facility water regulation valve to maintain both a user-selected differential pressure set point and a user-selected IT/pump-side water temperature set point. The iCDU utilizes digital on/off control incorporating hysteresis to control a vacuum pump or an air inlet valve to maintain a user-selectable vacuum pressure set point. The iCDU provides for internal communication through the Modbus protocol with a VFD, and external communication through a selectable protocol for BMS monitoring and control. It also detects and reports alarms indicating critical or non-critical operating conditions.
For more information and component locations, see "iCDU components."
iCDU setup Powering on the iCDU
IMPORTANT: Before powering on the iCDU verify: • The Hand/Auto is in the Off position. • There are no visible water leaks, especially around the connections and fittings.
To power on the iCDU: 1. Turn the iCDU power switch to the On position. Power is now supplied to the iCDU controller and
sensors. The water pump does not start until the switch is set to Auto. For the location of the power switch, see "iCDU components."
2. Press the PRG key to log in to the iCDU. o Enter password "1" to log in to CONTROL mode—enables the user to change set points
throughout the system o Enter password "3" to log in to CONFIG mode—enables the user to change set points and
additional settings throughout the system 3. Navigate to the Unit selection screen, and then select the preferred units to display.
iCDU 70
4. Verify the local atmospheric pressure value is accurate, or update it if necessary.
5. Navigate to the Supply Temp PI screen and the Pmp Diff Pres PI screen and verify the rack supply
water temperature set point values are appropriate for the existing operational condition.
NOTE: Do not make changes to the following values to avoid an unstable output: • DB • Prop Band • Int Time • Deriv
iCDU 71
6. Navigate through the Alarms threshold screens and verify that the threshold values are appropriate for the existing operational condition.
iCDU 72
7. Verify there are no active alarms on the iCDU display. 8. Verify there are no water leaks around the iCDU. 9. Turn the iCDU Hand/Auto switch to the Auto position. 10. Navigate to the IT Iso Valve Status screen and the Fac Iso Valve Status screen to verify the iCDU
isolation valves are open on both the IT and facility side.
iCDU 73
11. Navigate to the Temperature screen and Pressures screen and verify the values are within normal ranges.
12. After a short delay, the iCDU pump starts circulating water through the system. 13. Navigate to the Diff Pressure screen and verify that the pump differential pressure is no more than
20 psid.
14. Navigate to the Tank Level screen and verify that the tank water level is above 2/3 full (or at least 20
inches).
15. Allow the iCDU to run for several minutes and monitor for startup alarms that might cause a
shutdown.
iCDU 74
16. Navigate back to the Temperature screen and verify that the IT supply water (HEX inlet) temperature is below 30°C (90°F).
17. If there is more than one iCDU, follow the same sequence for each iCDU, powering up each unit one
at a time. 18. Navigate to the Network Status screen and verify that there is communication between each of the
iCDUs.
19. Open the rack isolation valves, and verify that water is flowing to the rack. 20. Power on the IT racks.
Powering off the iCDU 1. Verify that it is safe to shut down the iCDU.
a. Verify the remaining iCDUs can support the existing IT load. If not, reduce the IT load before shutting down a iCDU.
b. Verify that all remaining iCDUs are within normal operating parameters and have no alarm conditions that could potentially result in the inability to support the IT load.
2. Turn the Auto/Manual switch to the Off position. The iCDU water pump stops.
3. Verify that the IT isolation valves are closed. 4. Disconnect the power to the iCDU.
BMS network communication Each iCDU can communicate externally to a BMS through a Modbus/TCP connection.
To configure the iCDU to communicate to a BMS: 1. Configure the iCDU to communicate to the pcoWeb card. 2. Update the pcoWeb card network settings:
a. Using a laptop or computer on the same network subnet, browse to the pcoWeb card default IP address: 172.16.0.1.
iCDU 75
b. Under Administrator utilities, select Go to Administrator Area. c. Log in to the pcoWeb card.
— User Name—admin
— Password—fadmin
iCDU 76
d. From the menu on the left side of the page, select Configuration.
e. Select the Network tab, and enter the following information:
— IP address
— Subnet mask
— Gateway
iCDU 77
f. Click Submit to save the updates. g. Reboot the pcoWeb card to verify the new network settings. Then, using the new IP address,
browse back to the card and verify that the settings saved.
iCDU display Password security level
The iCDU utilizes password control to restrict access to parameters based on a security level. The password entry screen can be accessed at any time by pressing the PRG button on the iCDU display.
The available password level is Status—Password 0. No control parameters can be modified (control screens are read-only). Forced values can be cleared but not entered. Configuration screens and diagnostic screens are not displayed. The date is displayed in place of a security level on most screens.
Press ESC to exit the password entry screen.
Screen time-out Unless the Screen Lock is active, the iCDU automatically resets the security level to 0 (status) and returns to the main screen after 60 seconds.
Screen lock Screen lock prevents the iCDU from resetting the security level or returning to the Main Status screen automatically. Scroll through the screens using the up and down arrows. Reset the security level manually on the Password screen.
If screen lock is active, the selected security level screen is displayed indefinitely and a padlock icon with the letters "SL" appears in the upper right corner of most screens.
To activate the screen lock, press Enter, and then press PRG on the keypad on any screen except the main status screen. To deactivate the screen lock, press and Enter and PRG on any screen except the Main Status screen for approximately three seconds.
Always available The following screens are always available from the iCDU display ("iCDU display components" on page 14). They are used to display status and do not include modifiable parameters.
Program Screen
Example Description
Main Status
The Main Status screen summarizes the control points and status of iCDU components. The status bar indicates program name, version.
iCDU 78
Program Screen
Example Description
IT Isolation Valves
The IT Isolation Valve Status screen lists the current IT isolation valve positions. There are no modifiable parameters on this screen.
Facility Isolation Valves
The Facility Isolation Valve Status screen lists the current facility isolation valve positions. There are no modifiable parameters on this screen.
Temperatures
The Temperature screen lists all available temperature sensor values in the selected unit of measure. There are no modifiable parameters on this screen.
Pressures
The Pressure screen lists all available pressure sensor values in the selected unit of measure. There are no modifiable parameters on this screen.
iCDU 79
Program Screen
Example Description
Differential Pressures
The Differential Pressures screen lists three calculated differential pressures in the selected unit of measure: • Vacuum (Vacuum
Pressure – IT Supply [Out])
• Pump side heat exchanger (IT Return [In] – IT Supply [Out])
• Facility side heat exchanger (Facility Supply [In] – Facility Supply [Out])
There are no modifiable parameters on this screen.
Nominal Motor Values
The Nominal Motor Value screen lists nominal values from the water pump motor as reported by the VFD. There are no modifiable parameters on this screen.
iCDU 80
Program Screen
Example Description
Modbus VFD Values 1, 2, and 3
The Modbus VFD Values screens display the actual values related to the water pump motor as reported by the VFD. If the VFD communication has failed, all values will be at zero. There are no modifiable parameters on these screens.
Network Status
The Network Status screen displays the current Network (pLAN) status, iCDU status, lead iCDU, and iCDU pLAN pulse.
Always visible The following screens are always visible from the iCDU display ("iCDU display components" on page 14). They are used to display status and can be modified at a CNTRL level or higher.
iCDU 81
Program Screen
Example Description
Vacuum pump
The Vacuum Pump screen displays the current vacuum pressure, and enables users to view or modify the vacuum pressure set point (SP), cut-in, and cut-out. When the vacuum pump is activated, *pump on* flashes.
Pump differential pressure PI
The Pump Differential Pressure PI screen displays differential pressure in, VFD percent out, and enables users to view or modify the PI loop controlling the pump VFD. Differential pressure set point (SP), proportional band (Prop Band), and integral time (Int Time) can also be modified at CNTRL level or higher.
Supply temperature PI
The Supply Temperature PI screen displays pump/IT temperature in and valve percent out, and enables users to view/modify the PI loop controlling the facility water valve. The pump/IT supply temperature set point (SP), proportional band (Prop Band), integral time (Int Time) and dead band (DB) can also be modified at the CNTRL level or higher.
iCDU control Control switch
Control of the unit is managed by the HAND/OFF/AUTO control switch and through adjustable parameters for each component of the unit using the controller display. For the location of the HAND/OFF/AUTO switch, see "iCDU components."
IMPORTANT: The HAND and OFF modes are only for use by authorized service personnel.
When the switch is in AUTO mode (normal operating mode), the unit operates as described in the AUTO mode section. Additionally, PRG illuminates when the unit is in either HAND or AUTO mode, even if there are no alarms.
iCDU 82
AUTO mode When the switch is in AUTO mode, the iCDU attempts to maintain pump differential pressure to the preferred set point using a PID control method. The control output modulates the pump VFD speed to maintain the pump differential pressure automatically. No manual overrides are available.
The iCDU also modulates the water regulating valve on the facility water side to control the pump discharge water temperature (IT supply water), based on the IT water outlet temperature setpoint.
iCDU isolation valves Each iCDU uses two pairs of isolation valves to isolate the iCDU from the rest of the system when it is not powered on, or if the power switch is set to the OFF position. The facility and IT isolation valve pairs open and close based on the following conditions.
Events Facility Isolation Valve IT Isolation valve Power Off Closed Closed Power On, Switch in Off position Open Closed Power On, Switch in Auto position Open Open Power On, Switch in Hand position Open Open iCDU critical shutdown (not leak related) Open Open iCDU critical shutdown (facility leak #2 detected)
Closed Closed
When the IT and facility isolation valves are open, the following status message appears.
iCDU 83
When the valves are closed, the following status message appears.
The command Yes sends the signal from the iCDU to open the valve. It takes a few moments for the valves to completely open or close from their previous status. If the OPEN/CLOSED status is blank, the valves are in interim movement.
Vacuum pressure Vacuum pressure is maintained by use of a vacuum pump (to let air out) and an air inlet solenoid valve (to let air in). Both are operated by a digital signal from the controller. The vacuum pump and air inlet solenoid valve are enabled when the iCDU has power. They can operate while the pump is in OFF mode, if the setting on the Continuous Run is set to ON on the Vacuum Pump screen.
Vacuum pressure is monitored by the vacuum pressure sensor. If the pressure is too high, the pump is enabled until pressure is reduced to the cut-out pressure. If the pressure is too low, the air inlet valve is enabled until pressure is increased to the cut-out pressure.
Differential pressure Differential pressure from the vacuum tank to the pump and IT water outlet is maintained by a VFD-driven water pump. The differential pressure is calculated from the vacuum pressure sensor value subtracted from the pump and IT supply outlet water pressure. A PI loop attempts to maintain the differential pressure to the set point. If the differential is below the set point, the VFD increases the pump speed. If the differential is above the set point, the VFD decreases the pump speed.
Pump and IT supply out water temperature Supply temperature to the IT loop is maintained by modulating a water regulation valve on the facility side. A PI loop attempts to maintain the temperature at a set point. If the temperature is below the set point, the facility valve is modulated closed. If the temperature is above the set point, then the facility valve is modulated open. The facility valve is fully open when the unit is off.
iCDU 84
iCDU maintenance Water testing and maintenance schedule
Water quality is essential for ensuring normal iCDU operation. The Apollo 8000 System requires that the water loop be maintained and monitored every three months by qualified personnel. For all IT or secondary water quality-related issues, contact Hewlett Packard Enterprise Service.
Condensation management 85
Condensation management
Condensation management
CAUTION: To reduce the risk of condensation and avoid damage to the equipment when operating with chilled water, keep the rack doors closed during operation.
Condensation should not be a concern, unless the Apollo 8000 System is operating with the doors open while running water temperatures below recommended dewpoint parameters.
Frost damage To avoid frost damage, the water temperature must not fall below the minimum permissible temperature of +4°C (+39.2°F) at any point in the water cycle.
If the Apollo 8000 System must be shipped after previous running in normal operation mode, contact Hewlett Packard Enterprise Service to assess the risk of trapped water that might be subject to freezing during transportation.
Software and configuration utilities 86
Software and configuration utilities
HPE Insight Cluster Management Utility The Insight CMU is an efficient and robust hyperscale cluster lifecycle management framework and suite of tools for large Linux clusters. A simple graphical interface enables an at-a-glance view of the entire cluster across multiple metrics, provides frictionless scalable remote management and analysis, and allows rapid software provisioning to all system nodes. Insight CMU makes cluster management more user friendly, efficient, and error-free than if it were being managed by scripts, or on a node-by-node basis. Insight CMU is highly flexible and customizable, offers both GUI and CLI interfaces, and is used to deploy a range of software environments, from simple compute farms to highly customized, application-specific configurations.
For more information on Insight CMU features and links to technical documentation, QuickSpecs, and a product demo, see the Hewlett Packard Enterprise website (http://www.hpe.com/info/cmu).
To download the product, go to the Hewlett Packard Enterprise Software Depot (http://www.hpe.com/support/softwaredepot). Click HP Insight Management, then click Insight Cluster Management.
HPE Advanced Power Manager The APM utility module is a point of contact for HPE ProLiant Scalable System administration.
APM enables:
• Discovery of server rack components by topology
• Node-level power monitoring and control
• Easy node and identification through UID indicators
• Support of logging through standard syslog servers
NOTE: The use of the term server represents nodes, server trays, server nodes, and cartridges.
For more information on APM, including commands specific for Apollo 8000 System, see the Advanced Power Manager User Guide for Apollo 8000 (http://www.hpe.com/support/Apollo8000_APM_UG_en).
Troubleshooting 87
Troubleshooting
HPE Apollo 8000 System Troubleshooting Rack and mechanical issues
Issue or Symptom Possible Cause(s) Troubleshooting The rack doors have opened automatically.
Could indicate: • High temperature event • Loss of power to the
magnetic door locks • Excessively high fan
speeds
1 Verify that the rack is receiving power. 2 Check the APM for an error indicating high
temperature. 3 Verify the magnetic door locks are
receiving power.
The rack doors will not fully close.
Could indicate: • Obstruction of door path
due to cables, servers, or other rack IT components
• Bent or un-square door frame
• Incorrect installation of brackets
• Unsecured gasketing • Loss of power to the
magnetic door locks
1 Verify all cables are properly secured using tie-wraps and cable brackets. Ensure adequate clearances between cables and the rack doors.
2 Verify all servers and IT components are fully seated and not protruding from the rack.
3 Check the installation of rear door brackets and ensure adequate clearance between the brackets and the rear rack doors.
4 Visually inspect the rack doors to make sure they have not been damaged.
5 Verify the magnetic locks are receiving power.
The doors will not open, or are hard to open.
Could indicate: • A screw holding the door
closed during installation was not removed.
• The magnetic locks are too strong.
• Obstruction from the outside of the rack, such as a neighboring rack
• Obstruction from the inside of the rack, such as a cable
1 Verify all screws have been removed from the doors upon completing rack installation.
2 Verify there is the correct amount of overlap between the door magnet and the striker plate, and the alignment creates the right amount of tension to hold the door closed.
3 Verify there is nothing obstructing or overlapping the door on the outside of the rack.
4 Verify there are no cables pinched between the rack door.
There is air leaking from the rack door.
Could indicate: • Normal air flow from the
rack, as long as not excessive
• Weakened or missing gasketing
1 Verify the gasketing is installed securely. 2 Replace any missing or weakened
gasketing.
There is gasketing material falling off the inside of the rack door.
Could indicate: • Normal wear and tear • Cables or other IT
components snagging and pulling on the gasketing
1 Verify there is nothing that caught on the gasketing and caused it to fall off the rack door.
2 Replace the gasketing.
Troubleshooting 88
Issue or Symptom Possible Cause(s) Troubleshooting The server cannot be installed.
Could indicate: • Various issues, depending
on which point of installation causes the problem.
1 Verify the server is correctly oriented. 2 Verify there is no obstruction in the server
slot, and nothing has previously been installed.
3 Verify the thermal busbar is fully retracted and the red interlock bracket is not blocking the guide channel.
4 Verify the bracket for the interlock device is correctly installed.
5 Check the thermal busbar on the rear of the server for scratched metal or damaged interface.
The server installation is inhibited from the start.
Could indicate: • The server slot might be
too small. • There might be something
protruding from the server. • There might be screws
misplaced in the wall rails.
1 Try installing the server in an adjacent server slot.
2 Verify there are no protruding screws from the server.
3 Verify the server lid is fully installed. 4 Verify there are no screws or other
obstructions in the server slot, including on the walls and pre-installed rails.
The server installation begins normally, but then gets stuck halfway.
Could indicate: • The heatsink screws are
not fully installed, and drag or catch on the server slot.
• The interlock bracket is broken or lose, and engages early, getting caught on the server slot.
1 Verify there are no protruding screws from the server.
2 Verify the interlock bracket is working correctly, and has not engaged with anything other than the thermal busbar.
The server does not fully install in the f8000 rack.
Could indicate: • Damaged or misaligned
connectors on the rear of the server.
• Broken server locking lever
1 Check the rear of the server for signs of damage.
2 Check the backplane in the rack for signs of damage.
3 Verify the server handle latches and locking levers are fully open.
4 Install another server into the same server slot to verify the issue is not with the server slot.
5 Install the server into a different slot to verify the issue is not with the server.
The server handle latches or locking levers do not stay closed.
Could indicate: • Damaged hardware,
including handle latches or locking levers
1 Check the functionality of the hardware, making sure they handle latch and locking lever open and close easily.
2 If the hardware is broken, call Hewlett Packard Enterprise Service.
Power issues
Issue or Symptom Possible Cause Troubleshooting The LED on one of the redundant power supplies is not illuminated.
Could indicate: • Defective power supply • Damaged power supply
connector • Power supply not fully
seated
1 Reseat the power supply. 2 Verify all high voltage DC connections on
the power shelf are working. 3 Replace the power supply.
Troubleshooting 89
Issue or Symptom Possible Cause Troubleshooting The LED on one of the Titan power supplies is not illuminated.
Could indicate: • Defective power supply • Damaged power supply
connector • Power supply not fully
seated
1 Remove the Titan power supply long enough for the associated fan to stop running (approximately 15 seconds).
2 Reinstall the power supply. 3 If the LED on the power supply does not
illuminate green, replace the Titan power supply.
The LED on adjacent Titan power supplies within the same power shelf are not illuminated.
Could indicate: • EPO has tripped • Defective power supply • AC input power to the
power shelf is not connected
1 Verify the AC input to the rack is connected properly and power is feeding into the rack.
2 Verify both power cords feeding the power shelf are connected.
3 Verify the breaker status on the rear of the power shelf. If a breaker has tripped, reset the breaker by pushing the post back in.
4 Replace the Titan power supplies. The LED on all four Titan power supplies within the power shelf are not illuminated.
Could indicate: • EPO has tripped • Defective power supply • AC input power to the
power shelf is not connected
1 Verify the AC input to the rack is connected properly and power is feeding into the rack.
2 Verify both power cords feeding the power shelf are connected.
3 Verify the breaker status on the rear of the power shelf. If a breaker has tripped, reset the breaker by pushing the post back in.
4 Replace the Titan power supplies. A single server tray is not receiving power.
Could indicate: • Poor connection • Broken connectors on the
server tray • Defective server tray
1 Re-seat the server tray. 2 Verify the server slot is turned on via the
APM connection. 3 Install the server tray into a different slot. 4 Replace the server tray.
A cell of server trays are not receiving power.
Could indicate: • Poor connection between
the management module and the power shelf
• A breaker tripped on the power shelf
• One or both of the redundant power supplies in the associated management module assembly are defective
1 Verify a green LED status on the redundant power supplies in the associated management module assembly.
2 Verify the breaker status on the rear of the power shelf. If a breaker has tripped, reset the breaker by pushing the post back in.
3 If the LED on the power supply does not illuminate green, replace the power supply.
The power shelf is not communicating or sending data to the APM.
Could indicate: • Poor connection • Defective component,
such as the Atlas module or Janis communication card
1 Verify all communication connections between the power shelf and the utility shelf.
2 Replace the Atlas module. 3 Replace the Janis communication card.
The power shelf is communicating to the APM, but it is missing specific data information.
Could indicate: • Poor connection • Defective component,
such as the input meter or Titan power supply
1 Verify all communication connections between the power shelf and utility shelf.
2 If the AC input data is missing, replace the input meter.
3 If the Titan data is missing, replace the Titan power supply.
Troubleshooting 90
Temperature and cooling issues
Issue or Symptom Possible Cause Troubleshooting There is an over-temperature alert error on one server.
Could indicate: • An issue with the server • Stress on the server • Seating of the heatsink • An issue with the thermal
busbar for that server
1 Install a different server in the same slot to verify there is no issue with the thermal busbar.
2 Follow the instructions in the server documentation to resolve the over temperature issue.
3 Call Hewlett Packard Enterprise service. There are over-temperature alert errors on multiple servers within the same cell.
Could indicate: • A bad fan within the
management module assembly
• An issue with the water wall
• A clogged filter
1 Check for errors in the APM indicating fan or power failures.
2 Check the APM for HEX and general cell status.
3 Check the sensor and inlet temperature on the rack controller board.
4 Check the power usage compared to the temperature. Determine the temperature delta.
5 Verify that water is flowing into the rack via the flow sensors.
6 Call Hewlett Packard Enterprise service. There are over-temperature alert errors on multiple servers within the half rack.
Could indicate: • A bad fan within the
management module assembly
• An issue with the heat exchanger
• A clogged filter
1 Check for errors in the APM indicating fan or power failures.
2 Check the APM for HEX and general cell status.
3 Check the sensor and inlet temperature on the rack controller board.
4 Check the power usage compared to the temperature. Determine the temperature delta.
5 Verify that water is flowing into the rack via the flow sensors.
6 Call Hewlett Packard Enterprise service. There are over-temperature alert errors on multiple servers throughout the rack.
Could indicate: • Water flow problems
within the secondary water loop
• A clogged filter
1 Check for errors in the APM indicating fan or power failures.
2 Check the APM for HEX and general cell status.
3 Check the sensor and inlet temperature on the rack controller board.
4 Check the power usage compared to the temperature. Determine the temperature delta.
5 Verify that water is flowing into the rack via the flow sensors.
6 Call Hewlett Packard Enterprise service. There are over-temperature alert errors on multiple servers throughout the cluster.
Could indicate: • An issue with the iCDU • A clogged filter
1 Check for errors in the APM indicating fan or power failures.
2 Check the APM for HEX and general cell status.
3 Check the sensor and inlet temperature on the rack controller board.
4 Check the power usage compared to the temperature. Determine the temperature delta.
5 Verify that water is flowing into the rack via the flow sensors.
6 Call Hewlett Packard Enterprise service.
Troubleshooting 91
Issue or Symptom Possible Cause Troubleshooting There is a high rack water inlet temperature alert error within the f8000 rack.
Could indicate: • Improper valve position • An issue with the iCDU • A clogged filter
1 Check for water temperature alarms in the APM or CMU interface.
2 Verify that water is flowing into the rack via the flow sensors.
3 Check the rack isolation valve positions. 4 Check the water wall valve positions. 5 Check the health of the iCDU. 6 Check the differential pressure to
determine if a filter needs to be replaced. 7 Call Hewlett Packard Enterprise service.
There is a low water flow alert error.
Could indicate: • Improper valve position • An issue with the iCDU • A clogged filter
1 Verify that water is flowing into the rack via the flow sensors.
2 Check the rack isolation valve positions. 3 Check the water wall valve positions. 4 Check the health of the iCDU. 5 Check the differential pressure to
determine if a filter needs to be replaced. 6 Call Hewlett Packard Enterprise service.
There is a high water temperature alert error on the iCDU rack.
Could indicate: • High facility water
temperature • Low facility water pressure • An issue with the iCDU
1 Check for water temperature alarms in the APM or CMU interface.
2 Check the rack water flow. 3 Check the facility water temperature in the
CMU interface or using the CMU controller.
4 Check the facility water differential pressure.
5 Verify the iCDU is running correctly. 6 Call Hewlett Packard Enterprise service.
There is hot air coming out of the rack.
Could indicate: • The doors are not properly
sealed. • There is gasketing
missing from a portion of the rack.
1 Determine where the hot air is coming from.
2 Verify all gasketing is in place. 3 Call Hewlett Packard Enterprise service.
Communication issues
Issue or Sympton Possible Cause Troubleshooting The utility module is not receiving information from a compute cell.
Could indicate: • Poor connection between
the utility module and the compute cell
• Damaged display port cable
1 Check that the display port cables are connected and there is no signs of physical damage.
2 Replace the display port cable.
The utility module is receiving incorrect or incomplete information from the compute cells.
Could indicate: • The display port cables
have been switched between compute cells.
• Poor connection between the utility module and the compute cell
• Damaged display port cable
1 Check that the correct compute cell display port cable is connected to the correct port on the utility module.
2 Verify there are no signs of physical damage to the display port cable.
Troubleshooting 92
Issue or Sympton Possible Cause Troubleshooting The utility module is not receiving information from the power shelf.
Could indicate: • Poor connection between
the utility shelf and the compute cell
• Damaged display port cable
1 Check the serial cable connection between the power shelf and the utility module, and there is no sign of physical damage.
2 Call Hewlett Packard Enterprise Service.
The utility module is receiving incorrect or incomplete information from the power shelf.
Could indicate: • The serial cables have
been switched between power shelves
• Poor connection between the utility module and the power shelf
• Damaged serial cable
1 Check the serial cable connection between the power shelf and the utility module, and there is no sign of physical damage.
2 Call Hewlett Packard Enterprise Service.
The utility module is not receiving information from the thermal controller.
Could indicate: • Poor connection between
the utility module and the thermal controller
• Damaged display port cable
1 Check that the display port cable is connected to the correct port on the utility module.
2 Verify there are no signs of physical damage to the display port cable.
I cannot log in via direct connection to the utility module.
Could indicate: • Disconnected Ethernet
cable from the utility module
• Poor APM module within the utility module
• Ethernet network is incorrectly configured
1 Check the username and password. 2 Check the configuration of the utility
module. 3 Check the connection between the utility
module and the management switch. 4 Might need to replace the utility module.
Call Hewlett Packard Enterprise Service.
I cannot log in remotely to the utility module.
Could indicate: • Disconnected Ethernet
cable from the utility module
• Poor APM module within the utility module
• Ethernet network is incorrectly configured
1 Check the username and password. 2 Check the configuration of the utility
module. 3 Check the connection between the utility
moduleand the management switch. 4 Might need to replace the utility module.
Call Hewlett Packard Enterprise Service.
I cannot communicate with the utility module in either the f8000 rack IT rack or the iCDU rack.
Could indicate: • Ethernet network is
incorrectly configured
1 Check all of the cables. 2 Check the server configuration. 3 Call Hewlett Packard Enterprise Service.
I cannot communicate with any devices through iLO.
Could indicate: • Poor connection • Damaged iLO aggregator
1 Check the connection between the management switch and the iLO aggregator pca in the 2U shelf.
2 Check the display port connection at each management module cell.
3 Might need to replace the iLO aggregator. Call Hewlett Packard Enterprise Service.
There are no link lights between the server and the management switch.
Could indicate: • Poor Ethernet cable • Server is incorrectly
configured • Management switch is
incorrectly configured • There is no power
1 Check Ethernet cable. 2 Check server configuration. 3 Verify the adapter is enabled. 4 Verify power to the management switch. 5 Call Hewlett Packard Enterprise Service.
Troubleshooting 93
iCDU issues
Issue or Sympton Possible Cause Troubleshooting The red light on the iCDU is illuminated.
An alarm condition on the iCDU
1 Check the iCDU display for an error message and additional information.
2 Call Hewlett Packard Enterprise Service. There are loud or strange noises coming from the iCDU.
Could indicate: • Faulty pump
1 Check the iCDU display for an error message and additional information.
2 Power down the iCDU. If you have redundancy, power down the iCDU. If you do not have redundancy, power down the f8000 rack first, and then power down the iCDU.
3 Call Hewlett Packard Enterprise Service. The iCDU has stopped running.
Could indicate: • A critical alarm has
powered down the iCDU. • An EPO event
1 Check the iCDU display for an error message and additional information.
2 Check any critical alarms on the active alarm list.
3 Call Hewlett Packard Enterprise Service. The iCDU will not power on. Could indicate:
• A pending critical alarm is still considered active.
• A loss of power to the iCDU rack
1 Verify that all active critical alarms have been cleared.
2 Verify the iCDU rack has power. 3 Call Hewlett Packard Enterprise Service.
The iCDU has stopped communicating with APM or the BMS.
Could indicate: • A network problem • Hardware issues with the
APM, network switch, or the pcoWEB card
1 Verify that the network is working correctly. 2 While on the same network with the same
subnet, ping the iCDU via the IP address. 3 Verify the network switch is working
correctly. 4 See the iCDU Communication Setup
section of the Apollo 8000 System User Guide.
5 Check the pcoWEB card functionality.* Connect directly, bypassing the network switch. If you can connect successfully, reboot the card.
Looking at the site glass of the iCDU, the water level appears low.
Could indicate: • Low water in the tank
1 If auto-fill is enabled, verify that it is working correctly.
2 If auto-fill is not enabled, fill the iCDU and check the chemistry.
3 Call Hewlett Packard Enterprise Service. There is water coming from the iCDU.
Could indicate: • A leak • High water pressure • Excess water
1 Power down the iCDU. If you have redundancy, power down the iCDU. If you do not have redundancy, power down the f8000 rack first, and then power down the iCDU.
2 Call Hewlett Packard Enterprise Service.
*An administrative username and password are required to connect directly to the pcoWEB card.
iCDU alarm troubleshooting Critical alarms
When a critical alarm is triggered, the iCDU shuts down immediately.
Troubleshooting 94
Alarm text Description Possible cause Action VFD VFD general alarms VFD generated alarms Check VFD display for alarm
details and then correct. Low Water Tank water is low (low
and mid level sensors detected).
Loss of water (through leakage) or imbalance of water between iCDUs
Check for water leakage.
iCDU Isolation Valve Failed to Open
— iCDU isolation valve did not open from command.
Verify the valve is functional and check wire connectivity.
VFD Current Overload VFD overload condition detected when VFD current exceeds motor Full Load Amperes (FLA) x multiplier (1.1) for at least 20 seconds
Possible pump/motor failure or overloading conditions
Verify pump equipment, and then make corrections.
Intermittent Low Differential Pressure on IT
Intermittent low pump discharge pressure condition for a least 5 cycles every minute
Potential small air leakage
Check for air leakage, and then make corrections.
Sustained Low Differential Pressure on IT
Sustained low pump discharge pressure condition for at least 5 seconds
Potential faulty pump Verify pump equipment, and then make corrections.
Warning alarms
When a warning alarm is triggered on an iCDU in lead/lag mode, operation is rotated to the standby iCDU unless the standby iCDU is experiencing the same or a more severe alarm than the lead iCDU. In parallel mode, the iCDU continues to operate.
Alarm Description Possible cause Action Leak Detection A water leak is
detected within the iCDU.
Water leak sensor detected
Check for water leakage.
Tank Fill (Mid-level Alert) Pre-low water condition (mid level sensor detected)
Loss of water (through leakage) or imbalanced amount of water between iCDUs
Check for leak or water level on iCDUs.
High Discharge Press IT water pressure at iCDU pump discharge exceeds threshold.
Clogged secondary water line
Check for clogged IT water line.
High IT (Supply) HEX Outlet Temperature
IT supply water (to racks) temperature exceeds threshold.
Facility water temperature exceeds threshold or excessive IT load.
Verify if any of the conditions exist, and then correct them.
Low Suction/Vacuum Pressure
Vacuum pressure in secondary water loop is below threshold.
Potential faulty vacuum pump or air leakage in line
Verify if any of the conditions exist, and then correct them.
Vacuum pump on and vacuum pressure greater than cut in for more than 20 minutes
Vacuum pump continuous runtime exceeds threshold.
Potential water leak condition
Check for water leakage.
Informational alarms
These alarms are informational only and do not alter the operation of the iCDU.
Alarm Description Possible cause Action Lead/Lag Rotation Failure
Lead iCDU failed attempt to rotate to lag iCDU.
Lag iCDU not available or also in similar or higher alarm status.
Check for probable causes which prevented iCDU from rotating.
Troubleshooting 95
Alarm Description Possible cause Action High Water Tank water level is high
(high-level sensor detected).
Imbalanced amount of water between iCDUs
Stop iCDU as part of maintenance to balance water level.
iCDU Isolation Valve Failed to Close
iCDU isolation valve did not close from command.
Malfunctioning ISO valve or valve input voltage wire is disconnected.
Verify valve is functional or wire connectivity.
VFD Comm Failure Loss of communication between the iCDU controller and the VFD. VFD can still control the pump via analog signal. VFD data is not available for monitoring.
Invalid/incorrect communication settings for VFD
Correct the communication settings for the VFD.
pLAN communication error
Loss of communication to the other iCDU in the cluster
pLAN cable disconnected, incorrect pLAN communication settings, or partner iCDU is off.
Verify pLAN cable connection or if other iCDU has power, and verify pLAN communication settings.
High facility Inlet Temperature
Facility supply water (to iCDU) temperature exceeds the threshold.
Facility supply water temperature is too high.
Reduce the temperature of the facility supply water.
Vacuum Pump Runtime Alert
Accumulated vacuum pump runtime exceeds threshold.
Potential water leak condition
Check for water leakage.
Low Facility Heat Exchanger Differential Pressure
Differential pressure on facility water side is below threshold.
Potential chiller pump failure or facility water leakage
Verify facility water condition and correct.
Using HPE APM for sensor monitoring The APM serial console can be used to monitor and control various devices connected to the controller.
Troubleshooting 96
Using the help command displays various information for the sensors.
The controls indicate the current status for each of the sensors displayed. However, the status of the sensors is useful in troubleshooting various errors.
NOTE: If the status displays Cable Not Detected, there is a connection issue with the particular sensor. Either the sensor is not properly connected or there is something wrong with the cable itself. Verify the cable connection.
Sensor/Controls Status Troubleshooting Leak Detector No Leaks
Leaks If Leaks: Visually inspect the rack for leaks.
Door Sensor • 0—Front door • 1—Rear door
Closed Open
If Open: 1 Verify that the door is actually open. 2 If closed, verify there is a tight connection
between the door and the sensor. 3 Verify the status of the Door Lock is set to
Closed. Door Lock • 0—Front door • 1—Rear door
Closed Open
If Open: Set to Closed.
Valve Sensor • 0—Supply • 1—Return
Open Closed
If Closed: Verify the valve sensor cable is connected.
Valve Control • 0—Supply • 1—Return
Open Closed
—
Troubleshooting 97
Sensor/Controls Status Troubleshooting RTD Temperature • 0—Supply • 1—Intermediate • 2—Return
Temperature in °Celsius Temp Out Of Range
If Temp Out Of Range:
1 Verify the sensor cable is connected. 2 Check the temperature of the supply water
from the iCDU.
H2O Pressure Pressure in nU Pressure Out Of Range
If Pressure Out Of Range:
1 Verify the sensor cable is connected. 2 Check the pressure gauges on the iCDU.
Temp-Humid-board Temp in °Celsius, % Relative Humidity Humidity Out Of Range
If Humidity Out Of Range: Verify the sensor cable is connected.
1-Wire Temp • 0—HEX inlet temp • 1—HEX outlet
temp • 6—HEX inlet temp • 7—HEX outlet
temp
Temperature in °Celsius Temp Out Of Range
If Temp Out Of Range: Verify the sensor cable is connected.
ARPO Running Stopped
If Stopped: ARPO has tripped and the Apollo 8000 System has shut down. You will not see this error displayed, for the power will be off.
HEX Power Supply Present- OK Not Present- Not OK
If Not Present-Not OK:
1 Verify the green LED on the power supply is illuminated.
2 Verify the power supply is connected. 3 Restart the power supply. 4 Replace the power supply.
Specifications 98
Specifications
Physical specifications The table below lists the approximate physical specifications of a single HPE f8000 rack or 8000 iCDU Rack.
f8000 rack
Parameter Packaged system (as shipped on pallet)
Unpackaged system (off pallet, unwrapped)
Height 2575 mm (102 in) 2382 mm (94 in) Width 1016 mm (40 in) 607 mm (24 in) Depth 1676 mm (66 in) 1427 mm (57 in) Weight1 1522 kg (3356 lb)2 1225 kg (2700 lb)3 Height top assembly 1253 mm (50 in) 1060 mm (42 in) Height mid and bottom assembly
1555 mm (62 in) 1362 mm (54 in)
Weight top assembly1 Up to 642 kg (1415 lb)2 Up to 413 kg (910 lb)3 Weight mid and bottom assembly1
880 kg (1941 lb)2 693 kg (1528 lb)3
1There can be a +/- 5% difference in weight 2Weight for a completely packaged system with no server trays installed 3Weight for an unpackaged system with no server trays installed
8000 iCDU Rack
Parameter Packaged system (as shipped on pallet)
Unpackaged system (off pallet, unwrapped)
Height 2575 mm (102 in) 2382 mm (94 in) Width 1016 mm (40 in) 607 mm (24 in) Depth 1676 mm (66 in) 1427 mm (57 in) Weight1 1006 kg (2218 lb)2 896 kg (1974 lb)3 Height top assembly 1253 mm (50 in) 1060 mm (42 in) Height mid and bottom assembly
1555 mm (62 in) 1362 mm (54 in)
Weight top assembly1 383 kg (844 lb)2 273 kg (600 lb)3 Weight mid and bottom assembly1
734 kg (1618 lb)2 624 kg (1374 lb)3
1There can be a +/- 5% difference in weight 2Weight for a completely packaged system with no hose kits or IT equipment installed 3Weight for an unpackaged system with no hose kits or IT equipment installed
Electrical specifications The following table lists the electrical specifications for the f8000 rack.
Specifications 99
Parameter Value Comments Operating Voltage: Minimum Maximum
220/380VAC, 3 phase with N + PE 277/480VAC, 3 phase with N + PE
There are two styles of input power meters: • North American—Supports 480VAC • International—Supports 380VAC to
415VAC AC line frequency 50/60 Hz — AC line phase Three True 3-phase input power conversion for
inherent phase current balancing Rated input current • 380VAC source—415VAC 32A
per input meter • 480VAC source— 24A per input
meter
Per line cord, two cords per power shelf, two shelves per IT rack
Maximum inrush current Less than 2 times the peak full load current at any line condition
Per line cord
Dropout/hold-up time at minimum line voltage
20 ms —
Circuit breaker rating • 380-415VAC source —32A • 480VAC source—30A
Per cord
Power factor > 0.98 At all loads above 20% Ground leakage current < 3.0mA Per cord Maximum power consumption (full IT rack)
79,775W capable power shelf This is the maximum power at the input of the HVDC power shelves. There are two power shelves per rack, each with dual 3-phase AC feeds.
Power cords (included with power shelves)
220/380 -240/415VAC 277/480VAC
3 meter, 8AWG, 5-wire with IEC 60309-32 plug 3 meter, 8AWG, 5-wire with NEMA L22-30P plug
The following table lists the electrical specifications for the 8000 iCDU Rack.
Parameter Value Comments Operating Voltage: Minimum Maximum
220/380VAC, 3 phase with N + PE 288/480VAC, 3 phase with N + PE
There are two CDU versions: • North American—Supports 480VAC • International—Supports 380VAC to
415VAC AC line frequency 50/60 Hz — AC line phase Three phase — Rated input current • 480VAC source—7.1 A
• 380VAC to 415VAC source—9.5 A
Per line cord
Maximum inrush current 580 A peak-to-peak available Per line cord Circuit breaker rating 15 A Per cord Power factor Typical value = 0.80 At all loads Maximum power consumption
5.7 kVA Steady state
Power cords 30A • 480VAC source—NEMA L22-30P watertight connector
• 380VAC to 415VAC source—IEC 60309 watertight connector
Maximum power cords per rack
4 —
Specifications 100
Thermal and air flow performance
Maximum thermal and air flow performance parameters
Single rack Dual rack (Specification per rack)
Air temperature—inlet to rack-mounted components
25ºC (68ºF) 25ºC (68ºF)
Chilled water temperature 7º–15°C (45º–57°F) 7º–15°C (45º–57°F) Total rack-mounted component air flow
2,600 cfm or less at 0 or more pressure drop across the rack-mounted components
1,300 cfm or less at 0 or more pressure drop across the rack-mounted components
Chilled water flow rate 15.5 gal/min (60 l/min) 7.9 gal/min (30 l/min) Chilled water pressure differential at flow needed to meet thermal specifications
1.0 bar delta pressure 1.0 bar delta pressure
Heat lost to room Approximately 10% maximum depending on the MCS settings and room temperature
Approximately 10% maximum depending on the MCS settings and room temperature
Server heat load 35 kW maximum 17.5 kW maximum
Environmental specifications
Features Specifications Operating temperature 5ºC to 35ºC (41ºF to 95ºF) Non-operating temperature 0ºC to 60ºC (32ºF to 140ºF) Transit temperature -30ºC to 60ºC (-22ºF to 140ºF), up to 72 hours Storage temperature -20ºC to 60ºC (-4ºF to 140ºF) Operating humidity 20 to 80% relative humidity (non-condensing) Non-operating humidity 5 to 95% relative humidity (non-condensing) Operating altitude -76.2 to 3,048 m (-250 to 10,000 ft) Non-operating altitude -76.2 to 9,144 m (-250 to 30,000 ft)
Safety considerations 101
Safety considerations
Important safety information
IMPORTANT: Before servicing this product, read the Important Safety Information document provided with the server.
Symbols on equipment
This symbol indicates the presence of hazardous energy circuits or electric shock hazards. Refer all servicing to qualified personnel. WARNING: To reduce the risk of injury from electric shock hazards, do not open this enclosure. Refer all maintenance, upgrades, and servicing to qualified personnel.
This symbol indicates the presence of electric shock hazards. The area contains no user or field serviceable parts. Do not open for any reason. WARNING: To reduce the risk of injury from electric shock hazards, do not open this enclosure.
This symbol on an RJ-45 receptacle indicates a network interface connection. WARNING: To reduce the risk of electric shock, fire, or damage to the equipment, do not plug telephone or telecommunications connectors into this receptacle.
This symbol indicates the presence of a hot surface or hot component. If this surface is contacted, the potential for injury exists. WARNING: To reduce the risk of injury from a hot component, allow the surface to cool before touching.
217.7 kg
480 lb
This symbol indicates that the component exceeds the recommended weight for one individual to handle safely. WARNING: To reduce the risk of personal injury or damage to the equipment, observe local occupational health and safety requirements and guidelines for manual material handling.
These symbols, on power supplies or systems, indicate that the equipment is supplied by multiple sources of power. WARNING: To reduce the risk of injury from electric shock, remove all power cords to completely disconnect power from the system.
Warnings and cautions
WARNING: This equipment must be installed in a restricted access area. Only authorized technicians trained by Hewlett Packard Enterprise should attempt to repair this equipment. All troubleshooting and repair procedures are detailed to allow only subassembly/module-level repair. Because of the complexity of the individual boards and subassemblies, no one should attempt to make repairs at the component level or to make modifications to any printed wiring board. Improper repairs can create a safety hazard..
Safety considerations 102
WARNING: To reduce the risk of fire, that the racks must be installed only on a concrete or other non-combustible floor.
WARNING: To reduce the risk of personal injury or damage to the equipment, be sure that: • The leveling feet are extended to the floor. • The full weight of the rack rests on the leveling feet. • The racks are coupled together. • Single rack installations are adequately secured to the building structure.
WARNING: To reduce the risk of personal injury, fire, or damage to the equipment, do not overload the AC supply branch circuit that provides power to the rack. Consult the electrical authority having jurisdiction over wiring and installation requirements of your facility. The branch circuit feeding each power cord connection must be rated as follows: • Units with NEMA L22-30 Plugs: 30A Circuit Breakers max • Units with IEC 60309-1 Plugs: 32A Circuit Breakers max.
WARNING: To reduce the risk of electric shock or damage to the equipment: • Do not disable the power cord grounding plug. The grounding plug is an important safety
feature. • Plug the power cord into a grounded (earthed) electrical outlet that is easily accessible at all
times. • Unplug the power cord from the wall outlet to disconnect power to the equipment. • Do not route the power cord where it can be walked on or pinched by items placed against
it. Pay particular attention to the plug, electrical outlet, and the point where the cord extends from the server.
WARNING: To reduce the risk of injury from electric shock, remove all power cords to completely disconnect power from the system. • Each rack has two or more power supply cords. Removing any single source of power does
not necessarily remove power from any portion of the system. When performing any service other than hot-plug module replacement, you must completely disconnect all power to that portion of the system.
• When performing service procedures on the rack, disconnect all power cords from the outlets before servicing.
WARNING: A risk of electric shock from high leakage current exists. Before connecting the AC supply to the power enclosures, be sure that the electrical outlets are properly grounded (earthed).
WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the internal system components to cool before touching them.
Support and other resources 103
Support and other resources
Accessing Hewlett Packard Enterprise Support • For live assistance, go to the Contact Hewlett Packard Enterprise Worldwide website
(http://www.hpe.com/assistance).
• To access documentation and support services, go to the Hewlett Packard Enterprise Support Center website (http://www.hpe.com/support/hpesc).
Information to collect • Technical support registration number (if applicable)
• Product name, model or version, and serial number
• Operating system name and version
• Firmware version
• Error messages
• Product-specific reports and logs
• Add-on products or components
• Third-party products or components
Accessing updates • Some software products provide a mechanism for accessing software updates through the product
interface. Review your product documentation to identify the recommended software update method.
• To download product updates, go to either of the following: o Hewlett Packard Enterprise Support Center Get connected with updates page
(http://www.hpe.com/support/e-updates) o Software Depot website (http://www.hpe.com/support/softwaredepot)
• To view and update your entitlements, and to link your contracts and warranties with your profile, go to the Hewlett Packard Enterprise Support Center More Information on Access to Support Materials page (http://www.hpe.com/support/AccessToSupportMaterials).
IMPORTANT: Access to some updates might require product entitlement when accessed through the Hewlett Packard Enterprise Support Center. You must have an HP Passport set up with relevant entitlements.
Websites • Hewlett Packard Enterprise Information Library (http://www.hpe.com/info/enterprise/docs)
• Hewlett Packard Enterprise Support Center (http://www.hpe.com/support/hpesc)
• Contact Hewlett Packard Enterprise Worldwide (http://www.hpe.com/assistance)
Support and other resources 104
• Subscription Service/Support Alerts (http://www.hpe.com/support/e-updates)
• Software Depot (http://www.hpe.com/support/softwaredepot)
• Customer Self Repair (http://www.hpe.com/support/selfrepair)
• Insight Remote Support (http://www.hpe.com/info/insightremotesupport/docs)
• Serviceguard Solutions for HP-UX (http://www.hpe.com/info/hpux-serviceguard-docs)
• Single Point of Connectivity Knowledge (SPOCK) Storage compatibility matrix (http://www.hpe.com/storage/spock)
• Storage white papers and analyst reports (http://www.hpe.com/storage/whitepapers)
Customer Self Repair Hewlett Packard Enterprise products are designed with many Customer Self Repair (CSR) parts to minimize repair time and allow for greater flexibility in performing defective parts replacement. If during the diagnosis period Hewlett Packard Enterprise (or Hewlett Packard Enterprise service providers or service partners) identifies that the repair can be accomplished by the use of a CSR part, Hewlett Packard Enterprise will ship that part directly to you for replacement. There are two categories of CSR parts:
• Mandatory—Parts for which customer self repair is mandatory. If you request Hewlett Packard Enterprise to replace these parts, you will be charged for the travel and labor costs of this service.
• Optional—Parts for which customer self repair is optional. These parts are also designed for customer self repair. If, however, you require that Hewlett Packard Enterprise replace them for you, there may or may not be additional charges, depending on the type of warranty service designated for your product.
NOTE: Some Hewlett Packard Enterprise parts are not designed for customer self repair. In order to satisfy the customer warranty, Hewlett Packard Enterprise requires that an authorized service provider replace the part. These parts are identified as "No" in the Illustrated Parts Catalog.
Based on availability and where geography permits, CSR parts will be shipped for next business day delivery. Same day or four-hour delivery may be offered at an additional charge where geography permits. If assistance is required, you can call the Hewlett Packard Enterprise Support Center and a technician will help you over the telephone. Hewlett Packard Enterprise specifies in the materials shipped with a replacement CSR part whether a defective part must be returned to Hewlett Packard Enterprise. In cases where it is required to return the defective part to Hewlett Packard Enterprise, you must ship the defective part back to Hewlett Packard Enterprise within a defined period of time, normally five (5) business days. The defective part must be returned with the associated documentation in the provided shipping material. Failure to return the defective part may result in Hewlett Packard Enterprise billing you for the replacement. With a customer self repair, Hewlett Packard Enterprise will pay all shipping and part return costs and determine the courier/carrier to be used.
For more information about the Hewlett Packard Enterprise CSR program, contact your local service provider. For the North American program, go to the Hewlett Packard Enterprise CSR website (http://www.hpe.com/support/selfrepair).
Réparation par le client (CSR) Les produits Hewlett Packard Enterprise comportent de nombreuses pièces CSR (Customer Self Repair = réparation par le client) afin de minimiser les délais de réparation et faciliter le remplacement des pièces défectueuses. Si pendant la période de diagnostic, Hewlett Packard Enterprise (ou ses partenaires ou mainteneurs agréés) détermine que la réparation peut être effectuée à l'aide d'une pièce CSR, Hewlett Packard Enterprise vous l'envoie directement. Il existe deux catégories de pièces CSR :
Support and other resources 105
• Obligatoire—Pièces pour lesquelles la réparation par le client est obligatoire. Si vous demandez à Hewlett Packard Enterprise de remplacer ces pièces, les coûts de déplacement et main d'œuvre du service vous seront facturés.
• Facultatif—Pièces pour lesquelles la réparation par le client est facultative. Ces pièces sont également conçues pour permettre au client d'effectuer lui-même la réparation. Toutefois, si vous demandez à Hewlett Packard Enterprise de remplacer ces pièces, l'intervention peut ou non vous être facturée, selon le type de garantie applicable à votre produit.
REMARQUE: Certaines pièces Hewlett Packard Enterprise ne sont pas conçues pour permettre au client d'effectuer lui-même la réparation. Pour que la garantie puisse s'appliquer, Hewlett Packard Enterprise exige que le remplacement de la pièce soit effectué par un Mainteneur Agréé. Ces pièces sont identifiées par la mention "Non" dans le Catalogue illustré.
Les pièces CSR sont livrées le jour ouvré suivant, dans la limite des stocks disponibles et selon votre situation géographique. Si votre situation géographique le permet et que vous demandez une livraison le jour même ou dans les 4 heures, celle-ci vous sera facturée. Pour toute assistance, appelez le Centre d’assistance Hewlett Packard Enterprise pour qu’un technicien vous aide au téléphone. Dans les documents envoyés avec la pièce de rechange CSR, Hewlett Packard Enterprise précise s'il est nécessaire de lui retourner la pièce défectueuse. Si c'est le cas, vous devez le faire dans le délai indiqué, généralement cinq (5) jours ouvrés. La pièce et sa documentation doivent être retournées dans l'emballage fourni. Si vous ne retournez pas la pièce défectueuse, Hewlett Packard Enterprise se réserve le droit de vous facturer les coûts de remplacement. Dans le cas d'une pièce CSR, Hewlett Packard Enterprise supporte l'ensemble des frais d'expédition et de retour, et détermine la société de courses ou le transporteur à utiliser.
Pour plus d'informations sur le programme CSR de Hewlett Packard Enterprise, contactez votre Mainteneur Agrée local. Pour plus d'informations sur ce programme en Amérique du Nord, consultez le site Web Hewlett Packard Enterprise (http://www.hpe.com/support/selfrepair).
Riparazione da parte del cliente Per abbreviare i tempi di riparazione e garantire una maggiore flessibilità nella sostituzione di parti difettose, i prodotti Hewlett Packard Enterprise sono realizzati con numerosi componenti che possono essere riparati direttamente dal cliente (CSR, Customer Self Repair). Se in fase di diagnostica Hewlett Packard Enterprise (o un centro di servizi o di assistenza Hewlett Packard Enterprise) identifica il guasto come riparabile mediante un ricambio CSR, Hewlett Packard Enterprise lo spedirà direttamente al cliente per la sostituzione. Vi sono due categorie di parti CSR:
• Obbligatorie—Parti che devono essere necessariamente riparate dal cliente. Se il cliente ne affida la riparazione ad Hewlett Packard Enterprise, deve sostenere le spese di spedizione e di manodopera per il servizio.
• Opzionali—Parti la cui riparazione da parte del cliente è facoltativa. Si tratta comunque di componenti progettati per questo scopo. Se tuttavia il cliente ne richiede la sostituzione ad Hewlett Packard Enterprise, potrebbe dover sostenere spese addizionali a seconda del tipo di garanzia previsto per il prodotto.
NOTA: alcuni componenti Hewlett Packard Enterprise non sono progettati per la riparazione da parte del cliente. Per rispettare la garanzia, Hewlett Packard Enterprise richiede che queste parti siano sostituite da un centro di assistenza autorizzato. Tali parti sono identificate da un "No" nel Catalogo illustrato dei componenti.
In base alla disponibilità e alla località geografica, le parti CSR vengono spedite con consegna entro il giorno lavorativo seguente. La consegna nel giorno stesso o entro quattro ore è offerta con un supplemento di costo solo in alcune zone. In caso di necessità si può richiedere l'assistenza telefonica di un addetto del centro di supporto tecnico Hewlett Packard Enterprise. Nel materiale fornito con una parte di ricambio CSR, Hewlett Packard Enterprise specifica se il cliente deve restituire dei component. Qualora sia richiesta la resa ad Hewlett Packard Enterprise del componente difettoso, lo si deve spedire ad Hewlett Packard Enterprise entro un determinato periodo di tempo, generalmente cinque (5) giorni lavorativi. Il componente difettoso deve essere restituito con la documentazione associata nell'imballo di
Support and other resources 106
spedizione fornito. La mancata restituzione del componente può comportare la fatturazione del ricambio da parte di Hewlett Packard Enterprise. Nel caso di riparazione da parte del cliente, Hewlett Packard Enterprise sostiene tutte le spese di spedizione e resa e sceglie il corriere/vettore da utilizzare.
Per ulteriori informazioni sul programma CSR di Hewlett Packard Enterprise, contattare il centro di assistenza di zona. Per il programma in Nord America fare riferimento al sito Web (http://www.hpe.com/support/selfrepair).
Customer Self Repair Hewlett Packard Enterprise Produkte enthalten viele CSR-Teile (Customer Self Repair), um Reparaturzeiten zu minimieren und höhere Flexibilität beim Austausch defekter Bauteile zu ermöglichen. Wenn Hewlett Packard Enterprise (oder ein Hewlett Packard Enterprise Servicepartner) bei der Diagnose feststellt, dass das Produkt mithilfe eines CSR-Teils repariert werden kann, sendet Ihnen Hewlett Packard Enterprise dieses Bauteil zum Austausch direkt zu. CSR-Teile werden in zwei Kategorien unterteilt:
• Zwingend—Teile, für die das Customer Self Repair-Verfahren zwingend vorgegeben ist. Wenn Sie den Austausch dieser Teile von Hewlett Packard Enterprise vornehmen lassen, werden Ihnen die Anfahrt- und Arbeitskosten für diesen Service berechnet.
• Optional—Teile, für die das Customer Self Repair-Verfahren optional ist. Diese Teile sind auch für Customer Self Repair ausgelegt. Wenn Sie jedoch den Austausch dieser Teile von Hewlett Packard Enterprise vornehmen lassen möchten, können bei diesem Service je nach den für Ihr Produkt vorgesehenen Garantiebedingungen zusätzliche Kosten anfallen.
HINWEIS: Einige Hewlett Packard Enterprise Teile sind nicht für Customer Self Repair ausgelegt. Um den Garantieanspruch des Kunden zu erfüllen, muss das Teil von einem Hewlett Packard Enterprise Servicepartner ersetzt werden. Im illustrierten Teilekatalog sind diese Teile mit „No“ bzw. „Nein“ gekennzeichnet.
CSR-Teile werden abhängig von der Verfügbarkeit und vom Lieferziel am folgenden Geschäftstag geliefert. Für bestimmte Standorte ist eine Lieferung am selben Tag oder innerhalb von vier Stunden gegen einen Aufpreis verfügbar. Wenn Sie Hilfe benötigen, können Sie das Hewlett Packard Enterprise Support Center anrufen und sich von einem Mitarbeiter per Telefon helfen lassen. Den Materialien von Hewlett Packard Enterprise, die mit einem CSR-Ersatzteil geliefert werden, können Sie entnehmen, ob das defekte Teil an Hewlett Packard Enterprise zurückgeschickt werden muss. Wenn es erforderlich ist, das defekte Teil an Hewlett Packard Enterprise zurückzuschicken, müssen Sie dies innerhalb eines vorgegebenen Zeitraums tun, in der Regel innerhalb von fünf (5) Geschäftstagen. Das defekte Teil muss mit der zugehörigen Dokumentation in der Verpackung zurückgeschickt werden, die im Lieferumfang enthalten ist. Wenn Sie das defekte Teil nicht zurückschicken, kann Hewlett Packard Enterprise Ihnen das Ersatzteil in Rechnung stellen. Im Falle von Customer Self Repair kommt Hewlett Packard Enterprise für alle Kosten für die Lieferung und Rücksendung auf und bestimmt den Kurier-/Frachtdienst.
Weitere Informationen über das Hewlett Packard Enterprise Customer Self Repair Programm erhalten Sie von Ihrem Servicepartner vor Ort. Informationen über das CSR-Programm in Nordamerika finden Sie auf der Hewlett Packard Enterprise Website unter (http://www.hpe.com/support/selfrepair).
Reparaciones del propio cliente Los productos de Hewlett Packard Enterprise incluyen muchos componentes que el propio usuario puede reemplazar (Customer Self Repair, CSR) para minimizar el tiempo de reparación y ofrecer una mayor flexibilidad a la hora de realizar sustituciones de componentes defectuosos. Si, durante la fase de diagnóstico, Hewlett Packard Enterprise (o los proveedores o socios de servicio de Hewlett Packard Enterprise) identifica que una reparación puede llevarse a cabo mediante el uso de un componente CSR, Hewlett Packard Enterprise le enviará dicho componente directamente para que realice su sustitución. Los componentes CSR se clasifican en dos categorías:
Support and other resources 107
• Obligatorio—componentes cuya reparación por parte del usuario es obligatoria. Si solicita a Hewlett Packard Enterprise que realice la sustitución de estos componentes, tendrá que hacerse cargo de los gastos de desplazamiento y de mano de obra de dicho servicio.
• Opcional—componentes cuya reparación por parte del usuario es opcional. Estos componentes también están diseñados para que puedan ser reparados por el usuario. Sin embargo, si precisa que Hewlett Packard Enterprise realice su sustitución, puede o no conllevar costes adicionales, dependiendo del tipo de servicio de garantía correspondiente al producto.
NOTA: Algunos componentes de Hewlett Packard Enterprise no están diseñados para que puedan ser reparados por el usuario. Para que el usuario haga valer su garantía, Hewlett Packard Enterprise pone como condición que un proveedor de servicios autorizado realice la sustitución de estos componentes. Dichos componentes se identifican con la palabra "No" en el catálogo ilustrado de componentes.
Según la disponibilidad y la situación geográfica, los componentes CSR se enviarán para que lleguen a su destino al siguiente día laborable. Si la situación geográfica lo permite, se puede solicitar la entrega en el mismo día o en cuatro horas con un coste adicional. Si precisa asistencia técnica, puede llamar al Centro de asistencia técnica de Hewlett Packard Enterprise y recibirá ayuda telefónica por parte de un técnico. Con el envío de materiales para la sustitución de componentes CSR, Hewlett Packard Enterprise especificará si los componentes defectuosos deberán devolverse a Hewlett Packard Enterprise. En aquellos casos en los que sea necesario devolver algún componente a Hewlett Packard Enterprise, deberá hacerlo en el periodo de tiempo especificado, normalmente cinco días laborables. Los componentes defectuosos deberán devolverse con toda la documentación relacionada y con el embalaje de envío. Si no enviara el componente defectuoso requerido, Hewlett Packard Enterprise podrá cobrarle por el de sustitución. En el caso de todas sustituciones que lleve a cabo el cliente, Hewlett Packard Enterprise se hará cargo de todos los gastos de envío y devolución de componentes y escogerá la empresa de transporte que se utilice para dicho servicio.
Para obtener más información acerca del programa de Reparaciones del propio cliente de Hewlett Packard Enterprise, póngase en contacto con su proveedor de servicios local. Si está interesado en el programa para Norteamérica, visite la página web de Hewlett Packard Enterprise CSR (http://www.hpe.com/support/selfrepair).
Customer Self Repair Veel onderdelen in Hewlett Packard Enterprise producten zijn door de klant zelf te repareren, waardoor de reparatieduur tot een minimum beperkt kan blijven en de flexibiliteit in het vervangen van defecte onderdelen groter is. Deze onderdelen worden CSR-onderdelen (Customer Self Repair) genoemd. Als Hewlett Packard Enterprise (of een Hewlett Packard Enterprise Service Partner) bij de diagnose vaststelt dat de reparatie kan worden uitgevoerd met een CSR-onderdeel, verzendt Hewlett Packard Enterprise dat onderdeel rechtstreeks naar u, zodat u het defecte onderdeel daarmee kunt vervangen. Er zijn twee categorieën CSR-onderdelen:
• Verplicht—Onderdelen waarvoor reparatie door de klant verplicht is. Als u Hewlett Packard Enterprise verzoekt deze onderdelen voor u te vervangen, worden u voor deze service reiskosten en arbeidsloon in rekening gebracht.
• Optioneel—Onderdelen waarvoor reparatie door de klant optioneel is. Ook deze onderdelen zijn ontworpen voor reparatie door de klant. Als u echter Hewlett Packard Enterprise verzoekt deze onderdelen voor u te vervangen, kunnen daarvoor extra kosten in rekening worden gebracht, afhankelijk van het type garantieservice voor het product.
OPMERKING: Sommige Hewlett Packard Enterprise onderdelen zijn niet ontwikkeld voor reparatie door de klant. In verband met de garantievoorwaarden moet het onderdeel door een geautoriseerde Service Partner worden vervangen. Deze onderdelen worden in de geïllustreerde onderdelencatalogus aangemerkt met "Nee".
Afhankelijk van de leverbaarheid en de locatie worden CSR-onderdelen verzonden voor levering op de eerstvolgende werkdag. Levering op dezelfde dag of binnen vier uur kan tegen meerkosten worden aangeboden, indien dit mogelijk is gezien de locatie. Indien assistentie is gewenst, belt u het Hewlett Packard Enterprise Support Center om via de telefoon ondersteuning van een technicus te ontvangen.
Support and other resources 108
Hewlett Packard Enterprise vermeldt in de documentatie bij het vervangende CSR-onderdeel of het defecte onderdeel aan Hewlett Packard Enterprise moet worden geretourneerd. Als het defecte onderdeel aan Hewlett Packard Enterprise moet worden teruggezonden, moet u het defecte onderdeel binnen een bepaalde periode, gewoonlijk vijf (5) werkdagen, retourneren aan Hewlett Packard Enterprise. Het defecte onderdeel moet met de bijbehorende documentatie worden geretourneerd in het meegeleverde verpakkingsmateriaal. Als u het defecte onderdeel niet terugzendt, kan Hewlett Packard Enterprise u voor het vervangende onderdeel kosten in rekening brengen. Bij reparatie door de klant betaalt Hewlett Packard Enterprise alle verzendkosten voor het vervangende en geretourneerde onderdeel en kiest Hewlett Packard Enterprise zelf welke koerier/transportonderneming hiervoor wordt gebruikt.
Neem contact op met een Service Partner voor meer informatie over het Customer Self Repair programma van Hewlett Packard Enterprise. Informatie over Service Partners vindt u op de Hewlett Packard Enterprise website (http://www.hpe.com/support/selfrepair).
Reparo feito pelo cliente Os produtos da Hewlett Packard Enterprise são projetados com muitas peças para reparo feito pelo cliente (CSR) de modo a minimizar o tempo de reparo e permitir maior flexibilidade na substituição de peças com defeito. Se, durante o período de diagnóstico, a Hewlett Packard Enterprise (ou fornecedores/parceiros da Hewlett Packard Enterprise) concluir que o reparo pode ser efetuado pelo uso de uma peça CSR, a Hewlett Packard Enterprise enviará a peça diretamente ao cliente. Há duas categorias de peças CSR:
• Obrigatória—Peças cujo reparo feito pelo cliente é obrigatório. Se desejar que a Hewlett Packard Enterprise substitua essas peças, serão cobradas as despesas de transporte e mão-de-obra do serviço.
• Opcional—Peças cujo reparo feito pelo cliente é opcional. Essas peças também são projetadas para o reparo feito pelo cliente. No entanto, se desejar que a Hewlett Packard Enterprise as substitua, pode haver ou não a cobrança de taxa adicional, dependendo do tipo de serviço de garantia destinado ao produto.
OBSERVAÇÃO: Algumas peças da Hewlett Packard Enterprise não são projetadas para o reparo feito pelo cliente. A fim de cumprir a garantia do cliente, a Hewlett Packard Enterprise exige que um técnico autorizado substitua a peça. Essas peças estão identificadas com a marca "No" (Não), no catálogo de peças ilustrado.
Conforme a disponibilidade e o local geográfico, as peças CSR serão enviadas no primeiro dia útil após o pedido. Onde as condições geográficas permitirem, a entrega no mesmo dia ou em quatro horas pode ser feita mediante uma taxa adicional. Se precisar de auxílio, entre em contato com o Centro de suporte técnico da Hewlett Packard Enterprise para que um técnico o ajude por telefone. A Hewlett Packard Enterprise especifica nos materiais fornecidos com a peça CSR de reposição se a peça com defeito deve ser devolvida à Hewlett Packard Enterprise. Nos casos em que isso for necessário, é preciso enviar a peça com defeito à Hewlett Packard Enterprise, você deverá enviar a peça com defeito de volta para a Hewlett Packard Enterprise dentro do período de tempo definido, normalmente em 5 (cinco) dias úteis. A peça com defeito deve ser enviada com a documentação correspondente no material de transporte fornecido. Caso não o faça, a Hewlett Packard Enterprise poderá cobrar a reposição. Para as peças de reparo feito pelo cliente, a Hewlett Packard Enterprise paga todas as despesas de transporte e de devolução da peça e determina a transportadora/serviço postal a ser utilizado.
Para obter mais informações sobre o programa de reparo feito pelo cliente da Hewlett Packard Enterprise, entre em contato com o fornecedor de serviços local. Para o programa norte-americano, visite o site da Hewlett Packard Enterprise (http://www.hpe.com/support/selfrepair).
Support and other resources 109
Support and other resources 110
Support and other resources 111
Support and other resources 112
Remote support Remote support is available with supported devices as part of your warranty or contractual support agreement. It provides intelligent event diagnosis, and automatic, secure submission of hardware event notifications to Hewlett Packard Enterprise, which will initiate a fast and accurate resolution based on your product’s service level. Hewlett Packard Enterprise strongly recommends that you register your device for remote support.
For more information and device support details, go to the Insight Remote Support website (http://www.hpe.com/info/insightremotesupport/docs).
Warranty and regulatory information 113
Warranty and regulatory information
Warranty information HPE ProLiant and x86 Servers and Options (http://www.hpe.com/support/ProLiantServers-Warranties)
HPE Enterprise Servers (http://www.hpe.com/support/EnterpriseServers-Warranties)
HPE Storage Products (http://www.hpe.com/support/Storage-Warranties)
HPE Networking Products (http://www.hpe.com/support/Networking-Warranties)
Regulatory information Safety and regulatory compliance
For important safety, environmental, and regulatory information, see Safety and Compliance Information for Server, Storage, Power, Networking, and Rack Products, available at the Hewlett Packard Enterprise website (http://www.hpe.com/support/Safety-Compliance-EnterpriseProducts).
Belarus Kazakhstan Russia marking
Manufacturer and Local Representative Information
Manufacturer information: Hewlett Packard Enterprise Company, 3000 Hanover Street, Palo Alto, CA 94304 U.S.
Local representative information Russian:
• Russia:
• Belarus:
• Kazakhstan:
Local representative information Kazakh:
Warranty and regulatory information 114
• Russia:
• Belarus:
• Kazakhstan:
Manufacturing date: The manufacturing date is defined by the serial number.
CCSYWWZZZZ (serial number format for this product)
Valid date formats include:
• YWW, where Y indicates the year counting from within each new decade, with 2000 as the starting point; for example, 238: 2 for 2002 and 38 for the week of September 9. In addition, 2010 is indicated by 0, 2011 by 1, 2012 by 2, 2013 by 3, and so forth.
• YYWW, where YY indicates the year, using a base year of 2000; for example, 0238: 02 for 2002 and 38 for the week of September 9.
Turkey RoHS material content declaration
Ukraine RoHS material content declaration
Acronyms and abbreviations 115
Acronyms and abbreviations
ARPO Automatic Rapid Power Off
BMS building management system
CMU HPE Insight Cluster Management Utility
HEX heat exchanger
HPE APM HPE Advanced Power Manager
iCDU Intelligent Cooling Distribution Unit
PI proportional integral
PID proportional integral differential
PRG program
RTD resistance temperature detector
TBB thermal bus bar
VFD variable frequency drive
Documentation feedback 116
Documentation feedback
Hewlett Packard Enterprise is committed to providing documentation that meets your needs. To help us improve the documentation, send any errors, suggestions, or comments to Documentation Feedback (mailto:[email protected]). When submitting your feedback, include the document title, part number, edition, and publication date located on the front cover of the document. For online help content, include the product name, product version, help edition, and publication date located on the legal notices page.
Index 117
A ARPO 58 authorized reseller 103 AUTO mode 81, 82
B before you contact Hewlett Packard Enterprise 103 BMS, connections 74
C cabling 59 CMU 67, 86, 114 compute cell 26, 27, 29, 65 compute cell numbering 28 condensation management 85 configuration 6, 15 configuration of system 86 connecting communication 62 connecting power 61, 64 connecting the CMU solution 67 connecting the management module 60, 62 connecting the power shelves 59, 61, 64, 66 connecting the rack integrated switch 65, 67 connecting the racks 65 connecting the utility module 59, 60, 62, 64 contacting Hewlett Packard Enterprise 103 control switch 81 customer self repair (CSR) 104
D differential pressure 83 display modes 77, 80
E electrical specifications 98 environmental specifications 100
F fan numbering 30 firmware, updating 50 frost damage 85
H HAND mode 81 Hewlett Packard Enterprise Technical Support 103
HEX 49, 51, 114 HPE Advanced Power Manager 114 HPE APM sensor monitoring 95 HPE Insight Cluster Management Utility 86
I iCDU 69, 74, 77, 81, 84, 114 iCDU components 14, 69, 74, 77 iCDU control 81, 82 iCDU display 77 iCDU HEX connections 66 iCDU maintenance 93 iCDU rack management switch 67 iCDU troubleshooting 93 infiniband switch 53, 56 infiniband switch port numbering 54 infiniband switch, components 54, 56 installation services 15 installing hardware 21, 24, 35, 46, 56 isolation valve 82
L Leak detector sensor is not working properly 57 Leakage detected 57 LED indicators 19, 24, 29, 40, 44, 46, 55 LEDs, troubleshooting 87 locks, rack door 52
M management module 35
N numbering, fans 30 numbering, infiniband switch ports 54
O overview, product 6
P passwords 77 PI loop 83 power 69, 74 power components 17, 18 power considerations 69, 74 power redundancy 23, 24 power shelf components 18, 20, 21 power shelf procedures 21, 22, 24
Index
Index 118
power shelf, installation 21, 24 power supply 23, 24 power usage 24 product overview 6 program screen 77, 80
R rack components 11 rack door sensors and locks 52 regulatory information 112
S safety information 101, 112 safety, component 57 screen lock 77 screen time-out 77 security 77 security settings 77 sensors, temperature 57 service information 15 solution, CMU 67 specifications 98, 100 specifications, environmental 98 support 103 support and other resources 103
T technical support 103 telephone numbers 103 thermal air flow performance 100 Thermal componets 48, 50 Thermal controller 50 troubleshooting 87
U utility module 37, 38, 39, 44, 46 utility module, components 39, 41, 45 utility module, connectors 40, 42 utility module, different models 38, 39 utility module, location 37 utility module, specifications 46
V vacuum pressure 83 vacuum pump 83
W water maintenance 84 water quality 85 water temperature 83 water testing 85 Water walls 49, 51