118
HPE Apollo 8000 System User Guide Abstract This guide provides user information for the HPE Apollo 8000 System. Part Number: 747365-003R November 2015 Edition: 4

HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

  • Upload
    others

  • View
    9

  • Download
    0

Embed Size (px)

Citation preview

Page 1: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

HPE Apollo 8000 System User Guide

Abstract This guide provides user information for the HPE Apollo 8000 System.

Part Number: 747365-003R November 2015 Edition: 4

Page 2: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

© Copyright 2015 Hewlett Packard Enterprise Development LP

The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.

Page 3: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Contents 3

Contents

Overview................................................................................................................................................ 6 Product overview ...................................................................................................................................................... 6 Example configurations ............................................................................................................................................ 6

Component identification ..................................................................................................................... 11 f8000 rack components .......................................................................................................................................... 11 iCDU components .................................................................................................................................................. 13

iCDU display components ........................................................................................................................... 14

Installation and configuration ............................................................................................................... 15 Installation included with service ............................................................................................................................ 15 Installation expectations ......................................................................................................................................... 15

Power components .............................................................................................................................. 17 About the power shelf ............................................................................................................................................. 17

Power shelf front components ..................................................................................................................... 18 Power shelf rear components ...................................................................................................................... 19 Installing the power shelf ............................................................................................................................. 21 Configuring the power shelf ......................................................................................................................... 22

Redundant power supplies ..................................................................................................................................... 23 Redundant power supply LED indicator ...................................................................................................... 24 Installing the redundant power supply ......................................................................................................... 24

Monitoring power usage ......................................................................................................................................... 24

Compute cell ........................................................................................................................................ 26 About the compute cell ........................................................................................................................................... 26

Compute cell components ........................................................................................................................... 27 Compute cell numbering.............................................................................................................................. 28 Management module connectors and LED indicators ................................................................................. 29 Fan numbering ............................................................................................................................................ 30

Installing the management module ........................................................................................................................ 35

Utility module ....................................................................................................................................... 37 About the utility module .......................................................................................................................................... 37

Utility module products ................................................................................................................................ 38 HPE Apollo 8000 UM1 ........................................................................................................................................... 39

Utility module front components .................................................................................................................. 39 Utility module rear components ................................................................................................................... 41

HPE Apollo 8000 UM2 ........................................................................................................................................... 42 Utility module front components .................................................................................................................. 43 Utility module rear components ................................................................................................................... 45

Installing the utility module ..................................................................................................................................... 46 Configuring the utility module ................................................................................................................................. 47

Thermal components ........................................................................................................................... 48 About the thermal components .............................................................................................................................. 48 HEX ........................................................................................................................................................................ 49

HEX components and connectors ............................................................................................................... 49 Thermal controller .................................................................................................................................................. 50

Thermal controller components and connectors ......................................................................................... 50 Updating the thermal controller firmware..................................................................................................... 50

Water walls ............................................................................................................................................................. 51 Water wall components and connectors...................................................................................................... 51

Rack door sensors and locks ................................................................................................................................. 52 Activating and deactivating door locks ........................................................................................................ 52

Page 4: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Contents 4

Infiniband switch .................................................................................................................................. 53 About the infiniband switch ..................................................................................................................................... 53

Infiniband switch front components and connectors.................................................................................... 54 Infiniband switch port numbering ................................................................................................................. 54 Infiniband switch LED indicators.................................................................................................................. 55 Infiniband switch rear components and connectors .................................................................................... 56

Installing the infiniband switch ................................................................................................................................ 56

Safety components .............................................................................................................................. 57 About the safety components ................................................................................................................................. 57 Leak detection sensors .......................................................................................................................................... 57 Temperature sensors and automatic door release mechanism ............................................................................. 57 (Optional) ARPO .................................................................................................................................................... 58

Cabling ................................................................................................................................................ 59 Cabling overview .................................................................................................................................................... 59 Cabling when the HPE Apollo 8000 UM1 is installed ............................................................................................. 59

Connecting the utility module to the power shelves .................................................................................... 59 Connecting the utility module to the management modules ........................................................................ 60

Cabling when the HPE Apollo 8000 UM2 is installed ............................................................................................. 61 Connecting power........................................................................................................................................ 61 Connecting communication ......................................................................................................................... 62 Connecting the f8000 rack to the iCDU rack ............................................................................................... 65

Connect the CMU solution hardware ..................................................................................................................... 67

iCDU .................................................................................................................................................... 69 About the iCDU ...................................................................................................................................................... 69 iCDU setup ............................................................................................................................................................. 69

Powering on the iCDU ................................................................................................................................. 69 Powering off the iCDU ................................................................................................................................. 74 BMS network communication ...................................................................................................................... 74

iCDU display ........................................................................................................................................................... 77 Password security level ............................................................................................................................... 77 Screen time-out ........................................................................................................................................... 77 Always available .......................................................................................................................................... 77 Always visible .............................................................................................................................................. 80

iCDU control ........................................................................................................................................................... 81 Control switch .............................................................................................................................................. 81 iCDU isolation valves................................................................................................................................... 82 Vacuum pressure ........................................................................................................................................ 83 Differential pressure .................................................................................................................................... 83 Pump and IT supply out water temperature ................................................................................................ 83

iCDU maintenance ................................................................................................................................................. 84 Water testing and maintenance schedule ................................................................................................... 84

Condensation management ................................................................................................................. 85 Condensation management ................................................................................................................................... 85 Frost damage ......................................................................................................................................................... 85

Software and configuration utilities ...................................................................................................... 86 HPE Insight Cluster Management Utility ................................................................................................................ 86 HPE Advanced Power Manager ............................................................................................................................. 86

Troubleshooting ................................................................................................................................... 87 HPE Apollo 8000 System Troubleshooting ............................................................................................................ 87

Rack and mechanical issues ....................................................................................................................... 87 Power issues ............................................................................................................................................... 88 Temperature and cooling issues ................................................................................................................. 90 Communication issues ................................................................................................................................ 91 iCDU issues ................................................................................................................................................. 93

iCDU alarm troubleshooting ................................................................................................................................... 93 Using HPE APM for sensor monitoring .................................................................................................................. 95

Page 5: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Contents 5

Specifications ...................................................................................................................................... 98 Physical specifications ........................................................................................................................................... 98 Electrical specifications .......................................................................................................................................... 98 Thermal and air flow performance ........................................................................................................................ 100 Environmental specifications ................................................................................................................................ 100

Safety considerations......................................................................................................................... 101 Important safety information ................................................................................................................................. 101 Symbols on equipment ......................................................................................................................................... 101 Warnings and cautions ......................................................................................................................................... 101

Support and other resources.............................................................................................................. 103 Accessing Hewlett Packard Enterprise Support ................................................................................................... 103

Information to collect ................................................................................................................................. 103 Accessing updates ............................................................................................................................................... 103 Websites ............................................................................................................................................................... 103 Customer Self Repair ........................................................................................................................................... 104 Remote support .................................................................................................................................................... 112

Warranty and regulatory information .................................................................................................. 113 Warranty information ............................................................................................................................................ 113 Regulatory information ......................................................................................................................................... 113

Safety and regulatory compliance ............................................................................................................. 113 Belarus Kazakhstan Russia marking ......................................................................................................... 113 Turkey RoHS material content declaration ................................................................................................ 114 Ukraine RoHS material content declaration .............................................................................................. 114

Acronyms and abbreviations .............................................................................................................. 115

Documentation feedback ................................................................................................................... 116

Index .................................................................................................................................................. 117

Page 6: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Overview 6

Overview

Product overview The HPE Apollo 8000 System is a high-density, energy-efficient, sustainable high-performance computing solution that uses an innovative warm-liquid cooling technology to fuel the future of supercomputing. It offers hundreds of teraflops per rack to accelerate results, with a modular, rack-based system that is easy to install, maintain, and monitor. The hot water "waste heat" can be recycled to heat the data center efficiently.

The Apollo 8000 System comprises the following components:

• One or more HPE 8000 iCDU Racks and their associated onboard iCDU control system

• One or more f8000 racks and their associated control system

• Primary plumbing interface assembly kit that connects the individual iCDUs to the facility water system

• Secondary plumbing system that connects the individual iCDUs to the f8000 racks. This secondary closed loop system provides cooling for the f8000 rack and isolates the f8000 rack from untreated, potentially incompatible primary water.

• Drain and leak detection system

The iCDU performs the following functions:

• Accepts water supplied from the facility water system

• Returns heated water back to the facility water system

• Supplies cooled water to the f8000 racks through the secondary plumbing system

• Accepts heated water returned from the f8000 racks

• Maintains the preset water temperature and pressure in the secondary plumbing system

• Monitors system health

• Isolates primary and secondary water in case of a leak

Example configurations The Apollo 8000 System is a self-contained IT product integrated with a closed-loop cooling system that offers two configuration options.

The following figures demonstrate examples of the f8000 rack (1) to iCDU (2) ratio. The doors are removed for clarity.

CAUTION: To reduce the risk of condensation and avoid damage to the equipment when operating with chilled water, keep the rack doors closed during operation.

Example single iCDU configuration (with up to four f8000 racks)

Page 7: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Overview 7

• Front view

Item Description 1 HPE Apollo f8000 Racks 2 HPE Apollo iCDU Racks

Page 8: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Overview 8

• Top view

Item Description 1 Facility water in 2 Facility water out 3 Secondary water out from f8000 rack 4 Secondary water in to f8000 rack 5 Heat exchanger

Example redundant iCDU configuration (with up to four f8000 racks)

Page 9: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Overview 9

• Front view

Item Description 1 Apollo f8000 Racks 2 Apollo iCDU Racks

Page 10: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Overview 10

• Top view

Item Description 1 Facility water in 2 Facility water out 3 Secondary water out of f8000 rack 4 Secondary water in to f8000 rack 5 Heat exchanger

The iCDU transfers heat to the facility water plant and has a secondary loop that provides cooling to the servers in the IT racks.

Page 11: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Component identification 11

Component identification

f8000 rack components The Apollo 8000 System is composed of several key components, described below, which work together to provide a modular, rack-based, cooling solution.

f8000 rack front components

Item Component Description 1 Servers Provides services for other computers and programs

within the network 2 Fan units Circulate the cool air from the HEX through the f8000 rack 3 Power shelves The top power shelf provides power to the components in

the top half of the f8000 rack, and the bottom power shelf provides power to the components in the bottom half of the f8000 rack.

4 Utility module Gathers and monitors data from the f8000 rack components to display in HPE APM

Page 12: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Component identification 12

Item Component Description 5 Door sensor Monitors the front door to detect if the door is open or

closed

The utility module might appear different based on the model installed. For more information on the different models available, see "Utility module products (on page 38)."

f8000 rack rear components

Item Component Description 1 Management module assembly Control center for 10U rack compute cell 2 Fan units Circulate the cool air from the HEX through the f8000 rack 3 Cable tray Routes cables from the front to the rear of the f8000 rack 4 Rack drain pan Collects any condensation from the HEX 5 Rack door magnet Gathers and adjusts temperature requirements based on

pre-configured setpoints 6 RTD sensors Measures water temperature going in and coming out of

the HEX

Page 13: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Component identification 13

iCDU components

Item Description 1 iCDU display 2 USB connector for firmware updates 3 Ethernet connector for network connection 4 DB9 connector for pLAN connection 5 Hand/Off/Auto switch 6 VFD control button 7 Power On/Off switch 8 Facility water actuating valve 9 Vacuum pump 10 Water level site glass 11 Water return from upper iCDU rack 12 Water return from f8000 rack 13 iCDU drain 14 Leak detection sensors in the iCDU drain pan 15 Auto/Manual water fill connection 16 Water supply to upper iCDU rack 17 Manual water fill button 18 Water supply to f8000 rack 19 Facility supply to iCDU 20 Facility return from iCDU

Page 14: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Component identification 14

iCDU display components

Item Description 1 Alarm button 2 Program button 3 Escape button 4 Up arrow button 5 Enter button 6 Down arrow button

Page 15: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Installation and configuration 15

Installation and configuration

Installation included with service The HPE Apollo 8000 System Startup Service purchased with your product includes the installation of Apollo f8000 racks and HPE Apollo 8000 iCDU racks, including all the associated HPE ProLiant servers and network switches. There are many benefits to the service agreement, including:

• Deployment management

• Service planning

• Site readiness verification checklist

• Installation and startup

• Installation verification tests (IVT)

• Customer orientation session

For more information on what is included in the service agreement, see the Apollo 8000 System Startup Service Data Sheet (http://www.hpe.com/support/hpesc).

Installation expectations Each installation has many variables, which dictate how the installation is performed and how long it might take, including:

• Size and configuration of the installation

• Site-specific variables, such as facility water and power

• Site-specific requirements, such as networking, safety, and security

A deployment manager will ensure all variables are accounted for prior to installation. For more information on what is included in the service agreement, see the Apollo 8000 System Startup Service Data Sheet (http://www.hpe.com/support/hpesc).

For the purpose of this explanation, a common installation includes:

• One iCDU rack

• Two f8000 racks

IMPORTANT: The following table is a general overview of what a customer can expect during the installation process, which is by the Apollo 8000 Startup Service. It is not intended to be used as a procedure or schedule but is only a sample.

Installation requirement

Description Responsible party

Complete site audit questionnaire

Determine a general baseline for site readiness. Hewlett Packard Enterprise Sales or Customer

Page 16: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Installation and configuration 16

Installation requirement

Description Responsible party

Site audit Determine any modifications required to ensure site readiness for delivery and installation based on specifications outlined in the Apollo 8000 System Site Preparation Guide.1 After the site audit is complete, Hewlett Packard Enterprise Services can provide a quote for custom services, if the Customer desires.

Hewlett Packard Enterprise Sales, Hewlett Packard Enterprise Services, and Customer

Site modifications Complete all required modifications to ensure site readiness for delivery and installation.

Customer, Third party, or Hewlett Packard Enterprise Services2

Install and test plumbing Varies depending on existing plumbing and modifications required.

Customer, Third party, or Hewlett Packard Enterprise Services2

Install rack solution Includes: • Deliver all racks.3 • Level and bolt the racks in place. • Make all facility connections, including power,

water, and networking.

Hewlett Packard Enterprise Services

Infiniband cabling between racks

Cabling all infiniband network components for the rack solution.

Customer, Third party, or Hewlett Packard Enterprise Services2

Complete water preparation

Includes: • Pressure testing. • Add water and required chemistry. • Test chemistry.

Hewlett Packard Enterprise Services

Power on the system Includes: • Power on the iCDU rack. • Power on the CMU solution. • Power on the f8000 racks.

Hewlett Packard Enterprise Services

Test the system Includes: • Run rack-level system tests. • Run system-level tests. • Clean up fabrics. • Verify all f8000 rack hardware components are

functioning. • Verify the APM is functioning and reporting data. • Verify the CMU solution is functioning and reporting

data.

Hewlett Packard Enterprise Services

Test the server functionality

Includes: • Verify all server nodes boot. • Run CT single-node tests. • Verify all nodes pass CT node tests.

Hewlett Packard Enterprise Services

Customer orientation session

Varies depending on customer agreement and requirements.

Hewlett Packard Enterprise Services

1To view the Apollo 8000 System Site Preparation Guide, see the Hewlett Packard Enterprise website (http://www.hpe.com/info/windows/docs). 2Based on a separate, custom scope of work to be performed by Hewlett Packard Enterprise Services. 3White Glove Delivery is standard for installing the rack solution at the Customer site. This includes equipment delivery, transportation off the truck and into the final destination, and then removing all trash from the site within a few days of delivery.

Page 17: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Power components 17

Power components

About the power shelf The power shelf takes power input from the facility and provides the output power to the f8000 rack components. The power meters installed within the power shelf are specific to meet the input voltage requirement based on location:

• North America—Power meters are configured for 480/277VAC; 3-wire plus Neutral plus ground; L22-30P connections (two per power shelf)

• International—Power meters are configured for 380/220–415/240VAC; 3-wire plus Neutral plus ground; Hubbell 532P6W connections (two per power shelf)

Page 18: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Power components 18

Two power shelves are installed in each f8000 rack, and each is responsible for providing power to half of the rack. They also provide redundant power to some of the major f8000 rack components, such as the HEX and utility module.

Power shelf front components

Page 19: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Power components 19

Item Component Description 1 High voltage regulator Four high voltage regulators convert the facility power into

accessible power for the f8000 rack components.

Power shelf LED indicators

Health status LED:

• Green—Normal operation

• Off—No power indicates an issue

Power shelf rear components

Item Component Description 1 Power shelf controller Powers the power shelf on and off and reports status out to

APM 2 High voltage DC output connectors Sends power out to the f8000 rack components 3 Communication pass-through module Provides a serial communication port to interface with the

APM 4 Input power modules Brings power in from the facility to be used by the f8000

rack components 5 Grounding post connection Connection point for grounding the power shelf

Page 20: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Power components 20

Power shelf rear connectors

Item Component Description 1 Appliance outlet Provides a breaker-protected outlet that is connected from

one phase to neutral. The Rong Feng RF-203P-HP-1.2 outlet is 277VAC for the 480VAC input meter and a standard IEC 60320-2-2F for the 380/220VAC to 415/240VAC input meter. Not used for the Apollo 8000 System

2 Appliance breaker switch Breaker switch that controls the appliance outlet

3 Main 3-phase, 5-wire input cord Main power input to the power shelf 4 Compute cell connection Provides power to the top, left compute cell within the half

rack 5 Breaker indicator posts Pops out to indicate that one of the two output connectors

on either side of the post has tripped 6 HEX connection Provides power to the HEX in the iCDU rack 7 Torx driver Attached driver is used for removing the input power

modules. 8 Utility module connection Provides power to the utility module 9 Compute cell connection Provides power to the top, right compute cell within the half

rack 10 Compute cell connection Provides power to the bottom, right compute cell within the

half rack 11 Discovery services connection Reserved for future use 12 EPO connection Provides an EPO port for the power shelf. Supports either

normally open or normally closed configurations. Not used for the Apollo 8000 System

13 Bottom HEX connection* Provides power to one of the bottom* water wall assemblies

14 Top HEX connection* Provides power to one of the top* water wall assemblies 15 APM data connection Connects to the utility module to provide power shelf data

and control to APM 16 Compute cell connection Provides power to the bottom, left compute cell within the

half rack

*Each power shelf provides power to the top and bottom HEX water wall assemblies, therefore creating redundant power in the event of one power shelf losing functionality. The exact connection to the top or bottom HEX varies based on the top and bottom power shelf locations.

Page 21: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Power components 21

Installing the power shelf 1. Align and slide the power shelf into its bay in the front of the rack until it is fully inserted. 2. Insert four P2 Phillips screws, two in each side, to secure the power shelf to the front of the rack.

3. Install two protective covers, one on each side, over the front ends of the power shelf. 4. Install two P2 Phillips screws, one in each protective cover, to secure it the power shelf.

5. With the release lever in the open position, align and slide the voltage regulator into the bay in the

front of the power shelf until the release lever begins to engage.

Page 22: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Power components 22

6. Close the release lever to lock it and fully seat the voltage regulator into the backplane of the power shelf.

7. Close the rack front door. 8. From the rear of the rack, install the power cables and serial control cables into the rear of the power

shelf. 9. Insert both input modules into the rear of the power shelf.

Configuring the power shelf The power shelf is pre-configured to meet the input voltage requirements, based on location:

• North America—Power meters are configured for 480/277VAC; 3-wire plus Neutral plus ground; L22-30P connections (two per power shelf)

• International—Power meters are configured for 380/220–415/240VAC; 3-wire plus Neutral plus ground; Hubbell 532P6W connections (two per power shelf)

There are no additional configuration steps required after installation.

Page 23: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Power components 23

Redundant power supplies The redundant power supplies exist in pairs throughout the rack for the purpose of providing redundant power in the event of power failure. They are hot-swappable and can be replaced without having to power down the f8000 rack.

There are 11 redundant power supply pairs throughout the f8000 rack, responsible for providing redundant power to rack components:

• One pair of redundant power supplies in the front, center of the top heat exchanger

• One pair of redundant power supplies in the front, center of the bottom heat exchanger

• One pair of redundant power supplies in each of the management module assemblies

• One pair of redundant power supplies in the front of the utility shelf

Page 24: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Power components 24

Redundant power supply LED indicator

Health status LED:

• Green—Normal operation

• Off—No power indicates an issue

Installing the redundant power supply 1. Align and slide the power supply into the power supply bay. 2. Push the power supply until it is fully seated and locks into place.

Monitoring power usage Power component status and usage is monitored through APM. Use the SHOW POWER command to retrieve the following details:

Page 25: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Power components 25

• Input AC power information

• AC voltage

• AC current

• AC power

• Power factor

• Total harmonic distortion

• High voltage DC output voltage

• High voltage DC output current

• High voltage DC output power

• Health status

To view instructions on retrieving power status and details, see the HPE Advanced Power Manager User Guide for Apollo 8000 Solution on the Hewlett Packard Enterprise website (http://www.hpe.com/support/Apollo8000_APM_UG_en).

Page 26: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Compute cell 26

Compute cell

About the compute cell One compute cell is installed in each 10U quadrant space within the f8000 rack. Each compute cell assembly is composed of the following items:

• The management module

• Five fan units

• One pair of redundant power supplies

There is a total of eight compute cell assemblies in the rear of each f8000 rack; four in the top of the rack and four in the bottom of the rack.

Page 27: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Compute cell 27

The compute cell is the control center for a 10U cell of servers. It is responsible for the following tasks:

• Providing an iLO aggregation point for all servers within the cell

• Providing a connection point to APM to communicate and access data including power and temperature information for the cell

• Controlling the power output and fan speeds for the cell, based on pre-determined setpoints within APM

Compute cell components

Item Component Description 1 Redundant power supplies Provides redundant power to the management module

assembly 2 Power distribution module assembly Distributes power from the redundant power supplies 3 Fan units Provides air flow throughout the rear of the rack 4 Ethernet ports Provides an Ethernet connection between the utility

module and the compute cell 5 Management module Provides a connection point for communication between

the APM and the 10U cell

Page 28: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Compute cell 28

Compute cell numbering

Page 29: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Compute cell 29

Management module connectors and LED indicators

Item Description 1 Display port connector to the APM utility shelf 2 Identification LED

• Solid blue = Management module is receiving power and communicating.

• Off = No power or communication

Page 30: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Compute cell 30

Fan numbering Front rack fan numbering

The front rack fan health status or failures are only reported after entering the show thermal status command in APM.

For example: Upper Rack Zone:

Sensor Index Status

------------------ ------ -------------

Leak Detector 0 no leaks

Leak Detector 1 no leaks

Leak Detector 2 no leaks

Door Sensor 0 opened ( front )

Page 31: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Compute cell 31

Door Sensor 1 opened ( rear )

Valve Sensor 0 not present

Valve Sensor 1 not present

Water Temp (RTD) 0 29.4613 degC ( supply )

Water Temp (RTD) 1 29.8675 degC ( intermediate )

Water Temp (RTD) 2 31.6488 degC ( return -bot )

Flow 0 2.21365 PSI (740.00 mV reading)

Temp-Humid -board 0 33.66 degC, 21.35 %RH

Dew Point Temp. 0 8.52 degC

Air Temp. 0 31.0000 degC (Dev 0:0 )

Air Temp. 6 36.2500 degC (Dev 2:0 )

DC/DC Power Supply 0 112.00 mV (PS 0 - present - OK)

DC/DC Power Supply 1 191.00 mV (PS 1 - present - OK)

Control Index Status

------------------ ------ -------------

Door Lock 0 closed ( front )

Door Lock 1 closed ( rear )

Valve Control 0 not present

Valve Control 1 not present

EPO 0 running

Fan State Health RPM R1 Status R2 Status Fan mV Fan%

---- -------- ----- ------- --------- ------ ----- -----

1 : Fan installed healthy 4828 healthy healthy 341 29

2 : Fan installed healthy 4828 healthy healthy 341 29

3 : Fan installed healthy 4828 healthy healthy 341 29

4 : Fan installed healthy 4828 healthy healthy 341 29

5 : Fan installed healthy 4828 healthy healthy 341 29

6 : Fan installed healthy 4828 healthy healthy 341 29

7 : Fan installed healthy 4828 healthy healthy 341 29

8 : Fan installed healthy 4828 healthy healthy 341 29

9 : Fan installed healthy 4828 healthy healthy 341 29

10 : Fan installed healthy 4828 healthy healthy 341 29

Cell Fan %

---- -----

5 050

6 050

Page 32: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Compute cell 32

7 050

8 050

Lower Rack Zone:

Sensor Index Status

------------------ ------ -------------

Leak Detector 0 no leaks

Leak Detector 1 no leaks

Leak Detector 2 no leaks

Door Sensor 0 (not present)

Door Sensor 1 (not present)

Valve Sensor 0 not present

Valve Sensor 1 not present

Water Temp (RTD) 0 29.4925 degC ( supply )

Water Temp (RTD) 1 29.8675 degC ( intermediate )

Water Temp (RTD) 2 31.8050 degC ( return -bot )

Flow 0 2.11198 PSI (722.00 mV reading)

Temp-Humid -board 0 33.75 degC, 21.19 %RH

Dew Point Temp. 0 8.49 degC

Air Temp. 0 30.7500 degC (Dev 0:0 )

Air Temp. 6 35.0000 degC (Dev 2:0 )

DC/DC Power Supply 0 146.00 mV (PS 0 - present - OK)

DC/DC Power Supply 1 148.00 mV (PS 1 - present - OK)

Control Index Status

------------------ ------ -------------

Door Lock 0 not present

Door Lock 1 not present

Valve Control 0 not present

Valve Control 1 not present

EPO 0 running

Fan State Health RPM R1 Status R2 Status Fan mV Fan%

---- -------- ----- ------- --------- ------ ----- -----

1 : Fan installed healthy 4828 healthy healthy 336 29

2 : Fan installed healthy 4828 healthy healthy 336 29

3 : Fan installed healthy 4828 healthy healthy 336 29

4 : Fan installed healthy 4828 healthy healthy 336 29

Page 33: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Compute cell 33

5 : Fan installed healthy 4828 healthy healthy 336 29

6 : Fan installed healthy 4828 healthy healthy 336 29

7 : Fan installed healthy 4828 healthy healthy 336 29

8 : Fan installed healthy 4828 healthy healthy 336 29

9 : Fan installed healthy 4828 healthy healthy 336 29

10 : Fan installed healthy 4828 healthy healthy 336 29

Cell Fan %

---- -----

1 050

2 050

3 050

4 050

Page 34: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Compute cell 34

Compute cell fan numbering

The compute cell fans health status or failures are reported after entering the show rack command in APM or by reviewing the Fan tab within iLO.

For example: 1: Dist. Module

2: Enclosure

Product Name :Apollo 8000

Product SKU :000000-000

Serial Number:0000000000

UID :off

Status :

1 : Compute node (active)

2 : (unused)

Page 35: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Compute cell 35

3 : Compute node (active)

4 : (unused)

5 : Compute node (active)

6 : (unused)

7 : Compute node (active)

8 : (unused)

9 : Compute node (active)

10 : (unused)

11 : Compute node (active)

12 : (unused)

13 : Compute node (active)

14 : (unused)

15 : Compute node (active)

16 : (unused)

17 : Compute node (active)

18 : (unused)

19 : Compute node (active)

20 : (unused)

Power supply slot 1 is occupied, status good

Power supply slot 2 is occupied, status good

Fan slot 1 is occupied, status good

Fan slot 2 is occupied, status good

Fan slot 3 is occupied, status good

Fan slot 4 is occupied, status good

Fan slot 5 is occupied, status good

Installing the management module 1. Align and slide the management module into its bay in the compute cell until the release lever begins

to engage.

Page 36: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Compute cell 36

2. Close the release lever until it locks under the release button. The module should now be fully seated in the tray infrastructure backplane of the compute cell.

3. Connect all cables, including the display port cable.

Page 37: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Utility module 37

Utility module

About the utility module The utility module serves as a central location for gathering and monitoring data for the Apollo 8000 system, including:

• Managing rack Ethernet infrastructure via an embedded 160 port 1GB switch

• Gathering data from the f8000 rack sensors

• Controlling power output and fan speed for each management module cell

• Updating component status within APM

• Maintaining alarm conditions within APM

Page 38: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Utility module 38

There is one utility module installed in each f8000 rack within the Apollo 8000 System.

The utility module might appear different based on the model installed. For more information on the different models available, see "Utility module products (on page 38)."

Utility module products There are two utility module products that can be installed in the Apollo 8000 System. Although they might appear different and have slightly different features, their basic functionality within the Apollo 8000 System is the same.

Page 39: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Utility module 39

For the purpose of clarity in this document, when removing and replacing spare parts, the components will be referred to as shown in the following illustration and table.

Item Description 1 HPE Apollo 8000 Utility Module v1 (Apollo 8000 UM1) 2 HPE Apollo 8000 Utility Module v2 (Apollo 8000 UM2)

HPE Apollo 8000 UM1

Utility module front components

Page 40: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Utility module 40

Item Component Description 1 iLO aggregation module A hot-pluggable switch module that aggregates all server

iLOs to one port to provide an interface for the System Manager server to APM

2 APM module The main management module that contains the processor monitoring rack health for APM

Utility module front connectors

Item Component Description 1 Expansion port/System management

port Connects to the CMU solution

2 iLO A port iLO aggregation port collects uplink information from all of the compute nodes.*

3 Reset button Used to reset the APM 4 Serial console RJ-45 serial interface to the APM management CLI 5 Service port RJ-45 serial console port for local service, excluding any

changes that require a password, such as configuration or set points

6 Ethernet connection Ethernet interface to access the APM remotely 7 Facility Redundant reset, On-Gen, and Aux signals

*If the iLO A port is used to provide IP addresses, do not connect the Ethernet cable to the management module. The two connections at the same time result in a network loop condition.

Utility module front LEDs

Item Component Behavior 1 UID LED Flashes to indicate:

• The firmware is being flashed. • Identification by use of the "Set UID on" or "Set UID

off" commands

Page 41: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Utility module 41

Item Component Behavior 2 Active LED Flashes to indicate Ethernet activity 3 UID LED Flashes to indicate:

• The firmware is being flashed. • Identification by use of the "Set UID on" or "Set UID

off" commands

4 Health LED Solid Green—No active faults logged Solid Red—One or more faults has been logged.

Utility module rear components

Item Component Description 1 Serial ports Reserved for future use. Set up using the "Connect port"

command. 2 Power distribution module ports for

power shelf Connects via serial interface to the internal power shelves

3 Management module connection ports

APM connection to the management module and thermal controller

4 Redundant power supplies Provide redundant power for the utility module 5 Power connectors High voltage DC power connection to the power shelves 6 Fan unit Circulates cool air through the utility module

Page 42: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Utility module 42

Utility module rear connectors

Item Component Description 1 Power supply board connector Connects the utility module to the redundant power

supplies 2 Serial port Reserved for future use 3 Serial port Reserved for future use 4 Power distribution port A Connects the utility module to the top power shelf 5 Power distribution port B Connects the utility module to the bottom power shelf 6 Management module and rack

controller display port connections Connects the utility module to each of the management modules and thermal controllers

7 Fan connector Connects to the fan module on the utility module

HPE Apollo 8000 UM2

Page 43: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Utility module 43

Utility module front components

Item Component Description 1 Reset button Resets the APM 2 Serial console RJ-45 serial interface to the APM management CLI 3 Service port RJ-45 serial console port for local service, excluding any

changes that require a password, such as configuration or set points

4 Ethernet connection Ethernet interface to access the APM remotely 5 Facility connection Redundant reset, On-Gen and Aux signals 6 Expansion port Connects to the CMU solution

7 iLO A port/ Ethernet switch configuration

iLO aggregation port collects uplink information from all of the compute nodes. Provides access to configure the Ethernet switch

8 Blank* Pre-installed blanks to ensure thermal consistency for internal utility module components

9 Redundant power supply Provides redundant power to the utility module 10 Redundant power supply Provides redundant power to the utility module 11 Ethernet switch Provides Ethernet network connectivity

*Blanks are pre-installed on the utility module to ensure that the thermal requirements for the internal utility module components are maintained. Blanks should remain installed at all times to reduce the risk of overheating the internal electronics.

Page 44: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Utility module 44

Utility module front LEDs

Item Component Behavior 1 Active LED Flashes to indicate Ethernet activity 2 UID LED Flashes to indicate:

• The firmware is being flashed. • Identification by use of the "Set UID on" or "Set UID

off" commands

3 Health LED Solid Green—No active faults logged Solid Red—One or more faults has been logged.

4 UID LED Flashes to indicate: • The firmware is being flashed. • Identification by use of the "Set UID on" or "Set UID

off" commands

5 Power health LED Solid Green—Normal operation Flashing Red—One or more faults has been logged.

6 Power health LED Solid Green—Normal operation Flashing Red—One or more faults has been logged.

7 Access panel sensor Not activated* 8 UID LED Flashes to indicate:

• The firmware is being flashed. • Identification by use of the "Set UID on" or "Set UID

off" commands

9 Downlink activity LED Off—No activity Flashing Green—Activity for port

10 Uplink activity LED Off—No activity Flashing Green—Activity for port

11 Health status LED Solid Green—Normal operation Solid Red—One or more faults has been logged.

12 Power LED Off—No power Solid Green—Normal power operation

*Reserved for future use

Page 45: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Utility module 45

Utility module rear components

Item Component Description 1 Affixed power input cables Connects to both power shelves to provide redundant

power to utility module 2 Mini DB-9 connector Connects to the internal power shelf (bottom) 3 Mini DB-9 connector Connects to the internal power shelf (top)

4 Mini DB-9 connector Reserved for future use 5 Mini DB-9 connector Reserved for future use

6 Mini SAS HD connector Connects to the compute cell management module (cells 1 and 2)

7 Mini SAS HD connector Connects to the compute cell management module (cells 3 and 4)

8 Mini SAS HD connector Connects to the compute cell management module (cells 5 and 6)

9 Mini SAS HD connector Connects to the compute cell management module (cells 7 and 8)

10 Display port connector Connects to the internal HEX (top f8000 rack) 11 Display port connector Connects to the internal HEX (bottom f8000 rack) 12 Display port connector Reserved for future use 13 Reset button Utility module reset

14 Ethernet switch management serial console

Management serial console

15 QSFP connector Q1 Ethernet Uplink 16 QSFP connector Q2 Ethernet Uplink 17 QSFP connector Q3 Ethernet Uplink 18 QSFP connector Q4 Ethernet Uplink 19 J15 and J16 connector Connects to the Ethernet ports of compute cell # 8 20 J13 and J14 connector Connects to the Ethernet ports of compute cell # 7 21 J11 and J12 connector Connects to the Ethernet ports of compute cell # 6 22 J9 and J10 connector Connects to the Ethernet ports of compute cell # 5 23 System fan Provides system cooling 24 System fan Provides system cooling

Page 46: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Utility module 46

Item Component Description 25 J7 and J8 connector Connects to the Ethernet ports of compute cell # 4 26 J5 and J6 connector Connects to the Ethernet ports of compute cell # 3 27 J3 and J4 connector Connects to the Ethernet ports of compute cell # 2 28 J1 and J2 connector Connects to the Ethernet ports of compute cell # 1

Utility module rear LEDs

Item Component Behavior 1 UID LED Flashes to indicate:

• The firmware is being flashed. • Identification by use of the "Set UID on" or "Set UID

off" commands

2 Health LED Solid Green—No active faults logged Solid Red—One or more faults has been logged.

3 Q1-Q4 Uplink LEDs Solid green indicates an uplink connection is established. 4 Q1-Q4 Activity LED Flashing green indicates link activity. 5 Power LED (fan) Illuminates to indicate power 6 Power LED (fan) Illuminates to indicate power

Installing the utility module 1. Align and slide the utility shelf into the rack until the front hinges can be rotated in place and the

captive thumbscrews are aligned with the front of the rack. 2. From the front of the f8000 rack, tighten the two thumbscrews, one on each side, to secure the utility

shelf the front of the rack.

Page 47: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Utility module 47

3. From the rear of the f8000 rack, tighten the two thumbscrews, one on each side, to secure the utility shelf to the inside of the rack.

The utility module might appear different based on the model installed. For more information on the different models available, see "Utility module products (on page 38)."

4. Connect all appropriate cables to the rear of the utility shelf.

Configuring the utility module For information on configuring the utility module, see the Advanced Power Manager for Apollo 8000 User Guide (http://www.hpe.com/support/hpesc).

Page 48: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Thermal components 48

Thermal components

About the thermal components

CAUTION: To reduce the risk of condensation and avoid damage to the equipment when operating with chilled water, keep the rack doors closed during operation.

The thermal components are responsible for providing cooling to the rack components and servers. The Apollo 8000 System uses both water cooling and air circulation to maintain thermal parameters for optimal server and IT health. The following components work together to maintain the thermal requirements:

• HEX with embedded thermal controller

• Water walls

• Fans

• Rack door sensors

There are thermal components located in the top and bottom half rack of the f8000 rack and the top half of the iCDU rack.

Item Component Description 1 Left water wall Provides cooling to the servers on the left side of the f8000

rack

Page 49: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Thermal components 49

Item Component Description 2 HEX Removes the heated air from within the f8000 rack 3 Right water wall Provides cooling to the servers on the right side of the

f8000 rack 4 Fans Provides rack-level air circulation for system cooling

Thermal gasketing is installed throughout the racks to ensure air flow maintains the pre-configured thermal parameters. The rack doors must remain closed during normal operation to ensure thermal parameters.

HEX The HEX removes the heated air from within the f8000 rack, and supplies rack-level air cooling that is circulated by the fans.

The HEX units are located in both the f8000 rack and the iCDU rack; one in the top of the f8000 rack, one in the bottom of the f8000 rack, and one in the top of the iCDU rack.

HEX components and connectors

Item Component Description 1 HEX Removes the heated air from within the f8000 rack 2 Fan unit Provides rack-level air circulation for system cooling 3 Redundant power supplies Provides redundant power to the fans and embedded

thermal controller on the HEX 4 Embedded thermal controller Adjusts temperature requirements for the HEX

Page 50: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Thermal components 50

Thermal controller The embedded thermal controller on each HEX gathers data and adjusts temperature requirements based on pre-configured setpoints within APM as follows:

• Gathers information for all of the sensors within the rack

• Sends data from the sensors and management module to APM

• Receives feedback from APM alarm conditions, and adjusts fan speeds to maintain pre-configured temperature setpoints

• Opens the rack doors during an over temperature event

There are three embedded thermal controllers; two in the f8000 rack and one in the iCDU rack.

Thermal controller components and connectors

Item Component Description 1 Display port connector Connects the thermal controller to APM 2 Ethernet connector Debug only 3 Serial port connector Debug only 4 Sensor cable connector 1 Connects to the cable harness that gathers all sensor data

for the thermal controller 5 Sensor cable connector 2 Connects to the cable harness that gathers all sensor data

for the thermal controller 6 Fan 6-10 management connector Connector for fan management signals 7 Fan 1-5 management connector Connector for fan management signals 8 Fan 1-5 power connector Connector to supply power to the fans 9 Fan 6-10 power connector Connector to supply power to the fans

Updating the thermal controller firmware 1. Log in to the APM CLI interface using a serial port, Telnet, or SSH connection. 2. Enter the CLI command: UPGRADE CONTROLLER RS485 <PORT#>, where <port#> is 9 or 10.

If the thermal controller is already running the latest firmware, the update will not start.

Page 51: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Thermal components 51

Following the firmware update, the APM is automatically updated and reset to run the new image.

For more information, see the Advanced Power Manager for Apollo 8000 User Guide (http://www.hpe.com/support/Apollo8000_APM_UG_en).

Water walls The water walls provide cooling to the servers in the Apollo 8000 System through the TBB interface. The water flows through the water wall, condensing the vapor within the cooling pipes of each server. Additionally, the water walls do the following:

• Link the HEX to the servers

• Supply filtered cooler water to the servers through the TBB interface

• Remove the heated water from the servers, returning it to the secondary plumbing

There are two water walls, a left and a right, on either side of each HEX. Each water wall provides cooling to the servers in that rack quadrant.

NOTE: The HEX in the iCDU rack does not have water walls or a TBB interface. The fans circulate cooled air throughout the iCDU rack, but there is no direct link between the HEX and any installed servers in that rack via TBB interface.

Water wall components and connectors

Item Component Description 1 Water wall vertical manifold Pathway for water supply and return 2 TBB interface Provides the cooler water to the servers, while removing

the heated water 3 TBB pull bar Provides a stop for server installation and engages the

TBB with the server

Page 52: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Thermal components 52

Rack door sensors and locks The rack door sensors help maintain the thermal conditions within the f8000 rack, by activating the door locks to keep the doors closed when the temperature is correct and deactivating the door locks to automatically open the rack doors when the temperature gets too high.

CAUTION: Do not force the rack door open when the door locks are activated. Instead, use the APM CLI to release the magnetized door locks, and then open the doors. Trying to force the door open while the door locks are activated can result in a bent or broken rack door.

There is one door sensor and lock on the rack front door, and one door sensors and three locks on the rack rear door. For more information on deactivating the magnetized door locks, see "Activating and deactivating door locks (on page 52)."

Activating and deactivating door locks The magnetic door locks should be activated and deactivated through the APM CLI: 1. Open VT reflections for Unix. 2. Select Connection>Connection Set-Up. 3. Select Secure Shell. 4. Log in to the f8000 rack APM, using the following settings:

o Host name—Rack IP address o User name—Administrator

5. Select Connect to enter your Administrator password and access the CLI. 6. Enter one of the following commands:

o SET DOOR OPEN FRONT—Disengages the magnetic door locks on the rack front door. o SET DOOR OPEN REAR—Disengages the magnetic door locks on the rack rear doors. o SET DOOR CLOSE FRONT—Engages the magnetic door locks and closes the rack front door. o SET DOOR CLOSE REAR—Engages the magnetic door locks and closes the rack rear doors.

Page 53: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Infiniband switch 53

Infiniband switch

About the infiniband switch The 1U infiniband switch communicates data between processors and I/O devices from the servers within its 10U cell. Data is transmitted in packets that together form a message, exchanging information across the network.

One infiniband switch is installed for each 10U compute cell. There are a total of eight infiniband switches in the f8000 rack; four in the top of the rack and four in the bottom of the rack.

NOTE: If an infiniband switch is not installed in a switch bay, install a switch blank to ensure thermal requirements are maintained.

Page 54: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Infiniband switch 54

Infiniband switch front components and connectors

NOTE: The orientation of the infiniband switch might appear different depending on which rack slot it is installed in.

Item Component Description 1 QSFP ports Connects the infiniband switch to the outer rack network

switches via optical cables 2 Power button Powers the infiniband switch on and off

Infiniband switch port numbering Infiniband switch port numbering is used to identify or map cables within the solution.

Page 55: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Infiniband switch 55

Infiniband switch LED indicators

Item Component Description 1 Status LED Off—No power to the switch

Flashing Green—Switch is powering up Solid Green—Switch has power and normal operation Solid Amber—Non fatal error, which requires troubleshooting, but does not require shutdown Solid Red—Fatal error

2 Blank Reserved for future use 3 Bad Port LED Off—All ports are in normal operation

Flashing Amber—One or more ports has a symbol error, such as a bad cable, connection, or connector

4 UID LED Flashes to indicate identification of infiniband switch 5 Link LED Flashing Green—Data activity. The flashing speed is proportional

to the data transfer speed. Solid Amber—Physical link up Flashing Amber—Fault or error with the physical link

6 Activity LED Flashing to indicate infiniband activity 7 Health LED Solid Green—No active faults logged

Flashing Amber or Red—One or more faults has been logged. 8 Power LED Green—Normal power to the infiniband switch

Amber—Auxiliary power to the infiniband switch

Page 56: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Infiniband switch 56

Infiniband switch rear components and connectors

Item Component Description 1 High voltage DC power connector Provides power input from the power shelf to the infiniband

switch 2 Management and Auxiliary power

connector Provides infrastructure management and auxiliary power from the compute cell to the infiniband switch

3 Infiniband server cartridge connector Connects the infiniband switch to 9 servers via the cable bundle

Installing the infiniband switch 1. Align the infiniband switch to the appropriate bay within the chassis. 2. Slide the infiniband switch into the bay, and secure it with the locking tab. 3. Connect cables to the appropriate connectors in the front and rear of the infiniband switch.

Page 57: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Safety components 57

Safety components

About the safety components There are many built-in safety components throughout the Apollo 8000 System. These components are designed to prevent major catastrophic damage to your equipment in the event of an emergency. Safety components include the following:

• Primary/Facility side leak detection sensors

• Secondary leak detection sensors

• Temperature sensors

• Automatic door opening release mechanism

• (Optional) Automatic Rapid Power Off (ARPO) system

Leak detection sensors The leak detection sensors are configured to detect extra moisture in the HEX drip pans. There are three leak detection sensors in each HEX drip pan, at the bottom each HEX unit. In order for the moisture to be considered a leak, two of the three sensors must indicate excess moisture.

If the sensors indicate a leak and ARPO is installed: 1. An alarm is triggered. 2. The rack with the leak is isolated. 3. The water valve is shut off, preventing further water flow to the HEX. 4. Power is automatically shut off to the rack and rack components.

If the sensors indicate a leak and ARPO is not installed: 1. An alarm is triggered. 2. The rack with the leak is isolated. 3. The water valve is shut off, preventing further water flow to the HEX.

Temperature sensors and automatic door release mechanism

The temperature sensors located throughout the Apollo 8000 System are configured to read a stream of temperatures and indicate when a sensor exceeds an over temperature threshold.

If an over temperature threshold is exceeded within a rack, the following events occur:

• The front and rear rack doors open automatically to enable additional air flow to the rack components.

• The system fans on the HEX and compute cells ramp up to provide extra cooling to the rack components.

Page 58: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Safety components 58

(Optional) ARPO ARPO is a unique safety option that can be installed in each rack (f8000 rack and iCDU rack) for the Apollo 8000 System that enables the immediate shutdown of power to the entire rack and rack components in the event of an emergency, such as a major water leak within a powered rack.

Page 59: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Cabling 59

Cabling

Cabling overview The cabling throughout the Apollo 8000 System is quite extensive and should be handled by an experienced service technician.

The images included in this chapter are designed to give an overview of the various cabling connections required throughout the Apollo 8000 System, not to designate specific instructions for cabling the product.

The cabling scenarios differ depending on which utility module you have installed. For more information on the different models available, see "Utility module products (on page 38)."

Cabling connections when the Apollo 8000 UM1 is installed include the following:

• Connecting the utility module to the power shelves (on page 59)

• Connecting the utility module to the management modules (on page 60)

Cabling connections when the Apollo 8000 UM2 is installed include the following:

• Connecting power (on page 61)

• Connecting communication (on page 62)

• Connecting the f8000 rack to the iCDU rack (on page 65)

• Connecting the CMU solution hardware ("Connect the CMU solution hardware" on page 67)

Cabling when the HPE Apollo 8000 UM1 is installed Due to the difference of the rear connections of the two utility module versions, the cabling is very different depending on which utility module is installed.

Cabling connections when the Apollo 8000 UM1 is installed include the following:

• Connecting the utility module to the power shelves (on page 59)

• Connecting the utility module to the management modules (on page 60)

Connecting the utility module to the power shelves There are two main connection points between the utility module and each of the power shelves:

• Blue—Indicates the power connection between the power shelves and the utility module (set up for redundant power via the dual power shelves)

Page 60: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Cabling 60

• Green—Indicates the communication connection via serial port between the power shelves and the utility module (transferring data from each power shelf to APM)

Connecting the utility module to the management modules The display port cables connecting the utility module to the compute cell management modules and rack thermal controllers must be connected to the correct ports on the rear of the utility module in order for accurate data to appear in APM.

Item Component Description 1 Management module number 1 Display port 1 on the utility module connects to the bottom,

right management module in the bottom of the rack 2 Management module number 2 Display port 2 on the utility module connects to the top,

right management module in the bottom of the rack

Page 61: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Cabling 61

Item Component Description 3 Management module number 3 Display port 3 on the utility module connects to the bottom,

left management module in the bottom of the rack 4 Management module number 4 Display port 4 on the utility module connects to the top, left

management module in the bottom of the rack 5 Management module number 5 Display port 5 on the utility module connects to the bottom,

right management module in the top of the rack 6 Management module number 6 Display port 6 on the utility module connects to the top,

right management module in the top of the rack 7 Management module number 7 Display port 7 on the utility module connects to the bottom,

left management module in the top of the rack 8 Management module number 8 Display port 8 on the utility module connects to the top, left

management module in the top of the rack 9 Embedded rack controller, top HEX* Connects the utility module to the rack thermal controller in

the top of the rack 10 Embedded rack controller, bottom

HEX* Connects the utility module to the rack thermal controller in the bottom of the rack

*The rack controller is not accessible to a user, and is accessed by Hewlett Packard Enterprise Service personnel only.

Cabling when the HPE Apollo 8000 UM2 is installed Due to the difference of the rear connections of the two utility module versions, the cabling is very different depending on which utility module is installed.

Cabling connections when the Apollo 8000 UM2 is installed include the following:

• Connecting power (on page 61)

• Connecting communication (on page 62)

• Connecting the f8000 rack to the iCDU rack (on page 65)

• Connecting the CMU solution hardware ("Connect the CMU solution hardware" on page 67)

Connecting power Input power from the facility comes into the power shelves and is then filtered out to the f8000 rack components.

In many instances, the dual power shelves provide redundant power to major components.

Connect power to the rack components The power shelves provide the power the to rack components within the f8000 rack.

• Blue—Indicates the connection between the power shelf and the compute cell management modules

• Green—Indicates the connection between the power shelf and the top and bottom HEX (set up for redundant power via the dual power shelves)

Page 62: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Cabling 62

• Red—Indicates the connection between the power shelf and the utility module (set up for redundant power via the dual power shelves)

Connecting communication The components in the f8000 rack all communicate with one another, and then send the information out via the APM utility module to the network. Therefore, many connections are required to ensure the components can accurately communicate data including the following connections:

• Connect the HEX and management modules to the utility module.

• Connect the power shelves to the utility module.

• Connect the compute cell Ethernet to the rack integrated Ethernet switch on the utility module.

Connect the HEX and management modules to the utility module The utility module serves as the central point of communication for many of the rack components, and therefore has many connection points to the various components. Each of these connections uses a display port cable that transfers the information from the component to the utility module.

Page 63: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Cabling 63

• Blue—Indicates the connection between the utility module and the top and bottom f8000 rack HEX

• Black—Indicates the connection between the utility module and the management module of compute cells 7 and 8

• Purple—Indicates the connection between the utility module and the management module of compute cells 5 and 6

• Green—Indicates the connection between the utility module and the management module of compute cells 1 and 2

• Red—Indicates the connection between the utility module and the management module of compute cells 3 and 4

Page 64: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Cabling 64

Connect the power shelves to the utility module The utility module collects power information from the two power shelves and displays the information in APM.

Page 65: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Cabling 65

Connect the compute cell management modules to the utility module The multi-port Ethernet cables connecting the utility module to the compute cell must be connected to the correct ports on the rear of the utility module for accurate data to appear in APM. Two cables from each port in the rear of the utility module connect to the two Ethernet ports on each compute cell.

Connecting the f8000 rack to the iCDU rack There are a few connections between the f8000 rack and the iCDU rack that include the following:

• Connect the iCDU HEX to the power shelves in the adjacent f8000 racks.

• Connect the iCDU rack management switch to the utility module in the f8000 rack.

Page 66: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Cabling 66

Connect the iCDU HEX to the power shelves The HEX in the top of the iCDU rack receives redundant power from the power shelves in the two adjacent f8000 racks.

Page 67: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Cabling 67

Connect the iCDU rack management switch to the f8000 utility module The iCDU rack management switch communicates data from the CMU solution and the iCDU to the f8000 rack utility module via a network connection.

Connect the CMU solution hardware The CMU solution requires several specific hardware components, including the Advanced Power Manager utility module, HPE DL360e Gen8 Server, and rack management switch. These hardware components are located within the top half of the iCDU rack, and there are many connection points between them.

• Green—Indicates the network connection between the APM and the switch

• Blue—Indicates the display port connection between port 9 of the APM and the HEX

Page 68: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Cabling 68

• Red—Indicates the network connection to the iCDU (for pcoWeb interface)

Page 69: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

iCDU 69

iCDU

About the iCDU The iCDU controller evaluates, queries, and manages various measurements and warning and alarm messages from the 8000 iCDU Rack.

The controller analyzes measurements provided by the cooling unit, generates any necessary warning or alarm messages, and sends the messages to the iCDU display.

When a new warning or alarm is triggered, the following events occur:

• The warning and alarm messages appear on the iCDU display ("iCDU display components" on page 14).

• The red alarm LED lights on the iCDU appear for Warning and Critical alarms.

The iCDU monitors sensor values and utilizes PI control to modulate a VFD-controlled water pump and a facility water regulation valve to maintain both a user-selected differential pressure set point and a user-selected IT/pump-side water temperature set point. The iCDU utilizes digital on/off control incorporating hysteresis to control a vacuum pump or an air inlet valve to maintain a user-selectable vacuum pressure set point. The iCDU provides for internal communication through the Modbus protocol with a VFD, and external communication through a selectable protocol for BMS monitoring and control. It also detects and reports alarms indicating critical or non-critical operating conditions.

For more information and component locations, see "iCDU components."

iCDU setup Powering on the iCDU

IMPORTANT: Before powering on the iCDU verify: • The Hand/Auto is in the Off position. • There are no visible water leaks, especially around the connections and fittings.

To power on the iCDU: 1. Turn the iCDU power switch to the On position. Power is now supplied to the iCDU controller and

sensors. The water pump does not start until the switch is set to Auto. For the location of the power switch, see "iCDU components."

2. Press the PRG key to log in to the iCDU. o Enter password "1" to log in to CONTROL mode—enables the user to change set points

throughout the system o Enter password "3" to log in to CONFIG mode—enables the user to change set points and

additional settings throughout the system 3. Navigate to the Unit selection screen, and then select the preferred units to display.

Page 70: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

iCDU 70

4. Verify the local atmospheric pressure value is accurate, or update it if necessary.

5. Navigate to the Supply Temp PI screen and the Pmp Diff Pres PI screen and verify the rack supply

water temperature set point values are appropriate for the existing operational condition.

NOTE: Do not make changes to the following values to avoid an unstable output: • DB • Prop Band • Int Time • Deriv

Page 71: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

iCDU 71

6. Navigate through the Alarms threshold screens and verify that the threshold values are appropriate for the existing operational condition.

Page 72: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

iCDU 72

7. Verify there are no active alarms on the iCDU display. 8. Verify there are no water leaks around the iCDU. 9. Turn the iCDU Hand/Auto switch to the Auto position. 10. Navigate to the IT Iso Valve Status screen and the Fac Iso Valve Status screen to verify the iCDU

isolation valves are open on both the IT and facility side.

Page 73: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

iCDU 73

11. Navigate to the Temperature screen and Pressures screen and verify the values are within normal ranges.

12. After a short delay, the iCDU pump starts circulating water through the system. 13. Navigate to the Diff Pressure screen and verify that the pump differential pressure is no more than

20 psid.

14. Navigate to the Tank Level screen and verify that the tank water level is above 2/3 full (or at least 20

inches).

15. Allow the iCDU to run for several minutes and monitor for startup alarms that might cause a

shutdown.

Page 74: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

iCDU 74

16. Navigate back to the Temperature screen and verify that the IT supply water (HEX inlet) temperature is below 30°C (90°F).

17. If there is more than one iCDU, follow the same sequence for each iCDU, powering up each unit one

at a time. 18. Navigate to the Network Status screen and verify that there is communication between each of the

iCDUs.

19. Open the rack isolation valves, and verify that water is flowing to the rack. 20. Power on the IT racks.

Powering off the iCDU 1. Verify that it is safe to shut down the iCDU.

a. Verify the remaining iCDUs can support the existing IT load. If not, reduce the IT load before shutting down a iCDU.

b. Verify that all remaining iCDUs are within normal operating parameters and have no alarm conditions that could potentially result in the inability to support the IT load.

2. Turn the Auto/Manual switch to the Off position. The iCDU water pump stops.

3. Verify that the IT isolation valves are closed. 4. Disconnect the power to the iCDU.

BMS network communication Each iCDU can communicate externally to a BMS through a Modbus/TCP connection.

To configure the iCDU to communicate to a BMS: 1. Configure the iCDU to communicate to the pcoWeb card. 2. Update the pcoWeb card network settings:

a. Using a laptop or computer on the same network subnet, browse to the pcoWeb card default IP address: 172.16.0.1.

Page 75: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

iCDU 75

b. Under Administrator utilities, select Go to Administrator Area. c. Log in to the pcoWeb card.

— User Name—admin

— Password—fadmin

Page 76: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

iCDU 76

d. From the menu on the left side of the page, select Configuration.

e. Select the Network tab, and enter the following information:

— IP address

— Subnet mask

— Gateway

Page 77: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

iCDU 77

f. Click Submit to save the updates. g. Reboot the pcoWeb card to verify the new network settings. Then, using the new IP address,

browse back to the card and verify that the settings saved.

iCDU display Password security level

The iCDU utilizes password control to restrict access to parameters based on a security level. The password entry screen can be accessed at any time by pressing the PRG button on the iCDU display.

The available password level is Status—Password 0. No control parameters can be modified (control screens are read-only). Forced values can be cleared but not entered. Configuration screens and diagnostic screens are not displayed. The date is displayed in place of a security level on most screens.

Press ESC to exit the password entry screen.

Screen time-out Unless the Screen Lock is active, the iCDU automatically resets the security level to 0 (status) and returns to the main screen after 60 seconds.

Screen lock Screen lock prevents the iCDU from resetting the security level or returning to the Main Status screen automatically. Scroll through the screens using the up and down arrows. Reset the security level manually on the Password screen.

If screen lock is active, the selected security level screen is displayed indefinitely and a padlock icon with the letters "SL" appears in the upper right corner of most screens.

To activate the screen lock, press Enter, and then press PRG on the keypad on any screen except the main status screen. To deactivate the screen lock, press and Enter and PRG on any screen except the Main Status screen for approximately three seconds.

Always available The following screens are always available from the iCDU display ("iCDU display components" on page 14). They are used to display status and do not include modifiable parameters.

Program Screen

Example Description

Main Status

The Main Status screen summarizes the control points and status of iCDU components. The status bar indicates program name, version.

Page 78: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

iCDU 78

Program Screen

Example Description

IT Isolation Valves

The IT Isolation Valve Status screen lists the current IT isolation valve positions. There are no modifiable parameters on this screen.

Facility Isolation Valves

The Facility Isolation Valve Status screen lists the current facility isolation valve positions. There are no modifiable parameters on this screen.

Temperatures

The Temperature screen lists all available temperature sensor values in the selected unit of measure. There are no modifiable parameters on this screen.

Pressures

The Pressure screen lists all available pressure sensor values in the selected unit of measure. There are no modifiable parameters on this screen.

Page 79: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

iCDU 79

Program Screen

Example Description

Differential Pressures

The Differential Pressures screen lists three calculated differential pressures in the selected unit of measure: • Vacuum (Vacuum

Pressure – IT Supply [Out])

• Pump side heat exchanger (IT Return [In] – IT Supply [Out])

• Facility side heat exchanger (Facility Supply [In] – Facility Supply [Out])

There are no modifiable parameters on this screen.

Nominal Motor Values

The Nominal Motor Value screen lists nominal values from the water pump motor as reported by the VFD. There are no modifiable parameters on this screen.

Page 80: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

iCDU 80

Program Screen

Example Description

Modbus VFD Values 1, 2, and 3

The Modbus VFD Values screens display the actual values related to the water pump motor as reported by the VFD. If the VFD communication has failed, all values will be at zero. There are no modifiable parameters on these screens.

Network Status

The Network Status screen displays the current Network (pLAN) status, iCDU status, lead iCDU, and iCDU pLAN pulse.

Always visible The following screens are always visible from the iCDU display ("iCDU display components" on page 14). They are used to display status and can be modified at a CNTRL level or higher.

Page 81: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

iCDU 81

Program Screen

Example Description

Vacuum pump

The Vacuum Pump screen displays the current vacuum pressure, and enables users to view or modify the vacuum pressure set point (SP), cut-in, and cut-out. When the vacuum pump is activated, *pump on* flashes.

Pump differential pressure PI

The Pump Differential Pressure PI screen displays differential pressure in, VFD percent out, and enables users to view or modify the PI loop controlling the pump VFD. Differential pressure set point (SP), proportional band (Prop Band), and integral time (Int Time) can also be modified at CNTRL level or higher.

Supply temperature PI

The Supply Temperature PI screen displays pump/IT temperature in and valve percent out, and enables users to view/modify the PI loop controlling the facility water valve. The pump/IT supply temperature set point (SP), proportional band (Prop Band), integral time (Int Time) and dead band (DB) can also be modified at the CNTRL level or higher.

iCDU control Control switch

Control of the unit is managed by the HAND/OFF/AUTO control switch and through adjustable parameters for each component of the unit using the controller display. For the location of the HAND/OFF/AUTO switch, see "iCDU components."

IMPORTANT: The HAND and OFF modes are only for use by authorized service personnel.

When the switch is in AUTO mode (normal operating mode), the unit operates as described in the AUTO mode section. Additionally, PRG illuminates when the unit is in either HAND or AUTO mode, even if there are no alarms.

Page 82: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

iCDU 82

AUTO mode When the switch is in AUTO mode, the iCDU attempts to maintain pump differential pressure to the preferred set point using a PID control method. The control output modulates the pump VFD speed to maintain the pump differential pressure automatically. No manual overrides are available.

The iCDU also modulates the water regulating valve on the facility water side to control the pump discharge water temperature (IT supply water), based on the IT water outlet temperature setpoint.

iCDU isolation valves Each iCDU uses two pairs of isolation valves to isolate the iCDU from the rest of the system when it is not powered on, or if the power switch is set to the OFF position. The facility and IT isolation valve pairs open and close based on the following conditions.

Events Facility Isolation Valve IT Isolation valve Power Off Closed Closed Power On, Switch in Off position Open Closed Power On, Switch in Auto position Open Open Power On, Switch in Hand position Open Open iCDU critical shutdown (not leak related) Open Open iCDU critical shutdown (facility leak #2 detected)

Closed Closed

When the IT and facility isolation valves are open, the following status message appears.

Page 83: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

iCDU 83

When the valves are closed, the following status message appears.

The command Yes sends the signal from the iCDU to open the valve. It takes a few moments for the valves to completely open or close from their previous status. If the OPEN/CLOSED status is blank, the valves are in interim movement.

Vacuum pressure Vacuum pressure is maintained by use of a vacuum pump (to let air out) and an air inlet solenoid valve (to let air in). Both are operated by a digital signal from the controller. The vacuum pump and air inlet solenoid valve are enabled when the iCDU has power. They can operate while the pump is in OFF mode, if the setting on the Continuous Run is set to ON on the Vacuum Pump screen.

Vacuum pressure is monitored by the vacuum pressure sensor. If the pressure is too high, the pump is enabled until pressure is reduced to the cut-out pressure. If the pressure is too low, the air inlet valve is enabled until pressure is increased to the cut-out pressure.

Differential pressure Differential pressure from the vacuum tank to the pump and IT water outlet is maintained by a VFD-driven water pump. The differential pressure is calculated from the vacuum pressure sensor value subtracted from the pump and IT supply outlet water pressure. A PI loop attempts to maintain the differential pressure to the set point. If the differential is below the set point, the VFD increases the pump speed. If the differential is above the set point, the VFD decreases the pump speed.

Pump and IT supply out water temperature Supply temperature to the IT loop is maintained by modulating a water regulation valve on the facility side. A PI loop attempts to maintain the temperature at a set point. If the temperature is below the set point, the facility valve is modulated closed. If the temperature is above the set point, then the facility valve is modulated open. The facility valve is fully open when the unit is off.

Page 84: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

iCDU 84

iCDU maintenance Water testing and maintenance schedule

Water quality is essential for ensuring normal iCDU operation. The Apollo 8000 System requires that the water loop be maintained and monitored every three months by qualified personnel. For all IT or secondary water quality-related issues, contact Hewlett Packard Enterprise Service.

Page 85: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Condensation management 85

Condensation management

Condensation management

CAUTION: To reduce the risk of condensation and avoid damage to the equipment when operating with chilled water, keep the rack doors closed during operation.

Condensation should not be a concern, unless the Apollo 8000 System is operating with the doors open while running water temperatures below recommended dewpoint parameters.

Frost damage To avoid frost damage, the water temperature must not fall below the minimum permissible temperature of +4°C (+39.2°F) at any point in the water cycle.

If the Apollo 8000 System must be shipped after previous running in normal operation mode, contact Hewlett Packard Enterprise Service to assess the risk of trapped water that might be subject to freezing during transportation.

Page 86: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Software and configuration utilities 86

Software and configuration utilities

HPE Insight Cluster Management Utility The Insight CMU is an efficient and robust hyperscale cluster lifecycle management framework and suite of tools for large Linux clusters. A simple graphical interface enables an at-a-glance view of the entire cluster across multiple metrics, provides frictionless scalable remote management and analysis, and allows rapid software provisioning to all system nodes. Insight CMU makes cluster management more user friendly, efficient, and error-free than if it were being managed by scripts, or on a node-by-node basis. Insight CMU is highly flexible and customizable, offers both GUI and CLI interfaces, and is used to deploy a range of software environments, from simple compute farms to highly customized, application-specific configurations.

For more information on Insight CMU features and links to technical documentation, QuickSpecs, and a product demo, see the Hewlett Packard Enterprise website (http://www.hpe.com/info/cmu).

To download the product, go to the Hewlett Packard Enterprise Software Depot (http://www.hpe.com/support/softwaredepot). Click HP Insight Management, then click Insight Cluster Management.

HPE Advanced Power Manager The APM utility module is a point of contact for HPE ProLiant Scalable System administration.

APM enables:

• Discovery of server rack components by topology

• Node-level power monitoring and control

• Easy node and identification through UID indicators

• Support of logging through standard syslog servers

NOTE: The use of the term server represents nodes, server trays, server nodes, and cartridges.

For more information on APM, including commands specific for Apollo 8000 System, see the Advanced Power Manager User Guide for Apollo 8000 (http://www.hpe.com/support/Apollo8000_APM_UG_en).

Page 87: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Troubleshooting 87

Troubleshooting

HPE Apollo 8000 System Troubleshooting Rack and mechanical issues

Issue or Symptom Possible Cause(s) Troubleshooting The rack doors have opened automatically.

Could indicate: • High temperature event • Loss of power to the

magnetic door locks • Excessively high fan

speeds

1 Verify that the rack is receiving power. 2 Check the APM for an error indicating high

temperature. 3 Verify the magnetic door locks are

receiving power.

The rack doors will not fully close.

Could indicate: • Obstruction of door path

due to cables, servers, or other rack IT components

• Bent or un-square door frame

• Incorrect installation of brackets

• Unsecured gasketing • Loss of power to the

magnetic door locks

1 Verify all cables are properly secured using tie-wraps and cable brackets. Ensure adequate clearances between cables and the rack doors.

2 Verify all servers and IT components are fully seated and not protruding from the rack.

3 Check the installation of rear door brackets and ensure adequate clearance between the brackets and the rear rack doors.

4 Visually inspect the rack doors to make sure they have not been damaged.

5 Verify the magnetic locks are receiving power.

The doors will not open, or are hard to open.

Could indicate: • A screw holding the door

closed during installation was not removed.

• The magnetic locks are too strong.

• Obstruction from the outside of the rack, such as a neighboring rack

• Obstruction from the inside of the rack, such as a cable

1 Verify all screws have been removed from the doors upon completing rack installation.

2 Verify there is the correct amount of overlap between the door magnet and the striker plate, and the alignment creates the right amount of tension to hold the door closed.

3 Verify there is nothing obstructing or overlapping the door on the outside of the rack.

4 Verify there are no cables pinched between the rack door.

There is air leaking from the rack door.

Could indicate: • Normal air flow from the

rack, as long as not excessive

• Weakened or missing gasketing

1 Verify the gasketing is installed securely. 2 Replace any missing or weakened

gasketing.

There is gasketing material falling off the inside of the rack door.

Could indicate: • Normal wear and tear • Cables or other IT

components snagging and pulling on the gasketing

1 Verify there is nothing that caught on the gasketing and caused it to fall off the rack door.

2 Replace the gasketing.

Page 88: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Troubleshooting 88

Issue or Symptom Possible Cause(s) Troubleshooting The server cannot be installed.

Could indicate: • Various issues, depending

on which point of installation causes the problem.

1 Verify the server is correctly oriented. 2 Verify there is no obstruction in the server

slot, and nothing has previously been installed.

3 Verify the thermal busbar is fully retracted and the red interlock bracket is not blocking the guide channel.

4 Verify the bracket for the interlock device is correctly installed.

5 Check the thermal busbar on the rear of the server for scratched metal or damaged interface.

The server installation is inhibited from the start.

Could indicate: • The server slot might be

too small. • There might be something

protruding from the server. • There might be screws

misplaced in the wall rails.

1 Try installing the server in an adjacent server slot.

2 Verify there are no protruding screws from the server.

3 Verify the server lid is fully installed. 4 Verify there are no screws or other

obstructions in the server slot, including on the walls and pre-installed rails.

The server installation begins normally, but then gets stuck halfway.

Could indicate: • The heatsink screws are

not fully installed, and drag or catch on the server slot.

• The interlock bracket is broken or lose, and engages early, getting caught on the server slot.

1 Verify there are no protruding screws from the server.

2 Verify the interlock bracket is working correctly, and has not engaged with anything other than the thermal busbar.

The server does not fully install in the f8000 rack.

Could indicate: • Damaged or misaligned

connectors on the rear of the server.

• Broken server locking lever

1 Check the rear of the server for signs of damage.

2 Check the backplane in the rack for signs of damage.

3 Verify the server handle latches and locking levers are fully open.

4 Install another server into the same server slot to verify the issue is not with the server slot.

5 Install the server into a different slot to verify the issue is not with the server.

The server handle latches or locking levers do not stay closed.

Could indicate: • Damaged hardware,

including handle latches or locking levers

1 Check the functionality of the hardware, making sure they handle latch and locking lever open and close easily.

2 If the hardware is broken, call Hewlett Packard Enterprise Service.

Power issues

Issue or Symptom Possible Cause Troubleshooting The LED on one of the redundant power supplies is not illuminated.

Could indicate: • Defective power supply • Damaged power supply

connector • Power supply not fully

seated

1 Reseat the power supply. 2 Verify all high voltage DC connections on

the power shelf are working. 3 Replace the power supply.

Page 89: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Troubleshooting 89

Issue or Symptom Possible Cause Troubleshooting The LED on one of the Titan power supplies is not illuminated.

Could indicate: • Defective power supply • Damaged power supply

connector • Power supply not fully

seated

1 Remove the Titan power supply long enough for the associated fan to stop running (approximately 15 seconds).

2 Reinstall the power supply. 3 If the LED on the power supply does not

illuminate green, replace the Titan power supply.

The LED on adjacent Titan power supplies within the same power shelf are not illuminated.

Could indicate: • EPO has tripped • Defective power supply • AC input power to the

power shelf is not connected

1 Verify the AC input to the rack is connected properly and power is feeding into the rack.

2 Verify both power cords feeding the power shelf are connected.

3 Verify the breaker status on the rear of the power shelf. If a breaker has tripped, reset the breaker by pushing the post back in.

4 Replace the Titan power supplies. The LED on all four Titan power supplies within the power shelf are not illuminated.

Could indicate: • EPO has tripped • Defective power supply • AC input power to the

power shelf is not connected

1 Verify the AC input to the rack is connected properly and power is feeding into the rack.

2 Verify both power cords feeding the power shelf are connected.

3 Verify the breaker status on the rear of the power shelf. If a breaker has tripped, reset the breaker by pushing the post back in.

4 Replace the Titan power supplies. A single server tray is not receiving power.

Could indicate: • Poor connection • Broken connectors on the

server tray • Defective server tray

1 Re-seat the server tray. 2 Verify the server slot is turned on via the

APM connection. 3 Install the server tray into a different slot. 4 Replace the server tray.

A cell of server trays are not receiving power.

Could indicate: • Poor connection between

the management module and the power shelf

• A breaker tripped on the power shelf

• One or both of the redundant power supplies in the associated management module assembly are defective

1 Verify a green LED status on the redundant power supplies in the associated management module assembly.

2 Verify the breaker status on the rear of the power shelf. If a breaker has tripped, reset the breaker by pushing the post back in.

3 If the LED on the power supply does not illuminate green, replace the power supply.

The power shelf is not communicating or sending data to the APM.

Could indicate: • Poor connection • Defective component,

such as the Atlas module or Janis communication card

1 Verify all communication connections between the power shelf and the utility shelf.

2 Replace the Atlas module. 3 Replace the Janis communication card.

The power shelf is communicating to the APM, but it is missing specific data information.

Could indicate: • Poor connection • Defective component,

such as the input meter or Titan power supply

1 Verify all communication connections between the power shelf and utility shelf.

2 If the AC input data is missing, replace the input meter.

3 If the Titan data is missing, replace the Titan power supply.

Page 90: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Troubleshooting 90

Temperature and cooling issues

Issue or Symptom Possible Cause Troubleshooting There is an over-temperature alert error on one server.

Could indicate: • An issue with the server • Stress on the server • Seating of the heatsink • An issue with the thermal

busbar for that server

1 Install a different server in the same slot to verify there is no issue with the thermal busbar.

2 Follow the instructions in the server documentation to resolve the over temperature issue.

3 Call Hewlett Packard Enterprise service. There are over-temperature alert errors on multiple servers within the same cell.

Could indicate: • A bad fan within the

management module assembly

• An issue with the water wall

• A clogged filter

1 Check for errors in the APM indicating fan or power failures.

2 Check the APM for HEX and general cell status.

3 Check the sensor and inlet temperature on the rack controller board.

4 Check the power usage compared to the temperature. Determine the temperature delta.

5 Verify that water is flowing into the rack via the flow sensors.

6 Call Hewlett Packard Enterprise service. There are over-temperature alert errors on multiple servers within the half rack.

Could indicate: • A bad fan within the

management module assembly

• An issue with the heat exchanger

• A clogged filter

1 Check for errors in the APM indicating fan or power failures.

2 Check the APM for HEX and general cell status.

3 Check the sensor and inlet temperature on the rack controller board.

4 Check the power usage compared to the temperature. Determine the temperature delta.

5 Verify that water is flowing into the rack via the flow sensors.

6 Call Hewlett Packard Enterprise service. There are over-temperature alert errors on multiple servers throughout the rack.

Could indicate: • Water flow problems

within the secondary water loop

• A clogged filter

1 Check for errors in the APM indicating fan or power failures.

2 Check the APM for HEX and general cell status.

3 Check the sensor and inlet temperature on the rack controller board.

4 Check the power usage compared to the temperature. Determine the temperature delta.

5 Verify that water is flowing into the rack via the flow sensors.

6 Call Hewlett Packard Enterprise service. There are over-temperature alert errors on multiple servers throughout the cluster.

Could indicate: • An issue with the iCDU • A clogged filter

1 Check for errors in the APM indicating fan or power failures.

2 Check the APM for HEX and general cell status.

3 Check the sensor and inlet temperature on the rack controller board.

4 Check the power usage compared to the temperature. Determine the temperature delta.

5 Verify that water is flowing into the rack via the flow sensors.

6 Call Hewlett Packard Enterprise service.

Page 91: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Troubleshooting 91

Issue or Symptom Possible Cause Troubleshooting There is a high rack water inlet temperature alert error within the f8000 rack.

Could indicate: • Improper valve position • An issue with the iCDU • A clogged filter

1 Check for water temperature alarms in the APM or CMU interface.

2 Verify that water is flowing into the rack via the flow sensors.

3 Check the rack isolation valve positions. 4 Check the water wall valve positions. 5 Check the health of the iCDU. 6 Check the differential pressure to

determine if a filter needs to be replaced. 7 Call Hewlett Packard Enterprise service.

There is a low water flow alert error.

Could indicate: • Improper valve position • An issue with the iCDU • A clogged filter

1 Verify that water is flowing into the rack via the flow sensors.

2 Check the rack isolation valve positions. 3 Check the water wall valve positions. 4 Check the health of the iCDU. 5 Check the differential pressure to

determine if a filter needs to be replaced. 6 Call Hewlett Packard Enterprise service.

There is a high water temperature alert error on the iCDU rack.

Could indicate: • High facility water

temperature • Low facility water pressure • An issue with the iCDU

1 Check for water temperature alarms in the APM or CMU interface.

2 Check the rack water flow. 3 Check the facility water temperature in the

CMU interface or using the CMU controller.

4 Check the facility water differential pressure.

5 Verify the iCDU is running correctly. 6 Call Hewlett Packard Enterprise service.

There is hot air coming out of the rack.

Could indicate: • The doors are not properly

sealed. • There is gasketing

missing from a portion of the rack.

1 Determine where the hot air is coming from.

2 Verify all gasketing is in place. 3 Call Hewlett Packard Enterprise service.

Communication issues

Issue or Sympton Possible Cause Troubleshooting The utility module is not receiving information from a compute cell.

Could indicate: • Poor connection between

the utility module and the compute cell

• Damaged display port cable

1 Check that the display port cables are connected and there is no signs of physical damage.

2 Replace the display port cable.

The utility module is receiving incorrect or incomplete information from the compute cells.

Could indicate: • The display port cables

have been switched between compute cells.

• Poor connection between the utility module and the compute cell

• Damaged display port cable

1 Check that the correct compute cell display port cable is connected to the correct port on the utility module.

2 Verify there are no signs of physical damage to the display port cable.

Page 92: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Troubleshooting 92

Issue or Sympton Possible Cause Troubleshooting The utility module is not receiving information from the power shelf.

Could indicate: • Poor connection between

the utility shelf and the compute cell

• Damaged display port cable

1 Check the serial cable connection between the power shelf and the utility module, and there is no sign of physical damage.

2 Call Hewlett Packard Enterprise Service.

The utility module is receiving incorrect or incomplete information from the power shelf.

Could indicate: • The serial cables have

been switched between power shelves

• Poor connection between the utility module and the power shelf

• Damaged serial cable

1 Check the serial cable connection between the power shelf and the utility module, and there is no sign of physical damage.

2 Call Hewlett Packard Enterprise Service.

The utility module is not receiving information from the thermal controller.

Could indicate: • Poor connection between

the utility module and the thermal controller

• Damaged display port cable

1 Check that the display port cable is connected to the correct port on the utility module.

2 Verify there are no signs of physical damage to the display port cable.

I cannot log in via direct connection to the utility module.

Could indicate: • Disconnected Ethernet

cable from the utility module

• Poor APM module within the utility module

• Ethernet network is incorrectly configured

1 Check the username and password. 2 Check the configuration of the utility

module. 3 Check the connection between the utility

module and the management switch. 4 Might need to replace the utility module.

Call Hewlett Packard Enterprise Service.

I cannot log in remotely to the utility module.

Could indicate: • Disconnected Ethernet

cable from the utility module

• Poor APM module within the utility module

• Ethernet network is incorrectly configured

1 Check the username and password. 2 Check the configuration of the utility

module. 3 Check the connection between the utility

moduleand the management switch. 4 Might need to replace the utility module.

Call Hewlett Packard Enterprise Service.

I cannot communicate with the utility module in either the f8000 rack IT rack or the iCDU rack.

Could indicate: • Ethernet network is

incorrectly configured

1 Check all of the cables. 2 Check the server configuration. 3 Call Hewlett Packard Enterprise Service.

I cannot communicate with any devices through iLO.

Could indicate: • Poor connection • Damaged iLO aggregator

1 Check the connection between the management switch and the iLO aggregator pca in the 2U shelf.

2 Check the display port connection at each management module cell.

3 Might need to replace the iLO aggregator. Call Hewlett Packard Enterprise Service.

There are no link lights between the server and the management switch.

Could indicate: • Poor Ethernet cable • Server is incorrectly

configured • Management switch is

incorrectly configured • There is no power

1 Check Ethernet cable. 2 Check server configuration. 3 Verify the adapter is enabled. 4 Verify power to the management switch. 5 Call Hewlett Packard Enterprise Service.

Page 93: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Troubleshooting 93

iCDU issues

Issue or Sympton Possible Cause Troubleshooting The red light on the iCDU is illuminated.

An alarm condition on the iCDU

1 Check the iCDU display for an error message and additional information.

2 Call Hewlett Packard Enterprise Service. There are loud or strange noises coming from the iCDU.

Could indicate: • Faulty pump

1 Check the iCDU display for an error message and additional information.

2 Power down the iCDU. If you have redundancy, power down the iCDU. If you do not have redundancy, power down the f8000 rack first, and then power down the iCDU.

3 Call Hewlett Packard Enterprise Service. The iCDU has stopped running.

Could indicate: • A critical alarm has

powered down the iCDU. • An EPO event

1 Check the iCDU display for an error message and additional information.

2 Check any critical alarms on the active alarm list.

3 Call Hewlett Packard Enterprise Service. The iCDU will not power on. Could indicate:

• A pending critical alarm is still considered active.

• A loss of power to the iCDU rack

1 Verify that all active critical alarms have been cleared.

2 Verify the iCDU rack has power. 3 Call Hewlett Packard Enterprise Service.

The iCDU has stopped communicating with APM or the BMS.

Could indicate: • A network problem • Hardware issues with the

APM, network switch, or the pcoWEB card

1 Verify that the network is working correctly. 2 While on the same network with the same

subnet, ping the iCDU via the IP address. 3 Verify the network switch is working

correctly. 4 See the iCDU Communication Setup

section of the Apollo 8000 System User Guide.

5 Check the pcoWEB card functionality.* Connect directly, bypassing the network switch. If you can connect successfully, reboot the card.

Looking at the site glass of the iCDU, the water level appears low.

Could indicate: • Low water in the tank

1 If auto-fill is enabled, verify that it is working correctly.

2 If auto-fill is not enabled, fill the iCDU and check the chemistry.

3 Call Hewlett Packard Enterprise Service. There is water coming from the iCDU.

Could indicate: • A leak • High water pressure • Excess water

1 Power down the iCDU. If you have redundancy, power down the iCDU. If you do not have redundancy, power down the f8000 rack first, and then power down the iCDU.

2 Call Hewlett Packard Enterprise Service.

*An administrative username and password are required to connect directly to the pcoWEB card.

iCDU alarm troubleshooting Critical alarms

When a critical alarm is triggered, the iCDU shuts down immediately.

Page 94: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Troubleshooting 94

Alarm text Description Possible cause Action VFD VFD general alarms VFD generated alarms Check VFD display for alarm

details and then correct. Low Water Tank water is low (low

and mid level sensors detected).

Loss of water (through leakage) or imbalance of water between iCDUs

Check for water leakage.

iCDU Isolation Valve Failed to Open

— iCDU isolation valve did not open from command.

Verify the valve is functional and check wire connectivity.

VFD Current Overload VFD overload condition detected when VFD current exceeds motor Full Load Amperes (FLA) x multiplier (1.1) for at least 20 seconds

Possible pump/motor failure or overloading conditions

Verify pump equipment, and then make corrections.

Intermittent Low Differential Pressure on IT

Intermittent low pump discharge pressure condition for a least 5 cycles every minute

Potential small air leakage

Check for air leakage, and then make corrections.

Sustained Low Differential Pressure on IT

Sustained low pump discharge pressure condition for at least 5 seconds

Potential faulty pump Verify pump equipment, and then make corrections.

Warning alarms

When a warning alarm is triggered on an iCDU in lead/lag mode, operation is rotated to the standby iCDU unless the standby iCDU is experiencing the same or a more severe alarm than the lead iCDU. In parallel mode, the iCDU continues to operate.

Alarm Description Possible cause Action Leak Detection A water leak is

detected within the iCDU.

Water leak sensor detected

Check for water leakage.

Tank Fill (Mid-level Alert) Pre-low water condition (mid level sensor detected)

Loss of water (through leakage) or imbalanced amount of water between iCDUs

Check for leak or water level on iCDUs.

High Discharge Press IT water pressure at iCDU pump discharge exceeds threshold.

Clogged secondary water line

Check for clogged IT water line.

High IT (Supply) HEX Outlet Temperature

IT supply water (to racks) temperature exceeds threshold.

Facility water temperature exceeds threshold or excessive IT load.

Verify if any of the conditions exist, and then correct them.

Low Suction/Vacuum Pressure

Vacuum pressure in secondary water loop is below threshold.

Potential faulty vacuum pump or air leakage in line

Verify if any of the conditions exist, and then correct them.

Vacuum pump on and vacuum pressure greater than cut in for more than 20 minutes

Vacuum pump continuous runtime exceeds threshold.

Potential water leak condition

Check for water leakage.

Informational alarms

These alarms are informational only and do not alter the operation of the iCDU.

Alarm Description Possible cause Action Lead/Lag Rotation Failure

Lead iCDU failed attempt to rotate to lag iCDU.

Lag iCDU not available or also in similar or higher alarm status.

Check for probable causes which prevented iCDU from rotating.

Page 95: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Troubleshooting 95

Alarm Description Possible cause Action High Water Tank water level is high

(high-level sensor detected).

Imbalanced amount of water between iCDUs

Stop iCDU as part of maintenance to balance water level.

iCDU Isolation Valve Failed to Close

iCDU isolation valve did not close from command.

Malfunctioning ISO valve or valve input voltage wire is disconnected.

Verify valve is functional or wire connectivity.

VFD Comm Failure Loss of communication between the iCDU controller and the VFD. VFD can still control the pump via analog signal. VFD data is not available for monitoring.

Invalid/incorrect communication settings for VFD

Correct the communication settings for the VFD.

pLAN communication error

Loss of communication to the other iCDU in the cluster

pLAN cable disconnected, incorrect pLAN communication settings, or partner iCDU is off.

Verify pLAN cable connection or if other iCDU has power, and verify pLAN communication settings.

High facility Inlet Temperature

Facility supply water (to iCDU) temperature exceeds the threshold.

Facility supply water temperature is too high.

Reduce the temperature of the facility supply water.

Vacuum Pump Runtime Alert

Accumulated vacuum pump runtime exceeds threshold.

Potential water leak condition

Check for water leakage.

Low Facility Heat Exchanger Differential Pressure

Differential pressure on facility water side is below threshold.

Potential chiller pump failure or facility water leakage

Verify facility water condition and correct.

Using HPE APM for sensor monitoring The APM serial console can be used to monitor and control various devices connected to the controller.

Page 96: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Troubleshooting 96

Using the help command displays various information for the sensors.

The controls indicate the current status for each of the sensors displayed. However, the status of the sensors is useful in troubleshooting various errors.

NOTE: If the status displays Cable Not Detected, there is a connection issue with the particular sensor. Either the sensor is not properly connected or there is something wrong with the cable itself. Verify the cable connection.

Sensor/Controls Status Troubleshooting Leak Detector No Leaks

Leaks If Leaks: Visually inspect the rack for leaks.

Door Sensor • 0—Front door • 1—Rear door

Closed Open

If Open: 1 Verify that the door is actually open. 2 If closed, verify there is a tight connection

between the door and the sensor. 3 Verify the status of the Door Lock is set to

Closed. Door Lock • 0—Front door • 1—Rear door

Closed Open

If Open: Set to Closed.

Valve Sensor • 0—Supply • 1—Return

Open Closed

If Closed: Verify the valve sensor cable is connected.

Valve Control • 0—Supply • 1—Return

Open Closed

Page 97: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Troubleshooting 97

Sensor/Controls Status Troubleshooting RTD Temperature • 0—Supply • 1—Intermediate • 2—Return

Temperature in °Celsius Temp Out Of Range

If Temp Out Of Range:

1 Verify the sensor cable is connected. 2 Check the temperature of the supply water

from the iCDU.

H2O Pressure Pressure in nU Pressure Out Of Range

If Pressure Out Of Range:

1 Verify the sensor cable is connected. 2 Check the pressure gauges on the iCDU.

Temp-Humid-board Temp in °Celsius, % Relative Humidity Humidity Out Of Range

If Humidity Out Of Range: Verify the sensor cable is connected.

1-Wire Temp • 0—HEX inlet temp • 1—HEX outlet

temp • 6—HEX inlet temp • 7—HEX outlet

temp

Temperature in °Celsius Temp Out Of Range

If Temp Out Of Range: Verify the sensor cable is connected.

ARPO Running Stopped

If Stopped: ARPO has tripped and the Apollo 8000 System has shut down. You will not see this error displayed, for the power will be off.

HEX Power Supply Present- OK Not Present- Not OK

If Not Present-Not OK:

1 Verify the green LED on the power supply is illuminated.

2 Verify the power supply is connected. 3 Restart the power supply. 4 Replace the power supply.

Page 98: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Specifications 98

Specifications

Physical specifications The table below lists the approximate physical specifications of a single HPE f8000 rack or 8000 iCDU Rack.

f8000 rack

Parameter Packaged system (as shipped on pallet)

Unpackaged system (off pallet, unwrapped)

Height 2575 mm (102 in) 2382 mm (94 in) Width 1016 mm (40 in) 607 mm (24 in) Depth 1676 mm (66 in) 1427 mm (57 in) Weight1 1522 kg (3356 lb)2 1225 kg (2700 lb)3 Height top assembly 1253 mm (50 in) 1060 mm (42 in) Height mid and bottom assembly

1555 mm (62 in) 1362 mm (54 in)

Weight top assembly1 Up to 642 kg (1415 lb)2 Up to 413 kg (910 lb)3 Weight mid and bottom assembly1

880 kg (1941 lb)2 693 kg (1528 lb)3

1There can be a +/- 5% difference in weight 2Weight for a completely packaged system with no server trays installed 3Weight for an unpackaged system with no server trays installed

8000 iCDU Rack

Parameter Packaged system (as shipped on pallet)

Unpackaged system (off pallet, unwrapped)

Height 2575 mm (102 in) 2382 mm (94 in) Width 1016 mm (40 in) 607 mm (24 in) Depth 1676 mm (66 in) 1427 mm (57 in) Weight1 1006 kg (2218 lb)2 896 kg (1974 lb)3 Height top assembly 1253 mm (50 in) 1060 mm (42 in) Height mid and bottom assembly

1555 mm (62 in) 1362 mm (54 in)

Weight top assembly1 383 kg (844 lb)2 273 kg (600 lb)3 Weight mid and bottom assembly1

734 kg (1618 lb)2 624 kg (1374 lb)3

1There can be a +/- 5% difference in weight 2Weight for a completely packaged system with no hose kits or IT equipment installed 3Weight for an unpackaged system with no hose kits or IT equipment installed

Electrical specifications The following table lists the electrical specifications for the f8000 rack.

Page 99: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Specifications 99

Parameter Value Comments Operating Voltage: Minimum Maximum

220/380VAC, 3 phase with N + PE 277/480VAC, 3 phase with N + PE

There are two styles of input power meters: • North American—Supports 480VAC • International—Supports 380VAC to

415VAC AC line frequency 50/60 Hz — AC line phase Three True 3-phase input power conversion for

inherent phase current balancing Rated input current • 380VAC source—415VAC 32A

per input meter • 480VAC source— 24A per input

meter

Per line cord, two cords per power shelf, two shelves per IT rack

Maximum inrush current Less than 2 times the peak full load current at any line condition

Per line cord

Dropout/hold-up time at minimum line voltage

20 ms —

Circuit breaker rating • 380-415VAC source —32A • 480VAC source—30A

Per cord

Power factor > 0.98 At all loads above 20% Ground leakage current < 3.0mA Per cord Maximum power consumption (full IT rack)

79,775W capable power shelf This is the maximum power at the input of the HVDC power shelves. There are two power shelves per rack, each with dual 3-phase AC feeds.

Power cords (included with power shelves)

220/380 -240/415VAC 277/480VAC

3 meter, 8AWG, 5-wire with IEC 60309-32 plug 3 meter, 8AWG, 5-wire with NEMA L22-30P plug

The following table lists the electrical specifications for the 8000 iCDU Rack.

Parameter Value Comments Operating Voltage: Minimum Maximum

220/380VAC, 3 phase with N + PE 288/480VAC, 3 phase with N + PE

There are two CDU versions: • North American—Supports 480VAC • International—Supports 380VAC to

415VAC AC line frequency 50/60 Hz — AC line phase Three phase — Rated input current • 480VAC source—7.1 A

• 380VAC to 415VAC source—9.5 A

Per line cord

Maximum inrush current 580 A peak-to-peak available Per line cord Circuit breaker rating 15 A Per cord Power factor Typical value = 0.80 At all loads Maximum power consumption

5.7 kVA Steady state

Power cords 30A • 480VAC source—NEMA L22-30P watertight connector

• 380VAC to 415VAC source—IEC 60309 watertight connector

Maximum power cords per rack

4 —

Page 100: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Specifications 100

Thermal and air flow performance

Maximum thermal and air flow performance parameters

Single rack Dual rack (Specification per rack)

Air temperature—inlet to rack-mounted components

25ºC (68ºF) 25ºC (68ºF)

Chilled water temperature 7º–15°C (45º–57°F) 7º–15°C (45º–57°F) Total rack-mounted component air flow

2,600 cfm or less at 0 or more pressure drop across the rack-mounted components

1,300 cfm or less at 0 or more pressure drop across the rack-mounted components

Chilled water flow rate 15.5 gal/min (60 l/min) 7.9 gal/min (30 l/min) Chilled water pressure differential at flow needed to meet thermal specifications

1.0 bar delta pressure 1.0 bar delta pressure

Heat lost to room Approximately 10% maximum depending on the MCS settings and room temperature

Approximately 10% maximum depending on the MCS settings and room temperature

Server heat load 35 kW maximum 17.5 kW maximum

Environmental specifications

Features Specifications Operating temperature 5ºC to 35ºC (41ºF to 95ºF) Non-operating temperature 0ºC to 60ºC (32ºF to 140ºF) Transit temperature -30ºC to 60ºC (-22ºF to 140ºF), up to 72 hours Storage temperature -20ºC to 60ºC (-4ºF to 140ºF) Operating humidity 20 to 80% relative humidity (non-condensing) Non-operating humidity 5 to 95% relative humidity (non-condensing) Operating altitude -76.2 to 3,048 m (-250 to 10,000 ft) Non-operating altitude -76.2 to 9,144 m (-250 to 30,000 ft)

Page 101: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Safety considerations 101

Safety considerations

Important safety information

IMPORTANT: Before servicing this product, read the Important Safety Information document provided with the server.

Symbols on equipment

This symbol indicates the presence of hazardous energy circuits or electric shock hazards. Refer all servicing to qualified personnel. WARNING: To reduce the risk of injury from electric shock hazards, do not open this enclosure. Refer all maintenance, upgrades, and servicing to qualified personnel.

This symbol indicates the presence of electric shock hazards. The area contains no user or field serviceable parts. Do not open for any reason. WARNING: To reduce the risk of injury from electric shock hazards, do not open this enclosure.

This symbol on an RJ-45 receptacle indicates a network interface connection. WARNING: To reduce the risk of electric shock, fire, or damage to the equipment, do not plug telephone or telecommunications connectors into this receptacle.

This symbol indicates the presence of a hot surface or hot component. If this surface is contacted, the potential for injury exists. WARNING: To reduce the risk of injury from a hot component, allow the surface to cool before touching.

217.7 kg

480 lb

This symbol indicates that the component exceeds the recommended weight for one individual to handle safely. WARNING: To reduce the risk of personal injury or damage to the equipment, observe local occupational health and safety requirements and guidelines for manual material handling.

These symbols, on power supplies or systems, indicate that the equipment is supplied by multiple sources of power. WARNING: To reduce the risk of injury from electric shock, remove all power cords to completely disconnect power from the system.

Warnings and cautions

WARNING: This equipment must be installed in a restricted access area. Only authorized technicians trained by Hewlett Packard Enterprise should attempt to repair this equipment. All troubleshooting and repair procedures are detailed to allow only subassembly/module-level repair. Because of the complexity of the individual boards and subassemblies, no one should attempt to make repairs at the component level or to make modifications to any printed wiring board. Improper repairs can create a safety hazard..

Page 102: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Safety considerations 102

WARNING: To reduce the risk of fire, that the racks must be installed only on a concrete or other non-combustible floor.

WARNING: To reduce the risk of personal injury or damage to the equipment, be sure that: • The leveling feet are extended to the floor. • The full weight of the rack rests on the leveling feet. • The racks are coupled together. • Single rack installations are adequately secured to the building structure.

WARNING: To reduce the risk of personal injury, fire, or damage to the equipment, do not overload the AC supply branch circuit that provides power to the rack. Consult the electrical authority having jurisdiction over wiring and installation requirements of your facility. The branch circuit feeding each power cord connection must be rated as follows: • Units with NEMA L22-30 Plugs: 30A Circuit Breakers max • Units with IEC 60309-1 Plugs: 32A Circuit Breakers max.

WARNING: To reduce the risk of electric shock or damage to the equipment: • Do not disable the power cord grounding plug. The grounding plug is an important safety

feature. • Plug the power cord into a grounded (earthed) electrical outlet that is easily accessible at all

times. • Unplug the power cord from the wall outlet to disconnect power to the equipment. • Do not route the power cord where it can be walked on or pinched by items placed against

it. Pay particular attention to the plug, electrical outlet, and the point where the cord extends from the server.

WARNING: To reduce the risk of injury from electric shock, remove all power cords to completely disconnect power from the system. • Each rack has two or more power supply cords. Removing any single source of power does

not necessarily remove power from any portion of the system. When performing any service other than hot-plug module replacement, you must completely disconnect all power to that portion of the system.

• When performing service procedures on the rack, disconnect all power cords from the outlets before servicing.

WARNING: A risk of electric shock from high leakage current exists. Before connecting the AC supply to the power enclosures, be sure that the electrical outlets are properly grounded (earthed).

WARNING: To reduce the risk of personal injury from hot surfaces, allow the drives and the internal system components to cool before touching them.

Page 103: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Support and other resources 103

Support and other resources

Accessing Hewlett Packard Enterprise Support • For live assistance, go to the Contact Hewlett Packard Enterprise Worldwide website

(http://www.hpe.com/assistance).

• To access documentation and support services, go to the Hewlett Packard Enterprise Support Center website (http://www.hpe.com/support/hpesc).

Information to collect • Technical support registration number (if applicable)

• Product name, model or version, and serial number

• Operating system name and version

• Firmware version

• Error messages

• Product-specific reports and logs

• Add-on products or components

• Third-party products or components

Accessing updates • Some software products provide a mechanism for accessing software updates through the product

interface. Review your product documentation to identify the recommended software update method.

• To download product updates, go to either of the following: o Hewlett Packard Enterprise Support Center Get connected with updates page

(http://www.hpe.com/support/e-updates) o Software Depot website (http://www.hpe.com/support/softwaredepot)

• To view and update your entitlements, and to link your contracts and warranties with your profile, go to the Hewlett Packard Enterprise Support Center More Information on Access to Support Materials page (http://www.hpe.com/support/AccessToSupportMaterials).

IMPORTANT: Access to some updates might require product entitlement when accessed through the Hewlett Packard Enterprise Support Center. You must have an HP Passport set up with relevant entitlements.

Websites • Hewlett Packard Enterprise Information Library (http://www.hpe.com/info/enterprise/docs)

• Hewlett Packard Enterprise Support Center (http://www.hpe.com/support/hpesc)

• Contact Hewlett Packard Enterprise Worldwide (http://www.hpe.com/assistance)

Page 104: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Support and other resources 104

• Subscription Service/Support Alerts (http://www.hpe.com/support/e-updates)

• Software Depot (http://www.hpe.com/support/softwaredepot)

• Customer Self Repair (http://www.hpe.com/support/selfrepair)

• Insight Remote Support (http://www.hpe.com/info/insightremotesupport/docs)

• Serviceguard Solutions for HP-UX (http://www.hpe.com/info/hpux-serviceguard-docs)

• Single Point of Connectivity Knowledge (SPOCK) Storage compatibility matrix (http://www.hpe.com/storage/spock)

• Storage white papers and analyst reports (http://www.hpe.com/storage/whitepapers)

Customer Self Repair Hewlett Packard Enterprise products are designed with many Customer Self Repair (CSR) parts to minimize repair time and allow for greater flexibility in performing defective parts replacement. If during the diagnosis period Hewlett Packard Enterprise (or Hewlett Packard Enterprise service providers or service partners) identifies that the repair can be accomplished by the use of a CSR part, Hewlett Packard Enterprise will ship that part directly to you for replacement. There are two categories of CSR parts:

• Mandatory—Parts for which customer self repair is mandatory. If you request Hewlett Packard Enterprise to replace these parts, you will be charged for the travel and labor costs of this service.

• Optional—Parts for which customer self repair is optional. These parts are also designed for customer self repair. If, however, you require that Hewlett Packard Enterprise replace them for you, there may or may not be additional charges, depending on the type of warranty service designated for your product.

NOTE: Some Hewlett Packard Enterprise parts are not designed for customer self repair. In order to satisfy the customer warranty, Hewlett Packard Enterprise requires that an authorized service provider replace the part. These parts are identified as "No" in the Illustrated Parts Catalog.

Based on availability and where geography permits, CSR parts will be shipped for next business day delivery. Same day or four-hour delivery may be offered at an additional charge where geography permits. If assistance is required, you can call the Hewlett Packard Enterprise Support Center and a technician will help you over the telephone. Hewlett Packard Enterprise specifies in the materials shipped with a replacement CSR part whether a defective part must be returned to Hewlett Packard Enterprise. In cases where it is required to return the defective part to Hewlett Packard Enterprise, you must ship the defective part back to Hewlett Packard Enterprise within a defined period of time, normally five (5) business days. The defective part must be returned with the associated documentation in the provided shipping material. Failure to return the defective part may result in Hewlett Packard Enterprise billing you for the replacement. With a customer self repair, Hewlett Packard Enterprise will pay all shipping and part return costs and determine the courier/carrier to be used.

For more information about the Hewlett Packard Enterprise CSR program, contact your local service provider. For the North American program, go to the Hewlett Packard Enterprise CSR website (http://www.hpe.com/support/selfrepair).

Réparation par le client (CSR) Les produits Hewlett Packard Enterprise comportent de nombreuses pièces CSR (Customer Self Repair = réparation par le client) afin de minimiser les délais de réparation et faciliter le remplacement des pièces défectueuses. Si pendant la période de diagnostic, Hewlett Packard Enterprise (ou ses partenaires ou mainteneurs agréés) détermine que la réparation peut être effectuée à l'aide d'une pièce CSR, Hewlett Packard Enterprise vous l'envoie directement. Il existe deux catégories de pièces CSR :

Page 105: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Support and other resources 105

• Obligatoire—Pièces pour lesquelles la réparation par le client est obligatoire. Si vous demandez à Hewlett Packard Enterprise de remplacer ces pièces, les coûts de déplacement et main d'œuvre du service vous seront facturés.

• Facultatif—Pièces pour lesquelles la réparation par le client est facultative. Ces pièces sont également conçues pour permettre au client d'effectuer lui-même la réparation. Toutefois, si vous demandez à Hewlett Packard Enterprise de remplacer ces pièces, l'intervention peut ou non vous être facturée, selon le type de garantie applicable à votre produit.

REMARQUE: Certaines pièces Hewlett Packard Enterprise ne sont pas conçues pour permettre au client d'effectuer lui-même la réparation. Pour que la garantie puisse s'appliquer, Hewlett Packard Enterprise exige que le remplacement de la pièce soit effectué par un Mainteneur Agréé. Ces pièces sont identifiées par la mention "Non" dans le Catalogue illustré.

Les pièces CSR sont livrées le jour ouvré suivant, dans la limite des stocks disponibles et selon votre situation géographique. Si votre situation géographique le permet et que vous demandez une livraison le jour même ou dans les 4 heures, celle-ci vous sera facturée. Pour toute assistance, appelez le Centre d’assistance Hewlett Packard Enterprise pour qu’un technicien vous aide au téléphone. Dans les documents envoyés avec la pièce de rechange CSR, Hewlett Packard Enterprise précise s'il est nécessaire de lui retourner la pièce défectueuse. Si c'est le cas, vous devez le faire dans le délai indiqué, généralement cinq (5) jours ouvrés. La pièce et sa documentation doivent être retournées dans l'emballage fourni. Si vous ne retournez pas la pièce défectueuse, Hewlett Packard Enterprise se réserve le droit de vous facturer les coûts de remplacement. Dans le cas d'une pièce CSR, Hewlett Packard Enterprise supporte l'ensemble des frais d'expédition et de retour, et détermine la société de courses ou le transporteur à utiliser.

Pour plus d'informations sur le programme CSR de Hewlett Packard Enterprise, contactez votre Mainteneur Agrée local. Pour plus d'informations sur ce programme en Amérique du Nord, consultez le site Web Hewlett Packard Enterprise (http://www.hpe.com/support/selfrepair).

Riparazione da parte del cliente Per abbreviare i tempi di riparazione e garantire una maggiore flessibilità nella sostituzione di parti difettose, i prodotti Hewlett Packard Enterprise sono realizzati con numerosi componenti che possono essere riparati direttamente dal cliente (CSR, Customer Self Repair). Se in fase di diagnostica Hewlett Packard Enterprise (o un centro di servizi o di assistenza Hewlett Packard Enterprise) identifica il guasto come riparabile mediante un ricambio CSR, Hewlett Packard Enterprise lo spedirà direttamente al cliente per la sostituzione. Vi sono due categorie di parti CSR:

• Obbligatorie—Parti che devono essere necessariamente riparate dal cliente. Se il cliente ne affida la riparazione ad Hewlett Packard Enterprise, deve sostenere le spese di spedizione e di manodopera per il servizio.

• Opzionali—Parti la cui riparazione da parte del cliente è facoltativa. Si tratta comunque di componenti progettati per questo scopo. Se tuttavia il cliente ne richiede la sostituzione ad Hewlett Packard Enterprise, potrebbe dover sostenere spese addizionali a seconda del tipo di garanzia previsto per il prodotto.

NOTA: alcuni componenti Hewlett Packard Enterprise non sono progettati per la riparazione da parte del cliente. Per rispettare la garanzia, Hewlett Packard Enterprise richiede che queste parti siano sostituite da un centro di assistenza autorizzato. Tali parti sono identificate da un "No" nel Catalogo illustrato dei componenti.

In base alla disponibilità e alla località geografica, le parti CSR vengono spedite con consegna entro il giorno lavorativo seguente. La consegna nel giorno stesso o entro quattro ore è offerta con un supplemento di costo solo in alcune zone. In caso di necessità si può richiedere l'assistenza telefonica di un addetto del centro di supporto tecnico Hewlett Packard Enterprise. Nel materiale fornito con una parte di ricambio CSR, Hewlett Packard Enterprise specifica se il cliente deve restituire dei component. Qualora sia richiesta la resa ad Hewlett Packard Enterprise del componente difettoso, lo si deve spedire ad Hewlett Packard Enterprise entro un determinato periodo di tempo, generalmente cinque (5) giorni lavorativi. Il componente difettoso deve essere restituito con la documentazione associata nell'imballo di

Page 106: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Support and other resources 106

spedizione fornito. La mancata restituzione del componente può comportare la fatturazione del ricambio da parte di Hewlett Packard Enterprise. Nel caso di riparazione da parte del cliente, Hewlett Packard Enterprise sostiene tutte le spese di spedizione e resa e sceglie il corriere/vettore da utilizzare.

Per ulteriori informazioni sul programma CSR di Hewlett Packard Enterprise, contattare il centro di assistenza di zona. Per il programma in Nord America fare riferimento al sito Web (http://www.hpe.com/support/selfrepair).

Customer Self Repair Hewlett Packard Enterprise Produkte enthalten viele CSR-Teile (Customer Self Repair), um Reparaturzeiten zu minimieren und höhere Flexibilität beim Austausch defekter Bauteile zu ermöglichen. Wenn Hewlett Packard Enterprise (oder ein Hewlett Packard Enterprise Servicepartner) bei der Diagnose feststellt, dass das Produkt mithilfe eines CSR-Teils repariert werden kann, sendet Ihnen Hewlett Packard Enterprise dieses Bauteil zum Austausch direkt zu. CSR-Teile werden in zwei Kategorien unterteilt:

• Zwingend—Teile, für die das Customer Self Repair-Verfahren zwingend vorgegeben ist. Wenn Sie den Austausch dieser Teile von Hewlett Packard Enterprise vornehmen lassen, werden Ihnen die Anfahrt- und Arbeitskosten für diesen Service berechnet.

• Optional—Teile, für die das Customer Self Repair-Verfahren optional ist. Diese Teile sind auch für Customer Self Repair ausgelegt. Wenn Sie jedoch den Austausch dieser Teile von Hewlett Packard Enterprise vornehmen lassen möchten, können bei diesem Service je nach den für Ihr Produkt vorgesehenen Garantiebedingungen zusätzliche Kosten anfallen.

HINWEIS: Einige Hewlett Packard Enterprise Teile sind nicht für Customer Self Repair ausgelegt. Um den Garantieanspruch des Kunden zu erfüllen, muss das Teil von einem Hewlett Packard Enterprise Servicepartner ersetzt werden. Im illustrierten Teilekatalog sind diese Teile mit „No“ bzw. „Nein“ gekennzeichnet.

CSR-Teile werden abhängig von der Verfügbarkeit und vom Lieferziel am folgenden Geschäftstag geliefert. Für bestimmte Standorte ist eine Lieferung am selben Tag oder innerhalb von vier Stunden gegen einen Aufpreis verfügbar. Wenn Sie Hilfe benötigen, können Sie das Hewlett Packard Enterprise Support Center anrufen und sich von einem Mitarbeiter per Telefon helfen lassen. Den Materialien von Hewlett Packard Enterprise, die mit einem CSR-Ersatzteil geliefert werden, können Sie entnehmen, ob das defekte Teil an Hewlett Packard Enterprise zurückgeschickt werden muss. Wenn es erforderlich ist, das defekte Teil an Hewlett Packard Enterprise zurückzuschicken, müssen Sie dies innerhalb eines vorgegebenen Zeitraums tun, in der Regel innerhalb von fünf (5) Geschäftstagen. Das defekte Teil muss mit der zugehörigen Dokumentation in der Verpackung zurückgeschickt werden, die im Lieferumfang enthalten ist. Wenn Sie das defekte Teil nicht zurückschicken, kann Hewlett Packard Enterprise Ihnen das Ersatzteil in Rechnung stellen. Im Falle von Customer Self Repair kommt Hewlett Packard Enterprise für alle Kosten für die Lieferung und Rücksendung auf und bestimmt den Kurier-/Frachtdienst.

Weitere Informationen über das Hewlett Packard Enterprise Customer Self Repair Programm erhalten Sie von Ihrem Servicepartner vor Ort. Informationen über das CSR-Programm in Nordamerika finden Sie auf der Hewlett Packard Enterprise Website unter (http://www.hpe.com/support/selfrepair).

Reparaciones del propio cliente Los productos de Hewlett Packard Enterprise incluyen muchos componentes que el propio usuario puede reemplazar (Customer Self Repair, CSR) para minimizar el tiempo de reparación y ofrecer una mayor flexibilidad a la hora de realizar sustituciones de componentes defectuosos. Si, durante la fase de diagnóstico, Hewlett Packard Enterprise (o los proveedores o socios de servicio de Hewlett Packard Enterprise) identifica que una reparación puede llevarse a cabo mediante el uso de un componente CSR, Hewlett Packard Enterprise le enviará dicho componente directamente para que realice su sustitución. Los componentes CSR se clasifican en dos categorías:

Page 107: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Support and other resources 107

• Obligatorio—componentes cuya reparación por parte del usuario es obligatoria. Si solicita a Hewlett Packard Enterprise que realice la sustitución de estos componentes, tendrá que hacerse cargo de los gastos de desplazamiento y de mano de obra de dicho servicio.

• Opcional—componentes cuya reparación por parte del usuario es opcional. Estos componentes también están diseñados para que puedan ser reparados por el usuario. Sin embargo, si precisa que Hewlett Packard Enterprise realice su sustitución, puede o no conllevar costes adicionales, dependiendo del tipo de servicio de garantía correspondiente al producto.

NOTA: Algunos componentes de Hewlett Packard Enterprise no están diseñados para que puedan ser reparados por el usuario. Para que el usuario haga valer su garantía, Hewlett Packard Enterprise pone como condición que un proveedor de servicios autorizado realice la sustitución de estos componentes. Dichos componentes se identifican con la palabra "No" en el catálogo ilustrado de componentes.

Según la disponibilidad y la situación geográfica, los componentes CSR se enviarán para que lleguen a su destino al siguiente día laborable. Si la situación geográfica lo permite, se puede solicitar la entrega en el mismo día o en cuatro horas con un coste adicional. Si precisa asistencia técnica, puede llamar al Centro de asistencia técnica de Hewlett Packard Enterprise y recibirá ayuda telefónica por parte de un técnico. Con el envío de materiales para la sustitución de componentes CSR, Hewlett Packard Enterprise especificará si los componentes defectuosos deberán devolverse a Hewlett Packard Enterprise. En aquellos casos en los que sea necesario devolver algún componente a Hewlett Packard Enterprise, deberá hacerlo en el periodo de tiempo especificado, normalmente cinco días laborables. Los componentes defectuosos deberán devolverse con toda la documentación relacionada y con el embalaje de envío. Si no enviara el componente defectuoso requerido, Hewlett Packard Enterprise podrá cobrarle por el de sustitución. En el caso de todas sustituciones que lleve a cabo el cliente, Hewlett Packard Enterprise se hará cargo de todos los gastos de envío y devolución de componentes y escogerá la empresa de transporte que se utilice para dicho servicio.

Para obtener más información acerca del programa de Reparaciones del propio cliente de Hewlett Packard Enterprise, póngase en contacto con su proveedor de servicios local. Si está interesado en el programa para Norteamérica, visite la página web de Hewlett Packard Enterprise CSR (http://www.hpe.com/support/selfrepair).

Customer Self Repair Veel onderdelen in Hewlett Packard Enterprise producten zijn door de klant zelf te repareren, waardoor de reparatieduur tot een minimum beperkt kan blijven en de flexibiliteit in het vervangen van defecte onderdelen groter is. Deze onderdelen worden CSR-onderdelen (Customer Self Repair) genoemd. Als Hewlett Packard Enterprise (of een Hewlett Packard Enterprise Service Partner) bij de diagnose vaststelt dat de reparatie kan worden uitgevoerd met een CSR-onderdeel, verzendt Hewlett Packard Enterprise dat onderdeel rechtstreeks naar u, zodat u het defecte onderdeel daarmee kunt vervangen. Er zijn twee categorieën CSR-onderdelen:

• Verplicht—Onderdelen waarvoor reparatie door de klant verplicht is. Als u Hewlett Packard Enterprise verzoekt deze onderdelen voor u te vervangen, worden u voor deze service reiskosten en arbeidsloon in rekening gebracht.

• Optioneel—Onderdelen waarvoor reparatie door de klant optioneel is. Ook deze onderdelen zijn ontworpen voor reparatie door de klant. Als u echter Hewlett Packard Enterprise verzoekt deze onderdelen voor u te vervangen, kunnen daarvoor extra kosten in rekening worden gebracht, afhankelijk van het type garantieservice voor het product.

OPMERKING: Sommige Hewlett Packard Enterprise onderdelen zijn niet ontwikkeld voor reparatie door de klant. In verband met de garantievoorwaarden moet het onderdeel door een geautoriseerde Service Partner worden vervangen. Deze onderdelen worden in de geïllustreerde onderdelencatalogus aangemerkt met "Nee".

Afhankelijk van de leverbaarheid en de locatie worden CSR-onderdelen verzonden voor levering op de eerstvolgende werkdag. Levering op dezelfde dag of binnen vier uur kan tegen meerkosten worden aangeboden, indien dit mogelijk is gezien de locatie. Indien assistentie is gewenst, belt u het Hewlett Packard Enterprise Support Center om via de telefoon ondersteuning van een technicus te ontvangen.

Page 108: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Support and other resources 108

Hewlett Packard Enterprise vermeldt in de documentatie bij het vervangende CSR-onderdeel of het defecte onderdeel aan Hewlett Packard Enterprise moet worden geretourneerd. Als het defecte onderdeel aan Hewlett Packard Enterprise moet worden teruggezonden, moet u het defecte onderdeel binnen een bepaalde periode, gewoonlijk vijf (5) werkdagen, retourneren aan Hewlett Packard Enterprise. Het defecte onderdeel moet met de bijbehorende documentatie worden geretourneerd in het meegeleverde verpakkingsmateriaal. Als u het defecte onderdeel niet terugzendt, kan Hewlett Packard Enterprise u voor het vervangende onderdeel kosten in rekening brengen. Bij reparatie door de klant betaalt Hewlett Packard Enterprise alle verzendkosten voor het vervangende en geretourneerde onderdeel en kiest Hewlett Packard Enterprise zelf welke koerier/transportonderneming hiervoor wordt gebruikt.

Neem contact op met een Service Partner voor meer informatie over het Customer Self Repair programma van Hewlett Packard Enterprise. Informatie over Service Partners vindt u op de Hewlett Packard Enterprise website (http://www.hpe.com/support/selfrepair).

Reparo feito pelo cliente Os produtos da Hewlett Packard Enterprise são projetados com muitas peças para reparo feito pelo cliente (CSR) de modo a minimizar o tempo de reparo e permitir maior flexibilidade na substituição de peças com defeito. Se, durante o período de diagnóstico, a Hewlett Packard Enterprise (ou fornecedores/parceiros da Hewlett Packard Enterprise) concluir que o reparo pode ser efetuado pelo uso de uma peça CSR, a Hewlett Packard Enterprise enviará a peça diretamente ao cliente. Há duas categorias de peças CSR:

• Obrigatória—Peças cujo reparo feito pelo cliente é obrigatório. Se desejar que a Hewlett Packard Enterprise substitua essas peças, serão cobradas as despesas de transporte e mão-de-obra do serviço.

• Opcional—Peças cujo reparo feito pelo cliente é opcional. Essas peças também são projetadas para o reparo feito pelo cliente. No entanto, se desejar que a Hewlett Packard Enterprise as substitua, pode haver ou não a cobrança de taxa adicional, dependendo do tipo de serviço de garantia destinado ao produto.

OBSERVAÇÃO: Algumas peças da Hewlett Packard Enterprise não são projetadas para o reparo feito pelo cliente. A fim de cumprir a garantia do cliente, a Hewlett Packard Enterprise exige que um técnico autorizado substitua a peça. Essas peças estão identificadas com a marca "No" (Não), no catálogo de peças ilustrado.

Conforme a disponibilidade e o local geográfico, as peças CSR serão enviadas no primeiro dia útil após o pedido. Onde as condições geográficas permitirem, a entrega no mesmo dia ou em quatro horas pode ser feita mediante uma taxa adicional. Se precisar de auxílio, entre em contato com o Centro de suporte técnico da Hewlett Packard Enterprise para que um técnico o ajude por telefone. A Hewlett Packard Enterprise especifica nos materiais fornecidos com a peça CSR de reposição se a peça com defeito deve ser devolvida à Hewlett Packard Enterprise. Nos casos em que isso for necessário, é preciso enviar a peça com defeito à Hewlett Packard Enterprise, você deverá enviar a peça com defeito de volta para a Hewlett Packard Enterprise dentro do período de tempo definido, normalmente em 5 (cinco) dias úteis. A peça com defeito deve ser enviada com a documentação correspondente no material de transporte fornecido. Caso não o faça, a Hewlett Packard Enterprise poderá cobrar a reposição. Para as peças de reparo feito pelo cliente, a Hewlett Packard Enterprise paga todas as despesas de transporte e de devolução da peça e determina a transportadora/serviço postal a ser utilizado.

Para obter mais informações sobre o programa de reparo feito pelo cliente da Hewlett Packard Enterprise, entre em contato com o fornecedor de serviços local. Para o programa norte-americano, visite o site da Hewlett Packard Enterprise (http://www.hpe.com/support/selfrepair).

Page 109: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Support and other resources 109

Page 110: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Support and other resources 110

Page 111: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Support and other resources 111

Page 112: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Support and other resources 112

Remote support Remote support is available with supported devices as part of your warranty or contractual support agreement. It provides intelligent event diagnosis, and automatic, secure submission of hardware event notifications to Hewlett Packard Enterprise, which will initiate a fast and accurate resolution based on your product’s service level. Hewlett Packard Enterprise strongly recommends that you register your device for remote support.

For more information and device support details, go to the Insight Remote Support website (http://www.hpe.com/info/insightremotesupport/docs).

Page 113: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Warranty and regulatory information 113

Warranty and regulatory information

Warranty information HPE ProLiant and x86 Servers and Options (http://www.hpe.com/support/ProLiantServers-Warranties)

HPE Enterprise Servers (http://www.hpe.com/support/EnterpriseServers-Warranties)

HPE Storage Products (http://www.hpe.com/support/Storage-Warranties)

HPE Networking Products (http://www.hpe.com/support/Networking-Warranties)

Regulatory information Safety and regulatory compliance

For important safety, environmental, and regulatory information, see Safety and Compliance Information for Server, Storage, Power, Networking, and Rack Products, available at the Hewlett Packard Enterprise website (http://www.hpe.com/support/Safety-Compliance-EnterpriseProducts).

Belarus Kazakhstan Russia marking

Manufacturer and Local Representative Information

Manufacturer information: Hewlett Packard Enterprise Company, 3000 Hanover Street, Palo Alto, CA 94304 U.S.

Local representative information Russian:

• Russia:

• Belarus:

• Kazakhstan:

Local representative information Kazakh:

Page 114: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Warranty and regulatory information 114

• Russia:

• Belarus:

• Kazakhstan:

Manufacturing date: The manufacturing date is defined by the serial number.

CCSYWWZZZZ (serial number format for this product)

Valid date formats include:

• YWW, where Y indicates the year counting from within each new decade, with 2000 as the starting point; for example, 238: 2 for 2002 and 38 for the week of September 9. In addition, 2010 is indicated by 0, 2011 by 1, 2012 by 2, 2013 by 3, and so forth.

• YYWW, where YY indicates the year, using a base year of 2000; for example, 0238: 02 for 2002 and 38 for the week of September 9.

Turkey RoHS material content declaration

Ukraine RoHS material content declaration

Page 115: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Acronyms and abbreviations 115

Acronyms and abbreviations

ARPO Automatic Rapid Power Off

BMS building management system

CMU HPE Insight Cluster Management Utility

HEX heat exchanger

HPE APM HPE Advanced Power Manager

iCDU Intelligent Cooling Distribution Unit

PI proportional integral

PID proportional integral differential

PRG program

RTD resistance temperature detector

TBB thermal bus bar

VFD variable frequency drive

Page 116: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Documentation feedback 116

Documentation feedback

Hewlett Packard Enterprise is committed to providing documentation that meets your needs. To help us improve the documentation, send any errors, suggestions, or comments to Documentation Feedback (mailto:[email protected]). When submitting your feedback, include the document title, part number, edition, and publication date located on the front cover of the document. For online help content, include the product name, product version, help edition, and publication date located on the legal notices page.

Page 117: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Index 117

A ARPO 58 authorized reseller 103 AUTO mode 81, 82

B before you contact Hewlett Packard Enterprise 103 BMS, connections 74

C cabling 59 CMU 67, 86, 114 compute cell 26, 27, 29, 65 compute cell numbering 28 condensation management 85 configuration 6, 15 configuration of system 86 connecting communication 62 connecting power 61, 64 connecting the CMU solution 67 connecting the management module 60, 62 connecting the power shelves 59, 61, 64, 66 connecting the rack integrated switch 65, 67 connecting the racks 65 connecting the utility module 59, 60, 62, 64 contacting Hewlett Packard Enterprise 103 control switch 81 customer self repair (CSR) 104

D differential pressure 83 display modes 77, 80

E electrical specifications 98 environmental specifications 100

F fan numbering 30 firmware, updating 50 frost damage 85

H HAND mode 81 Hewlett Packard Enterprise Technical Support 103

HEX 49, 51, 114 HPE Advanced Power Manager 114 HPE APM sensor monitoring 95 HPE Insight Cluster Management Utility 86

I iCDU 69, 74, 77, 81, 84, 114 iCDU components 14, 69, 74, 77 iCDU control 81, 82 iCDU display 77 iCDU HEX connections 66 iCDU maintenance 93 iCDU rack management switch 67 iCDU troubleshooting 93 infiniband switch 53, 56 infiniband switch port numbering 54 infiniband switch, components 54, 56 installation services 15 installing hardware 21, 24, 35, 46, 56 isolation valve 82

L Leak detector sensor is not working properly 57 Leakage detected 57 LED indicators 19, 24, 29, 40, 44, 46, 55 LEDs, troubleshooting 87 locks, rack door 52

M management module 35

N numbering, fans 30 numbering, infiniband switch ports 54

O overview, product 6

P passwords 77 PI loop 83 power 69, 74 power components 17, 18 power considerations 69, 74 power redundancy 23, 24 power shelf components 18, 20, 21 power shelf procedures 21, 22, 24

Index

Page 118: HPE Apollo 8000 System User Guideh20628. · 2017-10-13 · Accessing updates ... • Installation verification tests (IVT) ... IMPORTANT: The following table is a general overview

Index 118

power shelf, installation 21, 24 power supply 23, 24 power usage 24 product overview 6 program screen 77, 80

R rack components 11 rack door sensors and locks 52 regulatory information 112

S safety information 101, 112 safety, component 57 screen lock 77 screen time-out 77 security 77 security settings 77 sensors, temperature 57 service information 15 solution, CMU 67 specifications 98, 100 specifications, environmental 98 support 103 support and other resources 103

T technical support 103 telephone numbers 103 thermal air flow performance 100 Thermal componets 48, 50 Thermal controller 50 troubleshooting 87

U utility module 37, 38, 39, 44, 46 utility module, components 39, 41, 45 utility module, connectors 40, 42 utility module, different models 38, 39 utility module, location 37 utility module, specifications 46

V vacuum pressure 83 vacuum pump 83

W water maintenance 84 water quality 85 water temperature 83 water testing 85 Water walls 49, 51