©2009 HP Confidential
Ed Ansett
Critical Facilities Services
HP Technology Consulting
24th May 2011
Using Data Centre Energy
Benchmarking to Enhance
Efficiency & Performance
Coypright HP 2011 - Subject to Change Without Notice HP Confidential
Background
• Data centres consume vast amounts of power
• Typically 30x an office of comparable size
• In 2008 data centres accounted for about 1.5% of US power consumption
• Server/storage capacity has roughly doubled since then
• Typical LEGACY data centre power efficiency (DCiE) stand at
Coypright HP 2011 - Subject to Change Without Notice HP Confidential
Legacy Data Centre Issues
• Energy costs nowadays dominate OPEX and
TCO; yet when they were designed energy
efficiency was not a consideration
• Pressure on owners to measure, manage and
reduce energy efficiency
• Pressure to make data centres last longer
• Virtualization limited by power and cooling
available
• Built to the highest Tier Level - resulting in over-
engineering
• IT and Facilities misaligned
Coypright HP 2011 - Subject to Change Without Notice HP Confidential
Technology Facility ParadoxProblem Definition
‘Technology is typically changed every 3 years. Data Centre lifetime is
typically15 - 20 years’
• Assuming a facility lifetime of 20 years: Tech refresh at yrs 3,6,9,12,15 & 18 –
Six refreshes
• Assuming a facility lifetime of 15 years: Tech refresh at yrs 3,6,9,12 - FOUR
refreshes
Coypright HP 2011 - Subject to Change Without Notice HP Confidential
Future Generation Technology Refreshes
Moore’s law intact for next 10 years
• 130 nm → 90nm → 45nm → 30 nm → ?
Transistor density – 16G devices/chip by 2013
• Molecular devices ?
• Quantum devices ?
• Single electron transistors ?
Disk Media – Capacity doubles every 16 months
Memory – new materials
• Flash memory, Ferro-electric (FeRAM)
• Polymer, Magnetoresistive RAM (MRAM)
• Tunneling Magnetic Junction RAM (TMJ-RAM)
Guilder’s law
• Bandwidth doubles every 12-18 months
Nanotechnology
• Memristor - Fourth electronic circuitry element
• Real-time data analysis with low-cost, self-powered sensors
Higher Performance =>
COMPACTION =>
• Higher power density
• Automation
• Optimised Infrastructure
• Dynamic resource allocation
Coypright HP 2011 - Subject to Change Without Notice HP Confidential
Data Centre Environmental Performance Standards, Guidelines & Metrics - Gaining Acceptance
• EU Code of Conduct for Data Centres
• Singapore Standard SS 564: 2010
• ASHRAE TC9.9 2011 Thermal Guidelines for Data Centres
• EPA Energy Star Rating for Data Centres
• LEED / BREEAM
• PUE – Power Usage Effectiveness
• DCiE – Data Centre Infrastructure Efficiency
• WUE – Water Usage Effectiveness
• CUE – Carbon Usage Effectiveness
• TP-E/C/H – Transaction Processing
Coypright HP 2011 - Subject to Change Without Notice HP Confidential7 1 June 2011
PUE (Power Usage Effectiveness) DCiE (Data Centre infrastructure Efficiency) DCiE=1/PUE
Emergency generator
Utility transforme
r
Total data center power
(kw)
Data center power usage effectiveness (PUE)
Total data center power (kw)
Total IT power (kw)
Main
power
House
power
Misc.
power
IT
power In
Mech.
power
Total data center power
(kw)
House power (kw)
(+)=
(+) (+) =
=
Data center
Main service
PP-Gen service
Lighting panel
Generator block Heaters & lighting
Lighting, Exit lights, BMS, EPMS & security
House panels
UPS
STS
PDUs
RPPs
Mech. Swgear
Data centerCRAC units,
AHUs, Chillers & Dry coolers
Misc. support Mech. support
Power
To IT
IT-critical load
Coypright HP 2011 - Subject to Change Without Notice HP Confidential
Data Centre Energy Efficiency Assessment
• Provides metrics of data centre facility’s energy efficiency
• Provides power efficiency benchmarking data of facility in comparison
with other facilities
• Determines the carbon footprint of your facility
• Identifies mechanical and electrical sources of inefficiency
• Identifies operational and maintenance practices that may affect
energy efficiency
• Provides understanding of mechanical and electrical best practices that
can increase energy efficiency
• Outlines mechanical and electrical concepts that may improve
efficiency with associated high-level cost-benefit analysis
Coypright HP 2011 - Subject to Change Without Notice HP Confidential
EEA Preparation:
• Obtain MEP documents & Utility data
• Determine an approach to the Audit (Hi-Level or Detailed audit)
• Create EEA file with equipment according to drawings (enter all design data) at this stage
• Define test and measurement points for all equipment
EEA Execution:
• Site Visit and kick-off meeting/presentation
• Conduct initial walk-through & ensure as-built condition
• Collect Data from name plate/ manuals/BMS
• Prepare instrumentation & gather power/Temp measurements
• Analyse data
EEA Report:
• Perform detailed analysis and evaluation
• Analyse what if scenarios
• Insert additional data into the EEA tool if needed
• Formulate findings, observations and recommendations
• Report Output
Audit Preparation
1 week
Step-1
Audit Execution
1 week
Step-2
Audit Report
4 weeks
Step-3
6 WEEKS
Data Centre Energy Efficiency Assessment
Coypright HP 2011 - Subject to Change Without Notice HP Confidential
PUE Results
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
Percent of Total Facility Power
Facility Number
ICT Load (%) Mechanical (%) Electrical Losses (%) Electrical and Lighting (%)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
Percent of Total Facility Power
Facility Number
ICT Load (%) Mechanical (%) Electrical Losses (%) Electrical and Lighting (%)
Average Singapore PUE = 2.39
DCiE = 0.42
©2009 HP Confidential11
Data Centre Benchmarking - Key Findings
•High Bypass and Recirculation
•Mixed aisle configurations
•No blanking panels/cable skirts
•No air management strategies
•Data centre too cold
•UPS rooms too cold
•Too many AC units/Turn some AC units off
•Controlled on return air
•Low air temp & %RH set points!
•Fixed speed fans, no VFDs
•Old fan motors with low efficiency
• No centralized communications
•Mixed up flow/down flow units
• Low chilled water set points
• No VFDs on pumps/CT fans
• Plant maintenance issues
• Plant age
• Chillers sequencing/staging
• Fixed speed chillers, no VFDs
• No free cooling
• No condenser water temp reset
• Low UPS loads “load factor”
• Plant efficiency, age
• Maintenance issues
• Lights ON all the time in data
centers
−No occupancy zoned LED lights
−More lights require more cooling
©2009 HP Confidential12
Typical Output
Saving energy reduces OPEX which increases the bottom line. Often the ROI is a
Matter of weeks or months rather than years:
Recommendations
Est. first cost or incremental cost increase
Est. annual elec. savings
(kWh)
Est. annual cost savings
(USD)
Simple payback (years)
Increase UPS room temperature setpoint – $101,376 $12,064 Immediate
Increase switchgear room temperature setpoint – $30,660 $3,649 Immediate
Reduce condenser water temperature setpoint (verify with chiller manufacturer) – $37,450 $4,457 Immediate
Increase CRAC return air set point and tolerance range – $51,500 $6,100 Immediate
Shut down 3 CRAC units – $109,620 $13,000 Immediate
Shut down 44 CRAC units – $1,599,600 $190,350 Immediate
Raise primary chilled water temperature setpoint – $198,000 $23,500 Immediate
Disable all individual CRAC infrared humidifiers and rely on centralized humidification in main AHUs – $525,600 $62,500 Immediate
Widen CRAC unit return air humidity deadband to meet ASHRAE standards $3,000 $158,000 $18,800 0.2
Add air side economizers to AHUs $24,000 $1,260,400 $150,000 0.2
Use liquid-cooled server cabinets in lieu of cabinets cooled by rejecting heat to data center (large DC) $85,000 $1,280,000 $128,000 0.7
Use chillers with variable speed compressors in lieu of constant speed compressors with solid state starter $300,000 $6,000,000 $360,000 0.8
Shut off 3 CRAC/H units $5,000 $62,500 $5,000 1.0
Fast ROI
©2009 HP Confidential13
Thermal Scans – Computer Racks
1 1 CF L O OROP E N I N G2 0 CS ER V ER SE X H A U ST
HOT AISLE SHORT CIRCUITING
FROM UNSEALED CABLE
CUTOUTS IN THE RAISED
FLOOR
H O TA I S L E
H O TA I S L EHOT AISLE SHORT CIRCUITING
FROM UNSEALED CABLE
CUTOUT DIRECTLY BELOW
EMPTY SERVER RACK
1 3 C A I RC O M I N G FR O MI N S I D E A NE M P T Y S ER V ERR A CK
2 0 CS ER V ER SE X H A U ST
©2009 HP Confidential1414 1 June 2011
Aisle ContainmentExample: Enclosed Hot Aisle Layout
• Air supplied from side walls down cold aisle
• Air is returned above drop ceiling
• Enclosed Hot Aisle
14
• Pros:
• Eliminates Bypass Airflow
• Eliminates Recirculation/ Hot Spots
• Allows Variable Supply Airflow to Match Server Airflow
• Lower Energy Cost
• Cons:
• First Cost of Enclosures
• Additional Lighting and Fire Protection Devices
Coypright HP 2011 - Subject to Change Without Notice HP Confidential
Server Level
Consider cloud SaaS, IaaS, EaaS etc. There are huge opportunities for
Energy savings provided server utilization is high.
Coypright HP 2011 - Subject to Change Without Notice HP Confidential
Need to look at entire life cycle to determine total energy and environmental impact
Includes material embodied energy and operating phase contributions
Techniques are not new, but data sets and standardized techniques are still emerging
Embedded Energy
Life Cycle Assessment Concepts
Analyzing total environmental foot print
General findings:
• The operational phase of a data center has the greatest impact to life cycle energy use and CO2 emissions
• Materials used in structural and HVAC systems are the two largest contributors when looking at embodied energy
Coypright HP 2011 - Subject to Change Without Notice HP Confidential
Embedded Energy in a Data Centre
Coypright HP 2011 - Subject to Change Without Notice HP Confidential
Air Management Objective: Increase air & chw set points- Minimise Negative flow and Bypass- Minimise Recirculation
Mechanical- Free Cooling (air or chw)- Plant / system optimise
- Chillers- CRACs- Humidifiers, etc
Electric performance- UPS- Gen heaters - Lights- dc power
Renewable power (mains /on-site)- Mains (wind, hydro)- Site (bio-fuel cogeneration)
Data Centre Energy & Sustainability Strategy
IT Server:(> Efficient)(> Utilisation)> T (& RH) range
Coypright HP 2011 - Subject to Change Without Notice HP Confidential
Data Centre Facility - Trends
• Modular data halls & containers
• Various free cooling techniques
• Variable density
• Aisle Containment
• Higher Densities
• Gradually moving to Liquid Cooling
• Real time measurement of PUE/TP-E
• Move away from conventional ac power to dc power
• Variable reliability:-
• Multi-tiering
• Clustering HW/SW
• Network diversity
•Widening of acceptable server inlet thermal limits
Coypright HP 2011 - Subject to Change Without Notice HP Confidential
End of Session