Embed Size (px)
Citation preview
Rhode Island Convention Center • Providence, Rhode Island
Maximizing Mission Capability byOptimizing Data Center Capacity
Session 6 – Data Center Energy Management Strategies
Craig Barclay, PEDepartment of Defense
August 10, 2016
Energy Exchange: Federal Sustainability for the Next Decade
An insight to power usage in the US and Federal Data Centers
2
2013In 2013, 91 billion kWh were consumed by data centers in the US.
Approximately 10 billion kWh were federal servers.(source: NRDC, 2014)
2008In 2008, 61 billion kWh were consumed by data centers in the US.
Approximately 6 billion kWh were federal servers.(source: Brown et al, 2008)
Energy Exchange: Federal Sustainability for the Next Decade
Lets put these energy consumption values into perspective
3
In 2020, the projected energy use by data centers in the US if energy use efficiencies are not
implemented will be 139 billion kWh at a cost of
$13.7 billion.
= 47% increase over 2013!
(source: NRDC, ComputerWorld, 2015)
39 billion kWh is the annual electricity
consumption for all the households in the state of Michigan at a cost of $3.8
billion
(source: Brown et al, 2008)
Energy Exchange: Federal Sustainability for the Next Decade
Upward trend will continue due to Exascale Computing, increase of Cloud Services, and Government Regulations
4
61
91
139
6170 73
0
20
40
60
80
100
120
140
160
2008 2013 2020
ENER
GY CONSU
MED
YEARS
Energy Consumption by Data Centers in the U.S.
Previous Reports
LBNL 2016 EnergyReduction Efforts
Energy Exchange: Federal Sustainability for the Next Decade
PUE is only one metric and is not a good metric for data center energy and infrastructure management
5
Power Usage Effectiveness (PUE) only looks at the power efficiency with respect to input power delivered to a system and the power consumed by the compute system.
It does not measure how the installed power capacity resources in a data center are being managed, which was the first and foremost consideration when the data center was first constructed!
There is a need to understand all power capacities to mange energy consumed by data centers. An IT Infrastructure Library (ITIL*) which captures all power capacity is essential
*(Source: Craig Barclay, 2014, Information used and adapted from Rasmussen, 2007
ilization of the full capacity of a data center is ramount to optimize infrastructure and mission
o build the frastructure for a 5MW, Tier III data nter costs $1.5 llion.
quates to an frastructure cost $23 million per egaWatt.
reflection on the power usage statistics
Imagine, just for a moment,the impact if the Utility
over built their infrastructure by 30‐40%
randed and Trapped Capacity has a devastating effect n Data Center Infrastructure Utilization
Is there a difference between Stranded and Trapped Capacity?
Does Trapped Capacity affect power, cooling, and space resourcing in a data
center?
YES!
StrandedTrappedCapacity
Data CenterInfrastructure
TRANDED CAPACITY is a PHYSICAL constraint
Stranded capacity is that capacity for which there is nsufficient power, space or cooling distribution – there’s a bottleneck in either a parent tier or in another system that prevents all the input power from being distributed.
Stranded Capacity = Insufficient Power or Space or Cooling to support intended loads
Power Permit system is one way to manage RANDED Capacity
Tracks power, space, and cooling on existing and new systems
Any refreshes or changes to IT systems go through a vetting process
On site walk down verification to confirm any infrastructure changes
Creates awareness across all users on power management
RAPPED CAPACITY is a MANAGERIAL constraint
During the summer 2015 on site measurements were performed on several Cloud and HPC systems to understand the magnitude of the problem.
The empirical data and research proved that ITpower resources supplied to systems were being under utilized by 40‐50% of their provisioned power.
The question was WHY?
Energy Exchange: Federal Sustainability for the Next Decade
An explanation of Trapped Capacity –Identifying the Problem
12
IT = Soft Requirement (Large Error Bar)
Infrastructure Organization = Hard Requirement (Provision and Build)
The same requirement means different things to different people!
Energy Exchange: Federal Sustainability for the Next Decade
OK – and so why is this mis‐communication so important?
13
The allocated power for the intended system is removed from the available site power until the system is designed, bid, installed, and comes on line. This process can take up to three years. In addition, monitoring the system load for one year to confirm actual power use is a total of 4 years!Lets consider actual examples of a Cloud and HPC System..….
Energy Exchange: Federal Sustainability for the Next Decade
Planning and Provisioning for a new IT system
14
POWER PROVISIONED at XXX MW3 YEARS
Energy Exchange: Federal Sustainability for the Next Decade
Actual energy use for a large HPC and Cloud System
15
trapped capacity
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 50 100 150 200 250 300 350 400
Power, M
W
Time
HPC Obfuscated Data, NormalizedProvisioned 259 kW
TrappedCapacity3 MW
Energy Exchange: Federal Sustainability for the Next Decade16
Analyzing the SCADA data from the power PDU produced expected and unexpected results.
0.00
20.00
40.00
60.00
80.00
100.00
120.00
140.00
160.00
7/24 7/25 7/26 7/27 7/28 7/29 7/30
System Total kWPDU G kWPDU H kWPDU L kWPDU M kW
TrappedCapacity140.5 kW
Mode = 118.5 kW
Maximum = 134.6 kW
Minimum = 98.9 kW
Provisioned 259 kW
Actual energy use for a Cloud System
Energy Exchange: Federal Sustainability for the Next Decade
Under utilization of expensive power and cooling resources
17
High Installation, Operating, and Maintenance costs
Energy Exchange: Federal Sustainability for the Next Decade
Under utilization of expensive IT components in the hardware stack
18
Processing (CPU), Memory (RAM), Storage (Disk)
Energy Exchange: Federal Sustainability for the Next Decade
On site measurements indicated that millions of dollars of infrastructure were not being fully utilized
19
TRAPPED CAPACITY indicated a reduced availability of site power, implying that there is a shortage of power to support mission
Energy Exchange: Federal Sustainability for the Next Decade
Executive Order 13693 (Replaced 13514)
20
Mandates that all federal agency heads must implement energy conservation measures.
Energy Exchange: Federal Sustainability for the Next Decade
Federal agencies must control data center power because it impacts Utility infrastructure costs.
21
Institute a strategy to manage power capacities of Cloud and HPC’s data centers
Energy Exchange: Federal Sustainability for the Next Decade
TRAPPED CAPACITY mitigation strategy
22
Gives data center designers, mangers, and operators insight to energy requirement and data center utilization.
Implement power monitoring at node, rack, and system level to obtain empirical
power data and insert a SoW in all new purchases.
Power permit process to include Stranded and Trapped
capacity management.
Develop an information technology infrastructure library (ITIL) to monitor and control all power capacities.
Institute a total cost of operation/ownership (TCO) metric to keep track of the costs associated with Data
Center Energy Use
Trapped Capacity Mitigation Strategy
Energy Exchange: Federal Sustainability for the Next Decade
Where we are now and where we want to be , requires a visual solution
23
What we have What we want
Energy Exchange: Federal Sustainability for the Next Decade
Thank you
24
Craig Barclay at craig.a.barclay [email protected]
Energy Exchange: Federal Sustainability for the Next Decade
Back up Slides
25
Provisioned 259 kW
Following slides is an example of ACTUAL load profiles of
Trapped Capacity at a RACK LEVELIn a CLOUD SYSTEM
Energy Exchange: Federal Sustainability for the Next Decade26
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
7/24 7/25 7/26 7/27 7/28 7/29 7/30
CY50 kW
Maximum = 5.79 kW
Minimum = 4.36 kW
Mode = 4.45 kW
TrappedCapacities6.96 KW
Provisioned 11.41 kW
A rack depicting a higher workload, notice the high activity.
0.00
.00
.00
.00
4.00
.00
.00
.00
.00
.00
7/24 7/25 7/26 7/27 7/28 7/29 7/30
CY51 kW
Maximum = 7.78 kW
Minimum = 4.30 kW
Provisioned 11.41 kW
Mode = 5.59 kW
TrappedCapacities5.82 KW
Infrastructure Library ‐ ITIL
