Forecasting Data Center Power Requirements:
Tips from the Trenches
Frank BereznayMoses Chu
IBM
Agenda
Overall Facility Utilization / Usage Profiles Measuring Power Usage at the Equipment
Level Reporting and Forecasting Techniques Summary
Facility Overview
Two Story Facility Built in the early 1990’s to accommodate mainframe class
equipment. 2 UPS Systems
Primary UPS (A/B) 2,400 KW Secondary UPS (C/D) 600 KW Added later
3,500 Items Connected to UPS Systems Equipment Mix Described as typical for today’s Data
Center. Newer blade farms creating hot spots for A/C
systems
Initial Measurements
There were a lot of opinions about the overall utilization profile for the UPS systems, but nobody had hard empirical data.
A low tech ‘clip board’ agent was installed and activated at the UPS panels at four hour intervals.
Initial MeasurementsUsage Profile for Primary UPS System
85.0%
85.3%
85.5%
85.8%
86.0%
86.3%
86.5%
86.8%
87.0%
0 4 8 12 16 20
Hour of the Day
Sun Mon Tue Wed Thur Fri Sat
Initial MeasurementsUsage Profile for Secondary UPS System
79.5%
80.0%
80.5%
81.0%
81.5%
82.0%
0 4 8 12 16 20
Hour of the Day
Sun Mon Tue Wed Thur Fri Sat
Usage Profile by Equipment Type
WinTel Servers52%
Non-WinTel Server8%
Mainframe3%
Storage21%
Network - Data10%
Telecom1%
Other5%
Measuring Power Usage at the Equipment Level
Not as easy as it sounds. Engineers were reluctant to open up the back
side of PDU cabinets to attach reading devices.
Power cables coming out of the cabinet were not labeled.
Similar situation at equipment end of power run.
Measuring Power Usage at the Equipment Level
Looked for some equipment documentation on actual power usage. Generally speaking, the documentation only
provided rating factors for the circuit breakers. Blade servers were the exception, they provided
actual power calculators. Looked online (and the CMG proceedings)
for articles providing this kind of information. Came up empty.
Measuring Power Usage at the Equipment Level
Pursued two efforts in parallel Captured before and after power readings at the
UPS level for any change that projected a 5KW or greater impact
Worked with staff at a newer Data Center that had the capability to provide circuit level power usage profiles
Developed spreadsheet of findings and created some initial forecasts
Measuring Power Usage at the Equipment Level
Initial forecasts tended to underestimate the impact of power events. Part of the learning curve for this type of analysis.
A ‘life cycle’ usage profile for equipment types began to emerge from the dual measurement efforts being conducted.
Measuring Power Usage at the Equipment Level
Power Usage Profile for WinTel Servers
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
Power Up Idle TypicalLoad
HeavyLoad
Idle PowerDown
Measurement Points
Measuring Power Usage at the Equipment Level
Power Usage Profile for Midrange and Mainframe Servers
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
Power Up Idle TypicalLoad
HeavyLoad
Idle PowerDown
Measuring Power Usage at the Equipment Level
Power Usage Profile for Storage Equipment
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
Power Up Idle TypicalLoad
HeavyLoad
Idle PowerDown
Reporting and Forecasting Techniques
All data capture and analysis was done in EXCEL, nothing fancy.
Experimented with a variety of reporting techniques before we came up with the final set of reports.
Reporting and Forecasting Techniques
Project Name / Identifier Equipment Type Qty KW Install date Week of Status 7-Jul 14-Jul 21-Jul 28-Jul 4-AugBlade Farm Upgrade L20p G4 64 0.3 07/11/08 07/07/08 16.0 Retire Development Regatta IBM 9119-590 -1 20.3 07/13/08 07/07/08 (20.3) New Prod Storage Frame DMX3-24 2 34.7 07/16/08 07/14/08 69.4 Retire Reporting Servers ML530 G2 -36 0.6 07/24/08 07/21/08 (20.2)New OLAP Reporting Servers ProLiant ML530 8 0.7 07/25/08 07/21/08 5.5Retire Prod Storage Frames DMX3-24 -2 34.7 07/30/08 07/28/08 (69.4)New zSeries Development Host IBM 2094-S38 1 12.7 08/06/08 08/04/08 12.7New Devl Storage Frames DMX3-24 2 34.7 08/08/08 08/04/08 69.4 16.0 69.4 5.5 0.0 82.1 (20.3) 0.0 (20.2) (69.4) 0.0 (4.3) 69.4 (14.7) (69.4) 82.1Utilization Threshold 2,400 Baseline Peak KW Usage 2,045 Projected KW Consumption 2040.7 2110.1 2095.4 2026.0 2108.1Projected Peak Utilization 85.0% 87.9% 87.3% 84.4% 87.8%
Weekly IncreaseWeekly DecreaseWeekly Change
Sample Table to Capture Power Events and Forecast UPS Utilization
Reporting and Forecasting Techniques
Sample Graphics Based Reporting Format
(80.0)
(60.0)
(40.0)
(20.0)
0.0
20.0
40.0
60.0
80.0
100.0
7-Jul
14-Jul
21-Jul
28-Jul
4-Aug
KW
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
Weekly Increase Weekly Decrease Projected Peak Utilization
Reporting and Forecasting Techniques
Physical Capacity of UPS System in KW 2400 Utilization Threshold for UPS System 85%
Utilization Threshold of UPS System in KW 2040
Monthly Peak Utilization as of July 7 2045
Available KW -5
Project Name / Identifier Equipment Type Qty KW Install date Week of Status Total KWAvailable
KWBlade Farm Upgrade L20p G4 64 0.3 07/11/08 07/07/08 16.00 (21.00)Retire Development Regatta IBM 9119-590 -1 20.3 07/13/08 07/07/08 (20.30) (0.70)New Prod Storage Frame DMX3-24 2 34.7 07/16/08 07/14/08 69.40 (70.10)Retire Reporting Servers ML530 G2 -36 0.6 07/24/08 07/21/08 (20.20) (49.90)New OLAP Reporting Servers ProLiant ML530 8 0.7 07/25/08 07/21/08 5.52 (55.42)Retire Prod Storage Frames DMX3-24 -2 34.7 07/30/08 07/28/08 (69.40) 13.98 New zSeries Development Host IBM 2094-S38 1 12.7 08/06/08 08/04/08 12.67 1.31 New Devl Storage Frames DMX3-24 2 34.7 08/08/08 08/04/08 69.40 (68.09)
Sample Tabular Report to Track Available KW for UPS System
Summary
Primary goal of the paper was to share some lessons learned.
Power analytics are fairly straightforward. The data can be elusive and difficult to get.
The paper contains a table of power factors developed over time.
Date Updated Vendor Category Model Configuration Watts Source of Metric Notes / Comments
EMC Storage Frame DMX 4500 14,400 UPS Measure
Actual power usage varies by storage configuration - See SI staff for an estimate.
EMC Storage Frame DMX1000 325 SI Planning Docs Actual power usage varies by storage
EMC Storage Frame DMX2000 500 SI Planning Docs
Actual power usage varies by storage configuration - See SI staff for an estimate.
EMC Storage Frame DMX3 (DMX1500) 20,000 UPS Measure
Actual power usage varies by storage configuration - See SI staff for an estimate.
EMC Storage Frame DMX3-2462 TB / 98 GB 228 Disks 18,500
Secondary DC Measure - 2/28/08
A linear regression was done on DMX3 frames. Use 2,351 + 204 * TB of Stg to get Watts estimate.
EMC Storage Frame DMX3-24163 TB / 163 GB 600 Disks 34,700
Secondary DC Measure - 2/28/08
A linear regression was done on DMX3 frames. Use 2,351 + 204 * TB of Stg to get Watts estimate.
EMC Storage Frame DMX3-615 TB / 32 GB 56 Disks 4,100
Secondary DC Measure - 2/28/08
A linear regression was done on DMX3 frames. Use 2,351 + 204 * TB of Stg to get Watts estimate.
EMC Storage Frame DMX3-610 TB / 16 GB 52 Disks 3,500
Secondary DC Measure - 2/28/08
A linear regression was done on DMX3 frames. Use 2,351 + 204 * TB of Stg to get Watts estimate.
5/7/2008 HP Windows Server DL360 174Data Captured for SCE Credits
HP Windows Server ProLiant DL740 G1 1,175 BDNA HP Windows Server ProLiant DL760 G1 1,555 UPS MeasureHP Windows Server ProLiant ML370 1,294 UPS MeasureHP Windows Server ProLiant ML530 690 UPS MeasureIBM Mainframe Server IBM 2094-S28 12,672 IBMIBM Mainframe Server IBM 2094-S38 12,672 IBMIBM Midrange Server 7040 Model 681 8,000 UPS MeasureIBM Midrange Server IBM 7013-590 19,200 BDNA IBM Midrange Server IBM 7017-S80 2,120 BDNA IBM Midrange Server IBM 7017-S85 2,120 BDNA IBM Midrange Server IBM 7026-6M1 650 UPS MeasureIBM Midrange Server IBM 7026-B80 460 BDNA IBM Midrange Server IBM 7026-H50 1,122 BDNA IBM Midrange Server IBM 7026-M80 1,122 BDNA IBM Midrange Server IBM 7028-6C4 783 BDNA
The End
Questions??? Article References
Power Monitoring 101 http://viewer.bitpipe.com/viewer/viewDocument.do?ac
cessId=10352520 Avoid Common Pitfalls when Calculating Data
Center Power Loads http://searchdatacenter.techtarget.com/tip/0,289483,si
d80_gci1363455_mem1,00.html?track=NL-455&ad=718537USCA&asrc=EM_NLN_8862833