02/24/09

Green Data Center project

Alan Crosswell

02/24/09 2

Agenda

• The Data Center Status Quo• Future State Goals• Our NYSERDA Advanced Concepts Datacenter proposal

02/24/09 3

The Data Center Status Quo

• Architectural– Built in 1963, updated somewhat in the 1980's.– 5000 sf (465 m^2) raised floor machine room space.– 1750 sf (163 m^2) raised floor space, now offices.– 12” raised floor– Adequate support spaces nearby

• Staff• Staging• Storage• Mechanical & fire suppression

02/24/09 4

The Data Center Status Quo (cont'd)

• Electrical– Supply: 3-phase 208V from automatic

transfer switch.– Distribution: 208V to wall-mounted panels;

120V to most servers.– No central UPS; lots of rack-mounted units.– Generator: 1750 kW shared with other

users.• At capacity.

– No metering. (Spot readings every decade or so:-)

– IT demand load tripled from 2001-2008• 2001: 107 kVA• 2008: 335 kVA

02/24/09 5

The Data Center Status Quo (cont'd)

• Mechanical– On floor CRAC units served by campus

chilled water.– Also served by backup glycol dry coolers.– Supplements a central overhead chilled air

system.– Heat load is shared between the overhead

and CRAC.– No hot/cold aisles– Rows are in various orientations– Due to tripling of demand load, the backup

(generator-powered) CRAC units lack sufficient capacity.

02/24/09 6

The Data Center Status Quo (cont'd)

• IT systems– A mix of mostly administrative (non-

research) systems.– Most servers dual-corded 120V

power input.– Many ancient servers.– Due to lack of room UPS, each rack

has UPSes taking up 30-40% of the space.

– Lots of spaghetti in the racks and under the floor.

02/24/09 7

The Data Center Status Quo (cont'd)

• Other data centers– Many small school, departmental &

research server rooms all over the place.– Most lack electrical or HVAC backup.– Many could be better used as academic

space (labs, offices, classrooms).– Growth in research HPC putting

increasing pressure on these server rooms.

– Lots of wasted money building new server rooms for HPC clusters that are part of faculty startup packages, etc.

02/24/09 8

Future State Goals – Next 5 years

• Begin phased upgrades of the Data Center to Improve power and space efficiency. Overall cost ~ $20M.

• Consolidate and replace pizza box servers with blades (& virtualization).

• Consolidate and simplify storage systems.• Accommodate growing demand for HPC research clusters• Accommodate server needs of new Interdisciplinary

Science Building.• Develop internal cloud services.• Explore external cloud services.

02/24/09 9

Future State Goals – Next 10 years

• Build a new data center of 10,000-15,000 sf• Consolidate many small server rooms.• Significant use of green-energy cloud

computing resources.

From www.jiminypeak.com

02/24/09 10

Our NYSERDA Proposal

• New York State Energy Research & Development Authority Program Opportunity Notice 1206

• ~$1M ($447K from NYSERDA awarded pending contract)• Improve space & power efficiency of primarily

administrative servers.• Contribute to Columbia's PlaNYC carbon footprint

reduction goal.• Make room for shared research computing in the existing

data center.• Measure and test vendor claims of energy efficiency

improvements.

02/24/09 11

Our NYSERDA Proposal – Specific Tasks

• Identify 30 old servers to replace.• Instrument server power consumption and data center

heat load. “Real time” with SNMP.• Establish PUE profile (use DoE DC Pro survey tool)• Implement 9 racks of high-density cooling (in-row/rack).• Implement proper UPS and high-voltage distribution.• Compare old & new research clusters' power consumption

for the same workload.• Implement advanced server power management and

measure improvements.• Review with internal, external and research faculty

advisory groups.• Communicate results.