Scandinavian countries develop carbon-neutral, shared-resource HPC data center using Intel and HP technologiesDue to growing power consumption and soaring power costs, supercomputing costs are an increasing economic burden for researchers and theiruniversities in Scandinavia. To address these challenges, the national academic research infrastructure organizations of Sweden, Denmark and Norwayworked together to develop a green, low-cost data center to provide resources that could be remotely shared. Located in Reykjavik, the capital ofIceland, the Advania Thor Data Center (AT/DC) uses powerful natural resources and cost-efficient natural air cooling. Its high-performance computing(HPC) platform consists of 288 HP ProLiant* BL280c G6 servers powered by Intel® Xeon® processors 5600 series. This innovative facility providesa new model for shared, energy cheap and energy-efficient HPC resources, potentially changing the future delivery of computing services.

CASE STUDY

Intel®Xeon®Processor 5600 Series

Energy/Utilities Energy Efficiency and Government/Public SectorIT Efficiency, Environment, and Performance

Natural Resources

CHALLENGES

• Cost-efficient HPC. Scandinavian countries were paying high charges for HPC and, faced withspiraling energy costs, were concerned about diminishing returns for supercomputing investments

• Working together. Representatives from Norway, Denmark and Sweden aimed to develop anenergy-efficient data center that provided HPC resources that researchers across the regioncould have shared access to

SOLUTIONS

• Carbon-neutral. Advania, one of the largest IT companies in the region, was charged withdeveloping a carbon-neutral, energy-efficient data center

• Natural resources. The data center was located in Iceland, where natural power resourcesand free cooling mean much cheaper energy costs

• Powerful platform. Advania developed a 288-node cluster based on HP ProLiant 280c G6servers and the Intel Xeon processor 5600 series

IMPACT

• Low cost. Lower energy costs could result in the total cost of ownership (TCO) falling by up to50 percent

• Shared resources. Universities and research organizations from across the region are beginningto share the HPC resources, signaling a political shift in, and willingness to share, supercomputingresources

• Elite benefits. AT/DC provides high-performance computing with low TCO, low power usageand green power usage

• Global reputation. AT/DC is gaining a powerful reputation for industry-leading, carbon-neutral,low-cost operations with governments and enterprises around the world seeking to benefitfrom its use

Diminishing returnsWithin the context of HPC, it’s the norm to have supercomputing hardware physically close tothe research institutes that use it. Yet the hosting and operational cost is becoming increasinglyprohibitive as energy costs spiral upwards each year.

This problem was particularly acute in the European Nordic regions, where countries spend millionsof Euros each year on supercomputers, yet often have weak accounting of energy costs.

Rene Belso, chief technical officer, Computing, Danish e-Infrastructure Cooperation (DeIC), said:“Research institutes across Scandinavia recognized the cost of HPC infrastructure operations, namelyelectricity supplies is not transparent. In the worst case, with the cost of electricity in key marketsegments rising annually, long-term costs could quickly spiral out of control, jeopardizing HPC inour region.

“In the best case we simply lose the significant electricity savings we could have made by placingour computer centers at a more favorable place. Savings we otherwise could have used to geta near doubling of our compute power”.

This will result in universities and research organizations seeing ever diminishing returns from theirHPC budgets, or in budgets increasing in size for lesser returns. Recognizing the implications, researchand government organizations in Sweden, Denmark and Norway came together to explore thepossibility of saving costs by jointly developing HPC resources that could be shared across the region.

Belso added: “Opening a supercomputer cluster is no longer rocket science, and ensuring high-level operations is quite straightforward today. Our aim was to see if we could bring down theTCO by getting organizations to share HPC resources.”

“Cost-efficient HPC delivery and

power and cooling were the criteria

for the AT/DC. Intel® Xeon®

processor 5600 series provides

the dense, powerful HPC required

by the data center and is also

designed for energy efficiency.”

Kolbeinn Einarsson Business Development Manager

Advania Nordic

“However, this was potentially fraught with diffi-culties.“ Belso added: “University computer cen-ters understandably often do not want to sharetheir resources with other centers. They arguethat they need to have resources close by, evenif the operational costs are significantly higher.“

“While this is sometimes justified, we wantedto prove to universities operating these com-puter centers that they could benefit by lettingtheir computer centers focus on more advan-ced operations and state-of-the-art user sup-port, and not on hardware maintenance, oftennot amounting to more than on-location cab-ling and hardware replacements.”

The reason that many computers maintain an oldoperational way is to retain control, influence,funding, and jobs. Only a few are ready to movefrom mostly cabling and hardware maintenanceto advanced control of remote front-end serversin collaboration with colleagues placed remotelywithin other organizations and cultures.

However with electricity costs growing year-on-year and ever more advanced network connec-tivity, traditional computer centers face thesame challenge to innovate as did wagon wheelmakers with the arrival of rubber tires. Thereis certainly resistance to change from a hard-ware focus to remote operations, and local usersupport, but the need for change is clear and thepotential advantages of doing so are compelling.

Hidden powerTo prove this, and with an eye on developingjoint national infrastructures for computationalscience that deliver high quality computationalservices, the trio of countries decided to developa data center that would act as a proof ofconcept pilot.

Because of its unique climate and natural powerresources Iceland, was chosen as the location.Geothermal energy lies just beneath the surface,while hydro power sources are easily tapped.This means the price of electricity, compared tothe Scandinavian countries, is almost negligible.The prospect of reducing TCO up to 50 percentand using carbon-neutral sources was a com-pelling reason to locate the data center in Ice-land.

Advania, the largest IT company in Iceland andthe ninth largest in the Nordic regions, wastasked with developing a data center that madeuse of these natural resources to provide greencompute services for Denmark, Sweden andNorway, as well as Iceland.

Kolbeinn Einarsson, business developmentmanager for Advania Nordic, said: “Iceland hasgreen energy in relative abundance, with lowand stable power costs and plenty of free naturalcooling. These factors are ideal for developing

an energy-efficient data center, but we alsoneeded to develop an HPC platform that wouldprovide efficiency gains.”

To meet these objectives, Advania developeda 288-node cluster based on HP ProLiant 280cG6 servers and the Intel Xeon processor 5600series. The platform provides a compelling HPCtheoretical peak performance of 35 Tflop, whilealso offering 6912GB of memory.

Industry-leading technologyEinarsson adds that the energy-efficiency andpower provided by the platform were the ultimatedeciding factors in the choice of hardware: “Weare vendor agnostic, but people need to packmore and more computing power into smallerspaces. Cost-efficient HPC delivery and powerand cooling were the criteria for the AT/DC.The Intel Xeon processor 5600 series providesthe dense, powerful HPC required by the datacenter and is also designed for energy efficiency.”

Energy prices in Iceland are extremely low butoverall efficiency was important on severalcounts. Advania wanted to combine high-pro-duction quality with low TCO and dovetailthese benefits with high power usage effec-tiveness (PUE) to deliver a globally elite datacenter on all counts.

The Intel Xeon processor 5600 series regulatespower consumption and intelligently adjustsserver performance according to applicationdemand. By automatically shifting the CPU andmemory into the lowest available power state,this maximizes both energy cost savings andperformance. In the near future processor up-grades to the Intel® Xeon® processor E5 family,which is optimized for cloud operations, arelikely to take place.

The HP ProLiant G6 line also includes advancesin energy efficiency, virtualization and auto-mation. For example, one of its features is theHP Sea of Sensors.* Thirty-two smart sensorsautomatically track thermal activity across theserver and dynamically adjust system compo-nents such as fans, memory and input/outputprocessing to optimize system cooling and in-crease efficiency.

Karl Hansen, business manager for HP, said:“From the HPC, political and environmental per-spectives, the Thor data center is a landmarkproject. It’s the first time that disparate organi-zations have come together to create a greenoperation that utilizes natural resources andshares HPC resources.”

Transformation and collaborationEbba Thora Hvannberg, professor of computersciences at the University of Iceland, says theAT/DC is set to transform collaborative projects

between research centers across the region:“Evaluation of energy costs was one of thesuccess criteria for all organizations involvedin the project, and together we expect to saveat least half of the costs we were previouslypaying.

“By moving equipment away from the Universityof Iceland, we gain access to more powerfuland scalable HPC resources than we had onour own. Ultimately, we expect collaborationbetween centers to be very fruitful. This willalso overcome the resistance to shared HPCresources displayed by some organizations.”

Another advantage to locating the data centerin Iceland is the existence of network (Internet)submarine cables strung across the AtlanticOcean floor. Although at the moment the AT/DCis primarily being used by Scandinavian researchorganizations, there is plenty of spare computingand storage capacity and Advania has beenreceiving enquiries from organizations throug-hout Europe and America.

Kolbeinn Einarsson explains: “The infrastructurealready exists to provide remote computingservices to other countries beyond the Nordicregions. For example, computer services canbe delivered to the UK from Iceland with onlya 40 millisecond latency delay.”

Einarsson says the data center will expand asand when it needs to, but for the present, thefocus is on providing HPC resources to researchorganizations across Norway, Sweden, Denmarkand Iceland. While the carbon-neutral energysources available in Iceland are clearly the foun-dation for the AT/DC, the hardware provided byIntel and HP complements and galvanizes a datacenter model that could, in the future, becomea template for compute services delivery.

Governments and large organizations are alreadyenquiring about the services delivered by thedata center and Karl Hansen believes that ulti-mately its carbon-neutral, geo-thermal, energy-efficient model could signal the beginning ofnew forms of compute power delivery in whichcompute power and storage are delivered re-motely at a fraction of the cost paid today.

Copyright © 2012 Intel Corporation. All rights reserved. Intel, the Intel logo, and Intel Xeon are trademarks of Intel Corporation in the U.S. and other countries.This document and the information given are for the convenience of Intel’s customer base and are provided “AS IS” WITH NO WARRANTIES WHATSOEVER, EXPRESS OR IMPLIED,INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT OF INTELLECTUAL PROPERTY RIGHTS. Receipt orpossession of this document does not grant any license to any of the intellectual property described, displayed, or contained herein. Intel® products are not intended for use inmedical, lifesaving, life-sustaining, critical control, or safety systems, or in nuclear facility applications.*Other names and brands may be claimed as the property of others. 0612/JNW/RLC/XX/PDF 327619-001EN

Sustainable, low-cost energy powers asupercomputing platform that delivers35 Tflop

Find the solution that’s right for your organi-zation. Contact your Intel representative, visitIntel’s Business Success Stories for IT Managers(www.intel.co.uk/Itcasestudies) or explore theIntel.com IT Center (www.intel.com/itcenter).

Spotlight on Thor Data Center

The AT/DC is a 28,000 square foot facility10 minutes from Reykjavik, Iceland. It is oneof the most energy efficient data centersin the world. Using free-cooling elements,power utilization is extremely efficient. Thedata center is powered by clean, renewablehydroelectric and geothermal energy sourcesand offers a unique value proposition forcompanies who require a reliant and costeffective hosting service in a 100 percentgreen, zero-carbon footprint environment.