What’s a Supercomputer Good for Anyway?

Ruth Poole – IBM Software Engineer Blue Gene Control System

Overview

• Applications– Current Customers– Future Directions

• Architecture Foundations

• Hardware Comparisons

• Trends

Application Categories – Current Customers

• Government and University– Nuclear stockpile stewardship– Other research

• Computational Fluid Dynamics – CFD– Aerodynamics – aerospace, automotive, engine design– Weather/climate modeling

• Physics / Astronomy– QCD – Quantum Chromodynamics– Flash – simulate supernovae explosions– LOFAR – radio telescope consisting of many small

receivers• Biomedical

– Computational chemistry– Drug discovery– DNA sequencing and search

• Others– Finance– Oil / Gas exploration

NAMD - NAnoscale Molecular Dynamics

• Theoretical and Computational Biophysics Group (TCB) and Parallel Programming Laboratory (PPL) at the University of Illinois at Urbana-Champaign

• Simulate systems of millions of atoms• Use to build models of cellular processes• Current goal: model one micro-second of cell

behavior

Blue Brain - EPFL

• EPFL-IBM joint research project to create a cellular level, software replica of the Neocortical Column

• Phase 1, rat brain with 10,000 neuron Neocortical Column with automatically generated, biologically accurate neurons completed November 26, 2007

• Study its function and dysfunction and to lay the foundation for large scale modeling of the mammalian brain

Materials Science

• Blue Waters IBM / NSF collaboration project for open research

• University of Illinois at Urbana-Champaign planned for 2011

• Design a substance from atoms that meets specific parameters for conductivity and strength

Robot Surgeons

• Laser prostate surgery on a dog• Real-time data from MRI used to guide laser• Computer in Austin, Dog in Houston• TACC Lonestar, #38 on Top500.org

Simulated Plants

• Computer model that mimics the process of evolution.

• First model to simulate every step of the photosynthetic process.

• Researchers have built a better plant, one that produces more leaves and fruit without needing extra fertilizer.

• University of Illinois; Photo by Don Hamerman

Background and Definitions

• Processors (Nodes)– Multicore – more than one processor on a chip, dual = 2,

quad = 4, etc.

– Hybrid – a main processor with one or more secondary processors for specific computations (aka. accelerator)

• “Macro” Architecture– Massively Parallel Processing system (MPP) –

Supercomputer with nodes, memory, networking tightly integrated

– Cluster – A network of commodity machines connected by a network

• Interconnect (between nodes)– Crossbar – all nodes connected through crossbar

– Mesh/Torus – nodes connected in a grid

– Tree – nodes connected in a (fat) tree

Hardware Comparisons

• IBM Road Runner

• IBM Blue Gene L/P

• Sun Constellation

• Cray XT5/h

• Cluster Systems

• Top500.org

IBM Roadrunner

• # 1 – 1 PetaFlop• Hybrid blades• 1-AMD Dual-core Opteron 1.8 GHz / 2-PowerXCell 8i

3.2 Ghz • 6,948 / 12,960 processors• Infiniband and Gigabit Ethernet• Los Alamos

“I know how to make 4 horses pull a cart - I don't know how to make 1024 chickens do it.”

Enrico Clementi - former IBM fellow

IBM Blue Gene L/P

• # 2, 3, 6, 9, others – 478 TFlops / 450 TFlops• Large number of modest processors• PowerPC 440/450 – 700/850 Mhz• 212,992 / 163,840 cores• 3-D torus• Communication over the torus network• Lawrence Livermore National Lab, Argonne, NCAR,

others

Sun Constellation – Ranger

• # 4 – 326 TFlops• AMD Quad-core Opteron 2 GHz Blades• 62,976 cores• Full-Clos Infiniband• Texas Advanced Computing Center

Cray XT5

• “Adapt the system to the application”• Quad-Core AMD Opteron 2.1 GHz• 30,976 (XT4 - Oak Ridge National Lab, # 5 –

205 TFlops)• Cray SeaStar2 – 3-D Torus• XT5h – hybrid processors

– AMD Opteron– Configurable FPGA accelerator

• Oak Ridge (planned)

Clusters

• 400 of the Top500.org• Many different vendors, configurations• Low cost, commodity systems• More management tools becoming available• Slower communication between nodes

Trends• Multicore – more processors on a chip• Hybrid processors

– Accelerators for specific kinds of computation– More difficult to take advantage of

• “Greenness” – Green500.org• Liquid cooling – Cray XT5 planned for Oak Ridge• More tools and systems integration

– Blue Waters– Microsoft Windows HPC

• Deskside supercomputers?– IBM BG/S– Cray CX1

• Mainstream customers – not just for research anymore– Financial– Engineering

References

• Overview of Recent Supercomputers - 2007 Edition http://top500.org/2007_overview_recent_supercomputers

• Cray XT5 product brochure http://www.cray.com/Products/XT/Product/Specifications.aspx

• Sun Constellation system information http://www.sun.com/servers/hpc/sunconstellationsystem/

• IBM Journal of Research and Development, Volume 52, Number 1/2, 2008

• “TACC Supercomputer Performs Laser Cancer Surgery on Canine” HPCWire, May 27, 2008, Aaron Dubrow

• “Researchers successfully simulate photosynthesis and design a better leaf” News Bureau: University of Illinois at Urbana-Chapaign, November 2007, Diana Yates

• The Blue Brain Project http://bluebrain.epfl.ch• Blue Waters http://www.ncsa.uiuc.edu/BlueWaters/

Background and Definitions

• Architectural classes– SIMD – single instruction, multiple data

(vectorprocessors) – multiple processors in lock-step, each with its own set of data

– MIMD – multiple instruction, multiple data – the bulk of High Performance Computing (HPC) systems

• Memory– Shared – all nodes have access to same memory– Distributed – each node has its own memory– NUMA – non-uniform memory access – logically

shared, physically distributed

• Interconnect (between nodes)– Crossbar – all nodes connected through crossbar– Mesh/Torus – nodes connected in a grid– Tree – nodes connected in a (fat) tree