HPC, the Department of Energy and the Obama Administration:

A Strategic Perspective

Victor ReisSenior AdvisorDepartment of Energy

October 9, 2009

Extreme Computing

Outline

• Strategic Planning (One Slide)– Vision, Assets, Strategy

• President Obama’s Vision• DOE as a strategic asset• The ASCI Strategy – An example• Current DOE HPC Strategies • Potential DOE HPC strategies & applications• Summary

Strategic Planning

Vision

Assets

Strategies

“We must harness the power of nuclear energy on behalf of our efforts to combat climate change, and to advance peace opportunity for all people.”

President ObamaPrague, April 5, 2009

“The United States will take concrete steps towards a world without nuclear weapons…. Make no mistake: As long as these weapons exist, the United States will maintain a safe, secure and effective arsenal to deter any adversary, and guarantee that defense to our allies.”

Vision: Nuclear

And my budget doubles funding for the Department of Energy's Office of Science, which builds and operates accelerators, colliders, supercomputers, high-energy light sources, and facilities for making nano-materials -- because we know that a nation's potential for scientific discovery is defined by the tools that it makes available to its researchers.

REMARKS BY THE PRESIDENTAT THE NATIONAL ACADEMY OF SCIENCES ANNUAL

MEETING

April 27, 2009

Vision: Science & DOE

“We have been talking about energy for decades.  But there is no longer a disagreement over whether our dependence on foreign oil is endangering our security.  It is.  There is no longer a debate about whether carbon pollution is placing our planet in jeopardy.  It’s happening.” 

WEEKLY ADDRESS: President Obama Calls Energy Bill Passage Critical to Stronger American Economy

June 27, 2009

Vision: Energy & Climate

“We know what needs to be done. We know that our planet's future depends on a global commitment to permanently reduce greenhouse gas pollution. We know that if we put the right rules and incentives in place, we will unleash the creative power of our best scientists, engineers, and entrepreneurs to build a better world.”

“That so many of us are here today is a recognition that the threat from climate change is serious, it is urgent, and it is growing.”

President Obama: Speech to the United Nations On Climate Change. 22 Sept 2009

Vision: Climate Change

Assets: DOE has done a lot in the last ~ 15 years: e.g.:

• Major New Science Installations– Spallation Neutron Source – Oak Ridge– Advanced Photon Source - Argonne– National Ignition Facility - Livermore– Linac Coherent Light Source - SLAC– Relativistic Heavy Ion Collider - Brookhaven– Microsystems and Engineering Sciences Application (MESA) – Sandia– Dual Axis Radiographic Hydrodynamic Test - Los Alamos

• Nuclear Weapons - Stockpile Stewardship– Much deeper understanding of nuclear explosive process

• 114 Annual Weapon Certifications without underground testing • Simulation has driven High Performance Computing Leadership • Stockpile Life Extension (W87, B61, W76)

• Non-Proliferation– Russian Weapons and Material Security much improved– International Safeguards, Global Threat Reduction Inititiative– “Megatons to Megawatts” (367 MTons of Russian HEU eliminated)

• Defense Clean Up– Rocky Flats– Fernald– Mound

• Radioactive Waste Disposition– Waste Isolation Pilot Plant (WIPP)

29 Nobel Prizes

0

500

1000

1500

2000

2500

LANL SNL LLNL LBNL ORNL PNNL ANL BNL FNAL SLAC INL NREL SRNL JLAB NETL Ames

Numberof

PhDs

PPPL

Office

NNSA

Science

NuclearEnergy

Efficiency& Renew

Enviro.Mgmt

FossilEnergy

Total = 12,440 PhDs *

Plus ~25,000 visiting scientists

ASSETS: DOE has a large collection of PhD scientistsat 17 labs

17 labs* October 2007

DOE Assets - Starting with Stockpile Stewardship (ASCI/ASC) DOE has maintained US Global Computing Leadership

8 of top 10 U.S.5 of top 10 DOE

ASCI Goal LANL/ IBM

Sandia/Intel

LLNL/IBM

LLNL/IBM

LLNL/IBM

July 3, 1993

President Clinton:

Seek a CTBT, & directs the DOE to

"establish a stewardship program to ensure the preservation of the core intellectual and technical competencies of the U.S. in nuclear weapons."

Accelerated Strategic Computing Initiative:Aligning and Executing Toward a Vision.

A comprehensive program to develop the applications, development tools, computers, networks, and visualization to create the simulation capabilities

to enable a science based approach to stockpile stewardship.

An integrated, Balanced Program

Robust, Balanced and Long Term Funding

$-

$100.0

$200.0

$300.0

$400.0

$500.0

$600.0

$700.0

$800.0

1996199719981999200020012002200320042005200620072008

Fiscal Year

$ M

illion

s

Facility Ops & User SupportPlatforms & EnvironmentsMulti-Purpose & ConstructionRelationshipsEnvironmentsPlatformsApplications

ASCI was built with Partnerships

Labs Hardware

Software

Academia

What are the models (strategies) for HPC development?•Science Model•Applications Model•Commercial Model

DOE

Science Model - DOE SciDAC – Scientific Discovery Through Advance Computing• Broad, Open, Peer Reviewed, Balanced, Important Science Problems• Best Buyable HPC’s • HPC support tools – math, algorithm’s, software etc.• Manage user computer cycles to priority problems

Applications Model – DOE NNSA ASCI/ASC• Focus on time urgent national problem

•“closed” community• Develop new computers/software/environments

• Partnerships with HPC industry• Targeted Investments in industry R&D

• Big Investment, “Tight” management

Potential HPC Strategies

How “applications driven” will the next generation of HPC architectures be?

What are the DOE HPC applications?• Who cares?• How much performance, when?• Scale of the investment?• Breadth of applications

What is the level of DOE/Industry partnership?

What is the model of DOE HPC program?

From the Perspective of HPC Development:

Commercial Spinoff?

Scientific Grand Challenges Workshop Series • BER/Climate Workshop: Challenges in Climate Change Science and the Role of

Computing at the Extreme Scale, November 6-7, 2008• HEP/High Energy Physics Workshop: Scientific Challenges for Understanding the

Quantum Universe and the Role of Computing at the Extreme Scale, December 9-11, 2008

• NP/Nuclear Physics Workshop: Forefront Questions in Nuclear Science and the Role of High-Performance Computing, January 26-28, 2009

• FES/Fusion Workshop: Extreme Scale Computing Challenges in Fusion Science, March 18-20, 2009

• NE/Nuclear Energy Workshop: Extreme Scale Computing Challenges in Nuclear Energy, May 11-12, 2009

• BER/Biology Workshop: Extreme Scale Computing Challenges in Biology, August 17-19, 2009

• BES/Materials Workshop: Extreme Scale Computing Challenges in Materials Science, August 13-15, 2009

• NNSA/ASCR Workshop: Science Grand Challenges, October, 6-8, 2009

November 4, 2008

DOE Response

The Imperative of High-Performance Computing: Pushing Back the Frontiers of

Science and Technology

Workshop on Scientific Challenges in National Security:

the Role of Computing at the Extreme Scale

October 6, 2009

Dr. William F. BrinkmanDirector, Office of ScienceU.S. Department of Energy

Signature Computations in Pushing Back the Frontiers of Science

• Energy for environmental sustainability– Climate change: incorporating carbon cycle in models, improving spatial/temporal

resolution

– Combustion: obtaining quantitative predictions within flames

– Bioenergy: modeling and solving recalcitrance in cellulosic ethanol

– Solar Energy: detailed modeling of solar cells

– Energy storage: next-generation electrical storage materials modeling

– Energy transmission: quantitative prediction of grid performance

– Fusion: ITER design, optimization, and operation, IFE design

– Nuclear energy: fully-resolved reactor system prediction, qualification of advanced fuels

• Materials and nanoscience– Predicting the structure and performance of materials under extreme conditions

Dr. William F. BrinkmanDirector, Office of ScienceU.S. Department of Energy

Signature Computations in Pushing Back the Frontiers of Science

• Fundamental science– Astrophysics: improved software and models fro astrophysical phenomena

– Chemistry: quantitative modeling of macromolecules and assemblies of molecules

– Nuclear physics: developing ab initio techniques for heavier atoms

– High energy physics: managing and analyzing extreme datasets

– Biology: improved models system-level biology

• National Security Science & Technology (DOE/NNSA)– Stockpile Stewardship: 3-dimensional simulation tools with improved resolution

and subgrid-scale physics

– Nuclear Nonproliferation: improved techniques for materials detection and control

– Climate Change: improved computational models for understanding global transport mechanisms in the atmosphere

Dr. William F. BrinkmanDirector, Office of ScienceU.S. Department of Energy

High Energy Physics (2)

Almost Perfect Fermion

Future ASC Computing RequirementsFuture ASC Computing Requirements 2121

The overarching ASC goal is to provide SSP with a sufficient science & technology base to make decisions

Energy Balance

BoostDrivers:

• Resolve key simulation unknowns (e.g. boost)

• Transition 2D to 3D UQ• Master exascale parallelism

• Field predictive, 3D UQ + key capabilities

07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 2697 98 99 00 01 02 03 04 05 0696

• Create 3D integrated design codes

• Master terascale parallelism

Secondary Performance

Develop and demonstrate capability

to certify aging weapons with codes

calibrated to past UGTs

Certify weapons without relying on

past or future UGTs - using predictive UQ

Certify Stockpile (near neighbors to the test

base)

Purple100

BG/L360

Red1

Blue3

White10

Q20

SSP Goals:

ASC Goals:

Platforms:

EnablesEnables

Exascale systems

InitialConditionsFor Boost

UQ and key unknowns will drive platform requirements until 2020-2025 - thereafter it will be predictive UQ alone

Uncertainty Quantification

Zia Sequoia12x Purple20x BG/L

Si RadiationDamage

Fracture

8x Purple

April 2009April 2009

Recommended DOE Program for Nuclear Energy

Summary• Solving current DOE problems could provide a major opportunity for HPC• HPC a major tool is solving current DOE problems

– Help Fulfill Obama vision– But HPC not quite recognized by the Obama Administration

• FY 2012 Budget first real opportunity

• What will be the DOE problems?– Climate– National Security– Energy – Science– …..

• Historically Computer Architecture drives the rate of growth– Driven by Problems & Partnerships > Programs >Platforms

• ASCI experience

• DOE Model of Development – Balance between Science & Applications– Level of investment in industry

• Opportunity for Obama Administration & DOE– Role of DOE labs – Academia– Industry