Realizing the Promiseof Grid Computing

Ian Foster

Mathematics and Computer Science Division

Argonne National Laboratory

and

Department of Computer Science

The University of Chicago

Presentation to the NSF Advisory Committee on CyberInfrastructure, November 30, 2001

foster@mcs.anl.gov ARGONNE CHICAGO

The Grid Opportunity

What Grids are about:

“Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations”

= entirely new tools, with often revolutionary impacts

The opportunity: advance transition to routine use by multiple years

foster@mcs.anl.gov ARGONNE CHICAGO

Why Grids? A biochemist exploits 10,000 computers to

screen 100,000 compounds in an hour 1,000 physicists worldwide pool resources for

petaop analyses of petabytes of data Civil engineers collaborate to design, execute,

& analyze shake table experiments Climate scientists visualize, annotate, &

analyze terabyte simulation datasets An emergency response team couples real time

data, weather model, population data

foster@mcs.anl.gov ARGONNE CHICAGO

Why Grids? (contd) A multidisciplinary analysis in aerospace

couples code and data in four companies A home user invokes architectural design

functions at an application service provider An application service provider purchases

cycles from compute cycle providers Scientists at a multinational company

collaborate on the design of a new product A community group pools members’ PCs to

perform environmental impact study

foster@mcs.anl.gov ARGONNE CHICAGO

Grids: Why Now?

Moore’s law improvements in computing produce highly functional endsystems

The Internet and burgeoning wired and wireless provide universal connectivity

Changing modes of working and problem solving emphasize teamwork, computation

Network exponentials produce dramatic changes in geometry and geography– 9-month doubling: double Moore’s law!

– 1986-2001: x340,000; 2001-2010: x4000?

foster@mcs.anl.gov ARGONNE CHICAGO

The Grid World: Current Status An exciting time, in many ways

– Dozens of major Grid projects worldwide> Deployment, technology, application

– Consensus on key concepts & technologies> E.g., Globus Toolkit as de facto standard

– Growing industrial interest But also:

– Funded by an inadequate patchwork of diverse, mostly short-term sources

– No long-term coordinated plan aimed at injecting Grid technologies into community

– International programs outpacing U.S. efforts!

foster@mcs.anl.gov ARGONNE CHICAGO

PACIs and Grids

PACIs play critical role in Grid development– Act very effectively as nucleation point, bully

pulpit, technology explorer

– Major resource providers for community But grid technologies & applications are

essentially unfunded mandates for PACIs– “Grids” a tiny fraction of total PACI budget

– Situation only worse for TeraGrid! Current situation untenable long term

– New scientific tools are not created for free

foster@mcs.anl.gov ARGONNE CHICAGO

What is Needed:A National Grid Program

Goal: Accelerate “Grid-enablement” of entire science & engineering communities– Don’t wait the 20 years it took the Internet!

Program components

1) Persistent R, D, outreach, support organization

2) Application-oriented “Grid challenge” projects

3) Infrastructure: campus, national, international

4) Basic research, engaging CS community

5) Explicit international component Explicit and strong interagency coordination

foster@mcs.anl.gov ARGONNE CHICAGO

A Persistent GridTechnology Organization

We’re talking about a complete retooling of entire science and engineering disciplines– Not a part-time, or three-year, or graduate

student business

– Also not something we can buy (yet) We need a persistent national organization

that can support this process– Technology R&D, packaging, delivery

– Training, outreach, support

– GRIDS Center an (unproven) existing model

foster@mcs.anl.gov ARGONNE CHICAGO

Application “Grid Challenge” Projects

Goal: Engage significant number of communities in the transition to Grids– GriPhyN, NVO, NEESgrid existing models

Emphasize innovation in application of technology and impact to community– May be data-, instrumentation-, compute-,

and/or collaboration-intensive

– Aim is to achieve improvement in the quality and/or quantity of science or engineering

– And to entrain community in new approaches

foster@mcs.anl.gov ARGONNE CHICAGO

Upgrade National Infrastructure

Seed nation with innovative Grid resources– iVDGL one existing model

Encourage formation of campus Grids– Re-think campus infrastructure program?

– U.Tenn SinRG one existing model Enhance national & international networks,

link with TeraGrid– Advanced optical nets, StarLight, etc.

Operations and monitoring

foster@mcs.anl.gov ARGONNE CHICAGO

Basic Research

Engage researchers in imagining & creating new tools & problem-solving methods– In a world of massive connectivity, data,

sensors, computing, collaboration, … And in understanding and creating the new

supporting services & infrastructure needed This is not “CS as usual”

foster@mcs.anl.gov ARGONNE CHICAGO

Explicit International Component

International connections are important, to– Support international science & engineering

– Connect with international Grid R&D

– Achieve consensus and interoperability But international cooperation, especially in

technology R&D, is hard An National Grid Program needs to provide

explicit support for international work– Infrastructure: networks

– Support for projects

foster@mcs.anl.gov ARGONNE CHICAGO

Resource Requirements

Persistent technology/support org: $30M– ~200 people

Application “Grid challenges”: $20M– 10 teams, with application & CS involvement

Infrastructure upgrades: $20M– Tb networks, campus infrastructures

Research: $15M– Grids of tomorrow, & 100s of grad students

International projects: $5M– Support international work in other projects