University of VirginiaORNL NeSSI Workshop Computational Infrastructures for Science Marty Humphrey Assistant Professor Computer Science Department University

University of Virginia ORNL NeSSI Workshop

Computational Infrastructures for Science

Marty HumphreyAssistant ProfessorComputer Science DepartmentUniversity of Virginia

NeSSI WorkshopOctober 13, 2003


“Traditional” Computational Science

• SP3, O2K, Linux clusters, etc.• PBS, LSF, LoadLeveler, etc.• Archival storage• MPI• Viz• SSH, SCP


Grid Definition (Foster and Kesselman)

• “Coordinates resources that are not subject to centralized control….”

• “Using standard, open, general-purpose protocols and interfaces…”

• “To deliver non-trivial qualities of service.”


Grid Computing

Host/OS 1,1 Host/OS 2,1

Grid “Operating System”

Host/OS 3,1


Grid User Wish-List• Who cares where it is?

– It must always be available when I need it

• Make it secure– no one can steal my data– no one can pretend to be

me– don’t tell me who I

will/can trust• Choose secure, fast,

cheap resources • Give me reasonable

quality of service

• Don’t make me manually move/copy stuff around

• Don’t make me learn a new OS– Allow me to run my existing

apps• I don’t want errors

– If errors occur, tell me in plain English how I can avoid them next time

• Allow me to more easily collaborate

Darnit, make my life easier !


Example: Transparent Remote Execution

• User initiates “run”• User/Grid SW selects site/resource• Grid SW copies binaries (if necessary)• Grid SW copies/moves input files• Grid SW starts job(s)• Grid SW monitors progress• Grid SW copies output files

Forms the basis of parameter-space or monte carlos


Web Interface

for CHARMM

and Amber


Status of CHARMM or Amber Run


Grid Focus: Virtual Organizations

• Logical grouping of resources and users• Support community-specific discovery• Specialized “views”• Dynamic collaborations of individuals and

institutions• Policy negotiation and enforcement will be

key issues looking forward


Grid Landscape Today: Globus

• Grid Resource Allocation and Management (GRAM)– Gatekeeper, Jobmanager (RSL

“schedulerspeak”)

• Grid Security Infrastructure (GSI)• Metacomputing Directory Service (MDS) (via

OpenLDAP)– Grid Index Information Service (GIIS)– Grid Resource Information Service (GRIS)

• GridFTP


Grid Landscape Today: Globus (cont.)

• “Add-ons”: – MPICH-G2– Replica Catalog and Management– Community Authorization Service (CAS)– Condor-G– etc.

• Basis of many large-scale Grids…


Selected Major Grid Projects (Oct 2001)

Name URL & Sponsors

Focus

Access Grid www.mcs.anl.gov/FL/accessgrid; DOE, NSF

Create & deploy group collaboration systems using commodity technologies

BlueGrid IBM Grid testbed linking IBM laboratories

DISCOM www.cs.sandia.gov/discomDOE Defense Programs

Create operational Grid providing access to resources at three U.S. DOE weapons laboratories

DOE Science Grid

sciencegrid.orgDOE Office of Science

Create operational Grid providing access to resources & applications at U.S. DOE science laboratories & partner universities

Earth System Grid (ESG)

earthsystemgrid.orgDOE Office of Science

Delivery and analysis of large climate model datasets for the climate research community

European Union (EU) DataGrid

eu-datagrid.orgEuropean Union

Create & apply an operational grid for applications in high energy physics, environmental science, bioinformatics

g

g

g

g

g

g

New

New


Selected Major Grid ProjectsName URL/Sponsor Focus

EuroGrid, Grid Interoperability (GRIP)

eurogrid.orgEuropean Union

Create tech for remote access to supercomp resources & simulation codes; in GRIP, integrate with Globus Toolkit™

Fusion Collaboratory

fusiongrid.orgDOE Off. Science

Create a national computational collaboratory for fusion research

Globus Project™ globus.orgDARPA, DOE, NSF, NASA, Msoft

Research on Grid technologies; development and support of Globus Toolkit™; application and deployment

GridLab gridlab.orgEuropean Union

Grid technologies and applications

GridPP gridpp.ac.ukU.K. eScience

Create & apply an operational grid within the U.K. for particle physics research

Grid Research Integration Dev. & Support Center

grids-center.orgNSF

Integration, deployment, support of the NSF Middleware Infrastructure for research & education

g

g

g

g

g

g

New

New

New

New

New



Grid Application Dev. Software

hipersoft.rice.edu/grads; NSF

Research into program development technologies for Grid applications

Grid Physics Network

griphyn.orgNSF

Technology R&D for data analysis in physics expts: ATLAS, CMS, LIGO, SDSS

Information Power Grid

ipg.nasa.govNASA

Create and apply a production Grid for aerosciences and other NASA missions

International Virtual Data Grid Laboratory

ivdgl.orgNSF

Create international Data Grid to enable large-scale experimentation on Grid technologies & applications

Network for Earthquake Eng. Simulation Grid

neesgrid.orgNSF

Create and apply a production Grid for earthquake engineering

Particle Physics Data Grid

ppdg.netDOE Science

Create and apply production Grids for data analysis in high energy and nuclear physics experiments

g

g

g

g

g

New

New

g



TeraGrid teragrid.orgNSF

U.S. science infrastructure linking four major resource sites at 40 Gb/s

UK Grid Support Center

grid-support.ac.ukU.K. eScience

Support center for Grid projects within the U.K.

Unicore BMBFT Technologies for remote access to supercomputers

g

g

New

New


PetaScale Virtual-Data Grids

Virtual Data Tools

Request Planning &

Scheduling ToolsRequest Execution & Management Tools

Transforms

Distributed resources(code, storage, CPUs,networks)

Resource Management

Services

Resource Management

Services

Security and Policy

Services

Security and Policy

Services

Other Grid ServicesOther Grid

Services

Interactive User Tools

Production TeamIndividual Investigator Workgroups

Raw data source

~1 Petaflop~100 Petabytes

Slide courtesy of Paul Avery


Data Grid Architecture

Application

Planner

Executor

Catalog Services

Info Services

Policy/Security

Monitoring

Repl. Mgmt.

Reliable TransferService

Compute Resource Storage Resource

DAG

DAG

DAGMAN, Kangaroo

GRAM GridFTP; GRAM; SRM

GSI, CAS

MDS

MCAT; GriPhyN catalogs

GDMP

MDS

Globus

Slide courtesy of Ian Foster


US-iVDGL Data Grid

UF

Wisconsin

Fermilab BNL

Indiana

Boston USKC

Brownsville

Hampton

PSU

J. Hopkins

Caltech

Tier1Tier2Tier3

FIU

FSUArlington

Michigan

LBL

Oklahoma

Argonne

Vanderbilt

UCSD/SDSC

NCSA

Partners?EUCERNBrazilAustraliaKoreaJapan

Slide courtesy of Paul Avery


Image courtesy Harvey Newman, Caltech

Data Grids for High Energy Physics

Tier2 Centre ~1 TIPS

Online System

Offline Processor Farm

~20 TIPS

CERN Computer Centre

FermiLab ~4 TIPSFrance Regional Centre

Italy Regional Centre

Germany Regional Centre

InstituteInstituteInstituteInstitute ~0.25TIPS

Physicist workstations

~100 MBytes/sec

~100 MBytes/sec

~622 Mbits/sec

~1 MBytes/sec

There is a “bunch crossing” every 25 nsecs.

There are 100 “triggers” per second

Each triggered event is ~1 MByte in size

Physicists work on analysis “channels”.

Each institute will have ~10 physicists working on one or more channels; data for these channels should be cached by the institute server

Physics data cache

~PBytes/sec

~622 Mbits/sec or Air Freight (deprecated)




Caltech ~1 TIPS

~622 Mbits/sec

Tier 0Tier 0

Tier 1Tier 1

Tier 2Tier 2

Tier 4Tier 4

1 TIPS is approximately 25,000

SpecInt95 equivalents


Global Grid Forum (GGF)

• Grid standards• Best practices• Broad academic, national lab, and industry

involvement• Areas: Applications and programming environments,

architecture, data, information systems and performance, Peer-to-Peer, Scheduling and Resource Management, Security

• GGF9 was last week in Chicago


Many Excellent DOE Grid and Middleware Projects

• Reliable and Secure Group Communication• Commodity Grid Kits (CoGKits)• Middleware for Science Portals• Scientific Annotation Middleware (SAM)• Storage Resource Management for Data Grid

Applications• Common Component Architecture (CCA)• Scalable Software Initiative


Next-Generation Grids

• Web Services– “Semantically encapsulate discrete

functionality”– Loosely coupled, reusable components– XML, SOAP, WSDL, UDDI, etc.– Broad industrial support: Microsoft, IBM, Sun,

BEA, etc.

• Open Grid Services Architecture (OGSA)– Combine Grids (Globus, Legion) with Web

Services– GT3: Java, AXIS, J2EE, etc.


OGSI.NET

• University of Virginia hosting environment for Grid Services based on Microsoft Web Services approach– Focus: Grid security (e.g., explicit trust

management)– Focus: Grid programming models– Focus: Connection between UNIX and Win*


Biomolecular VO based on OGSI.NET


Grid Challenges: “UK E-Science Gap Analysis”

(Fox and Walker, Jun 30 2003)

• Security: VPNs/Firewalls, fine-grain access control

• Workflow (“orchestration”) specs and engines• Fault tolerance• Grid adaptability (e.g., real-time support)• Ease of use• Grid federations


Future Directions

• Grid has come a long way• Merging of Grid and Web Services shows

promise• Many difficult issues remain

– Manageable security– Integration with legacy applications/tools

• Challenge for SNS: Identify and meet requirements not being met by current Grid technologies

Documents

University of VirginiaORNL NeSSI Workshop Computational Infrastructures for Science Marty Humphrey Assistant Professor Computer Science Department University