TeraGrid Program Year 5 Overview

John TownsChair, TeraGrid Forum

Director, Persistent InfrastructureNational Center for Supercomputing

ApplicationsUniversity of Illinois

2

Our Vision of TeraGrid•Three part mission:

– support the most advanced computational science in multiple domains

– empower new communities of users– provide resources and services that can be extended to a broader

cyberinfrastructure

•TeraGrid is…– an advanced, nationally distributed, open cyberinfrastructure

comprised of supercomputing, storage, and visualization systems, data collections, and science gateways, integrated by software services and high bandwidth networks, coordinated through common policies and operations, and supported by computing and technology experts, that enables and supports leading edge scientific discovery and promotes science and technology education

– a complex collaboration of over a dozen organizations and NSF awards working together to provide collective services that go beyond what can be provided by individual institutions

3

Strategic Objectives

•Objectives determined from considering numerous inputs– user input via various mechanisms

• surveys, user contacts, advisory bodies, review panels, etc.

– technical input from TG staff

•Planning for PY5 started by identifying 5 high level project strategic objectives– Enable science that could not be done without TeraGrid– Broaden the user base– Simplify users lives– Improve Operations– Enable connections to external resources

Advanced User Support – PY5 Objectives

•Continue to work collaboratively across TG areas to provide advanced support– activities selected based on user input, TRAC

recommendations, ad hoc requests, needs indentified by staff, etc.

•Advanced Support for TeraGrid Applications (ASTA)– maintain ~25 ASTA projects throughout the year– contributions to Science Highlights

•Advanced Support for Projects (ASP)– “foundation work” to allow use of scientific codes and

applications on the HPC resources.– projects with potential to benefit a number (10+) of users in

a domain science or TeraGrid users in general•Advanced Support for EOT (ASEOT)

– content development and presentation; workshop, panel and BOF participation

– outreach to new CI programs: DataNet, iPlant, etc.

4

User Services – PY5 Objectives

•Further improve:– the responsiveness and quality of problem resolution

for our users– the feedback mechanisms by which our users

become active partners in enhancing the ability of the TeraGrid

•By July 2010, ticket statistics will show that consultants communicate at least once per 7-day period with the user regarding the status of any ticket under investigation

•2010 user satisfaction survey will show promptness and quality of user support are rated at least 85%

5

User Facing Projects and Core Services – PY5 Objectives

• Resource Description Repository– further consolidate, integrate

resource information in central environment, IIS

– simplify tasks for RPs– improve consistency of info for users

• Migration to integrated backend for Portal, Web, and Wiki– more monitoring, measurement

for users• Work toward user-created portal logins

– integration with Shibboleth authentication

• IIS-based software catalog– combine CTSS/3rd-party software

information• New access tools

– job submission, metascheduling in TGUP

• Customization & Personalization– personalized, dynamic TGUP Home

page– domain views– User Discussion Forums– $PORTAL_HOME file space for TG

Users

6

Science Gateways – PY5 Objectives

•Support Services– ability to routinely

count end gateway users

– create a gateway software listing capability

– step by step integration instructions for gateways

– straightforward process for making use of and securing community accounts

•Targeted Support Projects– GridChem– PolarGrid– OSG cloud on TeraGrid via

NIMBUS– SIDgrid– SimpleGrid– Earth Systems Science

Gateway– Computational

Infrastructure for Geodynamics

– SCEC and NEES– Cyberinfrastructure for

End-to-End Environmental Exploration

7

Networking, Operations, and Security – PY5 Objectives

•Single Sign On– complete PSC deployment of backup MyProxy service– complete integration of Shibboleth support into

Internet Framework/TGUP– bridging authorization with OSG and EGEE to support

other activities

•Maintain support for Core Services– TGCDB, POPS, account and allocations management,

and the TeraGrid Ticketing System

• Inca improvements– “views” based on user needs and output of QA and

CUE– interface for RP administrators to execute tests on-

demand– Knowledgebase for errors, causes and solutions

8

RP Operations – PY5 Objectives

•New Resources– NICS’ Kraken system upgrade– NCAR’s Sun Ultra 40 system dedicated to data

analysis and visualization

•Retiring Resources and TeraGrid Extension– all non-Track 2 resources slated to discontinue as

TeraGrid resources after March 31, 2010– TeraGrid Extension proposal pending

• requests continued support for several resources –12 months: Abe, QueenBee, Lonestar, Steele, Quarry, Lincoln–3 months: Pople

– orderly transition of users to resources with ongoing support

9

Data and Visualization – PY5 Objectives

• Implement Data Architecture recommendations– User Portal integration; Data Collections infrastructure;

archival replication services; continued investigation of new location-independent access mechanisms (Petashare, Reddnet)

• Improve data movement performance and reliability •Complete production deployments of Lustre-WAN

– develop plans for next-generation Lustre-WAN and pNFS technologies

•Architect a comprehensive TeraGrid-wide data analysis plan– establish policies, documentation and training that provides a

comprehensive data analysis picture to the user community.

•Expand resources and capabilities in the TG Visualization Gateway via integration of RP tools into TG Vis Gateway

10

Software Integration – PY5 Objectives

•Expand the TeraGrid system information available via TeraGrid's integrated information services

•Deliver an application/service hosting capability on one or more TeraGrid resources

•Spin-off Software Build & Test capability from Core into its own capability kit

• Improved coordinated capability user documentation– e.g. CTSS documentation is re-organized around user

focused use cases

11

Quality Assurance/Common User Environment – PY5 Objectives

•Quality Assurance Working Group– expediting problem resolution when service failures

detected– improving the use of the Inca monitoring framework– validate common user environment (with CUE WG)– develop/propose a more formal process for CTSS

software deployment (with Software WG)

•Common User Environment Working Group– remove barriers to user movement between TeraGrid

resources• CUE Documentation (CUED)• CUE Management System (CUEMS)• CUE Build Environment (CUBE)• CUE Testing Platform (CUETP)• CUE Variable Collection (CUEVC)

12

Training – PY5 Objectives

•Offerings to include live, synchronous and asynchronous offerings covering all TeraGrid resources and services– expansion of HPC University offerings– conduct at least 50 training sessions, of which 10 will

include new content– at least 30 sessions offered synchronously; at least 8

new async topics

•Establish mechanism for users to provide TeraGrid with feedback on their evolving training needs

•Work with other regional, national and international groups (e.g. OSG, NCSI, DOE, NASA, etc.)– avoid duplication of effort

13

Education and Outreach – PY5 Objectives

• Education - continue K-12 and undergrad professional development and curriculum development efforts– SC09 Education Program - 10 summer workshops and Nov event– plan to conduct over 100 workshops, institutes and tutorials

• Outreach - expand outreach, especially with under-served communities– expand Campus Champions to > 60 campuses; focus on under-represented

campuses (e.g. EPSCoR sites)– TeraGrid Pathways program– conduct at least two petascale workshops with Blue Waters; outreach to at

least 20 Professional Society Meetings• Student Engagement - expand opportunities

– Student competitions @ TG09, SC09; computational science problem of the week

– Graduate Research Fellows seminars and allocations; student internships, REUs and workshops

– support at least 300 K-12 students and 500 college students• EOT-wide

– EOT highlights •EOT Highlights produced in time for SC09; EOT monthly newsletter launched and being distributed

– Evaluation•pilot instruments through April; move into production in May•create database for longitudinal impact studies; conduct first 6-month analysis by December•continue to review and improve metrics of long-term impact 14

External Relations – PY5 Objectives

•Provide easily accessible information– revise the public web site to improve the content,

navigation and ease of access to information

•Convey the impact of TeraGrid on research, education and society – develop science impact stories– Science Highlights

•Work with other regional, national and international groups– e.g. iSGTW and other professional societies– focus on under-represented groups

15

Project Management – PY5 Objectives

•Tracking and reporting– track progress against the IPP– report progress against the IPP quarterly and

annually• provide updates as otherwise requested

•Change Management– manage change management process for IPP

•Planning– anticipate coordinating development of IPP for

proposed TeraGrid Extension• timing issue

16

TeraGrid → XD Transition Planning

•[[Need to pull bullets together yet..]]

•All activity areas have effort reserved for TeraGrid → XD transition effort as appropriate

PY5 Total TeraGrid Budget Distribution

•TeraGrid budget most closely resembles the RP budgets – RP expenditures ~4x those of the GIG

•Bulk of TeraGrid funds go directly to – providing facilities to scientific researchers – assuring they have necessary support to make

productive use of them

18

Management,Finance, and Admin

13%

User Support andScience Outreach

29%

Infrastructure,Resources, and Services

50%

Science Gateways5%

CI Development3%

PY5 Total TeraGrid Budget Detail

Functional AreaStaffing Level

(FTE)Staffing

Costs ($k)Other Costs

($k)

HPC Operations 38.2 $ 4,147 $ 4,629

Network Operations and Security 18.2 $ 2,588 $ 1,219

Core Services 13.2 $ 1,435 $ 106

Data and Visualization 35.0 $ 4,738 $ 1,674

Scheduling 3.8 $ 615 $ 70

Science Gateways 14.0 $ 2,153 $ 30

Maintaining Data Collections 1.9 $ 168 $ 155

Helpdesk Support 11.0 $ 945 $ 47

User Consulting Support 33.5 $ 4,304 $ 465

Advanced Applications Support 15.5 $ 2,638 $ -

Training 8.4 $ 1,140 $ 120

Education and Outreach 12.8 $ 1,745 $ 692

Software Development 6.1 $ 1,247 $ 5

Project Management 8.1 $ 1,555 $ 146

Administrative Costs 11.8 $ 2,258 $ 1,767

Material and Supplies 0.0 $ - $ 414

Publication Costs 0.0 $ - $ 42

Consultant Services 0.0 $ - $ -

Computer Services 0.0 $ - $ -

Subawards 0.0 $ - $ 906

Facilities Costs 0.0 $ - $ -

Other Costs 11.8 $ 2,258 $ 406

Conference Costs (TG08 and SC08) 2.8 $ 294 $ 291

Total 234.0 $31,971 $11,416

19