Project Coordination R. Cavanaugh University of Florida

Project Coordination

R. Cavanaugh

University of Florida

Important• UltraLight is a project with two equal and symbiotic activities

– Application driven Network R&D– Not just a networking project, nor solely a distributed data analysis

project

• UltraLight is a Physics ITR– Ultimate goal

• Enable and produce physics (more generally e-science), which could not otherwise be performed

– Network Technical Group is the “backbone” of the Project• Activity: Perform network R&D• Metric: CS Publications, demonstrations

– Applications Technical Group is the “driver” of the Project• Activity: Perform “middleware” R&D, perform LHC Physics research• Metric: CS and Physics Publications, demonstrations, community adoption

Relationship between UltraLight and LHC

• CMS– CCS

• APROM

– US-CMS S&C • Tier-1 UAF• Tier-2 Program• DYSUN Tier-2c

• ATLAS

ATLAS and CMS will integrate the UltraLight Application Services respectively into their own separate Software Stacks

LHC Computing Model: Requirements and Scale

Reminder from the Proposal

• Phase 1 (12 months)– Implementation of network, equipment and

initial services

• Phase 2 (18 Months) – Integration

• Phase 3 (18 Months)– Transition to Production

Connecting to the LHC schedule

CCS TDR Physics TDR 20 % DC Commissioning UltraLight Phases (0,1,2,3) Key goals deliverables Tests2003 Summer 5 % DC

Winter 5 % DC

2004 Spring 5 % DC, op. 10 % DC

Early adopters participate in 5 % operations

Summer CCS TDR 10 % DC Milestone 1: link the critical components together with multiple clients

Clarens-ROOT, IGUANA, COBRA / Sphinx, ShahKar, Chimera, VDT-Client, VDT-Server, RLS, MonALISA

Autumn CCS TDR 10 % DC, Vol1,Vol2Milestone 2: multi-instantiations of services, multi-clients with varying access rights

VO-Clarens, including policies (quotas) and ACLs, Metadata Catalogue,

Early adopters participate in Vol1, Vol2

Winter 10 % DC, Vol1,Vol2


2005 Spring 10 % DC, Vol2 Milestone 3: distributed system that handles some failures

Job/request state and grid monitoring information

Early adopters participate in Vol2

Summer Vol2 20 % DCEarly adopters participate in Vol2

Autumn Physics TDR 20 % DC T0,T1,T2 setupMilestone 4: client sees a grid, does not worry about physical locations of resources

Job splitting, data management, Collaborative versioning tool, resource usage estimator

Winter Physics TDR 20 % DC pre core SW

2006 Spring 20 % DCMilestone 5: richer interaction of clients with execution environment, steering of workflows

MonALISA sensors/agents, Interactive Sphinx

Early adopters participate in data analysis development in preparation for 1st beam

Summer Review

Autumn Core SWMilestone 6: self-organising grid, robust distributed system that handles most failures

job/request suspension to service high priority requests, Sphinx Quality of Service

Winter

2007 Spring 20 % capacityMilestone 7: client interaction with grid-enabled analysis environment via GUI

Summer First beamAutumnWinter

2008 Spring 50 % capacitySummerAutumnWinter


CMS Milestones

and activitie

s

UltraLight

Deliverables

UltraLight

Users

Now

6-mfromnow

Phase 1

Phase 2

Phase 3

UltraLight

Milestones

Original UltraLight Synchronisation Plan with CMS

Scope of UltraLight

• Original Proposed Programme of Work

• Current Amended Programme of Work

Relationship between Ultralight, and proposed GISNET

Project Management

• Web-page portal– Wiki, etc

• Mail-lists• Regularly scheduled phone and video meetings• Periodic face-to-face workshops• Persistent VRVS room for collaboration• Reports

– Technical– Annual

Project Structure of UltraLight

• In the Proposal– HEP Application-layer

Services– e-VLBI Application-

layer Services– Global Services– Testbed Deployment

and Operations – Network Engineering– Education & Outreach

• Current– Applications

• CMS• ATLAS• e-VLBI• Global Services• Testbed

– Network• Global Services• Testbed• Engineering

– Education & Outreach

Connection between Application and Network Tech. Groups

Project Plan

• Short term

• Longer term

6-month Overall Project Goals• Establish early ties to the LHC

– Tightly couple with LHC experiment needs and timelines • Possibly take on official roles within experiments?

– Must aggressively come up to speed to be meet LHC milestones– Already underway…

• Establish collaborative ties with external partners– OSG, Grid3, CHEPREO, AMPATH, etc– Already underway…

• Establish scope of the project– Evaluate trade-offs between

• R&D interests and application needs• Functionality and LHC timeline

– Determine what technology we• Can adopt off the shelf• Must develop to meet project goals

• Establish initial UltraLight infrastructure and user community

Early Focus of the UltraLight Technical Groups

• Networking– Construct the UltraLight Network Testbed (UNT)

• Applications– Construct the UltraLight Applications Testbed (UAT)

• Leverage GAE, Grid3, CMS, ATLAS, etc– Prepare applications that will exploit the Network Testbed

• E&O– Build relationships with external partners

• CHEPREO, CIARA, AMPATH, etc

• To first order, the Network Testbed can be instantiated independently from the Application Testbed and E&O activities– This will be our early strategy– Later, bring the (largely orthogonal) testbeds together in an evolutionary

way

Longer Term Project Goals (past initial testbed building phase)

• Global Services– Develop handles which monitor, control, and

provision network resources• Manually at first, then move to automate

– Close collaborative effort between Applications Group and Networking Group

• Requires the that the UNT and UAT work together as a coherent whole

– Combine to operate single UltraLight Testbed (UT)

• Smooth transformation to UT expected as UNT and UAT activities are very complementary

6-month Network Goals

• Inventory the different UltraLight sites– What are the current connections available at each site?– What type of switches, how many, and where are they located,

etc?

• Construct the UltraLight Network Testbed– Persistent development environment

• Piece together all the different network components– Create the NOC Team

• Begin thinking about disk-to-disk transfers – Interface with Storage Elements– Important for later integration work with Applications

6-month Application Goals• Establish UltraLight Application Grid Testbed

– Persistent development environment• Deploy Application-layer services and middleware• Deploy HEP Applications

– Create the GOC Team

• Perform System Integration Tests: Demonstrate– Interoperability of existing UltraLight Application-layer Services– Operability of HEP Applications on top of existing Application-layer

Services

• Study HEP application (ORCA & ATHENA) behaviour in-depth – Execution environment, Data/Metadata Model, Performance Profiles– Current and future versions

6-month E&O Goals

• Earni and Fabian Network engineers– Networking to brasil

• $ for E&O connected with CHEPREO

• Julio and Heidi

• EO (CHEPREO and UltraLight)

• QuarkNet– Bring people together to do science– Community aspect

6-month E&O Goals

• Quark net– Research emphasized– Monte Carlo at FNAL

• Z0’s with fake detector• Teachers see what it is like to be a particle

physicists

– HS teachers are a different group• Need to be long term oriented, substantial,

something teachers can sink teeth into

6-month E&O Goals

• Ideas for what is practical • How to fit together• Had a grid needs/assesment workshop

– There is a writeup

• Look at live data from CMS• Coordinate with applications group

– ORCA -> comma separated file -> excel

• Web browser driven– No special software

6-month E&O Goals

• Showing students how scientists collaborate– Have meetings between scientists and students (via

VRVS?)– Ask a scientist day to highlight networking and HEP– Thinkquest competition

• Awards

• Put together ad-hoc grid networks– Workshops and teaching

• Seamless bridge Ultralight/HEP tools with java applet

6-month User Community Goals

• Feedback loop between developers and users – Update and confirm the Proposed UltraLight Grid-

enabled Analysis Environment– Use actual LHC data analysis work

• Contribute to the CMS Physics TDR Milestones– Many UltraLight members are strongly engaged

• Effort recognised in CMS

– Application driver for UltraLight!• Challenging and essential!• Very tight, ambitious schedule to meet however…

CCS TDR Physics TDR 20 % DC Commissioning UltraLight Phases (0,1,2,3) Key goals deliverables Tests2003 Summer 5 % DC

Winter 5 % DC

2004 Spring 5 % DC, op. 10 % DC

Early adopters participate in 5 % operations

Summer CCS TDR 10 % DC Milestone 1: link the critical components together with multiple clients

Clarens-ROOT, IGUANA, COBRA / Sphinx, ShahKar, Chimera, VDT-Client, VDT-Server, RLS, MonALISA

Autumn CCS TDR 10 % DC, Vol1,Vol2Milestone 2: multi-instantiations of services, multi-clients with varying access rights

VO-Clarens, including policies (quotas) and ACLs, Metadata Catalogue,


Winter 10 % DC, Vol1,Vol2


2005 Spring 10 % DC, Vol2 Milestone 3: distributed system that handles some failures

Job/request state and grid monitoring information

Early adopters participate in Vol2

Summer Vol2 20 % DCEarly adopters participate in Vol2

Autumn Physics TDR 20 % DC T0,T1,T2 setupMilestone 4: client sees a grid, does not worry about physical locations of resources

Job splitting, data management, Collaborative versioning tool, resource usage estimator

Winter Physics TDR 20 % DC pre core SW

2006 Spring 20 % DCMilestone 5: richer interaction of clients with execution environment, steering of workflows

MonALISA sensors/agents, Interactive Sphinx

Early adopters participate in data analysis development in preparation for 1st beam

Summer Review

Autumn Core SWMilestone 6: self-organising grid, robust distributed system that handles most failures

job/request suspension to service high priority requests, Sphinx Quality of Service

Winter

2007 Spring 20 % capacityMilestone 7: client interaction with grid-enabled analysis environment via GUI

Summer First beamAutumnWinter



Demonstrated at SC04

Much Delivered for SC04

CMS P-TDR AnalysisEfforts AlreadyUnderway (veryambitious schedule)

Application Group is already working hardto ramp up in time for the LHC

Current UltraLight Application Group Status

Current UltraLight Network Group Status

• Also working hard– Refer to Shawn’s UltraLight Network

Technical Document

Major Milestones over the Next 6 Months

• Dec– Initial UltraLight Collaboration Meeting at CIT

• Jan– UltraLight Week at CIT: UltraLight Applications Testbed Started

• Feb: CMS PRS Meeting– UM connects to MiLR, UF connects to FLR

• Mar: CMS Week– FLR declared to be “Production”

• Apr:• May: CMS CPT Week

– First round of CMS approvals for analyses to enter P-TDR • Jun: CMS Week

– UltraLight Meeting (UM?): UltraLight Network Testbed in place

Major Project Milestones

Conclusion

Notes

• Rick – Setting LHC requirements and scale– Connect to LHC schedule – Project plan (short term + longer term)– Project management strategy

• Regularly scheduled phone and video conf• Persistent VRVS room for collaboration

– Relationship to GISNET, DYSUN, UltraLight– Make connection between application and networking groups

• Frank – CMS (LHC) use-cases– Summary of the application services document– Short term plan for Application Technical Group

Documents

Project Coordination R. Cavanaugh University of Florida