Slide David Britton, University of Glasgow IET, Oct 09 1 Prof. David Britton GridPP Project leader University of Glasgow GridPP delivering The UK Grid

Slide Slide David Britton, University of Glasgow IET, Oct 09 1

Prof. David BrittonGridPP Project leaderUniversity of Glasgow

GridPPdelivering

The UK Grid for Particle Physics

RCUK e-Science Review9th December 2009

Slide Slide David Britton, University of Glasgow 2

Monday 23rd November, 2009

…and in preparation:

2001 2002 2003 2004 2005 2006 2007 2008 2009--------- GridPP1 ------------------ GridPP2 ---------

--------- GridPP3 ---------GridPP2+

From Web to Grid From Prototype to Production

From Production to Exploitation

The LHC experiments saw first collisions; data was streamed off the detectors; and distributed to a world-wide network of computers.

RCUK e-Science Review

Slide Slide 3

Institutes

CERN computer centre

RAL,UK

ScotGrid NorthGrid SouthGrid London

FranceItalyGermanyUSA

Glasgow Edinburgh Durham

Tier 0

Tier 1National centres

Tier 2Regional groups

Offline farm

Online system

Workstations

Useful model for Particle Physics but not necessary for others

What is GridPP?

David Britton, University of Glasgow

GridPP delivers the UK part of the Worldwide LHC Computing Grid (wLCG), a global project that the UK, with funding via GridPP, played a pivotal role in creating.

The LHC will produce unprecedented amounts of data, expected to rise beyond 10PB/year.


GridPP is an internationally recognised collaboration of particle physicists and computer scientists from 19 UK universities, STFC and CERN, who have built a Grid for particle physics.

Slide Slide

“National Impact”

• GridPP has successfully developed and deployed a Grid across ~19 sites in the UK with ~20,000 cores and ~11 PB of storage.

• RAL Tier-1: Oxfordshire.• ScotGrid: Durham, Edinburgh, Glasgow.• NorthGrid: Lancaster, Liverpool,

Manchester, and Sheffield.• SouthGrid: Birmingham, Bristol,

Cambridge, Oxford, RAL PPD and Jet.• London: Brunel, IC, QMUL, RHUL, and UCL.

• This has changed the landscape of HEP computing in the UK, placing UK physicists at the forefront of research by allowing seamless access to local, national and international resources through the Grid paradigm.

David Britton, University of Glasgow RCUK e-Science Review 4

UK Tier-2s – Leverage University funding, Local

publicity

Tier-2: 15,000 CPUs, 3PB Disk

New kit from Viglen at QMUL

UK Tier-2s – Enabled development of new

University-based facilities

New machine room at Glasgow

Slide Slide


• GridPP’s impact has been broader than just the LHC experiments; with support for other experimental collaborations (such as BaBar, D0, Mice, T2K, ILC) and for theoretical physics collaborations (such as UKQCD, and PhenoGrid).

• The national impact of GridPP goes beyond Particle Physics through the support of a wide range of Virtual Organisations from other disciplines in the UK through the European EGEE project.

• (Plots show normalised CPU time per VO, Jan-Sep 2009).


All Regions Contribute

Supports more than LHC

Wide range of VO’s

Slide Slide


• GridPP has complemented the development of the UK National Grid Service, by developing a larger-scale, but more specialised Grid. In so doing GridPP is a foundation stone for the future UK NGI, which will relate to the EU structures.

• Many GridPP trained staff have moved to external positions, contributing to other areas of science, business, teaching, and software development.


EGI.euEGI.eu

HEP SSCHEP SSC

GridPPGridPP NGSNGS

UK NGIUK NGI

EMIEMI

CUECUE

European Framework-7 proposals

Slide Slide

“International Impact”


LCG Phase 1 Agreed External Personnel Profile

0

10

20

30

40

50

60

70

2001 2002 2003 2004 2005Years

FT

E *

We

igh

t

EU

USA

CERNMat

Sweden

Israel

Hungary

Portugal

Switzerland

Spain

France

Germany

Italy

UK

UK funded LCG posts at CERN

The UK, with funding via GridPP, played a pivotal role in initiating the Large Hadron Collider Computing Grid (LCG) project by funding 23 posts at CERN for three years.

Also via GridPP, the UK played a large role in the EDG and EGEE projects (a set of FP6 and FP7 European projects) that created much of the Grid middleware together with an overlapping European Grid infrastructure that embraces other disciplines in addition to HEP.

Slide Slide



56 Countries

295 Sites

180,000 CPUs

21 Sites

20,000 CPUs

Worldwide

UK

Slide Slide David Britton, University of Glasgow 9

RAL UK Tier-1 – International role

Tier-1: 4,500 CPUs, 3PB Disk, 5PB Tape

New kit from Viglen at QMUL



An international role for the RAL Tier-1 as one of 10 worldwide T1 centers. Demanding service level required for receiving and processing data flowing from the LHC. RAL has been central in developing Grid accounting for EGEE and wLCG. RAL hosts the GOCDB, developed by GridPP staff, that collects and records the configuration of all EGEE sites.

Slide Slide David Britton, University of Glasgow RCUK e-Science Review 10


Delivering Resources

The UK Tier-1 and Tier-2s provide a large resource contribution to EGEE and wLCG.

Delivering Expertise

GridPP members play leading roles:• RAL started and chaired the European Group of Certification Authorities.• RAL leads and chairs the Joint Security Policy Group and Security Vulnerabilities Group.• GridPP members have Global responsibilities within the ATLAS and LHCb computing projects.• The UK chairs the wLCG Grid Deployment Board and previously the EGEE Project Board.• Members have worked on standardization in many areas in the international Open Grid Forum.

Slide Slide

“Wealth Creation”


GridPP has supported:

• imense and iLexIR start-up companies at Cambridge (Camtology – see below) with the GridPP/ATLAS/LHCb developed Ganga interface, and resources.• Econophysica (mathamatical models for commodity trading) at QMUL, with access to resources.• Total Oil (geoscience research) from Aberdeen, with help establishing a Virtual Organisation to run their applications on the EGEE Grid.

GridPP’s involvement in wLCG and EGEE has led directly to the formation of Constellation Technologies, a pioneer in Global Grid infrastructure software.

Slide Slide

“Wealth Creation”


Slide Slide

“Quality of Life”


The Grid allows rapid deployment of large-scale resources to tackle topical problems. In 2006, GridPP resources made the largest contribution to a pan-european search for a cure for bird-flu.

Slide Slide David Britton, University of Glasgow RCUK e-Science Review 14

Search for a cure for Malaria:

In 2007, the WISDOM docked over 41 million compounds in just six weeks, the equivalent of 80 years work for a single PC. The WISDOM team identified some 5000 interesting compounds, from which they found three families of molecules that could be effective against the malaria parasite.

The UK (GridPP) contributed almost half of the total computing resources.

“Quality of Life”

Slide Slide

“Impact on e-Science”


GridPP’s key impact on e-science is to demonstrate the potential of a Grid for science across multiple disciplines.

GridPP has built and runs a Grid in a complex, data-intensive environment at a scale well above anything previously done.

A Grid infrastructure now exists that can run tens of millions of jobs annually.

Slide Slide

“Impact on e-Science”


In June 2009 a world-wide full-scale test of the computing for LHC data-taking was performed (STEP09):

The UK provided over 2 million hours of CPU time in 2 weeks. ATLAS data reprocessing recalled more than 110TB of data from tape (7x higher than nominal rate for 2010 and the best performance world-wide). CMS data was imported to RAL at 100 MB/s and achieved an aggregated recall from tape of 180 MB/s. The UK contributed 32% of the Global CPU time for the LHCb experiment. The Glasgow Tier-2 ran more analysis jobs than any of the other ~60 sites world-wide.

STEP09 was a notable success for the UK. Worldwide, the wLCG Grid demonstrated groundbreaking performance.

LHCb CPU contributions

Jobs at RAL

ATLAS reprocessing jobs.

Slide Slide

Panel Questions


Please provide any available data, or at least anecdotes, about how the e-science program to date has in fact enhanced collaboration in you area of research. The LHC experiments require collaboration on a global scale to tackle the volume and complexity of the data from the LHC. The worldwide LHC Computing Grid, of which GridPP represents 10-15%, is an essential infrastructure, without which the LHC data could not be analysed. In essence, the e-infrastructure has realized the goal of global collaboration in High Energy Physics at a scale that could not be supported by the traditional computer centre.

Enhanced collaboration has been observed: • within the regional groups of universities making up the distributed Tier-2s, driven by GridPP organisation and funding;• between the Tier-2s and with the UK Tier-1, driven by the GridPP deployment team and support structure and by sharing the UK contribution to WLCG and EGEE. • between the UK and other countries both within the WLCG and EGEE projects. The coordination of GridPP has allowed the UK to present a credible, organized and common face to these international projects

Slide Slide

Panel Questions


What evidence (hard data or anecdotes) of data sharing has occurred in your e-science project? The STEP09 exercise in May 2009 exercised the worldwide LHC Computing Grid at full scale and across all areas. Data were shipped out from CERN via the OPN optical network to the ten Tier-1 centers across the world. There, it was reconstructed, archived, restored, and derived data sets made available to the network of Tier-2 centers. At the latter, the data were analysed and physics quantities extracted. This exercise demonstrated that the infrastructure works at, and in many areas, beyond, the levels required for data taking in 2010 and demonstrates both the divide-and-conquer philosophy of the Grid, and the intrinsic sharing of physics data across the globe.

Slide Slide

Panel Questions


What evidence of sharing mechanisms or services has occurred in your e-science project? Through e-science funding of GridPP, the UK has been a major contributor to both wLCC and EGEE. In the year to July 2009, GridPP provided 17% of EGEE’s computer time and contributed nearly 85 million CPU hours (KSI2K hours) to more than 40 EGEE virtual organizations across a wide range of disciplines. Of this, approaching 70 million CPU hours were provided to LHC VOs, 14% of WLCG’s worldwide total. The entire wLCG infrastructure is a mechanism and service for sharing data between members of a recognized collaboration. Collaborations are recognized via the Virtual Organisation Management System (VOMS) that verifies the membership and rights of individual. Higher-level tools are provided to help locate and move data (the LHC file catalogue, LFC, and the File Transfer Service, FTS). GridPP’s production grid includes a variety of services that are shared by all UK institutes and users, such as the BDII Information Service; Workload Management Services and Nagios monitoring.

Slide Slide

Panel Questions


What software have you built that has been used outside your project? GANGA – User interface and job submitter – used extensively by HEP, other disciplines and commercial users. Described in later talks.

APEL accounting – used to populate the GOCDB with information on all sites in EGEE.

GridSite - a certificate-based website management system, that allows members of an organisation to collaborate in maintaining web pages etc. Used to create a dozen or so external (to HEP) websites.

The RealTimeMonitor – high profile tool produced by GridPP; used throughout the world to demonstrate the Grid in action.

R-GMA – an information and monitoring tool developed at RAL, which was been deployed in LCG middleware and gLite.

Contributions to many others.

Slide Slide

Panel Questions


What disciplines have been involved in your project? Describe their interaction.

GridPP is fundamentally a collaboration between Particle Physicists and Computer Scientists.

The PEGASUS project based at LSE studied GridPP over a number of years – involved a lot of interaction (observation; interviews; talks).

GridPP supports over 40 Virtual Organisations covering a wide range of scientific disciplines.

Slide Slide

Panel Questions


What major investments made by the e-science core program and/or JISC (such as the National Grid Service) have you used in your research?

GridPP is dependent on the academic network hosted by JANET-UK and the dedicated OPN optical network to CERN.

GridPP developed the original UK Certificate Authority and then passed this over to the NGS/JISC to run.

Slide Slide

Panel Questions


What has your project contributed back to the larger research community?

Leadership roles (in ATLAS, LHCb, wLCG, EGEE, international security)

Software (already described).

The UK CA just mentioned.

Trained staff who have moved on into the wider world.

Standardisation work in the international OGF context (eg information schema).

Resources for 40 VOs.

Driven technology – pushed agenda for lightpath service in the UK (UKLIGHT project).

Slide Slide

Panel Questions


What has been the major impact of the e-science program to date on your research and allied education program?

The e-science programme has enabled the UK to participate in the worldwide e-infrastructure for LHC computing: without the e-Science program, UK physicists would not be in a position to exploit the data expected shortly from the LHC.

Slide Slide

Panel Questions


What is your vision for e-science going into the future? Stress what is most critical to the future of your and/or your communities research aspirations?

See next slide.

Slide Slide

Conclusion

David Britton, University of Glasgow 27

GridPP has achieved everything that it set out to do, but this is the start, not the end point: The Grid will have to serve the UK particle physics community and others, for the next 15-20 years. It will need to evolve to deal with increasing demands (e.g. SuperLHC luminosities), cope with new discoveries (e.g. Higgs), and respond to new challenges (e.g. ILC). It will need to incorporate new technology (e.g. many-core processors, GPUs), adapt to new structures (e.g. the UK NGI), and service a growing number of communities without losing focus.

What is needed is the sort of forward-thinking vision of Sir John Taylor in 2001 accompanied by a plan for long-term support.


Documents

Slide David Britton, University of Glasgow IET, Oct 09 1 Prof. David Britton GridPP Project leader University of Glasgow GridPP delivering The UK Grid