SweGrid in practice,Grid(s) in Sweden

TREFpunkt Karlshamn, 20-21 April,2005

Balázs KónyaLund University

NorduGrid Collaboration

The Grid, as seen by Ursula The Grid, as seen by Ursula Wilby, Sydsvenskan 10.2.2002Wilby, Sydsvenskan 10.2.2002

2

outline

• Grid Computing as of Today The Grid Vision How Grids work Production Grids

• SweGrid Hardware Operation & Services Middleware Users & Applications

• (Grids in Scandinavia)• Summary

3

Some history:TREFpunkt 2002

• Expression of interest: Development of GRID testbed in Sweden (Swedish informal GRID consortium, T Ekelöf). SWEGRID has been put into

production

• Design and implementation of the NorduGrid Middleware Architecture (started February 2002) Advanced Resource Connector

(ARC) from the NorduGrid Collaboration has become one of the major grid middlewares

4

Grid Computing as of Today:An old vision ...

The “Grid book” 1998: Computational Power Grid A future infrastructure of

computing and data management

a new utility, next to the existing water, heating, electricity, ...,

Computing from the tap

source: IBM

5

Grid computing as of Today:Turning into reality...

• Europe is building its E-Infrastructure FP5: ~50 million Euro

FP6: ~140 million Euro

• Grids become part of the European computing infrastructure

• Production quality Grids are being used on daily basis to solve scientific problems

• Vision: the grid layer should be seamlessly “integrated” with the network

5

“The Grid, for Europe, is far more than resource sharing. It is a big step forward to build the Cyberinfrastructure for a united research community tackling the grand challenges of our universe. It is a coordinated, single economic engine preparing to compete with Asia and the United States.” Wolfgang Gentzsch

6

Grid Computing as of Today

• The Grid Phenomenon (or hype) continues infecting the academics, the IT industry and the governments (bureaucrats).

• Grid computing has become the Holy Grail of distributed computing and a major marketing tool for the IT sector.

• The next BIG thing promised after the internet: World Wide Web access to information World Wide Grid access to computing capacity and

beyond ...

• Meanwhile a lot of research/development has been carried out: Matured middlewares & production grids

7

The rational behind the Grids remains the same

Network vs. computer performance

Computer speed doubles every 18 months

Network speed doubles every 9 months

Even data storage outperforms CPU

Science is pursued in collaborations

Teamwork: large experiments, instruments, data sets

Need for dynamic, flexible, secure, coordinated resource sharing

Need for sharing of geographically distributed resources

The world gets super-connected

Moore’s Law vs. storage improvements vs. optical improvements. Graph from

Scientific American (Jan-2001) by Cleo Vilett, source Vined Khoslan, Kleiner, Caufield and

Perkins.

8

Standa

rds &

Inter

opera

bility

???

Grid Computing as of Today:middlewares

The middleware is the Operating system of the Grid A collection of software which implements grid functionalities No complete solution

Major Middlewares (in alphabetic order):1)EDG-line: EDG/LCG/Glite

2)Globus Toolkit (incompatible versions)• GT v2 (pre-WS Globus)• GT v3 (deprecated)• Globus Toolkit v4

• WS-RF Framework• alpha/beta quality, first release in May

3)Grid3/OSG (GT2 + Condor)

4)NorduGrid/ARC

5)Unicore

9

How Grid works:overview of a Grid session

The GRID middleware:• Finds convenient places for the scientists "job" (computing task) to be run

• Optimizes use of the widely dispersed resources

• Organizes efficient access to scientific data

• Deals with authentication to the different sites that the scientists will be using

• Interfaces to local site authorization and resource allocation policies

• Runs the jobs

• Monitors progress

• Recovers from problems

•... and ...

•Tells you when the work is complete and transfers the result to the requested location

10

Grid session (animated)

RSL

input

output

output

outputCluster

ClusterClusterinfosys

program input

?? ?

?

11

Production Grids: EGEE

• The largest Grid project, funded by the EU FP6 http://eu-egee.org Time span: April 2004 to April 2006, with planned 2-years extension Partners: 70+ institutes worldwide Major activities:

• Raise Grid awareness• Provide resources and operational support via Grid technologies for

scientists• Maintain and improve Grid software

• Is a follow-up to the European DataGrid project (EDG), inheriting large parts of its Grid solutions Middleware: gLite, based on Globus and Condor

• Is based on the resources contributed to the LHC Computing Grid (LCG)

• The first release of the gLite middleware is to come out this month• Is widely expected to become the largest production Grid• Sweden is participating with SweGrid

12

Production Grids: Grid3

• Grid3: originally, provided infrastructure and simple Grid-like solution for High Energy Physics computing in USA http://www.ivdgl.org/grid2003/ Collaboration of several research centers, active: 2003-2004 Uses Globus and Condor, plus few own developments

• Was proven to be able to provide reliable services to other applications

cms dc04

atlasdc2

13

Production Grids:Open Science Grid

• Continuation and extension of Grid3 achievements http://www.opensciencegrid.org/ Consortium, aims at creating a national US Grid infrastructure Focus on general services, operations, end-to-end performance Takes over Grid3 in Spring 2005 (NOW)

14

Production Grids: NorduGrid

• Collaboration of Nordic researchers, developing an own Grid middleware solution (ARC) since 2001 http://www.nordugrid.org

• A Grid based on ARC-enabled sites Driven (so far) mostly by the

needs and resources of the LHC experiments

Dozens of other applications• Assistance in Grid

deployment outside the Nordic area

• SweGrid is part of this Grid

15

Production Grids: GRID.IT

• National Grid infrastructure in Italy http://www.grid.it/ Funded for 2003-2005 Also triggered by HEP community

needs, but expands to many other applications

• Like EGEE, heavily based on the EDG Grid middleware Does specific developments, most

notably, portals and monitoring tools

• Contributes to EGEE

16

Production Grids: SweGridbasic facts

• SweGrid is a computational Grid consisting of six dedicated clusters

• A Swedish National Computational resource The hardware is funded by a grant from the

Wallenberg foundation Operational costs and personnel for support and

maintenance are funded by the Swedish Research Council

• Official inauguration: March 2004 It was the first dedicated Grid resource in

Scandinavia

• SweGrid operates in a production mode since then Runs on the reliable ARC middleware Offers support and maintenance services

ww

w.s

weg

rid.

se

17

SweGrid:Architecture

• Six dedicated clusters located at the Swedish academic computing centres.

• Each of the 6 clusters consists of 100 computing nodes and 2 TB disk storage

• Homogeneous hardware to simplify initial development and deployment

• The sites are connected through the 10 Gb/s GigaSunet network

• The OS installed differs between the clusters: RedHat Linux 7.3, Fedora Core 1, Debian 3.0

• The primary Grid middleware is ARC, LCG/Glite is also being deployed

18

SweGrid: Grid node

•100 compute nodes •IA-32, 1 processor/node

•2.8 GHz Intel P4

•2 Gbyte memory

•875P chipset

•800 MHz FSB

•dual memory channel

•2 TByte temporary storage •FibreChannel for bandwidth

•14 x 146 GByte 10000 rpm

•1 Gigabit internal interconnect •Not full bisectional bandwidth

•Access server •Limited login

•1 Gigabit to SUNET, directly attached

19

SweGrid middleware: ARC

• One of the few matured Grid solutions Continous development Good support

• Attractive for resource owners Non-intrusive Portable (variety of OS, LRMS) Simple installation procedure Scalable, reliable Performs well in ATLAS Data

Challenges Scalabe: serves a grid of ~50

sites and 5000 CPUs, 50000 jobs/month

• Attractive for users Robust, portable Relatively feature rich Client can be installed everywhere

by anyone Plenty of documentation

IANA registered Grid ports:2135, 2811

20

Swegrid Users: Resource allocation

• SweGrid is part of the Swedish National Infrastructure for Computing (SNIC)

• Potential users must apply for resource allocations (CPU quotas)

• Resource allocations for users are done on a peer-review basis via the Swedish National Allocation Committee (SNAC), upon requests 1/3 is (pre-)allocated for LHC computing The rest is distributed between chemistry,

genomics, meteorology etc – whoever applies Allocation is done twice a year 20 applications in the last round requesting

153740 "CPU-hours". 18 was granted to the sum of 82300 "CPU-hours"

21

SweGrid services

• Operation, Support, Maintenance Six assigned Swegrid system administrators, one per site Detect and resolve hardware failures Installation and upgrades of software Monitoring and performance enhancements Tutorials Helpdesk Application support

• SweGrid Accounting System (SGAS) Developed within the SweGrid project Resource allocation and enforcement system Grid bank Usage tracking Test deployment on SweGrid

22

SweGrid Users:Application areas & Research

• SweGrid is designed mainly for through-put computation, to quickly process large numbers of loosely coupled non-parallel computations

• Users come from various fields of science: climate research, material science, physics, chemistry and biology www.pdc.kth.se/grid/swegrid-vo/volist.txt

• The SweGrid also hosts research activities in various fields in IT such as Distributed data bases, Scheduling, brokering Data management, replication

• SweGrid has also been developing the SweGrid Accounting System (SGAS)

23

Swegrid Users:Main User Group

• High Energy Physics (HEP) Community has been the driving force behind Grid

• Initially the main customers of SweGrid 1/3 of SweGrid is reserved

for HEP

• Data Challenges ~110.000 jobs in the second

half of 2004

24

SweGrid:a multidisciplinary grid

Date: 04/15/2005 02:38 PMTo: nordugrid-discuss@nordugrid.orgSubject: [NG-disc] Multidisciplinary grid

Right now bluesmoke has jobs running belonging to:

- climate simulation- astrophysics- bioinformatics- materials science

Actually no HEP at all, at the moment.

I think we might be underestimating the importance and coolness of this.

-- Leif Nixon - Systems expert ------------------------------------------------------------ National Supercomputer Centre - Linkoping University ------------------------------------------------------------

25

Grid of Grids

• Eesti Grid

• Finn activities

• SweGrid

• NorGrid

• DCGC

• NDGF

• Germany

• Switzerland

• Slovenia

• Slovakia

26

Estonian Grid

• Technical details:- 122 CPUs in 9 clusters- 3.8 TB storage- NorduGrid ARC middleware- GÉANT connection 622Mbit/s, between the clusters 1Gbit/s

• Operations: EG CA EG CA is member of EUGridPMA

Technical support and coordination group:

Steering committee established at the Ministry of

Education and Research of Estonia

• Challenges: Estonian electronic ID-card infrastructure on the Grid (700 000

valid electronic ID-cards issued in Estonia!)

Local experiences with E-money and rental software

Interoperability of ARC and UNICORE

27

DCGC

Danish Center for Grid Computing

• Danish Center for Grid Computing, established in August 2003 http://www.dcgc.dk

• 3 years project, aiming at Provide Grid-access to test facilities (including HPC) Host development activities Provide user support

• Aims at getting industrial partners and users involved

• Uses ARC for middleware

28

Finland: Material Sciences Grid (M-grid)

• First large initiative to put Grid middleware into production use in Finland:http://www.csc.fi/proj/mgrid/

• Based on ARC and Linux clusters, currently 443 CPUs, targeted for serial and ”pleasantly parallel” applications, clusters can be accessed both locally and via ARC

• Joint project between seven Finnish universities, Helsinki Institute of Physics and CSC founded by the universities and the Academy of Sciences

• Users mainly from the physics and chemistry departments in the partner universities Material physicists, particle physicists, chemists,

some bioscientists Typical applications: Gromacs, Gaussian, Dalton

29

NORGRID

• Norwegian project for Grid competence building, January – December 2004 http://norgrid.uio.no ca. 3 FTE Partners : NTNU, UiB, UiO, UiT, UNINETT Funding 50% NFR and 50% partners

• Objectives: competence building on Grid middleware and related

technologies in Norway Prepare a middleware infrastructure for the next HPC

project (starting 2005) Emphasis on distributed data management,

metascheduling, portals• Presently uses ARC

Aims to evaluate it against UNICORE, GT4.x and LCG2/gLite

30

NDGF

• Nordic Data Grid Facility pilot project was launched in spring 2003 Initial success of NorduGrid provided grounds for a Nordic Grid

facility http://www.ndgf.org Funds for 1 director + 4 postdocs in each country Strong emphasis towards portal development and storage facilities Aimed to evaluate various Grid solutions, uses ARC

• Will produce recommendations for the Nordic Grid facility Aims to harness all the resources in the Nordic countries Grid of Grids with a large centralized storage facility This facility is expected to become the Nordic Tier1 candidate

31

NGN

• Nordic Grid Neighborhood is a networking project funded by the Nordplus program, started in September 2004 http://www.nicpb.ee/NordicGrid/

• Expands to the Baltic states and North-West Russia (St.Petersburg) 20 partners supports and strengthens contacts in the field of Grid

technologies activities cover education, reciprocal knowledge transfer

and Grid research and development• Plans to set up a testbed to deploy and

demonstrate ARC and AliEn The scope is to attract and educate users that can

benefit from Grid, e.g., medical applications

32

Summary

• With the creation of SweGrid Sweden made the first dedicated Scandinavian investment in a Grid computing Infrastructure. The SweGrid model has been followed by other Scandinavian countries

• SweGrid offers a production quality grid infrastructure since the beginning of 2004

• SweGrid is part of the Swedish National Infrastructure for Computing and being heavily used by a multidisciplinary user base

• Through the SweGrid resources Sweden joins to major world-class Grids: NorduGrid, EGEE

• STAC (Swedish Technical Advisory Committee) is determined that the SweGrid project will have a continuation