Grids in CYFRONET

Cracow Grid Workshop, November 20-23, 2005

Grids in CYFRONET

Marian BubakZofia Mosurska

Michał Turała

CYFRONET Academic Computer Centre of the AGH University of Science and Technology


Very Short History

• Established in 1973 as an independent, non-profit organization.

• In 1999 it became a separate unit of the University of Science and Technology in Krakow.

• Main goal is to provide the universities and research institutions with:– computing power combined with a wide range of scientific

software– communication infrastructure and network services

• CYFRONET is also engaged in numerous important projects, both domestic and international.

• Strong collaboration with – Institute of Computer Science AGH– Institute of Nuclear Physics

*


10 Gb/s PIONIER

10 Gb/s GEANT

MAN

Computer Resources

s g i

user servermail serverWWW server

ftp servernews serverWWW server

SGI Onyx300SGI 2800 Claster PC SunFire 6800

ATL 2640 Sun Store

HP RP7410

HP N4000HP Superdome

klastersieciowy

6*IBM X345


INSTALLED COMPUTER POWER - Rpeak

WORLD

EUROPE

ACC CYFRONET


• 70 km of fiber-optic links• Spans the entire metropolitan area of

Cracow• Links universities, research centers,

government institutions, • 155Mbps ATM backbone (expandable

to 622Mbps)• Linked to POL-34 (622 Mbps)

Metropolitan Area Network of the City of Krakow

CYFRONET serves as the operator of the Cracow academic MAN, providing connectivity and services to academic institutions dispersed throughout the city via a fiber-optic network several dozen kilometers in length.

The Centre functions as the main network node in southern Poland.


Conferences organized by Cyfronet

The 6th Annual European Convex Users Conference (18-21 X 1994)

Infofestival ’96 „Informatyka dla Nauki” (12-15 XI 1996)

High Performance Computing in Europe on IBM Platforms

(8-11 IX 1996)

The 8th Joint EPS-APS International Conference on Physics Computing

(17-21 IX 1996)

High Performace Computing on Hewlett-Packard Systems

(5-8 XI 1997)

SGI Users’ Conference

(11-14 X 2000)


Conferences organized by Cyfronet (contd.)

3rd IFIP Conference on Distributed Applicatios and Interoperable Systems

(17-19 X 2001)

The 1st Cracow Grid Workshop

5-6 XI 2001

The 1st EU CrossGrid Conference

(18-20 III 2002)

The 2nd Cracow Grid Workshop

11-14 XII 2002

Seminar: Sun Application Tuning

18-20 II 2003

Perimeter, Server and Desktop Security Solutions of WatchGuard Technologies(2 IV 2003)


International Conferenceon Computational Science7 – 9 VI 2004

The 3nd Cracow Grid Workshop

27 – 29 XI 2003

E-health in Common Europe

5-6 VI 2003

Conferences organized by Cyfronet (contd.)

The 7th Conference on Software Engineering (KKIO)18-21 X 2005

E-health in Common Europe

17-18 XII 2004

The 4th Cracow Grid Workshp

13-15 XII 2004

The 5th Cracow Grid Workshp

20-23 XI 2005


SEMINARS

They are held weekly or bi-weekly, mainly for CYFRONET’s Users • Supercomputing and Networking (for many years)

• Cracow Grid Seminar (since 2002)


MAN

The Cracow Telemedicine Centre (where CYFRONET is one of the principal partners) has performed research in different telemedicine fields for years, helping many hospitals and health centres secure better medical services. The Centre has had many achievements in the development and implementation of IT solutions for different projects sponsored by the Polish Ministry of Science, as well as by EU 5FP.

PIONIER: Polish Optical Internet - Advanced Applications, Services and Technologies for Information Society. PIONIER program has triggered many supporting activities, (such as e-Polska, Wrota Polski, etc.) aiming at building the base mechanisms of information society and providing Poland for equal partnership opportunities with other countries.

National Projects


MAN

High Performance Computing and Visualization with the SGI Grid for Virtual Laboratory Applications - the SGI Grid project aims to design and implement:•state-of-the-art, broadband services for remote

access to expensive laboratory equipment•backup computational center•remote data-visualization serviceThese services will be based on the national HPC infrastructure and advanced visualization.

PROGRESS stands for Polish Research On GRid Environment for Sun Servers. It started as a grant co-financed by the Polish State Committee for Scientifc Research and Sun Microsystems Poland. The grant involved three research institutions: Poznan Supercomputing and Networking Center, Academic Computer Centre Cyfronet AGH and Technical University of Lodz, and lasted since December 2001 until December 2003.

National Projects (contd.)


MAN CLUSTERIX - this project contains the concept of building the National Cluster of Linux Systems, a distributed PC-cluster (or metacluster) of a new generation. Its implementation makes it possible to deploy a production-class Grid environment, which consists of local PC-clusters with 64- and 32-bit architectures, located in geographically distant independent centers. The management software (middleware) being developed will allow dynamic changes in the hardware configuration. This project is being implemented by 12 Polish supercomputing centers and metropolitan area networks (MANs), with the Technical University of Czestochowa as the project coordinator.

National Projects (contd.)


International Projects

with the financial support of the European Commission

• 5th Framework Program– CYFRONET involved into various 5

projects– Already finished

• 6th Framework Program– Cyfronet participates in 3 Grid projects– About 10 new proposals have been

submitted in September 2005 to the EC


Successful CrossGrid Final Review. The EU IST CrossGrid Project IST-2001-3224 had its Final Review on February 16-17, 2005 in Amsterdam. The results of the Project were demonstrated at the European Grid Conference in a series of technical talks and live demonstrations of applications, software services and tools, deployed on the CrossGrid testbeds. More the results were appreciated by the audience, by the EU Project Officer and by the Reviewers; all internal deliverables and presentations concerning technical, managerial and exploitation issues were approved. The Final CrossGrid Review concluded successfuly. About 200 persons from 21 institutions, representing 11 countries, contributed to the Project. CYFRONET was a Coordinator of the project.

Research Projects in the 5th Framework Program


Research Projects CrossGrid Architecture

T e s t b e d

Applications

Services

Tools

• 17 sites• 9 countries

• over 200 CPUs• 4 TB of storage

RoamingAccess Server

MigratingDesktop

OCM-G

PerformancePrediction

Data Access Globus

Toolkit

MPI Verification

MPI Library

Portal

Post-processing

InfrastructureMonitoring

Plugin

Plugin

Plugin

SOAP

SOAP

SOAP

SOAP

SOAP

Protocol

API

API

API

API

Links

API

API API

SOAP

SOAP

API

API(JMX)

(OMIS)

VisualizationKernel

Links

PerformanceAnalysis

API

ApplicationMonitoring

Benchmarks

NetworkMonitoring

MedicalSupport

ParticlePhysics

Meteo/Pollution

FloodSimulatio

n

DataGrid

Scheduler


CrossGrid

•21 partners•2002-2005•Research areas

• CrossGrid Applications• Grid Tool Environment • New Grid Services• International Testbed • Architecture

www.eu-crossgrid.orgwww.eu-crossgrid.org


PRO_ACCESS - The ImPROving ACCESS of Associated States To Advanced Concepts In Medical Informatics (PRO-ACCESS) project focuses on creating a platform for promotion, dissemination and transfer of advanced health telematics concepts and experiences from development and deployment of telemedicine solutions to NAS.

PELLUCID - A Platform for Organizationally Mobile Public Employees. The project is coordinated by the Council for the Central Laboratory of the Research Councils, RAL, UK, with two academic partners: CYFRONET and the Institute of Computer Science AGH.

GRIDSTART - this project clusters all of the 5FP IST-funded Grid research projects with the intention to stimulate wide deployment of appropriate technologies and to support early adoption of best practices.

5th Framework Program (cont.)

6-WINIT - an IPv6 wireless Internet initiative. The project will validate the introduction of the new Mobile Wireless Internet in Europe.


CoreGRID - the CoreGRID Network of Excellence (NoE) aims at strengthening and advancing scientific and technological excellence in the area of Grid and Peer-to-Peer technologies.

Research projects in the 6th Framework Program

K-Wf Grid - Knowledge-based Workflow System for Grid Applications - addresses the need for a better infrastructure for the future Grid environment. The Grid as a vast space of partially-cooperating, partially-competing Grid services will be a very dynamic and complex environment. In order to address the complexity in using and controlling the next generation Grid, the consortium will adopt the approaches envisioned by semantic Web and Grid communities in a novel, generic infrastructure. The K-Wf Grid system will assist its users in composing powerful Grid workflows by means of a rule-based expert system.


K-WfGridWorkflow Applications and Knowledge

•Integrating services into coherent application scenarios

•Enabling automatic construction and reuse of workflows with knowledge gathered during operation

•Involving monitoring and knowledge acquisition services in order to provide added value for end usersTechnologiesTechnologies: service-oriented Grid architecture,: service-oriented Grid architecture, software agents, ontologies, dynamicsoftware agents, ontologies, dynamic instrumentationinstrumentation..

Execute workflow

Capture knowledge

Reuse knowledge

K-WfGrid

Monitor environment

Analyze information

Construct workflow


User

Workflow Orchestration and Execution

Web Portal

Grid Resources

Scheduler

Grid Performance Monitoringand Instrumentation Service

Automatic Application Builder

Workflow Composition Tool

KnowledgeAssimilation Agent

Grid OrganizationalMemory

Ontological store of

knowledge

Low Level Grid Middleware (WS-RF)

Knowledge

Grid Workflow Execution Service

PerformanceAnalysis

User Assistant Agent

Grid Workflow User Interface

User interactionthrough the Portal

Guidancesfor the user

Workflow composition and execution visualizationUser’s decisions in crucial points of execution

Information onavailable

resources andtheir description

Execution of chosenGrid services

Informationabout workflow

execution

Analysed andextractedknowledge

Information aboutresources andenvironment

Information aboutperformance of

particular resources

K-WfGrid - Architecture and Flow of Actions


• Fraunhofer FIRST, Berlin, Germany• Institute of Computer Science,

University of Innsbruck, Innsbruck, Austria• Institute of Informatics of the Slovak

Academy of Sciences, Bratislava, Slovakia

• ACC CYFRONET AGH, Kraków, Poland• LogicDIS S.A., Athens, Greece• Softeco Sismat SpA, Genova, Italy

Bratislava

Athens

Genova

KrakówInnsbruck

Berlin

K-WfGrid - Partners

www.kwfgrid.net


EGEE - Enabling Grids for e-Science in Europe - the most prestigious of all 6FP Projects, aims to integrate current national, regional and thematic Grid efforts in order to create a seamless European Grid infrastructure for the support of the European Research Area (ERA). The EGEE project is coordinated by CERN.

CYFRONET, together with ICM (Warsaw) and PSNC (Poznan), form a Regional Operations Centre for the Central European Grid Consortium, comprising Austria, the Czech Republic, Hungary, Poland, Slovakia and Slovenia.

6th Framework Program (contd.)

EGEE Regional Operations Center (ROC) for Central European (CE) Federation is one of the several centers which comprise the European Grid Support, Operation and Management activity of EGEE. This activity's role is to create, operate, support and manage a production quality European grid infrastructure which will provide computing, storage, instrumentation and informational resources at many resource centres (RCs) across Europe.


ViroLab - Project Objectives

The long term mission of ViroLab is to provide researchers and medical doctors in Europe with a virtual laboratory for infectious diseases.

1. To develop a VO that provides the “glue” for binding the various components of our virtual laboratory

2. To develop a virtual laboratory infrastructure for transparent workflow, data access, experimental execution and collaboration

3. To virtualize and enhance state-of-the-art in genotypic resistance interpretation tools and integrate them into the virtual laboratory

4. To establish epidemiological validation to correctly and quantitatively predict virological and immunological outcomes and disseminate the results to European stakeholders


ViroLab - Consortium• Universiteit van

Amsterdam

• University Medical Centre Utrecht

• High Performance Computing Center Stuttgart

• CYFRONET AGH

• Gridwise Technologies

• Institute de Recerca de la SIDA

• Catholic University Leuven

• University College London

• Catholic University Rome

• Eötvös Loránd University

• Institute of Infectious and Tropical Diseases, University of Brescia

• Virology Education B.V.


grid resources

grid middleware

GLOBUS, EDG, CrossGrid, other

VO

man

agem

e nt ,

AA

A,

Dat

a A

c ce s

s &

Lo

gg

ing

fo

r P

rove

na n

c e

application

RuleEngines

Rule-basedSystem

Rule Derivers

Rule Elucidators

BiostatApps

Scientific Tools

StochasticModeling

Apps CAs VIZ SIM

Runtime System

Session Manager

Clinical PortalDriver

WebBrowser

PDADriver

PDAClient

presentation

Collaboration

Data Access(OGSA-

DAI)

data resources

Medical Apparatus

Measured Data

Patient Cohort

DB

Immunology DB

Medical Knowledge Data

Scientific Literature

Stanford Virology Lab DB

Pharmacology DB

Derived Rules DB Intermediate

Experimental Data

PFT DB

computationalresources

hospital CE, WN, SE

Ind. Patient

Data

Exp. Apparatus

ViroLab - Component Perspective

vir

tua

l o

rga

niz

ati

on


http://www.cyfronet.pl

Documents

Grids in CYFRONET