Instruments and Sensors in Grid – ICTP Trieste – 23 April 2007 1 Instruments in Grid: the New Instrument Element C. Vuerli (1,2), G. Taffoni (1,2), I

Instruments and Sensors in Grid – ICTP Trieste – 23 April 2007 1

Instruments in Grid:the New Instrument Element

C. Vuerli (1,2), G. Taffoni (1,2), I. Coretti (1), F. Pasian (1,2),P. Santin (1), M. Pucillo (1)

(1) INAF – Astronomical Observatory of Trieste(2) INAF – Informative Systems Unit


The Goal

Gridification of Telescopes and attached astronomical instrumentation to allow remote monitoring and control in Quasi-Real Time Step 1. Remote monitoring of telemetry and

scientific data under acquisition Step 2. Remote control: the astronomer

interacts with night assistant by remotely sending commands and receiving feedback


The Origin: The GRID.IT Project

GRID.IT: National multi-disciplinary project to validate the Italian Grid infrastructure

INAF participates with 3 applications Access to Astronomical Databases via Grid Porting on Grid the VST Pipelines Monitoring/Control of astronomical

instrumentation


Astro DBs in Grid: drivers

“Customer whishes”: Transparent access to data sources (DBs); Transparent access to computing resources; Complex workflows;

IVOA standards for data access; GRID computing and storing resources;

Integration of the VObs with Grid technology is of key importance to

provide users with proper computing power



Our Grid

It is the Grid middleware that we used in the framework of GRID.IT Project

Set up by INFN (National Institute of Nuclear Physics) to run jobs of the LHC (Large Hadron Collider) experiments (CERN, Geneva)

LCG (LHC Computing Grid) based on Globus 2.4 native computing and data storage resources

LCG now migrated to gLite (WebServices enabled)

No native access to non-computational resourcesG-DSE: a low level integration framework

In Globus and not on Globus


Control systems in Grid

Grid TeleControl Protocol (GTCP) JavaServices + GT4 container

Xport (X-ray Cristallography) WebServices

GridCC Web Services

G-ICS: a low level integration framework on a production grid


Recycling work on G-DSE to enable G-ICS

Provide a suitable formal definition of a Grid Abstract Computing Machine using Formal specification language

Review the architectural definition of a Data Source Engine (DSE), to build an Instrument Control System

Provide ICS integrated in the existing Grid Resource Layer Framework (G-ICS)

Extend GIS capabilities to understand and handle monitoring and control requests to ICS

Extend GIS capabilities to describe ICS Resources Provide a new Manager for ICS integrated in Globus Extend semantic and provide it to RSL to use G-ICS Etc…


Query DB specific driver

G-DSE and G-ICS

gatekeeper

JobManger QueryManger

JobProcessQueryProcess

Scheduler p-in

Pbs/LSF

query plug-in

GRAM GIS

RDBMS

MDS

GRIS

Ldapldif

Grid Providers (snmp)

Instrument Control System

ICSManger

ICS plug-in

ICSProcess

ICS specific driver

ICS

RDBMS


LCG-2: DB + ICS enabled middleware

Computing cluster Network resources Data storage

Operating system Local schedulerFile system

User access SecurityData transferInformation schema

Resource Broker Data managementApp monitoring system

User interfaces Applications

Hardware

System software

“Basic” services

“Collective” services

Application level services

HPSS, CASTOR…HPSS, CASTOR…

Scientific LinuxScientific Linux NFS, …NFS, … PBS, Condor, LSF,…PBS, Condor, LSF,…

VDT (Condor, Globus, GLUE) + XMLDB + XMLICSVDT (Condor, Globus, GLUE) + XMLDB + XMLICS

EU DataGrid +Glite + G-DSE + G-ICSEU DataGrid +Glite + G-DSE + G-ICS

Information system

RDBMS

MySQL Ora, etc.MySQL Ora, etc.

ICS

Instruments


From G-DSE to QE

With the “G-DSE enabled” collective and resource Grid layers, the new QE Grid Element is introduced

QE is a CE able to manage Queries and to connect RDBMS resources via Grid

It speaks RSL to interact with Grid services, VOQL when exchanging data with RDBMS resources and processing query requests coming from UIs (Users)


From G-ICS to IE

With the “G-ICS enabled” collective and resource Grid layers, the new IE of the Grid is introduced

IE is a CE able to manage, monitor and control requests to interact with ICS resources via Grid

It speaks RSL to interact with Grid services, and ICSL (Instrument Control System Language) when connecting an ICS


From G-ICS to IE

What we need is: ICSP: a ICS Protocol to make IE and ICS

able to communicate each other ICSL: a ICS Language used by IE to speak

with ICS according to protocol ICSP ICSC: an Instrument Control System

Connector, being it a driver able to speak ICSL and understand ICSP-compliant

requests and map them in requests to the specific ICS sitting behind it

handle answers to these requests


From G-ICS to IE

ODBC client

QE

ODBCDriver

Oracle

Mysql

Postgres

Custom

Odbclang

ICS 2ICS 3ICS 4

ICS 1

ICSL

ICSC client

IE

ICSCDriver

Mapping the DSE schema into ICS: The ODBC Driver translates standard SQL in the

SQL (extensions included) of the specific DBMS to be contacted

Similarly, the ICSC Driver translates standard ICSL in the specific ICSL (extensions included) of the ICS to be contacted

Extensions retrieved from metadata repository via MDS


Grid Instrument Control System

Local Resource Manager

ICS Grid Access Enabled

GRAM Protocol

Gridinfo

system

MDS

LDAP

Job Manager

ICSC Manager

ICS Instance Man

ICSC DriverICS Instance

Worker Node

Meta Machine

GANGLIA

snmpd

snmpd

ICS

Synchro CS

Tel CS

Synchro

ODBC Manager

DSE Instance Man

ODBC DriverDSE Instance

DSE

Internal DB

User DB


Long Term Goal: Grid and VObs

UI

VO

QL

Astro

DB

Grid Information and Brokering

System

RSL(VOQL)QE

MetaD

ata

QE

VOQL

AstroDB

QE

VOQL

AstroDB

External IVOA Resource (Registry)

RSL(VOQL)

VOQL

RSL

RSL

VO

QL

Outside Grid (VObs)

Data

Data not available in Grid

Data found in Grid


Long Term Goal: Grid and VObs

UI

Grid Information and Brokering

System

Mon/Ctrl requestsQE

MetaD

ata

IE

ICSL

IE

ICSL

RSL(VOQL)

Data


RSL RSL

ICSC


ICSC


In Summary: What is the IE?

The IE inherits from the CE, so it has all the characteristics of a classic computing element

The IE, moreover, makes possible to Publish on the Grid any kind of scientific instrumentation.

Telescopes and related astronomical instrumentation are just an example of it

Share on the Grid infrastructure any kind of scientific instrumentation

Get access to any kind of scientific instrumentation shared on the Grid infrastructure

Perform remote monitoring sessions in which the Grid technology only is used

Perform remote control sessions in which the Grid technology only is used


Current Status

Institutes that collaborated in the framework of GRID.IT Project INAF Trieste

IE project coordination and development/test G-DSE development/test

INAF Padova G-DSE development/test

INAF Capodimonte (Naples) IE development/test

INFN-CNAF G-DSE development/test


Current Status

We are going on through subsequent refinement steps

For the time being monitoring only implemented

ICS not fully implemented on Grid yet


Current Status

Two test-bed ICS identified (at Trieste and Naples observatories)

Tests successfully carried out on LCG implementation of the Grid middleware (no G-ICS extension)

New project proposals and requests for funding under evaluation to carry out the implementation of the Instrument Element


Test-bed and the first prototype of UIF

First prototype of UIF written in Java

The Celestron C-14 located at theBasovizza branch station of INAF-OATs


The final goal: An Integrated System

Grid-UI

The GRID

Grid-GUI

IE

IE

QE

QE

DBMS

DBMS

CECE

SE

SE

IE

Synchrotron


End of Presentation

Thank you for your attention

Documents

Instruments and Sensors in Grid – ICTP Trieste – 23 April 2007 1 Instruments in Grid: the New Instrument Element C. Vuerli (1,2), G. Taffoni (1,2), I