DataTAG WP4 Meeting CNAF Jan 14, 2003 Interfacing AliEn and EDG
1/13 Stefano Bagnasco, INFN Torino Interfacing AliEn to EDG Stefano
Bagnasco, INFN Torino Pablo Saiz, CERN Slide 2 DataTAG WP4 Meeting
CNAF Jan 14, 2003 Interfacing AliEn and EDG 2/13 Stefano Bagnasco,
INFN Torino Server AliEn Site AliEn CE AliEn SE Data Catalogue WNs
File registration Any data produced is stored on the local SE, and
files are registered in the Data Catalogue Job submission The CE
pulls a job from the server and sends it to a WN Status report The
job, once on the WN, reports to the server, identifying and
authenticating itself via a ProcID/JobToken pair AliEn: the works
Slide 3 DataTAG WP4 Meeting CNAF Jan 14, 2003 Interfacing AliEn and
EDG 3/13 Stefano Bagnasco, INFN Torino Server Data Catalogue
Interface Site AliEn CE AliEn SE An interface site is thus an EDG
User Interface machine which runs the AliEn client suite -
ClusterMonitor, CE and SE grid006.to.infn.it (EDG 1.2.3)
testbed010.cern.ch (EDG 1.4.0) grid010.to.infn.it (EDG 1.4.2) EDG
UI The philosophy: The whole Grid (namely, a Resource Broker) is
seen by the server as a single AliEn CE, and the whole Grid storage
is seen as a single, large AliEn SE. Exploiting DataGRID Slide 4
DataTAG WP4 Meeting CNAF Jan 14, 2003 Interfacing AliEn and EDG
4/13 Stefano Bagnasco, INFN Torino Server Interface Site AliEn CE
EDG UI EDG RB Job Submission 1 Job submission The interface client
pulls a job from the server; The Resource Broker interface is
implemented as a Local Queue object (like PBS, for example) A new
JDL file is generated, adding: - requirements for AliEn runtime
environment - requirement for outgoing connectivity - environment
variables containing the ProcID/JobToken pair SpecialRequirements
ClassAd attribute is used to pass requirements not meaningful to
AliEn The job is submitted to the Resource Broker AliEn SE Slide 5
DataTAG WP4 Meeting CNAF Jan 14, 2003 Interfacing AliEn and EDG
5/13 Stefano Bagnasco, INFN Torino Server Interface Site AliEn CE
Job submission EDG UI EDG Site EDG CE EDG RB WN AliEn EDG SE Job
Submission 2 Status report Job submission A DB of EDG/AliEn JobIDs
is kept by the interface CE to keep track of jobs that did not
report yet to the server Problem tracking yet to come... AliEn SE
Slide 6 DataTAG WP4 Meeting CNAF Jan 14, 2003 Interfacing AliEn and
EDG 6/13 Stefano Bagnasco, INFN Torino Server Data catalogue
registration 1 Data Catalogue EDG RC An output file is generated by
an AliEN job running on some EDG site The CloseSE is determined by
AliEn code (running on the WN) from the.BrokerInfo file The file is
stored in the selected SE and registered in EDG Replica Catalogue
LFN uniqueness is enforced by AliEn EDG Site EDG CE WN AliEn EDG SE
LFN PFN Slide 7 DataTAG WP4 Meeting CNAF Jan 14, 2003 Interfacing
AliEn and EDG 7/13 Stefano Bagnasco, INFN Torino Server Data
catalogue registration 2 Data Catalogue EDG RC A intermediate file
name is generated using the EDG LFN and the VO name (to allow form
multiple VO/RC) The file is now registered in AliEn Data Catalogue
using this intermediate FN as PFN The file is registered in both
catalogues, thus ensuring accessibility from both worlds. EDG Site
EDG CE WN AliEn EDG SE EDG:// / Slide 8 DataTAG WP4 Meeting CNAF
Jan 14, 2003 Interfacing AliEn and EDG 8/13 Stefano Bagnasco, INFN
Torino Data access - basic case AliEn Job running on EDG site
accessing AliEn-generated data stored in EDG resources Two-step
process via the AliEn Data Catalogue and the EDG Replica Catalogue,
all file transfers are local to the site: Data Catalogue EDG Site
EDG CE EDG SE EDG RC AliEn LFN EDG:// / InputData requirements are
not yet passed automatically to the RB via the JDL. As a temporary
workaround, input file information has to be specified by hand
using the SpecialRequirements ClassAd attribute in the AliEn JDL.
WN AliEn PFN LFN Slide 9 DataTAG WP4 Meeting CNAF Jan 14, 2003
Interfacing AliEn and EDG 9/13 Stefano Bagnasco, INFN Torino Data
access - EDG case Non-AliEn job running on EDG accessing
AliEn-generated data on an EDG SE Files are registered in both
catalogues with consistent LFNs. The job can use directly the EDG
Replica Catalogue. Data Catalogue EDG RC EDG Site EDG CE WN EDG SE
Slide 10 DataTAG WP4 Meeting CNAF Jan 14, 2003 Interfacing AliEn
and EDG 10/13 Stefano Bagnasco, INFN Torino AliEn job running on
EDG accessing data on an AliEn SE The requirement for OutgoingIP
ensures AliEn Data Catalogue from the EDG WN. Data Catalogue Data
access - exotic cases AliEn Site AliEn Client AliEn CE Data
Catalogue EDG Site EDG CE EDG SE EDG RC Interface Site AliEn Client
AliEn SE EDG UI AliEn CE WN AliEn AliEn LFN AliEn PFN Slide 11
DataTAG WP4 Meeting CNAF Jan 14, 2003 Interfacing AliEn and EDG
11/13 Stefano Bagnasco, INFN Torino Job running on AliEn site
accessing AliEn-generated data stored in EDG resources Two-step
process via the AliEn Data Catalogue and the interface SE
(implementing an MSS object, like Castor, to interface to EDG):
AliEn Site AliEn Client AliEn CE Data Catalogue EDG Site EDG CE EDG
SE EDG RC Interface Site AliEn Client AliEn SE EDG UI AliEn CE WN
Data access - exotic cases Slide 12 DataTAG WP4 Meeting CNAF Jan
14, 2003 Interfacing AliEn and EDG 12/13 Stefano Bagnasco, INFN
Torino Open issues Error and problem tracking not yet complete
InputData requirements are not yes passed automatically to the RB
(because of the intermediate LFN) The interface site may be a
bottleneck for cross- world jobs (i.e. jobs on EDG needing data
from AliEn and vice-versa) Much testing still to be done Slide 13
DataTAG WP4 Meeting CNAF Jan 14, 2003 Interfacing AliEn and EDG
13/13 Stefano Bagnasco, INFN Torino Conclusions System is in place,
but... Full functionality still to be tested - The work just
started, until now only tested separated pieces A couple more
issues to address Many small (even trivial) problems: minor bugs,
EDG releases, different RC schemas, not all EDG tools correctly
return error value, AliEn installation in EDG sites...
