Upload
sara-henry
View
220
Download
0
Embed Size (px)
DESCRIPTION
Enabling Grids for E-sciencE INFSO-RI Athena framework In practice framework is a skeleton of the application into which developers plug in their code and provides most of the common functionality. Athena - Atlas software framework based on GAUDI. Athena provides common services such as the transient data store, (interactive) job configuration and auditing, data(-base) access, message streams, etc. for Atlas software. The idea is to improve the coherency of the different software domains within Atlas, and thereby the ease of use for end-users and developers, by having them all use the same well-defined interfaces.
Citation preview
INFSO-RI-508833
Enabling Grids for E-sciencE
www.eu-egee.org
Using of GANGA interface for Athena applicationsA. Zalite / PNPI
Enabling Grids for E-sciencE
INFSO-RI-508833
Overview
• General introduction to ATLAS software framework - Athena: structure, applications, user interface.
• Introduction to GANGA.
• Running Athena using Ganga interface.
Enabling Grids for E-sciencE
INFSO-RI-508833
Athena framework
• In practice framework is a skeleton of the application into which developers plug in their code and provides most of the common functionality.
• Athena - Atlas software framework based on GAUDI. Athena provides common services such as the transient data store, (interactive) job configuration and auditing, data(-base) access, message streams, etc. for Atlas software. The idea is to improve the coherency of the different software domains within Atlas, and thereby the ease of use for end-users and developers, by having them all use the same well-defined interfaces.
Enabling Grids for E-sciencE
INFSO-RI-508833
Athena framework
• The overall principle of the Athena framework is separation of algorithm and data.
• Each algorithm implements 3 methods for invocation by framework: an initialize() function which is executed at the start of the job, an execute() function which is executed every event and a finalize() function which is executed at the end of the job.
• An algorithm has a number of control and data parameters which allow run-time configuration. They can be set in the startup jobOptions files.
• The idea is you can write data to the Transient Store (TS) from one algorithm. You can then read in this data in another algorithm and do some processeing on that data and then write resulting data to the TS and so forth.
Enabling Grids for E-sciencE
INFSO-RI-508833
CVS, CMT
• All ATLAS offline code is stored in Concurrent Versions System (CVS), which manages the evolution of the source files.
• The build of the binaries (compilation option, include files, libraries…) as well as the run-time environment are managed by Configuration Management Tool (CMT).
• The software is grouped into separate packages, where some packages may be related or "depend" upon each other.
Enabling Grids for E-sciencE
INFSO-RI-508833
Running Athena
• First step is setup of Atlas software environment (controlled by CMT)
• > athena By default, it looks for jobOptions.py
• > athena myJobOptions.py Athena will use myJobOptions.py for input
configuration
Enabling Grids for E-sciencE
INFSO-RI-508833
Ganga
• Ganga (Gaudi / Athena and Grid Alliance) is an interface to the Grid that is being developed jointly by Atlas and LHCb.
• It is optimised for the configuration and management of jobs that use the Gaudi / Athena Framework which is common to the both of these experiments.
Enabling Grids for E-sciencE
INFSO-RI-508833
Ganga
• Ganga aims to allow a user to run a job at any site where the user has authorisation, and in particular at remote sites to be accessed through the Grid. To this end, Ganga creates a site-independent job representation, based on information supplied by the user, and takes care of any file transfers necessary between the user's machine and the remote site.
• In the current Ganga model, each job entails running a single executable. This may correspond to a single binary, for example to perform event reconstruction, or may be a script that runs in sequence any number of binaries, for example event generation followed by detector simulation followed by event reconstruction.
Enabling Grids for E-sciencE
INFSO-RI-508833
Ganga
• To run from account on lxplus, or over afs, perform setup with: > source /afs/cern.ch/sw/ganga/install/etc/setup.sh
• After setup, the Ganga Graphical User Interface (GUI) is started by giving the command:
> ganga
Enabling Grids for E-sciencE
INFSO-RI-508833
Ganga GUI
Enabling Grids for E-sciencE
INFSO-RI-508833
Ganga GUI
• The Ganga Graphical User Interface (GUI), as shown at startup, has the following features: – Panel for browsing job and directory trees– Drop-down menus and toolbar – Panel for job configuration and monitoring– Embedded python shell
Enabling Grids for E-sciencE
INFSO-RI-508833
Ganga GUI (Job tree)
Enabling Grids for E-sciencE
INFSO-RI-508833
Ganga
• To run from account on lxplus, or over afs, perform setup with: > source /afs/cern.ch/sw/ganga/install/etc/setup.sh
• After setup, the Ganga Graphical User Interface (GUI) is started by giving the command:
> ganga
Enabling Grids for E-sciencE
INFSO-RI-508833
Ganga GUI (File tree)
Enabling Grids for E-sciencE
INFSO-RI-508833
Ganga: examples
Let’s try to perform practical examples
• Run /bin/hostname• Run /bin/hostname with output to the file
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job Creation
• The job-creation window
Batch system to submit: Local, LSF, EDG/LCG, PBS, Glite, DIRAC.Applications are divided into two classes: Generic and known applications, for example Athena.
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Submitting
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Submitting
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Submitting
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: outputs
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: outputs
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: outputs
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Submitting
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: output
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: output
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: NewJob3.output
Enabling Grids for E-sciencE
INFSO-RI-508833
Running Athena• Ganga uses Athena application handler to run Athena
within the Ganga framework.• The handler benefits from Ganga's job submission and
job monitoring capabilities• The handler has the following features:
– AthASK is used to hide away Atlas software configuration complexities. The user simply provides AthASK commands needed to run Athena job from within the GUI.
– User packages (i.e. binary builds, source code) can be specified and these directory trees will be compressed and transported (together with the rest of the job) to the worker node ready to be used by the submitted job.
– Integrated with AMI browser. Job option files can be updated with selected files from AMI using POOL.
– Integrated with JOE. Job option files can be edited using JOE before submission.
Enabling Grids for E-sciencE
INFSO-RI-508833
Athena Startup Kit (ASK)
• The Athena Startup Kit (ASK) integrates the ATHENA framework, and the software based on it, with the official releases and the various supporting tools in order to provide an easier, more consistent, and more robust user interface to Atlas offline software.
• ASK operates completely in user-space: it performs only those tasks that are otherwise left to the end-user by the "standard" Atlas tools.
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job for Athena
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job for Athena
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job for Athena
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job for Athena
•In case of local instalation
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job for Athena
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Submitting
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Run
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: output
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: output
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: output
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: output
Enabling Grids for E-sciencE
INFSO-RI-508833
LHCb prototype: GANGA
• Framework for job creating-submitting-monitoring
• Can be used with different physical applications and different submission back-ends
• Was also chosen as a prototype component by the Atlas experiment
GANGAGUI
JobOptionsAlgorithms
Collective&
ResourceGrid
Services
Submitting jobsMonitoring
Retrieving results
GAUDI Program
GAUDI – LHCb analysis framework
ExperimentBook-keeping
DB
Gaudi/Athena aNd Grid Alliance