H1OO – the analysis framework for H1 · 4-6-2007 judith katzy - DESY DV Seminar Project history ν HERA operation and H1 data taking since 1992 ν 35 TB of raw data, 3.5 TB reconstruction

H1OO – the analysisframework for H1

Judith Katzy (DESY)

4-6-2007 judith katzy - DESY DV Seminar

Project historyν HERA operation and H1 data taking since 1992ν 35 TB of raw data, 3.5 TB reconstruction output collected until

2000 ( HERA I )ν Original reconstruction and analysis in FORTRAN and

FORTRAN based packages BOS, look, fpack, hbook, PAW,…ν 2000-02 Upgrade of accelerator and detector for high luminosity

runningν High data rates and volumes expected

¬ Adjust analysis software to needs of largedata sets and high rates

¬ Lower turn-around time for physics analysis


H1OO - Goals and Requirements

ν Improved analysis code to facilitate re-use of software:ν Unification and modularization of softwareν Centralization of expert knowledge on physics algorithms

ν Common particle and energy flow concept:ν Unified physics particle definition used by all analysis across

all physics working groupsν Centralized calibration

ν Improved data access:ν Simplify access to correlated dataν Fast event selection


Development constraints

ν Backward compatibility to existing data formatsrequired

ν Easy to use (competition to existing codes!)ν Quick turn around time of software


H1OO

ν An object oriented framework for HEP analysisν Written in C++ and based on ROOTν One single framework for user analysis and standardized

physics algorithmsν Easy feed-back of new developments into official framework

ν Framework comprised off:ν Persistent storage of particles found by physics algorithmsν Event tagging schemeν Root i/o extensionsν Database utilities for run dependent quantitiesν Common set of physics algorithms rewritten and enhancedν Event display


H1OO architectureH1Steering

H1Skeleton

H1Pointers

H1Geom

H1Clusters H1Tracks

H1Benchmark

H1HatH1ModsH1Ods

H1Filler H1Calculator H1PartEmFinder

IO: root extensions BOS formats

Base library (IO, event loop)

Persistent Data ObjectsandEvent Container

Geometry & RunDependence

Physics Algorithms H1Finder

H1OOBanks

ROOT


Code organization

ν 500 classes split into ~50 packages stored in CVSν Each package compiles into a shared library

¬ Low compilation time♣No circular dependencies between core packages

allowed♣ Fortran libraries dynamically loaded on demand♣Core packages are standalone H1OO/ROOT

->enough for most analysis


H1oo environmentν Linux directory oriented

ν no absolute pathnamesν Working with different releases in parallel

supportedν Script to set-up full working environment with single

command:H1oo –n 3.0.12


H1OO data layers

Raw data(POT)

Reconstructionoutput(DST)

• H1Tree links parallel ROOT and BOS/fpack files¬ DST file location not needed as input¬ one event loop

Reconstruction(FORTRAN)

Conversion

H1OOPhysicsalgorithms

ROOT/H1Tree

User code

Physics particles: µODS3 kB/event

H1 Analysis Tag: HAT0.4 kB/event

User Tree

Reconstr. output: ODS13kB/event

BOS/fpack


Object Data Storeν Identical content as DSTν Contains H1OO bank, H1Track, H1Cluster objectsν One class for each BOS bank on DST auto-

generated for persistency and with accessorfunctions

ν H1Cluster and H1Track objects filled from BOSbanks

ν Generic reading of BOS banks possible


H1OO physics algorithmsν Particle finders operating on reconstruction

output (dst/ods)ν Completely rewritten using best H1 knowledgeν Continuously updated and extendedν Up-to-date best calibration applied to 4 vectors

ν All aspects of physics are covered:ν Heavy flavor, inclusive, searches, hadronic final

state,…ν Background finders, event kinematics, lumi

calculation, neuronal network,…


µODS – the physics analysis layer

Identified particles:4 VectorSpecific characterizing informationPointer to its particle candidateComposed particles:4 VectorPointer to the identified particles it is composed ofParticle candidates:Track, cluster or track-cluster-link associated with one or more identified particles4 Vector of the most likely particle hypothesis-> final state energy correspond to the sum of the particle candidate energiesPointers to the tracks and clusters on ODSPointers to the associated physics particles

Track …Cluster…

Particle candidates

Electrons

Muons

J/Psi

D*

Jets Sel.Tracks

ODS µODS


Event tagging - HATν List of ~300 basic type variables for event selection in

physics analysis and detector studies (eventkinematics, vertex, hv setting, trigger info,…)

ν Stored as ROOT ntuple

ν Hat selection 1.2 sec / 106 events(H1 total: ~ 1.1 109 events)

ν Also used as storage for basic type event variables


I/O extensions in H1OOν Root I/O with H1OO extension to make pointer

relations persistent¬ Partial event reading within one and across different

data layers possible

♣Encapsulated ROOT Branch access¬ Simple user interface to access H1OO objects: H1PartEmArrayPtr electrons;

electrons[0]->GetEnergy();

¬ Pointers for each ROOT branch auto-generated


Conditions data base handling

H1Tree synchronizes event data and run dependent db access transparent to the user

Raw data(POT)

Reconstructionoutput(DST)

Reconstruction(FORTRAN)

Conversion

ROOT/H1Tree

Physics particles: µODS3 kB/event

H1 Analysis Tag: HAT0.4 kB/event

User Tree

Reconstr. output: ODS13kB/event

BOS/fpack

Oracle data baseRun based quantities DB snapshot (ROOT)

33 MBConversion ofrelevant quantities


Data productionν Make best calibration and new particles persistent

with fast turn around time

ν 2-3 times per year mods + hat of full data setproducedν Incoming data are produced onlineν Web-interface for automatic mc job submission by users

ν Benchmarks:ν Production of µODS and HAT needs 0.05 sec/eventν ~ 4 weeks of processing for complete data setν Complete data set of µODS and HAT 1.7TB

ν For comparison: RAW data: ~200TB, DST 15 TB


Event sub-samples

ν Event selection based on HAT variables directly withH1Tree:ν HAT variables are of basic type, selectable quantities such as

event kinematics, number of particles of a given type, detectorstatus,…

ν List of selected events storable as H1EventList:ν Allows to make event selection based on any data layer

persistentν Stores name and location of associated filesν Keeps selection string (if any)

ν H1Eventlist of special data taking periods provided

=> Allows sub-samples of events without duplication ofRoot files on disk


Add-on packages to H1OOν Steering using CINT for parsing:

ν Used to steer file i/o, physics algorithms, etc.ν Same scripting language for steering as in interactive ROOT and

as compiled code

ν Framework for data quality checks of incoming data:ν Simple ntuple of relevant quantitiesν Additional pointers to raw dataν Web interface to fill and display the relevant histogram


New event display – motivation

ν Original event display based on LOOK graphicssystem from 1990 with command line input fordisplay of selected BOS banks

ν Visual checks of reconstruction and physicsalgorithms crucial – new event display needed tovisualise new data layers

ν Modern graphics and GUIs allows for easyimplementation of useful features (click and inspect)


New event display – implementation

ν Add-on program to h1oo based on ROOT GUIsν Easy integration in physics analysis chain or event selection

ν Visualisation of h1 data and their structureν Displays information of all or only selected data layers

(even to raw data!)ν Allows navigation between layers

ν Complete backward compatibility with the previous event displayh1redν input of h1red commands still possibleν significant increase of code complexity


event display featuresν Various views in 2D and 3Dν Selection options for display:

ν sub-detectorsν single or several data layers (even raw data!)ν Specific identified particlesν Cut on pt of tracks or particles (interactively applicable)ν η range of tracks and particles (interactively applicable)

ν Access to detailed particle information by inspectingsingle objects:ν E.g. electron: 4 vector, cluster radius, linked track,..

ν Color coded display of identified particles


µODS:Color coded identified particlesInspection of full detailed information of single particle by click

ODS clusters


All particle candidates of energy flow measurement


Project organization

ν Core team of 4 (6) personsν Strong support to feed-back users code into frameworkν Act as governance body for QA and future directions

ν Special workshops to unify physics algorithm across physicsworking groups

ν User support taken seriously:ν H1OO User Guide and Tutorialsν Vivid information exchange in special HyperNews forums


Development & release scheme♣Multi-developer project:

♣ Tight code quality checks done by official librarians (one for eachpackage)

♣ Coding conventions♣ Standardized error logging♣ Special release validation package for QA

♣Releases with tagged package versions every 2weeks

♣Production (= “pro”) release 2-3 times per year♣Continuous updates to the latest root version♣Supported on Linux (DL5, SL4)


SummaryH1OO successfully established as the standard tool for final

physics analysis in H1 with• Improved code and centralized physics algorithms Additional

data layers for physics analysis and fast event selection• Capability of navigation between different data layers and

partial event reading

H1OO used for almost all physics results presented in recentconferences and papers

H1oo suits needs for analysis of full HERA data set by providinggood performance and high precision


H1 data volumesν ~600 pb-1 data useable for physics collected

ν (Low and medium energy run total of 20 pb-1 expected)

Data/TB 192TB raw 15TB reco

1.7 TB mods+hat

Data taking year

Higher cross section – more events!


µODS event model

TObject

H1Part

H1PartCand H1PartComp H1PartId H1PartMc

Jet K0 D* µ e- …

ROOT I/O

Abstract base class for Streaming and 4 vector interfaceAbstract base class for Different particle types

Implementation of physicsParticles with 4 vectors and Detailed information

Documents

H1OO – the analysis framework for H1 · 4-6-2007 judith katzy - DESY DV Seminar Project history ν HERA operation and H1 data taking since 1992 ν 35 TB of raw data, 3.5 TB reconstruction