Blueprint RTAGs1 Coherent Software Framework a Proposal LCG meeting CERN- 14 June Ren é Brun

Blueprint RTAGs 1

Coherent Software Frameworka Proposal

LCG meetingCERN- 14 June

René Brunftp://root.cern.ch/root/blueprint.ppt

LCG 14 June Rene Brun Blue print RTAG 2

Way 1: the component model

E

D

F

IH

JK

L

G

B C

A

...

PROs: In principle easy to add

or replace a component

because of weak coupling

In reality, you simply postpone

the integration problem if the

number of components is > N

eg N> 10

C’

J’


The main software areas

GRID

middleware

RDBMS

run/file

catalogs

Object

persistencyv

2-d, 3-d

graphics

GUI

Toolkits

Math Libs

Statistics

Detector

Geometry

Event

Generators

Dectector

Simulation

Object

persistency

Histograming

Fitting

Event Models

Folders

Event Display

Ntuple

analysis

Interpreters

DAQ

Online

System

services

ETC...


Any box connected to many boxes

GRID

middleware

RDBMS

run/file

catalogs

Object

persistencyv

2-d, 3-d

graphics

GUI

Toolkits

Math Libs

Statistics

Detector

Geometry

Event

Generators

Dectector

Simulation

Object

persistency

Histograming

Fitting

Event Models

Folders

Event Display

Ntuple

analysis

Interpreters

DAQ

Online

System

services



GRID

middleware

RDBMS

run/file

catalogs

Object

persistencyv

2-d, 3-d

graphics

GUI

Toolkits

Math Libs

Statistics

Detector

Geometry

Event

Generators

Dectector

Simulation

Object

persistency

Histograming

Fitting

Event Models

Folders

Event Display

Ntuple

analysis

Interpreters

DAQ

Online

System

services



GRID

middleware

RDBMS

run/file

catalogs

Object

persistencyv

2-d, 3-d

graphics

GUI

Toolkits

Math Libs

Statistics

Detector

Geometry

Event

Generators

Dectector

Simulation

Object

persistency

Histograming

Fitting

Event Models

Folders

Event Display

Ntuple

analysis

Interpreters

DAQ

Online

System

services


Way 2: Framework with Object bus

Object bus: Object dictionary

Data Interface (I/O): Functional Interface

User

Applications

Higher level

framework

services

Higher level

framework

services

Experiment

framework

Higher level

framework

services

Higher level

framework

services

Higher level

framework services

User

Applications



Evidence

It takes time to build solid foundations more than 6 years for Geant3, PAW, ROOT

The process must be validated by users in different experiments in different configurations

In the long term, users prefer general purpose frameworks to experiment-specific frameworks because users work in 3, 4 experiments during the life time of any framework

People vote with their feet


ROOT libs structurepcnotebrun) #[1006] cd $ROOTSYS/lib(pcnotebrun) #[1007] ls -l total 26860 1246870 Jun 10 19:58 libCint.so 5468790 Jun 13 18:26 libCore.so 428183 Jun 13 18:30 libEG.so 156149 Jun 13 18:30 libEGPythia.so 164398 Jun 13 18:30 libEGPythia6.so 163824 Jun 13 18:30 libEGVenus.so 320933 Jun 13 18:30 libGX11.so 186870 Jun 13 18:30 libGX11TTF.so 995291 Jun 13 18:28 libGpad.so 1508147 Jun 13 18:28 libGraf.so 1031451 Jun 13 18:28 libGraf3d.so 3505642 Jun 13 18:29 libGui.so 1605028 Jun 13 18:30 libHbook.so 1840708 Jun 13 18:27 libHist.so 332928 Jun 13 18:29 libHistPainter.so 114938 Jun 13 18:29 libHtml.so 170068 Jun 13 18:30 libMC.so 461357 Jun 13 18:27 libMatrix.so 326767 Jun 13 18:29 libMinuit.so 270753 Jun 13 18:30 libMySQL.so 21917 Jun 12 19:52 libNew.so 90902 Jun 13 18:30 libPgSQL.so 345287 Jun 13 18:29 libPhysics.so 199557 Jun 13 18:29 libPostscript.so 500237 Jun 13 18:29 libProof.so 148509 Jun 13 18:30 libRFIO.so 2015746 Jun 12 20:05 libRGL.so 181086 Jun 13 18:29 libRint.so 35410 Jun 12 20:07 libSRPAuth.so 1123930 Jun 13 18:30 libTable.so 317808 Jun 13 18:30 libThread.so 1059588 Jun 13 18:27 libTree.so 369594 Jun 13 18:29 libTreePlayer.so 416115 Jun 13 18:29 libTreeViewer.so 158887 Jun 13 18:30 libX3d.so


Proposal

The existing set of ROOT libs is the starting core of the LCG software.

Because the system is already widely distributed and used, it guarantees

the initial acceptance to a wide community.

We invite architects and key developers to review the current organisation

of libraries and to propose an evolution if it proves necessary, keeping

in mind that this may affect thousands of existing users.

Several actions are proposed in the following slides

(non exhaustive list)


Review of libCore components

Because libCore is the essential library, it is important to understand the rationale behind its current organisation.

We want to keep this core as small as possible.

It will not be possible to agree on many other things if we cannot find an agreement on this fundamental component.


Object Persistency

GRID

middleware

RDBMS

run/file

catalogs

Object

persistencyv

2-d, 3-d

graphics

GUI

Toolkits

Math Libs

Statistics

Detector

Geometry

Event

Generators

Dectector

Simulation

Object

persistency

Histograming

Fitting

Event Models

Folders

Event Display

Ntuple

analysis

Interpreters

DAQ

Online

System

services

Continue current

developments in ROOT I/O

Foreign classes

Support for STL

Improved Trees

TLongRefs

Interface to Catalogs

Large files > 2 Gbytes

Implement in TStreamerInfo

in interpreted mode

what is currently

generated by Rootcint

for STL containers

{

{

{

Collaboration with

FNAL and BNL

Collaboration with

LCG

Collaboration with

BNL


Object Persistency

Propose a restricted discussion on the interface to the dictionary following the initial work by Craig.

Discussions and prototyping on access to CINT dictionary (feed/retrieve info) from external sources (eg XML/IDL, or other languages).

Idem for a restricted group discussing, prototyping LongRefs.

Would like to develop a variant of TStreamerInfo class generating XML for low volume exchange of objects.

Implementation of the RDBMS layer is an independent project (Dirk)


GRIDS Run/file catalogs

GRID

middleware

RDBMS

run/file

catalogs

Object

persistencyv

2-d, 3-d

graphics

GUI

Toolkits

Math Libs

Statistics

Detector

Geometry

Event

Generators

Dectector

Simulation

Object

persistencyv

Histograming

Fitting

Event Models

Folders

Event Display

Ntuple

analysis

Interpreters

DAQ

Online

System

services

Interface with Grid middleware

Interface with Alien (see Predrag)

PROOF (GRID oriented)

Interfaces with RDBMS

Oracle, MySQL, Postgres, etc

TSQL, TSQLResult

Discussions with GriPhynExpecting

close relationship

with LCG


DAQ Online

GRID

middleware

RDBMS

run/file

catalogs

Object

persistencyv

2-d, 3-d

graphics

GUI

Toolkits

Math Libs

Statistics

Detector

Geometry

Event

Generators

Dectector

Simulation

Object

persistencyv

Histograming

Fitting

Event Models

Folders

Event Display

Ntuple

analysis

Interpreters

DAQ

Online

System

services

Multi-Threading support

Shared memory

Sockets/Monitors

client/server

Network classes

System interface/Signals

Interpreter

Histograming

Event Display

Browsers/Inspectors

Persistency

Electronic logbook at FNAL

well integrated with ROOT

JavaRoot interface

possibly interesting


Object Dictionary

Follow the new C++ standard proposals in the area of instrospection, RTTI.

Too much emphasys so far on the transient class dictionary.

The real difficulty is the dictionary support for automatic schema evolution (persistent views and relationship with current transient classes).

We did not discuss enough this essential area. Remove as much as possible automatic

generated code in favour of dynamic interpretation.


CINT, Python, Java, C#

GRID

middleware

RDBMS

run/file

catalogs

Object

persistencyv

2-d, 3-d

graphics

GUI

Toolkits

Math Libs

Statistics

Detector

Geometry

Event

Generators

Dectector

Simulation

Object

persistencyv

Histograming

Fitting

Event Models

Folders

Event Display

Ntuple

analysis

Interpreters

DAQ

Online

System

services

CINT: Smooth transition from interpreted code

to compiled code dynamically unlinked/linked :

root > .x script.C (interpreted)

root > .x script.C++ (compiled)

Facilitate automatic interfaces

to Python and Java

Current implementations are slow

Root dictionary could be

exploited to improve run time.


Interpreters

Propose CINT as the main line because the same script can be interpreted or/and compiled.

CINT matches very well with the GUI signal/slots. Interpred code can easily be called from compiled code and vice-versa.

Propose to extend Pere’s Python interface and support it. Pere’s example with Excel is a good example of use of Python. Interactive analysis with loops filling histograms is not realist.

Propose to consolidate the JavaRoot interface from Subir Sarkar. I hope that Tony and Julius will be interested.

Investigate an interface with C#


Event models: Folders/Tasks

Event Models

Folders/tasks

Class 1 Class 2

Class 3

Class 4

Class 5Class 6

Class 7

Class 8

Det

Det

Det

Det

Det

Det

DetTask

Task

Task

TaskTask

Task

TaskTask

Task

TaskTask

Task

TaskTask

Task

TaskTask

Task

TaskTask

Task

Some convergence with Gaudi

to be exploited

Whiteboard communication

Tasks = Algorithms

Folders <--> Trees

to be developed


Event Models

Encourage discussions on the transient event store organisation and tools to facilitate the export/import to persistent containers like the ROOT trees.

This is in my mind an important item: role of collection classes, whiteboard communication style.

How to work (transient/persistent view) with external class libs (eg G4) collections.


GUI Toolkits

GRID

middleware

RDBMS

run/file

catalogs

Object

persistencyv

2-d, 3-d

graphics

GUI

Toolkits

Math Libs

Statistics

Detector

Geometry

Event

Generators

Dectector

Simulation

Object

persistencyv

Histograming

Fitting

Event Models

Folders

Event Display

Ntuple

analysis

Interpreters

DAQ

Online

System

services

Consolidate the TVirtualX interface

Complete TVirtualX/Qt implementation

TVirtualX/Win32-FOX (free/fast on Windows)

Export script from a running GUI

Build GUI from a given script

GUI editor/builder

TVirtualX

TGWin32++

TQt

TGX11

TGWin32

The ROOT event loop

has proven to work

with all known

graphics systems:

X11, Xt, Motif, Qt,

Open Inventor, etc


GUIs An important area. Take advantage of the existing

expertise and experience with different solutions. Take a long term view and not just what is

fashionable today. Non blocking GUIs. GUIs in distributed

applications, including Web. We would like to discuss the Carrot project

see http://carrot.cern.ch

We want to consolidate the existing ROOT GUI We would like a full TVirtualX/Qt implementation

(Valery)


2-D, 3-D graphics

GRID

middleware

RDBMS

run/file

catalogs

Object

persistencyv

2-d, 3-d

graphics

GUI

Toolkits

Math Libs

Statistics

Detector

Geometry

Event

Generators

Dectector

Simulation

Object

persistencyv

Histograming

Fitting

Event Models

Folders

Event Display

Ntuple

analysis

Interpreters

DAQ

Online

System

services

Rich set

of 2-D and 3-D

primitives

with integrated picking

and context menus

+

Geometry primitives

+

X3D viewer

OpenGL viewer

OpenInventor viewer

+

Postscript,SVG

TVirtualPad

TPad

TVirtualGL

TVirtualPS

TCanvas

Hope to see

a cooperation

Olivier, Guy


2-d graphics

A lot of work to be done, eg; Support for TrueTypeFonts 2 New objects for DAQs, time series,

statistics Rethink existing organisation (TVirtualPad)

to optimize cooperation. Can we agree on the level of functionality (eg polymorphic graphics containers, picking) before discussing interfaces.


3-D graphics

Pros & Cons of existing 3-d viewers Do we assume that 3-D graphics is only on a

local PC or also across LANs/WANs? Do we want to duplicate the objects collections

between the user application and the renderer? Picking strategy (graphics list or really object

oriented)? Can we do the 2-D with 3-D / (as Guy proposes) many 3-D objects to be developed Is a cooperation really impossible?


Event Displays

Assume client-server model with access to remote files.

We have many very successful implementations


Histograming & Fitting

GRID

middleware

RDBMS

run/file

catalogs

Object

persistencyv

2-d, 3-d

graphics

GUI

Toolkits

Math Libs

Statistics

Detector

Geometry

Event

Generators

Dectector

Simulation

Object

persistencyv

Histograming

Fitting

Event Models

Folders

Event Display

Ntuple

analysis

Interpreters

DAQ

Online

System

services

Much more than HBOOK/PAW

Fix & var bin size for 1-d, 2-d 3-D

Profile 1, 2 & 3D

All kinds of projections, slices

Errors for all dims

Filling with strings (auto sort)

time axis

associated fitting

Random n. generation

+ auto binning

+ auto addition

support for parallelism

Fairly complete

no requests

for extensions

Fitting

new ideas

collaborators


Ntuples & Trees analysis

GRID

middleware

RDBMS

run/file

catalogs

Object

persistencyv

2-d, 3-d

graphics

GUI

Toolkits

Math Libs

Statistics

Detector

Geometry

Event

Generators

Dectector

Simulation

Object

persistencyv

Histograming

Fitting

Event Models

Folders

Event Display

Ntuple

analysis

Interpreters

DAQ

Online

System

services

PAW-like queries on attributes

tree.Draw(“varx”,”sqrt(x*y) <z”)

tree.Draw(“event.tracks.GetPt()”)

+

Tree browser and viewer

+MakeClass

(generation of skeleton analysis code)

+MakeSelector

same as MakeClass for PROOF

Collaboration

with FNAL

Collaboration

with MIT


Math Libs & Statistics

GRID

middleware

RDBMS

run/file

catalogs

Object

persistencyv

2-d, 3-d

graphics

GUI

Toolkits

Math Libs

Statistics

Detector

Geometry

Event

Generators

Dectector

Simulation

Object

persistency

Histograming

Fitting

Event Models

Folders

Event Display

Ntuple

analysis

Interpreters

DAQ

Online

System

services

In ROOT today

TVector2,3

TLorentzRotation

TLorentzVector

TRandom,2,3

TMatrix

TMath

TFeldmanCousins

TPrincipal

TMultidimFit

Can generate random numbers

from basic distributions; gaus, poisson, etc

from parametric analytic functions 1,2,3-d

from histograms, 1,2,3-d

A collection of many algorithms

CERNLIB, Numerical Recipees in C/C++

Would like to see an interface

to GSL

to Numerical Recipees in C++

Collaboration with

Fred James, Louis Lyons, Pushpa Bhat

Many algorithms classes

developed by a huge user community

See recent FNAL meeting

and effort organized within ACAT


Math Libs , Statistics Not much to keep from CLHEP. Assuming that a team will investigate GSL and its

interface to interpreters (should be trivial). Would like to see a better integration of the new

nice “Numerical Recipees in C++” Plenty of people “outside” willing to help. See

ACAT working group or “Confidence Levels” workshops (Fred James, Louis Lyons, Pushpa Bhat, Jim Linnemann, Prosper Harrison, etc)

Linear algebra important (including persistent objects)


Detector geometry

GRID

middleware

RDBMS

run/file

catalogs

Object

persistencyv

2-d, 3-d

graphics

GUI

Toolkits

Math Libs

Statistics

Detector

Geometry

Event

Generators

Dectector

Simulation

Object

persistency

Histograming

Fitting

Event Models

Folders

Event Display

Ntuple

analysis

Interpreters

DAQ

Online

System

services

A very important element

Request number 1 at ROOT FNAL workshop 2001

Work in progress with ALICE

Huge interest in many experiments

I have a complete talk on this

Geometrypackage

Reconstructionprogram

Simulationprogram

Geant3-basedGeant4-basedFluka-based

C++classes

MySQL

Modelling

Visualisation

Interactivity

Where am I?

Distance to boundary

Closest boundary

Persistency

We want to make it

transparent to Geant3 users

Will be used by our Virtual MC


Detector Geometry package

I strongly hope we can cooperate on this subject. We have very nice results now. We have

concentrated the efforts on the caching techniques: speed is a key element.

We support the full detailed geometry on most existing detectors (tested with 37 !), including 4 LHCs.

We invite interested people in discussions and real work on the final API, functionality and testing.

We know how to interface it to G3.


TGeo performance vs Geant3

Number gtmedi Root Geant3/ gtmedi Root Geant3/ nodes physics physics Root random random RootGexam1 425 3.08 1.84 1.67 6.60 4.39 1.50Gexam3 86 2.87 2.15 1.33 3.47 2.50 1.38Gexam4 12781 2.51 2.20 1.14 12.09 11.18 1.08Brahms 2649 5.82 3.04 1.91 4.17 1.93 2.16Tesla 15370 6.56 5.58 1.17 12.95 7.15 1.81CDF 24422 14.81 4.31 3.43 20.94 5.85 3.57Minos_near 30988 30.93 20.99 1.47 21.57 13.70 1.57BTeVEcal 52 1.57 1.08 1.45 1.78 0.73 2.43BTeV 295310 45.27 25.88 1.75 197.06 26.83 7.34CMSEcal 251713 5.60 1.81 3.09 5.69 1.74 3.27CMS 1166310 33.57 8.76 3.83 39.09 24.98 1.56LHCb 1533488 7.98 6.75 1.18 12.58 2.89 4.35Alice 3080198 11.50 8.63 1.33 11.45 7.28 1.57Atlas 29046966 8.90 9.94 0.89 32.48 23.39 1.38


Detector Simulation

Propose to discuss the Virtual MC approach.

Goal is to help in comparing different trackers/physics engines.

VirtualMC guarantees common input for the geometry common input for the kinematics common output for hits/digits common visualisation


Detector Simulation

GRID

middleware

RDBMS

run/file

catalogs

Object

persistencyv

2-d, 3-d

graphics

GUI

Toolkits

Math Libs

Statistics

Detector

Geometry

Event

Generators

Dectector

Simulation

Object

persistency

Histograming

Fitting

Event Models

Folders

Event Display

Ntuple

analysis

Interpreters

DAQ

Online

System

services

ROOT can provide a solid base

for: geometry, visualisation,

interactivity, interpreter

and persistency

TVirtualMC

Geant3

Fluka

Geant4Kinematics Geometry

Hits, Digits

This strategy facilitates

migration or comparisons

with a common input

and a common output

Geant3.tar.gz includesan upgraded Geant3

with a C++ interface

Geant4_mc.tar.gz includesthe TVirtualMC <-->Geant4

interface classes


Virtual MC example (with G3)

{ // Load basic libraries gSystem->Load("libPhysics"); gSystem->Load("libMC"); gSystem->Load("$(MCINSTALL)/lib/Linux-g++/libexample01");

// Load Geant3 libraries // … skipped

// MC application Ex01MCApplication* appl = new Ex01MCApplication("Example01", "The example01 MC

application");

appl->InitMC("g3Config.C"); appl->RunMC(1);

} void Config() g3Config.C{ cout << "Create geant3" << endl; new TGeant3("C++ Interface to Geant3");}


Virtual MC example (with G4)

{ // Load basic libraries gSystem->Load("libPhysics"); gSystem->Load("libMC"); gSystem->Load("$(MCINSTALL)/lib/Linux-g++/libexample01");

// Load Geant4 libraries // … skipped

// MC application Ex01MCApplication* appl = new Ex01MCApplication("Example01", "The example01 MC

application");

appl->InitMC("g4Config.C"); appl->RunMC(1);}

void Config() { g4Config.C

// RunConfiguration for Geant4 TG4RunConfiguration* runConfiguration = new TG4RunConfiguration();

// TGeant4 new TGeant4("TGeant4", "The Geant4 Monte Carlo", runConfiguration);}

Documents

Blueprint RTAGs1 Coherent Software Framework a Proposal LCG meeting CERN- 14 June Ren é Brun