CMT Tutorial How to use CMT in Atlas Christian Arnault [email protected]

Christian Arnault – CMT tutorial – dec. 2001 1

CMT

CMT Tutorial

How to use CMT in AtlasChristian Arnault

[email protected]://www.lal.in2p3.fr/SI/CMT/CMT.htm


CMT

Introduction

• Presentation of the tutorial– It is composed of three short lectures (~ 20’ each)

• The Atlas work model and conventions

• Working on Atlas packages

• Advanced features

– It includes materials for documented examples (rather than exercises) with screen dumps

• Goal, scope– It is really aimed at Atlas context rather than being a general

purpose CMT tutorial– The expected audience ranges from the normal software operator up

to the more expert developer

• Announce– CMT workshop at LAL Feb-28/March-01 2002


CMT

Plan

• Lecture #1 :The Atlas work model and conventions– Introduction

– Work model, conventions

– How to setup CMT

– The CMT user interface

– The CMT architecture

• Lecture #2 : Working on Atlas packages– Working with a package

– Creating a new package

– Running an application

– Examples in Atlas

• Lecture #3 : Advanced features– Policy and patterns in Atlas

– Package types and areas

– Details on configuration parameters


CMT

Lecture #1 The Atlas work model and

conventions


CMT

Work model, conventions (1)

• The roles– So far, we only consider “developers” and “package coordinators”

• Developers work (edit, commit, tag, test, register) on individual packages• Package coordinators accumulate package tags and validate

• The scenarios– Working on a package in a private development area– Connecting to the official release

We are talking about the current practices in Atlas rather than the standard or the final definitions (since this is not yet fully defined / understood / adopted). A working group is currently being setup to define :

the roles (developer, coordinators, librarian, etc…)

the scenarios (developing, tagging, testing, integrating, installing, deploying, …)


CMT


• The software organization– The software base is a hierarchy of packages:

LArCalorimeterLArCalorimeter/LArBookkeepingLArCalorimeter/LArCellRecLArCalorimeter/LArClusterRecLArCalorimeter/LArCnvLArCalorimeter/LArCnv/LArAthenaRootLArCalorimeter/LArCnv/LArHitZebraCnvLArCalorimeter/LArConditionsLArCalorimeter/LArDetDescrLArCalorimeter/LArDetDescrObjyDDLLArCalorimeter/LArDigitizationLArCalorimeter/LArEventLArCalorimeter/LArEventObjyDDLLArCalorimeter/LArExampleLArCalorimeter/LArExample/LArBookkeepingExampleLArCalorimeter/LArExample/LArConditionsExampleLArCalorimeter/LArExample/SimpleCellBuilderLArCalorimeter/LArExample/TestLArDigitLArCalorimeter/LArExample/TestLArHardwareID

containers


CMT


• The software organisation– Packages may have particular semantics

• Normal packages provide software or documents

• Container packages manage other packages

• Policy packages hold Atlas specific conventions, policies, patterns

• Interfaces convert conventions and policies of external packages

– We must differentiate • The physical hierarchy (the directory structure)

• The logical hierarchy (relationships between packages)– Those two hierarchies are totally independent with each other

– A release is the unique and exhaustive set of all [package-version] duets validated as consistently built and cooperating. (see next screen)

• It is materialized by one container package AtlasRelease


CMT

use InDetSpacePoint InDetSpacePoint-00-01-10 InnerDetector/InDetRecuse SiClusterization SiClusterization-00-01-03 InnerDetector/InDetRecuse SiTrackerCluster SiTrackerCluster-02-01-00 InnerDetector/InDetRecuse SiTrackerSpacePoint SiTrackerSpacePoint-01-00-08 InnerDetector/InDetRecuse TRT_Rec TRT_Rec-00-00-28 InnerDetector/TRTuse LArBookkeeping LArBookkeeping-00-00-05 LArCalorimeteruse LArCellRec LArCellRec-01-01-06 LArCalorimeteruse LArClusterRec LArClusterRec-01-01-03 LArCalorimeteruse LArAthenaRoot LArAthenaRoot-00-01-05 LArCalorimeter/LArCnvuse LArHitZebraCnv LArHitZebraCnv-00-00-07 LArCalorimeter/LArCnvuse LArConditions LArConditions-00-00-04 LArCalorimeteruse LArDetDescr LArDetDescr-00-02-01 LArCalorimeteruse LArDetDescrObjyDDL LArDetDescrObjyDDL-00-00-01 LArCalorimeteruse LArDigitization LArDigitization-00-00-15 LArCalorimeteruse LArEvent LArEvent-00-00-15 LArCalorimeteruse LArEventObjyDDL LArEventObjyDDL-00-00-01 LArCalorimeteruse LArBookkeepingExample LArBookkeepingExample-00-00-06 LArCalorimeter/LArExampleuse LArConditionsExample LArConditionsExample-00-00-04 LArCalorimeter/LArExampleuse SimpleCellBuilder SimpleCellBuilder-01-00-04 LArCalorimeter/LArExampleuse TestLArDigit TestLArDigit-00-00-12 LArCalorimeter/LArExampleuse TestLArHardwareID TestLArHardwareID-00-00-06 LArCalorimeter/LArExampleuse TestLArHECTBCnv TestLArHECTBCnv-00-00-09 LArCalorimeter/LArExampleuse TestLArTBCnv TestLArTBCnv-00-00-08 LArCalorimeter/LArExampleuse LArGraphics LArGraphics-00-00-12 LArCalorimeteruse LArROD LArROD-01-00-03 LArCalorimeteruse LArSim LArSim-00-01-06 LArCalorimeteruse LArHECTBAna LArHECTBAna-00-00-03 LArCalorimeter/LArTestBeamuse LArHECTBCnv LArHECTBCnv-00-00-12 LArCalorimeter/LArTestBeamuse LArHECTBPed LArHECTBPed-00-00-03 LArCalorimeter/LArTestBeamuse LArTBCnv LArTBCnv-00-00-10 LArCalorimeter/LArTestBeamuse MagneticFieldAge MagneticFieldAge-01-00-07 MagneticFielduse MagneticFieldAgeHeader MagneticFieldAgeHeader-01-00-07 MagneticFielduse MagneticFieldAthena MagneticFieldAthena-01-00-07 MagneticFielduse MagneticFieldAthenaHeader MagneticFieldAthenaHeader-01-00-07 MagneticFielduse MagneticFieldCore MagneticFieldCore-01-00-07 MagneticFielduse MagneticFieldCoreHeader MagneticFieldCoreHeader-01-00-07 MagneticFielduse MagneticFieldStand MagneticFieldStand-01-00-07 MagneticField


CMT


• The software organisation– A release can be frozen or under development

• The latter will likely become the integration release

– Releases are maintained in

ATLAS_DIST_AREA/ATLAS_RELEASE• Where

– ATLAS_DIST_AREA takes one specific value per site where the Atlas software is installed (CERN, LBL, LBNL, laptops, …)

– ATLAS_RELEASE is a 3-number tag describing the release level

% ls /afs/cern.ch/atlas/software/dist/1.0.1 1.1.6 1.2.2 1.3.2 1.3.5 2.0.1 2.1.1 2.3.11.1.4 1.2.0 1.3.0 1.3.3 1.3.7 2.0.2 2.2.0 2.4.01.1.5 1.2.1 1.3.1 1.3.4 2.0.0 2.1.0 2.3.0 2.4.1


CMT


• The internal work model of CMT– Each package is identified by a triplet

[package - version - location]

– Packages are located and searched for in package areas– The search list for all possible areas is specified in one path-like

environment variable named CMTPATH

software/dist/2.4.0 % cmt show path# Add path /afs/cern.ch/atlas/offline/external/Gaudi/0.8.3 from ${CMTPATH}# Add path /afs/cern.ch/atlas/software/dist/2.4.0 from ${CMTPATH}# Add path /afs/cern.ch/sw/contrib from default path#/afs/cern.ch/atlas/offline/external/Gaudi/0.8.3:/afs/cern.ch/atlas/software/dist/2.4.0:/afs/cern.ch/sw/contrib

software/dist/2.4.0 % echo $CMTPATH/afs/cern.ch/atlas/offline/external/Gaudi/0.8.3:/afs/cern.ch/atlas/software/dist/2.4.0

Gaudi area

Release area


CMT


• The internal work model of CMT– Version specification accepts a semantics based on a 3-level

identifier scheme:<letters><major-id>[<letters><minor-id>[<letters><patch-id>]]

• Major-id describes major evolutions (that imply code change to client packages)• Minor-id describes minor evolutions (backward compatible change to the

interface, only imply re-compilation)• Patch-id describes simple patches (i.e. no change to any interface)

LArDetDescrObjyDDL-00-00-01LArDigitization-00-00-15LArEvent-00-00-15LArEventObjyDDL-00-00-01LArBookkeepingExample-00-00-06LArConditionsExample-00-00-04SimpleCellBuilder-01-00-04TestLArDigit-00-00-12

v1v10v10r1v13r10v1r0v1r24p0v1r6p2v2v2001v2r0p2


CMT


• The internal work model of CMT– The resulting directory structure of a package reflects this

organization:

LArCalorimeter/LArCellRec/LArCalorimeter/LArCellRec/LArCellRec-01-01-06LArCalorimeter/LArCellRec/LArCellRec-01-01-06/LArCellRecLArCalorimeter/LArCellRec/LArCellRec-01-01-06/ChangeLogLArCalorimeter/LArCellRec/LArCellRec-01-01-06/PACKAGELArCalorimeter/LArCellRec/LArCellRec-01-01-06/configureLArCalorimeter/LArCellRec/LArCellRec-01-01-06/srcLArCalorimeter/LArCellRec/LArCellRec-01-01-06/cmtLArCalorimeter/LArCellRec/LArCellRec-01-01-06/cmt/MakefileLArCalorimeter/LArCellRec/LArCellRec-01-01-06/cmt/requirementsLArCalorimeter/LArCellRec/LArCellRec-01-01-06/shareLArCalorimeter/LArCellRec/LArCellRec-01-01-06/Linux-gcc-dbgLArCalorimeter/LArCellRec/LArCellRec-01-01-06/Linux-gcc-opt

Location

Package

Version

Branches

Header files

Sources

CMT

Binary files


CMT


• The internal work model of CMT– Each package holds its own entire configuration specification into

one single text file named requirements• This file must reside in the cmt sub-directory

– By default everything belonging to the package (source files, or generated files) will stay within the context of the package

• Customizations are possible (eg to install generated files elsewhere)

– There is one single relationship established between packages, qualified by the use concept. (see next screens)

– CMT relies on this structure to address used packages, resolving complete address from

• The name

• The required version expression

• The possible access paths from CMTPATH


CMTLArCellRec/LArCellRec-01-01-06/cmt % cat requirementspackage LArCellRec

author RD Schaffer <[email protected]>author Srini Rajagopalan <[email protected]>

use AtlasPolicy AtlasPolicy-*use CxxFeatures CxxFeatures-00-* Utilitiesuse GaudiInterface GaudiInterface-01-* Externaluse StoreGate StoreGate-02-* Controluse Identifier Identifier-00-* DetectorDescriptionuse AtlasDetDescr AtlasDetDescr-00-* DetectorDescriptionuse EventAthena EventAthena-00-* Eventuse EventStructure EventStructure-00-* Eventuse ZebraTDREvent ZebraTDREvent-00-* Eventuse EventRawData EventRawData-00-* Eventuse LArDetDescr LArDetDescr-00-* LArCalorimeteruse LArEvent LArEvent-00-* LArCalorimeteruse LArROD LArROD-01-* LArCalorimeteruse LArSim LArSim-00-* LArCalorimeteruse CaloRec CaloRec-01-* Calorimeter


CMT

(Excerpt from the graphical representation of the use hierarchy by CMT)


CMT

How to setup CMT (1)

• The bootstrap operation: how to connect for the very first time with CMT– Call the setup script of CMT itself.– This selects one version of CMT– This only needs to be done once until the next change of CMT

version

For switching to another version of CMT, see later …

user/c/carnault % source /afs/cern.ch/sw/contrib/CMT/v1r10p20011126/mgr/setup.sh

user/c/carnault % cmt versionV1r10p20011126

user/c/carnault % echo $CMTROOT/afs/cern.ch/sw/contrib/CMT/v1r10p20011126


CMT


• Setting up an Atlas environment1. Install a HOME requirements file eg. in ${HOME}/cmtdir

2. Install in this file:a. The site identification (CMTSITE)

b. The location to the distribution area (ATLAS_DIST_AREA)

c. The selected release level (ATLAS_RELEASE)

d. The connection to the standard AtlasSettings definitions

3. … user/c/carnault % mkdir cmtdiruser/c/carnault % cd cmtdir

user/c/carnault/cmtdir % vi requirements

set CMTSITE CERN

macro ATLAS_DIST_AREA "/afs/cern.ch/atlas/software/dist“

#macro ATLAS_RELEASE "2.3.1"macro ATLAS_RELEASE "2.4.0"

use AtlasSettings v* $(ATLAS_DIST_AREA)

1

2-a

2-b

2-c

2-d


CMT


• Setting up Atlas environment in various sites– Change or select CMTSITE– Provide the possible values for ATLAS_DIST_AREA (some sites are

already defined)– A proper SITEROOT env. variable is expected for other sites. This

will provide logical root entry point towards various software

user/c/carnault/cmtdir % vi requirements

set CMTSITE CERN#set CMTSITE BNL#set CMTSITE LBNL#set CMTSITE EXTSITE

macro ATLAS_DIST_AREA "/afs/cern.ch/atlas/software/dist" \ CERN "/afs/cern.ch/atlas/software/dist" \ BNL "/afs/rhic/usatlas/software/dist" \ LBNL "/auto/atlas/local/software/dist" \ EXTSITE "${SITEROOT}/software/dist“

${SITEROOT}/cern => Cernlib${SITEROOT}/external => Atlas external software${SITEROOT}/ROOT => ROOT${SITEROOT}/sw => CERN Anaphe etc…${SITEROOT}/software => Atlas software


CMT


• Setting up an Atlas environment…

3. Configure for CMT by cmt config (this creates the setup and cleanup scripts)

4. Apply the setup script (only this last step has to be redone at each session startup).

c/carnault/cmtdir % cmt config------------------------------------------Configuring environment for standalone package.CMT version v1r10p20011126.System is Linux-gcc-dbg------------------------------------------Creating setup scripts.Creating cleanup scripts.

user/c/carnault/cmtdir % source setup.sh

3

4


CMT


• Setting up an Atlas environment…

5. Declare a private development area into the HOME requirements file

6. Create if needed the development area

7. Apply this definition to upgrade CMTPATH

c/carnault/cmtdir % vi requirements...path_remove CMTPATH ${HOME}path_prepend CMTPATH ${HOME}/devarea

user/c/carnault/cmtdir % mkdir –p ../devareauser/c/carnault/cmtdir % source setup.sh

user/c/carnault/cmtdir % cd ../devarea

5

67


CMT


• Understanding what is the current settings1. cmt version (the version of CMT)

2. cmt show path (the CMTPATH entries)

3. cmt show tags (the active configuration tags)

4. cmt show packages | grep dist (all reachable package-versions)

c/carnault/cmtdir % cmt versionv1r10p20011126

c/carnault/cmtdir % cmt show path# Add path /afs/cern.ch/atlas/offline/external/Gaudi/0.8.3 from initialization# Add path /afs/cern.ch/atlas/software/dist/2.4.0 from initialization# Add path /afs/cern.ch/sw/contrib from default path#/afs/cern.ch/atlas/offline/external/Gaudi/0.8.3:/afs/cern.ch/atlas/software/dist/2.4.0:/afs/cern.ch/sw/contrib

c/carnault/cmtdir % cmt show tagsLinux (from uname) package AtlasSettings implies [gcc debug]i386_linux22 (from CMTCONFIG) package CMT implies [Linux]CERN (from CMTSITE)debug (from package CMT) package AtlasSettings excludes [opt optimized]Default (from Default)gcc (from package AtlasSettings) excludes [egcs]Linux-gcc-dbg (from package AtlasSettings) package AtlasSettings implies [debug]Linux-gcc (from CMTCONFIG in package AtlasSettings)

1

2

3


CMT

The CMT user interface

• Principle : one command driver : cmt– All CMT actions are operated via this command line driver– gmake internally makes a wide use of it

• Basic syntaxcmt <verb> [ -<option>=<value> <parameters…> ]

• Main commands cmt config re-configure a package

cmt create … create a new package

cmt show … monitors configuration parameters

cmt broadcast … launch shell command over several packages

cmt checkout … interface to CVS’ checkout operation

cmt help get the list of possible commands

source cleanup.sh undo the source setup


CMT

The CMT user interface

• Parameter monitoring cmt show …

path package search paths

uses the use tree

macros all macro definitions

sets all set and path definitions

tags currently active tags

macro <macro-name> complete definition for a macro

set <set-name> complete definition for a set

constituents all constituents for this package

fragments all available make fragments

clients all direct clients of this package

packages all reachable packages


CMT

The CMT architecture

• The database– The requirements file

• It is the unique source of configuration parameters for all packages• CMT automatically connects requirements files of all used packages (belonging to

the current use tree)

• The use relationship– The unique relationship between packages managed by CMT– Parameterized by the version– Similar to inheritance (includes private or public qualification)

• The configuration parameters– Inherited parameters

• Use relationships : relate packages to each other• Symbols : define make macros, environment variables, aliases• Tags : define value selectors for symbols • Patterns : factor out recurrent configuration schemes• Make fragments : extend the CMT behaviour

– Local definitions (never inherited)• Constituents : for libraries, applications, document generators


CMT

End of lecture #1

Documents

CMT Tutorial How to use CMT in Atlas Christian Arnault [email protected]