P. Chochula ALICE Week Colmar, June 21, 2004 Status of FED developments

P. Chochula ALICE Week Colmar, June 21, 2004

Status of FED developments


Outline

• This is a summary talk, based on presentations given at previous DCS Workshops and MAY TB

• As a reminder, FED architecture will be presented• We will give an overview of commands and

services recognized by the FED server• The aim of this talk is to collect your comments

– Performance tests of FED are scheduled for this summer

– Generic FED API is due be released in September– FED API will be frozen in December


Present Status• FERO access model has been discussed several times

– No negative feedback so far– Very useful feedback from SPD,TRD and TPC – Integration with ECS is already ongoing– Unfortunately some groups are still not familiar with the concept– We encourage detector representatives to communicate the status

to software developers within your groups –

• FED architecture was presented to the ALICE TB (May 24, 2004)

• DCS focuses now on standardizing of the FED commands and services


Reminder - FERO Access Architectures in ALICE

FERO

DDL

Control Monitoring

Class A

Class C Class DControl

Control MonitoringMonitoring

FEROFERO

Non-DDLNon-DDL

Class B

Control Monitoring

FERO

DDL

Non-DDL


Reminder - FED Architecture

Class B,C,D Control

Class A+B Control

ECSECS

DAQ/RCDAQ/RC DCSDCS

Control CPU

FERO Hardware Layer

FED Server

FED Client

Profibus, JTAG, etc.

Control CPU

DDL SW

DDL SW

FEDFED

DDL

Monitoring of all classes


Reminder - Architecture of the FED Server

(PVSS) DIM Client

CA1 CAi MA1 MAi

Hardware

HW access

Database

DIM server

Serv

ices

DIM Interface layer

allows for communication

with higher levels of software

Hardware access layer contains device drivers

FE

DSe

rver

Cli

ent

Soft

war

e

Com

mands

& D

ata

Application layer contains detector

control and monitoring code

(agents)


Implementing the FED Server• Implementation of the FED Server remains detector’s

responsibility

• All low-level logic (e.g. collision protection, acquisition of data etc.) is implemented in the Application layer

• Operational logic (complex actions such as calibration or DAQ-DCS-TRG synchronization) is implemented in upper layers, using the ECS

• FED server should be implemented in C++ (some architectures allow also for direct implementation in PVSS)


Standard FED Operation

OFF

Configured

Running

Configuring

Configure

Re - Configure

RunStop

Switch-Off

Error

Recover


Detector-Specific FED Operation

Calibrating

Configured

Testing SEU

Verifying JTAG

Verify JTAG

Calibrate

Off

ConfigureVerifying Readout

Verify Readout

Test SEU

Some FEDs move to Configured stateSome FEDs do not implement this feature at all

Example: SPD


FED Data flow

FED

DCS

Com

mands

Ser

vice

s


FED Server API

• Standard command structure:– Type of command– Target– Payload

• Target is either the sub-detector or its part – Need for detector naming scheme

• Payload consists of data to be written to the FED server or a database tag (data is retrieved from DB by FED Server)


FED Server Commands

• Two groups of standard commands are implemented by all FED servers– FED OPERATION commandsFED OPERATION commands allow for integration of

FED with upper layers of software– FED MONITORING commandsFED MONITORING commands allow for integration with

DCS (setting of monitoring and logging parameters, external triggering of FED data acquisition and debugging)

• Detector-specific FED commands facilitate the implementation of detector specific features (such as agent control, internal checks etc.)


Standard (Mandatory) Commands Recognized by FED Servers

• FED OPERATION commands:– Configure and Re-Configure– Run– Stop– Switch-Off– Ignore

• FED MONITORING commands:– Set_Monitoring_Parameters (deadbands, rates …)– Start/Stop_Monitoring– Set_Messenger_Parameters (logging mode, …)– Read_Value


Detector Specific Commands (SPD example)

• Verify_JTAG – to test the integrity of the bus

• Verify_Readout_Chain – to check the bus configuration

• Test_SEU – verify settings of internal registers

• Calibrate – perform DAC and Threshold scans

• Start/Stop Agents – for debugging purposes


Services Published by the FED Server

• FED OPERATION service – contains data describing the status of the FED

• MESSENGER Service – published messages (errors, warning, debugging information)

• Detector related DCS DATA (temperatures, voltages, etc.)


Standard (Mandatory) Services Provided by FED Servers

• FED Operation Service: FED_Status provides structured information of internal status. Published states are: – OFF– Configured (Configuring)– Running– Ignoring (published via separate channel)– Error

• Using the structured information provided by FED, the DCS computes the overall state for the sub-detector


The Messenger Service

• Publishes information on FED server operation

• Each action results in a message – only requested types of messages will be published

• Main subscriber to Messenger data is the PVSS client

• Information provided by the Messenger service is logged by DCS and integrated with standard Alarm and Error handling


DCS Data Published by FED

• Structure and contents of the service differs from detector to detector

• Published data should be grouped in a pragmatic way:– Reasonable size of data published by one

service channel– Published data should be preferably organized

in the same way as readout – this will simplify correlation of physics with DCS data (e.g. all temperatures from a sector serviced by the same DDL)


Temperatures from this Sector are published as a single service

channel containing 12

values

Example of DCS data organization (SPD)

This SPD Sector is readout by a single DDL

Reminder: Numbering Convention has been approved and must be followed


Configuring the FED• Two types of configuration data:

– DCS related FED Server configuration (alarm limits, monitoring rates and deadbands)

– FERO configuration data (thresholds, DAC settings, etc.)

• FED Server configuration is downloaded from PVSS at system startup and can be modified during the operation (concept of recipes)

• FERO configuration data is loaded directly from the FED server in order to reduce the amount of data passed through PVSS. The configuration request originates in ECS/DCS.


FED Server Monitoring• All monitored data is published as DIM service

– Standard data is published by each FED server• server state information• messenger service

– Detector specific parameters are published by individual FED servers

• DCS Data acquisition is auto-triggered by FED server at predefined intervals (set by PVSS)

• Implemented commands allow for acquisition of DCS data on external request

DCS data provided by FED is treated in PVSS as any other device data


Device Monitoring Principle in DCSP

ub

lish

ing

d

ead

ban

d

Publishedvalue

Acquiredvalues

Samplinginterval

Value recordedIn DCS

PVSS Alarm Limit

PVSS Alarm Limit


Integration of FED with ALICE online systems

• DCS treats the FED as its sub-system

• FED is modeled as FSM using the FSM tools based on SMI++

• Integration into ALICE online systems is done via ECS

DCS DAQ/RC

TPC

SPD

TRG HLT

ECS

…TPC

SPD

…TPC

SPD

…

FERO

LV

HV

Gas

LV

HV

FERO


Present developments

• SPD prototype built at CERN:– SPD HW emulator– SPD FED Server– PVSS FED Client

– Launched integration of FED server with FSM tools; first prototype exists (credits to Mike Swanger)

SPD (PVSS) DIM Client

CA1 CAi MA1 MAi

SPD Router

VISA

DB

DIM server

Serv

ices

Com

mands

& D

ata


TPC/TRD/PHOS developments

• Custom layer implemented and tested (credits to S. Bablock and Ch. Kofler, U. Frankenfeld, M. Stockemier)

• Software is implemented with real hardware (DCS board)

• FSM logic being implemented• Agreed on prototype tests (performance, stability)

Excellent collaboration between TPC, TRD and DCS teams.


Comparing TPC/TRD architecture with the generic model

DIM Server

FEE Client

InterCom LayerDB

PVSS(DIM - Client)

FEE Server

Original Picture provided by WORMS group

FEE ServerFEE Server

CA1 CAi MA1 MAi

HW access

DIM server

(PVSS) DIM Client

FEE servers are equivalent of Agents


Next Steps• Written version of this talk will be circulated to

detector groups in June/July– Detailed description of the FED concept– Implementation details of the FED server:

• Structure of commands and services• Specifications of the Messenger

– Simplified version of the SPD prototype will be provided as a generic example of working FED server and PVSS client

• We expect feedback from detectors, as the final FED specifications should be ready for approval in September


What is still needed

• Feedback• Description of detector structure

– Naming and numbering scheme of detector components

• Description of detector operation– Calibration procedures, detector specific procedures,

sequence of actions etc.

• Realistic estimate of published data• It is essential to dedicate a person responsible for

software developments and to start prototyping. FED servers must be fully debugged before we start integration with ALICE


Test in summer 2004• We plan to launch a big test involving a big number of

dummy FED servers– The aim of the exercise is to test the performance (e.g. ability of

PVSS to digest the large number of parameters provided by FEDs) – this is a crucial test which has to be performed

• We need and estimate of data to be read from your detectors and of the update frequencies– Please provide these number by the end of July– Even preliminary numbers are much better than none

• By the end of July we need:– List of additional commands (if any) to be recognized by the FED– List of parameters to be downloaded and monitored


Warning

• Implementation of FED Server is a very delicate task– Mixing of monitoring and control data could lead to

serious problems– Performance has to be carefully studied

• Working FED server is just the first step, operational details are even more complex– Need for synchronization between online systems– Problems caused in one sub-system could “silently”

propagate to different sub-systems without being immediately spotted (recovery procedures must be able to predict this and act)


Conclusions

• After March DCS workshop and TB presentation we consider the FED as an approved approach

• Development now focused on API definition

• Our next milestone is September – Release of the structure of generic commands

and services

Documents

P. Chochula ALICE Week Colmar, June 21, 2004 Status of FED developments