-
LHCb and its electronics.
J. Christiansen, CERNOn behalf of the LHCb collaboration
[email protected]
AbstractThe general architecture of the electronics systems
in
the LHCb experiment is described with special emphasison
differences to similar systems found in the ATLASand CMS
experiments. A brief physics background anddescription of the
experiment are given to understand thebasic differences in the
architecture of the electronicssystems. The current status of the
electronics and itsevolution since the presentation of LHCb at
LEB97 isgiven and critical points are identified which will
beimportant for the final implementation.
I. PHYSICS BACKGROUNDLHCb is a CP (Charge & Parity)
violation experiment
that will study subtle differences in the decays of Bhadrons.
This can help explaining the dominance ofmatter over antimatter in
the universe. B hadrons arecharacterised by very short lifetimes of
the order of apico-second, resulting in decay lengths of the order
of afew mm. A typical B Hadron event is shown in Fig. 1 andFig. 2
to illustrate the types of events that must behandled by the
front-end and DAQ systems in LHCb. Atypical B Hadron event contains
40 tracks inside thedetector coverage close to the interaction
point and up to400 tracks further down stream.
Figure 1: Typical B event in LHCb
Figure 2: Close-up of typical B event
II. MAJOR DIFFERENCE FROM CMS AND ATLASThe LHCb experiment is
comparable in size to the
existing LEP experiments and it is limited in size by theuse of
the existing DELPHI cavern. The size, the budgetand the number of
collaborators in LHCb are of the orderof 1/4 of what is seen in
ATLAS and CMS. It consists of~1.2 million detector channels
distributed among 9different types of sub-detectors. Precise
measurements ofB decays, close to the interaction point, requires
the useof a special Vertex detector, located in a secondary
LHCmachine vacuum a few cm from the interaction point andthe LHC
proton beams. The need for very good particleidentification
requires the inclusion of two Ring ImagingCHerenkov (RICH)
detectors.
The layout of sub-detectors resembles a fixed targetexperiment
with layers of detectors, one after the other asshown in Fig. 3 and
Fig. 4. This layout of detectors,which can be opened as drawers to
the sides, ensuresrelatively easy access to the sub-detectors,
compared tothe enclosed geometry of ATLAS and CMS.
B hadrons are abundantly produced at LHC with a rateof the order
of 100kHz. Efficient triggering on selected Bhadron decays is
though especially difficult. This hasenforced a four level trigger
architecture, where thebuffering of data during the two first
trigger levels istaken care of in the front-end. The first level
trigger,named L0, has been defined with a 4.0 us latency(ATLAS/CMS:
2.5 us and 3.2 us) and an accept rate of 1MHz (ATLAS/CMS: 50 – 100
kHz). To obtain thistrigger rate, in the hardware driven first
level triggersystem, it has been required to limit the interaction
rate toone in three bunch crossings to insure “clean events”
withsingle interactions (ATLAS/CMS: ~30 interactions per
-
bunch crossing). It has even been required to have aspecial veto
mechanism in the trigger system to preventmultiple interactions to
saturate the available triggerbandwidth. The difficulty of
triggering on B events can beillustrated by the fact that the first
level trigger in LHCbis 3x30x1MHz/100KHz = 1000 times more likely
toaccept an interaction than what is seen in ATLAS/CMS.The high
trigger rate has forced a tight definition of theamount of data
that can be extracted for each trigger, andmade it important to be
capable of accepting triggers inconsecutive bunch crossings
(ATLAS/CMS: gap of 3 ormore). It is also necessary to buffer event
data during thesecond level trigger in the front-end electronics,
toprevent moving large amounts of data over long
distances(ATLAS/CMS: Only one trigger level in front-end).
Figure 3: Configuration of sub-detectors in LHCb
Figure 4: LHCb detector in DELPHI cavern
III. LHCB EVOLUTION SINCE LEB97Since the presentation of the
LHCb experiment at the
LEB97 workshop in London significant progress hastaken place.
LHCb was only officially accepted as a LHCexperiment in September
1998. The main architecture ofthe experiment and its electronics
has been maintainedand most detector technologies have now been
defined.
Detailed studies of the triggering have shown thattrigger
latencies were required to be prolonged. A L0
trigger latency expansion from 3.0 µs to 4.0 µs was
foundappropriate, as compatibility with existing ATLAS andCMS
front-end implementations was not an issue. The L1trigger
processing was found to be more delicate andsensitive to background
rates than initially expected. Thelatency has been prolonged from
50 µs to 1000 µs, as thecost of additional memory was found to be
insignificant.The Architecture of the trigger implementations has
nowbeen chosen after studying several alternative approaches.
The two levels of triggering in the front-end, withhigh accept
rates, has called for a tight definition of buffercontrol and
buffer overflow prevention schemes that workacross the whole
experiment. For the derandomizerbuffer, related to the first level
trigger, a scheme based ona central emulation of the occupancy has
been adopted.For the second level triggering and the DAQ system
anapproach based on trigger throttling has been chosen, asdata here
in most cases will be zero-suppressed andtherefore can not be
predicted centrally.
In a complicated experiment the front-end, trigger andDAQ
systems rely on a partitioning system to performcommissioning,
testing, debugging and calibration. Aflexible partitioning system
has been defined such thateach sub-system can run independently or
be clusteredtogether in groups.
For the DAQ system a push based event buildingnetwork,
distributing event data to the DAQ processingfarm, has been
maintained after simulating severaldifferent event-building schemes
on alternative networkarchitectures.
IV. FRONT-END AND DAQ ARCHITECTURELHCb has a traditional
front-end and DAQ
architecture with multiple levels of triggering and
databuffering as illustrated in the figure below.
Analog
L0 pipelinePile-up Muon Cal
L0 derandomizer
VertexL1 FIFO
CPU
CPU
Event N
Event N+1
X 100
CPU
Event N Event N+1
CPU
4 µs
16 events
1000 events
Throttle
X 1000
L0 derandomizer control
1 MHz
40KHz
200Hz x 100KB
40MHz
Event building network: 4GB/s
Reorganize
Event buffers
2GB/s
L2 & L3
Figure 5: General front-end, trigger and DAQ architecture.
-
Traditional pipelining is used in the first level triggersystem
together with a data pipeline buffer in the front-end electronics.
A special L0 derandomizer controlfunction is used in the final
trigger decision path toprevent any derandomizer overflows.
A second level trigger (L1) event buffer in the front-end is a
peculiarity of the LHCb architecture. Theinclusion of this buffer
in the front-end was forced by the1MHz accept rate of the first
level trigger. The associatedL1 trigger system determines primary
and secondaryvertex information in a system based on high
performanceCPU’s interconnected by a high performance SCInetwork.
For each event, accepted by the L0 trigger,Vertex data is sent to
one out of hundred processors thatmakes the decision of accepting
or rejecting the event.The processing of individual events is
performed onindividual processors and the processing time
requiredvaries significantly with the complexity and topology ofthe
event. This results in trigger decisions to be taken outof order.
To simplify the implementation of the L1buffers in the front-end,
the trigger decisions arereorganised into their original order.
This allows the L1buffers in the front-ends to be simple FIFO’s, at
the costof increased memory usage. An interesting effect of
thereorganisation of the L1 trigger decisions is that the
L1latency, seen by the front-end, is nearly constant eventhough the
processing time of events have large variationsas illustrated in
Fig. 10. After the L1 trigger all buffercontrol is based on a
throttle of the L1 trigger (enforcingL1 trigger rejects).
Finally event data is zero suppressed, properlyformatted and
sent to the DAQ system over a fewhundred optical links to a
standard module called thereadout unit as shown in Fig. 6. This
unit handles theinterface to a large processor farm of a few
thousandprocessors via an event building network. The processorfarm
takes care of the two remaining software driventrigger levels. An
alternative configuration of the readoutunit enables it to be used
to concentrate data from up to16 data sources and generate a data
stream which can bepassed to a read out unit or an additional level
of datamultiplexing. The readout unit is also used as an
interfacebetween the Vertex detector and the L1 trigger system.
The general architecture has been simulated withdifferent
simulation tools. The front-end and first leveltrigger systems have
been simulated at the clock levelwith hardware simulation tools
based on VHDL. TheCPU based systems (L1 trigger and DAQ) have
beensimulated with high level simulation tools like Ptolemy.
Event building network( 100 x 100 )
Readoutunit
Frontend
Readoutunit
Readoutunit
Front-endmultiplexing
Frontend
Farmcontroller
CPU
CPU
CPU
CPU
CPU
CPU
Farmcontroller
CPU
CPU
CPU
CPU
CPU
CPU
Farmcontroller
CPU
CPU
CPU
CPU
CPU
CPU
Storage
~1000 front-end sources
~100 readout units
~100 CPU farms
~1000 CPU’s(1000MIPS or more)
Front-end multiplexing basedon Readout Unit
4GB/s
< 50MB/s per link
Figure 6: DAQ architecture.
A. L0 derandomizer controlAt a 1 MHz trigger rate it is critical
that the L0
derandomizer buffer is used efficiently and overflows inany part
of the front-end system are prevented. The highaccept rate also
dictates the high bandwidth required fromthe L0 derandomizers to
the L1 buffer. This bandwidthmust though for cost reasons be kept
as low as possible,enforcing additional constraints on the L0
derandomizerbuffer. To ensure that all front-end implementations
arepredictable, it was decided to enforce a synchronousreadout of
the L0 derandomizer at 40MHz. A convenientmultiplexing ratio of
data from 32 channels is appropriateat this level. To be capable of
identifying event data andverifying their correctness additional 4
data tags (BunchID, Event Id and Error flags are obligatory) are
appendedto the data stream, resulting in a L0 derandomizer
readouttime of maximum 900 ns per event. To obtain a dead timebelow
1% a L0 derandomizer depth of 16 events isrequired as illustrated
in Fig. 7.
L 0 D e r a n d o m iz e r l o s s v s R e a d o u t s p e e
d
0
2
4
6
8
1 0
1 2
1 4
5 0 0 6 0 0 7 0 0 8 0 0 9 0 0 1 0 0 0
R e a d o u t s p e e d ( n s )
D e p th = 4 D e p t h = 8 D e p th = 1 6 D e p t h = 3 2
Figure 7: L0 derandomizer dead time as functionof readout time
and buffer depth
To simplify the control and prevent overflows of
thederandomizers, it is defined that all front-endimplementations
must have a minimum buffer depth of 16
-
events and a maximum readout time of 900ns. With sucha strict
definition of the derandomizer requirements, it ispossible to
emulate centrally the L0 derandomizeroccupancy. This is used to
centrally limit trigger acceptsas illustrated in Fig. 8. Remaining
uncertainties in thespecific front-end implementations are handled
byassuming a derandomizer depth of only 15 in the
centralemulation.
L0 pipeline
Derand.
L0 trigger
X 32
Data merging
4 data tags (Bunch ID, Event ID, etc.)
Data @ 40MHz
32 data
L0 derandomizer emulator
Not full
Readout supervisor
1MHz
Same state
Figure 8: Central L0 derandomizer overflow prevention.
In addition a simple throttle mechanism is available toenable
the following L1 trigger system to gate the L0accept rate if it
encounters internal saturation effects.
B. Consecutive triggersTriggers in consecutive bunch crossings
are in all
other LHC experiments not supported, to simplify thefront-end
implementations for sub-detectors that need toextract multiple
samples per trigger. In LHCb, with amuch higher trigger rate, a 3%
physics loss is associatedwith each enforced gap between trigger
accepts. It hasbeen determined that all sub-detectors can be made
tohandle consecutive triggers without major complicationsin their
implementation.
Consecutive triggers can also have significantadvantages during
calibration, timing alignment orverification of the different
sub-detector systems. Withthe defined size of 16 events in the L0
derandomizer, alldetectors can handle a sequence of up to 16
consecutivetriggers. Firing such a sequence of triggers can,
asillustrated in Fig. 9, be used to monitor signal pulses fromthe
detectors, study spill-over effects (in some casescalled pile-up)
and ease the time alignment of detectorchannels.
To insure the best possible use of this feature in theLHCb
front-end system, a simple and completelyindependent trigger based
on a few channels ofscintillators is under consideration. Such a
simple triggercan be programmed to generate triggers with
anycombination of interactions within a given time window
(No interaction, Single interaction, two interactions
inconsecutive bunch crossings, etc.).
Time alignment Pulse width
Baseline shifts Spill-over
Figure 9: Use of consecutive triggers for calibration,
timealignment and spill-over monitoring.
C. L1 and DAQ buffer controlThe control of L1 buffers in the
front-end and buffers
in the DAQ system can not be performed centrally basedon a
defined set of parameters. Event data are assumed tobe
zero-suppressed before being sent to the DAQ systemand large
fluctuations in buffer occupancies will thereforeoccur. A throttle
scheme is the only possible means ofpreventing buffer overflows at
this level. A throttlenetwork with up to 1000 sources and a latency
below 10us can throttle the L1 trigger when buffer space
becomesparse. A highly flexible fan-in and switch module is usedto
build the throttle network. The throttle switch allowsthe throttle
network to be configured according to thepartitioning of the whole
Front-end and DAQ system. Inaddition it has a history buffer,
keeping track of who wasthrottling when, to be capable of tracing
sources ofsystem dead time.
L1 buffer
4 tags32 data
900ns per event36 words per event @ 40MHz
36 words @ 40 MHz
Max 1000 events
L1 trigger
CPU
CPU
Event N
Event N+1 Reorganize
L1 buffer monitor(max 1000 events)
L1 decisionspacing (900ns)
TTC broadcast (400ns)
Readout supervisor
Zero-suppression< 25 µs
L1 derandomizer
Data merge
Output buffer
L1 Throttleaccept -> reject
DAQ
Data to DAQ
Nearly full
Nearly full
BoardSystem
L0Throttle
History trace
Throttle L0 triggers
Vertex
40 kHz
Figure 10: L1 and DAQ buffer control with plots ofestimated L1
trigger latency distributions before and after
reorganisation.
Because of the CPU based L1 trigger system, wherethe trigger
decision latency varies significantly fromevent to event, an
additional monitoring of the number ofevents in the L1 buffers is
implemented centrally. If it isseen that the L1 buffer occupancy
gets close to its
-
maximum (1000 events) a central throttle of the L0trigger will
be generated to prevent overflows.
The input data to the L1 buffers, from the L0derandomizers, are
as previously state specified to occurwith a spacing of 900ns
between each event. To preventconstraining the L1 buffers on their
readout side, the L1trigger decisions are specified to arrive via a
TTCbroadcast with a minimum spacing of the same 900 ns.This
simplifies the implementation of the L1 buffers andtheir control in
the different front-end systems.
D. Readout supervisorThe readout supervisor is the central
controller of the
complete front-end system via the TTC distributionnetwork. It
receives trigger decisions from the triggersystems but only
generates triggers to the front-end thatare guaranteed not to
overflow any buffer, according tothe control mechanisms previously
described. In additionit must generate special triggers needed for
calibration,monitoring and testing of the different front-end
systems.Generation of front-end resets on demand or at
regularintervals is also specified. During normal running onlyone
readout supervisor is used to control the completeexperiment.
During debugging and testing a bank ofreadout supervisors are
available to control different sub-systems independently, via a
programmable switch matrixto the different branches of the TTC
system. This allows avery flexible partitioning of sub-systems down
to TTCbranches. Each sub-detector normally consists of one or afew
branches.
DAQ
L0 trigger L1 trigger
L1trigger
Front-endDAQ
ECS
L0 interface L1 interface
L0 derandomizeremulator
Throttle
Sequenceverification
Special triggers
Buffer sizemonitoring
Throttle
TTC encoder
Resets
ECS interface
L0
L1
LHC interface
Ch. A Ch. B
Monitor
Monitoring
Control
Switch
TTC system
Figure 11: Architecture of readout supervisor.
The readout supervisor also contains a large set ofmonitoring
functions used to trace the function of the
system and the effective dead times encountered. This isread out
on an event by event basis to the DAQ systemtogether with the
normal event data and is also accessiblefrom the ECS system
(Experiment control system).
V. RADIATION ENVIROMENTThe LHCb radiation environment is in
first
approximation less severe than what is seen in ATLASand CMS
because of the much lower interaction rate(factor ~ 100 lower). On
the other hand, the LHCbdetector is a forward angle only detector,
where theradiation levels are normally the highest. The lessmassive
and less enclosed detector configuration alsoallows more radiation
to leak into the surrounding cavern.The total dose seen inside the
detector volume rangesfrom ~1Mrad/year in the Vertex detector to a
fewhundred rad/year at the edge of the muon detector asshown in
Fig. 12. The electronics located inside detectorsis limited to the
analogue front-ends and in some casesthe L0 pipeline and the
accompanying L0 derandomizer.
Figure 12: Radiation levels inside the LHCb detector.
At the edge of most detectors and in the cavern thetotal
ionising dose is of the order of a few hundred rad peryear and a
Neutron flux of the order of 1010 1Mevneutrons/cm2 per year. This
can be considered to besufficiently low radiation levels that most
electronics cansupport without significant degradation. The
electronicslocated in the cavern are in general the
front-endelectronics with the L0 pipelines and the L1 buffers,
andthe first level trigger systems. This electronics consists
ofboards located in crates, where individual boards can beexchanged
with short notice. It is assumed that short timeaccess to the LHCb
cavern, of the order of one hour, canbe granted with a 24-hour
notice. The installation ofpower supplies in the cavern is a
special critical issue astheir reliability in several cases have
been seen to be poor,even in low dose rate environments.
-
Because of the additional trigger level in LHCb, theelectronics
located in the cavern is of higher complexitythan seen in the other
LHC experiments. Significantamounts of memories will be needed for
this electronicsand the use of re-programmable FPGA’s is an
attractivesolution. The radiation level is though sufficiently
highthat single event upsets can still pose a significantproblem to
the reliability of the experiment.
A hypothetical front-end module handling ~1000channels, with L1
buffering and data zero-suppression,could use 32 FPGA’s with
300Kbit each for theirconfiguration. The estimated Hadron flux with
an energyabove 10Mev is of the order of 3 1010 cm-2 per year. Witha
measured SEU cross-section of 4 10-15 cm2/bit for astandard Xilinx
FPGA, one can estimate each module tohave a SEU upset every few
hours. At the single modulelevel this could possibly be acceptable.
In a system withthe order of 1000 modules, the system will be
affected bySEUs a few times per minute!. Unfortunately it will
berelatively slow to recover from such failures as theapproximate
cause must be identified and then the FPGAhas to be reconfigured
via the ECS system. This will mostlikely require several seconds to
accomplish. Thissimplified example clearly shows that SRAM
basedFPGA’s only can be used with great care, even in theLHCb
cavern where the radiation level is quite low.
VI. ERROR MONITORING AND TESTINGThe use of complex electronics
in an environment
with radiation requires special attention to be paid to
thedetection of errors and error recovery.
Frequent errors can be feared to occur, making it vitalthat
these can be detected as early as possible to preventwriting
corrupted data to “tape”. The format definition ofdata accepted by
the first level trigger has been made toinclude data tags that
allows data consistency checks tobe performed. The use of Bunch ID
and Event ID tags isenforced. Additional two data tags are
available for errorflags and data checksums when found appropriate.
Upthrough the hierarchy of the front-end and the DAQsystem the
consistency of these data tags must be verifiedwhen merging data
from different data sources. Thisshould ensure that most failures
in front-end systems aredetected as early as possible. All
front-end bufferoverflows must also be detected and signalled to
the ECSsystem, even though the system has been made to preventsuch
problems. The use of continuous parity checks on allsetup registers
in the front-end is strongly encouraged andthe use of self-checking
state machines based on one-hotencoding is also proposed.
To be capable of recovering quickly from detectederror
conditions, a set of well defined reset sequenceshave been
specified for the front-end system. These resetscan be generated on
a request basis from the ECS systemor can be programmed to occur
with predefined intervalsby the readout supervisor. To recover from
corruptedsetup parameters in the front-ends a relative fast
download of parameters from the ECS system has beenspecified.
Local error recovery, e.g. from the loss of asingle event fragment
in the data stream, is considereddangerous as it is hard to
determine on-line if the eventfragment is really missing or the
event identification hasbeen corrupted. Any event fragment with a
potential errormust be flagged as being error prone.
To be capable of performing efficient testing anddebugging of
the electronics systems, the normaltriggering path, the readout
data path and thecontrol/monitoring data path have been specified
to beindependent. All setup registers in front-endimplementations
must have read-back capability to becapable of confirming that all
parameters has beencorrectly downloaded. The use of JTAG boundary
scantesting is also strongly encouraged. To be capable ofperforming
efficient repairs of electronics in theexperiment, within short
access periods, it is importantthat failing modules can be
efficiently identified. It is alsoimportant that it can be
confirmed quickly if a repair hasactually solved the encountered
problem.
VII. EXPERIMENT CONTROL SYSTEMThe traditional slow control
system, or now often
called the detector control system, has in LHCb beenbrought one
level higher to actually control the completeexperiment. This has
given birth to a new system name:Experiment Control System (ECS).
In addition to thetraditional control of gas systems, magnet
systems, powersupplies, and crates the complete front-end, trigger
andDAQ system is under ECS control. This means that thedownloading
of all parameters to front-end and triggersystems is the
responsibility of ECS. These parametersinclude large look up tables
in trigger systems and FPGAconfiguration data on front-end modules
and will consistof Gbytes of data. The active monitoring of all
front-endand trigger modules for status and error information
isalso the role of ECS. In case of errors, ECS is responsiblefor
determining the possible cause of the error andperform the most
efficient error recovery. The DAQsystem, consisting of thousands of
CPU’s, is also underthe control of the ECS system which must
monitor/insureits correct function during running. With such a
widescope, the ECS system will be highly hierarchical with upto one
hundred PCs, each controlling clearly identifiedparts of the system
in a nearly autonomous fashion.
The ECS being the overall control of the wholeexperiment
requires it to have extensive support forpartitioning. The whole
front-end, trigger and DAQsystem must have hardware and software
support for suchpartitioning and the ECS will need a special
partitioningmanager function. The software framework for the
ECSsystem must be a commercially supported set of toolswith well
defined interfaces to standard communicationnetworks and links.
The physical interface from the ECS infrastructure tothe
hardware modules in the front-end and trigger systems
-
is a non-trivial part. The bandwidth requirements
varysignificantly between different types of modules and itmust
reach electronics located in environments withdifferent levels of
radiation. It has been emphasised thatthe smallest possible number
of different interfaces can besupported in LHCb. Currently an
approach based on thestandardisation of maximum three interfaces is
pursued.For environments with no significant radiation(underground
counting room) a solution based on a so-called credit card PC has
been found very attractive(small size, standard architecture,
Ethernet interface,commercial software support, and acceptable
cost). Forelectronics located inside detectors special radiation
hardand SEU immune solutions are required. The mostappropriate
solution for this hostile environment has beenfound to be the
control and monitoring system made forthe CMS tracker, being
developed at CERN. An interfaceto electronics boards located in the
low-level radiationenvironment in the cavern is not necessarily
well takencare of by the two mentioned interfaces. Here a
custom10Mbits/s serial protocol using LVDS over twisted pairsis
being considered with a SEU immune slave interfaceimplemented in an
anti-fuse based FPGA. All consideredsolutions supports a common set
of local board controlbusses: I2C, JTAG and a simple parallel
bus.
CreditcardPC
JTAGI2CPar
Serialslave
JTAGI2CParMaster
PC
Master
PC
Ethernet
LVDS
Opt.
LVDS daisy chain
Figure 13: Supported front-end ECS interfaces.
VIII. STATUS OF ELECTRONICSAs previously mentioned the
architecture of the front-
end, trigger and DAQ systems are now well defined. Keyparameters
are fixed to allow the differentimplementations to determine their
detailedspecifications. After the LHCb approval in 1998, manybeam
tests of detectors have been performed and the finalchoice of
detector technology is in most cases done. Theelectronics systems
are currently being designed. LHCb iscurrently in a state where the
different sub-systems are
progressing against the Technical Design Reports (TDR)within the
coming year. The choice of networktechnology for the event building
in the DAQ system ison purpose delayed as much as possible to
profit from thefast developments in this area from industry. The
requiredbandwidth of the network is though already available
withtoday’s technology, but prices in this domain are expectedto
decrease significantly within the coming years.
In most sub-detectors the architecture of the
front-endelectronics is defined and is progressing towards
theirfinal implementation.
For the Vertex detector the detailed layout of thedetector in
its vacuum tank is shown in Fig 14 and partsof its electronics is
shown in Fig 15.
Figure 14: Vertex detector vacuum tank and detectorhybrid.
Figure 15: Vertex detector hybrid prototype
A critical integration of electronics is required in theRICH
detector. A pixel chip (developed together withALICE) has to be
integrated into the vacuum envelope ofa hybrid photon detector tube
as shown in Fig. 16 and 17.Here a parallel development of a backup
solution basedon commercial Multi Anode Photo Multiplier Tubes
isfound necessary, in case serious problems in the
-
complicated pixel electronics and its integration into thevacuum
envelope is found.
Figure 16: RICH detector with HPD detectors
Figure 17: Pixel HPD tube
For the Calorimeter system, consisting of aScintillating pad
detector, a Preshower detector and anElectromagnetic and Hadron
calorimeter, most of thecritical parts of the electronics have been
designed andtested in beam tests. For the E-cal and the H-cal the
samefront-end electronics is used to minimise the designeffort.
Figure 18: Common Ecal and Hcal 12 bits digitisingfront-end.
IX. APPLICATION SPECIFIC INTEGRATEDCIRCUITS
The use of application specific integrated circuits isvital for
the performance and acceptable cost of all sub-detectors in LHCb.
ASIC’s are therefore criticalcomponents onto which the feasibility
of the wholeexperiment is based. ASIC design is complicated
andexpensive and any delays in their finalisation will oftenresult
in delays for the whole system. Delays of the orderof 1 year can
easily occur in the schedule of mixed-signalintegrated circuits as
no quick repairs can be made. In ourenvironment, which is not
accustomed to thedesign/testing and verification of large
complicatedintegrated circuits, the time schedules are often
optimisticand the time needed for proper qualification of
prototypesis often underestimated.
The rapidly evolving IC technologies offer enormousperformance
potentials. The use of standard sub-micronCMOS technologies being
radiation hardened usingenclosed gate structures has been a real
strike of “luck” tothe HEP community. Rapid changes in IC
technologiescan on the other hand pose a significant risk that
designsmade in “old” (~5 years) technologies can not befabricated
when finally ready. This fast pace in the ICindustry must be
considered seriously, when starting anIC development in the HEP
community, as the totaldevelopment time here is often significantly
longer thanwhat can be allowed in industry. The production of
IC’salso pose an uncomfortable problem. Any IC ready forproduction
today can most likely not be produced again ina few years. It is
therefore of outmost importance thatdesigns are properly qualified
to be working correctly inthe final application and that sufficient
spares areavailable.
As a rule of thumb, one can assume that each sub-detector in
LHCb relies on one or two critical ASIC’s.For harsh radiation
environments the use of sub-micronCMOS with hardened layout
structures is popular. Thetotal number of ASIC designs in LHCb is
of the order of10, spanning from analogue amplifiers to large
andcomplicated mixed signal designs. The productionvolume for each
ASIC is of the order of a few thousand.This low volume also poses a
potential problem as smallvolume consumers will get very little
attention from theIC manufactures in the coming years. The
world-wide ICproduction capacity is expected to be insufficient
over thecoming two years with a general under-supply as
aconsequence. In such conditions it is clear that smallconsumers
like HEP will be the first to suffer (not onlyfor ASIC’s).
X. HANDLING ELECTRONICS IN LHCBThe electronics community in LHCb
covering front-
end, trigger, ECS and DAQ is sufficiently small thatgeneral
problems can be discussed openly and decisionscan be reached. There
is a general understanding thatcommon solutions between sub-systems
and between
-
experiments must be used to manage to build the
requiredelectronics systems with the limited recourses
available(manpower and financial). One common ASICdevelopment is
made between the Vertex and the innertracker detector. This is also
assumed used in the RICHbackup solution. A common L1 trigger
interface, DAQinterface and data concentration module is designed
to beused globally in the whole experiment.
Regular electronics workshops of one week haveinsured that the
general architecture of the front-end,trigger, DAQ and ECS systems
are well understood in thecommunity and a set of key parameters has
been agreedupon. In addition, a specific electronics meeting, half
aday during each LHCb week, is held with no otherconcurrent
meetings. The electronics co-ordination is anintegral part of the
technical board with co-ordinatorsfrom each sub-system. In the
technical board it isunderstood that the electronics of the
experiment is acritical (and complicated and expensive and - - -)
part ofthe experiment that requires special attention in the LHCera
because of its complexity, performance and specialproblems related
to radiation effects.
XI. CHALLENGES IN LHCB ELECTRONICSSpecial attention must be paid
to certain critical points
in the development and production of the electronicssystems for
the LHCb experiment. Many of these will becommon to problems
encountered in the other LHCexperiments, but some will be LHCb
specific. The factthat the electronics systems in LHCb in many
cases stillare in a R&D phase will also bias the current
emphasisput on specific problems.
For problems common with the other LHCexperiments the most
efficient approach is obviously tomake common projects as has been
promoted by the LEBcommittee. Funding for such projects though seem
to bequite difficult to find. Specific areas where commonsupport
are crucial are the TTC system with itscomponents and support for
design of radiation hardenedASIC’s in sub-micron (0,25 um) CMOS
technology. Inaddition it would be of great value if the question
of usingpower-supplies in low – medium level radiationenvironments
(cavern) could be evaluated within such acommon project.
Time schedules of ASIC’s are critical as furtherprogress can be
completely blocked before working chipsare available. This is
especially the case where the front-end chips are an integral part
of the detector (RICHHPD). The out-phasing of commercial
technologies mayalso become critical in certain cases.
LHCb has a special need of using complicatedelectronics in the
experiment cavern. The total dose ishere sufficiently low that the
use of COTS can bejustified. The problem of SEU effects on the
reliability ofthe total system must though be carefully analysed.
Theuse of power supplies in the cavern is also a question thatmust
be considered.
It is clear that there is a lack of electronics designersin the
HEP community to build (and make working) theincreasingly
complicated electronics systems needed. Theelectronics support that
can be given by CERN, to all thedifferent experiments currently
under design, is alsolimited. Initiatives in LHCb have been taken
to involveother electronics institutes/groups in the
challengesinvolved in our systems. Engineering groups though
oftenprefer to work on industrial problems or specificchallenges
within their own domains. There is also acontinuous political push
for these groups to collaboratewith industry. With the currently
profitable andexpanding electronics industry it is also
increasinglydifficult to attract electronics engineers and
computerscientists to jobs in research institutions.
A new potential problem is surfacing in theelectronics industry.
The consumption of electronicscomponents is currently increasing
because of the successof computers/internet and mobile phones. Many
smallelectronics companies have serious problems
obtainingcomponents, as large customers always have precedence.This
problem of under-supply in the electronics industryis expected to
get even worse and potentially last thecoming few years. The need
of small quantities ofspecialised circuits for the electronics
systems for LHCexperiments may therefore bring unexpected delays in
thefinal production.
The verification and qualification of electronicscomponents,
modules and sub-systems, before they canbe considered ready for
production, is oftenunderestimated in our environment. The
complexity ofthe systems has increased rapidly with the last
generationsof experiments and the time needed for
properqualification often grows exponentially with complexity.This
problem is, as previously stated, especially criticalfor ASIC’s.
For programmable systems based of FPGA’sor processors this is to a
large extent less critical. Onemust though not forget that a board
based on a processoror FPGA’s is not worth much without the
properprogramming, which may take a significant amount oftime to
get fully functional.
We also have to worry about the usual problem ofdocumentation
and maintenance of the electronicssystems. The LHC experiments most
likely have to bekept running, with continuous sets of upgrades,
during tenyears or more. A set of schematics without any
otheradditional documentation is for complex systems far
fromsufficient. In many cases the schematics are not evenavailable
in a usable form, as the design of manyelectronics components will
be based on synthesis. Insome cases the tools used for the design
will even not beavailable after a few years.
