The optical readout chain for the ALICE Transition Radiation Detector Presentation at IRTG Seminar Felix Rettig 29. Juni 2007

The optical readout chain for theALICE Transition Radiation

Detector

Presentation at IRTG Seminar

Felix Rettig29. Juni 2007

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 3

Outline

• The ALICE Experiment

• The Transition Radiation Detector (TRD)

• Optical Readout Interface Board (ORI)

– features, high-speed laser modulation

– error rates, irradiation test results

• Readout datapath components

– Global Tracking Unit (GTU) and its sub-components

– low-latency transmission for triggering

– high-bandwidth buffering of raw data

– multi-event data buffering

• Current Status of GTU development and assembly

• Outlook


ALICE Experiment

• Pb-Pb collisions atup to 1150 TeV

• 8000 collisions persecond

• Goal: Quark-Gluon-Plasma

•Various detectors:TPC, TRD, ...

A Large Ion Collission Experiment (ALICE)


Transition Radiation Detector

• 1.2 million channels• 65664 MCMs


Transition Radiation Detector

• 1.2 million channels• 65664 MCMs

Radiator

Drift Chamber

Dri

ftA

mpl.

z

y

Cathode Pads

CathodeWires

AnodeWires


TRD Front-End Electronics (FEE)

• Multi-Chip Module (MCM)– 10-bit ADC for each of the 18 channels– up to 32 time bins, 10 MHz sampling rate

• ADC raw data reduction to stiff "tracklets" characterized byy-, z-position and deflection (compound 32-bit value)

Filt

er

Pre

pro

cess

or

Pro

cess

or

10

10

...

PASA

... 1

8x

...

An

alo

g s

ign

als

fro

m 1

8 c

ath

od

e p

ad

s

Read

ou

t Tre

eADC

ADC Netw

ork

Inte

rface

Data

Buff

ers

Tracklet Processor (TRAP)

Multi-Chip Module (MCM)

deflection

tim

e

bin

s

y

Local Tracking


TRD FEE Readout Tree• Readout Board (ROB):

– 16 MCMs in readout tree– 4 column mergers– 1 board merger

• Module:– 2x 4 (3) Readout Boards– 2 Half-Chamber Mergers– 2 Optical Readout Inter-

face Boards (ORI)

• half chamber data concen-trated by HC Merger

• optical transmission from ORI to Global Tracking Unit outside of L3 magnet

Board Merger

C C C C

Board Merger

C C C C

Board Merger

C C C C

Board Merger

C C C C

HC Merger

ORI-Board

z

y

2x per Module

12 or 16 Pad Rows

2 x

72

Channels

Optical laser link toGTU located outsideof L3 magnet

The full TRD comprises• 65664+ MCMs on 4104 ROBs • 540 modules in 90 stacks• 1080 optical readout links


Optical Readout Interface Board

HC-MCM

LVD

S

CPLD8B/10BSerial.

LaserDriver

PROM

Key features:• conversion 120 MHz 8-bit DDR to 125 MHz 16-bit SDR• 8B/10B encoding and serialization to 2.5 Gbit/s• high-speed modulation of laser driving signals• 850nm Laser Diode (VCSEL) with Monitoring Diode

ROB

120 MHz 125 MHz 2.5 GHz

I²C bus

LaserDiode


Optical Readout Interface Board

CPLD

LVDSTransceivers

LaserDriver

LaserDiode

Connectors toReadout Board

8B/10B EncoderSerializer


Laser Diode

• Vertical Cavity Surface EmittingLaser Diode (VCSEL)

• built-in monitoring diode forpower regulation feedback

C

Iforward


Laser Modulation - Shunt Switching

diff.data

signal

DAC

+

-

DACS

TempComp

TempSensor

+-

+-

+-

ADC

ADC

DACS

LOGAMP

CURRENTATTENUATOR

CTempComp

Laser DriverAPC Mode ADC

Servo ControllerModulation CurrentIM tc1/tc2

Servo ControllerSource Current

IM Gain

TNom

IM Nom

Peaking

IM TC1,2

IMD TC1,2APC Gain

IM Rng

IMD Nom

IM Rng

IMD

RT

IS

IM ILD

LaserDiode

MonitorDiode


Laser Modulation - Eye Diagrams

eye closed

• big parameter space: 15 setup parameters in combination with various properties of board components determinelow- and high-speed characteristics of laser driving signals

• hard to find a common setup to be used for all 1080 boards

• very fast signals to be optimized, intricate to measure

8 m

A

LaserForwardCurrent

250 ps

clear eye openingeye closed


Laser Modulation - Eye Diagrams

eye closed

• huge parameter space: 15 setup parameters in combination with various properties of board components determinelow- and high-speed characteristics of laser driving signals

• hard to find a common setup to be used for all 1080 boards

• signals to optimize are very fast, intricate to measure

8 m

A

LaserForwardCurrent

250 ps

clear eye openingeye closed6 m

A

final setup with "peaking"

125 ps


ORIs in first Supermodule at CERNN

um

ber

of

Opti

cal Li

nks

Optical Power [µW]

First Supermoduleinstalled at CERN

• high reliability of data transmission achieved:

• broad safety margin ensures sufficient opti-cal power in case of laser degeneration

• laser diodes operating in lower region of forward current and optical power output

min

imum

pow

er

for

BER

<1

0-1

0

Bit Error Rates < 10-15


ORI Irradiation Tests

• Irradiation tests carried out at OCL

• 27.7 MeV proton beam

• All components meet safety margin of factor 4 life-time

Assumptions about TRDradiation environment:

DIP ~ 1.6 Gy, Dtot ~ 1.8 Gy in 10 years of operation


Full TRD Readout Chain

• Global Tracking Unit (GTU)• Track Matching Unit (TMU)• Supermodule Unit (SMU), Trigger Generation Unit (TGU)

ORI

ORI

Module

6x

Stack

TriggerDesign

EventBufferingDesign

Trackletsonly

Tracklets&

Raw Data

RX

TMU

5x

SMU

TriggerHandling &

Control

GTU Supermodule Segment

Front-End Electronicswithin L3 magnet

Half Chamber

DCSBoardTTCrx

DDLSIU

DATESoftware

DIU

D-RORC

MassStorage

DAQ System

CentralTrigger

Processor

TGU18

x

5x

TrackConcentr.

GTU Racks

12

fib

res

Racks belowMuon System

Storage


ALICE Trigger Timing

Tracklet Building

Global Tracking

PreTrigger

-

Tracklet Shipping

Drift Time

Fit Calculation

0 1 2 3 4 5 6 7 8

Collision

Time after Collision [µs]

Raw Data Shipping

Level-0Trigger

Level-1Trigger

TRD Level-1 TriggerContribution Shipping

Time after Collision

Collision Level-0Trigger Level-2 Trigger Window

1.2 73.8 µs493.8 µs

Tracklets

6.2 µs

Raw Data Data Forward to DAQ

Accept/Reject

Level-1Trigger


GTU Requirements

• 1080 optical links deliver both– tracklets: low-latency required, small volume, high

rate– raw data: big volume, latency subordinate, lower

rate

• huge bandwidth– each opical link: 1.94 Gbit/s– links into one TMU: 23.3 Gbit/s– full detector: 2.1 Tbit/s = 244 GByte/s

• fast track reconstruction and decision taking for TRD'slevel-1 trigger contribution

• uni-directional optical links, no flow control– buffering of incoming raw data with full bandwidth

• forward buffered event data to DAQ system or discard

• support for trigger interlacing, buffering of multiple events


TMU Requirements

• low-latency deserialization and 8B/10B decoding ofoptical data received from front-end electronics

• 12 independent 16-bit data streams at 125 MHz clocks

• single 4-Mbit SRAM with 128-bit interface at 200 MHz

• multi-event buffering of up to five events,appropriate data forwarding or discarding on L2 messages

• error detection / handling for each separate link

• aborting of incomplete events, discarding associated data

• delicate balance between speed and ressource utilizationto accomodate trigger and buffer designs in one FPGA

→ complex FPGA design, many clock domains,highly pipelined data path design neededto allow for operation at 200 MHz


Receiving of Optical Data

• optical-electricalconverter modules

• Virtex-4 FX100 FPGA• 16 Multi-Gigabit

Transceivers– highly configurable,

~100 parameters– clock reconstruction– deserializing– comma detection,

comma aligning– 8B/10B decoding– rate matching

• incoming 16-bit data streams synchronized to one 125 MHz clock

SIP

O

ClockDividersPhaseAlign

2.5

GH

z se

rial d

ata

CommaDetect

8B/10BDecode

8B/10BDescram

16x52ring

buffer

PMA

ClockControl

PCS

Fab

ric

Inte

rface

RX

Data

Fla

gs

10GBASE-RBlock Sync

10GBASE-RDecode

Clo

cks


Event Buffering Design

16

/12

8

Event Shaper

SRAMController

write read

16

/12

8

ReadAddress

Logic&

Control

DataFormatting

TMU/SMUInterface

Readout Unit

4-MbitSRAM

SMU Control(L0-/L1-Trigger)

Scheduling

MemoryManagement

Even

t (n

, 0

)

Even

t (n

+1

, 0

)

Even

t (n

, 1

1)

Even

t (n

+1

, 1

1)

Event InfoFIFO Buffer

Superm

odule

Unit

Links

from

one S

tack

... 1

2x ...

SMU Control(L2-Message)

Data Block, Link 0 Data Block, Link 11

... 12x ...

em

pty

ad

dre

ss

data

status

DatastreamMerging


Data Stream Merging

• collation of 16-bit values to 128-bit lines for each link• padding at the end of datastreams• 12 data path clock domain crossings

L0

tracklet data raw data

commas

SRAM

END

END

16

16

128

...

12

lin

ks

...

L1

... 12 blocks ...

• 12 independent 16-bit data streams at 125 MHz with gaps• single SRAM with 128-bit wide interface at 200 MHz

12

8-b

it L

ine


Data Collation

=

FIFOring

counter

BRAM0

BRAM1

BRAM2

BRAM3

Link Data

Data Valid

End Marker

Line Data

Last Line

Line Advance

Line Valid

3216

128

readaddress

writeaddress

writeenable

• Block-RAM primitives

– 16-bit / 32-bit collation

– safe datapath clock domain crossing

– buffering of filled 128-bit lines until scheduled forward to SRAM write

• commas discarded, only valid data words are collated

• write enable logic performs padding without need for extra endmarker write cycles

• end of data flag stored explicitly, no 32-bit compares necessary in later stages

125 MHzclock domain

200 MHzclock domain

Aligning Buffer


Buffer Organization

• SRAM divided into separate parts, one for each link• blocks independently organized as ring buffers• data of multiple events dynamically• double write pointers allow discarding of incomplete

events, directly freeing occupied memory locations• overrun protection, anticipatory busy signaling

4-MbitSRAM

Even

t (n

, 0

)

Even

t (n

+1

, 0

)

Even

t (n

, 1

1)

Even

t (n

+1

, 1

1)

Data Block, Link 11

... 12x ...

em

pty

mem

ory

Data Block, Link 0

rp

end (n)

end (n+1)

rp

end (n)

end (n+1)


Buffer Address ManagementEvent

n-2

Event

n-1

Event

n (

inco

mp

lete

)

+

+

Write Pointer A1

Block RAM

Bank B

Bank A

writeaddress

readaddress

1

0

R 18SRAMwriteaddress

offset

Data Block, Link 0

ReadPointer

A BWrite pointers

Write Pointer A0

Write Pointer B1

Write Pointer B0

... 12x ...

base address

eventendpointer0

• 2 write pointers– first word offset– current offset

• normal operation:– first constant– increment current

• event complete:– pointers swap

meaning• discard incomplete

event:– use first word

pointer, copy to current pointer

• only one RAM block needed for all pointers,no 12:1 multiplexers


Event Buffering Design - Detailed








Tests and Integration Progress

• GTU segment running in Münster for super-module assembly and testing

• all segments being assembled and tested currently

• transfer to CERN and installation scheduled for end of july

• some CTP signal handling and DDL/DAQ issues open

• trigger design not yet integrated


Prospective Work

• Finish development of GTU

– a lot of open detail issues

– multi-event buffering under full CTP control

• Installation at CERN

• Beam and performance tests, refinements

• Monitoring system, GTU event display

• Physics applications of GTU

– various trigger schemes based oncomprehensive track and pt information,e.g. Jet Triggers


Thank You for Your Attention

Questions?


TMU Board - Final Layout


SMU Board - Final Layout


TMU-SMU-Hybrid

Documents

The optical readout chain for the ALICE Transition Radiation Detector Presentation at IRTG Seminar Felix Rettig 29. Juni 2007