RICH Data Flow

Jianchun Wang

2

HPD Readout Electronics

944 HPDs163 channels / HPD1 FE Hybrid / HPD

~160 FEMs6 FE Hybrids / FEM 1-13 Cables / FEM

20 - 30 DCBs~8 FEMs / DCB

Event Data

Timing and Control Data

3

Serial Line Cables

Timing and control link(tcdat) at 151 MHz (20refclk), 16-bit T/C word per BCO.

Reference: BTeV-doc-1920 by Walter Stuermer

Moving information between HPD multiplexers (MUX, FEM) and data combiner boards (DCB).

One or more cables per multiplexer board.

Each cable has 4 point-to-point serial lines, each with a differentially driven twisted pair (e.g. Category 6 cable).

All data is encoded in 8B10B format.

Reference clock link(refclk) at 7.6 MHz (132 ns BCO).

Event data links(evdat0, evdat1) at 640 MHz (712refclk), 267.2 bits data per BCO, or 263.6 Mbyte/s.

4

The HPD Occupancy

BTeV GEANT simulation with 2 interactions per bunch crossing.

Total about 14,000 events simulated (thanks to Raja).

Columns with odd ID are shifted down by a half HPD size.

Simplified row ID.

Electronic noise is not included in this plot.

5

Occupancy of The Hottest HPD

2 interactions per bunch crossing ( 132 ns ) are simulated.

Electronic noise is not included in the plot.

The hottest HPD has 19.9 hits per event, or 17.2 per bunch crossing.

There are 1.26 hits/event averaged over 944 HPDs.

6

Occupancy of The Hottest HPD

6 interactions per bunch crossing ( 396 ns ) are simulated.

Electronic noise is not included in the plot.

The hottest HPD has 52.6 hits per event, or 52.5 per bunch crossing.

There are ~3.7 hits/event averaged over 944 HPDs.

7

HPD Grouping

HPDs are grouped into 6-HPD modules. Each module is connected to 1 FEM board.

Some modules at the edge of HPD arrays contain less number of HPDs.

Only one of two HPD arrays is shown in the plot.

The last columns in both arrays are not grouped, and probable will be skipped.

There are 150 groups in total.

8

Occupancy of HPD Groups

2 Interaction per bunch crossing, no electronic noise.

The hottest group: 57 hits / bunch crossing.

9

Occupancy of HPD Groups

6 Interactions per bunch crossing, no electronic noise.

The hottest group: 180 hits / bunch crossing.

10

How Many Cables Needed?

The hottest HPD group with 2 interactions per bunch crossing: Occupancy: 57.0 hits / bunch crossing Noise level: 1% Total occupancy: hits / bunch

crossing Address length: 18 bits/hit (10 bits for 944 HPD, 8 bit for 163 channels) Data moving: 134.4 bits / bunch crossing (132 ns) Cables needed: ( cables + 20% overhead: ( cables

Total number of cables for 2 interactions per bunch crossing, 1% noise and 20% overhead: 458 cables.

With 6 interactions per bunch crossing, there are more physics hits (~ factor of 3) per bunch crossing, save level of noise, but more time to move data (factor of 3). In total 324 cables are needed.

11

Effect of Noise

2 interactions/bunch crossing (132 ns) 6 interactions/bunch crossing (396 ns)

No overhead, with 1% noise or no noise

12

Proposed Number of Cables

2 int

2 interactions/bunch crossing

Number of cables: 458

6 interactions/bunch crossing

Number of cables: 324

Including1% noise, 20% overhead

13

Data Flow Structure

This is just a guess. I am not quite sure how several lines will share the load, rotating to serve events or splitting an event. Currently I assume there is no time wasted.

128K FIFO is quite small. For the simulation it is big enough, and can save more than 100 bunch crossings for the hottest group.

FPG

A

128K FIFO

8B10B

encoder

buffer

buffer

buffer

buffer

14

Data Flow Simulation For each bunch crossing, the elastic and inelastic collisions are fully

simulated with BTeV GEANT with average 2 interactions per bunch crossing. The channels with no physics signal are simulated individually to have 1% change of noise hits. Note that 13.5% of the bunch crossing, there is no interaction but noise only.

For each bunch crossing a block of event data is generated, with 18-bit word for each hit. And two 18-bit words as event header: an event tag (3FFFF) and a BCO.

For each bunch crossing, the event data block is saved to FIFO first. Then part or all of the data saved in FIFO is transferred to outside via 8B10B encoder and cables within the bunch crossing time (132 ns).

If the buffer overflows, then only three 18-bit words are stored and transferred, with the extra word indicating the overflow status.

It is impossible to generate millions of events using BTeV GEANT. Instead, 14,000 events with 2 interactions per bunch crossing are fully simulated. The simulation then randomly pick events from this pool.

15

The FIFO Occupancy The highest occupied FIFO happens to be of the hottest group. With 4 million bunch crossings simulated, the maximum occupancy,

~30 Kbits is much smaller than FIFO capacity, 128 Kbits picked for this simulation.

2 interactions/bunch crossing 6 interactions/bunch crossing

16

Summary

This study is to roughly estimate what we need to move data out from front end electronics.

With noise level of 1% and 20% overhead, we need 458 cables to move the data from FEM to DCB at 2 interactions per bunch crossing and BCO time of 132 ns.

We need 324 cables at 6 interactions per bunch crossing and bunch crossing time of 396 ns.

The simulation shows that the FIFO occupancy reaches 26Kbits. A FIFO size of 128 Kbits is big enough.

17

Occupancy of The Hottest HPD Group

2 interactions per bunch crossing ( 132 ns ) are simulated.

Electronic noise is not included in the plot.

The hottest HPD group has 66.2 hits per event, or 57.0 per bunch crossing.

There are 7.9 hits/event averaged over all HPD groups.

18

Occupancy of The Hottest HPD Group

6 interactions per bunch crossing ( 396 ns ) are simulated.

Electronic noise is not included in the plot.

The hottest HPD group has180.2 hits per event, or 179.7 per bunch crossing.

There are 23.1 hits/event averaged over all HPD groups.

19

HPD Occupancy

2 interactions / bunch crossing, no electronic noise

The hottest HPD: 19.9 hits/event, 17.2 hits/bunch crossing

20

HPD Occupancy

6 interactions / bunch crossing, no electronic noise

The hottest HPD: 52.6 hits/event, 52.5 hits/bunch crossing

21

Row ID by Tomasz

22

The FIFO Occupancy With 4 million bunch crossings simulated, the maximum occupancy,

~30 Kbits is much smaller than FIFO capacity, 128 Kbits picked for this simulation.

2 interactions/bunch crossing 6 interactions/bunch crossing

23

Summary of Numbers

43DCBs

58 (100)52 (92)Cables (data lines)

2424FE Modules

PMT

3633DCBs

682 (1135)659 (1056)Cables (data lines)

332332FE Modules

MAPMT

3030DCBs

517 (951)498 (940)Cables (data lines)

200200FE Modules

HPD

16-bit format

(4 extra words)

18-bit format

(2 extra words)