Ideas for FE electronics: Features for commissioning, debugging etc

Upgrade Electronics, 13/06/2013 1

Ideas for FE electronics:Features for commissioning,

debugging etc

Ken Wyllie


Time alignment

== aligning the BXIDResets

Ken Wyllie


BX0 BX1 BX2 BX3 BX4 BX5 BX6 BX7 BX8 BX9 BX10 BX11 BX12 BX13 BX14 BX15

BXIDRst-SODIN

BXIDRstA

BXIDRstB

‘Beam’ data inA and B

3562 3563 0 1 2 3 4 5 6 7 8 9 10 11 12 13

3562 3563 0 1 2 3 4 5 6 7 8 9 10 1135613560

BXIDA

BXIDB

Beams collide in only one BX (or a few well-spaced BXs)Detector A sends a packet with data and BXID = 4Detector A sends a packet with data and BXID = 2

H3653 H0 H1 H2 H3 H4

Data4 H5 H6 H7

H3653 H0 H1 H2 H3 H4

Data2 H5

H6 H7

Note that this requires EITHER1. sufficient number of BXID bits to unambiguously identify the correct BX OR2. data packets from every BX (or both!)

Ken Wyllie

Difference used to align BXIDResets


Diagnostics

During commissioning, need some handles for debugging.We should have useful features like:

1. Substitute the data with fixed patterns or other infoSpecial running modes set by ECS (+TFC-alignment-mode)Pattern size can match the link bandwidth (no buffer overflow)

2. Transmit the full 12-bit BXID + length without data payload=> very useful for time alignment

3. Nominal running will have the occasional truncated event => buffer full or Big-Event

What if we have many of these (eg due to a bad threshold)? How will we tell what is wrong? Can we add info to the truncated events to indicate the reason for the truncation?

Ken Wyllie


Using the GBTX

Ken Wyllie

Upgrade Electronics, 13/06/2013 6Ken Wyllie

GBTX asserts ‘ready’ signals when it is ready to transmit/receivetime for locking of PLLs, DLLs, data alignment etc

After power-up, the FE should wait for these to asserttxRdy = readiness of transmitterrxRdy = readiness of receiver

tx/rxValid = 0: H = 1010tx/rxValid = 1: H = 1001

GBTX has two ‘frame-types’, differentiated by two different 4-bit headers

For transmitter: the FE selects the ‘frame-type’ using the txValid controlFor receiver: the GBTX asserts/deasserts rxValid according to the frame it receivesSame functionality mirrored in the GBT-FPGA (Tell40)

=> we can use this feature for flagging frames (eg invalid data)

Control Signals to/from GBTX


Data valid signals in transmission data frames

FE buffer GBTX FPGA

RX

80

txValid

80

rxValid

If there is more than one FE buffer driving a GBTX, then the FE must make its own dataValid bit in its data field. This is included in the GBTX frame.

FE buffer

GBTX FPGARX

39

dataValid

FE buffer

39

dataValid

txValid

39

dataValid39

dataValid

Ken Wyllie


Data protection

If you want to use Widebus Mode, remember that the data is NOT protected against SEUs (or more conventional bit errors on the link)

You should estimate the rate and impact such errors have on your system


Data interface between FE and GBTXGBTX uses ePorts: 80, 160 or 320 Mbit/sOne GBTX-frame per 40MHz: 80 bits (FEC mode) or 112 bits(wideBus)

80 each ePort handles 2 bits of frame per 40MHz160 4320 8

GBTX

10 links @ 320Mb e

Ports

ePorts

FEegFEC, 320 GBTX

5 links @ 320Mb

FE

5 links @ 320Mb

FE

4.8 Gb 4.8 Gb

GBTX10 @ 320MbFE

10 @ 320Mb 4.8 Gb

4.8 Gb

Same for received dataie TFC/ECS links


GBTX

data in (80,160,320)

data out (80,160,320)

clock out (40, 80,160,320)

Eport signals

eportRX

eportTX

eportRX

eportTX

eportRX

eportTX

data in (80,160,320)

data out (80,160,320)

clock out (40, 80,160,320)

data in (80,160,320)

data out (80,160,320)

clock out (40, 80,160,320)


Data from FE to GBTXData sampling in GBTX ePortRX: 320 Mbit/s example

320 Mbit/sbit-stream

Fine phase adjustmentAdjustable delay

Data arrives at unknown phase wrt to GBTX internal clock

320MHz

Shift register

40MHz

Data[7:0]

deserialiserData[15:8]

Data[79:72]

X 10

X 10

Delay line shifts phase of data to align it with local 320MHz clock.Fine adjustment.Automatic tracking or manual setting.

Deserialiser converts serial stream at 320Mbit/s into parallel 8-bit word synchronous with local 40MHz clock

In 320Mbit/s FEC mode, 10 ePorts are used. Each receives 8 bits of the 80-bit GBTX frame. In 320Mbit/s widebus mode, 4 extra ePorts are used to add 32 bits to the GBTX frame.

Ken Wyllie

DatafromFE chip

GBTX

GBTXframe


Y ZX

320 MHzclock

Y ZX

Y ZX

Y ZX

Y ZX

delay0

delay1

delay2

delay3

delay4

Fine phase adjustment made in GBTX


GBTX ePortRX will handle FINE time alignment of bit streamBUT have to ensure that the 8 bits in the 320Mbit/s streamarrive at the correct time wrt 40 MHz clock in GBTX deserialiser

320 Mbit/sbit-stream

320MHz Shift register

40MHz

Data[7:0]

deserialiserData[15:8]A

B

On edge of40MHz clock, we want this

C

Shift register

Not this

Not this

Ken Wyllie


Requirements for FE

Fine phase alignment: requires transitions in the data=> FE must guarantee edges in the data stream

Coarse alignment to get 80, 160 or 320 Mbit/s stream lined up with 40MHz in GBTX ie data packet is in one GBT frameSuggestion: Implement adjustable delay in FE serialiser

Ken Wyllie

8 bits data @ 40MHzor test pattern

Parallel load

Shift clock320 MHz

Output of FEePortTX

going toGBTX

select

Can use a test pattern like 11111110 to check the alignment


Data from GBTX to FE (TFC/ECS)NB ePort clock output goes with data streamSo this clock can be used to sample the data in the FE

But must ensure correct alignment with 40MHz clock in FE

80Mb serial stream from GBTX80MHz clock from GBTX

40MHz

select

80Mb serial stream from GBTX80MHz clock from GBTX

40MHz

Example with 80MbSimple deserialiser

Phase selectabledeserialiser

Alternative: repeat each TFC bit twice => effectively 40Mb (but introduces other limits)


Back-up slides

Ken Wyllie


Orbit from LHC

BXID resetin S-ODIN

C=20B=10A=5

03543 3563

BXID resetIn Det A

5

TFCBXIDDetABXID

20

03563

BXID resetIn Det B

10

03563DetBBXID

3548

3553

Alternative for time alignment:Use the BXID_counter offset

Ken Wyllie


FE Buffer implementation

WR

WR Frame 0 goes to GBT: txDataValid=1

W R

012340123401234

01234

Frame 1 goes to GBT: txDataValid=1

WR No frame to GBT: txDataValid=0

01234

WR

Frame 2 goes to GBT: txDataValid=1

01234

Ken Wyllie


Synchroniser

compression logic

data filterRaw data

BXID

Buffer control

(packing)

SynchNZS

Buffer full

BufferData packet

Data Length

NZS

Synch

Compressed/Uncompressed data

Data_available

data

dataValid

Big Event

HeaderOnlyBX Veto

40 MHzclock

HeaderOnly or BXVeto

NZS

Synch

Data Length

Ken Wyllie


0 0 0 Raw data NZS Synch BXVeto HOnly

0

0 Compressed data BXID 12b

BXID 12b

BuffFulllength

Input to compressionlogic

Input to data filter 0 0 0 NZS Synch BXVeto HOnly

0

Compressed data BXID n bits lengthpacket from TFC filter


1


BXID 12b

BuffFulllength



1

BXID n bits packet from TFC filter

0

1

T

T


0


BXID 12b

BuffFulllength



0

packet from TFC filter 0T

BXID 12 bits Synch pattern

Length = buffer width = link frame width

NB Synch bit can be set for a few cycles so that synch pattern is repeated a few timesKen Wyllie



0

0 BXID 12b

BXID 12b

BuffFull



0

packet from TFC filter 0T

Raw data

Length = all 1s Raw data BXID n bits

Ken Wyllie


T

Compromise for transmitting fewer BXID bits

TBXID n 12 bits Synch pattern

Frame

TBXID n+1 12 bits Synch pattern

T Synch pattern

Synch=1Synch=1Synch=1

Synch=0BX

n+3 datalength

BXID n+212 bits

TBX n+4 length

data TBX n+5 length data

BXID n bits HeaderOnly/BXVeto/BufferFull 1

Compressed data BXID n bits Length not 0 or F

BX with data 0

BXID n bits Length=0BX with no data 0

T

BXID n bits NZS 0 Length=F

BXID n bits Big event 1

Uncompressed data

Could be the same,without length field:But then can’t differentiate between the two cases

BXID n bits 10

1

Ken Wyllie

Documents

Ideas for FE electronics: Features for commissioning, debugging etc