Atlas SCT ROD FDR Implementation Model

Atlas SCT ROD FDRImplementation Model

August 20, 2002

Atlas Wisconsin GroupKhang Dao, Damon Fasching, Douglas Ferguson, Owen Hayes,

Richard Jared, John Joseph, Krista Marks, Mark Nagel

Sriram Sivasubramaniyan (Oklahoma), Alden Stradling and

Lukas Tomasek (Institute of Physics AV CR, Prague)

2

ROD Overview Notes• FPGA real time data path with DSPs for back-end processing and

operation control.– Different firmware and part loaded for SCT or Pixel RODs

• 9U VME card, custom back plane.• Back-of-Crate Card for optical interface to modules.• Clock and Trigger.

– Clock and trigger signals are available on the back plane from the Trigger Interface Module in the crate.

• Supports 100 kHz trigger rate without an upgrade.• SCT: 96 input links at 40 MHz• Diagnostic FIFO Memories on board for testing and data monitoring• Register based interface with DSP list processor.• Fiber Optic S-Link to Readout Buffers.• 12 FPGAs, 4.7 Million gates of programmable logic.• 5 DSPs, One for control and 4 for calculations.• Cypress Semiconductor VME Slave Interface chipset.

– Block Transfers - 2.2 Mbytes/second Slave DSP read access.

– A32/D32 data transfers

– Simple interface and supports standard VME transactions w/ interrupts.

8/20/2002 3

ROD Implementation Model: FPGA Front End / DSP Back End

96Data RCVR

ROD Controller: Operation,

Command, Trigger

43

Formatter and EFB

FPGA96

Bac

k of

Cra

te C

ard

/ Fro

nt E

nd E

lect

roni

cs

Input Memory & XCVR Control

VME Slave Interface

Sli

nk o

n B

OC

VM

E B

us

32

Cont / Addr

Data Bus

Con

trol

, Sta

tus,

Mas

k

Trig Data (TTC Bus)

Tim

ing

Inte

rfac

e

Mod

ule

BOCSetup Bus

Serial Output Link

Serial Input Link

Xon/Xoff

ROD Busy

Dec

. Tri

g C

ount

Control & Status for Block Xfer/ DMA / S-Link

DSP Event Trapping,

Histograms (Four DSP Chips)

DM

A 1

Router FPGAS-Link & DSP DMA

Host Port Interface

Data Valid

S-link Data

DM

A 4

Router Halt Output

DMA Cont.

Eve

nt D

ata/

Tir

g T

ype

R/W Bus

R/W Bus

R/W Bus

...

Bus

y

Program ResetManager

Tok

en a

nd D

ynam

ic M

ask

S-link Data

S-link WR

S-link Clk

S-link Test

48

R/W Bus

Cnf

g, R

eset

Cnfg, Reset

8/20/2002 4

48

Link or Diagnostic

Data to Formatter

48Bac

k pl

ane

Con

nect

or

Input FIFO

4k words

48

48

Data

DataInput FIFO

4k words

Link Input from BOC

R/W Bus Data


Control FIFO Operation

ADV OE

48 bits are read as 32 bit word bits 0:3116 bit word bits 32:47

Advance FIFO and latch new input

Enable FIFO output data

Read or write data

Select 16 or 32 bit part of word

Retransmit FIFO

Data Receiver Implementation

FIFO O

ut R

egister

FIFO In

Register

FIFO D

ata R

egister

FIFO D

ata R

egister

ADV OE RT

8/20/2002 5

• Input Diagnostic Memories.– Play or record data to perform diagnostics and tests.

– Start of record operation is triggered by the ROD Controller.

– Trapping (record) of Link Data can be synchronized to triggers or configuration commands.

– The controller can retransmit the list in a loop. The retransmit can be issued using the serial inputs (SP0 or SP1) from the DSP, or by a preset counter used in one of the Test Bench modes. Trigger information (from the ROD TIM FIFO) can also be retransmitted so that the ROD can have trigger rates up to 100kHz.

– The SCT ROD implementation of the Input Diagnostic Memories is two groups of 4k x 48 bit FIFOs. The Pixel ROD will use 64k deep FIFOs.

• Drivers and Registers– The input drivers allow the input link data to be routed to the Formatters and FIFOS

simultaneously. The FIFO can not be read by the DSP while the Input Links are connected to the Formatters.

– The Register ICs are used to latch the data from the ROD bus or the FIFO. The FIFOs receive data from the ROD Bus from the DSP through the Controller. The ROD Bus is not setup to handle a burst from a FIFO.

Data Receiver Notes

8/20/2002 6

32

96

Control, Status, MaskDec.

Trig Count

BusyMode Bits

Formatter 0Serial to Parallel ConversionFlag Header and Trailer ErrorsData Modes: Raw, Condensed, ExpandedHeader Trailer Limit Busy LimitDerandomizing FIFO4k Deep Debug Memories

R/W Bus

TokenCmd

Input Links

12Input Links

Formatter 1

Formatter 7

Formatter 6

Formatter 5

Formatter 4

Formatter 2

Formatter 3

12Input Links

12Input Links

12Input Links

12Input Links

12Input Links

12Input Links

12Input Links

*Proposed for Pixel ROD: Formatters 0,1,4,5 only

Event ID Checking,

Dual EFB Design

Event Formatting

Error Counters

Event Fragment Builder FPGA

Event ID Data /

Trig Type Dynamic

Mask

46

46

16Kx46 FIFO

32

Control

EV Fragment Header/Trailer

16Kx46 FIFO

Control

Data Valid

Gatherer Halt Output

Almost Full

Almost Full

Data

32

46

Halt Output

Halt Output

Data

Data Valid

Data Valid

Format #

Format #

R/W Bus

To/

Fro

m R

oute

r F

PG

A

To/From Controller FPGATo/From Controller FPGA

Token01

Cont header,trailer or data

Debug FIFO

4k x 39B

Debug FIFO

4k x 39B

FPGA Formatter and EFB

Token45

TokenCmd

Token30

Token74

8/20/2002 Page 7

Formatter 0: Top Level

Link FormatterAnd

Link OutputFIFOs

Link[11..0]DATA 12b

FifoDATA 32b

Link Number:12:4 Encoder

Link# hasTOKEN

HOLDoutput 1b

Formatter Writes Trailer Encoder:

latch Trailers and store new internal stateon ShowTrailerFLAGS

FormatterBUS 26bfrom RodContoller

ShowTrailerFLAGS 1b

L1TriggerPulse 1b

TOKENin 1bfrom previous chip

FormatterWritesTrailer 12b

DFF

HeaderTrailerLimit

EFB

RodController

FormatterBUSStrobe 1b

HeaderTrailerLimit[11..0] 12b

FormatterWritesTrailer[11..0] 12b

TimeOutError[11..0] 12b

Link[11..0]hasTOKEN 12b

TOKENout 1b

to next chip

ModeFifoEmpty[11..0] 12b ModeFifoEmpty 1b

FifoDATA 32b

HOLDoutput 1b

ModeBitsIN[11..0] 12b

RegTransferACK 1b OperationsController:

Register Block

MasterNSlave 1b

DataValid[11..0] 12bCondensedMode[11..0] 12b

RODBusyLimit[11..0] 12b

HeaderTrailerLimit[11..0] 12b

RODBusyLimit[11..0] 12b

Link[11..0]hasTOKEN 12b

ThisChipHasToken

RODBusyLimit

DataValid 1bDataValid[11..0] 12b

TimeOutError 1bTimeOutError[11..0] 12b

CondensedMode 1bCondensedMode[11..0] 12b

StoreTrailerHit

1ea2 state state machine

ClearCurrentFlag

TokenToReadOutTimeout 7bDataOverflowLimit 8b

HeaderTrailerLimit 8bRodBusyLimit 8b

OverflowError[11..0] 12b

OverflowError 1bOverflowError[11..0] 12b

RodBusyLimit 8b

Configuration (12 x 4b)

Status (12 x 16b)

TokenToReadOutTimeout

DataOverflowLimit

HeaderTrailerLimit

RodBusyLimit

Configuration(11:0) FIFO Word Counts(12 x 8b)

8/20/2002 8

Static Link Mask 1 bit per

link

Data Valid to EFB

EFB Halt Output

Condensed Mode and Raw Mode

Selects R/W Bus

Serial to Parallel Conversion and correction of 1 bit errors in header and trailerRecord error in data words

...

12 ea. converters

Link 0

Link 1

Link 11

Link 10

Config, Raw ABCD format per link bases Condensed 2 hits per 16 bits & not Config, RawNot condensed and not Config, Raw Expanded 32 bits for 1 or 2 hits

Serial to Parallel Conversion and correction of 1 bit errors in header and trailerRecord error in data words

FIFO 256 W by 32 bits

FIFO 256 W by 32 bits

...

12 ea. FIFOsandControl

Readout Control

Token from Controller

ContTrailer & Word in

Data to EFB

Token to Next Link

Readout Control

ContTrailer & Word in

Token to Next Formatter

Token from Previous Link

ValidTrailer

Readout controller provides: Readout of data on token Link time out error detection, header generation Data over flow detection Header trailer limit Busy function and limit Dynamic mode bits Value Meaning 00 Normal 01 Skip Readout 10 Mask 11 Dump 1st read 2nd Trailers to Controller

ValidTrailer

Busy

Busy

Busy toController

Trailers to ControllerDecrementTriggerCounters

Mux

Dynamic Mode Bits

2 bits per link

Link on/off

32 b

32 b32 b

7 b

7 b

39 b

39 b

2 b

2 b

HeaderTrailer Limit to Controller

FatalError to Controller

Edge ModeTimeoutdata_to_Large

R/W Bus

Formatter 0: 12 Links Shown

8/20/2002 9

.Formatter FPGAs have separate versions of logic for SCT and Pixel Modules.

.Link output FIFOs are implemented with FPGA internal memory. The XCV400E has 40 4Kbit RAM Blocks that can be implemented as 256x16 Dual Port RAM. The SCT Formatter must accept 12 input links, and each Link FIFO must be 32 bits wide. The maximum size of the Link FIFO with 12 links of input data is 256 words deep.

.The next version of the Formatter will provide dynamic memory allocation to provide a 512 words deep FIFO if one link from a module is masked off. This modification will combine 256x32 FIFOs on adjacent links in the case that the Static Mask Bit is set.

.Formatter can keep up with any input data rate (within memory availability).

.Only headers and trailers shall be written to the Link FIFO when the occupancy reaches almost full (Header/Trailer Limit). The Header Trailer Limit value is set by writing to the HT Limit Register in each Formatter. The value a global setting for each group of 12 links.

.Individual links can be put in raw, expanded or condensed hit data mode (SCT) by writing the appropriate value to the Formatter Configuration Register.

Formatter

8/20/2002 10

. Sorts data by sequentially transferring one event of data from each link to the Event Fragment Builder when the Controller issues a FE Command Pulse (Token Command).

. When FIFO Dual Port Memory is almost full:– The Decoder forces the links that are almost full (Header/Trailer Limit)into write

header/trailer only mode to reduce bandwidth.– If Memory continues to fill, and the Link FIFO reaches it set value of ROD Busy

Limit, full flag, the ROD Busy signal is issued to to stop triggers. At this point, an error flag is issued to interrupt the ROD controller - fatal error condition(ROD busy)

.This should not be possible unless flow control has been asserted by the S-Link for an extended period of time, Header/Trailer Limit is set to the same value as ROD Busy Limit, or the front end electronics have malfunctioned.

. Support flow control - EFB can assert pause signal. The Link FIFO will continue to fill and can become full if back pressure is asserted for an extended period of time. In this condition, the ROD will lose any event data that is on the FE links, but will retain all events that are in the Link FIFOs.

Formatter

8/20/2002 11

. If the token has been sent to a specific link, and there is no data in the FIFO, the link will timeout after an internal counter reaches the value set in the Token to Readout Timeout Register. In this case the Link Formatter will output a header and a trailer with the timeout error bit flagged, and pass the token to the next link.

. Events that exceed the length limit defined for a specific subset of links will have data deleted and a header trailer generated with an error flag.

. Individual links can be masked off by writing a 1 to the Static Mask bit in the Formatter Configuration Register or dynamically using the Mode Bits. This allows the user to mask off known malfunctioning links from a list or table or on the fly.

. Correct for loss of synchronization (module data is ahead or behind other modules) in a module (not required but is in VHDL). The dynamic mode bits correct for a link by advancing or withholding the link data with respect to the other links. The values are shown below: Dynamic mode bits: Value Meaning

00 Normal Readout Mode 01 Skip Readout of FIFO for this Trigger 10 Mask Link for this Trigger (full contents of the FIFO will be flushed) 11 Dump current event, read second for this trigger

The L1 ID number is corrected in the EFB. BCID numbers are ignored.

Formatter

8/20/2002 12

.Serial-to-Parallel conversion with error checking. Errors Detected: Header 1 bit error Trailer 1 bit error Condensed mode data error detection Link time out error in readout Header trailer limit Data too large limit

Header Errors Accepted: 111010 Head Det. Head Error 111010 Yes No1111010 Y Yes 011010 Y y 101010 Y y 110010 Y y 111110 Y y 111000 Y y 111011 Y y1110100 Y y

Trailer Errors : Trailer 1000000000000000A changed bit in any position is accepted but trailer error is set.If in state “new_cluster” and trailer_det=1 set trailer error. Example below:<11101><0><nnn><bbbb,bbbb><1><01><aaaa><ccc,cccc><1><ddd> chip channel hit ^ Trailer Detect error

Condensed mode error detection:The ROD variable “edge_mode” is used to determine if the input hit data is consistent with the state of this variable. If not a condensed mode error bit is set.

Link time out error detection:If a link has not received data in the time set in the time timeout register, a header and trailer will be written with error code.

Data too large error detection:This is a historic state that is not longer needed. If the event is to large (based on a register) a status bit is set.

Header trailer limit error detection:If the Formatter header trailer limit is passed. The formatter will not place new data in the FIFO. When a trailer is detected the trailer is written with the header trailer limit error code.

Synchronization error detection:If one of the field separator bits is incorrect. The decoding is confused. The link will be changed to raw data till a trailer is detected.

Formatter

8/20/2002 13

.Formatter Data Format SCT all data is in 16 bit packets packed in 32 bit words bit 31:0 Data bit 35;32 Link Channel address (for all 32 bit words) bit 36 Time Out Error (for all 32 bit words) bit 37 Condensed mode bit (for all 32 bit words)

Formatter Output (bits 31:0 bits 37:32 for all words)Name Bits 15:0 or 31:16header 001pLLLLBBBBBBBBtrailer 010zhvxxxxxxxxxx1 hit cond 1FFFFCCCCCCCxfx02 hits cond 1FFFFCCCCCCCsfx11st hit clus exp 1FFFFCCCCCCC0DDD1 hit clus exp 1xxxxxxx0xxx1DDD2 hit clus exp 1xxxxxxx1DDD1DDDflagged error 000xxxxxxFFFFEEEraw data 011nnnxxWWWWWWWWcond=condensed, clus=clusterexp=expanded

Key :x= do not care (The ROD fills these with 0’s)B=BCIDW=raw data C=cluster base addressD=3 bit hit dataE=ABC error codeF=FE numberh=header trailer limit errorL=L1IDn=count of raw dataf=error in condensed mode data, 1st hits=error in condensed mode data, 2nd hitp=preamble errorz=trailer bit errorv=data overflow error

Condensed example W1 001pLLLLBBBBBBBB 1FFFFCCCCCCCxfx0 Header+1 hitW2 010zhvxxxxxxxxxx 0000000000000000 trailerW1 001pLLLLBBBBBBBB 1FFFFCCCCCCCsfx1 Header+2 hitW2 1FFFFCCCCCCCxUx0 010zhvxxxxxxxxxx 1 hit+trailer

Raw data example W1 001pLLLLBBBBBBBB 011NNNxxBBBBBBBB Header+8bitW2 011NNNxxBBBBBBBB 010zhvxxxxxxxxxx 8 bit+trailer

Expanded example W1 001pLLLLBBBBBBBB 1FFFFCCCCCCC0DDD Header+1 hitW2 1xxxxxxx1DDD1DDD 1xxxxxxx0xxx1DDD 2 hit +1 hit W3 010zhvxxxxxxxxxx 0000000000000000 trailer

Condensed to raw on errorCorrectW1 001pLLLLBBBBBBBB 1FFFFCCCCCCCsfx1 Header+2 hitW2 1FFFFCCCCCCCxfx0 010zhvxxxxxxxxxx 1 hit+trailerError on 1 before 3rd hit (could have done it before 2nd hitW1 001pLLLLBBBBBBBB 1FFFFCCCCCCCsfx1 Header+2 hitW2 011nnnxxWWWWWWWW 010zhvxxxxxxxxxx raw+trailer

Format will switch to raw on the following errors for the rest of the event.0 after beam crossing numberfirst bit of hit data leader is 1bit before hit pattern is 0

Formatter

8/20/2002 14

39

Link or Diagnostic

Data to EFB

39

Input FIFO

4k words

39

39

Data

DataInput FIFO

4k words

Data Input from Formatter

R/W Bus Data


Control FIFO Operation

ADV OE

48 bits are read as 32 bit word bits 0:3116 bit word bits 32:47

Advance FIFO and latch new input

Enable FIFO output data

Read or write data

Select 16 or 32 bit part of word

Retransmit FIFO

Debug FIFO Implementation

FIFO O

ut R

egister

FIFO In

Register

FIFO D

ata R

egister

FIFO D

ata R

egisterADV OE RT

OE

OE

8/20/2002 15

• Debug Diagnostic Memories.– Play or record data to perform diagnostics and tests.

– Start of record operation is triggered by the ROD Controller.

– Trapping (record) of Formatter Output Data can be synchronized to triggers.

– The SCT and Pixel ROD implementation of the Debug Diagnostic Memories is two groups of 4k x 39 bit FIFOs.

Debug FIFO Notes

8/20/2002 16

39

Control, Status,

Event Data /

Trig Type

43

43

16Kx46 OutputFIFO

32

Output Control

EV Fragment Header/Trailer

Event Data Valid

Router Halt Output

Almost Full

Almost Full

Data

43

Halt Output

Data Valid

Format #

R/W Bus

To/

Fro

m R

oute

r F

PG

A

To/

Fro

m C

ontr

olle

r F

PG

A

EFB Data Engine 1Event ID checkingError CountersFormat event

39Data

Halt Output

Data Valid

Format #

DerandomizingFIFO 4k x 16b

Header FIFO 64w x 32b

Engine FIFO 32w x 107b2 ea.

EFB Data Engine 2Event ID checkingError CountersFormat event

Event Data

Event Data

Output Control

ErrorSummary

Word counts

Header/trailer EventFragment Generation

Data Output Control

16Kx46 OutputFIFO

Read Event

Dyn Mask &L1-BCO

Data Type

Event Fragment Builder (EFB)

8/20/2002 17

. Provides two parallel streams with data processing engine from half of the input links to mask the Formatters link-to-link latency.

– Each engine has a 16k word output FIFO for derandomizing of data and to hold large events.

– The two data processing engines helps achieve 40 MWords/sec bandwidth to the S-Link. . If “Router Halt Output” is issued, data can still flow through the EFB and fill the output memory.

. If the output memory is full back pressure will be applied to the Formatter.

. EFB checks Event ID (BCID and L1ID) and generates errors. The BCID error checking can be disabled by setting a bit in an EFB Internal Register.

. Performs summary error counts with mask Counting of errors can be masked on a link by link basis by writing the appropriate value to the Error Mask Registers in the EFB.

. The EFB formats input data event for the S-Link.

. Generates S-Link compatible headers and trailers. The information for this step is sent to the EFB from the Controller over the Dynamic Mask Data bus. The Event ID and Dynamic Mask for a specific Event must be written to the EFB before a Token can be issued to the Formatter to read out the link FIFOs.

EFB: Overview

8/20/2002 18

• The Event ID,Trigger Type, ECR, and Dynamic Mask are sent to the EFB from the Controller. This data arrives after TIM has transmitted an L1 trigger, Serial ID and Trigger Type data to the Controller and before a FE Command Pulse can be sent to the Formatters.

• The Event ID and Dynamic Mask is data set of 16 half words. The data format is shown on the next slide.

• The EFB stores the 16 16bit words per event in a 4kword x 16 FIFO, transfers this group of data to 3 FIFOs used for Event Header generation, L1ID/BCID checking, and synchronization correction.

• The Event Header FIFO is 16 words by 64 bits, stores the L1ID, BCID, ECR, and Trigger Type, and is used to build the Event Headers when the EFB is ready to send an event to the Router

• Each EFB engine require a Dynamic Mask FIFO. This FIFO is 16 words by 64 bits, stores the L1ID, BCID, and the Dynamic Mask bits required by each link for correction.

• The Event Header and Dynamic Mask FIFOs are ready for data processing 20 clocks after the EFB receives the full data set from the Controller.

EFB: Event ID and Dynamic Mask

8/20/2002 19

Word Name Engine FIFO Header FIFO Word 0 L1 ID [15:0] [3:0] yesWord 1 ECR ID [7:0] & L1 ID [23:16} no yesWord 2 BC ID [11:0] & 0000 [7:0] yesWord 3 TIM EvtType [1:0] & TrigType [7:0} no yesWord 4 ROD Event Type [7:0]/(spare bits) [15:8] no yesWord 5 Dyn Mask [7:0] yes noWord 6 Dyn Mask [15:8] yes noWord 7 Dyn Mask [23:16] yes noWord 8 Dyn Mask [31:24] yes noWord 9 Dyn Mask [39:32] yes noWord 10 Dyn Mask [47:40] yes noWord 11 Dyn Mask [55:48] yes noWord 12 Dyn Mask [63:56] yes noWord 13 Dyn Mask [71:64] yes noWord 14 Dyn Mask [79:72] yes noWord 15 Dyn Mask [87:80] yes noWord 16 Dyn Mask [95:88] yes no

Dynamic mask 95:0 is 2 bits per link, used to correct module synchronization. DM Bits [1:0] “00” no change to L1 ID for this event DM Bits [1:0] “01” Increment L1 ID for this event DM Bits [1:0] “10” Decrement L1 ID for this event

ECR = Event Counter Reset, TrigType = ATLAS Trigger Type

EFB: Event ID and Dynamic Mask Data Format

8/20/2002 20

3943

Data Link 1-48

BCID / L1ID

Check

Error Detect, Count

Format Data & Count

Halt OutputOutput FIFO Almost Full

Output FIFO Data

Error Summary Datalnput bits to error summary Header bit error (0) Trailer bit error (1) Flagged error (2) Sync error (3) Hit pattern error (4) L1ID error (5) BCID error (6) Timeout error (7) Almost full (8) Data overflow (9)If summary AND Mask =TrueCount = Count +1Summary Error = Summary Error | current Error

Data Valid

Format #

Add or subtract from L1 number

Dynamic Mask &L1-BCO

Mode bits add or sub 1

from L1ID

L1IDand

BCO

R/W Bus

Mask (96 ea.) Summary and Count to Error FIFO

Word count to WC FIFO

EFB: Data Engine - 1 Engine Shown

WR Output Data

Mask BCID Check

8/20/2002 21

• Correct L1 ID Number

Based on the Dynamic Mask bits, the L1 ID increments or decrements. This is used when a link is resynchronized on the ROD without stopping ATLAS triggers.

• BC ID / L1 ID Check:For Link Header, compare expected BCID and L1ID with the decoded data and flag any errors. All other data types are ignored.Separate Event FIFOs provide L1/BCID data so each EFB data path can work independently on different events.BCID check can be globally disabled if required.

• Error Detect:See EFB Data Engine above for error bit definition. Looks at data bit fields to identify errors.Generates the error summary block for the event.One maskable error counter per link.One global error counter BCID errors can be ignored for a link that has been resynchronized

EFB: Data Engine

8/20/2002 22

• Error Bits Summary in the Link Error Mask Register:

Bit(0) => Header bit errorBit(1) => Trailer bit errorBit(2) => Flagged errorBit(3) => Sync errorBit(4) => Hit pattern errorBit(5) => L1ID errorBit(6) => BCID errorBit(7) => Timeout errorBit(8) => Almost fullBit(9) => Data overflow

• Format Data and Count:1. Removes front end trailer words unless the trailer error flag or the almost

full flag was set.2. Removes header from links with no hit data and no errors.3. After unnecessary half words are removed, it packs all the data into 32 bit

words so there are no gaps in the data.4. Counts number of words sent to the output FIFOs over an event and

writes the count into a FIFO.

EFB: Data Engine

8/20/2002 23

Header/Trailer Event Fragment Generation

Gather Error Summary, Word Counts and Event ID data and raise Event Data Valid

Then Control Event Readout

Halt output suspends readout

32

Event Header Data

FIFO:Error Summary Data

FIFO:Word Count

FIFO: Header Event ID Data

Router Halt Output

FIFO:Word Count

FIFO:Error Summary Data

Engine 1Engine 1Engine 1 Engine 2

Event Trailer Data

32

Event Data Valid

Data Type

To\from

Rou

ter

Data FIFOs Output Control

3

EFB: Header/Trailer Generation

8/20/2002 24

• Header/Trailer Block Generation, Data Output Control:– After the data counts are available from both EFB engines, an event is

delivered to the Router then to the S-LINK (data is pushed).• A complete event must processed and written to the output FIFOs

before Event Data Valid is true.• An Event Header block is first transmitted to the S-Link.• Data is then transmitted from the output FIFOs. • Finally, the summary and event fragment trailers are transmitted.

• Pause:– If The DSP asserts Stop (calibration mode only) or the S-Link asserts Xoff,

data will not be transmitted, but the EFB can still process data and write to the output memories.

– If any of the internal EFB FPGA FIFOs are almost full or the output FIFOs are almost full, the FIFO controller will be told to pause and back pressure will be applied to the Formatters.

– Output FIFOs are big enough to hold largest SCT/Pixel Event.

EFB: Header Generation

8/20/2002 25

Event HeaderWord Contents Comment

0 0xB0F00000 (beginning of fragment)1 0xEE1234EE (start of header)2 0x9 (header size)3 TBD (format version number)4 TBD (source identifier)5 L1 ID (level 1 ID)6 BC ID (bunch crossing ID)7 L1 TT (ATLAS Level 1 trigger type)8 DET (detector event type ROD or TIM)

Event TrailerWord Contents Comment

0 Error count (Status 1: count of words with error)1 Error flags (Status 2: Bit error type occurrence)2 0x2 (Number of status words)3 nData (word count of data words)4 0x1 (status block position: 0/1 = before/after data)5 0xE0F00000 (end of fragment)

EFB: Output Format

8/20/2002 26

EFB Output (bits 31:0 bits for all words)Name Bits 15:0 or 31:16header 001pLLLLBBBBBBBBtrailer 010zhvxxxxxxxxxx1 hit condensed 1FFFFCCCCCCCxfx02 hits condensed 1FFFFCCCCCCCsfx11st hit clust expan 1FFFFCCCCCCC0DDD1 hit clust exp 1xxxxxxx0xxx1DDD2 hit clust exp 1xxxxxxx1DDD1DDDflagged error 000xxxxxxFFFFEEEraw data 011nnnxxWWWWWWWWclust=cluster, exp=expanded

Key :x= do not care (The ROD fills these with 0’s)B=BCIDW=raw data C=cluster base addressD=3 bit hit dataE=ABC error codeF=FE numberh=header trailer limit errorL=L1IDn=count of raw dataf=error in condensed mode data, 1st hits=error in condensed mode data, 2nd hitp=preamble errorz=trailer bit errorv=data overflow errort=time out errorK=condensed model=L1 errorb=BCID errorM=link number

.Link Data Format SCT all data is in 16 bit packets packed in 32 bit words bit 31:0 Data bit 42-32 are valid for header word only bits 42:32 blktMMMMMMM

EFB Output examples for two eventsBits 42:32 Bits 31:16 Bits 15:0 Event header 9 words (Normal event) blKtMMMMMMM link header CondensedblKtMMMMMMM Condensed link header Note:link trailers Condensed condensed are removed on noblKtMMMMMMM link header condensed trailer error, link condensed condensed by link event trailer 6 words

Event header 9 words (a trailer error) blKtMMMMMMM link header CondensedblKtMMMMMMM Condensed link header Condensed condensed blKtMMMMMMM link trailer link header Note:error in trailer condensed condensed condensed link trailer Note:special trailer

fill event trailer 6 words

If there are no errors in trailer and header and no data the link header is not written.

EFB: Output Link Format

8/20/2002 27

Router Implementation: Top Levelevent_info_in(45:0)

tristate_n_in(2:0)

data_valid_in(1:0)

slink_xoff_in

router_bus_data_inout(15:0)

slink_bad_in

slink_data_out(31:0)

slink_wen_out

slink_rst_out

dsp0_event_data_out(31:0)

dsp0_dataframeready_out

dsp3_event_data_out(31:0)dsp3_ dataframeready _out

slink_test_out

stop_output_out

dsp0_sbsram_clk_indsp0_send_data_in

dsp0_mask_dbr_in

dsp0_sbsram_me_in

router_bus_addr_in(7:0)

router_bus_rnw_in

router_bus_strb_n_inrouter_bus_ack_out

dsp1_cntrl_in(3:0)

dsp2_cntrl_in(3:0)

dsp3_ cntrl_in(3:0)

clk40_in

rst_n_in

Register Block

Write link and error info into Link Headers, Format data for S-Link and Error Checking

S-Link Write Control

46b

slink_cmd_reg(15:0)

Stop Output

slink_uctrl_out

DSP FIFO Buffer

Trap Event Datatrap_event_type_reg(31:0)

DSP 1

DSP 2

DSP 3



DSP 0

router_bus_registers

header detect

Error Format Data

slink_data_out(31:0)

dsp0_trap_enable_in

stop_out_override

stop_out_override

8/20/2002 28

43

Sli

nk o

n B

OC

Control & Status for Block Xfer/ DMA / S-Link

DM

A 1

DSP Error Data

DM

A 4 DMA

Cont....

S-link DataEvent Header Data

Event Trailer Data 32

Event Data Valid

Data Type

Output FIFO Data

32Read event data and format for S-Link and DSP error data.Detect Errors in the Link Headers, and create error code words for the Error Format.Detect Event Headers to enable trapping algorithms

R/W Bus

4 DMA transfer engines. Each engine has a 1k 32b word derandomizing buffer. Each buffer is broken into 256 word blocks.DMA moves full blocks of data. The blocks are filled as shown below:small event word count & continue bit in last word large event word count & continue bit in last word exactly one block word count & continue bit in block 2 last word

4 event trapping filtersFilters can trap on event types ATLAS, TIM , ROD or error dataEach trap has a divided by factor that will reduce data to DSPs

Header Detect

Router: Data Format and Trap

8/20/2002 29

. Write link and error information into the Link Headers for the S-Link Data Format.. When a link header is detected in the data, the Router moves the link number and error bits information from bits 45 down to 32 and overwrites the L1ID and BCID fields. (Shown on a later slide)

. Write to S-Link if Data Valid is true, and the User has not set the S-Link Masking bit or setup to mask specific Events.

. Detect errors in the Link Headers, and generate an error code for each link in Error Data Format. (Shown on a later slide). The error codes are stored in a block of registers for the duration of the Event. When the data

from EFB Engine 2 is done in the Router, the Error codes are written into the Event Trailer

Two bits error codes for each link00 = no data01 = good data10 = BCID or L1ID error11 = time out error

. Router FPGA routes data to the S-Link and the Slave DSPs.

- Internal FPGA dual port memory are used to asynchronously connect to DSPs.- During runtime monitoring, data is DMA’d into the internal SRAM of each DSP at

80 Mwords/sec. This channel must have the highest DMA priority or errors will be seen in the transfers.

• DSP captures the data it needs, throws away the rest.

Router: Overview

8/20/2002 30

Router: Overview• The Register Block allows VME access to the Router over the ROD Bus. The registers

in the Router include control and status for the DSP trapping algorithms and the S-Link.

• Because the Router needs to monitor header words 8 and 9 to sense an Event to trap, the Router uses a 9 clock pipeline for the Event Data path.

• If the S-Link asserts Xon/Xoff or S-Link Bad, the Router will mask the S-Link Write Enable to stop data transmission to the S-Link.

• In Calibration Mode only, if the DSP FIFO is almost full, and the StopOutputOverride bit is set, the Router can assert the StopOutput signal and temporarily apply back pressure to the EFB and Formatters. The Router can not apply back pressure from the DSPs to the ROD data path in Normal Data Taking Mode.

8/20/2002 31

• The Router supports four sets of trapping filters, one set per DSP. Each filter set is comprised of two filters per DSP.

• The first filter per DSP takes priority over second filter. Each filter has a mask bit to select a specific type of event (ATLAS, TIM, ROD).

• The first filter can be set to trap all events which disables the second filter.• Each filter has a Trap Match (8 bits per filter) value used to match the event type.• Each filter has a divide by factor (8 bits per filter) which allows pre-scalable trapping.• The trap variables must be loaded by setting the Load Trap Bit in the Reset/Load Trap Register.

This is done to protect from changing the trap type while the Router is in the middle of trapping an event.

• Each DSP DMA port has a bit to select between S-Link data format or error data format.

• Each DSP must set the Trap Enable bit to enable the Event trapping. Any the time that the Trap Enable bit is cycled from 0 to 1, the trap is reset and the FIFO counters are reset.

• The Mask Data Frame Ready input to the Router masks the Data Frame Ready signal until the DSP ISR is complete. If this bit stays high for an extended period, the FIFO will become full, and DSP data will be lost. This does not affect the Real Time Data Path signals

• When the an event has been trapped, a special event completion word is written to the last location of the FIFO block (256 words per block) that the event occupies. This word is made up of a pattern that does not appear in the data, and the total word count of the event. If an event end exactly on a block boundary, the word is written to the end of the next boundary.

Router: DSP Trapping Algorithm

8/20/2002 32

The format for the module link header is changed. The L1ID and BCID is replaced with error codes and the link number as shown below:

Key :x= do not care (The ROD fills these with 0’s)B=BCIDW=raw data C=cluster base addressD=3 bit hit dataE=ABC error codeF=FE numberh=header trailer limit errorL=L1IDn=count of raw dataf=error in condensed mode data, 1st hits=error in condensed mode data, 2nd hitp=preamble errorz=trailer bit errorv=data overflow errort=time out errorK=condensed model=L1 errorb=BCID errorM=link number

EFB Output (bits 31:0 bits for all words)Name Bits 15:0 or 31:16header 001pLLLLBBBBBBBBtrailer 010zhvxxxxxxxxxx1 hit condensed 1FFFFCCCCCCCxfx02 hits condensed 1FFFFCCCCCCCsfx11st hit clust expan 1FFFFCCCCCCC0DDD1 hit clust exp 1xxxxxxx0xxx1DDD2 hit clust exp 1xxxxxxx1DDD1DDDflagged error 000xxxxxxFFFFEEEraw data 011nnnxxWWWWWWWWclust=cluster, exp=expanded

Bits 42:32 blKtMMMMMMM

Router Output (bits 31:0 bits for all words)Name Bits 15:0 or 31:16header 001ptlbKxMMMMMMMtrailer 010zhvxxxxxxxxxx1 hit condensed 1FFFFCCCCCCCxfx02 hits condensed 1FFFFCCCCCCCsfx11st hit clust expan 1FFFFCCCCCCC0DDD1 hit clust exp 1xxxxxxx0xxx1DDD2 hit clust exp 1xxxxxxx1DDD1DDDflagged error 000xxxxxxFFFFEEEraw data 011nnnxxWWWWWWWWclust=cluster, exp=expanded

ReplacedL1ID andBCID in Event Header

Router: S-link Module Output Data Format

8/20/2002 33

The format for the header is not changed except for the addition of UCTL from the EFB input. The format is shown below:

Event HeaderWord Contents Comment

0 0xB0F00000 + UCTL (beginning of fragment)1 0xEE1234EE (start of header)2 0x9 (header size)3 TBD (format version number)4 TBD (source identifier)5 L1 ID (level 1 ID)6 BC ID (bunch crossing ID)7 L1 TT (ATLAS Level 1 trigger type)8 DET (detector event type ROD or TIM)

Event TrailerWord Contents Comment0 Error count (Status 1: count of words with error)1 Error flags (Status 2: Bit error type occurrence)2 0x2 (Number of status words)3 nData (word count of data words)4 0x1 (status block position: 0/1 = before/after data)5 0xE0F00000 + UCTL (end of fragment)

Router: S-link Event Header Output Format

8/20/2002 34

One of the slave DSPs is concerned with error diagnostics. It will need all error flags, module numbers and L1ID and BCID fields. The event header format is changes as shown below:

Router error data output same asEFB Output (bits 31:0 bits for all words)Name Bits 15:0 or 31:16header 001pLLLLBBBBBBBBAll other bits are also the same

EFB Event Header DSP Error HeaderWord Contents Word Contents Comment

0 0xB0F00000 0 same (beginning of fragment)1 0xEE1234EE 1 same (start of header)2 0x9 deleted (header size)3 TBD deleted (format version number)4 TBD deleted (source identifier)5 L1 ID 2 same (level I ID)6 BC ID 3 same (bunch crossing ID)7 L1 TT 4 same (ATLAS Level trigger type)8 DET 5 same (detector event type ROD or TIM)

EFB Event Trailer DSP Error Trailer CommentWord Contents Word Contents0 Error count 0 same (Status 1: count of words with error)1 Error flags 1 same (Status 2: Bitter error type occurrence)2 0x2 2 value (error status for links 1-16)3 nData 3 value (error status for links 17-32)4 0x1 4 value (error status for links 33-48) 5 value (error status for links 49-64) 6 value (error status for links 65-80) 7 value (error status for links 81-96) 5 0xE0F00000 8 same (end of fragment)

Two bits error codes for each link00 no data01 good data10 BCID or L1ID error11 time out error

Router: Error Data Output Format for the DSP

8/20/2002 35

Back End DSPs

• The Back End DSPs are TI TMS320C6701, floating point processors. The CPU clock is 160MHz, and each DSP has 32MBytes of SDRAM.

• The DSPs are removed from the data path allowing them to work independently without affecting the ROD data taking

• The VME host can access the Slave DSPs through a 16 bit Host Port Interface.

• SDSPs are hardwired for HPI Boot and Little Endian byte order.

• The interface to the Slave DSP is through the Master DSP to the ROD bus.

• The Slave DSPs respond to D32 block transfers from the VME master. (2.2Mbytes/S)

• The back end DSP performs histogramming, spying, and event trapping tasks.

– Calibrations: raw data, histograms, summaries to reduce VME traffic and load on crate processor, or comparison to reference histograms.

– Can trap specific events, i.e. embedded calibrations.

• Uses Trigger Type Info.

8/20/2002 36

TIM Data

Token and Mode Bits

Dec. Trig Count

R/W Bus to

FPGAs, BOC,

FIFOs, SDSPs

FE Command Streams48

Busy to TIM

Formatter

EFB

Input Memory & XCVRControl

ROD Controller: Implementation

ROD Controller

FPGAMaster DSP

TMS320C6201HPI Bus (VME)

EMIF Bus

ASYNC

DSP Serial Ports(1:0)

Interrupts

Event ID/ Dynamic Mask

FIFO Occupancy Status

Event ID Error

BOC

TIM

GPI/O PRM FPGA

8/20/2002 37

ROD Controller: Implementation

Test Bench Interface

Rod Controller FPGA:

RodController DSPmodule: SPI TCUTMS320C6201

HPI

EMIF

VMEbusIInterface:

FPGA Program Manager:Reset Manager:

RodResourcesInterfaceProgramBits 1b

RodResourcesInterfaceProgramBits, rst 5b

AFormattersProgramBits, rst 5b

BFormattersProgramBits, rst 5b

EventFragmentBuilderProgramBits, rst 5b

RouterProgramBits, rst 5b

EvntFrgBldrBUS 16b

FormatterABUS 16b

RouterBUS 8b

BOCsetupBUS 8b

busBuffer:

busBuffer:

busBuffer:

busBuffer:

busBuffer:

INMEM R/W & Control 32b

FormatterTrailerInformation 12b

Formatter Readout Mode Bits 16b

FE Command Streams 48b

EFB Dynamic Mask 12b

ControlREG,DebugMEMInterface:

busBuffer:DEBUGMEM R/W & Control 32b

FrontEndCommandEncoder:

FormatterReadoutModeBitsEncoder:

EvntFrgHeaderAndControlEncoder:

TIM 8b

busBuffer:

busBuffer:

busBuffer:

busBuffer:

BackEndDSP0 16b

BackEndDSP1 16b

BackEndDSP2 16b

BackEndDSP3 16b

BOOT Strap:PROM

Interrupt 1b

16b

32b

2b

DSP MEMORY:16Meg Bytes SDRAM

busBuffer:EVNTMEM R/W & Control 32b

En, AckR/NW

4b

Front Panel Status/Indicator LEDs 8b

32b

8b

32b

Interrupt 1b32b

DSP CE1 Mtype=000b rom

DSP CE2 Mtype=011b sdramDSP CE3 Mtype=011b sdram

BootMode=01101xBootProcess=8bitROM defualt timings MemoryMap=Map1

FrontEndOccupancyCounters:

AddressDecoder:

FormatterBBUS 16bbusBuffer:

dspReset[4..0] 5b

32b

32b

DSP CE0 Mtype=010b async

8/20/2002 38

• The Master DSP is a TI TMS320C6201B, fixed point processor. The CPU clock is 160MHz, and the DSP has 16MBytes of SDRAM (upgradeable to 32MBytes).

• The VME host can access the Master DSP through a 16 bit Host Port Interface.

• The Master DSP is hardwired for 8bit ROM Boot and Little Endian byte order.• The Boot ROM can be accessed on the VME Bus so that new versions of code can be

downloaded and started when required by the DAQ system.

• The Master DSP responds to D32 block transfers from the VME master. (2.2Mbytes/S)

• The Master DSP provides the main path of communication to the ROD from the VME through the Host Port Interface.

• The ROD Bus, controlled by the ROD Controller, is connected to the Master DSP EMIF asynchronous port, and utilizes CEn memory space 0. This is the path for access to all of the data path FPGAs, the Slave DSPs, the diagnostic memories, and the BOC.

• The Host Port Interrupt out from the DSP is wired to a VME interrupt input.

• The Master DSP has 2 serial communication ports (SP0 and SP1) that are used in the ROD to send serial commands to the FE Modules

ROD Controller: Master DSP

8/20/2002 39

• The Master DSP allows high level control operations to be performed on the ROD.– Because the control path of the ROD is not required to operate in real-time,

performance is less critical. – Using Primitive Lists, the MSDP performs control of ROD functional setup (the

specific mode of the ROD), histogramming, spying, and event trapping.

ROD Controller: Master DSP

• When the periodic reset is issued, the ROD raises busy and interrupts the master DSP. The master DSP then waits for all events to be played out of the front end chips by monitoring that the counts of all events pending is zero. Configuration as needed is then performed within approximately 1 thousandths of a second. Near the end of the cycle, the reset is issued to the modules and busy is removed.

• It is also possible to continue triggers while the ROD is correcting the state of the front end modules. In this mode busy would not be raised while the modules are configuring. Triggers would just return empty event with an error flag.

• The exact details of the periodic reset if needed will have to be worked out with ATLAS Trigger/DAQ.

Periodic Reset

8/20/2002 40

TIM Data

Event ID Data

Trig TypeDynamic

Mask FIFO

Bac

k-of

-Cra

te C

ard

Tok

en a

nd M

ode

Bit

s

Dec

. Tri

g C

ount

Back End DSPs

Host Port Interface (HPI) Bus

Master DSP r/w

R/W bus to

FPGAs and BOC

R/W Control

Registers

r/w

Control links 48 48

2 inputor

Circuits

Busy to TIM

48 bitMask

& Inputbit by bit

48 bitMask

& Inputbit by bit

Serial ExpansionEvent Data

TIM FIFO

4k W 8b

48

48

ToFormatter

Trigger OperationControl

ROD BCID, L1ID and Trigger Type

in ROD based triggers

To EFB

DSP SerialPort 0

DSP SerialPort 1

r/w

AllZero

Generate Dynamic

Mask

To data RCVR

Input Memory & XCVRControl

964 bit

Counters

Decr

Incr

FromFormatter

r/w

r/w

r/w

FromFormatter

ROD Controller FPGA

8/20/2002 41

• The main function of the ROD Controller FPGA is to control the operation of the ROD and coordinate the flow of data on the board in real time.

• Listing of the real time functions:1. Process commands from the TIM: Trigger, fast and slow commands and calibration signals2. Process serial data from the TIM: Event ID and Trigger Type.3. Count and store ECR commands, and include the count in the Event Header in the L1ID field.4. Send FE Commands to the Modules.5. Supply the Formatter Mode Bits and the EFB Event ID and Dynamic Mask Bits to the Data Path when an

L1 trigger has been received and the appropriate data has arrived from the TIM.6. Count and store triggers, and issue them to the Data Path when ROD is ready to process an Event.

• Listing of quasi real time functions:1. Issue and count corrective triggers sent to the data path.2. Issue Corrective Mode Bits for an Event.3. Issue a corrective Dynamic Mask and transfer it to the Default Mask.4. Change the masking of FE Command Streams.5. Interrupt the Master DSP to indicate the type of trigger sent to the data path.6. Trap Configuration or Event data in the Input Diagnostic FIFOs.

• Listing of non real time functions:1. Read/Write to the internal memory mapped register set.2. Read/Write to the BOC, diagnostic FIFOs, and the Slave DSPs.

ROD Controller: FPGA

8/20/2002 42

• The controller has an internal test bench block and a set of diagnostic registers to allow the ROD to be tested in a stand alone mode. This is required for initial debugging and some system testing.

• Listing of the Test Bench functions:1. Internal FIFO to mimic the TIM inputs2. Separate states to run stand alone test on the ROD.

A. Trap all Link inputs to Inmem FIFOsB. Trap all Link inputs to Inmem, send to Formatters, trap Formatter Output to Debug FIFOs, play data through the ROD

to the SDSPs (Event ID data must be provided by a TIM or internal registers) C. Load Inmem and TIM FIFOs with simulation data files and play through the RODD. Load Inmem and TIM FIFOs with simulation data files and play through the ROD with a set number of retransmits (1 to

65536).E. Load Inmem FIFOs with simulation data files and trap in the Debug FIFOsF. Load Debug and TIM FIFOs with simulation data and trap in the Event FIFOsG. Load Event FIFOs with 1 Event of simulation data and trap in the Slave DSPsH. Load Inmem FIFOs with 1 Event of simulation data, set the internal Event ID, Mode Bits and Dynamic Mask registers

to the required values, send L1 Triggers over SP0 or SP1, trap in the SDSPs

• Event ID FIFO stores a list of expected trigger IDs that the EFB can check against.

• Process the DSP Serial ports to provide for module configuration and ROD based trigger information.


8/20/2002 43

• The controller has a memory mapped set of internal registers required for all ROD operations.

• Listing of the internal registers by functional group:1. Main control and status2. FE Command Mask settings3. Calibration Commands4. Formatter Mode Bits/EFB Event ID and Dynamic Mask control and status5. Formatter Mode Bits /EFB Event ID and Dynamic Mask Look-up Tables6. Diagnostic control, status and counters7. DSP interrupts8. FE Occupancy Counter Control and Status9. Diagnostic Event ID, L1ID and BCID values10.Corrective Events FIFO access

• Can execute a long command sequence without tying up the ROD Crate Controller.

• Keeps count of the number of pending triggers to sense all events processed and for diagnostics. All events processed is used in a periodic reset to sense that the data is not in the front end chips.

• The ECR Counter can only be reset by writing a 1 to the ECR reset bit in the RCF Spare Control Register


8/20/2002 44

• Normal Data Taking

• The ROD receives L1A, Event ID, and Trigger Type signals from the TIM.

• With a latency of 2 clocks after receiving the L1A, the ROD sends the L1 Trigger command to the FE modules on the output links that are active.

• The FE Trigger Counter and the FE Occupancy Counters are incremented by 1.

• The Formatter Mode Bits Encoder sends the default Mode Bits to the Formatters.

• The EFB Event ID Dynamic Mask Encoder receives the deserialized ID and Trigger Type data, processes and sends the Event ID and default Dynamic Mask data to the EFB.

• The Token is issued to the Formatters, the FE Trigger Counter is decremented by 1, and an interrupt is sent to the Master DSP to indicate that a default trigger has been processed by the ROD Controller.

• The FE Occupancy Counters are decremented when a Link Formatter writes a trailer into the Link FIFO.


8/20/2002 45

• Resynchronization of modules during data taking

• Slave DSP determines what link requires L1 ID adjustment. The information is transferred to the Master DSP.

• The Master DSP writes the correction information to the Corrective Mode Bits and Corrective Dynamic Mask Registers in the ROD Controller FPGA then sets the “New DM Mask Ready” bit in the RCF Main Control Register.

• When the next L1A is detected from the TIM, the ROD performs the steps listed in the Normal Data Taking description with the following exceptions:

• The Corrective Trigger is trapped in the Corrective Events FIFO.

• The Mode Bits and Dynamic Mask Encoders issue the corrective Mode Bits and Dynamic Mask, and send an interrupt to the Master DSP to indicate that a Corrective Mask has been sent. This mask is valid for 1 L1 Trigger only for the Formatter. It is permanent for the EFB. (This feature is in the VHDL, but not tested)


8/20/2002 46

• FE triggers from Master DSP Serial Streams

• The ROD Controller FPGA contains a BC counter, L1 counter, and Event Type Register per serial stream. When the Calibration Trigger Decoder block detects a L1 Trigger from either serial stream, the L1 counter is incremented, the value of the BC counter is latched into a register, and the Event Type Register value is written to the Event ID FIFO.

• The Default Mode Bits and Dynamic Mask Look-up tables are used when a trigger is detected on SP0

• The Corrective Mode Bits and Dynamic Mask Look-up tables are used when a trigger is detected on SP1

• Fast Reset Commands will reset the counters as specified. A BCR command will reset the ROD BC counter and the FE module BC counter simultaneously, an ECR will not produce this result because the BC reset happens after other resets are performed by the chip controller.

• The BC ID register can be set to a static value.

• This mode is used for triggers during histogramming and diagnostics.


8/20/2002 47

ROD Program Reset Manager: Implementation

Flash Interface

FPGA Configuration

Data

FLASH 12MBytes

90ns

FPGA Config Control

ID(7:0)

FPGA Cnfg/RST Controller

Internal

R/W Registers

VME Address Decoder

and Data

Bridge

IA(18:0)

Control

ID(7:0)

ID(7:0)

FPGA Config Status

IA(20:0)

FPGA Reset Control

DSP Reset Control

Control

Status

D(31:0)

A(31:0)

Master DSP

Control

Control

AckN

D(31:0)

A(31:0)

Control

AckN

To VME

D(31:0)

Control

HR

DY

N

HHWIL

HD(15:0)

HCNTRL

HDS1

To MDSP

HPI

HRDYN

To FPGAsGP I/O

To ROD Controller

8/20/2002 48

ROD Program Reset Manager: Overview• Controls power on sequence of the ROD by configuring the data path

FPGAs in a specific order, and controlling resets to the DSPs.

• Allows VME host to configure and reset Data Path FPGAs.

• Allows VME Host to reset all DSPs.

• Allows VME Host to monitor status of the configuration status of Data Path FPGAs

• Acts as a bus-bridge between the ROD and the VME Bus.

• Enables D32 transfers between the VME and the Master DSP (16 bit HPI)

• Controls the read and write access to the 12 Mbytes of Flash memory that stores the FPGA configuration data.

• Stores the specific Serial Number of each board

8/20/2002 49

ROD Program Reset Manager

• Power on sequence of the ROD: • Configure ROD Controller FPGA

• Activate Master DSP

• Configure Formatters, EFB and Router

• Active Slave DSPs

• VME to Master DSP D32 access:

• Write Access: The PRM receives a D32 data word from the VME bus, splits it into 2 half words and sends the first half word to the HPI data port. After HRDY_N indicates that the HPI is ready for the second half word, the PRM toggles the half word ordering bit, moves the upper word to the lower half of the bus and writes it to the HPI data port. At this point, the PRM asserts LACKN to the VME Chipset, and is ready for the next transfer.

• Read Access: Same as above functionally.

8/20/2002 50

ROD Program Reset Manager• Controls the configuration and reset of Data Path FPGAs, resets of all DSPs, and status of

configuration and reset signals

• Writes to the Configuration and Reset registers on the VME bus allow the configuration and reset of all of the Data Path FPGAs and DSPs

• Reads to the Status registers show the state of configuration and reset signals

• Controls the read and write access to the 12 Mbytes of Flash memory that stores the FPGA configuration data.

• Internal registers and algorithms allow for simple access to the Flash memory.

• The Address and Write Data register is used to set the memory address for writes and reads and the data to write if required

• The Read Data register stores the data from the last read to the Flash

• The control register allows reads, host algorithm writes and PRM algorithm writes

• The host algorithm write gives complete control to the host to program the part. This is useful during chip or sector erase functions.

• The PRM algorithm write keeps control of the programming operation in the PRM. It is most useful for programming the part because it takes care of the repetitive commands required to program a Flash, and only requires address and data from the host.

8/20/2002 51

ROD Clock Distribution: Implementation

Clock Distribution

IC

1 In / 36 Out

PLL

BOC CLK+From BOC

PECL =>

LVTTL

40MHz XTAL OSC

BOC CLK-

SEL CLKTo

ROD36

To S-Link

S-LINK CLK

8/20/2002 52

ROD Clock Distribution: Overview

• Accepts the BOC clock PECL input, and converts to LVTTL to match the input of the Phase Locked Loop used on the ROD.

• Internal crystal oscillator can be used in stand alone mode

• Distributes separate clock lines for all parts that require clocking

Documents

Atlas SCT ROD FDR Implementation Model