Kchip: A Radiation Tolerant Digital Data Concentrator chip for the CMS Preshower Detector 9 th Workshop on Electronics for LHC Experiments Amsterdam, Sept

Kchip: A Radiation Tolerant Digital Data Concentrator chip for the CMS Preshower Detector

9th Workshop on Electronics for LHC Experiments Amsterdam, Sept. 30, 2003

KLOUKINAS KostasCERN, EP/CME-PS

Amsterdam, Sept. 30, 2003

KLOUKINAS Kostas EP/CME-PS 2

Outline

Preshower Front-End System

Kchip Architecture

Kchip Implementation

Prototype Test Results



Preshower Front-End System

DDU

Opt

ical

Rec

eive

rs

FPGADSP

Readout Path

FEC Module

TTCrx

TTCcimodule

Front-End Readout ASICs

Front-End Control ASICs

Control Path

Clk LV1I2C

DCUIV

I2CCCUCCUCCUCCUCCUCCU, PLL,

QPLL

Slow Control & Fast Timing SignalsRe

SubdetectorEvent Builder

K chip

FastTiming

Slow Control

FPGADSP

Rin

g co

ntro

ller

s

CLK & T1logic

TTCrx

VMEinterface

K chipK chip

K chipK chip

KchipPACEADC



Preshower Front-End Readout

Kchip

Si detector

CLKT1I2C

40MHzTo the next control chipsetFrom previous control chipset

DOHControl Chips

CCU, PLL, QPLLControl Token Ring

GOL

Readoutpath

Readout & Control

LogicTrigger commands

ADC_clk

Controlpath

GOH

PACE_clk

Column Addr. & Control

16

DeltaPACE-AM

x 32

800MbpsAD

AD

12

12

AD41240

PACE



Kchip Functionality

Data Concentration Can be configured to readout 1~4 PACE chips.

Event Data Formatting Align data into 16-bit words. Assemble an Event Packet. Assign a Bunch Count (BC) and Event Count (EC) Identifier. Link Protocol for transmission through a Gigabit Optical

Link.

Readout Controller Trigger Command Decoding PACE Readout Synchronization Monitoring Front-End Buffers Overflow Detection / Prevention PACE & ADC clock and Trigger Command Distribution



Kchip Block DiagramKchip

I2C_SCL

R e S y n c

ColAddr

Data_B

Data_C

Data_D

Co lAddr

ADC_B

ADC_C

ADC_D

TX_dataPacke t_Dat

a

Ev

en

tH

ea

de

r

Data_A

Co lumnAddress

F IFO

PACEControl ler

Tr iggerDecoder

Tr iggerHandler

I2C Block

PacketFormatter

GOL Interface

ADC_AB

ADC_A

De

-mu

ltiple

xe

rADC_CD

Clockand

Control

Da

taP

us

h

C o l A d d r _ P u s hColAddr_A

Co lAddr_B

Co lAddr_C

Co lAddr_D C o l A d d r _ F u l l

P L V 1

R e S y n c

C L K

ReSync_APLV1_APACE_CLK_A

ReSync_BPLV1_BPACE_CLK_B

ReSync_CPLV1_CPACE_CLK_C

ReSync_DPLV1_DPACE_CLK_D

CalPulseGenerator

R e S y n c

CalPu lse_ACa lPu lse_BCa lPu lse_CCa lPu lse_D

R e S y n c

L V 1

C a l P u l s e

B C 0

R e S y n c

C o l A d d r _ E m p t y

C o l A d d r _ P o p

D a t a _ P o p

T1

Tri

gg

er_

Em

pty T

rigg

er_

Po

p

R T S

C T S

R E A D Y

DAV / t x_enCAV / t x_er

R e S y n c

I2C_SDA

1 2

1 2

1 2

1 2

1 2

1 2

1 2

1 2

1 2

1 2

1 6 16

1 61 6

RE

SE

Tb

CL

K

P A C E _ F u l l

Error Logger

I2C_addr

2

ADC_CLK

Data FIFO

Data FIFO

Data FIFO

Data FIFO

DataVa l id_ADataVa l id_BDataVa l id_CDataVa l id_D

FIFO_Fu l l_AFIFO_Fu l l_BFIFO_Fu l l_CFIFO_Fu l l_D

EC

BC

L V 1

C L K

T r i g g e r _ P u s h

R e S y n c

D a t a _ F I F O _ F u l l

T r i g g e r _ F u l l

P A C E _ O u t _ o f _ S y n c

4

Tr iggerF IFO

Error F lags

to PACE

from PACE

from ADC

to GOL



Data Rates

3 columns/trigger32 samples/column

96 samples/trigger12-bit ADC100KHz Trigger Rate

14.4 MB/sec

• PACE generated traffic

• Gigabit Link Throughput (GOL chip) 80 MB/sec

• The difference in the readout times and the stochastic nature of trigger arrivals mandates the need of data buffering on the Kchip.

Traffic from 4 PACE chips = 57.6MB/sec

PACE event readout time = 6.9 μsecKchip event readout time = 7.8 μsec

80MB/sec

Controllogic

14.4MB/sec



Front-End System FIFOs

Trigger arrivals follow an exponential distribution. Kchip service time follow a uniform distribution.

An analytic queuing model is difficult to develop. A simulation model of the complete front-end system has

been developed.

P K

LV1

LV

1

PACE FIFOData & ColumnAddresses FIFO

Trigger FIFO

GOL

The 2 queues areserved together

Model of front-end system FIFOs.



Sizing the Kchip FIFOs

PACE FIFO can store up to 10 events. From simulations: Prejection= 1.9E-04 @ 100KHz trigger rate.

Kchip FIFOs: Kchip FIFOs should be sized for lower event rejection probability.

0

100

200

300

400

500

600

700

800

900

0,0E+00 2,5E+09 4,9E+09 7,3E+09 9,8E+09 1,2E+10 1,5E+10

Time [ns]

Oc

cup

ancy

[w

ord

s]

Data FIFO ColAddr FIFO Trigger FIFO

Time examined 15.102 s Number of events 1.5 106 Mean interarrival time of events 10.059 μs PACE rejected events 7 Kchip rejected events 0 Maximum Trigger FIFO occupancy 26 words Maximum Column FIFO occupancy 52 words Maximum Data FIFO occupancy 863 words Average Trigger FIFO occupancy 3 words Average Column FIFO occupancy 2 words Average Data FIFO occupancy 36 words



Implementation of Kchip FIFOs

FIFO Native Size of SRAM module

Actual Capacity

Data 1 Kword x 18 bits 10 events (1024/96)Column Address

128 words x 27 bits 10 events (matches Data FIFO)

Trigger 128 words x 27 bits 64 triggers (128/2)

128 x 27bits 1024 x 18bits

FIFOs are implemented using a “Configurable Dual-Ported SRAM macro cell”.

Two macro cells:

Column Addr. & Trigger FIFO 128 words x 27 bits

Data FIFO 1024 words x 18 bits



Packet Formatter

D a t a _ A

1 2

From/ to F IFOsFrom I2C

Block

C T S

D a t a _ B

D a t a _ C

D a t a _ D

D a t a _ A B

D a t a _ C D

D a t a _ B C1 2

1 2

1 2

1 6

1 6

1 6

C o l A d d r O u t

E v e n t H e a d e r

Da

ta re

alig

nm

en

t

T r i g g e r _B C

T r i g g e r _E C

1 61 6

1 6

1 6

1 4

Nul l_Data

K c h i p M o d e

t e s t l i n k

Statemachine

Co

lAd

dr_

Em

pty

Tri

gg

er_

Em

pty

Da

ta_

Po

p

Co

lAd

dr_

Po

p

Trig

ge

r_P

op

E r r o r F l a g s

8

R T SFrom/

to GOL

In te r fa

ce

From

I2C

Block

Fro

m

FIF

Os

P a c k e t _ D a t a

Link Testpattern

1 6

1 6

1 6MUX

PACE_fu l l

T r igger_Fu l l

KID

Nul l eventlogic

&

Headerformatter



GOL Interface

Fro

m/t

o G

OL

R T S

P a c k e t _ D a t a

decrement

Id le_counter

Busy_counter

exp i re

ex

pir

ep

res

et

decrement

p rese t

“ S O F ” c h a r a c t e r

C R CCRC generator

1 6

1 6“ C X T ” c h a r a c t e r

1 6

1 6

T X _ d a t a

Re

gis

ter1 6 1 6

C T S

D A V

C A V

IDL

E

BU

SY

8

8

R E A D Y

forc

e_

idle

R e S y n c

Fro

m/t

o P

ac

ke

t F

orm

att

er

From I2C B lock

From

Trigger

Decoder

s ta r t

Statemachine

MUX



Packet FormatH

EA

DE

RD

AT

A

015

Flags I

Slot 1

16-bit checksum

Slot 2

Slot 3

IDLE

SOF

Sample 18-bit EC

12-bit BCA-J1 B-J1

B-J1 C-J1

C-J1 D-J1

Col.# C Col.# DCol.# A Col.# B

A-J32 B-J32

B-J32 C-J32

C-J32 D-J32

A-J2 B-J2

B-J2 C-J2

C-J2 D-J2

Sample 32

KID

Flags II

TR

AIL

ER

F3 F2 F1 F0

K-chip Trigger FIFO FullPACE Trigger FIFO FullK-chip Data FIFO FullGeneral Error in PACE sync.

F3 F2 F1 F0

Channel Mask / PACE error

Trigger Inhibit Logic ModeCalibration EventReservedLink Test Packet flag

F7 F6 F5 F4

Flag Set I Flag Set II

F3 F2 F1 F0

K-chip Trigger FIFO FullPACE Trigger FIFO FullK-chip Data FIFO FullGeneral Error in PACE sync.

F3 F2 F1 F0

Channel Mask / PACE error

Trigger Inhibit Logic ModeCalibration EventReservedLink Test Packet flag

F7 F6 F5 F4

Flag Set I Flag Set II



Packet TypesH

EA

DE

RD

AT

A

015

Flags I

Slot 1

16-bit checksum

Slot 2

Slot 3

IDLE

SOF

8-bit EC

12-bit BC

KID

Flags II

TR

AIL

ER

HE

AD

ER

015

Flags I

16-bit checksum

IDLE

SOF

8-bit EC

12-bit BC

KID

Flags II

TR

AIL

ER

Normal Event Null Event

300 words

6 words



Link Layer

The Kchip employs a packet oriented data transmission protocol.

The Kchip Link Layer Protocol uses two uniquely defined transmission control characters, the IDLE and the SOH. The IDLE character allows the receiver to obtain and

maintain bit synchronization. . The SOF character indicates the beginning of the frame and

delimits the boundaries of subsequently transmitted frames.

data

GOLHDMP-1034

orTLK-2501

16

DAV/TX_en

CAV/TX_er800 Mbaud

data

16

RXDATA/RX_DV

RXCNTL/RX_ERREADY



Link Protocol

FF 1a 3F 80 DATA DATA DATA DATA CRCFF 1b FF 1a FF 1a FF 1b

Fill Frames SOF Data Packet Fill Frames

Packet format in CIMT encoding.

3F 80 DATA DATA DATA DATA CRCFF 1a FF 1b

Fill Frames SOF Data Packet

3F 80

SOF

DATA DATA

Data Packet

Packet format in CIMT encoding for back-to-back events.

IDLE CXT DATA DATA DATA DATA CRCIDLE IDLE IDLE IDLE

IDLE pattern SOF Data Packet IDLE pattern

Packet format in 8b/10b encoding.

IDLE = <K28.5, D5.6> or <K28.5, D16.2> : IdleCXT = <K23.7, K23.7> : Carrier Extend

CXT DATA DATA DATA DATA CRCIDLE IDLE

IDLE pattern SOF Data Packet

CXT

SOF

DATA DATA

Data Packet

Packet format in 8b/10b encoding for back-to-back events.

The Kchip can seamlessly use both encoding schemes supported by the GOL chip: the CIMT and the 8b/10b encoding.

The flexibility of using both encoding schemes is realized by properly choosing transmission control symbols (SOF, IDLE) which are supported in both encoding schemes.

CIMT encoding

8b/10b encoding



Calibration Circuit & DLL

The Kchip distributes to the PACE chips a calibration pulse of programmable delay with respect to the system clock and with programmable width (1 – 256 cycles).

The DLL from APV25 chip has been ported into the Kchip. Delay adjustment: 16 steps of 3.25ns

After an interval equal to the trigger latency a Trigger signal is generated by the Kchip in order to readout the Calibration Data. The automatic generation of the Calibration Event can be

disabled.

The DLL block as deliveredfrom RAL.



I2C Interface

The I2C interface can be used to access: The Kchip Internal Registers (There are 25 Status &

Control Registers). The Kchip FIFOS (Data, Column Addr. and Trigger FIFOs).

Supports 7-bit addressing Single Byte transfers. Synchronous design. (40MHz system clock)

DataFIFO

ColumnFIFO

TriggerFIFO

InternalRegisters

I2C interfaceState

Machine

synchronizer start

stop

SCL

SDAI2C SCL

I2C SDA



Buffer Overflow Handling

The Kchip can detect imminent overflow conditions and prevent buffers from actually overflowing and lose synchronization.

Trigger Inhibit Logic on Kchip. If a FIFO signals an Almost Full condition then the Trigger signal

will be gated until some data has been read out and space is made available.

The readout chain gets informed about the trigger gating condition and NULL Events are inserted to maintain Event Readout Synchronization.

The Trigger Inhibit logic can be enabled or disabled.



PACE synchronicity monitoring

Under normal operation the pipeline memories in the PACE chips should run synchronously.

The Kchip monitors the synchronization of the PACE chips by comparing the status of their control signals on a cycle to cycle basis. Cross-checks the 4 “DataValid” signals. Cross-checks the 4 “AlmostFull” signals.

“PACE out of sync” condition is signaled when the readout sequence in any of the PACE chips is not synchronous to the Kchip internal readout sequencer.

The “PACE out of sync” condition is flagged in the data packet header and in a special status register accessible through the I2C bus.



Implementation Issues

SEU Tolerant Design Protect Control Logic using Triple Module

Redundancy. All State Machines and Configuration Registers

have been triplicated.

Leave Data Path unprotected. SEU errors affect the integrity of small amount

of information and does not lead to a loss of readout synchronization.



Triplicated State Machine

Voter

VotedOutputs

error

F

SM

F

SM

F

SM

Voter

Inputs

F: combinatorial logicSM: state memory

erro

r

Voted Current State

Current State A

Current State B

Current State C

Outputs A

Outputs B

Outputs C

Next State A

Next State B

Next State C

A

B

C

Triplicated logic was described in Verilog and has been synthesized.

Erroneous state machine will recoverwithin a maximum of 3 clock cycles.



Kchip Design flow

Design implemented using the CERN DSM Design Kit in 0.25μm commercial CMOS technology.

Hardware Description Language: Verilog

Automatic synthesis and layout: Synopsys, Silicon Ensemble

Verilog XL simulation Static timing analysis: Pearl Design flow is “scripted”.

Design for Testability I2C port

can be used to access and test the on-chip SRAMs

Scan Path For design debugging and

production testing.

PostSynthesisSimulation

FunctionalSimulation

Verilogdescription

Static timinganalysis

Post LayoutSimulation

LVS

Tape Out

Syntesis

Static timinganalysis

Place &Route

DRCFinal layoutadjustments



The Synthesized Chip

Number of Digital Standard cells Registers: 1,400 Gates: 13,300

Clock tree statistics Number of buffers: 189 Number of Levels: 6 Max. delay: 685 ns Max. skew: 65 ps

Special Macro Cells 4x 1024 x 18bit, dual-port

SRAM 2x 128 x 27bit, dual-port

SRAM DLL

Number of pad cells I/O pins: 131 Power pins: 17 Total pins: 148

Size: 6 x 5 mm2

Pad Limited design.



Kchip Layout

Kchip designers: Kostas Kloukinas, CERN EP/CME Sandro Bonacini, CERN EP/MIC

Data FIFOs

Column Addr. FIFO

Trigger FIFO

DLL block

1st PrototypeCERN MPW10Submitted: Feb. 2003Received: June 2003



Prototype Tests

Scope Test functionality & validate

specifications conformity. Method

Make use of a Digital Tester at CERN (MIC group).

Use a generic “Test Fixture” board to host the chip.

Use a ceramic package (CPGA) to facilitate bonding.

“Test Vectors” were generated from Front-End system simulations.



Test Pattern sequences

(a) Kchip in Test Mode Access on-chip FIFOs and internal

registers through the I2C interface.

(b) Kchip in Test Mode Filling the on-chip FIFOs from the

ADC bus and then reading data back through the I2C interface.

(c) Kchip in Test Mode Loading data into the FIFOs

through the I2C interface and then reading it out from the GOL bus.

(d) Kchip in Normal Mode Sending triggers and reading out

events through the GOL.

KchipADC_data TX_data

I2C




The functionality of the Kchip has been successfully verified. Various data traffic patterns were simulated and test vectors

were loaded to the tester for verification. All modes of operation have been verified. No loss of readout synchronization up to 200KHz trigger rate

(Poisson distribution). I2C Interface

I2C interface maximum speed: 3.33 Mbit/s. Kchip has been tested with the CCU chip.

All Internal Registers were accessible. ID fuse bits were read out. Calibration Event Generation Logic

Verified the Programmable Timing of the CalPulse (on chip DLL)

A layout bug in the Data FIFO SRAM module has been identified. In specific traffic conditions, some data packets (0.5%) had errors

in the data field.




Maximum operating frequency: 60MHz The limiting factor was due to the bug in the SRAM module.

Power consumption Test Conditions: 40MHz, VDD=2.5V, T=25OC Icore = 68mA, Pcore = 170mW Iperi = 182mA, Pperi = 455mW Itotal = 250mA, Ptotal = 625 mW

Irradiation Tests Use of an X-ray machine at CERN. Step Irradiation at 1, 3, 5, 10, 20 MRad (SiO2). Dose Rate = 2.04 MRad/h

Devices were operational up to 20 MRad @ 2.5V, 40MHz. Observed a small drop in power dissipation.

ΔI = 2% @ 10MRad.



Conclusions

Conclusions The first prototype of the Kchip has been

successfully fabricated. Functionality has been verified on a digital tester.

Future Plans In system tests are under preparation. SEU tests will follow. Submission in next MPW.

Implement the fix for the bug in the SRAM module. Implement Hamming encoding in the FIFOs. Implement Built In Shelf Test circuit for the FIFOs.

Acknowledgements Microelectronics group in Rutherford Appleton Lab.

particularly Quentin Morrisey for the APV25 chip DLL macro cell.

Documents

Kchip: A Radiation Tolerant Digital Data Concentrator chip for the CMS Preshower Detector 9 th Workshop on Electronics for LHC Experiments Amsterdam, Sept