CMS ESR. May, 2004 HCAL TriDAS 1

HCAL TPG Status

Tullio GrassiUniversity of Maryland

May 2004

CMS ESR. May, 2004 HCAL TriDAS 2

Clock & BC0 Distribution

• “RX_CLK” and “RX_BC0” signals for all CAL TPGs transmission• Implement this requirement using 2 or 3-stage fanout scheme• Details to be defined with ECAL.

Rack-to-Rack CAT 6/7

HTR

DCC

HTR

HTR

HTR

FANOUT

HTR

DCC

HTR

HTR

HTR

FANOUT

FANOUT

FANOUT

FANOUT

TTC Minicrate

16 HCAL VME Crates

ECAL (18 crates)

CMS ESR. May, 2004 HCAL TriDAS 3

Fanout board - features

• Built by J. Mans and C. Tully at Princeton• Fanout 4 differential LVPECL signals over

cat6 with RJ45 connectors: – RX_CLK,– RX_BC0– TTC serial stream – 80MHz Ref_Clk

• Board operates in 2 modes– Global (unique board for ECAL and HCAL)– Crate

• Successfully used in 2003 testbeam and all current HCAL R&D test stands

• J.C. da Silva received one board for evaluation and “blessing”

CMS ESR. May, 2004 HCAL TriDAS 4

Fanout Board - diagram

TTC fiber

Clk80

P1

P2

Input from GLOBAL Fanout

18 Identical

Outputs

(also as timing)

40MHz TTC_CLK

RX_CLK

RX_BC0

INT_BC0

RX_CLK

RX_BC0

QPLL

80MHz

QPLL can run stand-alone

TTCrx

EXT_BC0

TTC Broadcast

QPLL implemented this way allows RX_CLK to be “cleaned” at each board if needed

G

C

G

C

FPGA

Progr.Delay

DecodingG

C

G

CMS ESR. May, 2004 HCAL TriDAS 5

Complete path of a 40MHz RX_CLK

TTC fiberCLK40_Des1

FPGACAT7 (RX_CLK,RX_BC0)

TTCrx

QPLL

FPGA

TTCrx

QPLL

Fanout board in Crate-mode

Fanout board in Global-mode

3.3V CMOS • Path is 3.3V differential PECL unless otherwise stated.• Path of RX_BC0 is similar but comes from the FPGA rather the QPLL• In the Global-mode card, do not mount the buffers for CLK80 and TTC

CAT7 (RX_CLK, RX_BC0, TTC, CLK80) HTR

SLB

SLB

SLB

SLB

SLB

SLBMax skew on HTR traces is 0.7 ns.

TPG spec is:Skew < 6 nsacross HCAL and ECAL

CMS ESR. May, 2004 HCAL TriDAS 6

TTC-PMC mezzanine (UMD)

– Carry TTCrx onto HCAL boards (HTR, DCC, Fanout)– Money-saver as it eliminated a lot of optical components

and Fine-Line BGAs on 9U boards.

Production is done (~350 boards)• The assembly house mounted all LEDs backward• Test in progress (100% yield so far)

CMS ESR. May, 2004 HCAL TriDAS 7

HTR Schematic

RX_CLK40

SLB

SLB

SLB

SLB

SLB

SLB

RX_BC0TTC

TTCrxmezzanine

CLK80

Crystal

Serial FE-Data

Ref Clk

Deserializers (8)

20Recovered Clk

TPG Path

SY

S4

0 C

lk

TTC Broadcast Async Fifo

PLLTTC 40 Clk x2

XILINX

LC

Fiber Data

SY

S8

0 C

lk

Fanout Card(1/VME crate)

CMS ESR. May, 2004 HCAL TriDAS 8

HTR Rev4 Status

• Current (Rev4) board – 3 in lab now– Testing:

Links/Clocks (same as Rev3, no problems, no mystery at UMD)

DAQ path exactly same as Rev3

Trigger Primitive Generation (basic features)

Integration with SLB (next slide)

CMS ESR. May, 2004 HCAL TriDAS 9

Trigger link and SLB testing

– Control of SLB (“Local Bus”) seems ok– JTAG working (connector on P3 area of motherboard)– Link tests between SLB and Wisconsin receiver board (STC)

• HP signal generator 120MHz clock for Receiver-STC• 120MHz divided by 3 and injected into TTC system• SLB runs with RX_CLK extracted from TTC+QPLL• 1.2Gbaud copper link: verified

– Some tests of Cat6 cable showed BER increases by ~10 compared to skew-clear cables.• This could be fine, depending on the final cable length

CMS ESR. May, 2004 HCAL TriDAS 10

Trigger link and SLB testing

Problems

In our setup we have seen 2 problems, but we had not much time to investigate:

1. Two pairs seem swapped (we see A,B,D,C instead of A,B,C,D)

2. Data are mis-aligned from pair-to-pair. The mis-alignement changes every time we restart.

We think that these are not a hardware or PCB problems, but cabling or firmware.

CMS ESR. May, 2004 HCAL TriDAS 11

Trigger Link testing

• So far, used Wisconsin STC boards. For commissioning and mass-production we need something different: – More channels– Programmable to optimize the test setup

• Layout underway for an SLB/Vitesse mezzanine transition – Allows to plug a Vitesse-Rx-mezzanine on each SLB post.– Run backwards into motherboard

• Board is now under design– Delays as an engineer left.

HTR

SLB site

UW Vitesse receiver mezzanine card

CMS ESR. May, 2004 HCAL TriDAS 12

Production Issues

Fiber Optics• Some problems not fully understood.• Seem more related to optical cables, connectors, laser• Changing the receiver (HTR) is not considered

Yield (excluding optics)• Last year batch (Rev3) had 5 bad boards out of 30• Only 1 was fixed (assembler mounted wrong part)• Not much time to find out the problems• Assembler seems the best in the area.

CMS ESR. May, 2004 HCAL TriDAS 13

HTR Plans

HTR-RCT Integration test in Wisconsin– Start on May 24th

– Will try to put energy on a given bunch and generate an L1A

– Main goal is to validate all hardware

If Wisconsin and Maryland groups are satisfied with the Integration Tests, we will launch the HTR

production.

Testbeam04 in CERN– At some point we want to use the last version (Rev4)– Issue: we only have 3 now.– HF needs 2 HTRs to run

CMS ESR. May, 2004 HCAL TriDAS 14

Level 1 Latency

Item HB HE HF

BX to QIE input 2 2 4

QIE to GOL (FE) 9 9 8

Optical Link (75-90m@1clock/5m)

15 15-18 15-18

HTR → SLB 12 12 12

SLB ? 4 4 4

TPG Cables (15m) 3 3 3

TOTAL 45 45-48 46-49

HCAL O-E QIE CCA HTR SLB RCTBX TOF To RBX Data To RCT

RBXHPD or PMT (HF)

46 clocks = 1,147.7ns

GOL

• Nothing has changed (46 clock tick budget)– FPGA logic does not include summing!– Estimates: probably 1 more clock cycle

in HF– Eliminating summing in overlap might be

ok for MET/Jet triggers• But would still have the 1→ 6 summing in

HF

12-13

CMS ESR. May, 2004 HCAL TriDAS 15

TPG Alignment

• Align so that all ECAL and HCAL data from same bucket reaches RCT inputs at same time– Achieved by delaying each channel individually

– Method for establishing this delay implemented inside SLB• Histogram data over threshold, look for LHC structure pattern

– Issue: For some detectors, occupancy is very low (HO, especially at low lumens)

Ch N

Ch N+1

Ch N+2

RX_BC0 (Global)

CMS ESR. May, 2004 HCAL TriDAS 16

Absolute (and Relative) Timing

• Relative timing within HCAL via:– Laser and LEDs

• Have to consider random latency variation after resynching optical links– Cross-check with BC0 from FE

• Absolute synchronization for Level 1– SLB histograms ET – Looks for LHC beam structure– Does this work? (esp at low luminosity?)

• Salavat has been working on this– Min bias events, some with Orca and

some with fast sim

• Occupancy at 1034 shown here– 250MeV per ADC count

• Question: how many orbits to establish the LHC beam structure per detector?

CMS ESR. May, 2004 HCAL TriDAS 17

Summary (from Salavat)

HCAL Element Cut (ADC counts) # LHC Orbits Required (88s/orbit)

HB, ~0 ≥6 5x105

HB, ~1.4 ≥6 105

HE, ~1.6 ≥5 104

HE, ~2.8 ≥5 102

HF, 13, ~2.9 ≥5 104

HF, 13, ~3.4 ≥5 103

HO, ring 2 ≥8 >107

• At 2x1033

CMS ESR. May, 2004 HCAL TriDAS 18

Summary (from Salavat)

• For HB, HE, and HF probably easy to remeasure “on the fly”– Fill SLB histograms, read out over VME, calculate offset…

• For HO, probably will take few hours to maybe even a day– OK as long as absolute (non random) latency is stable over long

periods– Need more simulation, checks…in progress

CMS ESR. May, 2004 HCAL TriDAS 19

Occupancy Implications and Alignment

• Bottom line: Will try to develop the following algorithm:– Whatever it takes, we measure the absolute alignment

• If it takes hours, then so be it...

– We hope that this absolute alignment will not change over time– Keep track of the relative alignment by:

• Sending up a BC0 signal from the FE– Keep track of this. If the links go down and we reset, then we measure the

relative alignment and adjust accordingly

• Problems:– If we are off by an order of magnitude or more...

• Needs some more simulation, computing resources, etc.

– Assumes long term stabilities which will have to be tracked

CMS ESR. May, 2004 HCAL TriDAS 20

Another Latency Issue

• TTCrx chip has a chip-to-chip variation in latency of ~20ns– Affect only the DAQ-path

• How well known is this?• Need a scheme for insitu calibration of the TTCrx latency

– Probably can come up with something for HTR/DCC/Fanout– Not sure what to do about TTCrx in FE...