HF Luminosity and Jet Triggering Drew Baden, Tullio Grassi, Jeremy Mans University of Maryland Chris Tully Princeton University Bob Hirosky University

HF Luminosity and Jet Triggering

Drew Baden, Tullio Grassi, Jeremy Mans University of Maryland

Chris Tully Princeton University

Bob HiroskyUniversity of Virginia

Fermilab HCAL Nov 2004 2

HF Basics

• HF covers ~3<<5• Steel absorber and rad hard quartz fibers

– Cerenkov light collected via phototubes, uniform HCAL readout

• HF and HF• 36 and 12 = 432 towers per side

– =10° and =0.166

• Each tower has a long and a short fiber running along z

– Short is in the back ~ “ETHAD”– Long is front to back ~ “ETEM+HAD”– Makes 2x432=864 towers per side

• Level 1 Trigger and HF– Combines 2 x 3 = 6 towers = 1 trigger

primitive (TPG)• =20° and =0.5

– Only used for MET and JETS• Not for electrons and photons

HF


Jets in Level 1

• TPGs– HB/HE: 0.087x0.087 x0.5 x 0.35 in HF

• CMS Level 1 Jets– Calorimeter organized into “Regions”

• 4x4 TPGs per region in HB and HE

• 1 TPG per region in HF

– Jet finding via “sliding window”• Sliding window using 3x3 regions: 12x12 TPGs

– Jet candidates are sent from RCT to GCT, which sorts and sends candidates to GT by category:• 12 jets, nominally 4 highest each of central, forward, and tau jet candidates

Calorimeter “Region” Barrel/Endcap

TPG (x) Region

HB/HE 0.087x0.087 4x4 TPGs

HF 0.5 x 0.35 1 TPG


Min Bias

Atlas CERN/LHC 96-40

• Min bias distributions @ 1034

– <n>~20 (assumes ~80mb inelastic)– <dn±/d> ~ <dn0/d> ~ 8 @ 14TeV

– <ET> ~ few GeV (and falls exponentially)

– 20<n> x 8ch x 10dx 2GeV = 3.2 TeV/interaction (Had)

– HF...40% of CMS in coverage• 640 GeV, 72 TPGs/side, ~10 GeV/TPG (@<n>=20)

– Current TPG = 0.5 x 0.348 x– s

• New Level 1 triggering will need to…– Sharpen efficiency– Move HLT-like algorithms and resolution as close to

L1 as possible


SLHC Background Reduction

• Beat down the background L1A rate– LHC Design luminosity of 1034 has large backgrounds:– Depending on the scheme for high luminosity

• <n> ~ current for 25ns SLHC, rates scale with Luminosity

• for continuous beam….smaller <n> but large pileup...not sure

– 100kHz L1A rate is ingrained, will most likely hold• Size of derandomizing buffers, etc.

• Bandwidth to HLT

• Number of HLT processors....

• CMS Calorimeter trigger based on TPGs– In HF…

• 1 TPG = 6 towers (3 x 2)

• Lack of granularity might make it useless for Level 1 jet triggers with large number of multiple interactions

– In HB and HE…• Jet-finding in Level 1 very inflexible, probably cannot be changed

• Ditto for isolated e and triggers

• We propose to start R&D on improvements

Condition Process Rate

1 ET>60 Jet0 10Hz

2 ET>15 Jets 0s 10Hz

1l pT>60 W l, jet l 10Hz

2l pT>15 Z l l 20Hz

ETmiss > 150 QCD jets 10Hz


SLHC Signal Enhancement

• Add functionality

– Higgs id without a tag is very hard• Gluon fusion backgrounds are too high, esp at 1035

– W Boson Fusion (WBF) dominant experimentally accessible rate

• Forward jets + central Higgs decay

• Tag jets are in HF+HE so HE will need to be included

– Current trigger at high luminosity will be difficult• Depends on scheme for increasing luminosity of

course…

C. Tully & H. Pi JetMetPRS Aug 2004

(tagged “forward” jets)


Simulation Results

• Chris Tully led effort to study current trigger @ 1035/cm2/s– Chose 25ns RF structure for lack of any better guidance

– Compared WBF qqH to QCD background

– Study “feature bit” for HF TPG• Current version: bit = 1 if any of the 6 towers in the TPG > % of total

TPG ET

• Used a 2x2 instead of 3x2 for expediency

– Not much discrimination at LHC or SLHC luminosities• At 1035 can see the effect of minbias/underlying event adding significant

energy to jet

– “feature bit” becomes “featureless”… 1035/cm2/s

1034/cm2/s


A first look…

• Current scheme– Jet candidates using 3x3 CR sum, x=1.5 x 1.0 – Slides window by 1 CR, x=0.5 x 0.35

• New scheme– Construct jet candidates from 4x4 tower sums,

x=0.67 x 0.7– Slide window by 1 tower, x=0.17 x 0.17

• Feature bit on if number of cells needed to sum 60% of ET (n60) in 4x4 cluster < cut– Use n60 < 7 to set feature bit– Prelim studies show QCD jets are narrow & well

contained• Jets from .5 cone all have 2nd moment < 0.3 in R

• Require jet candidate threshold && feature bit = 1• Can also require perimeter “quiet” (not studied

yet…)


n60

• LHC design luminosity 1034/cm2/s and SLHC 1034/cm2/s w ith 25ns RF structure…– High efficiency and background rejection for both LHC and SLHC

• > 80% and background rejection = 3 @ 1035/cm2/s

1034/cm2/s1035/cm2/s


WBF Signal vs background

• Signal: WBF qqH, MH=130 GeV

• Background: QCD from uniform sampling 30 – 300 GeV

• Central trigger H WW ℓℓℓ=,e)• Add forward jet trigger using current and new algorithm

– Compare 1034 and 1035– Current algorithm jet candidtes from x=1.5 x 1.0 sliding window and “feature bit”– New algorithm jet candidates from x=0.67 x 0.7 sliding window and n60 cut

• Comparison: – Signal:

• Efficiency goes up in both cases due to volunteer ’s in central – this needs more study

– Background:• Current algorithm: goes from 49% to 100%

• New algorithm: stable at 30%


Hardware Design

• We propose NOT changing current HTR– 48 QIE channels input

– 6 SLB sites• Each SLB transmits 2 TPGs/twisted pair over 4 pair

– For HF, 1TPG = 6 QIE channels• 48/6 = 8 TPG output• HF HTRs will only populate 1 SLB site• We will have 5 free SLB sites to use

• Proposal: – Luminosity: dedicate 1 SLB site

– Jet trigger: dedicate 4 SLB sites


Bottom view

SLB connectors

Virtex2PRO FPGA

Luminosity

• 1st prototype of combined Luminosity/SLB/RCTtest board made

– 1 SLB site footprint on bottom• SLB connectivity:

– 36bits from each HTR Xilinx + TTC+Localbus

– UW RCT receiver footprint on top– Xilinx Virtex2PRO FPGA

• Rocket I/O, embedded PPC, block ram...

• Uses:– Level 1/TPG commissioning

• Can host RCT Vitesse quad receiver– Will buffer TPGs and transmit back into HTR

FPGA for testing

• Simulating Level 1– Has RJ45 and SMA output which can drive

Jeremy’s HCAL trigger board

– Luminosity• 72 bits from HTR xilinx can accommodate full

48-channel single tower threshold

• Output: use RJ45 and drive gigabit ethernet– Under consideration...

– Plan to meet w/Marlow&Tully soon...

RJ45 I/O

Top View

RCT receiver connectors

SMA I/O


HF Trigger• Topology:

– Each HF HTR receives 12x 4 towers

– Need 9 HTRs per side for the long fibers

• Baseline Jet algorithm– This suggests a 4x4 sliding window to contain the jet

– For isolation another 2 on each side in (and maybe also in )• 6x6 for area for each jet candidate

• What are the I/O considerations here?– Need to be able to get the data from the HF/HTRs into some kind of cluster finder


HTR/Cluster Finder I/O

• It’s all in the edges...– Figure shows 3 HTR worth of data

– Find 4x4 jets sliding 2 in over each “edge”• There are 5 possible groupings of 4 towers within 2 of the edge• Heavy lines show the possible set of towers in

– For a 6x6 design, will need 3 HTRs worth of data for each cluster finder• Figure shows 2 cases: central HTR data and left+right for isolation, and a jet that

straddles the border

• Each HTR will have to send its 12x4 towers to 2 separate cluster finders– I/O calculations involve 48 towers

– Cable plant involves 48x2 towers


Bandwidth and Cabling

• HTR Xilinx →4-slot Trigger card (HFT, for “HF Trigger”)– Needs all 48 channels, but only enough lines to send data from 32 channels

@ 40MHz• Need to run the transmission at 80MHz.• 80MHz tests show that with proper termination and clock phasing, should work ok

• HFT → Cluster finder– Try to send data over cat6/cat7 quad twisted pair using 8B/10B

• We already know how to do 8B/10B since we are doing this now• Minimizes costs and engineering (no optical)

– Will need to make the copper cables as short as possible• Need to already consider rack topology

12.5ns


Rack Topology

• Chris Tully worked this out with Rohlf and Ianos• New “Luminosity” VME crate in the center minimizes distance from the 3 HF crates

and doesn’t break Wesley’s topology for Level 1– Goal: keep cable lengths < 5m


I/O Scheme Using Copper

• Current scheme:

– Use TTC clock into the QPLL to drive our links

– Asynchronous fifo at receiving end for phase synchronization

• Alternate scheme:

– Use crystal oscillators to drive links, and asynchronous FIFOs on TX

– Latency cost in frame clock ticks (80 or 120MHz frame clocks)• Small number of frame clock ticks on TX end• Logic on receiver end – links will be not be running at LHC frequency

– Would necessarily increase latency by some small number of frame clocks

• Note that current RCT+GCT latency is 40 clock ticks, so we would have that amount of time to “play with”

• Current HTR firmware uses ~12 clock ticks now• We think we will have plenty of time to produce jets for the global trigger

RxData

RX ClkTTC

QPLL

TxData

QPLL

TTC

Data

Fifo

TTC

Crystal

RxData

RX ClkTTC

QPLL

TxData

QPLL

Data

FifoFifoCompensation

Logic


I/O Considerations Drive Everything

• Current system SLB transmits using 1.2Gbps links– Fully populated HTR has links/SLB, each @ 1.2Gbps

– Each SLB drives 1 quad twisted pair cable running over 10m

• We plan to use 4 SLB sites– A single card would be 15.5cm high, capable of housing an 8way RJ45 connector ala

the Princeton Fanout card (which has 2 of these)• So we need to run our links on 4 quad twisted pair cat6/7 cables, sending the data to 2

separate receivers

– So we need to squeeze out all 48 channels onto 4 quad twisted pair (16 pair)

– Need to send 3 channels/twisted pair every 25ns, or 3 bytes @ 40MHz

– If we add hamming codes, that would make 4 byes @ 40MHz = 160MB/s

– Using the same 8B/10B encoding as now, that makes 1.6Gbps

– This is the same gigabit ethernet that we use now

– Note: Xilinx FPGAs have built-in I/O exceeds 3Gbps• XC2VP30 has 8 built-in transceivers and costs ~$500 today, would need 2 per HFT

– R&D is needed here. Note: da Silva uses these for the ECAL DCC….


HF Jet Board (HFJ)

• This will be a 9U VME board in the “Luminosity” crate• Straightforward design

– Links on RJ45 connectors, deserialized, fanout to FPGA• 9 HTR/side so try to service 3 HTR/HFJ• 8 RJ45/HLT means 24 RJ45/HFJ – HFJ would probably have to be 2 VME widths• Need 3 HFJ/side, or 6 total, or 12 VME slots

– Pin gymnastics• Each of 24 RJ45 will have 4 twisted pair, or 96 pair total per HFJ, or 96 deserializers!

– HTRs have 16….maybe this is doable…

– Might be better off using something like the Vitesse quad deserializers

• 96 deserializers, 20 pins each 1920 total parallel digital pins• Use 3 FPGA/HFJ, each HFJ services 32 inputs and needs 640 pins for I/O• Xilinx XC2V4000 flip-chip has 912 user I/O pins, there are others…

• Clustering algorithms…– Needs a lot of study – simulation and hardware implementation


FPGA Menu


Optical I/O Scheme

• Use optical to run faster by x4 and save connectors– 10Gigabit ethernet is becoming more and more common– Supported by the FPGA internal transceivers– Will definitely need to use crystals for transmitting– Single mode fibers are becoming more and more easy to use

• Would allow single-width VME boards for the HFJ– Fewer connectors (24/HFJ down to 6 optical)

• Board layout, mechanics of connectors, will be critical– R&D would have to start soon for this path


R&D Program

• Scrub design so that we can piggyback on existing system and build cards that can be implemented by LHC startup

– Keep entire current VME architecture, but add new capability– Run parasitically, collect data, study, iterate

• R&D list for Luminosity project– Settle on Luminosity requirements, finish prototype single-SLB width board

• R&D list for Trigger project– Simulation needed to settle on algorithm approach– HLT card:

• Learn how to use new FPGA’s with embedded processing, DSP, built-in deserializers….

• Verify HTR to HLT @ 80MHz

– HLT to HJF transmission• Study transmitting signals over <5m cat7 copper cables @ 1.6Gbps

• Use crystals to drive transmitter as an alternate scheme

• R&D quad serializers on the HLT and quad deserializers on HFJ

• 10G optical transmission using crystals and single-mode fibers

– HFJ• Study algorithms for clustering – lots of simulation needed here

– R&D on how to include HE to get the meat of the WBF signal and extend jet trigger to full CMS

Documents

HF Luminosity and Jet Triggering Drew Baden, Tullio Grassi, Jeremy Mans University of Maryland Chris Tully Princeton University Bob Hirosky University