Critical Data Errors

S. DurkinThe Ohio State University

USCMS EMU Meeting, FNAL, Oct. 20, 2005

Data Errors – A HistoryData Errors – A HistoryData Errors in 2004 Test Beam were the Dominant Problem

• DQM Crashes, DDU reports critical errors• How healthy is the electronics?

• DMB-TMB LCT time delay missing • Impossible to time in DAQ (Experts: don’t worry!)

September 2005 Slice Test

• full Peripheral crate of production TMB, DMB, CCP, MPC• FED crate with production DDU/DCC• missing new Crate Controller, Final LV supplies

• DMB block recycle increased to accommodate large Latency• DMB-TMB LCT delay kludged in DMB firmware UCLA must be added permanently to TMB firmware

Additional problems uncovered in Sept05 Slice Test

• CFEB L1A# ambiguous in overlap events (2 events/block)• word count overflow for ALCT time bin data (header size too small) • gigabit driver memory ring wrap around

Vindication 1 million event error-free runs!

Data Errors – Gone!Data Errors – Gone!

Preliminary Slice Test Track Fitting S. Durkin Sept. 8, 2005

Analyze RunNum1007Evs0to49999.bin using /AnalysisUtilities

• look for tracks in two separate CSC’s• trigger very loose, 9291/50000 such events found• single overall cross talk and noise constants used• events typically very well fit (see next two pages)• analyze data with CSC5 and CSC9 hit.

CSCID Chamber Type 4 ME2/1 16 no data 5 ME2/2 31 7 ME2/2 32 8 ME3/1 16 no data 9 ME3/2 31 10 ME3/2 32

CSCID1 CSCID2 Events 5 7 636 5 9 3129 5 10 1423 7 9 994 7 10 2326 9 10 670

Red Events chamber overlap.

************ Event ID 38 CSC ID 5 CSC type 0 Cathode Tracking: Number of Tracks 1

Track 0 Plane 0 Hits 0 y 61.8693 dy 0.07552 dif -0.0715 used Plane 1 Hits 0 y 61.2128 dy 0.02231 dif 0.0251 used Plane 2 Hits 0 y 60.6817 dy 0.00702 dif -0.0037 used Plane 3 Hits 0 y 60.1169 dy 0.01132 dif 0.0012 used Plane 4 Hits 1 y 59.5459 dy 0.01219 dif 0.0123 used Plane 5 Hits 0 y 59.0460 dy 0.03158 dif -0.0477 used

Cathode Track Fit (4 DOF) for track 0: Slope -0.55990 Intercept 61.7978 Chisqr 5.752332

Typical Track – Evt 38 CSCID 5 (Near Side)

Distance from track in Strips.Typically < 200m

************ Event ID 38 CSC ID 9 CSC type 0 Cathode Tracking: Number of Tracks 1

Track 0 Plane 0 Hits 0 y 36.5085 dy 0.00424 dif 0.0006 used Plane 1 Hits 0 y 36.0551 dy 0.00995 dif -0.0662 not used Plane 2 Hits 0 y 35.4979 dy 0.01635 dif -0.0292 used Plane 3 Hits 0 y 34.9292 dy 0.01200 dif 0.0194 used Plane 4 Hits 0 y 34.4546 dy 0.01630 dif -0.0261 used Plane 5 Hits 0 y 33.8762 dy 0.02779 dif 0.0321 used

Cathode Track Fit (4 DOF) for track 0: Slope -0.52015 Intercept 36.5090 Chisqr 9.727195

Typical Track – Evt 38 CSCID 9 Far Side

Cathode Strip Angle Matching

Black Curve – CSC5 Red Curve – (CSC5 minus CSC9 )

• Compare track angles• Ignore any geometry corrections

CSC9 (degrees)

CS

C5

(de

gree

s)

Nu

mbe

r o f

Eve

n ts

(degrees)

~ 4 degrees

Cathode Strip Intercept Matching• Extrapolate track intercept• Ignore any geometry corrections except for Z position• Note: CSC5 and CSC9 are back to back (mirror images)

CS

C5

plu

s C

SC

9 In

terc

ept

(str

ips)

CSC5 Slope

Black Curve – CSC5 InterceptRed Curve – CSC5 and CSC9 Corrected Intercept difference

Nu

mbe

r of

Eve

nts

Intercept (strips)

~ 2.5 strips

Best Fit DistanceBetween Chambers 101cm

Anode Wire Angle and Intercept Matching

• Anode tracks (units of wire groups) • Ignore any geometry corrections except for Z position

CS

C9

(d

egre

es)

CSC5 (degrees)

CS

C5

min

us

CS

C9

Inte

rcep

t (w

ire

grou

p)

CSC5 Slope

Cathode Strip Expected Resolution

• covariance matrix from the least-squares fits to the track can be used to estimate angular error and extrapolation error• using PDG notation

D/222 2

111211112 )/100(//1 Dposition

edextrapolat

• from the data- angular error 0.15 degrees extrapolated position 0.6 strips

Multiple Scattering Dominates

• from the particle data group it is easy to calculate

)(

5

cGeV

o

scatmult P

Real Time DDU MonitoringReal Time DDU MonitoringDDU/DCC Monitoring Controls Detector Resets Run Control: 1) reset initializes electronics 2) start DDU monitoring 3) take data

Continuously read DDU status register

Software Reset of Emu System on error (approx 200 sec)

FMM Reset from computer write to VME register

Firmware Real-time resets also possible (approx 30 sec)

• Spy Data DQM Superfluous, Monitoring done “real time”• DDU traps ALL critical errors in 25-100 sec and requests FMM reset

DDU Error CheckingDDU Error Checking

1 million events 16 Gb~ 16 seconds LHC running

LHC DDU/DCC OccupancyLHC DDU/DCC OccupancyHow well have we checked “real time” error detection?

Conclusion – Need More DataConclusion – Need More Data

Slice Test: 112K events(1 LHC sec) takes a 5 minutes run.

112:1

30 days running yields 6.4 LHC Hours

Data taking should be left on 24/7.Shifts are not necessary. No additional risks.

To Reset or Not to Reset:Rare Hardware failures have to be understood.