of 25 /25
T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet , T. Chaleil, C. Coquelet, X. de la Broïse, E. Delagnes, F. Druillole, A. Le Coguie, E. Monmarthe, J-M. Reymond, E. Zonca DSM/IRFU, CEA Saclay, France O. Ballester, IFAE, Barcelona, Spain

T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

  • View
    215

  • Download
    1

Embed Size (px)

Citation preview

Page 1: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

T2K Time Projection Chambers Front-end Electronics –

Experience Return

P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse, E. Delagnes, F. Druillole, A. Le Coguie, E. Monmarthe, J-M. Reymond,

E. Zonca

DSM/IRFU, CEA Saclay, France

O. Ballester,

IFAE, Barcelona, Spain

Page 2: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 2 Saclay 2-3 December 2009

Plan

Presentation of the T2K experiment

Architecture of TPC readout electronics

Principal components

Current status

Experience return

Page 3: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 3 Saclay 2-3 December 2009

Tokai to Kamioka (T2K) experiment

Main Physics Goal: neutrino oscillation

• μ disappearance for improved accuracy on 23

• e appearance to improve sensitivity to 13

50 kT water

Page 4: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 4 Saclay 2-3 December 2009

T2K Time-Projection Chambers

TPC features

• Resolution goal: 10% for p < 1 GeV/c

• Double wall design; inner wall is the field cage

• Total ~9 m2 instrumented with 72 Micromegas amplification modules segmented in ~7 mm x ~10 mm pads

• Custom readout electronics

1 of 3 TPCs shown

1728 pad (36 x 48)Micromegas module

34 cm x 36 cm

CalibrationGas system Electronics; on-line software

1 m

2 m

Page 5: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 5 Saclay 2-3 December 2009

TPC Readout Challenges

Environment

• Embedded front-end: limited access, need low power

• No radiation, low magnetic field (0.2 T), Japanese underground facility

Highly segmented detector

• 124.000 channels over 9 m2 of instrumented area

→ Massive replication of identical modular building blocks

Extreme burstiness of data

• 124.000 channels; sampling: 12-bit 33 MHz → 50 Tbps

→ beyond capability of fully digital solution for power budget target

Large raw event size but modest average dataflow• 50 Tbps during ~15 µs drift time → 90 MBytes

• Allowable event size for storage: ~250 KBytes

→ data reduction of ~400 to achieve in real time• Spill repetition period: ~3.5 s; Cosmic calibration: 20 Hz max.

• DAQ average rate: ~250 KBytes × 20 Hz = 5 MByte/s

→ commercial computers and networking techniques

Page 6: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 6 Saclay 2-3 December 2009

Plan

Presentation of the T2K experiment

Architecture of TPC readout electronics

Principal components

Current status

Experience return

Page 7: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 7 Saclay 2-3 December 2009

Logical Read-Out Flow

Architecture principles

• Custom front-end ASIC; analog memory (Switch Capacitor Array)

• ADC + digital buffer mounted close to the detector

• Multiple optical fibers send data to off-detector concentrator cards

• Interface to common DAQ via standard gigabit Ethernet network

Pre-amp and shapers

Samplers and analog memory buffers

Analog to digital conversion

Digital buffer – zero-suppression

Data concentrator Clock/Trigger fanout

~124.000 channels

1728 ASICs

On

-det

ecto

r

72 Optical fibers

~4 Tbaud*/s peak*1 baud = 12 bit

~2 ms retention max.

34 Gbaud/s peak

400 Gbit/s peak

~1-10 Gbit/s averaged

Shared DAQ~0.1-1 Gbit/s

432 FrontEnd Cards

18 DataConcentrator Cards

72 Front-EndMezzanines cards

Off

-det

ecto

r

Global Clock/TriggerGigabit Ethernet

Page 8: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 8 Saclay 2-3 December 2009

Architecture Implementation3 TPCs

1 m

2,5

m

2,5 m

Network

TCP/IP

PC Linux

DCCs

PrivateEthernet

GigabitEthernet

DAQ control

DetectorB

DetectorA

Global trigger

1 of 6 TPC end-plates(12-modules)

Outside magnetInside magnet

2 of 12 (or 18) DataConcentrator

Cardsx 6

12 duplexOptical fibres

1 of 72 modules

Front End MezzanineCard (FEM)

288 channelFront End Card (FEC)

1728 pad Micromegas plane

Slow controlnetwork

Optical fiberto/from DCC

Low voltagePower supply

1 of 1728 Front-End ASIC “AFTER”

72 channel x 511 time bucketsSwitched capacitor array

Page 9: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 9 Saclay 2-3 December 2009

Plan

Presentation of the T2K experiment

Architecture of TPC readout electronics

Principal components

Current status

Experience return

Page 10: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 10 Saclay 2-3 December 2009

ASIC “AFTER” Functional Diagram

AFTER

511 cells

SCAFILTER

100ns<tpeak<2us

CSA

1 channel

x72(76)

76 to 1BUFFER

SCA MANAGERSLOW CONTROL

Serial Interface

W / R

Mode

CK

CK

ADC

TEST

In Test

120fC<Cf<600fC

Power Supply Reference Voltage Reference Current

Asic Spy Mode

CSA;CR;SCAin (N°1)

Power

On Rese

t

Design features

• 72 channels x 511 analog memory cells; Fwrite: 1-50 MHz; Fread: 20 MHz

• 4 Charge Ranges (120 fC; 240 fC; 360 fC & 600 fC)

• Supports positive or negative input signals

• 16 Peaking Time Values (100 ns to 2 µs)

Asic

For

Tpc

Electronic

Read-out

Page 11: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 11 Saclay 2-3 December 2009

The AFTER chipDesign facts

• 0.35 µm CMOS - 500.000 transistors

• 7.8 x 7.4 mm die – 160-pin 0.65 mm pitch LQFP

• single 3.3V supply - 8 mW/channel

Performance

• Integral non-linearity < 1.2% full range; ENC: 350 electrons (no load)

• Stored charge degradation after 2 ms retention: 0.18 LSB

→ All specifications met on first silicon!

AFTER chip used to readout TPC Micromegas and FGD Silicon PM

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 12000 13000 14000 15000

DAC bin

Inte

gral

Non

_Lin

earit

y =

(dat

a - f

it)/fi

tmax

(%)

100ns

2µs

Page 12: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 12 Saclay 2-3 December 2009

AFTER chip test and production

Production of the AFTER chip

• 5300 chip produced; 4750 OK; Yield: 89%

• 1800 chips delivered to T2K (TPC + FGD + monitoring chamber)

LABVIEW Test Software

Protection 1 Protection 2 No Protection

AFTER Test Card

Test bench

• AFTER test card served as pre-prototype of Front-End Card

• Xilinx Virtex 2 Pro eval. kit as pre-prototype of FEM board for readout

• DAQ and Analysis with Ethernet PC and LabView

Page 13: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 13 Saclay 2-3 December 2009

Front-End Card (FEC)

Features

• All design concepts validated on AFTER test board

• 1 FEC reads out 288 channels – 6 layer PCB

• Consumption: 1 A – 4 W

Quad-channel12-bit ADC AD9229

Passive protection Circuits

4 AFTER chips

3.3V regulatorsSilicon ID chipV - I –T monitor

+4V input

Four dual row 80-pin 1.27 mm pitch connectors to detector

PulserClock fanout

PhotoMos relay

80-pin connector to FEM

Page 14: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 14 Saclay 2-3 December 2009

Main functions

• Drive 6 FECs and aggregate data produced (1728 channels, 5.7 Gbps)

• Buffer one event (raw data), i.e. ~10 Mbit

• Deliver data to DCC upon request: raw data or zero-suppressed (one programmable threshold per channel)

• Configuration and slow control, voltage, current, temperature monitoring

Front-end Mezzanine (FEM) card

ZBT SRAM2 Gbps optical transceiver

CANbus slow control (RJ45)

FPGA

µC

FEC#1

Voltage regulators

Voltage regulators JTAGPower In

ConnectorFEC#0Buffers FEC#2

FEC#3 FEC#4 FEC#5

Page 15: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 15 Saclay 2-3 December 2009

Detector Module Read-out Electronics

72-channel ASIC

Quad-channel ADC

digital Front-endMezzanine card (FEM)

Optical Transceiver

FPGA

80-pin connector

288-channel analogFront-End Card (FEC)

1728-pad detector plane

Slow-controlNetwork - CANbus

Fiber toDCC

Low voltagepower

Materializing the concept

• 3 year development of ~6 equivalent persons from ASIC specifications to working readout of one detector module

Page 16: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 16 Saclay 2-3 December 2009

Plan

Presentation of the T2K experiment

Architecture of TPC readout electronics

Principal components

Current status

Experience return

Page 17: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 17 Saclay 2-3 December 2009

Integration of TPC#0 at Triumf

Feb. ‘09

Page 18: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 18 Saclay 2-3 December 2009

Integration of TPC#0 and #1 at TokaiPhotos: Claudio Giganti

Page 19: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 19 Saclay 2-3 December 2009

Latest news from T2K nd280 m

TPC #2

• Tested at Triumf and now in the clean room at Tokai

• Installation in pit on Dec’ 17th

→ Fully commissioned TPCs by mid-Jan’10 for closing magnet

TPC #0 and TPC #1

• Installed in the magnet; DAQ commissioning in progress

• Awaiting final gas mixture to power central cathode and detector HV

→ Detector commissioning starts Dec. 7th

The nd280 collaboration announced the observation of the first beam neutrino candidate (in the INGRID detector) two weeks ago!

Page 20: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 20 Saclay 2-3 December 2009

Plan

Presentation of the T2K experiment

Architecture of TPC readout electronics

Principal components

Current status

Experience return

Page 21: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 21 Saclay 2-3 December 2009

Contribution of Irfu to T2K nd280m

Electronics

• ASIC design, production test, readout electronic cards (design and production supervision)

Physics

• Simulation and experimental data analysis of MicroMegas detectors

Detector engineering and integration

• Design of Micromegas detectors, production coordination, QA

• Cooling mechanics and structure, cabling, services (HV, LV, fibers…)

Firmware and software

• Embedded firmware and software in the front-end, slow control, DAQ for detector test bench, firmware and software for DCCs, LV control, etc.

Project management

• M. Zito co-leader of TPC project with D. Karlen (Canada), A. Delbart, D. Calvet coordinator of “Micromegas modules” and “Readout Electronics”

→ Project relies on the diversity of competence at Irfu, teams above critical size and adequate funding

Page 22: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 22 Saclay 2-3 December 2009

Project Difficulties Production

• First batch of wafers of the After chip did not pass QA

→ 2 months delay to get After chips

• First batch of FECs failed: humidity stored in After chip package during test requires extended baking procedure

→ more than 2 months to understand the problem and find workaround

• Defects on 10% FEM for first batches (broken via below the BGA)

no consensus reached between designer, PCB manufacturer, and assembly company on exact cause (coating on vias?).

→ three month delay: defect analysis, modifications to PCB, new batch

• Could not solder some connectors on FEMs using lead free process

Front-end production took 1 year while 6 months had been planned

Page 23: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 23 Saclay 2-3 December 2009

Project Difficulties (con’t)

Back-end electronics

• DCC prototype board did not become functional

Unanticipated contribution of Irfu to back-end HW/FW/SW and DAQ.

→ Back-up solution based on customized commercial evaluation boards deployed by common effort of Triumf/Irfu/Lpnhe.

CommercialXilinx Virtex 4 ML405

Evaluation board

CustomClock board

CustomOptical

Transceiverboard

FGD DCC Crate

Page 24: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 24 Saclay 2-3 December 2009

What really helped

Commercial FPGA evaluation boards

• Inexpensive, readily available, customizable hardware

• Can develop and debug firmware/software on proven hardware– Firmware/software development before a dedicated board is available– Simplifies dedicated board debugging because firmware/software already

tested and only migration issues need to be solved

• Common platform for collaborative development among various group

→ ~45 Xilinx ML405 boards purchased by the TPC-FGD groups! Used at Saclay, Paris, Rennes, Geneva, Barcelona, Hamburg, Tokai, Vancouver

$995.00Part Number:

HW-V4-ML405-UNI-G

Page 25: T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse,

[email protected] 25 Saclay 2-3 December 2009

Summary

• What?

A 124.000 channel readout system for the 3 TPCs of the T2K experiment

• How?

A new 72-channel ASICs based on a 511 time-bucket SCA

Custom made readout cards at the back of each 72 Micromegas detectors

Extensive use of commercial FPGA evaluation boards

• Principal merit?

High density, low power: ~16 mW/channel (2 kW for 124.000 ch.); ~3 €/ch.

• Summary and Status

5 year development effort from concept to installation on site

→ TPC commissioning at J-PARC starting in the next few days