Michael Begel

ATLAS R&D for Trigger & DAQ

Michael Begel

Michael Begel2

R&D Goal: continuously develop expertise in select detectors and electronics technologies to efficiently provide integrated solutions for future experiments

Front-End ReadoutDetectors

LAr Front-End Trigger Digitizer Board (LTDB)

Liquid Argon Electromagnetic Calorimeter (LAr)

Front-End Link Exchange (FELIX)Global L1 Calorimeter

Trigger (gFEX)

Cold Front-Endfor 35 ton TPC

Field Cage for 35-ton Time Projection Chamber (TPC)

Front-End Link Exchange (FELIX)at protoDUNE-SP

Low Gain Avalanche Detectors (LGAD)

FELIX & CaRIBOu calibration boards in

a CERN test beam

Control & Readout Board (CaRIBOu)

Signal Proc.

Sensor Front-End Board

Michael Begel

Front-End Signal Processing

● BNL focuses on overall system optimization for integrated detector readout

● LAr Trigger Digitizer Front-End Board○ ATLAS Phase-I LAr Calorimeter Front-End (for 2021)

3LTDBLAr Calorimeter FELIXgFEX

FELIX

LTDB

○ combined analog & digital board■ analog and digital parts designed

by different institutes⇒ Gerber files merged together

■ BNL leads architectural design■ collaboration with Saclay & Milan

○ 320 calorimeter signals (supercells)■ 80 custom 12-bit ADCs @ 40 MSPS digitization■ 20 custom MTx devices for data processing■ 5 GBTx/GBT-SCA for clock distribution, control & monitoring■ 40 optical links @ 5 Gb/s ⇒ 200 Gb/s output

○ pre-production module installed in ATLAS last week

Michael Begel

● Preamplifier & Shaper ASIC (HLC ASIC)○ ATLAS HL-LHC Upgrade LAr Calorimeter Front-End (for 2026)

○ fully differential low-noise amplifier■ simultaneous readout of two gain settings■ built-in driver to interface COTS or ASIC ADCs for testing■ built-in pulse generator & mask bit for individual channels

to calibrate electronic response and perform crosstalk studies

○ 65 nm CMOS process (TSMC)

○ engineered by BNL Instrumentation Division■ development continued

from VMM ASIC for ATLAS Phase-I UpgradeMuon Detector Front-End

4

Front-End Signal Processing

LTDBLAr Calorimeter FELIXgFEX

ZC706

FETB

test board

HLC1

Michael Begel

High-Throughput Trigger

● BNL is designing electronics that maximizes throughput while also runningfast trigger algorithms:○ FPGA with many transceivers satisfies high-bandwidth, fixed-latency & processing requirements

● Global Feature Extractor Module (gFEX)○ ATLAS Phase-I Level-1 Calorimeter Trigger Module (for 2021)

■ single board system receives data from full calorimeter■ hardware designed by BNL

○ ATCA blade:■ 30 layer Megtron6 PCB■ 3 VU9P FPGA and 1 ZU19 MPSoC■ 128 Gb DDR4 RAM■ 420 optical fibers in 35 miniPODs

○ capacity:■ input: 3.9 Tb/s on 312 fibers■ output: 1.4 Tb/s on 108 fibers■ inter-FPGA: 1.6 Tb/s (high-speed & low-speed)

○ pre-production module to be installed February 2018

5LTDBLAr Calorimeter FELIXgFEX

25.6 Gb/s in PCB

Collaboration between BNL and Chicago, Indiana, Lund, Oregon, Pittsburgh, Stockholm

Michael Begel

High-Throughput Trigger

● Global Common Module○ ATLAS HL-LHC Trigger/DAQ Upgrade (for 2026)

○ Global Trigger concentrates data for a full event from detector & trigger systems onto a single processor for analysis■ critical part of the TDAQ Upgrade■ up to ~100 Tb/s into Global Trigger■ exploits data aggregation and

serial-to-time multiplexing■ processes data with low latency (<4 s)

○ Global Common Module (GCM) is centerpiece of the Global Trigger■ single electronics module — based on gFEX —

used for all elements■ differences in functionality implemented via firmware

6LTDBLAr Calorimeter FELIXgFEX

Michael Begel

High-Bandwidth Readout

● BNL developing electronics for high-bandwidth detector readout○ factorize front-end electronics from data handling with compact,

high-density, scalable, low maintenance, easily upgradeable, commodity-based solution

○ reduces amount of custom hardware in favor of scalable detector-independent COTS hardware & software

○ paradigm shift adopted by ALICE, ATLAS, LHCb,ProtoDUNE-SP, DUNE and sPHENIX

7

custom commodityFELIX-711 used byProtoDUNE-SP

● Front End LInk eXchange (FELIX)○ baseline for ATLAS Phase-I and HL-LHC Upgrades○ ProtoDUNE-SP for 1 Anode Plane Assembly (APA)○ expected baseline for DUNE readout○ sPHENIX for TPC and MVTX readout○ NSLS-II with Vertically Integrated Photon Imaging Chip (VIPIC)○ proposed for Belle II, test beams, eRHIC, ANL Light Source

Cold Front-End TPC

FELIX

Michael Begel

High-Bandwidth Readout

● Front End LInk eXchange (FELIX)○ developed for ATLAS Phase-I Trigger/DAQ Upgrade (for 2021)

■ BNL designed hardware & co-developed firmware

○ generic PCIe card with Ultrascale Kintex FPGA ■ 48-channels Tx & Rx links in 8 miniPODs■ PCIe Gen3 x16 lanes interface to host■ supports versatile line rates & timing systems

● TTC; TTC-PON; White Rabbit…

○ capacity:■ 460 Gb/s input/output via optical fiber■ 128 Gb/s to host

○ planned upgrades for ATLAS HL-LHC:■ Ultrascale+ FPGA■ PCIe Gen4 x16 lanes

8

Collaboration between BNL and ANL, CERN, Irvine, NIKHEF, UCL, Weizmann [with FNAL (art_daq)]

LTDBLAr Calorimeter FELIXgFEX

Michael Begel

● Modular readout system for CMOS sensor R&D (CaRIBOu)○ open architecture with ZYNQ SoC to simplify firmware & software○ easily adapted to various sensors under development○ carefully defined interface to minimize design revisions○ used in several test beams with FELIX readout at CERN

■ trigger rate reaches 60+ kHz — much faster than previously available readout system (4 kHz)

○ outgrowth of FELIX & LDRD efforts at BNL

● Adapted for ATLAS HL-LHC Pixel Detector○ important for final development of pixel readout chain with FELIX ○ setup for pixel demonstrator readout development

9

Control and Readout

LGAD CaRIBOuSensor FE

CaRIBOu& FELIX

Control & ReadoutBoard (CaRIBOu)

Sensor Front-End BoardMounted Readout Chip

Telescope at CERN Test Beam with CaRIBOu & FELIX

Michael Begel

● BNL focuses on overall system optimization for integrated detector readout○ inspired by experimental requirements and our physics interests

■ common considerations: flexible, modular, scalable, high-density, high-throughput

○ front-end ASIC design optimized for detector technologies

○ front-end board design tailored for difficult experimental environments■ e.g., cryogenic temperatures, high-radiation environments

○ trigger/DAQ designs that exploit COTS solutions■ eg., high-speed optical links, DSP in FPGA firmware, SoC (ZYNQ+)

Summary

10

LTDBLAr Calorimeter FELIXgFEX

LGAD CaRIBOuSensor FE

CaRIBOu& FELIX

Cold Front-End TPC

FELIX

Michael Begel

Questions?

Al Ross, New YorkerJune 22, 1992

1111

Michael Begel

Close collaboration between BNL Physics & Instrumentation Division

Duke

Penn

YaleLBNL

NYU

Europe

Mechanics

(staves, s

upport, Q

A),

DC-DC converte

rs

Mechanics, Co-cured Stave, QA

UCSC

Mechanics, Serial Power,Systems, HV Multiplexing, etc

HV

Multiplexing,

TID

Irradiation

3D-Trench Detector

Fine Focus Laser Injection

Module testing

HV Multiplexing, LGADs

Stony Brook

3D Trench D

etector

RD50 CERN Rad-hard Detectors

CNM Rad-hard HV Mux JFET

Silicon Assembly

Iowa State

Stav

e C

ore

QC

Module P

ower Board

BNL Silicon

RAL

FBK

12

Brandeis

Michael Begel

High-Voltage Multiplexing for Silicon Strip Sensor Bias

● Radiation-hard switches to disconnect sensors in breakdown from HV bus○ >500 V and operates in magnetic fields○ original idea by BNL○ Panasonic GaNFET is promising candidate

● 3D Vertical JFET○ initial proposal by BNL○ CNM Barcelona designed & testing

3D Vertical JFET○ BNL developing Vertical JFET utilizing

less expensive planar processing

13

Panasonic GaNFET

CNM JFET

HV Mux Control Circuitry

● BNL & collaborators developing rad-hard control circuitry○ extensive irradiation campaign using neutrons, protons & gammas

6mm

BNL’s JFET

Michael Begel

New Silicon Technologies

14

● BNL invested in infrastructure to characterize and improve silicon sensor design for HEP○ class 100 Silicon Detector Laboratory in Instrumentation Division○ Transient Current Technique (TCT) system to map electric field○ TCAD silicon simulation software

● Low Gain Avalanche Detectors (LGAD) for fast timing and precision tracking detectors for HEP○ LGADs produced & tested at BNL and irradiated at LANL with protons○ applications in HEP, Nuclear, Photon, National Security, Medical○ collaborating with BNL Photon Sciences & Nuclear Physics, Santa Cruz,

Paris VI, LAL Orsay, CNM Barcelona, JSI Ljubliana, CERN, INFN Turin

● c e.g. HV-CMOS○ development, testing, electronics, & performance characterization @ BNL○ STAR HFT detector at BNL RHIC first large tracker to use MAPS○ synergistic with LGAD testing & characterization○ collaborating with Geneva, Bern, Liverpool, KIT Karlsruhe

BNL’s LGAD

CERN Test Beam

BNL’s FELIX DAQ & CaRIBOu calibration boards also used in this CERN test beam

Michael Begel

Microelectronics at BNL

● ASIC development successful because of concentration of resources & expertise and close collaboration with experimental teams○ advances in radiation detectors tightly

coupled to advances in front-end ASICs○ front-end ASICs rapidly becoming very

high functionality systems-on-chip (SOC)○ system integration is crucial!

15

ASIC for ATLAS Upgrade

Cryogenic Analog Low-Noise Front-End ASIC

adopted by MicroBooNE, Argontube, CAPTAIN, LArIAT, SBND, 50 L

ICARUS,DUNE 35 ton, proto-DUNE, DUNE

Cryogenic Front-End(8 FE ASICs, 8 ADC ASICs, 1 FPGA)

Michael Begel

Radiation-Tolerant Electronics

16

● Extensive studies on ADCs & FPGAs○ performance, survivability, single- and

double-event upsets○ commissioned 14 MeV neutron source @

BNL for initial component screening○ further tests at other facilities

● Critical need for HEP experiments○ many industrial components not extensively tested by manufacturers○ space- and military-qualified components are expensive○ significant interest from other fields: Silicon Engineering,

Nuclear Physics, Photon Sciences, Medical

● Systematically explore impact of radiation○ close collaboration with university and industry partners:

BYU (M.J. Wirthlin), MSU, SMU, CERN, INFN MilanoAltera, Analog, Hittite, Linear, Microsemi, TI

radiation testing of ABC130 ASIC

at BNL-SSIF

Michael Begel

Silicon Sensors● only U.S. Laboratory pursuing both

HV-CMOS Monolithic Active Pixel Sensors (MAPS) and Low Gain Avalanche Detectors (LGAD)

● depleted CCDs & single-photon detectors● Energy, Intensity, & Cosmic Frontier

experiments and Photon Sciences, Medical Imaging

● collaboration between Physics, Instrumentation, and Photon Sciences with ANL, Bern, Cambridge, CERN, CNM, Geneva, IFAE Barcelona, NYU, Pennsylvania, RAL, UC Santa Cruz, Stony Brook, Toronto, Wellesley, Yale

● supported by BNL LDRDs and DOE Early Career Award

17

The Generic Detector R&D research program relies on the unique capabilities and synergies available at BNL

Microelectronics● world-class development ASIC

development● low-power low-noise front ends● cold electronics enabling large

cryogenic detectors● high-functionality front-end ASICs● radiation-tolerant electronics● Energy & Intensity Frontier

experiments● collaboration between

Instrumentation & Physics with Arizona, Brigham Young, CEA Saclay, FNAL, LBNL, IFIN-HH Romania, IN2P3-LAL, INFN Milano, Michigan, NRL/NASA, Pennsylvania, Sao Paulo, SBU, SMU, Toronto, Washington Univ.

● supported by BNL LDRDs Data Acquisition & Trigger● readout for noble liquid environments● high-throughput data acquisition● high-bandwidth trigger processors● Energy & Intensity Frontier, Nuclear

Physics, Photon, Light Sources● collaboration between Physics &

Instrumentation and ANL, Bern, CERN, Chicago, Cincinnati, Columbia, FNAL, INFN Bologna, Indiana, UC Irvine, LANL, Lund, MSU, Oregon, NIKHEF, Pennsylvania, PNNL, Pittsburgh, SMU, Stockholm, Syracuse, UT Arlington, Weizmann, Wuppertal, Yale

Liquid Argon Detectors● BNL pioneered LAr-based detector

technology● precision sampling calorimeters● time-projection chambers● LAr properties● Energy & Intensity Frontier experiments● close collaboration between

Instrumentation & Physics with Bern, CERN, Chicago, Columbia, CSU, Duke, FNAL, LANL, LLNL, Liverpool, Manchester, MIT, Pennsylvania, Princeton, SLAC, Sheffield, SBU, SMU, Syracuse, Wisconsin, Yale

● supported by BNL LDRDs andtwo DOE Early Career Awards

Water Based Liquid Scintillator● unique facility developed by BNL● LZ, PROSPECT, 0νββ decay (SNO+),

Medical Imaging, Nonproliferation● collaboration between Chemistry &

Physics with Australia National, UC Berkeley, UC Davis, LBNL, Oxford, Penn, SBU, Tsinghua, Yale

● supported by BNL LDRDs

Michael Begel

Summary of Current BNL Contributions to ATLAS

18

class 10000 Clean Roomfor HL-LHC Silicon Tracker

VMM3 ASIC

Capability Name Capability DescriptionHiggs Physics BNL leads SM Higgs & precision Electroweak measurements in diboson final states;

H→ ZZ*, WW, ZZ, same sign WW, combinations & anomalous gauge couplingsIndirect Dark Matter Search for H→Zdark in ZZ final statesNew Physics Searches BNL established search for SUSY stop in all-hadronic decay channel;

Leadership in BSM HiggsSoftware & Computing Core, Trigger & Analysis Software for ATLAS including framework, reconstruction

algorithms, performance, analysis tools and supportDetector Comm. & Operations

ATLAS Detector Operations in Muon CSC, LAr Calorimeter, Calorimeter Trigger; Jet/ETmiss; Management of US ATLAS Operations Program

Software for Distributed Computing

Software for distributed computing on Grid, Cloud, & HPC; PanDA for ATLAS workflow management

BNL Scientific Data & Computing Center

ATLAS Tier-1 computing center; shared Tier-3 computing for US ATLAS community

Detector Design & Subsystem Fabrication

Key leadership in scope & design of ATLAS Phase-I Upgrade; BNL deliverables for LAr Calorimeter electronics (LTDB), Calorimeter Trigger (gFEX), DAQ (FELIX), Muon NSW ASIC (VMM) & Trigger Board (ADDC); Management of US ATLAS Phase-I Upgrade Project

Detector Integration & Testing

Critical roles in System Integration for Calorimeter, Trigger, & Muon in ATLAS Phase-I Upgrade

Detector Design & Subsystem Fabrication

Leadership of ATLAS HL-LHC Upgrade; BNL responsible for half of Silicon Strip Barrel Tracker; deliverables planned for Global Trigger, DAQ, LAr Front-End, high-granularity Timing Detector; Management of US ATLAS HL-LHC Project

Community Support BNL hosted 21 students, post-docs & professors in FY16-18 on ATLAS activities

H→ZZ*

Michael Begel19

BNL is a key leader of the ATLAS Experiment in Physics, Detectors & Computing

BNL is host laboratory for:● U.S. ATLAS Operations Program● U.S. ATLAS Phase-I Upgrade Project● U.S. ATLAS HL-LHC Upgrade Project● ATLAS Tier 1 Computing Facility● U.S. ATLAS shared Tier 3 Computing● U.S. ATLAS Center (with ANL, LBNL, SLAC)

BNL hosted 17 ATLAS students, post-docs & professors in FY17● 4 DOE Office of Science Graduate Student Research Fellows● others supported by U.S. ATLAS, China Scholarship Council,

and/or University funds ● from Amherst, Brandeis, Chicago, Dresden, Harvard, Oregon,

NIU, Oxford, Stony Brook, USTC

JHEP 10 (2017) 132

JHEP 01 (2016) 172

Searches for New Physics& Dark Matter

Higgs & ElectroweakPhysics

Detector Upgrades

Higgs Boson

Z Boson

ATLAS