Upload
arlene-barber
View
216
Download
0
Embed Size (px)
DESCRIPTION
3 A brief history of time … Oct. 2001: FNAL decided to build Pulsar board UofC decided to provide engineer support soon Upenn purchased obsolete interface components Mar. 2002: INFN built SVT-TSI-PCI daughter card Aug. 2002: ANL possibility with Atlas RoIB Oct. 2002: Pulsar works with ANL Gigabit Ethernet mezzanine card (essentially plug&play) ANL/FNAL/INFN/UofC/Upenn We have a strong group, now we need a good plan
Citation preview
First group planning meeting: Introduction
• Introduce new people• A brief history of time• Goals of this meeting• Project overview• Roadmap • Guideline for the project planning
…real talks by others…
• Some starting points for discussion session(see web). Initial thoughts (or targets for you to shoot at): near term hardware needs/production plan manpower/organization/near term goals … future meetings & L2 upgrade e-log
- Ted Liu, Jan. 30th, 2003
2
New people who joined the project recently:
• Burkard Reisert (FNAL new posdoc)• Cheng-Ju Lin (FNAL posdoc)• Paul Keener (Upenn, DAQ expert)• Joe Kroll & his $$ (Upenn)• Kristian Hahn (Upenn, Nigel’s student)• Shawn Kwang (UC, Mel’s student)• Bob Blair (ANL)• John Dawson (ANL)• Jimmy Proudfoot (ANL)• Franco Spinella (INFN)
“Old” people who is eager to come back:• Sakari Pitkanen (still waiting for visa @ Finland) => youngest collaborator
This project is attracting the best people/groups @ CDF …
3
A brief history of time …
Oct. 2001: FNAL decided to build Pulsar board UofC decided to provide engineer support soon Upenn purchased obsolete interface components
Mar. 2002: INFN built SVT-TSI-PCI daughter card
Aug. 2002: ANL possibility with Atlas RoIB
Oct. 2002: Pulsar works with ANL Gigabit Ethernet mezzanine card (essentially plug&play)
ANL/FNAL/INFN/UofC/Upenn
We have a strong group, now we need a good plan
4
Goals of this meeting
• Focus on planning, not technique details• Roadmap • Near term goals and long term plan• Manpower availability (current/future)• Who is interested in working on what&when• Get a clear picture of the issues/tasks involved will have a few technique talks
5
Project Overview
may roughly divide into four “parallel” efforts:
• E1: Pulsar hardware/firmware/VME software
• E2: Design spec. for data format/algorithm/system interface
• E3: RoIB option effort (see Bob’s talk)
• E4: PCI/CPU architecture/OS/Infrastructure SF/L2D SF …
E1E2E3E4
Phase A Phase B Phase C
CDF RunIIb L2
Atlas L2
6
Roadmap may divide into three phases/stages:
• Phase A: Teststand, core firmware, HW prod/testing Algorithm firmware specification PCI/CPU/OS/infrastructure software
will show some details • Phase B: Rx algorithm firmware implementation/testing, system integration in standalone mode
• Phase C: System integration in parasitic mode, test runs with cosmic and beam
7
TRK
Control/ Merger
TS
L1
Data IOTRK
Mezz cardconnectors
Pulsar design
9U VME(VME and CDF ctrl signals are visible to all three FPGAs)
P1
P2
3 Altera APEX 20K400_652 (BGA) FPGAs 502 user IO pins each
P3
SLINKsignal lines
Data IO
spare lines
SRAM
SRAM
128K x 36 bits
3 APEX20K400 FPGAs on board = 3 Million system gates/80KB RAM per board2 128K x 36 pipelined SRAMs with No Bus Latency: 1 MB SRAM (~5ns access time)
8
Gigabit EthernetRF clock
SRAMs
Pulsar is designed to be fully self-testable: at board level as well as system level
For every input/output, there can be an output/input, All interfaces are bi-directional (except VME)
It is this feature which allows us to develop&tune an upgrade system in standalone mode Overall planning should be based on this feature
SVT/XTRP
TSI
one for all andall for one
9Main Injector
(new)
TevatronDØCDF
Chicago
p source
Booster
αSVT
XTRP
CLIST
ISO
L2 decision crate
L1
RecesHP scope
LogicalAnalyzer
HP scope
The way we have been debugging the Level 2 decision crate: very often need to use CDF+Tevatron HUGE amount of work has been done this way by a few hardworking experts
The idea is to build test stand tools to “replace” CDF and Tevatron, to make life MUCH easier.
L2 inputs
TS
Why diagnostic tools?
An old slide from an old talk on Dec. 7th, 2001: original motivation to build Pulsar: reduce impact on running exp.
10
Main Injector(new)
TevatronDØCDF
Chicago
p source
Booster
A slightly modified version for funding agency
CDF can be used asWeapons of top mass construction
Pulsar can be used as Weapons against mass distractionto reduce impact on running experiment
L2Temulation
Pulsar can be used to emulate both CDF/Tevatron and ATLAS/LHC for L2T
realdata
11
Hardware setup during Phase A:
PC
(1) Pulsar Tx => Rx =>PCI=>PC w/ 4 hotlink mezzanine cards (muon/L1/SVT/XTRP/TSI paths)Fully develop teststand capability (should test with RunIIA boards)Develop core firmware for all casesRobustness/Production test setup (see Burkard’s talk) (2) Pulsar Tx => Rx => PCI=>PC 4 Taxi mezzanine cards (Reces path)(3) Pulsar Tx => Rx => PCI=>PC 2 Hotlink (Cluster) & 2 Taxi (Isolation)
Tx
Rx
12
Core firmware example: DataIO FPGA (Rx case) => Control FPGA is similar
DAQ buffersSRAMinterface
VME responses
RAM/Spy
buffers
RAM/ Spy
buffers
MezzanineCard interface
CDF Ctrlinterface
upstreamdownstream
Blocks in green: common to all FPGA types, blocks in whiteare data path specific (core firmware only deal with thesimplest case: pack ALL data into SLINK format)
Play/recordPlay/record
•Firmware was designed this way from day one
SLINKformatter
Filteralgorithm
13
Well organized firmware effort is one of the keys to have a system:clean/easy-to-understand/robust & with minimal maintenance effort
VME responsesSLINK formatter
DAQ buffersCDFCtrl responses
SRAM ctrlMezzanine interfaces
Diagnostic RAMsSpy buffers
…
Common core firmware
Knowing one data path Pulsar knows all PulsarsData specific algorithm difference is just minor details
hotlink
muon
cluster
Taxi Isolation
Reces
LVDS+ ext. FIFO
SVT
XTRP
LVDSL1
TSI
Need dedicated people responsible for developing the core firmware
14
(1) In God we trust, everything else we test
(2) One for all and all for one
Summary of what I have said orGuideline for the project planning:
Project planning should follow Pulsar design philosophy
Anyone who has a problem with this is in the wrong project
15
Goals of this meeting
• Focus on planning, not technique details• Roadmap • Near term goals and long term plan• Manpower availability (current/future)• Who is interested in working on what&when• Get a clear picture of the issues/tasks involved
will have a few technique talks…
By now, hopefully you are all warmed up…
16
Agenda
• Introduction (TL 20mins) • Pulsar hardware status (Mircea, 5mins)• Automating Pulsar test procedures (Burkard, 10mins)• L2 data format/real data patterns (Cheng-Ju, 10mins)• Pulsar algorithm firmware spec. for Iso&Reces (Bob, 20mins)• Status/plan of S32PCI64/CPU/OS study (Paul, 20mins)• ATLAS RoIB option (Bob, 20mins)• Discussions (ALL, 60mins)• Summary (5 mins)
• dinner at 6pm
17
Material (some initial thoughts) for discussion session
• Some details on hardware needs during phase A/B/C • Manpower/organization
• Near term goals for each effort
• Future meetings & L2 upgrade e-log
• More material can be found on the meeting web pagehttp://hep.uchicago.edu/~thliu/projects/Pulsar/ L2_upgrade_meeting.html
18
Hardware setup during Phase A1:
PC
(1) Current setup (since Nov 02): Pulsar Tx => Rx =>PCI=>PC with 4 hotlink mezzanine cards (for muon/L1/SVT/TSI path)• uses CERN AUX card• uses old SLINK to PCI card &• uses ANL Gigabit Ethernet card• uses an old PC
Tx
Rx
Will need to: • test with our AUX card (next month)• need 4 more hotlink mezz cards as reference cards to test new Pulsars
Some details
19
Hardware setup during Phase A2:
PC
(2) Next setup (March 03 -- ): Pulsar Tx => Rx =>PCI=>PC with 4 Taxi mezzanine cards (for Reces path)
Tx
Rx
Will need to: • test the two new Pulsars with the 4 hotlink mezz. reference cards first
•2 more Pulsars from prototype run (ready for testing next week)• 4 Taxi mezz. cards (next month)
20
Hardware setup during Phase A3:
PC
(3) Next setup (~April 02): Pulsar Tx => Rx =>PCI=>PC with 2 hotlink/Taxi cards (for Cluster/Isolation paths)
Tx
Rx
Will need to: • test the two new Pulsars with the 4 hotlink mezz. reference cards first• new PC with new PCI card
• 2 more Pulsars from pre-production• 2 hotlink/Taxi mezz. cards
Once this setup is fully developed&tested: time for production
21
Pulsar hardware needs during Phase A (summary):(1) Current setup: Pulsar Tx => Rx =>PCI=>PC with 4 hotlink mezzanine cards (for muon/L1/SVT/TSI paths) * 4 more hotlink mezzanine cards as reference cards for new motherboards testing * one AUX card (5V) (2) Next setup: Pulsar Tx => Rx => PCI=>PC with 4 Taxi mezzanine cards (Reces path) * 2 more Pulsar boards from prototype batch (next week) * 4 Taxi mezzanine cards from prototype run * one more AUX card (3.3V)(3) Next setup: Pulsar Tx => Rx => PCI=>PC with 2 Hotlink (Cluster) & 2 Tax(Isolation) * 2 more Pulsar boards from pre-production run * 2 Hotlink/Taxi mezzanine cards
Production starts after (1) –(3) fully tested
22
MUON
RECES
ROC
TRACER
CLU/ISO
RECES
RECES
RECES
MUON
RECES
ROC
TRACER
CLU/ISO
RECES
RECES
RECES
Tx
Rx
Teststand Area
…
Hardware setup for Phase B:Does RoIB need a separate crate ?
•Initial system level integration and performance studies•Finalize data path specific firmware, reduce the data size as necessary• online monitoring software• …
TSI
23
MUON
RECES
ROC
TRACER
CLU/ISO
RECES
RECES
RECES
Rx
L2 Central Crate inTrigger Room
…
Hardware setup for Phase C:
•Initial system level integration with real system• fine tune FW/SF/performance• parasitic running if possible• special test runs with cosmic and beam• …
In addition to the teststandarea setup in standalone mode,setup the full system (Rx) in L2 central crate next to thecurrent L2 decision crate
24
Project Overview
may roughly divide into four “parallel” efforts:
• E1: Pulsar hardware/firmware/VME software
• E2: Design spec. for data format/algorithm/system interface
• E3: RoIB option effort (see Bob’s talk)
• E4: PCI/CPU architecture/OS/Infrastructure SF/L2D SF …
E1E2E3E4
Phase A Phase B Phase C
25
(E1) Pulsar hardware/firmware/VME software
Pulsar/mezz/Aux production: UC shop Production testing: Burkard/Sakari+Kristian + … Technical support: UC(Harold/Mircea)/FNAL(Bob Demaat) Infrastructure support: Crates/TSI/Testclk/Tracer: CJLin/PWittich
Core firmware spec/implementation: Burkard/Sakari + others Core VME software(testing,online): Burkard/Kristian/Sakari subsystem specific algorithm firmware/VME software: muon path: Shawn with support from Sakari/Burkard Reces path: ? … Cluster path: Kristian … Isolation: Kristian … L1/XTRP/SVT: part of the core firmware Overall coordinator: Burkard Reisert Consultant: Mel Shochet/Bob Blair/John Dawson/Peter Wilson/Jonathan Lewis...+ volunteers
26
E1: possible near term goals
(1) Feb: have the automated testing procedure fully working for 16
hotlink channels (muon path): Tx->Rx->PC;
(2) March: finish testing 2 more Pulsar boards with hotlink/Taxi mezzanine
cards and AUX cards;
(3) April: initial proof of principle test with S32PCI64 with test setup
(4) May: core firmware (Tx/Rx) fully developed and tested with
hotlink/taxi mezzanine cards;
(5) June: production/testing for Pulsar/mezz cards/AUX cards
(6) July: have Tx->Rx-PC fully developed for Reces(taxi), cluster/Isolation (hotlink/taxi);
(7) Aug: document everything above: cdfnote
27
(E2) Firmware spec./SLINK data format/data patterns
System Level(TS handshake, DAQ etc): Peter/Mel/Bob Blair/John + others Subsystem specific: muon: Mel/Cheng-Ju/Shawn Kwang Reces: Bob Blair/Cheng-Ju/? Cluster: Cheng-Ju/Mel/Kristian Isolation: Bob/Cheng-Ju/Kristian L1/XTRP/SVT: Cheng-Ju/Burkard/Mel Overall coordinator: Cheng-Ju Lin Consultant: Jonathan Lewis/Peter Wilson/Henry Frisch/Jimmy Proudfoot/... + volunteers
28
E2: possible near term goals
(1) Feb: document ALL L2 data format; (2) March: software ready to derive all real data
patterns; (3) April: initial algorithm specification for each
path; (4) May: software modeling based on algorithm
specifications online monitoring software(5) June: finalize algorithm/system firmware
specification(6) July: document everything above --> cdf note
ready
29
(E4) PCI/CPU/OS/infrastructure software/L2D software
need more thinking here for the organization…need a posdoc in this area ?
R&D setup at Upenn: Paul Keener PCI/CPU/OS/infrastructure software development Upenn has funding from the university for this R&D effort.
Testing/integration setup at ANL: Bob Blair/John Dawson +?
Testing/integration setup at FNAL: Kristian + ?
L2 algorithm software: Peter Wittich/Bob/Cheng-Ju/Kristian/Burkard...
Overall coordinator: Peter Wittich
Consultant: Jim Patrick/Rick van Berg/Joe Kroll/Franco Spinella/Bill Ashmanskas...+
30
E4: possible near term goals
(1) Feb: at the very least repeat CERN's results on S32PCI64 performance;
(2) March: pick one PC platform/OS with reasonable performance
(doesn't have to be the best)(3) April: basic infrastructure software ready early April and
transfer to FNAL teststand, along with a PC/OS. This will be used
for proof of principle test at FNAL in April and teststand
needs(4) May-June: more studies to see if there is any better choice
for PC platform/OS. study on how to send the L2 decision
back to Pulsar (5) July: final decision on PC platform/OS... and
infrastructure software... (6) Aug: final decision on how to send L2 decision back to
Pulsar. (7) Sept: document everything above --> cdfnote ready
31
Future meetings • Group meetings: Once per month is reasonable I would suggest that each groupANL(Bob)/FNAL/INFN(Luciano)/UC(Mel)/Upenn(Joe) take turn to organize/chair the meetings
• Subgroup meetings during phase A:
bi-weekly regular meetings or let each coordinator decide when to call the meetings?
• create a L2 upgrade e-log? too many meetings too much BSWe have good people who want to spend more time to do real work Based on years of observation: much less BS on e-log an effective way to improve communication
32
RunIIA and RunIIB
2B or not 2B?