Plans for EPICS in Hall D at Jefferson Lab
Elliott WolinEPICS Collaboration Meeting
Vancouver, BC30-Apr-2009
Jefferson Lab: Newport News, Virginia6 GeV Continuous Electron Beam Accelerator Facility
Superconducting RFQ’s
Three existing experimental halls
Approved 12 GeV upgrade and new hall $310M
GlueX experiment:
200 kHz trigger
3 GB/s off detector
300 MB/s to tape
GlueX Experiment in Hall D
Search for mesons with gluonic excitations
• 200 kHz trigger rate• Deadtimeless readout• 15 kB event size• 3 GB/sec to L3 farm• Factor 10 L3 rejection• 300 MB/s to tape• Data taking in 2014
Experiment vs. Accelerator Operations
• Trigger/DAQ-centric, not controls-centric– trigger and DAQ hardware more problematic– high-speed data path is critical
• Run-oriented– many configuration changes– sometimes every 5 minutes– hardware download/readback speed critical– may bypass normal controls path for speed
5
ROC
ROC
ROC
ROC
ROC
ROC
EMU
EMU
EMU
EMU
EMU
EMU
Level 3 Farm
90% Rejection
EMU
RAID
SILO in Computer
Center
2 PB/year
60ROCs
8EMUs
8EMUs
ManyCores
1EMU
16 TB16 hrs
Event BuilderCrossbar
DAQ at High Luminosity3 GByte/sec
3 GB/s 3 GB/s3 GB/s
300 MB/s300 MB/s
300 MB/s
6
Type Number of Devices
Control Points per Device
Channel Count Control Mechanism
FDC cooling system 20 1 20 PLC
Cryogenic target 40 1 40 PLC
Solenoid magnet 200 1 200 PLC
Gas systems 40 1 40 PLC
Misc 50 1 50 PLC
Beamline Many Varies 200 PLC/EthernetGoniometer 6 axes 10 60 Ethernet
Tagger 750 4 3000 EthernetVXS, VME, HV, LV crates 80 15 1200 Ethernet
HV, LV channels 2300 4 9200 Ethernet
Discriminators 2800 1 2800 Ethernet
Cockroft-Walton bases 2800 5 14000 CAN
Board temperatures 1575 1 1575 SMBus
Misc 200 1 200 Ethernet
Software Channels 200 1 200 Ethernet
TOTAL 32785
Controls/Monitoring Channel Counts
Hardware
Electronics
7
Experiment Control System(ECS)
Supervisory Control and Data Acquisition System
(SCADA)
Detector Control System(DCS)
Hall D Three-Tier Controls Architecture
Allen-Bradley
PLC
Magnets Target Gas Cooling
HVSystem
LVSystem
OtherSystems
High-level state machines and operator GUI’s.
(AFECS)
Communication, archiving, alarms
(EPICS)
Autonomous detector control, low-level state machines.
(PLC, Ethernet, CAN, SMBus)
8
• Detector Control System– 24x7 autonomous operation– All PID and control loops reside here– No EPICS at this level
– Allen-Bradley PLC• engineers prefer simple analog sensors/actuators• will have some Ethernet devices
– High-level manufacturer-supplied controllers• HV, LV• goniometer• CAN, SMBus controllers
9
• SCADA - Supervisory Control and Data Acquisition– EPICS• JLab has extensive experience with EPICS
– accelerator and all three existing halls• Will use Linux-only EPICS, no VxWorks or RTEMS• CSS and/or LabView for non-expert graphics
– few restrictions on expert screens• Integrate with publish/subscribe package (cMsg)• Alarm system – BEAST (SNS) or LASER (CERN)• Many other tools in EPICS toolkit
– archiving – from SNS or JLab– strip charts – StripTool, SNS, JLab or Labview– save-and-restore facility – SCORE from SNS– web – CAML, WebCA– analysis - Labview
10
• Experiment Control System
– AFECS • Agent Framework for Experiment Control Systems• Java agent-based control system from JLab DAQ group• Used for run control• Allows integration of slow controls with run control
– high-level state machine scripting language– hierarchical – similar to SMI++ used at LHC
11
Alarm System• Goals – are these satisfied by BEAST? LASER?
1. All alarms require operator action• otherwise “alarm” is a nuisance
2. Single problem gives single alarm• alarm filtering/reduction, flood suppression
3. Context sensitive alarms• no alarm on device if not being used
4. Alarm shelving• remove from alarm screen
Labview
• Considering for majority of operator screens• Channel Access modules available– client for Linux (ORNL)– waiting for Linux server
• Basically just another GUI builder– like EDM, CSS– superior analysis capabilities
Conclusions
• Using EPICS for SCADA level (middle tier)– Linux only– no control loops in EPICS
• Cherry-picking best applications– Many from SNS/ORNL
• May use Labview extensively
Please give me comments and suggestions!
Backup
15
DetectorLevel 1
Energy SumTrigger
TriggerSupervisor
Module
Trigger
Inputs
Add’lTrigger
Logic
OutputPulser 1
Pulser n
Calib 1
Calib n
Trigger Signal
Full Experiment Trigger
16 bits
16 bits
16
PLC Redundant Chassis LayoutSeven Module Chassis
Power supplyPLC
Redundancy Module
CommunicationModule
Remote ChassisCommunicator
Blank Slots
17
Ten Module Remote Chassis
Field Devices:- temperature sensors- pressure gauges- vacuum gauges- cryogenic level gauges- relays- voltage, current gauges- flow meters- many others
Power supply
Remote ChassisCommunicator
DC Input Module