Introduction to Machine protection for CNGS (and TI2/8)

05.10.2006 CNGS Interlock System Training / J. Wenninger 1

Introduction to Machine protection for CNGS (and TI2/8)

• Introduction to the new Beam Interlock Systems

• CNGS extraction and protection requirements

• Interlocks : what, why, how?

• Operational aspects & issues

J. Wenninger AB-OP


Introduction

The aim of this presentation is to give an overview over :

• The new Beam Interlock Systems for SPS (and later LHC). In 2007 they will have replaced the present SPS Emergency System.

• Details on the interlocking for the CNGS fast extraction.

• Operational experience with the CNGS line and interlock system during the short CNGS run 18th -30th August 2006.

It is not possible to go into all details (time !). But the entire system is described in detail on the WEB page

https://cern.ch/sps-mp-operation/

https://sps-mp-operation.web.cern.ch/




Beam Interlock Systems for SPS/LHC

Requirements:• Generic solution for CERN• Reliable• Available• Fast


User Inputs

• The user inputs are partitioned into NOT MASKABLE and MASKABLE inputs.– Note : MASKED signal = DISABLED signal.

• NOT MASKABLE inputs are always active.

• The MASKABLE inputs can be masked provided the SAFE BEAM FLAG is TRUE.

• The SAFE BEAM FLAG is :– TRUE if BEAM_INTENSITY * BEAM_ENERGY is considered SAFE – i.e. no / little risk of damage. Needs input from BCT(s) and

potentially (not at the SPS) main converter currents.

– Presently distributed by the timing system (as a special telegram).

Beam

Interlock

System

Kicker SystemBEAM_PERMIT

MA

SK

AB

LE

NO

TM

AS

KA

BL

E

User System #1

User System #2

User System #n

User System #m

USER_PERMITS

SAFE_BEAM_FLAG

SBF= distributed information to provide some flexibility while maintaining safety.


Interlock Signal ConnectionsBIS Rack

User System

Permit ‘A’

Permit ‘B’

‘A’

‘B’

User System Rack

User InterfaceBeam Interlock

Controller

■ On copper cable up to 1.2km

■ Standard interface for all interlock ‘clients’ at the SPS & LHC

■ Receives the USER_PERMIT signal (redundant) and transmits it to the nearest BIC

■ Allows direct connection with many different types of hardware platform (PLC, VME, etc…)


BIS User Interface (CIBU)

• Standard interface for the user signals.

• Accepts various standards (VME/TTL/PLC) as input.

• Installed in the ‘client’ rack.

• Remote tests and diagnostics.

• Redundant power supply.


BIS VME Crate

• Cables are connected with a special patch panel at the back of the BIS VME rack.

• NO cabling inside the rack: signal routing proceeds over printed circuit boards.

• Fixed and well defined topology : cable VME module

One of the TT40 racks


BIC Module

The Beam Interlock Controller or BIC is the VME module that collects the User-permit (interlock) signals. It is the heart of the interlock system!

Each BIC has 14+1 inputs :

7 non-maskable HW inputs. Always active !!

7 maskable HW inputs, that can be masked provided the ‘Safe Beam Flag is TRUE. At the SPS the ‘Safe Beam Flag’ is generated 1 second after the last injection from an intensity measurement of the hadron BCT in LSS3. It is distributed by the timing system to the BICs. Presently the limit is set to 1012.

1 software input to be set through the standard controls network by SIS.

A standard BIC provides an output signal corresponding to the logical AND of all inputs.

IMPORTANT : unlike the SPS Emergency Dump System, the SPS BICs do not LATCH any signal !

BIC modules can be connected to each other:

Interlock loop (fiber optic link), used for the rings.

BIC output BIC User-permit input (‘daisy-chain’), used for transfer lines.

BIC internal history buffer:

All transitions are logged and time-stamped (to the s).

of the BIS


Schematic BIS Hardware LayoutPatch panel

1 or 2 BICmodules

CPU + CTRPModules

VME crate with LynxOS

Control Network

Application

UserPermits

User Interfaces

User Interface

Opticalmodule to Kicker

10

Future (2007) BIS Architecture SPS

2 Beam_Permit_LoopsClockwise & Anticlockwise

BPC

BIC

BPCBPC

BIC

BIC

BIC

BIC

BIC

BA3

2

BA2

BA1

BA6

BA5

BA4

•One BIC / BA.•Connected by an interlock

loop.•Connected to beam dump in

BA1.•Details to be sorted out.

SPS BEAM DUMPSYSTEM

User System User System User System

User System

User SystemUser SystemUser SystemUser System

User SystemUser SystemUser SystemUser System

User System User System User System

User System

User System

User System

User System

Use

r Sys

tem

User System

User System

User System

User S

yste

m

2006 status :•3 BICs for MKE & extraction

BLMs in BA4 and BA6.•Used by new Software

Interlock System (SIS).•Enters as ‘Summary BIS’

into SPS emergency dump system.


Extraction Interlocks for CNGS/TI8

TT41 line

TI8 line

TT40 line

CIBC-TT40A

CIBC-TT40B

CIBC-TT41A

CIBC-TT41B CIBC-TI8up CIBC-TI8dwTED TED

CIBC-EXT2

Injection region

& TI8 after TED

CNGS target

to MKE 4to MKE 4

Upstream part Downstream part

InstalledInstalled for 2007 Temporary link/loopCIBC = BIC module


CNGS Interlocking Overview


East Extraction for CNGS

Extract 2 x 10.5 s long SPS batches, nominally 2.4e+13 protons per batch at 400 GeV

EQUIPMENT:• 5 extraction kicker magnets (MKE4):

– rise time: 1 s, kick length: 11.3/12.1 s, voltage: 50 kV• 6 septum magnets (MSE.418)

• TPSG – protection element for MSE

• 4 extraction bumper magnets: – 31.5 mm extraction bump

• enlarged aperture quadrupole magnets

• instrumentation

CNGS

CNGS Extraction Timing• The CNGS cycle length is 6000 ms and the 400 GeV flat top is 90 ms long (from 4200 to 4290 ms).

• The two extractions are programmed at 4220 and 4270 ms.

• The RF extraction pre-pulse is programmed to arrive in the millisecond after (nominal) extraction time.

• For the extraction kicker 2 timings/moments are crucial : 13 ms before extraction trigger PFN charging (if EXTRACTION_PERMIT = TRUE)

arrival of extraction pre-pulse trigger magnets (if PFN charged and EXTRACTION_PERMIT = TRUE)

Legacy events New events RF extr. pre-pulse (RF2)

The timing should be the same on ALL CNGS users !Please do not change it - it has consequences on interlocks, logging…


CNGS Protection

Protection of the extraction channel and the transfer lines (TT40, TT41) – similar to the requirements for TI8 / LHC.

Protection of the T40 target (and what is behind) : At high intensity it is important to hit the target within 0.5 mm of the axis

(target rod 4 & 5 mm) to avoid damage to the rods.

A single (isolated) hit can (must !) be accepted, but we must avoid sending

two consecutive extractions (50 ms) off-axis.

T40 intercepts the fast extracted 400 GeV proton beam

with two 10.5 µs long spills spaced by 50 ms.

The nominal beam intensity is 4.8x1013 protons per

cycle, the ultimate intensity 7x1013 protons.

The nominal normalised emittance is ~ 12 mm mrad,

the beam size @ T40 is ~ 0.5 mm.

16

CNGS BICs & User Signals in 2006

CIBC.TT40A

Vacuum TT40 (1)

WIC TT40 (10)

MKE status (31)

MSE magnet (12)

TT40 PC (18)

MSE PC (19)

Bumper PC (17)

FMCM MSE (23)

CIBC.TT40B

OP Inhibit (13)

TED TT40 (4)

BTV TT40 (8)

BLM TT40 (28)

BPM LSS4 (26)

Beam Intensity (27)

SIS TT40 + LSS4

CIBC.TT41A

Vacuum TT41 (2)

WIC TT41 (11)

TT41 PC (20)

Main Bend PC (21)

Main Bend DCCT (22)

FMCM MBSG (24)

FMCM Main Bend (25)

SIS TT41 + CNGS

CIBC.TT41B

TBSE TT41 (3)

CNGS Shutter (5)

T40 Target (6)

BTV TT41 (9)

BLM TT41 (29)

BPM TT40+TT41 (30)

T40 Target (7)

Horn/Refl. Status (14)

Hadron Stop Cooling (15)

Fire Alarm (16)Red = un-maskable, Green = maskable, Magenta = Soft. Interlock

In () : the interlock index that I will referred to later when I describe the inputs.

The logical AND of all signals is sent as EXTRACTION_PERMIT to the extraction kicker MKE4 which fires only if the EXTRACTION_PERMIT is TRUE.


Fast Extraction Interlock Types

For the CNGS/TIx fast extractions there are 3 types of interlocks based on :

• Continuous surveillance of parameters. The associated permits change their state rather ‘rarely’ .

• Vacuum, WIC, TEDs, target…

• Pre-extraction surveillance where the user permits are evaluated a short time BEFORE extraction. The associated user permit is FALSE by default and switches to TRUE for a short time interval around extraction if all conditions are correct.

• Surveillance of the beam position around extraction point and of the PC currents.

• Post-extraction surveillance where the user permits are evaluated AFTER extraction. This type of surveillance concerns beam instrumentation. The associated user permit is switched to TRUE for a short time around extraction. The user permit is latched (FALSE) at the level of the client if a measured beam parameter is out of tolerance.

• Beam losses and beam positions in the transfer lines.

• Both Pre- and Post-extraction surveillance tasks are triggered by machine timing events coupled to the main extraction event.


Continuous Surveillance


Targets, Absorbers & Obstacles / 1

• Vacuum valves (1, 2) :• USER_PERMIT=TRUE only when all valves are OPEN. NOT MASKABLE !

• All valves are interlocked.

• TBSE block (3) :• USER_PERMIT=TRUE ONLY when TBSE is OUT of beam. NOT MASKABLE !

• TT40 TED (4) :• USER_PERMIT=TRUE when the TED is IN beam or OUT of beam (both positions are SAFE). NOT

MASKABLE !

• In 2007 are more subtle logic (with automatic masking/unmasking of downstream interlocks) will be implemented.

• CNGS (decay tunnel) shutter (5) :• USER_PERMIT=TRUE ONLY when the SHUTTER is OPEN (position ~ 0 mm). NOT MASKABLE !

• T40 Target (6,7) :• 2 USER_INPUTs, one NOT MASKABLE, one MASKABLE.

• For normal operation USER_PERMIT=TRUE ONLY when the target barrel is positioned with a rod assembly in beam.

• We (OP) have NO CONTROL over the target position !


Targets, Absorbers & Obstacles / 2

• TL screens (8,9) :• The TL screens have 4 positions: OUT, Carbon OTR, Titanium OTR and Alumina screen.

• USER_PERMITs are MASKABLE.

• USER_PERMIT = TRUE : OUT position + Carbon OTR

• USER_PERMIT = FALSE : Al screen, Titanium OTR or device moving (should move during beam out).


Magnet Interlocks

• A WIC (Warm Magnet Interlock Control) system surveys the magnet temperatures of TT40, TT41,

TI8, TT60 and TI2 (10, 11) :• USER_PERMIT=TRUE only when all temperatures are OK. NOT MASKABLE !

• When a sensor detects an over-temperature or a local IO module dies (partly installed in the tunnel), the system FIRST sets USER_PERMIT=FALSE, and 1 second later switches off the concerned PCs.

• Provides 2 separate inputs for TT40 and TT41.

• It is a PLC based system, at some time we should/will get the surveillance application.

• MSE.418 magnet and girder state (12) :• USER_PERMIT=TRUE ONLY when:

Girder is IN-BEAM and within tolerance (window of +- 2 mm, reference is HARDCODED).

Magnet temperatures & cooling are OK.

MSE PC is ON.

• NOT MASKABLE !

• Whenever an interlock on the magnet is generated, the USER_PERMIT is switched to FALSE ~ 100 ms BEFORE the PC is switched off.


Miscellaneous Inputs

• CCC button / OP inhibit (13) :• USER_PERMIT=FALSE if the button ‘LSS4 Inhibit’ is pressed (second button from the right).

• NOT MASKABLE !

• Equivalent of the console switch for the fast extraction.

• Horn & reflector status (14) :• USER_PERMIT=TRUE when both HORN and REFLECTOR are ON. MASKABLE !

• Hadron Stop Cooling (15) :• USER_PERMIT=TRUE when the cooling circuit of the CHGC hadron stop is running. MASKABLE !

• Fire Alarm (16) :• USER_PERMIT=FALSE when a fire is detected in the CNGS target cavern. MASKABLE !


Software Interlock System

• The new SIS (Software Interlock System) that will replace SSIS (2007 ?!) provides two software interlock inputs.

• It provides additional (and slower) cross-surveillance and diagnostics. Next year it will also directly cut the beam (based on destination) through the timing system.

• TT41 software surveillance:

• Summary of PC states (TT41), target position, BTV positions, TBSE position, Shutter, BLM gains (avoid saturation) and thresholds, etc is sent to BIC module CIBC.TT41A.

• The status is refreshed for every CNGS user (CNGS1/2/3/4) – at the end of the cycle.

• TT40 and LSS4 surveillance:

• Summary of PC states (TT40 & LSS4), BTV positions etc is sent to BIC module CIBC.TT40B.

• The status is refreshed for every CNGS or LHC user (CNGS1/2/3/4, LHC1/2, MD1/2/3/4) – at the end of the cycle.

• CNGS Protection does not depend critically on SIS. In case of SW problems you can therefore mask channels that give problems. This year we had quite frequently problems with the PC surveillance, and frequently had to mask all PC interlock (will have to be fixed next year).


Pre-extraction Surveillance


Powering failures

TT41 Main Bends

Tol.Tolerance

Powering ‘failures’ are among the most likely and most critical failures :

• Wrong converted setting surveillance of the current VALUE.

• Converter failure FAST surveillance of the current CHANGE.

Examples of simulated powering failures

Tolerance


ROCS PC Current Surveillance

• The ROCS system provides a pre-extraction surveillance, the so-called FEI (Fast Extraction Interlock). The current of selected converters has to match a reference within a pre-defined tolerance. The surveillance is performed at the last possible moment ~ 2 milliseconds before extraction.

• This system provides in total 6 inputs to the BICs, all inputs are MASKABLE:

LSS4 bumper converters (H+V) (17) TT41 converters (20)

TT40 converters (18) MBI main bend converter (21)

MSE.418 converter (19) Interlock DCCTs for shared main converter (22)

• Operational current tolerances :

MBHA, MBHC dipole strings 0.2%

Main dipole string 0.1%

Interlock DCCTs 1.0%

MBSG dipole string 0.1%

Septum 0.1%

Main quad strings (D/F) 0.2%

Matching quads 0.5%

Corrector magnets ~ 10 rad

Bumpers ~ 1 rad


ROCS Surveillance Timing• For each extraction, the ROCS system provides four 3 ms long pulses with PERMIT = TRUE (if all OK) which

sets a strong constraint on the timing event sequence (minimizes possible errors).

• The LEGACY events that trigger the ROCS are:• OEX.FINT1-CTM at -13 ms

• OEX.FINT201-CTM at 0 ms (first extraction)

• OEX.FINT202-CTM at 0 ms (second extraction)

Extraction kicker triggers ~ Nominal Extraction Time + 800 s (arrival time of pre-pulse)

Do NOT change the delays !!!!!!

PFN charge only startsif the PERMIT is TRUE !


Shared Main Bend Converter

MUGEF

DCCT

Ref.

Cyclestatus

MUGEF

ADC

ADC

Extractionpermit / abort

TI 8Main

Dipoles

CNGSMain

Dipoles

ControlSystem

3 600 V

5 400 Apeak

MB 1 MASTER

MB 2 SLAVE

Firing

Interlock DCCTs

MUGEF for ‘standard’surveillance

• The TT41 and TI8 main dipoles are powered by a single converter, with switches (presently mechanical, next year electronic) to send the current into the correct magnet string.

• To ensure that the switch position is REALLY correct, we have 2 ‘dummy’ ROCS channels that have only an interlock DCCT but no converter. The names of the ROCS are DCCT_TI8 and DCCT_CNGS (also accessible from equipstate).

• The 2 DCCTs are used to identify which branch is powered, and their current is surveyed like any other converter (22).


FMCMs• The FMCM is a special high sensitivity device that monitors the current change of a circuit

(through the voltage) and triggers an interlock when the current derivative is too large.

• 3 FMCMs (Fast Magnet Current Change Monitors) are installed on the MSE.418 (23), the MBSG.4000 (24) and the MBG dipole string (25) . The FMCM trigger thresholds have been determined by programming step functions into the converter reference (with and without beam).

SC time (ms)

PCFMCM trigger

(current change)ROCS survey

tol.

MSE.418 < 0.1% 0.1%

MBSG.4000 < 2.5×10-4 0.1%

MBG/MBI.816 < 2.5×10-4 0.1%

In all cases the position change @ target is maintained < 0.5 mm

(as specified).

• To avoid un-necessary triggering of the circuit during the converter ramps, the FMCMs are activated (input to TRUE) only at the end of the ramp when the currents stabilize. In addition a minimum current must be detected. The USER_PERMIT of the FMCMs is usually FALSE and switches to TRUE in a window of some hundred(s) milliseconds around the extraction. It switches to FALSE when the converter ramps down (looks like a failure…).


Ring BPMs

• A number of SPS ring BPMs in LSS4 are used to interlock the beam position at extraction as a pre-extraction surveillance (extraction bump amplitude) (26) .

• The interlock logic is implemented within the MOPOS system.

• Unfortunately MOPOS remains fragile (~ one reboot required / day, gain changes) and this interlock is our ‘talon d’Achille’ !

Interlock timing forMOPOS


Ring BPM Settings

• A priori one could interlock any BPMs in sextant 4. But MOPOS cannot manage the data (too slow).

• 8 positions (5 H and 3 V) are selected for the interlock around the extraction point (418).

• The ‘minimum’ system corresponds to 2 H (416/418) and 2 V (417/419) monitors.

• The nominal amplitude of the bump is 31 mm at BPCE.418 (corrected for BPCE non-linearity !) – this corresponds to ~ 29 mm on the normal display. The corrected value is available from the ‘Extraction Interlock’ menu in YASP (SPS ring, ‘Machine Specials’

• The tolerances on the positions are:

± 1 mm good protection

± 2 mm ~ at the limit (fallback if problems)


Ring BCT

• For LHC / TI8 test an interlock on the beam intensity is provided by the ion BCT4 (27) .

• The BCT can be configured to only provide USER_PERMIT=TRUE when the intensity is below a programmed threshold. The intensity measurement is performed 80 ms before extraction (warning event).

• This signal is not used/activated for CNGS. The USER_PERMIT must always be TRUE during CNGS operation.


Post-extraction Surveillance


Transfer line BLMs• Since BLMs in TLs can only measure AFTER the ‘action’, they provide a Post-extraction

surveillance. The USER_PERMIT is latched to prevent further extractions if an excessive loss is detected.

• Two user permits are provided for TT40 (28) and TT41 (29).

• The reset is done manually (for the moment via steering program).

• The USER_PERMIT transition from FALSE to TRUE is triggered by the ‘-20 ms extraction pre-warning’ event. If there is no (abnormal) loss, the USER_PERMIT resets to FALSE a few milliseconds after the extraction.


Transfer line beam loss & thresholds• Beam losses in TT40 & TT41 normally are very low (< 0.3 mGray for 1013 protons).

• When the C OTR screens are inserted losses increase to ~ 2 mGray in some locations.

Thresholds to 5 mGray in TT41.

• The gains must be ≤ 16 ! Saturation ! Surveyed by SIS !

• For the moment, the interfaces shown below are available from… the steering program !!!

BLM after TED : high thresholdto avoid interlocks when beam issent to the TED.

TT40 BLM losses & thresholds TT41 BLM losses & thresholds

Ignore…


Transfer line BPMs• A Post-extraction surveillance has also been implemented for the CNGS trajectory (30) .

• Each of the 23 H+V BPMs in TT40 (4 BPMs) and TT41 can be interlocked. They can be activated individually. Tolerances:

• ±4 mm for 20 transfer line BPMs TL aperture

• ±0.5 mm for last 3 or 4 BPMs target tolerance

• The interlock is latched (reset through steering application).

• A position is used for interlocking only if the BPM has seen beam (auto-triggered): to trigger the BPMs one needs one batch of ~ 2-3×1012 protons (or an equivalent DENSITY).

• The interlock timing follows the same logic than for transfer line BLMs.


Transfer line BPMs /2

Example : reference trajectory for the CNGS pilot run

Short-circuit on one electrode. De-activated for

interlocking.

Threshold ±4 mm Threshold ±0.5 mm

Misalignment transfer line-target-horn

MKE• The MKE extraction kicker provides a slow control interlock to the interlock system (31) .

• The MKE signal is only TRUE when the kicker is ON. NOT MASKABLE !

- PFN voltage

- BIC permit

- BETS window

- Beam

Timing of kicker, BIC permit and BETS:

Extr. 1 Extr. 2

• Internally the MKE has an energy tracking system (from a mains DCCT in BA3). This internal ‘BETS’ (Beam Energy Tracking System) system enforces the following interlock logic:

Kicker voltage 50 kV (nominal) ± 2 kV.

Energy must be in a window of 400 ± 5 GeV.

The BETS interlock is only visible in OASIS and on the scope display (example to the right) which is

available under: http://kestek4/

When the BETS interlocks, the magenta curve is flat !

A symptom for a BETS interlock is that the MKE does not kick although the

interlocks are OK!

http://kestek4/


Safe Beam Flag

• Masking of any MASKABLE input to the CNGS BIC system is conditioned by the SPS safe beam flag (SBF). SBF = TRUE if the intensity is below a predefined threshold.

• A MASKED INPUT is AUTOMATICALLY reactivated if SBF=FALSE.

• SBF Generation:

• The SPS high intensity BCT is used to measure the intensity 1 second after the start of the ramp (triggered by a timing event).

• The intensity is sent to the SPS MTG and compared to the safe beam threshold to generated the SBF.

• The SBF is distributed over the timing system to the BICs.

• Presently the limit for a safe beam is set to 1012 protons. This is a rather conservative limit and it is difficult to obtain CNGS beams of such a low intensity. I have therefore suggested to increase the limit to 3×1012 protons which should be sufficiently safe.


Extraction-Ring Interlock Systems Connection

Two systems that are part of the extraction interlock system are (also) connected to the ring BIC system:

• The extraction kicker MKE dumps the beam:

• When it is pulsing in local.

• When the kick is enabled and the interlock system has not given green light for extraction. After three consecutive (same USER) cycles in such a condition the SPS Emergency System cuts all beam.

• The BLMs in LSS4:

• Typical loss on the first BLM ~ 10-20 mGray – threshold at 50 mGray.

• The loss is due to beam in the gap. If losses become excessive, check the gap population with the fast BCT. Gap population depends on the PS CT, and our RF settings (injection phase, bfield, RF voltage on FB).

• Do not increase the thresholds ! Radiation in ECA4 !!

• ‘As usual’ : interlock is latched by SPS Emergency System after 3 consecutive interlocks (same USER !).

• Fabio’s new application will be used from now on to read & control those BLMs.


Radiation in ECA4

Radiation in ECA4:

H. Vincke


BIC Supervision


BIS Supervision Software

• The supervision applications for the SPS BIC systems (ring and CNGS) are available from the SPS console manager under:

Start Tasks SPS Control SPS Beam Interlocks

• Each ‘bubble’ on the top screens represents a BIC module, the color encodes the BIC output (RED=FALSE, GREEN=TRUE) and the state is sampled at ~ 1 Hz. Note that this is too slow to observe changes for the extraction where the signals are very short and fast.


BIC Supervision – Main Screen

Clicking on a ‘bubble’ opens the main supervision screen for the corresponding BIC module. On the main screen :

• Connected input list

• Input status

• Masks (status & actions)

• Safe beam flag

• Permit status

Refreshed at ~ 1 Hz.

From the main screen:• (Un-)masking of inputs.

• History buffer view.

• Expert screens view.

• Bar graph view.

Inputs 8 to 14 are always maskable

Inputs 1 to 7 are always un-maskable

Mask status

Safe Beam Flag status

Software Interlock System Input

BIC Supervision – history buffer

Time within SPS cycle in ms/s

Signal name / transition

State of BIC output

Timing marker : start of cycle

Signal transition : (A) and (B) refer to the 2 redundant signals

for each input !

• The history buffer logs all transitions and signal changes (also masks and SBF) as well as the start of every cycle (as a marker).

• Very powerful for debugging, but becomes complex when there are many signal transitions !

BIC Supervision – Bar Graph

Time offset wrt start of cycleDepth in seconds wrt

Time offset

The bar graph view shows the status of the inputs/output as a function of the CYCLE TIME: it is a graphical display of the history buffer.

• Display signals for one SPS cycle, or a sub-range of a cycle (for example around extraction)

• Refreshed every cycle (i.e. every 6-12 seconds)

Extraction permit switching to TRUE for extraction (3 ms wide)

Check this box to hold selected time range values

Check this box to freeze display


BIC supervision for CNGS operation

Graphical display of all signals for a (successful) CNGS extraction…


Supervision Software – new ?

Permit Time Delta MS Description

15:30:01 12000 Timing Marker

15:30:05 4220

15:30:05 4270 PC TT40, Vaccum TT40


15:30:11 4220

15:30:11 4270


15:30:17 4270 FMCM MSE, BPM Extraction

15:30:17 4270 FMCM MSE


Same columns that are also found in the history buffer.

List of inputs in state FALSE at the time of the snapshot, or indication for timing marker

I have recently proposed to implement a new table display that would summarize the status at selected times (corresponding to the extraction):

• If the output of the BIC is FALSE, the list of inputs that are false is listed.

• This provides a simple snapshot of the system status at the time of extraction.

The depth of this table should be ~ 50 to 100 cycles…


Diagnostics software

For continuous surveillance interlocks, the diagnostics is rather ‘easy’ since the errors are more or less static – one just has to find the appropriate application !

It is more tricky for highly dynamic interlocks ….


General Post-mortem• A post-mortem application to browse logged data has been written by Verena. It can be

found in the console manager in the same place as other interlock stuff :

Start Tasks SPS Control SPS Beam Interlocks Extraction post-mortem

• This application is very powerful, but it can be tricky to start using it just when a problem appears !

• It is very useful to analyze trends, changes over a few hours…

• Try it out when you have a quiet moment during a shift !

Diagnostics for position interlocks : steering application

• Post-mortem freeze of the display (background becomes ORANGE) when there is no valid data freezes the last acquisition for (interlock) diagnostics.

• The possibility to define a GOLDEN reference for the steering such that everyone uses the same reference (please use it !).

• The possibility to monitor the trajectory on all cycles (CNGS1,2,3) in the same application. But beware when steering since the context selection corresponds to a defined user !

‘Acq&Hw’ menu Acq Configuration ‘Timing’ tab

Change USER to : SPS.USER.CNGS*

Click on button ‘Set User’

• Interlock diagnostics …

Latch status


BLM diagnostics

• There is no application for the transfer line BLMs. The only tools for diagnostics are :

• For the LSS4 extraction BLMs, there is a new application from Fabio (replaces Labview !).

• Fixed displays.

• Verena’s PM (through the logging).

• A diagnostics panel in the steering application (also for thresholds !). The same panel also provides the RESET command for a latched BLM threshold.


ROCS diagnostics

• Diagnostics for the ROCS interlocks is in the form of an expert program, with references in bits… Not recommended for the non-initiated (so far Michel, Jorg, Rossano, Verena) since one can easily screw up the system.

• Fortunately this system proved very stable and gave no problem over the entire run (I use it since May).

• For the moment, refer to one of the ‘initiated’ – I will see if one can provide something for diagnostics only.


CNGS Operation

•Experience from Pilot run in August

•Tentative guidelines for the coming CNGS run


Extraction & TL Stability

• The good news from commissioning and pilot run is that the CNGS extraction & transfer line are VERY stable.

• Stability over 24 hours:

Horizontal ~ 100 microns rms Vertical ~ 40 microns rms

• Observed integrated changes during the 12 days of the pilot run:

Total drift in horizontal plane 0.2 mm rms

Total drift in vertical plane 0.4 mm rms (dominated by drift in SPS ring)

Vertical position change in the SPS ~ 0.5 mm

• Trajectory corrections:

• During the pilot run I made one correction in each plane every 1-2 days using 1-2 correctors (MICADO), which maintains the rms change around 0.1 mm or less.

• The corrections over a week are sufficiently small to fit into the interlock tolerances for the orbit correctors of TT40/TT41.

• In principle it is sufficient to steer 1-2/week. The interlock system will protect us against excessive trims (corrector limitations !).

• I will try to prepare an operational Autopilot for CNGS ...


Interlocks : the bad & the ugly

From the entire protection system, only 2 interlocks gave problems and/or required ‘interventions’ during the pilot run:

• The transfer line BPM interlock (30) due to unphysical readings in the region near the target where the tolerances are very tight.

• The LSS4 MOPOS BPM interlock (26) due to spurious interlocks and problems with gains during the second ½ of the pilot run (increased intensity):

The signal amplitudes were near the lower part of the range for the optimum gain setting (10 dB).

Gain increases (necessary to measure the first turn) led to saturated signals interlock.

Intensity drops (PS) led to insufficient signal amplitudes interlock.

We also observed that some MOPOS settings seemed to be occasionally ‘corrupted’. On 2 occasions during this period, the gain of the system changed without any actions from the shift crews. On such occasions the gain always returned to 20 dB! Has been observed again during recent MD periods.

Beam not extracted dumped out-gassing injection kicker inhibit (vacuum)

I fear the this problem will come back, in particular because the beam dump is not much more conditioned than in August !


Fake Interlock from TL BPMsExample for a trajectory leading to a FAKE (and latched !) position interlock in the transfer line.

• The interlock is generated by the 2 BPMs near the target that are interlocked to ±0.5 mm.

• Observed on 2 occasions (at the beginning and near the end of the run).

• False readings always occurred after a period without beam (at least a few cycles).

• After reset the readings are OK again.

Not really understood…

Example for a bad trajectory (difference wrt reference) leading to a fake interlock

Bad readings


Guidelines…

The extraction interlock system has significant redundancy for the most critical and tricky operation issues concerning PCs, beam positions and BLMs – I think we are well protected !

• Guideline 0: The coming run is for us, not for OPERA. We do not have to be stressed by them, and can therefore take time to study & learn.

• Guideline 1 : Do not touch interlock settings if you are not 100% sure of what you do ! It is better to be a bit inefficient (for this year) than to drill holes. I take the blame !!!

• Guideline 2 : in case of doubts,

• change the beam to MD3 (CNGS low intensity).

• test (you can mask !).

• Guideline 3 : after a long(er) stop, for example an MD,

• Put in the TED

• Make one extraction (nominal beam), check trajectory to TED (change < 1 mm).

• Make two extractions, check.

• If all OK, open TED, extract to target.

• If NOT OK Guideline 2 !


More guidelines…

• Guideline 4 : When an interlock is latched, and yoy have to decide if you should reset or not..

the TT40/41 position interlock is latched – must take a decision : reset or not !

• If you have the frozen steering display and if it looks like isolated large readings:

•Reset and try again.

• If you have no info on the trajectory:

•Start the steering display.

•Reset and try again.

• If the interlock latches again: think hard and be careful – possibly move to Guideline 2.

beam losses in the transfer line (latched) or REPEATED losses in LSS4.

• ABNORMAL !!!!

• Stop and check everything you can (abort gap etc…).

• Reduce intensity, try a single extraction. Be sure you have all diagnostics running for such test shots.

• Guideline 5 : people with (some) experience on this system

• Jorg (I will be away for ~ first days of the CNGS run), Verena, Rossano


WEB documentation

https://cern.ch/sps-mp-operation/A WEB page is installed for MP

operation at the SPS:

• Detailed system description

• Test documents & system status

• Settings:• References

• SPS timings relevant for MP/extraction

• Trouble-shooting

• Name of experts

• Sample screen shots for important information

• OP programs and diagnostics

• …

For use by the expert and by OP crews !





WEB documentation : example 1

Details on BICs and inputs – example for CIBC-TT40B



Details on machine timings :


Details on BPM interlock settings :


Documents

Introduction to Machine protection for CNGS (and TI2/8)