The Status of the LHC Controls System Shortly Before Injection of Beam

15 Oct 2007 ICALEPCS'07 - P.Charrue - CERN 1

The Status of theLHC Controls System

Shortly Before Injection of Beam

The Status of theLHC Controls System

Shortly Before Injection of Beam

Pierre CharruePierre Charrueon behalf ofon behalf of

CERN Accelerator and BeamsCERN Accelerator and BeamsControls GroupControls Group

Pierre CharruePierre Charrueon behalf ofon behalf of

CERN Accelerator and BeamsCERN Accelerator and BeamsControls GroupControls Group


Motivation of this talkMotivation of this talk

Present the controls challenges of the LHC machine

Give an overview of the LHC Controls Infrastructure

Propose links to further ICALEPCS’07 presentations

Present the controls challenges of the LHC machine

Give an overview of the LHC Controls Infrastructure

Propose links to further ICALEPCS’07 presentations

MOAB01


RelativityRelativity

““When you sit with a nice girl for two hours, you think it's only a minute. But when you sit on a hot stove for a minute, you think it's two hours. That's relativity” ” Albert EinsteinAlbert Einstein

Hence the title “Hence the title “LHC controls “shortly” before LHC controls “shortly” before beam injectionbeam injection” for us means:” for us means: a few months before a few months before LHC Beam CommissioningLHC Beam Commissioning 9 months after start of 9 months after start of LHC Hardware CommissioningLHC Hardware Commissioning

““When you sit with a nice girl for two hours, you think it's only a minute. But when you sit on a hot stove for a minute, you think it's two hours. That's relativity” ” Albert EinsteinAlbert Einstein

Hence the title “Hence the title “LHC controls “shortly” before LHC controls “shortly” before beam injectionbeam injection” for us means:” for us means: a few months before a few months before LHC Beam CommissioningLHC Beam Commissioning 9 months after start of 9 months after start of LHC Hardware CommissioningLHC Hardware Commissioning


OutlineOutline

LHC controls infrastructureLHC controls infrastructure

The major LHC challengesThe major LHC challenges

Technical Solutions to the LHC challengesTechnical Solutions to the LHC challenges

ResultsResults

SummarySummary




ResultsResults

SummarySummary


The LHC Controls InfrastructureThe LHC Controls Infrastructure

Based on the Based on the classical 3-tier modelclassical 3-tier model Based on the Based on the classical 3-tier modelclassical 3-tier model

• The The LHC Controls LHC Controls InfrastructureInfrastructure is a is a new new software and hardware software and hardware architecturearchitecture built with built with the the experienceexperience of of controlling the CERN controlling the CERN injector chaininjector chain

• Level of effort: Level of effort: 300 my, 300 my, 21 MSFR21 MSFR

• The The LHC Controls LHC Controls InfrastructureInfrastructure is a is a new new software and hardware software and hardware architecturearchitecture built with built with the the experienceexperience of of controlling the CERN controlling the CERN injector chaininjector chain

• Level of effort: Level of effort: 300 my, 300 my, 21 MSFR21 MSFR


The 3-tier LHC Controls InfrastructureThe 3-tier LHC Controls Infrastructure


The Resource TierThe Resource Tier VME cratesVME crates dealing with dealing with high high

performanceperformance acquisitions and acquisitions and real-real-timetime processing processing e.g. the LHC beam instrumentation e.g. the LHC beam instrumentation

and the LHC beam interlock systems and the LHC beam interlock systems use VME front-endsuse VME front-ends

VME cratesVME crates dealing with dealing with high high performanceperformance acquisitions and acquisitions and real-real-timetime processing processing e.g. the LHC beam instrumentation e.g. the LHC beam instrumentation

and the LHC beam interlock systems and the LHC beam interlock systems use VME front-endsuse VME front-ends

• PC based gateways interfacing systems where a large quantity of identical equipment is controlled through fieldbuses

• e.g. LHC power converters and LHC Quench Protection System• Programmable Logic Controllers (PLCs) driving various sorts of

industrials actuators and sensors for systems• e.g. LHC Cryogenics systems or the LHC vacuum system.

• Supported FieldBuses for local connections• e.g. Mil1553, WorldFIP, Profibus

• PC based gateways interfacing systems where a large quantity of identical equipment is controlled through fieldbuses

• e.g. LHC power converters and LHC Quench Protection System• Programmable Logic Controllers (PLCs) driving various sorts of

industrials actuators and sensors for systems• e.g. LHC Cryogenics systems or the LHC vacuum system.

• Supported FieldBuses for local connections• e.g. Mil1553, WorldFIP, Profibus


The Middle TierThe Middle Tier

ApplicationApplication servers hosting the software required to operate the LHC beams and running the Supervisory Control and Data Acquisition (SCADA) systems

– HP ProLiant dual CPU with 3Gbyte RAM running LINUX.

– Dual hot pluggable RAID-1 system disks– Redundant dual hot pluggable power

supplies.

ApplicationApplication servers hosting the software required to operate the LHC beams and running the Supervisory Control and Data Acquisition (SCADA) systems

– HP ProLiant dual CPU with 3Gbyte RAM running LINUX.

– Dual hot pluggable RAID-1 system disks– Redundant dual hot pluggable power

supplies.

Data serversData servers containing the LHC layout and the controls configuration containing the LHC layout and the controls configuration as well as all the machine settings needed to operate the machine or to as well as all the machine settings needed to operate the machine or to diagnose machine behaviors.diagnose machine behaviors.

File serversFile servers containing the operational applications containing the operational applications Central timingCentral timing which provides the cycling information of the whole which provides the cycling information of the whole

complex of machines involved in the production of the LHC beam and complex of machines involved in the production of the LHC beam and the timestamp referencethe timestamp reference

Data serversData servers containing the LHC layout and the controls configuration containing the LHC layout and the controls configuration as well as all the machine settings needed to operate the machine or to as well as all the machine settings needed to operate the machine or to diagnose machine behaviors.diagnose machine behaviors.

File serversFile servers containing the operational applications containing the operational applications Central timingCentral timing which provides the cycling information of the whole which provides the cycling information of the whole

complex of machines involved in the production of the LHC beam and complex of machines involved in the production of the LHC beam and the timestamp referencethe timestamp reference


The Presentation TierThe Presentation Tier

At the control room level, At the control room level, consolesconsoles running the running the Graphical User InterfacesGraphical User Interfaces (GUI) will allow machine operators to (GUI) will allow machine operators to control and optimize the LHC beams control and optimize the LHC beams and to supervise the state of key and to supervise the state of key industrial systems.industrial systems.

At the control room level, At the control room level, consolesconsoles running the running the Graphical User InterfacesGraphical User Interfaces (GUI) will allow machine operators to (GUI) will allow machine operators to control and optimize the LHC beams control and optimize the LHC beams and to supervise the state of key and to supervise the state of key industrial systems.industrial systems.

Dedicated fixed displays will also provide real-time summaries of key machine parameters

Operational consoles specifications : PCs with 2 Gbyte RAM running either LINUX or WINDOWS One PC, one keyboard, one mouse, and up to 3 screens Console capable of running any type of GUI software for LHC (and PS

and SPS) such as JAVA, WEB, SCADA or X-MOTIF.

Dedicated fixed displays will also provide real-time summaries of key machine parameters

Operational consoles specifications : PCs with 2 Gbyte RAM running either LINUX or WINDOWS One PC, one keyboard, one mouse, and up to 3 screens Console capable of running any type of GUI software for LHC (and PS

and SPS) such as JAVA, WEB, SCADA or X-MOTIF.


Inside CCCInside CCC

General Purpose Fixed Display

Operator Console

RPPB01


A typical Operator ConsoleA typical Operator Console

Interactive Consoles

Local Fixed Displays


CommunicationsCommunications

Technical Network (TN) For operational equipment Formal connection and access restrictions Limited services available

(e.g. no mail server, no external web browsing) Authorization based on MAC addresses Network monitored by CERN IT Department

Technical Network (TN) For operational equipment Formal connection and access restrictions Limited services available

(e.g. no mail server, no external web browsing) Authorization based on MAC addresses Network monitored by CERN IT Department

General Purpose Network (GPN) For office, mail,

www, development No formal connection

restrictions

General Purpose Network (GPN) For office, mail,

www, development No formal connection

restrictions


OutlineOutline




ResultsResults

SummarySummary




ResultsResults

SummarySummary

The LHC Challenges (1/3)The LHC Challenges (1/3) The LHC is the largest accelerator in the world

number of components diversity of systems

The complexity of operation will be extreme: very critical technical subsystems large parameter space need for online magnetic and beam measurements real time feedback loops.

Some 500 objects are capable of moving into the aperture of either the LHC Ring, or the transfer lines, ranging from passive valves up to very complex experimental detectors

P. Collier - AB/CO LHC Workshop January, 2005 The complexity of the accelerator is unprecedented and repair

of damaged equipment would take long, for example, the exchange of a superconducting magnet takes about 30 days

R. Schmidt - http://cern.ch/rudi/docs/VisitLHCWuppertal2006.ppt

The LHC is the largest accelerator in the world number of components diversity of systems

The complexity of operation will be extreme: very critical technical subsystems large parameter space need for online magnetic and beam measurements real time feedback loops.

Some 500 objects are capable of moving into the aperture of either the LHC Ring, or the transfer lines, ranging from passive valves up to very complex experimental detectors

P. Collier - AB/CO LHC Workshop January, 2005 The complexity of the accelerator is unprecedented and repair

of damaged equipment would take long, for example, the exchange of a superconducting magnet takes about 30 days

R. Schmidt - http://cern.ch/rudi/docs/VisitLHCWuppertal2006.ppt

The LHC Challenges (2/3)The LHC Challenges (2/3) The energy stored in the beams and the energy

stored in the magnets exceed present machines by more than 2 orders of magnitude

The LHC machine must be protected at all costs In the case of an operational incident operation

should be able to analyze what has happened and trace the cause.

Moreover no operation can be resumed if the machine is not back in a good state

The energy stored in the beams and the energy stored in the magnets exceed present machines by more than 2 orders of magnitude

The LHC machine must be protected at all costs In the case of an operational incident operation

should be able to analyze what has happened and trace the cause.

Moreover no operation can be resumed if the machine is not back in a good state

Energy stored in one beam at 7 TeV: 362 MJoule

The energy of one shot (5 kg) at 800 km/hour corresponds to the energy The energy of one shot (5 kg) at 800 km/hour corresponds to the energy stored in one bunch at 7 TeV. stored in one bunch at 7 TeV.

There are 2808 bunches.There are 2808 bunches.Factor 200 compared to HERA, TEVATRON and SPS. *Factor 200 compared to HERA, TEVATRON and SPS. *

Energy stored in one beam at 7 TeV: 362 MJoule

The energy of one shot (5 kg) at 800 km/hour corresponds to the energy The energy of one shot (5 kg) at 800 km/hour corresponds to the energy stored in one bunch at 7 TeV. stored in one bunch at 7 TeV.

There are 2808 bunches.There are 2808 bunches.Factor 200 compared to HERA, TEVATRON and SPS. *Factor 200 compared to HERA, TEVATRON and SPS. *

shot

Energy Stored in the BeamEnergy Stored in the Beam

*Rüdiger Schmidt

Energy stored in the magnetsEnergy stored in the magnets

The energy of an AirBus A380 at 700km/hr corresponds to the energy stored in the LHC magnet system*

The energy of an AirBus A380 at 700km/hr corresponds to the energy stored in the LHC magnet system*

AirBus A380

*Rüdiger Schmidt MAC 9 December 2005**Mike Harrison HCP2006

Kinetic Energy• One bunch out of 2808 carries the equivalent of a 5kg shot travelling at 800 km/h*• 1 small aircraft carrier of 104 tons going 30 km/h• 450 automobiles of 2 tons going 100 km/h**

Thermal Energy**• melt 500 kg of copper• raise 1 cubic meter of water 85ºC: “One ton of tea”

Chemical Energy**• 80 kg of TNT

The LHC Challenges (3/3)The LHC Challenges (3/3)

The LHC is the first superconducting accelerator built at CERN 4 large scale cryoplants with 1.8 K refrigeration capability

The LHC is the first superconducting accelerator built at CERN 4 large scale cryoplants with 1.8 K refrigeration capability


OutlineOutline




ResultsResults

SummarySummary




ResultsResults

SummarySummary


Technical solutions (1/3)Technical solutions (1/3)

Network security (CNIC) and equipment access control (RBAC, Machine Critical Settings)

Specific tools for the monitoring of the controls infrastructure LASER: general alarm system TIM: monitoring of technical infrastructure DIAMON (in development): monitoring and

diagnostics of controls infrastructure Post mortem data storage and analysis Logging system

Network security (CNIC) and equipment access control (RBAC, Machine Critical Settings)

Specific tools for the monitoring of the controls infrastructure LASER: general alarm system TIM: monitoring of technical infrastructure DIAMON (in development): monitoring and

diagnostics of controls infrastructure Post mortem data storage and analysis Logging system

RPPA35

RPPB03

RPPB13

WPPB38


Security : RBACSecurity : RBAC

Implement a ‘Implement a ‘role-basedrole-based’ access to equipment ’ access to equipment in the communication infrastructurein the communication infrastructure

Depending on Depending on WHICHWHICH action is made, on action is made, on WHOWHO is making the call, from is making the call, from WHEREWHERE the call is the call is issued and issued and WHENWHEN it is executed, the access it is executed, the access will be will be granted or deniedgranted or denied

This allows for This allows for filteringfiltering,, for for controlcontrol and for and for traceabilitytraceability on the settings modifications to the on the settings modifications to the equipmentequipment

Implement a ‘Implement a ‘role-basedrole-based’ access to equipment ’ access to equipment in the communication infrastructurein the communication infrastructure

Depending on Depending on WHICHWHICH action is made, on action is made, on WHOWHO is making the call, from is making the call, from WHEREWHERE the call is the call is issued and issued and WHENWHEN it is executed, the access it is executed, the access will be will be granted or deniedgranted or denied

This allows for This allows for filteringfiltering,, for for controlcontrol and for and for traceabilitytraceability on the settings modifications to the on the settings modifications to the equipmentequipment

TPPA04TPPA12WPPB08



Integrated suite of control room Integrated suite of control room applications applications LSA (LHC Software LSA (LHC Software Application) Application)

FESAFESA framework for front-end computers framework for front-end computers

UNICOSUNICOS framework for industrial controls framework for industrial controls

Integrated suite of control room Integrated suite of control room applications applications LSA (LHC Software LSA (LHC Software Application) Application)

FESAFESA framework for front-end computers framework for front-end computers

UNICOSUNICOS framework for industrial controls framework for industrial controls

WOPA04


The LSA frameworkThe LSA frameworkWOPA03RPPB05


UNICOS FrameworkUNICOS Framework WOAA03



Specific hardware developments (high SIL Specific hardware developments (high SIL levels) for levels) for machine protectionmachine protection (PIC, BIC…) (PIC, BIC…)

LHC Software InterlocksLHC Software Interlocks System (SIS) System (SIS) Fully integrated Fully integrated asset, layout, configurationasset, layout, configuration

database systemdatabase system

Extension to the Injector Extension to the Injector timing systemtiming system

Specific hardware developments (high SIL Specific hardware developments (high SIL levels) for levels) for machine protectionmachine protection (PIC, BIC…) (PIC, BIC…)

LHC Software InterlocksLHC Software Interlocks System (SIS) System (SIS) Fully integrated Fully integrated asset, layout, configurationasset, layout, configuration

database systemdatabase system

Extension to the Injector Extension to the Injector timing systemtiming systemWPPB02RPPB31FOAA03

WPPB03

RPPA03

TPPB23


OutlineOutline




ResultsResults

SummarySummary




ResultsResults

SummarySummary


LHC sector 7-8 : Successful cooldownLHC sector 7-8 : Successful cooldown

• The Cryogenics The Cryogenics controls system controls system has been a great has been a great aid to the aid to the operations to operations to achieve this first achieve this first cooldowncooldown


Primary cause of the 2004 TI8 accident identifiedPrimary cause of the 2004 TI8 accident identifiedPrimary cause of the 2004 TI8 accident identifiedPrimary cause of the 2004 TI8 accident identified

20000

20500

21000

21500

22000

22500

10/25/04 18:00 10/25/04 19:00 10/25/04 20:00 10/25/04 21:00 10/25/04 22:00 10/25/04 23:00 10/26/04 0:00

1E+08

1E+09

1E+10

1E+11

1E+12

1E+13

1E+14

MBHC.400129/CURRENT/EXTR1

MSE.4183M/CURRENT/EXTR1

I total

20000

20500

21000

21500

22000

22500

10/25/04 18:00 10/25/04 19:00 10/25/04 20:00 10/25/04 21:00 10/25/04 22:00 10/25/04 23:00 10/26/04 0:00

1E+08

1E+09

1E+10

1E+11

1E+12

1E+13

1E+14

MBHC.400129/CURRENT/EXTR1

MSE.4183M/CURRENT/EXTR1

I total

Conclusion : MSE current appears to be ~2.5% low at extraction

Reconstruction of Reconstruction of beam incidentbeam incident from logged data from logged data well working logging system SPS at 450 GeV (some 10^13 protons)SPS at 450 GeV (some 10^13 protons)

Tracking between the three main circuits of sector 78

Courtesy F.Bordry

2ppm

post mortem view of PC trackingpost mortem view of PC tracking


LHC Hardware Commissioning LHC Hardware Commissioning


OulookOulook

LHC Hardware Commissioning in the coming months, with massive parallelism early 2008

First LHC beam commissioning for summer 2008

LHC Hardware Commissioning in the coming months, with massive parallelism early 2008

First LHC beam commissioning for summer 2008

LHC Transfer Lines commissioning in the coming weeks

LHC Transfer Lines commissioning in the coming weeks


OutlineOutline



Controls Answers to the challengesControls Answers to the challenges

ResultsResults

SummarySummary



Controls Answers to the challengesControls Answers to the challenges

ResultsResults

SummarySummary


SummarySummary

We have addressed the LHC challenges and appropriate solutions have been deployed

The LHC infrastructure has been tested almost completely on previous milestones (LEIR, TI2/8 Transfer Lines, CNGS, SPS, LHC HWC) we are confident that we can meet the LHC challenges

Part of the enormous human resource effort invested for LHC comes from international collaborations Their contribution is highly appreciated Looking forward to more fruitful collaborations

We have scheduled our efforts for the 450 GeV engineering run in November 2007 We are now ready and eager to see beam in the LHC

We have addressed the LHC challenges and appropriate solutions have been deployed

The LHC infrastructure has been tested almost completely on previous milestones (LEIR, TI2/8 Transfer Lines, CNGS, SPS, LHC HWC) we are confident that we can meet the LHC challenges

Part of the enormous human resource effort invested for LHC comes from international collaborations Their contribution is highly appreciated Looking forward to more fruitful collaborations

We have scheduled our efforts for the 450 GeV engineering run in November 2007 We are now ready and eager to see beam in the LHC

Documents

The Status of the LHC Controls System Shortly Before Injection of Beam