19
Eric Prebys APC/LARP The truth is out there! 2/9/2009

Eric Prebys APC/LARP The truth is out there! 2/9/2009

Embed Size (px)

Citation preview

Page 1: Eric Prebys APC/LARP The truth is out there! 2/9/2009

Eric Prebys

APC/LARP

The

trut

h is

out t

here

!

2/9/2009

Page 2: Eric Prebys APC/LARP The truth is out there! 2/9/2009

The primary goal of this workshop was to provide the CERN Directorate with information and recommendations to define the scope of activities and final schedule and plan 2009/2010 running of the LHC.

This planning is obviously dominated by the consequences incident of September 19.

Questions about how this incident occurred were examined to the extent that they impact repair and remediation; however, questions regarding the decision process that allowed to occur were deferred to the external review committee.

2/9/2009E. Prebys, AEM

Page 3: Eric Prebys APC/LARP The truth is out there! 2/9/2009

When will the LHC start up? The DG has made a strong commitment to a physics run of the

LHC “in 2009”, provided it does not threaten the long term viability of the accelerator.

Run through the winter? How much does this cost?

How many sectors to warm up? There are currently four sectors warm, and a plan to install new

pressure relief flanges with dramatically increased capacity. To install these new relief flanges on the remaining four sectors

would require a warm up, which has it’s own risks and scheduling headaches.

The baseline plan is to leave the remaining sectors cold, and effect an intermediate pressure relief enhancement by replacing clamps with springs on a number of existing flanges.

Energy Quickly became clear the options were 4 or 5 TeV/beam.

2/9/2009E. Prebys, AEM

Page 4: Eric Prebys APC/LARP The truth is out there! 2/9/2009

After the close of the meeting, Steve Myers made all the workshop talks public on the CERN Indico site, which you can access with this shortcut: http://tinyurl.com/Chamonix2009 Of particular interest is Steve’s summary talk at the end

(very) Partial glossary of terms: “Consolidation” – CERNglish word referring to a set of

actions and/or repairs to address known shortcomings of a system.

“MLI” – multi-layer insulation, which was blown all over the tunnel and sucked into the beam pipe

“DN200” – 200 mm pressure relief flange (“bandaid du jour”).

“QPS” – quench protection system “MCI” – maximum credible incident

2/9/2009E. Prebys, AEM

Page 5: Eric Prebys APC/LARP The truth is out there! 2/9/2009

On September 19th, sector 3-4 was being ramped to 9.3 kA, the equivalent of 5.5 TeV All other sectors had already been ramped to this level Sector 3-4 had previously only been ramped to 7 kA (4.1 TeV)

At 11:18AM, a quench developed in the splice between dipole C24 and quadrupole Q24 Not initially detected by quench protection circuit Power supply tripped at .46 sec Discharge switches activated at .86 sec

Within the first second, and arc formed at the site of the quench Helium pressure rose beyond .13 MPa and ruptured into the

insulation vacuum. Vacuum also degraded in the beam pipe

The pressure at the vacuum barrier reached ~10 bar (design value 1.5 bar). The force was transferred to the magnet stands, which broke.

2/9/2009

*see talk by Philippe LeBrun, Monday AME. Prebys, AEM

Page 6: Eric Prebys APC/LARP The truth is out there! 2/9/2009

Incident on 19th of September 2008 => failure of some supports of SSS in sector 3-4 due to longitudinal loads

*O. Capatina, Tuesday PM

2/9/2009E. Prebys, AEM

Page 7: Eric Prebys APC/LARP The truth is out there! 2/9/2009

Under the rules at the time, there could have been people in adjacent sectors or service areas, who might have been killed or seriously hurt.

It could have occurred in, eg, the dispersion suppression (DS) region, for which spares would have been an issue.

It could have happened in the triplet (which BTW has much better designed joints), and damaged the experiments.

If the arc had cut into both He lines, it would have been 40 kg/s of He, a factor of 20 over design This MCI drives the new pressure relief specification

2/9/2009

*see talk Jim Strait, Wednesday PM

E. Prebys, AEM

Page 8: Eric Prebys APC/LARP The truth is out there! 2/9/2009

Why did the joint fail? Quality control problems Inherent weakness in joint design (no clamps)

Why wasn’t it detected in time? There was indirect (calorimetric) evidence of an ohmic

heat loss, but these data were not routinely monitored The bus quench protection circuit had a threshold of 1V, a

factor of ~1000 too high to detect the quench in time. Why did it do so much damage?

The pressure relief system was designed around an MCI Helium release of 2 kg/s, a factor of ten below what occurred.

2/9/2009E. Prebys, AEM

Page 9: Eric Prebys APC/LARP The truth is out there! 2/9/2009

Cable Junction Box Cross-section

Upper Tin/Silver Soldering alloy Layer

Inter-Cable Tin/Silver Soldering Alloy Layer

Superconducting Cable in Copper

Stabilizer

Upper Copper Profile

Lower Copper U Profile

Lower Tin/Silver Soldering Alloy Layer

Completed Junction

2/9/2009

*see talk by Arjan Verweij, Tuesday PME. Prebys, AEM

Page 10: Eric Prebys APC/LARP The truth is out there! 2/9/2009

Theory: A resistive joint of about 220 nTheory: A resistive joint of about 220 n with bad with bad electrical and thermal contacts with the stabilizerelectrical and thermal contacts with the stabilizer

No electrical contact between wedge and U-profile with the bus on at least 1 side of the

joint

No bonding at joint with the U-profile and the

wedge

A. Verweij

• Loss of clamping pressure on the joint, and between joint and stabilizer

• Degradation of transverse contact between superconducting cable and stabilizer

• Interruption of longitudinal electrical continuity in stabilizer Problem: this is where

the evidence used to be

2/9/2009E. Prebys, AEM

Page 11: Eric Prebys APC/LARP The truth is out there! 2/9/2009

Use calorimetric data to find suspicious joints, then check with precision resistance measurements: 1 sigma = 20 nOhms (design value .6 nOhm) Thermal runaway @ ~85 nOhms @ 5TeV Two additional bad joints found, both of them inside

magnets, which were removed. All new joints photographed and check with

ultrasound.

Cryodipole 2334 right splice

0

20

40

60

80

100

0 20 40 60 80 100Time (a.u.)

Tra

nsm

itte

d p

ow

er (

a.u

.)

right splice, far leftright splice, center leftright splice, center rightright splice, far rightacceptance limit

Cryodipole 2334 left splice

0

20

40

60

80

100

0 20 40 60 80 100Time (a.u.)

Tra

ns

mit

ted

po

wer

(a.

u.)

left splice, far leftleft splice, center leftleft splice, center rightleft splice, far rightacceptance limit

GOOD BAD

2/9/2009E. Prebys, AEM

Page 12: Eric Prebys APC/LARP The truth is out there! 2/9/2009

-10 mK

S1-2 S2-3 S3-4 S4-5

S5-6 S6-7 S7-8 S8-1

7 kA 7 kA 7 kA 9.3 kA

7 kA 7 kA 8.5 kA 7 kA

+40 mK

-10 mK

Rel

ativ

e te

mp

+40 mK

Rel

ativ

e te

mp

1-2 hour flat tops

6 February 2009

Smoking gun?

In the future, calorimetric data will be routinely monitored

2/9/2009

*See talk by Nuria Catalan Lasheras, Monday AME. Prebys, AEM

Page 13: Eric Prebys APC/LARP The truth is out there! 2/9/2009

Old quench protection circuit triggered at 1V on bus. New QPS triggers at .3 mV

Factor of 3000 Should be sensitive down to 25 nOhms (thermal runaway at 7

TeV) Can measure resistences to <1 nOhm

Concurrently installing improved quench protection for “symmetric quenches” A problem found before September 19th

Worrisome at >4 TeV

2/9/2009

*See talks by Arjan Verveij and Reiner Denz, Tuesday PM

E. Prebys, AEM

Page 14: Eric Prebys APC/LARP The truth is out there! 2/9/2009

2 kg/s

20 kg/s

40 kg/s

DP1

1.52

2.53

3.54

4.5

0 20 40 60 80 100 120

Vac

encl

osur

e P

[bar

]

Vac enclosure He T [K]

2 kg/s20 kg/s

40 kg/s

DP

1

1.1

1.2

1.3

1.4

1.5

1.6

0 20 40 60 80 100 120

Vac

encl

osur

e P

[bar

]

Vac enclosure He T [K]

New configuration on cold sectors: Turn several existing flanges into pressure reliefs (while cold). Also reinforce stands to hold ~3 bar

New configurationon warm sectors: new flanges(12 DN200 + 8 DN100)

(DP: Design Pressure)

L. Tavian2/9/2009

*Vittorio Parma and Ofelia Capatina, Tuesday PM E. Prebys, AEM

Page 15: Eric Prebys APC/LARP The truth is out there! 2/9/2009

15 SSS (MQ)1 not removed (Q19) 14 removed

8 cold mass revamped (old CM, partial de-cryostating for cleaning and careful inspection of supports and other components)

6 new CMs In this breakdown there is

consideration about timing (SSS cryostating tales long time; variants problems).

42 Dipoles (MBs)3 not removed

(A209,B20,C20)39 removed

9 Re-used (old CM, no decryostating –except one?)

30 new CMs New cold masses are

much faster to prepare than rescuing doubtful dipoles)

2/9/2009

*Lucio Rossi (magnets) and Francesco Bertinelli (connections), Tues AM

E. Prebys, AEM

Page 16: Eric Prebys APC/LARP The truth is out there! 2/9/2009

Most dirty beam pipes in removed magnets Several must be cleaned in tunnel

Soot: successive passes with wet and dry sponges MLI: little vacuum cleaner

2/9/2009

*Vincent Baglin, Tuesday AM

E. Prebys, AEM

Page 17: Eric Prebys APC/LARP The truth is out there! 2/9/2009

Magnet reinstallation begins this week. Work on flanges begins in a few weeks. Decision of Directorate (announced today)

Do NOT warm up remaining four sectors Start up in late September, collisions in late October Run through winter until next fall (with short break for

Christmas, of course) Raise energy to 5 TeV as soon as safe (symmetric QPS?) Collect ~200 pb-1 at 5+5

200 pb-1 @ 5+5 TeV ~ Tevatron Run II so far Possible ion run at the end

2/9/2009

*See Friday talks, particularly Steve Myers’ summary

E. Prebys, AEM

Page 18: Eric Prebys APC/LARP The truth is out there! 2/9/2009

We shouldn’t forget how incredibly well things were going before the incident. Beam was circulated in both directions within a few hours Beam was captured within a day Optics measurements uncovered a few minor wiring problems.

Right

Wrong

2/9/2009

*Thursday AM

E. Prebys, AEM

Page 19: Eric Prebys APC/LARP The truth is out there! 2/9/2009

Will NOT happen this year Problem with dipoles “forgetting their training”

after the period of storage Only occurs with one out of the three vendors (Noel) This was discovered before the turn on Project LOTS of quenches to go beyond 6 TeV

6.5 TeV: ~10 per sector 7.0 TeV: ~100 per sector

Can only do 2-3 quenches/day 1 to 2 months of quenching to get to 7 TeV Risk of damage?

2/9/2009

*Arjan Verveij, Monday AM and Ezio Todesco, Thursday PM E. Prebys, AEM