Sp

lice

Re

vie

w:

Re

po

rt f

rom

CE

RN

Au

dito

rs,

S.

Fe

he

r, 2

3 J

uly

. 2

01

3

Report from CERN AuditorsSplice Review 23rd of July 2013

S. Feher on behalf of the CERN Auditors’ team

Outline• Role of the Auditors• Modus Operandi of the Auditors• Findings, suggestions and

implementations• Future Plans

Sp

lice

Re

vie

w:

Re

po

rt f

rom

CE

RN

Au

dito

rs,

S.

Fe

he

r, 2

3 J

uly

. 2

01

3

Production InspectionsActor: Operators and supervisorsLevel: InterconnectionDocuments: IP for each activity

CERN AuditActor: Team of experienced CERN staffLevel: AllDocuments: IPs, TPs, CERN rules

Quality ControlActor: QC teamLevel: Interconnection, SubsectorDocuments: TP for each QC activity

QA TeamActor: QA teamLevel: AllDocuments: IPs, TPsActions: • Review and decision on NCs and border-line

cases• Follow-up of audits and production quality trends• Follow-up and timely completion of test reports

and MTF entries

Quality Assurance: Organization

Sp

lice

Re

vie

w:

Re

po

rt f

rom

CE

RN

Au

dito

rs,

S.

Fe

he

r, 2

3 J

uly

. 2

01

3

CERN Auditors

G. FavreS. FeherP. GalbraithM. LamontS. MathotR. OstojicA. SiemkoM. StruikH. Ten KateD. TommasiniL. Williams

Sp

lice

Re

vie

w:

Re

po

rt f

rom

CE

RN

Au

dito

rs,

S.

Fe

he

r, 2

3 J

uly

. 2

01

3

The role of CERN Auditors

The priorities for SMACC, as defined by the CERN management, are:

SafetyQualitySchedule

On the basis of the approved SMACC documents and their own experience, the auditors should inspect and audit the manner in which the work is done and the procedures are implemented by the production and QC teams. They should report their findings, with suggestions for improvement of the processes, where applicable.

The auditors should be alert, where necessary take immediate actions, and report to SMACC project leader any event in the tunnel that is at clear odds with the safety of workers and equipment and with CERN rules in general.

Sp

lice

Re

vie

w:

Re

po

rt f

rom

CE

RN

Au

dito

rs,

S.

Fe

he

r, 2

3 J

uly

. 2

01

3

Modus Operandi of the Auditors Visits to the tunnel, during working hours of the

SMACC train and QC teams, between 8h-15h. The QA-support will provide the latest information

about the status of activities, keep record of audit planning, and suggest best times for audits.

The audit should be recorded in a report within few days of the visit and returned to the QA-support. The reports will be stored in EDMS (SMACC-QA site).

The QA-support will analyse the audits and report to LSC on the findings.

Sp

lice

Re

vie

w:

Re

po

rt f

rom

CE

RN

Au

dito

rs,

S.

Fe

he

r, 2

3 J

uly

. 2

01

3

Auditors’ visits Visited the tunnel 14 times, in pairs Mainly inspected sector 56 Observing activities – but not witnessing the

entire procedure Asking questions about why and how the

operators do their job Listening to the operators’ suggestions –

tunnel experience Discussing these visits on Auditors’ meetings The visits are documented in Auditors’ Reports

Sp

lice

Re

vie

w:

Re

po

rt f

rom

CE

RN

Au

dito

rs,

S.

Fe

he

r, 2

3 J

uly

. 2

01

3

Findings, suggestions and implementationsQuestioning Procedures: The bus bars of the spool pieces need to be pulled aside in order

to be able to proceed with any of the splice consolidation work. There were no quantitative procedure determining the allowable force applied. Although it is not strongly related it was also observed that some of the spool buses are too lose and it is hard

to install the insulation box. Extensive discussions:• Bus could not be pulled out from the

spacer block• Checked by endoscopy the soundness

of the bus bars• US weld joint could be in danger? No,

each spool bus weld joint can handle 150 kg

Conclusions:• Does not appear that damage has

been done• Careful handling by hand (no tools):

not to apply any significant force

Sp

lice

Re

vie

w:

Re

po

rt f

rom

CE

RN

Au

dito

rs,

S.

Fe

he

r, 2

3 J

uly

. 2

01

3

Findings, suggestions and implementationsQuestioning Procedures cont’d: The ELQA Operators could set on the spot the HV

stand off test Voltage and leakage current values. They should strictly follow procedures. If threshold values need to be changed MP3 needs to be

involved. The high voltage PSs can go up to 3kV and have no

hardware limit. There are pre programmed scripts that run the HV ramp. Strong recommendation is to include a hardware limit switch if

it is possible so there is no easy way to change the setting neither easy to accidently run an undesired script.

Sp

lice

Re

vie

w:

Re

po

rt f

rom

CE

RN

Au

dito

rs,

S.

Fe

he

r, 2

3 J

uly

. 2

01

3

Findings, suggestions and implementationsTooling: The tooling during splice machining is not always turning in

the favourable direction (upmilling or downmilling). Does this affect the surface quality? • Eye exam: there are definite

differences on the different sides of the machined surfaces, however the surface quality by QC is OK.

• There is a preferred milling direction.

Conclusion:• Re-examine the procedure.• Perform bench tests.• Apply different procedure or

modify the tool to be able to change rotational direction.

Sp

lice

Re

vie

w:

Re

po

rt f

rom

CE

RN

Au

dito

rs,

S.

Fe

he

r, 2

3 J

uly

. 2

01

3

Findings, suggestions and implementations

Safety:• There is a potential problem with the soldering machines that can

be started prematurely before the control thermocouple is plugged in. (Could it be permanently mounted?)

Safety switch is being introduced to avoid accidental turn on the equipment.

Sp

lice

Re

vie

w:

Re

po

rt f

rom

CE

RN

Au

dito

rs,

S.

Fe

he

r, 2

3 J

uly

. 2

01

3

Findings, suggestions and implementations

Safety cont’d:• Hot swarf punching holes on the protection plastic -

Fire hazard

The material properties of the protection sheet has to be checked to ensure that nonflammable material is used.

Sp

lice

Re

vie

w:

Re

po

rt f

rom

CE

RN

Au

dito

rs,

S.

Fe

he

r, 2

3 J

uly

. 2

01

3

Findings, suggestions and implementationsHouse keeping: The milling machine is used for removing the insulation. Also for labelling

reasons tape was placed on the surface area reserved for solder joints. Is the cleaning with alcohol and Scotch Brite effective enough and how often are the pads changed? Should the freshly machined splices be temporarily protected with (household) aluminium foil? After machining is done there are only few days before the shunt installation takes

place. The procedure calls for surface preparation with Scotch Brite (to remove oxide layer) and alcohol. There were no requirements how often the Scotch Brite is replaced with new ones.

New Policy: for every different interconnect new Scotch Brite will be used.• Introducing the Aluminium foil would significantly complicate the work flow and the

benefits by using it is not substantial. Dust found inside the interconnects should be analysed for metal contents. Oil

drop found on the inner surface of the cryostat wall. Keep cleanliness as a high priority.

We found unmarked rolls of solder in the tunnel, what is done to prevent this? Better labelling? Labelling as a general practice is highly recommended, however only one type

of solder is used by a production team even if the solder for the splice installation team is different than the solder for the shunt installation team.

Sp

lice

Re

vie

w:

Re

po

rt f

rom

CE

RN

Au

dito

rs,

S.

Fe

he

r, 2

3 J

uly

. 2

01

3

Future Plans• Every activities need to be visited few times and

examined the procedures systematically not just picking parts of the procedures randomly.

As we go further it is expected that the number of new NCRs will be reduced. This does not mean that the importance of auditing will be reduce. In the contrary:• Routine and self confidence will kick in.• More reluctance to follow procedures.• The procedure will not be reviewed by the operators frequently.

Auditors’ visits will be very important.

Sp

lice

Re

vie

w:

Re

po

rt f

rom

CE

RN

Au

dito

rs,

S.

Fe

he

r, 2

3 J

uly

. 2

01

3

Findings, suggestions and implementationsQuestioning Procedures cont’d: Why are the available Ultem protection boxes not

always in place during soldering?: for better accessibility? No clear answer.

Why are Ultem boxes being modified to fit in the tunnel if we have 1000 units ready machined: no answer.

Who verifies the kapton insulation before installing the insulation box?: LMF: the same person who does the insulation also installs the insulation box..

Cutting and welding flanges will shrink them by a few mm, is this acceptable?: according to Cedric yes.

During M-line cutting swarf gets hot enough to burn through the sheets supposed to contain them?: no answer yet.