33
1 Technical Appendices T1 The Incidents T1.1 Transient Fault recordings and Fault Levels Transient fault records were obtained from transient fault recording apparatus connected at 66kV, at a location remote from Casuarina. The technique for filtering the fault currents from the considerable normal system load at these locations does have some limitations. Nevertheless these were useful in understanding the relative magnitudes of the faults on the three occasions. Transient Recording for the Breaker Fault Casuarina T/F 1 11CA05 Fault at 1629Hrs on 19/9/08 - Fault Current waveforms at 66kV Side of HC 132/66kV T/F 2 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 -0.25 -0.2 -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2 0.25 Time (sec) Fault Current (66kV CT sec Amp) Ia - Fault Ib - Fault Ic - Fault This recording only shows the first 0.37 seconds of the fault. The phase for the initial breakdown has been highlighted, for clarity. The initial single phase fault has continued for 6.5 cycles, before developing into a two phase fault. The two phase fault, with transient and an

T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

  • Upload
    lethuy

  • View
    229

  • Download
    8

Embed Size (px)

Citation preview

Page 1: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

1

Technical Appendices T1 The Incidents T1.1 Transient Fault recordings and Fault Levels

Transient fault records were obtained from transient fault recording apparatus connected at 66kV, at a location remote from Casuarina. The technique for filtering the fault currents from the considerable normal system load at these locations does have some limitations. Nevertheless these were useful in understanding the relative magnitudes of the faults on the three occasions.

Transient Recording for the Breaker Fault

Casuarina T/F 1 11CA05 Fault at 1629Hrs on 19/9/08 - Fault Current waveforms at 66kV Side of HC 132/66kV T/F 2

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

-0.25 -0.2 -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2 0.25

Time (sec)

Faul

t Cur

rent

(66k

V C

T se

c A

mp)

Ia - FaultIb - FaultIc - Fault

This recording only shows the first 0.37 seconds of the fault. The phase for the initial breakdown has been highlighted, for clarity. The initial single phase fault has continued for 6.5 cycles, before developing into a two phase fault. The two phase fault, with transient and an

Page 2: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

2

asymmetrical component has persisted for a further 6 cycles before developing into a full three phase fault with an initial transient and asymmetrical component. At some stage during the two phase segment the 66kV breaker operated. From the transient record this would appear to have been after three cycles. (Whilst the operation of the 66kV breaker is recorded in the SCADA Sequence of Events, the Sequence of Events recording time alignment with the transient recording is not precise.) The various changes in the magnitudes of the currents over time and the relative magnitudes of currents in the three different phases are all consistent with what is known of the transformer and other system element impedances and the different behaviours of single two and three phase faults. For comparison purposes Transient Recording for the First Endbox Fault

11CA22 1st Bushing Failure at 2314hrs on 20/9/08- Fault Current waveforms at 66kV Side of HC 132/66kV T/F 2

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

-0.1 -0.05 0 0.05 0.1 0.15 0.2

Time (seconds)

Faul

t Cur

rent

s (6

6kV

CT

sec

amp)

Ia - FaultIb - FaultIc - Fault

This single phase only fault has continued for four cycles, before the operation of the breaker, which has cleared the fault after a further four or five cycles. The magnitude of this fault is approximately 2.5 times as great as that of the single phase, phase of the breaker fault. The explanation for this difference lies primarily in the different impedances of the transformers involved. The

Page 3: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

3

No.1 and No 2 transformers of the breaker fault are both of relatively high impedance for single phase faults. Whereas the No.3 transformer, which operated in parallel with the No 2 transformer to supply the first Endbox fault has a much lower impedance to single phase faults. Transient Recording for the First Endbox Fault

11CA22 2nd Bushing Failure at 2024hrs on 2/10/08- Fault Current waveforms at 66kV Side of HC 132/66kV T/F 2

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

-0.1 -0.05 0 0.05 0.1 0.15 0.2

Time (sec)

Faul

t Cur

rent

(66k

V C

T se

c am

p)

Ir-faultIy-faultIb-fault

This single phase only fault has continued for four and an half cycles, before the operation of the breaker, which has cleared the fault after a further three or four cycles. The magnitude of this fault is approximately 1.6 times as great as that of the single phase, phase of the breaker fault and about 63% of that of the first endbox failure. The explanation for these differences is that the second endbox failure was supplied by the No 3 transformer only.

Page 4: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

4

T1.2 The Breaker Failure

SCADA Sequence of Events.

1629 hrs Friday 19 September 2008 11CA05 TF1 CB internal fault.

Pre-fault:

TF 1 & 2 in service, supply to Bus’s 1,2 & 3 and Rear, all 4 bus sections paralleled at bus ties. TF3 energised with 11kV CB open. At 1600: TF1 1036A, TF2 1019A No staff on site at Casuarina ZSS, PN workshops on RDO. Generation on line: C1,3,4,5,6,7; W1; B2; PC1,2,3,4,5,6; K1,2,3; LMS. System Demand at 1600: 242.8, 1700: 187.7

SOE events record L&N SCADA: 1629:27.405 11CA05 TF1 CB TRIPPED A BIT 1629:27.411 66CA704 TF1 LOCKOUT RELAY OPERATED 1629:27.414 11CA05 TF1 CB TRIPPED B BIT 1629:27.419 66CA704 TF1 LOCKOUT RELAY CLEARED 1629:27.420 11CA05 TF1 CB CLOSED B BIT 1629:27.456 11CA05 TF1 CB TRIPPED B BIT 1629:27.456 11CA02 CAP BANK 1 CB TRIPPED A BIT 1629:27.462 11CA05 TF1 CB CLOSED B BIT 1629:27.462 11CA02 CAP BANK 1 CB TRIPPED B BIT 1629:27.463 66CA TF1 BUCHHOLTZ GAS OPERATED 1629:27.483 66CA704 TF1 CB TRIPPED A BIT 1629:27.503 66BE202 BE-CA DIST RAZOA STARTOPERATED 1629:27.545 66CA TF1 BUCHHOLTZ GAS CLEARED 1629:27.554 66BE202 BE-CA DIST RAZOA STARTCLEARED 1629:27.561 66CA TF2 BUCHHOLTZ GAS OPERATED 1629:27.571 66CA704 TF1 CB TRIPPED B BIT 1629:28.686 11CA13 TF2 IDMT OCEF OPERATED 1629:28.752 11CA13 TF2 CB TRIPPED A BIT 1629:28.782 11CA13 TF2 CB TRIPPED B BIT 1629:28.812 11CA13 TF2 IDMT OCEF CLEARED 1629:28.832 22MM02 CAP BANK OVERVOLTAGE OPERATED 1629:28.899 66CA TF2 BUCHHOLTZ GAS CLEARED 1629:29.562 11CA25 CAP ABNK 2 OVER/UNDER VOLT OPERATED 1629:29.637 11CA25 CAP BANK 2 CB TRIPPED B BIT 1629:29.637 11CA25 CAP BANK 2 CB TRIPPED A BIT 1629:29.640 11CA25 CAP ABNK 2 OVER/UNDER VOLT CLEARED 1629:33 CA CO2 STORAGE FAULT ABNORMAL 1629:33 66CA704 TF1 COOLING NORMAL 1629:33 11CA05 TF1 CB TRUCK RACKED OUT 1629:35 11PP PCPS BUS FREQUENCY HIGH ALARM 50.301 HZ 1629:38 CA SSTN FIRE ALARM OPERATED 1630:13 CA CO2 DISCHARGED OPERATED

Page 5: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

5

Physical Observations

Both the substation site and the failed circuit breaker were inspected in the course of this investigation. Upon inspection of the circuit breaker it was clear that the main contacts were still closed, indicating that the circuit breaker had not attempted to isolate the fault. It was unable to be ascertained if the trip latch mechanism had activated as the circuit breaker had been tampered with prior to the inspection being carried out. The primary isolating contacts and main contacts showed no significant signs of discolouration. Inspection of the circuit breaker bushings revealed evidence of a flashover from the front of the ‘B’ phase transformer side bushing to the circuit breaker tank. The location of the flashover was between the bushings and the closest point of the tank above the oil level, where the electrical stresses would be greatest. The bushings are synthetic resin bonded paper (SRBP) condenser type bushings and the condition of the electrical insulating papers adjacent to the point of the failure was dry and flaky, and exhibited the characteristic signs of delamination due to long term heat exposure. Degradation could also be observed over the entire length of the bushing, and the insulation was brittle at the primary isolating contact end.

Fig 1 – Damage to Circuit Breaker Bushing at the point of failure inside the tank

Fig 2 – Evidence of Flashover from Bushing to tank

Mechanical damage was also observed to have occurred at the top of the bushing spout (external to the tank), just beneath the primary isolating contact tulip. The chunky nature of this damage is characteristic of dry brittle insulation.

Page 6: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

6

Fig 3 – Damage to Circuit Breaker Bushing at the top of the spout, immediately beneath the primary isolating contact tulip

Fig 4 – Close up of the damage of Fig 3

Relevant Failure Modes

This type of failure has been observed in the industry many times on this type of circuit breaker, as both conditional and functional failures. In the observed cases, the root cause was found to be high resistance contacts (primary isolating contacts or internal circuit breaker contacts) causing degradation of the paper bushing. The high resistance contacts result in the sustained production of heat within the circuit breaker thus resulting in delamination of the paper insulation layers. This progressive degradation of the bushing insulation eventually leads to a situation where the heat generated by the conduction of normal load current through the circuit breaker is sufficient to trigger ‘thermal runaway’ thus resulting in insulation failure which leads to the production of a flashover between the bushing and the tank of the circuit breaker. This generally occurs either outside or inside the tank, just above the oil level. This short circuit can develop ultimately to a three phase to earth fault depending on the electrical energy present and the degree of oil vaporisation within the circuit breaker tank. The duration of the fault also impacts dramatically on the extent of the damage caused.

Assessment

The dry, flaky delaminated condition of the electrical insulating papers adjacent to the point of failure supports the conclusion that the bushing has been subjected to a prolonged period of “drying out” due to heat exposure. The position of the failure, just above the oil line is consistent with a source of heat from below, indicating that high main contact resistance was the likely

Page 7: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

7

primary source of the heat. However heat from the primary isolation contacts may also have been a contributing factor. The fact that none of these contacts showed significant signs of discolouration is not considered significant as the temperatures required to cause discolouration are considerably higher than that required to cause insulation degradation. The nature of the mechanical damage, observed at the top of the bushing spout, immediately below the primary isolating contact tulips, also supports the conclusion that the bushing has been subjected to a prolonged period of “drying out” due to heat exposure. This mechanical damage was most likely sustained as a consequence of the mechanical forces of the explosion, or during its removal from the switchboard panel, (that is, it is considered to have been caused by the failure and not to have been part of the initial failure). The chunky nature of the mechanical damage suggests that the insulation was dry and brittle at the time of the explosion. Un-degraded insulation would have been crushed rather than chipped by such mechanical trauma. Due to the shortcomings of the maintenance and testing regimes, as well as the damage caused to the bushing by the failure, the condition of the bushing prior to the failure is unknown and unable to be determined. However, comparisons can be made with the condition of the No 2 Transformer 11kV circuit breaker. These comparisons are possible due to the similar load, operating, maintenance and environmental conditions. Several hours following the No1 Transformer 11kV circuit breaker failure, the No 2 Transformer 11kV circuit breaker was removed from service for testing and maintenance. The staff that removed the circuit breaker confirmed that it was excessively hot. As well as replacing the oil, both contact resistance (µΩ, “ductoring”) and insulation resistance (GΩ) tests were performed. The contact resistance results were as follows:

A Phase (µΩ) B Phase (µΩ) C Phase (µΩ) As found 1st op (19/09/08)

1408* 411* 714.9*

2nd op (19/09/08)

969* 79 383*

As left (20/09/08)

138 129 165

* These resistance readings are well above what would be classed as acceptable for service.

Page 8: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

8

The insulation resistance tests were carried out at 5kV with the circuit breaker closed. The results were as follows:

* These insulation readings are considered fair, however may warrant further investigation.

A Phase (GΩ) B Phase (GΩ) C Phase (GΩ) As found 3.21 2.94* 2.74* As left 14.6 13.8 13.3

The ‘as found’ tests results indicate that the same failure mode was also present on this circuit breaker and that without corrective actions to improve the resistance of the contacts it too would have eventually failed in a similar manner.

Root Cause

Therefore, through examination of the failed circuit breaker, knowledge of other failures of this type of circuit breaker, and the indicative condition of the asset, the root cause of this failure is considered to be the failure of the bushing due to paper degradation from the conduction of heat, originating from high resistance contacts.

T1.3 The First Endbox Failure

SCADA Sequence of Events. 2314 hrs Saturday 20 September 11CA22 Nakara Feeder circuit bus fault.

Pre-fault:

TF 2 & 3 in service, supply to Bus’s 2 & 3, transformers in parallel. TF3, Bus 3 and Rear bus O.O.S. At 2300: TF2 638A, TF3 657A No staff on site at Casuarina ZSS. Generation on line: C1,3,4,6,7; W1; PC1,2,3; LMS. System Demand at 2300: 171.6

SOE events record L&N SCADA: 2314:49.311 11CA05 TF1 CB TRIPPED B BIT 2314:49.314 11CA05 TF1 CB CLOSED B BIT 2314:49.316 11CA22 NAKARA INST OCEF OPERATED 2314:49.376 11CA22 NAKARA CB TRIPPED B BIT 2314:49.376 11CA22 NAKARA CB TRIPPED A BIT 2314:49.418 11CA22 NAKARA INST OCEF CLEARED

Page 9: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

9

2314:49.431 66CA702 TF3 LOCKOUT RELAY OPERATED 2314:49.451 11CA17 BUS TIE 2-3 CB TRIPPED A BIT 2314:49.467 11CA18 HOSPITAL CB TRIPPED A BIT 2314:49.470 11CA05 TF1 CB TRIPPED B BIT 2314:49.470 11CA05 TF1 CB CLOSED A BIT 2314:49.470 11CA18 HOSPITAL CB TRIPPED B BIT 2314:49.470 11CA19 WANGURI CB TRIPPED B BIT 2314:49.472 11CA19 WANGURI CB TRIPPED A BIT 2314:49.479 11CA25 CAP BANK 2 CB TRIPPED B BIT 2314:49.479 11CA25 CAP BANK 2 CB TRIPPED A BIT 2314:49.479 11CA27 PATTERSON CB TRIPPED B BIT 2314:49.479 11CA27 PATTERSON CB TRIPPED A BIT 2314:49.481 11CA17 BUS TIE 2-3 CB TRIPPED B BIT 2314:49.488 11CA28 NIGHTCLIFF CB TRIPPED A BIT 2314:49.488 11CA20 TF3 CB TRIPPED A BIT 2314:49.491 11CA28 NIGHTCLIFF CB TRIPPED B BIT 2314:49.493 11CA20 TF3 CB TRIPPED B BIT 2314:49.505 66CA702 TF3 CB TRIPPED A BIT 2314:49.567 66CA702 TF3 CB TRIPPED B BIT 2314:49.631 66CA702 TF3 CBF FAILED

Physical Observations An “out of service” endbox at Casuarina Zone substation was inspected, in conjunction with the “Breaker Failure” inspection. The recovered bushing and several “spare” and previously recovered bushings were closely inspected. These inspections revealed the presence of surface irregularities on most of the bushings. These irregularities were of varying magnitude. Most were small. Some of the out of service “recovered and spare” bushings exhibited evidence of flaking of the insulation layering and breaches of the protective outer moisture sealing. None were suitable for service. Inspection of the recovered first failed bushing revealed that insulation breakdown had occurred at the point midway between the earth ring and the end of the bushing, well in front of where the bushing passes through the Bakelite insulating barrier at the back of the busbar chamber. Apart from the puncture hole, the insulation of the bushing was substantially in tact, except over that part of the bushing between the earth ring and the end of the earth screen. Over this part of the bushing the layer of insulation covering the earth screen and parts of the earth screen was detached from the bushing or missing. Inspection of the actual end box was not possible due to the access restrictions in place at the time. Photographic evidence was however provided by the recovery team.

Page 10: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

10

R

Fig 5 – View of the endbox showing the cables bolted to the ends of the bushings and covered by heat shrink boots. The failed C phase bushing is on the left. Note the burn mark above.

Fig 6 – Close up of the damaged bushing showing the position of the puncture through the bushing insulation.

NOTE (for comparison with the second failure): 1. Insulation panels on both sides of the box, secured in place by clips at top and bottom 2. The four mounting bolts in the four corners of the Bakelite panel (behind the cable terminations) 3. Confined damage to insulation panels and absence of any crazing.

NOTE (for comparison with the second failure): 1. The substantial hole burned through the SRBP insulation and the re-solidified copper of the underlying bus bar. 2. There s no splitting or tearing of the insulation. 3. The outer layer of the insulation is missing between the puncture and the earth bangle position.

Relevant Failure Modes Failures of this type, are typical of SRBP bushings, and have been observed throughout the industry on many occasions. SRBP bushings suffer gradual insulation degradation due to aging and are subject to rapid insulation degradation when moisture ingress from external environmental sources occurs. The intrinsic water content of SRBP bushing insulation increases naturally with age, as the chemical make up of the paper insulation changes, under the influence of time and temperature. This natural and gradual degradation process is accelerated under conditions of external moisture ingress. When degradation reaches an advanced stage, leakage current in the insulation develops to the point at which it causes damage to the insulation through the generation of heat, which leads to thermal runaway and ultimately complete breakdown of the insulation. Such degradation is being experienced, to at least some degree, in OLX switchboards, throughout the industry. Normally the management of moisture ingress is achieved by protecting the bushings from exposure to moisture (including humidity) by encasing them in compound or by various surface treatments (the continued integrity of which

Page 11: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

11

is vital). In more recent times this approach has been enhanced by using air conditioning to provide a low humidity, temperature controlled environment. Assessment For most of their lives the SRBP bushings at Casuarina have been relying on the continued integrity of either the original surface treatments or the surface treatments applied at the time of the endbox conversions to “air insulation”. It is also apparent from visual inspection of the bushings that in some cases the surface treatment, applied at the time of the “air insulation” conversions, was applied over the top of residual traces of compound. These and the other surface irregularities observed would have had the effect of degrading the surface integrity against ingress of moisture and of disrupting the electrical stress distribution. Given this environmental history, it is highly likely that all the bushings at Casuarina have experienced some degree of degradation, and that some have experienced considerable degradation. Temperature swings and increased temperature or accidental mechanical damage can be sufficient to trigger failure, in badly affected bushings. The ongoing management of such boards is discussed briefly in Section 10.1 The Future of the Casuarina 11kV Switchboard.

The unusual loading pattern, associated with the Zone outage of the 19th September and the subsequent abnormal network switching arrangements, though well within the design capacity, would undoubtedly have given rise to abnormal temperatures and temperature swings. Root Cause Having regard to:

• The environmental history, • The observed physical condition of the insulation of the failed

bushings and that of other identical bushings and, • The physical damage to the bushings,

The root cause of the failure is considered to be insulation degradation, due to aging and moisture ingress, from a lifetime spent in the hot humid tropical environment of Darwin.

Page 12: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

12

T1.4 The Second Endbox Failure

SCADA Sequence of Events. 2023 hrs Thursday 02 October 2008 11CA22 Nakara Feeder circuit bus fault. (Second instance of this failure)

Pre-fault:

TF 2 & 3 in service, supply to Bus’s 2 & 3, individual TF per bus. TF3, Bus 3 and Rear bus O.O.S. At 2000: TF2 783A, TF3 1041A No staff on site at Casuarina ZSS. Generation on line: C1,4,5,6,7; W1; PC1,2,3; LMS. System Demand at 2000: 218.9

SOE events record L&N SCADA:

2023:55.954 11CA22 NAKARA INST OCEF OPERATED 2023:56.011 11CA22 NAKARA CB TRIPPED B BIT 2023:56.011 11CA22 NAKARA CB TRIPPED A BIT 2023:56.059 11CA22 NAKARA INST OCEF CLEARED 2024:09 CA SSTN FIRE ALARM OPERATED 2024:54 CA CO2 DISCHARGED OPERATED 2023 Called out operators to attend CA ZSS and report. SMS advice of Incident message sent out. 2043 Hv Ops arrived and requested that Fire brigade be called to attend as CO2 discharge was confirmed. 2047 Fire Brigade on site. Fire crew in BA entered 11kV room at 2050. 2052 CA ZSS 11kV Bus 3 tripped on Frame Earth Leakage, nothing seen or heard at the site when trip occurred.

SOE events record L&N SCADA: 2052:49.523 66CA702 TF3 LOCKOUT RELAY OPERATED 2052:49.558 11CA19 WANGURI CB TRIPPED B BIT 2052:49.558 11CA BUS FEL ZONE REAR OPERATED 2052:49.558 11CA BUS FEL ZONE 5 OPERATED 2052:49.558 11CA BUS FEL ZONE 3 OPERATED 2052:49.558 11CA BUS FEL ZONE 1 OPERATED 2052:49.558 11CA02 CAP BANK 1 O.O.BAL TRIP 2 OPERATED 2052:49.558 11CA01 BUS TIE 1-R INST OCEF OPERATED 2052:49.561 11CA BUS FEL ZONE REAR CLEARED 2052:49.561 11CA18 HOSPITAL CB TRIPPED B BIT 2052:49.561 11CA18 HOSPITAL CB TRIPPED A BIT 2052:49.561 11CA19 WANGURI CB TRIPPED A BIT 2052:49.561 11CA BUS FEL ZONE 5 CLEARED 2052:49.561 11CA BUS FEL ZONE 3 CLEARED 2052:49.561 11CA02 CAP BANK 1 O.O.BAL TRIP 2 CLEARED

Page 13: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

13

2052:49.561 11CA01 BUS TIE 1-R INST OCEF CLEARED 2052:49.563 66CA703 TF2 TF RAKZB TRIP OPERATED 2052:49.563 66CA703 TF2 INST OCEF OPERATED 2052:49.563 66CA TF2 BUCHHOLTZ GAS OPERATED 2052:49.565 66CA703 TF2 TF RAKZB TRIP CLEARED 2052:49.565 66CA703 TF2 INST OCEF CLEARED 2052:49.565 66CA TF2 BUCHHOLTZ GAS CLEARED 2052:49.573 11CA25 CAP BANK 2 CB TRIPPED A BIT 2052:49.573 11CA26 NTU CB TRIPPED B BIT 2052:49.573 11CA26 NTU CB TRIPPED A BIT 2052:49.576 11CA25 CAP BANK 2 CB TRIPPED B BIT 2052:49.576 11CA27 PATTERSON CB TRIPPED B BIT 2052:49.576 11CA27 PATTERSON CB TRIPPED A BIT 2052:49.579 11CA28 NIGHTCLIFF CB TRIPPED A BIT 2052:49.582 11CA28 NIGHTCLIFF CB TRIPPED B BIT 2052:49.585 11CA20 TF3 CB TRIPPED A BIT 2052:49.597 66CA702 TF3 CB TRIPPED A BIT 2052:49.612 11CA20 TF3 CB TRIPPED B BIT 2052:49.659 66CA702 TF3 CB TRIPPED B BIT 2052:49.725 66CA702 TF3 CBF FAILED 2052:54 11CA20 TF3 PROT ABNORMAL

Physical Observations The recovered second failed bushing and the two “not failed” bushings from the other phases of the same endbox were closely examined and sent away for testing and dissection by the Energy Australia Testing Laboratory. A copy of the Test Report is attached as Technical Appendix T1.5 Energy Australia Test Report – Bushings. The dissections carried out by Energy Australia showed that the bushings have only three foils, an outer earth screen foil and two inner foils. Examination of the second failed bushing revealed significant damage at the end of the bushing immediately adjacent to the cable attachment, beneath the “heat shrink” boot which insulates the cable connection. Despite considerable tearing of the insulation there was little evidence of arcing to the busbar itself. Moreover a second puncture is apparent further along the bushing towards the earth bangle. This second puncture did not appear to have penetrated through to the bus bar, but appeared to have only gone as far as the outer foils of the bushing. Deep serrations were also in evidence along a 5cm length of the top of the bushing, just in front of where the arcing to the bar occurred. There was also a noticeable depression in the surface of the bushing at this point.

Page 14: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

14

Fig 7 – View of the damaged C phase bushing, showing the full length tear from the earth ring to the bushing end. A small bright patch of re-solidified copper is just visible 7.4 cm from the end

Fig 8 – Close up of the damage to the end of the C phase bushing, more clearly showing the small bright patch of re-solidified copper. Note the serrated scratches along part of the top of the bushing

Fig 9 – View of the damaged C phase bushing, from the other side, showing the shallow second puncture clearly visible 13.3 cm from the end. Note also the residual black material on the end of the bushing, underneath where the Heatshrink boot is located.

Fig 10 – Close up of the second puncture mark.

The dissection, at the Energy Australia Test Laboratory, established that the second puncture had in fact reached the innermost of the three foils, but had not punctured through it. The insulation between the inner foil and the bar itself was intact. It also revealed that the visible damage to the bus bar, at the point of the primary puncture, was very minor indeed. Photos of the dissected bushing, showing the extent of the puncture and the damage to the foils, at the position of each foil are included in the Energy Australia Report, provided in Technical Appendix T1.5 Energy Australia Test Report – Bushings.

Page 15: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

15

Inspection of the end box was not possible due to the access restrictions in place at the time. Photographic evidence was however provided by the recovery team.

Fig 11 – View of the LHS of the endbox Fig 12 – View of the RHS of the endbox

The staff who attended to this failure confirmed that these photographs were of the “as found” condition. (The cables are still bolted to the ends of the bushings and covered by heat shrink boots.) INSTALLATION DEFECTS - APPARENT FROM THESE PHOTOGRAPHS: 1. There are no insulating panels on either side of the box. The empty clips at the top are clearly visible. The flash mark at the side of the box confirms that the panels were missing at the time of the failure. 2. All four mounting bolts are missing. EVIDENCE OF PHYSICAL DISLOCATION 1. The part of the middle boot, covering the end of the bushing appears to be in contact with the rear Bakelite panel, and appears to have been bowed out, as if under pressure from the Bakelite panel. DAMAGE APPARENT 1. The LHS boot appears to have been “blown off” the bushing and a shiny black slag has exuded from the boot. 2. There are burn marks on the Bakelite panel behind the cable connected to the failed LHS bushing. 3. There are burn marks, and possible evidence of tracking, on the rear insulation panel, behind the cable connected to the failed LHS bushing, and extending to the area behind the cable connected to the middle bushing. 4. A flash mark is apparent on the LHS of the box, immediately adjacent to the failed bushing.

Page 16: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

16

Fig 13 – View of the of the endbox Fig 14 – Close up of the damage to the end of the C phase bushing and the cable termination. (The rear Bakelite plate of the endbox is still in position)

The box is partially dismantled. (The cables have been unbolted from the bushings. The heat shrink boots have been removed.) ADDITIONAL INSTALLATION DEFECT - APPARENT FROM THESE PHOTOGRAPHS: 1 The offset of the cable lug is the wrong way round. EVIDENCE OF PHYSICAL DISLOCATION 2. The limited length of the bushing protruding from the Bakelite panel. (Note however that the box may have further dislocated during the dismantling.) DAMAGE APPARENT 1. The burning and possible tracking previously noted in fig 11 and fig 12 are more apparent in these photographs. Note, in particular, the burning on the insulation apron, between the blue and yellow phase cables.

Fig 15 –View of the endbox from the top. Fig 16 – A wider view from the top

The staff who attended to this failure confirmed that these photographs were of the “as found” condition. EVIDENCE OF PHYSICAL DISLOCATION 2. The box is physically located approximately 7.5 cm. from its intended mounting position. 2. Note the serrated scratches along part of the top of the failed C phase bushing 2. Note also the gap between the visible edge of the steel back of the endbox (underside of bushings) and the insulating apron of the back of the box.

Page 17: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

17

From an examination of these photographs of the endbox, taken following the second failure, and a comparison with the photographs of the endbox, taken following the first failure the following installation defects are apparent:

• The insulation panels at the side of the endbox are missing. • The four holding bolts which secure the endbox to the Current

Transformer (CT) Chamber are missing. • The cable termination lug is installed with the “offset” to the back,

instead of the front of the box.

It is also apparent: • That the endbox was dislocated from it’s mounting by approximately

7.5 cm. • There is considerable “burning” to the Bakelite and Insulating apron

of the back of the box in locations, remote from the apparent arc path, but in the vicinity of the blue and yellow phase cabling.

• That the bushing has split open.

The diagrams on the next two pages are approximate representations of the way the endbox was designed to be assembled and the way in which it was found. It is also apparent from the evidence that the various components of the endbox had been in (or very nearly in), the as found positions for a considerable period of time (for most of the twelve days since installation).

Page 18: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

18

The Intended Geometrical Configuration of the Box

0 kV

6.3kV4.2 kV

2.1 kV

Bakelite Panels (black) and Insulation apron (gray) are sitting over the top of the earth screen of the bushing (green). Consequently there is negligible electrical stress in the air gap between these insulants and the bushing.

Feeder cable (blue) is approximately 15cm from the Bakelite and insulating apron at the back of the box. Insulating apron is hard against the earthed steel of the rear of the box. The electrical potential on the surface of the apron is insignificant.

This diagram is only approximately to scale. Horizontal scale is approximately full scale. Vertical scale is approximately half scale.

Page 19: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

19

The Actual Geometrical Configuration of the Box

0 kV

6.3kV

Feeder cable (blue) is approximately 5 or 6cm from the Bakelite and insulating apron at the back of the box. Insulating apron is 1 to 2 cm. from the earthed steel of the rear of the box. The electrical potential on the surface of the apron is now significant. Potential induced by Blue and Yellow Phase cables are 120o apart, causing a potential gradient across the surface.

Second Bakelite Panel is still over the earth screen, but the induced voltage from the 6.3 kV cables causes electrical stress in the air gap.

First Bakelite Panel is now over the 4.2kV screen, with a resulting high electrical stress in the air gap.

This diagram is only approximately to scale. Horizontal scale is approximately full scale.

2.1 kV

Approximate position of punctures.

4.2 kV

Vertical scale is approximately half scale.

Page 20: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

20

From considerations of the electrical stress distributions within the box it is apparent that there were at least three stress related breakdown mechanisms potentially at work, the first two of which could explain the failure.

• Normally in this box both Bakelite insulating panels (and the insulating apron between them, would sit over a part of the bushing that was protected (in the electrical stress sense) by the bushings internal earth screen. This means that the electrical potential on the surface of the bushing under the Bakelite is at or near earth potential and that that the potential gradient in the air gap between the bushing and the Bakelite is very low.

• For the box, as found, the Bakelite panel which is interior to the box (the first Bakelite panel) was sitting over a part of the bushing where there was no earth screen and no outer foil. As a consequence a potential of the order of 4.2 kV was distributed across the bushing, the air gap and the Bakelite panel. In these circumstances the potential gradient within the air gap is significant and given the irregular shape of the air gap and texture of the surface of the Bakelite, the conditions for electrical discharges across parts of the air gap existed. These discharges could have, over a period of time (up to twelve days), eroded the bushing, all the way through to the bar itself. (The position of the puncture, directly under the Bakelite, slightly to the left of top is consistent with this explanation. At the very top, the Bakelite and the bushing insulation were in intimate contact and no air gap actually existed. And for most of the rest of the bushing surface the air gap was of such a width that the potential gradient was relatively low. But at the point where the intimate contact between the Bakelite and the bushing begins to separate, the potential gradient is greatest, and the irregularities in the Bakelite surface, where field intensities concentrate, are of greatest impact.)

• Normally also for this box the spacing between the cables and the back of the box, where the Bakelite panel is, is of the order of 15cm. Moreover there is sufficient of the bushings earth screen protruding in front of the Bakelite to ensure that the potential at the air gap and the gradient within the air gap is negligible.

• For the box, as found, the cables would appear to have been as close as 5cm from the inner Bakelite panel, with a further approximately 5cm to the outer Bakelite panel. Moreover there was no part of the earth screen protruding in front of the Bakelite. As a consequence there has been an electrically induced potential across the air gap and there is a possibility that the same mechanism as suggested for the interior Bakelite panel occurred here also. This however is considered unlikely as the induced voltage would be quite small and from what little can be ascertained from photographs, the air gap at the point of failure was quite large.

Page 21: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

21

• The more likely explanation is that once the erosion of the first puncture had established a conduction path between the 6.3 kV surface of the bar and the inner, and possibly even the outer, of the two foils the increased potential gradient between the foils and the earth screen has produced rapid internal breakdown at the location of the second puncture, which has ultimately led to conduction via the earth screen. The location of this second puncture at the very edge of the earth screen is explained by the concentrating effect of the edge, on the potential gradient at the edge. This was the point of greatest potential gradient in the bushing insulation. The positioning of the outer Bakelite panel at this point is likely to be no more than a coincidence.

• Under this scenario the electromagnetic forces of the current flowing in the foils and screen would have almost instantly burst open the insulation, which would explain the split along almost the full length between the end of the bushing and the earth bangle. On bursting open, the conduction path along the foils would have been broken and the arc, then established in the primary puncture, would have blown out to the external steel casing of the box. The arc within the secondary puncture would have been extinguished, for want of a voltage potential source. The puncture to the outside of the bushing has occurred as a consequence of the momentary arc formed during the brief moment of conduction and has occurred from the inside out, rather than the outside in.

• The quite minor damage to the bus bar at the point of the primary puncture suggests that the arc did not stay in this location for the full duration of the fault. A possible explanation for this is that the bursting open of the insulation of the bushing exposed the cable termination lug and that the arc, under the influence of the electromagnetic forces either moved along the exposed surface of the bar, or jumped to the cable lug. The cable lug is currently inaccessible due to the access controls. Once access is again available the condition of the end of the lug should be checked for signs of arc damage.

The third mechanism could explain the extensive burning and possible tracking marks on the interior Bakelite panel and the insulating apron.

• As noted above, for the box as found, the cables would appear to have been as close as 5cm from the inner Bakelite panel and insulating apron, with a further approximately 5cm to the outer Bakelite panel. Moreover for that part of the space where there is no “cut out” in the steel rear wall of the box there appears to be a spacing of the order of 2 cm. between the insulating apron and that steel wall. Under these circumstances there would be considerable electrical gradients across the surfaces of both the Bakelite and, perhaps more so, the surface of the insulating apron. The close proximity of both the blue and yellow phase cables would have been significant for the apron, in particular.

Page 22: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

22

The existence of these gradients could explain the burning and apparent tracking.

Root Cause

The second of the two cable endbox failures was attributable to an installation defect, which has arisen as a result the re-installation of a thirty six year old degraded Wagaman feeder endbox bushing into the Nakara endbox. Whilst there were a number of installation defects, any one of which could have eventually led to failure, the particular defect which was clearly the root cause of this failure was the omission of four mounting bolts. The omission of these bolts allowed the endbox to “walk away” from its mounting, due to vibration, and cause a relocation of critical insulation material within the box. The consequent changes in the electrical stress distribution within both the box and the bushing resulted in electrical breakdown. This breakdown is likely to have occurred, regardless of the initial condition of the bushing. The degraded bushing has simply failed earlier than an un-degraded bushing would have.

Page 23: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

23

T1.5 Protection Issues

For none of the above three incidents did the protection systems work fully in accordance with the design intentions. All of the malfunctions concerned the operation of the Frame Earth Leakage (FEL) systems. Only one is clearly understood and convincingly explained. This was the failure of one feeder breaker to open due to a burnt out coil, on the occasion of the first endbox failure. Those malfunctions that have not been convincingly explained are:

• The failure of the FEL systems to operate on the occasion of the breaker fault.

• The failure of the FEL systems to operate on the occasion of the second endbox fault.

• The spurious operation of the FEL system 29 min after the second endbox fault.

Additionally no credible explanation has been found for the operation of the No 1 Transformer 66kV breaker on the occasion of the breaker failure. Considerable effort was made by PAWC personnel, to assist the Enquiry in trying to understand these matters. A small “Protections Operations Investigation” group was convened and this group met on three occasions to collate and assess the available data. The work of the group was hampered by the lack of recent testing of the Frame Earth Leakage Systems and by the access restrictions put in place following the first endbox failure. Nevertheless considerable data was assembled and assessed and, as access restrictions eased, some relay testing and further inspection was undertaken. SCADA Sequence of Events reports, records of various relay status indicators, and the transient fault recording traces were all used. Whilst there may have been some gaps/discrepancies in the SCADA record, due to SCADA polling and queuing limitations, the record of most of the protection operations that did occur is known with confidence. Frame Earth Leakage (FEL) Failures In addition to the three FEL failures associated with incidents currently under investigation, Protection staff reported that there had been a history of spurious FEL operations. The precise circumstances of these earlier incidents were unclear. Some were reported to have been associated with cable faults,

Page 24: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

24

within the zone, remote from the substation, but others it seems were not associated with any actual fault at all. Given the lack of testing of the FEL systems it is tempting to conclude that the problems would be due to relay malfunction. However in the case of the second endbox failure, the FEL system concerned had operated substantially correctly twelve days previously and the one faulty component that had caused a problem had been replaced. Intermittent relay malfunctions are a possibility, but there are other possibilities as well. The main one being that the FEL earth path has been shorted to earth somewhere ahead of the FEL current transformer. Examination of the earth bus installation at Casuarina revealed that most of the installation sits within unrestrained close proximity to the steel frame of the switchboard. Whilst the available evidence is not conclusive it is possible that the cause of some of the problems, including that of the breaker failure, is that under certain fault conditions, the electromagnetic forces due to the high currents in the bar and surrounding components could cause sufficient movement of the bar to bring it into contact with the switchgear frame.

Fig 17 Shows earth bar and bottom connection – Wagaman Feeder

Fig 18 Shows earth bar and top connection – Nakara Feeder (pulled out of position)

Vertical earth bar lies between the switchgear frame and feeder cable, approx. 5cm from the frame and 10cm from cable. It is secured top and bottom only.

In the case of the No 1 Transformer, the sheaths of the nine 11kV transformer cables are bonded together and earthed at both the switchgear end and the transformer end. Under fault conditions this will have two effects. Firstly there will be substantial circulating currents in the cable sheaths and the bonding. The predominant return path for the fault induced current of the faulted phase will be the sheaths of the other six cables, although there will be

Page 25: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

25

a minor component flowing up the earth bar as well. Secondly the cable sheaths will present the least impedance return path to the actual fault current. As a consequence there will be a considerable component of fault current flowing up the vertical earth bar. The consequent electromagnetic forces on the unrestrained earth bar will undoubtedly cause it to be flung about somewhat. It is however, unlikely that a return earth path would be established with sufficient strength and duration to divert enough current away from the FEL system so as to cause it to malfunction. Moreover examination of the earth bar and surrounding switchgear frame and endbox, has failed to find any convincing evidence that such electrical contact has occurred. The No1 Transformer Endbox and Transformer Cable Earthing Bar

Fig 19 Indicative current directions – blue Forces on the earth bar - red Examination of the steel panel at the back of the switchgear also reveals that the bolts which hold the panel in place against the switchgear frame do not appear to have been in place at the time of the fault. The bolts are missing, there is no damage to the bolt hole of the sheet metal, where the bolt heads may have been pulled through, and the threads in the frame are not stripped. It is distinctly possible therefore that the steel panel may have been leaning outwards and contacting the earth bar, and thereby providing a conducting path between the two earth systems. Examination of the earth bar and the steel panel, has however failed to find any convincing evidence of electrical contact between the two.

Page 26: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

26

Fig 20 Empty bolt holes of the frame are clearly visible

Fig 21 Bolt holes of panel are undamaged

On the occasion of the second endbox failure it is possible that the dislocation of the endbox placed simple mechanical forces on the earth bar which caused it to bow and come into contact with the switchgear frame. Physical inspection of the panel behind reveals some evidence of marks on the panel, along the alignment of the bar and a distinct shadow of the bar, in the lower section. The evidence is however not conclusive. Further closer inspection, when access is more readily available, may help to clarify the matter. The Nakara Endbox (11CA22) – Feeder Earth Bar and Panel Proximity

Fig 22 A shadow is clearly visible on the panel behind the bar. (Note that the bar is pulled away from the panel in this photograph. It normally sits about 50mm from the panel, similiar to the arrangement that can be seen for the panel to the immediate right). A number of marks are also visible on the panel along the alignment of the bar.

Page 27: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

27

The problem of unconstrained proximity and the forces acting on the bars when fault currents flow, is also possibly one of the explanations for the spurious FEL operations reported by Protection staff. Any circumstance in which a fault current flows in the bar in the opposite direction to the net current flowing in the feeder cable would potentially initiate a spurious FEL operation. The reason that, on the occasion of the second endbox failure, the FEL operated 29 minutes after the event has not been credibly explained. The possibility of a relay malfunction lacks credibility. However the only other credible explanation lies with the possibility that the operation of the relay may have been triggered by the entry into the substation by the Fire Brigade. (As near as can be ascertained, there is time alignment between the two events.) The 66kV Breaker Operation (The Transformer Breaker Failure) It has not been possible from analysis of the available data to draw any conclusion as to the protection operation which caused the 66kV breaker to operate. The flag on the relay itself when checked by Protection staff indicated that the relay had not been operated. However when checked the relay proved to be functioning appropriately. Summary Regardless of the uncertainty about the actual causes of the FEL malfunctions and the 6kV protection operations, it is clear that there were at least two significant issues with the installation:

• The unconstrained proximity of the cable sheath copper earth bar to the frame of the switchgear and its proximity and configuration relative to the cables entering the endbox, is a design or installation weakness.

• The apparent failure, on the most recent previous occasion when the panel at the back of the No 1 Transformer breaker had been opened, to re-bolt the panel back in place at the conclusion of the work.

Page 28: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

28

T2 Substation Maintenance

T2.1 Policy Documentation Documentation Licensed from ETSA

The following is a list of the Substation Maintenance Policies, licensed to PAWC by ETSA. The Policies highlighted in yellow are the ones selected for evaluation.

Tab Document name TRIM Folder PWC SMI No

1 Job Safe Work Procedure – Substations F2006/11086 JSWP/001 Instructions common to all Circuit Breakers:

2 General Maintenance Procedures for CB’s F2006/11085 GI/CB/001 3 Handling Sampling & Use of Insulating Oil for CB’s F2006/11087 GI/CB/002 4 SF6 Safe Handling & Filling Procedures F2006/12585 GI/CB/003

Circuit Breakers

5 66kV ABB/Alstrom SF6 CB’s Type SI-145 & SI-72-5 F2006/12191 66/CB/001 6 66kV Delle Type HPGE9-12E SOV CB F2006/11268 66/CB/002 7 66kV ABB SF6 CB Type ELG F2006/11266 66/CB/003 8 11kV GEC & Alstom VCB Type SBV/SBV2 F2006/14606 11/CB/001 9 11kV Westinghouse/Email LC/A (J18 & J22) F2006/14607 11/CB/002

10 11kV Merlin Gerin SF6 CB Type LF F2006/14608 11/CB/003 11 11kV Siemens CB Type 3AF F2006/14610 11/CB/004

DC Supplies

12 General Commissioning & Maintenance Procedure For Substation & communication Battery Banks

F2006/15176 DC/BB/001

13 Replacement of Substation & Communication Battery Banks

F2006/15177 DC/BB/002

14 Replacement of Recloser Electronic Controller Type ME Form 3 Battery Banks

F2006/15178 DC/BB/003

Power Transformers, Regulators and OLTC

15 General Transformer Maintenance F2006/11088 TF/001 16 General Maintenance of CT, VT F2006/11090 TF/002 17 Maintenance of Transformer Valves F2006/11089 TF/003 18 Maintenance of Ferranti OLTC Type ES3 F2006/15179 OLTC/001 19 Maintenance of Rheinhausen OLTC Type M & MS F2006/15180 OLTC/002 20 Maintenance of English Electric OLTC/Regulator F2006/15181 OLTC/003 21 Maintenance of Crompton Parkinson OLTC/Regulator F2006/15182 OLTC/004

RMU

22 11kV South Wales Type C4X/D4X RMU F2006/14611 RMU/001

Page 29: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

29

23 Brown Boveri RMU Type RGBC & RGBF F2006/14612 RMU/002 24 English Electric 11kV Bulk Oil CB Type OLX Series 11

with Wing Disconnector RMU F2006/14613 RMU/003

Isolator

25 ETSA Duoroll 66/33/11kV Disconnectors F2006/15183 DIS/001 Internal Documentation

Formally documented policies for the protection systems were not sighted. However, acceptable test plans are programmed into the software package that is used to test the protection systems. There did not appear to be a policy regarding the testing of frame leakage busbar protection.

T2.2 Evaluation of Policies Policy Documentation Licensed from ETSA

In general, many of the required maintenance activities detailed in these documents are not described in adequate detail for staff to carry out. In some instances they are inadequately and incorrectly referenced and/or pertain to equipment that does not exist on the PAWC network. The level of detail in the policies, the use of references to other documents and applied format is inconsistent between policies. Some of the documentation is of such a poor standard that it should not be used for maintenance activities. Below is one example of issues that were found with the substation maintenance instruction RMU/003 reportedly for ‘English Electric 11kV bulk oil circuit breaker type OLX series 11 in 11kV RMU with Wing Disconnectors’. Similar issues exist in all six policy documents reviewed. 1. The English Electric OLX 11kV circuit breakers in PAWC do not have

‘11kV RMUs with Wing Disconnectors’. 2. The list of required ‘Tools & Equipment’ on the front page of the

instruction is clearly deficient upon reading the requirements of the instruction.

3. Section 3.1, Step 3 calls for Ductor tests to be carried out. It states “Acceptable values are.....’ but fails to state what the acceptable values are or where to reference these values.

4. Section 3.1, Step 5 calls for ‘Carry out Secondary Isolations in accordance with Section 3.....’ These instructions are in fact in Section 2.

5. Section 3.3, ‘Caution’ note has the ‘Danger’ symbol denoted beside it (the legend is shown in the front page of the instruction).

6. Section 3.3, ‘Caution’ note again calls for secondary isolations to be carried out, and again incorrectly refers to Section 3.

Page 30: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

30

7. Section 3.3.1, Step 4 refers the maintainer to a diagram contained in the instruction. In the printed version of the instruction provided for this investigation, the settings detailed in this diagram were found to be illegible. It is assumed that any document on site would be of similar quality, thus the required correct main contact settings would be unable to be ascertained during maintenance using this instruction.

8. Section 3.3.1, Step 5 calls for replacement of the oil, however, Step 6 calls for examination of the tank and tank liner. The instruction should sequentially list the requirements so as to avoid rework.

9. Section 3.6, Step 3 calls for a 500V insulation test of the trip coils, however, fails to refer to any acceptable values for the results of this test.

10. Section 3.6, Step 4 refers to tests in Section 7; however these tests are in fact detailed in Section 6.

11. Section 3.6, Step 5 calls for final Ductor tests, however, fails to refer to any acceptable values for the results of this test.

12. Section 3.7, Step 4 is unnecessary in PAWC as these circuit breakers have trailing leads.

13. Section 4, ‘Caution’ note does not have the ‘Caution’ symbol denoted next to it, as per the legend on the front page of the instruction.

14. Section 4, Note should be detailed in the step-by-step instructions as it is necessary for the successful completion of this part of the maintenance instruction.

15. Section 4.1, Step 1 refers to ‘Figure a’ which does not exist as a titled figure within this instruction. It is assumed that the untitled figure on the following page is referred to.

16. Section 5, refers to the maintenance of oil-filled RMUs – as mentioned above, these are not a feature of the equipment in PAWC.

17. Section 5, refers to required periodicies for maintenance of the 11kV RMUs. No other reference is made to required periodicy for maintenance of the circuit breakers.

18. Section 6, ‘Danger’ note does not have the ‘Danger’ symbol denoted next to it, as per the legend on the front page of the instruction.

19. Section 6, ‘Danger’ note again refers to secondary isolation procedures in Section 3, these procedures are in Section 2.

20. Section 7.3, Figure 3 is incorrectly titled ‘OLX Circuit Breaker Settings’ – these settings are shown in Figure 2.

The detailed review of the available policy documentation assessed the documentation to be of an inadequate standard on many levels: - The acquired documents from ETSA were not suitably tailored for the

PAWC work environment. There appears to have been some attempt to make the documents suitable as PAWC documents, and it is evident that there was no quality checking or rudimentary change control.

- It is understood that there was little or no field staff involvement in reviewing the accuracy or content of the policies. In some cases, the

Page 31: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

31

policies contained references to assets that PAWC do not have on their network; and in other cases the tasks listed were unable to be carried out as staff had neither the training nor knowledge of the tasks, nor the equipment to carry them out successfully.

- These policy documents could not stand alone as the only reference documents for staff to carry out maintenance activities due to the lack of clarity around the required tasks. Some shortcomings include, lack of pass/fail criteria where testing is required, incorrect ordering of tasks and, in many cases, lack of explanation of how to complete tasks.

- Generally in industry there has been a move away from these more intrusive, time-based styles of maintenance, to a condition based, diagnostic method. This newer type of maintenance helps to focus resources on the right amount and type of maintenance, in order to achieve the desired business and safety outcomes at the optimum cost.

Internal Policy Documentation (Protection Systems)

The test plans that are programmed into the software package used to test the protection systems appears to be generally satisfactory for highlighting the defects in the relays that are being tested during maintenance. There did not appear to be a policy regarding the testing of frame leakage busbar protection, and none was evaluated. Policy Documentation Structure Generally in industry, policy, standards and procedures for maintenance are not dealt with in one document, such as has been attempted by PAWC. A more thorough and systematic approach is to:

• Firstly have a high level policy that identifies what the organisation is trying to achieve within the specific areas of maintenance.

• Below this sits specific standards that articulate what is to be done and when. These standards usually go down to the level of equipment manufacturer and type, however, there may be additional levels required depending on the installation environments and the operational usage.

• The third level of documentation is the actual maintenance procedure. The procedure articulates how the job is to be done and by whom. This low level document should include information such as references to Safe Work Method Statements, required safety equipment, skill sets, materials and test instruments.

Although this systematic stepped approach to maintenance documentation is desirable, the real issue of criticality is that there are demonstrated clear links

Page 32: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

32

between what work is required, why it is required and how the work is to be done in accordance with the correct safety procedures.

T2.3 Evaluation of Practices Substation Maintenance (ex Protection)

The review of the actual field practices undertaken was carried out by interviewing the people responsible for the work required within the six policy documents. These interviews highlighted that there was no attempt in the field to implement these policies due to a distinct lack of knowledge regarding the policies. Therefore an assessment was made of the adequacy of the tasks that were actually being carried out, through questioning of the field staff.

The field practices assessed can be divided into three areas: 1. The tasks carried out 2. The documentation used to record this work and any defects 3. The knowledge of the staff of the tasks carried out

The tasks that are carried out were generally not being completed at regular intervals. In many cases the specified frequency of two years (for switchgear) was not being adhered to. An example of this is the fact that the most recent test results for the Casurina 11kV switchgear were from 2002. This was consistent with records sighted for a number of other substations.

The practices employed at PAWC for switchgear maintenance generally do not have any justification. The processes used and order of activities had many issues, and the staff interviewed appeared to have only ever been shown what to do by the “father-son” method. In addition, there were differences between staff interviewed in what tasks were undertaken due to differences in beliefs regarding what was considered adequate.

It became evident that the staff generally did the minimum possible work during these intrusive tasks and rarely highlighted defects. It appeared they were merely going through the motions so they could fill out the paperwork as this was regarded as low priority work. On questioning, it was clear they had never been given appropriate training or knowledge on why they were doing the tasks, and the critical importance of identifying and correcting defects. The knowledge and skills that have been given to these staff appears to be generally inadequate for them to successfully carry out the required maintenance activities, identify the appropriate failure modes and defects, and to successfully correct them.

Similar comments apply to the maintenance practices for transformers (excluding tap changers), and instrument transformers. The maintenance of

Page 33: T1 The Incidents - Power and Water Corporation · T1 The Incidents . T1.1 Transient ... Transient fault records were obtained from transient fault recording apparatus connected at

33

tap changers is outsourced and was not able to be reviewed as a part of these interviews.

Protection Maintenance

The testing of protection relays in accordance with the test plans that are programmed into the software package appears to be generally satisfactorily carried out. The failure to routinely test Frame Earth Leakage systems is a shortcoming. It is however recognised that gaining network access to test such a protection scheme can be difficult. It is critical that a policy, standard and procedure for this type of work should be documented and testing routinely implemented.

T2.4 Evaluation of Condition Monitoring Equipment and Capability

Generally PAWC have some equipment capable of carrying out condition monitoring but this equipment is used primarily for fault finding. If PAWC intend to utilise condition monitoring techniques as a maintenance strategy then there are many areas that need to be reviewed. PAWC does not have the appropriate equipment, staff skills and training required to fully implement condition monitoring at this time. It is recommended that a strategy be developed to progressively implement these techniques over a period of time. This strategy should include implementing in areas where the greatest benefits can be made. In order to do this a thorough review of appropriate available equipment must be undertaken, and staff must be adequately trained on the basis behind condition monitoring and the techniques involved.