Upload
hadung
View
214
Download
1
Embed Size (px)
Citation preview
Gloucester Water Supply
Gloucester’s water and sewerage services
handed over to MidCoast Water in July 2011
4,000 people in 1,900 homes
WTP capacity 4.5 ML/day
3 service reservoirs and 7 booster pump
stations, town reticulation includes 28km of
AC main
The incident – Day 1 – Tues 24 March
6am: Operators noticed a stronger than usual chlorine smell at the WTP
Morning – normal reticulation sampling taken, with chlorine levels 3.7 & 3.8 mg/L.
Operator reduced the chlorine dose at the WTP based on the samples
12.04pm – first call from a customer – water tasted like sulphur and was bitter
A work order was raised, mains flushing commenced
2pm to 3pm – 2 more calls, one being from Gloucester Hospital
Minor feedback commenced on social media
Plan to further investigate the following day to prevent the use of on-call
The incident – Day 2 – Wed 25 March
9am – first media enquiry – responded to by customer
service. Based on operational advice, the issue was
poorer river quality that had occurred over the
weekend
Customer calls increase rapidly (314, or more than
three time average)
Customer service recommend a recorded message be
added to assist with customers left on hold
Social and traditional media in full flight…
Day 2 was a long day…
Chlorine levels registered above field instrument range in reticulation (>8.8 mg/L)
11.20am Incident notified to Executive – an incident management team put together.
Telstra have planned shutdown of mobile service between approximately 11am and 3pm
Boil water alert was ruled out – needed to be a ‘do not drink’ alert
12 noon NSW Health and NSW EPA notified
Additional crews arrive at Gloucester from Taree to assist with flushing and testing
Door knock and letter drops rules out due to time of response required. Expectation of a
less than 1 day incident
Crates of bottled water purchased and conveyed to Gloucester
Incident management team
Incident controller – Executive Manager Service Delivery
Board members, television, radio – Acting General Manager
Media releases and social media – Public Relations Manager
Site incident control – Group Manager Catchment & Treatment
Customer service coordination and notification – Customer Service Mgr
Corporate support – Executive Manager Corporate Services
Social media
We started using Twitter in Feb 2014
during Level 3 water restrictions
Commenced Facebook in June 2014
after formalising a social media policy
and procedure
Day 3 – Thurs 26 March
Independent engineering investigator engaged by MidCoast Water to:
- identify the root cause of the incident
- comment on MidCoast Water’s response to the incident, and
- recommend any improvements to the response
Flushing and intensive chlorine monitoring continues
Confirmed to Newcastle media that the incident had ‘nothing to do with
Coal Seam Gas’
Day 4 – Fri 27 March
$50 rebate on water bill announced
9am MCW Councillors and Executive
arrive in Gloucester to speak with
community members
11am Independent investigator Hunter
H2O arrives on site in Gloucester to
commenced site review.
Likely causes
Hunter H2O established five likely causes
1. Dosing system ran on after plant shut down
2. Dosing system left in test mode (25mg/L)
3. Dosing system set to maximum dose rate (25mg/L)
4. Sabotage
5. Syphoning
Recommended the plant be run manually
until rectified
The following weeks Customer sample indicates worse than measured in reticulation
Incident debrief of MidCoast Water staff undertaken
Temporary online chlorine analyser installed, plus daily records being initiated
of hypochlorite storage tank levels
Plant run manually with hypochlorite tank isolated at end of each run
Computer modelling undertaken of a high chlorine dose by MidCoast Water
Further independent investigation
Joint strategic meeting with NSW Health and NSW Office of Water convened
to consider root causes of the incident
The technical cause
Siphoning of hypochlorite solution into the
treated water after plant shut down.
The problem was intermittent and couldn’t be
replicated until the hypochlorite tank was refilled
Hypochlorite dosing pumps and loading valves
improperly designed and commissioned
The root causes
A shortfall in the right staff resources
Inadequate training of current staff
Lack of quality systems and internal controls
No integrated business management system
Lack of system knowledge management
Facilities not up to current design standards and a lack of change management
Incomplete implementation of the drinking water quality plan
Areas for improvement
Time taken to realised there was an issue.
Clear evidence of a problem on Tuesday afternoon
No Incident Management Plan
There wasn’t a listing of critical customers (late notification)
There is a misalignment of MCW CCP, DWMP and the ADWG’s
Staff understanding of CCP’s
Areas for improvement
Shortage of resources:
- only one staff member had a sound understanding of the reticulation
- Staff were stood down for fatigue and concern with overtime
The level of chlorine in system was not quantified
The issues with the dosing system could have been picked up earlier
A chlorine analyser at Tyrell Street had not been calibrated or attended for a
number of years
Previous reviews of the WTP had identified the need for online instruments
Areas for improvement
There was a lack of reviews undertaken
on the hypo system
- Design basis and design reviews
- Compliance checks with AS3780
- HAZOP, Chair (1-3)
- Dosing location
- Functional description, etc.
Areas for improvement
There are numerous other risks at the plant
- Lack of control, automation and monitoring
- Compliance with standards – OHS, AS 3780
- Lack of documentation and update manuals
or procedures
- Training of staff and resources
There are other issues identified in the plant
that pose a risk to water quality
What worked well
Once the incident was declared everything worked as if there was a incident
management plan in place
‘Response by the Executive and Staff during the incident was exemplary’
Notification to regulators
Updates to the public (Mainstream and Social Media)
$50 Rebate and payments to commercial customers
Independent Review
‘Meet the Public Day’
Apology Letter to Public
Action planning
Many actions were able to be implemented in the week of the
incident
60 actions identified in regards to engineering/technical causes
and scheduled for implementation over a 6 month period
56 actions identified in regards to the root causes of the
incident and scheduled for implementation over a 2 year period
A step change in resources was required for implementation
Summary
Does this sound like a water treatment plant you might own and operate?
Blame the systems not the people
Hard lessons create a burning platform for change – a much sharper focus
business systems and risk management
Running ‘lean and mean’ can get you in trouble
‘Normal’ may actually be risky. Normal operation needs regular challenge
Transparency is critical for community and regulator confidence.
Have a plan to ensure that it won’t happen again. Share it widely.
Not sure how the situation would have unfolded prior to MidCoast Water
Credit
Independent engineers – Hunter H2O
- Paul Thompson, Alan Thornton, Clara Laydon, Evan Jack
MCW staff and councillors