Towards Targeted Reliability Oceans 2025 Theme 8 WP8.4

Risk and Reliability …

Towards Targeted ReliabilityOceans 2025 Theme 8 WP8.4

Gwyn Griffiths and Mario Brito

National Oceanography Centre, Southampton

2008 Moorings Workshop

Is there common ground between people working on moorings and those working on

improving the reliability of Autonomous Underwater Vehicles?


Fimbul Ice Sheet, Antarctica

Nic

k M

illa

rd


Inquiry following loss of Autosub2

“The Board has recognised the competence and commitment of the NOC AUV team; they have a high level of understanding of the importance of reliability and have employed sound reliability principles to influence their design decisions. However, they have not employed any formal systems reliability analysis methods. The Board believe this to be a major shortfall.”


Recommendations following loss of Autosub2

• NERC (or representatives) should define risk acceptance criteria

• AUV development team should implement formal risk and reliability management systems

• AUV development team should provide evidence of reliability achievement

Full report available:Strut, J. (editor), 2006. Report of the inquiry into the loss of Autosub2 under the Fimbulisen. NOCS Research and Consultancy Report: pdf at http://eprints.soton.ac.uk/41098/


Synopsis

Introduction to Targeted Reliability in a marine science context

Risk Management Process-AUV being used with Autosub

Informing the process

Detailed engineering and operational fault logs

Engineering and statistical follow-up

The use of Expert Judgement

Moorings

What might a Risk Management Process look like?

Example issues


Risk Management Process-AUV Start

e.g. P(loss)<20%


Campaign requirements, Dr Jenkins (BAS) forPine Island Glacier, Antarctica Feb. 2009

60 km open water missions to 200, 600, 1000 m depths, close to ice front.

3 x 60 km sub-ice-shelf missions to 600 m depth, in outer half of cavity.

3 x 120 km sub-ice-shelf missions to 1000 m depth, to the "minimum headroom" limit of the cavity

Sea ice may well be present in the area, beneath which Autosub3 would need to travel to reach the ice front.

Minimum

Desired


How might we predict probability of loss?

Part A - Gather fault history, document human error and all incidents with the AUVPart B - Set out the key features and risks of the operating environment

We postulate that combining Parts A and B cannot be done through scientific methods. Addressing Part A alone has been controversial.

*

* Estimated as cause of ~60% of US military UAV faults/incidents by Tvaryanas et al. (2005)


Part A: Example Autosub3 fault log entriesMission Distance (km) No. Faults Fault HIU? Comment 384 1.5 2 Y(1) 1. Mission aborted due to network failure. (Much)

later tests showed general problem with the harnesses (bad crimp joints). Harnesses repaired by manufacturer for next cruise. 2. Loop of recovery line came out from storage slot, New storage arrangement needed. GG note: the Terschelling 2006 trials showed no explicit evidence of harness-connector problems.

385 15.2 1 N 1. Autosub headed off in an uncontrolled way. Th is was due to a side effect of the removal of the upwards-looking ADCP. The SW vulnerability that caused this was corrected for later test cruise. This problem would be caught immediately after launch and before the vehicle would be committed to its mission.

386 26 1 N 1. GPS antenna failed. New design for future cruises. GG note: new GPS antenna design used, but different set of problems encountered on Terschelling 2006.


Autosub Engineering mitigation examples Replace wet-mateable connectors with

penetrators - prior experience of intermittent connection under pressure.

Autosub2 mission 313 Amundsen Sea


Part B. Eliciting Expert Judgement

1. Set out the Issues

2. Select the Experts

3. Clearly Define the Issues

4. Train the Experts

5. Elicit the Judgements

6. Analyze and Aggregate the Results

7. Complete Analysis and Write-up

Otway, H. and von Winterfeldt, D., 1992. Expert judgement in risk analysis and management: Process, context, and pitfalls. Risk Analysis, 12(1): 83–93.


EEJ example for the fault history of Autosub3 Ten AUV practitioners from Australia, Canada, USA from academia,

research, commercial, military backgrounds.

q Given the set of facts on all faults and incidents with Autosub3 throughout its life to date we seek to predict the probability of loss of the vehicle in four operating environments: > Open water > Coastal > Sea ice present > Under an ice sheet

1. In the course of evaluating each fault log entry, the expert is asked to assess the following question:

“What is the probability of loss of the vehicle in the given environment X given fault/incident Y?”


Your estimates for Autosub3 Mission 384

0.001 0.01 0.001 0.01 0.01 0.001 0.01

0.001 0.015 0.001 0.005 0.01 0.004 0.03

0.01 0.5 0.01 0.1 0.1 0.05 0.1

0.7 0.8 0.1 0.5 0.7 0.8 1.0

4, 2, 5, 4, 5, 3, 4 4, 2, 5, 4, 5, 4, 3 4, 3, 5, 3, 3, 2, 3 4, 3, 4, 3, 2, 1, 4

Range 0.001 - 0.01 0.001 - 0.03 0.01 - 0.5 0.1 - 1

Log Opinion Pool 0.0037 0.0051 0.0590 0.5523

Weighted Log Opinion Pool 0.0036 0.0043 0.0472 0.5056

Shelf Ice

Mission aborted (to surface)due to network failure.(Much) later tests showedgeneral problem with theharnesses (bad crimp joints).

Fault/incident description Open Coast Sea Ice

Weights 1-5


Experts’ estimates for Autosub3 Mission 384

0.001 0.01 0.001 0.01 0.01 0.001 0.01

0.001 0.015 0.001 0.005 0.01 0.004 0.03

0.01 0.5 0.01 0.1 0.1 0.05 0.1

0.7 0.8 0.1 0.5 0.7 0.8 1.0

4, 2, 5, 4, 5, 3, 4 4, 2, 5, 4, 5, 4, 3 4, 3, 5, 3, 3, 2, 3 4, 3, 4, 3, 2, 1, 4

Range 0.001 - 0.01 0.001 - 0.03 0.01 - 0.5 0.1 - 1

Log Opinion Pool 0.0037 0.0051 0.0590 0.5523

Weighted Log Opinion Pool 0.0036 0.0043 0.0472 0.5056

Shelf Ice

Mission aborted (to surface)due to network failure.(Much) later tests showedgeneral problem with theharnesses (bad crimp joints).

Fault/incident description Open Coast Sea Ice

Weights 1-5


Cumulative frequency statistics

Open Water Coastal Sea Ice Shelf Ice

Upper Q. .026 .037 .17 .40

Median .018 .020 .088 .17

Lower Q. .0085 .0083 .045 .072


Autosub Statistical (procedural) example

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Surviving

0 50 100 150 200 250 300

Distance(km) Kaplan-Meier method for estimating probability of survival with distance for all Autosub3 missions to date. Prevalence of faults leading to ‘infant mortality’ using GG’s judgement.

Operating procedure for under ice: Each mission has ‘open water’ 25km segment before committing under ice.

Only two missions beyond 250km, one of which failed, hence large step.


Estimated probability of loss

Minimum mission set, no sea ice in front of glacier P(loss) = 9%

Minimum mission set + 30km of sea ice in front of glacier P(loss) = 16%

Desired mission set with no sea ice in front of glacier

P(loss) = 24%

Desired mission set + 30km of sea ice in front of glacier

P(loss) = 30%

Based on Autosub3 history to end of March 2007. Will be updated after Terschelling June 2008 proving trials.


For no-fee consultations on risk and reliability issues for NERC marine science, contact either:Mario Brito ([email protected]) or Gwyn Griffiths ([email protected]).

http://www.noc.soton.ac.uk/OED/gxg/risk.html

Risk and Reliability: A new service to the NERC marine science community

Part of Oceans2025 Technology Work Package on Risk and Reliability

Mario Brito - (awaiting) PhD in software reliability

Access to engineering specialists in the Underwater Systems Lab.

Early discussions on PAP mooring-related problems have already taken place.


Example - Flooded Glider, April 2008Establish root cause of glider partially flooding with ~4 litres of water while on a tethered dockside post incident-free 3-month deployment.

Depth 0 to 7 m

Pressure reduction 9.5 to 6.0 in Hg

1 min


Systematic Fault Tree Analysis

Ammoniteflooded

Leak

Li batteryvented

Tapered ringin wrong order

Pressure tubesseparated

Chance events

Failure routes

Pressure tubesnot fully butted

Tie-rod crossthreaded

CTD

Stern tube

‘O’ rings

Pressure port

Vacuum plug

Establish actual cause or assign probabilities. Do NOT jump to conclusions!


The root cause: Assembly error

For no-fee consultations on risk and reliability issues, contact either:Mario Brito (mpb2o07 @noc.soton.ac.uk) or Gwyn Griffiths ([email protected]).

Courtesy Peter Stevenson


Glass buoyancy failure: WHOI VEX Mooring Array in ca. 5000m Western Atlantic.

Where can I find quantitative:

• Failure rates?

• Any difference in failure rates between brands e.g. Benthos, Vitrovex?

• The major causes of failure?

• What’s worse and why, time at depth or depth cycling?


Acoustic release failure: WHOI VEX Mooring Array in ca. 5000m Western Atlantic.

Batch of EG&G releases had improperly machined release mechanisms. Below 2000m, compression was such that the mechanism would never release.

Example of a common mode, human error.

Recovery using Isis ROV, April 2003.


Work programme for 2008 Analyze and write up Autosub3 Expert Judgement

Work with Autosub3 team on 2008 trials and risk management for 2009 Antarctic cruise

How do we incorporate quantitatively sea ice and vessel characteristics? Paper for IPY Conference, St. Petersburg July 2008.

Work with Autosub6000 team on Markov chain approach to stages of reliability and risk.

The reliability of deep ocean glass spheres. Factors affecting the reliability of the PAP moorings

Related to EuroSITES project and Oceans 2025 - propose to instrument a test mooring to establish in situ performance.

Discuss way forward for interaction with Rapid-Watch with NERC/Coordinator/Scientists


Conclusions

From open literature searches, AUV community lags the UAV community in analysis of incident and fault data. We should be more proactive, e.g. use of wikis, blogs, list servers …

Recording fault and incident data, and sharing outcomes is important for the community as a whole

Controversy still surrounds attempts to model statistically AUV faults, and more so, and the use of expert judgement to estimate probability of loss from fault history.

We need to do more to engage with ocean engineers working on moorings, landers etc.

Documents

Towards Targeted Reliability Oceans 2025 Theme 8 WP8.4