A Signal Detection Model to Interpret Safety Tests in ... · A Signal Detection Model to Interpret Safety Tests in Offshore Oil Drilling: A Case Study to Analyze Negative Pressure

A Signal Detection Model to Interpret Safety Tests in Offshore Oil Drilling: A Case Study to Analyze Negative Pressure Test (NPT)

Interpretation in Offshore Drilling

MaryamTabibzadeh,PhDa

DetlofvonWinterfeldt,PhDb

NajmedinMeshkati,PhDb

aDepartmentofManufacturingSystemsEngineering&ManagementCaliforniaStateUniversity,Northridge(CSUN)

bDanielJ.EpsteinDepartmentofIndustrialandSystemsEngineeringUniversityofSouthernCalifornia(USC)

PSAM14,September2018

Outline

!  VitalNeedtoOffshoreandDeep-waterDrilling!  OffshoreDrillingasaHigh-RiskIndustry!  WhyRiskAnalysisPractices?

!  AnalyzingHumanandOrganizationalFactors

!  ConcentratingonNegativePressureTest!  SignalDetectionModelParametricEquations

!  ACaseStudytoQuantifytheSignalDetectionModel

!  SensitivityAnalysis!  SummaryandConclusion

2

OffshoreDrilling:AVitalSourceofOilSupply

(EIA,2016)

3

WellsdrilledintheGulfofMexicobywaterdepthfrom1940to2010(ReporttothePresident,2011,page41)

WhyDeep-waterDrillingIsNoteworthy??

4

OffshoreDrilling&Production:High-RiskIndustry

Offshoredrillingisoneofthehigh-riskindustrieswith“tightlycoupled”and“interactivelycomplex”

operations.

PiperAlpha,NorthSea,1988

Fatalities:167Cost:$3.4Billion

Petrobras36,Brazil,2001

Fatalities:11Cost:$350Million

BPDWH,GOM,2010

Fatalities:11Cost:$40-$50Billion

5

CaseStudy:DeepwaterHorizon(DWH)Accident

!  April20,2010!  11peopledied,17injured!  5millionbarrelsofspilled

oil~682000tonsin87days!  Hugeenvironmental

damages,influencingsmalllocalbusinesses,andtourism

!  Billionsofdollarsofcost

6

7

DWHWasDuetoaSeriesofTechnicalFailures

!  Welldesign:•  Narrowdrillingmargin

•  Longstringinsteadofaliner

!  Cementing•  Cementmaterial

•  Numberofcentralizers

!  NegativePressureTest(NPT)misinterpretation!  BlowoutPreventer(BOP)failure

!  Mud-gasseparator!  Alarmsystems

8

HigherRiskofDeep-waterDrilling

!  Morecomplexcasingdesigns!  Higherpressure!  Moredifficultformations!  Higheruncertaintyofseismology!  Higherchallengesinaccessingthesiteandwellhead!  Loweravailabilityofexperiencedpersonnel

9

WhyRiskAnalysisPractices?

“Governmentagenciesthatregulateoffshoreactivityshouldreorienttheirregulatoryapproachesto

integratemoresophisticatedriskassessmentandriskmanagementpracticesintotheiroversightofenergy

developersoperatingoffshore.”

ReporttothePresident,NationalCommissionontheBPDWHOilSpill,2011,Page251

Trade-offbetweenhighriskofdeep-wateroffshoredrillingandtherisingdependenceofoilandgassupplytoit

10

WhyNPT?

NPT

Manageablescopeofwork

Theprimarywayofascertainingwell

integrityinoffshoredrilling

NPTmisinterpretation

wasamajorcontributingcauseoftheDWHaccident

NPT:NegativePressureTest11

SignificanceofNegativePressureTest

“Ifthenegativepressuretesthadbeencorrectlyinterpreted,theblowout,explosion,fire,andoilspillwouldhavebeenaverted.Consequently,theCourtfindsthatthemisinterpretationofthenegativepressuretestwasasubstantialcauseoftheblowout,explosion,fire,andoilspill.”

Findingsoffactandconclusionsoflawphaseonetrial,OilSpillbytheOilRig“DeepwaterHorizon”intheGOM,

TheUnitedStatesDistrictCourtfortheEasternDistrictofLouisiana,September2014,Page65

12

Dissecting“Standard”(Shou

ldbeDone)Negative

PressureTest

13

DWHcrewpracticeofthe

negativepressuretest

4pm

4:53pm4:58pm

5:10pm

5:26pm

5:53pm6:40pm

4pm

4:53pm

4:58pm

5:10pm

5:26pm

5:53pm

6:40pm

WhyHumanandOrganizationalFactors(HOFs)?

Long-termstudy(1988-2005)ofmorethan600welldocumentedmajorfailuresinoffshorestructures:approximately80%ofthemajorfailureswereduetoHOFs

LordCulleninthe25thanniversaryofPiperAlpha(2013):“asIdugdowntothebackgroundofwhathappened,Idiscovereditwasnotjustamatteroftechnicalorhumanfailure.Asisoftenthecase,suchfailuresareindicatorsofunderlying

weaknessesinmanagementofsafety.”

ThereisacriticalgapintheliteratureregardingtheexistenceofenoughriskassessmentapproachesanalyzingthecrucialroleofHOFs

ChiefCounsel’sreport(2011)ontheDWH:“whattheinvestigationmakesclear,aboveallelse,isthatmanagementfailures,notmechanicalfailings,werethe

ultimatesourceofthedisaster.”

15

ConceptualRiskAnalysisFrameworkforNPTMisinterpretation

OrganizationalFactors

Decisions/Actions

BasicEvents

16


ASnapshotoftheSignalDetectionModelforNPTInterpretation

!  APLeak&WellLeak:Yes/No

!  Targetvariable:Finitecontinuous

!  Decisions/judgments:OK/NOTOK

18

Possibleflowpathsforhydrocarbon(Sourceofimage:ChiefCounsel’sReport,2011,page39)

LeakintheBOPannularpreventer(Sourceofimage:ChiefCounsel’sReport,2011,page154)

TwoVariablesAffectingourTargetVariable(PressureDeviation)

19

DecisionProcessesinSignalDetectionTheory

PhysicalWorld

SensoryProcesses

Inference&DecisionProcesses

ResponseBehavior

Signaldetectiontheoryanddecisionprocesses(GreenandSwets,1974;DeplanckeandSparrow,2014)

20

SignalDetectionTheory

Hit

FalseAlarmMiss

CorrectRejection

Noise Signal

21

StatesoftheSystem

Classificationofstates:APLeak,WellLeak

!  Normalsate:h0:NN

!  Abnormalstates:h1:YNh2:NYh3:YY

!  P(h0)=P(NN)!  P(h1)=P(YN)!  P(h2)=P(NY)!  P(h3)=P(YY)

22

SignalDetectionModelNotations

Probabilityofeachstatefor(APLeak,WellLeak):

P(APLeak,WellLeak)=P(APLeak)*P(WellLeak)

e.g.P(NN)=P(APLeak=N)*P(WellLeak=N)

23

SignalDetectionModelforNPTInterpretation

Observeavalueforpressuredeviationfromthesystem;AP-EP=d

Whatisthejudgment?Say“H0”(OK)or“H1”(NOTOK)

Say“H0“orOKiff:

:LikelihoodRatio

:Prioroddforstate“hi”comparingto“h0”

:Conditionalprobabilityofthepressuredeviationknowingthatthestateis“hi”

Calculatethecut-offpointvalue“e”forthepressuredeviation:Judgmentis“H1”orNOTOKforanyobservedpressuremorethan“e”

Cij:Costofsaying“Hj”whenthestateis“hi”

SignalDetectionModelRequiredInputs

Thecut-offpointvaluedependsonthreemaininputs:

1)  P(hi):Priorprobabilityofthestate“hi”;i=0,1,2,3

2)  f(x|hi):Conditionalprobabilityofpressuredeviationforstate“hi”

3)  Cij:Costofsaying“Hj”whilethestateis“hi”;i=0,1,2,3andj=0,1

25

ResultsofSignalDetectionModelAnalysis

!  Cut-off point value for the above inequality knowing thevaluesforthemainthreeinputs:

e=247psi

!  Foranyobservedpressurebuilt-upmorethan247psi:say“H1”orNOTOK

Howapressuredeviationashighas“1400psi”was

acceptedbytheDWHcrew?

26

TheCut-offPointIllustrationandMeaning

0

0.005

0.01

0.015

0.02

0.025

0.03

f(x|h0)

f(x|h2)

f(x|h1orh3)

Pressuredeviation(psi)

27

Bias1:UnderestimatingPriorProbabilityofAbnormalStates

00.10.20.30.40.50.60.70.80.91

Cut-offpoint"e"Vs.P(APLeak=Y)

Cut-offpoint"e"

P(APLeak=Y)

0100200300400500600700800900100011001200130014001500160017001800190020002100

00.20.40.60.81

Cut-offpoint"e"Vs.P(WellLeak=Y)

Cut-offpoint"e"

P(WellLeak=Y)

Bias1:UnderestimatingPriorProbabilityofAbnormalStates-Cont’d

Cut-offpoint"e"Vs.P(APLeak=Y)

Cut-offpoint"e"

P(APLeak=Y)

0

500

1000

1500

2000

2500

Cut-offpoint"e"Vs.P(WellLeak=Y)

Cut-offpoint"e"

P(WellLeak=Y)

RootCausesofBiases

Organizationalfactorsaretherootcontributingcausesofbiases:

!  Economicpressure!  Personnelmanagementissues!  Issuesincommunicationandprocessingofuncertainties!  Lackofanintegrated,informedmanagement

30

!  There is aneed formore sophisticated riskanalysismethodologies to reducethehighriskofaccidentsandblowoutsinfutureoffshoredrilling.

!  The developed methodology in this study is an attempt of utilizingsophisticatedriskanalysispractices,andthismethodologycanbegeneralizedtootherapplicationsaswell.

!  Weproposedastructuredsignaldetectionmodelwithparametricequationsfor it in order to analyze critical decision making situations and involvedbiases.Thismodelcanbeusedindifferentsafety-criticalsystemssuchasoilandgasindustry,healthcare,transportationandfinancialsystems.

!  Biases, such as underestimating the prior probability of abnormal states,affect rational decision making and increase the risk of a false negativesituationormisinterpretinganegativepressuretest.

!  MisinterpretationofaconductedNPTcanmostlyoccurduetotheconfluenceofdifferentbiasesratherthanjustonespecificbias.

!  Organizationalfactorsaretherootcausesofinvolveddecisionmakingbiases.

SummaryandConclusion

31

Acknowledgements

!  Petroleumengineeringexperts:

" Mr.StanChristman,retiredExxonMobilexecutiveengineeringadvisor

" Mr.FredDupriest,retiredExxonMobilchiefdrillingengineerandlecturerattheTexasA&MUniversity

" Mr.RogerGatte,BPretiredwellssuperintendent" AretiredExxonMobilWorldwidedrillingmanager

32

Thankyou!

MaryamTabibzadeh,PhD

[email protected]

DepartmentofManufacturingSystemsEngineering&ManagementCaliforniaStateUniversity,Northridge(CSUN)

References•  Aven,T.,Sklet,S.,andVinnem, J.E. (2006)“BarrierandOperationalRiskAnalysisofhydrocarbon

releases (BORA-Release); Part I: Method description”. Journal of Hazardous Materials, 137 (2):681-691

•  Bea, R. (2002) “Human and Organizational Factors in Design and Operation of DeepwaterStructures”.ProceedingsoftheOffshoreTechnologyConference,Houston,Texas,May6-9

•  Bea, R. (2006) “Reliability and Human Factors in Geotechnical Engineering”. Journal ofGeotechnicalandGeoenvironmentalEngineering,132(5):631-643

•  Bea,R.(2011)PersonalCommunication.UniversityofCaliforniaBerkeley,memberoftheNationalAcademyofEngineering,December6

•  Beckwith, R. (2012) “The Post-MacondoWorld: Two Years After the Spill”. Journal of PetroleumTechnology,64(5):36-46

•  BOEMRE Report (2011) “Report Regarding the Causes of the April 20, 2010 Macondo WellBlowout”. The Bureau of Ocean Energy Management, Regulation, and Enforcement (BOEMRE).September14

•  BPreport(2010)“DeepwaterHorizonAccidentInvestigationReport”.BritishPetroleum

•  Chief Counsel’s Report (2011) “Macondo, the Gulf oil disaster”. National Commission on the BPDeepwaterHorizonOilSpillandOffshoreDrilling

•  Christman, S. A. (2013) Personal Communication, Retired ExxonMobil executive engineeringadvisor,December13

References•  Deplancke, A. and Sparrow, L. (2014) “Signal Detection Theory”, Presentation file,

http://ureca.recherche.univ-lille3.fr/sparrow/TLDocs/cours1314/PPNSA_UE7_SDT_2012.pdf,Accessdate:February24,2014

•  Dupriest,F. (2014a)PersonalCommunication,RetiredExxonMobilchiefdrillingengineer,LectureratTexasA&MUniversity,January18

•  Garcia, J. A. (2013) Personal Communication, Retired ExxonMobil Worldwide Drilling Manager,November25

•  Gentile,G.(2013)Regulations&Environment:ADeep-rootedCauseofMacondoisRaised,OilgramNew Column, November 4, http://blogs.platts.com/2013/11/04/macondo-why/, Access date:January28,2013

•  Green,D.M.andSwets,J.A.,“SignalDetectionTheoryandPsychophysics”,JohnWiley&Sons,Inc.,1974

•  EIA (2016) “Offshore production nearly 30% of global crude oil output in 2015“, U.S. EnergyInformation Administration (EIA), October 25, Retrieved fromhttps://www.eia.gov/todayinenergy/detail.php?id=28492,Accessdate:March25,2018

•  NAE/NRC Report (2011) “Macondo Well Deepwater Horizon Blowout: Lessons for ImprovingOffshoreDrillingSafety”.NationalAcademyofEngineeringandNationalResearchCouncil (NAE/NRC).TheNationalAcademiesPress,Washington,D.C.

•  Pate-Cornell,E. (1993)“Learning formthePiperAlphaAccident:Aplatformanalysisof technicalandorganizationalfactors”.RiskAnalysis.13(2):215-232

References•  Perrow,C.(1984)“NormalAccidents:LivingwithHigh-RiskTechnologies”.BasicBooks,NewYork•  Report to the President (2011) “Deep Water; the Gulf Oil Disaster and the Future of Offshore

Drilling”,NationalCommissionontheBPDeepwaterOilSpillandOffshoreDrilling•  Skogdalen, J.E.andVinnem, J.E. (2012) “QuantitativeRiskAnalysisofOilandGasDrilling,Using

DeepwaterHorizonasaCaseStudy”.ReliabilityEngineeringandSystemSafety.100:58-66

•  SINTEFexecutivesummary(2011)“TheDeepwaterHorizonAccident:Causes,LearningPointsandRecommendationsfortheNorwegianContinentalShelf”

•  Tabibzadeh,M.(2014)“ARiskAnalysisMethodologytoAddressHumanandOrganizationalFactorsin Offshore Drilling Safety:With an Emphasis on Negative Pressure Test”, Doctor of PhilosophyDissertation,UniversityofSouthernCalifornia,LosAngeles,California,May2014

•  Tabibzadeh,M.andMeshkati,N.(2014)“LearningfromtheBPDeepwaterHorizonAccident:RiskAnalysis of Human andOrganizational Factors in Negative Pressure Test”, Environment SystemsandDecisions,34(2):194-207,http://dx.doi.org/10.1007/s10669-014-9497-2

ProjectionofDeep&UltraDeep-waterDrilling

Librafield;Brazil:

!  OffRiodeJaneirocoast!  7000meters~23000ftdepth!  8-12billionbarrelsofoil

Economist,October26,2013

SignificanceofNegativePressureTestSlide38

TheHonorableDr.DonaldWinterinhisinterviewwithPlatts:BPDeepwaterHorizon

wasprecipitated“notbyapieceofhardware,butbythedecisiontoproceedtotemporaryabandonmentinspiteofthefactthatthenegativepressuretesthadnotbeenpassed”(November4,2013).


16

Observationsfromthe3-LayerConceptualModel

!  Organizationalfactorsarerootcausesofaccumulatederrorsandquestionabledecisions/actionsmadebypersonnelandmanagement.

!  Thefirstthreeorganizationalfactorswiththehighestinfluence:

1.  PersonnelManagementissues2.  Issuesincommunicationandprocessingofuncertainties3.  Economicpressure

ConditionalProbabilitiesandWellCharacteristics

!  Specificationsandrangeofeachconditionalprobabilitydistributionforthetargetvariableineachstatedependsontheanalyzedwellcharacteristics,suchasdepthofdrilling,depthofdisplacement,formationcharacteristics,andtypeandamountofusedfluids(e.g.oilbasedmudvs.waterbasedmud,spacer)

!  ConsideredvaluesforeachconditionalprobabilityarebasedoncharacteristicsofawellliketheMacondo.

Forexample:Whenthereisleakingintheannularpreventer(state“h1”),foracaseliketheDWH,basedonthe421bblsofusedspacer,intheworstcase,thebottomofthespacercanbeat8367ftandthetopatabout3000ft.

ParametricDecisionMakingEquations-1

Say“H0“orOKiff:

1.1

2.1

ExpectedvalueforsayingorjudgingHiafterobservingthevalue“d”fromthesystemforourtargetvariable

ParametricDecisionMakingEquations-2

Bysubstitutingtheequality(2.1)in(1.1):

Bysimplifyinginequality(1.2):

Fortheexpectedcost:1.3

1.2

1.4

Post-MortemAnalysisoftheDWHNPT

UnderwhatcircumstancescouldtheDWHcrewacceptthenegativepressuretestresultswithapressurebuilt-upof“1400psi”?!  Basicscenario:

•  P(APLeak=Y)=0.01•  P(WellLeak=Y)=0.02•  C20/C01andC30/C01=2000

Cut-offpoint=247psi

!  Scenario1:•  P(APLeak=Y)=0.01•  P(WellLeak=Y)=0.00001•  C20/C01andC30/C01=300

Cut-offpoint=837psi(Whichisstilllessthan1400psi)

!  Iftheabovecostratiosreduceto250:thecut-offpointwillbeinfinity,whichmeansacceptingthetestforanyobservedpressurebuilt-up;nomatterhowhighitis.

RootCausesofBiases

Organizationalfactorsaretherootcontributingcausesofthestatedbiases:

!  Economicpressure;ifthereistoomuchpressureoncostandtimesaving,that can cause underestimation of the described cost ratio (cost ofacceptingthetestforanabnormalstatetothecostofrejectingthetestforanormalstate).

!  Personnel management issues; if personnel does not receive propertrainingordoesnothaveenoughexperience,thatcancausethedescribedbiases.

!  Issuesincommunicationandprocessingofuncertainties;ifmanagersdo not communicate the risk of complex operations such as NPTprocedurestopersonnel,thatcancontributetothedescribedbiases.

!  Lackof integrated, Informedmanagement;existenceofno integratedfeedbacksystemfrommanagers (bothonshoreandoffshore) to thecrewcancontributetothedescribedbiases.

Documents

A Signal Detection Model to Interpret Safety Tests in ... · A Signal Detection Model to Interpret Safety Tests in Offshore Oil Drilling: A Case Study to Analyze Negative Pressure