@nicolefv

HOWMETRICSSHAPEYOURCULTURE

NICOLEFORSGREN,PHD

@NICOLEFV

DIRECTOR,ORGANIZATIONALPERFORMANCEANDANALYTICS,CHEF

CO-FOUNDER,DEVOPSRESEARCHANDASSESSMENT

@nicolefv

WEDON’TALWAYSTHINKOFITTHISWAY,BUT…

measurementis culture

@nicolefv

@nicolefv

"AsgoodasAmazonwas,itdidnotyethaveaculturethatfullyembracedmeasurementanddebate. Ithinkbuildingthiscultureisthekeytoinnovation. Creativitymustflowfromeverywhere.WhetheryouareasummerinternortheCTO,anygoodideamustbeabletoseekanobjectivetest,preferablyatestthatexposestheideatorealcustomers.Everyonemustbeabletoexperiment,learn,anditerate.Position,obedience,andtraditionshouldholdnopower.Forinnovationtoflourish,measurementmustrule.”- GregLindenhttp://glinden.blogspot.com/2006/04/early-amazon-shopping-cart.html

@nicolefv

YOUCAN’TIMPROVEWHATYOUDON’TMEASURE

@nicolefv

@nicolefv

Evaluatingwell-designedandexecutedexperimentsthatweredesignedtoimproveakeymetric,onlyabout1/3weresuccessfulatimprovingthekeymetric!

Online Experimentation atMicrosoft, Kohavi etal.http://stanford.io/130uW6X

@nicolefv

ALWAYSMEASURETHINGSTHATMATTER

@nicolefv@nicolefv

THATWHICHISMEASUREDGETSMANAGED

Inbusiness, youarewhatyoumeasure

@nicolefv

IFYOUCAPTUREONLYONEMETRIC,YOUKNOWWHATWILLBEGAMED

@nicolefv

@nicolefv@nicolefv

Sometimes, wejustmeasure ALLTHETHINGS.ButSTOPandaskyourself:

• Whatdoesthistellme?

• HowcanIusethistomakeadecision?

• Howdoesthisrelatetoanoutcome?

• Does itstillmakesense tomeasureandtrackthismetric?

• Whataretheunintended consequences ofimplementing this?

NOTEVERYTHINGTHATCANBECOUNTED…

COUNTS

@nicolefv

METRICSINFORMOURINCENTIVES…ANDSHAPEBEHAVIOR

@nicolefv

METRICSINFORMOURINCENTIVES…ANDSHAPEBEHAVIOR

Metric=delivercodeonschedule

Behavior=sprinttodelivercodeat5pmonFriday

Unintended consequence =opsteamspends theweekendputtingoutfires

@nicolefv

@nicolefv

JUSTBECAUSESOMETHINGISHARDTOMEASUREDOESN’TMEANITISN’TWORTHMEASURING

@nicolefv

WESTRUM ORGANIZATIONALCULTUREPATHOLOGICALPower-oriented

BUREACRATICRule-oriented

GENERATIVEPerformance-oriented

Lowcooperation Modest cooperation Highcooperation

Messengers shot Messengers neglected Messengers trained

Responsibilities shirked Narrowresponsibilities Risks areshared

Bridging discouraged Bridging tolerated Bridging encouraged

Failure leads toscapegoating Failure leads tojustice Failure leads toinquiry

Novelty crushed Novelty leads toproblems Novelty implemented

Westrum,R.(2004).Atypologyoforganisational cultures. Qualityandsafetyinhealthcare, 13(suppl 2),ii22-ii27.

@nicolefv

HOWTOMEASUREWESTRUMORGANIZATIONALCULTUREINASTATISTICALLY VALID&RELIABLEWAYRatehowstronglyyouagree(7)ordisagree (1)tothefollowingstatements:

• Onmyteam, information isactivelysought.

• Onmyteam,failuresarelearningopportunities, andmessengers ofthemarenotpunished.

• Onmyteam,responsibilities areshared.

• Onmyteam,cross-functional collaboration isencouragedandrewarded.

• Onmyteam,failurecausesenquiry.

• Onmyteam,newideasarewelcomed.

Forsgren,N.,J.Humble(2016).“TheRoleofContinuousDeliveryinITandOrganizationalPerformance.”IntheProceedingsoftheWesternDecisionSciencesInstitute(WDSI)2016,LasVegas, NV.

@nicolefv

WHATCANIDO?

• ReadandshareDataDriven,by@dpatil and@hmason• Askothersaboutmetrics:

• Isithappening?Where?

• Isitimportant?

• Thinkaboutyourownmetrics:• Startwithoutcomes:what’simportant?

• Thenthinkaboutwhatinfluences youroutcomes

• Defineandcommunicate

@nicolefv

METRICSHELPUSCOMMUNICATE

KEY:• Givethemmeaningfulnames• Definethemwell• Communicatethemoftenandacrossboundaries

@nicolefv

WE’VETALKEDABOUT:

MEASUREMENT ISCULTURE

• Youcan’timprovewhatyoudon’tmeasure

• Alwaysmeasurethingsthatmatter

• Thatwhichismeasured getsmanaged

• Ifyoucaptureonlyonemetric,youknowexactlywhatwillbegamed

• Metrics informourincentives… andshapebehavior

• Noteverything thatcanbecounted...Counts

• Justbecause something ishardtomeasure doesn’tmean itisn’tworthmeasuring

@nicolefv

IFMEASUREMENTISCULTURE&YOUAREN’TMEASURINGYOURWORK...

howisitaffectingyourculture?

@nicolefv

THANKYOUNICOLEFV.COM