FosterProvost–11/17/17

1

So You’ve Built a Machine Learning Model…

Now What?

Foster Provost

Thanks to Josh Attenburgh, Henry Chen, Brian Dalessandro, Sam Fraiberger, Thore Graepel,

Panos Ipeirotis, Michal Kosinski, David Martens, Claudia Perlich, David Stillwell

TheDataScienceProcessisausefulframeworkforthinkingthroughlotsofmodeling&managerialdecisionsaboutsolvingproblemswithAI/MachineLearning/DataScience

Formore,seeDataScienceforBusinessProvost&FawceF.O’ReillyMedia2013

FosterProvost–11/17/17

2

Justafewissues:•  Misalignmentofproblem

formulaQon•  Leakageinfeatures•  Samplingbias•  Learningbias(MLfavors

largersubpopulaQons)•  Labelingbias•  EvaluaQonbias

TheDataScienceProcessisausefulframeworkforthinkingthroughlotsofmodeling&managerialdecisionsaboutsolvingproblemswithAI/MachineLearning/DataScience

InReality…

FosterProvost–11/17/17

3

InthistalkI’llfocusontwocommonproblemsfacedwhendeployingmachinelearnedmodels

•  Lackoftransparencyintowhymodel-drivensystemsmakethedecisionsthattheydo–  importantforawholebunchofreasons

•  useracceptance,managerialacceptance,debugging/improving

–  ofcurrentinterest:areyourdecisionsfair?•  “UnknownUnknowns”

–  doyouknowwhatyourmodelismissing?Especiallywhatit’smissingand“thinks”it’sge[ngright?

6GabrielleGiffordsShooQng,Tucson,AZ,Jan2011

FosterProvost–11/17/17

4

7

WhywasMarikoshownthisPoFeryBarnad?

FosterProvost–11/17/17

5

Whywasthisdecisionmade?

evidence ? decision

data-drivenmodel

Customer Manager

DataScienceTeam

Explana5onsforwhom?

FosterProvost–11/17/17

6

!"#$%&'()"*$'$+,

TheComplexWorldofModels

(Martens&FP,“ExplainingData-drivenDocumentClassificaQon.”MISQ2014)

AnoQonofexplanaQonTheEvidenceCounterfactual

•  Modelscanbeviewedasevidence-combiningsystems•  Weareconsideringcaseswhereindividualpiecesofevidenceareinterpretable

•  Thus,foranyspecificdecision*fromanymodelwecanask:

Whatisaminimalsetofevidencesuchthatifitwerenotpresent,

thedecision*wouldnothavebeenmade?*The“decision”canbeathresholdcrossingforaprob.esQmaQon,scoringorregressionmodel

see(Martens&FPMISQ2014);(Chen,Moakler,Fraiberger,FP,BigData2017)(Moeyersomsetal.;Chen,etal.;ICML’16WkshponHumanInterpretabilityInML)

(cf.Hume1748)

FosterProvost–11/17/17

7

WhywasMarikoshownthisPoFeryBarnad?

WhywasMarikoshownthisPoFeryBarnad?

Becauseshevisited:

•  www.diningroomtableshowroom.com•  www.mazeltovfurniture.com•  www.realtor.com•  www.recipezaar.com•  www.americanidol.com

FosterProvost–11/17/17

8

Let’sfocusonthedevelopersExplanaQonsaidthedatascienceprocess

•  HelptounderstandfalseposiQves–omenrevealingproblemswiththetrainingdata

•  Canrevealproblemswiththemodel

FosterProvost–11/17/17

9

Withtheincreasinguseofpredic=vemodelsfrommassivefine-grainedbehaviordata…

Consumersareincreasinglyconcernedaboutthe

inferencesdrawnaboutthem.

Kosinski,M.,SQllwell,D.,&Graepel,T.(2013).ProceedingsoftheNaQonalAcademyofSciences,110(15),5802-5805.

FosterProvost–11/17/17

10

EffectofremovingselectedFacebookLikesfromconsideraQonbythepredicQvemodel

Twoguyspredictedtobegay:

Model:logisQcregressiononthetop100latentdimensionsfromanSVDoftheuser/Likematrix.

(Chen,Moakler,Fraiberger,…BigData2017)(Chen,etal.,ICMLWkshpInterpretability2016)

FosterProvost–11/17/17

11

Whywasthisguypredictedtobesmart?

Opportunityforofferinguserscontrolviaa“cloakingdevice”?

EffectofremovingselectedLikesfromconsideraQonbythepredicQvemodel

FalsePosiQves

(Chen,Moakler,Fraiberger,…BigData2017)(Chen,etal.,ICMLWkshpInterpretability2016)

FosterProvost–11/17/17

12

Butthere’satwist…

Afirmcouldpurporttogiveuserstransparencyandcontrol……butactuallymakeitcumbersomeforuserstoaffecttheinferencesdrawnaboutthem:

(Chen,Moakler,Fraiberger,…BigData2017)(Chen,etal.,ICMLWkshpInterpretability2016)

FosterProvost–11/17/17

13

So.ExplanaQonsofindividualdecisionscanhelpwithmanyissuesintheprocessofbuildingandusingmachinelearnedmodels.Butweneedmorehelpwithoneveryimportantproblem…

TheproblemofUnknownUnknowns•  Whatisyourmodelmissing?Whatisitmissinganditreallythinksthatit’scorrect?

•  Whywoulditbemissingthings?

FosterProvost–11/17/17

14

Weneedtothinkcarefullyaboutthedata-generaQngprocess(es)andthedatapreparaQonprocesses–especiallytheprocessofge[nglabeledtraining&tesQngdata.

TheproblemofUnknownUnknowns•  Whatisyourmodelmissing?Whatisitmissinganditreallythinksthatit’scorrect?

•  Whywoulditbemissingthings?– Samplingbias– Learningbias(MLfavorslargersubpopulaQons)– Labelingbias– EspeciallysevereforNon-self-revealingproblems

(AFenberg,IpeiroQs&ProvostJDIQ2015)

FosterProvost–11/17/17

15

HarnessHumanstoImproveMachineLearning

•  Withnormallabeling,humansarepassivelylabelingthedatathatwegivethem

31

Instead ask humans to search and find positive instances of a rare class

Searchinginsteadoflabelinghasintriguingperformance

(AFenberg&FPKDD2010)

FosterProvost–11/17/17

16

Active learning missing disjunctive subconcepts

33

(AFenberg&FPKDD2010)

NIPS 2016

FosterProvost–11/17/17

17

35

BeFer,but…..•  Classifierseemsgreat:Cross-validaQontestsshowexcellent

performance

•  Alas,classifierfailson“unknownunknowns”

“Unknown unknowns” à classifier fails with high confidence

(AFenberg,IpeiroQs&ProvostJDIQ2015)

FosterProvost–11/17/17

18

37

BeattheMachine!

Askhumanstofindexamplesthat•  theclassifierwillclassifyincorrectly•  anotherhumanwillclassifycorrectly

Example: Find hate speech pages that the machine

will classify as benign

(AFenberg,IpeiroQs&ProvostJDIQ2015)

38

BeattheMachine!

Example: Find hate speech pages that the machine

will classify as benign

IncenQvestructure:•  $1ifyou“beatthemachine”

•  $0.001ifthemachinealreadyknows (AFenberg,IpeiroQs&ProvostJDIQ2015)

FosterProvost–11/17/17

19

AAAI 2017

(AFenberg,IpeiroQs&ProvostJDIQ2015)

AAAI 2017

FosterProvost–11/17/17

20

Summary

•  WecanprovidetransparencyintothereasonswhyAIsystemsmakethedecisionsthattheydo

•  Wecancreatemechanismstohelpfindthe“UnknownUnknowns”

•  Asaresearcharea,there’ssQllalottodo

Somereading

Martens&FP,“ExplainingData-drivenDocumentClassificaQon.”MISQ2014

Moeyersomsetal.2016,ICML’16WkshponHumanInterpretabilityInML

Chen,etal.2016,ICML’16WkshponHumanInterpretabilityInMLChen,Fraiberger,Moakler,Provost.BigData5(3)2017

AFenberg,J.&Provost,F.Whylabelwhenyoucansearch?AlternaQvestoacQvelearningforapplyinghumanresourcestobuildclassificaQonmodelsunderextremeclassimbalance.InKDD2010.AFenberg,J.,IpeiroQs,P.&Provost,F.BeattheMachine:ChallengingHumanstoFindaPredicQveModel's“UnknownUnknowns”.JournalofDataandInformaQonQuality(JDIQ),6(1)2015.