Using Predictive Modeling to Identify Patients Who Need ...systemsforaction.org/sites/default/files/documents/... · Using predictive modeling to identify patients who need social

Funded by the Robert Wood Johnson Foundation

Strategies to Achieve Alignment, Collaboration, and Synergy across Delivery and Financing Systems

Using Predictive Modeling to Identify Patients Who Need Social Services

Research In Progress WebinarWednesday, January 10, 2017

12:00-1:00 pm ET/ 9:00 am-10:00 pm PT

AgendaWelcome: CBMamaril,PhD

ResearchFaculty,RWJFSystemsforAction NationalCoordinatingCenter,UniversityofKentuckyCollegeofPublicHealth

Presenter: JoshuaR.Vest,PhD,MPHDirector,CenterforHealthPolicyAssociateProfessor,HealthPolicy&ManagementIndianaUniversityRichardM.FairbanksSchoolofPublicHealthatIUPUIjoshvest@iu.edu

CommentarySpeaker:MichaelShafer,PhDDirectorCenterforAppliedBehavioralHealthPolicyProfessorArizonaStateUniversitymichael.shafer@asu.edu

QuestionsandDiscussion:ModeratedbyDr.Mamaril

2

Presenter

JoshuaR.Vest,PhD,MPHDirectorCenterforHealthPolicyAssociateProfessorHealthPolicy&ManagementIndianaUniversityRichardM.FairbanksSchoolofPublicHealthatIUPUIjoshvest@iu.edu

3

Commentary Speaker

4

MichaelShafer,PhDDirectorCenterforAppliedBehavioralHealthPolicyProfessorArizonaStateUniversitymichael.shafer@asu.edu

Webinars

5

Archives http://systemsforaction.org/research-progress-webinars

UpcomingWednesday, January 24, 12-1pm ET/ 9-10am PTOPTIMIZING GOVERNMENTAL HEALTH AND SOCIAL SPENDING INTERACTIONSJohns Hopkins University Bloomberg School of Public HealthPrincipal Investigators: Beth Resnick, DrPH, MPH, and David Bishai, MD, MPH, PhD

Wednesday, February 7, 12-1pm ET/ 9-10am PTSTRENGTHENING THE CARRYING CAPACITY OF LOCAL HEALTH AND SOCIAL SERVICE NETWORKSTrailhead Institute in ColoradoPrincipal Investigators: Danielle Varda, PhD, and Katie Edwards, MPA

Wednesday, February 21, 12-1pm ET/ 9-10am PTLINKING MEDICAL HOMES TO SOCIAL SERVICE SYSTEMS FOR MEDICAID POPULATIONSNational Committee for Quality AssurancePrincipal Investigators: Sarah Scholle, DrPH, and Keri Christensen, MS

Thank you for participating in today’s webinar!

For more information about the webinars, contact:[email protected]

111 Washington Avenue #201, Lexington, KY 40536859.218.2289

www.systemsforaction.org

6

Speaker Bios

7

Joshua R.Vest,PhD,MPH,isahealthservicesresearcherwithinterestsinorganizationaldeterminantsandeffectivenessofhealthinformationtechnologyandsystems,specificallytheadoption,utilization,andpolicyissuesoftechnologiesthatfacilitatethesharingofpatientinformationbetweendifferentorganizations.Heiswidelypublishedandhisworkhasemployedavarietyofresearchtechniquesfromlargescaledatabaseanalyses,togeographicalinformationsystemmapping,tosurveyresearch,toqualitativefocusgroupsandinterviews.Asaformerlocalpublichealthpractitioner,Dr.Vesthasaparticularinterestineffectivepublichealthinformationsystemsincludingtheroleofinformationtechnologygovernancestructuresonlocalpublichealthdepartments'adoptionofinformationtechnologyandsystems,thestructureofstateandlocalpublichealthinformationsystems,aswellasanevaluationofemailinterventiontoimprovediseasenotificationefforts.

MichaelS.Shafer,Ph.D.,isaprofessorintheSchoolofSocialWorkatArizonaStateUniversity’sCollegeofPublicServiceandCommunitySolutionswherehealsoholdsaffiliateappointmentsintheCenterforHealthInformationResearchandtheSchoolofCriminologyandCriminalJustice. Dr.ShaferisthefoundingdirectoroftheCenterforAppliedBehavioralHealthPolicywhichhas,forthepast25years,conductedcuttingedgeresearchontheadoptionandimplementationofinnovativepracticesinbehavioralhealthcare. Dr.Shaferhasauthoredmorethan40peer-reviewedresearcharticlesandgeneratedmorethan$45millioningrantsandcontractsthattargetcapacitybuildingandinnovationinbehavioralhealthservices. Dr.ShaferearnedhisPh.D.inEducationin1988fromVirginiaCommonwealthUniversity. Hehasreceivednumerousawardsandcitations,includingrecognitionfromtheU.S.DepartmentofJusticeforthedevelopmentofcrisisinterventiontrainingforlawenforcementpersonnel. Dr.Shaferisafrequentcontributortoprofessionalliteratureandheconsultswithbehavioralhealthagenciesthroughoutthecountry.

Usingpredictivemodelingtoidentifypatientswhoneedsocialservices

JoshuaRVest,PhD,MPHDirector,CenterforHealthPolicy

AssociateProfessor,HealthPolicy&ManagementIndianaUniversityRichardMFairbanksSchoolofPublicHealthatIUPUI

AffiliatedScientist,RegenstriefInstitute,Inc.

ThisworkwassupportedbytheRobertWoodJohnsonFoundationthroughtheSystemsforActionNationalCoordinatingCenter,ID73485.

Acknowledgements

IndianaUniversity• Suranga N.Kasthurirathne• Nir Menachemi• PaulKHalverson• ShaunJ.Grannis• JenniferWilliams• KarenComer

EskenaziHealth• DawnHaut• JenniferFerrell

• DonnaBurke• AlishaJessup

Kasthurirathne etal2017doi:10.1093/jamia/ocx130

Predictivemodelinginhealthcare:statisticalapproachestoidentifyingpatientsathighrisk(morelikely)fornegativeoutcomes

alliconsfromflaticon.com

Diagnosis&Utilization

Predictivemodelingiswidelyapplied...

Limitationsofcurrentpredictivemodeling

• LimitedtoEHRorclaimsdata• Socialdeterminantsoftenabsent• Oftensingle-sitedata

• Focuson“toolate”outcomes(reactivenotproactive)

• Don’tprovideinsightsintowhatservicespatientsshouldget

Objective1:Evaluatepredictivemodelsthatusecombinationsofclinical,socioeconomic,andpublichealthdata

Diagnosis&UtilizationEnvironmental&socialcontext

Neighborhoodhealthcontext

HealthBehaviors&System-widehealthdata




Dataonhealthstatus




Dataondriversofhealth




Acomprehensivepatientview

s

HEALTH

Economic&socialopportunities&resources

Living&workingconditionsinhomes&communities

Medicalcare

Personalbehavior

Frameworkfororganizingthefactorsincludedinriskidentificationtool

“Social Determinants of Health Model” by Braveman et al (2011) Annu. Rev. Public Health, 32:381-398

Theonlydataincludedinmostpredictionmodels

Whatweareadding

Fitday.com

Objective2:Contributionofthesedataonthenoveloutcomeofreferralstosocialservices

http://cchci.org/_services/behavioral-health/

www.chelseajew

ish.org/celebrating-social-w

ork-month-senior-care/

Pplay.google.com

guide.berkeley.edu/undergraduate

dmkelite.com

usf.edu

Fitday.com

Toberesponsivetonewpaymentstrategies,healthcareorganizationsintheUSarebeginningtoofferthesenon-medicalservices.

http://cchci.org/_services/behavioral-health/

www.chelseajew

ish.org/celebrating-social-w

ork-month-senior-care/

Pplay.google.com

guide.berkeley.edu/undergraduatedmkelite.com

usf.edu

Objective1:Evaluatepredictivemodelsthatusecombinationsofclinical,socioeconomic,andpublichealthdata.

Objective2:Contributionofthesedataonthenoveloutcomeofreferralstosocialservices.

Approach

Comparetheperformanceofriskpredictionmodelswith:

1)clinicaldataonly

2)clinicaldatawith community-levelsocioeconomic&publichealthindicators

Setting&sample

• EskenaziHealthoutpatientclinics– Indianapolis

safety-netprovider(formedicalindigent)

– urbanpopulation– allsocialservicesoffered

onaco-locatedbasis(noreferralstootherorganizations)

• 84,317adultpatients– atleast1outpatientvisit

between2011-2016

Demographics

Age (mean, sd) 43.9(15.6)

Male gender 35.1Race / ethnicityWhite, non-Hispanic 25.2African American, non-Hispanic 37.2Hispanic 19.5DiagnosesHypertension 38.7Asthma 7.9Cancer 7.6COPD 9.5Depression 19.0Diabetes 20.3Substance abuse 15.1Tobacco use 21.3

Sampledemographics

Data&measures(outcome)

Referraltosocialservices– Socialwork– Dietitian– Mentalhealth– Allotherservices(duetolowfrequency)

Datasources– EskenaziEHRbillingandencounterdata– schedulingsystemdata(includingkept,missed,&cancelledappointments)– unstructuredEHRordersandnotes

s

HEALTH



Medicalcare

Personalbehavior

Data&measures(predictors)



Medicalcare

Personalbehavior



• Diagnoses• Asthma• Coronaryarterydisease• Chronickidneydisease• Congestiveheartfailure• COPD• Stroke/cerebrovascularaccident• Depression• Diabetes• Hypertension• Ischemicvasculardisease• Obesity• Pregnancy• ….

• EDvisits(number)• Inpatientadmissions• PCPvisits• Mentalillness

• Smoking• Substanceabuse

EHR

• IndianaNetworkforPatientCare

• US’oldestHIE• StartedatRegenstriefInstitutein1995

• Oneofthenation’slargest• >80hospitals’medicalrecords• 17.2millionindividualpatients• 4.6billionclinicalobservations• 165milliontextreports• Over68%ofIndianapopulationcapturedin2014

• Datainclude:• admissionanddischarge• labreports• Microbiology• Pathology• Radiology• Cardiology• EKGdata

s



Medicalcare

Personalbehavior



• Employmentrates• Taxdelinquentproperties• Crimeindices• Educationrates• Voterparticipation• Income

• Smokingprevalence• Perceivedsafety• Mortalityrates• Infantmortalityrates• Maternalsmoking• Overweight/obesity

prevalence• Walkability

Fromsurveyorcensusdataandlinkedbygeolocation.

Frameworkfororganizingthefactors

kff.org/disparities-policy/issue-brief/beyond-health-care-the-role-of-social-determinants-in-promoting-health-and-health-equity/

Analyticapproach:performanceofpredictionmodelswithnoveldata

1) Clinicaldataonly(41variables)2) Clinicalplussocioeconomic&

publichealth(48variables)Splitsamplesfortraining&testing

Randomforestclassificationalgorithm

Modeleachoutcometwice.

Prevalenceofsocialservicereferralneed

Type of service %

Any service 53.0

Mental health 18.5

Social work 8.7

Dietitian 32.6

Other services 20.0

Predictionforsocialservicesreferralswasinthe“useful”range.

Clinicaldata Clinical +socioeconomic&publichealth

Anyreferral 0.745 0.741Mentalhealth 0.785 0.778Socialwork 0.731 0.714Dietitian 0.743 0.730Otherreferral 0.711 0.708

AreaundertheROCcurvevaluesforeachdecisionmodel

Consistentwithperformanceofmodelson:• Mortality• Readmissions• Diseasedevelopment• Carecoordinationneed

Socioeconomic&publichealthdatadidnotcontributesignificantly.

Clinicaldata Clinical +socioeconomic&publichealth

Anyreferral 0.745 0.741Mentalhealth 0.785 0.778Socialwork 0.731 0.714Dietitian 0.743 0.730Otherreferral 0.711 0.708

AreaundertheROCcurvevaluesforeachdecisionmodel

Socioeconomic&publichealthdatadidnotcontributesignificantly.

Limitations

• Socioeconomicmeasuresataggregatelevel– smallgeographicarea,butstillaggregate– limitedgeographicvariationbecauseonlywithinasingleurbanarea– individuallevelmeasuresgenerallyunavailablefromEHRs

• Highneed,vulnerablepopulation– limitedgeneralizability– probablylotsofunmetneed

• Allserviceswereco-locatedwithprimarycare– Maynotapplytoreferralstooutsideservices/otherorganizations

• Noassessmentwhetherornotthereferralwasappropriateorappointmentwaskept

Predictivemodelsforreferralstosocialservicesarecurrentlylive.

5:00AM

Beforeclinicsopen

Impactofpredictedmodelsonreferralratescurrentlybeingevaluated.

August September October November December JanuaryJulyJune

Next3cliniclocationsliveBaseline

3cliniclocationsliveBaseline

Last3cliniclocationsliveBaseline

Usingpredictivemodelingtoidentifypatientswhoneedsocialservices.

• Indicationsthatpredictivemodelingforsocialservicesmaybeuseful– modelsleveragedEHR andHIE data– performancecouldbeimproved,but

consistentwithliterature• Socioeconomic&publichealthmeasures

(attheaggregatelevel)didnotimprovemodelperformance

JoshuaRVest,PhD,[email protected]

Documents

Using Predictive Modeling to Identify Patients Who Need ...systemsforaction.org/sites/default/files/documents/... · Using predictive modeling to identify patients who need social