SPARCS Race/EthnicityCoding Concordance Analysis

October 7th 2013

So How Does New York State Facilities Measure Up

• Preformed Concordance Analysis on SPARCS Race and Ethnicity data elements

• Used NYS Medicaid and Vital Statistics as comparison data sets.

Assumptions

• Medicaid provides more accurate coding for race and ethnicity that SPARCS. This is primarily because:

– Medicaid has a higher rate of self reporting than SPARCS.

– Medicaid supports the ability to report multiple races per member.

Assumptions – cont.

• Vital Statistics provides more accurate coding for race and ethnicity than SPARCS since it is has:

– A higher rate of self reporting

– Both mother and fathers race to compare to infant

– More complete coverage for birth events since it covers all NYS births

Summary of Findings

• Native American and Hawaiian are poorly coded– Percent of population is low.

• SPARCS race coded as Other is quite high:– 38% for Outside NYC– 24% for NYC

• Best facilities are coding > 95% accurately

• Worst facilities are coding < 10% accurately

SPARCS-Medicaid Population

• Select SPARCS records with 2012 discharge date.

• Remove newborns and mothers from the comparison because Vital Stats provides a more complete (and presumably more accurate) record of newborn and mothers race/ethnicity than Medicaid.– SPARCS newborns were removed when the APR Major

Diagnostic Code (MDC) equaled ‘15’– SPARCS mothers birth records were removed when the

APR Diagnosis Risk Grouper (DRG) equaled 540, 541, 542, or 560

SPARCS-Medicaid Population-Cont-

• Select Medicaid records matched by the SPARCS matching process.

• For Medicaid, we are relying solely on the races coded in the ethnicity array rather than the race field. An analysis of these fields showed that the ethnicity array provided more specific (albeit minor) race information than the single valued race field.

• For recipient with > 1 race coded, we apply the following rules:– When 1 of the Medicaid races match the SPARCS race, that is

considered a match.– When there are > 1 Medicaid races coded, we add 1 to the "Multiple"

bucket. This basically provides an "asterisk" to the match indicated there were other races coded.

Race/Ethnicity Comparison Reports

• Three reports were produced for both NYC and Outside NYC:– Overall Dashboard Comparison of SPARCS to Medicaid Race

• Matrix of SPARCS Race to Medicaid Race

– Facility Level Scorecard of SPARCS to Medicaid Race concordance• Includes Kappa Statistic and Multiple Race %

– Facility-Race Level Comparison of SPARCS to Medicaid Race• Matrix of SPARCS Race to Medicaid Race• Sorted by Facility Name

Race/Ethnicity Comparison Reports

• The same reports are also produced for Ethnicity

• The same reports are also produced for SPARCS-Vital Stats Births– Mom to Mom Comparison– Baby to Mom/Dad Comparison

Overall NYC Concordance• Native American and Hawaiian are poorly coded but percent of population is low.• SPARCS race coded as Other is quite high at 38%

Overall Outside-NYC Concordance• Black and White coding is better than NYC by 15% or better.• SPARCS race coded as Other is lower than NYC by 24%

174 Delaware Valley Hospital Inc Asian 100.0% 0.0% 0.0% 0.0% 0.0% 0.7% Black or African

American0.0% 100.0% 0.0% 0.0% 0.0% 4.2%

Other Race 0.0% 0.0% 0.0% 33.3% 66.7% 2.1% White 0.0% 0.0% 0.0% 1.5% 98.5% 93.0%

Concordance Rate

• The Concordance Rate is an indicator of how well (or poorly) a facility is coding race and ethnicity compared to either Medicaid or Vital Statistics.

• We removed Unknown or Other from the calculation.

Correct Observations Concordance Rate = ---------------------------- Compared Observations

• The Kappa Statistic should also be considered when using the concordance rate.

Kappa Statistic• We identified a need to adjust for the varying diversity across facilities.

• Reason was that some facilities service areas of predominately one race, while others have their admission spread over multiple races.

• Cohen’s Kappa Statistic was used to support this adjustment.

Kappa Statistic• Cohen's Kappa's formula is: P(a) - P(e)

------------- 1 - P(e)

• where P(a) is the observed percentage agreement and P(e) is the probability of chance agreement • We exclude those facilities where >80 percent of matches are Other/Unknown

• The closer the kappa is to 1, the more agreement between the two raters (SPARCS and Medicaid or SPARCS-Vital Statistics)

• Facilities with a higher Kappa have coded a more diverse population.

• Concordance rating can be adjusted based on higher Kappa.

Top NYC Facilities

Top Outside-NYC Facilities