ELLEN ZAGER HILL , MS

TB PROGRAM EPIDEMIOLOGISTIDAHO DEPARTMENT OF HEALTH &

WELFARE

Refugee and Immigrant B Notification Program Evaluation

—Idaho

Idaho Geography (1)

Geographically large western state with a low population 19.2 people per square

mile (ranked 44th in US)

Population ≈ 1.58 million people 37% of population is in

the Boise metro area (Ada & Canyon Counties)

Idaho Geography (2)

44 counties divided into 7 multi-county regional public health districts

95%+ of refugees and immigrants settle in Boise (Ada County) and Twin Falls (Twin Falls County).

Background

For a state with the size population that Idaho has, the state receives an unusually large number of refugees and immigrants

2 local public health districts receive essentially all of the refugees arriving in the state: PHD4 (90%+) and PHD5 (~10%)

Immigrants arriving in Idaho are pretty evenly distributed; in SFY 2007–2009, 83 immigrants with TB notifications arrived

SFY

Idaho Populatio

n

No. of Refugees Arriving

Per capita ratio

(per 10,000 people)

No. with TB notifications

2007 1,499,402 834 5.6 30 (3.5%)2008 1,523,816 1,080 7.1 54 (5.0%)2009 1,545,801 1,291 8.4 76 (5.8%)

Evaluation Plan

Logic Model

Refugee/ Immigrant with a TB

notification

Inputs

Private Provider

State TB Program

Staff

LPHD

Short term Long term

Outcomes

The LPHD will increase the

proportions of completion of

medical evaluations within

90 days.

Increase the proportions of refugees/immigrants who

started on LTBI treatment who complete treatment.

Reduce the number of

foreign born TB cases.

Outputs

Every LPHD has a process for contacting, evaluating, and treating refugees and

immigrants with TB notification.

Data for grant progress report.

Refugees and immigrants with a TB notification are medically evaluated for

TB.

Refugees or immigrants treated who need to be

treated.

Activities

Form turned into state TB

program

Fill out CDC TB Notification

form

Complete medical evaluation

Complete treatment for

LTBI

OEFI receives notification from CDC

Start treatment for LTBI

District receives notification from OEFI

Contact Refugee or immigrant

Initiate medical evaluation

Evaluation Goal Evaluation Team

Baseline analysis of medical screen activities of refugees and immigrants arriving in Idaho with a TB notification

Lead Evaluator Ellen Zager Hill – TB

PEN Focal Point/TB Epi

Team Members Christine Hahn – TB

Controller/State Epi Adele Smith – MPH

student from ISU

Evaluation Plan (1)

Evaluation Plan (2)

Big Picture Questions Corresponding Specific Questions

Are the medical evaluations for refugees and immigrants with TB notifications done in a timely manner?

What is the proportion of refugees or immigrants that were evaluated within 90 days of arrival?

Are the PHDs sending notifications back to the OEFI in a timely manner?

What is the proportion of TB notification forms that are received within 120 days of receiving the notification?

What is the process that the PHDs go through to contact the refugees and immigrants with TB notifications for medical evaluations?

7 out of 7 PHDs have a process to contact refugees or immigrants with TB notifications

Evaluation Plan (3)

Big Picture Questions (continued)

Corresponding Specific Questions (continued)

Do the PHDs have a different process for evaluating and treating refugees and immigrants with TB notifications?

7 out of 7 PHDs have no difference in evaluation or treatment of refugees and immigrants with TB notifications

What is the process the PHDs perform to evaluate refugees and immigrants with TB notifications?

7 out of 7 PHDs have a process to evaluate refugees or immigrants with TB notifications.

What is the process the PHDs perform to treat refugees and immigrants with TB notifications?

7 out of 7 PHDs have a process to treat the refugees or immigrants with TB notifications who have LTBI.

Data Sources Execution Needs

Idaho Refugee & Immigrant Log (MS Excel spreadsheet)

CDC’s Electronic Disease Notification System

Interviews with PHD staff who coordinate TB activities (via phone)

TimeSurvey questionsPhoneSoftware to map

processes (MS Visio) (relatively) Clean TB

notification dataData analysis tool (MS

Excel and SAS)

Evaluation Plan (3)

Carrying Out theEvaluation Plan

Carrying Out the Evaluation Plan (1)

Developed a set of questions to ask the PHD staff who carry out TB activities

Developed a script to go with the questions

Carrying out the Evaluation Plan (2)

A flow of TB notification data from the state-level perspective was developed (shown at left)

Based on survey questions, flow charts of the evaluation and treatment processes were developed for each PHD

Data Cleaning Data Analysis

Started with our Idaho Refugee & Immigrant Log Took out all records that

were not for TB notifications

Filled in what missing data we could from hard copy records and the EDN system

Initially done with MS Excel

Final analysis done with SAS Benefit of using a

program like SAS is that you have a record of exactly how the data was cleaned, what analysis was done, and how it was done.

Carrying out the Evaluation Plan (3)

Carrying out the Evaluation Plan (4)

Some lessons learned…

Logs (especially those with no data standardization programmed in) are a difficult data source to analyze

(A lesson re-learned) small denominators can be a problem

Clearly document how you collect, clean, and analyze data An evaluation report is like a lab report. Your methods section may be your most important section.

(Sometimes even more important than your results.) SAS or other statistical analysis programs are a great way to

document how you are cleaning and analyzing data

Evaluation Results

Evaluation Results (1)

Are the medical evaluations for refugees and immigrants with TB notifications done in a timely manner? What is the proportion

of refugees or immigrants that were evaluated within 90 days of arrival?

Overall (SFY2007–2009) 67% were evaluated within 90 days of arrival.

Evaluation Results (2)

Are the PHDs sending notifications back to the OEFI in a timely manner? What is the proportion

of TB notification forms that are received within 120 days of receiving the notification?

Overall 76% of TB notification forms were returned within 120 days of receiving the original notification.

Evaluation Results (3)

Process Questions Results7 out of 7 PHDs have a process to contact refugees or immigrants with TB notifications

• All 6 PHDs interviewed have a process (the 7th is known (anecdotally) to have a process.• Most have a process for contacting hard-to-reach R/Is and a defined point when they declare someone “lost-to-follow-up”

7 out of 7 PHDs have no difference in evaluation or treatment of refugees and immigrants with TB notifications

• Of the 6 PHDs interviewed only 4 have received notifications for both refugees and immigrants. Of these 4, only 1 reported a difference in protocol (a different staff member follows each group)

7 out of 7 PHDs have a process to evaluate refugees or immigrants with TB notifications

• All PHDs have a process for evaluating refugees and immigrants• There is significant variation in these processes between PHDs

7 out of 7 PHDs have a process to treat the refugees or immigrants with TB notifications who are diagnosed with LTBI.

• Of the 6 PHDs interviewed, 5 actively follow R/Is on treatment for LTBI. The remaining PHD prefers that R/Is diagnosed with LTBI are treated and followed in their medical home.

Evaluation Results (4)

Extra analysis tied to differences in processes…

*Excludes data from 1 PHD (the PHD with the most notifications) because we were unable to interview them.

PHD-level State-level

Ensure there is a system in place to track TB notifications

Review process for coordinating medical evaluation on a semi-regular schedule

Consult OEFI TB staff as needed when a problem related to medical evaluation arises

Consider adding language to TB contracts stipulating a time frame for returning TB follow-up forms

Update B Notifications chapter of Idaho TB manual

Provide at least 1 NTIP-like report a year to each PHD that contains an analysis of their TB notification statistics

Review data from calendar years 2009, 2010, and 2011 for improvement. If improvement is not seen, then consider re-evaluating this activity.

Recommendations

What has been done as a result of the evaluation

We are no longer using the Idaho Refugee and Immigrant log to monitor performance for this objective. We are using solely EDN data.

We added language to contracts stipulating when TB follow-up data needs to be submitted.

B Notifications chapter of TB manual has been reviewed an updated, but has not yet been finalized.

Completion of medical evaluation statistics as of 1/31/2012: (calendar years)

2008 2009 2010

2011

N 48 93 88 71

Mean (days) 147 132 143 70

Std Dev (days)

162 163 155 43

Low CI (days)

99 98 111 60

High CI (days)

194 166 176 80

Min (days) 15 8 15 4

Max (days) 885 795 743 1901. This data reflects only notifications for which data has been received.

2. Please note that data for all years are incomplete for various reasons.

3. The improvements in data are mainly due to process changes at the PHD that receives 90+% of refugees in the state. (This PHD was not interviewed for the evaluation.)

Questions