HMIS Performance Measurement & EvaluationTechnical Assistance Training Series

January 16th, 2015

INTRODUCTIONS

Presenters:• Amanda Borta: Program Manager• Liz Weck: Programming Intern

You:• Name• Agency & what types of projects you have• What you hope to learn today.

HOUSEKEEPING

• Training will be from 9am-11:30

• Breaks

• Access to bathroom/keys

• Please be mindful of noise when entering/exiting room

• Questions

LEARNING OBJECTIVES

Participants will review and understand….

– Performance measures identified in HEARTH

– Impact of Project performance on System Performance

– How we measure performance within the Chicago CoC

WHAT ARE PERFORMANCE

MEASUREMENTS?

Performance measurement is a process that systematically evaluates whether the Continuum

of Care’s efforts are making an impact on the clients you are serving.

HEARTH ACT PERFORMANCE

MEASURES

1. Length of time homeless2. Returns to homelessness3. Access/coverage (thoroughness in reaching persons who are

homeless)4. Overall reduction in number of persons who experience

homelessness5. Job and income growth for persons who are homeless6. Reduction in first time homeless7. Other accomplishments related to reducing homelessness

CORE HEARTH ACT MEASURES

Reduce New Episodes of

Homelessness

Reduce Returns to

Homelessness

Reduce Length of Homeless

Episodes

WHY MEASURE PERFORMANCE?

• To understand whether current activities are working to achieve intended results.

• To drive program improvement and share information on effective practices with others.

• To ensure a common understanding among all partners, staff, and consumers of what you intend to achieve and how you intend to do it.

• To communicate and advocate for community support.

• To identify areas for improvement.

• To accomplish your goals.

PERFORMANCE MEASURMENT

STRUCTURE:

Identify & Develop Performance

Indicators

Set Performance Targets

Measure Performance

Report Progress

Identify & Make Improvements

BUILDING BLOCKS OF

PERFORMANCE MEASUREMENT

Inputs: Include resources dedicated to or consumed by the system—e.g., money, programs, facilities.

Activities: what the system does with the inputs to fulfill its mission, such as providing shelter, feeding the homeless, or providing job training

Outputs: the direct products of system activities. They usually are presented in terms of the volume of work accomplished—e.g., number of participants served,

number of service engagements, number of shelter nights.

Outcomes: benefits or changes among clients during or after participating in system activities. Outcomes may relate to change in client knowledge, attitudes, values, skills,

behaviors, conditions, or other attributes.

EVALUATION INSTRUMENT

The Evaluation Instrument is:

Part of a HUD requirement of CoC’s to evaluate all projects

A tool the CoC uses to evaluate a project’s performance, structure and policies

Modified annually by a sub-committee of the HUD McKinney Vento (HMV) Committee to ensure it is in line with HUD’s and Chicago’s priorities

Utilized during ranking procedure

PERFORMANCE IN THE CoC

HMIS: OVERVIEW

• Information system designated by The Continuum of Care to comply with 24 CFR578.

• Locally administered data system used to record and analyze client, service and housing data for individuals and families who are homeless or at risk of homelessness.

• Integrate unduplicated data across programs in a community.

• Understand size, characteristics and needs of the homeless at multiple levels: project, system, locality, state, nationally.

• HMIS is the official method of measuring outcomes for the federal strategic plan.

DATA QUALITY REPORTS

1. HUD CoC APR 0625 – This data quality report facilitates the

extraction of data for the completion of the CoC APR. A companion

ART report, 0631, provides details to assist in addressing data

quality issues. This report should be run once a month at a

minimum to confirm that complete and accurate data is being

properly recorded in ServicePoint.

2. HUD CoC APR Detail 0631 – This funding report is a companion

to report 0625 and displays the details behind the CoC APR report.

This report consists of several tabs, each tab focusing on specific

portions of the CoC data. The report also includes features to

assist in data quality monitoring

• Program Capacity/Occupancy: Persons Served compared to Proposed # Served measured on a quarterly basis

• Measures are for Leavers (exited program) and Stayers (continued program enrollment)

• Client Demographics, Disability, Health Condition and at Entry, Follow‐Up and Exit

• Client Income Amount $ and Income Sources at Entry, Follow‐Up and Exit

• Client Non‐Cash Benefits at Entry, Follow‐Up and Exit

• Length of Stay (Participation) in the program

• Destination at Exit: Permanent, Temporary, Institutional, and Other

• Individual Agency Determined Project Goal

• Attainment Based on Proposed Project

– Housing Stability

– Income

– Service linkages

APR ANNUAL PERFORMANCE

REPORT

UNDERSTANDING THE APR

• Tab A: Data Quality

• Tab B: Persons and Households Served

• Tab D: Gender

• Tab E: Age, Race, and Ethnicity

• Tab F: Disabling Conditions and DV History

• Tab G: Prior Living Situation

• Tab H: Veteran Status, Physical and Medical Health Condition Types

• Tab I: Cash Income (Leavers vs. Stayers)

• Tab J: Cash Income Types (Leavers vs. Stayers)

• Tab K: Non Cash Benefits Types (Leavers vs. Stayers)

• Tab L: Length of Stay (Leavers vs. Stayers)

• Tab M: Destinations Among Leavers

TAB A: DATA QUALITY

Total # of client records included

in the report

Total # of clients with an unknown value for various date elements.

TAB B: PERSONS & HOUSEHOLDS

SERVEDTotal # of clients

served by household type

Total # of households served by household

type

Daily Averages for persons served

Quarterly PIT of persons

served

Quarterly PIT of households

served

CAPACITY vs. OCCUPANCY

The following formula is used to calculate bed utilization rates:

Bed Utilization Rate Formula

(# of clients served on a given night) ÷ (total # of beds) = bed utilization rate

Common utilization rate problems include under utilization, over

utilization and flat utilization:

• Under utilization (rates below 80%)

• Over utilization (rates over 105%)

Total # of clients served

on a given night

Total # of bedsBED

UTILIZATION RATE

TAB I: CASH INCOME

(LEAVERS vs. STAYERS)

Reports Income

By cash income range By cash income category

TAB I: CASH INCOME

(LEAVERS vs. STAYERS) Cont.

Reports changes in Income

Stayers

Leavers

Combined

CoC APR DETAIL REPORT

To clean up data using this report, follow these instructions:

1. Make sure your clients are not missing any data: a. Open the APR Detail Report (631 Report)

b. Click Tab B: Summary Detail

c. Do you see the word “Missing” anywhere on that page? If so, it means a

client is “Missing” a piece of data

d. To clean up the “Missing” data:

i. On Tab B: Summary Detail, look to the left. The 1st column lists the

Client IDs. The second column lists the client names. Each subsequent

column lists a data element. If a client is missing a data element, the word

“Missing” will appear next to their name, in the appropriate column.

The CoC APR Detail Report will help you answer two important

data quality questions:

Do my clients have all their data entered?

Are Households entered correctly?

CoC APR DETAIL REPORT Cont.

If a client is missing data, open that client’s HMIS record, and correct

the “Missing” pieces of data.

If the “Missing” data is already in the Client’s Assessment:

1. You might find that when you open the Client’s Assessment, the “Missing” piece of

data is already there. If this happens to you:

a. Make sure the start date for the Income/Non-Cash Benefit/Disability is before

the Program Entry Date/Time

b. Make sure you have answered all of the detail questions for Income/Non-Cash

Benefit/Disability in the Sub-Assessment (popup box)

c. If you have done Steps a and b, and you are still “Missing” data, contact your

Agency Admin.

TAB J: CASH INCOME TYPES

(LEAVERS vs. STAYERS)

Reports source of income

Detail

Summary

TAB K: NON CASH BENEFIT TYPES

(LEAVERS vs. STAYERS)

Reports source of Non-Cash Benefits

Detail

Summary

TAB L: LENGTH OF STAY

(LEAVERS vs. STAYERS)

Length of Program Participation (Length of Stay = LOS)

LOS by duration

LOS by average

& median

TAB M: DESTINATION AMONG

LEAVERS

Exit Destination by household type

AFTER an enrollment period of 90 days

CONSIDERATIONS

Outcomes can be calculated consistently for all clients if,

and only if:

• Staff are collecting the required information at entry and

exit for each client.

• Staff are recording this information into the HMIS in a

timely fashion (Real time, 24 hours, two business days)

• The target population is clearly defined.

• The program goals are clearly defined for the target

population.

EXAMPLES OF WHAT TO MEASURE

Prevention/Diversion

Length of Stay

Return to homelessness

•% clients diverted from shelter

•% clients prevented from homelessness

•% clients entering shelter within 1, 3, 6, 9, or 12

months of diversion/ prevention

•% exiting shelter (for permanent housing) within

30,60, 90 days

•% clients re-presenting at or entering shelter

within 3, 6, 9, or 12 months of program

completion

EXAMPLES OF WHAT TO MEASURE

Prevention/Diversion

Length of Stay

Return to homelessness

•Determine if prevention recipients entered

shelter

•Compare shelter entries for those who received

assistance with those who did not

•Examine length of stay for each program

•Track persons with multiple program entries

•Compare permanent housing exits from one period with new entries in another period •Compare HMIS ID’s for old and new entries

Tool Data you can get Ways to Use

HMIS – “canned reports”

•Entries, exits during the year

• Demographic characteristics

• Destinations

•Missing answers and rates of don’t know

answers (important)

•Run the CoC APR for the entire system or a component of the system, such as all shelters• Compare performance programs to component average, or year to year on exits to permanent housing•Evaluate missing data and improve data quality

HMIS custom reports

• Everything in your HMIS (entries, exits, demographics, outcomes, time frames)

Select metrics you are interested in ( for example: exits to permanent housing within 45 days)

Point –in –time Count

Information about total population including chronic homelessness, demographics, etc. at a particular date/week over time

• Look at changes in size of total population and subpopulations over time• Compare to system APR or AHAR for effectiveness of reach of system and with specific populations

AHAR • Population estimates, demographics, points of entry, lengths of stay, system capacity and turnover

• Look at turnover rates for singles and families, transitional housing and shelter•Compare to need from PIT count, to turnover rates of rapid rehousing programs

HPRP reports • Information on prevention/rehousing population•Costs of financial assistance•Time of assistance•Destinations

•Use to look at persons served, demand for prevention, costs of prevention•Run Coc APR using HPRP data to compare with regular APR – look at differences in population• Look in HMIS for people served in HPRP in other parts of the system

WRAPPING UP

Measure

• Gather data & establish baseline

Analyze

• Consider which programs and activities lead to strong performance

• Examine data collection policies and practices

Plan

• Consider where to adjust your program “portfolio” funding or investments to achieve better performance

• Consider policies and practices that encourage high quality data

What are some of the current policies/practices in place at your agency to ensure performance outcomes are met and

improved?

NEXT STEPS

Review Project Models Chart.

Look at previous evaluation scores & rankings.

Determine where YOU are as an agency.

Meet with your team!

Others?