Performance Measurement 201: Best practices in performance measure design & implementation

Ia Moua, Deputy Director, Grants & Program Development

Patrick Gianelli, Grants Specialist

Session Objectives

Review how CV uses performance measures

Share best practices in performance measure design

Share best practices in implementation and data collection

Explore how CV assess performance measure review

PM Overview….systematic process for measuring program outputs and outcomes.

Outputs: Amount of service provided

(people served, products created, or programs developed)

Outcomes: Changes or benefits that occur as the result of the intervention

•Can reflect changes in individuals, organizations, communities, or the environment

•Typically include changes in attitudes/beliefs, knowledge/skills, behavior, or conditions

•Must have a logical connection to the intervention and be aligned with outputs

Why Measure Performance? Recognition of progress; reflects change

• Collect reliable information about the intervention’s implementation and progress toward outcome

Accountability to funders and stakeholders

• Communicate achievements in a meaningful and

compelling way

Program Improvement

• Spot and correct problems

• Strengthen the intervention

• Determine technical assistance needs

• Determine where to allocate limited resources

How CV Uses Performance Measures

PMWs trumps the program narrative in the contract

CA PMWs:

Tell the story of individual and collective impact of AmeriCorps programs

Serve as the “blue print” for understanding a program’s allowable member activities and intended output and outcome targets

Serve as a monitory tool for CV to assess individual program progress against the awarded program design and theory of change

Best Practices: Performance Measure Design

Select PMs that fit your program design and theory of change, not vice versa

Less = more: focus on a small number of high-quality measures

Measure outputs and outcomes for program beneficiaries

Clearly define all terms used

All PMW elements are logically aligned with the program’s theory of change (community need, proposed interventions, and desired outcomes)

For longer-term outcomes, set targets that are achievable in a single grant year

Best Practices: Performance Measure Design (cont.) Use numerical targets, not percentages

For outcomes that require participant follow-up, set targets that take into account response rate and attrition

Clearly distinguish outcomes from outputs while maintaining logical alignment

Choose outcome measures that are ambitious but realistic; ensure that the program can realistically document or track the required information

Best Practices: Performance Measure Implementation & Data Collection

Set up MOUs with service sites that clearly lay out data collection responsibilities/expectations

Provide up-front training in data collection for members, site supervisors, and other program staff

Obtain baseline data so that changes can be objectively assessed, rather than assessing perceptions of change retroactively

Select data collection instruments that are valid (measure what they are supposed to measure) and reliable (yield consistent results)

Keep data collection procedures consistent over time and across different sites

Best Practices: Performance Measure Implementation & Data Collection (cont.)

Choose data collection instruments that are accessible and yield timely data

Develop creative ways to improve beneficiary responses to data collection efforts (incentives, etc.)

Allocate sufficient resources toward data collection efforts: money, time, personnel

Build in time for data review and verification prior to compiling/submitting reports

Incorporate data quality review protocols into monitoring visits to sites

Share best practices between programs and site partners

YOUR TURN…Small Group Activity

Using the CV Performance Measure Review Checklist,

1.Take 10 minutes to individually assess the performance measure against the criteria assigned to your table.

2.Take 5 minutes to share your analysis with participants at your table.

3.Select a reporter to share your group’s analysis to the larger group.

Additional ResourcesVisit available on-line performance measurement courses on the CNCS Knowledge Network:

How to Use the CNCS National Performance Measure InstructionsBuilding Evidence of EffectivenessCollecting High Quality Outcome Data, Part 1 Collecting High Quality Outcome Data, Part 2 Designing Effective Action for Change High Quality Performance Measurement Overview of Performance Measurement

Link:http://www.nationalservice.gov/resources/online-courses

Questions?

THANK YOU!