DRILLING DOWN SMALL GROUP

“Data Matters”

2014 Technical Assistance Institute

Seating for this session

Please locate your State/Territory name at the table

Attendees from the same Council should sit at the same table.

There will be multiple Councils at the

table, but please make sure your Council colleagues are sitting together.

Purpose of session

Review specific elements for a robust APPR annual progress report narrative for objective(s)

Apply review prompts from self-evaluation tool to sample APPR annual progress report narrative(s)

Utilize APPR narrative rubric to determine areas of strengths and areas for improvement

Provide feedback to strengthen the self-evaluation tool and Identify areas where more information is needed

The Annual Progress Report

DD Councils provide an annual progress report for each objective that was addressed or scheduled to be addressed during the previous federal fiscal year.

The narrative should communicate descriptive information about activities that were implemented toward achieving the objective.

Our materials…

Materials continued…

2 annual progress report narratives

Elements

As you conduct the mock peer review of the narrative samples, please make any notes about any thoughts, feedback and ideas you may have to strengthen this tool – we will review and collect at the end of the session

The self-evaluation tool

Elements of the annual progress report

Introduction/Background Implementation of activities Performance measures Deliverables/Products Evaluation of Outcomes Success Stories (individual and/or policy) Timelines

Introduction/Background

An introduction or background for the annual progress report narrative helps provide a context (or framework) for the activities described.

Helpful information to frame the activity is whether or not the activity is new, or is a continuation activity; location of where the activity took place (specific areas or statewide); other information that adds value to the description

Implementation of Activities

The annual progress report narrative should include a clear description of the activity/activities.

Suggestions for information include: How the activity was implemented Who implemented the activity Summary of results and accomplishments Barriers to achievement

Performance Measures/Outputs Performance measure numbers should be

supported by the narrative description Information should explain how the

performance measure numbers were achieved

Information should clearly relate to the activity described

Measure numbers should be reported only once, in the category that best fits the data

If the objective includes a measurable number, the information should include a status statement.

Deliverables and/or Products A deliverable is often described as something

that is completed or delivered under the terms of an agreement or contract. Examples of deliverables and products are:

training modules, reports, booklets, brochures, web-sites.

The annual progress report narrative should include information about the product that is clearly connected to the activity and describe how the deliverable/product impacted the activity

Evaluation/Outcomes

The annual progress report narrative should contain information to explain what was achieved and how the achievement was assessed.

Include a description of evaluation methods.

Include enough information to clearly explain which activities or projects achieved the outcomes and how this was done.

Success Stories – 2 types

If you have stories to support the information in your annual progress report narrative, please include them! 1. Stories of people with developmental

disabilities who lives are better because of DD Council work

2. Stories of policy or legislative changes that happened as a result of Council work likely to positively impact the lives of people with developmental disabilities or that will prevent a potential negative public policy impact

Timelines

As you know, the timelines are imported into the APPR template, then the Council checks off a box that indicates the timeline status (met, not met or partially met)

The narrative should support the timeline assessment and if the timeline is not met, does a state plan amendment need to be made?

As you conduct the mock peer review of the narrative samples, please make any notes about any thoughts, feedback and ideas you may have to strengthen this tool – we will review and collect at the end of the session

The Rubric

The Rubric

What is a rubric? A rubric describes the important components of the work being reviewed. Criterion is included that described different levels of content.

The rubric can assist Council staff with understanding expectations for the PPR narrative elements.

We recognize the rubric criteria are subjective, meaning based on how the reader perceives the information, different conclusions could be reached.

Let’s apply the tools!

At your table, we’d like you to work with other colleagues from your own DD Council and other Council peers

Using APPR Narrative #1, use the Annual Progress Report Narrative Self-evaluation Review Tool to review the narrative.

Identify areas of strength(s) Identify areas for improvement(s) Where would the elements of this narrative

best fit on the Rubric?

Narrative #1

Did you find clear information about the activity and how it was implemented?

Was there enough background information to help provide context for the activity?

Were there any success stories to help demonstrate the impact of the activity?

What was the impact of the activity? Was there information about how outcomes were achieved?

Are the performance measure numbers supported by the narrative?

Were there any deliverables/products?

Was there information provided to support the

assessment of the timeline?

Annual Progress Report on Objective

Objective 4.1:

Support the development and dissemination of at least two (2) resources to be utilized throughout [State] in trainings and technical assistance, provided by other stakeholders, to facilitate overall health.

Possible areas to strengthen?Did your table participants identify areas?

Elements of narrative & the rubric

 

QUALITY INDICATOR

EXEMPLARY ACCEPTABLE

NEEDS IMPROVEMENT

MISSING

Introduction/Background

EXEMPLARY ACCEPTABLE

NEEDS IMPROVEMENT

MISSING

Implementation of Activities

EXEMPLARY ACCEPTABLE

NEEDS IMPROVEMENT

MISSING

Performance Measures/Outputs

EXEMPLARY ACCEPTABLE

NEEDS IMPROVEMENT

MISSING

Deliverables/Products EXEMPLARY ACCEPTABLE

NEEDS IMPROVEMENT

MISSING

Evaluation/Outcomes EXEMPLARY ACCEPTABLE

NEEDS IMPROVEMENT

MISSING

Success Stories EXEMPLARY ACCEPTABLE

NEEDS IMPROVEMENT

MISSING

Timelines EXEMPLARY ACCEPTABLE

NEEDS IMPROVEMENT

MISSING

Using the same tools and process, review the next narrative

Narrative #2

Did you find clear information about the activity and how it was implemented?

Was there enough background information to help provide context for the activity?

Were there any success stories to help demonstrate the impact of the activity?

What was the impact of the activity? Was there information about how outcomes were achieved?

Are the performance measure numbers supported by the narrative?

Were there any deliverables/products?

Was there information provided to support the

assessment of the timeline?

Annual Progress Report on Objective

Objective 2.1: Monitor state and federal legislation and policy, inform policymakers as necessary.

The Council has monitored all federal and state legislation and regulatory changes that might affect Council members. Information and advice was provided at the request of lawmakers.

Federal legislation included proposed acts on: Medicaid, IDEA, voting, health insurance, SSI, Housing, Employment, etc... Council staff has open lines of communication with the state’s congressional delegation and continues to be available to provide information or education as requested.

On state legislation the Council followed over 40 pieces of legislation, Monitoring 17, Testifying on 12, researching 8, and referring 3 to the Cross Disability Coalition. (no action was taken on the others).

Possible areas to strengthen

Provide background information for the activity Provide information on how the activity was implemented.

For example, include information on how legislation and policies were selected to be monitored (committee? Council members? There was mention of a PAC – how is that related to the activity?)

Provide information on what advice was requested by lawmakers; how many requests for information and advice? Is there information about requests from congressional delegates vs. state level policymakers?

What activities are included in monitoring legislation? Research legislation? Who provided testimony? Council staff or Council members? People with DD and their families?

What factors were considered that caused a referral to the Cross Disability Coalition?

What factors were considered when no action was taken? Provide information about the outcomes of the activity

For example, as a result of testifying, what happened with the legislation?

As a result of monitoring and researching legislation, what happened?

As a result of referring to the coalition, what happened?

Provide information about success stories that correlate to legislative changes as a result of Council work. Information might include stories about self-advocates testifying; how legislation passed or opposed would impact an individual with developmental disabilities or their families.

Elements of narrative & the rubric

 

QUALITY INDICATOR

EXEMPLARY ACCEPTABLE

NEEDS IMPROVEMENT

MISSING

Introduction/Background

EXEMPLARY ACCEPTABLE

NEEDS IMPROVEMENT

MISSING

Implementation of Activities

EXEMPLARY ACCEPTABLE

NEEDS IMPROVEMENT

MISSING

Performance Measures/Outputs

EXEMPLARY ACCEPTABLE

NEEDS IMPROVEMENT

MISSING

Deliverables/Products EXEMPLARY ACCEPTABLE

NEEDS IMPROVEMENT

MISSING

Evaluation/Outcomes EXEMPLARY ACCEPTABLE

NEEDS IMPROVEMENT

MISSING

Success Stories EXEMPLARY ACCEPTABLE

NEEDS IMPROVEMENT

MISSING

Timelines EXEMPLARY ACCEPTABLE

NEEDS IMPROVEMENT

MISSING

Feedback on tools

We’d like to hear your thoughts and ideas of how to make these tools better!

If you’d like to email your feedback, please send to smatney@nacdd.org