42
Plan, Evaluate and Improve Your Health Literacy Interventions and Activities Sabrina Kurtz-Rossi, M.Ed. Health Literacy Consultant April 10, 2013

Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

  • Upload
    mada

  • View
    35

  • Download
    0

Embed Size (px)

DESCRIPTION

Plan, Evaluate and Improve Your Health Literacy Interventions and Activities. Sabrina Kurtz-Rossi, M.Ed. Health Literacy Consultant April 10, 2013. Objectives. After attending the session, participants will be able to: Conceptualize a health literacy program evaluation - PowerPoint PPT Presentation

Citation preview

Page 1: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

 

Sabrina Kurtz-Rossi, M.Ed. Health Literacy Consultant

April 10, 2013

Page 2: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Objectives

• After attending the session, participants will be able to:– Conceptualize a health literacy program evaluation 

– Write measurable health literacy outcome objectives

– Identify indicators to assess both process and outcomes 

– Identify user-friendly evaluation resources for future use 

Olney, C. and Barnes, Susan (2006). Planning and Evaluating Health Information Outreach. NN/LM Outreach Evaluation Resource Center. Seattle, WA.

Page 3: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Ten Attributes of Health Literate Health Care Organizations

Brach C, Keller D, Hernandez LM, et al. Ten Attributes of Health Literate Health Care Organizations. Washington DC: National Academy of Sciences. 2012

Page 4: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Case Study

Providence, RI

Dorcas Place Adult and Family Learning Center Computer Lab

Western Maine

Auburn Public Library Computer Training Class

Page 5: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Research vs. Evaluation

• Research: To draw causal links between observed phenomena and to add to the knowledge base of the field

 

• Evaluation: To demonstrate effectiveness, and determine what works and what does not for the purpose of program improvement

Boulmetis, J and Dutwin, P (2005). The ABCs of Evaluation. Jossey-Bass. San Francisco, CA

Page 6: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Health Literacy Program / Evaluation Planning

• Step 1:  Know Your Audience

• Step 2: Identify Goals and Objectives

• Step 3: Develop Program (and evaluation) Plan

• Step 4: Implement and Evaluate

• Step 5: Apply and Share Results

Page 7: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Step 1: Know Your Audience

• Start with an Assessment – Identify needs in context – Guide your program – Build on already existing capacity– Identify champions within your organization

• Methods– Informal – discussion, observation– Formal – Key informant interviews, focused group 

discussions, survey

Page 8: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Attribute 1:

In a Health Literate Organization leadership makes health literacy integral to its mission,

structure, and operations. –

Page 9: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Case Study

• Methodology:  Leaders at community-based organizations interviewed. Semi-structured interview protocol designed to explore community confidence in using the Internet to find health information and local resources to support access. All interviews conducted in person. Data were coded and analyzed into themes. 

• Key Findings:•  Maine has a growing senior population and community-based organizations 

support public health efforts. Public schools, adult education, and libraries offer computer access and support (n=9). 

• Latinos get their health information from sources they know and trust, including the radio. A network of minority health programs offer computers access, training and support (n=10).

Page 10: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Step 2: Identify Goals & Objectives

• Goal -- Why you are doing what you are doing and for whom

• Example: – Increase use of health literacy interventions

– Improve community access to reliable online health information

Page 11: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Attribute 2:

A Health Literate Organization integrates health literacy into planning, evaluation,

patient safety, and quality improvement. –

Page 12: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Case Study

Focused Health Information Outreach Project

Goal:  Increase access to health information among communities experiencing health disparities

National Network of Libraries of Medicine – New England Region (NN/LM NER) http://nnlm.gov/ner/ 

Page 13: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Step 3: Develop Program(and Evaluation) Plan

Logic Model: A planning tool connecting activities to outcomes

If you have these resources

Resources Activities/Outputs Outcomes

and conduct these activities

you will get these outcomes

Page 14: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Why a Logic Model?

• Provides a framework for planning• Helps organize your thoughts• Clarifies intended outcomes• Links activities to outcomes • Acts as a communication tool

ActivitiesResources

OutputsOutcomes

Page 15: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Attribute 3:

A Health Literate Organization prepares its workforce.

Page 16: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Case StudyGoal: Increase access to health information among communities experiencing health disparities

Resources Activities Outputs Outcomes

Increase knowledge of NLM resources

Increase use of NLM resources

Increase community experts to help others

Number of trainings held

Number trained

Types of materials developed

Number of materials distributed

Develop training materials

Distribute materials at community events

Consultant

Outreach coordinators

NLM materials

NLM funding

Conduct trainings

Working student

Librarians in communities

Page 17: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Activity

• Choose a goal (or use your own)– To increase staff awareness and use of health literacy interventions

– To improve community access to reliable online health information

– To improve health information literacy skills among patients and their families 

• Complete the logic model worksheet - identify outcomes, activities, outputs, and resources 

Page 18: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Attribute 4:

A Health Literate Organization involves the population served

in evaluation of health information and services.

Page 19: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Case Study

• Community Assessment– Key informant interviews 

• Program– Tailored efforts based on findings – Community-based partnerships

• Evaluation – Outcome assessment:  pre-post training evaluations, 

follow-up assessment– Process assessment:  story-based evaluation, 

community partner exit interviews 

Western Maine (rural)

Providence, RI(Latino)

Page 20: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Evaluation Plan

• Process– To determine what worked and what didn’t– For program improvement

• Outcome– To demonstrate effectiveness, value, and influence– For comparison purposes, to determine which programs 

are more effective

Page 21: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Attribute 5:

A Health Literate Organization meets the needs of the populations

and avoids stigmatization.

Page 22: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Process Assessment

• Methods – Activity logs– Teacher feedback forms– Focused group discussion– Interviews

Page 23: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Outcome Assessment

• Methods– Existing records– Pre/post tests– Follow-up surveys– Observation – Interview – Story-based 

Page 24: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Attribute 6:

In a Health Literate Organization providers and staff use health literacy strategies in verbal communication

and confirm understanding.

Page 25: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Case Study

Outcomes (what you want) Indicators (how you will know)

Increased knowledge of NLM resources

Increased use of NLM resources

Increased community experts to help others

Will report use two weeks later of follow up

Will name resources on post-surveys

Will document helping others on story-based evaluation forms

Outcomes & Indicators

Page 26: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Measurable Objectives

Four components of a measurable objective1. Condition

• Following the training…• After watching the video…

2. Who• …the learner…• …the patient…• …the caregiver

Page 27: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Measurable Objectives

3. Performance (knowledge, attitudes, behavior)

• …will name…• … will agree…• … will use…

4. Criterion (quality/quantity) • …5 out of 6…• … at least 2…• … the correct way…   

Page 28: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Characteristics of a Measurable Objective

• Is audience focused, not intervention focused (the participant will…) 

• Includes action verbs

• Targets one expectation

• Includes observable criteria for evaluation

• Matches goals to activities/strategies

Page 29: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Attribute 7:

In a Health Literate Organization consumers have easy access to health

information and services and navigation assistance.

Page 30: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Bloom’s Taxonomy

• Cognitive – explain, recall, list, predict, select, describe, define

• Affective  – agree, avoid, help, join 

• Psychomotor – use, develop, design, repair, taste, use, move

Page 31: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Verbs to Avoid

KnowComprehendUnderstandAppreciate

StudyGain knowledge of

Learn 

Page 32: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Attribute 8:

A Health Literate Organization develops plain language materials that

are easy to understand and act on.

Page 33: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Case Study

Condition Who Performance Criterion

 

 

Immediately after training

participants will name

Two weeks after training participants will report using

Three months training

participants

NLM resources on post-survey

NLM resources on follow up interview

will write a story Describing how they helped others find information on story-based evaluation form

Writing Measurable Objectives

Page 34: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Activity

• Choose an outcome (or use your own)– Increase confidence in using plain language/teach-back– Increased awareness of available online health 

information resources– Increased interaction with diabetes educators

• Complete the writing measurable objectives worksheet - identify condition, audience, performance, and criterion

Page 35: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Step 4: Implement and Evaluate

• Expect the Unexpected

– Be flexible

– Adjust to the environment

– Address learners’ needs

– Tweak your notion of success

Page 36: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Attribute 9:

A Health Literate Organization addresses health literacy in high-risk

situations, including transitions and about medicines.

Page 37: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Case Example

Rhode Island: Pre/Post: Confidence in Ability to Find Health Information on the Internet

Pre N= 520

Post N=503

Maine: Pre/Post: Ever Used NLM Resources

Pre N=72

Post N=55

Page 38: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Case Example

• Successes– Training participants shared resources with friends and 

neighbors– Partners will continue to promote and distribute resources– Providers used resources to find accurate information to 

use in their work 

• Challenges – Pre- and post-evaluations were hard to administer

Page 39: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Step 5: Apply and Share Results

• Apply Lessons Learned

– Make changes based on what you learned

– Replicate with similar or different audiences 

• Share Your Findings

– Partners 

– Colleagues 

Page 40: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Attribute 10:

At a Health Literate Organization they clearly communicates about health

plans and health care costs.

Page 41: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Evaluation Resources

• Planning and Evaluating Health Information Outreach Projects

• Measuring the Difference: Guide to Planning and Evaluating Health Information Outreach

http://nnlm.gov/evaluation/guide/index.html

Page 42: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities

Questions & Comments

Sabrina Kurtz-Rossi, M.Ed.Principal | Health Literacy Consultant [email protected]

Kurtz-Rossi & Associates Health*Literacy*Education*Evaluationwww.kurtz-rossi.com 

Member, Clear Language Groupwww.clearlanguagegroup.com 

Health Literacy Leadership Institute, June 10-14, 2013 http://healthliteracyleadreship.com