Plan, Evaluate and Improve Your Health Literacy Interventions and Activities Sabrina Kurtz-Rossi, M.Ed. Health Literacy Consultant April 10, 2013

  • Published on
    25-Dec-2015

  • View
    213

  • Download
    0

Embed Size (px)

Transcript

  • Slide 1
  • Plan, Evaluate and Improve Your Health Literacy Interventions and Activities Sabrina Kurtz-Rossi, M.Ed. Health Literacy Consultant April 10, 2013
  • Slide 2
  • Objectives After attending the session, participants will be able to: Conceptualize a health literacy program evaluation Write measurable health literacy outcome objectives Identify indicators to assess both process and outcomes Identify user-friendly evaluation resources for future use Olney, C. and Barnes, Susan (2006). Planning and Evaluating Health Information Outreach. NN/LM Outreach Evaluation Resource Center. Seattle, WA.
  • Slide 3
  • Ten Attributes of Health Literate Health Care Organizations Brach C, Keller D, Hernandez LM, et al. Ten Attributes of Health Literate Health Care Organizations. Washington DC: National Academy of Sciences. 2012
  • Slide 4
  • Case Study Providence, RI Dorcas Place Adult and Family Learning Center Computer Lab Western Maine Auburn Public Library Computer Training Class
  • Slide 5
  • Research vs. Evaluation Research: To draw causal links between observed phenomena and to add to the knowledge base of the field Evaluation: To demonstrate effectiveness, and determine what works and what does not for the purpose of program improvement Boulmetis, J and Dutwin, P (2005). The ABCs of Evaluation. Jossey-Bass. San Francisco, CA
  • Slide 6
  • Health Literacy Program / Evaluation Planning Step 1: Know Your Audience Step 2: Identify Goals and Objectives Step 3: Develop Program (and evaluation) Plan Step 4: Implement and Evaluate Step 5: Apply and Share Results
  • Slide 7
  • Step 1: Know Your Audience Start with an Assessment Identify needs in context Guide your program Build on already existing capacity Identify champions within your organization Methods Informal discussion, observation Formal Key informant interviews, focused group discussions, survey
  • Slide 8
  • Attribute 1: In a Health Literate Organization leadership makes health literacy integral to its mission, structure, and operations.
  • Slide 9
  • Case Study Methodology: Leaders at community-based organizations interviewed. Semi- structured interview protocol designed to explore community confidence in using the Internet to find health information and local resources to support access. All interviews conducted in person. Data were coded and analyzed into themes. Key Findings: Maine has a growing senior population and community-based organizations support public health efforts. Public schools, adult education, and libraries offer computer access and support (n=9). Latinos get their health information from sources they know and trust, including the radio. A network of minority health programs offer computers access, training and support (n=10).
  • Slide 10
  • Step 2: Identify Goals & Objectives Goal -- Why you are doing what you are doing and for whom Example: Increase use of health literacy interventions Improve community access to reliable online health information
  • Slide 11
  • Attribute 2: A Health Literate Organization integrates health literacy into planning, evaluation, patient safety, and quality improvement.
  • Slide 12
  • Case Study Focused Health Information Outreach Project Goal: Increase access to health information among communities experiencing health disparities National Network of Libraries of Medicine New England Region (NN/LM NER) http://nnlm.gov/ner/http://nnlm.gov/ner/
  • Slide 13
  • Step 3: Develop Program(and Evaluation) Plan Logic Model: A planning tool connecting activities to outcomes If you have these resources ResourcesActivities/OutputsOutcomes and conduct these activities you will get these outcomes
  • Slide 14
  • Why a Logic Model? Provides a framework for planning Helps organize your thoughts Clarifies intended outcomes Links activities to outcomes Acts as a communication tool ActivitiesResources OutputsOutcomes
  • Slide 15
  • Attribute 3: A Health Literate Organization prepares its workforce.
  • Slide 16
  • Case Study Goal: Increase access to health information among communities experiencing health disparities ResourcesActivitiesOutputsOutcomes Increase knowledge of NLM resources Increase use of NLM resources Increase community experts to help others Number of trainings held Number trained Types of materials developed Number of materials distributed Develop training materials Distribute materials at community events Consultant Outreach coordinators NLM materials NLM funding Conduct trainings Working student Librarians in communities
  • Slide 17
  • Activity Choose a goal (or use your own) To increase staff awareness and use of health literacy interventions To improve community access to reliable online health information To improve health information literacy skills among patients and their families Complete the logic model worksheet - identify outcomes, activities, outputs, and resources
  • Slide 18
  • Attribute 4: A Health Literate Organization involves the population served in evaluation of health information and services.
  • Slide 19
  • Case Study Community Assessment Key informant interviews Program Tailored efforts based on findings Community-based partnerships Evaluation Outcome assessment: pre-post training evaluations, follow-up assessment Process assessment: story-based evaluation, community partner exit interviews Western Maine (rural) Providence, RI (Latino)
  • Slide 20
  • Evaluation Plan Process To determine what worked and what didnt For program improvement Outcome To demonstrate effectiveness, value, and influence For comparison purposes, to determine which programs are more effective
  • Slide 21
  • Attribute 5: A Health Literate Organization meets the needs of the populations and avoids stigmatization.
  • Slide 22
  • Process Assessment Methods Activity logs Teacher feedback forms Focused group discussion Interviews
  • Slide 23
  • Outcome Assessment Methods Existing records Pre/post tests Follow-up surveys Observation Interview Story-based
  • Slide 24
  • Attribute 6: In a Health Literate Organization providers and staff use health literacy strategies in verbal communication and confirm understanding.
  • Slide 25
  • Case Study Outcomes (what you want)Indicators (how you will know) Increased knowledge of NLM resources Increased use of NLM resources Increased community experts to help others Will report use two weeks later of follow up Will name resources on post- surveys Will document helping others on story-based evaluation forms Outcomes & Indicators
  • Slide 26
  • Measurable Objectives Four components of a measurable objective 1. Condition Following the training After watching the video 2. Who the learner the patient the caregiver
  • Slide 27
  • Measurable Objectives 3. Performance (knowledge, attitudes, behavior) will name will agree will use 4. Criterion (quality/quantity) 5 out of 6 at least 2 the correct way
  • Slide 28
  • Characteristics of a Measurable Objective Is audience focused, not intervention focused (the participant will) Includes action verbs Targets one expectation Includes observable criteria for evaluation Matches goals to activities/strategies
  • Slide 29
  • Attribute 7: In a Health Literate Organization consumers have easy access to health information and services and navigation assistance.
  • Slide 30
  • Blooms Taxonomy Cognitive explain, recall, list, predict, select, describe, define Affective agree, avoid, help, join Psychomotor use, develop, design, repair, taste, use, move
  • Slide 31
  • Verbs to Avoid Know Comprehend Understand Appreciate Study Gain knowledge of Learn
  • Slide 32
  • Attribute 8: A Health Literate Organization develops plain language materials that are easy to understand and act on.
  • Slide 33
  • Case Study ConditionWhoPerformanceCriterion Immediately after training participantswill name Two weeks after training participantswill report using Three months training participants NLM resources on post-survey NLM resources on follow up interview will write a story Describing how they helped others find information on story-based evaluation form Writing Measurable Objectives
  • Slide 34
  • Activity Choose an outcome (or use your own) Increase confidence in using plain language/teach-back Increased awareness of available online health information resources Increased interaction with diabetes educators Complete the writing measurable objectives worksheet - identify condition, audience, performance, and criterion
  • Slide 35
  • Step 4: Implement and Evaluate Expect the Unexpected Be flexible Adjust to the environment Address learners needs Tweak your notion of success
  • Slide 36
  • Attribute 9: A Health Literate Organization addresses health literacy in high-risk situations, including transitions and about medicines.
  • Slide 37
  • Case Example Rhode Island: Pre/Post: Confidence in Ability to Find Health Information on the Internet Pre N= 52 0 Post N=503 Maine: Pre/Post: Ever Used NLM Resources Pre N=72 Post N=55
  • Slide 38
  • Case Example Successes Training participants shared resources with friends and neighbors Partners will continue to promote and distribute resources Providers used resources to find accurate information to use in their work Challenges Pre- and post-evaluations were hard to administer
  • Slide 39
  • Step 5: Apply and Share Results Apply Lessons Learned Make changes based on what you learned Replicate with similar or different audiences Share Your Findings Partners Colleagues
  • Slide 40
  • Attribute 10: At a Health Literate Organization they clearly communicates about health plans and health care costs.
  • Slide 41
  • Evaluation Resources Planning and Evaluating Health Information Outreach Projects Measuring the Difference: Guide to Planning and Evaluating Health Information Outreach http://nnlm.gov/evaluation/guide/index.html
  • Slide 42
  • Questions & Comments Sabrina Kurtz-Rossi, M.Ed. Principal | Health Literacy Consultant 781-835-6488 sabrina@kurtz-rossi.com Kurtz-Rossi & Associates Health*Literacy*Education*Evaluation www.kurtz-rossi.com Member, Clear Language Group www.clearlanguagegroup.com Health Literacy Leadership Institute, June 10-14, 2013 http://healthliteracyleadreship.com