Facilitating Use of Strategic Information

Outline

• Creating Data Demand and Use

• Analyzing and reporting results according to

the target audience

• Data-utilization strategies

• Tools for facilitating use of M&E results for

policy and program development

Information for Decision Making

• Information must be based on quality data in order to be useful

• Information must be communicated effectively in order to be useful

• Information must be used to be good M&E

Information Needs

• Different for different stakeholders

– view activities from different perspectives

– have different degrees of understanding of the program

– need different information

– want different information

– need/want information at different levels of complexity

– have different intensities of interest

Rationale for Data Demand and Utilization

Central to achieving evidence-based decisions are:

• Increasing user demand for quality information

• Increasing availability of data and analysis methods

• Increasing use of data and analysis methods

Components of Strategic Information

Surveillance Population Surveys

Basic/Academic Research

Programs

Routine HMIS Facility Survey

Monitoring, Analyzing and

Triangulating Core Indicators

Program Evaluation Operations Research Targeted Evaluation (Including, outcome evaluations, rapid ethnographies, economic evaluations, case studies)

National Level Response

Coverage Surveys

National Level Impact/Collective

Effectiveness Modeling

Surveillance/Impact M&E

Identifying Needs

Designing & Implementing Intervention

Measuring Outcome

and Impact

The Big Picture – improved health outcomes

Demand for Information

TechnicalTechnicalApproachApproach

SystemsSystemsApproachApproach

IndividualIndividualBehaviorBehavior

Data Demand and Use

Political, Economic, Cultural, Social Environment

Use of data, analyses, and other information for decision making occurs within a political context

Technical Approach

Individual Behavior

OrganizationalApproach

Information for Decision Making

DecisionDecisionmakersmakers DecisionsDecisions

Why

?

Why

?W

ho else?

Who else?

How?How?

What What information?information?

Look at:• Current practices

• Organizational incentives and culture

• Individual attitudes and motivation

Develop a Data Demand and Use Profile:

• specify decision makers at various levels

• data needed to make decisions

• timing of data requirements

• the format for data presentation

Systems Approach to DDUSystems Approach to DDU

Technical ApproachTechnical Approach

Technical resources for transforming data into information

– Data sources

– Tools for data collection

– Methodologies

Analyzing M & E Data

Focus on the main information the M&E data need to provide?

• Process:– Was the program carried out as planned?– How well was it carried out?

• Results:– Did the expected change occur?– How much change occurred?

• Impact: – Is the change attributable to the program?– Does the change mean program “success”?

Increase availability of data

• Explain indicators – design

– interpretation

– significance

– meaning of changes in value

• Explain data – collection processes– calculation processes– quality

Explain Changes

• Meaning of changes in indicator values – what can explain the change

• What other information should M&E communicate to help use this indicator to make program decisions?

Explaining Contradictory Data and Results

• Results from various data sources – are the sources providing complimentary information?

– immunization rates for DPT-3 in children are very different between official government HMIS figures versus survey with verification of children’s immunization cards

• Results at various times – are there unexpected changes over time?

Clarify Analysis Steps

• Data calculation processes

– Percentage of population reporting any risky sexual behavior in last six months

– Denominator could be (a) all; (b) all sexually active; (c) all within certain age groups; (d) all within defined “high risk” groups

• Which calculation would you recommend? Why? What are key factors to consider?

• How would you explain the significance and meaning of the different options to the program manager(s)?

Data Quality

• Data quality

– a child-immunization survey, verifying with cards, provides data for several Child Survival indicators

– you find out that the non-response rate on many key questions is 40 percent

• What are the implications for data quality of this information?

• How would you explain this to the program manager(s)?

• What options could be recommended?

Triangulation

• “Short-hand term” for data synthesis and integrated data analysis for decision making

• Originally coined by social scientists: basis for validity/truth/credibility

• Process in which multiple data sources (quantitative or qualitative) are brought together and analyzed to better understand the same public health phenomenon

Input Process Output Outcome Impact

Triangulation for HIV Programs Evaluation

HIV/STI Surveillance

Patients’ tracking/ARV resistance surveillance

Programme Monitoring

Second-Generation HIVSurveillance

Second-Generation HIVSurveillance

HMIS in PEPFAR, CDC, MAP, GFATMHMIS in PEPFAR, CDC, MAP, GFATM

Reproductive Health Survey/ CDC, MICS/ UNICEF, MACRO/ DHS and

AIDS Indicators Surveys,Service Availability Mapping (SAM),

Health Facility Surveys (HSPA), Coverage Surveys

Reproductive Health Survey/ CDC, MICS/ UNICEF, MACRO/ DHS and

AIDS Indicators Surveys,Service Availability Mapping (SAM),

Health Facility Surveys (HSPA), Coverage Surveys

Core Consensus Indicators for all Initiatives Based on United Nations General Assembly Special Session (UNGASS) and Recent Meetings

Core Consensus Indicators for all Initiatives Based on United Nations General Assembly Special Session (UNGASS) and Recent Meetings

Household Surveys, e.g. BSSFacility Surveys

Coverage Surveys

Data Useand Communication

Improve Data Use & Communication

To increase use M&E Data must:

– be manageable and timely

– be presented according to the audience’s

• interest

• capacity to understand, analyze

• time, competing demands on time

– have transparent quality

– focus on activities, results of interest

– focus on meaning and direction for action

Communicating M&E information

Often hard to communicate, because:

• Technical issues are important but boring

• Decision makers are busy and distracted

• Program managers have tunnel vision

• Program implementers are too busy

• Data experts/wizards have few communication skills

• Communication experts/wizards have few data skills

Support Services

Information systems

Quality assurance

Community input

Financing

Leadership and management

Logistics

Training and skills maintenance

Supervision

Resources

Staff

Equipment and suppliesDiagnostic

services

Pharmacy

InfrastructureTransport and communication

Client

Care Process

Triage

HistoryCounseling

Examination Treatment

Diagnosis

It’s complex!

Effective communication

• Identify your audience and their needs.

• Focus on what your audience needs to know, not what you know.

• Consider your audience’s technical knowledge, motivations, and interests. Tailor the points accordingly.

• Decide what you hope the audience will do as a result of the presentation.

• Anticipate possible questions.

• Make sure your message source is someone your audience trusts and respects.

Desired Characteristics of Information

• Needs to be simple

• Needs to be timely

• Needs to be compelling

• Needs to be consistent with stakeholder values

Tools for Facilitating Use of

M&E Results for Policy and

Program Development

Types of Tools and Resources

• Tools – printed documents– computer programs (often on CDs)– Internet-based applications

• Methods/techniques resources include– analytical techniques– case studies and best practices– data resources

Types of Tools

• Policy Development

• Program Development and Management

• Technical Resources

• Best Practices and Case Studies

• HIV/AIDS Programs

• Data Sources

Simple Tool for Analysis

Policy Development

• The Policy Circle

– The Policy Circle is a framework through which the dynamic components of policy development and implementation can be better understood and analyzed.

– The Policy Circle is used to analyze different levels of policy, including national and local policies, and sectoral and operational policies.

Program Development and Management

• The Health Manager’s Toolkit

The Health Manager's Toolkit is an electronic compendium of tools designed to assist health professionals at all levels of an organization to provide accessible high-quality and sustainable health services.

The tools have been developed by organizations working throughout the world to improve the delivery of health services. http://erc.msh.org

Technical Resources

• REPROLINE: Reproductive Health Online

Reproductive Health Online is an Internet resource for health-care professionals, providing technical information and training materials on the full range of reproductive-health topics, including family planning (contraceptive methods and other family planning topics), maternal and neonatal health, cervical cancer, and STDs.

www.reproline.jhu.edu

Technical Resources

• IGWG: Gender Publications and Tools

The Interagency Gender Working Group (IGWG) Gender Publications and Tools. The IGWG promotes gender equity within population, health, and nutrition programs with the goal of improving reproductive health/HIV/AIDS outcomes and fostering sustainable development.

www.igwg.org/pubstools.htm

Best Practices and Case Studies

• Advance Africa: Compendium of Best Practices

The Compendium of Best Practices is a searchable online database of best practices in reproductive health and family planning service delivery. It can be used by program managers who have identified gaps, needs, and opportunities in their programs. By reviewing the lessons learned from other practices, program managers can apply their own practices with greater confidence that they will lead to improved program performance.

www.advanceafrica.org/compendium

Best Practices and Case Studies

• UNAIDS: Best Practice Collection

The UNAIDS Best Practice Collection – hundreds of UNAIDS publications documenting successful approaches to particular HIV/AIDS problems and issues.

www.unaids.org

HIV/AIDS Programs

• Synergy Project: APDIME Toolkit

The Synergy Project APDIME Toolkit is a resource to support program designers and managers in HIV/AIDS prevention, care and support programming. Tools for Assessment, Planning, Design, Implementation Monitoring, and Evaluation are included.

www.synergyaids.com

HIV/AIDS Programs

• HEARD: AIDS Toolkits

The AIDS Toolkits developed by the Health Economics and HIV/AIDS Research Division, University of KwaZulu-Natal encompass a broad range of interventions for HIV/AIDS including assessment of the impact of AIDS on different sectors and ministries.

www.nu.ac.za/heard/toolkits

HIV/AIDS Programs

• UNAIDS: M&E Toolkit for National AIDS Programs

The Guide to Monitoring and Evaluation for National AIDS Programs was developed by UNAIDS in collaboration with MEASURE Evaluation and many other partner organizations. It includes a toolkit providing guidelines and instruments for the development and use of indicators for all components of a national program.

www.cpc.unc.edu/measure/publications/unaids-00.17e

Data Sources

• DHS: Statcompiler

The Demographic and Health Surveys STATcompiler builds customized tables from hundreds of DHS+ surveys and indicators. The HIV/AIDS Survey Indicators Database includes data from DHS+, MICS, BSS, and other surveys.

www.measuredhs.com

Data Sources

• U.S. Census Bureau

International Data Base: The International Data Base (IDB) is a computerized data bank containing statistical tables of demographic and socio-economic data for 227 countries and areas of the world.

www.census.gov/ipc/www/idbnew.html

Data Sources

• U.S. Census Bureau

HIV/AIDS Surveillance Data Base: The HIV/AIDS Surveillance Data Base is a compilation of information from those studies appearing in the medical and scientific literature, presented at international conferences, and appearing in the press.

www.census.gov/ipc/www/hivaidsd.html

Data Sources

About DOLPHN

 The Data Online for Population, Health and Nutrition (DOLPHN) system is an online statistical data resource containing selected current and historical country-level demographic and health-indicator data. The DOLPHN system is designed to provide users with quick and easy access to frequently-used statistics and can be helpful as both a reference and analytical tool.

www.phnip.com/dolphn

Remember!!

We learn• 1% through taste• 2% through touch• 3% through smell• 11% through hearing• 83% through seeing

We remember• 10% of what we read• 20% of what we hear• 30% of what we see• 50% of what we see and hear