Working with real world Data: The importance of diving deeper

into reports & the data behind them

Brief by Robert Sentamu

Wilsken Agencies Ltd

Data--information--knowledge--wisdom

Information increasingly forms the basis of competitive advantage, and organizations must explicitly articulate and define the role information will play in the design and execution of their competitive strategies.

However, we must also remember that information is not free and does not always flow freely and in this regard information is often open to manipulation to sometimes achieve dubious ends.

We therefore must conduct due diligence on the information we use we are

to…

Be accountable

Be transparent

Support our decisions and policy development

Provide information to the public

Provide great value to us as public figures

How to conduct due diligence on data and information

Key parameter/questions to consider

Significance

Representation

Methodology

Context

Analysis

Implementer (who are the actors?)

Where do you stand in all this? (Triangulation)

What is polling?

A count

A census

A vote

A question

A survey

A sample

Question: what is common to each of the above?

Therefore public opinion polling is:

A count/survey/sample/census of public

opinion regarding various issues or

questions

Why do we do public opinion research?

To replace our assumptions about what the public thinks with facts.

To contribute to the debate on important public issues.

To contribute to decision making about policy priorities and positions.

To provide input to strategies for addressing Peoples’ needs.

To provide indicators for evaluation and impact assessment.

Alternative ways/Approaches employed in conducting

public opinion research

Qualitative methodologies e.g. Focus Group

discussions, workshops, in-depth one-on-one

interviews, rapid appraisal assessments, desk

assessments, literature reviews etc.

Quantitative methodologies such as:

representative sample surveys, Census counts,

polls etc

Highlights of the qualitative approach

It is structured mainly to probe and understand depth and the WHY for opinions, perceptions and attitudes held by the public.

Major advantage: enriches depth of understanding and reasons behind public opinion

Major disadvantage: When conducted in isolation, results cannot be quantified and therefore projected on the main stream population

Highlights of the quantitative approach

It structured mainly to measure, gauge and determine public opinion

Major advantage: Methodology allows you to fairly accurately estimate/rank importance and determine prioritization of issues by the public

Major disadvantage: Methodologies are not very good at investigating depth of opinion

The quantitative approach of

conducting public opinion research

Highlights of the quantitative methodology

Revolves a round surveying a representative sample of the public or the main research/survey target population

Utilizes mainly a survey instrument (a structured questionnaire) made up of various question items that measure “reliably” constructs of the issues being investigated in a valid manner

What is a representative sample of the main

research/survey target population?

A sample of the main population with the same

demographic characteristics (distribution and

variations) as the main survey target population

(Universe)

It is the minimum number of cases required to

allow statistical inferences of the diversity

existing in the main survey target population at a

set margin of error and degree of confidence in

the results

Analogy

Surveying a representative sample of the main target population is like testing for salt in a dish of soup; you do not have to scoop whole plate to determine the salt concentration.

Likewise, surveying a representative sample of the main survey target population is preceded by sampling the various homogeneous groupings (‘dishes of soups’) Making up the diversity of the population

Sampling

How do you determine the various homogeneous (diversity) groups

existing in the main target population?

Establish how opinion, perceptions & attitudes regarding the survey investigation issue (s) may vary by the demographic variables of the main target population such as: Urban-rural divide, region, gender, age, education etc.

Then employ rigorous scientifically proven random sampling procedures to selected representative sample from each of the identified demographic segments by which opinion regards the survey investigation issue varies

Sampling

The Idea behind sampling is to maintain the principle of randomness throughout the process of selecting sample areas to selection of the final individual survey respondent

The principle of randomness entails that all elements of the demographic segments of the target survey population should have an equal and proportionate probability of being selected into the sample

NB: There exists time tested scientifically proved procedures for doing representative random sampling (refer to: sample surveys with special reference to Africa by: prof. Ben Kiregyera)

Reliability of polling results

Reliability of quantitative public opinion research (poll) is dependent on the following:

1. Employment of appropriate sampling procedures

2. Strict field implementation of the designed sampling procedures

3. Having a valid instrument (questionnaire) measuring the investigation issues consistently and reliably

4. Ensuring that any translations of the instrument retains the meaning and structure of the origin question

5. Ensuring that field implementation of the questionnaire uses the English version or the translation and not paraphrases of the interviewers

How do validate reliability of public

opinion results

Match demographic distribution of results of survey/poll with census data of the main survey target population

Check for scientific rigor of sampling procedures employed

Check for internal consistence of the results vis-a-vis your individual perceptions of extent of issues measured

Compare consistence or divergence of current poll results with results from previous polls similar polls

Check for facial validity of the questions asked to establish if they measure the issues intended to be measured

Assess degree of accuracy in predicting outcomes

Do’s and Don’ts of utilizing Public opinion poll results

Read the entire report and not just part of the report. There is no such thing as an inaccurate poll before you read it.

Refrain from making up your mind before reading the entire poll report (always keep an open mind). There is often something for every one

Consider the results carefully even if you do not like or disagree with them. What if they are accurate, you will miss out on acquiring an informed position

Please remember that there is no such thing as useless information, even knowing about inaccurate information is good because it elevates your informed position

Do’s and Don’ts final point

You cannot dispute a poll by just

disagreeing with the results; if you want to,

do it on scientific grounds or provide

alternative prove in the same scientific

vein and rigor.

Group assignments

Examine AfroBarometer Uganda R5

results handed out and assess the validity

and reliability of the results

Thank you for being good participants