KEY TOPICS FOR CHILD INDICATOR REPORTS

DR. WILLIAM P. O’HARE

GOAL OF SESSION

• COVER SEVERAL KEY ASPECTS OF PRODUCING A CHILD INDICATORS REPORT

TOPICS IN SESSION

• ORGANIZING PROJECT

• GETTING DATA

• PRESENTING DATA– ANALYSIS, PAGE LAYOUTS, ETC

• DISSEMINATING PRODUCT

• EVALUATION

MOST SUCCESSFUL PROJECTS COMBINE 3 KEY ELEMENTS

1. GOOD DATA

2. GOOD COMMUNICATION

3. SUSTAINED EFFORT

(POLICY ANALYSIS???)

WHERE IS THE MONEY COMING FROM?

• HOW MUCH FOR HOW LONG– GOVERNMENT– ACADEMICS – NON-GOVERNMENT

• FOUNDATIONS• RICH INDIVIDUALS• INTERNATIONAL GROUPS

ORGANIZING PROJECT

2 FUNDAMENTAL APPROACHES TO GETTING STARTED

1. DETERMINE WHAT INDICATORS YOU WANT THEN GO LOOK FOR THEM

2. DETERMINE WHAT DATA IS AVAILABLE AND CHOSE WHICH MEASURES YOU WANT TO USE

LEAD ORGANIZATION

– GOVERNMENT

– NON-GOVERNMENT ORGANIZATION (NGOs) OR “NON-PROFITS” OR “CIVIL SOCIETY”

– ACADEMIC INSTITUTION

– COALITION OR PARTNERSHIP

VALUE OF COMMITTEES

• DATA COMMITTEE

• PUBLICATIONS COMMITTEE

• DISSEMINATION COMMITTEE

• POLICY COMMITTEE

FUNDAMENTAL DECISIONS

• DO YOU PLAN TO PRODUCE ONE REPORT OR SERIES OR REPORTS?

– SIMILAR REPORTS OVER TIME (BEST APPROACH)

– REPORTS ON DIFFERENT TOPICS

MULTIPLE REPORTS AND BRANDING

• MAIN AUDIENCE?

WHO IS YOUR MAIN AUDIENCE?

• SCHOLARS/SCIENTISTS– MORE LIKELY TO BE INTERESTED IN TOPIC AND WILLING

TO SPEND MORE TIME TO ABSORB REPORT BUT USUALLY NOT A POWERFUL GROUP

• GOVERNMENT OFFICIALS– POWERFUL GROUP IN SHAPING POLICIES BUT

• HAVE MANY COMPETING INTERESTS, CAN’T SPEND MUCH TIME ON ONE REPORT,

• MAY BRING IDEALOGICAL PERSPECTIVE,• LEVEL OF INTEREST MAY CHANGE WITH NEW ELECTIONS

• PUBLIC - MEDIA/PRESS– IMPORTANT IN REACHING BROADER PUBLIC BUT MUST

BE A GOOD STORY (USE OF RANKINGS)

THEORETICAL FOUNDATIONS

• PROS OF CHOSING A THEORETICAL ORIENTATION– PROS – HELPS GUIDE SELECTION OF

INDICATORS– PROS – BUILDS ON SCIENTIFIC

KNOWLEDGE

- MORE APPEALING TO ACADEMIC/SCIENTISTS AUDIENCE

CONS OF CHOSING A THEORETICAL PERSPECTIVE• DIFFICULT TO GET AGREEMENT

ACROSS DISCIPLINES– SOCIOLOGISTS, PSYCHOLOGISTS, POLICY ANALYSTS

• MAY SOUND ELITIST TO PUBLIC

• MAY CONSTRAIN DATA SELECTION

VARIOUS THEORETICAL PERSPECTIVES

• A CHILDREN’S RIGHTS PERSPECTIVE

• A SOCIOLOGY OF CHILDHOOD PERSPECTIVE

• ECOLOGICAL PERSPECTIVE

• WELL-BEING VERSUS WELL-BECOMING

QUESTIONS?

GETTING DATA

WHAT TO MEASURE

• WHO WILL DECIDE?– THE VALUE OF COMMITTEES

• THE UNIT OF OBSERVATION– CHILD– FAMILY– GEOGRAPHIC UNIT LIKE A STATE

DATA COLLECTION

• ABILITY TO COLLECT ORIGIN DATA

– ALLOWS ONE TO MEASURE EXACTLY WHAT YOU WANT TO MEASURE

– VERY COSTLY

– IF REPORT IS MEANT TO BE UPDATED REGULARLY, MUST PLAN FOR ON-GOING DATA COLLECTION – NOT JUST A ONE-TIME COST

DATA MOSTLY FROM TWO SOURCES

1. SURVEYS INCLUDING CENSUS– DEMOGRAPHIC AND HEALTH SURVEY– MULTIPLE INDICATOR CLUSTER SURVEY

2. ADMINISTRATIVE DATA– BIRTH AND DEATH DATA– SCHOOL ENROLLMENT

SURVEY DATA

• QUESTIONS OF COST AND QUALITY

– RESPONSE RATES (BIAS)– SAMPLE SIZE NEEDED

• GEOGRAPHIC AREAS• SPECIAL GROUPS

– GENDER– ETHNIC GROUP– AGE

ADMINISTRATIVE DATA VERSUS SURVEY DATA

• ADMINISTRATIVE DATA– MUST BE COLLECTED SAME WAY EVERYWHERE

IN THE COUNTY• DEFINITIONS MAY VARY ACROSS PROVINCES OR STATES• DILIGENCE MAY VARY ACROSS AREAS AND AGENCIES• FORM OF DATA STORAGE (PAPER VERSUS COMPUTER)

MAY VARY

– MUST GET ACCESS FROM GOVENMENT TO USE

– OFTEN NOT EXACTLY WHAT YOU WANT TO MEASURE

QUESTIONS OF CREDIBILITY OF DATA SOURCE(S)

• OFTEN MORE THAN JUST SCIENTIFIC QUESTION

• CREDIBILITY MAY DEPEND ON MESSENGER AND/OR AUDIENCE

• IF DATA IS NOT CREDIBLE, REPORT IS OF LITTLE VALUE

• GOVERNMENT VERSUS ACADEMIC VERUS NGO (LOCAL OR INTERNATIONAL)

CRITERIA FOR GOOD INDICATORS

1. The statistical indicator must be from a reliable source.

2. The statistical indicator must be available and consistent over time.

3. The statistical indicator must be available and consistent for all subnational areas (provinces/states/cities).

4. The statistical indicator should reflect a salient outcome or measure of well-being.

5. The statistical indicator must be easily understandable to the public.

6. The statistical indicators must have a relatively unambiguous interpretation.

7. There should be a high probability that the measure will continue to be produced in the near future.

QUESTIONS?

PRESENTING DATA

PRINTED REPORT VERSUS ON-LINE REPORT VERSUS ONLINE DATA (MIX)

• PROS AND CONS FOR DIFFERENT AUDIENCES– PRINTED REPORTS ARE EXPENSIVE BUT TEND

TO REACH POWERFUL PEOPLE BETTER– ONLINE REPORTS ALLOWS REPORT TO BE

DISSEMINATED MORE WIDELY BUT NOT LIKLEY TO BE READ BY POWERFUL PEOPLE

– ON-LINE DATA ONLY MAY LEAVE INTERPRETATION UP TO OTHERS.

POWER OF COMPARISONS

• WITHOUT SOME POINT OF COMPARISON, MOST DATA WILL NOT BE MEANINGFUL TO PUBLIC AUDIENCE

• VERY POWERFUL WAY TO CONVEY MEANING TO PUBLIC AUDIENCE

• REQUIRES MORE (RELIABLE) DATA

KINDS OF COMPARISONS

• COMPARISONS ACROSS GEOGRAPHIC AREAS LIKE PROVINCES

• COMPARISONS OVER TIME = MONITORING• COMPARISONS AMONG GROUPS

– MALES COMPARED TO FEMALES– INFANTS COMPARED TO TEENS– VARIOUS ETHNIC GROUPS

• ALL OF THE ABOVE

USE OF STATISTICAL SIGNIFICANCE TESTS

• CLASH OF SCIENCE AND COMMUNICATION

• WIDELY USED BY SCIENTISTS TO TEST DIFFERENCES

• NOT WIDELY UNDERSTOOD BY PUBLIC

• HARD TO USE IN DATABOOK CONTEXT WITH

– MANY INDICATORS

– FOR MANY AREAS

– OVER TIME

DESCRIPTION VERSUS ANALYSIS

“WHAT IS”

VERSUS

“WHY IT IS”

DESCRIPTION

PROS• SIMPLE PRESENTATION• LESS COMPLICATED STORY• LESS THREATENING TO THOSE IN POWER• BUILDS CREDIBILITY• GOOD FOR STARTING DIALOGUE

CONSHARDER TO CONNECT WITH POLICIES

LEAVES INTERPRETATION UP TO READER

MAY IGNORE MORE COMPLEX SCIENTIFIC EVIDENCE

ANALYSIS

PRO– EASIER TO LINK TO POLICIES– DOESN’T LEAVE INTERPRETATION UP TO

READER

CON

- MORE DIFFICULT TO COMMUNICATE TO PUBLIC AUDIENCE

-MORE THREATENING TO SOME

AGENCIES/ REGIMES/ IDEOLOGIES

ONE APPROACH

• START WITH DESCRIPTION TO BUILD CREDIBILITY THEN SLOWLY MOVE TO ANALYSIS OVER TIME (YEARS)

NUMBER OF INDICATORS TO INCLUDE

• BALANCE OF BEING COMPREHENSIVE VERSUS PARSIMONIOUS – KEEP IT SIMPLE

• WHAT DATA ARE AVAILABLE?

• WHAT DOMAINS TO BE REFLECTED

• NEED TO BALANCE DOMAINS– AVAILABILITY OF HEALTH DATA VERSUS

COMMUNITY DATA

DOMAINS USED IN VARIOUS STUDIES

UNICEF Innocenti Research Centre AMERICA’S CHILDREN

FOUNDATION FOR CHILD DEVELOPMENT

1.MATERIAL SITUATION1.FAMILY AND SOCIAL ENVIRONMENT

1. FAMILY ECONOMIC WELL-BEING

2.HEALTH AND SAFETY2.ECONOMIC CIRCUMSTANCES 2. HEALTH

3.EDUCATION 3.HEALTH CARE3. SAFETY/BEHAVIORAL CONCERNS

4.CHILDREN’S RELATIONSHIPS

4.PHYSICAL ENVIRONMENT AND SAFETY 4. EDUCATIONAL ATTAINMENT

5.SUBJECTIVE WELL-BEING 5.BEHAVIOR

5. COMMUNITY CONNECTEDNESS

6.BEHAVIOUR AND LIFESTYLES 6.EDUCATION 6. SOCIAL RELATIONSHIPS

7.HEALTH 7. EMOTIONAL/SPIRITUAL WELL-BEING

CALCULATION OF AN INDEX OF OVERALL CHILD WELL-BEING

• POWERFUL WAY TO CONVEY COMPLEX DATA TO THE PUBLIC

• ALLOWS OVERALL RANKING AMONG AREAS (COUNTRIES / PROVINCES)

• ALLOWS EASY TRACKING OF TRENDS OVER TIME

INDEX CALCULATION IS FROUGHT WITH METHODOLOGICAL ISSUES.

• IS QUALITY OF DATA THE SAME ACROSS ALL DOMAINS?

• DOES EACH DOMAIN NEED TO HAVE THE SAME NUMBER OF INDICATORS?

• SHOULD INDICATORS OR DOMAINS BE EQUALLY WEIGHTED?

• DOES INDEX HIDE IMPORTANT DETAILS (ONE EXTREME INDICATOR DRIVES THE WHOLE INDEX)?

DATA SELECTION CONTINUED

• POSTIVE AS WELL AS NEGATIVE MEASURES?

• SURVIVAL AND BEYOND?

• WELL-BEING OR WELL-BECOMING

RESIST TEMPTATION TO INCLUDE EVERYTHING

“KEEP IT SIMPLE” • TOO MANY INDICATORS BECOME

CONFUSING TO PUBLIC AND POLICY MAKERS

• TOO MANY INDICATORS MAKE PRESENTATION DIFFICULT

• MAY BE DIFFERENT FOR AN ACADEMIC AUDIENCE

• USE EXECUTIVE SUMMARY/ OVERVIEW REPORT

QUESTIONS?

DISSEMINATING THE PRODUCT(S)

DISSEMINATION• REPORT/DATA IS OF LITTLE VALUE UNLESS IT IS

DISSEMINATED WIDELY• WORKING WITH PUBLIC MEDIA

– PRINT– TV AND RADIO– INTERNET

– IMPORTANCE OF PRESS RELEASE AND ADVANCE WORK PRIOR TO RELEASE

• WORKING WITH GOVERNMENT AGENCIES AND NGOS – STAGING RELEASE OF REPORT– PROVIDING ADVANCE COPIES– GETTING TRUSTED VOICES TO PARTICIPATE IN RELEASE

KIDS COUNT DATA BOOK PRESS PACKAGE

– DATA BOOK– ESSAY AND DATA BRIEF– DATA WHEEL– FACT SHEET– PRESS RELEASE– POST CARD ON WEBSITE– CUSTOMIZED FOR EACH STATE

STATE DATA

STATE KIDS COUNT RELEASE

EXAMPLE OF IMPORTANCE OF MEDIA ATTENTION

Overall child well-beingCountry Average

rank Material Situation

Health and

Safety

Education Children’s relationships

Subjective Well-being

Behaviour and

lifestyles

Netherlands 4.2 10 2 6 3 1 3 Sweden 5.0 1 1 5 15 7 1 Finland 7.3 3 3 4 17 11 6 Spain 8.0 12 5 16 8 2 5 Switzerland 8.0 5 9 14 4 6 10 Denmark 8.2 4 4 8 9 12 12 Norway 8.3 2 8 9 10 8 13 Belgium 10.0 7 12 1 5 16 19 Italy 10.0 14 6 20 1 10 9 Ireland 10.2 19 19 7 7 5 4 Germany 11.2 13 11 10 13 9 11 Greece 11.8 15 18 17 11 3 7 Canada 12.0 6 14 2 18 15 17 France 12.5 9 7 15 12 18 14 Poland 12.5 21 16 3 14 19 2 Czech Republic 12.7 11 10 11 19 17 8 Austria 13.7 8 20 19 16 4 15 Portugal 14.0 16 15 21 2 14 16 Hungary 14.5 20 17 13 6 13 18 United States 18.0 17 21 12 20 20 United Kingdom 18.5 18 13 18 21 20 21

DIFFERENCE IN ATTENTION IN UK AND U.S.A.

• IN U.K. A LOT OF PUBLIC ATTENTION

-FRONT PAGE NEWSPAPER STORIES SUSTAINED COVERAGE

-RESPONSE FROM TONY BLAIR

- HEARINGS HELD• VERY LITTLE ATTENTION IN U.S

- ONLY A FEW NEWSTORIES- NO GOVERNMENT REACTION

REPORTING RESULTS

• WRITTEN REPORT VERSUS ON-LINE DATA

• LARGE REPORT WITH SMALL SUMMARY

• HOW MUCH TEXT AND HOW MUCH DATA

• HOW MUCH INTERPRETATION VERUS SIMPLE PRESENTATION OF DATA

• USE OF GRAPHIC PRESENTATION

QUESTIONS?

EVALUATING EFFORT

• FORMAL EVALUATIONS SELDOM DONE BECAUSE THEY ARE EXPENSIVE

– DOMAINS OF EVALUATION• MEDIA COVERAGE OF REPORT• IMPACT ON KEY AUDIENCES (GOVERNMENT,

SCHOLARS, ETC)• CHANGED POLICIES• IMPROVED LIVES

• KIDS COUNT REPORT AND THE MEDIA

Newspaper Articles and Readership 2001 to 2006 based on

Release of Data BookYear 2006 2005 2004 2003 2002 2001

            

Total number of articles

1,100 981 1,160 917 753 1,108

             

Total readership

59 60 67 46 41 49

(in millions)  

Television Coverage of KIDS COUNT Data Book Release

  2006 2005 2004 2003 2002

Number of Airings

548 560 509 625 462

           

Number of Stations

252 236 245 294 221

           

Number of Viewers (in millions)

15 19 16.3 21.6 17.7

REACHING KEY AUDIENCES

PERCENT SAYING THEY USE KIDS COUNT

Group

Percent Saying They Use KIDS

COUNT *Congressional Aides 36State Legislators 55State Legislative Staff 73State Legislative Leaders 73Business Leaders 26County officials 27Data Users 66*use a little or a lot as opposed to not at all

Perceived Impact of KIDS COUNT  Percent Saying KIDS COUNT had:  

 

a major or moderate impact on public awareness of children's issues in your state

a major or moderate impact on public policy in your state 

State Legislators 64% 60%  

State Legislative Staff 60% 46%  

State Legislative Leaders 72%   69%

Business Leaders 23%   25%

County Officials 46%   39%

Data users and congressional aides were not asked this set of questions

ROLE OF CHILDREN

• IN DETERMINING WHAT DATA TO COLLECT

• IN COLLECTING DATA

• IN INTERPRETING THE DATA

• IN DISSEMINATION– PRESS CONFERENCE ETC

– SOUTH AFRICA VERUS UNITED STATES

QUESTIONS?