Assuring Data Quality at the UNESCO Institute for Statistics

CCSA Conference on Data Quality for International Organisations

Helsinki, Finland, 6–7 May 2010

UIS Quality Framework

0. Prerequisites of Quality

1990s UNESCO data quality is compromised

Evaluation recommends:

1. Autonomous Institute, guaranteed by its own Board1999: UNESCO Institute for Statistic

2. Relocation from UNESCO Headquarters2001: move to Montreal

Consequences:

UIS can do its own resource mobilization

UIS can do its own recruitment

0. Prerequisites (cont’d)

Data quality is the first priority in the 

UIS Medium Term Strategy 2008‐2013

Internal restructuring:Centralized services for common processes

Regional teams (all 6 UN languages)

Resource mobilization very successful

Domain specialists in main fields of competence: teachers, funding for education, TVET, higher education, data sources, quality of learning

1. Assurances of Integrity

Developing staff competencies 

by investing in institutional training:technical competences

management

Recruitment through competitive process

2. Methodological Soundness

International standards:International Standard Classification of Education (ISCED) – revision in 2011Statistical Data and Metadata Exchange (SDMX)Framework for Cultural Statistics (FCS)

Development of international                                    indicators for monitoring:

EFA Global Monitoring ReportIndicators on teachersICTs in education indicators

3. Accuracy and Reliability4.  Serviceability

Assistance to countries:Workshops (regional, national)

Site visits (20+)

DQAF diagnostic studies

Implementation of Data Plans 

UIS presence in the field:Cluster and Regional Advisors

Data Plansdescribe metadata and specific instructions on how to 

complete the UIS questionnaires

Objectives:

Ensure the comparability and quality of education data

Provide essential metadata about national data

Provide a clear mapping between national data sources and international data requirements

Pre‐requisite:

High level of commitment from countries

Data Plans (cont’d)

Content:

Description of national education structure

Mapping to international standards and definitions (ISCED)

Specific instructions for completing UIS questionnaires

Recommendations for improvement from the international reporting perspective

Data Plans: Outcomes

For countriesCollaborative relationships between different reporting authorities

Clear instructions for use by current and future ministry staff

For UISImproved communicationEssential guidelines to insure data quality

For everyoneBetter quality data

Essential reference metadata

Detailed documentation regarding data sources and definitions (trained teacher in country x)

UIS Presence in the Field

Objectives:

Assist UIS National Counterparts by becoming a resource person

Supporting and facilitating the implementation of the UIS initiatives and programmes

Providing technical expertise to the UNESCO offices; 

Responding to the requests from other partners in the field (UN family, development banks, donors etc.)

UIS presence in the field

5. Accessibility

Renovation of the UIS website (2010):easier access to publication, data and metadata

improved lay‐out of texts, tables and charts

Restructuring UIS Data Centre to improve presentation (2011)

ISCED mappings published on website

UIS Quality Monitoring

Performance indicators to assess the timeliness and completeness of:

Survey processes (collect and process)

Education Database

For reporting to stakeholders and donors 

Completeness – C2. Response rate

4580.077.8Sub-Saharan Africa

01100.088.9South and West Asia

1286.282.8North America and

Western Europe

2390.287.8Latin America and the

Caribbean

2464.758.8East Asia and the Pacific

0088.988.9Central Asia

0185.781.0Central and Eastern Europe

4180.095.0Arab States

131782.280.3TOTAL

LossGain20082007

CountriesResponse rate

Completeness –C4. Coverage of key indicators

-9.166.775.8Sub-Saharan Africa

-6.882.789.5South and West Asia

-1.785.687.3North America and Western Europe

-5.266.471.6Latin America and the Caribbean

15.474.058.6East Asia and the Pacific

24.992.067.1Central Asia

2.789.386.6Central and Eastern Europe

-21.578.5100.0Arab States

0.777.176.4TOTAL

Change20082007Change between 2007 and 2008

South and W

est Asia

North A

merica and

Western Europe

LAC

Sub-Saharan Africa

East Asia and

the PacificArab States

Central A

sia

Central and Eastern

Europe

-25-20-15-10-505

10152025

Completeness – C1. Update rate

2008 in relation to target 85%

0.0

20.0

40.0

60.0

80.0

100.0

120.0

Arab States

Central and Eastern

Europe

Central A

sia

East Asia and the Pacific

Latin Am

erica and theC

aribbean

North A

merica and

Western Europe

South and West A

sia

Sub-Saharan Africa

8593.277.3Sub-Saharan Africa

8588.9100.0South and West Asia

8589.789.7North America and Western Europe

8582.995.1Latin America and the Caribbean

8576.585.3East Asia and the Pacific

85100.088.9Central Asia

8589.594.7Central and Eastern Europe

8578.9100.0Arab States

8586.389.3TOTAL

Target20082007

T2. Timeliness of data submission

Absolute change between 2007 and 2008

Arab States

Central A

sia

Sub-Saharan Africa

East Asia and

the Pacific

Latin Am

erica and the C

aribbean

North A

merica and

Western Europe

South and West

Asia

Central and Eastern

Europe

0

1

2

3

4

1.510.511.9Sub-Saharan Africa

3.710.314.0South and West Asia

1.018.519.5North America and Western Europe

0.311.011.2Latin America and the Caribbean

0.113.013.1East Asia and the Pacific

1.79.411.1Central Asia

1.217.118.3Central and Eastern Europe

1.911.012.9Arab States

1.112.713.8TOTAL

Change20082007

T4. Timeliness of internal processing

Change between 2007 and 2008

Central and Eastern Europe

Central AsiaLatin America

and the Caribbean

Sub-Saharan Africa

South and West Asia

North America and Western

Europe

East Asia andthe PacificArab

States

0

1

2

3

0.92.93.8Sub-Saharan Africa

0.02.52.5South and West Asia

0.21.41.6North America and Western Europe

1.83.65.4Latin America and the Caribbean

1.12.94.1East Asia and the Pacific

2.72.95.6Central Asia

0.11.51.6Central and Eastern Europe

0.93.94.9Arab States

1.02.83.8TOTAL

Change20082007

T6. Data freshness

Change between 2007 and 2008

C entral A sia

East A sia and

the P acif ic

A rab States

Sub-Saharan A frica

Lat in A merica

and the

C aribbean

C entral and Eastern Euro pe

N o rth A merica and

Western Euro pe

So uth and West A sia

-5

-3

-1

1

3

5

7

9

9.920.830.7Sub-Saharan Africa

1.423.925.3South and West Asia

-0.432.432.0North America and Western Europe

2.422.324.7Latin America and the Caribbean

1.026.927.9East Asia and the Pacific

8.217.625.8Central Asia

1.327.829.1Central and Eastern Europe

2.323.325.6Arab States

3.424.728.1TOTAL

Change20082007

Thank you!

For more information:Hendrik van der Pol  

h.van‐der‐pol@uis.unesco.org

Anuja Singha.singh@uis.unesco.org