Conference on Data Quality for International Organisations, Newport, 27-28- April 2006 1 Assessment of statistical data quality: The example of the Occupational

Conference on Data Quality for International Organisations, Newport, 27-28- April 2006Conference on Data Quality for International Organisations, Newport, 27-28- April 200611

Assessment of statistical data quality: Assessment of statistical data quality:

The example of the Occupational The example of the Occupational Wages part of the ILO October InquiryWages part of the ILO October Inquiry


ILO October InquiryILO October Inquiry

Worldwide Survey of:Worldwide Survey of:

Wages and hours of workWages and hours of work- 159 occupations- 159 occupations- 49 industry groups- 49 industry groups

Retail pricesRetail prices- 93 food items- 93 food items

Reference to October of each yearReference to October of each year


Problems of data quality and comparabilityProblems of data quality and comparability

Unavoidable differences between:Unavoidable differences between:

Concepts usedConcepts used

Specifications of occupations and itemsSpecifications of occupations and items

Reference periodsReference periods

Types of sourcesTypes of sources

Methods of data collectionMethods of data collection


Steps taken to ensure data qualitySteps taken to ensure data quality

Detailed instructions for respondentsDetailed instructions for respondents

Detailed descriptions of occupations and Detailed descriptions of occupations and industrial activities according to ISCO and ISICindustrial activities according to ISCO and ISIC

Detailed descriptions of food items including Detailed descriptions of food items including pricing unitspricing units

Questionnaire includes information on coverage, Questionnaire includes information on coverage, reference period, currency, source of the data etc.reference period, currency, source of the data etc.

Pre-filled data for the previous yearPre-filled data for the previous year


Quality assessment Quality assessment

Regular checks for deviations and vague trendsRegular checks for deviations and vague trends

Reporting agencies are asked to verifyReporting agencies are asked to verify

Discrepancies due to methodological revisions, Discrepancies due to methodological revisions, sample changes and different coveragesample changes and different coverage

Quality of data was assessed in 2001 asQuality of data was assessed in 2001 as- excellent for 32 %- excellent for 32 %

- acceptable/good for 53 %- acceptable/good for 53 %

- poor for 14 %- poor for 14 %

- not acceptable for one data set- not acceptable for one data set


Next stepsNext steps

Revision and expansion of the October Inquiry Revision and expansion of the October Inquiry

- changes in labour force structures- changes in labour force structures

- hourly wages/salary- hourly wages/salary

Improved collection of metadataImproved collection of metadata

Computerized validation checks in the databaseComputerized validation checks in the database

ILO experience with gathering ILO experience with gathering and disseminatingand disseminating

meta-data on household income meta-data on household income and expenditure statisticsand expenditure statistics

ILO Bureau of Statistics, GenevaILO Bureau of Statistics, Geneva

Volume 6 of the ILO publications on Sources and MethodsVolume 6 of the ILO publications on Sources and Methods


Previous PracticePrevious Practice

Method 1:Method 1:

– Questionnaire with open response (in English, French, Spanish) orQuestionnaire with open response (in English, French, Spanish) or

– Extract metadata from national publications, other documentsExtract metadata from national publications, other documents

– Then manual preparation of description, and translate into other two Then manual preparation of description, and translate into other two languageslanguages

Method 2Method 2

– Referring previous descriptions back to countries for updating, then Referring previous descriptions back to countries for updating, then translate revision into other two languagestranslate revision into other two languages


Problems with previous methodProblems with previous method

Countries were not always be sure of the level of Countries were not always be sure of the level of detail to provide, requiring follow-updetail to provide, requiring follow-up

Extensive effort to prepare standardised Extensive effort to prepare standardised description of metadatadescription of metadata

Extensive translation of results (about $50,000 Extensive translation of results (about $50,000 each publication)each publication)


New method developed in 2003New method developed in 2003

Questionnaire with tick boxes offering choice of Questionnaire with tick boxes offering choice of answers (plus “Other, specify”)answers (plus “Other, specify”)

Tested and revisedTested and revised

Translated into French and SpanishTranslated into French and Spanish

Computer programmes developed to:Computer programmes developed to:– Enter data into a databaseEnter data into a database– Edit the input Edit the input – Produce analyses of database (cross-tabulations)Produce analyses of database (cross-tabulations)– Produce HTML output in three languages in standard Produce HTML output in three languages in standard

formatformat


Advantages of new systemAdvantages of new system

Is believed to reduce reporting burden for countries Is believed to reduce reporting burden for countries – Tick boxes are faster and easier to completeTick boxes are faster and easier to complete– Gives guidance on the detail required in the response)Gives guidance on the detail required in the response)

Provides a database that can be analysed and cross-Provides a database that can be analysed and cross-tabulated (not previously available)tabulated (not previously available)

Reduces translationReduces translation

Computer development is re-useable to some extent Computer development is re-useable to some extent – has already been adapted for a metadata system on has already been adapted for a metadata system on statistics on

the employment situation of persons with disabilities


Disadvantages of new systemDisadvantages of new system

Requires more care in questionnaire design Requires more care in questionnaire design and extensive knowledge of possible and extensive knowledge of possible alternativesalternatives

Requires more computer inputRequires more computer input

Documents

Conference on Data Quality for International Organisations, Newport, 27-28- April 2006 1 Assessment of statistical data quality: The example of the Occupational