Census Data Archiving Seminar 20th – 23rd September, 2011.

Ethopia

The Gambia’s ExperienceSession 10: Archiving.

Session 10 – Country case studies, on Census Da Census data.

The Gambia Bureau Of statistics. (GBOS)

What is A Population Census?

• A Population Census is the official enumeration of persons in a country at a specific time.

• This enumeration also implies the collection, compilation, evaluation, analysis and publication of demographic, social and economic statistics relating to the population.

Objectives of the census• The objectives of the Census are to count all the people in the country and

to provide the Government with their number in each Local Government Area and District, by age, sex and several other characteristics.

• These figures are required for various aspects of economic and development planning. The ultimate aim of such planning is to provide a better way of life for the people of the Gambia, and to conquer what have been called the Five Giants: Disease, Ignorance, Squalor, Idleness and Want.

• Census Data Archiving:-

• Provides a central access point for statistical information of a country. With an increase demand for census microdata from developing countries particularly with emphasis on objective monitoring and evaluation of poverty reduction policies as well as tracking progress on the MDGs, comes the need to archive census data to allow easy access. The tools stores (Organise) present data about Indicators and variables collected during enumeration.

Geographic requirements

• The Geographic Area of the census, consists of :-

• Country : Level 1 The Gambia• Regions : Level 2 LGA’s• Provinces : Level 3 Districts• Villages : Level 4

Settlements?

Databases

• Censuses: • 1963 Not available• 1973 Hard copies• 1983 Archived on Tape• 1993 Archived on Tool kit and other formats• 2003 Archived on Tool kit and other formats

Getting Started

• Census Archiving, consists of the following elements:

–Databases–Questionaires– manuals– Programs– Sample frames– Urban / Rural definition– All other technical documents involve

during the census process

The Enumeration & Receiving Of Questionaires from the field”

• The training of enumerators and supervisors• Deploy the selected enumerators and supervisors to

the field,and start pilot census enumeration, coding , data entry and produce tables. Improve on your errors

• Start Actual Census enumeration• Collect all completed EAs from the DCO’s• Scrutinise the filled questionaire at random• Count number of EAs for all LGAs from returning DCOs• Collect the quick count form (GPC1) as summaries of

the enumerated population at settlement level. Provisional census results were compiled. When the preliminary figures were reveiwed and accepted, they were released for publication.

•Archived census questionaires by LGAs on the shelves or racks, in the archiving room.•Place an inventory form on each EA, labeled clearly•Enter date and sign whenever, the EA Folder is move from the rack so as to keep track of the history of the EA Movement from the archive room. •IF EA folder is return to the archive room it should be dated and sign (indicating return to rack).

Coding & Data Entry Processing

• Received EAs from the Questionaire Administrators from the archiving room, enter date received and signed

• Coders will be asign ID numbers to keep track of coders errors during EA coding

• After coding is finished return Folder to archive room, dated and signed on the inventory form as to keep history and movement of the folder.

• Data entry clerks to received EAs (folders) from the QA dated and sign as to start data entry keying. To return the EA to storage room dated and sign after EA is completed, to keep track of the EA movement.

• The process continues until all the EAs are coded and data entry completed.

Data Cleaning & Tabulation

• Clean the datafiles till clean by EA’s

• Concatenate data again

• Draw your format tables and test all formats to be • consistent with the table headings

• Tabulation & Analysis

• Produce the tables• Hand over the tables to the Statisticians to check the

consistency. Correct errors till complete.

Report writing

• Topics will be divided to report writers

• Each report writer will focus on his/her area• Census reports are as follows:• Spatial distribution -Directory of Settlement• Fertility -Economic Characteristics• Mortality -Settlement Profile• Migration -Census Atlas• Elderly -Education• Disability -Gender• Housing Characteristics -Methodology• Building -etc

Archiving Documents Preparation

• Convert all documents to PDF Format:

• Divide it into 5 Groups:-• Data folder, Docs, Misc, Programs, and Technical

Documents

Folders• Data Folder: Census data in Spss, Cspro or

other formats.

• Programs: Edits, Sums, Ap, Tab and Cn.

• Docs: Fmts, Dcf, Manuals, Questionaires, Batch ID, etc

• Technical Docs: Programs, Sample frame, etc

Microdata Management Toolkit

• Start by clicking the Metadata Editor• Click on the census name eg.

Gam_census_household• The study content will appear: i.e.• Document Description, Study Description,

Datasets and External Resources

Document Description• Study Title• POPULATION AND HOUSING CENSUS 2003

• Metadata Producer

• GAMBIA BUREAU OF STATISTICS• LOLLEY KAH JALLOW• SAI JASSEH

• Date of Production• 2007-02-27

• DDI Document Version • VERSION 1, 2003

• DDI Document ID Number• DDI-GMB-2003-001

• Under the document description you have the Metadata preparation

Study Description:

• Identification, Version, Overview, Scope,• Coverage, Producers and Sponsors, Data collection, Data Processing, Data Access, • Disclaimer and copyright and Contacts

The description will be filled on all the above.

Identification• Title

• POPULATION AND HOUSING CENSUS 2003

• Subtitle

• POP CEN 2003

• Abbreviation

• POPCEN2003

• Study Type

• Population and Housing Census [hh/popcen]

• Series Information

• 1.1 Background of the Census• A population census is defined as the total process of collecting, compiling, evaluating, analysing and publishing or

otherwise disseminating demographic, economic and • social data pertaining, at a specified time, to all persons in a country or in a well-delimited part of the country. A

housing census is the total process of collecting, • compiling, evaluating, analysing and publishing or otherwise disseminating statistical data pertaining at a specified

time, to all living quarters and occupants thereof in a • country or in a well-delimited part of a country. The 2003 Population and Housing Census of The Gambia was in

accordance with these definitions. Further, it contained • all the essential features of a census, namely individual enumeration, universality within the country and simultaneity.

Version • Description

• Version 1: Final datafile and census tables, but the final different reports of the 14 analyst not included. (not ready).

• Production Date

• 2003-04-15

• Notes

• VERSION 1 , FINAL DATAFILE AND CENSUS TABLES. BUT THE FINAL 14 REPORTS NOT INCLUDED. (NOT READY)

Overview:• Abstract

• POPULATION AND HOUSING CENSUS 2003. CENSUSES ARE CONDUCTED EVERY TEN YEARS. TO COUNT THE ACTUAL POPULATION OF THE COUNTRY. IT IS LATER USED FOR PLANNING PURPOSES AND DECISION MAKING.

• THE CENSUS WAS CONDUCTED 15TH APRIL 2003. THIS IS THE FIFTH CENSUS CONDUCTED IN THE

GAMBIA, FUNDED BY GAMBIA GOVERNMENT • THROUGH WORLD BANK. OTHER DONOR AGENCIES ALSO PROVIDE EQUIPMENT, E.G UNFPA.

• Kind of Data

• Census/enumeration data [cen]

• Unit of Analysis

• HOUSEHOLDS • INDIVIDUALS •

Scope

Description of Scope

• HOUSEHOLDS

• INDIVIDUALS

• BUILDINGS

Coverage

Geographic Coverage

– WHOLE COUNTRY

– Universe

– COVERS EVERYBODY

Producers & Sponsors• Primary Investigator

• GAMBIA BUREAU OF STATISTICS•

• Other Producers

• GAMBIA GOVERNMENT• WORLD BANK

• Funding

• GAMBIA GOVERNMENT • WORLD BANK

• Other Acknowledgments

• ALIEU S. M. NDOW• LOLLEY KAH JALLOW• GAMBIA GOVERNMENT•

Data Collection Dates of Collection

2003-04-15

Time Periods

2003-04-15

Mode of Data Collection

Face-to-face [f2f]

Notes on Data Collection

FEW RESPONDENTS REFUSED TO BE INTERVIEWED. DURATION OF INTERVIEW THREE WEEKS AND WE HAD, FEW CALL BACKS.

Questionnaires

FORM A- NORMAL HOUSEHOLD, FORM B- INSTITUTIONAL POPULATION, FORM C- BUILDING AND COMPOUND PARTICULARS AND FORM G- GRADUATE CARD.

Supervision

THEY WERE SUPERVISED BASED ON THE CENSUS METHODOLOGY. GPC 6 WAS A FORM FILLED BY SUPERVISORS FOR SPOT CHECKS IN THE COMPOUNDS. TO CHECKED THE LEVEL OF GOOD

ENUMERATION.

Data Processing

• Data Editing

• DATA CLEANING STARTED DURING DATA ENTRY (VERIFICATION OF 100 PERCENT) STRUCTURE CHECKS, RANGE CHECKS AND THE VALIDATION ON

VARIABLES. LOGIC CONTROL AND CONCOR EDIT CHECKS.

Access• Access Authority

• GAMBIA BUREAU OF STATISTICS•

• Confidentiality:

• SORT APPROVAL FROM GOVERNMENT AND GBOS BEFORE DIESEMINATING THE DATA.• • THE DATA SHALL NOT BE USED FOR ANY OTHER PURPOSE THAN THE ABOVE SPECIFIED REQUEST.• • ALL COST INVOLVED IN MAKING THE DATA AVAILABLE SHALL BE THE RESPONSIBILITY OF THE REQUESTER.

• Access Conditions:

• THIS DATA SHOULD NOT BE GIVEN FOR COMMERCIAL CONDITIONS ONLY FOR RESEARCH PURPOSES.

• Citation Requirement:

• POPULATION AND HOUSING CENSUS, 2003. • GAMBIA BUREAU OF STATISTICS

Disclaimer and copyright:

• Disclaimer

• THE GAMBIA BUREAU OF STATISTICS (GBOS) PROVIDES THIS DATA TO EXTERNAL USERS WITHOUT ANY RESPONSIBILITY IMPLIED. GBOS ACCEPTS

• NO RESPONSIBILITY FOR THE RESULTS FROM THE USE OF THIS DATA.

• Copyright

• USERS HAVE NO RIGHT TO COPY THE CD WITHOUT THE CONCENTS OF GBOS.

Contacts

• Contact Persons

• ALIEU S.M. NDOW SG• LOLLEY KAH JALLOW P.I.A

Dataset• 2003 Household

• Variable Count

• 38

• Case Count

• 157,114

Dataset cont.• 2003 Individual

• Variable Count

• 99

• Case Count

• 1,360,681

Dataset cont.

• 2003 Building

• Variable Count

• 501

• Case Count

• 112,200

File Description.• Contents

• HOUSEHOLD DATA FILE 2003 CENSUS

• Producer

• GBOS

• Version

• VERSION 1 (FINAL)

• Processing Checks

• STRUCTURE CHECKS DURING AND AFTER DATA ENTRY• RANGE CHECKS DURING DATA ENTRY• CONSISTENCY CHECKS• VALIDATION CHECK ON VARIABLES

• Missing Data

• USING CODE 9 0R 99 0R 999 FOR BLANK•

• Notes

• DATA PROCESSING WAS DONE WITH 100% VERIFICATION.

Datafile.• Contents

• HOUSEHOLD DATA FILE 2003 CENSUS

• Producer

• GBOS

• Version

• VERSION 1 (FINAL)

• Processing Checks

• STRUCTURE CHECKS DURING AND AFTER DATA ENTRY• RANGE CHECKS DURING DATA ENTRY• CONSISTENCY CHECKS• VALIDATION CHECK ON VARIABLES

• Missing Data

• USING CODE 9 0R 99 0R 999 FOR BLANK• • Notes

• DATA PROCESSING WAS DONE WITH 100% VERIFICATION.

Variables (eg)

Lga banjulDistrict banjul_southVil 1Ur/ru urbanEa 10001Comp 001Hh 01etc for all the three questionnaires

External Resources

• Reports

• Questionnaires

• Technical Documents

• Programs

Reports

 Resource

DOCS\2003 census topics.pdfDOCS\COMPOUND IN SETTLEMENT.pdfDOCS\Enumerator_manual.pdfDOCS\OCCUPATION CODE.pdfDOCS\INDUSTRY CODE.pdfDOCS\DIRECTORYS04.pdf

Questionnaire

Resource

DOCS\CENSUS FORM A.pdf

Technical Documents

• Resource

• DOCS\TECHNICAL DOCUMENT\BUILDING.FMT

Programs

• Resource

• PROGRAMS\BUILD EDIT CN.pdf

Advantage of Archiving

• To lead in developing statistical data and information of high quality, and to advance their effective use in both public and private policy decision making

• A power tool that facilitates the process of releasing census microdata to user community and allows the institution to exercise control over its data.

• Data Access portal: provides a clear way to implement data dissemination policies allowing full control to the publishing institutions. It does so by providing information on data access policies and access to procedural forms and licenses for :

• Public use file• Licensed files• Data enclave admission

Our Mission• The Gambia national data archive has been established to:

• Promote best practice and international standards for the documentation of microdata amongst data producers in the country

• Provide equitable access to microdata in the interest of all citizens, by protecting confidentiality and following international recommendations and good practice

• Promote the effective use of existing census data for statistical and research purposes thereby encouraging a diverse range of analytical work through secondary research.

• Ensure the long term preservation of microdata and the related metadata, and their continued viability and usability in the future.

• The Gambia National data Archive pursues these objectives within the framework of the Statistical Act and the United Nations Fundamental Principles of Statistics. Where microdata cannot be released due to confidentiality or other reasons, the National Data Archive provides the public with detailed metadata and other publicly available materials

Our activities• Activities:Include the acquisition, documentation, anonymization, dissemination,

and preservation of micro-data and related metadata.• Acquisition: Primarily established to archive census microdata produced by

GBOS and other official data producers. It also serve as a repository for non-official datasets. Data producers interested in depositing data in the data archive are invited to contact us.

• Documentation: Data documentation serves several important functions. It helps data producers build institutional memory, and helps researchers to :

• Find the data they are interested in• Locate the datasets and variables that meet their research requirements• Understand what the data are measuring and how the data have been created,

and assess their quaility• Understand the survey design and the methods used when collecting and

processing the data, thereby reducing the risk that data will be misunderstood or misused.

• The Gambia National Data Archive adopted the Data Documentation Initiative (DDI) and the Dublin Core (DCMI) international metadata standards

Cont.• Anonymization: Gambia Bureau of statistics is charged with legal and ethical

obligations to protect the confidentiality of census respondent. The Gambia National Data archive aims to protect confidentiality of the data by:

• Restricted access to data that present a potential disclosure risk to scrutinized users only, under formal conditions.

• Anonymizing data when necessary, by altering or supressing variables which could potentially identify a physical or legal individual. This may make the data less useful for analysts. The Natinal Data Archive seeks to minimise the information loss while ensuring an acceptable level of disclosure risk. Principles and methods applied for measuring the risk and for anonymizing data are those provided or recommended by the International Household Survey Network.

• Dissemination: Data dissemination increases the quality, use and potential impact of data, by:

• 1. Making it possible for analytical work to be replicated, a critical step to good science ;

• 2. Creating the potential to use old data to test new ideas;• 3. reducing the costs of data collection and the burden on respondents, by avoiding

the need for researchers to undertake their own surveys.• 4. Demonstrating transparency and credibility in data production, which are at the

heart of good governance; and• 5. Improving the relevance and quality of data by incorporating users feedback in

future data collection.

Cont.• Obviously, making microdata available also has down sides. It exposes data producers to criticism,

it increase the risk of breach to confidentiality, and it can result in conflicting outputs being generated. Having faith in the ethical consuct of data users and in their willingness to contribute to the quality and usefulness of the data, The Gambia National Data Archive considers that the benefits outweigh the disadvantages. We insist however that access to microdata must not be seen as a right. Access will only be permited to bona fide users, and for statistical and research purposes only.

• Preservation: Micro-datasets can be damaged or lost because of human error, because of technical problems, or because of disasters such as fire of flood. New technologies can also render old data unreadable, because of either hardware or software advances. The Gambia National Data Archive is implementing standard procedures for ensuring the physical security and long-term usability of its resources, together with associated backup arrangements for minimizing the impact of adverse events.

• Policies and procedures: Micro-datasets are categorised into three groups, according to the sensitivity of their content and their inherent disclosure risks:

– Public use files: made available on-line to all interested users, for research– Licensed files: involves a signed agreement between GNDA and external trusted

users, to permit them to access semi-anonymized datafiles.– Files accessible on-site (data enclave): sensitive data, access is only provided on site

in our data enclave under strict conditions, and only for research purposes.– GNDA: scrutinise the generated outputs in a full disclosure review before they are

released.

Way forward:• Seminar is timely as it will help the NSO’s to work on one platform, to help to

identify the core challenges in census data archiving

• Help in formulating and implementing an effective archiving plan suited to the needs and requirements of NSO’s

• Seminar will help in identifying good practices and lessons learned in census data archiving.

• Statisticians to give out census documents on time to the archivers to start archiving on time. (laugh)

• DevInfo / Gaminfo can also be used for archiving census indicators and disemination

• Special thanks to my fellow participants, by working hard and sharing their country’s experience on census data archiving.

• Thanks to UNSD and UNECA for their idea in organising this timely seminar

• Bravo

End of Presentation By:

Mrs. Lolley Kah Jallow

Principal Information AnalystGambia Bureau Of Statistics.

(GBOS)

Thank You.