NSS ASSESSMENT TOOLS (PARIS21 Study)

Inter-regional Seminar on national statistical systems reviews in SIDS 1-3 August 2017, Nadi, Fiji

Millicent Gay Tejada Regional Programme Coordinator, Asia-Pacific and SIDS

2

Outline 1 Why the study

2 Typology of NSS Assessment Tools

3 Considerations for undertaking NSS assessment

3

1 - Why the Study

Understand the various tools available to assess statistical capacity and/or current state of the NSS (and what aspects it

reviews/assesses and why)

Help developing countries find

the appropriate tool for their NSS review/ assessment (intent of the review and where the results will be used for)

Provide a Guide to NSS

assessment (harmonised approach

to assessing NSS in light of new developments in data)

4

15 different tools that assess/reviews national statistical systems (NSOs, statistical products, specific domain or area, etc.)

The tools reviewed point to 3 distinct (but often overlapping) purposes: statistical capacity, data quality, and compliance to good practice.

Scope

5

2 - Typology of NSS Assessment Tools

Type A national statistics plan/strategy or statistical advocacy for NSS modernization/transformation

Type B statistical programme design and monitoring

Type C global monitoring of statistical performance/ capacity

Type D data quality assurance and compliance with codes of practice, norms, and standards

6

Type A national statistics plan/strategy or statistical advocacy

for NSS modernization/transformation

Tools intended to assess NSS performance and statistical capacity to address data users needs.

Some tools utilise questionnaires intended to be completed by statistics stakeholders at the national level (e.g., NSO, Ministry of Planning or Finance, other

stakeholders) as a form of self-assessment; in some cases, questionnaires are completed with the guidance or facilitation of external expert/consultant.

These tools include: Statistical Evaluation and Progress Tool (STEP) NSDS Evaluation Tool Data4SDG Toolkit FAO Country Assessment Questionnaire and In-depth Country

Assessment PARIS21 Peer Reviews Adapted Global Assessment

7

Type B statistical programme design and monitoring

Tools to assess strengths and weaknesses of the NSS related to designing statistical programmes or projects that would address some identified issues/challenges

Results is a mixture of factual and descriptive information, interpretation is somewhat subjective

Tools can be administered by national authorities as part of a diagnostic exercise; questionnaire is administered by experts and analysed on behalf of development agencies

The results of the assessments using these tools are not available in the public domain

Example of Tools under this type: Snapshot Tool by EuropeAid/EUROSTAT Tool for Assessing Statistical Capacity (TASC) by the U.S. Census Bureau

8

Type C global monitoring of statistical performance/capacity

Aimed at presenting global estimates on specific elements of statistical capacity of the NSS and the statistics they produced (but does not reflect institutional or organizational capacity)

Country performance can be compared across time and space

Information/data are collected from publicly available sources and are published with corresponding metadata and methodology to describe it; countries however are not involved in the assessment process as information are collected from

Results are impartial, based on clear criteria, uses facts, scoring is not subjective

These tools include: World Bank Statistical Capacity Indicator (SCI) Open Data Inventory (ODIN) by Open Data Watch

9

Type D data quality assurance and compliance with

codes of practice, norms, and standards

Use of tools are largely mandatory and intended to ensure compliance with standards (or with statistical codes of practice) set by external bodies

Also used by countries to assess suitability of their NSS as part of the accession process to join a regional body

Recommendations are likely to be acted upon by national authorities as non-compliance often has serious consequences or will prevent the country from joining a regional body

Results are likely to be publicly available and stakeholders use them in making decisions

Example of these tools include: IMF Data Quality Assessment Peer Reviews developed by UNECE and widely used by Eurostat OECD Peer Reviews African Statistical Development Indicators

10

STEP Tool (GAMSO) (Type A)

NSDS Evaluation Tool (Type A)

SNAPSHOT (ESS CoP) (Type B)

TASC (DQAF & SCI) (Type B)

FAO CAQ for GSARS (Type A)

1. NSS Vision 1. Information & Status of the NSO

1. Strategy for statistics (in national dev’t )

1. Institutional Capacity

1. Institutional Infrastructure

2. Governance & Leadership of NSS

2. Resources and Expenditure

2. Organisation of the NSS (Legal, institutional and strategic foundations)

2. Planning and Managing

2. Resources

3. Manage Strategic Collaboration

3. Demand Responsive

3. Demand Responsiveness Resources

3. Mapping 3. Statistical Methods & Practices

4. Capability Management

4. Dissemination and Use

4. Quality determinants

4. Questionnaire Content and Testing

4. Availability of Core Data

5. Corporate Support Finances, Human Resources, IT, etc.

5. Data Quality

5. Relations with users

5. Sampling 6. Field Operations

6. GSBPM (dissemination)

6. Results & Performance

6. Key Indicators

7. Data Processing

8. Data Analysis and Evaluation

7. Output indicators

9. Data Dissemination

11

3 - Considerations for undertaking NSS Assessment

1. Purpose/objective of the assessment

Why undertake a review? Who is the target audience of the review? Who is to receive and act on the results?

2. Scope of the assessment

Review of the whole NSS or only parts of it? Is there an issue or sector which requires

special attention?

3. Managing the assessment

Who will conduct the assessment? Which stakeholders to be consulted/key

informants? Self-assessment or with support from an

international expert, or an independent assessment by a development partner?

4. review/assessment process (include various options offered; pros and cons; costs and timing)

How urgent is the assessment – some are rapid others take several months

Face-to-face meetings, consultations with decision makers and stakeholders

12

In the succeeding group work, consider…

13

National regulatory environment

Home user demand

Freedom of press

Rule of law

Strength of economy

Statistical Laws

Codes of Practice

Funding

Resources

Management

Business processes

Dissemination

Quality processes

Training Attitude Skills

twitter.com/ContactPARIS21

facebook.com/ContactPARIS

21 youtube.com/PARIS21OECD

PARIS21 Secretariat

OECD/DCD

4 Quai du Point du Jour

92100 Boulogne-Billancourt,

France

contact@paris21.org

www.paris21.org