The systems approach to official statistics Bo Sundgren 2010 [email protected]

The systems approach to official statistics

Bo Sundgren

[email protected]

http://sites.google.com/site/bosundgren/

mailto:[email protected]

http://sites.google.com/site/bosundgren/

My presentation• Why a systems approach – and what does it mean?

Some history

• Some different views of statistical systems

• Quality and efficiency of statistical systems

• Different perspectives on statistical systems:customers, stakeholders, organisation, information system (contents/data + processes), technical

• Modelling the contents of a statistical system:helicopter, zoom, contents by example, everything clickable

• Conclusion: Statistical systems are unperceivable systems (Langefors), and the systems approach is a strategy for coping with statistical systems without losing either overview or precision in details


Some history






Why a systems approach?• Modern societies are complex and interdependent

• Governments demand more, better, and more coherent official statistics for decision-making, planning, and evaluation

• Businesses also demand good official statistics, national and international, to discover and exploit business opportunities

• Other stakeholders: analysts, researchers, students, journalists, and interested citizens

• But there are budget and time constraints forcing all producers of official statistics to economise with resources, especially in data collection

• Respondents expect producers to reduce the response burden, and to harmonise data collection for official statistics with the natural business processes of the data providers

• Hence a holistic approach is needed, the systems approach

Paradigms in the history of statistics• Data about the state: bookkeeping of citizens and resources

(from Farao and onwards) – ”statisticus” (Latin) = statesman, politician

– “statistics” = “a comprehensive description of the social, political, and economic features of a state”, Gottfried Achenwall in “Staatsverfassung der heutigen vornehmsten europäischen Reiche und Völker im Grundrisse” (1749)

• Statistics as a mathematically founded theory: probability theory (17th century and onwards)

• Probability-based sample surveys (20th century)

• Reuse of data from administrative registers and data archives

• Reuse of data generated by business processes and other processes on the Internet

• The systems approach: a holistic and combined approach

What is a system, and what is ”the systems approach”?

• The systems approach is not a bad idea, according to Charles West Churchman in the summary statement of his book on the topic in 1968

• von Bertalanffy focused on open systems (in biology) as opposed to closed systems (studied in physics)

• Open systems interact with other systems outside themselves through inputs and outputs

• A system and its environment are in general separated by a boundary, an interface

• The output of a system a direct or indirect result from the input

• But the system is not just a passive tube, but an active processor, a transformer

• The transformation of input into output by the system is usually called throughput

Figure 1. A system in interaction with its environment. From Heylighen (1998).

Figure 2. A system as a transparent box, containing a collection of interacting subsystems, and as a black box, without observable components. From Heylighen (1998).

A system, its environment, and its parts

• The environment of a system consists of systems interacting with their environments

• A collection of such systems which interact with each other could again be seen as a system

• The mutual interactions of the component systems, or parts makes the system as a whole something more than the sum of its parts

• With respect to the whole, the parts are seen as subsystems

• With respect to the parts, the whole is seen as a supersystem

• If we look at a system as a whole, we need not be aware of all its parts; we can just look at its total input and total output: the black box view of a system

• When necessary, we may adopt the transparent box view, where we can see the system's internal processes

Different types of systems and views of systems

• open systems vs closed systems• black box views vs transparent views• hard systems vs soft systems

– hard systems, e.g. technical systems, are typically associated with quantifiable variables

– soft systems typically involve people and both quantifiable variables, and variables that are not easy to quantify; see, for example, Checkland (1981) and later works by the same author

• man-designed systems vs given systems (by ?)

• man-independent systems, e.g. buildings, vs man-dependent systems, e.g. enterprises – what Berger&Luckmann (1966) call social constructions of reality


Some history






Statistical systems• open, soft, man-designed, “reality as a statistical construction”

DESIGN &IMPLEMENT

EVALUATE

request

response

request

response

act

perceive

perceive

STATISTICAL PRODUCTION SYSTEMRESPONDENTS USERS

stakeholderreqirements

systemspec

evaluationresults

MONITOR

SERVE

request response

PLANNING AND CONTROL SYSTEM

INPUT THRUPUT OUTPUTrawdata

obsdata

statendprod

request response

requirement

requirement

requirement

requirement

EXTERNAL REALITY

Figure 3. A statistical system in its environment. From Sundgren (2004a).

Statistical systems: one view

DESIGN,CONSTRUCTION,

IMPLEMENTATION

Stakeholderinterests

Knowledge and metadata (global and local)

Production systemspecification

Rules,instructions,

metadata

MONITORING

Control andfeed-back

information

PRODUCTIONSYSTEM

"Reality"

Data and metadata

Informationproducts and

servicesUSAGE

Decisions made,problems solved;user satisfaction


information

EVALUATION

Production andusage experiences

Knowledge Experiences DocumentationsMetadata (inclprocess data)

Figure 4. Control and execution of a statistical production system. Metadata and paradata (process data). Feedback. From Sundgren (2004a).

Databases

Data Warehouse

DATA COLLECTION

Rules, instructions, metadata

MONITORING

Control and feed-back information

OBSERVATION/MEASUREMENT

"Reality"

Frame

Informationproducts and

services

DATAPREPARATION

Observationdatabase


information

ESTIMATION

Final output database


information

DISSEMINATION

Finalobservation

database

Outputdatabase

Production experiences

Figure 5. Basic operations in a database-oriented statistical production system. From Sundgren (2004a).

Statistical systems: another view

KNOWLEDGE BASE

DATA WAREHOUSE

DESIGN AND PLANNING

OPERATION

EVALUATION

Objectivesand contents

Measurement,data collection,and preparation

Presentation anddissemination

Frame andsampling

Estimation

Global metadata

Frames

Localproductiondatabase

Disseminationdatabase

Localproductiondatabase

Final observationregisters

Classifications andstandards

LEGEND

control flow

data/metadataflow

process

data/metadata

Local metadata about thesurvey under consideration

Metadata about all surveys:- SCBDOK documentations- Product descriptions- Quality declarations- Formalized metadata

Process data

Process data

Establish frameand sample

Measure, collectand prepare data

Aggregate andestimate

Present anddisseminate

input datainput datastatistical

outputsinput

knowledge

Expert knowledge:- CBM documents- Handbooks- Scientific literature- Expert systems

Figure 6. Statistics production with integrated data and metadata management. From Sundgren (2004a).

Statistical systems: yet another view


Some history






Quality and efficiency of statistical systems• quality as fitness for purpose: “the totality of features and

characteristics of a product or service that bear on its ability to satisfy stated or implied needs” (ISO 8402, 1994)

• quality components (Eurostat, 2003):– relevance

– accuracy

– timeliness and punctuality

– accessibility and clarity

– comparability

– coherence

• quality of individual surveys vs quality of statistical systems– special purpose vs general purpose

– known needs vs unknown needs

– present needs vs future needs

• efficiency

• needs for new methods and methodologies?


Some history






Statistical systems from different perspectives• the customers perspective, the statistical system as seen from the perspective

of customers, or users of statistics

• the stakeholders perspective, including also sponsors, respondents, etc

• the organisational perspective, how a statistical organisation is set up for a statistical system, e.g. the statistical system of a country or an international organisation

• the information systems perspective – statistical systems in general, and systems for production of official statistics in particular, are examples of information systems, systems for obtaining, storing, processing, and communicating information

• the contents perspective, how the statistical system relates to some kind of theoretical framework, where the data from the statistical system is used, e.g. the system of national accounts (SNA)

• the data/process perspective, a dualistic pair of views, where the data view focuses on the logical organisation of the data and metadata obtained, stored, transformed, and produced by the statistical system, and the process view focuses on the processes for obtaining, storing, transforming, and producing statistical data and metadata

• the technical perspective, the hardware/software environment, where the statistical system is implemented and running

Customers - Users• The customer in focus• The customer is always right• How to reconcile conflicting interests between many customers, some

of whom are even unknown, in a general-purpose system?

socialresearcher

teacher,student

interestedcitizen

politician

analystadvisor tominister

journalistactor on the

finance market

Figure 7. How to satisfy partially conflicting needs of different users of official statistics? From Sundgren (2004a).

Stakeholders

• Customers – Users• Respondents• Sponsors (the taxpayers and their representatives)• Producing staff: designers, operators, managers,

business partners and professional colleagues• Researchers on official statistics• Others concerned, ”victims” – cf group privacy

Organisation

Matrix organisation

Autonomousstovepipes

Co-ordinated stovepipes:standardised processes

around coroporatedata warehouse

Functional organisation

Organisationalintegration

Productionsystemintegration

High

Low

HighLow

Figure 8. Different ways of organising a statistical organisation.

Why have a statistical agency at all?

Better use of the potentialsof the organisation

Better use of data and production systems

Emerging: Network organisations

Organisation

Figure 9. Four ways of organising tasks and responsibilities in a statistical organisation.

Emerging network organisations

• Coordination (in one network node) of loosely coupled, services in other nodes of the network

• Cf the Internet• Cf Service-Oriented Architecture (SOA)• Services should be welldefined and easy to use through

standardised messages• There may be many coordination nodes using overlapping

sets of service nodes• Some of the coordination nodes may be competing with

each other (cf different Internet shops using partially the same warehouses, transportation firms, etc)

Information systems: data/process dualism

C O U N T R Y P R O D U C TI M P O R T

T R A N S A C T I O N

V o l u m e

V a l u e

W H E R EF R O M ?

W H A T ?

P r o d u c t I d

P r o d u c t N a m e

D e s c r i p t i o n

C o u n t r y I d

C o u n t r y N a m e

G N P

C O M P A N Y

C o m p a n y I d

N u m b e r O f E m p l o y e e s

B r a n c h O f I n d u s t r y

BY

WH

OM

?

I N P U T D A T AC O L L E C T I O N

e . g . s e t o f c o m p l e t e dq u e s t i o n n a i r e s f r o m o n e

d a t a c o l l e c t i o n p r o c e s s

F I N A LO B S E R V A T I O N

R E G I S T E Re . g . a r e l a t i o n a l d a t a b a s e ,

a S A S d a t a s e t , o r as e t o f f l a t f i l e s ,

o r g a n i s e d a so n e s e t o f m i c r o d a t a

F I N A L S E T O FS T A T I S T I C S

e . g . a r e l a t i o n a l d a t a b a s e ,o r a s e t o f

m u l t i d i m e n s i o n a l c u b e s( s t a r s ) ,

o r g a n i s e d a so n e s e t o f m a c r o d a t a

O U T P U T D A T AC O L L E C T I O Ne . g . a p u b l i c a t i o n o r a

s e t o f t a b l e s o n aw e b s i t e ,

o r g a n i s e d a s o n ei n t e g r a t e d s e t o f s t a t i s t i c a l

o u t p u t s

T h e S C B D O Kc o n c e p t u a lf r a m e w o r k

- d a t a c o n t e n t s- p r o c e s s e s

Q u a l i t yD e c l a r a t i o nM a c r o M e t a

M E T A D O KM i c r o M e t a

S C B D O K S C B D O K

S C B D O K

C O N C E P T U A LM O D E L O F

C O N T E N T SC l a s s i f i c a t i o nD a t a b a s e

( K D B )

Figure 16. Fundamental data/metadata interfaces and metadata objects in a statistical production system.

Process view

Figure 17. A process view of a system for production of official statistics.

Technical perspective: Service-Oriented Architecture (SOA)

Design principles:

• Loose coupling – Services maintain a relationship that minimises dependencies and only requires that they retain an awareness of each other.

• Service contract – Services adhere to a communications agreement, as defined collectively by one or more service descriptions and related documents.

• Autonomy – Services have control over the logic they encapsulate.

• Abstraction – Beyond what is described in the service contract, services hide logic from the outside world.

• Reusability – Logic is divided into services with the intention of promoting reuse.

• Composability – Collections of services can be coordinated and assembled to form composite services.

• Statelessness – Services minimise retaining information specific to an activity.

• Discoverability – Services are designed to be outwardly descriptive so that they can be found and assessed via available mechanisms.


Some history






Modelling the contents of a statistical system

• A statistical system must be able to provide relevant information contents to its customers

• But how can customers and designers of statistical systems find out and specify in a precise way, what is relevant contents, given often rather broadly and vaguely defined information needs.

• And how can they reconcile conflicting information needs within the same model

• There are different approaches to this problem. One may distinguish between theory-driven and data-driven approaches; see for example Hox (1997)

• In practice different approaches are combined and iterated

Theory-driven approaches

• derive the desirable information contents from theories and analytical models based on these theories, for example the analytical models and macroeconomic theories behind the System of National Accounts (SNA)

• “top-down”

Data-driven approaches

• may take their starting-point from existing data within fields and sectors of society that are relevant to the customers of the statistical system to be designed

• these data are carefully defined and described in descriptive data models

• these models usually need to be further harmonised between themselves in order to become useful as subdomains of an integrated data model

• “bottom-up”

A practical approach

• brainstorming to create a tentative conceptual (data) model based on existing theories and existing or foreseen data

• Sundgren (1973), Rosén&Sundgren (1991), Sundgren (2005), Sundgren (2006), and Sundgren (2007)

• useful both as specification and documentation• the conceptual model may very easily (even automatically)

be transformed into a physical data model, e.g. a relational data model to be implemented by relational database software

• microdata views and macrodata views

Macrodata view

• On a general level all official statistics are – estimated values of parameters of populations of objects (statistical units)

• where the parameters are – summarised (aggregated) values of variables of the individual objects in

the populations; aggregation function: “sum”, “average”, “correlation”, etc

Figure 10. A statistical cube based on observations of trade transactions.

Macrodata view

• Very often the population (e.g. a population of Persons) is broken down into subpopulations, or domains of interest, by crossclassifying the objects in the population by means of a number of classification variables (e.g. Sex, Region, AgeGroup)

• Data about the population and its subpopulations may be thought of as belonging to cells in a multidimensional cube, a hypercube, where the dimensions of the hypercube are spanned by the classification variables defining the subdomains of the population, and where the cells contain estimated values of the parameters for the respective subdomains obtained by crossclassifying the population, corresponding to the whole cube.

• Sundgren (2001) describes and explains this so-called -model of multidimensional statistical data.

Microdata view• A user of official statistics may not be able to state exactly which parameters of

which populations she is interested in, but faced with a short list of object types (or types of populations), and/or topics, and/or parameters/variables, she may be able to select a subset of official statistics of potential interest by selecting, step by step, a subset of object types (populations), variables, and parameters.

• In order to provide a user with short lists of object types and variables as a starting-point for the user’s “drill-down” operations, we must be able to give an overview of the contents of statistics in terms of a small number of concepts.

PersonEvent(Birth, Death, Marriage,Divorce, Migration, etc)

- ClassVariables- SumVariables

Person


Dwelling


Household


Residence

Figure 11. Population statistics: a basic model.

Microdata view• The populations occurring in official statistics are based upon a number of

basic object types.

• Some of these object types may be described as (conscious) actors, objects that are capable of purposeful acting, e.g. persons and organisations.

• Other objects are acted upon by the actors but are not capable of purposeful acting themselves, e.g. natural resources, products, assets; a common label for basic objects of this kind is “things” (in a broad sense) or utilities.

• In addition to the basic object types there are different kinds of complex object types, relating one of more basic objects. Examples: events/transactions (instantaneous), and relationships/processes/activities, lasting for some time.

PersonEvent(Birth, Death, Marriage,Divorce, Migration, etc)


Person


Dwelling


Household


Residence

Figure 11. Population statistics: a basic model.

Generic model of the contents of official statistics

• All objects are described by means of variables.

• Some variables are classification variables, e.g. “Sex of Person”.

• Other variables are summation variables, e.g. “Income of Person”.

• By using the relations between the objects, one may define derived variables, adjoined variables, such as “DwellingSize of Person” = “Size of Dwelling of Residence of Person”.

• By using summation operators on summation variables, one may derive values of parameters of populations or domains of objects of a certain type, e.g. “average(Income of Persons)”. The two types of derivations may also be combined, even repeatedly and recursively.

• Claim: The contents of all branches of official statistics can be expressed as specialisations of this generic model. This has been verified in a large number of practical examples, and no counter-examples have been found.

RelationshipActivityEvent

Transaction

- ClassVariables- SumVariables- AdjVariables

- EstParameters

Actor


- EstParameters

Utility


- EstParameters

ProgramExecution.countunique(Provider.Id,

Program.Type,Program.Level,

Program.Orientation)

EducationProvider(Institution)

x Sector (Public/Private)

Teacher

x Sex

Provides

TeacherEngagement.count

x TeacherEdStatusx PartTimeStatus- PartTimeFraction.sum

EducationSystem(Utility)

- Country- Currency- CompulsoryEdBegAge- CompulsoryEdEndAge- CompulsoryEdLength- AcadYearBegMonth- AcadYearEndMonth

IsEngagedIn

EducationProgram(Utility)

- Name - Year - EntranceAge - Duration x Type x Level (ISCED97) x Grade x Orientation x PositionInDegreeStructure x FieldOfEducation

BelongsTo

Of

Pupil

x Sexx Agex CountryOfOriginx AttendedPrePrimary

PupilEnrolment.count

x PartTimeStatusx Repeaterx Completer/DropOutx CumulatedTime- PartTimeFraction.sum

IsEnrolledIn

Expenditure

x EducationalStatusx Sourcex Nature- Amount.sum

Funder(Actor)

x Sector (Public/Private/...)

Pays For

For

For

LEGEND:

one-to-many relationship

many-to-one relationship

one-to-one relationship

many-to-many relationship

x Variable: indicates that the ”Variable” variable has a classifying role

Object.count – indicates that ”Object” objects are counted

Variable.sum – indicates that the ”Variable” variable is summarised

reading direction

For

UNESCOexample,version 1

What can a statistical agency do, in order to help a user - find potentially relevant statistical data? - judge the relevance of data retrieved?

• Provide overviews of available data

• Provide search tools

• Provide informative metadata

Helicopter overview + zoom (cf Google Maps, Google Earth, etc)

Contents By Example (based on a simple generic model)

PERSONVARIABLE

VARIABLE

VARIABLE

VARIABLE

VARIABLE

x

m

>0

x

ORGANISATIONVARIABLE

VARIABLE

VARIABLE

VARIABLE

VARIABLE

x

p

<5

x

RESOURCEVARIABLE

VARIABLE

VARIABLE

VARIABLE

VARIABLE

g

PRODUCTVARIABLE

VARIABLE

VARIABLE

VARIABLE

VARIABLE

x

ACTIVITYVARIABLE

VARIABLE

VARIABLE

EVENTVARIABLE

VARIABLE

VARIABLE

RELATIONVARIABLE

VARIABLE

VARIABLE

x x

xx

Actors Utilities

Complexobjects

Everything ”clickable”OBJECT

VARIABLE

Lefthand click Righthand click

Select:- object- variable

Retrieve metadata:- definition- value set, classification- questionnaire- quality declaration- survey documentation


Some history






Unperceivable systems (Langefors)(imperceivable, imperceptible)

• In summary, the systems approach is a paradigm for managing the design, operation, and evaluation of unperceivable systems, systems which are too big and complex for the human brain to grasp in one go

• Statistical systems are such systems• Combination of top-down and bottom-up, overview and

detail, comprehensiveness and precision, soft and hard

Documents

The systems approach to official statistics Bo Sundgren 2010 [email protected]