Upload
dennis-townsend
View
215
Download
0
Embed Size (px)
Citation preview
CONCEPTUAL ISSUES CONCEPTUAL ISSUES IN CONSTRUCTING IN CONSTRUCTING
COMPOSITE INDICESCOMPOSITE INDICESNadia FarrugiaNadia Farrugia
Department of Economics, University of MaltaDepartment of Economics, University of Malta
Paper prepared for the Paper prepared for the INTERNATIONAL CONFERENCE ON SMALL STATES AND INTERNATIONAL CONFERENCE ON SMALL STATES AND
ECONOMIC RESILIENCE ECONOMIC RESILIENCE Organised byOrganised by
The Islands and Small States Institute The Islands and Small States Institute of the Foundation for International Studies at the University of of the Foundation for International Studies at the University of
MaltaMaltaand the and the
Commonwealth Secretariat, LondonCommonwealth Secretariat, LondonValletta, MaltaValletta, Malta
23 - 25 April 200723 - 25 April 2007
Presentation OutlinePresentation Outline IntroductionIntroduction Desirable Attributes for Developing Desirable Attributes for Developing
Statistics and Composite IndicesStatistics and Composite Indices Main Conceptual IssuesMain Conceptual Issues
Indicator SelectionIndicator Selection Dealing with Missing DataDealing with Missing Data NormalisationNormalisation Weighting and AggregationWeighting and Aggregation Testing and Reviewing the Results ObtainedTesting and Reviewing the Results Obtained
ConclusionConclusion
DefinitionDefinition
A composite index, A composite index, is a weighted (linear) aggregation of a is a weighted (linear) aggregation of a
number of variablesnumber of variables wwjj is a weight, with 0≤ is a weight, with 0≤wwjj≤1 and ∑≤1 and ∑wwjj=1=1
XXcjcj is the variable of country is the variable of country cc in in dimension dimension jj
for any country for any country cc the number of policy the number of policy variables are equal to j=1,…,m.variables are equal to j=1,…,m.
1
m
c j cjj
I w X
UsesUses Describe complex phenomena in a single Describe complex phenomena in a single
indicatorindicator Cross-national comparisons of country Cross-national comparisons of country
performanceperformance Benchmarking exercisesBenchmarking exercises General trendsGeneral trends Policy priorities and performance targetsPolicy priorities and performance targets Several examples of renowned composite Several examples of renowned composite
indices, stock market indices, RPI, GDPindices, stock market indices, RPI, GDP
StrengthsStrengths
Summarises complex and multi-Summarises complex and multi-dimensional issuesdimensional issues
Helps set the direction for Helps set the direction for policymakers and to focus the policymakers and to focus the discussiondiscussion
Supports decision makingSupports decision making Helps disseminate informationHelps disseminate information Make stakeholders and the public more Make stakeholders and the public more
aware of certain problemsaware of certain problems Generates academic discussionGenerates academic discussion
WeaknessesWeaknesses
Subjectivity in computationSubjectivity in computation May send misleading policy May send misleading policy
messages and can easily be misusedmessages and can easily be misused May conceal divergences between May conceal divergences between
different components different components Increase difficulty in identifying Increase difficulty in identifying
proper remedial actionproper remedial action Measurement problemsMeasurement problems
Quality FrameworksQuality Frameworks IMF – Data Quality Assurance FrameworkIMF – Data Quality Assurance Framework Eurostat FrameworkEurostat Framework OECD – Quality Framework and OECD – Quality Framework and
Guidelines for OECD StatisticsGuidelines for OECD Statistics Booysen – Dimensions for Classifying and Booysen – Dimensions for Classifying and
Evaluating Development IndicatorsEvaluating Development Indicators Briguglio – Desirable Characteristics for Briguglio – Desirable Characteristics for
Developing Vulnerability IndicesDeveloping Vulnerability Indices JRC-OECD – Handbook on Constructing JRC-OECD – Handbook on Constructing
Composite IndicatorsComposite Indicators
Desirable Attributes of Desirable Attributes of Composite IndicesComposite Indices
1.1. AccuracyAccuracy2.2. Simplicity and Ease of ComprehensionSimplicity and Ease of Comprehension3.3. Methodological SoundnessMethodological Soundness4.4. Suitability for International and Suitability for International and
Temporal ComparisonsTemporal Comparisons5.5. TransparencyTransparency6.6. AccessibilityAccessibility7.7. Timeliness and FrequencyTimeliness and Frequency8.8. FlexibilityFlexibility
Main Conceptual IssuesMain Conceptual Issues
Indicator SelectionIndicator Selection Dealing with Missing DataDealing with Missing Data NormalisationNormalisation Weighting and AggregationWeighting and Aggregation Testing and Reviewing the Results Testing and Reviewing the Results
ObtainedObtained
Indicator SelectionIndicator Selection
Define the conceptDefine the concept Select indicators which satisfy desirable Select indicators which satisfy desirable
attributesattributes Do not select variables which beg the Do not select variables which beg the
questionquestion Draft an initial indicator set and review the Draft an initial indicator set and review the
available dataavailable data Keep the number of variables as small as Keep the number of variables as small as
possible but not fewer than necessary possible but not fewer than necessary (PCA, FA)(PCA, FA)
Indicator Selection Indicator Selection (Cont.)(Cont.)
Check for correlation between the Check for correlation between the variables or sub-indices (rank variables or sub-indices (rank correlation test, Cronbach correlation test, Cronbach coefficient alpha, cluster and coefficient alpha, cluster and discriminant analysis)discriminant analysis)
Review the indicators selected and Review the indicators selected and seek external advice and opinionseek external advice and opinion
Dealing with Missing Dealing with Missing DataData
Exclude the country from the Exclude the country from the analysisanalysis
Imputation methods: Single or Imputation methods: Single or MultipleMultiple
Single Imputation Single Imputation MethodsMethods
Case deletionCase deletion Mean/median/mode estimationMean/median/mode estimation Hot deck imputationHot deck imputation Regression imputationRegression imputation
Multiple Imputation Multiple Imputation MethodsMethods
Regression MethodRegression Method Propensity Score MethodPropensity Score Method Markov Chain Monte Carlo Markov Chain Monte Carlo
AlgorithmAlgorithm
Quantifying Qualitative Quantifying Qualitative DataData
Using a mapping (Likert) scaleUsing a mapping (Likert) scale Optimal spread of the scaleOptimal spread of the scale Permits non-linearityPermits non-linearity Defect relates to subjectivityDefect relates to subjectivity
NormalisationNormalisation
RescalingRescaling Standardisation (or z-scores)Standardisation (or z-scores) Percentage differences over Percentage differences over
previous yearsprevious years RatiosRatios RankingsRankings Measuring the relative position vis-à-Measuring the relative position vis-à-
vis a specified pointvis a specified point
Weighting and Weighting and AggregationAggregation
Equal WeightingEqual Weighting Differential WeightingDifferential Weighting Country-Specific or Indicator-Country-Specific or Indicator-
Specific WeightsSpecific Weights Weights Over Time: Constant or Weights Over Time: Constant or
ChangingChanging
Differential WeightingDifferential Weighting
Weights Reflecting the Statistical Weights Reflecting the Statistical Quality of the DataQuality of the Data
Stochastic WeightsStochastic Weights Participatory MethodsParticipatory Methods Precautionary PrinciplePrecautionary Principle Regression MethodRegression Method Benefit-of-the-Doubt Weighting Benefit-of-the-Doubt Weighting
SystemSystem
AggregationAggregation
Linear or geometric aggregationLinear or geometric aggregation Aggregation methods and weighting Aggregation methods and weighting
systemssystems Non-compensatory multi-criteria Non-compensatory multi-criteria
aggregationaggregation
Testing and Reviewing the Testing and Reviewing the Results ObtainedResults Obtained
Uncertainty and Sensitivity AnalysisUncertainty and Sensitivity Analysis OutliersOutliers Expert OpinionExpert Opinion Analysing the Results ObtainedAnalysing the Results Obtained
ConclusionConclusion
Composite indices have their pros and cons.Composite indices have their pros and cons. Hard to imagine that the debate on the use of Hard to imagine that the debate on the use of
composite indices will be ever settled.composite indices will be ever settled. Composite indices should be identified for Composite indices should be identified for
what they are.what they are. However, their importance should not be However, their importance should not be
undermined. undermined. Provided they are built on sound Provided they are built on sound
methodological considerations they are very methodological considerations they are very useful to portray complex phenomena in a useful to portray complex phenomena in a simple manner.simple manner.