Multicriteria model to improve the use of performance evaluation instruments with focus on...

Multicriteria Model to Improve the Use of Performance Evaluation Instruments with Focus

on Indicators

Msc. Edilson Giffhorn, PMP, IPMA - Speaker

PhD. Leonardo Ensslin

PhD. Sandra Rolim Ensslin

Msc. William Barbosa Vianna

ALIO-INFORMS Joint International Meeting6. – 9.6.2010, Buenos Aires - Argentina

Objective:

The aim of this paper is to present a process for improving the way to use the instruments of performance assessment on their way to identify, organize, measure and use the Performance Indicators.

2

3

1494

18th century

20th century – until 1950

20th century – after 1950

Tratactus de Computis et Scripturis - Summa de arithmetica, geometrica, proportioni et proportionalita(Brudan, 2009; Ensslin; Ensslin, 2009)

Industrial Revolution:Large corporations - production control(Leão, 1998)

Scientific Management:Evaluation of merit (people)(Guimarães et al. 1998)

Evaluate processes (Brandão, Guimarães, 2001)

The use of indicators as a management tool / control

Introduction

Instrument of Intervention

Evaluation Model

Conclusions

4

Until1960

1960 - 1995

After 1995

1st Generation Indicators

2nd Generation Indicators

3rd Generation Indicators

Performance Indicators

Ensslin, Ensslin (2009)

Introduction

Instrument of Intervention

Evaluation Model

Conclusions

5

1st Generation Indicators

Neely (1999); Ensslin, Ensslin (2009); Oliveira et al (2009)

Emphasis

Control

Introduction

Instrument of Intervention

Evaluation Model

Conclusions

6

Bititci, Suwignjo, Carrie; (2001); Ensslin, Ensslin; (2009)

2nd Generation Indicators

Emphasis

Management

Introduction

Instrument of Intervention

Evaluation Model

Conclusions

7

Ensslin, Ensslin (2009)

Characteristics of instruments

New requirements of the context

Generic context Specific context

Generic decision makers Specific decision makers

Monitoring Continuous Improvement

Ordinal scales Ordinal and Cardinal scales

Does not compare performance indicators

Comparison of performance by integration

Available quantitative properties

Quali-quantitative properties

Introduction

Instrument of Intervention

Evaluation Model

Conclusions

8

Ensslin, Ensslin; (2009)

3rd Generation Indicators

Emphasis

Improve specific contexts in a personalized way

Introduction

Instrument of Intervention

Evaluation Model

Conclusions

9

Ensslin, Ensslin (2009)

3rd Generation Indicators

Paradigm exchange

Isolated measures Integrated measures

Quantitative measures Quali-quantitative measures

Generic measures Specific measures for each context

Measures to control Measures to improve performance

Introduction

Instrument of Intervention

Evaluation Model

Conclusions

10

Traditional historical and

financial orientation

Neely, Powell (2004)

Focus on the measure,

misaligned with the goals

and unbalanced

Leandri (2001)

Generic or pre-existing

Denton (2005), Tangen (2003)

Even with the recognition of new requirements, many applications of performance measurement continue to use the

1st and 2nd generation.Introduction

Instrument of Intervention

Evaluation Model

Conclusions

11

Consequences

Schneiderman (1999), Neely (2000), Bourne et al. (2002)

Denton (2005)

Measures not aligned to strategy.

Performance Evaluations poorly developed, misaligned with the strategy and decision-makers do not understand the impacts of their decisions and actions.

Introduction

Instrument of Intervention

Evaluation Model

Conclusions

12

Research Theme

Explore ways of improving the identification / construction of the

Performance Indicators.

Provide greater chance of achievement the strategic objectives based on the tools

selected.

Introduction

Instrument of Intervention

Evaluation Model

Conclusions

13

State of the Art

Process to identify the theoretical referencial

35 Articles

Article

Article

Article

Keywords

Introduction

Instrument of Intervention

Evaluation Model

Conclusions

Ensslin, Dutra, Ensslin (2000)

14

MCDA-C Phases

Instrument of Intervention

Introduction

Evaluation Model

Conclusions

15

Ensslin et al. (2010)

Identify the important aspects in accordance with the decision maker.

Expand the understanding of the context of the decision maker.

Allow to take into account the interests of the actors involved with the monitoring of

the decision maker.Allow the decision maker to revise his

views in the course of the construction of their knowledge.

Recognize the ordinal and cardinal properties of the indicators.

Ensure that the decision maker legitimize the process with tools scientifically valid.

MCDA-C selected due:

Instrument of Intervention

Introduction

Evaluation Model

Conclusions

16

Structuring Phase

Identification of the Actors Subsystem

Evaluation Model

Introduction

Instrument of Intervention

Conclusions

17

Label:

Build a model to improve the mapping of the indicators on the instruments of performance evaluation.

Structuring Phase

Evaluation Model

Introduction

Instrument of Intervention

Conclusions

18

Best possible performance

Worst possible performance Scales used allow only statistical operations.

Objective underlying the EPA : Capactity to perform mathematical operations between scales.

Scales are constructed so as to enable mathematical operations between them.

EPA 61: mathematical operations

Intensity: Very Strong

Concept 61: Ensure that mathematical operations can be performed between the scales... use scales that allow only statistical operations.

Structuring Phase

Process to transform the Primary Assessment Elements in Concepts

Evaluation Model

Introduction

Instrument of Intervention

Conclusions

Structuring Phase

Evaluation Model

Introduction

Instrument of Intervention

Conclusions

Label

Areas of Concern

Concepts

21

Competitive

Excelence

Endanger

Definition LegitimationMathematical Foundations

Improve the Mapping of Indicators in the Performance Assessment

Instruments

PVF 6 – Ordinal Scales

PVF 7 – Cardinal Scales

PVF 3PVF 4 –

Value Function

PVF 5 – Substitution

RatesPVF 1 PVF 2

100

0

100

0

100

0

100

0

100

0

100

0

100

0

Good

Neutral

X% Y% Z%

A% B% C% D% E% F% G%

Impact profile of the Performance Measurement tool 1

Impact profile of the Performance Measurement tool 2

Impact profile of the Performance Measurement tool 3

Legend:

Global Evaluation

PM Tool 1

PM Tool 2

PM Tool 3

72

68

40

Process

W%

Evaluation Model

Introduction

Instrument of Intervention

Conclusions

General view of the final model.

22

Conclusion

The model will serve as a support instrument to make Performance Evaluations, in order to have greater accuracy and alignment between the operational level, tactical and strategic organization.

The model will give ways for improving the identification / construction of the Performance Indicators.

Conclusions

Introduction

Instrument of Intervention

Evaluation Model

23

THANKS

END

Msc. Edilson Giffhorn, PMP, IPMA: edilson.giffhorn@gmail.com

PhD. Leonardo Ensslin: ensslin@deps.ufsc.br

PhD. Sandra Rolim Ensslin: sensslin@deps.ufsc.br

Msc. William Barbosa Vianna: wpwilliam@hotmail.com