Management IS

Version 3.1 – December 2013

© Maurizio Morisio, Marco Torchiano, 2013

2

Licensing Note Attribution-NonCommercial-NoDerivs 2.5

!  You are free: to copy, distribute, display, and perform the work

Under the following conditions: !  Attribution. You must attribute the work in the manner specified by

the author or licensor. !  Noncommercial. You may not use this work for commercial

purposes. – 

!  No Derivative Works. You may not alter, transform, or build upon this work.

!  For any reuse or distribution, you must make clear to others the

license terms of this work. !  Any of these conditions can be waived if you get permission from the

copyright holder.

Your fair use and other rights are in no way affected by the above.

This is a human-readable summary of the Legal Code (the full license) found at the end of this document

MANAGEMENT IS

What gets measured gets done Attributed to P. Drucker

3

4

DATA WORKERS

STRATEGIC LEVEL SENIOR MANAGERS

MANAGEMENT LEVEL MIDDLE MANAGERS

OPERATIONAL OPERATIONAL LEVEL MANAGERS

KNOWLEDGE LEVEL KNOWLEDGE &

Organizational level Group served by IS

Management Control

5

Goal definition (Planning)

Identification of corrective actions

Outcome control (Delta analysis)

EXECUTION / MONITORING

Information about outcomes

Strategic planning

SOFTWARE ENGINEERING INSTITUTE | 4

Figure 1 shows the typical strategic plan elements and their relationships to one another.

Figure 1: Typical Strategic Planning Elements

Note that goals and the steps to achieving them (i.e., the whats and hows) can take on a different tenor at different hierarchical levels of an organization. For example, an organizational strategy might serve as a set of goals for managerial staff, for which that staff will create a sub-strategy; that sub-strategy might translate into goals for technical or production staff.

2.2 Strategic Planning Elements

A well-documented strategic plan is critically important for organizing thinking and communicat-ing thoughts. Strategic plans include elements that describe an organization’s present state, aspira-tions, intentions for the future, and approach for going forward.

Table 1 contains the definitions of the terms typically used to describe strategic planning elements (see also Figure 1). Understanding these elements and their relationship to one another supports not only strategic thinking and planning, but also the effective use of CSFs and future scenarios in strategic planning efforts.

7

Indicators !  What gets measured, gets done !  Management and Strategic level IS

should support managers in " monitoring and controlling the

organization " Using few, reliable indicators

!  How to define these indicators?

8

Approaches !  Management accounting !  CSF !  KPI !  Balanced scorecards !  (Customer and market profiling) !  (Strategic analysis methods)

Approaches

9

BSC (Balanced Score Cards)

Management Accounting KPI & SCOR

Strategy Management

Matrix

Customer profiling &

Market analysis

CSF (Critical Success Factors)

Financial performance monitoring

Process performance monitoring

Customers and market

monitoring

Innovation and critical

resources monitoring

MEASUREMENT

10

Measurement I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind. If you can not measure it, you can not improve it.

Lord Kelvin

11

Measurement

the process of empirical objective assignment of

numbers to entities, in order to characterize a specific

attribute thereof

12

Measurement !  Entity:

" an object or event !  Attribute:

" a feature or property of an entity !  Objective:

" the measurement process must be based on a well-defined rule whose results are repeatable

13

Examples of measures Entity Attribute Measure Person Age Year of last birthday

Person Age Months since birth

Source code Length # Lines of Code (LOC)

Source code Length # Executable statements

Testing process Duration Time in hours from start to finish

Tester Efficiency Number of faults found per KLOC

Testing process Fault frequency Number of faults found per KLOC

Source code Quality Number of faults found per KLOC

Operating system

Reliability Mean Time to Failure

Rules !  Specify both entity and attribute

" The entity must be defined precisely !  You must have a reasonable, even

intuitive understanding of the attribute before you propose a measure.

!  You must not re-define an attribute to fit in with an existing measure.

15

Common mistake !  Proposing a ‘measure’ when there is no

consensus on what attribute it characterizes.

!  E.g. Results of an IQ test "  Intelligence? " or verbal ability? " or problem solving skills?

!  E.g. # defects found / KLOC " quality of code? " quality of testing?

16

Type and use !  Types

" Direct measurement "  Indirect measurement

!  Uses of measurement: " for assessment " for prediction

– Measurement for prediction requires a prediction system

17

Direct measures !  Length of source code

" E.g. measured by LOC !  Duration of testing process

" E.g. measured by elapsed time in hours !  Number of defects discovered during

the testing process " E.g. measured by counting defects

!  Effort of a programmer on a project " E.g. measured by person months worked

18

Indirect measures Programmer productivity =

Module defect density =

Defect detection efficiency =

Requirements stability =

Test effectiveness ratio =

System spoilage =

19

LOC produced person months of effort

number of defects module

size

number of defects detected total number of defects

# of initial requirements total #of requirements

number of items covered

total number of items

effort spent fixing faults total project effort

Predictive measurement !  Requires a prediction system

" Mathematical model –  e.g. ‘E=aSb’ where E is effort in person months (to

be predicted), S is size (LOC), and a and b are constants.

" Procedures for determining model parameters

–  e.g. ‘Use regression analysis on past project data to determine a and b’.

" Procedures for interpreting the results –  e.g. ‘Use Bayesian probability to determine the

likelihood that your prediction is accurate to within 10%’

20

Entity classes

21

Process

Resource

Product an item which is input to a

process

a software related activity

or event

an object which results from a

process

Internal vs. External Given an entity (process, product,

resource) !  Internal attributes can be measured

purely in terms of the entity itself " e.g. length or structuredness of source code

(product) !  External attributes can only be measured

with respect to how the entity relates to its environment " e.g. reliability or maintainability of source

code (product)

22

Metrics Attributes

Entities Internal External PRODUCTS Specification Source Code ....

Length, functionality modularity, structuredness, reuse ....

maintainability reliability .....

PROCESSES DesignTest

time, effort, #spec faults found time, effort, #failures observed ....

stability cost-effectiveness ....

RESOURCES People Tools

age, price, CMM level price, size ....

productivity usability, quality ....

23

MEASUREMENT THEORY

24

Evolution of measures !  More sophisticated measures can be

defined as understanding of an attribute grows

!  E.g. temperature of liquids: " 200BC: rankings, “hotter than” " 1600: first thermometer still “hotter than” " 1720: Fahrenheit scale " 1742: Centigrade scale " 1854: Absolute zero, Kelvin scale

25

Measurement theory !  Scientific basis to determine formally:

" When we have defined an actual measure " Which statements involving measurement

are meaningful " What the appropriate scale type is " What types of statistical operations can be

applied to measurement data

26

Empirical relation system !  A set of entities !  The relations which are observed on

entities in the real world which characterize our understanding of the attribute in question " e.g. ‘Fred taller than Joe’ (for height of

people)

!  The closed operations that can be performed on the objects

27

Measurement mapping !  Mapping from the empirical relation

system onto a formal relation system " Measure " Relation mapping

!  A.ka. representation, homomorphism !  Measure: the number (formal element)

assigned to an entity in order to characterize an attribute

28

Measurement mapping

29

Joe

Fred

Joe IS TALLER THAN Fred

Height

M(Joe) = 180

M(Fred) = 172

M(Joe) > M(Fred)

Representation condition !  Measurement mapping implies that all

empirical relations are preserved in formal (numerical) relations and no new relations are created " e.g. M(Fred) > M(Joe) precisely when Fred

is taller than Joe !  Admissible measure if the

representation condition holds " Measurement scale

30

MEASUREMENT SCALES

31

Issues !  Representation problem

" How do we know if a particular empirical relation system has a representation in a given numerical relation system?

!  Uniqueness problem " How do we deal with several possible

alternative representations (scales) in the same numerical relation system?

!  Pragmatic problem " Which is the preferred numerical relation

system for a given empirical relation system?

32

Relation System richness !  RSA is richer than RSB if all relations in

RSB are contained in RSA

!  The richer the empirical system the more sophisticate the scale

33

Scale types

! Nominal ! Ordinal ! Interval ! Ratio ! Absolute

34

Rich

ness

Admissible measure !  Measure that is able to represent all the

empirical relations " There may exist several admissible measure

–  E.g. Length: inch, cm, feet, meter, miles !  Admissible transformation

" Mapping between two admissible measures " The more sophisticated the scale the more

restricted the class of admissible transformation

–  E.g. Admissible Length transformation: M’ = a*M –  Inadmissible transformation: M’ = a*M + b

35

Nominal scale !  Places elements in classification

scheme !  Empirical scale: different classes

" No ordering relation !  Any distinct numbering or symbolic

representation is acceptable " No notion of magnitude

36

Nominal scale example !  Empirical system

" Entity: person " Attribute: origin

–  Italy, EU, Extra-EU

!  Admissible mapping " M(p) =

– 1 if p is from Italy – 2 if p is from any EU country – 3 if p is from a non EU country

37

101

3.1

5

I

E

X

Nominal scale example !  Empirical system

" Entity: fault " Attribute: location

– Specification, design, code

!  Admissible mapping " M(x) =

– 1 if x is a specification fault – 2 if x is a design fault – 3 if x is a code fault

38

101

3.1

5

S

D

C

Ordinal scale !  Empirical system: classes ordered wrt

the attribute !  Acceptable mapping: any mapping

preserving the order " Measure represent ranking only " Acceptable transformations are the set of

all monotonic mappings " <C1, C2, .. Cn> # <a1, a2, .. an> " Where ���i>j, ai > aj

39

Ordinal scale example !  Empirical system

" Entity: statement " Attribute: agreement

– Completely disagree, Mostly disagree, Mostly agree, Completely agree

!  Admissible mapping " M(x) =

– 1 if x is Completely disagree – 2 if x is Mostly disagree – 3 if x is Mostly agree – 4 if x is Completely agree

40

-2

-1

1

10

Interval scale !  Empirical system: order and

differences between classes !  Acceptable mappings: preserve order

and difference " Addition and subtraction make sense " The ratio makes no sense

!  Acceptable transformations are affine transformations " M’ = a * M + b

41

Interval scale example !  Empirical system

" Entity: project activity " Attribute: time from start

– Months since the beginning of the project – Years since funding available

!  Admissible transformation " PM counts month since project start:

– April 1, 2010 " CEO counts years since received funds:

–  January 1, 2011 " MPM = 12*MCEO + 9

42

Ratio scale !  Preserves ordering, size of intervals, and

ratios between entities !  There is a zero element

" Represents total lack of attribute " Measurement starts at zero and increases at

equal intervals: called units " All arithmetic can be applied meaningfully to

classes in the range of the mapping !  Admissible transformation

" Ratio transformation " M’ = a*M

43

Ratio scale example !  Empirical system

" Entity: person " Attribute: age

– Years, Months

!  Admissible transformation " MMonths = a * MYear

– Where a = 12

44

Absolute scale !  Measurement made simply counting

items in the entity set " Number of occurrences " Only one possible mapping " All arithmetic analysis is meaningful

45

Absolute scale (counter)examples !  Empirical system

" Entity: project " Attribute: full time staff

– Number of full time developers

!  The attribute definition implies the items to be counted! " Length is not measurable on an absolute

scale, # of lines it is " Age is not measurable on absolute scale

46

Scales Scale Admissible

Transformations Example

Nominal 1-to-1 mapping Labeling, classifying entities

Ordinal Monotonic increasing function

Preference, hardness

Interval M’ = a*M+b With: a>0

Relative time, temperature

Ratio M’ = a*M With: a>0

Time interval, length

Absolute M’ = M Counting entities 47

Meaningful statements !  A statement involving measurement is

meaningful if its truth is invariant of transformation of allowable scales

48

Meaningful statements !  Statements

" The number of errors discovered during the integration testing was at least 100

" The cost of fixing each error is at least 100 " A semantic error takes twice as long to fix as

a syntactic error " A semantic error is twice as complex as a

syntactic error

49

?

✓

✓

Meaningful statements? !  Fred is twice as tall as Jane

!  The temperature in Tokyo today is twice that in London

!  The difference in temperature between Tokyo and London today is twice what it was yesterday

50

Statistical operations !  Central tendency

51

Type Mean Median Mode Nominal ✗ ✗ ✓ Ordinal ✗ ✓ ✓ Interval ✓ ✓ ✓ Ratio ✓ ✓ ✓ Absolute ✓ ✓ ✓

Objective vs. Subjective !  Objective measures do not depend on

the environment or the person collecting the measure " A small portion of subjectivity cannot be

avoided !  Subjective measures depend on the

context where they are collected " Can change according to the person " They reflect the perception and judgment of

the person performing the measurement

52

MANAGEMENT ACCOUNTING

53

54

Accounting !  Accounting

" Focuses on cost, revenues, cash flow, investment, capital

" Financial accounting – Public data, accounting standards and laws – Historical perspective

" Management accounting – Private, sensible data – Fit for use of company/managers (no

standards)

55

Management accounting !  Cost accounting

"  Budget and actual cost of operations, processes, departments, products

"  Analysis of variances and profitability

!  Direct and indirect costs " Direct: directly traceable to a product/service

sold to customer "  Indirect: all others

!  Fixed and variable costs "  Fixed: do not depend on number of units produced "  Variable: depend on the

number of units

56

Activity based costing !  A technique in management accounting !  Developed to overcome problems in direct/

indirect costs "  Traditionally, indirect costs were attributed

proportionally to all products –  Ex, direct cost 100, indirect cost 40%

"  Since indirect costs grow, proportional allocation hides costs of some products

–  Ex. Product1 consumes much more design or manufacturing than product2, true cost of product1 is higher

57

Activity based costing !  Activity based costing

" From indirect (taxes, administration, security) to direct costs (traceable to product or service)

" Allocates cost of each activity/resource to product and services in function of actual consumption

" Aims at knowing true cost of product/service, identify profitable ones, define selling costs

CSF Critical Success Factors

58

59

CSF !  Critical Success Factors !  Concept

" Few areas (4-5) in a company are responsible for business success (failure)

" They should be monitored constantly !  CSF refer to internal areas, not

objectives or targets " Ex. company objective: be market leader " Ex. Company target : acquire 25% of

market share by ..

[Rockart 79]

60

CSF - levels !  CSF exist at different levels, !  Following organization structure

" Corporate " Function

– Production, product design, etc. " Role

– Manager

61

CSF - example !  Corporate

"  Brand recognition, image "  Dealers network "  Equipment of cars "  Reliability of cars "  After sales service

!  Function (manufacturing) "  Production costs "  Quality of product "  Environment issues "  Relationship with trade unions

!  Manager (quality manager) "  Reputation w.r.t. other functions/roles "  Skills of technicians "  Process certification "  Technology for monitoring quality

CSF - Types !  Industry / Domain

"  the structure of the particular industry !  Strategy

" competitive strategy, industry position, and geographical location

!  Environment "  the macro environment

!  Temporal / Contingency " problems or challenges to the organization

!  Management " management perspective

62

63

CSF objective !  To identify in top down mode essential

information for managers " Cfr. bottom up: starting from currently

available information " Cfr. Information vs. data

!  To define / modify a reporting system within IS

64

CSF method 1.  Identify candidate CSFs 2.  Interview managers and identify

indicators 3.  Indicator robustness analysis 4.  Refinement, presentation,

implementation

65

1) Identify candidates !  For each candidate CSF

" Level (corporate, function, role) " Name " Type " Description

CSF Examples !  Business domain

" Key areas for all companies in same business domain

" Ex: cost for PC manufacturers, skill of personnel for consulting companies

!  Competitive factors within business domain " Factors that differentiate company from

others " Ex: for airlines, low cost vs. quality of service

66

CSF Examples !  Environmental factors

" Constraints from outside such as norms, rules, standards

" Ex.: for car manufacturers, euroX pollution norms

!  Contingency factors " Temporary constraint " Ex.: merge IS of two companies after

financial merge/acquisition " Ex.: recover brand reputation after failures

(see Benz Class A)

67

CSF – Politecnico (teaching) !  Corporation: brand recognition

" Business domain factors (teaching) " Competitive factors

– 1 Teaching quality – 2 Level of service (canteen, sport, housing)

–  2.a administrative services – 3 Cost (w.r.t. universities in the world) –  (degree offering)

" Environmental – Transports, weather

68

CSF – Politecnico (teaching) !  Corporation

" Contingent – DM 509 to DM 270

!  Departments " Teaching quality " Costs (professors)

!  Central administration " Services " Costs (infrasctructure)

69

CSF.1: Teaching quality (corp) !  University Ranking !  Delay to first employment

" Goal: few weeks !  % of graduate with a job within 1 year

" Goal 100% !  Salary after 1 year, 5 years

70

Example: Restaurant CSF CRITICAL SUCCESS

FACTOR TARGET

Customer Satisfaction 83% extremely satisfied

Market Share 20% of 10 mile radius

Employee Turnover 25% per year Food Quality 5% returned meals

71

Example: Non-profit Org. CSF

CRITICAL SUCCESS FACTOR TARGET

Number of Donors! 15,000 monthly donors!People Served! 2000 per month!Volunteers! 350 active volunteers!Customer Satisfaction! 92% extremely satisfied!

72

Factor 1, quality of education !  KPI

" Customer (studente) satisfaction " Ritardo sulla laurea medio " Voto medio esami " Voto laurea medio

73

CSF.2a service level, corp !  Administrative services

" Enrollment " Tax payment " Exam booking " Registrazione esame

– KPI: LT grade recording – KPI: unitary cost of recording

74

75

2) Interview managers !  Using output of step 1 interview each

manager !  Each one proposes CSF for function

and role (max 5), and measures (1 or more) for each CSF

!  Output " For each CSF

– Measures, source of (raw) data, rationale

2) Output Indicator Unit Source Motivation

Cost Unit product cost EUR ERP Key factor for

production process competiveness Unit overhead EUR ERP

Quality Production defect Ratio ERP

Measure of objective quality

Support defect Ratio Post-sale IS

76

2) Output

77

Indicator Unit Source Motivation

Quality

Customer judgment Score Sample

interviews Perceived quality

Comparison to competitors Score

Test and sample interviews

Actual gaps

Comparison with historical data Score Test Achieved

improvements

Environ

Produced waste Tons Ad-hoc measure Key for green

image of organization Recyclable

materials Tons Ad-hoc measure

Energy consumption Kw Ad-hoc

measure Energy efficiency

3) Indicator Robustness !  Understandability !  Processing cost !  Significance !  Frequency !  Structuredness !  Robustness

78

79

3) Indicator Robustness

80

3) Indicator Robustness !  Comprehensibility / Understandability

"  How simple !  Processing Cost

"  Cost and delay to process "  Cost and delay to collect raw data "  Initial and incremental

!  Significance / Meaningfulness "  How much the indicator covers the CSF

!  Frequency "  How often indicator varies

!  Structuredness "  How much the indicator is objective/not ambiguous

81

Robustness

Indicator Unde

stan

d.

Cost

Rele

vanc

e

Freq

uenc

y

Stru

ctur

e

Robu

stne

ss

Unit direct cost 5 4 5 5 5 4.8

Unit overhead 4 5 4 2 4 3.8

Production defects 4 5 4 5 5 4.6

Support defects 5 4 4 4 5 4.4

Customer rating 5 2 2 3 3 3.0

Competitor comparison 2 3 5 3 3 3.2 Past comparison 4 3 3 2 3 3.0 Waste processing 4 3 3 5 4 3.8 Recyclable materials 4 3 3 3 4 3.4 Energy consumption 2 5 4 3 5 3.8

83

4) Refinement, presentation !  Refinement

" Aggregate, simplify CSFs !  Presentation

" And acceptance from managers !  Implementation

" Define requirements and design for IS "  Implement

KPI - KEY PROCESS INDICATOR

84

85

KPI !  Process perspective

" Cfr CSF, focuses on areas – May correspond to process but in general

wider and cross processes " Cfr financial indicators (traditional

management accounting), focus on finance only

" Cfr. SLA (service level agreement), ITIL, focuses on process

86

KPI

Request

for a

service

Output (product

s and services)

Organization 1

Organization 2

Organization 3

Activity 1

Activity 2

Activity 3

Activity 4

!  Process view "  Involves one or more hierarchical nodes " Financial and non financial indicators " Process as a chain of services

87

KPI description !  Name

"  Average delay to satisfy order (DO) "  Average productivity of resource

!  Definition "  How the KPI is computed

–  DO11: 30 / number of orders satisfied in month –  Computed per calendar month: jan, feb, ..

–  DO12: 30 / number of orders satisfied in month –  Computed every day, for last 30 days

–  DO2: 365 / number of orders satisfied in year –  DO3: averagei( delay to satisfy order i)

"  Measurement unit !  Type

"  actual, target, estimate, benchmark

SMART KPI !  Specific purpose for the business, !  Measurable !  Achievable !  Relevant !  Timely

88

KPI

89

" Ex. Average delay to satisfy order " Ex. Average productivity of resource

Request

for a

service

Output

(products and

services)

Organization 1

Organization 2

Organization 3

Activity 1 Activity 2

Activity 3

Activity 4 Service KPI

Quality KPI

Efficiency KPI

KPI General framework

90

91

KPI – general framework !  General

"  Input volume "  Output volume "  Human resources "  Non human resources (plants, machines, facilities) "  Inventory "  Other resources

!  Efficiency "  Cost per unit "  Productivity of resources "  Utilization of resources

!  Quality "  Conformity "  Reliability "  Customer satisfaction

!  Service "  Response time, Lead time "  On time "  Perfect orders "  Flexibility toward customer

General KPIs – examples

92

General KPIs Hotel reservation

Lift maintenance

Product sale supermarket

Book sale on web

Building license (e.gov)

Input volumes # reservation requests, modif, delete

#urgent requests, #normal requests

#sales (person passing at register)

# orders #licences requested

Output volumes

#reserved rooms

#services completed

#invoices #products sold

# shippings # books sold

#licences issued

Human resources

#full time, part time employees

#personnel for maintenance (technical)

#personnel (at cash register, security)

#personnel for sales and distribution

#employees

Material resources

Call center, reservation system, workstations, supplies

Reservation and dispatch system, tools

Sales building, storage building, products

CRM, call center, web site, storage building

Supporting IS

Inventory #rooms -- #products on shelves #books --

Other resources -- -- -- -- Laws

93

Efficiency KPIs !  Cost per unit

" Total cost/ I/O volume !  Productivity

" Volume/resource !  Utilization

" Used resource /available resource

Volumes and Resources refer to any general # indirect measures

Efficiency KPI

94

Unit cost Productivity Utilization

Input Cost per unit of input -- --

Output Cost per unit of output

Human resources -- Output/ #

employees Used / Available

Non human resources -- Output/ resource

(ex # machines) Used capacity / available capacity

Inventory -- Sales/ stock Load factor

Time -- Time to produce/output

Time to service/ total time

Information Amount information/ output -- Amount info / total

amount information

Efficiency KPI

95

Unit cost Productivity Utilization

Input Cost per unit of input -- --

Output Cost per unit of output

Human resources -- Output/ #

employees Used / Available

Non human resources -- Output/ resource

(ex # machines) Used capacity / available capacity

Inventory -- Sales/ stock Load factor

Time -- Time to produce/output

Time to service/ total time

Information Amount information/ output -- Amount info / total

amount information

Design and Sw Industries

Depends on industry

Inventory turnover

Immaterial sales e.g. plane seats

Exercise !  List efficiency KPIs for the following

processes " Hotel reservation " Product sale supermarket

96

Hotel Reservation Unit cost Productivity Utilization

Input/output -- --

Human resources --

Non human resources --

Inventory --

Time --

Information --

Hotel Reservation Unit cost Productivity Utilization

Input/output

Total cost / # reservation reqs Total cost / # reserved rooms

-- --

Human resources --

#reservation reqs/ #employees #reserved rooms/ #employees

Time servicing/shift duration

Non human resources -- #reservation reqs /

#workstations #hours worked(call center)/24hrs

Inventory --

Time -- Distribution of requests per hours

Information --

Product sales Unit cost Productivity Utilization

Input/output -- --

Human resources --

Non human resources --

Inventory --

Time --

Information --

Efficiency KPI - examples

100

Hotel reservation Lift maintenance

Product sale supermarket

Book sale on web

Building licence (e-gov)

Cost per unit Total cost (call center)/ # reservation requests Total cost (call center)/ # reserved rooms

# Totalcost(registers)/# person passing at register Totalcost(registers)/# products sold Totalcost(registers)/# invoices

Productivity = Volume/Resources

#reservation request/ #employees #reserved rooms/ #employees

#Invoices/#employees #products sold/ #employees

Utilization= #hours worked(call center)/24hrs #hours worked(human resource)/8hrs

#hours worked(register)/open hours for shop #hours worked(human operator)/8

101

Efficiency KPI !  Volume input - Volume output/

Volume input

" Reservation request/rooms sold " Person passing at register/ invoices

" Products sold/invoices

102

Quality KPIs !  Conformity

"  With defined service/product description "  Non conform items/total # items "  Items

–  Input requests (from customer) –  Intermediate output –  Final output (defects, complaints from customer)

!  Reliability "  Probability that product /system satisfies its function after

time T "  MTTF – mean time to failure "  MTTR – mean time to repair "  MTBF – mean time between failures (= MTTF + MTTR)

!  Customer satisfaction "  Satisfaction through interviews/questionnaires

–  Qualitative scales (very high, high ..)

Quality KPIs

103

Input Internal Output

Conformity Non conforming requests

Number discarded Reject ratio Rework cost/total costs

Complaints Non conformity to to requests, contract, or expectations

Reliability -- MTTF MTBF MTTR

MTTF MTBF MTTR

Satisfaction -- --

Satisfied customers ratio Evaluation of product/service

Quality KPIs

104

Input Internal Output

Conformity Non conforming requests

Number discarded Reject ratio Rework cost/total costs

Complaints Non conformity to to requests, contract, or expectations

Reliability -- MTTF MTBF MTTR

MTTF MTBF MTTR

Satisfaction -- --

Satisfied customers ratio Evaluation of product/service

Depends on industry

Judgment collected through polls

Exercise !  Define quality KPI for

" Hotel reservation process

105

Quality KPIs

106

Hotel Reservation

Input Internal Output

Conformity

Reliability --

Satisfaction -- --

Quality KPIs

107

Hotel Reservation

Input Internal Output

Conformity #reservations with problem/ #reserved rooms

#cancelled reservations/ #reserved rooms

Complaints from customers

Reliability -- #lost reservations/ #reserved rooms

Satisfaction -- -- Customers’ opinion

Service KPIs !  Response time (supplier pov), Lead time (customer pov)

"  Time to satisfy order, from reception to delivery of good/service –  To be checked in peak periods

!  Punctuality "  delay = actual lead time - nominal lead time "  Average delay "  #delayed orders

!  Perfect orders "  On time and within specifications

!  Flexibility towards customer "  # modified orders/ total # orders "  value modified orders/ total value of orders

–  It is NOT internal flexibility = how internal resources can respond to changes in mix/number of orders

108

109

Service KPIs - example Hotel reservation Lift

maintenance

Product sale supermarket

Book sale on web

Building license (e.gov)

Lead time (customer view, with queues) Response time (producer view, no queues)

T customer service = t end call – t response T operator = t end call – t operator answer

T customer service = t invoice issued – t begin queue T cashier service = t invoice issued – t first product scan

T license granted – t request issued T license granted - T begin processing

Punctuality t actual issue – t promised issue

Flexibility

Perfect orders

110

Processes and stakeholders !  Process has several stakeholders

" Operator " Manager "  Customer

!  Process (and consequently KPIs) should be designed by considering all stakeholders "  Ex cost

–  Cost for operator: work fatigue –  Cost for manager: financial cost –  Cost for customer: price tag + cost for finding ordering

and obtaining the product

111

KPIs and stakeholders Cost Quality Service

Operator •  T non value

activity / T total •  T occupied / T total •  T info access

•  Conformance and internal reliability (System error rate)

•  Operator satisfaction

•  System response time by Operator process

Manager •  Unit cost •  Resource

Productivity •  Resource saturation •  Time saturation

•  Conformance (input & output quality) •  Internal reliability (MTBF, MTTR) •  Customer satisfaction

Customer •  Price / Supplier

cost •  Time and cost to

get product or service

•  Conformance to request

•  Product/service reliability

•  Satisfaction

•  Response time, lead time

•  Timeliness •  Perfect orders •  Flexibility

112

KPI – steps 1.  Select processes to monitor 2.  For each process, select KPIs 3.  Profile KPI 4.  Robustness and CSF 5.  Dimensions 6.  Requirements and design

113

1.Select processes !  Starting from models

" SCOR " AP " Business domain specific

114

2.Select KPIs !  Using KPI templates

" General " Cost " Quality " Service

115

3.Profile KPI !  For each KPI detail

" Name " Goal " Metric definition

– how computed " Unit of Measure " Scale "  Interpretation " Source

– where the data comes from

116

4.Robustness !  Are KPIs covering strategic areas?

" Cross check with CSF !  Are KPIs robust?

" Same attributes as for CSF – Comprehension – Cost – Meaningfulness – Frequency – Structure

117

4.Indicator Robustness !  Comprehensibility / Understandability

"  How simple !  Processing Cost

"  Cost and delay to process "  Cost and delay to collect raw data "  Initial and incremental

!  Significance / Meaningfulness "  How much the indicator covers the CSF

!  Frequency "  How often indicator varies

!  Structuredness "  How much the indicator is objective/not ambiguous

118

5.Dimensions !  Or segmentation:

" Entities to which indicator is associated and therefore

" Data the indicator can be aggregated on

119

Common dimensions !  Time window

"  Sales per hours/per day/per month .. !  Hierarchical node in organizational – geographical

structure "  Sales per country/per region/per shop "  Expenses per company/per division/per group/per person

!  Product / product category "  Sales per phone xy / per business phones

!  Customer/ customer category !  Activity in process

"  Cost per design / production "  Defects from design/ from production

!  Project "  Cost per project "  Defects per project

120

6.Requirements and design !  Define requirements

" Use cases, class diagram etc !  Define design !  For supporting IS

KPI vs CSF !  CSF

" vital for a strategy to be successful. " drives the strategy forward, it makes or

breaks the success of the strategy " “Why would customers choose us?”

!  KPI " quantify management objectives

– along with a target or threshold " enable the measurement of strategic

performance.

121

Examples !  IBM Intelligent Operations Center

" http://www-01.ibm.com/software/industry/intelligent-oper-center/

122

BALANCED SCORECARDS

123

124

Balanced scorecards !  Financial perspective on company

performance is limited !  Managers can only partially act on

financial outcome of a company !  Better to focus on more perspectives !  [Kaplan and Norton1992]

125

Perspectives !  Financial

"  To succeed financially how should we appear to our shareholders?

!  Customer "  To achieve our vision, how should we appear to

our customers? !  Internal process

"  To satisfy our shareholders and customers, what business processes must we excel at?

!  Innovation and learning "  To achieve our vision, how will we sustain our

ability to change and improve?

Perspectives

126

4

Balanced Scorecard for Performance Measurement

Figure 1 shows the original structure for the Balanced Scorecard (BSC). The BSC retains

financial metrics as the ultimate outcome measures for company success, but supplements these

with metrics from three additional perspectives – customer, internal process, and learning and

growth – that we proposed as the drivers for creating long-term shareholder value.

Figure 1: Translating Vision and Strategy: Four Perspectives

Vision andStrategy

Objectives Measures Targets InitiativesFINANCIAL

“To succeed financially, how should we appear to our shareholders?”

Objectives Measures Targets InitiativesLEARNING AND GROWTH

“To achieve our vision, how will we sustain our ability to change and improve?”

Objectives Measures Targets InitiativesCUSTOMER

“To achieve our vision, how should we appear to our customers?”

Objectives Measures Targets InitiativesINTERNAL BUSINESS PROCESS

“To satisfy our shareholders and customers, what business processes must we excel at?”

1.1. Historical Roots: 1950-1980

The Balanced Scorecard, of course, was not original for advocating that nonfinancial

measures be used to motivate, measure, and evaluate company performance. In the 1950s, a

General Electric corporate staff group conducted a project to develop performance measures for

Indicators !  Objectives !  Measure !  Target !  Initiative

127

128

Perspectives and indicators !  Financial

"  Cash flow "  Return on investment "  Financial result "  Return on capital invested "  Return on equity

!  Customer (the value proposition) "  Customer satisfaction " Market share " Quality "  Service

129

Perspectives and indicators !  Internal process (that deliver the customer value

proposition) "  Number of activities "  Opportunities success rate "  Accident ratios "  Manufacturing indicators (loading, availability,

performance quality) !  Innovation and learning

"  Investment rate "  Illness rate "  Internal promotions % "  Employee turnover "  Gender ratios

Dashboard BSC

Financial perspective -turnover -ROI

Customer perspective

Internal processes Innovation and learning

130

131

How to implement !  Define vision, translate into

operational goals !  Communicate vision, link to individual

performance !  Business planning !  Feedback

132

License (1) !  THE WORK (AS DEFINED BELOW) IS PROVIDED UNDER THE TERMS OF THIS CREATIVE COMMONS PUBLIC LICENSE ("CCPL"

OR "LICENSE"). THE WORK IS PROTECTED BY COPYRIGHT AND/OR OTHER APPLICABLE LAW. ANY USE OF THE WORK OTHER THAN AS AUTHORIZED UNDER THIS LICENSE OR COPYRIGHT LAW IS PROHIBITED.

!  BY EXERCISING ANY RIGHTS TO THE WORK PROVIDED HERE, YOU ACCEPT AND AGREE TO BE BOUND BY THE TERMS OF THIS LICENSE. THE LICENSOR GRANTS YOU THE RIGHTS CONTAINED HERE IN CONSIDERATION OF YOUR ACCEPTANCE OF SUCH TERMS AND CONDITIONS.

!  1. Definitions –  "Collective Work" means a work, such as a periodical issue, anthology or encyclopedia, in which the Work in its entirety in

unmodified form, along with a number of other contributions, constituting separate and independent works in themselves, are assembled into a collective whole. A work that constitutes a Collective Work will not be considered a Derivative Work (as defined below) for the purposes of this License.

–  "Derivative Work" means a work based upon the Work or upon the Work and other pre-existing works, such as a translation, musical arrangement, dramatization, fictionalization, motion picture version, sound recording, art reproduction, abridgment, condensation, or any other form in which the Work may be recast, transformed, or adapted, except that a work that constitutes a Collective Work will not be considered a Derivative Work for the purpose of this License. For the avoidance of doubt, where the Work is a musical composition or sound recording, the synchronization of the Work in timed-relation with a moving image ("synching") will be considered a Derivative Work for the purpose of this License.

–  "Licensor" means the individual or entity that offers the Work under the terms of this License. –  "Original Author" means the individual or entity who created the Work. –  "Work" means the copyrightable work of authorship offered under the terms of this License. –  "You" means an individual or entity exercising rights under this License who has not previously violated the terms of this

License with respect to the Work, or who has received express permission from the Licensor to exercise rights under this License despite a previous violation.

2. Fair Use Rights. Nothing in this license is intended to reduce, limit, or restrict any rights arising from fair use, first sale or other limitations on the exclusive rights of the copyright owner under copyright law or other applicable laws.

3. License Grant. Subject to the terms and conditions of this License, Licensor hereby grants You a worldwide, royalty-free, non-exclusive, perpetual (for the duration of the applicable copyright) license to exercise the rights in the Work as stated below:

a.  to reproduce the Work, to incorporate the Work into one or more Collective Works, and to reproduce the Work as incorporated in the Collective Works;

b.  to distribute copies or phonorecords of, display publicly, perform publicly, and perform publicly by means of a digital audio transmission the Work including as incorporated in Collective Works;

The above rights may be exercised in all media and formats whether now known or hereafter devised. The above rights include the right to make such modifications as are technically necessary to exercise the rights in other media and formats, but otherwise you have no rights to make Derivative Works. All rights not expressly granted by Licensor are hereby reserved, including but not limited to the rights set forth in Sections 4(d) and 4(e).

133

License (2) !  4. Restrictions.The license granted in Section 3 above is expressly made subject to and limited by the following

restrictions: a.  You may distribute, publicly display, publicly perform, or publicly digitally perform the Work only under the terms of this

License, and You must include a copy of, or the Uniform Resource Identifier for, this License with every copy or phonorecord of the Work You distribute, publicly display, publicly perform, or publicly digitally perform. You may not offer or impose any terms on the Work that alter or restrict the terms of this License or the recipients' exercise of the rights granted hereunder. You may not sublicense the Work. You must keep intact all notices that refer to this License and to the disclaimer of warranties. You may not distribute, publicly display, publicly perform, or publicly digitally perform the Work with any technological measures that control access or use of the Work in a manner inconsistent with the terms of this License Agreement. The above applies to the Work as incorporated in a Collective Work, but this does not require the Collective Work apart from the Work itself to be made subject to the terms of this License. If You create a Collective Work, upon notice from any Licensor You must, to the extent practicable, remove from the Collective Work any credit as required by clause 4(c), as requested.

b.  You may not exercise any of the rights granted to You in Section 3 above in any manner that is primarily intended for or directed toward commercial advantage or private monetary compensation. The exchange of the Work for other copyrighted works by means of digital file-sharing or otherwise shall not be considered to be intended for or directed toward commercial advantage or private monetary compensation, provided there is no payment of any monetary compensation in connection with the exchange of copyrighted works.

c.  If you distribute, publicly display, publicly perform, or publicly digitally perform the Work, You must keep intact all copyright notices for the Work and provide, reasonable to the medium or means You are utilizing: (i) the name of the Original Author (or pseudonym, if applicable) if supplied, and/or (ii) if the Original Author and/or Licensor designate another party or parties (e.g. a sponsor institute, publishing entity, journal) for attribution in Licensor's copyright notice, terms of service or by other reasonable means, the name of such party or parties; the title of the Work if supplied; and to the extent reasonably practicable, the Uniform Resource Identifier, if any, that Licensor specifies to be associated with the Work, unless such URI does not refer to the copyright notice or licensing information for the Work. Such credit may be implemented in any reasonable manner; provided, however, that in the case of a Collective Work, at a minimum such credit will appear where any other comparable authorship credit appears and in a manner at least as prominent as such other comparable authorship credit.

d.  For the avoidance of doubt, where the Work is a musical composition: i.  Performance Royalties Under Blanket Licenses. Licensor reserves the exclusive right to collect, whether

individually or via a performance rights society (e.g. ASCAP, BMI, SESAC), royalties for the public performance or public digital performance (e.g. webcast) of the Work if that performance is primarily intended for or directed toward commercial advantage or private monetary compensation.

ii.  Mechanical Rights and Statutory Royalties. Licensor reserves the exclusive right to collect, whether individually or via a music rights agency or designated agent (e.g. Harry Fox Agency), royalties for any phonorecord You create from the Work ("cover version") and distribute, subject to the compulsory license created by 17 USC Section 115 of the US Copyright Act (or the equivalent in other jurisdictions), if Your distribution of such cover version is primarily intended for or directed toward commercial advantage or private monetary compensation.

–  Webcasting Rights and Statutory Royalties. For the avoidance of doubt, where the Work is a sound recording, Licensor reserves the exclusive right to collect, whether individually or via a performance-rights society (e.g. SoundExchange), royalties for the public digital performance (e.g. webcast) of the Work, subject to the compulsory license created by 17 USC Section 114 of the US Copyright Act (or the equivalent in other jurisdictions), if Your public digital performance is primarily intended for or directed toward commercial advantage or private monetary compensation.

134

License (3) !  5. Representations, Warranties and Disclaimer !  UNLESS OTHERWISE MUTUALLY AGREED BY THE PARTIES IN WRITING, LICENSOR OFFERS THE WORK AS-IS AND MAKES NO

REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE WORK, EXPRESS, IMPLIED, STATUTORY OR OTHERWISE, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF TITLE, MERCHANTIBILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, ACCURACY, OR THE PRESENCE OF ABSENCE OF ERRORS, WHETHER OR NOT DISCOVERABLE. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO SUCH EXCLUSION MAY NOT APPLY TO YOU.

!  6. Limitation on Liability. EXCEPT TO THE EXTENT REQUIRED BY APPLICABLE LAW, IN NO EVENT WILL LICENSOR BE LIABLE TO YOU ON ANY LEGAL THEORY FOR ANY SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR EXEMPLARY DAMAGES ARISING OUT OF THIS LICENSE OR THE USE OF THE WORK, EVEN IF LICENSOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

!  7. Termination a.  This License and the rights granted hereunder will terminate automatically upon any breach by You of the terms

of this License. Individuals or entities who have received Collective Works from You under this License, however, will not have their licenses terminated provided such individuals or entities remain in full compliance with those licenses. Sections 1, 2, 5, 6, 7, and 8 will survive any termination of this License.

b.  Subject to the above terms and conditions, the license granted here is perpetual (for the duration of the applicable copyright in the Work). Notwithstanding the above, Licensor reserves the right to release the Work under different license terms or to stop distributing the Work at any time; provided, however that any such election will not serve to withdraw this License (or any other license that has been, or is required to be, granted under the terms of this License), and this License will continue in full force and effect unless terminated as stated above.

8. Miscellaneous a.  Each time You distribute or publicly digitally perform the Work or a Collective Work, the Licensor offers to the

recipient a license to the Work on the same terms and conditions as the license granted to You under this License. b.  If any provision of this License is invalid or unenforceable under applicable law, it shall not affect the validity or

enforceability of the remainder of the terms of this License, and without further action by the parties to this agreement, such provision shall be reformed to the minimum extent necessary to make such provision valid and enforceable.

c.  No term or provision of this License shall be deemed waived and no breach consented to unless such waiver or consent shall be in writing and signed by the party to be charged with such waiver or consent.

d.  This License constitutes the entire agreement between the parties with respect to the Work licensed here. There are no understandings, agreements or representations with respect to the Work not specified here. Licensor shall not be bound by any additional provisions that may appear in any communication from You. This License may not be modified without the mutual written agreement of the Licensor and You.