Performance Management Framework

Performance Management Framework

Pernilla Ulfvengren

KTH Royal Institute of Technology

pernilla.ulfvengren@indek.kth.se

Page 2MASCA

MASCA change process

Page 3

Identification of needs and goals

Plan and preparechange

Execute –Implement plan and

Monitor change

GAP

Review and Reinforce

GAP

Enablers for change•CMS toolbox•Capability•Competence

GAP

Develop / implement:

•Learning, Training, Mentoring•Enabling technologies•Organisational processes

Develop Enablers for ChangeHumans, Technology, Information Apply Enablers on IPER

Management control

• The purpose is to have intentional impact on operations, executives and personnel

• towards certain objectives.

4MASCA

Performance management

• The essential components of PM is to measure and monitor.

• Monitoring a process is to systematically measure it.

• Process control basics (Single or Double loop)

• Performance indicators:

“Indicators are proxy measures for items identified as important in the underlying model(s) of safety.

As such they are uncertain and often only distantly connected to idealised measures that are rarely available in practice”.

– A word of caution by Wreathall (2009)

Indicators objectives

measures of routine safety related activities

measures of failures (deviations) detected during these activities

measures of events (incidents) that occur

during operations

7MASCA

Safety Management System

• SMS (ICAO, 2009)

• Performance-based vs Outcome-based

• Integrated management system for safety

1. Safety policy

Safety model, management commitment, appointments, documentation etc.

2. Safety risk management

Hazard identification, Risk assessment and mitigation

3. Safety assurance

Safety performance monitoring and measurement, management of change, continuous improvement of the SMS

4. Safety promotion and training

Training, eduction, safety communication

Safety performance

• From ”outcome based” to ”performance based”

• Safety Performance Indicators, Targets and Requirements

• Effectiveness of the SMS, how well it performs and manage to improve the system.

• It’s change capability!

MASCA 9

Purpose of SPI’s

• Primarily:“Support of and control in the risk management process.”

• But also:“Support the relation with over-sight authorities.”

MASCA 10

Process and Performance

• What influence the process risk and performance?

• Where in process is it most likely safety is affected, risk emerge?

• Where in process might consequences of this be manifested?

• Classification of process influences using the SCOPE model:

EffectEffect

ContextPerson; Organisation; Environment

ContextPerson; Organisation; Environment

ProcessProcess

OrganisingCompetence; Procedures; Leadership; Coordination; Feedback; Automation

OrganisingCompetence; Procedures; Leadership; Coordination; Feedback; Automation

SupplyTeam

InformationMaterial

SupplyTeam

InformationMaterial

Effect

ContextPersonal; Organisational; Environmental

Process

OrganisingCompetence; Procedures; Leadership; Coordination; Automation; Feedback

SupplyTeam

InformationMaterial

SCOPE – Process & Performance Analysis

Process Performance Indicators

Safety Performance Management Framework

PROCEDURALPM

STRUCTURAL PM

MASCA/CMS/New HUMAN FACTORS model

Procedural framework, 8-step process

MASCA 14

3

4

5

6

7

8

1

2

Safety data dissemination loop1. A performance measure and monitoring framework with a model for key performance indicators.2. Capability to aggregate data to appropriate level for decision makers. 3. A PM framework for strategic decision making and priority.4. A PM framework to dis-aggregate the data triggering performance measure(s) and stakeholder(s).

Safety data driven change loop5. Develop appropriate enablers 6. Apply enablers to IPER process7. Coordinate and evaluate change with other ongoing change initiatives.

Evaluation and improvement of PM framework8. Evaluating current KPIs/SPIs using measuring and monitoring and operational performance data.

FSQB

CMS

Structural framework• Enabling technologies

– Information support technologies

– Statistical tools

– Data management (gathering, storing, retreiving) tool

– SAS’s Vision Monitor

• Process models

– Dependencies

– Process analysis, indicator identification

– SCOPE model and tool

• Organisational structure

• Indicators list, logic and weights

– List of indicators and measures

– Definitions and logic descriptions

– Weight and dependencies (antecedents, leading or lagging)

• Data model

– SPIs, KPIs, combined SPI’s/KPIs

– Indicators linked to operational process and system functionality

– Data mining model

– Statistical analyses models

Safety - Efficiency

Safety Measures (SPI’s) OF• Rejected takeoff high speed• Emergency• Takeoff warning• Runway/taxiway incursion• Runway /taxiway excursion• Altitude penetration• Hard Landing• GPWS mode 1-4• Stick shaker• Tail strike• Birdstrike• Birdstrike damage• TCAS• Wake turbulence• Significant weather• Airport facilities• Cabin safety

Normal operations (SPI/KPI) OF• Line check level 1• Line check level 2• Line check level 3• Cabin safety monitoring• Unstabilized approaches at 500 ft• Unstabilized approaches at 1000 ft• FDM, 200 kts / 2 mins• FDM, G• PC/OPC result (D• Punctuality P3 events (• Punctuality P15• Planning precision• Fuel efficiency index• Regularity• Operational costs

MASCA 16

Analysis methods

• Data mining and analysis applications

– Association analysis (Market basket)

– Correlation

– Regression

– Trend analysis

– Multi-attribute

– Comparative analysis

MASCA 17

Page 18

Thank you for your attention …