1 What Gets Measured Gets Undone Dr. Jim Mirabella Director of Institutional Research Professor of Statistics Florida Community College - Jacksonville (FCCJ) J. Michael Adams Corporate Manager, Quality Services Florida Power & Light/ FPL Group, Inc. Florida Sterling Conference Wednesday, May 31, 2000 Disney World, Orlando, Florida
2 Whats getting undone ? The desired outcome and supporting processes that the measure(s) intended to describe Why???? Poor measures drive poor performance!
3 Objectives of the workshop Know good measurements from poor measurements Distinguish the implications of good and poor measures on performance Balance an array of measures that favorably tracks a processs outcome for planning, assessing performance, and analysis Assess attendees measures Any other expectation?
4 Why Measure at All? Measurement is the language of progress and comparison Provides a sense of where we are AND where we are going (planning) Can guide a steady advancement toward established goals (tracking) Can identify goal shortfalls, or over- achievement (analysis) Communicates to the work force what is important to the organization (behavioral)
5 Good measures are born from SMART goals Specific Measurable Agreed upon Realistic Time-Bound
6 The Challenge of Measuring Workers might perceive it as a threat Workers might disregard organizational goals, customers, products and services Workers might focus on obtaining favorable measurements Measuring items with no influence on organizational success = waste of time result is a bean-counting approach that focuses on irrelevant details Expected to do something!
7 The Challenges- examples Apples and oranges Impact on comparisons, benchmarking and performance turnover consistency defects vs. defectives commutes (measured in time or miles) school achievement
8 Where are these measures Organizational Process Department/work unit Individual Exercise
9 Your measures Jot down your various performance measures for you, your department, unit and organization. Indicate the purpose of the measure ( 1= planning, 2= tracking and analysis; 3: behavioral changes) Organization Process Department/ Unit Individual
10 What Do We Measure? Quality Delivery Cycle Time Waste Cost Defects Satisfaction Complaints Financials Price
11 Measurement Examples 4 Operations-related measures -reliability -timeliness of delivery -order processing time -errors / defects -product lead time -inventory turnover -cost of quality -employee
12 Measurement Examples 4 Customer-related measures -customer satisfaction -customer complaints -customer retention 4 Financial measures -market share -sales per employee -return on assets -return on sales
13 Baldrige/ Sterling: Results Category Private, Education, Health Care 7.1: Customer Focus Results; Student Performance, Patient and other Customer Focus 7.2: Financial and Market; Student and Stakeholder Focused Results, 7.3: Human Resource Result; Budgetary and Financial Results, Staff and Work System 7.4: Supplier and Partner Results; Faculty and Staff Results 7.5: Organizational Effectiveness
14 Determining and Reviewing Measurements for Balance Baldrige Results 7.0 Customer Financial Human Resources Supplier/ Partner Organizational supplier dept process
16 Airline Measurement System Individual measures for each flight process Group measures for overall airline If problems occur with a flight, who do you blame -- the flight crew or the airline? What measures do you affiliate with the flight crew? What measures do you affiliate with the airline? What is important to you for a satisfactory flying experience?
17 Review your measures Are the organization measures reflective of all Baldrige results items? Do the measures reward favorable behaviors? Is there alignment with the contributing departments and suppliers? Are the overall results managed as an outcome of a process?
18 Whats the opportunity? Many organizational measurement systems are too short, too rigid, or used like a strict teachers ruler... to whack rather than to motivate Need to replace these outdated measurement systems with more dynamic measurement system that motivates continuous improvement in customer satisfaction, flexibility, and productivity SIMULTANEOUSLY
19 Mis-measurement Systems Unless specifically tuned to flight plan, measurement systems may: yield irrelevant / misleading information provoke behavior not conducive to strategy Traditional measures ignore requirements & perspectives of customers (internal/external) Bottom-line measures (profitability) too late for mid-course correction / remedial action
20 Mis-measurement Systems Many measurement systems overlook key non-financial performance indicators Measures often used for punishment rather than to promote learning Many measurement systems are inflexible and limited in what they can do
21 What Should a Measurement System Do? Measures must link operations to strategic goals departments should know how they contribute separately and together toward strategic mission System has to integrate financial / non- financial info in a way usable by managers managers need right info at right time Measurement systems real value lies in its ability to focus all business activities on customer requirements
22 What Should a Measurement System Do? Measure what is important to the customers Motivate operations to continually improve against customer expectations Identify and eliminate waste -- of both time and resources Help to accelerate organizational learning and build a consensus for change when customer expectations shift
23 Measurement Mismanagement In an effort to increase market share, a computer service bureau strategizes to improve the timeliness of its voicemail service. The goal is to provide new customer with service w/in 24 hours. To help speed the process, a program is developed to accept verbal phone orders. Most new customers were online w/in a day as promised, causing the company to celebrate their success. A later audit revealed 70% error rate in order entry, with 30% of customers disputing their bill and eventually canceling service..
24 Measurement Mismanagement A Hi-Tech company sets objective to become highly profitable by being a product leader. To measure the performance of its marketing and R&D functions, the number of new products developed is the most watched barometer. An internal review reveals that in a sample of 20 new product introductions, 80% were delivered over a month late, and significant waste piles up in production. Management cannot understand why the accountants ink is red after all, their yardstick tells them they are developing new products at a record rate.
25 Beware of .. Seemingly Simple Measures How satisfied are your customers? What is your employee turnover? What is your client retention rate? Is the value of our service worth the price?
26 Beware of .. Treating customer perceptions as objective measures Customer satisfaction product quality Customer satisfaction is a complex phenomenon A well-handled complaint results in higher customer satisfaction than does no complaint More reasons for complaint more dissatisfaction
27 Beware of .. Non-specific measurements Results are actionable Hard to improve what you cannot assess Failing to measure adequately Think BALANCED SCORECARD Dont give employees an outlet for gaming success Identify all areas important to customers
28 Beware of .. Using results incorrectly Dont tie results to employee pay unless employees can directly influence results Dont base employee pay on results that cannot be measured
29 Recap of Objectives and Expectations Know good measurements from poor measurements Distinguish the implications of good and poor measures on performance Balance an array of measures that favorably tracks a processs outcome for planning, assessing performance, and analysis Assess attendees measures