Welcome, Examiners!

Washington State Quality Award

Return Examiner Training

2010

First things first!

• THANK YOU!• WSQA’s mission: Improving the way we live, learn,

and work in Washington by helping organizations improve through the use of the Baldrige Criteria for Performance Excellence

2

Objectives For This Class• Learn changes to process• Be aware of upcoming changes from

National• Using Baldrige values to build key themes• Reinforce comment writing• Using the expected results• Reinforce scoring

3

Introductions

• Name, organization, what you do• Your experience with WSQA

4

Signed Ethical Statements

• Collect Signed Statement- You need to submit a signed ethics statement each year you review an application.

5

5/10/06 Washington State Quality Award

Council

6

Learning Opportunities

• WSQA offers workshops• Introduction• Preparation for Application

• Online self assessment tool• Annual conferences• Consulting• Performance Excellence Collaborative• Become an Examiner

Discuss the linkage between “Key Factors” and “Key Themes.”

Over the years, some of us have learned that when there are multiple questions in an “Area to Address” then generally the first question is more important than the second one, which is more important than

the third question, etc. Is this true?

Can you describe “work systems” more fully and

perhaps compare/contrast with “work processes” and “core

competencies.”

Are “projections” the same or different than “targets or goals?” (See page 58 of the criteria, which seems to say that projections can be targets.) Do performance projections impact the scoring in Cat. 7?

How does an Examiner assess innovation in an application?

How to determine if a results item should be given credit in

7.5 rather than 7.1?

What are the likely changes to the 2011-2012 criteria?

Describe the fact-based process the Board of Overseers uses in shaping and improving the Baldrige criteria.

Harry’s Session on Upcoming ChangesProcess Changes• Category 3: Rewritten to improve logic flow,

also to discuss more of the organization—less of the content

• Category 5: Consolidation/reorder to improve the logic flow

• Category 6: Reorganized around “Operations Focus” with better separation between 6.1 and 6.2. Examiners were struggling between Work Systems and Work Processes.

11

Upcoming Changes to Results

• Category 7: Will have only 5 items – 7.1 and 7.5 are to be combined and 7.4 will now have to address Governance and Societal Responsibilities

• Performance Projections will only be a factor up in the 90-100% scoring ranges. The “hows” will be expected to be addressed in Categories 2 and 4.

12

General Upcoming Changes

• The criteria will now “gently introduce” both topics of Social Media and Intelligent Risk.

• The sub-areas listed within the items will now have Titles to help improve the outlines and as a guide for better organization.

13

Key Theme – new process

14

What is a Key Theme?

A high level perception or observation, relevant to the applicant’s key factors, that recurs in the scorebook and is expanded upon with evidence during a site visit. Key themes are often common to more than one category or item. They may address one or more Baldrige core value.

A role model result, practice, process, or innovation, that might be used by other organizations, or an observation so important, strength or OFI, process or result, that it should be brought to leadership’s attention.

The executive summary of the feedback report. It may be the only part of the feedback report that some people in the organization read.

Cross Cutting Themes

• In the early days of the Baldrige, Key Themes were called Cross Cutting Themes, or Cross Cutting Issues, because they described topics that cut across the categories and items.

Core Values-The Foundation of the Baldrige Criteria

• Visionary Leadership• Customer driven excellence• Organizational and personal learning• Valuing workforce members and

partners• Agility• Focus on the future• Managing for innovation• Management by fact• Societal responsibility• Focus on results and creating value• Systems perspective

• If you think of the 7 Baldrige categories as the vertical beams in a house frame, the 11 Baldrige Core Values are the cross members that provide strength to the frame.

Process for using Baldrige Core Values to Develop Key Themes

• Each Team Member:• Review the applicant – Conduct individual review• Read the Baldrige Core Values• Write 11 post-its rating the applicant against each Baldrige Core

Value with a +, 0, or – and write a few short thoughts for each core value explaining why you rated the applicant as you did.

• Key Themes Lead• Sum the plusses and the minuses. Core Values that have many

plusses, or minuses, may be candidates for Key Themes. Typically only a few will stand out.

• Report results to team. • Draft Key themes for these high scoring values using feedback

report comments and post-it thoughts as inputs.

Using Core Values to ID Key ThemesBaldrige Core Value Score (+,0,-)

Visionary Leadership +,+,+,+,+,+ (6+) Probable KT

Customer driven excellence +,+,0,0,0,-

Organizational and personal learning +,+,+, 0,0,0

Valuing workforce members and partners +,+,+,+,+,- (5+) Possible KT

Agility +,+,0,0,-,-

Focus on the future +,+,+,0,-,-

Managing for innovation 0,0,-,-,-,-

Management by fact -,-,-,-,-,-, (6-) Probable KT

Societal responsibility +,+,+,+,+,0 (5+) Possible KT

Focus on results and creating value 0,-,-,-,-,- (5-) Possible KT

Systems perspective +,+,0,0,-,-

Caveat to this approach

• While the Baldrige Core Values are the foundation upon which the Criteria are based, they are not examination items. Key themes are written based on comments developed relating to the criteria. The feedback report comments MUST support the Key Themes.

WSQA Assessment Process

Objective:

Introduce the Assessment process

What Examiners Do…

Analyze Response

Read Criteria

Read Applicant Response

Meet with co-examiners to

consolidate analysis

Visit the Applicant to clarify/verify

Co-write an analysis report

Record Analysis

Preparing for independent review

• Read the criteria• Read the application• Familiarize yourself with the online toolhttp://www.wsqa.net/extraining.php• Check in with Team Lead or Mentor after preparing

a few Key Factors

23

Independent review:Process Items

Objective: Be able to diagnose a process item

for independent review

Independent review steps - process

1. Review related criteria2. Read the category under review, take notes3. Select relevant key factors from list4. Identify & record processes5. Record observations on worksheet under strengths

and Opportunities6. Indicate

• Approach• Deployment• Learning• Integration

25

Identify and record processes

• “Process” includes• Inputs• Steps (related activities)• Time frames• Outputs• End users (internal/external customers)• Standards• Key measures to evaluate and improve the process

• Not always formal or named – you might have to read between the lines

• Look to criteria for guidance

26

“Systematic” refers to processes that• Are well-ordered• Are repeatable• Use data and information so learning is possible• Build in the opportunity for evaluation,

improvement, and sharing

Identify and record processes

27

• “Effective” refers to how well a process or measure addresses its intended purpose

• Determining effectiveness requires evaluating• How well the approach is aligned with the organization’s

needs• How well it is deployed, and/or• How the evaluation of the outcome of its measures are

used, i.e., for learning and improvement

Identify and record processes

28

Indicating ADLI- Approach

• “Approach” refers to• Methods used to accomplish the process• How appropriate the methods are to the item

requirements• The effectiveness of their use• The degree to which the process is repeatable

and is based on reliable data and information (i.e., systematic)

29

Indicate “ADLI”- Deployment

“Deployment” refers to the extent to which• The approach is applied in addressing relevant

and important item requirements• It is applied consistently• It is used by all appropriate work units

30

Indicate “ADLI”- Learning

“Learning” refers to • Refining the approach through cycles of

evaluation and improvement• Encouraging breakthrough change to your

approach (innovation)• Sharing refinements and innovations with other

relevant work units and processes

31

Indicate“ADLI”- Integration

“Integration” refers to the harmonization of• Plans• Processes• Information• Resource decisions• Actions• Results• Analyses

to support key, organization-wide goals

32

Indicate ADLI- Integration

Questions to ask in analyzing “Integration” • Do the individual components of a performance

management system operate as a fully interconnected unit?

• Is the approach aligned with the organizational needs identified in the Organizational Profile?

33

ADLI-key concepts

Integration examples:

• Alignment of objectives and action plans with strategic challenges AND mission, vision, values

• Alignment of product/service delivery methods with KEY customer requirements

• Alignment of key process measures with KEY customer requirements

34