NG BB 28 MEASURE Tollgate

Black Belt Measure

Tollgate Briefing

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

Project NameDEPMS Project Number

Name of Belt

OrganizationDate

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

2

Measure Tollgate Requirements

PROJECT DELIVERABLES NGB COMMENTS

MEASURE

Current State Detailed Process Map / VSM Mandatory VSM (value stream map)

Key Input, Process, and Output Metrics Mandatory What to measure

Operational Definitions Mandatory What to measure

Data Collection Plan Mandatory How, who, when, etc

Baseline Statistics Mandatory Center, variation

Control Charts Recommended Varies by project

Process Capability Analysis Mandatory Ability to meet cust req

Measurement System Analysis Mandatory Accuracy of data

Estimated Financial & Operational Benefits Mandatory

Storyboard / A3 Mandatory 1-page proj summary

Barriers/Issues/Risks Mandatory

Quick Wins Recommended

Lessons Learned Optional

NG CPI BLACK BELT TOLLGATE REQUIREMENTS

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

3

NOTE: THIS IS A TEMPLATE FOR ALL NG CPI BELTS

NG has developed this template as a basic format with standard deliverables to

help guide NG CPI belts through the NG tollgate requirements for certification.

It is recognized that each project is unique and has unique deliverables with

unique flows. Therefore, this format does not have to be followed exactly to the

letter of the law for your project.

(DELETE THIS SLIDE FOR YOUR PROJECT)

Measure Tollgate Templates

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

4

Project CharterProblem Statement:

Business Case:

Goal Statement:

Unit:

Defect:

Customer Specifications:

Process Start:

Process Stop:

Scope:

Project TimelinePhase Start Stop Status

Define mm/dd/yy mm/dd/yy

Measure mm/dd/yy mm/dd/yy

Analyze mm/dd/yy mm/dd/yy

Improve mm/dd/yy mm/dd/yy

Control mm/dd/yy mm/dd/yy

Take away

message goes

here

(impact of

problem)

Team Members

Name Role Affiliation DACI

Black Belt Driver

Master Black Belt Driver

Project Sponsor Approver

Process Owner Approver

Required Deliverable

Charter and Timeline

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

5

- Example -

Detailed “As Is” Process Map

Required Deliverable - VSM or Process Map or Both

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

6

Order Mgmt Supervisor

CUSTOMER

DISTOrder MgmtOrder MgmtOrder Mgmt Order Mgmt

Screen for Acct Mgr

Order Mgmt

P/T = 2 min

Error Rate=2%

Volume=800

P/T = 6 Min

Error Rate=0%

Volume=800

P/T = 6 Min

Error Rate=2%

Volume=800

P/T = 2 Min

Error Rate=1%

Volume=800

20 Orders

3 min

Phone Call

Large

Business

Home

6 Customers

5 Customers

3 Customers

Small

Business

Customer

Info4

Product

Need4Pricing

4

Shipping

Info4

P/T = 3 min

Lost calls=10%

Volume=1200

Pick

Pack & Ship

P/T = 120 Min

Error Rate=1%

Volume=1200

10

2 min 6 min 6 min 2 min 120 min

240 min5 min

Customer call time = 24 min

Service lead time = 384 min

SUPPLIERSManual Update

Weekly Update

Phone Call

Trigger:

Completion Criteria:

Cycle Time:

Takt Time:

Number of People:

Number of Approvals:

Items in Inbox:

% Rework:

# of Iterations (cycles):

# of Databases:

Top 3 Rework Issues:

1.

2.

3.

- Example -

Required Deliverable - VSM or Process Map or Both

Value Stream Map

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

7

Key Input, Process, and Output Metrics

Suppliers Inputs Process Outputs Customer

Start

Step1Step 2 Step 3

Step 4Step 5

Input Metrics Process Metrics Output Metrics

Quality

Speed

Cost

VOC/

VOB


UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

8

Operational Definitions

Define each of the Key Input, Output, Process Metrics from your SIPOC that you are going to collect data on (via the Data Collection Plan) as well as any other terms that need clarification for the data collectors and everyone else on the team.

Examples:

Award Process PLT: The time from when a Director submits the Award recommendation to the time when the employee is presented the Award in a ceremony.

Number of Claims Processed: The number of Claims processed per weekday (M-F).

Total Hours Worked: The total number of hours worked in the facility including weekends and holidays.

Number of Personnel: The total number of military and civilian personnel working (not including contractors).

Include other unique terms that apply to your project that require clear operational definitions for those collecting the data and for those interpreting the data.


UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

9

Data Collection PlanPerformance Measure

Operational Definition

Data Source and

Location

How Will Data Be Collected

Who Will Collect Data

When Will Data Be

Collected

Sample Size

Stratification Factors

How will data be used?

Ability to update projects and build tollgate reviews

X1

– Steps to update projects

In DEPMS By counting steps Name ASAP 1 None To find VA, BNVA, NVA

Ability to update projects and build tollgate reviews

X2

– Tollgate template slides that match POI

In DEPMS By determining % of activity steps identified in “Introduction to _____” modules in POI that are adequately addressed in templates

Name ASAP 40 None To determine consistency with POI

Easy Access to LSS tools and references

X3 – Availability of

LSS tools and references

In DEPMS By determining the percentage of tools, with their references, listed on DMAIC Road Map slides that can be found in PS

Name ASAP 63 None To determine availability of tools and references

Easy Access to LSS tools and references

X4

– Steps required to find tools and references

In DEPMS By counting # steps required to find the tools and their references

Name ASAP 37 None To find VA, BNVA, NVA


- Example -

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

10

Measurement Systems AnalysisThe Measurement System used to collect data has been calibrated and is considered to have no potential for significant errors. The data collection tool is reliable, can be counted on, has good resolution, shows no signs of bias and is stable.

Type of Measurement

ErrorDescription Considerations to this Project

Discrimination (resolution)

The ability of the measurement system to divide measurements into “data categories”

Work hours can be measured to <.25 hours. Radar usage measure to +- 2 minute.

BiasThe difference between an observed average measurement result and a reference value

No bias - Work hours and radar start-stop times consistent through population.

Stability The change in bias over timeNo bias of work hours and radar usage data.

Repeatability The extent variability is consistentNot an issue. Labor and radar usage is historical and felt to be accurate enough for insight and analysis.

ReproducibilityDifferent appraisers produce consistent results

Remarks in usage data deemed not reproducible, therefore were not considered in determining which radars were used in each op

Variation The difference between parts N/a to this process.Required Deliverable

- Example -

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

11

Measurement Systems Analysis

The Measurement System is acceptable with the Total Gage R&R % Contribution <10%

Percent

Part-to-PartReprodRepeatGage R&R

100

50

0

% Contribution

% Study Var

Sam

ple

Range 0.10

0.05

0.00

_R=0.0417

UCL=0.1073

LCL=0

1 2 3

Sam

ple

Mean

10.00

9.75

9.50

__X=9.7996

UCL=9.8422

LCL=9.7569

1 2 3

Part

10987654321

10.00

9.75

9.50

Operator

321

10.00

9.75

9.50

Part

Average

10 9 8 7 6 5 4 3 2 1

10.00

9.75

9.50

Operator

1

2

3

Gage name:

Date of study :

Reported by :

Tolerance:

Misc:

Components of Variation

R Chart by Operator

Xbar Chart by Operator

Response by Part

Response by Operator

Operator * Part Interaction

Gage R&R (ANOVA) for ResponseGage R&R

%Contribution

Source VarComp (of VarComp)

Total Gage R&R 0.0015896 3.70

Repeatability 0.0005567 1.29

Reproducibility 0.0010330 2.40

Operator 0.0003418 0.79

Operator*Part 0.0006912 1.61

Part-To-Part 0.0414247 96.30

Total Variation 0.0430143 100.00

Study Var %Study Var

Source StdDev (SD) (6 * SD) (%SV)

Total Gage R&R 0.039870 0.23922 19.22

Repeatability 0.023594 0.14156 11.38

Reproducibility 0.032140 0.19284 15.50

Operator 0.018488 0.11093 8.91

Operator*Part 0.026290 0.15774 12.68

Part-To-Part 0.203531 1.22118 98.13

Total Variation 0.207399 1.24439 100.00

Number of Distinct Categories = 7

- Example -Optional BB Deliverable

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

12

“As Is” Baseline Statistics

The current process has a non-normal distribution with the P-Value < 0.05

Mean = 44 days

Median = 22 days

Std Dev = 61 days

Range = 365 days


360300240180120600

Median

Mean

6050403020

1st Q uartile 12.000

Median 22.000

3rd Q uartile 52.000

Maximum 365.000

33.647 55.981

17.000 29.123

54.308 70.246

A -Squared 12.65

P-V alue < 0.005

Mean 44.814

StDev 61.251

V ariance 3751.674

Skewness 2.87329

Kurtosis 9.54577

N 118

Minimum 1.000

A nderson-Darling Normality Test

95% C onfidence Interv al for Mean

95% C onfidence Interv al for Median

95% C onfidence Interv al for StDev95% Confidence Intervals

Summary for Workdays

- Example -

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

13

Process Control Chart

The current baseline delivery time is stable over time with both the Moving Range (3.22 days) and Individual Average (29.13 days) experiencing common cause variation

255 data points collected with zero subgroups, thus the I&MR control chart selected

Observation

In

div

idu

al

Va

lue

2442171901631361098255281

40

35

30

25

20

_X=29.13

UC L=37.70

LC L=20.56

Observation

Mo

vin

g R

an

ge

2442171901631361098255281

10.0

7.5

5.0

2.5

0.0

__MR=3.22

UC L=10.53

LC L=0

I-MR Chart of Delivery Time

Required As Applicable- Example -

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

14

118 data points collected

Non-normal distribution

Mean = 44 days

Lower Cust Spec = 0 days

Upper Cust Spec = 15 days

65% of observations outside customer spec

Z Bench = -.31


- Example -

Process Capability

420360300240180120600

LSLUSL

LSL 0

Target *

USL 15

Sample Mean 44.8136

Sample N 118

Location 3.09501

Scale 1.26378

Process Data

Z.Bench -0.31

Z.LSL 3.07

Z.USL -0.02

Ppk -0.01

O v erall C apability

% < LSL 0.00

% > USL 65.25

% Total 65.25

O bserv ed Performance

% < LSL 0.00

% > USL 62.03

% Total 62.03

Exp. O v erall Performance

Process Capability of WorkdaysCalculations Based on Lognormal Distribution Model

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

15

Project Barriers/Issues/Risks

Barriers

Issues

Risks


UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

16

Measure StoryboardDefine

Measure

Sigma Performance Level of

1.3BUS CASE: Be #2 Fin Service Provider

GOAL: Reduce Loan/Lease CT from9.2 to 8.0 days by July 1

FIN IMPACT: $2.7M per year

Project Charter1.2 Day CCR Gap


UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

17

Company: Department: Date: Prepared by:

1. Define the problem situation

2. Problem Analysis

3. Action plans to correct problems

4. Results of Activity

5. Future Steps

8-Step A3 Project Summary Report

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

18

NG CPI Tollgate Tool

Control Charts Process Capability

Process Capability High Level Process Map (SIPOC)

(Finance Owner)(Finance Owner)

(MBB)(MBB)(MBB)(MBB)(MBB)

(Sponsor)(Sponsor)(Sponsor)(Sponsor)(Sponsor)

I accept the ControlTollgate

I accept the Improve Tollgate

I accept the Analyze Tollgate

I accept the MeasureTollgate

I accept the Define Tollgate

(Process Owner)(Process Owner)(Process Owner)(Process Owner)(Process Owner)

Project Charter• Problem Stmt• Defect Definition• Goal Statement• Project Scope• Business Impact• Strat Alignment

Detailed As Is Process Map

Value Stream Map Key Process Metrics

Potential Xs• Brainstorming• 5 Whys • Fishbone • Affinity Diagram

“To Be” Process Map

Solution Generation / Prioritization

Process Control Plan

Process Owner Accountability

Sponsor & Team

Replication Check

Data Collection Plan

Measure Systems

Analysis

Data Collected

Critical Xs • Cause & Effect Matrix• Hypothesis Testing• Regression• Time Studies• Theory of Constraints

Improvement Strategy• Improvement Model• Implementation Plan• Pilot • “X” Improvement Target

Updated Financial Benefits

Replication Opportunities

Measurable Y

Voice of Customer

Customer Specs

Voice of Business

Baseline Data Analysis

• Descriptive Stats

• Graphs

• Pareto Charts

FMEA• Risk Analysis • Risk Mitigation Plan

Mistake Proofing Project Documentation of revised policies, SOP’s, procedures, and training

Project Timeline

Communication Plan• Stakeholder Analysis

Est Financial Benefits

Control Charts (as needed)

FMEA• Risk Analysis • Risk Mitigation Plan

Visual Process Control Tools (Optional)

1. ValidateProblem

4. Find Root Cause

3. Set Targets

5. Develop C-Ms

6. See C-Ms Through

2. IdentifyGaps

7. Confirm Results

8. Standardize

Define Measure Analyze ControlImprove

8-STEP PROCESS

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

19

Appendix

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

20

Team MembersName Role Affiliation DACI

Black Belt Driver

Master Black Belt Driver

Project Sponsor Approver

Process Owner Approver

Contributor

Contributor

Contributor

Contributor

Inform

Inform

Inform

Inform


Cross Functional Team

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

21

I confirm that:

DEPMS (DoD Enterprise Performance Management System) has been searched for similar projects: Yes / No

Replication: List relevant initiatives / potential replication projects found (if any):

• Project 1: (DEPMS # or other tracking tool project number)

• Project 2:

Collaboration: Identify organizations that can/should be considered for working this project collaboratively:

• Organization 1:

• Organization 2:Required Deliverable

Replication Check

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

22

Strategic Alignment

The Define Tollgate requires a linkage to organizational strategy.

Include an organizational metric/metrics for which your project will help improve

Refer to your organization’s Strategic Plan and/or other referenced documents


UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

23

Business Impact

Insert as much information as possible about the potential operational and/or financial benefits

Include any assumptions upon which these estimates are based

Example: Operational benefits – This project is expected to reduce PLT by 35%, improve SQL from 1.2 to 3.0, save 20 man hours per shift

Example: Financial benefits – This project is expected to save $xx in FYxx


UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

24

Measurable Y:


Suppliers Inputs Process Outputs Customers

High-Level Process Map (SIPOC)

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

25

Voice of the Customer /

Key Customer Issue(s)Critical Customer

Requirement

What does the customer want from us? What does the customer want from us? We need to identify the issue(s) that

prevent us from satisfying our customers.

We should summarize key issues and translate them into specific and measurable

requirements


Voice of Customer / Voice of Business

Voice of the Business

Key Process Issue(s)CriticalBusiness

Requirement

What does the business want/need from us? What does the business want/need from us? We need to identify the issue(s) that prevent us from meeting strategic

goals/missions.

We should summarize key issues and translate them into specific and measurable

requirements

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

26

Stakeholder

Name/Group

Project Impact

On Stakeholder

(H, M, L)

Stakeholder

Level of

Influence on

Success of

Project (H,M,L)

Stakeholder’s

Current Attitude

Toward Project

( +, 0, - )

Explanation of

Current

Stakeholder

Attitude

(list)

Stakeholder

Score

(H=3, M=2, L=1,

+=3, 0=1, -=-3)

Action Plan

For

Stakeholder

Recommended Deliverable

Stakeholder Analysis

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

27

Audience Media PurposeTopics of

Discussion/

Key MessagesOwner Frequency Notes/Status

Communication Plan


Education

NG BB 28 MEASURE Tollgate