The Use Of Systems Engineering Tools To Improve Processes in

Healthcare:

Six Sigma and Lean Improvement Methodologies as they Apply to

Healthcare

The Use Of Systems Engineering Tools To Improve Processes in

Healthcare:

Six Sigma and Lean Improvement Methodologies as they Apply to

Healthcare

D M A I CSpeaker Contact info Speaker Contact info

Muhanad Hirzallah Quality Improvement Black Belt Quality Management Services

Mayo Clinic200 First St. SW

Rochester, MN, USA(507) 266-3216 (phone)

(507) 255-9689 (fax)Hirzallah.Muhanad@mayo.edu

D M A I CEngineering the Healthcare System Engineering the Healthcare System

D M A I CMayo Clinic’s Quality Experience Mayo Clinic’s Quality Experience• Process Improvement Section within the

Quality Management Services Department

• The Quality Academy: now part of Mayo Clinic College of Medicine.

• Mayo Clinic’s System – Wide Initiatives:– IRB review cycle time

• Process Improvement Section within the Quality Management Services Department

• The Quality Academy: now part of Mayo Clinic College of Medicine.

• Mayo Clinic’s System – Wide Initiatives:– IRB review cycle time

D M A I CThe IRB experienceThe IRB experience

Under federal regulations, an Institutional Review Board (IRB) is a constituted group formally designated to review and monitor research involving human subjects.

Its’ work must comply with the Federal Wide assurance agreement with the government in order to receive federal funding for research.

Delays in making a decision may result in loss of funding and delays in patients being offered access to potential new treatments.

Under federal regulations, an Institutional Review Board (IRB) is a constituted group formally designated to review and monitor research involving human subjects.

Its’ work must comply with the Federal Wide assurance agreement with the government in order to receive federal funding for research.

Delays in making a decision may result in loss of funding and delays in patients being offered access to potential new treatments.

D M A I CProblem & GoalProblem & GoalThe IRB turnaround time for protocol review has been identified as a source of dissatisfaction by the researchers and the IRB Staff.

On average, the review of protocols took 38 days. The customers expect a more predictable and timely turnaround.

The goal is to reduce the protocol turnaround time to no more than 21 days, while maintaining the IRB responsibility to uphold the regulations.

The IRB turnaround time for protocol review has been identified as a source of dissatisfaction by the researchers and the IRB Staff.

On average, the review of protocols took 38 days. The customers expect a more predictable and timely turnaround.

The goal is to reduce the protocol turnaround time to no more than 21 days, while maintaining the IRB responsibility to uphold the regulations.

D M A I C

Team members:

Project Sponsor : Chair, Research AdministrationProcess Owner : IRB Medical Director Core Team: IRB Administrator

Measurement Coordinator / IRB Operations Manager

Project ManagerProcess Improvement Expert

Team members:

Project Sponsor : Chair, Research AdministrationProcess Owner : IRB Medical Director Core Team: IRB Administrator

Measurement Coordinator / IRB Operations Manager

Project ManagerProcess Improvement Expert

D M A I CResultsResults

Metrics: Before AfterAverage Cycle time 38d 16d- 21 days compliance 27% 83%

For new submissionFull board

Average CT (w/out PI) 31d 11d- 21 days compliance 38% 100%

For new submissionFull board

Metrics: Before AfterAverage Cycle time 38d 16d- 21 days compliance 27% 83%

For new submissionFull board

Average CT (w/out PI) 31d 11d- 21 days compliance 38% 100%

For new submissionFull board

D M A I C

DMAIC MethodologyDMAIC Methodology

DDefineefine::

MMeasureeasure::

AAnalyzenalyze::

IImprovemprove::

CControlontrol::ControlControl

ImproveImprove

AnalyzeAnalyze

MeasureMeasure

DefineDefine

D M A I C

Project CharterProject Charter• An explanation for the project• Goals and desired results in measurable

terms • Project plan and milestones• Roles and responsibilities

• An explanation for the project• Goals and desired results in measurable

terms• Project plan and milestones• Roles and responsibilities

D M A I CSurvey of Investigators Survey of Investigators

Ove

rall

sati

sfac

tion 531

5

3

1

531 531

531 4.53.52.5 432

5

3

1

4.53.52.5

5

3

1

Quality Speed Clarity

Communication Expertise Consistency

Training

Scatterplot of Overall Satisfaction vs the Dimensions

Ove

rall

sati

sfac

tion 531

5

3

1

531 531

531 4.53.52.5 432

5

3

1

4.53.52.5

5

3

1

Quality Speed Clarity

Communication Expertise Consistency

Training

Scatterplot of Overall Satisfaction vs the Dimensions

D M A I C

Potential Impact for the CustomerPotential Impact for the Customer• Patients / Participants

Greater access to clinical trials• Investigator

Improved satisfaction• Industry

Improved satisfaction• IRB

Enhanced complianceImproved Morale

• Federal AgenciesEnhanced compliance

• Patients / ParticipantsGreater access to clinical trials

• InvestigatorImproved satisfaction

• IndustryImproved satisfaction

• IRBEnhanced complianceImproved Morale

• Federal AgenciesEnhanced compliance

D M A I C

High Level Process MappingHigh Level Process MappingResearch Team submits protocol

IRB Intake person assigns a board

IRB Specialist reviews protocol & assign reviewer

IRB meets/ discusses/ decides

IRB Board member reviews protocol

IRB specialist communicates decision

D M A I CMeasureMeasure

DDefineefine::

MMeasureeasure::

AAnalyzenalyze::

IImprovemprove::

CControlontrol::ControlControl

ImproveImprove

AnalyzeAnalyze

MeasureMeasure

DefineDefine

D M A I C

Value Stream Mapping ToolValue Stream Mapping Tool

Examine the flow of information and workLocate the largest sources of wasteEnvision a less wasteful stateDevelop plans for future activities

D M A I CCurrent State Value Stream Map Current State Value Stream Map

High Error Rate in processing

detected after 5 days

Rework on submissions take

about 6 days

2-week Board Cycle means

waiting time of at least 10-14 days

Minute Excerpts take about 5 days

to go out to PIs

Board capacity not well defined, leads

to inequitable distribution

Double layer ‘intake’ results in

redundancy

D M A I C

Results of the Current State Value Stream Map:Results of the Current State Value Stream Map:

Value Added Ratio: 1% - 1.43%

Total Wait Time: 38 daysTotal Process Time: 169.4 - 245.4 min

0.38 - 0.55 daysTotal Lead Time (Turnaround): 38.38 - 38.55 days

D M A I C

18.2%

80.4%

1.4%

VA NVA - E NVA

D M A I CIRB WastesIRB Wastes

TYPES TYPES OFOF

WASTEWASTE

I

C

O

MW

P

T

Correction / Defects

OverProduction

Motion

Transportation/Material

Movement

Waiting

Inventory

Over Processing

Mis-utilization of skillsRe-prioritization

D M A I C

Data CollectionData Collection

• # of IRB submissions• # of submission outside the 21d target.• Time to process a new submission• Errors

• # of IRB submissions• # of submission outside the 21d target.• Time to process a new submission• Errors

D M A I CAnalyzeAnalyze

DDefineefine::

MMeasureeasure::

AAnalyzenalyze::

IImprovemprove::

CControlontrol::ControlControl

ImproveImprove

AnalyzeAnalyze

MeasureMeasure

DefineDefine

D M A I C

Analysis for New SubmissionsAnalysis for New Submissions• Working d/yr =

365 - weekends- PTO= 230 d/ yr• Annual Demand of new full protocols for yr 2005

= 345 • Daily Demand =

345/230= 1.5 protocols/day• Work time a day =

(8 hrs X 60 min) – (2 X 15 min Breaks) = 450 minutes

• Working d/yr = 365 - weekends- PTO= 230 d/ yr

• Annual Demand of new full protocols for yr 2005 = 345

• Daily Demand = 345/230= 1.5 protocols/day

• Work time a day = (8 hrs X 60 min) – (2 X 15 min Breaks) = 450 minutes

D M A I C

• Takt* = ( 450 min/ day) / (1.5 protocols/ day) = 300 min/ protocol

• Variation adjusted Takt = (.85)X(300)= 255 min

*The pace (German)

• Takt* = ( 450 min/ day) / (1.5 protocols/ day) = 300 min/ protocol

• Variation adjusted Takt = (.85)X(300)= 255 min

*The pace (*The pace (GermanGerman))

Analysis for New SubmissionsAnalysis for New Submissions

D M A I COperator ChartOperator Chart

0

50

100

150

200

250

300

Time

Time

Time 110 61.3 60

Specialist Assistant Reviewer

Un-Utilized Capacity

D M A I C

Analyze PhaseAnalyze Phase

• Intake person has a very low yield (i.e. quality)

• IRB has more capacity than the demand• RCA for errors required.

Conclusion: Opportunities for improvement

• Intake person has a very low yield (i.e. quality)

• IRB has more capacity than the demand• RCA for errors required.

Conclusion: Opportunities for improvement

D M A I CImproveImprove

DDefineefine::

MMeasureeasure::

AAnalyzenalyze::

IImprovemprove::

CControlontrol::ControlControl

ImproveImprove

AnalyzeAnalyze

MeasureMeasure

DefineDefine

D M A I C

Brainstorm ImprovementsBrainstorm Improvements

• Eliminate the Intake function • IRB Specialist reviews 1st • Standardize work elements.• Mistake proofing to reduce errors.• Communicate with the Research audience• Continuously improve

Seek feedback from customersAnalyze data

• Eliminate the Intake function • IRB Specialist reviews 1st • Standardize work elements.• Mistake proofing to reduce errors.• Communicate with the Research audience• Continuously improve

Seek feedback from customersAnalyze data

D M A I C

Standardize + waste

elimination

Standardize + waste

elimination

Enhanced board

functioning

Enhanced board

functioning

Future State mapFuture State map

Error reduction/ knowledge

management/ training

Error reduction/ knowledge

management/ training

Communication plan

Communication plan

PDC format improvement PDC format

improvement

D M A I C

Future State map resultsFuture State map results

7 days178.3 min/ 0.4 days7.4 days5.40%

Total Processing:Total Turnaround Time:Value added ratio:

Total Wait: 7 days178.3 min/ 0.4 days7.4 days5.40%

Total Processing:Total Turnaround Time:Value added ratio:

Total Wait:

D M A I C

0.0% 5.4%

94.6%

VA NVA - E NVA

D M A I C

Validate Improvement – ComplianceValidate Improvement – Compliance

Improvement over time

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

Baseline 9/25 -9/29

10/2 -10/6

10/9 -10/13

10/16 -10/20

10/23 -10/27

10/30 -11/3

11/6 -11/10

11/13 -11/17

3/19-3/23 3/26-3/30 4/2 - 4/6

Time

Perc

enta

ge

Improvement

D M A I C

Validate Improvements – Error FrequencyValidate Improvements – Error Frequency

Error Frequency over time

0

5

10

15

20

25

30

35

Application/Information

Clarity

Adim/Clerical Problems

SponsorProblems

Multi SiteProblems

IRBeProblems

ConsentForm

Problems

FDAProblems

BudgetProblems

SubCommitteeProblems

Type of Error

Freq

uenc

y

9/27 - 10/6 10/6 - 10/27 10/30 - 11/13 11/13 - 12/6

Error Frequency over time

0

5

10

15

20

25

30

35

Application/Information

Clarity

Adim/Clerical Problems

SponsorProblems

Multi SiteProblems

IRBeProblems

ConsentForm

Problems

FDAProblems

BudgetProblems

SubCommitteeProblems

Type of Error

Freq

uenc

y

9/27 - 10/6 10/6 - 10/27 10/30 - 11/13 11/13 - 12/6

D M A I CPareto Chart for Errors Pareto Chart for Errors

Count 20 17 5 3 2 2 2Percent 39.2 33.3 9.8 5.9 3.9 3.9 3.9Cum % 39.2 72.5 82.4 88.2 92.2 96.1 100.0

Coun

t

Perc

ent

ErrorsOthe

r

FDA P

roble

ms

Adim

/Cler

ical P

roble

ms

Sub C

ommitte

e Pro

blems

Budg

et Pr

oblem

s

Cons

ent F

orm Pr

oblem

s

Appli

catio

n/Info

rmati

on Cl

arity

50

40

30

20

10

0

100

80

60

40

20

0

Pareto Chart for Errors (10/6 - 10/27)

D M A I CAverage Days Turnaround of New Full Board Protocols

Average Days Turnaround of New Full Board Protocols

Average Turnaround Time (without PI delay)

31

23

16

11

0

5

10

15

20

25

30

35

Baseline Sept /Oct 06 Nov. 06 Apr. 07

Period

Day

s

D M A I C

ControlControl

DDefineefine::

MMeasureeasure::

AAnalyzenalyze::

IImprovemprove::

CControlontrol::ControlControl

ImproveImprove

AnalyzeAnalyze

MeasureMeasure

DefineDefine

D M A I CControl ChartsControl ChartsIRB Full Board Control Chart

0.00

5.00

10.00

15.00

20.00

25.00

30.00

07-00119 406-00550 306-00940 407-00052 906-00630 806-00700 807-00249 707-00109 207-00242 807-00148 707-00247 407-00233 807-00114 706-00941 007-00161 107-00268 507-00009 907-00154 607-00078 806-00953 707-00168 206-00577 307-00252 406-00643 706-00565 407-00076 307-00241 007-00113 606-00557 006-00945 807-00071 006-00665 607-00016 106-00526 907-00206 707-00131 407-00130 406-00885 207-00041 907-00254 907-00122 807-00316 1

IRB Delay (DAYS) UCL LCL CL USL

D M A I CControl PhaseControl Phase

• Consistent and improved process • Within control: Control Charts

• Control plan–Measurement System Assurance: Including monitor (Alerts) to ensure the process success. –Reaction plan.

• Consistent and improved process • Within control: Control Charts

• Control plan–Measurement System Assurance: Including monitor (Alerts) to ensure the process success.–Reaction plan.

D M A I CLessons to shareLessons to share• DMAIC proved to be effective in Healthcare

processes including administrative/ business processes.

• Research Administration, as a result launched 3 system-wide process improvement teams and established their own “Research Quality Office” recently.

• Other institutions benchmark with Mayo IRB

• DMAIC proved to be effective in Healthcare processes including administrative/ business processes.

• Research Administration, as a result launched 3 system-wide process improvement teams and established their own “Research Quality Office” recently.

• Other institutions benchmark with Mayo IRB

D M A I CLessons to shareLessons to share• Staff resistance to change!

• Information Technology (IT) support

• Process owners commitment is critical to the success of the initiative.

• The right skills on the team.

• Customer and External effect.

• Staff resistance to change!

• Information Technology (IT) support

• Process owners commitment is critical to the success of the initiative.

• The right skills on the team.

• Customer and External effect.

D M A I C

Thank You.Thank You.

Questions?Questions?