Risk-Based Commissioning & Qualification Benchmarking

Educational Track #1

AgendaPart 1 (8:30 – 10:00)

• ISPE Baseline® Guide Volume 5: Commissioning & Qualification vs. ASTM E2500 – Mr. Robert Chew

• Overview of C&Q approaches – Abbott, Lilly, Pfizer

Break – Vendor areas open (10:00 – 10:30)

Part 2 (10:30 – Noon) • Panel Q&A

Wrap-up

ISPE Baseline Guide Volume 5: Commissioning and Qualification, vs. ASTM E2500

Robert E. Chew, PE

President, Commissioning Agents, Inc.

My Background in Subject

• Official trainer for ISPE Baseline Commissioning and Qualification Guide

• Member of team that wrote ASTM E2500

Key Points to Compare

• Terminology

• Qualification practices

• Focusing the effort

• Roles and responsibilities

• Managing changes during the project

Terminology

• C&Q Baseline guide• Commissioning• Qualification

• ASTM E2500• Verification

Qualification Practices

• C&Q Baseline Guide• “To meet the demands of the regulators”

• ASTM E2500• Not mentioned

Focusing the Effort

• C&Q Baseline Guide• Impact assessments

• Direct impact systems and critical components

• V-model

• ASTM E2500• Risk assessment -> critical aspects

Roles and Responsibilities

• C&Q Baseline Guide• Quality pre and post approves protocols; quality control

role

• ASTM E2500• Quality involved in process requirements, risk

assessments, critical aspects, and acceptance and release

• Subject matter experts used throughout

Managing Change

• C&Q Baseline Guide• QA pre-approved change control

implemented at start of IQ/OQ

• ASTM E2500• QA pre-approved change control

implemented at start of manufacture of product for distribution

Common Elements

• Good engineering practices

• Planning

• Requirements definition

• Engineering design reviews

• Project change management

• Documented inspections and testing

Risk-Based Commissioning & Qualification Approach

Abbott LaboratoriesGlobal Pharmaceutical Operations

Steve Connelly

Overview

Where Abbott Is Today

C&Q Approach

Commissioning & Qualification Activities

How We Manage Cost

Lessons Learned

Where Abbott is TodayWhere Abbott is TodayRisk Based Maturity Model*Risk Based Maturity Model*

*Adapted from 2007 ISPE Tampa Conference Presentation

Commissioning & Qualification Approach C&Q Approach:

• Application of ISPE Baseline® Guide Volume 5: Commissioning & Qualification (last 2 years)

• Little to no implementation of ASTM E2500-07*• Level 3 (Risk-based Maturity Model)

Implementation:• Corporate policies support Science & Quality Risk

Management and includes Good Engineering Practice (GEP) reference

• Training ( RBA, GEP, ECM. Commissioning, Risk Assessments)

• Tools and Templates • Discretion of the Validation Review Board (VRB)

*ASTM E2500-07: A Standard Guide for the Specification, Design, & Verification of Pharmaceutical and Biopharmaceutical Manufacturing Systems and Equipment

Science & Quality Risk

Management Qualification

Process Flow

START

Does system directly impact

product quality?

Does system support a direct impact system?

Design Qualification

Installation Qualification

Evaluate Risk/Impact

Assessment

VCR Submittedw/Assessment

Report and Rationale

System have operational functions?

Yes

Operational Qualification

Support of CPPs tested in

PV?

Performance Qualification

CommissioningPlan

Pre-Qualification Engineering tests /FAT as applicable

VCR Submittedw/Assessment

Report and Rationale

Process Complete

VCR Closed

VCR Submittedw/Assessment

Report and Rationale

User Requirements

Functional Design

No

Yes

No

Yes

No

Yes

Engineering tests /FAT as applicable

CommissioningPlan

No

Direct Impact

Systems

Indirect Impact

Systems

No Impact

Systems

Good Engineering Practices Formal acknowledgement of GEP in corporate policy

Divisional GEP guideline consists of minimum requirements

Additional divisional guidelines are in development for life-cycle management

– Calibration

– Maintenance

– Engineering change management

Training to aid site support (implementation)

Requirements Based On Impact Category

*System = equipment, facility, utility

Three methods:–ISPE-Style System* Impact Assessment Questionnaire –Risk Assessment with any standard method (FMEA, HAZOP, etc.)

–Technical Evaluation by SMEs

•Documented and VRB approved•Determining impact to identifying critical elements

Quality Risk Control–Risk Assessment (i.e. FMEA, etc) used to enable:•Qualification focus on critical functions and/or components•Leverage limited commissioning testing (FAT/SAT) and •Eliminate some redundant activities (i.e. alarms, etc)

Commissioning When supporting Qualification, Commissioning must

include or address:• Engineering Change Management (ECM)

– After point commissioning data intended to support Qualification

• Testing– Methods described in sufficient detail for repeatability– Documentation confirms critical elements met criteria– Acceptance criteria documented prior to execution– Criteria based on User Requirements– Documentation of observed results

Commissioning & Qualification (C&Q) Plans typically contain run rules for project

Qualification – Equipment User requirements

• Product and Process User Requirement (PURS)• General User Requirements (GURS)

Commissioning (in support of qualification)• Test it once (or one set of multiple test repetitions, where

appropriate); document it correctly• Primarily installation activities, operational testing

leveraged in some instances (VRB discretion);compile, summarize test results for VRB approval

Qualification • When approved, commissioning reports complete the

IQ/OQ phase of Qualification with no additional “protocol” testing

Qualification – Software (APC/HMI/DCS)

Categorization (based on product quality impact)• Managing control issues for critical elements

• Supplier Assessments for direct impact systems only

• VRB discretion with technical advisory from Subject Matter Expert

Holistic Approach • Integrated Qualification document (IOQ for

equipment/controls; as applicable)

• APC specific qualification requirements

– Traceability matrix = Design Qualifications (DQ)

– Commissioning = Qualification except critical items

Qualification – Facilities

Indirect Impact Facilities - Commission

– Design and construction conforming to GEP

– Abbott Engineering Standards

Direct Impact Facilities – Commission/Qualify

– Meet quality specifications (i.e. Aseptic facility, etc)

– Design and construction conforming to GEP

– Abbott Engineering Standards

Qualification – HVAC Categorization

• Interdependent with facility quality attributes

– For example: Continuous monitoring for critical environmental aspects; therefore facility and HVAC can be Indirect Impact (HVAC controls would then be qualified)

– For example: Temperature, RH, and room pressure differentials typically critical elements which are monitored and recorded continuously, alarming when parameters exceed specifications; VRB discretion as to qualify

Qualification – Utilities

Impact Categories - Typically critical or non-critical

Utility systems typically not big source of Validation Change Request (VCR) sources

Only regulatory requirement concern is typically Potable Water feed to purification

– Purpose of pretreatment usually to preserve life of purification equipment rather than direct impact of water quality

How We Manage Cost…$ In 2007, reviewed C&Q costs for several capital projects

to:– Establish a baseline – Compare costs for the “traditional, validate everything” C&Q

approach versus the risk-based C&Q approach

$ Cost comparisons across projects difficult because– Inconsistent reporting and project scope varies widely

$ Drive cost reporting model for capital projects needed to: – Address scope differences, track additional parameters:

Number of Qualification Protocols per project Number of Qualified Systems Total Number of Systems

– Three parameters documented in C&Q Plan

• Monthly Capital Project Status Reports are used to track costs and progress

• There is a wide variation in the tracking systems used by the engineers for the project costs

• Most project engineers / project managers use spreadsheets of their own design to track costs

How We Manage Cost…Commissioning & Qualification Cost Tracking Details

Opportunities Identified for Consistent Reporting

Engineering/Commissioning/Qualification costs categorized below

Activities not linear, categorization to clarify “gray” areas of cost reporting

Defines appropriate cost “bucket” for consistency across projects

How We Manage Cost…

ENGINEERING COMMISSIONING QUALIFICATION (CERTIFICATION)FEP 1-3 Risk Assessment Validation PlanUser Requirements Failure Mode Effect Analysis Design QualificationFunctional Specification (Draft) I/O checkout Equipment Installation QualificationPreliminary Design Calibration Equipment Operational QualificationDetailed Design Functional Specification (Revisions) Computer Hardware Installation QualificationConstruction Factory Acceptance Testing Computer Software Installation QualificationLoop Tuning Startup Controller Modification QualificationEquipment Installation/Alignment Trouble Shooting HVAC Installation/Operational Qualification

Sequence Testing Utility Installation QualificationAs-Builts (Non Qualified Systems) Process Qualification

Validated DrawingsSamples/TestingCleaning/PassivationInspections

Engin

eering W

ork

Ord

ers

/ C

ontr

acts

Lessons Learned

Risk-Based C&Q is an iterative process (one tiny step at a time)

Corporate & divisional policies in place, tools developed, although success for various sites, concern about corporate & regulatory audits

Sites want to maintain autonomy with guidance; common elements, not same approach (i.e. all engineers comply however variations due to specialty)

Application: Site readiness tools - identifying opportunities for closing gaps

Lessons Learned

Listen, Speak, Deliver. Communication is key.

Quality may wonder: What if engineering doesn’t do its job?;ensure compliance

Engineering may wonder:Quality = police & spell checkers; “right” the first tyme

Application:

Regular/frequent communication with VRB & project team

Lessons Learned

Sustainable Paradigm Shift

Sites: Implementation, accountability & ownership a must for sustainability

Resources: People, time & money, site retaining technical knowledge

Application: Training & support implementation as applicable

Thank you!

• Questions will be addressed during the Panel Q&A session (after break)

Eli Lilly & Company

Commissioning and Qualification Approach

Eli Lilly & Company Approach to Commissioning & Qualification

• C&Q Delivery Strategy

• C&Q Metrics

• C&Q Lessons Learned

• Improved C&Q Delivery Strategy

Recent GFD C&Q Delivery Strategy~ Baseline Guide approach (’02 – ‘07)

• Developed procedures governing C&Q activities

• Fully commissioned all assets ~ GEP

• Reserved qualification for critical aspects of DI systems only

• Leveraged commissioning documentation to support qualification – commissioning subject to GDPs

• Involved Quality at C&Q strategy level

• In addition, • Simulation has been used to commission automation aspects• CSV testing has been integrated within C&Q test documents

Fundamental C&Q Delivery Strategy~ for Global Facilities Delivery (GFD) Capital Projects

Profile

RequirementsProcurement

Design

Qualification

Commissioning

Construction/Installation

Validation

Planning

Operations/ Maintenance

Project

Owner

System Classification

Component Classification

DQ/DR

Simulation/ Offline Testing

RV

IV

FAT

SATFT

IQ OQ PQ

Initial Cals

Startup

Commissioning Package

Construction QA

Approve CQMP

Approve System-level Plans

Loops

This strategy has resulted in:

• Earlier resolution of issues

• Opportunity to understand systems

• Reduced Qualification time

• Reduced Qualification costs

• Fewer Qualification comments/discrepancies

C&Q Delivery Strategy ~ Results

C&Q Metrics

• Schedule Metrics

• Cost Metrics

• Discrepancy Metrics

Typical C&Q Schedule Metrics ~ Durations

• 10 months from facility mechanical completion to OQ completion

• Total Commissioning duration: 24 mos. (172 days/system)

• Total IQ/OQ duration: 6 months

• FT duration/system: 42 days

• IQ duration/system: 11 days

• OQ duration/system: 17 days

• Average # of days from MC to OQ completion: 227 days

Note: Metrics for new powder filling facility consisting of approx. 40 systems

C&Q Cost Metrics ~ Historical view

No C&Q prog1998- 2001

Full C&Q prog2002- pres.

Part. C&Q prog2001-2003

C&Q Costas % TIC

40%

20%

10%5%

25%

10%5% GOAL3%

C&Q Budget Metrics ~ capital projects completed in ‘07

Total Installed Cost C&Q Cost (% TIC)$319MM 6.0%

$186MM 1.5% (comm. only)

$180MM 5.5%

$145MM 5.7% (incl. non-CQ activities)

$14.5MM 4.3%

$11.6MM 4.3%

$7.2MM 2.2%

Average: 4.7% (C&Q projects only)C&Q delivery strategy has resulted in a step change in C&Q savings!!

Lilly C&Q Work Breakdown Structure

Lilly budgets/tracks C&Q expenses via 4 primary WBS categories:

• Planning & Design• Commissioning• Qualification• Administration

New C&Q Cost Data spec recently approved to provide guidance

Planning & Design - Subcategory

• C&Q plans and strategy• Initial schedule development• Estimating• System classification• Component classification• System C&Q plans• Vendor assessments• User requirements• Design review, spec review, drawing review reviews/tag list• Engineering requirements• Design Qualification• Traceability Matrices

Commissioning - Subcategory

• FAT development• FAT execution• Receipt Verification development• Receipt Verification execution• Installation Verification development• Installation Verification execution• SAT development• SAT execution• Construction turnover packages

• Start-up protocol development• Start-up protocol execution• Functional testing development• Functional testing execution• Commissioning packages• C&Q plan summary reports• FAT travel• Supporting documents

Qualification - Subcategory

• IQ development• IQ execution• OQ development• OQ execution• PQ development• PQ execution

Administration - Subcategory

• Training• Project management• C&Q resource travel• Administrative assistance• Audits• CSV support

Typical C&Q Budget Metrics ~ Cost Breakdown (%)

URs 3%

Procedures 2% Doc Revs 1%

Planning 5%IV 7%

SU 9% RV 10%

CSV Support 3%

Construction QA 4%

DQ/EDR 3%

CCs 2%

Impact Assessment 1%RI

1%

Training1%OQ 2%

IQ 2%Doc mgmt 12%

Project mgmt 11%

FT 11%

Travel 10%

Scheduling 0%

SAT1%FAT

1%

Note: Only includes cost data for C&Q contractor

C&Q Discrepancy Metrics ~ Philosophy

C&Q delivery strategy based on risk mitigation approach

• DR/DQ reduced risk during Construction/Build, Commissioning, Qualification

• Construction quality assurance activities reduced Commissioning risks

• Simulation reduced risk during Commissioning (Startup, SAT, FT)

• Commissioning (RV, IV, FAT, SAT, FT) reduced risk during subsequent Commissioning activities and Qualification

• As risk decreased, the number of discrepancies also decreased - resulted in cleaner qualification documentation and better schedule adherence during qualification

Typical C&Q Discrepancy Metrics

1230 discrepancies resulting from Construction QA activities 2800 discrepancies during commissioning 130 discrepancies during qualification

0

100

200

300

400

500

600

700

RV SIM FAT IV SU SAT IL IC FT CP IQ OQ PQ CQP US

C&Q Activities

# o

f D

isc

rep

an

cie

s

Typical C&Q Discrepancy Metrics ~ Categories

0

100

200

300

400

500

600

700

800

900

# o

f D

iscr

ep

an

cie

s

Accepted

Corrective Action

1. Properly define User Requirements Testable, verifiable, applicable

Not too prescriptive, no design details, no guesses

Include C&Q engineers in review/approval process

2. Clearly define data and documentation requirements

3. Change management Don’t implement formal change control too soon

Don’t confuse change control with version control

Ensure commissioning change process is well understood

Ensure construction changes are communicated to team

Ensure automation changes are reflected in design docs

C&Q Lessons Learned

Current GFD C&Q Delivery Strategy~ Enhanced Baseline Guide approach (last 2 years)

• Conducted several six sigma initiatives to improve C&Q processes

• Developed additional procedures and best practices pertaining to C&Q• GDP SOP• Change Management SOP• Transfer of Care, Custody and Control SOP • Discrepancy Management Best Practice

• Developed new specifications governing the following

• Vendor data and documentation• FAT/SAT• C&Q cost data

• Defined new project information management role

• Implementing new requirements definition process based on systems engineering principles

• Implementing formal Construction QA program

Current GFD C&Q Delivery Strategy~ Enhanced Baseline Guide approach (cont.)

• Developed Automation Lifecycle Management engineering standard to accommodate integration of CSV and C&Q resulting in:

• Elimination of duplicate requirements, design reviews, testing• Elimination of “all or nothing” approach to software testing• Allowance for risk-based delivery strategies Leverage offline or simulation testing to

meet requirements

• Assessing risk in areas beyond product quality to streamline C&Q delivery

• Vendor risk• Technology risk• Automation strategy

• Leveraging vendor testing to support commissioning vs. just using as contractual gate/risk mitigation

The resulting C&Q delivery strategy will hopefully be realized as another step change in C&Q savings! Potential C&Q costs as low as 3% of TIC

Questions?

Q&A

Risk Based Commissioning & Qualification Benchmarking

Presented by:

Nick Andreopoulos Senior Manager/Team leader

Pfizer Global EngineeringPeapack, NJ

Chris BeltzSenior Manager/Team Leader

Pfizer Quality OperationsKalamazoo, MI

Agenda

• Basis of C&Q Practices

• C&Q Program Management Approach

• C&Q Cost Management

• Key Lessons

Current C&Q Practices

• Primarily based on C&Q Baseline® Guide 5 • Certain sites have streamlined C&Q process

• C&Q “Lite” version for small projects and minor equipment changes

• Leveraging of Commissioning tests into IOQ• Applying some ASTM concepts – Focusing IOQ on risk to

product quality/patient safety• Functional Level Impact Assessment - Replacing

Component Level Impact Assessment• Currently developing and piloting full ASTM E2500

based Verification approach at targeted sites

C&Q Program Management

• C&Q supported jointly by Quality, Engineering and Automation/IT

• Quality requirements defined in Corporate Quality Standards

• C&Q Approach defined in Global Practices• Sites develop their own procedures and

validation master plans based on the global documents

C&Q Cost Management

• C&Q costs that are associated with equipment qualification are included in Project Capital budgets. Process Validation and Cleaning Validation are not included in project budgets

• C&Q cost managed by Project team

Key C&Q Lessons Learned• Upfront involvement by key stakeholders in the

Assessment steps to define the appropriate scope for qualification

• IOQ scope should be focused on critical aspects/functions. All other testing is GEP

• Testing documents should be based on SME execution and leveraged, where possible, from vendors. Level of documentation should be commensurate with risk to product quality/patient safety.

• Roles/Responsibilities need to be clearly defined at the outset of projects

Risk-Based Commissioning & Qualification Benchmarking

Educational Track #1

Part 2

AgendaPart 1 (8:30 – 10:00)

• ISPE Baseline® Guide Volume 5: Commissioning & Qualification vs. ASTM E2500 – Mr. Robert Chew

• Overview of C&Q approaches – Abbott, Lilly, Pfizer

Break – Vendor areas open (10:00 – 10:30)

Part 2 (10:30 – Noon) • Panel Q&A

Wrap-up

Thank You!!

• Robert Chew, Commissioning Agents• Matt Warhover, Commissioning Agents• Steve Connelly, Abbott Laboratories• Dave Dolgin, Abbott Laboratories• Scott Hamm, Eli Lilly• Rick Gunyon, Eli Lilly• Nick Andreopoulos, Pfizer • Chris Beltz, Pfizer• Great Lakes Chapter Board and Directors• Ron Dunn, GLC Manager