Common Sense Practices for Agile SDLC Compliance … · Common Sense Practices for Agile SDLC Compliance –The Auditable OQ Verification Package Dr. Teri Stokes, ... QMS with SQC

IVT Session 15 April 27, 2017

Common Sense Practices for Agile

SDLC Compliance –The Auditable

OQ Verification Package

Dr. Teri Stokes, Director, QA Compliance

Cytel, Inc. – USA, Europe, India

What is Common Sense?

• Working Definition:

Common sense is knowledge based on “native good judgment” and personal experience. It is that combination of “book” smart, “street” smart, and “work” smart that a person can apply to scoping computer validation projects.

Slide 2

FDA Inspection View of GXP System

Operating environment (Clinical Research -GCP)

Computerized system (CDM System)

People

Software

Hardware

Computer system

SOPs

People

Equipment

Work Process

Infrastructure Platform Software application system - CDMSSlide 3

Defining Terms

Is SW Product GXP or Not?

Risk Analysis for Safety, Efficacy, and Quality (SEQ).

– SEQ Form to record a specific system risk assessment

– Impact of SW failure on People, Product, or Data in clinical studies or other regulated work activities (GCP/GLP/GMP = GXP)

– Priority #1 Safety Impact, #2 Efficacy Impact, #3 Other Quality

– If GXP yes = OQ/IQ/PQ If GXP no = general business practice

Slide 4

OQ Verification Package

Slide 5

GAMP 4 – The “V” Model

Functional

Specification

Design

Specification

System

Build

Installation

Qualification

IQ

Operational

Qualification

OQ

Performance

Qualification

PQ

User

Requirements

Specification

Specification & Qualification Basic Framework

Slide 6

Verifies

Verifies

Verifies

Ref. GAMP-4 Figure 6.2

Installation

Qualification

IQ&

SW Project Definition

SDLC SOP.DEV.001 The Agile Software Development Process for Custom Software

Slide 7

Product

Ideas PRD FRS

SRS OQ Trace

Matrix

SW Architect.

Document

SW Design

Description

SDLC SOPs,

Name/Code Stds

Developer Handbook

(Product Rqts. Doc.)

SAD SDD

(Functional Rqts.)

(Software Rqts.)

Developer Coding of SW

SDLC SOP.DEV.001 SDLC - Sand Box Sprints

CodeMaster OQ

Verification

Plan

Code Review

Form

SW Architect.

Description

SW Design

Description

Name & Code

Conventions

Developer Handbook

Unit Test

Smoke Test

Code ReviewSOP.DEV.003

Unit, Smoke &

Code Review

FormMaster OQ

Test Plan

SOP.DEV.002 SPRINT-N SAND BOX

Release

Testing

Sprint Plan

Sprint Report

SAD SDD

Issues are

tracked and

recycled until

resolved

OQ Testing of Release Candidate

QC Server OQ Testing

Code

Code Review

Form/Report

Unit Test

Smoke Test

Code ReviewSOP.DEV.003

Unit, Smoke &

Code Review

Form/ReportMaster OQ

Test Plan

SOP.DEV.002 SPRINT SAND BOX

OQ Release

Testing

OQ Test

Cases

OQ Test

Scripts

Test log,

Printout, P/F Report

OQ Test

Summary

Report

OQ

Trace

Matrix

QC Test

Server for

OQ Release

Release Candidate

Master OQ

Verification

Plan

OQ Verif.

Summary

Report

Sprint Plan

Sprint Report

SDLC Control Docs for SW Quality

• SDLC SOP.DEV.001 SDLC– Custom Software Project

Slide 10

Ideas MRS PRD Product

Rqts. Doc

FRS - SRS SW Rqts.

Spec.

Master OQ

Verification

Plan

Master

OQ Test

Plan

OQ Test

Sum. Rpt.

QC Server

IQ Report

Customer IT performs IQ to PRS

End Users perform PQ to URS

Customer

Audits

Authority

Inspections

OQ-VSR

Verification

Summary

Report

Developer Handbook

N Sprint Reports

Trace Matrix

User Acceptance at Go-Live

2.SYSTEM PLAN

What? URS

3.DESIGN

How? FRS &

SDD

4.BUILD

Program or

Configure

5.TEST

Verify to SDD

& Release

6.COMMISSION

Accept &

Validate to URS

9.RETIRE

Decommission

& Replace

Test Fit

to Design

Test Fit to

Work Process

8.MAINTAIN

Fix & Modify

7.OPERATE

Use&Monitor

1.SYSTEM IDEA

Needs Analysis,

RFP & Contract

Application

Life Cycle

SDLC

OQ – Operational

Qualification

IQ – Installation

Qualification

PQ – Performance

Qualification

Platform System

1.

Configure

2.

Install

3.

Test

Test Fit to Install Specs.

PQ

OQ

IQ

System User

Software

Supplier

SDLC - SW

Development

Life Cycle

Slide 11

Ongoing Testing &

Change Control

2.SYSTEM PLAN

What? URS

3.DESIGN

How? SDD

4.BUILD

Program or

Configure

5.TEST

Verify to SDD

& Release

6.COMMISSION

Accept &

Validate to URS

9.RETIRE

Decommission

& Replace

Test Fit

to Design

Test Fit to

Work Process

8.MAINTAIN

Fix & Modify

7.OPERATE

Use & Monitor

1.SYSTEM IDEA

Needs Analysis,

RFP & Contract

Application

Life Cycle

SDLC

Platform System

1.

Change

2.

Install

3.

Test

Test Fit to Design & Work

SDLC - Software

Development

Life Cycle

Software

Supplier

System User

Change

Symbol

Slide 12

Life Cycle Qualification at

New Version or Retirement

2.SYSTEM PLAN

What? URS

3.DESIGN

How? SDD

4.BUILD

Program or

Configure

5.TEST

Verify to SDD

& Release

6.COMMISSION

Accept &

Validate to URS

9.RETIRE

Decommission

& Replace

Test Fit

to Design

8.MAINTAIN

Fix & Modify

7.OPERATE

Use & Monitor

1.SYSTEM IDEA

Needs Analysis,

RFP & Contract

Application

Life Cycle

SDLC

RQ –

Retirement

Qualification &

Archive

Management

Platform System

1.

Back Up

2.

Archive

3.

Test

Test Media for RetrievalOQ

System User

Software

Supplier

SDLC - Software

Development

Life Cycle

Slide 13

Replacement

System

Replacement Plan

Data Migration Plan

Archive Plan

CSV Package Model

Control of Software &

Platform Environment

CSV Package Plan Verification/Validation Plan

Control of System & SW

Changes & Audit of

CSV Package, QMS

CSV Package Summary Report Verification/Validation Summary Report

Standard CSV

package

SOPs, Training &

Materials for Human

Work with System

Prepared and maintained by a CSV Package Team for IQ, OQ, or PQ

Requirements &

Spec.s for SW, System

& Services - SLA

Slide 14

1. System Control 2. Human Control 3. Testing Control

Test PlansStartup & Ongoing

Test Cases, Scripts,

Data & Result Logs

Test Summary

Reports

QMS = Quality Management System SLA = Service Level Agreement

Trace

IEEE Format for a Software

V&V Plan

IEEE Std. 1012-1986

1. Purpose

State plan identifier. Describe system. State assumptions and scope for this plan. What’s included, excluded, and limited?

2. Referenced Documents

Identify key documents associated with plan activities.

3. Definitions

Define terms and acronyms needed to understand this Plan.

4. Verification & Validation (V&V) Overview

Team structure, master task schedule, assigned responsibilities, tools, techniques, and methodologies to be used for tasks.

Slide 15

IEEE Format for a Software

V&V Plan

IEEE Std. 1012-1986

5. Life-Cycle Verification & Validation (V&V)

Management of V&V, concept phase, requirements phase, design phase, implementation phase, test phase, installation and checkout phase, operation and maintenance phase

6. Software Verification & Validation Reporting

Required reports, optional reports

7. Verification & Validation Administrative Procedures

Anomaly reporting and resolution, task iteration policy, deviation policy, control procedures, standards, practices, and conventions

Slide 16

XQ Validation Plan (VP)

Task Schedule

Slide 17

VP Task for OQ or IQ

or PQ (= XQ)

Role Responsible Status –Date

Write XQ Ver/Val Plan Team Leader with CSV

Team

Week 0

Review & Approve V.Plan System Sponsor & QA Week 1

Write XQ XRS* – Tables of

Work & System Tasks

CSV Team Week 0-1

Write XQ Test Plan for

Startup

Test Coordinator Week 1-2

*XRS = System Requirements = Functional (FRS) for OQ, Platform

(PRS) for IQ or User (URS) for PQ

1. Package Summary Report identifier - Unique ID traceable to associated Validation Plan.

2. Summary of all Life Cycle Tasks - Identifies CSV Package Team. Includes Task List from Validation Plan with updated status and roles responsible. References Audit Reports and/or Test or Package Summary Reports to cover Platform and Application Provider phases of Life Cycle.

3. Summary of Anomalies and Resolutions - Describes any deviations from Validation Plan and reasons why. Highlights any anomalies pending from the Test Summary Report.

CSV Package Summary Report

IEEE Std. 1012-1992 Adapted

Slide 18

4. Assessment of overall software quality - Discusses assumptions and limits to scope of CSV Package. Gives highlights from Test Summary Report. Describes how Package items are sufficient to assess system quality and user readiness for Go-Live in a regulated environment.

5. Recommendations - Recommends a Go-Live release status to the System Sponsor as Full Pass, Pass with Specified Conditions, or System Fail.

6. Approvals - Names, titles, signatures, dates and meaning of signatures.

7. Appendix - Table of Contents listing for Package items.

IEEE Std. 1012-1992 Adapted

Slide 19

CSV Package Summary Report

CSV Package Team Model –

Any Package - OQ/IQ/PQ

Package Sponsor

Test Coordinator

Package Manager

(Pkg. QC)

CSV Package Team

Quality

AssuranceTeam Leader

Ad hoc

Members

Trains Team and

Audits CSV Pkg.

Funds and Approves

Validation Work

Team to Develop and

Maintain a CSV Pkg.

Slide 20

Software Supplier’s (SDLC) OQ Package

Code & Tools Mgt. Logs,

SDLC, SAD, Controlled SW

Devel. Platform, Tools IQ

OQ Verification Plan

SW Upgrade Plan, Bug

Tracking, Audit Logs,

QMS with SQC & QA

OQ Verification Summary Report

Standard SW supplier’s

CSV package

SW Engr. SOPs, Pgmr.

Trng, Product Manuals,

User Help Desk

Prepared by internal or external software supplier team

MRS, FRS, SDD, Code,

SDLC Docs., Support

Contract (SLA), (Escrow)

Slide 21

1. System Control 2. Human Control

Software Supplier’s (SDLC) OQ Package

Code & Tools Mgt. Logs,

SDLC, SAD, Controlled SW

Devel. Platform, Tools IQ

OQ Verification Plan


Data & Result Logs

Master Test Plan &

Sub Test Plans –

U,F,S,M,S,I,R*

SW Upgrade Plan, Bug

Tracking, Audit Logs,

QMS with SQC & QA

OQ Verification Summary Report

Standard SW supplier’s

CSV package

SW Engr. SOPs, Pgmr.

Trng, Product Manuals,

User Help Desk

Prepared by internal or external software supplier team

MRS, FRS, SDD, Code,

SDLC Docs., Support

Contract (SLA), (Escrow)

Slide 22

Test Summary

Report (per Plan)

*U,F,S,M,S,I,R = Unit, Function, Smoke, Module, System, Interface, Regression


Trace

Platform System IQ Package

Data Ctr. Config. Mgt.

Logs, Ops. SOPs, System

Manuals & Security WI

IQ Validation Plan


Data & Result Logs

IQ Test Plan(s)

Startup & Ongoing

Dis. Rec. Plan, Change

Control Log, Backup &

Archive Logs, Supplier

Records, IT Audit Log

IQ Test Summary

Report(s)

IQ Validation Summary Report

Standard platform

CSV package

IT/IS CVs, Training

Records, Dept. SOPs,

WIs, Check Lists

Prepared and maintained by IS/IT Department Team

SLA, SLA Monthly

Reports, Security

Logs, Help Desk Log

Platform URS

Slide 23

3. Testing Control 1. System Control 2. Human Control

Trace

Application User’s PQ Package

Applic. Admin. SOP &

Applic. Config. Mgt. Logs

PQ Validation Plan


Data & Result Logs

PQ Test PlansStartup & Ongoing

Change Control Log,

QA Audit Log, Supplier

Reports & BDG* Minutes

PQ Validation Summary Report

Standard user’s

CSV package

User Manuals, CVs,

Trng Records, Work

SOPs, WIs, Help Logs

Prepared and maintained by User Department(s) Team

Needs Analysis, RFP,

Contract, URS, User

Access Profiles, SLAs

Slide 24

PQ Test Summary

Reports

* BDG = Business Decision Group


Trace

CSV Package XQ Definitions

• Operational Qualification (OQ) – Documented evidence

that a system operates as intended throughout its expected

range as per design/functional specifications. (Supplier)

• Installation Qualification (IQ) – Documented evidence that

all system components are installed to supplier instructions &

user requirements. (Infrastructure/IT)

• Performance Qualification (PQ) – Documented evidence

that a system operates as intended in the user’s work process.

(User – Start-up & Ongoing)

Slide 25

CSV Package Contents – GXP Systems

Slide 26

Agile SW OQ Package SW Application PQ Package

1. OQ Verification Plan (OQ VeP) 1. PQ Validation Plan (PQ VP)

2. Software Requirements Spec. (SRS) 2. User Requirements Spec. (URS)

3. OQ Master Test Plan (OQ MTP) 3. PQ Test Plan (PQ TP)

4. OQ Test Cases, Scripts & Fix Cycle 4. PQ Test Cases, Scripts & Fix Cycle

5. OQ Trace Matrix (Scripts to SRS) 5. PQ Trace Matrix (Scripts to URS)

6. OQ Test Summary Report (OQ TSR) 6. PQ Test Summary Report (PQ TSR)

7. SOP/WIs (SDLC Sprint Tasks) 7. SOP/WIs (User Work Process Tasks)

8. Disaster Recovery Plan & DRP Test 8. Business Continuity Plan & BCP Test

9. QA Audit Certificate of SDLC Site 9. QA Audit Certificate of SW Vendor

10. OQ Change Management Plan 10. PQ Change Management Plan

11. OQ Verification Summary Report 11. PQ Validation Summary Report

12. QA Audit Certificate of OQ Pkg. 12. QA Audit Certificate of PQ Package

OQ Package

&

PQ Package

CSV* Regulatory Themes Matrix

Management Control

Auditable Quality

System Reliability

Data Integrity & Privacy

Slide 27 *CSV = Computerized System Validation

1. Verification/Validation Plan

2. Requirements Specifications

7. SOP/WIs

9. Vendor QA Audit

11. Validation Summary Report

12. QA Audit of CSV Package

3. Test Plan

4. Test Cases & Scripts

5. Trace Matrix

6. Test Summary Report

8. DRP/BCP – Test Report

10. Change Management Plan

CSV Package Contents – IT Platform IQ

Platform System IQ Package Document ID

1. IQ Validation Plan CTX.IQ.VP.v.1.0

2. Platform Requirements Spec. (PRS) CTX.IQ.PRS.MTX.v.2.1

3. IQ Test Plan CTX.IQ.TP.v.1.0

4. IQ Test Cases & Scripts & Fix Cycle 9 Test Cases & 24 Test Scripts

5. IQ Trace Matrix (Scripts to PRS) CTX.IQ.PRS.MTX.v.2.1

6. IQ Test Summary Report SOP/WIs CTX.IQ.TSR.v.1.0

7. SOP/WIs (Data Center Tasks) SOP IT_C002, CO23, C024 & C026

8. Disaster Recovery Plan & Test IT_C001, TBD in Standby Configuration

9. QA Audit Certificate of IT Facility TBD in Production Configuration

10. IQ Change Management Plan CTX.IQ.TC09 & SOP IT_C012

11. IQ Validation Summary Report CTX.IQ.VSR.v.1.0

12. QA Audit Certificate of IQ Package PQ Team Walk-through Report

IQ Package

Slide 28

Validation Focus for CSV* Packages

• Operational Qualification (OQ): Provision of documented evidence by system supplier that the software reliably operates as expected to its functional requirements and design specifications.

• Installation Qualification (IQ): Provision of documented evidence by IT that the network and infrastructure systems reliably operate to platform requirements for user’s application.

• Performance Qualification (PQ): Provision of documented evidence by end users that a software application reliably operates as intended to the user requirements of the regulated work process.

Slide 29 * CSV = Computerized System Validation

Computer Validation Training

Thank You!

Tak – Tack – Takk

Mahalo - Shukria

Bangalore – 2012

Any Questions or Comments?

??

OQ Group Exercise #1:

Inspection Themes for the 11’s –

FDA Part 11 & EMA Annex 11

Slide 32

Assignment: Prepare a regulation-based quick review chart for assessing

the status of your GXP systems.

1. Divide into paired teams for Annex 11 and 21 CFR Part 11.

2. Use the respective regulation to identify which CSV package item(s)

satisfy each stated document requirement.

3. Use the regulation’s paragraph numbering to plot the relationship on the

table of CSV Package Items.

(Example: Validation Plan = Annex 11 Principle & Sections 4.1-4.8)

(Example: Validation Plan = Part 11 Section 11.10 (a))

4. Are there any regulations not fulfilled by the package?

5. If so, how would you address the issue?

SDLC OQ/PQ Group Exercise:

Validation Inspection Themes for the 11’sAnnex 11 SDLC Software OQ Package SW Application User’s PQ Package 21 Part 11

Principle; 4.1-4.8 1. OQ Verification Plan (OQ VeP) 1. PQ Validation Plan (PQ VP) 11.10 a

2. Software Requirements Spec. (SRS) 2. User Requirements Spec. (URS)

3. OQ Master Test Plan (OQ MTP) 3. PQ Test Plan (PQ TP)

4. OQ Test Cases, Scripts & Fix Cycle 4. PQ Test Cases, Scripts & Fix Cycle

5. OQ Trace Matrix (Scripts to SRS) 5. PQ Trace Matrix (Scripts to URS)

6. OQ Test Summary Report (OQ TSR) 6. PQ Test Summary Report (PQ TSR)

7. SOP/WIs (SDLC Sprint Tasks) 7. SOP/WIs (User Work Process Tasks)


9. QA Audit Certificate of SDLC Site 9. QA Audit Certificate of SW Vendor

10. OQ Change Management Plan 10. PQ Change Management Plan

11. OQ Verification Summary Report 11. PQ Validation Summary Report

12. QA Audit Certificate of OQ Pkg. 12. QA Audit Certificate of PQ Package

Name: _____________________ Date: ________

End Users’ IQ/PQ Group Exercise:

Validation Inspection Themes for the 11’sAnnex 11 Platform System IQ Package Software Application PQ Package 21 Part 11

Principle; 4.1-4.8 1. IQ Validation Plan 1. PQ Validation Plan 11.10 a

2. Platform Requirements Spec. (PRS) 2. User Requirements Spec. (URS)

3. IQ Test Plan (IT Infrastructure PRS) 3. PQ Test Plan (GXP Work URS)

4. IQ Test Cases, Scripts & Fix Cycle 4. PQ Test Cases, Scripts & Fix Cycle

5. IQ Trace Matrix (Scripts to PRS) 5. PQ Trace Matrix (Scripts to URS)

6. IQ Test Summary Report 6. PQ Test Summary Report

7. SOP/WIs (Data Center Tasks) 7. SOP/WIs (User Work Process Tasks)


9. QA Audit Certificate of IT Facility 9. QA Audit Certificate of SW Vendor

10. IQ Change Management Plan 10. PQ Change Management Plan

11. IQ Validation Summary Report 11. PQ Validation Summary Report

12. QA Audit Certificate of IQ Package 12. QA Audit Certificate of PQ Package

Name: _____________________ Date: ________

OQ Group Exercise #2 :

CSV Policy: GXP SEQ Risk Assessments

Slide 35

Assignment: Prepare an SEQ Risk Assessment for a known SW Product.

1. Divide into teams for performing GXP SEQ Risk Assessments.

2. Use the SEQ Attachments 1 and 2 to identify which Software products

are GXP applications that need OQ. What is their validation priority?

3. Pick a SW Product used in your company to analyze.

4. Are there any software products that are not GXP?

5. If so, how would you address SDLC quality for them?