Acc Criteria

8/10/2019 Acc Criteria

1/25

Agency Name

Project Name

Deliverable Review Process and

Acceptance Criteria

Version Number: (n) Date: (mmddyyyy)


2/25

Document !istory and Distribution

1. Revision History

Revision # Revision Date Description of Change Author

2. Distribution

Recipient Name Recipient Organization Distribution Method


3/25

Deliverable Review Process and Acceptance Criteria

Table of Contents

1 Purpose and Scope............................................................................................................................1

2 Goals of the Deliverable Review Process........................................................................................13 Glossary of Terms.............................................................................................................................2

!eetin" Participants# Roles# and Responsibilities...........................................................................3

$ Review Process.................................................................................................................................3

$.1 Review Process %low................................................................................................................3$.2 Review Process Steps................................................................................................................$

& Review Dispositions.........................................................................................................................'

' ()it *riteria for Reviews..................................................................................................................++ ,cceptance Process for Pro-ect Deliverables...................................................................................+

,cceptance *riteria for !ilestones and Deliverables....................................................................121/ ,pprovals.......................................................................................................................................1+,ppendi) ,0 o" of Recommended ,ctions*han"es............................................................................1

,ppendi) 0 Review 4erification %orm.................................................................................................2/

,ppendi) *0 Re5uest for ,cceptance %orm...........................................................................................21

12/19/2014 2:36 AM i


4/25


1 Purpose and Scope

,cceptance criteria are defined as 6the list of re5uirements that must be satisfied prior to the customer

acceptin" delivery of the product7.

This document defines the acceptance process# the acceptance criteria# and the reviewapprovalre5uired for customer acceptance of the 8,"ency name9 8pro-ect name9 pro-ect deliverables.

The purpose of this document is to define a standardi:ed Deliverable Review Process# which will

provide a structured method to support the ,"ency Software 4erification and 4alidation Plan 8S44P9

to ensure that appropriate# correct# consistent# and complete deliverables are created for the pro-ect.

This document describes0

Goals of the review process;

Definitions;

!eetin" participants# roles# and responsibilities;

Review process;

Dispositions for the review meetin"; and

Review e)it criteria.

,ppendices , and contain forms that aid the !oderator# Deliverable R!* ,pproved Principles# Policies# and Standards7section# 6Pro-ect Reportin"7 cate"ory.

2 Goals of te Deliverable Review Process

The primary "oal of the Deliverable Review Process is to detect and remove deliverable defects as

early as possible in the Software Development ife *ycle 8SD*9 process.

Secondary "oals to be attained are0

*onsistency with >((( Std 1/2+?1'# Standard for Software Reviews;

12/19/2014 2:36 AM 1


5/25


(nsure correctness# completeness# consistency# and accuracy of deliverables and products

for all life cycle activities within the development process;

Provide reviewers with a common understandin" of the review criteria and the deliverable;

and

Serve as a criterion for the 8,"ency name9 Deliverable ,cceptance Process.

! Glossary of Ter"s

%ollowin" are definitions of commonly used terms in this document0

Deliverable:, non?technical or technical component item to be reviewed.

Deliverable "wner:The person responsible for the deliverable and the initiation of the

review.

Document Control:The or"ani:ation responsible for documentation filin" procedures.

#tem:, thin" written down at the Review Process meetin". This consists of recommended

action items# point clarifications# and improvement su""estions.

$eeting:, forum used to formally present the deliverable durin" which all participants are

able to communicate and interact with each other at the same time.

$oderator:The person who leads the meetin" and is responsible for the overall review

meetin".

Peer:, member of the a"ency staff who is of e5ual standin" with the author of the wor@

document.

Peer review:, formal peer meetin" held to allow discussion of issues# alternatives# options#

and wor@ document approval.

Review process:The structured process to review the deliverable; meet to allow discussion of

recommended chan"es# actions# alternatives to improve the 5uality of the deliverable and reach

a consensus on the disposition of the deliverable; record the feedbac@; and verify completion of

the review; and archive the deliverable and review forms.

Review criteria: The stated re5uirements# policies# standards# procedures# methods# and

"uidelines applicable to the deliverable.

Reviewer: , review meetin" participant who evaluates the deliverable a"ainst an established

review criteria.

12/19/2014 2:36 AM 2


6/25


%uality Assurance: 8Product9 *onformance to technical and business functional

re5uirements to ensure defect?free products 8completeness of software or system features and

functions# error?free operation9. 8Process9 4erification and validation of pro-ect activities and

resultant artifacts with respect to established policies# standards# procedures# methods# and

"uidelines for software development.

%uality Assurance (%A) representative: The person responsible for verifyin" compliance to

the established =uality ,ssurance policies# standards# procedures# methods# and "uidelines for

the pro-ect or the a"ency.

&cribe:, peer review meetin" participant who records items identified at the meetin".

&o'tware veri'ication and validation (&VV): , disciplined =uality ,ssurance approach used

to assess software products throu"hout the Software ifecycle Process. Verificationis defined

as the process of determinin" whether products of a "iven phase of the software development

process fulfill the re5uirements established durin" the previous phase# and validationis definedas the process of evaluatin" software at the end of the software development process to ensure

compliance to software re5uirements.

or document: The material to be reviewed.

# $eetin% Participants& Roles& and Responsibilities

The Deliverable Review Process team members include the moderator# the deliverable owner# thereviewers# a scribe# and a =, representative# as defined in the definitions section of this document.

The deliverable owner may serve as moderator. The =, representative may also serve as themoderator. , separate scribe may be selected to record all recommended chan"es or action items

discussed at the meetin".

' Review Process

This section details the review process to initiate# conduct# and finali:e reviews. Aote that the

deliverables may be partitioned into units to allow for incremental reviews. The deliverable reviews

are complete after all incremental reviews have been held.

'.1 Review Process (low

The review process flow is shown below.

12/19/2014 2:36 AM 3


7/25


8/25


'.2 Review Process Steps

This section details the steps to complete the review process.

Phase Task Oner

Prior to Review1. *omplete deliverable sub-ect to review. Deliverable


9/25


During t*e review$. *onduct the Review !eetin". Reviewers# Deliverable

ntroduce meetin" "oals and e)pectations !oderator

$.2 Solicit feedbac@ in manner appropriate to deliverable# review

criteria# and meetin" format.

!oderator

$.3 Document su""ested chan"es and actions. !oderator

$. Summari:e review results. Scribe

& Determine and document review disposition Reviewers# Deliverable


10/25


A'ter t*e review+ ,Approved with Changes disposition+ >f 6Approved with Changesdisposition

+.1 !a@e the recommended chan"es andor resolve actions# and

return the document to =, representative.

Deliverable


11/25


Rewor: if recommended chan"es are re5uired thatsi"nificantly alter the deliverable# the deliverablewill enter the rewor@ phase# and the same "roup of participants will be as@ed to review the rewor@ed

document.

* +,it Criteria for Reviews

>n order to closely mana"e the process# the e)it criteria for the process must be clearly defined. The

e)it criteria for the Deliverables Review Process includes0

>tems lo""ed on the o" of Recommended *han"es and ,ctions %orm have been addressed and

verified as complete.

Review 4erification %orm is completed and si"ned.

The deliverable was placed under confi"uration mana"ement system.

*ompleted o" of Recommended *han"es and ,ctions and Review 4erification forms are placedin the Pro-ect ibrary.

- Acceptance Process for Proect Deliverables

The acceptance process for 8Pro-ect Aame9 provides a roadmap for incremental acceptance by thecustomer of the software application and associated pro-ect deliverables at the followin" @ey

milestones.

Pro-ect Phase *oncept *omplete

Phase Re5uirements *omplete

Phase Desi"n *omplete

Phase ,pplication Ready %or Pilot

Phase ,pplication Ready %or Statewide Rollout

Phase *omplete

The followin" pro-ect deliverables are sub-ect to acceptance within the conte)t of the abovemilestones.

12/19/2014 2:36 AM -


12/25


$ilestone Deliverables

Pro-ect Phase *oncept *omplete Pro-ect >nitiation and >mplementation

Document# Software Pro-ect !ana"ement

Plan

Phase Re5uirements *omplete Software Re5uirements Specification

Template

Phase Desi"n *omplete Software Desi"n Specification

Phase ,pplication Ready %or Pilot ,pplication# Software Test Plan# SoftwareTransition Plan# Trainin" Plan# BserCs

andboo@# usiness *ontinuity Plan

Phase ,pplication Ready %or Statewide Rollout ,pplication# Software Test Plan# SoftwareTransition Plan# Trainin" Plan# BserCs

andboo@# usiness *ontinuity Plan

Phase *omplete *loseout Review# essons?learned

The followin" table defines the se5uence of activities that must be performed in support of the

acceptance process and who is responsible for those activities.

Activity #ndividual(s) Responsible

Define acceptance criteria for milestones and

deliverables in the current pro-ect phase

=, !ana"er# Pro-ect !ana"er# >S Sponsor#

and usiness Sponsor8s9 for current phase

>dentify and plan for verification and validation

activities necessary to support acceptance criteria fordeliverables sub-ect to acceptance

=, !ana"er and Pro-ect !ana"er

*omplete pro-ect deliverables for milestone Pro-ect team members responsible for

pro-ect deliverable

(nsure completion of any necessary verification and

validation activities for deliverables

=, !ana"er and Pro-ect !ana"er

12/19/2014 2:36 AM 9


13/25



*omplete the Re5uest for ,cceptance form

8,ppendi) ,# first three sections9. Refer to Section# ,cceptance *riteria and ,pproval Re5uired for

Document Deliverables# for "uidelines on document

content and approvers.

Pro-ect !ana"er

%orward the Re5uest for ,cceptance form attached

to the deliverables for the milestone and any outputsfrom verification and validation activities to the list

of approvers for the milestone and its deliverables

.

Pro-ect !ana"er

Schedule and conduct a meetin" for approvers to

review the milestone and its deliverables with

respect to their acceptance criteria.

Pro-ect !ana"er

Durin" acceptance review meetin"# si"n?off onacceptance and include any desired comments on

si"nature pa"e for milestonedeliverables.

8Aote0 Re-ected deliverables are returned to the

deliverable owner to rewor@ alon" with the Re5uest

for ,cceptance forms and will be reviewed foracceptance a"ain once the necessary chan"es have

been made. Reasons for re-ection are documented

on the si"nature form. >f si"n?off is not obtained

within five 8$9 business days# then the pro-ect will

proceed as thou"h acceptance were obtained# and anissue will be lo""ed to escalate the official

acceptance.9

=, !ana"er# Pro-ect !ana"er# >S Sponsor#and usiness Sponsor8s9 for current phase

Return Re5uest for ,cceptance Si"nature pa"e toPro-ect !ana"er

=, !ana"er# Pro-ect !ana"er# andusiness Sponsor8s9 for current phase

Submit completed Re5uest for ,cceptance form and

si"nature pa"es to =, mana"er for inclusion in the

Pro-ect Aoteboo@

Pro-ect !ana"er

%ile Re5uest for ,cceptance form and si"naturepa"es in Pro-ect Aoteboo@

=, !ana"er

Place accepted deliverables in the Pro-ect Aoteboo@ =, !ana"er

12/19/2014 2:36 AM 10


14/25



(nsure the new version of the deliverable is in the

Pro-ect %ile repository and is mar@ed as the currentversion

=, !ana"er

Report overdue Re5uest for ,cceptance si"naturepa"es as issues on status reports to mana"ement

Pro-ect !ana"er

12/19/2014 2:36 AM 11


15/25


/ Acceptance Criteria for $ilestones and Deliverables

The acceptance criteria in the table below define the conditions under which the PRnitiation

and>mplementation

Document

Document has been reviewed and approved by0

Prioriti:ed scope and hi"h level re5uirements have

been reviewed by the Desi"n Team# which consists of

sta@eholders from the usiness Sponsor or"ani:ationin addition to @ey members of the pro-ect team.

Aote0 ,cceptance of this document is completed by

si"nin" off on the documentCs si"nature pa"e ratherthan a Re5uest %or ,cceptance form.

Pro-ect Phase

*oncept *omplete

Software Pro-ect

!ana"ement Plan

Document has been reviewed and approved by0

Prioriti:ed scope and hi"h level re5uirements havebeen reviewed by the Desi"n Team# which consists of

sta@eholders from the usiness Sponsor or"ani:ation

in addition to @ey members of the pro-ect team.

Aote0 ,cceptance of this document is completed by

si"nin" off on the documentCs si"nature pa"e rather

than a Re5uest %or ,cceptance form.

Phase

Re5uirements*omplete

Software

Re5uirementsSpecification

The Software Re5uirements Specification describes

whatcapabilities the application should have andincludes0

usiness process

usiness re5uirements Bse cases

%unctional re5uirements

Data re5uirements

Aon?functional re5uirements

12/19/2014 2:36 AM 12


16/25


$ilestone Deliverable Acceptance Criteria

The Desi"n Team has reviewed the Specification and

its prioriti:ed re5uirements for correctness#completeness# and consistency with respect to the

sponsor or"ani:ationCs business processes and

business needs within the scope established in thePro-ect >nitiation and >mplementation Document.

The application development team has reviewed the

Specification to ensure its sufficiency for be"innin"application desi"n and to determine which

re5uirements can be met within the constraints of the

current pro-ect phase.

The software test team has reviewed the Specification

to ensure that all re5uirements are testable.

The pro-ect mana"er has reviewed the Specification

to ensure that all re5uirements are traceable to the

scope# "oals# and ob-ectives established in the Pro-ect>nitiation and >mplementation Document.

Phase Desi"n

*omplete

Software Desi"n

SpecificationThe Software Desi"n Specification describes how the

application should function and be constructed to

provide the capabilities defined in the SoftwareRe5uirements Specification. This document includes0

Screen flows

Screen desi"ns

%orm and letters desi"n

Report desi"ns

o"ical data model

Physical data model

Bser roles

Security model

Data mappin"

Detailed system interface desi"ns

The Desi"n Team has reviewed the Specification forcorrectness# completeness# and consistency with

respect to the prioriti:ed re5uirements established in

the Software Re5uirements Specification.

12/19/2014 2:36 AM 13


17/25



The business team has reviewed the specification toensure that all re5uirements are fully represented in

the desi"n and that the desi"n includes no items that

are not part of the established re5uirements.

The application development team has reviewed the

Specification to ensure its sufficiency for be"innin"

application development and to validate thefeasibility of implementin" the desi"n within the

constraints of the current pro-ect phase.

The software test team has reviewed the Specification

to ensure that the desi"n is testable.

Phase ,pplicationReady %or Pilot

Software Test Plan The completed application was delivered into the testenvironment with functionality as specified in the

Software Re5uirements Specification# Supplemental

Specification for Aonfunctional Re5uirements# andSoftware Desi"n Specification.

The completed application passed throu"h systemtestin" with the followin" results0

1. ,ll test cases completely e)ecuted for

functional modules with a system test priority

ran@in" between 1 and 3 inclusive2. Fero severity 1 8critical9 defects

3. Fero severity 2 8ma-or9 defects in businesspriority 1 functional modules

. Ao more than 2 severity 3 defects 8minor9 per

business priority 1 functional module and nomore than 2/ severity 3 defects total in

business priority one functional modules

$. Ao more than 1 severity 2 defect per business

priority 2 functional module and no more than$ severity 2 defects total in business priority 2

functional modules&. Ao more than $ severity 3 defects per

business priority 2 functional module and no

more than / severity 3 defects total in

business priority 2 functional modules

12/19/2014 2:36 AM 14


18/25



>n the above criteria# business priority refers to the

business priority assi"ned to a functional module inthe Software Re5uirements Specification# where

priority 1 is business critical# priority 2 is moderately

important# and priority 3 is optional.

Severity 1# or *ritical# defects include issues that

result in de"raded system performance beyond stated

performance criteria# deny access to functionality#andor corrupt data or display data incorrectly.

Severity 2# or !a-or# defects include issues thatprevent a user from correctly completin" a tas@ in the

system# but can be mana"ed by a wor@around# andor

issues that promote data errors or reduce data 5uality

substantially.

Severity 3# or !inor# defects include issues that

interfere with# but do not prevent a user fromperformin" normal tas@s. These are fre5uently

usability or performance issues or defects that cannot

be reliably reproduced. These can also be issues thatrepresent an inaccuracy in the application but have no

impact on functionality or performance# such as

spellin" errors# inconsistent fonts# etc.

%or user acceptance testin"# the completedapplication was delivered in the same condition or

better as when it was delivered with the test fi)esre5uired to meet the system test acceptance criteria

described above. The completed application passed

throu"h all acceptance testin" scenarios with noseverity 1 defects in any functional module and no

severity 2 defects in any business priority 1

functional module 8priorities and severities as definedabove9. ,cceptance testers a"ree that the application

can move into pilot rollout with all the @nown non?

minor defects or with a limited number of defect fi)esthat can be completed and tested by the pro-ect teamin no more than two 829 business days.

The accepted application has been moved from thetest environment to the pilot environment with all test

12/19/2014 2:36 AM 1,


19/25



data removed and is functionin" properly 8as "ood as

or better than it was durin" acceptance test9.


BserCs andboo@ The BserCs andboo@ describes in detail theprocedures for usin" all of the functionality provided

in the application in terms understandable to the

typical user.


SoftwareTransition Plan

The Software Transition Plan describes the tas@s andactivities that need to ta@e place to efficiently and

effectively move the application from the

development or Pilot environment to the production#

operations and maintenance environment and tointe"rate use of into the business processes of the

impacted or"ani:ation8s9.

The Plan includes deployment schedules# resource

estimates# identification of special resources and

staffin". The transition plan also definesmana"ement controls and reportin" procedures# as

well as the ris@s and contin"encies. ,n impact

statement outlinin" the potential impact of thetransition to the e)istin" infrastructure# operations#

and support staff and to the user community is

included.

The Plan has been reviewed and approved by the

operations# maintenance# and support or"ani:ation#

includin" Technical Services.


Trainin" Plan The Trainin" Plan describes what trainin" willprovided to users to prepare them to use the

application and how this trainin" will be performed.

Phase ,pplication

Ready %or Pilot

usiness

*ontinuity Plan

The usiness *ontinuity Plan describes how business

functions supported by will be performed in the event

that the application is unavailable for a period oftime.

Phase ,pplication

Ready %or

Statewide Rollout

,pplication ,ny application issues that the Desi"n Team and Pilot

users identified as needin" to be addressed prior to

statewide rollout have been addressed.

12/19/2014 2:36 AM 16


20/25



The application still meets or e)ceeds the acceptancecriteria established for the 6Phase ,pplication Ready

%or Pilot7 milestone.

Phase ,pplication

Ready %orStatewide Rollout

BserCs andboo@ The BserCs andboo@ was updated to include any

chan"es necessary to support application chan"esmade after the Pilot.

Phase ,pplication

Ready %or

Statewide Rollout

Software

Transition Plan

The Software Transition Plan was updated to include

any chan"es necessitated by problems with the Pilot

rollout.

Phase ,pplication

Ready %orStatewide Rollout

Trainin" Plan The Trainin" Plan was updated to include any

chan"es necessitated by problems with the Pilottrainin".

Phase ,pplication

Ready %or

Statewide Rollout

usiness

*ontinuity Plan

The usiness *ontinuity Plan was updated to include

any chan"es necessitated by problems with the

evaluation of business continuity procedures durin"the Pilot.

Phase *omplete *loseout Reviewdocument

,ll pro-ect activities defined in the Pro-ect >nitiationand >mplementation Document and any approved

chan"e re5uests have been completed.

,ll users have been trained and provided access to

the application as specified in the Software Transition

Plan and Trainin" Plan.

The Pro-ect *loseout Review document includes all

pro-ect outcomes# costs# and lessons learned for thecurrent pro-ect phase.

12/19/2014 2:36 AM 1.


21/25


10 Approvals

The si"natures below indicate the approval of the 8,"ency9 8Pro-ect Aame9 Deliverable ReviewProcess and ,cceptance *riteria process and document.

Si"nature Date

Si"nature Date

Si"nature Date

12/19/2014 2:36 AM 1-


22/25


23/25


Appendi, 7 Review 5erification (or"

DeliverableProduct Name and Purpose:

DeliverableProduct "wner:

RevisionVersion Number:

Reason 'or Review:

Veri'ication:

..................................... .......................

%AProject $anager Date

12/19/2014 2:36 AM 20


24/25


Appendi, C Re8uest for Acceptance (or"

Re8uest for Acceptance

Section 1

Date:

&ubmitted /y:

&ubmitted -o:

Project:

&ection 0 Deliverable Description:

(Provide a brief description of the ilestone and deliverables and any necessary coents!"

&ection 1 &ignatures:

(#ist individuals whose signatures are re$uired for deliverable in %ection &!Pro'ect anager will input date of signature!"

Approval Date

12/19/2014 2:36 AM 21


25/25


Si%nature Pa%e for Re8uest for Acceptance

&ignature Date

Name -itle

Rejection Comments:

>nsert comments e)plainin" re-ection of deliverable and the date of re-ection0

Date0

*omment0

2eneral Comments:

>nsert any comments and date of comment0

Date0 *omment0

12/19/2014 2:36 AM 22

Documents

Acc Criteria