Nevada Silver State Exchange // Status report 10.31.13

8/12/2019 Nevada Silver State Exchange // Status report 10.31.13

1/15


2/15

BOS IV&V Monthly Status Report

Deliverable 5.4.2.4Reporting Period:

September 1 October 31, 2013

www.pcgtechnologyconsulting.com


3/15

BOS Monthly Status ReportTopics

Project Summary

IV&V Activities Summary

IV&V Risks Summary

Planned Activities for Next Reporting Period

BOS IV&V Monthly Status Report 2


4/15

IV&V Project Summary


This section provides the consolidated status of IV&V and Quality Assurance activities in the table

below:Project Area* Risk

Status

IV&V Observations

Project Planning and

Administration

IV&Vs ability to assess vendor project planning and administration activities has been

constrained by the lack of visibility into these activities. While IV&V, Xerox and the SSHIX

staff attended bi-weekly Steering Committee meetings, many of these formal status meetings

were significantly abbreviated during this reporting period in order for Xerox to address

critical project concerns. IV&Vs access to project documentation has been minimal and

participation in ad hoc meetings limited.

Although Xerox presented an updated project schedule during this reporting period, most

critical milestones were not met and project delays occurred.

Requirements Specification /

Validation & Requirements

Specification Document

Significant failures occurred during integration testing which indicated a lack of coordination

between BOS and HCR-EE requirements definition, resulting in additional coding, testing,

and frequent builds.

In August, Xerox agreed to provide system extracts and update system documentation to

provide additional requirements traceability information to IV&V, which has yet to occur. As a

result, IV&V could not trace the BOS requirements to any of the testing results as of the date

of this report.

Data Retention Recovery

Protection and Management Plan

No change: IV&V Data Retention Recovery Protection and Management Plan review

completed; no risks identified at this time.

Hosting Plan No change: Hosting Plan review completed; no risks identified at this t ime.

* Project areas assessed depend on the deliverables that are available for review per the DD&I project schedule.

M

NA

NA

M


5/15

IV&V Project Summary (Continued)


Project Area Status IV&V Comments

Project Management Plan

(PMP)

IV&V has identified multiple risks (see IV&V Assessment workbook) in this area, which the State has

chosen to accept. IV&V continues to keep these risks open, however, to track the overall affect to the

project.

Risk Management IV&V continues to experience limited insight into formal risk management practices performed on behalf

of the project and believes that the project is not implementing best practices in relation to identifying,

tracking and mitigating project risks.

Implementation IV&V participated in BOS implementation meetings 3-times per week leading up to implementation,

which provided IV&V additional insight into the project.As 10/1/13 approached, concerns arose over the projects ability to go live on time. As a result, the NV

Governors Office stepped in to monitor the project. In one of the meetings, alternatives were presented

to delay implementation, which was rejected by the Governors Office. The BOS went live on 10/1/13

with (known) missing functionality, undefined business processes and incomplete testing, (including

CMS Blueprint testing). After implementation many critical defects were identified that required multiple

production builds. In the coming days / weeks IV&V believes that the project needs to prioritize defects

to be fixed, outstanding functionality to be implemented, and develop a schedule for doing so.

M

NA

H


6/15


BOS IV&V Monthly Status Report5


Test Management

User Acceptance Testing

Xerox completed wave testing as required by CMS and IV&V submitted the States Attestation Letter in

accordance with CMS regulations. BOS did not fully complete end-to-end testing by 10/1/2013 and

therefore IV&V was unable to fully assess if testing was successful during this reporting period.

The project did not complete Blueprint testing by 10/1/31 (with CMS approval), which has yet to be

scheduled as of the date of this report.

As IV&V had limited insight into the BOS User Acceptance Testing (UAT) process, IV&Vs assessment

in this area is based on the results of wave testing and defect reporting.

Interfaces Delayed design and development of interfaces created delays in testing and coordinated mapping ofdata. Since implementation, the project continues to identify data errors in the interfaces which could

not be identified before implementation due to the delays in interface development.

Release Management While IV&V completed its assessment of Xeroxs Release Management Plan several months ago,

release management is not scheduled to start until the BOS is in full production.

Defect Management BOS & HCR-EE are co-managing defects identified during testing. Teams do not share a defect

repository but are executing a process to ensure defects are recorded in each tool to ensure

coordination of identified defects.

Status has been updated to high due to the fact that a number of high magnitude critical defects

continue to occur.

Configuration Management No change: Configuration Management Plan released.

NA

NA

H

M

H


7/15



Change Management No change: Change Management assessment completed.

Performance and Scalability

Plan

IV&Vs system assessment of Performance and Scalability has been delayed to allow project resources

to focus on development of the BOS.

Security Management Single sign-on (SSO) functionally is still in development. The BOS has escalated this problem for

resolution. Lack of SSO negatively affecting testers ability to successfully complete testing. Should

SSO functionality not be feasible, workarounds will be required.

IV&V reviewed 3rdparty vendor security assessment to determine if vendor fulfilled contractual

obligations. No issues identified. IV&V is monitoring security related artifacts and has offered

assistance, if needed.

Detailed System Design Xeroxs second version of the SDD did not address significant issues/concerns that were raised in the

initial reviews of the draft documentation. IV&V rejected this version of the SDD and recommended that

the State do the same. IV&Vs assessment cannot be completed with the limited information provided

in this document. The SDD does not follow standard guidelines for acceptable design documentation. If

a well-documented and more complete version of the SDD is not delivered, the State will be unable to

determine if the BOS was built as designed.

NA

NA

L

M


8/15




Contingency Planning: The State should define contingency workarounds/processes and stakeholder communications based

on limited functionality available for 10/1/2013 implementation date. Work efforts should be coordinated

with HCR-EE to ensure all issues are resolved before implementation.

Status has been updated to medium. The BOS was implemented with multiple workarounds (i.e. paper

applications) due to missing functionality. The State has not provided updated contingency plans to

IV&V for review.

Risk Assessment Rating Legend

Not Applicable Low Medium High CompletedL M HNA C

M


9/15

IV&V Activities Summary (Continued)


IV&V activities completed in the current reportingperiod:

Participated in Exchange Steering Committee Meetings

Attended conference calls and webinars with CMS/CCIIO as appropriate

Participated in test planning including UAT, BluePrint, and joint HCR-EE / BOS testing

meetings

Testing ongoing and attestation letters submitted as required by CMS

Monitoring UAT testing progress

Attended daily defect meetings

Attended daily BOS programmer status meetings

Attended UAT progress meetings

Attended Governors Aide status meeting

Attended defect prioritization planning meetings


10/15


Contractors Project Management

e-pm.51122 Scope Management

e-pm.51235 Schedule (Time) Management

e-pm.51237 Contingency Planning

e-pm.52054 Implementation Communication Logistics e-pm.52067 Business Continuity Plan Lacks Business Operations

Representation

e-pm.52068 Command Center Model/Communications Difficulties

e.pm.52069 Implementation Activities/Objectives

e.pm.52070 Exchange Business Readiness

Testing

e-te.52073 UAT Defect Management

Security

b-se.52061 System Security Plan and Safeguards Procedure Report

IV&V Risks

A recap of the risks listed in the Monthly IV&V Assessment Workbook is provided

below:


11/15


IV&V Assessment Metrics

The IV&V assessment metrics provide a summary of the risks and findings reported

in the IV&V Assessment Workbook.

Base Risk Metrics:

Total Open22

Total Closed1

Total Accepted0

Total Updated8

Total Closed (30-day) 0

Total Open (30-day) 0

Note: 30-day period since last status

update

Total Open71%

Total Closed

3%

TotalAccepted

0%

TotalUpdated

26%

Total Closed(30-day)

0%

Total Open(30-day)

0%

Risks


12/15





Project Management11

Requirements0

Quality0

Operating Environment 0

Design & Development 0

Testing 8

Data Management 0

Security1

Open Risks by Section:

0

2

4

6

8

10

12

No.ofR

isks

Risk Sections

Open Risks by Section


13/15





Total Open79

Total Closed240

Total Accepted0

Total Updated 0

Total Closed (30-day) 0

Total Open (30-day) 1

Base Findings Metrics:

Note: 30-day period since last status update Total Open25%

Total Closed75%

Total Accepted0%

Total Updated0%

Total Closed

(30-day)0%

Total Open(30-day)

0%

Findings


14/15

IV&V Planned Activities


The following activities are planned for the next reporting period:

Continue participation in Exchange Steering Committee Meetings

Continue onsite presence when required

Participate in BOS daily status meeting

Initiate IV&V system assessments

Continue IV&V testing validations

Continue with Xerox deliverable reviews

Monitor defect resolution and Fast Follow development


15/15


Public Consulting Group Technology Consulting

2150 River Plaza Drive, Suite 380 Sacramento, CA 95833

(916) 565-8090 www.publicconsultinggroup.com

Documents

Nevada Silver State Exchange // Status report 10.31.13