Mitigating Technology Implementation Risks: Embedding

Mitigating Technology Implementation Risks: Embedding Software Engineering and Controls Practices into the Organization’s Project

Structure

ETM 5121 Project ProposalJoseph Robinson

July 3, 2004

Joseph Robinson Page 1 4/13/2023

1.0: Introduction

A Catalyst for ChangeThe Enron and WorldCom corporate scandals have forever changed the way organizations will conduct business operations. Reason: These events have provided the catalyst for the creation of government regulations, appropriately named Sarbanes-Oxley after the two members of the U.S. Congress and Senate who were instrumental in the bill’s passing, mandating corporate governance; these new regulations culminate with the attestation by CEO’s and CFO’s to the integrity and accuracy of the particular organization’s financial reporting. At the core of this integrity, however, lies a company’s information system: the source, repository, and reporting mechanism of financial data for most publicly held organizations. Accordingly, a proposal that seeks to mitigate, or at best minimize, the risk of newly implemented information systems’ data integrity is likely appropriate.

Proposal for an Embedded Risk StructureIn an effort to further stabilize new technology implementations and ensure information systems’ data integrity risks are mitigated, it is proposed that the organization utilize certain key processes and steps that will be embedded within the current project structure. The employment of this embedded risk mitigation structure will enhance the current project process, and more importantly the technology implementation, in several ways: 1) Provide clear guidance to all project participants around the mitigation of data integrity risks 2) Outline the risk mitigation process to be employed 3) Ensure risk mitigation activities are formalized and 4) Ensure the completeness of these activities through the identification of specific project milestones. Ultimately, this risk mitigation structure provides assurance that the organization is applying due diligence in addressing the accuracy and integrity of its financial data. More importantly, however, this structure will allow the organization to certify its financial information according to the guidelines set forth by Sarbanes-Oxley. And recent events further substantiate the need for this mandate in the Gap Inc. organization.

Lessons from Recent HistoryDuring the phased implementation of the company’s newly purchased point-of-sale system, certain problems have surfaced: 1) transactions are causing a random printer freeze condition, 2) transactions are being dropped from the system record without warning, and 3) customers’ credit card purchases are being captured incorrectly. Although these software and hardware problems were purportedly identified and acknowledged during the testing phase of the implementation project, a decision to continue with the scaled rollout was determined.

The risk posed to financial reporting as the point-of-sale technology rollout continues should be obvious: Missing and incorrect transactions aggregated over the total population of retail locations, some 4,000 stores, will likely have a substantial impact on the company’s reported sales. Furthermore, these failed transactions also require human effort to mitigate or correct. Consequently, as the organization continues to ramp up this new technology, this manual effort will also increase exponentially until the errant


transactions are corrected. Obviously, this breakdown in risk analysis and mitigation further exhibits the need for a formal structure that addresses and provides transparency to such conditions. Accordingly, the proposal that follows addresses this need.

This proposal is structured in the following manner:

Section 1.0; IntroductionSection 2.0; Problem StatementSection 3.0; Measures of the ProblemSection 4.0; Project ObjectivesSection 5.0; Specific DeliverablesSection 6.0; Alternatives to be consideredSection 7.0; Project Approach and Criteria for Evaluating AlternativesSection 8.0; Project PlanningSection 9.0; Planned Use of ETM Materials

2.0: Problem Statement

A Mandate for Expanded Project ScopeGap Inc. has a number of impending technology implementations that will not only have a substantial impact on business processes, but that will also create, store, and provide the data for financial reporting. For this reason, it is proposed that the company expands the scope of the current project structure to include the minimum activities, or components, necessary to support a technology risk mitigation structure; this will ensure the level of risk to financial reporting inaccuracy is minimized through a formal and repeatable process. This approach also aligns with the company’s current Sarbanes-Oxley compliance efforts: documentation of business processes, identification of the control structure1and any control gaps, and testing of the control environment2.

Closing the Gap in New Technology RiskAlthough, current activities encompass the assessment of the level of control within existing technologies, the assessment of the control environment of new technologies is not addressed with the same level of rigor and formality. Consequently, this absence of applied structure creates a potential lapse in the assessment of impending implementations, potentially leading to a condition wherein the accuracy of financial reporting is at risk. Given the number of impending implementations during the next few years, the organization’s level of risk will likely increase exponentially if this lapse is not addressed. Impending exposure points will include the accuracy of inventory data, sales data, and lease management data, as new technologies are either being planned or currently underway for these areas. This exposure is also compounded by the large size of the IT Department (the diffusion of information is difficult) and their relative unfamiliarity with the requirements of Sarbanes-Oxley.

1 A control structure refers to the manual (human process) and automated (system process) controls that provide a level of assurance that risk is minimized or prevented. 2 The control environment consists of a series of manual and/or automated controls that, combined, provide a level of assurance that risk is minimized or prevented. This can also be referred to as a “web of controls”.


The IT FactorCurrently, the Information Technology (IT) Department supporting the organization includes approximately 1500 employees. Within this large population of employees are many sub-organizations that support specific technology implementation projects. Historically, communicating new requirements across the global IT organization has been cumbersome and lethargic, as separate entities within IT are slow to react to these corporate-wide mandates. Additionally, the level of importance placed on these new mandates varies widely across the IT hierarchy, resulting in varying degrees of adaptation; this has resulted in sporadic adherence and performance relating to any new requirements and correlating measures. Currently, Sarbanes-Oxley requirements, related to new technology implementations, are generally unclear to the IT Department; this condition is further exacerbated by the lack of formal structure around these risk mitigation efforts.

3.0 Measures of the Problem The new technology risk mitigation structure’s measures will directly support the major objectives of the initiative (previously outlined in Section 4): 1) Provide clear guidance to all project participants around the mitigation of data integrity risks 2) Outline the risk mitigation process to be employed 3) Ensure risk mitigation activities are formalized and 4) Ensure the completeness of these activities through specific project milestones.

1. Increased understanding of Sarbanes-Oxley requirements and the relationship to technology; this will be measured through surveys conducted via the company’s intranet, prior to and following the proposed program’s rollout. The survey will include specific questions around IT employees’ level of knowledge related to Sarbanes-Oxley requirements, their role in meeting these new requirements, etc. (where 1 = Unfamiliar, 2 = Somewhat Unfamiliar, 3 = Familiar, 4 = Somewhat Familiar, 5 = Very Familiar). [Supports Project Objectives 1, 2, and 3]

2. Reduction in the number of post-implementation problem tickets opened for the specific technology (vs. past implementations). This measure will be accomplished by comparing the incidence (frequency) and severity of problem tickets relating specifically to reporting and the accuracy of data. Currently, the organization’s problem ticket software, Service Center 5.0, specifically identifies a problem ticket category, including reporting problems, the specific application the problem relates to, etc. These problem queues can then be queried to identify trends. Assumption: If the level of rigor related to financial reporting accuracy and data integrity is embedded into the project structure, a decrease in the number of the post-implementation incidents should occur. [Supports Project Objectives 2, 3, 4, and 5]

3. Increased management satisfaction with level of rigor related to new technology risk mitigation. Currently, management’s confidence in the level of rigor applied to new implementations risk related to financial reporting is low; this will be measured through management feedback on specific questions related to the perception of new technology risk mitigation efforts and processes, conducted prior to and after the proposed program’s rollout. Management responses will be


rated on a scale from 1-5, where 1 = Below Expectations, 2 = Somewhat Below Expectations, 3 = Meets Expectations, 4 = Somewhat Above Expectations, 5 = Exceeds Expectations. [Supports Project Objectives 3 and 4]

4.0 Project Objectives The objective of this project is to develop a formalized and repeatable risk mitigation structure, which:

1. Is embedded in the organization’s existing project structure;2. Provides clear guidance to all project participants around the mitigation of data

integrity risks;3. Outlines the risk mitigation process to be employed;4. Ensures risk mitigation activities are formalized;5. And ensures the completeness of these activities through the identification of

specific project milestones.

5.0 Specific Deliverables The specific deliverables that address the problem statement and achieve the objectives stated for developing a formalized and repeatable risk mitigation structure are:

1. A revised project template, including Project Charter, Standard MS Project Template, and Test Plan Template that includes specific risk mitigation activities, objectives, requirements, and milestones. [Supports Project Objectives 1, 2, 3, 4, and 5]

2. A formalized process, including a Process Flow Document, for risk assessment activities, including pre-implementation, project phase, and post-implementation process steps. [Supports Project Objectives 3, 4, and 5]

3. A formal communication document that includes a summary of the Sarbanes-Oxley impact, requirements resulting from this impact, and other relevant information. [Supports Project Objective 2]

4. A global Sarbanes-Oxley calendar that outlines the timeline for certification and the annual recertification (sustainability) timeline and the key activities required supporting these milestones. [Supports Project Objectives 2, 4, and 5]

5. Summary of measures (scorecard), including the surveys and problem ticket (pre- and post- program) analysis. [Supports Project Objectives 1 and 5]

6.0: Alternatives to be considered In order to effectively develop and deploy a formalized and repeatable risk mitigation structure, meet the project objectives and provide the defined deliverables, certain alternatives will be considered. It should be noted that the alternatives involve both the method of achieving the objectives and ownership and accountability for the program itself; and in some cases, the alternative is simply the specific combination of these variables.


6.1: Risk mitigation processes owned and administered by the business (100% business accountability)

Description: The business segment, or functional area, receiving the technology solution would own both the process of risk mitigation and its administration (project documentation). Note: In some cases, multiple departments may be the recipients of the solution; consequently, the main, or largest, user (department) would be identified. In this scenario, the functional area would provide the resources to complete the risk mitigation activities and completion of related project milestones.

Technology: The business would require a relatively high level of technical competence to support the risk mitigation effort, as much of this work will be completed in the new information system (data structure, reporting analysis, testing analysis, etc.). It is not apparent that this level of technical competence is readily available or exists today, however. Additionally, this risk mitigation work would also require some understanding of general control principles (manual and automated controls, control environment, etc.).

Successful Use: Many functional areas already support many of the technical processes around applications, including user administration, query and report writing, the initiation of batch jobs, etc. Consequently, the functional areas likely have resources capable of supporting some level of risk mitigation activities.

Feasibility: Although the business could likely support this approach to some degree, the level of technical knowledge needed to perform a thorough and detailed analysis of the new system may exceed business expertise and capabilities. Moreover, access to much of the IT infrastructure required to complete this work may be difficult, as this exceeds the normal access level granted by the IT Department. Finally, business areas are less familiar with the project structure and this might result in some level of risk to project milestones and timelines.

Known Preconceived Biases: It’s likely that the functional area resources would approach the analysis work with a high degree of probity3. More importantly, results of these analysis activities would likely be reported to leadership without bias for the success or failure of the project.

6.2 Risk mitigation processes owned and administered by IT (100% IT accountability)

Description: IT owns both the process of risk mitigation and its administration (project documentation). IT would provide the resources to complete the risk mitigation activities and completion of related project milestones.

3 According to Arthur M. Squires, in his text The Tender Ship, “an applied scientist or engineer shall display utter probity toward the engineered object, from the moment of its conception through its commissioning for use”. [2]


Technology: IT would require a relatively high level of technical competence to support the risk mitigation effort, as much of this work will be completed in the new information system (data structure, reporting analysis, testing analysis, etc.). Additionally, this risk mitigation work would also require some understanding of general control principles (manual and automated controls, control environment, etc.). Although IT has a high degree of technical competency, the general controls knowledge is not developed.

Successful Use: IT directly supports the organization’s technical processes and has a track record of successful technology implementations.

Feasibility: Feasibility is considered medium: IT has the technical competency to support this approach, but bias must also be considered (see bias section below). IT could easily support these activities, as the technical knowledge needed to perform a thorough and detailed analysis of the new system exists today. In addition, IT has ready access to the system infrastructure required to complete this work. IT is also extremely familiar with the project structure and this will likely enable achievement implementation milestones.

Known Preconceived Biases: As IT has an interest in the successful implementation of the new technology (performance appraisals are directly related to this measure), certain priorities and biases may preexist. Consequently, certain failures in testing, weaknesses in the control environment, and other risks may be overlooked, or under-prioritized, in the interest of a successful technology implementation.

6.3 Risk mitigation processes owned by IT and administered by functional (business) areas (50% IT accountability; 50% business accountability)

Description: IT owns the process of risk mitigation and the business owns its administration (project documentation). IT would provide the resources to complete the risk mitigation activities; however, the functional areas would ensure completion of related project milestones. This scheme ensures that the technical work can be supported by IT, while the functional areas will have a certain degree of oversight to ensure an acceptable level of rigor has been applied to the analysis activities and that variances, or deficiencies, in system performance receive due scrutiny. Approval and signoff of milestone completion would be completed by the business, not by IT.

Technology: IT would require a relatively high level of technical competence to support the risk mitigation effort, as much of this work will be completed in the new information system (data structure, reporting analysis, testing analysis, etc.). Additionally, this risk mitigation work would also require some understanding of general control principles (manual and automated controls, control environment,


etc.). In addition, the functional areas would require a high degree of project understanding to ensure the administration process is effectively supported.

Successful Use: In the past, IT and the functional areas have demonstrated a high level of collaboration and cooperation in completing new technology implementations.

Feasibility: Feasibility would be considered high, as this structure likely exploits the skills and interests of both areas (IT and functional department). IT could easily support the risk analysis efforts due to existing technical knowledge. While, functional areas will ensure project milestones are not only achieved, but that the results are valid.

Known Preconceived Biases: In this scenario, biases are likely removed due to joint ownership and accountability for the process and corresponding results.

7.0: Project Approach and Criteria for Evaluating AlternativesBecause this particular proposal involves the improvement and enhancement of an existing process, the initial structure employed to support the definition, measurement, and analysis will be Six Sigma’s DMAIC (Define, Measure, Analyze, Improve, and Control) Cycle. Assuming approval of this proposal (and for the purposes of the assignment), a project approach, utilizing the requirements, design, construction, implementation/deployment, and post-implementation review phases will be employed for the “Improve” portion of DMAIC. Consequently, the project plan for this proposal will include elements of both DMAIC and a normal, phased technology implementation.

The sponsor for this project will be the Sarbanes-Oxley Steering Committee; this executive group represents an adequate cross section of all area that will likely be impacted by this proposed project. The author will execute and directly support the project, including defining project requirements (in collaboration with business partners), scope, measurement, analysis, the presentation of findings to the Steering Committee, and management of the implementation, including communications. No additional resources will be required to support this effort, as the brunt of the project involves the development of a system or process, not the delivery of a technical solution, per se.

7.1 Define Phase

1. Clarify business goals and objectives of the project Develop high-level goals and objectives. Example: In an effort to further

stabilize new technology implementations and ensure the information system’s data integrity risk is mitigated, it is proposed that the organization utilize certain key processes and steps that will be embedded within the current project structure.

2. Define detailed objectives of the project The objectives stated in Section 4 of the document are an appropriate

definition of the project objectives


3. Identify team members4. Plan overall project (includes task list, Gantt Chart, etc.)5. Evaluate Alternatives

Develop alternatives based on understanding of organization’s current and past capabilities and limitations

Assess feasibility of each alternative, based on capabilities and limitations and rank these according to a high, medium, or low value. A decision matrix will be used for this process, utilizing specific decision criteria that are then weighted according to importance (Figure 1). Note: The example only includes the criteria from previous section (Section 6); however, actual matrix will likely include additional criteria.

6. Develop and Present Proposal, Alternatives and Recommendation Present problem definition, high-level solution, objectives, measures,

alternatives, recommendation of alternatives, and project plan to organizational

Approval from Steering Committee to continue with project7. Create Project Charter

Further define requirements Outline objectives at a level that identifies specific deliverables,

including service levels, system response times (if applicable), etc. Ensure requirements are understood and improved by all impacted parties

Define project scope: This section of the Project Charter should outline the boundaries (if any) of the project (e.g. “European locations not in scope”)

6. Communicate project details, including 1 – 5, to all impacted parties

Figure 17.2 Measure Phase (Understanding the Process)

1. Collect Data Document (flow chart) current process supporting Sarbanes-Oxley (for

understanding of change impact) [1]


Create Cause & Effect (C&E) Diagram4 to better understand the problem or breakdown

Gather information related to intended measures (as-is state): 1) Understanding of Sarbanes-Oxley Survey 2) Problem ticket volume 3) Management Satisfaction Survey and assess/record current performance. Note that these measures support the performance objectives (directly related to the customer).

Create a Control Chart5 to capture as-is performance of system or process, if applicable [1]

7.3 Analyze Phase (Determine Major Causes of Defects or Process Breakdowns)

1. Assess and record results from the surveys and problem ticket performance Create Multi-Vari Charts6

2. Establish target measures for post-implementation results3. Develop key project metrics scorecard [Section 5.0; Item #5]

7.4 Improve Phase (Removing the Major Causes of Defects)

1. Design New Solution Complete to-be process design : Utilize information from previous project

phases to develop a process model (to-be process map); this model will be used to create the formal risk assessment process, including pre-implementation, project phase, and post-implementation process steps [Section 5; Item #2]

Complete updates to project documentation : Based on the new process, the project documentation will be updated with these new activities [Section 5; Item #1]

Complete formal communication : Develop global communication draft based on completion and design of previous deliverables [Section 5; Item #3]

Complete Sarbanes-Oxley Calendar [Section 5; Item 4] Receive approval for design from Steering Committee

2. Implement the New Process Conduct training : Train impacted parties on roles/responsibilities related

to the new process Collect Data : Collect and analyze the new system performance data to

validate improvement Develop Plan : Develop plan to quantify and fix causes of variation in the

new process

4 This technique, also referred to as a fishbone diagram, exposes possible causes of defects from such sources as materials, equipment, methods or processes, or personnel [1].5 A statistical tool used to track an important condition over time and watch for changes in both the average and the variation of the condition [1].6 A chart depicting variances within units, between units, samples, and lots [1].


Test the Process: Test the new process to ensure design is effective; make adjustments to design where failures occur

7.5 Control Phase (Maintaining the Improvements)

1. Develop a Control Plan7

2. Put controls in place3. Monitor and control process variation

7.6 Post-Implementation Review Phase (Close Out)

1. Post-Implementation Review (PIR) : Conduct PIR to capture lessons learned (related to project challenges, opportunities, etc.)

2. Prepare presentation and report out to leadership

8.0 Project PlanThe project plan follows the project approach outlined in Section 7.0 of this proposal. Target dates, predecessors, successors, and baseline durations have been assigned to each task. Note that a number of approval points, or milestones are included in the project structure. Moreover, many of these dates may appear excessive; however, much of this work will be completed by a single resource (the author) and many of the business units providing information for analysis are dispersed. Consequently, a certain degree of slack was purposely built into the timeline in appreciation of these potential constraints.

The first screenshot (Figure 2) below includes the initial proposal work and the final presentation; these activities encompass approximately six (6) months of work effort. The second screenshot (Figure 3) outlines three project phases: Define, Measure, and Analyze. Finally, Figure 4 details the Improve, Control, and Close Out Phases.

Figure 2

7 A description of the inputs to a process that should be monitored or error-proofed for the purpose of maintaining satisfactory output, or system performance [1]


Figure 3

Figure 4


9.0 Planned Use of ETM MaterialsAlthough each of the MSETM courses has provided, and continue to provide, key insights and learning, the courses outlined below are especially significant in terms of this project proposal:

Management of Technology Implementations: Much of the MSETM coursework provided insight to the challenges and obstacles in implementing technology and process change. For example, during the ETM 5111 course, a text entitled Strategic Management of Technology and Innovation provided many case studies that provided value in determining the approach for this project proposal; in fact, the text has been oft referred to during the past year for several projects.

Currently, the IEM 5010 course, Leading and Managing Technology Implementation, has provided key insights to issues that were never considered: probity in development and engineering projects, the lessons that failures provide, etc. This course should continue to provide a plethora of valuable learning that can be readily applied to this proposed project; and this information is fascinating, as well.

Strategic Quality Leadership/Management: Much of the impact of misguided technology implementations is unknown and unknowable. This paraphrased Deming principle is definitely applicable to the point-of-sale example presented earlier in this proposal document. For example, the impact of transactions not accurately captured could, in many ways, have a downstream impact on customer service and, ultimately, the customers’ perception of the brand. Accordingly, this guiding principle, among other quality principles will definitely be employed during this project.

References

1. Breyfogle, F., Cupello, J, and Meadows, B. Managing Six Sigma: A practical guide to understanding, assessing, and implementing the strategy that yields bottom-line success. New York, NY: John Wiley & Sons, Inc., 2001.

2. Squires, A.M. The Tender Ship: Governmental Management of Technological Change. Cambridge, MA: Birkhauser Boston, Inc., 1986.


Documents

Mitigating Technology Implementation Risks: Embedding