Embed Size (px)
Citation preview
1
How to Audit the System Development Life Cycle
William J. Papanikolas, CISA, CFSA
Western Michigan Chapter ISACA
October 20, 2011
2
Today’s Agenda
Defining SDLC
How to Impact the SDLC Process
What to Audit at Each Step
3
Defining SDLC
The System Development Life Cycle (SDLC) is the entire systems process from identifying a need through the final implementation of a solution.
SDLC is one of the best places for an auditor - and yet one of the least audited.
4
Defining SDLC
Successful SDLC projects are measured three ways:
Creating a Quality Product
Completing at Budgeted Cost
Completing on Approved Timetable
NOTE: The majority of SDLC projects fail to achieve even two of these goals!
5
Defining SDLC
Project
Initiation
Business
Requirements
Definition
Technical
Requirements
Definition
Software
Selection /
Coding
Testing Data
Conversion
Training and
Documentation
Final
Implementation
6
How to Impact the SDLC Process
Creating a Quality Product Provide assurance the final product is going to
deliver what has been promised
Ensure proper controls (automated and manual) have been designed into the new process
Completing at Budgeted Cost Ensure SDLC oversight includes controls for costs
Completing on Approved Timetable Ensure SDLC oversight includes controls for
timeliness
7
Project Initiation
Project champion determined.
Project charter developed.
High level timelines and budgets determined.
Project team assigned; roles and responsibilities established.
Project monitoring and accounting set up.
8
Project Initiation - Audit Ideas
Quality Product
Appropriate stakeholders are involved.
Project champion represents the key stakeholders.
Project is consistent with the organization’s strategic plans.
9
Project Initiation - Audit Ideas
On Time and On Budget
Budget was properly determined (watch out for approval cutoffs!).
Timeline is realistic given project magnitude and past organizational experience.
Appropriate metrics and reporting schemes are developed.
10
Business Requirements Definition
Primary system functions defined.
Usability targets established (e.g., 24x7, 1 second response time).
Management reporting requirements understood.
Regulatory and legal implications considered.
End user screen requirements determined.
11
Business Requirements Definition - Audit Ideas
Quality Product
Appropriate stakeholders are represented.
Security requirements are defined.
Automated and manual controls are considered.
12
Business Requirements Definition - Audit Ideas
On Time and On Budget
Project plan and budget remain realistic given business requirements.
Business requirements do not overly rely on new and/or unproven technologies (e.g., a requirement that all transactions will process over the intranet).
13
Technical Requirements Definition
Processing platform(s) determined
Necessary hardware acquisitions outlined.
System capacity requirements understood (both processing speed and data storage).
Network modifications defined.
Data structures created.
14
Technical Requirements Definition - Audit Ideas
Quality Product
Technical requirements support the business requirements.
Members of all impacted technical units represented.
Technology assumptions are properly validated through internal experience or external site visits.
Links to existing applications are defined and controlled (e.g., control totals)
15
Technical Requirements Definition - Audit Ideas
On Time and On Budget
Project plan and budget remain realistic given technical requirements.
Lead times for purchasing, receiving, installing and testing new hardware have been properly reflected in the timeline.
16
Software Selection/Coding
Request for Proposal created.
Vendor and software selection criteria established.
Contract terms established.
Programming teams assigned for coding and modification.
Software loaded in test environment.
17
Software Selection/Coding – Audit Ideas
Quality Product RFP and vendor assessments come directly from business and technical requirements.
Selected vendor has experience in your industry, with companies your size, and with similar setups.
Vendor is financially stable and will be around for long term support (alternatively, the source code could be owned by your organization).
Proper change management and security controls are set up for the coding environment.
18
Software Selection/Coding – Audit Ideas
On Time and On Budget
Vendor contract terms are favorable, and include clauses on cost overruns.
Vendor contract includes rewards/penalties for project timeliness.
Project plan appropriately reflects the resources and time necessary to install, code and modify.
19
Testing
Unit testing completed for each system element.
Integrated testing completed for each system module.
System testing completed for overall system and related interfaces.
Stress testing completed for online performance and data storage/retrieval.
End user testing completed.
20
Testing - Audit Ideas
Quality Product
All testing is performed in an appropriate environment with adequate security.
All issues noted during testing are communicated to the proper owner within the project.
Test cases reasonably reflect the environment as it will appear in production.
Change management controls are in place as system elements progress through the testing cycle.
21
Testing - Audit Ideas
On Time and On Budget
Resolution of test issues is focused on items that are necessary to achieve business or technical requirements (not all issues must be solved prior to going live!).
Project plans are properly updated to reflect issues noted in testing that must be resolved.
22
Data Conversion
Data from the old system(s) is properly cleansed prior to conversion.
Converted data is evaluated to ensure it is accurate and complete.
23
Data Conversion - Audit Ideas
Quality Product
Data is accurately mapped from the old system to the new.
Key data elements are screened using software (or manually in some cases) to ensure anomalies are removed.
After conversion, sample data reflects accurate transfer.
Control totals of key data fields/tables show consistency in the old and new data structure.
24
Data Conversion - Audit Ideas
On Time and On Budget
Project plans are properly updated to reflect issues noted in data conversion that must be resolved.
25
Training and Documentation
Users, operators, database administrators, management, etc. receive the training required to operate and use the system.
Documentation is provided for all users and operators of the system.
26
Training and Documentation – Audit Ideas
Quality Product
Training addresses both system usage and business process.
Training includes all affected parties.
Training is provided close enough to implementation to allow participants best retention.
Documentation (online and paper) is organized in a way to be useful to users and operators.
27
Training and Documentation – Audit Ideas
On Time and On Budget
Training and documentation are properly included in the project plan and budget.
28
Final Implementation
Final system running in the production environment.
New hardware, networking, etc. comes online.
Business processes change over to accommodate new system.
29
Final Implementation - Audit Ideas
Quality Product
Promotion to production environment follows established change management procedures.
Parallel processing with old system(s) commences.
Help desk and “swat teams” are in place.
System backout procedures are established.
30
Final Implementation - Audit Ideas
On Time and On Budget
Final costs are captured and summarized (watch out for implementation problems being defined as “on-going maintenance”).
Project teams are closed down as the implementation continues.
31
What’s Next?
Post-Implementation Review
Lessons Learned
Final Reporting
Questions?
