December 1992 Computer Audit Update

developing such systems significantly underbid the other tenderers. Tenderers who had successfully developed such systems for other ambulance services. Did not the alarm bells ring that this was risk writ large? That they were adding additional unnecessary risks to the ones that had scuttled the previous attempts. Or was the choice dictated by cost. Trying to develop computer systems on the cheap has cost the service dearly, but not as dear a cost as some of the would-be users of the ambulances have suffered.

IT SECURITY TESTING, A P R A C T I C A L GUIDE u PART 2

THE SECURITY TESTING PROCESS

Bernard Robertson and David Pullen PA Consulting Group

IT security testing consists of a number of discrete stages. This article, which is the second in a series of nine on the subject of IT security test ing, descr ibes these phases and the documents which should be produced at each stage.

Definition of the system

The first step in running a security testing programme is to define the scope of the system to be tested. This scope should be agreed with management and may be hardware based (e.g. all systems in one physical location) or functional (e.g. all components involved in the processing of a specific business function).

System familiarization

Once the scope is defined, the testers should gain a thorough understanding of the system under test. All available documentation should be carefully studied. Any queries, inconsistencies or omissions should be noted and discussed with

the individual most able to help. The type and quantity of documentation which is available will vary from system to system, so it is impossible to give a complete list of what should be reviewed. Key documents which, if available, should be reviewed include:

• User statement of requirements.

• Functional requirements.

• Detailed design documents.

The quality of the documentation will also vary, therefore one of the important items to evaluate is the currency, completeness, accuracy and availability of the documentation.

Business and security control identification

Once the team has a working knowledge of the system, they can identify the core functions of the system (e.g. the payment of invoices, the production of accounts, etc.) and the procedures and controls to ensure that the functions operate securely and effectively. These may be access controls to the system (passwords, procedures, etc.) and/or financial limits (e.g. no cheque can exceed a certain amount). The controls and procedures should be identified and listed. Most of this information will normally be found in the functional specification.

Identification of risks

The next step is to determine the risks to the system which could have a significant impact on the business, or which could stop critical functions from being performed (e.g. fraud or loss or damage to equipment could mean that the cheques intended to be produced cannot be processed or the system is not available when required).

The process of risk identification involves identifying vulnerabilities in the system and determining the threats which may be used to exploit them. The team should identify the areas of the system which are subject to real threats and give a priority to each area selected for

©1992 Elsevier Science Publishers Ltd 3

Computer Audit Update December 1992

testing. It is probable that the time allocated for the testing will not allow for every detail to be tested, so the most important and fruitful areas should be tested first.

Selection of the appropriate testing techniques

There are a wide variety of testing techniques which may be used, from conventional functional testing of software to authorized 'hacking'. The test team should select the most appropriate testing techniques based on:

• The type of system to be tested.

• The maturity of the system.

• The scale and depth of testing required.

• The timescales and resources available.

• The experience of the testing team.

(Descriptions of commonly used testing techniques will be included later in this series.)

Determine overall strategy for testing

At this stage of the test process the test team should determine the overall strategy for the test programme. The strategy should be documented and should include: the overall objectives of the security testing, the resources required and a description of how the security testing will fit within the overall testing objectives of the system in question.

Specification of the tests

The tests which are to be performed on the system should be clearly specified at this stage. The test specification is a relatively detailed document which describes the content and structure of the test programme and enables staff outside the test team to understand the scope and focus of the testing. (Typical section headers of this document are highlighted in bold print.)

The introduction to the specification should clearly set out:

The objectives of the specification (i.e. to describe the tests to be carried out on a particular system in order to check its func- tionality).

The scope of the document (i.e. what is to be tested and what is regarded as being outside the scope of the testing programme).

• The intended readership of the document.

What assumptions have been made (e.g. that the testing will be on a current version of the system, and that a suitable test environment and data will be available).

• What related documentation is available (e.g. user documentation, etc.).

• How the test specification is structured.

The test background should cover:

The test rationale (i.e. why the testing pro- gramme was started and what it is trying to achieve).

The results of the risk identification phase (i.e. what are the main risks the key business functions face).

The test environment should specify exactly what equipment will be used for the test ing. This sect ion should include any requirements for test tools.

The requirement for test data should be specified in terms of 'live' data and/or test data. Responsibility for the production of test data should be defined.

The test groups identified at the end of the risk identification phase should be defined in this section. The objectives and expected results of

4 @1992 Elsevier Science Publishers Ltd

December 1992 Computer Audit Update

each test should be clearly described so that they can be agreed with system management. The test scripts (which are produced later) will describe these tests in more detail.

The problem reporting mechanisms should describe how problems will be recorded, reported, corrected and re-tested. Reports of identified problems should include details of:

• Problem type and importance (i.e. priority and impact)

• Problem title and reference number.

• How the problem was generated.

• The result of the problem.

• Exposures arising from the problem.

• The date the problem was discovered.

• Who discovered the problem.

Scripting

When all the tests have been specified, the test team should produce the test scripts. It is vital that each test to be performed is described in detail, with every action in the test and the expected result of that action clearly detailed. Scripts should be written in a simple but thorough manner so that a relatively unskilled tester can perform them without reference to the author of the script. The reliability and reproducibility of the test results depend on the quality of the test scripts produced.

If, for example, the test is designed to check that the input data is correctly validated, the script should describe what information should be input (field by field), and the expected result. In this case the first element of test data input would be intentionally incorrect and therefore fail the validation test. This should be repeated several times by inputting more invalid data before

inputting the correct data (which would be accepted).

Structured testing

Once the scripting of the tests has been completed, the structured testing can commence. The scripted tests should be carried out exactly as scripted and every keystroke should be recorded so that if a test fails, the circumstances of the failure can be recreated. The tester should keep extensive notes of what action was performed at each step of the script and also log exactly what happened. Each time a test fails, one or more problems may be identified and should be entered into the problem log. Two different types of structured tests will normally be performed, which are positive and negative tests.

Positive functional tests are conducted to provide confirmation of the correct functioning of the security features of a system. An example would be to test that only passwords longer than the specified number of characters are accepted by the system.

Negative functional tests are conducted to uncover any security weaknesses or system errors which create security problems. For example testing the effect of a field overflow in a financial amount field.

If the system documentation is incomplete or out of date the test specification and the scripts may not be correct. In these circumstances, it may be necessary to change the actions taken during the testing process and clearly note these changes in the scripts. In this event every keystroke must be recorded.

System penetration (or attack testing)

By this stage in the testing cycle the test team will be very familiar with the system under test. The next stage of the testing process is to try to discover and explore any weaknesses which have become apparent during the earlier phases, but which have not yet been fully exploited. The process starts with the development of attack scenarios which, in our experience, are best determined in a 'brainstorming' session with test team members and system users.

©1992 Elsevier Science Publishers Ltd 5

Computer Audit Update December 1992

All of the 'attacks' developed during the 'brainstorming' session may then be performed. If an attack scenario fails to achieve system penetration as originally designed, it may then be modified and re-tested. As before, each action taken must be recorded so that the tests can be recreated. Each time a security breach is encountered the identified problems should be logged.

System penetration testing differs in two ways from the structured testing process: firstly, it may use a variety of testing techniques; secondly, it is much less structured and far more experience based. It may be said that a good attack tester requires the mental i ty of a determined hacker.

The use of attack testing techniques can incur a substantial resource expenditure in terms of the tes te r ' s r equ i r emen t for sys tem familiarization, test planning and execution. However, these costs must be weighed against the risks of undetected security weaknesses remaining in the system during the operation.

Problem classification

The next stage in the test process is to examine the problems identified in the testing and system penetration phases and classify them into high, medium or low risk problems. A high risk p rob lem shou ld be f ixed urgent ly as it compromises the security of the system and jeopardizes the functioning of the system. A medium risk problem should be corrected when all high risk problems have been eliminated. A low risk problem should be fixed when resources are available.

Report overview

A report overview should be produced next. This should detail all the problems found and should give an early assessment to management of the severity of the problems with respect to the risks identified at the specification stage. The overview allows management to make early decisions on the action and the resources which are likely to be required.

Report ing

Using all the information recorded in the previous steps, the final report may be written. This report should be given a restricted circulation as there may be good reasons why all the recommendations cannot be implemented, particularly in the short term. It is therefore essential that this information is not made available to anyone who might use it for illicit purposes. The report will normally contain the following sections:

• Management summary.

• Introduction.

- - Background to the testing.

- - A description of the system that was tested.

- - The scope and objectives of the testing performed.

- - The approach taken to the testing.

- - The structure of the report.

Test process (i.e. a description of the actions taken during the test process). This may in- clude:

- - Risks identified.

- - Testing techniques used.

" Test environment (i.e. a precise description of the test environment).

• Major findings and a classification ofthe prob- lems found.

" Conclusions.

- - Number and severity of problems and their possible causes.

- - Effectiveness of security testing process.

6 @1992 Elsevier Science Publishers Ltd

December 1992 Computer Audit Update

Qual i ty of the sys tem documenta t ion reviewed.

- - Overall security level of the system.

Recommendations (these will, of course, vary from system to system) and suggested im- provements / alterations.

• Appendices.

- - Structured test results.

- - Attack testing results.

- - Problem classification.

- - Problem descriptions.

The final report is the formal output of the test process. However, it is essential that the documents produced during the test process are up to date and complete at the end of the testing. This allows errors to be recreated and verified, if required.

When security testing has been completed and the final report delivered, follow up action will be required to correct the problems identified during testing. A later article in this series entitled 'The Follow Up Process' details the objectives, process and common issues associated with a fol low up programme and gives hints and examples of how an effective follow up may be achieved.

Bernard Robertson is a principal consultant in the Security Consult ing Practice of PA Consulting Group. He has extensive experience in per forming a range of secur i ty testing programmes for public and financial sector clients. Bernard is a regular speaker on IT security issues and holds degrees in Economics and Business Administration. David Pullen is a senior consultant within the same Security Consulting Practice. Over the last five years he has conducted several security testing projects, including one lasting two years with a team of 15 security testing. David is a physics graduate and has produced security testing educat ional material.

PRACTICE MANAGEMENT SYSTEMS & PRACTICE PRODUCTIVITY

John Oates

This article considers the use of IT for practice productivity and management in small and medium-sized firms. Why should it be done? The answer to this question is - - inevitability. Although this may seem to be a rather strange reason, it is necessary to consider the following:

There are 87 million micros in the world (well that was last month!), consider the inevita- bility of it.

• Many clients will be starting to expect it.

• Competitors are doing it more and more.

It will become much harder to manually pro- cess increasing volumes of data cost-effec- tively.

It is necessary for the firm concerned to identify why the route is going to be tollowed (or rather for the majority of firms, to justify going further down the route). The benefits should become apparent as the elements of this article are further discussed.

In order for the process to work, and work well, it is essential that the company identifies a 'champion', an individual who is going to drive the project forward, in the face of the obstacles that will inevitably arise. This 'champion' needs to be highly placed within the practice, not necessarily at partner level, but there needs to be a partner committed to it. If the correct people are put in place at the initial stages, this should go a long way towards ensuring the success of the project. Any project like this needs to be driven. If the firm does not support it then it should not be embarked upon. In other words, do it right, or don't do it at all!

@1992 Elsevier Science Publishers Ltd 7