CS 5032 L12 security testing and dependability cases 2013

Dependability and Security Assurance, 2013 Slide 1

Dependability and Security Assurance

Lecture 2


Security testing

• Testing the extent to which the system can protect itself from external attacks.

• Problems with security testing

– Security requirements are ‘shall not’ requirements i.e. they specify what should not happen. It is not usually possible to define security requirements as simple constraints that can be checked by the system.

– The people attacking a system are intelligent and look for vulnerabilities. They can experiment to discover weaknesses and loopholes in the system.

• Static analysis may be used to guide the testing team to areas of the program that may include errors and vulnerabilities.


Security validation

• Experience-based validation

– The system is reviewed and analysed against the types of attack that are known to the validation team.

• Tiger teams

– A team is established whose goal is to breach the security of the system by simulating attacks on the system.

• Tool-based validation

– Various security tools such as password checkers are used to analyse the system in operation.

• Formal verification

– The system is verified against a formal security specification.


Examples of entries in a security checklist

Security checklist

1. Do all files that are created in the application have appropriate access permissions? The wrong access permissions may lead to these files being accessed by unauthorized users.

2. Does the system automatically terminate user sessions after a period of inactivity? Sessions that are left active may allow unauthorized access through an unattended computer.

3. If the system is written in a programming language without array bound checking, are there situations where buffer overflow may be exploited? Buffer overflow may allow attackers to send code strings to the system and then execute them.

4. If passwords are set, does the system check that passwords are ‘strong’? Strong passwords consist of mixed letters, numbers, and punctuation, and are not normal dictionary entries. They are more difficult to break than simple passwords.

5. Are inputs from the system’s environment always checked against an input specification? Incorrect processing of badly formed inputs is a common cause of security vulnerabilities.


Process assurance

• Process assurance involves defining a dependable process and ensuring that this process is followed during the system development.

• Process assurance focuses on:– Do we have the right processes? Are the processes

appropriate for the level of dependability required. Should include requirements management, change management, reviews and inspections, etc.

– Are we doing the processes right? Have these processes been followed by the development team.

• Process assurance generates documentation– Agile processes therefore are rarely used for critical

systems.


Processes for safety assurance

• Process assurance is important for safety-critical systems development:

– Accidents are rare events so testing may not find all problems;

– Safety requirements are sometimes ‘shall not’ requirements so cannot be demonstrated through testing.

• Safety assurance activities may be included in the software process that record the analyses that have been carried out and the people responsible for these.

– Personal responsibility is important as system failures may lead to subsequent legal actions.


Safety related process activities

• Creation of a hazard logging and monitoring system.

• Appointment of project safety engineers who have explicit responsibility for system safety.

• Extensive use of safety reviews.

• Creation of a safety certification system where the safety of critical components is formally certified.

• Detailed configuration management.


Hazard analysis

• Hazard analysis involves identifying hazards and their root causes.

• There should be clear traceability from identified hazards through their analysis to the actions taken during the process to ensure that these hazards have been covered.

• A hazard log may be used to track hazards throughout the process.


A simplified hazard log entry Hazard Log Page 4: Printed 20.02.2009

System: Insulin Pump SystemSafety Engineer: James Brown

File: InsulinPump/Safety/HazardLogLog version: 1/3

Identified Hazard Insulin overdose delivered to patient

Identified by Jane Williams

Criticality class 1

Identified risk High

Fault tree identified

YES Date 24.01.07 Location Hazard Log, Page 5

Fault tree creators Jane Williams and Bill Smith

Fault tree checked

YES Date 28.01.07 Checker James Brown

System safety design requirements

1. The system shall include self-testing software that will test the sensor system, the clock, and the insulin delivery system.

2. The self-checking software shall be executed once per minute.

3. In the event of the self-checking software discovering a fault in any of the system components, an audible warning shall be issued and the pump display shall indicate the name of the component where the fault has been discovered. The delivery of insulin shall be suspended.

4. The system shall incorporate an override system that allows the system user to modify the computed dose of insulin that is to be delivered by the system.

5. The amount of override shall be no greater than a pre-set value (maxOverride), which is set when the system is configured by medical staff.


Safety and dependability cases

• Safety and dependability cases are structured documents that set out detailed arguments and evidence that a required level of safety or dependability has been achieved.

• They are normally required by regulators before a system can be certified for operational use. The regulator’s responsibility is to check that a system is as safe or dependable as is practical.

• Regulators and developers work together and negotiate what needs to be included in a system safety/dependability case.


The system safety case• A safety case is:

– A documented body of evidence that provides a convincing and valid argument that a system is adequately safe for a given application in a given environment.

• Arguments in a safety case can be based on formal proof, design rationale, safety proofs, etc. Process factors may also be included.


The contents of a software safety case

Chapter Description

System description An overview of the system and a description of its critical components.

Safety requirements The safety requirements abstracted from the system requirements specification. Details of other relevant system requirements may also be included.

Hazard and risk analysis Documents describing the hazards and risks that have been identified and the measures taken to reduce risk. Hazard analyses and hazard logs.

Design analysis A set of structured arguments (see Section 15.5.1) that justify why the design is safe.

Verification and validation A description of the V & V procedures used and, where appropriate, the test plans for the system. Summaries of the test results showing defects that have been detected and corrected. If formal methods have been used, a formal system specification and any analyses of that specification. Records of static analyses of the source code.

Review reports Records of all design and safety reviews.

Team competences Evidence of the competence of all of the team involved in safety-related systems development and validation.

Process QA Records of the quality assurance processes (see Chapter 24) carried out during system development.

Change management processes

Records of all changes proposed, actions taken and, where appropriate, justification of the safety of these changes. Information about configuration management procedures and configuration management logs.

Associated safety cases References to other safety cases that may impact the safety case.


Structured arguments

• Safety/dependability cases should be based around structured arguments that present evidence to justify the assertions made in these arguments.

• The argument justifies why a claim about system safety/security is justified by the available evidence.


Structured arguments


Insulin pump safety argument

• Arguments are based on claims and evidence.

• Insulin pump safety:– Claim: The maximum single dose of insulin to be

delivered (CurrentDose) will not exceed MaxDose.

– Evidence: Safety argument for insulin pump (discussed later)

– Evidence: Test data for insulin pump. The value of currentDose was correctly computed in 400 tests

– Evidence: Static analysis report for insulin pump software revealed no anomalies that affected the value of CurrentDose

– Argument: The evidence presented demonstrates that the maximum dose of insulin that can be computed = MaxDose.


Structured safety arguments

• Structured arguments that demonstrate that a system meets its safety obligations.

• It is not necessary to demonstrate that the program works as intended; the aim is simply to demonstrate safety.

• Generally based on a claim hierarchy. – You start at the leaves of the hierarchy and

demonstrate safety. This implies the higher-level claims are true.


A safety claim hierarchy for the insulin pump


Safety arguments

• Safety arguments are intended to show that the system cannot reach in unsafe state.

• They do NOT demonstrate that the system is correct.

• They are generally based on proof by contradiction

– Assume that an unsafe state can be reached;

– Show that this is contradicted by the program code

• The contradiction means that the computation is therefore NOT UNSAFE i.e. SAFE.


Construction of a safety argument

• Establish the safe exit conditions for a component or a program.

• Starting from the END of the code, work backwards until you have identified all paths that lead to the exit of the code.

• Assume that the exit condition is false.

• Show that, for each path leading to the exit that the FINAL VALUE ASSIGNMENTS made in that path contradict the assumption of an unsafe exit from the component.


Insulin dose computation with safety checks

-- The insulin dose to be delivered is a function of blood sugar level, -- the previous dose delivered and the time of delivery of the previous dose

currentDose = computeInsulin () ; // THIS IS THE AMOUNT DELIVERED

// Safety check—adjust currentDose if necessary. // if statement 1if (previousDose == 0) //Assigns value to CurrentDose{

if (currentDose > maxDose/2)currentDose = maxDose/2 ;

}else

if (currentDose > (previousDose * 2) )currentDose = previousDose * 2 ;

// if statement 2if ( currentDose < minimumDose ) // ONLY EXECUTED IF DOSE UNSAFE

currentDose = 0 ;else if ( currentDose > maxDose )

currentDose = maxDose ;administerInsulin (currentDose) ;


Informal safety argument based on demonstrating

contradictions


Neither branch of if-statement 2 is executed

IF-THEN branch of if-statement 2 is executed

IF_ELSE branch of if-statement 2 is executed

currentDose = 0 currentDose = maxDose

CurrentDose is >= minimumDose and

<= maxDose

In all cases, the post conditions contradict the unsafe condition that the dose administered is greater than maxDose.


Key points

• Security validation is difficult because security requirements state what should not happen in a system, rather than what should. Furthermore, system attackers are intelligent and may have more time to probe for weaknesses than is available for security testing.

• Security validation may be carried out using experience-based analysis, tool-based analysis or ‘tiger teams’ that simulate attacks on a system.

• It is important to have a well-defined, certified process for safety-critical systems development. The process must include the identification and monitoring of potential hazards.


Key points

• Safety and dependability cases collect all of the evidence that demonstrates a system is safe and dependable. Safety cases are required when an external regulator must certify the system before it is used.

• Safety cases are usually based on structured arguments. Structured safety arguments show that an identified hazardous condition can never occur by considering all program paths that lead to an unsafe condition, and showing that the condition cannot hold.