Case Study Experiences from the DIAMONDS Project · 2013-01-17 · Case Study Experiences from the DIAMONDS Project 8th ETSI Security Conference 16. -17. January, ... or process monitor

Case Study Experiences from the DIAMONDS Project8th ETSI Security Conference

16. -17. January, Sophia Antipolis - France

© DIAMONDS Consortium 2010-2013

Ina Schieferdeckerwww.itea2-DIAMONDS.org

DIAMONDS ProjectIn six countries

Project Duration: October 2010 – March 2013

Project Partner:• Large companies (6)• Small companies (9)• Universities (3)


• Universities (3)• Research institutes (4)

Introduction & RelevanceEffective and Efficient Security Testing

DIAMONDS will enable efficient and automated security testing methods of industrial relevance for highly secure systems in multiple domains.

Objectives:• Security test methodologies and test patterns • Automatic monitoring techniques• Open source platform for security test tool integration


• Open source platform for security test tool integration

Business Impact:• 6 different industrial domains• Pre-standardization work• Novel integration of testing, security analysis and risk orientation

Innovation & Expected ResultsCombination of Approaches


Security Testing

Model-based Testing

DIAMONDS Innovation ExpectedResults Achieved

• Advanced model-based security testing methods: Model-

based fuzz testing

• Autonomous testing techniques based on automatic

monitoring techniques: Passive symbolic monitoring,

Integration of monitoring and model-based test


Integration of monitoring and model-based test

generation

• Pre-standardization work on multi-domain security test

methodologies and test patterns: Risk-based security

testing methodology, security test pattern

• Open source platform for security test tool: Traceability

platform for security testing, Malware reverse engineering

and its application to testing

Security testing solutions for six industrial domains

• Banking

• Automotive

Case StudiesSix Industrial Domains


• Automotive

• Radio protocols

• Smart cards

• Telecommunication

• Industrial automation

DIAMONDS Results So FarTechniques and their Application

• Risk-Based Testing (Banking, Automotive):

– Risk-based test identification & risk-based test selection (FOKUS, SINTEF)

• Advanced Fuzz Testing (Banking, Radio Protocols, Automotive, Telecommunication):

– Model-based behavioural fuzzing (FOKUS)


– Model-based behavioural fuzzing (FOKUS)

– Model inference assisted smart fuzzing (INPG)

• Active Testing Techniques (Banking, Radio Protocols)

– Model-based security testing from behavioral models and test purposes (SMARTESTING)

– Active Intrusion Testing (FSCOM)

• Passive testing techniques (Radio Protocols, Industrial Automation):

– Events-based passive testing (monitoring) (MONTIMAGE)

– Anomaly detection with Machine Learning (INPG)

Banknote processing machine that counts, sorts and assess banknotes by

their currency, denomination, condition and

G&D Banking Case StudyCase Study Characterization


external peripherals

external peripherals

CP CP CP

RS RS VMS

CCCC

CC / GW

LANWAN

CP = Currency Processor

RS = Reconciliation Station

CC = Control Center

VMS = Vault Management System

Firewall

denomination, condition and authenticity

� Security challenges• Restricted access to functions: The access to functions is restricted to authorized users.

• Operation system access restriction: The access to the operation system, i.e. file system, or process monitor is restricted to authorized users.

G&D Banking Case StudyCase Study Characterization


users.• Prevent Admin Hijacking: Hijacking an administrator account is used to get the privileges of an administrator account as a user that is not assigned to the administrator group.

• Prevent infiltration/manipulation of software: Software manipulation can be used to fake data or to provoke errors on the currency processor application.

• Prevent manipulation of application configuration: Manipulation could possibly change the classification of banknotes.

G&D Banking Case StudyApproach: Risk-based Security Testing

Risk Analysis Security Test Test

Test Code Test

CORAS Risk AnalysisDeliverable D1.WP2*

Behavioural FuzzingDeliverable D2.WP2* (see also next slide), D3.WP2*

Data Fuzzing with TTCN-3Deliverable D3.WP3*


*project deliverables are available atwww.itea2-DIAMONDS.org “publications”

Risk Analysis

(CORAS)

Security Test Pattern

Identification

Test Generation

Test Code Generation(TTCN-3)

Test Execution

Pattern name Usage of Unusual Behavior Sequences

Context Test pattern kind: BehaviorTesting Approach(es): Prevention

Problem/Goal Security of information systems is ensured in manycases by a strict and clear definition of whatconstitutes valid behavior sequences from the securityperspective on those systems. For example…

Solution Test procedure template:

1. …

2. …

Known uses Model-based behavioural fuzzing of sequencediagrams is an application of this pattern

Security Test Pattern CatalogueDeliverable D3.WP4.T1*

G&D Banking Case StudyApproach Behavioural Fuzz Testing

• Test cases are generated by fuzzing one or more valid sequences.

• This concrete fuzzing of behaviour is realized by changing the order and appearance of messages in two ways:

– By rearranging messages directly. This enables straight-lined sequences to be fuzzed.Fuzzing operators are for example remove, move or repeat a message.

– By utilising control structures of UML 2.x sequence diagrams, such as combined fragments, guards, constraints and invariants. This allows more sophisticated behavioural


TC SUT

1: logon("OP1")

2: selectDenomination(…)

valid sequence

Remove Message1: logon

BehaviouralFuzzing

Fuzzer Apache

1: selectDenomination(…)

invalid sequence

fragments, guards, constraints and invariants. This allows more sophisticated behavioural fuzzing that avoids less efficient random fuzzing.

• By applying one ore more fuzzing operators to a valid sequence, invalid sequences (= behavioural fuzzing test cases) are generated.

G&D Banking Case StudyResults

� Focus on risks related to� unauthorized access� machine/configuration modification

� Until now, no weaknesses were found� confidence in the security of the system is strengthened


� confidence in the security of the system is strengthened

� Metrics� different security levels depending on the covered risks/vulnerabilities by • number of test cases (one or more) per risk/vulnerabilityunauthorized access, configuration modification: more

• number of test methods to generate these test casesdata fuzzing and behavioural fuzzing: 2 test methods

� CORAS method for risk analysis has been proven to be of value� graphical modelling� specification of assets to be protected

� Saved resources due to

G&D Banking Case StudyExploitation


� Saved resources due to� reuse of functional test cases and� reuse of test execution environment for non-functional security testing

� integration of data fuzzing in the TTCN-3 execution environment• keeps the behavioural model clean and concise• allows easy combination of data and behavioural fuzzing

� Standardization of DIAMONDS results provides certification options for products with security requirements

� Bluetooth connectivity module for mobile devices that allows direct communication between car’s head unit and a mobile phone

Automotive Case StudyCase Study Characterization

� Security challenges:


� Security challenges:� Access to the car’s infrastructure by malfunctioning or hostile mobile phones or by misuse of the Bluetooth interface

� Modification of the Bluetooth module in order to interfere with the car’s normal operation and its security and safety

� Technical challenges:� Simulation of Bluetooth device/mobile phone and integration of CAN bus

� specialized Bluetooth stack for security testing

Security

Automotive Case StudyApproach: Risk-based Security Testing


Security Risk

Analysis

System Model

Test Model

Functional test cases

Security Test Case Templates

Fuzzing techniques

� Fuzzing Library developed by Fraunhofer FOKUS� Library is called by FuzzingContainer to inject fuzzed test data

� Improved fuzzing heuristics based on Peach and Sulley

� Interface uses XML for requests and generated fuzz test data

Automotive Case StudyApproach: Data Fuzzing


fuzz test data� Example: Device name and PIN was fuzzed within this case study

� Generators:

� So far, about 150 test cases have been executed

� Test purposes� break Bluetooth connectivity module� compromise the head unit by anomalous Bluetooth messages

Automotive Case StudyResults


� Until now, a few anomalies were found� need further investigation

� Metrics� several vulnerabilities resulted from risk analysis were covered� further metrics have to be found

DEMONSTRATOR DESCRIPTION

� OMNeT++ simulation platform of mobile ad-hoc networks.

� Vulnerability analysis based on over-the-air exchanged PDU at mac and physical layers.

Radio Protocol Case StudyCase Study Characterization

OBJECTIVES

� Security and risk analysis� Formal security flaws identification� Testing tools chain complementarity� …mitigation strategies (on-going)

� Model-based generation of test


generation of test cases (Smartestingand FSCOM) and their execution based on OMNeT++.

� Online analysis of captured traces in order to detect security flaws (Montimage)

Radio Protocol Case StudySecurity Testing Approach


Radio Protocol Case StudyResults

• Integration of the tools in the TCS validation framework, use of standardized API to help on the integration on different validation environment and industrial domains.

• Validation of the framework with the validation of 19 security properties. Implementation of 7 intrusion attacks.


properties. Implementation of 7 intrusion attacks.

• Further work • distributed detection of several attackers at routing layer • genetic testing for static analysis of memory overflow.

Radio Protocol Case StudyExploitation

� DIAMONDS satisfies the requirements of higher securitytesting, in particular on Over The Air threats.

� Evolution of security testing from the critical components to the whole parts of the radio equipment (Hw platform, midleware and radio protocol application).


radio protocol application).

� DIAMONDS is a first response the security testing analysis of theseapplications for which tools and methodologies are lacking.

� Next step might be the integration of Intrusion Detection & PreventionSystem in the radio equipments.

Montimage, CodenomiconProduct update that integrate features that have been developed in DIAMONDS iTrust consultingMalwasm, an open-source tool allowing to monitor a executable during execution by stepping forward and back like in a video and observing and tracing changes to all kind of system parameter.Trick-tester, a Linux distribution allowing Pentesters to have all kind of open source tools

Overall Exploitation Results in DIAMONDSFrom Case Studies to Industry


Trick-tester, a Linux distribution allowing Pentesters to have all kind of open source tools corrected configured with tailored script, and perform efficient pentests. Testing TechnologiesTTworkbench will integrate the TTCN-3 Fuzzing Support SmartestingSecurity test purpose language and a test generation mechanism extend the current Smartesting product with a dedicated feature for model-based security test generation.Giesecke & DevrientAdoption of the Risk Analysis method CORAS for the product development life cycle. …

Standardisation levels:• International: ISO, ITU, …• European: ETSI, ENISA, …• National: NIST, AFNOR, DIN, …• Industrial communities: IEEE, OMG, …

DIAMONDS focusses on ETSI:

DIAMONDS Standardisation WorkStandardisation Bodies


DIAMONDS focusses on ETSI:

• TC MTS: Methods for testing and specification, Model-based testing, Security Special Interest Group;

• TC TISPAN/E2NA: Threat, vulnerability and risk analysis (TVRA)

• TC INT: IMS network testing (concrete test case catalog)

• ISG ISI: Operational Security Indicators measuring IT security policy enforcement & effectiveness (in cooperation with national R2GS Clubs)

DIAMONDS Standardisation WorkStandardisation Approach

System definition & analysis1) TOE, subjects, assets, 2) threats, policies, assumptions3) security objectives4) Security functional requirements

identifying

…System (risk) analysis methods & models:e.g. CORAS, UMLsec

TOE1, TOE2, TOEn


testing

| | | | | |

TCL, JUnit, C++, TTCN-3, manual tests…

Test developer plana) concepts/architectureb) purposesc) Test suite structure� coverage of security

relevant TSFI

enforcingTSFI supporting SFRs

non-interfering

UMLsec

Test tools & techniques:e.g. fuzzing, partitioning

SFRs (specification)

TSFI (realisation)

usecase▼

testcase

Co-summit 2011 and 2012ITEA Exhibition Award


Contact

Fraunhofer Institute forOpen Communication Systems FOKUS

Kaiserin-Augusta-Allee 31 10589 Berlin, Germany

Tel. +49 (30) 34 63 -7000Fax +49 (30) 34 63 -8000

Innovation Center forCost-Effective Systems Qualityhttp://s.fhg.de/sqc

Prof. Dr.-Ing.Ina SchieferdeckerTel. +49 (30) [email protected]


Fax +49 (30) 34 63 -8000

[email protected]

26

Documents

Case Study Experiences from the DIAMONDS Project · 2013-01-17 · Case Study Experiences from the DIAMONDS Project 8th ETSI Security Conference 16. -17. January, ... or process monitor