Data-based Metrological Support Services

Data-based Metrological

Support Services

Progress Report on Work Package 4

Dr.-Ing. Marko Esche

▪ Objectives of Work Package 4

▪ Exemplary support service: risk assessment

▪ Report on task 1: Inter-institutional comparison

▪ Report on task 2: Closing the risk assessment looü

▪ Summary

▪ Further work

2019-05-28 2 Data-based Metrological Support Services

Overview

▪ Objectives

▪ Use data volumes created by measuring instruments

employed in the EU single market

▪ Based on connected databases, develop new data-driven

metrological services such as methods for closing the

risk assessment loop.

▪ Exemplary support service: software risk assessment


WP 4: Data-based Metrological Support Services

Connect separate databases

4

NotifiedBody

Market Surveillance

Manufacturer User

ServiceData

Within individual “Data Shells” Within joint “Data Shells”

AdministrativeData Measurement

DataMI

ICSMS AUX

=> Trustworthy information exchange

2019-05-28 Data-based Metrological Support Services

Exemplary support service

▪ MID (Directive 2014/32/EU) requires an “analysis and

assessment of the risks” to be part of the documentation

submitted for conformity assessment.

▪ Within the Metrology Cloud, a framework for risk

assessment is investigated which has been accepted by

WELMEC WG 7 and can be used by manufacturers and

Notified Bodies.

▪ The structure of ISO/IEC 27005 is used for the analysis.

▪ Methods from ISO/IEC 15408 and 18045 are employed to

provide reproducible numerical risk scores.



Aspects of Directive 2014/32/EU

Manufacturer / NB: Risk assessment => What can go wrong?

MSA: Risk assessment => What has gone wrong?

Knowledge bases:

National, e.g. Germany: SAM

EU: ICSMS

Supports RA and

definition of threats

Placing on the market

Method description

▪ ISO/IEC 27005: “Risk is a combination of the consequences that would follow from the occurrence of an unwanted event and the likelihood of the occurrence of the event.”

▪ Needed components:

▪ threats to assets

▪ impact/hazard/consequence

▪ probability/likelihood


RiskIdentification

Risk Estimation Risk Evaluation

Risk Assessment

Software requirements in the MID

Assets derived from the MID

Number Asset Security Property

A1 metrological software integrity, authenticity

A2 evidence of an intervention availability, integrity

A3 measurement data integrity, authenticity

A4 metrological parameters integrity

A5inadmissible influence on thesoftware unavailability

A6 indication of the result availability, integrity


Risk assessment procedure (ISO/IEC 27005)

Identification of Assetsand

Security Properties

IdentificationOf

Attack Vectors

CalculatingProbability of Occurence

AndRisk Score

Legal Requirements Documentation

Market Surveillance

Public Databases

(ENISA, CVE/MITRE, etc.)

Expert Knowledge

Public Databases

(ENISA, CVE/MITRE, etc.)


Risk assessment procedureCalculation of Attack Probability

(ISO 18045 Part 2, B.4.2.2 ff)

Attacker Model(ISO 15408 SPD)

Assets(ISO 15408 SPD) Adverse Actions

(ISO 15408 SPD)

Primary AssetsIntended Use

(ISO 15408 SPD)Source: MID

Secondary Assets(ISO 15408 SPD)

Source: Manufacturer, NBs

Filter: intended use vs. misuse

Impact(WELMEC Guide 5.3)Source: NBs, market

surveillance, manufacturer

Threat Definition(ISO 15408 Part 1 SPD)

“Attacker X executes adverse action Y on asset Z”

Definition + Impact

Attack VectorsSource: NBs, market surveillance (SAM,

ICSMS), manufacturer

Implemented AttackDefinition + Impact+

Implementation

Vulnerability Analysis1) Elapsed Time (1-19 points)2) Expertise (0-8 points)3) Knowledge of the TOE (0-11 points)4) Window of Opportunity (0-10 points)5) Equipment (0-9 points)

TOE resistance

Value Resistance Probability Score

0-9 No rating 5

10-13 Basic 4

14-19 Enhanced Basic 3

20-24 Moderate 2

>24 High 1

S

Risk Evaluation

Calculation of the Risk associated with an attack

risk = impact*probability


M. Esche and F. Thiel, “Software risk assessment for measuring instruments in legal metrology”, FedCSIS 2015

▪ No link between probability of occurrence and incident

information from the field

▪ Accuracy depends on the assessor’s skill and knowledge

about the measuring instrument.

▪ No objective way to quantify attacker motivation in current

method

▪ Parties involved consider risk to be a theoretical concept.

▪ Data available within the Metrology Cloud may help to solve

these issues.


Challenges

▪ Task 1: Inter-institutional comparison

▪ Test and evaluate existing risk assessment method.

▪ Different market actors (notified bodies, manufacturers market surveillance) assess generic instruments to investigate objectiveness of the method.

▪ Goal: Suggestions for the improvement of the method

▪ Task 2: Closing the risk assessment loop

▪ Investigate strategies for the inclusion of incident data.

▪ Investigate data sources made available via the Metrology Cloud.

▪ Develop and test a concept for incident data inclusion.


Tasks of Work Package 4


Task 1: Inter-institutional comparison

NotifiedBody A

NotifiedBody B

NotifiedBody C

MI

Manufacturers’Associations

provide

assess

Result A

Result B

Result C

Evaluation

+

improved method

▪ March 2018: Joint training session with WELMEC WG7

Subgroup Risk Assessment.

▪ June – August 2018: Proposal and development of an

abstract measuring instrument by CECIP as a reference

point.

▪ September 2018 – January 2019: Assessment of the

abstract measuring instrument by five notified bodies

▪ February 2019: Collection and evaluation of results, proposal

of a formalized risk assessment template

▪ Threat 1: introduction of false measurement results

▪ Threat 2: modification or replacement of software


Report on task 1


Cloud-based measuring instrument

▪ two categories of display devices (full control, receive only)

▪ communication between separate components via Wi-Fi with

WPA encryption

▪ Cloud offers data storage and display server (DSP).

Results for the cloud-based instrument


0

5

10

15

20

25

30

35

Impact Elapsed

Time

Expertise Knowledge

of the TOE

Window of

Opportunity

Equipment Sum Probabilty

Score

Risk

NB1 NB2 NB3 NB4 NB5

0

5

10

15

20

25

30

35

Impact Elapsed

Time

Expertise Knowledge

of the TOE

Window of

Opportunity

Equipment Sum Probability

Score

Risk

NB1 NB2 NB3 NB5

Threat 1: introduction of false measurement results

Threat 2: modification or replacement of software

Simple weighbridge


▪ Sealed communication path from load cell to terminal

▪ Evaluator units and terminal are based on the same microprocessor.

▪ Data can be read from the terminal via RS 485 or can be written to a USB stick.

▪ Terminal checks the authenticity of all other units at startup.

▪ Flash memory for parameters and software is protected by a hardware switch.

▪ Legally relevant log is stored on an SD-card.

Results for the simple weighbridge


0

5

10

15

20

25

30

Impact Elapsed

Time

Expertise Knowledge

of the TOE

Window of

Opportunity


Score

Risk

NB1 NB3 NB4 NB5

0

5

10

15

20

25

30

35

Impact Elapsed

Time

Expertise Knowledge

of the TOE

Window of

Opportunity


Score

Risk

NB1 NB3 NB5

Threat 1: introduction of false measurement results

Threat 2: modification or replacement of software

▪ Objective comparison of risk assessment results for software

is only possible if certain prerequisites are fulfilled.

▪ Instructions for new evaluators on how to perform the risk

assessment according to ISO 18045 shall be readily

available.

▪ Examples for evaluation of common attack vectors to

reduce the workload for evaluators shall be supplied.

▪ Proper documentation of the complete attack vector and

justification for the evaluation shall be required of all

assessors to allow for better comparability of assessment

results.


Task 1: observations

Solution: risk assessment template


▪ June – December 2018: Investigation of inclusion strategies

for incident data in software risk assessment

▪ February 2019: Presentation of proposals by PTB and

CECIP

▪ Conclusions:

▪ Risk is an inherently theoretical concept.

▪ Incident data is not available prior to putting the instrument

on the market.

▪ Incident data can be used to calculate risk scores during a

second assessment round.

▪ Attacker motivation should be reflected in the assessment.


Report on task 2

▪ Concerning the incident data:

▪ Data available via the cloud would need to be very

specific.

▪ WP4 will need to make it clear to market surveillance/other

WPs, which kind of data is needed for risk assessment.

▪ PTB will update the proposal to close the loop.

▪ CECIP is currently developing an extended proposal to

determine attacker motivation.

▪ The motivation (score) shall then be incorporated into the

risk assessment result as well.


Report on task 2

Task 2: target workflow


requirements

declaration of conformity + risk assessment

observed attacks &irregularities

common attackvectors

attack vectors

Market Surveillance

Notified BodyManufacturer

IncidentData

AdministrativeData

riskassessment

time, expertise,…

▪ Task 1 has progressed to the stage of publishing a first

template for risk assessment formalization.

▪ The template will now be tested by the partners and modified

where necessary.

▪ Task 2 will produce draft concept within the next two months.

▪ WP1 and market surveillance will be contacted once a

precise definition of the needed data has been formulated.


Summary

▪ These results will be presented to WELMEC WG7 in September.

▪ WG7 will decide how to include the results in its work program.

▪ The applicability of a simplified risk assessment method for components/modules of a measuring instrument should be investigated as well.

▪ Intended outcome:

▪ simplify the method

▪ make results more easily reusable

▪ pave the way towards risk-based evaluation of measuring instruments


Further work

Physikalisch-Technische Bundesanstalt

Braunschweig and Berlin

Abbestraße 2-12

10587 Berlin

Dr.-Ing. Marko Esche

Telefon:+49 30 3481-7975

E-Mail: [email protected]

Version: 05/19

http://www.ptb.de/

Documents

Data-based Metrological Support Services