Three Examples on the Real-Life Applications of Model Checking · 1)M. Webster et al. Toward...

Matemaattis-luonnontieteellinen tiedekunta

Three Examples on the Real-Life Applications of

Model CheckingJohn Lång

3 December 2019

2 / 41

Where Can I Use MC?

● During the past four decades, MC has been utilised as a verification and validation method in many diverse domains– e.g. nuclear power plants, financial software, aerospace

domain, dam control systems, cyber-physical systems, particle collider experiment control systems, telecomms protocols and libraries, computer hardware design, graph theory, medicine, arms industry, etc., etc.

3 / 41

The Scope of My Report

● I considered and read 16+ case studies on MC● I selected three papers for my work based on:

– How recent those articles were– Their mutual diversity of topics and techniques– Pedagogical considerations

4 / 41

Selected Papers

● The papers (see references on the last slide) discussed the verification of1) An autonomous robotic assistant (Webster et al.)

2) Satellite onboard software (Gan et al.)

3) The ARP protocol (Bruschi et al.)

5 / 41

Case 1: The Robot Study

● Webster et al. verified an autonomous robotic assistant, Care-O-bot, manufactured by Frauenhofer IPA, and deployed at the University of Hertfordshire’s Robotic House

● The study is part of the Trustworthy Robotic Assistants research project (www.robosafe.org)

6 / 41

The Robot House

● Two-storey, three-bedroom suburban house in Hatfield, Hertfordshire, England

● Equipped with 50+ real-time sensors– E.g. refridgerator door open/closed sensor, toilet flush

sensor, pressure sensors on chairs to detect sitting● Also equipped with the Care-O-bot

7 / 41

The Care-O bot

● Good Old-Fashioned (i.e. rule-based) AI● Commercial product (a “robot butler”) of

Frauenhofer IPA● An arm with three fingers and a tray● Stereo sensors serving as eyes● Text-to-speech synthesizer and a LED light

8 / 41

The Robot OS

● Care-O-bot’s software is based on the Robot Operating System (ROS) and ROS packages

● The ROS Script Server mechanism is used for high-level commands– e.g. “say hello”, “raise tray”, “move to location x”

● Decision rules (31) and sensor data in SQL DB

9 / 41

The Modeling Languages

● The authors used an intelligent agent modeling language and simulation environment Brahms

● The Brahms code was used to generate a model in the PROMELA modeling language and model checked with the Spin model checker– Spin is an enumerative model checker

10 / 41

IF-THEN Notation

● The authors defined an IF-THEN syntax for representing the decision rules. For instance:IF tray_is_raised AND tray_is_emptyTHEN set_light(yellow) move_tray(lowered_position) wait() set_light(white) wait() set(tray_is_raised, false) set(tray_is_lowered, true)

11 / 41

Brahms Workframes

● ‘A key concept in Brahms is the “workframe,” which specifies a sequence of things to be done when a given condition holds.’ (p. 188)

● The first step in the modeling process was to convert the IF-THEN rules into (more verbose) Brahms workframes

12 / 41

Brahms to PROMELA

● Brahms has a belief system, unlike PROMELA● However, the authors assumed perfect

knowledge and complete sensor accuracy● The authors used an already existing

translation tool, BrahmsToPromela – Beliefs of agents convert into arrays of booleans

13 / 41

The Properties

● The authors investigated four LTL properties of the form (□ P→◇Q)– e.g. (□ BC-O-b(BPersonguiGoToKitchen)

→◇BC-O-b(location = Kitchen))– BC-O-b and BPerson are the belief arrays of the agents

Care-O-bot and the human user

14 / 41

The Models

● Four models of the Robot House:1)Non-deterministic human inhabitant, six choices

2)Deterministic simulation replicating actual events

3)Non-deterministic, based on actual events, 26 choices

4)Non-deterministic with 133 “conjoined” activities

15 / 41

The First Model

● The first model considers a time interval from 12 P.M. to 9 P.M.

● The person may choose freely one action from six options, once per hour during this period

● This model is rather weak

16 / 41

The Second Model

● The second model was based on data that was logged when a person lived in the Robot House for about four and half days

● A custom Java application translated the log entries into Brahms workframes for modeling

● Due to technical limitations of PROMELA, the length of the modeled period was only 21 hours

17 / 41

The Third Model

● The authors improved the precision of the environment in their third model

● In this non-deterministic model, there were in total 26 activities for the person agent

● It took about three times longer to verify the LTL properties than with the first model

18 / 41

The Fourth Model

● The previous non-deterministic models failed to account for the fact that some user actions are “conjoined”– e.g. the event in_bathroom needs to be active before,

during, and after the event toiletting

● These dependencies significantly increased the state space size of the fourth model

19 / 41

Results of the Robot Study

● All of the four models satisfied the four LTL properties. They were reasonably small

● These results give idea on how increasing non-determinism increases state space size

● The study demonstrates how models can be built incrementally and with automated tools

20 / 41

Case 2: The Satellite Study

● Gan et al. used the symbolic model checker NuSMV to verify properties of an satellite Attitude and Orbit Control System (AOCS) written in Ada by Space Systems Finland

21 / 41

AOCS

● The task of AOCS is to maintain stable orbit● Four software managers that are executed one

after each other in periodical sequence● Six operational modes● Seven hardware units

22 / 41

The Model

● The AOCS implementation was serial and non-recursive, and it didn’t use all features of Ada– This simplified model construction in NuSMV– NuSMV uses a symbolic ROBDD-based approach

● The authors wrote a model with a realistic level of detail by hand

23 / 41

The Main Focus

● Of the seven units in the system, the authors focused on the Payload Instrument (PLI), which is responsible for collecting science data

● Other units were abstracted as booleans indicating the presence/absence of an error– Any subset of these six units may fail at any time

24 / 41

Mode Transitions

● Not all of the mode transitions in the AOCS software were atomic

● Steps of execution were recorded with integer timers in the Ada code

● Naïve modeling of the timers would likely have resulted in state space explosion

25 / 41

Representing Timers

● The authors employed data abstraction● Instead of representing counters as integers,

sets of relevant values are enumerated● Abstraction introduced spurious counter

examples, which were suppressed with LTL fairness constraints

26 / 41

Unfair Executions

● The authors gave this example of a spurious counter-example: The timer may loop in the state 0..99 forever and always pick inc

0..99 100 101..

incinc,rst inc inc

inc

rst

rst

27 / 41

Properties

● The authors used the state machine diagrams of the original AOCS design document to generate an LTL specification

● This was done with a Python script● The properties were also converted to and

verified with CTL

28 / 41

Liveness-to-Safety

● The authors developed a new liveness-to-safety translation mechanism, called acceptance counting for improving verification performance

● If an upper bound for violations of a liveness property can be found, then the property holds

● Only partially decidable

29 / 41

Results of the AOCS Study

● All 28 properties generated from the state machine diagrams were successfully verified

● The acceptance-counting method turned out to outperform a rival method (state recording) and plain NuSMV models

30 / 41

Case 3: The ARP Protocol

● Bruschi et al. investigated the ARP protocol using SMT-based model checking

● https://tools.ietf.org/html/rfc826● https://tools.ietf.org/html/rfc3927● https://tools.ietf.org/html/rfc5227

31 / 41

The ARP Protocol

● The Address Resolution Protocol (ARP) maps IPv4 to machine (e.g. MAC) addresses

● Hosts have their ARP caches● A host broadcasts a request message accross

LAN to get machine addresses of another host● The owner of an IP address sends a response

32 / 41

Aspects of ARP

● ARP is an infinite state system– The number of participating hosts is unbounded– Safety of ARP can be finitarily disproved

● ARP messages are not encrypted or signed● Two announcements force a host to change IP

33 / 41

Man-in-the-Middle

● A Man-in-the-Middle (MitM) attack for ARP means that some host h has its ARP cache entry mapping the IP address a of the victim host v to the MAC address of malicious host x.– i.e. h believes that a belongs to x– x can intercept messages that h sends to v

34 / 41

Denial of Service

● In case of the ARP protocol, the Denial of Service (DoS) attack means that there is some host v that is unable to obtain a stable IP address, because some attacker keeps forcing it to try new addresses– The victim of a DoS attack cannot communicate

35 / 41

Results of the ARP Study

● ARP turns out to be vulnerable against MitM and DoS (not RFC 5227 though) attacks– The authors managed to construct attacks as SMT

counter-examples to safety properties

● MC found feasible for protocol verification● RFC 826 is from 1982. This paper is from 2017

36 / 41

Common Challenges

● Time to market● Total lifetime cost vs. cost of verification● Different culture in industry and academia● Understanding design, specification, limitations● State space explosion

37 / 41

Modeling Techniques

● We saw several modeling techniques in action:– Working on a sublanguage of a programming language– Building models incrementally– Using log data from real life to increase model fidelity– Building models/properties manually/automatically– Fairness constraints exclude spurious counter-examples

38 / 41

State Space Explosion

● We saw some performance improving methods:– Abstraction– Verifying properties one at a time– Converting LTL properties into CTL– Converting liveness properties to safety properties

39 / 41

Conclusions

● Formal verification often brings added value● Useful for both design and implementation● Use cases often restricted to critical systems● Many businesses don’t seem to be aware of FV● I see a lot of potential for monetization

40 / 41

Thank You!

● Questions?● These slides are available at:

https://www.cs.helsinki.fi/u/jllang/applications_of_model_checking-2019-12-03.pdf

41 / 41

References1) M. Webster et al. Toward Reliable Autonomous Robotic Assistants Through

Formal Verification: A Case Study. IEEE Transactions on Human-Machine Systems 46(2):186-196 (2016).

2) Xiang Gan, Jori Dubrovin, Keijo Heljanko. A symbolic model checking approach to verifying satellite onboard software. Science of Computer Programming, 82:44-55 (2014).

3) Bruschi D., Di Pasquale A., Ghilardi S., Lanzi A., Pagani E. Formal Verification of ARP (Address Resolution Protocol) Through SMT-Based Model Checking - A Case Study. In: Polikarpova N., Schneider S. (eds) Integrated Formal Methods. IFM 2017. Lecture Notes in Computer Science, vol 10510. Springer 2017.