Bridging the gap between specification and implementation

11/9/04 1

Bridging the gap between specification and implementation

Insup LeeDepartment of Computer and Information Science

University of Pennsylvania

November 9, 2004

11/9/04 2

The gap between specification and implementation

► Problem– gap between an abstract model and the

implementation– scalability challenge (software size and

complexity)► Approaches

– Software model checking– Model-based code generation– Test generation from specification– Run-time verification/checking

► Model checking– Formal, Complete– Does not scale well– Checks design, not implementation

► Testing– Tests an implementation directly– Informal, Incomplete

Requirements

Designspecification

Implementation

11/9/04 3

Run-time Verification and its Application to Security

► Run-time verification► The MaC framework

– Java-MaC– Steering

► Model-based security checking– Security automata – Model carrying code

► Current work

11/9/04 4

Runtime Verification

Program Verifier

Execution

InformationCheck

Sat / UnsatFeedback

User

11/9/04 5

Run-time verification► Run-time monitoring and checking w.r.t. formal specification► Ensures the runtime compliance of the current execution of a

system with its formal requirement► Steps

1. Specify formal requirements2. Extract information from current executing program3. Check the execution against formal requirements4. Steer the computation to a safe state

► Complementary methodology to formal verification and program testing

– Validate implementation– Not complete: guarantee for current execution– Prevention, avoidance, and detection & recovery

► Joint work with S. Kannan, M. Kim, U. Sammapun, O. Sokolsky, M. Viswanathan

11/9/04 6

The MaC Framework

ProgramProgram

Static Phase

Run-time Phase

low-levelbehavior

high-levelbehaviorProgramProgram Filter

Automatic Instrumentation

Human

Monitoring ScriptsMonitoring ScriptsLow-level

SpecificationHigh-level

Specification

Event Recognizer

Event Recognizer

AutomaticTranslation

Run-time CheckerRun-time Checker

AutomaticTranslation

Input

Informal Requirement

Spec

[Kim et al, ECRTS 99]

11/9/04 7

Design of the MaC Languages

► Must be able to reason about both time instants and information that holds for a duration of time in a program execution.– Events and conditions are a natural division, which is also found in

other formalisms such as SCR. – Conditions, which are true or false for a finite duration of time (e.g.,

is variable x >5?)– Events, which are either present or absent at some instant of time

(e.g., is the control right now at the end of method f?).► Need temporal operators combining events and conditions in order to

reason about traces.

start(position==100) end(position==100)

1:00:10 1:00:301:00:15

raiseGate

Time

position == 100

11/9/04 8

Logical Foundation

► Conditions interpreted over 3 values: true, false and undefined.

► [., .) pairs a couple of events to define an interval.► start and end define the events corresponding

to the instant when conditions change their value.

212121 | | | ),[ | )(defined:: CCCCCEEC c |C

CE

EEEECCeE

when

| | | )(end | )(start | :: 2121

[Lee et al, ICPDP 99]

e1 e2

[e1 ,e2) [e1 ,e2)

11/9/04 9

Meta Event Definition Language (MEDL)

ReqSpec <spec_name>

/* Import section */ import event <e>; import condition <c>;

/*Auxiliary variable */ var int <aux_v>;

/*Event and condition */ event <e> = ...; condition <c>= ...;

/*Property and violation */ property <c> = ...; alarm <e> = ...;

/*Auxiliary variable update*/ <e> -> { <aux_v'> := ... ; }End

► Expresses requirements using the events and conditions, sent by event recognizer.

► Expresses the subset of safety properties.

► Describes the safety requirements of a system, in terms of conditions that must always be true, and alarms (events) that must never be raised.

– property safeRRC = IC -> GD;

– alarm violation = start (!safeRRC);

► Auxilliary variables may be used to store history.

– endIC-> { num_train_pass’ = num_train_pass + 1; }

11/9/04 10

The MaC languages

Run-time state:•method call•object state•local variables

Abstract state:•events•conditions

MEDL

PEDL

SADL

► PEDL: abstraction► MEDL: abstract transformation► SADL: feedback

11/9/04 11

PEDL (Primitive Event Definition Language)

► Primitive Event Definition Language (PEDL)– Maps the low-level state information of the system to high-level events.

– Depends on target program implementation

– Design Issues

• What can be observed?

• Passive and active probes

• Software probe vs. hardware probe

• Programming language-based vs. systems API-based

► Java-PEDL– Provides primitives to refer to values of variables and to certain points in

the execution of the program.

– PEDL is defined so that events can be recognized in time linear to the size of the PEDL specification

[Kim et al, FMSD 04]

11/9/04 12

Java-MaC Static Phase

Java Bytecode

InstrumentedBytecode

MaC Compilers

PEDL MEDL

MaC Specifications

EventRecognizer

Checker

MaC Verifiers

SADL

Steerer

11/9/04 13

Instrumented Java Program MaC Verifier

Execution

Information

Sat / UnsatFeedback

User

EventRecognizer

(PEDL)

Steerer

Checker(MEDL)

Java-MaC Dynamic Phase

11/9/04 14

MaC Language - PEDL

Java Program PEDL

Abstraction

- When train position is between 30 and 50

- When gate starts/ends being down

export event startGD, endGD; export condition cross;

// specify what to monitor monobj Train.position; monmeth Gate.up(); monmeth Gate.down();

// specify abstraction condition cross = (30 < Train.position) && (Train.position < 50); event startGD = endM(Gate.down()); event endGD = startM(Gate.up());

Railroad Crossing Property: - If train is crossing, then gate must be down- Train is crossing when position is between 30 and 50

position = 0

position = 20

position = 40

Gate.down()

position = 55

Gate.up()

position = 60

startGD

endGD

cros

s

cross = true

cross = false

11/9/04 15

MaC Language - MEDL

PEDL MEDL

import event startGD, endGD; import conditions cross;

// specify abstraction condition gateDown = [startGD, endGD);

// specify property property safeRRC = cross -> gateDown;

Violation

gateD

ow

n

cros

s

Abstraction

- When gate is down

Property

- If train is crossing, then gate must be down


startGD

endGD

cros

s

cross = true

cross = false

11/9/04 16

Instrumentation

class Train { int position; main() { position = 0; position = 20; position = 40; position = 55;} }

monobj Train.position;

class Train { int position; main() { position = 0; send(x,0); position = 20; send(x,20); position = 40; send(x,40); position = 55; send(x,55);} }

+=

Sent to Event Recognizer:[ (position,0), (position,20), (position,40), (position,55) ]

11/9/04 17

MaC Language - Components

Java Program PEDL MEDL

Abstraction

- When train position is between 30 and 50

- When gate starts/ends being down


Abstraction

- When gate is down

Property

- If train is crossing, then gate must be down

position = 0

position = 20

position = 40

Gate.down()

position = 55

Gate.up()

position = 60

Violation

gateD

ow

n

cros

s

startGD

endGDcro

ss

cross = true

cross = false

11/9/04 18

Evaluation Overview► Static Phase: Each property is represented as a tree

– The most basic events/conditions/aux vars are at the leaf level• For PEDL, variable updates, start/end method events are leaves• For MEDL, imported events from PEDL and auxiliary variables are

leaves– Composition of events and conditions link to events/conditions

that are composed of– PEDL and MEDL are forests

► Dynamic Phase: In both Event Recognizer (ER) and Checker, evaluation starts at the leaves and traverses up to the root– Evaluation starts only at leaves representing occurred events or

changed conditions– Otherwise, no evaluation is done– Efficient

11/9/04 19

PEDL Graph

export event startGD, endGD;

export condition cross;

monobj Train.position;

monmeth Gate.up();

monmeth Gate.down();

condition cross = (30 < Train.position) && (Train.position < 50);

event startGD = endM(Gate.down());

event endGD = startM(Gate.up());

Train.position Gate.up()Gate.down()

30 < position < 50 endM(Gate.down()) startM(Gate.up())

cross startGD endGD

11/9/04 20

MEDL Graph

import event startGD, endGD; import conditions cross;

condition gateDown = [startGD, endGD); property safeRRC = cross -> gateDown;

cross

cross -> gateDown

endGDstartGD

[startGD, endGD)

gateDown

safeRRC

11/9/04 21

Algorithm► Static Phase:

– Create PEDL and MEDL graphs– Assign a height to each node in both graphs

► Dynamic Phase:– Maintain an evaluation list sorted by height– Add all occurred primitive events and changed conditions to the evaluation list at

height 0– For each event/condition in the evaluation list,

• Call evaluate() method• Add its parent in the evaluation list (if not already in)• Repeat until the list if empty

– Finishing• ER sends occurred events/changed condition to checker for each exported

events/condition • Checker notifies user for each event in the alarm list.

► Complexity– The size of the PEDL or MEDL graph is linear in the size of the formula.– Evaluation of a MEDL or PEDL formula on a single observation is linear in the

size of the graph.

11/9/04 22

Instrumented Java Program MaC Verifier

Execution

Information

Sat / UnsatFeedback

User

EventRecognizer

(PEDL)

Steerer(SADL)

Checker(MEDL)

Java-MaC Dynamic Phase

11/9/04 23

Steering process

system

checker

violation

action invoked

action executed

detection

steeringconditionsatisfied

actioninvocationreceived

event received

11/9/04 24

Simplex Architecture

Safety

Experimental

DecisionModule

PhysicalSystem

us

ue

xu

SC

EXx0

Equilibriumstate

► Experimental controllers provide improved performance but uncertain stability properties– Can be dynamically added or replaced

► Safety controller has the largest stability region

[L. Sha]

11/9/04 25

Inverted Pendulum (IP) Example

DeviceDrivers

angle,track

volts

Decision Module

Switching logic

Written in C

SafetyController

ExperimentalControllerExperimental

ControllerExperimentalController

[L. Sha]

m

l

x

g

Muf

11/9/04 26

Steering Action Definition Language

► SADL script– identifies object used in steering

// the target of steering is the object dm of type DecisionModule

// located in the class IP

DecisionModule IP:dm;

– defines steering actions// setSC() method of dm is invoked

steering action change2SC = { call (IP:dm).setSC(); }

– specifies steering location• locations in the code where the actions can be executed

before read DecisionModule:volts;

11/9/04 27

IP and MaC

DeviceDrivers

angle,track

volts

Decision Module

Written in Java

SafetyController

ExperimentalControllerExperimental

ControllerExperimentalController

JNI

monitor

steer

MaC:Switching

logic

[Kim et al, RV 02]

11/9/04 28

Applications

► Where can we get specifications?– During the design specification and analysis phase, many properties may

be identified and verified. • Reuse properties model checked during the design phase• E.g., more than 1000 properties during designing of flight control systems

– Extract from (informal) requirements and specification documents– Security Policy– Extract from the target program

• To ensure that the program has not been tampered• Model Carrying Code

► Other application areas– Network routing simulation– Hardware design– Adaptable sensor network systems– Cheat detection in distributed game

► Annual run-time verification workshop (01, 02, 03, 04, …)

11/9/04 29

Monitoring and Checking for Security Properties

11/9/04 30

Similar techniques, different purposes

► Check security policy– Security automata, edit automata – Model-Carrying Code (MCC)

► Intrusion detection – Extract from the target program to ensure that the program has

not been tampered– Signature-based approach

11/9/04 31

Security Policy in Security/Edit Automata

Example (modified from [BLW02]): Limit the amount of memory that an application can allocate for itself

Property: application must not allocate memory more than n

a = (malloc(q), q’=q’+q, q’< n)

a

a a = (malloc(q), q’=q’+q, q’ >= n)

halt

a = (malloc(q), q’=q’+q, q’< n)

11/9/04 32

Must not allocate more than n

export event mallocCall;monmeth int malloc(int);event mallocCall = startM(malloc(int));

PEDL

import event mallocCall;import action halt;var int memory;alarm violateMemoryPolicy = end(memory < 1000);mallocCall -> { // value(mallocCall,0) returns arg of malloc() memory’ = memory + value(mallocCall,0);}violateMemoryPolicy -> { invoke(halt); }

MEDL

steering action halt = // exit before next malloc() call { call System.exit(); } before call malloc(int);

SADL

11/9/04 33

Model-Carrying Code (MCC)

► How can we run untrusted code on our machine?– Untrusted code comes with a model of its security-relevant

behavior– Users have their own security policies– Employ two types of checking

► Static checking: to ensure that untrusted program’s model respects user’s security policy– Use model checking to check that Beh(Model) are in

Beh(Policy)► Run-time checking: to ensure that program behaves as

specified by model – Use runtime checking with

• Model is a specification (Automata)• Events are system calls

11/9/04 34

MCC Framework

[SVB+03]

11/9/04 35

Current Work on MaC

► Java-MaC available for download– www.cis.upenn.edu/~rtg/mac

► Minimum trace of run-time verification► MEDL-RE: MEDL with regular expressions► MaC with probability► Hierarchical IDS► Using MaC to detect failed/malicious sensor

nodes

11/9/04 36

Minimal Trace for run-time verification

► Large amounts of monitored data hampers communication between the system and the checker– How to extract minimum information necessary to check a given

property?► Redundant information in traces

– alarm dropReq = fault when respondingcondition responding = [acceptReq,response)event acceptReq = request when ready==trueevent fault = start(abort == true)

– Trace with redundant data: request ready(t) abort(t) request abort(f) request abort(t)

– Trace without redundant data: ready(t) request abort(t)

► Approach: define minimally adequate trace, and MEDL automata

11/9/04 37

MEDL-RE: MaC with regular expression

► Regular expression over events– Statement: RE R {Ē} = < R >, – Grammar of R: R ::= e | R.R | R+R | R*– Alphabet of R contains events used in R and events in its relevant set {Ē}

► Regular expressions are neither events nor conditions and cannot be used alone– Events associated with RE R:

startRE(R), success(R), fail(R) ► Example

– Three components of a media must start in the following order: video, caption, audio

– RE media {} = < startVideo . startCaption . startAudio >– alarm notOrdered = fail(media)

► Challenges– (Possibly infinite) multiple instances of (possibly overlapping) regular

expressions

11/9/04 38

► Definition of Probability– P(E) = |E| / |S| Ex. Sample Space S = {H,T} and Event E = {H}, P(E) = 0.5

– Long run frequency: P(E) = lim n(E) n n• where n = the number of experiments performed, n(E) = the number of outcomes

belonging to the event E• Ex. experiment is performing coin flipping, P(H) is the number of times the

outcome is H in relative to the number of times the coin flipping is performed

► Challenges– How to detect experiments from execution trace– Accuracy of probability calculated from execution trace

► Possible approach– Detecting experiment using events or regular expressions– Calculate probability from execution trace with confidence interval– Possible syntax: E ~ [ p, Exp] and C ~ [ p, Exp] where = < | > | <= | >=, p is probability, Exp is an experiment

MaC with probability

11/9/04 39

Extend MaC Architecture for IDS

11/9/04 40

Thank You!

Documents

Bridging the gap between specification and implementation