Reasoning In Discrete Event Systemsusers.cecs.anu.edu.au/~patrik/lss/lss13.pdfReasoning In Discrete...

Reasoning In Discrete Event Systems

P@trik Haslum

Logic Summer School 2013

Reasoning In Discrete Event Systems

1. Introduction & basic concepts.2. Planning, verification & diagnosis.3. State-space exploration.4. Relaxations and heuristics.5. What about logic?6. Infinite-state systems.

Introduction

Example: The Optical Telegraph Protocol

* Communications Protocol:- Message vocabulary.- Rules govern behaviour of

protocol participants.

* Model protocol participants.

* Model system bycomposition.

Introduction

Idle

Take CtlSend Start

Recv Attn

Send Data Recv Data

Send Stop

Recv Stop

Release Ctl

Recv StartTake Ctl

Send Attn

Recv DataSend Data

Recv Stop

Send Stop

Release Ctl

Discrete Event Model of a Telegraph Operator

Introduction

Synchronised Composition

* Discrete event system: S = (Q,L,qI ,T ), where- Q is the set of states;- L is the set of event labels (alphabet);- qI ∈ Q is the initial state; and- T ⊆ Q × L×Q the transition relation.

* S1 × S2 = (Q1 ×Q2,L1 ∪ L2, (qI1,qI2),T ), where- T ((q1,q2), (a,b), (q′1,q

′2)) iff T1(q1,a,q′1), T2(q2,b,q′2)

and aRb;- T ((q1,q2), (a, ), (q′1,q2)) iff T1(q1,a,q′1) and¬∃b ∈ L2 : aRb;

- T ((q1,q2), ( ,b), (q1,q′2)) iff T2(q2,b,q′2) and¬∃a ∈ L1 : aRb; and

- R ⊆ L1 × L2 is the label synchronisation relation.

Introduction

0 1

S1.a

S1.c

2S1.dS1.b

×0 1

S2.a

2

S2.bS2.e

3S2.a

S2.c

=(0, 0) (1, 1)

(a,a)(2, 1)

(d , ·)

(0, 2)

(b,b)(·,e)

(1, 3)(a,a)

(c, c)

(1, 0) (2, 0)(d , ·)

(1, 2)

(·,e)

(2, 2)(d , ·)

(·,e)

(0, 1) (0, 3) (2, 3)

Introduction

24 states

215 states

1911 states

16984 states

150945 states

Planning, Verification & Diagnosis

Planning

“Planning is the art and practice of thinking before acting.”

* Given a world model, and initial world state, and a desiredgoal condition, find a (cheapest, fastest, etc.) course ofactions that transforms the world from the initial state to astate where the goal condition holds.

* The Classical Planning Assumptions:- The (model of the) world is finite, discrete and

deterministic.- The world model complete and correct.- There is no interference.

* In what sitation could such assumptions possibly hold?

Planning, Verification & Diagnosis

Puzzles

Planning, Verification & Diagnosis

(Ruml, Do, Zhou & Fromherz 2011)

(c) MYDATA automation AB

(Helmert & Lasinger 2010)

Industrial Automation

Planning, Verification & Diagnosis

(Hatzack & Nebel 2001)

(Trug, Hoffmann & Nebel 2004)

Airport Ground Traffic Control

Planning, Verification & Diagnosis

S4

G11S6S7

S3

S1

−G84 S5

S2

→ S4

−S7

−S6G11

S3

S1

−G84 S5

S2

→ S4

−S7

−S3−S6

G11

S1

−G84 S5

S2

→ S4

−S7

−S3S6

G11

S1

−G84 S5

S2

(Blanchette et al. 1999)

Genome Edit Distance Computation

Planning, Verification & Diagnosis

The Classical Planning World Model

* States are assignments of truth values to a finite set ofatomic propositions (V ).

* Transitions are instances of actions, characterised by:- Preconditions: Must hold in the state for action to be

applicable.- Effects: Will hold in the state after action application.- Assumption of inertia: Atoms not affected by the action

remain as they were.

* There is a single initial state.

* The set of goal states is a defined by a formula over V .

Planning, Verification & Diagnosis

Example: Sokoban

(empty s22)(empty s23)(empty s32)(empty s33)(box s42) (man s43) (empty s44)(empty s45)(empty s52)(empty s53)(box s54) (empty s55)(empty s62)(empty s63)

Planning, Verification & Diagnosis

Verification

* Given a system model, decide if the system meetsspecified requirements:- Safety.- Liveness (absence of deadlock states).- Progress / responsiveness.

* Distributed & concurrent systems:- Set of finite-state processes, may be dynamic.- Inter-processes communication: shared memory;

synchronised transitions; or message queues (pipes).- Non-deterministic: unpredictable relative speeds; or

abstract model.

Planning, Verification & Diagnosis

Protocols & Sequential Circuits

Planning, Verification & Diagnosis

Diagnosis & Diagnosability

* Given a system model, some knowledge of the initialsystem state and a series of observations of events fromthe running system, decide- what are possible current states of the system?- what unobservable events can / cannot have happened?- is system behaving according to the model?

* Given a system model, decide- what unobservable (failure) events, if any, can go

undiagnosed infinitely?- where to place additional sensors to make important

(e.g., failure) events diagnosable, at minimum cost?

Planning, Verification & Diagnosis

Example: The Vadstena Water Plant

Planning, Verification & Diagnosis

Industrial Automation

Planning, Verification & Diagnosis

(Williams & Nayak 1996)

(c) NASA

Autonomous Systems

Planning, Verification & Diagnosis

(Sohrabi, Udrea & Riabov 2013)

client1.connect(server)client1.high NX volumeserver.connect(client1)server.port scanning(client2)server.port scanning(client3)client3.connect(server)client2.connect(server)client2.many requests to one domainclient3.many requests to one domain

Security

Planning, Verification & Diagnosis

Example: WITAS UAV

10877070000 NAV.0.0 Created { MissionExecutor { { Position { 265.0 -66.0 20.0 ...10893800000 NAV.0.0 CheckPt 4 { { SegmentSeq { { { { 289.7897914429 -72.7260848284 ...10895050000 F3D.0.0 Created { NAV.0.0 { { { SegmentSeq { { { { 289.7897914429 ...10895770000 F3D.0.0 CheckPt 11 { { { double 19.1655151302 } } } }10896280000 F3D.0.0 CheckPt 2110990120000 Heli 0 FINISHED10990270000 F3D.0.0 CheckPt 2410990380000 F3D.0.0 Mode:done10990660000 F3D.0.0 Destroyed

Planning, Verification & Diagnosis

1

4

4

4

4

4

4

3

4

2

2

4

2

2

2 2

4

4

4

Planning, Verification & Diagnosis

1

4

4

4

4

4

4

3

4

2

2

4

2

2

2 2

4

4

4

Planning, Verification & Diagnosis

1

4

4

4

4

4

4

3

4

2

2

4

2

2

2 2

4

4

4

Planning, Verification & Diagnosis

Diagnosability

* Let Mod be the set of all event sequences allowed by thesystem model.

* A fault f is diagnosable iff ∃n∀s ∈ Mod : f ∈ s∀s′ ∈ Mod : s′ = s, t ; |obs(t)| ≥ n∀s′′ ∈ Mod : obs(s′′) = obs(s′)f ∈ s′′,

i.e., f is detectable after at most n observations.

* Twin-plant construction:- Compose S with a copy S′ of itself, synchronising on

observations.- If no infinite (i.e., looping) trajectory in S × S′ such that f

occurs in S but not in S′ exists, f is diagnosable.

Planning, Verification & Diagnosis

Problem Reductions

* Planning-as-model checking- Create a deadlock when goal achieved.

* Diagnosis-as-planning- Goal is to generate the given observations.- Need 0/1 action costs to minimise fault events.

* Planning-as-diagnosis- Explain observation “goal achieved”, with/without

“cheating” (fault).

* Diagnosability-as-planning- “Mark & return” for looping acceptance path.

* Model checking-as-planning- Synchronise with Buchi automaton.

Planning, Verification & Diagnosis

Domain Independence

* System models required for reasoning typically given in aformal description language.

* A reasoner/algorithm is domain-independent if it isapplicable to a class of models that is defined by what thereasoners input language can express.

* Why, or why not, domain-independence?- Generality of algorithms & underlying principles.- Improve reuse, rapid prototyping.- Opportunities for improvement (efficiency, reasoning

capability) by specialisation to a domain / application.

* No reasoning systems are truly domain-independent.

Planning, Verification & Diagnosis

0 : (move-up g)1 : (move-right h)2 : (move-down f)3 : (move-down f)4 : (move-right b)5 : (move-right b)6 : (move-up d)7 : (move-up d)8 : (move-right a)9 : (move-down c)

10 : (move-down c)11 : (move-down c)12 : (move-left a)13 : (move-down d)14 : (move-down d)15 : (move-left b)16 : (move-left b)17 : (move-up f)18 : (move-up f)19 : (move-left h)20 : (move-left b)21 : (move-up d)22 : (move-left h)23 : (move-left h)24 : (move-down f)25 : (move-down f)

26 : (move-down c)27 : (move-left h)28 : (move-down d)29 : (move-right b)30 : (move-right b)31 : (move-right b)32 : (move-up d)33 : (move-right h)34 : (move-up c)35 : (move-up d)36 : (move-right a)37 : (move-up c)38 : (move-up c)39 : (move-left h)40 : (move-up e)41 : (move-left i)42 : (move-left i)43 : (move-left i)44 : (move-down f)45 : (move-down g)46 : (move-right a)47 : (move-right a)48 : (move-down g)49 : (move-down g)50 : (move-right a)

* Trying to find a solution for ∼ 3 days.

* Writing a PDDL model took ∼ 45 minutes.

* Planner to find an optimal plan: 0.1 seconds.

State-Space Exploration

State-Space Exploration

* Given an implicit representation of S = S1 × . . .× Sn,generate (and store) only as much of S as is needed tosolve the problem.

* Forward & Regression search.

* Graph search algorithms:- Best-First Search.- Memory-limited search.

State-Space Exploration

......

......

...

......

State-Space Exploration

Microban #1

State-Space Exploration

Regression (backward search)

��� � ����� ��� ��� ���������������� ������ ����� � �

!"���������������� ������ �����#� �

!$�

% '&�� &(� & �$��� &����) ��*�� � �!"�

��) ��*�� � �!$�

��+ � � ��, � ��� ��+ � � ��,-� ��� ��+ � � ��,-� ��� ��+ � � ��, � ���

* φ holds in apply(a, s) iff regress(φ,a) holds in s.- apply(a, s) |= ϕ iff s |= regress(ϕ,a).

* In STRIPS notation:

regress(c,a) =

{(c − add(a) ∪ pre(a) if del(a) ∩ c = ∅FALSE otherwise

State-Space Exploration

(box s42),(box s23)

(push-down s22 s32 s42)

(empty s42),(man s22),(box s32),(box s23)

(move-down s42 s52)

(empty s52),(man s42),(man s22),(box s32),(box s23)

(move-up s42 s32)

(man s42),(empty s32),(man s22),(box s32),(box s23)

(move-up s32 s22)

(empty s42),(man s32),(empty s22),(box s32),(box s23)

. . .

(push-up s62 s52 s42)

(man s62),(empty s42),(box s52),(box s23)

. . .

(push-up s43 s33 s23)

(man s43),(empty s23),(box s42),(box s33)

. . . . . .

(push-left s44 s43 s42)

(man s44),(empty s42),(box s43),(box s23)

. . .

State-Space Exploration

(box s42),(box s23)

(push-down s22 s32 s42)

(empty s42),(man s22),(box s32),(box s23)

(move-down s42 s52)

(empty s52),(man s42),(man s22),(box s32),(box s23)

(move-up s42 s32)

(man s42),(empty s32),(man s22),(box s32),(box s23)

(move-up s32 s22)

(empty s42),(man s32),(empty s22),(box s32),(box s23)

. . .

(push-up s62 s52 s42)

(man s62),(empty s42),(box s52),(box s23)

. . .

(push-up s43 s33 s23)

(man s43),(empty s23),(box s42),(box s33)

. . . . . .

(push-left s44 s43 s42)

(man s44),(empty s42),(box s43),(box s23)

. . .

State-Space Exploration

(box s42),(box s23)

(push-down s22 s32 s42)

(empty s42),(man s22),(box s32),(box s23)

(move-down s42 s52)

(empty s52),(man s42),(man s22),(box s32),(box s23)

(move-up s42 s32)

(man s42),(empty s32),(man s22),(box s32),(box s23)

(move-up s32 s22)

(empty s42),(man s32),(empty s22),(box s32),(box s23)

. . .

(push-up s62 s52 s42)

(man s62),(empty s42),(box s52),(box s23)

. . .

(push-up s43 s33 s23)

(man s43),(empty s23),(box s42),(box s33)

. . . . . .

(push-left s44 s43 s42)

(man s44),(empty s42),(box s43),(box s23)

. . .

State-Space Exploration

(box s42),(box s23)

(push-down s22 s32 s42)

(empty s42),(man s22),(box s32),(box s23)

(move-down s42 s52)

(empty s52),(man s42),(man s22),(box s32),(box s23)

(move-up s42 s32)

(man s42),(empty s32),(man s22),(box s32),(box s23)

(move-up s32 s22)

(empty s42),(man s32),(empty s22),(box s32),(box s23)

. . .

(push-up s62 s52 s42)

(man s62),(empty s42),(box s52),(box s23)

. . .

(push-up s43 s33 s23)

(man s43),(empty s23),(box s42),(box s33)

. . . . . .

(push-left s44 s43 s42)

(man s44),(empty s42),(box s43),(box s23)

. . .

Relaxations and Heuristics

Relaxations and Heuristics

* A relaxation is a problem transformation that preservessolution existence and cost.

* If the relaxed problem is easier to solve (e.g., polynomial),optimal solution cost for the relaxed problem is usable asan admissible heuristic for the original problem.

* Relaxations of the reachability problem:- Abstraction.- Monotonic relaxation.

Relaxations and Heuristics

Abstraction

* An abstraction is a mapping from (states of) S to someabstract space S′, which preserves labelled paths andgoal states.

Relaxations and Heuristics

Monotonic Relaxation

* State variables/components accumulate values.

* Propositionally, facts, once true, never become false.- In planning, this is known as the delete relaxation (P+).

* Relaxation: any plan for P is also valid for P+.

* Solving P+ is in P.- Applying an action (transition) cannot disable any other

action.- No action needs to be applied more than once.

* Solving P+ optimally is NP-hard.

Relaxations and Heuristics

Example: Monotonic Relaxation

B

A

C

(on-table A)(on-table B)(on-table C)(on A B)(on A C)(on B A)(on B C)(on C A)(on C B)(clear A)(clear B)(clear C)(MoveToT+ A B)pre: (on A B),

(clear A)add: (on-table A),

(clear B)

A

A

B C

(on-table A)(on-table B)(on-table C)(on A B)(on A C)(on B A)(on B C)(on C A)(on C B)(clear A)(clear B)(clear C)(MoveFromT+ B C)pre: (on-table B),

(clear B), (clear C)add: (on B C)

A

B

A

A C

B

(on-table A)(on-table B)(on-table C)(on A B)(on A C)(on B A)(on B C)(on C A)(on C B)(clear A)(clear B)(clear C)

Logical Reasoning About Discrete Event Systems

Using Logic

* Proving invariance.

* Bounded reachability and propositional logic.

* Tense-modal logic- Logic as a specification language.- LTL and LTL resolution.

* Proving non-reachability.

Logical Reasoning About Discrete Event Systems

Invariants (for Planning)

* A formula ϕ is an invariant iff s |= ϕ and enabled(a, s)implies apply(a, s) |= ϕ, for s and a.- If ϕ is invariant and holds in s0, ϕ holds in any reachable

state.

* Testing invariance w.r.t. action schemas:- Regression: apply(a, s) |= ϕ iff s |= regress(ϕ,a).- ϕ is invariant iff for each a

∀~x(ϕ ∧ pre(a(~x))→ regress(ϕ,a(~x))

is a tautology.

Logical Reasoning About Discrete Event Systems

Regression over Action Schemas

* Regressing an atomic formula: apply(a(~y), s) |= p(~x) iff- p(~z) ∈ effects(a(~y)) and ~x = ~z; or- s |= p(~x) and for any ¬p(~z) ∈ effects(a(~y)), ~x 6= ~z

* Regressing any formula: substitute condition for atoms.

* move-right(f , t)precond: right-of(f , t),man(f ),empty(t)effects: ¬man(f ),empty(f ),man(t),¬empty(t)

* regress(man(x) ∨ box(x) ∨ empty(x),move-right(f , t))= ((x = t) ∨ (man(x) ∧ x 6= f )) ∨ box(x)∨

((x = f ) ∨ (empty(x) ∧ x 6= t)).

Logical Reasoning About Discrete Event Systems

Example: Invariants (?)

1. ∀x(man(x) ∨ box(x) ∨ empty(x))

2. ∀x((¬man(x) ∧ ¬box(x)) ∨ (¬man(x) ∧ ¬empty(x))∨(¬box(x) ∧ ¬empty(x)))

3. ∀x , y(man(x) ∧ man(y)→ (x = y))

4. ∀x , y , z(box(x) ∧ box(y) ∧ box(z)→((x = y) ∨ (x = z) ∨ (y = z)))

5. ∃x , y(box(x) ∧ box(y) ∧ (x 6= y))

Logical Reasoning About Discrete Event Systems

Horizon-k Planning as SAT

p1@0p2@0...pn@0

a1@1a2@1......am@1

p1@1p2@1...pn@1

a1@2a2@2......am@2

p1@2p2@2...pn@2

. . .

a1@ka2@k......am@k

p1@kp2@k...pn@k

* The set of valid plans up to (parallel) length k can beencoded as a propositional logic formula φ.

Logical Reasoning About Discrete Event Systems

* a@i →∧

p∈pre(a) p@(i − 1).- If a takes place at i , pre(a) must hold at i − 1.

* a@i →∧

p∈add(a) p@i .

* a@i →∧

q∈del(a) ¬q@i .- If a takes place at i , it’s effects hold at i .

* (p@i ∧ ¬p@(i − 1))→∨{a | p∈add(a)} a@i .

* (¬p@i ∧ p@(i − 1))→∨{a | p∈del(a)} a@i .

- If p changes from i − 1 to i , some action must havecaused the change.

Logical Reasoning About Discrete Event Systems

* p@0 if p ∈ s0.

* ¬p@0 if p 6∈ s0.- The first atom layer is the initial state.

* p@k if p ∈ G.- The last atom layer satisfies the goal.

* ¬a@i ∨ ¬a′@i if del(a) ∩ (pre(a′) ∪ add(a′)) 6= ∅ ordel(a′) ∩ (pre(a) ∪ add(a)) 6= ∅.- More than one action can take place at i as long as they

do not interfere.- Weaker requirement: actions at i can be performed in

some sequence.

Logical Reasoning About Discrete Event Systems

SAT-based planning

* How to find the right k?

* Test Φk , k = 1,2, . . .,until a satisfyingassignment (plan) isfound.

* Can be very inefficient.

* This is what SATPLANdoes.

(Rintanen, Heljanko & Niemela, 2006)

Logical Reasoning About Discrete Event Systems

SAT-based planning

* Try many values inparallel, allocatinggeometrically less timeto higher values.

* As SAT formulasproved unsolvable, thefrontier moves up.

* Limited by memory.(Rintanen, Heljanko & Niemela, 2006)

Logical Reasoning About Discrete Event Systems

Tense-Modal Logic

* LTL, CTL and CTL* are tense modal logics.- LTL is interpreted over paths (sequences of states).- CTL and CTL* are interpreted over trees.

* Property specification language:- Safety: �¬deadlock (LTL), ∀�¬deadlock (CTL)- Responsiveness: �(p → ♦q) (LTL), ∀�(p → ∀♦q) (CTL)- Possibility: ∀�∃♦goal (CTL)- ♦�p (LTL)

Logical Reasoning About Discrete Event Systems

Linear Temporal Logic (LTL)

* LTL is interpreted over infinite sequences of states.

* w = s0, s1, . . .:- w |= p iff s0 |= p.- Interpretation of logical connectives (¬, ∧, ∨) is standard.- w |=©α iff w1 = s1, s2, . . . |= α.- w |= (αU β) iff ∃k .wk |= β and ∀0 ≤ i < k .w i |= α.

* Common abbreviations;- ♦α ≡ (TRUE U α)- �α ≡ (αU FALSE)- αWβ ≡ (αU β) ∨�α

Logical Reasoning About Discrete Event Systems

Planning as LTL-SAT

* The set of valid plans can be encoded as an LTL formula:- �(a→

∧p∈pre(a) p) (action implies prec.)

- �(a→∧

p∈add(a)©p)

- �(a→∧

p∈del(a)©¬p) (action implies eff.)- �(¬p ∧

∧{a | p∈add(a)} ¬a→©¬p)

- �(p ∧∧{a | p∈del(a)} ¬a→©p) (frame axioms)

- �(¬a ∨ ¬a′) (single action per step)- p if p ∈ s0, ¬p if p 6∈ s0 (init state)- ♦

∧p∈G p (eventually goal)

Logical Reasoning About Discrete Event Systems

LTL Resolution

* Clausal normal form:- �(L1 ∨ . . . ∨ Lm) (state clause)- �(A→©B) (step clause)- �(A→©♦L) (eventuality clause)- (L1 ∨ . . . ∨ Lm) (initial clause)(A conjunction of literals, B disjunction of literals).

* State and step resolution:- Resolve state/step clauses with conflict literals of

“matching” modality.

* ♦-resolution.- (∨

i Ai)→©�¬L, C →©♦L⇒ C → (∧

i ¬Ai)WL- (∨

i Ai)→©�¬L is constructed from a loop in ¬L.

Logical Reasoning About Discrete Event Systems

* A merged step clause is a conjunction of step clauses,∧j �(Aj →©Bj), rewritten as�(Aj ∧ . . . ∧ Am →©(B1,1 ∧ . . . ∧ Bk ,1) ∨ . . . ∨©(B1,m ∧ . . . ∧ Bk ,m)).

* A set of merged step clauses �(Ai →©Bi) such that- Bi |= ¬L, and- Bi |=

∨i Ai ,

is a loop in ¬L.

* A loop is a conditional invariant: (∨

i Ai)→©�¬L.

Logical Reasoning About Discrete Event Systems

Proving Unreachability

* Exhaustive state-space search.

* Bounded SAT-encoding combined with induction.

* Constructive:- LTL-Resolution.- Relaxed regression.

* Constructive proof search can be guided.

Searching for something that exists is usually easier thansearching for something that does not exist.

Infinite-State Systems

Infinite-State Systems

* The undecidability problem.- Counter machines.

* Petri nets.

* Timed & hybrid automata.

Infinite-State Systems

k -Counter Machines

* Deterministic finite program with kcounters of infinite capacity.

* Typical instruction set:- Set ci to 0.- Increment or decrement ci by 1.- goto n.- if ci = 0 goto n else goto n′.- Halt.

* Halting of k -CMs is undecidable fork > 2.

1 clear c22 if c1 = 0 goto 73 dec c14 inc c25 inc c26 goto 27 if c2 = 0 goto 118 dec c29 inc c1

10 goto 711 ...

Infinite-State Systems

Petri Nets* Directed bipartite graph of places and transitions.- Places are counters: A marking assigns each place

n ∈ N tokens.- Transition t applicable at marking n iff ∀p ∈ •t ,m(p) > 0

and leads to marking m′ = m − •t + t•.

* Marking reachability for Petri nets is decidable.

p1

t1

p2

p3t2

t3

t1−→p1

t1

p2

p3t2

t3

t3−→p1

t1

p2

p3t2

t3

t1−→p1

t1

p2

p3t2

t3

t2−→p1

t1

p2

p3t2

t3

t2−→p1

t1

p2

p3t2

t3

Infinite-State Systems

Timed Automata

* Non-deterministic finite automatawith real-valued clocks.- Transition guards: Linear

conditions on clocks and clockdifferences.

- Transitions can reset clocks.- Clocks evolve at uniform rate.

* Reachability is PSPACE-completefor a single automaton.