Automated Discovery of Declarative Process Models

The Automated Discovery of Declarative Process ModelsClaudio Di Ciccio

claudio.di.ciccio@wu.ac.at

Humboldt-Universität zu Berlin, 7 December 2016

www.wu.ac.atwww.wu.ac.at/infobizwww.wu.ac.at/infobiz/team/diciccio

Control-flow discovery

?

Objective: understanding the temporal structure that best describes the process behind the event log

SEITE 3

Knowledge-intensive Processes

SEITE 4

Knowledge-intensiveProcesses

SEITE 5

Mining flexible processes

SEITE 6

Declarative modelling ofprocesses

Usage of constraints “Open model”

Declare state-of-the-art language

If A is performed,B must be performed,

no matter if before or afterwards(responded existence)

Whenever B is performed,C must be performed afterwards

and B can not be repeateduntil C is done

(alternate response)SEITE 7

Workflow Netsas process models

SEITE 8

Imperative v declarative

SEITE 9SEITE 9

Declarative processesDECLARE

Declarative process modelling

“Open model” Specify constraints for

permitted behaviour Every execution that

complies with them is acceptable

Works best with flexible processes

The set of DECLARE templates is extendible

SEITE 11

A fragment of declarative process model If an abstract is submitted, a new paper

had been written or will be written

After the paper submission, a confirmation email is sent

After the paper submission, the paper will be reviewed;there can be no review without a preceding submission

A paper can be accepted only after it has been reviewed

After the rejection, no further submission follows

Paper cannot be both accepted and rejected

SEITE 12

Submit abstract Write new paper

Submit paper Send confirmation email

Submit paper Review paper

Review paper Accept paper

Reject paper Submit paper

Accept paper Reject paper

= activation task

Responded existence(Submit abstract, Write new paper)

Response(Submit paper, Send confirmation email)

Succession(Submit paper, Review paper)

Precedence(Review paper, Accept paper)

Not succession(Reject paper, Submit paper)

Not co-existence(Accept paper, Reject paper)

Template Tasks

A fragment of declarative process model

SEITE 13

Submit abstract

Write new paper Submit paper

Send confirmation email

Review paper Accept paper

Reject paper

Declare:Existence Constraint Templates

Existence(n, A)Activity A occurs at least n times in the process instanceBCAAC ✓ BCAAAC ✓ BCAC ✗ (for n = 2)

Absence(A)Activity A does not occur in the process instanceBCC ✓ BCAC ✗

Absence(n+1, A)Activity A occurs at most n-1 times in the process instanceBCAAC ✗ BCAC ✓ BCC ✓ (for n = 2)

Exactly(n, A)Activity A occurs exactly n times in the process instanceBCAAC ✓ BCAAAC ✗ BCAC ✗ (for n = 2)

Init(A)Activity A is the first to occur in each process instanceBCAAC ✗ ACAAAC ✓ BCC ✗

Absence(2, A) ≡ AtMostOne(A)

Existence(1, A) ≡ Participation(A)

SEITE 14

Declare:Relation Constraint Templates

RespondedExistence(A, B)If A occurs in the process instance, then B occurs as wellCAC ✗ CAACB ✓BCAC ✓ BCC ✓

SEITE 15

Declare:Relation Constraint Templates

RespondedExistence(A, B)If A occurs in the process instance, then B occurs as wellCAC ✗ CAACB ✓BCAC ✓ BCC ✓Response(A, B)If A occurs in the process instance, then B occurs after ABCAAC ✗ CAACB ✓CAC ✗ BCC ✓

SEITE 16

Declare:Relation Constraint Templates

RespondedExistence(A, B)If A occurs in the process instance, then B occurs as wellCAC ✗ CAACB ✓BCAC ✓ BCC ✓Response(A, B)If A occurs in the process instance, then B occurs after ABCAAC ✗ CAACB ✓CAC ✗ BCC ✓AlternateResponse(A, B)Each time A occurs in the process instance, then B occurs afterwards, before A recursBCAAC ✗ CAACB ✗ CACB ✓CABCA ✗ BCC ✓ CACBBAB ✓

SEITE 17

Declare:Relation Constraint Templates

RespondedExistence(A, B)If A occurs in the process instance, then B occurs as wellCAC ✗ CAACB ✓BCAC ✓ BCC ✓Response(A, B)If A occurs in the process instance, then B occurs after ABCAAC ✗ CAACB ✓CAC ✗ BCC ✓AlternateResponse(A, B)Each time A occurs in the process instance, then B occurs afterwards, before A recursBCAAC ✗ CAACB ✗ CACB ✓CABCA ✗ BCC ✓ CACBBAB ✓ChainResponse(A, B)Each time A occurs in the process instance, then B occurs immediately afterwardsBCAAC ✗ BCAABC ✗ BCABABC ✓

Activation Target

Declare:Relation Constraint Templates

RespondedExistence(B, A)If B occurs in the process instance, then A occurs as wellCAC ✓ CAACB ✓BCAC ✓ BCC ✗Precedence(A, B)B occurs in the process instance only if preceded by ABCAAC ✗ CAACB ✓CAC ✓ BCC ✓AlternatePrecedence(A, B)Each time B occurs in the process instance, it is preceded by A and no other B can recur in betweenBCAAC ✗ CAACB ✓ CACB ✓CABCA ✓ BCC ✗ CACBAB ✓ChainPrecedence(A, B)Each time B occurs in the process instance, then B occurs immediately beforehandBCAAC ✗ BCAABC ✗ CABABCA ✓

Target Activation

Declare:Relation Constraint Templates

CoExistence(A, B) ≡ Resp’dEx.(A, B) Resp’dEx.(B, A)If B occurs in the process instance, then A occurs, and viceversaCAC ✗ CAACB ✓BCAC ✓ BCC ✗Succession(A, B) ≡ Response(A, B) Precedence(A, B)A occurs if and only if it is followed by B in the process instanceBCAAC ✗ CAACB ✓CAC ✗ BCC ✗AlternateSuccession(A, B) ≡ Alt.Resp.(A, B) Alt.Prec.(A, B)A and B occur in the process instance if and only if the latter follows the former, and they alternate each other in the traceBCAAC ✗ CAACB ✗ CACB ✓CABCA ✗ BCC ✗ CACBAB ✓ChainSuccession(A, B) ≡ ChainResp.(A, B) ChainPrec.(A, B)A and B occur in the process instance if and only if the latter immediately follows the formerBCAAC ✗ BCAABC ✗ CABABC ✓

Activation Target

Target Activation

Declare: NegativeRelation Constraint Templates

NotCoExistence(A, B)A and B never occur together in the process instanceCAC ✓ CAACB ✗BCAC ✗ BCC ✓NotSuccession(A, B)A can never occur before B in the process instanceBCAAC ✓ CAACB ✗CAC ✓ BCC ✓NotChainSuccession(A, B)A and B occur in the process instance if and only if the latter does not immediately follows the formerBCAAC ✓ BCAABC ✗ CBACBA ✓

Activation Target

Target Activation

Subsumption hierarchyof constraint templates

SEITE 22

Declarative processesDiscovery of DECLARE models

Constraints mining

?

Objective: understanding the constraints that best define the allowed behaviour of the process behind the event log

SEITE 24

Mining declarative processes:ingredients

“Submit abstract”,“Submit paper”,“Accept paper”,…

SEITE 25

A,B,C,…

Activities Process alphabet

Mining declarative processes

RespondedExistence(A, B) ?RespondedExistence(A, C) ?…Response(A, B) ?Response(A, C) ?…

SEITE 26

• Support:fraction of cases fulfilling the constraint

• Confidence:support scaled by fraction of traces in which the activation occurs

• Interest factor:confidence scaled by fraction of traces in which the target occurs

Support Conf. I.F.

Threshold

Threshold

Threshold

Mining declarative processes

RespondedExistence(A, B) ?RespondedExistence(A, C) ?…Response(A, B) ?Response(A, C) ?…

SEITE 27

Support Conf. I.F.

• Support:fraction of cases fulfilling the constraint

• Confidence:support scaled by fraction of traces in which the activation occurs

• Interest factor:confidence scaled by fraction of traces in which the target occurs

Mining declarative processes

RespondedExistence(A, B) RespondedExistence(A, C) …Response(A, B) ?Response(A, C) …

SEITE 28

Support Conf. I.F.

• Support:fraction of cases fulfilling the constraint

• Confidence:support scaled by fraction of traces in which the activation occurs

• Interest factor:confidence scaled by fraction of traces in which the target occurs

Mining declarative processes

RespondedExistence(A, B) RespondedExistence(A, C) …Response(A, B) Response(A, C) …

SEITE 29

Support Conf. I.F.

• Support:fraction of cases fulfilling the constraint

• Confidence:support scaled by fraction of traces in which the activation occurs

• Interest factor:confidence scaled by fraction of traces in which the target occurs

Constraints miningAn example

SEITE 30

A A B C A B C A C B C D

A C C A B B C B C A C B B D

C C C C C A A B C A A B A A B

A B B B D

C B A B D

A B B D

A A A C A C B D

A B C D

C A B A A C C B B D

B C C D

C A A C C C A A B C B C C B D

The role of Support(event-based)

A A B C A B C A C B C DA C C A B B C B C A C B B DC C C C C A A B C A A B A A BA B B B DC B A B DA B B DA A A C A C B DA B C DC A B A A C C B B DB C C DC A A C C C A A B C B C C B D

Support for Response(A, B)

1.0 Precedence(A, B)

0.926 (25/27) Pruning on the basis of a

support threshold E.g., 0.95

A threshold equal to 1.0 for a constraint means “always valid in the log”

ActivationTargetTarget

Activation

SEITE 31http://github.com/cdc08x/minerful

Prom Nightly Builds > “Declare MINERful” plug-in

The role of Support(trace-based)

A A B C A B C A C B C DA C C A B B C B C A C B B DC C C C C A A B C A A B A A BA B B B DC B A B DA B B DA A A C A C B DA B C DC A B A A C C B B DB C C DC A A C C C A A B C B C C B D

Support for Response(A, B)

1.0 Precedence(A, B)

0.818 (9/11) Pruning on the basis of a

support threshold E.g., 0.95

A threshold equal to 1.0 for a constraint means “always valid in the log”

SEITE 32 http://sourceforge.net/projects/prom/files/ProM/6.3

Declarative processesChallenges

SEITE 34

Objective

SEITE 35

Make mined modelsintelligible

Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells Increase the expressiveness of discovered

models Specify templates

SEITE 36

Make mined modelsintelligible

Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells Increase the expressiveness of discovered

models Specify templates

SEITE 37© Granger, NYC, source: https://www.granger.com/

Redundancies in discovered constraints

E.g., A trace likeA B A B C A B C Csatisfies (w.r.t. A and B): RespondedExistence(A, B), RespondedExistence(B, A),

Response(A, B), AlternateResponse(A, B), ChainResponse(A, B),Precedence(A, B), AlternatePrecedence(A, B), ChainPrecedence(A, B),CoExistence(A, B), CoExistence(B, A),Succession(A, B), AlternateSuccession(A, B), ChainSuccession(A, B)

Among them, the mining algorithm should return only the most restricting constraint – i.e., ChainSuccession(A, B)

In order to face this issue, returned constraints are pruned on the basis of the subsumption hierarchy of constraints

SEITE 38

Pruning redundant constraints

1.0

1.0

1.0

1.01.0

1.0

1.0

1.0

1.0

1.0

1.0

✖✖✖

✖✖✖✖

✖✖✖

SEITE 39

Pruning redundant constraints

0.8

0.8

0.9

0.90.9

0.9

0.7

0.7

0.9

0.9

0.9

✖ ✖??

✖

✖

✖

??

Support ✖

✖✖✖

SEITE 40

Mining declarative processes

RespondedExistence(A, B) RespondedExistence(A, C) ?…Response(A, B) Response(A, C) …

SEITE 41

Support Conf. I.F.

Mining declarative processes

RespondedExistence(A, B) RespondedExistence(A, C) ?…Response(A, B) Response(A, C) …

SEITE 42

Support Conf. I.F.

Mining declarative processes

RespondedExistence(A, B) RespondedExistence(A, C) …Response(A, B) Response(A, C) …

SEITE 43

Support Conf. I.F.

http://github.com/cdc08x/minerfulProm Nightly Builds > “Declare MINERful” plug-inhttp://sourceforge.net/projects/prom/files/ProM/6.3

Make mined modelsintelligible

Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells Increase the expressiveness of discovered

models Specify templates

SEITE 44

Make mined modelsintelligible

Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells Increase the expressiveness of discovered

models Specify templates

InterludeDECLARE & Regular Expressions

SEITE 45

Mining declarative processes

RespondedExistence(A, B)RespondedExistence(A, C)andResponse(A, B)Response(A, C)and…

SEITE 46

Semantics of Declare:Regular Expressions

SEITE 47

From constraints-based model to FSA

SEITE 48

[^a]*((a.*b.*)|(b.*a.*))*[^a]* [^a]*(a.*c)*[^a]*

RegularExpression

DeterministicFiniteState

Automaton

RespondedExistence(A, B)RespondedExistence(A, C)andResponse(A, B)Response(A, C)and…

Make mined modelsintelligible

Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells Increase the expressiveness of discovered

models

Reprise

SEITE 49

While mining a real-life log…

SEITE 50

Submit

draft

Write

deliverableOrganise

agenda

Time to challenge the X

SEITE 51

Submit

draft

Write

deliverableOrganise

agenda

The result

SEITE 52

The problem

When support threshold is lower than 100%,constraints can be valid through most of the log, though being in conflict

Example: an event log consists of two traces:1. <A, B, A, B, A, B, C>2. <A, B, A, B, A, C>

Support threshold: 0.7• a is always the first

Init(A)• c is always the last

End(C)• In 6 cases over 8 (75%), a and c do not directly follow each

other NotChainSuccession(A, C)

• In 5 cases over 7 (71.143%), b and c do not directly follow each other

NotChainSuccession(B, C)

SEITE 53

The problem

When support threshold is lower than 100%,constraints can be valid through most of the log, though being in conflict

Example: an event log consists of two traces:1. <A, B, A, B, A, B, C>2. <A, B, A, B, A, C>

Support threshold: 0.7• a is always the first

Init(A)• c is always the last

End(C)• In 6 cases over 8 (75%), a and c do not directly follow each

other NotChainSuccession(A, C)

• In 5 cases over 7 (71.143%), a and b do not directly follow each other

NotChainSuccession(B, C)

Question: what can be done right before C? inconsistency!

SEITE 54

The algorithm to detect inconsistencies

SEITE 55

Init(a)

Participation(b)AtMostOne(c)End(c) ChainPrecedence(a, b)…

ChainSuccession(a, b)Response(a, b)ChainResponse(b, a)NotChainSuccession(a, c) ChainSuccession(b, a)NotChainSuccession(b, c)…

…

…

1

Event log

Declare constraints

Automata

Safe constraints

(support 100%)

Unsafe

constr

aints

Product

automaton

The algorithm to detect inconsistencies

Init(a)

Participation(b)AtMostOne(c)End(c) ChainPrecedence(a, b)…

ChainSuccession(a, b)Response(a, b)ChainResponse(b, a)NotChainSuccession(a, c) ChainSuccession(b, a)NotChainSuccession(b, c)…

…

…

1

2

Event log

Declare constraints

Automata

Safe constraints

Unsafe

constr

aints

Product

automaton

SEITE 56 http://github.com/cdc08x/minerful

SEITE 57

Make mined modelsintelligible

Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells Increase the expressiveness of discovered

models

SEITE 58© Granger, NYC, source: https://www.granger.com/

The application of the method to minimise the model

Rationale:1. How to find redundancies among constraints?

Use the automaton-model correspondence Same language recognised after the product?

Main difference with the inconsistency-checking algorithm

Constraints having support 100% are checked for redundancies

More details in the paper

SEITE 59 http://github.com/cdc08x/minerful

The application of the method to minimise the model

SEITE 60

BPIC 2012

Make mined modelsintelligible

Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells

Inspect the imperative counterpart Simulate runs

Increase the expressiveness of discovered models

Specify templates

SEITE 61

From declarative to imperative models

SEITE 62

Event log

Declare

constraintsResponse(Queued, Accepted)NotChainSuccession(Queued, Completed)Response(Queued, Completed)NotChainSuccession(Queued, Unmatched)Response(Accepted, Completed)End(Completed)NotSuccession(Completed, Unmatched)AtMostOne(Unmatched)RespondedExistence(Unmatched, Accepted)AlternateResponse(Unmatched, Completed)

…

Automata

Product

automatonTheory ofRegions

Petri Net

Petrify

Make mined modelsintelligible

Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells

Inspect the imperative counterpart Simulate runs

Increase the expressiveness of discovered models

Specify templates

SEITE 63

Generation of logs

SEITE 64

http://github.com/cdc08x/minerfulhttp://github.com/processmining/synthetic-log-generator

Make mined modelsintelligible

Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells

Inspect the imperative counterpart Simulate runs

Increase the expressiveness of discovered models

Specify templates

SEITE 65

Target-Branched Declare

Support for Response(A, {B, C})

1.0

B A A B A B C DA A D B B A A D CB A D D A D D A B EC A B CD A D B DA E A CA A A D BD A D D B DA A A D C DA D D D A CA A BE D E B C E E C B E

SEITE 66

ActivationTarget

Branching factor = 2

Objective

SEITE 67

+ Expressive Power+ Conciseness

Evaluation:constraints pruning

SEITE 68

BPI Challenge 2012(0) 6,654,480 (1) 10,676 (2) 1446 (3) 12

Evaluation:performance

SEITE 69

BPI Challenge 2012(KB) 00:07.274 (Const’s) 25:51.678 (Total) 26:11.380

http://github.com/cdc08x/minerful

Make mined modelsintelligible

Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells

Inspect the imperative counterpart Simulate runs

Increase the expressiveness of discovered models

Specify templates

SEITE 70

Semantics of Declare:LTL and LTLf

Linear Temporal Logic (LTL) initially was a specification language for the execution of (endless) concurrent programs (Pnueli, 1977) Syntax (let A be a propositional symbol):

DECLARE was initially based on LTL

SEITE 71

“Until”

“Eventually”“Always”

“Next”

Semantics of Declare:LTL

SEITE 72

Declarative process modelling

“Open model” Specify constraints for

permitted behaviour Every execution that

complies with them is acceptable

Works best with flexible processes

The set of DECLARE templates is extendible

SEITE 73

Extendibility of DECLARE:A clear business impact

SEITE 74Source: http://businessvalueexchange.com/

Semantics of Declare:LTL and LTLf

Linear Temporal Logic (LTL) initially was a specification language for the execution of (endless) concurrent programs (Pnueli, 1977) Syntax (let A be a propositional symbol):

Interpretation over infinite traces,i.e., an infinite sequence of consecutive instants of time

LTLf formulae are meant to be interpreted over finite traces

“Until”

“Eventually”“Always”

“Next”

SEITE 75

Semantics of Declare:LTLf

SEITE 76

Semantics of Declare:SCIFF

SEITE 77

Semantics of Declare:R/I-nets

SEITE 78

Semantics of Declare:FOL over finite traces

SEITE 79

Semantics of Declare:Regular expressions

SEITE 80

More alternatives for DECLARE spec.: A clear business impact

SEITE 81Source: http://businessconsultantsnewyork.blogspot.co.at/

My long-term objective

Establish an algebra of constraint templates, built on a basis of behavioural relations that are: orthogonal to one another (no entailments,

subsumptions…) covering multiple trace-based behavioural relations

definition languages by linear composition have clear semantics (last but for sure NOT least) ...

SEITE 82

Further reading

Presented: About MINERful:

Claudio Di Ciccio, Massimo Mecella: On the Discovery of Declarative Control Flows for Artful Processes. ACM Trans. Management Inf. Syst. 5(4): 24:1-24:37 (2015)

Simulation of DECLARE models: Claudio Di Ciccio, Mario Luca Bernardi, Marta Cimitile, Fabrizio Maria Maggi: Generating Event Logs Through the

Simulation of Declare Models. EOMAS@CAiSE 2015: 20-36 Discovery of target-branched DECLARE models:

Claudio Di Ciccio, Fabrizio Maria Maggi, Jan Mendling: Efficient discovery of Target-Branched Declare constraints. Inf. Syst. 56: 258-283 (2016)

Getting rid of redundancies and inconsistencies: Claudio Di Ciccio, Fabrizio Maria Maggi, Marco Montali, Jan Mendling: Ensuring Model Consistency in Declarative Process

Discovery. BPM 2015: 144-159 From DECLARE to Petri nets:

Johannes Prescher, Claudio Di Ciccio, Jan Mendling: From Declarative Processes to Imperative Models. SIMPDA 2014: 162-173

More: First steps towards data awareness of discovered DECLARE models:

Stefan Schönig, Claudio Di Ciccio, Fabrizio Maria Maggi, Jan Mendling: Discovery of Multi-perspective Declarative Process Models. ICSOC 2016: 87-103

Against the little learnibility of DECLARE: Michael Hanser, Claudio Di Ciccio, Jan Mendling: A New Notational Framework for Declarative Process Modeling.

Softwaretechnik-Trends 36(2) (2016) Guarantee of the significance of discovered DECLARE models:

Fabrizio Maria Maggi, Marco Montali, Claudio Di Ciccio, Jan Mendling: Semantical Vacuity Detection in Declarative Process Mining. BPM 2016: 158-175SEITE 83