56

process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

  • Upload
    others

  • View
    5

  • Download
    0

Embed Size (px)

Citation preview

Page 1: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique
Page 2: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 1

Page 3: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 2

Conclusion

How about data mining and

business process management?

When did process mining

start?

What are the main PM developments

in this century?

How did PM tooling develop

over time? Three key observations

Why is process discovery so

difficult?

What are the main research

challenges?

Page 4: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 3

Conclusion

How about data mining and

business process management?

When did process mining

start?

What are the main PM developments

in this century?

How did PM tooling develop

over time? Three key observations

Why is process discovery so

difficult?

What are the main research

challenges?

Page 5: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Positioning Process Mining

4

process

mining

Data Mining (DM)

(clustering, classification, rule discovery, etc.)

Business Process Management (BPM)(process analysis/modeling, enactment,

verification, etc.)

pe

rform

an

ce-o

rien

ted

qu

es

tion

s,

pro

ble

ms

an

d s

olu

tion

s

co

mp

lian

ce

-orie

nte

d q

ue

stio

ns

,

pro

ble

ms

an

d s

olu

tion

s

Page 6: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

History and Origins of BPM

PAGE 5

database

system

user

interface

database

system

user

interface

database

systema

pp

lica

tio

n

BP

M s

yste

m1960 1975 1985 2000

ap

plic

atio

n

ap

plic

atio

n

ap

plic

atio

n

BPM

WFM

office

automation

data

modeling

operations

management

scientific

management

business

intelligence

software

engineering

formal

methods

business

process

reengineering

Skip Ellis, Office Talk, 1979

Michael Zisman, SCOOP, 1977

Anatol Holt, Information Systems Theory

Project, 1968

Carl Adam Petri, Petri nets, 1962

Page 7: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

History and Origins of Data Mining

PAGE 6

Classical statistics (since 500 BC): descriptive statistics (e.g., sample mean) statistical inference (e.g., confidence interval, regression, hypothesis testing).

Artificial intelligence (since 1950): making intelligent machines by applying human-thought-like processing to statistical problems.

Machine learning (since 1950): construction and study of systems that can learn from data.

Page 8: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Data Mining: Supervised Learning

•  Labeled data, i.e., there is a response variable that labels each instance.

•  Goal: explain response variable (dependent variable)

in terms of predictor variables (independent variables).

•  Classification techniques (e.g., decision tree

learning) assume a categorical response variable

and the goal is to classify instances based on the predictor variables.

•  Regression techniques assume a numerical

response variable. The goal is to find a function that

fits the data with the least error.

PAGE 7

Page 9: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Example: Decision tree learning

PAGE 8

logic

failed

(79/10)

- ≥ 8

passed

(31/7)

failed

(101/8)

linear

algebra

program

ming

operat.

research

cum laude

(20/2)

<8

<6

<6

passed

(82/7)

≥ 6

≥ 6

passed

(87/11)

≥ 7

<7

linear

algebra ≥ 6

<6

failed

(20/4)

Page 10: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Unsupervised Learning

•  Unsupervised learning assumes unlabeled data, i.e., the variables are not split into response and

predictor variables.

•  Examples: clustering (e.g., k-means clustering and

agglomerative hierarchical clustering) and pattern

discovery (association rules)

PAGE 9

Page 11: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Example: Association rules

PAGE 10

Page 12: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Example: Episode Mining

PAGE 11

a

b

c

d

E1

b

c

E2

a

b

c

d

E3

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37

a c b de c b b cf a e eb c d c b

E1

E2 (16x)

E1 E3

Page 13: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 12

Conclusion

How about data mining and

business process management?

When did process mining

start?

What are the main PM developments

in this century?

How did PM tooling develop

over time? Three key observations

Why is process discovery so

difficult?

What are the main research

challenges?

Page 14: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Language identification in the limit

(Mark Gold 1967)

•  Mother uses sentences from some language {aab, ab, ab, abc, …}.

•  "Perfect child" listens to mother and

hypothesizes what the full language is like (given all sentences so far).

•  Eventually the perfect child’s

hypothesis is correct and never

changes again (without knowing), i.e., only finitely many wrong

hypotheses are generated.

•  A language is learnable in the limit if

such a perfect child exists.

PAGE 13 Language identification in the limit by E Mark Gold, Information and Control, 10(5):447–474, 1967.

Page 15: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Language identification in the limit

(E. Mark Gold 1967)

•  Gold showed that most languages cannot be

learned in the limit (including the most simple

ones like regular languages (ab*(c|d)).

•  He noted that it matters whether the child gets positive and negative examples (corrections),

whether the mother is evil, etc.

•  Frequencies matter!

•  Representational bias matters!

PAGE 14

sentence ≅ trace in event log

language ≅ process model

Page 16: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Myhill-Nerode Theorem (1958) and the

Biermann/Feldman Algorithm (1972)

•  There is a unique minimal deterministic finite automaton recognizing a regular language L ( shown

by John Myhill and Anil Nerode in 1958).

•  The equivalence classes defined by ≅ determine the

states of the automaton: x ≅ y if there is no z such

that xz∉L and yz∈L.

•  Cannot be applied to example traces: overfitting and

no generalization.

•  Alan W. Biermann and Jerome A. Feldman propose

in 1972 techniques to learn finite state machines

from examples (e.g., considering k-tails).

PAGE 15

Nerode. Linear automaton transformations. Proc. Amer. Math. Soc. 9 1958 541-544. Biermann and Feldman. On the synthesis of finite-state machines from samples of their behaviour.

IEEE Transactions on Computers, 21:592–597, 1972.

Page 17: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique
Page 18: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Where/when did process mining start?

•  Myhill/Nerode(1958)?

•  Gold (1967)?

•  Baum/Welch (1970)?

•  Biermann/Feldman (1972)?

•  Rakesh Agrawal (1994)? −  Apriori algorithm for frequent patterns, later extended to sequences, episodes, …

•  Jonathan Cook and Alexander Wolf (1998)? −  "Discovering Models of Software Processes from Event-Based Data"

−  using techniques similar to Biermann/Feldman (k-tails) and Baum/Welch (Markov models)

•  Rakesh Agrawal, Dimitrios Gunopulos, Frank Leymann? −  "Mining Process Models from Workflow Logs" (1998)

−  Flowmark process models without discovering type of splits and joins, no loops, etc.

•  Anindya Datta (1998)? −  Automating the Discovery of AS-IS Business Process Models

−  Biermann/Feldman style work, embedded in BPM PAGE 17

Page 19: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique
Page 20: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Initial team

PAGE 19

Page 21: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 20

Conclusion

How about data mining and

business process management?

When did process mining

start?

What are the main PM developments

in this century?

How did PM tooling develop

over time? Three key observations

Why is process discovery so

difficult?

What are the main research

challenges?

Page 22: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Workflow

Mining

PAGE 21

diagnosis/

requirements

configuration/

implementation

enactment/

monitoring

adjustment

(re)designmodelsdata

insight

discussion

verification

performance

analysisanimation

specificationdocumentation

configuration

Page 23: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Models, data, and systems coexist

PAGE 22

(re)

design

implement/configure

run & adjust

model-b

ased

analysis

data-based

analysis

Page 24: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique
Page 25: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Team in November 2007

PAGE 24

Some people are missing, e.g., Peter van den Brand.

Page 26: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Current process mining spectrum (including alignments, operational support, and multiple perspectives)

PAGE 25

information system(s)

current

data

“world”people

machines

organizationsbusiness

processes documents

historic

data

resources/

organization

data/rules

control-flow

de jure models

resources/

organization

data/rules

control-flow

de facto models

provenance

exp

lore

pre

dic

t

reco

mm

en

d

de

tect

ch

eck

co

mp

are

pro

mo

te

dis

co

ve

r

en

ha

nce

dia

gn

ose

cartographynavigation auditing

event logs

Models

“pre

mortem”

“post

mortem”

Page 27: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 26

Conclusion

How about data mining and

business process management?

When did process mining

start?

What are the main PM developments

in this century?

How did PM tooling develop

over time? Three key observations

Why is process discovery so

difficult?

What are the main research

challenges?

Page 28: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Pre-ProM (figure from March 2002!)

PAGE 27

Staffware

InConcert

MQ Series

workflow management systemen

FLOWer

Vectus

Siebel

case handling / CRM systemen

SAP R/3

BaaN

Peoplesoft

ERP systems

gemeenschappelijk XML formaat voor

het opslaan van workflow logs

EMiTLittle

Thumb

mining tools

InWoLvEProcess

Miner

Exper-

DiTo

alpha algorithm

including time

analysis

(BvD)

predecessor

of MXML

format

predecessor of ProM's

heuristic miner (TW) mining with

duplicate tasks

(Joachim Herbst)

mining block

structured models

(Guido Schimm)

evaluation tool

(Laura Maruster)

The first tool to support

the alpha algorithm for

process mining was the

MiMo (Mining Module)

tool based on ExSpect.

Later it was

implemented in EMiT

and ProM.

Tobias Blickle (ARIS PPM)

Page 29: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 28

EMiT

MiMo

Little Thumb

Process Miner

Page 30: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique
Page 31: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique
Page 32: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique
Page 33: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique
Page 34: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique
Page 35: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique
Page 36: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique
Page 37: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique
Page 38: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 37

Conclusion

How about data mining and

business process management?

When did process mining

start?

What are the main PM developments

in this century?

How did PM tooling develop

over time? Three key observations

Why is process discovery so

difficult?

What are the main research

challenges?

Page 39: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

How good is my model: Four forces

PAGE 38

fitness

simplicity

generalization

precision

Process

Mining

ability  to  explain  

observed  behavior

avoiding  

underfitting

Occam’s  Razor

avoiding  

overfitting

lift

gravity

thrust drag

Leaving out one of these dimensions during discovery will lead to degenerate cases!

Page 40: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique
Page 41: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 40

formal (not just a

picture)

fast (should not

take years) sound (result should

at least be free of deadlocks,

etc.)

ability to balance all conformance

dimensions (fitness, precision,

generalization, and simplicity) incl.

noise

provide guarantees

(not just a best effort)

1

2

3 4

5

Page 42: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 41

Conclusion

How about data mining and

business process management?

When did process mining

start?

What are the main PM developments

in this century?

How did PM tooling develop

over time? Three key observations

Why is process discovery so

difficult?

What are the main research

challenges?

Page 43: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 42

•  conformance checking to diagnose deviations

•  squeezing reality into the model to do model-based

analysis

#1 Alignments are essential!

Page 44: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 43

#2 Models are like the glasses required to

see and understand event data!

Page 45: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique
Page 46: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 45

Conclusion

How about data mining and

business process management?

When did process mining

start?

What are the main PM developments

in this century?

How did PM tooling develop

over time? Three key observations

Why is process discovery so

difficult?

What are the main research

challenges?

Page 47: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 46

Finding

sheep with

five legs

we are getting close…

Page 48: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 47

Distributing

process

mining

problems to

cope with

big data

Page 49: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 48

On-the-fly

process mining

Operational

support

Page 50: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Concept drift

PAGE 49

Concept drift

Page 51: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

Cross-organizational mining

PAGE 50

cross-organizational /

comparative process mining

Page 52: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 51

context aware

process mining

Page 53: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 52

Supporting the process

of process mining

Page 54: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique

PAGE 53

Conclusion

How about data mining and

business process management?

When did process mining

start?

What are the main PM developments

in this century?

How did PM tooling develop

over time? Three key observations

Why is process discovery so

difficult?

What are the main research

challenges?

Page 55: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique
Page 56: process-mining-camp2013-history-of-process-mining · business process reengineering Skip Ellis, Office Talk, 1979 Michael Zisman, SCOOP, 1977 Anatol Holt, ... • There is a unique