A benchmark evaluation for incremental pattern matching in graph transformation

Budapest University of Technology and Economics

Fault-tolerant Systems Research Group

A Benchmark Evaluation for Incremental Pattern Matching in

Graph Transformation

Gábor Bergmann, Ákos Horváth,

István Ráth, Dániel Varró

2Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

Talk Overview

IntroductionGT &

incremental overview

Case studies Measurements

3Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

Benchmarking

Aim:− systematic and reproducible measurements− on performance− under different and precisely defined circumstances

Overall goal:− help transformation engineers in selecting tools− Serve as reference for future research

Popular approach in different fields− AI− relational databases− rule-based expert systems

4Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

Benchmarking in graph transformation

Specification examples for GT− Goal: assessing expressiveness− UML-to-XMI, object-relational mapping,

UML-to-EJB, etc. „Generic” Performance benchmarks for GT

− Varró benchmark − R. Geiß and M. Kroll: On Improvements of the Varró

Benchmark for Graph Transformation Tools− (Agtive Tool Contest, Grabats ’08, …)

In our paper:Benchmarks for graph transformation with incremental pattern matching− simulation− synchronization

5Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

Talk overview

IntroductionGT &

incremental overview

Case Studies Measurements

6Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

Metamodel

Metamodeling

tkn3:tokens

a4:outarc

a1:inarc a2:outarc

a3:inarc

inarc

outarc 11

* *

At most one

Arbitrary

p1:Place p3:Place

t2:Transition

t1:Transition

to1:Token

Transition

Place

Object

Link

Slot Instance model

Class

Association

Multiplicity constraint

Token

1

*

tokens

7Budapest University of Technology and Economics

Fault-tolerant Systems Research Group

Phases of GT matching– Pattern Matching phase– Updating phase: delete+ create

LHS RHS

Place

Token

Tran. Placea1:inarc a2:outarc

Place

Token

Tran. Plana1:inarc a2:outarc

ttn1:tokens tkn2:tokens

matching

Graph Transformation

updating

Pattern Matching is the most critical issue from performance viewpoint

8Budapest University of Technology and Economics

Fault-tolerant Systems Research Group

Incremental Pattern Matching

Precondition of GT rules pattern matchingGoal

− Store matching sets− Incremental update− Fast response

Good performance expected when: − frequent pattern matching− Small updates

Possible application domain− E.g. synchronization, constraints, model simulation, etc.

An adapted RETE algorithm

9Budapest University of Technology and Economics

Fault-tolerant Systems Research Group

p2

p1

p3

INPUT

• RETE net– nodes: intermediate matchings– edge: update propagation

• Example – input: Petri net– pattern: firable transition– update: new transition

Place

t2

TransitionToken

t1

k2k1

k2k1

p1p2 t2t1

p2, k2p1, k1

p2, k2, t3

p1, k1, t1, p3

t3t3

t3t3 t3

p1p2p3 t2t1

t3

t3

p1, k1, t1

p2, k2, t2, p3

p3

t3

p2, k2, t3

Incremental Pattern Matching Example

10Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

Talk overview

IntroductionGT &

incremental overview

Case Studies Measurements

11Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

Test Cases

Model Simulation− Visual languages− Static graph structure− Small local model

manipulation− ALAP execution

Synchronization− Match reusability− Unidirectional− Complex rules− Batch like execution

Petri Net simulation ORM synchronization

12Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

Petri net Simulation Test Case generation

− Inverse property preservation reduction operation

− Random weighted select− Few tokens− Transition/Place ~ 1.1− Starting from:

Execution− Random fireable transition

selection− 1000 / 1000000 iterations− Measured:

Total execution time Small/ Long

transformation sequence

length

PDmatchings

smallfan-out

smallLHS size

Petri NetParadigm Features

Characteristics

Sequential Transition reduction

13Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

Object-Relational Mapping: Synchronization

orphanTable

C: Class

T : Table

: tablerRef

NEG

OR

A: Association: tableRef

{DEL}

: Package

: Class

: Attribute : Attribute

: Schema

: Table

: Column : Column

:schemaRef

:columnRef

:tableRef

:columnRef

Sync order 1. Orphan schema delete 2. Orphan Tableclass delete 3. Orphan Tableassoc delete 4. Orphan Columnattr. Delete 5. Renaming 6. new Classes/Assocs/Columns

created

14Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

Object-Relational Mapping: Synchronization

Test Case generation− Fully connected graph− N classes − N(N-1) directed association− K attributes

Execution− Phases

1. Generation

2. Build

3. Modification

4. Synchronization

− Measured: Synchronization phase

PD

transformation sequence

length

PDmatchings

mediumfan-out

largeLHS size

ORM Syn.Paradigm Features

Characteristic

Source modification− 1/3 classes deleted− 1/5 associations deletes − ½ attributes renamed− 1 new class added and the fully

connected graph is rebuilt

15Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

Talk overview

IntroductionGT &

incremental overview

Case Studies Measurements

16Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

Environment

Hardware and OS− 2 GHz Core2− 2048 MB RAM− Linux kernel of version 2.6.22− Sun JVM 1.6.0_05 for VIATRA− .Net 3.0 on Windows Vista

Tool related− VIATRA with and GrGen withouth GUI− Standard services of the default distribution

17Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

Results of the Petri Net Simulation

•Constant time!•Predictable runtime•Comparable with compiled LS

18Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

ORM Build and Synchronization

1

10

100

1000

10000

100000

85 705 1925 7600 17025 30200 67800

Model size [N]

Exe

cutio

n tim

e [m

s]

build – RETE

sync – RETE

build – LS

sync – LS

Result of the ORM Synchronization

•Execution time:•Rete: linear•LS: exponential

in number of affected element

19Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

Varró: STS BenchmarkPerformance on STS mutex benchmark

100

1000

10000

100000

1000000

100 200 500 1000 3000 10000 30000

Size of process ring [node count]

Exe

cuti

on

tim

e (m

s)

VIATRA/RETE

VIATRA/LS

GrGen.NET

•Non reusable model elements•Suites better for LS

20Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

Summary Rete based incremental PM

− Increased memory consumption− Fix overhead− Predictable linear scaling for practical problems− Comparable performance with fastest LSs in some cases.

Contribution to Benchmarking of GTs− 2 new benchmarks for assessing incremental PM− Larger models− ALAP execution

Different matching strategies for different− Execution phases− Patterns in a transformation?

In overall the performance of GT tools are satisfying. Benchmark example descriptions, specifications, and

measurement results available at:● http://wiki.eclipse.org/VIATRA2/Benchmarks

21Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08

orphanTable

C: Class

T : Table

: tablerRef

NEG

OR

A: Association: tableRef

{DEL}

: Package

: Class

: Attribute : Attribute

: Schema

: Table

: Column : Column

:schemaRef

:columnRef

:tableRef

:columnRef

22Budapest University of Technology and Economics

Fault-tolerant Systems Research Group ICGT '08