101
Budapest University of Technology and Economics Department of Measurement and Informa<on Systems Budapest University of Technology and Economics Fault Tolerant Systems Research Group A TUTORIAL ON EMFINCQUERY Incremental evalua;on of model queries over EMF models Gábor Bergmann, Ákos Horváth, Ábel Hegedüs, Zoltán Ujhelyi, Balázs Polgár, István Ráth, Dániel Varró

A tutorial on EMF-IncQuery

Embed Size (px)

DESCRIPTION

Tutorial at EC-MFA 2011.

Citation preview

Page 1: A tutorial on EMF-IncQuery

Budapest  University  of  Technology  and  EconomicsDepartment  of  Measurement  and  Informa<on  Systems

Budapest  University  of  Technology  and  EconomicsFault  Tolerant  Systems  Research  Group

A  TUTORIAL  ON  EMF-­‐INCQUERYIncremental  evalua;on  of  model  queries  over  EMF  models

Gábor  Bergmann,  Ákos  Horváth,  Ábel  Hegedüs,  Zoltán  Ujhelyi,  Balázs  Polgár,  

István  Ráth,  Dániel  Varró

Page 2: A tutorial on EMF-IncQuery

Contents§ Mo;va;on

o Model  queries…?o State  of  the  art  of  EMF  model  querieso Goals

§ Technology  overview§ Hands-­‐on  part  1

o Basics  (UML  model  sta;s;cs)§ Break§ Hands-­‐on  part  2

o Advanced  well-­‐formedness  valida;on  for  UMLo Integra;on  with  Papyrus  UML

§ Conclusiono Performance  benchmarkso Q&A

§ Feel  free  to  ask  ques4ons  along  the  way!

Page 3: A tutorial on EMF-IncQuery

Downloads§ h_p://viatra.inf.mit.bme.hu/incquery/events/ecmfa2011#Files§ What  you’ll  need

o Eclipse  Helios  (Modeling)• Make  sure  to  allocate  enough  heap  by  passing  “-­‐Xmx1024m”  through  eclipse.ini

o Op;onally:  Papyrus  UMLh_p://www.eclipse.org/modeling/mdt/papyrus/updates/index.php  

o Everything  from  the  update  site  h_p://viatra.inf.mit.bme.hu/update/ecmfa11/  

o In  your  workspace:• Sample  IncQuery  project  h_p://viatra.inf.mit.bme.hu/downloads/demo/ecmfa11/ecmfa-­‐incquery-­‐project.zip  

• Sample  Papyrus  UML  models  h_p://viatra.inf.mit.bme.hu/downloads/demo/ecmfa11/ecmfa-­‐umlsamples-­‐project.zip  

Page 4: A tutorial on EMF-IncQuery

MOTIVATION

While  you’re  downloading  J

Page 5: A tutorial on EMF-IncQuery

First  of  all…§ What  is  a  model  query?

o A  piece  of  code  that  looks  for  certain  parts  of  the  model.

§ “Mathema;cally”o Query  =  set  of  constraints  that  have  to  be  sa;sfied  by  (parts  of)  the  model.

o Result  =  set  of  model  elements  (element  configura;ons)  that  sa;sfy  the  constraints  of  the  query.

§ A  query  engine?o Supports  the  defini;on/execu;on  of  model  queries.

Page 6: A tutorial on EMF-IncQuery

Hi  Jane,  what  do  you  do  at  work?

Jane

Page 7: A tutorial on EMF-IncQuery

Hi  Jane,  what  do  you  do  at  work?

Jane

Page 8: A tutorial on EMF-IncQuery

Hi  Jane,  what  do  you  do  at  work?

Jane

Boss

Page 9: A tutorial on EMF-IncQuery

Hi  Jane,  what  do  you  do  at  work?

Jane

Boss

Detect

Constraint

Page 10: A tutorial on EMF-IncQuery

Hi  Jane,  what  do  you  do  at  work?

Jane

Boss

Detect

Report

Constraint

Page 11: A tutorial on EMF-IncQuery

Hi  Jane,  what  do  you  do  at  work?

Jane

Boss

Detect

ReportView

Constraint

Page 12: A tutorial on EMF-IncQuery

Hi  Jane,  what  do  you  do  at  work?

Jane

Boss

Detect

ReportView

Constraint

Gen

Page 13: A tutorial on EMF-IncQuery

Model  queries§ Queries  are  at  the  heart  of  MDD.

o Viewso Reportso Generatorso Validatorso…

§ Development  issueso Complex  queries  are  hard  to  write

Page 14: A tutorial on EMF-IncQuery

Issues  with  query  development§ Hard  to  write?§ Your  op;ons

o Java  (or  C/C++,  C#,  …)o Declara;ve  languages  (OCL,  EMF  Query  1-­‐2,  …)

Impera;ve  query  languages Declara;ve  query  languages

Expressive  power L  (you  write  lots  of  code) J (very  concise)

Safety JJ (precise  control  over  what  happens  at  execu;on)

JL(unintended  side-­‐effects)

Learning  curve J (you  already  know  it) L (may  be  difficult  to  learn)

Reusability J (standard  OO  prac;ces) LL (???)

Performance LJ (considerable  manual  op;miza;on  necessary)

JL (depends  on  various  factors)

Page 15: A tutorial on EMF-IncQuery

Issues  with  query  execu;on§ Query  performance

o =  Execu;on  ;me,  as  a  func;on  of• Query  complexity• Model  size  /  contents• Result  set  size

§ Incrementalityo Don’t  forget  previously  computed  results!oModels  changes  are  usually  small,  yet  up-­‐to-­‐date  query  results  are  needed  all  the  ;me.

o Incremental  evalua;on  is  an  essen;al,  but  not  a  very  well  supported  feature.

Page 16: A tutorial on EMF-IncQuery

Model  query  engine  wish  list§ Declara;ve  query  language

o Easy  to  learno Good  bindings  to  the  impera;ve  world  (Java)o Safe  yet  powerfulo Reusable

§ High  performanceo Fast  execu;on  for  complex  queries  over  large  modelso First-­‐class  support  for  incremental  execu;on

§ TechnologyoWorks  with  EMF  out-­‐of-­‐the-­‐box

Page 17: A tutorial on EMF-IncQuery

STATE  OF  THE  ART

Page 18: A tutorial on EMF-IncQuery

Problem  1:  Expressiveness§ EMF  Query  (declara;ve)

o Low  expressivenesso Limited  navigability• no  „cycles”

§ OCL  (declara;ve)o Verbose  o Lack  of  reusability  supporto Local  constraints  of  a  model  element

o Poor  handling  of  recursionàChallenging  to  use

Page 19: A tutorial on EMF-IncQuery

Problem  2:  Incrementality§ Goal:  Incremental  evala;on  of  model  queries

o Incremental  maintenance  of  result  seto Avoid  unnecessary  re-­‐computa;on

§ Related  work:  o Constraint  evalua;on  (by  A.  Egyed)

• Arbitrary  constraint  descrip;on– Can  be  a  bo_leneck  for  complex  constraints– Always  local  to  a  model  element

• Listen  to  model  no;fica;ons  • Calculate  which  constraints  need  to  be  reevaluated

o No  other  related  technology  directly  over  EMFo Research  MT  tools:  with  varying  degrees  of  support

Page 20: A tutorial on EMF-IncQuery

Problem  3:  Performance§ Na;ve  EMF  queries  (Java  program  code):  Lack  of  o Reverse  naviga;on  along  referenceso Enumera;on  of  all  instances  by  typeo Smart  Caching

§ Scalability  of  (academic)  MT  toolso Queries  over  >100K  model  elements  (several  proofs):  FUJABA,  VIATRA2  (Java),  GrGEN,  VMTS  (.NET),  Egyed’s  tools

Page 21: A tutorial on EMF-IncQuery

EMF-­‐IncQuery§ Expressive  declara;ve  query  language  by  graph  pa_erns

§ Incremental  cache  of  matches  (materialized  view)

§ High  performance  for  large  models

Page 22: A tutorial on EMF-IncQuery

INCQUERY  TECHNOLOGY  OVERVIEW

Page 23: A tutorial on EMF-IncQuery

Technology  Overview§ What  is  EMF-­‐INCQuery?

o Query  language  +  incremental  pa_ern  matcher  +  development  tools  for  EMF  models• Works  with  any  (pure)  EMF  applica;on• Reusability  by  pa_ern  composi;on• Arbitrary  recursion,  nega;on• Generic  and  parameterized  model  queries• Bidirec;onal  navigability• Immediate  access  to  all  instances  of  a  type• Complex  change  detec;on

§ Benefitso Fully  declara;ve  +  Scalable  performance

Page 24: A tutorial on EMF-IncQuery

Contribu;ons§ Expressive  declara;ve  query  language  by  graph  pa_erns

o Capture  local  +  global  querieso Composi;onality  +  Reusabilility  o „Arbitrary”  Recursion,  Nega;on

§ Incremental  cache  of  matches  (materialized  view)

§ High  performance  for  large  models

Page 25: A tutorial on EMF-IncQuery

Origins  of  the  idea§ Model  transforma;ons  by  VIATRA2

o Transforma;on  language• Declara;ve  pa_erns  for  model  queries• Graph  transforma;on  rules  for  elementary  mapping  specifica;ons  • ASM  rules  for  control  structure

o Matching  strategy• Local  search-­‐based  (op;mized  search  plans)• Incremental  pa@ern  matching  (based  on  RETE)• Hybrid  pa_ern  matching  (adap;ve  combina;on  of  INC  and  LS)

§ Developed  by  BUTE  and  OptXware  Ltd.§ More  info

o h_p://eclipse.org/gmt/VIATRA2o h_p://wiki.eclipse.org/VIATRA2

Page 26: A tutorial on EMF-IncQuery

ECore  models  as  graphs§ Ecore

o EClasso EReferenceo EA_ribute

§ VIATRA2o En;ty  (Node)o Rela;on  (Edge)o Node+Edge

Car:  EClass

Part:  EClass+weight:  EInt

parts:  EReference

MyCar:  Carweight  =  1500parts  =  [MyWheel:Part]

MyCar:  CarMyWheel:  Part

:parts

:  Integervalue  =    1500

:weight

Car

Part

parts

Integerweight

Page 27: A tutorial on EMF-IncQuery

implementssuperClass

Class

Iface

UI  Model

Bu_on ImageBu_on

GComp Cmd

C1 C2

S

Bu_on:Class ImageBu_on:Class

GComp:Class Cmd:Iface

s1:superClass

s1:implement

s2:included

i2:implement:  Iface

C1:  Class C2:  Class

S:  Class

S1:superClass S2:superClass

pa_ern  siblingClass(C1,C2)

Instance  Model

Graphical  nota;onPa_ern  defini;on

matching§ Graph  Pa_ern:  

o Structural  condi;on  that  have  to  be  fulfilled  by  a  part  of  the  model  space

§ Graph  pa_ern  matching:o A  model  (i.e.  part  of  the  model  space)  can  

sa;sfy  a  graph  pa_ern,  o if  the  pa_ern  can  be  matched  to  a  

subgraph  of  the  model  

Page 28: A tutorial on EMF-IncQuery

Graph  pa_erns  (VTCL)// B is a sibling class of A pattern siblingClass(A, B) = { Class(A); Class.superClass(S1, A, P); Class(P); Class.superClass(S2, B, P); Class(B);

A:  Class B:  Class

P:  Class

s1:superClass s2:superClass

siblingClass(A,B)

Page 29: A tutorial on EMF-IncQuery

Graph  pa_erns  (VTCL)// B is a sibling class of A pattern siblingClass(A, B) = { Class(A); Class.superClass(S1, A, P); Class(P); Class.superClass(S2, B, P); Class(B);

OR  pa_ern

A:  Class B:  Class

P:  Class

s1:superClass s2:superClass

siblingClass(A,B)

A:  Class S:Class

X:  Iface

I1:implements I2:implements

locallySubs  (A,  S,  X)

OR

A:  Class S:Class

X:  Class

s1:superClass s2:  superClass

// S is locally substitutable by A pattern localSubs(A, S, X) = {

Class(A); Iface(X);

Class.implements(I1, A, X); Class(S);

Class.implements(I2, S, X);} or {

Class(A); Class(X);

Class.superClass(P1, X, X); Class(S);

OR  pa_ern

Page 30: A tutorial on EMF-IncQuery

Contribu;ons§ Expressive  declara;ve  query  language  by  graph  pa_erns

o Capture  local  +  global  querieso Composi;onality  +  Reusabilility  o „Arbitrary”  Recursion,  Nega;on

§ Incremental  cache  of  matches  (materialized  view)o Cheap  maintenance  of  cache  (only  memory  overhead)o No;fy  about  relevant  changes  (new  match  –  lost  match)o Enable  reac;ons  to  complex  structural  events

§ High  performance  for  large  models

Page 31: A tutorial on EMF-IncQuery

RETE  nets

INPUT

 T  :  typeC  :  Class

TE:  TypedElement

INPUTTE:  TypedElement

INPUTC  :  Class

JOINC  :  Class

TE:  TypedElement

 T  :  type

ANTI-­‐JOIN

NEG

 T  :  typeC  :  Class

TE:  TypedElement

UnusedClass(C)

§ RETE  networko node:  (par;al)  matches  

of  a  (sub)pa_erno edge:  update  

propaga;on

§ Demonstra;ono input:  UML  modelo pa_ern:  UnusedClasso change:  delete/retarget  type  reference

PATTERNUnusedClass(C)

C:  Class

TE:  TypedElement

NEG

T:  ty

pe

Page 32: A tutorial on EMF-IncQuery

RETE  nets

INPUT

 T  :  typeC  :  Class

TE:  TypedElement

INPUTTE:  TypedElement

INPUTC  :  Class

JOINC  :  Class

TE:  TypedElement

 T  :  type

ANTI-­‐JOIN

NEG

 T  :  typeC  :  Class

TE:  TypedElement

UnusedClass(C)

§ RETE  networko node:  (par;al)  matches  

of  a  (sub)pa_erno edge:  update  

propaga;on

§ Demonstra;ono input:  UML  modelo pa_ern:  UnusedClasso change:  delete/retarget  type  reference

PATTERNUnusedClass(C)

C:  Class

TE:  TypedElement

NEG

T:  ty

pe

UnusedClass(C)A  class  to  which  no  type  reference  points•Associa;on•Property•Parameter•Variable

Page 33: A tutorial on EMF-IncQuery

RETE  nets

INPUT

 T  :  typeC  :  Class

TE:  TypedElement

INPUTTE:  TypedElement

INPUTC  :  Class

JOINC  :  Class

TE:  TypedElement

 T  :  type

ANTI-­‐JOIN

NEG

 T  :  typeC  :  Class

TE:  TypedElement

UnusedClass(C)

§ RETE  networko node:  (par;al)  matches  

of  a  (sub)pa_erno edge:  update  

propaga;on

§ Demonstra;ono input:  UML  modelo pa_ern:  UnusedClasso change:  delete/retarget  type  reference

Input  nodes

Intermediate  nodes

In-­‐memory  model(EMF  ResourceSet)

Produc;on  node

PATTERNUnusedClass(C)

C:  Class

TE:  TypedElement

NEG

T:  ty

pe

Page 34: A tutorial on EMF-IncQuery

RETE  nets

INPUT

 T  :  typeC  :  Class

TE:  TypedElement

INPUTTE:  TypedElement

INPUTC  :  Class

JOINC  :  Class

TE:  TypedElement

 T  :  type

ANTI-­‐JOIN

NEG

 T  :  typeC  :  Class

TE:  TypedElement

UnusedClass(C)

§ RETE  networko node:  (par;al)  matches  

of  a  (sub)pa_erno edge:  update  

propaga;on

§ Demonstra;ono input:  UML  modelo pa_ern:  UnusedClasso change:  delete/retarget  type  reference

PATTERNUnusedClass(C)

C:  Class

TE:  TypedElement

NEG

T:  ty

pe

Page 35: A tutorial on EMF-IncQuery

RETE  nets

INPUT

 T  :  typeC  :  Class

TE:  TypedElement

INPUTTE:  TypedElement

INPUTC  :  Class

JOINC  :  Class

TE:  TypedElement

 T  :  type

ANTI-­‐JOIN

NEG

 T  :  typeC  :  Class

TE:  TypedElement

UnusedClass(C)

§ RETE  networko node:  (par;al)  matches  

of  a  (sub)pa_erno edge:  update  

propaga;on

§ Demonstra;ono input:  UML  modelo pa_ern:  UnusedClasso change:  delete/retarget  type  reference

a b c

x z wp s

 Class  objects

TypedElement  objects

TypedElement.type  edges

x z w a b c

p s

p x s z

c

PATTERNUnusedClass(C)

C:  Class

TE:  TypedElement

NEG

T:  ty

pe

Page 36: A tutorial on EMF-IncQuery

RETE  nets

INPUT

 T  :  typeC  :  Class

TE:  TypedElement

INPUTTE:  TypedElement

INPUTC  :  Class

JOINC  :  Class

TE:  TypedElement

 T  :  type

ANTI-­‐JOIN

NEG

 T  :  typeC  :  Class

TE:  TypedElement

UnusedClass(C)

§ RETE  networko node:  (par;al)  matches  

of  a  (sub)pa_erno edge:  update  

propaga;on

§ Demonstra;ono input:  UML  modelo pa_ern:  UnusedClasso change:  delete/retarget  type  reference

a b c

x z wp s

 Class  objects

TypedElement  objects

TypedElement.type  edges

x z w a b c

p s

p x s z

c

p

p

p x

PATTERNUnusedClass(C)

C:  Class

TE:  TypedElement

NEG

T:  ty

pe

Page 37: A tutorial on EMF-IncQuery

RETE  nets

INPUT

 T  :  typeC  :  Class

TE:  TypedElement

INPUTTE:  TypedElement

INPUTC  :  Class

JOINC  :  Class

TE:  TypedElement

 T  :  type

ANTI-­‐JOIN

NEG

 T  :  typeC  :  Class

TE:  TypedElement

UnusedClass(C)

§ RETE  networko node:  (par;al)  matches  

of  a  (sub)pa_erno edge:  update  

propaga;on

§ Demonstra;ono input:  UML  modelo pa_ern:  UnusedClasso change:  delete/retarget  type  reference

a b c

x z wp s

 Class  objects

TypedElement  objects

TypedElement.type  edges

x z w a b c

p s

p x s z

c

p

p

p x

a

PATTERNUnusedClass(C)

C:  Class

TE:  TypedElement

NEG

T:  ty

pe

Page 38: A tutorial on EMF-IncQuery

RETE  nets

INPUT

 T  :  typeC  :  Class

TE:  TypedElement

INPUTTE:  TypedElement

INPUTC  :  Class

JOINC  :  Class

TE:  TypedElement

 T  :  type

ANTI-­‐JOIN

NEG

 T  :  typeC  :  Class

TE:  TypedElement

UnusedClass(C)

§ RETE  networko node:  (par;al)  matches  

of  a  (sub)pa_erno edge:  update  

propaga;on

§ Demonstra;ono input:  UML  modelo pa_ern:  UnusedClasso change:  delete/retarget  type  reference

a b c

x z wp s

 Class  objects

TypedElement  objects

TypedElement.type  edges

x z w a b c

p s

p x s z

c

p

p

p x

a

PATTERNUnusedClass(C)

C:  Class

TE:  TypedElement

NEG

T:  ty

peNo4fica4on

Transparent:  user  modifica;on,  model  imports,  results  of  a  transforma;on,  external  modifica;on,  …  à RETE  is  always  updated!

Page 39: A tutorial on EMF-IncQuery

RETE  nets

INPUT

 T  :  typeC  :  Class

TE:  TypedElement

INPUTTE:  TypedElement

INPUTC  :  Class

JOINC  :  Class

TE:  TypedElement

 T  :  type

ANTI-­‐JOIN

NEG

 T  :  typeC  :  Class

TE:  TypedElement

UnusedClass(C)

§ RETE  networko node:  (par;al)  matches  

of  a  (sub)pa_erno edge:  update  

propaga;on

§ Demonstra;ono input:  UML  modelo pa_ern:  UnusedClasso change:  delete/retarget  type  reference

a b c

x z wp s

 Class  objects

TypedElement  objects

TypedElement.type  edges

x z w a b c

p s

p x s z

c

p

p

p x

a

PATTERNUnusedClass(C)

C:  Class

TE:  TypedElement

NEG

T:  ty

peNo4fica4on

Transparent:  user  modifica;on,  model  imports,  results  of  a  transforma;on,  external  modifica;on,  …  à RETE  is  always  updated!

Experimental  results:good,  if…o There  is  enough  

memoryo Transac;onal  model  

manipula;on

Page 40: A tutorial on EMF-IncQuery

EMF-­‐INCQUERY  Architecture

RETE  Core

EMF  INC  PM  Core VIATRA2  INC  PM

EMF  App

Pa_ern/Query  spec

Generated  PM  and  RETE  builder

Page 41: A tutorial on EMF-IncQuery

EMF-­‐INCQUERY  Tooling§ Genera;ve  approach

o compile  queries  into  • RETE  network  builders• Query  interface  wrappers

o end-­‐to-­‐end  solu;on:  generates  Eclipse  plug-­‐in  projects

§ Relies  on  the  VIATRA2  environmento query  developmento debugging  

Page 42: A tutorial on EMF-IncQuery

Tooling  architecture

VIATRA2  model  representa;on

Declara;ve  Queries  in  VIATRA2

Generated  Query  Wrappers

EMF  Model1

2 4

3

EMF  App

Development  ;me Run;me

Page 43: A tutorial on EMF-IncQuery

Development  workflow

Import  EMF  domain  into  VIATRA2

Develop  and  test  queries  over  VIATRA2

Generate  INCQuery  code

Integrate  into  EMF  applica;on

Page 44: A tutorial on EMF-IncQuery

Development  workflow

Import  EMF  domain  into  VIATRA2

Develop  and  test  queries  over  VIATRA2

Generate  INCQuery  code

Integrate  into  EMF  applica;on

AutomatedAutomated

Page 45: A tutorial on EMF-IncQuery

Development  workflow

Import  EMF  domain  into  VIATRA2

Develop  and  test  queries  over  VIATRA2

Generate  INCQuery  code

Integrate  into  EMF  applica;on

AutomatedAutomated

Supported  by  the  VIATRA2  

IDE

Page 46: A tutorial on EMF-IncQuery

Development  workflow

Import  EMF  domain  into  VIATRA2

Develop  and  test  queries  over  VIATRA2

Generate  INCQuery  code

Integrate  into  EMF  applica;on

AutomatedAutomated

Supported  by  the  VIATRA2  

IDE

Semi-­‐automated  for  typical  scenarios,

 some  manual  coding

Page 47: A tutorial on EMF-IncQuery

PART  1  -­‐  BASICS

Page 49: A tutorial on EMF-IncQuery

Model  measures§ Total  number  of

o Packageso Classeso A_ributes

§ Number  ofo Childreno A_ributeso A_ributes  inherited

§ …

Page 50: A tutorial on EMF-IncQuery

Solu;on:  Development  workflow

Import  EMF  domain  into  VIATRA2

Develop  and  test  queries  over  VIATRA2

Generate  INCQuery  code

Integrate  into  EMF  applica;on

Page 51: A tutorial on EMF-IncQuery

DEMO§ Create  IncQuery  project§ Import  the  UML  metamodel§ Import  UML  instance  models  § VTCL  development  environment

o Create  and  debug  pa_ernso Visualize  pa_ern  matcheso Test  queries/pa_erns  in  transforma;onso Create  pa_erns  using  instance  model  “examples”

Pa_ern  development  in  VIATRA2

Page 52: A tutorial on EMF-IncQuery

EXAMPLE Our  first  IncQueries

Page 53: A tutorial on EMF-IncQuery

DEMO§ Generate

o EMF-­‐IncQuery  Source  code:  Pa_ern  matcherso Sample  UI  project:  UI  integra;on

§ Run  Sample  UI  project  commands  to  see  model  measures  

Genera;ng  IncQuery  code

Page 54: A tutorial on EMF-IncQuery

Generated  pa_ern  matchers§ INCQuery  run;me

o Eclipse  plugin• Depends  only  on  EMF  and  the  INCQuery  core• No  VIATRA2  dependency!  

o Private  code:  pa_ern  builders• Parameterize  the  RETE  core  and  the  generic  EMF  PM  library

o Public  API:  Pa_ern  matcher  access  layer• Query  interfaces• Data  Transfer  Objects  (DTOs)• Used  to  integrate  to  EMF  applica;ons  

Page 55: A tutorial on EMF-IncQuery

Generated  Sample  UI§ Command  contribu;ons

o Project  explorer,  Naviga;on,  Package  Explorero Perform  model  loading  and  query  execu;ono Display  the  results  on  the  UI• List  (default)

– Pre_y  prints  a  list  of  matches

• Counter– Prints  the  number  of  matches

Page 56: A tutorial on EMF-IncQuery

IncQuery  Run;me

Generic  Query  API

Generated  Change  API

Generated  Query  API

Generic  Change  API

Page 57: A tutorial on EMF-IncQuery

Generated  Query  API§ Basic  queries

o Uses  tuples  (object  arrays)  corresponding  to  pa_ern  parameterso Object[] getOneMatch()o Collection<Object[]> getAllMatches()

§ Parameterized  querieso getOneMatch(Object X, Object Y, …)o getAllMatches(Object X, Object Y, …)o Null  input  values  =  unbound  input  variables

Based  on  pa_ern  signature

Page 58: A tutorial on EMF-IncQuery

Query  Signatures§ Data  Transfer  Objects  generated  

for  pa_ern  signatures

§ Signature  query  methodso SiblingClassSignature getOneMatch()o SiblingClassSignature

getOneMatchAsSignature(Object C1, Object C2)

o Collection<SiblingClassSignature> getAllMatchesAsSignature()

o Collection<SiblingClassSignature> getAllMatchesAsSignature(Object C1, Object C2)

§ C1,  C2:  EObjects  or  datatype  instances  (String,  Boolean,  …)

public class SiblingClassSignature{ Object C1; Object C2;}

// B is a sibling class of A pattern siblingClass(C1, C2) = { /* contents */ }

Page 59: A tutorial on EMF-IncQuery

Query  Signatures

Ongoing  work:  use  domain-­‐specific  types  in  generated  code

public class SiblingClassSignature{ UMLClass C1; UMLClass C2;}

§ Data  Transfer  Objects  generated  for  pa_ern  signatures

§ Signature  query  methodso SiblingClassSignature

getOneMatchAsSignature()o SiblingClassSignature

getOneMatchAsSignature(UMLClass C1, UMLClass C2)

o Collection<SiblingClassSignature> getAllMatchesAsSignature()

o Collection<SiblingClassSignature> getAllMatchesAsSignature(UMLClass C1, UMLClass C2)

§ C1,  C2:  EObjects  or  datatype  instances  (String,  Boolean,  …)

// B is a sibling class of A pattern siblingClass(C1, C2) = { /* contents */ }

Page 60: A tutorial on EMF-IncQuery

Integra;ng  into  EMF  applica;ons§ Pa_ern  matchers  may  be  ini;alized  for

o Any  EMF  No;fier• e.g.  Resources,  ResourceSets

o (Transac;onalEdi;ngDomains)

§ Ini;aliza;ono Possible  at  any  ;meo Involves  a  single  exhaus;ve  model  traversal  (independent  of  the  number  of  pa_erns,  pa_ern  contents  etc.)

Page 61: A tutorial on EMF-IncQuery

Typical  programming  pa_erns1. Ini;alize  EMF  model

o Usually  already  done  by  your  app  J

2. Ini;alize  incremental  PM  whenever  necessaryo Typically:  at  loading  ;me

3. Use  the  incremental  PM  for  querieso Model  updates  will  be  processed  transparently,  with  minimal  

performance  overheado Delta  monitors  can  be  used  to  track  complex  changes

4. Dispose  the  PM  when  not  needed  anymoreo +  Frees  memoryo -­‐  Re-­‐traversal  will  be  necessary

Page 62: A tutorial on EMF-IncQuery

PART  2  –  VALIDATION  CASE  STUDY

Page 63: A tutorial on EMF-IncQuery

Example  2§ Inspired  by  the  “Verifying  UML  profiled  models”  ATL  case  studyo h_p://www.eclipse.org/m2m/atl/usecases/Verifying_UML_profiled_models/  

§ Well-­‐formedness  checking  of  UML  models,  with  a  crucial  difference:o Incremental,  on-­‐the-­‐fly  valida;onoMaintain  error  markers  as  the  user  is  edi;ng  the  model

Page 64: A tutorial on EMF-IncQuery

UML  well-­‐formedness  rules§ Tradi;onally  specified  by  OCL  constraints

o Note:  OCL  constraints  included  in  UML  models  serve  a  different  purpose

o OCL  constraints  can  be  a_ached  to  any  EMF  instance  model  via  EMF  Valida;on

§ Rules  specified  byo Tool  developerso (End  users)

Page 65: A tutorial on EMF-IncQuery

Well-­‐formedness  checking  from  a  tool  developer’s  perspec;ve

§ Well-­‐formedness  ruleso Express  constraints  not  (easily)  possible  by  metamodeling  techniques

o Ensure  “sane”  modeling  conven;ons  &  best  prac;ceso Aid  code  genera;on  by  design-­‐;me  valida;on

§ Example

Page 66: A tutorial on EMF-IncQuery

Well-­‐formedness  checking  from  a  tool  developer’s  perspec;ve

§ Well-­‐formedness  ruleso Express  constraints  not  (easily)  possible  by  metamodeling  techniques

o Ensure  “sane”  modeling  conven;ons  &  best  prac;ceso Aid  code  genera;on  by  design-­‐;me  valida;on

§ Example

Page 67: A tutorial on EMF-IncQuery

Well-­‐formedness  checking  from  a  tool  developer’s  perspec;ve

§ Well-­‐formedness  ruleso Express  constraints  not  (easily)  possible  by  metamodeling  techniques

o Ensure  “sane”  modeling  conven;ons  &  best  prac;ceso Aid  code  genera;on  by  design-­‐;me  valida;on

§ Example 2

1

The  number  of  parameters  of  a  Behavior’s  

specifica;on  should  match  with  the  defini;on  of  the  

Behavior  itself.

Page 68: A tutorial on EMF-IncQuery

IncQuery  Valida;on  Engine§ Simple  valida;on  engine

o Supports  on-­‐the-­‐fly  valida;on  through  incremental  pa_ern  matching  and  problem  marker  management

o Uses  IncQuery  graph  pa_erns  to  specify  constraints

§ Simulates  EMF  Valida;on  markerso To  ensure  compa;bility  and  easy  integra;on  with  exis;ng  editors

o Doesn’t  use  EMF  Valida;on  directly• Execu;on  model  is  different

Page 69: A tutorial on EMF-IncQuery

How  it  works  –  IncQuery  Change  API§ Track  changes  in  the  match  set  of  pa_erns  (new/lost)§ Delta  monitors

o May  be  ini;alized  at  any  ;meo DeltaMonitor.matchFoundEvents / DeltaMonitor.matchLostEvents• Queues  of  matches  (tuples/Signatures)  that  have  appeared/disappeared  since  ini;aliza;on

§ Typical  usageo Listen  to  model  manipula;on  (transac;ons)o A}er  transac;on  commits:

• Evaluate  delta  monitor  contents  and  process  changes• Remove  processed    tuples/Signatures  from  .matchFound/LostEvents

Page 70: A tutorial on EMF-IncQuery

Well-­‐formedness  rule  specifica;on  by  graph  pa_erns

§ WFRs:  Invariants  which  must  hold  at  all  ;mes§ Specifica;on  =  set  of  elementary  constraints  +  contexto Elementary  constraints:  Query  (pa_ern)o Loca;on/context:  a  model  element  on  which  the  problem  marker  will  be  placed

§ Constraints  by  graph  pa_ernso Define  a  pa_ern  for  the  “bad  case”• Either  directly• Or  by  nega;ng  the  defini;on  of  the  “good  case”

o Assign  one  of  the  variables  as  the  loca;on/context

Match:  A  viola;on  of  the  invariant

Page 71: A tutorial on EMF-IncQuery

EXAMPLE

§ “All  Behaviors  must  have  an  Opera;on  as  their  specifica;on.”o Otherwise  they  do  not  have  any  

“interface”  through  which  they  could  be  accessed  à  “dead  code”  

§ Bad  case:

A  simple  UML  valida4on  constraint

Page 72: A tutorial on EMF-IncQuery

EXAMPLE

§ “All  Behaviors  must  have  an  Opera;on  as  their  specifica;on.”o Otherwise  they  do  not  have  any  

“interface”  through  which  they  could  be  accessed  à  “dead  code”  

§ Bad  case:

A  simple  UML  valida4on  constraint

Iden;fy  pa_ern  variable  “Behavior”  as  

the  loca;on

Page 73: A tutorial on EMF-IncQuery

EXAMPLE

§ “All  Behaviors  must  have  an  Opera;on  as  their  specifica;on.”o Otherwise  they  do  not  have  any  

“interface”  through  which  they  could  be  accessed  à  “dead  code”  

§ Bad  case:

A  simple  UML  valida4on  constraint

• Subs;tute  the  model  element  match  to  “Behavior”.• Syntax:  $Behavior.featureName$• $Behavior$:  shortcut  for  $Behavior.name$

Iden;fy  pa_ern  variable  “Behavior”  as  

the  loca;on

Page 74: A tutorial on EMF-IncQuery

DEMO§ Generate  sample  valida;on  project§ Observe  on-­‐the-­‐fly  valida;on  in  ac;on

Constraint  code  genera;on

Page 75: A tutorial on EMF-IncQuery

Generated  Sample  Valida;on  project§ Java  classes:  Constraint  descendants§ Plugin.xml

• Constraint  registra;on• UI  integra;on

– Editor  ID  from  genmodel

Page 76: A tutorial on EMF-IncQuery

Valida;on  lifecycle§ Constraint  viola;ons

o Represented  by  Problem  Markers  (Problems  view)oMarker  text  is  updated  if  affected  elements  are  changed  in  the  model

oMarker  removed  if  viola;on  is  no  longer  present

§ Lifecycleo Editor  bound  valida;on  (markers  removed  when  editor  is  closed)

o Incremental  maintenance  not  prac;cal  outside  of  a  running  editor

Page 77: A tutorial on EMF-IncQuery

Valida;on  UI  integra;on§ A  menu  item  (command)  to  start  the  valida;on  engine

§ Generic  (part  of  the  IncQuery  Valida;on  framework)o GMF  editor  command• Appears  in  all  GMF-­‐based  editor’s  context  menu

o Sample  Reflec;ve  Editor  command• Appears  on  the  toolbar

§ Generatedo EMF  generated  tree  editor  command• Appears  on  the  toolbar

Page 78: A tutorial on EMF-IncQuery

EXAMPLE

§ “Unused  signal”o A  Class  defines  a  Recep;on  for  a  Signal,  but  it  doesn’t  use  it  in  its  StateMachine  for  triggering  a  transi;on.

§ Good  case:

A  complex  UML  constraint

Page 79: A tutorial on EMF-IncQuery

EXAMPLE

§ “All  Recep;ons  of  a  Class  must  use  Signals  that  are  used  in  the  defining  Class’  StateMachine  as  a  Transi;on  Trigger.”

§ Bad  case:

A  complex  UML  constraint

Page 80: A tutorial on EMF-IncQuery

EXAMPLE

§ “All  Recep;ons  of  a  Class  must  use  Signals  that  are  used  in  the  defining  Class’  StateMachine  as  a  Transi;on  Trigger.”

§ Bad  case:

A  complex  UML  constraint

below  expresses  a  transi;ve  

containment  rela;onship.

Page 81: A tutorial on EMF-IncQuery

EXAMPLE

§ “All  Recep;ons  of  a  Class  must  use  Signals  that  are  used  in  the  defining  Class’  StateMachine  as  a  Transi;on  Trigger.”

A  complex  UML  constraint

BELOW

Page 82: A tutorial on EMF-IncQuery

EXAMPLE

§ “All  Recep;ons  of  a  Class  must  use  Signals  that  are  used  in  the  defining  Class’  StateMachine  as  a  Transi;on  Trigger.”

A  complex  UML  constraint

BELOW

Transi;ons  can  be  located  in  any  Region  of  the  

StateMachine.

Page 83: A tutorial on EMF-IncQuery

DEMO§ Generate  Sample  Valida;on  code§ Load  modelGenera;onExample_2000.uml§ Observe  on-­‐the-­‐fly  valida;on

o Performance  is  capped  by  the  Eclipse  Marker  management  layer.

On-­‐the-­‐fly  valida;on  on  a  large  model

Page 84: A tutorial on EMF-IncQuery

EXAMPLE

§ The  number  of  parameters  of  a  Behavior’s  specifica;on  Opera;on  should  match  with  the  Behavior’s  defini;on.

§ Bad  case:

Extra  assignment

2

1

Page 85: A tutorial on EMF-IncQuery

EXAMPLE Solu;on

Cardinality  constraints  expressing  the  

“number  of  matches”

Page 86: A tutorial on EMF-IncQuery

PERFORMANCE  BENCHMARKING

Page 87: A tutorial on EMF-IncQuery

Challenges§ Performance  evalua;ons  are  hard

o Fair?o Reliable?o Reproducible?o Can  the  results  be  generalized?

§ Benchmark  example:  on-­‐the-­‐fly  constraint  valida;on  over  AUTOSAR  modelso Conference  presenta;on  at  MODELS  2010o Mo;va;on:  the  Embedded  Architect  Tool  of  OptXware  Ltd.o AUTOSAR  models  can  be  very  large  (>>1m  elements)

Page 88: A tutorial on EMF-IncQuery

What  is  measured?§ Sample  models  were  generated

o matches  are  scarce  rela;ve  to  overall  model  size

§ On-­‐the-­‐fly  valida;on  is  modeled  as  follows:1. Compute  ini;al  valida;on  results2. Apply  randomly  distributed,  small  changes3. Re-­‐compute  valida;on  results

§ Measured:  execu;on  ;mes  ofo Ini;aliza;on  (model  load  +  RETE  construc;on)o Model  manipula;on  opera;ons  (negligible)o Valida;on  result  (re)computa;on

§ Compared  technologieso MDT-­‐OCLo Plain  Java  code  that  an  average  developer  would  write

Page 89: A tutorial on EMF-IncQuery

IncQuery  Results§ Hardware:  normal  desktop  PC  (Core2,  4GB  RAM)§ Model  sizes  up  to  1.5m  elements§ Ini;aliza;on  ;mes  (resource  loading  +  first  valida;on)

o <1  sec  for  model  sizes  below  50000  elementso Up  to  40  seconds  for  the  largest  model  (grows  linearly  with  the  model  size)

§ Recomputa;on  ;meso Within  error  of  measurement  (=0),  independent  of  model  sizeo Retrieval  of  matches  AND  complex  changes  is  instantaneous

§ Memory  overheado <50  MB  for  model  sizes  below  50000  elementso Up  to  1GB  for  the  largest  model  (grows  linearly  with  model  size)

§ How  does  it  compare  to  na;ve  code  /  OCL?

Page 90: A tutorial on EMF-IncQuery

Ini;aliza;on  ;me

Page 91: A tutorial on EMF-IncQuery

Ini;aliza;on  ;me

•  Includes  ;me  for  first  valida;on•  Linear  func;on  of  model  size,  orders  of  magnitude  faster

Page 92: A tutorial on EMF-IncQuery

Recomputa;on  ;me

Page 93: A tutorial on EMF-IncQuery

Recomputa;on  ;me

Recomputa;on  ;me  is  uniformly  near  

zero(independent  of  model  size)

Page 94: A tutorial on EMF-IncQuery

Performance  overview

Page 95: A tutorial on EMF-IncQuery

Assessment  of  the  benchmark§ High  performance  complex  queries  are  hard  to  write  manually  in  Java.

§ IncQuery  can  do  the  trick  nicely  as  long  as  you  have  enough  RAM.

§ Metamodel  structure  has  huge  impact  on  performance  when  using  “conven;onal”  query  technologies  such  as  OCL,  due  too Lack  of  reverse  naviga;ono Lack  of  type  enumera;on  (.allInstances())

Page 96: A tutorial on EMF-IncQuery

Contribu;ons§ Expressive  declara;ve  query  language  by  graph  pa_erns

o Capture  local  +  global  querieso Composi;onality  +  Reusabilility  o „Arbitrary”  Recursion,  Nega;on

§ Incremental  cache  of  matches  (materialized  view)o Cheap  maintenance  of  cache  (only  memory  overhead)o No;fy  about  relevant  changeso Enable  reac;ons  to  complex  structural  events

§ High  performance  for  large  modelso Linear  overhead  for  loading  o Instant  response  for  querieso >  1  million  model  elements  (on  a  desktop  PC)

Page 97: A tutorial on EMF-IncQuery

CONCLUSION

Page 98: A tutorial on EMF-IncQuery

Closing  thoughts§ On-­‐the-­‐fly  valida;on  is  only  one  scenario

o Early  model-­‐based  analysis  o Language  engineering  in  graphical  DSMLso Model  execu;on/analysis  (stochas;c  GT)o Tool  integra;on  o Model  op;miza;on  /  constraint  solvingo …

§ The  tutorial  exampleso Do  not  make  use  of  advanced  features  such  as  parameterized  queries  or  complex  structural  constraints  (recursion)

o Are  meant  only  as  a  star;ng  pointo The  project  website  has  many  more  examples!

Page 99: A tutorial on EMF-IncQuery

Model  transforma4ons  based  on  IncQuery§ High  performance  model  transforma;ons

o Hybrid  query  approach• Use  IncQuery  for  

– Complex  queries– Frequently  used  queries– Parameterized  queries

• Plain  Java  for  simple  queries  (saves  RAM)

o Java  for  control  structure  &  model  manipula;on

§ High-­‐level  transforma;on  languages  (VIATRA2,  ATL,  Epsilon,  …)  could  be  “compiled”  to  run  on  this  infrastructure

§ Ongoing  research:  automa;c  mapping  of  SPARQL,  OCL  &  others  to  the  IncQuery  language

Page 100: A tutorial on EMF-IncQuery

Wish  list  IncQuery  features  J§ Declara;ve  query  language

o Easy  to  learno Good  bindings  to  the  impera;ve  world  (Java)o Safe  yet  powerfulo Reusable

§ High  performanceo Fast  execu;on  for  complex  queries  over  large  modelso First-­‐class  support  for  incremental  execu;on

§ TechnologyoWorks  with  EMF  out-­‐of-­‐the-­‐box

Page 101: A tutorial on EMF-IncQuery

Pointers§ Open  source  project:

o h_p://viatra.inf.mit.bme.hu/incquery

§ VIATRA2  (grandfather)o h_p://eclipse.org/gmt/VIATRA2o h_p://viatra.inf.mit.bme.hu

§ Istváno [email protected]

§ The  development  teamo viatra-­‐[email protected] We’re  glad  to  help  with  any  problems  you  might  have.o Check  the  site  for  FAQ,  examples  &  more.

§ Ques;ons  &  answers