131010 jim cordy - submodel pattern extraction for simulink models

Submodel Pattern Extraction for Simulink Models

James R. CordyQueen’s University

NECSIS Automotive Partnership Canada

MPE near-miss clones

near-miss clone detection Simone

analysis of GM models pattern extraction

pattern evolution




pattern evolution

Model Pattern Engineering

discover, catalogue and formalize submodel patterns

emergentdomain-specific, client-specific

discovery analysis, identification methodology, techniques

classification characterization, formalization notation, tooling, catalogues

application deployment, analysis organization, documentation, use cases

why?

reuse in model development

standards/consistency analysis/enforcement

failure/change propagation in model maintenance

verification/test optimization

deployment variation/optimization

model product lines




pattern evolution

code clonescopy-paste programming

efficient, widely usedproblematic

code clonestype 1 - exact

bool ConfNextToken (char **p) { // skip white space while (1) switch (**p) { case '\t' : // ignore case ' ' : (*p)++; break; case '\0' : return FALSE; default : return TRUE; };}


[Roy, Cordy, Koschke SCP 2009]

code clonestype 1 - exact


bool ConfNextToken (char **p) { while (1) switch (**p) { case '\t': case ' ': // just skip (*p)++; break; case '\0': // eof return FALSE; default: // something we want return TRUE; };}


code clonestype 2 - renamed


bool NextToken (char **bp) { while (1) // not really switch (**bp) { case '\t': case ' ': // next (*bp)++; break; case '\0': return 0; default: return 1; };}



code clonestype 3 - near miss

bool NextToken (char **bp){ while (1) // not really switch (**bp) { case '\t': (*bp)++ case ' ': break; case '\0': return 0; default: return 1; } }


model clonestype 1 - exact

Tfmaxs2

Tfmaxk1

TorqueConversion

2/3*R*muk

Ratio of staticto kinetic

mus/muk

Fn1

Tfmaxs2

Tfmaxk1

TorqueConversion

2/3*R*muk

Ratio of staticto kinetic

mus/muk

Fn1

[Alalfi, Cordy, Dean, Stephan, Stevenson ICSM 2012] [Störrle SSM 2013]

model clonestype 2 - renamed


model clonestype 3 - near miss





pattern evolution

ConQATgraph-based model clone detection

[Deissenboeck et al. IWSC 2010]

graph flattening ignores hierarchical structure

problems with near-miss

graph flattening ignores hierarchical structure

problems with near-miss

code-based near-miss works well NiCad, iClones, others

mature, accurate, efficient handles unexpected differences

threshold-based, tunable scalable

NiCadparse - extract - normalize - diff threshold

Pretty-printedPotential Clones

1234

Parsing & Potential Clone

Extraction

OriginalCode Base

1. Parse / Extract 2. Rename / Filter / Normalize

Renaming, Filtering,

Normalization

NormalizedPotential Clones

1234

CloneClasses

5.pc23.pc67.pc . . .

12.pc17.pc22.pc . . .

15.pc18.pc78.pc . . .

21.pc63.pc97.pc . . .

37.pc39.pc44.pc . . .

Choose Next Potential Clone

as Exemplar

3. Clone Analysis

Comparable Size Potential Clone Cluster

Pairwise Comparison

with Exemplar

(Repeat)

Cluster Comparable

Size PCs


1234

Pretty-printedPotential Clones

1234

Parsing & Potential Clone

Extraction

OriginalCode Base

1. Parse / Extract 2. Rename / Filter / Normalize

Renaming, Filtering,

Normalization


1234

CloneClasses

5.pc23.pc67.pc . . .

12.pc17.pc22.pc . . .

15.pc18.pc78.pc . . .

21.pc63.pc97.pc . . .

37.pc39.pc44.pc . . .

Choose Next Potential Clone

as Exemplar

3. Clone Analysis

Comparable Size Potential Clone Cluster

Pairwise Comparison

with Exemplar

(Repeat)

Cluster Comparable

Size PCs


1234

[Roy, Cordy ICPC 2008]

crazy idea: can we use near-miss text code methods on graphical models?

“Models are source code too” Mark Harman, keynote at SCAM 2010

[Harman SCAM 2010]




pattern evolution

Simone

Simulink near-miss clone detectionexperiment

adapt NiCad near-miss code clone detector to graphical modelsvalidate vs. ConQAT

for types 1 & 2hand validate type 3 (near-miss)

[Alalfi, Cordy,Dean, Stephan, Stevenson ICSM 2012]

Simulink

hybrid hardware/software modelswidespread in industry - automotive, aerospace, embedded systems

mature and interesting at GM

Simulink

hierarchical models


Challenge #1 code methods require text

NiCad requires a parserSolution: grammar inference

on Simulink’s internal form

...! System {! Name "onoff"! Location [168, 385, 668, 686]! Open on! ModelBrowserVisibility off! ModelBrowserWidth 200! ScreenColor "automatic"! PaperOrientationi "landscape"! PaperPositionMode "auto"! PaperType "usletter"! PaperUnits "inches"! ZoomFactor "100"! AutoZoom on! ReportName "simulink-default.rpt"! Block {! BlockType DiscretePulseGenerator! Name "Discrete Pulse\nGenerator"! Position [45, 25, 75, 55]! Amplitude "1"! Period "2"! PulseWidth "1"! PhaseDelay "0"! SampleTime "1"! }! Block {! BlockType Product! Name "Product"! Ports [2, 1, 0, 0, 0]! Position [145, 67, 175, 98]! Inputs "2"! SaturateOnIntegerOverflow on! }! ...! }! ...!

Challenge #2 what granularity?

NiCad requires candidates for comparisonSimulink: model (too big) block (too small) system (just right!)


even with raw text, find some subsystem clones

but:90% irrelevant Simulink internal “formatting” systems

some identical systems only 70% sameentirely missed exact copies displayed differently

neutral_up_down1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1

/Users/manaralalfi/Desktop/NiCad 3.2 Simulink/Simulink/Publicmodel/automotive/powerwindow03.mdl

printed 13 Apr 2012 14:56 page 1/1

powerwindow03/power_window_control_system/validate_driver

neutral_up_down1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1




Challenge #3 problems with “noise”

solution: “agile parsing” to filter out irrelevant elements


[Dean, Cordy, Malton, Schneider JASE 2003]

filtering removes more than 300 kinds of irrelevant elements and blocks

increases signal-to-noise ratio in text

...! System {! Name "onoff”! Block {! BlockType DiscretePulseGenerator! Name "Discrete Pulse\nGenerator”! Amplitude "1"! Period "2"! PulseWidth "1"! PhaseDelay "0"! SampleTime "1"! }! Block {! BlockType Product! Name "Product"! Ports [2, 1, 0, 0, 0]! Inputs "2”! }! ...! }! ...!

filtering significantly improved performance precision - 10x fewer false positives

hand validation of results

recall - many fewer false negatives fewer missed clones

much larger clones

but: some clones we could clearly see by hand still not detected - why?

Challenge #4 no linear order of model elements

Challenge #4 solution: topological sort by block, line, port, branch

sorting increases recall, to find many more clones

neutral_up_down1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1



powerwindow03/power_window_control_system/validate_driverneutral_up_down

1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1




sorting increases recall, to find many more clones

neutral_up_down1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1



powerwindow03/power_window_control_system/validate_driverneutral_up_down

1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1




neutral_up_down1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1




neutral_up_down1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1




neutral_up_down1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1




neutral_up_down1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1




neutral_up_down1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1




neutral_up_down1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1




neutral_up_down1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1




neutral_up_down1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1




neutral_up_down1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1




neutral_up_down1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1




neutral_up_down1

mutually_exclusive

neutral

up

down

validated_neutral

validated_up

validated_down

check_up

action

resetchecked_action

check_down

action

resetchecked_action

Goto1[reset]

From2[reset]

From1[reset]

reset4

down3

up2

neutral1




...! System {! Name "onoff”! Block {! BlockType DiscretePulseGenerator! Name "Discrete Pulse\nGenerator”! Amplitude "1"! Period "2"! PulseWidth "1"! PhaseDelay "0"! SampleTime "1"! }! Block {! BlockType Product! Name "Product"! Ports [2, 1, 0, 0, 0]! Inputs "2”! }! ...! }! ...!

Challenge #5 finding type 2 (renamed) requires anonymization

names in Simulink not like other languages

solution: context-dependent anonymizer

validation - Simone vs. ConQAT on Matlab Central public model systems

finds all type1(exact) and type 2 (renamed) clones found by ConQAT

finds many new type 3 (near-miss) clones not found by ConQAT

finds larger clones and larger clone classes


Simone vs. ConQAT

Total nontrivial subsystems

357 Extractor only Filtered Filtered & Sorted Filtered, Sorted &

Renamed

Clone Type Type 1 Type 3-1 @30% Type 1 Type 3-1

@30% Type 1 Type 3-1 @30% Type 2 Type 3-2

@30%

Clone Pairs 116 / 10* 364 / 164* 204 204 303 181 279 1938

Clone Classes 8 / 4* 57 / 56* 44 55 45 52 48 24

Clone Coverage 8% / 3% 52% / 46% 37% 48% 42% 45% 49% 75%

Simone near-miss clones in Simulink public automotive model variants


!"#$%&%'()(*+&$%'( ,+-.".$%-$(/%-01"-2( 3#"-'(/%-01"-2(

*4.$%1.(56$&07$%'( ,#+-%.( ,#0..%.( ,#+-%.( ,#0..%.( ,#+-%.( ,#0..%.(

!"#$"%&'()#*"$& +,-& ,,.& ,+& ,/+& ,.& -0,& ,1&

2345$3)6&7(48& 1/9:& ;0-& .,& ,11& .,& 1.;0& --&

'8)<"#&=>"4?$(@& --0& 1+.& -0& ;;;& -0& -,.& -,&

89('+$)::( 099& ;11& ;.& ;90& ;0& /::& ;:&

89('+$):;( 099& ;11& ;/& ;.-& ;.& /1.& ;1&

89('+$):<( 0./& ;1;& ;+& ;.-& ;/& /;-& ;,&

89('+$):=( 1:01& ;/-& ,+& ,9;& -1& 11-9& ,:&

Simone near-miss pattern mining in Simulink models




pattern evolution

Case studyGM fuel system models

SimGraph visualizationunderstanding Simulink model subsystem similarity

GM Fuel System ModelsSubsystem similarity overview

Largesubsystems

Midsize subsystems

Smallsubsystems

Z models (red)

Y models (green)

X models (blue)

GM Fuel System ModelsSubsystem similarity overview

Many subsystems unique - not similar to any others in

these models

GM Fuel System ModelsRemove unique subsystems Connecting lines represent subsystem similarity -

thick lines, 90-100% similarthin lines, 70-80% similar

Similar subsystems “near-miss clones”

both within and between models

GM Fuel System ModelsRearrange to cluster similar subsystems Clusters reveal groups of

similar subsystems -“clone classes”

GM Fuel System ModelsInfer common subsystem patterns

Patterns characterize common repeated similar

subsystem paradigms

Small groups of relatively large

similar subsystems both within and across models

Large groups of small to mid-sized similar subsystems across

models




pattern evolution

using patterns

SimNavexemplar sets as patterns

SimPatmodeling variance

SimNav presenting and integrating results in Simulink

exemplar sets as patterns

SimNav

&'()*'+,

-'./0

&=!>-?

!"#$"%#12345/61078*9+170131839:7;69):8178901<

&'()*'+,78*9+170131839:7;(901*6<&'(-/@

SimPat characterizing and representing subsystem patterns

modeling variance

SimPat

!901*7D/E1:+6!901*7.*9+17.*/66A7B76)56C631(78*9+16

!901*7D/E1:+7FA!1:G17#H:9E*177/+07D:166):17?6I(/I9+

!901*7D/E1:+7JA777!1:G17/**73H:11777777776)56C631(67

!901*7D/E1:+6!901*7.*9+17.*/66A7B76)56C631(78*9+16

!901*7D/E1:+7BA.9((9+71*1(1+3677777777777779+*C

!901*7D/E1:+6!901*7.*9+17.*/66A7B76)56C631(78*9+16




pattern evolution

!"#$

%&$

%"$

&'$ &($ &)$

&'$

&'$

&*$

%!$

&($ &)$ &*$

&($ &)$ &*$

!"#$

model pattern evolutionSimCCT

evolution of patterns across versions

pattern variance in two dimensionsinstance, time

[Stephan, Alalfi, Cordy, Stevenson ME 2013]

model patternMCC - model clone class

elements of a model patternMCI - model clone instance

evolution of patterns migration of MCIs between MCCs across versions of the system

evolution of patterns1-1 pattern is stable across versions

1-1* pattern exists, but loses or gains MCIs

1-many pattern splits into multiple patterns

1-many* pattern splits, losing or gaining MCIs

1-0 pattern unifies or disappears

SimCCT

SimCCT - Power Window MCC 3 !" #"

!" $" %" &" '(" '!" '#"

!" $" %" &" '(" '!" '#" )("

*'"

*)"

*+"

*,"-"*!" ./01"2/3"405/26"3/"728"9::"

!"#"$#%&'()#*+,-!,.,

!"#"$#%/0)#1$2",-!,3,

45"$6%78,-!,9,

!"#

!$#

$# %# &#

%# &#

%# &#

!'# ()*+#,)-#.*/),0#-)#1,2#344#

%# &#

!%#

!&#

SimCCT - Power Window MCC 2

!"#$

%&$

%"$

&'$ &($ &)$

&'$

&'$

&*$

%!$

&($ &)$ &*$

&($ &)$ &*$

!"#$

SimCCT - AVS MCC 7




pattern evolution

current workStateflow models

deployment at GManalysis of more systems

Thank you! Manar H. AlalfiThomas R. DeanMatthew StephanAndrew Stevenson

Joseph d’AmbrosioCheryl Williams

James R. CordyQueen’s University

NECSIS Automotive Partnership Canada

Submodel Pattern Extraction for Simulink Models

Technology

131010 jim cordy - submodel pattern extraction for simulink models