Meta-models and Infrastructure for Smalltalk Omnipresent History … · Etoys 3.0 Etoys 4.0 Squeak 3.8 Croquet Squeak 3.9 Squeak 3.10 Cuis Squeak 3.6 Closure Relicensing TextEditor

Argentina - Nov 12th 2010

Meta-models and Infrastructure for Smalltalk Omnipresent HistoryVerónica Uquillas-GómezStéphane DucasseTheo D’Hondt

Andy Kellens, VUB

Monday 29 November 2010

Source Code History Analysis

2



✤ Linear history

2



✤ Linear history

2

Pharo 0.x



✤ Linear history

2

Pharo 0.xPharo 1.0



✤ Linear history

2

Pharo 0.xPharo 1.0

Pharo 1.1



✤ Linear history

2

Pharo 0.xPharo 1.0

Pharo 1.1Pharo 1.2



✤ Linear history

✤ Cross history

2

Pharo 0.xPharo 1.0

Pharo 1.1Pharo 1.2



✤ Linear history

✤ Cross history

2

Pharo 1.0

Squeak 3.7

Etoys 3.0

Etoys 4.0

Squeak 3.8

CroquetSqueak 3.9

Squeak 3.10 Cuis

Squeak 3.6

Closure

Relicensing

TextEditor

MessageTally

Closure

Unicode

Unicode fixes

Pharo 0.xPharo 1.0

Pharo 1.1Pharo 1.2



✤ Linear history

✤ Cross history

2

Pharo 1.0

Squeak 3.7

Etoys 3.0

Etoys 4.0

Squeak 3.8

CroquetSqueak 3.9

Squeak 3.10 Cuis

Squeak 3.6

Closure

Relicensing

TextEditor

MessageTally

Closure

Unicode

Unicode fixes

Squeak

Pharo 0.xPharo 1.0

Pharo 1.1Pharo 1.2



✤ Linear history

✤ Cross history

2

Pharo 1.0

Squeak 3.7

Etoys 3.0

Etoys 4.0

Squeak 3.8

CroquetSqueak 3.9

Squeak 3.10 Cuis

Squeak 3.6

Closure

Relicensing

TextEditor

MessageTally

Closure

Unicode

Unicode fixes

Squeak

Pharo

Pharo 0.xPharo 1.0

Pharo 1.1Pharo 1.2


Querying the History of a System

3



3

Co-change analysis what are the entities that changed together with entity Number in version 3 ?

Queries as in the past

what were the senders of the method #asString in Squeak 3.9?

Queries as in the present

what are the messages sent by method #printOn: in version 2?

Bug spot was the method #printString regularly changed over the last 5 years?

Global analysis what is the whole history of method #detect:ifNone:?

Forks analysis if the version of method #isNil changed in Squeak 3.9, should it be changed in Pharo?

Comparison profiler what are the differences or similarities of running versions 4 and 5?



3












3












3












3












3












3











We need ...

4


We need ...

✤ History meta-model

4


We need ...


✦ Linear history

4


We need ...


✦ Linear history

✦ Cross history

4


We need ...


✦ Linear history

✦ Cross history

✤ Infrastructure

4


We need ...


✦ Linear history

✦ Cross history

✤ Infrastructure

✦ Store lot of data

4


We need ...


✦ Linear history

✦ Cross history

✤ Infrastructure


✦ Efficient /Fast

4


We need ...


✦ Linear history

✦ Cross history

✤ Infrastructure


✦ Efficient /Fast

✦ Accessible

4


Multiple Meta-ModelsSmalltalk model RB source code model

MC source code model Tools source code model

5


Multiple Meta-Models

localSelectors

Metaclass

selectorpragmas

CompiledMethod

instanceVariables

ClassDescription

Behavior

*

nameclassPoolsharedPoollocalSelectors

Class

Object

thisClass

methodDict contains

subclasses

methodClassselector category

MethodReference

methodReference

1

categoriesallMethodSelectors

ClassOrganizer

organization

1

subject1

superclass

1

*

classComment commentStamp

BasicClassOrganizer

methodClass

1

SystemDictionary

environment

1

SystemOrganizercategoryArray

Categorizercategory is managed by

namespace=

Smalltalk model RB source code model


5



localSelectors

Metaclass

selectorpragmas

CompiledMethod

instanceVariables

ClassDescription

Behavior

*


Class

Object

thisClass

methodDict contains

subclasses


MethodReference

methodReference

1


ClassOrganizer

organization

1

subject1

superclass

1

*


BasicClassOrganizer

methodClass

1

SystemDictionary

environment

1



namespace=

categorycommentclassVariableNamespoolVariableNames

RBClass

compiledMethodsourceselector

RBMethod

instanceVariableNames

RBAbstractClass

implementorssenders

RBNamespace

*newMethods

RBMetaclass

*removedMethods

1 class

*

newClasses

*

changedClasses

BrowserEnvironment1

environment

*

removedClassessuperclasses

subclasses

* *

model

1

theMetaClass

theNonMetaClass



5



localSelectors

Metaclass

selectorpragmas

CompiledMethod

instanceVariables

ClassDescription

Behavior

*


Class

Object

thisClass

methodDict contains

subclasses


MethodReference

methodReference

1


ClassOrganizer

organization

1

subject1

superclass

1

*


BasicClassOrganizer

methodClass

1

SystemDictionary

environment

1



namespace=


RBClass


RBMethod


RBAbstractClass

implementorssenders

RBNamespace

*newMethods

RBMetaclass

*removedMethods

1 class

*

newClasses

*

changedClasses

BrowserEnvironment1

environment

*


subclasses

* *

model

1

theMetaClass

theNonMetaClass


MC source code modelidnamemessagedatetimeauthor

MCVersionInfo

MCDefinition

nameworkingCopy

MCPackage dependencieschangesworkingCopy

MCVersion

patchRelativeToBase:MCSnapshot

1

snapshot

definitions

info

*

snapshot

1package1

name

MCClassDefinition categories

MCOrganizationDefinition

selectorsourcecategory

MCMethodDefinition

name

MCVariableDefinition

variables

*

packageName

MCScriptDefinition

name

MCTraitDefinition

MCPostscriptDefinition

MCPreambleDefinitionMCInstaceVariable

DefinitionMCClassVariable

Definition

Tools source code model

5



localSelectors

Metaclass

selectorpragmas

CompiledMethod

instanceVariables

ClassDescription

Behavior

*


Class

Object

thisClass

methodDict contains

subclasses


MethodReference

methodReference

1


ClassOrganizer

organization

1

subject1

superclass

1

*


BasicClassOrganizer

methodClass

1

SystemDictionary

environment

1



namespace=


RBClass


RBMethod


RBAbstractClass

implementorssenders

RBNamespace

*newMethods

RBMetaclass

*removedMethods

1 class

*

newClasses

*

changedClasses

BrowserEnvironment1

environment

*


subclasses

* *

model

1

theMetaClass

theNonMetaClass



MCVersionInfo

MCDefinition

nameworkingCopy


MCVersion


1

snapshot

definitions

info

*

snapshot

1package1

name




MCMethodDefinition

name


variables

*

packageName

MCScriptDefinition

name

MCTraitDefinition




Definition

Class

Method

Attribute

Access

Inheritance

Package

Namespace

*

parentType

*

parentType

containerparentPackage

superclass

subclass

* *

Invocationsender

candidates

*

accessor

*

variable

*

*

*

*

Reference

*source

* target

*


5



localSelectors

Metaclass

selectorpragmas

CompiledMethod

instanceVariables

ClassDescription

Behavior

*


Class

Object

thisClass

methodDict contains

subclasses


MethodReference

methodReference

1


ClassOrganizer

organization

1

subject1

superclass

1

*


BasicClassOrganizer

methodClass

1

SystemDictionary

environment

1



namespace=


RBClass


RBMethod


RBAbstractClass

implementorssenders

RBNamespace

*newMethods

RBMetaclass

*removedMethods

1 class

*

newClasses

*

changedClasses

BrowserEnvironment1

environment

*


subclasses

* *

model

1

theMetaClass

theNonMetaClass



MCVersionInfo

MCDefinition

nameworkingCopy


MCVersion


1

snapshot

definitions

info

*

snapshot

1package1

name




MCMethodDefinition

name


variables

*

packageName

MCScriptDefinition

name

MCTraitDefinition




Definition

Class

Method

Attribute

Access

Inheritance

Package

Namespace

*

parentType

*

parentType


superclass

subclass

* *

Invocationsender

candidates

*

accessor

*

variable

*

*

*

*

Reference

*source

* target

*


Class

Method

Attribute

Access

Inheritance

Package

Namespace

*

parentType

*

parentType


superclass

subclass

* *

Invocationsender

candidates

*

accessor

*

variable

*

*

*

*

Reference

*source

* target

*

5



localSelectors

Metaclass

selectorpragmas

CompiledMethod

instanceVariables

ClassDescription

Behavior

*


Class

Object

thisClass

methodDict contains

subclasses


MethodReference

methodReference

1


ClassOrganizer

organization

1

subject1

superclass

1

*


BasicClassOrganizer

methodClass

1

SystemDictionary

environment

1



namespace=


RBClass


RBMethod


RBAbstractClass

implementorssenders

RBNamespace

*newMethods

RBMetaclass

*removedMethods

1 class

*

newClasses

*

changedClasses

BrowserEnvironment1

environment

*


subclasses

* *

model

1

theMetaClass

theNonMetaClass



MCVersionInfo

MCDefinition

nameworkingCopy


MCVersion


1

snapshot

definitions

info

*

snapshot

1package1

name




MCMethodDefinition

name


variables

*

packageName

MCScriptDefinition

name

MCTraitDefinition




Definition

Class

Method

Attribute

Access

Inheritance

Package

Namespace

*

parentType

*

parentType


superclass

subclass

* *

Invocationsender

candidates

*

accessor

*

variable

*

*

*

*

Reference

*source

* target

*


Class

Method

Attribute

Access

Inheritance

Package

Namespace

*

parentType

*

parentType


superclass

subclass

* *

Invocationsender

candidates

*

accessor

*

variable

*

*

*

*

Reference

*source

* target

*

Class

Method

Attribute

Access

Inheritance

Package

Namespace

*

parentType

*

parentType


superclass

subclass

* *

Invocationsender

candidates

*

accessor

*

variable

*

*

*

*

Reference

*source

* target

*

5



localSelectors

Metaclass

selectorpragmas

CompiledMethod

instanceVariables

ClassDescription

Behavior

*


Class

Object

thisClass

methodDict contains

subclasses


MethodReference

methodReference

1


ClassOrganizer

organization

1

subject1

superclass

1

*


BasicClassOrganizer

methodClass

1

SystemDictionary

environment

1



namespace=


RBClass


RBMethod


RBAbstractClass

implementorssenders

RBNamespace

*newMethods

RBMetaclass

*removedMethods

1 class

*

newClasses

*

changedClasses

BrowserEnvironment1

environment

*


subclasses

* *

model

1

theMetaClass

theNonMetaClass



MCVersionInfo

MCDefinition

nameworkingCopy


MCVersion


1

snapshot

definitions

info

*

snapshot

1package1

name




MCMethodDefinition

name


variables

*

packageName

MCScriptDefinition

name

MCTraitDefinition




Definition

Class

Method

Attribute

Access

Inheritance

Package

Namespace

*

parentType

*

parentType


superclass

subclass

* *

Invocationsender

candidates

*

accessor

*

variable

*

*

*

*

Reference

*source

* target

*


Class

Method

Attribute

Access

Inheritance

Package

Namespace

*

parentType

*

parentType


superclass

subclass

* *

Invocationsender

candidates

*

accessor

*

variable

*

*

*

*

Reference

*source

* target

*

Class

Method

Attribute

Access

Inheritance

Package

Namespace

*

parentType

*

parentType


superclass

subclass

* *

Invocationsender

candidates

*

accessor

*

variable

*

*

*

*

Reference

*source

* target

*

Class

Method

Attribute

Access

Inheritance

Package

Namespace

*

parentType

*

parentType


superclass

subclass

* *

Invocationsender

candidates

*

accessor

*

variable

*

*

*

*

Reference

*source

* target

*

5



localSelectors

Metaclass

selectorpragmas

CompiledMethod

instanceVariables

ClassDescription

Behavior

*


Class

Object

thisClass

methodDict contains

subclasses


MethodReference

methodReference

1


ClassOrganizer

organization

1

subject1

superclass

1

*


BasicClassOrganizer

methodClass

1

SystemDictionary

environment

1



namespace=


RBClass


RBMethod


RBAbstractClass

implementorssenders

RBNamespace

*newMethods

RBMetaclass

*removedMethods

1 class

*

newClasses

*

changedClasses

BrowserEnvironment1

environment

*


subclasses

* *

model

1

theMetaClass

theNonMetaClass



MCVersionInfo

MCDefinition

nameworkingCopy


MCVersion


1

snapshot

definitions

info

*

snapshot

1package1

name




MCMethodDefinition

name


variables

*

packageName

MCScriptDefinition

name

MCTraitDefinition




Definition

Class

Method

Attribute

Access

Inheritance

Package

Namespace

*

parentType

*

parentType


superclass

subclass

* *

Invocationsender

candidates

*

accessor

*

variable

*

*

*

*

Reference

*source

* target

*


Class

Method

Attribute

Access

Inheritance

Package

Namespace

*

parentType

*

parentType


superclass

subclass

* *

Invocationsender

candidates

*

accessor

*

variable

*

*

*

*

Reference

*source

* target

*

Class

Method

Attribute

Access

Inheritance

Package

Namespace

*

parentType

*

parentType


superclass

subclass

* *

Invocationsender

candidates

*

accessor

*

variable

*

*

*

*

Reference

*source

* target

*

Class

Method

Attribute

Access

Inheritance

Package

Namespace

*

parentType

*

parentType


superclass

subclass

* *

Invocationsender

candidates

*

accessor

*

variable

*

*

*

*

Reference

*source

* target

*

5

How easy is ... interacting,

maintaining, synchronizing?


Meta-Models Plague

6


Meta-Models Plague

✤ Multiple models API

6


Meta-Models Plague


6


Meta-Models Plague


✦ conversion or transformation

6


Meta-Models Plague



✦ duplication of data, tests

6


Meta-Models Plague




✦ more maintenance tasks

6


Meta-Models Plague




✦ more maintenance tasks

6

Mass duplic

ation of e

fforts

/tasks


Versioning Systems vs. Source Code Meta-Model

7



GIT MC SVN

Versioning System

7



GIT MC SVN

Versioning System

Source Code Meta-Model

localSelectors

Metaclass

selectorpragmas

CompiledMethod

instanceVariables

ClassDescription

Behavior

*


Class

Object

thisClass

methodDict contains

subclasses


MethodReference

methodReference

1


ClassOrganizer

organization

1

subject1

superclass

1

*


BasicClassOrganizer

methodClass

1

SystemDictionary

environment

1



namespace=

7



GIT MC SVN

Versioning System


localSelectors

Metaclass

selectorpragmas

CompiledMethod

instanceVariables

ClassDescription

Behavior

*


Class

Object

thisClass

methodDict contains

subclasses


MethodReference

methodReference

1


ClassOrganizer

organization

1

subject1

superclass

1

*


BasicClassOrganizer

methodClass

1

SystemDictionary

environment

1



namespace=

7



GIT MC SVN

Versioning System


✤ algorithm✤ changes✤ versions

localSelectors

Metaclass

selectorpragmas

CompiledMethod

instanceVariables

ClassDescription

Behavior

*


Class

Object

thisClass

methodDict contains

subclasses


MethodReference

methodReference

1


ClassOrganizer

organization

1

subject1

superclass

1

*


BasicClassOrganizer

methodClass

1

SystemDictionary

environment

1



namespace=

7



GIT MC SVN

Versioning System



localSelectors

Metaclass

selectorpragmas

CompiledMethod

instanceVariables

ClassDescription

Behavior

*


Class

Object

thisClass

methodDict contains

subclasses


MethodReference

methodReference

1


ClassOrganizer

organization

1

subject1

superclass

1

*


BasicClassOrganizer

methodClass

1

SystemDictionary

environment

1



namespace=

7



GIT MC SVN

Versioning System



localSelectors

Metaclass

selectorpragmas

CompiledMethod

instanceVariables

ClassDescription

Behavior

*


Class

Object

thisClass

methodDict contains

subclasses


MethodReference

methodReference

1


ClassOrganizer

organization

1

subject1

superclass

1

*


BasicClassOrganizer

methodClass

1

SystemDictionary

environment

1



namespace=

7


Requirements for Source Code and History Modeling

8



✤ No duplication of meta-models

8




✤ Model update as cheap as possible

8




✤ Model update as cheap as possible

✤ Tool reusability relying on common APIs

8


Unifying and Foundational Model Infrastructure

9



9



✤ Provide a common API (structural & runtime level)

9




✤ Allow tools to interact and integrate directly with the host environment

9





✤ Become the foundational model in

9






✤ Support history analysis

9






✤ Support history analysis

9


Ring Overview

10


Ring Overview

10

Structural Model


Ring Overview

10

Structural Model

Runtime Model

runtime API

structural API


Ring Overview

10

Structural Model

Runtime Model

runtime API

structural API

ChangeModel


Ring Overview

10

Structural Model

Runtime Model

runtime API

structural API

RB Source Model

ChangeModel


Ring Overview

10

Structural Model

Runtime Model

runtime API

structural API

RB Source Model

VersioningModel

ChangeModel


Ring Overview

10

Structural Model

Runtime Model

runtime API

structural API

RB Source Model

VersioningModel

MergingModel

ChangeModel


Ring Overview

10

Structural Model

Runtime Model

runtime API

structural API

RB Source Model

VersioningModel

MergingModel

ChangeModel

ToolsToolsTools


The Onion Structure of the Ring

11



11

Definition

ClassDefinition

MethodDefinition

Core Source Model

CommentDefinition

Properties



11

Definition

ClassDefinition

MethodDefinition

Core Source Model

CommentDefinition

Properties

Runtime Model



11

Definition

ClassDefinition

MethodDefinition

Core Source Model

CommentDefinition

Properties

Runtime Model

Basic ToolsBasic Tools

Basic Tools



11

Definition

ClassDefinition

MethodDefinition

Core Source Model

CommentDefinition

Properties

Runtime Model


Basic Tools

PackageDefinition

RB Source ModelVariableDefinition



11

Definition

ClassDefinition

MethodDefinition

Core Source Model

CommentDefinition

Properties

Runtime Model


Basic Tools

PackageDefinition


RB Source Model



11

Definition

ClassDefinition

MethodDefinition

Core Source Model

CommentDefinition

Properties

ScriptDefinition

Versioning Source ModelPoolDictionaryDefinition

Version

Runtime Model


Basic Tools

PackageDefinition


RB Source Model



11

Definition

ClassDefinition

MethodDefinition

Core Source Model

CommentDefinition

Properties

ScriptDefinition


Version

Runtime Model


Basic Tools

PackageDefinition


MergingModel

RB Source Model


Full off line analysis Source Model...


11

Definition

ClassDefinition

MethodDefinition

Core Source Model

CommentDefinition

Properties

ScriptDefinition


Version

Runtime Model


Basic Tools

PackageDefinition


MergingModel

RB Source Model




11

Definition

ClassDefinition

MethodDefinition

Core Source Model

CommentDefinition

Properties

ScriptDefinition


Version

Runtime Model


Basic Tools

PackageDefinition


MergingModel

ChangeModel

RB Source Model




11

Definition

ClassDefinition

MethodDefinition

Core Source Model

CommentDefinition

Properties

ScriptDefinition


Version

Runtime Model


Basic Tools

PackageDefinition


MergingModel

ChangeModel

RB Source Model

HistoryModel


Open Questions

12


Open Questions

✤ Expensive queries

12


Open Questions


✤ Version ids

12


Open Questions


✤ Version ids

✤ Annotations

12


Open Questions


✤ Version ids

✤ Annotations

✤ Meta-model vs. Database schema

12


Open Questions


✤ Version ids

✤ Annotations


✤ Core code model API

12


Open Questions


✤ Version ids

✤ Annotations



✤ Meta-models extensibility

12


Open Questions


✤ Version ids

✤ Annotations



✤ Meta-models extensibility

✤ Unifying models12


Summary

13


Summary✤ Goal:

✦ perform linear and cross history analysis

13


Summary✤ Goal:


✤ Problem:

✦ unsuitable infrastructure for querying the history

✦ several meta-models overlapping & non-polymorphic APIs

13


Summary✤ Goal:


✤ Problem:



✤ Proposal:

✦ unifying and foundational model infrastructure, Ring

13


Summary✤ Goal:


✤ Problem:



✤ Proposal:

✦ unifying and foundational model infrastructure, Ring

✤ Current work:

✦ implementation of the Ring

13



Meta-models and Infrastructure for Smalltalk Omnipresent History

Verónica Uquillas-Gómez, [email protected]


mailto:[email protected]














Documents

Meta-models and Infrastructure for Smalltalk Omnipresent History … · Etoys 3.0 Etoys 4.0 Squeak 3.8 Croquet Squeak 3.9 Squeak 3.10 Cuis Squeak 3.6 Closure Relicensing TextEditor