1

On MDE to implement a Component Assembly Compiler for HPC

Journées IDM, June the 7th 2011

Julien Bigot*, Christian Pérez*Laboratoire de l’Informatique du Parallelisme (LIP) ENS Lyon

2

Context: High Performance Computing

•Scientific applications• Domain specialist code

• Long life cycle (~30 years)

• Computation intensive

•Hardware• Heterogeneous

– Supercomputers– Computing clusters– Computing grids– Clouds

• Parallel

• Short life cycle (~3 years)

•Programming model• Low level, fine-tuning of code

• Hand made … and remade

Thermal

Optics

Dynamics

Structural Mechanics

SAN SAN

Cluster

LAN

WAN

Supercomputer

GridSupercomputer

Cluster

Satellite modeling ? programmingmodels

3

Outline

•Context: High Performance Computing (HPC)

•High Level Component Model (HLCM) Modeling and Implementation

•Feedback on the Model Driven Approach

•Conclusion

4

Programming model requirements

•Ease of development• Domain specialist: Domain code / support code separation

• Long life cycle: clean code behavior and interface specification

•High Performance• Minimal overhead

• Support for parallelism

•Hardware evolution• Algorithm optimization

• Interaction optimization

– E.g. distributed memory vs. shared memory

Software Components

5

Software components

•Components• black boxes

• Interactions through well defined interaction points

•Application• Set of interconnected instances

• High level view of architecture

•Separation of concern• Domain code: components

• Support: component model

•Eased reuse• Well defined interface & behavior

CmpA CmpB

a

b

c

d

c1 c2

6

Software Component Models

•Industrial Component Models (CORBA Component Model, Enterprise JavaBeans, …)

• Transparent distribution support (CORBA, JavaRMI, …)

• Overhead for same host components

•Research Component Models (Fractal, …)• Interesting features (Hierarchy, Connectors, …)

• Features handled at runtime, additional costs

•High Performance Component Models (CCA, …)• Acceptable overheads

• Low level of abstraction => Hardware adaptation complex

7

Programming model requirements

•Ease of development• Domain specialist: Domain code / support code separation

• Long life cycle: clean code behavior and interface specification

•High Performance• Minimal overhead

• Support for parallelism

•Hardware evolution• Algorithm optimization

• Interaction optimization

– E.g. distributed memory vs. shared memory

Software Components

Component implementation choice

8

Multiple alternative implementations

•Distinction type/implementation• Type: name + ports

• Implementation: behavior

•Choice• Depending on available hardware

CmpA

I1 I2

9

Programming model requirements

•Ease of development• Domain specialist: Domain code / support code separation

• Long life cycle: clean code behavior and interface specification

•High Performance• Minimal overhead

• Support for parallelism

•Hardware evolution• Algorithm optimization

• Interaction optimization

– E.g. distributed memory vs. shared memory

Software Components

Component implementation choice

1st class Software Connectors

with implementation choice

10

1st class software connectors

•1st class entity• A Name

• Set of roles

• Multiple alternative implementations

•Hardware optimization possible• Interaction code well identified

• Multiple versions possible

• More application specific interactions

UP

CmpA CmpB

CmpA CmpB

ports

roles

user provider

11

Programming model requirements

•Ease of development• Domain specialist: Domain code / support code separation

• Long life cycle: clean code behavior and interface specification

•High Performance• Minimal overhead

• Support for parallelism

•Hardware evolution• Algorithm optimization

• Interaction optimization

– E.g. distributed memory vs. shared memory

Software Components

Algorithmic skeletons,

generic programming & hierarchy

Component implementation choice

1st class Software Connectors

with implementation choice

12

Generic programming & hierarchy:Algorithmic skeletons

Disp<I> Coll<O>

W

W

W

Farm<W, I, O, N>

.

.

.N times

I O

Farm

13

Programming model requirements

•Ease of development• Domain specialist: Domain code / support code separation

• Long life cycle: clean code behavior and interface specification

•High Performance• Minimal overhead

• Support for parallelism

•Hardware evolution• Algorithm optimization

• Interaction optimization

– E.g. distributed memory vs. shared memory

Software Components

Compilation based approach

Algorithmic skeletons,

generic programming & hierarchy

Component implementation choice

1st class Software Connectors

with implementation choice

No need for new components, only a new assembly model

14

HLCM Compilation

Reuse an existing component model (L²C)

HLCM: Genericity Hierarchy Connectors Multi-Implem

L²C: Low Level Primitive components

only Various interaction

C++ MPI CORBA …

Hardware description

PIM PSM

PDM

MDA

15

HCLM (Meta)Modeling

•Using the Eclipse Modeling Framework (EMF)

•PIM• Main classes: ComponentType, PortType, Connector, …

• Primitive implementations: abstract

• Some patterns identified: genericity, implementation choice

• 130 Ecore metaclasses (490 lines of EMFatic)

• 2000 utility Java lines

•PSM• Main classes: ComponentInstance, Connexion, …

• References to the PIM in each instance

• 41 Ecore metaclasses (160 lines of EMFatic)

• 800 utility Java lines

16

HCLM Transformation

•Four step transformation• Parsing HLCM: 400 lines Xtext annotated grammar

• Compilation: 4000 lines plain Java

• PSM to Backend

• Backend Model Dump

•Compilation• Choosing implementations (delegated to an external “oracle”)

• Concretizing generic elements

• Flattening composites

• Until convergence: only primitive elements

•Final dump• Often single hand made step

Backend specific

17

Feedback (1/2)

•Modeling tools: huge gain of time• Modeling HLCM

– Initial attempt with plain Java: ~1 month work

– Second attempt with Ecore: ~2 days work

• Direct concrete syntax to model parsing (Xtext)

•Ease of comprehension• Most concepts intuitive with UML background

• Various level of modeling confusing (model vs. metamodel)

– E.g. connectors

– Fixed set of interactions: various connectors in the metamodel

– 1st class connectors: various connectors in the model

– Combination predefined connectors + user defined : ???

– E.g. genericity: manipulation of metamodel elements in the model

About same state

18

Feedback (2/2)

•Model transformation• Transformation tools not ready 2 years ago (Kermeta , QVT, ATL)

• Plain java transformation

– API not human friendly

– Collection access => whole collection traversal

– Omnipresent type casts (constraint not expressible in Ecore)

– Error complex to associate with user error

• Constraint language evaluation required

•Real use• Applications: Parallel method calls, shared memory, Domain

decomposition

• Backends: L²C , Pure Java, CCM, Charm++, …

• Transformation speed: 1500+ components, ≤ 10secs

19

Conclusion

•HLCM: a component assembly model for HPC• Hierarchy, Generic Programming, Connectors, Alternative

Implementations

– Efficiency possible on a wide range of hardware

• A model based implementation

– Lot of time gained

– Some tools not used due to immaturity

• Usable in real world

•Perspectives / future work• Re-evaluate modeling tools (ATL, OCL, …)

• Introduce the time dimension in HLCM

– Models@runtime ?

20