PaaSport Marketplace Infrastructure First Release ...linc.ucy.ac.cy › file › PaaSport › PaaSport_D5.2_finalrelease.pdfPaaSport is funded by the European Commission Seventh Framework

PaaSport is funded by the European Commission Seventh Framework Programme Contract no.: 605193

A semantically enhanced marketplace of interoperable platform-as-a-service offerings for the deployment and

migration of business applications of SMEs

PaaSport Marketplace Infrastructure

- First Release

Deliverable 5.2

Date: May 2015

Deliverable Title PaaSport Marketplace Infrastructure – First Release

Filename 1.0 Author(s) Giannis Ledakis (SILO), Kostas Kalaboukas

(SILO), Nick Bassilades (IHU), Demetris Trihinas (UCY), Gerald Hübsch (CAS), Islam Hassan (NUIG)

Reviewers Nick Bassiliades (IHU), Andreas Papadopoulos (UCY)

Date May 2014


Start of the project: 01/12/2013 Duration: 36 months Project coordinator organisation: BITMI “BUNDESVERBAND IT-MITTELSTAND EV” Deliverable title: Unified PaaSport Marketplace Infrastructure – First Release Deliverable no.: 5.2 Due date of deliverable: May 2015 Actual submission date: June 2015 Dissemination Level

PU Public

PP Restricted to other programme participants (including the Commission Services)

RE Restricted to a group specified by the consortium (including the Commission Services)

CO CO Confidential, only for members of the consortium (including the Commission Services)

Deliverable status version control

Version Date Organisation Authors

0.1 10 April 2014 SILO Giannis Ledakis

0.2 27 April2014 SILO Kostas Kalaboukas

0.3 04 May 2014 IHU Moisis Symeonidis

Nick Bassilades

0.4 08 May 2014 NUIG Islam A. Hassan

0.5 15 May 2014 CAS Gerald Hübsch

0.6 20 May 2014 UCY Demetris Trihinas

0.7 24 May 2014

SILO

Giannis Ledakis

0.8 29 May 2014 Giannis Ledakis

0.9 03 June2014 Giannis Ledakis

1.0 15 June 2014 Giannis Ledakis


Abstract

The present document is Deliverable 5.2 “Unified PaaSport Marketplace Infrastructure – First Release” (henceforth referred to as D5.2) of the PaaSport project. This deliverable documents the technical progress that is related to the various components that have been described in the architectural deliverable D5.1. The PaaSport marketplace is a broker that offers matchmaking between applications and PaaS offerings while also supports the deployment and management of applications to PaaS offerings. PaaSport marketplace has several end-users since both developers and PaaS providers can use the platform in order to benefit from its functionalities. Towards this end, the efforts of the first development period were devoted to the various backend components and user-interface components (a.k.a. dashboards). Furthermore, this deliverable reports on the progress of integration of the entire PaaSport platform.

Keywords

PaaS Broker, PaaS interoperability, PaaS portability, PaaSport project

D5.2 –PaaSport Marketplace Infrastructure – First Release 4 / 59

Table of Contents Executive Summary .......................................................................................................... 9

1 Introduction ............................................................................................................ 10

1.1 Document Scope ........................................................................................................ 10

1.2 Document Structure ................................................................................................... 10

2 Overview of PaaSport Marketplace ......................................................................... 11

2.1 Unauthorized user ...................................................................................................... 11

2.2 Developer Dashboard ................................................................................................. 13

2.3 Provider Dashboard .................................................................................................... 18

2.4 Administrator Dashboard ........................................................................................... 19

3 Overview of Revised Architecture ............................................................................ 22

3.1 Overview of the Reference Architecture ..................................................................... 22

4 Features of the First Release of PaaSport Marketplace Infrastructure ...................... 24

4.1 Semantic Model .......................................................................................................... 24

4.1.1 DOLCE Extensibility Model ................................................................................. 24

4.1.2 Offering Model ................................................................................................... 26

4.1.3 Application Model .............................................................................................. 26

4.1.4 SLA Model ........................................................................................................... 27

4.1.5 Roadmap ............................................................................................................ 28

4.2 PaaS Offering Recommendation Layer ........................................................................ 28

4.2.1 PaaS Offering Search Component ...................................................................... 29

4.2.2 Semantic Query Handling Component ............................................................... 29

4.2.3 Semantic PaaS Offering Discovery Component ................................................. 29

4.2.4 Application to PaaS Offering Matchmaking Component ................................... 30

4.2.5 PaaS Offering Shortlist Component ................................................................... 32

4.2.6 PaaS Offering Selection Component .................................................................. 32

4.2.7 Interaction with the Persistency Layer............................................................... 33

4.2.8 Roadmap ............................................................................................................ 33

4.3 Monitoring and SLA Enforcement Layer ..................................................................... 34

4.3.1 SLA Monitoring Component ............................................................................... 34

4.3.2 Deployed Application Monitoring Component .................................................. 35

4.3.3 Roadmap ............................................................................................................ 37

4.4 Persistency, Execution and Coordination Layer .......................................................... 37


4.4.1 Persistency Layer ................................................................................................ 37

4.4.2 Execution Layer .................................................................................................. 43

4.4.3 Coordination Layer ............................................................................................. 46

4.4.4 Roadmap ............................................................................................................ 48

4.5 Adaptive Front-Ends ................................................................................................... 48

4.5.1 Roadmap ............................................................................................................ 50

5 Reporting on the Development Lifecycle Setup ........................................................ 52

5.1 Deployment of PaaSport Marketplace with Continuous Integration ........................... 52

5.2 Source Code Quality ................................................................................................... 54

5.3 Development Statistics ............................................................................................... 55

6 Conclusion............................................................................................................... 57

REFERENCES ................................................................................................................... 58

LIST OF ABBREVIATIONS ................................................................................................. 59


Table of Figures: FIGURE 1: PAASPORT MARKETPLACE WELCOME PAGE ........................................................................................ 11 FIGURE 2: LOGIN PAGE .......................................................................................................................................... 12 FIGURE 3: PAAS PROVIDER REGISTRATION PAGE ................................................................................................. 12 FIGURE 4: DEVELOPER REGISTRATION PAGE ........................................................................................................ 13 FIGURE 5: DEVELOPER MAIN PAGE ....................................................................................................................... 13 FIGURE 6: CREATION OF A MATCHMAKING REQUEST FOR AN APPLICATION. ..................................................... 14 FIGURE 7: VIEWING RECOMMENDED PAAS PROVIDERS FOR AN APPLICATION. ................................................. 14 FIGURE 8: DEPLOYMENT OF AN APPLICATION...................................................................................................... 15 FIGURE 9: MANAGEMENT OF DEPLOYED APPLICATIONS APPLICATION ............................................................... 15 FIGURE 10: MONITORING OF DEPLOYED APPLICATIONS. ..................................................................................... 16 FIGURE 11: DEVELOPER HELP CENTER .................................................................................................................. 17 FIGURE 12: ADDING KEYS FOR SPECIFIC PROVIDER .............................................................................................. 18 FIGURE 13: PAAS PROVIDER MAIN PAGE .............................................................................................................. 18 FIGURE 14: USER MANAGEMENT ......................................................................................................................... 19 FIGURE 15: SERVICES MANAGEMENT ................................................................................................................... 20 FIGURE 16: SERVICE CATEGORY MANAGEMENT .................................................................................................. 20 FIGURE 17: SERVICE PARAMETERS MANAGEMENT .............................................................................................. 21 FIGURE 18: HIGH-LEVEL VIEW OF THE PAASPORT MARKETPLACE ARCHITECTURE .............................................. 22 FIGURE 19: DNS PATTERN ..................................................................................................................................... 25 FIGURE 20: DUL AND PAASPORT ASSOCIATION .................................................................................................... 25 FIGURE 21: OFFERING MODEL OVERVIEW ............................................................................................................ 26 FIGURE 22: APPLICATION MODEL OVERVIEW ...................................................................................................... 27 FIGURE 23: SLA MODEL OVERVIEW ...................................................................................................................... 27 FIGURE 24: PAAS OFFERING RECOMMENDATION LAYER COMPONENTS ............................................................ 28 FIGURE 25: OVERVIEW OF THE PAASPORT MATCHMAKING AND RECOMMENDATION ALGORITHM ................. 30 FIGURE 26: THE DIFFERENT FUNCTIONS SUPPORTED BY SCORING FUNCTION .................................................... 32 FIGURE 27: WORKFLOW OF DATA FOR THE RECOMMENDATION LAYER ............................................................. 33 FIGURE 28: SLA MONITORING COMPONENT ........................................................................................................ 34 FIGURE 29: ARCHITECTURE OF THE PAASPORT MONITORING SERVICE ............................................................... 35 FIGURE 30: SAMPLE CODE FOR INCLUSION OF MONITORING AGENT THROUGH MAVEN .................................. 36 FIGURE 31: SAMPLE CODE FOR INCLUSION OF MONITORING AGENT AS FILTER ON APPLICATION .................... 36 FIGURE 32: DATABASE TABLES OF PAASPORT MARKETPLACE ............................................................................. 38 FIGURE 33: ORM OBJECTS ..................................................................................................................................... 39 FIGURE 34: INHERITED REPOSITORY METHODS ................................................................................................... 40 FIGURE 35: REPOSITORY INTERFACES AVAILABLE IN FIRST RELEASE OF PAASPORT MARKETPLACE ................... 41 FIGURE 36: RELATIONAL TO RDF MAPPING .......................................................................................................... 41 FIGURE 37: D2RQ EXAMPLE; CONNECTING TO THE DATABASE AND MAPPING A TABLE .................................... 42 FIGURE 38: D2RQ EXAMPLE; MAPPING A PROPERTY. .......................................................................................... 43 FIGURE 39: SAMPLE OF METHODS SPECIFIED IN THE IADAPTER INTERFACE CLASS............................................. 45 FIGURE 40: SAMPLE CODE FROM CREATEAPPLICATION METHOD ON CLOUDFOUNDRY ADAPTER .................... 45 FIGURE 41: USAGE OF PAASPORT PORTABILITY LIBRARIES. ................................................................................. 46 FIGURE 42: PROXYING OF CREATEAPPLICATION METHOD ................................................................................... 47 FIGURE 43: BACKEND CODE EXAMPLE FOR USER REGISTRATION ........................................................................ 49 FIGURE 44: FRONTEND HTML CODE EXAMPLE. .................................................................................................... 49 FIGURE 45: FRONTEND JAVASCRIPT CODE EXAMPLE. .......................................................................................... 50 FIGURE 46: FRONTEND UI EXAMPLE. .................................................................................................................... 50 FIGURE 47: CONTINUOUS DELIVERY IN PAASPORT .............................................................................................. 52 FIGURE 48: SAMPLE AUTOMATED BUILD OF PAASPORT ON JENKINS .................................................................. 53 FIGURE 49: INDICATIVE UNIT TESTS ...................................................................................................................... 54 FIGURE 50: SOURCE CODE QUALITY THROUGH SONARQUBE .............................................................................. 55


FIGURE 51: STATISTICS OF COMMIT LOG ............................................................................................................. 55 FIGURE 52 : DAILY ACTIVITY .................................................................................................................................. 56


List of Tables: TABLE 1: BASIC UNIFIED CLOUD API CALLS ........................................................................................................... 43 TABLE 2: UNIFIED CLOUD API EXTENDED FUNCTIONALITY CALLS ........................................................................ 44


Executive Summary The present document is Deliverable 5.2 “PaaSport Marketplace Infrastructure – First Release” (henceforth referred to as D5.2) of the PaaSport project. This deliverable documents the technical progress that has been achieved during the development period of the first release of the PaaSport platform. PaaSport architecture is a multi-layered one, therefore several components that correspond to these layers have been developed until M18. The developed components are classified as back-end components and front-end components. The goal of the project during the first half of the second year was to achieve progress in all components simultaneously. However, it can be argued that some components have been finalized. Specifically, these components include the matchmaking component and the repository component. The matchmaking relies on a specific algorithm that has been implemented leveraging semantic technologies. In order to do so, specific concepts of the relational database have been ‘uplifted’ to ontological concepts in order to make use of existing reasoning engines. Since these components are considered quite mature, an extensive documentation is provided for them in respective chapters. Moreover, this deliverable introduces the reader to the various graphical environments that have been developed per different role. Intuitive interfaces per role that have been defined in the PaaSport ecosystem are designed. The majority of them have already been implemented based on the State Of The Art Angular1 technology. The developed interfaces per role are also documented in this deliverable.

Finally this document describes and documents the integration progress of PaaSport platform. This process has been described in the PaaSport deliverable D5.1[11]; yet in the current deliverable some statistics that refer to the development process are analysed. The purpose is to highlight the successful agile paradigm that has been adopted prior to the initiation of the development.

1 https://angularjs.org/


1 Introduction The PaaSport project aims to create a marketplace where on the one hand, different PaaS providers can advertise their cloud offerings and on the other hand developers can benefit from a PaaS abstraction layer in order to avoid the vendor lock-in problem. The goal of the first development cycle that took place between M12 and M18 was to achieve progress in all layers. Moreover, PaaSport project focuses on resolving the data and application portability issues that exist in the Cloud PaaS market through a flexible and efficient deployment and migration approach. The vision of the PaaSport project is to enable Cloud vendors to roll out semantically interoperable PaaS offerings while benefiting European Software SMEs by allowing them to deploy or migrate business applications on the best-matching Cloud PaaS offerings.

1.1 Document Scope In order for all the functional requirements to be achieved, PaaSport implements a semantically-enhanced Cloud-broker architecture. This architecture consists of multiple layers that have been analysed in PaaSport deliverable D5.1[11]. This deliverable is a complementary one since it actually provides the development status of the various layers. Emphasis is also given to the various changes or minor deviations that occurred during the implementation phase. Furthermore, the purpose of the document is to also introduce the reader with the user experience of the marketplace through screenshots of the user interface. Moreover, as far as back-end functionality is concerned, specific emphasis was provided during the first period in the finalization of the matchmaking algorithm, the implementation of the relational and semantic artefacts that are required and monitoring. Specific analysis for all these artefacts is provided. Finally, the purpose of this document is report on the development lifecycle that is being followed by the R&D performers.

1.2 Document Structure The document consists of six (6) main chapters. The first chapter is the introduction to this document. The second chapter provides an overview of PaaSport Marketplace by exploring the already developed functionalities. The third section provides a high level technical overview of the platform while further details on each layer regarding the status and technical implementation are provided in chapter four. In chapter five, information about the deployment of the first release along with an analysis of the methodology followed in order to support continuous delivery of the platform is provided. Finally, before concluding, the document presents some analytics related to the development of the PaaSport platform.


2 Overview of PaaSport Marketplace This chapter provides an overview of PaaSport Marketplace by describing the functionalities that are offered in the first release. More functionalities are under development and will be included in the final release of PaaSport Marketplace. A specific video that highlights the existing functionality is provided at the following url: https://www.youtube.com/watch?v=zZIRJAfp6ug .

2.1 Unauthorized user The first page that an unauthorized user sees is the welcome page. Welcome page provides information about the project, and also links for the login and registration forms. These pages are depicted in Figure 1, Figure 2, Figure 3 and Figure 4 that follow.

Figure 1: PaaSport Marketplace welcome page

https://www.youtube.com/watch?v=zZIRJAfp6ug


Figure 2: Login Page

Figure 3: PaaS provider registration page


Figure 4: Developer registration page

2.2 Developer Dashboard Many of the most important PaaSport Marketplace functionalities are accessible through the developer dashboard. Developer dashboard is displayed only after a successful registration and login of a developer account (Figure 5). Offered functionalities can be grouped under application matchmaking and deployed applications management.

Figure 5: Developer main page

Developers using PaaSport are provided with the capability of of matchmaking an application in order to find the best matching PaaS providers and deployment to a specific PaaS provider. For this reason developer provides a name to the application and selects the required services of the application (Figure 6). For each service that is added, service parameters based on the user needs are defined, and the matchmaking request is created and is ready to be submitted, as depicted in the rightmost part of Figure 6.


Figure 6: Creation of a matchmaking request for an application.

After submitting of the matchmaking request, PaaS providers are suggested, sorted by terms of better matching to application needs, as displayed in Figure 7.

Figure 7: Viewing recommended PaaS providers for an application.

Finally, after user has selected the PaaS offering of his choice, deployment takes place, as presented in Figure 8.


Figure 8: Deployment of an application.

After a successful matchmaking and deployment, the application can be viewed in the deployed applications list as shown in Figure 9. The management of application lifecycle (Start, Stop and Delete) functionalities are also provided in this first release of PaaSport Marketplace.

Figure 9: Management of deployed applications application

Monitoring of the application is also supported with a plethora of available monitoring metrics that user can select (Figure 10). More details on PaaSport monitoring system can be found in chapter 4.3.


Figure 10: Monitoring of deployed applications.

In order to enable monitoring user has to include a specific PaaSport monitoring library to the deployed application by following the related instructions that are provided in the Developer Center, the part of PaaSport marketplace that gathers all information needed by developers (Figure 11).


Figure 11: Developer Help Center


Developers can also manage account setting. In order to have the ability to deploy an application to a specific PaaS provider, user has to provide the required keys. These keys are encrypted and stored to database, and are used from PaaSport Marketplace in order to achieve deployment through the offered PaaS provider API.

Figure 12: Adding keys for specific provider

2.3 Provider Dashboard A PaaS provider that joins PaaSport Marketplace has specific needs, like managing PaaS offering details, both in infrastructure and software level. A comprehensive list of PaaS provider requirements has been presented in PaaSport deliverable D1.2[4]. Although PaaS provider role has been created and the appropriate user menu has been created (Figure 13), the functionalities of PaaS provider are under development will be integrated in the final release of PaaSport Marketplace.

Figure 13: PaaS provider main page


2.4 Administrator Dashboard Administrator is role of PaaSport Marketplace that is responsible for the management of administrative tasks. The basic tasks of PaaSport administrator are to manage users (Figure 14) and Services (Figure 15).

Figure 14: User management

Services are available for the description of PaaS offerings, and they are organized in various categories. Moreover, each service is characterized by a set of parameters that are mainly being used by the application matchmaking algorithm so as to provide recommendations of the most suitable PaaS offerings.


Figure 15: Services management

Administrator can also manage the available service categories (Figure 16) and service parameters (Figure 17).

Figure 16: Service Category management


Figure 17: Service Parameters Management

Infrastructural parameters and offering models management through PaaSport Marketplace UI is under development.


3 Overview of Revised Architecture The purpose of this section is to provide an overview of the architecture in order to make the technical understanding of PaaSport Marketplace easier. More details about architecture can be found in PaaSport deliverable 1.2 [4]. Instead of putting emphasis in the various changes/amendments that have been occurred, a holistic view of the architecture is provided.

3.1 Overview of the Reference Architecture PaaSport Marketplace constitutes a thin, non-intrusive broker that mediates between PaaS offerings and PaaS users. It relies on open standards and introduces a scalable, reusable, modular, extendable and transferable approach for facilitating the deployment and execution of resource intensive business services on top of semantically-enhanced Cloud PaaS offerings. PaaSport Reference architecture was consolidated in PaaSport deliverable 1.2 [4] and it is also illustrated in Figure 18.

Cloud Marketplace Catalogue

DevOps Engineer personalised space

PaaS provider personalised space

PaaS model

SLA model

Application model

PaaS offering selection

PaaS offering shortlist

PaaS offering search

PaaS offering rating

Semantic Query handling

Semantic PaaS offering discovery

Application to PaaS offering matchmaking

Semantic models

Adaptive Front-ends

PaaS Offering Recommendation Layer

User profilesPaaS offering

profilesApplication

profiles

Search and Discovery Interfaces

Tunnelling and Virtual Execution

PaaSport Unified PaaS API

Persistency, Execution and Coordination Layer

Monitoring and SLA

Enforcement Layer

Orchestration

Deployed application monitoring

PaaSport Adapter

PaaSport Adapter

SLA Matchmaking

SLA Enforcement

Interoperability Libraries

Figure 18: High-level view of the PaaSport Marketplace Architecture

PaaSport Marketplace comprises of the following five layers:

1. The PaaSport Semantic Models that serve as the conceptual and modelling pillars of the marketplace infrastructure, for the annotation of the registered PaaS offerings and the deployed applications profiles;

2. The PaaS Offering Recommendation Layer that implements the core functionalities offered by the PaaSport Marketplace Infrastructure, such as PaaS offering discovery, recommendation and rating;

3. The Monitoring and SLA Enforcement Layer that realizes the monitoring of the deployed business applications and the corresponding Service Level agreement;


4. The Persistency, Execution and Coordination Layer that puts in place the technical infrastructure, e.g. repositories, on top of which the PaaSport marketplace is built. It also includes the PaaSport Unified PaaS API that is a common API exploited in order to uniformly interact with the heterogeneous PaaS offerings and, in addition, it realizes the lifecycle management of the deployed applications.

5. The Adaptive Front-ends that support seamless interaction between the users and the PaaSport functionalities, through a set of configurable utilities that are adapted to the user’s context;

In chapter 4 all five layers are described with focus on the technical achievements and the functionalities each layer currently offers in the first release of PaaSport Marketplace. The planned work and roadmap of each layer is also provided. All components are mapped in MAVEN components which are available at: https://github.com/ubitech/paasport.

https://github.com/ubitech/paasport


4 Features of the First Release of PaaSport Marketplace Infrastructure

This chapter presents the current status of each layer of PaaSport Marketplace. Following analysis is mostly focused on the technical details of each component that together consist the PaaSport Marketplace and offer the features of first release.

4.1 Semantic Model The PaaSport Semantic Models are the conceptual and modelling pillars of the marketplace infrastructure and they are used in order to provide a semantic annotation means for the registered PaaS offerings and the deployed applications profiles. Specifically, the roles that the PaaSport Semantic Models play in the various modules of the PaaSport platform are the following:

They provide a common vocabulary for the various modules of the system.

Concerning the PaaSport Offering Recommendation layer, matchmaking of application requirements to PaaS offerings is performed using concepts and properties of the offering and application Semantic Models.

Concerning the Persistence layer, the database schema follows exactly the conceptual model of the ontologies, in order to avoid syntactic/semantic mismatch between tables/concepts and attributes/properties when the Offering Recommendation layer retrieves PaaS offerings from the database, based on the Semantic Models. The PaaS Offering profiles stored in the database are mapped onto the Semantic Model layer (RDF graph) using a relational-to-ontology mapping tool, namely the D2RQ2 platform.

Concerning the adaptive front-ends layer, the UIs for managing application and PaaS offering semantic profiles use concepts and properties from the semantic models.

Concerning the monitoring and SLA layer, the SLA model has been considered for defining the PaaSport SLA policy model used by the SLA Enforcement component, while the monitoring system has been designed so as to support the metrics defined in the semantic SLA model.

In the following sub-sections, an overview of the PaaSport semantic models is provided. A more detailed description can be found in PaaSport deliverable 1.3 [5].

4.1.1 DOLCE Extensibility Model The core model of the ontology has been developed as a specialized instantiation of the descriptions and situations (DnS) pattern that is part of DOLCE+DnS Ultralite. DnS does not put restrictions on the type of entities and relations that one may want to represent, either as a domain specification, or as an upper ontology, and it allows for context-sensitive “redescriptions” of the types and relations represented by other ontologies [3]. The core modelling pattern of DnS allows the representation of the following concepts:

2 http://d2rq.org/


Situations. A situation defines a set of entities that are involved in a specific pattern instantiation and they are interpreted on the basis of a description. Each situation is also correlated with one user/agent.

Descriptions. An activity description serves as the descriptive context of a situation, defining the concepts that classify the entities of a specific pattern instantiation, creating views on situations.

Concepts. The DUL concepts classify domain entities describing the way they should be interpreted in a particular situation. Each concept may refer to one or more parameters, allowing the enrichment of concepts with additional descriptive context.

DnS is used to formally provide precise representations of contextualized situations and descriptions on Offerings, Applications and SLAs (Figure 19). The Offering Pattern is represented through the combination of the OfferingModel and GroundOffering classes, the Application Pattern is represented by the ApplicationRequirement class, and finally, the SLA Pattern is represented by the SLATemplate class.

DnS Pattern

Core PaaSport Pattern

Offering Pattern

Application Pattern

SLA Pattern

extension

extension

extension

extension

Figure 19: DnS pattern

The extensibility of model depends on the parameters’ definition (Figure 20). A situation (offering, application, SLA agreement) could have a set of concepts (description). A concept (for example QoS) could have one or more parameters. Every parameter has a value (dul:hasParameterDataValue) and a “quality” (dul:parametrizes), which is the “physical” or “logical” dimension of the parameter (e.g. storage, duration, etc) and it is usually (not always) accompanied by measurement units. For this reason, the quality of the value is known comparison of two parameters and by extension two concepts is possible. Figure 20 displays the association between DUL and the PaaSport model.

PaaSSituation

DUL:Satisfies[allValuesFrom]

PaaSConcept

muo:QualityValue

PaaSParameter

dul:hasParameter[allValuesFrom]

rdfs:Literal

PaaSDescription

DUL:Defines[allValuesFrom]

DUL:Situation PaaSDescription DUL:Concept DUL:Parameter

dul:hasParameterDataValue

[allValuesFrom]

dul:Parametrizes[allValuesFrom]

Figure 20: DUL and PaaSport association


In order to add a new parameter to a concept the quality value of this parameter has to be declared. The same applies for concepts.

4.1.2 Offering Model The Offering Model helps PaaS providers semantically describe their PaaS offerings. Specifically, it contains all available characteristics of a PaaS offering. These characteristics are part of the PaaSConcept hierarchy and can be technical, performance, geographical etc. The core entities of the model are the PaaS Offering and its description (Figure 21). Moreover, the PaaSGroundedOffering is a subclass of PaaSOffering, referring to the existing PaaS offering. This offering is related to software-based, hardware-based, QoS, location and pricing policy parameters. Two classes are defined for the offering description:

1. The OfferingModel represents the description of an offering model, i.e. a preinstalled PaaS software instance, e.g. “openshift origins”. The offering model description describes general attributes of a model, such as programming language, databases, server etc. These attributes are common in every PaaS offering with a certain installation.

2. The GroundOffering represents the description of a grounded PaaS offering and consists of PaaS concepts such as programming environment, services, resources, QoS, location, pricing etc. The resource, QoS, Pricing and location concepts define when an Offering Model becomes a GroundOffering (necessary and sufficient conditions). In practice, when a user creates a new offering by grounding an existing offering model, a new instance of this class will be created by copying all the concepts of the model (service and programming environment).

Figure 21: Offering Model overview

4.1.3 Application Model The Application Model encapsulates definitions for classes used to capture knowledge related to Cloud-based applications by using a simple, open and extendable vocabulary, which allows the semantic annotation of developers’ applications.


Figure 22: Application Model overview

Based on the Application Model and the PaaSConcept Model (Figure 22), the Cloud-based application developer creates and manages the semantic profile of his applications. The developer can define functional (programming language, servers, database etc.) and non-functional (performance, price etc.) characteristics of his application. Specifically, these characteristics refer to the deployment environment and allow developers to match the PaaS Offerings that are the most relevant to their applications.

4.1.4 SLA Model SLA (Service level agreement) is an agreement between two parties, the Service Consumer (the PaaSport user who deployed an application) and a PaaS Offering Provider. The level of service is formally defined in terms of performance and reliability. The SLA has a period of validity that is defined in terms of the StartDate and EndDate properties. The performance is described by QoS parameters and the pricing by the pricing policy parameters, similarly to the QoS and pricing parameters of PaaSConcept (Figure 23).

Figure 23: SLA Model overview


4.1.5 Roadmap The current status of the PaaSport semantic models is that they are finalized. So, there is no work to be done, apart from maintenance issues that might arise in the integration and testing phase of the PaaSport platform.

4.2 PaaS Offering Recommendation Layer The PaaS Offering Recommendation Layer is responsible of filtering and recommending the best suitable PaaS Offerings. Offerings recommendations are based on the degree of similarity between the applications deployment profiles and the available PaaS characteristics, which is computed based on the similarity of their semantic descriptions.

Recommendation Layer has been developed as part of the PaaSport Cloud-broker Architecture specification. It interacts with PaaSport semantic models described in section 4.1 and also with the Persistence, Execution and Coordination Layer described in 4.4 via the search and discovery interfaces.

Figure 24: PaaS Offering Recommendation Layer Components

PaaS Offering Recommendation Layer includes eight main sub-components that are depicted in Figure 24 and are:


1. PaaS Offering Search 2. Semantic Query Handling 3. Semantic PaaS Offering Discovery 4. Application to PaaS Offering Matchmaking 5. PaaS Offering Shortlist 6. PaaS Offering Selection 7. SLA Matchmaking 8. PaaS Offering Rating

All components except SLA Matchmaking component and PaaS Offering Rating component have been already developed in the first release of PaaSport Marketplace and presented in following subsections.

4.2.1 PaaS Offering Search Component PaaS Offering Search component is an interface that receives software SMEs application requirements and sends these requirements to the semantic query handler. The semantic query handler component continue with the rest of the process through interacting with the semantic PaaS offering discovery, application PaaS offering matching and PaaS Offering Shortlist components. This component is invoked by the SearchPaaSOfferingWidget and returns list of the matched offers combined with scores.

4.2.2 Semantic Query Handling Component The semantic (SPARQL) query-handling component is the main component of the Offering Recommendation Layer and it controls all the interaction between all the other components. It receives the application requirements of the Software SMEs Engineers from the “PaaS offering search” component and performs accordingly one of the following tasks:

1. It triggers the “Semantic PaaS offering discovery” component to get all PaaS offerings according to the user preferences that come from the user profile.

2. It sends PaaS offerings along with the application requirements to the “Application to PaaS Offering Matchmaking” component which is responsible to perform the matchmaking based on the matching algorithm and return a list of the matched offers to the semantic query handling.

3. It sends a list of the matched offerings to the PaaS Offering Shortlist component where the matched offerings are combined with scores based on the offering scoring algorithm.

4. It sends the scored matched offerings to the “PaaS offering search” component where a response is generated and sent to the SearchPaaSOfferingWidget.

4.2.3 Semantic PaaS Offering Discovery Component The PaaS Offering Discovery component receives the semantic query from the “semantic query handling” component. This component applies two different search methods in order to find among the available PaaS offerings the ones that meet best the user’s requirements. The PaaS Offering Discovery component search is based on the semantic profile of the


application and also on manually inserted searching criteria. It is responsible to get SLA offers through matchmaking with the PaaS Offerings that meet the user’s requirements.

Semantic PaaS offering discovery component interacts with the repository layer, SLA matchmaking component and semantic query-handling component in order to achieve the required functionality.

4.2.4 Application to PaaS Offering Matchmaking Component The PaaS offering matchmaking implements the semantic matchmaking between the application requirements and the available PaaS offerings. The matchmaking criteria are flexible and configurable thus allowing the Software SMEs Engineer to decide in a range between exact (full) match and partial matches. This component is invoked by the semantic query-handling component and is the base of PaaSport matchmaking.

4.2.4.1 PaaSport Recommendation Algorithm and Model At the heart of the PaaS Offering Recommendation Layer there is a recommendation algorithm that selects and scores-ranks the most appropriate PaaS offerings that best match the requirements of the application a developer wants to deploy. In the following section the recommendation algorithm is briefly described. A more detailed description can be found at PaaSport deliverable D2.1 [6].

Figure 25: Overview of The PaaSport Matchmaking and Recommendation algorithm

This algorithm, which is pictorially presented in Figure 25, takes as input the application requirements instance as defined by the DevOps Engineer from the GUI and returns as


output the list of offerings that are compatible with the functional requirements, sorted in descending order according to the scoring function, which is based on the non-functional requirements. The algorithm initially retrieves the IDs of all offerings stored in the Persistency Layer and stores them in-memory into the offerings list. Then, there are two main loops in the algorithm that implement the two steps mentioned above:

1. Matchmaking using Functional Parameters and 2. Ranking using Non-Functional Parameters

The first step of the algorithm, where the offerings are filtered according to whether they satisfy the functional parameters or not, consists of three nested loops. The first loop iterates over all PaaS concepts of the Application Requirement (retrieved by the GUI), the second loop iterates over all offering entries in the candidate offerings set and the third loop iterates over all functional parameters of the currently examined application’s concept. An appropriate SPARQL ASK query checks if the offering satisfies the functional parameter or not. The second step of the algorithm scores and ranks the offerings that satisfy the functional parameters, using the non-functional parameters. It consists of the following sub-steps:

1. Retrieval of the values of the non-functional parameters 2. Calculation of the score of each offering 3. Sorting of the offerings in descending order of their score

The scoring function for each non-functional parameter actually tries to measure how far the value of the current offering is from the worst offering (for this parameter), so that offerings can be compared in a fair manner according to this parameter. Furthermore, in order for the comparison to be fair across all parameters, this function always returns a value between 0 and 1. Of course, since each parameter counts differently according to the DevOps engineer, the value of this scoring function is multiplied by a weighing factor. The score of an offering for a parameter is given by the following function:

Eq. 1 where OffParVal is the value of the parameter for the offering, AppReqParVal is the required value for the parameter by the application, wpar is the coefficient of the parameter, and fdfpar is the difference function selected for the parameter. In case the value of the parameter for

the offering is unbounded () it means that it should be assigned the maximum scoring value (1). On the other hand, if the value of the offering parameter is less than what the application requires, then it is assigned the minimum scoring value (0). In any other case, the score of the difference function is multiplied by the corresponding weight. The sum of all weights for all parameters equals to 1. We support the following difference functions, which are better illustrated in Figure 26:

Flat Scoring Function:


Linear Scoring Function:

Super-linear Scoring Function:

Sub-linear Scoring Function:

Spike Scoring Function: . The total score of the offering is a sum over all parameters of the offering that are requested by the application request:

Figure 26: The different functions supported by scoring function

4.2.5 PaaS Offering Shortlist Component The PaaS offering shortlist component provides suggestions of PaaS offerings regarding the application that the Software SMEs Engineers want to deploy. This component receives the results of the matchmaking component through semantic query handling component. It is envoked by the semantic query-handling component and it is mainly based on the offering scoring algorithm. The output of this component is a list of scored matched offers, which is send back to semantic query handling component, and consequently to the user interface as depicted in Figure 7 in Section 2.

4.2.6 PaaS Offering Selection Component PaaS Offering Selection Component is serving as an interface for the PaaS Offering Recommendation Layer that interacts with the semantic query handling component. It passes a list of ranked PaaS offerings and a list of selection criteria as inputs and returns a selected offering. The Offerings Selection mechanism utilizes lightweight semantic models

and multiple criteriain order to support the user to make his final decision and selection with regard to the most suitable PaaS offering.


4.2.7 Interaction with the Persistency Layer The Recommendation layer and the matchmaking algorithm interact with the persistency layer in order to:

1. retrieve the PaaS offerings stored in the PaaSport Marketplace database 2. get the application requirements that the DevOps engineer has posted through the

UI Figure 27 shows the workflow of data from the persistence layer to the recommendation layer, concerning the inputs needed for the matchmaking/recommendation algorithm. The PaaS offerings are stored in the relational database of the persistency layer and they are mapped to RDF data, using the concepts and properties of the Semantic Models, using the D2RQ platform presented in section 4.4.1.4. Then the offerings are fed into the matchmaking algorithm. After that, the application requirements are queried from the persistency layer and they are used to construct an application object that is also used as input of the matchmaking/recommendation algorithm, which then proceeds as already described above.

Matchmaking Algorithm

Mapping File

Offering Repository

D2RQ scriptRead

OntologyRead Application

ObjectExecute the Algorithm

Application Repository

Ontology of the offerings

Scored Offerings

Figure 27: Workflow of Data for the Recommendation layer

In order to implement the above functionality three custom java classes have been created which are needed for the matchmaking algorithm: ApplicationInstance, Concept and Parameter. ApplicationInstance is the class that describes the application and that class has some instances of concepts. The Concept class has a type and two sets of parameters, a set of functional parameters and a set of non-functional parameters. Finally, the class Parameter has some appropriate data such as type, value etc.

4.2.8 Roadmap The presented components have been implemented and the matchmaking of applications is possible in the first release of PaaSport Marketplace. Some changes may occur before the final release of PaaSport Marketplace and the two components of SLA Matchmaking and PaaS Offering Rating will be included.


4.2.8.1 SLA Matchmaking Component The SLA Matchmaking Component receives its input from application PaaS offering matching component. This input contains the application requirements, while output provides the SLA matchmaking results between the application requirements and the multi-parametric characteristics of the PaaS providers that offer the necessary SLA.

4.2.8.2 PaaS Offering Rating Component The PaaS offering rating component facilitates the rating of a particular PaaS offering by a Software SMEs Engineer. Each engineer can leave comments and rate particular PaaS offerings, thus expressing their satisfaction or dissatisfaction with regards to quality, usability, reliability and good user support.

4.3 Monitoring and SLA Enforcement Layer The Monitoring and SLA Enforcement layer is in charge of SLA enforcement (detection of SLA violations) by collecting monitoring metrics from both PaaS applications and platforms. The Monitoring and SLA Enforcement layer consists of the following two components: (i) the SLA Monitoring; and (ii) the Deployed Application Monitoring. Efforts so far focused on the definition of the PaaSport SLA policy model; and the development of the PaaSport monitoring system.

4.3.1 SLA Monitoring Component For the development of the SLA Monitoring Component, the innovative PaaSport Service Level Agreement Policy Model has been defined and documented in PaaSport deliverable 3.1 [7]. PaaSport SLA policy model is used for the instantiation of Service Level Agreements (SLAs) among the PaaS offering providers and the end-user customers who deploy their applications using the PaaSport Marketplace Infrastructure. The proposed SLA Policy Model is PaaS provider agnostic and is compatible to the SLA Model developed under Task 1.3. It is leveraged by the SLA Matchmaking component of the PaaS Offering Recommendation Layer to instantiate the SLAs among the PaaSport users and the PaaS providers. The SLA matchmaking process is based on WS-Agreement protocol.

Additionally, the PaaSport SLA policy model enables the user to define its own custom metrics for which he would like to get notified when a specified criteria, i.e. above a threshold, is not met. The SLA Monitoring Component depicted in Figure 28 consists of two sub-components, namely SLA Enforcement and SLA Violations Handler.

Figure 28: SLA Monitoring Component

SLA Monitoring

SLA Enforcement

SLA Violations Handler


Implementation of the SLA Monitoring Component is expected to be completed and integrated in the second release of PaaSport Marketplace during the upcoming months. The functionalities of the current version of the SLA Enforcement component are limited to detecting SLA violations on simple metrics. Furthermore, the SLA Violations Handler has been designed and it is under development. It currently provides a subscription mechanism that enables every interested component to receive alerts/notifications for SLA violations.

4.3.2 Deployed Application Monitoring Component The Deployed Application Monitoring Component is mainly based on the PaaSport Monitoring Service. The PaaSport Monitoring Service is utilized to collect monitoring metrics from deployed PaaS applications and platforms, and share them with the other PaaSport components such as the SLA Monitoring component and the User Interface. Monitoring results are also offered to the user though a specific UI, as displayed in Figure 10 that has been provided in Section 2. The figure below depicts an abstract overview of the architecture of the PaaSport Monitoring Service.

Figure 29: Architecture of the PaaSport Monitoring Service

Monitoring Agents are light-weight monitoring instances embedded in PaaS applications. Monitoring Agents are responsible for collecting metrics from both the PaaS application and the underlying PaaS container. An agent can be configured to monitor, besides the built-in metrics such as CPU and Memory load, custom user- and application- tailored metrics. This configuration can be achieved by either loading monitoring probed from the PaaSport Monitoring Library or by using the respective API of the monitoring library. To embed a Monitoring Agent to a PaaS application, a developer is only required to add the Monitoring Agent dependency to his respected Dependency Manager (e.g. Maven for a J2EE application), as follows:


Figure 30: Sample code for inclusion of Monitoring Agent through Maven

After downloading the Monitoring Agent artifacts via the Dependency Manager, the Monitoring Agent is ready to start monitoring, upon deployment, the PaaS application and the underlying container. However, one simple stage remains: the developer, for security reasons, must add via his application configuration file, his Monitoring API key, provided upon registration to the PaaSport Platform. Moreover, users can optionally configure advance settings, such as logging, monitoring agent naming, and collecting periods for various metrics, as depicted in the figure below. Additionally, NO changes at source code level are required by the developers.

Figure 31: Sample code for inclusion of Monitoring Agent as filter on application

Monitoring Servers are responsible for receiving, processing, storing and exposing to users via a rich REST API, monitoring metrics. The Monitoring Server tier forms a completely distributed and de-centralized tier of the Monitoring Service, but is exposed to users, as well as PaaSport components, as a central endpoint to ease interaction. Monitoring Server mechanisms are PaaS-platform agnostic, as none of the collecting, distributing or storage mechanisms depend on the underlying platform or infrastructure. As one of the greatest novelties of the Monitoring Service, is that the complete monitoring infrastructure is self-adaptive and self-manageable, offering high-availability, fault-tolerance and recoverability. Specifically, when a Monitoring Server crashes Monitoring Agents distributing metrics to it will not become unavailable and, in turn, the topology will automatically re-configure itself, with Monitoring Agents assigned automatically to other Monitoring Servers. Moreover, scalability is a feature of the service, dynamically adjusting to the imposed workload in an


elastic manner to keep consumers satisfied when the workload increases and reduce costs by releasing resources when not required.

4.3.3 Roadmap Currently, the exposed functionalities of the Monitoring and SLA Enforcement layer meet the requirements of the first release of the PaaSport marketplace. In the following months we focus on extending the SLA Enforcement component to support more complex SLAs and metrics as well as the implementation of the SLA Violation subscription mechanism. Concerning the Application Monitoring Component we focus on: (i) enriching the Monitoring Probes Library; (ii) adoption of adaptive filtering and sampling at the container level so as to dynamically adapt the monitoring process to the current workload resulting in minimum communication, storage and computation overhead; (iii) incorporate security mechanism that will provide secure monitoring by establishing secure channels for distributing monitored metrics. Finally, other improvements include the performance and scalability evaluation of both the SLA Monitoring and Deployed Application Monitoring components, which’s development ends by M24 of the project.

4.4 Persistency, Execution and Coordination Layer The Persistency, Execution and Coordination layer is responsible for the storing of all data needed for the proper function of PaaSport Marketplace and also for the encapsulation of functionalities that allow the multi-cloud deployment and management of applications.

4.4.1 Persistency Layer Repository layer is used in order to persist the various PaaSPort models that are mapped to the semantic model and other entities that are needed for the proper function of PaaSPort Marketplace, while also offering appropriate search and discovery interfaces that allow the usage of persisted information from other components.

4.4.1.1 Database Description Relational database is the main component of persistency layer and is used to store the data that are necessary for the operation of the platform. The database used by PaaSport Marketplace is presented using the Entity–Relationship model (ER model), a data model commonly used for describing the data or information aspects that are implemented in relational database. As seen in Figure 32 it consists of 31 tables that store information related to users, PaaS offerings and applications.


Figure 32: Database Tables of PaaSport Marketplace

The repository that is constructed by these tables contains the three main types of data objects that PaaSport Marketplace needs to store: a) the PaaS Offering Profiles constituting the semantic profiles of the PaaS offerings advertised in the PaaSport Marketplace; b) the Deployed Application Profiles constituting the semantic profiles of the deployed business software applications; and c) the User Profiles constituting the semantic representation of the profiles that Software SMEs Engineers and PaaS Providers maintain on the PaaSport Marketplace. In the following sections we present the way that these tables are used.


4.4.1.2 Adoption of Object-Relational Mapping (ORM) scheme Object-relational mapping (ORM) is a programming technique in which a metadata descriptor is used to connect object code to a relational database. Object code is the core of programs made in Java language that is used in PaaSport and an ORM converts data between type systems that are unable to coexist within relational databases and OOP languages. For PaaSport, Hibernate3 framework has been used for connecting object code to the relational database. For the implementation of ORM scheme on PaaSport with Hibernate, a class has been created for each table of the created database, as shown in Figure 33. For each of table that it is mapped to a class, the table columns have been also correlated to java variables of the appropriate type.

Figure 33: ORM objects

3 http://hibernate.org/


4.4.1.3 Search and Discovery Interfaces Repository layer also provides the appropriate interfaces that are exploited for the usage of relational database from other components of PaaSport marketplace. For the creation of interfaces of all entities of PaaSport database we used Spring Data4 as it auto-generates SCRUD (Search Create Retrieve Update Delete) business logic based on existing JPA annotations. This technology allowed us to avoid all boilerplate code during implementation since the SCRUD-SQL-business logic is inherited and auto-generated by the core JPARepository as illustrated in Figure 34.

Figure 34: Inherited Repository methods

As methods are inherited from interfaces CrudRepository, JpaRepository, PagingAndSortingRepository of Spring Data, by default all model created support the following methods:

Methods inherited from interface CrudRepository o count, delete, delete, delete, deleteAll, exists, findOne, save

Methods inherited from interface JpaRepository o deleteAllInBatch, deleteInBatch, findAll, findAll, findAll, flush, getOne, save,

saveAndFlush

Methods inherited from interface PagingAndSortingRepository o findAll

This functionality is used for the creation Data Access Objects (DAO) logic and is separated from the ORM model of the entities. The DAOs that have been already created and offer the needed Search and Discovery Interfaces are presented in Figure 35.

4 http://projects.spring.io/spring-data-jpa/


Figure 35: Repository Interfaces available in First Release of PaaSport Marketplace

4.4.1.4 The relational-to-RDF mapping A crucial aspect of the repository layer is the ability to expose a specific part of its data in RDF format in order for the semantic matchmaking to take place (Figure 36), in a process called as RDF-ization. Data stored in relational systems can be extracted via queries, stored procedures, or any other process that will extract the data from the database. Table columns and rows have to be mapped to concepts and attributes defined in an ontological model.

Figure 36: Relational to RDF mapping


D2RQ mapping language is used to achieve the RDF-ization with mapping between offering profiles for the matchmaking algorithm.

4.4.1.4.1 Description and Examples of a Real Mapping D2RQ language is also used to create all essential mapping rules that are stored in a file called paasport-mapping.ttl. Every time that a PaaS provider inserts a new offering instance in the database, a script is responsible to recreate the PaaSport ontology file (paasport.ttl). The D2RQ language can connect RDF triples to database tuples and attributes. At the example below (Figure 37), the mapping rule for the grounded offerings is presented. First of all, there is a rule for connecting to the database, so it defines the port of the database endpoint, username and password. After that, a mapping rule is defined, with the name of the rule-triplet, the defined data storage, the URI pattern of the triplets and the name of the class. Thus, for every tuple in the table groundedpaasoffering the mapping rule creates a new triplet.

paas:database a d2rq:Database;d2rq:jdbcDriver "com.mysql.jdbc.Driver";d2rq:jdbcDSN "jdbc:mysql://127.0.0.1/paasport";d2rq:username "paasport";d2rq:password "!paasport!";jdbc:autoReconnect "true";jdbc:zeroDateTimeBehavior "convertToNull";.

paas:Offering a d2rq:ClassMap;d2rq:dataStorage paas:database;d2rq:uriPattern "http://paasport-project.eu/ontology/paasport #offering_@@groundedpaasoffering.name@@_@@groundedpaasoffering.id@@";d2rq:class paas:Offering;.

Figure 37: D2RQ example; connecting to the database and mapping a table

In Figure 38a property mapping example is presented. First, there is a rule that creates the property domain class (GroundOffering class). After that, there is a rule that creates the property range class (ExecutionContainer class). Finally, there is a rule that makes the property mapping, in our case, the property paas:offers. The connection is based on a join between the tables of the two classes above on the id attribute. Thus, every GroundOffering instance connects to a corresponding ExecutionContainer instance through the property offers, based on the id of the groundpaasoffering tuple.


paas:GroundOffering a d2rq:ClassMap;d2rq:dataStorage paas:database;d2rq:uriPattern "http://paasport-project.eu/ontology/

paasport#description_@@groundedpaasoffering.name@@_@@groundedpaasoffering.id@@";

d2rq:class paas:GroundOffering;.

paas:infrastructuralservice a d2rq:ClassMap;d2rq:dataStorage paas:database;d2rq:uriPattern "http://paasport-project.eu/ontology/

paasport#ExecutionContainer_@@executioncontainer.id@@"; d2rq:class paas:ExecutionContainer;

.

paas:offersinfrastructure a d2rq:PropertyBridge; d2rq:belongsToClassMap paas:GroundOffering; d2rq:property paas:offers; d2rq:refersToClassMap paas:infrastructuralservice; d2rq:join "groundedpaasoffering.id => executioncontainer.groundedpaasofferingid"; .

Figure 38: D2RQ example; mapping a property.

4.4.2 Execution Layer Execution layer is responsible for the cross-platform deployment and management for PaaSport. Its main parts are the Unified Cloud API that is an API of methods that are common among PaaS offerings and allows the cross-platform deployment of applications, and the PaaSport Adapters that are reference implementation of PaaSport Unified Cloud API for a specific PaaS offering. Moreover these components are supported from the Application Lifecycle Management component, the PaaSport Portability Libraries and the PaaS Offering Management component.

4.4.2.1 Unified Cloud API Unified Cloud API provides a common interface for interaction with different PaaS providers. As documented in PaaSport deliverables 4.1 [9] and 4.2 [10], the common PaaS API characteristics and entities have been identified and contained in the Unified Platforms Interface Model and Unified Cloud API.

The Unified PaaS API integrates the application management, the user management, container and deployment package management, service management, service bindings, domain management, and policy management. The management of each part contains the CRUD functionality, and additional services as necessary as the monitoring and scaling for application. The basic methods provided by the interface and are needed to be supported from a PaaS offering that it is part of PaaSport are presented in Table 1 that follows.

Table 1: Basic Unified Cloud API calls

Basic Methods (Required for PaaSport Inclusion)

Create Application

Retrieve Application

Delete Application

Manage Application


Create Deployment Package

Update Deployment Package

Retrieve Service

Get List of Available Services

Create Service Binding

Delete Service Binding

Additionally, the following functions presented in Table 2 are provided for essential management tasks of the PaaS instances controlled by the PaaSport Broker:

Table 2: Unified Cloud API extended functionality calls

Extended Methods (Might not be supported from all providers)

Update Application Create Deployment Package Retrieve Domain

Get List of Applications Retrieve Deployment Package

Update Domain

Scale Application Delete Deployment Package Delete Domain

Create User Create Service Get List of configured Domains

Retrieve User Retrieve Service Binding Create Security Policy

Update User Update Service Binding Retrieve Security Policy

Delete User Create Container Update Security Policy

Add Key Retrieve Container Delete Security Policy

Delete Key Get List of Bound Services Get List of configured Security Policies

List available Containers

Create Domain Monitor Application

Regarding the implementation of the Unified Cloud API, the methods have been defined in the IAdapter interface class. A sample part of this interface is provided in Figure 39.


Figure 39: Sample of methods specified in the IAdapter interface class

This class is real implementation of the Common PaaS Platforms Interface and can be used for the implementation of Adapters that are providing PaaS specific support.

4.4.2.2 PaaSport Adapters PaaS specific adapters are separate projects that implement the IAdapter Interface. For each method supported, adapter creator should implement the method and use PaaS specific API calls for the connection with the PaaS, as depicted in Figure 40.

Figure 40: Sample code from createApplication method on CloudFoundry Adapter

PaaSport Adapters that offer support for OpenShift and CloudFoundry based PaaS offerings have been already implemented and supported in the first release of PaaSport, while Heroku and AWS Beanstalk adapter implementation is under final testing and soon will be part of the PaaSport Marketplace. Implementation of Adapters for Apache Stratos and CloudControl offering has also started and the adapters will be part of the final release of PaaSport Marketplace.


4.4.2.3 PaaSport Dynamic Configuration Interface and PaaSport Portability Libraries PaaSport Dynamic Configuration Interface and the PaaSport portability libraries are used in order to assure portability of applications deployed to the cloud, and have been described with more details in PaaSport deliverables 4.1 [9] and 4.2 [10].

Portability refers to the ability of the Application to be ported from one container to the other and remain functional without any changes at the source code level. A blocking factor on the vision for cloud portability and interoperability is the binding of resources to the application. This issue is making the migration scenarios to be only limited to applications not using resources, or to make use of additional effort from the application developer in order to make the application compatible to the new PaaS.

PaaSport Dynamic Configuration Interface is based on the open source project of Spring Cloud Connectors5 and has been extended to support more PaaS offerings, like OpenShift and more services and variables that are available for each PaaS. In Figure 41 sample usage of PaaSport Portability libraries is provided.

Figure 41: Usage of PaaSport Portability libraries.

PaaSport Portability Libraries is a component that is totally independent from main PaaSport project but it has been developed in close relation with PaaSport Adapters, so portability libraries for Heroku, OpenShift and CloudFoundry based PaaS offerings have been already implemented. Creation of Portability libraries for AWS Beanstalk, Apache Stratos and CloudControl, is under research status.

4.4.3 Coordination Layer The coordination layer in the Persistency, Execution and Coordination layer serves is responsible for the management of offerings as well as the application lifecycle.

4.4.3.1 PaaS Offerings Management Component The designed mechanism exposes an API to the rest of the components for the management of PaaS offerings semantic models. It provides the ability to manage both the semantic profiles of PaaS offering model and PaaS offerings. Hence, it allows the reuse of PaaS offering models by various PaaS providers for creation of multiple PaaS offering.

5 http://cloud.spring.io/spring-cloud-connectors/


Additionally, it enables the creation of PaaS offering models and PaaS offerings and handles their properties and policies, i.e. SLA template. Furthermore, it enables the filtering of PaaS offerings based on custom filters in order to provide PaaSport components a mechanism for easy and transparent access to PaaSport repositories. The first release of the PaaS Offering Management component has been documented in PaaSport deliverable 3.2 [8]. The next release of the component will include more advanced filtering of the available PaaS offerings according to the user demands.

4.4.3.2 Application Lifecycle Management Component Under the development of the Application Lifecycle Management (ALM) component, the PaaSport application lifecycle has been defined. The defined application lifecycle must be strictly followed by all the applications in PaaSport system. ALM component uses the functionality provided by the PaaSport Unified PaaS API and it enables the creation, management and deletion of applications, while it guarantees that the applications are always in a consistent state. The first release of the ALM component has been documented in PaaSport deliverable 3.2 [8].

In the code implementation of PaaSport Marketplace, the main connection point of Unified Cloud API with Coordination layer is the PaaSInteractionProxyComponent. In PaaSInteractionProxyComponent core business logic is implemented in class PaaSInteractionProxyService that currently implements the proxying for the most important Application Lifecycle Management methods, with a sample code of this class provided in Figure 42.

Figure 42: Proxying of createApplication method

For better extendibility of the PaaSport Platform, the PaaS specific adapters are provided as external dependencies to the main project. This was made possible with the usage of the Service Provider Interface (SPI) design pattern in PaaSInteractionProxyService. For this reason the Adapter methods are called using Reflection6, a technique that is commonly used by programs which require the ability to examine or modify the runtime behaviour of applications running in the Java virtual machine. More technical details will be provided in the corresponding deliverable D4.3 that will describe the final release of Persistency and Execution layer.

6 http://docs.oracle.com/javase/tutorial/reflect/index.html


4.4.4 Roadmap Currently Adapters for OpenShift and CloudFoundry adapters have been created and support the required methods of Unified Cloud API (defined in deliverable D4.1 [9]), while are continuously extended for supporting more of the extended functionality methods (defined in deliverable D4.2 [10]). Meanwhile Adapters for Heroku and AWS Beanstalk are under final testing, and adapters for CloudControl and Apache Stratos are under development and will be available for the final release of PaaSport Marketplace. PaaSInteractionProxyService is supporting the proxying of basic methods but is being continuously extended with support of more advanced methods.

4.5 Adaptive Front-Ends The adaptive front-ends of the PaaSport marketplace are offering to developers, PaaS providers and PaaSport marketplace administrators the user interfaces for the offered functionalities. General categorization of the functionalities currently offered through UI elements in the first release of PaaSport Marketplace can be simply described in the following categories:

Cloud Marketplace Catalogue

User Management

Application Semantic Profile Management

PaaS Offerings search

Application Deployment

Application Lifecycle Management

Application Monitoring

PaaS Offerings Management

PaaS Offering Semantic Profiles Management These functionalities have been presented with screenshots in Section 2, but more information regarding specific UI elements is provided in PaaSport deliverable 5.1 [11]. For front-end creation, AngularJS7 is used with REST backend that is developed step-by-step in order to follow the development process of the PaaSport marketplace and to reflect the reference architecture as described in PaaSport deliverable 1.2 [4]. AngularJS is a UI implementation framework that is working with usage of Model View View-Model (MVVM) binding framework. The main features of AngularJS include the support of two-way binding on regular fields, built-in validation, JSON support, the ability to define modules and inject dependencies, good testing support and built-in routing.

Front-end part created with AngularJS is only using a specific backend that uses REST calls, and does not communicate with any other components. The REST backend creates and consumes JSON responses that AngularJS frontend is using. An example page implementation follows. Particularly, we demonstrate the code needed both in REST backend and AngularJS frontend. Figure 43 shows the creation of a REST call that is responsible to undertake the registration of a new user.

7 https://angularjs.org/


Figure 43: Backend code example for user registration

For the frontend creation Angular uses a blend of JavaScript and html in order to display the content provided by the backend. The html page creates the actual placeholders and is extended with some Angular specific attributes that allow placeholders connection with the backend, as shown in Figure 44.

Figure 44: Frontend html code example.

An example of the JavaScript code used by Angular is provided in Figure 45, where the parameters that are connected with the placeholders on the html page are defined and received from an appropriated JSON object.


Figure 45: Frontend JavaScript code example.

Finally, the resulting page that is connected to the REST backend with JSON objects is displayed in Figure 46.

Figure 46: Frontend UI example.

4.5.1 Roadmap Creation of Adaptive Frontends is a continuous task that is using the available backend functionality. Although creation of the UI elements though Adaptive Frontends is at initial steps according to the description of work[12], much of the offered functionality has already been implemented, in order to have the First Release of PaaSport Marketplace that supports all functionalities that backend layers support. More functionalities offered through


Adaptive Frontends will become available during the next months and will be integrated in the final release of PaaSport marketplace.


5 Reporting on the Development Lifecycle Setup This section describes the methodology followed for the deployment of First Release of PaaSport Marketplace. This methodology allows the continuous delivery of updated versions and allows the easier development and support of PaaSport Marketplace.

5.1 Deployment of PaaSport Marketplace with Continuous Integration Deployment of PaaSport Marketplace is based on a Continuous Delivery process. Continuous Delivery (CD) is a software engineering approach in which teams keep producing valuable software in short cycles and ensure that the software can be reliably released at any time [1]. It is used in software development to automate and improve the process of software delivery. Continuous delivery basis is a series of techniques designed to ensure that code can be rapidly and safely deployed to production by delivering every change to a production-like environment and ensuring proper functionalities through automated testing. Every change is delivered to a staging environment using complete automation, so that it is guaranteed that the created application is deployable at any time, and can be deployed to production with a push of a button.

Continuous deployment is also another option that is even more automated. In Continuous deployment, every change that passes the automated tests is deployed to production automatically8. For PaaSport Marketplace the final decision for deployment to production is done manually, in order to have total control of the releases that go into production, so Continuous Delivery is performed. The overview of this process followed in order to deploy a new version of PaaSport marketplace to production is presented in Figure 47.

Figure 47: Continuous Delivery in PaaSport

The whole process is automated except two basic manual steps. The starting point of this process is the commit of the code by the developer to the version control repository. Git is used as the primary Version Control system. The Git repository is located here: https://github.com/UBITECH/paasport and it is a private repository with access limited to the consortium developers for the time being. For every commit made, a git hook9 is configured to start the Automated Build – Continuous Integration with Jenkins.

8 https://puppetlabs.com/blog/continuous-delivery-vs-continuous-deployment-whats-diff

9 http://git-scm.com/docs/githooks

https://github.com/UBITECH/paasport


Jenkins is configured to build in an automated manner new releases of PaaSport platform on a staging environment. Semi-auto (after manual approval) deployment to production environment is also configured. Automated testing has also been configured with usage of unit tests for every build made with Jenkins. Unit testing is a task that every artefact developer is responsible and is performed before the integration of the artefact in the marketplace. Beyond integration, functional tests have been also created for several main functionalities of the marketplace in order to ensure its proper functioning. The configuration of the development cycle is in-line with the one described in PaaSport deliverable 5.1[11].

Figure 48: Sample automated build of PaaSport on Jenkins

The purpose of the Continuous Integration (CI) platform is twofold. On the one hand, the latest valid snapshot is always deployed to a specific server which is used for continuous testing. On the other hand, when the development will be about to finish the CI server will operate as a pre-staging environment. This practically means that any upgrades that will be released during the production phase will be performed automatically through Jenkins. This is extremely valuable since in our project the entire development lifecycle has been done using corporate standards. During the built-process, all unit-tests (see indicative figure below) are executed (unless they are marked as @Ignored). This practically means that each release has a functional guarantee regarding its stability. However, this also means that the responsibility of the developers is high since the test coverage is under their jurisdiction. In order to leverage the sense of responsibility we relied on quality control mechanisms as explained in the next chapter.


Figure 49: Indicative Unit Tests

5.2 Source Code Quality As already described the development lifecycle relies on SonarQube10 tool that perform the quality testing at the source level. SonarQube is configured in order to perform a set of analysis such as static code analysis, analysis of best practices, analysis of conventions etc. The figure below illustrates an indicative quality report of a specific component. As it is depicted specific reports regarding the blocking/critical/major issues are presented along with several indications of architectural quality (duplications, reusability, testing coverage etc). A critical indicator regarding the quality is the technical depth that attempts to quantify the amount of days that are required in order to make a specific snapshot a proactive release. It should be argued that the calculation presented by the SonarQube reporting tool is overestimated by approximately 50%. In any case, one month before the final release the PaaSport development team will minimize the technical depth.

10

http://www.sonarqube.org/


Figure 50: Source Code Quality through SonarQube

At this point it should be noted that the emphasis of the development team is not to tackle all quality issues that are raised by SonarQube but to finish the entire set of features that are required for the first release.

5.3 Development Statistics Finally, this sub-chapter is devoted to some reports that refer to the development process. As depicted in the figure below the first development cycle was rather intense since the nature of the agile process that is adopted requires continuous delivery.

Figure 51: Statistics of commit log

Furthermore, the technical team relied on weekly releases which practically resulted in non-homogeneous commits during the week.


Figure 52 : Daily activity


6 Conclusion The present document is PaaSport Deliverable 5.2 “PaaSport Marketplace Infrastructure – First Release” of the PaaSport project and its main objective is to present the first release of the prototype of the integrated PaaSport Marketplace Infrastructure as well as to document the development and integration progress of PaaSport.

An overview the status of PaaSport Marketplace has been provided with the presentation of First Release in conditions of real usage by end users. The presented conditions were documented for unregistered users, registered developers and PaaSport Marketplace administrators.

After providing the latest updates on its architecture, the status of PaaSport Marketplace is provided with more details. The details are mostly presented in terms of technical achievements for each layer identified in PaaSport architecture, namely Semantic Model, PaaS Offering Recommendation Layer, Monitoring and SLA Enforcement Layer, Persistency, Execution and Coordination Layer and Adaptive Front-Ends. The achievements so far are in line with the requirement of the first release of the PaaSport Marketplace and the related Milestones defined in DoW [12].

Finally, PaaSport has a well-defined methodology that is used for development of PaaSport Marketplace and also for the deployment of new releases. This methodology is analysed and details about the first release are provided for the production environment used, but also for the code that implements the First Release of PaaSport Marketplace.


REFERENCES [1] Chen, Lianping. Continuous Delivery: Huge Benefits, but Challenges Too (2015) [2] Evren Sirin, Bijan Parsia, Bernardo Cuenca Grau, Aditya Kalyanpur, and Yarden Katz.

2007. Pellet: A practical OWL-DL reasoner. Web Semant. 5, 2 (June 2007), 51-53. [3] Gangemi, A., Mika, P. Understanding the semantic web through descriptions and

situations. In: Proceedings of the International Conference on Ontologies, Databases and Applications of Semantics. pp. 689--706 (2003)

[4] PaaSport Deliverable D1.2, “PaaSport Reference Architecture”, PaaSport project, FP7-605193

[5] PaaSport Deliverable D1.3, “PaaSport Semantic Models”, PaaSport project, FP7-605193

[6] PaaSport Deliverable D2.1, “PaaSport Recommendation Algorithm and Model”, PaaSport project, FP7-605193

[7] PaaSport Deliverable D3.1, “Service Level Agreement Policy Model”, PaaSport project, FP7-605193

[8] PaaSport Deliverable D3.2, “Monitoring and Accounting Layer - First Release”, PaaSport project, FP7-605193

[9] PaaSport Deliverable D4.1 - “Unified Cloud Platforms Interface Model and API”, PaaSport project, FP7-605193

[10] PaaSport Deliverable D4.2, “Persistence and Execution Layer– First Release”, PaaSport project, FP7-605193

[11] PaaSport Deliverable D5.1, “PaaSport Integrated Technical Architecture”, PaaSport project, FP7-605193

[12] PaaSport Description of Work, PaaSport project, FP7-605193


LIST OF ABBREVIATIONS

ALM Application Lifecycle Management API Application Programing Interface CRUD Create, Retrieve, Update and Delete DnS Descriptions and Situations DOLCE Descriptive Ontology for Linguistic and Cognitive Engineering DoW Description of Work DUL DOLCE and DnS HTTP Hypertext Transfer Protocol JSON JavaScript Object Notation IaaS Infrastructure as a Service LDAP Lightweight Directory Access Protocol MVVM Model View View-Model SLA Service-level agreement SME Small-Medium Enterprise LDAP Lightweight Directory Access Protocol ORM Object-relational mapping PaaS Platform as a Service QoS Quality of Service RDF Resource Description Framework REST Representational State Transfer SLA Service-level agreement SOA Service-oriented architecture SPI Service Provider Interface SME Small and medium-sized enterprises UI User Interface XML Extensible Markup Language

Documents

PaaSport Marketplace Infrastructure First Release ...linc.ucy.ac.cy › file › PaaSport › PaaSport_D5.2_finalrelease.pdfPaaSport is funded by the European Commission Seventh Framework