Cloud Computing Automation: Integrating USDL and TOSCA

Jorge Cardoso (1,2), Tobias Binz (3), Uwe Breitenbucher (3), Oliver Kopp (3) Frank Leymann (3)

(1) CISUC/Dept. Informatics Engineering, University of Coimbra, Portugal(2) Karlsruhe Service Research Institute, Karlsruhe Institute of Technology, Germany

[email protected]@dei.uc.pt

(3) Institute of Architecture of Application Systems, University of Stuttgart, Germany{lastname}@iaas.uni-stuttgart.de

Cloud Computing Automation: Integrating USDL and TOSCA

Departamento de Engenharia InformáticaFCTUC FACULDADE DE CIÊNCIAS E TECNOLOGIA da

UNIVERSIDADE DE COIMBRA

Institute of Architecture of Application Systems (IAAS) Universität Stuttgart, Germany

Cloud Computing Automation: Integrating USDL and TOSCA, J. Cardoso, T. Binz, U. Breitenbucher, O. Kopp, F. Leymann, CAiSE 2013, Springer, LNCS 7908, pp. 1—16.

Genessiz: Center for Large-Scale Service System Research 2

Standardization

2013

See also http://cloud-standards.orgBMWi: The standardisation environment for cloud computing. Technical report, Germany Federal Ministry of Economics and Technology (Feb. 2012)


Interoperability• Standards are being

developed in isolation

• It is not clear to which extend they can be integrated

• Lack of certainty as to which standards can inter-operate

• Streamline the lifecycle of cloud applications

2013

Standardization does not imply interoperability

Definition (Interoperability): the ability of various systems and organizations to work together (inter-operate)


Motivation (1)• Discovery, Selection and

Customization– Done manually by

consumers

• Keyword querying – No advanced mechanism

• Multiple queries– Different marketplaces use

different query interfaces

2013

AppDirect

Google


Motivation (2)• After a purchase decision...

– Customization– Deployment – Management

• No …• Formalization of the

executables• Management best practices• Etc.

• Problem– Manual – Error-prone

2013

Exceptions: Saleforce, Google Apps,Microsoft Office 365, etc.

6BMWi: The standardisation environment for cloud computing. Technical report, Germany Federal Ministry of Economics and Technology (Feb. 2012)

“In the course of the six months in which the study was produced, a lot of new publications appeared, not all of which could be taken into account” (e. g. TOSCA, http://www.oasis-open.org/committees/tosca/)


Research Question

2013

When used in conjunction, can they automate parts of the lifecycle of cloud applications, namely discovery, selection, deployment, and management?

Can USDL and TOSCA be integrated seamlessly? How can interoperability be achieved? What are the challenges?


Approach

2013

DISCOVERY AND

SELECTION

DEPLOYMENT AND

MANAGEMENT

USDL/Service Description

TOSCA/Service Management

Use Case

Describes the structure of an application and its management (which is executable)

Goal >>Portability and full-automated management of applications

Describes the functional and non-functional requirements, capabilities, and interactions of a service

Goal >>Description of a cloud service to make it searchable, comparable, and tradable

Topology Management Plans

…Interaction &

functional capabilities

Offerings

Interactions

Providers

…Non-functional

capabilities

Pricing

Legal

Service Level

Topology and Orchestration Specification for Cloud Applications


USDL:CoreMaster Schema


Linked USDL

2013

http://www.linked-usdl.org/ https://github.com/linked-usdl/


TOSCA

2013

OsApache(OperatingSystem)

VmApache(VirtualMachine)

ApacheWebServer(ApacheWebServer)

(HostedOn)(InstalledIn)

SugarCrmApp(SugarCrmApp)

PhpModule(PhpModule)

(DependsOn)

OsMySql(OperatingSystem)

VmMySql(VirtualMachine)

MySql(MySqlRDBMS)

SugarCrmDb(SugarCrmDb)

(HostedOn)

(DbConnection)

Acquire VM

Install OS on

VM

Install & Start Web

Server

Install PHP

Module

Deploy PHP App Establish

DB ConnectionInstall

OS on VM

Install & Start MySQL RDBMS

CreateSugarCRMDB Module

Build Plan

Topology

Acquire VM

Su

garC

RM


Solution• How to access service descriptions in a dynamic world?

– Global service identification and service description access using Linked USDL

• What if the provider has ceased its operations and transferred its obligations to some other provider? Who still handle the original functions? • Intelligent routing of service requests

• What would happen if the TOSCA descriptor associated with a USDL description would no longer be valid? – Dynamic binding of deployment descriptors

2013

WWW + Semantic web = Distributed, scalable, reliable, extensible, …[11]

Service Cloud

14

USDL-based Marketplace

USDL-based Service Offerings

Billing / CRM System

UI

…

Service 1

Service N

TOSCA-based Provider

TOSCA Service Archives

T

T

…

Service 1

Service N

TOSCA Runtime Environment

Global Routing

Layer

LocalRouting

Layer

Cloud Management System

USDL URI … Provider Endpoint

http://sugarcrm.org?enterprise … 192.182.1.3

http://redmine.org?professional … 147.11.4.79

TOSCA Routing Layer

USDL URI … Plan Endpoint

http://sugarcrm.org?enterprise … 111.121.12.1/SugarCRMPlan

http://redmine.org?professional … 111.121.12.1/RedminePlan

SIOPP

Architecture

ReasoningEngine

ReasoningEngine

RoutingTable

RoutingTable

5

4

32

1

6

7

Genessiz: Center for Large-Scale Service System Research

Service Identification & Access

201315

Cloud

USDL-compliant

http://rdfs.genssiz.org/SugarCRM?pricePlan=pricing_SugarCRM_Ultimate

5

4

3

2

1Query stringsare a W3C recommendation

Service offerings are modeled with Linked USDL

A. Simple way to create unique global identifiers for services. Compared to, e.g., a universally unique identifier (UUID), Linked USDL URIs are more adequate to service distribution networks since they are managed locally by service providers

B. The HTTP URI also serves as endpoint to provide uniform data access to the service description. A Linked USDL URI can be used by, e.g., RDF search engines, and web query agents looking for cloud service descriptions

6


Benefits• Global service identification and remote description access

– Unique service identification schema using Linked USDL URIs – Uniform data access [12] to service descriptions using Linked USDL

HTTP URIs – Decentralized management of unique service identifiers is scalable

• Intelligent routing of service requests– SPARQL for the content-based routing [14]– Flexible querying mechanism (cf. web APIs)– Full access to the service specifications is possible remotely

• Dynamic binding of deployment descriptors – Publish-subscribe pattern [15] – Scalable by distributing USDL requests to TOSCA Runtime

Environments – Higher degree of decoupling (cf. BPM or integration by web services)2013


Intelligent Routing of Service Requests

• Separation of Concerns (routing logic)– GRL -- high level information

• e.g., information about the country of the provider for legal aspects

– LRL -- lower level aspects • e.g., load balancing information

– TRL -- management actions• e.g., implementing security aspecTOSCA ts directly in management plans

2013

Runtime Environment

TOSCA Routing Layer

Marketplace

Global Routing Layer

Provider

Local Routing LayerService 1

Service 2

…


Intelligent Routing of Service Requests

201320

TOSCA-based Provider

TOSCA Service Archives

T

T

…

Service 1

Service N

TOSCA Runtime Environment

LocalRouting

Layer

TOSCA Routing Layer

USDL URI … Plan Endpoint

http://sugarcrm.org?enterprise … 111.121.12.1/SugarCRMPlan

http://redmine.org?professional … 111.121.12.1/RedminePlan

ReasoningEngine

RoutingTable

5

4

6

7Acquire

VM

Install OS on

VM

Install & Start Web

Server

Install PHP

Module

Deploy PHP App

EstablishDB

ConnectionInstall OS on

VM

Install & Start MySQL RDBMS


Build Plan

Acquire VM

A SPARQL query to inquire about the options

input message used by the build plan to deploy SugarCRM on Amazon EC2


Benefits• Global service identification and remote description access

– Unique service identification schema using Linked USDL URIs – Uniform data access [12] to service descriptions using Linked USDL

HTTP URIs – Decentralized management of unique service identifiers is scalable

• Intelligent routing of service requests– SPARQL for the content-based routing [14]– Flexible querying mechanism (cf. web APIs)– Full access to the service specifications is possible remotely

• Dynamic binding of deployment descriptors – Publish-subscribe pattern [15] – Scalable by distributing USDL requests to TOSCA Runtime

Environments – Higher degree of decoupling (cf. BPM or integration by web services)2013


Performance• Settings

– Win7-64bit, JRE 1.7, Intel i5-2410M, 2,3GHz. – GRL: hash table with 500,000 entries and looked up 5,000 entries – LRL: hash table with 10,000 entries and looked up 1,000 entries

• GRL: 3 ms• LRL: 2 ms• Build plan adaptation: 289 ms (σ = 76)• Deployment: 4-7 min

– Depends on the provisioning time of the VMs at Amazon EC2

• Conclusions– In our scenario, the overhead, even for peak demands, is negligible

2013


Limitations• Scalability

– Adopt a peer-to-peer architecture using an overlay network• e.g., use the Simple Knowledge Organization System (SKOS)

– Network partitioned according to service domains • e.g., healthcare, finance, and logistics

– Route requests domain to domain/subdomains using SKOS • e.g., skos:narrower and skos:member

• Customization – The customization string works well with simple customization– Inadequate for condition-based based customization

• i.e. if logical conditions need to be sent along with service requests

• Inputs to build plans – Associating USDL URIs with concrete input values for build plans has been

found to be difficult if there is no description on how the values affect the deployment

2013


Conclusion• We explored the interoperability of two cloud specifications

– Linked USDL and TOSCA

• Solution– Open and decentralized– Semantic web and Linked Data technologies– Link description/customization with deployment/management

• Results– Interoperability is possible– End-to-end solutions can be developed– Engineered solution – Support the lifecycle of cloud services

2013

Thank You

for Listening


References• 10. Cardoso, J.; Barros, A.; May, N. and Kylau, U. Towards a Unified Service

Description Language for the Internet of Services: Requirements and First Developments. In IEEE International Conference on Services Computing, IEEE Computer Society Press, Florida, USA, 2010.

• 11. Hors, A.L., Nally, M.: Using read/write Linked Data for Application Integration: Towards a Linked Data Basic Profile. In: Linked Data on the Web (2012)

• 12. Ziegler, P., Dittrich, K.: Three decades of data intecration – all problems solved? In: Jacquart, R. (ed.) Building the Information Society. IFIP, vol. 156, pp. 3–12. Springer, Boston (2004)

• 14. Carzaniga, A., Rutherford, M.J., Wolf, A.L.: A routing scheme for content-based networking. In: Proceedings of IEEE INFOCOM 2004, Hong Kong, China (2004)

• 15. Hohpe, G., Woolf, B.: Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions. Addison-Wesley, Boston (2003)

2013


29BMWi: The standardisation environment for cloud computing. Technical report, Germany Federal Ministry of Economics and Technology (Feb. 2012)

Standardization

Fig 7. Involvement of the standardisation organisations in cloud computing


Lifecycle

2013

12 steps towards creating a cloud service


challengers leaders

niche players visionaries

Complexity of the model

Com

ple

tene

ss o

f th

e m

odel

OWL-S

WSDL

SAWSDLhREST

WSMO-Lite

USDL v1USDL v2

Linked USDL

USDL v3

microWSMO

REST


Complexity vs Acceptance

http://craft.de/simplizitaet/

WO IST WAS?WSDLOWL-SWSMOSAWDLWSMO LiteRESThRESTmicroWSMO

„Ok, but I need more…“

„Nice improvement.“

„Cool“

„I‘m a hero“ „I have to look it up in the manual…“

„Where the heck do I find it?“

„I can‘t even do the simplest things….“

I‘m a looser

Maximum Customer Satisfaction

COMPLEXITY

ACCE

PTAN

CE


Classes

Properties

Service Oriented Computing II – SS 2013

LINKED USDL MODULES

• USDL-Core• USDL-Pricing• USDL-SLA

22.05.2013

• Additional modules– USDL-Legal

• Domain specific– USDL-EDU– USDL-Logistics


TOSCA

2013

Service Structure Service Orchestration forDeployment & Management

Start VM Install Tomcat

OperatingSystem(Ubuntu 12.04 LTS)

VirtualServer(AWS EC2 Server)

WebServer(Tomcat)

EC2


TOSCA Topology Concepts

2013

Application(WAR)

OperatingSystem(Ubuntu 12.04 LTS)

VirtualServer(AWS EC2 Server)

WebServer(Tomcat)

EC2

Node Template

Relationship Template

Node Type

hosted-on

ubuntu.amiubuntu.ami

app.war

Deployment Artifacts

AppSpecific

DeployStart, Stop

installPkg

TerminateCreateVM

execScript

Management Operations


SugarCRM/Use Case/Silver

2013

OperatingSystem(OperatingSystem)

VirtualMachine(VirtualMachine)

(HostedOn)

ApacheWebServer(ApacheWebServer)

(HostedOn)

SugarCrmApp(SugarCrmApp)

(HostedOn)

PhpModule(PhpModule)

(InstalledIn)

(DependsOn)

MySql(MySqlRDBMS)

(HostedOn)

SugarCrmDb(SugarCrmDb)

(HostedOn)

(MySqlDbConnection)


Intelligent Routing• Listing 1.3 shows an example of an

input message used by the build plan to deploy SugarCRM on Amazon EC2 (described in Section 3.4).

• The message contains credentials of the Amazon account to be used (line 2 and 3), the geographic region where the virtual machines should be located (line 4), and a pointer to the USDL offering (line 5).

• The USDL URI is used by the plan to query the Linked USDL offering by using SPARQL and adjust the deployment.

• In our prototype, deciding between the deployment options enterprise or ultimate is done based on the selected USDL pricing plan.

2013

Acquire VM

Install OS on VM

Install & Start Web Server

Install PHP Module

Deploy PHP App

EstablishDB Connection

Install OS on VM Install & Start MySQL RDBMS


Build Plan

Acquire VM


Intelligent Routing• Listing 1.4 shows the

SPARQL query used by the build plan to inquire about the options which are attached to the pricing plan included by the (customized) USDL URI.

• The options are then installed automatically.

2013