Proceedings of IEEE CCIS2011 MULTI CLOUD MANAGEMENT FOR UNIFIED

CLOUD SERVICES ACROSS CLOUD SITES Tiancheng Liu, Yasuharu Katsuno, Kewei Sun, Ying Li, Takayuki Kushida,

Ying Chen, Mayumi Itakura IBM Research, China and IBM Research, Tokyo, Japan

{liutc, sunkewei, lying, yingch}@cn.ibm.com, {katsuno, kushida, mitakura}@jp.ibm.com Abstract Recently, there is appearing a multi Infrastructure as a Service (IaaS) cloud site environment. It makes server users/administrators annoying because each cloud site is managed separately by each cloud owner and they have to make use of cloud sites individually. In this paper, we propose the Multi Cloud Management Platform that locates between cloud users and cloud sites and provides unified cloud services. It can decrease workloads of server users/administrators under a multi IaaS cloud site by a service catalog federation, a collaborative management, and an application virtual server migration services. We implement a prototype system, and show our approach is feasible. Keywords: Cloud Computing Management, Service Federation, Unified Management, Application Migration.

1 Introduction 1.1 Background An Infrastructure as a Service (IaaS) cloud computing is an attractive computing concept in enterprises from viewpoints of both a server user who makes use of servers and a server administrator who prepares servers for server users and manages them [1].

CloudManager Cloud Resource Pool

VirtualServer

VirtualServer

VirtualServer

ServerUser

ServerAdministrator

Service Catalog

Service CatalogCloud

ManagementPortal

CloudServicePortal

Cloud Site

Figure 1 IaaS cloud site Figure 1 shows a general IaaS cloud site. A server administrator creates virtual server templates, and enrolls them as service catalogs into a cloud manager in advance, using a cloud manager portal. Server users open a cloud service portal and select an appropriate service catalog when they would like to use servers. Then, the service catalog request is sent from the service portal to the cloud manager. The cloud manger creates a new virtual server on cloud resource pool by copying the template linked with the requested catalog, and informs the users that requested virtual server is ready. A server administrator manages virtual servers on the cloud management portal. Server users can immediately make use of virtual servers, exactly when they want without contacting a server administrator. A server administrator does not need to answer user’s requests each time and can manage servers centrally.

A multi IaaS cloud site is appearing as cloud computing benefits spread [2]. As the result, both a server user and a server administrator are annoyed because cloud sites are separated (Figure 2). First, a server user has to select an appropriate cloud site and service catalog satisfying with requirements by himself/herself, using a different cloud service portal located on each cloud site. Second, a server administrator needs to manage virtual servers with different manners using a different cloud management portal on each cloud site. Finally, it requires heavy workloads to migrate an application equipped virtual server from one cloud site to another (e.g. from test cloud to production cloud).

CloudManager Cloud Resource Pool

VirtualServer

VirtualServer

VirtualServer

CloudManager Cloud Resource Pool

VirtualServer

VirtualServer

VirtualServer

CloudManager Cloud Resource Pool

VirtualServer

VirtualServer

VirtualServer

ServerUser

ServerAdministrator

Service CatalogService Catalog

Service CatalogService Catalog

Service CatalogService Catalog

CloudManagement

Portal

CloudManagement

Portal

CloudManagement

Portal

CloudServicePortal

CloudServicePortal

CloudServicePortal

Cloud Site 1

Cloud Site 2

Cloud Site 3

Issue #1

Issue #2

Issue #3

Figure 2 Issues on multi IaaS cloud sites 1.2 Our contribution In this paper, we propose the Multi Cloud Management Platform to resolve the issues on multi IaaS cloud sites (Figure 3). The Multi Cloud Management Platform is located between users and cloud sites, and provides the unified cloud service portal and the unified cloud management portal for a server user and a server administrator respectively. It has three features to tackle the issues: a service catalog federation, a collaborative management, and an application virtual server migration. A service catalog federation automatically picks up service catalogs that meet server user’s requirements from multi cloud sites, creates a single service catalog that links with them, and provides it as a federated service catalog for the user on the unified cloud service portal. The user can select an appreciate service catalog on the single portal without considering multi cloud sites. A collaborative management extracts virtual server information from each cloud site, and provides the server administrator on the unified cloud management portal. The administrator can centrally manage virtual servers running on different cloud sites. An application virtual server

___________________________________ 978-1-61284-204-2/11/$26.00 ©2011 IEEE

migration provides the service that migrates an application equipped virtual server on one cloud site to another site. The user can migrate the virtual server to an appreciate cloud site easily. We implement prototype systems for a service catalog federation, a collaborative management, an application virtual server migration, and show they are feasible.

Multi CloudManagement Platform

CloudManager Cloud Resource Pool

VirtualServer

VirtualServer

VirtualServer

CloudManager Cloud Resource Pool

VirtualServer

VirtualServer

VirtualServer

CloudManager Cloud Resource Pool

VirtualServer

VirtualServer

VirtualServer

ServerUser

ServerAdministrator

Service Catalog

Service Catalog

Service Catalog

Service Catalog

Service Catalog

Service Catalog

Unified CloudManagement Portal

Unified CloudService Portal

Federated Service Catalog

Federated Service Catalog

Migration

Application VirtualServer Migration

Service CatalogFederation

CollaborativeManagement

Cloud Site 1

Cloud Site 2

Cloud Site 3

Figure 3 Multi cloud management platform This paper is structured as follows: Section 2 shows the concept of the Multi-Cloud Management Platform. We describe a service catalog federation, a collaborative management, an application virtual server migration in Section 3, 4, 5 respectively. Related work is discussed in Section 6. Finally, we conclude in Section 7 with a summary of our results and consider future works that still remain to be addressed.

2 Multi cloud management platform Different cloud service providers, based on different technologies, support a large variant number of cloud services, which can be categorized into Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and Software as a Service (SaaS). Cloud service consumers select what fit their requirements (functions, QoS, price, etc.) from the cloud services. After the consumers make the selection, they follow the process of the corresponding service provider to use the service. This is the most common scenario of how cloud computing services are provided and consumed. Different to the above described way, our Multi Cloud Management Platform introduces a new way to consume cloud services. Instead of providing concrete cloud services, it focuses on federating existing Cloud services and providing integrated and collaborative operation supports. The consumer only accesses portals from Multi Cloud Management Platform, instead of the portals of different cloud sites. All available services can be requested from a single entry. And the related management can be done through this entry, besides some reusable management services are provided. Figure 4 shows the framework of Multi Cloud Management Platform. At the front-end, there are two portals: Unified Cloud Service Portal and Unified Management Portal. These two portals both connect to Service Catalog Federation, which stores all the available services. The difference is unified cloud service portal shows the catalogs containing normal services, e.g. virtual server services, while unified management portal shows

the management services, e.g. monitoring service and middleware management services.

Multi CloudManagement Platform

ServerUser

ServerAdministrator

Unified CloudManagement Portal

Unified CloudService Portal

Federated Service Catalog

Federated Service Catalog

Migration

Cloud Service Connections &

Adaptation

Management Assets

Script Lib JavaLib

Workflow

CloudManager

Service Catalog

Service Catalog

CloudManager

Service Catalog

Service Catalog

CloudManager

Service Catalog

Service Catalog

Others

Collaborative Management Process & Automation

Figure 4 Multi cloud management platform architecture For the normal services, Service Catalog Federation directly connects to the selected cloud service via Cloud Service Connection and Adaptation. The details of how to select cloud service from the candidates is described in Section 3. For the management services, the Collaborative Management Process and Automation executes the corresponding management service process, which is composed by reusable management assets stored in the repository. To access the cloud services or the virtual servers in remote cloud sites, Cloud Service Connection and Adaptation is used. Cloud Service Connection and Adaptation is a module that encapsulates the difference of APIs provided by different cloud service provider or cloud site, so that other Multi Cloud Management Platform modules only needs to be aware of a single set of APIs. There is a specific management service provided in Multi Cloud Management Platform, migration service. One cloud service consumer might use multiple cloud services from different cloud sites. In some cases, the consumer wants to migrate one application from one cloud site to another one, for reliability, price or other reasons. Multi Cloud Management Platform provides a migration service to automate the migration process. This will save the consumer a lot of time and efforts.

3 Service catalog federation Under multi-cloud environments, a server user has to open a cloud service portal on each cloud site, find service catalog candidates that meet their requirements, and select an appropriate catalog. For example, a server user is about to choose a service catalog composed of Linux OS and a web server from cloud sites A, B, C (Figure 5). The user opens each cloud service portal, finds that candidates are “RHEL Standard”, “Linux Middleware”, “Open Source Advanced” catalogs on cloud sites A, B, C respectively, and selects the most suitable one. These tasks impose a heavy burden on the user. The user opens each cloud service portal, finds that candidates are “RHEL Standard”, “Linux Middleware”, “Open Source Advanced” catalogs on cloud sites A, B, C respectively, and selects the most suitable one. These tasks impose a heavy burden on the user.

CloudManager

CloudManager

CloudManager

RHEL Basic

RHEL Standard

Linux Middleware

Windows Middleware

Open Source BasicOpen Source Advanced

CloudService Portal

CloudService Portal

CloudService Portal

ServerUser

Cloud Site 1

Cloud Site 2

Cloud Site 3

Figure 5 Multi cloud environments without Service catalog federation Our service catalog federation (Figure 6) automatically finds service catalog candidates that meet the requirements, generates the single service catalog that links with them, and provides it as federated service catalog on the unified cloud service portal. Then, it automatically selects the most appreciate catalog from the candidates and requests to the owned cloud site when the user requests the federated service catalog. In case of Figure 5, “Linux + Web” federated catalog is generated, and “RHEL Standard” catalog is selected and requested to cloud site 1.

ServerUser

Multi CloudManagement Platform

Unified CloudService Portal

Linux + Web

CloudManager

CloudManager

CloudManager

RHEL Basic

RHEL Standard

Linux Middleware

Windows Middleware

Open Source BasicOpen Source Advanced

Cloud Site 1

Cloud Site 2

Cloud Site 3AutomaticGeneration

Automaticselection

Figure 6 Multi cloud environments with service catalog federation A federation tag tree is used for automatically generating a federated service catalog from service catalogs (Figure 7). It is composed of parent tags selected by a server user and child tags attached to service catalogs on cloud sites. First, a server user selects a few parent tags that match requirements on the unified cloud service portal. Then, multi cloud management platform automatically collects service catalogs whose child tags under the selected parent tags are attached, generates federated service catalogs, and provides the user with it on the service portal. Finally, when the user selects a federated service catalog, an appreciate service catalog is automatically picked out in the service catalogs composed of the selected federated service catalog by service catalog federation rules and requested to the cloud site. It is considered cloud resource availability, performance, security, and user organization as the metrics of service catalog federation rules.

FederationTag Tree Linux

Windows

Web

RHEL

SuSE

CentOS

Windows Server 2003

Windows Server 2008

WebSphere

Tomcat

Commercial VMM

OSS VMM

Hyper-V

KVM

Xen

VMware

Child TagParent Tag

Figure 7 Federation tag tree

We give an example of service catalog federation in Figures 6 and 7 with the following example of relationships among cloud sites, service catalogs, and child tags. {“Cloud Site 1”, {“RHEL Basic”, {“VMware”, “RHEL”}},

{“RHEL Standard”, {“Vmware”, “RHEL“, “WAS”}}},

{“Cloud Site 2”, {“Linux Middleware”, {“KVM“, “SuSE“, “WAS“}},

{“Windows Middleware”, {“KVM“, “Windows Server 2008“,

“WAS“}}}, {“Cloud Site 3”, {“Open Source Basic”, {“Xen“, “CentOS“}},

{“Open Source Advanced”, {“Xen“, “CentOS“, “Apache“}}}}

The unified cloud service portal shows a server user parent tags: “Commercial VMM”, “OSS VMM”, “Linux”, “Windows”, “Web”. 1. The user selects “Linux” parent tag. 2. The following federated service catalog candidates are automatically created and provided to the user. • Linux : federate with “RHEL Basic”, “RHEL Standard”, “Linux Middleware”, “Open Source Basic”, “Open Source Advanced”. • Linux - Web : federate with “RHEL Standard”, “Linux Middleware”, “Open Source Advanced”. • Linux - Commercial VMM : federate with “RHEL Basic”, “RHEL Standard”. • Linux - OSS VMM : federate with “Linux Middleware”, “Open Source Basic”, “Open Source Advanced”. • Linux - Commercial VMM - Web : federate with “RHEL Basic”. • Linux - OSS VMM - Web : federate with “Linux Middleware”, “Open Source Advanced”. 3. The parent tag “Windows” is automatically disappeared on the portal, because of exclusive relationship with “Linux” tag. 4. The user selects “Web” parent tag in addition to “Linux” parent tag from "Commercial VMM", "OSS VMM", "Web“ tags on the portal, then the following federated service catalogs are automatically provided. • Linux - Web • Linux - Commercial VMM - Web • Linux - OSS VMM - Web 5. The user requests “Linux - Web” federated service catalog. 6. The service catalog “RHEL Standard” is automatically selected and requested to cloud site 1.

4 Collaborative management To manage cloud services across multiple cloud sites, the server administrators face a similar situation as server users: they need to access different management interfaces, portals of different cloud site or in some cases without any user interface. The problem is, even if one administrator wants to do exactly the same

management operations on the virtual servers in different cloud sites, he/she is very likely to have to do those operations via different manners, for the multiple cloud sites provides different management interfaces. This situation has a big impact on the management efficiency, and makes it almost impossible to reuse the management knowledge. To resolve this problem, we leverage our previous work on Management as a Service [3] and enhance it with the service automation technology.

Collaborative Management Process & Automation

Management Assets

Script Lib JavaLib

Workflow Others

Service Automation (TPAe)

Management Service

Definition

Management Service

Definition

… Remote Service Call &

Delivery

Figure 8 Collaborative management framework The framework of Collaborative Management framework is shown in Figure 8. There are three major components in the framework: • Service automation: is a service process engine to automate and manage the management process. We use Tivoli Process Automation engine. It defines each management operation as a management service definition. • Management assets: is an asset repository that stores reusable management operation implementations. Typically the assets can be scripts, Java classes, workflows or other kinds. These assets are used in the management service definition to do real operations • Remote service call & delivery: is a component that enables the assets to run on remote cloud sites. In some cases management service definition can directly use this module to invoke a service from one cloud site. 4.1 Management service model The management service model follows the service model specified by Tivoli Service Automation Manager. A management service definition consists of three parts: • Structural model: is the target managed object topology. It typically has the virtual servers, software and applications running on the virtual servers. They have relationships and also some interfaces that can be called from external. • Build plan in operational model: is the process to be execute to make this management service available to the consumers. The preparation of the service process can be defined here. • Management plans in operational model: is a set of processes to do actual management work. Process has activities, which can be either the management assets or available cloud services provided by cloud sites. 4.2 Management assets Management assets are reusable management operation implementations, in the forms of scripts, workflows, Java classes, etc. In management asset module, there are two major functions: asset management and asset runtime.

An asset management provides standard asset creation, retrieval, update and removal operations. Each asset has a description containing both the basic information and the constraints. An asset runtime is to run the assets. It has a wrapper to the assets. The wrapper is to enable each asset can be invoked and used via a RESTful or SOAP interface, no matter what the form of the asset itself. The wrapper interface is unified so that each asset can be used in the service definition, and different forms of assets can work together for a complex management task. On the other hand, the asset runtime contains a set of middleware to support asset running. For example for workflow Tivoli Provisioning Manager is used to support its workflow. And Java virtual machine is used to support Java classes. One special case is script assets. The scripts are to run on the managed servers. So the runtime calls remote service call and delivery. 4.3 Remote service call and delivery To access cloud services provided by cloud sites or run scripts on the virtual servers, a lot of interactions are needed. In most cloud sites, web service interfaces (especially REST interface) are provided, in addition to web user interface. Since cloud service federation has catalogs of all cloud services, a simple broker is sufficient here. To run scripts on remote virtual servers, the script firstly needs to be uploaded to the target server, and then runs on the target virtual server. The implementation leverages existing remote execution and access library, like JSch which allows connecting to an sshd server and use port forwarding, X11 forwarding, file transfer, etc., and can be integrated with Java programs. The script execution result is lastly wrapped up in a string and returns to the one who triggers the script execution. Actually security and account management is very important part here. It handles the credentials of remote cloud sites and virtual servers, users’ access rights, etc. But for the limit of space, this will not be touched in this paper.

5 Application virtual server migration Application migration from one cloud environment to another is a very complicated task. The configuration information and its change log is not expected to be documented well. Therefore, the first step of the migration approach is to discover the configuration from the original cloud environment, including configuration parameters of hardware, operating system, middleware and applications. After the configuration information is gathered, they need to be transformed to these that are suitable for target virtualized environment based on the migration and mapping knowledge. And then the transformed configurations will be provisioned on the targets. Manually execution of the configuration discovery, mapping and provisioning of the configuration data is quite a challenge job for the system administrator. The process is error prone. It would be of great help if both discovery and provisioning are executed automatically by migration agents.

Normally, cloud environment are diversity in hardware equipment and software components, and it is nearly impossible to have one suit of migration agents covering all types of hardware and software components in data centers. Therefore, it’s extremely important to have an extensible migration framework. In this paper, we introduce a framework that can 1) perform automatic migration; 2) be extended on demand. As illustrated in Figure 9, the extensible automatic migration framework consists of Adaptive Communication Channel Building Module (ACCBM), Migration Knowledge Base (MKB), Agent Scheduler (AS), Configuration Data Repository (CDR), and Configuration Discovery & Provisioning Agent (CDPA).

Figure 9 Extensible migrationr overview 5.1 Adaptive communication channel building module Adaptive Communication Channel Building Module (ACCBM) takes the responsibility to build the connection and verify the communication among the migration tool and its applied system. Different system opens different communication channels based on their own security consideration. For example, by default, AIX system will open FTP channel but forbids SSH. ACCBM will test the available communication channels available on the remote environment and perform basic information exchange, like upload and download configuration files. Different level of migration operations requires different level of access right to the remote system. For instance, reading the user and group settings from a UNIX system will require root or super user access right. ACCBM will analysis the feedback of the migration results and suggest for more access right. After getting the access right, ACCBM will automatically build a communication channel, perform the migration operation and destroy the channel afterwards. 5.2 Migration knowledge base Migration Knowledge Base (MKB) stores knowledge which is needed for migration, including configuration transformation, and dependency analysis. Generally speaking, the data center configuration could be classified into two major categories – Micro configuration and Macro configuration. Micro configuration describes detailed configuration parameters of a certain software or hardware components. Macro configuration includes software and hardware components and dependencies among them. Configuration transformation knowledge focuses on mapping micro configuration information from physical to virtualized environment. Taking

operating system migration as the example, the configuration transformation might happen among heterogonous operating systems, e.g. from SUN Solaris on standalone machines to AIX on Virtual LPARs of pSeries or zSeries servers. Dependency knowledge describes the relationship between hardware equipment and software components, and among software components. During migration progress, the relationship can be classified into two categories: Host-by and Co-exist. Component A Host-by Component B means that B is the prerequisite to deploy A. While A Co-exist with B means that A and B can only function well with each other. Dependency analysis knowledge is essential for checking the configuration information integrity. This kind of knowledge is consumed by migration data reconciliation module implemented in the migration agent. Figure 10 shows the typical dependency knowledge of a 3-tier application.

Figure 10 Dependency of typical 3-tier application Migration knowledge base covers the transformation knowledge of micro-configuration and dependency knowledge of macro-configuration. Migration framework leverages MKB on mapping data from physical to virtualized environment, and at the same time, verifying the information integrity. 5.3 Extensible agent repository Configuration discovery and provisioning agent (CDPA) is in charge of gathering configuration information and provisioning the transformed configuration information on to the virtualized environment. CDPA plays an important role in the migration framework, executing the discovery and provision task.

Figure 11 Configuration discovery & provisioning agent Figure 11 shows the overall architecture of CDPA. CDPA contains two adapters: Discovery adapter includes configuration discovery scripts for collecting configuration data. Discovery adapter is wrapped by the Configuration prober that invokes Adaptive Communication Channel Building Module (ACCBM) to perform the discovery task.

Provisioning Adapter includes the similar function but the process is in contrary with discovery adapter. Provisioning adapter will obtain model data and invoke provisioning scripts to set the information into the target virtualized environment. Meantime, its wrapper, Configuration Setter will interact with dependency analysis knowledge in MKB to check the property of the configuration setting. Another important module in the CDPA is configuration data reconciliation module. Reconciliation module interacts with dependency analysis module in MKB and checks the integrity of the collected configuration information. If there are any software/hardware components missing or system configuration parameters absent, the reconciliation module would propose to the migration framework for a new round of collection and involve other related CDPA.

6 Related work Virtual Private Cloud [4] focuses on multi-cloud environment as well as ours. It interconnects among cloud resource pools over multi-cloud sites, while our Multi Cloud Management Platform interconnects among cloud portals by a service catalog federation and a collaborative management. So, multi-cloud environments become more usable, using both Multi Cloud Management Platform and Virtual Private Cloud. Both Jamcracker [8] and RightScale [9] provide the capability of connecting an enterprise with different Cloud service providers. Jamcracker focuses more on Business Supporting Services, like account management, metering and billing etc., without a lot of attention to the operations support. RightScale does not provide automated service selections and migrations. The business need of multi-cloud brings the need of moving data among different cloud environment. Related research have been made on migrating different pieces of data into the new environment, for example Database migration[5], session migration[6] and event process migration[7] during a non-stop scenario. In this paper, we look into the migration problem from a new aspect of configuration migration, which makes sure the consistency on both functional and non-functional requirement of the source and target environment.

7 Summary The Multi Cloud Management Platform that we propose in this paper offers a service catalog federation, a collaborative management, an application virtual server migration on unified cloud portals in order to decrease workloads of server users and administrators under multi IaaS cloud sites. A service catalog federation enables server users to select an appreciate service catalog without considering multi cloud sites. A collaborative management brings server administrators unified virtual server management across multi cloud sites. An application virtual server migration provides server users with a means to migrate application-equipped virtual servers across cloud sites.

One of future work that still remain to be addressed is cloud service mashup across multi-cloud sites, such as a combination of virtual server creation service on one cloud site and virtual server monitoring service on an another site. A common service description scheme will be needed to achieve it.

References [1] M. Armbrust and A. Fox et al. Above the

clouds: A Berkeley view of cloud computing, Technical Report EECS-2009-28, UC Berkeley, 2009.

[2] A. Li and X. Yang et al. CloudCmp: Shopping for a Cloud Made Easy, Proceeding of the 2nd USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 10), 2010.

[3] T. Liu, Y. Li, and X. Li, A Collaborative Management as a Service Framework for Managing Internetware Systems, First Asia-Pacific Symposium on Internetware (Internetware 2009), 2009.

[4] T. Wood and R. Shenoy et al. The Case for Enterprise-Ready Virtual Private Clouds, Proceeding of the 1st USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 09), 2009.

[5] Elamparithi, M. Database Migration Tool (DMT) - accomplishments & future directions, Communication and Computational Intelligence (INCOCCI), 2010, p.481

[6] jgaard-Hansen, K. ; Huan Cong Nguyen ; Schwefel, H. Session mobility solution for client-based application migration scenarios, Wireless On-Demand Network Systems and Services (WONS), 2011, p.76

[7] Malkawi, M.; Abaza, M.; Knox, D.;Process migration in virtual memory multicomputer systems, Proceeding of the Twenty-Sixth Hawaii International Conference on System Sciences, 1993, Page(s): 90 - 98 vol.2

[8] Jamcracker, Inc., Unified Cloud Whitepaper, http://www.jamcracker.com/JSDN-Enterprise

[9] RightScale Inc., Right Scale Cloud Management, http://www.rightscale.com/products/features/

[10] Windows is trademarks of Microsoft Corporation. Linux is a registered trademark of Linus Torvalds. Other product or company names are trademarks or registered trademarks of their respective holders.