Simulation Modelling Practice and Theory 49 (2014) 180–192

Contents lists available at ScienceDirect

Simulation Modelling Practice and Theory

journal homepage: www.elsevier .com/locate /s impat

CloudExp: A comprehensive cloud computing experimentalframework

http://dx.doi.org/10.1016/j.simpat.2014.09.0031569-190X/� 2014 Elsevier B.V. All rights reserved.

⇑ Corresponding author.E-mail addresses: yijararweh@just.edu.jo (Y. Jararweh), mjarrah@just.edu.jo (M. Jarrah), kharbutli@just.edu.jo (M. kharbutli), maalshbool@ju

(Z. Alshara), malsaleh@uncc.edu (M.N. Alsaleh).

Yaser Jararweh a,⇑, Moath Jarrah b, Mazen kharbutli b, Zakarea Alshara a,Mohammed Noraden Alsaleh c, Mahmoud Al-Ayyoub a

a Department of Computer Science, Jordan University of Science and Technology, 22110, Jordanb Department of Computer Engineering, Jordan University of Science and Technology, 22110, Jordanc University of North Carolina Charlotte, Charlotte, NC, USA

a r t i c l e i n f o

Article history:Received 5 February 2014Received in revised form 4 August 2014Accepted 10 September 2014

Keywords:Cloud computingCloud computing modeling and simulationCloudSimNetwork topologiesMapReduceSLA managementRain workload generator

a b s t r a c t

Cloud computing is an emerging and fast-growing computing paradigm that has gainedgreat interest from both industry and academia. Consequently, many researchers areactively involved in cloud computing research projects. One major challenge facing cloudcomputing researchers is the lack of a comprehensive cloud computing experimental toolto use in their studies. This paper introduces CloudExp, a modeling and simulation environ-ment for cloud computing. CloudExp can be used to evaluate a wide spectrum of cloud com-ponents such as processing elements, data centers, storage, networking, Service LevelAgreement (SLA) constraints, web-based applications, Service Oriented Architecture(SOA), virtualization, management and automation, and Business Process Management(BPM). Moreover, CloudExp introduces the Rain workload generator which emulates realworkloads in cloud environments. Also, MapReduce processing model is integrated in Clou-dExp in order to handle the processing of big data problems.

� 2014 Elsevier B.V. All rights reserved.

1. Introduction

Cloud computing is an emerging computing paradigm that is continuously evolving and spreading. Many experts believeit will become the dominant IT service delivery model by the end of the decade [1]. Cloud computing is built on a wide rangeof different computing technologies such as high-performance computing, grid and utility computing, distributed systems,virtualization, storage, networking, security, management, automation, Service-Oriented Architecture (SOA), etc. Moreover,other concepts like Business Process Management (BPM), Service-Level Agreement (SLA), Quality of Service (QoS). This com-plexity presents a major challenge for researchers to conduct comprehensive cloud computing-related experiments for twomain reasons: First, conducting experiments on real systems is expensive and the criticality and frangibility of the systemposes many limitations and risks. Such experiments may adversely affect the system’s availability, reliability, and security.Second, while simulation tools would mitigate the problems that arise from using a real system, there are no comprehensivecloud computing experimental tools that cover the wide spectrum of cloud computing components. Unfortunately, current

st.edu.jo

Y. Jararweh et al. / Simulation Modelling Practice and Theory 49 (2014) 180–192 181

cloud computing research tools only cover a subset of the different components listed above, which make them limited andnot applicable when considering new cloud computing technologies.

One of the earliest and most popular cloud computing simulators is CloudSim which an environment developed at theUniversity of Melbourne [14]. CloudSim is a very promising tool for conducting cloud computing experiments. However,CloudSim has several limitations and shortcomings: First, it is built on top of a grid computing environment which puts lim-itations on the infrastructures that can be simulated. This is a major limitation since current cloud systems are built using awide range of different hardware configurations. Second, it only includes a basic and simplified network model with limitedworkload traffic generator. Third, it does not provide efficient modeling for embarrassingly parallel data problems. Fourth, itlacks several important cloud computing components such as BPM and SLA. Fifth, it lacks a Graphical User Interface (GUI)which can be useful and easier for researchers to conduct their experiments.

As a result of the lack of a comprehensive cloud computing experimental tool and due to the limitations in CloudSim, weset out to develop CloudExp as a comprehensive and effective cloud computing experimental framework. CloudExp capabil-ities cover most of cloud computing related technologies such as big data management and mobile cloud computing, etc.CloudExp uses CloudSim as the base design platform and introduces many new enhancements and extensions on top ofit. These enhancements and extensions include:

� Integrating of the Rain cloud workload generator from Berkeley into CloudSim simulator [16].� Integrating of a MapReduce framework into CloudSim to handle embarrassingly parallel data processing paradigms

and Big Data problems.� Adding new modules related to SLA and BPM.� Adding new cloud computing network models (VL2, BCube, Portland, and DCell) that represent actual topologies in

real cloud environments.� Introducing of the Mobile Cloud Computing (MCC) simulation framework.� Introducing a monitoring outlet for most of the cloud system components.� Adding an action module to allow researchers to reconfigure a cloud system and study the overall system behavior

and performance.� Adding a GUI for the simulator to make it more user-friendly.

This paper presents CloudExp and discusses its different components in detail. The rest of the paper is organized as fol-lows: Section 2 provides a comprehensive survey in the field of cloud computing simulation and modeling. CloudSim, whichis the base for CloudExp, is introduced in sub-Section 2.1. CloudExp is fully detailed in Section 3. A use case scenario for Clou-dExp is presented in Section 4 where different simulation results are presented. Finally, Section 5 concludes the paper.

2. Related work

Simulations tools are essential for carrying out research experiments in cloud computing. Several cloud computing sim-ulators have been developed, each of which supports certain aspects/components of cloud computing. This section describessome of the most popular simulators in the fields of cloud computing and distributed systems.

2.1. CloudSim: The underlying framework for CloudExp

CloudSim is a cloud computing modeling and simulation tool that was developed at the University of Melbourne [14]. Itaims to provide cloud computing researchers with an experimental tool to conduct cloud computing-related research. It sup-ports the modeling and simulation of various cloud computing components, including power management, performance,data centers, computing nodes, resource provisioning, and virtual machines provisioning [12].

CloudSim is a layered design framework written in Java and was initially built on top of SimJava and GridSim [4]. BothSimJava and GridSim are discrete event simulators that are widely used to simulate parallel and distributed computing sys-tems. However, SimJava and GridSim have several scalability limitations that led the developers of CloudSim to implement anew discrete event management framework which is the CloudSim core simulation engine [14]. The following is a briefdescription of the different cloud computing features and components modeled by CloudSim.

The Data centers represent the core infrastructure in a cloud system and include the hardware and software stack. Indefining the data centers, the modeler needs to identify the number of hosts in each data center. The Hosts are modeled usingthe class Host which models a physical resource such as a computer or a server. A data center can have multiple hosts. Cloudtasks are represented by Cloudlets which are the cloud-based application services. In this class, computational metrics areused to model the complexity of applications. Brokering is performed through the use of the DatacenterBroker class. Virtualmachines (VM) are modeled in the class VirtualMachine. Coordination is performed using the CloudCoordinator class whichmodels the communication between different cloud coordinators and cloud brokers. In addition, it monitors the internalstate of each data center that is connected to it. Bandwidth provisioning services are modeled using the BWProvisioner class.Memory provisioning and the policies for allocating memory to VMs are modeled using the MemoryProvisioner class. Virtualmachine provisioning is performed using the VMProvisioner class which allocates VMs to hosts.VM allocation policies models

182 Y. Jararweh et al. / Simulation Modelling Practice and Theory 49 (2014) 180–192

the policies for allocating processing power to VMs. Allocation can be space-shared or time-shared. Power consumption ismodeled in the PowerModel class which models data centers power consumption. Due to the space limit, the readers aredirected to the work presented in [14] for more details about CloudSim components.

2.2. iCanCloud

iCanCloud [18] is a cloud computing simulation platform capable of conducting large-scale experiments. It provides ascalable, flexible, fast, and easy to use tool that allows organizations to optimize the trade-offs between the cost and perfor-mance of their cloud computing system. The building block of a cloud computing system in iCanCloud simulator is the vir-tual machine. A virtual machine can be built by selecting and configuring its four main components: CPU, memory, storage,and network. Furthermore, iCanCloud provides a graphical interface that helps users in configuring their cloud computingexperiments. The configuration of a cloud computing system in iCanCloud consists of three main parts: (1) the cloud envi-ronment definition, (2) the cloud hypervisor definition, and (3) the users configuration. The cloud environment definition isspecified using the NED language and defines the names of virtual machine instances to be simulated in the model and thenumber of machines for each instance. The characteristics of the CPU, memory, storage, and network for each virtualmachine are determined by users as well. The CPU is characterized by the processor speed measured in Million InstructionsPer Second (MIPS) and the scheduler (iCanCloud provides a number of defined strategies for the CPU scheduler). The memorysystem configuration defines the size of the memory space and the access latency. The storage system in iCanCloud is mod-eled as file system, volume manager and disk drivers. The data requests are translated through these three components intodata blocks and then read or written to the appropriate disk drive. Applications running on different virtual machines in thesimulated cloud computing model can communicate and interchange data through a defined network. iCanCloud provides anetworking framework that simulates different network protocols (e.g. TCP or UDP) with a high degree of accuracy. It alsoprovides a set of modules reflecting real network devices such as routers, switches and other network protocols which givesthe user the flexibility to simulate different wired or wireless network architectures.

The second part of iCanCloud configuration is the cloud hypervisor definition which specifies the hypervisor model to beused in the simulation. The hypervisor module is responsible for managing the virtual machines and executing the jobsbased on customized brokering policies. iCanCloud provides an interface for customizing the hypervisor module by integrat-ing new brokering policies that determine how to select the jobs to be executed and on which virtual machine in the cloud.The last part of the configuration is the users configuration. The users are modeled as entities that submit jobs to be executedby the existing virtual machines in the cloud. iCanCloud is provided with a list of users who are currently submitting jobs tothe computing system. For each user we have a list of jobs and the configuration of virtual machines that can execute eachjob. The responsibility of selecting the appropriate virtual machine to execute a job can be delegated to the hypervisor sincemost users do not have such expertise. The datacenters in iCanCloud can be represented by a set of virtual machines. How-ever, iCanCloud does not currently provide models for power consumption evaluation.

CloudAnalyst [19] is a simulator that enables the user to model software as a service data centers in different geographicallocations. The output of the experiments in CloudAnalyst consists of the response time for each request and the cost toaccomplish the requested task. The cost is based on EC2 cost policy from Amazon. GreenCloud simulator [20] is an extensionof the popular network simulator NS-2. GreenCloud focuses on simulating the communication between processes running ina cloud computing infrastructure. It simulates the packets exchanged and provides power consumption analysis of data cen-ter components (e.g. server and switches).

Table 1 shows a comparison between CloudExp and the two major simulators in the field: CloudSim and iCanCloud. Thetable shows that CloudExp covers a much wider spectrum of features compared to the other simulators making it more effec-tive and comprehensive for cloud computing research. Its worth to mention here that while CloudSim is already supportingsome of the functionalities like the power consumption calculation, CloudExp extend this functionality by using a better cal-culation model that take in consideration power consumption for all the components of the data center including the net-work power consumption.

Table 1Comparison of cloud simulators.

Metric CloudSim iCanCloud CloudExp

GUI support No Yes YesDynamic workload No No YesNetworking models Limited Moderate FullMobile cloud computing simulation No No YesPower consumption calculations Yes No YesService level agreement Yes No YesMapReduce support No No Yes

Y. Jararweh et al. / Simulation Modelling Practice and Theory 49 (2014) 180–192 183

3. CloudExp framework

The features that CloudSim provides are promising for cloud computing research. However, it lacks several componentsneeded for the simulation of comprehensive cloud computing experiments. CloudExp integrates several new features andimportant components to CloudSim. This makes it a more comprehensive simulation environment for current cloud comput-ing systems. The added components include a more-detailed workload generator Section 3.1, modules for MapReduce Sec-tion 3.2, modules for mobile cloud computing Section 3.3, extensions for SLA and BPM Section 3.4, a wide range of networktopologies Section 3.5, and a Graphical User Interface (GUI). The new extensions and additions are explained in details in thissection.

A GUI extension is provided in order to make it convenient for users to conduct research experiments by creating themain components in a cloud system. For example, Fig. 1 shows how a user can create data centers and define their param-eters such as the number of hosts on each data center. Also, its shows the creation of virtual machines and brokers inCloudExp.

3.1. Workload generation and the rain workload generator

Cloud computing systems promise to handle and fulfill different customers’ applications, requested services and work-load requirements. These applications and services vary widely ranging from data intensive (i.e. big-data) applications tofinancial applications and web services. Moreover, cloud workloads differ in their size, execution times, memory footprints,network BW requirements, frequency, and QoS requirements, among other factors. Such diverse workloads are very dynamicand exhibit different patterns of execution phases during an application’s life span. Hence, cloud computing systems arerequired to dynamically adapt their operational environment on the fly to cope with these workload dynamics. Workloadscharacterization is very crucial in a cloud computing system in order to achieve proper resources provisioning, cost estima-tion, and SLA considerations. Cloud computing application developers and hosting companies need to predict changes in theworkload patterns to be able to adjust promptly when variations in the patterns occur. Moreover, a detection of any possiblesystem bottlenecks is needed before actually porting the application to the real system. Cloud application updates afterdeployment may also cause problems in the actual cloud system because they might change the workload patterns anddynamics. These changes affect the cloud system resources allocation and QoS measurements. As a result, a mechanismto test cloud applications before their actual deployment is of the utmost necessity.

Based on the aforementioned facts, an accurate and comprehensive workload modeling environment is essential in anycloud computing simulation tool. Current workload modeling environments for systems such as grid computing do not fit ormatch the characteristics of real workloads in cloud computing environments. CloudSim provides a basic cloud workloadmodeling structure that fails to capture many of the characteristics of current cloud computing workloads. Hence, CloudSimusers need to extend the available modules to generate the required patterns. CloudSim models the workload as an

Fig. 1. Creating a data centers with virtual machines and cloud brokers in CloudExp.

184 Y. Jararweh et al. / Simulation Modelling Practice and Theory 49 (2014) 180–192

extension to the Cloudlet entity and introduces the utilization model which is an abstract class that needs to be extended bythe users to implement the workload patterns related to the application[14]. The Utilization Model provides methods andvariables in order to specify the resources requirements of the applications during the deployment phase. The input to thismethod is a discrete time parameter, while the output is a percentage of the resources that are required by the Cloudlet. Themain drawback of this approach stems from the fact that the user manually needs to implement an accurate model for theapplication’s workload patterns. This is not an easy task and incur an extra overhead on the user.

In order to solve this problem, CloudExp is integrated with Rain workload generator framework from the University ofCalifornia at Berkeley [16] which is an open-source software. The Rain framework presents an accurate and comprehensivecloud computing workload characteristics. Hence, CloudExp replaces the basic workload generator framework available inCloudSim by integrating the Rain workload generator. The integration process is simplified by the fact that both CloudSimand Rain were written in Java. The three main workload characteristics that Rain provides are: (1) Variations in workloadintensity (e.g. small, medium, and large). (2) Variations in the mix of operations that are performed by the system compo-nents (e.g. data intensive, computing intensive, and task dependencies). (3) Variations in the data access patterns and fre-quency, also known as data hot spots. Predicting data hot spots enables efficient access to them which reduces accesstime and overhead. next we address the architecture of the Rain workload generator.

3.1.1. Rain workload generator architectureRain is a unique statistics-based workload generator. It provides the user with the ability to easily reuse, configure, and

schedule cloud applications workloads. The variations in the requirements of the applications in Rain are achieved usingempirical probability distributions. Rain architecture handles the variations using load scheduling mechanisms. The changesin the mix of operations, the access patterns, and data hot spots are done using application-specific request generators [16].Moreover, Rain architecture differs from other workload generators in handling workload requests. While current generatorscouple request generation with its execution, Rain decouples request generation from request execution which adds flexi-bility how to manage requests. Another limitation of current generators is related to the fact that the thread that generatesthe request is also responsible for its execution; this is known as thread-affinity problem. Thread-affinity problem impactsthe performance of the generator as one thread cannot generate new requests until it completes the current one. Rain solvesthis problem by allowing any request generated by a certain thread to be executed by any available thread and not only bythe one who generates it. This solution provides flexibility, scalability and improves the performance.

The Rain architecture consists of the following components:

(1) The Scenario component is responsible for the experiment configuration parameters (e.g. maximum number of users,experiment duration, etc.).

(2) The Generator component uses the configuration from the Scenario component to create requests and operations to beused by other components.

(3) The Scoreboard component is used to present experiment results and summary.(4) The Threading component contains the available threads for executing the tasks produced by the request generator

component.(5) The Benchmark component is responsible for running the entire experiment. It interacts with other components by

loading the Scenario that needs to be handled by the threading components. It also initializes the Scoreboard and pre-sents the results at the end of the experiment.

3.1.2. Rain workload generation scenarioA new workload will be initialized by using the Scenario where each entity, i.e. user, will be assigned to a certain generator

and a thread. The thread requests a new operation from the generator. When the thread finishes executing the operationsassigned by the generator, it generates a summary (e.g. status, execution results, etc.) and writes its details on the Scoreboard.All these steps are controlled by the Benchmark component. CloudExp users can use the workload generator in differentways. First, users can experiment with different workload patterns and mix of operations while using a fixed system config-uration (e.g. VMs, network, etc.). This can help in understanding how a workload can impact the utilization of a cloud com-puting components and the SLA. Second, users can conduct experiments with different provisioning algorithms and assesshow these algorithms react to workload variations. Third, users are able to study the impact of different workload patternsand characteristics. Fourth, users can apply an intensive workload scenario to detect bottlenecks in a cloud infrastructure.

3.2. MapReduce model

MapReduce is widely used as a powerful parallel data processing model [5]. It is a programming model that has efficientlysolved problems of large datasets using large clusters of machines. Examples of applications that utilize MapReduce includedistributed grep, distributed sorting, web-link graph reversal, web-access log stats analysis, document clustering, andmachine learning [7]. Cloud computing offers a suitable environment for processing and analyzing terabytes of data throughthe utilization of many resources connected through a network topology [9]. Most cloud providers such as Amazon EC2,Microsoft Azure, Google, Yahoo and Facebook, adopted MapReduce in their computing environments.

Y. Jararweh et al. / Simulation Modelling Practice and Theory 49 (2014) 180–192 185

Due to the fact that it is difficult to perform benchmarking experiments for MapReduce in real infrastructures, CloudExpprovides a simulation solution to overcome that. Users can use CloudExp toolkit to perform experiments under non-staticconditions (e.g. availability and workload pattern) in a controllable environment where tests can be re-executed and re-configured.

3.2.1. Design and implementation of MapReduceThis subsection provides finer details about the fundamental classes of MapReduce. The class design diagram for MapRe-

duce and its correlation with CloudSim is shown in Fig. 2. The list of classes are:

1. Master: This class contains instances of all mappers and reducers that belong to the same user. Also, it contains infor-mation about the status and data locations during the execution of the mappers and reducers.

2. Map: It models the mappers’ information and behavior such as the input dataset size, output dataset size, and utili-zation model.

3. Reduce: It models the reducers’ information and behavior such as the input dataset size, output dataset size, and uti-lization model.

4. MapReduceSchedulerSpaceShared: This class extends an abstract class CloudletScheduler. It represents the behaviorpolicy between the mappers and reducers. For example, it determines the share policy of processing power amongmappers and reducers in a virtual machine.

In the MapReduce approach, the input data is partitioned into a set of sub-problems which in turn can be partitioned fur-ther into smaller sub-problems. This allows the computing grid or cluster to process small size problems in parallel. The out-put is constructed by collecting all the results back from the sub-problems in a fashion that yields a correct result for theinitial problem. The illustration in Fig. 3 shows the flow of a MapReduce operation [5]. When a user program simulatesMapReduce tasks, the following sequence of actions occurs:

1. The workload in the user experiment is read and parsed to initiate the list of map and reduce instances. Each instancehas parameters that are used to simulate the MapReduce model. The parameters include: ID, input data size, outputdata size, and utilization models.

2. The master node assigns the instances created in the previous step to computing nodes in the cluster. The master alsokeeps track of mappers and reducers information such as datacenter id, host id, virtual machine id, and a status (cre-ated, ready, waiting, running, success, fail, canceled). Then each mapper or reducer is submitted to the cloud comput-ing environment using the predefined utilization models with an idle state called ready state.

3. After the submission of the mappers (initially in ready state), the simulation of MapReduce starts while taking intoaccount each mapper’s status and the utilization model.

4. When a mapper finishes processing, it stores the dataset results in a specific location and informs the master of theaddress.

5. When all mappers finish processing, the master sends signals to all reducers to start working on the datasets thatwere produced.

6. During MapReduce simulation, CloudExp toolkit continuously collects data to be displayed for statistical analysis afterthe simulation is over.

Fig. 2. MapReduce class design diagram.

Workload

Master

Worker

Worker

Worker

Worker

Worker

Output File

Output File

Input File

Input File

Input File

(1) fork (1) fork

(1) fork

(2) assign map

(2) assign reduce

(3) read (4) map finished (5) start reduce

(6) reduce finished

Input files Map phase Intermedate files Reduce phase Output files

Fig. 3. Flow of MapReduce operation.

186 Y. Jararweh et al. / Simulation Modelling Practice and Theory 49 (2014) 180–192

As shown in Table 5, CloudExp provides many MapReduce configuration parameters such as the scheduling algorithmslike FIFO, Matchmaking and Delay algorithm. Number of Mappers and Reducers, the input and the output files size, numberof files per task and number of CPUs per task. A complete example for using CloudExp to simulate MapReduce workload isshown in Section 4.3.

3.3. Mobile cloud computing

Nowadays, smartphones and tablet PCs are more widespread and users are relying more on them compared to con-ventional personal computers. This has lead to an unprecedented growing market for mobile devices. However, mobiledevices suffer from many limitations related to their limited resources. Limitations include: connectivity issues, limitedbandwidth, security vulnerabilities, applications compatibility, and a restricted power since they are mostly poweredby batteries.

An emerging concept of Mobile Cloud Computing (MCC) is showing a great momentum nowadays within mobile ser-vice providers and users. It is based on the integration of cloud computing providers and the mobile service system. Thisintegration is achieved by the possibility of offloading mobile data and intensive computation requirements to cloud com-puting infrastructures [21]. This enables the mobile device to migrate the intensive computation tasks to cloud infrastruc-tures and preserve its limited resources. In addition, mobile devices would no longer require large storage capacity on thedevice as the user data could be stored in a cloud storage system. The user can easily retrieve his/her data whenever isneeded. MCC research is expanding quickly and many researchers are engaged in MCC research topics such as data off-loading, mobile-cloud subscription, and security. To the best of our knowledge, there are no simulation tools that considerMCC simulation so far. On CloudExp, we present a new simulation module that handles MCC simulation. The MCC mod-ule in CloudExp provides the basic capabilities for users to conduct MCC related experiments. Using CloudExp, theresearcher can configure the MCC module to create and simulate MCC- related experiments. Many parameters can beused to configure the MCC related experiments. These parameters are ranging from number of mobile users, connectiontype, connection bandwidth, data rate, and authentication mechanism. The authors in [22,23] exploit CloudExp to suc-cessfully simulate MCC related experiments.

3.4. Service level agreement and business process management

Service Level Agreement (SLA) and Business Process Management (BPM) are considered among the basic pillars of today’scloud computing environments [11]. In CloudExp we made efforts to introduce SLA to the users in a simple and effectivemanner. Starting from the basic definition of SLA as a contract between the service provider and the service requester. Clou-dExp provides a set of measurable terms that the user can change while studying their impact on other system components.

Y. Jararweh et al. / Simulation Modelling Practice and Theory 49 (2014) 180–192 187

These measurable terms include, but are not limited to: number of users who can use the system simultaneously (e.g. 1000active user), service availability (e.g. 99.9%), service cost, service outage handling terms, business continuity and disasterrecovery, network performance, security measures, service metering tools available to the user, user compensation in caseof SLA violation, and customer support. To make things more realistic, CloudExp connects the SLA terms related to an appli-cation’s sensitivity to the availability of the service. This is because some applications require an availability of not less than100%. CloudExp enables the users to study the effect of SLA terms on the different cloud computing components. In addition,it provides the service cost with different SLA configurations and different QoS requirements. Furthermore, CloudExp inte-grates the Web Service Level Agreement (WSLA) framework that handles the SLA monitoring and enforcement [13]. SLAparameters used in Cloudexp are number of users, service availability, service cost, network performance and SLA violation.

3.5. Network topologies in CloudExp

Data centers in a cloud computing infrastructure normally contain thousands of physical servers that are connectedthrough switches, routers and other network devices. Hence, the proper design of the data center network plays a criticalrole in the cloud environment because it directly affects the performance and the throughput of cloud applications. VMFlow[17], for example, is a framework for placement and migration of VMs to reduce the cost. It is shown in the paper that thenetwork topology along with network traffic demands should be taken into consideration in order to meet the objective ofnetwork power reduction. Conventional data centers are organized in a hierarchy of three layers: core, aggregation and serv-ers. The core layer consists of the core routers that connect the aggregation switches to the Internet. In the servers layer, theservers are organized in racks and connected to access switches (normally called Top-of-Rack switches) which are in turnconnected to the aggregation switches. The aggregation layer may provide some functions such as domain service, locationservice, server load balancing, and more [1]. This simple design suffers from some limitations as stated in [2]. First, whilegoing up in the hierarchy, it becomes very hard technically and financially to sustain high bandwidth which results in pre-venting idle servers from being assigned to overloaded servers. This affects the data center’s performance. Second, the pro-tocols that are normally used in the core and aggregation layers limit the utilization of the links to less than 50% of themaximum utilization capacity. Third, the performance of communication depends on the distance and the addressingscheme of the hierarchy which causes the services to scale up to nearby servers for rapid response time and performance.This requires some unused capacity to be reserved for a single service for future scale up and not to be shared with others.

CloudSim provides a module for network topology which is the NetworkCloudSim. It was developed to simulate thebehavior of parallel and distributed applications in cloud computing environments [6]. NetworkCloudSim offers the basicentities and classes to simulate a network through the connection of Switchs (RootSwitch, AggregateSwitch and Edge-Switch), NetworkDatacenter and NetworkDatacenterBroker.

However, NetworkCloudSim does not support the common network topologies in cloud computing such as VL2 [2],BCube [10], Portland [15], and DCell [3]. Hence, these topologies were integrated into our CloudExp tool [8]. Also, CloudExpsupports a graphical interface to easily drag and drop entities, define their properties, and establish the connections betweenthem which results in a customized network topology.

4. Simulation and experiments scenarios using CloudExp

CloudExp offers a wide spectrum of functionalities and capabilities to its users, which would help them in understandingthe different aspects of cloud computing environments. Users can simulate experiments by specifying some infrastructureconfiguration. To demonstrate some of the functionalities of CloudExp, we will use two different set of experiments as ashowcase examples and present a the results obtained using CloudExp. The first set of experiments is showing cloud datacenter resources utilization and power consumption with different work load scenario and SLA constrains. The second setof experiments is presenting Map Reduce task scheduling experiments using CloudExp. The workload characteristics usedin both cases are described in the next section.

4.1. Workload characteristics

One of the main contributions of CloudExp is the integration of the Rain workload generator into the CloudSim system.Rain workload generator demonstrates its near optimality in correctly characterizing cloud like workloads [16]. All theexperimental results presented in the next two sections are based on the Rain workload generator.

The workload used in the experiments represents a social-event calendar web application named Olio with many oper-ations like adding events, viewing event details, attending events, adding new users, viewing user profile details, searching[16]. The workload will be initialized by using a Scenario where each entity, i.e. user or a task, will be assigned to a certaingenerator and a thread. The thread requests a new operation from the generator. This operation will be executed in theassigned VM. When the thread finishes executing the operations assigned by the generator, it generates a summary (e.g. sta-tus, execution results, etc.) and writes its details on the Scoreboard. The number of processes executing in each VM is basedon the VM utilization and may vary with time. The experiments are lasted for 24 h with a space-shared VM allocation policybased on the resource utilization.

188 Y. Jararweh et al. / Simulation Modelling Practice and Theory 49 (2014) 180–192

4.2. General CloudExp experiments

The first scenario assumes that a user with ID ¼ 0 has a cloud environment that consists of one data center (datacenter1)with the parameters listed in Table 2. Moreover, the data center has ten identical physical nodes (hosts) each has the prop-erties listed in Table 3. Up to five virtual machines (VM) are executing on each host which results in fifty VMs for the wholedata center. In addition, in this scenario, VL2 network topology is used with the specifications listed in Table 4.

Fig. 4 shows the utilization of different components in the data center (datacenter1) along with its power consumption asa result of a continuous and steady increase in the workload intensity over time. The workload intensity is represented by thenumber of VMs running on the data center. As expected, the figure shows that the utilization of RAM, CPU, and bandwidthincrease as the workload intensity (number of VMs) increases. The power consumption also increases.

Fig. 5 shows the utilization of different components the data center (datacenter1) along with its power consumption inthe case of mixed workload with dynamically-changing intensity over time. As expected, the figure shows that the systemutilization and power consumption increases/decreases over time based on the workload intensity.

Fig. 6 shows the data center’s utilization and power consumption with an increasing workload intensity and a predefinedSLA. The example assumes that the system starts violating the SLA when the utilization of system components reaches a cer-tain threshold (e.g. RAM utilization reaches 87% and above). The user can monitor and predict possible SLA violations andcan, consequently, increase the physical resources (nodes) to guarantee that the system is always running in a balancedbehavior. The figure demonstrates how an increase in the number of nodes results in a drop in the system utilization towithin the SLA constraints.

4.3. MapReduce experiments

In our second scenario, we will use the same cloud configuration for the data center (datacenter1), the hosts and the net-work presented in Tables 2–4, respectively, to demonstrate CloudExp’s capabilities of simulating MapReduce related exper-iments with a large number of tasks. Table 5 shows the MapReduce task properties and the experiment configurations. Theuser can customize his experiments using the MapReduce configuration file which contains the parameters of Table 5. In this

Table 2Data center configuration.

ISA x86

Operating system LinuxVirtual Machine Monitor (VMM) XenTime zone current timeCost per processing second 0.03 centsCost per memory unit 0.05 centsCost per storage unit 0.001 centsCost per bandwidth unit 0.01 centsVirtual machine allocation policy Allocate VM to the host with lowest utilization

Table 3Host properties.

Host ID A unique ID automatically generated in the range 0–3

Storage capacity 1 TBNumber of CPUs 2Cores per CPU 4MIPS for each core 1024Memory capacity 4 GBBandwidth 10 MbpsVirtual machine scheduler Space shared

Table 4VL2 network topology structure.

Switching levels 3 levels

Types of switches One root switch, two aggregate switches, two edge switchesDelays Root switch = 0.00285 ms; aggregate switch = 0.00245 ms; edge switch = 0.00157 msBandwidth Root switch = 100 Mb; aggregate switch = 100 Mb; edge switch = 40 Mb

10

20

30

40

50

60

70

80

90

10 15 20 22 25 30 33 35 40 50 50U

tiliz

atio

n %

VMs

CPUBW

Ram

2.7

2.8

2.9

3

3.1

3.2

3.3

3.4

3.5

3.6

10 15 20 22 25 30 33 35 40 50 50

Pow

er /

Wat

t

VMs

Power

Fig. 4. Datacenter1 utilization and power consumption with a continuous increase in the workload intensity.

10

20

30

40

50

60

70

80

90

20 15 10 25 33 50 40 30 50 22 35

Util

izat

ion

%

VMs

CPUBW

Ram

2.7

2.8

2.9

3

3.1

3.2

3.3

3.4

3.5

3.6

20 15 10 25 33 50 40 30 50 22 35

Pow

er /

Wat

t

VMs

Power

Fig. 5. Datacenter1 utilization and power consumption with a dynamic workload intensity mix.

Y. Jararweh et al. / Simulation Modelling Practice and Theory 49 (2014) 180–192 189

10

20

30

40

50

60

70

80

90

100

10 15 20 22 25 30 33 35 40

50/1

0

50/1

0

50/1

1

50/1

2

50/1

3

50/1

4

50/1

5

50/1

6

Util

izat

ion

%

VMs/Number of Hosts

BWCPURam

threshold 2threshold 1

2.5

3

3.5

4

4.5

5

5.5

6

10 15 20 22 25 30 33 35 4050

/10

50/1

050

/11

50/1

250

/13

50/1

450

/15

50/1

650

/17

Pow

er /

Wat

t

VMs / Number of Hosts

TotalThroughput 1Throughput 2

Fig. 6. Datacenter1 utilization and power consumption with dynamic resources configuration and SLA constraints.

Table 5MapReduce task properties.

Task ID ID (0-number of tasks)

Scheduling algorithm MatchmakingTask length Random between (1–500) MIInput file size 64 MBOutput file size 64 MBNumber of files per task 1Number of mappers 20 CPUs with 4 cores eachNumber of reducers 4 CPUs with 4 cores eachNumber of VMs per node Up to 8 VMs per node

190 Y. Jararweh et al. / Simulation Modelling Practice and Theory 49 (2014) 180–192

experiment, we simulated different numbers of tasks (Olio search operations), e.g. 10000, 30000, 50000 and 100000, usingthe Matchmaking scheduling algorithm with a random task length measured in Million Instructions (MI) unit. Both inputand output files’ sizes are set to 64 MB. All the computing capacity for (datacenter1) is used in this experiments with a totalof 20 CPUs (i.e. 80 cores), 4 MB of the total memory and a maximum of 8 VMs per physical node. The number of Mappers isthe default value which is 20 CPUs in our experiments representing the total computing capacity available in (datacenter1).The user can define any other number of Mappers. The number of Reducers is set to be 4; the default number of reducer is 1.In our experiments the MapReduce workload is represented as a fixed number of Olio search tasks with the specificationsavailable in Table 5. These parameters are user defined parameters and can be changed based on the user needs. The outputof this experiment include the total simulation time, which represent the total execution time of each set of tasks, and thetotal power consumptions with different number of VMs.

Fig. 7 shows the total execution time (in seconds) and the total power consumption (in KWH) for different MapReduceworkload sizes ranging from 10000 tasks to 100000 tasks using the same physical nodes with different numbers of VMsper physical node. With increasing of the workload size, more execution time and power are needed. On the other hand,using more VMs per physical node will reduce the total execution time and power consumption as we are processing moretasks in parallel without using more resources.

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

1 VM 2 VMs 4VMs 6 VMs 8 VMs

Sim

ulat

ion

Tim

e

Number of VMs

10000 Tasks30000 Tasks50000 Tasks

100000 Tasks

0

10

20

30

40

50

60

70

80

1 VM 2 VMs 4 VMs 6 VMs 8 VMs

Pow

er C

onsu

mpt

ion

(KW

H)

Number of VMs

10000 Tasks30000 Tasks50000 Tasks

100000 Tasks

Fig. 7. MapReduce tasks simulation time and power consumption.

Y. Jararweh et al. / Simulation Modelling Practice and Theory 49 (2014) 180–192 191

5. Conclusion and future work

This paper presented CloudExp, a comprehensive, easy-to-use, and efficient cloud computing modeling and simulationtoolkit. CloudExp fills a large gap in cloud computing research caused by the lack of having a comprehensive and easy-to-use tool, in addition to the limited availability and the high costs of conducting experiments in real cloud computing envi-ronments. CloudExp provides a rich, yet simple, GUI to build cloud infrastructure and present results and charts to theresearch to enable configuration analysis and evaluation. CloudExp allows users to customize all aspects in a cloud infra-structure from the host processing nodes to the network topology. In addition, CloudExp provides models for popular andstate-of-the-art technologies such as MapReduce and Mobile Cloud Computing. CloudExp allows users to investigate theSLA and other business aspects into the tool. Furthermore, it includes an extensive workload generator capable of accuratelyrepresenting real cloud infrastructure workload. Moreover, the modularity in CloudExp’s design allows users to integratenew components or extend existing ones easily and effectively.

CloudExp makes it easy for users to comprehend the different cloud system components and their roles in the whole sys-tem. Users can modify the different components and their parameters, run simulations, and analyze results.

References

[1] Q. Zhang, L. Cheng, R. Boutaba, Cloud computing: state-of-the-art and research challenges, J. Internet Services Appl. 1 (1) (2010) 7–18.[2] A. Greenberg, J.R. Hamilton, N. Jain, S. Kandula, C. Kim, P. Lahiri, D.A. Maltz, P. Patel, S. Sengupta, VL2: a scalable and flexible data center network,

SIGCOMM Comput. Commun. Rev. 39 (4) (2009) 51–62.[3] C. Guo, H. Wu, K. Tan, L. Shi, Y. Zhang, S. Lu, Dcell: a scalable and fault-tolerant network structure for data centers, in: Proceedings of the ACM

SIGCOMM 2008 Conference on Data Communication, Seattle, WA, USA, 2008, pp.75–86.[4] R. Buyya, R. Ranjan, R.N. Calheiros, Modeling and simulation of scalable Cloud computing environments and the CloudSim toolkit: Challenges and

opportunities, in: Proceedings of the International Conference on High Performance Computing and Simulation (HPCS), 2009, Leipzig, Germany, 2009,pp.1–11.

[5] J. Dean, S. Ghemawat, Mapreduce: simplified data processing on large cluster, in: Proceedings of the 6th Conference on Symposium on OperatingSystems Design and Implementation, 2004, USENIX Association, Berkeley, CA, USA, 2004.

[6] S.K. Garg, R. Buyya, NetworkCloudSim: modelling parallel applications in cloud simulations, in: Proceedings of the 4th IEEE International Conferenceon Utility and Cloud Computing (UCC 2011), Melbourne, Australia, 2011.

192 Y. Jararweh et al. / Simulation Modelling Practice and Theory 49 (2014) 180–192

[7] Z. Xiao, Y. Xiao, Accountable MapReduce in cloud computing, in: Proceedings of the IEEE Conference on Computer Communications Workshops(INFOCOM WKSHPS), 2011, pp. 1082–1087.

[8] Y. Jararweh, Z. Alshara, M. Jarrah, M. Kharbutli, M.N. Alsaleh, TeachCloud: Cloud Computing Educational Toolkit, in: Proceedings of the 1st InternationalIBM Cloud Academy Conference (ICA CON 2012), North Carolina, USA, April 2012.

[9] M. Armbrust, A. Fox, R. Griffith, A.D. Joseph, R.H. Katz, A. Konwinski, G. Lee, D.A. Patterson, A. Rabkin, I. Stoica, et al. Above the clouds: a Berkeley view ofcloud computing, EECS Department, University of California, Berkeley, Tech. Rep. UCB/EECS-2009-28, 2009.

[10] C. Guo, G. Lu, D. Li, H. Wu, X. Zhang, Y. Shi, C. Tian, Y. Zhang, S. Lu, BCube: a high performance, server-centric network architecture for modular datacenters, in: Proceedings of the ACM SIGCOMM 2009 Conference on Data communication, Barcelona, Spain, 2009, pp. 63–74.

[11] F. Faniyi, R. Bahsoon, Engineering proprioception in SLA management for cloud architectures, in: Proceedings of the 9th Working IEEE/IFIP Conferenceon Software Architecture (WICSA), Boulder, CO, USA, 2011, pp. 336–340.

[12] K.M. Kim, A. Beloglazov, R. Buyya, Power-aware provisioning of cloud resources for real-time services, in: Proceedings of the 7th InternationalWorkshop on Middleware for Grids, Clouds and e-Science, Urbana Champaign, Illinois, USA, 2009.

[13] P. Patel, A. Ranabahu, A. Sheth, Service level agreement in cloud computing, in: Cloud Workshops at OOPSLA09, 2009.[14] R.N. Calheiros, R. Ranjan, A. Beloglazov, C.A.F. De Rose, R. Buyya, CloudSim: a toolkit for modeling and simulation of cloud computing environments

and evaluation of resource provisioning algorithms, Software: Pract. Exp. 41 (1) (2011) 23–50.[15] R.N. Mysore, A. Pamboris, N. Farrington, N. Huang, P. Miri, S. Radhakrishnan, V. Subramanya, A. Vahdat, PortLand: a scalable fault-tolerant layer 2 data

center network fabric, SIGCOMM Comput. Commun. Rev. 39 (4) (2009) 39–50.[16] A. Beitch, B. Liu, T. Yung, R. Griffith, A. Fox, D.A. Patterson, Rain: A Workload Generation Toolkit for Cloud Computing Applications, Technical Report

UCB/EECS-2010-14, 2010.[17] V. Mann, A. Kumar, P. Dutta, S. Kalyanaraman, VMFlow: leveraging VM mobility to reduce network power costs in data centers, in: Proceedings of the

10th International IFIP TC 6 Conference on Networking, Valencia, Spain, 2011.[18] Alberto Nez, Jose L. Vzquez-Poletti, Agustin C. Caminero, Gabriel G. Casta, Jesus Carretero, Ignacio M. Llorente, iCanCloud: a flexible and scalable cloud

infrastructure simulator, J. Grid Comput. 10 (1) (2012) 185–209.[19] Wickremasinghe Bhathiya, N. Rodrigo, Rajkumar Buyya Calheiros, CloudAnalyst: a CloudSim-based visual modeller for analysing cloud computing

environments and applications, in: Proceedings of the 24th International Conference on Advanced Information Networking and Applications (AINA2010), Perth, Australia, April 20–23, 2010.

[20] Kliazovich Dzmitry, Bouvry Pascal, Samee Ullah Khan, GreenCloud: a packet-level simulator of energy-aware cloud computing data centers, in: 2010IEEE Global Telecommunications Conference (GLOBECOM 2010).

[21] Niroshinie Fernando, Seng W. Loke, Wenny Rahayu, Mobile cloud computing: a survey, Future Gener. Comput. Syst. 29 (1) (2013) 84–106. ISSN: 0167-739X, doi:10.1016/j.future.2012.05.023.

[22] Muhannad Quwaider, Yaser Jararweh, Cloudlet-based for big data collection in body area networks, in: Proceedings of the International Conference forInternet Technology and Secured Transactions (ICITST), London, UK, December 2013.

[23] Muhannad Quwaider, Yaser Jararweh, Cloudlet-based efficient data collection in wireless body area networks, Simulat. Modell. Pract. Theory 0 (2014)5. doi:10.1016/j.simpat.2014.06.015.