Embed Size (px)
Citation preview
Simulation of heterogeneous cloud
infrastructures
Konstantinos Giannoutakis
Information Technologies Institute/
Centre for Research and Technology
Hellas
ITI/CERTH
Overview • Introduction
• Simulation
• Towards a framework for simulating heterogeneous clouds
• Conclusions
Cloud
Environments
• Cloud environments are becoming more popular during the past
decades.
• This fact is due to the flexibility of Cloud environments in resource
allocation as well as resiliency in both software and underlying
hardware.
• According to Forbes, 92% of the total workload will be executed in
Cloud environments by 2020.
• Moreover, by 2020 hyper-scale data-centers will be 485 (259 in
2015 or 21% of all installed data-center servers) or 47% of all
installed data-center servers. (Cisco Global Cloud Index)
• Three major Cloud providers (Microsoft, Amazon, Google) have
almost 1.5 million data-centers.
Cloud
Environments
• Traditional Cloud environments are formed using CPU-based data-
centers and their architecture is based on the Warehouse Scale
Computer (WSC) (Barroso and Holzle, 2009).
• Recently, heterogeneous hardware such as:
- GPUs
- Intel MICs
- FPGA
- High Performance Clusters
started to be integrated in the Cloud in order to be used for
processing more demanding and specialized workloads (i.e. HPC
applications), while simultaneously decreasing energy
consumption.
• However, the addition of such hardware substantially increases
complexity of monitoring this hardware, provision it or developing
software that can fully take it into advantage.
Cloud
Environments
Cloud
Environments
Despite the Pros in modern Cloud environments there are also Cons
which include:
• Overprovisioning: More resources are installed than actually
required in order to match user requests.
• Underutilization: Utilization of modern Clouds is very low (20%-
30%) resulting to increased power consumption.
• Management Issues: Due to the continuously growing scale and
heterogeneity of modern Cloud environments the centralized
management is not effective since more and more outdated data is
used for resource provisioning.
• Organization Issues: Organization of resources to maximize
utilization and "sharpen" the choice of adequate hardware to match
accurately end-user needs is based on global decisions.
Cloud
Environments
• Most of these problems can be tackled effectively by local
decisions based on a hierarchical self-organization and self-
management system.
• The CloudLightning project aims in using self-organization and
self-management strategies to effectively manage heterogeneous
resource at hyper-scale.
• However, one question remains: How we evaluate, study or
improve hyper-scale Cloud environments, especially since most
hyper-scale environments belong to private companies?
• The answer is: Simulation
Requirements
for hyper-scale
simulations
What are the key requirements, that limit existing DES simulators, for
hyper-scale simulations?
• Very large amount of computations.
• Accurate models for power consumption based on adequate
interpolating models.
• Native parallel design in order to be able to execute in HPC
environments.
• Support for tasks that can span across multiple Virtual Machines
(VMs).
• Support for accelerators (GPU,MIC,DFE).
• The simulator should be designed in a language that is build for
high performance computations (i.e. C or C++).
With the above in mind, a new simulator has been build
(CloudLightning simulator).
Architecture • In order to design a simulator for large scale phenomena we can
borrow the design from large scale Engineering and Physics
simulations.
• These simulations are based on a time-advancing loop with
prescribed time granularity.
• The time advances from t0 = 0 to tend with a prescribed sampling
step tstep. (in seconds, milliseconds, etc.).
• This design enables for integration of dynamical components,
since the state of these components can be updated with respect
to tstep.
• This time-stepping approach allows for a dynamic resolution of the
results, since a large time-step will only reveal a coarse picture of
the system, while a small time-step will reveal more fine
interactions.
Architecture
Abstract Cloud architecture with one Cell with one data-center
Architecture Note: A Cell can contain multiple data-centers (WSCs) but one broker
(Cell Manager).
Algorithm 1 Driver for the hyper-scale Cloud Simulator
1: Initialize data-centers, network, storage
2: for t = t0 = 0 to tmax with step tstep do
3: Create task queue in Gateway Service at time t
4: Send tasks to Broker
5: Receive tasks from Broker and find adequate resources
6: Assign tasks to the resources
7: Perform update on the affected components from the task-assignment
8: end for
Parallelization • The parallelization of the simulator is a two stage process (Coarse-
Fine grain parallelization).
• The Gateway Service is residing in the Head Node and is
responsible for creating the task queue and sending tasks to the
Cells.
• Each Cell resides in one multi-core compute node.
• Coarse grain parallelization can be performed via the Message
Passing Interface (MPI).
• The communications between the Gateway Service and the Cells
are minimal, thus even for large number of incoming tasks the
overall running time is not affected significantly.
• These communications are limited to sending the task resource
requirements and parameters: (1) Number of VMs, (2) Number of
vCPUs per VM, (3) Memory per VM, (4) Storage per VM and (5)
Network Bandwidth.
Parallelization • The most computationally intensive task is the search for adequate
resources for a task and the update of the state of all the
components inside a Cell.
• However, these actions can be performed in parallel using the
multiple cores in each compute node. This, fine-grain
parallelization substantially accelerates simulation locally.
• This inherently parallel design scales both in terms of number of
Cells (horizontally) and number of resources in the Cell (vertically).
• Moreover, this design enables the use of dynamic components
(dynamic Brokers) which change their logical architecture based
on characteristics of the underlying resources.
ParallelizationAlgorithm 2 Parallel driver for the hyper-scale Cloud Simulator
1: Initialize local data-centers, network, storage in each Cell
2: for t = t0 = 0 to tmax with step tstep do
3: if Head Node then
4: Create task queue in Gateway Service at time t
5: Send tasks to Broker
6: else
7: Receive tasks from Broker and find adequate resources in parallel
8: Assign tasks to the resources
9: Perform update in parallel on the affected components
10: end if
11: Barrier synchronization of distributed threads
12: end for
Power
consumption
models
The power consumption models used for servers in Cloud simulators
are of two kinds:
• Global models based on minimum and maximum power
consumption. For example:
𝑃 𝑢 = 𝑃𝑖𝑑𝑙𝑒 + (𝑃𝑚𝑎𝑥 − 𝑃𝑖𝑑𝑙𝑒)u, u ∈ [0, 1]
where u is the utilization of the server.
• Piecewise linear interpolation from data obtained from
organizations such as spec.org. This data is the measured power
consumption under certain utilization. For example:
𝑃 𝑢 = 𝑃(𝑢𝑖)+(𝑃(𝑢𝑖+1) −𝑃(𝑢𝑖))(𝑢−𝑢𝑖), 𝑢𝑖 ≤ 𝑢 ≤ 𝑢𝑖+1, 𝑢 ∈ [0,10)
The two examples are the usual practices for computing the power
consumption in present simulators.
For piecewise interpolation models, "not-a-knot" piecewise cubic
interpolation can be used.
Support for
accelerators
• Available simulators do not support accelerators such GPUs, MICs
and FPGAs.
• The execution model of the these devices is similar to that of the
CPUs, however, accelerators cannot be shared by multiple users
in the Cloud.
• Thus, if a user acquires an accelerator its computational power is
totally utilized by that instance.
• The power consumption of these devices is either minimum (idle
state) or maximum. Thus, the power consumption of a server with
accelerators is:
𝑃 𝑢 = 𝑃𝑐𝑝𝑢 𝑢 +
𝑖=1
𝑛𝑎𝑐𝑐
𝜌𝑖𝑃𝑚𝑎𝑥𝑎𝑐𝑐 +
𝑖=1
𝑛𝑎𝑐𝑐
(1 − 𝜌𝑖)𝑃𝑖𝑑𝑙𝑒𝑎𝑐𝑐
where 𝜌𝑖 ∈ [0,1] is the average utilization of the i-th accelerator
and 𝑛𝑎𝑐𝑐 the number of accelerators.
Execution
models
There are three basic types of scheduling execution for the VMs residing on a
server:
• Modern execution models are
primarily based on Space-
Time.
• Gang scheduling is applied by
modern operating systems. In
Gang scheduling threads
belonging to the same
application are scheduled
together.
• Bag of Gangs scheduling is
used when simultaneous
application with multiple
threads are scheduled
together.
Space shared:
Time shared:
Space-Time shared:
Design and
Extensibility
• The simulator based on the presented analysis has been designed
and implemented in C++.
• The Message Passing Interface (MPI), for distributed memory
parallel systems, as well as the Open Multi-Processing (OpenMP),
for shared memory parallel systems, libraries and extensions are
supported in C++.
• The C++ STL includes all the required libraries to build required
lists, queues and maps.
• C++ is also a compiled language and offers fine grain control in
memory and threads.
Design and
Extensibility
Design and
Extensibility
• The selected decomposed approach enables for easy extension of
the simulator.
• The extension procedure requires only to insert methods to the
appropriate class. In example, a new power consumption model
can be inserted in the Power Consumption component.
• Adding models can be performed with minimal interaction with the
source code.
• The addition of a new component, in example, a second statistics
engine, requires designing the new class, updating the Cell class
to include it and add the update procedure in the Update and
Statistics Engine.
• Finally, the MPI is responsible for scaling across Compute Nodes,
while OpenMP is responsible for scaling update and search
procedures across the available Cores of a compute node.
Conclusions • A new hybrid parallel framework for hyper-scale Cloud simulations
has been presented that takes advantage of HPC clusters.
• The new framework is extensible in terms of new models and
components with small software additions.
• Improved power consumption models have been considered that
are more natural to the actual power consumption of modern
CPUs.
• Execution and power consumption models for accelerators have
been given.
References • L.A. Barroso and U. Holzle, The Datacenter as a Computer: An
Introduction to the Design of Warehouse-Scale Machines, Morgan
& Claypool Publishers, 2009.
• Cisco, Cisco Global Cloud Index: Forecast and Methodology,
2015-2020, 2016.
• CloudLightning, http://www.cloudlightning.eu, 2016.
