Contrail Infrastructure - Virtual Execution Platform - Inria

Contrail Infrastructure


Piyush Harsh

Myriads Project Team, INRIA

July 27, 2012


Introduction

What is Contrail

Collaborative project partly funded by European Commissionunder FP7 directive (Contract No: FP7-ICT-257438)

Duration 3 years (Oct 2010 - Sept 2013)

Budget: e11.3 M

Integrated Cloud Federation Software Suite

Individual software suites to manage IaaS clouds, PaaS, SLAsFederation services


Introduction

In a nutshell ...


Introduction

General Guiding Principles

Minimize possibility of a vendor lock-in

supporting an open application description format to increaseapplication portabilityproviding standard application interfaces for improvingcomponent level interoperability

Open development process (open source)

source code available at OW2 subversion repositoryJira for bug reporting and tracking


Introduction

Tasks for Infrastructure Modules

Provision physical resources (compute,storage, network) to deploy federationapplications (on behalf of end users).

Allow monitoring of deployedapplications

Enable application lifecyclemanagement

Provide support for SLA and QoP

Properly deploy and configure securitytools and mechanisms


Introduction

Tasks for Infrastructure Modules

Provision physical resources (compute,storage, network) to deploy federationapplications (on behalf of end users).

Allow monitoring of deployedapplications

Enable application lifecyclemanagement

Provide support for SLA and QoP

Properly deploy and configure securitytools and mechanisms


Introduction

Contrail Infrastructure at a Glance

Components

Virtual Execution Platform (VEP)

Virtual Infrastructure Network (VIN)

Infrastructure Monitoring

Services

Contrail Authorization Services (PDP)

Contrail Certification Services (CA)


Virtual Execution Platform




Introduction

What is VEP

VEP is a software suite that facilitates membership of an IaaSprovider in the Contrail Cloud Federation (CCF). It provisions thecompute resources from the IaaS platform and deploys userapplications under a negotiated SLA.

Highlights

Provides key features to enhance module interoperability, andenables application portability to minimize vendor lock-in.



Distributed View



Component View

Component View of VEP



Component View

Component View - Submodules

Figure: REST module Figure: Image Provisioning



Services

Virtual Execution PlatformConfiguring the datacenter

VEP enables a provider to configure the datacenter layout.

VEP maintains the full layout along with clusters, racks, andinterconnect technology used.

System administors can pick and choose their physicalmachines for participations in the federation.

VEP manages the VM scheduling of hosts under its control

currently best-effort round-robin scheduling is supportedadvance resource reservation and SLA based scheduling isunder development



Services

Virtual Execution PlatformManaging Application Lifecycle

VEP allows VM’s lifecycle control (VM start, stop, suspend,resume)

VEP performs VM’s contextualization as needed

VEP helps bootstrap VIN agents for setting up secure VMnetworking across providers

These application lifecycle management operations are performedby the federation modules on behalf of end users. VEP acceptsREST requests only from trusted federation modules.


VIN

Virtual Infrastructure Network


VIN

Virtual Infrastructure NetworksArchitecture

Group A

Cloud A Cloud BPublicInternet TT

Central VINController

Group ACentral Application

Controller


VIN

Virtual Infrastructure Networks

Contrail support application deployment over multiple cloudproviders. VEP enables bootstrapping the secure network servicesprovided by the VIN modules.

VMM

App

OS

OS

VM

TVIN

Agent

Figure: VIN agent in host

VMM

App

T

OS

OS

VM

VINAgent

Figure: VIN agent in VM


VIN

Virtual Infrastructure NetworkTiming Diagram

Centralcontroller VIN Node 1 Node 2 Node 3

Register VM 1

return VM 1 identifier

Requestcontroller address

Register VM1 agent

Start VM 1,start VIN agentwith ID & address

Register VM 2

Register VM2 agent

Start VM 2 & agent

Register VM 3

Register VM3 agent

Start VM 3 & agent

Register Network 1Broadcast NW 1 properties

Add VM1 to NW1Broadcast VM1 in NW1


Make tunnelto VM2

Make tunnelto VM1


Make tunnelto VM3

Make tunnelto VM3

Make tunnelto VM1

Make tunnelto VM2Remove VM2 from NW1

Broadcast VM2 out of NW1

Stop tunnelto VM2

Stop tunnelto VM2

Start VIN session Sessionthread

Agent

Agent

Agent

Starting of agents on thephysical hosts iscontrolled by VEP

Agent bootstrapping andinitial configuration is alsodone by VEP

if agent inside VM -parameters passed aspart of VMcontextualization


Monitoring

Infrastructure Monitoring


Monitoring

Contrail MonitoringKey Points

monitoring infrastructure built using RabbitMQ

publish-subscribe queuing system

designed to withstand high traffic load - finagle along withKrestrel (twitter!) is being used.


Monitoring

Monitoring Architecture

Federation

Provider 1Provider 1

MsgQ

MsgQ 1

MsgQ 2

MsgQ 3Hub

Hub

Hub

Component Request messages


Monitoring

Component View - Monitoring

Cluster

Sensor

StartMonitoring() SlaExtractor QueryAPI

Listener

Alerting Queue

Monitoring Hub

MongoDB

MongoDB

MongoDB

Data Manager

Reporting Manager

Billing Manager

Pricing Manager

VM

Sensor

Host

Sensor

Listener

Listener

Listener

Listener

Listener


Monitoring

Component View - OpenNebula Monitoring


Security

Securing Resources and Services


Security

Securing the provider’s resource

Fed-local Account Mapping

VEP implements a fully randomized mapping to the localresource (user id). This provides a certain level of securityagainst a compromised federation account.

The mapping table is maintained at the provider site and isindependently generated at each site.


Security

Use of Delegated Certificates

OAuth 2.0

VEP will use time and role restricted delegated X.509 certificatesto allow access to local and remote cloud resources such asstorage, secure tunnels, etc. OAuth 2 is being utilized asdelegation protocol.


Security

Security Bootstrap Process for VIN


Interoperability

Efforts towards Portability and Interoperability


Interoperability

Interoperability and VEP

VEP strives to become interoperable with 3rd party cloud tools bysupporting open standards. Using an open standard to describeyour cloud application further makes it portable.


Interoperability

Open Standards

A short overview of Cloud Standards

Standards enable interoperabilityMajor cloud standardization bodies

OGF - Open Grid ForumDMTF - Distributed Management Task ForceSNIA - Storage Network Industry Association

Key Upcoming Cloud Standards

OCCI - Open Cloud Computing Interface (OGF)OVF - Open Virtualization Format (DMTF)CIMI - Cloud Infrastructure Management Interface (DMTF)CDMI - Cloud Data Management Interface (SNIA)

De-Facto Cloud Standards

Amazon EC2 API


Interoperability

Open Virtualization Format


It is an Open standard by Distributed Management Task Force(DMTF), an industry non-profit organization. Includes big-wigssuch as Intel, HP, IBM, Cisco, Vmware, Microsoft, US DoD etc.

With approval of major players in the cloud industry, it is morelikely to succeed.


Interoperability


OVF - a bit of detail

provides a standard way of describing a virtual application

ability to describe a VM hardware specifications

ability to specify the network and storage parameters

individual VM contextualization support

provision for controlling VM start-up order

container description for a self-contained application in asingle .ova package

support for elasticity in the upcoming 2.0 draft


Interoperability


OVF Example

Snippet 1


Interoperability


OVF Example - Snippet 2


Interoperability


OVF Centric View of VEP


Interoperability

Cloud Infrastructure Management Interface

Cloud Infrastructure Management Interface

CIMI is an upcoming DMTF standard defining model and protocolfor management of interactions between IaaS clouds and users ofIaaS services.

CIMI system comprises of network, volumes, and machines

System can be instantiated from templates supplied by cloudproviders and/or users

specification for generating a CIMI system template from OVF

process for generating an OVF from a deployed applicationsnapshot is described

OVF and CIMI standards works seamlessly with each other!


Interoperability

Standards in VEP

What is the current status?

VEP roadmap has development plans for providing full OVFstandards compliance and a CIMI with extensions (for supportingall of Contrail’s requirements) support to enhance interoperability.


Conclusion

Wrapping Up


Conclusion

Feature List

Contrail Infrastructure Features

Ability to deploy cloud applications described in an OVFstandard format.

Real time resource monitoring

Ability to setup networks across multiple providers

Full application lifecycle control through REST

multi-level authorization and access control

Multi-pronged approach to security including ability to secureremote entities using delegated X.509


Conclusion

Feature List

Features (contd.)

API drivers to fully support several upcoming open-sourcecloud technologies (OpenNebula, OpenStack (planned))

open standards support (DMTF’s OVF and CIMIspecifications)

Intelligent resource provisioning guided by QoP constraints

Application admission control module, a cloud-centricresource reservation module (planned)


Conclusion

Additional Information

Need more info?

VEP: http://vep.gforge.inria.fr/Monitoring: http://contrail.xlab.si/VIN and other packages: http://contrail.ow2.org/Contrail: http://contrail-project.eu/

http://vep.gforge.inria.fr/

http://contrail.xlab.si/

http://contrail.ow2.org/

http://contrail-project.eu/

Documents

Contrail Infrastructure - Virtual Execution Platform - Inria