Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Cloud Portability, Lifecycle Management and more!
@mrutkowski
Wednesday, 18 May, 2016 @ 11:00 AM EDT
Matt Rutkowski IBM STSM, Cloud Open Technologies OASIS TOSCA Chair, Simple Profile WG
2
▪What is TOSCA? ▪ milestones & participation
▪What Makes TOSCA Unique? ▪ intent model
▪Key Modeling Concepts Topology, Composition, Portability, Lifecycle (management), Policy
▪ TOSCA’s Growing Eco-System ▪ in open source & standards
▪What’s Next ▪ work group activities, version 1.1
An important Open standard, that is enabling a unique Cloud eco-system supported by a large and growing number of international industry leaders…
TOSCA uses a domain-specific language (DSL) to define interoperable descriptions of :
• Cloud applications, services, platforms, infrastructure and data components, along with their relationships, requirements, capabilities, configurations and operational policies…
• …thereby enabling portability and automated management
across cloud providers regardless of underlying platform or infrastructure thus expanding customer choice, improving reliability and time-to-value while reducing costs.
3
Associated Companies • TOSCA Version 1.0 Specification approved as an OASIS Standard
— published Nov 2013, XML format
• TOSCA Simple Profile v1.0 Specification (YAML format)
— final public review, ended March 2016, towards OASIS Standard
— TOSCA Simple Profile v1.1 Specification (target: June 2016)
Supports Domain-Specific Profile Specifications:
– Network Function Virtualization (NFV) Profile v1.0
• Government and Corporate Awareness:
– OASIS: 600+ participant organizations. 5000+ participants spanning 65+ countries
– TOSCA Committee: 170+ people 45+ companies/orgs
– International Standards & Research: ISO/IEC JTC 1 liaison, EU FP7, ETSI NFV liaison, etc.
• Multi-company Interoperability Demonstrated:
– EuroCloud 2013, Open Data Center Alliance 2014, OSCON 2015, OpenStack Summit 2016 (Indigo DataCloud)
4
Includes contributors, reviewers, implementers, users or supporters of the TOSCA Standard via OASIS
incorporates both Data and Information Model features and concepts …
… but brings unique orchestration concepts focus in Lifecycle mgmt. and State
Information Models Typically, used to model a constrained
domain that can be described by a closed set of entity types, properties, relationships and operations.
Data Models Typically, describe the structure (format), enabling manipulation (via interfaces) of the data stored in data management systems assuring integrity.
• Topology • Composition • Requirements - Capabilities • State (Nodes, Relationships) • Lifecycle (Management) • Policy
Intent Model Adds:
TOSCA is an Intent Model which is declarative (integration points for imperative)
• Structure • Format • interfaces
• Types, Relationships • Properties • Operations
TOSCA is can work with imperative scripts (e.g., Ansible, Chef, Bash, Ant, etc.)
TOSCA can include other data models (e.g., JSON, YANG)
• Topology • Composition
• Portability
• Lifecycle
• Policy
Tier (Group Type)
TOSCA is used first and foremost to describe the topology of the deployment view for cloud applications and services
7
source_resource
Node_Type_A
target_resource
Node_Type_B
Requirement
connect_relationship
ConnectsTo Capability
Nodes - are the resources or components that will be materialized or consumed in the deployment topology
Relationships express the dependencies between the nodes (not the traffic flow)
Node templates to describe components in the topology structure
Relationship templates to describe connections, dependencies, deployment ordering
Requirement - Capability Relationships can be customized to match specific source requirements to target capabilities
Groups Create Logical, Management or Policy groups (1 or more nodes)
8
TOSCA Service Template (container) Application Tier
(container)
Web Server (container)
Web App
PHP Module
Database Tier (container)
DB Server (container)
Database
Service Templates provide the “container” to exchange and reuse topologies: • Reusable models extend investments by making it easy to compose more valuable
and complex apps from existing apps • Determines dependency boundaries to maximize parallelism of deployments • Models (dependencies) can be validated by automation to ensure application-aware,
policy-aligned configuration, deployment and operational semantics
Cont
ainm
ent
Connectivity
Example: a simple, 2-Tier Cloud application expressed in a TOSCA Service Template
• Topology
• Composition • Portability
• Lifecycle
• Policy
Application Tier (container)
Application Tier
(container)
10
Logging/Monitoring Tier (ELK)
nodejs
WebServer
app_server
Compute
paypal_pizza store
WebApplication
collectd
logstash SoftwareComponent
Requirements
Container
Capabilities log_endpoint
logstash_server
Compute
Capabilities
Container
elasticsearch SoftwareComponent
Requirements Container
Capabilities search_endpoint
elasticsearch _server
Compute
Capabilities
kibana SoftwareComponent
Requirements
Container
kibana_server
Compute
Capabilities
search_endpoint
ConnectsTo
HostedOn HostedOn HostedOn
ConnectsTo
mongo_dbms
DBMS
mongo_server
Compute
mongo_db
Database
rsyslog
search_endpoint
Container Container
ConnectsTo
Example: Connect a Logging / Monitoring Service composed of ElasticSearch, LogStash and Kibana (ELK)
Enabling the description of complex, multi-tier (hybrid) Cloud applications
Analytics Service
(Topology)
Cloud Application (Topology)
Orchestrators can “substitute” for abstract nodes… … as long as all declared “requirements” are met: • Monitoring Service can be substituted in Cloud Application • Analytics Service can be substituted in Monitoring Service
Abstract nodes in one TOSCA topology can be substituted with another topology
Monitoring Service
(Abstract)
Java Application
Web Application
Server
SQL Datastore
Monitoring Service (Topology)
Collector
Logger
Monitoring Framework
Analytics Service
(Abstract)
Analytics Engine
Hadoop Service Template 1
Service Template 2
Service Template 3
• Topology
• Composition
• Portability • Lifecycle
• Policy
TOSCA Service Template
Storage
Compute1
DB
Compute2
App
Network
Scaling Policy
TOSCA’s defines Normative Types for different domains, for example:
Application, IaaS Types are part of “core” specification
e.g., Web Server, Database, Compute, Block Storage, Network
Cloud Application’s declarative modelled from these normative types …
… Can be understood by any Cloud Provider
unfulfilled Application Requirements
can be exported for Orchestrators to fulfill
Templates include (or reference) all necessary configuration and Infrastructure requirements
TOSCA applications, using normative types, are portable to different Cloud infrastructures
TOSCA Meta-Model Normative Types
Nodes • Properties • Attributes
Relationships • Properties • Attributes
Capabilities
Interfaces (Operations)
Groups
Policies
Requirements
Interfaces
com
pose
d fr
om
base
d up
on
Example: TOSCA applications are portable to different Cloud infrastructures
Application Requirements
TOSCA Orchestration
TOSCA Service Template
Storage
Compute1
DB
Compute2
App
Network
Scaling Policy
Cloud Provider C
Cloud Provider A
Cloud Provider B
by expressing application Requirements…
independently from cloud provider Capabilities…
& Optimization Automatic Matching
Infrastructure Capabilities
Orchestrators concern themselves dealing with disparate cloud APIs 14
• Topology
• Composition
• Portability
• Lifecycle • Policy
TOSCA models have a consistent view of state-based lifecycle have Operations (implementations) that can be sequenced against state of any dependent resources fits into any Management Framework or Access Control System
my_resource_name
My_Resource_Type
Lifecycle.Standard
create
configure
start
stop
delete
Standardize Resource Lifecycle Standardize Relationship Lifecycle Lifecycle Customization
source_resource
Type_A
A
target_resource
Type_B
B
my_relationship
ConnectsTo
Ope
ratio
ns Lifecycle.Configure
pre_config_target
post_config_target
add_target
remove_target
pre_config_source
post_config_source
add_source
remove_source
Operations
Lifecycle.Configure.NFV
Ope
ratio
ns
Lifecycle.Standard
create
configure
start
stop
delete
Create new Lifecycles or Augment existing (via subclassing)
pre_config
pre_delete
16
my_resource_name
My_Resource_Type
Lifecycle.Standard
create
configure
start
stop
delete
Node Lifecycle
Ope
ratio
ns
Implementations (e.g., imperative scripts) can be bound to operations.
source_resource
Type_A
A
target_resource
Type_B
B
my_relationship
ConnectsTo
Lifecycle.Configure
pre_config_target
post_config_target
add_target
remove_target
pre_config_source
post_config_source
add_source
remove_source
Operations
The Orchestrator moves the nodes through their Lifecycle States by executing their Lifecycle Operations in topological order • Orchestrators can work to deploy nodes in parallel based upon node relationships
Relationship Lifecycle Nodes have their own Lifecycle
Operations which are invoked in order to achieve a target state
Relationships also have their own Lifecycle Operations to configure or allocate and de-configure or deallocate Node related resources
• Topology
• Composition
• Portability
• Lifecycle
• Policy
v1.0 includes the groundwork for Placement (Affinity), Scaling and Performance Policies ‒ Orchestrators can evaluate Conditions based on Events that trigger Automatic or Imperative Actions
Policies can be declared independently and ttached to various points in your models 1. That can be attached to Interfaces or specific Operations, 2. Nodes and 3. Groups of Nodes
my_app_1
Compute
Capabilities
Container
... Lifecycle
create
configure
...
Policy • Type • Event, Condition • Action
my_scaling_group
backend_app
Compute
Policy • Type • Event, Condition • Action
my_database
Compute web-app
Compute Policy • Type • Event, Condition • Action
1
2
3
Scaling
“Policies are non-functional Requirements independent of nodes”
TOSCA Policy Definition (e.g., Placement, Scaling, Performance) : <policy_name>: type: <policy_type_name> description: <policy_description> properties: <property_definitions> # allowed targets for policy association targets: [ <list_of_valid_target_resources> ] triggers: <trigger_symbolic_name_1>:
event: <event_type_name>
target_filter: node: <node_template_name> | <node_type> # (optional) reference to a related node # via a requirement requirement: <requirement_name> # (optional) Capability within node to monitor capability: <capability_name> # Describes an attribute-relative test that # causes the trigger’s action to be invoked. condition: <constraint_clause>
action: # implementation-specific operation name <operation_name>: description: <optional description> inputs: <list_of_parameters> implementation: <script> | <service_name> ...
Event • Name of a normative TOSCA Event Type • That describes an event based upon a
Resource “state” change. • Or a change in one or more of the
resources attribute value.
Condition Identifies: • the resource (Node) in the TOSCA
model to monitor. • Optionally, identify a Capability of the
identified node. • Describe the attribute (state) of the
resource to evaluate (condition)
1..N
Trig
gers
can
be
decl
ared
Describes: • An Operation (name) to invoke when
the condition is met • within the declared Implementation • Optionally, pass in Input parameters to
the operation along with any well-defined strategy values.
Action
• Reference by other Standards
• Open Source • OpenStack
22
Topology, Type & LCM Design http://alien4cloud.github.io/
alien4cloud
Service Orchestration & Management http://getcloudify.org/
Data/computing platform targeted at scientific communities
http://information-technology.web.cern.ch/about/projects/eu/indigo-datacloud
https://wiki.openstack.org/
Heat-Translator (IaaS, App Orchestration) Tacker (Network Function Orchestration)
http://ariatosca.org//
Multi-Cloud Orchestration (Amazon, Azure, VMware, OpenStack)
Open Sourced from Cloudify
www.seaclouds-project.eu/media.html Open, Multi-Cloud Management
Parser Deployment Template Translation
https://wiki.opnfv.org/display/parser/Parser Note: ETSI NFV ack. TOSCA can be used as an input model/format
TOSCA-Parser
TOSCA Plugin
TOSCA Integration
apps.openstack.org
OpenStack Client (OSC) Plugin
Senlin Clustering & Policy
(on roadmap)
Tacker NFV Orchestration Parser
Heat-Translator App. Orchestration
https://wiki.openstack.org/
Core
Compute (Nova)
Storage (Cinder, Swift)
Network (Neutron)
Shar
ed
Image (Glance)
Database (Trove)
Identity (Keystone) … Metering
(Ceilometer) Orchestration (Heat / HOT)
(Heat-Translator / TOSCA)
HOT Pattern Domain
TOSCA CLI App. Catalog
Library export
https://pypi.python.org/pypi/heat-translator https://pypi.python.org/pypi/tosca-parser
• Interoperability (Conformance) • Goal: Conformance test suite for v1.0; includes tests for each section of Simple Profile v1.0 specification.
• Each test is a TOSCA Service Templates with metadata describing test using the OASIS Test-Assertion (TAG) Standard • Work underway to publish in new GitHub repo., announcement (target ~May 2016)
• Container (Clustering) • Goal: Finish new Cluster capability definitions, Data Cluster use cases. for Simple Profile v1.1
• Instance Model • Goal: new schema for an Instance Model (reuse existing schema where possible)
• Discussing API potentially enabling capture, export and management of deployed application
• Monitoring • Goal: Create normative event types for basic operational events
• Focus on events types for Health, Scaling & Performance • Support basic “Red-Yellow-Green” and Percentage-based monitoring for dashboards
• Network Function Virtualization (NFV) • Expanded Scope: include Software-Defined Network (SDN) use cases • Goal: Complete v1.0 Specification, v1.0 Public Review Draft 3 Published (17 March 2016)
• Can model complete ETSI MANO specification: Network Services, Virtual Network Functions (NFV)s, Virtual Links, with Forwarding Paths,
• Orchestration demonstrated with OpenStack Tacker Project, multi-VNF use cases for next release
Specification Release Targets • Public Review Draft 01 - target June 2016
• “Final” Public Review Draft - target 3Q 2016
New Features • Metadata (completed)
• now supported in all Types (Node, Relationship, Capability, Data, etc.) • Conformance Testing metadata
• Group Type (completed) • Expanded Group Type to allow management of member resources (i.e., Lifecycle) • Has its own Capabilities and Requirements
• Policy Definition (completed) • Event-Condition-Action model • Includes Event Filters and Triggers
• Workflow (80% completed) • Intermix declarative with Imperative (e.g., Ansible, Chef, Ant, Bash) • Preserve investment in existing scripts for complex installations / configurations
• Cluster Type (75% completed) • Add support for Cluster normative type; based upon new Group Type • Will support new normative LoadBalancer , Scalable and Router Capability Types • Data Clusters (e.g., Cassandra, MongoDB, etc.) – In-Progress
25
26
• TOSCA Technical Committee Public Page (latest documents, updates, and more)
— https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=tosca
• OASIS YouTube Channel, TOSCA Playlist — https://www.youtube.com/user/OASISopen , http://bit.ly/1BQGGHm
• LinkedIn Group: “TOSCA OASIS Standard”: — https://www.linkedin.com/groups/8505536
• TOSCA Simple Profile in YAML v1.0 (final public review draft, 04, Feb. 2016) — http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.0/TOSCA-Simple-Profile-YAML-v1.0.html
• TOSCA Simple Profile for NFV v1.0 (latest public review draft, 17 March 2016)
– http://docs.oasis-open.org/tosca/tosca-nfv/v1.0/tosca-nfv-v1.0.html
• Contact the Technical Committee Co-Chairs: – Paul Lipton, [email protected]; Simon Moser, [email protected]
Q&A
Complete Source: https://github.com/openstack/heat-translator/blob/master/translator/toscalib/tests/data/tosca_single_server.yaml
tosca_definitions_version: tosca_simple_yaml_1_0 description: > Template for deploying a single server with predefined properties and input parameter topology_template: inputs: cpus: type: integer description: Number of CPUs for the server. constraints: - valid_values: [ 1, 2, 4, 8 ] node_templates: my_server: type: tosca.nodes.Compute capabilities: host: properties: num_cpus: { get_input: cpus } disk_size: 10 GB mem_size: 512 MB os: properties: architecture: x86_64 type: linux distribution: rhel outputs: server_address: description: IP address of server instance. value: { get_attribute: [server, private_address] }
Logical Network Model
AppServerTier
AppServer
App Component
DBServerTier
DBServer
APP DB
Required EndPoint
Provided EndPoint
HTTP Client
Application Architecture: Structure, dependencies, requirements, and interconnectivity
AppServerTier
DB Client Port
App Server Port
DBServerTier
DB Server Port
Private Logical
Network
Public Logical
Network
Network Technology Independent Representation
Concrete Network Model
Technology specific rendering of Logical Network
Logical Router Logical Switch 10.0.2.0/24
DB Client
Intf
Logical Router Logical Switch
X.X.X.X/24
App Server
Intf
Logical Switch
10.0.1.0/24 DB
Server Intf
Logical Switch
X.X.X.X/24
Edge Intf
Logical FW
Rules
Logical FW
Rules
HTTP Client
Edge Services Gateway
Service Topology Layered Topologies