Upload
thirdwaveinsights
View
49
Download
1
Tags:
Embed Size (px)
Citation preview
1
Serve Meals, Not Ingredients
John [email protected]@tkdsweitz
Christine [email protected]@CristineDraper
@3rdwaveinsightshttp://www.thirdwaveinsights.com
2
Topology
Meals are topologies, nodes are key ingredients
Nodes
Software on a Node
Connections
MealIngredients
3
Emulating topologies in Chef starts with grouping nodes
Topology JSON
Minimize collisionsTopology-as-codeOptimized for ChefFaster iterations
{ "name": "inttest1", “provisioning” : { }, "services": [ { "name": "appserver-elb", "type": "load_balancer" } ], "nodes": [ { "name": "appserver" }, { "name": "dbserver" } ]}
topo.json
Chef Server
Environments
Data Bags
Cookbooks
Nodes
Chef Server
Nodes Node
Environments
Data Bags
Cookbooks
● Ways to emulate:– Environments – Tags – Node attributes– Data bags
knife-topo
4
Multi-* Enterprise - This may not be you, exactly, but...
● Multiple lines of business running business systems
● Multiple deployment across application pipeline
● Technology Adoption Model
First team First wave Broad adoption Full adoption
Asset Tracker Inventory
CustomerOrders
Human Resources
Application pipeline
Test Stage DeliveryDevelop
5
Jake's charter
More meetingsLess autonomy/controlEndless debates/escalationsTedious data gathering
Reduce costShare skillsShare content
Why increase consistency?
INCREASE CONSISTENCY ACROSS TEAMS WITHOUT DECREASING THEIR AGILITY
6
The WORK (and it is work) to increase consistency...
Gather
Organize
Decide
Change
Monitor
Waste Innovation
7
Organize: Topology-Blueprint-Inventory Pattern
Blu
eprin
tsP
ipel
ine
Topo
logi
es
Test Stage DeliverDevelop
Inve
ntor
y
Optimized Arrangements
Particular Deployment
Automate.Insights
Organizational Flow
Bill of Materials
An IT business system is a collection of IT resources delivering a business function
● Blueprints evolve
● Topologies are specialized
● Visibility encourages collaboration
8
Decide: What is expected to be common in all deployments?
Software Version Default recipe
mongodb 2.6.5 coolco_mongodb::default
nodeapp 1.0 coolco_nodeapp::deploy_from_package
nodejs 0.10.29 coolco_nodejs::default
Recipe Inventory
apt
chef_client::config
coolco_handlers::default
coolco_mongodb::default
coolco_nodeapp::deploy_from_package
coolco_nodejs::default
Software Attribute Value
mongodb xxx yyy
xxx yyy
xxx yyy
Software Attribute Value
nodeapp nodeapp.name asset_tracker
nodeapp.user jake
nodeapp.dir /home/automation
RunlistNode attributes
9
Decide: Look across blueprints...
Automate. Insights
● Node Types● Software and Version● Attribute Values● Recipes● Cookbooks
Software attribute value by blueprint
10
Change – All does not need to change at once
Blu
eprin
tsP
ipel
ine
Topo
logi
es
Test Stage DeliverDevelop
Inve
ntor
y
Optimized Arrangements
Particular Deployment
Organizational Flow
Bill of Materials
Chef Server
Actual IT Resources
Environments
Data Bags
CookbooksN
odes
Roles
● Prime● Synchronize● Local Override
Sta
ged
Ch
ang
es
nodejs 0.10.28 to 0.10.30
11
Gathering and Changing details in Chef Server
knife topo
export YourTopology
node1 node2 node3 …
> YourTopology.jsonTopology
JSONChef Server
knife-topo
Automate.Insights
knife topo
import YourTopology
knife topo
create YourTopology
Create blueprints from nodesAlign topologies with blueprint
Propagate changes to Chef server
Gather
Organize
Decide
Change
Monitor
12
Fred's Scenario
Meanwhile Shadow IT has sprung up in various lines of business, using a range of providers
Fred's one of the technical leads who's been successful bringing consistency across CoolCo's
core business systems...
CoolCo wants Fred to help the Lines of Business adopt Chef and become more consistent in their
delivery, without impacting their agility
13
Fred meets with one of the lines of business to understand what they have...
appserver AMI
appserverauto scaling groupElastic Load
Balancer(appserver-
elb)
dbserver
AWS cloud
Launch Configuration
db url
protocol & port
instanceprotocol & port
Resources": { "elbappserverelb": { "Type": "AWS::ElasticLoadBalancing::LoadBalancer", "Properties": { "AvailabilityZones": [ "us-west-2c", "us-west-2b", "us-west-2a" ], "Listeners": [ { "InstancePort": "3001", "LoadBalancerPort": "3001", "Protocol": "HTTP", "InstanceProtocol": "HTTP" } ]} },...
Create multiple test topologies rapidly & consistentlyDelete topologies when done
CloudFormation Template
CloudFormation Stacks
CloudFormation
AWS Icons: http://aws.amazon.com/architecture/icons/
Hook-Em... a recently-developed customer conversion system
14
… and what their challenges are
app+
nodejs
mongodb
application & middleware
"DBServer": { "Type" : "AWS::EC2::Instance", "Properties" : { "UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [ "#!/bin/bash\n", "yum update -y aws-cfn-bootstrap\n", <snip 20 lines> "yum -y install mongo-10gen-server > /tmp/yum-mongo.log 2>&1\n", <snip 50 lines> "/etc/init.d/mongod start > /tmp/mongod-start.log 2>&1\n", <snip 20 lines> "/opt/aws/bin/cfn-signal -e 0 -r \"Mongo setup complete\" '", { "Ref" : "WaitHandleMongo" }, "'\n" ] ] } }, ... } ... }}}
It's hard to configure & update the softwareOur admin who created our templates has left...
15
A shared goal emerges...
Deployer Interface that makes it easy to customize and update the topology
Topology-as-codeDeployment lifecycle actionsFull stack provisioning
Reuse organization cookbooks & practices
Chef Server
Git
TopologyJSON
chef-provisioning
...Now to make it happen!
ProvisioningRecipes
Convert from existing templates
16
Chef Server
What are the steps?For the Hook'Em business system:
1) Set up Hook'Em System Inventory in Automate.Insights
2) Create a Blueprint based on the existing CloudFormation template
3) Create Topologies and provision them using generated recipes
app +nodejs
mongodbapplication &
middleware
infrastructure
TopologyJSON
chef-provisioning
Automate. Insights
Provisioning recipes
● System inventory● Existing template
17
1 – Setup Hook'Em System Inventory
Software Version Default recipe
mongodb 2.6.5 coolco_mongodb::default
nodeapp 1.0 coolco_nodeapp::deploy_from_package
nodejs 0.10.29 coolco_nodejs::default
Recipe Inventory
apt
chef_client::config
coolco_handlers::default
coolco_mongodb::default
coolco_nodeapp::deploy_from_package
coolco_nodejs::default
Software Attribute Value
mongodb xxx yyy
xxx yyy
xxx yyy
Software Attribute Value
nodeapp nodeapp.name hookem
nodeapp.user automation
nodeapp.dir /home/automation
RunlistNode attributes
Fred primes the System Inventory in Automate.Insights with CoolCo's standard recipes and recommended attribute values for the software in the HookEm System.
System Inventory
Automate. Insights
mongodb
nodeappnodejsBlueprint
18
2 - Create a Blueprint for Hook'Em
Topology JSON Automate.
Insights
The Hook'Em team converts the existing CloudFormation template using topo-provision, and uses it and the System Inventory to create a Blueprint and Topologies in Automate.Insights
topo-provision
CloudFormationtemplate
Software inventory
Full-StackTopology JSON
19
3 - Generate recipes & provision the full stack topology
chef-provisioning can handle network resources too!
app +nodejs
mongodbapplication &
middleware
infrastructure
Full StackTopology JSON
topo-provision
Chef-Provisioning recipes
chef-provisioning
with_driver “aws”
load_balancer ... do ... end
machine … do … end
aws_launch_config … do ... end
aws_autoscaling_group … do ... end
● Deploy● Undeploy
load_balancer "appserver-elb" do load_balancer_options({ :listeners=> [{ :port=>3001, :protocol=>"HTTP", :instance_port=>3001, :instance_protocol=>"HTTP" }], :availability_zones=> ["us-west-2c", "us-west-2a", "us-west-2b"]})end
machine "dbserver" do run_list ["recipe[coolco::db]"] attribute 'topo', {"node_type"=>"dbserver", "name"=>"test"} end
deploy.rb:Supports deployment lifecycle actionsSensitive to resource relationships
Using topo-provision, the Hook'Em team generate topology provisioning recipes that enable chef-provisioning to provision the full stack.
20
Summary of transformation
Infrastructure templates
Stack create and delete
Customization of parameters
Full stack Topology JSONConversion from existing templates
Topology deploy and un-deployIntegrated provisioning and configuration management
Customization of attributes & run listLine-of-sight across topologiesTool assistance for change propagation
TopologyTemplate
Deployment
Deployer Interface
As-Is To-Be
Practices Reuse organization cookbooksBenefit from Chef test tools
CloudFormation Template
CloudFormation Stacks
CloudFormation
Topology JSON
ChefAutomate.Insights
21
A Cloud-neutral topology template has wider value to the Chef community
TopologyJSON
chef-provisioning
ProvisioningRecipesConvert from other
formatsIncrease portability
CloudFormation
Terraform
OASIS TOSCA
Home-grownIntegrate with multiple tools, from Chef and others
Adoption of a Chef-oriented topology template that is not tied to one Cloud Provider helps increase portability and tool integration going forward.
Deploy to multiple providers
This is a vision with proof points for CloudFormation, AWS and Vagrant.
Test Kitchen
kitchen.yml
Automate. Insights
22
Thank you!
Tools:http://www.thirdwaveinsights.com/automateinsightshttps://github.com/christinedraper/topo-provisionhttps://github.com/christinedraper/knife-topo
Find Out MoreSpeakers:[email protected]@[email protected]@CristineDraper