RICON 2014 - Build a Cloud Day - Crash Course Open Source Cloud Computing

  • Published on

  • View

  • Download

Embed Size (px)


This crash course is designed to give an overview of cloud computing architecture and the open source software that can be used to deploy and manage a cloud computing environment. Topics to be discussed in this session will include virtualization (KVM, LXC, and Xen Project), orchestration (Apache CloudStack, Eucalyptus, Open Nebula, and OpenStack), and storage (GlusterFS, Ceph, and others). The talk will also provide insight into how to deliver Platform-as-a-Service (PaaS) and what technologies can be used to compliment this evolving cloud computing paradigm. Systems administrators and IT generalists will leave the discussion with a general overview of the options at their disposal to effectively build and manage their own cloud computing environments using free and open source software and understand the capabilities and benefits of a host of technologies.


  • 1. RICON 2014: Build a Cloud DayCrash Course inOpen Source Cloud ComputingMark HinkleSenior Director, Open Source SolutionsCitrix Inc.mark.hinkle@citrix.commrhinkle@gmail.com@mrhinkle

2. By Mark R. Hinkle@mrhinklemrhinkle@gmail.comABOUT MEI Help Build Open Source EcosystemsOpen Source Experience Manage Citrix Open Source Business Office Apache CloudStack Committer and PMC Member Advisory boards Gluster and Xen Project Joined Citrix via Cloud.com acquisition July 2011 Zenoss Core open source project to 100,000 users,1.5 million downloads Former LinuxWorld Magazine Editor-in-Chief Open Management Consortium organizer Author - Windows to Linux Business DesktopMigration Thomson NetDirector Project - Open Source ConfigurationManagementRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 3. http://www.slideshare.net/socializedsoftwareAttribution You must give appropriate credit, provide a link to the license, and indicate if changeswere made. You may do so in any reasonable manner, but not in any way that suggests the licensorendorses you or your use.ShareAlike If you remix, transform, or build upon the material, you must distribute your contributionsunder the same license as the original.By Mark R. Hinkle@mrhinklemrhinkle@gmail.comSlides Available on Slideshare:Creative Commons Attributions-ShareAlike 4.0 InternationalShare copy and redistribute the material in any medium or formatAdapt remix, transform, and build upon the materialfor any purpose, even commercially.The licensor cannot revoke these freedoms as long as you follow the license terms.RICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 4. By Mark R. Hinkle@mrhinklemrhinkle@gmail.comAgenda The State of Cloud Computing Vetting Open Source Cloud Projects Virtualization Infrastructure-as-a-Service Platform-as-a-Service SDN Open Source for Amazon Web ServicesRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 5. By Mark R. Hinkle@mrhinklemrhinkle@gmail.comThe State of CloudComputingRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 6. THE PUBLIC CLOUD PLAYERS$356 Billion Market CapBy Mark R. Hinkle@mrhinklemrhinkle@gmail.com$141 Billion Market Cap$363 Billion Market CapRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 7. Revenue (in Billions)0 0.5 1 1.5 2 2.5 3 3.5 4Source: Company data, Evercore Group LLC, Research. Azure based on MSFT comments about a $1 billion rev run rate in May 2013.Google based on estimate by TBR (Technology Business Research)By Mark R. Hinkle@mrhinklemrhinkle@gmail.comPublic Cloud RevenueRackspaceGoogleAzureAmazonRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 8. IS IT PRACTICAL TO TRY TODUPLICATE AMAZON,GOOGLE AND MICROSOFT?By Mark R. Hinkle@mrhinklemrhinkle@gmail.comRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 9. Managed Cloud SP/SI CloudBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comPublic Cloud Global Footprint Massive Scale Extreme VelocityRICON 2014 - Buiid A Cloud Day - Open Source Cloud ComputingzCloud Industry Shake OutzzVendorsAdvantagesChallenges Stability Security Privacy End to End Network Security & SLA App QoS SI Capabilities Enterprise Trust SMB Channel Higher price thanPublic Cloud Limited servicescapabilities Agility Stack lock-in, notbest of breed 10. Enterprise Solution ProvidersMove to Converged InfrastructureBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comIDC estimates that totalworldwide spending onconverged infrastructurewill hit $17.8 billion in2016. Will open sourcehardware make thosedollars go farther?RICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 11. By Mark R. Hinkle@mrhinklemrhinkle@gmail.comThe Death of Net Neutrality Pay for play overinternet started (e.g.Netflix peeringagreement) What if you had to payto gain access to yourcloud provider? Would this change thedynamics of publiccloud computing?RICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 12. Open Source Hardware the NextBig Thing Frank Frankovsky, Chairman and President,Open Compute Project Foundation Jason Taylor, Director of Infrastructure, Bill Laing, Corporate Vice President of Cloud Jason Waxman, General Manager, High DensityComputing, Data Center Group, Intel Mark Roenigk, Chief Operating Officer, Andy Bechtolsheim, Arista Networks Don Duet, Managing Director, Goldman SachsBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comFacebookand Enterprise, MicrosoftRackspace HostingRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 13. By Mark R. Hinkle@mrhinklemrhinkle@gmail.comVettingOpen Source ProjectsRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 14. Indicators of Open Source Health Code Velocity Committers Committer Reputation User-driven or Vendor-DrivenInnovation User Activity Corporate Support* Reputation of Foundation*By Mark R. Hinkle@mrhinklemrhinkle@gmail.comHow can you tell if a Project is LegitRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 15. Open Source isnt a Zero-SumGamethe future of technological innovation is not stealing limitedresources away from one another, but creating new resources and new opportunities to create new resources together ina rich ecosystem.By Mark R. Hinkle@mrhinklemrhinkle@gmail.comAllison RandalOpen Source HackerFormer OSCON Program Chair@allisonrandalRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 16. http://www.openhub.net http://activity.openstack.orgBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comOpen Source AnalysisVisualizing Community ActivityRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 17. DevOpsToolchainBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comPlatform-as-a-Service (PaaS)? ?Infrastructure-as-a-Service (IaaS)Orchestration?Compute Storage Networking(Networking-as-a-Service)RICON 2014 - Buiid A Cloud Day - Open Source Cloud ComputingOrchestrationConfigurationManagementMonitoringOpen Source Cloud Stack 18. By Mark R. Hinkle@mrhinklemrhinkle@gmail.comVirtualizationCarving up compute resourcesOPEN SOURCE Xen Project Citrix XenServer KVM VirtualBox OpenVZ LXC libcontainerPROPRIETARY VMware Microsoft Hyper-V OracleVM (Based on Xen Project)RICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 19. By Mark R. Hinkle@mrhinklemrhinkle@gmail.comHypervisors and ContainersDifferences in virtualizationType 1 HypervisorsVMware, Xen Project, Hyper-VType 2 HypervisorsKVM, VirtualBoxRICON 2014 - Buiid A Cloud Day - Open Source Cloud ComputingContainersLXC, libcontainer 20. The Portability Problem Different file formats for virtual machines(VMware uses vmdk file format, Xen andHyper-V use VHD, KVM uses Raw or QCOW2) Guest images may be processor architecture VMware and Xen can manage SCSI devices, but KVM and Xen can use virtio drivers but not VMware uses a proprietary agent inside theguest OS (VMware tools) which does not workwith Xen or KVMBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comContainers compared to Hardware VirtualizationboundKVM cannotVMware Yada, Yada, YadaRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 21. Lets your run a Linux system within A container is a group of processes on aLinux box, put together the provide anisolated environment From the inside, it looks like a VM Externally it looks like normal processes chroot on steroidsBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comContainersLightweight Linux Virtualizationanother Linux systemRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 22. Docker Container PackagingDocker is an open-source project to easilycreate lightweight, portable, self-sufficientcontainers from any application. The samecontainer that a developer builds and testson a laptop can run at scale, in production,on VMs, bare metal, public clouds andmore.By Mark R. Hinkle@mrhinklemrhinkle@gmail.comOpen source Container Packaging EngineTo learn more please visit:www.docker.ioRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 23. Managed daemonized processes on Linux Create ability to re-use and manage similar Content agnostic Hardware agnostic Easy to automate Integrated with other tools: Chef, OpenShift,By Mark R. Hinkle@mrhinklemrhinkle@gmail.comWhat is DockerSystem for Managing and Deploying Linux Containersusing libcontainerapplicationsPuppet, VMware, etc.RICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 24. By Mark R. Hinkle@mrhinklemrhinkle@gmail.comDockersGrowingEcosystemRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 25. Kubernetes builds on top of Docker toconstruct a clustered container schedulingservice. Kubernetes enables users to aska cluster to run a set of containers. Thesystem will automatically pick workernodes to run those containers on, whichwe think of more as "scheduling" than"orchestrationKubernetesTo learn more please visit:https://github.com/GoogleCloudPlatform/kubernetes Greek for ShipmasterBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comContainer Cluster Management SchedulerRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 26. Fig -Fast, isolated development environments Flynn - Next-generation application platform Panamax Drag-and-Drop Docker Containerization Project Atomic JEOS designed to run Dockercontainers SocketPlane Docker Networking (coming soon) Weave Docker Networking 13,000+ Docker-related repos on GithubBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comDocker Related ProjectsRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 27. Continuous Integration Code Application is stored Build Code is built (Jenkins) Test Unit tests areBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comRebuild Applications on any Cloud and/or Virtualized Infrastructurein a repository(Subversion,Git)automated (Jenkins) Deploy Deploy code toserver various waysCodeBuildTestDeployThoughtworks Go Open SourceContinuous Deliver SystemRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 28. By Mark R. Hinkle@mrhinklemrhinkle@gmail.comCloud APIsEverything (should) have an API in the CloudRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing Deltacloud(ruby) Daisein(java) Jclouds(java) Libcloud(python) Fog(ruby) 29. One to many tools for managing large numbers of devicesAnsible Ansible's SSH-key based access allows contributors to the Fedora Project to assist inautomating infrastructure while having access limited appropriately. (Originally authored Func)Capistrano Utility and framework for executing commands in parallel on multiple remote machines, via SSH.It uses a simple DSL that allows you to define tasks, which may be applied to machines incertain rolesRunDeck Rundeck is an open-source process automation and command orchestration tool with a webFunc Func provides a two-way authenticated system for generically executing tasks, integrations withMCollective The Marionette Collective AKA MCollective is a framework to build server orchestration orSalt Execute arbitrary shell commands or choose from dozens of pre-built modules of common (orScalr Provide scaling across multiple cloud computing platforms, integrates with Chef.By Mark R. Hinkle@mrhinklemrhinkle@gmail.comCloud Automation ToolsProject Descriptionconsole.puppet and cobbler.parallel job execution systems.complex) commands.RICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 30. Project Year Started License VirtualizationBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comMinimum Viable CloudInfrastructure-as-a-Service | IaaS | Compute OrchestrationRICON 2014 - Buiid A Cloud Day - Open Source Cloud ComputingTechnologiesApacheCloudStack2008 Apache (Bare Metal), Xenserver,KVM, LXC VMware Hyper-VEucalyptus 2006 GPL Xen, KVM, VMware(commercial version)OpenNebula 2005 Apache Xen, KVM, VMwareOpenStack 2010 (Developed byNASA by Anso Labspreviously)Apache VMware ESX and ESXi, ,Xen, XenServer, KVM,LXC, QEMU and VirtualBox 31. By Mark R. Hinkle@mrhinklemrhinkle@gmail.comOpenStackThe Boy Band of the Open Source Cloud RICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 32. By Mark R. Hinkle@mrhinklemrhinkle@gmail.comSpan Compute, Storage and NetworkingIDENTITYSERVICEIMAGESERVICETELEMETRYSERVICERICON 2014 - Buiid A Cloud Day - Open Source Cloud ComputingORCHESTRATIONSERVICEOpenStack Shared Services 33. Even More OpenStack Projects TroveDatabase Service IronicBare Metal (Ironic) MarconiQueue ServiceBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comSpan Compute, Storage and Networking CinderBlock Storage Service CeilometerMetering/Monitoring HeatOrchestrationRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 34. Open Source Solution ProvidersIf you cant do it yourselfOpenStack is not a product. If you are building a large infrastructure, itsmore like a tool kit. It gives you a lot of technologies that do take a lot ofeffort to integrate.Chris Kemp, OpenStack Board Member and Co-FounderBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comRICON 2014 - Buiid A Cloud Day - Open Source Cloud ComputingCEO of Piston Computing 35. Project DescriptionCeph Distributed file storage system developed by DreamHost ->GlusterFS Scale Out NAS system aggregating storage over Ethernet orRiak CS Riak CS is open source software designed to provide simple,available, distributed cloud storage at any scale. Riak CS is S3-API compatible and supports per-tenant reporting for billing andmetering use cases. (object)Sheepdog Distributed storage for KVM hypervisors, distributed iSCSIBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comCloud StorageVirtualized, Distributed usually on Commodity HardwareInkTank -> Red Hat (block, object, file)Infiniband (file)OpenStackStorageLong-term object storage system (object)RICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 36. What about Open Source PaaS?By Mark R. Hinkle@mrhinklemrhinkle@gmail.comRICON 2014 - Buiid A Cloud Day - Open Source Cloud Computing 37. Project Sponsors Languages/FrameworksBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comPlatform-as-a-Service (PaaS)Abstracted Cloud-Scale Run-Time EnvironmentsCloudFoundry VMware -> Pivotal -> CloudFoundryFoundationRICON 2014 - Buiid A Cloud Day - Open Source Cloud ComputingSpring for Java, Ruby for Rails andSinatra, node.js, Grails, Scala onLift and more via partners (e.g.Python, PHP)Cloudify Gigaspaces [Groovy for deployment recipes]OpenShift Origin Red Hat Java, Ruby, PHP, Perl and PythonApache Stratos WSO2 - >Apache Stratus PHP, Tomcat, MySQL cartridges 38. Apache Mesos is a cluster manager that simplifies thecomplexity of running applications on a shared pool ofservers. Largely supported by Twitter, used byLinkedIn, AirBNB too.Features Fault-tolerant replicated master using ZooKeeper Scalability to 10,000s of nodes Isolation between tasks with Linux Containers Multi-resource scheduling (memory and CPU Java, Python and C++ APIs for developing newBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comApache Mesosaware)parallel applications Web UI for viewing cluster stateTo learn more please visit:http://mesos.apache.org/RICON 2014 - Bu...