EUCALYPTUS: An Open Source Infrastructure for Elastic Computing Research

  • Published on

  • View

  • Download

Embed Size (px)


EUCALYPTUS: An Open Source Infrastructure for Elastic Computing Research. Rich Wolski Chris Grzegorczyk, Dan Nurmi, Graziano Obertelli, Shriram Rajagopalan, Sunil Soman, Lamia Youseff, Dmitrii Zagorodnov The MAYHEM Lab Computer Science Department University of California, Santa Barbara. - PowerPoint PPT Presentation


<ul><li><p>EUCALYPTUS:An Open Source Infrastructure forElastic Computing ResearchRich WolskiChris Grzegorczyk, Dan Nurmi, Graziano Obertelli, Shriram Rajagopalan, Sunil Soman, Lamia Youseff, Dmitrii ZagorodnovThe MAYHEM LabComputer Science DepartmentUniversity of California, Santa Barbara </p></li><li><p>ElastaCloudility Computing</p></li><li><p>The Basic Model: Computing as a ServiceSLAsWeb ServicesVirtualization</p></li><li><p>Elasticity, Cloudiness, and UtilityElastic Computing, Cloud Computing, and Utility Computing are, at some level, synonymsSLA-driven interfaceFee-for use and pay-as-you-goAllocations can vary dynamicallyTypical scenarioUser registers a credit card and gets credentialsProvider hosts Linux images and/or application services User instantiates one or more images and pays an occupancy fee for eachLocal system administration not provided by hosting serviceSLAs and tools differentiate between systemsCoordinated allocationsLeasesInstalled software packagesManagement tools</p></li><li><p>Commercial OfferingsAmazon EC2Linux image hosting3 flavors of SLA (small, medium, large)S3 storage facility3TeraOffers a variety hosting services and SLAsappLogic =&gt; visual configuration tool for SaaSIBM Blue CloudBased on Tivoli blade management utility computing platformBasis for current Google/IBM collaborationSun Microsystems network.comGrid-based solution with many pre-packaged applicationsAbout 10x as expensive as EC2</p></li><li><p>Open Source Clouds?Nimbus (Freeman and Keahey, University of Chicago)Client-side cloud-computing interface to Globus-enabled TeraPort cluster at U of CBased on GT4 and the Globus Virtual Workspace ServiceLots of cool featuresGreat if local resources are GT4 proficientTutorial here in 4:00 PM sessionEnomalismStart-up company planning to distribute open source REST APIsUser dashboardDownloads appear to be disabled</p></li><li><p>Elastic Utility Computing Architecture Linking Your Programs To Useful SystemsWeb services based implementation of elastic/utility/cloud computing infrastructureLinux image hosting ala Amazon and IBM/GoogleInterface compatible with EC2Works with command-line tools from Amazon w/o modificationEucalyptus 1.0 == EC2@HomeFunctions as a software overlayOne-button installVersion. 1.0 comes as an RPM and Rocks roll for easy cluster deploymentSystem Administrators are people too.</p></li><li><p>Goals for EucalyptusFoster research in elastic/cloud/utility computing models of service provisioning, scheduling, SLA formulation, hypervisor portability and feature enhancement, etc.Experimentation vehicle prior to buying commercial servicesTech Preview using local machines with local system administration supportProvide a debugging and development platform for EC2 (and other clouds)Allow the environment to be set up and tested before it is instantiated in a for-fee environmentProvide a basic software development platform for the open source communityE.g. the Linux Experience</p></li><li><p>Challenges</p><p>ExtensibilitySimple architecture and open internal APIsClient-side interfaceAmazons EC2 interface and functionality (familiar and testable)NetworkingVirtual private network per cloudMust function as an overlay =&gt; cannot supplant local networkingSecurityMust be compatible with local security policiesPackaging, installation, maintenancesystem administration staff is an important constituency for uptake</p></li><li><p>Eucalyptus Architecture: WS-CloudClient-side APITranslatorCloud ControllerCluster ControllerNode ControllerAmazon EC2 Interface</p></li><li><p>EC2 CompatibilityVersion 1.0 Interface is based on Amazons published WSDL2008 compliant except for static IP address assignmentSecurity groupsAvailability zones correspond to individual clustersUses the EC2 command-line tools downloaded from AmazonREST interface S3 support/emulation: not yet, but on its wayImages accessed by file system name instead of S3 handle for the momentUnless user wants to use the actual S3 and pay for the egress chargesSystem administration is differentEucalyptus defines its own Cloud Admin. tool set for user accounting and cloud management</p></li><li><p>NetworkingEucalyptus does not assume that all worker nodes will have publically routable IP addressesEach cloud allocation will have one or more public IP addressesAll cloud images have access to a private network interfaceTwo types of networks internal to a cloud allocationVirtual private networkUses VDE interfaced to Xen and VLANs set up dynamicallySubstantial performance hit within a clusterAllows a cloud allocation to span clustersHigh-performance private network (availability zone)Bypasses VDE and uses local cluster network for each allocationRuns at native network speed (I.e. with Xen)Cloud allocations cannot span clustersAvailability zone approach fits with Amazons high-level semantics</p></li><li><p>Network Performance Comparison</p></li><li><p>SecurityAll Eucalyptus components use WS-security for authenticationEncryption of inter-component communication is not enabled by defaultConfiguration optionSsh key generation and installation ala EC2 is implementedCloud controller generates the public/private key pairs and installs them User sign-up is web basedUser specifies a password and submits sign-up requestCert is generated but withheld until admin. approves requestUser gains access to cert. through password-protected web pageSimilar to EC2 model without the credit cards </p></li><li><p>Packaging, Installation, and DeploymentRocks Roll per clusterOne-button installRequires Rock V (the most curret release) for Xen supportMultiple clusters requires a configuration file edit at Version 1.0Multi-cluster configuration tools ala Rocks not readily availableRequires Xen version 3.1 to be installed and functioningDoes not require modification to dom0Does require Xen-bridge (not an IP tables approach yet)All needed packages are bundled in the rollRev. 1.0 is not smart enough to determine if local versions of the dependencies will work or notFull version (minus images) is 55 MB</p></li><li><p>Eucalyptus: The MovieThe Movie</p></li><li><p>Software TechnologiesAxis2 and Axis2c version 1.4.0Hibernate 3.2.2HSQLDB 1.8.0jetty 6.1.9JiBX (March 30th sourceforge)Mule 2.0.1Rampart version 1.3libvirt version 0.4.2socat-1.6.0VDE version 2.2.0-pre2</p></li><li><p>Release StatusEucalyptus version 1.0 will be available for public release 5/28/08http://eucalyptus.cs.ucsb.eduEC2 interfaceSimple load-balancing cloud controllerSimple web-based user accounting and system administration toolsetIn testing nowMinor releases for version 1.0 will fix bugs and add limited IP tables supportStatic IPs and security groupsVersion 2.0 (planned)smart private networkingS3 emulation/supportSQS and SimpleDB</p></li><li><p>FutureTest deployment at SDSC using On-demand ClusterShake down by June 1stFriendly user community after thatSLA researchWe built Eucaltyptus so that we could study how SLAs could be automatically formulatedLeverage TeraGrid QBETS technology to determine whether an SLA can be signed or notVGrADS at SC08Virtual Grid Application Development Software project (NSF Large ITR)Single unifying programming abstraction for large-scale workflowsPlanning to demo TeraGrid, EC2, and multiple Eucalyptus Clouds using Linked Environments for Atmospheric Discovery (LEAD) workflowsDevelopment underway now</p></li><li><p>Thanks and More InformationNational Science The Eucalyptus Development Team at UCSB isChris Grzegorczyk -- grze@cs.ucsb.eduDan Nurmi -- nurmi@cs.ucsb.eduGraziano Obertelli -- graziano@cs.ucsb.eduShriram Rajagopalan -- shriram@cs.ucsb.eduSunil Soman -- sunils@cs.ucsb.eduLamia Youseff -- lyouseff@cs.ucsb.eduDmitrii Zagordnov -- dmitrii@cs.ucsb.edurich@cs.ucsb.edu</p><p>What is the problem?-- there is a great deal of interest in cloud computing and no infrastructure for studying it</p><p>Why should you care?-- because investing in an untried technology without prior study can be very expensive</p><p>Defining terms: What is cloud computing?-- At a high level, it is that the user gets compute service rather than direct access to computers</p><p>Definition continued: the trade press tries to claim the term-- need to be clear on what we mean when we say cloud computing for the purpose of this presentationWhat are the current approaches?-- we didnt invent this idea. What technologies are out there?What is the problem with the alternative approaches?-- they are all either proprietary, very complex, or non-exitantOverview of the solution-- summary of how it meets the requirements</p><p>In solving this problem, what do we hope to achieve?-- what good things will happen if we succeed?</p><p>What challenges do we need to meet to solve the problem?-- enumerate them</p><p>Challenge 1: extensibility-- solved by using a simple, straight forward architecture and open APIs that use web services</p><p>Challenge 2: client interface-- solved by implementing Amazons EC2 interface-- familiar to users and already commercially successful</p><p>Challenge 3: networking-- solved using VDE but there is a performance problem-- performance problem can be addressed by leveraging Amazons mechanism =&gt; they have the same problem</p><p>Challenge 4: security-- solved using industry-standard web technologiesChallenge 5: installation and maintenance-- solved through the use of emerging tools that make standard software easier to install and maintain</p><p>How we did it</p><p>What is the state of the system now?</p><p>What is there left to do?</p></li></ul>


View more >