LARubyConf PoolParty presentation

Embed Size (px)

Citation preview

  • 8/14/2019 LARubyConf PoolParty presentation

    1/77

    Ari LernerMichael

    Fairchild

    Interactive R&D

  • 8/14/2019 LARubyConf PoolParty presentation

    2/77

    I have a great idea!

    This site,s lets you post messages to your friends,

    and they can follow you...

  • 8/14/2019 LARubyConf PoolParty presentation

    3/77

    write an application...

  • 8/14/2019 LARubyConf PoolParty presentation

    4/77

    write an application...instantiate servers...

  • 8/14/2019 LARubyConf PoolParty presentation

    5/77

    write an application...

    instantiate servers...configure servers...

  • 8/14/2019 LARubyConf PoolParty presentation

    6/77

    write an application...instantiate servers...configure servers...

    scale them up...

  • 8/14/2019 LARubyConf PoolParty presentation

    7/77

    write an application...instantiate servers...

    configure servers...scale them up...

    reconfigure...

  • 8/14/2019 LARubyConf PoolParty presentation

    8/77

    write an application...instantiate servers...

    configure servers...scale them up...

    reconfigure..scale down...

  • 8/14/2019 LARubyConf PoolParty presentation

    9/77

    and pay for it ?!?

  • 8/14/2019 LARubyConf PoolParty presentation

    10/77

    Wouldnt it be nice if you could

    just write one fileto do all that?

  • 8/14/2019 LARubyConf PoolParty presentation

    11/77

    clouds.rb

  • 8/14/2019 LARubyConf PoolParty presentation

    12/77

    cloud :twitterclonedo

    end

  • 8/14/2019 LARubyConf PoolParty presentation

    13/77

    cloud :twitterclonedoinstances 1..40

    end

  • 8/14/2019 LARubyConf PoolParty presentation

    14/77

    cloud :twitterclonedo

    instances 1..40using :ec2

    end

  • 8/14/2019 LARubyConf PoolParty presentation

    15/77

    Its just Ruby

  • 8/14/2019 LARubyConf PoolParty presentation

    16/77

    cloud :twitterclonedoinstances 1..40using :ec2has_file "/etc/motd"do

    content "hello you"

    endend

  • 8/14/2019 LARubyConf PoolParty presentation

    17/77

    Lets go already!

  • 8/14/2019 LARubyConf PoolParty presentation

    18/77

    write an application...instantiate servers...

    configure servers...scale them up...

    reconfigure..scale down...

  • 8/14/2019 LARubyConf PoolParty presentation

    19/77

    Ew

  • 8/14/2019 LARubyConf PoolParty presentation

    20/77

    cloud-start

  • 8/14/2019 LARubyConf PoolParty presentation

    21/77

    cloud-start -s examples/fairchild.rb

    *** Starting cloud pp1

    0 running instances (1 - 3)

    Adding default gem depdendencies

    ...

    75.101.181.64 stderr => [Sat, 04 Apr 2009 16:46:01 +0000] INFO: Ranexecute[line_in_/etc/profile] successfully

    [Sat, 04 Apr 2009 16:46:01 +0000] INFO: Chef Run complete in 29.883499

    seconds

    Your cloud has started. Your ip is 75.101.181.64Your cloud has started. Your ip is 75.101.181.64 Your cloud has started. Your ip is 75.101.181.64 Your cloud has started. Your ip is 75.101.181.64

    Your cloud has started. Your ip is 75.101.181.64 Your cloud has started. Your ip is 75.101.181.64 Your cloud has started. Your ip is 75.101.181.64 Your cloud has started. Your ip is 75.101.181.64

  • 8/14/2019 LARubyConf PoolParty presentation

    22/77

    launching...

  • 8/14/2019 LARubyConf PoolParty presentation

    23/77

    bootstrapping...

  • 8/14/2019 LARubyConf PoolParty presentation

    24/77

    configuring...

  • 8/14/2019 LARubyConf PoolParty presentation

    25/77

    cloud-list

  • 8/14/2019 LARubyConf PoolParty presentation

    26/77

    $ cloud-list -s examples/fairchild.rb*** Listing cloud pp1****** Active instancesmaster 75.101.181.64 i-516f0f38

  • 8/14/2019 LARubyConf PoolParty presentation

    27/77

    cloud-ssh

    cloud-ssh -s examples/fairchild.rb

    master 75.101.181.64 i-516f0f38Welcome to your PoolParty instanceroot@ip-10-251-201-157:~#

  • 8/14/2019 LARubyConf PoolParty presentation

    28/77

    poolparty coreresources

    crondirectoryexecfilehostmountpackageremote_filesymlinkuservariable

  • 8/14/2019 LARubyConf PoolParty presentation

    29/77

    Puppet

  • 8/14/2019 LARubyConf PoolParty presentation

    30/77

  • 8/14/2019 LARubyConf PoolParty presentation

    31/77

    include_recipe "apache2"include_recipe "passenger"

    include_recipe "rails"

    web_app "paparazzi"dodocroot "/var/www/paparazzi/public"

    template "paparazzi.conf.erb"server_name "www.paparazzi.com"server_aliases [node[:hostname],

    node[:fqdn],"paparazzi.com"]

    rails_env "production"end

  • 8/14/2019 LARubyConf PoolParty presentation

    32/77

    chefdoinclude_recipies "~/recipies/*"has_git_repos :name => "/srv/paparazzi"do

    source "git://github.com/auser/paparazzi.git"

    at "/srv"end

  • 8/14/2019 LARubyConf PoolParty presentation

    33/77

    cloud :twitterclonedoinstances 1..1000using :ec2chefdoinclude_recipes "~/recipies/*"has_git_repos :name => "/var/www/paparazzi"dosource "git://github.com/auser/paparazzi.git"at "/var/www"

    endend

  • 8/14/2019 LARubyConf PoolParty presentation

    34/77

    cloud-configure

  • 8/14/2019 LARubyConf PoolParty presentation

    35/77

    Pushesconfiguration

    changes to thecloud

  • 8/14/2019 LARubyConf PoolParty presentation

    36/77

    building file list ... donedr_configure/clouds.jsond

  • 8/14/2019 LARubyConf PoolParty presentation

    37/77

    Idempotent

  • 8/14/2019 LARubyConf PoolParty presentation

    38/77

    unchanged whenmultiplied by itself

  • 8/14/2019 LARubyConf PoolParty presentation

    39/77

    Run it more thanonce and it wont

    change thingsunless it should

  • 8/14/2019 LARubyConf PoolParty presentation

    40/77

  • 8/14/2019 LARubyConf PoolParty presentation

    41/77

    cloud-configure

  • 8/14/2019 LARubyConf PoolParty presentation

    42/77

    cloud-configure

  • 8/14/2019 LARubyConf PoolParty presentation

    43/77

    Recipes already

    available?

  • 8/14/2019 LARubyConf PoolParty presentation

    44/77

  • 8/14/2019 LARubyConf PoolParty presentation

    45/77

    OMG!Oprah mentioned

    twitterclone!

  • 8/14/2019 LARubyConf PoolParty presentation

    46/77

  • 8/14/2019 LARubyConf PoolParty presentation

    47/77

    cloud :twitterclonedoinstances 1..40expand_when 'load > 0.8'contract_when 'load < 0.3'

    end

  • 8/14/2019 LARubyConf PoolParty presentation

    48/77

    collective metricsveraged across the clou

  • 8/14/2019 LARubyConf PoolParty presentation

    49/77

  • 8/14/2019 LARubyConf PoolParty presentation

    50/77

    y database server is overloadebut my web server isnt

  • 8/14/2019 LARubyConf PoolParty presentation

    51/77

    multiple clouds

  • 8/14/2019 LARubyConf PoolParty presentation

    52/77

    pool :twitterclonedoinstances 1..4000

    cloud :dbdoexpand_when "memory > 0.9"

    end

    cloud :webserverdomaximum_instances 10expand_when "load > 1.2"

    end

    end

    l d i l d

  • 8/14/2019 LARubyConf PoolParty presentation

    53/77

    :instances => 2..5:using => :ec2:ami => 'alestic ubuntu 8.1'

    :contract_when => "load < 0.65":expand_when => "load > 1.9"

    cloud :twitterclonedoend

    Defaults

  • 8/14/2019 LARubyConf PoolParty presentation

    54/77

  • 8/14/2019 LARubyConf PoolParty presentation

    55/77

    Decentralized

  • 8/14/2019 LARubyConf PoolParty presentation

    56/77

    PoolParty does not make

    application assumptions

  • 8/14/2019 LARubyConf PoolParty presentation

    57/77

    Only cloudarchitecture

  • 8/14/2019 LARubyConf PoolParty presentation

    58/77

  • 8/14/2019 LARubyConf PoolParty presentation

    59/77

    Whatever

  • 8/14/2019 LARubyConf PoolParty presentation

    60/77

    web applicationsdata mining

    transcoding cloudemail blasting cloudparticle beam simulation cloud

    (getting girls) social networks

  • 8/14/2019 LARubyConf PoolParty presentation

    61/77

    the sky is the limit

    f

  • 8/14/2019 LARubyConf PoolParty presentation

    62/77

    future

  • 8/14/2019 LARubyConf PoolParty presentation

    63/77

    more remoterbases

  • 8/14/2019 LARubyConf PoolParty presentation

    64/77

    using :ec2

    using :vmwareusing :gogridusing :slicehost

    using :vboxusing :ssh

    :ssh:ssh

  • 8/14/2019 LARubyConf PoolParty presentation

    65/77

    Add Your Own!

  • 8/14/2019 LARubyConf PoolParty presentation

    66/77

    launch_new_instance!terminate_instance(id)

    describe_instance(id)describe_instances

  • 8/14/2019 LARubyConf PoolParty presentation

    67/77

    scripteddeployment

    testing

  • 8/14/2019 LARubyConf PoolParty presentation

    68/77

    aska

    parentingbutterfly

    commander (soon)

    commander (soon)commander (soon)

    commander (soon) Butterfly

  • 8/14/2019 LARubyConf PoolParty presentation

    69/77

    Butterfly

  • 8/14/2019 LARubyConf PoolParty presentation

    70/77

    HTTP for internodecommunication

    of DNS

  • 8/14/2019 LARubyConf PoolParty presentation

    71/77

    of DNS

    round-robin load

    balancinground-robin load

    balancinground-robin load

    nom na e ac ons

  • 8/14/2019 LARubyConf PoolParty presentation

    72/77

    nom na e ac onsbased

    on local state

  • 8/14/2019 LARubyConf PoolParty presentation

    73/77

    Alternatives

    console

  • 8/14/2019 LARubyConf PoolParty presentation

    74/77

    cloud-terminate

    http://poolpartyrb.com/
  • 8/14/2019 LARubyConf PoolParty presentation

    75/77

    http://poolpartyrb.c

    #poolpartyrb(@freenode)(@freenode)(@freenode)

    (@freenode)[email protected]@atti.com

    http://poolpartyrb.com/http://poolpartyrb.com/http://poolpartyrb.com/http://poolpartyrb.com/mailto:[email protected]://poolpartyrb.com/mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]://poolpartyrb.com/http://poolpartyrb.com/http://poolpartyrb.com/http://poolpartyrb.com/http://poolpartyrb.com/http://poolpartyrb.com/
  • 8/14/2019 LARubyConf PoolParty presentation

    76/77

    Questions?

  • 8/14/2019 LARubyConf PoolParty presentation

    77/77

    Thank you

    Interactive R&D