RabbitMQ at Skills Maer Cloud Exchange Cloud Messaging · PDF fileCloud Messaging Use Cases April 2010 ... Pubsub Hubs are Cloud Messaging Technology ... Check out the Google teamʼs

RabbitMQatSkillsMa-erCloudExchange

CloudMessagingUseCases

April2010AlexisRichardsonVMwareInc.

copyright (c) VMware Inc.

Friday, 23 April 2010

RabbitMQisamessagingserverthatjustworks

Thank you!

Im in yr serverz,queueing yr messagezPhoto credit: http://flickr.com/photos/53366513@N00/67046506/



Wees7mate300‐500ormoreinproduc7on,>20Kindev



RabbitMQintegratedwithmorethan100developerplaForms

RabbitdistributedbymostLinuxtypeplaForms

WeofferpackagesformanyotherOSplaForms

RabbitMQ.NETandMicrosoOIVApartnership

UsingRabbitMQiseasy



545

RabbitdistributedbymostLinuxtypeplaForms

WeofferpackagesformanyotherOSplaForms

RabbitMQ.NETandMicrosoOIVApartnership

Cloudmessagingisthefuture

RabbitMQintegratedwithmorethan100developerplaForms

UsingRabbitMQiseasy


Cloudhasre‐energisedIT



CloudmaIersbecauseweoperateatreal7meonaglobalscale

Mobile, video, presence, live streams - anywhere, anytime

How to deliver this without lock-in to any one platform or location?

Massive amounts of data everywhere and changing all the time

Security, privacy, consistency of customer experience eg latency, relevance

Cloud Messaging matters!



http://railsdog.com/blog/2009/12/generating-pdfs-on-ec2-with-ruby/copyright (c) VMware Inc.


http://railsdog.com/blog/2009/12/generating-pdfs-on-ec2-with-ruby/

http://railsdog.com/blog/2009/12/generating-pdfs-on-ec2-with-ruby/

9http://railsdog.com/blog/2009/12/generating-pdfs-on-ec2-with-ruby/


Cloud+QueuesmakesiteasytomanageanARMYofworkerbots



WealsocareaboutSCALE...



WealsocareaboutSCALE...whenwecannotpredictdemand



Cloudapplica7onsarecomponentbasedandneedintra‐appmessagingtoscale

See more here: http://aws.typepad.com/aws/2008/12/running-everything-on-aws-soocialcom.htmlcopyright (c) VMware Inc.


Inter‐cloudapplica7onsuseinter‐applica7onmessagingtorouteanddeliverdata

!

WestEuropedatacentre

NEEuropedatacentre

SouthernEuropedatacentre



Inter‐cloud“securenetworkoverlay”‐usesRabbitMQunderthecovers

See http://www.cohesiveft.com/vpncubed/copyright (c) VMware Inc.


http://www.cohesiveft.com/vpncubed/

http://www.cohesiveft.com/vpncubed/

DMAC

PUBSUBatmassivescale:“TwiIerforDataintheCloud”

Service.DomainName

Region Virtual IP

Cloud IPs

See: http://www.oceanobservatories.org/spaces/display/CIDev/HomeFriday, 23 April 2010

http://www.oceanobservatories.org/spaces/display/CIDev/Home

http://www.oceanobservatories.org/spaces/display/CIDev/Home

Pubsub?wZ?



At scale, coping with changing data can be a problem



19

Socialapplica7onsstoredata



20

And thereʼs terabytes of it




21

And thereʼs terabytes of it

And itʼs in the cloud




Tell me when THIS changes



PubsubwinsbecausePollingSucks...“Arewethereyet?”,“Arewethereyet?”,..

Pubsubhubbub provides web pubsub

Pubsub Hubs are Cloud Messaging Technology

Check out RabbitHub on github for more info/code

Check out the Google teamʼs video at

http://www.youtube.com/watch?v=B5kHx0rGkec





Monitoringdonewrong



Monitoringdoneright



Monitoringdoneright

See: Ben Hyde’s post at http://enthusiasm.cozy.org/archives/2009/02/listening-to-the-system



Monitoringdoneright

27copyright (c) VMware Inc.


Monitoringdoneright

28Lots more event processing goodness here: http://blog.cloudeventprocessing.com/



29

PubsubgetsyouMonitoring‐andifyouaddQueuesyougetLogging



Recordinginforma7onisoffundamentalhumanvalue



Sowhatismessaging?

Messages

Pubsub

Queues

Log Storage

messagingis‘datainmoSon’anditisreallyimportantand‘themovetocloudcompuSng’willputmessagingeverywhere



ThereareLOTSofcloudmessagingtechnologies!

Messages

Pubsub

Queues

Log StorageAMQP, NoSQL, SQL

Routing, transforming, alertingeg SNS, ESBs and Esper CEP

Buffering, caching, filteringeg SQS, Redis

Email, Chat, Comet, BOSH, ..SMS, Twitter, ..

Trading, Cash transferSMTP, HTTP, XMPP, AMQP

AMQP, HTTP PSHB, XMPP

AMQP, (XMPP)

Archiving + searcheg Riak, Cassandra



...andRabbitMQ’sgoalistomakeallofthisreallysimple

Messages

Pubsub

Queues

Log StorageAMQP, NoSQL, SQL

Routing, transforming, alertingeg SNS, ESBs and Esper CEP

Buffering, caching, filteringeg SQS, Redis

Email, Chat, Comet, BOSH, ..SMS, Twitter, ..

Trading, Cash transferSMTP, HTTP, XMPP, AMQP

AMQP, HTTP PSHB, XMPP

AMQP, (XMPP)

Archiving + searcheg Riak, Cassandra

Thank you!

Im in yr serverz,queueing yr messagezPhoto credit: http://flickr.com/photos/53366513@N00/67046506/



RabbitMQ

TowardsaPLATFORMforCloudMessaging



IfcloudrequiresaCloudOS,thenismessagingtheCloudBus?



ForInfra,yes.Manyclouds,egNASANebula,haveaRabbitMQcloudbus:

Usecases:Inter‐VMchaIerAndlotsofManagement:RequestcapacityStart100VMs.Shutthemdown.“Iamoverloaded”“Findme‐whoamI?”“Moveme”“Backmeup”Scalebackcapacity



That’sgreatbutwealsoneedaPLATFORMifwearetodeployandrunapps

SaaS cloud applications

PaaSeg RabbitMQ on Heroku

IaaSeg Rabbit at NASA



PlaZormasaserviceexample:Heroku!

SaaS cloud applications

PaaSeg RabbitMQ on Heroku

IaaSeg Rabbit at NASA



Conceptually:aplaZormrepresentsfreedomfromcomplexity

App AppApp

Scalablecloudbus‐runasmanyasyoulike(Rabbitisopensourceandcanrunonanycloudasits‘centralnervoussystem’)

IaaS infra is hidden



Pubsubismadesimple‐likeappstwee7ngateachother!

App AppApp

“hey” “hey”“hey”



Queuesareeasytoo

App AppApp

“hey”

“hey”



Queuesareeasytoo

App AppAppApp

“hey”



Bybeing100%open‐wecanalsodointercloudjustassimply!

Intra-App Intra-Cloud Inter-Cloud

Rabbit Cloud Service

Rabbit Cloud Service

Rabbit Cross-cloud Services

Rabbit inside the Cloud VM

RabbitMQ as infra: “Cloud Bus”

Federated RabbitMQIaaS

PaaS



44

Tobridgeprivateandpublicclouds,opensourceandopenstandardsarebest

SAY NO TO

CLOUD

LOCK IN

(CC) Giara @flickr.com copyright (c) VMware Inc.


Summary‐cloudsbreakthelinkbetweenapplica7onandloca7on

Can’t predict where apps and VMs are running

Can’t predict scale

Design for DATA IN MOTION

In motion VMs need PUBSUB and ROUTING like with cell phones

VMs suspended or offline need QUEUES

Messaging lets you ‘dial up’ VMs and broadcast changes safely



Solu7on‐RabbitMQistheopencloudmessagingplaZorm

Fantastic community support for messaging intra app

Language neutral open protocols for messaging inter app

Federation - bridge private and public ‘intercloud’

Cloud Bus for IaaS

Messaging PaaS



Solu7on‐RabbitMQasanopencloudmessagingplaZorm

We’ll manage messaging for you

Scalable, easy to use

Completely portable - no lock in

Pre-provisoned - zero install, pay as you go, self-service

Latency is low: Application logic and data are CO-LOCATED

WIN



RichWolski,CTOandFounder,Eucalyptus

“pre-ymucheverythingyouownisgoingtobetryingtosendyoudata”

StefanNorberg,ChiefArchitect,Unibet

“Ifyouremovetheneedtoinvestininfrastructure,theneedtotrainpeopleontheoperaSonalaspectsandthengetexcellentscalabilityandlowlatencyguaranteedbycontract,I’dbuyitinasecond.WhowillprovidemewiththeRealTimeWebasaservice?”



Thank‐you

Email:[email protected]

Getstarted:h-p://www.rabbitmq.com/how.html



mailto:[email protected]

mailto:[email protected]

http://www.rabbitmq.com/how.html

http://www.rabbitmq.com/how.html

Documents

RabbitMQ at Skills Maer Cloud Exchange Cloud Messaging · PDF fileCloud Messaging Use Cases April 2010 ... Pubsub Hubs are Cloud Messaging Technology ... Check out the Google teamʼs