Scaling CometD by Kevin Nilson

Scaling CometD to The Masses

by Kevin Nilson

Principal Architect / E*Trade Financial

ww

w.d

evo

xx.c

om

Learn about CometD and Scaling CometD Apps

ww

w.d

evo

xx.c

om

Agenda

Quick Intro to CometD

Basic Steps (Outside CometD)

Advanced Steps (Inside CometD)

ww

w.d

evo

xx.c

om

Kevin Nilson

Java Champion

Co-authored Web 2.0 Fundamentals Book

Leader Silicon Valley Web Developer JUG

Lead Silicon Valley Google Tech User Group

Leader Silicon Valley JavaScript Meetup

Taught 7 Courses College Of San Mateo

Winner of MySpace Editor's Choice Award for project YumieDate, at Open Social Weekend Apps hosted at Google

ww

w.d

evo

xx.c

om

Intro To CometD

What’s Comet? What’s Bayeux?What’s CometD?

Pushing Data to The Net over HTTPChannel & JSON Based Comet Protocol

Comet Client and Server Implementation using BayeuxWhat’s WebSockets?

DEMO

Polling Interal

Polling

Long Polling

Streaming

ww

w.d

evo

xx.c

om

Basic Steps (Outside CometD)

Reduce overheadAllow MultithreadingRemove Simple Bottlenecks

ww

w.d

evo

xx.c

om

Remove Apache

Apache Has 1 Thread Per Request Limit

Browse Apache Jetty

ww

w.d

evo

xx.c

om

Separate Client Into Tiers

Some high throughput some lowSpreads broadcast over multiple threads

Delivery to 5 channels can be handled by 5 threadsDelivery to 1 Channel can be handled by only 1

thread

/GameUpdatePublisher

/Player/GameUpdate

/Watcher/GameUpdate

High Volume

Low Volume

ww

w.d

evo

xx.c

om

Only Deliver Changed Fields

Message 1 Message 2 Message 3 Message 4 Message 5

Green Score 0 0 0 1 2

White Score 0 1 2 2 2

Total Players 10 11 11 9 9

50% Field Delivery Reduction

ww

w.d

evo

xx.c

om

Advanced Steps

Move Customers to slower tier if they are not consuming fast enough

Disconnect users with very “large” queues

Auto adjust throttling if overall we are producing faster than we can consume (deliver to customers)

ww

w.d

evo

xx.c

om

Before, No Internal Visiblity

Jetty

JMSBayeuxServic

e

Browser

I can see

I can’t see

LEDGEND

filter

ww

w.d

evo

xx.c

om

Before – Full Story

Jetty

JMSBayeuxServiceBrowser

Queue

I can see

I can’t see

LEDGEND

filter

ww

w.d

evo

xx.c

om

After

Jetty


Queue

I can see

I can’t see

LEDGEND

QueueListenerDeliverListener

filter

ww

w.d

evo

xx.c

om

Queue & Deliver Listener Code

client.addListener(new DeliverListener( ) {public void deliver(Client arg0, Queue<Message> queue) {

…}

}

client.addListener(new QueueListener() {public boolean queueMaxed(Client client, Client client1, Message arg2) {

…}

}

ww

w.d

evo

xx.c

om

Prevent Huge Queue

Jetty


Queue

I can see

I can’t see

LEDGEND

Check Queue Size:Disconnect Client if

“Large”

filter

ww

w.d

evo

xx.c

om

Measure Average Time in Queue

Jetty


Queue

I can see

I can’t see

LEDGEND

Check Time In Queue:Move Client to Slower Tier or Increase Overall

Throttling

filter

ww

w.d

evo

xx.c

om

Adjust Tier: Avg Queue Size of Client

JMSMyBayeuxService

Platinum

GoldSilver

Bronze

LEDGEND

Platinum = 100 updates / minuteGold = 50 updates / minuteSilver = 25 updates / minuteBronze = 10 updates / minute

ww

w.d

evo

xx.c

om

Broadcast in CometD, not JMS

JMSMyBayeuxServiceBrowserBrowserBrowser

JMSMyBayeuxServiceBrowserBrowserBrowser

ww

w.d

evo

xx.c

om

Channel Create & Remove Listener

JMSMyBayeuxService

channelAddedchannelRemoved

ww

w.d

evo

xx.c

om

ChannelBayeuxListener Code

bayeux.addListener(new ChannelBayeuxListener() {public void channelAdded(Channel channel) {

// Code Here}

public void channelRemoved(Channel channel) {// Code Here

}});

ww

w.d

evo

xx.c

om

Adjust Tier Rate based on Avg Queue Size

JMSMyBayeuxService

Platinum

GoldSilver

Bronze

LEDGEND



ww

w.d

evo

xx.c

om

Auto Throttle: Avg Message Delay

LEDGEND

Platnium = 100 updates Gold = 50 updatesSilver = 25 updatesBronze = 10 updates

Jetty


Queue

filter

Platnium = 50 updates Gold = 25 updatesSilver = 10 updatesBronze = 5 updates

ww

w.d

evo

xx.c

om

Summary

CometD Intro

Basic Steps

Advanced Steps

. . .

ww

w.d

evo

xx.c

om

Thanks

[email protected]

Web2-book.com - Web2.0 Fundamentals

http://web2-book.com



Technology

Scaling CometD by Kevin Nilson