Windows Azure AppFabric Service Bus: New Capabilities

Clemens VastersPrincipal Technical LeadMicrosoft Corporation

MID312

The Cloud.

All Apps Move To The Cloud.

No They Don’t.

CRM ERPLOBCampaig

nHR

Time Card

Benefits

HealthVendor Market

TravelLoyalty

BI Portal

BI Shop

Integrated Enterprise

CRM ERPLOBCampaig

nHR

Time Card

Benefits

Health

Sourced

Vendor Market

TravelLoyalty

BI Portal

BI Shop

?

?

Firewall

NAT

Security Realm

Very Few Go ‘All In’ On The Cloud.

CRM ERPLOBCampaig

nHR

Time Card

Benefits

Health

Sourced

Vendor Market

TravelLoyalty

BI Portal

BI Shop

?

?

Firewall

NAT

Security Realm

So How Can This Work?

• Existing Web Service Endpoint.• No Mutual Trust Between Systems,

VPN not an option.• Sits Behind Firewall and NAT. • Not built for non-corporate identity

access control

More Scenarios

Service Bus

Sender Receiver

Frontend Nodes

Ctrl

1

2

3

4

Forwarderou

tbou

nd

sock

et

connec

t

outbound

socket

rendezvous

Ctrl

NLB

Connectivity Relay

• Using Outbound Connections Only• No open inbound firewall/NAT ports• Outbound connections default to TCP• Fall back to HTTP when needed• Guarded by Datacenter Firewall• Secured with Access Control

Service Bus

Receiver

Frontend Nodes

Connectivity Relay

So Far So Good.

(“I’ve seen that already! What’s New?”)

Service BusCommercially Available since January 2010

Brand-New Community Technology PreviewAvailable Now

… in medias res ….

IntegrationRouting

CoordinationTransformation

Rich, declarative

content-based routing,

document transformation,

and process coordination.

Service Management

Naming, DiscoveryMonitoring

?

Consistent, modeling-friendly

management surface and

service virtualization capabilities

MessagingQueuingPub/SubReliable Transfer

Reliable, transaction-aware cloud messaging

infrastructure for business

apps.

ConnectivityService Relay

Protocol TunnelEventing, Push

Rich options for interconnecting

apps across network

boundaries

We’re Adding a Whole New Set Of Capabilities

Retail Points of Sale

Inventory and Fulfillment

Centers

Business Operations

and Marketing

Real Time Retail

Analytics

?

Store Queue

Service Bus

Inventory

Store Queue

Service Bus

Inventory

Store

Store

Store

Load Leveling

Store Queue

Service Bus

Inventory

Store

Store

Store

Load BalancingInventor

y

Store Data Collection Topic

Service Bus

Inventory

Sub

Sub

Audit

Audit Taps

Store Data Collection Topic

Service Bus

Inventory

Sub

Sub

Audit

Sub

Dashboard

Real-Time Observation

Store Data Collection Topic

Service Bus

Inventory

Sub

Sub

Audit

Sub

Dashboard

Sub

MartVue Analyzer

Analysis Topic Sub

Real-Time Analysis

Store Data Collection Topic

Service Bus

Inventory

Sub

Sub

Audit

Sub

Dashboard

Sub

MartVue Analyzer

Analysis Topic Sub

Catalog TopicSubScale-Out Eventing

QueueP C

Queues!

• Reliable, durable storage – up to 1GB per Queue*

• Queues stay in the system once created, no TTL limit

• Max message size 256KB, Sessions allow grouping

• Messaging API, WCF, and HTTP/REST Interfaces*Limited to 100MB per Queue for CTP

Why Queues?

Queue

Load Leveling

P C

Flatten spiky traffic into a predictable stream of work

Why Queues?

Queue

Load Balancing

P C

• Balance Work Across a Pool of Workers• Add and Remove Workers Based on Need (Queue

Length)

C

C

C

Why Queues?

Queue

Temporal Decoupling

P C

• Publish Work To Workers That Are Temporarily Offline

• Enable Scheduled Batch Processing

Why Queues?

Queue

Integrity

P C

• Don’t Lose Work If Errors Occur Before Processing Completes

• Peek/Lock plus De-Dupe can often replace full Tx semantics

Balancing Work With Queues

Work Queue

ASP.NET Frontend

Worker

ASP.NET Frontend

ASP.NET Frontend

ASP.NET Frontend

ASP.NET Frontend

Worker

Worker

Worker

Worker

Reply Queue

Reply Queue

Reply Queue

Reply Queue

• Can’t Stress Out The Backend• Frontend queries by correlation ID with timeout• Notifies user if timeout passes without response

Topics!

TopicP S

S

S

C

C

C

• All the Features of Queues + Publish/Subscribe• Up to 2000 Subscriptions on a Topic• Each Subscription is a virtual queue getting message

copies• Subscriptions can have filters and actions

Rules and Filters

• Rules:– Rules select messages based on conditions– More than one rule for a subscription– Each matching rule yields copy of a message

• Filter Conditions and Actions:– Condition represented by expressions– Actions can modify msg props during

retrieval

Subscription

Rule

Rule

Condition

Action

Condition

Action

Filter Expressions

• InvoiceTotal > 10000.00 OR ClientRating <3• ShipDestCtry = ‘USA’ AND ShipDestState=‘WA’• LastName LIKE ‘V%’

Filter conditions operate on message properties and are expressed in SQL’92 syntax

SET AuditRequired = 1

Filters actions may modify/add/remove properties as message is selected

Why Topics?

Topic

Event Fan Out

P S

• Publish Events To Many Subscribers• Subscription Can Be Shared By Competing

Consumers

S

S

C

C

C

C

C

Why Topics?

Topic

Tap

P S

• Primary Subscription Used Like a Queue• Secondary Subscription Used As Audit Tap

S

C

C

C

C

Why Topics?

• Filters applied to distribute messages to partitions

• Mutually exclusive filters covering ranges

Topic

Partitioning

P S

S

S

C

C

C

C

C0 < x < 33

67 < x < 100

34 < x < 66

“Ok, I Get It”

Code? Code!

Creating Things on The NamespaceServiceBusNamespaceClient namespaceClient = new ServiceBusNamespaceClient(baseAddress, TransportClientCredentialBase.CreateSharedSecretCredential(acc, key));

// Create a queue with default settingsnamespaceClient.CreateQueue("DefaultQueue");

// Create a queue with custom settingsQueueDescription qd = new QueueDescription();qd.DefaultMessageTimeToLive = new TimeSpan(1, 0, 0);qd.EnableDeadLetteringOnMessageExpiration = true;namespaceClient.CreateQueue("CustomQueue", qd);

// Delete entitiesnamespaceClient.DeleteQueue("DefaultQueue");

36

The Service Bus Namespace Client

• Management operations are performed using the ServiceBusNamespaceClient

– CreateTopic, GetTopic, DeleteTopic

– CreateQueue, GetQueue, DeleteQueue

• Yields Queue and Topic instances– Provide access to metadata and settings

• Takes QueueDescription and TopicDescription instances

– Inputs for CreateTopic/Queue

/

Q T

ServiceBusNamespaceClient

create delete

clemens.servicebus.appfabriclabs.com

Runtime API Choices

WCF Bindings

Service Bus Relay Protocol

Implementation(private)

Apps

Messaging API

WCF Service Model

.NET Framework 4.0

Service Bus

HTTPREST

SOAP WS-*

(Relay Clients)

Any Platform

The Messaging API

WCF Bindings

Service Bus Relay Protocol

Implementation(private)

Messaging API

WCF Service Model

.NET Framework 4.0

HTTPREST

SOAP WS-*

(Relay Clients)

Any Platform

Messaging-Specific API That Provides Access To All Features

39

MessagingFactory• Clients for sending/receiving messages

are created via MessagingFactory• Factory creates entity-specific “client”

– QueueClient, TopicClient, SubscriptionClient

• Clients create send and receive objects– MessageSender, MessageReceiver,

SessionReceiver– Provides an abstraction that allows client

send/receive code to be written to Queues, Topics, and Subscription in an identical fashion.

MessagingFactory

QueueClient

Connection Manager (private)

TopicClient

MessageSender

MessageReceiver

SessionReceiver

SubscriptionClient

40

Sending MessagesMessagingFactory factory = MessagingFactory. Create(baseAddress, factorySettings);

QueueClient queueClient = factory.CreateQueueClient("DefaultQueue"); // Create a MessageSenderMessageSender ms = queueClient.CreateSender();

// Send a message with a String bodyms.Send(BrokeredMessage.CreateMessage("Test message"));

// Send a message with system and application properties setBrokeredMessage bm = BrokeredMessage.CreateMessage();bm.Label = "PurchaseOrder123";bm.TimeToLive = new TimeSpan(0, 5, 0);bm.Properties["PurchaseOrderID"] = 1234;

ms.Send(bm);

41

Receiving Messages// Create a MessageReceiver with ReceiveAndDelete semantics MessageReceiver mr = queueClient.CreateReceiver(ReceiveMode.ReceiveAndDelete);BrokeredMessage rm = mr.Receive();

// Create a MessageReceiver with PeekLock semanticsMessageReceiver mr2 = queueClient.CreateReceiver(ReceiveMode.PeekLock);

BrokeredMessage rm2 M= mr.Receive();

try{ // ProcessMessage(rm2);

rm2.Complete();}catch (Exception e){

rm2.Abandon();}

42

Sessions// Create a SessionReceiver to get next available sessionSessionReceiver sr = queueClient.AcceptSessionReceiver(ReceiveMode.PeekLock); BrokeredMessage rm3 = sr.Receive();try{ ProcessMessage(rm3); sr.Session.SetState(bytes); rm3.Complete();}catch (Exception){ rm3.Abandon();}

43

TopicMessagingFactory factory = MessagingFactory.Create(baseAddress, factorySettings); // Create a TopicClientTopicClient topicClient = factory.CreateTopicClient("DefaultTopic");// Create a MessageSenderMessageSender ms = topicClient.CreateSender();// Send a message with a String bodyms.Send(BrokeredMessage.CreateMessage("Test message"));// Create a SubscriptionClientSubscriptionClient subscriptionClient = factory.CreateSubscriptionClient("DefaultTopic/MySubscription"); // Create a MessageReceiverMessageReceiver mr = subscriptionClient.CreateReceiver(ReceiveMode.ReceiveAndDelete);BrokeredMessage rm = mr.Receive();

44

Subscription Filter Rules// Create a topic and 3 subscriptions.Topic topic = namespaceClient.CreateTopic(Program.TopicName); // Create a subscription for all messages sent to topic.topic.AddSubscription(SubsNameAllMessages, new MatchAllFilterExpression()); // Create a subscription that'll receive all messages with// color property with value "blue" and brushSize with value 10. string filterExpression = "color = 'blue' AND brushSize = 10"; topic.AddSubscription(SubsNameFilteredMessages, new SqlFilterExpression("color = 'blue' AND brushSize = 10"));

// Create a subscription that'll receive all messages which have CorrelationId set to footopic.AddSubscription(SubsNameCorrelatedMessages, new CorrelationFilterExpression("foo"));

45

Subscription Filter Actions//Create a topicmyTopic = namespaceClient.CreateTopic("IssueTrackingTopic");// Create a Rule that stamps Importance property value 'Low'RuleDescription defaultPriRuleDescription = new RuleDescription();defaultPriRuleDescription.FilterAction = new SqlFilterAction("set Importance = 'Low';");// Create a Rule that selectively stamps Priority and Importance valuesRuleDescription highPriRuleDescription = new RuleDescription();highPriRuleDescription.FilterExpression = new SqlFilterExpression("SupportPackage = 'Premium' OR Severity <= 1");highPriRuleDescription.FilterAction = new SqlFilterAction("set Priority = 1;set Importance = 'High';");// Create a Subscription and apply rulesSubscription mySub = myTopic.AddSubscription("IssueTrackingSubscription", defaultPriRuleDescription);mySub.AddRule("High Priority", highPriorityRuleDescription);

WCF ServiceBusMessagingBinding

WCF Bindings

Service Bus Relay Protocol

Implementation(private)

Messaging API

WCF Service Model

.NET Framework 4.0

HTTPREST

SOAP WS-*

(Relay Clients)

Any Platform

Convenient, Familiar Experience For Many Apps

47

WCF Experience

• ServiceBusMessagingBinding– One-Way Messaging

– Automatic Pump – Pull/Push Translation

– IInputChannel, IOutputChannel, IInputSessionChannel

– Transport-Level Security

• ServiceBusMessagingFactorySettings available for tweaking as binding properties

• BrokeredMessage available as a message property of the WCF message on receive

Application / Dispatcher

.Net API

Service Bus Transport Channel

SOAP over SBMP

Summary

AppFabric is the New Middle-TierBuilding Blocks for Building Real Apps

Service Bus is the enabler for ‘hybrid apps’Connectivity and Reliable Messaging Very rich new Publish/Subscribe CapabilitiesCTP Available Now

Related Content

COS311 - Introduction to Windows Azure AppFabric Composite Applications

MID271-INT - Futures: Integration Capabilities in Windows AppFabricMID373-INT - Accessing Enterprise Data from the Cloud and Mobile Devices Using Microsoft BizTalk Server and Windows Azure AppFabric

Track Resources

http://www.microsoft.com/windowsazure/appfabric/overview/

https://portal.appfabriclabs.com

Track Resources

Windows Azure Platform Training Kit

Windows Server AppFabric Training Kit

BizTalk 2010 Developer Training Kit

Windows Azure AppFabric on MSDN

Windows Server AppFabric on MSDN

AppFabric Team Blog

Resources

www.microsoft.com/teched

Sessions On-Demand & Community Microsoft Certification & Training Resources

Resources for IT Professionals Resources for Developers

www.microsoft.com/learning

http://microsoft.com/technet http://microsoft.com/msdn

Learning

http://northamerica.msteched.com

Connect. Share. Discuss.

Complete an evaluation on CommNet and enter to win!

Scan the Tag to evaluate this session now on myTech•Ed Mobile

© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to

be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS

PRESENTATION.