Message Oriented Web Services

8/3/2019 Message Oriented Web Services

1/76

ThoughtWorks, 2006

Message-Oriented Web ServicesA Tutorial using WCF

Dr. Jim Webber

Service-Oriented Systems Practice LeadThoughtWorks


2/76

ThoughtWorks, 2006

Roadmap

Introduction to Web Services patterns Designing a Web Services-based application

In an iterative manner

Building out some simple services

Thinking about messages and metadata

Demos (hopefully!)

You guys participate!

Q&A


3/76

ThoughtWorks, 2006

In the Beginning...

In the old days we used SOAP-RPC RPC style SOAP plus encoding rules for serialising and deserialising

application-level objects into messages.

A kind of XML RPC

Similar to RMI etc but without all the good stuff like garbagecollection, security, and performance.

Remote object method

invocations

Network

This is not SOA


4/76

ThoughtWorks, 2006

Web Service Architecture

Todays Web Services architecture is based on the notionof:

Services, which exchange

SOAP Messages, which are constrained in form and exchange

pattern by

WSDL, which may also advertise

Or SSDL!

Quality of Service Protocols, defined by the

WS-* specifications


5/76

ThoughtWorks, 2006

Message-Orientation

The action taken in response to receiving a message is upto the recipient

At the highest architectural level, a message exchange is simply atransfer of information from sender to receiver.

The business semantics of a message exchange are service-specific.

Transfer of messages is transport protocol independent

The means of transporting a message carries no semantics I.e. POSTing or GETting does not change the semantics of a message.


6/76

ThoughtWorks, 2006

SOA and Web Services Aims

Integration Reuse

Agility

Lower cost of development/ownership for applications

No such thing as a free lunch

Have to architect and implement to achieve these features


7/76 ThoughtWorks, 2006

High Level Architecture

Service

Administrative

domain

Service

Service

Service

Administrativedomain

Service

Service Administrative

domain

...

Not validated

WSDL contract



SOAP + WS-Addressing

http tcp jms

logical point-to-point SOAP message transfer

intermediate intermediate

Web Service

transport

Web Service

transport



Implementation

The internal architecture of aservice is relatively mundane.

The layering of messaging,logic, and state is similar to the

classic N-tiered architecture.

The message processing layeris interesting though...

BPEL,

Java,.N

ET

serv

icelogic

Databases

Humans

ProgramsComputational resources

Devices

resour

ces

...

...

messageprocessing

SOAP,

WSDL,and

otherWS-*protocols

processing

transport

SOAP messages

TCP, UDP, HTTP, SMTP, JMS, etc.



Message Processing Layer

Deals with the conversion ofmessages on the wire (in SOAPformat) to programmaticabstractions.

Does not try to abstractmessage exchanges into RPC

Presents a truthful picture of theunderlying message exchangesto the service logic

Allows the service logic to betolerant of the underlyingmessaging infrastructure

Latent, lossy, asynchronous etc

Messages are the abstractionthat the service logic binds to,not other services

Loose coupling!

messageprocess

ing

transportbod

y

header

SOA

PEnvelope

bod

y

header

SOA

PEnvelope

incoming SOAP message

outgoing SOAP message

servicelogic

messageobject

messageobject

Network

Client Application

Server objects and

stubs

WSDL



Todays SOAP API

public class GameServiceStub

{

public void StartGame()

{...}

public GridPos Move(GridPos gp)

{...}

// Other remote methods omitted for brevity

}



A Message-Oriented API

public interface IMessage

{...}

public class StartGameMessage : IMessage

{...}

// Other messages omitted for brevity

public class GameMessagingLayer

{

/* Outbound messages */

public void SendMessage(IMessage msg){...}

/* Inbound messages */

public event MoveMessageReceivedDelegate MoveMessageArrived;

public event StartGameMessageReceivedDelegateStartGameMessageArrived;

// Other messaging events omitted for brevity

}



Service Logic

The arrival of a message at a service causes someprocessing to occur

From the MEST architectural style

Service implementation is stateless

All the information it needs to perform a specific task is containedwith the message that initiates that task, is computed, or iscontained in the persistent data storage tier.

c.f. architecture of the web.

Non-functional requirements (e.g. security, transactions) setcontext for message processing

But are orthogonal to the business logic



Enterprise Issues

Web Services are not an enterprise platform in the traditional sense

Though an individual Web Service will usually be deployed onto such aplatform (J2EE app server, .Net)

Just a collection of specs and implementations

messages, message exchange patterns, and semantics

Cannot rely on the infrastructure to manage dependabilitycharacteristics

Security, reliable message transfer, transactionality, scalability etc

Need to understand how the WS-* specs can help

And how then cannot help.



Security

WS-Security does not make your Web Services secure! It can help with message-level security

Privacy

Integrity

Non-repudiation HTTPS is not sufficient

Messages traverse arbitrary networks

Point to point mechanisms are difficult to scale

And insecure for multi-hop paths

WS-Security elements are embedded within the message

And easily traverse arbitrary networks



Secure Message Transfer

http tcp jms

logical point-to-point SOAP message transfer

resources

message processing

Web Service logic

transport

resources

message processing

Web Service logic

transport

intermediate intermediate



Reliable Messaging

In theory, reliable messaging is transparent to the architecture

Reliable messaging gives us at most once/exactly once notification ofdelivery of messages

No indication whatsoever of whether a message was processed

Messages can go missing or be duplicated, the protocol will detect that

Not in a catastrophic failure case

Can rely on the underlying protocol to smooth out the lumps in a large-scale Web Service networks

No such thing as a free lunch

The lumps will always be there, RM can only reduce their frequency andsize



Scalability

Stateless services scale easily.

Add more hardware, deployservices.

Leave state management tounderlying enterprise-gradeDatabase.

DB guys know how to managestate in a dependable manner

Server Farm

Data Tier



Dependability

Statelessness makes failover fault tolerance trivial

Well mostly...

If it can be detected that a Web Service or a host server has crashed,simply route messages to live Web Services on live hosts

Standard systems management stuff for the hardware WS-Management may help for Web Services

Stateful services will cause you pain

Do you really want to have to build state migration capabilities into your WScode?

Avoid WS-RF and dependent toolkits/specs.



State Sucks

Server Farm

Unknown Conversation

Multi-Protocol Session-Affinity Aware Router$$$



State is the DBAs Problem!

Server Farm

Data Tier

Inexpensive Round-Robin Router


22/76

ThoughtWorks, 2006

Implementing in the Brave New World

Both Java and.Net have message-oriented APIs Give much greater flexibility in terms of wire format

Reinforce the message-passing architectural paradigm and enable loosecoupling

ASMX and JAX-RPC will hurt you, dont use them!

With the caveat that you can avoid hurting yourself with them, but its hardwork

Toolkits like WCF are the way forward.

WSE, WSIT, and CeltiXFire are cool too.

Alpine and Dingo are interesting ideas in progress.

Well look at the features of WCF in this talk, but the architecturalprinciples apply more broadly.


23/76

ThoughtWorks, 2006

WCF Roadmap

WSE 2.0, 3.0 are a halfway house between the current WS stack and WCF (Indigo)

WCF provides implementations of (some of) the various WS-* protocols that Microsoftsupports

Provides new APIs and libraries for developing WS applications

WCF hasnt quite deprecated WSE yet...

WSE2

.0,3.0,...

Indigo

ProtocolSupport

Time

WSE Converges with

Indigo(Longhorn, 2007?)


24/76

ThoughtWorks, 2006

Messaging

Messages are at the core of Web Services WCF provides a transport-independent messaging infrastructure

Build message-exchanges (protocols) over arbitrary underlyingtransport protocols

WCF does not understand transfer protocols per se

WCF provides four levels of API out of the box:

RPC services (automatic parameter to message conversion)

Typed Services (custom messages)

Untyped Services (processMessage style)

Messaging Layer (low level)

We will prefer typed services in this session.

Though it restricts the message-exchange patterns that we can support torequest-response or less.


25/76

ThoughtWorks, 2006

WS-Addressing

WS-Addressing supports two main features:

Transport neutral addressing and identification mechanisms

Standard means of sharing Web Services addresses

endpoint reference

Addressing features are mapped onto particular transport protocol features attransfer time

Makes possible muti-hop messages over arbitrary transport

uuid:aaaa-cccc-dddd-eeee

http://example.com/consumer

http://example.com/service

http://example.com/doSomething

uuid:11-22-33-44-55


26/76

ThoughtWorks, 2006

WS-Addressing Specifics

Each WS-Addressing header block consists of: [destination] : URI (mandatory)

[source endpoint] : endpoint reference (0..1)

[reply endpoint] : endpoint reference (0..1)

[fault endpoint] : endpoint reference (0..1) [action] : URI (mandatory GRRR!)

[message id] : URI (0..1)

[relationship] : (QName, URI) (0..unbounded)

We will see how these are used by WCF to enable transport-neutraladdressing and message correlation.


27/76

ThoughtWorks, 2006

WCF Knobs and Levers

ServiceContract behaviours: Session (boolean)

OperationContract behaviours:

IsInitiating (boolean

IsTerminating (boolean)

Bound to channel lifetime

ServiceBehaviour behaviours:

InstanceContextMode

PerCall default stateless implementation

PerSession one instance per session

Shareable share instance between sessions

Single singleton instance for all

All utilising features from WS-Addressing at the messaging layer


28/76

ThoughtWorks, 2006

Demo: Uncorrelated Message Exchange

Use a service proxy to send a message Use an in-process hosted service instance to receive a message

This demo uses binary SOAP over TCP

MTOM is the standard here

Decouple the Web from Web Services

Using WCF SvcTraceViewer.exe utility to spy on messages going

through to the Indigo stack


29/76

ThoughtWorks, 2006

Demo: Correlated Message Exchange

Use session flag to get WS-Addressing to correlate messages exchangeswithin the scope of a session

IsOneWay = false allows request-response correlation

Session controls allow multiple exchanges within a single service sessionscope

Note: WS-Context (or better) is the general solution here! Sessions not advertised in WSDL, but individual exchanges are

See SSDL for a better contract language...

WCF uses features from WS-Addressing to manage message correlation(relatesTo)

Look no HTTP! This demo again uses SOAP over TCP

Decouple the Web from Web Services

You get the message


30/76

ThoughtWorks, 2006

Message Fundamentalism

Messages are the fundamental abstraction of all Web Services Were back to the explicit message-passing paradigm

The examples in the previous demos can be used as the basis for hugeenterprise applications

Were just going to use them to play a game of noughts and crosses....


31/76

ThoughtWorks, 2006

Would you like to play a game?


32/76

ThoughtWorks, 2006

Design Requirements: Iteration 1

A computerised version of the two player game of noughts-and-crosses Played on a 3x3 grid


33/76

ThoughtWorks, 2006

Iteration 1 Output

X O

O

X X

Player 1 Player 2


34/76

ThoughtWorks, 2006



Playable by distant opponents

Using commodity network infrastructure


35/76

ThoughtWorks, 2006

Iteration 2 Output

X O

O

X X

Player 1 Player 2

X O

O

X X

Player 1 Player 2Network


36/76

ThoughtWorks, 2006



Playable by distant opponents

Using commodity network infrastructure

Allow 3rd-party vendors to interoperate


37/76

ThoughtWorks, 2006

Iteration 3 Output

X O

O

X X

Player 1 Player 2

X O

O

X

Poms Aussies

WS Stack WS Stack

Have todesign thisapplication

protocol(and decide

QoS features)
http://www.flags.net/ASTL.htmhttp://www.flags.net/ENGL.htmhttp://www.flags.net/ENGL.htmhttp://www.flags.net/ASTL.htmhttp://www.flags.net/ENGL.htmhttp://www.flags.net/ASTL.htmhttp://www.flags.net/ASTL.htmhttp://www.flags.net/ASTL.htmhttp://www.flags.net/ENGL.htmhttp://www.flags.net/ENGL.htmhttp://www.flags.net/ASTL.htm


38/76

ThoughtWorks, 2006

Noughts and Crosses Architecture

The application consists of:

(SOAP) message set;

Pluggable messaging layer whichencapsulates the MEPs andmessages;

Application logic which deals interms of domain-specific objects(moves, grid squares, noughts,crosses etc), and understands therules of the game;

Game UI which allows players toinitiate games and make and receivemoves.

This is the canonical architecture forall Web Services

Application-level objects are notdirectly exposed

We instead use WCF messageclasses

.

NetFramework

WSE

2.0

(and

ASP.Net)

Messaging Layer

SOAP Messages

GUI

Network

Policy

Noughts and

Crosses

Implementation

User-

Extensible

Components

Underlying

Platform

Application Logic

Message

Sender

Message

Listener


39/76

ThoughtWorks, 2006

Architectural Principles

The service is an entity which exchanges messages; Messages exchanges are governed by an immutable contract (usually

specified in WSDL)

No changes to the service implementation are allowed to violate thatcontract;

No information about the service implementation, or its resources is allowedto propagate outside they service;

Service is stateless

All the information it needs to perform a specific task is contained with the

message that initiates that task - c.f. architecture of the web.

Non-functional requirements (e.g. security, message integrity) are outof scope for service implementation

They are administrative tasks (mostly!).


40/76

ThoughtWorks, 2006

Demo: Noughts and Crosses

A simple noughts and crosses game Using SOAP messaging over HTTP

Can use arbitrary protocols

TCP, HTTP, MSMQ (via IIS, WAS, or in-process) supported out of the box, otherfolks (MS and non-MS) developing additional transports

Want to demonstrate (again) decoupling of Web Services from WebProtocols

Using SvcTraceViewer.exe or TCPTrace to spy on messages


41/76

ThoughtWorks, 2006

Code Oddities

No use of WCF sessions Want to use any underlying binding

WCF uses WS-Addressing for session management on the wire

basicHttpBinding is WS-I BP 1.1 conformant, therefore no WS-

Addressing available


42/76

ThoughtWorks, 2006

Demo: Cheating The messages are sent in plain text

With no integrity guarantees

Or privacy for that matter

Can deduce the message format

Because it isnt private

Can send forged messages to upsetthe game
http://www.charlesgale.co.uk/jpgs/2562-taller-top-hat-1b.jpg


43/76

ThoughtWorks, 2006

WS-Security

WS-Security is well supported in WCF

Also in WSIT (Sun)

Annoying to get working in Axis2!

Security is multifaceted consisting of:

Privacy

Making sure messages cannot be seen by others Transport: SSL, HTTPS (whole messages)

XML-Encryption (individual message elements)

Authenticity

Ensuring messages can be attributed to a particular sender

Integrity Ensuring message contents have not been tampered with in transit.

Non-repudiation

A sender cannot deny sending a message

In a Web Services environment, we concentrate on the security of messages


44/76

ThoughtWorks, 2006

Transport Security Issues

Transport security only useful for communication between twoendpoints;

If intermediaries are involved, they will process garbage (encryptedmessages);

Cannot solve Web Services security with HTTPS!

Need finer granularity of control.

Message-level security is the answer.


45/76

ThoughtWorks, 2006

Certificates

All security starts with trust.

Certificate authorities are trusted third parties.

And offer services to sign public keys.

A key signed by a trusted party, is trusted!

E.g. Verisign

Use your own certificates for development and testing

.Net tools: makecert and certman

Toolkey (Java)

Public and Private keys are used for encryption and signing.

Encrypt with recipients public key

Sign with senders private key and strong one-way algorithm (hash)

Lets take a look at how these are used


46/76

ThoughtWorks, 2006

Public Key Encryption Overview

Sender Receiver

Get Public Key


47/76

ThoughtWorks, 2006


Sender Receiver

Public Key


48/76

ThoughtWorks, 2006


Sender Receiver

Encryptmessage withKey


49/76

ThoughtWorks, 2006


Sender Receiver

Send Encrypted Message


50/76

ThoughtWorks, 2006


Sender Receiver

Decryptmessage

with privatekey


51/76

ThoughtWorks, 2006

Problems

Public key encryption is really computationally intensive;

Shared key encryption isnt:

But relies on the secure transfer of shared keys between parties.

Therefore can combine the two strategies:

Use public key encryption to encrypt a shared key;

Which is small

Use that shared key to encrypt the message.


52/76

ThoughtWorks, 2006

XML Encryption Overview

Core element is EncryptedData

EncryptedKey element is used to transport encryption keys from theoriginator to a known recipient.

Encrypted with public key of recipient.

Result of encryption is an XML encryption element that contains orreferences the cipher data.

Can encrypt:

Arbitrary character data;

Entire XML document;

Individual elements.

EncryptedData cannot be the parent or child of another EncryptedDataelement

The actual data encrypted can be anything including existing EncryptedDataor EncryptedKey elements.

Cant nest them, but can re-encrypt


53/76

ThoughtWorks, 2006

Encrypting Elements

We want to protect sensitive details

But not necessarily all of the document:

Dr.

Webber

Jim

1234 5678 9101 1121

0505


54/76

ThoughtWorks, 2006

Key-Encrypted Element Form

Jim Webber

DEFDBACF

Jim Webber

xyzabc

Expensiveencryption for

shared key(public key)

Cheapencryption for

data

(shared key)


55/76

ThoughtWorks, 2006

XML Digital Signatures

Used to ensure XML is:

Non-repudiable

If you sign something, the CA can prove its your signature

Tamper-proof

If you sign the manifest, the of the document cannot be secretly tampered with.


56/76

ThoughtWorks, 2006

XML Signature Basics

Canonical form is imperative

(http://www.w3.org/TR/2001/REC-xml-c14n-20010315)

Two XML documents might be logically equivalent according to their schema:

Webber

Jim

Jim


57/76

ThoughtWorks, 2006

Digital Signature Overview

Sender Receiver

e04286085deb749d7094b0472b5c8276

Hash

Hashmessage


58/76

ThoughtWorks, 2006


Sender Receiver

453487de87b3875487c8787d1eef90ab1

Encrypt

Encrypt hash(Private Key)


59/76

ThoughtWorks, 2006

Appendencrypted

hash


Sender Receiver

453487de87b3875487c8787d1eef90ab1

Signing the message


60/76

ThoughtWorks, 2006


Sender Receiver

Send Signed Message


61/76

ThoughtWorks, 2006


Sender Receiver

Get Public Key


62/76

ThoughtWorks, 2006


Sender Receiver

Public Key


63/76

ThoughtWorks, 2006


Sender Receiver

Decryptsignature with

Senders public

key

453487de87b3875487c8787d1eef90ab1


64/76

ThoughtWorks, 2006


Sender Receiver

Check hashagainst

message

e04286085deb749d7094b0472b5c8276


65/76

ThoughtWorks, 2006


Sender Receiver

Checksignatureagainst

Certificate

authority

Signing of a public key by a trusted third party allowsothers to check that a signature really belongs to thesender

CA

Signature

Owner


66/76

ThoughtWorks, 2006

XML Digital Signature Fundamentals

XML Signatures are applied to arbitrary message content.

Contents are digested (hashed).

The resulting value is placed in an element (with other information) .

That element is then encrypted.

XML digital signatures are represented by the Signature element:


67/76

ThoughtWorks, 2006

Signature Element

j6lwx3rvEPO0vKtMup4NbeVu8nk=

MC0CFFrVLtRlk=...

...

...

......

How the XML wascanonicalised


68/76

ThoughtWorks, 2006

Signature Element


MC0CFFrVLtRlk=...

...

...

...

...

The hash (usuallySHA1)


69/76

ThoughtWorks, 2006

Signature Element


MC0CFFrVLtRlk=...

...

......

...

The encryptedhash (the

signature!)


70/76

ThoughtWorks, 2006

Signature Element


MC0CFFrVLtRlk=...

...

...

...

...

Key Information (usually acertificate)

Helps receiver verifysignature easily


71/76

ThoughtWorks, 2006

All that Security Stuff is REALLY Hard

Security is a tough problem, even if you understand the maths

The XML formats used in WS-Security seem to defy human reading

WCF gives us programming abstractions for dealing with this stuff.

Its still difficult!

There is a better way...

WS-Policy and WS-SecurityPolicy allow us to specify securityrequirements (e.g. signed and encrypted messages) declaratively

And WCF lets us specify WS-SecurityPolicy settings declaratively usingbindings

Phew!


72/76

ThoughtWorks, 2006

WS-Policy

A family of related specs:

WS-Policy defines the base syntax

WS-PolicyAssertions contains a set of common message assertions (e.g. only once)

WS-PolicyAttachment describes how to attach policies to WSDL contracts and UDDIentities

WS-SecurityPolicy contains security assertions which dovetail with WS-Securitymechanisms

Etc...WS-Policy

WS-PolicyAssertions WS-PolicyAttachment WS-SecurityPolicy


73/76

ThoughtWorks, 2006

WS-Policy: Dual Aims

To be able to capture and implement policy requirements for a WebService (and its users) in a declarative way.

To be able to advertise the policy requirements for a Web Service aspart of or as an adjunct to the WSDL contract.

Standardisation helps a little in the first case

Can move my policy file between (say) .Net and Axis

Standardisation helps inordinately with the second case

Can advertise and consume/process policies in a standard way


74/76

ThoughtWorks, 2006

Demo: Message Integrity and Privacy

Two major problems:

Messages are visible

Messages are easily tampered with

A minor problem

Replay attacks

Allow only authorised users to send messages to the Web Services embedded

within the Noughts-and-Crosses game Can use similar WS-SecurityPolicy mechanisms to bootstrap WS-Security-based

message encryption

Well use Windows-only methods for simplicity

Defaultneed to do nothing more with WSHttpBinding Well use username/passwordbased mechanisms

Can use active directory-based authentication or local user accounts

Username token only over HTTPs

WCF doesnt let you do insecure things by default

Prefer X509 or Kerberos in production environments


75/76

ThoughtWorks, 2006

Wrapping Up

Wheres the WSDL?

WSDL is generated automatically for you, but it doesnt mean it can be ignored.

You must still implement sensible message-oriented patterns to make WSDL contracts hideyour service implementation

WSDL is still a critical part of the picture when you want others to bind to yourservice or vice versa

SSDL is better, but unsupported

Metadata is the new cool area to be in. The better the metadata, the less code youhave to build from scratch.

Clearly some projects could be tackled with automated tooling

The architectural principles still applydont turn them Web Services into XML-iseddistributed objects

But not when you need fine control over the wire format of the messages andmessage exchange patterns

When implementing specific protocols (system or business)


76/76

Questions?http://jim.webber.name

Documents

Message Oriented Web Services