Communication Protocols and Web Services

Created by Omer Katz, 2010

Omer Katz © Some Rights Reserved

Structure of a standard networkingstack for the Web

•HTTP

Presentation + Application Layer

•SSL

•RPC

Session Layer

•TCP/IP

Transport Layer

•IPv4

•IPv6

Network Layer

•LLC

•MAC

Data Link Layer

•LAN

•MAN

Physical Layer

Common serialization formats JSON

Natively supported within JavaScript Capable of referencing other records by convention Human readable format Lightweight Easy to parse Uses JSON-Schema to validate itself Key-Value based syntax

XML JavaScript exposes a proprietary DOM API Human readable format for hierarchal data Hard to parse Consumes more memory than JSON Supports XSLT and XPath Uses XML-Schema or DTD to validate itself Markup based syntax

What is a web service? A web service is typically an application programming interface (API) or Web API that is

accessed via Hypertext Transfer Protocol (HTTP) and executed on a remote system, hosting the requested service.

Web services do not provide the user with a GUI. Web services instead share business logic, data and processes through a programmatic

interface across a network. Developers can then add the Web service to a GUI (such as a Web page or an executable

program) to offer specific functionality to users.

Web services communication protocols RPC

XML-RPC HTTP Bound

JSON-RPC SMD – Service Mapping Description

JSONP Allows cross domain Ajax requests

SOA SOAP

XML Based Strongly typed WSDL dependant

REST Stateless Cacheable Code On Demand Client and Server are unaware of each other’s state Layered System Uniform Interface Service may be described through SMD, WSDL, WADL or not at all

What is RPC? Remote Procedure Call. Allows to access server-side/inter-process operations from a client through a well-defined

protocol. Examples: COBRA, XML-RPC, JSON-RPC, DCOM, RMI Client is tightly coupled with the server

JSON-RPC Request/Response model

Request Notification Response Error Batch operations

Advantages Lightweight JSON-Schemas can be extended to validate specific message Not bound to HTTP by the specification, but might be used with it Can be natively serialized from within JavaScript Service discovery can be achieved through SMD or WADL

Disadvantages Procedural The web service's methods grow linearly with the number of client request types Not bound to a URI

Conclusion Fits for low resources environment (embedded devices) Does not fit for rapidly growing web services Does not fit for web services operating on many entities unless the web service is an HTTP service

JSON-RPC Examples A Notification: {     ”jsonrpc”: ”2.0”,     ”method”: ”update”,     ”params”: [1,2,3,4,5] }

JSON-RPC Examples A Request{     ”jsonrpc”: ”2.0”,     ”method”: ”subtract”,     ”params”: {                    ”subtrahend”: 23,                    ”minuend”: 42                },     ”id”: 3 }

JSON-RPC Examples An Error:

{”jsonrpc”: ”2.0”,”error”: {

”code”: -32601, ”message”: ”Procedure not found.”

},”id”: ”1”

}

JSON-RPC Examples Batch Request:

[{

”jsonrpc”: ”2.0”, ”method”: ”sum”,”params”: [1,2,4],”id”: ”1”

},{

”jsonrpc”: ”2.0”,”method”: ”notify_hello”,”params”: [7]

},{

”jsonrpc”: ”2.0”,”method”: ”subtract”,”params”: [42,23],”id”: ”2”

}]

JSON-RPC Examples Batch Response:

[{      ”jsonrpc”:”2.0”,      ”result”:7,      ”id”:”1” }, {      ”jsonrpc”:”2.0”,      ”result”:19,      ”id”:”2  

  },{      ”jsonrpc”:”2.0”,      ”error”:{         ”code”:-32601,         ”message”:” Procedure not found.”      },      ”id”:”5”},

]

XML-RPC Request/Response model

Request Response Error

Advantages Bound to a URI Can represent hierarchical data in a readable way XSLT & XPath Can be validated through XML-Schema or DTD

Disadvantages Procedural The web service's methods grow linearly with the number of client request types Bound to HTTP Only appropriate for the web Hard to parse Heavy payload Consumes a lot of memory No bulk operations

Conclusion Does not fit for rapidly growing web services Does not fit for embedded devices Fits to represent hierarchical data

XML-RPC Examples Request: <?xml version=”1.0”?> <methodCall>

<methodName>examples.getStateName</methodName>             <params>                 <param>                     <value>                         <i4>40</i4>                     </value>                 </param>             </params> </methodCall>

XML-RPC Examples Response:<?xml version=”1.0”?>     <methodResponse>         <params><param>                 <value>                     <string>South Dakota</string>                 </value>             </param>         </params>     </methodResponse>

XML-RPC Examples Error:<?xml version=”1.0”?>     <methodResponse>         <fault>             <value>                 <struct>                     <member> <name>faultCode</name>                         <value><int>4</int></value>                     </member>                     <member>                         <name>faultString</name>                         <value><string>Too many parameters.</string></value>                     </member>                 </struct>             </value>     </fault> </methodResponse>

What is SOA? Service Oriented Architecture. Allows to access server-side from a client through a well-defined protocol. Examples: SOAP, REST, SOAPjr Client does not have to be tightly coupled with the server. Unlike RPC based web services which usually are not discoverable there is a way to discover

services in SOA based web service.

Service Description Standards WSDL

Web Services Description Language Well known Verbose Uses XML Not human readable Used by WCF Feels natural for SOAP services Endorsed by Microsoft WSDL 1.1 supports only GET and POST requests

Service Description Standards - Continued WADL

Web Application Description Language Not very well known A bit less verbose than WSDL Uses XML A bit more human readable Used by Java Feels natural for REST Services Endorsed by Sun Microsystems

Service Description Standards - Continued SMD

Service Mapping Description Working Draft status Less verbose Uses JSON Human readable Used by the Dojo Toolkit Not stable Endorsed by Sitepen and The Dojo Foundation

WSDL Example <description>  <types>     <schema>         <element name=”myMethod”>             <complexType>                 <sequence>                     <element name=”x” type=”xsd:int”/>                     <element name=”y” type=”xsd:float”/>                 </sequence>             </complexType>         </element>         <element name=”myMethodResponse”>             <complexType/>         </element>     </schema> </types>

…

WSDL Example <message name=”myMethodRequest”>     <part name=”parameters” element=”myMethod”/> </message> <message name=”empty”>     <part name=”parameters” element=”myMethodResponse”/> </message>

<portType name=”PT”>     <operation name=”myMethod”>         <input message=”myMethodRequest”/>         <output message=”empty”/>     </operation> </portType>

Hold your horses, there’s more…

WSDL Example <binding interface=”...”>         <!-- The binding of a protocol to an interface, same structure              as the interface element -->      </binding>      <service interface=”...”>         <!-- Defines the actual addresses of the bindings, as in 1.1,              but now ”ports” are called ”endpoints” -->         <endpoint binding=”...” address=”...”/>      </service> </description>

And it goes on and on… I actually tried to keep it short.

Have I already mentioned that WSDL is verbose?

SMD Example{

target:”/jsonrpc”, // this defines the URL to connect for the services transport:”POST”, // We will use POST as the transport envelope:”JSON-RPC-1.2”, // We will use JSON-RPC SMDVersion:”2.0”, services: {   add : { // define a service to add two numbers   parameters: [     {name:”a”,type:”number”}, // define the two parameters     {name:”b”,type:”number”}],   returns:{”type”:”number”} }, foo : {   // nothing is required to be defined, all definitions are optional.   //This service has nothing defined so it can take any parameters   //and return any value }, getNews : {   // we can redefine the target, transport, and envelope for specific services   target: ”/newsSearch”,   transport: ”GET”,   envelope: ”URL”,   parameters:[ { name: ”query”, type: ”string”, optional: false, default: ”” } ],   returns:{type:”array”} }

}

But that’s not even a standard yet

WADL Example <application xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance      xsi:schemaLocation=http://research.sun.com/wadl/2006/10 wadl.xsd      xmlns:xsd=http://www.w3.org/2001/XMLSchema      xmlns:ex=http://www.example.org/types      xmlns=http://research.sun.com/wadl/2006/10>

     <grammars>         <include href=ticker.xsd/>      </grammars>

…

WADL Example <resources base=http://www.example.org/services/>         <resource path=getStockQuote>            <method name=GET>                 <request>                    <param name=symbol style=query type=xsd:string/>                 </request>                 <response>                    <representation mediaType=application/xml                        element=ex:quoteResponse/>                    <fault status=400 mediaType=application/xml                        element=ex:error/>                 </response>            </method>         </resource>     <! many other URIs>      </resources> </application>

Looks a bit better, but it’s not widely adopted.

So what do we do?

Service Discovery - Summery Use whatever your platform allows you (WSDL for .NET, WADL for JAVA and SMD for Dojo) Don’t allow service discovery – this may happen if:

You intend the web service to be used only internally. You are not using a tool to generate a proxy. You intend the web service to always be on a fixed IP

Allow gradual service discovery Each call will expose different operations that are available Occasionally adds an unnecessary overhead

Can be solved using a parameter in the request to turn off and on gradual discovery Only exposes a part of the service that is related to the call

Service Discovery – WS-Discovery Web Services Dynamic Discovery A multicast discovery protocol that allows to locate web services over local network Uses web services standards like SOAP

SOAP Simple Object Access Protocol Message model:

Request Response Error

Advantages: Type safe WCF support feels more natural More appropriate for event driven use cases Services are discoverable Can carry any kind of data

Disadvantages: Verbose WSDL Heavy payload, especially over HTTP Does not fit for streaming HTTP already knows how to handle requests/responses Accessing from a non-WCF client is nearly impossible Uses XML for the envelope, which makes the possibility to send any kind of data pretty much obsolete

Conclusion Major pain in the ass

SOAP Example SOAP RequestPOST /InStock HTTP/1.1Host: www.example.orgContent-Type: application/soap+xml; charset=utf-8Content-Length: nnn

<?xml version=”1.0”?><soap:Envelope xmlns:soap=”http://www.w3.org/2001/12/soap-envelope” soap:encodingStyle=”http://www.w3.org/2001/12/soap-encoding”>

<soap:Body xmlns:m=”http://www.example.org/stock”>  <m:GetStockPrice>    <m:StockName>IBM</m:StockName>  </m:GetStockPrice>

</soap:Body></soap:Envelope>

SOAP Example SOAP ResponseHTTP/1.1 200 OKContent-Type: application/soap+xml; charset=utf-8Content-Length: nnn

<?xml version=”1.0”?><soap:Envelopexmlns:soap=”http://www.w3.org/2001/12/soap-envelope” soap:encodingStyle=”http://www.w3.org/2001/12/soap-encoding”>

<soap:Body xmlns:m=”http://www.example.org/stock”>  <m:GetStockPriceResponse>    <m:Price>34.5</m:Price>

  </m:GetStockPriceResponse></soap:Body>

</soap:Envelope>

Oh dear, take this abomination away

REST Representational State Transfer Constrains

Stateless Uniform Interface Layered System Cacheable Code On Demand (Optional)

Guiding principles Identification of resources Manipulation of resources through these representations Self-descriptive messages Hypermedia As The Engine Of Application State (HATEOAS)

REST is an architecture, not a standard.

This means we relay on whatever application protocol we have to

use.

REST is not bound to HTTP but feels natural with HTTP

REST – Web Services A RESTful web service is implemented over HTTP URI – Unique Resource Identifier

Provides unified interface to access resources. /Foos/ - A collection of resources /Foos/[Identifier]/ - An object instance inside a collection

Uses HTTP Verbs GET

Used to get content, no side effects are guaranteed POST

Used to update content PUT

Used to create/replace content DELETE

Used to delete content OPTIONS HEAD

Discoverable through OPTIONS and HEAD Verbs Allows request to refer to different representation through MIME-types Responds with HTTP Status Codes

Hands On – A Simple RESTful Web Service using ASP.NET MVC We’re going to write a simple blog application Why ASP.NET MVC?

Easy to implement Appropriate for small services

What are our entities? A Post A Tag A Comment A User

Both tags and comments entities are sub-entities of a post Both posts and comments entities are sub-entities of a user

Hands On – A Simple RESTful Web Service using ASP.NET MVC The URI Scheme:

To represent a resource collection of posts: /Posts/ To represent a specific post resource: /Posts/[id]/ To represent a resource collection of comments of a specific post resource: /Posts/[id]/Comments/ To represent a specific comment resource of a specific post resource: /Posts/[id]/Comments/[commentId] Well, you got the idea, right?

How to implement: Go to your Global.asax file Define the routes Use constrains to avoid routes clashes Use constrains to define allowed HTTP Verbs

Hands On – A Simple RESTful Web Service using ASP.NET MVC Implementing the URI Scheme:

public static void RegisterRoutes(RouteCollection routes) { // … routes.MapRoute(”DeleteComment”, ”Comments/{commentId}/”,

new { controller = ”Comments”, action = ”Delete” }, new { commentId = @”\d+”, httpMethod = new HttpMethodConstraint(”DELETE”) });

routes.MapRoute(”EditComment”, ”Comments/{commentId}/”, new { controller = ”Comments”, action = ”Edit” }, new { commentId = @”\d+”, httpMethod = new HttpMethodConstraint(”PUT”) });

 routes.MapRoute(”AddPostComment”, ”Posts/{postId}/Comments/”, new { controller = ”Comments”, action = ”Add” },

new { postId = @”\d+”, httpMethod = new HttpMethodConstraint(”POST”) });

 routes.MapRoute(”PostComment”, ”Posts/{postId}/Comments/{commentId}/”, new { controller = ”Comments”, action = ”PostComment” }, new { postId = @”\d+”, commentId = @”\d+”, httpMethod = new HttpMethodConstraint(”GET”) });

 routes.MapRoute(”PostComments”, ”Posts/{postId}/Comments/”, new { controller = ”Comments”, action = ”PostComments” }, new { postId = @”\d+”, httpMethod = new HttpMethodConstraint(”GET”) });

routes.MapRoute(”Comment”, ”Comments/{commentId}”, new { controller = ”Comments”, action = ”Comment” }, new { commentId = @”\d+”, httpMethod = new HttpMethodConstraint(”GET”) });

 routes.MapRoute(”UserComments”, ”Users/{user}/Comments/”, new { controller = ”Comments”, action = ”UserComments” }, new { user = @”^[a-zA-Z0-9_]*$”, httpMethod = new HttpMethodConstraint(”GET”) });

 routes.MapRoute(”UserComment”, ”Users/{user}/Comments/{commentId}/”, new { controller = ”Comments”, action = ”UserComment” },

  new { user = @”^[a-zA-Z0-9_]*$”, httpMethod = new HttpMethodConstraint(”GET”) }); // …}

Hands On – A Simple RESTful Web Service using ASP.NET MVC The URI Scheme - Explained:

Defines a route that maps to an action that allows to delete a comment commentId is limited to integral values (for more information, read about Regular

Expressions) You can reach to this action only by a DELETE request

public static void RegisterRoutes(RouteCollection routes) { // … routes.MapRoute(”DeleteComment”, ”Comments/{commentId}/”,

new { controller = ”Comments”, action = ”Delete” }, new { commentId = @”\d+”, httpMethod = new HttpMethodConstraint(”DELETE”) });

// …}

Hands On – A Simple RESTful Web Service using ASP.NET MVC The URI Scheme - Explained:

Defines a route that maps to an action that allows to edit a comment commentId is limited to integral values You can reach to this action only by a PUT request Why are we using PUT instead of POST?

The Comments/{commentId}/ URI refers to a specific item in a collection, since we are not sending only what is updated in our request we prefer to replace the resource instead of updating it

public static void RegisterRoutes(RouteCollection routes) { // … routes.MapRoute(”EditComment”, ”Comments/{commentId}/”,

new { controller = ”Comments”, action = ”Edit” }, new { commentId = @”\d+”, httpMethod = new HttpMethodConstraint(”PUT”) });

// …}

Hands On – A Simple RESTful Web Service using ASP.NET MVC The URI Scheme - Explained:

Defines a route that maps to an action that allows to add a comment to a post postId is limited to integral values You can reach to this action only by a POST request Why are we using POST instead of PUT?

The Posts/{postId}/Comments/ URI refers to a collection, PUT will replace all of our existing comments with a new one

POST only updates a resource, so a new comment is added to our existing comments collection

public static void RegisterRoutes(RouteCollection routes) { // … routes.MapRoute(”AddPostComment”, ”Posts/{postId}/Comments/”,

new { controller = ”Comments”, action = ”Add” }, new { postId = @”\d+”, httpMethod = new HttpMethodConstraint(”POST”) });

// …}

Hands On – A Simple RESTful Web Service using ASP.NET MVC The URI Scheme - Explained:

Those are the routes we are familiar with, the ones that use GET requests and most of the time return HTML

In this example, this route maps to an action that allows us to access a specific comment on a post

public static void RegisterRoutes(RouteCollection routes) { // … routes.MapRoute(”PostComment”, ”Posts/{postId}/Comments/{commentId}/”,

new { controller = ”Comments”, action = ”PostComment” }, new { postId = @”\d+”, commentId = @”\d+”, httpMethod = new HttpMethodConstraint(”GET”) });

// …}

Hands On – A Simple RESTful Web Service using ASP.NET MVC The comments controller:

Now that our routes are defined we know what actions we need to implement All actions should use valid HTTP Status Codes All actions must be able to represent themselves in different formats (JSON, XML, HTML, RSS etc.)

How to implement: Create a new controller called CommentsController Follow your routing scheme and create the needed actions Create a new action result decorator that allows you to select the HTTP Status code Use attributes to define allowed content types and HTTP Verbs

Hands On – A Simple RESTful Web Service using ASP.NET MVC Implementing the comments controller:

public class CommentsController : Controller {         // …        [AcceptVerbs(HttpVerbs.Delete)] // Or [HttpDelete] if you are using MVC 2 and above

public ActionResult Delete(int commentId)         {              // Delete operation occurs here

return new HttpResponseCodeActionResultDecorator(204); // 204 No content }

        [AcceptVerbs(HttpVerbs.Post)] // Or [HttpPost] if you are using MVC 2 and above public ActionResult Add(int postId)

        {         // Create a new comment that belongs to a post with a post Id of postId

return Json(new {CommentId = newCommentId}) ;         }

        [AcceptVerbs(HttpVerbs.Put)] // Or [HttpPut] if you are using MVC 2 and above public ActionResult Add(int commentId)

        {             // Create a new comment that belongs to a post with a post Id of postId

return new HttpResponseCodeActionResultDecorator(201, Json(new {CommentId = newCommentId}) ); // 201 - Created }

        // …}

Hands On – A Simple RESTful Web Service using ASP.NET MVC Implementing the HttpResponseCodeActionResultDecorator:

public class HttpResponseCodeActionResultDecorator : ActionResult {         private readonly int statusCode;         private readonly ActionResult actionResult;

        public HttpResponseCodeActionResultDecorator(int statusCode)         {             this.statusCode = statusCode;         }

        public HttpResponseCodeActionResultDecorator(int statusCode, ActionResult actionResult)             : this(statusCode)         {             this.actionResult = actionResult;         }

        public override void ExecuteResult(ControllerContext context)         {             context.HttpContext.Response.StatusCode = statusCode;

if (actionResult != null)             actionResult.ExecuteResult(context);

        } }

Hands On – A Simple RESTful Web Service using ASP.NET MVC Resources with multiple representations:

Our list of posts should be representable through RSS and Atom feeds but also display them as HTML. An HTTP request containing an Accept header with a value other than application/rss+xml,

application/atom+xml, text/html or application/xhtml+xml will return the HTTP response code “406 Not Acceptable”.

The implementation is not trivial An example of an implementation can be seen here: http://

aleembawany.com/2009/03/27/aspnet-mvc-create-easy-rest-api-with-json-and-xml however it does not conform to HTTP due to the fact that it will either return 404 or the default view.

How to implement: Create an ActionResult class that inspects the request’s Accept header and executes the requested

ActionResult or return 406 Not Acceptable

Hands On – A Simple RESTful Web Service using ASP.NET MVC Implementing the AcceptTypeResult:

public class AcceptTypeResult : ActionResult {         private readonly int? successStatusCode;

private readonly object result;

        public AcceptTypeResult(object result)         {            this.result = result;        }

        public AcceptTypeResult(int successStatusCode, object result) : this(result)         {            this.successStatusCode = successStatusCode;        }

        public override void ExecuteResult(ControllerContext context)         {

var request = context.HttpContext.Request;            context.HttpContext.Response.StatusCode = successStatusCode ?? 200;

if (request. AcceptTypes.Contains(“application/rss+xml”))Rss(result).ExecuteResult(context);

else if (request. AcceptTypes.Contains(“application/atom+xml”))Atom(result).ExecuteResult(context);

else if (request. AcceptTypes.Contains(“application/xhtml+xml”) || request.AcceptTypes.Contains(“text/html”))View(result).ExecuteResult(context);

else             context.HttpContext.Response.StatusCode = 406;        } }

Bibliography http://www.infoq.com/articles/webber-rest-workflow http://www.infoq.com/articles/mark-baker-hypermedia http://barelyenough.org/blog/2007/05/hypermedia-as-the-engine-of-application-state/ http://third-bit.com/blog/archives/1746.html http://www.learnxpress.com/create-restful-wcf-service-api-step-by-step-guide.html http://www.infoq.com/articles/rest-soap-when-to-use-each http://www.prescod.net/rest/ http://www.prescod.net/rest/rest_vs_soap_overview/ http://www.devx.com/DevX/Article/8155 http://tomayko.com/writings/rest-to-my-wife http://en.wikipedia.org/wiki/Representational_State_Transfer http://en.wikipedia.org/wiki/HATEOAS http://en.wikipedia.org/wiki/SOAP http://en.wikipedia.org/wiki/SOA http://en.wikipedia.org/wiki/Web_Application_Description_Language http://en.wikipedia.org/wiki/WSDL http://en.wikipedia.org/wiki/JSON-RPC http://en.wikipedia.org/wiki/XML-RPC http://www.sitepen.com/blog/2008/03/19/pluggable-web-services-with-smd/

For further reading http://jcalcote.wordpress.com/2009/08/10/restful-authentication/ http://jcalcote.wordpress.com/2009/08/06/restful-transactions/ http://www.artima.com/lejava/articles/why_put_and_delete.html http://www.markbaker.ca/2002/08/HowContainersWork/ http://www.w3.org/Protocols/rfc2616/rfc2616.html http://www.elharo.com/blog/software-development/web-development/2005/12/08/post-vs-put/ http://blog.whatfettle.com/2006/08/14/so-which-crud-operation-is-http-post/ http://cafe.elharo.com/web/why-rest-failed/ http://en.wikipedia.org/wiki/Windows_Communication_Foundation

Thank you for listening!