Remote Procedure Calls and Web Services

Zachary G. IvesUniversity of Pennsylvania

CIS 455 / 555 – Internet and Web Systems

April 19, 2023

2

Today

Reminder HW2 Milestone 2 due tonight

HW3 “pre-release” today

What Does MapReduce Do Well?

What are its strengths?

What about weaknesses?

3

MapReduce is a ParticularProgramming Model

… But it’s not especially general (though things like Pig Latin improve it)

Suppose we have autonomous application components that wish to communicate

We’ve already seen a few strategies: Request/response from client to server

HTTP itself Asynchronous messages

Router “gossip” protocols P2P “finger tables”, etc.

Are there general mechanisms and principles?(Of course!)

… Let’s first look at what happens if we need in-order messaging

4

5

Communication Mechanisms

We’ve already seen a few: Request/response from client to server

HTTP itself Asynchronous messages

Router “gossip” protocols P2P “finger tables”, etc.

Are there general mechanisms and principles?(Of course!)

… Let’s first look at what happens if we need in-order messaging

6

Message-Queuing Model (1)

Four combinations for loosely-coupled communications using queues.

2-26

7

Message-Queuing Model (2)

Basic interface to a queue in a message-queuing system.

Primitive Meaning

Put Append a message to a specified queue

Get Block until the specified queue is nonempty, and remove the first message

Poll Check a specified queue for messages, and remove the first. Never block.

NotifyInstall a handler to be called when a message is put into the specified queue.

8

General Architecture of a Message-Queuing System (1)

The relationship between queue-level addressing and network-level addressing.

9

General Architecture of a Message-Queuing System (2)

The general organization of a message-queuing system with routers.

2-29

10

Benefits of Message Queueing

Allows both synchronous (blocking) and asynchronous (polling or event-driven) communication

Ensures messages are delivered (or at least readable) in the order received

The basis of many transactional systems e.g., Microsoft Message Queue (MMQ), IBM

MQseries, etc.

Some Common Modes of Building Distributed Applications

Data-intensive: XQuery (fetch XML from multiple sites, produce new XML)

Turing-complete functional programming language Good for Web Services; not much support for I/O, etc.

MapReduce (built over DHT or distributed file system) Single filter (map), followed by single aggregation (reduce) Languages over it: Sawzall, Pig Latin, Dryad, …

Message passing / request-response: e.g., over a DHT, sockets, or message queue

Communication via asynchronous messages Processing in message handler loop

Function calls: Remote procedure call / remote method invocation

11

12

Fully Synchronous Request/Response: Remote Procedure Calls

Remote procedure calls have been around forever, including: COM+ CORBA Java RMI The basic idea: put a function elsewhere in the system, call in

distributed fashion but using standard languages, methods

An RPC API defines a format for: Initiating a call on a particular server, generally in a reliable way Sending parameters (marshalling) to the server Receiving a return value, which may require marshalling as well

And an RPC call is synchronous (i.e., it generally blocks)

13

A Remote Procedure Call Visualized

time

working

server is busyre

ques

t

function server waits for req.

client blocks

RPCServer

RPCClient

14

How RPC Generally Works

You write an application with a series of functions One of these functions, F, will be distributed

remotely You call a “stub generator”

A caller stub emulates the function F: Opens a connection to the server Requests F, marshalling all parameters Receives F’s return status and parameters

A server stub emulates the caller: Receives a request for F with parameters Unmarshals the parameters, invokes F Takes F’s return status (e.g., protection fault), return value,

and marshals it back to the client

15

Passing Value Parameters

Steps involved in doing remote computation through RPC

2-8

16

RPC Components

Generally, you need to write: Your function, in a compatible language An interface definition, analogous to a C header file, so

other people can program for F without having its source

Generally, software will take the interface definition and generate the appropriate stubs

(In the case of Java, RMIC knows enough about Java to run directly on the source file)

The server stubs will generally run in some type of daemon process on the server Each function will need a globally unique name or

GUID

17

Parameter Passing Can Be TrickyBecause of References

The situation when passing an object by reference or by value.

2-18

18

What Are the Hard Problems with RPC? Esp. Inter-Language RPC?

Resolving different data formats between languages (e.g., Java vs. Fortran arrays)

Reliability, security

Finding remote procedures in the first place Extensibility/maintainability

(Some of these might look familiar from when we talked about data exchange!)

19

Web Services

Goal: provide an infrastructure for connecting components, building applications in a way similar to hyperlinks between data

It’s another distributed computing platform for the Web Goal: Internet-scale, language-independent, upwards-

compatible where possible

This one is based on many familiar concepts Standard protocols: HTTP Standard marshalling formats: XML-based, XML

Schemas All new data formats are XML-based

One Alternative: REST(Representational State Transfer)

Not really a standard – a style of development Data is represented in XML, e.g., with a

schema Function call interface uses URIs

Server is to be stateless

And the HTTP request type specifies the operation e.g., GET http://my.com/rest/service1 e.g., POST http://my.com/rest/service1 {body} adds

the body to the service

20

21

The “Standard” for Web Services: Three Parts

1. “Wire” / messaging protocols Data encodings, RPC calls or document

passing, etc.

2. Describing what goes on the wire Schemas for the data

3. “Service discovery” Means of finding web services

22

The Protocol Stacks of Web Services

Enhanced + expanded from a figure from IBM’s “Web Services Insider”,http://www-106.ibm.com/developerworks/webservices/library/ws-ref2/

Other extensions

SOAP Attachments

WS-Security

WS-AtomicTransaction,WS-Coordination

SOAP, XML-RPC

XML XML Schema

Service Description (WSDL)

Service Capabilities(WS-Capability)

MessageSequencing

Orchestration (WS-BPEL)

Inspection

Directory(UDDI)

Wire Format Stack Discovery StackDescription Stack

WS-Addressing

High-levelstate transition + msgingdiagramsbetween modules

23

Messaging Protocol: SOAP

Simple Object Access Protocol: XML-based format for passing parameters Has a SOAP header and body inside an envelope As a defined HTTP binding (POST with content-type of

application/soap+xml) A companion SOAP Attachments encapsulates other (MIME)

data

The header defines information about processing: encoding, signatures, etc.

It’s extensible, and there’s a special attribute called mustUnderstand that is attached to elements that must be supported by the callee

The body defines the actual application-defined data

24

A SOAP Envelope

<SOAP-ENV:Envelopexmlns:SOAP-ENV=“http://www.w3.org/2001/12/soap-envelope”xmlns:xsd=“http://www.w3.org/www.w3.org/2001/XMLSchema-instance”>

<SOAP-ENV:Header><t:Transaction xmlns:t=“www.mytrans.com” SOAP-ENV:mustUnderstand=“1” />

</SOAP-ENV:Header> <SOAP-ENV:Body>

<m:PlaceOrder xmlns:m=“www.somewhere/there”><orderno xsi:type=“xsd:string”>12</orderno>

</m:PlaceOrder> </SOAP-ENV:Body></SOAP-ENV: Envelope>

25

Making a SOAP Call

To execute a call to service PlaceOrder:

POST /PlaceOrder HTTP/1.1Host: my.server.comContent-Type: application/soap+xml;

charset=“utf-8”Content-Length: nnn

<SOAP-ENV:Envelope>…

</SOAP-ENV:Envelope>

26

SOAP Return Values

If successful, the SOAP response will generally be another SOAP message with the return data values, much like the request

If failure, the contents of the SOAP envelop will generally be a Fault message, along the lines of:

<SOAP-ENV:Body><SOAP-ENV:Fault xmlns=“mynamespace”>

<faultcode>SOAP-ENV:Client</faultcode><faultstring>Could not parse

message</faultstring>…

27

How Do We Declare Functions?

WSDL is the interface definition language for web services Defines notions of protocol bindings, ports, and

services Generally describes data types using XML

Schema

In CORBA, this was called an IDL In Java, the interface uses the same

language as the Java code

28

A WSDL Service

Service

Port Port Port

PortTypeOperation

Operation

PortTypeOperation

Operation

PortTypeOperation

Operation

Binding Binding Binding

29

Web Service Terminology

Service: the entire Web Service Port: maps a set of port types to a

transport binding (a protocol, frequently SOAP, COM, CORBA, …)

Port Type: abstract grouping of operations, i.e. a class

Operation: the type of operation – request/response, one-way Input message and output message; maybe

also fault message Types: the XML Schema type definitions

30

Example WSDL

<service name=“POService”><port binding=“my:POBinding”>

<soap:address location=“http://yyy:9000/POSvc”/></port>

</service><binding xmlns:my=“…” name=“POBinding”>

<soap:binding style=“rpc” transport=“http://www.w3.org/2001/...” />

<operation name=“POrder”><soap:operation soapAction=“POService/POBinding” style=“rpc” /><input name=“POrder”>

<soap:body use=“literal” … namespace=“POService” …/></input><output name=“POrderResult”>

<soap:body use=“literal” … namespace=“POService” …/></output>

</operation></binding>

31

JAX-RPC: Java and Web Services

To write JAX-RPC web service “endpoint”, you need two parts: An endpoint interface – this is basically like the

IDL statement An implementation class – your actual code

public interface BookQuote extends java.rmi.Remote {public float getBookPrice(String isbn) throws java.rmi.RemoteException;

}public class BookQuote_Impl_1 implements BookQuote {

public float getBookPrice(String isbn) { return 3.22; }}

32

Different Options for Calling

The conventional approach is to generate a stub, as in the RPC model described earlier

You can also dynamically generate the call to the remote interface, e.g., by looking up an interesting function to call

Finally, the “DII” (Dynamic Instance Invocation) method allows you to assemble the SOAP call on your own

33

Creating a Java Web Service

A compiler called wscompile is used to generate your WSDL file and stubs You need to start with a configuration file that

says something about the service you’re building and the interfaces that you’re converting into Web Services

34

Example Configuration File

<?xml version="1.0" encoding="UTF-8"?><configuration

xmlns="http://java.sun.com/xml/ns/jax- rpc/ri/config"><service name="StockQuote"

targetNamespace="http://example.com/stockquote.wsdl" typeNamespace="http://example.com/stockquote/types" packageName="stockqt">

<interface name="stockqt.StockQuoteProvider" servantName="stockqt.StockQuoteServiceImpl"/>

</service>

</configuration>

35

Starting a WAR

The Web Service version of a Java JAR file is a Web Archive, WAR

There’s a tool called wsdeploy that generates WAR files

Generally this will automatically be called from a build tool such as Ant

Finally, you may need to add the WAR file to the appropriate location in Apache Tomcat (or WebSphere, etc.) and enable it

See http://java.sun.com/developer/technicalArticles/WebServices/WSPack2/jaxrpc.html for a detailed example

36

Finding a Web Service

UDDI: Universal Description, Discovery, and Integration registry

Think of it as DNS for web services It’s a replicated database, hosted by IBM, HP,

SAP, MS

UDDI takes SOAP requests to add and query web service interface data

37

What’s in UDDI

White pages: Information about business names, contact info, Web site

name, etc.

Yellow pages: Types of businesses, locations, products Includes predefined taxonomies for location, industry, etc.

Green pages – what we probably care the most about: How to interact with business services; business process

definitions; etc Pointer to WSDL file(s) Unique ID for each service

38

Data Types in UDDI

businessEntity: top-level structure describing info about the business

businessService: name and description of a service

bindingTemplate: how to access the service tModel (t = type/technical): unique identifier

for each service-template specification publisherAssertion: describes relationship

between businessEntities (e.g., department, division)

39

Relationships between UDDI Structures

publisherAssertion

businessEntity

businessService bindingTemplate

tModel

n

2

1n

1 n

m

n

40

Example UDDI businessEntity<businessEntity businessKey=“0123…” xmlns=“urn:uddi-

org:api_v2”><discoveryURLs>

<discoveryURL useType=“businessEntity”>http://uddi.ibm.com/registery/uddiget?businessKey=0123 ...

</discoveryURL><name>My Books</name><description>Technical Book Wholesaler</description>…<businessServices>

…</businessServices><identifierBag>

<!– keyedReferences to tModels </identifierBag><categoryBag> … </categoryBag>

</businessEntity>

41

UDDI in Perspective

Original idea was that it would just organize itself in a way that people could find anything they wanted

Today UDDI is basically a very simple catalog of services, which can be queried with standard APIs It’s not clear that it really does what people

really want: they want to find services “like Y” or “that do Z”

42

The Problem: With UDDI and Plenty of Other Situations

There’s no universal, unambiguous way of describing “what I mean” Relational database idea of “normalization” doesn’t

convert concepts into some normal form – it just helps us cluster our concepts in meaningful ways

“Knowledge representation” tries to encode definitions clearly – but even then, much is up to interpretation

The best we can do: describe how things relate pollo = chicken = poulet = 雞 = 鸡 = jī = मु�र्गी� = murg Note that this mapping may be imprecise or situation-

specific! Calling someone a chicken, vs. a chicken that’s a bird

43

This Brings Us to XQuery,Whose Main Role Is to Relate XML

Suppose we define an XML schema for our target data and our source data

Can directly translate between XML schemas or structures Describes a relationship between two items

Transform 2 into 6 by “add 4” operation Convert from S1 to S2 by applying the query described by view V

Often, we don’t need to transfer all data – instead, we want to use the data at one source to help answer a query over another source…

44

Lazy Evaluation: A Virtual View

Source2.xml

Source1.xmlVirtual

XML doc.XQuery Query

Form

Browser/AppServer(s)

QueryResults

XQuery

Source2.xml

Source1.xmlComposedXQuery

HTMLXSLT

45

Let’s Look at Some SimpleMappings

Beginning with examples of using XQuery to convert from one schema to another, e.g., to import data

First: let’s review what our mappings need to accomplish…

46

Challenges of Mapping Schemas

In a perfect world, it would be easy to match up items from one schema with another Each element would have a simple correspondence to an

element in the other schema Every value would clearly map to a value in the other

schema

Real world: as with human languages, things don’t map clearly! Different decompositions into elements Different structures Tag name vs. value Values may not exactly correspond It may be unclear whether a value is the same

It’s a tough job, but often things can be mapped

47

Example Schemas

Bob’s Movie Database<movie> <title>…</title> <year>…</year> <director>…</director> <editor>…</editor> <star>…</star>*</movie>*

Mary’s Art List<workOfArt> <id>…</id> <type>…</type> <artist>…</artist> <subject>…</subject> <title>…</title></workOfArt>*

Want to map data from one schema to the other

48

Mapping Bob’s Movies Mary’s Art

Start with the schema of the output as a template:<workOfArt> <id>$i</id> <type>$y</type> <artist>$a</artist> <subject>$s</subject> <title>$t</title></workOfArt>

Then figure out where to find the values in the source, and create XPaths

49

The Final Schema Mapping

Mary’s Art Bob’s Moviesfor $m in doc(“movie.xml”)//movie, $a in $m/director/text(),

$i in $m/title/text(), $t in $m/title/text()return <workOfArt>

<id>$i</id> <type>movie</type> <artist>$a</artist> <title>$t</title></workOfArt>

Note the absence of subject…We had no reasonable source,so we are leaving it out.