Towards Zero-Code Towards Zero-Code CompositionComposition

Emre KEmre Kııccııman, Laurence Melloul, Armando Foxman, Laurence Melloul, Armando Fox

Software Infrastructures Group, Stanford Software Infrastructures Group, Stanford UniversityUniversity

Zero-code CompositionZero-code Composition

Zero-Code == not writing any glue code to Zero-Code == not writing any glue code to compose services into a systemcompose services into a system

Zero-Code makes it easier to build static Zero-Code makes it easier to build static applicationsapplications Reuse of existing componentsReuse of existing components

No need to write code to adapt components to each otherNo need to write code to adapt components to each other

Zero-Code is necessary for building ad hoc and Zero-Code is necessary for building ad hoc and dynamic applicationsdynamic applications Ad hoc Ad hoc We don’t know what we need to build until we We don’t know what we need to build until we

need to build itneed to build it

Without zero-code composition, it takes too much time to Without zero-code composition, it takes too much time to build an application, ad hoc apps are not worth building…build an application, ad hoc apps are not worth building…

… … and Ubiquitous Computingand Ubiquitous Computing

Ubiquitous computing means:Ubiquitous computing means: Many heterogeneous devices (including legacy and Many heterogeneous devices (including legacy and

COTS devices)COTS devices)

Many services in the infrastructure, providing varied Many services in the infrastructure, providing varied and interesting functionality (but all developed and interesting functionality (but all developed separately, and not coordinated with one another)separately, and not coordinated with one another)

And these services and devices are all expected And these services and devices are all expected to work together!to work together!

A Zero-Code composition framework helps us A Zero-Code composition framework helps us to compose these devices and services to compose these devices and services together to perform taskstogether to perform tasks

Composition GuidelinesComposition Guidelines

General idea: Use loosely-coupled servicesGeneral idea: Use loosely-coupled services

Component frameworks today impede Component frameworks today impede compositioncomposition CORBA, JavaBeans, etc.CORBA, JavaBeans, etc.

The methods used to build components do not work The methods used to build components do not work well for composing components. well for composing components.

Two major types of impedimentsTwo major types of impediments Control impediments relate to ordering of execution Control impediments relate to ordering of execution

of components.of components.

Interface impediments occur when components have Interface impediments occur when components have dependencies on others’ interfaces.dependencies on others’ interfaces.

Control GuidelinesControl Guidelines

1.1. Make control flow between components explicitMake control flow between components explicit Difficult to analyze and manipulate compositions Difficult to analyze and manipulate compositions

because connections are spread out as function calls because connections are spread out as function calls across the codeacross the code

Explicitly specifying component interactions makes it Explicitly specifying component interactions makes it easier to understand and change the compositioneasier to understand and change the composition

2.2. Separate control flow between services from Separate control flow between services from the flow within a servicethe flow within a service Control model (assumptions about which components Control model (assumptions about which components

control sequencing of operations).control sequencing of operations).

Giving each service a separate locus of control. Only Giving each service a separate locus of control. Only dependency is data dependency dependency is data dependency use data flow use data flow network between componentsnetwork between components

Interface GuidelinesInterface Guidelines

1.1. Use named inputs and outputs, and a level of Use named inputs and outputs, and a level of indirection for component interactionindirection for component interaction APIs force components to know in advance the APIs force components to know in advance the

interface/method names of other componentsinterface/method names of other components By using a level of indirection, we can compose any By using a level of indirection, we can compose any

components without prior coordinationcomponents without prior coordination Example: Unix PipesExample: Unix Pipes

2.2. Use evolvable, self-describing type systemsUse evolvable, self-describing type systems Semantically compatible components cannot work Semantically compatible components cannot work

together because they do not agree on names of data together because they do not agree on names of data types types

By using self-describing type systems, we can get By using self-describing type systems, we can get necessary sub-elements of data structuresnecessary sub-elements of data structures

Example: Named tuples, XML, …Example: Named tuples, XML, …

Putting it TogetherPutting it Together

Components are loosely-coupled, interact Components are loosely-coupled, interact through an indirection of named, typed inputs through an indirection of named, typed inputs and outputsand outputs

Within a composition, services are connected Within a composition, services are connected via coarse-grained data flow networksvia coarse-grained data flow networks

A generic run-time system instantiates a A generic run-time system instantiates a composition based on an explicit description of composition based on an explicit description of the data flow networkthe data flow network

Data types are described using an evolvable, Data types are described using an evolvable, self-describing type systemself-describing type system

Paths PrototypePaths Prototype

Paths is a prototype zero-code composition Paths is a prototype zero-code composition framework that follows most of these framework that follows most of these guidelinesguidelines

To create a composition, one writes a To create a composition, one writes a description of the data flow network, description of the data flow network, specifying services, connectors, queues, and specifying services, connectors, queues, and the connections between them.the connections between them.

Web ServicesWeb Services

Web services already follow many of these guidelines:Web services already follow many of these guidelines:

Control: They are autonomous, have their own Control: They are autonomous, have their own separate loci of control, and do not expose their separate loci of control, and do not expose their internal control modelinternal control model

Interface: They are accessible through a common Interface: They are accessible through a common protocol (HTTP), and because web forms only protocol (HTTP), and because web forms only accept strings, there is no danger of artificial type accept strings, there is no danger of artificial type mismatchesmismatches

But…But…

There are semantic type mismatches, and no There are semantic type mismatches, and no standard method of passing data between standard method of passing data between services… services…

Exploiting CompositionExploiting Composition

How can we take advantage of composition?How can we take advantage of composition?

Building Ad Hoc applicationsBuilding Ad Hoc applications Application TemplatesApplication Templates

Providing more sophisticated Web ServicesProviding more sophisticated Web Services Customized services (e.g., comparison shopping with Customized services (e.g., comparison shopping with

user-specified services and products, or “here is my user-specified services and products, or “here is my profile, plan my evening”)profile, plan my evening”)

Subscription services (e.g., “let me know when a Subscription services (e.g., “let me know when a historian presents his/her new book in Downtown SF”) historian presents his/her new book in Downtown SF”)

Smart services (e.g., guess intelligent compositions of Smart services (e.g., guess intelligent compositions of random services put in a basket by the user)random services put in a basket by the user)

Application TemplatesApplication Templates

Application templates describe the data-flow Application templates describe the data-flow connections between a set of generic services.connections between a set of generic services.

Here is a Here is a simple simple template for template for data data aggregation.aggregation.

To build an To build an application, fill-application, fill-in the template in the template with specific with specific data source, data source, aggregator, aggregator, and formatter and formatter servicesservices

Application Templates (cont.)Application Templates (cont.)

Application templates ease automatic Application templates ease automatic application creationapplication creation It’s hard to decide how to connect arbitrary services It’s hard to decide how to connect arbitrary services

together—the template puts that decision into a together—the template puts that decision into a person’s handsperson’s hands

It’s easy to decide which services to compose togetherIt’s easy to decide which services to compose together—search for services based on service type, and rank —search for services based on service type, and rank them based on context, user preferences, etc.them based on context, user preferences, etc.

With zero-code composition, we don’t worry With zero-code composition, we don’t worry about mechanics of particular servicesabout mechanics of particular services Can use application template concept in your projects Can use application template concept in your projects

without zero-code comp. by standardizing your without zero-code comp. by standardizing your interfacesinterfaces

The WeSCoS SystemThe WeSCoS System

A general composition frameworkA general composition framework

Allows developers to add services and define Allows developers to add services and define composition requestscomposition requests

Relies on compact, descriptive XML service Relies on compact, descriptive XML service specificationspecification

Same interface used for composite and atomic Same interface used for composite and atomic services (transparent to the caller)services (transparent to the caller)

Once a specification is loaded into WeSCoS, Once a specification is loaded into WeSCoS, requests are automatically executed by the requests are automatically executed by the systemsystem

Architecture Walk-throughArchitecture Walk-through

A composition exampleA composition example Get restaurants nearby theaters playing “Men in Get restaurants nearby theaters playing “Men in

black”, within given zipcode areablack”, within given zipcode area

2 HTTP atomic services2 HTTP atomic services

1.1. Get movie showtime info, by theater, for given Get movie showtime info, by theater, for given zipcode (moviefone.com)zipcode (moviefone.com)

2.2. Get restaurant info, nearby given address Get restaurant info, nearby given address (mapquest.com)(mapquest.com)

For given zipcode, invoke 1; for each found theater, For given zipcode, invoke 1; for each found theater, extract its address and invoke 2extract its address and invoke 2

The WeSCoS ArchitectureThe WeSCoS Architecture

Web Service

Web Service

Http ServiceProcessor

ServiceRequest

Processor

Converters(xml/html,xml/wml)

CompositionEngine

HTTP

HTTP

XML data

HTML data

XML spec

XML spec

Service Processing Core

XML spec

Servicerequest

Requestresults

Database

User

Front-End

Request/Response document

The Service SpecificationThe Service Specification

A compact, XML-based documentA compact, XML-based document

Includes identifier, input/output schemas, and Includes identifier, input/output schemas, and execute nodeexecute node

The execute node contains invocation The execute node contains invocation information, useful to the composition systeminformation, useful to the composition system All HTTP invocation information (atomic service)All HTTP invocation information (atomic service)

Or all composition execution steps (composite Or all composition execution steps (composite service)service)

Recommended: 1 specification per Web Recommended: 1 specification per Web service functionalityservice functionality

A Service Specification ExampleA Service Specification Example

<service version = “1.0”><identifier>

<name>Get CD list by author</name><provider>Amazon.com</provider><version>1.0</version><description>Return the list of CDs, with title and price, given the author name </description>

</identifier><input-schema>

<element name=“field-keywords” desc=“authorName” type=“String”/></input-schema><output-schema>

<element name=“info” minOccurs=“0” maxOccurs=“unbounded”> <element name=“title” desc=“CDTitle” type=“String”/> <element name=“price” desc=“CDPrice” type=“float” minOccurs=“0”/></element>

</output-schema><execute method=“http”>…</execute>

</service>

The HTTP Execute NodeThe HTTP Execute Node

<execute method=“http” version=“1.0”> <url>http://www.amazon.com/exec/obidos/search-handle-form/102-0977834-7085735 </url> <method>post</method> <version>1.0</version> <parse method=“html>

<result> <![CDATA[<tr…<td…<a…>title</a>…~author<…Our Price:…$price<…</tr>]]> </result>

</parse></execute>

XPL: an XML Programming XPL: an XML Programming LanguageLanguage

Scripting language to define composition stepsScripting language to define composition steps

Syntactically compatible with specification Syntactically compatible with specification syntaxsyntax

Restricts composition instructions, for more Restricts composition instructions, for more control over the system behaviorcontrol over the system behavior

Could substitute other “integration” Could substitute other “integration” languages, e.g. Tcllanguages, e.g. Tcl

The Composition Execute NodeThe Composition Execute Node <execute method=“comp” version=“1.0”>

<invoke ref=“A”> <identification name=“Get CD list by author” provider=“Amazon.com”

version=“1.0”/> <param name=“field-keywords” source=“input/author”/></invoke><foreach ref=“res” data=“A/result”> <invoke ref=“B”>

<identification name=“Get CD info by title” provider=“Buy.com” version=“1.0”> <param name=“qu” separator=“ “> <value source=“res/title”/> <value source=“res/author”/> </param>

</invoke> <add element=“result”>

<add element=“author” source=“res/author”/> <add element=“title” source=“res/title”/> <add element=“price1” source=“res/price”/> <add element=“price2” source=“B/result/price”/>

</add></foreach>

</execute>

Prototype EvaluationPrototype Evaluation

+ Ease of service creation+ Ease of service creation

+ Ease of composition+ Ease of composition

+ Simple system implementation and + Simple system implementation and deploymentdeployment

Document exchange => easy, modular upgrade of the Document exchange => easy, modular upgrade of the composition systemcomposition system

- Naïve (and slow) implementation- Naïve (and slow) implementation

- Not so stable (see open challenges)- Not so stable (see open challenges)

Open ChallengesOpen Challenges CorrectnessCorrectness

Unstable servers (tracking changes): submit-and-Unstable servers (tracking changes): submit-and-scrape rules are fragile; plus independent evolutionscrape rules are fragile; plus independent evolution

Problem for deploymentProblem for deployment Semantic mappings are user’s responsibilitySemantic mappings are user’s responsibility

Multi-step interactionsMulti-step interactions e.g. multi-screen sites, authentication, etc.e.g. multi-screen sites, authentication, etc. May be able to help with tools such as WebLMay be able to help with tools such as WebL

Failure semantics: if one service in the “chain” Failure semantics: if one service in the “chain” fails...fails... Retry whole chain? Or only the failed service?Retry whole chain? Or only the failed service? What about side effects or non-idempotent services?What about side effects or non-idempotent services? Currently limited to “lookup-like” servicesCurrently limited to “lookup-like” services

Open Challenges (cont.)Open Challenges (cont.)

““Distributed systems challenges”Distributed systems challenges” No guarantee/control on service availability/quality; No guarantee/control on service availability/quality;

trusttrust

potentially low bandwidth and high latencypotentially low bandwidth and high latency

Time-out estimates (transparent nested Time-out estimates (transparent nested compositions)compositions)

““Fundamental” Challenge: Lack of support for Fundamental” Challenge: Lack of support for composition validation (due to lack of composition validation (due to lack of systematic typing)systematic typing)

Arguments are Arguments are namednamed, not necessarily typed, not necessarily typed Observation: human knowledge is generally Observation: human knowledge is generally

required at some level to “validate” compositionsrequired at some level to “validate” compositions

Related WorkRelated Work

CNET, comparison shopping => no CNET, comparison shopping => no composition frameworkcomposition framework

HP’s E-SPEAK => closest but not a zero-code HP’s E-SPEAK => closest but not a zero-code composition frameworkcomposition framework

MICROSOFT’s .NET?MICROSOFT’s .NET?

Arriba, Commerce One => specific service Arriba, Commerce One => specific service interface per application domaininterface per application domain

Future WorkFuture Work

Think a data flow model to specify compositionsThink a data flow model to specify compositions

Front-end to add and validate service Front-end to add and validate service specificationsspecifications

Interface converters for wireless devicesInterface converters for wireless devices

Take advantage of composition to provide Take advantage of composition to provide system-wide properties (dependability, etc.)system-wide properties (dependability, etc.) Time-outs & errors: retries can be specified as part of Time-outs & errors: retries can be specified as part of

the composition specificationthe composition specification

Availability vs. quality: less-quality, competitive Availability vs. quality: less-quality, competitive services can be used as backup services; again services can be used as backup services; again specified in the compositionspecified in the composition

ConclusionConclusion

Use loosely-coupled services:Use loosely-coupled services: Keep services autonomous, with flexible data types Keep services autonomous, with flexible data types

and named I/Oand named I/O

Use a data flow model to represent the compositionUse a data flow model to represent the composition

Question: Question: Will the benefits of Web Service Will the benefits of Web Service Composition outweigh its challenges?Composition outweigh its challenges?