An SoA for a massively scalable next-gen ILS - principles and practice.

Paul Harvey <Paul.Harvey@oclc.org>

OCLC

Web-Scale Library Management ServicesWeb-Scale Library Management ServicesELAG 2011

Web-Scale Library Management ServicesWeb-Scale Library Management Services

Þ What is the WMS?

Þ Key Architectural Goals

Þ SoA Service Groups

Þ Standard Service Group Architecture

Þ Circulation Service Group

Þ Modular Staff Interface

Þ Service Directory

Þ Scaling, Responsiveness and Fault Tolerance

Þ The Complete Picture

What is the WMS? What is the WMS?

Þ A Globally Available, Centrally/Cloud Deployed, Web Based, Next-Gen Integrated Library System

Þ Operates at Web Scale

Þ Functionality

Þ Circulation, Acquisitions, Cataloguing, License Management, User Management/IDM, OPAC

Þ Physical and Electronic Materials

ÞThe Cooperative Open Platform

Infrastructure

WCL

DublinCirculation

Workflow

LMAN

Dublin

IDM

SM

Configuration

Dublin

Acquisitions

Vendor Info

Key Architectural ImperativesKey Architectural Imperatives

Þ Responsive

Þ Massively scalable

Þ Highly fault tolerant

Þ Flexible

Þ Extensible and Integratable

Þ Hosted - SaaS and PaaS

Þ Builds on existing OCLC data and software

How we are Achieving our Goals?How we are Achieving our Goals?

Þ Service Orientated (SoA)

Þ “Shared nothing” architecture

Þ Replication and failover

Þ Intra and Inter data centre

Þ Stateless services

Þ Partition by domain and data

Þ Avoid distributed transactions

Þ Optimistic locking

Þ Network savvy APIs

Þ Versioned services/APIs

Þ Judicious Caching

Þ Embrace open standards

Þ Layered

Þ Multi-tenanted

Þ Discoverable services

Þ Organisationally and technically

Þ Service abstraction models business function

How we are Achieving Our Goals?How we are Achieving Our Goals?

Þ Loosely coupled/independent services

Þ Cohesive and composable services

Þ Common infrastructure services

Þ Common technological platform

Þ Standard meta-protocol

Þ Common architecture framework and archetypes

Þ Only one tool/service for one job

Þ Data redundancy

Þ Allow (temporary) data inconsistency

Þ Asynchrony

Þ Temporal De-coupling

Þ Failure De-coupling

SoA Service Groups - Dividing by FunctionSoA Service Groups - Dividing by Function

Dom

ain

Acquisitions Circulation

Vendor Information

License Manageme

ntUser DataKBWC

ConnexionResource Discovery

Business Intelligence

WCLModular

Staff Interface

Inte

gra

tion

UI Cookbook

Notification

Infr

astr

uctu

re

AuthN AuthZRegistry

ProxyCollection Service

Asynchronous Updating B2B

Service Directory

WorkflowHardware

Integration

Managed Storage

Auditing Fiji XServe

Service Directory

Modular Staff

Interface

Circulation17 Service

Group Types

69 Service Types

18 Data Models

Layering and Standard Service Group ArchitectureLayering and Standard Service Group Architecture

Web UIOpen Platform

Proxy

Business Layer

Database

DAO

Þ Public facing serviceÞ Authentication

(sometimes)Þ De facto and De jure

protocolsÞ AtomPub/JSON/XMLÞ May aggregate other

services

Þ Provides user interfaces

Þ Authentication

Þ The core business logic!Þ Well defined APIÞ Non-invasive fast binary

protocolÞ Where authorization is

enforced

Þ Abstracts away the database

Þ Presents CRUD type operations

Þ No business logicÞ Enforces integrity

constraintsÞ Database agnostic

Þ Needs in itself to be reliable

Þ Needs to scale to the number of partitions it holds! • Standard technology

stack• Standard usage

guidelines

MSIExternal Callers

Circulation Service Group as an ExampleCirculation Service Group as an Example

Circulation Business Service

MSI

Circulation Database

Circulation UI

IDM AuthZ

IDM AuthN

Registry

Registry Proxy

“WorldCat” Union

Cat

NCIP

WCL

Availability Service

Asynchronous Sync

Modular Staff InterfaceModular Staff Interface

Panel Manager

Accordion Manager

Card Manager

• Sovereign Application

• Configuration driven

• All client apps are equal

• Handles “SSO”

• Loose coupling

• Via Events

• Managers

• Open to new applications

• Highly scalable

• Very thin proxy

• Client side weaving

• Common look and feel

• The UI Cookbook

Acquisitions Context

Menu

...

Acquisitions UI

VIC UI

...

Top Menu

Circulation UICirculation

Context Menu

Top Menu

Circulation Context

Menu

Circulation UI

The Service DirectoryThe Service Directory

Service Directory

Point of Service

Rendezvous

Service

Version

Institution

Url

Circulation

1.2

Sheffield City

https://.../

Service

Version

Institution

Url

Accounts

1.0

Bibsys

https://.../

Service

Version

Institution

Url

Acquisitions

1.1

Sheffield City

https://..

./

Service

Version

Institution

Url

Acquisitions

1.1

Sheffield City

https://..

./

Service

Version

Institution

Url

Circulation

1.1

OHIO State

https://.../

Service

Version

Institution

Url

Acquisitions

1.1

Sheffield City

https://.../

Service

Version

Institution

Url

Acquisitions

1.1

Sheffield City

https://..

./

Service

Version

Institution

Url

Acquisitions

1.1

Sheffield City

https://..

./

Service

Version

Institution

Url

Acquisitions

1.1

Sheffield City

https://..

./

Service

Version

Institution

Url

Acquisitions

1.1

Sheffield City

https://..

./

Service

Version

Institution

Url

Acquisitions UI

1.1

Sheffield City

https://.../

Service

Version

Institution

Url

Acquisitions

1.1

Sheffield City

https://..

./

The Service DirectoryThe Service Directory

Service Directory

Service

Version

Institution

Url

Acquisitions

1.1

Sheffield City

https://.../

Service

Version

Institution

Url

Acquisitions

1.1

Sheffield City

https://.../

Service

Version

Institution

Url

Acquisitions

1.1

Sheffield City

https://.../

Service

Version

Institution

Url

Acquisitions

1.1

Sheffield City

https://.../

Service

Version

Institution

Url

Acquisitions

1.1

Sheffield City

https://.../

Service

Version

Institution

Url

Acquisitions

1.1

Sheffield City

https://.../

Service

Version

Institution

Url

Acquisitions

1.1

Sheffield City

https://.../

Service

Version

Institution

Url

Acquisitions

UI

1.1

Sheffield City

https://.../

Service = “Acquisitions” && Version = “1.2” && Institution = “Sheffield City”

?

Scaling, Responsiveness and Fault ToleranceScaling, Responsiveness and Fault Tolerance

WMS Server

I

I

II

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

?$I

I

I

I

I

I

I

I

I

I

WMS Server

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

II

I

I

I

I

II

I

I

I

I

II

I

I

I

I

II

I

I

I

I

II

I

I

I

I

II

I

I

I

I

II

Partitions/Fault Tolerance

Vert

ical

Horizontal

The Complete PictureThe Complete Picture

DM

ZIn

tern

al N

etw

ork

UI

Platform

Business

Load-balance, fail-over

Layers

Circulation Acquisitions LicenseManagement

I1, I2, I3

I1, I2, I3

I1, I2, I3

I1, I2, I3

I1, I2, I3

I1, I2, I3

Domains

Database

“Data” Partition Serving I1, I2 and I3

\

I1, I2, I3 I1, I2, I3 I1, I2, I3

Load-balance, fail-over and routing

Load-balance, fail-over

Web-Scale Library Management ServicesWeb-Scale Library Management Services

?www.oclc.org/info/elag11