Distributed Systems - CSE SERVICEScrystal.uta.edu/~elmasri/os2/slides/steen/ch9.pdf · Distributed Systems Principles and Paradigms Chapter 09 (version 27th November 2001) Maarten

Distributed SystemsPrinciples and Paradigms

Chapter 09(version 27th November 2001)

Maarten van Steen

Vrije Universiteit Amsterdam, Faculty of ScienceDept. Mathematics and Computer Science

Room R4.20. Tel: (020) 444 7784E-mail:[email protected], URL: www.cs.vu.nl/ � steen/

01 Introduction02 Communication03 Processes04 Naming05 Synchronization06 Consistency and Replication07 Fault Tolerance08 Security09 Distributed Object-Based Systems10 Distributed File Systems11 Distributed Document-Based Systems12 Distributed Coordination-Based Systems

00 – 1 /

Distributed Object-Based Systems

� CORBA

� DCOM

� Globe

09 – 1 Distributed Object-Based Systems/

CORBA

CORBA: Common Object Request Broker Architecture

Background:

� Developed by the Object Management Group(OMG) in response to industrial demands for object-based middleware

� Currently in version #2.4 with #3 (almost) done� CORBA is a specification: different implementa-

tions of CORBA exist� Very much the work of a committee: there are

over 800 members of the OMG and many of themhave a say in what CORBA should look like

Essence: CORBA provides a simple distributed-objectmodel, with specifications for many supporting ser-vices � it may be here to stay (for a couple of years)

09 – 2 Distributed Object-Based Systems/9.1 CORBA

CORBA Overview (1/2)

Client application

StaticIDL

proxy

DynamicInvocationInterface

Client ORB Server ORB

Skeleton DynamicSkeletonInterface

Objectadapter

Object implementation

ORBinterface

ORBinterface

Local OS Local OS

Client machine Server machine

Network

Object Request Broker (ORB): CORBA’s object bro-ker that connects clients, objects, and services

Proxy/Skeleton: Precompiled code that takes careof (un)marshaling invocations and results

Dynamic Invocation/Skeleton Interface (DII/DSI): Toallow clients to “construct” invocation requests atruntime instead of calling methods at a proxy, andhaving the server-side “reconstruct” those requestinto regular method invocations

Object adapter: Server-side code that handles incom-ing invocation requests.


CORBA Overview (2/2)

Interface repository: Database containing interfacedefinitions and which can be queried at runtime

Implementation repository: Database containing theimplementation (code, and possibly also state) ofobjects. Effectively: a server that can launch ob-ject servers.


CORBA Object Model

Essence: CORBA has a “traditional” remote-objectmodel in which an object residing at an object serveris remote accessible through proxies

Observation: All CORBA specifications are given bymeans of interface descriptions, expressed in an IDL.CORBA follows an interface-based approach to ob-jects:

� Not the objects, but interfaces are the really im-portant entities

� An object may implement one or more interfaces

� Interface descriptions can be stored in an inter-face repository, and looked up at runtime

� Mappings from IDL to specific programming arepart of the CORBA specification (languages in-clude C, C++, Smalltalk, Cobol, Ada, and Java.


CORBA ServicesService DescriptionCollection Facilities for grouping objects into lists, queue, sets, etc.

Query Facilities for querying collections of objects in a declara-tive manner

Concurrency Facilities to allow concurrent access to shared objects

Transaction Flat and nested transactions on method calls over multi-ple objects

Event Facilities for asynchronous communication throughevents

Notification Advanced facilities for event-based asynchronous com-munication

Externalization Facilities for marshaling and unmarshaling of objects

Life cycle Facilities for creation, deletion, copying, and moving ofobjects

Licensing Facilities for attaching a license to an object

Naming Facilities for systemwide naming of objects

Property Facilities for associating (attribute, value) pairs with ob-jects

Trading Facilities to publish and find the services an object hasto offer

Persistence Facilities for persistently storing objects

Relationship Facilities for expressing relationships between objects

Security Mechanisms for secure channels, authorization, and au-diting

Time Provides the current time within specified error margins


Communication Models (1/2)

Object invocations: CORBA distinguishes three dif-ferent forms of direct invocations:

Request type Failure sem. Description

Synchronous At-most-once Caller blocks

One-way Unreliable Nonblocking call

Deferred syn-chronous

At-most-once Nonblocking, but can pick-up results later

Event communication: There are also additional fa-cilities by means of event channels:

Event channelConsumer

Consumer

Supplier

Supplier

Supplier

Push event to consumers

Event channelConsumer

Consumer

Supplier

Supplier

Supplier

Ask suppliers for new event


Communication Models (2/2)

Messaging facilities: reliable asynchronous and per-sistent method invocations:

Clientproxy

Callbackinterface

ClientORB

Client application

2. Request to server

4. Call by the ORB

1. Call by theapplication

3. Response from server

Clientproxy

Pollinginterface

ClientORB

Client application

2. Request to server


3. Response from server



Processes

Most aspects of processes for in CORBA have beendiscussed in previous classes. What remains is theconcept of interceptors:

Clientproxy

ClientORB

Client application

Request-levelinterceptor

Message-levelinterceptor

To server

Local OS

Invocation request

Request-level: Allows you to modify invocation se-mantics (e.g., multicasting)

Message-level: Allows you to control message-passingbetween client and server (e.g., handle reliabilityand fragmentation)


Naming

Important: In CORBA, it is essential to distinguishspecification-level and implementation-level object ref-erences

Specification level: An object reference is consideredto be the same as a proxy for the referenced ob-ject � having an object reference means you candirectly invoke methods; there is no separate client-to-object binding phase

Implementation level: When a client gets an objectreference, the implementation ensures that, oneway or the other, a proxy for the referenced objectis placed in the client’s address space:

�� "!$#%��&('*),+�-�.0/21 35476�-�8(9�-�8;:=<$/?>�'"6@/BADCE�

Conclusion: Object references in CORBA used to behighly implementation dependent: different imple-mentations of CORBA could normally not exchangetheir references.09 – 10 Distributed Object-Based Systems/9.1 CORBA

Interoperable Object References(1/2)

Observation: Recognizing that object references areimplementation dependent, we need a separate refer-encing mechanism to cross ORB boundaries

Solution: Object references passed from one ORB toanother are transformed by the bridge through whichthey pass (different transformation schemes can beimplemented)

Observation: Passing an object reference ��

fromORB A to ORB B circumventing the A-to-B bridge maybe useless if ORB B doesn’t understand ��

��


Interoperable Object References(2/2)

Observation: To allow all kinds of different systemsto communicate, we standardize the reference that ispassed between bridges:

Repositoryidentifier

IIOPversion Host Port Object key Components

ProfileID

Tagged Profile

Objectidentifier

POAidentifier

Other server-specific information

Profile

Interoperable Object Reference (IOR)


Naming Service

Essence: CORBA’s naming service allows servers toassociate a name to an object reference, and haveclients subsequently bind to that object by resolvingits name

Observation: In most CORBA implementations, ob-ject references denote servers at specific hosts; nam-ing makes it easier to relocate objects

Observation: In the naming graph all nodes are ob-jects; there are no restrictions to binding names to ob-jects � CORBA allows arbitrary naming graphs

Question: How do you imagine cyclic name resolu-tion stops?

Observation: There is no single root; an initial con-text node is returned through a special call to the ORB.Also: the naming service can operate across differentORBs � interoperable naming service09 – 13 Distributed Object-Based Systems/9.1 CORBA

Fault Tolerance

Essence: Mask failures through replication, by puttingobjects into object groups. Object groups are trans-parent to clients: they appear as “normal” objects.This approach requires a separate type of object ref-erence: Interoperable Object Group Reference:

Repositoryidentifier

IIOPver.

IIOPver.Host-1 Host-NPort-1 Port-N

Objectkey-1

Objectkey-NComponents Components

ProfileID

ProfileID

TAGPRIMARY

TAGBACKUP

Other group-specific information

Other group-specific information

Profile-1 Profile-N

Interoperable Object Group Reference (IOGR)

Note: IOGRs have the same structure as IORs; themain difference is that they are used differently. InIORs an additional profile is used as an alternative; inIOGR, it denotes another replica.


Security

Essence: Allow the client and object to be mostly un-aware of all the security policies, except perhaps atbinding time; the ORB does the rest. Specific poli-cies are passed to the ORB as (local) objects and areinvoked when necessary:

Client application Object implementation

Policyobject

Policyobject

Securityservice

Securityservice

Policyobject

Policyobject

Securityservice

Securityservice

Set ofclient-specificpolicy objects

Set ofobject-specificpolicy objects

Set of relevantORB security

servicesLocal OS Local OS

Client ORB Server ORB

NetworkInvocation

Examples: Type of message protection, lists of trustedparties.


Distributed COM

DCOM: Distributed Component Object Model

� Microsoft’s solution to establishing inter-processcommunication, possibly across machine bound-aries.

� Supports a primitive notion of distributed objects

� Evolved from early Windows versions to currentNT-based systems (including Windows 2000)

� Comparable to CORBA’s object request broker

09 – 16 Distributed Object-Based Systems/9.2 Distributed COM

DCOM Overview (1/2)

Somewhat confused? DCOM is related to many thingsthat have been introduced by Microsoft in the pastcouple of years:

In-placeediting

Dragand drop

Interprocessdata transfer

Persistentstorage

Persistentreferences

Documentlinking

Objectactivation

ScriptingGrouping(Controls)Documents

Core COM library

Embedding

COM

OLE

ActiveX

DCOM: Adds facilities to communicate across pro-cess and machine boundaries.


DCOM Overview (2/2)

Classobject ObjectSCM SCM

Microsoft RPC

Clientproxy

Objectstub

Proxymarshaler

Proxymarshaler

Client application

COM COM

Local OS Local OS

Network

Client machine Object server

Registry Registry

SCM: Service Control Manager, responsible for ac-tivating objects (cf., to CORBA’s implementationrepository).

Proxy marshaler: handles the way that object refer-ences are passed between different machines


COM Object Model

� An interface is a collection of semantically relatedoperations

� Each interface is typed, and therefore has a glob-ally unique interface identifier

� A client always requests an implementation of aninterface:

– Locate a class that implements the interface

– Instantiate that class, i.e., create an object

– Throw the object away when the client is done


DCOM ServicesCORBA DCOM/COM+ Windows 2000Collection ActiveX Data Objects –Query None –Concurrency Thread concurrency –Transaction COM+ Automatic

TransactionsDistributed Transac-tion Coordinator

Event COM+ Events –Notification COM+ Events –Externalization Marshaling utilities –Life cycle Class factories, JIT

activation–

Licensing Special class facto-ries

–

Naming Monikers Active DirectoryProperty None Active DirectoryTrading None Active DirectoryPersistence Structured storage Database accessRelationship None Database accessSecurity Authorization SSL, KerberosTime None None

Note: COM+ is effectively COM plus services thatwere previously available in an ad-hoc fashion


Communication Models

Object invocations: Synchronous remote-method callswith at-most-once semantics. Asynchronous invoca-tions are supported through a polling model, as inCORBA.

Event communication: Similar to CORBA’s push-style model:

Supplier Consumer

Consumer

Event classobject

Eventobject

m_event

m_event

Eventstore

Interfacecontainingm_event

Invocationis stored

Invocationis passed

to consumer

Objectimplementing

m_event

Messaging: Completely analogous to CORBA mes-saging.


Processes: Passing ObjectReferences

Observation: Objects are referenced by means ofa local interface pointer. The question is how suchpointers can be passed between different machines:

Clientproxy

Clientproxy

Proxy(un)marshaler

Proxy(un)marshaler

Process A Process B

Client application Client applicationMarshaledclient proxy

Objectstub

Object

Object server

Binding information Same bindinginformation

Question: Where does the proxy marshaler comefrom? Do we always need it?


Naming: Monikers

Observation: DCOM can handle only objects as tem-porary instances of a class. To accommodate objectsthat can outlive their client, something else is needed.

Moniker: A hack to support real objects

� A moniker associates data (e.g., a file), with anapplication or program

� Monikers can be stored� A moniker can contain a binding protocol, spec-

ifying how the associated program should be“launched” with respect to the data.

1 Client Calls BindMoniker at moniker2 Moniker Lookup CLSID and tell SCM to create object3 SCM Loads class object4 Class object Creates object, returns int. pointer5 Moniker Instructs object to load previously stored state6 Object Loads its state from file7 Moniker Returns interface pointer of object to client


Active Directory

Essence: a worldwide distributed directory service,but one that does not provide location transparency.

Basics: Associate a directory service (called domaincontroller) with each domain; look up the controllerusing a normal DNS query:

DNS

DomaincontrollerClient

1. Ask for host addressof domain controllerin a given domain

2. Requested address

3. LDAP query

4. LDAP reply

Note: Controller is implemented as an LDAP server


Fault Tolerance

Automatic transactions: Each class object (from whichobjects are created), has a transaction attribute thatdetermines how its objects behave as part of a trans-action:

Attr. value DescriptionREQUIRES NEW A new transaction is always started at

each invocationREQUIRED A new transaction is started if not al-

ready done soSUPPORTED Join a transaction only if caller is al-

ready part of oneNOT SUPPORTED Never join a transaction (no transaction

support)DISABLED Never join a transaction, even if told to

do so

Note: Transactions are essentially executed at thelevel of a method invocation.


Security (1/2)

Declarative security: Register per object what thesystem should enforce with respect to authentication.Authentication is associated with users and user groups.There are different authentication levels:

Auth. level DescriptionNONE No authentication is requiredCONNECT Authenticate client when first con-

nected to serverCALL Authenticate client at each invocationPACKET Authenticate all data packetsPACKET INTEGRITY Authenticate data packets and do in-

tegrity checkPACKET PRIVACY Authenticate, integrity-check, and en-

crypt data packets


Security (2/2)

Delegation: A server can impersonate a client de-pending on a level:

Impersonation DescriptionANONYMOUS The client is completely anonymous to

the serverIDENTIFY The server knows the client and can do

access control checksIMPERSONATE The server can invoke local objects on

behalf of the clientDELEGATE The server can invoke remote objects

on behalf of the client

Note: There is also support for programmatic secu-rity by which security levels can be set by an appli-cation, as well as the required security services (seebook).


Globe

� Experimental wide-area system currently being de-veloped at Vrije Universiteit

� Unique for its focus on scalability by means oftruly distributed objects

� Prototype version up and running across multi-ple machines distributed in NL and across Europeand the US.

09 – 28 Distributed Object-Based Systems/Globe

Object Model (1/3)

Essence: A Globe object is a physically distributedshared object: the object’s state may be physicallydistributed across several machines

Local object

Distributed shared object

Process

Interface

Network

Local object: A nondistributed object residing a sin-gle address space, often representing a distributedshared object

Contact point: A point where clients can contact thedistributed object; each contact point is describedthrough a contact address


Object Model (2/3)

Observation: Globe attempts to separate function-ality from distribution by distinguishing different localsubobjects:

Controlsubobject

Replicationsubobject

Semanticssubobject

Communicationsubobject

Communicationwith other local

objects

Same interface as implementedby semantics subobject

Semantics subobject: Contains the methods that im-plement the functionality of the distributed sharedobject

Communication subobject: Provides a (relatively sim-ple), network-independent interface for communi-cation between local objects


Object Model (3/3)

Controlsubobject

Replicationsubobject

Semanticssubobject

Communicationsubobject

Communicationwith other local

objects

Same interface as implementedby semantics subobject

Replication subobject: Contains the implementationof an object-specific consistency protocol thatcontrols exactly when a method on the semanticssubobject may be invoked

Control subobject: Connects the user-defined inter-faces of the semantics subobject to the generic,predefined interfaces of the replication subobject


Client-to-Object Binding

Client Naming service

Location service

1. Name

Object handle2. Object handle

Addresses &protocols

Register contactaddress

Local object

5. Make contact

(Trusted)class repository

Distributed shared object

Class object

4. Load & instantiateclass(es)

3. Selectaddress

Observation: Globe’s contact addresses correspondto CORBA’s object references


Globe ServicesService Possible implementation Av?Collection Separate object that holds refer-

ences to other objectsNo

Concurrency Each object implements its ownconcurrency control strategy

No

Transaction Separate object representing atransaction manager

No

Event/Notif. Separate object per group ofevents (as in DCOM)

No

Externalization Each object implements its ownmarshaling routines

Yes

Life cycle Separate class objects combinedwith per-object implementations

Yes

Licensing Implemented by each object sepa-rately

No

Naming Separate service, implemented bya collection of naming objects

Yes

Property Separate service, implemented bya collection of directory objects

No

Persistence Implemented on a per-object basis YesSecurity Implemented per object, combined

with (local) security servicesYes

Replication Implemented on a per-object basis YesFault tolerance Implemented per object combined

with fault-tolerant serversYes


Object References

Essence: Globe uses location-independent object han-dles which are to be resolved to contact addresses(which describes where and how an object can becontacted):

� Associated with a contact point of the distributedobject

� Specifies (for example) a transport-level networkaddress to which the object will listen

� Contains an implementation handle, specifyingexactly what the client should implement if it wantsto communicate through the contact point:–

��

�� ! "��#�$�%��&''

''(�*),+-�� '. +/�0)21 � �#+-�3)!�,4�) �

4&�#+5�6)21�879)

�

– “slave/master-slave/tcp/ip”

Observation: Objects in Globe have their own object-specific implementations; there is no “standard” proxythat is implemented for all clients


Naming Objects

Observation: Globe separates naming from locatingobjects (as described in Chapter 04). The currentnaming service is based on DNS, using TXT recordsfor storing object handles

Observation: The location service is implemented asa generic, hierarchical tree, similar to the approachexplained in Chapter 04.


Caching and Replication

Observation: Here’s where Globe differs from manyother systems:

� The organization of a local object is such thatreplication is inherently part of each distributedshared object

� All replication subobjects have the same interface:

Method Description� ��0�

Called to synchronize replicas of the se-mantics subobjects, obtain locks if neces-sary, etc.

� ��!Provide marshaled arguments of a specificmethod, and pass invocation to local ob-jects in other address spaces� ��E��!Called after the control subobject has in-voked a specific method at the semanticssubobject

� This approach allows to implement any object-specific caching/replication strategy


Security

Essence: Additional security subobject checks for au-thorized communication, invocation, and parameter val-ues. Globe can be integrated with existing securityservices:

Kerberos

Process B

Process A

Local object A

Local object B

Principal objectof process B

Principal objectof process A

1 2

34

5

6

7

Security domain of B

Security domain of A

Distributedsharedobject


Comparison


Issue CORBA DCOM GlobeDesign goals Interoperability Functionality ScalabilityObject model Remote objects Remote objects Distributed objectsServices Many of its own From environment FewInterfaces IDL based Binary BinarySync. communication Yes Yes YesAsync. communication Yes Yes NoCallbacks Yes Yes NoEvents Yes Yes NoMessaging Yes Yes NoObject server Flexible (POA) Hard-coded Object dependentDirectory service Yes Yes NoTrading service Yes No NoNaming service Yes Yes YesLocation service No No YesObject reference Object’s location Interface pointer True identifierSynchronization Transactions Transactions Only intra-objectReplication support Separate server None Separate subobjectTransactions Yes Yes NoFault tolerance By replication By transactions By replicationRecovery support Yes By transactions No