1 Topics in Distributed Systems CS 5580F. ©2005 D. J. Foreman 2 Topics 1. Course requirements 2. Introduction 3. Communications 4. Naming 5. Synchronization

1

Topics in Distributed Systems

CS 5580F

©2005 D. J. Foreman

2

Topics

1. Course requirements

2. Introduction

3. Communications

4. Naming

5. Synchronization

6. Consistency & Replication

7. Fault Tolerance

8. Security


3

Course Requirements


4

Grading, etc

Paper 50% Presentation 15% Programming project 35% Scale:

90-93.9=A- 94-100=A 80-83.9=B- 84-86.9=B 87-89.9=B+ 70-73.9=C- 74-76.9=C 77-79.9=C+ 0-59=F 60-69=D


5

Student Responsibilities

Read the suggested papers Complete the required programming project Select a topic for presentation Write a paper Present the paper


6

Programming Project

Design & implement a distributed application Project must be approved before beginning Design criteria

Ignore: System outages Network failures (e.g., partitioning)

Consider: Communication type (C/S, P-P) Groups, multi-casting

Teams: see instructor


7

Sample Projects

An "ATM machine" A remote file access mechanism A database A calculator A chat room A computing "grid" (like SETI) Or ???????? (you pick one)


8

Semester Project – continued

System types: Client/Server Peer-Peer Processor Pool

All must have: Multi-threaded operation User interface (need not be graphical) Point(s) of control (any system shuts-down all)

Must be demonstrated before end of semester allowed time depends on # of projects in class


9

Introduction


10

Introductory Papers

The papers listed in the following section, while dated in the early 1970s and 80's, form some of the foundations of contemporary implementations and research.


11

Basic Concepts

Synchronizing Resources, G. Andrews, 1981 Specification of Synchronizing Processes, K.

Ramanathan, 1983 Concepts and Notations for Concurrent

Programming, G. Andrews, 1983 Distributed Operating Systems, A.

Tanenbaum, 1985


12

Programming & Concept Papers

The Temporal Semantics of Concurrent Programs, A. Pneuli, 1981

Time, Clocks, and the Ordering of Events ina D.S., L. Lamport, 1978

Determining the Last Process to Fail, D. Skeen, 1985

Correctness Proofs of D. Termination Algorithms, K. Apt, 1986

Monitors: AN OS Structuring Concept, C.A.R. Hoare, 1974


13

Actual System Papers

An Introduction to the V System, E. Berglund, 1986

The Design of the Saguaro D.O.S., G. Andrews,1987

XMS: A Rendezvous-based D. S. Software Architecture, N. Gammage, 1985


14

Definition of a Distributed System

A collection of functions implemented across a set of independent devices that appears to its users as a single coherent system.

Horizontal - all devices are peers Vertical - there is a control hierarchy

Note: Devices may be computers or ….


15

Some Problems (topics)

System Character codes (ASCII/EBCDIC) Number representation (big- vs. little-Endian)

Size of an "int" (16/32/64 - bit) Float

Shared Memory Operational Compatibility

Thread control System services Process handling

Pointers & References Objects


16

Hardware Systems

Multiprocessors (modularized shared memory) Bus traffic cache incoherence Limited scalability (network cost/speed) NUMA – Non-Uniform Memory Access

(some local, some shared via network)

Homogeneous Multicomputers Private memory + switching circuits Bus-based connection: ethernet routing Switch-based: routing via h/w net (mesh, n-cube)

e.g., Clusters of Workstations

Heterogeneous Multicomputers No commonality, no global view


17

Software

Tightly coupled (true "Distributed System") Maintains global view

Multiprocessor (one O/S)Multicomputer (homogeneous only)

Loosely coupled (NOS) Each heterogeneous computer runs its own O/S Not much transparency as seen by applications BUT: Local services are shared on the network


18

Some Classical Distributed Systems

Locus – distributed UNIX Lightweight kernel systems w/UNIX emulation

Amoeba – groups of threads (processor pool) Mach – synch msg-based IPC, threads

Argus ('79) – "Guardians" for transactions Clouds – object-based (like methods) V-system (1984)– IPC via kernel (msgs only),

true multi-threading (unavailable in UNIX '84) Cedar - µcoded VM, IDE, concurrent execution


19

Classical Systems, continued

XDFS - ca. 1977 - servers Cambridge File Server – ca.1982 - indexes on

servers used by OS to build its directory Sun NFS – networked UNIX FS


20

Open vs Closed Systems

Open Extensible Services external to kernel

Closed Non-extensible Services internal to kernel Acts as server to remote requestors

Services are: File system Process mgmt Network communications


21

Atomicity

Semaphores Unstructured – requires app to test/signal

Monitors A programming construct

Shared variablesMutexesProcedures

lock and unlock functions (may be kernel-provided)

Threads (Pthreads, POSIX threads) Butenhof, "Programming with POSIX threads", AW Lewis & Berg, "Multithreaded Programming with

Pthreads", PTR-PH


22

Memory Problems

a. Little Endian (Intel): lowest # byte = highest value

b. Big Endian (Sun): highest # byte = highest value Receiver must know data types

Original(Sun)

as received(Windows)

improper correction

5*2245


23

Number Representations

What is an "int" 16 bits??? 32 bits???

FP number storage Many mechanisms, depending on h/w Less portable than integers


24

An example using int's

32-bit compiler, treats "int" as 32 bitsint x; x=1; // x is 00000000 00000000 00000000 00000001

Y(x); // all 32 bits ---> 16-bit target system Target of call compiled on 16-bit system

function Y (int val) // "int" is 16 bits{print val;} // prints 0 ---- WHY???? ---

'Y' only gets the 1st 16 bits of x because it doesn't KNOW there are more to get !!! ---

WHY??? --

Same problem for 32-bit 64-bit calls and whatever comes next. Remember Y2K!


25

Operational Compatibility

Thread controls Pthreads on Windows (compatible w/POSIX) POSIX threads

seen on Linux & Solaris MS threads Solaris threads others (OS/2, mainframe)

System services (commands, signals, file manipulation)

Process handling (creation, deletion) Communication (IPC: msgs vs pipes)


26

Memory


27

Shared Memory

Inhibits distribution (coherency lost, cost to build) When NOT shared, problems arise:

Consistency Access (speed, mechanisms) Compatibility (endian) etc.

Atomicity Program level

Compare & Swap Test & OP (add, subtract, set-bit)

Kernel/library level Semaphores, Mutexes, Condition variables


28

Distributed Memory

Partial solutions for heterogeneous systems Small local (non-shared) memory Bus and caches to a main store Bus and caches to each processor

Full solutions for multiprocessors Interconnection (h/w switching) networks

Limited scalability VERY expensive, but VERY fast


29

Communications

2


30

Methods

Shared variables - coherence?? Message passing

RPC – Remote Procedure Call Very good for Client-server Hides message passing

RMI – Remote Method Invocation (also Remote Object Invocation)

Like RPC, but with system-wide objects Java server implements a Security Manager & registry

Very much like DCE MOM – Message Oriented Middleware

Streams (TCP/IP) Needed for continuous information flow (video, audio)


31

RPC Steps

1. Client procedure calls client stub in normal way

2. Client stub builds message, calls local OS

3. Client's OS sends message to remote OS

4. Remote OS gives message to server stub

5. Server stub unpacks parameters, calls server

6. Server does work, returns result to the stub

7. Server stub packs it in message, calls local OS

8. Server's OS sends message to client's OS

9. Client's OS gives message to client stub

10.Stub unpacks result, returns to client

2


32

Passing Value Parameters

a) Original message on the PCb) The message after receipt on a SPARC WSc) The message after being inverted. Numbers in

dotted-boxes indicate the address of each byte

2


33

Berkeley Sockets (1)

Delete the connectionClose

Input dataReceive

Output dataSend

Allow a specific connection request to an ethereal port, selected by the O/S, freeing the "listen" port

Accept

Signal availability for connection requestsListen

Assign a "handle" to the socketBind

Create an endpoint from a connectionSocket

Basic TCP/IP 'commands'

2


34

Connection-oriented communication pattern using sockets. Note the ordering of Reads & Writes vs time.

Berkeley Sockets: Client-Server

S o c ket B ind L is ten A c c ep t R ead W rite C lo s e

S e rve r

C lien t

S o c ket C o nnec t W rite R ead C lo s e

2

t0


35

Berkeley Sockets: Peer-Peer

S o c ket, B ind ,L is ten, A c c ep t R ead W rite

P e e r 1

P eer 2

C o nnec t W riteR eadS o c ket, B ind ,L is ten, A c c ep t

C o nnec t

Read & Write are independent threads

2


36

Berkeley Sockets: Producer-Consumer

S o c ket B ind L is ten A c c ep t R ead C lo s e

C onsum e r

P ro d u cer

S o c ket, B ind C o nnec t W rite C lo s e

Note the one-way nature of communication

2


37

Naming

4


38

What is a name?

Basic objects File Value Program

Data structures across a network How to reference How to modify


39

Synchronization

5


40

Mechanisms

Explicit signals Semaphores, conditions, events

Test and Set shared variables Consider chip cycle vs. bus cycle


41

Methods

Semaphores Conditional critical sections Messages Event counts and sequencers Monitors Modules, common procedures and entries Path expressions Managers I/O commands


42

Problems

Race conditions - cause loss of data

Action sequences - hierarchy to be maintained

Data protection


43

Consistency and Replication

6


44

World view

Users must see same data at all times Updates must be controlled Rollbacks must maintain consistency


45

Fault Tolerance

7


46

Partitioning

Problem: when is the system partitioned? Response time? Keep-alive? Other?

Solutions Rollback? Re-start? Cut-off? Other?


47

Security

8


48

Whose data is it?

Must protect data and programs Viruses Worms Errors Theft

Documents

1 Topics in Distributed Systems CS 5580F. ©2005 D. J. Foreman 2 Topics 1. Course requirements 2. Introduction 3. Communications 4. Naming 5. Synchronization