EEC-681/781 Distributed Computing Systems Lecture 10 Wenbing Zhao [email protected] Cleveland State University

EEC-681/781EEC-681/781Distributed Computing Distributed Computing

SystemsSystems

Lecture 10Lecture 10

Wenbing ZhaoWenbing [email protected]@ieee.org

Cleveland State UniversityCleveland State University

Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao

22

OutlineOutline

• Clock Synchronization issues• Clock Synchronization Algorithms

– Centralized– Distributed

• Event ordering and logical clocks• Due date for project progress report

– 11/20 Monday mid-night– No extension!


33

Motivation for Clock SynchronizationMotivation for Clock Synchronization

• In everyday life, we are relying on clocks to coordinate our activities– For example, in EEC681, we meet every Monday and

Wednesday between 6-7:50pm

• In computer systems, it is also convenient to use clock as a way to coordinate different activities– For example, the “make” program relies on files’

timestamp to decide if a recompilation is necessary

Apr 19, 2023Apr 19, 2023 Wenbing ZhaoWenbing Zhao

44

Motivation for Clock SynchronizationMotivation for Clock Synchronization

• Stock market buy and sell orders• Secure document timestamps (with cryptographic

certification)• Aviation traffic control and position reporting• Radio and TV programming launch and monitoring• Intruder detection, location and reporting• Multimedia synchronization for real-time

teleconferencing• Network monitoring, measurement and control• Differentiated services traffic engineering


55

Universal Coordinated TimeUniversal Coordinated Time

• Universal Coordinated Time (UTC):– Based on the number of transitions per second of

the cesium 133 atom (pretty accurate)– At present, the real time is taken as the average of

some 50 cesium-clocks around the world– Introduces a leap second from time to time to

compensate that days are getting longer

• UTC is broadcast through short wave radio and satellite.– Satellites can give an accuracy of about 0.5 ms


66

Physical Clocks in Computer SystemsPhysical Clocks in Computer Systems

• Every machine has a timer that generates an interrupt H times per second

• There is a clock in machine p that ticks on each timer interrupt– Denote the value of that clock by Cp(t), where t is UTC

time

– Ideally, for each machine p, Cp(t) = t, or, in other words, dC/dt = 1


77

Clock Time and UTCClock Time and UTC• The relation between clock

time and UTC when clocks tick at different rates

• In practice:

1 - < dC/dt < 1 +

• Maximum drift rate: C

lock time, C


88

TermsTerms

• Clock drift rate (or clock accuracy)– The amount of deviation from UTC per unit of time (a

day or a week, etc.)

• Clock precision: – Resolution of the clock, e.g., 1s, or 1ns

• Clock skew– The difference in time values of two clocks is called

clock skew– Maximum clock skew of a group of clocks is

determined by the two clocks that have the largest clock difference


99

Clock SynchronizationClock Synchronization

• Two clocks are said to be synchronized at a particular instance of time if the clock skew of the two clocks is less than some specified constant δ

• A set of clocks are said to be synchronized if the clock skew of any two clocks in this set is less than δ


1010

Clock Synchronization IssuesClock Synchronization Issues

• A distributed system requires:– External Synchronization

• Synchronize with an external time source

– Internal Synchronization• Clocks within the same network synchronize with each other

• Clock Synchronization requires:– Each node can read the other nodes’ clock values

• Must consider unpredicted communication delay

– Time must never run backward• Smooth adjustments => must maintain the order of the events


1111

Message Propagation TimeMessage Propagation Time

• Estimate of message propagation time: (T1-T0-I)/2


1212

Message Delay DistributionMessage Delay Distribution

T1

T3T2

T4

)()( 2314 TTTT )]()[(2

14312 TTTT

Server

Client

x

0


1313

Clock Synchronization AlgorithmsClock Synchronization Algorithms

• Centralized– Passive Time Server Centralized Algorithm

• Cristian’s algorithm

– Active Time Server Centralized Algorithm• Berkeley Algorithm

• Distributed– Global Averaging Distributed Algorithms– Localized Averaging Distributed Algorithms


1414

Passive Timer ServerPassive Timer Server• Each node periodically sends a message (time=?) to the

time server at the current local clock time, T0

• The server responds with a message (time = T), T is the current time of the server

• The client receives the message at the local clock time T1, and adjusts its local clock time to T+(T1-T0-I)/2– The time taken by the server to handle the request message is I,

T+(T1-T0-I)/2

– Several measurements of T1-T0, discard the unreliable ones


1515

Active Time ServerActive Time Server

• The time server periodically broadcasts its clock time (T)• Other nodes receive the message to correct their own

clocks– Each node has the knowledge of the approximate time (Ta)

required for the propagation of the message, T+Ta

– Each nodes replies with the local clock time

• The server – Knows the approximate propagation time from each node– Takes fault-tolerant average of clock values as current time– The server adjusts it own, and sends the amount by which each

node clock requires adjustment to each node


1616

Centralized Algorithms – Centralized Algorithms – DrawbacksDrawbacks

• Single-point failure

• Scalability


1717

Global Averaging Distributed AlgorithmsGlobal Averaging Distributed Algorithms

• Each node broadcasts its local clock time periodically• Each node waits for time T

– The node collects the messages broadcast by other nodes– For each message received, the node keeps the local time– At the end of T, the node estimates the skew of its clock with

respect to each of the other nodes on the basis of the times at which it received

– The node computes a fault-tolerant average of the estimated skews and uses it to adjust its local clock


1818

Localized Averaging Distributed AlgorithmsLocalized Averaging Distributed Algorithms

• Each node exchanges its clock time with its neighbors

• Then sets its clock time to the average of its own clock and the clock times of its neighbors


1919

ExerciseExercise

• Consider the behavior of two machines in a distributed system. Both have clocks that are supposed to tick 1000 times per millisecond. One of them actually does, but the other ticks only 990 times per millisecond. If UTC updates come in once a minute, what is the maximum clock skew that will occur?


2020

Event OrderingEvent Ordering

• “Time, Clocks, and the Ordering of Events in a Distributed System”, by Leslie Lamport, Communications of the ACM, July 1978, Volume 21, Number 7, pp.558-565

• He showed that it is possible to synchronize all the clocks to produce a single, unambiguous time standard

• He pointed out the clock synchronization need not to be absolute– What usually matters is not that all processes agree on exactly

what time it is, but rather, that they agree on the order in which events occur


2121

Happens-Before RelationHappens-Before Relation

• With perfectly accurate physical time– An event a happened before an event b if a happened at an

earlier time than b

• Without using the physical clocks– Assume that the system is composed of a collection of

processes, each process consists of a sequence of events– The events of a process form a sequence, where a occurs

before b in this sequence if a happens before b– Assume sending and receiving a message is an event in a

process


2222

Happens-before RelationHappens-before Relation• “Happens-before” relation, denoted by “→”, is

defined as follows:– The relation “→” on the set of events of a system is the

relation satisfying the following three conditions:• If a and b are events in the same process, and a comes

before b, then a → b• If a is the sending of a message by one process and b is

the receipt of the same message by another process, then a → b

• If a → b and b → c, then a → c

– Event a causally affects event b


2323

Partial OrderingPartial Ordering

• Two distinct events a and b are said to be concurrent if a → b and b → a– Neither event can causally affect the other– This introduces a partial ordering of events in

a system with concurrently operating processes


2424

Logical ClocksLogical Clocks

• Logical clocks: Use the clock just as a way of assigning a number to an event, where the number is the time at which the event occurs– Define a clock Ci for each process Pi

• Assigns a number Ci(a) to any event a in that process

• The entire system of clocks is represented by the function C which assigns to any event b the number C(b), where C(b) =Cj(b) if b is an event in process Pj

• The clocks Ci are logical clocks rather than physical clocks


2525

Implementation of Logical ClocksImplementation of Logical Clocks

• The logical clocks is correct if the events of the system that are related to each other by the happens-before relation can be properly ordered using these clocks

• Clock condition: – For any event a, b, if ab then C(a) <C(b)


2626

Implementation of Logical ClocksImplementation of Logical Clocks

• According to our definition of the happens-before relation, the clock condition is satisfied if the following two conditions hold:– C1: if a and b are events in process Pi, and a comes

before b, then Ci(a) < Ci(b)– C2: if a is the sending of a message by process Pi

and b is the receipt of that message by process Pj, then Ci(a) < Cj(b)


2727

Implementation of Logical ClockImplementation of Logical Clock

• To meet C1: – Each process Pi increments Ci between any two

successive events

• To meet C2: – (a) if event a is the sending of a message m by

process Pi, then the message m contains a timestamp Tm = Ci(a).

– (b) Upon receiving a message m, process Pj sets Cj greater than or equal to its present value and greater than Tm


2828

Implementation of Implementation of Logical Clocks by CountersLogical Clocks by Counters

• A Lamport logical clock is a monotonically increasing software counter

• Each process Pi keeps its own logical clock Ci which is used to apply Lamport timestamps to events

• To capture the happens-before relation →, processes update their logical clocks and transmit the values of their logical clocks in messages as follows:– Before each event at Pi: Ci := Ci+1

– When Pi sends a message m, it piggybacks t = Ci

– When Pj receives (m,t): Cj := max(Cj,t) + 1

• e → e’ => C(e) < C(e’)


2929

Implementation of Implementation of Logical Clocks by CountersLogical Clocks by Counters

Question: Which two events are concurrent?


3030

Total Ordering of EventsTotal Ordering of Events

• We can use the logical clocks satisfying the Clock Condition to place a total ordering on the set of all system events– Simply order the events by the times at which occur– To break the ties, Lamport proposed the use of any

arbitrary total ordering of the processes, i.e. process id


3131

Total Ordering of EventsTotal Ordering of Events

• Using this method, we can assign a unique timestamp to each event in a distributed system to provide a total ordering of all events

• Very useful in distributed system– Solving the mutual exclusion problem– Totally ordered reliable multicast => needed to build

fault tolerant systems

Documents

EEC-681/781 Distributed Computing Systems Lecture 10 Wenbing Zhao [email protected] Cleveland State University