12/17/2015Distributed Systems - Comp 6551 Consistency and Replication The problems we are trying to solve Types of consistency Approaches to propagation

04/21/23 Distributed Systems - Comp 655 1

Consistency and Replication

• The problems we are trying to solve

• Types of consistency• Approaches to propagation


Transparency in a Distributed System

Transparency Description

AccessHide differences in data representation and how a resource is accessed

Location Hide where a resource is located

Migration Hide that a resource may move to another location

RelocationHide that a resource may be moved to another location while in use

Replication Hide that a resource is replicated

ConcurrencyHide that a resource may be shared by several competitive users

Failure Hide the failure and recovery of a resource


What problems does replication solve?

• Some capacity and performance problems– Keep replicas on both sides of a bottleneck– Keep replicas on both sides of a connection

with long delays

• Two kinds of incoherence:– Replication provides some location

transparency– Replication provides some failure transparency

(aka fault tolerance)Continue to work if one copy goes downContinue to work if the network goes down


What problems does replication cause?

• Consistency– To maintain concurrency

transparency, system has to keep replicas updated

• Complexity – To maintain replication transparency,

system has to be able to locate and select appropriate replicas

• Overhead can take back capacity and performance gains


If you remember onlyone two thing(s) …

1. There are many types of consistency, known as “consistency models”

2. As the consistency model gets stronger

• The system gets easier to use.• The system gets harder to

implement.• The system gets slower and

consumes more resources.






Types of (data-centric) consistency


Ground rules for examples• Examples deal with a company’s inventory

data• Inventory data is replicated at every

location• This is for illustrative purposes; we are NOT

claiming this is a good architectural idea• Assume each process in each example is at

a different location• Assume that invisible processes propagate

each update to all other copies• Time moves left to right


Strict consistency

P1:

P2:

P1:

P2:

D=H

D:DVD players, H:high, L:low, set:=, get:?

D?H

D=H

D?L D?H

Yes

No


Sequential consistency 1

P1:

P2:

P3:

P4:

D=H

D=L

D?L

D?L D?L

D?L



Sequential consistency 2

P1:

P2:

P3:

P4:

D=H

D=L

D?L

D?L D?H

D?H



Sequential consistency - not

P1:

P2:

P3:

P4:

D=H

D=L

D?L

D?H D?L

D?H



Cost of sequential consistency

• Assume we have a distributed system with a replicated store and a feature to maintain consistency across the replicas

• Let r be the expected read time• Let w be the expected write time• Let t be the minimal packet transfer

time between nodes in the system• Then r + w t


Causal consistency

P1:

P2:

P3:

P4:

D=L

D?L

D?L

D?HD?L

D?H

D=M

D=H

D?M

D?M

No causal relationship

Not sequentially consistent

Potential causal relationships

D:DVD players, H:high, M: medium, L:low, set:=, get:?


Causal consistency - not

P1:

P2:

P3:

P4:

D=L

D?L

D?H

D?H

D=H

D?L

D?L

Potential causal relationship



Causal consistency - ok

P1:

P2:

P3:

P4:

D=L

D?M

D?M

D=M

D?L

D?L

No causal relationship

D:DVD players, M:medium, L:low, set:=, get:?


Types of (data-centric) consistency


Eventual consistency• DNS

– A domain can only be updated by its naming authority (therefore, no write-write conflicts)

– Updates propagated on a schedule– Most accesses work from cache– Can refresh a cached value on failure

• Web– A page can only be updated by its owner

(again, no write-write conflicts)– Most accesses work from cache– Page can be refreshed if user suspects a

failure


Client-centric consistency


Client-centric consistency models

Model The ideaMonotonic reads

Each read by a process returns the same value as the previous read, or a more recent value

Monotonic writes

Each write by a process must complete before the next write of the data item by the process

Read your writes

Each write by a process will be visible in any subsequent read by that process

Writes follow reads

Each write by a process after a read will be done at all replicas on a value that is at least as recent as the value read


Client-centric examples• Imagine a discussion group with

replicas in Columbus and Nairobi• Imagine these events

– Alice starts thread A– Bob starts thread B– Alice edits A to make A’– Alice adds attachment to make A’’– Carol posts reply AC


Monotonic reads

Columbus Nairobi

Carol reads againOK

B

A A

Carol reads

BNot mono read


Monotonic reads

Columbus Nairobi

Carol reads againOK

A

A B

Carol reads

BNot mono read


Monotonic writes

Columbus Nairobi

OK

A’

A A

A’’

A’Not mono write


Read your writes

Columbus Nairobi

Alice reads againOK

A

A B

AC

BNot read your writes

Alice reads

AC

Read your writes, butNon-monotonic reads


Writes follow reads

Columbus Nairobi

AC arrivesOK

A

A

Carol reads

AC

Not writes follow reads


Writes follow reads to prevent this

Columbus Nairobi

A

AC

A

Carol reads

AC

Bob reads Dave readsDave sees replybefore original






Types of replicas

Any experience with server-initiated replicas?


What to propagate?• Notifications• Updated data• Update operations


Who initiates propagation?• Server (push-based protocol)• Client (pull-based protocol)

Documents

12/17/2015Distributed Systems - Comp 6551 Consistency and Replication The problems we are trying to solve Types of consistency Approaches to propagation