MongoUK 2011 Replication

8/4/2019 MongoUK 2011 Replication

1/24

[email protected]

Replication


2/24

MongoDB Replication

MongoDB replication like MySQL replication

Asynchronous master/slave

Variations:

Master / slave

Replica Sets


3/24

A cluster of N servers Any (one) node can be primary Consensus election of primary Automatic failover Automatic recovery All writes to primary Reads can be to primary (default) or a secondary

Replica Set features


4/24

How MongoDB Replication works

Member1

Member2

Member3

Set is made up of 2 or more nodes


5/24


Member1

Member2PRIMARY

Member3

Election establishes the PRIMARYData replication from PRIMARY to SECONDARY


6/24


Member1

Member2DOWN

Member3

negotiate

newmaster

PRIMARY may failAutomatic election of new PRIMARY if majorityexists


7/24


Member1

Member2DOWN

Member3

PRIMARY

New PRIMARY electedReplication Set re-established


8/24


Member1

Member2RECOVERING

Member3

PRIMARY

Automatic recovery


9/24


Member1

Member2

Member3

PRIMARY

Replication Set re-established


10/24

Creating a Replica Set

>cfg={

_id:"acme_a",members:[

{_id:0,host:"sf1.acme.com"},

{_id:1,host:"sf2.acme.com"},

{_id:2,host:"sf3.acme.com"}]}

>useadmin

>db.runCommand({replSetInitiate:cfg})


11/24

Replica Set Options

{arbiterOnly: True}

Can vote in an election

Does not hold any data

{hidden: True}

Not reported in isMaster()

Will not be sent slaveOk() reads

{priority: n}

{tags: }


12/24

Priorities

Prior to 2.0.0{priority:0}//NevercanbeelectedPrimary

{priority:1}//CanbeelectedPrimary

New in 2.0.0

Priority, floating point number between 0 and 1000 During an election

Most up to date

Highest priority

Allows weighting of members during failover


13/24

Priorities - example Assuming all members are up to date Members A or B will be chosen first

Highest priority Members C or D will be chosen next if

A and B are unavailable A and B are not up to date

Member E is never chosen

priority:0 means it cannot be elected

E

p:0

C

p:1

D

p:1

A

p:2

B

p:2


14/24

Tagging

New in 2.0.0 Control over where data is written to Each member can have one or more tags e.g.

tags:{dc:"ny"}

tags:{dc:"ny",ip:"192.168",rack:"row3rk7"}

Replica set defines rules for where data resides Rules can change without change application code


15/24

Tagging - example{

_id:"mySet",members:[{_id:0,host:"A",tags:{"dc":"ny"}},{_id:1,host:"B",tags:{"dc":"ny"}},

{_id:2,host:"C",tags:{"dc":"sf"}},{_id:3,host:"D",tags:{"dc":"sf"}},{_id:4,host:"E",tags:{"dc":"cloud"}}]settings:{getLastErrorModes:{allDCs:{"dc":3},someDCs:{"dc":2}}}}

>db.blogs.insert({...})

>db.runCommand({getLastError:1,w:"allDCs"})


16/24

Using Replicas for Reads

slaveOk()

- driver will send read requests to Secondaries- driver will always send writes to Primary

Java examples-DB.slaveOk()-Collection.slaveOk()

-find(q).addOption(Bytes.QUERYOPTION_SLAVEOK);


17/24

Safe Writesdb.runCommand({getLastError:1,w:1})- ensure write is synchronous- command returns after primary has written to memory

w=norw='majority'

- n is the number of nodes data must be replicated to- driver will always send writes to Primary

w='myTag' [MongoDB 2.0]- Each member is "tagged" e.g. "US_EAST", "EMEA", "US_WEST"- Ensure that the write is executed in each tagged "region"


18/24

Safe Writesfsync:true- Ensures changed disk blocks are flushed to disk

j:true- Ensures changes are flush to Journal


19/24

When are elections triggered?

When a given member

See's that the Primary is not reachable The member is not an Arbiter Has a priority greater than other eligible members


20/24

Typical Deployments

Use?Set

sizeDataProtection HighAvailability Notes

X One No NoMustuse--journaltoprotectagainst

crashes

Two Yes NoOnlossofonemember,survivingmemberisreadonly

Three Yes Yes-1failureOnlossofonemember,survivingtwomemberscanelectanewprimary

X Four Yes Yes-1failure**Onlossoftwomembers,surviving

twomembersarereadonly

Five Yes Yes-2failures

Onlossoftwomembers,surviving

threememberscanelectanewprimary


21/24

Use Cases - Multi Data Center write to three data centers

allDCs:{"dc":3}

>db.runCommand({getLastError:1,w:"allDCs"})

write to two data centers and three availability zonesallDCsPlus:{"dc":2,"az":3}

>db.runCommand({getLastError:1,w:"allDCsPlus"})

US-EAST-1tag:{dc:"JFK",az:"r1"}

US-EAST-2

tag:{dc:"JFK"az:"r2"}

US-WEST-1tag:{dc:"SFO",az:"r3"}

US-WEST-2

tag:{dc:"SFO"az:"r4"}

LONDON-1tag:{dc:"LHR",az:"r5"}


22/24

Use Cases - Data Protection &High Availability

A and B will take priority during a failover C or D will become primary if A and B become unavailable E cannot be primary

D and E cannot be read from with a slaveOk()

D can use be used for Backups, feed Solr index etc. E provides a safe guard for operational or application error

Dpriority:1

hidden:True

Apriority:2

Bpriority:2

Cpriority:1

E

priority:0hidden:True

slaveDelay:3600


23/24

Replication features

Reads from Primary are always consistent

Reads from Secondaries are eventually consistent

Automatic failover if a Primary fails

Automatic recovery when a node joins the set

Control of where writes occur


24/24

@mongodb

conferences,appearances,andmeetupshttp://www.10gen.com/events

http://bit.ly/mongo

Facebook|Twitter|LinkedInhttp://linkd.in/joinmongo

download at mongodb.org

Were Hiring [email protected]

Documents

MongoUK 2011 Replication