Embed Size (px)
Citation preview
HIGH AVAILABILITY SCENARIOS
Single Node
•Will have downtime•If node crashes human intervention might be needed
How MongoDB Replication works
Member 1
Member 2
Member 3
•Set is made up of 2 or more nodes
How MongoDB Replication works
Member 1
Member 2
PRIMARY
Member 3
•Election establishes the PRIMARY•Data replication from PRIMARY to SECONDARY
How MongoDB Replication works
Member 1
Member 2
DOWN
Member 3
negotiate new master
•PRIMARY may fail•Automatic election of new PRIMARY if majority exists
How MongoDB Replication works
Member 1
Member 2
DOWN
Member 3
PRIMARY
•New PRIMARY elected•Replication Set re-established
How MongoDB Replication works
Member 1
Member 2RECOVERING
Member 3
PRIMARY
•Automatic recovery
How MongoDB Replication works
Member 1
Member 2
Member 3
PRIMARY
•Replication Set re-established
DEMO
Replica Set 0•Two Node•Network failure can cause the nodes to slip which will result in the the whole system going READ_ONLY
NOT RECOMMENDED!
Replica Set 1
Arbiter
•Single datacenter•Single switch & power•Points of failure:•Power•Network•Datacenter•Two node failure
•Automatic recovery of single node crash
Replica Set 2
Arbiter
•Single datacenter•Multiple power/network zones•Points of failure:•Datacenter•Two node failure
•Automatic recovery of single node crash•w=2 not viable as losing 1 node means no writes
Replica Set 3•Single datacenter•Multiple power/network zones•Points of failure:•Datacenter•Two node failure
•Automatic recovery of single node crash•w=2 viable as 2/3 online
Replica Set 4
•Multi datacenter
•DR node for safety
•Can’t do multi data center durable write safely since only 1 node in distant DC
Replica Set 5
delayed •Three data centers•Can survive full data center loss •Can do w= { dc : 2 } to guarantee write in 2 data centers
HARDWARE SIZING
Collection 1
Index 1
Virtual Address Space 1
Collection 1
Index 1
Virtual Address Space 1
Collection 1
Index 1 This is your virtual memory size (mapped)
Virtual Address Space 1
Physical RAM
Collection 1
Index 1
Virtual Address Space 1
Physical RAM
Collection 1
Index 1
This is your resident memory size
Virtual Address Space 1
Physical RAM
DiskCollection 1
Index 1
Virtual Address Space 1
Physical RAM
DiskCollection 1
Index 1
100 ns
10,000 ns
=
=
Sizing RAM and Disk• Working set• Document Size• Memory versus disk• Data lifecycle patterns
• Long tail• pure random• bulk removes
Figuring out working Set> db.wombats.stats() { "ns" : "test.wombats", "count" : 1338330, "size" : 46915928, "avgObjSize" : 35.05557523181876, "storageSize" : 86092032, "numExtents" : 12, "nindexes" : 2, "lastExtentSize" : 20872960, "paddingFactor" : 1, "flags" : 0, "totalIndexSize" : 99860480, "indexSizes" : { "_id_" : 55877632, "name_1" : 43982848 },
Size of data
Size on disk (and in memory!)
Size of all indexes
Average document size
Size of each index
Disk configurations
~200 seeks / second
Single Disk
Disk configurations
~200 seeks / second
~200 seeks / second ~200 seeks / second ~200 seeks / second
Single Disk
RAID 0
Disk configurations
~200 seeks / second
~200 seeks / second ~200 seeks / second ~200 seeks / second
~400 seeks / second ~400 seeks / second ~400 seeks / second
Single Disk
RAID 0
RAID 10
SSD?• Seek time of 0.1ms vs 5ms
(200 seeks / sec => 10000 seeks / sec)
• But expensive
Takeaway• Know how important page faults are
–If you want low latency, avoid page faults• Size memory appropriately
–To avoid page faults, fit everything in RAM–Collection Data + Index Data
• Provision disk appropriately–RAID10 is recommended–SSD’s are fast, if you can afford them
MONITORING
Monitoring Toolsmongostat -
MMS! - http://mms.10gen.com
munin, cacti, nagios -http://www.mongodb.org/display/DOCS/Monitoring+and+Diagnostics
Monitoring is your friend!
SUMMARY
Typical DeploymentsUse? Set
sizeData Protection
High Availability Notes
X One No No Must use --journal to protect against crashes
Two Yes No On loss of one member, surviving member is read only
Three Yes Yes - 1 failureOn loss of one member, surviving two members can elect a new primary
X Four Yes Yes - 1 failure** On loss of two members, surviving two members are read only
Five Yes Yes - 2 failuresOn loss of two members, surviving three members can elect a new primary
Summary
• Reads from Primary are always consistent
• Reads from Secondaries are eventually consistent
• Automatic failover if a Primary fails
• Understand your Working Set
• Monitoring is your Friend!
