MongoDB

Replication and Sharding

Workshop by Harun Yardımcı

@h_yardimci

What we will do today?- Following Replicated Shard

What is MongoDB?

● Document-oriented database

● in MongoDB you store JSON-like documents with dynamic schemas

● Bridge the gap between key-value stores and relational databases

What is MongoDB?

Why MongoDB?

● Document-oriented

● High performance

● High availability

● Easy scalability

● Rich query language + MapReduce

Use CasesWell Suited

● Archiving and event logging● Document and Content Management Systems● ECommerce Often in combination with an RDBMS for the final order processing and accounting

● Gaming Small read/writes are a good fit for MongoDB● Mobile Specifically, the server-side infrastructure of mobile systems. Geospatial.

● Operational data store of a web site MongoDB is very good at real-time inserts, updates, and queries. Specific web use case examples:● content management● comment storage, management, voting● user registration, profile, session data

● Real-time stats/analytics

What is a Document?

{'id':1, 'category_name' : 'Computer'}

Mongo Data Model

Key:

'category_name'

Value:

'Computer'

Field: key-value pair

'category_name':'Computer'

Document: set of fields

{'id':1, 'category_name' : 'Computer'}

Mongo Data Model

Collection: set of documents

(say categories)

{'id':1, 'category_name' : 'Computer'},

{'id':2, 'category_name' : 'Mobile'},

...

Database: set of collections

categories

products

members

Simple Usage and Introduction

First Run

$ mkdir -p /data/db/

$ chown -R mongod:mongod /data/db/

$ mongod [--dbpath /data/db/] &

$ mongo

> show dbs

> use admin

> show collections

Create a data path and give permissions

Start mongod deamon

Connect to mongod

Intro

CRUD Operations

> use testdb

switched to db testdb

> j = { name : "deneme" };

{"name" : "deneme"}

> t = { x : 3 };

{ "x" : 3 }

> db.col.save(j);

> db.col.insert(j); /* see the error message */

> db.col.save(t);

> for (var x = 1; x <= 20; x++) db.col.save({x:x, j:x*x})

>

Intro

> db.col.find();

{ "_id" : ObjectId("4c2209f9f3924d31102bd84a"), "name" : "mongo" }

{ "_id" : ObjectId("4c2209fef3924d31102bd84b"), "x" : 3 }

{ "_id" : ObjectId("4f971ab51c3fde3bc55f286f"), "x" : 1, "j" : 1 }

{ "_id" : ObjectId("4f971ab51c3fde3bc55f2870"), "x" : 2, "j" : 4 }

{ "_id" : ObjectId("4f971ab51c3fde3bc55f2871"), "x" : 3, "j" : 9 }

…

has more

> db.col.remove({"x":"3"});

>

The Big Picture!

Replication

Two Types of Replications● Master / Slave Replication● ReplicaSet

Master-Slave Rep.

$ mkdir -p /data/db/ms

$ mkdir -p /data/db/sl

$ bin/mongod --master [--port <port>] [--dbpath /data/masterdb/]

$ bin/mongod --slave [--port <port>] --source <masterhostname>[:<port>] [--dbpath /data/slavedb/]

…

> db.printReplicationInfo() - on master

> db.printSlaveReplicationInfo() - on slave

> use admin

> db.runCommand({resync: 1})

Different data paths for each instances

ReplicaSet - Voting

Consensus Vote

For a node to be elected primary, it must receive a majority of votes by the following formula

(floor(5/2)+1)

Arbiters - {_id: 2, host: 'localhost:27019', arbiterOnly: true}

Reachable Node – heartbeat

Each replica set is limited to 12 total nodes and 7 voting nodes.

ReplicaSet

Starting The Nodes

$ mkdir -p /data/r0

$ mkdir -p /data/r1

$ mkdir -p /data/r2

$ mongod --replSet myRSet --port 27017 --dbpath /data/r0

$ mongod --replSet myRSet --port 27018 --dbpath /data/r1

$ mongod --replSet myRSet --port 27019 --dbpath /data/r2

myRSet

localhost27017

/data/r0

localhost27018

/data/r1

localhost27019

/data/r2

ReplicaSet

Initiating The Set

> config = {_id: 'myRSet', members: [

{_id: 0, host: 'localhost:27017'},

{_id: 1, host: 'localhost:27018'},

{_id: 2, host: 'localhost:27019'}]

}

> rs.initiate(config);

{

"info" : "Config now saved locally. Should come online in about a minute.",

"ok" : 1

}

> rs.status();

Sharding

Sharding Components

● Shard Servers

● Config Servers

● mongos Router

Sharding

Sharding

$ mkdir -p /data/db/s1 /data/db/s2 /data/db/config

$ mongod --shardsvr --port 27001 --dbpath /data/db/s1 &

$ mongod --shardsvr --port 27002 --dbpath /data/db/s2 &

$ mongod --configsvr --port 27003 --dbpath /data/db/config &

$ mongos --port 27017 --configdb localhost:27003 &

Sharding

Add Shards to Config

> use admin

> db.runCommand( {addShard : "localhost:27001"} );

{"ok" : 1 , "added" : "localhost:27001"}

> db.runCommand( {addShard : "localhost:27002"} );

{"ok" : 1 , "added" : "localhost:27002"}

Connect to mongos to add shards

Sharding

Enable Sharding

> db.runCommand( { enablesharding: "test_database"} );

{ "ok" : 1 }

> db.runCommand( { shardcollection : "test_database.myCollection", key : {"_id" :1} })

{ "collectionsharded" : "test_database.myCollection", "ok" : 1 }

ReplicaSet + Sharding

host1$ mongod --shardsvr --replSet rs_a

host2$ mongod --shardsvr --replSet rs_a

host3$ mongod --shardsvr --replSet rs_a

> cfg = {

_id : "rs_a",

members : [

{_id : 0, host : "host1:27018", priority : 1},

{_id : 1, host : "host2:27018", priority : 1},

{_id : 2, host : "host3:27018", priority : 0}

]

}

> rs.initiate(cfg)

Same replica set name

ReplicaSet + Sharding

host1$ mongod --configsvr

host2$ mongod --configsvr

host3$ mongod --configsvr

$ mongos --configdb host1:27019,host2:27019,host3:27019

$ mongo

> db.adminCommand( { addShard : "rs_a/host1:27018,host2:27018,host3:27018" } )

> db.adminCommand( { addShard : "rs_b/host4:27018,host5:27018,host6:27018" } )

> db.adminCommand( { addShard : "rs_c/host7:27018,host8:27018,host9:27018" } )

What is Next?

Please

Try it Yourself

and

Share Your Experiences

Thanks