Five Things you Need to Know About Scaling

The 5 Things You Need To Know

About ScalingA. Jesse Jiryu Davis

Senior Python Engineer

1. Indexes

db.people.find({name: 'Jesse'}).explain()!{!! "cursor" : "BasicCursor",!! "n" : 10,!! "nscanned" : 100000,!! "millis" : 40,!}!!db.people.createIndex({name: 1});!!db.people.find({name: 'Jesse'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 10,!! "nscanned" : 10,!! "millis" : 0,!}!

1. Indexes

db.people.find({name: 'Jesse'}).explain()!{!! "cursor" : "BasicCursor",!! "n" : 10,!! "nscanned" : 100000,!! "millis" : 40,!}!!db.people.createIndex({name: 1});!!db.people.find({name: 'Jesse'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 10,!! "nscanned" : 10,!! "millis" : 0,!}!

1. Indexes

db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 1,!! "nscanned" : 10,!! "millis" : 0,!}!!db.people.createIndex({name: 1, hacks: 1});!!db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 1,!! "nscanned" : 1,!! "millis" : 0,!}!

1. Indexes

db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 1,!! "nscanned" : 10,!! "millis" : 0,!}!!db.people.createIndex({name: 1, hacks: 1});!!db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 1,!! "nscanned" : 1,!! "millis" : 0,!}!

1. Indexes

db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 1,!! "nscanned" : 10,!! "millis" : 0,!}!!db.people.createIndex({name: 1, hacks: 1});!!db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 1,!! "nscanned" : 1,!! "millis" : 0,!}!

1. Indexes

db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 1,!! "nscanned" : 10,!! "millis" : 0,!}!!db.people.createIndex({name: 1, hacks: 1});!!db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1_hacks_1",!! "n" : 1,!! "nscanned" : 1,!! "millis" : 0,!}!!db.people.dropIndex({name: 1})!!

1. Indexes

db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 1,!! "nscanned" : 10,!! "millis" : 0,!}!!db.people.createIndex({name: 1, hacks: 1});!!db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1_hacks_1",!! "n" : 1,!! "nscanned" : 1,!! "millis" : 0,!}!!db.people.dropIndex({name: 1})!!

1. Indexes

2. File System

• ext3 is bad• ext4 and xfs are good

• Set noatime • This goes in /etc/fstab

2. File System

3. Working Set Size

3. Working Set Size

Disk!

RAM!

4. Disks

4. DisksOption 1: Spinning disk.

100 seeks / sec

4. DisksOption 2: Spinning disks!

~100 seeks / second!

~100 seeks / second! ~100 seeks / second! ~100 seeks / second!

4. DisksOption 3: SSD

5. Shard

Primary!

Secondary!

Secondary!

MongoS!

Primary!

Secondary!

Secondary!

Primary!

Secondary!

Secondary!

Primary!

Secondary!

Secondary!

5. Shard

1. Indexes 2. File System 3. Working Set 4. Disks 5. Shard

1. Indexes 2. File System 3. Working Set 4. Disks 5. Shard

The 5 Things You Need To Know About Scaling

A. Jesse Jiryu Davisbit.ly/scaling-five