High Performance MongoDB on Storage-Optimized AWS EC2

Partner Technical Solutions, MongoDB

Sandeep Parikh

#MongoDBWorld

High Performance MongoDB on Amazon EC2

Solutions Architect, Amazon Web Services

Kyle Lichtenberg

Wait, isn’t there a whitepaper on this?

• Much has changed since March 2013!– (and an update is in the works!)

New EC2 Instances Types

• I2 released in December 2013– Next generation High I/O Instance Family– 4-32 vCPUs – 30-244 GiB of Memory– 800-6,400 GB of SSD Storage (up to 315,000+

IOPS)

• R3 released in April 2014– Next Generation Memory Optimized Instance

Family– 2-32 vCPUs – 15-244 GiB of Memory

Latest Generation Instance Features

• Enhanced Networking (SR-IOV)– Improved packet per second (PPS) performance– Lower inter-instance latencies– Very low network jitter

• SSDs– No mechanical seeking!– Higher I/O than magnetic

• Updated Hardware Architecture – Ivy Bridge Intel Xeon E5-2670 v2– Higher sustained memory bandwidth (up to

63,000 MB/s)

Storage I/O Optimized Instance Evolution

hi1.4xlarge

Memory

60.5 GiB

Storage

2x 1,024GB SSD

Network

10 Gigabit

i2.8xlargeMemory 244 GiB

Storage 8x 800GB SSD

Network 10 Gigabit



Network High



Network High

i2.xlargeMemory 30.5 GiB

Storage 800GB SSD

Network Moderate

Memory Optimized Instance Evolution

cr1.8xlargeMemory 244 GiB


Network 10 Gigabit

m2.4xlargeMemory 68.4 GiB

Storage 2x 840GB

Network High

m2.2xlargeMemory 34.2 GiB

Storage 850GB

Network High

m2.xlargeMemory 17.1 GiB

Storage 420GB

Network Moderate

r3.8xlargeMemory 244 GiB


Network 10 Gigabit

r3.4xlarge

Memory 122 GiB

Storage 320GB SSD

Network High

r3.2xlarge

Memory 61 GiB

Storage 160GB SSD

Network High

r3.xlarge

Memory 30.5 GiB

Storage 80GB SSD

Network Moderate

r3.large

Memory 15.25 GiB

Storage 32GB SSD

Network Moderate

AWS Block Storage Review

• Amazon Elastic Block Store (“EBS”)– Network attached– Durable– Snapshot backups to S3

• Amazon EBS Provisioned IOPS (“PIOPS”)– Standard EBS + definable, consistent

performance

• Instance Storage (“Ephemeral Storage”)– Lowest latency– Included in instance cost– Volatile!

Storage ConsiderationsEBS EBS PIOPS Instance Store

~100 IOPS Magnetic

3 IOPS/GB General Purpose (SSD)

Up to 1TB/Volume

Linear performance increases up to 12

volumes on 8xlarge instances

Max 36,000 IOPS/Instance

Snapshots!

Up to 30 IOPS/GB(max

4,000/volume)

Up to 1TB/Volume

Linear performance increases up to 12

volumes on 8xlarge instances


Snapshots!

Up to 365,000 IOPS/instance

Up to 6.4TB

Fewer IOPS on smaller instance

sizes


No built-in backup functionality

Storage Considerations

• When using EBS– Use EBS Optimized Instances!– Use PIOPS for consistent performance– Use snapshots for backups

• When using Instance Storage– Replicate! Replicate! Replicate!– Use a replica with EBS volumes for backups

Cost Considerations

All EBS All EBS PIOPS Hybrid Instance Store

47x 1TB General Purpose (SSD)

6x r3.2xlarge(EBS Optimized)

Volumes: $4,700.00Compute: $3,294.00

Total: $7,994.00/mo

35x 134GB @ 4K IOPS

6 r3.2xlarge(EBS Optimized)

Volumes: $586.25EBS IOPS: $9,100.00Compute: $3,294.00

Total: $12,394.25/mo

4x 800GB SSD2x i2.2xlarge

4x 1TB General Purpose (SSD)1x i2.2xlarge

(EBS Optimized)

Volumes: $400.00Compute: $3780.78

Total: $4,180.78/mo

Assuming a performance target of 140,000 IOPS:

All costs are estimations and are based on published pricing as of 6/18/2014.

Cost is an important performance metric!

• Reserved Instances– Pricing model designed for steady state workloads– Pay a small upfront fee for a reduced hourly rate– Light, Medium, and Heavy utilization– 1 and 3 year terms

Pricing Models

Hybrid Instance Store

4x 800GB SSD2x i2.2xlarge

4x 1TB General SSD1x i2.2xlarge

(EBS Optimized)

Volumes: $400.00

• On-Demand– One-time fee: $0.00– Monthly Cost: $4,180.78– Annual Cost: $50,169.36

• 1 YR Heavy Reserved– One-time fee: $10,920– Monthly Cost: $1,119.58– Annual Cost: $13,434.96 ($24,354.96)

• 3 YR Heavy Reserved– One-time fee: $16,440.00– Monthly Cost: $965.86– Annual Cost: $11,590.32 ($17,070.32)

All costs are estimations and are based on published pricing as of 6/18/2014.

Functional Deployment Patterns

• Proper instance type and storage selections play a huge part in price and performance.

• Not all storage architectures are created equal, but neither are all environments!

High Performance MongoDB

Instance Storage

Ephemeral

Set number of volumes per instance

RAID with LVM

Snapshots via LVM

EBS Volumes

Persistent

Multiple volumes per instance

RAID with LVM or MD

Snapshots

MongoDB Storage Considerations

Storage Differences

PIOPS EBS Instance SSDs

Highest durability Highest performance

Requires EBS-optimized instances

Available on new generation instances

Higher price per IOP Included in instance pricing

Development, Testing

MongoDB

App

MongoDBPrimary

App

MongoDBSecondar

y

MongoDBSecondar

y

Single Node Replica Set

Production, High Performance

MongoDBPrimary

App

MongoDBSecondar

y

MongoDBSecondar

y

Replica Set

MongoDBPrimary

MongoDBSecondar

y

MongoDBSecondar

y

Sharding

MongoDBPrimary

App

MongoDBSecondar

y

MongoDBSecondar

y

MongoDBPrimary

MongoDBSecondar

y

MongoDBSecondar

y

mongos

App

mongos

App

mongos

Deployment Strategies: SSDs

• SSD deployments– Replica Sets

and – MMS Backup

• High performance

• Highly available

• Continuous backup

MongoDB

Primary

MongoDB

Secondary

MongoDB

Secondary

MMS Backup Agent

MongoDB Management Service: Backup

Fully-managed, agent-based, continuous

backup

Custom snapshot

scheduling and retention

Point in-time recovery

Performance impact similar to Secondary

Encrypted data transfer

Restores require 2-factor authentication

Deployment Strategies: Hybrid

Mixed SSD/EBS deployment

Secondary with PIOPS

EBS

Powerful, high-bandwidth instance

Aggregate multiple

volumes for max IOPS

ex. r3.8xlarge

with 12 PIOPS EBS volumes

Key is avoiding

replication lag

Monitor with MMS

Measuring Performance

MMS Monitoring

MongoDB metrics

Visualizations

Dashboards

Alerts

System stats tools

iostat

mpstat

sar

mongoperf

MongoDB ops/second

Set threads, fileSize, caching

Partner Technical Solutions, MongoDB

Sandeep Parikh

#MongoDBWorld

Thank You!

Solutions Architect, Amazon Web Services

Kyle Lichtenberg

Technology

High Performance MongoDB on Storage-Optimized AWS EC2