47
November 12 th , 2014 | Las Vegas, NV Rahul Bhartia, John Loughlin, & Matt Yanchyshyn

(BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

Embed Size (px)

DESCRIPTION

Deploying self-managed NoSQL databases on Amazon Web Services (AWS) is more straightforward than you would think. This session focuses on three popular self-managed NoSQL systems on Amazon EC2: MongoDB, Cassandra, and Couchbase. We start with an overview of each of these popular NoSQL databases, discuss their origins and characteristics, and demonstrate the use of the AWS ecosystem to deploy these NoSQL databases quickly. Later in the session, we dive deep on use cases, design patterns, and discuss creating highly-available and high-performance architectures with careful consideration for failure and recovery. Whether you're a NoSQL developer, architect, or administrator, join us for a comprehensive session on looking at three different NoSQL systems from a uniform perspective.

Citation preview

Page 1: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

November 12th, 2014 | Las Vegas, NV

Rahul Bhartia, John Loughlin, & Matt Yanchyshyn

Page 2: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 3: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 4: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

• Couchbase: http://amzn.to/1xAacOL

• MongoDB: http://amzn.to/1wyE7rb

• Datastax: http://bit.ly/1xAk3UF (Community AMI)

Page 5: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 6: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

With AWS, you

can always

change later!

Page 7: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

Amazon Elastic Block Store Instance Store (ephemeral)

• General Purpose (SSD): 3

IOPS/GB, 3000 IOPS/volume max

• PIOPS: 4000 IOPS/volume max

• Max 48,000 IOPS/Instance

• Up to 1TB/volume

• Linear performance increases up to

12 volumes on 8xlarge instances

• Snapshots

• Direct-attached storage

• SSD on new instance types

• Up to 365,000 IOPS and

6.4 TB/instance (i2.8xlarge)

• Fewer IOPS on smaller

instances

• No built-in backup functionality

Page 8: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

• When using instance store (ephemeral), replicate sufficiently and backup often

• When using Amazon EBS, leverage EBS snapshots

• Stripe multiple volumes for increased performance and storage capacity

Page 9: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 10: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 11: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

GENERAL PURPOSE DOCUMENT DATABASE OPEN-SOURCE

Page 12: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 13: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

Provision

Upgrade

Scale

Continuous backup

Point-in-time recovery

Performance alerts

Page 14: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 15: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 16: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 17: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 18: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 19: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

MongoDB

Primary

MongoDB

Secondary

MMS

Backup

Agent

Page 20: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 21: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 22: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 23: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 24: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 25: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 26: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 27: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 28: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 29: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 30: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 31: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

• Allows a node joining the cluster to pull a proportional

amount of data from other nodes in the cluster.

• Speeds node rebuilds as data is shipped incrementally

from every other node in the cluster.

Page 32: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 33: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 34: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 35: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

us-east-1

a

a

aa

b

c

b

cb

c

b

c

us-west-2

a

a

aa

b

c

b

cb

c

b

c

S3 S3

Page 36: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 37: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 38: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

CB Bucket Priority

Disk I/O Priority (New in CB 3.0)

Page 39: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

++++

• Tunable memory (new in CB 3.0)

• Shared thread pool (new in CB 3.0)

Page 40: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

• Graceful failover (New in CB 3.0)

• Delta node recovery (New in CB 3.0)

Page 41: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

• Rack Awareness

Page 42: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

• XDCR enables uni or bi-directional

• Database Change Protocol (New in CB 3.0)

• Pause and resume (New in CB 3.0)

Page 43: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

cbbackup

• Incremental backup and restore (New in CB 3.0)

Page 44: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
Page 45: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

(New in CB 2.5)

(New in CB 3.0)

(New in CB 3.0)

Page 46: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

XDCR XDCR

Rack A

ware

ness

Rack A

ware

ness

Rack A

ware

ness

Page 47: (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

http://bit.ly/awsevals