MongoDB on AWS in 5 min

MongoDB on AWS

[email protected]

[email protected]

David TurnerI look after the internet

125

[email protected]@davidmturner.com124

[email protected] [email protected]

[email protected]

[email protected]

M102: MongoDB for DBAs

[email protected]

100%

116

[email protected]

M101P: MongoDB for Programmers

[email protected]

Less than 100%

114

[email protected]

What is MongoDB?

112

[email protected]

Prebaked AMI

109

MongoDB with 1000 or 4000 PIOPS

[email protected]

Up and running in minutes

107

[email protected]

Not ready for production

106

[email protected]

[email protected]

Approach

1. configure AWS objects

2. instantiate instances using AWS CLI tools

3. scripted install with user data bash script

4. initialise the replica set

102

[email protected]

Configure VPC

101

[email protected]

Subnets

100

[email protected]

$ aws ec2 create-subnet --vpc-id vpc-xxxxxxxx --cidr-block 10.0.1.0/24 --availability-zone eu-west-1a

99

[email protected]

$ aws ec2 create-subnet --vpc-id vpc-xxxxxxxx --cidr-block 10.0.2.0/24 --availability-zone eu-west-1b

[email protected]

Route Table

97

[email protected]

Assuming you have a NAT instance already

96

[email protected]

$ aws ec2 create-route-table --vpc-id vpc-xxxxxxxx

95

[email protected]

$ aws ec2 create-route --route-table-id rtb-xxxxxxxx --destination-cidr-block 0.0.0.0/0 --instance-id i-xxxxxxxx

94

[email protected]

$ aws ec2 associate-route-table --route-table-id rtb-xxxxxxxx --subnet-id subnet-xxxxxxx1

93

[email protected]

$ aws ec2 associate-route-table --route-table-id rtb-xxxxxxxx --subnet-id subnet-xxxxxxx2

[email protected]

Security Group

91

[email protected]

$ aws ec2 create-security-group --group-name MongoDB --description MongoDB --vpc-id vpc-xxxxxxxx

90

[email protected]

Ingress

89

[email protected]

$ aws ec2 authorize-security-group-ingress --group-id sg-xxxxxxxx --protocol tcp --port 27017 --source-group sg-xxxxxxxx

88

[email protected]

$ aws ec2 authorize-security-group-ingress --group-id sg-xxxxxxxx --protocol tcp --port 27017 --source-group sg-yyyyyyyy

87

[email protected]

$ aws ec2 authorize-security-group-ingress --group-id sg-xxxxxxxx --protocol tcp --port 22 --source-group sg-zzzzzzzz

86

[email protected]

Egress

85

[email protected]

$ aws ec2 authorize-security-group-egress --group-id sg-xxxxxxxx --protocol tcp --port 27017 --source-group sg-xxxxxxxx

84

[email protected]

$ aws ec2 authorize-security-group-egress --group-id sg-xxxxxxxx --protocol tcp --port 80 --cidr 0.0.0.0/0

83

[email protected]

$ aws ec2 authorize-security-group-egress --group-id sg-xxxxxxxx --protocol tcp --port 443 --cidr 0.0.0.0/0

[email protected]

$ aws ec2 authorize-security-group-egress --group-id sg-xxxxxxxx --protocol udp --port 123 --cidr 0.0.0.0/0

[email protected]

Network ACLs

80

[email protected]

Lazy. Default ALLOW.

79

[email protected]

Placement Groups

77

[email protected]

$ aws ec2 create-placement-group MongoDB-a

76

[email protected]

$ aws ec2 create-placement-group MongoDB-b

[email protected]

IAM

74

[email protected]

TrustPolicy.json: { "Version": "2012-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Principal": { "Service": "ec2.amazonaws.com" }, "Action": "sts:AssumeRole" } ] }

73

[email protected]

MongoDBPolicy.json

{ "Version": "2012-10-17", "Statement": [ { "Action": ["ec2:DescribeVolumes", ec2:DescribeTags, "ec2:DescribeInstances", "ec2:DescribeInstanceAttribute"], "Resource": ["*"], "Effect": "Allow" }, { "Action": ["ec2:DeleteSnapshot", ec2:CreateSnapshot", "ec2:DescribeSnapshots"], "Resource": ["*"], "Effect": "Allow" }, { "Action": ["ec2:CreateTags"], "Resource": ["*"], "Effect": "Allow" } ] }

72

[email protected]

Role

71

[email protected]

$ aws iam create-role --role-name MongoDB --assume-role-policy-document file://TrustPolicy.json

70

[email protected]

$ aws iam put-role-policy --role-name MongoDB --policy-name MongoDB-Policy --policy-document file://MongoDBPolicy.json

69

[email protected]

Instance Profile

68

[email protected]

$ aws iam create-instance-profile --instance-profile-name MongoDB

67

[email protected]

$ aws iam add-role-to-instance-profile --instance-profile-name MongoDB --role-name MongoDB

66

[email protected]

fire up instances with a bash userdata script

65

[email protected]

$ data=$(cat ./user-data.sh)

64

[email protected]

# volume sizes in GB data_size=500 log_size=15 journal_size=25

63

[email protected]

$ aws ec2 run-instances --region eu-west-1

62

[email protected]

--security-group-ids sg-xxxxxxxx

61

[email protected]

--key-name TopSecretKeyPair

60

[email protected]

--iam-instance-profile {Arn:"arn:aws:iam::012345678901:instance-profile/MongoDB_Instance_Profile"}'

59

[email protected]

--instance-type r3.large

(2 vCPU, 15.25GB RAM)

58

[email protected]

--block-device-mapping [{ "DeviceName": "/dev/xvdf", Ebs: {"VolumeSize":'$data_size', "VolumeType": "io1", "Iops": 1000}}, {DeviceName": "/dev/xvdg", "Ebs": {VolumeSize":'$data_size', VolumeType: "io1", "Iops": 1000}}, {"DeviceName": "/dev/xvdh", Ebs": {VolumeSize":'$journal_size', "VolumeType": "io1", "Iops": 250}}, {DeviceName": "/dev/xvdi", "Ebs": {VolumeSize":'$log_size', "VolumeType": "io1", "Iops": 150}}]

57

(data)

(journal)

(log)

[email protected]

--placement AvailabilityZone=eu-west-1a,GroupName=MongoDB

56

[email protected]

--disable-api-termination

55

[email protected]

--image-id ami-a10897d6

Amazon Linux AMI 2015.03 (HVM), SSD Volume TypeRoot device type: ebs Virtualization type: hvm

54

[email protected]

--subnet-id subnet-xxxxxxxx

53

[email protected]

user-data $data

52

[email protected]

count 2

51

(then same again for AZ eu-west-1b)

[email protected]

yum -y update

50

[email protected]

mdadm --verbose --create --name=mongo /dev/md0 --level=0 --chunk=256 --raid-devices=2 /dev/xvdf /dev/xvdg

mdadm --detail --scan | tee -a /etc/mdadm.conf

49

[email protected]

mkdir /mnt/data /mnt/journal /mnt/log

mkfs.ext4 /dev/md0 mkfs.ext4 /dev/xvdh mkfs.ext4 /dev/xvdi

48

[email protected]

uuid=`blkid -o value -s UUID /dev/md0`

echo "UUID=$uuid /mnt/data ext4 defaults,auto,noatime,noexec 0 0 /dev/xvdh /mnt/journal ext4 defaults,auto,noatime,noexec 0 0 /dev/xvdi /mnt/log ext4 defaults,auto,noatime,noexec 0 0" >>

/etc/fstab mount -a

47

[email protected]

ln -s /mnt/journal /mnt/data/journal

46

[email protected]

echo "[MongoDB] name=MongoDB Repository baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64 gpgcheck=0 enabled=1" >> /etc/yum.repos.d/mongodb.repo

45

[email protected]

yum install -y mongodb-org-2.6.1 mongodb-org-server-2.6.1 --exclude mongodb-org, mongodb-org-server

service mongod stop

44

[email protected]

yum install -y sysstat gcc python27 python27-pip python27-devel

pip-2.7 install pymongo boto

43

[email protected]

chown mongod:mongod /mnt/data chown mongod:mongod /mnt/journal chown mongod:mongod /mnt/log

42

[email protected]

echo "dbpath=/mnt/data logpath=/mnt/log/mongodb.log logappend=true fork=true replSet = AWSUserGroup " > /etc/mongod.conf

41

[email protected]

echo "mongod soft nofile 64000 mongod hard nofile 64000 mongod soft nproc 32000 mongod hard nproc 32000" > /etc/security/limits.d/90-mongo.conf

40

[email protected]

echo 'ACTION=="add", KERNEL=="md*", ATTR{bdi/read_ahead_kb}="16"' >> /etc/udev/rules.d/85-ebs.rules echo 'ACTION=="add", KERNEL=="xvdf", ATTR{bdi/read_ahead_kb}="16"' >> /etc/udev/rules.d/85-ebs.rules echo 'ACTION=="add", KERNEL=="xvdg", ATTR{bdi/read_ahead_kb}="16"' >> /etc/udev/rules.d/85-ebs.rules echo 'ACTION=="add", KERNEL=="xvdh", ATTR{bdi/read_ahead_kb}="16"' >> /etc/udev/rules.d/85-ebs.rules echo 'ACTION=="add", KERNEL=="xvdi", ATTR{bdi/read_ahead_kb}="16"' >> /etc/udev/rules.d/85-ebs.rules

39

[email protected]

echo "/mnt/log/mongodb.log { daily rotate 30 compress dateext missingok notifempty sharedscripts prerotate grep query /mnt/log/mongodb.log | logger -t mongodb -p warn endscript postrotate /bin/kill -SIGUSR1 \$(/bin/cat /mnt/data/mongod.lock) rm -f /mnt/log/mongodb.log.[0-9][0-9][0-9][0-9]-* endscript } " > /etc/logrotate.d/mongod

38

[email protected]

echo "net.ipv4.tcp_keepalive_time = 120" > /etc/sysctl.d/01-mongod.conf

37

[email protected]

chkconfig mongod on

36

[email protected]

reboot35

[email protected]

Route53

34

[email protected]

Skipping it for time

33

[email protected]

[email protected]

Configure MongoDB

31

[email protected]

rs.initiate()

30

[email protected]

rs.add(x.x.x.x)

29

[email protected]

rs.add(y.y.y.y)

28

[email protected]

rs.status()

27

[email protected]

Backups

26

[email protected]

[email protected]

Hidden Member

24

[email protected]

EBS Snapshots

23

[email protected]

Hourly cronjob

22

[email protected]

Python Script

Lock the database

Lock the filesystem

EBS Snapshot

Unlock filesystem

Unlock database

Trim snapshots

21

[email protected]

if __name__ == "__main__": conn = boto.ec2.connect_to_region(REGION) client.admin.command("fsync", lock=True) p = subprocess.Popen( '/usr/bin/sudo /sbin/fsfreeze -f /mnt/data', shell=True) p.wait() create_snapshots(conn, get_volume_ids(conn)) p = subprocess.Popen( '/usr/bin/sudo /sbin/fsfreeze -u /mnt/data', shell=True) p.wait() client.admin[$cmd'].sys.unlock.find_one()

conn.trim_snapshots(hourly_backups=3, daily_backups=7, weekly_backups=4, monthly_backups=True)

20

[email protected]

Development instances

19

[email protected]

--block-device-mapping [ { "DeviceName": "/dev/xvdf", "Ebs": {"SnapshotId": snap-xxxxxxxx, "VolumeSize":'$data_size', "VolumeType": gp2"} },{"DeviceName": "/dev/xvdg", "Ebs": {"SnapshotId": snap-xxxxxxxx", "VolumeSize":'$data_size', "VolumeType": gp2"} }

18

[email protected]

Development instance in less than 15 min

17

[email protected]

Havent Shown

16

[email protected]

EBS Encryption and KMS

15

[email protected]

Multi-region

14

[email protected]

Sharded configuration

13

[email protected]

Log management

12

[email protected]

Monitoring

11

[email protected]


Would like to meet...

[email protected]

[email protected]

Images4: http://i.ytimg.com/vi/-xcecNrpChQ/maxresdefault.jpg

15: http://elginsweeper.com/portals/0/Images/Application_Photos/Runway.gif

17: http://arabhardware.net/wp-content/uploads/2014/12/ram-04.jpg

18: http://www.public-domain-image.com/full-image/objects-public-domain-images-pictures/an-old-west-style-padlock-in-old-town-san-diego.jpg-free-stock-photo.html

20: http://orbital.comp.nus.edu.sg/wp-content/uploads/2013/02/STS-133_Discovery_Lift_Off_Launch_Pad_39A_KSC.jpg

23: http://www.layman.org/wp-content/uploads/2013/10/face.jpg

26: http://blog.mongodb.org/post/4982676520/mongodb-on-ec2-best-practices

122: https://dogchow.com/media/28148/how_do_i_stop_my_dog_from_begging_istock_000012144475small.jpg

124: http://www.salus-wellness.com/wp-content/uploads/2010/11/Business.jpg

126: http://www.twinfinite.net/wp-content/uploads/2015/03/Lego.jpg

Software

MongoDB on AWS in 5 min