61
AWS re:Invent © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Jayme Cox, Cloud Computing, Pinterest Kaartik Viswanath, Product Manager, AWS October 2015 NET307 Pinterest The Road from EC2-Classic to EC2-VPC

(NET307) Pinterest: The road from EC2-Classic To EC2-VPC

Embed Size (px)

Citation preview

Page 1: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Jayme Cox, Cloud Computing, PinterestKaartik Viswanath, Product Manager, AWS

October 2015

NET307

PinterestThe Road from EC2-Classic to EC2-VPC

Page 2: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

Frequently asked Questions

1. When should I start adopting Amazon VPC?

2. Why should I adopt Amazon VPC?

3. How do I go about the process?

Page 3: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Overview

3

Howwhat tools, processes, and procedures

Whyreasons to migrate to VPC

Whentimelines and schedules

Lessons Learnedwhat to think about before migrating

1

2

3

4

What we will talk about

Page 4: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent 4

•100 million pinners

•150,000 requests/sec at peak

•100% in Amazon EC2

What is Pinterest?

4

We help people discover things they

love, and inspire them to do those

things in their daily lives

Page 5: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent 5

Why

How

When

Lessons

Page 6: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Performance

Security

Access

Benefits of VPC

6

Page 7: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Enhanced networking

● EC2-Classic: 250,000 pps

● EC2-VPC: 900,000 pps

● EC2-Classic: ~8.5 Gbit/sec

● EC2-VPC: ~9.9 Gbit/sec

Performance

7

make it fast

Page 8: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Enhanced networking

● faster!

Internal ELB

● replace DNS roundrobin

● replace haproxy/nginx nodes

● health checks

Performance

8

make it fast

Page 9: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Performance

Security

Access

Benefits of VPC

9

Page 10: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Network level controls

● Private Amazon S3 endpoint

Security

10

make it secure

Page 11: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Network level controls

● Private Amazon S3 endpoint

● Subnets and full IP address control

Security

11

make it secure

Page 12: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Network level controls

● Private Amazon S3 endpoint

● Subnets and full IP address control

● Network-level Access Control Lists

Security

12

make it secure

Page 13: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Performance

Security

Access

Benefits of VPC

13

Page 14: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Access Options

● VPC Peering allows secure connections to multiple VPCs

Access

14

make it available

Page 15: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Access Options

● VPC Peering allows secure connections to multiple VPCs

● AWS Direct Connect private connectivity to VPC

Access

15

make it available

Page 16: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Access Options

● VPC Peering allows secure connections to multiple VPCs

● AWS Direct Connect private connectivity to VPC

● Virtual Private Gateway allows VPN access

Access

16

make it available

Page 17: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent 17

Why

How

When

Lessons

Page 18: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

The Pinterest migration to VPC required that

we have zero downtime.

A hard cutover would have duplicated 40%

of our current infrastructure, which was too

expensive.

ClassicLink to the rescue!

18

Page 19: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

ClassicLink● Phased migration on a service-by-service basis

Tools

19

make it go

Page 20: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

ClassicLink● Phased migration on a service-by-service basis

● VPC address space must not overlap EC2-Classic

Tools

20

make it go

Page 21: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

ClassicLink● Phased migration on a service-by-service basis

● VPC address space must not overlap EC2-Classic

● Automate linking classic EC2 nodes to VPC

Tools

21

make it go

Page 22: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

ClassicLink● Phased migration on a service-by-service basis

● VPC address space must not overlap EC2-Classic

● Automate linking classic EC2 nodes to VPC

Service discovery● What services do you need to migrate?

Tools

22

make it go

Page 23: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

ClassicLink● Phased migration on a service-by-service basis

● VPC address space must not overlap EC2-Classic

● Automate linking classic EC2 nodes to VPC

Service discovery● What services do you need to migrate?

● And what services do they depend on?

Tools

23

make it go

Page 24: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

ClassicLink● Phased migration on a service-by-service basis

● VPC address space must not overlap EC2-Classic

● Automate linking classic EC2 nodes to VPC

Service discovery● What services do you need to migrate?

● And what services do they depend on?

Tracking system● Master ticket for each service

Tools

24

make it go

Page 25: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

ClassicLink● Phased migration on a service-by-service basis

● VPC address space must not overlap EC2-Classic

● Automate linking classic EC2 nodes to VPC

Service discovery● What services do you need to migrate?

● And what services do they depend on?

Tracking system● Master ticket for each service

● Migration status of each service

Tools

25

make it go

Page 26: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

ClassicLink● Phased migration on a service-by-service basis

● VPC address space must not overlap EC2-Classic

● Automate linking classic EC2 nodes to VPC

Service discovery● What services do you need to migrate?

● And what services do they depend on?

Tracking system● Master ticket for each service

● Migration status of each service

● Track migration rate

Tools

26

make it go

Page 27: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

ClassicLink● Phased migration on a service-by-service basis

● VPC address space must not overlap EC2-Classic

● Automate linking classic EC2 nodes to VPC

Service discovery● What services do you need to migrate?

● And what services do they depend on?

Tracking system● Master ticket for each service

● Migration status of each service

● Track migration rate

● Track problems and areas to improve

Tools

27

make it go

Page 28: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

NAT gateways● Private subnets require a NAT gateway for egress

● Possible network bottleneck

● Large subnets may need multiple NAT gateways

Design

28

each piece of the puzzle

Page 29: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

NAT gateways● Private subnets require a NAT gateway for egress

● Possible network bottleneck

● Large subnets may need multiple NAT gateways

Subnets● Allow for future growth in each subnet

● Balance size of subnet vs. network segmentation

● Use public subnets for high traffic Internet egress

Design

29

each piece of the puzzle

Page 30: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

NAT gateways● Private subnets require a NAT gateway for egress

● Possible network bottleneck

● Large subnets may need multiple NAT gateways

Subnets● Allow for future growth in each subnet

● Balance size of subnet vs. network segmentation

● Use public subnets for high traffic Internet egress

Security groups● Plan out based on number of rules allowed

● Contiguous subnets allow for CIDR-based rules

● Plan for ClassicLink access from EC2 private address space

Design

30

each piece of the puzzle

Page 31: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Gather information● Create a template for service owners to fill in

Document

31

the migration process

Page 32: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Gather information● Create a template for service owners to fill in

● Document current setup

Document

32

the migration process

Page 33: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Gather information● Create a template for service owners to fill in

● Document current setup

● Document changes required by VPC

Document

33

the migration process

Page 34: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Gather information● Create a template for service owners to fill in

● Document current setup

● Document changes required by VPC

● Service dependencies

Document

34

the migration process

Page 35: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Gather information● Create a template for service owners to fill in

● Document current setup

● Document changes required by VPC

● Service dependencies

Service migration plan● Service runbooks and dashboards

Document

35

the migration process

Page 36: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Gather information● Create a template for service owners to fill in

● Document current setup

● Document changes required by VPC

● Service dependencies

Service migration plan● Service runbooks and dashboards

● Canary and Testing

Document

36

the migration process

Page 37: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Gather information● Create a template for service owners to fill in

● Document current setup

● Document changes required by VPC

● Service dependencies

Service migration plan● Service runbooks and dashboards

● Canary and Testing

● Full rollout to VPC

Document

37

the migration process

Page 38: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Gather information● Create a template for service owners to fill in

● Document current setup

● Document changes required by VPC

● Service dependencies

Service migration plan● Service runbooks and dashboards

● Canary and Testing

● Full rollout to VPC

● Rollback plan

Document

38

the migration process

Page 39: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Stateful clusters● Sync to replica hosts in VPC

Architect

39

for each type of service

Page 40: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Stateful clusters● Sync to replica hosts in VPC

● Promote VPC cluster to master

● Decom classic hosts

Architect

40

for each type of service

Page 41: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Stateful clusters● Sync to replica hosts in VPC

● Promote VPC cluster to master

● Decom classic hosts

Stateless● Mixed pools

● Grow VPC

Architect

41

for each type of service

Page 42: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Stateful clusters● Sync to replica hosts in VPC

● Promote VPC cluster to master

● Decom classic hosts

Stateless● Mixed pools

● Grow VPC

● Decom classic hosts

Architect

42

for each type of service

Page 43: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Stateful clusters● Sync to replica hosts in VPC

● Promote VPC cluster to master

● Decom classic hosts

Stateless● Mixed pools

● Grow VPC

● Decom classic hosts

Frontend traffic● Create new VPC ELB

Architect

43

for each type of service

Page 44: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Stateful clusters● Sync to replica hosts in VPC

● Promote VPC cluster to master

● Decom classic hosts

Stateless● Mixed pools

● Grow VPC

● Decom classic hosts

Frontend traffic● Create new VPC ELB

● Register VPC hosts

Architect

44

for each type of service

Page 45: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Stateful clusters● Sync to replica hosts in VPC

● Promote VPC cluster to master

● Decom classic hosts

Stateless● Mixed pools

● Grow VPC

● Decom classic hosts

Frontend traffic● Create new VPC ELB

● Register VPC hosts

● Migrated traffic via DNS

Architect

45

for each type of service

Page 46: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent 46

Why

How

When

Lessons

Page 47: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Timeline● Build dependency tree to avoid overlap

● Schedule each service

● Meet weekly to track progress

Coordinate

47

each service

Page 48: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Timeline● Build dependency tree to avoid overlap

● Schedule each service

● Meet weekly to track progress

Tracking● Tag each service before and after

● Track cost via tags

● Build tooling/dashboard

Coordinate

48

each service

Page 49: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Timeline● Build dependency tree to avoid overlap

● Schedule each service

● Meet weekly to track progress

Tracking● Tag each service before and after

● Track cost via tags

● Build tooling/dashboard

Service owners● Add to service owners’ roadmap

● Service owners fill out migration template

● Use master ticket to keep in sync

Coordinate

49

each service

Page 50: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent 50

Why

How

When

Lessons

Page 51: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Limit scope of changes● Try to limit the number of changes during migration● Do not change application versions, OS revs, etc.

Favorite things

51

to think about

Page 52: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Limit scope of changes● Try to limit the number of changes during migration● Do not change application versions, OS revs, etc.

Network ACL● Optional; you may only need security groups

● Tricky with the private S3 endpoint using public IPs

Favorite things

52

to think about

Page 53: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Limit scope of changes● Try to limit the number of changes during migration● Do not change application versions, OS revs, etc.

Network ACL● Optional; you may only need security groups

● Tricky with the private S3 endpoint using public IPs

Service discovery● Difficult to do, so start before migration, it will pay off

● Building the service dependency map will smooth the migration

● Helps to track down port numbers used for security groups

● Build tooling to automate and report

Favorite things

53

to think about

Page 54: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Subnets● Spend time to get it right

● Public vs. Private

● Contiguous IP address space (CIDR)

Favorite things

54

to think about

Page 55: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Subnets● Spend time to get it right

● Public vs. Private

● Contiguous IP address space (CIDR)

Security groups● Start with deny all

● Work towards open

Favorite things

55

to think about

Page 56: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Subnets● Spend time to get it right

● Public vs. Private

● Contiguous IP address space (CIDR)

Security groups● Start with deny all

● Work towards open

amazonaws.com● split DNS works in classic

● but in VPC it only returns public IP

Favorite things

56

to think about

Page 57: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Long tail● ClassicLink allows mixed environment

● Encourage service owners to migrate

Favorite things

57

to think about

Page 58: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Long tail● ClassicLink allows mixed environment

● Encourage service owners to migrate

Cost● Someone cares about cost

● So track it from the beginning

Favorite things

58

to think about

Page 59: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Questions?

Page 60: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Thank you!

Page 61: (NET307) Pinterest: The road from EC2-Classic To EC2-VPC

AWS re:Invent

Remember to complete

your evaluations!