Upload
amazon-web-services
View
5.087
Download
6
Embed Size (px)
DESCRIPTION
In this session, you learn why Twilio chose to migrate from Amazon EC2-Classic to VPC and how they leveraged features available only in VPC, specifically: - AWS CloudHSM: Build out a secure key encryption or role-based access control for internal use; also used to securely store and encrypt data for external customers.- Elastic Network Interface (ENI): Allows multiple Elastic IPs per instance and the ability to move network interface between instances.- Hardware Virtual Machine (HVM) instances w/SRV-IO: New hardware virtualized instances that allow line-level performance of network interfaces for up to 10g Ethernet speeds. Secure data-in-transit by default, which ensures all machines communicate via a software-defined network and work in the same manner as VLAN tagging for compliance reasons. Sponsored by Twilio.
Citation preview
© 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
November 12, 2014 | Las Vegas
SDD302
A Tale of One Thousand Instances - Migrating from
EC2-Classic to EC2-VPC
Donald Sumbry, Twilio
Jonas Borjesson, Twilio
Hi, I’m Jonas
@borjessonjonasTech Lead of the SIP Connectivity team at Twilio
Hi, I’m Sumbry
@sumbryDirector of Cloud Services at Twilio
Background
We provide a communications API that enables
phones, VoIP, and messaging to be embedded into
web, desktop, and mobile software.
twilio
twilio
How does it work?
A user calls your
number
Twilio receives the call Your app responds
twilio6 years ago, things were simple
• Few machines on EC2-Classic
• Single service - PSTN to HTTP
• US-based customers
• Single AWS region
ProcessingNode
SIP
RTP
ProcessingNode
ProcessingNode
Load
Balancer
Load
Balancer
Database
twilioNow, not so simple anymore
• Thousands of instances
• Many services
• Global customers
• Resources in every AWS region
SIP
RTP
Virginia
IrelandBrazil
SIP
RTP
Signalling
Issues
All regions completely separated
– Traffic has to go through known endpoints
• Un-necessary hops
• Complicates deployments
• More difficult to debug
• Easier to create routing bugs
Forcing us to…
• Open up firewalls
• Secure traffic between regions using our own “VPN”
• Traffic has to go through known endpoints
• Known endpoints assigned EIPs
Which means…
• Un-necessary hops
• Complicates deployments
• More difficult to debug
• Easier to introduce bugs
• Cannot deploy nodes behind EIPs without affecting
traffic
That translates to…
• Fewer deploys
• Riskier deploys
• Harder to nail down bugs
• Takes longer to get fixes out
• Less happy customers!
Why VPC?
“EC2 2.0” (aka EC2-VPC)
• Global routing tables
• Enhanced Networking with SR-IOV
• Elastic network interfaces
• Software defined network
• Hardware security manager
Twilio considers VPC an evolutionary step or upgrade of the Amazon EC2
platform.
Global routing tables• Per subnet or per VPC routing tables
• Route traffic to instances
• Tunnel traffic between regions
Routing traffic to instances enables the easy creation of things like load
balancers, tunnels, or even VPCs inside of VPCs.
HVM and SR-IOV• HVM images with Enhanced Networking
• PCI Express speeds to network adapter
• Low-latency access to network adapter
• Up to 10gb network speeds
Enhanced Networking with SR-IOV means fast performance even under
virtualized hardware.
Elastic network interfaces• Multiple EIPs and multiple private IPs
• Multiple ENIs per instance
• Security groups follow an ENI
• ENI has a MAC address
ENIs are more like network cards that you can move around and attach to
different instances.
Software defined network• Control over my instances’ routes
• Number my own network
• Network ACLs
• Data-in-transit protected by more than just a
security group
• Provision networks like virtual machines
Use of a software defined network solves the data-in-transit issue that many
certifications require.
Hardware security manager• Easily integrates with IAM policies
• Centralized management of keys and certificates
• Easily and quickly encrypt customer data
Use of the HSM solves the data-at-rest issue that many certifications require.
Twilio Cloud Requirements
Twilio Cloud Requirements• Services can be deployed anywhere
• Services can communicate anywhere
• Services can be discovered anywhere
Solving the issue of global service discovery is easy once the underlying cloud
infrastructure is in place.
US1US2
BR1 AU1
JP1
SG1
IE1
DE1
EC2-VPC Building Blocks• Global routing tables
• HVM and SR-IOV
• Elastic Network Interfaces
• Software Defined Network
• Hardware Security Manager
Region-to-region connectivity
Performing routing among multiple VPCs in different regions is a bit more
complicated and necessitates the use of a routing protocol.
router router
us-east-1 / 10.1.0.0 us-west-2 / 10.2.0.0
vpc-abcdef vpc-zyxwv
IPSEC tunnelhosthosthosthost
US1US2
BR1 AU1
JP1
SG1
IE1
DE1
VPC-enabled infrastructure
SIP
RTP
Virginia
IrelandBrazil
Which may look insignificant, but...
• A single global network
– Global service discovery
• Much easier call flow
– Easier to debug
– Less risk to deploy
– More frequent deploys
– Call setup latency down 25%
• Less infrastructure and complexity
Also…
• Blocking firewall rules
– Important for stopping attacks
• ENI
– Aid us in deploying new edge services
– Improved network performance
– Better audio quality
Happier customers!
Migrating from EC2-Classic to
EC2-VPC
Migration Requirements• Equivalent to moving a datacenter
– Zero downtime
– Bridge traffic between services in a region
– Easily discover services in EC2-Classic or
EC2-VPC
Peering vs bridging
Peering is two VPCs talking in different regions.
Bridging is EC2-Classic and EC2-VPC in the same account talking in the same region.
vpc-bbb
vpc-aaa
vpc-aaa
classicus-east-1
us-west-2us-east-1
Migrating from EC2-Classic to EC2-VPC
• Use IP Tunnel Manager for bridging traffic
• Use software routers for peering traffic
• Use Service Discovery for discovering new
services as they move
Make sure any services you want to move from EC2-Classic to EC2-VPC
share the same AWS account and are in the same region!
Conclusion
• Services can be deployed globally
• Services can communicate globally
• Services can be discovered globally
• New VoIP infrastructure deployed in:
– all regions around the world
– taking live traffic for new products
– existing carrier traffic is being migrated
Where we are today
How could this have been easier?
• Feature to bridge EC2-Classic and EC2-VPC
• Feature to connect VPCs in different regions
Are you listening, AWS? Maybe. :-)
http://bit.ly/awsevals