Upload
plesk
View
1.961
Download
0
Embed Size (px)
Citation preview
Jan Löffler, CTO Plesk
AutoScaling WordPress with Docker & AWS
code.talks 2016
Sebastian Vettel
Who uses WordPress?
Usain Bolt Katy Perry Snoop Dog LL Cool J
Kim Kardashian The Rolling Stones Sylvester Stallone Mötley Crüe
Beyoncé Justin Bieber Justin Timberlake Alanis Morissette
Usain Bolt Katy Perry Snoop Dog LL Cool J
Kim Kardashian Mötley Crüe
Beyoncé Alanis MorissetteJustin Bieber
TechCrunch Metro The New Yorker BBC America
MTV News The Official Star Wars Blog Sony Music Facebook Newsroom
Time.com CNN UPS Mozilla Blog
http://w3techs.com/technologies/history_overview/content_management/all/y
# WEBSITES WORLDWIDE
1.085.358.537http://www.internetlivestats.com/total-number-of-websites/
# WEBSITES WORLDWIDE
1.085.358.692http://www.internetlivestats.com/total-number-of-websites/
38% of E-Shops
26,6%
2,7%
2,2%
#SPEED
High Traffic Sites???
NEED
FOR
SPEED
my-wordpress-site.com
my-wordpress-site.com
MYSQL ServerLoad-Balancer
my-wordpress-site.com
MYSQL ServerLoad-Balancer
File Storage
my-wordpress-site.com
MYSQL Server
Load-Balancer
File Storage
CDN
MYSQL Server
Easy-peasy, isn’t it?
DEPLOYMENT
doubled to 30%http://www.rightscale.com/blog/cloud-industry-insights/cloud-computing-trends-2016-state-cloud-survey
my-wordpress-site
Modern Web-Development
2016
IMMUTABLE INFRASTRUCTURE
IMMUTABLE STACKS
ELB myapp-v1
jans-wordpress.com
EC2+ Docker
EC2+ Docker
EC2+ Docker
100%
IMMUTABLE STACKS
ELB myapp-v1
EC2+ Docker
EC2+ Docker
EC2+ Docker
ELB myapp-v2
EC2+ Docker
EC2+ Docker
jans-wordpress.com
90% 10%
IMMUTABLE STACKS
ELB myapp-v1
EC2+ Docker
EC2+ Docker
EC2+ Docker
ELB myapp-v2
EC2+ Docker
EC2+ Docker
jans-wordpress.com
0% 100%
APP INSTANCE
VM (EC2)
Basis Image (AMI)
Docker Container
Application
✓ VPC✓ Security
Groups✓ AutoScaling✓ CloudWatch✓ CloudFront✓ Route53
✓ Docker Runtime✓ Logging✓ Monitoring✓ Security config
✓ Debian✓ Nginx✓ php7-fpm✓ NewRelic
AWS
API
Docker Registry
docker pull
docker push
AMI
THE WISH
THE REALITY
$ accept-vpc-peering-connection$ $ allocate-address$ $ allocate-hosts$ $ assign-private-ip-addresses$ $ associate-address$ $ associate-dhcp-options$ $ associate-route-table$ $ attach-classic-link-vpc$ $ attach-internet-gateway$ $ attach-network-interface$ $ attach-volume$ $ attach-vpn-gateway$ $ authorize-security-group-egress$ $ authorize-security-group-ingress$ $ bundle-instance$ $ cancel-bundle-task$ $ cancel-conversion-task$ $ cancel-export-task$ $ cancel-import-task$ $ cancel-reserved-instances-listing$ $ cancel-spot-fleet-requests$ $ cancel-spot-instance-requests$ $ confirm-product-instance$ $ copy-image$ $ copy-snapshot$ $ create-customer-gateway$ $ create-dhcp-options$ $ create-flow-logs$ $ create-image$ $ create-instance-export-task$ $ create-internet-gateway$ $ create-key-pair$ $ create-nat-gateway$ $ create-network-acl$ $ create-network-acl-entry$ $ create-network-interface
$ create-reserved-instances-listing$ $ create-route$ $ create-route-table$ $ create-security-group$ $ create-snapshot$ $ create-spot-datafeed-subscription$ $ create-subnet$ $ create-tags$ $ create-volume$ $ create-vpc$ $ create-vpc-endpoint$ $ create-vpc-peering-connection$ $ create-vpn-connection$ $ create-vpn-connection-route$ $ create-vpn-gateway$ $ delete-customer-gateway$ $ delete-dhcp-options$ $ delete-flow-logs$ $ delete-internet-gateway$ $ delete-key-pair$ $ delete-nat-gateway$ $ delete-network-acl$ $ delete-network-acl-entry$ $ delete-network-interface$ $ delete-placement-group$ $ delete-route$ $ delete-route-table$ $ delete-security-group$ $ delete-snapshot$ $ delete-spot-datafeed-subscription$ $ delete-subnet$ $ delete-tags$ $ delete-volume$ $ delete-vpc$ $ delete-vpc-endpoints$ $ delete-vpc-peering-connection$
$ delete-vpn-connection$ $ delete-vpn-connection-route$ $ delete-vpn-gateway$ $ deregister-image$ $ describe-account-attributes$ $ describe-addresses$ $ describe-availability-zones$ $ describe-bundle-tasks$ $ describe-classic-link-instances$ $ describe-conversion-tasks$ $ describe-customer-gateways$ $ describe-dhcp-options$ $ describe-export-tasks$ $ describe-flow-logs$ $ describe-host-reservation-offerings$ $ describe-host-reservations$ $ describe-hosts$ $ describe-id-format$ $ describe-identity-id-format$ $ describe-image-attribute$ $ describe-images$ $ describe-import-image-tasks$ $ describe-import-snapshot-tasks$ $ describe-instance-attribute$ $ describe-instance-status$ $ describe-instances$ $ describe-internet-gateways$ $ describe-key-pairs$ $ describe-moving-addresses$ $ describe-nat-gateways$ $ describe-network-acls$ $ describe-network-interface-attribute$ $ describe-network-interfaces$ $ describe-placement-groups$ $ describe-prefix-lists$ $ describe-regions$
$ describe-reserved-instances$ $ describe-reserved-instances$ describe-reserved-instancesmodifications$ $ describe-reserved-instances$ describe-route-tables$ $ describe-scheduled-$ describe-scheduled-$ describe-security-group$ describe-security-groups$ $ describe-snapshot-attribute$ $ describe-snapshots$ $ describe-spot-datafeed$ describe-spot-fleet$ describe-spot-fleet$ describe-spot-fleet$ describe-spot-instance$ describe-spot-price$ describe-stale-security$ describe-subnets$ $ describe-tags$ $ describe-volume-attribute$ $ describe-volume-status$ $ describe-volumes$ $ describe-vpc-attribute$ $ describe-vpc-classic$ describe-vpc-classic$ describe-vpc-endpoint$ describe-vpc-endpoints$ $ describe-vpc-peering$ describe-vpcs$ $ describe-vpn-connections$ $ describe-vpn-gateways$ $ detach-classic-link$ detach-internet-gateway$ $ detach-network-interface$ $ detach-volume$ $ detach-vpn-gateway$
210 calls for EC2 only
$ accept-vpc-peering-connection$ $ allocate-address$ $ allocate-hosts$ $ assign-private-ip-addresses$ $ associate-address$ $ associate-dhcp-options$ $ associate-route-table$ $ attach-classic-link-vpc$ $ attach-internet-gateway$ $ attach-network-interface$ $ attach-volume$ $ attach-vpn-gateway$ $ authorize-security-group-egress$ $ authorize-security-group-ingress$ $ bundle-instance$ $ cancel-bundle-task$ $ cancel-conversion-task$ $ cancel-export-task$ $ cancel-import-task$ $ cancel-reserved-instances-listing$ $ cancel-spot-fleet-requests$ $ cancel-spot-instance-requests$ $ confirm-product-instance$ $ copy-image$ $ copy-snapshot$ $ create-customer-gateway$ $ create-dhcp-options$ $ create-flow-logs$ $ create-image$ $ create-instance-export-task$ $ create-internet-gateway$ $ create-key-pair$ $ create-nat-gateway$ $ create-network-acl$ $ create-network-acl-entry$ $ create-network-interface
$ create-reserved-instances-listing$ $ create-route$ $ create-route-table$ $ create-security-group$ $ create-snapshot$ $ create-spot-datafeed-subscription$ $ create-subnet$ $ create-tags$ $ create-volume$ $ create-vpc$ $ create-vpc-endpoint$ $ create-vpc-peering-connection$ $ create-vpn-connection$ $ create-vpn-connection-route$ $ create-vpn-gateway$ $ delete-customer-gateway$ $ delete-dhcp-options$ $ delete-flow-logs$ $ delete-internet-gateway$ $ delete-key-pair$ $ delete-nat-gateway$ $ delete-network-acl$ $ delete-network-acl-entry$ $ delete-network-interface$ $ delete-placement-group$ $ delete-route$ $ delete-route-table$ $ delete-security-group$ $ delete-snapshot$ $ delete-spot-datafeed-subscription$ $ delete-subnet$ $ delete-tags$ $ delete-volume$ $ delete-vpc$ $ delete-vpc-endpoints$ $ delete-vpc-peering-connection$
$ delete-vpn-connection$ $ delete-vpn-connection-route$ $ delete-vpn-gateway$ $ deregister-image$ $ describe-account-attributes$ $ describe-addresses$ $ describe-availability-zones$ $ describe-bundle-tasks$ $ describe-classic-link-instances$ $ describe-conversion-tasks$ $ describe-customer-gateways$ $ describe-dhcp-options$ $ describe-export-tasks$ $ describe-flow-logs$ $ describe-host-reservation-offerings$ $ describe-host-reservations$ $ describe-hosts$ $ describe-id-format$ $ describe-identity-id-format$ $ describe-image-attribute$ $ describe-images$ $ describe-import-image-tasks$ $ describe-import-snapshot-tasks$ $ describe-instance-attribute$ $ describe-instance-status$ $ describe-instances$ $ describe-internet-gateways$ $ describe-key-pairs$ $ describe-moving-addresses$ $ describe-nat-gateways$ $ describe-network-acls$ $ describe-network-interface-attribute$ $ describe-network-interfaces$ $ describe-placement-groups$ $ describe-prefix-lists$ $ describe-regions$
$ describe-reserved-instances$ $ describe-reserved-instances$ describe-reserved-instancesmodifications$ $ describe-reserved-instances$ describe-route-tables$ $ describe-scheduled-$ describe-scheduled-$ describe-security-group$ describe-security-groups$ $ describe-snapshot-attribute$ $ describe-snapshots$ $ describe-spot-datafeed$ describe-spot-fleet$ describe-spot-fleet$ describe-spot-fleet$ describe-spot-instance$ describe-spot-price$ describe-stale-security$ describe-subnets$ $ describe-tags$ $ describe-volume-attribute$ $ describe-volume-status$ $ describe-volumes$ $ describe-vpc-attribute$ $ describe-vpc-classic$ describe-vpc-classic$ describe-vpc-endpoint$ describe-vpc-endpoints$ $ describe-vpc-peering$ describe-vpcs$ $ describe-vpn-connections$ $ describe-vpn-gateways$ $ detach-classic-link$ detach-internet-gateway$ $ detach-network-interface$ $ detach-volume$ $ detach-vpn-gateway$
210 calls for EC2 only
S3ELB
RDS
CloudFrontCloudWatch
AutoScaling
Route53
Events
IAM
S3api
SNS
PLESK WORDPRESS AWS SCALER
https://github.com/plesk/wordpress-aws-scaler
AWS
manage-wordpress.sh
Docker Registry
docker pull
docker push
AMI
PLESK WORDPRESS AWS SCALER
manage-wordpress.sh create
Too quick? How does it work internally?Install AWS CLI (https://docs.aws.amazon.com/cli/latest/userguide/installing.html)
$ sudo pip install awscli$ aws configure$ aws --version
Clone WordPress AWS Scaler git repo (https://github.com/plesk/wordpress-aws-scaler)
$ git clone https://github.com/plesk/wordpress-aws-scaler.git$ cd wordpress-aws-scaler
Build Docker Image and Upload to Docker Hub (https://hub.docker.com/explore/)
$ docker build -t janloeffler/wordpress-aws-scaler:latest .$ docker push janloeffler/wordpress-aws-scaler:latest
List existing AWS resources
List existing AWS Resources
$ aws ec2 describe-vps $ aws ec2 describe-security-groups $ aws ec2 describe-instances$ aws rds describe-db-instances[...]
Do this for all AWS components
$ aws autoscaling $ aws ec2 $ aws cloudfront $ aws cloudwatch $ aws elb $ aws rds $ aws s3 $ aws route53 $ aws events
Create new resources
Create new AWS Resources
$ aws ec2 create-vpc$ aws ec2 create-security-group[...]
Do this for all AWS components
$ aws autoscaling $ aws ec2 $ aws cloudfront $ aws cloudwatch $ aws elb $ aws rds $ aws s3 $ aws route53 $ aws events
Create new Launch ConfigurationGenerate EC2-User-Data Script to run directly after booting EC2 VM
$ cat >ec2-user-data.sh <<EOL#!/bin/bashdocker pull janloeffler/wordpress-aws-scaler:latestdocker run -d -p 80:80 -p 443:443 -e WORDPRESS_DB_HOST='${DB}' -e WORDPRESS_DB_USER='${DB_USERNAME}' -e WORDPRESS_DB_PASSWORD='${DB_PASSWORD}' -e WORDPRESS_DB_NAME='${DB_NAME}' -e WORDPRESS_DB_PREFIX='${WORDPRESS_DB_PREFIX}' -e WORDPRESS_URL='http://${ELB}' -e WORDPRESS_TITLE='${WORDPRESS_TITLE}' -e WORDPRESS_USER_EMAIL='${WORDPRESS_USER_EMAIL}' -e NEWRELIC_KEY='${NEWRELIC_KEY}' -e NEWRELIC_NAME='${NEWRELIC_NAME}' -e S3_KEY='${IAM_USER_KEY}' -e S3_SECRET='${IAM_USER_SECRET}' -e S3_BUCKET='${S3_BUCKET_NAME}' -e S3_BUCKET_URL='${S3_URL}' -it janloeffler/wordpress-aws-scaler:latestEOL
Create new Launch Configuration for AutoScaling
$ aws autoscaling create-launch-configuration --launch-configuration-name $LC_NAME --image-id $AMI --instance-type $INSTANCE_TYPE --key-name $KEYNAME --security-groups $SEC_GROUP_ID --user-data file://ec2-user-data.sh
You must love JSON !!! ;-){
"Reservations": [{
"OwnerId": "699328319947","ReservationId": "r-041ac13209e0d4eef",
"Groups": [],
"RequesterId": "226008221399","Instances": [
{"Monitoring": {
"State": "enabled"
},"PublicDnsName": "ec2-52-210-200-157.eu-west-1.compute.amazonaws.com",
"State": {"Code": 16,
"Name": "running"
},"EbsOptimized": false,
"LaunchTime": "2016-09-05T12:19:54.000Z","PublicIpAddress": "52.210.200.157",
"PrivateIpAddress": "172.31.25.83",
"ProductCodes": [{
"ProductCodeId": "4jvb72q6a56js2x7jzd24jar5","ProductCodeType": "marketplace"
}
],"VpcId": "vpc-fffbe19a",
"StateTransitionReason": "","InstanceId": "i-0ea3384a24a3c6aba",
"ImageId": "ami-64385917",
"PrivateDnsName": "ip-172-31-25-83.eu-west-1.compute.internal","KeyName": "Marko Heijnen",
"SecurityGroups": [{
"GroupName": "WordPress-Docker-Test",
"GroupId": "sg-1b8dde7c"}
],
"ClientToken": "70aa81e1-10a3-4b83-acc7-62702015d06e_subnet-cff90cb9_1",
"SubnetId": "subnet-cff90cb9","InstanceType": "m3.medium",
"NetworkInterfaces": [{
"Status": "in-use",
"MacAddress": "06:c0:22:65:b4:ed","SourceDestCheck": true,
"VpcId": "vpc-fffbe19a","Description": "",
"Association": {
"PublicIp": "52.210.200.157","PublicDnsName": "ec2-52-210-200-157.eu-west-1.compute.amazonaws.com",
"IpOwnerId": "amazon"},
"NetworkInterfaceId": "eni-2f4efa63",
"PrivateIpAddresses": [{
"PrivateDnsName": "ip-172-31-25-83.eu-west-1.compute.internal","Association": {
"PublicIp": "52.210.200.157",
"PublicDnsName": "ec2-52-210-200-157.eu-west-1.compute.amazonaws.com","IpOwnerId": "amazon"
},"Primary": true,
"PrivateIpAddress": "172.31.25.83"
}],
"PrivateDnsName": "ip-172-31-25-83.eu-west-1.compute.internal","Attachment": {
"Status": "attached",
"DeviceIndex": 0,"DeleteOnTermination": true,
"AttachmentId": "eni-attach-7f9031a0","AttachTime": "2016-09-05T12:19:54.000Z"
},
[…]
WP CUP – RACE DAY 1
single-server
avg 3.86 sec for 947 hits
auto-scaling
avg 1.45 sec for 9.866 hits
Fastest Lap 216 ms180 RPS < 3sec
Fastest Lap 99 ms1.000 RPS < sec
No Caching!!!
Plesk Mission
“We simplify the lives of web professionals.”
Docker
WordPress CDN
ELB & AutoScaling
DB
Route53
code.talks 2016, Hamburg
“High traffic sites with WordPress, Docker & AWS”
How to auto-scale WordPress on AWS and make it fly!
Passion for Web Development
Jan Löffler
CTO Plesk
● @jlsoft2
Marko Heijnen
WordPress Contributor
● @markoheijnen
https://github.com/plesk/wordpress-aws-scaler
https://www.slideshare.net/plesk
“When you walk through your website architecture and you wonder who could be so stupid to test it with 10.000 parallel requests per second. But if you want to be successful and play in the first league, it is important to think about it before your traffic increases”
Jan Löffler @jlsoft2
https://github.com/plesk/wordpress-aws-scaler
BACKUP
Web Development Key Tools – what web developers need and use
59
26,6% of all websites worldwide
2,7% 2,2%(high traffic sites)
70% of developers use git as primary source management solutionIncreasing usage by web agencies
38% of all online stores worldwide
runs on top of WordPress as plugindeveloped by automattic
used by 30% of IT companiesincreased from 13% to 30% in 12 monthsfurther 35% plan introduction
Sources: w3techs.com, Rightscale, StackOverflow Survey 2015, 2016, Netcraft
311% growth
17.2% usage26.8% growth
17.9% usage14.9% growth
16% of all webservers22% of all active sites
continuously growing, while Apache is shrinking
WordPress Management
Free SSL everywhere
http2 50+ extensions
CLI
Automatic Updates
Site Migration Free support
30+ languages nginxServer Security DNS
✔
Plesk Onyx: Continuous Delivery Deployment Pipeline (sample)
61
Jenkins runs as Docker container managed by Plesk
Plesk Multi Server Extension installs three environments
Apps managed by Plesk via Docker