Embed Size (px)
Citation preview
DOCKER MEETUP BERLIN
2017-07-19
SANDOR SZÜCS
@sszuecs
Automatic infrastructure for Kubernetes ingress in AWS
2
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
ZALANDO
15 markets
6 fulfillment centers
20 million active customers
3.6 billion € net sales 2016
165 million visits per month
12,000 employees in Europe
3
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
ZALANDO TECHNOLOGY
HOME-BREWED,CUTTING-EDGE& SCALABLEtechnology solutions
>1,700employees from
tech locations+ HQs in Berlin6
77nations
help our brand toWIN ONLINE
4
Put images in the grey dotted box "unsupported placeholder" - behind the orange box (left side stays white)
Write the quote in all capital letters
ZALANDO TECH’SINFRASTRUCTURE
5
Please write the title in all capital letters
FOUR ERAS AT ZALANDO TECH
ZOMCATPHP STUPS KUBERNETES
2010 2015 2016
Data center
WAR
LXC
AWS
Docker
Cloud Formation
Low level (AWS API)
AWS
Docker
Cloud Formation
Kubernetes manifest
Higher abstraction level
Data center
PHP files
6
Please write the title in all capital letters
LARGE SCALE?
9
Put images in the grey dotted box "unsupported placeholder" - behind the orange box (left side stays white)
Write the quote in all capital letters
KUBERNETES:ARCHITECTURE
10
Please write the title in all capital letters
ISOLATED AWS ACCOUNTS
Internet
*.abc.example.org *.xyz.example.org
Product ABC Product XYZ
EC2
LBLB
11
Please write the title in all capital letters
KUBERNETES ON AWS
12
Put images in the grey dotted box "unsupported placeholder" - behind the orange box (left side stays white)
Write the quote in all capital letters
DEPLOYMENT
13
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
DEPLOYMENT CONFIGURATION
.├── apply│ ├── cf-iam-role.yaml # AWS IAM Role│ ├── cf-rds.yaml # AWS RDS Database│ ├── kube-ingress.yaml # K8s Ingress│ ├── kube-secret.yaml # K8s Secret│ └── kube-service.yaml # K8s Service├── deployment.yaml # K8s Deployment└── pipeline.yaml # CI/CD config
14
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
INGRESS.YAML
apiVersion: extensions/v1beta1kind: Ingressmetadata: name: "..."spec: rules: # DNS name your application should be exposed on - host: "myapp.foo.example.org" http: paths: - backend: serviceName: "myapp" servicePort: 80
15
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
JENKINS DEPLOY PIPELINE
16
Put images in the grey dotted box "unsupported placeholder" - behind the orange box (left side stays white)
Write the quote in all capital letters
CHALLENGES
17
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
1. Getting Started
2. Ingress
CHALLENGES
18
Put images in the grey dotted box "unsupported placeholder" - behind the orange box (left side stays white)
Write the quote in all capital letters
CHALLENGE 1:GETTING STARTED
19
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
GETTING STARTED
https://github.com/hjacobs/kubernetes-on-aws-users
20
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
GETTING STARTED
https://github.com/hjacobs/kubernetes-on-aws-users
21
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
CLUSTER PROVISIONING
• Two Cloud Formation stacks
• Master & worker ASGs + etcd
• Nodes w/ Container Linux
• K8s manifests applied separately
• kube-system Deployments
• DaemonSets
22
Please write the title in all capital letters
CLUSTER PROVISIONING
23
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
GETTING STARTED
Other questions we asked ourselves..
• Single AZ vs. Multi AZ? ⇒ Multi AZ
• Federation? ⇒ No, not ready yet
• Overlay network? ⇒ Flannel, “rock solid”
• Authnz? ⇒ OAuth, webhook
24
Put images in the grey dotted box "unsupported placeholder" - behind the orange box (left side stays white)
Write the quote in all capital letters
CHALLENGE 2: Ingress
25
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
Ingress
• System view
• Developer point of view
• Enhancing Kubernetes with
weighted traffic switching
26
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
System View
Goal: use Kubernetes API as primary interface
• External DNS → Route53
• Kubernetes Ingress Controller for AWS → ALB+TLS
• Skipper - http router → your dockerized app
⇒ we wrote new componentsto achieve our goal
27
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
System View - traffic flow
https://github.com/zalando/skipper
ALB
Node Skipper Node Skipper
MyApp MyApp MyApp
Service Service
K8s network
EC2 network
TLS
HTTP
28
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
Kubernetes - ingress - configuration
https://github.com/zalando/skipper https://github.com/zalando-incubator/kube-ingress-aws-controller / https://github.com/kubernetes-incubator/external-dns
29
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
Kubernetes - service - configuration
https://github.com/zalando/skipper https://github.com/zalando-incubator/kube-ingress-aws-controller / https://github.com/kubernetes-incubator/external-dns
Select PODs by label
Target of Ingress definition
Port of the my-app
30
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
Kubernetes - POD - configuration
https://github.com/zalando/skipper https://github.com/zalando-incubator/kube-ingress-aws-controller / https://github.com/kubernetes-incubator/external-dns
Target port
Selected by service
31
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
Skipper
https://github.com/zalando/skipper https://github.com/zalando-incubator/kube-ingress-aws-controller / https://github.com/kubernetes-incubator/external-dns
• Skipper used as main shop router
• Skipper can be used as ingress implementation
32
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
Skipper
https://github.com/zalando/skipper https://github.com/zalando-incubator/kube-ingress-aws-controller / https://github.com/kubernetes-incubator/external-dns
33
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
System View - skipper - configuration
https://github.com/zalando/skipper https://github.com/zalando-incubator/kube-ingress-aws-controller / https://github.com/kubernetes-incubator/external-dns
Skipper reads
34
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
kube-ingress-aws-controller
https://github.com/zalando/skipper https://github.com/zalando-incubator/kube-ingress-aws-controller / https://github.com/kubernetes-incubator/external-dns
35
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
System View - controller - configuration
https://github.com/zalando/skipper https://github.com/zalando-incubator/kube-ingress-aws-controller / https://github.com/kubernetes-incubator/external-dns
kube-ingress-aws-controller
Optional read
write
36
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
System View - external DNS
https://github.com/zalando/skipper https://github.com/zalando-incubator/kube-ingress-aws-controller / https://github.com/kubernetes-incubator/external-dns
37
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
System View - external-dns - configuration
https://github.com/zalando/skipper https://github.com/zalando-incubator/kube-ingress-aws-controller / https://github.com/kubernetes-incubator/external-dns
external-dnsreads
38
Please write the title in all capital letters
Use bullet points to summarize information rather than writing long paragraphs in the text box
System View - summary
● DNS Name pointing to ALB
● ALB + TLS
● Skipper routes
39
Please write the title in all capital letters
Use bullet points to summarize information rather than writing long paragraphs in the text box
Developer point of view
● Define templates
● Execute pipelines
40
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
Developer point of view - defines template
Pipeline expands some variables from pipeline.yaml
41
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
Developer point of view - executes pipeline
42
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
WEIGHTED TRAFFIC SWITCHING
• STUPS - former AWS platform toolkit
• Kubernetes
• supports rolling updates
• enhancement to ingress required
43
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
WEIGHTED TRAFFIC SWITCHING - STUPS
• STUPS uses weighted Route53 DNS records
• Allows canary, blue/green, slow ramp up
• Problem: DNS caching
44
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
WEIGHTED TRAFFIC SWITCHING - Kubernetes
• Approach: add weights to Ingress backends
https://github.com/zalando/skipper
45
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
Traffic Switching - Ingress configuration
https://github.com/zalando/skipper
Skipper reads
46
Please write the title in all capital letters
Put images in the grey dotted box "unsupported placeholder"
Use bullet points to summarize information rather than writing long paragraphs in the text box
LINKS
Running Kubernetes in Production on AWShttp://kubernetes-on-aws.readthedocs.io/en/latest/admin-guide/kubernetes-in-production.html
Kube AWS Ingress Controllerhttps://github.com/zalando-incubator/kube-ingress-aws-controller
Skipper HTTP Ingress Routerhttps://github.com/zalando/skipper/
External DNShttps://github.com/kubernetes-incubator/external-dns
Zalando Cluster Configurationhttps://github.com/zalando-incubator/kubernetes-on-aws
PostgreSQL Operatorhttps://github.com/zalando-incubator/postgres-operator
Please write contact name, department and position in all capital letters
QUESTIONS?
SANDOR SZÜCS
TECH INFRASTRUCTURE
CLOUD ENGINEER
@sszuecs
Slides based on @try_except_
Illustrations by @01k
Please write contact name, department and position in all capital letters
