Kubernetes Networking

Bryan Boreham, Director of Engineering

@bboreham

What does Weave do?

Weave lets devops

iterate faster with:

• observability &

monitoring

• continuous delivery

• container networks &

firewalls

Kubernetes is our #1

platform

What you should learn

1. How clients talk to services in Kubernetes

2. Connecting containers: overlay vs native

3. Connecting into your cluster: NodePort, HostPort, LoadBalancer, Ingress

4. Be better equipped to troubleshoot

https://media.timeout.com/images/103755908/630/472/image.jpg

• Docker

• Kubernetes

• Weave

Who is working with...

What is Kubernetes?

https://image.shutterstock.com/z/stock-photo-vintage-photo-of-a-man-working-on-complex-machine-1392745.jpg

What is Kubernetes?

NodeNode Node

Master

NodeNode Node

Kubernetes runs Services

Let’s talk about Ports

A service listens on a Port at an IP address– e.g. http on port 80, postgres on port 5432– or your own custom service on 9090

192.1.6.4

9090foo

Suppose we want to run two?

• Only one thing can be listening on a port• We could give the second one a new port

number

192.1.6.4

9090

9091

foo1

foo2

How do we keep track?

• Fiddling with port numbers needs a bit of book-keeping• Could have another service where we register all the

port numbers we’ve chosen• Downside: this requires that every client uses the

registry

192.1.6.4

9090

9091

Service Registry

“Where is foo2?”

foo1

foo2

Give every service its own IP address

• Container Networking means never having to say “what port is it on?”

• Every service uses its native port number

10.20.30.429090

foo1

9090foo2

10.20.30.43

Kubernetes Concepts

Pod

IP addr

Node

Container

Just one thing though

• Now, when we contact a service, we need to know its IP address.

• There is a standard way to do that

DNS

“Where is foo2?”

10.20.30.42

foo1

foo2

10.20.30.43

• Run multiple instances of a service• Clients should call one of them, don’t care

which

foo

10.20.10.1

foo

10.20.29.13

Now add Scaling and Redundancy

foo

10.20.30.42

DNS can do this. Right?

• Some clients will re-query on every call

• Some clients will cache the result too long

• Most clients will not round-robin

http://gunshowcomic.com/648

• DNS name resolves to a stable Virtual IP address

• Kube-proxy translates VIP to one Pod IP

Kubernetes Cluster IPs

kube-dns

10.20.30.42

kube-proxy100.96.0.30->10.20.30.42

foo

“Where is foo?”- 100.96.0.30 192.1.6.4

We need network packets to go from one pod to another pod, across whatever sits in the middle

Let’s talk about Pod Networking

10.20.30.42

192.1.6.4192.1.6.3

10.20.9.1

Who controls your network?

http://philippel.deviantart.com/art/DUNE-Sandworm-Rising-403336019

If you have the IP space, and you control the network, just program the routers

Pod Network: Routes

10.20.30.42

192.1.6.4192.1.6.3

10.20.30.0/24:via 192.1.6.4

10.20.9.0/24:via 192.1.6.3

10.20.9.1

Packets are encapsulated before they leave the machine

Pod Network: Overlay

10.20.30.42

192.1.6.4192.1.6.3

10.20.9.1

192.1.6.3->192.1.6.4[10.20.9.1->10.20.30.42]

The Three Commandments

...of Kubernetes Networking:

• All containers can communicate with all other

containers

• All nodes can communicate with all containers

(and vice-versa)

• The IP that a container sees itself as is the same

IP that others see it as

CNI: the Container Network Interface

kubelet

Pod

InterfacePlugin

Pod Network

ADD

• One high-numbered port, on every Node in the cluster• Can bounce from one machine to another

Exposing services: NodePort

10.20.30.42

kube-proxy:30021->10.20.30.42:80

foo

192.1.6.4 :30021

• Specific port is mapped locally on the host• “Don’t use hostPort unless it is absolutely necessary”

Exposing services: HostPort

10.20.30.42

:8080->10.20.30.42:80

foo

192.1.6.4 :8080

• Layer 4 - works for any TCP-based protocol• Available for specific implementations, e.g. ELB

Exposing services: LoadBalancer

LB

foo

cloud-controller

Programs endpoints

kube-proxy / iptables

foo

• Layer 7 - defined for http only• Available for specific implementations, e.g. nginx, ALB

Exposing services: Ingress

ingress controller

foo foo

Master

Example Ingress configapiVersion: extensions/v1beta1kind: Ingressspec:

rules:- host: foo.bar.com

http:paths:- path: /foo

backend:serviceName: s1servicePort: 80

- path: /barbackend:serviceName: s2servicePort: 80

Recap: all you need to know

• Kubernetes runs Pods which implement Services

• Pods need a Pod Network - routed or Overlay

• Pod network is driven via CNI

• Clients connect to Services via virtual Cluster IPs

• Kubernetes has many ways to expose a Service outside the cluster - each has pros and cons

Thanks! Questions?

We are hiring!Engineers in SF & London

weave.works/hiring

What’s Next?

• Try Weave Cloud– https://cloud.weave.works

• Join the Weave user group!– meetup.com/pro/Weave

• Get in touch! (Slack, Email, etc.)– weave.works/help