Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Trust is good, control is betterA short story about Network Policies
Maximilian Bischoff, Johannes M. Scheuermann
Hamburg, 26. June 2019
Maximilian Bischoff
Unofficial: Chaos Monkey
@johscheuer2
Cloud Platform Engineer
Cloud Platform Engineer
Johannes M. Scheuermann
What can you expect ?
● Get an overview about challenges with network policies
● Get an overview on different aspects of testing / validating your setup
3
Why should I test my policies ?
Many adjustment screws
https://www.pexels.com/photo/colorful-toothed-wheels-171198 6
Why should I test my policies ?
Kubernetes doesn’t implement the policies
7
kube-apiserver
Netpol
CNI PluginRead NetPol
Implements them
Why should I test my policies ?
Kubernetes doesn’t implement the policies
8
kube-apiserver
CNI PluginRead NetPol
No Feedback !
Netpol
Implements them
...
spec:
ingress:
- from:
- namespaceSelector:
matchLabels:
team: operations
podSelector:
matchLabels:
type: monitoring
Why should I test?Hard to read policies
https://github.com/ahmetb/kubernetes-network-policy-recipes/blob/master/07-allow-traffic-from-some-pods-in-another-namespace.md10
...
spec:
ingress:
- from:
- namespaceSelector:
matchLabels:
team: operations
- podSelector:
matchLabels:
type: monitoring
and or
Node
Why should I test my policies ?
Component updates
11
kube-proxy
iptables CNI-Plugins
Node
kube-proxy
ipvs CNI-Plugins
Node
kube-proxy
iptables CNI-Plugins v1
Node
kube-proxy
iptables CNI-Pluginsv2
Node
Why should I test my policies ?
Component updates
12
kube-proxy
iptables CNI-Plugins
Node
kube-proxy
ipvs CNI-Plugins
Node
kube-proxy
iptables CNI-Plugins v1
Node
kube-proxy
iptables CNI-Pluginsv2
Is everything still working after an update?
Node
Why should I test my policies ?
Component updates
13
kube-proxy
iptables CNI-Plugins
Node
kube-proxy
ipvs CNI-Plugins
Node
kube-proxy
iptables CNI-Plugins v1
Node
kube-proxy
iptables CNI-Pluginsv2
Conformance tests don’t test network policies!
What to test
15
PolicyPolicyPolicy
SDN Control
PlaneData Plane
Conformance - Effect of policies- Synchronisation
How to test
17
PolicyPolicyPolicy
SDN Control
PlaneData Plane
Conformance - Effect of policies- Synchronisation
Testing strategiesCopy-pod
18
kind: podapiVersion: v1metadata: name: foo namespace: default labels:
app: foospec: containers: - name: foo image: foo:latest ...
kind: podapiVersion: v1metadata: name: foo-test-copy namespace: default labels:
app: footesting.framework: “”
spec: containers: - name: test image: test/runner:latest ...
copy
Testing strategiesLinux namespaces
20
pause(of pod
foo)
test-runner
underlying OS
...
Network
IPC
CGroup
...
IPC
CGroup
kubectl exec -it foo -- wget -qO - --timeout=2 http://bar.defaultwget: download timed out
Manually
https://github.com/ahmetb/kubernetes-network-policy-recipes/blob/master/03-deny-all-non-whitelisted-traffic-in-the-namespace.md21
kind: NetworkPolicyapiVersion: networking.k8s.io/v1metadata: name: default-deny-all namespace: defaultspec: podSelector: {} ingress: []
default
foo barbar
node 1
netassert
https://github.com/controlplaneio/netassert22
default
foo barbar
config.yaml---k8s: deployment: default:foo: default:bar: TCP:80
ssh
docker run --net ...
test.js
nmap
netassert
kind: NetworkPolicyapiVersion: networking.k8s.io/v1metadata: name: demo namespace: defaultspec: podSelector: matchLabels: app: prometheus ingress: ...
Test case generationPreface
24
isolated from every pod
except for those matching
Test case generationTwo kinds of tests
25
pod Aapp=prometheus
pod Bapp=grafana
pod C
...spec: podSelector: matchLabels: app: prometheus ingress: - from: - podSelector: matchLabels: app: grafana
pod Aapp=prometheus
Test case generationMultiple policies
26
pod Bteam=opsapp=foo
...spec: podSelector: matchLabels: app: prometheus ingress: - from: - podSelector: matchLabels: app: grafana
...spec: podSelector: {} ingress: - from: - podSelector: matchLabels: team: ops namespaceSelector: {}
?
How do these tools complement
28
PolicyPolicyPolicy
kubeaudit/
SDN Control
PlaneData Plane
netassert/illuminatio
Recap
● Test your assumptions!
● Regression testing makes your life easier
● Network Policies are still hard to get right○ Missing feedback○ Does it work for Services and Pods?
29
Thank You
Maximilian BischoffIT Engineering & Operations
inovex GmbHLudwig-Erhard-Allee 676131 Karlsruhe
Johannes ScheuermannIT Engineering & Operations
inovex GmbHLudwig-Erhard-Allee 676131 Karlsruhe