Embed Size (px)
Citation preview
Professorship for Computer ScienceCommunication Services, Telecommunication Systems and Computer Networks
High Availability Performanceof Kubernetes and Docker Swarmon a Raspberry Pi ClusterHighLoad++ Conf | Moskow
November 8, 2016
@MathiasRenner | Hypriot.com
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Content
1 Motivation
2 Why Kubernetes and Docker?
3 Related Work
4 Test Setup
5 Testing Scenarios
6 Results
7 Live Demo
8 Outlook
9 Lessons Learned
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 2/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Motivation
Observing paradigm shift: From large to small devices, in size and power.
Architecture size shrinksEnergy efficiency increasesLow-cost hardware availability increases
→ Internet of Things
Contributing factors
Educational added value (physical, bare metal). Same as big machines, just lesspowerful.Cluster for Distributed Systems/Microservices/SOAVery popularOwn contribution for Docker on ARM
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Motivation
Observing paradigm shift: From large to small devices, in size and power.
Architecture size shrinks
Energy efficiency increasesLow-cost hardware availability increases
→ Internet of Things
Contributing factors
Educational added value (physical, bare metal). Same as big machines, just lesspowerful.Cluster for Distributed Systems/Microservices/SOAVery popularOwn contribution for Docker on ARM
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Motivation
Observing paradigm shift: From large to small devices, in size and power.
Architecture size shrinksEnergy efficiency increases
Low-cost hardware availability increases→ Internet of Things
Contributing factors
Educational added value (physical, bare metal). Same as big machines, just lesspowerful.Cluster for Distributed Systems/Microservices/SOAVery popularOwn contribution for Docker on ARM
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Motivation
Observing paradigm shift: From large to small devices, in size and power.
Architecture size shrinksEnergy efficiency increasesLow-cost hardware availability increases
→ Internet of Things
Contributing factors
Educational added value (physical, bare metal). Same as big machines, just lesspowerful.Cluster for Distributed Systems/Microservices/SOAVery popularOwn contribution for Docker on ARM
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Motivation
Observing paradigm shift: From large to small devices, in size and power.
Architecture size shrinksEnergy efficiency increasesLow-cost hardware availability increases
→ Internet of Things
Contributing factors
Educational added value (physical, bare metal). Same as big machines, just lesspowerful.Cluster for Distributed Systems/Microservices/SOAVery popularOwn contribution for Docker on ARM
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Motivation
Observing paradigm shift: From large to small devices, in size and power.
Architecture size shrinksEnergy efficiency increasesLow-cost hardware availability increases
→ Internet of Things
Contributing factors
Educational added value (physical, bare metal). Same as big machines, just lesspowerful.
Cluster for Distributed Systems/Microservices/SOAVery popularOwn contribution for Docker on ARM
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Motivation
Observing paradigm shift: From large to small devices, in size and power.
Architecture size shrinksEnergy efficiency increasesLow-cost hardware availability increases
→ Internet of Things
Contributing factors
Educational added value (physical, bare metal). Same as big machines, just lesspowerful.Cluster for Distributed Systems/Microservices/SOA
Very popularOwn contribution for Docker on ARM
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Motivation
Observing paradigm shift: From large to small devices, in size and power.
Architecture size shrinksEnergy efficiency increasesLow-cost hardware availability increases
→ Internet of Things
Contributing factors
Educational added value (physical, bare metal). Same as big machines, just lesspowerful.Cluster for Distributed Systems/Microservices/SOAVery popular
Own contribution for Docker on ARM
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Motivation
Observing paradigm shift: From large to small devices, in size and power.
Architecture size shrinksEnergy efficiency increasesLow-cost hardware availability increases
→ Internet of Things
Contributing factors
Educational added value (physical, bare metal). Same as big machines, just lesspowerful.Cluster for Distributed Systems/Microservices/SOAVery popularOwn contribution for Docker on ARM
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Why Kubernetes and Docker
Docker
Official support on ARM available since Aug 18, 2016LightweightAnnounced HA features that are very basic, but complete for enough use cases
Kubernetes (K8s)
Recently support for ARM has been extendedKnown as very reliable (based on Google’s Borg DC Manager)Not that ressource-intensive
What about...?
Mesos: Requires too much resources (Netflix got it working), complex setupNomad: Still experimental, in general and on ARM archOpenStack, Pacemaker, Marathon, Apache Spark, Open MPI, ROOT...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Why Kubernetes and Docker
Docker
Official support on ARM available since Aug 18, 2016
LightweightAnnounced HA features that are very basic, but complete for enough use cases
Kubernetes (K8s)
Recently support for ARM has been extendedKnown as very reliable (based on Google’s Borg DC Manager)Not that ressource-intensive
What about...?
Mesos: Requires too much resources (Netflix got it working), complex setupNomad: Still experimental, in general and on ARM archOpenStack, Pacemaker, Marathon, Apache Spark, Open MPI, ROOT...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Why Kubernetes and Docker
Docker
Official support on ARM available since Aug 18, 2016Lightweight
Announced HA features that are very basic, but complete for enough use cases
Kubernetes (K8s)
Recently support for ARM has been extendedKnown as very reliable (based on Google’s Borg DC Manager)Not that ressource-intensive
What about...?
Mesos: Requires too much resources (Netflix got it working), complex setupNomad: Still experimental, in general and on ARM archOpenStack, Pacemaker, Marathon, Apache Spark, Open MPI, ROOT...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Why Kubernetes and Docker
Docker
Official support on ARM available since Aug 18, 2016LightweightAnnounced HA features that are very basic, but complete for enough use cases
Kubernetes (K8s)
Recently support for ARM has been extendedKnown as very reliable (based on Google’s Borg DC Manager)Not that ressource-intensive
What about...?
Mesos: Requires too much resources (Netflix got it working), complex setupNomad: Still experimental, in general and on ARM archOpenStack, Pacemaker, Marathon, Apache Spark, Open MPI, ROOT...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Why Kubernetes and Docker
Docker
Official support on ARM available since Aug 18, 2016LightweightAnnounced HA features that are very basic, but complete for enough use cases
Kubernetes (K8s)
Recently support for ARM has been extendedKnown as very reliable (based on Google’s Borg DC Manager)Not that ressource-intensive
What about...?
Mesos: Requires too much resources (Netflix got it working), complex setupNomad: Still experimental, in general and on ARM archOpenStack, Pacemaker, Marathon, Apache Spark, Open MPI, ROOT...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Why Kubernetes and Docker
Docker
Official support on ARM available since Aug 18, 2016LightweightAnnounced HA features that are very basic, but complete for enough use cases
Kubernetes (K8s)
Recently support for ARM has been extended
Known as very reliable (based on Google’s Borg DC Manager)Not that ressource-intensive
What about...?
Mesos: Requires too much resources (Netflix got it working), complex setupNomad: Still experimental, in general and on ARM archOpenStack, Pacemaker, Marathon, Apache Spark, Open MPI, ROOT...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Why Kubernetes and Docker
Docker
Official support on ARM available since Aug 18, 2016LightweightAnnounced HA features that are very basic, but complete for enough use cases
Kubernetes (K8s)
Recently support for ARM has been extendedKnown as very reliable (based on Google’s Borg DC Manager)
Not that ressource-intensive
What about...?
Mesos: Requires too much resources (Netflix got it working), complex setupNomad: Still experimental, in general and on ARM archOpenStack, Pacemaker, Marathon, Apache Spark, Open MPI, ROOT...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Why Kubernetes and Docker
Docker
Official support on ARM available since Aug 18, 2016LightweightAnnounced HA features that are very basic, but complete for enough use cases
Kubernetes (K8s)
Recently support for ARM has been extendedKnown as very reliable (based on Google’s Borg DC Manager)Not that ressource-intensive
What about...?
Mesos: Requires too much resources (Netflix got it working), complex setupNomad: Still experimental, in general and on ARM archOpenStack, Pacemaker, Marathon, Apache Spark, Open MPI, ROOT...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Why Kubernetes and Docker
Docker
Official support on ARM available since Aug 18, 2016LightweightAnnounced HA features that are very basic, but complete for enough use cases
Kubernetes (K8s)
Recently support for ARM has been extendedKnown as very reliable (based on Google’s Borg DC Manager)Not that ressource-intensive
What about...?
Mesos: Requires too much resources (Netflix got it working), complex setupNomad: Still experimental, in general and on ARM archOpenStack, Pacemaker, Marathon, Apache Spark, Open MPI, ROOT...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Why Kubernetes and Docker
Docker
Official support on ARM available since Aug 18, 2016LightweightAnnounced HA features that are very basic, but complete for enough use cases
Kubernetes (K8s)
Recently support for ARM has been extendedKnown as very reliable (based on Google’s Borg DC Manager)Not that ressource-intensive
What about...?
Mesos: Requires too much resources (Netflix got it working), complex setup
Nomad: Still experimental, in general and on ARM archOpenStack, Pacemaker, Marathon, Apache Spark, Open MPI, ROOT...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Why Kubernetes and Docker
Docker
Official support on ARM available since Aug 18, 2016LightweightAnnounced HA features that are very basic, but complete for enough use cases
Kubernetes (K8s)
Recently support for ARM has been extendedKnown as very reliable (based on Google’s Borg DC Manager)Not that ressource-intensive
What about...?
Mesos: Requires too much resources (Netflix got it working), complex setupNomad: Still experimental, in general and on ARM arch
OpenStack, Pacemaker, Marathon, Apache Spark, Open MPI, ROOT...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Why Kubernetes and Docker
Docker
Official support on ARM available since Aug 18, 2016LightweightAnnounced HA features that are very basic, but complete for enough use cases
Kubernetes (K8s)
Recently support for ARM has been extendedKnown as very reliable (based on Google’s Borg DC Manager)Not that ressource-intensive
What about...?
Mesos: Requires too much resources (Netflix got it working), complex setupNomad: Still experimental, in general and on ARM archOpenStack, Pacemaker, Marathon, Apache Spark, Open MPI, ROOT...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Related Work
In 2012: first public project about cloud computing with Raspberry Pis (the firstmodel) has been started by researchers at the University of Glasgow
LXC was state of the art, everything non-container virtualization tooresource-hungry
Largest project: Bolzano University with 300 RPis
In 2015: Turning point: Lucas Kaldstrom and University of Glasgow started atsame time porting Kubernetes to ARM - independently.
Lucas implemented to most features and recently became Kubernetes maintainer.
In June 2016: Kaspar Nissen and Martin Jensen from Ahrhus University were firstto run comprehensive load and high availability tests with Kubernetes on RaspberryPis. Most versatile investigation about Cloud Computing at small scale.
This work’s contribution: Extend tests of Nissen/Jensen and compare K8sagainst Docker Swarm.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Related Work
In 2012: first public project about cloud computing with Raspberry Pis (the firstmodel) has been started by researchers at the University of Glasgow
LXC was state of the art, everything non-container virtualization tooresource-hungry
Largest project: Bolzano University with 300 RPis
In 2015: Turning point: Lucas Kaldstrom and University of Glasgow started atsame time porting Kubernetes to ARM - independently.
Lucas implemented to most features and recently became Kubernetes maintainer.
In June 2016: Kaspar Nissen and Martin Jensen from Ahrhus University were firstto run comprehensive load and high availability tests with Kubernetes on RaspberryPis. Most versatile investigation about Cloud Computing at small scale.
This work’s contribution: Extend tests of Nissen/Jensen and compare K8sagainst Docker Swarm.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Related Work
In 2012: first public project about cloud computing with Raspberry Pis (the firstmodel) has been started by researchers at the University of Glasgow
LXC was state of the art, everything non-container virtualization tooresource-hungry
Largest project: Bolzano University with 300 RPis
In 2015: Turning point: Lucas Kaldstrom and University of Glasgow started atsame time porting Kubernetes to ARM - independently.
Lucas implemented to most features and recently became Kubernetes maintainer.
In June 2016: Kaspar Nissen and Martin Jensen from Ahrhus University were firstto run comprehensive load and high availability tests with Kubernetes on RaspberryPis. Most versatile investigation about Cloud Computing at small scale.
This work’s contribution: Extend tests of Nissen/Jensen and compare K8sagainst Docker Swarm.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Related Work
In 2012: first public project about cloud computing with Raspberry Pis (the firstmodel) has been started by researchers at the University of Glasgow
LXC was state of the art, everything non-container virtualization tooresource-hungry
Largest project: Bolzano University with 300 RPis
In 2015: Turning point: Lucas Kaldstrom and University of Glasgow started atsame time porting Kubernetes to ARM - independently.
Lucas implemented to most features and recently became Kubernetes maintainer.
In June 2016: Kaspar Nissen and Martin Jensen from Ahrhus University were firstto run comprehensive load and high availability tests with Kubernetes on RaspberryPis. Most versatile investigation about Cloud Computing at small scale.
This work’s contribution: Extend tests of Nissen/Jensen and compare K8sagainst Docker Swarm.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Related Work
In 2012: first public project about cloud computing with Raspberry Pis (the firstmodel) has been started by researchers at the University of Glasgow
LXC was state of the art, everything non-container virtualization tooresource-hungry
Largest project: Bolzano University with 300 RPis
In 2015: Turning point: Lucas Kaldstrom and University of Glasgow started atsame time porting Kubernetes to ARM - independently.
Lucas implemented to most features and recently became Kubernetes maintainer.
In June 2016: Kaspar Nissen and Martin Jensen from Ahrhus University were firstto run comprehensive load and high availability tests with Kubernetes on RaspberryPis. Most versatile investigation about Cloud Computing at small scale.
This work’s contribution: Extend tests of Nissen/Jensen and compare K8sagainst Docker Swarm.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Related Work
In 2012: first public project about cloud computing with Raspberry Pis (the firstmodel) has been started by researchers at the University of Glasgow
LXC was state of the art, everything non-container virtualization tooresource-hungry
Largest project: Bolzano University with 300 RPis
In 2015: Turning point: Lucas Kaldstrom and University of Glasgow started atsame time porting Kubernetes to ARM - independently.
Lucas implemented to most features and recently became Kubernetes maintainer.
In June 2016: Kaspar Nissen and Martin Jensen from Ahrhus University were firstto run comprehensive load and high availability tests with Kubernetes on RaspberryPis. Most versatile investigation about Cloud Computing at small scale.
This work’s contribution: Extend tests of Nissen/Jensen and compare K8sagainst Docker Swarm.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Related Work
In 2012: first public project about cloud computing with Raspberry Pis (the firstmodel) has been started by researchers at the University of Glasgow
LXC was state of the art, everything non-container virtualization tooresource-hungry
Largest project: Bolzano University with 300 RPis
In 2015: Turning point: Lucas Kaldstrom and University of Glasgow started atsame time porting Kubernetes to ARM - independently.
Lucas implemented to most features and recently became Kubernetes maintainer.
In June 2016: Kaspar Nissen and Martin Jensen from Ahrhus University were firstto run comprehensive load and high availability tests with Kubernetes on RaspberryPis. Most versatile investigation about Cloud Computing at small scale.
This work’s contribution: Extend tests of Nissen/Jensen and compare K8sagainst Docker Swarm.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Related Work
In 2012: first public project about cloud computing with Raspberry Pis (the firstmodel) has been started by researchers at the University of Glasgow
LXC was state of the art, everything non-container virtualization tooresource-hungry
Largest project: Bolzano University with 300 RPis
In 2015: Turning point: Lucas Kaldstrom and University of Glasgow started atsame time porting Kubernetes to ARM - independently.
Lucas implemented to most features and recently became Kubernetes maintainer.
In June 2016: Kaspar Nissen and Martin Jensen from Ahrhus University were firstto run comprehensive load and high availability tests with Kubernetes on RaspberryPis. Most versatile investigation about Cloud Computing at small scale.
This work’s contribution: Extend tests of Nissen/Jensen and compare K8sagainst Docker Swarm.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Hardware Setup with PicoCluster
Figure 1: Pico Cluster 5xRPi 3, https://www.picocluster.com
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 6/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Setup aspects
Goal: Meet all requirements for good experiments.
Hardware
All SD cards of same typeCheck network switchIsolate network...
Software
Which OS? Raspbian vs. HypriotOS vs. ArchLinuxFlashing (I stopped counting), device-init...Network: 2 interfaces per device, check DHCPs ...Update SoftwareMonitoring system that supports all required capabilitiesConfigure Desktop machine: No network interference, appropriate software tomanage the cluster (terminator)Setup NTPSSH listen on two interfacesSetup SwapspaceAutomate, automate, automate!...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Setup aspects
Goal: Meet all requirements for good experiments.
HardwareAll SD cards of same type
Check network switchIsolate network...
Software
Which OS? Raspbian vs. HypriotOS vs. ArchLinuxFlashing (I stopped counting), device-init...Network: 2 interfaces per device, check DHCPs ...Update SoftwareMonitoring system that supports all required capabilitiesConfigure Desktop machine: No network interference, appropriate software tomanage the cluster (terminator)Setup NTPSSH listen on two interfacesSetup SwapspaceAutomate, automate, automate!...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Setup aspects
Goal: Meet all requirements for good experiments.
HardwareAll SD cards of same typeCheck network switch
Isolate network...
Software
Which OS? Raspbian vs. HypriotOS vs. ArchLinuxFlashing (I stopped counting), device-init...Network: 2 interfaces per device, check DHCPs ...Update SoftwareMonitoring system that supports all required capabilitiesConfigure Desktop machine: No network interference, appropriate software tomanage the cluster (terminator)Setup NTPSSH listen on two interfacesSetup SwapspaceAutomate, automate, automate!...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Setup aspects
Goal: Meet all requirements for good experiments.
HardwareAll SD cards of same typeCheck network switchIsolate network
...
Software
Which OS? Raspbian vs. HypriotOS vs. ArchLinuxFlashing (I stopped counting), device-init...Network: 2 interfaces per device, check DHCPs ...Update SoftwareMonitoring system that supports all required capabilitiesConfigure Desktop machine: No network interference, appropriate software tomanage the cluster (terminator)Setup NTPSSH listen on two interfacesSetup SwapspaceAutomate, automate, automate!...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Setup aspects
Goal: Meet all requirements for good experiments.
HardwareAll SD cards of same typeCheck network switchIsolate network...
Software
Which OS? Raspbian vs. HypriotOS vs. ArchLinuxFlashing (I stopped counting), device-init...Network: 2 interfaces per device, check DHCPs ...Update SoftwareMonitoring system that supports all required capabilitiesConfigure Desktop machine: No network interference, appropriate software tomanage the cluster (terminator)Setup NTPSSH listen on two interfacesSetup SwapspaceAutomate, automate, automate!...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Setup aspects
Goal: Meet all requirements for good experiments.
HardwareAll SD cards of same typeCheck network switchIsolate network...
SoftwareWhich OS? Raspbian vs. HypriotOS vs. ArchLinux
Flashing (I stopped counting), device-init...Network: 2 interfaces per device, check DHCPs ...Update SoftwareMonitoring system that supports all required capabilitiesConfigure Desktop machine: No network interference, appropriate software tomanage the cluster (terminator)Setup NTPSSH listen on two interfacesSetup SwapspaceAutomate, automate, automate!...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Setup aspects
Goal: Meet all requirements for good experiments.
HardwareAll SD cards of same typeCheck network switchIsolate network...
SoftwareWhich OS? Raspbian vs. HypriotOS vs. ArchLinuxFlashing (I stopped counting), device-init...
Network: 2 interfaces per device, check DHCPs ...Update SoftwareMonitoring system that supports all required capabilitiesConfigure Desktop machine: No network interference, appropriate software tomanage the cluster (terminator)Setup NTPSSH listen on two interfacesSetup SwapspaceAutomate, automate, automate!...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Setup aspects
Goal: Meet all requirements for good experiments.
HardwareAll SD cards of same typeCheck network switchIsolate network...
SoftwareWhich OS? Raspbian vs. HypriotOS vs. ArchLinuxFlashing (I stopped counting), device-init...Network: 2 interfaces per device, check DHCPs ...
Update SoftwareMonitoring system that supports all required capabilitiesConfigure Desktop machine: No network interference, appropriate software tomanage the cluster (terminator)Setup NTPSSH listen on two interfacesSetup SwapspaceAutomate, automate, automate!...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Setup aspects
Goal: Meet all requirements for good experiments.
HardwareAll SD cards of same typeCheck network switchIsolate network...
SoftwareWhich OS? Raspbian vs. HypriotOS vs. ArchLinuxFlashing (I stopped counting), device-init...Network: 2 interfaces per device, check DHCPs ...Update Software
Monitoring system that supports all required capabilitiesConfigure Desktop machine: No network interference, appropriate software tomanage the cluster (terminator)Setup NTPSSH listen on two interfacesSetup SwapspaceAutomate, automate, automate!...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Setup aspects
Goal: Meet all requirements for good experiments.
HardwareAll SD cards of same typeCheck network switchIsolate network...
SoftwareWhich OS? Raspbian vs. HypriotOS vs. ArchLinuxFlashing (I stopped counting), device-init...Network: 2 interfaces per device, check DHCPs ...Update SoftwareMonitoring system that supports all required capabilities
Configure Desktop machine: No network interference, appropriate software tomanage the cluster (terminator)Setup NTPSSH listen on two interfacesSetup SwapspaceAutomate, automate, automate!...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Setup aspects
Goal: Meet all requirements for good experiments.
HardwareAll SD cards of same typeCheck network switchIsolate network...
SoftwareWhich OS? Raspbian vs. HypriotOS vs. ArchLinuxFlashing (I stopped counting), device-init...Network: 2 interfaces per device, check DHCPs ...Update SoftwareMonitoring system that supports all required capabilitiesConfigure Desktop machine: No network interference, appropriate software tomanage the cluster (terminator)
Setup NTPSSH listen on two interfacesSetup SwapspaceAutomate, automate, automate!...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Setup aspects
Goal: Meet all requirements for good experiments.
HardwareAll SD cards of same typeCheck network switchIsolate network...
SoftwareWhich OS? Raspbian vs. HypriotOS vs. ArchLinuxFlashing (I stopped counting), device-init...Network: 2 interfaces per device, check DHCPs ...Update SoftwareMonitoring system that supports all required capabilitiesConfigure Desktop machine: No network interference, appropriate software tomanage the cluster (terminator)Setup NTP
SSH listen on two interfacesSetup SwapspaceAutomate, automate, automate!...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Setup aspects
Goal: Meet all requirements for good experiments.
HardwareAll SD cards of same typeCheck network switchIsolate network...
SoftwareWhich OS? Raspbian vs. HypriotOS vs. ArchLinuxFlashing (I stopped counting), device-init...Network: 2 interfaces per device, check DHCPs ...Update SoftwareMonitoring system that supports all required capabilitiesConfigure Desktop machine: No network interference, appropriate software tomanage the cluster (terminator)Setup NTPSSH listen on two interfaces
Setup SwapspaceAutomate, automate, automate!...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Setup aspects
Goal: Meet all requirements for good experiments.
HardwareAll SD cards of same typeCheck network switchIsolate network...
SoftwareWhich OS? Raspbian vs. HypriotOS vs. ArchLinuxFlashing (I stopped counting), device-init...Network: 2 interfaces per device, check DHCPs ...Update SoftwareMonitoring system that supports all required capabilitiesConfigure Desktop machine: No network interference, appropriate software tomanage the cluster (terminator)Setup NTPSSH listen on two interfacesSetup Swapspace
Automate, automate, automate!...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Setup aspects
Goal: Meet all requirements for good experiments.
HardwareAll SD cards of same typeCheck network switchIsolate network...
SoftwareWhich OS? Raspbian vs. HypriotOS vs. ArchLinuxFlashing (I stopped counting), device-init...Network: 2 interfaces per device, check DHCPs ...Update SoftwareMonitoring system that supports all required capabilitiesConfigure Desktop machine: No network interference, appropriate software tomanage the cluster (terminator)Setup NTPSSH listen on two interfacesSetup SwapspaceAutomate, automate, automate!
...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Setup aspects
Goal: Meet all requirements for good experiments.
HardwareAll SD cards of same typeCheck network switchIsolate network...
SoftwareWhich OS? Raspbian vs. HypriotOS vs. ArchLinuxFlashing (I stopped counting), device-init...Network: 2 interfaces per device, check DHCPs ...Update SoftwareMonitoring system that supports all required capabilitiesConfigure Desktop machine: No network interference, appropriate software tomanage the cluster (terminator)Setup NTPSSH listen on two interfacesSetup SwapspaceAutomate, automate, automate!...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Software Setup: Docker
Easy since included in HypriotOS.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 8/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Software Setup: Kubernetes
Using project: ”Kubernetes-on-arm” by Lucas Kaldstrom
Bleeding edge: Setup non-deterministic and unreliable. Many trials, experimentsand bug shooting.
Installed Kubernetes via automated interactive call:/bin/echo -e " rpi-3\nhypriotos\nnode-green\nEurope\/Berlin\n
\n\ny\nY" | sudo kube-config install
Recently, setup became as easy as with Docker.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 9/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Software Setup: Kubernetes
Using project: ”Kubernetes-on-arm” by Lucas Kaldstrom
Bleeding edge: Setup non-deterministic and unreliable. Many trials, experimentsand bug shooting.
Installed Kubernetes via automated interactive call:/bin/echo -e " rpi-3\nhypriotos\nnode-green\nEurope\/Berlin\n
\n\ny\nY" | sudo kube-config install
Recently, setup became as easy as with Docker.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 9/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Software Setup: Kubernetes
Using project: ”Kubernetes-on-arm” by Lucas Kaldstrom
Bleeding edge: Setup non-deterministic and unreliable. Many trials, experimentsand bug shooting.
Installed Kubernetes via automated interactive call:/bin/echo -e " rpi-3\nhypriotos\nnode-green\nEurope\/Berlin\n
\n\ny\nY" | sudo kube-config install
Recently, setup became as easy as with Docker.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 9/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Software Setup: Kubernetes
Using project: ”Kubernetes-on-arm” by Lucas Kaldstrom
Bleeding edge: Setup non-deterministic and unreliable. Many trials, experimentsand bug shooting.
Installed Kubernetes via automated interactive call:/bin/echo -e " rpi-3\nhypriotos\nnode-green\nEurope\/Berlin\n
\n\ny\nY" | sudo kube-config install
Recently, setup became as easy as with Docker.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 9/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Software stack for experiments
Kubernetes Stack Docker Swarm Stack
rpicloud/greeting-rpi rpicloud/greeting-rpiK8s@ARM
Docker DockerHypriotOS HypriotOS
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 10/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Testing Scenarios
Resource Usage
Load Test
Resilience Test
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 11/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Testing Scenarios
Resource Usage
Load Test
Resilience Test
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 11/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Testing Scenarios
Resource Usage
Load Test
Resilience Test
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 11/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Results – Resource Usage
Table 1: First results of resource consumption measurements
Sc. CPU usage, % Mem usage, % Proc CPU D., % Proc mem D., MB1 0.44 4.21 0.05 2.472 1.43 / 0.83 7.87 / 7.4 1.1 / 0.44 57.45 / 2.423 14.74 / 2.01 34.21 / 12.68 5.76 / 1.31 479.51 / 160.77
Legend:Scenario 1: Fresh install, monit and Docker daemon runningScenario 2: After docker initScenario 3: After starting K8s
Also, in Scenario 3, Docker created 41 process children. In Scenario 1 & 2, Docker has 1.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 12/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
And then the Big Bang...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 13/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Results – HA
Kernel panics and systemd services/daemons crashed. Often reconfiguration andreboots required.
Mesh routing: Produce timeouts in both K8s & Docker.
Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during arequest, and sometimes not even accepting inbound connections at all.”1
→ Both tools were basically not working to the minimum extent.
1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-modeHigh Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Results – HA
Kernel panics and systemd services/daemons crashed. Often reconfiguration andreboots required.
Mesh routing: Produce timeouts in both K8s & Docker.
Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during arequest, and sometimes not even accepting inbound connections at all.”1
→ Both tools were basically not working to the minimum extent.
1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-modeHigh Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Results – HA
Kernel panics and systemd services/daemons crashed. Often reconfiguration andreboots required.
Mesh routing: Produce timeouts in both K8s & Docker.
Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during arequest, and sometimes not even accepting inbound connections at all.”1
→ Both tools were basically not working to the minimum extent.
1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-modeHigh Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Results – HA
Kernel panics and systemd services/daemons crashed. Often reconfiguration andreboots required.
Mesh routing: Produce timeouts in both K8s & Docker.
Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during arequest, and sometimes not even accepting inbound connections at all.”1
→ Both tools were basically not working to the minimum extent.
1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-modeHigh Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Results – HA
Kernel panics and systemd services/daemons crashed. Often reconfiguration andreboots required.
Mesh routing: Produce timeouts in both K8s & Docker.
Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during arequest, and sometimes not even accepting inbound connections at all.”1
→ Both tools were basically not working to the minimum extent.
1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-modeHigh Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Results – HA
Kernel panics and systemd services/daemons crashed. Often reconfiguration andreboots required.
Mesh routing: Produce timeouts in both K8s & Docker.
Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during arequest, and sometimes not even accepting inbound connections at all.”1
→ Both tools were basically not working to the minimum extent.
1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-modeHigh Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Network Issues
Figure 2: Timeout of curl command against the cluster
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 15/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Network Issues
Figure 3: Incoming network traffic
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 16/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Network Issues
Figure 4: Incoming network traffic
Figure 5: Incoming network traffic
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 17/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Network Issues
Figure 6: One of the resolved routing issues for Docker 1.12.2
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 18/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Plan B...
Throw most of the effort away
and figure out a Plan B...
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 19/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Final Software stack of experiments
Kubernetes Stack Docker Swarm Stack(Nissen/Jensen)
rpicloud/greeting-rpi rpicloud/greeting-rpiK8s v1.2 + K8s@ARM v0.7
Docker v1.11.x Docker v1.12.3ArchLinux ARM latest HypriotOS v1.1.0
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 20/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Results - Load Test
Figure 7: Breakpoints of load test with Kubernetes
Test has performed with 10.000 req at eatch step. Stepsize=100
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 21/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Results - Load Test
Figure 8: Breakpoints of load test with Docker Swarm Mode
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 22/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Results – Load Test
Table 2: Breakpoints of Kubernetes and Docker Swarm Mode
Replica Count Kubernetes Docker Swarm Mode1 ca. 900 reqests/seconds ca. 600 reqests/seconds5 ca. 1400 reqests/seconds ca. 900 reqests/seconds10 ca. 1500 reqests/seconds ca. 900 reqests/seconds
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 23/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Results - Resilience Test
Figure 9: Successful requests onto the Kubernetes cluster with 1 replica
Kubernetes: Total outage of 47 secondsDocker Swarm Mode: Total outage of 72 seconds (+53%)
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 24/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Results - Resilience Test
Figure 10: Successful requests onto the cluster with 2 replica
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 25/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Results – Resilience Test
Table 3: Success rates of Kubernetes and Docker Swarm Mode
Replica Count Kubernetes Docker Swarm Mode1 74.65% 61.74%2 99.87% 99.85%5 99.98% 99.93%
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 26/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Live Demo:”Let’s crash the cluster!”
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 27/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Outlook
Add and test high available storage (GlusterFS, Ceph, HDFS) and extendapplications to a microservices level
Use single board computers with higher network I/O...Paper of this work will soon be released.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 28/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Outlook
Add and test high available storage (GlusterFS, Ceph, HDFS) and extendapplications to a microservices levelUse single board computers with higher network I/O...
Paper of this work will soon be released.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 28/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Outlook
Add and test high available storage (GlusterFS, Ceph, HDFS) and extendapplications to a microservices levelUse single board computers with higher network I/O...Paper of this work will soon be released.
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 28/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Lessons Learned
Don’t trust! Measure!
Don’t trust! Measure!
Don’t trust! Measure!
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 29/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Lessons Learned
Don’t trust! Measure!
Don’t trust! Measure!
Don’t trust! Measure!
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 29/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Lessons Learned
Don’t trust! Measure!
Don’t trust! Measure!
Don’t trust! Measure!
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 29/30
Motivation
Why Kubernetes andDocker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
Questions ?
@MathiasRennerwww.Hypriot.com
High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 30/30
![Hypriot Cluster Lab Marcel Großmann · An ARM-Powered Cloud Solution Utilizing Docker Marcel Großmann Andreas Eiermann ... Raspberry Pi Zero [Picture: raspberrypi-spy.co.uk] Hypriot](https://img.pdfslide.net/doc/110x75/5fc6fd9ab2af7505d21b26ed/hypriot-cluster-lab-marcel-gromann-an-arm-powered-cloud-solution-utilizing-docker.jpg)
