Stonefly is the innovator of worldwide storage standard protocol
DOCKER EXPLAINEDDocker Basics & Fundamentals
By Ahmed Murkaz
Stonefly Inc.26250 Eden Landing Hayward, California, USA
Docker Explained
Absract
This course will cover the basics and fundamentals of Docker and Docker infrastruc-
ture, how to manage your Docker containers and images; you will learn the advanced
usage of Docker and the Docker containers and your Docker infrastructure, you will
learn how to run your containers in an enterprise environment with external storage
providers and with advanced networking infrastructure. You will learn how to run
your Docker containers on a public cloud just like Amazon or Google cloud engine.
The difference between traditional virtualization and containers will be covered,
what's the difference between full containers or OS containers and application con-
tainers. You will learn the basics of a Docker image, and how a Docker image looks
like, and you will learn the basics of a Docker infrastructure and what parts it com-
prises.
1
Docker Explained
What is tradi�onal virtualiza�on and how it works?
In traditional virtualization you use your general infrastructure, your servers, storages,
switches and so on, and you put an “Operating System” (OS) like Windows or Linux
onto your servers and after that you put a so-called “Hypervisor” on top of the OS. In
case of Linux you will install the KVM, and in case of Windows or Windows Core
Server you will install the Hyper-V feature onto that. So Hyper-V and KVM are hypervi-
sors, but there are several specialized OSs which consist the “Host Operating System”
and the “Hypervisor” in one (Just like the VMware ESX or XenServer which provides
you with an OS and a hypervisor itself in one installable package).
2
After you install your “Host Operating System” and the “hypervisor”, you can create
Virtual Machines (VMs) and the hypervisor will emulate the hardware to these VMs. It
will emulate the Motherboard, CPU, memory, disk, and Network Interface Card (NIC),
and you will install the “Guest OS” into these VMs. The guest OS installation is simple,
Docker Explained
there are no special requirements because the hardware is emulated, however, it
comes with a price and performance penalty as everything is emulated. So when you
write data onto the disk in the VM, first you have to write it out in the VM disk, but the
VM disk is a real file on the host OS, so you have to write out that data onto the real
disk into a real file. There is at least double write or cures and the same happens with
the NIC and so on.
We have specialized drivers called para-virtualized drivers and when you install it into
the “Guest OS” it boosts the performance as it evades the performance penalty by
talking directly to the disk or the file on the “Host Operating System”. It helps you a
lot but still it's just like a real server, so you will have to administer the server's VM just
like a normal server. You have to patch all the applications of the OS.
Traditional virtualization has the advantage that you can install different OSs on top
each other. Let's say you have an Ubuntu host OS with a KVM Hypervisor, you can
install a windows server or a Linux server or a FreeBSD server on top of that. It doesn't
matter because there is a separate kernel for all the VMs.
3
Docker Explained
In the first place traditional virtualization is used to consolidate the hardware and the
infrastructure, so we have a better usage of the infrastructure, because you can put
more OSs, more machines on the real servers, since in real life 80% or 90% of all the
time the OS is just idle, or just uses a 10% or 20% of the hardware resources. If you
put more machines, more servers into one real server you have better average con-
sumption, you will spend less money by putting more servers on one physical ma-
chine.
One physical machine consumes less electricity than three different servers. This is
the main reason behind the traditional virtualization; better usage of the infrastruc-
ture. On the second thought you can just think about different software versions and
different libraries which are not compatible with each other.
So let's say you have four applications and you want to put them onto one server, but
they are requiring different versions of a software which are not compatible with
each other. Consider that you want to use MySQL 6 and you want to use MySQL 5,
PHP 7 and PHP 5.6 and they are not compatible with each other. But your applica-
tions maybe they just require different versions and you can just manage it but it's
very hard, because you can install them into different directories, work with different
environment variables, but it will be very hard to maintain patch and update all
these things.
On the other hand you use traditional virtualization to isolate the processes and to
give the different applications different virtual machines. That's much easier to
manage, i.e. different virtual machines with applications than one real server with five
different applications. We have lots of tools to orchestrate and schedule these virtual
machines. You can just install them with the help of a “Chef” or “Puppet” or anything
like that so you can automate all these processes. Yet still you have performance and
management overhead as well because you have to patch all the OSs and it con-
sumes more hard disk, more storage and so on.
Why use tradi�onal virtualiza�on?
4
Docker Explained
So we have another virtualization method called containers. The first occurrence of
the containers happened just like Full containers of OS containers, and they are
almost the same as the traditional virtualization because you use them to isolate and
separate the different roles, different customers or different applications and you
install whole OSs into the containers. You have a full boot sequence, there's an SSHD
or a remote desktop. If it is a windows you have a MySQL server and an Apache server
in one container, and you just put another version of a particular server and PHP and
MySQL going to another container and they're with their full boot sequence, and you
can use the same upgrade methods just like with the traditional virtualization. You
can use almost the same orchestration tools, you can use “Chef” and “Puppet” to
deploy these containers onto the “Host Operating System”.
There's a very big difference between traditional virtualization and a container virtual-
ization. This difference is that you cannot use different OS on top of each other. You
cannot use a Linux on top of a windows server, and you cannot install a windows
server on top of an “Ubuntu” host OS
Containers
5
Docker Explained 6
Docker Explained
With a container virtualization you have the real servers and the host OS, and you
have a container management service just like V-server, Alex-C and so on. There are
lots of different flavors of this container management service but one thing is coming
in all of them that they isolate different processes. So when you install guest OS you
just install it into a directory of the host OS, and when you boot the container it just
starts the processes and the processes are isolated from each other. So if you have
this green guest OS and this orange guest OS, they are just processes and there's no
hardware emulation.Despite the obvious benefits of cloud storage, many organiza-
tions remain wary of moving to the cloud. Traditional storage may be costly,
resource-intensive and cumbersome, but it also performs well, is secure, highly avail-
able, and just works.
7
Docker Explained
So in traditional virtualization because the hardware, i.e. the motherboard, CPU and
everything is emulated, you can install different Kernels into these virtual machines,
but with containers you use the same Kernel on the host OS. So the Kernel is one you
cannot use different kernels in guest OSs other than the host OS because a container
is just a process and the processes in the container are just processes on the host OS,
they are just separated from each other and separated from the host OS processes as
well, and you can put limits on them. So you can limit the memory and CPU usage of
a container, but you can only run Linux container on top of Linux OS, and you can run
only windows containers on a windows server. But it has a very big advantage that is
it’s very fast. It is nearly as fast as a real server because there is no emulation.
It's much faster than a traditional virtualization. If you run Linux on Linux, it doesn't
mean that if your host OS is an “Ubuntu” that you cannot run CentOS Linux as a guest
OS, only the Kernel will be the same but all the libraries, server versions and so on will
be the guest OSs’ version.
8
Docker Explained 9
So you can run CentOS, SUSE or a different version of “Ubuntu” on an “Ubuntu” host
OS but still it looks like a real server with all the boot sequences and all the softwares.
You can manage and login to the guest OSs as we have discussed before.
The very big difference between an application container, an Operating System (OS)
container and the traditional virtualization is that you won’t ship full OSs in an appli-
cation container. We have several application container engines, but Docker is the
most famous and the most mature.
Full Containers vs. Applica�on Containers
So you still use your servers and you put an OS on top of that, let's say you will install
an “Ubuntu” operating system, but you will install the Docker engine on top of that
What is an applica�on container?
Docker Explained 10
and we are not ship and deploy for guest OSs with full boot sequences, SSH, Daemon
or mail server Daemon or an apache web server in one package or in one directory,
you just put your application and all the required libraries and software in one pack-
age. So if you have a WordPress installed in an OS container, you would install an SSH
daemon, an apache web server, a PHP library to MySQL and mail server on top of
that guest OS.
With an application container you use different containers and let's say again that
WordPress example works like the following; that you will put your WordPress and
your custom theme into the green application container, and we will put the MySQL
server into another container let's say the orange one.
Then you link them together, so you will only pack your WordPress and WordPress
team into one container just like the apache web server and the PHP FISE, and you
will use standard MySQL image in another container.
Docker Explained
Why is it good for you?
If you are a developer you can just pack your application into one standardized
format and you can just deploy your station anywhere Docker is supported. Let's
say you have your custom software package into the red application container
and it doesn't really matter whether you deploy it onto Amazon, Microsoft Azure,
Google Cloud Engine (GCE) or your own laptop, it will work the same way because
all the required dependencies, binaries, and libraries are in one package.
11
You can just compare an application container to a portable executable container,
that if the runtime environment supports your application, you can run it just like
portable executable versions of Mozilla Firefox. You can run it on any windows version;
it’s the same.
If you have a Docker container and this Docker container consists of Linux binaries
and Linux applications, you can run it on any server that supports the Docker Engine
and the operating system is Linux and it works in the same way with windows server
as well.
Docker Explained 12
From a developer point of view it’s just very easy to pack your applications into one
package and run it anywhere, and from the infrastructure guys’ point of view if you
want an infrastructure engineer you don't have to work with dependency house, and
you don't have to provide several scripts just to maintain state of the servers.
You only have to provide the Docker Engine and the basic infrastructure and devel-
opers will take care of everything else. As the application container just like a Docker
container is very lightweight, you can run more applications on one OS on one server
than with OS containers over the traditional virtualization. A traditional virtualization
just depends on your hardware, but sometimes you can just run 10-20 virtual ma-
chines (VMs) on one server, however, with a Docker application container you can run
just like 100 containers; It just depends on how much memory it uses and how much
memory do you have, but you have the possibility, and this application container
format is very good at running the so-called Micro-Services.
So if you have one web application and you need to deploy like 50 more from that
web application you just deploy 50 containers and you don't have to wait for an OS
installation or use it from a template, because you just deploy your web application
and Docker Engine will take care of all the other things. Another big advantage
beside the standard format and the lower resource usage, is that it's much easier to
deploy new versions, because let's say App 1 and App 2 are just two different versions
but from the same software. When you want to deploy a newer version from your
software you just deploy the App 2 container and just stop the App 1 and if anything
goes wrong it's very easy to downgrade or roll back the software installation because
you just start up the App 1 container and just shut down the App 2 container. And
when you deploy you don't have to wait for infrastructure guys, and as an infrastruc-
ture guy you won't have extra work just to make sure that the newer version of a
software can be rolled out. These are the biggest advantages of a Docker environ-
ment.
Docker Explained 13
Basics of a Docker image
Docker Explained 14
The Docker environment works with Docker images, so the containers are the run-
ning versions of an image and an image is executable and it has several layers. When
you start a container you will start up an image and make it run and let's say in this
example you have an Apache container and you have that Apache image but
“Apache” needs an OS as well to run all the libraries, so it doesn't require a full OS
because the host OS provides all of the memory management and so on, so you only
have to have the binaries required to run the Apache. You put your Apache into one
image and maybe it has dependencies so it has references to another parent image
as shown in this example from Docker.com.
It will just reference to that other image and it just grabs that image as well, and that
image references to the Debian “Base Image” and the Debian “Base Image” is not
referenced to anything because it's a base image. When you want to run the Apache
container you grab the Apache image and the image will point you to other images
and at the end Docker will put all the layers on top of each other.
It will just grab the “Base Image”, put the “emacs image” on top of that and the
“Apache image” on top of that, and finally it will create a “read/writeable” layer
Docker Explained 15
on top of that so any change will be written into that “read/writeable” layer because
the images are immutable. This means you can use the same image for several con-
tainers. So if you want to run 100 Apache containers, you use the same image and
they will be only different with the “read/writeable” layer, because every image will
have a different “read/writeable” layer. But this layer is ephemeral. This means when
you remove the container that “writeable” layer will be lost. If you delete all the
Apache binaries you just remove the container and start a new container from the
same image, and because the images is immutable all the changes are lost, because
they leave only interactive container, nothing is lost.
Everything is just happening in the “writeable” layer but not in the “Base Image”, and
because you can use the same image for several containers, it consumes less storage
space because in a full container or in a traditional virtualization if you have an image
with one gigabyte in size and you start or deploy 100 containers it would consume
100 gigabytes. But in a Docker infrastructure if you have an image with one gigabyte
in size and you deploy 100 containers you still use the same image because only the
“read/writeable” layer is different. It will only consume one gigabyte disk space and
the “read/writeable” layer’s disk space.
All you have to know is that Docker containers are coming from images and these
images are immutable and the “Containers” are ephemeral or the “read/writeable”
layer of the containers are ephemeral.
So if you remove a container everything will be lost which has been written into this
“read/writeable” layer.
You will see in later that there are persistent data stores, what you can use with con-
tainers. So if you have a real application you won’t lose your data if you just remove a
container. So don't be scared when you hear that the “read/writeable” layer is ephem-
eral and the images are immutable because you can store your changes.
Docker Explained 16
Docker infrastructure
Docker Explained 17
At the end let's see how a Docker infrastructure looks like. This is a very simple Docker
infrastructure because you have your Docker host let's say that's an “Ubuntu” or
CentOS Linux and you have the Docker Daemon on top of that Docker host.
When you want to start up a container right now you know that Docker uses images
to create containers. So the Docker daemon will look for the images locally, and if it
doesn't find the images it will pull down from the so-called registry, so you store your
images on a registry and these images will be downloaded when necessary and will
be stored on a Docker host to create containers from these images.
So the Docker client just issues comment to Docker demon and the Docker demon
will behave just instructed. These are the basics of the infrastructure. The image for
the container can be pulled from a public registry or a private registry. It just depends
on the configuration, and it will just start up a container.
Later you can remove the locally stored images, and you can upgrade the images
from the registry and so on.