Building Web Scale Apps with Docker and Mesos
Alexander RukletsovSoftware Engineer @ Mesosphere
Why should we care?
Applications in the Cloud Era
Client-Server Era:Small apps, big servers
Cloud Era:Big apps, small servers
Serv Serv Serv Serv
VirtualizationAggregation
App
Server
App App App App
Major Components
● Hardware
● Orchestration, deployment and isolation
● Cluster and resource management
● Scale-aware applications, service discovery, etc.
Major Components
● Hardware
● Orchestration, deployment and isolation
● Cluster and resource management
● Scale-aware applications, service discovery
Applications in the Cloud Era
Client-Server Era:Small apps, big servers
Cloud Era:Big apps, small servers
Serv Serv Serv Serv
VirtualizationAggregation
App
Server
App App App App
“Divide and rule [and package]”
—Philip II of Macedon, paraphrased
“Divide et impera [et sarcina]”
Containers
● Lightweight Linux execution environment
● Static application composition
● Reliable deployment
● Unit of resource isolation
● Execution isolation
● Multi-tenancy without heavyweight VMs
Docker
● Open source
● Configurable layers
● Reproducible
● Version-controlled
● Plenty of other people’s containers
Docker
● Open source● Configurable layers● Reproducible● Version-controlled● Plenty of other people’s containers
● First-class citizen in Mesos and the Mesosphere stack
● Kubernetes employs and promotes Docker
© Gerard Julien/AFP
Run everything in containers!
What about container management?
© Unknown
© Unknown
Apache Mesos facts
● Created in 2009 at UC Berkeley, hardened in Twitter● Top-level Apache project● Mesosphere, Twitter, and Airbnb are major users /
contributors● Scales to 10 000s of nodes, production grade● Packages and support through Mesosphere● Google officially endorsed Mesos for Kubernetes● Built-in containerization, including Docker
Mesos as a Distributed OS kernel
● Two level resource scheduling● Launch tasks across the cluster● Communication between tasks (like IPC)● APIs for building “native” applications (aka frameworks):
program against the datacenter● APIs in C++, Python, JVM-languages, Go and counting● Pluggable CPU, memory, IO isolation● Multi-tenant workloads● Failure detection● Easy failover and HA
How Mesos works (HA mode)
Zookeeper
MesosMasterMesos
MasterMesosMaster
MesosMaster
MesosSlave
Executor Task
Task
Framework TaskScheduler
Application
Executor
Task
TaskExecutor
MesosSlave
© ect.nl
Manage containers with Mesos!
Mesos + Docker =
Native Docker support in Mesosmessage DockerInfo {
required string image = 1;
// Network options.
enum Network {
HOST = 1;
BRIDGE = 2;
NONE = 3;
}
message PortMapping {
required uint32 host_port = 1;
required uint32 container_port = 2;
optional string protocol = 3; // Protocol to expose (ie: tcp, udp).
}
optional Network network = 2 [default = HOST];
repeated PortMapping port_mappings = 3;
optional bool privileged = 4 [default = false];
// Allowing arbitrary parameters to be passed to docker CLI.
repeated Parameter parameters = 5;
}
Native Docker support in MesosCommandInfo command;
command.set_value("dd if=/dev/zero of=/dev/null");
ContainerInfo::DockerInfo dockerInfo;
dockerInfo.set_image("mesosphere/inky");
ContainerInfo containerInfo;
containerInfo.set_type(ContainerInfo::DOCKER);
containerInfo.mutable_docker()->CopyFrom(dockerInfo);
TaskInfo task;
task.set_name("");
task.mutable_task_id()->set_value("1");
task.mutable_slave_id()->CopyFrom(offer.slave_id());
task.mutable_resources()->CopyFrom(offer.resource());
task.mutable_command()->CopyFrom(command);
task.mutable_container()->CopyFrom(containerInfo);
vector<TaskInfo> tasks;
tasks.push_back(task);
driver.launchTasks(offer.id(), tasks);
Native Docker support in Marathon// nginx-task.json
{
"container": {
"type": "DOCKER",
"docker": {
"image": "nginx",
"network": "BRIDGE",
"portMappings": [
{ "containerPort": 80,
"hostPort": 0,
"servicePort": 80,
"protocol": "tcp" }
]
}
},
"id": "nginx",
"instances": "1",
"cpus": "0.25",
"mem": "256",
"uris": []
}
$ cat nginx-task.json | http http://dev1.mesosphere.com:8080/v2/apps
Future Docker Swarm API support
Mesos
Mesos Framework
Serv
DockerApp
Docker Swarm API
DockerApp
DockerApp
DockerApp
DockerApp
DockerApp
DockerApp
Serv Serv Serv Serv
What Mesos contributes
● Multi-framework: weighted fair sharing, roles, etc.● Run Docker containers alongside other popular frameworks
(e.g. Spark, Rails, Hadoop, …)● Run services and batch apps in the same cluster● Advanced scheduling: resources, constraints, global view● High resource availability, cluster self-healing● Proven at scale, battle-tested in production● GUI / CLI cluster management console
Ways of running Dockers
● Marathoneasy to setup, reliable orchestration
● Multiple Marathonsbenefit from two level scheduling, reservations, framework
roles
● Custom frameworkfine-grained management
“L’homme est libre au moment qu’il veut l’être”
“Man is free at the instant he wants to be”
—Voltaire
Cluster configuration example #1
AWS, DigitalOcean, GCE
Mesos
Hardware
Kernel
Apps
API
DockerApp
DockerApp
Services
Services REST API“Marathon”
DockerApp
AppDocker
AppApp
Cluster configuration example #2
AWS, DigitalOcean, GCE
Mesos
Hardware
Kernel
Apps
API
Services
Services REST API“Marathon”
Kubernetes
DockerApp
DockerApp
DockerApp
AppDocker
AppApp
Docker
Cluster configuration example #3
Serv Serv Serv Serv
Mesos
Spark, MPI, Hadoop, Storm
ServServServServ
Mesos SDKJava, Python, C++, Go
Services REST API“Marathon” (init)
Batch REST API“Chronos” (cron)
Serv
App Recurring Jobs(ETL, backups)
Hardware
Native Long running Batch
Apps
API
Kernel
AppDocker
App
AWS, DO, GCE
Docker
Cluster configuration example #4
Mesos
Spark, MPI, Hadoop, Storm
ServServ
Mesos SDKJava, Python, C++, Go
Services REST API“Marathon” (init)
Batch REST API“Chronos” (cron)
Serv
App Recurring Jobs(ETL, backups)
Hardware
Native Long running Batch
Apps
API
Kernel
AppDocker
App
Serv
© Reuters
Mesos is not only for “big” players!
Tiny clusters for everybody
● Mesosphere on GCE
● Mesosphere on DigitalOcean
● Mesosphere on AWS
● Portable Mesosphere on a USB stick
google.mesosphere.com
digitalocean.mesosphere.com
elastic.mesosphere.io
Demo: Mesos-on-Mesosphere
TaskTaskTask
Mesos-on-Mesosphere
Zookeeper
MesosSlaveMarathon
MesosMaster
MesosMasterMesos
MasterMesosMaster
MesosSlave
...
Task
Docker
DockerDockerDocker
MesosSlave
Mesos-on-Mesosphere
Zookeeper
MesosSlaveMarathon
MesosMaster
MesosMasterMesos
MasterMesosMaster
MesosSlave
...
MesosMaster
Docker
MesosSlave
MesosSlave
Docker
MesosSlave
MesosMaster
Summary
● Complete stack for large (and small) distributed apps
● Multi-tenancy
● Resource optimizations
● Easy to deploy
● No vendor lock-in
Thank You.
[email protected]#mesos on irc.freenode.net