Upload
london-hashicorp-user-group
View
494
Download
1
Embed Size (px)
Citation preview
NomadHASHICORP
HASHICORP
Mitchell Hashimoto @mitchellh
HASHICORP
HashiConf 2016June 13-15 Amsterdam
One day of training followed by two days of talks in Amsterdam on all things HashiCorp
HASHICONF.EU >
by HashiCorp
NomadHASHICORP
Distributed
Op5mis5cally Concurrent
Scheduler
NomadHASHICORP
Distributed
Op5mis5cally Concurrent
Scheduler
HASHICORP
Schedulers map a set of work to a set of resources
HASHICORP
CPU Scheduler
Web Server -Thread 1
CPU - Core 1
CPU - Core 2
Web Server -Thread 2
Redis -Thread 1
Kernel -Thread 1
Work (Input) Resources
CPU Scheduler
HASHICORP
CPU Scheduler
Web Server -Thread 1
CPU - Core 1
CPU - Core 2
Web Server -Thread 2
Redis -Thread 1
Kernel -Thread 1
Work (Input) Resources
CPU Scheduler
HASHICORP
Schedulers In the Wild
Type Work Resources
CPU Scheduler Threads Physical Cores
AWS EC2 / OpenStack Nova Virtual Machines Hypervisors
Hadoop YARN MapReduce Jobs Client Nodes
Cluster Scheduler Applica5ons Servers
Human Scheduler Mee5ngs Hours in a Day
HASHICORP
Advantages
Higher Resource U5liza5on
Decouple Work from Resources
BeMer Quality of Service
HASHICORP
Advantages
Bin Packing
Over-Subscrip5on
Job Queueing
Higher Resource U5liza5on
Decouple Work from Resources
BeMer Quality of Service
HASHICORP
Advantages
Abstrac5on
API Contracts
Standardiza5on
Higher Resource U5liza5on
Decouple Work from Resources
BeMer Quality of Service
HASHICORP
Advantages
Priori5es
Resource Isola5on
Pre-emp5on
Higher Resource U5liza5on
Decouple Work from Resources
BeMer Quality of Service
HASHICORP
NomadHASHICORP
NomadHASHICORP
Cluster Scheduler
Easily Deploy Applica5ons
Job Specifica5on
HASHICORP
example.nomad
# Define our simple redis jobjob "redis" { # Run only in us-east-1 datacenters = ["us-east-1"]
# Define the single redis task using Docker task "redis" { driver = "docker"
config { image = "redis:latest" }
resources { cpu = 500 # Mhz memory = 256 # MB network { mbits = 10 port “redis” {} } } }}
HASHICORP
Job Specification
Declares what to run
HASHICORP
Job Specification
Nomad determines where and manages how to run
HASHICORP
Job Specification
Powerful yet simple
HASHICORP
# Define our simple redis jobjob "redis" { # Run only in us-east-1 datacenters = ["us-east-1"]
# Define the single redis task using Docker task "redis" { driver = "docker"
config { image = "redis:latest" }
resources { cpu = 500 # Mhz memory = 256 # MB network { mbits = 10
port “redis” {} } } }}
HASHICORP
Containerized
Virtualized
Standalone
Docker
Qemu / KVM
Java Jar
Static Binaries
HASHICORP
Containerized
Virtualized
Standalone
Docker
Jetpack
Windows Server Containers
Qemu / KVM
Hyper-V
Xen
Java Jar
Static Binaries
C#
NomadHASHICORP
Application Deployment
Docker
Multi-Datacenter and Multi-Region
Flexible Workloads
Bin Packing
HCL Job Specifications
NomadHASHICORP
Easy for developers
Operationally simple
Built for scale
HASHICORP
Easy for Developers
HASHICORP
Nomad for Developers
Simple Data Model
Declarative Job Specification
Sane Defaults
HASHICORP
job “foobar” {
# Restrict the parallelism in updates update { stagger = “60s” max_parallel = 3 }… }
HASHICORP
job “foobar” {
group “api” { # Scale our service up count = 5 … }}
HASHICORP
job “foobar” {
group “api” { # Scale our service down count = 3 … }}
HASHICORP
job “foobar” {
group “hdfs-data-node” { # Ensure the scheduler does not put # multiple instances on one host constraint { distinct_hosts = true } … }}
HASHICORP
job “foobar” {
group “hdfs-data-node” { # Attempt restart of tasks if they # fail unexpectedly restart { attempts = 5 interval = “10m” delay = “30s” } … }}
HASHICORP
job “foobar” {
task “my-app” { # Ensure modern kernel available constraint { attribute = “kernel.version” version = “>= 3.14” } … }}
HASHICORP
job “foobar” {
task “my-app” { # Inject environment variables env { MY_FEATURE_FLAG = “ON” } … }}
HASHICORP
job “foobar” {
task “my-app” { # Register with Consul for service # discovery and health checking service { port = “http” check { type = “tcp” interval = “10s” } } … }}
HASHICORP
job “foobar” { # Make sure this task runs everywhere type = “system”
# Nothing should evict our collector priority = 100
task “stats-collector” { … }}
Terminal
HASHICORP
$ nomad agent -dev==> Starting Nomad agent...==> Nomad agent configuration:
Atlas: <disabled> Client: true Log Level: DEBUG Region: global (DC: dc1) Server: true
==> Nomad agent started! Log data will stream in below:
[INFO] serf: EventMemberJoin: nomad.global 127.0.0.1 [INFO] nomad: starting 4 scheduling worker(s) for [service batch _core] [INFO] raft: Node at 127.0.0.1:4647 [Follower] entering Follower state [INFO] nomad: adding server nomad.global (Addr: 127.0.0.1:4647) (DC: dc1) [DEBUG] client: applied fingerprints [storage arch cpu host memory] [DEBUG] client: available drivers [docker exec]
NomadHASHICORP
Infrastructure As Code
Declarative Jobs
Desired State
Emergent State
HASHICORP
Operationally Simple
HASHICORP
Client Server
HASHICORP
Built for Scale
HASHICORP
Built on Experience
gossip consensus
HASHICORP
Built on Research
gossip consensus
HASHICORP
Single Region Architecture
SERVER SERVER SERVER
CLIENT CLIENT CLIENTDC1 DC2 DC3
FOLLOWER LEADER FOLLOWER
REPLICATIONFORWARDING
REPLICATIONFORWARDING
RPC RPC RPC
HASHICORP
Multi Region Architecture
SERVER SERVER SERVERFOLLOWER LEADER FOLLOWER
REPLICATIONFORWARDING
REPLICATION
REGION B GOSSIP
REPLICATION REPLICATIONFORWARDING
REGION FORWARDING
REGION A
SERVERFOLLOWER
SERVER SERVERLEADER FOLLOWER
NomadHASHICORP
Region is Isolation Domain
1-N Datacenters Per Region
Flexibility to do 1:1 (Consul)
Scheduling Boundary
HASHICORP
Thousands of regions
Tens of thousands of clients per region
Thousands of jobs per region
HASHICORP
Optimistically Concurrent
HASHICORP
Data Model
HASHICORP
Evaluations ~= State Change Event
HASHICORP
Create / Update / Delete Job
Node Up / Node Down
Allocation Failed
HASHICORP
“Scheduler” =
func(Eval) => []AllocUpdates
HASHICORP
Scheduler func’s can specialize
(Service, Batch, System, etc)
HASHICORP
Evaluation Enqueue
HASHICORP
Evaluation Dequeue
HASHICORP
Plan Generation
HASHICORP
Plan Execution
HASHICORP
External Event
Evalua?on Crea?on
Evalua?on Queuing
Evalua?on Processing
Op?mis?c Coordina?on
State Updates
HASHICORP
Server Architecture
Omega Class Scheduler
Pluggable Logic
Internal Coordination and State
Multi-Region / Multi-Datacenter
HASHICORP
Client Architecture
Broad OS Support
Host Fingerprinting
Pluggable Drivers
HASHICORP
Fingerprinting
Operating System
Hardware
Applications
Environment
Type Examples
Kernel, OS, Versions
CPU, Memory, Disk
Java, Docker, Consul
AWS, GCE
HASHICORP
Fingerprinting
Constrain Placement and Bin Pack
HASHICORP
Drivers
Execute Tasks
Provide Resource Isolation
NomadHASHICORP
Workload Flexibility:
Schedulers
Fingerprints
Drivers
Job Specification
NomadHASHICORP
Operational Simplicity:
Single Binary
No Dependencies
Highly Available
HASHICORP
HASHICORP
Nomad 0.3
Docker, Qemu, Exec, Java, Raw Drivers
Service, Batch, System Schedulers
Periodic Jobs
Service Discovery
Rolling Updates
Job Queueing
Filesystem API (Logging, stdout/stderr, etc.)
Flexible Constraints
HASHICORP
Nomad Near Future
Log Shipping
Plugin System
Volumes
NomadHASHICORP
Cluster Scheduler
Easily Deploy Applica5ons
Job Specifica5on
NomadHASHICORP
Higher Resource U5liza5on
Decouple Work from Resources
BeMer Quality of Service
HASHICORP
Thanks! Q/A