Embed Size (px)
Citation preview
Killing containers to make weather
beautifulJacob Tomlinson
Met Office Informatics Lab @_jacobtomlinson
Who am I?
Visualising 4D data in a
web browser
Confidence in
integration
Visualising 4D data in a web browser
Ingestion
Supercomputer
Binary files
PNG images
Videos
The browser
Service types
Long running daemons
Short running workers
Kubernetes
Mixing job types
Docker Host
nginx
Syst
em C
apac
ity
Docker Host
nginxSy
stem
Cap
acity
Docker Host
nginx
Syst
em C
apac
ity
Docker Host
Syst
em C
apac
ity
Docker Host
Syst
em C
apac
ity
Docker Host
Syst
em C
apac
ity
nginx nginx nginx
Docker Host
nginx
Syst
em C
apac
ity
Docker Host
nginxSy
stem
Cap
acity
Docker Host
nginx
Syst
em C
apac
ity
Docker Host
nginx
Syst
em C
apac
ity
Docker Host
nginxSy
stem
Cap
acity
Docker Host
nginx
Syst
em C
apac
ity
worker
worker
worker
worker
worker
worker
worker
worker
worker
worker
worker
worker
Docker Host
nginx
Syst
em C
apac
ity
Docker Host
Syst
em C
apac
ity
Docker Host
Syst
em C
apac
ity
worker
nginx nginx
workerworker
Docker
nginxSyst
em
worker
Docker
Syst
em
worker
nginx
Docker
Syst
em
worker
nginx
Docker
Syst
em
worker
worker
worker
worker
worker
Docker
Syst
em
worker
Docker
Syst
em
worker
Docker
Syst
em
worker
Docker
Syst
em
worker
nginx
worker
nginxnginxnginx
workerworkerworker
Persistence
EBS backed persistent volumes
Docker Host
mongo
EBS Volume
Docker Host Docker Host
Docker Host
EBS Volume
Docker Host Docker Host
mongo mongo
Docker Host
EBS Volume
Docker Host Docker Host
mongo
Scaling by queue
AWS cost vs scale
Confidence in integration
Short containers
Long containersDaemon processes which listen to requests
Asynchronous processes doing repetitive workloads
Short containers
∞ jobs 1 job
n jobs
Pros Conceptual simplicity
Confidence
Cons Overhead
Long containers
Run forever
Finite runtime
Scaling down
Being confident
Conclusion
Questions
Jacob Tomlinson @_jacobtomlinson
