Upload
datawire
View
306
Download
1
Embed Size (px)
Citation preview
Developer-first workflows (for microservices) on Kubernetes
Richard Li
datawire.io
● Building a cloud application using microservices in 2013
● Distributed systems engineers● Multiple services● Prototyping was really fast
● … then velocity slowed down by a lot when we started getting users and needing to add more features
2
Microservices at Datawire ...
datawire.io
Part 1: How do I gain the benefits of microservices?
(How do I break up my monolith?)(How do I architect my app with microservices?)(What infrastructure do I need in place before I
can benefit from microservices?)
3
datawire.io 4
Stability/Maturity
Velocity
Prototype Production Mission critical
datawire.io 5
Stability/Maturity
Velocity
Prototype Production Mission critical
datawire.io
How do I break up my monolithic architecture?How do I break up my monolithic process into
multiple independent processes?
6
datawire.io 7
Define
Code
Test
Release
Prod
datawire.io 8
Define
Code
Test
Release
Prod
Centralized process
● Specialized teams● Fixed policies (e.g.,
release criteria)
datawire.io 9
Microservices lets you run multiple processes!
datawire.io 10
Industrial Revolution.
datawire.io
Microservices is a distributed development architecture
workflow.
11
datawire.io 12
Stability/Maturity
Velocity
Prototype Production Mission critical
● How do I get to Continuous Deployment incrementally?● How do I limit the scope of PCI (audit process)?● How do I ship feature X as fast as possible?
datawire.io
Microservices is ...
● Multiple workflows○ Including your existing workflow!○ Workflows designed for different
stability/velocity tradeoffs● Simultaneous workflows
13
datawire.io
Doing things this way shifts how people operate!
● Specialists become generalists → design better holistic systems● Dev vs Ops vs QA → roadmap / API conversations● Challenging for engineers → More fun!
14
datawire.io
Part 2: How do I create multiple, simultaneous workflows?
15
datawire.io
Create self-sufficient, autonomous software teams.
16
datawire.io
Why self-sufficiency and autonomy?
● Self-sufficient○ Team does not need to rely on other teams to achieve its goals
● Autonomy○ Team is able to independently make (process) decisions on how to achieve its goals
17
datawire.io
Centralized specialist functions are a common barrier to self-sufficiency and autonomy
18
Centralized architectureCentralized infrastructure / ops*
(You might need a platform team, eventually)
datawire.io 19
datawire.io 20
Monolith
datawire.io 21
Monolith
(soon-to-be) self-sufficient, autonomous team
datawire.io
I’m stranded on API island. Do I have to build my workflow from scratch?
22
datawire.io
No! Kubernetes / Docker / Envoy give you the infra you need
23
datawire.io
Docker
● A consistent build/runtime for your service
● Run locally or in Kubernetes cluster● Massive community & ecosystem
○ 14M Docker hosts○ 3300+ contributors○ Lots of third party tools
24
datawire.io
Kubernetes
● Cloud POSIX● Declarative definition of your cloud
infrastructure
● Supported by all major vendors: Google, Microsoft, Mesos, Docker, Oracle, Red Hat, IBM all support it
● … except for Amazon.○ Yet 62% of K8S workloads on AWS
25
datawire.io
Envoy
● Modern L7 proxy, designed for cloud applications○ Resilience: global rate-limiting, circuit
breakers○ APIs for managing fleets of Envoys○ L7 observability (incl distributed tracing)
● Fast growing ecosystem & community○ Maintained by Lyft○ Google, IBM, Apple, Datawire, Verizon○ CNCF project
26
datawire.io
Cloud Native Compute Foundation
27
● Istio● SPIFFE● App dev
datawire.io
Summary
1. Microservices is a distributed workflow: multiple & simultaneous processes.2. Start building your distributed workflow by creating a self-sufficient, autonomous
team.3. Kubernetes / Docker / Envoy give you the basic operational primitives you need.
28
datawire.io
Part 3: How do I actually use these technologies to build my workflow?
29
datawire.io 30
Monolith
Prototype new feature, FAST
datawire.io 31
Stage 1: Prototyping
Existing workflow: Designed for mature app, not prototyping
Challenge: Prototyping workflow that is fast but not one-off
Strategies:● Dev infra ~= Prod infra● Self-service operational
infrastructure
datawire.io
Strategy: Dev infrastructure ~= Prod infrastructure
32
● End-to-end○ Have developers access the service the same way as end users (e.g., through an
API Gateway, ingress, etc.)
● Environmental consistency○ Insure that your dev runtime is the same as your initial production runtime○ Environmental differences between dev & prod will cause bugs. So eliminate
them upfront.
Your dev infrastructure should resemble your production infrastructure as much as possible.
datawire.io
Strategy: Self-service operational infrastructure
33
Infrastructure needs to be configured by the service team, not by operations (otherwise, there’s a bottleneck.)
Service team needs to be able to publish new services, without operations.
Service team has access to the Kubernetes manifest (maybe they use a standard template).
datawire.io
Implementing Stage 1 strategies
● Eliminate environmental differences○ Containerize your service and use the same container for dev & prod○ Template your Kubernetes manifests so you can use the same manifest for dev & prod
● End-to-end○ Use an API Gateway for dev and prod (you can even use the same one)
● Self-service operational infrastructure○ Expose the right configuration options, in a rational way, for your infrastructure to the service
team■ Kubernetes manifest■ Configuring your API Gateway
34
datawire.io 35
Monolith
Now in production, and need to ITERATE
datawire.io 36
Stage 2: Production users
Existing workflow: App-level integration testing & roll out; dedicated QA team; unit tests
Challenge: Update service quickly, while not letting bugs impact the end user
Strategies:● Software redundancy
datawire.io
Strategy: Multiple versions for software redundancy
37
End user
Primary version
Canary version
Dev version
datawire.io 38
Stage 3 Internal service consumption
Existing workflow: None
Challenge: Communicate to consumers the service contract, and support that contract in the service
Strategy:● Service level objectives● Network-level observability &
resilience
datawire.io
● Define the target level of your service as part of your API● Common SLOs relate to throughput, latency, and availability
○ This API will support up to 5,000 requests per second per consumer○ The expected latency at p50, p90, p99
● Simple implementation of SLOs can be via monitoring
Strategy: Service level objectives
39
datawire.io
Strategy: Network level observability & resilience
● Observability○ Figure out which service is causing the problem○ Need visibility at a network level○ Distributed tracing (e.g., Jaeger, Zipkin)
● Resilience○ Avoid cascade failure○ Network-level rate limiting, circuit breaking, etc.
● Service mesh○ Strategy for implementing network-level observability & resilience that’s transparent to end users○ Istio just proposed to the CNCF
40
datawire.io
Summary
1. Microservices is a distributed workflow.2. Start building your distributed workflow by creating a self-sufficient, autonomous
team.3. Kubernetes / Docker / Envoy give you the basic operational primitives you need. 4. Your workflow requirements depend on your service maturity (prototype,
production, internal dependencies).5. Strategies for building your workflow
a. Prototype: Dev ~= Prod, self-service operational infrastructureb. Production: Software redundancyc. Internal dependencies: Service level objectives, network-level observability/resilience
41
datawire.io
So how does this work in practice?
42
datawire.io
Your traditional monolith workflow ...
● Intended for iterative improvement & prevent downtime○ Code locally in dev environment○ Pull request○ Branch for release○ QA branch & integration test on staging environment○ Release on production environment
● Different environments, specialized teams
43
datawire.io
Versus microservice infrastructure & workflow ...
● Eliminate environmental differences○ Container -- same run time environment○ Kubernetes -- same deployment environment
● Self-service operational infrastructure○ Templated Kubernetes manifest○ Self-service API Gateway
● Multiple versions for software redundancy○ Facilitated by API Gateway / router
44
Will let you prototype / launch / update a microservice with a workflow that’s tailored to feature development.
datawire.io
Thank you!
● [email protected]● Getting ready to launch hands-on tutorials on building your own distributed
workflow. If you’re interested in being notified, sign up here:○ https://d6e.co/montrealk8s
● If you’re interested in any of our open source tools, check them out:○ https://forge.sh for deployment○ https://www.telepresence.io for real-time live coding○ https://www.getambassador.io self-service API Gateway built on Envoy
45
END