27
Programming for Hostile Environments Our adversary: bare metal infrastructure June 2018

Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

  • Upload
    others

  • View
    5

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

Programming for Hostile EnvironmentsOur adversary: bare metal infrastructure

June 2018

Page 2: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

About Me Nathan Goulding, SVP Engineering

~15 years frontline engineer forinfrastructure/cloud and media companies

Currently lead engineering team at Packet

me = n+3

@NathanGoulding / [email protected]

Page 3: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

APP

What Packet Does“serverless”

(still runs on a server)

CONTAINER

HYPERVISOR

SERVERS

We automate bare metal, physical infrastructure

Founded in 2014 by infrastructure geeks

Over 15,000 users

x86 and ARM CPU architectures

16 locations around the world

20 supported operating systems

50,000 installs per month

@NathanGoulding / [email protected]

Page 4: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

“DRIVERLESS”

@NathanGoulding / [email protected]

Page 5: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

Programming for Hostile Environments

@NathanGoulding / [email protected]

Topics we’ll cover:

Transitioning from monolith (ruby) to microservices (golang)

Turning antipatterns into patterns

Applied best practices

Goals we set for ourselves

Ephemeral nanoservices

Page 6: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

@NathanGoulding / [email protected]

Hostility of the Environment

APP

CONTAINER

HYPERVISOR

SERVER

Page 7: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

packet.net / @packethost

The Problem, Abstract

Page 8: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

REST API

PORTAL

Datacenter #2Out-of-band

DHCP

Power Control

VPN

Metadata

OS Images

Bare metal racks

IPAM

Datacenter #1

DNS

Out-of-band

DHCP

Power Control

VPN

Metadata

OS Images

Bare metal racks

@NathanGoulding / [email protected]

Page 9: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

From monolith to microservices

API Internal and

External Services

Client PortalDevice, Project, Billing,

Token Management

SOREN

Sflow Agg & Analysis

NARWHAL

Physical SwitchAutomation

S.O.S

Serial ConsoleOut of Bound

Access

DOORMAN

Customer Backend

VPN

MAGNUM IP

Multi-TenantIPAM

PB&J

Power and Boot Control

TINKERBELL

DHCP & iPXE Server

KANT

EC2 StyleMetadata

OSIE

In MemoryInstallation

Environment

PENSIEVE

Forward and rDNS

@NathanGoulding / [email protected]

Page 10: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

Moving to golang

@NathanGoulding / [email protected]

Compiled

Static typing

Very little “magic”

The best of prior programming languages minus the cruft

Page 11: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

An emerging pattern

Page 12: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

@NathanGoulding / [email protected]

An emerging pattern

Page 13: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

Best Practices, in Practice

#1 - gRPC for communication / rpc

#2 - Get your data as close to where you need it as quickly as possible

#3 - Don’t hide code you don’t like

@NathanGoulding / [email protected]

Page 14: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

#1 gRPC for communication / rpc

packet.net / @packethost

Handles backoff / retry

Straightforward service definition for request / response

Streaming data and authentication via SSL

Paradigm for dealing with message format changes

Page 15: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

#2 Get data close to where it needs to be, quickly

packet.net / @packethost

The network is unreliable, the network is unreliable, the network is unreliable

Speed up access times + experience for everyone

Be careful of “I’ll just request it (remotely) whenever I need it”

Page 16: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

#3 Don’t hide code you don’t like

packet.net / @packethost

Don’t use interfaces / providers to hide code you wish didn’t exist

Use drivers / implementations where it counts

Page 17: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

Why Does it Matter?

@NathanGoulding / [email protected]

Page 18: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

Goal #1: Can we provision in under 60 seconds?

@NathanGoulding / [email protected]

Page 19: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

Provisioning Timing Distribution

@NathanGoulding / [email protected]

Page 20: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

Provisioning Timeline

@NathanGoulding / [email protected]

Page 21: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

@NathanGoulding / [email protected]

Page 22: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

Ephemeral Nanoservices

@NathanGoulding / [email protected]

Function Job Nanoservice Microservice Monolith

Ephemeral ✓ ✓ ✓ ✕ ✕

Encapsulated ✓ ✓ ✓ ✓ ✕

Logging ? ✓ ✓ ✓ ✓

Complex tasks ✕ ✓ ✓ ✓ ✓

Monitored ✕ ✕ ✓ ✓ ✓

Page 23: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

Nanoservice Use Cases

@NathanGoulding / [email protected]

Services that have complex tasks or functionality to perform, and...

Need to communicate with other services, and...

Need to be kept up and running, but...

Will never be used past their “life”

Analogy: an ephemeral nanoservice is an “instantiation” of a microservice

Page 24: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

Goal #2: Can we go a full day without a single provisioning failure?

@NathanGoulding / [email protected]

Page 25: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

@NathanGoulding / [email protected]

Page 26: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

What’s next?

#1 - Flexible workflows via directed graphs

#2 - Distributed tracing for service logs

@NathanGoulding / [email protected]

Page 27: Programming for Hostile Environments · Transitioning from monolith (ruby) to microservices (golang) Turning antipatterns into patterns Applied best practices Goals we set for ourselves

Q&A

(we're hiring)

@NathanGoulding / [email protected]