understanding production€¦ · + MARKETING FINANCE CRMs •Flexible data format •Quick to...

@cyen @honeycombio

understanding production through your customers’ eyes

DEV OPS

2012

velocity! reliability!

DEV OPS

2012

DEV OPS

SUPPORT

SALES

PRODUCT+ MARKETING

FINANCE

?DEV OPS

SUPPORT customerSALES

PRODUCT+ MARKETING

FINANCE

?DEV OPS

SUPPORT customer

logs+exception trackers

helpdesks

monitoring

product analytics

business intelligence tools

SALES

PRODUCT+ MARKETING

FINANCE

CRMs

• Flexible data format • Quick to iterate

• High fidelity • Full context from user

• Customer-oriented

• UX-oriented

• Uses business identifiers • Historical analyses • Arbitrarily complex

• Realtime • Graph-based

2012

OPSDEV

living in: development living in: production

!@#$%

OPS

2012

DEV

logs+ exception trackers

dashboards+ metrics

2012

OPSDEV

CPU is up and write throughput is down on half the nodes in the Cassandra cluster! What did you do?

2012

OPSDEV

!@#$%

2012

OPSDEV

The latest build increased API latency for our biggest customer, on our most expensive endpoint!

+ build IDs + customer IDs + endpoints

logs+ exception trackers

dashboards+ metrics

DEV OPS

2012living in: development

living in: production

2019

OPSDEV

THE FIRST WAVE: getting ops folks to code

OPSDEV2019

APP

API GATEWA

YUSER

MGMT BILLING

WEB UI PARTNERMGMT

PAYMENTS

INTERNAL WEB

UITXN

MGMTNOTIFICATION

SYSTEM

THE FIRST WAVE:

THE SECOND WAVE: teaching devs to own code in production

getting ops folks to code

OPSDEV2019

it’s all about sharing SOFTWARE OWNERSHIP

observability

OPSDEV2019

observabilitya.k.a. understanding the behavior of a system based on knowledge of its

external outputs.a.k.a. "what is my software doing, and why

is it behaving that way?"

2019

observability(being able to ask new questions of my

systems in production)

2019

tools... + people + processes

OPSDEVlogs+exception trackers

monitoring

• Flexible data format • Quick to iterate

• Realtime • Graph-based

OPSDEVData: •Structured •High Cardinality •High Dimensionality

= Context

logs+exception trackers

monitoring

• Flexible data format • Quick to iterate

• Realtime • Graph-based

observability

"NOTHING IS NEW"

observability

?DEV OPS

SUPPORT customer

logs+exception trackers

helpdesks

monitoring

CRMs

product analytics

business intelligence tools

SALES

PRODUCT+ MARKETING

FINANCE

• Flexible data format • Quick to iterate

• High fidelity • Full context from user

• Customer-oriented

• UX-oriented

• Uses business identifiers • Historical analyses • Arbitrarily complex

• Realtime • Graph-based

?DEV OPS

customer

logs+exception trackers

helpdesks

monitoring• Flexible data format • Quick to iterate

• High fidelity • Full context from user

• Customer-oriented

• UX-oriented

• Uses business identifiers • Historical analyses • Arbitrarily complex

• Realtime • Graph-based

CRMs

product analytics

business intelligence tools

SALES

PRODUCT+ MARKETING

FINANCE

SUPPORT

• Realtime

?DEV OPS

SUPPORT customer

logs+exception trackers

helpdesks

monitoring• Flexible data format • Quick to iterate

• High fidelity • Full context from user

• Customer-oriented

• UX-oriented

• Uses business identifiers • Historical analyses • Arbitrarily complex

• Realtime • Graph-based

CRMs

product analytics

business intelligence tools

SALES

PRODUCT+ MARKETING

FINANCE

• Realtime• Flexible data format • Quick to iterate

?DEV OPS

SUPPORT customer

logs+exception trackers

helpdesks

monitoring• Flexible data format • Quick to iterate

• High fidelity • Full context from user

• Customer-oriented

• UX-oriented

• Uses business identifiers • Historical analyses • Arbitrarily complex

• Realtime • Graph-based

CRMs

product analytics

business intelligence tools

SALES

FINANCE

PRODUCT+ MARKETING

• Flexible data format • Quick to iterate

?DEV OPS

SUPPORT

logs+exception trackers

helpdesks

monitoring• Flexible data format • Quick to iterate

• High fidelity • Full context from user

• Customer-oriented

• UX-oriented

• Uses business identifiers • Historical analyses • Arbitrarily complex

• Realtime • Graph-based

CRMs

product analytics

business intelligence tools

SALES

PRODUCT+ MARKETING

FINANCE

• Realtime• Flexible data format • Quick to iterate

customer

?DEV OPS

SUPPORT customerSALES

PRODUCT+ MARKETING

FINANCE

and by the way…

?DEV OPS

SUPPORT customerSALES

PRODUCT+ MARKETING

FINANCE

thanks!@cyen@honeycombio

ASK NEW QUESTIONS

SHIP BETTER SOFTWARE