Containers, DevOps, Microservices and Kafka: Tools used by our Monolith wrecking crew.

Jonathan Owens, Senior Site Reliability Engineer & Jose Fernandez, Lead Software Engineer

This document and the information herein (including any information that may be incorporated by reference) is provided for informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc. (“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby is proprietary to New Relic and may not be replicated or disclosed without New Relic’s express written permission. Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future events is a forward-looking statement. These forward-looking statements can often be identified as such because the context of the statement will include words such as “believes,” “anticipates,”, “expects” or words of similar import. Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date hereof, and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties transacting business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement or success of the matters covered by such forward-looking statements are based on New Relic’s current assumptions, expectations, and beliefs and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual results, performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further information on factors that could affect such forward-looking statements is included in the filings New Relic makes with the SEC from time to time. Copies of these documents may be obtained by visiting New Relic’s Investor Relations website at ir.newrelic.com or the SEC’s website at www.sec.gov. New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New Relic makes no warranties, expressed or implied, in this document or otherwise, with respect to the information provided.

Confidential ©2008-15 New Relic, Inc. All rights reserved.  

GOOD PROBLEM

Confidential ©2008-15 New Relic, Inc. All rights reserved.  

6

Shopping list for replacing a Monolith

New Relic APM + Insights

Kafka

Microservices

DevOps

Containers

CollectCollectCollectCollectCollect

Collect

Collect

Collect

CollectCollect

Collect

Collect

Collect

Collect

Collect

Great job refactoring services this year!“– No Manager Ever

Updating last-seen timestamps for metrics

Metric ID resolution

Computing cluster agent metrics

Writing metric metadata

Writing summary records

Applying black / whitelist rules to accounts

Processing and storing environment values

Legacy alerting

Agent connect() calls

Agent run state

Orchestrating thread profiles

Applying rename rules to incoming metrics

Aggregation of 1-minute and 1-hour timeslices

from raw harvestsWriting 1-minute and 1-hour

timeslices to MySQL

Updating last-seen timestamps for metrics

Metric ID resolution

Computing cluster agent metrics

Writing metric metadata

Writing summary records

Applying black / whitelist rules to accounts

Processing and storing environment values

Agent connect() calls

Agent run state

Orchestrating thread profiles

Applying rename rules to incoming metrics

Legacy alertingAggregation of 1-minute

and 1-hour timeslices from raw harvests

Writing 1-minute and 1-hour timeslices to MySQL

Shopping list for replacing a Monolith

New Relic APM + Insights

Kafka

Microservices

DevOps

Containers

Collect ConsumeProcess

Process

Tap into the monolith and emit data for your use

collector minuteaggregator

houraggregator

hourwriter

houraggregator

collector minuteaggregator

houraggregator

hourwriter

houraggregator

Validate your data

MySQL

query

QueryService

“In the dark”

Insights

Cassandra

query, result, and performance

QueryValidator

MySQL

query

QueryService

“In the dark”

Insights

Cassandra

query, result, and performance

QueryValidator

Shopping lists for replacing a Monolith

New Relic APM + Insights

Kafka

Microservices

DevOps

Containers

collector minuteaggregator

hourwriter

collector minuteaggregator

houraggregator

hourwriter

houraggregator

collector minuteaggregator

houraggregator

hourwriter

houraggregator

minuteaggregator

summaryrecord

generator

Experimentally proven value

Service boundaries match communication boundaries

Photo credit: theimpulsivebuy

cc by Aranami http://www.flickr.com/photos/z0/

Shopping list for replacing a Monolith

New Relic APM + Insights

Kafka

Microservices

DevOps

Containers

Alice GoldfussSite Reliability Engineer New Relic @alicegoldfuss

DevOps means empathy and communication over the wall until there is no more wall.

JonathanOwens

JoseFernandez

OpsDev

… long wait… frustration due

to lack of visibility…

cyclerepeats

“I need 3 Cassandra servers

ASAP!!”(vague request)

“Here they are. Sorry it took so long.

Our queue was backed up.”

<different goals>

“Doh. I needed the

newer version of Cassandra…”

OpsDev

Team Culture

How did we do it? How can you do it?

* insert picture for Upscale event

Upscale

Product Reliability Engineer

Site Reliability Engineer

Shopping list for replacing a Monolith

New Relic APM + Insights

Kafka

Microservices

DevOps

Containers

Docker is opinionated about software architecture in a way that encourages more robustly crafted applications.

“– Karl Matthias & Sean P. Kane

Docker: Up and Running

PUSH and then

PULL door

Push plate?

Policy statement

Push on a pull handle

Visual instruction to use badge

(with light indicator)

PUSH PUSH PULL

This part fits the problem

This part fits the person

Start Small

Shopping list for replacing a Monolith

New Relic APM + Insights

Kafka

Microservices

DevOps

Containers

Q&A

Thank you.

Jonathan Owens, Senior Site Reliability Engineer & Jose Fernandez, Lead Software Engineer