Hopelessness and Confidence in Distributed Systems Design

Hopelessness and Confidence in Distributed Systems Design Camille Fournier, Senior Thinker and

Raconteur @skamille

The easiest way to scale!

Hassle-free scaling!

Easiest. Database. Ever.

Sync state in a cinch at a distributed system scale.

Real-time data

Real-time Analytics

Real-time Decisions

@aphyr  

Distributed Systems: Ugly, Hard, and Here to Stay

TRADE/OFF�

TRADE/OFF�

Embrace Hopelessness Gain Confidence

A brief interlude: The goals of Distributed Systems

SCALE �

FAILURE TOLERANCE �

A DISTRIBUTED CACHE FOR RISK ANALYSIS�

Scaling �

/�

"DSM  FIGURE"  by  Mehrnazzhian  -­‐  Own  work.  Licensed  under  CC  BY-­‐SA  4.0  via  Commons  

Azul

SYNCHRONIZATION � PERFORMANCE�/�

REWRITING� MODIFICATION �/�

BEST practice: TESTING

GLOBAL SERVICE DISCOVERY USING ZOOKEEPER�

GLOBAL service discovery system

Failure Tolerance�Scaling �

/�

Naïve approach: Central Coordinator!

•  Should all data be in one system? That would be easy!

•  But then…

DISTANCE� PERFOMANCE �/�

•  Speed of light •  Limitations of bandwidth and latency

DISTANCE� PERFOMANCE �/�

Fallacies of Distributed Computing Deutsch et al, Sun Microsystems

1.  The network is reliable. 2.  Latency is zero. 3.  Bandwidth is infinite. 4.  The network is secure. 5.  Topology doesn't change. 6.  There is one administrator. 7.  Transport cost is zero. 8.  The network is homogeneous.

Wide Area Networks Add Complexity

FEATURES � CONTROL�/�

BEST PRACTICE: Remember that clients are part of your system

A second brief interlude…

MICROSERVICES AT RENT THE RUNWAY�

Evolution off of a monolith

Scaling (systems and humans) and failure tolerance (??) �

/�

COMMON UNDERSTANDING � THE PERFECT

TOOL FOR THE JOB�

/�

@coda  

RATE OF CHANGE � THOROUGHNESS�/�

@adrianco  

BEST PRACTICE: DATA DESIGN (AGAIN!) �

BEST PRACTICE: Data Design (again!)

•  Silence comes via good design! •  You can always design poorly and have to

overcommunicate/coordinate

Distributed Systems: Ugly, Hard and Here to Stay

COMBAT HOPE AND FEAR BY UNDERSTANDING TRADE/OFFS�

Hopeless and Confident

Thank you! Special Thanks: @randommood

@SeanTAllen @chrisk5000

@HenryR @aphyr @caitie

@pbailis (aka Littlefinger)

@giltene @MikeM6174

@tsantero

?s: @skamille