155
THIRDCHANNEL Hi.

Richer data-history-event-sourcing

Embed Size (px)

Citation preview

Page 1: Richer data-history-event-sourcing

THIRDCHANNEL

Hi.

Page 2: Richer data-history-event-sourcing

First, A Quick Scenario

Page 3: Richer data-history-event-sourcing
Page 4: Richer data-history-event-sourcing
Page 5: Richer data-history-event-sourcing

THIRDCHANNEL

Balance: $100

Page 6: Richer data-history-event-sourcing

THIRDCHANNEL

Balance: -$100

Page 7: Richer data-history-event-sourcing

THIRDCHANNEL

Page 8: Richer data-history-event-sourcing

THIRDCHANNEL

Page 9: Richer data-history-event-sourcing
Page 10: Richer data-history-event-sourcing

THIRDCHANNEL

Page 11: Richer data-history-event-sourcing

THIRDCHANNEL

Page 12: Richer data-history-event-sourcing

THIRDCHANNEL

Richer Data History With Event

Sourcing

Steve Pember

CTO, ThirdChannel

Page 13: Richer data-history-event-sourcing

THIRDCHANNEL

Agenda• Event Sourcing

• Event Sourcing Challenges

• Don’t Worry, ES Is Worth It

• Implementation (Theory)

• Implementation (Reality)

• Event Storage

• Querying

• Demo

Page 14: Richer data-history-event-sourcing

What Is Event Sourcing?

Page 15: Richer data-history-event-sourcing

THIRDCHANNEL

Event Sourcing• Alternative Storage Pattern

Page 16: Richer data-history-event-sourcing

THIRDCHANNEL

Page 17: Richer data-history-event-sourcing

Instead, Store Facts

Page 18: Richer data-history-event-sourcing
Page 19: Richer data-history-event-sourcing

THIRDCHANNEL

Page 20: Richer data-history-event-sourcing

THIRDCHANNEL

Page 21: Richer data-history-event-sourcing

THIRDCHANNEL

Event Sourcing• Alternative Storage Pattern

• Series of Facts

• Aggregates << Events

Page 22: Richer data-history-event-sourcing

Event: Something Which Has

Occurred In The System

Page 23: Richer data-history-event-sourcing

Aggregate: The Object of Events’

Affections

Page 24: Richer data-history-event-sourcing

THIRDCHANNEL

Event Sourcing• Alternative Storage Pattern

• Series of Facts

• Aggregates << Events

• Purely Additive

Page 25: Richer data-history-event-sourcing

There is No Delete

Page 26: Richer data-history-event-sourcing

THIRDCHANNEL

Page 27: Richer data-history-event-sourcing

Full History of the Aggregates

Page 28: Richer data-history-event-sourcing

THIRDCHANNEL

Page 29: Richer data-history-event-sourcing

THIRDCHANNEL

Event Sourcing• Alternative Storage Pattern

• Series of Facts

• Aggregates << Events

• Purely Additive

• Optimization: Snapshots

Page 30: Richer data-history-event-sourcing

THIRDCHANNEL

• snapshot photo

Page 31: Richer data-history-event-sourcing

THIRDCHANNEL

Page 32: Richer data-history-event-sourcing

THIRDCHANNEL

Event Sourcing• Alternative Storage Pattern

• Series of Facts

• Aggregates << Events

• Purely Additive

• Optimization: Snapshots

• Example: Shopping Cart

Page 33: Richer data-history-event-sourcing

THIRDCHANNEL

Page 34: Richer data-history-event-sourcing

THIRDCHANNEL

Page 35: Richer data-history-event-sourcing

THIRDCHANNEL

Page 36: Richer data-history-event-sourcing

THIRDCHANNEL

Event Sourcing• Alternative Storage Pattern

• Series of Facts

• Aggregates << Events

• Purely Additive

• Optimization: Snapshots

• Example: Shopping Cart

• Transient Objects

Page 37: Richer data-history-event-sourcing
Page 38: Richer data-history-event-sourcing

Objects are Transient Derivatives of Your

Events

Page 39: Richer data-history-event-sourcing

THIRDCHANNEL

Event Sourcing• Alternative Storage Pattern

• Series of Facts

• Aggregates << Events

• Purely Additive

• Optimization: Snapshots

• Example: Shopping Cart

• Transient Objects

• Natural

Page 40: Richer data-history-event-sourcing
Page 41: Richer data-history-event-sourcing

THIRDCHANNEL

Page 42: Richer data-history-event-sourcing

THIRDCHANNEL

Page 43: Richer data-history-event-sourcing

THIRDCHANNEL

Questions So Far?• Events & Aggregates

• Snapshots

• Additive, loss-less model

• Transient Derivatives

Page 44: Richer data-history-event-sourcing

THIRDCHANNEL

Agenda• Event Sourcing

• Event Sourcing Challenges

Page 45: Richer data-history-event-sourcing

“I Think I Get What You’re Laying

Down, But It Still Seems Bananas”

Page 46: Richer data-history-event-sourcing
Page 47: Richer data-history-event-sourcing

So much work!

Page 48: Richer data-history-event-sourcing

“That’s gotta be a ton of CPU overhead”

Page 49: Richer data-history-event-sourcing

“Where’d my Models go? I can’t live without my

MVC!”

Page 50: Richer data-history-event-sourcing

THIRDCHANNEL

Event Sourcing Challenges• Additional Work To Apply

• Additional CPU Processing

• Non-Traditional Concept of Models

Page 51: Richer data-history-event-sourcing

THIRDCHANNEL

Event Sourcing Challenges• Additional Work To Apply

• Additional CPU Processing

• Non-Traditional Concept of Models

• More Storage Required VS non-ES

Page 52: Richer data-history-event-sourcing

THIRDCHANNEL

Page 53: Richer data-history-event-sourcing

THIRDCHANNEL

Event Sourcing Challenges• Additional Work To Apply

• Additional CPU Processing

• Non-Traditional Concept of Models

• More Storage Required VS non-ES

• Reduced Database Level Constraints

Page 54: Richer data-history-event-sourcing

Make Use of Transactional Code

Page 55: Richer data-history-event-sourcing

THIRDCHANNEL

Event Sourcing Challenges• Additional Work To Apply

• Additional CPU Processing

• Non-Traditional Concept of Models

• More Storage Required VS non-ES

• Reduced Transactions / Database Level Constraints

• May Be Confusing For Junior Engineers

Page 56: Richer data-history-event-sourcing

THIRDCHANNEL

Page 57: Richer data-history-event-sourcing
Page 58: Richer data-history-event-sourcing

THIRDCHANNEL

Page 59: Richer data-history-event-sourcing

So Why Use ES?

Page 60: Richer data-history-event-sourcing

THIRDCHANNEL

Agenda• Event Sourcing

• Event Sourcing Challenges

• Don’t Worry, ES Is Worth It

Page 61: Richer data-history-event-sourcing

THIRDCHANNEL

Why Event Sourcing• More Than an Audit Log

Page 62: Richer data-history-event-sourcing

Audit Logs Tell the History

Events Tell the Intent of History

Page 63: Richer data-history-event-sourcing
Page 64: Richer data-history-event-sourcing

ES is also a perfect Audit Log

Page 65: Richer data-history-event-sourcing

THIRDCHANNEL

Why Event Sourcing• More Than an Audit Log

• Data Storage is Inexpensive

Page 66: Richer data-history-event-sourcing

THIRDCHANNEL

Why Event Sourcing• More Than an Audit Log

• Data Storage is Inexpensive

• Used By All Long-Running Businesses

Page 67: Richer data-history-event-sourcing

THIRDCHANNEL

Page 68: Richer data-history-event-sourcing
Page 69: Richer data-history-event-sourcing

All Business Problems Can Be Modeled With

Event Sourcing

Page 70: Richer data-history-event-sourcing

THIRDCHANNEL

Why Event Sourcing• More Than an Audit Log

• Data Storage is Inexpensive

• Used By All Long-Running Businesses

• Only Structural Model That Does Not Lose Information

Page 71: Richer data-history-event-sourcing

Remember: There Is No Delete

Page 72: Richer data-history-event-sourcing

THIRDCHANNEL

Why Event Sourcing• More Than an Audit Log

• Data Storage is Inexpensive

• Used By All Long-Running Businesses

• Only Structural Model That Does Not Lose Information

• Simplified Testing and Debugging

Page 73: Richer data-history-event-sourcing

Unit Test Events, Assert In Integration Tests

Page 74: Richer data-history-event-sourcing

Time Travel Eases Debugging

Page 75: Richer data-history-event-sourcing

THIRDCHANNEL

Page 76: Richer data-history-event-sourcing

THIRDCHANNEL

Page 77: Richer data-history-event-sourcing

THIRDCHANNEL

Page 78: Richer data-history-event-sourcing

THIRDCHANNEL

Page 79: Richer data-history-event-sourcing

THIRDCHANNEL

Page 80: Richer data-history-event-sourcing

THIRDCHANNEL

Page 81: Richer data-history-event-sourcing

THIRDCHANNEL

Why Event Sourcing• More Than an Audit Log

• Data Storage is Inexpensive

• Used By All Long-Running Businesses

• Only Structural Model That Does Not Lose Information

• Simplified Testing and Debugging

• Ideal for Business Analysis

Page 82: Richer data-history-event-sourcing

Future-Proof your Data

Page 83: Richer data-history-event-sourcing

THIRDCHANNEL

Page 84: Richer data-history-event-sourcing
Page 85: Richer data-history-event-sourcing

But With Event Sourcing…

Page 86: Richer data-history-event-sourcing

–Johnny Appleseed

“Type a quote here.”

Page 87: Richer data-history-event-sourcing

THIRDCHANNEL

Page 88: Richer data-history-event-sourcing

Current State is a Projection

Page 89: Richer data-history-event-sourcing

A 3C Example

Page 90: Richer data-history-event-sourcing

THIRDCHANNEL

Page 91: Richer data-history-event-sourcing

Still…

Page 92: Richer data-history-event-sourcing

THIRDCHANNEL

Page 93: Richer data-history-event-sourcing

THIRDCHANNEL

Have you ever built?• Versioning of objects

• An ‘undo’ action

• Audit Log

• Tracking object value changes over time?

• Time series?

Page 94: Richer data-history-event-sourcing

THIRDCHANNEL

Questions so far?

Page 95: Richer data-history-event-sourcing

Next Up: Implementation

Page 96: Richer data-history-event-sourcing

THIRDCHANNEL

Agenda• Event Sourcing

• Event Sourcing Challenges

• Don’t Worry, ES Is Worth It

• Implementation (Theory)

Page 97: Richer data-history-event-sourcing

THIRDCHANNEL

Implementation (Theory)• Simple Base Objects

Page 98: Richer data-history-event-sourcing

Three Base Objects

Page 99: Richer data-history-event-sourcing

THIRDCHANNEL

Page 100: Richer data-history-event-sourcing

THIRDCHANNEL

Page 101: Richer data-history-event-sourcing

Past-Tense Event Names

Page 102: Richer data-history-event-sourcing

THIRDCHANNEL

Page 103: Richer data-history-event-sourcing

THIRDCHANNEL

Implementation (Theory)• Simple Base Objects

• Aggregate and Event SubClasses Have Transient Properties

Page 104: Richer data-history-event-sourcing

THIRDCHANNEL

Page 105: Richer data-history-event-sourcing

THIRDCHANNEL

Page 106: Richer data-history-event-sourcing

THIRDCHANNEL

Implementation (Theory)• Simple Base Objects

• Aggregate and Event SubClasses Have Transient Properties

• Aggregates Receive and Play Events

Page 107: Richer data-history-event-sourcing

Events Modify Aggregates

Page 108: Richer data-history-event-sourcing
Page 109: Richer data-history-event-sourcing

THIRDCHANNEL

Implementation (Theory)• Simple Base Objects

• Aggregate and Event SubClasses Have Transient Properties

• Aggregates Receive and Play Events

• Aggregates Require Distinction Between New and Historical Changes

Page 110: Richer data-history-event-sourcing

THIRDCHANNEL

Page 111: Richer data-history-event-sourcing

THIRDCHANNEL

Implementation (Theory)• Simple Base Objects

• Aggregate and Event SubClasses Have Transient Properties

• Aggregates Receive and Play Events

• Aggregates Require Distinction Between New and Historical Changes

• Event Service Layer Required

Page 112: Richer data-history-event-sourcing
Page 113: Richer data-history-event-sourcing

THIRDCHANNEL

Agenda• Event Sourcing

• Event Sourcing Challenges

• Don’t Worry, ES Is Worth It

• Implementation (Theory)

• Implementation (Reality)

Page 114: Richer data-history-event-sourcing

THIRDCHANNEL

Implementation (Reality)• Freely Add New Transient Properties

Page 115: Richer data-history-event-sourcing

THIRDCHANNEL

Implementation (Reality)• Freely Add New Transient Properties

• Be Wary of Modifying Events

Page 116: Richer data-history-event-sourcing

THIRDCHANNEL

Implementation (Reality)• Freely Add New Transient Properties

• Be Wary of Modifying Events

• Be Wary of Removing Aggregate Transients

Page 117: Richer data-history-event-sourcing

THIRDCHANNEL

Implementation (Reality)• Freely Add New Transient Properties

• Be Wary of Modifying Events

• Be Wary of Removing Aggregate Transients

• Snapshot Rarely

Page 118: Richer data-history-event-sourcing

Plus so much more…

Page 119: Richer data-history-event-sourcing

THIRDCHANNEL

Agenda• Event Sourcing

• Event Sourcing Challenges

• Don’t Worry, ES Is Worth It

• Implementation (Theory)

• Implementation (Reality)

• Persistance

Page 120: Richer data-history-event-sourcing

THIRDCHANNEL

Persistence & Event Storage• No Real Need for an ORM

Page 121: Richer data-history-event-sourcing

THIRDCHANNEL

Page 122: Richer data-history-event-sourcing

THIRDCHANNEL

Persistence & Event Storage• No Real Need for an ORM

• Try a Library

Page 123: Richer data-history-event-sourcing

THIRDCHANNEL

Event Sourcing Libraries• NEventStore (.NET)

• ProophEventStore (PHP)

• Akka Persistence (JVM)

Page 124: Richer data-history-event-sourcing

THIRDCHANNEL

Persistence & Event Storage• No Real Need for an ORM

• Try a Library

• Database Agnostic

Page 125: Richer data-history-event-sourcing

THIRDCHANNEL

Page 126: Richer data-history-event-sourcing

THIRDCHANNEL

Page 127: Richer data-history-event-sourcing

THIRDCHANNEL

Page 128: Richer data-history-event-sourcing

THIRDCHANNEL

Persistence & Event Storage• No Real Need for an ORM

• Database Agnostic

• Consider Using a Relational DB

Page 129: Richer data-history-event-sourcing

THIRDCHANNEL

Questions?

Page 130: Richer data-history-event-sourcing

THIRDCHANNEL

Agenda• Event Sourcing

• Event Sourcing Challenges

• Don’t Worry, ES Is Worth It

• Implementation (Theory)

• Implementation (Reality)

• Persistance

• Querying

Page 131: Richer data-history-event-sourcing
Page 132: Richer data-history-event-sourcing

THIRDCHANNEL

Querying Events• All Queries are Projections, including Current State

Page 133: Richer data-history-event-sourcing

THIRDCHANNEL

Querying Events• All Queries are Projections, including Current State

• Returning Current State is Easy

Page 134: Richer data-history-event-sourcing

THIRDCHANNEL

Page 135: Richer data-history-event-sourcing

THIRDCHANNEL

Querying Events• All Queries are Projections, including Current State

• Returning Current State is Easy

• Other Projections Can Be Tough

Page 136: Richer data-history-event-sourcing

Query across Event Types

Page 137: Richer data-history-event-sourcing

THIRDCHANNEL

Page 138: Richer data-history-event-sourcing

THIRDCHANNEL

Querying Events• All Queries are Projections, including Current State

• Returning Current State is Easy

• Other Projections Can Be Tough

• Try Reactive Streams

Page 139: Richer data-history-event-sourcing

THIRDCHANNEL

Page 140: Richer data-history-event-sourcing

Use Streams to Filter Events in Code

Page 141: Richer data-history-event-sourcing

THIRDCHANNEL

Page 142: Richer data-history-event-sourcing

THIRDCHANNEL

Querying Events• All Queries are Projections, including Current State

• Returning Current State is Easy

• Other Projections Can Be Tough

• Try Reactive Streams

• Initial Projection Work May Require Dev Time

Page 143: Richer data-history-event-sourcing

Find Someone with a Statistics

Background

Page 144: Richer data-history-event-sourcing

THIRDCHANNEL

Querying Events• All Queries are Projections, including Current State

• Returning Current State is Easy

• Other Projections Can Be Tough

• Try Reactive Streams

• Initial Projection Work May Require Dev Time

• Consider Cacheable Query Layers

Page 145: Richer data-history-event-sourcing

Separate Event Storage / Querying

Services

Page 146: Richer data-history-event-sourcing

THIRDCHANNEL

Page 147: Richer data-history-event-sourcing

THIRDCHANNEL

Agenda• Event Sourcing

• Event Sourcing Challenges

• Don’t Worry, ES Is Worth It

• Implementation (Theory)

• Implementation (Reality)

• Persistance

• Querying

• Demo Time

Page 148: Richer data-history-event-sourcing

Pray to the demo gods.

Page 149: Richer data-history-event-sourcing

THIRDCHANNEL

Page 150: Richer data-history-event-sourcing

In Summary

Page 151: Richer data-history-event-sourcing

Event Sourcing -> Additive Only,

Lossless. Can Be Tricky To Work With

Page 152: Richer data-history-event-sourcing

Not for your Mom’s Blog

Page 153: Richer data-history-event-sourcing

Thank You!

Page 154: Richer data-history-event-sourcing

THIRDCHANNEL

Any Questions?

Steve Pember

@svpember

www.thirdchannel.com

Page 155: Richer data-history-event-sourcing

THIRDCHANNEL

Image Credits• BOA: http://marketmadhouse.com/wp-content/uploads/2014/11/BofA.jpg

• Khaaaaan: http://d1oi7t5trwfj5d.cloudfront.net/38/0d9b80ae5311e1bcc4123138165f92/file/5-things-you-might-not-know-about-star-trek-ii-wrath-of-khan-30th-anniversary.jpg

• tribbles: http://de.memory-alpha.org/wiki/Tribble

• bank ledger: http://www.iowall.net/accounting-ledger-book.html

• ORM: https://guides.codepath.com/android/ActiveAndroid-Guide

• hoarders: http://www.aetv.com/hoarders

• snapshot stream: https://geteventstore.com/

• office space bobs: http://galleryhip.com/office-space-bobs.html

• Nye, Obama, Tyson photo: http://en.wikipedia.org/wiki/Selfie

• Printing Press: http://www.directindustry.com/prod/koenig-bauer-ag/offset-printing-presses-high-volume-40462-602574.html