Upload
vikash-kodati
View
22
Download
0
Embed Size (px)
Citation preview
Event driven Microservices (μs) and CQRSVikash Kodati
8/10/2016
T-Mobile Confidential2
AGENDA
4/6/2016
• Why Event driven Microservice (μs)• Overview of event sourcing• Implementing Queries in an event sourced application
3
• Let’s imagine you are building a large, complex application, e.g., an online store
6/13/2016 T-Mobile Confidential
T-Mobile Confidential4
Monolithic Architecture
6/13/2016
Store front UI Module
WAR
Catalog Module
Reviews Module
Orders Module
Browser
Client App
HTML
REST/JSON
SQL Database
T-Mobile Confidential5
Limitations
6/13/2016
• Monoliths = Trouble
• Very intimidating to change things
• State of art = deploy many times in a day = difficult with Monolith
• The large the app = slower the IDE = slower container
• Obstacle to development and ties to a technology choices at the start of the project.
• RDBMS issues around Scale, Schema updates, OR mismatch • Don’t deal well with unstructured data
T-Mobile Confidential6
Microservice Architecture
6/13/2016
Browser
Mobile Device
Store Front UI
API Gateway
Catalog Service
Review Service
Order Service
….Service
Catalog Database
Review Database
Order Database
….Database
HTML Rest
Rest
Rest
T-Mobile Confidential7
NoSQL db options
6/13/2016
• Avoids the limitations of RDBMS
• Text Search Solr/CloudSearch
• Social (graph) data Neo4J
• Highly distributed and available database Cassandra.
T-Mobile Confidential8
Drawbacks
6/13/2016
• Complexity of developing a distributed system• Implementing inter-process Communication• Handling partial failure
• Complexity of implementing business transactions that span multiple databases (without 2pc)
• Complexity of testing a distributed system• Complexity of deploying and operating a distributed system• Managing the development and deployment of features that span
multiple services
T-Mobile Confidential9
Issues to address
6/13/2016
• How to deploy the services?• How do the services communicate?• How do client of application communicate with the services?• How to partition the system into services?• How to deal with distributed data management problems?
T-Mobile Confidential10
#1: SQL + Text search engine
6/13/2016
T-Mobile Confidential11
#2: Update 2 entities in NoSQL db
6/13/2016
T-Mobile Confidential12
#3: Indexing in Cassandra
6/13/2016
NOSQL Landscape
6/13/2016 T-Mobile Confidential13
T-Mobile Confidential14 6/13/2016
How to deal with distributed data management problems?
Solution: Event-Based Architecture
4/6/2016 T-Mobile Confidential15
• Components (e.g., services) publish events when state changes• Components subscribe to events• Maintain eventual consistency across multiple aggregates• Synchronize replicated data
T-Mobile Confidential16
#1: Shared Databases (today’s world)
6/13/2016
Order Service Customer Service ….Service
Tight Coupling
Customer table
Credit Limit
Order table
Order total…..
The DatabaseSimple and
ACID
T-Mobile Confidential17
#2: Database per services
6/13/2016
Order Service
Order table
Order total
Customer table
Credit limit
Customer service
Order Database Customer Database
18
To maintain consistency a service must atomically publish an event whenever a domain object changes.
2PC (aka. Distributed transactions) is not viable choice for most modern applications. NOSQL DBs do not support it!
6/13/2016 T-Mobile Confidential
T-Mobile Confidential19
How to maintain data consistency without 2PC
6/13/2016
Order service
Place order()
Customer service
updateCreditlimit()
Invariant:Sum(open order.total)<=Customer.creditLimit
Order
Total
CustomerCredit limit
Order management Customer management
Belongs to Has orders
T-Mobile Confidential20
Use event-driven, eventually consistent order processing
6/13/2016
OrderService
Customer Service
Place Order Order Created
Credit Reserved
ORCredit Check Failed
21
BEFORE: Update State + Publish EventsTwo actions must be atomic
NOW: Persist (and publish) EventsSingle action that is atomic
6/13/2016 T-Mobile Confidential
Designing Domain Events
6/13/2016 T-Mobile Confidential22
• For each domain object (i.e.,DDD aggregate)• Identify(State changing) domain event• Define Event Classes• Event Enrichment• ID: TimeUUID
• For example,• Shopping Cart:ItemAddedEvent, ItemRemovedEvent,
OrderPlacedEvent• Order:OrderCreated,OrderCancelled,OrderApproved,OrderReje
cted, OrderShipped
Persists events NOT Current state
6/13/2016 T-Mobile Confidential23
Order
Status
101 Accepted
Order table
Persists events NOT Current state
6/13/2016 T-Mobile Confidential24
Entity Id Entity Type Event ID Entity Type Event data
101 Order 901 Order Created …..
101 Order 901 Order Approved …..
101 Order 901 Order shipped ….
Replay events to recreate state
6/13/2016 T-Mobile Confidential25
Order
State
Events
OrderCreated(..)orderAccepted(…)OrderShipped(….)
Periodically snapshot to avoid loading all events…
Aggregates : Command Events
6/13/2016 T-Mobile Confidential26
Command AggregateEvent
Request handling in an event sourced application
6/13/2016 T-Mobile Confidential27
HTTPHandler Order
Event Store
PastEvents = findEvents(entityID)
New()
applyEvents(PastEvents)
NewEvenets=processCmd(someCmd)
applyEvents(NewEvents)
saveEvents(NewEvents) (optimistic locking)
Order Service
Event store publishes events consumed by other services
6/13/2016 T-Mobile Confidential28
Event Store
EventSubscriber
Customer
Subscribe(EventTypes)
Publish(event)
Publish(event)
Update()
Customer Service
Event store = database + message broker
6/13/2016 T-Mobile Confidential29
Event Store
SaveAggregate Events
GetAggregate Events
Subscribe to Events
• Hybrid database and message broker• Implementations:
• Home Grown/DIY• getEventStore.com by greg Young• http://eventuate.io by Chris Richardson
Benefits of event Sourcing
6/13/2016 T-Mobile Confidential30
• Solves data consistency issues in a microservice/NoSQL based architecture
• Reliable event publishes events needed by predictive analytics etc, user notifications
• Eliminates O/R mapping problem(mostly)• Reifies state changes:
• Built in, reliable audit log• Temporal queries
• Preserved history More easily implement future requirements
Drawbacks of event Sourcing
6/13/2016 T-Mobile Confidential31
• Requires application rewrite• Weird and unfamiliar style of programming• Events = a historical record of your bad design decisions• Must detect and ignore duplicate events:
• Idempotent event handlers• Track most recent event and ignore old ones
• Querying the event store can be challenging• Some queries might be complex/inefficient e.g. accounts with balance> x• Event store might only support lookup of events by entity id• Must use command Query Responsibility Segeration(CQRS) to handle queries application
must handle eventually consistent data
T-Mobile Confidential32
Anatomy of a microservice
6/13/2016
Microservice
Event Store
Messages AdapterHTTP Adapter
HTTP Request Messages Requests
AggregateAggregate
Event AdapterEvents
Events
Cmd
Cmd
Implementing queries in an event sourced application
4/6/2016 T-Mobile Confidential33
Find recent, Valuable Customers
6/13/2016 T-Mobile Confidential34
SELECT * FROM CUSTOMER c, ORDER o WHEREc.id = o.ID AND o.ORDER_TOTAL > 100000AND o.STATE = 'SHIPPED‘ AND c.CREATION_DATE > ?
Customer Service Order Service
What if sourcing is used?
Command query Responsibility Segregation(CQRS)
6/13/2016 T-Mobile Confidential35
Application Logic
Commands Queries
Command query Responsibility Segregation(CQRS)
6/13/2016 T-Mobile Confidential36
Aggregate
Command Side
MaterializedView
Query Side
Event Store
Commands Queries
Events Events
Command query Responsibility Segregation(CQRS)
6/13/2016 T-Mobile Confidential37
Benefits and Drawbacks of CQRS
6/13/2016 T-Mobile Confidential38
Benefits Drawbacks
• Necessary in an event sourced architecture
• Separation of concerns = simpler command and query models
• Support multiple denormalized views
• Improve Scalability and performance
• Complexity• Potential code duplication• Replication lag/eventually
consistent view
Summary
6/13/2016 T-Mobile Confidential39
• Use microservices to accelerate development• Use an event-driven architecture to maintain data consistency• Implement an event-driven architecture using event sourcing• Use CQRS to implement materialized views for queries