How an Enterprise Data Fabric (EDF) can improve resiliency and performance

Copyright © 2009, GemStone Systems Inc. All Rights Reserved.

How an Enterprise Data Fabric (EDF) can improve

resiliency and performance while reducing TCO

Presented by

Mike Stolz, VP Architecture &

Martin Hand, Business Development Director for EMEA

GemStone Systems Inc.

2 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.

Gaming Scalability - Agenda

What is an EDF3 Relevant Use Cases

EDF Resiliency Story Affiliate Banner Management Global Data Distribution (Near Caching)


What is an Enterprise Data Fabric (EDF)

Publisher

Subscriber

Update Data

Subscriber

Subscriber

• Data Management that pools memory, CPU, and network across many servers. • Very high performance – memory speed not disk speed

• Secure, elastic, resilient data storage• Horizontal Partitioning of the Domain Data with Co-location of related data

• Dynamic scalability and elasticity for both data and behavior• Automatic striping and replication• Continuous availability – “RAID for the Enterprise”

• Replication to Disaster Recover Sites• Global Data Distribution – Consistent Global Views


Enterprise Data Fabric Resiliency Story

High Availability, Disaster Recovery and

Business Continuity are a pure distributed caching play


Use-Case: In Play Pricing and Risk Management

Business requirement/drivers Global view, speed, stability, scalability, etc.

• Multiple geographical sites (i.e., Hong Kong, London, India) must see all betting positions

• Sub-second position calc and global distribution• Betting volumes doubling year on year• Number of users doubling year on year• Burst rates of over 1000 bets per second• Consideration for broadcast latency• High availability/fast failover and recovery

– NO TOLERANCE FOR DOWN TIME


Use Case: In Play Pricing and Risk Management

Traditional Solution Product Set: Database Instances In Every Region Local Area Messaging product to move

data around among processes WAN-based Messaging Product for global

distribution N+1 Clustering for High Availability Storage Level Replication for Disaster

Recovery


Classic High Availability

External Data Sources.

Clients

Node Failure:

In the event of failure of a node in the N+1 cluster, the +1 node is pressed into service

Shared Storage

• Detect Failure

• Boot +1 Node

• Mount Shared Storage

• Start up DB

• Start up Applications

• Fetch data from DB

• Re-create Objects

• Reconnect exchanges

• Reconnect clients

• Recovery complete

Typically takes about 15 minutes to

recover from a single node failure


Classic Disaster Recovery

External Data Sources

Clients

Catastrophic Failure:

In the event of failure of the entire cluster or Data Center, DR is invoked.

Replicated StoragePrimary

DR

• Detect Failure

• Swing Storage

• Boot DR Nodes

• Fetch data from DB

• Start up Applications

• Start up DB

• Mount Storage

• Recovery Complete


• Reconnect externals

• Re-create objects

In most installations it takes between

1 and 4 hours to recover from

a data center outage


Node Failure:

In the event of failure of a single node.

Catastrophic Failure: In the event of failure

of the entire cluster or Data Center, DR is invoked.

Enterprise Data Fabric H/A and DR

External Data Sources/

Exchanges, etc.

Clients

Shared Nothing Storage

Hot #1

Hot #2

Shared Nothing Storage

Transactional Replication

Data is striped across the distributed cache

• Detect Failure


• Reconnect Clients

• Detect Failure


• Reconnect Externals


On the order of 1 second for recovery

from a single node failure

1 second plus any external connectivity

to recover from a full data center outage


EDF Simplifies your Architecture

What did we replace? Database Instances In Every Region

• Still need 1 instance for archival purposes

Separate Local Area Messaging Product WAN Distribution Product Traditional N+1 Clustering for H/A Storage Level Replication for DR

This architecture simplification represents a very significant savings in

terms of TCO.


Improve your Affiliate Banner Advertising

How an Enterprise Data Fabric can help you scale


Affiliate Banner System

Bet.com


Affiliate Banner Delivery

Banners are delivered to the affiliate’s web pages each time

they are viewed by a member of the public.

The HTML code on the affiliate’s web site for the banner does

not have a static image address. Instead it calls to your image

server with parameters that indicate which banner should be

delivered.

The parameters identify the Product (i.e. “Sport”), Classification

(i.e. “Cricket”), Language (i.e. “German”) and Dimensions of the

requested Banner.

Once the banner has been displayed, you need to track every

time a user actually clicks on the banner so that you know how

effective your banner advertising actually is in terms of traffic

generation

The issue here is the ability to scale this

to hundreds of thousands of hits.



VB.NetWeb Server

Internet

VB.NetWeb Server

VB.NetWeb Server

Banner DB Clicks DB

Geo DB

The traditional way:

A tangled web of connections from the web tier into the Geo, Banner and Click tracking databases which become the bottleneck.



VB.NetWeb Server

Internet

VB.NetWeb Server

VB.NetWeb Server

Banner DB Clicks DB

Geo DB

EDF

With an EDF:

Geo and Banner data are pre-loaded into cache, Offloading the DB’s and reducing latency, thereby enabling scale to hundreds of thousands of hits.

Clicks are written to the cache, which in turn does a lazy write-behind to the click tracking database.

The net result is that the time spent

generating the appropriate banner is

minimized, thus enhancing the end-user

experience, and the likelihood that he will

click-through.


EDF for Global Data Distribution

Advantages of global data consistency and regional edge caching


Global Data Distribution – Edge Caching

How do you deliver best end-user experience to gamers globally?

Answer: Global Edge Caching.


Gateway Gateway

Gateway

US

AP EU

Processing EnginesCentralization of Odds Management:

Odds can be managed centrally or if you prefer, can be managed at multiple geographical sites (nearest point to the action).

AP Users

US Users

EU Users

RDBMS



Gateway Gateway

Gateway

US

AP EU


Odds data is distributed around the globe and cached locally in the regional edge caches.

AP Users

US Users

EU Users

RDBMS



Gateway Gateway

Gateway

US

AP EU


So that users only need to make a local hop to read the most current data.

AP Users

US Users

EU Users

RDBMS



Gateway Gateway

Gateway

US

AP EU

Processing Engines

AP Users

US Users

EU Users

Centralization of Transaction Management:

Transaction requests are sent to the centralized resource manager.

RDBMS



Gateway Gateway

Gateway

US

AP EU

Processing EnginesCentralization of Transaction Management:

The transaction is processed locally by the resource manager.

AP Users

US Users

EU Users

RDBMS



Gateway Gateway

Gateway

US

AP EU

Processing Engines

AP Users

US Users

EU Users

Centralization of Transaction Management:

Response is sent to the originating user’s node.

RDBMS



Gateway Gateway

Gateway

US

AP EU


And changes to Odds or other data are automatically propagated around the world.

AP Users

US Users

EU Users

RDBMS



Gateway Gateway

Gateway

US

AP EU


And notification is published to all interested users.

AP Users

US Users

EU Users

RDBMS


Global Edge Caching ensures the most

efficient end-user experience without

replicating the whole application

environment.


Summary

What did the EDF achieve? Fast end-user experience Secure, Reliable Persistence Keep up with fast moving environment Updates published from location of action Rapid, Event Based Odds Calculation Distribution of Updates Globally Consistent Global Views High Availability Disaster Recovery Reduced complexity and TCO


Q & A

Technology

How an Enterprise Data Fabric (EDF) can improve resiliency and performance