©Continuent 2013

Marketing Automation at Scale: How

Marketo Solved Key Data

Management Challenges with

Continuent Tungsten

Nick Bonfiglio – VP Operations, Marketo

Robert Hodges – CEO, Continuent

About Us

• Nick Bonfiglio - Marketo VP of Operations / Responsible for Cloud Services and Security at

Marketo

/ Extensive background in scaling SaaS products and services

• Robert Hodges - Continuent CEO / Tungsten Replicator designer

/ DBMS internals and applications for three decades

/ Author of Scale-Out Blog (http://scale-out-blog.blogspot.com)

Marketo is the global leader in Revenue Performance

Management.

Marketo’s powerful, yet easy-to-use marketing automation

and sales effectiveness solutions, transform how

marketing and sales teams of all sizes work — and work

together — to drive dramatically increased revenue

performance and fuel business growth.

The company’s proven technology, comprehensive

services, and expert guidance are helping corporations

around the world to turn marketing from a cost center to a

business-building revenue driver.

About Marketo

/ The leading provider of data replication and

clustering for open source databases

/ Our Product: Continuent Tungsten

• Clustering – Commercial-grade HA, performance

scaling, and data management for MySQL

• Replication – Flexible, high-performance data

movement

About Continuent

4

Customers from hot startups to Fortune 500 companies

©Continuent 2013

Marketo’s Data

Management Challenge

Nick Bonfiglio

Marketo Architecture

• Marketo uses a mixed stack architecture to support its customers’ subscriptions, leveraging the best technologies for the solution.

• Marketo is a platform consisting of a collection of systems that provide user interface, landing pages/forms, fast secondary search (solr), big data (hadoop) and archiving (cassandra).

• Our systems employ several custom queue-based background processing, which are key to a successful marketing automation platform. And, custom email assembly, scripting and tracking facilities.

Systems Architecture

Marketo SaaS Platform

File Services Database Services

Batch

Processes

User Interface

Munchkin (tracking)

API Landing Pages/Forms

Email Services

Infrastructure Architecture

Marketo DB Design Goals

• Deliver increased availability by providing redundancy and real-time resiliency at all layers.

• Deliver an architecture that allows us to upgrade software and routine database schema changes without incurring downtime to customers.

• Improve overall performance, scalability and management, including backups and disaster recovery of large 7TB+ DB servers.

Marketo DB Architecture

DB on

FlexVol DB on

FlexVol

DB on

FlexVol

Replicator Replicator Replicator

Master Slave Slave

Marketo Web Cluster Marketo Queue Cluster

MySQL Client Library MySQL Client Library

Manager Manager Manager

Pod Data

DBMS Vendor Client Library DBMS Vendor Client Library Tungsten Connector Tungsten Connector

Mo

nito

ring an

d co

ntro

l

Mo

nito

ring an

d co

ntro

l

Marketo DB Hardware Architecture

• Leveraging NetApp FlexVols for snapshot, near-line and DR replication

• Symmetric DB servers so failover yields full performance

• Aggregates are 300GB drives for increased spindles

• Each DB Server ~6-8T

• Leverage flexcache for certain volumes

Performance Metrics

• Over 600 Million MySQL Transactions per day

• Over 600TB of data

• Processing 25 Million Campaigns per day

• Sending over 500M Emails per month

• Serving 15M Landing Page Visits per day

• A few proof points: – Several failovers which would have caused an

outage

– Parallel replication allows is to keep lag to <300sec

– Several roll rotations for maintenance (schema changes, dropping large dbs, db pruning, etc.)

©Continuent 2013

So… How Does Tungsten

Do That?

Robert Hodges

Challenges to Hosting Critical Data on MySQL

/ Get back online quickly after DBMS failures

/ Perform DBMS maintenance and SQL upgrades

without stopping applications

/ Load balance SQL across replicas to use

hardware efficiently

/ Ensure slaves keep up with high master

transaction loads

/ Address all of this without migrating or rewriting

applications

Tungsten Database-as-a-Service for MySQL

Replicator Replicator

• Cluster constructed from

off-the-shelf MySQL

servers

• Automatic failover

• Online maintenance

• Query Load balancing for

speed and h/w utilization

• No migration of data or

application changes

Manager

db2

Manager

Replicator

db1

Manager

db3

Slave Slave Master

Application Stack

Tungsten Connector

Application Stack

Tungsten Connector

Replicator Replicator

Transparent connectivity enables illusion of single

DBMS to applications

• Locate appropriate

server for transactions

• Select most up-to-date

server for reads

• Shift connections from

offline nodes

• Hide offline databases

• Supports SSL

Send writes to master Load balance

reads over slaves

Application Stack

Application Stack Application

Stack

Tungsten Connector

db2 db1 db3

©Continuent 2013

Demo:

Introducing a Tungsten

Data Service

What Causes Slave Lag?

Application Application

Application

Master

InnoDB Buffer Pool

NetApp Storage

Slave

InnoDB Buffer Pool

NetApp Storage

Typical SaaS Application: 1.) Data size >> buffer pool 2.) Persistent stored on disk

SaaS Applications Can Replicate Tenants in

Parallel!

• Sharded replication divides updates into

independent streams (“shards”)

tenant_1

tenant_2

tenant_3

tenant_4

Tenants independent from each other

Tenant database updates applied in parallel

tenant_1

tenant_2

tenant_3

tenant_4

High-Speed Parallel Replication Lets Slaves Keep

Pace with Busy Masters

• Shard transactions by

schema on master

• Transfer log serially

• Apply shard updates

over “channels”

• 5x or more speed boost

in best cases

• Works on all MySQL

versions from 5.0 to 5.6

Replicator

db1

Master

db2

Slave

Replicator

Application Stack

Application Stack Application

Stack

Problem: Uneven SaaS Tenant Size

Problem: “Hot” Tables within Single

Schemas that Block Other Tenants

Tungsten’s Innovative Caching Minimizes

Effects of Lumpy SaaS Workload

Slave

InnoDB Buffer Pool

NetApp Storage

Tungsten Replication Service

NetApp Storage

Fetch from Master

Write to Log

Apply to Slave

Linux Page Cache

Read log from page cache

(No reads from storage)

Write log to page cache

Fsync to storage

Shards apply Independently!

Innovative Features of Tungsten Parallel

Replication Support

• Tungsten leans on OS page cache to keep

long updates from blocking progress

• Use any number of apply channels

• Support for clean failover and backup

• Completely crash-safe for InnoDB

• Configuration checks to prevent accidental

slave corruption

• Sharding algorithm is configurable!

©Continuent 2013

Demo:

Maintenance on MySQL

master under load

What Else Can Tungsten Do?

• Cross-site data services

– Primary/disaster recovery sites

– Multi-master replication between sites

• Fast data warehouse/NoSQL loading from

MySQL/Oracle

• Oracle-to-Oracle replication

What’s Next? Push-Button Cloud Operation

Configure DBMS type and

resource levels for servers

Define backup and

recovery policies

Confirm configuration

and launch

Choose Service Type Manage Deployment

1

2 3 4

Making it Better – Visual Management

Making it Better – Visual Management

Failure detected

and failover

completed within 15

seconds

Making it Better – Visual Management

Failed db instance

is isolated from the

service

Making it Better – Visual Management

©Continuent 2013

Conclusion and Q&A

In Summary…

33

• Marketo performs 600M transactions daily

on MySQL running over NetApp

• Tungsten delivers fast failover, rolling

maintenance, and parallel replication to

make architecture work

• Marketo and Continuent together have

enabled an architecture that supports

economical scaling to 1000s of customers

Contact Us!

Continuent, Inc.

560 S. Winchester Blvd

Suite 500

San Jose, CA 95128

Tel +1 (866) 998-3642

Fax +1 (408) 668-1009

e-mail: sales@continuent.com

www.continuent.com

Marketo, Inc.

901 Mariners Island Blvd

Suite 200

San Mateo, CA 94404

Tel +1 (877) 260-MKTO

(6586)

Fax +1 (650) 376-2331

e-mail: sales@marketo.com

www.marketo.com