19
VoltDB David Rolfe Director of Solution Architecture, EMEA Doug Jauregui Sr. Solutions Engineer

3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

  • Upload
    others

  • View
    8

  • Download
    0

Embed Size (px)

Citation preview

Page 1: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

VoltDB

David RolfeDirector of Solution Architecture, EMEA

Doug JaureguiSr. Solutions Engineer

Page 2: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

Legacy database technology is obsolete• Legacy RDBMS designs date from about

1985.

• Vendors are finding legacy databases increasingly uneconomic.

• Legacy databases struggle to scale beyond 2 nodes.

• But demand for transactions is increasing all the time.

• Meanwhile Moore’s law means hardware and RAM keeps getting cheaper

$1

$10

$100

$1.000

$10.000

$100.000

$1.000.000

$10.000.000

1

10

100

1.000

10.000

100.000

1.000.000

10.000.000

100.000.000

1.000.000.000

10.000.000.000

100.000.000.000

1980 1990 2000 2010

Internet Traffic (GB Month), Transistors per CPU and Cost of RAM over time

Total Internet Bandwidth (GB/Mo)

Transistors per CPU

Price of Ram ($/GB)

Page 3: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

21st Century Requirements for transaction processing

• Virtualization friendly .• ACID transactions.• Millisecond response times.• No ”Long Tail” • Supports complicated logic• Easily scalable beyond 2 nodes.• HA “Just Happens”• Geo replication • “Translytics”/”HTAP”

Page 4: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

RDBMS - How We Thought an RDBMS Worked

SELECT * FROM PRODUCTS WHERE ID = 1 FOR UPDATE OF qty;

ID=1, Qty= 200, LastDate= 23 /March/18

UPDATE users SET BAL = 190 WHERE ID =1;

INSERT INTO sales Iuserid, productId, cost) VALUES (42,1,10);

UPDATE products SET qty = 199 WHERE ID = 1;

COMMIT;

SELECT * FROM PRODUCTS WHERE ID = 1 FOR UPDATE OF qty;

ID=1, Qty= 200

UPDATE users SET BAL = 190 WHERE ID =1;

INSERT INTO sales Iuserid, productId, cost) VALUES (43,1,10);

UPDATE products SET qty = 199 WHERE ID = 1;

COMMIT;

Inflight Transactions

WAΙTΙNG

WAΙTΙNG

RAMDATA

DISKDATA

CPU

Page 5: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

RDBMS - What Actually Happens – Part 1…

SELECT * FROM PRODUCTS WHERE ID = 1 FOR UPDATE OF qty;

ID=1, Qty= 200, LastDate= 23 /March/18

UPDATE users SET BAL = 190 WHERE ID =1;

INSERT INTO sales Iuserid, productId, cost) VALUES (42,1,10);

UPDATE products SET qty = 199 WHERE ID = 1;

COMMIT;

SELECT * FROM PRODUCTS WHERE ID = 1 FOR UPDATE OF qty;

ID=1, Qty= 200

UPDATE users SET BAL = 190 WHERE ID =1;

INSERT INTO sales Iuserid, productId, cost) VALUES (43,1,10);

UPDATE products SET qty = 199 WHERE ID = 1;

COMMIT;

Inflight Transactions

WAΙTΙNG

WAΙTΙNG

RAMDATA

DISKDATA

CPU

CPU

Page 6: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

RDBMS - What Actually Happens – Part 2

Inflight Transactions

WAΙTΙNG

WAΙTΙNG

Inflight Transactions

WAΙTΙNG

WAΙTΙNG

RAMDATA

RAMDATA

CPU

CORE

CPU

CORE

SAN

Page 7: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

If we tried this in a supermarket…

Page 8: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

Dr. Michael Stonebraker found a solution..

Index Management

11%

Logging20%

Locking18%

Latching10%

Buffer Management

29%

Useful Work12%

Page 9: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

How VoӏtDB works

Inflight Transactions

Inflight Transactions

RAMDATA

RAMDATA

WAΙTΙNG

WAΙTΙNG

WAΙTΙNG

WAΙTΙNG

BOOK

BOOK

PAY

PAY

BOOK

PAY

BayItem 1

BayItem 2

BOOK

BOOK

PAY

BayItem 1

BayItem 2

BOOK

PAY

BOOK

BOOK

BOOK

PAY

PAY

BOOK

PAY

BayItem 1

BayItem 2

BOOK

BOOK

PAY

BayItem 1

BayItem 2

BOOK

PAY

BOOK

CORE

CORE

CORE

CORE

Local File System

Local File System

Page 10: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

How a supermarket works…

Page 11: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

VoltDB’s Role

Tranactions

Scale

MillisecondsACIDTransactions

Batch Processing/

HDFSNoSQL

Legacy OLTP

Page 12: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

The only 3 ways to interact with any databaseApproach Examples Strengths Weaknesses

Many SQL Statements + Commit or Rollback

JDBC, ODBC, Liked by developers, initial development is rapid

• Doesn’t handle scaling OLTP loads well – DB spends its time figuring out who can see what instead of working

• Constant locking problems for shared, finite resources• Failure of a client to Commit or Rollback causes a

temporary resource leak

Move all the data to the client and back again

NoSQL, KV Stores

Very developer friendly

• Multiple updated copies of the data can arrive at the same time for scaling OLTP loads

• All of the data gets moved across the network, every time.

Stored Procedures VoltDB, PL/SQL

Predictablespeed and best possible scaling characteristics

• Not in fashion with developers. • PL/SQL created perception of complexity.• Other implementations of Java Stored Procedures really

slow.

Page 13: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

A Proven and Reliable Partner

loT Platforms, Energy, SensorSmart grid/meters, asset tracing & management

TelcoBilling/rights management, subscriber data, etc.

Financial ServicesRisk, market data management, customer mgt.

Personalize, Customize, TargetAd optimization, audience segmenting, customer service

Infrastructure, Dashboards, KPIsData pipeline, system performance, streaming ETL.

Page 14: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

VoltDB & Machine Learning

Page 15: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

Near Real Time Data for Models and Rules

VoltDB

Spark + Hadoop

New Data

Rules

Fraud Prevention

Single Sign-on Manager

Consumer Banking Risk Management

Credit Card & Mobile Pay

Consumer Banking System

Mobile log-in

Message Queue Real-Time Decision Making

VoltDB

Application/Use Case• Fraud Prevention

• Single sign-in of all Huawei phones

• Consumer banking risk management

Why VoltDB?

• > 50% reduction in fraud cases

• > $15M/year saved from fraud loss

• 10k complex Transactions Per Second

• 99.99% transactions finish < 50ms

• 10x better performance than

traditional fraud detection

Page 16: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

VoltDB & Machine Learning• VoltDB has a C++ core with a Java layer on top for running stored

procedures

• VoltDB implements High Availability by running the same code in two places at once.

• Any Java class can be used in a stored procedure call provided:• It’s deterministic (all copies of the code have to act the same way…)• It doesn’t access network resources (which would make it non-deterministic)

• Examples: H20.AI and (J)PMML

Page 17: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

ML Example – User Defined Function in H20

Page 18: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

ML Example – Calling JPMML from a Procedure

Page 19: 3 - David Rolfe VoltDB IMDB Oct 1st 2018D · 2018-11-15 · VoltDB & Machine Learning • VoltDB has a C++ core with a Java layer on top for running stored procedures • VoltDB implements

For more information:[email protected]