VoltDB and Erlang

TechPlanet 2012H. Diedrich

http://www.eonblast.com – twitter @hdiedrich

Your Host

Henning Diedrich

• Founder, CEO Eonblast• Maintainer of Emysql, Erlvolt

This Presentation

Review With Leasure• Video & Slides will be online at SK planet• Slides will be online at slideshare.net

Eonblast

VoltDB and Erlang

Open Source Tech for Scale

1. What makes Them Special?2. How do They Look?3. Are They For Me?

Made for a Concrete Need Made for Distribution Made for Multi-Core Truly Different Approaches Based on Parallelism Improving on Previous Solutions Corporate-Created Open Source Professional Support Known by Those in the Know

The free, scaling, SQL DB

Redis + SQL + Scale = VoltDBRedis + SQL + Scale = VoltDB

Why VoltDB?

Business Perspective

• Reduce Costs• Scale Seamlessly• Maintain Strategic Flexibility

Why VoltDB?

Production Perspective

• Keep The Good of SQL• Use the Speed of In-Memory• Take Pain out of Scaling

In-Memory

The Redis of clusters• Today good for 100s of GB of data• Sheds 75% of DBM system activity• Made from scratch for distribution• Full disk persistence

SQL

The MySQL of NewSQLs• SQL Has Flaws• Essentially Math, Not Syntax• You'd be missing Queries• VoltDB is even 'more SQL than SQL'

CAP

• Distributed• Consistent• Highly-Available• Partition-Tolerant

… have it all!Brewer on CAP 2012: http://www.infoq.com/articles/cap-twelve-years-later-how-the-rules-have-changed

ACID

• Atomicity• Consistency• Isolation• Durability

… for granted?

Double Bookkeeping

• Not Every App needs It• Requires ACID Transactions• Neigh Impossible to emulate• Impossible With BASE

(Eventual Consistency)

The Magic

• Compiled-In Queries• Single-Thread Execution• Snowflakes & Clones

Partitions

State#

K Safety 1

State#

Snowflake Structures

PK

FK

FK

FK

FK

FK

FK

PK

Snowflake Structures

FK

FK

FKFK

FKPK

Other Data

PK

PK

PK

Partitions

State#

A B C E F G H I J K L M N O P

Partitions

State#

A B C E F G H I J K L M N O P

A-Z A-Z A-Z A-Z A-Z

Single Thread Execution

• One Thread per Partition• One Thread per Transaction• One Thread can't race itself• Main Design Criterion

Benchmark

• Node.js• Amazon EC2• 64 core node.js clusters + 96 core VoltDB cluster• 695,000 transactions per second (TPS)• 2,780,000 operations per second• 100,000 TPS per 8 core client• 12,500 TPS per node.js core• Stable even under overload• Pretty much linear scale

Details: http://voltdb.com/company/blog/695k-tps-nodejs-and-voltdb

BenchmarkTV Contest

• Millions of callers • Small set of candidates• Massive peak• One transaction is one vote• Callers are identified by their telephone number• Callers must not be allowed to vote more than once

BenchmarkCREATE TABLE contestants ( contestant_number integer NOT NULL , contestant_name varchar(50) NOT NULL , CONSTRAINT PK_contestants PRIMARY KEY ( contestant_number ) );

CREATE TABLE votes ( phone_number bigint NOT NULL , state varchar(2) NOT NULL , contestant_number integer NOT NULL);

CREATE TABLE area_code_state ( area_code smallint NOT NULL , state varchar(2) NOT NULL , CONSTRAINT PK_area_code_state PRIMARY KEY ( area_code ) );

Benchmark

// Check if the vote is for a valid contestant

SELECT contestant_number FROM contestants WHERE contestant_number = ?;

// Check if the voter has exceeded their allowed number of votes

SELECT num_votes FROM v_votes_by_phone_number WHERE phone_number = ?;

// Check an area code to retrieve the corresponding state

SELECT state FROM area_code_state WHERE area_code = ?;

// Record a vote

INSERT INTO votes (phone_number, state, contestant_number) VALUES (?, ?, ?);

The Transaction in 4 Operations

Drivers

• Java• C, C++• node.js• Python• Ruby• Go• Erlang

• VoltDB, Inc. 2009 – commercial developer, support• Open Source – 100% dictatorial by VoltDB, Inc• Made for OLTP – fast cheap writes, high throughput• CA of CAP – 100% consistent & highly available• Simple SQL – real queries• In-memory – 100x faster than MySQL• ACID transactions – double bookkeeping• Distributed – for painless growth• Linear scale – predictable, low cost• Replication, Snapshots – disk persistence, hot backup• More SQL than SQL – clean separation of data

Getting Started

• Download and Install from voltdb.com• Open Getting Started inside the installation• Run Hello World locally• Read M.I.T. Paper A New Architecture• Watch webinars at voltdb.com• Free Help also from VoltDB staff

http://community.voltdb.com/forum• Use EC2 to benchmark your ideas

Upcoming Blog Post

• Looking at 14 databases• Riak, Cassandra, Membase, H-Base, Voldemort, MySQL, MySQL Cluster, Redis, Redis

Cluster, Tokyo Cabinet, Memcached, CouchDB, Couchbase, VoltDB, MongoDB, Postgres

• In the light of what games need• Unbiased comparison• Twitter @hdiedrich

Erlang may be to Javawhat Java was to C++

C++ – pointers = JavaJava – deadlocks = Erlang

Why Erlang?

Business Perspective

• Reduce Costs• Reduce Downtimes• Shorten Time To Market

Why Erlang?

Production Perspective

• High Productivity• Low Hardware Requirements• Fewer Errors

Who Is using It?

“You probably use systemsbased on Erlang/OTP every day

without knowing it.“

Mike Williams

Erlang Game Servers

Zynga: FarmVille via membase, Activision Blizzard: Call of Duty, Bigpoint: Battle Star Galactica, Wooga: Magic Land

Erlang Poster Childs

Klarna AB

• Financial Services for E-Commerce• 30 seconds downtime in 3 years

Distributed Databases• Membase

• Riak

• BigCouch

Sweet Spots• Stateful Servers with High Throughput• Cluster Distribution Layers• Chats

Origins

PLEX• Ericsson makes billions with telecom switches• They used PLEX, an all proprietary software• PLEX delivers, but has bad productivity

Origins• The 80's: Ericsson Computer Science Lab

Joe Armstrong, Robert Virding, Mike Williams

„What aspects of computer languages make it easier to program telecom systems?“

Erlang was Built For

• Reliability• Maintenance• Distribution• Productivity

Concurrency Built In

• In the Language, not OS• Easier Testing• Massively Battle-Tested• Closer To Real Problems

The Magic

• Microprocesses• Pattern Matching*

• Immutable Variables

* Not your familiar Regex string matching

Thinking Erlang

• The Actor Model• Thinking Parallel• Thinking Functional• Thinking Processes• Pattern Matching• Let It Crash!

The Actor ModelCarl Hewitt 1973• Behavior• State• Parallel• Asynchronous Messages• Mailboxes• No Shared State• Self-Contained Machines

Benefits• More true to the real world• Better suited for parallel hardware• Better suited for distributed architectures• Scaling garbage collection (sic!)• Less Magic

Data Code

Object

Data Code

Actor

Process

Asynchronous Messages

Message dispatch is one-way, truly asynchronous.Not function calls but something in their own right.Clean break from the FP paradigm.

Pid ! Msg

Thinking ProcessesWhat should be a Process?

Processes• Don’t share State• Communicate Asynchronously• Are Very Cheap to create And keep• Monitor Each Other• Provide Contention Handling• Constitute the Error Handling Atom

“It's easy!” Joe Armstrong

Objects share Threads

Multiple objects share threads.Objects can be accessed across threads.Threads - and objects - share state.

State, code and process form a unity: the actor.Like processes, actors do not share state.In fact, like humans. Who mostly work quite well.

Actors are Processes

Objects and Threads

C C++ C# Java JavaScript Node Lua Python

Objects and Threads

Idle Threads

Objects and Threads

Unwanted surviving objects

Leeroy Jenkins!!!

Objects and Threads

Prematurely destroyed objects

2*B || !2*B ?

Erlang Actors

Erlang Actors: State + Code + Process

Erlang Actors

One dies.

Erlang Actors

It's restarted.

Errors propagate

Loss of state can affect multiple modules.State errors cross thread boundaries.Defensive code multiplies.

Objects need Locks

● System design is disrupted by explicit locks.● Overly cautious locking slows things down.● Forgotten locks create errors that show under load.

Locked Locked

Waiting

Doing X1

Doing X2

Resource

Worker 2

Worker 1

Crashed Locks Stall

● Locks can need cross-thread error handling.● Stalling and time outs aggravate load.

Locked

Waiting

Doing X1

Resource

Worker 2

Worker 1

Doing X2

Locked

Processes are Transactional

● One actor is one process, obviously cannot “race itself”.● Mandating a job type to an actor creates a transactional funnel.● Only one such job will ever be executing at any one time.

Do X1 for me!

Do X2 for me!

Doing X1 Doing X2Funnel

Thinking Parallel

“It's not easy.” Robert Virding

• The Generals’ Problem• Lamport Clocks• No Guarantees

Generals’ Problem

State

Two generals need to agree when to attack.

Generals’ Problem

State

One sends a messenger.

Generals’ Problem

State

Acknowledge.

Generals’ Problem

State

ACK the ACK. Etc.

Generals’ Problem

State

The messenger may get lost.

Byzantine Generals

State

The generals, actually, too.

Lamport Clocks

Source: Lamport http://research.microsoft.com/users/lamport/pubs/time-clocks.pdf

Order matters more than time.

Thinking Functional

Small Functions

+ Immutable Variables

→ Don’t assign variables: return results!

Complete State in Plain Sight

→ Awful for updates in place.

→ Awsome for debugging & maintenance.

Erlang is not side-effect free at all.

Immutable Variables

Can’t assign a second time:

A = A + 1.

A = 1, A = 2.

Immutable VariablesIt has to be:

B = A + 1.

A = 1, B = 2.

● Prevent Coding Errors● Provide Transactional Semantic● Allow For Pattern Matching Syntax● Can be a Nuisance

Pattern Matching

This can mean two things:

A = func().

The meaning depends on whether

A is already assigned.

Pattern Matching

The common, mixed case:

{ok, A} = func().

ok is an assertion AND

A is being assigned.

Pattern Matching

The common, mixed case:

{ok, A} = func().

“This makes it hard to give Erlang a syntax like e.g. Lua.”

Robert Virding

Let It Crash!

• No Defense Code• On Error, restart Entire Process• Built-In Process Supervision & Restart• Missing Branches, Matches cause Crash

→ Shorter, Cleaner Code

→ Faster Implementation

→ More Robust: handles All Errors

Syntax

● Small● Easy● Stable● Declarative● Inspired by Prolog and ML● Obvious State, Implicit Thread

Fibonacci

Looks like the math description

fib(0) -> 0;fib(1) -> 1;fib(N) when N>1 -> fib(N-1) + fib(N-2).

Hello, World!

-module(hello).

-export([start/0]).

start() ->

io:format("Hello, World!~n").

Hello, World! -module(hello).-export([start/0, loop/0]).

start() -> Pid = spawn(hello, loop, []), Pid ! hello.

loop() -> receive hello -> io:format("Hello, World!~n"), loop() end.

From Edward Garson's Blog at http://egarson.blogspot.de/2008/03/real-erlang-hello-world.html

Start-module(hello).-export([start/0, loop/0]).

start() -> Pid = spawn(hello, loop, []), Pid ! hello.

loop() -> receive hello -> io:format("Hello, World!~n"), loop() end.

From Edward Garson's Blog at http://egarson.blogspot.de/2008/03/real-erlang-hello-world.html

Output-module(hello).-export([start/0, loop/0]).

start() -> Pid = spawn(hello, loop, []), Pid ! hello.

loop() -> receive hello -> io:format("Hello, World!~n"), loop() end.

From Edward Garson's Blog at http://egarson.blogspot.de/2008/03/real-erlang-hello-world.html

Process Spawning-module(hello).-export([start/0, loop/0]).

start() -> Pid = spawn(hello, loop, []), Pid ! hello.

loop() -> receive hello -> io:format("Hello, World!~n"), loop() end.

From Edward Garson's Blog at http://egarson.blogspot.de/2008/03/real-erlang-hello-world.html

New Process

Blocking Receive-module(hello).-export([start/0, loop/0]).

start() -> Pid = spawn(hello, loop, []), Pid ! hello.

loop() -> receive hello -> io:format("Hello, World!~n"), loop() end.

From Edward Garson's Blog at http://egarson.blogspot.de/2008/03/real-erlang-hello-world.html

Message Passing-module(hello).-export([start/0, loop/0]).

start() -> Pid = spawn(hello, loop, []), Pid ! hello.

loop() -> receive hello -> io:format("Hello, World!~n"), loop() end.

From Edward Garson's Blog at http://egarson.blogspot.de/2008/03/real-erlang-hello-world.html

Pattern Matching-module(hello).-export([start/0, loop/0]).

start() -> Pid = spawn(hello, loop, []), Pid ! hello.

loop() -> receive hello -> io:format("Hello, World!~n"), loop() end.

From Edward Garson's Blog at http://egarson.blogspot.de/2008/03/real-erlang-hello-world.html

Tail Recursion-module(hello).-export([start/0, loop/0]).

start() -> Pid = spawn(hello, loop, []), Pid ! hello.

loop() -> receive hello -> io:format("Hello, World!~n"), loop() end.

From Edward Garson's Blog at http://egarson.blogspot.de/2008/03/real-erlang-hello-world.html

Productivity

Motorola Study running 2002 – 2006:

“Erlang shows …• 2x higher throughput• 3x better latency• 3 - 7x shorter code … than the equivalent C++ implementation.”

• Ericsson AB 1983 – commercial developer, support• Open Source – 100% dictatorial by Ericsson• Professional Support – by veterans at Erlang Solutions• Made for telecom – complexity & robustness• Functional – improves maintainability• Actors Model – better than OO for parallel worlds• Distributed – seemless use of multi-cores• Parallel – Processes as basic building block• Simple Language – Easy to learn and maintain• Huge Library – OTP is made for server development• Battle-tested – billion $ projects, 1 million LOC• Great Community – bright and friendly devs

Erlang

Learning Erlang

• Scour Post-Mortems• Download and Install from erlang.org• Fred’s site Learn You Some Erlang! • Joe’s book Programming Erlang• IRC #erlounge• Erlang Mailing List• Local Erlounge Meetings• Erlang Factories & User Conferences

Erlvolt

Erlvolt

• The Erlang VoltDB Driver• Stable Synchronous Version• Upcoming Asynchronous Version

https://github.com/Eonblast/Erlvolt

Erlvolt

Inserterlvolt:callProcedure(Connection, "Insert", ["안녕하세요", "세계", "Korean"]),

SelectResponse = erlvolt:callProcedure(Connection, "Select", ["Korean"]),

Row = erlvolt:fetchRow(Table, 1),

io:format("~n~n~s, ~s!~n",

[ erlvolt:getString(Row, Table, "HELLO"),

erlvolt:getString(Row, Table, "WORLD") ]);

ResourcesErlang

Webhttp://www.erlang.org/ http://learnyousomeerlang.com/http://www.erlang.org/static/getting_started_quickly.html

Listhttp://erlang.org/mailman/listinfo/erlang-questionshttp://groups.google.com/group/erlang-programming

Bookshttp://pragprog.com/book/jaerlang/programming-erlanghttp://shop.oreilly.com/product/9780596518189.do

Referenceshttp://erldocs.com/ http://www.erlang.org/doc/

Post Mortemshttp://www.facebook.com/note.php?note_id=14218138919 http://www.slideshare.net/wooga/erlang-the-big-switch-in-social-games

Erlvolt

https://github.com/Eonblast/Erlvolt

Eonblast

https://www.eonblast.com

VoltDB

Downloadhttp://voltdb.com/products-services/downloads

Webinarshttp://community.voltdb.com/weninars

Forumhttp://community.voltdb.com/forum

Voter Examplehttps://github.com/VoltDB/voltdb/tree/master/examples/voter

Benchmark Blog Posthttp://voltdb.com/company/blog/695k-tps-nodejs-and-voltdb

Volt’s Magic Saucehttp://nms.csail.mit.edu/~stavros/pubs/hstore.pdf

Upcoming

• Erlvolt 2 – asynchronous Erlang VoltDB driver• 14 New Databases for Games – Blog Post

TBA on Twitter @hdiedrich

Questions

• Email: hdiedrich*eonblast.com• Twitter: @hdiedrich• VoltDB: http://community.voltdb.com/forum• Erlang: erlang-questions@erlang.org • Eonblast: http://www.eonblast.com• Game News: @eonblast