To SQL or No(t)SQL - PFCongres 2012

PFCongres 2012 Jeroen van Dijk

To SQL

OR

No T SQL?( )

JEROEN VAN DIJK

∂Nerd chief @ ENRISE

∂PHPBenelux board member∂Zend Certified Engineer∂Web technology freak∂Open source addict

JEROEN@ENRISE.COM @NEOREY

∂ You engineers are our top chefs!

THE ENRISE RESTAURANT

∂ We want to prepare the best dishes∂ With the best ingredients∂ To create a magical client experience!

WHAT IS

ACID?

EVER HEARD OF

THEOREM?CAP

KNOW

ABC?∂ Courtesy of Tim Anglade

Always

∂ Courtesy of Tim Anglade

AlwaysBe

∂ Courtesy of Tim Anglade

AlwaysBeCaching

∂ Courtesy of Tim Anglade

AlwaysBeCaching

∂ Courtesy of Tim Anglade

JOIN THE HYPE?

WANT!==HAVE

13

HAVE TO USE NOSQL?

SCALABILITY&

PERFORMANCE

14

HAVE TO USE NOSQL?

SCALABILITY&

PERFORMANCE

15

RECAP RDBMS GREATNESS

∂ Standard Query Language∂ ACID: Atomicity, Consistency, Isolation, Durability∂ Supported by everyone and everything∂ Tuning options∂ Battle tested!∂ Open source?!

16

NOT ENOUGH?

MAKING LOADS OF MONEY?

17

BUY A BIGGER BOX

18

∂ Vertical scalability

19

NOT SO GREAT RDBMS FEATURES

NOT SO GREAT RDBMS FEATURES

∂ Vertical scalability

∂ Horizontal scalability

20

∂ Vertical scalability

∂ Horizontal scalability

∂ Schema changes!

21

NOT SO GREAT RDBMS FEATURES

SINCE 2004

DATA++++++

22

WHO NEEDS

ACID!?

CAP THEOREM

PC

A

CONSISTENCY PARTITION TOLERANCE

AVAILABILITY

CAP THEOREM

P

CA AP

CP

C

AAVAILABILITY

PICKTWO

CONSISTENCY PARTITION TOLERANCE

CAP THEOREM

P

CA AP

CP

C

A

PARTITION TOLERANCE

PICKTWO

CONSISTENCY

MySQL (InnoDB, not MyISAM)

PostgreSQL SQL Server

Dynamo Voldemort

Cassandra

SimpleDB

CouchDB

Riak

BigTable

Hypertable Hbase

MongoDB Terrastore

Couchbase MemcacheDB Redis

AVAILABILITY

Oracle RAC Neo4J

NOSQL TYPES4

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

4 NOSQL TYPES

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

4 NOSQL TYPES

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

4 NOSQL TYPES

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

4 NOSQL TYPES

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

∂ 122 known NoSQL databases

4 NOSQL TYPES

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

∂ Focus from Redis, Riak, Neo4J, MongoDB

4 NOSQL TYPES

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

KEY - VALUE

∂ Schema-less design∂ Just strings of data∂ Hard to query∂ Mostly in memory

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

KEY - VALUE

[“key1” => “value1”,“key2” => “value2”,“key3” => “value3”,

]

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

REDIS

∂ Blazing fast key-value implementation∂ Master - slave replication∂ Lots of methods to query data∂ Notable options∂ Data types : Strings, hashes, lists, sets∂ Data expiration∂ Pub/Sub for messaging

∂ Reconsider when using Memcached

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

COLUMN

∂ BigTable or Dynamo style∂ Consistent hashing∂ Vector clocks∂ Hinted hand off

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

COLUMN

∂ Data stored in a ring

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

COLUMN

A

B

C

D

∂ Consistent hashing with 4 nodes

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

COLUMN

∂ Partitioning as done by Riak

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

COLUMN

∂ Read / Write....

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

COLUMN

∂ Anywhere in the ring

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

COLUMN

∂ First node joins the cluster, claims all partitions

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

COLUMN

A

B

C

∂ Reading / writing is done to 3 nodes

N = 3

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

COLUMN

A

B

C

N = 3W = 2

R = 2

∂ Reading / writing succeeds with 2 valid responses

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

COLUMN

A

B

C

D

N = 3W = 2

R = 2[ Ov1 ]

[ Ov1,v2 ]

∂ Node C down, while new write action

[ Ov1,v2 ]

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

COLUMN

A

B

C

D

[ Ov1,v2 ]

N = 3

∂ Node D hands the new version off

N = 3W = 2

R = 2

[ Ov1,v2 ]

[ Ov1,v2 ]

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

RIAK

∂ Dynamo implementation∂ MapReduce query style∂ Multiple storage backends∂ Notable options

§ Link walking (like Graph solutions)§ Solr-like search interface§ Secondary indexes

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

∂ Relations more important then entities∂ From RDBMS perspective : SELF JOINS

GRAPH

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

∂ Facebook style

GRAPH

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

GRAPH

∂ Betweenness centrality

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

GRAPH

∂ Degree centrality

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

GRAPH

∂ Closeness centrality

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

GRAPH

∂ Twitter style

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

GRAPH

∂ TomTom style

2

1

1

33

92

31

2

2

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

GRAPH

2

1

1

33

92

31

2

2

∂ TomTom style

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

GRAPH

2

1

1

33

92

31

2

2

∂ TomTom style

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

NEO4J

∂ ACID compliant∂ Enterprise product for HA ($$$)∂ Custom query language∂ Notable options

§ Self contained web admin

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

DOCUMENT

∂ Largest resemblance with RDBMS∂ MapReduce∂ Software architect more important

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

MONGODB

∂ MySQL of it’s generation?!∂ Master - slave structure∂ MapReduce∂ Notable options

§ Geo indexes

SMALLEST LEARNING CURVE!

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

∂ Rapid changing data which fits in memory∂ Analytics, logging, real-time data collection

USE CASES

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

∂ Rapid changing data which fits in memory∂ Analytics, logging, real-time data collection

∂ Very good availability & fault tolerance ∂ Applications where seconds of downtime hurt

USE CASES

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

∂ Rapid changing data which fits in memory∂ Analytics, logging, real-time data collection

∂ Very good availability & fault tolerance ∂ Applications where seconds of downtime hurt

∂ For rich interconnected data∂ Social relational data, geo & maps data

USE CASES

KEY-VALUE

COLUMN

GRAPH

DOCUMENT

∂ Rapid changing data which fits in memory∂ Analytics, logging, real-time data collection

∂ Very good availability & fault tolerance ∂ Applications where seconds of downtime hurt

∂ For rich interconnected data∂ Social relational data, geo & maps data

∂ MySQL like usage with indexes∂ Any type of data you’d fit in MySQL

USE CASES

65

ONE USEFUL INGREDIENT

66

MORE GREAT TASTES

∂ KLIK VOOR FOOTER

67

∂ KLIK VOOR FOOTER

68

Polyglot persistence?

∂ THANK YOU! FEEDBACK? JOIND.IN/7084

∂ MORE DETAILS? SCAN THIS CODE.