Embed Size (px)
DESCRIPTION
A whirlwind tour of a few NoSQL solutions, learning the very different ways they represent data and seeing their unique strengths and weaknesses in various kinds of applications. Along the way, we'll learn why new technologies must be introduced to address today's scaling challenges, and what compromises we'll have to make if we want to abandon the databases of our youth.
Citation preview
BerglundTim
NoSQLSMACKDOWN
1
@tlberglund
#nosql2
Voldemort
3
negativavia
4
SQL5
tuple (n.)
relation (n.)An unordered set of tuples of the same type.
6
tuple (n.)A function that maps attributes to values.
7
tuple (n.)(A bundle of key-value pairs—but don’t tell anyone!)
8
id username pwd_hash born_at monkey
1 mluther d8c82af9 Nov 1483 FALSE
2 aaugustine 329b8dae Nov 354 FALSE
3 gnyssa e50ec9e0 Jun 335 FALSE
4 bonzo 330e01f2 Apr 2007 TRUE
9
Relations
10
Comparing “NoSQL” to “relational” is a bit of a shell game.
—Eben Hewittauthor of Cassandra: The Definitive Guide
11
Transactions
12
C
PA
CAP Theorem
13
Tradeoff Between
ConsistencyAvailability
Partition Tolerance
14
Between what?
15
consistency (n.)All clients always have the same view of the data.
16
availability (n.)All clients can always read or write within some maximum latency.
17
partition tolerance (n.)
No set of failures less than total network failure is allowed to cause the system to respond incorrectly.
18
Cluster Node
Cluster Node
Cluster Node
Cluster Node
19
Cluster Node
Cluster Node
Cluster Node
Cluster Node
Switch
20
Cluster Node
Cluster Node
Cluster Node
Cluster Node
Switch
21
C
PA
CAP Theorem
22
Strongly Consistent
C
PA
MongoDBCassandra
23
Always Available
C
PA
CouchDBRiak
VoldemortCassandra
24
Partition Intolerant
C
PA
MySQLOracle
SQL ServerNeo4JRedis
25
C
PA
26
negativavia
a way forward
27
NoSQL is a set of different approaches to storing and
retrieving data.
28
What’s Different?
Data models
Querying
Approaches to scale
29
Tradeoffs
Complex transactions vs. scalability
Consistency vs. availability (often)
Performance vs. durability
Horizontal vs. vertical scale
Cheap writes vs. cheap reads
30
OriginLicenseImplementation language
Data model
How does it scale?
API/Query language
Deployments
Support and community
31
32
Voldemort
33
34
Origin-Facebook Inbox search
back in 2007
License-Apache Public License 2.0
35
Implementation Language-
Java 6
Data Model-
It’s a Big-Table-based
“column store.”
36
Column
TimestampValueName
37
Row
Column
Key
Column
Column
Column
Column
39
Column Family
ColumnColumnKey Column
ColumnColumnKey
ColumnKey
40
“Contacts” Column Family
emailfull_name050fe74e2 mobile
emailfull_namebbf77f01d
full_name8b20d8f6
41
SuperColumn
Name
Columnkey
Columnkey
Columnkey
42
Contact Info SuperColumn
4145bfaf15f10c2e6033f8b9c3143297a36f5fe3
20101011T120502ZTim Berglundfull_name
19940217T145637Z[redacted]mobile
20101011T120452Z80123postal_code
full_name
mobile
postal_code
43
SuperColumn Family
Key
SuperColumnKey
Key
SuperColumn
SuperColumn
SuperColumn
SuperColumn
44
Keyspace
SuperColumn Family
SuperColumn Family
SuperColumn Family
Column Family
Column Family
45
A what?
46
Nested Hash Table
Cluster.Keyspace.ColumnFamily[key1][key2] = <column>
...SuperColumnFamily[key1][key2] = <row>
Cluster.Keyspace.ColumnFamily[key] = <row>
...SuperColumnFamily[key1][key2][key3] = <column>
...SuperColumnFamily[key] = <map of rows>
47
Scalability-
Rock star!(see Amazon Dynamo)
2000
4000
6000
8000
A000
C000
E000
0000
48
2000
4000
6000
8000
A000
C000
E000
0000
49
Scalability
- Consistent hashing
- No distinguished nodes
- Add and remove nodes
on a live cluster
50
API- Thrift RPC
- Easy to fetch columns
by key
- Hadoop integration- Native clients
51
Deployments-
52
Support/Community-
- Plus, it is an Apache
project...
- www.datastax.com
- Eben Hewitt’s book
53
Voldemort
54
55
Origin-
Founders of DoubleClick were totally going to
take over the Cloud
License-Database: GNU Affero 3.0
Drivers: APL 2
56
Implementation Language-
C++
Data Model-
JSON document database
(this is so simple!)
57
{ "_id" : ObjectId("4cbd00455280f73d395922a4"), "contact" : { "tags" : ["man", "", "", ""] "firstName" : "Myron", "lastName" : "Dalton", "address1" : "4322 Maple Street", "city" : "Santa Ana", "state" : "CA", "postalCode" : "92705", "email" : "[email protected]" }, "occupation" : "Long haul truck driver" }
58
Does it scale?
Well...it shards!
59
API- Native JavaScript
console
- Binary drivers
- Ad-hoc query language
(but it’s NOT SQL, okay?)
60
db.address.find().limit(5)
db.contact.find({ “lastName”: “Berglund” })
db.address.find({ $query: { “stateProvince”: “CO” }, $orderBy: { “city”: 1 } })
db.address.find({ “contact.city”: “Chicago” })
db.address.remove({_id: ObjectId("4cbcfd7df72291161b1d1bf2")})
61
API- Can write MapReduce
jobs in JavaScript
- Morphia for Java
- Mongoose for node.js
62
Deployments-
63
Community/Support
- www.10gen.com
- Probably the NoSQL
mindshare leader
64
Concerns
- Write durability?
- Sharding performance
- But everyone still wants
to date her
Journaling comingin 1.8!
65
Voldemort
66
67
Origin
-Neo Technologies in 2003
-Malmö and San Francisco
68
License
- GPL3, full-featured
- Commercial
$49/mo antiviral
$499/mo advanced
$1,999/mo enterprise
69
Maturity
- Production since 2003
- 1.0 in Feb 2010
- Java 6Implementation Language
- Easily embeddable!
70
Data Metaphor
- Graph
- Nodes, relationships
71
4CG
-;NNB?Q
"LC;H
(IFFSQII>4SJ?M
+HIQM
7LCN?M QCNB7ILEM QCNB
3J?;EM
QCNB%HA;A?M CH
>CMJON;NCIH QCNB
All nodes and relationships have arbitrary properties
72
Query Model
- REST/JSON
- Java traversal API
- JTA/JTS XA
- Bindings in Clojure, Ruby,
Python, PHP, Scala, Grails
73
Scale Idiom- Traditionally focused on
single-node performance
- Recent HA support
- Master/slave
- ZK master election
- Writeable slaves74
Support
- Neo Technologies
Deployments
- Box.net
- Box.net
- ThoughtWorks
75
Voldemort
76
77
Origin-
Internal datastore forBasho’s Salesforce.comapps
(Hey, it seemed like a good idea at the time!)
78
License-
APL 2 for OSS version
Closed-source “Enterprise DS” version
79
Implementation Language-
Erlang, C, SpiderMonkey
JavaScript VM
Data Model-
Key/value store, but
with buckets!
80
ValueKey
That’s it.
81
Bucket A
ValueKey
ValueKey
ValueKey
ValueKey
Bucket B
ValueKey
ValueKey
ValueKey
ValueKey
82
Bucket A
Timname
Developeroccupation
061972birthday
Littletoncity
Bucket B
Aureliusname
Bishopoccupation
110354birthday
Hippocity
83
Does it scale?
- Like a boss!
- No distinguished node
- Tunable consistency, replication
- Add nodes without taking the cluster down
84
API- HTTP interface (slow,
but featureful)
- Protocol Buffers (a
performance beast)
85
API
- Key CRUD
- MapReduce in
JavaScript
- Graph traversals
translate to MapReduce
86
Deployments-
87
Community/Support
- www.basho.com
88
Voldemort
89
90
Origin-
Salvatore Sanfilippo wrote it for his analytics
site, llogg.com
Open Source-Brand open source
License-
91
Implementation Language
- ANSI C, baby
- Wants a POSIX OS
- 340kB download!
92
Data Model
-Key/value store++
-Strings
-Hashes, Sets
-Lists
-Sorted Sets
93
Does it scale?
- Vertically, sure
- Plus it’s really fast
- Master/slave options
- Technically a CA system
94
API- Binary socket interface
- Commands look like assembly language
- Drivers for 22+ languages
95
96
97
98
99
Deployments-
craigslist100
Community/Support
- Officially sponsored by VMware
101
Voldemort
102
Do you need this?
Maybe.103
104
Further ReadingBrewer’s Conjecturehttp://www.podc.org/podc2000/
Proof of Brewer’s Conjecture (the “CAP Theorem”)http://bit.ly/cap-theorem-proof
Amazon Dynamohttp://bit.ly/amazon-dynamohttp://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
Google BigTablehttp://bit.ly/big-table
The CAP Theorem Explainedhttp://www.julianbrowne.com/article/viewer/brewers-cap-theorem
Visualzing NoSQL Databases on the CAP Venn Diagramhttp://blog.nahurst.com/visual-guide-to-nosql-systems
Redishttp://redis.io/
Cassandrahttp://cassandra.apache.org
MongoDBhttp://mongodb.org
106
Further ReadingCouchDBhttp://couchdb.apache.org
Riakhttp://basho.com
Voldemorthttp://project-voldemort.com
Neo4Jhttp://neo4j.org
Pretty Much Everything About NoSQLhttp://nosql.mypopescu.com
107
Photo CreditsWrestlershttp://www.flickr.com/photos/stigster/4573851095
Desert Roadhttp://www.flickr.com/photos/kenlund/2439199670
Kindergarten Graduationhttp://www.flickr.com/photos/moyermk/3102262394
Clipboardhttp://www.flickr.com/photos/wheatfields/264890076
Winning Wrestlerhttp://www.flickr.com/photos/jrandallc/2259174414
108
