View
408
Download
0
Category
Tags:
Preview:
DESCRIPTION
Understand how you can get the benefits you're looking for from NoSQL data stores without sacrificing the power and flexibility of the world's most popular open source database - MySQL.
Citation preview
NoSQL & MySQL: The Best of Both Worlds
Andrew Morgan (@andrewmorgan)www.clusterdb.com
Principal MySQL Product Manager
11th June 2014
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.2
Safe Harbour Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.3
MySQL Customers
World’s Most Popular Open Source Database
WebSaaS, Hosting
Enterprise OEM / ISV’s
Telecom
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.4
SQL NoSQL
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.5
Session Agenda
NoSQL – What are people looking for? RDBMS – What advantages do they still have? How MySQL Delivers the Best of Both Worlds
– MySQL Cluster NoSQL attributes: Scale-out, performance, ease-of-use, schema
flexibility, on-line operations NoSQL APIs
– Key-Value store access to InnoDB (Memcached)
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.6
Types of NoSQL stores
Key-Value
• Cassandra• Memcached• BigTable• Hadoop• Voldermort
Document
• MongoDB• CouchDB
Graph
• Neo4J• FlockDB
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.7
Key-Value Store
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.8
Document Store
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.9
Graph Database
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.10
What NoSQL must deliver
Massive scalability– No application-level sharding
Performance High Availability/Fault Tolerance Ease of use
– Simple operations/administration– Simple APIs– Quickly evolve application &
schema
ScalabilityPerformance
HAEase of use
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.11
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.12
Still a role for the SQL (RDBMS)?
NoSQL
Simple access patterns
Compromise on consistency for performance
Ad-hoc data format
Simple operation
SQL
Complex queries with joins
ACID transactions
Well defined schemas
Rich set of tools
No best single solution fits all
Mix and matchScalability
PerformanceHA
Ease of useSQL/Joins
ACID Transactions
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.13
The Temptation to Jettison Relational Model Relational Model:
– Data stored in multiple tables
– Many lookups to retrieve a user’s timeline
– Is it worth the effort in setting up this complex data model?
Simpler just to store as one document?
The allure of document stores
Examples borrowed from @sarahmei https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy
user
friend
post
comment
like
liker
commenter
many
many
many
one
one
many
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.14
The Temptation to Jettison Relational Model
Document Model:– Entire timeline in a single
document (row)– Single lookup to retrieve the
user’s timeline– Brilliantly efficient model when
the document truly contains self-contained information
Like a real-world document!
The allure of document stores
Examples borrowed from @sarahmei https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy
{name: ‘Joe’, url: ‘…’ stream:[ {friend:{ name: ‘Jane’, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: ‘Joe’}, {…} ], likes: [‘Joe’, ‘Fred’, ‘Billy’] }, {…},{…},… ] }, {…},{…},…}
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.15
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.16
The Temptation to Jettison Relational Model
These are all people who have their own data that users will want to view or click through:
– Name– url– Timeline
Easy to represent with FKs in a relational model
But when the data isn’t self contained…
Examples borrowed from @sarahmei https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy
user
friend
post
comment
like
liker
commenter
many
many
many
one
one
many
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.17
The Temptation to Jettison Relational Model
Do I store all data for all friends and likers again at every point they appear in the document?
– Massive amount of repeated data
Wasted space Needs to be kept in sync (and
you don’t have transactions to provide consistency
The allure of document stores
Examples borrowed from @sarahmei https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy
{name: ‘Joe’, url: ‘…’ stream:[ {friend:{ name: ‘Jane’, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: ‘Joe’}, {…} ], likes: [‘Joe’, ‘Fred’, ‘Billy’] }, {…},{…},… ] }, {…},{…},…}
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.18
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.19
The Temptation to Jettison Relational ModelThe allure of document stores
Examples borrowed from @sarahmei https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy
{name: 83746251, url: ‘…’ stream:[ {friend:{ name: 9384726153, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: 83746251}, {…} ], likes: [83746251, 750730283, 2938493820] }, {…},{…},… ] }, {…},{…},…}
The reality is that the developer will store the user-ids instead
Developer is responsible for implementing ‘joins’ in the application
– e.g. scanning all timelines for where a user made a comment or liked a post
No transactions to ensure consistency between documents
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.20
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.21
MySQL Cluster: Overview• Auto-Sharding, Multi-Master• ACID Compliant, OLTP + Real-Time Analytics
HIGH SCALE, READS + WRITES
• Shared nothing, no Single Point of Failure• Self Healing + On-Line Operations
99.999% AVAILABILITY
• Open Source + Commercial Editions• Commodity hardware + Management, Monitoring ToolsLOW TCO
• Key/Value + Complex, Relational Queries• SQL + Memcached + JavaScript + Java + JPA + HTTP/REST & C++SQL + NoSQL
• In-Memory Optimization + Disk-Data• Predictable Low-Latency, Bounded Access TimeREAL-TIME
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.22
Who’s Using MySQL Cluster?
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.23
MySQL Cluster Architecture
MySQL Cluster Data Nodes
Data Layer
Clients
Application Layer
Management
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.24
MySQL Cluster Architecture
MySQL Cluster Data Nodes
Data Layer
Clients
Application Layer
ManagementManagement
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.25
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.26
MySQL Cluster Architecture
MySQL Cluster Data Nodes
Data Layer
Application Layer
ManagementManagement
Clients
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.27
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.28
On-line Operations
Scale the cluster (add & remove nodes on-line) Repartition tables Upgrade / patch servers & OS Upgrade / patch MySQL Cluster Back-Up Evolve the schema on-line, in real-time
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.29
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.30
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.31
Scale-Out Reads & Writes on Commodity Hardware
8 x Commodity Intel Servers– 2 x 6-core processors 2.93GHz – x5670 processors (24 threads)– 48GB RAM
Infiniband networking flexAsynch benchmark (NDB API)
2 4 80
200
400
600
800
1,000
1,200SELECT Queries per Minute
Number of Data Nodes
Mill
ions
4 80
204060
80100
120UPDATE Queries per Minute
Number of Data Nodes
Mill
ions
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.32
JOIN Performance
33K rows over 11 tables
Must Analyze tables for best results
mysql> ANALYZE TABLE <tab-name>;
MySQL Cluster 7.1 MySQL Cluster 7.20
102030405060708090
100
Query Execution Time Seconds
Scalability aPerformance a
HA aEase of useSQL/Joins a
ACID Transactions
a
70xMore
performance
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.33
MySQL Cluster 7.4
Performance gain over 7.3– 47% (Read-Only)– 38% (Read-Write)
Better performance and operational simplicitylabs.mysql.com
Faster node restarts– Recovering nodes rejoin the cluster
faster
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.34
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.35
Creating & running your first ClusterThe traditional way (pre-MCM & Auto-Installer)
Download & Extract•edelivery.oracle.com
•www.mysql.com•dev.mysql.com
Configure•Cluster-wide “config.ini”
•Per-mysqld “my.cnf”
Start processes•Management Nodes•Data Nodes•MySQL Servers
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.36
Scalability aPerformance a
HA aEase of use aSQL/Joins a
ACID Transactions
a
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.37
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.38
MySQL Cluster 7.3: Auto-Installer
Fast configuration Auto-discovery Workload optimized Repeatable best practices
Specify Workload
Auto-Discover
Define TopologyDeploy
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.39
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.40
MySQL Cluster Manager
1. Download MCM/Cluster package from edelivery.oracle.com:2. Unzip3. Run agent, define, create & start Cluster!
$> bin\mcmd --bootstrapMySQL Cluster Manager 1.1.2 started
Connect to MySQL Cluster Manager by running "D:\Andrew\Documents\MySQL\mcm\bin\mcm" -a NOVA:1862
Configuring default cluster 'mycluster'...
Starting default cluster 'mycluster'...
Cluster 'mycluster' started successfully
ndb_mgmd NOVA:1186
ndbd NOVA
ndbd NOVA
mysqld NOVA:3306
mysqld NOVA:3307
ndbapi *
Connect to the database by running "D:\Andrew\Documents\MySQL\mcm\cluster\bin\mysql" -h NOVA -P 3306 -u root
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.41
MCM: Upgrade Cluster
mysql> upgrade cluster--package=7.3 mycluster;
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.42
Scalability aPerformance a
HA aEase of use aSQL/Joins a
ACID Transactions
a
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.43
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.44
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.45
NoSQL Access to MySQL Cluster data
ClusterJMySQL
JDBC
Apps
JPA
JNIPython Ruby
ClusterJPA
Apps Apps Apps Apps Apps
Node.js
JS
Apps
mod-ndb
Apache
Apps
ndb-eng
Memcached
Apps Apps
NDB API (C++)
MySQL Cluster Data Nodes
Apps
PHP PERL
Apps
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.46
Schema-Free apps Rapid application evolution
– New types of data constantly added
– No time to get schema extended
– Missing skills to extend schema
– Initially roll out to just a few users
– Constantly adding to live system
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.47
Cluster & Memcached – Schema-Free
<town:maidenhead,SL6>
key value
<town:maidenhead,SL6>key value
Key Valuetown:maidenhead SL6
generic table
Application view
SQL view
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.48
Cluster & Memcached - Configured Schema
<town:maidenhead,SL6>
prefix key value
<town:maidenhead,SL6>key value
Prefix Table Key-col Val-col policytown: map.zip town code cluste
rConfig tables
town ... code ...
maidenhead ... SL6 ...
map.zip
Application view
SQL view
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.49
MySQL 5.6 Memcached with InnoDB
8 32 128 5120
10000
20000
30000
40000
50000
60000
70000
80000
MySQL 5.6: NoSQL Benchmarking
Mem-cached APISQL
Client Connections
TPS
Clients and Applications
MySQL ServerMemcached Plug-in
innodb_memcached
local cache(optional)
Handler API InnoDB API
InnoDB Storage Engine
mysqld process
SQL Memcached Protocol
Up to 9x Higher “SET / INSERT” Throughput
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.50
MySQL Cluster 7.3: Node.js NoSQL API Native JavaScript access to MySQL Cluster
– End-to-End JavaScript: browser to the app & DB
– Storing and retrieving JavaScript objects directly in MySQL Cluster
– Eliminate SQL transformation Implemented as a module for node.js
– Integrates Cluster API library within the web app Couple high performance, distributed apps,
with high performance distributed database Optionally routes through MySQL Server
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.51
MySQL Cluster NoSQL API for Node.js & FKs FKs enforced on all APIs:{ message: 'Error', sqlstate: '23000', ndb_error: null, cause: { message: 'Foreign key constraint violated: No parent row found [255]',
sqlstate: '23000', ndb_error: { message: 'Foreign key constraint violated: No parent row found', code: 255, classification: 'ConstraintViolation', handler_error_code: 151, status: 'PermanentError' }, cause: null } }
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.52
Next StepsLearn More• www.mysql.com/cluster• Authentic MySQL Curriculum:
http://oracle.com/education/mysql
Try it Out• dev.mysql.com/downloads/cluster/
Let us know what you think• clusterdb.com• @clusterdb• forums.mysql.com/list.php?25
Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.53
Recommended