Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
1Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
MySQL High-Availability andScale-Out architectures
Oli SennhauserSenior Consultant
2Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Introduction
• Who we are?• What we want?
3Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Table of Contents
• Scale-Up vs. Scale-Out• MySQL Replication• MySQL HA solution• MySQL Cluster• Application architectural solutions• Other architectural stuff• It's your turn...• Let us build a replication
4Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
When do we think about architecture?
• Performance problems• HA requirements
5Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Scale-up vs. Scale-Out
• Scale-up
• Scale-out
t
Relaxation of constraints
6Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
MySQL Replication
• The MySQL answer: Master-Slave-Replication:
Even Oracle uses the expression “scale-out” now!
Master
Slave1
Slave2
Slave3
Application
LB
...
ro
rtw
2 types ofsessions!!!
7Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
MySQL Replication in detail
M S
Appl
bin-logbin-log... relay-log
IO thread
SQL thread
asynchronous!
relay-log
rtw
ro
• wide distance possible!• thin line possible!
8Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
MySQL Replication varieties
M
SI
SI
SI
• Cascaded replication:
for example Yahoo!
S1
S2
S3
S4
S6
S5
S7
S8
...
...
black hole SE!
9Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
MySQL Replication varieties
M
S2
S3
• HA with “hot standby” Slave:
• active – active fail over!
S1
Sbackup
Sstandby
Appl
VIP HA solution
10Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
MySQL HA solution
M
S2
S3
• HA with MySQL:
• 99.99% HA (four nine)• active – passive fail over!
S1
M'
Appl
VIP
DRBD
heartbeat
11Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
MySQL HA solution
M
S2
S3
• HA with MySQL and SAN:
S1
M'
Appl
VIP
SAN
heartbeat
12Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Multi-Master Replication
M1
S12
S13
• actually Master-Master Replication
• This architecture does NOT solve your write problems!!!
S11
Appl
M2
Appl
VIP/LB
S22
S23
S21
13Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Multi-Master Replication
M1
S12
S13
• HA Multi-Master Replication
S11
M2
S22
S23
S21
M2'
Appl
VIP
DRBD
M1'
Appl
VIP
DRBD
VIP/LB
14Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Shared disk cluster
DB1
• This is in 99.9% of the cases NOT your solution!!!• It looks like Oracle RAC but MySQL does NOT
(yet) have the instruments needed!• Can be useful in some special kinds of reporting.
DB2
shareddisk
DB2
ro!
15Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Where are we now?
• The architectures above solved:– your read problems.– some HA requirements.
• But they did not solve:– your write problems!
• And what when we need:– higher HA?– synchronous replication?– more write speed?
16Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
MySQL Cluster
• Shared-nothing architecture• Synchronous replication (2-Phase commit)• Fast automatic fail over• High performance (also writing)• High transactional throughput• No special components required• In-Memory database (in 5.1 also disk support)• Scalable, 1000's of transactions per second• 99.999% HA (five nine)• On-line upgrade path (at least on GA within same
version)
17Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
MySQL Cluster architecture
Application Application Application
MySQLServer
NDB Cluster(Data nodes)
MySQLServer
MySQLServer
DB DB
DBDB
MGM Server
MGM client
ApplicationApplicationApplication
config.ini
my.cnfSQLNodes
DataNodes
MgmtNodes
18Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
MySQL Cluster HA features
MySQLServerMySQLServer
DataNodeDataNode
DataNodeDataNode
NDBStorage Engine
ManagementServer
ManagementServer
NDB APINDB API
MySQL Cluster
DataNodeDataNode
DataNodeDataNode
MySQLServerMySQLServer
MySQLServerMySQLServer
ManagementServer
ManagementServer
MySQLServerMySQLServer
MySQLServerMySQLServer
X X X X
X XX
19Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Cluster with Replication
S2
S3
• for read scale-out (Reporting):
S1
Appl
ndbd ndbd
ndbd ndbd
mysqld mysqld mysqld
Reporting
rw
20Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Cluster with Replication
• Cluster-Cluster replication for disaster fail over (MySQL 5.0):
Appl
ndbd ndbd
ndbd ndbd
mysqld mysqld mysqld
rw
ndbd ndbd
ndbd ndbd
mysqld mysqld mysqld
fail over
21Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Cluster with Replication
• Cluster-Cluster replication for disaster fail over (MySQL 5.1):
Appl
ndbd ndbd
ndbd ndbd
mysqld mysqld mysqld
rw
ndbd ndbd
ndbd ndbd
mysqld mysqld mysqld
fail over
22Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Cluster with Replication
• circular Cluster-Cluster replication (>= MySQL 5.1.18?):
23Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Cluster examples
• Session handling• Telecom (Mobile)• VoIP• RSS-Feed aggregation• Mail• On-line Games
• Use cluster where– you need HA– you have high write load– you do little Joins and Grouping
24Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
How to go on?
• Read is a caching problem!➔ More RAM or scale-out.
● Write is a batching problem!➔ Batch your load,➔ Buy stronger I/O system.➔ Use MySQL Cluster.➔ SSD!
● What then?
25Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
SSD disruption
• I/O system without any movable parts!• SSD = Solid State disk (Flash memory, NAND,
NOR chips, RAM-SAN).• During the last year(s)• Price from 15'000 USD / 160 Gbyte -> 100 USD /
32 Gbyte• 10-50 times faster than mechanical I/O systems• 1 Mio write cycles dead :-( (special FS!)
• RAM: Huge amount of memory is cheap!
➔ This will disrupt the database world!
26Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
What then?
• With or without SSD we will reach a physical limit. What then?
• Application partitioning:– Split applications
• OLTP vs. OLAP• all in one
– Segment your application
27Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Application partitioning
• OLTP vs. OLAP– hot business data (trx) vs. “old” reporting data
• “All in one”– Sessions, user tracking, ads, chat, booking
• Segment– split by for example 1 Mio users (split by user_id).
28Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Architecture examples
29Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Architecture examples
30Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Architecture examples
31Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Architecture examples
32Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Some more architecture stuff?
• Databases are slow!!!➔ Whenever possible try to avoid databases! Use
memcached for example.
• The SP trap.➔ Stored Procedures are a lock in! Try to avoid SP!
• Use Materialized Views (MV) and/or shadow tables.
• VM/SAN is nice for consolidation but not for performance/scale-out!
• Backup and Staging• MySQL – Proxy
33Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database
Now it's your turn...
• Your problems?
• Let us build a replication...