20 Typical MySQL Mistakes and how to avoid them - Percona › sites › default › files › ... · 20 Typical MySQL Mistakes and how to avoid them Short real world examples and

©2016Percona©2016Percona©2016Percona©2016Percona©2016Percona©2016Percona©2016Percona1

JanosRuszo

20TypicalMySQLMistakesandhowtoavoidthemShortrealworldexamplesandtipstoavoidthem

SeniorConsultantPerconaUniversityBudapest11/05/2017


OSsidemistakesopenfileslimit,storage,timezones

©2016Percona

Openfileslimit/security.conf/systems▪ Defaultlimit:1024openfiles▪ Defaultconfigurationfile:/etc/security/limits.conf▪ RHEL6+/Fedora9+:/etc/security/limits.d▪ IncreasethelimitsforMySQL!

3

©2016Percona

DatadirectoryonNFS▪ Avoidifpossible▪ LatencyofSSD:0.031ms/LatencyofNFS~=networklatency(0.5-1ms)▪ Higherlatency->lowerIO/s▪ NFSservercangoaway(shortnetworkissue/serversideissue)▪ UsuallyNFSserverisshared,noguaranteedperformance▪ UsuallyDBteamcannotaccesstheNFSserver->hardertodebugperformanceissues

4

©2016Percona

Timezones▪ MySQLusesthesystem’stimezone▪ Master/slavecanhavedifferenttimezone->differentresults▪ Functionswhichreliesonthelocaltime,willproducedifferentresultsonreplication[NOW()/FROM_UNIXTIME()]• Binlogformat“ROW”replicatestheactualvalue,whichmakesitresistanttothisissue

https://dev.mysql.com/doc/refman/5.7/en/replication-features-timezone.html

5


Configurationmistakesvariousconfigurationvaluesandwhattosetthemto

©2016Percona

BufferPoolSizing▪ Generalrecommendation:75-80%oftotalmemory▪ InnoDBcachesdata+index▪ Dataserverfrommemoryisfast,servedfromdiskisslower

▪ SHOWENGINEINNODBSTATUS▪ Bufferpoolhitrate999/1000

7

©2016Percona

Charactersets▪ Charactersetlevels:▪ Server->Database->Table->Column▪ Clientandstoringcolumn/tablecanhavedifferentcharactersets->that’sbad

8

©2016Percona

Charactersets- exampleCreate Table: CREATE TABLE `text` (`content` text

) ENGINE=InnoDB DEFAULT CHARSET=utf8

mysql> SHOW VARIABLES LIKE 'character_set_client%';+----------------------+--------+| Variable_name | Value |+----------------------+--------+| character_set_client | latin1 |+----------------------+--------+1 row in set (0.00 sec)mysql> insert into test.text set content='árvíztűrő tükörfúrógép’;mysql> select * from text;+-------------------------------+| content |+-------------------------------+| árvízt?r? tükörfúrógép |+-------------------------------+1 rows in set (0.00 sec)

9

©2016Percona

MyISAMtables▪ MyISAMisnotcrashsafe->youcanlosedataonservercrash▪ Notcachingdata,onlytheindexes▪ MyISAMsupportonlytablelevellocking▪ NotMVCCcompilant,notransactions▪ Slowlydeprecated

10

©2016Percona

MultiColumn/UUIDPKonInnoDB▪ InnoDBorganisesdataorderedbythePrimaryKey▪ SecondaryindexalwayscontainsthePrimaryKey,whichresultsinlargeindexes

▪ UUIDisnotsequentialwhichresultsinfragmentationandrandomwrites

11

©2016Percona

MultiColumn/UUIDPKonInnoDB

12

©2016Percona

Toomanytablesontheserver▪ Onfilesystem,eachdatabaseis1directory▪ Eachtableisatleast2files(.frm+.idb)atInnoDB▪ Standarddrupalinstall:74tables▪ Sharedhostwith1000drupalinstalls->74.000tables▪ Variousinformation_schemaquerieshavetoopenalltables▪ evenfilesystemfind/lscanbeslow

13

©2016Percona

Straightalter(directalter)▪ Straightalterlockswriteaccesstothetable▪ Forlargetablesthiscantakealongtime▪ SinceMySQL5.6onlinealterpossibleinsomecases(SeeMySQLdocumentation)

▪ Fornon-blockingschemamodificationsusept-oscorgh-ost

14

©2016Percona

Versionmismatch▪ Ansible:- name: Install MySQL-Oracle Server

yum: pkg={{ item }} state=presentwith_items:

- MySQL-server- MySQL-client

▪ Centos/Redhat:yum install MySQL-server

▪ Debian/Ubuntu:apt-get install percona-server-server-5.6

15

©2016Percona

Versiondifferences▪ Duetooptimisationsversiondifferencecanresultinperformancedifferences

[root@server-1~]#mysql--versionmysqlVer14.14Distrib5.6.34-79.1,forLinux(x86_64)using6.2

[root@server-2~]#mysql--versionmysqlVer14.14Distrib5.6.35-81.0,forLinux(x86_64)using6.2

▪ Youcanusethepackagesdirectly▪ Useownrepository

16

©2016Percona

Configdifferences▪ Configdifferencesbetweenmy.cnfandruntimeconfiguration▪ Afterrestartthingscanbreak/unexpectedbehaviour▪ Differentconfigbetweenslaves

./pt-config-diff /etc/my.cnf h=localhost3 config differencesVariable /etc/my.cnf localhost========================================innodb_thread_concurrency 0 4wait_timeout 600 3600read_only ON OFF

17

©2016Percona

Usingserver’sIPasserver_id▪ Don’tusemethodslikeremovingdotsfromipaddresses

▪ 10.77.12.3(1077123)▪ 10.77.1.23(1077123)

▪ MySQLtruncatestheserver_idifit’sbiggerthanthemaxvalueforanINT(4294967295)▪ 192.168.111.222(192168111222->4294967295)▪ 192.168.222.222(192168222222->4294967295)

▪ 4508354421957495439->4294967295▪ 12354356476576->4294967295

18

©2016Percona

wait_timeout,max_connections▪ Defaultvalueofwait_timeout:28800seconds▪ Usemax_user_connections,thiswillpreventasingleuserfromusingallconnections

▪ MySQLreservesanextraconnectionslot(max_connections+1)forauserwithSUPERprivileges

▪ gdb-p$pid -ex"set max_connections=3000--batch

19

©2016Percona

Replicationlag▪ Replicationissinglethreaded!▪ Alongrunningdelete/update/altercancausehugereplicationlags,whichcausesapplicationstoreadstaledata

▪ 5.6introducedmultithreadedreplicationfordifferentdatabases▪ 5.7introducednewmethodsformultithreadedreplication

20

©2016Percona

Toomanyslaveson1master▪ Masterlogsallthewritesinthebinarylog▪ Allslavesneedstopullthebinarylogs

▪ Withhighnumberofslaves(10-20),theycausehugeIOandNetworkoverheadonthemaster

▪ eg.:15slaves,100MBwrite/minute▪ 15*100->1.5GB/minute->25MB/sec(200Mbps)

▪ Useintermediateslavesorshardthedatabaseintosmaller“blocks”

21

©2016Percona

Badqueries▪ Querieswhicharenotusingindexesoraggregatinglotofdatacanbedeadly

▪ LIKE‘%something’orWHEREwithoutindexmysql> explain select * from text where data like '%something';+----+-------------+-------+------+---------------+------+---------+------+---------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+-------+------+---------------+------+---------+------+---------+-------------+| 1 | SIMPLE | text | ALL | NULL | NULL | NULL | NULL | 2092230 | Using where |+----+-------------+-------+------+---------------+------+---------+------+---------+-------------+

▪ JOINoncolumnswithoutindexmysql> explain select text.* from text join text2 on text.data = text2.data where text2.data = 'something';+----+-------------+-------+------+---------------+------+---------+------+---------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+-------+------+---------------+------+---------+------+---------+-------------+| 1 | SIMPLE | text2 | ALL | NULL | NULL | NULL | NULL | 2091815 | Using where || 1 | SIMPLE | text | ALL | NULL | NULL | NULL | NULL | 2092230 | Using where |+----+-------------+-------+------+---------------+------+---------+------+---------+-------------+

22

©2016Percona

Badqueries▪ Querieswhicharenotusingindexesoraggregatinglotofdatacanbedeadly

▪ Monitorslowqueries:▪ PerconaMonitoringandManagementQueryAnalyser▪ pt-query-digest▪ Anemometer▪ Vividcortex

▪ Handlequeries:▪ pt-kill▪ Optimise

▪ Cache!

23

©2016Percona

MaximumInteger▪ INT:MosttypicalcolumntypesforPrimaryKeys(2147483647)▪ UNSIGNEDNOTNULL(4294967295)▪ ArolledbackTXstillincreasestheAUTO_INCREMENT

▪ MonitortheusageofINTcolumnswithAUTO_INCREMENT▪ Starttoplanintime.▪ PrepareapplicationtouseBIGINT(18446744073709551615)

▪ https://github.com/RickPizzi/pztools/blob/master/findmax.sh

24

©2016Percona

LOADDATALOCALINFILE▪ “InaWebenvironmentwheretheclientsareconnectingfromaWebserver,ausercoulduseLOADDATALOCALtoreadanyfilesthattheWebserverprocesshasreadaccessto(assumingthatausercouldrunanystatementagainsttheSQLserver).“▪ https://dev.mysql.com/doc/refman/5.6/en/load-data-local.html

▪ Todisable:▪ SETGLOBALlocal_infile=OFF▪ my.cnf:local_infile=OFF

25

©2016Percona

LOADDATALOCALINFILE- example▪ mysql> LOAD DATA LOCAL INFILE '/etc/shadow' INTO TABLE `text` (data);▪ Query OK, 27 rows affected (0.00 sec)▪ Records: 27 Deleted: 0 Skipped: 0 Warnings: 0▪ mysql> select * from text limit 0,1;▪ +-----------------------------------------------------------------------------------------+▪ | data |▪ +-----------------------------------------------------------------------------------------+▪ | root:$6$DytrzB7fNRZakzdw$.UM./RzgQ2s1p.hJ<redacted>AJnx3z6u1amvHWUHYBo.n/::0:99999:7::: |▪ +-----------------------------------------------------------------------------------------+▪ 1 row in set (0.00 sec)

26

©2016Percona

Backupcaveats▪ Mostcommonbackupmethods▪ Logical▪ mysqldump▪ mydumper

▪ Cold▪ stopmysql,archivedatafiles

▪ Snapshots▪ LVM(performanceoverhead)▪ EBSsnapshots

▪ HotorOnline▪ MySQLEnterprisebackup(expensive)▪ PerconaXtraBackup

27

©2016Percona

Backupcaveats- Logical▪ mysqldump▪ Runsforalongtimeforlargerdatabases▪ Singlethreaded▪ Locksthedatabasesforconsistentbackup(exceptInnoDBtableswith—single-transactionoption

▪ mydumper▪ Multithreaded▪ StillneedtolockMyISAMtables,butforashortertime▪ Fasterrestoresduetomultiplethreads

28

©2016Percona

Backupcaveats- Snapshot▪ LVMsnapshot▪ Verylargeoverheadforwrites!▪ Readspeeddegradesassnapshotspaceisconsumed▪ https://www.percona.com/blog/2013/07/09/lvm-read-performance-during-snapshots/

▪ EBSsnapshot▪ quickandlight▪ SnapshotisstoredinS3,quicktorestore▪ !!EBScreatedfromsnapshotiscold!!▪ “Ifyouaccessapieceofdatathathasn'tbeenloadedyet,thevolumeimmediatelydownloadstherequesteddatafromAmazonS3,andthencontinuesloadingtherestofthevolume'sdatainthebackground”

▪ Inourtests,readspeedfornotyetdownloadeddatais4-6MB/s▪ http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSSnapshots.html

29

©2016Percona

Backupcaveats- HotorOnline▪ PerconaXtraBackup▪ OnlineopensourcebackuptoolfromPercona▪ Parallel▪ Compression▪ Encryption▪ Streaming▪ Throttle(Doesn’tworkwithstreaming!)

▪ MyISAMtablesstillneedstobelockedduringbackup!

30

©2016Percona

Backupcaveats- mysqlbinlog▪ ForPoint-in-timerestore,weneedthebinarylogstobebackedupaswell!

▪ BackupstakenwithXtrabackupareconsistentacrossdatabases▪ Backupsarenotconsistentacrossclusters(endtimematters)

▪ Consistentrestorerequirespointintimerecovery▪ mysqlbinlogsupports--read-from-remote-serverstartingfromMySQL5.5▪ expire_logs_days=10bydefault

31

DATABASE PERFORMANCEMATTERS

DatabasePerformanceMattersDatabasePerformanceMattersDatabasePerformanceMattersDatabasePerformanceMattersDatabasePerformanceMatters

32

Documents

20 Typical MySQL Mistakes and how to avoid them - Percona › sites › default › files › ... · 20 Typical MySQL Mistakes and how to avoid them Short real world examples and