View
233
Download
0
Category
Preview:
Citation preview
8/13/2019 Andrey Mysqlnd
1/42
1Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
ysqlndA new raPHP Vikinger
Skien, NorwayJune, 9 th200
Andrey Hristo! "andrey# ysql$%o &
8/13/2019 Andrey Mysqlnd
2/42
2Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
'ho is talking to you(
Andrey Hristov works for MyS! as a Software Developer MyS! "onnectors departe#ent author of MyS! $vent Scheduler
%#a&ister e'a#( hacks PHP since a&es has developed #ost of #ys)lnd takes care of e't*#ys)li toðer with +eor& ,ichter
--- tries to answer your )uestions . /ust ask0 --- replies to %so#e( #ails send to andrey1#ys)l-co#
8/13/2019 Andrey Mysqlnd
3/42
3Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
'hat does )mysqlnd* mean(
MySQL Server
The world's most populr ope! sour"e dt#se 2LAM$2 %!inu'3 Apache3 MySQL3 $%$* Perl * Python(
http4**dev-#ys)l-co#*
mys&l!d MySQL !tive driver (or $%$
)tive driver to "o!!e"t (rom $%$ to the MySQL Server Ti&htly inte&rated into PHP3 able to support PHP &oodies )ot !ew progrmmi!g A$* http4**dev-#ys)l-co#*downloads*connector*php5#ys)lnd*
8/13/2019 Andrey Mysqlnd
4/42
+Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
'hat is mysqlnd(
!ibrary and not a new e'tension not e'posed to the userland i#ple#ented in "
Drop5in replace#ent for lib#ys)l works with e'istin& software al#ost4 so#e e'peri#ental functions are no lon&er available
"o#pile5ti#e switch for e't*#ys)li e't*#ys)l support in the develop#ent tree PDO*MyS! support planned
PHP version independent4 PHP 63 PHP 7 theoretically even PHP 83 you #i&ht be able to port it0
8/13/2019 Andrey Mysqlnd
5/42
,Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
lib#ys)l#ys)lnd
MyS! native driver for PHP 5 #ys)lnd
How PHP %onne%ts to +yS-
9end $n&ine
e't*#ys)l PDO*MyS!e't*#ys)li
MyS! Server
MyS! "lient !ibrary 5 lib#ys)l
: MyS! "lient !ibrary option re#ains3 lib#ys)l is not deprecated
: ;ou can
8/13/2019 Andrey Mysqlnd
6/42
-Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
'hy mysqlnd( .he /ros 01
+P!*Dual5!icensehttp4**www-&nu-or&*copyleft*&pl-ht#l
http4**www-#ys)l-co#*co#pany*le&al*licensin&*
f you want closedsource and earn #oney3 share your win and pay
for a co##ercial license=
?SD5Stylehttp4**www-php-net*license
8/13/2019 Andrey Mysqlnd
7/427Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
'hy mysqlnd( .he /ros 02
!ib#ys)l co#es with the MyS! Server updatin& lib#ys)l #eans updatin& the MyS! Server bound to the server release cycle
#ys)lnd is a library %stored in the e't*#ys)li directory( up&radin&3 e-&- after a bu& fi' is si#ple has its own release and develop#ent cycle no lon&er linkin&*co#pilin& issues no need to install lib#ys)l on the build server
leaner code
Ti&htly coupled with PHP and 9end $n&ine can do #a&ic to &et #ore /uice out of your "P@%s( better inte&ration into PHP %see below(
8/13/2019 Andrey Mysqlnd
8/42.Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
'hat are the %ons(
!ess tested than e't*#ys)li 5 lib#ys)l co#bo we have #ore than doubled the nu#ber of tests
and pass the# all we have even found lib#ys)l bu&s with the new tests we have done #ainly "!> tests no web tests we have no*few user feedback
So#e %e'peri#ental( functions are no lon&er available mysqli_embedded_*
,eplication related3 e-&- mysqli_enable_rpl_parse
+ood old ti#es are &one---
8/13/2019 Andrey Mysqlnd
9/42/Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
A3ra%ada3ra4 memory limit
Syste# Me#ory
9end $n&inephp-ini4 #e#oryBli#it C EFM
EF
e't*#ys)li #ys)lnd
e#alloc%(
#alloc%(
lib#ys)l
Me#ory li#it now supported0
8/13/2019 Andrey Mysqlnd
10/4210Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
A3re%ada3ra4 /ersistent %onne%tions
Apache
Worker
"onnection "ache
Worker E
"onnection "ache
,e)uest ,e)uest ,e)uest
Persistent connections like e't*#ys)l and lib#ys)l saves connection overhead4 G6 for Dell DID Store works with Apache prefork*threads and ast5"+> Per process resp- per thread cache
uture4 optional fla& to reset connections properly mysqli_connect('p:localhost', ...)
8/13/2019 Andrey Mysqlnd
11/4211Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
A3ra%ada3ra4 PHP streams
#ys)lnd uses PHP strea#s portable and robust !ot impleme!tede'pose strea# to users theoreti"lly4 you could i#ple#ent a pro'y
8/13/2019 Andrey Mysqlnd
12/4212Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
A3ra%ada3ra4 one %ode 3ase
PHP 6 and PHP 7 use the sa#e code base all e'tensions use one code base of #ys)lnd one fi'3 fi'es it all %one bu&3 breaks it all( @nicode is co#piled out for PHP 6
nevertheless the code has only few Jifdef and is readable >f php-net wants #ys)lnd they can &o for one or two branches
PHP 6 PHP 7
#ys)lnd
8/13/2019 Andrey Mysqlnd
13/4213Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
A3ra%ada3ra4 memory sa!ings
8/13/2019 Andrey Mysqlnd
14/421+Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
A3ra%ada3ra4 read5only !aria3les 02
What e't*#ys)li %or e't*#ys)l3 ---( used with lib#ys)l does4 #ys)li sends a )uery %( result set &et fetched into lib#ys)l buffers %E( #ys)li allocates vals3 then new buffers
%L( #ys)li copies data fro# lib#ys)l to its own buffers calls mysql_ree_result()and deallocates lib#ys)l buffers
What e't*#ys)li used with #ys)lnd does4 #ys)li sends a )uery %( result set is fetched row by row3 every row is different buffer %E( #ys)lnd creates a result set of vals3 pointin& to the buffers
calls mysqlnd_ree_result()which could be li&htwei&ht
8/13/2019 Andrey Mysqlnd
15/421,Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
What e't*#ys)li %or e't*#ys)l3 ---( with lib#ys)l does4 one e'tra allocation for #ys)li buffers one e'tra data copy one e'tra val allocation3
which can be saved with the val cache
Tricks and &oodies of #ys)lnd4 #ys)lnd does thread5safe val cachin& 5
9end $n&ine does this too3 but we brin& it to an e'tre#e0
#ys)lnd records cache statistics %e-&- phpino()output( #ys)lnd collects )uery statistics %#ore later( statistics are per process3 even in a threaded environ#ent
A3ra%ada3ra4 read5only !aria3les 06
8/13/2019 Andrey Mysqlnd
16/421-Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
7ead5only !aria3le im/lementation
zval.type = stringzval.value = *ptr_row1_col1zval.refcount = 1
,ow 5 k L Noe 6 loves F #ys)lnd ---
Misc
zval.type = stringzval.value = *ptr_row1_col1zval.refcount = 2 (or 3)
,ow 5 k L Noe 6 loves F #ys)lnd ---
Misc
,esult set consists of4
ED val bufferresult butter#iscellaneous
val owned by result set4
refcount C
etchin& data4
fast:refcountGG
unlike lib#ys)l always correct hash sie and no rehashin&
unset(ro)4 refcount55
zval.type = stringzval.value = 'Joe'zval.refcount = 1
#ys)liBfreeBresult%(4
Separate %valBcopyBctor%(( all vals for which refcount 3 for e'a#ple4 /oe ro['irstname']
8/13/2019 Andrey Mysqlnd
17/4217Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
A3ra%ada8sure(84 read5only !aria3les
Me#ory allocators are synchronied synchronisation is slow happens even in sin&le5threaded applications
#ys)lnd allocates bi&&er chunks are we wastin& #e#ory #ini#ies heap fraentation3 less work for the allocator
#ys)lnd tries to reuse the vals saves "P@ cycles less #e#ory fraentation ?ut4 will you notice it
8/13/2019 Andrey Mysqlnd
18/421.Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
9:/ertada3ra4 ;!al %a%he
: On #odule start5up the val cache is initialied- Specific nu#ber ofvals3 ini settin&3 are pre5allocated on a conti&uous block of #e#ory-
: When #ys)lnd needs a val3 it asks the cache3 if the cache has afree entry3 it returns a pointer to one- Otherwise3 allocates a val-
: When a result set is freed3 the vals are destructed throu&h thecache- >f a val is not fro# the cache nor#al 9$ is called- Of coursewe need to separate values3 if refcount -
: >f the val is fro# the cache3 efree%( is not called3 which #eans nocritical sections in the allocator- The val can be used a&ain-
: >f a cache val had refcount 3 it won2t be returned for re5usa&e tillthe end of the script- !i#itation3 which can be lifted in furtherversions by usin& &arba&e collection on ne't free result-
8/13/2019 Andrey Mysqlnd
19/421/Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
9:/ertada3ra4 ;!al %a%he 02
The &lobal %per process( cache consists of4 a pre5allocated val block a free list stack a reference counter
a #ute' other variables for operation and statistics created at M>Q>T
The local %per thread( cache consists of4 a &arba&e collection stack reference counter pointer to the &lobal cache created at ,>Q>T
8/13/2019 Andrey Mysqlnd
20/4220Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
9:/ertada3ra4 ;!al %a%he 06
: ?ecause co#ple'ity of se)uential scan of the block for findin&free val is O%n(3 the access ti#e would &row with the cache sie-Thus3 an opti#isation called free5list5stack was introduced-
: ree5list5stack holds pointers to free vals3 thus +$T operationhas O%( co#ple'ity-
: P@T is also is also lei&ht5wei&ht because the val has to beadded to the free5list-
: The stack is or&anised backwards- Qew ele#ents &et lower#e#ory addresses- Thus if there is a batch of +$T operationsthe "P@ will cache the line and the first +$T will hit the DATA
!*!E cache-: +arba&e collection at end of re)uest is very cheap as those
8/13/2019 Andrey Mysqlnd
21/4221Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
9:/ertada3ra4 ;!al %a%he /ro3lems
: ,ecall that on #ys)liBfreeBresult%( the result set has to bescanned and vals with refcount have to be separated-However3 as there is %are( user variable%s( pointin& to a valcache entry3 we cannot perfor# a P@T operation- Thus3 we arestuck with this till the end of the script-
: This #eans that actually with #ys)lnd it2s better if the userdoesn2t call very )uickly mysqli_ree_result()-
: This li#itation can be lifted by tryin& checkin& the &arba&ecollection list of the local cache for vals with refcount C
8/13/2019 Andrey Mysqlnd
22/4222Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
et%h=all0
8/13/2019 Andrey Mysqlnd
23/4223Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
8/13/2019 Andrey Mysqlnd
24/422+Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
8/13/2019 Andrey Mysqlnd
25/42
2,Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
8/13/2019 Andrey Mysqlnd
26/42
2-Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
8/13/2019 Andrey Mysqlnd
27/42
27Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
9:/erimental4 ..-53ased
$'peri#ental i#ple#entation in the SIQ not part of the released version3 e'peri#ental3 will chan&e0 Ti#e5to5live %TT!( invalidation4 stale data can be delivered On the AP>5level4 si#ple to use3 no e'tra tools needed "usto#ieable stora&e %#e#cached3 file3 ---( planned mysqli_query(strin0 query, boolean cache)
php.ini: mysqli.qc_hash_buer_si1e
php.ini: mysqli.qc_ttl
8/13/2019 Andrey Mysqlnd
28/42
2.Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
.ime5to5li!e 0..- /ro/erties
4KF4KK %&& /0(*) users 445 until 11678677
4KF4K %&& /0(*) users 45 until 11678677
4KF4KE %&& /0(*) users 45 until 116786774KF4KL 9&&& users 7
4KF4K8 %&& /0(*) users 45 until 11678677
4K4KK %&& /0(*) users 75 until 11617677 7
4K4K %&& /0(*) users 75 until 11617677
: TT! is si#ple to i#ple#ent and to use: ,eturnin& stale data is possible
: MyS! %Server( uery "ache never serves stale data
: MyS! uery "ache invalidates on a per table basis
: We drea# of . !o promisewe ever #ake it . per table and perrow invalidation
8/13/2019 Andrey Mysqlnd
29/42
2/Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
A3ra%ada3ra4 Per>orman%e
Ti#e
MyS!Apache "o#pilin& $'ecutin& #ys)lnd
Be!"hmr your prti"ulr ppli"tio! yoursel(5
The AP> i#pact in a web setup can be very s#all it #akes no sense to opti#ie so#ethin& down fro# E to
$'cellent #icro bench#ark results can #elt down !et overall ti#e be Ks and the ti#e spend in lib#ys)l K-s !et #ys)lnd be twice as fast %EKK( as lib#ys)l4 K-K6s ;ou &et down fro# Ks to -6s3 that2s only 6 faster0
8/13/2019 Andrey Mysqlnd
30/42
30Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
A3ra%ada3ra4 @ell @V@ Store
O!TP application with s#all result sets: Online shop si#ulation3 uses transactions3 re#inds of T"P5"
: Pro/ect pa&e4 http4**linu'-dell-co#*pro/ects-sht#lJdvdstore
: @sed for a contest in which LAM$had by far outperfor#ed others
: "2t contest4 http4**firebird-sourcefor&e-net*connect*ct5db"ontest-ht#l
: "ontest sub#ission4 http4**www-heise-de*ct*dbcontest*teilneh#er-sht#l
: Qot perfect for #ys)lnd3 because of s#all result sets
We found for the contest sub#ission4
: e't*#ys)li with #ys)lnd4 fastest: e't*#ys)li with lib#ys)l4 7 slower
: e't*#ys)l with lib#ys)l4 L slower
mys&l!d is #out s (st s li#mys&l6 sometimes (ster
http://linux.dell.com/projects.shtml%22%20%5Cl%20%22vdstorehttp://firebird.sourceforge.net/connect/ct-dbContest.htmlhttp://www.heise.de/ct/dbcontest/teilnehmer.shtmlhttp://www.heise.de/ct/dbcontest/teilnehmer.shtmlhttp://firebird.sourceforge.net/connect/ct-dbContest.htmlhttp://linux.dell.com/projects.shtml%22%20%5Cl%20%22vdstore8/13/2019 Andrey Mysqlnd
31/42
31Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
Sometimes >aster(
DER E66 6DE DKE8 EK8F 8K7 FDE D7LF8 LER7F 76KKK
K
EK
8K
7K
FK
DKK
DEK
D8K
D7K
DFK
#ys)liBselectBvarcharBbuffered-php
!ib#ys)l
#ys)lnd
Percent
!ower bars are faster3 lib#ys)l is blue3 #ys)lnd is purpleetchin& KK rows with one varchar colu#n of a variable sie- Data shown forKK rows3 varchar%n(3 KK' overall-
8/13/2019 Andrey Mysqlnd
32/42
32Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
'e like this mi%ro 3en%hmark %hart$$$
!ower bars are faster3 lib#ys)l is blue3 #ys)lnd is purple"onnect3 create KK5rows with varchar%#(3 S$!$"T all3 KKK' ti#es #ys)liBdataBseek%#tBrand%((3 close-
DKK'varchar%DEF(
DKK'varchar%E67(
DKK'varchar%6DE(
DKK'varchar%DKE8(
DKK'varchar%EK8F(
DKK'varchar%8DSE(
DKK'varchar%FLF8(
DKK'varchar%D7R7
DKK'varchar%LL6L
DKK'varchar%766K
K
DKK
EKK
LKK8KK
6KK
7KK
RKK
FKK
SKK
DKKK
DDKKDEKK
DLKK
#ys)liBdataBseekBse)uential-php %DKKK rows3 overall(
!ib#ys)l
#ys)lnd
Percent
8/13/2019 Andrey Mysqlnd
33/42
33Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
.hese are also a%%e/ta3le$$$
DKK row s4 DKKKK' overa ll DKKK rows 4 DKKKK' overall
K
E6
6K
R6
DKK
DE6
D6K
DR6
EKK
EE6
E6K
ER6
LKK
#ys)liBaffectedBrows-php
!ib#ys)l
#ys)lnd
Percent
D KK KK Krows4 o ve ra ll DK KKK Krows4 i ns er tB id %(
K
EK
8K
7K
FK
DKK
DEK
D8K
D7K
DFK
#ys)liBinsertBid-php
!ib#ys)l
#ys)lnd
Percent
E kS$5
6 kS$5
DK kS$5!$"T
LK kS$5!$"T
6K kS$5!$"T
DKK kS$5!$"T
6KK kS$5!$"T
6KKK kS$5!$"T
DKKKKk S$5!$"T
K
EK
8K
7K
FK
DKK
DEK
D8K
D7K
DFK
EKK
#ys)liBfetchBlon&Bbuffered-php %DK rows3 overall(
!ib#ys)l
#ys)lndPercent
6KKKKK'overall
DKKKKK'overall
DKKKK'overall
DKKK' overall DKK' overall
K
6K
DKK
D6K
EKK
E6K
LKK
L6K
8KK
86K
6KK
#ys)liBrealB)uery-php
!ib#ys)l#ys)lnd
Percent
8/13/2019 Andrey Mysqlnd
34/42
3+Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
can be very s#all
--- lib#ys)l and #ys)lnd run thousands of op- per second
--- fro# K-KKKs to K-KKKKLs often #akes no difference
--- we know we #ade it faster3 the charts are real3 butcheck twice if you really profit fro# #ys)lnd
perfor#ance i#prove#ents
--- if you really want #ore charts4 we are preparin& a LKpa&es perfor#ance docu#ent
Typically we found456 to G6-
?ut really no pro#ise0
8/13/2019 Andrey Mysqlnd
35/42
3,Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
7oadma/
: "o#plete the work . be there in ti#e for PHP 7 SS! support "o#pressed protocol support write docu#entation3 educate users
: Support PDO*MyS! Qobody has volunteered for the task--- --- looks like we have to do it
: Move develop#ent and source to php-net Of course4 only if the PHP fellows want #ys)lnd on php-net Of course4 not before its stable enou&h not to break PHP---
8/13/2019 Andrey Mysqlnd
36/42
3-Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
.he >uture is yours
;our wishes are4
Hands up3 please0 There is plenty of roo# on this slide0 We can add #ore slides with your ideas and wishes0
8/13/2019 Andrey Mysqlnd
37/42
37Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
Possi3le >utures$$$
: Prepared state#ents "ache handles4 prepare is e'pensive3 e'ecute is fast Si#ilar to persistent connections or val5cache
: "lever client5side )uery cache TT!5based invalidation as a default ,e#ote and confi&urable stora&e of cache entries Drea#4 MyS! uery "ache style invalidation Drea#4 row5level invalidation
$'pose PHP strea#s to users Transparent Pro'y Translatin& Pro'y
8/13/2019 Andrey Mysqlnd
38/42
3.Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
?dea to try4 3a%kground >et%h
PHP scripts run slow co#pared to " pro&ra#s Server4 #e#ory and locks are released faster3 lowers overall load
"lient4 reduces latency3 SMP support3 works with prefork and threads
Send )uery $'ecute )uery
"ontinue script
Start fetchin&
"reate*invoke back&round
Deliver resultsPrefetch all %asynchronously(
Process row
ree result set
,elease locks
etch row E
Deliver data
Process row E
Deliver dataQetwork
Dear Presenter3do not #ention that this is new to the PHPworld- We have no e'perience with it-
Deal !istener3do not ask for the first perfor#ance fi&urewe saw when tryin& the idea out in a hack-
>t is an idea3 no #ore0>t #i&ht never be i#ple#ented0
8/13/2019 Andrey Mysqlnd
39/42
3/Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
8/13/2019 Andrey Mysqlnd
40/42
+0Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
Some randomly sele%ted Bontri3utors
: !ukas ahwe S#ith AP> ideas and #ore--- http4**pooteeweet-or&*
Peter 9aitsev Statistics3 back&round fetch3 bench#arkin& hints and #ore--- http4**www-#ys)lperfor#anceblo&-co#*about*
http://pooteeweet.org/http://www.mysqlperformanceblog.com/about/http://www.mysqlperformanceblog.com/about/http://pooteeweet.org/8/13/2019 Andrey Mysqlnd
41/42
+1Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database
uestions (
Ok3 you do not dare to ask-However3 at any ti#e3 #eet
#e on >,"3 ask #e by #ail3 ---
8/13/2019 Andrey Mysqlnd
42/42
Ceed3a%k, /lease
Avera&e . +ood . $'cellent . Hands up0
My overall ratin& for the talk is--- This talk was too difficult to understand---
This talk did not &ive enou&h technical infor#ation---
>f > were you3 > would chan&e this--- . raise your voice0
Thank you and &ood bye0
Recommended