Embed Size (px)
Citation preview
about• Data Engineer• @Crowdcube
uses• serialised Arrays• JSON• large strings• images*
how does it work?
alter table•creates a temporary table with the new structure
•copies the data from the old table to the new one
•consolidates the new table
example
alter PK: INT -> BIGINT
- 2.147.483.648 TO
2.147.483.647
INT
- 922.337.203.854.775.808 TO
922.337.203.854.775.807
BIGINTIllus
trat
ion
out o
f sca
le
case• > 750 GB• > 380 million lines• 3 TEXT fields• Auto increment: 898.191.090
how long did it take?
2 days
why?
speed
fast slow
storage• 1 file in the disk for each row for each TEXT
field• stored in different location than the table
data itself• each field up to 4 MB
engines
MyISAM• fastest read speed• supports FULLTEXT indexes• non transactional• less data reliability
InnoDB• transactional• better data integrity• until MySQL 5.6 it didn’t support FULLTEXT
indexes in TEXT fields
querying into TEXT fields
• inefficient search with LIKE statement• slow DDL operations (like alter table)• unnecessary increase of the table size
possible alternatives
fastest search
search servers• indexes large bodies• api bindings• decoupled
smart retrieval
content delivery network
• high availability• cheap
TEXT field within the RDBMS
conclusion• there is no silver bullet solution• some implementations may add an
additional layer of complexity to the application
• some implementation works better with decoupled applications
thanks!come say hi :)
