Embed Size (px)
Citation preview
From bigGORM-centeredinto a cloud offast nodes
Jorge Uriarte Aretxaga@jorgeuriarte
http://www.slideshare.com/gailen/
About me
Origins
In the beginning...
In the beginning...
Well-known territory
Well-known territory
JVM based!
Well-known territory
Old good SQL& RDBMs
JVM based!
Well-known territory
Old good SQL& RDBMs
JVM based!
GORM magic!
Productive development environment
Productive development environment
Affordable learning curve
Productive development environment
Affordable learning curve
Good-enough performance
Evolution
“Mmmhhh... I wouldn’t build it that way
nowadays...”Enrique Amodeo
“Mmmhhh... I wouldn’t build it that way
nowadays...”Enrique Amodeo
“Mmmhhh...nowadays...
neither would I!”Me
“Mmmhhh... I wouldn’t build it that way
nowadays...”Enrique Amodeo
New brands& countries
Reality Check
Reality bites
Reality bites
-server -Xmx1024m -XX:MaxPermSize=128m -XX
:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:
+UseTLAB -XX:+CMSIncrementalMode
-XX:+CMSIncrementalPacing -XX:CMSIncrement
alDutyCycleMin=0 -XX:CMSIncrementalDutyCyc
le=10 -XX:MaxTenuringThreshold=0
-XX:SurvivorRatio=256 -XX:CMSInitiatingOcc
upancyFraction=60 -XX:+DisableExplicitGC
-server -Xmx1024m -XX:MaxPermSize=128m -XX
:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:
+UseTLAB -XX:+CMSIncrementalMode
-XX:+CMSIncrementalPacing -XX:CMSIncrement
alDutyCycleMin=0 -XX:CMSIncrementalDutyCyc
le=10 -XX:MaxTenuringThreshold=0
-XX:SurvivorRatio=256 -XX:CMSInitiatingOcc
upancyFraction=60 -XX:+DisableExplicitGC -server
-Xms3g -Xmx4g
-XX:MaxPermSi
ze=128m -XX:+U
seParNewGC
-XX:+UseConcMa
rkSweepGC -XX:
+UseTLAB -XX:+
CMSIncremental
Mode
-XX:+CMSIncrem
entalPacing -X
X:CMSIncrement
alDutyCycleMin
=0
-XX:CMSIncreme
ntalDutyCycle=
10 -XX:MaxTenu
ringThreshold=
0
-XX:SurvivorRa
tio=256 -XX:CM
SInitiatingOcc
upancyFraction
=60
-XX:+DisableEx
plicitGC
Productivity slows down...
Fragility...
Evolution stopped...
Slow development
Slow development
Fragile system... prone to errors
Slow development
Fragile system... prone to errors
Tuning hell with JVM heap
Slow development
Fragile system... prone to errors
Tuning hell with JVM heap
Terracota cluster issues
Slow development
Fragile system... prone to errors
Tuning hell with JVM heap
Terracota cluster issues
Overall excessive complexity
Change?Surgery?Refactor?
keep tuning?
Meanwhile... in another mental thread...
A solution looking for a problem?
But get back to the previous point...
Will you dare jump off it?
But get back to the previous point...
• New side-project• Potentially deadly traffic• Still to be tighly connected with our core system
• New side-project• Potentially deadly traffic• Still to be tighly connected with our core system•Solutions?
•Dedicated system?•More machines & tuning?•Catch that train!
Proof of concept and... JUMP!
Some details?
Prototype- show me the code -
RedisFast Controllers
Ajax (fewer requests)
Fast importer from existing XMLs
Quick & dirty (1w)
Groovy!!!(XML parsing, JSON
rendering, grails controllers, ...)
Elasticsearch
• Simply groovy.util.XmlSlurper
• Simply groovy.util.XmlSlurper• Two importers, really
• Simply groovy.util.XmlSlurper• Two importers, really• Bulk
• Simply groovy.util.XmlSlurper• Two importers, really• Bulk• Incremental (continuous)
• Simply groovy.util.XmlSlurper• Two importers, really• Bulk• Incremental (continuous)
• Born as a temporary hack...
• Simply groovy.util.XmlSlurper• Two importers, really• Bulk• Incremental (continuous)
• Born as a temporary hack...soon to give way to a better system
• Redis master/slaves replication
• Redis master/slaves replication• Fast
• Redis master/slaves replication• Fast• Trivial
• Redis master/slaves replication• Fast• Trivial• Fast
• Redis master/slaves replication• Fast• Trivial• Fast• Trivial
• Redis master/slaves replication• Fast• Trivial• Fast• Trivial• Did I mention fast and trivial?
Deadly simple Grails app
Deadly simple Grails app
Deadly simple Grails app
Deadly simple Grails app
“Look mum, no GORM!”
From ‘searchable’ to ElasticSearch
• Fat-trimmed new fronts• Fast, *light* application• Information still safe in the core system
• Much more capacity with less resources• Simple integration
Wait... where did the red arrows go?
New system
Old system
Subdomain based write-through
Subdomain based write-through
• Login, session, tracking...
Subdomain based write-through
• Login, session, tracking...• Purchases, payments, subscriptions
Subdomain based write-through
• Login, session, tracking...• Purchases, payments, subscriptions• Everything goes to our *good old core system*
Subdomain based write-through
• Login, session, tracking...• Purchases, payments, subscriptions• Everything goes to our *good old core system*• ACID, consistency, GORM, SQL, transactions...
Subdomain based write-through
• Login, session, tracking...• Purchases, payments, subscriptions• Everything goes to our *good old core system*• ACID, consistency, GORM, SQL, transactions...• ...but now with a small fraction of the load
Evaluation
Evaluation
Terracota/Clustering approach vs “new thing”
Evaluation
Terracota/Clustering approach vs “new thing”
Memory, speed, size, codebase
Evaluation
Terracota/Clustering approach vs “new thing”
Memory, speed, size, codebase
Decoupled search system
Evaluation
Terracota/Clustering approach vs “new thing”
Memory, speed, size, codebase New problems,
synchronization
Decoupled search system
Productivity increasing again...
New landscape
New landscapeGet rid of XML importer(event driven pub/subs...?)
New landscapeGet rid of XML importer(event driven pub/subs...?)
Delete old code (yeah!)
New landscapeGet rid of XML importer(event driven pub/subs...?)
Keep pushing for speed
Delete old code (yeah!)
New landscapeGet rid of XML importer(event driven pub/subs...?)
Keep pushing for speed
Delete old code (yeah!)
Keep it simple (this time)
Just ask me... in two years
@jorgeuriarte
External sources
PITAhttp://www.quicksales.com.au/ad/p-i-t-a-pain-in-the-ass-car-number-plate/2606513
Hamsterhttp://www.flickr.com/photos/sualk61/3117477410/
Cliff viewhttp://www.flickr.com/photos/epsos/4376727123/sizes/l/
Knight in the white horsehttp://www.flickr.com/photos/photoplod/7982134342/
Snailhttp://www.flickr.com/photos/raphaelquinet/693813637/
Fast snailhttp://www.flickr.com/photos/zorro13/4086173313
Timanfaya devilhttp://www.flickr.com/photos/yelacis/6941891989/
Light patternshttp://www.flickr.com/photos/aerosolhalos/6676192013/
Kick in the top of the cliffhttp://www.flickr.com/photos/4elevenpix/5056089861/
Homer evolutionhttp://www.flickr.com/photos/photonquantique/1858685882/
Lego machinehttp://www.flickr.com/photos/djimison/3058818283/
Landscapehttp://www.flickr.com/photos/blmiers2/6112610781/
Examhttp://www.flickr.com/photos/albertogp123/5843577306/
Future is pasthttp://www.flickr.com/photos/tind/7533681980/
"From big GORM-centered into a cloud of fast nodes" por Jorge Uriarte se encuentra bajo una
Licencia Creative Commons Atribución-CompartirIgual 3.0 Unported.
