Upload
skaramanlakis
View
229
Download
0
Tags:
Embed Size (px)
DESCRIPTION
insights on building a microservices architecture!
Citation preview
BUILDING A MODERNMICROSERVICES
ARCHITECTURE @
QCON LONDON - 2015YONI (JONATHAN) GOLDBERG
- GiltDirect, Sale Personalization, Loyalty, SEO, Post-purchase, Login/Registration
- MIT CS BS/Meng | Google | IBM | IDF
- Israel | Brooklyn | Coffee | JS/Node | Arduino | Running | Kite Surfing | Poker
THREE TAKE AWAYSFROM GILT'S STORY
Which problems will be solved?Which challenges will you face?Is it the right choice for you?
Flash Sales Business Founded in 2007
Top 50 Internet-Retailer~150 Engineers
WHAT IS GILT?
ANOTHER WAY TO LOOK ATGILT
THE CLASSIC STARTUPSTORY
THE EARLY DAYS2007 - Ruby on Rails the hottest new thing
The goal was to get to market fast
WE WERE ABLE TO HANDLE OURTRAFFIC PRETTY WELL
UNTIL LOUBOUTIN CAME TOGILT
TECHNOLOGY PAIN POINTS -2009
Spike required to launch 1,000s of ruby processesPostgres was overloadedRouting traffic between ruby processes sucked
DEV PAIN POINTS1000 Models/Controllers, 200K LOC, 100s of jobsLots of contributors + no ownershipDifficult deployments with long integration cyclesHard to identify root causes
WE NEEDED TO SOLVETHE PROBLEM FAST
THREE THINGS HAPPENEDStarted the transition to the JVMM(a/i)cro-Service Era StartedDedicated data stores
WHY JVM?Widely adoptedStableBetter support for concurrencyBetter GC vs MRI
FIRST 10 SERVICES
WE SOLVED 90% OF OUR SCALINGPROBLEMS
BUT NOT THE DEVELOPERS PAINPOINTS
SOLVED PAIN POINTSSpike required to launch 1,000s of ruby processesPostgres was overloadedRouting traffic between ruby processes sucked
STILL OPEN PAIN POINTSNew services became semi-monolithic1000 Models/Controllers, 200K LOC, 100s of jobsLots of contributors + no ownershipDifficult deployments with long integration cycles
WHY WE DOUBLED DOWN ON MICRO-SERVICES
Empower teams and ownershipSmaller scopeSimpler and Easier deployments and rollbacks
WE BEGAN THE TRANSITION TO SCALA AND PLAY
LOSA - LOTS OF SMALL (WEB) APPS [SAME AS MICRO-SERVICES BUT FOR WEB-APPS]
AS OF LAST WEEK WE HAVE AROUND 300 SERVICES IN PROD
WHY THE INCREASE?
APP BOOTSTRAP
rake bootstrap:admin-web # Bootstrap a admin-web servicerake bootstrap:client-server-core # Bootstrap a client-server-core servicerake bootstrap:play # Bootstrap a play servicerake bootstrap:play-ui-build # Bootstrap a play-ui-build servicerake bootstrap:sbt-library # Bootstrap a sbt-library servicerake bootstrap:schema # Bootstrap a schema service
DEMO
HOW TO DEFINE AMICROSERVICE?
FUNCTIONALITY SCOPENUMBER OF DEVS INVOLVED
"A SERVICE-ORIENTED ARCHITECTURECOMPOSED OF LOOSELY COUPLEDELEMENTS THAT HAVE BOUNDED
CONTEXTS"
- ADRIAN COCKCROFT
CURRENT CHALLENGESDeployments and Testing (Functional/Integration)Dev/Integration EnvironmentsService DiscoverabilityWho owns this service!?Monitoring
MICRO SERVICEDEPLOYMENT V3
CHALLENGES THAT WERE SOLVED IN V2:
Frustrating to deploy semi-manually (Capistrano)Scary to deploy other teams servicesHard to execute functional tests between services
SBTMotivation: Scala adoptionComplex Scala syntaxCool features: ~test, shell, consoleHard to debug
GILT-SBT-BUILDSimple config for all the servicesPulls many plugins: [nexus, testing, RPMs, run scripts, Monitoring, SemVer, ...]Custom commands (e.g 'sbt release')
ION-CANNON + SBTRun tests on dedicated EnvDark canary releasesEasy rollbacksIntegrated health checks
DATA CENTERLIMITATIONS
&& MORE TECHNOLOGY
OWNERSHIP
IMMUTABLEINFRASTRUCTURE
V3Per department AWS account and budgetLonger deploys [New instances / CNAME swaps]Each team has AWS expertise
API LOVE
WELL DEFINED REST APIS SOLVE DISCOVERABILITY, DOCUMENTATION
AND INNER ADOPTION
APIS @ GILTand
Describe the API in simple jsonAuto generates versioned docs, routes and clientsPer team - API design committee
www.apidoc.me Swagger.io
DEPENDENCY FUN [DEMO]
"MID-TIER MICRO-SERVICE"
BIGGEST PERFORMANCE CHALLENGENETWORK IO
ON DEV/INTEGRATION ENVIRONMENTSThe hardware is not strong enoughNo one wants to compile 20 servicesService Dependencies
EACH TEAM HAS A STAGINGENV
SERVICE_PORTS=[4001, #listing-service8235, #svc-user-set9420, #svc-free-fall7895, #svc-Loyalty8155, #web-loyalty9410, #web inventory status7898, #admin-loyalty7899, #notification7102, #rouge9530, #svc-component6802, #svc-waitlist-submit4066, #svc-action-sale....
PORT_FORWARD_ARGS=SERVICE_PORTS.map { |port|['-L', "#{port}:localhost:#{port}"]}
STAGING DIFFICULTIES:Hard to keep all the services up to dateMaxed our staging env capacitiesShared data across legacy apps
WITH OUR MIGRATION TO AWS [TEAM RESPONSIBILITY]A pool of integration instances and dbsDevelop with prod instances [Not a fan]
ON OWNERSHIP
"code stays much longer than people" - SB
CODE OWNERSHIP
CURRENT APPROACHCode Review!Code Review!Code Review!Team owns services, not individual developersOwnership transfer
DATA OWNERSHIP
WE TRANSITIONED TOMICRO-DBS
MICROSERVICES NEEDS THEIR OWNMONGODB | POSTGRES | RDS |
VOLDEMORT
MANAGE MICRO-RELATIONALDBS
SCHEMA EVOLUTIONMANAGER
https://github.com/gilt/schema-evolution-manager
PRINCIPLES OF SCHEMA EVOLUTION
MANAGERIndependent from the service codeManages the schema evolutions in a Git repoSchema changes are deployed as tar fliesNo rollbacksSchema changes are required to be incremental
ON MONITORING
THE TOOLS WE USE
GRAPHITE / OPENTSDB
BUT IT WASN'T ENOUGH
CAVE
HTTP://CAVELLC.GITHUB.IO
CONTINUOUS | AUDIT | VAULT |ENTERPRISE
ALERT EXAMPLES [SCALA]orders [shipTo: US].sum.5m < 1000response-time [svc: svc-team, env: prod].p999 > 1000
SUMMARY
WHAT'S NEXT ?BUILD YOUR NEXT FEATURE
IN A NEW SERVICE
@yoni_goldberg [email protected]
QUESTION TIME
www.yonigoldberg.com