Upload
phil-calcado
View
1.222
Download
2
Tags:
Embed Size (px)
Citation preview
No free lunch, indeed:Three years
Phil CalçadoSoundCloud
of microservicesat SoundCloud
> 11 hours of audio uploaded every minute
~ 300 million people every month
heaps have been written about microservices in
the past year-ish
tl;dr
• Rapid provisioning• Basic Monitoring• Rapid application deployment
These make sense
• Rapid provisioning• Basic Monitoring• Rapid application deployment
this makes sense
why does it make me so nervous?
the SoundCloud story you might know
the pre-history
SoundCloud, circa 2011
let’s prepare for the “microservices
explosion"
#1 provisioning
what was cool in 2010-11
much better than anything else at the
time
a problem
no resource limits (i.e. cgroups)+
naïve scheduling=
loud neighbour in your own datacentre
a problemmade for
most of our services migrated to
the problem
time
start workon v1
v1 100% deployed
start work on v2
before we go sophisticated, let’s
simplify what we have
warmed up pool
machineintake
not the final solution, but will buy us some
time
#2 telemetry
state of telemetry tools circa 2011-12 wasn’t
great
StatsD
let’s build our own!
but that’s not what broke…
obvious with a monolith
monolith
not so much now
?
?
?
standardise dashboards
standardise operations
https://twitter.github.io/twitter-server/Features.html#http-admin-interface
visualise
add management lines, all the way up, to
escalation policies
#3 deployment
> git
SquashFS
> make
unit tests integration tests
acceptance tests perf tests
> make
/dev/null
we ended up with 7 different deployment
scripts
> make > gitunit tests integration tests
unit tests integration tests
acceptance tests perf tests
containers let you spawn your mini-
SoundCloud
but why was I so nervous?
because wemessed up
there are simple and incremental ways to address these
• Rapid provisioning• Basic Monitoring• Rapid application deployment
“uh? do you think Netflix got it right the
first time?"
some good things
Q&A