Upload
emerson-macedo
View
261
Download
1
Embed Size (px)
Citation preview
CONTEXT
User Profile API was developed in 2012.
It’s a well written Ruby on Rails
application , responsible for track
logged user actions
CONTEXT
User Profile API was developed in 2012.
It’s a well written Ruby on Rails
application , responsible for track
logged user actions
CONTEXT
Our Video Player POST Watched
Percentage to our Endpoint so we can
provide Keep Watching Percentage
to our Logged Users. It does it
every 10 seconds
CONTEXT
Our Video Player POST Watched
Percentage to our Endpoint so we can
provide Keep Watching Percentage
to our Logged Users. It does it
every 10 secondsx
x
CONTEXT
Globo is making Binge Watching experiments combined
with Online First for new TV Shows
TV Shows
THE PROBLEM
The average response time
was good, but percentiles were hurting the application
Avg
70ms
70ms
THE PROBLEM
The average response time
was good, but percentiles were hurting the application
99
2.5s
95
1s
Avg
FIRST CHANGE
We increased the
computational resources
from 2 to 4 bare metals Each one with 24 CPUs and 64GB of RAM
SECOND CHANGE
We decided to try a deployment with Tsuru containers with auto scaling, each one with 1-4 vCPU and 2GB of RAM
SECOND CHANGE
We decided to try a deployment with Tsuru containers with auto scaling, each one with 1-4 vCPU and 2GB of RAM
93 containers
CONTEXT
Our Video Player POST Watched
Percentage to our Endpoint so we can
provide Keep Watching Percentage
to our Logged Users. It does it
every 10 seconds
CONTEXT
Our Video Player POST Watched
Percentage to our Endpoint so we can
provide Keep Watching Percentage
to our Logged Users. It does it
every 10 secondsx
x
ARCHITECTURAL OVERVIEW
User Profile API is a
classical Ruby on Rails
application, which also
uses Resque for
background jobs
ARCHITECTURAL OVERVIEW
User Profile API is a
classical Ruby on Rails
application, which also
uses Resque for
background jobs
ARCHITECTURAL OVERVIEW
User Profile API is a
classical Ruby on Rails
application, which also
uses Resque for
background jobs
BLOCK
BLOCK
BLOCK
ARCHITECTURAL OVERVIEW
User Profile API is a
classical Ruby on Rails
application, which also
uses Resque for
background jobs
CONTAINER DISTRIBUTION
Our containers was distributed for the Rails App, Resque Workers and Resque Scheduler
93 containers
CONTEXT
Our Video Player POST Watched
Percentage to our Endpoint so we can
provide Keep Watching Percentage
to our Logged Users. It does it
every 10 seconds
CONTEXT
Our Video Player POST Watched
Percentage to our Endpoint so we can
provide Keep Watching Percentage
to our Logged Users. It does it
every 10 secondsx
x
THIRD CHANGE
We saw that just ONE endpoint was
responsible for 80%, of application throughput
Pareto
THIRD CHANGE
We saw that just ONE endpoint was responsible for 80%, of application throughput
Pareto
20% of effort will solve 80% of the problem
WHY ELIXIR
Elixir generates Erlang byte code and runs
on BEAM VM which has 30 years of development
x
THIRD CHANGE
We rewrote the Ruby POST
endpoint from scratch to an
Elixir version
SAME APPLICATION
ERL PROCESS
ERL PROCESS
TOOLS
We chosse phoenix
framework to create
our Elixir API and
we’re using many other
community libs
TOOLS
We chosse phoenix
framework to create
our Elixir API and
we’re using many other
community libs
Ecto HTTPoison
Exrm CacheX
GenRetry FakeServer
Corsica
PROBLEMS
MongoDB Driver did not has
Replica Sets support. We had
to implement it
http//github.com/emerleite/mongox
http//github.com/emerleite/mongox_ecto
PROBLEMS
Elixir did not has NewRelic support.
We need to create an ad-hoc implementation using Exometer
PROBLEMS
Elixir did not has NewRelic support.
We need to create an ad-hoc implementation using Exometer
https://github.com/Feuerlabs/exometer_core
2013 2016
10k
60kFINAL RESULT
After the Olympics and with
Binge Watching, the
throughput increased ~50%
2013 2016
10k
60kFINAL RESULT
After the Olympics and with
Binge Watching, the
throughput increased ~50%
2016 / 2
90k