View
807
Download
1
Category
Preview:
Citation preview
!"
#
$
%
&'
$./run
{
Created by Fabrício Epaminondas ©2015-2020
Designing SCALABLE APPLICATIONS
Short Talk
WHAT IS SCALABILITY?
The concept
“Scalability is the capability of a system, network, or process to handle a growing amount of work, or its potential to be enlarged in order to accommodate that growth.”
Bondi, André B. (2000). Characteristics of scalability and their impact on performance.
3
Let's go deeper…
4
5Scalability for sandwiches
How hungry are you?
A bigger sandwich may fit your need (vertical scaling)
6
7
How many people are hungry?
Think about it…
8
Fast-food restaurant CHAINS
You need to scale the entire solution (horizontal scaling)
Scaling Computing Resources 9
• Processing (CPU)• Memory• Storage• Network
Scale-up vs Scale-out
Vertical Scaling (Scale-up) 10
Generally refers to adding more processors and RAM, buying a more expensive and robust server. • Pros
•Generally less challenging to implement •Less power consumption than running multiple servers •Less licensing costs • Less network hardware than scaling horizontally.
• Cons• Hardware costs •Greater risk of hardware failure causing bigger outages
•Generally severe vendor lock-in and limited upgradeability in the future
Horizontal Scaling (Scale-out): 11
• Generally refers to adding more servers with less processors and RAM. This is usually cheaper overall and can possibly scale infinitely.
• Pros•Much cheaper than scaling vertically •Easier to run fault-tolerance •Easy to upgrade
• Cons•More licensing fees •Bigger footprint in the Data Center •Higher utility cost (Electricity and cooling) •Possible need for more networking equipment
(switches/routers)
Related concepts 12
• Performance•Average Response Time (ART) •Latency (usually network and disk I/O) •Resource consumption
• High Availability•Single Point of Failure (SPOF)•Fault tolerance
USE CASE Weather Forecast
Designing the software
14
• Mobile App to show world-wide weather forecasts
• You planned to reach 1k users at first week.
• Important notes:•How many users will access
the system at same time? •How to model workload? •How to predict the future?
Weather Forecast App
15
• Use performance tests based on scenarios.•Test types:
• Performance•Validates the ART on a scenario (e.g.: systems
response takes 3secs with 100k users)• Load
• Aims to understand required resources for running the system (planning infrastructure).
• Stress•Look for system failures under stress conditions (DoS,
outages, etc.)
Notes
16Solution #1: Two Tier Architecture
(O )Client Server
Internet
4Gb Mem 1 vCPU*
* This virtual CPU unit represents the processing power of a CPU with 2 cores
17Solution #1: Two Tier Architecture
• Pros• Easy setup
• Cons•Single point of failure (1 SPOF) • Vertical scaling costs (buy better hardware, CPU, etc.)
18
• Your app is growing fast and now it reached 10k users after one month.
Weather Forecast App
19Solution #2: Three Tier Architecture
O )Client
Web Server
Data Server
(
4Gb Mem 1 vCPU
4Gb Mem 1 vCPU
Network Latency
Internet
Local/Private Network
20
• Pros• Doubled the computing power • Dedicated and optimized infrastructure (disk, memory, etc.)
• IT operations specialists (database, server management).
• Cons• More points of failure on server side (2 SPOFs) •Vertical scaling costs • More network latency
Solution #2: Three Tier Architecture
21
• Forecast app is being used in most important country cities and it has at about 500k users after two months.
•Some users are noticing failures and outages while trying to get the latest weather information.
Weather Forecast App
22Solution #3: Clustering
O )Client
Web Cluster
Data Server
(
4Gb Mem 1 vCPU
4Gb Mem 1 vCPU
Internet
Local/Private Network
(Load
Balancer
+
23
• Pros•High availability and fault tolerance •Grid computing •Out-of-box setup scaling for stateless applications
• Cons•IT management more complex, e.g.: Deployment and
rollback procedures. •Need to use a data sharing service for stateful
applications or use sticky sessions which may not scale properly
•If Data Server is the bottleneck, you may also need a cluster for it.
•More network latency
Solution #3: Clustering
24Solution #3: Stateful vs Stateless
O)
User2Web
Cluster
Data Server
( Internet
Local/Private Network
(Load
Balancer
A
B
OUser1
Session 1
Session 2
Web session stickiness example
+
25
• After a global market campaign, some users started to complain about a slow UI loading in mobile application, when opening the latest weather information from the internet.
Weather Forecast App
26
• Example of browser caching
Solution #4: Caching
[1] Reference: https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching#cache-control
27Solution #4: Caching
O)
User2Web
Cluster
Data Server
(
Internet +
(Load
Balancer
OUser1
(Cache Server
In Memory Caching
Local Caching
Local Caching
Client Caching
Client Caching
Proxy Caching
In Memory Caching
Example: CDN for Static Web Content: html, css, images, etc.
28Solution #4: Caching
Content Delivery Network(CDN)
29
• Pros•Reduce wait time caused by latency problems •Avoid waste of computing resources by caching long-
running operations like remote calls, disk I/O, etc. •Cons
• Need to define metadata for control the cache storage. •May require to minimal hardware for getting better
results (SSD, RAM, etc.), some devices such as mobile have critical limitations.
• You may see out-of-date results while the old cache was not invalidated yet.
Solution #4: Caching
30
• Your app is growing too fast, it is known throughout the world with more than 1 million users in two month only.
•It is time to have your product running in a cloud solution.
Weather Forecast App
31Solution #5: Full Distributed Cloud Service
Comments 32
,
!"
#
$
%
&'
$./run
{
Created by Fabrício Epaminondas ©2015-2020
ContactFABRICIO EPAMINONDAS - linkedin.com/in/fabricioepa
. fabricioepa.wordpress.com
/ fabricioepa
0 @fabricioepa
Recommended