Upload
ngobao
View
244
Download
3
Embed Size (px)
Citation preview
Topics
1. What is load balancing? 2. Load balancing techniques 3. Load balancing strategies 4. Sessions 5. Elastic load balancing
What is load balancing?
“load balancing is a technique to distribute workload evenly across two or more computers, network links, CPUs, hard drives, or other resources, in order to get optimal resource utilization, maximize throughput, minimize response time, and avoid overload.”
─Wikipedia
Load balancing techniques
• Round robin DNS • DNS global server load balancing • Layer 4 load balancer • Web switch • Reverse proxy
Round Robin DNS Configure multiple A records for single name
– Return next IP address from circular list – Some servers may try to return closest IP address.
Problems – Clients cache addresses. – No awareness of availability status of addresses.
Round Robin DNS Entry
> dig google.com ;; ANSWER SECTION: google.com. 21 IN A 74.125.225.18 google.com. 21 IN A 74.125.225.19 google.com. 21 IN A 74.125.225.20 google.com. 21 IN A 74.125.225.16 google.com. 21 IN A 74.125.225.17
DNS Global Server Load Balancing 1. Client makes a DNS query to ISP
DNS server. 2. ISP DNS server looks up DNS
records. 3. GSLB receives DNS query. 4. GSLB algorithm examines site
health and latency to determine which site best.
5. GSLB returns IP address to client of site that is available and has lowest latency.
Problem: DNS servers and clients cache responses for long periods of time.
Layer 4 Load Balancing Switch
Network Address Translation (NAT) for LB – One external IP address – Switch maps incoming connections to one of the
pool of servers. – Switch maps outgoing connection source IP
address to IP address of the switch.
Additional features – Better load balancing algorithms – Detection of availability status of servers
Application
Presentation
Session
Transport
Network
Data Link
Physical
Web Switch
AKA content switch or application switch Balances load at application layer
– Multiple load balancing algorithms – Handles SSL for servers – May also perform compression – May also add firewall features
Application
Presentation
Session
Transport
Network
Data Link
Physical
Reverse Proxy
• Proxy server has network identity • Forwards requests to backend web servers • Features can include caching, compression, SSL
Open Source Load Balancers
Perlbal – used by LiveJournal, TypePad
HAProxy – used inside hw + sw load balancers
Varnish – HTTP accelerator, caching + balancing
Pound – Security-focused HTTP accelerator
mod_proxy_balance – Apache module
NGinx – lightweight, high performance web proxy
Load balancing strategies
• Round robin • Least connections/time • Predictive • Random • Weighted strategies
Least Connections / Response Time
Send requests to server with the least number of connections or lowest response time.
– Good for balancing between requests with different requirements or servers with different performance levels.
– Problems can arise with multiple load balancers making decisions in parallel.
– Weights can be added for manual tweaking.
Predictive
Round robin or least connections with additional heuristics to compensate for information staleness issue arising from many short rapid transactions.
Random
• Either select server at random or combine with resource-based algorithm to deal with information staleness problems.
• Weighted random adds manual constant to probability of choosing particular servers. – A dual-core isn’t twice as fast as a single CPU. – Requires trial and error experimentation.
The Problem
User interactions with a server have a certain amount of state.
– Authentication – Current stage of transaction – Shopping carts, etc.
How can the application preserve state when the load balancer sends next request to a different server?
Session Stickiness
Stickiness requires session-aware load balancer – Ensures that future requests from same session
always go to same server.
Problems – Lack of failover since requests tied to single server – Difficult to allocate resources effectively since
sessions vary tremendously in duration and size
Client-side State
Why not store all state in client cookies? – Insufficient client storage (ameliorated by HTML5) – Insecure (can’t trust client to control prices, id)
Solution – Store frequently accessed low security data in
client cookies. – Store rarely accessed data in central backend
storage and take performance hit to access when necessary.
Elastic Load Balancing
Distributes traffic across EC2 instances – Instances can be located across multiple AZs – Supports any TCP based protocol – Monitors instance health and will not distribute
traffic to unhealthy instances – Supports SSL termination – Supports session stickiness – Provides metrics to CloudWatch
ELB Algorithms
1. Weighted round robin – Sends new request to instance handling smallest
number of requests. – Round robin choice if multiple instances have
same smallest number of requests. – Can configure health checks to prevent ELB from
sending requests to unresponsive instances.
2. Session sticky – Weighted round robin, but all requests that are
part of a session go to the same server.
ELB Lifecycle
1. Create an ELB – List of AZs – Parameters for health check – List of listeners
2. Add instances to ELB – By instance ID – ELB will track status: InService, OutOfService
3. Advertise public DNS name of ELB 4. Modify number of instances to match traffic
– Or setup CloudWatch/AutoScale to handle for you 5. Delete ELB when unneeded
CloudWatch Monitoring service for EC2
– CPU utilization, Data transfer, Storage usage Pricing
– Basic Monitoring with 5 minute granularity free – Detailed Monitoring (1 minute) for 1.5₵ per hour – 10₵ per alarm after first 10 alarms
CloudWatch Terminology A namespace represents a source of data
AWS/EC2 AWS/ELB
A measure is a raw, observed data value One minute’s worth of observation
A unit is an attribute of a measure Seconds, %, bytes, bits, counts, bytes/s, bits/s
A dimension is a refined view of a type of data AvailabilityZone, ImageType, InstanceID, …
A metric is a stored, processed measure A statistic is a computed attribute of a metric
Minimum, maximum, average, sum
AutoScaling
AutoScaling Group – Set of EC2 instances that should scale together
Triggers – Scale on CloudWatch alerts – Scale on time-based schedule – Fixed number of healthy instances
Examples – Add 3 instances if CPU > 50% – Remove 3 instances if CPU < 10%
AutoScaling Setup
1. Create an ELB 2. Create AutoScaling launch configuration
– ID of AMI to be launched – Instance type – Key pair to authenticate to instances – List of EC2 security groups for instances
3. Create an AutoScaling Group 4. Create a trigger for the group
AutoScaling Operation
1. CloudWatch metrics specified in AutoScaling group’s trigger are retrieved at specified times.
2. Metrics checked against trigger thresholds – If metrics larger than UpperThreshold and the
number of instances less than MaxSize, a scale-out event is initiated, launching new instances.
– If metrics are smaller than LowerThreshold and number of instances is greater than MinSize, a scale-in event is initiated, terminated excess instances.
Key Points 1. Load balancing distributes transactions across multiple
servers, CPUs, links, or storage devices 2. Techniques
– Round robin DNS (configuration) – DNS-based GSLB (software + configuration) – Layer 4 switch (hardware) – Web switch (hardware) – Reverse proxy (software)
3. Algorithms – Round robin – Weighted: least connections/fastest response time – Predictive heuristics – Random
4. Session stickiness
References 1. Jeff Barr, Host Your Web Site in the Cloud: Amazon Web
Services Made Easy, Sitepoint, 2010. 2. Theo Schlossnagle, Scalable Internet Architectures, Sams
Publishing, 2007. 3. Willy Tarreau, “Making Applications Scalable with Load
Balancing,” http://1wt.eu/articles/2006_lb/, 2006. 4. Pete Tenereillo, Why DNS Based Global Server Load Balancing
(GSLB) Doesn’t Work, http://www.tenereillo.com/GSLBPageOfShame.htm, 2004.