Introduction to Load Balancing - Faculty Web Serverfaculty.ccc.edu/mmoizuddin/CISCO LIVE 2008/APP/BRKAPP... · 2012-12-10 · © 2006, Cisco Systems, Inc. All rights reserved. 14503_04_2008_c2.scr

© 2006, Cisco Systems, Inc. All rights reserved.14503_04_2008_c2.scr

1

© 2008 Cisco Systems, Inc. All rights reserved. Cisco PublicBRKAPP-100114503_04_2008_c2 2

Introduction toLoad Balancing

BRKAPP-1001


2

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 3BRKAPP-100114503_04_2008_c2

Agenda

Introduction

Load Balancing and Health Monitoring

Flow Management

Server Offload

High Availability

Deployments

Geographic Load Balancing

What’s Next ?


WAN AccelerationData redundancy eliminationWindow scalingLZ compressionAdaptive congestion avoidance

Application AccelerationLatency mitigationApplication data cacheMeta data cacheLocal services

Application OptimizationDelta encodingFlashForward optimizationApplication securityServer offload

Application NetworkingMessage transformationProtocol transformationMessage-based securityApplication visibility

Application ScalabilityServer load-balancingSite selectionSSL termination and offloadVideo delivery

Network ClassificationQuality of serviceNetwork-based app recognitionQueuing, policing, shapingVisibility, monitoring, control

Cisco Application Delivery Networks

WAN


3


Other Cisco Live Breakout Sessions that You May Want to Attend

BRKAPP-2014 Deploying AXG

BRKAPP-2013 Best Practices for Application Optimization illustrated with SAP, Seibel and Exchange

BRKAPP-2011 Scaling Applications in a Clustered Environment

BRKAPP-2010 How to build and deploy a scalable video communication solution for your organization

BRKAPP-1009 Introduction to Web Application Security

BRKAPP-1008 What can Cisco IOS do for my application?

BRKAPP-3006 Troubleshooting WAASBRKAPP-2005 Deploying WAAS

BRKAPP-2018 Optimizing Oracle Deployments in Distributed Data Centers

BRKAPP-2017 Optimizing Application DeliveryBRKAPP-1016 Running Applications on the Branch Router

BRKAPP-1015 Web 2.0, AJAX, XML, Web Services for Network Engineers

BRKAPP-1004 Introduction WAAS

BRKAPP-3003 Troubleshooting ACEBRKAPP-2002 Server Load Balancing Design

ApplicationsISRGSS WAAS ACE AXGACNS

Relevancy


The Application Delivery Journey

EarlyTechnologies

QoS

Load Balancing

1995–2000

Message VisibilityVirtualization

Deep Packet Inspection

2006 and Beyond

L4-7 Switching

WANOptimization

Web Acceleration

2000–2006

Application Aware Networks

Multi-GigabitPerformance

Client/Server

Centralized

Few Connections

ApplicationTrends

CiscoSolution

Web Enabled

Decentralized

1000s of Connections

SOA/Web 2.0

Distributed

Exponential Increase in Connections

End-to-End Application Delivery Networks


4


How It All StartedDirect Communication Clients/Servers

BenefitSimple solution

IssueNo fault tolerance

Limited performance and scalability

Web Server

IP TCP http Data

X


Scaling to a Few ServersThe Software Approach

BenefitAddresses some of the fault tolerant and performance issues

IssueStill limited in scale/performance.

Leverages server resources for LB and HA

Proprietary clustering technologies

S/W Load BalancerClustering Technologies


5


Scale and High Availability for Larger Deployments

BenefitAddresses fault tolerant, performance and scalability issues

Future proof: architecture includes hardware co-processors tosupport resource-intensive features (i.e., SSL, compression)

…

The Hardware-Based Solution


The Main Functions of a Load Balancer

Represents multiple server farms with public IP addresses Virtual IP’s or VIP’s (which clients resolve via DNS)

Monitors the health of servers

Intelligently distributes incoming requests according to configurable rules

Clients Load Balancer/ContentSwitch

ServersWeb

Streaming

Database


6


TerminologyClients Content

Switch-

LoadBalancer

ServersServerfarm

Client-SideGateway

Keepalive (Probe)

172.16.2.100TCP port 80

Virtual IP Address (VIP)URL = /news

User-Agent = WindowsCEClient = 192.0.0.0/8

Class-Map

Load BalancingAlgorithm(Predictor)

Round Robin

XML Gateways

If match class-map Xthen use serverfarm Xelse use serverfarm y

Policy-Map


Devices Being Load Balanced

Server

Proxies

Accelerators (compression engines, SSL offloaders)

Caches (reverse and transparent)

Firewalls (Layer 3 and Layer 2)

VPN concentrators

Routers

Generic IP device requiring load distribution and/orredundancy


7


Traffic Being Load Balanced

Generic IP traffic (i.e. IPSec tunnels)

Generic UDP and TCP (i.e. proprietary protocols)

Network services (i.e. LDAP, DNS, Radius)

HTTP (i.e. Web Presentation Layer, Web Services, SOAP/XML)

Voice and Video (i.e. RTSP, SIP, H.323)

Remote terminals (i.e. Windows Terminal Services)

Multi-connection protocols (i.e. FTP, RTSP)

Multi-tier packaged applications (i.e. SAP, Oracle, Microsoft, BEA)

Vertical specific applications (i.e. medical, finance, education)

EthernetHeader

IPHeader

TCPHeader

TCPHeader

EthernetTrailer

Payload

Layer 3 Layer 4Layer 5-7

Layer 2

HTTPHeader


HTTPThe Most Common Load Balanced Protocol

RFC 2616,HTTP 1.1 IETF draft standard:“The hypertext transfer protocol (HTTP) is an application-level protocol for distributed, collaborative, hypermedia information systems”

Three important elements of an HTTP request:Method (GET, POST, …)

URI

Headers (include cookies)

Carried over TCPMultiple HTTP requests can be “tunneled” over the sameTCP connection


8


HTTP 1.0—Single Request

Client

SYNSYN_ACK

ACK

FINFIN_ACK

ACK

GET / HTTP 1.0ACKHTTP/1.0 200 OKContinuation

ACK

WebServer


HTTP 1.1—Two Requests, No Pipelining

SYNSYN_ACK

ACK

FINFIN_ACK

ACK

GET /a.gif HTTP 1.1ACKHTTP/1.1 200 OK

Continuation

ACKGET /b.jpg HTTP 1.1

ACKHTTP/1.1 200 OK

ACK

Client WebServer


9


HTTP 1.1—Building an Entire Page

TCP 3102 > 80

logo1.gif globe.gif footpage.jpg

TCP 3103 > 80

/cgi-bin/count

TCP 3104 > 80

bannertop.jpg menu.jpg

TCP 3101 > 80

index.html

The behaviourdepends

on the browser


FTP—File Transfer ProtocolA Multi-Connection Protocol

Active FTP

Client FTPServer

3016 211

23017 20

34

C:>ftp test.cisco.comFTP server testUser: abcPassword: xxx230 User abc


10


FTP—File Transfer ProtocolA Multi-Connection Protocol

3018 211

23019 2036

34

Client FTPServer

Passive FTPC:>ftp test.cisco.comFTP server testUser: abcPassword: xxx230 User abc


Load Balancing and Health Monitoring

How Connections Are Distributed to the Best Available Servers


11


ServerfarmClient

Load Balancing Algorithms

How to Distribute Requests Across Servers?Enhanced Predictors Improve Serverfarm Efficiency


Load Balancing Algorithms

(Weighted) Round RobinVery simple, servers receive equal (or proportional) amount of requests

(Weighted) Least ConnectionsDynamic, based on open connections, optimizes load across servers

Hash on IP (source/destination, with mask)No state required for persistence

Hash on URL or portion of URLUseful for transparent cache redirection

Based on LoadServer load retrieved via SNMP or feedback protocols

FastestBased on response time: fastest servers receive newer connections

Least BandwidthReal-time amount of traffic considered to select less active server


12


Session Persistence—Stickiness

Browse

Select

Buy

1

2

3

Empty?!?

The “Shopping Cart” Problem

I’ll Never Shop Here

Again!


Session Persistence—Stickiness

Session: logical aggregation of multiple simultaneous or subsequent connections Sessions are limited in time (timeout)Servers might keep session state locallyLoad distribution across multiple servers introduces the problem

The content switch needs to identify a session and send connections belonging to the same session (i.e. from the

same client) to the same server

Methods to identify the session or client:Source IP address, HTTP session cookie, SIP session ID,SSL ID, generic protocol session data, …


13


Health CheckingThe content switch needs to continuously monitor the back-end servers

Failed servers have to be identified and removed from rotation:the load balancing algorithms adapt to the change

Server failures should be transparent to clients

Servers recovering from failures should be checked and put back in the available pool, avoiding flapping

Any failure affecting client-server interaction should be detected: connectivity, application or back-end servers malfunctions

ServerfarmClients

X X


Active Probing—Keepalives

Intended to run periodicallyGenerated by the load balancer: a correct reply is expectedEither predefined health checks or user-configurable scriptsExamples: ICMP (L3 connectivity), TCP (stack), HTTP (application)For each probe:

Interval, retry timesMaximum TCP open timeMaximum receive time (max response time)Failed retry time, successful retries before back in service

Serverfarm


14


In-Band Health Monitoring

The load balancer monitors server-to-client “inband” traffic and keep counters for consecutive errors

Can catch basic errors:No replies from server

RST’s from server

For HTTP traffic, can perform return error code checking (i.e. 500-type errors should remove servers from rotation)

Clients Serverfarm


Flow Management

“Layer 4” and “Layer 7” Processing


15


Flows, Connections, Sessions

Three main types of flowsTCP: IP protocol, src/dst IP, src/dst L4 port, TCP state

UDP: IP protocol, src/dst IP, src/dst L4 port

Generic IP: source/destination IP

TCP flows (connections) require setup

Multiple flows between the same client and server might be logically grouped into a session

A Load Balancer MaintainsMuch More State than a Router

on a Per-Flow Basis


Layer 4 Switching

L2–L4 information is always present in the first packet of the flow (unless it is a fragment!)

IP protocol

Source/destination IP addresses

Source/destination L4 ports (for TCP/UDP)

Source VLAN, MAC address

The load balancing decision can be made on the first packet


16


Layer 4 Flow Setup—Basic Load BalancingDecisions Made on First Packet

SYN

SYN_ACK

ShortcutACK

ShortcutData GET/HTTP 1.1

ShortcutDataHTTP/1.1 200 OK

Shortcut

Matches Existing Flow

Rewrites L2/L3/L4

Matches VIPSelects Server

Rewrites L2/L3/L4


Layer 7 Switching

L5–L7 information is only received after the TCP setup and might span multiple packets

HTTP URLs, cookies, header fieldsSSL session IDFTP data channel portGeneric application data

Requires TCP termination and buffering!


17


Layer 7 Flow Setup for HTTP (1/3)Load Balancing Decisions Require More Data

SYN

SYN_ACK

Starts Buffering

ACK

Data GET/HTTP 1.1

ACK ACK’s Client PacketsKeeps Buffering

Matches VIP w/L7 rule

Chooses SEQ #Replies w/SYN_ACK



ACK

DataGET Continuation

SYN

SYN_ACK

ACK

Data—GETData—GET Continuation

Empties BufferSends Data to Server

Acts as ClientDoes Not Forward

SYN_ACK

Parses the DataSelects ServerInitiates TCP


18



ACK

DataHTTP/1.1 200 OK

ShortcutACK

ShortcutDataContinuation

Shortcut

Matches Existing FlowRewrites L2/L3/L4

and SEQ/ACK

Does Not Forward ACKReady to

Splice the Flows


Full ProxyIndependentclient&

serverconnections

Layer 7 Flow Setup—Full ProxyThe Most Flexible Approach

SYNSYN_ACK

ACKData GET/HTTP 1.1

ACK SYNSYN_ACK

ACKData—GET

ACK

ACKData

DataHTTP/1.1 200 OKHTTP/1.1 200 OK

Client connection Server connection

… …


19


Content Switching MetricsConnections per Second (CPS)

L4 vs. L7

HTTP requests per Second (“CPS”)HTTP 1.1 vs. 1.0

Concurrent Connections (CC)Bandwidth (in Gbps) and Packets per SecondLatencyKeepalives per secondNumber of virtual servers/real serversNumber of policies/rules


Server Offload

Freeing Up Server CPU and Resources


20


Server Offload Overview

What is it ?Perform resource intensive functions on application traffic in the content switch on behalf of the server. Often hardware accelerated.

Why ?Servers can dedicate more resources to processing and serving client requests: faster application response!

What can be offloaded ?SSL processing, TCP setup/close, HTTP compression, XML processing,…

ServersApplicationSwitch


Offloading SSLOffload CPU-intensive SSL processing

Servers resources are dedicated to serving requests and running applications, rather than encrypting data

Centralized key/certificate storage/managementAllows advanced content switching (URL-based, cookie-sticky, payload parsing) and inspection of SSL trafficScalability: easy to add more SSL “performance”

WebServers

ContentSwitch

Encrypted toVIP:443

Clear Text toServers:80


21


SSL—HandshakeFull

Client Hello

Server HelloCertificate *

Server Key Exchange *Certificate Request *

Server Hello Done

* CertificateClient Key Exchange* Certificate VerifyChange Cipher SpecFinished

Change Cipher SpecFinished

Application Data

AbbreviatedRe-use same SSL session ID

Less latency - Faster applications

Client Hello

Server HelloChange Cipher Spec

Finished

Change Cipher SpecFinished

Application Data


Building an Encrypted Web Page

TCP 3102 > 443TCP 3102 > 443

logo1.gif globe.gif footpage.jpg

TCP 3103 > 443TCP 3103 > 443

bannertop.jpg menu.jpg

TCP 3101 > 443TCP 3101 > 443

index.htmlSSL ID123

SSL ID123

SSL ID123


22


SSL Offload Metrics

New transactions per second (TPS)Full SSL setup (asymmetric)

Depends on key size

Different from chipset RSA operations

Raw throughput (in Mbps or Gbps)Symmetric

Concurrent connections (CC)

Number of SSL ID cached entries (for SSL ID re-use)

Number of services

Number of certificates


TCP1

TCP1 Pool1

TCP2

TCP3

TCP2 Pool2

Offloading TCPTCP Reuse (Multiplex)

Offload TCP (HTTP) setup processing from serversServers resources are dedicated to serving requests and runningapplications, rather than opening and closing TCP connections

TCP connections to the server are kept open(HTTP 1.1 Connection Keepalive)Client requests multiplexed to existing server connections


23


High Availability

Protecting Against Single Points of Failure


Redundancy

Internet VIP Active192.1.1.100

IP Interface10.1.1.254

ACTIVE

BACKUP

Heartbeat and State Synchronization link


24


Terminology

Each VIP Can Independently Be Active or

Standby

An Entire Load Balancer Is Either Active or Standby

All VIPs Arein the Same State

Granularity

Per-VIP RedundancyBox-to-Box Redundancy

Only One Entity Can Process Traffic at Any Given

Time

(The Other Is Standby/Monitoring)

Multiple Entities Can Process Traffic at the Same

TimeState

Active-StandbyActive-Active


Redundancy—Statefulness

Medium

Session Stateful

Sticky Tables

Sticky Stateful

HighLowLB Resources

Long Living FlowsStateless ContentIdeal For

Full Flow TablesSync/MonitorLB Communication

Full StatefulStateless

Adaptive RedundancyStateful Level Configurable

Independently on Each Policy


25


Deployments

Network Integration Options and Examples


Router Mode

Servers in private IP subnet

VIPs usually in different, routable subnet from servers

Requires two IP subnets

Easy to deploy with many server IP subnets

Servers Default Gateway:Content Switch IP

Content Switch “Routing”

Subnet A Subnet B


26


Bridge Mode

Servers in routable IP subnet

VIP’s can be in the same or different subnet

Requires one IP subnets for each farm

Easy deploy for firewall or cache load balancing

Servers Default Gateway:Upstream Router

Content Switch “Bridging”

Subnet A


L3 One-Arm Mode

L2-rewrite not possibleContent switch not inline

Does not see unnecessary traffic

Requires PBR, server default gateway pointing to load balancer or client source NAT

The return traffic is needed!

Not as common as bridge or routed mode due to problems with forcing traffic back to CSM in return direction

Servers Default Gateway:Upstream Router

Subnet B

Subn

et B

PBR—Policy Based Routing, NAT—Network Address Translation


27


L3 One-Arm Mode—Flows

L2 to the Server Default GatewayRouting Would Break; Need to Use Either PBR, SNAT, or Server Default GatewayJust Routing to the Client IP

VIP ServerIP

1 23

3’4

Just Routing Traffic to the VIPJust Routing Traffic to the Server IP2

3

3’

4

1

PBR—Policy Based Routing, sNAT—Source Network Address Translation


L2 One-Arm ModeReturn Traffic Bypassing Load Balancer

Bypass for return traffic: high throughput!Requires MAC rewrite, L2 adjacencyServers need identical loopback addresses (one per VIP)TCP termination not possible: no L7 features!Load balancer blind to return traffic (inband, accounting)

ServersDefault Gateway:Upstream Router

Same IP Subnet


28


DBHosts

APPHosts

IDMHosts

OIDHosts

Application Servers(portal, Java,

caching)

Identity Management(login functions)

Internet Directory(LDAP)

A Multi-Tier Example of DeploymentApplication Server Suite 10g

3 serverfarm in 3 distinctIP subnets configured in

bridge mode

Separate Data-Basefarm not requiring

load balancing


Firewall Load BalancingFWLB + SLB

InsideNetwork

ServerfarmFirewallfarm

InternalLoad Balancer

Internal Load Balancerdistributes traffic to servers

and stores source MAC addressfor return traffic to firewalls

ExternalLoad Balancer

1

2 3

45

678


29


GeographicLoad Balancing

Disaster Recovery and Load Distribution Across Data Centers


InternetServiceProvider A

Front-End Tier(Web)

ServiceProvider B Internal

Network

ApplicationTier

DatabaseTier

InternalNetwork

Distributed Data Center Topology


30


Site Selection Mechanisms

Site selection mechanisms depend on the technology or mix of technologies adopted for request routing:

1. HTTP Redirect

2. DNS Based

3. Route Health Injection and L3 Routing

Health of servers and applications need to be taken into account

Optionally, also other metrics (like load and distance) can be measured and utilized for a better selection


DNS-Based Site Selection

Client

DNS Proxy

Data Center 1

http://www.cisco.com/

Root DNS for/ Root DNS for .com

Authoritative DNScisco.com

AuthoritativeDNS

www.cisco.com

Keepalives

1

23 4

56

78

9

10

Keep

alive

s

Data Center 2


31


DNS-Based Site Selection

Client

DNS Proxy

http://www.cisco.com/

Root DNS for/ Root DNS for .com

Authoritative DNScisco.com

AuthoritativeDNS

www.cisco.com

Keepalives

1

23 4

56

78

9

10

TCP:80

Keep

alive

s

Data Center 1 Data Center 2

© 2008 Cisco Systems, Inc. All rights reserved. Cisco PublicBRKAPP-100114503_04_2008_c2 62

What’s Next ?

Load Balancing, Content Switching, Application Delivery …and Cisco Products


32


Advanced Requirements: From Load Balancing to Application Delivery

Server OffloadFree up server CPU and resources

Application AccelerationBetter user experience, faster transactions

Bandwidth ReductionEfficient WAN resources utilization

Application and Protocol InspectionProtection against sophisticated application-specific attacks

VirtualizationOne physical device behaves as many: maximum deployment flexibility and separation of resources

Flexible Network ManagementAllows multiple users, with different responsibilities, tosimultaneously manage the device


XML Switching and PCI

Global Products and Tools

Application Switching

Cisco Application Control Engine Family

ACE Module8 Gbps

ACE Module16 Gbps

ACE Module4 Gbps

Module(4-16 Gbps) +

Multi-Module(64 Gbps)

ACE 47101 Gbps

CSS 11501Up to 1 Gbps

ACE XML Gateway30,000 TPS

ANM

ACE 47102 Gbps

ACE GSS20K DNS RPS

ACE WebApplication

Firewall

ACE XML Gateway Manager

“One-Click”Migration

Tools

Appliance(1-2 Gbps)


33


Q and A


Recommended Reading

Continue your Cisco Live learning experience with further reading from Cisco Press

Check the Recommended Reading flyer for suggested books

Available Onsite at the Cisco Company Store


34


Complete Your Online Session Evaluation

Give us your feedback and you could win fabulous prizes. Winners announced daily.

Receive 20 Passport points for each session evaluation you complete.

Complete your session evaluation online now (open a browser through our wireless network to access our portal) or visit one of the Internet stations throughout the Convention Center.

Don’t forget to activate your Cisco Live virtual account for access to all session material on-demand and return for our live virtual event in October 2008.

Go to the Collaboration Zone in World of Solutions or visit www.cisco-live.com.


Documents

Introduction to Load Balancing - Faculty Web Serverfaculty.ccc.edu/mmoizuddin/CISCO LIVE 2008/APP/BRKAPP... · 2012-12-10 · © 2006, Cisco Systems, Inc. All rights reserved. 14503_04_2008_c2.scr