View
1.450
Download
0
Category
Preview:
DESCRIPTION
Citation preview
Copyright © 2002 Wensong Zhang. Page 1
Free Software Symposium 2002
Linux Virtual Server: Linux Linux Virtual Server: Linux Server Clusters for Scalable Server Clusters for Scalable Network ServicesNetwork Services
Wensong ZhangWensong Zhang
China National Laboratory for Parallel & China National Laboratory for Parallel & Distributed ProcessingDistributed Processing
Free Software Symposium 2002Free Software Symposium 2002
October 22, 2002October 22, 2002
Copyright © 2002 Wensong Zhang. Page 2
Free Software Symposium 2002
AgendaAgenda
IntroductionIntroductionThe Linux Virtual Server frameworkThe Linux Virtual Server frameworkBuilding Scalable Network Services Building Scalable Network Services
using LVSusing LVSFuture Work of LVSFuture Work of LVSCharacteristics of LVSCharacteristics of LVS
Copyright © 2002 Wensong Zhang. Page 3
Free Software Symposium 2002
IntroductionIntroduction InternetInternetExplosive Growth of the InternetExplosive Growth of the Internet
–100% annual growth rate100% annual growth rate
Sites receiving unprecedented Sites receiving unprecedented workloadworkload–Yahoo! 625 million views per dayYahoo! 625 million views per day
–AOL Web cache system receiving 5 AOL Web cache system receiving 5 billion requests per daybillion requests per day
Copyright © 2002 Wensong Zhang. Page 4
Free Software Symposium 2002IntroductionIntroduction
The requirements of The requirements of Network ServicesNetwork ServicesIncremental scalabilityIncremental scalability24x7 availability24x7 availabilityManageabilityManageabilityCost-effectivenessCost-effectiveness
Copyright © 2002 Wensong Zhang. Page 5
Free Software Symposium 2002
IntroductionIntroduction Possible SolutionsPossible SolutionsSingle server upgradeSingle server upgrade
–complicated, high cost, single point of complicated, high cost, single point of failure, ...failure, ...
Cluster of servers is becoming a Cluster of servers is becoming a viable architecture for SNSviable architecture for SNS–Redundancy --> HARedundancy --> HA
–Divide-and-Conquer --> High-performanceDivide-and-Conquer --> High-performance
–High performance/cost ratioHigh performance/cost ratio
Copyright © 2002 Wensong Zhang. Page 6
Free Software Symposium 2002LVS FrameworkLVS Framework
The Linux Virtual The Linux Virtual Server ProjectServer ProjectThe goal of the Linux Virtual Server The goal of the Linux Virtual Server
projectproject– to provide a basic framework for building to provide a basic framework for building
highly scalable and highly available highly scalable and highly available network services using a large cluster of network services using a large cluster of commodity servers.commodity servers.
Copyright © 2002 Wensong Zhang. Page 7
Free Software Symposium 2002
LVS FrameworkLVS Framework FrameworkFramework
Copyright © 2002 Wensong Zhang. Page 8
Free Software Symposium 2002
IPVS is an advanced Layer-4 switchingIPVS is an advanced Layer-4 switching
LVS FrameworkLVS Framework IP Virtual ServerIP Virtual ServerImplemented in the Linux kernelImplemented in the Linux kernelThree IP load balancing techniquesThree IP load balancing techniques
–Virtual Server via NATVirtual Server via NAT
–Virtual Server via IP TunnelingVirtual Server via IP Tunneling
–Virtual Server via Direct RoutingVirtual Server via Direct Routing
Eight scheduling algorithmsEight scheduling algorithms
Copyright © 2002 Wensong Zhang. Page 9
Free Software Symposium 2002
LVS FrameworkLVS Framework VS/NATVS/NAT
Copyright © 2002 Wensong Zhang. Page 10
Free Software Symposium 2002
LVS FrameworkLVS Framework VS/TUNVS/TUN
Copyright © 2002 Wensong Zhang. Page 11
Free Software Symposium 2002
LVS FrameworkLVS Framework VS/DRVS/DR
Copyright © 2002 Wensong Zhang. Page 12
Free Software Symposium 2002
LVS FrameworkLVS Framework ComparisonComparison
VS/ NAT VS/ TUN VS/ DR
Server any Tunneling Non-arp device
server network private LAN/ WAN LAN
server number low (10~20) High (100) High (100)
server gateway load balancer own router Own router
Note: those numbers are estimated based on the assumption that load balancer and backend servers have the same hardware configuration.
Copyright © 2002 Wensong Zhang. Page 13
Free Software Symposium 2002
LVS FrameworkLVS Framework Scheduling AlgorithmsScheduling AlgorithmsRound-RobinRound-RobinWeighted Round-RobinWeighted Round-RobinLeast-ConnectionLeast-ConnectionWeighted Least-ConnectionWeighted Least-Connection
Copyright © 2002 Wensong Zhang. Page 14
Free Software Symposium 2002LVS FrameworkLVS Framework
Scheduling Algorithms Scheduling Algorithms (Continued)(Continued)Locality-Based Least-ConnectionLocality-Based Least-ConnectionLocality-Based Least-Connection with Locality-Based Least-Connection with
ReplicationReplicationSource HashingSource HashingDestination HashingDestination Hashing
Copyright © 2002 Wensong Zhang. Page 15
Free Software Symposium 2002
LVS FrameworkLVS Framework State SynchronizationState Synchronization
Conn Sync daemon (kernel thread)Conn Sync daemon (kernel thread)UDP MulticastUDP Multicast
Copyright © 2002 Wensong Zhang. Page 16
Free Software Symposium 2002
Layer-7 switching in user-spaceLayer-7 switching in user-space–high overhead of context switching and high overhead of context switching and
memory copyingmemory copying
– limited scalabilitylimited scalability
LVS FrameworkLVS Framework KTCPVSKTCPVS
Copyright © 2002 Wensong Zhang. Page 17
Free Software Symposium 2002
kernel threadkernel threadloadable scheduling moduleloadable scheduling module
LVS FrameworkLVS Framework KTCPVS (Cont’d)KTCPVS (Cont’d)
Copyright © 2002 Wensong Zhang. Page 18
Free Software Symposium 2002LVS FrameworkLVS Framework
Cluster Monitoring Cluster Monitoring softwaresoftwareRed Hat Cluster Server / PiranhaRed Hat Cluster Server / Piranha
–LVS + PiranhaLVS + Piranha
UltraMonkeyUltraMonkey–LVS + lvs-gui + heartbeat + ldirectordLVS + lvs-gui + heartbeat + ldirectord
KeepalivedKeepalivedNetparseNetparseetc.etc.
Copyright © 2002 Wensong Zhang. Page 19
Free Software Symposium 2002
Building SNS using LVSBuilding SNS using LVS General ArchitectureGeneral Architecture
3 3 tierstiers–Load balancerLoad balancer
–Server clusterServer cluster
–Shared storageShared storage
Copyright © 2002 Wensong Zhang. Page 20
Free Software Symposium 2002
Building SNS using LVSBuilding SNS using LVS Building BlocksBuilding BlocksLoad balancerLoad balancer
–Layer-4 switching, Layer-7 switchingLayer-4 switching, Layer-7 switching
ServerServer–HTTP, HTTPS, FTP, SMTP, POP3, IMAP4, HTTP, HTTPS, FTP, SMTP, POP3, IMAP4,
most TCP and UDP servicesmost TCP and UDP services
Shared storageShared storage–database, network file system, distributed database, network file system, distributed
file system, ...file system, ...
Copyright © 2002 Wensong Zhang. Page 21
Free Software Symposium 2002
Building SNS using LVSBuilding SNS using LVS High AvailabilityHigh AvailabilityServer failoverServer failover
–ping, service detection, etcping, service detection, etc
Load balancer failoverLoad balancer failover–state synchronizationstate synchronization
–heartbeatheartbeat
Copyright © 2002 Wensong Zhang. Page 22
Free Software Symposium 2002
Building SNS using LVSBuilding SNS using LVS Web ClusterWeb Cluster
Copyright © 2002 Wensong Zhang. Page 23
Free Software Symposium 2002
Building SNS using LVSBuilding SNS using LVS Cache ClusterCache Cluster
Copyright © 2002 Wensong Zhang. Page 24
Free Software Symposium 2002
Building SNS using LVSBuilding SNS using LVS Mail ClusterMail Cluster
Copyright © 2002 Wensong Zhang. Page 25
Free Software Symposium 2002
Building SNS using LVSBuilding SNS using LVS Some Sites using LVSSome Sites using LVSUK National JANET Web Cache UK National JANET Web Cache
(wwwcache.ja.net)(wwwcache.ja.net)linux.comlinux.comsourceforge.netsourceforge.netvalinux.comvalinux.comreal.comreal.comOne of largest PC manufacturersOne of largest PC manufacturersetc.etc.
Copyright © 2002 Wensong Zhang. Page 26
Free Software Symposium 2002
Future Works of LVSFuture Works of LVS Future WorksFuture WorksAdding more load balancing Adding more load balancing
algorithmsalgorithmsDeveloping more advanced/flexible Developing more advanced/flexible
cluster monitoring softwarecluster monitoring softwareMaking KTCPVS ready for productionMaking KTCPVS ready for productionExploring TCP handoffExploring TCP handoffetc.etc.
Copyright © 2002 Wensong Zhang. Page 27
Free Software Symposium 2002
Characteristics of LVSCharacteristics of LVS
CharacteristicsCharacteristicsLVS extends Linux kernel to support LVS extends Linux kernel to support
three IP load balancing techniquesthree IP load balancing techniquesEight scheduling algorithmsEight scheduling algorithmsHigh scalability (up to 100 nodes)High scalability (up to 100 nodes)High availabilityHigh availabilitySupporting most TCP and UDP Supporting most TCP and UDP
services, no modifications to either services, no modifications to either clients or serversclients or servers
Copyright © 2002 Wensong Zhang. Page 28
Free Software Symposium 2002Characteristics of LVSCharacteristics of LVS
Compared to Other Compared to Other Commercial ProductsCommercial ProductsMore IP load balancing techniquesMore IP load balancing techniquesMultiple scheduling algorithmsMultiple scheduling algorithmsState synchronizationState synchronizationA robust and stable code base, a large A robust and stable code base, a large
user and developer base.user and developer base.Reliability proven in big real world Reliability proven in big real world
applicationsapplicationsFree to everyoneFree to everyone
Copyright © 2002 Wensong Zhang. Page 29
Free Software Symposium 2002
SummarySummary
IntroductionIntroductionThe Linux Virtual Server frameworkThe Linux Virtual Server frameworkBuilding Scalable Network Services Building Scalable Network Services
using LVSusing LVSFuture Work of LVSFuture Work of LVSCharacteristics of LVSCharacteristics of LVS
Copyright © 2002 Wensong Zhang. Page 30
Free Software Symposium 2002
Call to ActionCall to Action
Building scalable network services is Building scalable network services is complicated and expensivecomplicated and expensive
LVS is here to help make your life LVS is here to help make your life easiereasier
LVS is proven stable, and is being LVS is proven stable, and is being deployed by more and more sites.deployed by more and more sites.
Copyright © 2002 Wensong Zhang. Page 31
Free Software Symposium 2002
CollateralCollateral
The Linux Virtual Server ProjectThe Linux Virtual Server Project
(http://www.LinuxVirtualServer.org)(http://www.LinuxVirtualServer.org)
wensong@linux-vs.orgwensong@linux-vs.org
Recommended