26 Sep 2003
Transparent Adaptive Resource Management for
Distributed Systems
Department of Electrical Engineering and Computer Science
Vanderbilt University, Nashville
Jaiganesh Balasubramanian
Ossama Othman
Dr. Douglas C. Schmidt
{jai, ossama, schmidt}@dre.vanderbilt.edu
Vanderbilt University
Balasubramanian et all Transparent Resource Management
Distributed Systems• Typical issues with distributed systems
– Heterogeneous environments– Concurrency– Large bursts of client requests– 24/7 availability– Stringent QoS requirements
• Examples of distributed systems– E-commerce– Online trading systems– Mission critical systems
• Defense– Ship systems
Vanderbilt University
Balasubramanian et all Transparent Resource Management
Motivation• Development and maintenance of QoS-
enabled distributed systems– Non-trivial– Requires expertise that application developers
often lack
• Solution: Middleware (e.g. CORBA)– Can shield distributed system developers from
the complexities involved with developing distributed applications
– Can facilitate manipulation of QoS requirements and management of resources
Vanderbilt University
Balasubramanian et all Transparent Resource Management
Distributed System Resource Management
• Middleware can alleviate resource management difficulties
• Doing so in a transparent and efficient manner is difficult
• Managing resources transparently is important for legacy DRE systems– Often cannot be easily modified to introduce support
for improved distributed resource management– May not be feasible to do so
• Middleware in use may need to be enhanced to support this functionality– Such an enhancement can be found in a
middleware-based load balancing service
Vanderbilt University
Balasubramanian et all Transparent Resource Management
Middleware-Based Load Balancing• Load balancing service can be used to manage
resources for middleware-based distributed systems– Can improve the efficiency and overall
scalability of a distributed system– Allows additional components to be added to
distributed systems with minimal impact to performance
– Improves availability due to inherent redundancy
– Can be composed with other services (e.g. fault tolerance, security, etc)
– Can take into account request content
Vanderbilt University
Balasubramanian et all Transparent Resource Management
Basic Scenario• Multiple clients making
request invocations– Potentially non-
deterministic• Members
– Multiple instances of the same object implementation
• Object groups– Collections of members
among which loads will be distributed equitably
– Logically a single object• Load balancer
– Transparently distributes requests to members within an object group
LoadBalancer
Clients
Req
uest
s
Rep
lies
MembersObject Groups
Vanderbilt University
Balasubramanian et all Transparent Resource Management
TAO Load Balancer (Cygnus)• Cygnus makes it easier to develop distributed
applications in heterogeneous environments providing application transparency, scalability, flexibility, adaptability and interoperability.
• Uses the following strategies: - RoundRobin - Random - LeastLoaded
Vanderbilt University
Balasubramanian et all Transparent Resource Management
Load Balancing Strategies• Client binding granularity
– Per-session• Client permanently
forwarded to a replica– Per-request
• Requests forward on client’s behalf
– On-demand• Client can be rebound
to another replica whenever necessary
• Balancing policy– Non-adaptive
• No load feedback used when binding clients
– Adaptive• Load feedback taken in
to account
: Client : Replica
: Load Balancer
1. send_re
quest() 2. send_request()
: Client : Replica
: Load Balancer
1. send_re
quest()
2. L
OCATION_FORW
ARD()
3. send_request()
Per-session / On-Demand
`
Per-request
: Load Balancer
load_advisory()report_load()
: Replica
Adaptive
Vanderbilt University
Balasubramanian et all Transparent Resource Management
Load Balancing Architectures• Load balancing
architecture comprised of a combination of client binding granularity and balancing policy
• Given the strategies just described, there are six possible architectures
• Three common architectures– Non-adaptive per-
session– Adaptive per-request– Adaptive on-demand
: Client : Server Replica
: Load Balancer
1. send_request()
2. LOCATIO
N_FORWARD()
3. send_request()
: Client : Server Replica
: Load Balancer
1. send_request()
2. send_request()
load_advisory()
report_load()
The front-endserver proxy.
: Client : Server Replica
: Load Balancer
1. send_request()
2. LOCATIO
N_FORWARD() load_advisory()
report_load()
3. send_request()
Non-adaptive Per-Session
Adaptive Per-request
Adaptive On-demand
Vanderbilt University
Balasubramanian et all Transparent Resource Management
Load Balancer Components• Load Monitor
– Provides load feedback• Load Analyzer
– Determines location and member load conditions
• Member Locator– Binds client to
appropriate object group member
• Load Alert– Facilitates load control
(load shedding)• Load Manager
– Mediates all interactions between load balancer components
Client
Location/Node
LoadManager
MemberLocator
LoadAnalyzer
next_member
push_loads
POArequests
member
requests
*
* LoadMonitor
LoadAlert alert
Vanderbilt University
Balasubramanian et all Transparent Resource Management
Load Monitor• Facilitates feedback
– Monitor and report loads
• Load monitoring is location-oriented– The loads at a given
location are monitored and reported
– Contrast with loads on a given object group member
Load Balancer Host
LoadManager
Location
LoadMonitor
Member - Group 1
Member - Group N
LoadAlert
Vanderbilt University
Balasubramanian et all Transparent Resource Management
Load Analyzer• Load Analyzer
– Decides which member will receive the next client request
– Determines load condition at each location
– Induces load shedding
– Extensible load balancing strategies
• Set via the PropertyManager interface
• Each object group may use a different strategy
LoadManager
CustomStrategy1 CustomStrategy2
LoadAnalyzer
«interface»Strategy
«interface»Strategy
RoundRobin
Random
LeastLoaded
«interface»CustomStrategy
RoundRobin
Random
LeastLoaded
Vanderbilt University
Balasubramanian et all Transparent Resource Management
Member Locator• Implements the Interceptor design pattern• Transparently forwards client requests to
member retrieved from the load analyzer– In CORBA, redirection induced via standard GIOP LOCATION_FORWARD message
– Conforming client side ORB will transparently re-issue request to member chosen by load balancer
Vanderbilt University
Balasubramanian et all Transparent Resource Management
Load Alert• Facilitates “control” aspect of
load balancing– Load shedding– Only used for adaptive
load balancing strategies– Forwards client requests
back to load balancer
Location/Node
LoadManager
LoadAnalyzer
Member
LoadAlert
Client
Vanderbilt University
Balasubramanian et all Transparent Resource Management
Load Manager• Mediates interactions between all load
balancer components• Manages object groups• Manages load monitor and load alert location
maps• Acts as a specialized event channel
– Load events are published by load monitors– Load events are consumed by load
analyzers
Vanderbilt University
Balasubramanian et all Transparent Resource Management
Load Balancing Instructions
Client LoadManager MemberLocator LoadAnalyzer LoadMonitor LoadAlert Member
1: send_request
2: send_request
3: next_member
4: LOCATION_FORWARD
5: send_request
6: push_loads
7: is_overloaded
8: alert
9: LOCATION_FORWARD
Vanderbilt University
Balasubramanian et all Transparent Resource Management
Future Work• Define stateful load balancing model• Decentralized load balancing
– Reduced network overhead– Improved reliability
• Integrate multicast support• Examine compatibility with real-time systems• Examine similarities and parallels to dynamic
scheduling
Vanderbilt University
Balasubramanian et all Transparent Resource Management
Concluding Remarks• Load balancing architecture and model is
– Flexible• Extensible load balancing strategies
– Allows both non-adaptive and adaptive (including self-adaptive) strategies to be used
• Freedom to implement in a variety of ways– Centralized– Decentralized / Federated / Cooperative– Hierarchical
– Generic• Supports multiple object groups• Not application-specific
– Familiar• Uses many of the same group management concepts
found in existing fault tolerance models
Vanderbilt University
Balasubramanian et all Transparent Resource Management
References• Ossama Othman, Jaiganesh Balasubramanian, Douglas
C. Schmidt “ The Design of an Adaptive Load Balancing and
Monitoring Service” http://www.cs.wustl.edu/~schmidt/IWSAS_2003.pdf• Ossama Othman, Jaiganesh Balasubramanian, Douglas
C. Schmidt “Performance Evaluation of an Adaptive Load Balancing
and Monitoring Service” http://www.cs.wustl.edu/~schmidt/ICDCS_2004.pdf