Center Selection and Migration for Wide-area MulticastRoutingMichael J. Donahoomjd@cc.gatech.edu Kenneth L. Calvertcalvert@cc.gatech.eduCollege of ComputingGeorgia Institute of TechnologyAtlanta, GA 30332{0280Ellen W. Zeguraewz@cc.gatech.edu

AbstractInterdomain multicast routing is a challenge, requiring a scalable solution that is suitable for diverse applicationcharacteristics. Center-based routing trees overcome some of the limitations associated with traditional shortest-path tree routing, however they present a new set of routing problems. Speci�cally, the choice of the center routercan be critical in determining the multicast routing performance. We present and assess algorithms to solve twoproblems: selection of an initial center router, and migration of the center router in response to dynamic changes inthe group membership during the application lifetime. These two problems, and our proposed solutions, may havemore general applicability in wide-area networking, e.g., in placing shared resources in the network or estimatingbandwidth utilization for multicast distribution.Keywords: Multicast, Routing, Internetworking 1

21 IntroductionThere is little question that emerging networking applications will require multicast capability. From video confer-encing to replicated database access, the need exists to transmit from one or more sources to multiple destinations.Supporting multicast in local area networks is relatively uncomplicated, particularly with broadcast technologies suchas Ethernet. Supporting interdomain multicasting is a more signi�cant challenge, requiring a solution which dealswith issues such as scaling along many dimensions (e.g., network size, number of groups), incomplete or inaccurateinformation, and diverse application characteristics.A key aspect of multicast service deployment is the problem of interdomain multicast routing: given a set ofsources and destinations that are distributed in a large internetwork, construct and maintain routing paths fromthe sources to the destinations. Other problems arise in interdomain multicast, including end-to-end reliability and ow control; solutions to these problems are largely orthogonal to the problem of routing. Two main approacheshave emerged for implementation of interdomain multicast routing. In the �rst, a shortest-path tree is constructedfrom each source to the set of destinations. In the second, a shared tree is constructed from a designated core orcenter router to the set of destinations. In addition, a path is maintained from each source to the shared tree. Bothapproaches assume the existence of an underlying unicast routing mechanism.Shortest-path trees o�er several advantages in multicast routing. They minimize the \cost" to each receiver ina multicast group, where cost is the metric used to construct the shortest-path trees, typically hop count. Further,shortest-path trees can be constructed1 relatively easily in a distributed fashion, given information used by theunderlying unicast mechanism [6, 12]. Why, then, are center-based methods being considered in recent multicastrouting schemes such as Core-Based Trees [3] and Protocol Independent Multicast [7, 8]? Among the reasons arereduction in router storage and control message overhead; both of these issues are addressed by the use of center-basedtrees, which require amounts of router storage proportional to the number of receiver groups rather than the numberof source/receiver group pairs. Mechanisms for fault tolerance of centers are speci�ed in the speci�c center-basedmulticast routing protocol.A new set of routing problems are encountered when using shared-tree multicast, due to the presence of the center1For simplicity, we assume paths are symmetric, and we adopt the underlying routing protocol's metric for shortest-path. One coulddevelop a separate multicast routing mechanism.

3router. Speci�cally, the choice of the designated center router plays a role in determining the shape (and thereforethe performance) of the routing tree. As we will see, the performance gain can be signi�cant with a well-chosencenter.Selecting a center becomes more di�cult if the make-up of the group changes during the lifetime of the connection.As an example application, consider the multicast requirements of Distributed Interactive Simulation (DIS) [11] ordistributed game-playing: each entity must multicast information about its location and activity to the other entitieswithin its current region-of-in uence. The region-of-in uence can change radically during the simulation, as entitiesmove about within the virtual environment. Moreover, these changes are not predictable a priori since movementoccurs in response to evolving situations.In this paper, we present and assess algorithms to solve two problems:� Selection of an initial center router.We consider the relationship between the routing performance and the choice of the center router. We developalgorithms to e�ciently identify center routers that o�er good performance, using information of several types:independent of network and multicast group, based on network topology information, and based on multicastparticipant locations.� Migration of the center router.We develop algorithms to identify when the participants in an application have changed su�ciently to warrantmigration of the center router. We also propose algorithms to identify a suitable new center. Note that thenegative impact of center migration such as packet loss and duplication is beyond the scope of this paper.In the next section we describe the wide-area multicast routing problem in more detail, and give an overview ofshortest-path and shared-tree routing. Assessing solutions to the initial selection and migration problems requiresan experimental framework that accounts for input variables such as network topology and multicast group con�gu-rations, as well as evaluation of performance; our experimental methodology is described in Section 3. We then takeup the problem of Center Selection, considering group performance for di�erent methods in Section 4, and tra�cconcentration e�ects in Section 5. In Section 6, we propose and evaluate solutions to the Center Migration problem.Finally, we summarize contributions of the work and discuss some remaining open questions.

4These two problems, and our proposed solutions, may have more general applicability in wide-area networking.Speci�cally, the problem of selecting an initial center router is fundamentally similar to the problem of placementof any shared resource in a wide-area network. Identifying optimal locations of other shared entities |e.g., shareddatabases or network caches| may be possible using adaptations of our algorithms. Our solution to the problem ofmigrating the center router includes a method for estimating bandwidth utilization for multicast distribution, usingunicast routing information. This method may be useful, for example, in deciding whether to deliver often-requestedinformation from a server via unicast or multicast. We also re�ne a method for e�ciently soliciting responses froma selected number of receivers in the multicast group. This is useful whenever response implosion is a concern.2 Review of Multicast RoutingIn what follows we represent the internetwork |i.e. the collection of routers and networks or links connecting them|as a weighted undirected graph G = (V;E). The vertices in V represent routing nodes, and the edges in E representlogical connectivity between them. Thus a set of four routers that communicate via a single LAN is represented inthe graph by a clique of four vertices. Edge weights represent properties of the logical links, for example propagationdelay or usage cost; these weights are used by the underlying unicast routing algorithm to determine the sequenceof nodes traversed by a packet traveling from one node to another. The \shortest" path between two nodes is theone with the smallest total edge weight. Edge weights can be adjusted in the model to control the characteristics ofthe paths found by shortest-path routing algorithms, as described below.We de�ne an instance of multicast communication to be a set of sources (S � V ) and a set of receivers (R � V ).We refer to S [R as the participants of the multicast group. For each source s 2 S, the multicast routing algorithmdetermines a distribution tree, T (s;R). This tree is a subgraph of G that speci�es the path from the source to thereceivers taken by the multicast packets generated by s. Note that our model does not explicitly represent hosts;in the methods we consider, hosts participate in multicast only via a designated local router, which is responsiblefor maintaining multicast group information for all hosts connected to a particular network (typically a broadcastLAN). Thus \source" or \receiver" actually refers to the designated local router for the host that is the actual sourceor receiver.

5In the shortest-path approach to multicast routing, the distribution tree from source s to receivers R is a shortest-path tree rooted at s, i.e. packets from s follow the shortest path through the graph to each vertex in R. In general,distribution trees for di�erent sources will be disjoint. For the center-based approach, on the other hand, thedistribution trees are determined by the participants and the choice of the center router. We consider both PIM andCBT routing schemes in our evaluation. In PIM, a packet from a source s follows the shortest path from s to thecenter router, and then follows the shortest path from the center to each receiver. In CBT, the routing is the sameas PIM except if a node in the center to receiver shortest path tree is encoutered on the shorest path from a sourceto the center, a copy of the packet is sent to the downstream receivers from that point as well as to the center. Thedistribution tree for s is thus the union of the shortest path from s to the center with the shortest-path tree fromthe center to all the receivers. In general, the trees for each source have considerable overlap, since they di�er onlyin the path from source to center router.Note that in each method, wherever the distribution tree forks, the router at that vertex is responsible forduplicating the packet and sending it over each outgoing link; because multiple-access (broadcast) networks arerepresented by cliques, this may or may not involve multiple transmissions of the packet. In any case, only one copyof each packet originated by a source traverses each link in the distribution tree.Figure 1 shows an example of a PIM center-based multicast routing instance, with two sources (marked \S1"and \S2") and six receivers (marked \R"); the center router is marked \C". The shortest-path tree from the centerto all receivers is indicated by the darker edges. The links traversed by packets sent by S1 are marked by dottedarrows; the path followed by packets sent from S2 are marked by solid arrows. The CBT routing looks the same withthe exception that all double arrows (e.g. from S2 to C) are eliminated). Note that in general, with center-basedmulticast routing, a packet does not follow the shortest path from its source to each receiver; this is evident inFigure 1 for the path followed by packets from S1 to the lower-left-most receiver.In considering the performance of center selection and migration algorithms, we will make our comparisons to theperformance that would be achieved with shortest-path routing. This allows for some normalization of the results,as well as a comparison to a standard method of doing wide-area multicast routing.

63 Experimental MethodologyLater in this paper we propose and evaluate solutions for the Center Selection and Migration problems. Our evaluationis based on simulations involving assorted network topologies and group \scenarios" that re ect various distributionsof multicast participants, both static and dynamic. Clearly our results depend on our choices of network topologyand group scenarios. In this section we describe these models as well as the metrics we use to evaluate performance.3.1 Graph ModelsWe use two di�erent graph models to represent internetworks in our simulations. Each is intended to capture some ofthe (unicast) routing characteristics of real internetworks, particularly locality. In discussing both models, a domain isa group of interconnected nodes representing the set of routers under a single administration, e.g. a campus networkor a backbone network. Domains are represented as graphs constructed by �rst placing nodes at random locationsin the plane, and then placing edges between nodes randomly with a probability that is a decreasing function of theEuclidean distance between the nodes.In each method, graphs representing individual domains are connected together to form the internetwork graph;the methods di�er in how this is accomplished. In the two-level model, a single domain graph is �rst constructed,and then each of its nodes is replaced with an entire domain graph. This replacement is done in such a way that\leaf" nodes (i.e. nodes that have only one link to another router) are preserved. If the �rst graph has N nodes, andeach domain graph has M nodes, the entire graph has N �M nodes.In the transit-stub model, each domain is designated as either a transit or a stub domain. With respect to therouting of packets, a stub domain is one whose edges carry only tra�c whose origin or destination is within thatdomain, while a transit domain is one that carries tra�c between stub domains. Transit domains model \backbone"networks, which interconnect stub domains, which model campus or regional networks. In general, each transitdomain node has edges to a number of stub domains, to other transit nodes in its own domain, and possibly to nodesin other transit domains; nodes in stub domains, however, connect only to other nodes in the same domain, or to anode in a transit domain (see Figure 2). (The model allows for stub-stub edges, and for \multi-homed" stub domainsthat connect to multiple transit domains; in our experiments, however, none of these additional edges were present.)

7In the transit-stub graphs, edge weights are assigned in such a way that routing policies similar to those in realnetworks hold when shortest paths are computed using the weights (See [17] for details on weights). In particular:� The shortest path between any two nodes in the same domain stays entirely within that domain.� The shortest path from node u in stub domain U to node v in stub domain V passes through U , through oneor more transit domains, and through V , without touching any other stub domains.These methods can easily generate very large graphs; in our study we use graphs with 441 nodes, an averagetransit node degree of 5.8, and an average stub node degree of 4.6. For more details and motivation of the graphmodels used here, see [17]. The two-level graph model is used in evaluating center placement(Section 4) and tra�cconcentration (Section 5). The transit-stub graph model is used in evaluating center migration (Section 6).3.2 Multicast Group ScenariosWe model two aspects of multicast groups: the initial (or static) location of sources and receivers, and the dynamicbehavior of the group over time.3.2.1 Static Multicast InstancesA multicast routing instance represents the distribution requirements of a particular set of users for a particularapplication. An example of such an application would be a video conference among a small set of participants, ordistribution of a lecture series to a large, widely distributed group of students. As discussed earlier, an instance isde�ned by specifying which nodes are designated as receivers, and which are sources; these two sets may or maynot intersect. An instance may be characterized by the number and overlap of sources and receivers, and by theirdistribution throughout the internetwork. To evaluate the e�ect of center choice methods, we consider instances ofthree di�erent types:All Receivers Sources Each receiver is also a source; receivers are distributed randomly throughout the entirenetwork. This scenario models, for example, a video conference application.Single Source, Distributed Receivers Receivers and a single source are distributed randomly in the network.This scenario can model a video broadcast of a lecture or meeting.

8Localized Receivers In this scenario the receivers are constrained to be located within the same domain. Thesources are distributed randomly throughout the whole network. An example of an application with thesecharacteristics might be a distributed database: database clients (sources) submit transaction requests toservers (receivers) via multicast, with each response being unicast from a server back to its originator.3.2.2 Dynamic Multicast Group ScenariosPrevious work in multicast routing relies on very simple speci�cations of group activity[9, 14, 15], i.e. new participantsjoining and existing ones leaving. We extend the concept of group activity speci�cation to allow evaluation of ouralgorithm in the contexts of various \real-world" scenario simulations. We use the term scenario to describe atime-dependent characterization of an application's behavior. Instead of simulating with purely random behavior,scenarios provide a framework for classi�cation of applications. This classi�cation might be used, e.g., to allowthe center migration algorithm to adapt on an application-speci�c basis, instead of trying to develop one generalmigration algorithm that works well for all applications. In this paper, however, we focus on a general algorithmthat adapts on the basis of on-line observations.We characterize a scenario by the following parameters:Migration Probability The probability that a new participant is generated in a domain that is not representedby the current set of participants.New Participant The frequency with which new participants appear. in a time interval.Life The percentage of the total simulation time that a participant exists.Participant Source The probability that a new participant is a source.All of these parameters can be time-dependent distribution functions. For example, the New Participant probabilityis a function mapping the percentage of time that has elapsed in the simulation to the probability of a new participantgeneration event. This can be used, for example, to model applications that have many participants join early in thelifetime of the group but few or none that join after a certain percentage of time has elapsed.Of course, the space of possible multicast application behaviors is vast, and it is not yet clear what are the \right"scenarios. As more multicast applications are developed, more data will become available about the characteristics;

9at present it seems reasonable to consider a wide range of behavior. Other researchers have characterized multicastgroups using parameters including the duration (or lifetime) of a connection, the holding time between modi�cationevents, and the probability an event is an addition [9, 15]. We have not found any other use of a migration function,relating the location of new participants to those of existing ones. Characterizing migration via a time-varyingdistribution provides a general framework for specifying diverse application behavior, though it may be limited inits ability to specify precisely some real application of interest. Conversely, specifying the dynamic behavior of awell-known application allows the study of one (or a few) real applications, but may not provide results that aregenerally applicable. Additional work is needed to collect measurements from actual multicast applications, and touse these measurements to select and specify dynamic multicast application parameters.We select four particular scenarios for study: 2Scenario VC (Video-Conferencing) This scenario has early creation of new participants distributed widelythroughout the network, with most participants remaining for the duration of the connection. The connection,therefore, reaches a steady state that lasts for the majority of the connection time.Scenario IS (Distributed Interactive Simulation) In this scenario, the group has creation of new participantsthroughout the life of the connection, with more receivers than sources. The participants migrate moderatelythroughout the network, and last on average 20% of the application lifetime. (Note that a mobile componentcan be approximated as the departure of one participant and the creation of another. Note that this does notcapture any spatial correlation of mobility, but it is not clear that correlations in the physical world correspondto correlations in the network topology)Scenario RC (Relay Chat) The group in this scenario is highly dynamic, with participants being created fre-quently throughout the network, generally lasting a very short period of time.Scenario SM (Seminar) This scenario mimics the membership characteristics of the UCB Multimedia Seminarsession[1].Table 1 gives the probability distributions that characterize each instance. Figure 3 shows how the number and2Scenarios are named for representative applications believed to exhibit the given characteristics; this is clearly debatable. Theimportant feature of our scenarios is that they represent a diverse set of characteristics.

10distribution of participants varies over time in each instance; the four curves in each plot give the number of sourcesand receivers, and the number of represented transit and stub domains. Each scenario begins with one source andone receiver. A large number of represented domains indicates a widely spread multicast instance, while a smallnumber of represented domains corresponds to a tightly localized instance. Note that there is considerable variationacross the four scenarios, allowing us to better determine what types of instances have dynamic behavior that canbene�t from center migration.3.3 Performance MetricsFor a given multicast instance, a particular multicast distribution tree determines three quantities of interest incharacterizing routing algorithm performance:� The average and maximum number of links traversed by any packet in traveling from a source to a receiver.The maximum gives an upper bound on the non-queuing component of delay experienced by multicast packets.For a given routing subgraph, we measure the length (in hops) of the longest path between a source and areceiver, referring to this as the maximum delay. We also measure the average length (in hops) of the pathbetween a source and a receiver, referring to this as the average delay.In Figure 1, for example, the maximum delay is 5 hops and the average delay is 3.83 hops.� The total number of (single hop) packet transmissions required to deliver a packet from each source node to allreceiver nodes. This gives an indication of the resources used by the algorithm, in particular overall bandwidth.For a given routing subgraph, we compute the sum, over all sources, of the total number of edges traversed bya packet sent from that source to all receivers, and call that sum the bandwidth measure of the algorithm.In Figure 1 the bandwidth is 13 packet hops for packets originating at S1 and 12 packet hops for packetsoriginating at S2, for a total bandwidth of 25 packet hops.3.4 Knowledge ModelIn any distributed algorithm, it is important to specify the assumptions about the knowledge available at theprocessing entities without the need for distributed interaction or complex calculation. Assumptions that are too

11strong may lead to solutions that are impractical for implementation, while assumptions that are too weak may leadto overly complex solutions. We are interested in routing algorithms operating in the Internet, thus we base ourassumptions on the knowledge available at Internet routers. Speci�cally, we assume the following:� A router knows the next hop to be used in (unicast) routing to any given destination.� For center-based multicast, a center knows the sources of the multicast instance, up to some time resolution.The center may impose some time window, and consider the \current" set of sources to be all sources whichhave been active within the most recent time window. Sources can be detected by observing the tra�c which ows through the center.� A router that is part of a multicast distribution tree knows the outgoing (downstream) interfaces on which tosend a received multicast message.4 Center SelectionWe now turn to the problem of selecting an initial center that provides good performance. The next several sectionspresent our analysis for four di�erent classes of center selection methods, spanning a range of complexity. They are:Worst-Case We investigate the limits of the negative e�ects of center choice on performance. In other words, thissection answers the question, \How bad can a center be?" We �nd that an worst-case choice gives very poordelay performance across all scenarios, and poor bandwidth performance unless the group has a large numberof well-distributed participants.Random With this method, a center is selected randomly from among all the nodes of the graph. We show thaton average this method can improve the delay and bandwidth, however the variance is considerable.Topology-Based These methods make use of coarse information about the topology of the network. In particu-lar, we analyze the relationship between performance and nearness of the center to a topological center (seesection 4.3) of the network.

12Group-Based These methods make use of information about both the network topology and the location of thenodes (sources and receivers) that make up the group. The availability of such information can potentiallyenhance the performance.To evaluate these methods, we use the static multicast instances and the two-level graph model. These graphs allhave 20 domains of 20 nodes each, thus 400 total nodes. All results in this section are the average over 10 di�erenttwo-level graphs, and we consider the CBT routing protocol.4.1 Worst-Case Center ChoiceWe begin by examining the performance of a worst-case center. This is intended to evaluate the heuristic of simplypicking any center. This is the least complex of the methods we consider, requiring no knowledge of the group, thenetwork topology, or previously chosen centers. This method will be appropriate only when center choice does notsigni�cantly a�ect the performance of the center-based tree.The left plot of Figure 4 shows the bandwidth (delay) performance of the center with the highest bandwidth(delay). Recall that the performance is actually an average of the performance for ten graphs. The performance isshown as a ratio to the performance in a shortest-path routing tree (The shortest-path tree is a shortest path fromeach source to each receiver). We compare the performance to a shortest-path tree because it is a practical, currentlyused routing approach. Using a Steiner tree for comparison gives less practical results. Note that bandwidth variesthe most across scenario because the probability of a common multicast path, which reduces bandwidth, varieswidely with scenario. Each scenario covers a range of counts of multicast participants. For All Receivers Sources,the number of participants ranges from �ve to 15 by ones. For Single Source, Distributed Receivers, the number ofreceivers ranges from ten to 100 by tens. For Localized Receivers, the number of receivers ranges from �ve to 50 by�ves. The x-axis is the percentage of participants from the range speci�ed for the scenario (e.g. for All ReceiversSources the number of participants ranges from 5 to 15 so 50% on the x-axis represents 10 group members for thatscenario).Observe that for delay, all three scenarios performed approximately 2.4 times worse than the shortest-path routing,demonstrating that a worst-case center choice can result in signi�cantly higher maximum delays. For bandwidth,

13the performance is sensitive to scenario, but for the All Receivers Sources and Localized Receivers scenarios, theperformance is signi�cantly worse than shortest-path routing. The Single Source scenario performs better than theother scenarios for bandwidth because it has a large number of well distributed receivers; the placement of the centerdoes not a�ect performance as much as it does for the other scenarios.Note that the worst-case center choice, by de�nition, tells an administrator the extent of the consequences ofpicking a bad center. Basically, this measure indicates the worst that can happen by just simply picking a centerwithout giving any consideration to application or center history. If worst-case center selection produces acceptableresults, it indicates that an administrator need not put any e�ort into center selection, making an arbitrary centerchoice very attractive.4.2 Random Center ChoiceThe random center choice method must select a center at random, and therefore requires some mechanism to selectnodes at random. This method allows hedging against outliers that perform poorly (Note that random center choicecan actually pick the worst-case center); if only a few nodes make poor centers, the average performance may stillbe reasonable. Random center choice requires no knowledge of the scenario for which the center is selected.To determine the feasibility of random center choice, we take the average of the center performance in eachscenario. The right plot of Figure 4 shows the average bandwidth and delay ratios of a random center choice tothe shortest-path routing. Observe that the �gure looks very similar to the left plot except that the range of y-axis is reduced by almost 40%; as noted earlier, the delay performance is similar across the three scenarios, whilethe bandwidth performance varies somewhat more across the scenarios. Not surprisingly, the random center choicemethod performs better than the worst-case center choice method. To further explore random center choice, weconsider the variance of center performance. We observed high variability of the random center choice method forboth the Localized Receivers and the All Receivers Sources scenarios, implying that some applications may experiencea wide range of bandwidth performance across instances, possibly making a random center choice unacceptable.

144.3 Topology-Based Center ChoiceDespite the improvement we get from random over worst-case, a random center still results in average delay that is60-70% worse than the shortest-path routing delay. Note that work by Wei and Estrin[16] reports ratios of center-based to shortest-path based protocols which are much smaller, but this is for optimal centers. While the bandwidthapproaches the shortest-path routing bandwidth, the variance is quite high for some scenarios. We now move tomethods that use information about the gross topological characteristics of the network to make better center choices.We de�ne a single method that chooses a center according to the depth of the shortest-path tree from each node.Formally, let d:u denote the depth of the shortest-path tree rooted at u. A topological center3 of a graph is anynode, c, that minimizes d:c. Our method takes a parameter t and selects a center, u, at random from those satisfyingd:u � d:c+t. In e�ect t speci�es a tolerance on the depth of the nodes that are considered. By varying the parametert, we get a family of methods between the extreme of choosing a topological center (t = 0) and choosing a node atrandom. We will let Tol-t denote this method with parameter t.For Tol-0, the results are similar to the results for worst-case and random choice, but with a reduction in therange of the y-axis by 25% from the random choice. There are at least two reasons to consider the performance of atopological center choice as the tolerance t is increased. First, since the graph center would likely be determined ona coarse time scale, we need to examine the e�ects of imperfect center information on center performance. Second,to distribute the network load, we might want to select centers from a larger set than the set of topological centers.This issue is examined further in Section 5.Figure 5 shows the ratio of delay for center-based trees to delay for shortest-path routing when the tolerance t isincreased. All Receivers Sources is shown on the left, and Single Source Scenarios on the right. (Localized Receiversis similar and is excluded for brevity.) An increase in delay as the tolerance increases is consistent across all groupsizes, with a signi�cant di�erence between choosing from a center (Tol-0) and choosing at random. It is interestingthat the shapes of the curves for a particular scenario remain similar across all tolerances. The e�ect of increasingthe tolerance is less pronounced for bandwidth performance, particularly for the All Receivers Sources and SingleSource Scenarios.3Our goal is to assess the performance of center-based routing when using network centers, not to develop e�cient methods for �ndingnetwork centers.

154.4 Group-Based Center Choice MethodsFor worst-case, random, and topologically based center choice methods, the localized receiver scenario is consistentlythe worst with respect to bandwidth. This leads us to consider several center choice methods that make use ofinformation about the locations of the receivers and/or sources. Note that the group-based center choice requires themost complex knowledge of the methods we consider; the characteristics must be determined per group rather thanper network. The practical application of these methods depends not only on the availability of information aboutthe receivers (group members) and their locations, but may also require e�cient and reliable methods of changingthe center when the participant distribution changes. (See Section 6.)For comparison we also consider performance-based center choice methods: these select from a set of nodes(receivers, sources, or all nodes) the node with the best value of some weighted combination of bandwidth and delay.Such a method requires even more information than group-based methods, because the performance of all nodes inthe subset must be known (or estimated) to make the choice.We consider the following methods of choosing a center:Random Receiver One of the receivers is selected as center. Results reported represent the ratio of averagebandwidth (delay) over all receivers to the shortest-path routing bandwidth (delay).Center of Receivers A center is chosen from among the topological centers of the subgraph induced by the re-ceivers. This is only possible when the induced subgraph is connected4 , which is only guaranteed in theLocalized Receivers scenario. Reported results represent the ratio of the average over all centers of the inducedsubgraph to the shortest-path routing.Best Receiver The bandwidth and delay for the given con�guration is measured with each receiver as center, andthen the best is chosen according to a weighted combination of bandwidth and delay. Results reported forthis metric are either strongly weighted towards delay or bandwidth, and represent the ratio of that center'smeasures to the shortest-path routing.We compare the group-based methods, along with the Tol-0 topological choice, for each of the three scenarios.4The induced subgraph is the graph consisting of all receivers and all edges between pairs of receivers.

16As a point of reference, we also include the performance of optimal center choices, with curves labeled Best-BW andBest-DL. These curves indicate the bandwidth (delay) required by the best center for a heavily weighted towardsbandwidth (delay) combination of bandwidth and delay. Best-BW-Rcvr (Best-DL-Rcvr) indicates the bandwidth(delay) of the center among the receivers with the best value of the same metric.Consider the Localized Receivers scenario where the knowledge of localization can be used to improve bothbandwidth and delay. (See Figure 6.) Choosing the topological center of the subgraph induced by the receivers orjust choosing a receiver at random results in bandwidth and delay performance which is better than the performanceof a graph center. In fact, the bandwidths of the receiver center and the random receiver are almost as good asthe bandwidth performance of the optimum center for bandwidth. Observe that while the receiver center providesthe best performance for localized participants, if �nding the center is computationally unacceptable, selection of arandom participant e�ectively gives a comparable bandwidth performance and less than 10% increase in delay.To highlight the di�erences in center performance along the two performance dimensions (bandwidth and delay),we present in Figure 7 a representative scatter plot showing the absolute (bandwidth, delay) measure for each ofthe 400 nodes in one graph under the All Receivers Sources scenario. Receivers (\Group Members") and the threecenters of the graph are marked di�erently; the shortest-path routing value is shown with a plus. There are fewerthan 400 points in the scatter plot since multiple nodes can have the same (bandwidth, delay) measure. Nodes closestto the origin represent good performance along both dimensions; note that several of these nodes are neither centersnor receiver/sources indicating the capability for further improvement.5 Tra�c ConcentrationOne of the purported drawbacks of center-based trees is their tendency to concentrate tra�c on certain links of thenetwork, due to the sharing of the center-based routing tree by all sources in a group [7, 16]. We demonstrate thattra�c concentration is not inherent in the center-based tree approach; rather it is sensitive to center choice and canbe signi�cantly reduced (if desired) by choosing centers at random.To measure tra�c concentration, we model a large number of connections in the same network. For each connec-tion, we determine the routes for shortest-path and center-based trees, and determine the tra�c concentration on

17each link. The results presented here are for a 49 node, two-level graph with 300 groups and 10 members per group,with each group an All Receivers Sources scenario5. Figure 8 shows the histogram of link load for shortest-path trees.Most links carry a load of under 1000 ows (A ow is the tra�c carried from a source across a link to downstreamreceivers); the average load per link is 440. At the extremes, there is one link which carries over 2000 ows; theminimum number of ows carried by any link is eight.We now consider center-based trees with the center for each group chosen from one of the two topological centersof this particular graph. The left plot of Figure 9 shows the link load histogram for this method of center choice.Considerably more links carry load over 1000 ows; the average load per link is 580. The most signi�cant tra�cconcentration occurs on a cluster of links with load around 2000 ows, due to the use of only two centers by all ofthe groups. There are 50 links which carry no tra�c whatsoever and are not included on this plot. These results arequalitatively similar to the tra�c concentration reported by other authors [16].To demonstrate that tra�c concentration is in uenced (and can be alleviated) by center choice, we also computethe link load when the center for each group is chosen at random from all nodes in the graph. The right plotin Figure 9 shows this result. In marked contrast to the topological-center, this method of center choice does anexcellent job of distributing the tra�c. In fact, the average load per link is 420 and the maximum is just under 2000| both improvements on the shortest-path tree measures!Just as delay and bandwidth are a�ected by center choice, so too is tra�c concentration. The key to avoidingoverloaded links in center-based trees is to choose centers from a reasonably large set of routers.6 Center MigrationWe now turn to the problem of identifying when and where to move the center, due to dynamic changes in theapplication over time. We consider PIM center-based routing for this evaluation. Figure 10 demonstrates that ascenario with moderate migration can experience poor performance with a �xed center compared to an optimalcenter. Thus we know that migrating the center has the potential to improve performance for some scenarios. Notethat we do not discuss the actual protocol mechanism for migrating a center. Most likely the approach would be5Results for the other scenarios were similar and are not presented here.

18similar to the routing protocol technique for switching to a secondary center after the primary fails.The outline of the center migration algorithm is as follows:1. Determine a set of candidate centers.2. Request from each candidate center, c0, the expected performance, were c0 to be the center.3. If the candidate center with the best performance is signi�cantly better than the current center, migrate tothis best candidate.The algorithm proceeds by allowing the current center, at regular, possibly application-dependent intervals, tocompare its multicast performance to the performance possible with some other node as center. If another nodeis identi�ed with signi�cantly better performance, it becomes the new center. The regularity of the probing andthe threshold used to decide what constitutes \signi�cantly better performance" are dictated by both applicationbehavior and the overhead cost associated with moving the center (This approach could be used to identify the bestn centers, providing a list of backup centers in the event of primary center failure). Later in the paper we discuss anadaptive scheme to control the frequency of probing; we also examine the overhead caused by the evaluation process.There are two major issues associated with this algorithm:� Candidate center selectionWe must ensure that the pool of candidate centers includes those nodes which are likely to improve performanceover the current center. In addition, we must keep this pool small, to reduce control message overhead andmaintain scalability. In Section 6.1 we develop a policy to select a good, but small, pool of candidate centers.� Candidate center evaluationThe information necessary to evaluate a candidate center is not available at any single, centralized location inthe network. We develop distributed methods to gather a reasonable amount of information to allow accurateestimates of performance. Section 6.2 describes our evaluation methods.

196.1 Selecting Candidate CentersIn order for our Center Migration algorithm to improve performance, the set of candidate centers must include some(at least one) of the nodes that are signi�cantly better than the current center. The objective of including highquality centers must be balanced with the need to keep the number of candidate centers low. Later we will see thateach candidate center separately receives messages from the receivers, thus the amount of control message overheadscales linearly with the number of candidates. We aim to identify a small number of candidates with a high likelihoodthat one will be signi�cantly better than the current center.Before suggesting a policy for selecting candidate centers, we examine the question of where the good centers arelocated. Given the structure of the transit-stub graph model, we determined the ratio of the following four centersto an optimal center: the best transit center, the average transit center, the best stub center, and the average stubcenter. For the various scenarios, we found that the optimal transit node performed at or very near optimal forboth delay and bandwidth. The average transit and optimal stub nodes performed the next best while the averagestub was signi�cantly worse than optimal. Further investigation reveals that when the group has a small number ofreceivers and sources concentrated in a single stub domain, the optimal center tends to be a stub node, speci�callya node from the domain with all the participants.We now propose the following candidate selection policy which attempts to exploit the previous observationsregarding center location and performance: one candidate center is chosen from each transit domain. In addition,one candidate is chosen from the stub domain with the most representation by sources and the stub domain with themost representation by receivers. If the network has a prohibitively large number of transit domains, we augmentthis policy with the speci�cation that candidates be chosen from randomly selected transit domains. By selecting arandom center from each transit domain, we approximate using the average transit node from each transit domain.Selecting a random node from the most populated stub domains approximates selecting the best stub node since, ifthe participants are highly localized, we know that a very good center likely comes from that locale.The policy for candidate center selection is orthogonal to the candidate center evaluation algorithm, and manyplausible policies exist for selecting candidate centers. Comparing various candidate center selection policies is beyondthe scope of this paper; however, in Section 7 we do compare our policy to the (non-practical) alternative of selecting

20the optimal center.6.2 Evaluating Candidate CentersThe previous section developed a method for selecting candidate centers, based on observations about where goodcenters were likely to be located. Left out of the discussion was the question of whether the bandwidth and delay ata candidate center can be evaluated exactly. In fact, given routers may have little or no topology information andthe algorithm described thus far, exact evaluation requires information not available at the candidate centers. Wenow consider how to provide and use information at a candidate center to evaluate performance; our evaluation willbe exact for delay performance but approximate for bandwidth performance.We obtain information to use in performance evaluation by soliciting help from the participants (i.e., the des-ignated routers for the sources and receivers). Thus we propose augmenting the control messages that are part ofthe multicast protocol to include messages that facilitate center migration. Note that we build our protocol basedon the assumptions in Subsection 3.4 so the center does not know the identity of the participants. To initialize theevaluation, the center sends a multicast message to the receivers and a unicast message to each source6, listing thecandidate centers. Each participant responds (The centers use timeouts to determine when all responses have beenreceived. The speci�cs of the timeout mechanism is beyond the scope of this paper) by sending a unicast to eachcandidate center, containing a hop-count �eld that is updated as the message makes its way from the participantto a candidate center. This allows a candidate center to know the hop count from itself to each participant in thegroup. Each candidate center then evaluates and reports its cost to the current center. Figure 11 demonstrates themessage ow of the protocol. The bandwidth overhead for each evaluation consists of packet hops required for theunicast/multicast to the sources/receivers from the center, the unicast from the participants to the candidate centers,the unicast from the candidate centers to the center, and, if migration occurs, the noti�cation of the participants ofthe center change.Let r denote the number of receivers and di denote the distance (in hops) from the candidate center to receiveri. Let s denote the number of senders and si denote the distance (in hops) from sender i to the candidate center.6The center knows the sender by monitoring the sources of multicast packets it has routed over some time interval.

21The delay performance metrics are easy to compute. The average and maximumdelay for messages from source i is:Davg(i) = si + 1r r�1Xj=0 djDmax(i) = si +maxj fdjgThe bandwidth is not directly computable from the distance information, since bandwidth depends on the amountof overlap in the shortest paths from the center to the receivers. In order to estimate the multicast bandwidth, we�rst derive an upper bound, and then relax the bound to obtain an estimate.We observe that the capability to estimate multicast performance from unicast measurements could be of valuein other situations, beyond the context of dynamic multicast routing. For example, this type of information would beuseful for deciding whether to deliver information (e.g., from a server) via unicast or multicast. A server may beginby unicasting, and switch to multicast when the performance bene�t warrants the switch. Our methods for inferringmulticast performance from unicast performance would be useful to the server to determine bandwidth bene�t.6.2.1 Evaluating Multicast BandwidthWe want to determine an upper bound on the bandwidth performance at a candidate center, corresponding to theleast amount of overlap in the shortest paths reaching the receivers. Note that in addition to the distance information,the candidate center also knows the outgoing interface for each receiver. Thus the set of receivers can be partitionedaccording to outgoing interface, and the bandwidth can be evaluated separately over each interface. We also assumethat the candidate center knows the maximum degree of any node in the network, denoted �.Since each interface can be treated separately, we develop an upper bound for bandwidth to the receivers reachedover a given interface. We modify the earlier notation to let r denote the number of receivers reached over thegiven interface, and di denote the distance from the interface node to receiver i. (Note that this is one less than thedistance from the candidate center to receiver i.) The least amount of path overlap (and hence the largest multicastbandwidth) results when the shortest path tree from the candidate to the receivers branches as much as possible asclose to the center as possible. The degree of branching at a node is limited to �� 1 by the node degree constraint.Figure 12 shows the distribution tree from candidate center C out the interface corresponding to (C; 1), with levels

22indicated.The worst case distribution tree is one which is \full" for as many levels as possible (each node branches by� � 1). A multicast distribution tree containing a full subtree, S, with i levels reaches at least (� � 1)i nodesdownstream from and including the leaves of S. The maximum depth of S is then the maximum value h such thatjfdijdi � hgj � (� � 1)h. The intuition is that there must be enough receivers at distance h or greater from thecandidate to allow full branching to a depth of h. The link (C; 1) from the candidate center to the interface node,plus the full subtree S, contribute multicast bandwidth equal to Phi=0(�� 1)i.After building a full subtree, S, to depth h, there may be some \extra" receivers that are at distances greaterthan h. Note that the number of extras is strictly less than (� � 1)h+1, otherwise the full tree could be extendedby one more level. Since there are fewer than (�� 1)h+1 extra receivers, they can be reached from the leaves of S,using disjoint paths to each extra receiver. Thus the multicast bandwidth to the receivers out the given interface isbounded by: hXi=0(�� 1)i + Xj s:t: dj>h(dj � h)where the �rst term accounts for the full subtree, S, and the second term accounts for the extra receivers.The total bandwidth, B, for a candidate center is the sum of the bandwidth from each source to all receivers.Each source unicasts to the candidate then multicasts using the receiver distribution tree. Recall that si denotes thedistance from source i to the candidate, and s denotes the total number of sources. Let Bj denote the maximummulticast bandwidth to the receivers out interface j for the candidate center. The total bandwidth, B, for a multicastfrom all sources is bounded by Pi si + sPj Bj . Each candidate center computes the upper bound and sends thisas an estimate of bandwidth to the current center. If the current center decides to perform center migration, thisoccurs using a center change protocol, such as in PIM [7].We evaluate the e�ectiveness of this method by inspecting the ratio of our estimated multicast bandwidth tothe actual multicast bandwidth over randomly generated instances using the transit-stub graphs. The left plot ofFigure 13 shows the ratio as a function of the number of receivers. Each data point corresponds to a randomlychosen center and set of receivers. The striking feature of this plot is the tendency of the data to fall into threebands. Further investigation reveals that the bands are distinguished by the proximity of the center to a transit

23domain. The points marked by the diamond are transit nodes, those marked by the plus are stub nodes directlyconnected to transit nodes, those marked by the box are stub nodes two hops away from a transit node, and thosemarked by the \x" are three or more hops from a transit node. The lowest band consists entirely of transit nodesand directly connected stub nodes. The middle band is made up of stubs that are two hops away from a transitnode. The intuition is that the closer the center is to a transit domain, the more accurate the assumption aboutmaximum node degree (since transit nodes have higher average node degree than stubs). In addition, transit nodesare nearer to the \middle" of the network, thus the shortest paths from a transit center to a set of random receiversare more likely to be disjoint.This result is signi�cant since our candidate centers come primarily from transit domains. While the estimationcan be o� by a factor of four or more for some stub nodes, it is generally within a factor of two for transit nodes.Our analysis of multicast bandwidth above is an upper bound that assumes branching by the maximum possible(i.e., ��1) at each level of the tree. A single node with large degree can in uence the tightness of our upper bound.Relaxing this, we can use the average node degree to get a closer estimation to the actual multicast bandwidth. (Ofcourse, we no longer have a guaranteed upper bound.) The right plot of Figure 13 repeats the previous experimentusing that average node degree instead of the maximum. We see the same banding e�ect when using average nodedegree; we note that the estimation for transit nodes is now generally within a factor of 1.25.We can do a �t to each band of data to get equations to use in estimating multicast bandwidth. A node whichknows it is a transit node can use the 1.25 factor to more accurately estimate bandwidth based on computation usingaverage node degree. Note that this factor is graph model speci�c.We note that Thaler and Ravishankar [13] also give a method to estimate bandwidth from information on distancesto receivers. Their estimate is the average of a lower and upper bound; their upper bound is not as tight as ours,using only the degree at the tree root to tighten the naive bound consisting of the sum of the distances to all receivers.Their lower bound is fairly straight forward, based on maximizing the amount of path overlap.6.2.2 Random Participant EvaluationAt this point, we have presented an algorithm which, using participant feedback, evaluates the performance atcandidate centers. Unfortunately, the message complexity of the algorithm increases with the number of participants.

24This not only creates additional network load but can lead to message implosion at the candidate centers. In thissection we introduce Random Participant Evaluation, which works in the same manner as above except that itutilizes probabilistic multicast[2] to control the number of responses sent by participants to each candidate center.In probabilistic multicast, a packet is multicast to all receivers. The header of this multicast packet contains anacceptance probability, �, which each participant uses to determine whether or not to respond. That is, eachparticipant will send a message to the candidate center with probability � (Recall that the sender solicitation issent by unicast so the current center approximates probabilistically multicasting to the senders by unicasting toeach sender with a probability of �). The candidate centers will use the information obtained from the limitedset of participants to estimate performance. In what follows we describe our method for estimating the number ofparticipants, and then evaluate the accuracy of the performance estimates.6.2.3 Determining the Number of ParticipantsTo maintain scalability, we must determine � based on the number of participants. We adapt the receiver congestiondiscovery protocol proposed by Bolot [4] to determine the number of receivers. We do not describe Bolot's techniquehere, but since ours is a simple modi�cation, the description of our approach is, for the most part, a description ofBolot's approach. Our algorithm chooses an initial value of �, and the center sends a probabilistic multicast to thereceivers. The initial value of � is chosen conservatively, allowing for few or no responses. We modify the receiverresponse to send a message to the current center, in addition to sending a unicast message to each candidate center.If no responses are received at the current center in the initial round,7 � is increased, and a probabilistic multicast,again, solicits responses from receivers. This process is repeated until a response is received or � = 1. (Note thatwhen � = 1, every receiver will respond.) We estimate the number of receivers based on the �rst round in which aresponse is received.Let M be a bound on the maximum number of receivers, and A be the desired number of responses sent to eachcandidate center from the receivers. Initially � = A=M , clearly conservative since in general M is larger than thenumber of receivers and considerably larger than A. The value � is increased exponentially until a receiver responds.7A timeout of twice the maximum round trip time is used to determine the end of a round. See [4] for a description of the round triptime discovery.

25As in [4], we analyze the algorithm to determine the expected round in which a �rst response occurs. Following thenotation in [4], we let rj denote the number of responses at round j, pj denote the probability that a particularreceiver replies during round j, and n denote the number of receivers. We note that these de�nitions assume thatwe proceed through all of the rounds, even after a response has occurred. We expect rj to initially be close to 0and then to monotonically increase as the rounds proceed and � is increased. In our algorithm, unlike Bolot's, pj isindependent of the responses from previous rounds.Assuming no messages are lost in the network, the probability we get at least one response in round j isPr(rj > 0) = 1� (1 � pj)nwhere pj = 8>><>>: AM � 2j j � blog2MA c1 otherwiseThe expected round of the �rst reply isE(First) = dlog2 MA eXj=0 jPrf�rst response at round jgPrfresponse at round j and no response in rounds 0...j-1g= dlog2 MA eXj=0 j(1 � (1 � pj)n) j�1Yk=0(1� pk)n!From this we can show that our method consistently gives a �rst response in an earlier round than Bolot's method,with a response about three rounds earlier for A=M = :0001 and as many as 10 rounds earlier for A=M = :01,independent of n. The intuition for the earlier responses is that the initial value of � can get us closer to the correctnumber of responses than Bolot's bit masking scheme. Of course, Bolot's method can be adapted to begin in ahigher indexed round. In addition, we are able to solicit responses with a �ner granularity by using a probability �rather than the bit mask approach of Bolot. With bit masking, the granularity with which responses are solicitedis restricted to powers of 2. The Bolot scheme does reduce the number of random number generations required atreceivers; we require a new random number at each receiver for each round, while Bolot requires only one randomnumber per \epoch", or series of rounds until a response is received.

26We can �t the A=M = :0001 curve to a straight line, derivingn � e�0:714�E(First)+8:4Assuming the boundM is well chosen (i.e., is larger than n), this algorithm estimates the number of receivers withoutimplosion. To show this, consider the probability that we reach round j and m responses are received in round jPr(rj = m ^ no prior response) = Pr(rj = m)Pr(8k; 0 � k � j � 1; rk = 0)Pr(rj = m) is distributed according to a binomial distribution with p = � AM � 2j� and number n, andPr(8k; 0 � k � (j � 1); rk = 0) = j�1Yk=0(1 � pk)n:The probability of more than 10 responses in a round for n = 10; 000 and A=M = :0001 reaches its maximum of0.00017 in round 3 which means that the probability of implosion if very small.Since the number of receivers may change considerably over the lifetime of the multicast group, we must pe-riodically re-estimate the number of receivers. Recall that the smaller the number of receivers is, the larger theexpected round of the �rst response; consequently, a small group could experience signi�cant delays8 in estimatingthe number of receivers. To avoid this, we propose changing the value ofM after each receiver count estimation. Wecannot simply change M to the number of receivers we discover, since message implosion may result if the numberof receivers signi�cantly increases between receiver count estimations. To attempt to avoid this, after the initialreceiver count estimation, we set M to twice the number of estimated receivers. If the number of receivers stays thesame, the estimation should take only two rounds since the probability of receiver response doubles each round. Inaddition, the number of receivers would have to double to cause message implosion at the next estimation. Afterthe initial receiver count estimation, we adjust M using exponential increase if too many receivers respond on the�rst round and multiplicative decrease if it takes more than two rounds to perform the receiver count estimation.8Recall that each round with no response takes at least twice the maximum roundtrip time.

27E�ectively, this ensures that only a large increase in receivers will cause message implosion.6.2.4 Estimating Performance with Limited ParticipationEach candidate center must estimate performance given data about the unicast routes between itself and a limitedset of group participants. We now must estimate average and maximumdelay, in addition to bandwidth, since we donot have the full knowledge necessary to compute exact delays. We use the average delay of the limited participantsas the estimate of average group delay, relying on the assumption that the limited participants are, on average, agood representation of the total group. Our experimentation shows that even a small percentage of participants cangive a good approximation to the average delay. For maximum delay, we know that the limited participants willnot give us a accurate estimate, unless we happen to have heard from the furthest source and destination. However,all of the candidates centers will be similarly limited in accuracy, thus for the purposes of comparison we use themaximum delay of the limited participants.To estimate bandwidth, we assume that each responding participant serves as a \proxy" for the participants thathave not responded. Thus, if � = :1 and we hear from a receiver that is �ve hops away, we assume there are 10receivers (1=�) at distance �ve hops. We then calculate multicast bandwidth using the algorithm of Section 6.2.1.Repeating the experiments of section 6.2.1 with 25% of the participants providing data to the candidate centerand using the average node degree, we get very similar results to �gure 13 except the bands are not as cleanlyseparated. The accuracy for transit and directly connected stub nodes is still quite good, with most values in therange 1.0 to 1.25. If we decrease to 10% participation, we see bands of data that are more muddled, but the generaltrends are preserved. For transit and directly connected stub nodes, most values are in the range .75 to 1.25.7 Summary and ConclusionsFor the problem of center selection, our results yield no single best selection method; instead, they suggest points ofconsideration when making a center choice and demonstrate the potential consequences. To select from our centerchoice methods, an administrator must consider the tradeo�s between performance and required information. At oneextreme, no information about network topology is required for performance at least as good as the worst-case. In

28fact, an administrator may designate one center for use by all multicast groups. The advantage is simplicity, but badcenter performance and increased tra�c concentration may result. At the other extreme, the administrator acquiresinformation about network topology and group distribution (e.g. whether receivers are localized) to designate acenter that o�ers good performance for that particular instance. In this case, the center choice method uses theapplication semantics to decide between a topological or group based center choice. These tradeo�s are importantbecause in general, information about the network and group topology is not trivial to come by.For each scenario and each of the four center choice methods, Tables 2 and 3 show bandwidth and delay perfor-mance measures, respectively, for the smallest group instance of each scenario9. The improvement of bandwidth anddelay performance for random center choice over worst-case is approximately 30% for both bandwidth and delay. Inaddition, moving from worst-case to random center choice also reduces the chance for center clustering which resultsin tra�c concentration. In choosing between worst-case (arbitrary) and random, the administrator must decide ifinsuring random center placement is worth avoiding the performance extremes of the worst-case center choice.Observe the improvement in using the topology-based center choice instead of the random center choice. Whilebandwidth performance has almost negligible improvement for the Localized Receivers and All Receivers SourcesScenarios, the delay decreases by approximately 20%. If delay is important and several groups will use the topologicalcenter of the network before it is recalculated, choosing the topology-based center choice makes sense. Finally, forgroup-based choice, note that the administrator needs to know the application semantics because the performance ofthe group-based choice is scenario dependent. If the scenario has highly localized participants, then the group-basedchoice o�ers a 22% improvement in bandwidth over the topology-based choice and loses nothing in delay performance.However, if the participants are not localized, bandwidth performance improves nominally but delay increases byapproximately 25% from the topology-based choice. In this case, the administrator should use a method other thangroup-based to place the center.For center migration, we summarize our results by evaluating the full algorithm on each of the four scenariosdescribed earlier. The simulation proceeds by updating participants according to the multicast routing scenarioparameter speci�cations at each time step. Every 10 time units, the simulation evaluates the performance of the9Recall that this performance is the ratio or center-based routing the shortest-path routing. Center-based approaches can performbetter than shortest-path tree yielding ratios less than one.

29current multicast trees. The initial center evaluation frequency is :01 which means the �rst center evaluation occurs attime step 100. The center evaluation frequency adapts thereafter with a maximum frequency of :01. The simulationstops after 1000 time steps.Figure 14 shows the ratio of the bandwidth, average delay, and maximumdelay performance with no migration toour center migration scheme for scenario RC (top left), scenario VC (top right), scenario IS (bottom left), and scenarioSM (bottom right). The migration decisions are based exclusively on the bandwidth metric. The B/W(Amortized)plot is the ratio of the non-migration bandwidth to the sum of the migration bandwidth and the amortized centerevaluation overhead (amortized over the all multicasts during the same interval). For example, let N and M be thenon-migration bandwidth and migration bandwidth for the multicast at time t, respectively. Let O be the numberof packet hops required by control messages for the last center evaluation, and let there be C multicast messagessent by sources between the last and the next center evaluation. The amortized bandwidth ratio for time t isB=W (Amortized) = NM + OCTable 4 gives the average improvement of migration over non-migration for each scenario and performance mea-sure. We use the simulation results to reach the following conclusions about center migration:Bandwidth. Our center migration algorithmprovides a modest, consistent improvement in bandwidth performance.In fact, even migration bandwidth with amortized overhead costs are always at least as good as non-migrationbandwidth costs.Delay. Even when using the bandwidth metric to make center migration decisions, maximum and average delayimprove markedly (e.g. on average 142% for average delay in the SM scenario) 10Adaptation. The points near the x-axis indicate when center evaluation and migration occurs. Note from Figure 14that the more dynamic scenarios (e.g. IS and RC) have frequent center evaluation while less dynamic scenarios(e.g. SM and VC) have signi�cantly fewer center evaluations. This demonstrates that an adaptive center10Reducing multicast bandwidth correlates closely with reducing delay; using delay as the center migration criteria produces resultssimilar to these.

30evaluation frequency mechanism does adapt itself to application behavior, reducing center evaluation overheadfor less dynamic applications.Overhead. We allow exibility in controlling overhead by determining the frequency and scope of evaluation.Avenues for future research include improving upon the set of parameters used for multicast scenario control,investigating the relationship between performance and graph topology, further re�ning the candidate center selectionpolicy and comparing the center migration performance to other center selection methods.AcknowledgmentsWork supported by NSF MIP-9502669. Preliminary versions of the work appeared in [5, 10].References[1] K. C. Almeroth and M. H. Ammar. Characterization of mbone session dynamics: Develop-ing and applying a measurement tool. Technical Report GIT-CC-95/22, Georgia Institute ofTechnology, June 1995.[2] M. H. Ammar. Probabilistic multicast: Generalizing the multicast paradigm to improve scal-ability. In Proceedings of INFOCOM 1994, pages 848{55, 1994.[3] T. Ballardie, P. Francis, and J. Crowcroft. Core based trees (CBT) an architecture for scalablemulticast routing. In Proceedings of ACM SIGCOMM '93, pages 85{95, 1993.[4] J. Bolot, T. Turleeti, and I. Wakeman. Scalable feedback control for multicast video distributionin the internet. In Proceedings of SIGCOMM, pages 58{67, 1994.[5] K. Calvert, E. Zegura, and M. Donahoo. Core selection methods for multicast routing. InProceedings of IC3N '95, 1995.

31[6] S. E. Deering and D. R. Cheriton. Multicast routing in datagram internetworks and extendedLANs. ACM Transactions on Computer Systems, 8(2):85{110, May 1990.[7] S. E. Deering, D. Estrin, D. Farinacci, V. Jacobson, C. Liu, and L. Wei. Protocol independentmulticast (PIM): Motivation and architecture. Working Draft, March 1994.[8] S. E. Deering, D. Estrin, D. Farinacci, V. Jacobson, C. Liu, and L. Wei. Protocol independentmulticast (PIM), sparse mode protocol speci�cation. Working Draft, March 1994.[9] M. Doar and I. Leslie. How bad is naive multicast routing? In Proceedings of INFOCOM 1993,pages 82{9, 1993.[10] M. Donahoo and Zegura E. Core migration for dynamic multicast routing. In Proceedings ofIC3N '96, 1996.[11] Institute for Simulation and Training. Standard for Distributed Interactive Simulation |application protocols (version 2.0.4 draft IEEE standard). Technical Report IST-CR-94-50,University of Central Florida, March 1994.[12] C. Partridge, D. Waitzman, and S. Deering. Distance vector multicast routing protocol. InternetRequest for Comments 1075, November 1988.[13] D.G. Thaler and C.V. Ravishankar. Distributed center-location algorithms: Proposals andcomparisons. In IEEE Infocom '96. IEEE, March 1996.[14] B. M. Waxman. Routing of multipoint connections. IEEE Journal on Selected Areas in Com-munications, 6(9):1617{22, December 1988.[15] B. M. Waxman. Performance evaluation of multipoint routing algorithms. In IEEE INFOCOM93, volume 3, pages 980{6, March 1993.

32[16] L. Wei and D. Estrin. The trade-o�s of multicast trees and algorithms. In Proceedings ofICCCN '94, 1994.[17] E. W. Zegura, K. L. Calvert, and S. Bhattacharjee. How to model an internetwork. In Pro-ceedings of INFOCOM 96, 1996.Scenario Mig Prob New Part Life Part SourceVC U(0.8) E(50,0.9) U(1) U(.5)IS U(0.2) U(0.2) U(0.2) U(0.5)RC U(0.9) U(0.9) U(0.02) U(0.6)SM U(0.8) N(600,150,0.8) U(0.1) U(0)U (I) = I;N (�; �2; I) = I � exp�(T IME��)22�2 ; E(�; I) = I � exp�TIME�Table 1: Scenario parametersLocalized Receivers Single Source All Receivers SourcesWorst-Case 1.9 1.5 1.6Random 1.2 1.1 1.15Center 1.1 1.05 1.05Group 0.85 1.0 1.0Table 2: Comparison of bandwidth performance for center choice methodsLocalized Receivers Single Source All Receivers SourcesWorst-Case 2.5 2.6 2.5Random 1.6 1.65 1.7Center 1.3 1.25 1.3Group 1.15 1.6 1.65Table 3: Comparison of delay performance for center choice methods

33Scenario B/W(Amortized) B/W Avg Delay Max DelayIS 22% 28% 137% 74%RC 32% 48% 139% 85%SM 15% 37% 142% 112%VC 8% 10% 117% 74%Table 4: Average (%) Improvement of Migration over Non-migration

R

R

R

C

S2

R

R

R

S1

Path of S2’s Packets

Path of S1’s Packets

Figure 1: Multicast routing subgraph.

34

����

����

���� �

���

����

����

����

����

Stub Domains

Stub-Stub edge

Transit Domains Multi-homed Stub

Figure 2: Transit-Stub domain structure

35

0

2

4

6

8

10

12

14

16

18

20

0 200 400 600 800 1000

Cou

nt

Time

Distribution of Participants for Scenario IS

ReceiversSources

Transit DomainsStub Domains

0

2

4

6

8

10

12

14

0 200 400 600 800 1000

Cou

nt

Time

Distribution of Participants for Scenario RC

ReceiversSources

Transit DomainsStub Domains

0

5

10

15

20

25

30

35

40

0 200 400 600 800 1000

Cou

nt

Time

Distribution of Participants for Scenario SM

ReceiversSources

Transit DomainsStub Domains

0

2

4

6

8

10

12

14

16

18

20

0 200 400 600 800 1000

Cou

nt

Time

Distribution of Participants for Scenario VC

ReceiversSources

Transit DomainsStub Domains

Figure 3: Scenario Participant Distribution

361

1.2

1.4

1.6

1.8

2

2.2

2.4

2.6

2.8

0 20 40 60 80 100Percentage of Participants in Scenario

Bandwidth/Delay Performance of Worst Center Choice for Bandwidth/Delay

All-rcvrs-srcs-delayOne-src-dist-rcvrs-delay

Localized-rcvrs-delayAll-rcvrs-srcs-bw

One-src-dist-rcvrs-bwLocalized-rcvrs-bw

1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

0 20 40 60 80 100Percentage of Participants in Scenario

Performance of Random Center Choice

All-rcvrs-srcs-delayOne-src-dist-rcvrs-delay

Localized-rcvrs-delayAll-rcvrs-srcs-bw

One-src-dist-rcvrs-bwLocalized-rcvrs-bw

Figure 4: Bandwidth and delay for worst and random center choice1.2

1.25

1.3

1.35

1.4

1.45

0 20 40 60 80 100Percentage of Participants in Scenario

Impact of Center Choice Tolerance on Delay

All-rcvrs-srcs-Tol-0All-rcvrs-srcs-Tol-1All-rcvrs-srcs-Tol-2

1.15

1.2

1.25

1.3

1.35

1.4

0 20 40 60 80 100Percentage of Participants in Scenario

Impact of Center Choice Tolerance on Delay

One-src-dist-rcvrs-Tol-0One-src-dist-rcvrs-Tol-1One-src-dist-rcvrs-Tol-2

Figure 5: Impact of tolerance on delay for ARS and SS

370.8

0.9

1

1.1

1.2

1.3

1.4

0 20 40 60 80 100Percentage of Participants in Scenario

Bandwidth and Delay Performance for Localized Receivers

Tol-0-BWRand-Rcvr-BW

Ctr-Rcvrs-BWTol-0-DL

Rand-Rcvr-DLCtr-Rcvrs-DL

Best-BW-RcvrBest-BW

Best-DL-RcvrBest-DL

Figure 6: E�ect of center choice method for Localized Receivers10

15

20

25

30

35

40

45

350 400 450 500 550 600 650 700 750

Max

Del

ay (

Hop

s)

Bandwidth (Hops)

All Receivers Sources, Group size 10

Shortest-Path PerformanceGroup Members

Graph Centers

Figure 7: Representative bandwidth-delay plot

380

2

4

6

8

10

0 500 1000 1500 2000 2500

Fre

quen

cy

Link LoadFigure 8: Tra�c concentration in shortest-path trees0

2

4

6

8

10

0 500 1000 1500 2000 2500

Fre

quen

cy

Link Load

Topological Center Choice

0

2

4

6

8

10

0 500 1000 1500 2000 2500

Fre

quen

cy

Link Load

Random Center Choice

Figure 9: Tra�c concentration with topological and random center

39

1

2

3

4

5

6

7

8

9

0 100 200 300 400 500 600 700 800 900 1000

Rat

io o

f Non

-Mig

ratio

n C

ente

r to

Opt

imal

Cen

ter

Time

Non-Migration Performance

B/WAvg DelayMax Delay

Figure 10: Bandwidth and delay performance of a �xed center...

S

R R R

C

S

?

?

Init and New Core

Reply

Cost Report

? Candidate CoresFigure 11: Message ow to evaluate candidate center

40C 1

2

3

4

5

Level 1

Level 2

Level 0

-1

-1Figure 12: Distribution tree

1

2

3

4

5

6

7

0 50 100 150 200 250 300

Rat

io o

f Est

imat

ed to

Act

ual M

ultic

ast B

andw

idth

Number of Receivers

"Transit""Stub-1""Stub-2""Stub-3"

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5

6

0 50 100 150 200 250 300

Rat

io o

f Est

imat

ed to

Act

ual M

ultic

ast B

andw

idth

Number of Receivers

"Transit""Stub-1""Stub-2""Stub-3"

Figure 13: Multicast estimation with maximum (left) and average (right) node degree

41

0

0.5

1

1.5

2

2.5

3

0 200 400 600 800 1000

Rat

io o

f Non

-Mig

ratio

n to

Mig

ratio

n C

ore

Per

form

ance

Time

Performance Ratios for Scenario IS

B/W(Amortized)B/W

Avg DelayMax Delay

Core MigrationsCore Evaluations

0

0.5

1

1.5

2

2.5

3

3.5

4

0 200 400 600 800 1000

Rat

io o

f Non

-Mig

ratio

n to

Mig

ratio

n C

ore

Per

form

ance

Time

Performance Ratios for Scenario RC

B/W(Amortized)B/W

Avg DelayMax Delay

Core MigrationsCore Evaluations

0

0.5

1

1.5

2

2.5

3

3.5

0 200 400 600 800 1000

Rat

io o

f Non

-Mig

ratio

n to

Mig

ratio

n C

ore

Per

form

ance

Time

Performance Ratios for Scenario SM

B/W(Amortized)B/W

Avg DelayMax Delay

Core MigrationsCore Evaluations

0

0.5

1

1.5

2

2.5

3

0 200 400 600 800 1000

Rat

io o

f Non

-Mig

ratio

n to

Mig

ratio

n C

ore

Per

form

ance

Time

Performance Ratios for Scenario VC

B/W(Amortized)B/W

Avg DelayMax Delay

Core MigrationsCore Evaluations

Figure 14: Performance Ratios with Bandwidth Migration Criteria