Research of P2P overlay in heteromerous network

September 2008, 15(Suppl.): 24–30www.buptjournal.cn/xben

The Journal of ChinaUniversities of Posts and Telecommunications

Research of P2P overlay in heteromerous networkYANG Ya, SONG Jun-de

School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China

AbstractMost of the existing overlays are designed for only one network. This paper analyses the characteristic of heteromerous

network and proposes a tree structure overlay for it. We classify the nodes into two types: common node and hybrid node. Wedesign new routing table and new algorithms for node (especially for hybrid node) joining, leaving and searching. The simulationresult shows that the overlay is well suit for heteromerous network. The algorithm is convergent in large-scale situation.

Keywords P2P, overlay network, heteromerous network

1 Introduction

P2P overlay network is the most important research area in P2P technology. Chord [1], Tapestry [2], Pastry [3] andCAN [4] are the most popular algorithms. They have their own merit in node’s join, left, query and routing table’supdating. But they are all based on one kind of network. Thispaper analyes and research the heteromerous network.

This paper analyes the essential feature of P2Pheteromerous network and proposes an overlay network and algorithms based on heteromerous conditions. According to the node characteristic, they are divided them into two kinds: common node and hybrid node. New algorithms are designed for hybrid node join, left and query. An experimentalassessment validates the practicality of our proposal.

2 Characteristic of heteromerous network

Most of exsiting P2P overlay networks are based on only one kind of network, especially based on IP network. Every node has IP address and they are completely coordinated node. But in two heteromerous network condition (such as network A and network B), IP is not the only protocol. Other protocol like BlueTooth may be used. So we can not simply consider all nodes are completely coordinated peer.

We only anaylse two type of nwetworks in this paper. Thereare three kinds of nodes in this enviroment: nodes can only access network A, nodes can only access network B and nodescan access network A and B both.

The three kinds of node can be logically divided into two

Received date: 18-03-2008Corresponding author:YANG Ya, E-mail: [email protected]

categories: node can only access one kind of network(common node) and node can access two networks (hybridnode). Figure 1 shows the logical distribution and coverage of two heteromerous network.

Note：○ node can only access network A□ node can only access network B

hybrid node can access network A and B both Fig. 1 Logical node distribution and coverage of heteromerousnetwork

As network A and network B are two different networks, their mechanisms and protocols are also not same. Even if there is physical overlap region, nodes can not communicate to each other. But if hybrid node located in overlap region,network A and network B can communicate to each otherbecause hybrid node can achieve the conversion of twonetwork protocols. So the hybrid node is the bridge of the two networks, its load is much higher than common node.

Hybrid node is a special circumstance which can not appearin a single network and it is also the focus of the study.

3 Overlay suit for heteromerous network

BATON [5] and BATON* [6], propose the balanced treestructure for P2P network. They can effectively resolve the

Supplement YANG Ya, et al. / Research of P2P overlay in heteromerous network 25

unbalance between root node and leaf node. Related domesticresearch such as MPPBTree [7] also uses balanced treestructure. Queries are made to an approximate balance of the whole distribution.

Referening the tree structure and combining thecharacteristics of heteromerous network, we design hybridnode as the root node. The two sub-tree of root node arecorresponding to network A and network B. As the nodenumber of two networks may vary greatly, the entire tree may not be a balanced binary tree. But the sub-tree of root node is still maintained as a balanced binary tree.

So Fig. 1 can be converted into a tree structure which is shown in Fig. 2:

Fig. 2 Tree structure overlay

In BATON, node can easily access its neighbor nodes in the same level through horizontal links. But in the condition of heteromerous network, this kind of access can only happen within one sub-tree. If the necessary information can not be found in its sub-tree, we must find the opposite side of the tree. But the nodes on two sides can not communicate to each other, there is no way to access and store information of node in the opposite sub-tree. So how to access the node on other side is the key problem of the whole design.

These accesses across the tree can be completed by rootnode (for it is a hybrid node) because the hybrid node can apperceive all nodes on both sides. But the most obviousdisadvantage is the high load of root node.

If there are several hybrid nodes in the tree (we assume that 10% of the total nodes are hybrid nodes), how to put them in the tree to achieve load sharing and improve the performance?

We propose that putting them into different levels. Theideal situation is that every level has at least one hybrid node.So if the node in left sub-tree wants to access the other one in right sub-tree, it just easily access the hybrid node in the same level at first, and then can access its destination node on the other side. This strategy can reduce the burden on the root node and speed up the routing query.

We assume that the opportunity of query opposite side for

each node is equal. The more of the nodes, the more of their request. So we should deploy more hybrid node on levelswhich contain more nodes to share the whole load further.

4 Routing table

For the introduction of hybrid node, routing table isdiffenent from BATON’s.

Figure 3 represents a typical tree structure. We will follow this example to descirpt the routing table constrution ofdifferent nodes.

Fig. 3 Typical heterogeneous network binary tree with hybrid node

4.1 Routing table of root node

Root node must be a hybrid node. Its routing table is in Table 1.

Table 1 Routing table of root node (number of hybrid node is 1)Left sub-tree Right sub-tree

Level 1 2 3 4 1 2 3Common node 1 2 3 4 1 2 3Hybrid node 0 0 1 0 0 0 1

Rightmost node b e x k c f YHybrid No No Yes No No No Yes

Note: a level=0, No=1, BackupRoot=x, LeftChild=b, RightChild=c,LeftAj=k, RightAdj=y.

We add a backup root node in root node’s routing table. We can use this backup node to replace the root node when root node leaves. And we add the rightmost node information in left sub-tree which is useful when nodes join and quit.

4.2 Routing table of hybrid node

The routing table of hybrid node is shown in Tables 2–4.We choose node x as a example:

Table 2 Left routing table

Node Left child Right child Lowerbound

Upperbound Hybrid

1 j Null Null No2 i p Null No

Table 3 Right routing table

Node Leftchild

Rightchild

Lowerbound

Upperbound Hybrid

26 The Journal of China Universities of Posts and Telecommunications 2008

Table 4 Opposite routing table

Node Leftchild

Rightchild

Lowerbound

Upperbound Hybrid

0 l Null Null No1 m Null Null No2 n Null Null No

In Tables 2–4, x level=3, No=4, root node=a, Hybrid=Yes,Parant=e, LeftChild=q, RightChild=k, LeftAj=q, RightAdj=k.

Compared with BATON, besides adding ‘Hybrid’ flag inleft and right routing table, we add opposite routing table to record node on other side in the same level.

If the hybrid node is the rightmost hybrid node, a table which records all hybrid nodes in its level must be added into its routing table. We can choose a replace hybrid node when this rightmost hybrid node quit.

4.3 Routing table of common node

The routing table of common node are shown in Tables 5 and 6. We choose node h as a example.

Table 5 Left routing table

Node Leftchild

Rightchild

Lowerbound

Upperbound Hybrid

Table 6 Right routing table

Node Leftchild

Rightchild

Lowerbound

Upperbound Hybrid

1 i p Null No2 j Null Null No

In tables 5 and 6, h Level=3, No=1, root node=a, hybrid=No,parant=d, left child=o, right child=null, leftAj=o, rightAdj=d,rightmost hybridnode=x. As the rule in BATON, everycommon node saves the node which the distance betweenthem is 2i. If the node is just a hybrid node, then the ‘Yes’ flag will be filling into ‘hybrid’ field. In addition, the rightmost hybrid node in the same level is added into routing table. So enev if there is no hybrid node in its left and right routing table, it is still keeping the information of the rightmosthybrid node in the same level to transfer query message to opposite side.

5 Algorithms of node join, left and query

5.1 Node join

As common node has only one kind of access ability, it can find nodes in left sub-tree or right sub-tree. And then it can use join method introduced by BATON: determineing where thenew node should join and then actually joining into the network.

Hybrid node is much special for it has two kinds of access capability. Firstly, we should determine which side to add, left side or right side. For the number of nodes in two sides may

be different, we can distribute the hybrid node according to the proportion of sub-tree to the whole tree.

Algorithm of node join is shown as following:JoinLeftSubTree(Node n ) int L = JudgeLevel();

Node a = MostRightNodeOfLevel(L);IF ( NumberOf(a) < 12L− ) THEN{JoinAsRightChild(MostRightNodeOfLevel(L− 1));

Node y＝root.RightTable[L];IF(y!=NULL) THEN

BuildOtherSideTable(GetRightTableFrom(y))ELSE

{ Node z= NearestParentHybridNode(L− 1);IF z !=NULL THEN

BuildOtherSideTable(GetOtherSideTable(z))ELSE

{ p= root.RightTable[MaxLevel]; BuildOtherSideTable(GetRightTableFrom(p);

}}}ELSE Replace(n, a)

The process of node join: for example, we choose hybrid node z to join the tree shown in Fig. 3. We put node z into left sub-tree for the number of left sub-tree is much more than the right’s. Then from the routing table of root node, we cancaculate the 4th level has more nodes than other levels and still has no hybrid node. So the new hybrid node should be put into this level. For the rightmost position in the 4th level is occupied by node k, node k must left and rejoin the tree. Node z replaces node k and becomes the right child of node x.For there are no nodes in the same level on the right side of the tree, node z’s oppssite routing table will copy from node x’s. After the joining of node z, a new tree will be as Fig. 4:

Fig. 4 After the joining of a hybrid node z

The routing table of root node will be upated as Tables 7 and 8.

Table 7 Left sub-tree when number of hybrid Node is 2Level Common node Hybrid node Rightmost node Hybrid

1 1 0 b No2 2 0 e No3 3 1 x Yes4 4 1 z Yes


Table 8 Right sub-tree when number of hybrid node is 1Level Common node Hybrid node Rightmost node Hybrid

1 1 0 c No2 2 0 f No3 3 1 y Yes

In tables 7 and 8, a level=0, No=1, backuproot=x,leftchild=b, rightchild=c, leftAj=z, rightAdj=y.

5.2 Node depature

When node depature the network, we divided them into four kinds: common node, root node, hybrid node on therightmost (leftmost) in its level and hybrid node on other position.

Switch NodePosition(n){Case CommonNode:

LeaveAsBaton(n);NotifyRoot();

Case Root:Node a = ExistOtherHybridNode();IF a != null THEN{ Replace(n,a);

NotifyAllNode(‘RootChanged’,a;}ELSE

NotifyAllNode(‘RootLeft’);Case MostRightOrLeftHybrid:

Node a = OtherNodeOnSameLevel(n);IF a!= null THEN { Replace(n,a);

NotifyRoot();}ELSE{ WaitForNewHybirdNode(PresetPeriod);

ReplaceByHybridNodeOnOtherLevel(n);}Case Others:

LeaveAsCommonNode(n);NotifyMostRightNode(n);NotifyRoot();

}//End of Switch1) Common node depatures: just like depature method

introduced in BATON, but it must notify root node to update its routing table.

2) Root node depatures: if the root node is the last hybrid node in the whole tree, this depature means left sub-tree and right sub-tree can not access to each other and will comebecome 2 sole network. They will combine to form a new network when a new hybrid node appears.

If there is a backup hybrid node in the tree, we can make it to become the root node and notify all nodes in the tree. Thecost is O(N).

3) The depature node is the rightmost and there are other hybrid nodes at the same level. We can chooose one of remain

bybrid nodes to replace the leaving one. And then notify root node to update its routing table.

If the depature node is the last hybrid node in its level, this depature operation will make the left and right sub-tree can not communicate to each other through this node. In order to keep communicating, they must use the upper level or lower level’s hybird node. This action will increase the cost.

One solution is to replace it with a hybrid node in other level. We can adjust them according to the proportion of each level. But this solution cost much because it will make nodes on other levels to update their routing table.

Another solution is do not make any adjust in a scheduledtime, and anticipate a new hybrid node will join the network in this period. The cost is much lower than the first solution’s.But the scheduled time must be set by the number of hybrid nodes and the frequency of their join operation. If one node in this level queries opposite side, the request must be transfer to its child or neigbor. If there is still no hybrid node in its childnode or neigbor node’s level, the request will direct transfer to the root node to avoid too much transfer operation. The whole cost is one depature operation plus one join operation.

4) The leaving hybrid node is not the rightmost node in its level: we regard it as a common node except it will also send update message to its neigbors, rightmost hybrid node and root node.

5.3 Query operation

When nodes in one network queries nodes in other one, all message must be transferred by hybrid nodes. The load ofhybrid node will much higher than common ones’. So query opertaion should carry out in its own sub network. Only if no result can be found, the query message can be transfer toopposite network through hybrid nodes.

When a node receive query message, there are 3 kinds ofsituation:

1) The query value is not within its own range: transfer the query request to other nodes (neigbor nodes, child nodes or adjacent nodes) in its sub-tree.

2) The query value is within its own range and the data is just stored in it: returns the data and stops search.

3) The query value is within its own range but the data is NOT stored in it: this means the data is not stored in the whole sub-tree but may be stored in the opposite side. Thequery message must be transfered to the other side through hybrid nodes or root node. For every node keeps theinformation of hybird nodes or root node in its routing table, it only need 1 step to transfer the query message to them.

SearchOnOtherSide(Node n,Key:V) IF NodeIsRoot(x) THEN{ IF NodeIsInLeftSubTree(n)


THEN{ FOR(i=0;i<x.RightTableLenth();i++)

{ IF(x.RightTable[i]<=V<x.RightTable[i])THEN Send (x.RightTable[i], ‘ExactSearch’,n,V);}j=Random(1, x.RightTableLenth());Send (x.RightTable[j], ‘ExactSearch’,n,V);}}

ELSE //node n in right sub-tree need to query left}ELSE

IF NodeIsInLeftSubTree(n) THEN { Node m=FindFarthestInRightTable(lower<=V);

IF m != NULL THEN Send(m, ‘ExactSearch’,n,V)ELSE Send(root, ‘SearchOnOtherSide’,n,V) }

ELSE//node n in right sub-tree}//End of SearchOnOtherSideThe process of query: Example 1, transfer by hybrid node

(not the root node): Start from node o, query data 74 (see Fig. 5).Data 74 is stored in node n.

Fig. 5 Transfer query message through hybrid node at thesame level

Node o starts a query in left sub-tree. This query will go through o-k-j path according to the algorithm. Node jresponse for range from 73 to 76, but the data 74 is not stored in it. This means data 74 is not stored in whole left sub-tree.Node j will send a new query request to opposite side.

Node j checks its right routing table to find a hybrid node in the same level. If there is one hybrid node, the query will be sent to hybrid node and then be transfered to right sub-tree.In this case, node x is the hybrid node. Node x receive the query message and check its right routing table to find afarthest node whos lower bound less than 73. Node l is the next node, and then node m, n. Node n is the final destination.

Example 2, transfer by root node: start from node m, query data 43. Data 43 is stored in node p (see Fig. 6).

From node m, find data in right sub-tree. Node c cover the range but it doesn’t sotre the data. There are no hybrid nodes in the same level. So node c transfer the query to root node.

Fig. 6 Transfer query message through root node

Root node checks node in its left routing table and find no one can cover data 43, so it choose randomly from its routing table. We suppose node e is selected. Then a new query starts from node e in left sub-tree. The query path is e-d-i-p.

6 Algorithm cost analysis

6.1 Node join

The cost of common node join is almost the same asBATON’s except one additional message send to root node. The cost is still O (logN).

When hybrid node joins the network, the main cost exists in joinleftsubtree (node n). The first step to judge which level should join in cost logN, which is the height of the sub-tree.The 2nd step JoinAsRightChild(MostRightNodeOfLevel ( 1)L − ,we can easily find the rightmost node in level 1L − withinleft routing table of root node. For example, node z joins the network and becomes the child of node x. Node x must notify all neigbor nodes in its routing table to inform its new right child node z. This operation will need at most 2L1 messages ( L1 is the level number of node x).

And these neigbor nodes will send at most 2L2 messages (L2 is the level number of node z) to imform their children. The children nodes will send back 2L2 messages to node z toupdate its routing table.

Node z will send a message to adjcent node a to update its routing table. Node z also need send a message to root node to confirm a new hybrid node joining.

So the total message is 2L1＋2L2＋2L2＋2 <6logN. Thecost is still O (logN).

6.2 Node depature

Common node: just like BATON except a message must be sent to root node. The total message is 2L1 + 2L2 + 3 < 4logn(L2 is the level number of the leaving node, L1 is the level


number of its parent)Root node: when root node quits, it sends messages to all

node on both left and right sub-tree. The cost is O(N).Hybrid node: we suppose hybrid nodes are 10% of total

nodes. So there are 2L×10% hybrid node on level.1) The leaving node is not the rightmost node in its level

and there are other hybrid nodes on the same level. Thisprobability is (2 10% 1) (2 10%)L L× − × . It can be regarded as common node. Besides sending message to its neigbors, it must send message to the rightmost hybrid node in the same level. And it must send message to root node too. Whenhybrid node depatures, messages number is 1 more thancommon’s. It is 2L1 + 2L2 + 4＝4L2+2.

2) The leaving hybrid node is the rightmost node in its level and there are other hybrid nodes on the same level. Thisprobability is 1 (2 10%)L × . We can choose a hybrid node to replace the leaving one. Then root node will be notified for this replace operation. The cost is sum of one time quit and one time join. It is 2L1+2L2+4+2L1+2L2+2L2+2= 10L2+3.

6.3 Query operation

If the detination node and the original node belong to the same network, the query operation can be completed in one sub-tree. It costs O(logN). N is the node number of sub-tree.

If the detination node and the original node belong to the different networks, the cost will from 3 parts:

1) Cost in its own sub-tree: O(logA). A is the node number of its own sub-tree.

2) Message transferred to opposite sub-tree: because every node saves hybrid node or root node information, only 1message is enough.

3) Cost in its opposite sub-tree: O(logB). B is the node number of its opposite sub-tree.

So the total cost is O(logA＋logB).

7 Experimental study

We design experimental enviroment based on BATON and Planetlab [8] to evaluate the performance of our proposedsystem over different scale networks. The total number ofnode N is 1 000 to 10 000. And we suppose hybrid nodes are10% of total nodes.

For a network of size N, 1 000xN data values in the domain of [1, 1 000 000 000) are inserted in batches. For each test, 1000 exact queries are executed, and the average cost is taken.To simulate different sequences of events (order in whichnodes join and leave), the experiments are executed 10 times using 10 different sequences and the average is taken.

7.1 Cost of join and leave operations

The number of message is a little more than BATON’sbecause every join and leave operation must notify root node(see Fig. 7).

Fig. 7 Message number of node join and leave

7.2 Cost of updating routing table

Common node can only join left or right sub-tree. Hybrid node can join them both but it must join one of them. So the cost of two kinds of join operation is nearly equal.

Fig. 8 Message number of updating routing table

Figure 8 shows the cost of node joining is much more than node leaving. And when hybrid nodes leave, it cost a little more than common nodes because bybrid nodes take moreinformation on them.

7.3 Cost of data insert and delete

When data is inserted and deleted, it is only need to operate in one sub-tree. So the cost is a little less than BATON’s (see Fig. 9).

Fig. 9 Message number of data insert and delete

7.4 Cost of exact query

We assume that data insertion is nearly balanced in both


sub-tree.From Fig. 10, we can find that our cost is a little more than

BATON’s. The reason is that if the destination node is not in the same side as the original node, it must transfer querymessage to opposite side through hybrid node or root node and start a new query on opposite sub-tree.

Fig. 10 Message number of exact query

From above results we can aggregate that all operation cost do not increase linear with the linear increasing of nodenumber N. It keeps O(logN). In the larger number of nodes,our algorithm has good convergence.

8 Conclusions

To implement P2P application in heteromerous networkenviroment brings new challenge to overlay network. Hybrid node can be used as common node to provide resourcesharing, and aslo becomes a bridge to the two differentnetworks. This paper focuses on heteromerous enviroment

and hybrid node, proposes new algorithm of node joining, leaving and querying. We test our system on experimentalenviroment. And the result shows that this overlay caneffectively adapt to the situation of heterogeneous network. In large-scale application, the algorithm still has a goodconvergence.

References

1. Karger D, Kaashoek F, Stoica I, et al. Chord: a scalable peer-to-peerlookup service for internet applications. Proceedings of the 2001 ACM SIGCOMM Conference, 2001: 149–160

2. Rowstron, Druschel P. Pastry: scalable, distributed object location and routing for large-scale peer-to-peer systems. Proceedings of the 18thIFIP/ACM International Conference of Distributed Systems Platforms, 2001: 329–350

3. Zhao Y, Kubiatowicz J D, Joseph A D. Tapestry: an infrastructure for fault-tolerant wide-area location and routing. Technical ReportCSD-01-1141, Univ. CaliFornia, Berkeley, CA, Apr. 2001

4. Ratnasamy S, Francis P, Handley M, et al. A scalable contentaddressablenetwork. Proceedings of 2001 ACM Annual Conference of the Special Interest Group on Data Communication, 2001: 161–172

5. Jagadish H V, Ooi B C, Vu Q H. BATON: a balanced tree structure for peer-to-peer networks. Proceedings of the 31st VLDB Conference, 2005:661–672

6. Jagadish H V, Ooi B C, Vu Q H, et al. Speeding up search in peer to peernetworks with a multiway tree structure. Proceedings of the 2006 ACM SIGCOMM Conference, 2006: 1–12

7. Xu L B, Yu K, Wu G X. A P2P semantic routing model with match path and probability balance tree. Journal of Software, 2006, 17(10):2106–2117

8. Chun D, Culler T, Roscoe A, et al. Planetlab: an overlay testbed for broad-coverage services. ACM SIGCOMM Computer Communication Review, 2003, 33(3): 25–32

Documents

Research of P2P overlay in heteromerous network