View
218
Download
0
Category
Tags:
Preview:
Citation preview
Searching In Peer-To-Peer Networks
Chunlin Yang
What’s P2P - Unofficial Definition
• All of the computers in the network are equal
• Each computer functions as a client as well as a server with no administrator
• User on each computer decides what data on their computer will be shared on the network
What’s P2P – Continue
• To share huge volumes of data among peers in the network
• No dedicated servers or hierarchy among the computers in the network
• Examples: Gnutella, Freenet, and Napster
Why P2P• Three internet fundamental assets: information,
bandwidth, storage space
• Increasing amount of information, find useful information in real time is increasingly difficult
• Bandwidth: more have been done, however hot sites like Yahoo, eBay get more and more traffic bottleneck
Why P2P - Continue
• Computing resource: processors speed increase and storage device capacity get bigger, but data center accumulate more and more computation tasks
• P2P networking can greatly improve the utilization of the internet resources
Why P2P - Continue
• Load balance traffic to reduce the peak load on network
• Increase reliability and fault tolerance of the global system
• Fault tolerance for server down time, such as email delivery or slice big email package to small packets and transfer through multi-path.
Basic Searching Algorithms
• Gnutella: BFS
• Freenet: DFS
• Napster: Index Server
Basic Search Algorithm Gnutella
• Each node of the network simultaneously acts as a client as well as a server
• Conducts searching while listening for incoming queries
• Completely decentralized, every node is equal
Basic Searching Algorithm Gnutella - Continue
• A node send query to all its neighbors and each neighbor searches in its own resource and forward the message to all it’s own neighbors
• If a query is satisfied, a response will be sent back to the original requester using the reverse path
Basic Searching Algorithm Gnutella - Continue
• Queries are assigned GUIDs to avoid repetition
• Use a TTL of 7 (about 10000 nodes) to not congest the network
• Problem: can be cyclical, and cause excessive traffic
Basic Searching Algorithm Freenet
• Cooperative file distribution to improve documentation distribution efficiency by sharing bandwidth and disk
• Each file has a unique id and its locations
• Network of equal nodes, each acting as client and server
Basic Searching Algorithm Freenet - Continue
• Information stored on hosts under searchable keys
• Uses a depth-first search with depth limit D. Each node forwards the query to a single neighbor, and waits for a definite response from the neighbor
• If the query was not satisfied, the neighbor forwards the query to another neighbor
Basic Searching Algorithm Freenet - Continue
• If the query was satisfied, the response will be sent back to the query source using the reverse path
• Each node along the path copies data to its own database as well
• More popular information becomes
easier to access
Basic Searching Algorithm Napster
• Centralized server has information of online users and songs location in database for quick search
• Client use peer-to-peer file transfer when a location of a song found from server
• Legal problem: ignores copyright• Problem: same issue for client-server
bottleneck and if the index server down
Improving Search Algorithms In Peer-to-Peer Network
• Iterative Deepening
• Directed BFS
• Local Indices
• Routing Indices
• NEVRLATE
Iterative Deepening
• Multiple breadth-first searches initiated with successively larger depth limits, until the query is satisfied or the
• Maximum depth has been reached.• Example: policy P(a,b,c) first depth a,
second depth b, and third depth c.
Iterative Deepening - Continue
• A Source mode S first initiates a BFS of depth a, When a node at depth a receives and process the query, it will store the query temporarily
• All messages frozen at nodes of a hops from the source
• S receives response messages from nodes that have processes the query
Iterative Deepening - Continue
• After a time period of predefined W, if the query has been satisfied, S does nothing
• Otherwise S starts another round of iteration by initiating a BFS of depth b
• S send a resend message of TTL of a, all node will only forward the resend message until to nodes at a hops
Iterative Deepening - Continue• A node at hop a will drop the resend message
and unfreeze the corresponding query by forwarding the query to all its neighbor with a TTL of b-a
• When message reach to node of hop b, the process continues in a similar fashion
• When process to level c, query will not be frozen, S will not initiate another iteration even the query is not satisfied. Problem ?
Directed BFS
• A node sends query to a subset of its neighbors that could return many results for minimum response time
• A node maintains simple statistics on its neighbors for past queries or the latency of the connection with that neighbor
• From these statistics, some rules can be used to pick up a node to send a query:
Directed BFS - Continue
• Neighbors that has returned highest number of results for previous queries
• Neighbors that returns response message having the lowest average number of hops
• Neighbors that has forward the largest number of message
• Neighbors that has the shortest message queue
Local Indices
• Each node n maintains an index over the data of all nodes within r hops of itself
• r is a system-wide variable known as the radius of the index
• When receive a query, a node can process it on behalf of every node with in r hops, data can be searched on fewer nodes to reduce the cost while keep the satisfaction
Local Indices - Continue• A system-wide policy specifies the depths at
which the query should be processed
• All nodes at the depths not listed in the policy simply forward the query
• Example P(1,5), Only nodes with a depths of 1 and 5 process the query while nodes at other depth just forward the query,
• Reason: Each node has information of its neighbors within 4 hops.
Routing Indices
• To allow a node to select the “best” neighbors to send a query to,
• Routing Indices is a data structure and associated algorithms that, given a query, returns a list of neighbors, ranked according to their goodness for the query,
• The goodness should in general reflect the number of documents in nearby nodes.
Routing Indices - Continue
• Each node has a local index for quickly finding local documents when a query is received.
• Nodes also have a Compound Routing Indices containing:
• The number of documents along each path,• The number of documents on each topic of
interest,
Routing Indices Example
Routing Indices Example
Documents with topics--------------------------------------------------
Path #docs DB N T L A 150 30 20 0 100B 100 20 0 10 30 C 1000 0 300 0 50 D 200 100 0 100 150
Routing Indices - Maintain
• When a connection is established between two nodes, they exchange their routing indices, and update its own indices and send message to its neighbors,
• When a node I disconnected from the network, node D detected, it will remove the row for I, and send a new routing indices of its own to all its neighbors to update.
NEVRLATE
• Network-Efficient Vast Resource Lookup At The Edge
• Directory servers to be organized into a logical 2-dimensional grid, or a set of sets of servers
• Enabling registration in one “horizontal” dimension and
• Lookup in the other “vertical” dimension.
NEVRLATE - Continue
• Each node is a directory server
• Each set of servers, the vertical cloud, can reach each other member of the set
• The set of sets of servers is the entire NEVRLATE network.
NEVRLATE - Continue
NEVRLATE - Continue
• Each host register its resource and location to one node of each set
• When a query comes, only one set need to be searched to get all location containing the satisfied information
• Can also register to two nodes in each sets for fault tolerance
Extension
• Total rank of neighbor’s : weighed sum of all key ranks
• Assumption: high rank nodes should always be better to access or close to resource
• Dominating-set mark process: rule1/rule2, when remove a node from the DS, choose the one with less rank instead of uid
Extension - Continue
• Based on Mark Process (Wu & Li), the connected dominating set nodes will have relatively higher connectivity than non-DS nodes.
• The dominating set nodes need to have resource information and location of resource for their neighbor nodes.
• When search, request will be sent only to DS nodes to reduce cost and traffic while keep satisfactions.
Extension - Continue
• Clustering: when construct a cluster, choose the one with highest rank instead of lowest uid, choose the node with lowest rank as the gateway – low traffic
• Consider not only its own rank but also total ranks of its neighbors
• Max-min ranking: when searching, choose max as well as min for the key index rank
Extension - Continue
• Reason: max could be high traffic, min, low traffic
• Networks are dynamic, resources are dynamic, help to re-rank the networks
• Example: Glades Rd/Palmetto Park Rd
• SW NE
Summary
Recommended