www.neresc.ac.ukwww.neresc.ac.uk

A Peer-to-Peer Database Server based on BitTorrent

A Peer-to-Peer Database Server based on BitTorrent

John ColquhounPaul Watson

John ColquhounPaul Watson

www.neresc.ac.ukwww.neresc.ac.uk 2

Introduction

• If a database server receives queries faster than it can process them, performance becomes unacceptable

• Similar problems have been addressed in the domain of file-sharing by the use of Peer-to-Peer (P2P) technologies

• Can we utilise the combined processing power, disk space and memory of individual clients to reduce the load on the server?

• We examine how P2P techniques could be applied within a database environment and introduce the Wigan P2P database, derived from the BitTorrent file-sharing protocol

• Potential applications in e-Science and e-Commerce

www.neresc.ac.ukwww.neresc.ac.uk 3

System Architecture

Advertise: SELECT *FROM t

Advertise: SELECT id, valueFROM tWHERE t.Id < 100

Query: SELECT id, valueFROM tWHERE t.Id < 100

Advertise: SELECT id, valueFROM tWHERE t.id < 100

Query: SELECT id, valueFROM tWHERE t.Id < 10

Advertise: SELECT id, valueFROM tWHERE t.Id < 10

Query: SELECT id, valueFROM tWHERE t.id < 100

Tracker

www.neresc.ac.ukwww.neresc.ac.uk 4

Implementation

• A simulator of Wigan

• The TPC-H benchmark database was used to evaluate the design

• Identified cases where Wigan offered a performance advantage over a Client-Server database and those areas where it did not

• Experiment results – a busy system where peers submitted one of a choice of five queries, however some submitted an entirely random query over a table of 10,000 tuples

www.neresc.ac.ukwww.neresc.ac.uk 5

Simulator Results

0

20

40

60

80

100

120

P2P Client-Server

Database type

Ave

rag

e re

spo

nse

tim

e (s

)

All peers

Random queries

Repeating queries

www.neresc.ac.ukwww.neresc.ac.uk 6

Current work

• Live Wigan system

• Currently under development, using algorithms developed for the simulator

• Written in Java and uses OGSA-DAI

• Also uses the TPC-H benchmark database

• Experiments are ongoing; in the future the live version will be used to investigate extensions to the Wigan system

www.neresc.ac.ukwww.neresc.ac.uk 7

Live System Results (1)

• Initial experiments with the live system involved comparing Wigan against accessing data directly from SQLServer via JDBC

• One of the TPC-H tables is large (approx 6 million tuples) and hence SQLServer takes some time to evaluate queries on this table, regardless of the result set size

• However, in Wigan, connecting to a peer that already has the query results reduces the response time

• In this experiment, all peers submitted the same query

www.neresc.ac.ukwww.neresc.ac.uk 8

Live System Results (2)

0

5

10

15

20

25

30

35

40

45

Wigan SQL Server

Ave

rag

e re

spo

nse

tim

e (s

)

www.neresc.ac.ukwww.neresc.ac.uk 9

Summary

• We designed, implemented & evaluated the Wigan Peer-to-Peer Database System• Derived from the popular BitTorrent file-sharing protocol

• The first database server that uses P2P to scale over multiple peers

• Simulator shows Wigan can outperform client-server db when:• There are enough peers available to reduce the load on the

seed• There is sufficient overlap between the queries• The system is sufficiently busy so a traditional database

server would become overloaded