Evaluation of Partial Path Queries on XML Data

Evaluation of Partial Path Queries on XML Data

Stefanos Souldatos (NTUA, GREECE)Xiaoying Wu (NJIT, USA)Dimitri Theodoratos (NJIT, USA)Theodore Dalamagas (NTUA, GREECE)Timos Sellis (NTUA, GREECE)

Partial path queries

Query processing

Query evaluation

Experiments

Conclusion


3

Difficulties on Querying XML Data

Creta

theHotel.grtheHotel.gr

CretaCreta

CityCity

CityCity

ChaniaChania

IslandIsland

AthensAthens

IslandIslandLocationLocation

PorosPoros

CityCity

HeraklioHeraklioCenterCenter

Athens Creta

4


Creta

Search problemName: Xiaoying WuPlace: Athens Center, HeraklioPurpose: Sightseeing

Problem:

structural difference

Search problemName: Xiaoying WuPlace: Athens Center, HeraklioPurpose: Sightseeing

Problem:

structural difference

Parthenon (438 BC)

Phaistos’ Disk (1700

BC)


CretaCreta

CityCity

CityCity

ChaniaChania

IslandIsland

AthensAthens


PorosPoros

CityCity


Athens Creta

5


Creta

Search problemName: Theodore DalamagasPlace: IslandsPurpose: Sea sports

Problem:

structural inconsistency

Search problemName: Theodore DalamagasPlace: IslandsPurpose: Sea sports

Problem:

structural inconsistency


CretaCreta

CityCity

CityCity

ChaniaChania

IslandIsland

AthensAthens


PorosPoros

CityCity


Athens Creta

Windsurf

Jet ski

6


Creta

Search problem Name: Dimitri TheodoratosPlace: HeraklioPurpose: HDMS Conference

Problem:

unknown structure

Search problem Name: Dimitri TheodoratosPlace: HeraklioPurpose: HDMS Conference

Problem:

unknown structure


CretaCreta

CityCity

CityCity

ChaniaChania

IslandIsland

AthensAthens


PorosPoros

CityCity


Athens Creta

HDMS 2008

7


Creta


Search problem Name: Stefanos SouldatosPlace: Any islandPurpose: Escape from PhD!

Problem:

multiple sources

Search problem Name: Stefanos SouldatosPlace: Any islandPurpose: Escape from PhD!

Problem:

multiple sources

hotels.grhotels.gr

holidays.grholidays.gr

1400 islands

8


Creta


CretaCreta

CityCity

CityCity

ChaniaChania

IslandIsland

AthensAthens


PorosPoros

CityCity


Athens Creta

Can we use existing query languages (XPath, XQuery) to express our queries?

Can we use existing techniques to evaluate our queries?

9

Path Queries in XPath


CityCity IslandIsland

partial path queries


CityCity

IslandIsland


CityCity

IslandIsland

//theHotel.gr [descendant-or-self::*[ancestor-or-self::City] [ancestor-or-self::Island]]

/theHotel.gr/City//Island//theHotel.gr//City [descendant-or-self::*[ancestor-or-self::Island]]

no structure(keywords)

full structure(path patterns)

10

Partial Path Queries

root node (optional)

query node labelled by “a”

child relationship

descendant relationship

r

aa

b

r

c

da

c

partial path query

11

Partial Path Queries

a

b

r

c

da

cQUERY

PROCESSING a

b

r

c

d

a

partial path query partial path query

in canonical form

QUERYEVALUATION



Query processing

Query evaluation

Experiments

Conclusion

13

Query Processing

a

b

r

c

da

c

1. Full form2. Satisfiability3. Redundant nodes4. Canonical form

14

Query Processing

a

b

r

c

da

c

IR1

INFERENCE RULES(IR1) |- r//ai

(IR2) x/y |- x//y (IR3) x//y, y//z |- x//z(IR4) x/ai, x//bj |- ai//bj(IR5) ai/x, bj//x |- bj//ai(IR6) x/y, y/w, x//z, z//w |- x/z(IR7) x/y, x//z, w/z, w//y |- x/z(IR8) x/y, y/w, x/z |- z/w(IR9) x//y, y//w, x/z |- z//w(IR10) x/y, w/y, w/z |- x/z(IR11) x//y, w/y, w//z |- x//z(IR12) x/y, y/w, z/w |- x/z(IR13) x//y, y//w, z/w |- x//z

x,y,z,w: query nodesai/bj: nodes labelled by a/b


15

Query Processing

a

b

r

c

da

cIR4


INFERENCE RULES(IR1) |- r//ai(IR2) x/y |- x//y (IR3) x//y, y//z |- x//z(IR4) x/ai, x//bj |- ai//bj(IR5) ai/x, bj//x |- bj//ai(IR6) x/y, y/w, x//z, z//w |- x/z(IR7) x/y, x//z, w/z, w//y |- x/z(IR8) x/y, y/w, x/z |- z/w(IR9) x//y, y//w, x/z |- z//w(IR10) x/y, w/y, w/z |- x/z(IR11) x//y, w/y, w//z |- x//z(IR12) x/y, y/w, z/w |- x/z(IR13) x//y, y//w, z/w |- x//z


16

Query Processing

a

b

r

c

da

c

IR4




17

Query Processing

a

b

r

c

d

a

c




18

Query Processing

a

b

r

c

d

a

c


yx

A query is unsatisfiable if its full form contains a trivial

cycle:

19

Query Processing

c

a

b

r

c

d

a


yx

y

yz

y

yx

yz

yx

y

zy

A node y is redundant if one of the following patterns occur:

a)

b)

c)

d)

20

Query Processing

a

b

r

c

d

a


canonical form of satisfiable query

=full form

– IR2 – IR3 – redundant nodes

canonical form of satisfiable query

=full form

– IR2 – IR3 – redundant nodes

The canonical form of a query is a directed acyclic graph

(dag)



Query processing

Query evaluation

Experiments

Conclusion

22

Evaluation Algorithms

Based on PathStack [Bruno et al. ’02]

Produce all possible path queries… Decompose into root-to-leaf paths… PartialMJ: Decompose a spanning tree into paths…

Extending PathStack [Bruno et al. ’02]

PartialPathStack: Produce a topological order of the query nodes and extend PathStack to handle it…

24

Based on PathStack

dc

e

b

r

a

g

fd

c

e

b

r

a

g

fd

c

e

b

r

a

g

f

c

e

b

r

a

d

g

fd

c

e

b

r

a

g

f

1. Producing all possible path queries…

25

Based on PathStack

d

c

e

b

r

a

g

f

c

e

b

r

a

d

g

fd

c

e

b

r

a

g

f

d

c

e

b

r

a

g

f

d

c

b

r

a

e

g

f


26

Based on PathStack

c

e

b

r

a

d

g

f

Problems:

too many queries to evaluate

multiple traversal of the XML tree


27

b

r

a

d

g

f

r

ac

de

Based on PathStack

2. Decomposing into root-to-leaf paths…

b

r

a

de

r

ac

d

g

f

28

Based on PathStack


b

r

a

d

g

f

r

ac

de

b

r

a

de

r

ac

d

g

f

PathStack

29

b

r

a

d

g

f

r

ac

de

Based on PathStack


b

r

a

de

r

ac

d

g

fProblems:

path overlaps

more than one components to evaluate

intermediate results

30

Based on PathStack

PartialMJ. Using a spanning tree…

Remove edges to create a spanning tree

b

r

a

d

g

f

r

acb

r

a

de

31

Based on PathStack


b

r

a

d

g

f

r

acb

r

a

de

c

e

b

r

a

d

g

f

32

Based on PathStack


b

r

a

d

g

f

r

acb

r

a

de

c

e

b

r

a

d

g

f

PathStack

33

Based on PathStack


b

r

a

d

g

f

r

acb

r

a

de

c

e

b

r

a

d

g

f

Join conditions (identity, structural, path)

34

Based on PathStack


b

r

a

d

g

f

r

acb

r

a

de

c

e

b

r

a

d

g

f


35

Based on PathStack


b

r

a

d

g

f

r

acb

r

a

de

c

e

b

r

a

d

g

f


36

Based on PathStack


b

r

a

d

g

f

r

acb

r

a

de

c

e

b

r

a

d

g

f

37

Based on PathStack


c

e

b

r

a

d

g

f Problems:

path overlaps

more than one components to evaluate

intermediate results

38

Extending PathStack

dc

e

b

r

a

g

f

PartialPathStack. Employ a topological order…

c

e

b

r

a

d

g

f

39

Extending PathStack

PartialPathStack. Employ a topological order…

c

e

b

r

a

d

g

fd

c

e

b

r

a

g

f

PartialPathStack

40

PartialPathStack Examplequerytree

Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

db

r

a

c esink

nodes

results

41

PartialPathStack Exampletree

Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r

sink nodes

results

42


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

results

43


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1 b1

sink nodes

results

44


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1 b1 d1

sink nodes

results

45


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1 b1 d1 c1

sink nodes

results

46


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

results

OUTPUT!!!

47


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

results

OUTPUT!!!

48


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

results

OUTPUT!!!

49


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

results

OUTPUT!!!

50


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

results

OUTPUT!!!

51


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

OUTPUT!!!

results

ra1b1d1c1e1

52


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

results

ra1b1d1c1e1

d2

53


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

d2 c2

OUTPUT!!!

results

ra1b1d1c1e1

54


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

d2 c2

OUTPUT!!!

results

ra1b1d1c1e1

55


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

d2 c2

OUTPUT!!!

results

ra1b1d1c1e1

56


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

d2 c2

OUTPUT!!!

results

ra1b1d1c1e1

57


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

d2 c2

OUTPUT!!!

results

ra1b1d1c1e1

58


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

d2 c2

OUTPUT!!!

results

ra1b1d1c1e1

ra1b1d1c2e1

59


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

d2 c2

results

ra1b1d1c1e1

ra1b1d1c2e1

60


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

d2

results

ra1b1d1c1e1

ra1b1d1c2e1

e2

OUTPUT!!!

61


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

d2

results

ra1b1d1c1e1

ra1b1d1c2e1

e2

OUTPUT!!!

62


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

d2

results

ra1b1d1c1e1

ra1b1d1c2e1

e2

OUTPUT!!!

63


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

d2

results

ra1b1d1c1e1

ra1b1d1c2e1

e2

OUTPUT!!!

64


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

d2

results

ra1b1d1c1e1

ra1b1d1c2e1

e2

OUTPUT!!!

65


Sr Sa Sb Sd Sc Se

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

query

db

r

a

c e

r a1

sink nodes

b1 d1 c1 e1

d2

results

ra1b1d1c1e1

ra1b1d1c2e1

ra1b1d1c1e2

e2

OUTPUT!!!

66

PartialPathStack Examplequerytree

d2

e1

c1

d1

c2 e2

d1

b1

a1

r

db

r

a

c e

results

ra1b1d1c1e1

ra1b1d1c2e1

ra1b1d1c1e2

only one component to evaluate

no intermediate results

67

Evaluation Algorithms

Problems:

Algorithm:

Many queries /

components to evaluate

Path overlaps

Intermediate results

Produce all path queries…

Decompose into paths…

PartialMJ (spanning tree)

PartialPathStack

68

PartialPathStack vs PathStack

PathStack• Path queries• Indegree = 1• Outdegree = 1• O(input + output)

d

c

e

b

r

a

g

f

d

c

e

b

r

a

g

f

PartialPathStack• Partial path queries• Indegree > 1• Outdegree > 1• O(input*indegree + output*outdegree)



Query processing

Query evaluation

Experiments

Conclusion

70

Queries Used in the Experiments

d

c

e

b

r

a

f

d

c

eb

r

a

f d

e

r

a

fc

b

d

e

r

a

fc

b

Q1/Q5 Q2/Q6 Q3/Q7 Q4/Q8

71

Experiment 1

Execution time on Treebank…2.5 million nodes

72

Experiment 1

path queries


73

Experiment 1

too many results


74

Experiment 1

2.5 million nodes(IBM AlphaWorks

XML generator)

Execution time on Synthetic data…

75

Experiment 2

PartialMJ

PartialPathStack

PartialPathStack

PartialMJ

PartialPathStack

PartialMJ

Q2

Q3 Q7

Execution time varying the size of the XML tree…(1 - 3 million nodes)



Query processing

Query evaluation

Experiments

Conclusion

77

Conclusion

Evaluation Containment

Heuristics for

Containment

Partial Path Queries CIKM ’07 SSDBM ’06 CIKM ’06

Queries with repetitions

? SSDBM ’06 CIKM ’06

Partial Tree Queries ? SSDBM ’06 CIKM ’06

Questions?


Query processing

Query evaluation

Experiments

Conclusion

Documents

Evaluation of Partial Path Queries on XML Data