Magic Sets

7/31/2019 Magic Sets

1/24

Magic Sets [Bancilhon et al., 1985]

March 31, 2011

(STI Innsbruck) March 31, 2011 1 / 22
http://find/


2/24

Table of contents

1 Introduction

2 Magic sets

3 Magic sets algorithm

http://find/


3/24

Introduction

Magic sets

Optimization technique for recursive Datalog.

Combines benefits of both top-down and bottom-up processing of logic.

http://find/


4/24

Logic programming

An exampleA logic program for cousins at the same generation:

r1 : sg(X, X).

r2 : sg(X, Y) : par(X, X1), par(Y, Y1), sg(X1, Y1).

Assume a database contains a parenthood relation par, and par(U, V) meansthat V is a parent of U

sg(U, V) means that U and V are cousins.

U and V have a common ancestor W.

There are lines of descent from W to U and V covering the same number ofgenerations.r1 says that anyone is his own cousin, i.e., U = V = W, descent is throughzero generations.

http://find/


5/24

Top-down (backward chaining)

Top-down (backward chaining)

Query: sg(a, W) , i.e., find all the cousins of a particular individual

r2 : sg(X, Y) : par(X, X1), sg(X1, Y1), par(Y, Y1).

Will consider each parent of a, say b and c.

Recursively, all bs cousins, then all cs cousins, then all the children of b andc.

Seeks to discover all proofs rather than just all answers, runtime may beexponential.

http://find/


6/24

Bottom-up (forward chaining)

Bottom-up (forward chaining)

r1 : sg(X, X).

r2 : sg(X, Y) : par(X, X1), par(Y, Y1), sg(X1, Y1).

Start by assuming only the facts in the database, i.e., the par relation, sg isempty initially.

The first pass, r2 yields nothing, r1 yields the set of all (X, X) pairs.

The second pass, r2 gives all the facts sg(U, V) where U and V have acommon parent.

The third pass, r2 gives all the facts sg(U, V) where U and V have acommon grand parent.

and so on..

Converges in time that is polynomial

http://find/http://goback/


7/24

Limitations of the top-down and bottom-up approaches

Limitations of top-down and bottom-up approaches

Top-down approach may repeatedly establish the same fact.Bottom-up approach may generate many facts that do not contribute to theanswer to the query.



8/24

Magic sets

Magic sets

A desirable property: only discovers relevantfacts.



9/24

Magic sets

Magic sets

A desirable property: only discovers relevantfacts.

Relevant facts

The relevant sg facts for query sg(a, W).

The cone of a is all the ancestors of a.

Generate all the facts sg(b, c) such that b is inthe cone; c may or may not be in the cone.

Start with the pair sg(b, b) for every b in thecone.

Apply r2 bottom-up.

Facts like sg(d, f) do not need to be generated; d,f both not in the cone.

We do want to establish facts like sg(e, f), ifpar(e, b) and par(f, c).



10/24

Magic sets

Magic predicates

Introduce magic predicates to represent the bound arguments in queriesthat a top-down search would ask.

For a goal sg(a,W), the first argument of sg is either a or the parent of avalue that appears.

r2 : sg(X,

Y) :

par(X,

X1),

par(Y,

Y1),

sg(X1,

Y1).

Goal sg(b, W), where b is an ancestor of a.

Applying r2: par(b, X1), par(W, Y1), sg(X1, Y1)

The first argument X1 of the new goal sg, is a parent of b and therefore

ancestor of a.In rules:

r3 : magic(a).

r4 : magic(U) : magic(V), par(V, U).



11/24

Magic sets

Rewriting the rules

The possible values of the first argument (assuming top-down evaluation ofsg) used to filter bottom-up evaluation.

After rewriting the rules r1 and r2 to insist that the first argument of sg arein the magic set:

r5 : sg(X, X) : magic(X).

r6 : sg(X, Y) : magic(X), par(X, X1), par(Y, Y1), sg(X1, Y1).



12/24

Magic sets

Rule/goal graphThe nodes corresponds to the predicatesymbols and rules.

Nodes have adornments indicating whicharguments of predicates are bound and which

variables of rules are bound.Arcs lead to a predicate (goal) node from eachrule with the predicate on the left side.

Arcs lead to a rule node node from each of theliterals ( = goal) on the right side.

r1 : sg(X, X).

r2 : sg(X, Y) : par(X, X1), par(Y, Y1),

sg(X1, Y1).



13/24

Magic sets

Sideways information passing

Evaluating goals in any order (e.g.par(X, X1), sg(X1, Y1), par(Y, Y1)).

A variable that appears in a goal may beregarded as bound for all subsequent goals.

r1 : sg(X, X).


sg(X1, Y1).



14/24

Magic sets algorithm

AlgorithmTransform the rules into equivalent rules that can be implemented moreefficiently bottom-up.

Modification of rules by magic sets.

INPUT: A portion of a rule/goal graph (one of the goal node is the query

node).OUTPUT: A modification of the rules (perhaps no change) that is designedto permit efficient evaluation of the rules bottom-up.

MethodFor each node g whose predicate is recursive, construct a relation magicg.

Which is the set of tuples of values (for those arguments that the adornmentsof g says are bound) that can actually contribute to the answer to the query.



15/24


Method: In summery

r : p : L, q, R.g : L, h, R.

For a query, g with bound value a:

magicg(a).

magich(X) : L, magicg(Y).

g : magicg(Y), L, h, R.



16/24


MethodLet us assume that the rules are all linear recursive, and a rule is written as:

r : p: L, q, R.

q is the recursive predicate in the body.L is the set of literals that are evaluated before q.

R is the set of literals that are evaluated after q.

MethodLet g be a goal node corresponding to predicate p with some adornment.Then g has a predecessor rule node corresponding to rule r.

And the rule node for r has a predecessor h corresponding to the predicate q.


M i l i h


17/24


Method: Magic set rules

Suppose, to answer the query, g with bound values b (i.e. b is in magicg)needs to be evaluated.

Y, the list of arguments in r that appear in arguments of p that g says arebound (so b is a tuple of bindings for Y).

X, the list of arguments in r that appear in arguments of q that h says are

bound.

Conclude, tuple c of bindings for X is in magich:

if the bindings in c are consistent with binding for bplus some set of legal bindings for the variables in literals of L

Formally, magich(X) : L, magicg(Y).

Create such a magic set rule for each rule that has a recursive

predicate in its body, and for each pair of g and h.

magicg(a). Here g is the goal node corresponding to the query, a is the tupleof constants provided by the query.


M i l i h


18/24


Method: Rewriting the original rules

First, replace each recursive predicate by new predicates corresponding to itsnodes in the rule/goal graph.

Second, replace a rule such as r : p :

L,

q,

R. as:Replacing p and q by all possible pairs of goal nodes g and h, related to eachother and to p and q as above,Adding literal magicg(Y) to the body; Y is the list of variables appearing inthe argument positions of p that g asserts are bound.In rule:

g :

magicg(Y),

L,

h,

R.


M i l i h


19/24


Cousins at the same generation

Recursive call to sg occurs in reverse order:

r1 : sg(X, X).


sg(Y1, X1).

The query: sg(a, W).


M i t l ith


20/24


g is sgbf and h is sgfb

The original rules:r1 : sg(X, X).

r7 : sg(X,

Y) :

par(X,

X1),

par(Y,

Y1),

sg(Y1, X1).

Order for sideways information passing:par(X, X1), sg(Y1, X1), par(Y, Y1).

Let us use magic1 for node sgbf and magic2

for sgfb

.


M i t l ith


21/24


g is sgbf and h is sgfb

The original rules:r1 : sg(X, X).r7 : sg(X, Y) : par(X, X1), par(Y, Y1),sg(Y1, X1).

Magic set rules (applyingmagich(X) : L, magicg(Y).):

magic2(X1) : par(X, X1), magic1(X).

Applying g : magicg(Y), L, h, R., the rulesfor sg are modified to:

sgbf(X, X) : magic1(X).sgbf(X, Y) : magic1(X), par(X, X1),par(Y, Y1), sgfb(Y1, X1).




22/24


Similarly for g is sgfb and h is sgbfOrder for sideways information passing:par(Y, Y1), sg(Y1, X1), par(X, X1).

The magic set rule: magic1(Y1) : par(Y, Y1), magic2(Y).

The rules for sg are modified to:

sgfb(X, X) : magic2(X).sgfb(X, Y) : magic2(X), par(X, X1), par(Y, Y1), sg

bf(Y1, X1).

For query sg(a, W), query node is sgbf

The basic magic set rule:magic1(a).




23/24


All together

magic1(a).magic1(U) : par(V, U), magic2(V).magic2(U) : par(V, U), magic1(V).

sgbf(X, X) : magic1(X).sgbf(X, Y) : magic1(X), par(X, X1), par(Y, Y1), sg

fb(Y1, X1).

sgfb(X, X) : magic2(X).

sgfb(X, Y) :

magic2(X), par(X, X1), par(Y, Y1), sgbf(Y1, X1).


Bancilhon, F., Maier, D., Sagiv, Y., and Ullman, J. (1985).


24/24

, , , , S g , , U , J ( 985)Magic sets and other strange ways to implement logic programs (extendedabstract).In Proceedings of the fifth ACM SIGACT-SIGMOD symposium on Principlesof database systems, pages 115. ACM.


Documents

Magic Sets