Zahid Irfan & Dr. Asim Karim (Advisor)

(zahidi, akarim @lums.edu.pk)CS-509-Masters of Science (CS) Project

Lahore University of Management Sciences,Lahore, Pakistan

8 May 2004

Approximate Query Processing (AQP) in Data Streams

Acknowledgement

This work is primarily based on the research paper “One-pass wavelets decompositions of data streams” by Gilbert, Muthukrishnan, Strauss and Kotidis, IEEE Trans. Knowledge and Data Engineering May/June, 2003.

Work by Muthukrishnan, Piotr Indyk and of course Johnson-Lindenstrauss.

Introduction Streams and Streaming Models Wavelet Transform & Embedded

Vectors Pseudo-Random Number Generator Implementation Details Test Results Conclusions and Future Work

AQP in Data Streams

Lets solve a puzzle. Guess the missing number in a random sequence of numbers [1…N] without repetition.

Introduction

Space Requirements O (1). Time Complexity O (n).

What about two numbers, three numbers …. and so on…

Data Stream “A sequence of digitally encoded

signals used to represent information in transmission”.

Input stream is the sequence a [i], arrives sequentially item by item.

Data Streams

Applications Networks Data Monitoring.

Applied to Traffic Flow Analysis World Wide Web.

Website hits, statistics etc. Online Transactions Processing

System Large Databases Query Processing

Data Streams Applications

Time Series Comprises value of the same quantity

over different time intervals. Typical examples

Daily closing values of Stock Exchange Traffic at an IP-Link at time intervals.

Stream Models

Cash Register Model Positive updates arrive over period of

time. Typical examples

well … Cash Register Cricket Scores Internet web-site hits or other statistics.

Stream Models

Turnstile Model Fully dynamic model Updates are both negative & positive

e.g. Passengers in an airport

Relative Hardness Turnstile > Cash Register > Time Series “Depends and varies from application to

application”.

Stream Models

Wavelets A mathematical hierarchical tool for

decomposition of signals/ functions. Types of Wavelets

Haar Wavelets Daubechies Wavelets Many more…

Wavelet Transform

Haar Wavelet Example

Resolution Averages Detail CoefficientsD = [2, 2, 0, 2, 3, 5, 4, 4]

[2, 1, 4, 4] [0, -1, -1, 0]

[1.5, 4] [0.5, 0]

[2.75] [-1.25]

----3

2

1

0

Haar Wavelet Decomposition [2.75, -1.25, 0.5, 0, 0, -1, -1, 0]

Wavelet in <,> Space Haar Wavelets can be represented as

the following. Example vector A of N=4, 4 coefficients.

W1= 1/N*[1 1 1 1], W2 = 1/N*[1 1 -1 -1], W3=1/N*[1 -1 1 -1], W4=1/N*[1 1 1 -1]

1st Coefficient = <A,W1>. Average Coefficient 2nd Coefficient = <A,W2>. Detail Coefficient 3rd Coefficient = <A,W3>. Detail Coefficient 4th Coefficient = <A,W4>. Detail Coefficient

Embedding Vectors

Embedding Vectors Any n-point metric space can be

embedded into an O(log2 n) dimensional Euclidean space and L1 metric with 1+є distortion

f(v) = embedding for vector v = < <v, r1>, <v, r1>, … <v, rk>

>

Johnson-Lindenstrauss (JL) Lemma Simply stated <a,b>~<a,rj>*<b,rj>

Where j=1…k, k<<N rj is random vector= {1, -1 with equal

probability} Implications

Represent a vector in RN space in k-dimensional space.

Benefits : Approximate Queries… ??

Johnson-Lindenstruass Lemma

<a,b>~<a,rj>*<b,rj> Approximate queries can be used by

choosing special b. Query ith value choose b=[ 0..010…0] Range Query (i,j) value choose

b=[ 0..01..10…0], where b[x]=1 for i<=x<=j.

What's the catch?? … rj is also size of N. So where to store the random vectors??

AQP & JL-Lemma

Solution to large space over head is generate the random vectors on the fly!!

Such as : for (i=0;i<k;i++) { srand (i);

for (j=0;j<N;j++) {rand (); }

} This solution works but there is a more

elegant solution to this problem. Reed-Muller Codes Extractor.

Pseudo-Random Generator

Reed-Muller Generator The Matrix

values represent RM codes.

RM (x,y)= Replace

01 & 1 -1 we get wavelet basis vectors.

2 mod 2 1X

y

Benefits of Reed-Muller Pseudo Random generator Generated on the fly. Every value is independently

computed without anything to do with the previous values.

Most nearly imitates Wavelet basis vectors.

Hence the sketch contains most of the energy of the signal.

Reed-Muller PR Generator

Things learnt so far There is a way to embed the N data

into k<<N vectors JL-Lemma : <a,b>~<a,r><b,r> Reed-Muller Codes excellent imitators

of both wavelet basis vectors as well as random vectors.

Query Processing is possible thanks to JL- Lemma.

Lessons so far !!

Implementation Details

Implementation Trivia Implemented in Visual C++ 6.0 Design follows Classes and Objects

paradigm Test Results and graphs from MS

Excel

Data Flow Diagram

Dataset

DatasetGenerator

Data StreamGenerator

Wavelets-basedDecomposition

Reed-MullerGenerator

SketchQueryProcessing

Engine

Dataset Generator Synthetic Data Set was generated

using Random Distributions. Normal Distribution

Calling Telephone Number 9497000~9497999 (1000 lines)

Receiving Telephone Number Exponential Distribution

Call Time 0~512 minutes

Data Streamer

The data streaming class offers methods, which help in useful imitation of a real-time data stream by continuously presenting the program with data. Type DataStreamer::getData();

Pseudo Random Generator

2 mod 2

),(1X

yyxG

This class calculates the Reed-Muller based Pseudo-random Numbers.

type PseudoRandomGenerator::getRandom (int X,int Y);

Uses the formula

Data Decomposition

The data is decomposed into a sketch by calculating the dot product of data stream with O (log N) random vectors.

The sketch is stored into Main Memory to be utilized by the query processing engine. Sketch [j]+=Data [i]*Random (i, j);

Here i=(1,N) and j=(1,k);

Query Processing Engine

The Query Processing Engine uses the sketch and a new vector b. Uses the same old JL-Lemma

<a,b>~<a,rj>*<b,rj> Setting various values of b result in

theoretically any sort of query.

Point Query Processing

Point Query Point Query can be processed by

asking for any single value in the whole data stream.

Point Query Algorithm Prepare b[i]={0 for i !=j , 1 for i=j} and

generate <b,r> QuerySketch[j] +=B[i] * Random (i,j); Result = (DataSketch * Query Sketch)/ N

Range Query Processing Range Query

Range Queries specify the low and high between which the query is to be processed.

Even multiple ranges can be specified Query Algorithm

Prepare b[i]={0 for i !=j , 1 for i=j} and generate <b,r>

QuerySketch[j] +=B[i] * Random (i,j); Result = (DataSketch * Query Sketch)/ N

AQP Test

Time Complexity Analysis Query Processing Accuracy with

Data Size Query Processing Accuracy with

Sketch Size

Time Complexity

Time Complexity The following Time complexities were

found to be linear in size of data. Sketching Time Query Processing Time

Time Complexity (Sketching)

Sketching Time versus Data Size (Sketch Size assumed to be log N)

0

20

40

60

80

100

120

10,000.00 100,000.00 1,000,000.00 10,000,000.00

Ske

tch

ing

Tim

e (s

eco

nd

s)

`

Time Complexity (Query)

Querying Time versus Data Size (Sketch Size assumed to be log N)

0

20

40

60

80

100

120

10,000.00 100,000.00 1,000,000.00 10,000,000.00

Qu

eryi

ng

Tim

e (s

eco

nd

s)

`

Accuracy versus Data Size

Data Size versus Accuracy of Query PSNR (dB) versus Data Size

Data Size is increased by Power of 2 Sketch size assumed to be log N

PSNR (dB) versus Data Size

PSNR(dB) versus Data Size (Sketch Size assumed to be log N)

100

105

110

115

120

10.00 1,000,010.00

2,000,010.00

3,000,010.00

4,000,010.00

5,000,010.00

`

Accuracy versus Sketch Size

Accuracy of Query against the Sketch Size. PSNR (dB) versus Sketch Size

Data Size is assumed to be constant = 32768

Sketch Size is varied

PSNR (dB) versus Sketch Size

PSNR(dB) versus Sketch Size(DataSize N=32768)

100110120

130140150

0 20 40 60 80 100 120

Sketch Size

PS

NR

Conclusions

Space Complexity Reduction Prohibitively large data stream in sub-

linear space. Time Complexity Reduction

one-pass data stream algorithm. Scalability to multi-dimensions

Applications and Future Work Data Mining Streams Multimedia & Databases

Trying it with Video coding might be fun or disaster

Graph Theory Problems MST, Matching etc. need to be solved in the

streaming model. Computational Geometry

Earth observation data streams or weather data streams

Solve any problem that can be modeled as a data stream

References S. Acharaya, P.B. Gibbons, V. Poosala and S. Ramaswamy, “Join

Synopsis for Approximate Query Answering”, ACM In Proceedings of the 1999 ACM SIGMOD International Conference on Management of Data, 1999.

J. M. Hellerstein, P. J. Haas and H. J. Wang, “Online Aggregation”, In the Proceedings of 1997 ACM SIGMOD International Conference on Management of Data, 1997.

Y. E. Iaonnidis and V. Poosala, “Histograms-Based Approximation to Set-Valued Query Answers”, In the proceedings of 25th International Conference on Very Large Databases, 1999.

K. Chakrabarti, M. Garofalakis, R. Rastogi and K. Shim, “Approximate Query Processing Using Wavelets”, The Proceedings of the 26th Conference on Very Large Databases, Eygpt, 2000.

F. Olken, “Random Sampling in Databases”, PhD Thesis, University of California at Berkeley, 1993.

A.C. Gilbert, Y. Kotidis, S. Muthukrishnan and M. J. Strass, “One-pass wavelet Decomposition of Data Streams”, IEEE Transactions of Knowledge and Data Engineering, Vol. 15, No.3, May/June 2003.

A. Ta-Shma, D. Zuckerman, and S. Safra, “Extractors from Reed-Muller Codes” In Proceedings of 42nd Annual IEEE Symposium on Foundations of Computer Science, 2001.

Questions & Answers

Thanks to the following for their sincere help in this projectDr. Asim Karim, Dr. Sarmad Abbasi, Dr. Asim Loan, Dr. Sohaib A.

Khan and all my friends speciallyLaeeq Aslam and Aimal Tariq Rextin.