46
SciQL Bridging the Gap between Science and Relational DBMS Ying Zhang, Martin Kersten, Niels Nes CWI Amsterdam Dec. 03, 2012

SciQL, Bridging the Gap between Science and Relational DBMS

Embed Size (px)

DESCRIPTION

by Ying Zhang, Martin Kersten, Niels Nes CWI Amsterdam Dec. 03, 2012

Citation preview

Page 1: SciQL, Bridging the Gap between Science and Relational DBMS

Virtual(Observatory(Infrastructure(for(Earth(Observation(Data

Project(acronym:( TELEIOSGrant(agreement(no:(( FP7(8 257662

SciQLBridging the Gap between Science

and Relational DBMS

Ying Zhang, Martin Kersten, Niels NesCWI Amsterdam

Dec. 03, 2012

Page 2: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Who needs arrays anyway?

2

Scientists love arrays: FITS, MSEED, NETCDF, HDF5…but also use: lists, tables, XML, ...

Seismology – 1-D waveforms, 3-D spatial dataAstronomy – temporal ordered rastersClimate simulation – temporal ordered gridRemote sensing – images of 2-D or higherGenomics – ordered DNA stringsForensics – images, strings, graphs

Page 3: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Arrays In DBMS

3

Research issues already in the 80’s

Algebraic frameworks

(S)RAM, AQL, AML, ...

SQL language extension

RasQL, AQuery, SRQL, …

a notion of order

SQL:1999, SQL:2003

collection type, C-style arrays

aggregation functions over arrays

OODB, multi-dimensional DBMS, Sequence DBMS, ...

The Longhorn Array Database

RasDaMan

Store large arrays in chunks as BLOBs

Array query (RasQL) optimisation on top of DBMS

Known to work up to 12 TBs!

PostgreSQL 8.1

SciDB

Array DBMS from scratch

Overlapping chunks for parallel execution

Page 4: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

SQL is difficult

Appropriate array denotations?

Functional complete operation set?

DBMSs are slow

Too much overhead

Size limitations (due to BLOB representations)?

Existing foreign files?

Scale

...

What Not RDBMS?

4

Page 5: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

SciQL

An array query language based on SQL:2003

Lower the entrance fee to RDBMSs

Reveal optimisation opportunities

Distinguish features (Kersten et al., AD’11; Zhang et al. IDEAS2011):

Arrays and tables as first class citizens of DBMSs

Seamless integration of relational and array paradigms

Named dimensions with constraints

Flexible structure-based grouping

5

Page 6: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

CREATE ARRAY A1 (x INT DIMENSION [0:1:4], y INT DIMENSION [0:1:4],v FLOAT DEFAULT 0.0);

Array Definitions

6

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0x

y

0 1

0

1

2 3

23

Page 7: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

CREATE ARRAY A1 (x INT DIMENSION [0:1:4], y INT DIMENSION [0:1:4],v FLOAT DEFAULT 0.0);

Array Definitions

6

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0x

y

0 1

0

1

2 3

23

dimensions,any scalar data type

Page 8: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

CREATE ARRAY A1 (x INT DIMENSION [0:1:4], y INT DIMENSION [0:1:4],v FLOAT DEFAULT 0.0);

Array Definitions

7

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0x

y

0 1

0

1

2 3

23

dimensional range:[(start|∗) : (step|∗) : (stop|∗)]

Page 9: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

CREATE ARRAY A1 (x INT DIMENSION [0:1:4], y INT DIMENSION [0:1:4],v FLOAT DEFAULT 0.0);

Array Definitions

8

cell values,any column data type

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0x

y

0 1

0

1

2 3

23

Page 10: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

CREATE ARRAY A1 (x INT DIMENSION [0:1:4], y INT DIMENSION [0:1:4],v FLOAT DEFAULT 0.0);

Array Definitions

9

fixed array

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0x

y

0 1

0

1

2 3

23

Page 11: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

null

Array Definitions

10

CREATE ARRAY A2 (x INT DIMENSION[1:*],y INT DIMENSION,v FLOAT DEFAULT 0.0);

Unbounded array

x

y

0 1

0

1

2 3

23

Page 12: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

null

Array Definitions

10

CREATE ARRAY A2 (x INT DIMENSION[1:*],y INT DIMENSION,v FLOAT DEFAULT 0.0);

INSERT INTO A2 VALUES(1,0,5.5), (1,1,0.4), (2,2,4.5);

Unbounded array

x

y

0 1

0

1

2 3

23

Page 13: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

null0.0 4.50.4 0.0

5.5 0.0

null

null

null

Array Definitions

10

CREATE ARRAY A2 (x INT DIMENSION[1:*],y INT DIMENSION,v FLOAT DEFAULT 0.0);

INSERT INTO A2 VALUES(1,0,5.5), (1,1,0.4), (2,2,4.5);

Unbounded array

x

y

0 1

0

1

2 3

23

Page 14: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

null0.0 4.50.4 0.0

5.5 0.0

null

null

null

Array Definitions

10

CREATE ARRAY A2 (x INT DIMENSION[1:*],y INT DIMENSION,v FLOAT DEFAULT 0.0);

INSERT INTO A2 VALUES(1,0,5.5), (1,1,0.4), (2,2,4.5);

Unbounded array

x

y

0 1

0

1

2 3

23

current range

Page 15: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Array & Table Coercions

11

SELECT x, y, v FROM A1;

x y v0 0 0.00 1 0.00 2 0.00 3 0.01 0 0.01 1 0.01 2 0.01 3 0.02 0 0.02 1 0.02 2 0.02 3 0.03 0 0.03 1 0.03 2 0.03 3 0.0

CREATE ARRAY A1 (x INT DIMENSION[0:1:4],y INT DIMENSION[0:1:4],v FLOAT DEFAULT 0.0);

0.0 0.0 0.0 0.00.0 0.0 0.0 0.00.0 0.0 0.0 0.00.0 0.0 0.0 0.0

x

y

0 1

0

1

2 3

2

3

full materialisation!

Page 16: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Array & Table Coercions

12

SELECT [x], [y], v FROM T2;

CREATE TABLE T2 (x INT, y INT,v FLOAT DEFAULT 0.0);

INSERT INTO T2 VALUES(1,0,5.5), (1,1,0.4),(2,2,4.5), (1,1,1.3);

x y v1 0 5.51 1 0.42 2 4.51 1 1.3

An unbounded array

dimension ranges derived from the minimal bounding box

cells values from the table or the column default

duplicates are overwritten arbitrarily

dimension qualifiers: ‘[’, ‘]’

0.0 4.50.4 0.0

5.5 0.0

null

null

null

null

x

y

0 1

0

1

2 3

23

Page 17: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

0.0 null 0.0 0.0

0.0 null 0.0 0.0

0.0 null 0.0 0.0

0.0 null 0.0 0.0x

y

0 1

0

1

2 3

23

Array Modifications

13

DELETE FROM A1 WHERE x = 1;

creates holes in the array

Page 18: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.5 0.5 0.5 0.5

0.0 0.0 0.0 0.0x

y

0 1

0

1

2 3

23

Array Modifications

14

UPDATE A1 SET v = 0.5 WHERE y = 1;

INSERT INTO A1 VALUES(0,1,0.5), (1,1,0.5), (2,1,0.5), (3,1,0.5);

set/change cell valuesoverwrite existing values

Page 19: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Array Views

15

CREATE VIEW A2 AS SELECT [x], [y], v FROM A1[1:3][1:3];

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.5 0.5 0.5 0.5

0.0 0.0 0.0 0.0x

y

0 1

0

1

2 3

23

array view A2

Page 20: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Array Views

16

CREATE ARRAY VIEW A3 (x INT DIMENSION [-1:1:5],y INT DIMENSION [-1:1:5],w FLOAT DEFAULT 0.0) AS

SELECT [x], [y], v FROM A1;

-1.0 -1.0 -1.0 -1.0

-1.0 -1.0 -1.0 -1.0

-1.0 0.5 0.5 0.5

-1.0 -1.0 -1.0 -1.0x

y

0 1

0

1

2 3

2

3

Page 21: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

0.0 0.0 0.0 0.0 0.0 0.0

0.0 -1.0 -1.0 -1.0 -1.0 0.0

0.0 -1.0 -1.0 -1.0 -1.0 0.0

0.0 -1.0 0.5 0.5 0.5 0.0

0.0 -1.0 -1.0 -1.0 -1.0 0.0

0.0 0.0 0.0 0.0 0.0 0.00 1

0

1

2 3

2

3

x

y

-1

4

-1 4

Array Views

17

CREATE ARRAY VIEW A3 (x INT DIMENSION [-1:1:5],y INT DIMENSION [-1:1:5],w FLOAT DEFAULT 0.0) AS

SELECT [x], [y], v FROM A1;

Page 22: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Array Tiling

18

SELECT [x], [y], AVG(v) FROM A1 GROUP BY A1[x:x+2][y:y+2];

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.5 0.5 0.5

0.0 0.0 0.0 0.0x

y

0 1

0

1

2 3

2

3

Page 23: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Array Tiling

18

SELECT [x], [y], AVG(v) FROM A1 GROUP BY A1[x:x+2][y:y+2];

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.5 0.5 0.5

0.0 0.0 0.0 0.0x

y

0 1

0

1

2 3

2

3

Anchor point:A1[x][y]

Page 24: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Array Tiling

18

SELECT [x], [y], AVG(v) FROM A1 GROUP BY A1[x:x+2][y:y+2];

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.5 0.5 0.5

0.0 0.0 0.0 0.0x

y

0 1

0

1

2 3

2

3

Anchor point:A1[x][y]

Page 25: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Array Tiling

18

SELECT [x], [y], AVG(v) FROM A1 GROUP BY A1[x:x+2][y:y+2];

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.5 0.5 0.5

0.0 0.0 0.0 0.0x

y

0 1

0

1

2 3

2

3

Anchor point:A1[x][y]

Page 26: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Array Tiling

18

SELECT [x], [y], AVG(v) FROM A1 GROUP BY A1[x:x+2][y:y+2];

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.5 0.5 0.5

0.0 0.0 0.0 0.0x

y

0 1

0

1

2 3

2

3

Anchor point:A1[x][y]

Page 27: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Array Tiling

18

SELECT [x], [y], AVG(v) FROM A1 GROUP BY A1[x:x+2][y:y+2];

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.5 0.5 0.5

0.0 0.0 0.0 0.0x

y

0 1

0

1

2 3

2

3

Anchor point:A1[x][y]

Page 28: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Array Tiling

19

SELECT [x], [y], AVG(v) FROM A1 GROUP BY A1[x:x+2][y:y+2];

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.125 0.25 0.25 0.25

0.125 0.25 0.25 0.25x

y

0 1

0

1

2 3

2

3

Page 29: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Array Tiling

20

SELECT [x], [y], AVG(v) FROM A1 GROUP BY A1[x:x+2][y:y+2] HAVING x mod 2 = 0 AND y mod 2 = 0;

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.5 0.5 0.5

0.0 0.0 0.0 0.0x

y

0 1

0

1

2 3

2

3

Anchor point:A1[x][y]

Page 30: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Array Tiling

21

SELECT [x], [y], AVG(v) FROM A1 GROUP BY A1[x-1][y], A1[x][y-1],

A1[x][y], A1[x+1][y], A1[x][y+1];

0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0

0.0 0.5 0.5 0.5

0.0 0.0 0.0 0.0x

y

0 1

0

1

2 3

2

3

Page 31: SciQL, Bridging the Gap between Science and Relational DBMS

the LOw Frequency ARray radio telescope

Page 32: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

LOFAR Catalogue

meridian

zone (Gray et al. 2006)

0 1 2 3 ...

90

...

357 358 359

-2

0-1

12

-90...

time

frequency

IQUV

t1 t2 t3 t4 ...

ν1

ν2

ν3

ν4

...

ra DOUBLE,decl DOUBLE,ra_err DOUBLE,decl_err DOUBLE,flux DOUBLE,...

23

Page 33: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

LOFAR Catalogue

CREATE ARRAY LOFARsrc ( zone INT DIMENSION[-90:1:91], mrdn INT DIMENSION[0:1:360], ts TIMESTAMP DIMENSION, freq INT DIMENSION[30:10:241], id INT DIMENSION[0:1:*], stks CHAR(1) DIMENSION CHECK(stks=`I' OR stks=`Q' OR stks=`U' OR stks=`V'), ra DOUBLE, decl DOUBLE, ra_err DOUBLE, decl_err DOUBLE, flux DOUBLE, ...);

meridian

zone (Gray et al. 2006)

0 1 2 3 ...

90

...

357 358 359

-2

0-1

12

-90...

time

frequency

IQUV

t1 t2 t3 t4 ...

ν1

ν2

ν3

ν4

...

ra DOUBLE,decl DOUBLE,ra_err DOUBLE,decl_err DOUBLE,flux DOUBLE,...

23

Page 34: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

LOFAR Use Case

Similarity of the flux of a LOFAR source at frequencies 30 MHz and 200 MHz

cross-correlation of two time series

0 1 2 3 ...

90

...

357 358 359

-2

0-1

12

-90...

time

frequency

IQUV

t1 t2 t3 t4 ...

ν1

ν2

ν3

ν4

...

ra DOUBLE,decl DOUBLE,ra_err DOUBLE,decl_err DOUBLE,flux DOUBLE,...

meridian

zone (Gray et al. 2006)

24

Page 35: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Cross-Correlation

Cridx -3 -2 -1 0 1 2

Crval

Gidx 0 1 2

Gval 1 5 7

Fidx 0 1 2 3

Fval 4 3 6 2

25

Page 36: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Cross-Correlation

Cridx -3 -2 -1 0 1 2

Crval 2

Gidx 0 1 2

Gval 1 5 7

Fidx 0 1 2 3

Fval 4 3 6 2

Cr.idx = -3

F [3 : 4]

G [0 : 1]

26

Page 37: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Cross-Correlation

Cridx -3 -2 -1 0 1 2

Crval 2 16

Gidx 0 1 2

Gval 1 5 7

Fidx 0 1 2 3

Fval 4 3 6 2

Cr.idx = -2

F [2 : 4]

G [0 : 2]

27

Page 38: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Cross-Correlation

Cridx -3 -2 -1 0 1 2

Crval 2 16 47

Gidx 0 1 2

Gval 1 5 7

Fidx 0 1 2 3

Fval 4 3 6 2

Cr.idx = -1

F [1 : 4]

G [0 : 3]

28

Page 39: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Cross-Correlation

Cridx -3 -2 -1 0 1 2

Crval 2 16 47 61

Gidx 0 1 2

Gval 1 5 7

Fidx 0 1 2 3

Fval 4 3 6 2

Cr.idx = 0

F [0 : 3]

G [0 : 3]

29

Page 40: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Cross-Correlation

Cridx -3 -2 -1 0 1 2

Crval 2 16 47 61 41

Gidx 0 1 2

Gval 1 5 7

Fidx 0 1 2 3

Fval 4 3 6 2

Cr.idx = 1

F [0 : 2]

G [1 : 3]

30

Page 41: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Cross-Correlation

Cridx -3 -2 -1 0 1 2

Crval 2 16 47 61 41 28

Gidx 0 1 2

Gval 1 5 7

Fidx 0 1 2 3

Fval 4 3 6 2

Cr.idx = 2

F [0 : 1]

G [2 : 3]

31

Page 42: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Cross-Correlation

Cridx -3 -2 -1 0 1 2

Crval 2 16 47 61 41 28

Gidx 0 1 2

Gval 1 5 7

Cr.idx = 2

F [0 : 1]

G [2 : 3]

Fidx 0 1 2 3

Fval 4 3 6 2

32

Page 43: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

LOFAR Use Case

0 1 2 3 ...

90

...

357 358 359

-2

0-1

12

-90...

time

frequency

IQUV

t1 t2 t3 t4 ...

ν1

ν2

ν3

ν4

...

DECLARE fcnt INT, gcnt INT;SET fcnt = SELECT COUNT(*) FROM LOFARsrc[*][*][*][30][11][‘I’];SET gcnt = SELECT COUNT(*) FROM LOFARsrc[*][*][*][200][11][‘I’];

CREATE ARRAY VIEW F (idx INT DIMENSION[0:1:fcnt], flux DOUBLE DEFAULT 0.0) AS SELECT flux FROM LOFARsrc[*][*][*][30][11][‘I’];

CREATE ARRAY VIEW G (idx INT DIMENSION[0:1:gcnt], val DOUBLE DEFAULT 0.0) AS SELECT flux FROM LOFARsrc[*][*][*][200][11][‘I’];

CREATE ARRAY CrCorr30_200 (idx INT DIMENSION[-fcnt+1:1:gcnt], val DOUBLE DEFAULT 0.0);INSERT INTO CrCorr SELECT SUM(F.flux * G.flux) FROM F, G, CrCorr30_200 AS C GROUP BY F[MAX(0, -C.idx) : MIN(fcnt, gcnt-C.idx)], G[MAX(0, C.idx) : MIN(gcnt, fcnt+C.idx)];

ra DOUBLE,decl DOUBLE,ra_err DOUBLE,decl_err DOUBLE,flux DOUBLE,...

meridian

zone (Gray et al. 2006)

33

Page 44: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

LOFAR Use Case

0 1 2 3 ...

90

...

357 358 359

-2

0-1

12

-90...

time

frequency

IQUV

t1 t2 t3 t4 ...

ν1

ν2

ν3

ν4

...

DECLARE fcnt INT, gcnt INT;SET fcnt = SELECT COUNT(*) FROM LOFARsrc[*][*][*][30][11][‘I’];SET gcnt = SELECT COUNT(*) FROM LOFARsrc[*][*][*][200][11][‘I’];

CREATE ARRAY VIEW F (idx INT DIMENSION[0:1:fcnt], flux DOUBLE DEFAULT 0.0) AS SELECT flux FROM LOFARsrc[*][*][*][30][11][‘I’];

CREATE ARRAY VIEW G (idx INT DIMENSION[0:1:gcnt], val DOUBLE DEFAULT 0.0) AS SELECT flux FROM LOFARsrc[*][*][*][200][11][‘I’];

CREATE ARRAY CrCorr30_200 (idx INT DIMENSION[-fcnt+1:1:gcnt], val DOUBLE DEFAULT 0.0);INSERT INTO CrCorr SELECT SUM(F.flux * G.flux) FROM F, G, CrCorr30_200 AS C GROUP BY F[MAX(0, -C.idx) : MIN(fcnt, gcnt-C.idx)], G[MAX(0, C.idx) : MIN(gcnt, fcnt+C.idx)];

ra DOUBLE,decl DOUBLE,ra_err DOUBLE,decl_err DOUBLE,flux DOUBLE,...

meridian retrieve the time series

zone (Gray et al. 2006)

33

Page 45: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

LOFAR Use Case

0 1 2 3 ...

90

...

357 358 359

-2

0-1

12

-90...

time

frequency

IQUV

t1 t2 t3 t4 ...

ν1

ν2

ν3

ν4

...

ra DOUBLE,decl DOUBLE,ra_err DOUBLE,decl_err DOUBLE,flux DOUBLE,...

meridian

zone

dynamic grouping for every iteration

DECLARE fcnt INT, gcnt INT;SET fcnt = SELECT COUNT(*) FROM LOFARsrc[*][*][*][30][11][‘I’];SET gcnt = SELECT COUNT(*) FROM LOFARsrc[*][*][*][200][11][‘I’];

CREATE ARRAY VIEW F (idx INT DIMENSION[0:1:fcnt], flux DOUBLE DEFAULT 0.0) AS SELECT flux FROM LOFARsrc[*][*][*][30][11][‘I’];

CREATE ARRAY VIEW G (idx INT DIMENSION[0:1:gcnt], val DOUBLE DEFAULT 0.0) AS SELECT flux FROM LOFARsrc[*][*][*][200][11][‘I’];

CREATE ARRAY CrCorr30_200 (idx INT DIMENSION[-fcnt+1:1:gcnt], val DOUBLE DEFAULT 0.0);INSERT INTO CrCorr SELECT SUM(F.flux * G.flux) FROM F, G, CrCorr30_200 AS C GROUP BY F[MAX(0, -C.idx) : MIN(fcnt, gcnt-C.idx)], G[MAX(0, C.idx) : MIN(gcnt, fcnt+C.idx)];

34

Page 46: SciQL, Bridging the Gap between Science and Relational DBMS

2012-12-03 Ying Zhang

Discussion

35

Your use cases?

Your problems?

What do you need?

How can SciQL help?