XML und Datenbanken - UZHe4923bf4-f78b-46bc-9275...Lecture "XML and Databases" - Dr. Can Türker 11-20 XMLCAST Target data type can be any (alpha)numeric SQL data-type, including CLOB/BLOB

Chapter 10

Database Systems with XML Support

Oracle12c

DB2 11.0

Microsoft SQL Server 2016

PostgreSQL 9.6

11-2Lecture "XML and Databases" - Dr. Can Türker

Example:XML Schema Document


Example: XML Documents

Gunter SaakeIngo SchmittCan TürkerObjektdatenbanken

Konzepte, Sprachen, Architekturen

International Thomson Publishing

Dieses Lehrbuch ...

Meike KlettkeHolger MeyerXML & Datenbanken

XML-Dokumente effizient speichernund verarbeiten

dpunktMit der wachsenden ...

XML Document book1.xml XML Document book2.xml

Can TürkerSQL:1999 & SQL:2003

Objektrelationales SQL, SQLJ & SQL/XML

dpunktSQL ist ...

XML Document book3.xml

Gunter SaakeKai-Uwe SattlerDatenbanken & JavaJDBC, SQLJ, ODMG & JDOdpunkt2 Das Zusammenspiel ...

XML Document book4.xml


Oracle: Architecture (1)

Figure taken from Oracle® XML Developer's KitProgrammer's Guide 11g Release 1 (11.1) April 2008


Oracle: Architecture (2)

Figure taken from Oracle® XML DBDeveloper’s Guide 11g Release 1 (11.1) October 2007


Mapping XML to Databases

Approaches

– XML column approach: column of type XMLTYPE

– XML table approach: table of type XMLTYPE

Exploit object-relational extensions

– XMLTYPE as predefined object type with SQL/XML functions as methods

– Intermedia-Text package with full-text functionality

– DBMS_XMLDOM package with DOM methods

– DBMS_XMLSCHEMA package with methods for creating and managing XML schemas

– DBMS_XMLGEN package with methods to generate XML from SQL

Storage options

– Text-based (unstructured as CLOB)

– Binary (compact in binary format)

– Schema-based (object-relational)

– Hybrid (semi-structured)

Storage Options (1)



Storage Options (2)




XML Column vs. XML Table

CREATE TABLE ( XMLTYPE)[XMLTYPE [COLUMN] [STORE AS { OBJECT RELATIONAL | CLOB () | BINARY XML ()}][XMLSCHEMA ELEMENT [#]]]

CREATE TABLE OF XMLTYPE [XMLTYPE STORE AS { OBJECT RELATIONAL | CLOB () | BINARY XML() }][XMLSCHEMA ELEMENT [#]]]

schema-based storage text-based storage

Default: BINARY XML

Table with XML column

Table with XML row

binary storage


User-Defined Function to Read XML Documents

CREATE DIRECTORY xmldir AS 'c:\xmldir';GRANT READ ON DIRECTORY xmldir TO PUBLIC WITH GRANT OPTION;

CREATE FUNCTION getDocument(filename VARCHAR2) RETURN CLOBAUTHID CURRENT_USER IS

xbfile BFILE;xclob CLOB;

BEGINxbfile := BFILENAME('xmldir', filename);DBMS_LOB.open(xbfile);DBMS_LOB.createTemporary(xclob, TRUE, DBMS_LOB.session);DBMS_LOB.loadFromFile(xclob, xbfile, DBMS_LOB.getLength(xbfile));DBMS_LOB.close(xbfile);

RETURN xclob;END;/


XML Column

Table with XML column

Insert XML value as XML column value

CREATE TABLE book (content XMLTYPE

) ;

INSERT INTO bookVALUES (XMLTYPE('

Meike KlettkeHolger Meyer ...

'));

INSERT INTO book VALUES (XMLTYPE(getDocument('book1.xml')));


XML Table

Table with XML rows

Insert XML value as XML row value

CREATE TABLE book OF XMLTYPE;

INSERT INTO bookVALUES (XMLTYPE('

Meike KlettkeHolger Meyer ...

'));

INSERT INTO book VALUES (XMLTYPE(getDocument('book1.xml')));


XML Schema Handling

Package DBMS_XMLSCHEMA provides methods to register, compile, generate, and remove XML schemas

DBMS_XMLSCHEMA.registerSchema('schema-url', 'xml-schema');

DBMS_XMLSCHEMA.generateSchema('schema', 'type-name');

DBMS_XMLSCHEMA.deleteSchema('schema-url', deleteoption);

DELETE_RESTRICT CONSTANT NUMBER := 1;DELETE_INVALIDATE CONSTANT NUMBER := 2;DELETE_CASCADE CONSTANT NUMBER := 3;DELETE_CASCADE_FORCE CONSTANT NUMBER := 4;

DBMS_XMLSCHEMA.compileSchema('schema-url');


Schema-based XML Table

Register XML schema

Create XML table based on this schema

Alternative syntax

BEGINDBMS_XMLSCHEMA.registerSchema('book.xsd', getDocument('book.xsd'));

END;/

CREATE TABLE book OF XMLTYPEXMLSCHEMA "book.xsd" ELEMENT "book";

CREATE TABLE book OF XMLTYPEELEMENT "book.xsd#book";


Queries

Support for SQL/XML functions

– XMLQUERY, XMLTABLE, XMLEXISTS, XMLCAST

– XMLELEMENT, XMLFOREST, XMLCONCAT, XMLAGG

– XMLPI, XMLCOMMENT, XMLSERIALIZE, XMLPARSE

Plus additional functions

– XMLROOT

– XMLCOLATTVAL

– XMLCDATA

Full-text search


XMLTYPE

creates XML value

Example:

XMLTYPE( )

Joe

Jim

2000

3500

name salaryemployee SELECT XMLTYPE(' ' || 12 * salary || ' ') AS worker

FROM employee;

24000

42000

WORKER

creates XML value from an XQuery expression

Example:


Datenbanken & JavaGunter SaakeKai-Uwe Sattler

BOOK

ObjektdatenbankenGunter Saake Ingo SchmittCan Türker

XMLQUERY

XMLQUERY( [PASSING [BY VALUE] ]RETURNING CONTENT [NULL ON EMPTY])

:= AS ""

SELECT XMLQUERY('{$i//title/text()}{for $a in $i//author return {$a/text()}}'

PASSING VALUE(b) AS "i" RETURNING CONTENT) AS bookFROM book bWHERE XMLQUERY('$i/book[author[1] = "Gunter Saake"]' PASSING VALUE(b) AS "i"

RETURNING CONTENT) IS NOT NULL;


XMLTABLE

creates SQL table from an XQuery expression. Columns option allows user-defined mapping to the table columns. Without this option, an XML table is created.

Example:

XMLTABLE ([] [PASSING [BY VALUE] ] [COLUMNS ])

:= [PATH ] [DEFAULT]

3-8266-0258-7

3-89864-219-4

3-89864-148-1

ISBN

3-89864-228-3

Objektdatenbanken

SQL:1999 & SQL:2003

XML & Datenbanken

BOOKTITLE

Datenbanken & Java

SELECT t.*FROM book b, XMLTABLE('$i' PASSING VALUE(b) AS "i" COLUMNS

isbn VARCHAR(30) PATH '/book/@isbn', booktitle VARCHAR(50) PATH '/book/title') AS t;

returns true if the XQuery expression yields a non-null value

Example:

XMLEXISTS( [PASSING [BY VALUE] ])

:= AS ""


XMLEXISTS

Objektdatenbanken

Datenbanken & Java

BOOKTITLE

SELECT XMLQuery('$i//title/text()' PASSING VALUE(b) AS "i"RETURNING CONTENT) AS booktitle

FROM book bWHERE XMLEXISTS('$i//author[text() = "Gunter Saake"]' PASSING VALUE(b) AS "i");


XMLCAST

Target data type can be any (alpha)numeric SQL data-type, including CLOB/BLOB plus REF XMLTYPE

Example:

XMLCAST( AS )

Objektdatenbanken

SQL:1999 & SQL:2003

XML & Datenbanken

BOOKTITLE

Datenbanken & Java

SELECT XMLCAST(XMLQUERY('$i//book/title' PASSING VALUE(b) AS "i" RETURNING CONTENT) AS CLOB) AS booktitle

FROM book b;


XMLELEMENT

created XML element

Example:

XMLELEMENT(NAME [, XMLATTRIBUTES()][, ])

:= [AS ]

Joe

Jim

WORKER

Joe

Jim

2000

3500

name salaryemployee SELECT XMLELEMENT(NAME "worker", XMLATTRIBUTES(salary), name) AS worker

FROM employee;

Joe24000

Jim42000

WORKER


XMLFOREST

creates forest of XML elements with optional attributes

Example:

XMLFOREST()

:= [AS ]

Joe

Jim

2000

3500

name salaryemployee SELECT XMLFOREST(name, XMLELEMENT(NAME "euro", 12*salary) AS income) AS worker

FROM employee;


XMLCONCAT

concatenates XML elements to a forest

Example:

XMLCONCAT()

Joe

Jim

2000

3500

name salaryemployee

Joe24000

Jim42000

WORKER

SELECT XMLCONCAT(XMLELEMENT(NAME "employee", name), XMLELEMENT(NAME "euro", 12*salary))

AS workerFROM employee;


XMLAGG

aggregates XML elements

Example:

XMLAGG([ORDER BY ] )

SELECT name, XMLAGG(XMLELEMENT(NAME "salary", salary))AS income

FROM employeeGROUP BY name;

Joe

Jim

2000

3500

name salaryemployee

Jim 5000

2000

35005000

Joe

Jim

NAME INCOME


XMLPARSE, XMLSERIALIZE, XMLCOMMENT & XMLPI

creates XML value (WELLFORMED avoids validity checking)

converts XML value to an SQL string or LOB (in case of schema-based XML values, SHOW DEFAULTS uses the defaults of the given XML schema)

creates XML comment node

creates XML processing instruction node

XMLCOMMENT ()

XMLPI (NAME [, ]))

XMLPARSE({DOCUMENT | CONTENT} [WELLFORMED])

XMLSERIALIZE({DOCUMENT | CONTENT} [AS ] [ENCODING ] [VERSION ] [{SHOW|HIDE} DEFAULTS])


XMLCDATA, XMLROOT & XMLCOLATTVAL

creates XML node of form ]]>

creates XML node (if available the XML attributes version and standalone are set)

creates XML elements of the form Value

XMLCDATA()

XMLROOT(, VERSION {|NO VALUE} [, STANDALONE {YES|NO [VALUE]}])

XMLCOLATTVAL()

:= [AS ]

Index Types

– Functional (value)

index

– Full-text index

– XML index

available for binary or text-based XML columns/tables and creates a set of secondary indexes

Path index on all XML tags and fragments

Value index on the order of the document nodes

Value index on the values of the document nodes

Command to avoid using XML index in query

Command to avoid using XML index in query and performing XQuery optimization

CREATE INDEX xmlfunctionalidx ON book b (XMLCAST(XMLQUERY( '$i//@year' PASSING VALUE(b) AS "i" RETURNING CONTENT) AS INT));


Indexing (1)

CREATE INDEX xmlfulltextidx ON book b (VALUE(b)) INDEXTYPE IS CTXSYS.CONTEXT;

CREATE INDEX xmlidx ON book b (VALUE(b)) INDEXTYPE IS XDB.XMLIndex;

SELECT /*+ NO_XMLINDEX_REWRITE */ ...

SELECT /*+ NO_XML_QUERY_REWRITE */ ...


Indexing (2)

Query exploiting the functional index

Query exploiting the full-text index

Query exploiting the XML index

SELECT XMLQUERY('' PASSING VALUE(b) AS "i" RETURNING CONTENT) AS book

FROM book bWHERE XMLCAST(XMLQUERY('$i//@year' PASSING VALUE(b) AS "i" RETURNING CONTENT)

AS INT) > 2000;

SELECT XMLQUERY('$i//book/title' PASSING VALUE(b) AS "i" RETURNING CONTENT) AS booktitleFROM book bWHERE XMLEXISTS('$i//book/title' PASSING VALUE(b) AS "i");

SELECT XMLQUERY('$i//@isbn' PASSING VALUE(b) AS "i" RETURNING CONTENT) AS isbnFROM book bWHERE CONTAINS(VALUE(b), 'Das book', 0) > 0ORDER BY SCORE(0) DESC;

Manipulation

Persistent updates only possible via SQL UPDATE

Text-based storage supports only complete replacement of XML values ; other storage types also support partial replacement

SQL functions use XPath expressions for selective update

– updateXML – replace XML node

– insertChildXML – insert XML node as child of a given XML element node

– insertChildXMLbefore – insert XML elements as child of a given element node before the child elements of the given type

– insertChildXMLafter – insert XML elements as child of a given element node after the child elements of the given type

– insertXMLbefore – insert XML node before the given node

– insertXMLafter – insert XML node after the given node

– appendChildXML – insert XML node as last child element node of the given node

– deleteXML – remove XML node

XQuery Update Facility is not supported!



Manipulation — UPDATEXML

updates the part of the XML value given by the XPath query

Example:

UPDATEXML(, [, ])

:= ,

...dpunkt ...

... International Thomson Publishing ...

...dpunkt ...

book SYS_NC_ROWINFO$

...dpunkt ...

UPDATE book bSET VALUE(b) = UPDATEXML(VALUE(b), '//publisher[text()="dpunkt"]/@city', 'Zürich');


Manipulation — DELETEXML

deletes a part of the XML value given by the XPath query

Example:

DELETEXML(, [,])

UPDATE book bSET VALUE(b) = DELETEXML(VALUE(b),

'//book[@isbn="3-89864-148-1"]/author[text()="Holger Meyer"]');

...

Meike KlettkeXML & Datenbanken ...

book SYS_NC_ROWINFO$


XML Views

Based on the concept of object views using XMLTYPE as object type

Example: CREATE VIEW dpunktbook OF XMLTYPE WITH OBJECT ID DEFAULT AS

SELECT VALUE(b)

FROM book b

WHERE XMLEXISTS('$i//publisher[text()="dpunkt"]' PASSING VALUE(b) AS "i");


Export SQL Data to XML

Standard SQL-XML mapping using

– Table content mapped to elements

– Column values of each row are mapped to child elements of a element

– Structured (complex) columns are mapped to elements with corresponding child (hierarchically nested) elements

– Collections mapped to list of elements

– Object reference and foreign keys mapped to ID/IDREFs

User-defined transformation from SQL to XML possible via XSLT

… … …

DBMS_XMLGEN.getXML('query')


Summary: Oracle

XML Storage

Model extensional, object-relational

Schema validation possible

Storage Type binary, text-based, schema-based

SQL-XML Mapping Using SQL/XML functions, schema generators, XML views

XML Data Type available

XML Indexing

Value/Functional Index available

Full-text Index available

Path Index available (implicitly as part of XML indexes)

Queries and Manipulation

Queries SQL/XML with XQuery support

Full-text Search using Intermedia-Text package

Manipulation complete replacement using SQL/XML functions in SQL UPDATE (selective SQL UPDATE possible in case of schema-based XML values but XQuery Update Facility not supported)


IBM DB2

Database

File-system

File-system

File-system

FileSystem

DB2

XML DocumentsApplication



After IBM has deprecated the DB2 XML Extender, DB2 supports only an XML column approach

Table with column of type XML

– Binary/text-based storage

– XML schema validation possible


Table with XML Column

Register XML schema

Create table with XML column

Load XML documents from a file

REGISTER XMLSCHEMA 'book.xsd' FROM 'file://C:\XMLDIR\book.xsd' AS book.xsd COMPLETE

IMPORT FROM 'C:\XMLDIR\book.del' OF DEL XML FROM C:\XMLDIRXMLVALIDATE USING XDS DEFAULT book.xsdINSERT INTO book

CREATE TABLE book (Id INT PRIMARY KEY,content XML

)

1, 2, 3, 4,

File book.del


Queries

Use XQuery in SQL via SQL/XML functions

– XMLQUERY, XMLTABLE, XMLEXISTS, XMLCAST

– XMLELEMENT, XMLFOREST, XMLCONCAT, XMLAGG

– XMLPARSE, XMLSERIALIZE, XMLCOMMENT, XMLPI

– XMLTEXT, XMLDOCUMENT, XMLVALIDATE, XSLTRANSFORM

– XMLROW, XMLGROUP

Use SQL in XQuery

– XQUERY db2-fn:xmlcolumn ('t1.xml1')

yields value of column xml1 of table t1 as a node sequence provided the column is of type XML

– XQUERY db2-fn:sqlquery ('SELECT xml1 FROM t1')

yields value of the column xml1 of table t1 as a node sequence provided the column is of type XML

Full-text search requires installation and activation of DB2 Text Extender

creates XML value from an XQuery expression. Difference to Oracle: resulting XML value is never NULL; an empty sequence is delivered as such!

Example:



BOOK

ObjektdatenbankenGunter Saake Ingo SchmittCan Türker

XMLQUERY

XMLQUERY( [PASSING [BY REF] ][RETURNING SEQUENCE [BY REF] [EMPTY ON EMPTY])

:= AS ""

SELECT XMLQUERY('{$i//title/text()}{for $a in $i//author return {$a/text()}}'

PASSING content AS "i") AS bookFROM book bWHERE XMLEXISTS ('$i/book[author[1] = "Gunter Saake"]' PASSING content AS "i");


XMLTABLE

creates SQL table from an XQuery expression. Without the COLUMNS option, a table with an XML column is created. Difference to Oracle: XMLQUERY und XMLTABLE can receive the XML input via reference.

Example:

XMLTABLE ([] [PASSING [BY REF] < xquery-argument-list>] [COLUMNS ])

:= [BY REF] [PATH ] [DEFAULT] }

3-8266-0258-7

3-89864-219-4

3-89864-148-1

ISBN

3-89864-228-3

Objektdatenbanken

SQL:1999 & SQL:2003

XML & Datenbanken

BOOKTITLE

Datenbanken & Java

SELECT t.*FROM book, XMLTABLE('$i' PASSING content AS "i" COLUMNS

isbn VARCHAR(30) PATH '/book/@isbn', booktitle VARCHAR(50) PATH '/book/title') AS t;

return true if the XQuery expression yields a non-null value

Example:

XMLEXISTS( [PASSING [BY REF] ])

:= AS ""


XMLEXISTS

Objektdatenbanken

Datenbanken & Java

BOOKTITLE

SELECT XMLQuery('$i//title/text()' PASSING content AS "i") AS booktitleFROM bookWHERE XMLEXISTS('$i//author[text() = "Gunter Saake"]' PASSING content AS "i");


XMLCAST

Difference to Oracle: SQL data type XML can be used here as target type!

Example:

XMLCAST( AS )

Objektdatenbanken

SQL:1999 & SQL:2003

XML & Datenbanken

BOOKTITLE

Datenbanken & Java

SELECT XMLCAST(XMLQUERY('$i//book/title' PASSING content AS "i") AS CLOB) AS booktitleFROM book;


XMLELEMENT

creates XML-Element (XMLBINARY determines the encoding of the input)

Example:

Joe

Jim

WORKER

Joe

Jim

2000

3500


FROM employee;

XMLELEMENT(NAME [, XMLATTRIBUTES()][, ] [XML-content-option-list>])

:= [AS ]

:= {NULL | EMPTY} ON NULL | XMLBINARY [USING] {BASE64 | HEX}


XMLFOREST

creates forest of XML elements

Example:

XMLFOREST( [XML-content-option-list>])

:= [AS ]

Joe

Jim

2000

3500


FROM employee;

Joe24000

Jim42000

WORKER


XMLCONCAT


Example:

XMLCONCAT()

Joe

Jim

2000

3500

name salaryemployee

Joe24000

Jim42000

WORKER




XMLAGG


Example:




Joe

Jim

2000

3500

name salaryemployee

Jim 5000

2000

35005000

Joe

Jim

NAME INCOME


XMLPARSE, XMLSERIALIZE, XMLCOMMENT, XMLPI

creates XML value. Default option STRIP WHITESPACE eliminates spaces from text nodes which consist only out of spaces

converts XML value to an SQL string. Option INCLUDING XMLDECLARATION additionally inserts the XML declaration . Default: EXCLUDING.



XMLCOMMENT ()

XMLPI (NAME [, ]))

XMLPARSE (DOCUMENT [{STRIP|PRESERVE} WHITESPACE])

XMLSERIALIZE([CONTENT] [AS ] [VERSION ˈ1.0ˈ] [{EXCLUDING|INLUDING} XMLDECLARATION])


XMLTEXT, XMLDOCUMENT, XMLVALIDATE, XSLTRANSFORM

creates XML text node

creates XML document node

returns copy of XML value enriched by information from the schema validation, including the default values

transforms XML value using an XSL style sheet

XMLTEXT()

XMLDOCUMENT(


XMLROW

returns XML document node where each value expression is attached as a child node. Option AS ATTRIBUTES creates attribute nodes instead of child nodes. Option ROW can be used to rename the document element.

XMLROW( [OPTION ]

:= [AS ""]

:= ROW "" | AS ATTRIBUTES

SELECT name, XMLROW(name, salary) AS incomeFROM employee;

Joe

Jim

2000

3500

name salaryemployee

Joe2000

Jim3500

Joe

Jim

NAME INCOME


XMLGROUP

returns XML document element node with child element nodes for each expression. Option AS ATTRIBUTES creates attribute nodes instead of element nodes. Option ROWSET can be used to rename the document element node.

XMLROW( [ORDER BY ] [OPTION ]

:= [AS ""]

:= ROOT "" | ROW "" | AS ATTRIBUTES

SELECT name, XMLGROUP(salary) AS incomeFROM employeeGROUP BY name;

Joe

Jim

2000

3500

name salaryemployee

Jim 5000

2000

35005000

Joe

Jim

NAME INCOME


XQuery returns single-column table with a row for each item of the resulting sequence

XMLQuery returns table with a (possibly empty) sequence for each table row

XQuery vs. XMLQuery

Gunter SaakeKai-Uwe Sattler

Meike Klettke

Holger Meyer

Can Türker

Gunter Saake

Kai-Uwe Sattler


Meike KlettkeHolger Meyer

Can Türker


XQUERY db2-fn:sqlquery('SELECT content FROM book')//book[@year > 2000]/author;

SELECT XMLQUERY('$i//book[@year > 2000]/author' PASSING content as "i") FROM book;

Index Types

– XML index

– Full-text index

Query using the XML index

Query using (and requiring) the full-text index

CREATE INDEX xmlbooktitleidx ON book (content) GENERATE KEY USING XMLPATTERN '/book/title' AS SQL VARCHAR(50);


Indexing

CREATE INDEX xmlfulltextidx FOR TEXT ON book(content);

SELECT XMLQUERY('$i/book/title' PASSING content AS "i") AS booktitleFROM bookWHERE XMLEXISTS('$i/book/title' PASSING content AS "i");

SELECT id, INTEGER(SCORE(content, '(XML OR book)')*1000) AS score FROM book WHERE CONTAINS(content, '(XML OR book)')=1ORDER BY score DESC;

11-53

Only complete replacement of XML values possible via SQL UPDATE

XQuery Update Facility supported to manipulate a copy of an XML value

Manipulation

Jim Bob Meike KlettkeHolger MeyerXML & Datenbanken

XML-Dokumente effizient speichernund verarbeiten

dpunktMit der wachsenden ...

UPDATE book SET content = XMLQUERY('transform copy $pi := $i modify do

insert document { Jim Bob } before $pi//book/author[1] return $pi' PASSING content AS "i")

WHERE XMLEXISTS('$i/book[author/text() = "Holger Meyer"]' PASSING content AS "i");


Summary: IBM DB2

XML Storage

Model extensible, object-relational

Schema Validation possible

Storage Type binary/text-based (XML stored as LOBs)

SQL-XML Mapping XQUERY + XMLQUERY


Indexing

Value Index available (as part of the XML index)

Full-text Index using Text Extender*

Path Index available (as part of the XML index)


Queries SQL/XML with XQuery support

Full-text Search using Text Extender*

Manipulation complete replacement via SQL UPDATE but no selective update; XQuery Update Facility not supported

*Text Extender must explicitly be installed and activated!


Microsoft SQL Server - Architecture

Database

MS SQL Server

SQLOLEDB


Internet Information Server (IIS)

ADOMiddleware



Storage Types

– Native (binary) storage

– Text-based storage as CLOB

– Model-based storage using the EDGE approach (extended SQL)

– Schema-based storage using STORED queries (extended SQL)

Data type XML with XQuery methods

– Query() – evaluates XQuery and returns a value of type XML

– Value() – evaluates XQuery and returns a scalar SQL value

– Exist() – returns true if the XQuery yields a non-empty result

– Modify() – updates a value of type XML

Integrated usage of SQL and XQuery

– Access to SQL data in XQuery using the functions sql:column("columnname") and sql:variable("variablename"), respectively

– Evaluation of XQuery in SQL using XML methods above


Native Storage – Table Definition

Register XML schema

Create table with XML column

Insert XML document from file

CREATE XML SCHEMA COLLECTION bookXSD AS '…'

INSERT INTO bookSELECT 1, xColFROM (SELECT *

FROM OPENROWSET (BULK 'C:\XMLDIR\book1.xml', SINGLE_BLOB) AS xCol) AS R(xCol)

CREATE TABLE book (Id INT PRIMARY KEY,content XML (bookXSD)

)


Native Storage - Indexing

Create primary XML index

– creates clustered index with entries of the form (ID, ORDPATH, TAG, NODETYPE, VALUE, PATH_ID, …)

– needed to created secondary XML indexes

Secondary XML index types:

– Path index (Path, Value)

– Property index (Primary key , Path, Value)

– Value index (Value, Path)

Secondary XML (path plus property) indexes

Full-text index

CREATE PRIMARY XML INDEX contentidx ON book (content)

CREATE XML INDEX contentpathidx ON book (content) USING XML INDEX contentidx FOR PATH CREATE XML INDEX contentpropertyidx ON book (content) USING XML INDEX contentidx FOR PROPERTY

PATH | PROPERTY | VALUE

CREATE UNIQUE INDEX bookkeyidx ON book (Id)CREATE FULLTEXT CATALOG xmldbftcat AS DEFAULTCREATE FULLTEXT INDEX ON book (content) KEY INDEX bookkeyidx


Example: XQuery using SQL data to create new XML values

Native Storage – Queries (1)

SELECT content.query('{//title}{for $a in //author return {$a}}') AS book

FROM bookWHERE content.exist('/book[author[1] = "Gunter Saake"]') = 1

ObjektdatenbankenGunter SaakeIngo SchmittCan Türker


book


Example: Query using the XML index

Example: Query using the full-text index

Native Storage – Queries (2)

SELECT content.query('//author') AS authorsFROM bookWHERE content.exist('/book[author[1] = "Gunter Saake"]') = 1


Gunter SaakeIngo SchmittCan Türker

authors

SELECT id, contentFROM book WHERE CONTAINS(content, '(XML OR book)')


Example: insert new element

Example: update attribute value

Native Storage – Manipulation

UPDATE bookSET content.modify('replace value of (//publisher[. = "dpunkt"]/@city)[1] with "Zürich"')

UPDATE bookSET content.modify('insert Unknown before (//author)[1]')


Model-based Storage using EDGE (1)

SQL extension OPENXML transforms XML values to SQL tables

OPENXML without the WITH clause returns an EDGE table (result see next foil)

DECLARE @xmldoctext VARCHAR(8000) = ' Meike KlettkeHolger MeyerXML & DatenbankenXML-Dokumente effizient speichern und verarbeitendpunkt ...'

DECLARE @hdoc INT

EXEC sp_xml_preparedocument @hdoc OUTPUT, @xmldoctext

SELECT * FROM OPENXML(@hdoc, '', 0)

EXEC sp_xml_removedocument @hdoc


Model-based Storage using EDGE (2)

BIGINT BIGINT INT NVARCHAR NVARCHAR NVARCHAR NVARCHAR BIGINT NTEXT

id parentid nodetype localname prefix namespaceuri datatype prev text

0 NULL 1 book NULL NULL NULL 1 NULL1 NULL 7 xml NULL NULL NULL NULL NULL2 1 2 version NULL NULL NULL NULL NULL3 1 2 encoding NULL NULL NULL NULL NULL5 0 2 isbn NULL NULL NULL NULL NULL6 0 2 year NULL NULL NULL NULL NULL7 0 1 author NULL NULL NULL NULL NULL8 0 1 author NULL NULL NULL 7 NULL9 0 1 title NULL NULL NULL 8 NULL10 0 1 subtitle NULL NULL NULL 9 NULL11 0 1 publisher NULL NULL NULL 10 NULL12 11 2 city NULL NULL NULL NULL NULL13 11 3 #text NULL NULL NULL NULL dpunkt14 0 1 description NULL NULL NULL 11 NULL15 2 3 #text NULL NULL NULL NULL 116 3 3 #text NULL NULL NULL NULL ISO-8859-117 5 3 #text NULL NULL NULL NULL 3-89864-148-118 6 3 #text NULL NULL NULL NULL 200319 7 3 #text NULL NULL NULL NULL Meike Klettke20 8 3 #text NULL NULL NULL NULL Holger Meyer21 9 3 #text NULL NULL NULL NULL XML & Datenbanken22 10 3 #text NULL NULL NULL NULL XML-Dokumente ...23 12 3 #text NULL NULL NULL NULL Heidelberg24 14 3 #text NULL NULL NULL NULL ...


Schema-based Storage using STORED Queries (1)

OPENXML supports custom mappings

OPENXML with the WITH clause creates user-defined table (result see next foil)

DECLARE @xmldoctext VARCHAR(8000) = ' Meike KlettkeHolger MeyerXML & DatenbankenXML-Dokumente effizient speichern und verarbeitendpunkt ...'

DECLARE @hdoc INT

EXEC sp_xml_preparedocument @hdoc OUTPUT, @xmldoctext

SELECT * FROM OpenXML(@hdoc, '//book', 0) WITH (title NVARCHAR(3000) './title',isbn NVARCHAR(15) './@isbn',publisher NVARCHAR(200) './publisher')

SELECT * FROM OpenXML(@hdoc, '//author', 0) WITH (name NVARCHAR(200) '.',isbn NVARCHAR(200) '../@isbn')

EXEC sp_xml_removedocument @hdoc


Schema-based Storage using STORED Queries (2)

Resulting table "book"

Resulting table "author"

title isbn publisher

XML & Datenbanken 3-89864-148-1 dpunkt

name isbn

Meike Klettke 3-89864-148-1

Holger Meyer 3-89864-148-1


Mapping Databases to XML

Variant 1: Standard mapping using SQL SELECT and FOR XML

– FOR XML AUTO: Generates XML element names and reflects foreign keys in hierarchies

– FOR XML RAW: Maps to ROW-XML elements und XML attributes

– FOR XML PATH: Generates complex XML elements using XPath

– FOR XML EXPLICIT: User controls XML mapping (EDGE)

Variant 2: User-defined XML view

– Use XML schema

– Annotate the schema with information over tables and columns

– Access XML view via IIS functionality or ADO (ActiveX Data Objects)


FOR XML AUTO

SELECT name, title, publisher, book.isbnFROM author JOIN book ON (author.isbn = book.isbn)FOR XML AUTO

XML_XXX: NTEXT (CLOB)

SELECT name, title, publisher, book.isbnFROM author JOIN book ON (author.isbn = book.isbn)FOR XML AUTO, ELEMENTS

Meike KlettkeXML & Datenbanken dpunkt3-89864-148-1Holger MeyerXML & Datenbanken dpunkt3-89864-148-1


Option XMLSCHEMA ('uri') generates XML Schema


FOR XML RAW

SELECT name, title, publisher, book.isbnFROM author JOIN book ON (author.isbn = book.isbn)FOR XML ROW


SELECT name, title, publisher, book.isbnFROM author JOIN book ON (author.isbn = book.isbn)FOR XML AUTO, ELEMENTS

Meike KlettkeXML & Datenbanken dpunkt3-89864-148-1Holger MeyerXML & Datenbankendpunkt3-89864-148-1



FOR XML PATH

SELECT book.isbn AS "@isbn", name AS "author", publisher AS "title/@publisher", title AS "title/text()"FROM author JOIN book ON (author.isbn = book.isbn)FOR XML PATH

Meike KlettkeXML & DatenbankenHolger MeyerXML & Datenbanken

XML_XXXX: NTEXT (CLOB)

SELECT book.isbn AS "@isbn", name AS "author", publisher AS "title/@publisher", title AS "title/text()"FROM author JOIN book ON (author.isbn = book.isbn)FOR XML PATH ('books'), ROOT ('bookstore')

Meike KlettkeXML & DatenbankenHolger MeyerXML & Datenbanken

XML_XXXX: NTEXT (CLOB)


User-Defined XML Views

Annotated XML schema

– Namespace: xmlns:sql="urn:schemas-microsoft-com:mapping-schema"

– Annotation sql:relation: Relationship between XML and a table

– Annotation sql:field: Relationship between XML and a table column

– Annotation sql:relationship: Foreign key relationship

XML view access via XPath

– Some XPath construct are not supported

– XPath transformed to corresponding SQL


User-Defined XML Views (Example)

Book Publisher Title isbn

Thomson Objektdatenbanken 3-8266-0258-7

Objektdatenbanken 3-8266-00258-7 Thomson

Definition ofXML structure

Definition of table columns


Manipulation using Updategrams

Schema-based mapping required

– Updates formulated in an XML document

– New namespace: xmlns:updg="urn:schemas-microsoft-com:xml-updategram" Element before: Definition of the state to be updated

Element after: Definition of the new state

– Update operations Insert: empty element

Delete: empty element

Update: both elements are non-empty

Example:

Thomson

International Thomson Publishing


Summary: SQL Server

XML Storage

Model relational

Schema inline DTD or XML schema

Storage Type native: XML column text-based: CLOB columnmodel-based: OPENXMLuser-defined, schema-based: OPENXML

SQL-XML Mapping automatic: FOR XMLuser-defined: XSD annotations


Indexing

Value Index available

Full-text Index available

Path Index available


Queries SQL extensions (query und value are not SQL/XML compatible), XQuery

Full-text Search supported

Manipulation XML method modify or using updategrams; XQuery Update Facility not supported


PostgreSQL - Architecture

Database

PostgreSQL Server



Storage and Queries

XML data type

– Text-based storage

– No schema validation

SQL/XML functions

– XMLEXISTS, XMLELEMENT, XMLFOREST, XMLCONCAT, XMLAGG

– XMLPARSE, XMLSERIALIZE, XMLCOMMENT, XMLPI

– Plus XMLROOT

– No support for XMLQUERY und XMTABLE

– XPath instead XQUERY support

Full-text search


XML

creates XML value

Example:

XML( )

Joe

Jim

2000

3500

name salaryemployee

SELECT XML('' || 12 * salary || '') AS worker

FROM employee;

24000

42000

worker


XPATH (1)

extracts part(s) of an XML value using an XPath expression. Resulting XML sequence represented by a value of the SQL type SET OF.

Example:

XPATH(, )

{"Gunter Saake"}

{"Can Türker"}

{"Meike Klettke"}

firstauthor

{"Gunter Saake"}

Gunter Saake

Can Türker

Meike Klettke

firstauthorunnested

Meike Klettke

SELECT XPATH('//author[1]', content) AS firstauthor, TEXT(UNNEST(XPATH('//author[1]', content))) AS firstauthorunnested

FROM book;


XPATH (2)

{"Meike Klettke""Holger Meyer"

}

authors

{"Gunter Saake""Ingo Schmitt""Can Türker"}

{"Can Türker"}

{"Gunter Saake ""Kai-Uwe Sattler"}

SELECT XML('' || TEXT(XPATH('//author', content))|| '') AS authorsFROM book b;


XPATH (3)

Meike KlettkeHolger Meyer

authors

Gunter SaakeIngo SchmittCan Türker

Can Türker


SELECT XMLELEMENT(NAME "authors", XPATH('//author', content)) AS authorsFROM book b;

returns true if the XQuery expression yields a non-null value

Example:

XMLEXISTS( PASSING [BY REF] )


XMLEXISTS

{Objektdatenbanken}

{Datenbanken & Java}

booktitle

SELECT XPATH('//title/text()', content) AS booktitleFROM bookWHERE XMLEXISTS('//author[text() = "Gunter Saake"]' PASSING content);


XMLELEMENT

creates XML element (XMLBINARY defines encoding)

Example:

Joe

Jim

worker

Joe

Jim

2000

3500


FROM employee;

XMLELEMENT(NAME [, XMLATTRIBUTES()][, ])

:= [AS ]


XMLFOREST

creates forest of XML elements

Example:

XMLFOREST()

:= [AS ]

Joe

Jim

2000

3500


FROM employee;

Joe24000

Jim42000

worker


XMLCONCAT


Example:

XMLCONCAT()

Joe

Jim

2000

3500

name salaryemployee

Joe24000

Jim42000

worker




XMLAGG


Example:




Joe

Jim

2000

3500

name salaryemployee

Jim 5000

2000

35005000

Joe

Jim

name income


XMLPARSE, XMLSERIALIZE, XMLCOMMENT, XMLPI

creates XML value

converts XML value to SQL string



XMLCOMMENT ()

XMLPI (NAME [, ]))

XMLPARSE ({DOCUMENT|CONTENT} )

XMLSERIALIZE({DOCUMENT|CONTENT} AS )

XMLROOT(, VERSION {|NO VALUE} [, STANDALONE {YES|NO [VALUE]}])


XMLROOT

creates XML node (if available XML attribute version and standalone are set too)


Indexing

Index support

– Functional Index

(Value index)

– Full-text index

Query using the functional index

Query using full-text index

CREATE INDEX xmlfunctionalidx ON book USING BTREE (((XPATH('//author/text()', content))[1]::TEXT));

SELECT * FROM bookWHERE (XPATH('//author/text()', content))[1]::TEXT = 'Gunter Saake';

CREATE INDEX xmlfulltextidx ON book USING GIN(to_tsvector('german', content::TEXT));

SELECT *FROM bookWHERE to_tsvector(content::TEXT) @@ plainto_tsquery('german','Gunter');


Export Database to XML

Standard mapping of SQL to XML using the following functions

Variant (xxx for table, query, schema, database):

– xxx_to_xmlschema returns corresponding XML schema

– xxx_to_xml_and_xmlschema returns XML value and corresponding XML schema

table_to_xml(tablename text, nulls boolean, tableforest boolean, targetns text)

query_to_xml(query text, nulls boolean, tableforest boolean, targetns text)

schema_to_xml(schemaname text, nulls boolean, tableforest boolean, targetns text)

database_to_xml(nulls boolean, tableforest boolean, targetns text)


TABLE_TO_XML (1)

exports table to XML

1) null values are mapped to empty XML elements or are ignored

2) document: false maps row to XML sequence; true to an XML document element

3) optional namespace

Example:

TABLE_TO_XML(, , , )

Joe2000.00

Jim3500.00

table_to_xml

SELECT TABLE_TO_XML('employee', true, true, '');


TABLE_TO_XML (2)

Example:

Joe2000.00

Jim3500.00

table_to_xml

SELECT TABLE_TO_XML('employee', true, false, '');


TABLE_TO_XMLSCHEMA

table_to_xmlschema

SELECT TABLE_TO_XMLSCHEMA('employee', true, false, '');


QUERY_TO_XML

Example:

Joe24000.00

Jim42000.00

query_to_xml

SELECT QUERY_TO_XML('SELECT name, 12*salary AS annualsalary FROM employee', true, false, '')


Summary: PostgreSQL

XML Storage

Model extensible, object-relational

Schema not supported

Storage Type text-based

SQL-XML Mapping using SQL/XML functions


Indexing

Value Index available

Full-text Index using tsearch package

Path Index not supported


Queries SQL/XML without XMLQUERY und XMLTABLE but with XPATH support

Full-text Search using tsearch package

Manipulation complete replacement using SQL UPDATE but no selective update; XQuery Update Facility not supported


Comparison – SQL/XML FunctionsTyp SQL:2003 Oracle DB2 MSSQL PostgreSQL

XML ✓ ✓ (XMLTYPE) ✓ ✓ ✓

XMLPARSE ✓ ✓ ✓ — (CAST) ✓

XMLSERIALIZE ✓ ✓ ✓ — (CAST) ✓

XMLCOMMENT ✓ ✓ ✓ — ✓

XMLPI ✓ ✓ ✓ — ✓

XMLDOCUMENT ✓ — ✓ — —

XMLTEXT ✓ — ✓ — —

XMLVALIDATE ✓ — ✓ — —

XMLQUERY ✓ ✓ ✓ — (query) — (XPATH)

XMLTABLE ✓ ✓ ✓ — —

XMLEXISTS ✓ ✓ ✓ — (exist) ✓ (XPATH)

XMLCAST ✓ ✓ ✓ — (CAST) —

XMLELEMENT ✓ ✓ ✓ — ✓

XMLFOREST ✓ ✓ ✓ — ✓

XMLCONCAT ✓ ✓ ✓ — ✓

XMLAGG ✓ ✓ ✓ — ✓

XMLROOT — ✓ — — ✓

XMLCDATA — ✓ — — —

XMLCOLATTVAL — ✓ — — —

XMLROW — — ✓ — —

XMLGROUP — — ✓ — —

FORXML — — — ✓ —

OPENXML — — — ✓ —