View
247
Download
0
Category
Tags:
Preview:
Citation preview
Integrating Keyword Search into XML Query Processing
Presentation By:Alex Kremer
Ariel Rosenblatt
XML Query Language (XML-QL)Extending XML-QL with Keyword SearchExtended XML-QL Implementation Using
RDBMS
Bibliography(well-formed, but invalid) Bibliography Article elements are from different
sources Same information, but using
different XML Scheme / DTDs (Document Type Descriptors)
XML Queries XML is becoming the Data Storage
and Exchange Format of choice in many applications
Handling of XML data requires a rich and powerful Query Language Allow for querying the content and
structure of an XML document Varying or unknown structures can
make formulating queries very difficult
XML Queries: Why not SQL/OQL XML is not rigidly structured In XML the schema can exists with the
data as tag names If DTD is not available, schema is build while
the document is parsed Missing elements or multiple
occurrences of the same element This flexibility is crucial for EDI
(Electronic Document Interchange)
XML Query Requirements W3C Working Group Goals:
Support different usage scenarios Define data model + query operators Define query language syntax Interoperate with other XML working
groups
XML Query Requirements: Usage Scenarios Human-readable documents
Manuals, Books, Articles Data-oriented documents
XML representation of: Database data, Object data, …
XML representation might be either: Physical or Virtual
XML Query Requirements: Usage Scenarios Contd. Mixed model documents:
Hybrid of document oriented and data-oriented
Catalogues, Patient health records, … Administrative data:
Configuration files, User profiles, Administrative logs
XML Query Requirements: Usage Scenarios Contd. Filtering streams:
On-line: filtering / extracting / transforming / routing, of XML data streams
Logs of email messages, Network packets, Stock market data, Newswire feeds
Document Object Model (DOM) Perform queries on DOM structures to
return sets of nodes that meet the specified criteria
XML Query Requirements: Usage Scenarios Contd. Multiple syntactic environments for
queries embedded in: URL, XML, JSP or ASP pages, a string
in a general-purpose programming language
…
XML Query Requirements: Interoperability Results must be returned in a DOM
compatible manner XPath (used in XPointer and XSLT)
XPath expressibility and search facilities should be used in query syntax
Usage of XML Schema (XSDL) and/or DTD
XML Query Languages: Proposals to W3C XQL (heavily based on XPath) XML-QL
XML-QL It is declarative It is “relational complete”; in
particular it can express joins Simple enough to enable
optimizations It can extract data from existing
XML documents and construct new documents (transformations)
XML-QL: Syntax
WHERE clause specifies how to filter data from the input XML dataset
CONSTRUCT clause specifies how to assemble the query results in XML
WHERE ( xml-pattern [ ELEMENT_AS $elem_var ] )*
IN url, ( predicate )*
CONSTRUCT xml-pattern | $variable
XML-QL: Example #1
Yields the following result
WHERE <article>
<author><name>$N</name></author>
<title>$T</title>
<article> ELEMENT_AS $E IN “bibliography.xml”,
$N like *Florescu*
CONSTRUCT <result> $E </result>
XML-QL Explained:The Data Model A Set of XML documents must be
represented (XML Data Set) XML elements in a dataset can be
partitioned according to their types Need to represent information in a
loss-less manner (original data set must be recreatable from the representation)
XML-QL Explained:Data Model Representation
ID00
ID01
article
ID02
“20000815” “1” “http:…”
id linkdate
title
ID03
source
“XML Query…” “W3C”
ID04
article
ID05
“3” “http:…”
title
ID06
author
“A Query…”
“Daniela Florescu”
ID07
author
id link
name name“Alon L…”
ID08
ID09
“4” “http:…”
title
ID10
author
“Integr…”
“Daniela Florescu”
ID12
author
id link
name
“Donald K…”
article
ID11 ID13
name
ID14article
id
“6”
“@article…Florescu…}”
Bibliography:
XML-QL Explained:Data Model Representation
Dataset D is represented as a graph GD: Nodes:
Element e node Ne uniquely labeled IDe
Data value v leaf Lv uniquely labeled v Edges:
(Ne , Ne’) labeled with the tag of e’, if e’ is directly nested within e (<e><e’>…</e’></e>)
(Ne , Lv) labeled with “”, if v is directly contained within e (<e>v</e>)
(Ne , Lv) labeled with attribute name a, if v is the value of atribute a of element e (<e a=“v”>…</e>)
XML-QL Explained:Query Processing An XML pattern can be also modeled by a
graph Some labels in the graph are now variables
The result of the evaluation of query q on the input D, is: Each mapping from the graph Gq to the graph
GD which preservers the constant labels This mapping induces a substitution of the
variables in the query on the set of constant values
XML-QL Explained:A Query Graph for Example #1
WHERE <article>
<author><name>$N</name></author>
<title>$T</title>
<article> ELEMENT_AS $E IN “bibliography.xml”,
$N like *Florescu*
CONSTRUCT <result> $E </result>
title author
$T
“*Florescu*”
name
article
XML-QL Explained:Query Processing, Example #1
ID00
ID01
article
ID02
“20000815” “1” “http:…”
id linkdate
title
ID03
source
“XML Query…” “W3C”
ID04
article
ID05
“3” “http:…”
title
ID06
author
“A Query…”
“Daniela Florescu”
ID07
author
id link
name name“Alon L…”
ID08
ID09
“4” “http:…”
title
ID10
author
“Integr…”
“Daniela Florescu”
ID12
author
id link
name
“Donald K…”
article
ID11 ID13
name
ID014article
id
“6”
“@article…Florescu…}”
Bibliography:
title author
$T
“*Florescu*”
name
article
No <author>
No <name>“name” is an attribute
Match! Add ID08 to Results$E = ID08$T = “Integrating Keyword Search…”
XML-QL: Advanced QueriesExample #2 (More Florescu)
We now look for articles where the author name can be also an
attribute!, result
WHERE <article> <*><author><name>$N</name></author></*> <title>$T</title>
<article> ELEMENT_AS $E IN “bibliography.xml”,$N like *Florescu*
CONSTRUCT <result> $E </result>unionWHERE <article>
<*><author><_ name=$N></_></author></*> <title>$T</title>
<article> ELEMENT_AS $E IN “bibliography.xml”,$N like *Florescu*
CONSTRUCT <result> $E </result>
Back
XML-QL: Disadvantages We need to know the XML
structure in order to query We can still perform more efficient
queries, where we get all the information available, but
These queries can easily grow very complex as seen previously
XML-QL: Keyword Search Extension Addition of special predicate called
contains to XML-QL Tests the existence of a given word
within an XML element Works on partially known or not-
known XML structure Allows querying several XML
documents with different structure
Extended XML-QL: The contains Predicate The contains predicate has 4 arguments,
($E, word, depth, location): $E is an XML element variable Word – the word we are searching for Depth is an integer expression limiting the
depth at which the word is found within the element
Location is a boolean expression over the set of constants,
{tag_name, attribute_name, content, attribute_value}
Extended XML-QL:Example #3 We can use the extended XML-QL
to formulate a query which yields the same result as Example #2
WHERE <article>
<author></author> ELEMENT_AS $A
<title>$T</title>
<article> ELEMENT_AS $E IN “bibliography.xml”,
contains($A, “Florescu”, 3,
content or attribute_value)
CONSTRUCT <result> $E </result>
Back
Extended XML-QL:Example #4
WHERE <article></article>
ELEMENT_AS $E IN “bibliography.xml”,
contains($E, “Florescu”, 3, any)
CONSTRUCT <result> $E </result>
We are able to query unstructured data (full text search) within a set of articles:
Yielding the result
Implementing the contains predicate The authors suggest an
implementation of the XML-QL extension on top of a Commercial RDBMS: Oracle 8, IBM DB2, MS-SQL, …
Implementation Using RDBMS Reasons:
Easy to implement an extended XML query processor
Universally available RDBMS allow to mix XML data and
other (relational data) Very good performance over large
volumes of data
Relational Support forFull-text Indexing Use of extended Inverted Files to
implement: The contains predicate Finding of relevant XML data sources
(URLs) in a distributed environment We will use RDBMS to implement
Inverted Files
Inverting Files For our needs the inverted file will
contain tuples of the following format: <word, elID, depth, location>
Examples from bibliography.xml: <“article”, elID01, 0, tag> <“id”, elID01, 1, attr> <“Requirements”, elID01, 2, value>
Storing Inverted Files in RDBMS: Unique Internal elIDs Unique element IDs are modeled
as records containing: Document locators (URLs) Element locators within the document
Using absolute positions (start, end) Using unique identifiers specified by DTD
(explicit id attribute)
Why not XPointer?
Storing Inverted Files in RDBMS: Unique elID Schemes After normalization the authors
propose the following scheme: Elements(elID, docid, start_pos,
end_pos, type, id_val) Documents(docid, URL)
From this point elID can be used as an internal key used for faster processing
Storing Inverted Files in RDBMS Natural way – using scheme:
contains(elID, word, depth, location) Huge! We partition it into word tables
for each keyword <word> in the dataset: <word>(elID, depth, location)
Virtually all IR (Information Retrieval) systems use partitioning by word
Back
Storing Inverted Files in RDBMS: Further Partitioning We use further partitioning to optimize
the query processing: The type (tag) of the element is usually known
at predicate evaluation time by looking at the XML pattern of the query
We further partition the individual <word> tables by the type of the element they are in: <word>-<type>(elID, depth, location)
Table examples: Name-author, Florescu-name bibliography.xml
Back
Implementation: Extended XML-QL Query Processing Two Ways:
Replicating the whole XML data in an RDBMS
XML-QL processing is entirely performed in an RDBMS
Distributed XML Query Processing only index (contains) is stored in an
RDBMS
Replicating the XML Data in an RDBMS The binary table approach:
For each type (tag name or attribute name), a table is built with the following scheme:
<type>(parent, element, value) The parent element contains the element of
type <type> element is null if a <type> has no sub-
elements or if <type> is an attribute name (in that case we are usually interested in the value)
bibliography.xml
Replicating the XML Data in an RDBMS: XML-QL Queries Every XML-QL query can be
translated into an equivalent SQL query
The SQL query will process the binary tables of the replicated XML Data
Back
XML-QL to SQL: Example #5 (from Example #1)
WHERE <article> <author><name>$N</name></author> <title>$T</title>
<article> ELEMENT_AS $E IN “bibliography.xml”,$N like *Florescu*
CONSTRUCT <result> $E </result>
SELECT article.elementFROM article, author, name, title WHERE article.element = author.parent AND
author.element = name.parent AND article.element = title.parent AND /* title exists */
name.value like “Florescu”
Extended XML-QL to SQL: Keyword Search Processing the contains predicate
involves usage of inverted file tables
The word-type table has to be joined with the previous result
The word-type table is the resulting table of the word by type partitioning
Extended XML-QL to SQL: Example #6
SELECT title.valueFROM article, author, name, title, Florescu-author, Integrating-titleWHERE article.element = author.parent AND author.element = Florescu-author.elID AND
article.element = title.parent AND title.element = Integrating-title.elID
WHERE <article> <author></author> ELEMENT_AS $A <title>$Ttext</title> ELEMENT_AS $T
<article> ELEMENT_AS $E IN “bibliography.xml”,contains($A, “Florescu”, 3, any)contains($T, “Integrating”, 3, any)
CONSTRUCT <result> $Ttext </result>
Distributed XML Query Processing XML data can be indexed in RDBMS, but The XML data cannot be stored in the
RDBMS Reasons: volume (entire www) or legal
The mediator (query interface): Uses inverted files in RDBMS, but Accesses the data sources to compute the full
query result (Expensive!) Load relevant documents/elements into RDBMS
and process the query as described before (XML-QL to SQL)
Distributed XML Query Processing: Elements Retrieval Use of Inverted Files for the retrieval
of relevant documents/elements: Evaluate contains predicates to
disqualify irrelevant elements Further reduce the dataset needed to
process the remaining basic XML-QL query
This is an optimization since retrieval of remote data is expensive
Load the relevant documents/elements
Distributed XML Query Processing: Reducing Retrieval
WHERE <article>
<author><name>$N</name></author>
<title>$T</title>
<article> ELEMENT_AS $E IN “bibliography.xml”,
$T like *XML*
CONSTRUCT <result> $N </result>
Get the intersection of elIDs sets from: author-article name-article title-article XML-article
Conclusions XML-QL can be extended to support keyword search Use of RDBMS:
Inverted Files can be stored an queried using an RDBMS XML data itself can be replicated and queried in the RDBMS Keyword search and overall XML query processing can be
carried out very efficiently Data structure influence:
The more structure is known, the faster a query will be executed
Totally unstructured queries can be executed very fast The more structure is known, the higher is the quality of
the query results
Recommended