XGI: A Graphic Interface for XQuery Creation and XML Schema Visualization

XGI: A Graphic Interface for XQuery XGI: A Graphic Interface for XQuery Creation and XML Schema Creation and XML Schema

Visualization Visualization

Xiang LiXiang Li

University of WashingtonUniversity of Washington

Biomedical and Health InformaticsBiomedical and Health Informatics

Master Thesis Committee:Master Thesis Committee:

James Brinkley, John GennariJames Brinkley, John Gennari

OutlineOutline

Needs for graphically querying XML Needs for graphically querying XML Existing graphical query solutionsExisting graphical query solutions Requirements for a successful systemRequirements for a successful system My approach and XQuery Graphic My approach and XQuery Graphic

Interface (XGI) system overviewInterface (XGI) system overview Using XGI to build a XQueryUsing XGI to build a XQuery System validation and evaluationSystem validation and evaluation Discussion and conclusionsDiscussion and conclusions

OutlineOutline

Needs for graphically querying XMLNeeds for graphically querying XML Existing graphical query solutionsExisting graphical query solutions Requirements for a successful systemRequirements for a successful system My approach and the XQuery Graphic Interface My approach and the XQuery Graphic Interface

(XGI) system overview(XGI) system overview Using XGI to build a XQueryUsing XGI to build a XQuery System validation and evaluationSystem validation and evaluation Discussion and conclusionsDiscussion and conclusions

XML and XQueryXML and XQuery

XMLXML De facto standard language for data exchange and De facto standard language for data exchange and

representation in biomedical researchrepresentation in biomedical research Semistructured data modelSemistructured data model Wide-spread adoption and many data sources in XML Wide-spread adoption and many data sources in XML

formatformat XQueryXQuery

W3C-candidate, textual XML querying languageW3C-candidate, textual XML querying language Turing-complete and well-supportedTuring-complete and well-supported Required programmatic skill similar to SQLRequired programmatic skill similar to SQL

ExamplesExamples<root> {<root> {

<patient> <patient>

<year_of_birth> </year_of_birth> <year_of_birth> </year_of_birth>

<wada_comments> </wada_comments> <wada_comments> </wada_comments>

<wada_memory> </wada_memory> <wada_memory> </wada_memory>

<wada_language> </wada_language> <wada_language> </wada_language>

<handedness> </handedness> <handedness> </handedness>

<sex> </sex> <sex> </sex>

<type> </type> <type> </type>

<gao_research_num> <gao_research_num> </gao_research_num> </gao_research_num>

<description> </description> <description> </description>

<pre> </pre> <pre> </pre>

<size> </size> <size> </size>

<pnum> </pnum> <pnum> </pnum>

<viq> </viq><viq> </viq>

<age_at_registration> <age_at_registration> </age_at_registration></age_at_registration>

....

……

……..

</patient></patient>

</root></root>

<result><result>

{{

for $pa in $root/patientfor $pa in $root/patient

where $pa/surgery/csmstudy/trial/stimulated/text() = 'Y'where $pa/surgery/csmstudy/trial/stimulated/text() = 'Y'

return <patient>return <patient>

{$pa/pnum}{$pa/pnum}

{$pa/viq}{$pa/viq}

{$pa/age_at_registration>{$pa/age_at_registration>


}</result>}</result>

XML source document

XQuery expression

Motivations for a graphic query Motivations for a graphic query systemsystem

Creating XQuery is difficultCreating XQuery is difficult Unfamiliar XQuery syntax and semanticsUnfamiliar XQuery syntax and semantics Need explicit knowledge of the source data schemaNeed explicit knowledge of the source data schema

Goals:Goals: Implement the Query-By-Example (QBE) paradigmImplement the Query-By-Example (QBE) paradigm Simplify query construction for novice usersSimplify query construction for novice users Expedite query generation process for expert usersExpedite query generation process for expert users Balance between usability and expressivityBalance between usability and expressivity

OutlineOutline

Needs for graphically querying XML Needs for graphically querying XML Existing graphical query solutionsExisting graphical query solutions Requirements for a successful systemRequirements for a successful system My approach and the XQuery Graphic My approach and the XQuery Graphic


Design approachesDesign approaches Structured query approachStructured query approach

Graphical interface is simpleGraphical interface is simple

Query process is quick and uncomplicatedQuery process is quick and uncomplicated

Output format is pre-determinedOutput format is pre-determined

Users are not exposed to the underlying querying languageUsers are not exposed to the underlying querying language

Unstructured query approachUnstructured query approachArbitrarily construct the output formatArbitrarily construct the output format

Able to generate more expressive XQuery statementsAble to generate more expressive XQuery statements

Complex query interfaceComplex query interface

Existing systemsExisting systems

Structured query approachStructured query approach QSByEQSByE QURSEDQURSED XBrainXBrain

Unstructured query approachUnstructured query approach BBQBBQ XQBEXQBE Stylus StudioStylus Studio

Existing systemsExisting systems

Structured query approachStructured query approach QSByEQSByE QURSEDQURSED XBrainXBrain

Unstructured query approachUnstructured query approach BBQBBQ XQBEXQBE Stylus StudioStylus Studio

QURSEDQURSED

Query and Reporting Semistructured DataQuery and Reporting Semistructured Data Developers create XML-querying forms for end-Developers create XML-querying forms for end-

usersusers

Displays source schema in a treeWeb-based query forms

Developers have to create the query form andthe output template for every single query

End-users cannot modify the query form, the output template, and the generated query

Uses output template to define structure

XBrainXBrain

Distributed query systemDistributed query system Developers create saved queries for end-users to Developers create saved queries for end-users to

executeexecute

Allows users to modify and save custom queriesVery simple query interface

Novice users need developers to create more complex queriesOutput format is pre-determined

XQBEXQBE

XQuery By ExampleXQuery By Example Users use the interface define the source and the Users use the interface define the source and the

construct treeconstruct tree

Able to generate fairly complex queries

Accepts multiple source schema formatsFree and still is being supported

Requires users to have explicit knowledgeof the source data schemaElaborate syntax and semantics

Stylus StudioStylus Studio

Comprehensive XML editing, publishing, and querying Comprehensive XML editing, publishing, and querying tooltool

Novel query interface design by explicitly constructing Novel query interface design by explicitly constructing XQuery expressionXQuery expression

Able to create very expressive XQuery statements

Displays the source schema as a tree

Expensive

Not open source or web-based

OutlineOutline

Needs for graphically querying XML Needs for graphically querying XML Existing graphical query solutionsExisting graphical query solutions Requirements for a successful Requirements for a successful

systemsystem My approach and the XQuery Graphic Interface My approach and the XQuery Graphic Interface

(XGI) system overview(XGI) system overview Using XGI to build a XQueryUsing XGI to build a XQuery System validation and evaluationSystem validation and evaluation Discussion and conclusionsDiscussion and conclusions

System requirementsSystem requirements

Reduced cost of implementationReduced cost of implementation Support for multiple schema formatsSupport for multiple schema formats Navigable source treeNavigable source tree Powerful XQuery creationPowerful XQuery creation Easy integration with other applicationsEasy integration with other applications Support collaborationSupport collaboration

OutlineOutline

Needs for graphically querying XML Needs for graphically querying XML Existing graphical query solutionsExisting graphical query solutions Requirements for a successful systemRequirements for a successful system

My approach and the XQuery Graphic My approach and the XQuery Graphic Interface (XGI) system overviewInterface (XGI) system overview

Using XGI to build a XQueryUsing XGI to build a XQuery System validation and evaluationSystem validation and evaluation Discussion and conclusionsDiscussion and conclusions

Reduced cost of implementationReduced cost of implementation Web-based using Asynchronous JavaScript Web-based using Asynchronous JavaScript

and XML (AJAX) technologyand XML (AJAX) technology Shared installation and resourcesShared installation and resources

Support for multiple schema formatsSupport for multiple schema formats XML Schema, Document Type Data (DTD), XML Schema, Document Type Data (DTD),

and auto-extract schema from XML documentand auto-extract schema from XML document Navigable source treeNavigable source tree

Uses hierarchical tree to represent the nested Uses hierarchical tree to represent the nested XML source schemaXML source schema

My approachMy approach

My approach (cont.)My approach (cont.)

Powerful XQuery creationPowerful XQuery creation Implements a subset of XQueryImplements a subset of XQuery

Easy integration with other applicationsEasy integration with other applications Uses open source toolkitsUses open source toolkits Implements modular design Implements modular design

Support collaborationSupport collaboration Allows users to save queries with self-Allows users to save queries with self-

explanatory namesexplanatory names

System architectureSystem architectureWeb BrowserLocal

Filesystem

Graphic interface scripts

Client

Schemas File

Management

Schemas

Data Model

ServerFilesystem

TranslationEngine

SchemaPreprocessor

Server

Data ModelController

XQueryGeneration

Engine

XQuery generation engineXQuery generation engine

Uses an Extended Uses an Extended Backus-Naur Form Backus-Naur Form grammargrammar

Retrieve the query Retrieve the query schema from the data schema from the data model controllermodel controller Preprocess the schema Preprocess the schema

to fit the grammarto fit the grammar Return the generated Return the generated

query to the query query to the query interfaceinterface

<query> <query> := <flwor_exp> | <startTag> ‘{‘ := <flwor_exp> | <startTag> ‘{‘ <query> ‘}’ <endTag> <query> ‘}’ <endTag>

<flwor_exp><flwor_exp> := <for> <where>? := <for> <where>? <return><return>

<for><for> := ‘for’ <some_var> ( ‘,’ <some_var> )*:= ‘for’ <some_var> ( ‘,’ <some_var> )*

<some_var><some_var> := ‘$’<var_name> ‘in’ := ‘$’<var_name> ‘in’ <path_exp><path_exp>

<where><where> := ‘where’ <constraint>?:= ‘where’ <constraint>?

<constraint><constraint> := <predicate> ( ‘and’ := <predicate> ( ‘and’ <predicate> )*<predicate> )*

<predicate><predicate> := ‘exists(’ <path_exp> ‘)’ | := ‘exists(’ <path_exp> ‘)’ | < pred_exp>< pred_exp>

<pred_exp> <pred_exp> := <expression> <operator> := <expression> <operator> <expression><expression>

<expression><expression> := <const> | <variable> := <const> | <variable>

<operator><operator> := ‘=’ | ‘<’ | ‘>’ | ‘<=’ | ‘>=’ | := ‘=’ | ‘<’ | ‘>’ | ‘<=’ | ‘>=’ | ‘!=’‘!=’

<return><return> := ‘return’ ( <emptyTag> | <path_exp> := ‘return’ ( <emptyTag> | <path_exp> ) | ‘return’ <query>) | ‘return’ <query>

<startTag><startTag> := ‘<’<name> <attr>? ‘>’:= ‘<’<name> <attr>? ‘>’

<endTag><endTag> := ‘</’ <name> ‘>’:= ‘</’ <name> ‘>’

<emptyTag><emptyTag> := ‘<’ <name> <attr>? ‘/>’:= ‘<’ <name> <attr>? ‘/>’

<attr><attr> := (<name> ‘=’ <path_exp> ) + := (<name> ‘=’ <path_exp> ) +

<name><name> := valid name of schema elements:= valid name of schema elements

<path_exp><path_exp> := XPath expression:= XPath expression

<variable><variable> := any variable that has := any variable that has been bound to an outer schema elementbeen bound to an outer schema element

<const><const> := a constant value of the schema := a constant value of the schema elementelement

<var_name><var_name> := an automatically := an automatically generated, non-duplicative name for a variablegenerated, non-duplicative name for a variable

OutlineOutline



Interface layoutInterface layoutDocument name panel

Toolbar panel

Search box

Source panelQuery panel

Information panelSaved predicate panel

Example query 1Example query 1<results><results>

{{


return <patient id="{$pa/pnum/text()}">return <patient id="{$pa/pnum/text()}">

{$pa/sex}{$pa/sex}

{$pa/viq}{$pa/viq}


}</results>}</results>

<root> {<root> {

<patient> <patient>

<year_of_birth> </year_of_birth> <year_of_birth> </year_of_birth>

<wada_comments> </wada_comments> <wada_comments> </wada_comments>

<wada_memory> </wada_memory> <wada_memory> </wada_memory>

<wada_language> </wada_language> <wada_language> </wada_language>

<handedness> </handedness> <handedness> </handedness>

<sex> </sex> <sex> </sex>

<type> </type> <type> </type>

<gao_research_num> <gao_research_num> </gao_research_num> </gao_research_num>

<description> </description> <description> </description>

<pre> </pre> <pre> </pre>

<size> </size> <size> </size>

<pnum> </pnum> <pnum> </pnum>

<viq> </viq><viq> </viq>

<age_at_registration> <age_at_registration> </age_at_registration></age_at_registration>

....

……

……..


</root></root>CSM database schema

Load a source schemaLoad a source schema

Insert the root nodeInsert the root node

Add nodes to the query schemaAdd nodes to the query schema

Add an attributeAdd an attribute

Create XQueryCreate XQuery

Example query 2Example query 2<results><results>

{{


where exists($pa/is_public) and $pa/surgery/csmstudy/trial/stimulated/text() = 'Y'where exists($pa/is_public) and $pa/surgery/csmstudy/trial/stimulated/text() = 'Y'

return <patient id="{$pa/pnum/text()}">return <patient id="{$pa/pnum/text()}">

{$pa/sex}{$pa/sex}

{$pa/viq}{$pa/viq}

<age><age>

{$pa/age_at_registration/text()}{$pa/age_at_registration/text()}

</age></age>

{{

for $tr in $pa/surgery/csmstudy/trialfor $tr in $pa/surgery/csmstudy/trial

return <trials>return <trials>

{$tr/trial_num}{$tr/trial_num}

</trials></trials>

}}


}</results>}</results>

Add a user-defined nodeAdd a user-defined node

Change node relationChange node relation

Add existential quantificationAdd existential quantification

Add a predicateAdd a predicate

Change node nameChange node name

Create XQueryCreate XQuery

OutlineOutline



ValidationValidation

XGI vs. XQBEXGI vs. XQBE Compare featuresCompare features Compare capable queriesCompare capable queries

Recreate queries from XBrainRecreate queries from XBrain Language-error queriesLanguage-error queries Custom saved queriesCustom saved queries

Compare features with XQBECompare features with XQBEFeatureFeature XQueryXQuery XQBEXQBE XGIXGI

Existential Existential QuantificationQuantification

YesYes YesYes YesYes

ConjunctionConjunction YesYes YesYes YesYes

Breadth ProjectionBreadth Projection YesYes YesYes YesYes

Depth ProjectionDepth Projection YesYes YesYes YesYes

RenamingRenaming YesYes YesYes YesYes

New ElementNew Element YesYes YesYes YesYes

JoinJoin YesYes YesYes PartialPartial

Cartesian ProductCartesian Product YesYes YesYes PartialPartial

FlatteningFlattening YesYes YesYes YesYes

NestingNesting YesYes PartialPartial NoNo

FilteringFiltering YesYes YesYes NoNo

NegationNegation YesYes PartialPartial NoNo

AggregatesAggregates YesYes YesYes NoNo

Arithmetic Arithmetic ComputationsComputations

YesYes YesYes NoNo

SortingSorting YesYes YesYes NoNo

Recreate queriesRecreate queries

Able to generate 13/23 queries capable by Able to generate 13/23 queries capable by XQBEXQBE

Generate all language error queries from XBrainGenerate all language error queries from XBrain

Expert users created custom queries in XBrainExpert users created custom queries in XBrain Fully or partially recreate 22/43Fully or partially recreate 22/43 Cannot recreate 21/43Cannot recreate 21/43

User EvaluationUser Evaluation

Only one expert user used the system informallyOnly one expert user used the system informally InstallationInstallation

Require multiple packages that should be bundled Require multiple packages that should be bundled into a deployable archive fileinto a deployable archive file

Need to change a few system settingsNeed to change a few system settings Certain features could be more easily accessibleCertain features could be more easily accessible Need to support more featuresNeed to support more features More feedbacks on the limitations of the More feedbacks on the limitations of the

interfaceinterface

DiscussionDiscussion

XGI is capable of generating a large XGI is capable of generating a large subset of XQuerysubset of XQuery

Limitations:Limitations: Does not support functions and some query Does not support functions and some query

constructsconstructs Saved queries are constrained by the query Saved queries are constrained by the query

schemaschema

Future workFuture work

Support more extensive collaborationSupport more extensive collaboration Implement more XQuery featuresImplement more XQuery features

““Let”, “Order by”, etcLet”, “Order by”, etc XQuery functions and user-defined functionsXQuery functions and user-defined functions

Improve UI and enable browser Improve UI and enable browser compatibilitycompatibility

ConclusionConclusion

XGI is a simple and effective graphical XGI is a simple and effective graphical query tool for assisting both novice and query tool for assisting both novice and expert users to create complex XQuery expert users to create complex XQuery

statements quickly and accurately. statements quickly and accurately.

Thank You!Thank You!

Documents

XGI: A Graphic Interface for XQuery Creation and XML Schema Visualization