SQL Server 2000 XML Annotated Schemata Michael Rys

Copyright 2000, Microsoft Corp.Copyright 2000, Microsoft Corp.

SQL Server 2000 XML SQL Server 2000 XML Annotated SchemataAnnotated Schemata

Michael RysMichael Rys

Program ManagerProgram ManagerSQLServer XML TechnologiesSQLServer XML TechnologiesMicrosoft CorporationMicrosoft Corporation

[email protected]@microsoft.com


ScenariosScenarios

Web applications with Dynamic DataWeb applications with Dynamic Data Browser based applications that require Browser based applications that require

data from a databasedata from a database

Business-to-business data Business-to-business data processing; messagingprocessing; messaging Data interchange using XML as Data interchange using XML as

ubiquitous, extensible, platform ubiquitous, extensible, platform independent transport formatindependent transport format


Loosely Coupled SystemsLoosely Coupled Systems Scalable. Many to Many.Scalable. Many to Many.

Changes in Implementation do not break Changes in Implementation do not break each othereach other

AppAppLogicLogic DataDataMappingMapping MappingMapping

Application SystemApplication System Data SystemData System

ObjectObject(XML)(XML)

Move data in a standardized format:Move data in a standardized format: XML SyntaxXML Syntax XML Schemata for domain standardXML Schemata for domain standard


ChallengesChallenges Providing XML views Providing XML views

to relational tablesto relational tables Allow efficient querying of XML Allow efficient querying of XML

viewsviews Allow updating of XML viewsAllow updating of XML views Provide other useful applications of Provide other useful applications of

XML views:XML views: BulkloadBulkload Relational schema generationRelational schema generation


OrdersID 222Cust 243

Order LinesOrder ID 222Line No 1Item dt334Quantity 1

View ExampleView Example

<order ordid=""222"" cust=""243""> <line qty=""1"" item=""dt334""/></order>


XML ViewsXML Views Defines an XML View on the databaseDefines an XML View on the database Uses XML-Data Reduced (XDR) syntax Uses XML-Data Reduced (XDR) syntax

with annotations (mapping schemas)with annotations (mapping schemas) Similar to DTD but using XML grammarSimilar to DTD but using XML grammar Public schema grammar used by BizTalkPublic schema grammar used by BizTalk

Annotations specify the XML to Annotations specify the XML to relational database mappingrelational database mapping For column valuesFor column values For relationships between contained tagsFor relationships between contained tags

XML View MapperXML View Mapper


<?xml version=""1.0""?><Schema name=""Orders""

xmlns=""urn:schemas-microsoft-com:xml-data""xmlns:dt=""urn:schemas-microsoft-com:datatypes""><ElementType name=""line"" model=""closed"">

<AttributeType name=""qty"" /><AttributeType name=""item"" /><AttributeType name=""line"" /><attribute type=""qty"" /><attribute type=""item"" />

</ElementType><AttributeType name=""ordid"" /><AttributeType name=""cust"" /><ElementType name=order model=""closed"">

<attribute type=""ordid"" /><attribute type=""cust"" /><element type=""line""/>

</ElementType></Schema>

Default MappingDefault Mapping


Default MappingDefault Mapping

Element name matches table nameElement name matches table name Attribute and textOnly subelement Attribute and textOnly subelement

names match column namesnames match column names No way to map hierarchies: resulting No way to map hierarchies: resulting

XML is flatXML is flat


AnnotationsAnnotationssql:relation sql:relation (map table-element name)(map table-element name)sql:field sql:field (map column-element/attribute name)(map column-element/attribute name)sql:relationship sql:relationship (hierarchies, joins)(hierarchies, joins)sql:limit-field & sql:limit-value sql:limit-field & sql:limit-value (horizontal partitions)(horizontal partitions)sql:key-fields sql:key-fields (keys)(keys)sql:id-prefix sql:id-prefix (XML prefix for IDs)(XML prefix for IDs)sql:is-constant sql:is-constant (exists in XML, do not map to DB)(exists in XML, do not map to DB)sql:map-field sql:map-field (exists in schema, do not map at all)(exists in schema, do not map at all)sql:overflow sql:overflow (column for unmapped XML)(column for unmapped XML)sql:use-cdata sql:use-cdata (generate CDATA section)(generate CDATA section)sql:url-encode sql:url-encode (contains dbobject reference)(contains dbobject reference)sql:datatype sql:datatype (SQL datatype hint)(SQL datatype hint)xmlns:sql=“urn:schemas-microsoft-com:xml-sql"xmlns:sql=“urn:schemas-microsoft-com:xml-sql"


AnnotationsAnnotations<?xml version="1.0" ?><?xml version="1.0" ?><Schema xmlns="urn:schemas-microsoft-com:xml-data"<Schema xmlns="urn:schemas-microsoft-com:xml-data" xmlns:sql="urn:schemas-microsoft-com:xml-sql">xmlns:sql="urn:schemas-microsoft-com:xml-sql"> <ElementType name="Customer" <ElementType name="Customer" sql:relation="Customers"sql:relation="Customers">> <AttributeType name="ID" /><AttributeType name="ID" /> <attribute type="ID" <attribute type="ID" sql:field="CustomerID"sql:field="CustomerID" /> /> <element type="Order"><element type="Order"> <sql:relationship key-relation="Customers"<sql:relationship key-relation="Customers" key="CustomerID"key="CustomerID" foreign-relation="Orders"foreign-relation="Orders" foreign-key="CustomerID"/>foreign-key="CustomerID"/> </element></element></ElementType></ElementType> <ElementType name="Order" <ElementType name="Order" sql:relation="Orders"sql:relation="Orders">> <AttributeType name="OrderID" /><AttributeType name="OrderID" /> <attribute type="OrderID" <attribute type="OrderID" sql:field="OrderID"sql:field="OrderID"/>/></ElementType></ElementType></Schema></Schema>


XPath to define XML ViewsXPath to define XML Views W3C RecommendationW3C Recommendation Allows to navigate XML trees:Allows to navigate XML trees:

Specifies set of nodesSpecifies set of nodes Allows specification of predicatesAllows specification of predicates

SQLServer 2000 XPath:SQLServer 2000 XPath: Subset of W3C XPath 1.0 RecommendationSubset of W3C XPath 1.0 Recommendation Defines virtual XML view over relational database Defines virtual XML view over relational database

together with Annotated Schemastogether with Annotated Schemas Usable in URLs, Templates, and in ADO/OLE-DBUsable in URLs, Templates, and in ADO/OLE-DB

NOTE: OpenXML uses MSXML XPath NOTE: OpenXML uses MSXML XPath implementationimplementation


SyntaxSyntax General SyntaxGeneral Syntax

/a::n[p]/b::m[q]/a::n[p]/b::m[q]where a::n[p] location step; a, b axes; m, n where a::n[p] location step; a, b axes; m, n nodetests; p, q predicatesnodetests; p, q predicates

Shortform for most common axesShortform for most common axes

Example:Example: /child::Customer[attribute::State="WA"]/child::Customer[attribute::State="WA"] Shortform: /Customer[@State="WA"]Shortform: /Customer[@State="WA"]


SemanticsSemantics General Semantics:General Semantics:

Select all nodes with name Select all nodes with name nn for which for which predicate predicate pp holds and can be reached via holds and can be reached via axis axis aa

pp is existentially quantified is existentially quantified

root

Customer CustomerCustomer

@state=WA @state=CA @state=WAOrder OrderOrder Order

/root /Customer [@state="WA"] /Order


SQLServer 2000 XPath SupportSQLServer 2000 XPath Support Axes: child, attribute, parent, selfAxes: child, attribute, parent, self Boolean-valued predicatesBoolean-valued predicates Relational operators =, !=, <=, >, >=Relational operators =, !=, <=, >, >= Arithmetic operators +, -, *, divArithmetic operators +, -, *, div Boolean operators/functions AND, OR, Boolean operators/functions AND, OR,

true(), false(), not()true(), false(), not() Explicit conversion functions number(), Explicit conversion functions number(),

string(), boolean()string(), boolean() VariablesVariables Not supported: See Books-OnLineNot supported: See Books-OnLine


XPath and Annotated SchemaXPath and Annotated Schema

SQL DB

XPath Query

/A/B/C

First location step Adefines instance ofXML view over DB

AnnotatedSchema

Defines setof potential XML views over DB


QueryingQuerying XPath QueryXPath Query

Used with a reference to the Used with a reference to the mapping schemamapping schema

/Customer[@ID=“ALFKI”] /Customer[@ID=“ALFKI”] Results in:Results in:

<Customer ID="ALFKI"><Customer ID="ALFKI"> <Order OrderID="10643" /> <Order OrderID="10643" /> <Order OrderID="10692" /> <Order OrderID="10692" /> <Order OrderID="10702" /> <Order OrderID="10702" /> <Order OrderID="10835" /> <Order OrderID="10835" /> <Order OrderID="10952" /> <Order OrderID="10952" /> <Order OrderID="11011" /> <Order OrderID="11011" /> </Customer></Customer>


Functional XPath LimitationsFunctional XPath Limitations Not a full Query Language, only navigationNot a full Query Language, only navigation XPath cannot projectXPath cannot project

Tailor annotated schema to project only Tailor annotated schema to project only accessible dataaccessible data

Use XSLT postprocessing to project data needed Use XSLT postprocessing to project data needed in query but not exposed in viewin query but not exposed in view

XPath cannot prune subtrees XPath cannot prune subtrees (e.g., give me all (e.g., give me all customers of state X and only their orders from today)customers of state X and only their orders from today) Static pruning: map to SQL viewsStatic pruning: map to SQL views Dynamic pruning: XSLTDynamic pruning: XSLT

XPath cannot transformXPath cannot transform Tailor annotated schemaTailor annotated schema Use XSLT to transformUse XSLT to transform


URL QueryURL Query

http://server/vroot/http://server/vroot/vnamevname//xpathxpath??paramsparams Virtual name (Virtual name (vnamevname))

Direct reference to mapping schemaDirect reference to mapping schema Path to directoryPath to directory

http://server/vroot/http://server/vroot/vnamevname/nwind.xml//nwind.xml/xpathxpath/?/?paramsparams

XPath Query (XPath Query (xpathxpath)) Specifies set of nodes from virtual document Specifies set of nodes from virtual document

to returnto return Encode + etc. into %xx in URLEncode + etc. into %xx in URL

Parameters (Parameters (paramsparams)) xslxsl, , outputencodingoutputencoding, , contenttypecontenttype, , rootroot, …, … User definedUser defined


TemplateTemplate

http://server/vroot/http://server/vroot/vnamevname??paramsparams Same as template query with Same as template query with

embedded SQL Query except:embedded SQL Query except: Embedded query references mapping Embedded query references mapping

schema fileschema file Query is expressed as XPathQuery is expressed as XPath

Supports querying over namespace Supports querying over namespace qualified namesqualified names


Template (Example)Template (Example)

<root xmlns:sql="urn:schemas-microsoft-<root xmlns:sql="urn:schemas-microsoft-com:xml-sql"com:xml-sql"

sql:xsl="sql:xsl="path to XSLT filepath to XSLT file" >" > <sql:header><sql:header> <sql:param name="state">WA</sql:param><sql:param name="state">WA</sql:param> </sql:header></sql:header> <sql:xpath-query <sql:xpath-query mapping-mapping-

schema="nwind.xdr"schema="nwind.xdr">> /Customers[@Region=$state]/Customers[@Region=$state] </sql:xpath-query></sql:xpath-query></root></root>


Template ProcessingTemplate Processing

ISAPIISAPI DataDataSQLSQL

FOR XMLFOR XML

Middle-TierMiddle-Tier SQL Server 2000SQL Server 2000

URLURLCustStateCustState.xml.xml

Computed Computed FOR XML FOR XML EXPLICITEXPLICIT

XMLXML

ComputesComputesRequest fromRequest fromCustState.xmlCustState.xml

Using nwind.xdrUsing nwind.xdr

nwindnwind.xdr.xdr

TDS/ TDS/ XMLXML


BizTalk IntegrationBizTalk Integration Uses same schema format as BizTalk Uses same schema format as BizTalk

framework and serverframework and server

BizTalkBizTalkSchemaSchema

copycopy BizTalkBizTalkSchemaSchema

AnnotationsAnnotations

++ == MappingMappingSchemaSchema

SQLSQLServerServer

++XPATHXPATH/Customers/Customers

Que

ryQ

uery

XMLXMLinin

BizTalkBizTalkGrammarGrammar

Results

Results


UpdateGramsUpdateGrams Expresses Modifications to an Expresses Modifications to an

XML documentXML document Uses optimistic concurrency controlUses optimistic concurrency control Captures inserts, deletes, and updatesCaptures inserts, deletes, and updates Supports hierarchical updates Supports hierarchical updates

to multiple tables using to multiple tables using annotated schemasannotated schemas

Allows specification of null imageAllows specification of null image Values can be parameterized (uses Values can be parameterized (uses

$param, see DB202 for syntax)$param, see DB202 for syntax) Currently ships in beta as web releaseCurrently ships in beta as web release


<r xmlns:updg="urn:schemas-microsoft-com:xml-updategram"><r xmlns:updg="urn:schemas-microsoft-com:xml-updategram"> <updg:sync mapping-schema="nwind.xml" <updg:sync mapping-schema="nwind.xml"

updg:nullvalue="ISNULL">updg:nullvalue="ISNULL"> <updg:before><updg:before> <Customer CustomerID="LAZYK" CompanyName="ISNULL"<Customer CustomerID="LAZYK" CompanyName="ISNULL" Address="12 Orchestra Terrace" > Address="12 Orchestra Terrace" > <Order oid="10482"/><Order oid="10482"/> <Order oid="10545"/><Order oid="10545"/> </Customer></Customer> </updg:before></updg:before> <updg:after><updg:after> <Customer CustomerID="LAZYK"<Customer CustomerID="LAZYK" CompanyName="Lazy K Country Store"CompanyName="Lazy K Country Store" Address="12 Opera Court" >Address="12 Opera Court" > <Order oid="10354"/><Order oid="10354"/> <Order oid="10545"/><Order oid="10545"/> </Customer ></Customer > </updg:after></updg:after> </updg:sync></updg:sync></r></r>

UpdateGramsUpdateGrams


BulkloadBulkload Allows loading of large amount of XML Allows loading of large amount of XML

into database via annotated schema.into database via annotated schema. Allows generation of relational schemaAllows generation of relational schema Provided as COM objectProvided as COM object Can be called from DTSCan be called from DTS Uses SAX to parse XMLUses SAX to parse XML Provides transacted and non-transacted Provides transacted and non-transacted

modesmodes Beta will ship Oct 2000 as web releaseBeta will ship Oct 2000 as web release


Bulkload - RulesBulkload - Rules Only limited buffering while loadingOnly limited buffering while loading Record Lifetime RuleRecord Lifetime Rule

Data for a record needs to be grouped in the same Data for a record needs to be grouped in the same element subtreeelement subtree

Basic Record Generation RulesBasic Record Generation Rules A new record is generated for a new relation on A new record is generated for a new relation on

node or if a sql:relationship exists for this nodenode or if a sql:relationship exists for this node

Subset Lifetime RuleSubset Lifetime Rule FK of 1:n relationships is buffered only as long as FK of 1:n relationships is buffered only as long as

parent that defines FK is in scopeparent that defines FK is in scope

Key Ordering RuleKey Ordering Rule FK of 1:n needs to appear before many childrenFK of 1:n needs to appear before many children


Bulkload - InterfaceBulkload - InterfaceISQLXMLBulkLoad {ISQLXMLBulkLoad { ConnectionString(…);ConnectionString(…); ConnectionCommand(…);ConnectionCommand(…); BulkLoad(…); BulkLoad(…); KeepNulls(…);KeepNulls(…); KeepIdentity(…);KeepIdentity(…); CheckConstraints(…);CheckConstraints(…); ForceTableLock(…);ForceTableLock(…); Transaction(…);Transaction(…); XMLFragment(…);XMLFragment(…); SchemaGen(…);SchemaGen(…); SGDropTables(…);SGDropTables(…); SGUseID(…);SGUseID(…); ErrorLogFile(…);ErrorLogFile(…); Execute(…); }Execute(…); }


SummarySummary Use XML-centric application Use XML-centric application

programming modelprogramming model Queryable and updatable XML view Queryable and updatable XML view

over databaseover database Provides:Provides:

Simple, declarative method for Simple, declarative method for defining XML views on the databasedefining XML views on the database NO CODING REQUIREDNO CODING REQUIRED

Mechanism to retrieve XML Mechanism to retrieve XML conforming to Biztalk conforming to Biztalk schema definitionsschema definitions

XSL for arbitrary outputXSL for arbitrary output


FutureFuture Web Release ModelWeb Release Model Updategrams (Aug-Nov 2000)Updategrams (Aug-Nov 2000) Bulk Load (Oct-Nov 2000)Bulk Load (Oct-Nov 2000) XML View Mapper (Sep-Nov 2000)XML View Mapper (Sep-Nov 2000) W3C Schemas (XSD)W3C Schemas (XSD) W3C Query LanguageW3C Query Language PerformancePerformance

Documents

SQL Server 2000 XML Annotated Schemata Michael Rys