Crossing the Structure Chasm

Alon HalevyUniversity of Washington

FQAS 2002

Outline The structure chasm:

Old problem Reasons for renewed interest (Semantic Web).

Crossing the chasm at the U. of Washington: Getting people to structure their data (Darwin) Large-scale data sharing by Peer-data

management (Piazza) What can you do with a corpus of 1 million

schemas? (Let’s build it and see). Challenges going forward.

The Structure ChasmThe U-World Authoring: easy, learned

in grade-school.

Querying: easy, keywords, I don’t need to know the database.

Data sharing: put our documents into the same corpus/search engine.

Results: approximate. For human viewing.

The S-World Authoring: need to design

structure first. Learned in college.

Querying: I need to know how you structured your data.

Data sharing: negotiations, possibly committee work.

Results: need to be precise; may affect bank accounts.

Why Should We Care? The chasm is limiting the use of S-

World technology: Losing potential customers, applications. Being ignored by friends and family.

People often end up using more lightweight solutions: Causes loss of functionality No easy migration path back to the S-

World.

Creating The Semantic Web We want a web of structured data,

where searches are more meaningful: It’s a gigantic distributed database. Content authors are not DB/KR

specialists. The SW has not taken off yet

precisely because of the chasm. Claim: We do not know how to build

large-scale data sharing systems.

Crossing the Structure Chasm

Goal: import some of the nice properties of the U-world into the S-world.

Make authoring, querying and sharing of data easier.

No illusions: The S-world will always be harder than the U-world.

Some non-solutions: (from KR people): more expressive

representation languages. (from DB people): didn’t XML solve this problem?

Pause

Corpus-basedDesignTools

Corpus ofStructured Data

Peer 1

U2SContent

AnnotationTool

HTML AnnotatedHTML

Peer 2

Peer 3

Peer 4 Schema

Mapping

Schem

a

Mapping

SchemaMapping

Sch

ema

Map

ping

Schema

Storage

Schema

Storage

Schema

StorageSchema

Storage

Queryover Peer4Schema

Results fromAll Mapped

Peers' StoredData

Schema

Sch

ema

Map

ping

s

Statistics overStructure

Entice people to structure their data

Enable sharing of data without central control

Tools for facilitating authoring and sharing of data

Darwin

Piazza

The Corpus

Crossing the Chasm with Revere

Crossing the Chasm w/ Revere Key components:

Darwin: Get people to structure their data. Piazza: Enable people to share their data. Statistics over structures: Import the main

technique of the U-world into the S-world. Goal: create infrastructure for building

semantic web applications: First case study: creating a SW from data

that is already on people’s web pages.

Outline The structure chasm:

Old problem Reasons for renewed interest (Semantic Web).

Crossing the chasm at the U. of Washington: Getting people to structure their data (Darwin) Large-scale data sharing by Peer-data

management (Piazza) What can you do with a corpus of 1 million

schemas? (Let’s build it and see).

Darwin: an evolutionary approach to the semantic web

Two challenges: Can we create conditions to entice people to

create semantic content? Can a database evolve rather than being

created in the traditional fashion? Goal: create a semantic web from data

that is already on web pages: events, contact info,… Large number of very heterogeneous web

pages. Wrapper technology does not apply. Accessing at query-time not scalable.

Joint work with: Etzioni, Gribble, Levy, McDowell, Vlasseva

Key Ideas of Darwin Make it easy: tool for annotating HTML

pages No need to replicate data.

Immediate gratification: A set of applications that provide immediate

benefit (calendar, phone book) Illustrate that even partial data is useful.

Defer checking integrity constraints Start local, reach out to others later.

Darwin and the Chasm Addresses first step: getting data

into structured form. Challenges:

How to entice people to create content

How to evolve a database/knowledge base?

How to do this in a scalable fashion.

Outline The structure chasm:

Old problem Reasons for renewed interest (Semantic Web).

Crossing the chasm at the U. of Washington: Getting people to structure their data (Darwin) Large-scale data sharing by Peer-data

management (Piazza) What can you do with a corpus of 1 million

schemas? (Let’s build it and see).

Large-Scale Data Sharing

Goal: to share structured data across multiple autonomous sites.

Current solution: data integration Query a set of data sources through a

mediated schema. Use XML as a data sharing format, and

XQuery. Information Manifold (96), Tukwila (99),

Nimble Technology (www.nimble.com).

(With Ives, Mork, Suciu, Tatarinov)

ReviewsSh ip p in gO rd ersIn ven toryBooks

m ybooks .com M edia ted S chem a

W e s t

...

F e dE x

W A N

a lt.bo o ks .re v ie w s

In te rne tIn te rne t In te rne t

UP S

E a s t O rde rs C us to me rR e v ie w s

NY Time s

...

M o rga n-K a ufma n

P re ntic e -Ha ll

Data Integration Systems

Limitations of Data Integration The mediated schema:

Creating it is hard, often infeasible. Mapping to it may involve repetitive

work. Querying it can be hard for users

familiar with their own schema. Note: much better than warehousing.

Goal: share data without a single mediated schema.

Peer Data-Management

PDMS: a network of peers Peers can:

Export base data Provide views on base data Serve as logical mediators for other peers

A peer can be both a server and a client.

Semantic relationships are specified locally (between small sets of peers).

Advantages of PDMS No need for a central mediated schema. Can map data opportunistically, as is most

convenient. Queries are posed using the peer’s

schema. Answers come from anywhere in the system.

Relationship to peer-to-peer file sharing: Data has rich semantics Probably not as dynamic in membership.

Example PDMS

Hospitals(H)

FirstHospital (FH)

LakeviewHospital (LH)

LH:CritBed(bed, hosp, room, PID, status) H:CritBed(bed, hosp, room), H:Patient(PID, bed, status)

Ad-hoc Additions to a PDMS

MedicalAid (MA)

EarthquakeCommand

Center (ECC)

Search &Rescue (SR)

EmergencyWorkers (EW)

WashingtonState

NationalGuard

PDMS Research Directions Schema mediation:

Languages for specifying mappings. Algorithms for answering queries. Easy generation of mappings.

Efficiency and optimization: Avoiding redundant paths, following best ones. Propagating updates efficiently (w/ Mork,

Gribble). Distributed indexing of views (Dalvi, Suciu).

Schema Mediation in PDMS

The formalism for the semantic glue. From data integration, we have:

Global-as-view (GAV): mediated schema is defined as views over the sources [query composition].

Local-as-view (LAV): sources are defined as views over mediated schema [answering q’s u/views]

GLAV: a combination of both: Qsource = Qschema

Qsource Qschema

Query answering is understood for a two-tier network: a mediator over multiple sources.

Hospitals(H)

FirstHospital (FH)

LakeviewHospital (LH)

LH:CritBed(bed, hosp, room, PID, status) H:CritBed(bed, hosp, room), H:Patient(PID, bed, status)

Mediation: the Relational Case A mediation language that uses

GLAV locally. Precise conditions for when global

query answering in a PDMS is tractable/decidable.

A query answering algorithm that combines chains of query composition and answering queries using views.

See ICDE-03 paper for details.

Mediation: the XML Case Mediation language:

XQuery is inappropriate. Our language allows incremental

specification of mappings. Uses subset of XQuery.

Query answering algorithm: New techniques for answering queries

using views: Challenge: nesting in XML structure.

Implementation: based on XML.

Additional Mediation Issues Mapping composition:

Given A-B and B-C mappings, is there an A-C mapping that doesn’t lose information?

Yes, and no. Even when yes, it may be infinite. [w/ Madhavan].

Basic framework and properties of mappings: KR community needs to consider mappings

as first-class citizens. See [AAAI-02].

Piazza and the Chasm Enable data ad-hoc large-scale data

sharing. No need for central control or schema.

Open issues: Optimization (follow only good paths?) Annotations on mappings? Intelligent data placement. Update propagation.

Outline The structure chasm:

Old problem Reasons for renewed interest (Semantic Web).

Crossing the chasm at the U. of Washington: Getting people to structure their data (Darwin) Large-scale data sharing by Peer-data

management (Piazza) What can you do with a corpus of 1 million

schemas? (Let’s build it and see).

Corpus Based Tools Information retrieval works by:

Large corpora of text Statistics over word occurrences in

texts. Can we do the same in the S-World?

Create a corpus of schemas. Use it to build tools that facilitate

authoring, querying and sharing data.

The Corpus Contents:

Schemas, ontologies, meta-data, data, queries.

Sample statistics: How often does a word appear as a

relation name? When it does, what tend to be the

attribute names? What other tables are there? What

are the foreign keys?

Sample Tools Auto-complete:

I start creating a schema, and the tools suggests a completion (perhaps I start only with data, not schema).

Schema matcher: I can map two between two schemas by

relating them both to the corpus. Query reformulator:

I ask a query using my terminology, and the tools reformulates it to a particular database schema.

Why are we Optimistic? Because of our work on LSD and

GLUE [w/ Doan, Domingos, Madhavan]: We computed classifiers for attributes of

schemas. Classifiers are a particular kind of

statistic. This is a huge community project:

We need your help Or at least, your schemas

Summary Takeaway questions:

How can we entice people to structure data? Can we generalize Peer-to-peer systems to

structured data? What can we do with a corpus of schemas,

and how can we build it? For more details,

www.cs.washington.edu/homes/alon [CIDR-03], [AAAI-02], [ICDE-03], [WWW-02],

[SIGMOD-01].