Towards a Vocabulary for Data Quality Management in Semantic Web Architectures

Towards a Vocabulary for

DQM in Semantic Web

Architectures (Research in Progress)

Christian Fürber and Martin Hepp

[email protected], [email protected]

Presentation @ 1st International Workshop on Linked Web

Data Management,

March 25th, 2011, Uppsala, Sweden

Part 1:

What‘s the Problem?

C. Fürber, M. Hepp:

Towards a Vocabulary for DQM

In SemWeb Architectures

2

Various Data Quality Problems

3

Character alignment violation

Invalid characters

Word transpositions

Invalid substrings Mistyping / Misspelling errors

False values

Misfielded values

Meaningless values

Missing values

Out of range values

Functional Dependency

Violation

Incorrect reference

Referential integrity violation

Contradictory relationships

Imprecise values

Existence of Synonyms

Existence of Homonyms

Unique value violation

Inconsistent duplicates

Approximate duplicates

Outdated values Outdated conceptual elements

Cardinality violation

Missing classification

Untyped literals

Incorrect classification



in SemWeb Architectures

Refe

rence: L

inkin

g O

pen D

ata

clo

ud d

iagra

m, b

y

Ric

hard

Cygania

k a

nd A

nja

Jentz

sch. h

ttp://lo

d-c

loud.n

et/

The Problem




4

Data retrieved on 2011-03-12 from http://loc.openlinksw.com/sparql

Negative Population

Weird Population Values

Invalid URL‘s

http://loc.openlinksw.com/sparql


Part 2:

What are high quality data?




5

What is Data Quality?

• Data‘s „fitness for use by data consumers“ (Wang, Strong 1996)

• „Conformance to specification“ (Kahn et al. 2002)

• „Data are of high quality if they are fit for their intended

uses in operations, decision making, and planning. Data

are fit for use if they are free of defects and possess

desired features.“ (Redman 2001)




6

• Requirements as „Benchmark“

Perspective-Neutral Data Quality




7

Data quality is the degree to which

data fulfills quality requirements

…no matter who makes the quality requirements.

The Problem




8

Data retrieved on 2011-03-12 from http://loc.openlinksw.com/sparql

Negative Population

Weird Population Values

Invalid URL‘s

Population

cannot be

negative

Population is

indicated by

numeric values

URL‘s usually

start with http://,

https://, etc.

Quality-

Requirements



Satisfying Quality Requirements




9

Status

Quo

Desired

State

= Individuals

Groups

Standards,

etc.

Problem 3: Satisfying

Requirements

Desired

State

Desired

State

Problem 2: Harmonizing

Requirements Problem 1: Expressing

Quality Requirements

Part 3:

Research Goal




10

Major Research Goal

• Represent Quality-Relevant information for

automated…

– Data Quality Monitoring

– Data Quality Assessment

– Data Cleansing

– Filtering of High Quality Data

…in a standardized vocabulary.




11

Motives for DQM-Vocabulary

• Support people to explicitly express data quality

requirements in „same language“ on Web-Scale

• Support the creation of consensual agreements

upon quality requirements

• Reduce effort for DQM-Activities

• Raise transparency about assumed quality

requirements

• Enable consistency checks among quality

requirements C. Fürber, M. Hepp:



12

Part 4:

Our Approach




13

Basic Architecture




14

DQM-Vocabulary

Knowledgebase

SPARQL-Query-Engine

RDB A RDB B Data Acquisition

Assessment

Scores Problem

Classification

Cleansed

Data

HQ Data

Retrieval

Main Concepts of DQM-Vocabulary




15

Express

Requirements

Express

Cleansing

Tasks

Annotate

Quality

Scores

Classify Quality

Problems

Account for

Task-Dependent

Requirements

Data Quality Problem Types:

Source for Potential Requirements

16

Character alignment violation

Invalid characters

Word transpositions

Invalid substrings Mistyping / Misspelling errors

False values

Misfielded values

Meaningless values

Missing values

Out of range values

Functional Dependency

Violation

Incorrect reference

Referential integrity violation

Contradictory relationships

Imprecise values

Existence of Synonyms

Existence of Homonyms

Unique value violation

Inconsistent duplicates

Approximate duplicates

Outdated values Outdated conceptual elements

Cardinality violation

Missing classification

Incorrect classification




Data Quality Requirements




17

Syntactical Rules

Semantic Rules

Redundancy Rules

Completeness Rules

Timeliness Rules

Quality-Influencing Artifacts




18

Data

Current Focus

of DQM-Vocabulary

Design Alternatives:

Statements about Classes & Properties

(1) Using classes and properties as subjects

(2) Using datatype properties with xsd:anyURI

(3) Mapping class and property URI‘s to new URI‘s




19

Part 5:

Application Examples




20

Example 1: Legal Value Rule (1/3)




21

What instances have illegal values for property foo:country ?





22

dqm:LegalValueRule Class

Instance

Literal value

“foo:Countries“

“foo:countryName“ “tref:countryName“

“tref:Countries“

foo:LegalValueRule_1





23

Example 2: DQ-Assessment (1/2)




24

How syntactically accurate are all

properties that are subject to

LegalValueRules?

Example 2: DQ-Assessment (2/2)




25

Part 6:

Conclusions &

Planned Work




26

Advantages of DQM-Voabulary

• Minimizes human effort for DQM

• Web-Scale sharing/reuse of data quality

requirements

• Consistency checks among data quality

requirements

• Transparency about applied data quality

rules




27

Limitations

• Representation of complex functional

dependency rules and derivation rules

• Limited experience on real world-data sets

• Currently no own concepts for classes and

properties

• Research still in progress




28

Future Work • Evaluation of design alternatives

• Development of processing framework

• Representation of more complex

functional dependency rules / derivation

rules

• Extension of DQM-Vobulary

• Evaluation on real-world data sets

• Publication at http://semwebquality.org




29

http://semwebquality.org/

30

Christian Fürber Researcher

E-Business & Web Science Research Group

Werner-Heisenberg-Weg 39

85577 Neubiberg

Germany

skype c.fuerber

email [email protected]

web http://www.unibw.de/ebusiness

homepage http://www.fuerber.com

twitter http://www.twitter.com/cfuerber

Paper available at http://bit.ly/gYEDdQ

http://bit.ly/gYEDdQ

http://bit.ly/gYEDdQ

Technology

Towards a Vocabulary for Data Quality Management in Semantic Web Architectures