Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Wright State UniversityCORE Scholar
Kno.e.sis Publications The Ohio Center of Excellence in Knowledge-Enabled Computing (Kno.e.sis)
10-20-2008
Relationship Web: Trailblazing, Analytics andComputing for Human ExperienceAmit P. ShethWright State University - Main Campus, [email protected]
Follow this and additional works at: http://corescholar.libraries.wright.edu/knoesis
Part of the Bioinformatics Commons, Communication Technology and New Media Commons,Databases and Information Systems Commons, OS and Networks Commons, and the Science andTechnology Studies Commons
This Presentation is brought to you for free and open access by the The Ohio Center of Excellence in Knowledge-Enabled Computing (Kno.e.sis) atCORE Scholar. It has been accepted for inclusion in Kno.e.sis Publications by an authorized administrator of CORE Scholar. For more information,please contact [email protected].
Repository CitationSheth, A. P. (2008). Relationship Web: Trailblazing, Analytics and Computing for Human Experience. .http://corescholar.libraries.wright.edu/knoesis/264
Knowledge Enabled Information and Services Science 1
Knowledge Enabled Information and Services Science
Kno.e.sis Center -- Labs
Bioinformatics Lab (Dr Raymer)
Semantic Sciences Lab (Dr Sheth)
Metadata and Languages Lab (Dr Prasad)
Semantic Web Lab (Dr Sheth + Dr. S.Wang)
Data Mining Lab (Dr Dong)
Service Research Lab (Dr Sheth)
Sensor Networking Bin Wang
Knowledge Enabled Information and Services Science
Kno.e.sis Members – a subset
Knowledge Enabled Information and Services Science
Evolution of the Web
2005
1990sWeb of databases
- dynamically generated pages- web query interfaces
Web of pages- text, manually created links- extensive navigation
Web of services- data = service = data, mashups- ubiquitous computing
Web of people- social networks, user-created content- GeneRIF, Connotea
Web as an oracle / assistant / partner- “ask to the Web”- using semantics to leveragetext + data + services + people
2010
Computing for Human Experience
Knowledge Enabled Information and Services Science
Relationship Web: Trailblazing , Analytics and Computing for Human Experience
27th International Conference on Conceptual Modeling (ER 2008)Barcelona, Oct 20-23 2008
Amit ShethKno.e.sis Center, Wright State University, Dayton, OH
This talk also represents work of several members of Kno.e.sis team, esp. the Semantic Discovery and Semantic Sensor Data. http://knoesis.wright.edu
Thanks, K. Godam, C. Henson, M. Perry, C. Ramakrishnan, C. Thomas. Also thanks to sponsors: NSF (SemDis and STT), NIH, AFRL, IBM, HP, Microsoft.
Knowledge Enabled Information and Services Science
A Compelling Vision: Trailblazing
“(Human mind works) by association. With one item in its grasp, it snaps instantly to the next that is suggested by the association of thoughts, in accordance with some intricate web of trails carried by the cells of the brain.“
Dr. Vannevar Bush, As We May Think, 1945
Knowledge Enabled Information and Services Science
Documents, Objects, Events, Insight, Experience
“An object by itself is intensely uninteresting”. – Grady Booch, Object Oriented Design with Applications, 1991
Keywords
|
Search(data)
Entities
|
Integration(information)
Relationships,Events
|
Analysis,Insight(knowledge)
Knowledge Enabled Information and Services Science
Semantics and Relationships on a Web scale
Increasing depth and sophistication in dealing with semantics–From searching to integration to analysis, insight, discovery
and decision making–Relationships
• Heart of semantics• Allows to continue progress from syntax and structure to
semantics
A. Sheth, et al. Relationships at the Heart of Semantic Web: Modeling, Discovering, and Exploiting Complex Semantic Relationships, in Enhancing the Power of the Internet (Studies in Fuzziness and Soft Computing, V. 139). SpringerVerlag., 2004. http://knoesis.wright.edu/library/resource.php?id=00190
Knowledge Enabled Information and Services Science
Relationship Web takes you away from “which document” could have data/information I need, to interconnecting Web of information embedded in the resources to give knowledge, insights and answers I seek.
Amit P. Sheth, Cartic Ramakrishnan: Relationship Web: Blazing Semantic Trails between Web Resources. IEEE Internet Computing July 2007.
Knowledge Enabled Information and Services Science
Structured text (biomedical literature)
Informal Text (Social Network
chatter) Multimedia Content and Web data
Web Services
Metadata Extraction
Patterns / Inference / Reasoning
Domain Models
Meta data / Semantic Annotations
Relationship Web
Search
Integration
Analysis
Discovery
Question Answering
Knowledge Enabled Information and Services Science
Issues - Relationships
Understanding and modeling relationshipsIdentifying Relationship (extraction)Discovering and Exploring Relationships (reasoning) Hypothesizing and Validating Complex RelationshipsUsing/exploiting Relationships for Semantic Applications (in search, querying, analysis, insight, discovery)
Knowledge Enabled Information and Services Science
Data, Information, and Insight
What: Which thing or which particular one
Who: What or which person or persons
Where: At or in what place
When: At what time
How: In what manner or way; by what means
Why: For what purpose, reason, or cause; with what intention, justification, or motive
© Ramesh Jain
Knowledge Enabled Information and Services Science
EventWeb [Jain]RelationshipWeb [Sheth]
A Web in which each node is an event or object, and connected to other nodes using–Linguistic Relationships–Referential Links: hypertext links to related information
(HREF); links with metadata (MREF), links referring to model (model reference in SAWSDL, SA-REST)
–Structural Links: showing spatial and temporal relationships–Causal Links: establishing causality–Relational Links: giving similarity or any other relationship
Adapted/extended from Ramesh Jain
Knowledge Enabled Information and Services Science
Supporting Relationships in the Real World
Objects and event represent or model real world– More complex relationships
• Living organisms: Saprotropism, Antagonism, Exploitation, Predation, Ammensalism or Antibiosis, Symbiosis
• Relationships between humans: …
Knowledge Enabled Information and Services Science
Karthik Gomadam
AmitSheth
Attended Google IO
Moscone Center, SFOMay 28-29, 2008
Is advised by
Ph.D Student Researcheris_advised_by
Assistant Professor Professor
Research Paper
Journal Conference
Location
published_in published_in
has_location
Image Metadata
Event
Causal
kno.e.sis
DirectsRelational
Knowledge Enabled Information and Services Science
Identifying & Representing Relationships
Implicit Relationships– Statistical representation of interactions between entities,
co-occurrence of terms in the same cluster, tag cloud...
Linking of one document to another via a hyperlink…
– Two documents’ belonging to categories that are siblings in a concept hierarchy.
Knowledge Enabled Information and Services Science
Identifying & Representing Relationships
Explicit Linguistic Relationships
“He beat Randy Johnson” converted into the triple
Dontrelle WillisbeatRandy Johnson
Knowledge Enabled Information and Services Science
Identifying & Representing Relationships
Formal Relationships
–Subsumption, partonomy, ….
Domain specific vs domain-independent relationships
–Example of domain independent relationships: time, space/location
Complex entity and relationships (example later)
Knowledge Enabled Information and Services Science
Why is This a Hard Problem?
Are objects/entities equivalent/equal(same)?How (well) are they related? –Implicit vs explicit:
• Statistical representation of interactions between entities, co-occurrence of terms in the same cluster, tag cloud...
• formal/assertional vs social consensus based• powerful (beyond FOL): partial, probilistic and fuzzy match
–Degrees of relatedness and relevance: semantic similarity, semantic proximity, semantic distance, …• [differentiation, disjointedness]• related in a “context”
Knowledge Enabled Information and Services Science
Why is This a Hard Problem?
Semantic ambiguity When information (knowledge) is –Incomplete– inconsistent –Approximate
–Even is-a link involves different notions: identify, unity, essense (Guarino and Wetley 2002)
A.Sheth, et al, “Semantics for The Semantic Web: the Implicit, the Formal and the Powerful,” International Journal on Semantic Web & Information Systems, 1 (no. 1), 2005, pp. 1–18.
Knowledge Enabled Information and Services Science
Faceted Search and Semantic Analytics –Some Early Work where relationships played key role
InfoHarness/VisualHarness (1993-1998)InfoQuilt (1996-2000)MREF (1996-1998)OBSERVER (1996-2000)Taalee Semantic (Faceted) Search, Semantic Directory and Semagix Freedom enabled analytics (1999-2006)
Knowledge Enabled Information and Services Science
InfoQuilt (1996-2000): Using metadata PatchQuilt and user models/ontologies to support queries and analytics over globally distributed heterogeneous media repositories
Knowledge Enabled Information and Services Science
Physical Link to Relationship
<TITLE> A Scenic Sunset at Lake Tahoe </TITLE><p>Lake Tahoe is a popular tourist spot and<A HREF = “http://www1.server.edu/lake_tahoe.txt”>some interesting facts</A> are available here. The scenic beauty of Lake Tahoe can be viewed in this photograph:<center><IMG SRC=“http://www2.server.edu/lake_tahoe.img”></center>
Traditionally, correlation is achieved by using physical linksDone manually by user publishing the HTML document
Knowledge Enabled Information and Services Science
MREF (1996-1998)Metadata Reference Link -- complementing HREF
Creating “logical web” through Media independent metadata based on correlation
Knowledge Enabled Information and Services Science
Metadata Reference Link (<A MREF …>)
<A HREF=“URL”>Document Description</A>physical link between document (components)
<A MREF KEYWORDS=<list-of-keywords>; THRESH=<real>>Document Description</A>
<A MREF ATTRIBUTES(<list-of-attribute-value-pairs>)>Document Description</A>
Knowledge Enabled Information and Services Science
Correlation Based on Content-Based Metadata
Some interesting information on dams is available here“information on dams” defined by MREF to keywords and metadata (may be used for a query)
height, width and size
water.gif (Data) Metadata Storage
water.gif……mpeg……ppm
Major component(RGB)Blue
Content based Metadata
ContentDependentMetadata
Knowledge Enabled Information and Services Science
Abstraction Layers
METADATA
DATA
METADATA
DATA
ONTOLOGYNAMESPACE
ONTOLOGYNAMESPACE
MREFin RDF
Knowledge Enabled Information and Services Science
MREF (1998)
Model for LogicalCorrelation using Ontological Terms and Metadata
Framework for Representing MREFs
Serialization(one implementation choice)
K. Shah and A. Sheth, "Logical Information Modeling of Web-accessible Heterogeneous Digital Assets",Proc. of the Forum on Research and Technology Advances in Digital Libraries," (ADL'98),Santa Barbara, CA, May 28-30, 1998, pp. 266-275.
M R E F
R D F
X M L
Knowledge Enabled Information and Services Science
Domain Specific Correlation
Potential locations for a future shopping mall identified by all regions having a population greater than 500 and area greater than 50 sq meters having an urban land cover and moderate relief <A MREF ATTRIBUTES(population < 500; area < 50 & region-type = ‘block’ & land-cover = ‘urban’ & relief = ‘moderate’)>can be viewed here</A>
Knowledge Enabled Information and Services Science
Domain Specific Correlation
=> media-independent relationships between domain specific metadata: population, area, land cover, relief
=> correlation between image and structured data at a higher domain specific level as opposed to physical “link-chasing” in the WWW
Knowledge Enabled Information and Services Science
TIGER/Line DB
Population: Area:
Boundaries:
Land cover:Relief:
Census DB
Map DB
Regions(SQL)
Boundaries
Image Features(IP routines)
Repositories and the Media Types
Knowledge Enabled Information and Services Science
Knowledge Enabled Information and Services Science
Complex Relationships
Some relationships may not be manually asserted, but according to statistical analyses of text, experimental data, etc.– allow association of provenance data with classes,
instances, relationship types and direct relationships or statements
Knowledge Enabled Information and Services Science
Complex Relationships
Relationships (mappings) are not always simple mathematical / string transformationsExamples of complex relationships–Associations / paths between classes–Graph based / form fitting functions–Probabilistic relational
E 1 :Reviewer
E 6 :Person
E 5 :Person
E 2 :Paper
E 4 :Paper
E 7 :Submission
E 3 :Person
author _ ofauthor _ of
author _of
author _ of
author _ of
knows
knows
Knowledge Enabled Information and Services Science
A Simple Relationship ?
Smoking CancerCauses
Graph based / form fitting functions
Knowledge Enabled Information and Services Science
Complex Relationships Cause-Effects & Knowledge Discovery
VOLCANO
LOCATIONASH RAIN
PYROCLASTICFLOW
ENVIRON.
LOCATION
PEOPLE
WEATHER
PLANT
BUILDING
DESTROYS
COOLS TEMP
DESTROYS
KILLS
Knowledge Enabled Information and Services Science
Knowledge Discovery - Example
Earthquake Sources Nuclear Test Sources
Nuclear Test May Cause Earthquakes
Is it really true?
Complex Relationship:
How do you model this?
Knowledge Enabled Information and Services Science
Complex Relationships – Several Challenges
–Probabilistic relations
Number of earthquakes with magnitude > 7 almost constant. So if at all, then nuclear tests only cause earthquakes with magnitude < 7
Knowledge Enabled Information and Services Science
Entity, Relationship, Event Extraction and Semantic Annotation
From content with structure –Web pages–Deep Web
From well-formed text (edited for rules of grammar)From informal or casual text–social networking sites
From digital media
Knowledge Enabled Information and Services Science© Semagix, Inc.
Semagix Freedom for building ontology-driven information system
Extracting Semantic Metadata from Semistructured and Structured Sources
Knowledge Enabled Information and Services Science
WWW, EnterpriseRepositories
METADATA
EXTRACTORS
Digital Maps
NexisUPIAPFeeds/
Documents
Digital Audios
Data Stores
Digital Videos
Digital Images. . .
. . . . . .
Create/extract as much (semantics) metadata automatically as possible;
Use ontlogies to improve and enhance extraction
Information Extraction for Metadata Creation
Knowledge Enabled Information and Services Science
Automatic Semantic Metadata Extraction/Annotation
Knowledge Enabled Information and Services Science
Limited tagging(mostly syntactic)
COMTEX Tagging
Content‘Enhancement’
Rich Semantic Metatagging
Value-added Voquette Semantic Tagging
Value-addedrelevant metatagsadded by Voquetteto existing COMTEX tags:
• Private companies • Type of company• Industry affiliation• Sector• Exchange• Company Execs• Competitors
Semantic Annotation (Extraction + Enhancement)
Knowledge Enabled Information and Services Science
Video withEditorialized Text on the WebAuto
Categorization
Semantic Metadata
Automatic Classification & Metadata Extraction (Web Page)
Knowledge Enabled Information and Services Science
Extraction Agent
Enhanced Metadata AssetWeb Page
Ontology-Directed Metadata Extraction (Semi-Structured Data)
Knowledge Enabled Information and Services Science
Some real-world or commercial semantic applications
Knowledge Enabled Information and Services Science
Taalee’s Semantic (Facted) Search (1999-2001):Highly customizable, precise and fresh A/V search
Delightful, relevant information,exceptional targeting opportunity
Knowledge Enabled Information and Services Science
Sem
anti
c E
nri
chm
ent
Wh
at e
lse
can
a c
onte
xt d
o?
Knowledge Enabled Information and Services Science
Cre
atin
g a
Web
of
rela
ted
info
rmat
ion
Wh
at c
an a
con
text
do?
(a c
omm
erci
al p
ersp
ecti
ve)
Knowledge Enabled Information and Services Science
Wh
at e
lse
can
a c
onte
xt d
o?(a
com
mer
cia
l per
spec
tive
)
Sem
anti
c E
nri
chm
ent
Semantic Targeting
Knowledge Enabled Information and Services Science
BLENDED BROWSING & QUERYING INTERFACE
ATTRIBUTE & KEYWORDQUERYING
uniform view of worldwide distributed assets of similar type
SEMANTIC BROWSING
Targeted e-shopping/e-commerce
assets access
VideoAnywhere and Taalee Semantic Querying and Browsing (1998-2001)
Knowledge Enabled Information and Services Science
Semantic/Interactive Targeting (1999-2001)
Buy Al Pacino VideosBuy Russell Crowe VideosBuy Christopher Plummer VideosBuy Diane Venora VideosBuy Philip Baker Hall VideosBuy The Insider Video
Precisely targeted through the use of Structured Metadata and integration from multiple sources
Knowledge Enabled Information and Services Science
Keyword, Attribute and Content Based Access
Blended Semantic Browsing and Querying (Intelligence Analyst Workbench) 2001-2003
Knowledge Enabled Information and Services Science
Search for company
‘Commerce One’
Links to news on companies that compete against
Commerce One
Links to news on companies Commerce One competes
against
(To view news on Ariba, click on the link for Ariba)
Crucial news on Commerce One’s
competitors (Ariba) can be accessed easily
and automatically
Semantic Browsing/Directory (2001-….)
Knowledge Enabled Information and Services Science
System recognizes ENTITY & CATEGORY
Relevant portionof the Directory is automatically presented.
Semantic Directory
Knowledge Enabled Information and Services Science
Users can exploreSemantically related
Information.
Semantic Directory
Knowledge Enabled Information and Services Science
Semantic Relationships
Knowledge Enabled Information and Services Science
Focused relevantcontent
organizedby topic
(semantic categorization)
Automatic ContentAggregation
from multiple content providers and feeds
Related relevant content not explicitly asked for (semantic
associations)
Competitive research inferred
automatically
Automatic 3rd party content
integration
Semantic Application Example – Research Dashboard (Voquette/Semagix: 2001-2004)
Knowledge Enabled Information and Services Science
Watch list Organization
Company
Hamas
WorldCom
FBI Watchlist
Ahmed Yaseer
appears on Watchlistmember of organization
works for Company
Ahmed Yaseer:• Appears on Watchlist ‘FBI’
• Works for Company ‘WorldCom’
• Member of organization ‘Hamas’
Early Semantic Association: An Application in Risk & Compliance (Semagix 2004-2006)
Knowledge Enabled Information and Services Science
Global Investment Bank
Example of Fraud
prevention applicationused in financial services
User will be able to navigate the ontology using a number of different interfaces
World Wide Web content
Public Records
BLOGS,RSS
Un-structure text, Semi-structured Data
Watch ListsLaw
Enforcement Regulators
Semi-structured Government Data
Scores the entity based on the content and entity relationships
EstablishingNew Account
Knowledge Enabled Information and Services Science
demostration
Knowledge Enabled Information and Services Science
How Background Knowledge helps Informal Content Analysis
Knowledge Enabled Information and Services Science
A Community’s Pulse is often Informal
•Wealth of information available in blogs, social networks, chats etc.
•Free medium of self-expression makes mass opinions / interests available
•Polling for popular culture opinions is easier
•Social Production undeniably affects markets• geo-specific retail ads, demographic interests in music
Knowledge Enabled Information and Services Science
Background Knowledge Improves Content Analysis
Metadata creation Example – music comments Spot Artist, Track names and associated sentiments
Example comment “Keep your smile on Lil.”
Smile here is a track from Artist Lilly Allen’s album Background knowledge from Music Brainz taxonomy provides
evidence Annotate ‘smile’ as Track ‘Lil’ as Lilly Allen
Background knowledge from Urban Dictionary for understanding Slang I say: “Your music is wicked” What I really mean: “Your music is good”
Knowledge Enabled Information and Services Science
Results - Pulse of a Music Community
•Mining artist popularity from chatter on MySpace- Lists close to listeners preferences vs. Bill Boards
BB User Comments: May 07
User Comments: Jun 07
Rihanna
Biffy Clyro
Twang
Maroon 5
McCartney
Winehouse
Rascal
Rihanna
Winehouse
Maroon 5
Mccartney
Biffy Clyro
Twang
Rascal
Rihanna
Winehouse
Maroon 5
Mccartney
Biffy Clyro
Rascal
Twang
Knowledge Enabled Information and Services Science
830.9570 194.9604 2580.2985 0.3592688.3214 0.2526779.4759 38.4939784.3607 21.7736
1543.7476 1.38221544.7595 2.99771562.8113 37.47901660.7776 476.5043
parent ion m/z
fragment ion m/z
ms/ms peaklist data
fragment ionabundance
parent ionabundance
parent ion charge
Mass Spectrometry (MS) Data
Semantic Extraction/Annotation of Experimental Data
Knowledge Enabled Information and Services Science 73
Semantic Sensor Web
Semantically Annotated O&M
<swe:component name="time"><swe:Time definition="urn:ogc:def:phenomenon:time" uom="urn:ogc:def:unit:date-time">
<sa:swe rdfa:about="?time" rdfa:instanceof="time:Instant"><sa:sml rdfa:property="xs:date-time"/>
</sa:swe></swe:Time></swe:component><swe:component name="measured_air_temperature"><swe:Quantity definition="urn:ogc:def:phenomenon:temperature“
uom="urn:ogc:def:unit:fahrenheit"><sa:swe rdfa:about="?measured_air_temperature“
rdfa:instanceof=“senso:TemperatureObservation"><sa:swe rdfa:property="weather:fahrenheit"/><sa:swe rdfa:rel="senso:occurred_when" resource="?time"/><sa:swe rdfa:rel="senso:observed_by" resource="senso:buckeye_sensor"/>
</sa:sml></swe:Quantity></swe:component>
<swe:value name=“weather-data">2008-03-08T05:00:00,29.1</swe:value>
Semantic Sensor Web @ Kno.e.sis74
Knowledge Enabled Information and Services Science 75
Person
Company
Coordinates
Coordinate System
Time Units
Timezone
SpatialOntology
DomainOntology
TemporalOntology
Mike Botts, "SensorML and Sensor Web Enablement," Earth System Science Center, UAB Huntsville
Semantic Sensor ML – Adding Ontological Metadata
Knowledge Enabled Information and Services Science
Semantic Query
Semantic Temporal Query
Model-references from SML to OWL-Time ontology concepts provides the ability to perform semantic temporal queries
Supported semantic query operators include:– contains: user-specified interval falls wholly within a sensor reading
interval (also called inside)– within: sensor reading interval falls wholly within the user-specified interval
(inverse of contains or inside)– overlaps: user-specified interval overlaps the sensor reading interval
Example SPARQL query defining the temporal operator ‘within’
76
Knowledge Enabled Information and Services Science
demo of Semantic Sensor Webhttp://knoesis.wright.edu/research/semsci/application_domain/sem_sensor/
Knowledge Enabled Information and Services Science
Relationship/Fact Extraction from Text
Knowledge Engineering approach–Manually crafted rules
• Over lexical items <person> works for <organization>• Over syntactic structures – parse trees
–GATE
Knowledge Enabled Information and Services Science
Relationship/Fact Extraction from Text
Machine learning approaches–Supervised–Semi-supervised –Unsupervised
Knowledge Enabled Information and Services Science
Schema-Driven Extraction of Relationships from Biomedical Text
Cartic Ramakrishnan, Krys Kochut, Amit P. Sheth: A Framework for Schema-Driven Relationship Discovery from Unstructured Text. International Semantic Web Conference 2006: 583-596 [.pdf]
Knowledge Enabled Information and Services Science
Method – Parse Sentences in PubMed
SS-Tagger (University of Tokyo)
SS-Parser (University of Tokyo)
(TOP (S (NP (NP (DT An) (JJ excessive) (ADJP (JJ endogenous) (CC or) (JJ exogenous) ) (NN stimulation) ) (PP (IN by) (NP (NN estrogen) ) ) ) (VP (VBZ induces) (NP (NP (JJ adenomatous) (NN hyperplasia) ) (PP (IN of) (NP (DT the) (NN endometrium) ) ) ) ) ) )
• Entities (MeSH terms) in sentences occur in modified forms• “adenomatous” modifies “hyperplasia”• “An excessive endogenous or exogenous stimulation” modifies “estrogen”
• Entities can also occur as composites of 2 or more other entities• “adenomatous hyperplasia” and “endometrium” occur as “adenomatous
hyperplasia of the endometrium”
Knowledge Enabled Information and Services Science
Method – Identify entities and Relationships in Parse Tree
TOP
NP
VP
S
NPVBZ
induces
NPPP
NPINof
DTthe
NNendometrium
JJadenomatous
NNhyperplasia
NP PPINby
NNestrogenDT
theJJ
excessive ADJP NNstimulation
JJendogenous
JJexogenous
CCor
ModifiersModified entitiesComposite Entities
Knowledge Enabled Information and Services Science
Resulting Semantic Web Data in RDF
ModifiersModified entitiesComposite Entities
estrogen
An excessive endogenous or
exogenous stimulation
modified_entity1composite_entity1
modified_entity2
adenomatous hyperplasia
endometrium
hasModifier
hasPart
induces
hasPart
hasPart
hasModifier
hasPart
Knowledge Enabled Information and Services Science
Blazing Semantic Trails in Biomedical Literature
Cartic Ramakrishnan, Extracting, Representing and Mining Semantic Metadata from Text: Facilitating Knowledge Discovery in Biomedicine, PhD Thesis, Wright State University, August 2008.
Amit Sheth and Cartic Ramakrishnan, “Relationship Web: Blazing Semantic Trails between Web Resources,” IEEE Internet Computing, July–August 2007, pp. 84–88.
Knowledge Enabled Information and Services Science
Relationships -- Blazing the Trails
“The physician, puzzled by her patient's reactions, strikes the trail established in studying an earlier similar case, and runs rapidly through analogous case histories, with side references to the classics for the pertinent anatomy and histology.
The chemist, struggling with the synthesis of an organic compound, has all the chemical literature before him in his laboratory, with trails following the analogies of compounds, and side trails to their physical and chemical behavior.”
[V. Bush, As We May Think. The Atlantic Monthly, 1945. 176(1): p. 101-108. ]
Knowledge Enabled Information and Services Science
Semantic Browser
Knowledge Enabled Information and Services Science
demo of Semantic Browserhttp://knoesis.wright.edu/research/semweb/projects/textMining/iswc2008/
Knowledge Enabled Information and Services Science
Original Documents
PMID-10037099
PMID-15886201
Knowledge Enabled Information and Services Science
Semantic Trail
Knowledge Enabled Information and Services Science
“Everything's connected, all along the line. Cause and effect.
That's the beauty of it. Our job is to trace the connections and reveal them.”
Jack in Terry Gilliam’s 1985 film - “Brazil”
How are Harry Potter and Dan Brown related?
Leonardo Da Vinci
The Da Vinci code
The Louvre
Victor Hugo
The Vitruvian man
Santa Maria delle Grazie
Et in Arcadia EgoHoly Blood, Holy Grail
Harry Potter
The Last Supper
Nicolas Poussin
Priory of Sion
The Hunchback of Notre Dame
The Mona Lisa
Nicolas Flammel
painted_by
painted_by
painted_by
painted_by
member_of
member_of
member_of
written_by
mentioned_in
mentioned_in
displayed_at
displayed_at
cryptic_motto_of
displayed_at
mentioned_in
mentioned_in
How are Harry Potter and Dan Brown related?
Knowledge Enabled Information and Services Science
Semantic Trails Over All Types of Data
Semantic Trails can be built over a Web of Semantic (Meta)Data extracted (manually, semi-automatically and automatically) and gleaned from –Structured data (e.g., NCBI databases)–Semi-structured data (e.g., XML based and semantic
metadata standards for domain specific data representations and exchanges)
–Unstructured data (e.g., Pubmed and other biomedical literature)
and–Various modalities (experimental data, medical images, etc.)
Knowledge Enabled Information and Services Science
Discovering Complex Connection Patterns
Discovering informative subgraphsGiven a pair of end-points (entities) produce a subgraph with relationships connecting them such that the subgraph is small enough to be visualized and contains relevant “interesting” connections
Cartic Ramakrishnan, William H. Milnor, Matthew Perry, Amit P. Sheth. (2005). Discovering informative connection subgraphs in multi-relational graphs. SIGKDD Explorations, 7(2), pp. 56-63.
Knowledge Enabled Information and Services Science
Discovering Complex Connection Patterns
We defined an interestingness measure based on the ontology schema –In future biomedical domain the scientist will control this
with the help of a browsable ontology–Our interestingness measure takes into account
• Specificity of the relationships and entity classes involved• Rarity of relationships etc.
Cartic Ramakrishnan, William H. Milnor, Matthew Perry, Amit P. Sheth. (2005). Discovering informative connection subgraphs in multi-relational graphs. SIGKDD Explorations, 7(2), pp. 56-63.
Knowledge Enabled Information and Services Science
Heuristics
Two factor influencing interestingness
Knowledge Enabled Information and Services Science
Discovery Algorithm
• Bidirectional lock-step growth from S and T• Choice of next node based on interestingness measure• Stop when there are enough connections between
the frontiers• This is treated as the candidate graph
Knowledge Enabled Information and Services Science
Discovery Algorithm
Model the Candidate graph as an electrical circuit–S is the source and T the sink–Edge weight derived from the ontology schema are treated as
conductance values–Using Ohm’s and Kirchoff’s laws we find maximum current
flow paths through the candidate graph from S to T–At each step adding this path to the output graph to be
displayed we repeat this process till a certain number of predefined nodes is reached
Knowledge Enabled Information and Services Science
Discovery Algorithm
Results–Arnold Schwarzenegger, Edward Kennedy
Other related work–Semantic Associations
Knowledge Enabled Information and Services Science
Results
Knowledge Enabled Information and Services Science
Hypothesis Driven Retrieval Of Scientific Text
Knowledge Enabled Information and Services Science
Spatial
Temporal
Thematic
Events: 3 Dimensions – Spatial, Temporal and Thematic
Knowledge Enabled Information and Services Science
Events and STT dimensions
Powerful mechanism to integrate content–Describes the Real-World occurrences–Can have video, images, text, audio all of the same event–Search and Index based on events and STT relations
Knowledge Enabled Information and Services Science
Events and STT dimensions
Many relationship typesSpatial: –What events happened near this event? –What entities/organizations are located nearby?
Temporal: –What events happened before/after/during this event?
Thematic:–What is happening?–Who is involved?
Knowledge Enabled Information and Services Science
Events and STT dimensions
Going furtherCan we use –What? Where? When? Who?
To answer –Why? / How?
Use integrated STT analysis to explore cause and effect
Knowledge Enabled Information and Services Science 105
High-level Sensor (S-H)Low-level Sensor (S-L)
A-H E-HA-L E-L
H L
• How do we determine if A-H = A-L? (Same time? Same place?)
• How do we determine if E-H = E-L? (Same entity?)
• How do we determine if E-H or E-L constitutes a threat?
Example Scenario: Sensor Data Fusion and Analysis
Knowledge Enabled Information and Services Science 106
Sensor Data Pyramid
Raw Sensor (Phenomenological) Data
Feature Metadata
Entity Metadata
Relationship
Metadata
Data
Information
Semantics/Understanding/Insight
Data Pyramid
Knowledge Enabled Information and Services Science 107
Collection
Feature Extraction
Entity Detection
RDFKB
Semantic Analysis
Data
• Raw Phenomenological Data
TML
Fusion
SML-S
SML-S
SML-S
Ontologies
Information
• Entity Metadata
• Feature Metadata
Knowledge
• Object-Event Relations
• Spatiotemporal Associations
• Provenance Pathways
Sensors (RF, EO, IR, HIS, acoustic)
• Object-Event Ontology
• Space-Time Ontology
Analysis Processes
Annotation Processes
O&M
Sensor Data Architecture
Knowledge Enabled Information and Services Science
Current Research Towards STT Relationship Analysis
Modeling Spatial and Temporal data using SW standards (RDF(S))1
–Upper-level ontology integrating thematic and spatial dimensions
–Use Temporal RDF3 to encode temporal properties of relationships
–Demonstrate expressiveness with various query operators built upon thematic contexts
1. Matthew Perry, Farshad Hakimpour, Amit Sheth. "Analyzing Theme, Space and Time: An Ontology-based Approach", Fourteenth International Symposium on Advances in Geographic Information Systems (ACM-GIS '06), Arlington, VA, November 10 - 11, 2006
2. Matthew Perry, Amit Sheth, Farshad Hakimpour, Prateek Jain. “Supporting Complex Thematic, Spatial and Temporal Queries over Semantic Web Data", Second International Conference on Geospatial Semantics (GeoS ‘07), Mexico City, MX, November 29 –30, 2007
3. Claudio Gutiérrez, Carlos A. Hurtado, Alejandro A. Vaisman. “Temporal RDF”, ESWC 2005: 93-107
Knowledge Enabled Information and Services Science
Current Research Towards STT Relationship Analysis
Graph Pattern queries over spatial and temporal RDF data2
–Extended ORDBMS to store and query spatial and temporal RDF
–User-defined functions for graph pattern queries involving spatial variables and spatial and temporal predicates
–Implementation of temporal RDFS inferencing
1. Matthew Perry, Farshad Hakimpour, Amit Sheth. "Analyzing Theme, Space and Time: An Ontology-based Approach", Fourteenth International Symposium on Advances in Geographic Information Systems (ACM-GIS '06), Arlington, VA, November 10 - 11, 2006
2. Matthew Perry, Amit Sheth, Farshad Hakimpour, Prateek Jain. “Supporting Complex Thematic, Spatial and Temporal Queries over Semantic Web Data", Second International Conference on Geospatial Semantics (GeoS ‘07), Mexico City, MX, November 29 –30, 2007
3. Claudio Gutiérrez, Carlos A. Hurtado, Alejandro A. Vaisman. “Temporal RDF”, ESWC 2005: 93-107
Knowledge Enabled Information and Services Science
Upper-Level Ontology Modeling Theme and Space
OccurrentContinuant
Named_PlaceSpatial_OccurrentDynamic_Entity
Spatial_Region
Occurrent: Events – happen and then don’t existContinuant: Concrete and Abstract Entities – persist over time
Named_Place: Those entities with static spatial behavior (e.g. building)
Dynamic_Entity: Those entities with dynamic spatial behavior (e.g. person)
Spatial_Occurrent: Events with concrete spatial locations (e.g. a speech)
Spatial_Region: Records exact spatial location (geometry objects, coordinate system info)
occurred_at located_at
occurred_at: Links Spatial_Occurents to their geographic locationslocated_at: Links Named_Places to their geographic locations
rdfs:subClassOfproperty
Spatio-Temporal-Thematic Query Processing @ Kno.e.sis
Knowledge Enabled Information and Services Science
OccurrentContinuant
Named_Place
Spatial_OccurrentDynamic_Entity
PersonCity
Politician
SoldierMilitary_Unit
BattleVehicle
Bombing
Speech
Military_Eventassigned_to
on_crew_ofused_in
gives
participates_in
trains_at
Spatial_Region
located_at occurred_at
Upper-level Ontology
Domain Ontology
rdfs:subClassOf used for integrationrdfs:subClassOfrelationship type
Knowledge Enabled Information and Services Science
Temporal RDF Graph: Platoon Membership
E1:Soldier
E3:Platoon E5:Soldier
E2:Platoon
E4:Soldierassigned_to [1, 10]
assigned_to [11, 20]
assigned_to [5, 15]
assigned_to [5, 15]
E1 is assigned to E2 from time 1 to 10 and then assigned to E3 from time 11 to 20
Also need to handle inferencing:
(x rdf:type Grad_Student):[2004, 2006] AND(x rdf:type Undergrad_Student):[2000, 2004]
(x rdf:type Student):[2000, 2006]
Time interval represents valid time of the relationship
Knowledge Enabled Information and Services Science
ORDBMS Implementation: DB Structures
Unlike thematic relationships which are explicitly stated in the RDF graph, many spatial and temporal relationships (e.g., distance) are implicit and require additional computation
Knowledge Enabled Information and Services Science
Sample STT Query
Scenario (Biochemical Threat Detection): Analysts must examine soldiers’ symptoms to detect possible biochemical attackQuery specifies a relationship between a soldier, a chemical agent and a battle location (graph pattern 1)a relationship between members of an enemy organization and their known locations (graph pattern 2)a spatial filtering condition based on the proximity of the soldier and the enemy group in this context (spatial Constraint)
Knowledge Enabled Information and Services Science
Going places …
Formal
Social,InformalImplicit
Powerful
Knowledge Enabled Information and Services Science
looking into my crystal ball
Knowledge Enabled Information and Services Science
TECHNOLOGY THAT FITS RIGHT IN
“The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.
Machines that fit the human environment instead of forcing humans to enter theirs will make using a computer as refreshing as a walk in the woods.” Mark Weiser, The Computer for the 21st Century (Ubicomp vision)
Get citation117
Knowledge Enabled Information and Services Science
imagine
Knowledge Enabled Information and Services Science
imagine when
Knowledge Enabled Information and Services Science
meets
Farm Helper
Knowledge Enabled Information and Services Science
with this
Latitude: 38° 57’36” NLongitude: 95° 15’12” WDate: 10-9-2007Time: 1345h
Knowledge Enabled Information and Services Science
that is sent to
Geocoder
Farm Helper
Services Resource
Sensor Data ResourceStructured Data Resource
Agri DBSoil
Survey
Lat-Long
Soil InformationPest information …
Weather Resource
LocationDate /Time
Weather Data
Knowledge Enabled Information and Services Science
and
Knowledge Enabled Information and Services Science
Six billion brains
Knowledge Enabled Information and Services Science
imagination today
Knowledge Enabled Information and Services Science
impacts our experience tomorrow
Knowledge Enabled Information and Services Science
Knowledge Enabled Information and Services Science
Computing For Human Experience
Prof. Amit P. Sheth,LexisNexis Eminent Scholar and
Director, kno.e.sis centerWright State University
http://knoesis.org