Upload
isabel-barr
View
214
Download
0
Embed Size (px)
Citation preview
Data MiningData Mining
David EichmannSchool of Library and Information Science
The University of Iowa
David EichmannSchool of Library and Information Science
The University of Iowa
Why?Why?
Given enough data represented through enough dimensions, we loose the ability to see the patterns
Given enough data represented through enough dimensions, we loose the ability to see the patterns
How?How?
Decision TreesNearest Neighbor ClusteringNeural NetworksRule InductionK-Means Clustering
Decision TreesNearest Neighbor ClusteringNeural NetworksRule InductionK-Means Clustering
What is it?What is it?
The automated extraction of hidden predictive information from databases.
Key pointsAutomatedHiddenPredictive
The automated extraction of hidden predictive information from databases.
Key pointsAutomatedHiddenPredictive
The Typical ProcessThe Typical Process
Evaluation CriteriaEvaluation Criteria
Receiver Operating Characteristic Curves
Receiver Operating Characteristic Curves
But Nobody Said We Had To Do MATH….
But Nobody Said We Had To Do MATH….
Forms of DataForms of Data
StructuredDatabasesForms
Semi-StructuredTables on the WebBibliographic citationsGraphs & charts
UnstructuredFull text (e.g., journal articles, physician chart notes)
Images
StructuredDatabasesForms
Semi-StructuredTables on the WebBibliographic citationsGraphs & charts
UnstructuredFull text (e.g., journal articles, physician chart notes)
Images
Text MiningText Mining
Corpus now is a collection of text artifactsFull text when you’ve got it (e.g. newswire)Metadata when you don’t (e.g. MEDLINE)
The trick then becomes extracting ‘interesting’ relationships between ‘interesting’ entitiesWho killed whoWho works for whoWho makes what
Corpus now is a collection of text artifactsFull text when you’ve got it (e.g. newswire)Metadata when you don’t (e.g. MEDLINE)
The trick then becomes extracting ‘interesting’ relationships between ‘interesting’ entitiesWho killed whoWho works for whoWho makes what
The Classic EntitiesThe Classic Entities
PersonsOrganizationsPlaces (Geography)Events
PersonsOrganizationsPlaces (Geography)Events
A Newswire ExampleA Newswire Example
APW19981001.0262 [Israel(0.271), Jonathan Pollard (0.153), Benjamin Netanyahu(0.102), Bill Clinton(0.102), United States(0.055), ...]
Persons Bill Clinton (3) Jonathan Pollard (8) Moshe Fogel (2) Benjamin Netanyahu (2) Israeli Embassy (1)
Organizations Cabinet (1)
Places Israel (16) United States (5) Washington (2)
APW19981001.0262 [Israel(0.271), Jonathan Pollard (0.153), Benjamin Netanyahu(0.102), Bill Clinton(0.102), United States(0.055), ...]
Persons Bill Clinton (3) Jonathan Pollard (8) Moshe Fogel (2) Benjamin Netanyahu (2) Israeli Embassy (1)
Organizations Cabinet (1)
Places Israel (16) United States (5) Washington (2)
In the Medical/Health RealmIn the Medical/Health Realm
UMLS an excellent frameworkOrganismChemicalActivityDisease
UMLS an excellent frameworkOrganismChemicalActivityDisease
A MEDLINE ExampleA MEDLINE Example
Document: 89316090 - Reconstructive surgery in Nicaragua Provided MeSH Keywords
Human Nicaragua
Z01.107.169.690 Surgery, Plastic/*
G02.403.810.788 Phrases
[Reconstructive, surgery] [Nicaragua] [letter]
MeSH Terms Surgery (1) G02.403.810.762 Letter [Publication Type] (1)
Other Phrases Reconstructive surgery (1)
Document: 89316090 - Reconstructive surgery in Nicaragua Provided MeSH Keywords
Human Nicaragua
Z01.107.169.690 Surgery, Plastic/*
G02.403.810.788 Phrases
[Reconstructive, surgery] [Nicaragua] [letter]
MeSH Terms Surgery (1) G02.403.810.762 Letter [Publication Type] (1)
Other Phrases Reconstructive surgery (1)
Concept Extraction Example
Concept Extraction Example
“Roman forces under Julius Caesar invade Britain.”(S (NP (NP Roman forces) (PP under (NP Julius Caesar))) (VP invade (NP Britain)) .)
Entity Attributes: <organization Roman forces> <person Julias Caesar> <placename Britain>
Concepts: <Roman forces - under - Julius Caesar> <Roman forces - invade - Britain>
“Roman forces under Julius Caesar invade Britain.”(S (NP (NP Roman forces) (PP under (NP Julius Caesar))) (VP invade (NP Britain)) .)
Entity Attributes: <organization Roman forces> <person Julias Caesar> <placename Britain>
Concepts: <Roman forces - under - Julius Caesar> <Roman forces - invade - Britain>
And a Small Demo…And a Small Demo…