Data Mining E. Schubert, E. Ntoutsi Data Mining Tutorial

Data MiningTutorial

E. Schubert,E. Ntoutsi

ExamplesInduced metric

Data Mining TutorialSession 3: Distance functions homework

Erich Schubert, Eirini Ntoutsi

Ludwig-Maximilians-Universität München

2012-05-24 — KDD class tutorial

Data MiningTutorial

Distance functions

I Reflexive: x = y ⇒ d(x, y) = 0“Distance to self is 0”

I Symmetric: d(x, y) = a ⇔ d(y, x) = a“Order of arguments is irrelevant”

I Strict: d(x, y) = 0 ⇒ x = y“Only identical elements have distance 0”

I Triangle inequality: d(x, y) ≤ d(x, z) + d(z, y)“Directly x to y is at least as short as a detour over z”

You will need to know these properties!

Data MiningTutorial

An important reminder

You cannot prove by example.

... but you can disprove by example!

Please, show that it holds for all situations, or give acounterexample. Do not give a positive example.

Data MiningTutorial

Distances Homework – difference sum

d(x, y) =n∑

(xi − yi)

d((0), (−1)) = −1 – must not be negative!

Data MiningTutorial

Distances Homework – difference sum

d(x, y) =n∑

(xi − yi)

d((0), (−1)) = −1 – must not be negative!

Data MiningTutorial

Distances Homework – squared Euclidean

d(x, y) =n∑

(xi − yi)2

Reflexive, symmetric, strict: obvious. Triangle inequality?

How about o = (0, 0), p = (1, 0), q = (2, 0)?

d(o, q) = 4

d(o, p) + d(p, q) = 1 + 1 = 2

“Squared Euclidean distance” – not metrical.(1 dimensional counter example: 0, 1, 2)

Data MiningTutorial

d(x, y) =n∑

(xi − yi)2

How about o = (0, 0), p = (1, 0), q = (2, 0)?

d(o, q) = 4

d(o, p) + d(p, q) = 1 + 1 = 2

Data MiningTutorial

d(x, y) =n∑

(xi − yi)2

How about o = (0, 0), p = (1, 0), q = (2, 0)?

d(o, q) = 4

d(o, p) + d(p, q) = 1 + 1 = 2

Data MiningTutorial

d(x, y) =n∑

(xi − yi)2

How about o = (0, 0), p = (1, 0), q = (2, 0)?

d(o, q) = 4 6≤ d(o, p) + d(p, q) = 1 + 1 = 2

Data MiningTutorial

Distances Homework – projected Euclidean

d(x, y) =

√√√√n−1∑i=1

(xi − yi)2

Reflexive, symmetric: obvious. Triangle inequality requiressome work.

But not strict: dimension n is ignored.

Data MiningTutorial

d(x, y) =

√√√√n−1∑i=1

(xi − yi)2

Data MiningTutorial

d(x, y) =

√√√√n−1∑i=1

(xi − yi)2

Data MiningTutorial

Distances Homework – concordance

d(x, y) =n∑

{1 iff xi = yi

0 iff xi 6= yi

d is not reflexive – the other properties are irrelevant to us.

Data MiningTutorial

Distances Homework – concordance

d(x, y) =n∑

{1 iff xi = yi

0 iff xi 6= yi

d is not reflexive – the other properties are irrelevant to us.

Data MiningTutorial

Distances Homework – Hamming distance

d(x, y) =n∑

{1 iff xi 6= yi

0 iff xi = yi

Discordance on binary vectors.“Number of ones after an XOR of the two vectors”.Important metric from information theory.Reflexivity, strictness, symmetry are obvious.

Data MiningTutorial

d(x, y) =n∑

{1 iff xi 6= yi

0 iff xi = yi

Discordance on binary vectors.“Number of ones after an XOR of the two vectors”.Important metric from information theory.Reflexivity, strictness, symmetry are obvious.

Data MiningTutorial

Proof of triangle inequality by case distinction on theindividual positions (dimensions):

Data MiningTutorial

Proof of triangle inequality by case distinction on theindividual positions (dimensions):A) xi = yi ∧ yi = zi:

d(xi, yi) + d(yi, zi) ≥ d(xi, zi)

d(xi, xi) + d(yi, xi) ≥ d(xi, xi)

0 + 0 ≥ 0

Data MiningTutorial

Proof of triangle inequality by case distinction on theindividual positions (dimensions):B) xi = yi ∧ xi 6= zi:

d(xi, xi) + d(xi, zi) ≥ d(xi, zi)

0 + 1 ≥ 1

Data MiningTutorial

Proof of triangle inequality by case distinction on theindividual positions (dimensions):C) xi = zi ∧ xi 6= yi:

d(xi, yi) + d(yi, xi) ≥ d(xi, xi)

1 + 1 ≥ 0

Data MiningTutorial

Proof of triangle inequality by case distinction on theindividual positions (dimensions):D) xi 6= yi ∧ yi = zi:

d(xi, yi) + d(yi, yi) ≥ d(xi, yi)

1 + 0 ≥ 1

Data MiningTutorial

Proof of triangle inequality by case distinction on theindividual positions (dimensions):E) xi 6= yi ∧ yi 6= zi ∧ xi 6= zi:

1 + 1 ≥ 1

Data MiningTutorial

Proof of triangle inequality by case distinction on theindividual positions (dimensions):Which implies:

d(x, y) + d(y, z) =

d(xi, yi) +

d(yi, zi)

(d(xi, yi) + d(yi, zi))

≥n∑i

d(xi, zi) = d(x, z)

(We have just shown the step line 2 to 3!)

Data MiningTutorial

Distances Homework – Other examples

Other interesting distance functions (on sets X,Y ⊆ Rn),for existing distance measures d : Rn × Rn → R+

I single-link(X,Y) = minx∈X,y∈Y d(x, y)I average-link(X,Y) = 1

|X|·|Y| ·∑

x∈X,y∈Y d(x, y)

I complete-link(X,Y) = maxx∈X,y∈Y d(x, y)

They will be discussed in detail in the clustering chapter!

Data MiningTutorial

Distances Homework – Other examples

There are hundreds of distance functions.I For time series: DTW, EDR, ERP, LCSS, . . .I For text: Cosine and normalizationsI For sets – based on intersection, union, . . .I For clusters (single-link etc.)I For histograms: histogram intersection, “Earth movers

distance”, quadratic forms with color similarityI With normalization: Canberra, . . .I Quadratic forms / bilinear forms: d(x, y) := xTMy for

some positive (usually symmetric) definite matrix M.

Can be seen as a part of “preprocessing”:choosing the appropriate distance function!

Data MiningTutorial

Induced metric

Given a pseudo-metric d on the set A: d : A× A→ R+0 .

Define the equivalence relation ∼ such thatx ∼ y⇔ d(x, y) = 0.

Let A∼ be the set of equivalence classes of A wrt. ∼.

d∼ : A∼ × A∼ → R+0

with d∼(x∼, y∼) := d(x, y)

Properties?

Well defined?

Data MiningTutorial

Induced metric

Given a pseudo-metric d on the set A: d : A× A→ R+0 .

Define the equivalence relation ∼ such thatx ∼ y⇔ d(x, y) = 0.

Let A∼ be the set of equivalence classes of A wrt. ∼.

d∼ : A∼ × A∼ → R+0

with d∼(x∼, y∼) := d(x, y)

Properties? Well defined?

Data MiningTutorial

Induced metric

Verify for any z ∈ x∼ and w ∈ y∼ that d(z,w) = d(x, y).

Since z ∈ x∼ and w ∈ y∼ we have

z∼ = x∼ and d(z, x) = 0w∼ = y∼ and d(w, y) = 0

Use the triangle inequality twice:

d(z,w) ≤ d(z, x) + d(x, y) + d(y,w) ≤ d(x, y)

d(x, y) ≤ d(x, z) + d(z,w) + d(w, y) ≤ d(z,w)

Any element from the equivalence class gives the samedistance for d∼. ⇒ well defined on A∼.

Data MiningTutorial

Induced metric

Data MiningTutorial

Induced metric

Data MiningTutorial

Induced metric

Need to show:d∼(a∼, b∼) = 0⇔ a∼ = b∼

d∼(a∼, b∼) = 0⇒ d(a, b) = 0⇒ a ∼ b⇒ a∼ = b∼

Symmetry, triangle inequality inherited from d!

Data MiningTutorial

Induced metric

d∼(a∼, b∼) = 0⇒ d(a, b) = 0⇒ a ∼ b⇒ a∼ = b∼

Data MiningTutorial

Induced metric

d∼(a∼, b∼) = 0⇒ d(a, b) = 0⇒ a ∼ b⇒ a∼ = b∼

Data MiningTutorial

Induced metric

euclidxy((r1, x1, y1), (r2, x2, y2)) =√

(x1 − x2)2 + (y1 − y2)2

Record number x y1 0 12 1 13 0 1

Euclidean distance on X × Y. Metric in R2 ∼ X × Y,but only a Pseudo-metric on Record number× X × Y.“Duplicate” records have a distance of 0.

Data MiningTutorial

Induced metric

euclidxy((r1, x1, y1), (r2, x2, y2)) =√

(x1 − x2)2 + (y1 − y2)2

Euclidean distance on X × Y. Metric in R2 ∼ X × Y,but only a Pseudo-metric on Record number× X × Y.“Duplicate” records have a distance of 0.

Data Mining E. Schubert, E. Ntoutsi Data Mining Tutorial

Documents

A9 schubert

Data Mining Tutorial - LMU Munich · Data Mining Tutorial E. Schubert, A. Zimek Aufgabe 6-1 Aufgabe 6-2 Aufgabe 6-3 single-link average-link complete-link Data Mining Tutorial Clusteranalyse

Data Mining Tutorial - LMU Munich · Data Mining Tutorial E. Schubert, A. Zimek Outlier detection Aufgabe 8-1 Data Mining Tutorial Outlier Detection Erich Schubert, Arthur Zimek Ludwig-Maximilians-Universität

Schubert - Schwanengesang

schubert D959

SCHUBERT FANTAISIE

Data Mining Tutorial - Frequent Itemset Mining · Data Mining Tutorial E. Schubert, A. Zimek Apriori FIM Fazit NetFlix und Frequent Itemsets Assoziationsregeln aus NetFlix-Daten:

Data Mining E. Schubert, E. Ntoutsi Data Mining Tutorial · 2012-05-08 · Data Mining Tutorial E. Schubert, E. Ntoutsi Iris data Tools Weka ELKI SciPy GNU R Summary Data Mining Tutorial

Data Mining Tutorial - uni-muenchen.de · Tutorial E. Schubert, E. Ntoutsi Aufgabe 8-1 + Aufgabe 8-2 Old Faithful EM Demo Data Mining Tutorial Session 5: Clustering Erich Schubert,

Schubert franz Schubert Die schöne Müllerin · 2010-08-26 · 1 Schubert üllerin euniëre énor no Schubert üllerin euniëre énor Schubert Die schöne Müllerin La Belle Meuniëre

Advanced Topics in Text Mining - uni-heidelberg.de · Advanced Topics in Text Mining Summer Term 2017: Text Clustering & Topic Modeling Erich Schubert Lehrstuhl für Datenbanksysteme,

Schubert - Serenata

Tutorial on Schubert Varieties and Schubert Calculus - icerm

Serenata Schubert

Serenatade Schubert

Schubert Lieder

Schubert - Erlkönig

Schubert & Salzer - Your Partner for Mining and Metallurgy · Schubert & Salzer - Your Partner for Mining and Metallurgy jatuporn1979 - 123rf.de, industrieblick- Fotolia.com. Benefit

Data Mining Tutorial - Session 4: Classification · Data Mining Tutorial E. Schubert, E. Ntoutsi Aufgabe 3-3 Aufgabe 4-1 Aufgabe 4-2 Aufgabe 4-3 Aufgabe 5-1 Aufgabe 5-2 Aufgabe 5-3

Prof. Dr. Franz Schubert - HAW Hamburgusers.etech.haw-hamburg.de/users/schubert/bu/Task5_DCC.pdf · 21.05.2013 3 Bussystems and Sensors Prof. Dr. Franz Schubert Prof. Dr. Franz Schubert