Embed Size (px)
Citation preview
Graph Based Semi-supervised Learning
Aydın Gerek
Marmara University
May 26th, 2018
Aydın Gerek Graph Based Semi-supervised Learning
Learning Paradigms
Supervised Learning
Unsupervised LearningSemi-supervised Learning
InductiveTransductive
Aydın Gerek Graph Based Semi-supervised Learning
Learning Paradigms
Supervised LearningUnsupervised Learning
Semi-supervised Learning
InductiveTransductive
Aydın Gerek Graph Based Semi-supervised Learning
Learning Paradigms
Supervised LearningUnsupervised LearningSemi-supervised Learning
InductiveTransductive
Aydın Gerek Graph Based Semi-supervised Learning
Learning Paradigms
Supervised LearningUnsupervised LearningSemi-supervised Learning
Inductive
Transductive
Aydın Gerek Graph Based Semi-supervised Learning
Learning Paradigms
Supervised LearningUnsupervised LearningSemi-supervised Learning
InductiveTransductive
Aydın Gerek Graph Based Semi-supervised Learning
Graphs
V = set of vertices
E = set of edges
Auv =
{1 (u, v) ∈ E0 (u, v) /∈ E
Duv =
{∑w Auw u = v
0 u 6= v
∆ = D − A L = D−1/2∆D−1/2
Aydın Gerek Graph Based Semi-supervised Learning
Graphs
V = set of vertices E = set of edges
Auv =
{1 (u, v) ∈ E0 (u, v) /∈ E
Duv =
{∑w Auw u = v
0 u 6= v
∆ = D − A L = D−1/2∆D−1/2
Aydın Gerek Graph Based Semi-supervised Learning
Graphs
V = set of vertices E = set of edges
Auv =
{1 (u, v) ∈ E0 (u, v) /∈ E
Duv =
{∑w Auw u = v
0 u 6= v
∆ = D − A L = D−1/2∆D−1/2
Aydın Gerek Graph Based Semi-supervised Learning
Graphs
V = set of vertices E = set of edges
Auv =
{wuv (u, v) ∈ E0 (u, v) /∈ E
Duv =
{∑w Auw u = v
0 u 6= v
∆ = D − A L = D−1/2∆D−1/2
Aydın Gerek Graph Based Semi-supervised Learning
Graphs
V = set of vertices E = set of edges
Auv =
{wuv (u, v) ∈ E0 (u, v) /∈ E
Duv =
{∑w Auw u = v
0 u 6= v
∆ = D − A L = D−1/2∆D−1/2
Aydın Gerek Graph Based Semi-supervised Learning
Graphs
V = set of vertices E = set of edges
Auv =
{wuv (u, v) ∈ E0 (u, v) /∈ E
Duv =
{∑w Auw u = v
0 u 6= v
∆ = D − A
L = D−1/2∆D−1/2
Aydın Gerek Graph Based Semi-supervised Learning
Graphs
V = set of vertices E = set of edges
Auv =
{wuv (u, v) ∈ E0 (u, v) /∈ E
Duv =
{∑w Auw u = v
0 u 6= v
∆ = D − A L = D−1/2∆D−1/2
Aydın Gerek Graph Based Semi-supervised Learning
Gaussian Random Fields (GRF) (Zhu et al., 2003)
also known as Label Propagation
arg minY∈Rn×|Y|
∑`∈Y
∑(u,v)∈E
Auv (Yu` − Yv`)2
= arg minY∈Rn×|Y|
∑`∈Y
Y`T
∆Y`
s.t. Yu` = Yu`,∀` ∈ Y, ∀ labeled u
Y (t+1)v` ←
∑(u,v)∈E Auv Y (t)
u`
Y (t+1) ← D−1AY (t)
Harmonic Property: node = average of neighbors
Aydın Gerek Graph Based Semi-supervised Learning
Gaussian Random Fields (GRF) (Zhu et al., 2003)
also known as Label Propagation
arg minY∈Rn×|Y|
∑`∈Y
∑(u,v)∈E
Auv (Yu` − Yv`)2
= arg minY∈Rn×|Y|
∑`∈Y
Y`T
∆Y`
s.t. Yu` = Yu`,∀` ∈ Y, ∀ labeled u
Y (t+1)v` ←
∑(u,v)∈E Auv Y (t)
u`
Y (t+1) ← D−1AY (t)
Harmonic Property: node = average of neighbors
Aydın Gerek Graph Based Semi-supervised Learning
Gaussian Random Fields (GRF) (Zhu et al., 2003)
also known as Label Propagation
arg minY∈Rn×|Y|
∑`∈Y
∑(u,v)∈E
Auv (Yu` − Yv`)2 = arg min
Y∈Rn×|Y|
∑`∈Y
Y`T
∆Y`
s.t. Yu` = Yu`,∀` ∈ Y, ∀ labeled u
Y (t+1)v` ←
∑(u,v)∈E Auv Y (t)
u`
Y (t+1) ← D−1AY (t)
Harmonic Property: node = average of neighbors
Aydın Gerek Graph Based Semi-supervised Learning
Gaussian Random Fields (GRF) (Zhu et al., 2003)
also known as Label Propagation
arg minY∈Rn×|Y|
∑`∈Y
∑(u,v)∈E
Auv (Yu` − Yv`)2 = arg min
Y∈Rn×|Y|
∑`∈Y
Y`T
∆Y`
s.t. Yu` = Yu`,∀` ∈ Y, ∀ labeled u
Y (t+1)v` ←
∑(u,v)∈E Auv Y (t)
u`∑(u,v)∈E Auv
Y (t+1) ← D−1AY (t)
Harmonic Property: node = average of neighbors
Aydın Gerek Graph Based Semi-supervised Learning
Gaussian Random Fields (GRF) (Zhu et al., 2003)
also known as Label Propagation
arg minY∈Rn×|Y|
∑`∈Y
∑(u,v)∈E
Auv (Yu` − Yv`)2 = arg min
Y∈Rn×|Y|
∑`∈Y
Y`T
∆Y`
s.t. Yu` = Yu`,∀` ∈ Y, ∀ labeled u
Y (t+1)v` ←
∑(u,v)∈E Auv Y (t)
u`
Dvv
Y (t+1) ← D−1AY (t)
Harmonic Property: node = average of neighbors
Aydın Gerek Graph Based Semi-supervised Learning
Gaussian Random Fields (GRF) (Zhu et al., 2003)
also known as Label Propagation
arg minY∈Rn×|Y|
∑`∈Y
∑(u,v)∈E
Auv (Yu` − Yv`)2 = arg min
Y∈Rn×|Y|
∑`∈Y
Y`T
∆Y`
s.t. Yu` = Yu`,∀` ∈ Y, ∀ labeled u
Y (t+1)v` ←
∑(u,v)∈E Auv Y (t)
u`
Dvv
Y (t+1) ← D−1AY (t)
Harmonic Property: node = average of neighbors
Aydın Gerek Graph Based Semi-supervised Learning
Gaussian Random Fields (GRF) (Zhu et al., 2003)
also known as Label Propagation
arg minY∈Rn×|Y|
∑`∈Y
∑(u,v)∈E
Auv (Yu` − Yv`)2 = arg min
Y∈Rn×|Y|
∑`∈Y
Y`T
∆Y`
s.t. Yu` = Yu`,∀` ∈ Y, ∀ labeled u
Y (t+1)v` ←
∑(u,v)∈E Auv Y (t)
u`
Dvv
Y (t+1) ← D−1AY (t)
Harmonic Property: node = average of neighbors
Aydın Gerek Graph Based Semi-supervised Learning
Manifold Regularization (Belkin et al., 2006)
Manifold Hypothesis: The data lives in a low dimensionalmanifold embedded in high-dimensional ambient space.
f ∗(x) = arg minf∈HK
1nl
nl∑i=1
L(xi , yi , f ) + λA‖f‖2K + λI‖f‖2I
‖f‖2I =
∫x∈M‖∇Mf‖2dP(x) ≈ 1
nl + nufT ∆f
fTi = f (xi)
Aydın Gerek Graph Based Semi-supervised Learning
Manifold Regularization (Belkin et al., 2006)
Manifold Hypothesis: The data lives in a low dimensionalmanifold embedded in high-dimensional ambient space.
f ∗(x) = arg minf∈HK
1nl
nl∑i=1
L(xi , yi , f )
+ λA‖f‖2K + λI‖f‖2I
‖f‖2I =
∫x∈M‖∇Mf‖2dP(x) ≈ 1
nl + nufT ∆f
fTi = f (xi)
Aydın Gerek Graph Based Semi-supervised Learning
Manifold Regularization (Belkin et al., 2006)
Manifold Hypothesis: The data lives in a low dimensionalmanifold embedded in high-dimensional ambient space.
f ∗(x) = arg minf∈HK
1nl
nl∑i=1
L(xi , yi , f ) + λA‖f‖2K
+ λI‖f‖2I
‖f‖2I =
∫x∈M‖∇Mf‖2dP(x) ≈ 1
nl + nufT ∆f
fTi = f (xi)
Aydın Gerek Graph Based Semi-supervised Learning
Manifold Regularization (Belkin et al., 2006)
Manifold Hypothesis: The data lives in a low dimensionalmanifold embedded in high-dimensional ambient space.
f ∗(x) = arg minf∈HK
1nl
nl∑i=1
L(xi , yi , f ) + λA‖f‖2K + λI‖f‖2I
‖f‖2I =
∫x∈M‖∇Mf‖2dP(x) ≈ 1
nl + nufT ∆f
fTi = f (xi)
Aydın Gerek Graph Based Semi-supervised Learning
Manifold Regularization (Belkin et al., 2006)
Manifold Hypothesis: The data lives in a low dimensionalmanifold embedded in high-dimensional ambient space.
f ∗(x) = arg minf∈HK
1nl
nl∑i=1
L(xi , yi , f ) + λA‖f‖2K + λI‖f‖2I
‖f‖2I =
∫x∈M‖∇Mf‖2dP(x)
≈ 1nl + nu
fT ∆f
fTi = f (xi)
Aydın Gerek Graph Based Semi-supervised Learning
Manifold Regularization (Belkin et al., 2006)
Manifold Hypothesis: The data lives in a low dimensionalmanifold embedded in high-dimensional ambient space.
f ∗(x) = arg minf∈HK
1nl
nl∑i=1
L(xi , yi , f ) + λA‖f‖2K + λI‖f‖2I
‖f‖2I =
∫x∈M‖∇Mf‖2dP(x) ≈ 1
nl + nufT ∆f
fTi = f (xi)
Aydın Gerek Graph Based Semi-supervised Learning
Manifold Regularization (Belkin et al., 2006)
Manifold Hypothesis: The data lives in a low dimensionalmanifold embedded in high-dimensional ambient space.
f ∗(x) = arg minf∈HK
1nl
nl∑i=1
L(xi , yi , f ) + λA‖f‖2K + λI‖f‖2I
‖f‖2I =
∫x∈M‖∇Mf‖2dP(x) ≈ 1
nl + nufT ∆f
fTi = f (xi)
Aydın Gerek Graph Based Semi-supervised Learning
Manifold Regularization (Belkin et al., 2006)
Manifold Hypothesis: The data lives in a low dimensionalmanifold embedded in high-dimensional ambient space.
f ∗(x) = arg minf∈HK
1nl
nl∑i=1
L(xi , yi , f ) + λA‖f‖2K + λI‖f‖2I
‖f‖2I =
∫x∈M‖∇Mf‖2dP(x) ≈ 1
nl + nufT Lf
fTi = f (xi)
Aydın Gerek Graph Based Semi-supervised Learning
Graph Convolutional Networks (Kipf and Welling 2017)
X ∈ Rn×di
, Z ∈ Rn×do
H(l) ∈ Rn×dl , H(0) = X ,H(L) = Z
H(l+1) = f (H(l),A)
f (H(l),A) = σ(AH(l)W (l))
A = A + I, D−1/2AD−1/2
f (H(l),A) = σ(D−1/2AD−1/2H(l)W (l))
Aydın Gerek Graph Based Semi-supervised Learning
Graph Convolutional Networks (Kipf and Welling 2017)
X ∈ Rn×di , Z ∈ Rn×do
H(l) ∈ Rn×dl , H(0) = X ,H(L) = Z
H(l+1) = f (H(l),A)
f (H(l),A) = σ(AH(l)W (l))
A = A + I, D−1/2AD−1/2
f (H(l),A) = σ(D−1/2AD−1/2H(l)W (l))
Aydın Gerek Graph Based Semi-supervised Learning
Graph Convolutional Networks (Kipf and Welling 2017)
X ∈ Rn×di , Z ∈ Rn×do
H(l) ∈ Rn×dl
, H(0) = X ,H(L) = Z
H(l+1) = f (H(l),A)
f (H(l),A) = σ(AH(l)W (l))
A = A + I, D−1/2AD−1/2
f (H(l),A) = σ(D−1/2AD−1/2H(l)W (l))
Aydın Gerek Graph Based Semi-supervised Learning
Graph Convolutional Networks (Kipf and Welling 2017)
X ∈ Rn×di , Z ∈ Rn×do
H(l) ∈ Rn×dl , H(0) = X
,H(L) = Z
H(l+1) = f (H(l),A)
f (H(l),A) = σ(AH(l)W (l))
A = A + I, D−1/2AD−1/2
f (H(l),A) = σ(D−1/2AD−1/2H(l)W (l))
Aydın Gerek Graph Based Semi-supervised Learning
Graph Convolutional Networks (Kipf and Welling 2017)
X ∈ Rn×di , Z ∈ Rn×do
H(l) ∈ Rn×dl , H(0) = X ,H(L) = Z
H(l+1) = f (H(l),A)
f (H(l),A) = σ(AH(l)W (l))
A = A + I, D−1/2AD−1/2
f (H(l),A) = σ(D−1/2AD−1/2H(l)W (l))
Aydın Gerek Graph Based Semi-supervised Learning
Graph Convolutional Networks (Kipf and Welling 2017)
X ∈ Rn×di , Z ∈ Rn×do
H(l) ∈ Rn×dl , H(0) = X ,H(L) = Z
H(l+1) = f (H(l),A)
f (H(l),A) = σ(AH(l)W (l))
A = A + I, D−1/2AD−1/2
f (H(l),A) = σ(D−1/2AD−1/2H(l)W (l))
Aydın Gerek Graph Based Semi-supervised Learning
Graph Convolutional Networks (Kipf and Welling 2017)
X ∈ Rn×di , Z ∈ Rn×do
H(l) ∈ Rn×dl , H(0) = X ,H(L) = Z
H(l+1) = f (H(l),A)
f (H(l),A) = σ(AH(l)W (l))
A = A + I, D−1/2AD−1/2
f (H(l),A) = σ(D−1/2AD−1/2H(l)W (l))
Aydın Gerek Graph Based Semi-supervised Learning
Graph Convolutional Networks (Kipf and Welling 2017)
X ∈ Rn×di , Z ∈ Rn×do
H(l) ∈ Rn×dl , H(0) = X ,H(L) = Z
H(l+1) = f (H(l),A)
f (H(l),A) = σ(AH(l)W (l))
A = A + I
, D−1/2AD−1/2
f (H(l),A) = σ(D−1/2AD−1/2H(l)W (l))
Aydın Gerek Graph Based Semi-supervised Learning
Graph Convolutional Networks (Kipf and Welling 2017)
X ∈ Rn×di , Z ∈ Rn×do
H(l) ∈ Rn×dl , H(0) = X ,H(L) = Z
H(l+1) = f (H(l),A)
f (H(l),A) = σ(AH(l)W (l))
A = A + I, D−1/2AD−1/2
f (H(l),A) = σ(D−1/2AD−1/2H(l)W (l))
Aydın Gerek Graph Based Semi-supervised Learning
Graph Convolutional Networks (Kipf and Welling 2017)
X ∈ Rn×di , Z ∈ Rn×do
H(l) ∈ Rn×dl , H(0) = X ,H(L) = Z
H(l+1) = f (H(l),A)
f (H(l),A) = σ(AH(l)W (l))
A = A + I, D−1/2AD−1/2
f (H(l),A) = σ(D−1/2AD−1/2H(l)W (l))
Aydın Gerek Graph Based Semi-supervised Learning
![Phenotype prediction with semi-supervised learningloglisci/NFmcp17/NFMCP_2017_paper_3.pdf · Phenotype prediction with semi-supervised ... the semi-supervised cluster assumption [1]:](https://img.pdfslide.net/doc/110x75/5b8fbb9809d3f2103e8ccb95/phenotype-prediction-with-semi-supervised-logliscinfmcp17nfmcp2017paper3pdf.jpg)
