DiffusiononaGraph

Diffusion

Duetorandommotion,moleculesofahighconcentrationwilltendtoflowtowardsaregioninspacewheretheconcentrationislower.

Examples:Adyeinjectedintosolutionspreadingthroughacontainer,orheatspreadingfromaregionofhightemperaturetoaregionoflowertemperature.

TheDiffusionEquation

x=0 x=L

Considerdiffusioninonedimension(x)overtime(t)andletu(x,t)betheconcentrationofthesubstancethatisdiffusing.Then

𝜕𝑢𝜕𝑡 = 𝐷

𝜕&𝑢𝜕𝑥&

isthediffusionequationwithdiffusioncoefficientD.Onewouldalsoneedtosupplyinitialvaluesforu,u(x,0)=u0(x),andboundaryconditions ateachboundary.

TheDiffusionEquationTodescribediffusioninadomainwithmorethanonedimension,thesecondpartialderivativeoperatorisreplacedwiththeLaplacianoperator.Thenthediffusionequationis,

𝜕𝑢𝜕𝑡 = 𝐷𝛻

&𝑢

whereinthreedimensions

𝛻& =𝜕&

𝜕𝑥& +𝜕&

𝜕𝑦& +𝜕&

𝜕𝑧&

Laplacianoperator

DiffusiononaGraphWhatifthediffusingsubstancemovesalongedgesofagraphfromnodetonode?Inthiscase,thedomainisdiscrete,notacontinuum.

Letcbethediffusionrateacrosstheedge,thentheamountofsubstancethatmovesfromnodejtonodei overatimeperioddtisc 𝑢, − 𝑢. 𝑑𝑡 andfromnodei tonodejisc 𝑢. − 𝑢, 𝑑𝑡.So

𝑑𝑢.𝑑𝑡 = 𝑐 𝑢, − 𝑢.𝑑𝑢,𝑑𝑡 = 𝑐(𝑢. − 𝑢,)

ui ujc

DiffusiononaGraphDiffusiontoandfromnodei musttakeintoconsiderationallnodesinthegraph.Theconnectivityofthegraphisencodedintheadjacencymatrix.Hereweassumethatweareworkingwithasimplegraph.

𝑑𝑢.𝑑𝑡 = 𝑐𝐴.4 𝑢4 − 𝑢. + 𝑐𝐴.& 𝑢& − 𝑢. + ⋯+ 𝑐𝐴.6(𝑢6 − 𝑢.)

𝑑𝑢.𝑑𝑡 = 𝑐7𝐴.,(𝑢, − 𝑢.)

6

,84

or

DiffusiononaGraph

𝑑𝑢.𝑑𝑡 = 𝑐7𝐴.,𝑢, − 𝑐𝑢.7𝐴.,

6

,84

6

,84

Rewritingthelastexpression,

Degreeofnodei,di

= 𝑐7𝐴.,𝑢, − 𝑐𝑢.𝑑.

6

,84

WenowmakeuseoftheKronecker delta,𝛿.,

𝛿., = :0, if𝑖 ≠ 𝑗1, if𝑖 = 𝑗

DiffusiononaGraph

𝑑𝑢.𝑑𝑡 = 𝑐7𝐴.,𝑢, − 𝑐7𝛿.,𝑢,𝑑,

6

,84

6

,84

c𝑢.𝑑. = 𝑐 ∑ 𝛿.,𝑢,𝑑,6,84so

Definethen-dimensionalvector 𝑢 =𝑢4…𝑢6

Then c∑ 𝐴.,𝑢, = 𝑐 𝐴𝑢 .6,84

Nextdefinethe𝑛×𝑛degreematrix 𝐷 =𝑑4 0 00 … 00 0 𝑑6

Then c∑ 𝛿.,𝑢,𝑑, = 𝑐 𝐷𝑢 .6,84

TheGraphLaplacian

𝑑𝑢.𝑑𝑡 = 𝑐7𝐴.,𝑢, − 𝑐7𝛿.,𝑢,𝑑,

6

,84

6

,84

so

becomes𝑑𝑢𝑑𝑡 = 𝑐𝐴𝑢 − 𝑐𝐷𝑢

= 𝑐 𝐴 − 𝐷 𝑢

WenowdefinetheGraphLaplacianmatrix,

𝐿 ≡ 𝐷 − 𝐴

Theequationfordiffusiononagraphisthen

𝑑𝑢𝑑𝑡 + 𝑐𝐿𝑢 = 0

or 𝑑𝑢𝑑𝑡 + 𝑐 𝐷 − 𝐴 𝑢 = 0

TheGraphLaplacian

What’sinsideofL?

𝐿., = J𝑑., if𝑖 = 𝑗

−1, if𝑖 ≠ 𝑗andthereisanedge0, if𝑖 ≠ 𝑗andthereisnoedge

IsLsymmetric? Yes,why?

SolvingtheGraphDiffusionEquation

𝑑𝑢𝑑𝑡 + 𝑐𝐿𝑢 = 0

ThisisalinearsystemofODEs,soitissolvable.Also,sinceL issymmetricithasrealeigenvaluesandorthogonaleigenvectors,�⃗�., i= 1,⋯ , 𝑛.

Nowwritethesolutionasalinearcombinationoftheseeigenvectors,notingthatthecoefficientschangeovertime:𝑢 = ∑ 𝑎.(𝑡)�⃗�.6.84 .

InsertthisintotheODE, 7𝑑𝑎.𝑑𝑡 �⃗�. +7𝑐𝑎.𝐿�⃗�. = 0

6

.84

6

.84

7𝑑𝑎.𝑑𝑡 + 𝑐𝑎.𝜆. �⃗�. = 0

6

.84

where𝜆. isaneigenvalueofL.

SolvingtheGraphDiffusionEquation

Nowtaketheinnerproductofbothsidesofthelastequationwitheachoftheeigenvectors,recallingthattheyformanorthogonalset.Thisleadston differentialequationsforthecoefficients𝑎.(𝑡).

YZ[Y\+ 𝑐𝜆.𝑎. = 0 ,i=1,…,n

TheseODEsareuncoupledandlinear,sotheyhavesimpleexponentialsolutions:

𝑎. 𝑡 = 𝑎.(0)𝑒^_`[\

where𝑎.(0) istheinitialvalueofthecoefficient.

Sinceeachcoefficienthassuchasolution,thenbythesuperpositionprinciple,alinearcombinationoftheseisalsoasolution.Thus,thegeneralsolutiontothegraphdiffusiondifferentialequationis

𝑢 𝑡 =7𝑎.(0)𝑒^_`[\�⃗�.

6

.84

SpectralSolution

SolvingtheGraphDiffusionEquation

Howdowefindtheinitialvaluesofthecoefficients?

Usetheinitialdistributionofu amongthenodes.

𝑢 0 =7𝑎.(0)�⃗�.6

.84

Nowtaketheinnerproductofbothsideswithaneigenvector

𝑢 0 a �⃗�, = 𝑎,(0) �⃗�,&

𝑎, 0 =𝑢(0) a �⃗�,�⃗�,

&

SpectralPropertiesoftheGraphLaplacian

Byspectralproperties,wemeanpropertiesoftheeigenvaluesandeigenvectors.

SinceL issymmetric,itseigenvaluesarerealanditseigenvectorsareorthogonal

IsL singularornon-singular?

Lookatanyrowi. Thediagonalelementisthedegreeofthenode,𝑑..Alltheotherelementsareeither0or,foreachedge,-1.Thereareexactly𝑑. ofthese,soifyousumacrossanyrowofL youget𝑑. − 𝑑. = 0. Thisistrueforanyoftherows.Sothesumofallcolumnsofthematrixis0.Therefore,L issingular.Thatis,ithasatleastonezeroeigenvalue.Callit𝜆4 = 0.

SpectralPropertiesoftheGraphLaplacian

Whatistheeigenvectorassociatedwiththezeroeigenvalue?Thatis,thevector�⃗�4 suchthat𝐿�⃗�4 = 0?

Itmustbeavectorof1s,1.Why?

Because,𝐿1 isthesumofthecolumnsofL,whichweknowequals0.

DoesL haveanynegativeeigenvalues?

SoL hasnon-negativeeigenvalues,whichiscalledpositivesemidefinite.

Supposethat𝜆& < 0.Thentheterminthespectralsolution

𝑎&(0)𝑒^_`e\�⃗�& → ∞

whichweknowcan’thappen(thinkaboutspreadingdie,doesitsconcentrationgotoinfinityanywhereinthedomain?)

as 𝑡 → ∞

SpectralPropertiesoftheGraphLaplacianSupposethatthegraphhastwocomponents.Howisthatreflectedintheeigenvalues?

Labelthenodessothatthefirst𝑛4correspondtoonecomponentandthelast𝑛& = 𝑛 − 𝑛4 correspondtotheothercomponent.ThisresultsinablockdiagonalgraphLaplacianmatrix

L =

L1

L20

0

n1

n2

SpectralPropertiesoftheGraphLaplacian

L =

L1

L20

0

n1

n2

Define �⃗�4 = (1,1,1,⋯ , 0,0,0,0⋯)

n1

�⃗�& = (0,0,0,⋯ , 1,1,1,1⋯)

n2

𝐿�⃗�& = 𝐿&1 = 0Then 𝐿�⃗�4 = 𝐿41 = 0 and

So�⃗�4and�⃗�& arebotheigenvectorsofL with0eigenvalues,andL hastwo0eigenvalues.

SpectralPropertiesoftheGraphLaplacian

Ingeneral,thenumberof0eigenvaluesofthegraphLaplacianisequaltothenumberofcomponentsofthegraph.

OnecanordertheeigenvaluesofL fromsmallesttolargest.Then

𝜆4 = 0

If 𝜆& ≠ 0 thenthegraphisconnected

If 𝜆& = 0 thenthegraphisdisconnected

So𝜆&iscalledthealgebraicconnectivityofthegraph.

AsymptoticSolution

𝑢 𝑡 =7𝑎.(0)𝑒^_`[\�⃗�.

6

.84

Recallthatthespectralsolutiontothegraphdiffusionequationis

Supposethatthegraphisconnected.Then𝜆4 = 0 andallothereigenvaluesarepositive.Therearen termsinthesolutionabove.Whathappenstothesetermsas𝑡 → ∞?

Allapproach0,exceptforthefirstterm,whichisindependentoftime.Sotheasymptoticsolutionisjustthefirsttermofthespectralsolution,

𝑢h = 𝑎4 0 �⃗�4

AsymptoticSolution

𝑢h = 𝑎4 0 �⃗�4

But 𝑎4 0 �⃗�4 =𝑢(0) a �⃗�4�⃗�4 &

1 = 𝑢4 0 +⋯+ 𝑢6(0)

𝑛 1

So 𝑢h =𝑢4 0 +⋯+ 𝑢6(0)

𝑛 1

Howcanweinterpretthisphysically?

Inthelongterm,eachnodeintheconnectedgraphgetsthesameshareofthedye(orwhateverisdiffusing),whichisequaltothetotalamountinitiallypresentdividedbythenumberofnodes.

AsymptoticSolutionThiscanalsobederiveddirectlyfromthediffusionequation

𝑑𝑢𝑑𝑡 + 𝑐𝐿𝑢 = 0

Settimederivativeto0,

𝐿𝑢h = 0

SotheequilibriumvectorisaneigenvectorofthegraphLaplaciancorrespondingtothe0eigenvalue,whichiswhatwejustsawusingthedifferentapproach.

AsymptoticSolutionWecanrewritethisequilibriumequationbydeconstructingthegraphLaplacian

𝐿𝑢h = 0

or

(D-A)𝑢h = 0

D𝑢h = 𝐴𝑢h

𝑢h = 𝐷^4𝐴𝑢h

Holdonnow,isD invertible? Yes,aslongasthegraphisconnected

TheFilterMatrixDefineanewmatrix,W,whichI’llcallafiltermatrix

𝑊 ≡ 𝐷^4𝐴

𝑢h = 𝑊𝑢h

Then

Howcanweinterpretthis?Onewayistothinkaboutthefollowingfirst-order linearrecursionor differenceequation:

𝑢jk4 = 𝑊𝑢j

startingfromtheinitialvector𝑢l.Whathappensifyouiterateforever?

Thenultimatelyifthesystemconverges(anditwill),thek+1iteratewillbethesameasthekiterate.Thisgivestheequilibriumequationabove.Theequilibriumvector𝑢h isthereforethefixedpointoftherecursionwiththefiltermatrix.It’swhatyougetifyoukeepperformingthefilteringoverandoveragain.

TheFilterMatrix

Since 𝑢h = 𝐷^4𝐴𝑢h

𝑢h,. =∑ 𝐴.,𝑢h,,6,84∑ 𝐴.,6,84

wherewenotethat𝐴.. = 0whentherearenoself-edges.

If𝑢h isthesameateachnode,thenthisequationissatisfied.Sotheequilibriumvalueateachnodeistheaverageoftheinitialamountofdiffusiblesubstance.Thisisthepropertyofdiffusion,andmoreparticularly,theLaplacianoperator.Thusthename“graphLaplacian”forL.

HeterogeneousDiffusion

Sofarwehavethoughtofthegraphasanunweightedgraph.Thatis,diffusionbetweenanypairofnodeshasthesamerate,c.Thisiscalledhomogeneousdiffusion.Moregenerally,eachedgecanhaveitsowndiffusionrate,whichiscalledheterogeneousdiffusion.Sonowtheadjacencymatrixhasweightsasitselements(or0s),and

𝐿., = J𝑑., if𝑖 = 𝑗

−𝑐.,, if𝑖 ≠ 𝑗andthereisanedge0, if𝑖 ≠ 𝑗andthereisnoedge

wheredegreedi isthesumoftheweightededgesatnodei and𝑐., istheweightconnectingnodesi andj.

𝑢h,. =∑ 𝐴.,𝑢h,,6,84∑ 𝐴.,6,84

Again,

Asolutiontothisis𝑢h,. = 𝑢h foreachnodeinthenetwork.Thatis,onceagain,atequilibriumthenodesequallydividetheinitialamountofsubstance.

ImageProcessing

PixelatedImageasaNetwork

Eachpointonthegridhasagreylevel:0=white,1=black,withshadesofgreycorrespondingtointermediatevalues.Thesegridpointsarethenodes ofanetworkandtheirgreylevelisthevalueofu atthatnode.

PixelatedImageasaNetwork

Whataretheedges?

Mostgenerally,connecteachnodepairwithanedge,butmaketheedgesweighted(𝑐.,)bytheaffinity oftheconnectednodes.Here,affinitymeanshowsimilartheyare.

Similarcouldmeanlocation(nearestneighborsgethighestaffinity).Oritcouldmeanthegreylevelofthepixels(similaru valuesofnodes).Ifitisbasedonlocation,thentheedgeweightsarefixed.Ifbasedongreylevel,edgeweightsarefunctionsoftheu values.

ModifyinganImageWiththeFilterMatrix

Let𝑢l bethegreylevelvaluesoftheoriginalimage.Imageprocessingmultipliesthatvectorbythefiltermatrixtogetaproductvector,𝑢4 = 𝑊𝑢l.Ittypicallydoesthismorethanonce,applyingtherecursionrelationwesawearlier,𝑢jk4 = 𝑊𝑢j.

TheeffectthishasontheimagedependsonW.Iftheaffinitiesarechosentobeneighborsinphysicalspace,thentheeffectofthefilteringwillbetoaverageoutdifferencesamongneighbors.Theimagebecomessmootherandblurrier.

ModifyinganImageWiththeFilterMatrix

Iftheaffinitiesarechosenaccordingtosimilarityingreylevel,thenapplyingthefilterwillhavetheeffectofmakingsimilarpixelsmoresimilar,whichtendstosharpentheimage.Inthiscase,theweights𝑐.,andthereforetheelementsofWareupdatedwitheachiterationoftherecursionformula.

Inpractice,combinationsofaffinitybasedonlocationandbasedongreylevelsareused,anditerationstopswhensomemeasureofimagegoodnesshasbeenreached.

ForaGreatVideoonImageProcessing

Thefollowinglinkhasanhour-longlecturebyaresearcherfromGoogleonhowthegraphLaplacianisusedinimageprocessing:

https://www.youtube.com/watch?v=_ItmFYCr7ag

TheEnd