A Quantum Jensen-Shannon Graph Kernel using the Continuous-Time Quantum Walk

Lu Bai, Edwin R. Hancock Department of Computer Science, University of York, UK

Andrea Torsello and Luca Rossi Ca’ Foscari University of Venice, Italy

Outline

Motivation

State of the Art Graph Kernels

Existing graph kernel methods

New Kernel: Quantum Jensen-Shannon Graph Kernel

Continuous-time quantum walk on a graph

Classical and quantum Jensen-Shannon divergence

Quantum Jensen-Shannon divergence of graphs

Quantum Jensen-Shannon graph kernel

Experiments

Conclusion and Further Work

Background

Graph matching using continuous time quantum walks.

Classical Random walks on graphs

Determined by the Laplacian spectrum (and in continuous time case by heat-kernel).

Can be used to analyse graph structure and develop path-based algorithms on graphs.

Quantum Walks on Graphs

Have both discrete time (DQW) and continuous time (CQW) variants.

Use qubits to represent state of walk.

State-vector is composed of complex numbers rather than an probabilities. Governed by unitary matrices rather than stochastic matrices.

Admit interference of different feasible walks.

Reversible, non ergodic, no limiting distribution.

Sensitive to symmetry structure of graph (leads to faster hitting and commute times).

Quantum and Classical Walks on Graphs

Contrast continuous-time quantum walk and classical

random walk on a graph

Continuous-time quantum walk A) state space: set of vertices B) the state vector is complex-valued C) the evolution is governed by a time-varying unitary matrix

Classical random walk A) state space: set of vertices B) the state vector is real-valued C) the evolution is governed by a double stochastic matrix

Continuous-time quantum walk is reversible and non-ergodic, and does not have a limiting distribution.

Continuous Time Quantum Walk

Evolution governed by Schrodinger’s equation

Solution

tt iLdt

d ||

0|]exp[| iLtt

Walks compared

Classical walk Quantum walk

Literature

Fahri and Gutmann: CQW can penetrate family of trees in polynomial time, whereas classical walk can not.

Childs: CQW hits exponentially faster than classical walk.

Kempe: exploits polynomial hitting time to solve 2-SAT problem and suggest solution to routing problem.

Shenvi, Kempe and Whaley: search of unordered database.

Graph matching

Shiau, Joynt and Coppersmith: random walks obeying Fermi statistics yield invariants for distinguishing strongly regular graphs.

Douglas and Wang: compare amplitudes on DQW on different graphs to establish isomorphism or similarity.

Quantum walks

….have richer structure due to interference and complex representation. Provide deeper probes of network structure (symmetry). What can a quantum walker discover about a graph that a classical walker can not?

Prior work:

Explored role of interference in quantum walks to develop alternatives to classical random walks algorithm for inexact graph matching.

Problem: Approach is pseudo-quantum since issue of observation process is not addressed rigourously.

Journal Publications

Coined quantum walks lift the cospectrality of graphs and trees, Emms, Severini, Wilson, Hancock. Pattern Recognition in press

A matrix representation of graphs and its spectrum as a graph invariant, with Severini, Wilson, Hancock. Electronic Journal of Combinatorics

Graph Matching using Interference of Discrete Quantum Walks, Emms, Wilson and Hancock. Image and Vision Computing in press.

Graph Matching using Interference of Continuous Time Quantum Walks, Emms, Wilson and Hancock. Pattern Recognition in press.

Graph Embedding using Quantum Commute Time, Emms, Wilson and Hancock. Quantum Information and Computation in press.

Approach

Quantum interference can be used to ‘automatically’ compare the quantum walk on two graphs.

Create an auxiliary structure that connects all pairs of vertices from the two graphs to be compared via auxiliary vertices.

Simulate a quantum random walk on this structure which causes the two walks to interfere.

The key states where this interference is exact indicate possible matches between the vertices of the graphs.

Constructing the Auxiliary Graph Take two graphs to be

matched.

Auxiliary Vertices

Constructing the Auxiliary Graph Take two graphs to be

matched.

If graph A has n vertices

and graph has B m vertices,

create nm auxiliary vertices.

Constructing the Auxiliary Graph Take two graphs to be

matched.

If graph A has n vertices

and graph has B m vertices,

create nm auxiliary vertices.

Connect each vertex in A to

each vertex in B by way of

an auxiliary vertex.

Auxiliary Vertices

Constructing the Auxiliary Graph Take two graphs to be

matched.

If graph A has n vertices

and graph has B m vertices,

create nm auxiliary vertices.

Connect each vertex in A to

each vertex in B by way of

an auxiliary vertex.

Vertex connecting u єVA to

v єVB labelled α{u,v}.

Auxiliary Vertices

u

v

α{u,v}

Continuous Time QW

Simulate CTW on auxiliary graph

Starting state

Isomorphism

utVu

ut |)(|

otherwise

Vuud

ud

Vuud

ud

H

VVu

G

VVu

u

HG

HG

0

)(

)(

)(

)(

)0(2

2

0)(},{

thgv

If there exists some vertex u є VA such that I-(u,v) is non-

zero for all v єVB or some vertex v єVB such that I(u,v) is

non-zero for all uєVA then no isomorphism exists.

Otherwise check all functions which map u->v only if I-

(u,v)=0 and check if isomorphism.

May be a unique function that must be checked or more

than one.

Possible Isomorphisms

Interference Amplitudes in presence of noise

False matches modelled by a Gaussian distribution, σf:

True matches modelled by a double exponential distribution, σt<<σf :

Observations

Set up comparison structure in which corresponding nodes are symmetrically placed.

Set up CTQW on comparison structure.

Correspondences characterised by zero interference amplitude.

Problems

Current collapse of wave-function is quasi-classical.

Require more principled means of similarity assessment.

New Approach

Define information theoretic kernel over continuous time quantum walk.

Relies on density matrix formalism.

Similarity measure is quantum Jensen-Shannon divergence between density matrices.

Literature

Existing Graph Kernels (i.e Graph Kernels from the R-convolution [Haussler, 1999]). Fall into three classes:

Restricted subgraph or subtree kernels

Weisfeiler-Lehman subtree kernel [Shevashidze et al., 2002

Random walk kernels

Product graph kernels [Gartner et al., 2003]

Marginalized kernels on graphs [Kashima et al., 2003]

Path based kernels

The shortest path kernel [K. m. Borgwardt, 2005]

Information Theoretic Graph Kernels

The Jensen-Shannon graph kernel [L. Bai and E.R. Hancocki,

2013]

Defined using the classical Jensen-Shannon divergence between entropies over the graphs.

Entropy associated with a probability distribution of a graph can be directly computed without the need of decomposing the graph.

Avoids the computational burdensome of comparing the similarities between all the pairs of substructures of the graphs.

Density Matrix

In quantum mechanics, the density operator ρ is a matrix that describes an ensemble of pure states, i.e. a mixed state.

Pure state is a quantum state that can be described by a single state vector |ψ⟩ and its density operatorρ can be written as |ψ⟩ ⟨ψ| .

Mixed quantum state is an ensemble of pure states described by a density operator ρ. consider a quantum system that can be found in a number of pure states {(|ψn⟩ , n)|(n = 1, 2, . . . ,N)} each with a probability pn.

The density operator (i.e. density matrix) of the system is then defined as

Initialisation

For Continuous-time Quantum Walk define the initial state |ψo⟩ of

G(V,E) as

Von Neumann Entropy of Density Matrix

The von Neumann entropy of a graph: In quantum mechanics, the von Neumann entropy is an extension of the classical Shannon entropy, and is defined on a density operatorρ. Note that ifρ is the density matrix associated with a pure state, then the von Neumann entropy vanishes.

whereλ1;ρ:G, . . . , λj;ρ;G, . . . , λ|V |;ρ;G are the ordered eigenvalues of ρG.

Information theoretic divergences

The classical Jensen-Shannon divergence The classical JSD is a non-extensive mutual information measure between

probability distributions over structure data, and is related to the Shannon entropy. The classical JSD is always well defined, symmetric, negative definite and bounded.

Assume P and Q are two probability distributions, the JSD is defined as

The quantum Jensen-Shannon divergence The quantum JSD is extend from the classical JSD by replacing the Shannon

entropy of a probability distribution with the von Neumann entropy of a density operator. The quantum JSD is always well defined, symmetric, negative definite and bounded.

Assume ρ and σ are two density operators, the quantum JSD is

A Quantum Jensen-Shannon Graph Kernel

The Quantum Jensen-Shannon Graph Kernel The quantum Jensen-Shannon divergence between graphs: Let a continuous-

time quantum walk perform evolve on a pair of graphs Ga(Va,Ea) and Gb(Vb,Eb) with time t (t = 1, . . . , T). Then the density operators ρG;a and σG;b of Ga(Va,Ea) and Gb(Vb,Eb) can be computed from their mixed states. The quantum Jensen-Shannon divergence between Ga and Gb is defined as

The quantum Jensen-Shannon kernel between graphs: We define the quantum Jensen-Shannon kernel between the pair of graphs Ga and Gb as

where λ is a decay factor which satisfies λ ∈(0,1), and HN(.) is the von Neumann entropy of a graph associated with its density operator.

Lemma. The quantum Jensen-Shannon kernel is positive definite pd.

Proof. The follows the definition of a diffusion kernel. If a (dis)similarity or distance measure s(Ga,Gb) is symmetric, then a diffusion kernel ks=exp(λ s(Ga,Gb)) is pd.

Experiments

Experimental evaluations of the quantum Jensen-Shannon graph kernel Classification of graphs abstracted from bioinformatics databases.

Alternative graph kernels and graph entropies for comparisons include: a) the Weisfeiler-Lehman subtree kernel [Shevashidze et al., 2009]

b) the shortest path graph kernel [Borgwardt et al., 2005]

c) the Ihara zeta function of graphs [Ren et al., 2010]

d) the Shannon entropies using information functionals FV and FP [Dehmar et al. 2011]

Von Neumann entropy evaluation

Explore the relationship between the von Neumann entropy of a graph associated with its density operator and its corresponding increasing time t.

Experiments

Graph Datasets from the Bioinformatics Databases

MUTAG: The MUTAG dataset consists of graphs representing 188 chemical compounds. The maximum, minimum and average number of vertices are 28, 10 and 17.93 respectively.

ENZYMES: The ENZYMES dataset consists of graphs representing protein tertiary structures consisting of 600 enzymes from the BRENDA enzyme database. In this case the task is to correctly assign each enzyme to one of the 6 EC toplevel classes. The maximum, minimum and average number of vertices are 126, 2 and 32.63 respectively.

D&D: The PPIs dataset consists of protein-protein interaction networks (PPIs). The graphs describe the interaction relationships between histidine kinase in different species of bacteria. Histidine kinase is a key protein in the development of signal transduction. If two proteins have direct (physical) or indirect (functional) association, they are connected by an edge. There are 219 PPIs in this dataset and they are collected from 5 different kinds of bacteria. We select Proteobacteria40 PPIs and Acidobacteria46 PPIs as the testing graphs (i.e. 86 testing graphs). The maximum, minimum and average number of vertices of the selected testing graphs are 232, 3 and 109.60 respectively.

Experiments

Experimental comparisons of graph classification using the quantum Jensen-Shannon graph kernel (t=50), the alternative graph kernels and the graph entropies.

Experimental setup: For the kernel methods, we compute the kernel matrix of each graph kernel on each dataset, we then apply the PCA on the kernel matrix to embed the graphs into principle component space as feature vectors. For other methods, we compute the characteristics values of graphs on each dataset. We perform 10-fold cross-validation using the Support Vector Machine (SVM) Classification associated with the Sequential Minimal Optimization (SMO) on the graph feature vectors or characteristics values to evaluate the performance of our kernel and that of the alternative methods. We use nine samples for training and one for testing. We repeat the experiments ten times and report the average accuracies.

Experiments

Von Neumann Entropy Evaluation The left, middle and right subfigures show the results of the evaluation on the MUTAG,

Enzymes and PPIs datasets separately. The x-axis shows the time t which is from 1 to 50, and the y-axis shows the mean value of the von Neumann entropies of graphs belonging to the same class at each time t. Here the different lines represent the entropies of different classes of graphs separately. This evaluation suggests that the von Neumann entropies of different classes of graphs can be divided well, and tend to be stable with increasing time t.

Conclusion and Further Work

Conclusion: CTQW + density matrix promising tools for network analysis and kernelisation.

Further Work Deeper understanding of links to classical walks, and their fixed

points.

Extend the quantum graph kernel to attributed graphs and hypergraphs.

Explore role of decoherence in density matrix picture.

Acknowledgments

Prof. Edwin R. Hancock is supported by a Royal Society Wolfson Research Merit Award.

We thank Prof. Karsten Borgwardt and Dr. Nino Shervashidze for providing the Matlab implementation for the various graph kernel methods, and Dr. Geng Li for providing the graph datasets. We thank Dr. Peng Ren for the constructive discussion and suggestion about the further work.

Reference 1. Scholkopf, B., Smola, A.: Learning with Kernels. MIT Press (2002).

2. Haussler, D.: Convolution kernels on discrete structures. In: Technical Report UCS-CRL-99-10, Santa Cruz, CA, USA (1999).

3. Kashima, H., Tsuda, K., Inokuchi, A.: Marginalized kernels between labeled graphs. In: Proceedings of International Conference on Machine Learning. (2003) 321–328.

4. Borgwardt, K.M., Kriegel, H.P.: Shortest-path kernels on graphs. In: Proceedings of the IEEE International Conference on Data Mining. (2005) 74–81.

5. Shervashidze, N., Schweitzer, P., van Leeuwen, E.J., Mehlhorn, K., Borgwardt, K.M.: Weisfeiler-lehman graph kernels. Journal of Machine Learning Research 1 (2010) 1–48.

6. Lamberti, P., Majtey, A., Borras, A., Casas, M., Plastino, A.: Metric character of the quantum jensen-shannon divergence. Physical Review A 77 (2008) 052311.

7. Bai, L., Hancock, E.R.: Graph kernels from the jensen-shannon divergence. Journal of Mathematical Imaging and Vision (To appear).

8. Majtey, A., Lamberti, P., Prato, D.: Jensen-shannon divergence as a measure of distinguishability between mixed quantum states. Physical Review A 72 (2005) 052310.

9. Farhi, E., Gutmann, S.: Quantum computation and decision trees. Physical Review A 58 (1998) 915.

10. Dirac, P.: The Principles of Quantum Mechanics (4ed). Oxford Science Publications (1958).

11. Kempe, J.: Quantum random walks: an introductory overview. Contemporary Physics 44 (2003) 307–327.

12. Nielsen, M., Chuang, I.: Quantum computation and quantum information. Cambridge university press (2010).

13. Martins, A.F., Smith, N.A., Xing, E.P., Aguiar, P.M., Figueiredo, M.A.: Nonextensive information theoretic kernels on measures. Journal of Machine Learning Research 10 (2009) 935–975.

14. Konder, R., Lafferty, J.: Diffusion kernels on graphs and other discrete input spaces. In: Proceedings of International Conference on Machine Learning. (2002) 315–322.

15. Neuhaus, M., Bunke, H.: Bridging the gap between graph edit distance and kernel machines. World Scientific (2007).

16. Escolano, F., Hancock, E.R., Lozano, M.A.: Heat diffusion: Thermodynamic depth complexity of networks. Physical Review E 85 (2012) 036206.

17. Dehmer, M.: Information processing in complex networks: Graph entropy and information functionals. Applied Mathematics and Computation 201 (2008) 82–94.

18. Ren, P., Wilson, R.C., Hancock, E.R.: Graph characterization via ihara coefficients. IEEE Transactions on Neural Networks 22 (2011) 233–245.

19. Scholkopf, B., Smola, A.J., M¨uller, K.R.: Kernel principal component analysis. In: Proceedings of International Conference on Artificial Neural Networks. (1997) 583–588.

20. Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. Scholkopf, B., Burges, C.J.C., and Smola, A.J. (Eds.) Advances in Kernel Methods (1999) 185–208.

21. Witten, I.H., Frank, E., Hall, M.A.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann (2011).

22. Bai, L., Hancock, E.R., Ren, P.: A jensen-shannon kernel for hypergraphs. In: SSPR/SPR. (2012) 181–189.

23. Ren, P., Aleksic, T., Emms, D., Wilson, R., Hancock, E.: Quantum walks, ihara zeta functions and cospectrality in regular graphs. Quantum Information Processing 10 (2011) 405–417.

Thank you!