Multigrid Eigensolvers for Image Segmentation

Andrew Knyazev

Supported by NSF DMS 0612751.

This presentation is at http://math.ucdenver.edu/~aknyazev/research/conf/.

1

Eigensolvers for image segmentation:

Image Segmentation – the definition Image Segmentation as Clustering Clustering: how? Spectral clustering and partitioning Partitioning as image segmentation Eigensolvers for spectral clustering Multi-resolution image segmentation References Conclusions

2

Image Segmentation – the definitionWikipedia: In computer vision, segmentation refers to the process of partitioning a digital image into multiple segments (sets of pixels). Image segmentation is the process of assigning a label to every pixel in an image such that pixels with the same label (i.e. in the same segment) share certain visual characteristics. All pixels in a segment are similar with respect to some characteristic or computed properties, such as color, intensity, or texture. Adjacent segments are significantly different with respect to the same characteristic(s).

3

Image Segmentation as ClusteringA specific case of general Data Clustering, where data points are the image pixelsGeometric location of each pixel within the image is KNOWN a prioriThe similarity among the (typically only neighborhood) pixels is calculated using some function which for every two (neighborhood) pixels i and j determines a number a_ij between 0 and 1. If a_ij = 0, there is no similarity between the two pixels. On the other extreme, if a_ij = 1, the two pixels are “the same.” Weighted graph: vertices-pixels,edges-a_ij

4

Examples: Graph Partitioning

5

Examples: Image Segmentation

6

Clustering: how? The overviewThere is no good widely accepted definition of clustering! The traditional graph-theoretical definition is combinatorial in nature and computationally infeasible. Heuristics rule! Good open source software, e.g., METIS and CLUTO.

Clustering can be performed hierarchically by agglomeration (bottom-up) and by division (top-down).

7

Agglomeration clustering example

Clustering: how? AlgorithmsPartitioning means determining clusters. Partitioning can be used recursively for hierarchical division (top-down).

Many partitioning methods are known. Here we cover:

K-means (centroids) Spectral partitioning using Fiedler vectors =

Principal Components Analysis (PCA)

PCA/spectral partitioning is known to produce high quality clusters, but is considered to be expensive as solution of large-scale eigenproblems is required. Our expertise in eigensolvers comes to the rescue!

8

K-means (centroids) clustering The K-means algorithm assigns each point to the cluster whose center (called centroid) is nearest. The center is the average of all the points in the cluster.

9

The initial centroids are placed randomly (left), then are moved iteratively (center) until convergence (right).

The “relaxed” K-means is equivalent to the spectral clustering, discussed next.

Eigenproblems in mechanical vibrations

Free transversal vibration without damping of the mass-spring system is described by the ODE system

10

Standing wave

assumption leads to the eigenproblem

Images courtesy http://www.gps.caltech.edu/~edwin/MoleculeHTML/AuCl4_html/AuCl4_page.html Component xi describes the up-down movement of mass i.

11

Spectral clustering in mechanics

A 4-degree-of-freedom system has 4 modes of vibration and 4 natural frequencies: partition into 2 clusters using the second eigenvector:

Images Courtesy: Russell, Ketteriung U.

The main idea comes from mechanical vibrations and is intuitive: in the spring-mass system the masses which are tightly connected will have the tendency to move together synchronically in low-frequency free vibrations. Analysing the signs of the components corresponding to different masses of the low-frequency vibration modes of the system allows us to determine the clusters of the masses!

12

• A = symmetric adjacency matrix

• D = diagonal degree matrix

• Laplacian matrix L = D – A

Spectral clustering for simple graphs

Undirected graphs with no self-loops and no more than one edge between any two different vertices

L=K describes transversal vibrations of the spring-mass system (as well as Kirchhoff's law for electrical circuits of resistors)

13

•The Fiedler eigenvector gives bi-partitioning by separating the positive and negative components only

•By running the K-means on the Fiedler eigenvector one could find more then 2 partitions if the vector is close to piecewise-constant after reordering

•The same idea for more then one eigenvectors (multi-way)

31011

13101

01201

10010

11103

L

Rows sum to zero

3

1 2

4

5

13.

26.

44.

81.

26.

2x

Spectral clustering for simple graphs

22 83. xLx

Example Courtesy: Blelloch CMU

www.cs.cas.cz/fiedler80/

The Laplacian matrix L is symmetric with the zero smallest eigenvalue and constant eigenvector (free boundary). The second eigenvector, called the Fiedler vector, describes the partitioning.

PCA clustering for simple graphs14

• The Fiedler vector is an eigenvector of Lx=λx, in the spring-mass system this corresponds to the stiffness matrix K=L and to the mass matrix M=I (identity)

•Should not the masses with a larger adjacency degree be heavier? Let us take the mass matrix M=D -the degree matrix

•So-called N-cut smallest eigenvectors of Lx=λDx are the largest for Ax=µDx with µ=1-λ since L=D-A

• PCA for D-1A computes the largest eigenvectors, which then can be used for clustering by the K-means

•D-1A is row-stochastic and describes the Markov random walk probabilities on the simple graph

15

Partitioning as image segmentationImage pixels serve as graph vertices. We generate a sparse Laplacian, by comparing neighboring only 5 or 9 pixels here when calculating the weights for the edges by comparing pixel colors. Here is an example displaying on the right 4 Fiedler vectors of an image on the left. The actual values of vectors’ components are shown at every pixel.

16

Eigensolvers for spectral clustering Our BLOPEX-LOBPCG software has proved to be efficient for large-scale eigenproblems for Laplacians from PDE's and for image segmentation using multiscale preconditioning of hypreThe LOBPCG for massively parallel computers is available in our Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) packageBLOPEX is built-in in http://www.llnl.gov/CASC/hypre/ and is included as an external package in PETSc, see http://www-unix.mcs.anl.gov/petsc/On BlueGene/L 1024 CPU we can compute the Fiedler vector of a 24 megapixel image in seconds (including the hypre algebraic multigrid setup).

Multi-resolution image segmentation

Bipartitioning for 9-Point Stencil. Left to right: Original - Spectral - NCUT - Spectral Line - NCUT Line. Top to bottom: 2856, 53482, 95904, 128625 pixels images

17

Image ``Very Large Telescope at Paranal'' in the Atacama desert region in Chile from the European Southern Observatory web site www.eso.org

Multiresolution image segmentation

Direct bisection of the highest resolution image may be better quality compared to multiresolution segmentation

Multiway clustering using several eigenvectors may help multiresolution segmentation

18

10 smallest eigenvalues of L and D-1L for different image scaling

References

AK, Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method. SISC 23 (2001), 517-541.

AK and K. Neymeyr, Efficient solution of symmetric eigenvalue problems using multigrid preconditioners in the locally optimal block conjugate gradient method. ETNA, 15 (2003), 38-55.

AK, I. Lashuk, M. E. Argentati, and E. Ovchinnikov, Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) in hypre and PETSc. SISC. 25(2007), 2224-2239.

19

Recommended general reading Internet search for Spectral Clustering, Graph Partitioning, Image Segmentation, Normalized Cuts Spectral Clustering, Ordering and Ranking: Statistical Learning with Matrix Factorizations by Chris Ding and Hongyuan Zha, Springer, In Press. ISBN-13: 978-0387304489These slides and other similar talks on my Web at http://math.ucdenver.edu/~aknyazev/research/conf/

20

Possible Future Work

Code a similar driver using Hypre geometric multigrid preconditioner rather than algebraicDevelop and code in-house geometric multigrid specifically for image segmentation purposes No need for double precision: code a single precision driver In a dedicated image segmentation package, all calculations must be performed using a precision that matches the image format (great for GPUs!) Crucial mathematical issues not discussed here

21

22

Conclusions

Spectral Clustering is an intuitive and powerful tool for image segmentation. Our eigenxolvers efficiently perform image segmentation using Hypre’s algebraic multigrid preconditioning on parallel computers. Segmentation using smaller resolutions of the same image, e.g., to construct initial approximations for higher resolutions, could accelerate the process, but requires multi-way computations involving several eigenvectors.