Equazioni lineari matriciali propriet`a, metodi numerici ...simoncin/umi15.pdf · Equazioni lineari matriciali propriet`a, metodi numerici ed applicazioni V.Simoncini Dipartimento

Equazioni lineari matriciali

proprieta, metodi numerici ed applicazioni

V. Simoncini

Dipartimento di Matematica, Universita di Bologna (Italy)

[email protected]

1

Some matrix equations

• Sylvester matrix equation

AX+XB +D = 0

Eigenvalue problems, Control, Model Order Reduction, Assignment problems,

Riccati equation

Lyapunov matrix equation

AX+XA⊤ +D = 0, D = D⊤

Stability analysis in Control and Dynamical systems, Signal processing,

eigenvalue computations

2


• Sylvester matrix equation

AX+XB +D = 0

Eigenvalue problems, Control, Model Order Reduction, Assignment problems,

Riccati equation

• Lyapunov matrix equation

AX+XA⊤ +D = 0, D = D⊤

Stability analysis in Control and Dynamical systems, Signal processing,

eigenvalue computations

3


• Algebraic Riccati equation

AX+XA⊤ −XBB⊤X+D = 0, D = D⊤

Lancaster-Rodman ’95, Konstantinov-Gu-Mehrmann-Petkov, ’02,

Bini-Iannazzo-Meini ’12

Multiterm matrix equation

A1XB1 +A2XB2 + . . .+AℓXBℓ = C

Elliptic PDEs, PDEs with stochastic inputs, bilinear dynamical systems, etc.

Focus: All or some of the matrices are large (and possibly sparse)

4






• Multiterm linear matrix equation




5






• Multiterm linear matrix equation




6

The Lyapunov equation.

AX+XA⊤ +D = 0, A stable

A X + X A⊤ + D = 0

A = sparse, but ... X dense

Example: For D = I and A symmetric, it holds that X = − 12A

−1

7



A X + X A⊤ + D = 0

A = 0 1000 2000 3000 4000 5000 6000

0

1000

2000

3000

4000

5000

6000

nz = 31680 sparse, but ... X dense


−1

8



A X + X A⊤ + D = 0

A = 0 1000 2000 3000 4000 5000 6000

0

1000

2000

3000

4000

5000

6000

nz = 31680 sparse, but ... X dense


−1

9

The Lyapunov equation. Some characterizations

AX +XA⊤ +BB⊤ = 0, A ∈ Rn×n stable

• The Applied Mathematician perspective

X holds stability information of time-invariant dynamical system:

x′(t) = Ax(t) +Bu(t), x(0) = x0

• The Analyst perspective. Closed form solution:

X = − 1

2π

∫ ∞

−∞

(ıωI −A)−1BB⊤(ıωI −A)−∗dω =

∫ 0

−∞

eAtBB⊤eAtdt

• The Algebraist perspective. Kronecker formulation:

(A⊗ I + I ⊗A)x = b x = vec(X), b = vec(BBT )

with S := A⊗ I + I ⊗A ∈ Rn2×n2

10





x′(t) = Ax(t) +Bu(t), x(0) = x0


X = − 1

2π

∫ ∞

−∞


∫ 0

−∞

eAtBB⊤eAtdt




11





x′(t) = Ax(t) +Bu(t), x(0) = x0


X = − 1

2π

∫ ∞

−∞


∫ 0

−∞

eAtBB⊤eAtdt




12

Linear systems vs linear matrix equations

Large linear systems:

Sx = b,

• Krylov subspace methods (CG, MINRES, GMRES, BiCGSTAB, etc.)

• Preconditioners: find P such that

SP−1x = b x = P−1x

is easier and fast to solve

Large linear matrix equation:

AX +XA⊤ +BB⊤ = 0

No preconditioning to preserve symmetry

X is a large, dense matrix ⇒ low rank approximation

X ≈ X = ZZ⊤, Z tall

13

Linear systems vs linear matrix equations

Large linear systems:

Sx = b,

• Krylov subspace methods (CG, MINRES, GMRES, BiCGSTAB, etc.)

• Preconditioners: find P such that

SP−1x = b x = P−1x

is easier and fast to solve

Large linear matrix equations:

AX+XA⊤ +BB⊤ = 0

• No preconditioning - to preserve symmetry

• X is a large, dense matrix ⇒ low rank approximation

X ≈ X = ZZ⊤, Z tall

14

The Kronecker sum matrix

S := A⊗ In + In ⊗A,

with A symmetric and positive definite, banded with bandwidth b

• Quantum Chemistry and Quantum dynamics

• Signal processing

• Numerical analysis

- PDE discretizations: e.g., in Finite Differences, Finite Elements,

Legendre Spectral Methods, Isogeometric Analysis, ...

• Multivariate Statistics

Sparsity and quasi-sparsity pattern properties of

f(S)

f ∈ z−1, ez, z1

2 , ...

15

Discretization of 2D Laplace operator on the unit square

S := A⊗ In + In ⊗A, A = tridiag(−1, 2,−1)

Sparsity pattern:

0 10 20 30 40 50 60 70 80 90 100

0

10

20

30

40

50

60

70

80

90

100

nz = 4600 10 20 30 40 50 60 70 80 90 100

0

10

20

30

40

50

60

70

80

90

100

nz = 9380

Matrix S S−1

16

Discretization of 2D Laplace operator on the unit square

S := A⊗ In + In ⊗A, A = tridiag(−1, 2,−1)

Sparsity pattern:

0 10 20 30 40 50 60 70 80 90 100

0

10

20

30

40

50

60

70

80

90

100

nz = 460 020

4060

80100

0

20

40

60

80

1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Matrix S |(S−1)ij |

17

The exponential decay of the entries of S−1

The classical bound (Demko, Moss & Smith):

If S spd is banded with bandwidth b, then

|(S−1)ij | ≤ γq|i−j|

b

where

κ = λmax(S)/λmin(S) (cond. number of S)

q :=

√κ− 1√κ+ 1

< 1

γ := maxλmin(S)−1, γ, and γ =(1 +

√κ)2

2λmax(S)(λmin(S), λmax(S) smallest and largest eigenvalues of S)

Many contributions: Bebendorf, Hackbusch, Benzi, Boito, Razouk, Golub, Tuma,

Concus, Meurant, Mastronardi, Ng, Tyrtyshnikov, Nabben, ...

18

The actual decay

020

4060

80100

0

20

40

60

80

1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 20 40 60 80 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

row index

ma

gn

itu

de

column 35 of S−1

... a very peculiar pattern ⇒ much higher sparsity

19

Where do the repeated peaks come from?

For S = A⊗ In + In ⊗A ∈ Rn2×n2

:

xt := (S−1):,t = S−1et ⇔ Solve : Sxt = et

Let

Xt ∈ Rn×n be such that xt = vec(Xt)

Et ∈ Rn×n be such that et = vec(Et)

Then

Sxt = et ⇔ AXt +XtA = Et

20

Where do the repeated peaks come from?

For S = A⊗ In + In ⊗A ∈ Rn2×n2

:

xt := (S−1):,t = S−1et ⇔ Solve : Sxt = et

Let

Xt ∈ Rn×n be such that xt = vec(Xt)

Et ∈ Rn×n be such that et = vec(Et)

Then

Sxt = et ⇔ AXt +XtA = Et

21

The Poisson equation - revisited

−uxx − uyy = f, in Ω = (0, 1)2

+ Dirichlet b.c. (zero b.c. for simplicity)

22

The Poisson equation - revisited

−uxx − uyy = f, in Ω = (0, 1)2

+ Dirichlet b.c. (zero b.c. for simplicity)

FD Discretization: Ui,j ≈ uxi,yj , with (xi, yj) interior nodes, so that

uxx(xi, yj) ≈Ui−1,j − 2Ui,j + Ui+1,j

h2=

1

h2[1,−2, 1]

Ui−1,j

Ui,j

Ui+1,j

uyy(xi, yj) ≈Ui,j−1 − 2Ui,j + Ui,j+1

h2=

1

h2[Ui,j−1, Ui,j , Ui,j+1]

1

−2

1

AU+UA = F, Fij = f(xi, yj)

23

For S the 2D Laplace operator, t = 1, . . . , n2

t = 35, Sxt = et ⇔ AXt +XtA = Et

02

46

810

0

2

4

6

8

100

0.2

0.4

0.6

0.8

1

02

46

810

0

2

4

6

8

100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

matrix Et and matrix Xt

Et has only one nonzero element

Lexicographic order: (Et)ij , j = ⌊(t− 1)/n⌋+ 1, i = tn⌊(t− 1)/n⌋

24

For S the 2D Laplace operator, t = 1, . . . , n2

t = 35, Sxt = et ⇔ AXt +XtA = Et

02

46

810

0

2

4

6

8

100

0.2

0.4

0.6

0.8

1

02

46

810

0

2

4

6

8

100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

matrix Et and matrix Xt

Et has only one nonzero element

Lexicographic order: (Et)ij , j = ⌊(t− 1)/n⌋+ 1, i = tn⌊(t− 1)/n⌋

25

0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

component0

24

68

10

0

2

4

6

8

100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

ij

Left: Row of S−1 Right: same row on the grid

26

0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

component0

24

68

10

0

2

4

6

8

100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

ij


27

0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

component0

24

68

10

0

2

4

6

8

100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

ij


28

0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

component0

24

68

10

0

2

4

6

8

100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

ij


29

0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

component0

24

68

10

0

2

4

6

8

100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

ij


30

0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

component0

24

68

10

0

2

4

6

8

100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

ij


31

Qualitative bounds (more general than for the Laplacian!)

Let κ = λmax/λmin = cond(A)

i)Assume ℓ, i,m, j : ℓ 6= i, m 6= j. n2 := |ℓ− i|+ |m− j| − 2 > 0

|(S−1)k,t| ≤√κ2 + 1

2λmin

1√n2

.

ii)Assume ℓ, i,m, j : ℓ = i or m = j. n1 := |ℓ− i|+ |m− j| − 1 > 0

|(S−1)k,t| ≤κ√κ2 + 1

2

1√n1

.

02

46

810

0

2

4

6

8

100

0.2

0.4

0.6

0.8

1

02

46

810

0

2

4

6

8

100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

(i, j) and (ℓ,m)

32

Example. Legendre stiffness matrix (scaled to have unit peak)

A = tridiag(δk, γk, δk)

0 20 40 60 80 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

row index

ma

gn

itu

de

column 35 of S−1

new estimate γk =

2

(4k − 3)(4k + 1)

k = 1, . . . , n, and

δk =−1

(4k + 1)√

(4k − 1)(4k + 3)

k = 1, . . . , n − 1

Canuto, Simoncini, Verani 2014

33

Connections to point-wise estimates for discrete Laplacian

For the discrete Green function Gh on the discrete d-dimensional grid

Rh, there exist constants h0 and C such that for h ≤ h0, x, y ∈ Rh,

Gh(x, y) ≤

C log C|x−y|+h

if d = 2

C(|x−y|+h)d−2 if d ≥ 3

(Bramble & Thomee, ’69)

Our estimate: entries depend on inverse square root of the distance!

34

Typical decay plot for f(S), with S Laplace operator as before

020

4060

80100

0

20

40

60

80

1000

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0.018

020

4060

80100

0

20

40

60

80

1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

f(S) = e−5S f(S) = S− 1

2

(Bounds for Laplace or Stieltjes functions)

In general, S = A1 ⊕A2 := A1 ⊗ I + I ⊗A2, A1, A2 banded spd

Benzi, Simoncini 2015

35

Generalizations

• Three-dimensional case

• (banded) Non-symmetric matrices

• “Quasi” Kronecker structure

• Numerical solution of PDEs on structured grids

36

−∆u = 1, Ω = (0, 1)3 ⇒ S = (A⊗I⊗I+I⊗A⊗I+I⊗I⊗A)

CG for Sx = b vs Iterative solver for (I ⊗A+A⊗ I)U+UA = F

A ∈ Rn×n, S ∈ R

n3×n3

, n = 50

CG PCG Matrix Eqn solver

Comput. Time 2.91 0.56 0.08

37

−∆u = 1, Ω = (0, 1)3 ⇒ S = (A⊗I⊗I+I⊗A⊗I+I⊗I⊗A)

CG for Sx = b vs Iterative solver for (I ⊗A+A⊗ I)U+UA = F

A ∈ Rn×n, S ∈ R

n3×n3

, n = 50

0 10 20 30 40 50 60 70 8010

−9

10−8

10−7

10−6

10−5

10−4

10−3

10−2

10−1

100

number of iterations/ space dim

resid

ua

l n

orm

CG

PCG

Sylv

CG PCG Matrix Eqn solver

Comput. Time 2.91 0.56 0.08

38

Generalizations. Solutions to PDEs

−uxx − uyy = f, (x, y) ∈ Ω

In polar coordinates (r, θ): −urr − 1rur − uθθ = f

⇒ A1X+XA2 = F

39

Generalizations. Solutions to PDEs

−uxx − uyy = f, (x, y) ∈ Ω

0.5

1

1.5

2

0

0.5

1

1.50

0.01

0.02

0.03

0.04

0.05

0.06

0.07

In polar coordinates (r, θ): −urr − 1rur − uθθ = f

⇒ A1X+XA2 = F

40

Structured grids

Applications

• Computational Aero- and Fluid-Dynamics

• Seminconductor devices

• Object modelling

• Parallel computation

• ...

Classical strategies (building blocks)

• Conformal mappings (Boundary-fitted curvilinear coord.)

• Algebraic grid generators (Transfinite interpolation)

• Elliptic, hyperbolic grids with controls

• Variational methods

• ...

41

Grid generation. An example

(grids from http://www.math.fsu.edu/ okhanmoh/research.html)

42

Conclusions

• Matrix equations have very broad applicability

(structure recurrent in many application problems...)

• Recent appropriate computational devices

• Important tool for matrix sparsity analysis

references:

M. Benzi, V. Simoncini, SIMAX, v.36, 2015.

C. Canuto, V. Simoncini and M. Verani, LAA, v.452, 2014.

V. Simoncini, Survey on computational methods for matrix equations, to appear in

SIAM Review

43

Documents

Equazioni lineari matriciali propriet`a, metodi numerici ...simoncin/umi15.pdf · Equazioni lineari matriciali propriet`a, metodi numerici ed applicazioni V.Simoncini Dipartimento