Matrices and Libraries

7/23/2019 Matrices and Libraries

1/51

Mathematical Libraries for LinearAlgebra

William Oquendo, [email protected]

Credits: Computational Physics - Landau and Paez, Deitel,

cplusplus tutorial, GSL, Gnu free software fundation, eigen c++,armadillo, python

Thursday, November 14, 13
mailto:[email protected]:[email protected]


2/51

Matrices: What experts say

Landau y Paez



3/51

Matrices: Typical examples

Ax = b

ANNxN1 = bN1Ax =b

Solve a linear systemEigenvalue problems

C = AB

Matrix Multiplication

A = LU

Matrix Decomposition

C = Ax+By

Linear Combinations...



4/51

Matrices: Practical aspects



5/51




6/51




7/51

Solving typical linear problems Gauss-Jordan elimination: Well suited for finding inverses, not very

efficient, http://en.wikipedia.org/wiki/Gauss-Jordan_elimination

Gauss elimination: Very efficient, can also find the rank of a matrix,and compute the inverse of a square matrix, http://en.wikipedia.org/wiki/Gaussian_elimination

LU Decomposition: Write the original matrix as a product of an

Lower and an Upper triangular matrices, useful for computing thedeterminant, http://en.wikipedia.org/wiki/LU_decomposition

Cholesky decomposition: Valid for symmetric definite matrices,t w i c e e f fi c i e n t than LU, h t t p : / / e n . w i k i p e d i a . o r g / w i k i /Cholesky_decomposition

QR decomposition: Orthogonal and rigth triangular, linear squaresproblems, http://en.wikipedia.org/wiki/QR_decomposition

...

http://en.wikipedia.org/wiki/QR_decompositionhttp://en.wikipedia.org/wiki/QR_decompositionhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/QR_decompositionhttp://en.wikipedia.org/wiki/QR_decompositionhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/Gauss-Jordan_eliminationhttp://en.wikipedia.org/wiki/Gauss-Jordan_elimination


8/51

Matrix problems: best method forsolving .... Use scientific libraries!!!!



9/51

Why scientific libraries? Routines for common operations, like vector and matrix

operations, or Fourier transforms, implemented to take mostadvantages from cpu.

Can be optimized for each cpu.

Coding is easier since API is stable.

Increase portability (Dont use non-standard libraries!)

Better to help compiler to perform optimization.

Developer time is saved. Efficiency increased.

Can implement multithreaded operations out of the box.

Cache friendly.

...



10/51

Linear Algebra: Main numericallibraries

BLAS: Basic Linear Algebra Subroutines. Standard API for lowlevel matrix operations, like matrix multiplication.

LaPACK: Linear Algebra Package based on BLAS. Optimized.Real or Complex.

Fast(est) Fourier Transform (in The West) FFTW.

MKL: BLAS LAPACK routines optimized for Intel machines.

AMD Math Core Library - ACML : For AMD machines.

ATLAS: Automatically tuned BLAS and LAPACK

GSL, eigen, armadillo, ... : More general libraries



11/51

BLAS



12/51

GSL



13/51

What about GSL and BLAS?



14/51

How to use the GSL?

Please red the tutorial, or man gsl, or info gsl.



15/51

How to use the GSL? (once installed)

linking to gsl and gslcblas

linking to gsl and blas and atlas



16/51

How to use the GSL?



17/51

How to use the GSL?Check!



18/51

How to use the GSL?



19/51

Matrix solution example



20/51

Matrix solution example



21/51

Example of GSL usage



22/51

Example of GSL usage



23/51

Examples of GSL usage

introGSL.cpp

gslMatrixBasic.cpp

gslVectorBasic.cpp

blas.c

blasV2.c

gsllinalgLU.cpp

gsllinalgQRls.cpp

gsllinalgCholeski.cpp

eigen_nonsymmMOD.c



24/51

Matrices: Exercises



25/51

Matrices: Exercises



26/51

Eigen C++ LinAlg library

http://eigen.tuxfamily.org/index.php?title=Main_Page

http://eigen.tuxfamily.org/index.php?title=Main_Pagehttp://eigen.tuxfamily.org/index.php?title=Main_Page


27/51

Eigen C++ LinAlg library



28/51

How to use Eigen? headers only!



29/51

Fully templated



30/51

Several headers for several areas


Q i k f (1)


31/51

Quickref (1)


Q i k f (2)


32/51

Quickref (2)


Li Al b S S l i


33/51

Linear Algebra: System Solution


Some Linear Algebra Solving


34/51

Some Linear Algebra SolvingMethods



35/51

Eigen Solvers



36/51

Inverse and others



37/51

Armadillo C++ LinAlg library



38/51

Armadillo should be compiled



39/51

Matrix type



40/51

Linear Systems



41/51

Eigen decompositions (1)



42/51

Eigen decompositions (2)



43/51

Python scipy.linalg library



44/51




45/51




46/51

Python example 1

import


P h l II


47/51

Python example II


E P f C


48/51

Exercise : Performance Comparison

The goal is to compare the performance of each library for solvingrandomly-generated matrices as a function of the size, testing also theactual algorithm used to solve the system. Optionally, the eigen valuedecomposition can also be useful.


Ti i i i C/C


49/51

Timing routines in C/C++

gnu timecommand

std::chronosor clock


Ti i i i P h


50/51

Timing routines in Python

gnu timecommand

timeit


E i /H k


51/51

Exercise/Homework

For a linear system solved by QR and LU, compute the performance ofeigen, gsl+cblas, armadillo, and python, as function of the matrix size.

Final result: Several figures, one per method (QR and LU and simplesolve). Each figure showing user-time versus matrix size per library/method (several curves per figure).

Details:

Change matrix size from 2x2 up to 1024x1024 (or higher).

Initialize randomly the matrices at each time step using seed=0 and

srand48 or srand if needed explicitly.

Print only the average after 20 tries to improve statistics.

Before starting the benchmarks, make some test to check if thesolutions are the solutions.

Documents

Matrices and Libraries