Accelerated Linear Algebra Libraries

James Wynne IIINCCS User Assistance

2

Accelerated Linear Algebra Libraries

• Collection of functions to preform mathematical operations on matrices

• Designer has re-written the standard LAPACK functions to make use of GPU accelerators to speed up execution on large matrices

3

MAGMA HOST INTERFACE

Accelerated Linear Algebra Libraries

4

MAGMA - Host

• MAGMA stands for Matrix Algebra on GPU and Multicore Architecture

• Developed by the Innovative Computing Laboratory at the University of Tennessee

• Host interface allows easy porting from CPU libraries (like LAPACK) to MAGMA’s accelerated library– Automatically manages data allocation and transfer

between CPU (Host) and GPU (Device)

5

MAGMA - Host• Fortran:– To run MAGMA functions from Fortran, an Interface block needs to be

written for each MAGMA function that’s being called. These interfaces will be defined file magma.f90

– Example:module magma

Interface Integer Function magma_sgesv(n,nrhs,…)& BIND(C,name=“magma_sgesv”)

use iso_c_binding

implicit none

integer(c_int), value :: n

integer(c_int), value :: nrhs

…

end Function

end Interface

end module

6

MAGMA - Host

• Pseudo-code for a simple SGESV operation in magma

Program SGESV!Include the module that hosts your interfaceuse magma use iso_c_binding!Define your arrays and variablesReal(C_FLOAT) :: A(3,3), b(3)Integer(C_INT) :: piv(3), ok, status!Fill your `A` and `b` arrays then call!MAGMA_SGESVstatus = magma_sgesv(3,1,A,3,piv,b,3,ok)!Loop through and write(*,*) the contents of !array `b`

end Program

7

MAGMA - Host

• Pseudo-code for a simple SGESV operation in magma

Program SGESV!Include the module that hosts your interfaceuse magmause iso_c_binding!Define your arrays and variablesReal(C_FLOAT) :: A(3,3), b(3)Integer(C_INT) :: piv(3), ok, status!Fill your `A` and `b` arrays then call!MAGMA_SGESVstatus = magma_sgesv(3,1,A,3,piv,b,3,ok)!Loop through and write(*,*) the contents of !array `b`

end Program

8

MAGMA - Host

• Pseudo-code for a simple SGESV operation in magma

Program SGESV!Include the module that hosts your interfaceuse magmause iso_c_binding!Define your arrays and variablesReal(C_FLOAT) :: A(3,3), b(3)Integer(C_INT) :: piv(3), ok, status!Fill your `A` and `b` arrays then call!MAGMA_SGESVstatus = magma_sgesv(3,1,A,3,piv,b,3,ok)!Loop through and write(*,*) the contents of !array `b`

end Program

9

MAGMA - Host

• Before compiling, Make sure the MAGMA module, CUDA toolkit and the GNU programming environment is loaded– magma.f90: Contains the Interface block module– sgesv.f90: Contains the Fortran source code

• To compile:$ module swap PrgEnv-pgi PrgEnv-gnu

$ module load cudatoolkit magma

$ ftn magma.f90 –lcuda –lmagma –lmagmablas sgesv.f90

10

MAGMA - Host

• C:

• In C source code, no kind of interface block is needed like in Fortran

• Simply #include<magma.h> in your code

• When declaring variables to use with MAGMA functions, use magma_int_t instead of C’s int type. Matrices for MAGMA’s SGESV are of type float

11

MAGMA - Host• Example code for C:

#include<magma.h>#include<stdio.h>

int main(){ //Define Arrays and variables for MAGMA float b[3], A[3][3]; magma_int_t m = 3, n = 1, piv[3] ok;

//Fill matrices A and b and call magma_sgesv() magma_sgesv(m,n,A,m,piv,b,m,&info); //Loop through and print out returned array b}

12

MAGMA - Host• Example code for C:

#include<magma.h>#include<stdio.h>

int main(){ //Define Arrays and variables for MAGMA float b[3], A[3][3]; magma_int_t m = 3, n = 1, piv[3] ok;

//Fill matrices A and b and call magma_sgesv() magma_sgesv(m,n,A,m,piv,b,m,&info); //Loop through and print out returned array b}

13

MAGMA - Host• Example code for C:

#include<magma.h>#include<stdio.h>

int main(){ //Define Arrays and variables for MAGMA float b[3], A[3][3]; magma_int_t m = 3, n = 1, piv[3] ok;

//Fill matrices A and b and call magma_sgesv() magma_sgesv(m,n,A,m,piv,b,m,&info); //Loop through and print out returned array b}

14

MAGMA - Host• Example code for C:

#include<magma.h>#include<stdio.h>

int main(){ //Define Arrays and variables for MAGMA float b[3], A[3][3]; magma_int_t m = 3, n = 1, piv[3] ok;

//Fill matrices A and b and call magma_sgesv() magma_sgesv(m,n,A,m,piv,b,m,&info); //Loop through and print out returned array b}

15

MAGMA - Host

• Before compiling, Make sure the MAGMA module, CUDA toolkit and the GNU programming environment is loaded

• To compile:$ module swap PrgEnv-pgi PrgEnv-gnu

$ module load cudatoolkit magma

$ cc –lcuda –lmagma –lmagmablas sgesv.c

16

MAGMA DEVICE INTERFACE

Accelerated Linear Algebra Libraries

17

MAGMA - Device

• MAGMA Device interface allows direct control over how the GPU (device) is managed– Memory allocation and transfer– Keeping matrices on the device

18

MAGMA - Device

• Fortran:– To run MAGMA device functions from Fortran, an Interface

block needs to be written for each MAGMA function that’s being called. This is not required in C source code

– Device functions suffixed with _gpu– CUDA Fortran used to manage memory on the device– All interface blocks need to be defined in a module

(module magma) • If module exists in a separate file, file extension must be .cuf, just like

the source file

19

MAGMA - Device

• Example:module magma

Interface

Integer Function magma_sgesv_gpu(n,nrhs,dA…)& BIND(C,name=“magma_sgesv_gpu”)

use iso_c_binding

use cudafor

implicit none

integer(c_int), value :: n

integer(c_int), value :: nrhs

real (c_float), device, dimension(:)::dA(*)

…

end Function

end Interface

…

20

MAGMA - Device

• Also need the MAGMA initialize function (defined in the same interface block module):

…

Interface

Integer Function magma_init() & BIND(C,name=“magma_init”)

use iso_c_binding

implicit none

end Function

end Interface

end module

21

MAGMA - Device

• Pseudo-code for a simple SGESV operation in magma

Program SGESV!Include the module that hosts your interfaceuse magmause cudaforuse iso_c_binding!Define your arrays and variablesReal(C_FLOAT) :: A(3,3), b(3)Real(C_FLOAT),device,dimension(:,:) :: dAReal(C_FLOAT),device,dimension(:) :: dBInteger(C_INT),value :: piv(3), ok, status

22

MAGMA - Device

• Pseudo-code for a simple SGESV operation in magma

Program SGESV!Include the module that hosts your interfaceuse magmause cudaforuse iso_c_binding!Define your arrays and variablesReal(C_FLOAT) :: A(3,3), b(3)Real(C_FLOAT),device,dimension(:,:) :: dAReal(C_FLOAT),device,dimension(:) :: dBInteger(C_INT),value :: piv(3), ok, status

23

MAGMA - Device

• Pseudo-code for a simple SGESV operation in magma

Program SGESV!Include the module that hosts your interfaceuse magmause cudaforuse iso_c_binding!Define your arrays and variablesReal(C_FLOAT) :: A(3,3), b(3)Real(C_FLOAT),device,dimension(:,:) :: dAReal(C_FLOAT),device,dimension(:) :: dBInteger(C_INT),value :: piv(3), ok, status

24

MAGMA - Device

• Pseudo-code for a simple SGESV operation in magma

!Fill your `A` and `b` arrays then initialize !MAGMA status = magma_init()!Copy filled host arrays `A` and `b` to `dA`!and `dB` using CUDA FortrandA = AdB = b!Call the device functionstatus = magma_sgesv_gpu(3,1,dA,3,piv,dB,3,ok)!Copy results back to CPUb = dB!Loop through and write(*,*) the contents of !array `b`

end Program

25

MAGMA - Device

• Pseudo-code for a simple SGESV operation in magma

!Fill your `A` and `b` arrays then initialize !MAGMA status = magma_init()!Copy filled host arrays `A` and `b` to `dA`!and `dB` using CUDA FortrandA = AdB = b!Call the device functionstatus = magma_sgesv_gpu(3,1,dA,3,piv,dB,3,ok)!Copy results back to CPUb = dB!Loop through and write(*,*) the contents of !array `b`

end Program

26

MAGMA - Device

• Pseudo-code for a simple SGESV operation in magma

!Fill your `A` and `b` arrays then initialize !MAGMA status = magma_init()!Copy filled host arrays `A` and `b` to `dA`!and `dB` using CUDA FortrandA = AdB = b!Call the device functionstatus = magma_sgesv_gpu(3,1,dA,3,piv,dB,3,ok)!Copy results back to CPUb = dB!Loop through and write(*,*) the contents of !array `b`

end Program

27

MAGMA - Device

• Pseudo-code for a simple SGESV operation in magma

!Fill your `A` and `b` arrays then initialize !MAGMA status = magma_init()!Copy filled host arrays `A` and `b` to `dA`!and `dB` using CUDA FortrandA = AdB = b!Call the device functionstatus = magma_sgesv_gpu(3,1,dA,3,piv,dB,3,ok)!Copy results back to CPUb = dB!Loop through and write(*,*) the contents of !array `b`

end Program

28

MAGMA - Device

• Before compiling, Make sure the MAGMA module, CUDA toolkit and the PGI programming environment is loaded– magma.cuf: Contains module of Interface blocks– sgesv.cuf: Contains the Fortran source

• To compile:$ module load cudatoolkit magma

$ ftn magma.cuf –lcuda –lmagma –lmagmablas sgesv.cuf

29

MAGMA - Device

• C:

• In C Device source code, no kind of interface block is needed like in Fortran

• Simply #include<magma.h> in your code

• When declaring variables to use with MAGMA functions, use magma_int_t instead of C’s int type. Matrices for MAGMA’s SGESV are of type float

• Before running any MAGMA Device code, magma_init()must be called.

30

MAGMA - Device

• C:

• To interact with the device (Allocate matrices, transfer data, etc) use the built in MAGMA functions– Allocate on the device: magma_dmalloc()– Copy matrix to device: magma_dsetmatrix()– Copy matrix to host: magma_dgetmatrix()

31

MAGMA - Device• Example code for C:

#include<magma.h>#include<stdio.h>

int main(){ //Define Arrays and variables for MAGMA float b[3], A[3][3]; float *A_d, *b_d //Device pointers magma_int_t m = 3, n = 1, piv[3] ok;

//Fill matrices A and b and allocate device //matrices magma_dmalloc(&A_d, m*m); magma_dmalloc(&b_d, m);

32

MAGMA - Device• Example code for C:

#include<magma.h>#include<stdio.h>

int main(){ //Define Arrays and variables for MAGMA float b[3], A[3][3]; float *A_d, *b_d //Device pointers magma_int_t m = 3, n = 1, piv[3] ok;

//Fill matrices A and b and allocate device //matrices magma_dmalloc(&A_d, m*m); magma_dmalloc(&b_d, m);

33

MAGMA - Device• Example code for C:

#include<magma.h>#include<stdio.h>

int main(){ //Define Arrays and variables for MAGMA float b[3], A[3][3]; float *A_d, *b_d //Device pointers magma_int_t m = 3, n = 1, piv[3] ok;

//Fill matrices A and b and allocate device //matrices magma_dmalloc(&A_d, m*m); magma_dmalloc(&b_d, m);

34

MAGMA - Device• Example code for C:

//Transfer matrices to device magma_dsetmatrix(m,m,A,m,A_d,m); magma_dsetmatrix(m,n,b,m,b_d,m); //Call the device sgesv function magma_sgesv_gpu(m,n,A_d,m,piv,b_d,m,&info); //Copy back computed matrix magma_dgetmatrix(m,n,b_d,m,b,m); //Loop through and print out returned array b}

35

MAGMA - Device• Example code for C:

//Transfer matrices to device magma_dsetmatrix(m,m,A,m,A_d,m); magma_dsetmatrix(m,n,b,m,b_d,m); //Call the device sgesv function magma_sgesv_gpu(m,n,A_d,m,piv,b_d,m,&info); //Copy back computed matrix magma_dgetmatrix(m,n,b_d,m,b,m); //Loop through and print out returned array b}

36

MAGMA - Device• Example code for C:

//Transfer matrices to device magma_dsetmatrix(m,m,A,m,A_d,m); magma_dsetmatrix(m,n,b,m,b_d,m); //Call the device sgesv function magma_sgesv_gpu(m,n,A_d,m,piv,b_d,m,&info); //Copy back computed matrix magma_dgetmatrix(m,n,b_d,m,b,m); //Loop through and print out returned array b}

37

MAGMA - Host

• Before compiling, Make sure the MAGMA module, CUDA toolkit and the GNU programming environment is loaded

• To compile:$ module swap PrgEnv-pgi PrgEnv-gnu

$ module load cudatoolkit magma

$ cc –lcuda –lmagma –lmagmablas sgesv_gpu.c