L U Decomposition

Demonstrating the quick way to generate matrix elements

-Dave C, 2015

LU decomposition is a tedious darned process at the best of times, if you have to do it by hand. There is a strong incentive to minimise the number of steps so that construction time is reduced and the number of repetitions due to construction error is also reduced. Let’s be honest, if you have to do maybe fifty matrix operations by hand in order to obtain the L and U matrices, there’s a very high chance you got it wrong somewhere. So here’s the method as simple as I have seen it and explained as clearly as I can make it. -Dave C, 2015.

LU decomposition turns a dense, square matrix into a couple of triangular matrices so that a given column vector can be identified by a forward substitution and a back substitution. You’ve all had the lecture before at university so you all understand the basic idea. It’s just the fiddly details of producing the matrix elements that drive you insane. The thing to note is that there are different ways to do this, producing different L and U matrices. You can get LU decompositions in which the U matrix has the 1s on the diagonal, and LU decompositions in which the L matrix has 1s down the diagonal. It doesn’t matter if your only concern is using the matrices. But producing these matrices is a pain which can be minimised if you follow the standard routine. The best routine has 1s on the diagonal of the L matrix.

Here’s a matrix that I will convert to LU form.

The top line stays the same. The first diagonal element is special, as you’ll see in a moment. It’s called the pivot.

Divide the elements in the first column by the pivot (that’s the blue column).

Now be careful! You’re going to multiply A12 (dark-green) by A21 (blue) and subtract it from A22 (light-green). The result will be 5 – (2x2) = 1.

5-(2x2)

Process the other elements in the second row in the same way.

5-(2x2)

Process the other elements in the second row in the same way.

4-(2x1)

Done!

Now process the lower rows in exactly the same way. Be sure not to reverse the order of subtraction. You are subtracting the top from the bottom, not the other way around.

3-(1x2)

5-(1x2)

4-(1x1)

Now process the lower rows in exactly the same way. Be sure not to reverse the order of subtraction. You are subtracting the top from the bottom, not the other way around.

6-(2x2)

8-(2x2)

8-(2x1)

Done!

Now we’re going to ignore the first row and the first column and do the same process again on the remaining sub-matrix. This is a smaller matrix so things will go much faster now.

Divide the first column by the pivot. (In this example, dividing by the 1 in the pivot box makes no difference)

Process the elements of the second and third rows.

Process the elements of the second and third rows.

Done!

Now proceed inwards to the next smaller matrix, and process that in the same way. Remember not to reverse the order of subtraction or scale the lines in any other way.

Divide the first column by the pivot. In this example there is only one number to divide.

Process the line as before. In this case, since this is the last row, there is only one number to process. 2 – (1 x 1) = 1

Done! The LU decomposition is essentially complete. All you need to do now is separate it into its two parts.

Break the matrix so that the diagonal elements go with the upper triangle.

L U

Add a diagonal row of 1s to the L matrix. Fill the remaining spaces with zeroes. (I have used dots instead).

That’s it! Job done.

If you want to know why this procedure works, think of it as Gaussian Elimination with the elimination steps recorded in the margin. The ‘margin’ is the lower left part of the matrix. The numbers in blue are the scale factors you have to use in order to eliminate a given row.

For example, to eliminate the 6 from the second row of this matrix, I have to double the 3 above it before subtracting. Therefore the scale factor for the whole second row is 2. I am subtracting two times the top row from the second row to produce a new second row that has a zero in the first column. Since there is a zero in the first column, I can use that space to record the scale factor that I used. That later becomes part of the L matrix.

Done by hand, this is a very tedious process for matrices bigger than 3x3. I would hate to do it for a 5x5. Your teachers want you to do a few hand-drawn LU decompositions only so that you get familiar with the process and understand how it works. After that you can safely retreat to using a spreadsheet (as I have done in producing this example) or a bigger computer. Most engineering program libraries will have an LU matrix maker tucked away in it somewhere, so you won’t have to write the code yourself. Most of these programs will be older than you are, having been written back in the 1960s on mainframe computers and simply carried forward through succeeding generations of technology. But there is value in understanding how a thing works because then you can understand what has happened when things go wrong, and perhaps make things go better.

Can anything go wrong with LU decomposition? Consider the possibilities: 1: What would happen to a process like this if you had a singular matrix? Solving a matrix problem by LU decomposition this is a sneaky way of

inverting a matrix. It’s good but it’s still inverting a matrix. So if the matrix can’t be inverted then the process will break down somewhere, usually by dividing something by zero.

2: What happens if the matrix is invertible but puts a zero into one of the pivot

boxes anyway? In that case, the process I have described will fail, again manifesting as a division by zero. The process can be modified to swap zero pivots with other numbers in the same column but that requires a bit of decision-making in your program or you watching the steps of the process closely to swap rows manually if necessary.

3: Why do this? Good question. Your lecturers will tell you that LU decomposition is preferred

to matrix inversion because it requires less work and therefore obtains an answer cheaper and faster. These days that can only be true if the matrix system you are processing is absolutely huge.

How huge is huge? That one I can’t answer. 10,000 elements? A million

elements? I just used my spreadsheet to invert a 50x50 matrix just to see how long it would take and it finished the process in a split second. That’s 2500 elements to look at, theoretically 42,000 arithmetic operations to get the result. I then got my spreadsheet to invert a 200x200 matrix. That’s now 40,000 elements. My laptop had to think about that for about 3 seconds but still came up with a result. So how big does a matrix have to be to require economic alternatives like LU decomposition? Are we carrying into the 21st century a concern that really only applied to the smaller, slower computers of the 20th century? I don’t know.

If it takes my laptop 3 seconds to invert a 200x200 matrix, that’s 3 seconds to do approximately 2.7 million arithmetic operations. If I made the matrix ten times deeper and wider, the processing time would theoretically be a thousand times longer, amounting to 3000 seconds. That would be most of an hour. That’s a matrix with 4 milllion elements in it. An hour is a long time, but still small compared to what my lecturer bragged about when he was a student in the 1960s. In those days he would load a deck of punch cards into a chattering mainframe during the night and then retire to his cot in the corner of the room for several hours, loading another deck of cards whenever a bell rang to tell him that the previous job was done. Man, those were the days! So it seems to me that LU decomposition is only a viable economic alternative to brute force matrix inversion for matrices of dimension well over a thousand, possibly tens of thousands. I wonder what kind of projects require the simultaneous determination of tens of thousands of interrelated unknowns? The mind quivers. -Dave C, 2015.