Solving Linear Recurrence Relations

IntroductionRecall from the presentation on sequences and summation that a (k-step) recurrence relation is an equation that defines the elements of a sequence recursively, i.e. in terms of their own predecessors, going back k indices:

𝑎𝑛 = 𝑓(𝑎𝑛−1, 𝑎𝑛−2, … , 𝑎𝑛−𝑘)

A recursive definition involving a k-step recurrence relation requires the values of the first k terms: 𝑎0, 𝑎1, … , 𝑎𝑘 , called the initial conditions, to define a sequence properly.

The Fibonacci sequence, 𝑓0= 1, 𝑓1 = 1, 𝑓𝑛 = 𝑓𝑛−1 + 𝑓𝑛−2 for 𝑛 = 2,3,4, … is the most well known example of a sequence that is defined recursively.

Recursive definitions are not computationally effective for very large indices. How would you evaluate 𝑓𝑛 where 𝑛 = 10100? Computing this using the recursive definition would require about 10100

additions, which would take far longer than the age of the universe to carry out even on a very fast computer, and in light of the estimate we proved in the presentation on recursively defined functions and sets,

𝑓𝑛 >3

2

𝑛−2

𝑓𝑛 would have far more digits than the estimated number of atoms in the universe, so it could never even be stored.

Therefore , a natural question with a recursive definition is: can we find a closed-form definition, i.e. a definition that lets us compute (or find a good estimate of) 𝑎𝑛 as a function of 𝑛 alone?

While there is no general method for solving a recurrence relation, there is one that works for linear recurrence relations with constant coefficients, i.e. recurrence relations of the form

𝑎𝑛+𝑘 = 𝑐1𝑎𝑛+𝑘−1 + 𝑐2𝑎𝑛+𝑘−2 +⋯+ 𝑐𝑘𝑎𝑛

where the 𝑐1, … , 𝑐𝑘 are constants. An example of such a recurrence relation would be the 3-step recurrence relation

𝑎𝑛+3 = 𝑎𝑛+2 + 4𝑎𝑛+1 − 4𝑎𝑛

Bad News and Good News

Idea of the MethodOne of the simplest linear, constant-coefficient recurrence relation is 𝑎𝑛+1 = 2𝑎𝑛. Since the sequence value doubles with each step, we know immediately that solution sequences of this recurrence are exponential (aka geometric), i.e. 𝑎𝑛 = 𝑐2𝑛 where 𝑐 = 𝑎0 is the initial value. In particular, 𝑎𝑛 = 2𝑛 is a solution.

By a solution of a recurrence relation, we mean a sequence whose terms satisfy the recurrence relation. In the example above, 𝑎𝑛+1 =2𝑛+1 = 2 ∙ 2𝑛 = 2𝑎𝑛.

This gives us the idea: perhaps all linear, constant-coefficient recurrence relations have exponential solutions? The answer turns out to be affirmative, and this enables us to find all solutions.

Given a recurrence relation, our goal is to find an exponential solution 𝑎𝑛 = 𝑟𝑛, where the unknown base 𝑟 ≠ 0 remains to be determined.

We substitute 𝑎𝑛 = 𝑟𝑛 into the recurrence and find the values of 𝑟 for which the recurrence is satisfied.

Example 1𝑎𝑛+3 = 𝑎𝑛+2 + 4𝑎𝑛+1 − 4𝑎𝑛

Substituting 𝑎𝑛 = 𝑟𝑛, we get

𝑟𝑛+3 = 𝑟𝑛+2 + 4𝑟𝑛+1 − 4𝑟𝑛

or𝑟𝑛+3 − 𝑟𝑛+2 − 4𝑟𝑛+1 + 4𝑟𝑛 = 0

Since all terms have a factor 𝑟𝑛 in common, we can factor it out:

𝑟𝑛 𝑟3 − 𝑟2 − 4𝑟 + 4characteristic polynomial

= 0

We need this to be zero for all 𝑛. Since 𝑟 ≠ 0, the equation can only be satisfied if the cubic factor is zero. We can factor it by grouping:

𝑟3 − 𝑟2 − 4𝑟 + 4 = 𝑟2 𝑟 − 1 − 4 𝑟 − 1 = (𝑟2−4)(𝑟 − 1) = (𝑟 + 2)(𝑟 − 2)(𝑟 − 1).

We just found three 𝑟 values for which the recurrence is satisfied: 𝑟 = −2, 2, 1.

The cubic polynomial whose zeros we just found is called the characteristic polynomial of the recurrence relation.

Now we know that𝑎𝑛+3 = 𝑎𝑛+2 + 4𝑎𝑛+1 − 4𝑎𝑛

is satisfied for 𝑎𝑛= 2𝑛, 𝑎𝑛 = (−2)𝑛 and 𝑎𝑛 = 1.

Let’s verify the first of these solutions before we proceed. (The other one can be verified in the same way.)

If 𝑎𝑛= 2𝑛, then𝑎𝑛+2 + 4𝑎𝑛+1 − 4𝑎𝑛

= 2𝑛+2 + 4 ∙ 2𝑛+1 − 4 ∙ 2𝑛

= 4 ∙ 2𝑛 + 4 ∙ 2𝑛+1 − 4 ∙ 2𝑛

= 4 ∙ 2𝑛+1 = 2𝑛+3.

𝑎𝑛= 2𝑛, 𝑎𝑛 = (−2)𝑛 and 𝑎𝑛 = 1 are not the only solutions of

𝑎𝑛+3 = 𝑎𝑛+2 + 4𝑎𝑛+1 − 4𝑎𝑛however. Any constant multiple of a solution is again a solution, and a sum of solutions is again a solution.

Therefore, if 𝑎𝑛, 𝑏𝑛 and 𝑐𝑛 are solutions, then so must be 𝑝𝑎𝑛 + 𝑞𝑏𝑛 + 𝑟𝑐𝑛. Such a sum of the individual solutions with constant factors attached is known as a linear combination of solutions.

Therefore, for any numbers 𝑝, 𝑞 and 𝑟,

𝑎𝑛= 𝑝2𝑛 + 𝑞 −2 𝑛 + 𝑟

is a solution of our recurrence. In fact, it is the general solution. By that, we mean that any solution of the recurrence is contained in the above formula, for a specific value of 𝑝, 𝑞 and 𝑟. We will not prove this, but allude to the reason in non-rigorous terms. First though, we will discuss how initial conditions fit into the picture.

Initial Conditions

Suppose we have a sequence defined recursively by a recurrence relation and a set of initial conditions. Let us study how we can use what we just learned to find a non-recursive description of the sequence. Let us use the following example:

𝑎𝑛+3 = 𝑎𝑛+2 + 4𝑎𝑛+1 − 4𝑎𝑛

with 𝑎0 = 2, 𝑎1 = 0, 𝑎2 = 8.

We know from the calculation we just made that the general solution to the recurrence is

𝑎𝑛 = 𝑝2𝑛 + 𝑞 −2 𝑛 + 𝑟

That means that we can find 𝑝, 𝑞 and 𝑟 to match the initial conditions. Substituting the initial conditions leads to a linear system in 3 equations and 3 unknowns:

𝑎0 = 2 = 𝑝 + 𝑞 + 𝑟𝑎1 = 0 = 2𝑝 − 2𝑞 + 𝑟𝑎2 = 8 = 4𝑝 + 4𝑞 + 𝑟

We can now see why a rigorous, proof-based discussion of the solution theory of linear recurrence relations with constant coefficients and initial conditions is beyond the scope of this class: it requires knowledge of linear algebra. Intuitively though, we can glimpse the reason why the alleged general solution above is the general solution: it has three free variables, p,q,r, and every choice of these variables corresponds to exactly one initial condition 𝑎0, 𝑎1 and 𝑎2, and vice versa.

Solving the linear system

We may solve linear systems of equations by elimination. A full discussion of the elimination method belongs in a linear algebra class; we will only discuss the method by example.

𝑎0 = 2 = 𝑝 + 𝑞 + 𝑟𝑎1 = 0 = 2𝑝 − 2𝑞 + 𝑟𝑎2 = 8 = 4𝑝 + 4𝑞 + 𝑟

We subtract equation 1 from equation 2 to get a new equation, and we subtract equation 2 from equation 3 to get another new equation. In both equations, the variable 𝑟 has been eliminated:

−2 = 𝑝 − 3𝑞8 = 2𝑝 + 6𝑞

Now we add two times the first equation to the second to get 4 = 4𝑝, hence 𝑝 = 1. Back-substituting, we get 𝑞 = 1 and 𝑟 = 0. Therefore, we found

𝑎𝑛 = 2𝑛 + −2 𝑛.

The example we just finished is representative of the general situation, as long as the characteristic polynomial has distinct zeros, i.e. zeros that all have multiplicity 1. (Recall from algebra that the multiplicity of a zero of a polynomial is how many times the corresponding linear factor is repeated in the linear factorization of the polynomial.)

Generalization

We solve a linear, constant-coefficient, k-step recurrence relation

𝑎𝑛+𝑘 = 𝑐1𝑎𝑛+𝑘−1 + 𝑐2𝑎𝑛+𝑘−2 +⋯+ 𝑐𝑘𝑎𝑛

by finding the zeros of the characteristic polynomial

𝑟𝑘 − 𝑐1𝑟𝑘−1 − 𝑐2𝑟

𝑘−2 −⋯− 𝑐𝑘

If these zeros (aka characteristic values) 𝑟1, 𝑟2, … , 𝑟𝑘 are all distinct (equivalently, have multiplicity one), then the general solution of the recurrence relation is

𝑎𝑛 = 𝑝1𝑟1𝑛 + 𝑝2𝑟2

𝑛 +⋯+ 𝑝𝑘𝑟𝑘𝑛

where 𝑝1 to 𝑝𝑘 are constants. For each set of initial conditions, 𝑎0…𝑎𝑘 given, there is one and only one choice of the 𝑝1 to 𝑝𝑘 to match the initial conditions. We find these numbers by solving a linear system of k equations in k unknowns.

Repeated Zeros of the Characteristic Polynomial

When the characteristic polynomial has repeated zeros, then our solution method cannot work because the “general” solution does not have enough free variables. Therefore, the correct general solution must have a different form. We will not prove or explain the following theorem:

if 𝑟 is a repeated zero of the characteristic polynomial with multiplicity 𝑚, then not only is 𝑟𝑛 a solution, but also 𝑛𝑟𝑛, 𝑛2𝑟𝑛, … , 𝑛𝑚−1𝑟𝑛. The general solution is obtained by replacing the constant coefficient of 𝑟𝑛 by a general polynomial of degree 𝑚− 1 in 𝑛.

The required notation to represent this theorem symbolically is a little awkward, so we will illustrate the theorem with an example instead:

Suppose the characteristic polynomial of a linear recurrence relation with constant coefficients is

𝑟 + 1 𝑟 − 2 3 𝑟 + 4 2.Then the general solution is

𝑎𝑛 = 𝑝1 −1 𝑛 + 𝑝2 + 𝑝3𝑛 + 𝑝4𝑛2 2𝑛 + 𝑝5 + 𝑝6𝑛 −4 𝑛

Complex Zeros (I)Sometimes, the zeros of the characteristic polynomial are complex. In that case, the general solution will be formally complex as well, but produce real solutions for all real initial values. Let us consider the following example problem:

𝑎𝑛+2 = −𝑎𝑛

with 𝑎0 = 1 and 𝑎1 = 0.

We can immediately see that this sequence is 1,0,-1,0,1,0,.., but it is not obvious how to describe that with a closed form formula.

The characteristic polynomial is 𝑟2 + 1, hence the two characteristic values are ±𝑖. Thus, the general solution of the recurrence is 𝑎𝑛 = 𝑝𝑖𝑛 + 𝑞 −𝑖 𝑛, with complex numbers p and q. Substituting the initial conditions, we get

1 = 𝑝 + 𝑞0 = 𝑝𝑖 − 𝑞𝑖

By dividing the second equation by 𝑖, we get 0 = 𝑝 − 𝑞. We can add that to the first

equation to get 2𝑝 = 1 or 𝑝 =1

2. It follows that 𝑞 =

1

2. Hence, 𝑎𝑛 =

1

2(𝑖𝑛 + −𝑖 𝑛).

Complex Zeros (II)It is always possible to rewrite a formally complex, but real-valued solution solely in terms of real functions using a piecewise definition. The key to doing that lies in the fact that 𝑖4 = 1, which causes the integer powers of i to assume only 4 distinct values:

𝑖0 = 1, 𝑖1= 𝑖, 𝑖2 = −1, 𝑖3 = −𝑖, 𝑖4= 1, 𝑖5 = 𝑖, 𝑖6 = −1, 𝑖7 = −𝑖, 𝑖8 = 1, etc. More formally:

If n is an integer, then 𝑛 = 4𝑘 + 𝑟 for some integer 𝑘 and 𝑟 ∈ {0,1,2,3}. Then 𝑖𝑛 = 𝑖4𝑘+𝑟 = 𝑖4𝑘𝑖𝑟 =(𝑖4)𝑘𝑖𝑟 = 1𝑘𝑖𝑟 = 𝑖𝑟 . Using that 𝑟 = 𝑛 mod 4, that tells us that

𝑖𝑛 = 𝑖𝑛 mod 4 =

1 if 𝑛 mod 4 = 0𝑖 if 𝑛 mod 4 = 1−1 if 𝑛 mod 4 = 2−𝑖 if 𝑛 mod 4 = 3

For example, our previous solution 𝑎𝑛 =1

2(𝑖𝑛 + −𝑖 𝑛) can be simplified as follows:

𝑎𝑛 =

1 if 𝑛 mod 4 = 00 if 𝑛 mod 4 = 1−1 if 𝑛 mod 4 = 20 if 𝑛 mod 4 = 3

which is of course a piecewise description of 𝑎𝑛 that we could have come up with immediately based on the definition of 𝑎𝑛.

Complex Zeros (III)Let us study an example of a recurrence with complex but not purely imaginary characteristic values:

𝑎𝑛+2 = 𝑎𝑛+1 −2𝑎𝑛 with 𝑎0 = 0 and 𝑎1 = 1.

The characteristic polynomial is 𝑟2 − 𝑟 + 2. By applying the quadratic formula, we find the two characteristic values to be

𝑟1,2 =1 ± 𝑖 7

2.

Therefore, the general solution of the recurrence is 𝑎𝑛 = 𝑝1+𝑖 7

2

𝑛

+ 𝑞1−𝑖 7

2

𝑛

where p and q are

complex numbers. Substituting the initial conditions and simplifying, we get

0 = 𝑝 + 𝑞

2 = 𝑝 1 + 𝑖 7 + 𝑞 1 − 𝑖 7

The first equation implies 𝑞 = −𝑝. Substituting this into the second equation produces

2 = 𝑝 1 + 𝑖 7 − 𝑝 1 − 𝑖 7 = 𝑝 ⋅ 2𝑖 7.

This leads to 𝑝 = −𝑖

7and 𝑞 =

𝑖

7, and to the solution 𝑎𝑛 = −

𝑖

7

1+𝑖 7

2

𝑛

−1−𝑖 7

2

𝑛

.

Observe that this solution is formally complex, even though each 𝑎𝑛 is a real number (in fact, an integer).

Complex Zeros (IV)Since the characteristic roots in our second example are mixed real/imaginary quantities, it is more difficult to make it formally apparent that the solution

𝑎𝑛 = −𝑖

2𝑛 71 + 𝑖 7

𝑛− 1 − 𝑖 7

𝑛

is purely real-valued. We have to use the binomial theorem:

1 + 𝑖 7𝑛− 1 − 𝑖 7

𝑛=

𝑘=0

𝑛𝑛

𝑘𝑖𝑘 − −𝑖 𝑘 7

𝑘

𝑞𝑘

To simplify the quantity 𝑞𝑘, we once again have to make a case distinction here based on the remainder of 𝑘 mod 4. If 𝑘 mod 4 is 0 or 2, then 𝑞𝑘 = 0 since 𝑘 is even in these cases.

If 𝑘 mod 4 = 1, then 𝑞𝑘 = 2𝑖 7𝑘

, and if 𝑘 mod 4 = 3, then 𝑞𝑘 = −2𝑖 7𝑘

. After some

simplification and making the substitution j=2k+1 in the sigma sum, we get the explicitly real representation

𝑎𝑛 =1

2𝑛−1

𝑗=0

𝑛−12

𝑛

2𝑗 + 1−7 𝑗

which cannot be further simplified (and is not an efficient way of computing 𝑎𝑛).