Discrete orthogonal polynomials on equidistant nodes

International Mathematical Forum, 2, 2007, no. 21, 1007 - 1020

Discrete Orthogonal Polynomials

on Equidistant Nodes

Alfredo Eisinberg and Giuseppe Fedele

Dip. di Elettronica, Informatica e SistemisticaUniversita della Calabria

87036 Arcavacata di Rende (CS)Italy

Abstract

In this paper we give an alternative and, in our opinion, more simpleproof for the orthonormal discrete polynomials on a set of equidistantnodes. Such a proof provides a unifying explicit formulation of discreteorthonormal polynomials on an equidistant grid and an explicit formulafor the coefficients of the “three-term recurrence relation”. We show howto efficiently apply these formulas to the problem of least-squares fittingon equidistant nodes. Finally we investigate the problem to determinethe effect of adding a new basis function or taking one away. Thisis useful in the process of trying to discover the optimal set of basisfunctions and the problem to update them when a moving window overthe time series/data stream is used.

Mathematics Subject Classification: 05E35, 93E24

Keywords: Orthogonal polynomials, Least-squares

1 Introduction

Although orthogonal polynomials over discrete sets were considered as earlyas the middle of the nineteenth century by Chebyshev [5, 6, 10, 11], com-paratively little attention had been paid to them until now. They appearnaturally in combinatorics, genetics, statistics and various areas in appliedmathematics (see, for example, [7, 11]). Expansion in orthogonal polynomialsis used, for example, to avoid the difficulties with ill-conditioned systems ofequations which occur in a least squares applications [1]. The Gram polyno-mials [3, 1], orthonormal on the set of equidistant nodes in [−1, 1], are used

1008 A. Eisinberg and G. Fedele

in all the applications in which equidistant measures must be used. In thispaper we focus our attention on discrete orthonormal polynomials. We de-rive an analytic expression for orthonormal polynomials on a discrete gridZn = {zr|zr = a + hr, r = 0, 1, ..., n− 1} and the coefficients of the “three-term recurrence relation” by using some results of the pseudoinverse on integernodes [4]. We show how to efficiently apply these formulas to the problem ofleast-squares fitting on equidistant nodes. Finally we investigate the problemto determine the effect of adding a new basis function or taking one away aspart of a process of trying to discover the optimal set of basis functions andthe problem to update the basis functions when a moving window over thetime series/data stream is used.

We want to emphasize that although high degree approximations on equidis-tant nodes have a bad reputation in numerical analysis, a lot of researches havebeen made for the design of accurate algorithms for polynomial approximationon such a set of nodes. There is a considerable interest on equidistant nodesbecause this choice frequently occurs in many applications.

2 Preliminary results

Let f1 and f2 be real valued functions defined on the set of nodes Xn ={x1, x2, ..., xn} and introduce the inner product:

〈f1(x), f2(x)〉 =

n∑k=1

f1(xk)f2(xk). (1)

A family P = {p1(x), p2(x), ..., pm(x)} with m ≤ n of polynomials is or-thogonal respect to this inner product if the following properties hold:

〈pk(x), pq(x)〉 = 0, k �= q; k, q = 1, 2, ..., m〈pk(x), pk(x)〉 = ξk �= 0, k = 1, 2, ..., m.

(2)

If ξk = 1, k = 1, 2, ..., m then polynomials pk are said orthonormal on theset of nodes Xn. A well known result from the general theory of orthogonalpolynomials is that the orthogonal polynomials satisfy a three term recursionformula:

pk(x) = (αkx + βk)pk−1(x) + γkpk−2(x), k = 3, 4, ..., m. (3)

Proposition 1 Let Xn be a set of n nodes and let P be a family of morthogonal polynomials on Xn, then

Discrete orthogonal polynomials on equidistant nodes 1009

pi(xj) = (−1)i+1pi(xn+1−j), j = 1, 2, ..., n; i = 1, 2, ..., m (4)

if and only if

αi(xj + xn+1−j) + 2βi = 0, i = 1, 2, ..., m; j = 1, 2, ..., n. (5)

Proof. The proof is made by induction on the polynomial index i. Suppose(5) is true for i− 1 and i− 2 then

pi−1(xj) = (−1)ipi−1(xn+1−j), j = 1, 2, ..., n (6)

and

pi−2(xj) = (−1)i+1pi−2(xn+1−j), j = 1, 2, ..., n (7)

• (→) Taking into account (3), the (4) becomes:

(αixj + βi)pi−1(xj) + γipi−2(xj) =

(−1)i+1 [(αixn+1−j + βi)pi−1(xn+1−j) + γipi−2(xn+1−j)](8)

By using (6) and (7) the (5) follows.

• (←) Taking into account (6) and (7), pi(xn+1−j) can be written as

pi(xn+1−j) = (−1)i(αixn+1−j + βi)pi−1(xj)+

+(−1)i+1pi−2(xj).(9)

Considered that (αixn+1−j + βi) = −(αixj + βi) then (4) follows.

Here we review a result useful in the sequel. Let Bk be the matrix definedas follows:

Bk(i, j) =n∑

p=1

pi+j−2, i, j = 1, 2, ..., k (10)


then its inverse can be factorized as [4]:

B−1k = WkΛ

−1k W T

k . (11)

In (11), Wk and Λ−1k have the following expressions:

Wk(i, j) =(j − 1)!(

2j−2j−1

) n∑s≥1

(−1)i+j

(s− 1)!

(s + j − 2

s− 1

)(n− s

n− j

) [s

i

], i, j = 1, 2, ..., k,

(12)

where[

ij

]are the Stirling numbers of the first kind [8], and

Λ−1k = diag

{ (2i−2i−1

)[(i− 1)!]2

(n+i−12i−1

)}

i=1,2,...,k

. (13)

3 Discrete orthogonal polynomials

Let P = {p1(x), p2(x), ..., pm(x)} be the set of polynomials defined as

pk(x) =k∑

i=1

ck,ixi−1, k = 1, 2, ..., m. (14)

The set P is orthogonal on the set of nodes Xn = {1, 2, ..., n} if the followingequalities hold:

〈pk(x), xq−1〉 = 0, k = 2, 3, ..., m; q = 1, 2, ..., k − 1, (15)

〈pk(x), xk−1〉 = ρk �= 0, k = 1, 2, ..., m. (16)

Let ck and bk (k = 1, 2, ..., m), be two vectors defined as follows:

ck(i) = ck,i, i = 1, 2, ..., k

bk(i) = ρkδk,i, i = 1, 2, ..., k(17)

then by (15) and (16) and taking into account (11), it must be


ck = WkΛ−1k W T

k bk. (18)

It is straightforward to note that:

W Tk bk = bk (19)

and

[Λ−1

k W Tk bk

]i= ρk

(2k−2k−1

)[(k − 1)!]2

(n+k−12k−1

)δk,i, i = 1, 2, ..., k. (20)

Then, by simple algebraic manipulations we have:

ck,i = μk(−1)i+kk∑

s=1

1

(s− 1)!

(s + k − 2

s− 1

)(n− s

n− k

) [s

i

], i = 1, 2, ..., k, (21)

where

μk =ρk

(k − 1)!(

n+k−12k−1

) , k = 1, 2, ..., m. (22)

By using the identity [8]:

∑i

(−1)i[s

i

]xi−1 = (−1)s(s− 1)!

(x− 1

s− 1

)(23)

polynomial pk(x) becomes:

pk(x) = μk

k∑s=1

(−1)s+k

(s + k − 2

s− 1

)(n− s

n− k

)(x− 1

s− 1

), k = 1, 2, ..., m. (24)

If we put

ρk = (k − 1)!

(n + k − 1

2k − 1

), k = 1, 2, ..., m (25)

then the family P of polynomials


pk(x) =k∑

s=1

(−1)s+k

(s + k − 2

s− 1

)(n− s

n− k

)(x− 1

s− 1

), k = 1, 2, ..., m. (26)

is orthogonal on the set Xn = {1, 2, ..., n}. By an affine transformation on thevariable x in (26) we have that the polynomials

pk(x) =k∑

s=1

(−1)s+k

(s + k − 2

s− 1

)(n− s

n− k

)( x−ah

s− 1

), k = 1, 2, ..., m. (27)

are orthogonal on the set

Zn = {zr|zr = a + hr, r = 0, 1, ..., n− 1} (28)

Such polynomials satisfy the following three-term recurrence relation:

pk(x) = (αkx + βk)pk−1(x) + γkpk−2(x), k = 3, 4, ..., m (29)

where

⎧⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎩

αk = 4k−6h(k−1)2

,

βk = − [(2k−3)(2a+h(n−1))]h(k−1)2

,

γk = (k−2)2−n2

(k−1)2, k = 3, 4, ..., m

(30)

Proposition 2 The set of polynomials Q = {q1(x), q2(x), ..., qm(x)} where

qk(x) =1√(

n+k−12k−1

)(2k−2k−1

)k∑

s=1

(−1)s+k

(s + k − 2

s− 1

)(n− s

n− k

)(x− 1

s− 1

), (31)

k = 1, 2, ..., m

is orthonormal on the set of nodes Xn = {1, 2, ..., n}.Proof. It is straightforward to note that

〈pk(x), pk(x)〉 = ck,kρk, (32)


then by using (21) and the expression (25) for ρk, we have

〈pk(x), pk(x)〉 =

(n + k − 1

2k − 1

)(2k − 2

k − 1

)(33)

and the (31) follows.

Corollary 3.1 The generic polynomial tk(x) in the family of orthonormalpolynomials T = {t1(x), t2(x), ..., tm(x)} on the set of nodes

Zn = {zr|zr = a + hr, r = 0, 1, ..., n− 1} (34)

can be written as

tk(x) =1√(

n+k−12k−1

)(2k−2k−1

)k∑

s=1

(−1)s+k

(s + k − 2

s− 1

)(n− s

n− k

)(x−a

h

s− 1

), (35)

k = 1, 2, ..., m

The tk satisfies a three-term recurrence relation

t1(x) = 1√n

t2(x) =√

3

h√

n(n2−1)[2x− h(n− 1)− 2a]

tk(x) = (αkx + βk)tk−1(x) + γktk−2(x) k = 3, 4, ..., n

where

αk = 2h(k−1)

√(2k−1)(2k−3)n2−(k−1)2

βk = −12[2a + h(n− 1)]αk

γk = − αk

αk−1k = 3, 4, ..., n

(36)

Such explicit formulas are particularly useful because they allow to computeefficiently the orthonormal polynomials without using the Stieltjes procedurewhich can be sensitive to roundoff errors [13].


4 Least squares application

In this section we consider the problem of least squares on a discrete set

Zn = {zr|zr = a + hr, r = 0, 1, ..., n− 1} (37)

by using polynomials pk defined in (27) which are orthogonal on Zn. Suchpolynomials satisfies the three-term recurrence relation (29) with parameters(30). Note that 〈pk(x), pk(x)〉 is invariant under affine transformation, there-fore its value can be computed by using the (33). Here we want to expand agiven function y = f(x) in terms of orthogonal polynomials [9, 12]:

f(x) =

m∑i=1

wiqi(x). (38)

and to determine the coefficients w1, w2, ..., wm in (38) such that the Euclideannorm of the error function f − f is minimized,

||f − f || =n−1∑j=0

|f(zj)− f(zj)|2.

Since q1(x), q2(x), ..., qm(x) form an orthogonal system, the coefficients arecomputed more simple by

wj =〈qj(x), f(x)〉〈qj(x), qj(x)〉 , j = 1, 2, ..., m. (39)

If we define the following quantities:

b = H · f (40)

where

H(i, j) = qi(zj), i = 1, 2, ..., m; j = 0, 1, ..., n− 1, (41)

f = [f(z0), f(z1), ..., f(zn−1)] (42)

and


g(i) =

(2i− 2

i− 1

)(n + i− 1

2i− 1

), i = 1, 2, ..., m (43)

then

wi =b(i)

g(i), i = 1, 2, ..., m. (44)

Since the product b = H · f is also invariant under affine transformation,then it would be computed by considering, for example, the set of orthogonalpolynomials on [1, 2, ..., n] for which the matrix H is

H(i, j) =i∑

s=1

(−1)s+i

(s + i− 2

s− 1

)(n− s

n− i

)(j − 1

s− 1

), (45)

i = 1, 2, ..., m; j = 1, 2, ..., n.

The pseudo-code in Table 1 finds the coefficients wi, i = 1, 2, ..., m in (38).We compare our algorithm (EF) applied to the orthogonal polynomials on

the set of n equidistant nodes in [0, 1] with that proposed in [2] (CB) whichcosts 10mn flops and with the Mathematica built-in function Fit (MA) [15].The first two algorithms have been implemented in Mathematica package,which allows arbitrary precision numbers. Our algorithm costs 3.5mn flops.Infact we build an half part of the matrix H by the three-term recurrencerelation, and the product H · f by using the property in Proposition 1.

For some values of m with n = 1000, we have generated one thousandvectors f , with entries uniformly distributed in [−1, 1], and have computedthe exact solution of the problem (38)

f(x) =

m∑i=1

hiqi(x) =

m∑k=1

rkxk−1 (46)

using extended precision. For each algorithm we have computed the maximumcomponentwise relative errors

EEF = max1≤i≤m

|rEFi − ri||ri| (47)

ECB = max1≤i≤m

|rCBi − ri||ri| (48)


Table 1: Pseudo-code for least-square problem by discrete orthogonal polyno-mials.function lmsOrt

Input: a, h, m, fOutput: wi, i = 1, 2, ...,m

Calculate the quantities αk, βk and γk in (30).Calculate gk in (43).

Build the matrix CP of the coefficients of the polynomials pk

by using the three-term recurrence relation (29).

Build the elements i = 1, 2, ...,m, j = 1, 2, ..., n2 of the matrix H .

Build the two vectors f1 ={f(i), i = 1, 2, ..., n

2 }

and f2 ={f(n + 1− i), i = 1, 2, ..., n

2 }.

Build the vector b =

{b(i) =

�n2 �∑

j=1

H(i, j)(f1(j) + (−1)i+1f2(j))

}.

if n is odd thench = 1b(1) = b(1) + f((n + 1)/2)for i = 3 : 2 : m

ch = γ(i)ch; b(i) = b(i) + chf((n + 1)/2)end


EMA = max1≤i≤m

|rMAi − ri||ri| (49)

where rEF , rCB and rMA are the approximate solutions computed by EF, CBand MA algorithms respectively in machine precision.

The mean and the maximum of EEF , ECB and EMA over one thousand runshave then been computed and reported in Table 2. Table 2 reports also thefraction of trials in which the proposed algorithm gives equal or more accurateresult than CB and MA algorithm. Note that the matrix H in (45) has integerentries that can be stored without rounding errors and this is probably thecause of the robustness of this algorithm.

5 Some properties

A common problem in least-square problems is to determine the effect ofadding a new basis function, that is to determine the effect of using m + 1basis in (38). This is important as part of a process of trying to discoverthe optimal set of basis functions. The effect can, of course, be calculated byrestarting the algorithm in the previous section. However in our case, simpleanalytical formulas can be derived to cover this situation.

Let CPm be the matrix of the coefficients of the polynomial pk(x), k =1, 2, ..., m in (27), that is

CP m =

⎡⎢⎢⎢⎢⎢⎣

c1,1 0 0 ... 0c2,1 c2,2 0 ... 0c3,1 c3,2 c3,3 ... 0...

...... ...

...cm,1 cm,2 cm,3 ... cm,m

⎤⎥⎥⎥⎥⎥⎦ =

[CP m−1 0

wTm cm,m

](50)

and let Gm be the matrix of the gk, k = 1, 2, ..., m in (43):

Gm =

⎡⎢⎢⎢⎣

g1 0 ... 00 g2 ... 0...

... ... 00 0 ... gm

⎤⎥⎥⎥⎦ =

[Gm−1 0

0T gm

](51)

If we consider the matrix H in (41) as

Hm =

[Hm−1

hTm

](52)


then the coefficients of the approximant polynomial can be expressed in matrixform as

CPTmGmHmf =

[CP

Tm−1Gm−1Hm−1f + gm(hT

mf)wm

cm,mgm(hTmf)

]. (53)

Formula (53) is useful for the updating of the degree of the approximantpolynomial in least square sense since it gives the coefficients of the approxi-mant polynomial of degree m − 1 in terms of the approximant polynomial ofdegree m− 2.

In real-time applications, which process a stream of continuously arrivingdata, it is often required a processing in moving time windows. More specific,let Z1 and Z2 be two discrete sets of nodes such as

Z1 = {a + h · r, r = 0, 1, ..., n− 1} , (54)

Z2 = {a + p + h · r, r = 0, 1, ..., n− 1, p > 0} (55)

and CP1 and CP2 the matrix of the coefficients of the orthogonal polynomialon Z1 and Z2 respectively, then

CP2 = CP1Mp (56)

where

M(i, j) = (−1)i+j

(i− 1

j − 1

), i, j = 1, 2, ..., m. (57)

Proposition 3 The p-th power of the matrix M is

Mp(i, j) = (−1)i+j

(i− 1

j − 1

)pi−j, i, j = 1, 2, ..., m. (58)

The matrix Mp has the following useful recursive properties:

Mp(i, i) = 1, i = 1, 2, ..., m,

Mp(i, 1) = −pMp(i− 1, 1), i = 2, 3, ..., m,

Mp(i, j) = −pMp(i− 1, j) + Mp(i− 1, j − 1), i = 3, 4, ..., m, j = 2, 3, ..., i− 1.

(59)


m CB MA EFmax mean max mean max mean EF vs CB EF vs MA

2 9.53-13 6.15-15 1.92-13 4.30-15 4.19-13 3.26-15 0.85 0.753 1.17-09 1.82-12 4.46-11 9.57-14 4.62-12 8.06-15 1.00 0.984 9.57-11 1.06-12 2.22-11 3.16-13 6.94-13 6.53-15 1.00 0.995 5.56-10 2.43-12 4.60-10 1.61-12 3.76-13 5.64-15 1.00 1.006 2.97-10 2.79-12 1.14-09 9.37-12 1.80-12 1.37-14 1.00 1.007 2.70-08 8.85-11 4.56-09 3.29-11 4.70-12 1.66-14 1.00 1.008 4.05-07 1.61-09 8.20-08 2.48-10 1.41-11 4.95-14 1.00 1.009 3.13-07 5.85-10 5.72-07 1.80-09 4.97-12 2.55-14 1.00 1.0010 9.89-07 1.14-08 5.99-07 5.82-09 1.16-11 2.79-14 1.00 1.0020 5.34-01 5.75-03 2.02+03 2.01+01 1.08-11 1.34-13 1.00 1.0030 3.27+07 2.08+05 1.14+03 5.49+00 4.51-11 2.89-13 1.00 1.0040 4.06+60 8.45+57 3.75+02 3.11+00 2.76-10 7.88-13 1.00 1.0050 4.51+173 3.49+171 5.10+03 8.74+00 5.82-11 4.28-13 1.00 1.00

Table 2: Maximum and mean value of ECB, EMA and EEF over 1000 runs,n = 1000, f ∈ [−1, 1]. Success rate of EF algorithm.

6 Conclusions

In this paper we have given an alternative proof of the explicit formula for thediscrete orthogonal polynomials on equidistant nodes. Such a formula, with itsproperties, allows us to design an efficient algorithm for solving least squaresproblem. The numerical experiments indicate that our approach is more stablecompared with existing Conte-de Boor algorithm. We finally have shown someproperties satisfied by these polynomials.

References

[1] A. Bjorck, Numerical Methods for Least Squares Problems, SIAM, 1996.

[2] S.D. Conte and C. De Boor, Elementary Numerical Analysis, McGraw-Hill,Second. ed., 1972.

[3] G. Dahlquist and A. Bjorck, Numerical Methods, Prentice Hall, EnglewoodCliffs, 1974.

[4] A. Eisinberg, P. Pugliese and N. Salerno, Vandermonde matrices on integernodes: the rectangular case. Numer. Math., 87 (2001), 663-674.

[5] F.B. Hildebrand, Introduction to Numerical Analysis, New York: Mc-Graw-Hill, 1956.

[6] C. Jordan, Calculus Of Finite Differences, Chelsea Publ. Comp., New York,1965.

[7] S. Karlin and J. McGregor, Linear growth models with many types andmultidimensional Hahn polynomials, in Theory and applications of specialfunctions, 261?288, ed. R. A. Askey, Academic Press, New York, 1975.


[8] D.E. Knuth, The Art of Computer Programming,(2nd ed.), Addison-Wesley, Reading Mass., 1981.

[9] B.M. Mohan and B. Srinath, On the identification of discrete-time systemsvia discrete orthogonal functions, Computers Elec. Engng., 23 (1997), 329-345.

[10] R. Mukundan, Image Analysis by Tchebichef Moments, IEEE Trans. Im-age Proc., vol.10 no.9 (2001), 1357-1364.

[11] A.F. Nikiforov, S.K. Suslov, V.B. Uvarov, Classical orthogonal polynomi-als of a discrete variable, Springer-Verlag, Berlin, 1991.

[12] M. Okabe, Discrete orthogonal polynomials inherent to sampling pointsat equal intervals, Comput. Methods Appl. Mech. Engng., 123 (1995), 371-383.

[13] L. Reichel, Fast QR Decomposition of Vandermonde-Like Matrices andPolynomial Least Squares Approximation, SIAM J. Matrix Anal. Appl., 3(1991), 552-564.

[14] G. Szego, Orthogonal Polynomials, 4th ed., Amer. Math. Soc., Provi-dence, RI, 1975.

[15] S. Wolfram, Mathematica: a System for Doing Mathematics by Comput-ers, Second. ed., Addison-Wesley, 1991.

Received: August 10, 2006

Documents

Discrete orthogonal polynomials on equidistant nodes