SOME WAYS OF DETERMINING COMPUTATIONAL ERRORS IN CONSTRUCTING

REGRESSION LINES

V. P. Borovikov and N. L. Novozhilov UDC 53.088:681.3

Here we give methods of determining the computational errors arising on plotting re- gression lines by computer.

We consider a physics experiment providing a set of inputs X = (X I ..... Xn), X i = (xil .... , Xir), 1 <_ i _< n and a set of responses Y = (YI, "'', Yn)" Many such experiments are described by a typical regression equation

Yt= ~ blXilq-~i=(b}g)i-I-ai; l~;i<n, r<n, ( 1 ) I ~l-,5.r

i n w h i c h e i i s t h e random e r r o r w i t h w h i c h a r e s p o n s e i s o b s e r v e d i n e x p e r i m e n t i , X = { x i j } is the regression matrix, xij is the value j of the regressor in experiment i, and b (bl,

.... br).

The classical task is to use observations on the inputs XI, .... X n and corresponding responses YI, ''', Yn to estimate the unknown parameters bl, ..., b n in (i). The basic method is least-squares fitting, which provides estimators that subject to fairly general assumptions [I] have optimal features, which distinguishes them from other possible esti- mators.

Least-squares fitting routines exist in virtually all systems now used to process measurements. Such modules produce estimators having methodological and computational

errors. The output is bj J J = 6~ + 6~, j = 1 ..... r, in which 5j is the exact least-squares estimator, which includes the methodological error, while 6j is the computational error, which can be substantial, e.g., in a high-degree polynomial model.

The (i) b parameters usually have definite physical meanings, and in many experiments, constructing the most accurate estimates for them is a basic task. It is methodologically important to establish whether a model produces a substantial computation error from the input data, i.e., if one wishes to increase the accuracy, i.e., to reduce the variance in the error e i in (i), which may not lead to a substantial improvement in the model parameter estimators because these show a bias as a result of the computations, and reducing that bias is required to improve them. The computational error in a least-squares estimator is gov- erned by two types of factors: firstly, the input data, i.e., the X, Y set, and secondly the module structure, together with the computer characteristics (number of bits, rounding rule, etc.). The experimenter cannot control the factors in the second group, but the xij can usually be managed. One naturally chooses the xij in such a way that the computational error should not exceed some reasonable limit.

We now give some methods of estimating the computational errors for least-squares routines. Experience with them has shown that these methods give reliable information on that error (bias) for the b parameters, as the basis of the approach is very simple: the least-squares estimators are interpreted as coefficients in the expansion of the orthogonal projection of the Y = (YI, ''', Yn) vector on a hyperplane based on the regression vectors [i]. By 5 we denote the exact least-squares estimator and consider

~=~X, ( 2 )

w h e r e one c a n s a y t h a t t h e v e c t o r Y-~ i s o r t h o g o n a l t o t h e h y p e r p l a n e L(X 1, ' ' ' , Xr ) c o n - s t r u c t e d on the regressors Xj = (xij)i, i.e.,

(Y~-~i)x~i=0, i=1 . . . . . ~ . (3)

Now we replace b in (2) by the estimator b, which is derived from a module, and corre- spondingly in place of Y we have Y, and (3) is replaced by

Translated from Izmeritel'naya Tekhnika, No. 9, pp. 6-7, September, 1989.

0543-1972/89/3209-0841512.50 �9 1990 Plenum Publishing Corporation 841

~.~ ( F , - - ~ i ) x l i = a i , i = l . . . . . r. l~ i~ . (4)

Equation (4) can be interpreted qualitatively as the extent to which the vector Y-Y is orthogonal to the hyperplane L.

We subtract (4) from (3) to get

E 6 Y t x t l = a / , j = l . . . . , r , (5)

in which 6Y i = Yi-Yi, i = i, ..., n is the computational error on determining ~i" We re- write (5) from (2) as equations for the computational error in estimating the bj:

x~ (6) 6ic ik=ak , l ~ j ~r

in which Cjk l~i ....

Equation (6) gives the following estimator for the computation error:

maxl6il> max lahl

l<i<r

A l t h o u g h t h e s e a r g u m e n t s a r e s i m p l e , t h e y a r e u s e f u l f o r l e a s t - s q u a r e s s u b r o u t i n e s . We consider experiments with the RLFOR program from the IMSL scientific subroutines library for personal computers, which gives least-squares estimators for linear regressions models. We

~-i, , . , , . , consider polynomial re~ression models, in which xij = x j = 1 .. r, i = 1 .. n,

with the x i taken in the range [-i, I]. The degrees in the polynomials vary from 6 to 9.

We adopted the following rules in examining the rs we first examined the o = (el, ..., ar) vector governing the orthogonality of Y-Y to L(X I .... , Xr) , and when all the aj were i0 -z or less, the computational error in the b was i0 -~ or less. When there was one element or more in the u vector of order 1 or more, i.e., one cannot say that Y-Y was ortho- gonal to L, one can certainly say that the error (computational bias) in the b will be sub- s~antial. For example, one might estimate the coefficients in brxr-1 + ... + bx + b I meas- ured with errors e i at point i uniformly distributed in [-i, i] on a grid with a step 0.2. The following are some results: for r = 7, max aj = 0.003, and the maximum computational error was 6j = 0.0018, while for r = 8 we have correspondingly max aj = 23.4, max 6j = 2.1, and for r = 9, max aj = 23.59, max 6j = 1.951. I i

i I If there are large elements in a, we derive the u of (7); ~ was of the order of max6j.

i

e.g., for the above polynomials we found 7 as 0.0007, 1.3, and 0.99, while the maximal errors

in the coefficients were 0.00018, 2.1, 1.95.

Graphs were formulated for the maximal computational error, which showed that the (7) estimator is not a crude one, so if the computational accuracy is important, one should choose the regressor values to minimize 7. These experiments show that the dependence of the maximal error on y in the 70max6! plane is closely represented, by a straight line, so

I

one can use that subroutine with a few values of y and the corresponding max~ i to determine !

the maximal computational error for any data set.

The (7) estimator is a lower bound to the maximum computational bias, and it does not imply that the bias will occur in all the coefficients. We have found that in essence that error varies greatly, with significant errors occurring in only certain of the minor coeffi- cients. The error distribution over the coefficients is important. In an orthogonal plan, where (Xjl, Xj2) = 0, Jl ~ J2, as for example when one uses the trigonometric functions cos. (2njx) or sin (2~jx) as basis, geometrical considerations imply that the 6j ire ordered in the cos ~J sequence,_in which cos~ is the cosine of the angle formed by Y-Y with regressor

j: cos ~ = aj/(JY-YJlXjJ), in which the Euclidean norm is taken.

If the regressors are not orthogonal, this is not so. In tests with RLFOR on polynomial coefficients, the coefficient group where the error was maximal corresponded to the aj group differing most markedly from zero. Experiments with regression programs have shown that the coefficient estimators least stable under grid changes show the largest computational errors.

842

This approach and the formulas thus enable one to define estimators for the computa- tional errors in regression lines derived by means of least-squares programs. Such estima- tors enable one to decide whether it is desirable to increase the accuracy in the measure- ments or to use some software.

i.

LITERATURE CITED

A. A. Borovkov, Mathematical Statistics: Additional Chapters [in Russian], Nauka, Moscow (1987).

APPLICATION OF SPECIAL WEIGHT FUNCTIONS TO INPUT SIGNAL

RECOVERY IN INSTRUMENTAL TRANSDUCERS OF MECHANICAL PARAMETERS

A. P. Golikova UDC 53.083.72:53.087. 92[048.3]

Considerable dynamic errors caused by signal distortion in real instrumental transdu- cers (IT) affect the accuracy of parameter measurement in fast mechanical processes. In practical measurements of such processes (e.g., pulse pressure, impact loading) it is im- portant to know the maximum (peak) value of the measured parameter. In this paper we con- sider how the dynamic measurement error of such a parameter can be reduced by the method of input signal recovery. Let us solve the convolution equation with an approximately known right side which describes the operation of linear ITs:

t

0

where x(t) is the input signal, h(t) is the pulse response of the IT, y(t) is the measuring system (MS) output signal assumed to contain no random error, and D(t) is a random error of uniform spectral density and a standard deviation oq.

The general theory of input signal recovery has been developed in [I]. Subsequent in- vestigations consisted essentially in "adapting" the recovery algorithm to specific measure- ment problems. The existing methods have been reviewed in [2].

By taking into account the measurement specifics (e.g., error information [3], the MS accuracy class [4], etc.) it is possible to concretize the method and to improve the accur- acy of input signal recovery.

Let us limit the class of linear ITs to second-order dynamic sections whose normalized

frequency response is given by

H~(J~)=[l--(~/~o)~+J2~/~o] -1, (1)

where ~0 is the IT natural frequency, and ~ is a damping factor.

All input signals considered are processes whose spectral density, beginning at a cer- tain frequency, is practically zero. The output signal 9(t) then has low spectral density

at high frequencies.

The stabilization factor of the Tikhonov regularization method [i] can then be formed taken into account a priori information concerning the IT frequency response [5] and not the input signal and error spectral densities as usual. The recovered input signal is given by

~i)= F - l {~j~)fCo,a)/H(i~)} , (2)

where H(jw) = F{h(t)} is the complex frequency response, Y(jm) = F{9(t)} is the Fourier transform of the output signal, and F and F -z are symbols of the Fourier transform and its

inverse.

In (2) the stabilizing factor was a spectral "window" of the special form:

Translated from Izmeritel'naya Tekhnika, No. 9, pp. 8-10, September, 1989.

0543-1972/89/3209-0843512.50 �9 1990 Plenum Publishing Corporation 843