Three-dimensional space from optical flow correspondence

COMPUTER VISION, GRAPHICS, AND IMAGE PROCESSING 42, 306-317 (1988)

Three-Dimensional Space from Optical Flow Correspondence*

AMAR MITICHE

INRS-Tdldcommunications, 3, Place du Commerce, Ile-des-Soeurs, P. Q., H3E 11t6, Canada

Received May 4, 1986; accepted November 5, 1987

This study considers the use of two distinct images of a number of points at which optical flow is known, to recover the relative displacement of the viewing systems and the position and motion of the points in space. The viewing systems are assumed immobile and the points rigidly moving. The problem considered here can therefore be seen as a generalization of the structure-from-motion problem as it considers both the discrete case of point correspondences and the continuous case of optical flow. A linear method is proposed based on the observation of four (or more) points. The linear system of equations resulting from this observation is solved for the relative displacement of the viewing systems. Once this displacement is calculated, the position and motion of the points in space are given straightforwardly. �9 1988 Academic Press, Inc.

1. INTRODUCTION

Motion analysis has quickly become a subject of intense research in computer vision because of the many attractive theoretical and computational problems it offers and the ever growing number of potential applications [1]. Among the most popular problems in motion analysis is that of relating quantitatively events in space to the changes they cause in images resulting from their observation. In practice, these changes are those measured on significant image observables. Observables that have been used include points, lines, contours, optical flow, and range. Except for the use of optical flow, the c o m m o n approach has been to utilize a time-sequence of images or a set of images taken from different viewpoints. The advantage in using several distinct images has been largely demonstrated in this context [2]. But, although several studies have been devoted to the determination of structure and motion from optical flow [3-17], the use of several distinct optical flow images has been neglected. Exceptions, though, are the studies reported in [13-17]. Studies in [13, 14] consider the unification of the functions of stereoscopy and motion. A method was described in [13] by which optical flow was integrated to the stereo- scopic mechanism to achieve perception of motion and structure in space. In [14] the stereo and motion analyses were performed by two different modules and a third module was proposed to unify the two analyses and overcome some of their shortcomings. The study in [15] considered the problem of segregating an image on the basis of optical flow into areas representing differently moving objects in space. The process that has been proposed used orthographic projection to model imaging and involved first establishing correspondence between a large number of feature points in two distinct images, and then using a Hough space scheme to search for the unknowns of the problem. In [16] a small number of orthogonally projected

*This research was supported in part by the Natural Science and Engineering Research Council of Canada under Grant A6022 and Grant A4234.

306

0734-189X/88 $3.00 Copyright �9 1988 by Academic Press, Inc. All fights of reproduction in any form reserved.

3D SPACE FROM OPTICAL FLOW 307

points were observed at two instants of time. Structure and motion were then recovered by combining the optical flow equations at each instant of time with the equations of invariance of distance between the two instants of time. [17] considers the continuous formulation of the incremental rigidity scheme.

This study considers optical flow as observable. It is assumed that the observed motion is that of a single object. The specific problem addressed is the following: Given two distinct views taken by immobile viewing systems from unknown viewpoints, of a number of rigidly moving points at which optical flow is known, determine the relative displacement between the viewpoints, the position of the points in space, and their motion. At this point we stress the fact that the object is moving and the viewing systems are immobile. The problem considered here can therefore be seen as a generalization of the structure-from-motion problem [18-22] as it considers both the discrete case of point correspondences and the continuous case of optical flow. The problem can also be seen as a generalization of the problem of stereo motion described in [13] as the relative displacement between the views is here unknown.

We assume that optical flow is given to us. Although the problem of computing optical flow is not addressed in this paper, we would like to point out that it is a difficult problem for which a satisfactory answer has yet to be proposed. Methods that have been proposed had mixed success [23-30]. Obtaining accurate estimates of optical flow can be crucial for its accurate interpretation, especially by those schemes which have to cope with the availability of only a small number of points in a small number of views. Nevertheless, optical flow interpretation is important and interesting in its own right because it is (1) a primary cue to depth and motion of objects, (2) it allows the segregation of space into differently moving objects, and (3) it allows the prediction of placement of objects.

Given our problem and its related assumptions, we propose an answer based on the observation of four (or more) points for which a linear system of equations is written and solved for the parameters of the relative displacement of the viewpoints. The method is an extension to the linear method used in the context of point correspondences [20222]. Once this relative displacement is known, the position Of the points in space is obtained by simple triangulation. Recovering the motion of the points also becomes a simple matter when their position in space is known.

The remainder of this paper is organized as follows: Section 2 gives the mathematical details of the method to recover the various unknowns, Section 3 contains an experimental simulation, and Section 4 is a summary.

2. STRUCTURE AND MOTION FROM OPTICAL FLOW CORRESPONDENCE

The configuration of viewing systems is shown in Fig. 1. These will be referred to as the first viewing system and the second viewing system. Each is represented by a central projection model. These models are designated by cartesian coordinate systems S 1 and S 2 with origins 01 and 0 2 (which are also the projection centers) and image planes I 1 and 12, respectively. We assume, for convenience and without loss of generality, that the focal lengths are equal to one unit of measurement.

Let P be a point in space, with coordinates (X1, Y1, Zt) in S 1 and (X2, Y2, Z2).in S 2. Let the image of P be Pa on I x with coordinates (xl, Yl) and P2 on 12 with coordinates (x2, Y2). Finally, let the transform taking S a onto $2 be decompc;sed into a rotation R about an axis through origin O1, and a translation T. The rotation

308 AMAR MITICHE

Yj

O I

x'xYl ~ Pl

~ ~'1 / J / /

I I ~ 0 2 Z2

Z2

FIG. 1. Viewing system configuration

O~ ~t X 2

is described by the matrix

rl r2 r3 ) R = r4 r5 r6

r7 r8 r9

and the translation by the vector T = (tl, t2, t3).

2.1. Relative Displacement of the Viewpoints

We will proceed in two steps: First we write equations which exploit image position information and then we write those which exploit optical flow information. With the notations above we can write

(x2, Y2, z2)' = R. (Xx, Ya, z l ) ' + r (1)

or, in expanded form

Since

X 2 = r lX 1 + r2Y 1 + raZ 1 + t 1

Y2 = r4X1 + rsY1 + r6Zl + t2

Z 2= rTX 1 + rsY 1 + r9Z 1 + t 3.

(2)

x2= Z2 Y2= Z2' (3),(4)

we have

r l X 1+ r2Y 1+ r3Z 1 + t I (s) x 2-- r T X l + r s y l + r g Z l + t 3"


Dividing the numerator and denominator of the right-hand side by Z1, we obtain

f i x I + r2y x + r 3 + t l / Z 1 x2 = (6)

r7x 1 + rsYl + r 9 + t 3 / Z a

Similarly for Y2,

r4x 1 + rsy 1 + r 6 + t 2 / Z 1

Y2 = r 7 x l + rsYx + r9 q_ t a / Z 1 (7)

We pull Z 1 (or 1//Z1) out of (6) and (7) and obtain

Z 1 t 1 - - t 3 x 2

( r7x 1 + rsy 1 + r9 )x 2 - r l x x - r2y 1 - r 3

t 2 - t 3y z

(r7x 1 q- rsy x + r9)y 2 -- r4x 1 -- r sy 1 -- r6" (8)

The equation above has been used in [20-22] in the context of computing object motion from image point correspondences. In the following we will supplement it with another linear equation obtained by exploiting optical flow information�9 Equation (8) can be written as a linear expression:

X l X 2 a 1 + x 2 Y l a 2 -b x 2 a 3 + x l Y 2 a 4 q- y l y 2 a 5 + y2a6 -b x l a 7 q- y l a 8 + a 9 = 0, (9)

where the variables constitute the elements of matrix A:

A = al a2 a3

a4 a5 a6

a7 a8 a9

t3r4 - t2r 7 t3r 5 - t2r 8

= t l r 7 - tar 1 t i t 8 - t3r 2

I t2rl tlr4 t2r2 tit5

t3r6 - t2r9 1 I

tlr9 t3r 3 ] .

t2r3 &r6 ]

(10)

Equation (9) can be written in condensed matrix form:

(x2 Y2 1)A = 0.

Now we proceed to exploit optical flow information. Noting that Eq. (9) is valid at each instant of time and considering the fact that the viewing systems are stationary (matrix A is independent of time), we can derivate with respect to time and obtain

( u l x z + u 2 x x ) a 1 + (u2Y 1 -4- X2Vl )a 2 + u2a 3 + (Uly 2 q- XlO2)a 4

+(01y 2 + V z y l ) a 5 + v2a 6 + u la 7 q- o l a 8 = O, (11)

where (ul, vl) and (u2,/32) are the optical flow values at image points Pl and P2,

310 AMAR MITICHE

respectively. In condensed matrix form (11) can be written as

(x:) (no:) ( < o)A + 1)A = o.

The uncertainty of scale appears in Eq. (9) and Eq. (11) which still hold when the variables are multiplied by a scalar. Scale is fixed arbitrarily by fixing the value of one of the variables. For notational convenience we fix a 9 which is set to 1. We are then left with eight unknowns a 1 . . . . . a 8. Considering that each point contributes two equations ((9) and (11)), then one can attempt to solve for matrix A by observing k > 4 points. This will yield the following system of linear equations:

M

a l I

( / -~1

aal

aal

ar

a~l a~l

a ~ l

01" 01 01 01

(12)

For k = 4 matrix M is, the superscripts being point indicators,

M r

xlx 1 x l y I x~ xl)'~ y~y~ y~ x{ y]

4 4 x4y ~ x 4 x4y~ y4y4 y2 4 x~ y~ X1 X2 1 1 1 1 1 1 g l 1 1 1 1 1 1 1 1

1 1 q_ U2X1 _}. Vly 2 v2y 1 ulYi + + va2 u{ vll UlX2 U2Yl X2U1 N1U2

, , ,4 4 y 4 + ~ 4 v4y~+4y? ~4 4 4 uax~ + uax~ u~yl 4 + x2v 1

If Det (M) ~ 0 then the elements of A are determined uniquely. Once this matrix is computed, one can recover the components R and T of the relative displacement of the viewpoints the elements of which are related to those of A by (10). To compute translation, an examination of (10) reveals that

t l a 1 + t2a 4 + t3a 7 = 0

t ta 2 + t za 5 + t3a 8 = 0 (13)

t l a 3 + t2a 6 + t3a 9 = O.

Or, in matrix form,

A t T = O.


Since rank (A) = 2, we have two homogenous linear equations which can be solved for (the direction of) translation.

Finally, we recover the rotation matrix R. There are two cases: either T 4= 0 or T = 0 .

I f T = 0 then R cannot be obtained from (10) for an obvious reason. Then one has to go back to the original equations (6) and (7) which become linear in the rotat ion parameters. Writing the linear system of equations is straightforward.

For T ~ 0 it has been shown in [20] that the rotation components can be obtained f rom A by computing the singular value decomposition of A hence avoiding the nonlinear equations appearing in A. Indeed, if A has the singular value decomposition

then

A = USV' , (14)

R = UDV t, (15)

where D is either D 1 o r D2;

_ 1 D I = 0 ,

0 0 2 =

01 i) - 1 0 , 0 0

where d = Det(U)Det(V) . For more details refer to [20]. Only one of the two matrices above will yield positive Z-values in which we are interested (see Section 2.2). F rom matrix R one can finally obtain the parameters 0, n a, n 2, n 3, where the rotat ion is through angle 0 about an axis through the origin with direction cosines nl, n2, n 3 [20, 30]. In [30], for instance,

rl + rs + r g - 1 COS 0 =

2

~ r 1 - cos O

nl = 1 - cos 0

r2+r4

n2 = 2n1(1 - cos0) (16)

r3 + r 7

n3 = 2n1(1 - cos 0)

r 6 - r 8 sin 0 = - -

2nl

2.2. Recovering Position in Space

If T = 0 depth cannot be recovered from point correspondences (a pure rotation does not carry depth information as is clear from Eq. (6) and (7)). If T ~ 0 then depth is recovered from (8). The other two components of position are then computed from (3) and (4).

2.3. Recovering Object Motion

Since we are dealing with optical flow, we will then consider instantaneous motion. We can describe motion with respect to either of our coordinate systems.

312 AMAR MITICHE

Therefore, for convenience, we drop the subscripts which differentiated the two viewing systems. Then the velocity P ' of the motion of mobile object point P, relative to static S, is described in terms of rotational velocity f~ = (0:x, ~ 0:3) about an axis through the origin and a translational velocity L = (Ix, lz , 13) as P ' = L + f~ • OP . Differentiating the projective relations (3) with respect to time and after some algebraic manipulations we can obtain the expression of optical velocity (u, v) at (x, y):

11 - x l 3 U --- - - x y 0 : 1 + (1 + X2)0:2 --Y0:3 +

l 2 - y l 3 v = - ( 1 + y 2 ) 0 : 1 + xy0:2 + x0:3 +

(17)

When depth is known (Section 2.2) the two equations above are linear in the parameters of object motion (tot, 0:z, 0:3, lx, 12, 13). Since there are six unknowns, one can attempt to recover the motion from the observation of three or more points. Writing the linear system of equations is straightforward.

3. EXPERIMENTAL VERIFICATION AND DISCUSSION

The viewing systems model is in Fig. 1. The focal length is 1 cm, which is close to those in the range of common cameras. Points have been generated randomly at distances approximately 4 m from the projection planes. For points at such distances, the pixel size of the central projection model equivalent to a 1 cm focal length camera is approximately 10 -3 cm [31]. Table l a contains the image coordinates and depth (Z-value), with respect to the first viewing system, of all the points used in subsequent experiments. Note that image points are confined to a small area around the origin (0.25 cm 2) but are fairly separated from each other within this area. The corresponding optical flow values are given in Table lb. The displacement between the two viewing systems is such that the translation components are t I = - 1 m, t 2 = - 1 m, and t 3 = 1 m. The rotation is about the Y-axis through a - 1 5 ~ angle. The object motion that gave rise to the data in Table lb is such that (0:1, 0:2, 0:3) = (0.1,0.1,0.1) and (ll, 12, 13) = (1.0, 1.0, 1.0). This setup simulates rea-

TABLE la Image Coordinates and Depth (Z), in the First Viewing

of All the Data Points Used in the Experiment System,

x y Z

- 0.25 0.25 493.2 0.0 0.25 329.2

- 0.25 0.0 308.9 0.0 0.0 395.9 0.22 0.25 408.0

-0 .1 0.2 436.7 0.1 0.25 459.2 0.1 -0 .2 328.5

Note. The unit of measurement is the centimerer (cm).

3D SPACE FROM OPTICAL FLOW

TABLE lb Optical Flow Values, in the Image Plane of the First Viewing System, Corresponding

to the Image Points given in Table l a

313

U V

0.0900 -0.1360 0.0780 -0.1040 0.1103 -0.1220 0.1025 -0.0975 0.0760 -0.0794 0.0855 -0.1142 0.0755 -0.0921 0.1260 -0.0923

sonably a real case. This setup is also "clean" in the sense that it does not seem to exhibit pitfalls such as those associated with optical flow ambiguities indicated in [32]. This will allow us, therefore, to verify the mathematical derivations in the paper and observe the effect of perturbations of data. In actual situtations, several factors other than data perturbation will have an effect on results: field of view, displacement between the two viewing systems, spatial distribution of image points, degree of overdetermination (number of points used), etc.

As indicated earlier, the main thrust of this study is in developing the method to recover the relative displacement between the viewing systems. The computation of object position and object motion is actually a side effect of the computation of the relative displacement between the viewing systems. Indeed, knowing this displacement, one can recover object position and motion quite straightforwardly (Sections 2.2 and 2.3). Therefore, the following experiments simulate the algorithm by which the relative displacement of the viewing systems is computed.

Comparison of computed to actual parameters is performed the following way:

(a) For a direct evaluation of the computed direction of translation, the uncertainty of scale is resolved by setting the third component of translation (t3) to its actual value,

(b) The computed angle of rotation, 19, is compared directly to its actual value,

(c) The direction of the computed axis of rotation is compared to the direction of the actual axis of rotation by computing the angle between the two directions. This angle is designated by ~k-

Tables 2-4 describe the behaviour of the algorithm when the values from the two sources of data (image position and optical flow) are perturbated. We considered the case of four points (the minimal number of points required) and the case of eight points (as an example of an overdetermined system).

Tables 2a-c show that the algorithm responds well to perturbations on image positions. The performance with eight points is comparable to that with four points. Tables 3a-c indicate that the algorithm is more sensitive to perturbation of optical flow. Using eight points leads to slightly better results than using four points. When image positions as well as optical velocities are noisy, there is a sensible gain in accuracy when more points are used than the minimum four required.

314 A M A R M I T I C H E

T A B L E 2 a

E f f e c t o f n o i s y i m a g e pos i t ions . N o i s e is u n i f o r m l y d i s t r i b u t e d b e t w e e n - I p ixe l a n d + 1 p ixe l

P o i n t s tl /'2 0 t~

Actual - 100.0 - 100 ,0 - 15 ,0 0.0

4 - 98.8 - 102.5 - 14.9 2.7

8 - 100.9 - 102.8 - 15.4 2.2

T A B L E 2b

E f f e c t o f n o i s y i m a g e pos i t ions . N o i s e is u n i f o r m l y d i s t r i b u t e d b e t w e e n - 2 p ixe l s a n d + 2 p ixe l s

P o i n t s t 1 t 2 0

A c t u a l - i 0 0 . 0 - 100.0 - 15.0 0 .0

4 - 97 .4 - 105.0 - 14 .9 5.6

8 - 101.8 - 105.7 - 15.8 4.2

T A B L E 2c

E f f e c t o f n o i s y i m a g e pos i t ions . N o i s e is u n i f o r m l y d i s t r i b u t e d b e t w e e n - 5 p ixe ls a n d + 5 p ixe l s

P o i n t s t 1 t 2 0

A c t u a l - 100.0 - 100.0 - 15 .0 0 .0

4 - 92.5 - 112.5 - 1 4 . 8 15.5

8 - 104.3 - 115.4 - 17.3 9.4

As indicated earlier, there are several factors other than data perturbation which will influence results. We have run a number of cases by varying some of these factors: the position of the points in space, their motion, and the relative displacement between the viewing systems. The observations are:

(a) Results are less accurate when the observed points are farther from the image planes.

(b) Results are less accurate for smaller object images.

(c) The observation of an additional point will not bring significant information if its image is close to those already in use. This is due to the fact that points close to each other have similar optical velocities, especially in the presence of noise,

(d) Results are less accurate when the viewing systems are closer to each other,

(e) Results are comparable over a large range of object motions.

(f) The computation of rotation is more robust to noise than the computation of translation. This is probably due to the fact that the singular value decomposition of a matrix, on which the calculation of rotation is based here, is known to be numerically quite insensitive to matrix perturbations [33].

In general, results seem to indicate that accurate measurement of image variables (image positions and optical flow) is a prerequisite to accurate measurement of the related space variables (space positions and space motion). As mentioned earlier,

3 D S P A C E F R O M O P T I C A L F L O W 315

T A B L E 3 a

E f f e c t o f n o i s y o p t i c a l f low. N o i s e is z e r o - m e a n , u n i f o r m l y d i s t r i b u t e d i n a n i n t e r v a l 2% the a c t u a l v a l u e

P o i n t s t 1 t 2 0 ~b

A c t u a l - 100.0 - 100 .0 - 15.0 0 .0

4 - 95.5 - 99 .4 - 1410 7.5

8 - 96.6 - 94.9 - 14.6 0 .26

T A B L E 3b

E f f e c t o f n o i s y o p t i c a l f low. N o i s e is z e r o - m e a n , u n i f o r m l y d i s t r i b u t e d i n a n i n t e r v a l 4% t h e a c t u a l v a l u e

P o i n t s t 1 t 2 0 t~

A c t u a l - 100.0 - 100 .0 - 15.0 0 .0

4 - 90.9 - 98.7 - 13.3 16.0

8 - 93.6 - 90.6 - 14.2 0 .57

T A B L E 3c

E f f e c t o f n o i s y o p t i c a l f low. N o i s e is z e r o - m e a n , u n i f o r m l y d i s t r i b u t e d i n a n i n t e r v a l 10% t h e a c t u a l v a l u e

P o i n t s q t 2 0

A c t u a l - 100.0 - 100.0 - 15.0 0 .0

4 - 77.2 - 97.0 - 13.0 45.5

8 - 87.0 - 81.5 - 13.3 1.5

this is especially true for schemes which have to deal with the availability of only a small number of points in a small number of views. But schemes which can cope with data scarcity are much needed because optical flow fields are generally sparse especially with images that exhibit little texture. Indeed, when images have little significant texture which allows the observation of motion, then one has to rely on a small number of "characteristic" points, such as corners or edges, which are consistently observable from one instant of time to the other and are invariant with respect to motion.

4. S U M M A R Y

This study has addressed the problem of using two distinct views taken from unknown viewpoints of a number of points for which optical flow was known, to recover the position of the points in space, their motion, and the relative displacement of the viewing systems. The viewing systems were assumed stationary and the object moving. A linear method was proposed which was based on the observation of four (or more) points. First, the relation between the position of corresponding points in the two images allowed us write a linear equ~ition in eight variables related to the parameters of the relative displacement of the viewing systems. Then the relation between the optical flow values at the corresponding points allowed us to write another linear equation in the same unknowns. The observation of four (or more) points therefore gave a linear system of equations the resolution of which

316 AMAR MITICHE

TABLE 4a Effect of noisy image positions and noisy optical flow. Noise on image positions is uniformly distributed betweerL :-1 pixel and + 1 pixel. Noise on optical flow is zero-mean, uniformly distributed in an interval 4~ the actual value

Points t 1 t 2 0 t~

Actual - 100.0 - 100.0 - 15.0 0.0 4 - 109.1 - 115.2 - 17.7 5.7 8 -107.1 -117.6 -16.4 2.2

TABLE 4b Effect of noisy image positions and noisy optical flow, Noise on image positions is uniformly distributed between - 2 pixets and + 2 pixels. Noise on optical flow is zero-mean, uniformly distributed in an interval 496 the actual value

Points t 1 t 2 0

Actual - 100.0 - 100.0 - 15.0 0.0 4 -107.9 -118.2 -16.9 3.9 8 - 107.9 - 115.6 - 16.7 3.6

TABLE 4c Effect of noisy image positions and noisy optical flow. Noise on image positions is uniformly distributed between - 2 pixels and + 2 pixels. Noise on optical flow is zero-mean, uniformly distributed in an interval 10$g the actual value

Points t 1 t 2 0 t~

Actual - 100.0 - 100.0 - 15.0 0.0 4 - 122.7 135.1 - 20.8 14.5 8 - 120.4 - 135.7 - 18.2 4~5

d e t e r m i n e d t h e r e l a t ive d i s p l a c e m e n t o f t he v i e w p o i n t s . O n c e th i s d i s p l a c e m e n t w a s

c o m p u t e d , t h e p o s i t i o n of t he p o i n t s in space , a n d the i r m o t i o n , w e r e g i v e n ~

s t r a i g h t f o r w a r d l y .

REFERENCES

1. J. K. Aggarwal, Motion and time-varying imagery, Compat. Graphics 18, No. 1, 1984, 20-21. 2. W. N. Martin and J. K. Aggarwal, Dynamic scene analysis--A survey, Comput. Graphics Image

Process. 7, 1978, 356-374. 3. D. H. Ballard and O. A. Kimball, Rigid body motion from optical flow and depth, Comput. Vision

Graphics Image Process. 22, 1983, 95-115. 4. B. Neumann, Optical flow, Comput. Graphics 18, No. 1, 1984, 17-19. 5. H. C. Longuet-Higgins and K. Prazdny, The interpretation of a moving retinal image, Prac. R. Sac.

London B 208, 1980, 385-397. 6. K. Prazdny, On the information in optical flows, Comput. Vision Graphics and Image Process. 22,

1983, 239-259. 7. K. Prazdny, Egomotion and relative depth map from optical flow, Biol. Cybernet. 36, 1980, 87-102.


8. A. Mitiche, On kineopsis and computation of structure and motion, IEEE Tram Pattern Anal. Mach. Intell. $, No. 1, 1986, 109-112.

9. Y. Yasumoto and G. Medioni, Experiments in estimation of 3D motion parameters from a sequence of image frames, in Proceedings Conf. on Computer Vision and Pattern Recognition, San Francisco, CA, 1985, pp. 89-94.

10. A. R. Bruss and B. K. P. Horn, Passive navigation, Comput. Vision Graphics Image Process. 21, 1983, 3-20.

11. X. Zhuang and R. M. Haralick, Rigid body motion and the optical flow image, in Proceedings, First Conf. on Artificial Intelligence Applications, Denver, CO, 1984, pp. 366-375.

12. G. Adiv, Determining three-dimensional motion and structure from optical flow generated from several moving objects, IEEE Trans. Pattern Anal. Mach. Intell. 7, No. 4, 1985.

13. A. Mitiche, On combining stereopsis and kineopsis for space perception, in Proceedings, IEEE First Conf. on Artificial Intelligence Applications, Denver, CO, 1984, pp. 156-160.

14. A. Waxman and J. Duncan, Binocular image flows, in Proceedings, IEEE Workshop on Motion: Representation and Analysis, Charleston, SC, May 1986, pp. 31-38.

15. H. Tsuktme and J. K. Aggarwal, Analysing orthographic projection of multiple 3D velocity vector fields in optical flow, in Proceedings, Vision and Pattern Recognition, San Francisco, CA, 1985, pp. 510-517.

16. K. Sugihara and N. Sugie, Recovery of rigid structure from orthographically projected optical flow, Comput. Vision Graphics Image Process. 27, 1984, 309-320.

17. E. Hildreth and N. Grzywacz, "The incremental recovery of structure from motion: position vs velocity based formulations," Proceedings, IEEE Workshop on Motion: Representation and Analysis, Charleston, SC, May 1986, pp. 137-143.

18. S. Ullman The interpretation of structure from motion, Proc. R. Soc. London B 203, 1979, 405-426. 19. J. W. Roach and J. K. Aggarwal, Determining the movement of object from a sequence of images,

IEEE Tram. Pattern Anal. Mach. Intell. 6, 1980, 554-562. 20. R. Tsai and T. Huang, Uniqueness and estimation of three-dimensional motion parameters of rigid

objects with curved surfaces, IEEE Tram. Pattern Anal. Mach. Intell. 6, No. 1, 1984, 13-27. 21. H. Longuet-Higgins, A computer algorithm for reconstructing a scene from two projections, Nature

293, 1981, 133-135. 22. X. Zhuang and R. Haralick, Two-view motion analysis, in Proceedings, Conf. on Computer Vision

and Pattern Recognition, San Francisco, CA, 1985, pp. 686-690. 23. C. L. Fennema and W. B. Thompson, Velocity determination in scenes containing several moving

objects, Comput. Graphics Image Process. 9, 1979, 301-315. 24. B. K. P. Horn and B. G. Schunk, Determining optical flow, Artif. Intell. 17, 1981, 185-203. 25. R. Paquin and E. Dubois, A spatio-temporal gradient method for estimating the displacement field in

time-varying imagery, Comput. Graphics Image Process. 1983, 205-221. 26. H. H. Nagel, Displacement vectors derived from second-order intensity variation in image sequences,

Comput. Vision Graphics Image Process. 21, 1983, 85-117. 27. W. B. Thompson and S. T. Barnard, Lower-level estimation and interpretation of visual motion,

Computer 14, 1981, 20-28. 28. K. Wohn, H. )r.Je, L. S. Davis, and A. Rosenfeld, Smoothing optical flow fields, in Proceedings,

Image Understanding Workshop, Washington, D C, 1983, pp. 61-63. 29. E. C. Hildreth and S. Ullman, The Measurement of Visual Motion, MIT A1 Memo 699, Dec. 1982. 30. A. Mitiche and P. Bouthemy, Tracking modeled objects using binocular images, Comput. Vision

Graphics Image Process. 32, 1985, 384-396. 31. P. Rives, Caracttrisation du Dtplacement d'un Object Tridimensionnel. Application au Domaine des

Viddoconferences, INRS Tdtcommunications Technical Report 82-10, 1982. 32. G. Adiv. Inherent ambiguities in recovering 3D motion and structure from a noisy flow field, in

Proceedings, Conf. on Computer Vision and Pattern Recognition, San Francisco, CA, 1985, pp. 70-77.

33. G. Dahlqnist and A. BjSorck, Numerical Methods, p. 143, Prentice-Hall, Englewood Cliffs, NJ, 1974.

Documents

Three-dimensional space from optical flow correspondence