Embed Size (px)
Citation preview
Calibration Stability of an Underwater Stereo Video System:
Implications for Measurement Accuracy and Precision
Euan S. Harvey
Department of Marine Science University of Otago
P.O. Box 56, Dunedin NEW ZEALAND
Email: [email protected]
Mark R. Shortis
Department of Geomatics The University of Melbourne
Parkville, VIC, 3052 AUSTRALIA
Email: [email protected]
ABSTRACT
Assessment of population structure is often used to detect and analyse the impact on marine fauna of environmental factors or commercial fishing. A primary tool in the characterisation of population structure is the distribution of the lengths of a large sample of individual specimens of a particular species. Rather than use relatively unreliable visual estimates by SCUBA divers, an under-water stereo video system has been developed to improve the accuracy of the measurement of lengths of highly indicative species such as reef fish. In common with any system used for accurate measurements, calibration of the camera system is of paramount importance to realise the maximum possible accuracy from the system. Further, the determination of the relative orientation of the two cameras is vital to the correct estimation of fish lengths. Also at issue is the stability of the calibrations and relative orientation of the cameras during deployments to capture video sequences of marine life, as any variations will inevitably lead to systematic errors and therefore inaccuracies in the measured lengths. This paper describes a series of experiments concerning the determination and testing of camera calibration, relative orientation and stability of the underwater stereo video system. The strategy for the integrated determination of calibration and orientation is described. Variations in calibration and orientation parameters are quantified in terms of magnitude and significance. Finally, the detected variations are analysed for the propagated effects on object space accuracy.
INTRODUCTION
Traditional biological sampling techniques in the marine environment have involved the use of nets, bottles, grabs and cores to obtain samples. More recently visual estimation and photographic recordings have become used, particularly in reef ecology. Marine scientists rely almost exclusively on manual taxonomic classification and enumeration of these samples, techniques which are highly sensitive to human error, subjectivity and sampling bias. The flow on effects resulting from error in classification, enumeration and morphometric measurement must affect confidence in results which compare spatial and temporal data.
Accurate information about fish or invertebrate abundance and distribution is required for fisheries and natural resource management, and to further a basic understanding of the population dynamics and ecological interactions that occur within rocky reef, deep sea and pelagic marine environments. Of possibly greater value than estimates of relative abundance is a knowledge of the size frequency distribution or biomass of a population. For example, the size structure of a fish population, when linked with an even rudimentary knowledge of the biology of the species, can provide insights into recruitment to the adult population, fishing intensity and rates of recovery from fishing (McCormick and Choat, 1987). Harvey and Shortis (1996) describe an underwater stereo-video system which can be used to overcome the problem of subjectivity and inter-observer variability and bias in visual length or biomass estimates of marine organisms, particularly of reef fish (Harvey et al., 1998a, b, c, d).
STEREO-VIDEO SYSTEM CALIBRATION
Camera Calibration Techniques
In order to be used for accurate, quantitative measurement, any photographic system must be geometrically calibrated. With few exceptions, cameras do not provide a perfect central projection and calibration is necessary in order to model the small departures from an ideal perspective image (Figure 1). Lens distortions (Ziemann and El-Hakim, 1986) are an example of such departure which is always present in the lenses commonly used with video cameras. The calibration of video cameras also typically requires an affinity correction to allow for timing errors in the horizontal scan lines of the video images. Further, if the cameras are focussable, some of the fundamental characteristics of the cameras, such as the principal point position (the intersection of the optical axis with the focal plane) and the principal distance (separation between the perspective centre of the lens and the focal plane) are variable and must be determined for the specific focus setting.
Calibration of an underwater photographic system must incorporate the effects of refraction at the various air-glass and glass-water interfaces which are present when the camera is mounted in a housing for use underwater (inset Figure 1). A number of different approaches to the calibration problem have been proposed and tested, using both film and video camera systems. Perhaps the simplest approach is the use of grids to simultaneously correct for lens distortion, refraction and scale error at a specific distance (Adams, 1982). The clear deficiency of this approach is the limitation to a specific distance and the consequent restrictions on underwater operation. To allow greater flexibility of operation underwater, the camera system must have a more general calibration which allows its use over all distances, or perhaps across a useful range of distances within a calibration volume.
The general approach which has been widely adopted is the use of photography or images of a three dimensional control frame with a number of known positions signalised using targets. Locations of the targets are measured on multiple images taken from a number of different locations relative to the control frame, and then processed through a self-calibrating photogrammetric network solution (Granshaw, 1980). The network solution simultaneously computes the camera calibration parameters, the locations and orientations of the cameras, revised coordinates for the targets, and a number of statistical measures of the quality of the
solution. Incorporation of the refraction effects can be through explicit modelling using ray tracing for the optical paths (Li et al., 1997) or through the implicit incorporation within other parameters of the camera calibration (Schewe et al., 1996; Turner, 1992).
The ray tracing approach is a rigorous solution, but leads to a number of complications. Li et al. (1997) use a two phase calibration for an underwater stereo-video system. The cameras are first calibrated in air to derive the fundamental calibration parameters, such as the position of the principal point, the camera principal distance and lens distortions. The camera system is then calibrated in water to derive the geometry of the air-glass and glass-water interfaces. The refractive indices of the elements in the path are assumed or directly measured rather than being adopted as unknowns within the network adjustment.
Perhaps the clearest disadvantages of the rigorous ray tracing solution are the requirements for a two phase process and the necessary assumptions of refractive indices for the media. The approach does not account for changes in the relative geometry of the various interfaces in the optical path, or indeed the calibration of the cameras, due to handling of the system or changes in pressure and temperature. Although the adopted refractive index for the glass elements and the internal air path may be reliable, small changes in refractive index of the water path due to changes in pressure, temperature and salinity cannot be eliminated.
An analysis of the effects of the refractive surfaces in the optical path in an ideal waterproof camera housing shows that images are displaced radially from the principal point (Li et al., 1996). Whilst the assumptions that the optical components of the housing are symmetric around the optical axis of the camera and refractive surfaces are, in general, orthogonal to the optical axis are unlikely to be totally fulfilled in practice, it is clear that the principal component of the refractive effect is radial (Fryer and Fraser, 1986). The principal component of image distortion from camera lenses is also radial and the self calibration algorithm incorporates an odd-ordered polynomial to model radial lens distortion. The close analogy between, and the consequent confusion of the signal from, the effects of these two phenomena is partly the reason for the two stage calibration process used by Li et al. (1997).
As a consequence, the alternative approach which has been widely adopted has been to allow the refractive effects of the optical components and refractive interfaces to be absorbed by the camera calibration parameters. The principal component is implicitly taken up by the radial distortion model, whilst any residual effects from asymmetric components of the housing are partly or wholly absorbed into other parameters of the camera calibration, such as decentring lens distortion or the affinity term. No assumptions need to be made concerning the refractive indices of the air, glass or water media, and modelling of the optical components of the underwater housing is unnecessary. This approach was used to process tank calibrations of stereo film cameras used in the North Sea for oil rig inspections (Leatherdale and Turner, 1983) and later extended to video camera systems (Turner, 1992). More recently, the approach has been adopted for calibration of a single video camera used to monitor the shape of deployed fish nets (Schewe et al., 1996).
Leatherdale and Turner (1983) constructed an accurate, rigid control frame within the tank to pre-calibrate the stereo-video system, which raises the possibility of camera calibration
variations between the tank calibration and the inspection tasks in the North Sea. Schewe et al. (1996) employed a flat plate to avoid the difficulties associated with accurately maintaining the rigidity of a three dimensional structure in open water. Although attaching rigid structures to submersibles has been proposed in the past to provide an in situ, three dimensional control frame (Baldwin and Newton, 1982), such ideas have been limited by the requirement for accurate target locations on the frame.
The approach used in this research is to conduct in situ, self calibrations of the camera using a three dimensional control frame (Harvey and Shortis, 1996). To overcome the requirement for a rigid frame which maintains an accurate structural integrity, a free network solution is used (Granshaw, 1980) which requires only approximate coordinates for the targets on the frame. Accurate coordinates of the signalised points are determined as part of the self calibration solution, except that the overall scale of the frame, and therefore any subsequent measurements taken by the stereo-video system, may require a small correction. The photogrammetric advantage of this approach is that the camera calibration parameters are representative of the prevailing underwater environment, because factors such as housing geometry and the refractive index of the water are modelled implicitly in the calibration parameters. The practical advantage is that an open, light aluminium structure (Figure 2) can be employed as the control frame and must retain a rigid shape only for the duration of the image acquisition.
The practical disadvantages of the free network, self calibration solution are that it requires relatively many images to be acquired and measured, and a well defined set of images must be acquired. Through the combination of experience with underwater photography and simulation of photogrammetric networks, a standard procedure of sixteen images in a set pattern has been adopted as a compromise between the requirements for an accurate calibration and the limits on dive time and manoeuvrability of the stereo-video system and the control frame. The control frame is supported by a tether from the surface to allow it to be easily positioned and rotated by one diver. Rotating the frame with respect to the cameras is more easily accomplished than swimming around the frame with the cameras. This procedure is reinforced by the requirement to roll the camera at four different orientations with respect to the frame (0º, 90º, 180º and 270º), in order to reduce the effects of some correlations between calibration parameters. Hence the set of sixteen video images comprises four views of the control frame at four different roll angles.
The disadvantage of the algorithm with respect to accuracy is the incompleteness of the model. It is unlikely that the complete effects of the refraction will be modelled by the camera calibration parameters, and the quality of the photogrammetric solution will generally be less than optimal. Further, because the implicit solution is of an empirical nature, it could be expected that the model of the refraction effects will only be appropriate within the volume represented by the control frame. Measurements made outside of the corresponding range of distances could be expected to be increasingly inaccurate with increasing distance from the calibration volume.
Whilst the forgoing problems with the approach can be overcome or avoided by adhering to well defined procedures, the free network algorithm has a side effect that some camera
calibration parameters are not as confidently determined, as compared to a calibration based on a rigid frame with accurate point locations. Although some improvement could be expected with many more images, the limiting factor is correlations between the parameters of the camera calibration, and correlations between the camera calibration parameters and the parameters describing the locations and orientations of the cameras relative to the control frame. The significance of the correlations is enhanced by the relatively narrow field of view of the video cameras, caused by the very small CCD arrays typically used with small, portable video camcorders.
Correlations between the principal point location, decentring lens distortion and the locations and orientations of the cameras are well known phenomena for self-calibration networks. Suppression of the principal point or decentring distortion parameters is one option, but this is not desirable as these parameters are also absorbing some of the signal from the refraction effects. Hence this inherent weakness of the photogrammetric network solution has to be accepted, and in general has little consequence for measurements made in the object space within the range of distances defined by the control frame.
Relative Orientation
In addition to the calibration of the internal characteristics of the cameras, a stereo-video system also requires the spatial relationship between the two cameras to be determined. Commonly called the relative orientation of the stereo-video system, the parameters required are the separation of the perspective centres of the cameras on the base bar, also known as the camera base, and the rotations (phi, omega and kappa) of one camera with respect to the other (Figure 1). The positions of the cameras with respect to the control frame are no longer relevant once the calibration sequence is complete, other than to establish the relative orientation.
The relative orientation of the stereo-video system can be determined using an integrated algorithm (King, 1995) or post-processed by analysing the camera station locations and orientations produced from the photogrammetric network solution. The integrated solution is preferable for reasons of efficiency and a consequential strengthening of the network solution. The improvement in the network solution is due to the inclusion of constraints which demand that every stereopair has a fixed relative orientation (King, 1995). The values of the constraints become the relative orientation parameters, whereas the post-processing method requires the camera station data to be mathematically transformed into a coordinate system which is aligned on the camera base. The sixteen stereo camera data sets are averaged to produce the relative orientation data.
The disadvantages of the integrated network solution are the requirement for a much more complicated solution algorithm and the increased complexity of the analysis of the results of the network solution. The alternative of post-processing separates the two phases of the system calibration and thereby simplifies the analysis of each phase. A simplified post-processing calculation for relative orientation is particularly pertinent when parameter correlations are present. Correlations between the camera calibration parameters and the camera station locations tend to distort the otherwise fixed relative orientation of the stereo
cameras, leading to apparent errors in the post-processed data. A further factor is the imprecision of the synchronisation of the left and right images from the stereo-video system. The limitation, imposed by the current hardware (Harvey and Shortis, 1996), will also lead to small variations in the relative orientation.
The response to such errors is typically to remove the offending data from the solution and re-compute the relative orientation based on a data set of less than sixteen stereo pairs. The standard deviations of the average relative orientation parameters can be used as a guide to the quality of the solution, and as a statistical tool to provide a consistent process for the elimination of data.
Once the relative orientation of the stereo-video system is established, measurement of the positions of objects can subsequently be determined in three dimensions relative to the camera base. The only restriction is of course that the object is clearly visible within the fields of view of the stereo cameras. Two positions can be compared, within a single synchronised pair of images, to compute lengths, which is the primary data used in sampling populations of species such as reef fish. The only additional information which may be required is the scale factor correction, but in general this potential error is substantially less than the precision of the measurement of the position or length.
Stability Issues
There are many issues to be considered when predicting or testing the stability of the calibration of cameras used for quantitative measurements. Perhaps the first consideration should be that no camera will be perfectly stable because all cameras are handled in some way during routine operations. The influence of handling is greater for cameras which are not designed for quantitative measurement, such as video camcorders. Long term experience has shown that large format aerial cameras used for topographic mapping, which are constructed from a monolithic piece of die-cast metal, are extremely stable, whereas 35mm SLR based digital still cameras show demonstrable variations (Shortis and Beyer, 1997). The flexing of the body of the camera and the movement of the focal plane CCD sensor contribute to the variation, both instigated by handling during photography.
The same investigation also clearly showed that any interference with the camera produced dramatic changes in the camera calibration parameters (Shortis and Beyer, 1997). The simple act of separating the camera body for inspection, and the more invasive process of stabilisation of the CCD array, produced changes in the calibration parameters which were more than an order of magnitude greater than the effects of handling.
The effects of a change in the lens focus are well documented (Shortis et al., 1996). Conventional lenses used for video camcorders and 35mm SLR cameras change the focus by varying the principal distance of the lens. The variation of the principal distance is carried out by the motion of one or more of the lens elements on a screw thread. This motion typically effects the position of the principal point due to variation in the optical axis of the lens, effects decentring distortion by altering the alignment of the pattern of the distortion and has a minor affect on radial distortion, as this type of distortion is scaled by the magnitude of the principal
distance. The magnitude of these effects are generally small but significant for standard lenses, whereas the effects for zoom lenses are readily apparent (Burner, 1995).
A stereo-video system introduces the additional stability issue of the relative orientation. Handling of the system could be expected to vary the relative orientation for similar reasons and to similar magnitudes to the effects on camera calibration parameters. Small variations might be expected to result from routine use, whilst disassembly would result in larger changes.
In summary, small changes in the camera calibration and relative orientation of the stereo-video system might be expected for a single dive where the cameras are being manipulated in the underwater environment. Larger changes would be likely to occur between dives, as the cameras are handled more and the water proof housings must be opened to retrieve video tapes. Dramatic changes may occur if the cameras are refocused or the camera base bar or waterproof housings are disassembled for inspection or routine maintenance.
As the stereo-video system is to be used for making biological measurements of distances from subjects or lengths of morphometric features, it is important to estimate the variability in the accuracy and precision of any measurements made during any one particular deployment and over a series of deployments. Therefore, it is also desirable to estimate the variability in the calibration during one deployment and over a series of deployments. Accordingly, this paper has three aims:
1. To determine the level of stability of the calibration parameters during single deployments under ideal and field conditions in the case of the stereo-video system remaining intact
2. To determine the level of stability of calibration parameters over a series of deployments in the case of the stereo-video system being dismantled between dives
3. To determine the affects of variation in calibration parameters during one deployment and over a series of deployments on the accuracy and precision of measurements
The stereo-video system used in this research was originally designed to measure the size of reef fish ranging in length from 5 centimetres to 3 metres at distances ranging between 2.5 and 10 metres. The system utilises two Sony VX1E Hi 8 video cameras in water proof housings. Mounted on a frame, the two cameras are separated by a distance of 1.44 metres. The focal lengths are set at approximately 10 millimetres to obtain fields of view of 2.5 metres at a range of 5 metres. Each camera is inwardly converged at 8.5 degrees to gain an optimised stereoscopic field of view. A more comprehensive description of the system and design considerations may be found in Harvey and Shortis (1996).
METHODS
Calibration Stability Testing
Ten calibrations were recorded at the Moana Swimming Pool, Dunedin, New Zealand on December 22, 1997 at a depth of 5 metres. The calibrations were made one after the other over a 45 minute time period, with no changes being made to the stereo-video rig. Each calibration took on average 4 minutes 20 seconds with visibility being estimated at approximately 25 metres.
Fifteen calibrations were made under field conditions in open water during routine field work monitoring reef fish populations in Milford Sound, on the west coast of New Zealand’s South Island. Calibrations in the field were undertaken at a depth of eight metres and took on average 2 minutes and 49 seconds. Visibility was estimated to range between 6 and 15 metres. The dates of calibrations and visibility are listed in Table 1.
Five pairs of the fifteen field calibrations were recorded at the beginning and (a) and end (b) of 5 field deployments for the purpose of investigating variability between calibrations in the field during 1 deployment (Table 1). Between the start and end of each of the 5 paired calibrations the camera system was left in the water. All calibrations took place in approximately eight metres of water.
Handling of the underwater stereo-video system between the surveys at field sites is inevitable due to the requirements of transporting the system by road, transferring the system onto the boat and deploying the system into the water. In all cases the underwater housings were taken off the frame and the cameras were taken out of the housings in order to retrieve the video tapes and grab the video frames for further image measurements and processing.
For both the pool and field calibrations, the focal lengths for both cameras were set at the in-water equivalent of 10 millimetres to obtain an optimal field of view. However the focal lengths were reset between field and pool calibrations, disallowing a direct comparison between camera calibration parameters derived from the pool and field experiments.
Accuracy and Precision Variability Tests
For both pool and field experiments, a series of measurements on the images were made between points of a known location. The measurements were made of five different lengths with ten repeat measurements being made for each length from each calibration. The lengths derived from the video system were subsequently compared to known lengths determined by direct measurement with a steel surveyors tape.
Three of the measured lengths were recorded between identified targets on the control frame, corresponding approximately to the X, Y and Z coordinate axis directions (see Figure 2). With some minor variations, identical pairs of targets were recorded for all tests. The two additional measurements were made to a calibration check plate located 2.6 metres in front of the stereo-video cameras. The check plate is a flat, rectangular aluminium plate 25 by 5
centimetres in size in the horizontal and vertical dimensions respectively (see Figure 1). The plate is painted black and has 11 white dots 1 centimetre in diameter evenly distributed over it, so designed to enable automatic calibration checking in the future. This calibration check plate can be seen in the bottom right hand corner of the upper left image and the bottom left hand corner of the right upper image displayed in Figure 2. Periodic measurement of the points on the check plate can be made to detect any variation in measurement accuracy due to any potential calibration instability. Measurements were made horizontally across the top edge of the plate and vertically, from the top right corner to the bottom right corner of the plate.
Theoretically, if the calibrations are stable there should be little, if any, variation between lengths derived from image measurements from one stereopair taken from any particular deployment, computed using different calibration data established during other deployments. Any variation must be attributable to mistakes or misidentifications in the preliminary processing of data input into the calibration algorithm, or error in the calibration process itself. Therefore, additional measurements were made from one stereopair of images from the tenth calibration in the pool using all the calibration data sets from the pool deployment. The procedure was repeated using all the calibration data sets from the field sites to derive lengths from one stereopair from Site 15a, in order to determine the affect within field calibrations.
To provide a more rigorous understanding of the variation in the accuracy and precision of measurements, further measurements were made during a series of 13 subsequent dives at open water sites. In total, an additional 600 measurements of each of the horizontal and vertical dimensions of the check plate were made during these 13 field deployments. The dates of deployment, duration and number of measurements made are listed in Table 2. The tests aim to detect small variations in the accuracy and precision of measurements during single field deployments, which are attributable to variation in the actual calibration of the system during the duration of the dive.
ANALYSIS OF DATA AND RESULTS
Analysis of the Stability of Calibration Parameters
For each of the 10 pool calibrations at Moana and the 15 open water calibrations up to sixteen stereopairs were used in each of the photogrammetric network solutions. As previously noted, the network solution computes values for the camera calibration parameters, as well as statistical estimates of the expected variations of the parameters. The values of the calibration parameters can be considered as “average” values from all stereopairs in each network, although this is an over-simplification. The computed standard deviations of the parameters are based on a number of factors, the most important of which are the precision of the image measurements, the number of target images, the number of video images and the geometry of the network.
The post-processing to compute the relative orientation is similarly based on all stereopairs in each network. For each stereopair the length and spatial direction of the base between the perspective centre positions, as established by the network solution, is computed and then the
stereopair is mathematically transformed to a “zero” position where the phi, omega and kappa rotations are evaluated. Average values of the relative orientation parameters and estimates of their precisions, in the form of standard deviations, can be computed from the sixteen samples.
Analysis of the results of the camera calibrations and relative orientation calibrations showed consistent results in all but one case. In general, the camera calibrations produced an estimate of the image measurement precision of 3-4 micrometres, which is approximately one half of the dimension of a pixel in the video images. Precisions from target measurements are typically 0.2-0.3 pixels for in-air calibrations, and the poorer result in this work can be readily attributed to the influence of the dispersion effects of water, incompleteness of the refractive model and poor visibility in some cases. The results of the relative orientation computations were also reasonably consistent, after rejection of one or two stations of the sixteen in most cases. The exception to this rule was the second pool calibration which showed no real consistency in the relative orientation, despite having acceptable left and right camera calibrations. This pool calibration was rejected from the complete set, leaving 9 pool calibrations in total.
An initial impression of the variation of all the calibration parameters can be obtained from the standard deviations of the average parameters from the 9 individual pool calibrations and 15 individual open water calibrations. These values, shown in Table 3, can be used to determine a comparative variability between the pool and open water calibrations. Selected parameters only are shown in the table. As radial and decentring distortion are modelled by multiple parameters, comparisons for these employ the magnitude of the distortions at a radius of two millimetres from the principal point within the video image format.
The table clearly demonstrates that there is, overall, greater variability of calibration parameters in open water compared to the pool. The average ratio of the standard deviations is 2.7, indicating that in general the open water calibrations were almost three times more variable. This a general confirmation that the handling and disassembly of the stereo-video system inevitably leads to changes in the calibration parameters.
A finer level of detail for the variability can be derived by comparisons of consecutive calibrations, as for a completely stable system no differences should be recorded. The larger the magnitude of the difference, the greater the variability. However, raw differences are not sufficient for the comparison, as the variability of each of the average parameters from each network solution or relative orientation solution must also be considered. For example, due to the geometry of the relative orientation, omega rotations are derived much more confidently than phi rotations. As a consequence, the same magnitude of variation for an omega and a phi rotation is much more significant for the omega rotation. Hence, significance values are computed using the following formula
Significance = 1nn
1nn
variancevarianceparameterparameter
−
−
+
−
where the variance is the square of the standard deviation. The value effectively gives a weighted, dimensionless estimator of the significance of the change in the parameter. These values can be statistically tested against a distribution such as the Student T. For a 95% confidence level and 15 degrees of freedom the critical value is 2.1, however comparative values are generally sufficient for the purpose of this investigation.
Shown in Figure 3 are significance values for the right camera calibration parameters and the relative orientation parameters for the pool and open water calibrations. The left camera calibration significance values are similar, or slightly less, in magnitude and so are not shown. The graphs clearly show the greater variability of the open water calibrations. In comparison, the pool calibrations are quite stable, with some exceptions such as the omega rotations in the relative orientation and the decentring distortion in the camera calibrations. However, the significant changes in these parameters may be effected by correlations, so there is some doubt as to whether these values represent real variation or are an artefact of the calibration technique. The exactly parallel changes in the left and right omega values are a result of the mathematical transformation employed for the relative orientation, which seeks a mean rotation of the camera optical axes with respect to the camera base (see Figure 1).
In contrast, the variations in the parameters for the open water calibrations are extreme in some cases, such as the changes which occurred between two sites on the second day where the disassembly of the stereo-video system, or some other factor such as a refocusing of the lenses, has clearly had a dramatic effect. Although the influence of correlations cannot be ruled out completely, significance values which are 20 times the critical value must be indicative of extraordinary change which is not statistically acceptable.
With the exception of the omega rotations, the relative orientation shows good stability which indicates that the camera base bar and housings are maintaining their rigidity. The omega rotations are the most confidently determined parameters, and the high confidence of their determination has an influence on the results. However, the variations in the omega rotations may be attributed to the insertion of the camcorders into the waterproof housings, which is the least reliable component in the mounting process. The insertion process may also be one cause of the variation in the camera calibration parameters, as changes in the relative positions of the cameras and the glass ports of the housing would lead to different refraction regimes. Other factors, such as handling and re-focussing of the cameras, different water depths and different conditions in general will all contribute to the observed variations.
Analysis of the Accuracy and Precision of Stereo-video Measurement Data
The error of stereo-video measurements in the horizontal and vertical dimensions of the calibration check plate, and the X, Y and Z dimensions of the control frame were calculated by subtracting the stereo-video estimate from the known lengths of the check plate or distance between targets on the control frame. The mean error, standard error (SE) and coefficient of variation (CV) were then calculated. Standard error is defined here as the standard deviation (SD) of the mean, and coefficient of variation is defined as the ratio of the standard deviation of the estimate and the value of the estimate, expressed as a percentage.
Stereo-video estimates were derived from the stereopair comparator (see Figure 2) which computes locations using the camera calibration and relative orientation data provided from the system calibration, and a simple 3D spatial intersection based on image measurements (Harvey and Shortis, 1996). The comparator also provides a quality measure known as the root mean square (RMS) error, which indicates the exactness of the intersection.
Analysis of variance (ANOVA) was used to determine whether differences existed between measurement errors recorded between dimensions and also between calibrations. Least significant differences were used to determine differences between individual dimensions and calibrations when ANOVA tests detected differences.
Effects of Calibration Variation on Measurement Accuracy and Precision
All measurements made from pool and field calibrations (entitled Pool and Field) using the associated calibration data sets and also from one pair of images using all calibration data sets (entitled Pool:1cal and Field:1cal) show good accuracy and precision (Table 4). Surprisingly, the field measurements taken in open water were more accurate and more precise than the pool measurements taken in a controlled environment.
The accuracy for the measurements made from pool images is very close to the 1.27 centimetre error reported prior research (Harvey and Shortis, 1996). A smaller mean error was anticipated for this research work because the previous research used a control frame of 1 by 1 by 0.5 metres dimension, whereas this study employed a control frame of 2 by 2 by 1 metres. Plots of the mean error (Figure 4) and CV (Figure 5) show that higher than expected mean error is caused principally through deviations of measurements in the Y dimension. Measurements of the check plate were made at a distance of 2.6 meters from the camera, whilst measurements in the X, Y and Z dimension of the control frame ranged from 4.6 to 7.1 metres from the cameras with measurements of the Z dimension always being the furthest from the camera, followed by Y and X dimensions. Therefore it would be expected that the horizontal and vertical dimension measurements to have the best accuracy and precision, followed by measurements in the X, Y and finally Z dimension, as the precision of measurements taken with the stereo-video system deteriorates with increasing distance. This anticipated pattern is reflected in the results with larger errors and standard errors being recorded (Figure 4) and a trend to increasing CV (Figure 5) with the exception of the Y dimension Pool and Pool:1 cal data.
One way ANOVA testing was performed on the error recorded between the five measured dimensions for Pool, Pool:1cal, Field and Field:1cal (Table 5) showed significant differences and low correlations between the dimension types for Pool and Pool:1cal measurement errors. In contrast, the Field and Field:1cal did not show significant differences and exhibited high correlations.
Differences between dimension types were determined for Pool and Pool:1cal (Table 6) by calculating the least significant differences. The least significant differences show that differences existed between the measurement error in the Y dimension in comparison to all other dimensions. When the Y dimension data is removed from Pool and Pool:1cal data sets,
the mean errors decrease to 0.01 centimetres (SE = 0.05, SD = 0.33, sample size = 36) and 0.02 centimetres (SE = 0.12, SD = 0.71, sample size = 36) respectively, which is comparable to the data for Field and Field:1cal.
One way ANOVA testing was performed on data for Pool, Pool:1cal, Field and Field:1cal, testing for differences between the measurement errors recorded between calibrations for all dimesnions. No significant differences were recorded for the Pool, Pool:1cal and Field (Table 7). However significant differences were detected for Field:1cal (Table 7).
Analysis of the least significance differences at a 99% confidence level for the Field:1cal data showed that differences existed in the measurement error made with the calibrations from sites 2b and 16b, in comparison to measurements made with calibrations from other sites. Inspection of the data shows that measurements made in all dimensions for site 16b had a large error and high RMS errors (mean RMS of 6.3 centimetres for all dimensions) as calculated by the stereopair comparator. Site 2b had larger than average errors in the X and Y dimensions and high RMS errors (mean RMS of 3.6 centimetres for all dimensions). In general the pool and field data produced RMS errors of the order of 1-2 centimetres (Table 4), indicating that the system calibrations for sites 2b and 16b are not optimal.
A one way ANOVA test showed that there were no significant differences between the horizontal or vertical measurements (Table 8) of the calibration check plate between the 13 dives listed in Table 2. The range of error for any one particular deployment was relatively small as seen from the graphs plotting error against time for dives 2, 6 and 12 (Figures 6, 7 and 8).
DISCUSSION
The calibration strategy used for this research resulted in small variations within calibration parameters, both over the duration of one deployment and through a series of deployments over time. The dismantling of the housings from the frame, and removal of the cameras from the housings between calibrations at field sites is the most likely cause of the larger variations seen in omega rotation and, to a lesser extent, in the phi rotation in the field data. Even though the design for the system requires that the underwater housings bolt into position on the frame and the cameras slide into a groove in the underwater housings in an attempt to eliminate this problem, it is still possible for minor variations in re-assembly to occur. This is a technical problem that can resolved in the future as part of the development of an improved and technologically more sophisticated system.
However, for a biologist wishing to use a stereo-video system for field work, it is important that the system can be quickly dismantled and reassembled during transportation to and from field sites with minor variations in accuracy and precision. Our research shows that the small variations in the calibration parameters that we recorded when the rig was dismantled between deployments does not greatly affect the accuracy and precision of length or distance measurements, with data recorded in the field (overall mean error of 0.06 centimetres) being only slightly less accurate than data measured under ideal conditions in a swimming pool (overall mean error of 0.02 centimetres, excluding the Y dimension data).
Ideally, to ensure that sufficient information is recorded to account for variations in camera calibration parameters over any particular deployment, calibrations should be made at the start and end of every deployment. It is clear that small variations in calibration parameters between calibrations made during the same deployment have a negligible affect on the accuracy and precision of measurements, given that any single measurement from the stereo-video system has an RMS error of the order of 1-2 centimetres.
Although calibrations in the field only required approximately 3 minutes to carry out, biologists conducting subtidal research with the aid of the underwater stereo-video system are limited in the duration of the excursion by physiological constraints. Therefore, the time available for underwater survey work must be maximised and system calibrations on every second or third deployment would be advantageous. The results from measurements made from the same pair of images using all the field calibration files show that it is possible to use calibration files from previous deployments to make accurate and precise measurements for different sets of images. Hence it is feasible to use calibration data from earlier deployments. However the system should be calibrated whenever the underwater housings are removed from the base bar, the cameras are removed from the housings or the underwater visibility changes dramatically. As an additional safety measure, routine measurements should be made of a check plate or other objects of a known size during the analysis of recorded images for deployments in which the system is not calibrated. Measurements of the calibration check plate which exhibit and RMS error higher than 2.5 centimetres would indicate the presence of unacceptable error levels from variation in the system calibration, or other sources such as poor synchronisation of images.
Further investigation of the large variation in measured dimensions in the Y direction (see Figure 2) for the pool data indicated a correlation between the attitude of the cameras and under-estimation and over-estimation of distances. The required roll strategy of 0º, 90º, 180º and 270º changes the influence of gravity on the base bar and housings, which may induce local movement of the cameras within the housings and changes in the orientations of the housing with respect to the base bar. It is apparent from the results shown in Table 6 that these effects have the largest influence on the Y dimension for the pool data. The Y and Z directions are effected most strongly as these are the dimensions for which precision deteriorates most quickly with distance, degrading in proportion to the square of the range from the cameras, as opposed to degrading in proportion to the range for the X dimension. The analysis showed not only a lesser effect for the open water tests, but also the tendency for the open water and pool measurements to slightly under-estimate and clearly over-estimate distances respectively. The reasons for these phenomena are unknown at this time, although the presence of correlations is often a prime source of unexpected results.
Whilst future development of the system aims to minimise such errors, the influence on real measurements of the length and height of mobile and sessile marine species is generally much less significant for two reasons. The first is that, in general, the lengths and heights of individuals to be measured are less than 30 centimetres and the potential error will be reduced in proportion to the distance. The second reason is related to the preferred orientation of individual fish or other species to be measured. Previous research (Harvey and Shortis, 1996) has shown that measurement errors are minimised when the length or size to be measured is
approximately parallel to the plane of the images. Acute angles reduce the visibility of edges or key points (such as the fork of the tail of fish) used for measurement. Hence the vast majority of measurements are made in the X direction (see Figure 2), avoiding the Y dimension which is most influenced by the variability.
Variations in calibration parameters and correlations with camera attitude may also be attributable to flex in the control frame whilst it is being rotated as part of the calibration procedure. Flex has been noted during the pool and open water testing, particularly when the frame is being manipulated and rotated by more vigorous field assistants. The design of a more rigid frame may assist in the reduction of these small variations.
To date most of the measurements made with the system have been from images recorded at depths between 1 and 18 metres. It is possible that increasing water pressure due to increasing depth may deform components of the underwater housings and frames, causing deterioration in measurement accuracy and precision. This is an issue which needs to be investigated and resolved if stereo-video systems are to be mounted onto remotely operated vehicles or submersibles to assist in deep sea marine research.
CONCLUSIONS
This paper has presented a study of the calibration stability of an underwater stereo-video system designed for non-contact sampling of populations of marine species. In the controlled environment of the pool there were few significant changes to the calibration parameters, indicating that the system is very stable if the cameras and housings are not disturbed. The calibration was, as could be expected, less stable in the more realistic environment of use at different open water sites. The significant changes to some of the parameters for the system calibration is attributed to the disassembly of the system required for transport between sites and access to the recorded video tapes.
In terms of the derived lengths, extensive testing shows that the average mean errors in distances derived from the underwater stereo-video system are 0.02 and 0.06 centimetres in controlled and open water environments respectively. Calibrations of the camera system appear to be stable both within and between deployments of the system. The small variations in camera parameters appear to have minimal affect on the accuracy and precision of measurements made with the system. Therefore, marine biologists wishing to utilise this technology can be assured that there will be minimal variability in the accuracy and precision of measurements made at different times on any one deployment and over repeated deployments.
Future development of the system will include both hardware and algorithmic improvements. Improved housings will reduce the potential for relative movement of the cameras within the housings and the housings with respect to the base bar. The next revision of the system will incorporate a synchronisation generator and time code to enable the automatic, accurate matching of image pairs from the two cameras. Research is currently underway to automate the measurement of images required for the calibration using coded patterns on the control frame. Further, an advanced network solution is under development which will incorporate
scale constraints to reduce the impact of correlations between calibration parameters, as well as minimise the apparent correlation between the attitude of the cameras and the derived distances.
The improvements in automation and accuracy will allow the underwater stereo-video system to be a precise, reliable tool which can gather information on marine fauna quickly and conveniently. The ability to gather large samples of length and size data for populations of marine species, with little temporal or spatial variation in the integrity of the data, will allow much more powerful statistical testing. More powerful tests will in turn dramatically improve the ability of marine biologists to detect and predict the influence of environmental impacts on populations of marine species.
References Adams, L. P., 1982. Underwater analytical photogrammetry using non-metric cameras.
International Archives of Photogrammetry, 24(5) : 12-22. Baldwin, R. A. and Newton, I., 1982. A proposed system for underwater photogrammetry
from a manned submersible camera platform. International Archives of Photogrammetry, 24(5) : 39-52.
Burner, A. W., 1995. Zoom lens calibration for wind tunnel measurements. Videometrics IV,
SPIE Vol. 2598, pp 19-33. Fryer, J. G. and Fraser, C. S., 1986. On the calibration of underwater cameras.
Photogrammetric Record, 12(67) : 73-85. Granshaw, S. I., 1980. Bundle adjustment methods in engineering photogrammetry.
Photogrammetric Record, 10(56) : 181-207.
Harvey, E. S., Fletcher, D. and Shortis, M. R., 1998a. A comparison of the precision and accuracy of visual estimates of reef fish length made by divers and a stereo-video system. Submitted to Marine Ecology Progress Series.
Harvey, E. S., Fletcher, D. and Shortis, M. R., 1998b. Improving the statistical power of visual length estimates of reef: A comparison of divers and stereo-video. Submitted to Marine Ecology Progress Series.
Harvey, E. S., Fletcher, D. and Shortis, M. R., 1998c. Visual census of reef fish: The effect of observer bias and error in visual estimates of distance. Submitted to Marine Biology.
Harvey, E. S., Fletcher, D. and Shortis, M. R., 1998d. The accuracy and precision of diver field estimates of reef fish length. In preparation.
Harvey, E. S. and Shortis, M. R., 1996. A system for stereo-video measurement of sub-tidal organisms. Marine Technology Society Journal, 29(4) : 10-22.
King, B. R., 1995. Bundle adjustment of constrained stereopairs - mathematical models. Geomatics Research Australia, 63 : 67-92.
Leatherdale, J. D. and Turner, J., 1983. Underwater photogrammetry in the North Sea.
Photogrammetric Record, 11(62) : 151-167. Li, R., Li, H., Zou, W, Smith, R. G. and Curran, T. A., 1997. Quantitative photogrammetric
analysis of digital underwater video imagery. IEEE Journal of Oceanic Engineering, 22(2) : 364-375.
Li, R., Tao, C., Zou, W, Smith, R. G. and Curran, T. A., 1996. An underwater digital
photogrammetric system for fishery geomatics. International Archives of Photogrammetry and Remote Sensing, 31(5) : 319-323.
McCormick, M. I., and Choat, J. H. 1987. Estimating total abundance of a large temperate-
reef fish using visual strip-transects. Marine Biology, 96(4) : 469-478. Schewe, H., Moncreiff, E. and Gruendig, L., 1996. Improvement of fish farm pen design
using computational structural modelling and large-scale underwater photogrammetry. International Archives of Photogrammetry and Remote Sensing, 31(5) : 524-529.
Shortis, M. R., Robson, S. and Short, T., 1996. Multiple focus calibration of a still video camera. International Archives of Photogrammetry and Remote Sensing, 31(5) : 534-539.
Shortis, M. R. and Beyer, H. A, 1997. Calibration stability of the Kodak DCS420 and 460 cameras. Videometrics V, SPIE Vol. 3174, pp. 94-105.
Turner, J., 1992. The development of an operational digital photogrammetric system for the North Sea oil and gas industry. Videometrics, SPIE Vol. 1820, pp.136-144.
Ziemann, H. and El-Hakim, S. F., 1986. System calibration and self calibration - Part 1 :
Rotationally symmetrical lens distortion and image deformation. Photogrammetric Engineering and Remote Sensing, 52(10) : 1617-1625.
Site Date Duration
(minutes:seconds) Visibility (metres)
16a 16/02/96 2:50 15 16b 16/02/96 2:26 8 2a 17/02/96 1:05 12 2b 17/02/96 2:27 9 4a 17/02/96 3:00 12 9a 17/02/96 3:03 8
12b 18/02/96 2:30 8 10a 20/02/96 3:17 13 10b 20/02/96 2:48 13
17a 02/03/96 3:45 12 17b 02/03/96 3:04 6 15a 03/03/96 3:15 14 6b 04/03/96 2:57 10 5a 04/03/96 2:01 12 5b 04/03/96 3:42 13
Table 1. Details of calibrations of the underwater stereo-video system in open water for field deployments.
Dive Date Dive Time
(minutes) Visibility (metres)
Number of measurements
1 20/4/96 29 10 58 2 20/4/96 33 10 33 3 21/4/96 32 10 55 4 21/4/96 35 8 32 5 21/4/96 34 8 31 6 23/4/96 27 10 38 7 23/4/96 37 11 50 8 24/4/96 33 9 30 9 30/10/96 30 6 43 10 30/10/96 35 6 34 11 30/10/96 40 6 77 12 30/10/96 42 6 64 13 30/10/96 50 6 56
Table 2. Details of additional open water dives and calibration check plate measurements.
Left rotations
(degrees)
Right camera calibration parameters
(micrometres)
Data Base
(cm) Phi Omega Kappa PPx PPy PD Radial
Dist. Dec. Dist.
Pool
3.0 0.346 0.082 0.126 69.0 42.2 30.6 2.1 1.6
Open Water
12.2 1.231 0.206 0.214 154.0 78.3 100.3 3.9 5.4
Ratio 4.1 3.6 2.5 1.7 2.2 1.9 3.3 1.9 3.4
Table 3. Comparison of the standard deviations of selected calibration parameters for the 9 pool and 15 open water calibrations.
Data Error
(centimetres) SE
(centimetres) SD
(centimetres) Sample size
Pool 1.27 0.40 2.67 45 Pool:1cal 1.23 0.38 2.52 45 Field 0.03 0.10 0.86 75 Field:1cal 0.08 0.27 1.94 75
Table 4. Accuracy values and precision measures of the measured lengths for the pool and field calibration tests.
Degrees of Freedom Data
Dimension Error Total
Fisher Statistic
Correlation Factor
Pool 4 40 44 85.53 0.00 Pool:1cal 4 40 44 305.41 0.00 Field 4 70 74 0.35 0.84 Field:1cal 4 70 74 0.80 0.53
Table 5. Results of ANOVA tests of the 5 measurement dimensions for the pool and field calibration tests.
Comparison of
Measurement Dimensions Pool:1 cal Pool
Horizontal Vertical No No Horizontal X No No Horizontal Y Yes Yes Horizontal Z No No
Vertical X No No Vertical Y Yes Yes Vertical Z No No
X Y Yes Yes X Z No No Y Z Yes Yes
Table 6. Results of LSD tests of the 5 measurement dimensions for the pool calibration tests (Yes and no refer to significant differences at a 99% confidence level).
Degrees of Freedom Data
Dimension Error Total
Fisher Statistic
Correlation Factor
Pool 8 36 44 0.11 1.00 Pool:1cal 8 36 44 0.03 1.00 Field 14 60 74 1.01 0.45 Field:1cal 14 60 74 3.11 0.00
Table 7. Results of ANOVA tests of the errors between measurements for the pool and field calibration tests.
Degrees of Freedom Data
Dives Error Total
Fisher Statistic
Correlation Factor
Horizontal 12 587 599 1.10 0.36 Vertical 12 587 599 0.77 0.68
Table 8. Results of ANOVA tests of the errors between measurement dimensions of the calibration check plate for the additional dives.
Video camcorderand waterproof
housing
Controlframe
Calibrationcheck plate
Z
Y
X
x
y
z
Phi
Om ega
BaseBar Kappa
Radial lens distortionDecentring lens distortion
Focal planeCCD array
Lens
Target image
Target
PP
PC
PP = Principal PointPD = Principal D istancePC = Perspectiv e Centre
v
h
Figure 1. Schematic of the geometry of the camera calibration and relative orientation with
(inset) the underwater stereo-video system in operation.
Figure 2. The software interface showing the control plate, calibration frame, centroids and the direction of x, y, and z measurements
Pool Relative Orientations
0
5
10
15
20
1 2 3 4 5 6 7 8 9
Calibration
Cha
nge
Sign
ifica
nce Base
L-PhiL-OmegaL-KappaR-PhiR-OmegaR-Kappa
Pool Right Camera Calibrations
0
5
10
15
20
1 2 3 4 5 6 7 8 9
Calibration
Cha
nge
Sign
ifica
nce PPx
PPyPDRad DistnDec DistnAffinity
Open Water Relative Orientations
0
5
10
15
20
0 2 4 6 8 10 12 14 16
Calibration Date (Days)
Cha
nge
Sign
ifica
nce Base
L-PhiL-OmegaL-KappaR-PhiR-OmegaR-Kappa
Open Water Right Camera Calibrations
0
5
10
15
20
0 2 4 6 8 10 12 14 16
Calibration Date (Days)
Cha
nge
Sign
ifica
nce PPx
PPyPDRad DistnDec DistnAffinity
Figure 3. Graphs of significance values of various calibration parameters (the Fisher critical value is shown as the horizontal dashed line).
-2
0
2
4
6
8
H V X Y Z
Measurement dimension
Mea
n er
ror
(cm
) PoolFieldPool:1 calField:1 cal
Figure 4. Effects of calibration variation on measurement accuracy. Error bars ±1 SE.
0
0.5
1
1.5
2
H V X Y ZDimension
Coe
ffic
ient
of v
aria
tion
(%)
PoolFieldPool:1 calField:1 cal
Figure 5. Effects of calibration variation on the coefficient of variation of measurements.
Figures 6, 7 and 8. Range of error for measurements made of calibration check plate
throughout the three deployments.
-0.4-0.2
00.20.40.60.8
0 5 10 15 20 25 30Time (min)
Erro
r (cm
)Horizontal
Vertical
-0.4
-0.2
0
0.2
0.4
0.6
0.8
0 5 10 15 20 25 30 35 40Time (min)
Erro
r (cm
)
HorizontalVertical
-0.4
-0.2
0
0.2
0.4
0.6
0.8
0 10 20 30 40 50Time (min)
Erro
r (cm
)
HorizontalVertical
