Photogrammetric Record, 15(89): 657–664 (April 1997)

PRACTICAL AUTOMATION IN COMMERCIALDIGITAL PHOTOGRAMMETRY

By A. S. WALKER

Leica AG

(Paper read at the Thompson Symposium held at the University of York on20th April, 1996)

AbstractThe revolution promised by digital photogrammetry is dependent on

extensive, successful automation at many stages of the production process.The progress of automation is uneven. Digital cameras are not in wide-spread use. Scanners are available which include roll film transports andautomated orientation, but set up for optimum tonal transfer remainsinteractive. Extensive automation of several operations on digital photo-grammetric workstations, such as generation of digital terrain models,orthophotographs, mosaics and perspective scenes, contrasts with featureextraction, where robust products lag behind success in research. Progresstowards practical automation to assist, but not replace, the human operatoris significant, however, and examples are given from the Leica-Helavasystems.

INTRODUCTION

DIGITAL PHOTOGRAMMETRY has arrived with a vengeance. The analogue and analyticalworkstations which have been so familiar for so long are on the point of beingeclipsed by digital systems as the workhorses in the photogrammetric production line.A burgeoning literature has accompanied the diffusion of these systems and it isgratifying to see so many papers appearing from users, mostly satisfied adopters ofthe new technology. An interesting debate has taken place on the status of thischange. Are we witnessing just another stage in the evolution of our subject or are weon the threshold of something more dramatic, a revolution in our working methods?Almost 20 years ago, Ackermann noted that post-war photogrammetry would becharacterized not only by continuing incremental development, for example betteranalogue instruments and the widespread appearance of analytical plotters, but alsoby “a transition to a new level”, which would be marked by, “a most impressiveevolution of new techniques … which will supersede the classical concept of photo-grammetry” and would be “opened up bycomputers and electronic technology”(Ackermann, 1977, page 150). Fifteen years later, a similar view was restated(Ackermann, 1992) and the significance of the times was encapsulated by anotherluminary in the phrase, “the promise of softcopy photogrammetry” (Leberl, 1991). Ofcourse, since these words were written, the novelty of digital photogrammetry hasworn off, but the excitement and the potential have not been lost. Ackermann’sdelight in the latest achievements had if anything increased and he commented quiterecently, “The major technical breakthrough … is the step towards digital photo-grammetry … the development which now has started is so fantastic and hasunlimited potential that I am not at all worried about future development… The result

657

will supercede [sic] anything which we could ever have dreamt of, simply because ofthe power of digital technology” (Anon., 1995).

These views have attracted widespread support, but there remain some importantdoubts. One of these concerns the cost of scanners, which push the initial cost ofdigital photogrammetry too high for some tastes. Dowman (1996) commented on this,but perhaps took into account insufficiently the high performance of many modernscanners, which enable one unit to provide enough digital imagery to keep a numberof digital photogrammetric workstations (DPWs) busy or to acknowledge the differ-ent business economics implicit in the growth of service bureau companies whichoffer scanning as a service, often along with photoprocessing. The second worry isautomation. The enormous potential of digital photogrammetry, in terms both ofgreater productivity and a broader range of deliverables compared to the analogue oranalytical predecessors, has always been seen to be dependent on automation, thoughLeberl (op. cit.) accepted that there would be phases of the digital photogrammetricprocess in which automation would be but sparsely available for some time to come,but argued that the transition to digital was epochal nonetheless. Dowman (op. cit.)noted that automation was in progress but its success was rather variable across thedifferent work processes. Whilst firmly in the revolutionary school of thought, Wongwisely used the phrase, “… continued evolution in automation” (Anon., 1996, page54). This paper briefly examines the status of automation, drawing examples mainlyfrom the Leica digital photogrammetric systems by Helava.

CAMERAS AND SCANNERS

While it is clear that extensive efforts are being made to bring digital camerasto reality, as a result of which numerous airborne missions have been flown withdigital imaging devices, it remains the case that no digital camera has been introducedwith geometric performance or format in the same sphere as the traditional aerialphotogrammetric camera. Further discussion of the high levels of performance andautomation now achieved in film cameras would be out of place in this paper, so weturn to the most common precursor to digital photogrammetric work, the scanning, ina precision scanner, of aerial film images. Here automation has a role, however:several scanners on the market have roll film transport, accompanied by software forautomated film advance and interior orientation. The technical challenges of accuratefilm advance, totally autonomous interior orientation and the digital equivalent ofelectronic dodging and compensation for “hot spots” are considerable and at the timeof writing had not been met in their entirety.

DIGITAL PHOTOGRAMMETRIC WORKSTATIONS

The literature on digital photogrammetric workstations (DPWs) has becomevoluminous and it is pointless to attempt to summarize here. Instead, we shall focuson certain of the software functions and the role of automation in each.

Orientation and Triangulation

Relatively simple image processing should, in principle, enable interior andrelative orientation to be automated. Many vendors include these functions in theirDPWs, though the level of automation can vary considerably. Kersten and Stallmann(1995), for example, have developed sophisticated software for batch processing fortotally automated interior orientation and at the time of writing are interfacing thiswith the SOCET SET software of the Leica-Helava DPWs. Although automatedtriangulation is a major topic for discussion, it is worth remembering that DPWs havegreat potential for semi-automated triangulation, based on a similar approach to thatused to work along strips on analytical plotters (APs), but with the advantages thatloading and reloading images is trivial compared to inserting diapositives on the stageplate of the AP and that image matching can be invoked for very accurate pointtransfer. Kolbl (1996) described the advantages of Intergraph’s ImageStation in this

658

mode. This is important, because analytical triangulation is mature and productive, socustomers place high demands on this aspect of DPWs.

Three vendors, Inpho, Leica-Helava and Zeiss offer highly automated triangula-tion, with their MATCH-AT, HATS and PHODIS-AT products. These use entireimages and have superseded the patchwise approach of Helava’s DCCS (DigitalComparator Correlator System). The underlying principles of all three are similar, inthat the operator should have to measure the ground control points in at least oneimage, but all other processes are automatic. HATS (Miller and Walker, 1996)requests the operator to define a pattern of tie points and image matching proceedsaccordingly. The other two products prefer to select very large numbers of tie pointsby means of an interest operator. In all cases, the different levels of the imagepyramids are used to facilitate the process. Should matching fail entirely, the operatorwill be invited to make manual measurements, but if there are sufficient successfulpoints then failures can simply be discarded without compromising the block. HATShas its own built in block adjustment, but all three products can output imageco-ordinates formatted for the most popular bundle adjustment suites. Kersten andO’Sullivan (1996) is one of several papers which confirm that automated triangula-tion requires further development, such as the incorporation of GPS data, but isundoubtedly operational. Although huge data volumes are handled, modern hard disktechnology has fallen in cost sufficiently that workstations with up to 100 GBcapacity are in service at several sites.

A couple of related aspects of DPWs are often overlooked as being rather trivial,but represent applications of automation. After triangulation, it is effortless to loadimages and their orientation data: no diapositives have to be placed on stage platesand no interior orientation need be repeated; after triangulation, no more work isrequired to prepare images for further processing. Indeed, in the Leica-Helava system,it is possible to define stereomodels as pairs of already triangulated images and thenswitch between them with a mouse click in a couple of seconds. This is veryconvenient for temporal studies using photography captured at different epochs.Secondly, the transfer of parameters between DPWs and other systems is possibletoo, for example to enable images triangulated on a DPW to be set up easily on anAP. This can be done rigorously if the mathematical models are available, or it ispossible simply to pass image and ground co-ordinates and solve the model again onthe recipient system.

Digital Terrain Models

The generation of digital terrain models (DTMs) by image matching has alwaysbeen recognized as a major advantage of digital over analogue or analytical photo-grammetry. Although some form of matching technology was incorporated into a fewof the earlier instruments, the ready availability of complete images in digital form isalmost a prerequisite for methods which are to be successful in production. Equally,the problems encountered by automatic systems with large scale imagery showingtrees or buildings have become one of the frustrations of digital photogrammetry. Thesolution is a compromise, whereby good matching algorithms are supplemented byquality control information and powerful interactive editing functions. Many vendorsoffer DTM generation and, as in automated triangulation, the approaches may bedichotomized into area and feature based matching, the latter preferred by the Germancompanies. Some pundits, commenting on the success of automation in digitaltriangulation, have construed DTM generation as an area of failure, but others mightjudge one million DTM points per hour, most of them without gross errors, as quitegood. These high speeds enable the user both to experiment with different matchingstrategies according to terrain characteristics and to select dense post spacings foraccurate modelling on the ground surface. Experiments at the Helava offices haveprovided evidence that the results may be applied to change detection, for example tomonitor urban development or measure forest growth (Carsonet al., 1996). Thisauthor contends that DTMs represent the fullest, most successful implementation ofautomation in digital photogrammetry.

659

Orthophotographs, Mosaics, Image Maps and Perspective Scenes

The computation of digital orthophotographs is not difficult, it must be said.Given a digital image, a DTM and a set of orientation parameters, the algorithms toproduce an orthophotograph by, for example, nearest neighbour, bilinear or bicubicinterpolation are not too challenging. Moreover, DPWs usually offer several functionsfor image enhancement so that the final orthophotograph is at least as pleasing inappearance as one from an analytical device such as the Zeiss Orthocomp Z2. Indeed,not only the mainstream photogrammetric vendors but also a host of remote sensingand other software houses offer such functionality. Yet the fact that it is easy does notmean it is not successful automation. Moreover, current striking rates of a fewseconds per megabyte of the output orthophotograph are rather impressive. Recently,attention has been devoted to the lack of published standards for digital orthophoto-graphs, surely a sign that the technological problems have been solved. Somevendors, such as Leica-Helava, do a little better in the sense that a single orthophoto-graph is produced to cover the user’s choice of area, but from a set of input images,using a nearest nadir or some other suitable selection criterion. This allows wide areacoverage, most important in view of the increasing use of this type of product asraster layers in GIS systems.

One or two vendors also offer “true orthophotographs”. The effects of “buildinglean”, such as the covering of streets or other detail by buildings, the appearance ofthe sides of tall buildings or the incorrect size of roofs, which appear in theorthophotograph as well as the untreated aerial photograph, are corrected, providedthat the roof lines of buldings are available for the computation as well as the “bareearth” DTM.

The greater challenge is mosaicking. Whether mosaics are made by generatingorthophotographs and then connecting them by means of geometric and radiometricfeathering, or by producing a single output orthophotograph from multiple inputimages as described above, it is not trivial to balance the radiometry and perfect thefeathering such that the seamlines are invisible. Many vendors have some level ofautomation in the tackling of this process, but few would lay claim to total success.This is an area, furthermore, where users demand rigorous standards and speed is notsufficient reason for aesthetic compromise.

Automation has been fairly successful in the production of “walk throughs”or “fly throughs” consisting of animations obtained by fast viewing of series ofperspective scenes or “bird’s eye views”. These are computed by calculating theview that would be seen by the user if he were located in a particular position.As in a “true orthophotograph”, the components are a series of images, one or moreDTMs and a “feature database” consisting of the rooflines of buildings or otherstructures in the scene. Whereas many of us have been impressed with the tankcommander’s views of East Berlin produced by the US Defense Mapping Agencyduring the latter years of the Cold War, a classic application nowadays is in planningenquiries, for which the appearance of a new road or building in the landscapecan be simulated and assessed. The remote sensing vendors, such as PCI and EarthResource Mapping, offer this kind of functionality, as do some of the photogram-metric vendors, such as Leica-Helava and VirtuoZo. GDE Systems, Helava’s parentcompany, work with Evans and Sutherland to produce the RapidScene product, inwhich the rendering of the perspective view with fragments of imagery, chosenoptimally from the selection of images available, is done in real time, by sophisti-cated software and/or high end graphics accelerators developed for the simulationmarketplace.

Feature Extraction

Gargantuan research efforts for at least a generation have centred on automatedextraction of features from imagery. Such a task may be exemplified by “generateroof and ground lines for all the buildings in this stereopair”. The reality, however,is perhaps the greatest disappointment in the story of automation in digital photo-grammetry; Dowman (op. cit.) is but one of many commentators to have made this

660

point. Feature extraction, or stereocompilation, is the core activity of photogram-metric mapping, at which analogue and analytical systems excelled. Thus it is the areawhere digital photogrammetry must provide added benefits, yet it was not the primarygoal during the development of DPWs in the 1980s. Productive feature extraction isdemanding of the photogrammetrist in terms of accurate measurement and thuscauses fatigue; automation would reduce this and would enable less skilled personnelto be allocated to the task, freeing experienced photogrammetrists for other parts ofthe mapping process. Before we turn to the status of automation, however, let usremember that model setting and resetting are faster on DPWs than analyticalinstruments, as noted earlier, that the range of software packages for compilation isnow almost as wide on DPWs as on analytical plotters and that DPWs offer colourstereosuperimposition as a standard feature, in contrast to the expensive option itusually represents on the analytical instruments.

In practice, few vendors include much automated feature extraction in theircurrent products. Probably Leica-Helava is the leader, mainly because this vendor hasbeen practical and has admitted the limitations of fully automated extraction, prefer-ring to implement a few tools for semi-automated feature extraction (SAFE). Thephilosophy here parallels the one behind the DTM generation: full automation is acounsel of perfection for the moment, but useful tools to assist the operator arepracticable, within a process which is controlled by a human. The most robust tool inthe current version of SOCET SET enables the operator to place the measuringmark approximately near the corners of a building; the tool then finds the cornersand completes the building, in monoscopy or stereoscopy, in the latter case withaverage or different heights for the corners as required. Tests indicate increases inproductivity of two to four times in densely urban areas, accompanied by a reductionin fatigue of which the user was clearly conscious. Behind the scenes, more advanceddevelopment is in progress of tools capable of finding most of the buildings in adevelopment or suburb (Mueller and Olson, 1993 and 1995). The other tools in thecurrent product are a line follower and a region builder, but both are less robust thanthe building tool and must be used intelligently by the operator. Some users, such asTrinder and Li (1995), are working on more sophisticated tools in the same vein. Itis generally agreed that worthwhile success in this area is a necessity for digitalphotogrammetry to supersede analytical as the workhorse for compilation, as distinctfrom DTMs or orthophotographs, but at present productization lags significantlybehind research.

Project Management

Digital photogrammetry raises interesting challenges of its own. The user mustcope with digital images, on which a whole variety of processes may be in progress,sequentially or simultaneously as appropriate. Photogrammetrists, scanners andDPWs are major investments. The data volumes are massive and there is a host ofdigital images, image pyramids, epipolar rectified images, ground control files, GPSfiles, image point files, DTMs, orthophotographs, mosaics, and so on. As one thrusttowards increased convenience and productivity, the vendors have added morefacilities for batch processing in response to users’ demands, enabling non-interactiveoperations to proceed unsupervised without the need to complete menu boxes at everystage, yet easy to use editors to set up batch jobs are not found in every DPW. Moreambitious software to manage all these activities, both interactive and batch, and datais obviously useful. Leica-Helava, for example, have footprint tools and log files totry to monitor the progress of tasks in a user friendly manner. Vision Internationalhave tools to catalogue the imagery. Most vendors have something in this area. Butit is easy to envisage more. Software from the project management world, tuned fordigital photogrammetry, would be a godsend if it could allocate resources (people,scanners, DPWs, software licences, output devices, bureau services, for example) insuch a way as to optimize the flowline from raw imagery to output products.Experiences in the military world suggest that this is not easy.

661

TABLE I. An assessment of the achievements of the three photogrammetries.

Analogue Analytical Digital

Primary data acquisition a a dScanning n/a n/a bOrientation g b aTriangulation g a bDTMs g b aImage products d g aFeature extraction a a bProject management d d g

Overall b a b 1

Comment Complete Mature Developing

OUTPUT

Whether by automated or manual processes, a host of outputs emerges fromdigital photogrammetry, most commonly image mosaics and vector data files. Oftenthese must be merged and further refined into the final products. Again, there is scopefor automation. Many vendors include some functionality for map composition intotheir standard products, for example Earth Resource Mapping, ERDAS, Leica-Helava, Vision International and Zeiss. There are specialist products too, for exampleACE (Advanced Cartographic Environment) from Les Services CartoGraphiques21 1 and Mercator from Barco. The requirements can vary from modest titles, gridsand marginalia to sophisticated page and colour handling for map sheet or atlasproduction. In many cases, the result will be a merged raster and vector product in aformat such as PostScript, ready for onward transmission to the vast range of plottingdevices now available. Here again automation is critical, with further hardware,firmware or software from the plotter manufacturers provided to direct the photo-grammetric data to the output device, often an inkjet or laser raster plotter. The latter,for example, often come with raster image processors (RIPs) to translate fromPostScript or similar data into the final patterns of laser dots imprinted on the filmmedia. In order to take full advantage of the high resolution of these devices, thephotogrammetric vendors do well to output the vectors at higher resolution than theraster backdrop. Automation is essential, because it is neither practical nor economi-cal for the photogrammetrist to be intimately involved with the minutiae of theseprocesses.

THE FUTURE

So, where art thou, automation? We have examined the various stages of thedigital photogrammetric process in a practical way. We could summarize our findingsin Table I, which is no more than a heuristic view by the current author.

In the final column, the rating could be interpreted in two ways. The obviousassessment is the success of the automation, that is to what extent is it operational?A more subtle judgement of the same thing is the proportion of the vendors who havemanaged to include it in their current products. The two correspond well. It is evidentthat digital photogrammetry is successful in scanning and orientation, but less well sofar in triangulation. The generation of DTMs is a success story (it is churlish to brandthe provision of interactive editing as a failure) and digital orthophotographs may beregarded as a completely automated process. Mosaics are widely offered but completesuccess is rather subtle and difficult. Walk throughs and fly throughs work wellenough but are not yet popular outside the worlds of simulation and mission planning.Automation plays a useful role, too, on the output side.

More development effort must be applied, however, in the areas of featureextraction and project management. Several vendors have something useful in one orboth of these areas, but neither as yet enable the human operator to take a supervisory

662

role away from the small details. Progress has been made, but the promise of digitalphotogrammetry has not yet been fulfilled. Few experts doubt, however, that theintensive development work which is in progress will bear fruit and the resulting toolswill bring increasing advantages in productivity.

What will the future bring? It is not difficult to envisage a very high degree ofsuccess in most of the processes. Scanning, interior orientation, triangulation, DTMs,orthophotographs and mosaics will soon be combined into an automated flowline,with human intervention only in formulating batch commands, measuring the imageco-ordinates of ground control points and correcting matching errors in triangulationor DTM generation. True orthophotographs and fly throughs are refinements whichwill be available in an easy way to those users who need them. On the other hand,it is not easy to imagine equal success with feature extraction and project manage-ment. In the former case, however, good SAFE tools can certainly offer desirableimprovements in productivity and in the latter the line manager’s task can be muchsimplified, albeit with products which will fall short of complete workflow manage-ment for some time to come.

This assessment is not controversial and echoes the views of many writers,including academics, vendors and users. The author finds it a little surprising thatseveral eminent authorities have found the progress to be so disappointing. Cynicscould argue, one supposes, that stereocompilation on DPWs is the same as onanalogue or analytical systems, but with less sharp images in less attractive colours.To do so is wilfully to ignore not only the widespread success of automation inorientation and triangulation but also the much broader range of products, most ofthem highly automated, expecially DTMs and orthomosaics, which DPWs can createso much more effectively than their predecessors. A DPW is more than an analyticalplotter with digital images and should be assessed accordingly. Overall, therefore,automation is operational and makes a useful contribution to the photogrammetricprocess. Work is in progress to increase its effectiveness in the very near future.

REFERENCES

ACKERMANN, F., 1977. Some thoughts on the future of photogrammetry.Photogrammetric Record, 9(50):147–155.

ACKERMANN, F., 1992. Strukturwandel in der Photogrammetrie. Zeitschrift fur Photogrammetrie undFernerkundung, 60(1): 2–5.

ANON., 1995. GIM interviews Professor Fritz Ackermann, retired professor, Stuttgart University.GeomaticsInfo Magazine, 9(10): 40–45.

ANON., 1996. Facing the ‘soft’ future: GIM interviews Mr Patrick Wong, President of ISM. Ibid., 10(1):53–57.

CARSON, W. W., MILLER, S. B. and WALKER, A. S., 1996. Automated forest inventory using a digitalphotogrammetric workstation.Proceedings of the Second International Airborne Remote SensingConference and Exhibition, 3. 841 pages: 251–257.

DOWMAN, I., 1996. Digital photogrammetry—time for decision.Surveying World, 4(2): 5.KERSTEN, T. and O’SULLIVAN , W., 1996. Digital aerial triangulation with the Helava Automated Triangula-

tion System.OEEPE Official Publication33: 139–150.KERSTEN, T. P. and STALLMANN , D., 1995. Experiences with semi-automatic aerotriangulation on digital

photogrammetric stations.SPIE ProceedingsVolume 2646: 77–88.KOLBL, O., 1996. An overview on commercial software products for digital aerial triangulation.OEEPE

Official Publication33: 125–138.LEBERL, F., 1991. The promise of softcopy photogrammetry.Digital photogrammetric systems(Edited by

H. Ebner, D. Fritsch, and C. Heipke). Wichmann, Karlsruhe. 344 pages: 3–14.MILLER, S. B. and WALKER, A. S., 1996. Aerial triangulation—automation ups productivity!Surveying

World, 4(2): 23–25.MUELLER, W. J. and OLSON, J. A., 1993. Model-based feature extraction.SPIE ProceedingsVolume 1944:

263–272.MUELLER, W. J. and OLSON, J. A., 1995. Automatic matching of 3-D models to imagery.Automatic

extraction of man-made objects from aerial and space images(Eds. A. Gruen, O. Kuebler, and P.Agouris). Birkhauser Verlag, Basel. 321 pages: 43–52.

TRINDER, J. C. and LI, H., 1995. Semi-automatic feature extraction by snakes.Ibid.: 95–104.

Resume

La revolution que promet la photogramme´trie numerique depend del’emploi reussi et extensif de l’automatisation dans de nombreuses phasesdu processus de production.

663

Or les progres de l’automatisation sont ine´gaux. Les came´rasnumeriques ne sont pas d’un emploi largement re´pandu. Des scanneursadaptes aux films en bobines et incluant l’orientation automatique existent,mais le reglage permettant d’avoir le meilleur rendu des tonalite´s resteinteractif. Si l’automatisation est largement introduite dans de nombreusesoperations des stations de travail de photogramme´trie numerique, tellesque la confection de mode`les nume´riques du terrain, d’orthophotographies,de mosaı¨ques et de vues perspectives, on constate en revanche pourl’extraction des objets, de re´els retards de la production sur les re´sultatsreussis de la recherche. Toutefois, en pratique, des progre`s significatifsexistent pour une automatisation qui assiste l’ope´rateur humain sans leremplacer, et des exemples tire´s des syste`mes Leica-Helava sont fournis.

Zusammenfassung

Die durch die Digitalphotogrammetrie versprochene Revolution ha¨ngtvon der extensiven, erfolgreichen Automation wa¨hrend vieler Stufen desProduktionsprozesses ab. Der Fortschritt der Automation ist ungleich-maßig. Digitale Kameras sind noch nicht genu¨gend in Anwendung. Esexistieren Scanner, die den Transport von Rollfilm und die automatischeOrientierung ermo¨glichen, aber die Einstellung fu¨r die optimale Tonwer-tubertragung bleibt interaktiv. Die extensive Automation einigerOperationen auf digitalen photogrammetrischen Arbeitsstationen, wie dieErzeugung von digitalen Gela¨ndemodellen, Orthophotos, Bildmosaiks undperspektiven Ansichten steht im Gegensatz zur Merkmalsextraktion, wo dieeffektive Produktion hinter dem Erfolg in der Forschung zuru¨ckbleibt. Es istbedeutsam, den Fortschritt in Richtung praktischer Automation zu unter-stutzen, ohne dabei jedoch den menschlichen Auswerter zu ersetzen. Dazuwerden Beispiele von Leica-Helava-Systemen gegeben.

664