Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Principles of 30 Image Analysis and Synthesis
THE KLUWER INTERNATIONAL SERIES IN ENGINEERING AND COMPUTER SCIENCE
PRINCIPLES OF 3D IMAGE ANALYSIS AND SYNTHESIS
EDITED BY
BERND GIROD Information Systems Laboratory Stanford University Stanford, CA, USA
GUNTHER GREINER Lehrstuhl fur Graphische Datenverarbeitung Universitat Erlangen-Nurnberg Erlangen, Germany
HEINRICH NIEMANN Lehrstuhl fOr Mustererkennung Universitat Erlangen-Nurnberg Erlangen, Germany
Springer Science+Business Media, LLC
Library of Congress Cataloging-in-Publication Data
Principles of 3D image analysis and synthesis I edited by Bernd Girod, GUnther Greiner, Heinrich Niemann.
p. em - (The Kluwer international series in engineering and computer science ; SECS 556)
Includes bibliographical references and index.
I. Image processing-digital techniques. 2. Image analysis. I. Girod, Bernd. II. Greiner, GUnther. III. Niemann, Heinrich. IV. Series.
TA1637.P75 2000 621.36'7-dc21
Copyright © 2002 by Springer Science+ Business Media New York
Originally published by Kluwer Academic Publishers in 2002
00-029626
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, mechanical, photo-copying, recording, or otherwise, without the prior written permission of the publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work.
Permissions for books published in Europe: [email protected] Permissions for books published in the United States of America: [email protected]
Printed on acid-free paper.
ISBN 978-1-4419-4982-0 ISBN 978-1-4757-3186-6 (eBook) DOI 10.1007/978-1-4757-3186-6
Contents
Contributing Authors
Preface
1. OPTICAL 3D SENSORS
1.1 2D Image Acquisition for 'Perfect' 3D Sensors
1.2 3D Sensors-Principles, Potentials and Limitations
2. MULTIPLE VIEWS AND IMAGE SEQUENCES
2.1 Coordinate Systems and Camera Models
2.2 Motion Models
2.3 Structure From Multiple Views
2.4 Object Tracking in Image Sequences
2.5 Illumination Estimation
3. RECOGNITION AND INTERPRETATION
3.1 Segmentation
3.2 Semantic Models
3.3 Statistical Models
3.4 Object Recognition
3.5 Image Understanding
3.6 Active Vision
4. REPRESENTATION AND PROCESSING OF SURFACE DATA
4.1 Polygon Meshes
4.2 Feature Extraction and Registration
4.3 Discrete Modeling of Point Clouds
4.4 Fusion of Discrete Models
4.5 Splines
4.6 Fitting Free Form Surfaces
vii
xi
2
5
25
26
30
38
57
67
79
80
93
101
108
122
131
141
142
153
166
175
181
192
vi PRINCIPLES OF 3D IMAGE ANALYSIS AND SYNTHESIS
5. REALISTIC RENDERING
5.1 Models for Illumination and Reflection
5.2 Global Illumination
5.3 Texture Models
5.4 Image-Based Rendering
6. VOLUME VISUALIZATION
6.1 Volume Data
6.2 Volume Models
6.3 Data Acquisition
6.4 Visualization of Scalar Volume Data
6.5 Visualization of Vector Fields
7. ACOUSTIC IMAGING, RENDERING, AND LOCALIZATION
7.1 3D Acoustical Imaging
7.2 Aurealization
7.3 Fusion of Multisensor Data
7.4 Object Localization Using Audio and Video Signals
8. SELECTED APPLICATIONS
8.1 Digitizing 3D Objects for Reverse Engineering and Virtual Reality
8.2 Surface Interrogation in Automotive Design
8.3 Fluid Dynamics
8.4 Diagnosis Support of Patients with Facial Paresis
8.5 Surgical Planning
8.6 Image Communication
References
Index
203
204
213
223
232
243
244
245
252
261
270
279
280
295
309
322
335
336
347
353
365
378
389
399
457
Contributing Authors
Michael Breuer Lehrstuhl fiir Stromungsmechanik, Universitat Erlangen-Ni.irnberg
Swen Campagna Lehrstuhl fi.ir Graphische Datenverarbeitung, Universitat Erlangen-Ni.irnberg
Katja Daubert AG 4 Computergraphik, Max-Planck-Institut fi.ir Informatik, Saarbri.icken
Joachim Denzler Lehrstuhl fi.ir Mustererkennung, Universitat Erlangen-Ni.irnberg
Peter Eisert Lehrstuhl fiir Nachrichtentechnik, Universitat Erlangen-Ni.irnberg
Thomas Ertl Visualisierung und Interaktive Systeme, Universitat Stuttgart
Arnd Gebhard Lehrstuhl fi.ir Mustererkennung, Universitat Erlangen-Ni.irnberg
Bernd Girod Information Systems Laboratory, Stanford University
Sabine Girod Department of Functional Restoration, Stanford University School of Medicine
Gunther Greiner Lehrstuhl fi.ir Graphische Datenverarbeitung, Universitat Erlangen-Ni.irnberg
Peter Hastreiter Lehrstuhl fi.ir Graphische Datenverarbeitung, Universitat Erlangen-Ni.irnberg
viii PRINCIPLES OF 30 IMAGE ANALYSIS AND SYNTHESIS
Gerd Hausler Lehrstuhl fi.ir Optik, Universitat Erlangen-Ni.imberg
Wolfgang Heidrich AG 4 Computergraphik, Max-Pianck-Institut fi.ir Informatik, Saarbri.icken
Benno Heigl Lehrstuhl fi.ir Mustererkennung, Universitat Erlangen-Ni.imberg
Matthias Hopf Visualisierung und Interaktive Systeme, Universitat Stuttgart
KaiHormann Lehrstuhl fi.ir Graphische Datenverarbeitung, Universitat Erlangen-Ni.imberg
Manfred Kaltenbacher Lehrstuhl fiir Sensorik, Universitat Erlangen-Niimberg
Stefan Karbacher Lehrstuhl fiir Optik, Universitat Erlangen-Ni.imberg
Xavier Laboureux Lehrstuhl fi.ir Optik, Universitat Erlangen-Niimberg
Hermann Landes Lehrstuhl fiir Sensorik, Universitat Erlangen-Niimberg
Reinhard Lerch Lehrstuhl fiir Sensorik, Universitat Erlangen-Ni.imberg
Marcus Magnor Lehrstuhl fi.ir Nachrichtentechnik, Universitat Erlangen-Niimberg
Heinrich Niemann Lehrstuhl fiir Mustererkennung, Universitat Erlangen-Niimberg
Dietrich Paulus Lehrstuhl fiir Mustererkennung, Universitat Erlangen-Niimberg
Rudolf Rabenstein Lehrstuhl fi.ir Nachrichtentechnik, Universitat Erlangen-Niimberg
Frank Schafer Lehrstuhl fiir Stromungsmechanik, Universitat Erlangen-Niimberg
CONTRIBUTING AUTHORS ix
Annette Scheel AG 4 Computergraphik, Max-Planck-Institut fiir Informatik, Saarbrticken
Hartmut Schirmacher AG 4 Computergraphik, Max-Planck-Institut ftir Informatik, Saarbrticken
Nikolaus Schon Lehrstuhl fiir Optik, Universitat Erlangen-Ntimberg
Harald Schonfeld Lehrstuhl fiir Optik, Universitat Erlangen-Niimberg
Stephan Seeger Lehrstuhl ftir Optik, Universitat Erlangen-Niimberg
Hans-Peter Seidel AG 4 Computergraphik, Max-Planck-Institut fiir Informatik, Saarbrticken
Marc Stamminger AG 4 Computergraphik, Max-Planck-Institut fiir Informatik, Saarbriicken
Eckehard Steinbach Lehrstuhl fiir Nachrichtentechnik, Universitat Erlangen-Ntimberg
Norbert Strobel Lehrstuhl fiir Nachrichtentechnik, Universitat Erlangen-Niimberg
Matthias Teschner Lehrstuhl ftir Nachrichtentechnik, Universitat Erlangen-Niimberg
Christian Teitzel Lehrstuhl fiir Graphische Datenverarbeitung, Universitat Erlangen-Ntimberg
Lutz Trautmann Lehrstuhl ftir Nachrichtentechnik, Universitat Erlangen-Ntimberg
Christian Vogelgsang Lehrstuhl fiir Graphische Datenverarbeitung, Universitat Erlangen-Niimberg
Riidiger Westermann Visualisierung und Interaktive Systeme, Universitat Stuttgart
Matthias Zobel Lehrstuhl ftir Mustererkennung, Universitat Erlangen-Niimberg
Preface
Traditionally, say 15 years ago, three-dimensional image analysis (aka computer vision) and three-dimensional image synthesis (aka computer graphics) were separate fields. Rarely were expert<; working in one area interested in and aware of the advances in the other field. Over the last decade, this has changed dramatically. Maybe it is a result of the growing maturity of each of these areas that they are less concerned with themselves. Vision and graphics communities are today engaged in a mutually beneficial exchange, learning from each other and coming up with new ideas and techniques that build on the state-of-the-art in both fields. Many of us today believe that we will ultimately have one unified field, that, besides vision and graphics, also might encompass traditional image processing and image communication, and names such as 'Visual Computing', 'Imaging Sciences', or 'Image Systems Engineering' have been proposed.
Without doubt, three-dimensional image analysis and synthesis is very much an application-driven field. The declining cost of processors, memory, and sensors continues to expand the scope of viable applications at a breath-taking speed. New systems solutions are in reach by combining state-of-the-art techniques from vision and graphics. The thorough scientific treatment of the underlying principles, for example, the limitations of sensors, the reliability, accuracy, and complexity of image processing algorithms, the fidelity of interactive visualization schemes, or the appropriate mathematical formulation of the interaction of diverse methods from computer vision and computer graphics, is the prerequisite for such advanced systems.
This book is the result of a most fruitful collaboration between scientists at the University of Erlangen-Ntimberg, Germany, that, coming from diverse fields, are working together propelled by the vision of a unified area of three-dimensional image analysis and synthesis. As a formal framework for this collaboration, we set up the "Graduiertenkolleg Dreidimensionale Bildanalyse und -synthese" (Graduate Research Center 3D Image Analysis and Synthesis) initially, which is being supported by the Deutsche Forschungsgemeinschaft (German Research Foundation) since January 1996. The Graduiertenkolleg comprises a program of research and advanced studies for doctoral students, with special emphasis on problems of 3D image acquisition, computer vision, 3D graphics, and selected applications ranging from medicine to manufacturing. In January 1998, this program was substantially augmented and broad-
xii PRINOPLES OF 3D IMAGE ANALYSIS AND SYNTHESIS
ened by the addition of a "Sonderforschungsbereich (SFB)" (similar to an NSF Center of Excellence in the US). The SFB 603 "Model-bac;ed Analysis and Visualization of Complex Scenes and Sensor Data," again funded by the Deutc;che Forschungsgemeinschaft, complement<; the Graduate Research Center through currently 12 long-term collaborative research projects. These SFB projects address much more ambitious goals, often involving implementation and testing of large systems and investigations with a time-line exceeding five years. Within the SFB, a working group was formed that set out to systematically review and compile the state-of-the-art in image analysis and synthesis, and quickly we realized that publishing the results of our effort in the fonn of a book and thus sharing them with the scientific community might be worthwhile.
It is our long-term vision to develop a theoretically well-founded general methodology for the design of systems for image synthesis and interpretation. This methodology should be based on the formulation of relevant problems ac; optimization problems, the use of model-based techniques, sensor data fusion, and hierarchical processing algorithms and data structures. Models are the backbone of 3D image analysis and synthesis. Modeling 3D scenes is central to computer graphics, as it is for 3D vision techniques. Often, these models are implicitly built into the flow of an algorithm, but more and more we are seeing techniques that keep the model explicit and separate from the algorithm, thus deserving the label 'model-bac;ed' . Initially we considered to organize the contents of this book around the various types of models used, ranging from surface geometry, reflectance and illumination to statistical models or semantic networks. But since many important state-of-the-art techniques are not model-bac;ed in the sense of providing and using an explicit model, we decided to follow a more conventional outline which startc; out at the image acquisition end of a hypothetical processing chain, proceeds with analysis, recognition and interpretation of images, towards the representation scenes by 3D geometry, then back to images via rendering and visualization techniques.
3D image analysis and synthesis systems can process regular camera images, but often optical 3D sensors are a viable and more powerful alternative that can acquire 3D data directly. Chapter 1 discusses the principles, the potential and the limitations of such sensors. Chapter 2 considers the processing of multiples views of a scenes acquired by several camerae; simultaneously or by one camera ac; a video sequence over time. Discussion includes the famous structure-from-motion problem and approaches to incorporate illumination estimation into motion estimation techniques. Chapter 3 deals with image recognition and interpretation, including segmentation, statistical models for recognition and image understanding, and active vision, where interpretation results are fed back to the sensor. Chapter 4 "Representation and Processing of Surface Data" is a central chapter since it interfaces both to traditional image analysis and image synthesis techniques. The discussion includes polygon meshes and their reduction, splines, and techniques for reverse engineering. The next two chapters address topics from computer graphics. Chapter 5 presentc; techniques for synthesizing realistic images. The traditional approach bac;ed on scene geometric descriptions and on models for reflection and illumination is discussed, including texturing techniques and global illumination. In addition, the recent developments in image-based rendering are presented. Scientific visualization, discussed in Chapter 6, would not be possible
PREFACE xiii
without combining image analysis and synthesis techniques, and thus can serve a<> an excellent example for the spirit of this book. It treat<; the state-of-the-art in volume visualization and includes a presentation of flow visualization techniques. We have also included Chapter 7 on Acoustic Imaging, Rendering, and Localization, partly because there are many interesting parallels to the material presented in the previous chapters, partly because we expect to see more applications in the future that can benefit from joint audio-video analysis and rendering. The concluding Chapter 8 is a collection of selected applications that some of us are involved in, ranging from automotive design to surgical planning.
We hope that scientist<>, engineers, graduate students and educators will find this book a useful reference for their work and a readable, but concise introduction to areas that they might be less familiar with. We have included references to the important seminal papers in each area, and thus the book should be a reasonable starting point for in-depth study of a selected problem. There is far too much material to cover in a oneor two-semester course, but individual chapters might be suitable as supplementary reading material in an advanced graduate level course.
The comprehensive treatment of such a variety of research topics wa<; possible only by a close cooperation of many expert<;. This book is the result of a true team effort. As editors, we express our sincere appreciation to all the contributing authors listed above. In addition, our special thanks go to M. Magnor, K. Hormann and C. Vogelgsang for the technical assistance in preparing the manuscript. Finally we are grateful to the Gennan Research Foundation DFG for financially supporting many of the contributors.
Stanford and Erlangen, February 2000
BERND GIROD, GUNTHER GREINER, HEINRICH NIEMANN