Knowledge-Based Vision-Guided Robots

Studies in Fuzziness and Soft Computing

Editor-in-chief Prof. Janusz Kacprzyk Systems Research Institute Polish Academy of Sciences ul. Newelska 6 01-447 Warsaw, Poland E-mail: kacprzyk@ibspan.waw.pl http://www.springer.de/cgi-bin/search_book.pl?series = 2941

Further volumes of this series can be found at our homepage.

Vol. 83. S. Barro and R. Marin (Eds.) Fuzzy Logic in Medicine. 2002 ISBN 3-7908-1429-6

Vol. 84. L. C. Jain and J. Kacprzyk (Eds.) New Learning Paradigms in Soft Computing, 2002 ISBN 3-7908-1436-9

Vol. 85. D. Rutkowska Neuro-Fuzzy Architectures and Hybrid Learning. 2002 ISBN 3-7908-1438-5

Vol. 86. M. B. Gorzalczany Computational Intelligence Systems and Applications. 2002 ISBN 3-7908-1439-3

Vol. 87. C. Bertoluzza, M. A. Gil and D. A. Ralescu (Eds.) Statistical Modeling, Analysis and Management of' Fuzzy Data, 2002 ISBN 3-7908-1440-7

Vol. 88. R. P. Srivastava and T. J. Mock (Eds.) Belief Functions in Business Decisions, 2002 ISBN 3-7908-1451-2

Vol. 89. B. Bouchon-Meunier, J. Gutierrez-Rios, L. Magdalena and R.R. Yager (Eds.) Technologies for Constructing Intelligent Systems I, 2002 ISBN 3-7908-1454-7

Vol. 90. B. Bouchon-Meunier, J, Gutierrez-Rios, L. Magdalena and R.R. Yager (Eds.) Technologies for Constructing Intelligent Systems 2, 2002 ISBN 3-7908-1455-5

Vol. 91. J.J. Buckley, E. Eslami and T. Feuring Fuzzy Mathematics in Economics and EnRineering, 2002 ISBN 3-7908-1456-3

Vol. 92. P. P. Angelov Evolving Rule-Based Models, 2002 ISBN 3-7908-1457-1

Vol. 93. V. V. Cross and T. A. Sudkamp Similarity and Compatibility in Fuzzy Set Theory, 2002 ISBN 3-7908-1458-X

Vol. 94. M. MacCrimmon and P. Tillers (Eds.) The Dynamics of Judicial Proof', 2002 ISBN 3-7908-1459-8

Vol. 95. T. Y. Lin. Y. Y. Yao and L. A. Zadeh (Eds.) Data Mining, Rough Sets and Granular Computing, 2002 ISBN 3-7908-1461-X

Vol. 96. M. Schmitt, H.-N. Teodorescu, A. Jain, A. Jain, S. Jain and L. C. Jain (Eds.) Computational Intelligence Processing in Medical Diagnosis, 2002 ISBN 3-7908-1463-6

Vol. 97. T. Calvo, G. Mayor and R. Mesiar (Eds.) Aggregation Operators, 2002 ISBN 3-7908-1468-7

Vol. 98. L. C. Jain, Z. Chen and N. Ichalkaranje (Eds.) Intelligent Agents and Their Applications, 2002 ISBN 3-7908-1469-5

Vol. 99. C. Huang and Y. Shi Towards Efficient Fuzzy Information Processing, 2002 ISBN 3-7908-1475-X

Vol. 100. S.-H. Chen (Ed.) Evolutionary Computation in Economics and Finance, 2002 ISBN 3-7908-1476-8

Vol. 101. S. J. Ovaska and L. M. Sztandera (Eds.) Soft Computing in Industrial Electronics, 2002 ISBN 3-7908-1477-6

Vol. 102. B. Liu Theory and Praxis of Uncertain Programming, 2002 ISBN 3-7908-1490-3

Nick Barnes Zhi-Qiang Liu

Knowledge-Based Vision-Guided Robots

With 99 Figures and 3 Tables

Physica-Verlag A Springer-Verlag Company

Dr. Nick Barnes University of Melbourne Department of Computer Science and Software Engineering 3010 Victoria Australia nmb@cs.mu.oz.au

Professor Zhi-Qiang Liu City University of Hong-Kong School of Creative Media Tat Chee Ave., Kowloon Hong Kong P. R. China zliu@cs.mu.oz.au

ISSN 1434-9922 ISBN 978-3-662-00312-1

Library of Congress Cataloging-in-Publication Data applied for Die Deutsche Bibliothek - CIP-Einheitsaufnahme Barnes, Nick: Knowledge based vision guided robots: with 3 tables / Nick Barnes; Zhi-Qiang Liu. - Hei­delberg; New York: Physica-VerI., 2002

(Studies in fuzziness and soft computing; Vol. 103) ISBN 978-3-662-00312-1 ISBN 978-3-7908-1780-5 (eBook) DOI 10.1007/978-3-7908-1780-5

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Physica-Verlag. Viola­tions are liable for prosecution under the German Copyright Law.

Physica-Verlag Heidelberg New York a member of BertelsmannSpringer Science+Business Media GmbH

© Physica-Verlag Heidelberg 2002 Softcover reprint of the hardcover 1 st edition 2002

The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

Hardcover Design: Erich Kirchner, Heidelberg

SPIN 10876877 88/2202-5 4 3 2 I 0 - Printed on acid-free paper

Acknow ledgements

The authors would like to thank members of technical services of the de­partment, particularly, John Horvath, Andrew Peel, Thomas Weichert, and David Hornsby, for keeping robots and cameras going, despite our best efforts to burn, crash, and destroy them in anyway possible.

Nick Barnes would also like to thank Andrew Howard, Sandy Dance, and Les Kitchen for the challenging research discussions. Thank-you to my family, Betty, Barry, Lisa and Dale for supporting me in this strange enterprise called research. Also, my other family John and Wendy for support, encouragement and Sunday dinners. Finally to Nina, the greatest person whom I know. Her inspiration is a great source of strength to me. Her support of everything that I do makes challenges seem easy. Her company makes problems seem trivial. Her smile makes everything else seem unimportant. Her writing and editing skills have improved this book beyond recognition. Without her help this book would never be completed.

. . . if there were machines which bore a resemblance to our body and imitated our actions as far as it was morally possible to do so, we should always have two very certain tests by which to recognise that, for all that, they were not real men. R. Descarte. circa 1628. [95]1

I possessed the capacity of bestowing animation, yet to pre­pare a frame for the reception of it, with all its intricacies of fibres, muscles, and veins, still remained a work of in­conceivable difficulty and labour. Frankenstein. Mary Shelley, 1818. [201]

. . . to manufacture artificial workers is the same thing as to manufacture motors. The process must be of the simplest, and the product of the best from a practical point of view. R. U. R. (Rossum's Universal Robots). (English transla­tion) Karel Capek, 1923. [39]

lSee conclusion full text.

Contents

1 Introduction 1 1.1 Background .. 2

1.1.1 A vision-guided approach 2 1.1.2 Computer vision and vision-guided mobile robots 3 1.1.3 Applying high-level computer vision to guide mo-

bile robots. 4 1.2 Aims of the Research Presented in this Book: A Problem

in Robot Vision . 4 1.3 The Approach of this Book 5 1.4 About the Chapters 7

2 Related Systems and Ideas 9 2.1 Basic computer vision approaches . 9

2.1.1 Frame-based computer vision 10 2.1.2 Active vision 10

2.2 Vision-Guided Mobile Robot Systems 11 2.2.1 Mobile robot subsystems and concepts 12 2.2.2 Mobile robot object recognition . 18 2.2.3 Maps and path planning . 19 2.2.4 Temporal sequencing for complex tasks 20 2.2.5 Vision-guided mobile robot systems 20 2.2.6 Reactive navigation 22 2.2.7 Model-based vision systems for mobile robots 22 2.2.8 Knowledge-based mobile robotic systems. 23 2.2.9 Vision-guided mobile robots using stereo . 24 2.2.10 Active perception systems for mobile robots . 25 2.2.11 Application of vision-guided mobile robots. 28

2.3 Computer Vision for Mobile Robots 31

x Contents

2.3.1 Traditional model-based vision 3D object recog-

2.3.2 2.3.3

nition ....... . Shape-from-shading Pose determination .

2.4 Conclusion ........ .

32 35 43 43

3 Embodied Vision For Mobile Robots! 45 3.1 Introduction............ 46

3.1.1 Embodiment . . . . . . . . . . 46 3.1.2 Phenomena and noumena . . . 47

3.2 The Classical Computer Vision Paradigm 47 3.2.1 Non-classical computer vision. . . 48

3.3 Problems with Classical Computer Vision 49 3.4 Applying Embodied Concepts in Human Vision. 51

3.4.1 Models play an analogous role in computer vision 52 3.5 Embodiment of Vision-guided Robots ... 53

3.5.1 Embodiment, task and environment 54 3.5.2 The role of the task ...... 55 3.5.3 The role of the environment. . 55

3.6 Embodiment for Vision-guided Robots 55 3.6.1 Physical embodiment ..... 56 3.6.2 Embodiment in a task ..... 3.6.3 Embodiment in an environment.

3.7 Conclusion ............... .

57 58 59

4 Object Recognition Mobile Robot Guidance 63 4.1 Introduction..... 63 4.2 System Perspective . . . 65 4.3 Object Recognition . . . 65

4.3.1 Canonical-views 65 4.3.2 Match verification 73 4.3.3 Edge matching . . 74 4.3.4 Edge-based features for ground-based robots 74 4.3.5 View prediction. . . . . . . . . . . . . 77

4.4 Determining Object Pose and Distance. . . . 78 4.4.1 Active determination of the sign of (J • 82 4.4.2 Error analysis .

4.5 Conclusion .................. . 82 85

Contents

5 Edge Segmentation and Matching 5.1 Edge Extraction ......... .

5.1.1 Edge extraction ...... . 5.1.2 On the choice of window size and quantisation of

XI

87 87 89

p and e . . . . . . . 99 5.2 Edge Matching . . . . . . . 101

5.2.1 Evaluating matches 101 5.2.2 5.2.3 5.2.4 5.2.5 5.2.6 5.2.7

Spatial elimination . Edge coverage. . . . Position estimation consistency Geometric verification . . Quadratic edge extraction Further active processing

103 103 105 105 107 107

6 Knowledge Based Shape from Shading 109 6.1 Introduction................ 110

6.1.1 Motivation and system perspective 111 6.1.2 Assumptions . . . . . . . . . . . . 114 6.1.3 Knowledge-based representation of objects. 115

6.2 Using Object Model Knowledge for Shape-From-Shading 115 6.3 A New Boundary Condition for Shape-From-Shading. 116 6.4 Knowledge-based Implementation. . 120

6.4.1 Knowledge / frame topology 120 6.4.2 Fact knowledge . . . . . . 121 6.4.3 Procedural knowledge . . . 123 6.4.4 Shape processing rulebase . 124

6.5 Experimental Method and Results 6.5.1 Synthetic images . 6.5.2 Real images .... 6.5.3 Domain knowledge

6.6 Conclusion ....... .

124 124 130 131 133

7 Supporting Navigation Components 135 7.1 Model-based Path Planning . . . . . . . . . . . 135

7.1.1 Path planning and obstacle avoidance . 142 7.2 Odometry and Obstacle Avoidance Subsystem 142

7.2.1 Obstacle avoidance strategies 143 7.2.2 Coordinate transforms ......... . 145

XII Contents

8 Fuzzy Control for Active Perceptual Docking 147 8.1 Introduction................... 147

8.1.1 Fuzzy control . . . . . . . . . . . . . . 148 8.1.2 Fuzzy control for mobile robot control 150 8.1.3 TSK fuzzy model. . . . . . . . . . . . 151 8.1.4 Visual motion-based approaches to mobile robots

and the docking problem .. 153 8.2 Direction Control for Robot Docking. . . 154

8.2.1 The log-polar camera ....... 154 8.2.2 Docking for a ground-based robot 155 8.2.3 Noise in the input parameter 157

8.3 A Fuzzy Control Scheme. 158 8.4 Results... 159 8.5 Conclusion ........ 161

9 System Results and Case Studies 165 9.1 Evaluation of Components. 165

9.1.1 Experimental setup. . . . 166 9.1.2 View matching . . . . . . 166 9.1.3 Pose determination - power supply 168 9.1.4 Pose determination - model vehicle. 180

9.2 Case Studies ................. 183 9.2.1 Moving around the corner of an object. 183 9.2.2 Distinguishing a particular object among similar

objects. . . . . . . . . . . 186 9.2.3 Docking.......... 186 9.2.4 Object circumnavigation. 188 9.2.5 Obstacle avoidance . 190

9.3 Conclusion 191

10 Conclusion 203 10.1 Limitations of the Research Presented and Future Work 204 10.2 Extended quotation from Descartes ............ 207

Index 231