J. Fulcher, L. C. Jain (Eds.)

Applied Intelligent Systems

Springer-Verlag Berlin Heidelberg GmbH

Studies in Fuzziness and Soft Computing, Volume 153

Editor-in-chiefProf. Janusz KacprzykSystems Research InstitutePolish Academy of Sciencesul. Newelska 601-447 WarsawPolandE-mail: kacprzyk@ibspan.waw.pl

Further volumes of this series can be found on our homepage:springeronline.com

Vol 134. V.A. NiskanenSoft Computing Methods in Human Sciences, 2004ISBN 3-540-00466-1

Vol. 135. J.J. Buckley Fuzzy Probabilities and Fuzzy Sets for Web Planning, 2004ISBN 3-540-00473-4

Vol. 136. L. Wang (Ed.)Soft Computing in Communications, 2004ISBN 3-540-40575-5

Vol. 137. V. Loia, M. Nikravesh, L.A. Zadeh (Eds.)Fuzzy Logic and the Internet, 2004ISBN 3-540-20180-7

Vol. 138. S. Sirmakessis (Ed.)Text Mining and its Applications, 2004ISBN 3-540-20238-2

Vol. 139. M. Nikravesh, B. Azvine, I. Yager, L.A. Zadeh (Eds.)Enhancing the Power of the Internet, 2004ISBN 3-540-20237-4

Vol. 140. A. Abraham, L.C. Jain, B.J. van der Zwaag (Eds.)Innovations in Intelligent Systems, 2004ISBN 3-540-20265-X

Vol. 141. G.C. Onwubolu, B.V. Babu New Optimzation Techniques in Engineering, 2004ISBN 3-540-20167-X

Vol. 142. M. Nikravesh, L.A. Zadeh, V. Korotkikh (Eds.)Fuzzy Partial Differential Equations and Relational Equations, 2004ISBN 3-540-20322-2

Vol. 143. L. RutkowskiNew Soft Computing Techniques for System Modelling, Pattern Classifi cation and Image Processing, 2004ISBN 3-540-20584-5

Vol. 144. Z. Sun, G.R. FinnieIntelligent Techniques in E-Commerce, 2004ISBN 3-540-20518-7

Vol. 145. J. Gil-AlujaFuzzy Sets in the Management of Uncertainty, 2004ISBN 3-540-20341-9

Vol. 146. J.A. Gámez, S. Moral, A. Salmerón (Eds.) Advances in Bayesian Networks, 2004ISBN 3-540-20876-3

Vol. 147. K. Watanabe, M.M.A. HashemNew Algorithms and their Applications to Evolutionary Robots, 2004ISBN 3-540-20901-8

Vol. 148. C. Martin-Vide, V. Mitrana, G. Paun (Eds.)Formal Languages and Applications, 2004ISBN 3-540-20907-7

Vol. 149. J.J. BuckleyFuzzy Statistics, 2004ISBN 3-540-21084-9

Vol. 150. L. Bull (Ed.)Applications of Learning Classifi er Systems, 2004ISBN 3-540-21109-8

Vol. 151. T. Kowalczyk, E. Pleszczy ska, F. Ruland (Eds.)Grade Models and Methods for Data Analysis, 2004ISBN 3-540-21120-9

Vol. 152. J. Rajapakse, L. Wang (Eds.)Neural Information Processing: Research and Development, 2004ISBN 3-540-21123-3

˘

John FulcherLakhmi C. Jain (Eds.)

Applied IntelligentSystemsNew Directions

123

Professor John FulcherUniversity of Wollongong

School of Information

Technology & Computer Science

2522 Wollongong, NSW

AustraliaE-mail: john@uow.edu.au

ISSN 1434-9922

Library of Congress Cataloging-in-Publication-Data

A catalog record for this book is available from the Library of Congress.Bibliographic information published by Die Deutsche Bibliothek. Die Deutsche Bibliothek lists this publication in the Deutsche Nationalbibliographie; detailed bibliographic data is available in the Internet at http://dnb.ddb.de

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifi cally the rights of translation, reprinting, reuse of illustrations, recitations, broadcasting, reproduction on microfi lm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution under the German Copyright Law.

© Springer-Verlag Berlin Heidelberg

The use of general descriptive names, registered names trademarks, etc. in this publication does not imply, even in the absence of a specifi c statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

Cover design: E. Kirchner, Springer-Verlag, Heidelberg Printed on acid free paper 62/3020/M - 5 4 3 2 1 0

Professor Lakhmi C. JainUniversity of South Australia

Knowledge-Based Intelligent

Engineering Systems Centre

Mawson Lakes

5095 Adelaide

AustraliaE-mail: l.jain@unisa.edu.au

Softcover reprint of the hardcover 1st edition 2004

2004 Originally published by Springer-Verlag Berlin Heidelberg in 2004

ISBN 978-3-642-05942-1 ISBN 978-3-540-39972-8 (eBook) DOI 10.1007/978-3-540-39972-8

Dedicated to

Ione Lewis

Preface

Humans have always been hopeless at predicting the future…most peoplenow generally agree that the margin of viability in prophecy appears to beten years.1 Even sophisticated research endeavours in this arena tend to gooff the rails after a decade or so.2 The computer industry has beenparticularly prone to bold (and often way off the mark) predictions, forexample:

� ‘I think there is a world market for maybe five computers’ Thomas J.Watson, IBM Chairman (1943),

� ‘I have traveled the length and breadth of this country and talked withthe best people, and I can assure you that data processing is a fad thatwon’t last out the year’ Prentice Hall Editor (1957),

� ‘There is no reason why anyone would want a computer in their home’Ken Olsen, founder of DEC (1977) and

� ‘640K ought to be enough for anybody’ Bill Gates, CEO Microsoft (1981).

The field of Artificial Intelligence3 – right from its inception – has beenparticularly plagued by ‘bold prediction syndrome’, and often by leadingpractitioners who should know better. AI has received a lot of bad pressover the decades, and a lot of it deservedly so.4 How often have wegroaned in despair at the latest ‘by the year-20xx, we will all have…(insert your own particular ‘hobby horse’ here – e.g. autonomous robot vacuumcleaners that will absolve us of the need to clean our homes…etc)’ – andthis is to completely ignore the reality that most of the world’s population

1 Davies S (1996) Monitor: Extinguishing Privacy on the InformationSuperhighway, Pan Macmillan, Sydney.

2 Naisbitt J (1982) Megatrends: Ten New Directions Transforming Our Lives, Macdonald, London.

3 whatever that term means (we could present another entire book debating thistopic).

4 Fulcher JA (2001) Practical (Artificial) Intelligence, Invited Keynote Speech, 5th

National Thai Conf Computer Science & Engineering, Chiang Mai, 7-9November: i23-i28.

Preface VIII

does not have internet access5, let alone own a computer6 or telephone7 –indeed 1.3 billion do not have access to clean drinking water8. Obviouslythe ‘we’ of these predictions refers only to a small minority of first-worldcitizens. Don’t these advocates (zealots?) realize the harm that suchmisplaced predictions cause in the longer term? Indeed, their misplacedenthusiasm damages all of us who work in the field.

Decades of spectacular failures and unfulfilled promises can only serveto antagonize the public and get them offside for years to come. Morespecifically, some AI researchers have misjudged the difficulty ofproblems and oversold the prospects of short-term progress based on theinitial results.910 To put it another way, the methods that sufficed fordemonstration on one or two simple examples turned out to fail miserablywhen tried out on wider selections of problems.11

This then leads us to the motivation for the present book. Your Editorsfelt it was timely to take a step back from the precipice, as it were, to pauseand reflect, and rather than indulge in yet more bold predictions, to reporton some intelligent systems that actually work – now (and not at somemythical time in the future).

With this in mind, we invited the authors herein – all internationalexperts in their respective fields – to contribute Chapters on intelligenttechniques that have been tried and proved to work on real-worldproblems. We should also point out that our use of ‘intelligent’ in thiscontext reflects the intelligence of the authors who have created thesevarious techniques, and not on some nebulous ‘ghost in the machine’.

The book commences with Edelman & Davy’s application of GeneticProgramming, Support Vector Machines and Artificial Neural Networks tofinancial market predictions, and from their results they draw conclusionsabout the weak form of the Efficient Markets Hypothesis.

5 http://unstats.un.org/unsd (ITU estimates – e.g. Iceland 60%; Spain 14%; China1.74%; India 0.54%; Somalia 0.01% per 100 population)

6 http://unstats.un.org/unsd (ITU estimates – e.g. USA 57%; Slovenia 27.5%;China 1.59%; India 0.45%; Niger 0.056% per 100 population)

7 http://unstats.un.org/unsd (ITU estimates – e.g. Monaco 147%; Australia 100%;Estonia 75%; Mexico 27%; China 17.76%; India 3.56%; Afghanistan 0.13%per 100 population)

8 http://unstats.un.org/unsd (WHO/UNICEF estimates) 9 Allen J (1998) AI Growing Up; the Changes and Opportunities, AI Magazine,

Winter: 13-23. 10 Hearst M and Hirsh H (2000) AI’s Greatest Trends and Controversies, IEEE

Intelligent Systems, January/February: 8-11. 11 Russell S and Norvig P (1995) AI: A Modern Approach, Prentice Hall,

Englewood Cliffs, NJ.

Preface IX

Chapter 2 covers the Higher Order Neural Network models developedby Zhang & Fulcher. HONNs (and HONN groups) have been successfullyapplied to human face recognition, financial time series modeling andprediction, as well as to satellite weather prediction – it is the latter that isreported on in this volume.

Back demonstrates the power of Independent Component Analysis inChapter 3, and cites examples drawn from biomedical signal processing(ECG), extracting speech from noise, unsupervised classification (non-invasive oil flow monitoring and banknote fraud detection), and financialmarket prediction.

Chapter 4 focuses on the application of AI techniques to regulatoryapplications in health informatics. Copland describes an innovativecombination of Evolutionary Algorithms and Artificial Neural Networkswhich he uses as his primary Data Mining tool when investigatingservicing by medical doctors.

Swarms and their collective intelligence are the subject of the nextchapter. Hendtlass first describes several ant colony optimizationalgorithms, then proceeds to show how they can be applied both to theTravelling SalesPerson problem and sorting (of the iris data set).

In Chapter 6, McKerrow asks the question: ‘Where have all the mobilerobots gone?’, and in the process restores some sanity to counteract somebold predictions by practitioners who should know better. The real-worldcommercial applications covered in this Chapter include robot couriers,vacuum cleaners, lawn mowers, pool cleaners, and people transporters.

Zeleznikow’s expertise with intelligent legal decision support systems isbrought to the fore in Chapter 7, and places this emerging field within anhistorical context. Rule-based reasoners, case-based and hybrid systems,Knowledge Discovery in Databases and web-based systems are allcovered.

Chapter 8 is devoted to Human-Agent teams within hostileenvironments. Sioutis and his co-authors illustrate their ideas within thecontext of the Jack agent shell and interactive 3D games such as UnrealTournament.

The Fuzzy Multivariate Auto-Regression method is the focus of Chapter9. Sisman-Yilmaz and her co-authors show how Fuzzy MAR can beapplied to both Gas Furnace and Interest Rate data.

In Chapter 10, Lozo and his co-authors describe an extension of ART – Selective Attention Adaptive Resonance Theory – and illustrate itsusefulness when applied to distortion-invariant 2D shape recognitionembedded in clutter.

We hope these invited Chapters serve two functions: firstly, presentationof tried and proven ‘intelligent’ techniques, and more especially theparticular application niche(s) in which they have been successfully

X Preface

applied. Secondly, we hope to restore some much needed publicconfidence in a field that has become tarnished by bold, unrealisticpredictions for the future.

Lastly, we would like to thank all our contributing authors who sowillingly took time from their busy schedules to produce the qualityChapters contained herein.

Enjoy your reading.

University of Wollongong John Fulcher University of South Australia Lakhmi C JainSpring 2004

Table of Contents

1 Adaptive Technical Analysis in the Financial Markets UsingMachine Learning: a Statistical ViewDavid Edelman and Pam Davy

1.1 ‘Technical Analysis’ in Finance: a Brief Background 1 1.2 The ‘Moving Windows’ Paradigm 2 1.3 Post-Hoc Performance Assessment 3 1.3.1 The Effect of Dividends 5

1.3.2 Transaction Costs Approximations 6 1.4 Genetic programming 71.5 Support-Vector Machines 10 1.6 Neural Networks 12 1.7 Discussion 14

References 15

2 Higher Order Neural Networks for Satellite WeatherPredictionMing Zhang and John Fulcher

2.1 Introduction 17 2.2 Higher Order Neural Networks 18

2.2.1 Polynomial Higher-Order Neural Networks 20 2.2.2 Trigonometric Higher-Order Neural Networks 23

Output Neurons in THONN Model#1 24 Second Hidden Layer Neurons in THONN Model#1 27 First Hidden Layer Neurons in THONN Model#1 30

2.2.3 Neuron-Adaptive Higher-Order Neural Network 30 2.3 Artificial Neural Network Groups 33

2.3.1 ANN Groups 33 2.3.2 PHONN, THONN & NAHONN Groups 34

2.4 Weather Forecasting & ANNs 35 2.5 HONN Models for Half-hour Rainfall Prediction 36

2.5.1 PT-HONN Model 36 2.5.2 A-PHONN Model 37

XII Table of Contents

2.5.3 M-PHONN Model 38 2.5.4 Satellite Rainfall Estimation Results 38

2.6 ANSER System for Rainfall Estimation 39 2.6.1 ANSER Architecture 40 2.6.2 ANSER Operation 41 2.6.3 Reasoning Network Based on ANN Groups 43 2.6.4 Rainfall Estimation Results 45

2.7 Summary 47 Acknowledgements 47 References 47 Appendix-A Second Hidden Layer (multiply) Neurons 51 Appendix-B First Hidden Layer Neurons 54

3 Independent Component Analysis Andrew Back

3.1 Introduction 59 3.2 Independent Component Analysis Methods 60

3.2.1 Basic Principles and Background 60 3.2.2 Mutual Information Methods 62 3.2.3 InfoMax ICA Algorithm 64 3.2.4 Natural/Relative Gradient Methods 65 3.2.5 Extended InfoMax 66 3.2.6 Adaptive Mutual Information 66 3.2.7 Fixed Point ICA Algorithm 68 3.2.8 Decorrelation and Rotation Methods 69 3.2.9 Comon Decorrelation and Rotation Algorithm 71 3.2.10 Temporal Decorrelation Methods 71 3.2.11 Molgedey and Schuster Temporal Correlation

Algorithm 72 3.2.12 Spatio-temporal ICA Methods 73 3.2.13 Cumulant Tensor Methods 74 3.2.14 Nonlinear Decorrelation Methods 75

3.3 Applications of ICA 75 3.3.1 Guidelines for Applications of ICA 75 3.3.2 Biomedical Signal Processing 76 3.3.3 Extracting Speech from Noise 77 3.3.4 Unsupervised Classification Using ICA 78 3.3.5 Computational Finance 81

3.4 Open Problems for ICA Research 833.5 Summary 85

References 86 Appendix – Selected ICA Resources 95

Table of Contents XIII

4 Regulatory Applications of Artificial Intelligence Howard Copland

4.1 Introduction 97 4.2 Solution Spaces, Data and Mining 98 4.3 Artificial Intelligence in Context 102 4.4 Anomaly Detection: ANNs for Prediction/Classification 104

4.4.1 Training to Classify on Spare Data Sets 105 4.4.2 Training to Predict on Dense Data Sets 106 4.4.3 Feature Selection for and Performance of

Anomaly Detection Suites 109 4.4.4 Interpreting Anomalies 113 4.4.5 Other Approaches to Anomaly Detection 115 4.4.6 Variations of BackProp’ ANNs for Use with

Complex Data Sets 120 4.5 Formulating Expert Systems to Identify Common Events

of Interest 121 A Note on the Software 130 Acknowledgements 130 References 130

5 An Introduction to Collective Intelligence Tim Hendtlass

5.1 Collective Intelligence 133 5.1.1 A Simple Example of Stigmergy at Work 134

5.2 The Power of Collective Action 136 5.3 Optimisation 138

5.3.1 Optimisation in General 138 5.3.2 Shades of Optimisation 139 5.3.3 Exploitation versus Exploration 139 5.3.4 Example of Common Optimisation Problems 140

Minimum Path Length 140 Function Optimisation 140 Sorting 141 Multi-Component Optimisation 141

5.4 Ant Colony Optimisation 141 5.4.1 Ant Systems – the Basic Algorithm 143

The Problem with AS 143 5.4.2 Ant Colony Systems 144 5.4.3 Ant Multi-Tour System (AMTS) 145 5.4.4 Limiting the Pheromone Density – the

Max-Min Ant System 145

XIV Table of Contents

5.4.5 An Example: Using Ants to Solve a (simple) TSP 146

5.4.6 Practical Considerations 154 5.4.7 Adding a Local Heuristic 155 5.4.8 Other Uses for ACO 157 5.4.9 Using Ants to Sort 158

An Example of Sorting Using ACO 162 5.5 Particle Swarm Optimisation 165

5.5.1 The Basic Particle Swarm Optimisation Algorithm 165

5.5.2 Limitations of the Basic Algorithm 166 5.5.3 Modifications to the Basic PSO Algorithm 167

Choosing the Position S 167 The Problem of a finite t 168 Aggressively Searching Swarms 168 Adding Memory to Each Particle 169

5.5.4 Performance 170 5.5.5 Solving TSP Problems Using PSO 171

PSO Performance on a TSP 172 5.5.6 Practical Considerations 175 5.5.7 Scalability and Adaptability 176 References 177

6 Where are all the Mobile Robots? Phillip McKerrow

6.1 Introduction 179 6.2 Commercial Applications 181

6.2.1 Robot Couriers 182 6.2.2 Robot Vacuum Cleaners 183 6.2.3 Robot Lawn Mowers 187 6.2.4 Robot Pool Cleaners 189 6.2.5 Robot People Transporter 191 6.2.6 Robot Toys 193 6.2.7 Other Applications 195 6.2.8 Getting a Robot to Market 195 6.2.9 Wheeled Mobile Robot Research 196

6.3 Research Directions 197 6.4 Conclusion 198

A Note on the Figures 199 References 199

Table of Contents XV

7 Building Intelligent Legal Decision Support Systems:Past Practice and Future Challenges

John Zeleznikow

7.1 Introduction 201 7.1.1 Benefits of Legal Decision Support Systems to

the Legal Profession 202 7.1.2 Current Research in AI and Law 204

7.2 Jurisprudential Principles for Developing Intelligent Legal Knowledge-Based Systems 208 7.2.1 Reasoning with Open Texture 209 7.2.2 The Inadequacies of Modelling Law as a

Series of Rules 210 7.2.3 Landmark and Commonplace Cases 211

7.3 Early Legal Decision Support Systems 214 7.3.1 Rule-Based Reasoning 214 7.3.2 Case-Based Reasoning and Hybrid Systems 219 7.3.3 Knowledge Discovery in Legal Databases 222 7.3.4 Evaluation of Legal Knowledge-Based Systems 222 7.3.5 Explanation and Argumentation in Legal

Knowledge-Based Systems 231 7.4 Legal Decision Support on the World Wide Web 233

7.4.1 Legal Knowledge on the WWW 233 7.4.2 Legal Ontologies 234 7.4.3 Negotiation Support Systems 240

7.5 Conclusion 246 Acknowledgements 247 References 247

8 Forming Human-Agent Teams within Hostile Environments Christos Sioutis, Pierre Urlings, Jeffrey Tweedale, and Nikhil Ichalkaranje

8.1 Introduction 255 8.2 Background 256 8.3 Cognitive Engineering 257 8.4 Research Challenge 258

8.4.1 Human-Agent Teaming 258 8.4.2 Agent Learning 260

8.5 The Research Environment 262 8.5.1 The Concept of Situational Awareness 262 8.5.2 The Unreal Tournament Game Platform 263 8.5.3 The Jack Agent 263

XVI Table of Contents

8.6 The Research Application 264 8.6.1 The Human Agent Team 264 8.6.2 The Simulated World Within Unreal Tournament 265 8.6.3 Interacting With Unreal Tournament 267 8.6.4 The Java Extension 268 8.6.5 The Jack Component 269

8.7 Demonstration System 270 8.7.1 Wrapping Behaviours in Capabilities 271 8.7.2 The Exploring Behaviours 272 8.7.3 The Defending Behaviour 273

8.8 Conclusions 275 Acknowledgements 276 References 276

9 Fuzzy Multivariate Auto-Regression Method and itsApplication

N Arzu Sisman-Yilmaz, Ferda N Alpaslan and Lakhmi C Jain

9.1 Introduction 281 9.2 Fuzzy Data Analysis 282

9.2.1 Fuzzy Regression 282 9.2.2 Fuzzy Time Series Analysis 284 9.2.3 Fuzzy Linear Regression (FLR) 285

Basic Definitions 285 Linear Programming Problem 285

9.3 Fuzzy Multivariate Auto-Regression Algorithm 286 Example - Gas Furnace Data Processed by MAR 287

9.3.1 Model Selection 288 9.3.2 Motivation for FLR in Fuzzy MAR 289 9.3.3 Fuzzification of Multivariate Auto-Regression 290 9.3.4 Bayesian Information Criterion in Fuzzy MAR 291 9.3.5 Obtaining a Linear Function for a Variable 292 9.3.6 Processing of Multivariate Data 293

9.4 Experimental Results 295 9.4.1 Experiments with Gas Furnace Data 295 9.4.2 Experiments with Interest Rate Data 296 9.4.3 Discussion of Experimental Results 298

9.5 Conclusions 299 References 299

Table of Contents XVII

10 Selective Attention Adaptive Resonance theory and ObjectRecognition

Peter Lozo, Jason Westmacott, Quoc V Do, Lakhmi C Jain and Lai Wu

10.1 Introduction 301 10.2 Adaptive Resonance Theory (ART) 302

10.2.1 Limitations of ART’s Attentional Subsystem with Cluttered Inputs 303

10.3 Selective Attention Adaptive Resonance Theory 305 10.3.1 Neural Network Implementation of SAART 306

Postsynaptic Cellular Activity 309 Excitatory Postsynaptic Potential 309 Lateral Competition 309 Transmitter Dynamics 310

10.3.2 Translation-invariant 2D Shape Recognition 312 10.4 Conclusions 317

References 318

Index 321