VLSI FOR ARTIFICIAL INTELLIGENCE

THE KLUWER INTERNATIONAL SERIES IN ENGINEERING AND COMPUTER SCIENCE

VLSI, COMPUTER ARCHITECTURE AND DIGITAL SIGNAL PROCESSING

Other books in the series:

Consulting Editor

Jonathan Allen

Logic Minimization Algorithms jor VLSI Synthesis. R.K. Brayton, O.D. Hachtel, C.T. McMullen, and A.L. Sangiovanni-Vincentelli. ISBN 0-89838-164-9.

Adaptive Filters: Structures, Algorithms, and Applications. M.L. Honig and D.O. Messerschmitt. ISBN 0-89838-163-0.

Introduction to VLSI Silicon Devices: Physics, Technology and Characterization. B. El-Kareh and R.J. Bombard. ISBN 0-89838-210-6.

Latchup in CMOS Technology: The Problem and Its Cure. R.R. Troutman. ISBN 0-89838-215-7.

Digital CMOS Circuit Design. M. Annaratone. ISBN 0-89838-224-6. The Bounding Approach to VLSI Circuit Simulation. C.A. Zukowski.

ISBN 0-89838-176-2. Multi-Level Simulation jor VLSI Design. D.D. Hill and D.R. Coelho.

ISBN 0-89838-184-3. Relaxation Techniques jor the Simulation oj VLSI Circuits. J. White and

A. Sangiovanni-Vincentelli. ISBN 0-89838-1i:S-X. VLSI CAD Tools and Applications. W. Fichtner and M. Morf, editors.

ISBN 0-89838-193-2. A VLSI Architecture jor Concurrent Data Structures. W.J. Dally. ISBN 0-89838-235-1. Yield Simulation jor Integrated Circuits. D.M.H. Walker. ISBN 0-89838-244-0. VLSI Specification, Verification and Synthesis. o. Birtwistle and P.A. Subrahmanyam.

ISBN 0-89838-246-7. Fundamentals oj Computer-Aided Circuit Simulation. W.J. McCalla. ISBN 0-89838-248-3. Serial Data Computation. s.o. Smith and P.B. Denyer. ISBN 0-89838-253-X. Phonological Parsing in Speech Recognition. K.W. Church. ISBN 0-89838-250-5. Simulated Annealing jor VLSI Design. D.F. Wong, H.W. Leong, and C.L. Liu.

ISBN 0-89838-256-4. Polycrystalline Silicon jor Integrated Circuit Applications. T. Kamins.

ISBN 0-89838-259-9. FET Modeling jor Circuit Simulation. D. Divekar. ISBN 0-89838-264-5. VLSI Placement and Global Routing Using Simulated Annealing. C. Sechen.

ISBN 0-89838-281-5. Adaptive Filters and Equalisers. B. Mulgrew, C.F.N. Cowan. ISBN 0-89838-285-8. Computer-Aided Design and VLSI Device Development, Second Edition. K.M. Cham,

S-Y. Dh, J.L. Moll, K. Lee, P. Vande Voorde, D. Chin. ISBN: 0-89838-277-7. Automatic Speech Recognition. K-F. Lee. ISBN 0-89838-296-3. Speech Time-Frequency Representations. M.D. Riley. ISBN 0-89838-298-X A Systolic Array Optimizing Compiler. M.S. Lam. ISBN: 0-89838-300-5. Algorithms and Techniquesjor VLSI Layout Synthesis. D. Hill, D. Shugard, J. Fishburn,

K. Keutzer. ISBN: 0-89838-301-3. Switch-Level Timing Simulation oj MOS VLSI Circuits. V.B. Rao, D.V. Dverhauser,

T.N. Trick, I.N. Hajj. ISBN 0-89838-302-1

VLSI FOR ARTIFICIAL INTELLIGENCE

edited by

Jose G. Delgado-Frias

Department of Electrical Engineering State University of New York at Binghamton

Will R. Moore

Department of Engineering Science University of Oxford

KLUWER ACADEMIC PUBLISHERS BOSTON/DORDRECHT/LONDON

Distributors for North America: Kluwer Academic Publishers 101 Philip Drive Assinippi Park Norwell, Massachusetts 02061 USA

Distributors for the UK and Ireland: Kluwer Academic Publishers Falcon House, Queen Square Lancaster LAI lRN, UNITED KINGDOM

Distributors for all other countries: Kluwer Academic Publishers Group Distribution Centre Post Office Box 322 3300 AH Dordrecht, THE NETHERLANDS

Library of Congress Cataloging-in-Publication Data

VLSI for artificial intelligence.

(The Kluwer international series in engineering and computer science; 68)

Includes bibliographies and index. 1. Artificial intelligence-Data processing.

2. Integrated circuits-Very large scale integration. I. Delgado-Frias, Jose G. II. Moore, Will R. III. Series. Q336.V57 1989 006.3 88-37254 ISBN-13: 978-1-4612-8895-4 e-ISBN-13: 978-1-4613-1619-0

DOl: 10.1007/978-1-4613-1619-0

Copyright © 1989 by Kluwer Academic Publishers Softcover reprint of the hardcover 1 st edition 1989 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, mechanical, photocopying, recording, or other­wise, without the prior written permission of the publisher, Kluwer Academic Publishers, 101 Philip Drive, Assinippi Park, Norwell, Massachusetts 02061.

Contents

List of Contributors viii

Preface ix

Programme Committee x

Prologue xi

1 Prolog Machines 1

1.1 From Low Level Semantic Description of Prolog to Instruction Set and VLSI Design 3 Reinhard Eck

1.2 A 32 Bit Processor for Compiled Prolog 13 Pierluigi Civera, Dante Del Corso, Gianluca Piccinini and Maurizio Zamboni

1.3 CARMEL-I: A VLSI Architecture for Flat Concurrent Prolog 27 Ran Ginosar and Arie Harsat

1.4 VLSI for Parallel Execution of Prolog 38 Jeff Reynolds and Sergio Delgado-Rannauro

2 Functional Programming Oriented Architectures 47

2.1 Supporting Functional and Logic Programming Languages through a Data Parallel VLSI Architecture 49 John O'Donnell

2.2 Translating Declaratively Specified Knowledge and Usage Requirements into a Reconfigurable Machine Richard Frost, Subir Bandyopadhyay and Dimitris Phoukas

3 Garbage Collection

3.1 VLSI-Appropriate Garbage Collection Support Steven Krueger

3.2 A Self-timed Circuit for a Prolog Machine Yves Bekkers, Louis Chevallier, Serge Le Huitouze, Olivier Ridoux and Lucien Ungaro

61

73

75

85

vi Contents

4 Content-Addressable Memory

4.1 VLSI and Rule-Based Systems Peter Kogge, John Oldfield, Mark Brule and Charles Stormon

4.2 Unify with Active Memory Yan Ng, Raymond Glover and Chew-Lye Chng

4.3 The Pattern Addressable Memory: Hardware for Associative Processing Ian Robinson

5 Knowledge Based Systems

5.1 A High Performance Relational Algebraic Processor for Large Knowledge Bases Simon Lavington, Jerome Robinson and Kai- Yau Mok

5.2 A WSI Semantic Network Architecture Jose Delgado-Frias and Will Moore

6 Neural Architectures

6.1 A VLSI Implementation of Multilayered Neural Networks Bernard Faure and Guy Mazare

6.2 A Fully Digital Integrated CMOS Hopfield Network Including the Learning Algorithm Michel Weinfeld

6.3 A Neural Network for 3-D VLSI Accelerator Tadashi Ae and Reiji Aibara

6.4 Shift Invariant Associative Memory Donald Prados and Subhash Kak

7 Digital and Analog VLSI Neural Networks

7.1 VLSI Bit-Serial Neural Networks Zoe Butler, Alan Murray and Anthony Smith

7.2 A New CMOS Architecture for Neural Networks Michel Verleysen, Bruno Sirletti and Paul Jespers

7.3 A Limited-Interconnect, Highly Layered Synthetic Neural Architecture Lex Akers, Mark Walker, David Ferry and Robert Grondin

7.4 VLSI-Design of Associative Networks Ulrich Ruckert and Karl Goser

7.5 Fully-Programmable Analogue VLSI Devices for the Implementation of Neural Networks Alan Murray, Anthony Smith and Lionel Tarassenko

93

95

109

119

131

133

144

157

159

169

179

189

199

201

209

218

227

236

Contents vii

8 Architectures for Neural Computing 245

8.1 Are Special Chips Necessary for Neural Computing? Igor Aleksander

8.2 A VLSI Systolic Array Dedicated to Hopfield Neural Network Frant;ois Blayo and Philippe Hurat

8.3 An Integrated System for Neural Network Simulations Simon Garth and Danny Pike

Index

247

255

265

273

List of Contributors

T Ae, Hiroshima (Japan) R Aibara, Hiroshima (Japan) L A Akers, Arizona State (USA) I Aleksander, Imperial College (UK) S Bandyopadhyay, Windsor (Canada) Y Bekkers, INRIA (France) F Blayo, LGI (France) M Brule, Syracuse (USA) Z F Butler, Edinburgh (UK) L Chevallier, INRIA (France) C L Chng, Nanyang (Taiwan) P Civera, Torino (Italy) J G Delgado-Frias, Oxford (UK) S Delgado-Rannauro, Essex (UK) D Del Corso, Torino (Italy) REck, Erlangen-Nuernberg (FRG) B Faure, IMAG (France) D K Ferry, Arizona State (USA) R A Frost, Windsor (Canada) S Garth, Texas Instruments (UK) R Ginosar, Technion (Israel) R J Glover, Brunei (UK) K Goser, Dortmund (FRG) R 0 Grondin, Arizona State (USA) A Harsat, Technion (Israel) P Hurat, LGI (France) P G A Jespers, Louvain (Belgium) S Kak, Louisiana State (USA)

P M Kogge, IBM (USA) S D Krueger, TI (USA) S H Lavington, Essex (UK) S Le Huitouze, INRIA (France) G Mazare, IMAG (France) K-Y Mok, Essex (UK) W R Moore, Oxford (UK) A F Murray, Edinburgh (UK) Y H Ng, Imperial College (UK) J T O'Donnell, Glasgow (UK) J Oldfield, Syracuse (USA) D Phoukas, Windsor (Canada) D Pike, Cambridge (UK) G L Piccinini, Torino (Italy) D Prados, Louisiana State (USA) J Reynolds, Essex (UK) o Ridoux, INRIA (France) I N Robinson, HP (USA) J Robinson, Essex (UK) U Ruckert, Dortmund (FRG) B Sirletti, Louvain (Belgium) A V W Smith, Edinburgh (UK) C Stormon, Syracuse (USA) L Tarassenko, Oxford (UK) L Ungaro, INRIA (France) M Verleysen, Louvain (Belgium) M R Walker, Arizona State (USA) M Weinfeld, Poly technique (France) M Zamboni, Torino (Italy)

Preface

This book is an edited selection of the papers presented at the International Workshop on VLSI for Artiflcial Intelligence which was held at the University of Oxford in July 1988. Our thanks go to all the contributors and especially to the programme committee for all their hard work. Thanks are also due to the ACM-SIGARCH, the Alvey Directorate, the lEE and the IEEE Computer Society for publicising the event and to Oxford University for their active support. We are particularly grateful to David Cawley and Paula Appleby for coping with the administrative problems.

Jose Delgado-Frias Will Moore

October 1988

Programme Committee

Igor Aleksander, Imperial College (UK) Yves Bekkers, IRISA/INRIA (France) Michael Brady, University of Oxford (UK) Jose Delgado-Frias, University of Oxford (UK) Steven Krueger, Texas Instruments Inc. (USA) Simon Lavington, University of Essex (UK) Will Moore, University of Oxford (UK) Philip Treleaven, University College London (UK) Benjamin Wah, University of Illinois (USA)

Prologue

Research on architectures dedicated to artificial intelligence (AI) processing has been increasing in recent years, since conventional data- or numerically-oriented architec­tures are not able to provide the computational power and/or functionality required. For the time being these architectures have to be implemented in VLSI technology with its inherent constraints on speed, connectivity, fabrication yield and power. This in turn impacts on the effectiveness of the computer architecture.

The aim of this book is to present the state-of-the-art and future trends on VLSI implementations of machines for AI computing. In order to achieve this objective the papers are drawn from a number of research communities spanning the subjects of VLSI design through computer architectures to AI programming and applications.

This book has eight chapters which have been grouped into three major cate­gories: hardware support for artificial intelligence programming languages, computer architectures for knowledge oriented systems, and neural network hardware implemen­tations. This grouping covers the complete range from purely programmable systems to learning systems and from symbolic manipulation to connectionism.

Hardware support for artificial intelligence programming languages

Logic-oriented programming languages -such as Prolog- and functional languages -such as pure Lisp and Miranda- have been widely used as high-level languages for artificial intelligence applications. As a consequence, much research has been car­ried out to develop high performance computers for these programming languages. Chapter 1 contains papers which examine the implementations of Prolog machines. Although, the majority of these machines are based on the Warren abstract machine (WAM), there is a wide range of architectures: from reduced instruction set comput­ers (RISC) to complex instruction set computers (CISC) and from uni-processor to multiprocessor architectures. Chapter 2 presents two functional programming ori­ented VLSI architectures. Chapter 3 looks at hardware support for programming languages to overcome memory limitations. Garbage collection (GC) helps to reclaim memory space that is no longer used by the program. In this chapter two garbage collectors are discussed; the first is for Lisp-like machines and the second for Prolog computers.

Computer architectures for knowledge oriented systems

Knowledge representation and manipulation tasks are frequently required in AI sys­tems. These tasks have inherent parallelism which must be exploited in order to obtain reasonable execution times. Chapter 4 deals with content-addressable mem­ory (CAM) circuits. CAM circuits are useful for applications such as production systems and logic programming. The CAM implementations presented in this chap­ter illustrate the effective use of parallelism. In Chapter 5, two architectures for

xii Prologue

knowledge bases are described. The multiprocessor architectures are based on rela­tional algebraic operations and semantic networks.

Neural network hardware implementations

In recent years many computer scientists have become interested in neural network models. Such models are believed to have a potential for new architectures for com­puting systems; such systems may be able to achieve human-like performance in some fields. Chapter 6 looks at architectural implementations of neural networks which Il re based on the Hopfield model. Chapter 7 presents several digital and analog circuits to implement these networks. The implementations reveal contrasting approaches to exploiting the VLSI capabilities and for overcoming the limitations imposed by this technology. Chapter 8 gives some alternative designs for neural network computations. The computers presented here are not themselves based on a neural network model but they do, through more conventional conventional architec­tures provide high computational power for neural computing applications.

VLSI FOR ARTIFICIAL INTELLIGENCE