[Marvin Lee Minsky] Computation, Finite and Infini Copy

Prentice-Hall /( ~01 ~Pl ~T~ TIC ~N Series in Automatic Con put a ion

FIN I E AND George Forsythe, editor

i I NFI 'liTE MA CHI" ES;

BATES AND DOUGLAS, Program mingL anguag e/One BAC~ANN, FELICIANO, BAUER, ND SA 11ELSm , Imro 1uction toAL 'iOL BOWLES (editor), Computers in Huma istic l< esearc CESCHINO AND KUNTZMAN, Nu me rica So/uti on ofl ilia/ V a/ue p, obi ems DESMONDE, Computers and Th ir Use DESMONDE, Rea/- Time Data P ocessi gSyst ms: I troduc tory C ncepts EVANS, WALLACE, AND SUTHER LAND, ~imula ion Us ·ng Dig ita/ Co mpute s FORSYTHE AND MOLER, Compu er Sol tiono Linea Algeb aicSy terns MA RVIN L. ~I ~SK y GOLDEN, Fortran IV: Program ning a dCom puling GOLDEN AND LEICHUS, IBM 36 0: Pro ramming an~ Comp ling HARTMANIS AND STEARNS, Afg< braic tructu e Theo yofS quenti /Mac hines Profi ssor l if Ele trica Engi neerit g HULL, Introduction to Computi g Mas achuJ etts I stitu eof echn logy MARTIN, Design of Real-time ( om put rSyst ms MARTIN, Programming Rea/-Ti me Co nputer Systems MINSKY, Computation: Finite a ~d Infi ile Ma chines MOORE, Interval Analysis SCHULTZ, Digila/ Processing: A Syst m Ori nlatio SNYDER, Chebyshev Methods i Numf rica/ A pproxi nation STROUD AND SECREST, Gaussia n Qua~ rature Formu as TRAUB, Iterative Methods fort e So/u tionof Fquati ns VARGA, Matrix Iterative Analy is WILKINSON, Rounding Errors i A/get raic P ocesse ZIEGLER, Shared-Time Data P ocessi g

I l NJVE RSIT't' OF\ I.A. I,;<.: !\RY

~

PRENTICE-HALL INTER,NATION f'\L, IN ., Lor don

PRENTICE-HALL OF AUSTRAL! , PTY LTD., Sydne · ' IREN PRENTICE-HALL OF CANADA, LTD., froront ~ICE HALL, I" c. PRENTICE-HALL OF INDIA PR VATE TD., I ew D< /hi 11\1 INGL PRENTICE-HALL OF JAPAN, I~ c., To kyo EWO po c LIFF , N. J.

~ ~ 75 /!~ r,.

In r. emm Y of HENRY MINSKY

© 1967 by Prentice-Hall, Inc. Englewood Cliffs, N. 1.

All rights reserved. No part of this book may be reproduced in any form or by any means without permission in writin from the publisher.

Current printing (last digit):

10 9 8 7 6 5 4 3 2

Library of Congress Catalo Card No.6 -12342 Printed in the United State! of Arr erica

N

achines have any limitations

ings to cr stallize in the years ahead~ the formal methods that fill this

that theory which is yet to come.

vii

viii PREFACE

We know this is so for several reas ns. computability appears necess In a convergence of ideas r independent attempts to for in this direction; and b) Th and the extraordinarily shor pat fro surprising conclusions give he th ory guarantees it a permanent pl ce in com

THE MAIN GOAL

The main goal of this b ok is to in rodu e th stu ent t the oncept of effective procedure-a te hnic l ide tha has cryst llize onl fai ly recently but already promis s to e as important to th pra tical cult re of modern scientific life as ere t e ide s of eom try, alcul s, o a to s. It is a vital intellectual tool or w rkin wit or t ying o bu ld m dels of intricate, complicated syst ms- e they m nds r en inee ing yste s. Its most obvious applicati n is to c mpu atio and com uter , bu I believe it is equally valuabl for lear hink ng a out iolo ical, syc ological, mathematical, an (es eciall ) p iloso hica que tion . T is claim must seen impossibly pret ntiou , an I ha bet er ex lain it: he theory of effectiveness is us ful ot o ly to pro e thi gs a out omp ex systems, but is also necessar to rove hing abo t pro if itself!

The overall strategy oft e bo k is ery im pl . W will try t the concept of effective pr cedu e fr m su h a arie y of onte malisms, and viewpoints, t at its char cter ill become thor ughl und rstood-in the sense that on can ecog ize i and adap it, a most with ut thinking, wherever and wh neve it is appr pria e. This c ncep has so many different aspects tha one cann t hope to kno the all equ lly well. It appears unexpect dly i asp cts f pr ctical computat"on, inguistic theories, and mathe atic l lo ic. e must k ow orne of th se aspects very thoroughly, a dot ers a leas cas ally, f we ~ret be ble to sense its style of operat on i new situ tions Bo h th text and the problems are designed wit thi pol valence in min e a ew lines much further than m st, b t all these appr ach ing nd intersecting to form a we of eirdl di~ rent but onne ting thre ds. Most fortunately, the theory is almo t entirely elf-c ntai ed, so th t it has no formal mathemati al " rerequisite " be ond a re sona le highschool algebra course. A supe ficial glan e th ough thes pag s m"ght give a bewildering impress on o the se o many co plex and echn cal formalisms, but this is m st d finite y no the case. Th written in the formalisms, it is mer ly a out t em, and required nor expected toe er ha e se n the bef re!

PREFACE ix

Th book has also a sec nd, layf I theme. The many-headed ecti e co ility ften appears in an especially

Uni ersal mac ine r sys em. This character has an g places which we keep trying to ferret

t is asily foun in the p nder us o eration of a modern digital omp ter's rder code or t e gro s, re und nt structure of a FORTRAN r AL OL like rogr· mmi g Ia guag . (I is not important, at least in his b ok, t at y u kn w w at th se a e.) ut it can also lurk within the

t"ny s ruct re o a "com uter" wit tw memory registers or two i stru tion- apes (Cha ter I ), or in th see ingly childish bead-stringing

rimitives f a ost n rma syst m ( hapt r 13), or even in a machine hat ( !most) neither eads nor rites on i s tapes. While there is not

uch pract"cal se in finding t ese 'mini al Universal systems" they upport ou mai goal by aki g us awar of exactly what is essential ather than mere y pra tical or co veni nt in the concept of computation.

Pe haps even mor im orta t, we lear of many different ways in hich grea complexi y of beha ior an a ise from the interactions of

imp! devices, imp! acti ns, s mple desc iptions, or simple concepts. e se also man diff rent ays in wh ch th same complex behavior can

rise rom hat · ppea to b utte ly di eren kinds of interactive systems. T e rea er is prep· red f r bo h of these explorations in Part I, which

diti n int a s aile but essential (and in many ways ry. ith · n the fin it -stat machine area there exists as app aran es o its wn indigenous "central concept"

wee plore fewe oft ese, hough) an almost as surprising an assembly fits wn inimal hi ing p aces. I w sh I could ave aptu ed in the iffere t approaches to the theory a

ense of th diff rent style of t inki g th· t lead the early explorersost, hur h, T ring, Klee cCul och, and the others-to approach

he sa e p oble s with sue diff rent tech ically and esthetically) formal eth ds. The personaliti s ar ob cure because in my attempt to ode nize, simp ify, nd u ify t e res Its I have removed the individual

etail oft e original developm nts. t would be well worth one's time erus ng Davis' ( 1965) collectio just to ain additional personal conact ith the rna ters!

is I· rgely self-contained, given enough ab ut fu ctional notation (even this is

rna hem tical "nduction (reviewed in Chapan sop omo es h ve encountered no difficulty though they ay e more sophisticated mathe-

X PREFACE

matically than most college stude student or computer programmer avin seri imagine educators (even mat ema ician wh cursive function theory") be omin sus ICIO have somehow constructed a imp! fied r dil "leaving out the calculus." Quit the ontr ry: t e calculu was in it! This really is a branch of ath mati s th t ste s d recti non-numerical, logical fou dations. Inde d, a workers are currently attem ting to bri g to ethe of Newton and Leibniz; the resul ing ew s bjec cursive analysis") has barely otte off t a st· rt.

It is worth a few words t try to e plain why class cal analysis plays :;uch a small r le he e. I dev loping a Theor tation we are trying to deal with sy terns com osed of a reat man part , or very intricate structures. lass cal athe atic I me hods can o th s only in very special situati nd t Classically, one is unable to equations, to say nothing of few that under certain special co diti as it were, when the situation gets omp ex in such a wa that the arts f the system can be treated a individu lly a d in epe dent! random this is what happens in Stati tical Ther ody ami the ries. But t mu t be stated, explicitly and e phat cally that this is ju t what d es n t happen when, as in a comp tatio sys em, the st uctu e ha a m re o -ganized, purposeful structur . T e st tistic I an lysis work bea tiful y for things like gases. It wor s for prec· ous l"ttle e se. here simp y is o reason to suppose that as co put tions grow large one will thing non-trivial by trying to "a erag -out' the ffect of The effect of the "conditio al" ( ee Chapt r 10) is to anything like a "conservatio " co cept to ha e a place i heor .

Fortunately, the systems f co put tion ave ther eatu es th t rna e possible some analysis, tho gh o a v ry differe t kin . I stead of t e statistically defined events u ed in phys cs, w use logic 1/y-d fined class s of computations or expressi ns. hey re ti d tog ther, not Y ge metric or energetic properties, but y the r rei tions to si ilar mac lar definitions. We can use achi epa ts so simp e an wit such sim le interactions that we can ap ly t e utt rly t ansp rent Logi of ropo itions, where for an equivale tactual p ysic I rna hine we w uld ave solve hopelessly opaque ana ytic equati ns. hap ers I 5, a d 12 expl in detail how the problem s re uced until this ind f an lysis can made. One could even co plai that the ain resul s are obta ned a most deceptive sort of parlor tr ck: i stea oft ying we study their descriptions! It m st be adm tted hat i

PREFACE xi

etho avo ds en irely too any rob! e de elop ent f the Theo y of

ms, and we can be quite sure that utati n, in the years to come, will to b ck off a little from this sim-be cen erect r01md the prob em o

plifica ion, itho t goi g al the ay b ck. The trouble now is that we do too muc in o e ste .

Th s book has the f I have used earlier versions MI for evera yea s. St den s spe ializing in computer-related can bsor mos oft e rna erial · n a one-term course, since anyone

expe ienc is a! eady subconsciously primed nd n eds nly to see them precisely formu

ith s me higher mathematics courses, I art as a readi g exercise, have them work chin " wi h Pa t II. In any case, I feel that to e int grat d very early somewhere in curriculu ; th material of Parts II. and III

c n come c nsid rabl (wi h th exc ption of Computer Science s uden s). his e itio is a! o de igne for "ndependent reading outside

e cia sroo , and tha is w y I ave iven substantial hints for solving ost of the arde pro I ems

r sour es.

er of problems scattered throughutine: most of them point toward

f them are difficult research

Th rea er is heref re e JOme not to tu n too easily to the solutions; ot un ess a need d id a ha not orne for a day or so. Every such con

has a pri e-1 ss o the xper"ence obtained by solving a new pro !em. Besi es, e en i rea ing t e solutions were enough to the abilit · to olve such prob ems which it is not), one rarely set ich so legant and so accessible to

xii PREFACE

workers who have not had tc clim b OV( r a Ia rge sc t of ~ath( matic al pr -

requisites. Hence, it is an nusu ~lly ~ ood eld or pr act icc in t ainirg

oneself to formalize ideas an d eva luate and c omp re d ifferer t for ~aliz -

tion techniques.

ACKNOWLEDGMENTS

Writing this book requ ired a con idera ble a moun t of time and a co 'ITE ~TS number of sources of suppo t mu t be ackn wled ed. These in cia de t e

RAND Corporation and n1 mere us p< rts o MI , na tnely, the ~ath -matics Department, the Elec rica! Engi [leerir g De partrr ent, heR sear< h

Laboratory of Electronics, he L ncolr Lab prato y, a d Pr oject MAC.

A first draft was written f.vhile the ~uthc r wa a unio Fe! OW If

Harvard's Society of Fellow . I ( xplic "tly w nt tc ackr owle ~ge t e su -

port of Calvin Mooers of th e Zat pr Cc mpa y an ~ Ro f.vena Swar son f

the AFOSR for full-time w iting for < ne p rio d. I ~ ould also like o 1 PH SICAL MACI INES ~NO

thank 1a~es Winslow for t is re< ding and omm ent c fan ~arly man - TH IR AB ~TRAC cou NTERP "RTS 1

script, and James E. Rickets Jm fo man y ing niou sug! estio s lea ~ing o J.( Wha tIs a ~achi pe? the final version. Many te hnic I poi rts a e du Profe f..ianuel

I ~ to sor l.l Abo lltDei nitior s 3

Blum of MIT, who used the manu scrip as c< urse notes con truct d (ar d 1.< Mac hines s Phy ical !I .fodel of At stract Processes 4 solved) many of the problerr s, an con ribut ed m ch tc the mal ersion.

I want to thank my colleagu es, es pecia ly M rtin Davis ,Joh 11 Me ~arthy,

Seymour Papert, Hartley R ogers Jr., Dana Scot , Oli ver S If rid e,,ai d PA RT 0 ""E. INITI f'CHII ~ES Hao Wang, for innumerabl idea in ' nd a out his a Final y, it is

-STA EM ea.

even more obligatory to th; nk n y te< chers espc cially And ew ( jjleas1 n

and George Miller at Harv ard, Profe sors ~och er, I ox, I efshc tz, VI n

2 Neumann, Tucker, and Tul ey a Prin eton and Drs. Hert ert Z im a d FIN TE-ST< TE M CHINS 1 Alexander Joseph in earlier years for he se f:Iuenc e of pppo tun it es th at

led to this volume, and I al 0 th< nk m y wif , Gi< ria, I or he entl us1as m 2.0 Intn ducti1 n II

and encouragement. 2.1 State sand ~ignals 13 2.2 Equi walent His to ries: I ntern I Stat s 15

MA YIN ~INS y 2.3 State -Tran ition lrable and I l>iagra ms 20 2.4 The tate- ransi ionD iagrar fi of a Isolated Machine 23 2.5 State -Tran ition in the Prese nee of External Signals 25 2.6 The 'Mult "plica ion P oblen , : A Problem that

Cam ot be ~olvec by A 11y Fin ite-St te Machine 26 2.7 Prob I ems 27

3 NEI RAL I ETWO RKS.

AU OMA A MA PE UP OF P< RTS 2

3.0 lntn ductic n 32 3.1 The' 'Cells 'of IV cCull och ar d Pitt 33

xiii

xiv CONTENTS CONTENTS XV

3.2 Machines Compo sed of McC lloch Pitts lo/euro ~s 6 7 3.3 Decoders and En oders for Bi nary~ ignals. UNIVER AL TL RING MACH NES 132 Series-Parallel Cc nversi on 6

ring l\ es to< 3.4 Realization of M re Co mplex Stimt lus-R 7 0 Us ngTu achi~ omp te the Values of Functions 132 spon e 7 I Th Uni~ ersal vlachi e as 2 n Inte rpretive Computer Specifications. T e Beh avior fNet With outC cles I 137

3.5 Equivalence of N ural ~ ets \\ ith Fi nite-S ate 7 2 Th Mac ine D escrip tions 138 Machines in Gen( rat 5 7 3 An Exam pie 143

3.6 Universal Sets of ells 8 7 4 Re !narks 144

THE MEMORIES OF EV NTS 8 ui'omATI ONS C F EFF CliVE COM I UTABI ITY: SOME

4 IN FINITE-STATE MACI1 INES 67 P OBLE~ S NO SOL ABLE BY IN TRUC ION-OBEYING MACHINES 146

8 I Th Half ng Pr blem 146 4.0 Introduction 7 olvat thel 148 4.1 The Meaning of a nOut ut Sil nat: our E 18

8 2 Un ility 0 altin1 Prob em xamp es 8 3 Sm lte Re ated l nsolv able I ecisio n Problems

,, 150 4.2 Regular Expressi ~Reg lar Sc ts of~ equer I ', ns an ces 152 84Th Crea iveC aract r oft eUn olvability Argument 4.3 Kleene's Theoren : Fin teAu om at Can ~eco~ nize Only Regular Set ofSe > 8 5 Co sequc nces J or AI orith (ns an Computer Programs: uenc s 9

Th Deb gging Probl 153 4.4 Kleene's Theorerr (Con inued ): An Reg tarE em press on 8 6 No ~-Uns olvab· lity of Indiv dual I fatting Problems 153 Can Be Recogniz( d by~ orne I inite- ~tate vtachi ! 5 e 8 7 Re! ucibi ity of ::>neK indo Unso lvable Problem to Another 154 4.5 Problems (5 8 8 Pro blems 155

9 T E CO 1.\PUTA BLE Rl AL Nl MBER 1 ~ 7

PART TWO. IN FIN TE M A CHI NES 9 I Re iewo the R ea!N mber Syste n 157 9.2 Th( (Turi ng-) C ompu able eat 1'\ umbers 158 9.3 Th( Exist ence o Non Com utabl Real Numbers 159

5 COMPUTABILITY, EFFEC IVE P OCED ~RES, 9.4 Th( Com putab eNun bers, ~hile Countable, AND ALGORITHMS. INF NITE ~A CHI "'ES 103 Ca not B Effe tively Enurr erate! ! 160

9.5 De~ cripti ns an ~Con putal leNu fr!bers 162 5.0 Introduction 1(3 9.6 Pro blems Abou Com putab eNur pbers 167 5.1 The Notion of"E lfectiv Proc dure" 1(4 5.2 Turing's Analysis of Co fr!puta tion P ocess s 1(7 5.3 Turing's Argumer t 1(8

1 0 Tt- E RELJ TION! BE TV\ EEN 5.4 Plan of Part Two 112 TLRING ~ACH NES ~ ND Rl CURSI E FUI CTIONS 169 5.5 Why Study Infinit Mac pines? 114

1( .0 In roduc !On 169 1C.I Ar thme izatio pofT ring ~a chi res 170

6 I( .2 Th e Prin itive- ecur ive F nctio s 174 TURING MACHINES 117 10.3 Th Prot tern o Recu rsion With 5 everal Variables 177

10.4 Th e(Ger era!) ecur ive Ft nctio s 183 6.0 Introduction 117 10.5 To al-Re ursiv e Fun tions and P artial-Recursive 6.1 Some Examples o Turir gMa hines 120 Fu 1ction s: Ter minot ogy ar d The orems 185 6.2 Discussion ofTur ngM chine Effici ncy 128 10.6 Elf ective En urn eratio n ofth e Par tial Recursive Functions 187 6.3 Some Relations B tweer Diffe ent K inds o fTuri ng M2 chine 129 10.7 Co nditio nal Ex pressi :ms;T heMe Carthy Formalism 192

xvi CONTENTS CONTENTS xvii

10.8 Description of ~OID{: utatio ns Us ·ng Li~ t-Stru tures I 5 14 10.9 LISP I 6 'VERY S MPLE ~ASES FOR < OMPU TABILI Y 255

*·I u nivers a! Pro ~ram ~a chi neswi th Two Registers 255

1 1 MODELS SIMILAR 1~.2 u nivers a! Pro ~ram ~achi neswi th One Register 258

TO DIGITAL COMPUT RS 199 1~.3 G odell' umbt rs 259 14.4 T .vo-Ta peNo n-Wri ingT uring Machines 261

11.0 Introduction 1 9 14.5 u nivers a!No -Eras ingT! ring~ .1achines 262 II. I Program-Mad ines a nd Pr gram 2 0 14.6 T e Pro blem f"Ta g" an Mon ogenic Canonical Systems 267 11.2 Program for a urin! Mac ine 2 4 14.7 u nsolv bility of Po t's "C orres1 ondence Problem" 273 11.3 The Notions o Prog ammi ngLa guag sand Com1 ilers 2 4 14.8 " mall' Univ rsal uring Mach ines 276 11.4 A Simple U niv ersal I ase fo r aPr gram Com puter 2 6 11.5 The Equivalen e ofP rogra nMa< hines with

General-Recur ive F nctio ns 2 8 15 11.6 Replacement o the P redec ssor t ySuc essor andE qualit 2 1 SOLUTI< NS T< SELE TED ROBLE MS 282 11.7 Primitive and< en en IRec rsion Based onR petiti n 2 1 11.8 Survey of our I quiva lence Proof~ 2 5

16 SUGGE~ TIONS FOR URTH R REA DING

.IND D SCRIP OR-IN r>EXED BIBLIC GRAP y 297

PART THREE. SYM BOL-l it\ ANI ~»ULA ION SYST i=MS "ND COlt PUT~ BILITY

TABLE )F SPE CIAL 5 YMBO S J09

12 THE SYMBOL-MANIPlJ LATIO ~

SYSTEMS OF POST 219 IIIIDEX AND < LOSS .I RY 311

12.0 Introduction 219 12.1 Axiomatic Sys ems a nd the Log~ ticM thod 2 I 12.2 Effective Com utabi ity as a Prer equisi e for Proof 2'2 12.3 Proof-Finding Proce dures 2 4 12.4 Post's Product ons. anon ical F orms orRu es of nfere ce 2'6 12.5 Definitions of >rodu tion ndC nonic II Sys em 230 12.6 Canonical Sys ems f r Rep resen ation )fTur ingM achims 232 12.7 Canonical Ext nsion s. Au iliary AI ph bets 235 12.8 Canonical Sys ems f r Pro ram- \1achi nes 237

13 POST'S NORMAL-FOR M THI OREM 24)

13.0 Introduction :<40 13.1 TheNormal-f orm., heore m for

Single-Antece ent P oduc ions :<40 13.2 The Normal-f orm., he ore m for ~ulti le-Ar teced nt

Productions. ~educ ion t< Sing! ~-A xi mSy tern. 149 13.3 A Universal C ~noni a!Sy tern 151

..

1 I HY !iiCJ ,L r tiAC HI" ES AND , HE R J. BS" RA CT COlJ NTERPARTS

I

}

I

1.0 \ '/HAT IS A MAC I- INE?

w hen t he ter m "n a chi e" i~ usee in o rdinary discourse, it tends to evoke an u attra ctive pictu e. It bring to rr ind a big, heavy, complicated bbjec whic his n pisy, reas , and met~ llic; 1 erforms jerky, repetitive, and frlOnO ton01 s mo ions; and pass arp e dges that may hurt one if he does not m ainta n suf cien dista nee.

Tl ere a e rna ny re a sons why "mac hine' or "mechanical" have come o aro use f{ eling ofd stast{ , con empt , and fear. Even today, most of the :nach nery we se is co ncerr ed w th th use of brute power to distort and ransf prm c rude mater ials. ~ost pres{ nt-da y machines really are danger-bus. lJniik e our bodi s, pr pduct ion n achir es are made of large, sturdy barts hat n eed n p sof t shea bing orth ~ir pr ptection. Now while it is un-eces1 ary t o att ibut{ calle usne s or anti I athy to a rolling-mill, it is

Jracti ally "mpo sible to at tribu e an thing more friendly. Perhaps we ven f ar H at an y mo e syr npath etic a ttituc e might lead to a well-inten-ionec but dis as rous embr ace. Ther are occasional exceptions to this eel in . w{ may adm i e in 1 he we rks o fa sr all watch that craftsmanship equir ed to creat ~min iatun s; we may adm i e the quiet competence of a igh-speed bomp ~ter. Neit er o thes are involved in the distortion of

nater als; t ey a e stil I macl ines, howe ver, and not too far to b~ trusted. So meh ow the no tic ns ca lied ' mec hanisrr " and "determinism" always

eem o ge t mixe d up with hese feelir gs al out machines. Most people eliev tha t by t eir n ture , macl ines an de only what they are told, and

klill de it re en tie sly. Derha ps thi is \'. hy it seems to most people incred-i~le t hat it r night be pc ssible to b ~ild r nach inescapable of imaginative or < reati e ac t vity. And man peo ble fe I it e qually preposterous, or even

1

2 PHYSICAL AND ABSTRACT M CHIN S s c. 1.

insolent, to suggest that the theon of n achi es might ontribute o th explanation of the workings of any hing o ex( mpla y as )Ur o .vn m nds. 1

People reg'ularly resent tht allegation that hey r ight be in some sens predictable or predetermined. We f\'ill t~ ke UI som1 of He qut stion clus tered around the idea of "dett rmin sm" ater t>n. l here ·s an< ther, mucl simpler, basis for the feeling Htat it woul be t ndigi ified to be or b like, a machine. With few excepti ns, the m~ chim s tha man has desig ed i the past have not been inte lectu lly challe ging or ii teres ing. Eve where, as computers, they have he ped in problem solvi g, it wast suall a clear-cut matter of econom1 or labor-s ving No great acts pf unfintici pated discovery emerged dire tly frpm machine ope atiot . Juc ged t y pas performance, the machines h ve shown little fpr men to r~spec .

This popular conception of" nachi e" is no longer ar propr· ate since th avalanche of practical and theo etical devel pmen ts that accc mpan"ed the emergence of digital comruters n the 1950' . We are row in mers d in a new technological revolut"on c< ncern d wi h the meet aniza ion o intelectual processes. Today v e hav the beginn·ngs: n achin~s tha play game , machines that learn to pi y ga res; n achines th2t han~le al stractf--nornumerical~mathematical prob ems 2nd dt al wi h ordinary langu ge e -pressions; and we see many ot er ac ivitie formerly confined wit in the province of human intelli1 ence. With"n a g nerat on, I am cc nvinc d, few compartments of intellec will remain. out ide t e rna hine' real n~the problems of creating "arti cia! i tellig nee" will be substantial y solv~d.2

Such matters are not pro{ erly' ithin the s ope fwh tis studie~ her . However, it is important to understand rom the s art Hat otr concern s with questions about the ulti ~ate heon tical apacities nd lifn.itat ons c f machines rather than with the practic~ I engineer·ng a1 alysi of existing mechanical devices.

To make such a theoretical s udy, it is necessary o abstrac away many realistic details and fe, tures of m char ical s~sterr s. Fpr th mo t part, our abstraction is so ru hless that t lea es or ly a kelet bn re reser -tation of the structure of seq ence of ewents insid a mfichin~-a ort c f "symbolic" or "informatiom I" st ucture. We ign pre, i our abstrfictiot , the geometric or physical cc mpo ition of mechar ical arts. We igno e questions about energy. We even shred time into a seq ence of separat , disconnected moments, and we t tally igno e sp~ ce it~ elf! Can such a theory b_e a theory of any "tiling" at all? Ir crediply, i can indet d. E y abstractmg out only what a ~oun to c uesti ns a pout he lc gical cons -quenc~s of certain kinds of ( ause-~ffect reJat ons, ~e cap concentr te Ol r attentiOn sharply and clearh on a few eally fund men a! m tters On< e we have grasped these, we can I ring back to t e pr ctical wotld th · s

ts . upenor numbers refer to notes at th end o each hapter.

SEC. .1 HYSICAL AND ABSTRACT MACHINES 3

unde stan ing, IVhich we C< uld never btair while immersed in inessential detai and distraction

Actual y, th( expc sitior in th sed apters is as concrete and worldly as seem corr patib e wit~ the ideas. All the topics could be handled much more ~rec"sely and thprou~ hly b usir g more formal mathematical representatiOns In f< ct, most o the deas here ppeared first in mathematical p_u_blicatio s. I o no claim that it is 1 ossib le, by sufficiently skillful expoSitiOr, to' clear away all t e mathematics" without any loss. For, as we ~hall see, some )f th~ best ideas about the theory of machines are really mher ntly math mat1 al- r an aboz t rna hematics itself In discussions that oncnn th~ nat~Jre of syn bolic matttematical expressions, for instance, we will deve op bli def nitio s an~ examples the mathematical formfilism necessary lor ou pur oses. But my intention was to make the test access ble e en to read rs wi h no more than good high-school mathemat!< s backgro ~ds, and hav exp icitly marked as optio~al the few sectH ns w ere t 11s goal seemed< ompletely unreasonable. :

1.1 ~Boulr DEF NITIO NS

The te m "n achi e" n ises some eriou s problems of definition. For one thing, an in uitively ac equa e def nitio 1 would have to be very complica ed. This s because "machine' can ot usefully be defined as "a mem :Jer o [a C( rtain class of pl ysica objt cts]." For the decision as to whet per sc meth ng is a m chint dep nds n what that thing is actually used or, a d no just pn its com1 ositic n or tructure.

V hen ~e tal abo~t an achi ewe have ·n mind not only (1) an object of so me sort, bu (2) ~ n ide of" hat t at otject is supposed to do. When is su<? a I ai_r (I, 2) t t10ugl t of as a nach ne? A typewriter or printing pre~s IS co s1dered to be a machine; a :Japerweight is not; neither is a filing cabu et. There is a continuum b twee the nagnifying glass and the automati -scar ning, parfcle-c< untir g mi rose pe: the first certainly is not rega1 ded as a machine, tht latte certfiinly is. One expects a machine to have a. sigf'tificapt nu~ber of " arts' and to perform some reasonably com I lex operation or som thin~, 3 bu it is ifficult to capture in the same defin tion machines which peel appl s and machines which transform code j information si~ nals!

V e oft~n find gre t dif cult in g"ving precise definition to a word from ou~ c rdi~a ry nop-tec nicallang1 age. Over some ranges we are quite sure _wh1cl thmgs belong ~nd ' hich don' ; in other ranges we are uncerta n. 'Ve m y th nk we ha\e a c ear iptuitive notion of what is involv( d, bu ~hen we ry tc defir e it i tern s of precisely specified classes and rope ties, he ur certa"n area cau es tr< uble.

I ord r to nake a def nitio 1 pre ise, harp boundaries must be im-

4 PHYSICAL AND ABSTRACT M CHIN S s c. I. EC. 1.2 P~YSIC L AND ABSTRACT MACHINES 5

posed on something. This fc rces ~s to beco ~e a lvare pf th se a eas i ddin ~mac hine lvorb ," yo ~ sho ld bt able to recognize one. (Of course,

which our intuition itself is '-mcer ain. This is w y fin ding apprc priat Dne n cogn zes g ars, rank S, SCI ews, tc., s things people use to make

definitions is so often the maj pr eff prt in wolve bin c reati' e scit ntific work. !nach ·nes, l ut th ~t isn 't the samt thin . A "Meccano" set is not itself a

If a new definition helps cia sify bjec s wh se st ~tus fyas f Drme ly un- !nach ·ne.) You can even reco~ nize ~ "b oken" adding machine, i.e.,

certain, then some new notic n mu st be invol ed. ~hih on t pe su face~ omet hing whos arr ~nger ent ·s ve y m ~ch like that of an adding

definition is just a conventic n, in ellec ually its a cept~ nee ~ay fiVe ~ !nach ·ne ex ept i r son e rna tter c f det il th~ t will keep it from "working

much more active role. Jrope ly." Obvi ously the jistin tion betw< en "intact" and "broken" is Jne w hich an't Je de ned ully n ter TIS of physical objects-there must

Consider, for instance, the te m "li ing." Whe is an objec alive. HOIV Jean ingre dient of in ent ( r, at least of h istory). One has to have an about viruses, genes, cryst Is, se f-repr oduci g rna hines No one h s been dea o fwha t the hing posec to d ::>,an< , while this idea may involve able to give a definition of 'livin "tha , in st ch qu stion , sati fies s< ientis s ssup

Jnly a al co 1strai nt on the c etails of the material construction, in general. There are ob ects ' hich are cl arly iving, e.g., mice; objec s very gene

his c< nstra intis cruci I. ', which are clearly not, e . . , ro< ks; a d a ow-i1 nport nt ur certai n are Biologists (or rather, biolo gy tea chers) used omak eup r sts like: It does I 't m ch rr atter for exam pie, 111hat the parts of an adding

nach· near actu allyn a de Jf. The ge rs o a digit-wheel adder could be (I) Self-reproducing netal WOO j, or plasti -ju t so ong s the y are able to preserve certain (2) Irritable spec s of heir orm to a uffic ent d egree. If they are too flimsy, the (3) Metabolizing

NA,e nach newi II bet nreli ble, r she rt-liv d, bt t you can still tell that it was (4) Made of "protor lasm, 'or p otein, car be hydra tes, D c.

uppo ed tc be a n ad< ing nach· ne. In ot her machines, e.g., electrical But (l) puts out the mt le, (2) and ( ) the spore whilt if th< se co ditio s ircui s, ev n th< phy ical georr etric) forrr may be quite unimportant;

are dropped, (4) will adm t the frank urter. One can g o on to ext end t e orne ::>ther const raint ·smo e ess ntial list with more careful qua ificati ns, b t que tions emai until the li t gro\ s w hen a part cular 1ine · s des L:ribe to us, we do not first ask

can tr mac

to include special mention of ev rythi ink o . Us ally c ne en s g we ~uesti ons a bout· ts rna terial cons ructi< n. C iven an engineering drawing, up evading the problem b intr< ducin g sucl ex or c pro erties as "s ntien"

eire it di:: gram , a pa tent< escri Jtion -son ething must first convince us or "adaptive," which gi e ris to e qually serio us de iinitio al pr oblems.

Such evasions, while born ofne< essity show little elatio n to ir ventic n. hat" e unc ersta 1d ho wit" orks ·n pri ciple That is, we must see how it s "su ppose d" to wor . w inqt Ire o 1ly Ia ter whether this member will

When an intuitive idea s ::> stea dfast y res sts ac cepta ble p ecise defir i- tand the s ress, or wt ether that oscill tori stable under load, etc. But tion, one must consider the possi iii tie eith r tha t it i not a ver y goc d he id ea of a m chin usu lily c enter aro md some abstract model or idea or else that it is too co npre ensi' e or too q ualitc: tive o ser vein a Jroce s.

technical capacity. Thus, th re re maim toda y !itt e rea on tc be lit ve th at Th ere is a cur ious c ontra st be ween this i jea of a machine and the idea

the ancient notion of living- s.-ina nima e has any urtht r tee nical utili y; fa " theor y ." ~onsi jer s me' theo y" 0 physics, e.g., Newton's me-

there is no reason to expect that harp ning this ld di stinct ion w ill he lp hani s. n is the ory ( Jr an oth r the ory o physics) is supposed to be a

us formul~te good technical ques ions. For in t 1e bo rderr ne ar ea, o 1e ener lizati on ab outs mea spect ofth be he vior of objects in the physical

now needs finer, sharper too s. n e old term mo e fro m wi bin t e tee h- IVOrJd Iftl. e pre ictio s tha com 'fran the heory are not confirmed, then

nical discussions to the outs ide, t 1king succ ssive y the forrr s of chap er assun ing t lzat th e exp erime '1t is 1 mpec able) the theory is to be criticized

headings, book titles, and m mes < fpro fessio ns.4 ndm odifie i, as was '1/ewt n's t heory whe the evidence for relativistic nd q antu mph nom nab cam< cone lusiv . After all, there is only one nive sean j it is 1't th busi ness< fthe phys· cist to censure it, much as he

1.2 MACHINES AS PHYSICA MOt ELS C F ~ight like tp.

ABSTRACT PROCESSES Fe r rna hine~ , the situa ion i inv< rted! The abstract idea of a rna-hine, e.g., n ad ~ing 1 rae hi re, is a spe ifica ion for how a physical object

To recognize an actual n achi1 e, we have to h~ ve so me ie ea of wha it ught to we rk. f the mac ~ine hat I buil< wears out, I censure it and is supposed to do. The san e set of p:: rts c< uld t e arr nged eith< r as ~n erha ps fix it. Ju st as · n ph' sics, he pa rts ar d states of the physical object adding machine or as a m pderr scul pture. Bu if y< u kn ow " how an · re su ppOS( d to< orres pond to th DSe o the bstract concept. But in con-

6 PHYSICAL AND ABSTRACT MJ CHIN! S SIC. l.'

trast to the situation in physic , we riticilze th mataial p~rt of the ~ ysten

when the correspondence bre; ks de wn. Now we can formulate ore of he ce~tral ~uest ons hat ' ill c< ncen

us. How can we be sure, in ar y par icular casf, that then is an wa) at al to build the machine we have ·n mi d? l ow do we now that he pi ysical world will permit it? In som particula case , no easo f1able persc n will have any doubts. Can I realh builc an a~din~ macl ine? Whe we lpok at the structure of a simple adc ing n achi1 e we see 1 hat i sho1 ld wprk; ·t cannot help working-this is ~s cle~r as tis tl at 2 2 = 4.

The cliche is illuminati g. Sc me pepple b~lieve hat si~ple ~athefriatic~ I statements are "self-evider t." 0 her p ople paint~ in tha they are b sed o rather obscure but still em irical obser ations. Either vi w m: y be I eld f< r machines. That a conven ional ~ddin mac ine \1 ill wo k (if brope ly col -structed) seems on the on han< equi alent to th aritl metic prop sitiOI s involved, and on the other hand eems derivf d fron our t mpiri al ex1 erien e about the ways in which objec s beh ve in the v orld. Perh ps ore cou d even maintain the view tha belie in a1 arithmetic taten ent is equiv lent o the belief that certain mac ines, if proberly uilt. fvil/ w rk. hus, I kno~ that the order of summati n in <' ddition is ir eleva t. I c n thi1 k of 1 his as a property of abstract nurr ber, < r as : n empirica gene aliza ion f om e -perience with counting, or as a nece sary roper y of f!ny rr achin~ whi h adds numbers correctly. Of co rse, t ere c n be no be ief or confi ence n any process which forms mpir cal g• neral'zatior s whi h is ot ul imate y based on some sort of a pl iori, ursuppprted, or he ristic proposition

In any case, when a n alizat on of one o our friachi es fa'ls to < o wh t our logic and mathematic predi t ofi, we are qui e sur that either it mu t be defective or broken, o else ' ur lo ic mt st be neon istent An sure y most people would agree 1 hat it woul< be a1 disas rous, and a unth 'nkabl'e, for an adding machine to < rr but not b defe• tive a it w< uld b for , + 2 o occasionally equalS.

Out conclusions about silch q~estic ns w II fa! into two families. n the first part (chapters 2-4) ~e st~dy t~e kipds of rna• hine~ that can pe built of finite numbers of sin pie p~rts. lfhen is nc reas< nabl dou tat II that all these can be physica ly re lized within ea ily e tima ed econorr ic bounds, reliably and practic lly. n lat~r sections we *ow hat t ere t re certain kinds of behavior t at C< nnot be rc alize at II, e en t y idt a! machines with infinite numl ers of par s. T e tht ory 2lso tf lis us quitf a

bit about problems of interr edia e cha acte . There would be little pwfit i giving a more deta led 1= ictun of cur

exploration now. The next ew cl a pte s disc uss t e ca aciti s an< limi ations of the "finite" machir es. J fter hat, f\'e wi I be ble tb res1 me t !lis general discussion of intuiti e ide~s ab ut m~chin s anc thee ries.

EC. 1.2 P~YSIC L AND ABSTRACT MACHINES 7

l~OTE~

For disc~ssion, and thee ry, o why people find it difficult to accept the hypoth~~~ that ~ey a ~mac hmes pf any kind, see Minsky, "Matter, Mind and Mo~els [ 1965 . T IS paper a temp s to explain why the "mind-brain" prop~em is so d fficu.l to th nk cit arly a bout nd proposes some definite ways tot mk about cpnscic usnes , dete mini m, cr ativity, and such matters.

F01 an in rodm tion and su vey o the 1 ew fit ld of "artificial intelligence," see COJ/tputeJ and Thou5ht, a ource book of rei rints of important papers in this fi~l , edit d by eigen~aum and F ldmap [1963]. Computers and Thought contams a Ia ge inc exed piblio raph1 of e rlier fvork in this area. See also the aut or's 1 a per i[l the~ ept. I 966 is ue of ~·cientf{ic American. ·

Th< class cal id~a of "simile rna hine' -lev r, wheel, inclined plane, etc.doe not apturj.: the pirit c f wh<' t is inklolvec in today's machines because it doe n't hclp unc ersta d anything xcept the t1 ansmission of force. We cannot ~xp ain i~ thost term even some parts pf clo kwork, such as the ratchet (an mfc rmattbn-storage device) or the sprin (an erergy-storage device).

These ~in~s ~f ~u~stions cc me UJ cont'nuall' in discussions about computers anc artifi tal m elhge ce: V hat i~ intell gence? What is learning? When can om credi a m chine with solvir g a p obler , and when must one credit its des gner r its rogra mmer. A n ost ir cisive analysis of this kind of question of< efinit on is founc in cl apter I of ottk< 's Elements of Physical Biology [19 6], ~~ iginal y pub ished (in I< 24) w en the question of whether biological me hamslns copld bt acco nted for b ordif!ary physical principles was of mo e seribus cc ncern than t is tc day. Lottk concludes (and the years have cer ainly confi1 med) that here is lit le ha m and perhaps much gain in rec gnizi g our inabi ity to tate <!early the d fference between living and nonlivi[lg rna ter- fvitho t conr mitm nt on the q estion of whether a well-defined difl~rencc can be fo nd. I can[wt rc sist quoting Lottka's citation of Sir Wi liam aylis ' cital ion ol Clau e Bernard'! citation of Poinsot's statement: "If any ore askt d me o defi e tim , I sh uld rc ply: Do you know what it is that yot spea of? f he s~ id Ye , I shc uld s< y, Ve y well, let us talk about it. If he sai No, shou d say, Very ~ell, I t us t lk ab ut something else."

t St the Bibliogr phy f, r com lete d: ta on orks referred to in the text and notes.

PART I FIN ITE ST~ ,TE MA CHI NES

L

2 FIN ITE ST~ ,TE MA4 :HII IilES

.0 IIITROI UCTI )N

In the fi rst P< rt of this t ook Ne wi II be concerned with the structure nd b havi r of he cl ass o mac hines knO\ vn as finite-state machines or nile t utom at a. hese are t e rna hine whi h proceed in clearly separate

'discr ete" ~ teps rom ne tc a no hero fa fi ite number of configurations r· sta es. These high ly id~ alizec mac hines are of special interest for a umb ;!r of eaSOI s.

Be cause of t eir ~ eculi rly I mitec , fin te nature, the structure and Jehav ior o thes ~rna hines IS e sily jescri bed completely, without any mbig uity c r apr: roxin atior . It i muc h har jer to deal with more realistic

node s of mech amca syst ems, tnW ich ariable quantities like time, Jositi Jn, m omen tum, fricti Jn, e C., V<: ry sn oothly over continuous, im-Jerce tibly chan ging ange of alues. Tc analyse such "continuous "

ysten s, on e has to in rodu em<: them a tical abstractions and approxima-ions of all sort in o rder to ge a "' orka :Jle model, and the resulting ysten s of equa ions can't usua ly be solvt d in practice without intra-jucin fur her s mpli catio ns. ~urio usly t nough, it turns out that we jon't alwa s pa a h gh p ice fc r res rictir g our attention to the finite utorr ata- the ' 'digit I" S\ stem as t hey < re sometimes called. These ysten s ar sur risin ly p Jwerf ul, in some respects. And tn certain mpo tant ,vays hey c an b mac e to apprc ximate anything that can be jane by ot her fi nite hysic al sy terns (W will discuss this further in hapt r 5.)

We willd eal wi h the tinite · utom ta in wo ways. In this chapter we use ~ COffii letely abstr· ct for mulat on in whid we treat at once the entire con-

11

12 FINITE-STATE MACHINES SE . 2.0 I

figuration of the machine s a single c uantity (its a tal s ate) 1 hich hang s in accord with the histor~ of he m chine and its en irompent. In t e following chapters we thir k oft e rna< hine ~ s composed of many pm ts, ea h interacting with its neighbors, and lpok tp see what happ< ns, ir deta I, inside the machine. For hese wo vi wpoi ts we use differer t met~ods f description. In the first "total-s ate" ormu ation ~e ne d onl~ describe t e set of possible states and he conditio s wh ch ca se ore stat to ct ange o another. In the second formula ion "'e hav to d scribe preci ely h w ea h part works, and how the parts re in erconjlectec to in uenc one nothc r. To make this simple, we< hoose for o r par s son ever simp e dev ces, t e "neurons" of McCulloch ~nd P tts, a d we event ally <em on trate that t e theory of all finite au tom~ ta is ( quiva ent tc the t eory pf the e par iculatly simple elements.

The outstanding feature of finite-st te m chin s is tjl.e sir[lple elati< n between their structure and heir t ehav or. ( nee lve ha~e ( l the escnption of an automaton, (2) its initial con< ition (state), an~ (3) des ripti< n of the signals that will rea< h it rom ts en~iron fn.ent, we an c leu late what its sta~e ill be at each successive ~om nt. +his i worKed cut by a step-by-st~ .. evelopment of the n achit e's s ccessive st~tes tjuoul h tin e. This calcu tion is such a sin pie n atter that tis a ways easy, in pt incip e, to build physical models wit the arne equerces c f eve[lts. ( f cot rse, 1 e might not be able to affot d a !node large or fast enou h for sor e purposes.

2.0.1 Moments of Time

The theory of finite-state mact mes eals fvith , ime i a n ther ecuWu way. The behavior of a rna< hine s des ribec 'as a simp e, lir ears quence of events in time. These events ccur only at di cretf "me ment "- t etween which nothing happet s. 0 e may imagine these jnomf nts a occt rring regularly like the tick in of a clock, and we id ntify then orne ts with integers-the variable for t me, , tak~::s on only the alues 0, l 2, .... Since there is no particular r[leani[lg to an origin i[l tim , we are g nera ly careless about where t star s, but usuf.tlly \ e wt I taki;: t = 0 to be t~e moment when the machine was fi st pu into open tion.

If one likes, he may think of the machin as o eratir g con inuot sly, a d of our moments as corr spon• ing t the nstan s at 1 hich he tal es a equence of "snapshots" ol the m~chinc 's condition An1 determinat systc m may be viewed in this waw, but it is rot a very useful tl ing t< do u less t~e system meets the conditic ns we will ir pose f-that by the end c f each inter al the machine does, in elf ct, "settle < own" into pne o a finite number of describable states.

SIC. 2.\ FINITE-STATE MACHINES 13

2.1 S*TES ~NO IGNHS

From the: poin of v'ew of the user (c r of he environment), a machine c< n ust ally e regardec as a close~ box with input and output channels. (See F*. 2.1-1.) From time to t me t~e u er ac son hem chin thro~gh t e inr ut cl anne s, an~ fro In tin e to ime he n acl ine a ts 01 the ser throu~ h the: outr ut cl annes. Tl e usf r doe n't norma ly ne~d tc kno' just what reall take: s plac e ins de tl e bm . Th/lt is, unles he i par icula ly ir teres ed in und rstanding the 'works" o the mac! ine, or m mo< ifyin it, he n eds to kn w or ly w at a e its "inp t-

-~~~// ·.~~~~ Input M R channels r/

1 II llr Output channels

Fig. 2.1-1. A "black-box" machine.

o tput' pro ertie . Wl en dealing with machine in this way, we refer to it as a' blad box,' ind'catin our f.!ncor cern ~ith its interior.

We will reat the inbut a d ot tput hant els with similar unconcern. Fpr (it the theo y of auto !nata) we re n b more concerneq with the p ysic: I nat re o the igna s tha pas alor g the channels than we are with th\:! phy icalc omp sitio of tpe m· chine's parts. The signals might, fc r exa[nple, be el ctric pulses, as n a r odern computer. They might be electro hemical in pulsf s, as n ner e fib rs. Wr they might be mechanical ir puis s, lik~ tho e th2 t trar sfer ' earn" infprmation from one wheel to a othe m a add'ng m chin . Wt don' care which, in automata theory.

Wh~t do s co cern us ts the et of distil guishable states that might c arac enze a channel t a given roment. 1\.t each of our discrete mo-n ents ·n time, ea h ch nne! will e fo nd i one or another of a finite n m be of ppssible stales or cond tions Th se states may be given any v riety of sy~bol c nar es, e g., le ters bf thf alphabet, numbers, or even words ike "wes" nd "ro." In n ost c f the examples we will use, each c anne! will be c apab e of just wo possib e states- this ts the usual si uatic n in< omp ter d sign.

An lectri swit h mi1 ht be foun( in 3 n ~~on'' or an ""off'' position. Nomin: lly, th~re ar no o her ppsitio s for a simple switch. The state of a s~ itch c eterm nes, c f cour e, wh~ther ignal~ will go through it- i.e., appear at the o~tput ~fter I eing resen ed at the input. For our purposes such a s itch ould be a wa-st te de ice, ransn itting signals in one state and blockin then in the other state. (The e are lso "switches" with more than t\ o sta es; th~ stanbard j\meri an VHF television set has a twelve-state S\ itch or "p ograt)l" se ectior . At y twt lve symbols could be used to d signa e the tates; the syf11bols in act~al us are the integers 2, 3 ... , 13.)

The assuf11ptic n th: t the num~er of pmsible states of a channel is finite is one< f the key 1 ostulates < f autpmat· theory. Our theory simply

14 FINITE-STATE MACHINES SEC. .1.1

doesn't concern itself wit!" cont"nuoL s or "anal g" d vice (lik( van ble resistors or optical irises) .vhich can oe set to transm t any of an infinite set of fractions of a sigml. Of course, hen one ries o co nstru t a physical realization of a switc om IS I able indt ed c rtaim--to encounter some degree of non-e 1scre e beravio ; me st a tual switches will change their propertie just a !itt e wh~n th handle is pres ed rr ore firmly. But when this hap ens, ve cer sure he S'-' itch (and rot th aut mata theory) if it has any no iceable efft: ct on the operation of the s stem .1

Returning to our black box nachine, suppos that it has just c ne Input and one output channel. et tl ese t e cor nectt:d to ~n er vironment £. (See Fig. 2. I -2.)

s

Fig. 2.1-2. Machine coupled with environment.

\1 e will call the i put hann ::I S nd the utpu channel R (fc r "s im ul 1s"

and "respc nse" . The inp1t, S, is capable o cert in st tes < r "si nals" s 1 , ••• ,

sn an~ the outp t ha pos~ ible tates r1

,

... , m. At eac mo nent the en vir Jnment E de ermir es some input s ate of S as a timulus; the chosen igna will be deno ed b S(t . At each mor11ent he machineS( lects ~orne outp t sta e wh.ch will act in orne .vay on the environmt:nt;

the chosen response at timet will be called f, (t).

Now, in order to describe a lack box nachine more C)rnplttely, we have to _specify how it behaves-r-hov. its c utpu s dq end n it~ inpL ts. What will be the output (state) c f R at a cntain time ? In gene raJ, tpis will depend on the entire pr viou "his ory" of th system. C ne re trictwn we will introduce here is th s--t e ou puts ate at the mom nt t oes, ot depend on the input state at that arne 1me t. Thi is to say hat"' e won't admit the possibility of signals tr versing a Jnit i stan aneo~sly. But he output at time t + I will t suall dep nd, t lea~t in part, on tl e in~ ut signal at the previous moment t.

This restriction reflec s the r hysic· I impossibi ity of instartaneous tra smission from one place tc anot er. It is necessary here or a simple reasc n: to prevent unrealizabler- par doxic ai--d scrip wns of si uatio s when channels are interconnected, and to preve[H infinitely fast t ehavibr- a so physically unrealizable.

2. 1.1 The "momentary respc nse function" of a machine

~magine that a machine ~has been oper· ting n inte ractJ n wi h sorre environment E for some un nowr time Th ~ machine has been n ceivi g

ll

SEC. 2.2 FIN IT -STATE MACHINES 15

sig als f om l. and as be en re pon ing t J the e sig als. Let H(t) denote the histo y of his proces up o the pres<: nt time t. That is, H(t) is suppo ed to desc ibe, in son e wav, the entir recc rd o states of affairs concerring M, frc m tht: time M w sere ted G nd St t intc operation, up to the timet. 1 he hi tory, H(t), of M is supposed als to i elude a record of all the stimtli that hav ente ed M since it W< s sta~ted.

If, at time , we were to disc onne t M from its environment and insert the signal sj, them chin would re~pond (at t me t + I) with some signal rj. J st wl ich si~nal j occ rs at t + would de bend, of course, both on fvhich sign; I s i i cho~en at time t and on t e sta e of affairs inside M at t met. Ass1 ming that hiss ate o affa rs inside A is determined by the history h ( t) of M, t t ~ere n ust br> somr> rei a ion, , oft 'ze form

li(t + 1) = F(H( ), S( ))

~ nfo tuna ely a y sue h rei< tion fvoulc be t o hopelessly cumbersome to c eal ' ith c irectl~. be ause of i s exr licit ~eper dency on the entire history. t wollld b of g eat '<alue f we ould replc: ce it by some much mo e din ct relation bet we en the stirr ulus S(t) < nd tt e response R(t + 1 ). In the ne t sec ion v. e will shov. that by m king a spe ial restriction on the nat re of M-r.ame y th; t the mac ine i~ in wme sense finite-we can obt m su h a r~lation in a very simp! and useftl forrjl.

2.2 EQUIVALEI T HI!TORIES: IN ERNH STATES

or a given mac ine 1~ at give~ tim~ t, VIe car imagine an infinite vari~ty of poss ble h stori s. T e on tha has ctua ly occurred will deterrr ine tl e ma~hine s res~ onse to th next stim J!us. Now it may be that som eve1 ts frc m the very remote pa t rna contribute to determining this resppnse unctipn. I this s the case, one can say tha the machine shows som~ "tn ce," pr "r emo y," c f these re !note event . If every ancient evert left a sep~rate, indebendt: nt tn ce, tt e rna hine would need to have in fir ite capacit , in s Jme sense, o store them.

It is ot r pia 1 to s udy just th kin of n achi e thz t can be made from a finite se · of c iscre e pa ts, st ch a! swit hes nd relays, or cores and tran istor . Th abil ty of uch macpine to sto e inf rmation about past events mt st ha e s01re lin its. No *ch rrachijle co~Jld possibly store a complete ecor of tl e eve 1ts of arbit arily long his tor ies. 2

t I _could be, fo some machine M, that tte output R(t + I) i not in fact determined

by th mput and th histo y. In that c· se we ~ave a "non- eterm nate machine." For examp( • the input ard the histor migl t only deterr ine tl e pro/ abilities of each of the posst le res~onses. In th t case we wo~ld be dealin with "pro~abilistic machine." We mtgh also cpnside non-e etermi ate rr achine whic aren t probabilistic, but, except for certat~ technical ad antag s which won' concern us, here is no rea on to do so here.

16 FINITE-STATE MACHINES

If its memory is limited, has happened to it; it cannotldi!>tinlguishl histories. It is important to fnllchwin,o lclefim\tionslcalref)llly

We need to make p for a machine. Im machine M. But at time M 1 has history H 1(t) and H 1 and H 2 are his·toj•ies subsequent sequence of M 2 would yield the which machine isM 1 gn.-JI "'hi.~~

and observing their res:pqnse:s.

Now define the ''ec1uivalen•l:e histories equivalent to are equivalent to each lent in pairs. Clearly, must be completely se1Ja~ate t"l~ioir.i·rlt"

This brings us to the k We assume that the m behavior, between only These classes will be called designate the internal state and we will call the states thlt~msejves

Evidently, there is a cc:l>nn,t::c~wn

histories and the kinds of In chapter 4 we will exami very interesting that, given lence classes is finite, we structures of such classes.

2.2.1

By our definition of in to S(t) can depend only on things other than the di~timgul~;ha.b\e to put this is to write

tone can imagine, of course, things that are never expressed in have some parts with inputs information, and it might be Cf. Moore [1956]. Our states that have been simplified as far removed."

FINITE-STATE MACHINES 17

nput state at time t and the nal S(t) depends, of course, I state Q(t) depend?

on the whole history a statement which separates

lim•mtjdi12te and the remote past. sting of all the things that it was first put into oper-

red:ei\rerll after that. The history at hil,·•~·-.. '~• ly in having an extra term;

ua111en, the signal S(t). By hypothhappt!n s within the black box.

t + 1? The state Q(t + I) S ( t) and on the previous

d on more than this, or the lbeha·l!rio r, between two histories

class Q(t) and hence indislcot1cl,Jsion by writing

(G)

us as complete a descript, so long as we remain the black box. For, sup

f the machine at a certain (t + 2), etc., of stimuli that nsF and G tell us what outt + I and also the internal

Using this latter fact the output and internal

ca~'cu.lat,(on, step by step, to find the time (so long as we know

of internal states will be been used, e.g., finite-state

P1c,ce.l:srs·,fi•'l~U auitonwtla, h.nitl?-st'dte trd,nsaruq,f!rs, etc. We will now study nntte-st:ate restriction may at

machine we can build can defiqing sufficiently many states.

and S(t) has been stated, there not already included. The logic

mla,tht:m~ttically mature reader can follow it d~finil:iqn histories. A less intricate


2.2.2

Chapter 5 discusses th discuss machines which technical reasons which m of all machines, including negative results, as is usu machines only weakens elusions, so far as the

The idea of a fini state, is usually develo section. Our goal is ult.im~tc:ly that involve a rather ab:stbact introducing part of this alb!;tnHhiion jump (in chapter 4) will

The immediately foil we hasten to give,a more i,u,Jtlllll"'

Imagine that inside our parts. Suppose that each pa positions. Then we can describe simply listing the states (at there are only a finite num number of distinct totals are what we call its "inter capable of K states by i distinct total states.t (The operation might be much the parts, as is the case in ternal states (the possible q t, ... , q P• and the internal

Now, the input and o connect to some parts inside the

Theinputchannelconn input signals may affect the internal state at time t + I at time t and the signals tha

tBecause two of these "total external behavior, the "internal histories than do those defined in what follows, which definition we


parts of the machine so the of those parts. Since the on their states at time t

(F)

Je process (described cornbe many things going on

input signals directly affect mlc,mc:nit these parts affect the states

y be affected at later times. circulate to remote parts of we examine some of these

of things to be said of the

pouring signals into the S e machine may succeed in

ng to the earlier diagrams situation from the point of pretty much the same. The

the communication channels inside the box. If we are -box machine, we have the

ends and its environment definition. The body can be

rain r for the liver. The distinction njJa1chinle and the inessential trim depends on

t gr a system as a whole, we try d¢t:st<m~ each part separately, and a,lso

act. When we make such a treated in turn as the machine

"n'"lir.-.n•~l~.n•. One cannot usefully make l~rh;tr6''" an unnatural division of a sys-

r~2tson:ltble analysis.


A finite-state machine is com des ribed whe we "transition" functions F an G. Eac only a finite number of inp t val be represented as simple ta les. "delay" device whose transition f

EXAMPLE l: A "MEMOR " M

The input channel can tr nsmi just two s 1 , and there are just two inter al s ates, q0 a signals, r0 and r 1:

G F

INPUT So INP T So

s

The tables show that th

EXAMPLE 2: A "PARITY '

G F

0 0 0

0

EC. 2.3

r

s 0 a d outp t

n-ne s) and s 1

The transition tables for this achi e sh t the state (and outp t) remains the same when a 0 is e tere , but cha ges hen l is nter d. Hence any even number of l 's w·ll ca se no net hang of s ate. Thus if we knew that at some (perhaps r mot ) tim in t e pa t the mac ine as in state 0 (say), we could tel at a y later m men whe her t ber of I 's between the first time an the econ wa even or o d. (say) that the machine bega its e istence in state , we coul the total number of I 's that ad e er entered it wa odd or ev n.

s c. 2. FINITE-STATE MACHINES 21

feeble kind of memory. This

Fig. 2.3-2. Parity machine.

rpre ing t ese iagrams: We will always use

urs a the tail of one of the arrows hat will happen, given the state

he h ad points to the subsequent The symbol inserted in the

nal that will occur, as given

free to omit some of the

epres nts del y m chin which remembers the two p evio s digits; it stat at time t, or it out ut at time t + I, tells which s gnal nter d at t me t - 2.

22 FINITE-STATE MACHINES SIC. 2.

o*o G qw qo qlO qll

1_ 0 ql() ql qoo qw

do(,~,$ 4-,:1 1 ql ql qo1 qn

0 10 F qYJ qa1 q1c qll

0 0 0 1 1 Fig. 2.3-3. Two-moment delay

machine. 1 0 0 1 1

One could write, briefly

G(q;1,sk) = qj F(q; ,sk) = i

EXAMPLE 4. A THREE-MO \'lENT DELAY MACHIN

Figure 2.3-4 shows a machine vhich will emen ber hree digits It i easy to remember longer seq ence of digits y increasi g th~ number of states. Of course, the numb r of state will grow rapi ly. rhere is n escape from the requirement that here must be 2n states to emen ber binary digits. (Why' not? Descril e a echo que or m akin~ an -digi memory, and give an argume t for why it neec sat I ast 2 stah s.)

0011

0 ~ ~ //0 '\. 1 ~=~=S 0----<~----1 ·

1'--_.,-1.

0/0 \ i/c , / '"w

01o?f- 1 0~

0

q( 1

Fig. 2.3-4. Three moment dela machine.

EXAMPLE 5. A BINARY S RIAL ADDING MACHH E

A final example of a stat diagram ¥ill b a binary adding rna hine 4

This machine is given for it inp ts t\\O str ngs c f binary djgits, s1mutaneously, where each string represent a number in binar for n with least significant digits first. The utpu signals o this mad ine < re the binary digits of the sequence which repr sents the s Jm o the two n mbe s put into the machine. Beginning t some st< rting morr ent, he machine receives a sequence of pairs of bi :1ary digits (where ea h di it of a pa r belongs to one of the two n umbt rs being f d into the machine). Th s means that there are really f)ur possible inp t sig als, ¥hich we can call 00, 01, 10, II. Only two sta es a e needed; a "nc-carr " st· te q0 and a

I

'

I !

SE . 2.4 FINITE-STATE MACHINES 23

"carry' state qJ. See I ig. 2 3-5.) For xamJle, the sum 45 +57= 102 h< s the binary form

I I I I 0 I + I I 0 0 I

= I I ( 0 I I 0

< r

3 + 0+~ + 4 +0 + 3 + 16 + 1 + 0 + 0

6L + 3 + 0 + ( + 4 + 2

+I +I

+0

I 0

,( -~," !,. u\"1

0 10

00 Carry 0

11 01

Fig. .3-5. Serial binary adding machine.

and wo ld cause tt e sequence of events shown in Table 2.3-1.

Ta le 2.3-

0 I 2 3 4 5 6 7 ...

45 ( I 0 I I 0 t

(0) I 0

57 ( I 0 0 I I ly 0 (0)

sig al II 00 10 II 01 II 00 (00)

sta e qo ql qo qo ql ql ql qo

ou put - 0 I I 0 0 I ly tN te that t e digits fthe bi ary num ers are v ritten ba kwards.

PI OBL r..M 2 3-1. .V rite out th e Fa d G abies for the machine of Fig. 2. -5.

2. THE STA E-TR NSIT ON DIAGR1 M 01 AN ISC LATE MACHINE

Wh · t ha pens if a finite state machine i left to itself- that is, if it re eives no ir formation from the c utsid world? That would be the case if the n achi e receives only a co stan input-that is, a repetitive, unchanging sigr al. In thi case the tate- rans1tion diagram becomes very si nple. Sine there is c nly a singe sig al involved, each state uniquely determines what t e ne t sta e will be. In te ms of the diagram, there is only a single arrov. lead ng fr m earh stc. te. ( f course, it is still possible fo several a rows to le· d int? the same state The diagram must therefoe lock somethi g lik Fig. 2.4-1. Note th· t this is not a collection of exam pi s; it i the iagn m of a single m<Jchine.


Fig. 2.4-1. State diagra of a c rtain i olate mach ne. this is all one machi e-not sever I.

Choose some state for a s art, ma hine into The machine will progress fro sta e to state, chains in a diagram. Now ther are nly number o stat if the machine is run for a long enough ti e, it must even ually re-e ter a state it has previously been in. An this mea s th t it ust, from that time on, continue in a periodi ally repeating Ea~h chain ust eventually lead to a closed loop or 'eye! ." iffere t st rting states may lead into the sam eye! -i.e , di erent merge. But two paths, once me ged ( y en ering a co diverge again.t Hence the st te d agra co tains a fi ite umb r of separate, closed loops or "cy les." Ea h cy le h s, le ding number of distinct "trees" or ergi g se s of aths. There is pass from the tree system of one eye e to t at o anot er. 1

We can conclude from th se o serv tions abo t the stat dia ram, that any finite-state machine, i left ompl tely o its If, wi I fall even ually into a perfectly periodic repetit ·ve pattern. The dura ion f thi repe ting pattern cannot exceed the nu ber f int rna! tates of t e rna hine and could be much less. Of cour e, n he d of the poch

se, fi ite m chines are trivial. But in making such an in erpre ation one hould keep ·n mi very large numbers of state that racti a! m· chine may have. A m dern digital computer may have in it hig -spee "me ory,' the order of a million small parts, each o whi h has two The num er o total states of such a machine is he pr duct not t

tWhy not? 1 We call each such set, consisting of a ycle plus all the tr es att ched o it,

nected) "component" of the diagram The distinc com onents are n cessarily completely detached from one another. (Note th· t if we use a iff ere t "co stant" signal we mi ht get a different set of components-trees a d loo s.)

SEC. 2.5 INITE-STATE MACHINES 25

2.5

arts is possible. his

2.4 1. If the rea er is the operation code of some disc ver t e program with the longest

nning time befor repe ting r halting). He will find that the solu ion w II be, essenfally, pro ram t at tr ats the entire memory, save

r the prog am it elf, a a si gle g gantic accumulator. A fine poin : a c. mmo mist ke is to try to in Jude, in the giant cycle, the states oft e add tiona "har ware' ava !able, e.g., 'ndex registers and accumulato s. This tur s out to b a mi take unles there is a large systematic arra of s ch u its), r th addi ional rogr m required to exploit these regis ers w II con ume ores ates t an a e gain d by this tricky strategy.

ifficu t to ive a gene al de cription of what happens vari ble i puts to t e m chin . One kind of complete athe atic I ter s is given in c apter 4; here we can go a

in t is di ectio by onsi ering the case in which there is only nal s gnat rom he o tside al si nals can nduc sta e transiti ns not permitted in the

stat diagram. Nor ally the machine will be movano her c mp nent f its no-signal, tree-cycle state I sig al c n thr w the rna hine into a state of a difto a dista t par oft e sa e component). Since the bot the s'gnal nd t e old state, the component into

ill de end on the exact state (and not just the

nal leaves the machine in egar ed a mer ly advancing or retarding


that computation. If the co1mppnen~ the computation. A sel,eqtion regarded as storage in the m In the long run, memory canno but only as different componen~s

For example, the parity cotJ):lt,er 2.3-2) has a state diagram wh absence of l signals) divides components. The occurrence ways causes a transition from ponent to the oiher. (See Fig. no l occurs, this machine member" arbitrarily long.

The machine of Fig. 2.3-4 · ence of a constant signal, e.g., 0 has one component, as in Fig. 2.5-2. It ca remember anything from the indeed it can remember no thing that happened more moments ago.

When input signals change occasionally, this analysis components of the state very helpful, because the themselves are only defined to particular constant signal colndlitJ~ns. like those of chapter 4.

The machine of example 5 chine which can add, serially, two The reason that this is po , for is never necessary to "carry" or during addition~a single carry

Multiplication is different · numbers involved, it can amounts of information duri

tExcept for phase information, chronized clock.

syn-


answer. In fact we will

iply arbitrarily large pairs of

could multiply arbitrarily addition example (ex

t quantities are presented first. We will ask M to ultiply the number 2n by

tolli<Dwe:d by n zeros in binary amely 22

n, is a l followed in th product, and only n + 1 that machine has to print n

1. as a machine with

than n states, then, if ause it is in a loop of its

it must go on printuired l. Since for any

n larger than the number tmite1state lmiultJipl·yinli!; machine which will work

problem beyond the reach number of limitations we ue to the machine's finite

limitations that persist finite memory capacity~

of histories represented by is chapter.

For example, one might still bers presented with their digits

the idea of the proof will in


those represented by the i state , although they re cl sely r lated What history is represente by t e ou put 0 of th mac ine i Fig. 2.7.1 Output I?

PROBLEM 2.7-3. Show t sequence:

1 0 1 0 0 1 0 0

Fig. 2.7-1.

circulated by word of mo attra ought to be available in pr'nt. T e pr blem rst a ose i with causing all parts of a self-pr duci hine t bet rned The problem was first solv d by cCa thy a d M now that it is known to ha e a s lutio in logical design or comput r program time of two to four hours. The roble is directly analogous to pr of lo gramming, but it does not epen logical elements or the ins ructi urge those who know a s lutio those who are figuring it o t for this intriguing problem.

"Consider a finite (but e-di state machines, all of whi h are alike excep the machines are called soldi rs, a d on e en General. The machines ar sync rono s, an the tate f eac timet + I depends on the tates f itse f and of its two n ighb rs at ime t

The problem is to specify t e sta es an tran itions of th soldi rs in such way that the General can cause them to g into one artie lar te mina state (i.e., they fire their gu s) all t exa tly th sam time At t e be innin (i.e., t = 0) all the soldiers re as umed to be in a single tate, he qu escen state. When the General under oes t e tr the tate abele 'fire when ready,' he does ot ta e an initi up to the soldiers. The signal can prop gate soldier per unit oftime,.an their probl m is ow t get ll coo dinat d an in rhythm. The tricky par oft e pro !em i that he sa e ki d of oldie

s c. 2. FINITE-STATE MACHINES 29

particular, the soldier with K states uch larger than K. Roughly speak

coun as high as n. I an the soldier farthest from the rent rom the other soldiers in being oth sides of them, but their structure

ting solut on of this problem is to use a rizon al co rdinate representing the spatial inate repre enting time. Within the (i, j)

a sy bol ay b written, indicating the state of isual exam'natio of the pattern of propagation

icate what kinds of signaling must take place

"A y sol tion t the firings uad ynchr nization problem can easily be s own o req ire that the time rom the G neral's order until the guns go o mus be at least n - , whe en is he number of soldiers. Most persons s lve this pro !em i a wa whi h req ires etween 3n and Sn units of time, a thou h occ siona ly oth r sol tions re fo nd. Some such other solutions r quire ~nan of th or de of n unit of ti e, for instance. Until recently, it was not kn wn w at th smal est p ssible time for a solution was. Howe er, this was solve at .I.T. by Pr fesso E. Goto of the University of

okyo. The s lutio obt ined y Go o use a very ingenious construction, ith ea h sol ier having any t ousa ds of states, and the solution required

e actly 2 unit of ti e. In view f the difficulty of obtaining this solu-ore i teres ing p oble for b ginners is to try to obtain some

s een 3 and Sn u its of time, which, as remarked above, is r lative yeas to do."

OTES

I. Co puter speci lists often t lk of disti ction between "digital" and "analog" qua tities, wher a di ita! q antit is o e th t takes on only one of a fixed, finit set o valu s whil an a alog quant ty ha values from a continuum, e.g., the oltag measured cross resis or or capa itor. The term "analog" comes fro thee a in hich ost "com uters' were electrical or mechanical devices desi ned o ap ate t e di renti I eq ations of continuous physical

ld thi k of n analogy between the physical . In this sense, we are deciding finite mathematics rather than In chapter 9 we discuss one way

ote I) and were also to ignore t nois , qu ntum measurement uncertainty, er de ices like the following to have infinite

30 FINITE-STATE MACHINES SEC. ).7 SEC . 2.7 FINITE-STATE MACHINES 31

EPC memory: Su~ pose hat tl:l e devi ce E · n the ace on pany ng .vhere the as erisk hows where "car ies" h ave tc be made. Binary multiplica-figure is a vol age so urce t at ha been for in finite 1 ast ti ne,

' ion ( sed i secti on 2.( ) is d one, ~ !so in anal< gy to decimal arithmetic, by producing a oltag E(t) with he p opert that for e ch epeat d adc ition r, bet er, by shifte d add' tion: integer j, E(t = ei if j < t ~ j + l here 1 is e· ther 0 or I. Suppose al o tha Ran Car such that" henev er E(t ) is l 1011

zero, the charge on C will decay to half it valu in o e tim e unit. Th< n at ny X 5 X 101 - - - =

integer timet 0 , the voltage on C will be giv en by he ex ressic n 5 lOll 00 + oc 00

LE( 0 '- k )rk 1011 I

k~O = 110 Ill

so that by measuring this voltage w 'thin nite 1= recisi n anc expr ssing it as 2 .

real binary fraction, one can recove in fin ite inf ormat on ab out tl:l e pas . We AI nost ny in rodu< tory ext o com puter will give a thorough intro-do not want to admit such a physic al ab urdit' , and SO Wt builc our he or 1 juctio n to b nary~ rithm tic;" e will use it nly fc r examples. upon basically finite ingredients, e tendi g lat r to he in finite quest' on b\ another route. See also the deeper a a lysis inch pter 5.

3. The fact that we can make a dela -or memo ry-n a chin e with only one st ate, namely

G qo F qo ,i

so qo so ro I

S I qo s, ,, I

suggests that there is something u rmatu a! abc ut the way 1\'e def ned t e out put ; function. Indeed, we could refor mulat e the heory so th t the outpu t at t me >"

t + I depends only on the state ~ t tim< t + , or v e cou d defi ne it o dep nd I

only on the state at time t. Tee nical y, the se wo uld le d to the S<J me b sic : results; each has its own slight a vant2 ges ar d dis dvan ages. Our hoice of definition was made so that the c onstr ction of cJ- apter 3 and 4 wo uld cc me out with the least complication.

4. Most readers will be familiar " ith bi nary umbe rs. J st as, in th e deci mal system, a number like 3065 mean

~00+0 X 100 + § _25__!Q +2_ u a number like I 011, in the binary syster , me2 ns .

l

~ 22 + L2s_l' X 2° ' +~ + I ~ ,.

or 8 I II + + =

The rule for additron of binary n umbe s is g' venin detai in cl:l apter 3, sec ion I

3.2.6. It is JUSt like decimal add tion, xcept that hen c ne ha s to a d I I, one has to recognize that the sum is "I J," i.e , Hwr te zer o and carry the o e." Thus

45 I 1101 +57 +I 1001 ---

** * * * 102 I I OliO

3 NEURAL NE W AUTOMATA MA

3.0 INTRODUCTION

RK . E p F AR s

The black-box approach (secti n 2.1 is useful only when we stand a machine's external b havi r co plet ly a d don't c re w at i inside. Otherwise the machi e's structu e ha to detail, i.e., in terms of its p rts understand perfectly each pa t an ho have a chance of understandi g the mac

The machines discussed in this hapt parts. In fact, each part is othi g m machine. The interconnectio s, too, ar small variety or "stock" of th se si pie lent of any other finite-state achi e.

We will use for our parts the e erne ts de elop d by Me Pitts [ 1943] as models for cert in aspects of br in fu

It should be understood de cCul och, present writer considers t ese evices and es to serv as a curat physiological models of erve ells a d tis ues. They were ot d signe with that purpose in min . Th y are desig ed f r the repre entati n an analysis of the logic of sit ations that arise n any discr te pr cess, e it i brain, computer, or anywhere lse. n the ries hich are more s riousl intended to be brain mod Is, th "neu ons" have o be uch ore ompl -cated. The real biological neur n is uch ore ompl x than our simple logical units-for the evo ution of ne ve eel s has led to very intric te an specialized organs.' At this poin , the cCu loch- itts" ells" r "n urons' are quite sufficient for ou purp ses. Our resen goal how, starting with a se of ery s mple elem nts, machines of all sorts.

32

s c. 3.

3 1

a

a b

a b c d e

Ou

Majority

NEURAL NETWORKS 33

ade up o these elementary units are

interconnecting cells to form the formation of such nets is out to form arbitrarily many

Majority

a-~ b- 1 c-

OR

NOT

tThe diagra mati notation in use b logists seems startling when one di cover that i the di gram elow, the sig als fl w fro left to right! It is hard to believe th re exi ts a cu ture i which the ar ow po nts the wron way, so to speak. Of course, this is due t an a tempt to car·cature the a tual a pearance of some nerve cells in some n rvous ystem . I co ld not ring yself t furth r pro agate this traditional nuisance.

34 NEURAL NETWORKS SE . 3.1

input terminations, we do not allc w output fibers fn m dif ~rent cells to fu~e together. A glance at the illust ation in ths ch2 pter Ill stow ""hat IS

permitted.

Each cell is a finite-state rr achir e anc according y op rates in di cretf moments. The moments are assun ed s nchr::>nou among al cell . At each moment a cell is either firing or quiet; hese are the tv o possible states of the cell. For each sta e the e is an ass cia ted out put si nal, ransmitted along the cell's fiber b anches. It is cpnver ient o im~gine these signals as short waves or pul5es tra nsm1 ted v ry q ickly alon the_ fiber. Since each cell has only two r: ossib e sta es, it is evt n mere_ co~vem ~t tc think of the firing state as pn duci g a 1 ulse ~hile the < utet_ tate ~1el~~ no pulse. (One may think of ' no p lse" as th narr e of he SIJ nal a ssoc1 ated with the quiet state.)

Cells change state as a co 11sequence Jf the puis s rec ived at th ir in puts. In the center of the cir le represe ting each cell there ·s wri ten < number, called the threshold f the: t cell. Th · s threshol det rmin~s tht state-transition properties of l cell r in t e fol owin mar ner.

At any moment t, there wi I bel cert in di tribution < fact v!ty < n t~t fibers terminating upon C. 11\'e igpore ~ll filers v hich are quiet t th1 time and look to see if any il hibit1 ry inputs: re fir"ng. 'l' one or m'pre o the i~hibitors are firing, then ~ will not j re at time t +. 1. Ot~erwi e-i no inhibitor is firing-we cot nt ur the :mmb r of exc1t tory mputs tha are firing; if this number is equal to or g1eater than he thresho d of r (th number in the circle), then C ~ ill fire at timet I.

In other words: a cell will re at time t + I if an~ on/ if, a time t, th number of active excitatory inlputs 1 quais or e ceed the hreshold. 'nd n_~ inhibitor is active. A cell with thres old 1 will ~re if any xcita ory 1 ber I

fired and no inhibitor is. A cell fvith hresh old 2 requires 't lea t two excitations (and no inhibitio ) if i is to fire t the next mom nt. A c~ll with threshold 0 will fire at any t"me t.nless an i hibitor pr vent this. Fig. 3.1-2 illustrates the beha ior o a number of dif eren cells

REMARKS

(I) The reader will note th~t the state of a ell a t + 1 doesn't cepen on its state at time t. These l re ve y sin pie ' neur< ns" indeec . Ore not able property of real neurons is tha one having fir~d, tt ere i~ an i1 terva I during which they can't be fir d ag in (c lied the n fract ry interva ), an this illustrates a real depende cy o the previ us state. However, ~ e will see (section 3.6) that it is easy to "s mulate" st ch b havic r with groups o McCulloch- Pitts cells.

(2) Our "inhibition" is here ab olute, in tl at a ingle inhit itory signal can block response of a cell to (; ny a moun of xcita ion. we might equally well have used a diffl!rent system in ~hict a cell fires if tre di;ference between the amoun s of excit tion and nhibi ion ~xcee~s th

SEC. 3.1

a

b

c

0 1 0 1

0 p 1 1

NEURAL NETWORKS 35

0 1 0 1

0 0 1 1 Signals on input fibers

0 ) 0 0 1

a --8~ ) 1 0 1 0 1 0 1

a -r-f2l~ b -f-+-'=.1,

a-~~.-b -~ 2 c -lo

(

(

(

(

0 0 0 0 0 1

1 1 0 1 1 1

0 0 1 0 1 1 1

10 0 0100

00 01000

00 0 0 0 01

1 1 1 1 1 1 1

0010 0 0 0

0 1 0 1 p 1 0

Signals on

output fibers

Fig. .1-2. Behavior of orne c lis. ( he r~sponse columns are displa ed to how tt e alwa s-present del y .)

th eshold. This is equi alen to t avin! the inhibition signals increase tht thre hold Me Cullc ch, himsel , [ 19 0] h s adopted this model for ce1tain t ses.2

(3) ll>elay . We ass1 me a stan ard ~elay between input and output fo1 all o reels. In more pain~ takin~ ana yses, such as those of Burks and W ng [ 1957] nd C )pi, E !got, and \l righ [ 195 ], it has been found useful to epar te the tim -dep ndency frc m th::: oth r "logical" features of the eel s anc to i trod LICe s ecial time dela, cells along with instantaneous logical c:::lls. The use of insta 1tane Jus lc gical cells forces one to restrict the ways m w ich t e element5 can pe co nect d (lest paradoxical nets be dn wn). We v ill avpid Heme cept priefl in Sfction 4.4.1.

(4) ltf!atherrzatic~l No ations. Ir the origi1 al McCulloch-Pitts paper [ 19~3], tl e pr< perti s of ells nd tt eir ir tercopnections were represented

. 3.2 s c . 3 . . 2 NEURAL NETWORKS 37 36 NEURAL NETWORKS SE

Jying ~ssior s fro "tern poral rr ultipl thes num Jers t< gethe for a II the cells. Remember precisely why by a mathematical notation em pi expr m a tt e stat diag am (F ig. 2. -4) of the 3 delay net needs fully 8 = 2 x 2 x 2 propositional calculus." Bee use nost of ou r arg urn en s car be based s ates. on common-sense reasoning abou diag ams, it di j not seem nec1 ssary Thi' exam pie al eady show two easor s why we turn from state dia-here to bring in this mathem atica app rat us. Of cour se, a yone who g ams t o net\1 ork n a chin es rna :le of arts, as we start to consider compli-intends to design efficient con pi ex nets or co mput rs w ll ha e to learn c ted rr achin s. Fi st, th state diagr ams t ecome too large, as they grow to use the appropriate modern matt em at cal n prese ntatic ns. I or re a sons e pone tially with t henur nber o f disti ct ite ms or signals the machine must discussed in note 3 of chaptl r 4, he o igina Me< ~ulloc h-Pit s pa Jer is n memt er. S econd the ivisio n intc part rather than states makes the recommended not so much fc r its r1otat" on as fori s cor tent, philo so ph- d agrarr s mor emea 11ingfu I, at le ast to he ex ent that different aspects of the

ical as well as technical. b havic rare ocali2 ed in, or ac count d for , by different physical parts of ttemac hine.

3.2 MACHINES MADE UP OF I ~cCUL OCH PITTS 3 2.2 Gates and s witchE s. Co ntrol c f NEURONS flow c f info motion

In this section we show ho go at out c onstr uctin! som use Sui pose that i is de sired to co r1trol he fl. JW of signals or information won can anotl ful, more complicated machir ing a few seful kind of r eura nets f om o ne ph ce to er (F g. 3. -2). heir formation is traveling along es, u

Our goal is to collect enough evict: s ton ake < gene ral-p1 rpos com puter

r Trans rnitter Receiver I 3.2.1 Delays

F g. 3.2 2.

Examples l, 3, and 4 of s ction 2.3 ~ ive tl e sta e-tra sitio rt stru cture

for machines which remembe , anc read out, the I< st on ~. twc , anc thre a a ire ady 1 stab!" shed path, and we w ish tc control its flow by signals binary signals they have recei ed. hen ~tWO! k rna hine in F ig. 3. -l de u~der bur o wn c ntro. w intr pduct (Fig . 3.2-3) cells of threshold 2

('AND' cells into each fiber: in th ~ figu ewe use three parallel fibers just s- 1 p tc shO\ that wee< n ha die 2 ny m mbe at o ce. Now, during any inter-v I in which we\\ anti form ation to ftc w,w can send a string of pulses

! s-a-~( r,. a ong t ~e "c pntro I" or "gati ~g" fi ber. Solo hg as the control pulses are R

, the butpt t fibe rs wi I car y the signals that come into the ~ ' p esen sam

s-a- ~ r;-a (1 pper inpu s; if he co htrol pulse are< bsen there will be no output. t---( R The mse tion bf th AN[ cells into the ' ignal path will cause a unit \J' I

time de lay i the rans lnissic n of ignals. w could avoid this by admit-ig. 3. -1. Dt lay net .

precisely the same things. (< omp ~re \\ ith F igs. 2 .3-l, 3, ar d -4. Th ri~ ~ l Ootpo"

behaviors of these nets are de cribt d by he ec uatio rs

l I ~(t) S(t - l) Input ~~ ~ f R(t) S(t - 2) I

~@ ~ R(t) = S(t - 3) v respectively.

Note that while it req uires n sta es to remer nber l digit (see 2.3), t Contrc I fibe takes only n cells. Of cou rse, tt e net\ ork c elay n achin e witt n ce Is dots

Fig. 3. 2-3. ( :Jating networ k (faci itation type). indeed have 2n total state . For each cell ca n hav e 2 st tes, a nd or emu t

38 NEURAL NETWORKS SEC. . 2.2 SEC 0 3.2 . NEURAL NETWORKS 39

ting a gating, or inhibitory, efff ct wi h no del a -by som how inter up- Re eiver l. T he th resho d-2 ( AND) cells don :>t fire. When the control ting transmission along the fib er. his 1as b een r pres nted (e.g. by fibe r is a tive, the A ND C( lis tr nsmi the igna s to Receiver 2, just as in McCulloch [ 1960]) by a term in tion with Jne (i nhibi ory) tiber ncirc ling Fig 0 3.2- 0 AI 0, th e OR ells ' re nc wall "inh bited," and no signals can another, as in Fig. 3.2-4. But t is ' gains t our polic y her e to' ntroc uce get toRe ceive l.T hus t e act vatio n oft h.e co 1trol fiber has the effect of instantaneously responding elen ents. flip ping, dou ble-th row three pole swit h be tween the transmitter and

We can use the inhibitory ffect to co ntrol info rmatic n flo w, wi hin the two r ceivf rs.

the family of cells we are allow ed to use, y us· ng a 11etwc rk m e tha of

Fig. 3.2-5. Note that here, in form 1tion flow thrc ugh Jnly when the 3.2.3 Memory

p8 ~ The c ontro I syst em jt st de scrib( d is lOt V ry convenient because its

/ ope rator hast J con tinue to se nd cc ntrol puis :s throughout the interval

~ ~8 ~ dur 'ng w hich e wa nts si gnal OW. The et in Fig. 3.2-7 permits the op-v era or to initia e sig a! tn nsmi sion with

I ~8 ~ a si 11gle ~ ignal and to te rmin te it with lnj)lJI

~ Output

trt _.....- ther; a no no a lentil n is requi ed d ring

Cor the 'nterv ning perio 1 oftr ansm ssion. er The t rJ~Irol 2-5. ( :Jating

ick h ere is to us e a'' eedb ck" v Fig. 3.2-4 Fig. 3. networ k (inhi ition ype). .8,

fibe that runs from the o utput fiber of a ~ cell back exci ·natio

., control fiber does not fire. In the " facili ation ' (to use t he ne ysi- to an a tory term n at Sto t \)o'

urop 1

ologist's word) gate of Fig. 3.2- 5, inf< rmat 'on fl )WS 0 11lyw 1en t e co trol the 'nput oftht very same cell. Once this Sto

fiber does fire. cell has l een j red ( 'Jy a ignal from the -Now consider the network obta ned "'hen we u e bo h "ir hi bit on" (

"stc rt" ji ber) i will conti1 ue to fire G t all Fig. 3.2-7. Gating network with

and "facilitation" gates (Fig. 3 2-6)! Whf n the cont ol fit quiet the SUCl essivt mom ents, until 'tis h a/ted by a memory. er IS

signals fire the threshold-! (< R) cells, nd t he in form tion flow to sign a! 01 the in hi 'Jitory "stt p" J ber. Thr ough ut th is int rval of ac ivity 't wil send pulses to the "gate" cell

p~ and perm it the pass2 ge of in for nation.

l Tronsm1t1er IL F eceiver I This I ittle n et ha som men ory. It rer 11emb ers whether it has received

~ a" top" signa sine : it Ia st rec eived a "st art" ignal. We will shortly see

~ hov mor e com pi ex vent can be "r :mem berec " or "represented" by the I inte rna! s tate o fa net.

I ~()

tis ea sy to ee th t, in our n ets, at y kin d of more-or-less permanent

v t-r- men ory r aust c epenc on he ex stenc( of c losed or "feedback" paths.

/ ~ Oth rwise in th abse ce of exterr al sti nulati on, all activity must soon die out, lea vir g all ells ir the q uiet s ate. his w ould leave no representation 1---'1 -- of v. hath d har pened in th mor remc tepa t. Actually, the same is true ever in th( prese nee of exterr al stir auli o any ind, but this is harder to see

~() at n is poi nt. T e int( rna! s ate o a ne with :mt any "loops" or "cycles"

jepen can only on th stim latio that has o curred in a bounded portion j

~ ::;JR 2 I

ofH e imn ediat past. The bounc is ju t the ength of the longest path in .....__ eceiver the et. Ir a net with ycles there sn't a ny bo Lind on the length of possible path s witt in th net, so the re is o lirr it to he duration of information stor ge.

Control fiber I ere is a not er dif erenc e betv. een o ur "c lis" and biological neurons. Fig. 3.2-6. An in forma ion-ro te swi ching evice. It is evider t that in the brain notal infor matio 1 is stored in this "dynamic"

40 NEURAL NETWORKS SEC. 3.2.4

circulating form; it is not k own t what exte called, "reverberation," is i port nt.

3 latio , or s it is often

Although this "circulating" or ' dyna ic formation can be stored in ur cCulloch- itts started with component parts capa le of some kind of " tatic" me ory, e.g., the equivalent of a magne ic-co e, s itch, r "flip-flo ." he n ts of the next section show how we c n si ula e th se w th components.

3.2.4 Binary scalers

Another kind of memory output pulse for every two inp serve as the basis for countin in Fig. 3.2-8 is the simplest on

et is the b nary cafe , whi h pr due s one t pul es. In rna y co ch d vices and othe

An initial input pulse wil ation. A second pulse at som later time will inhibitory pulse on fiber y will exti guis the that same time a pulse will ppea on he o tput fiber of B, that two input pulses have n w o curr d. he n t th original "resting" state. Hen e th "sea er" ivide pulses by a factor of two. Note that Fig. 3.2-Fig. 3.2-8.

Inspection will show that t is si if input pulses occur too close to get er, i e., o may be seen more clearly if e d aw t e st (Fig. 3.2-9).

The machine has two p so there are four possibl tota state . T machine- "pulse" and "no pul e"- re re happens that the fourth sta e nev r occ rslead into it.

Fig. 3.2-8. Defective binary sc ler. Fig. 3.2-9.

rns o it r of inpu ed ithi \

c' a'• \ _./

SE . 3.2.5

In

pri we wh ass

C unter

NEURAL NETWORKS 41

h cells are firing; since the event it as a ghost. An output occurs e de ice is this: in the transition chin ignores its input. It will is condition occurs only after a

to @ ); hence pulses

ause difficulty. A slightly more

Start

t }

Evennumbered

l•s

}

Odd-numbered

l•s

State diagram of Fig. 3.2-10.

, and an output pulse oc

stat -i.e, afte each even-numbered I.

g pa agra h could be regarded as a oun up t two. By similar methods puis s, one for each M pulse inputs, mbe . Consider what happens if we

ence-the output of each .2-12.) We represent the

uts for pulse and reset signals eel s A and B of the nets just

for ach pair of input pulses.

t e ha e pro ided · n extr "Re et" ch nne! or for ibly returning the device to its resti g stat ; we w II use i in the next s ction.


Fig. 3.2-12. 2 K_a ry seal r.

Fiber A is quiet after even ptlses, but (~sua! y) jitles stt adily aftet odd pulses. t If we string together 1 sucl scal~rs, v e ob ain 2 net fVhich can count up to 2K and then start ver. If wt fire he cc mmor resc t fib<: r we can restart the net at any time.

How can we count up to anum er wllich i n't e act! a pc wer f 2? We can use a trick which invo vest e birary number sys em r pres ntation of that number. Take, fc r example M = 13. Then M = 8 + 4 + 1 = 1101 in binary form. Let T be he n mbe of 1 sin the bi ary umber. We create a single cell ith tlhreshpld 1 (3 ir this case and run excitatory connections to this cell f om he st ady A) o~tput of hose scalers which correspond to th posi ions of th 1 's c f the bina y form of M. (See Fig. 3.2-13.) Now, gc neral y spt akin!, the set o A tilers v hich

ln---r-----1 I t ,_

Fig. 3.2-13. M- ry sea er for 13 = + 4 + I.

will be active after the Mth in] ut p1 lse ~ill be just hose ·n tht posi ions for 1 's in the binary expansion of M . Ar d the new cell v ith tl reshc ld T willfire when first all these po ition bee< me a tive oget er, i.e., or the Mth count. The path labeled with an x will ause the r set I ne tc fire when this event occurs so that the counter will tart c ver after each g oup of M pulses. With this con nee tion n op ration, th whc le ne bee mes an "M-ary scaler."

t See problem 3.2-2, p. 44.

j

SEC. 3.2.5 NEURAL NETWORKS 43

PRC BLE~~. W y dor 't we need o rur inhil itory connections from the othe scale stages to tt e outi ut eel ?

tye rur aga n inti-> timing problen s if I ulses enter the net too rapidly. Cor ect c perat"on i cert inly assur d if pulse are spaced by intervals of !<. moments whe e K s the number c f binary scalers in the chain. Oth rwist ther is a chance tha the top c II w·ll miss its chance to fire. The trou\:; le is ue tc the act t at there a e tra sient states during which the< utpu of tt e chain do sn't •ratch the binary expansion of the number of in puts hat h~s oce urrec . Th se st1 tes o cur v hile the net is "propagating he c rries " ll appilty, th se tr~nsie t st2 tes never correspond to numbers 1 hich will cause puri< us fir"ng o the tpp cell (as the reader may care to ve ify), so th top cell v on't re when i shouldn't. But the total court might pass through M wh.ile t e carrying is going on and the top cell ould miss hat count. We can I revert this by spacing the inputs so that hen t has a chance t< "set le down" t etwe n counts.

I XAMI LE

C onsic era hree stage cour ter li e th t in Fig. 3.2-12. Table 3.2-1 shov s the fir in pat ern i res) onse to tt e inp ut sequence given in the top I ne. 'Nun ber" is tht bin a ry nu p1ber A,+ 2A 2 + 4A 3 that appears

able .2-1

Signal I 0 I 0 I 0 I 0 I 0 I 0 I 0 I ... A, 0 I I I 0 I I I 0 I I I 0 I I I ... 8, 0 0 0 I 0 0 0 I 0 0 0 I 0 0 0 I 0 ... A2 0 0 0 0 I I I I I 0 0 0 I I I I I 0 ... 82 0 0 0 0 0 0 0 0 I 0 0 0 0 0 0 0 I 0 0 ... A3 0 0 0 0 0 0 0 0 0 I I I I I I I I I 0 0 ... 83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I 0 0 0 ...

Nu nber 0 I I I 2 3 3 3 2 5 5 5 6 7 7 7 Shou d be 0 I I 2 2 3 3 4 4 5 5 6 6 7 7 8

currently 2t the A ou put f bers. Not thac the< ount '4' never appeared in the A ot tput bers· this is bee use we die not space our I 's far enough apart (sine k = 3) a d ne inp ~.~ts c~ me i while the net was "carrying" from 3 to 4. N e erthdess, he netwod as a who! e never really lost count, and i catc es u~ soo~

Another kine of c unte which is conce1 tuall much simpler, but also less encient in it use of eels, is shown in Fi~. 3.2-14. This counter might be ca led u~ary ather than binmy-it correspon s to counting on fingers withe ut a ' radix" or 1 ower base~ nun ber swstem . In this net the activity


0 t

Fig. 3.2-1'. Un ry sea er.

advances, with each input pulse, one cell furth r do NO th chain. \\hen i reaches the end of the chain it caus san :mtput pube and rese s the who! system to the resting state. (At this moment it i cap: ble c f missing count. This defect, as usual could be fixed by a ding extn cells.) T count up to M now requires lf/ cell inst ad o log 2 (M), ~hid is the ver much smaller number requind by he b nary scale chaip. T1 app eciat this the reader will recall tha log 2 ( lOOC) is a pout 0 an~ log (1 ,OC 0,000 is about 20, etc. Recall the 1 emar s in sectic n 3.2 1. Here VIe are usin one whole cell for each state c f the binarv machine.

PROBLEM 3.2-1. Verif' the s atement tha the A -ary 5 caler f this sectic n cannot yield spurious ot tputs because of carry-prop gation conditio~ , although it can miss prop~ rout{: uts. l his is ather comp icatec

PROBLEM 3.2-2. The s atem~ nt th2 t fibe A in Fig. .2-12 fires teadi y

after odd pulses is false be a use tate (~ d es not yielc an o tput n fib r

A 1. Can this be correcte< with ut in reasi1 g the over- II del y of he ou -puts? This defect need not affect the cc nstru tion < f Fig. 3.2-1 bec*se or e can run fibers from cell C swell as from cell A to theM- jetect r cell

PROBLEM 3.2-3. Desig a bir ary k -ary caler hat d )es not miss coun s under any circumstances. This is ve y con plica ed. (Note hat e en tt e non-binary scaler of Fig. 3.2-14 an miss an input hen it rese s. Fi it!) t is clear that the difficulties van is if we can b sure hat s·gnals are separated enough. The troubles ari e fro In att~ mptin~ to co cor plex ppera ions in times of the order of the espon se del ys of the i dividf!al p~ rts. t is r -markable how far comp ter d~signers ha e be~n abl to ~o, in gettirg around the carry-propaga ion d~lays ne m "ght s~ppos to b~ ines apab e in arithmetic devices. For a rev ew of some of th1 se tecrniqu s, see sever I papers in Proc. Inst. Radi Engrs. [Jar., 196 ].

3.2.6 Nets to do arithmetic

It is interesting that on1 a v~ ry si nple net i req ired to ac d two arbitrarily large numbers pr sentt d in a binary serial form. The sta e diagram for such a machine was I ictur d at the epd of sectipn 2. (Fi .

'

SEC . 3.2.6 NEURAL NETWORKS 45

2.3-5). (Note Unl ss th surr manes are pres nted with the least significapt digit firs, no nite ~Uach ne co~ld d this "ob. For the value of each dig it of the sum de{ ends unct onallw on d/ the input digits of lower order. Th higl est d"gits c n't b con pute< unti all t~e others are available and wo~ld n quin unli1pited mempry if arbit arily large numbers were to be ad1 ed, high d"gits first.)

Let.~- ; anc B; b the ith d gits c f the sum nands.1 Let S; be the ·ith digit of heir urn. Then S 0 is the' mod 2)" sum of A 0 and B0~that is, it i 1 or 0 as (A o + B o) s ode or e en. But, ust as in decimal addition, the high r dig ts of the s m dt pend not < nly c n the corresponding digits of he summands but also on whetl er or not here was a carry from the previous state Let~; be 1, or 0, ac ordi g to whether there is, or is not, a c rry f om t e (i 1)-t~ sta~ e. 0 cour e, C is 0.

Then the r les f1 r (bi1 ary) dditi on ar

1 il (A; .S;=

B; C;) is odd

0 if (A; B; C;) is even,

anc

ri+l . {~ if ( ;+ B;+ C;) is 2 or greater

if ( ;+ ~;+ ~;)is 1 or less.

No e the com I lete s mmt try o the wles c fA;, B;, and C; in these rules! !The ret in ig. 3 2-15 ealiz s bo h the e cal ulations with just six cells;

the outp t is ~elay d by thret timt unit . N pte how the firing of the thn sholc -2 ce I repr sent a ca ry to the fc llowing digit and how the input dig"ts an the arry. signal are reated syn metrically. (But note also that her we ave ' situ tion ·n which tl e sta e diagram is somewhat simpler tha the best McC lloch-Pitts net ¥e could i vent. Here is a machine tha doesn't di ide so wei into parts.)

A+--+--+-~+--- r..F 8 ~ires on pne or more inputs

clvrcar y) ~'+++--- t;c~;::'""''"'"' · \ ~ ~i es o.n wo or ~ S{f-3)

"-.. more 1n~s )f ._-+-+-t+-+--+---+-__. -~· Fires on exactly three inputs

-~~~- (needed just as a delay unit) " t--- \ 3 Fires on ~ three in~uts

Fi!. 3.2-15. A eriall::inary odditio network.

t hat is A 0 is he least sign if cant d git.

46 NEURAL NETWORKS EC. 3 3

If we use "subtractive in ibiti n" 2 o th tan 'nhibi ory to the threshold, we can use the e egan net of Fi . 3. -16,

A

8

ho

Fig. 3.2-16

PROBLEM 3.2-4. Giv n tha one f two num ers is oun in si e, sh w that a finite machine can be bu It (fo each boun ) whi h wil mult ply a y number so bounded by ny a bitra ily Ia ge n mber prese ted igit-b -digit. Show that if N is a boun ber o digit , the each digit f the product can be comp ted b g not man mor than N dig ts from the past. Exactly ho rna em or are r ally r quir d?

3.3 DECODERS AND ENCOD RS F R 81 ARY SIGNALS. SERIES-PARALL L CO VER ION

As we have noted (in s ction 3.2.5), th tota nu ber f sta es of a machine is the product of th nu bers f sta es of II of its p rts. t is t e huge numbers of states so obtain d th t rna es p actic I the digit I co -puter, and other interesting machines. If o e ha to r pres nt ea h st te of a process by the excitatio of diffe ent art o a m chin , on would be limited in practice to rat pie proc sses. But by u ing iffere t configurations of a relative) rese t states of a process can do much better.

This means that in efficie t m chin s, inf rmation ust b rep esent in something like binary-co ed f rm. (In c apte 6 w will disc ss-f r theoretical reasons only-s me achi es w ich se th una y fo m, a d it will be seen how inefficie t the are. Th stat s of he process carri d out by the machine will the be epres nted by c nfig ratio s of activ'ty

SEC. 3.3. NEURAL NETWORKS 47

in

3.3.

am (N bet

s. hine total states is by en

of ime- eque tial configurations. That tate y a fme c nfiguration of activity on f eel s. A d a ain, it will be necessary to

ble o de ode such confi urat ons to c nvert them to and from

***

1**

r fib rs. he w sections show various inC rmation fr m ne form to another; and bro ght oget er t give a powerful general itrar fini e-sta e rna hines.

**1*

1* *

ay ppear, from time to time,

*1*1 *11* *111

11*1 111* 1111

e pre ence of a ulse and '*' the absence of a pulse mo ent f tim . H re e have chosen as our ex

ssibl seq ence that may occur over an mig t not nece sarily want to distinguish

at there i avai able some special signal, en t e sequence transmission e able to distinguish between,

iagr· m c ntains a c py of the same two-cell

48 NEURAL NETWORKS SE . 3.3 I

m r

Signol~------...+--+--+--...-+---+--...-1--~~!!!r

111 P,..--...,.: +--+r--HII ~: 11........1

:----~ -.-... - ,+-1 --+1-+-~~ i r--11

l : : ~-~ L.J I I

Start ~ --' ~ n ~---...,:f+--+:-Hn=l \....l.._"TL1 ...L.._,r2 a

L ___ ~J L-J

Fig. 3.3- . Binary sec uence ecode .

~II:--** 1 * * * 1 1

t--..~: 1: ~ r--;= II-- * 1 1 •

-· 1 1 1

--.,...-1 ::~ 4- ""!:::--- 1 • 1 * -1• 11

k-1:::::: ll : ~ '-<-1::::-111 ~

network seen to the left.t It can l e seen tha nott ing em happen in th s net until a start pulse enters. If He sta t pul e ts ot ac comr aniec by an input pulse, the upper cell i sta~ e I \1 ill fire, wt ile if the s art 1 ulse s accompanied by an input pu se th low~r eel will fire ( nd tl e up er cell will not). In either case one< nd o ly one oft~e tw stag~ Ice Is wil fire.

At the next moment, the ulse putpt t fror~ sta~ e I w"ll serve as a sta t pulse for one section of stag II. The pther half pf st< ge II will ema n quiescent. (By repeating this argu lnent ~e set that the first el men of tl e input sequence has already determine wh"ch h If of the otal outp t bundle will have an output.) A s milar selection < ccur in stage I ; aft r the second moment there wi I be < puis~ on xactlw one of tile fou fibe s bridging stages II and III. 1\.nd < fter mof11ent~ ther~ will be j st 01 e pulse on the 2K fibers betwee~ the ~th and(/<, + I -th stages. At t~e en~, there will be just one output fiber excited, a1 d th"s wil be the fiber co -

SigMI' { ~~--h----#r-----h---+1-r--+-11-.--+-.f.-+--~-+.. ll ( IT .l .l ll J

~v•• 1 1 ~~ 2

0/:1 000 ( 01 010 011 100 101

*** "'" Fig. 3.3-2. hree- igit bi ary p: rallel c ecode .

~~

2

110

3

111

•

tThese are the same "switches" used in sec ion 3 .. 2, bt t note the n versal of inp~tconnection significance.

SE . 3.3.2 NEURAL NETWORKS 49

1101-

~-8 lf1put _ ~ 1 r; :y ([t 0

·~ ( erial)

~) >. tart..., ~Ci] (tt; ~

'( _!

~ ~ .b

2 ;

~Output "-... 1 (parallel)

'-.... 1

F g. 3.3 3. Fe ur-dig t seria 1 to-pa allel converter.

re pon_di~g tc t~e l inan number represented l y the input sequence. Thus ea h di~tlnct enal pulse sequ nee excites a dis inct output fiber.

We .vill ~ e ho "' to construct a different kind of binary sequence decoder bv ust g tht nets of t1 e ne t sec ion. (Specifically, one connects th ou_t ut o a net like Fig. 3.3-3 to tl e inr ut of a net like Fig. 3.3-2, m tch1r g, of ours , the numl ers o digi s.) If one counts the numbers of ce Is, th s altt rnati e me hod pigh seen mo1 e efficient than the present one. B~Jt if pne a !so c nsidc rs th~ cos of connections, and that highth esho d cell will prest mablw be xpen ive, t>ne sees that the binary net of Fig. .3-l ' ill h< ve advantages fc r Jar~ eN.

3. .2 ( ther ecod rs

If th~ cod d in forma ion comes in th fonr of single pulses occurring sir. ultauouslv alor g an Limbe of p ralle fibers, we can use the net below. H re we utili e hig -threshold cells with nany connections.

Figure 3.3-2 sh ws t e ne for V = ThP threshold of each cell has to eq~a~ its nt mbe of excitatc ry connect ons to prevent firing on a signal co ntammg fever l' thar required. The ell a the left has a threshold of 0; "t fire unit ss in ibite . (Unless such cells are used, the "null" signal

0 lnpu ::-

1-t--+-+---1----l-.

L_

~~--r<f€Jf-L<F~ ~r:::€) 1 n. :::01 Sta I l.J ' ) _) J ~

~Start

Fig. 3.3-4. Para lel-to-i erial c nvertt r. Nc te that it needs only one ell pe stage.

50 NEURAL NETWORKS SEC 3.3.3 s~c. 3.~ NEURAL NETWORKS 51

can't be distinguished activeh , unl ss sc me o ther ourc of s ignal ,likf r, ~

B the "start" fiber shown, is pro ided)

~ ~: -4 The serial and parallel n odes of o per at on a e, b oadh spe king g,

interchangeable. The nets in Figs 3.3- and 3.3-L perf prm onve sion '-:

in each direction. ...........

"' 1

~ ~· ~·~ Input

' ~ "-l -4 a -(] ~~ ~ ~a ----1 1101 9z

~ Out 'z

Fig. 3.3-5. Seria I encoc er. Fig. .3-7. Firing pattern encoder.

tions ir to th cost the c ost ri es m ore o less n proportion to the number 3.3.3 Encoders o patt rns t o be r ealize d. ~

Encoders are trivial for e ther direc ion. Too btain the erial signal In real phys cal S\ stem there are a !way limitations on the numbers

from a single pulse, for instan re, th ~net n Fig . 3.3- will suffice. An.d fo o com ectio ns th~ t can be b ough tog ther n one region. Basically, the

the parallel case one needs on y the prop r bra nchir g, as n Fig . 3.3- . lilnitat on is oneo fsign ~1-vs. noise levels. In practice one must make sure

By omb ning hese two ncod tl at th re is not t1 ogre '(It a · 'load' on c sour e which 'fans out" to drive

Input 1 in~ tee hniqt es v.e ca COl struc 11' any o ther f Ierner ts. 0 ther~ ise th ~sign a! lev I may drop below the noise

1 x-~0 more COm{ licatt d i put- utpu lc vel ir evita t>le in any ecei\ er de vice, and 1 he channel will become un-

' 1 relatior s. Spppo e thi t wh neve rc liablt . Tl ere c re si 'rnilar const aintj on l he amount of 'fanning in" 1 a lowec . Th otht blems in th truction of practical thresh-a pulse occu son some inpu fibe rear r pro e con

Fig. 3.3-6. Parallel encoder. fi we want to g nera e a • ertai o dele lnent -pre blem s of esolu tion ws. st ability-and at this writing

ti !ne-se ~uence fir ng patte1 n 0 c mpu ers < o nc t oft n u e ele ment am logous to McCulloch-Fitts

some output fibers g" ... , g n For exanr pie, i /I fi es at time t, we migh n euron s witl high thres ~olds . As ~ewi I see ·n section 3.6, everything we

want gi to fire at t + 2 and< t t + 3, a d gz to fi e at + 1 and + 3. "' ant tt do< an bt don witl a fi ed b t>und on the degree of "fanning"

Suppose also that when a pu se oc urs ~ t tim t on a not her ir put f ber j p rmit ed, b ut th s inv olves in esc apab e del ays-slowing down compu-tion 4

we want g 1 to fire at t + 1 G nd at t + 7, an gz o fin at t + 2 ·md a t< peed

t + 3. We can represent thes spec "ficati ons bv PROBl EM. Chara cteriz the b ehavi r oft he above encoder when signals t + 1 t + 2 t + t + 1 t 2 + 3 a rive c osely togett er alo ng tht inpu lines , so that the response patterns

/I (t): z(t): v a e fore ed to over! a p. H< w do s this beha' ior depend on the fact that all

gl X X gl y cells ha e thre shold I and here reno inhibi ors?

gz X X gz v y

The net in Fig. 3.3-7 real zest ese s pecifi atior s am sho .vs cit arly a 3.4 RE ALIZA TION OF M ::>RE C OMPL EX general method for making st ch m ts. T here ation betw een t e res ponse- ST MULL S-RES PONS SPEC I FICA TION~. specification table and the ne con ectio ns is een nore asily by d awing n E BEt AVIO ll OF ~ETS WITH )UT C YCLE 1

the net with signals propaga ing f om 1 ight o lef. w can exte d th"s method to handle any numb r of nput and • utpu fibe s, an d res ponst s One mig I twa t mo re el2 bora e stin ulus response behavior in which of any duration, by extendin~ the c iagra ~in ppro [Jfiatt dime nsior s. st ·muli, as w ~ll as respc nses, are f mep tterr s along a number of fibers.

If we measure the cost of am< chim only by it nun ber c f cell , sucjl " e car achic ve th s by omb ning ever< I oft jle techniques of section 3.3. nets are very economical. I takt s jus n ce Is to reali e, in one net, - n " ewil do tl is in uch way that here ultin g nets will have the peculiar different firing patterns. Ho fvever 'if 0 e co ~nts hem mbe of c pnnet- p oper y tha they cont inn< feed back 1-circ ~Jar signal paths-no routes

52 NEURAL NETWORKS SEC . 3.4 SE< . 3.4.1 NEURAL NETWORKS 53

along which a pulse can return to tt e san e eel agai nand agai1. It is of Our "re-c :>der" net has t hree parts. In the part to the right, the interest to see precisely what s ch n ~ts ca n do and v hat they c an no do, sp ce-ti ne in put p atterr s are COn '-I erted by a simple series-parallel con-for these are exactly the machin es wi hout any I mg-h rm m emory. ve ter in to pu rely patia pat erns. Thi~ redL ces the problem to one of

The method can be shown t est b exar 1ple. Supp ose tt at tht situ tion rec ogniz ing a pure y spa tial d strib tion of sig nals. In the middle of the is like that in section 3.3.3 exc pt tt at th stin uli a e ter npora I pat erns ne is a et o "rec ogniz ng" ells, one or ea ch of the specified stimuli. along several fibers e 1, ... , e, . We rr ight, for exam pie, want two Th eir in put c )nnec tions and t hresh olds re ar ranged so that each recog-stimulus response pairs niz ing c 11 wi I fire on r: recisf ly on e of he s imulus patterns to be de-

tee ted. The tart Julse is tn nsmi ted t J the recognizing cells with the t t + 1 t 1 + 2 pr perc elay o tha t they wod only whe the entire stimulus has arrived.

e, X g, X Nc te th tthe outp t (en oder part is the same as Fig. 3.3-7. sho llid yi ld R,

The outpl t fibe rs fro m th gnizi g eel s produce the required re-s, ez X gz X reco

e3 X X sp nse atter ns ex actly as m secti n 3 . . 3. he technique is easily ex-

and ter ded t o har die rr ore 1 1put and c utpu cha nels, more S-R pairs, and

e, X ~ lor ger p atterr dur tions. No e tha t this syste m requires only one recog-

shot ld yif ld g, I Rz niz ing c 11 for each des in d S-F pair Ofc ourse , there are further expenses Sz ez X X gz X the n ction ciate j wit h. such cell and in the in-m any onm a sse 1 eac

e3 X en a sed numb ers o conn ectio s to he ot er ce lis.

Of course, we cannot ask for t 1e re pons to begin unti afte the stimulus ends; there must be s :>me elay. Thf net of Fi 5· 3.4 I re lizes 3.~ . 1 De lays . In effie ency fthe anon calm ethod these specifications with a del ay of one addit on a! mom ~nt. Note how the tables above are "wired" in to thf net. Ther e are certa n ine scapa bled lays in an y computation, due to the

-t fac t tha cert in ev ents t ave t 0 occ ur be ore c ther events. In general, a

~ 0 1 -e, Ia~ oft IVO ti ne ur its (2 nd n :> mo e) is requ· red for an arbitrary finite ./ / tal le of stim ulus- espo se n quire ment . T 1is is shown by our con-

g,~--1) str uctio abo e. F ougt ly sp eakin g, on nee js a level (delay I) for the '"- ''"- -t 0 -e2

rec ogniz ing c lis o thei equi val en , and one needs another output level 1 (a1 othe del a ) to pro vi ie fo the ringi 5ether of the outputs from _,. / ng to

g2 1-tv--tV di eren reco gnizir g eel s. c f COl rse, c elay 2 is not always necessary,

rf) an d it is som time poss ible t :>real ize th e des ired behavior with delay I \. f, T 1 -e3 or even jelay 0 or I, et . (T is ca 1 hap pen if , as in the present example, ~

~~ f-on can dist nguis h the stim ulus patte ns b fore they are completed.)

) .C Th e net in Fi .3.4 2 ha , in his s nse, delay 0, since the output begins

¥ co ncurr ntly .vith he Ia t (sec on d) volle oft h.e stimulus. This net hap-

~ pe 11s to have the s arne WO S imul lis-res ponst reactions as does the net

'-I- )J: f- of Fig. .4-1 ( but\'< ill re ct di feren ly to othe stimuli). It turns out that

f.~ o; th( info mati non fiber 2 wa redu ndan ; so v e don't need it. 2 ~ ~ The stimu lus-re spon e rei tion used bove for our examples was not

I I I I I I 'I ch Jsen rbitr arily but "' as de signe d als to llustrate the possibility of

~ I I I I I

----4V ~ I I

~~ 0: I I

~ ex hang ing t e rol es of time and ocati n m a fiber bundle. The two I I I j .§..!g,!j

J ./ sti mulu -resp :>nse airs! 1ven xplic itly a e ins ances of a set of sixty-four Encoder Recog izers e con erter rei ation whic h ha e the prop erty hat if inpu e;fires at j moments after

Decoder th inpu t star puis , the out utgj will fi e at · moments (plus the system Fig. 3.4-1. Stimu Ius-res onse r ecoder de ay) a fter t he ou tput tart pulse Th s wh :>le set of relations can be

54 NEURAL NETWORKS

91-------92 _____ _

5

"canonical" encoder-decoder yste it would require sixty-four rec gniz r cells. B tion has such a simple structu e, we were able to re than that number of cells. T rea ize a com letel response relation, where the timu i ha e up to N = 6), requires on the order f 2 N cells, on t proportion of behavior functio s ca be r alize nets. But there is reason to eliev that this almost all of the functions that are e

SEC. 3.4.1

ig.

2

3

5

j

SE . 3.4.2

Th tech

3.4.2

th

is

NEURAL NETWORKS 55

atures of individual problems.

.4 a d 3.4 1 are all cycle-free; that is, s. e ha e seen that, even with this

sti ulus- espo se behavior function that in t e rna ner 'ndicated i our discussion. These are avior pat ems that an b realized by nets without

ior patterns so realized are must admit that, at least in ould behave in every respect

hat man. That is, one could ach equence of stimuli the "man"

be elicited. But in so far as nd even misleading to point

woul not now monstrous structure without a c mple e, ext nsive theor of h w th man operates. And if we had the latt r, we could build sma ler, g nuine y inte ligent machine.

3.5 THE EQUIVALE CE FINITE-ST TE

of t e kind we have been con-sid ring · s a finite-state mo ent, the total state of the net is gi en b the ring Q( + 1 (Q(t , S(t) is d term'ned y the connection structure of the net, and the utpu fun tion R(t 1) F( (t), S(t)) is determined by wh'ch fi ers a e des·gnat d as arryi g output ignals.

It is i tere ting nd e en surprisi g th t the e is a converse to this.

so cer ain eural net ha e pre isely like

nd can be "simulated" by, iven any nite- tate achine M, we can build a ich, egar ed a a b ack-box machine, will be-

56 NEURAL NETWORKS Sl C. 3.5

The construction of the c quiv< lent net-machine N"' is surpri ingly straightforward, once we agre on ow the in1 uts and o tput~ are o be represented. Suppose that the inputs a d outputs to M are S1, ... , Sm and R 1 , ••• , R. respectively, a1 d that the tates of Mare ( 1 , ••• , QP. Our net NM will then have input fi~ers. 1 , ••• , sm and o tput fiber r1, . . , '•· and it will contain m cells, C lJ, ... , ( mj, or e ch state Qj o the machine M. The cells Cij are tp be < rran~ ed in a twc -dim nsio1 a! (n x p) array, as depicted in Fig. 3.5-1

Each of the cells Cij has tt reshc ld 2. We explc it th< abil ty of such cells to detect coincidences. n fact the cell rij wi I fire (at t" me t + I) precisely when it receives an input f om.:; (at ime ) and whe the imulated machine M is in state Q . Til us the firir g of ,..,ij is to be equi alent to the pair of events (Qj. S;). To make t e ne equi alen tot e machine M we now have only to arrange thirgs so that t produces the I rope output F(Q;,Sj) and goes into treaprropriate interna state G(Q;, Sj). To see how this is done, consider he vertical colur ns o the c iagram.

G- unction connect on box ~G7.(o~2~.s~1~~-~o~1~--~~~r~~~~~r---r.---i---: A ~~ --~ , .. --~ v· !

'---..,.t+t-----lH+;I;-t-t+-+---ttH--,;tt----t--tttt---j

-+-.:--IHH---1!11---.1--I!+t-~---

F -function connecticn box I I

Outp t

Fig. 3.5-l. Tr e canonical ti ite-state neu al net.

SE . 3.5 NEURAL NETWORKS 57

At any m Jmen t there will be, am or g all the descending fibers in all th shac ed cc lumns, precisely one ctive fiber Suppose that this fiber is in the jth col mn. Ther the ~et NM is simul:: ting the state Qj of M. (It doesn't friatter whi h fib r of the CO Umn IS ac ive, since all have the same connect ons.) Sup ose a !so til at at each friom nt precisely one horizontal (ir put) ber i acti e, say, th ith ow f ber. This corresponds to some In ut si 15 nal k:o;. 1 hen recis~ly o e of the array cells will fire at time t 1; t iswi I be cell C;.

The ~ber rom Cu diwides into wo b anches. The descending branch go s to he" -fun< tion onne tion box" and there leads to the cell which re reserts th ap{:ropriate output F(Q ,S;). Thus the function F is ""ired" into his part of the net, and it n ed cc ncern us no further.

The ascending bram h of the output frorr Cu is responsible for the m chine's ch nge Jf state. ., his fiber goes up through the "G-function connect on b x" \\ hich is so wire< that this fiber enters the descending co umn for the appropri te new sta e (G(Qj. S ). If one and only one cell C; fires at tir~e t 1, "e are thus assu ed tt at at time t + 1 there will ag~in b pre< isely one ctive fiber amo~g th~ descending columns (and th tit" ill be in the col mn < orrespond"ng to the next state of the simulated m chin M). Thu~. if t~e signals< nteri~g NM are at each moment th sam as tl ose enterir g M, both fr!ach nes v ill go through the same sequences of st< tes a d ou puts! (Th out1 ut of NM will be delayed by one tir e un "t bee use c f the pR ce Is in he output< onnection box.) This compi tes tl e proof of he theoren

EXA .tPLE

The partie ular onnection sho vn in the diagram Fig. 3.5-1 show the remit o applving he general metl od t< a serial binary adder, e.g., the one des< ribed in section 2.3. 1 corresponds tJ the 00 input, S 2 to the 01 and 10 inputs and S 3 to the 11 inp t ca~ e. Q 1 is the no-carry state and Q2 is the carr state. On has o ge the nachine started bv introducin_g _a start pu se in o s~~~-e _d~ ~~!!.<! f!g__!i ~r; ~~_elects the ini.!i;!l_stat~_Q[..Jhe _n~_. Tc make the mac ine ipto a com entional two-input serial adder on has to a tach an ir put ncod~r of som sort, such as the one in Fi . 3.5 2.

Start ~ [)_ ~() 51

00

~ ~ .4 ~

,.,.....,.._ a 52 01 or 10

.............

X ? _...........

~ ~ a 53 8 II

Fig. 3 .5-2. npute coder for bir ary adder.

58 NEURAL NETWORKS SE . 3.6 SEC. 3.6 NEURAL NETWORKS 59

The "canonical net" for nite s ate m achim s has he in eresti gpro perty 0 allo wed alues oft e th eshol d n, or d d we put any limit on the that it is composed essential y of n othin mon than AND C ells. The r ader n mbe s of onnt ction to < nd f om a ny o e cell. It will be noticed, will agree that the OR cells i 11 the utpu conn ction box re no t real y in- h< weve r, tha tin n ost c f the nets exhib ited, the cells used thresholds of volved in the basic operati n of hem achine .) It folio s th2 t, in some 1 and 2. Inc eed ¥em ght s uspec , fro m tht results in sections 3.3 and sense, these are all we need. Wewi llloo into his m ore c< refull in se ction 3.5 tha wed o not need hres olds arger than this. This is true, as we will 3.6. In particular, it would seem hat t ere is non ed fo r inhi bitq!}: con-

see bel )W. But a thou ~h hi her hresh olds are not basically necessary, nections-for none of these appea in tt e dia ram! The ecret lies i our they m ke it possi ble tc perf Jrm < tatio ns with fewer cells and less ass urn p!_i<)Il_tJ!~L!h~_i_l1.2..!:1..t__ is.!!.'!!:> ~!.~. n "cc m_p!~_t _ly __ <:l~ coded ' for n:/ at omp

each moment one and of!ly_c: ne of the in put li es is xcite< . We now now d( lay. Cons der t he pa allel decoc er of Fig. 3.3-2. If there are n input

that if this is the case we ne ed on y thr shold 2 cell s to s mula e arb trary fi ers, his d code use~ cells with thres 10ld p to and including n. But machines. However, if the nput ignal don' t hav< this onve ient orm, the dec Jding dela\ is or ly 1 ime t nit. We c Juld construct a parallel de-then, in general, it will requ r(!.i!ll ibi_t9_r v __ ~Q.!l. (!(;_tiQ s (as in Fi . 3.5 2) to cc der c ut of thres hold and 2 eel s, bu the delays have to be longer; at bring them into that form. l ut it till se msq uite re mark ble tr at on y the least lo :;z(n) threshold-2 cells are requirec once the si~ nals a e soc ecoded.

PI OBL "'M.< onsti uct p rallel decoc ers o t of threshold-! and -2 cells. PROBLEM. Prove that on cann ot, in gener I, de< odes gnals using only C< nstru t the arne t sing c nly th eshol d-2 ce Is. AND and OR cells (without in ibitic n).

It i~ inte est in tha if w are willir g to pay the price in delay and Efficiency m mbe s of c ells, lve ca simz /tane bus/y restri t both the threshold number It is important to recogn ·ze th t we aves mula ed th< beha vior c f the ar d the numb k>r of onne lions to an fl fron each cell. We use a technique of

given machine M by a rath< r inefl cient mech nism. 1ft ere a e I si ~nals b anch· ng bi 11ary t rees. To St e the meth od, s ppose that we wish to con-and J states, our constructi on us s I X J ce Is (pi s the numl er us d in st uct n ets ec uival bnt tc the Ierne rts ir Fig. 3.6-2, which are typical of encoders and decoders). I;, hese ells c uld t ke on all p ssible firing con- the kir d W< ha\ e be n u ing reely figurations the machine N wou d hav e 2/J bossib e sta es. C nly a very T ese c ells c an b rep aced by t e lef ~l ~~ small fraction of these occu in th ope ation of ou mac in e. See tl e rear d rig ht ne s of ~ig. 3 .6-3 espec tively . ' marks in section 3.4.1. This sort o f resu t redt ces th ~ app icabil. ty of In any Ir the r et to he ri ht, v. ehad to lo ~d th f-(t---- -~ -powerful conceptual tools c f logi and analy is to the p actic2 I desi n of

di~gran witt extr ells s o tha som OR

'11 ~~ actual machines. Those m thod~ tram form ~way he SI ecial featur ~s of the net will have he s< me c elay or al I ,

devices or expressions by lo ding hem ~O\Yn ~ith t rms hich ~o no add to their content. This imprc ves tr eir "f rm." But i USU2 lly we rks a1 ainst si nals.

efficient design. The umb rs of input and butpl t con Fig. 3.6-2

There is also a great dea of tt eory: nd p1 actica kno ledgt cone rned n< ction , as ~ell s the thre hold num

with the efficient realization of b< havio patt rns ( witch ng fu pction s) by b< rs, aJ e hel to 2 in tl ese r ets. This ~ean s that we need not be con-nets of various kinds of eh ment~ -rel2 ys, tr nsist< rs, v2 cuum tubes and Ctrned f.vith rob I< mso "lo< ding' and noise , provided again that we do the like; but these practical n atter are o the r pain li re of< ur stl dy. n t mir d tht gene al sl wing dow ~oft jle OJ: eration of the system. And

it mean s that our~ tockr porn f pa ts ne ds tc contain nothing more than

cc pies fthe three elem nts" e hav b use< in F g. 3.6-3, shown individually 3.6 UNIVERSAL SETS OF CELLS in Fig. .6-4.

He ce tl is se of hree elem< nts f prms a "universal base" for the Up to this point we have onst ucte< our nets by fr ely < onne ting c nstn ction of fir ite rr achir es. I turn s out that, at the cost of a delay,

together elements of the form howr in F ·g. 3. -1, w here we se no I mits tl e sec Jnd ( ~ND) Ierne nt ca 11 be mittt d, fo it can be replaced by Fig.

~~ 3 6-5 sc that the fi st (o )anc thir (AN ) NOl ) elements alone could form

~~~ - , a univt rsal t In de( d, th se t\1 o ele ment can easily be combined to

~0) ~ - _.?..::;; ase. - - fc rm a sing! dev ce (F ·g. 3. -6) \\ hich orm~ , by itself, a universal base! -...;.,;_ - -on ~ ::::: - :::g (It is u ders ood t hat o e car leav e fibe s unconnected, when neces-- som

Fig. 3.6-1 s ry.) To u e thi~ as a base, one 1as tc com pensate for the fact that the

60 NEURAL NETWORKS

Fig. 3.6-3. OR and AND nets on each cell

s c. 3.

net in Fig. 3.6-5 for forming AND from thre AND NOT' tak s tw mo ments to respond. This cann t be avoi ed, s the elem nt is reall unt versa! only for systems in whi h signals nter nly n alt rnat mo ents The same remarks hold for se era! f the bases disc ssed elow.

Other simple combination base to

:------~~ Ofl b

(1) (3)

Fig. 3.6-4. (I)

a b

(avb) c

Fig. 3.6-5

(1

a-+---+ b--+--+c-+---+-

NEURAL NETWORKS 61

inhibited. Then the element

(1) (2)

Fig. 3.6-7. (I) NOR, (2) DELAY.

(3)

R bAND NOT C.

combined with e AND's and OR's, as shown f the other combinations.

a--o (€tf---- a

(2)

(I) M JORITY, (2) INVERTER.

er si pie ombinations, some mentioned in he r ader ill note that our exposition

(2)

(I AND, (2) OR. Fig. 3.6-12. Refractory cell.

62 NEURAL NETWORKS SEC. 3.6.1 SEC. 3.6.2 NEURAL NETWORKS 63

"refractory period" -it can no be red )n t\\ 0 su cessi em men s-a re alizea by a net c 7mpo ed of AND' ~and OR's. Among these are the ele-basic property of biological m uron . It is int resti g th t thi pro Jerty ' m ents s hown in Fi g. 3.6 13 (t ile las t of v hich is already known to be uni-can be used to replace the use of in hi bite ry co nnect ons. Why this is so I ve rsal t y itse If). '"II of these rem rks re su bject to the restriction that will be explained in the next se< tion. th e rna hine com pose< of t heel< ment s rna have to be operated with

J de lays t hat re quire the i1 puts ignal to b< suffi iently separated in time.

3.6.1 Monotonic functions

It is easy to see that the two elem ~nts A NO ai d OR don< t by hem5 elves I~ -~0 ._ ~ =~ make a complete universal ba ( lrhey do, a show ~d in 3.5, f we

- f----o r-e. s we -allow external decoders and e ncod rs, b ~t we are now consi ~erin 1 the complete problem.) They ar defi ient m a certa n ab lity, whicl the Fi~ . 3.6-1~. N< n-mor otonic elements.

reader will discover if he tries t p use them tom ake a net ' hich simu ates, e.g., the "A and not B" eleme1 t. T e trc uble s tha ever ~net comr osed It f II ow tha our "reft actor

, cdl is universal, since it is non-

only of AND's and oR's has a m no to ic st ·mult s-res onse pror erty. m ~mote nic,:: nd si ce it vide ~tly h ~sAN p and OR built in very directly. That is, it is impossible to me. ke th out ut sr a//er by n akin! the input larger. Suppose that a certain patt< rn sl at th e inp ~t pr< duce a ce rtain

3. .2 he de uble-1 ne tri k response pattern R 1 • Now s ppos tha we const uct nev inp t s2 which contains all of the impL lses i n sl ( in th< sens of s ctior 3.4) plus The discu ssion in 3. .I se ems t J say that Jne can do very little, given an additional pulse. Then the outp t R2 resu ting rom S'2 ffil st co ntain 01 ly m no to nic ce lis m e AN ) and OR. Yet section 3.5 and particularly all the pulses of the response R 1 to S 1 , anc poss bly rr ore. the disc ussio 1 ofF ig. 3. 5-l se ems o say that the most general finite-state

To see that this is true, co sider the I ropa atior of p ulses in an net mach in (e.g , Fig. 3.5-l ) can be as embl ed es entially from just these two composed exclusively of AND' and OR's At each stage , the activ" ty in el mens. H OW C< n bo h sta emer ts be true? The only difference is that response to s2 must include th e acti vity t hat w ould esult from sl a t the in secti n 3. the nput to t em< chine are assumed to be decoded-at same moment. If S 1 causes a c II to be fir d, th n s2 will t lea t me< t the each m Jmen ther IS a sign I on preci ely G ne of the input lines. This same threshold. There is no ' ay ir whi< h the pres nee c f the new pulse m ans hat i the desig 1 of a mac iline 1 sing those inputs, the occurence in S 2 can cause there to be It ss ac tivity than befo e. I a tt res he ld is o a sig a! or one ine rr eans hat v e car cou t on the non-appearance of already reached, there is no wa yan wpu se.ca n pre ent a nAN ) or < nOR si nals on an v oth r line . Th ref or , we don't have to be concerned about cell from firing. The argumer t con inue , stei by s ep, t roug ham net- injlibiti ~g re pons s du to s ignal on hose other lines, and it happens work whose cells are individua lly m no to nic (a ~are NO a nd OF). J that we don' have tow rry a ~out ny o her i hibition problems, either.

What must one add, to AN p anc OR, tom ~ke a com plete base? We It tl rns o ~tth:: t we on't P.eed ull d< cod it g at the source to eliminate know that admitting an inhibi tor c nnec ion, ~.g., i P. the form of th e "A the nee for inhib tion, but v e car get way with some weaker source of and not B" element, will suffi e. Ir fact it ca n be ~how ~ tha one need n< n-m< no to ~ic be havic r, an this can oe bu ilt into the signal source in only add any non-monotoni ele (nent what ver! Th pro pf ol this v: rious way5. Yo P. Ne man [19' 6) ffi( ntior s an elegant way to do this. (Minsky [1956)) is rather comi licate d anc will ot be deve oped here. L .t us I epres nt si! na/s l sing airs if jibe sins ead of single fibers. In each

It is easy to show that us ng o ly A o ar d OR one can ealiz any pt ir of ines, we as ume hat o eam only one i: firing, at any moment. (One monotonic function whatever. To ~0 th is, ot serve that one an s "mply Ji e cor respo nds t p the origi ral fi per; t he ot ~er has pulses at just those construct a decoder like that i n sec ion 3 .3.2 ~ ithou putt ·ng in the i hibi- ti lnes tl e ori ina! ber< oes n ot fir .) Tl en w can compose the functions tory connections. Some thoug ht sh uld how hat t he in ibito y co nnec- A D, 0 , an AN[ NOT by t ile ne s of Fig. .6-14, each of which has a tions are there precisely fo obt ainin no -mor oton c be havio -to

' p oper 'dou )(e-lir e" ot tput.

permit a larger stimulus to b ock a res onse that wou d oc ur w ith a I So ·r we LISe th e dot ble-1 ·ne tr ick, w that just AND and OR form e see proper part of that stimulus. This s not need d wl en w are rea liz ing a a unive rsal t ase, and i ncide tally that these functions can play entirely monotonic function. S)mme rica! roles! We can see f rther that NOR is universal alone: the

It follows from the above tater ents that, given AND and< R, W :::can le t-har d net ofF g. 3.( -15 r alize s the unive rsal single-cell base OR AND obtain a complete base by add ng an y oth r e/e nent hat c ~mnot a/rea dy be N)T bu is co mpos eden irely of NO R cells:

64 NEURAL NETWORKS

(1)

(3)

Fig. 3.6-14. The double-line rick. (4) NOT.

Fig. 3.6-15.

3.6.3 Other kinds of parts for fi

SEC 3.6.3

for his e posi ion for their simplicity. No one ese e erne ts in com uters today, although some compon use are uite imila . By and arge, the elements of computers d res our cells but for s o packaging and \\\iring econo y, ne u ually find mo e in unit ·package-the equivalent of a out t ree r fo r of ur c lis. time of this writing, computer construction is movi g in the directi n of more integrated "micro-circuitry" i whi h m ch ore omp icate fun tions are realized in circuits spray d, pi ted, iffused o oth rwise emb dde in a homogeneous matrix rna erial e.g., a se i-co duct r pi te. I isn' clear at this writing to what xten sue devices ill continu to sembled from small universa bas s of elem As fabrication methods improve , we an e pect ore elicate "t resh old-logic" kind of circuit to pi y a I rger ole.

NEURAL NETWORKS 65

ite-s ate achi es c n be found in relay-contact pera ed synchr nous y, relays c n be made to act as though in discret mo ent . It ·s ve y easy to construct our en

c ders, deco ers, and ther ogic I fun tion as networks of relays; this is done in th pap r of hannon [I 49]. ur hief reason for using "cells" i stead of relays i tha rela s ha e a seful but confusing peculiarityclosed onta ts c nduc curr nt i eith r dir ction. Hence, in relay con

o natural notio of irection of signal flow, and t is de racts fro exp sitor clarity. he relay computer, with its me-c anic lly li ited spee , has been obsol ce the electronic computers o the I te I 40's; ut it the ry is ing important in the design

ogenic (su r) elements and field-effect

ases g id el men s (cu rent! out

ith eld- ffect

refound i the vacuum-tube logic of multiof f shio , but bound to be resurrected in

and other multi-element semifor s of transistor-diode logic. ld p int away from the theoret-

t---~x·

1---y' t---~z·

'-------'

Fig. 3.6-16

Do not look at the solution

an this res It be appli d to itself-that is, how many N T's a e nee ed to obtai K si ulta eous complements? This leads to a w ilber [195 ] and arkov [1958].

N TES

I. For supe b presentat"on o what is kn wn a out the elements and networks that onsti ute re l ani al n rvous syste s, re d Part I of Bullock and Horridge's "St uctur and Fun tion in the Nerv us Systems of Invertebrates" [ 196 ]. Th re is till n gene ally accept d the ry of the mechanisms involved in le rning in ne vous syste s. ( ee n te 3.) The situation with respect to function of speci lized arts f the ervo s system is somewhat better: see, for exa pie t e wo k on he vi ual s stem descr bed in the papers of Lettvin, Mat rana, McC lloch, and itts [ l 59] a d of ubel and Wiesel [ 1959].

66 NEURAL NETWORKS SEC 3.6.3

2. The models in which inhibitic n anc exci ation play ymmt tric r les ir elude ours, one allows use of man v inh· biton inpu con ectio s frcm a single fiber. McCulloch, himself [ 195 9, 196 J] use subtr active inhibi ion ir his d mon-stration that one can construct neura net r nachir es wh ose bt havio is irr mune '

to certain kinds of fluctuations in the neuro ns' th esholc s. Th e que~ tion c fhow to make machines that rema n reh able 1 nder fiuctu ating condi ions .vi thin their parts is a fascinating top c (no othe wiser nentio ned ir this ook). The results of a number of theorie , not bly 0 von Neurr ann [ 956], Moo e and Shannon [ 1956], and the cited work of Me Cullo h, sh bw th t for ~ vari ~ty of

I kinds of disturbances one can r raker rachir es as eliabl as d sired, at tht price

~ , HE ME MO ~IE! ~ 01 E\ ENTS of introducing "redundancy"- dupli catior ofpa ts-ir appr bpriat way~

I~ F IN I~ "E-S TA, '"E ~ ~AC HINES While talking about subt active inhit ition, we s ould certai rly ci e the earlier work of Rashevsky [ 19 8, 19~ 0], wh o first had tl e sen! e and the c< urage to try to make mathematical rr odels pfcon plical ed net ralne s.

The introduction of subtr ctive inhib tion r aises orne surpri singly com-plicated questions, even for s ngle- euror netw orks! We ill n t tre t thi! subject here, either; it is callec "Thr esholc Logi "an< the eader migh look at Dertouzos [ 1965] for a sur ey ar d at vtinsk and Pape t [19( 6] for SOffit recent theoretical developmen s. 4.:1 IN RODl CTIOII

3. Memory. Unfortunately, thert is sti I very little :lefinit e kno .vledg abou t, and not even any generally accepte theo y of, ow in forma tion i store inn rvous Inc hapte 2, w talk d of macl ine s ates nd classes of histories. We systems, i.e., how they learn. Most of th evid nee I ads o ne to believ e tha ol serve d the e, an d in c hapt r 3, t hat t ere i s a connection between the there are several mechanisms -at I ast d fferen for hort-t rm a nd fo long id a of mach ne "r nemo y" a d cia sses o f hist ories. Now we examine the term memory. One form oft eory would prop seth t sho t-tern merr ory i~ ' n ture )f thi conr ectio n. Fi st Wt will take a closer look to see what an "dynamic"-stored in the forr ofpt lses r verbe ating aroun d clos ed ch ins o "t quiv< lence class of hi torie " is r ally ike. Then we study the structure neurons-while long-term me nory s stat c-stc red ir the f rm o chan ges in

0 thest curi )US 0 Jjects us in g mo e for mal n ethods. We find a way of connections, thresholds, of the micrc a nato my. I culat ve tht ories of thi or sp

dt scrib ng cc ncise y exa tly \\ hat k ·nds c f eve nts finite-state machines can variety, see Hebb's [1949] boo once ll-aSS( mblie . A r nore s atic r ode! ·s tha I er, \1 hat 1 inds of th hey c cognize, and what kinds of of Rosenblatt [ 1962]. Recen tly th re h< ve be numb r of public atiom rem em ngs an re en a

proposing that memory is st pred, like g ~netic in for rnatio , in he fo rm o ~ cc mpu ation the) can 1 erfor m.

nucleic-acid chains, but I hav not seen ; ny of these theor ·es we rked put tc Ofc when "ren em be r," or "recognize," we are speaking urse, we sa

include plausible read-in and ead-c ut me hani~ ms. at out p; rticul ride lized notio s of r em or y and recognition. These may

4. The allowed "fan-out" numbe s in t pday' com uter < ircuit y are of tht or de nc t be ir perfe t har [nony with c ur co [Timon -sense notions of such matters. of 6; higher when long lines o "bu ses" re dr ven b ~ spec ial ar plifie s. I r Tl e the ry is xplic tly on ly abc ut tht limit tions of finite-state machines. nervous systems the situation is so pnewh t dill. rent bee au e the nerv fibe 0 the c ther h and, t ere is no cu rent cienti pc reason to suspect that living itself acts like a continuous < mplifi er th< t mak es up for I< cal d ains, withir or~anisr ns can in is£ lation tran1 cend hese I imits in any way important for limits. In some cases the nun hers< f neu ons t at aft [ect, o are a ffecte by, th s con ext. l ater o n we ee th< t a cr ature s ability to use the environment given neuron is in the thousan s. fo stor; ge of recorc s or- almo t the arne hing from our viewpoint-for

5. For further discussion of thi poin t, rea the relate b pap ~rs b1 :Me ~-'arth m terial for gr owth, does r ake a differ nee i regard to these limitations. [ 1956], a paper whose import nee h as, I t ~ink, not bt en ge eralh reco nized; The ~a in ~atht rna tic I disc overit s in t is chapter were made by the Solomonoff [ 1964], also of gn at ph losop ical i mport a nee; nd J\. in sky [1959 lo ician Steph en C. Kleen e [ 195 fJ]. Ir turn this was based on the earlier for some remarks concerning hese · nd th e ana logo us mor prec ise, re~ ults o M cCull ch-Pi ts [ 1 ~ 43] e pi ora ion " ith til e same general goals. There Shannon [1949]. The present ook r rovid s mo e or I ss en ugh r ather natical remain nsolv ed m< ny irr porta nt pre blems centered around the relations background for reading these (exce Jt, pe haps, Shan on), ut th read er will between the s ructu es of nets, the st uctur es of state diagrams, and the have also to bring along a go od de I of ensiti ity a d ins ght t und rstan st uctur ofm mori s. the approaches to the philo ophic I pre blem of "i ducti e inf ere nee " sug The rgum ents ir secti n 4.3 and s ction 4.4 require some mathematical gested in the McCarthy and th e Solo mono ffpap rs. "sophis icatio n," th :mgh o spe ific p ior k nowledge on the reader's part.

67

68 MEMORIES IN FINITE-STATE MA HINE SEC. . I SEC . 4.1 MEMORI ES IN FINITE-STATE MACHINES 69

Fortunately, these arguments a reno requir ed in the s que!, so th reac er Wod ing b ackw ~rds, we o servt that a mu st also have fired at t - 2, who gets enmired here should not I esitat to n ove c n. ~ hile he Ia er t - 3, ai d inc eed c: t eve y me ment in th ~pas , back to the time when S chapters may look even more c ompli a ted obro ~sers, they rein fact rot wa firec -th< t is, when the mach ine ~ as fi st started into operation. so. Except perhaps for a few p ints r earn e end of tht book, secti pn 4.3 is Cle ~rly v e cm not t II ho f\1 ion ~ago that event occurred. But we can be the most intricate, but once unc erstoc d it is really quite imp it. sur that the f ring f R, t a c rtain time t i.mi= lies that a must have been

fire j ate ery r 1ome nt fro m the Start to th hmt t - 1. We might say that the firing of R mean s that this r et "r cogn · zes"

4.1 THE MEANING OF AN OUTPU SIGI AL: FOUR EXAMPLES

an' sed e ntirel ~of c ne or more a's. J sequ nee' omp Our first goal is to clarify the struc ure c f the "equ val en ce cl sses pf

histories," mentioned in chapter A few ~xam pies v ill ill ustra e wh~t f\'e as umed that he st rt pu lse S pccur only once. If this event is these classes are like. We discuss them first in for tnally and hen i r terrrs ace< mpan ied ~) an a ignal the 1 cell res. po long as a signals continue of a more mathematical description. to a rive, heR ell kt eps fi ing; c ne m y thi k of this as a reverberation

Each example to follow is bas ed on aM Cull JCh-P itts n t. F or uri- faci ita ted by tht a sigr als. I the a signa ever ails to appear, the reverbera-tion must jieou forev er, sin ce the e will be no further occasion to meet the formity we will suppose that each has fc ur in put fi ers c ailed 2, b, c , and d. reqt ired t res he ld of for n eRe II. Si nee w assumed that just one input For the present we assume that at each mom nt pr ecisel 1.1 one and c nly o>ze fibe is fir d at t ach n omen t, the occur renee of any of the signals b, c, d of these fibers is fired. That is, at ach rome rzt the net ~ ill re eive ·ust orie cau e the cess at on o R's ring, just t ecaus this would cause a to miss of the possible signals a, b, c, or tl; we will ot C< nside the simw taneofis a pu lse.

arrival of combinations of such sig ~als. (Tht y car thus be t rougl t of ~s arriving from the output of some ecod Fr ne work) pXAM PLE •

In each example, the machine oegin its c per at on w ith tt e occ urren e ::>ur s ~cone net ( Fig. 4 .1-2) .viii H cogn ze of a pulse along a special fiberS. This 1ber s call ~d th "sta rt fib r" a d the pulse is called the "start pulst " Forth pres ent w e will assu ne th at l equer ce th tend s wit! an a. J any s only one start pulse ever occurs; < nly i n sec ion ~ .4 wi II we cons 'der e A-

ceptions to this. One of the ordin ary ir puts ignal mus also occu at t e For if R f res a timt t, w con Jude that< mus t have fired at time t - l. time of the start pulse. Anc now then is si tnPIY pothi rg m' re w can say about what happened Finally, each net has a specia outp ut fib Fr R. We will sk n e sar e

"' befc re th t, fo the r f!verb l:!ratic n init iated by th S pulse does not require

question in each case: what is the signi ~can1 e of pul~ e alo gR. Mo e any parti ular nput signa patt rn fo r its n ainh nance. precisely, suppose that we observ ap lse a ong heR fiber and hat ' e ::;XAM PLE 3 do not know when the machine w s sta ted o r wha t sign als it has n ceive d. What can we say about what cou ld ha e ha opene din he p2 st his tory of he t ird r et irr pose~ a ra ther nore com~ licated constraint on the the machine? clas of st quen es th at mi ht h ve c2 used R to hre. (See Fig. 4.1-3.) We

obs rve t at th efirs (ear iest) igna 1-tha t is, the signal concurrent with EXAMPLE 1

In our first example, fibers b, ~da ren't cted to ar ythin a c, an conn ~· a -

I><~ (See Fig. 4.1-l.) If R fires at tim t, ~ e car ded ce tt at a r.red fit tin e b-1-- ) ~ b r...... t - I. Furthermore, since the thr shol1 of tl eRe II is , we ~re s1 re al 0 c - 1-- R

d--either that S fired at t - I or else hat} did. c -Th third possi Jility- that Sand R bo th j d -

;~ f red a t - l-is ruled out b ecaus befo re ~ ~ /~R R 'sun que fi ring there c ould have Jeen 0 s/ s _...,.,. d-

' ay fo Re v r to ave ~ ot sta rted. Wed e-5---- I berat ly igr ore a last possil ility- that R Fig. .1-2. Net f r Exa fnple 2: Fi~. 4.1-3. Net for Example 3: a(ba)•b. Fig. 4.1-1. Net for Example 1: a•a. las alv ays b en fir ing-i an ir finite past. 1 (a v b v c v d)•a.

70 MEMORIES IN FINITE-STATE M r'CHIN s SEC 4.1 SEC 4.2 MEMORI ES IN FINITE-STATE MACHINES 71

the S pulse-must have been an a. 0 herw seth start pulsc , al01 e, w< uld PI SCI SSIO!

have failed to fire any cell, and t pe ne wou d rer ain < uiet ver 2 fter. We h oft he ex mph ribe the class of sequences I n eac s we fvere ble t< desc

observe also that the most recen puis mw t hav beer b, el eR ould not I rec< gnizc d by hem achin , usi rg ex pressi ons ( n boxes) of a more or less

fire. A little thought will show that he st uctu e of the n ~t ca ses 1 to eve yday lang age. For nets of l reate r cor rplexity such expressions

recognize WOI ld bc come cum bersc me, mbi~ uous and finally incomprehensible. any sequence th at be! msw ith ar a, e ds w "th Kle ne's [1956 forn ulati pn de scribc s sue h cla5 ses in an elegant manner,

a b and in the interi m rer eats t he se< uenc baa ny usir g for mula he< ailed "reg Jar e xpres ions' ' and a number of asso-number (including none) of tir nes. ciat d no lOllS. The defin "tions in th e nex sect on are not precisely those

of 1 leen but are r nore ike t hose n Cc pi, E !got, and Wright's [1958)

Alternatively, one might say the: tR n cogn zes sim plifiec vers on of Klee ne's t eory 2

any sequence tt at be gins' ith a b foil owed by 4.2 REG JLAR EX PRE SSIOI'I SAN D REC:: ULAR SETS any number (inclu< ing n one) f rep etitio s of ab. OF~ EQUE NCES

Only such a sequence can pr serve the pulse reve bera ing ' ithin the Let U para phra5 e the four descr "ption s obt ined above for the sets of net; this pulse, which originate s fro n S, ·s req uired to rr eet t e th esh- seq ence reco gnize j by t he ne s of I igs. ~ . I-I t hrough 4.1-4 . oldR. I) A ny nu mber ofoc urre :1ces c fa fo II owe d by a.

EXAMPLE 4 2) A ny nu mber ofoc urre :1ces c f(a o r b or e or d) followed by a.

The class of histories define d by our fi nal e amp e is n ore ifficu It to 3) a folio\ ed b any numt er of occur renee s of ba followed by b.

describe. (See Fig. 4.1-4.) Th ·s net can becor ne ac ive c nly i the tart 4) A ny nu mber ofoc curre nces c f(a o r bed followed by bed. No N obs erve hat t hese xpre sions (and thos arising from more com-

a~ plic a ted ets) < anal be a semb led fr om a very ew terms and connectives.

c' d-.

~ We requ re, in fact, only the s ignal letter s the nselves and three connec-

r; t R tives:

\::... ' ) s of" "an\ num ber o occu rrenc \ "foil owed by"

I "'or'

5/ Ifv e ab Jrevi te th se tt ree b y "*', juxt posi ion,t and ' , respectively, v,

the abov four expr ssion s takc on t e cor npact forms:

Fig. 4.1-4. Ne forE ample 4: (a bed)• bed. (l) a *a

pulse on S is accompanied by a pu se on a or on l. Th e act vity ulti- (2) ( 7 v b v e v d)*a

mately required to fire R) can be ke pt al· ve or ly by (I) everb eratir g in (3) a (ba)*b

the cell connected to a or (2) 1: y eye ling roun d the loop form ed b the (4) ( 2 v be d)*b d

sequence bed. Firing of R can occu onl afte the omp etion of be d se- I

quence. Some thought will the r revc al th< tthe ~ring of R ecog rizes ., tjrhe te mjux apositi pn refe s to t e forn ation pf a n w expression by simply writing seve al old !;:xpres ions c nsecu ively \ ithout any st parati g punctuation. In this informal exp< sit ion fve int oduce parent eses c nly w~ ere ne ded t< mark the limits of the 'v" and

any sequence, rade ~p of any r umbc r of uc- "*. c nnect ves.

cessive or mixed pccur renee of he p ~tterr s a I is un ~ersto< d that juxta ositio is a "stron er" b nd than is 'v'. That is, a v be

and bed, which enc s wit ran ( ccun ence Df bee.. mea s a v (be) a pd no (a v )c. his i~ exact y as n school-algebra, where multi-plie< tion is denote d by ju xtapos tiona da + be me ns a (b x c) and not (a + b) x c.

I

72 MEMORIES IN FINITE-STATE M CHIN S SEC. 4.2 SEC 4.2. MEMOR ES IN FINITE-STATE MACHINES 73

We list just some of the segue ces 1 nclud ed In the sets r epres en ted by We h ave u sed v. hat n a the natic ans c all a "recursive definition." In expressions 1-4 above. thi forrr of d( finiti Jn on e beg· ns wi has at em ent that:

(1) a,aa,aaaa,aac. aa, ... BASI:: Cert ain (' prim tive" obje tsar defi itely in the class. (2) a, aa, ba, ca, de. a,ab , cca bca,

bcacdabbccbbd a abc 1aa, ... Th n the re art som rule whic h sta e tha:

(3) ab, abab, ababc. b,ab baba IJ, aha baba ab,. FIOCUR SION: !If c rtain kind s of c bject n the class, then are so (4) bcd,abcd,bcdb d, be dabcc., are certa n oth er ob ects f or me j from them.

aabcdaaaaaabc dbcda bed, ...

While it is perhaps clear how tl read, Fin ally, here s usu ally s a ted rest ictio that

ese e ~~.pres ions are rr eant o be we must state explicitly what is inv Jived We will< efine any tring of si! nal F ESTR CTIOt :IN o obj cts n ot re uirec to t e in the class by the letters, stars, v's, and parenthese s to t ear gulm expr ~ssion prov ided hat al over ules a rein he cl ss.

it can be constructed in accorc witt the follov ing ules. Eac reg 1lar expression will serve to represen ace tain f et of signa sequ ences. Su h a PR< BLE \II 4.2 1. 1 he in~ utto net i a se< uence ofO's and l's. Repre-set is called a regular set of seqt ence5. Th rule s belc w als o exp lain ow sent the n et-a a re ular xpres ion ( asy), as a state diagram (harder), "representing" works. and as a McC lloch Pitts netwo k (u ually very hard)-in each of the

folic wing ases.

DEFINITION OF THE CLASS 0 F RE< ULAF EXP ESSI< NS ""n ou putp lse oc curs w hen:

Any letter symbol xis, alone, a reg1 lar e pres.1 ion. I) th( num er of 'sin t he inp ut is d ·visibl by 3. 2) all I 's ha e bee n in bl ocks c fat le st 3.

It represents precisely the set cor sis tin ofth e sing e (one -letter sequ nee x 3) no I has occur ed at· time divisi le by or 3. If E and Fare regular expres ions, hens o is ( 1 F). 4) th( re has been neve n num ber of block of I 's, each of odd length. The set of sequences represente -1 by ( EF) is obtai 11ed as folio ;vs: ClOOSe any

sequences 1 from the set represente d by 1 and ny se quenc e s2 f om tt e set rep-resented by F. Then the sequence f ormec by a tachir g s2 t ) the nd of s 1 is se- 4.2.1 Re cursiv defir itions and ir ducti e pro )fS

quence in the set represented by (El ), and only uch s quen< es are in the set. Then otion of re cursi e de nitio n is i mpor ant in its own right, and If, E, F, ... , G are regula r exJ ressil ns, then 50 is the e xpres ion

will b d wit it a ain I ter i the book. Therefore it is ap-we e con cerne (E v F v ... v G). pro priate here to ta~ e tim e for a mo e de ailed study of what is involved.

E vI and only ' The set of sequences represente d by v .. v G) conta ·ns al Let us lo ok ag ain a the ecun ive d finiti on of "regular expression,'' this

those sequences which are already i any fthe ets re resen ted by E or For. . or with

G. (That is, simply form the "union "oft ose S( ts.) tim outs J muc h con cern bout its in erpr tation. Let K be the class

If Eisa regular expression, t en sc is E ofr gula expr r>ssions.

The set of sequences represen ted b E* is ob tainec as f ollow : le SJ,

s2, . .. , s k be any collection of seqt ences of th set epres nted t>y E. Ther the EASE: Any Jette sym ol a, b, c,. .. is a n exp ression in K.

sequence obtained by stringing the mall n a r w is seqt ence :>f E* and only FECUR SION: If E and F are e ~~.pres wns n K, hen so is (EF).

such sequences are. Warning: We perm "t also the c se of no se quenc s at II- If£ , Ez, .. ,E n are xpre swns in K, then so is

that is, we allow k to be zero. Th' s lead occa sional y to orne neon\ enien e in (EI £2~ En}

that we have to talk about the "null ce"- these quenc e ofn sigm Is at a ll. t ... v seque

The regular expressions are 'lll th se de fined by th abo e rul1 s, an 1 no If E san xpre SIOn nK, hens o is E*. others. t F ESTR CTIOt: On ly ex Jressi ons g nera ed b v the above rules are

The regular sets of sequences ' re all sets f seq ences de fin ed by the a t>ove io K. representation rules.

tOne can think of E• as: (null sequ nee) v EvEE vEEE v ... Very often, om doe· not stat the restr iction explicitly, it being i But we allow omission of parent eses \1 hen n amb guity an res ult; e. ., we write und ersto d th< tom IS n Jt co cern d wi h "r cher" systems, or "exten-

EFG for (E(FG)) or ((EF) G). Why is t is per nissibl ? sior s" w ich n ay h· ppen to co ntain K.

74 MEMORIES IN FINITE-STATE 1\- ACHil' ES SEC. 4.2.2 SEC. 4.2.3 ME MORIE S IN INITE-STATE MACHINES 75

The terms "recursion" or " ecun ive" re w ed in refer nee o the fact I- ere s a ecur IVe efini ion .vhich gen rates just the class of that, in such a definition, the n2 meo the c lass t eing jefine d rec JrS W thin pare nthes s stru cture that are g amm a tical. the definition itself, in an essen ial w< y. In the p resen case we a e defi ning a certain class K. The term K appe rs in the B ASE p art o the jefini ion,

F ECUF DEFil' ITIO/'. LASS P but not in a circular way; here we o nly S< y th<: t cer ain t hings are i n K, SIVE OF HE C

and one does not already have to k ow a nythi ng ab out/<.. to u nders and this. But in the RECURSION p rt of the ~efini ion, bne s ays t nings like B SE: ()is i n P. (Po)

" ... if E is in K, then ... ," an ~ her it a ppear s tha one has lreac y to R CUR~ ION:

know something about K~the thing bein! de fir ed~ 0 ffi( ke se rse 0 the (1) If Ei in P so is (£). (PI)

definition! Thus there is son ethir g ab but a recu sive defin tion that (2) If E a ndF ~rein P, so is El. ( Pz)

suggests that it might be "circ1 Jar," and he net som ehow unsa isfac ory. R STRH TION No hing lse is in P.

Indeed, this can easily be the case. Thu s, if fve WI re to Offil the ~ASE 1 his g part of the above definition of '-', tht rem ining de fir ition woul ~be c uite nera es, ar pong pther , the ex pre ~sion

useless, for one would have no f¥ay " hate' er to ell w jlich xpre sions are, ) in fact, in K and which expressi pns a e not ()) )()

In later chapters one of 01 r cer tral 1 oals ~ill t e to stud) just this ( ()) ) () ()) (( )() ()()() () (()) question of when a recursive d finiti Jn re lly \\ orks and v hen i tis d ~fee- ( ( () ) ) ( () ( ))) (( ) ) ()) (()()()) (()(())) tively circular. For the present it w II be usefu to s udy i n son e det ail a ())( )) ())( )() () )( ()) ()()()() few examples of recursive defi nitior s wh ch tL rn 01 t to be sa isfac ory. )( ( ( ) ) )(() ()) () ())() Our first example is taken fro m th e exp res sic ns o elen enta y set ool- ( ()) () ()() )() algebra.

whic h are all th well form ed str uctur s of p to ight symbols.

4.2.2 The set of well-formed par enthes is strir "t-ow I t us eturr to t e qu stion of h w to decide when a recursive gs de fin ition realh defi es sc meth ing, ' nd is not fatally circular. We will

Consider the expressions u ed in elem en tar algt bra. If w e exa mine not ry to give gen raJ a nswe1 to tt is qu estior. In fact, as will be seen the expressions in Ia er ch pter , the e is n o ho e of fin din gag neral answer. However,

(a + b) ):z + in p2 rticul ar ca es w can often solve the probl m. Here we can give a

a(b + c(d + e)) t<(b + c(d -+ e) simp le pre cedu e by f¥hich onec ante I whe ther given expression is or is ' noti n the class P. Tjlere ~ny v ays t p do his, but the one below is (a + b)(c + d) )~ + )l (c + d( rem

certa inly t neon that invol ves tt e lea~ t effo t for many expressions. a(b + (c + d) + !:>) c.)b + (c + d + t( J

we observe that those in the fi rst cc lumr are rnean ngfu whil e tho e in 4.2.3 The meth d of hesis-c ounti g the second column are not. 0 e ca thin k of his d stinc atter

aren IOn a am of grammar; the expression~ In the econ b co umn are some how c ur p oced re is base on he c nstn ction of a certain machine to "ungram rna tical." reco mze whicl par nthe is str ings re w 11-fo med~that is, are in the

Parentheses are punctuatio mar ks wt ich a esup posec to b pair d to class Pde ned bove . Tht mac ine' e USt ism ~ch like those considered mark out phrases or clauses. n the seco nd cc lumn it si j:nply isn't clear in fo rmer chap ers, c xcep that it ha s an infini e number of states: (see what clauses, if any, are defi ned. If v e ab tract out the aren hesis Fig. .2-1 ). structure alone, we can still rna keth disti nctio , for in eac h case

() ) Sta rt ............

(()) (() '\_ .J-rt_ ~( ~( .... ()() ))(( ~ ~ .{q1 lout q2 ~ 3L 1 q4r: •••• (()) )()( \I

--' ) ) ) )

we can see which are "well-fon ned" and v hich are n t!

~ Fig.' .2-1. The p· renthe is-cou nting n achine M.

76 MEMORIES IN FINITE-STATE MA CHINES EC. 4 2.3 SEC . 4.2. IVEMOR ES IN FINITE-STATE MACHINES 77

For any particular example,< nly a finite num Jer of state will be us ed, He 1ce so we need not discuss now what ISlill Jlied Jy ha ing n inf nite nachi ne;

Ql[( E)] = Q2[ this is a topic of later chapters. We will ee, d ough , tha we ave 10t )] = Q2[)l =·ql

gone over to infinite machines or fr volot s rea sons; there is in fact no So finite machine that can handle his j b, a leas for the ~ hole class of If E is in ~,so s (E) I expressions made up of parenthe es.

Now let E be an expression rr a de p of paren these . Dt fine he fu nc- Ne t, su ppose that E an d Fa re in C. hen in computing QdEF], the tion Q ;[E] to be the state that M will end if if it is sta ted i 'l stat e q; t nd rna hine retu ns tc stat ql fter see in! E( because QdE] = ql) and given the sequence of symbols J rom E. We ~ ill p ove he f :>llow ng ret rns fi nally to sta te q 1 fter eeing F(be cause QdF] = q 1). Hence theorem. ;

l If E" and Fan inC so is EF.\

THEOREM Now ' the tt ree s a tern nts i box s are exact y like the three statements

An arbitrary expression E is ir Pif, ando n/y if, it ha the p roper y that in here ursiv e defi nitior of l (sec ion 4 2.2). Can we conclude that the

~I[E] cia ses C and P ar the same. N< , but only because we have not yet

= ql pro vent eRE TRIC ION taterr ent. We< an, t owever, be sure that any-Note that this is a very strc ng tt eore n; it tells us nc t on y wh ·ch thi gin Pis al so in r. th

'""' t is, t he cia ssp i con ained in the class C, which expressions are in P but also w hich are n ot. The roof is in truct tve is~ hat~ ewa ted t :>pro e he e. Now w e hav e to prove that the class C because it shows how one can rei ate th e stn cture of a recur ive d efinit on IS C ontai edin the c lass l. (here, of P) to the structure of a nach· ne(h re, M ). Proo that if E is inC, then Eis i P:

Plan of proof Define C to be he cl ss of expn ssion wid the :>rope rty This s the more inter sting less Jbvio us, pa rt. We need a new method

Q 1[E] = q 1• Wereallyhavetop rove WO d ings, first t hat to eal ~ ith t e rec ursio , an< we hoos to Jse mathematical induction

sin l on he /e ngth if ex press! ons. Wha t we will do is to show that if the

If E is in C, th en E · the or em is tn e for all e of le s tha n a certain length, then it pres lOllS

and then that mu st be true or ex pressi ons o f tha I eng h. A n "induction" is a simple

if E is in P, tht n Ei inC for m of ecur ion. Let u s defi ne fer gth [ S'] to be the number of symbols m

Then it will follow that both clas es m st be the s me. I BASE: If le ngth E] = 2 an dE i in ( , the n E is in P. This is true,

Proof that if E is in P, then Q I[E] = ql; that i s, Eis inC: sm e ins pecti< n sh JWS t hat tt e on y exr resst n which meets the condi-We simply prove that the pr opert v hoi ds fo ever ythin in ", rec ur- tio sis ( ). Ar d thi is in Pbec a use Jf the BASE ( P 0 ) of the definition of P.

sively. INDU TIO!\ : Ass ume that we h ve s hown, for every expression of BASE: Qt(()] = ql. This is seer, sin ply t:>y p tting () nto he len gth /e s tha n k, hat if it is ·n C hen i is in P. Now consider any ex-

machine, starting in state q 1 • He nee pre ssion E of engtt k. We ca n sep a rate the s tuation into two cases. In t

l C. j the first ase i is p Jssibl to t reak E int o tw< parts F and G such that

) is i t E FG and l oth and G ar in C. In he se cond case there is no such

RECURSIONS: Suppose Qdl ]= I· 1 hen TVe kr ow t at st ate !J is par tition of E.

never reached if the machine is s tarte< in s ate q 1 and give 11 seq Jence E. CASE I. his i trivi al. For if E' = G an d both F and G are in C,

It follows that the n bot hFa 1d G are s orter than k (th e len th of E). Then by the in-

by translating everything up on~ 2[E] du< tion hypo hesis bott of t hem are ir P. Hence (by the P 2 of the = q2 def nitio of l ) it fc llows that E" mu t be i n P. unit to th righ . N< te al o tha t for the

sequence (E, we have CASE II. fEi inC then Qd ]= ql. I there is no partition of E int two shor er ex Jressi ons b oth ir C, v e car conclude that the testing

Qd(E = Q 2[E] = q2 rna hine does not r -ente stat e ql ntil t he en d of E. Therefore, it must

78 MEMORIES IN FINITE-STATE MACHINES SEC. 4.2.

enter state q2 immediately aft r the first symt ol of E, and it must ente state q2 on the next-to-last symbol of E for i Is o~ly from t ere hat i can get to state q 1 on the last symt ol of E). t follows t~at E m us have theform(F)andthatQ 2[F] 1= q 2 • No\ obscrvet atttema hineneve enters q 1 during the comput< tion pf Q [F]. The1efore if "e stat tht: machine in state q 1 , with inpu F, v e car com Jude lso tllat ~ 1 [ F] = q I (for the machine structure always lc oks the sane to the r"ght c f the start ing point, and we know that t is comput tion does not e cour ter what is to the left of the starting stat~:). Hence, by d finition, F is in C. Then since F is shorter than k (the length of E), mu t be in P, and hence (by P1) so is E = (F). This concludes t e pn of that if is ir C, t en E is in P and, together with tht: pre\ ious ection, th pro f th< t E 'nd C are the same.

REMARKS. We started by defini g the class P of well-f rmed pare thesi~ sequences by using a rathe simp e rec rsive ~efini ion. ~e shpwed hat i was possible to "recognize" or 'decic e," w~ich ~equen es bdong o this set, by using a simple sort o "counting nachi e." he use oft is machine, or "decision procedure," s ver stra ghtfo ward; one simply feec s the expression into the machine and observes the >tate o the rr achin~ at the end. There is no question about how long tl e machine's com I uta tic n will take: its number of steps is just the expr ssion s length.

One can imagine other ecurs ve de nitio s for which it wo~ld be more difficult to find such a dec sion rachi e-o e wh ch we uld d~finitely tell which expressions are in and whi h are out o the c efinec class As ' ill be seen later, there are even ca~ es in' hich ·tis impossit lefor any s ch decision machine to exist. At best, in sucl case , we can fin~ on I) a rna hine ~hich will sometimes decide, bt t will othe tim s get invo ved ip a everterminating computation. ntici ating hese evelopmen s, we an as here why things came out so we I. Th ansv er is hat in the c se of the class P, the recursive definition has ne very im ortant spec al pre perty Eac time an expression is admitted on the oasis c f oth r pre iousl adm "tted e pressions, the new expression as gr ater ength Tht s (£) is lor ger t an E and EF is longer thanE or 1 . This fact allow us tc work back .vards, since for any expression we nee inv< stigat only a lir ited lass f po entia I ancestors. This is reflected in th fact that ~e we e abl to cpmple e the proof by an induction on th lengt~ of e pressipns.

There are proof methoc s for makir g dec sions abou the e quiva ences of classes given by differe[lt rec ursivt defir itions using me hods more general than our simple le gth argum nts. he i terest d re der snould study the method of "Recursion Induct" on" d scrib d by McCarthy [ 960]. If we had described our test ng m chine by a ecurs ve dellnitio instead of by using an informal state c iagra n des ription, we could have proven the theorem by an application o the I< ecurs on In uction met od.

Curiously enough, the clas C o expr essio s is 1 ot itself a egul: r set of sequences. As will be seen the) canr ot be reccgnize~ by any f nite-

. !

• i

•.'

'i''

'I'

l

\

SEC. 4.3 l\1 EMOR ES IN FINITE-STATE MACHINES 79

sta e machine. We ave just shown hat t ey c n be recognized by a very simple ir finite-state mac~Iine. Wh~n w tun to the study of Turing rna hine , a variety pf infinite r a chi es, "e will use this class as one of our firs exan pies.

V\ pn of b1 "inc uction" is relat d to a recursive definition of a collect" on o "nu(nben d" q antities or obje ts. ~ uppose that we have a set of bjec s, ea h somehow as ociat d with a number (in the situation above, each e:x.press on is associated with its length). We want to prove son e sta emer t abc ut all the objects. f we can prove the statement sep rate) about the objects aswciat d with each number, then this proves it fc r all he ob ·ects. The scheme ca led "math rna tical induction" has as its! oal p ovin that

BASE: 1 he st teme nt he Ids fpr al objects with t e nu (nber

INpUCT ON: f the tater ent I olds for all objects vith numbn n, then i holds for all

< bjects with number n + I.

Thi clea ly establist es the statement step by-st p, for each number.

PR( BLEM: W at is rong with tP,e follpwing proof?

Propo ition: All n arble have he smrze color.

Con ider < cont iner • f rna bles. We I rove hat any handful of marbles are he sa [ne co or. 1 ASE: f we ake I mad le, then this set of marbles cert< inly has only one color. INDl cno~: Taken + I marbles. Choose n of tl em. y inductive assur ption we c n su pose these are all the same colo . No .v replace ore oft hese n marb es by the extra marble. Then the extn mart le mt st be the s2 me cc lor a the others, in the new set of n mar les. So alln arble mus beth same color

4.3 KLEE~E'S HEOAEM: F NITE AUTOMAU CAN RECC GNIZ ONI Y REC ULAR SETS OF SEQUEI~CES

11Ve nc w re! ume he study c f the capa ilitie and limitations of finitestate mac ines. We on sic er a tinite machine A-: which starts in a certain stat~: Qinit and ask, " N'hid sequences of in ut symbols cause M to end up in a certain st< te Q n ?" Any equence v ith t is property is said to be "recogniz d" bv the mach in e. The set of all such sequences is the set of sequ nces recognized by M. (0 cou se, b di~ rent choices of Qinit and Qfin the arne macl ine can r cognize a nun ber of different sets of

80 MEMORIES IN FINITE-STATE SE . 4.3

sequences.) The Kleene theo em s ates hat he se s so recog izab e by machines are precisely the regu/ r set of s quen es.

We divide the proof into t o pa ts. In thi secti n w tha any set recognized by a finite-state mac ine is regu ar; i secti we how the converse-that any regular set c n be ecog ized by so em chin

The proof (which is essen ially hat f Kl ene) is ~ sed n ~ thematical induction on the num er o stat s of he ach1 e. hat 1 , we show first that if the theorem olds for a I rna hine with n states then it holds for all machines with n I s ates. We how also hat i holds for all one-state machines. It f llows that the heor m is pro ed f r all machines. The proof is bas d on arg ment ab ut t e set of aths through the machine's state di gram from Qinit o Qfi .

It is actually easier to prov the theorem f r a s ightl pre-hensive class of diagrams tha just thos for mac ines; rgu ents hold also for "incomplete" di grams in hich any f the arro s m Y be deleted. This is an important phen mat emat'cs- hat i can be easier to prove a more gen raJ t . T e pr foun pro lem is to find the right generalization!

Consider a diagram comp necting some of the points. e r quire (tempora ily) t ere most one arrow in each direct on b twee an pair of p ints. Let each

Fig. 4.3-1

arro ca ry a diffe ent I tter abel. F o example, co side the iagra of ig. 4 3-1.

w co side pat s fro on vert x to the iagram. A path from x to is a seq ence of ar ows hie beg'ns at

vert x x nd e ds t ver ex y W pia e no additiona restrictio s on hat · s a path; i the exa pie, abca cdcd dgfj hhh repre ents a legiti ate ath f om 1 to Q4 • Now give

Qy (whi h m y be the ow t at t e set of a! sequences is represe ted y a egul r ex

pres ion n th lett rs I belli g th dia ram. The argu ent brea two cases depe ding

on whether Q x and Q v are diff rent. We denote by R xy the set f all path Q x o Q . W rna e an

important definition:

R~y = the set of all path fro x t y which enter and lat r lea e ag in th case that Q z • s the same as Q R~y contains aths hat egin

MEMO RIES I FINITE-STATE MACHINES 81

e th t to z in he interior of the

say t at

Rxx (R x)* (I)

= RY,yRyy) (2)

~Y. uch set i cerned with a new diagram Q" s mo e or ess e 1mm ted. If we can describe the

s o a di gram with a sta e del ted, we will be able to use hyp thes s; th t the sets of paths between vertex pairs in

s aller diagr ms re re ular. Let ab b the label for the arrow between Q and Qb, i ther is on . 0 herw seth symbol Cab is considered meanin less. A Ia el C a is lway mea ingf I; we will interpret it later. The k y ste in the arg men is to obse ve that th set R~y is composed of the p th C y (if ny) toget er ith a I me ningful sequences described by ex ress'onsofthe orm.

(3)

. W y is this true? Because any m x to Qy which is in R~y ust go either directly from Qx else hrou h so e c ain f stat s which does not touch Qz in its

in erior. Th re ust b so e state Q whi h is the first in this chain (a ter I avin Qx) and there mus be orne state Q. which is the last (before reaching y); th rem inde oft is ch in must be in R~ •. Observe that if ach et Rz n so enti ned n (3) is re ular, then so must be R~y, for it is com osed f these an the 's b 'v' a d concatenation.

The final step: by th ind hyp thes· s the sets R~. of (3) above m st b or ea h is set f pat s bet een two vertices in a graph w ich as fe er v rtices than occu red i the original. In fact, the graph is that btai ed b ving the ertex Q z t gether with all its arrows!

e set (3), (2), · nd (I are all regular and, hence, so are ave nail to s ow ( he induction base) that the

82 MEMORIES IN FINITE-STATE MACHII'ES SEC. 4.3.1

theorem holds for the one-vert x case. B t the set of seqt ence fron Q 1 to Q 1 is just ( C 11 )* which is regt Jar if C 11 is.

To apply the argument to t e sta e diagram of n achi es, \\ e ha\ e to interpret the symbols Cab prop rly. If Q and Qb a e dif eren mac~ine states, there may or may not be a y ar ows leadi g di ectly frorr Qa to Qb. If there are none, we simp)~ om't an' exp essio s of forrr (3) which contain a Cab term. If t ere 're se era) uch rrow , lab lied .vith letters a 1, ... , an, we repla e the synbol Cab by he expression (a 1 v ... van). We must treat the symbols c. diff( rent) . If there are arrows leading from Qa direc ly to Qa as in the case of C44 of the example), we replace Caa by the expression form d of he associa ed le ters connected by v's, just as for the Cabs. ( n the exar pie, C44 is repl ced simply by h.) But even if then are no atrows from Qa tack to Qa we must not eliminate entirely th cor espo ding expr ssior s of form (3), because an expression like

still can represent paths. In sU<h a case we simply remove the< symbol, obtaining the expression R~nCny, and simi) rly if the r ght-hand C has two identical subscripts. Carrying ut this pn gram, we tind that a I the C's and R's are eventually eliminat d, yi ldin1 a re,ular expression usin~ the input symbols of the machine. he base o the nduc ion- that the st t of signals recognized by a one-sta e machine is re ular c-is t ivial since this set is just (a v b v ... v z)* Wlhere ~' b, ... , z s the set o inpu symbols to the machine.

4.3.1 An example

Suppose that we wish to con pute R 11 f< r Fig. 4.3- . W( begi witl:

l11 = (Rll)*

using rule ( 1 ). Then we expand R l 1:

R :1 = c 2R~3c 31 v crl3R1 C31

using rule ( 3); one can go only to Q or { 3 dir ctly rom Q 1. Since C 12

is a, C 31 is c and C 13 is d, we car com pine tpese tp obtfiin

Next, we expand

R 11 = (aR~ c v d~13c)

c: I Rl Rl2 ,, 23 = 22 23

using rule (2); but R ~2 is null, so simply

R~3 , Rg

SEC. 4 3.2 MEM JRIES IN FINITE-STATE MACHINES 83

r ext, .ve expand R ~L v ritin in tJ: e val es of the C's as we generate them:

R 12 b'1 1 12 eR ~~ 1 eR 12 bR 12f 23 = " 33 v 44J v 43 v 34

and continu'ng,

RH = RW)f" = (gR,U3 )* = (gh*f)*

R,U = R,U4) = (11 v f1 U4g) = (h v fg)*

"here he te m h = C4 is non-trivial and has to be included. Similarly,

R~j = R,U ~~j4 = (h v fg)*f

~u = RH ~u3 = (gh f)*gh*.

E acking up, this~ Ives

R~3 i= b(g'fl*f)* v e(J v fg *f v 1 (h v g)*f v b(gh*f)*gh*f

V e an not s urpri ed to find erms dupli a ted since the same subgraph of ~ 3 and Q4 h s been app road ed in seve al ways. The above expression is e uiva ent t<

I rem2 ins tc expand

Hence, finall , we get

1 ~3 = b(gh f)* e(h fg)*f

R13 = (RB)*

= (g~.\lf gR! )*

= (g *fv ~h*f)*

= (g *f)

R1 = (t(b(gA*f)* v e(h vfg) f)c v d(gh*f)*C)*

P OBLEM 4.3-1. xplain in "ords he significance of the expressions in ea h of he ste s abo~e.

PI OBL~M 4 3-2. ind t e reg lar expressipn for R 14 in the above net.

4 . . 2 emar s

The e is o restricti< n th t the Jette s m he machine diagram be all different. Let us relabe the iagrc m of Fig. .3-1 so that just two letters ai pear Fig. .3-2). Sirr ply ty substitut'ng tt ex's andy's in for the correspon< ing )( tters of the old c iagram, W( obta · n the regular expression

f. 11 = (x(x(vy*x)* v y y v y)*X x v y(yy*X)*X)* (A)

84 MEMORIES IN FINITE-STATE M ACHII\ ES SEC. .3.2 s c. 4. MEMC RIES I N FINITE-STATE MACHINES 85

lJ f\ Kle< ne [I 56, p 24] ~ives he fc llowir g results, among others. We

q ote:

~ "AL f->EBRP IC TRJ NSFO MATI( NS OF REGU AR EXPRESSIONS. We list some

~ e< ualiti s for ets o table . (\V e hav scar ely begun the investigation of e< uival nces.)

;~ (I) E E = E. (2 E v F = F v E.

) (3) ( E v F) G = Ev F v G). (4 (EF)G = E(FG).

~ (5) (E * F)G = E (FG) (6 (E v F)G = EG v FG. (7) E( rvG = E F v E'J. (8 E * ( F v G) = E * F v E * G. (9) E* F= vE EF. ( 10 E*F=FvEE*F

(II) E* F = ps * (f v EF v E 21 v .. v Es- 'F) (s > I).

TJ prov e (I I) we ha e Fig. 4.3-2

s-1 s-l

for the set of paths from Q1 b< ck to Q1 i the new c iagra m. 1 his i the ' E*f =L: E"F =I L: Esq+ r F'=~ ""' Esq L E'F."

diagram of a genuine finite-stat e rna< hine, since each lette occu ctly n~ 0 q~ r~O q ""'o r::oO

rs ex once at each vertex. PROBL EM .3-5. Whict of th folio wing a re true?

This is not the only regula exp essio n tha t can repn sent he st t of ~',

paths in question. Each regula set as m any r egula exp essio 1S. T here E* F = ( EvE *)F

is a whole algebra for regula essio ns, w hich allow s rna : ' E*f * = ( Evf )* (E F)* exp ny t ans-formations and simplifications.

I E*f * = *EF vE FF* (See prob em 4. 3-3 b low. For ex an pie, we can simplify the above expn ssion for R II all hew y do .vn to

(1 v F * = ( E* v F *)(I * v E* )* E(FC E)* F G= F(Gl F)* (

( (XY* vy (YY* )*X) (B) PIWBL EM 4 .3-6. ~how that g 'vena ny re~ ular set E, the set consisting of

This expression has an interpn tatio :To get f oms ate { 1 bac k to tate the men bers f E, e ach n verse< in ti me, is also a regular set. (Prove by

Q1, one has to go through state QJ. here are t ,VOW< ysto get tc stat< Q3; u ing th recu sive d efiniti Jn of egula expn ssion.)

these are xy* x andy. (This is ot sc easy to se , in tself. On can then delay returning to state Q1 on y by shutt ing t etwet n sta tes Q 3 and Q4;

·,

yy * x arises from each such c cle. Fin a ly on emu st ret urn t J Ql; this KLEENE'

requires the signal x. It would Je qu te ha d to prove that A) a 1d (B 44 S THE OREM (con inued : AN~ are

the same without inventing a se t off< rmal trans form tion ules. REGULA ~ SET CAN BE RE COG II I ZED BY SOM E FIN TE-ST !!.TEN A CHI liE

PROBLEM 4.3-3. Show th t the ets re Jresen ted by We now Jrove the c onve se of the t eore n of 4.3; we show that there

b(ab v )*a and bb*a bb*a * e ists, or an y reg ular. et of seqw nces, a fin 'te-state machine that recog-

are the same. Use this to e tab lis h the equiv lence of (A ) and (B) a JOVe. n zes p ecise y tha set. This (com Jined with the result of 4.3) shows that

Realize that the statement are bsolt tely t ue, a nd de not depen d on tt e set. reco gnize by mach nes 2 nd tt e set represented by regular ex-reasoning from any paitic lar c iagra n. I went some tran form tions p essio ns, ar co- xtens Ive- hat t he nc tions of regular set and set recog-between equivalent regular < xpres ions nd d scove a co respc ndenc e be- n zable by so nem chim are< quiv<: lent. tween them and some transf< rmati pns o state diagr ms. Fore~ amplt , one Ou met! od is to sh pw tt at an ~ set epre entable by a regular expres-might justify each step of b ( ab v L )*a , b(b v ab) a = !J(b*a b)*b' a = sipn C' n be recc gnized by a 1\ cCu loch- IPitts machine. Since any bb*a(bb*a)* 1\ cCul och et is a fini e-st a e rna ~hine the heorem will be proven. We

PROBLEM 4.3-4. Interpre nalysi s usin ~ rela ions ( I), (2) and 3) of c uld ( with some diffic ulty) cons ruct he st ate diagram of the machine our<

4.3 in terms of operations wl ich di sect 5 tate d agra~ s, spl ·uing ach' ertex d recti , bu we will eave this as a ex<: rcise. Anyway, smce the

into several according to hm man arro fvs lea eit. 1\ cCul loch- Pitts et is a sp cia! ·ase pf tin ite-state machine, we get a

86 MEMORIES IN FINITE-STATE ES SEC. 4.4.1

stronger result. (For, showing that ny r gula special kind of finite machine is a s rong r res that any regular set can be re lize as

set an e rea ized by a It th n o e which s ows the ul/ s t of nite- tate

machines.) We will break

concerned with details revisin the eliminated.

4.4.1. Realizing regular expressi ns w th

instant-OR cells

We will build up network , rec rsive y, to reco nize regul r se s or sequences, using the structure f the asso iated regular ex ressi ns. First we have to realize the single letter s: T e ne s in Fig. .4-l realize the sets represented by a, b, t we have o pr due nets

a' s/~

Fig. 4.4-1.

forEF,E*,and(EvFv ... G) individual letters that appear we simply combine the output oft -the instant-OR cell (Fig. 4.4- ).

The instant-OR cell acts lik our usual thre hold- I cell assume that it introduces no tra smis ion we can tie different output fibe s to ether our original requirements on nite- tate ach · nes, nate it before we can claim to ave prove our heor m.

the G), ent

To realize the expression E , we have only to tie toge her t e ne s for

a b c

s

Fig. 4.4-2. Net for(£ v F v ... v ).

s

bl---1---+--+-.. Chr++--+--lo..

Fig. 4 4-3.

s c. 4 .. I

and F ser ally

RE ARK

Th

MEM RIES N FINITE-STATE MACHINES 87

ig. 4 4-3. The idea ere is that the output of the or F. Thus the F net should

eque ce o curs to pr vide a start pulse for F and uenc occ rs to prod ce the F output. There is a

at

an E. se-

ne. uenc rna e up of a number of E sequences

* recognize. trie to work without the notion ard ndeed to prove directly that

it s ea ier t theo em han a

88 MEMORIES IN FINITE-STATE M CHINES SEC. .4.2

Fig. 4.4-6

EXAMPLE

We realize a net for the exp essio

(av *h)*

The net for C* is shown in Fig. 4.4-5. Figure 4 4-6 gives t e ne for * b; and Fig. 4.4-7, the net for a C*b. Th co plet net for ( v c b)* is shown in Fig. 4.4-8.

s

4.4.2 Elimination of the instant-

way. It may

(!) (2) (3)

a

b

·c

s

IES I FINITE-STATE MACHINES 89

Sig als a rivin fro OR cells are i effe t transmitted directly from ts of those cells. An input o an R ce I is again either

Fig. .4-9

1) estatfibe, 2) a 2 c II, o 3) e of noth r OR cell.

wor bac through the net to find all inst ntly each its lower input, ignor-

b

c

s Fig. 4.4-10

90 MEMORIES IN FINITE-STATE ACHI ES SEC. 4.4.2

The new net has no advan age i wir ng si plic'ty o in numb rs of cells, but it does have a decide adv ntag in or erli t sho s th t we may realize any regular expre sion by a net ade p e cells like that shown in Fig. 4. -11. No

an extra terminal delay of one

utpu . s·milar y, i ge era! ase f n I wer nput we need on! set he th esho d to + 1, and give the uppe fibe n br nche . If e d this for the exa pie,

any regular set c loch-Pitts networ wit extra time unit.

There can be no way to limin te t e ter ina! OR nit, extra delay, if there is a fiber eadi g dir ctly rom the s art output, for we are not all owe to ie fi ers t geth r dir ctly. (Th re is some question about what s ch c nne tions coul me n. ndee , in Kleene's original discussion th s wa eire mve ted y de ning '*' to be a binary connective so that an xpre sion like * c uld ot o cur e cept within expressions of the form E *F. See Prob em 4 4-1.

SEC. 4.4.2 MEM RIES IN FINITE-STATE MACHINES 91

any regular expression E, we can rtain cell fires (with delay 1)

his ith 'v', the regular expressions of ach of tho e sta es to obtain the appr pria e regular expression for that ell's nd t is co pletes our circuit of proofs!

ao

c)* = (a v b v c) 0 a v a = ( b)(ab * = ab) 0

) *be = ( v be ) 0 bed v bed

92 MEMORIES IN FINITE-STATE MACHINES SEC. .4.2

Fig. 4.4-14

PROBLEM 4.4-3.

s~~------------~

(2)

d for urthe illust ation

(ab)* = ab) 0 null

In building up th nets for re ular' sets, can se f r £" the n t in ·ig. 4.4-14.

n the e can be no ber l ading from start utpu , and he p oble disc ssed · bove

s not · rise.

aves hese Obvi usly, o finite rna hine

ber f star pulses. If o fun tion rope ly, it som old c tart ulses ss of histo ies. hus,

Fig. 4.4-15. Stages in th const uction of (a b*) *: INSTAI'T-OR cells, 2) col ccting the i puts, (3) re placement w·th Me ·ulloc -Pitts ·ells.

s c.

3 a 4

b 5

MEMORIES I FINITE-STATE MACHINES 93

a 6

a 7

b 8

a 9

b 10

a II

ulses at times I, 3, 4, 6, 7, 8, 9, 7, 9, · nd II pia one role, while 8 and 10 play

with espec to th equi alene classes of histories. The start pulses a time 2 an 5 m y as ell b forg tten the machine's state should not s ow a y tra e oft eir ha ing o cur red. In working this out, one finds that t e (ac essibl ) stat s of he ne work achi e correspond to certain subsets o occu renee of th lette s oft e reg lar e pression itself Using this, one c uld g dire tly fr mar gular expre sion t the state-transition table of an e uival nt fi ite-st te rna hine, without going through our construction of a inte medi· te Me ullo h-Pit s net ork.

ressi n (a* b *) *, we obtain the . 4.4 15. As is shown, when the

Fig. 4.4-16. Simpler net for (a*b*)* = (a v b)*.

It i inter sting to no e tha no inhibitory connections appear in any of ese ets. his i beca se the signals ar decoded at the start-distinct

timuli occur as sin le pu ses o disti ct fib rs. The situation is the same as t at in secti n 3.4; the a sence of n n-mo otonic elements is something of

n illu ion si ce th yare eally mplic't in t e input source decoder.

be s en th t the e nets have the "strong recogni-tion" ction 4.4.1 that for each regular set of equences, t e cor espo et will rec gnize any sequence of that set hich ccur betw en a star puis and the present time (minus 2). As oted n pro !em .4-1, he st tes of the n t, vis-a-vis the start pulses, cores pond to c rtain subse s of l tters ·n the regular expression. Perhaps the est w y to s e this is to sk on self: hat is the meaning, with respect to the roces history, of he fir ng of ne o the cells in the net? Observe that each ell co respo ds u iquel to a letter occu renee in the regular expression. oncl de th· t the ring just what is needed to represent

he ess ntial eatur soft ulses.

ROBLEM 4.4-5. Arg e that there an really be no trouble resulting from llowi g the insta t-oR cells f section 4.4.1, so long as there are no in-

94 MEMORIES IN FINITE-STATE MA HINE s c. 4 .. 3

hibitory connections allowed n the net. Furthermo e, no trou le co ld occur even with instantaneou in hi itory conn ctions prov· ded t at th re

exist no clos d sig allo ps wi hout ny d lay. he rules for net form tion iven i Cop, Elg t, and Wrig t [19 8] ingeniously revent the onstr ction of an logi al pa adoxes such s that repre en ted by th net · n Fi . 4.4- 7, which uses dead y co bination of non- onot nic and delay-free el ment . (C. secfon 3.1, re ark ). ne cannot assi n an cons stent mean ng to the espo se function oft is net.

4.4.3 Remarks on regular sets

Fig. 4.4-18. E AND NOT F. Fig. .4-19. E AN F.

SEC. 4.5 EMOR ES IN FINITE-STATE MACHINES 95

a -+----1--...! b-+--+-!Ool c-+--+--...!

4.4.4

Fig. 4.4-21

uires at least one star to 't. The net of Fig. 4.4-21

Questions such as how rtain kind of sets, are quite difficult.3

ir cia sic p per, McC lloc and itts [1943] make some observatio s ab ut th con eque ces, or the the ry o knowledge, of the propositi n th t the brain is co pos d of basic lly nite-state logical elements. An eve tho gh t is may n t be preci ely he case, the general conclu ions rema n valid a! o for any finite state or probabilistic machine, pre uma ly in Judi g a rain. The obse vatio s are: (l) the inclusion of disjuncti e rei tion (the OR o 'v' c nne tive) eans that the description of prev ous s ate anno be c mpl tely eter ined from the description of the pr sent tate, and ( ) the cycli ity ( rom the presence of cyclic loo s) m kes ·t impossib e to st w en, in the past, the initial sti vent occu red.

" his i nora c unte part f the abstr

ow le ge u eful. "t

brains, is the h renders our

he ding ecti ns of their pape sketch out the possibility of a sys em at c inv stiga ion f the elati n be ween nervous structure and the neurolog of norma and disea ed m nds.

4.5

4.5 1. R consi ering problem 4. -6, try to prove, using only reasoning abou state diagr ms, t at if is the set of sequences that carry so e rna hine fro sta e Q 0 to Q , the there is a machine that so rec gnize the s t of the seq ences of S t· ken in reverse. Finding the direct con tructi n of his r verse -sequ nee r cognizer without using arguments abo t reg lar e press ons i diffi ult b t ins ructive enough to be worth

tOp. c t., p. I I.

96 MEMORIES IN FINITE-STATE MAC HINES SEC. .5

the effort. A solution metho< for herdson [1959].

rela ed p obi en is f< und in She p-

PROBLEM 4.5-2. There is ar analpgy b twee regtlar e> pressipns a d algebraic operations- for exan pie, t ose l sed b Mas n [ 19 0] fo elect ic network theory. For example, eplac a* b

I ~~

and make multiplication non-e mmutative ,-that is, do not p rmit Y exchanges. Then we obtain rul (9) o prob em 4. -4 as allow :

= IYX

E*F ""-1-·J

I - e

1 [e + I - e] · f

I - e

= [I + I ~ e]- f

= f + -:-rl- . . f ~ F v 'f.* Ef I - e

Think about this and see i you an fi d rea ons why thi is s , and (if you are familiar with "signal flow graph ") w at is the c:mnec ion \1 ith electrical networks. Of course there is an · lgebr· ic fac

I+ a+ a 2 + ... =I 2 I a[ I + a i a -+ •.. ] = --

1 r- a

For further developments· long his line, see Harin~ [ 196PL an Ott nd Feinstein [1961],

PROBLEM 4.5-3. Show that

(a*abvta)*a = (( vabvba)

PROBLEM 4.5-4. Find a regular e press onE or th sequ nces hat b ing the machine in Fig. 4.5-1 back to its startir g stat . The find 3 regulfir expres-

star·~2.~ :0

==============~:0~ 1 0/0 1 )

Fig. 4.5-1

si< n E R for the set pf seqpence each of w tch is here erse fa se~uenc in f. No~ dra~ a st te di gram to re resen E R. You will rnd th t yoL can't do it f you requi e a single state to repre ent th even . Prove thi~.

P OBLEM 4.5-5. Solut'ons t the e can be b· sed o the n ethoc s of section .6 ).

Whi h of the fpllowing se s of seque11ces (I 5) ca~ be ncogni ed by a finit state machine? A I inpLts are P or I.

I

SEC 4.5 MEMOR ES IN FINITE-STATE MACHINES 97

(I) T e set falls quen es, 0, I, 00, pl, 10 II, 000, ... (2) T e nun bers , 2, 4, 8, ... 2n, ... writ en in binary notation. (3) T esarreseti unarly: I, II, 111,111 1111, .... 4) T e set of sec uences in v hich he n mber of O's is equal to the

nu nberofl's. (5) T e seq ences 0, 101, 11011, ... I nO! , ....

6) If E is the set of sequenc s rec gnized by a machine M, is there another machir e tha recognizes any eque ce ending with one that M rec< gnize ?

7) If E is t~e set of se~uenc s rec gnize~ by machine M, is there a rna< hine t"'at re ogniz s any sequef1ce corztaini gone that M recognizes?

8) If E is tre set of sel'uenc s rec gnize~ by machine M, is there a rna< hine hat r cognizes a y two cor secuti e occurrences of the same seq ence f £?

PRI )BLE VI 4.5 -6. R place each of the folio wing by regular expressions that do nc t use t he v.

(a b)* (a v bb ba)* (l v (bb v ab)* )*

If£ is an regu ar ex ressic n, ca E* I way be written without any v's? Wh?

NOES

I. We thus avoid a n mber of fire poi ts wt ich a< count for some of the comI=lexity of Kl ene's [1956] papt r. We asst me tt at the machine is started in cperatipn by he in ectior of ar S pu se int an c therwise quiescent net. That i , we< ssumt that the machin real! did start t some distinct (though unknown mom~nt in he finite pa t. Otrerwi e, in he net of example I, it could be that the R cell \ as a! -vays ring, has a ways received a signals, and need n ver have seen an S pulse. The significa ce of this awesome (but perfectly I< gical possibility is disc ssed in the c rig inc I McC ulloch-Pitts [ 1943] paper.

2. I< leene [ 1956] defin s all hree < per at ons, t r, sta , and followed-by as binary operati ns. ~ o do ~e, fo or a~d fol owed-by, b t where Kleene allows only t e fonr £* , we llow * to mean conca enati g any number of selections f om E, inclu ing n ne. This mt ans w have to thi[Jk about the "null sequence" a d sorre resulting comrlicatic ns. n Copi, El ot, and Wright [1958], £* d esn't conta n the null s quen e. T eir p oof of Kleene's theorem makes a c ean separation of the "bgical ' pror erties of cells from the delay properties, a dour first constn ction using 'instant-OR" cells follows these lines; our final c nstru tion rings there ult back clo er to "'leen 's.

3. Cn the txtend d Kle ne AI ebra.

A remark< ble set of e uival nee t eoren s has been established recently by S Papert am R. !'I eN at ghtor. Ccnsider the lass of "extended" regular

~ 98 MEMORIES IN FINITE-STATE IV ACHil' ES SEC . 4.5 I SEC . 4.5 M MORI S IN FIN IT E-STATE MACHINES 99

;

expressions obtained by adding not a1 dana to th or, tar, ~ nd ju tapos 'tion (3) Any propo ition1 I forrr inm( mber of Pis in P.

of the Kleene Algebra. (This w s pro osed first i Mel' a ugh on an ~ Yan ada r (4) If A P ar d A c pntair s the free v ariable t;, then

[ 1960].) As we showed in sectior 4.3, ( nedo s not pbtair anytl ing.n w by this, ,~: (t ;)A and(l t ;)A 1 rein

because we still lie in the doma n of nite-~ tate n achin es, as show r by igs. let Po e the set of mem ers o Pwi h no ree v1 riables. Then Po defines the ,, 4.4-19 and 4.4-20. What we wan ttop into t is tl at, ac ~ordir g to I ig. 4. -20, c ass of word on X which satisf es it.

while we may need to introduce loop to ob ain nc t, we o not requi1 e any hing Ar gular set E is in 1 if it an b{ defin d as he set of words satisfying a

more than a loop around a sing! cell. Clea ly, if we C< nfine ourse ves tc ex- p edica e cal ulus ex pre sion, Po, c efined as ~ bove. The class L is the

tended regular expressions tha cont in no stars we c nob ain 1\ cCul och- "L-Ian1 uages" of IV eN au ghton [1960 and t hese t oo, turn out to be equivalent

Pitts nets that have only single-c llloo ps oft his ki1 d. It can b shov n tha the t the s ar-fre emac hines. This the or is de elop{ d in the paper of Papert and

converse is true- that nets wit h onl this kind f loo can be de cribe by I\ cNat ghton [1966 ].

star-free extended regular expre sions. It turns out that machines of til is cia s car not ' ' ount' eye ically; for

example, they cannot even achie ve the behav iors in sectic n 3.2 .4 that deter nine whether the number of l's in the input is eve nor c dd. l et us ~ive a more gen-era! definition of non-counting. :

A regular set E in a non- ounti g set if then exist anur r~ber n such that, for all strings U, V and ¥,an all I ositiv inte gersp, if UV"Wis in Et en so is UV +Pw.

It is easy to show that any star-! eeE s non coun ing; it is mt ch m< re dif cult to show the converse, which is ~lso t ue. 1 his th eorerr poin stow ~rd a con-nection between the formulatio rs we have een c onsid ring nd a rathe dif-ferent way of looking at machin ~s, na In ely' t heap road throt gh ser igrOl ps.

The connection between th theo ry of au ton ata a d th{ theo y of emi-groups is very close. (We have ut th serer arks 'n the Now so as not t put off the reader unacquainted w th gr ups ' nd se f11i-gr< ups; ee, f< r exa mple Rabin and Scott [1959].) In the s migr< up fc rmul2 tion, pne t inks of a machine as having a set of stat s, wh ich ar tran form( d intc one noth r by the input signals. In some rna chine , an i rput ignal may< nly p ~rmut ~ the states around-that is, no input signa drive stwo diffen nt sta es in o the arne r

state. Clearly, if this is the ase, hat n achin e is c apabl of s orne yclic counting ability, for there must bean n-tri iallo< pint e stat e diag am. rom the group-theory point of view the xister ce of a per nutat' on in uced by a signal on some of the states me2 nstha tthen is a s ubgro up in he se nigro p of transformations (on the states) gener ted b the nput ignal . Th{ impo rtant theorem here is that a machin e that is "p ermut ation- ree" n thi' sense is a ,,

star-free machine, and convers ly; an d it tl t tha this i n turn is eq uiva-t

rns o lent to the class of machines tha cant e con tructe d out Jfthe "unit au torr ata" of Krohn and Rhodes [1963]. rhea vane d rea er wi II war t also to se e the work of Schutzenberger [ 1965].

Finally, there is a connectic n wit still diffc rent a the natic~ I fam ly of 1

concepts- the predicate calcu us of quan tified prop< ..

Consid r SltJOnS. an ',

alphabetX = lx 1, ... ,x,l. F pr eac h i w ~ defi e the prop sitior al fur ction I

F;(t) which asserts that the t th i f'iput i X;. et t, t 2' .. . ' t s l ease of i1 teger ' valued variables and let P beth class ofexJ ressio ns del ned b

(I) F;(tJ) f P al i,j (2) tJ::;tkfP al i, k

,,

j

:I

... , I I I~FII ~ITI ~ M ~c• INE s

I

C4~MF UTJ~BII ITl , El FE4~Tn E F RO~~EDURI S, AUD aLGORITHh1S. IN FINITE MACH NE!i.

5.0 INTI ODU< TION

We now tL rn ot r attt ntion to S< me t as1c questions centered around the very otio of a mech~nica pro< ess. What can a machine do? What do{ s it mean t say hat a proc ss is roech mea ? When is a procedure so con pletely spt cified that a rna hine can < arry t out? In earlier chapters we xplo ed He lim"tatio~s of machines' ith f nite memory. What happens wh n wt lift this estriction? What problems can be solved by rna hine -by mechanical pro<esses -with unlimited memory? Are there pro esse~ that can t e pre isely desc ibed vet still cannot be realized in a rna hine';

~s w not d in chap er I, mos peo ble h ve a low opinion of the int{ llectual po entialities of machines. t is I.Jsually felt that although rna~ hine~ can e ve y fas , or ery s rong the' cannot be very smart. It is well known hat n achir es have be n made to do many things that meet hig hunan standards- to p ay g mes very well, to find solutions to rna hematical Jrobl ms o colltge-grade d fficul y, to find solutions to difficu t sys ems ~esigr prollems to c assif visual patterns of appreciable con plexi y. B~t it i usually fe t that this effects no credit on the machine -t at b caus the designer c r pre grammer as set down every small det il of he p oces~, the mach· ne h s on y to perform (however quickly and accu ately simi le clerical asks.

tis certair ly tn e th.: t pro~ram ning- the "ob of specifying the proced re that a < omp ter i to carry Jut-arnot nts to determining in advan e ev rything tt e co rputn wii do. In this sense, a computer's pro ram an st rve a a pr'.>cise ~escr [ption of th~ process the machine will

1)3

104 COMPUTABILITY SEC. 5.1 SEC . 5.1 COMPUTABILITY 105

:

carry out, and in this same sense it is neani ngful to sa v tha anyt hing hat Such quest 'ons ad c oncer ned r 11athe rna tic · ans for some time before can be done by a computer can be preci ely d scrib ed.t the adve t of omp uting mach ines. Thes e qm stions are associated with

We often hear a kind of co nvers e stat em en to t he efi ect t 1at " any the idea of a alg rithn -an effec tive roce1 uret-for calculating the procedure which can be precisely desc ibed can b e pro ram1 rzed t o be er- val e of orne quan ity 0 for ~ndin g the solut ion of some mathematical formed by a computer." I have h ard his a ~d sir ~ilar taten ents made on pro blem many occasions, and the propos tion i s usu lly st ~ted o be f1 con seqm nee Thei ~ea o an a gorit m 01 effec ive p oced ~re arises whenever we are of the work of the mathematici n AI nM Tur· ng. put it is nc t usu ~lly pre en tee witt a se of ir struc wns ~bou how to behave. This happens stated exactly what it is that Tu ring 1 rove ; in parti< ular t is 1 ot m a de wh n, in the c purse ofw prkin on pro !em, we discover that a certain clear what was his notion of "pr ~cisel w des ribec " T ~ring s cor cept t:Jf a pre cedUI e, ifJ rope I ly ca1 ried < ut, w ·ll enc iving us the answer. Once up g precise description of a process is es~ entia to t e fo lowir g ch< pters so we make such a dis< over , the ask c f find ·ng th e solution is reduced from we have to explain it in some de ail. arr atter of in ellec ual d scov ry to a m~ rem atter of effort; of carrying

out the d iscov red 1= rocec ure~ obey 'ng th e spe ified instructions. But h ow d oes o e tel , giv~ n wh at ap Jears to be a set of instructions,

5.1 THE NOTION OF EFFECTIVE PRO< EDURE tha wer ally have een t old e actly what to do ? How can we be sure that we an h ncef rth e ffecti\ ely a t, in ccor wit! the "rules," without ever

Our exploration of machine~ in P< rt II· s bas d, in large part on i eas ha\ ing tc mak e any furth r chc ice o inn a vatio n of our own? derived from the paper of Turi g [1£; 36] 0 11 the theo y of COmJ utabi lity.

This paper is significant not onl ~for the rr athe1 ratic I the pry w ~ich on- his q uestio r is e< sily a swen d if t e pro ess is supposed to terminate cerns us here, but also because it c ntair s, in essen ce, tt e inv ~ntio of in a certa· n fini e, aln ady k rown, time, becat se then we can just try it and the modern computer and sm re of the prog amm ng t chni ues hat see. Buti f the I ength of th~ procc ss isn t kno f.vn in advance, then "trying" accompanied it. While it is oft ~n sa d tha t the 1936 pape1 did ot n ally it rr ay no bed cisive, beca ~se if he pt ocess does go on forever-then at much affect the practical develo pmer t oft ~e co Input r, I < ould not a ~ree not imew II we ver be sure fthe nswe. Ou concern here is not with the to this in advance of a carefu stuc y of the i ~1telle tual his to y of the que tion f wh ther proc ss te min a es wi h a correct answer, or even

matter. eve stop~. Ou con< ern is whet er th next step is always clearly deter-

Turing's paper must be viev ed a ainst the i 11telle tual back~ roun j of min ed. T e oth r que tions will cc me u~ inch pter 8.

a variety of ideas concerning dt: scrip ions nd p roces es. \gain wet ink of a collection of questions. W scar bed r?scrib ed? ureh the Thq ositic n we willt ke is this: fthe proce dure can be carried out by at P' ocess notion of description entails s me I angu~ 2oulc any one fixed !an- son ever v sim J!e m chin 'sot hat th ere c n be no question of or need for ge.

~'in no vat ion" :>r "in tellig nee,' then an be sure that the specification guage admit description of all jescri bable sses? Can ther be J roc- ; we c proc is c ompl te a1 d th t we have an " effect ve p ocedure." We expect no esses which are, somehow, we I defi ned, vet c< nnot be d ~scrib ed at all?

It could be argued that there ~ight exist de fir ite p oces~ es wl ose < om- qm rrel v ith t is. I ut w will also maint ain,' ith Turing, a sort of con-

munication requires the transrr issio of< men tal a titud , or ~ dis osi- ver e, w ich ' ill s~ em a first quit extr me. We assert that any pro-

tion, which cannot be capture ~in ny fi nite umb r of wor< s-~ hich ced ure w hich c ould "nat1 rally 'be< alled effec ive, can in fact be realized by ~ (sin pie) !nach 'ne. !tho gh t ism y see m extreme, the arguments must remain intuitive. bel ow in its fa or a1 e har tor fute.

tit is important to note that this do s not mean hat th persc n who writes a com puter We r ust c mph size hat t his is a su pjecti we matter, for which only program automatically understands all he cor sequer ces of ~hath has d bne! I is per ectly arg ~men and pers asiot are appn priat ~; tht re is nothing here we can possible to write instructions which Ia nch tl e com uter i to a ! reat se f!rch p ocess, with

exJ ect tc . It i pt (o r even to read) the follow-prove not eces ary t ace many of the trial-and-error features of ic evol tion, nd th conse uent evelo ment organ ing argu nents to a preci ate th ema hem a tical jevelopment of the sequel. of structures of enormous and unexpected omple xity. Many comp ter p ogram s are

frankly experimental~to see what will be t e beh avior f a sr ecified syster -no well Th read er wt o fin s hin self n str ng d isagn ement either intellectually understood in advance. All scientists know that s pecific · tion d oes nc t mea imm diate or more like! ) em :>tion lly s ould not I ~t tha t keep him from apprecia-understanding. When we write down equati ns, e.! ., in rr a them tics o phys ics, it is not tio oftt e bea utifu I tech1 ical c onter toft he the; ory developed further on. enough to know that the solutions are thus eterm ined. 11/e usu ally w nt to now orne-

thing about the character of the solu ions~ and t his ent· ils son e kin of p ocess ailed Wew i I use t elatte term n the sequel. The t erms ar roughly synonymous. but there "solving the equations." are num er of hades of me2 ning u ed in differe t con exts, especially for "algorithm."

106 COMPUTABILITY SEC. 5.1.1 SEC 5.2 COMPUTABILITY 107

5.1.1 Requirements for a definit on of fou ndin chapt r 8; he tr ck is to su bstitL te mt rely quantitative increases effective procedure in r emo y size for c ualit tive i rcrea es in COffiJ lexity of the machine.

ft\ithc ugh' e sta e thi~ as fa t, th re re p-ta in a subjective aspect to the In trying to give a precise matt em at cal d efinit on t< "effi ~ctive pro- rna ter. piffei ent p ople may rot a ~ree n wl ether a certain procedure

cedure," one encounters vario us di ificult es. ~ One ann< t ex~ ect a ways sho ~ld b call d eff ~ctiv~ . Pe haps there are processes, one might sup-to find a simple and complete! y sati f'fact< ry fo mal quiv ~lent for a com- pos ,wh ch si1 pply j an no bed scrib ~din ~ny f rmal language, but which plex intuitive notion.) We will begir by s ying hat can neve thele ss be carri d ou t, e.g ., by mind. One might even argue

an effective proce jure s a s~ t of r ules vhich tell tha thos e imJ ortar t, bu still mys eriou , fun ctions which make minds us, from moment to rr omer t, pr cisel hov to sup rior to a! pre ently kno wn n echa nical processes must, by their behave. intt itive natur , esc pear y sys em at ·c des ripti on.

Turin g disc usses some ofth ese is ues i 1 his brilliant article, "Comput-This attempt at a definition is subjc ct to the c iticis p1 th~ t the inter ret a- ing Mac mes and In tell gene " [le 50], ~nd will not recapitulate his tion of the rules is left to depen don orne perso nor:: gent. No~ ape son's arg ~men s. Tl ey ar ount , in rr y vie ~.to sati~ factory refutation of many ability to obey instructions de pend on 1 is ba ckgrc und find i ~telli! ence. sue ~ objt ction .We will ~uta~ ide st ch m ~tters and turn first to the prob-If his intelligence is too small, herr ay fa I to nder tand wha1 we 1 pean. !em ofm akin~ prec seth intu itive dea c f obe ying a set of instructions. If his intelligence is too large, e r toe alier , he pay i vent some cons ·stent Ma ters: re ch rified by CC nfini g att ntio first to very concrete processes, interpretation of the rules that was 1 ot in en de j. "' e kn< who w oft nan sue as t e exe cutio ofn a the natic I con puta ions. apparently "simple" explan tion turn out to ave an unsusp ected ambiguity.

We could avoid the probler ns of inter preta ion- of u ders andir g-if 5.2 TURI NG'S ANAl YSIS OF CC >MPU ATION we could specify, along with t he st teme nt of the r les, t he de ails if the PRO ~ESSE

mechanism that is to interpret them. Thi wou d lea ve no amb iguit\. Of ~the course, it would be very cumb rsom e to l ave t p do ~II th s ove raga n for n his 1936 pape ,A. ~.Tu ring c efine class of abstract machines

each individual procedure; it i desi able to fin d sor e re:: son a ly u~ iform tha now bear ~ism me. ft\ Tu ing 11 achil e is :: finite-state machine asso-family of rule-obeying mech nisrr s. j mo st co nvem nt f prmu ation ciat dwi has ecial kind of en Iron rent-1- its t ppe-in which it can store would be one in which we set 1 p (an late reco wer) s quer ces o sym ols. ~ew ill describe these machines

in g reate deta ·1 in< haptc r 6 a ~d th seq el. irhey are very simple. At ( 1) a language in whic sets of b havi oral 1 ules eac mo nent he (fi nite-s tate I art o the) mac ine gets its input stimulus

are to be expr ssed, and by eadi~< g the sym JOJ W ritten at a certa in pc int along the tape. The (2) a single mach ine \1 hich can nteq:: ret s ate- rest onse of the mac hine r nay c lzange that symb 'Jl and also move the rna-

ments in the I nguo ge an d thu scar you the chir e as mall is tan ce eit her w ay a! ong t he ta pe. The result is that the steps of each s pecifi d pr cess. stin ulus for he n xt c cle f op ratie n WI II come from a different

rt of mach ine tc s of rules "sq pare' 'of th e tap , anc the mach ne m ~y th s read a symbol that was This suggests designing so me se accc pt se wn ten tl ere lc ng a1 o. T ~ism ans 1 hat t e m2 chine has access to a kind expressed in some language, :: nd tc do v hat t ~e ru es re ~uire. Her ~om ofr ~dim ntar exte rior I pemo ry in addi ion t p that provided within its might expect a new difficulty. For s '-uely one' ould not c xpec any ingle

~les l fini e-sta e par. Ar d sin ewe will place noli (nit on the amount of tape mechanism to be powerful en pugh to in terpn t anc exec ute r [or a/ !able this pry h ~s, in effec t, an in fin te capacity. But the re-ava mem effective procedures. Surely o [le we uld e pect to m ed at least a seq uenc~

stri ted n ann er in v hich the rr achir e an< its t pe are coupled (see chap-of more-and-more complex rr achir es fo the ~xecu wn f a ~ eque ce o ter ) mi ht m ake o ne th nk,a t firs t, tha t thei ossible uses of this poten-more-and-more complex proc dure . tial y infi 1ite rr em or ywo ld be rea II very limit ed. Curiously, this is no prot Jem! lttt rns c ut th at we can reali:~ e ou

urin g disc oven d, he weve ' th at he ould set these machines up to notion of an instruction-obey achin e in forr 1 wh ich re main con ng m rna ever v corr plex omp uta tic ns. I cha ters and 7 we will work out a stant, no matter how complex the proced ure ir que tion. Tha is,~ e car nun ber c f exa mple sand ee th at th tape merr ory really does escape the set up a rules-language and sing le "u niver al" i terp eta tic nm chim lim ·tatior s of the fi nite-st te m chine. which can handle all effectiv pro edur s. 1 he d tailec con true tion i~

·' 108 COMPUTABILITY SEC 5.3 SE . 5.3 COMPUTABILITY 109

' .,. Turing goes on to defend tt e fol ow in pro ositi n, n< w of en c< !led arl itrari y Sffia II ext nt.t Thee l"ect o f this restriction of the number of

Turing's thesis: sy nbols is not very eriou . It i alwa s pos sible to use sequences of sym-

Any process which coul d nat urally be c ailed bo s in t e pia e of s ingle ymbo Is. T us an Arabic numeral such as 17 or

an 99 9999 9999( 99 is rJOrm lly tre gle symbol. Similarly in any ated 2 sa si

effective procedure can be re lized by a Tur ng Eu ropea r1 lang uage ords are tr a ted s sin~ le symbols (Chinese, however, machine. att empts to ha e an nume rable infinit v of s mbols). The differences from

This proposition, in its nost forrr , is u sually calle Chu ch's ou pain of vi w bet .veen t he sin lean< com ound symbols is that the com-

en era und s mbol ' if th too I ngthy ot be observed at one glance.

thesis, after the work of Ala 1ZO C urch relatir g the intuit ve no tion c f ef-po ey are , canr Th is is ir a ceo dane< with ex per ence. We c nnot tell at a glance whether

fectiveness to formal logica proc sses. We r efer t it a Turi ng's t hesis 99 9999 9999C 999 a d 999 99999 99999 9 are the same.

because of our preoccupat on, i r1 the sequ I, wi h Tu ing's parti ular "The beha iour f the moment is determined by the

formulation of computabili y cor cepts. Par of , uring s 193 b pap er is camp uter a any

sy nbols which he is c bserv ng, a d his 'state of mind' at that moment. We concerned with demonstrati r1g the equi alene of h s anc Chu ch's Jrior

m< y sup JOSe t at th re is bou dB t J the umber of symbols or squares formulation. wll ich th com puter an ol serve at on e mar r1ent. If he wishes to observe

When one sees how primiti e the mac ines eally are, nco cept, this ffi( re, he must use st ccessi ve ob ervati ons. We will also suppose that the

thesis seems incredibly rash. Y:;t the have bon Turi ng's \ nu mber f stat s of nind hich need l e tak n into account is finite. The year e ou Iew.

or thi the s med ose which restrict the number Every procedure which math mati cians have rally d tc be

re sons are o aract r as t gen agre

of ymbc Is. If we ad mitte< an ir finity of sta es of mind, some of them will "effective" has been shown eq ivale nt, in one .vay o r anc ther, to a pro- be 'arbit arily lose' ndw· II be c on fuse d.t A gain, the restriction is not one cedure carried out by a Turing nach ne. wll ich se iousl affec scorn putati on, sir ce the use of more complicated states

One cannot expect to prove Turin g's th esis, ince he te rm" atur lly" of mind an be avoid ed by .vritin man symb ols on the tape. relates rather to human dispos tions than to an y pre isely defin :;d qu ality "Let us im gine he op ratio r~s per forme d by the computer to be split

of a process. Support must con e fro mint uitive argu ment , and we c auld up into' simp! oper tions whic h are so ele mentary that it is not easy to

hardly do better than to preser t son e of the a gum< nts i Tur ing's own im agine hem f urthe divid d. E ery su chop ration consists of some change

words. of the pl ysical syste n con isting of th com uter and his tape. We know th state of th e syst m if we kn OW th e seq ence of symbols on the tape, wt ich of these are ob servec by th com puter possibly with a special order),

( an d the tate o fmin of tll e com puter. We may suppose that in a simple

5.3 TURING'S ARGUMENT op ratio r1 not nore han o ne syr 1bol i alter d. Any other changes can be sp it up into s "mple chan~ es of this k in d. The situation in regard to the

The following is taken vert a tim from Turi rJg [1 36, sectio 9]. It is sq ares hose symb< Is rna be a! tered n this way is the same as in regard to

one of a sequence of argument in f f his posit on. he o ther rgu- th obse ved sc uares We nay, t erefo e, wit hout loss of generality, assume vor

th tthe swho bois a e cha r1ged a re always 'observed' squares. ments are reviewed clearly in leen [ 19 2]. hen achir es di scuss d in

quare esym

the quotation are essentially tl ose < escri Jed i the next chap er~t oday tlfwe regard a sym ol as I terally printe on a quare we may suppose that the square is

called "Turing machines." No e tha t the .vord 'com puter ' as t sed i 1 the 0 X < I, 0 < y < I . The ymbol is defi ed as a set o f points in this square, viz. the set

quotation means the person th< t Tur ing is go in~ tore place by a nach· ne. oc upied by pri ter's i k. If these s ets are rest ric ted to be measurable, we can define the 'di tance' bet we ntwo ymbo s as th cost c f trans form in g one symbol into the other if the co t of m oving unit rea of printe 'sink unit d istance is unity, and there is an infinite

"Computing is normally done y wri ing c rtain symbc Is on paper We su ply of ink at = 2, y = c . Wit this t opolog y the s ymbols form a conditionally com-may suppose this paper is divide d int< sqm res li e a hild's arith me tic pa t spac . [Turing's n te]. book. In elementary arithm etic th etwo dimer siona char cter c fthe a per 1In th second para raph, we car not ur dersta d wh t Turing could have meant by the is sometimes used. But sue a us is al ;vays < voida ble, a d I t ink t at it su gestio that "state~ of m· nd" c uld b "con used," if he is discussing psychological will be agreed that the two dimer siona char cter f pap er is o ess ntial m tters. f, inst ad, he has in mind p hysical states of a br ain, then one would indeed expect

of computation. I assume t en th at the camp utatio is ca rried Jut on one- th t for s fficien ly sim Jar sta es the ewill e a ch ance o random transitions, e.g., because

dimensional paper, i.e., on a tape ivide into I s hall a of herma I or qu an tum pheno nena. here i sa lim it to tt e amount of information that can squar s. so su pose that the number of symbols whicl be pr· nted i s finit e. If re to

be recove red fr m an physi cal sys tern o limit d size The same holds for whatever may we w ph sica!. ystem is used to rer resent the sy nbols ithin he "squares" of fixed dimensions.

allow an infinity of symbol , ther then wou d be ymbo Is diff ering o an [M .M.]

I

ll 0 COMPUTABILITY s c. 5.~

"Besides these change~ of sy[nbols the ~ imple opere tions must 'ncluc e changes of distribution of bserv d sqt ares. Then w ob erved squar s mu t be immediately recognisatle by the c mput r. I think it is easonable t:> suppose that they can only be sq ares' hose distan e fro n the loses of the immediately previously o servd squ res d es nc t exc ed a certair fixed amount. Let us say that each pf tht new obserwed s uares is w thin squares of an immediately previc usly observed squ re.

"In connection with 'i rmed'ate re ognisabilit, ,' it IT ay be thou~ ht th< t there are other kinds of square which are immediate! recognisalle. I 11

particular, squares marke by s ecial symb Is mi ht bt take as i nmedately recognisable. Now if these quarts are marked onh by single s mbol there can be only a finite numl er of them, and e sh< uld npt up et our theory by adjoining these !JTarked squ res tc the < bserv d sqt ares. If, o the other hand, they are m rked y a sequenc of symboh, we c~nnot regar the process of recognition s a simple rocess. Th s is a fundament< I point and should be illustrated. In m st rna them a tical I apers the e uations an theorems are numbered. Normally t e nu nbers do nc t go eyon (say) 1000. It is, therefore, po sible to recognise a th orem at a lance by it number. But if the pa er w s ve y long, we migl t rea h T eoren 157767733443477; then, further pn in the p per, f\'e mi ht fir d '.. henc (applying Theorem 15776 7334~ 3477) we h< ve . . . . In rder o rna e sur which was the relevant the< rem v e sho ld ha e to compa e the wo n mber figure by figure, possibly ickin~ the gures off ir pencil to nake ure of their not being counted twi e. If in sp'te of his it s still thou~ ht that ther are other 'immediately recpgnis< ble' s uares, it dces no upse my < onten tion so long as these squan scan e fou d by orne roces of w!lich n y typ of machine is capable.

"The simple operations must herefJre in lude:

(a) Changes of the s mbol on c ne of the < bserv d squares.

(b) Changes of ne of the squares obser ed to anotter square with n L s~uarei of o e of he pr viou ly observed sq ares.

"It may be that some c f thes cha ges necessarily ir valve a change o state of mind. The most genera sing! operation must heref:>re be take to be one of the following:

(A) A possible hang (a) c f syrr bol t< gethe with a possible ch: nge o state of mi1 d.

(B) A possible chan~ e (b) of ol serve squ res, together with a possible cl ange f stat of m nd.

"The operation actualh performed ·s dett rmined, as as be~n su~ gestec [above] by the state of min of tr e corn puter and t e obs~rved symbc Is. h particular, they determine the tate c f mir d of the c mpu er af er th operation.

I

SE . 5.3 I COMPUTABILITY lll

"We !JTay now cc nstru t a rr achin to d the work of this computer. To ach state o mm of the computer orresponds an 'm-configuration' of the mach ne. The machine scans B squ res c rresponding to the B squares obs rved by tht com1 uter. In anv mo\e the machine can change a symbol on scanned s< uare r can chan e any one < f the scanned squares to anoth~r squ re di~ tant npt mo e thar L sqt ares rom one of the other scanned squares.t The If"love f\'hich is do e, an~ the succeeding configuration, are

det rmin d by the scanned ymbc I and them configuration. The machines jus described co not diffe very essen ially rom computing machines as defined (I revio sly) and co responding to an machine of this type a comput ng m chine can b cons ructec to co!JTpute the same sequence, that is to say these uenc com1 uted y the omp ter."

5.3 l T e eql ivalence of nany ntuithe forr ulati< ns

Perh ps tt e stn nges argument in fa or o Turing's thesis is the fact that, ov r the year , all othe note wort y at empts to give precise yet int itive y sat sfact ry d finitipns c f "ef ectiv procedure" have turned ou to be equi alen -to ~efine esser tially the s arne class of processes. In the 1936 pape Turing pr ves that h s "comput bility" is equivalent to the "ef ectiv calculabi ity" )f A. Chu ch. 1\ very different formulation of effective ess, escribed a about the sam time by Emil Post, also turned ou to h::ve th same effe t; we sho" the quiv lence of Post's "canonical sys ems' and Turir g m:: chines in hapt r 14 Another quite different formulation, trat o "gen~ral r curs·ve fu ctior" due to S.C. Kleene and others IS also e quiv< lent, :ts we show in chapter 10 and 11. More recently an 1m be of o her f rmu ation hav appeared, with the same result (e.g., Smullyar's "IIemerJ.tary Fornal S'stem" [1<;62]). Whenever a system has been proppsed ;vhich is net equ valer t to these, its deficiencies or exces es hawe ah ays teen i tuiti ely e iden .

Why is thi an a rgum nt ir favor of urin 's thesis? It reassures us tha different ;vorke rs wi h dif erent approaches probably did really have the same intui ive c )ncept in n ind- and hence leads us to suppose that the e IS eally here an ' objective" or "absolute" notion. As Rogers [19 7]putit:

"In his s nse, he nbtion of ef ectively compt table function is one< f the few 'absolute' concepts p oduc d by mod rn "ork in the foundatic ns of mathemafcs."

Proof of the equiva ence oft" o or more defir itions always has a compell ng ef ect ' hen he d~finit ons · nse rom different experiences and motivatic ns.

t t\s we hall s e, the e is nc loss of gene ality i we re trict both L and B to be unity! [M. vf.]

112 COMPUTABILITY SEC. 5.4

5.4 PLAN OF PART II

This second part of the boo explores he pwpert es of our bxten~ed machines~finite-state machines with pas~ive b t unl"mite env ronment of "scratch paper." This explor tion ontir ues t devt lop t e tw b the jnes of the first part of the book. We try to a sess he range c f po\ sible behaviors~ to characterize what i~ wit in a1 d OL tside the each of tl ese machines. And we continue to exerc se OL r ave catio -· th coli ctior of very small "universal bases" fo1 the 'ssemply o stru tures which realize the full range of behavior.

In both these avenues we n w m et a mucl more ext nsiv( and exciting range of phenomena. Ore wo ld cntainly expbct tl e rar ge to be large, since we have argued th: t ou new macrines can ~ncor[lpass all effective procedures. For our he bby, ~e di cove sam rem arkalle ba es. ~he reader wi_ll find it inc~edib_l , at f rst si~ht, trat s< me c f the~ e set of s1mple operatwns could g1ve we to he fL II ra1 ge o pos ible <amp tations.

For the more serious objec ive- that pf charact nzm the effec 1ve procedures~we acquire a rerjlarkable tpol, the niver a/ cbmpu ing machine. It turns out (in chapte 7) tl at th~re e ists a cert< in Tt ring (nachine which can imitate the beh~vior of ar y other Turing jnachine, g1ven an adequate description of the stru< ture pf th~t ot er rr achir e. ( he description is to be written do ~n in the ~nvir nme t~t< pe~ nd reed not be built into the works of he u ivers~l m:: chine.) A a n suit, our exploration can be reduced to tt e stu y of this s · ngle machine. While we do not entirely so restrict our :: ttenfon, v e do freqL ently call ~pon the existence of such a machine thrc ughol1t the sequ I. T e um ersa mac ine also opens the road toward sim1le ba es.

The universal machine work , as c ne m ght e pect, by a berating on the description of another machine. It interpr< ts su h a < escnt>tion, one tep at a time, so that, in effect, it mita es th!! oth~r's bbhavipr. 1/Vith 'interpretative behavior" within the scobe of our r achi es, v e car begi to ask new kinds of questions abo~t m:: chines. F r mstance one can sk:

1 what happens when a machine is cc nfr01 ted ' ith i s O\\ n de cript on?

1 If this is done to the universal macl ine, hen, as ore mi ht e pect, the I 1 machine must become paralyz( d by an ir finite regr ssion of i terpreta-

tion cycles~it can never actually get to co~put any hing. At rst, uch phenomena seem entertaining, hen nno ing, nd f nally we :: re fo ced to conclude that they signal a p< rtentbus opstaclb to cur explora ion. We find that certain of the questions we na urall'v ask abou mac ines annat be answr;red, at least by any effecti e pr« cedu e for answ'ering quest'ons. Ind_eed, the m~st ~ign~fican_t result of t e secpnd 1 art o the pook are n gatlve results md1catmg hmitatic ns n< t on y on the ffect ve p oced~res

SE< . 5.4 COMPUTABILITY 113

tht mseh es, b!Jt alsp on our !tim:: te ability ver to characterize, effectiv ly, wrich 1 rocec ures ~rein fact« ffectilve. . _Chai ter 8 cont< ins ~~ ~ be~ innin~s of these results. The basic method IS ~1m ph and rathe stril mg. Beca~se tJ- e rna hines are capable of interpn tativ( operation we c~n imagine pass'ng certain of our questions over to he m~chin~s the~seh es. Such a question, for example, is: Which machine co.nputations ventz ally t"Prmin'(lte w'th a efinite result, and which go on orev«rWitl outaflyde.) nite onchsion? We how, by some simple technic a! tri« ks, tl at as uming the bxistt nee o a m chine that can answer this qu stion lead~ to a ontr diction; h( nee, o sue h machine can exist.

The on-e istence of~ cer*in k nd o macttine might not be, in itself, a great c isastt r. B t so ar as our oats ~ere re concerned, this result is inc eed 'ery s riou ; no Turir g m< chine can answer this question. But th( n, if ~e ac ept urin 's th sis, tpere <an b no effective way at all to an wer i . Th~t is, ~e ca nev rasp re to a con plete, systematic theory of tht cone ition und r wh ch a omputatic n is ure to terminate. We may be able o de ide, 1 or 01 e reason c r an< ther, that certain machines will we rk, a1 d th:: t certjlin ol hers ill n< t, bu we v ill never be able to put this art on a syste jnatic basis whicp will wod for « ll machines! I regard this, an~ sim ·tar other 'und cidal ility" results, t be among the most signif cant · ntell ctual disc< verie of rjlodern tirr es. One can reject its impli ati01 s on y by rejecting urin~'s tl esis, but there is no apparent pr spec of aJ y sat sf act ry replace (nent.

Witt the demc nstra ion pf th se el!!men ary undecidability results, se1 era! r ew li es of expl< ratiop are ppen( d (ev~n if the most desirable line is hus i revoc ably Iosee ). We could tn to n strict our machines so that th ir be~avior is ot q!Jite ~ o co jnplic ted; and we do this briefly in ch pter 10, i connecti< n with th so- ailed "primitive-recursive functions." Anotl'!er lijle (tt at W( do npt fol ow) s to study, as if by default, th< inte relat ons petw( en v riou kin s of unsolvability results; this fo ms rr uch e f the ubje t rna ter o the (node n mathematical "theory of re< ursiv fun< tions " 0 r m: in activity will ~e to examine a variety of alt rnat ve fo mula tions of e~ ctiveness nd tl rough this to try to underst:: nd someth ng o the ourct s of lvhat ~e kr ow is a hopelessly complex ra ge o behavior. In th cou se of this v e wil come to understand better th rei a ion b tweep the modern cc mput r (ar d its programs) on the one hapd an~ the e abs ract inach nes (: nd tf eir d( scriptions) on the other.

Cha ter II is ' once ned with ndin 5 a r iddle ground between the sir pie I ut in prac ically inefl cient Turitig m chine, and the more complicated but < pplic tion onen ed rr oder corr puter. Our result is a new fa jnily < f ab~ tract mad ines- called "p ogra jn machines" ~which combi e att activ~ features rom poth xtre1jles. It turns out that in almost ev ry n spect pro~ ram machmes re st perie r to Turing machines for


theoretical purposes; and, by ~ tudyif1g th m, v e alsp obtain s pme raetical insight both into the the ry o effe< tive omp~taticn an intc tlie basis of modern computer pro~ ramn ing. We < o no wist to a ouse wild hopes that this theory, by sam< magical i sight, will help ~nyo e dir ctly with practical computer prog amming problems. But ·t dm s pre vide some of the "cultural backgro Lind" from which a satisfa tory theo y of practical computation will ultin ate!) sprir g.

Chapter 9, optional and so new hat m< re ad vane< d th2 n the rest, discusses some connections between the thepry o effe tive ~roce~ures and the theory of real numbers.

5.5 WHY STUDY INFINITE MAC -liNES.

Up to this point, our stud of machines has be n based entire! on the finite-state point of view. We have L sed c uite few diffe ent < rguments to justify this emphasis eve Turing's argu !nents earl"er in this chapter are so directed.t In th~ seq el wt will deal f\'ith fr1ach nes c f an infinite character; our machine will have infin te ta) es, o sto age registers of unlimited capacity. Since no such mac! ine cpuld xist m a nite universe) and (even if the unive se is infini e) w can neve have one, why should we study their theory?

Our answer has a paradox cal qualit'. We shall no give the easy answer that, just as mathematician study infinite r umb rs th y cannot reach, it is instructive to study the r mitin~, im ccess ble e tensipn ol our ideas about machines. On the ontn ry, we tak the positipn that thi extension is actually needed to gain an real y pr ctica insi~ ht in o real-life computers! It is worth some < iscus ion o see how the infinit -rna< hine theory could be more realistic than the nite theor , foi .pra< tical purposes.

In the first place, the limi ation alrt ady stablished for nite- tate machines seem much too restri tive o take ser·ously. Ou intuitive "deas about machines require a mor comprehensive fram;:wod. W oug11t to be able to talk about machint s wh ch c< n mt ltipl) pairr:- of f!rbitrarily large numbers (shown in chapt r 2 t< be teyonfl any finite-state mac ine) or can verify the grammatical orre< tness of a bitra y ex ressi ns i1 the simplest of mathematicallangu~ges. (The argu (nents of chapter 4 sh< wed that this cannot be done by fini e-sta e rna hine .)

To be sure, we will always t e cor fined in r a! !if , to nach·nes v. hich are finite. But I assert that it i not alwa s the finitwess f the mac ines that limits their uses; more ust ally i is ei her (I) tht prac ical imita ions

tThe above quotation from Turing [ 1936] s, torr y kno ledge, the fi st clea descr ption of a finite-state machine in the literatur .

SE . 5.5 COMPUTABILITY 115

OJ runn ·ng-ti re or (2) 1 he C< ncep uaJ <ampJpxity of their structures or "1 rogr ms." Tht s, wt knofv that no pnite machine can ~numerate all the integers; it car not count, in ar y rea onat le sense, past Its number of di tinct acce sible inter a! states. But that s not the limitation met in pr actic<; eve 1 the smal est n odern computer has thousands of bits of accessit le memory, but or ore act ally o cot nt up to 2 1000 -even operating at the wove fnquen y of hard cosmic ra s-would take longer eons than e\ en o r m st c< smolpgica astrpnomers like to consider. This m~kes uspe t the prac ical guidar ce value o inferences based solely on th fini e-sta e lim.tatio

A rr ore s~riou practical imita ion , rises when we consider machines w ich consu ne a I rge i forn atior ston ge capacity in a relatively passive w y. V e might want 11 con ider, for ir stance, a machine which remembers all of it~ prev·ous input expenence t Ev n here the practical limitati< n rarely t2kes t e form of an outrigt t bot nd on machine size; it has in tead the 1 eculi r fo m o rele tless y increasing economic pressure. F r, gi en a real rrachif1e, w~ can alwa s ext nd it a bit more by adding p< rts (2nd ht nee s ates) in the for (n, sa , of external storage tapes. The eventual tern inatipn of~uch rowt~ will deperd on more or less irrelevant ci cum~ tances sur ounding t e prpject rather than on any particularly natural bound on hem chin size.

For nodern con pute s, we can usual y pre vide enough external memory for our r:roblems. he cJmputatior time still usually remains as the practic< llimitatior. Th sis aggravated y the fact that access to external m mar (e.g , rna netic tape ) m2 y ta~ e mi lions of times longer than OI erati ns w thin he certral fr1ach ne. 1 ~ven if this were not so (and one m Ilion times 2 1000 is stil like 2 1000

, in p1 actict ), our difficulty is less that th~ rna hine ends up ir a st te-dif!grar loo D than that we cannot wait u til it oes.

For such easo s, it woul< seen pro ]table to study the theory of machines in which th amcunt o machinery is not itself the limitation. But it woul< not be pr Jfitable, at least from our I oint of view, to. study n:'achines ' hich are r~ally nfini e eitl er in initial endowment or m effective speed o ope1 ation Tht s, it v ould seem unre listie to consider a machine w ich, f\'hen start< d, a! eady cont. ins t e co1 rect answers to all answerat le qu stions in E nglis ! Ncr wo1 ld it eem realistic to study a machine

tTher are s me w o beli ve the huma brain has tt is capacity, but I am inclined to believeth t this s due o a combina ion of wishful thinking and misinterpretation of strikin~, but e ceptional, in idents

1Ther rema ns a fundame tal pr blem ere, e en if ccess times for external memories rer ained constant wit! incre sing s·ze. F r as tl e men ory size grows then the length of tine, on tre ave age, for the c mputt r to c< mpute the aGdress in memory of a datum must gn w, too albeit much ~Tiore s owly.


which could test in finite time, an in inite numl er of cases or h vpoth eses (and thus tell us whether or n ot Fe rmat' s Las t The or em is tr 1e by ex-amining all cases). A compro mise seem~ inev itable ; we must com ider machines which have at each rr ome1 ton! vafi ite q uanti y of struc ure, but which are capable of bein exte nded in de nitel as ime! oes c n-"growing machines."

The Turing machine, with its tinite state com Jutin uni and its initially "almost-blank" tape, is perfe ctly s uited 0 ou purr oses. We eed not think of the machine's tap~ as ir finite. We ima! me1 stea< tha the

~) ,ruR lNG

machine begins with a finite ape, but hat, when ever an e 1d is en-h1AC HIN ES countered, another unit of tape (a " quar ") i~ atta hed. Thu s, ins ead

of an infinite tape, we need on y an nexh austil leta e fac ory. Sine , in fact, there is a fixed bound on t lle ra e at ' hich new ape< an b reqt ired -one square per moment is t he w< rst c se- he f2 ctory can be rr a in-tained by investing a fixed, finit amo unto mon ey in a (pe rfectl ) rei· able bank. This picture gives a re ssuri 1gly nite pictu e of the n ew st udy. '

Accordingly, although we are < one with t he stt dy 0 fin it aut< mat a , we 60 II'ITROD UCTIC N

need not jump directly to the s udy finfi nite < utom ata. Inste d we will work in what Burks [ 1959] apth calls the d omai of' grow ·ng a tomi ta." A unn mac hine i a fin ite-st te m achin ~ associated with an external We note in passing that math en atici ns de in de ed stu dy ge nuin~ ly inf nite s orag or rr em or y me< mm. This medi urn h as the form of a sequence of automata, e.g., in the theory of he "hyper arith netic func ions'

, of s1uare ,rna ked c ff on a lin( ar ta 7e. T hem achine is coupled to the tape

Kleene [1952]. t roug h a hE ad,w hich · s situ ted, t eac h mo ment, on some square of the t pe (F ig. 6. 3-l ). The h ead h as th ee fu nctio 1s, all of which are exercised i eact ope ation cycl~ of n e fini te-sta te rna chine. These functions are: r adin! the quar oft he ta pe be ng " cann ed," writing on the scanned s uare and rnovin g the mach ne to an ac jacen t square (which becomes the s anne d squ are in then ext o erati on cy !e).

'

l ~ I I I Ill I I I I I I ~ ' r---4 ~ ---Tape-

~ :II 1\ L ,II

,., ,/

"--'

I ig. 6.0-1

I

It viii b reca lied f oms ectior 2.2 hat a finite-state machine is char-acteriz d b an !ph a bet ( o, ... ' Sm of nput symbols, an alphabet ( o, .. , r n) of ou puts mbo s, a ~ et (q ' ... ' qp) of internal states, and a pair of func IOns

Q(t + I) = G( Q(t), S(t))

R(t + I) = F( Q(t), S(t))

117

118 TURING MACHINES SEC. 6.0

which describe the relation bet'>~ een i~put, inter al st· te, a d subsequent behavior.

In order to attach the external t pe, i IS c nvenient to modify his description a little. The input s mbo s (s 0 , .•. , m) w II ren ain the sane, and it will be precisely these that may be inscribe on the tare, on symbol per square. The input to the machin M, at the time t, will be ·ust that symbol printed in the square the mad ine is scam ing a that mom nt. he resulting change in state will th n be deter'rninec., as 1 efore. by he fi. nction G. The output of the mach in M } as now the dual unctwn of (1) v. riling on the scanned square (perhaps cnanging the syr bol alrea y th re) and (2) moving the tape one way c r the other.

Thus R, the response, has two ccmpor ents. Om compone t of the response is simply a symbol, fro n the same set ( 0 , ••• , sm) to b printed on the scanned square; the second compor ent i one pr th other of wo symbols '0' (meaning "Move lef ")an~ '1' ( 'Mo e rigl t"),' hich have the corresponding effect on the rna hine s pm ition. Ace ordir gly, it is cpnvenient to think of the Turing m chin as d scrib d by three functiOns

Q(t + I = C (Q(t, S(t)) R(t + I = F(Q(t) S(t) D(t + I = l ( Q(t , S(t))

where the new function 'D' tells fvhich way he m~chin will move. In each operation cycle the machi est< rts in som stat q;, Ieads the

symbol sj written on the square L nder the h~ad, r rints there the r ew s' mbol F(q;, sj), moves left or right according o D q;, s ), and then en ers the new state G(q;, sj).

When a symbol is printed 01 the ape, he S' mbol prev oush ther is erased. Of course, one can preserve it by rinti g the same syrr bol t~at was read, i.e., if F(q;, sj) happens t be j· B~caus~ the mac ine an move either way along the tape, it i~ possible for it o re urn 1o a r reviously printed location to recov r the information inscribed there. As we will see, this makes it possible to use tl e tape for the s1orage of a bitra ily large amounts of useful informa ion. We w II giv examples shorty.

The tape is regarded as in fin te in both direc ions. But we \\ill m ke the restriction that when the ml chine is st rted the t pe n ust l e bla~k. except for some finite number o sqw res. With this restri tion one an think of the tape as really finite ~ t any part"cular time but v ith tl e pr< vision, whenever the machine corr es to an er d of he fi ite portio , someone will attach another square.

Formal mathematical descrir tions ofT Jring machines nay I e fot nd in Turing [ 1936], Post [ 1943], K eene 1952 , Da is [ JC 58]. frhen are ~nimportant technical differences in the e forrnula1 ions. For our purpc ses it will usually be sufficient to use picto ial state d agrarns. C ur in mediate

l

J

I

. I ' !

Slo ·. 6.0 TURI:-.iG MACHI:-.;Es 119

purpose is tc sho~ ho~ Tt ring machines, with their unlimited tape m mon, can perf rrn cornpt tatio s be ond the capacity of finite-state rn· chin s; it is us ally easie to nder tand the examples in terms of di· grams than in errns of tables f fur ctions. While it is fresh in our rnmds, lowe\ er, le usn )te that the fin it( -stat( parts of our machines can be described icely by se s of~ uintuoles of the orrn

(old s ate, s mbol scannbd, ne~ stat , sym ol written, direction of motion)

i.e,

or

i.e , as uint pies in wl ich t ne th rd, f urth, and fifth symbols are de-te mine b by t ne fir t anc seco nd th ougl the hree functions G, F, and D rn ntior ed ab ove.t

Thu a ertai Tu ing mach ne ( ection 6.1.1 below) would be de cribe d by he fo lowi g six quint uples

(qo, 0, qo, 0, R) (q ' 0, ql, 0, R) ( qo, I, q" 0, R) (q I' I qo, 0, R) (qo, 8, HALT 0, - ) (q I' 8 HALT, I, - )

or ·ust

(0, 0, 0, 0, ) I, (' I, 0, I) (0, I, I, 0, ) I, I, 0, 0, I) (0 8, H, 0, ) I, 1 ' H, I, -)

w ere v e ha\ e reS( rved the s rnbo 'H'( or 'H LT') to designate a halting st· te.

One mort rem rk. Nhen we dealt v ith finite-state machines and the th ngs they could do, '>~ e ha to 1egarc the "nput data as corning from some efivironnent, so that th( descripticn of a computation was usually nc t con ainec cornpletellv in t~e de cription o the machine and its initial st te. \Vith ; Turing rn~chin tap( we l ave r ow a closed system, for the tahe ser es a~ envi onm( nt fo the f nite- tate 1!nachine part. Hence we can sp cify "co'11putction" com fete! by iving (I) the initial state of the ml chine and 2a) t 'ze co tents of th tapE. Of course we have also to say (2 ) wh ch square of the tape he scannin hea ~sees at the start. We will usually ssurr e the mad ine starts i stat q 0 •

tThe state de noted py q ij is defi ed to be th· t one of the q, 's given by the function G(b;,sj) nd sir ilarly ors;;: nd fordu.

' I j'

-- 121 ·/ TURING MACHINES SEC. .1.1

6.1 SOME EXAMPLES OF TURIN MA

The remainder of this ch pter so e o the thin s Turing machines can do to the inform tion place on heir tapes and cont asts these processes with those obtainable from finit -state mac ines. (Fo the comparison, one may think o a fi ite-st te achi e as a sp ciall restricted kind of Turing machine whic can mov in o lyon dire tion.

6. 1.1 A parity counter

We will set up a machine w

is represented on the Turing m

0

where we have printed the se in B. The machine starts (in state q0 ) at he b ginni g of the s que ce; t e B s to tell the machine where the sequ nee nds. The tes, one for odd and one for even parit , an it r it encounters a 1. The associat d fi ite-state by Table 6.1-1.

Table 6.1-1. QUI

q; Sj q;; s ij d;; q; Sj q;; dij

0 0 0 0 0 I 0 0 0 0 0 B H 0 ,, B H

qo q

If we trace the operation o e we find hat i goe thr ugh the configurations at the top of

The machine ends up at the form r sit oft e ter ina! B w ich i has replaced by the answer. The in ut se uen

PROBLEM. Change the quintupl

In this simple example the a case there is no possibility returning to it at a later time. thing that could not also be do e by n un ided finit -stat sequential input) and we know alrea y, from s ction 2.2, for this computation.

uch and ny

ine ( ith is is true

s c. 6 .. 2 TURING MACHINES 121

122 TURING MACHINES

Table 6.1-2. QUINTU LES F R PAF ENTHE IS CH CKER

Q s Q' S' D Q {' S' D Q

0 ) I X 0 I ) 0 2 0 ( 0 ( I I X I 2 0 A 2 A 0 I 0 - 2 0 X 0 X I I X 0 2

qo ql

Tracing out the operation, w e hav

A(()(( ...

[~ A ()((( ...

0 A ()((( ...

0 A (X((( ...

GJ A XX(( ...

~ A(XX (( ...

[~ A(XX (( ...

@ A(XX((( ...

G

EC. t .1.2

)' S' D

1 ever o curs ~ 0 -~ I -

X 0

and we see that one pair has been remove and the r[lachi~e is earcl ing for the next ')'. The state Q 0 is b sica) y a ight-!novir g state w ich searches (without changing symbols) until it en< ount rs a )'. It rem< ves this (by X-ing it) and goes to sta e Q 1 Sta e Q 1 is ba icalh a lelt-mo1 ing state which searches for a matching '( . If i find~ one, it X-s it an~ rett rns to state Q0 . This pairing-off continues until cne of two events occ~rs. (I) State Q 1 may find no'(' matt. If i thus read es an A('-' hich woul be the one to the left) the machine print a 0 mea ing not we !-for ned) and halts. Or (2) state Q0 may find no mo e T- . It knows this bye countering an A (which must be the one at the ri~ ht). The ,nachine th ~n en ers state Q 2 which checks to see i any '('-s remain. f there 1s one, the machine again prints 0 and halt>. If no left parer theses remain (' hich the machine finds out by reaching tt e left hand A), t e machine prints I ('-'ellformed!) and halts.

0'

s ·(. 6 .. 3 TURI:--G MACIII:\I:S 123

Thio is a compu ation whid cannot be done by a finite-state machine, a note in 4.2.2. t is possibl here bec~n sc our Turing machine can go b ck o er ar itrari y ion inte vals t to find '('-s that it had passed over e· rlier.

Let us no e in p ssing an in erestirg pr< perty of this machine. It does npt rna ever mud use c f any partic~lar t· pe square and, indeed, changes it cont nts n< mon than pnce. We w'll show later (in section 14.5) that any rr achin is eq ivale tin a sense o one with his strong restriction.

DI GRM COt VENl IONS

In nost ::>f ou machines each state will lave the character of an unidirecti Jnal search: eac1 stat 1s u. ually asso iated with moves in a single directi Jn. 1 he diagran s car be nade simper and more transparent by r cognizing his f ct. hus we c· n re1 reser t the machines of 6.1.1 and 6.1.2 bv the diagr· ms i Fig. 6.1-1 Each arr w in the diagram represents

Stalrt~~ o-,~r-t~h ~o~H~Start. 8 f..-:--'O 8 A !l--x- c 0 -1-1

Rll_t 1 Rl\ L eLl-a f"-0 V 1--X- ) A 1-H

qc q q qo q2 (1) (2)

Fig. .1-l

s me c uintuple (~ ;, sj, %· s j• diJ Then q; is the state at the tail of the a row, sj is he symbol at its tail, ij is Nritten in the middle of the arrow and on itted if the same as sj q;j is the s ate at the head of the arrow, and d 1 is tr e syn bol ' ritte msde tht hex<: gon or q;1 . If two q;/s name the s· me sate but their diJ' are c ifferent, w canr ot use this kind of diagram. 1

The most c mmc n quintup es, of the form (q;, 5j, q;, sj. diJ) are simply omitte

PROBI EM. The eader shout~ reconstruct Table 6.1-2 from Fig. 6.1-1 (b) t< be sure he 1 nders ands hese < onver tions.

6.1.3 A unc ry-to-pinar) conv rter

In ectiof!s 3 . .4 an~ 3.2 5 we desc ibed a network which converts a s quence of n pul es mto a arallel pa tern corresponding to the binary

t Th · t is no thew ole st ry, ho .vever. A rennrkabl theorem of Rabin and Scott [ 19591 s ows t at rna ·hines which an m ve bo h way , but can not change tape symbols, are no n ore (o less) owerf I than machi es whi ·h can move nly in one direction.

1Th s nota ion is based n a st ggesti< n of I' . Roc ester. It is important to recognize t~ at it c· n be sed or ly in t e pee liar ci cumst nee tr at e/1/ering each state is associated with a fi ed di ection It ju t hap ens tbt mo t of cur examples satisfy this condition. P oblem: Show that a y mac ine is quiva ent to ·uch a 'directed-state machine."

124 TURING MACHINES EC. ~ .1.3 SE . 6.1 4 TURING MACHINES 125

number representation of n. Of cour~ e sue 11 ad VICe an h ndle num ers w Start 1 only up to a certain size (depen ~ing < n th< ber o bim ler u nits ~~ -....... ~0 ~

X num ry sc 1

it contains). We can make a 1 uring do n is fo ,.

A L 0 R mac ~ine lvhich will all 1 h r--.A......_

A

sequences, however large. Let he se ~uenc e be epre~ en tee as a sequ< nee

~ I)Q-~ of l's on an infinite, otherwise b ank t ape: i r-;-:; H

~ f.--f.-- 8

~ IOIOIO' o olol ~ ~ @ L 1 x=fi) (§) \!!Jc X 0

d ha I Fig. 6.1-2 Fig. 6.1-3

The machine starts at the left most I the diag howr )l

an am In

Fig. 6.1-2. The paired right-me ving state~ act ike a pari y co nter (see section 6.1.1 ); they also remove alter nate 's (b X-ir g the m). N'hen the 6.1.4 I\ una y mul iplier

parity counter has passed throu h tht: data strin! , it e ters Jne o the eft- The mac ine i Fig 6.1- will multt ply t vo unary numbers m and n moving states, which enters an or< Bin the f rst a ailat le sp ce to the n prese nted s blo ks o I 'so nata pe of he fc rm left. When no I 's are left in th dat· strir g, th mac hine nalts. It t rns out that the A's and B's so wr tten are tl e bir ary c igits of th orig ina I ll< liOIOI IOIOI JIA 8 AIOIOIOI ••• IOIOIOI(

I, . . .

number of I 's. To see this, one an v rify t hat a the nome nts o ente mg ~ (') the right-hand R state the tape hast 1e fo belov. (T he re der

~ -m -------mss own should trace the machine throu1 h all he st ~ps.) Them chim start sat tl e sep aratir gB. If, fo example, the initial tape is

0 1 1 1 1 1 1 1 1 1 0 .. IAI1 11811 1I1IA . . . ~ A X 1 X X 1 X 1 X 1 0 0 0 tl e enc resu twill be

B A X X X X X X I X X 0

0 ) . . . AIOIO 8IXIX XIA 010101 ••• ! -m- ---n -- --~mxr ------A B A X X X ¥ X X X 1 X X 0 ;

0 :~c or 2 x 3 = 6, wh ere th e ans wer C nun ry fo rm) will be found as a block i

of X's to th righ oft ninal A. ; gain we have a computation that B A B A X X X ¥ X X X X X X 0

e ter

0 c nno bed nein any nite- tate nachi ne fo arbitrary values of m and n.

PROBL EM ( .1-2. Canst uct T uring nachi 1es which:

So the answer is BA BA = 10 10 = 10 (t ase "'). The cc mpu ation re- ( ) COil pute he sqt are o n, wh ren i repre sented as

peatedly divides by 2, writes d wn t he re main er (t 1at is , A o r B), and

repeats the process on the q uoti ent u tilth tient Jecon Ai1 1 . 1 18

e quo es ze 0. !

The structure of this proc ~ss is rath r m re c Jmplt: x th n m the ( ) con pute he su n oft\ o bin ary nu mbers represented in a form t

previous examples. The two pa rity-c Junte r stat s rna y be houg ht of as a subprocess which is repeated ov r anc ove unti som con iition , dete cted ( \A\• 1 1~1~1~ ... 1~1~18 6161. • • \6\6\c\ ~ by a "supervisory" process, is · ttain d. \ 'e ha e a' loop wit hi n a lc op" Q and can discern a rudimentary ierar hy 0 con rol. ) ssum that hey h veth sam< numt ers of digits.

This computation cannot bel one b , aji1 it e-st pte m 'pchin . ( ) dec idewl ether or not then mber is a p ime number (very complicated).

' , PROBLEM 6.1-1. It is int restir g tha exac tly th sarr e res It ca n be A 1 1 1 118

achieved by a binary counter scherr e whi h req uires nly t ~o sta es an ~ the same alphabet! Can you fine such two- tate uring mach ne? tw I the sq are cc ntains I or 0. , I ere

0 means

I

I

~~ 1~

126 TURING MACHINES

PROBLEM 6.1-3. D

PROBLEM 6.1-4.

of two binary numbers m

6. 1.5 An addressed memory

It is not particularly app Turing machine can be con"H·""·t .. rli file for storing and retriev how one might set up an o name.

LOCATION AND FILE.

Suppose that one has a ljUllliJI.fl

ciated with a name N;. Lett ranged along a tape in the markers. (We will use X's.) whose name is N. We assume the same number of digits, an

Name of

where the parentheses rep used as additional punctuat tiona] symbols, A and B, for lkee:plrtg

TURING MACHINES 127

the library, and when it g position, having changed

rd U(N) to A's and B's.

by re-

move the information to 6.1-6 to the above, it will

block in which its name

Then it will stop at the ; have the same length as

into the "address" block

how the machine works. If it seems

128 TURING MACHINES SE . 6.2 Sl C. 6. .2 TURING MACHINES 129

Again, it would be a simple m ~tter o ad ~ sta t s w hich \\ ould esto e the st: vera! tapes whi h br ing it lose in e I lcien y to the conventional com-

tape to its initial 0, 1 format. p ting mach ne, b ~t we will r ot loc k int p this Our real interest in Turing

The memory file we have cc nstn cted sa v< riety kno\\ n as n as ocia- machin es is ir the ir use ~san ode I ford< fin in effective computability.

tive memory. Items are recove ed di ectly by m ~tchir g wit ~an ~me' vhich is paired with the item. In a onv~:: ntion ~I ser ial cc mpu er or e ret ieves items from a memory by a m< re in ~irec t proc ~ss, n meh , thr< ugh efer- 6.~ SCME R LATI< NS 8 ~TWE N Dll FE REt T

ence to their location in a fixed arra of st orage devic es.t here are c rtain KIIIDS C F TUI lNG ~ACH NES

advantages to having the optic n of< ssoci tive neme ry in a con pute , but we cannot stop to discuss this ere. 6.3.1 wo-S mbo I Turin mac ines

We have place no 1 artie lar r1 strict IOno t1 the variety of symbols that

6.2 DISCUSSION OF TURING N A CHI ~E EF ICIEN CY may oc ur o the apes pf ou mac hines save that each machine can deal w th ju t sor e fin te se of S' mbols. It is int resting that we can restrict

The reader who has traced out tl e abc vern chin son ~ctua tape s will 01 r rna hine to t e US< oft wo sy 'rnbols , witl out loss of generality. (In

have observed two compleme tary featu es of the s "tuati pn. ( n th ti ite-a ~tom · ta tt eory this s ren inisc ~nt o f the equivalence of some one l\1 och-1

hand, he is becoming convin ed tl at or e car indt ed d P a ~ reat jnany cCul itts ret we rks, whic ~ ha lve b nary signals, to all less-

re strict d fin te-st2 te m~ chine s.) things with Turing machines that anne t be done with fin itt -stat< rna-chines-and also that one ca r do surpr ising! ~ cor t1plic< ted t ~ings with

To how thee UIVa ence, one ll.as t D do little more than replace the

rather simple machine structL res. On t e ot er h nd, e wi I hav ob- S)mbol by t>inar nun bers. Sur pose that ~ Turing machine T uses k

served the staggering inefficie ncy c f Tu ing nach "nes i 11here nt in their differer t syrr bois. Sup pose !so t ll.at t e nu mber k has n binary digits.

repeated passage over long blc cks o tape to pe rforrr the nost Ierne ntary Then v e ca ass ·gn tc eac syrr bol f T a distinctive n-binary-digit

operation of moving a tempor ry m arker No one h as se iousl pro Josed n mbe . No wwe c:an r place then a chi e T y a new machine T* which

the use of a Turing machine s ructu e fo any pract calc mpu atior. In- w II tre t its tape, in ef ect, t s tho ugh i s squ ares were grouped in blocks

deed, I do not even know of 01 e bui t for de me nstra ion p urpo es. OJ "fengt h n. < n ea h of hese lock IS W itt en an n-digit representation of

Nevertheless, despite the sl owne s of he rr achin es, th ey de not eces- sc me s mbo of tt e old macl ine J . Fo eac state of the old machine T

sarily make inefficient use of tl e tap mer ory 1 apac ty. w will give *a c ollect ion o stat<: s, an< thes will be arranged somewhat li ethe units ofth seq enti< I bin ry dt code 3.3.1. In fact we construct

For example, we can n ake < mac ine t b dec· de if an n-~igit inary a ~inar tree n Ia ers d eep, f rig t-mo ving tates. If the machine starts number is prime, using les s tha n ac ditio1 al ta e sqt ares. Perh aps it , a1 the I ft of a syr bol lock then whe r it h as reached the right end of should also be noted that tl e higl spee of o dinar com puters isba ed on Hat bl ck, t pe po 15ition of it stat in 1 hat t inary tree will tell precisely their use of "random-acce s" m morit s whi h on y app ear tc esc a e the wpat s1 mbol (fror r T's old a lphat et) w ~s rq resented there. It is then a problem of serial search al ng a tape. If or e thin ks of such fn.achi t1es a5

'

sifr1ple maw r to ttacl , to ach ermi a! st ~te of this tree, a chain of existing in a sort of three-di rnensi nal t pe, o re car cons ruct a rgum< nts tc n left-g ping tates whic writ e (in e or< er) the binary digits of the rever show that Turing machine s of ppro riate kinds are r ot fu fl.damc ntall} represe 11tatic n of that ymb I wh ch T d then have written there. slower than other kinds off nite-s ate-pi us-me mory machi nes.

WOU

F nally if T were o me ve ri! ht, w e hav e to< djoin n right-moving states,

As we will see, it is possi ble tc exec ute t em< st el bora te po ssible a d lik wise for rr otior to t he lef . A deta· led construction is given in

computation procedures witl Tur ng n a chi es w hose fixed stru ture S anne n [19 6]. s ee als o sec ion I .5 fo a sir niJar construction.

contain only dozens of parts. One can i [nagi1 e an in ten tellar robe t, f01 whom reliability is the prime onsi erati pn, p rforr hing · ts co [nput tion~ in such a leisurely manner, ovc r eon s of s are t me. 6.3.2 f>ingle and c ouble infini e tap s

There are modifications of the" urin! mac ~ine rame f,vork, e.g., us in~ It c n als o be how that Turi ng m ~chin s which have tapes infinite

tThe point is that. with the assoc iative (nemor , one does n bt hav to kn OW wh re tht in both direc ions ~ave fl.o ad ~anta ~e or disac vantage over machines with desired item is located. sifl.gly i rfinit tape s. It . s not wort sho fving his here in detail. Suffice it

130 TURING MACHINES

to note that one can show the tapes, and then mapping the squares for each half. Turing to problem 7.4-l.

6.3.3 Multiple-tape machines

6.3.4 Two-state Turing machines

PROBLEM 6.3-1. Design numbers, generate

SEC. .3.4

quiv lenc by oldin the doubly in nite into the ingle tape by using alte nate

1936] uses this eth d. ee th sol tion

o an

onthe of

he b nary tape and

gain

all ell-f rmed

The.order does not matter, but n seq ence may ppear mor than once. Foil ow the same ground rul as in first p oble .

Hint: Enumerate the bi ary umbe s as i pro lem 6.3-1. s th y are generated, interpret them s pa enthe is se uenc s an the for well-formedness (using 6.1. ). If detail beco e to nast , jus sket h out the problems and how to de I wit them.

s c. 6 . .4

ROB

(4) Prove that righ -mov ng T finite-s ate m chine.

(5) Can y u do rob! m (I) with (6) Can y u thi k of

y som Turi g m chine

TURING MACHINES 131

ring machine "accepts a finite

Turing

machine can be replaced by a

right moving machine? Prove it. sequ nces that is not the set accepted

EC. 7 0 UN I\ ERSAL TURING MACHINES 133

,~nott er wa y rna hem< ticiar s rna defi1 e a f1 nction is:

A func ion i a se t of rden d pa rs (x, y) such that th ~rear ~no WOp fiirs v ith tl e sar ~e first number, but fo eacl X, t ere i alw< ys or e pai r with that x as its fir st nu rber.

If wet hink ::>fa f uncti< n in this v ay, t 1en t e function F above is the set c f pai s

7 UNIVERSAL j

0) (2, 2 > (0, ( ' I) (3 0) (4, I (5, 2) (6, 0) ... TURING MA CH NE!,

Is here ny d If ere ce be tweer thes defi itions? Not really, but there < re se era! fine r oints Th sec< nd d efinit on is terribly neat; it avoids many rick' logic al po nts- to co mput the value of the function for any < rgum ent x one ·ust f nds t he pa ·r tha t star s with x and the value is the ~econc half ofth pai. Nc men tion s rna de of what the rule really is. (Then mig ht no t eve n be one, thou h th at leaves the uncomfortable

7.0 USING TURING MACHINE TO OMP JTE c uesti ::>n of what one c ould seth e fun tion for, or in what sense it really THE VALUES OF FUNCTIO liS exists. The first defini ion t ies th P. fun tion down to some stated rule for

; comp ting ts va ues, JUt th at Je:: ves u s the question of what to do if we In chapter 6 we saw a n urn be r of ways to m ake urin ~ rna hine can th ink o f difft rent ules or cc mpu ing t e same thing! For example,

manipulate the information o n the r tap es. I eacl case we s tartec witt (l divi de th nur 1ber by th 1d the value is the some information on the tape (usm lly a binar v or er, b

ee a nary num ut oc remain

casionally a string of symbols . Th hine \ rted ·n sor 1e sta ndarc er

mac as st (2 add up tt e nur 1ber' digit s (bas e 10, of course) and state and tape position and al lowec to r nun il it a! ted. w en (a nd if di vide t lzat b thre and take here nainder the machine stopped, we loo ed a the · nforn atior now on i s tap e anc regarded this as the result of a com utati on. ,' re tw o rule s tha , as n any c hildr n kn ::>w,g ve the same values (prove it!).

Now in general, what is o n the tape when the r 1achi ne stc ps w ·u de two ld be a nu isanc to t hink of th m a two different functions, but pend in some complicated w yon what was m th tape at th e sta t. Sc t hey a etw diffe rent ules; owe have a pro blem with our first definition. we can say that the tape res ult o the omp utatio n is fun tion Jf th In ordi ary ife, c r eve n in ordin ary r nathematics, worrying about

input. Exactly what functio 1 it i dep nds, of co urse, on '-" hat unn uch hai -split ing r 1atte WOU ld be silly One just says, informally,

machine was used. Hence w can think of a Turin g rna hine as de fil . nzng, 'Well ' if t'>' o rul s do in fa t cor 1pute the s arne values, we will think of

or computing, or even as being a fun ction. hem 1s de ning the s 1me f uncti n. he d ifferent definitions, while not

What is a function? Mathe mati ians have ever I mo e or esse =tuiva denti al, a e eqt ivale t." Now we w ill in fact take this common-sense )

lent ways of defining this. Pe haps the rr ost u sua! efini lOlli som ethin ppro ach, JUt \\- e ha e to be ca reful to be clear about it. This is not

like this: Jecau se we are t ying just t o be errib y car eful and logical and so forth,

A/unction is a rule v here Jy, gi en a numt er (ca lied t he JUt r< ther becau se w are study ing t e th ory of rules and definitions

argument), one is to d ho1 to c mpu te an< ther numt er nd t e lik e as ur p I mar sut ·ect r atte . We have to make a clear i

~istin (called the value oft re fUI ction fort at ar ~ume rt). tion ~etwe en w at a ule d pes a d ho w it is defined because that is xactl ~ wh< t we are s udyi g-r ot be cause of an obsessive, or evangel-

For example, suppose the rul that de fin s· a f ~ncti< n F · s "th rem ~in de stic, pr th rape tic d esire to pr pmot mo e clear and logical thinking when the argument is dividec by t ~uee.' Tht n (if ~e cc nside ron! non- ener lly. say his bt caus< man !! oft e ne er mathematics texts belabor negative integers for argumen ts) w< find hat he di tinct onst etwe< n fun ction and rules flnd ordered pairs to the point

F(O) = 0, F(i) = I, 1,;'(2) 2, F(3) = 0, F(4 ) = I, et . hat t e stu denti s mo e con fused than hew· s before, precisely because in

132

134 UNIVERSAL TURING MACHI ES SEC. 7.0.1

the subject matter he is studyi~g th distinctio~s an so uf!imp rtan that there is little or no danger of cc nfusipn to begin with

So we will think of a fun< tion s an asso iatio bet !veen argur[lents and values-as a set of orderec pair (x, J) sucl that there is ju t on( pair for each x. For each function here may ~e m ny d 'finiti ns o rule that tell how to find the value y, givbn tht argument x. T1wo de~nitic ns or rules are equivalent if they define th same function It nay l e very difl cult, given two different rules, to te I if t ey a e in fact ~ quiv~ lent. In f ct it may, in a sense, be impossible, an j this will form the entn I pre blem discussed in chapter 8!

7.0.1 Functions of non-negative integErs

Mathematicians use ~he nc tion )f fur ction very broadly ir that they allow almost any kind of thing to be an a gum( nt or valu . For instance, we can define the function wh )Se arguments a e sen ence in Englis and whose value, for an argument , is t e sub ·ect o x.

F(Mary hab a lit le la1rb.) f= Mary

We will not stop to discuss v beth r thi defipitior could ever be friade really precise but will just obs~ rve tl at hue w~ are t~inki~g of a function that has a sentence, rather th~n a numl er, fc r an argu (nent. An< ther, more common kind of function is he fu nctio ~ wh se a gum( nt is a set and whose value, for an argument x is th set olr subsets oj x:

F([a, b, c]) = [ [], [a], [b], [ ], [a, b], [a c], [l, c], [c , b, c ]

where-as most children today knc w-[] is tl e empty sd and, incl11ding that, there are eight possible mbsets of three bjects. n is kii d of function is called a set function I ecau e its argument and values ar sets instead of numbers.

In most of what follows, we \ ill deal with a very simr;le ki d of function-namely, functions IVhos~ argument and va)u( s are non-negative integers, i.e., are in

, I, 2 3, 4, ...

There is nothing really special abou this; the t eory woul j be he same in intellectual content, only just nore omp icate , if v e used ne~ ative numbers· or rational numbers as we II. (I would be diffe1 ent if we a loweb rea numbers; that is what the [op wna] chapterS is at out.) Ins me c f ou1 theory, particularly in the pre ent < haptc r, we encc untei som fun tion' whose arguments are a little 1hore ike s nten es ir Eng ish. But nles~

otherwise noted, we will tal on y ab[mt fi-Jnctic ns t at cpncerh th( non-negative integers.

EC. 7.0.3 UNI ERSAL TURING MACHINES 135

We fire a so in erest~d in the 1 ules or d( fining functions, and for the ~ext C w chapter we ~ill consider m inly ~ very special kind of rule~ame y, deJ ning unctibns in term of the beh vi or of Turing machines.

17.0.2 Functions c efine by T ring rochi es

We have already r oted, at the beg nnin of this chapter, that we can defin( a fm ction in te ms of the beha' ior of a Turing machine; the argument is wri ten on the tape t the start, and he value is found on the tape when the n achi e stops. In chapter ( we c iscussed how to do this for a numl:er of arithmetic unctions ir eluding

r 2, parity(n), + y, xy, etc.t

From now on, ~ e wil fix cur at ention alrr ost entirely on functions that an b defined i this way by T 1ring mad ines. In order to be able to alk ~ recis ly about s ch rr atter , we will ave to agree on some rather trict defin 'tions I w nt t< reas~ure he n ader about these in advance.

lfhe c efinit ons I elow may seem very f!arrolw and restrictive. The reader ~ill b abl( to th'nk o manw oth r wa she could use a Turing machine to ~efinc a fu ~ctiOJ , anc he n ay feb! th~ t in the restriction to doing it our !way < grea deal is beirg Jo t. H sho ld k ep these objections in reserve 1-Jntil ~e ha con plete~ thi~ cha1 ter. Only then can one see clearly the ncrec ible ange over whic~ striking! difeerent kinds of definitions, in his g nera area com out nnall to h vee~ actly the same character.

7.0.3 Turi g-cor puta le fur ctions

A funct ·on f( ) wil be sG id to be Turing- omputable if its values can be omp ted ly son e Tu ing n achir. e T1 whos£ tape is initially blank except or some strmdard rep esentr1tion of th argt ment x. The value of f(x) is

what emai11s on he taoe wh n the mac/ ine stops. Now there is a rea pro!: !em i the bove definition unless we agree on

a "st, ndari represent tion" for umb rs. For what doesf(lll) mean? If the n1mbe s are una y it nean f(3) if b'nary, f(7); if decimal, f(lll). Simi! rly, IVe have to choo e a tand rd c nvention for interpreting the value resul . It urns out, it doe n't n uch natter what we do, so long as wear con isten througho t oui theo y. Sc we will assume, unless otherwise oted that numl ers w'll be epre entec in unary notation. They will also~ omet mes I egin find e[!d wi h otr er sp cia! symbols (for punctuation,

t 0 r disc ssion bout< rderec pairs, etc., dpesn't ell us what to do about functions of severa argUirents, e.g., j(x, y) = x y. k::Iearh this needs something more-say, ordere triplts. Thi slight y com licate< matte is dis ussed in section 10.2.

136 UNIVERSAL TURING MAGill\ ES SEC. 7 .0.3

so that the Turing machine ca , for exan pie, ell where a number ~nds and where blank tape begins).

We have just said that if th::: rep esentation of m mbers rerr ams onsistent throughout our theory, ·t do{sn't much matter wh tit i On the other hand, if we do alter the represe tation of r umb{ rs wi hin t e tht: ory, we obviously do change the fu ctior that a Tu ing machine can be l:Onsidered to compute. Indeed, fo a gi\en Tt ring mach ne, t ere r ight be a great many different functions asso iated with diff{ rent ~ays of n presenting the arguments on the initi I ta1 e. n th s ch pter we show Turing's great discovery: that tl ere i~ a Tu ing r achi e so ensit ve in this respect that, by properly adjusfng tt e inp~t reJ resentatio , we can c~use it to compute any Turing-computabl~ fun tion t-tha1 is ary fun tion that can be computed by any other urin macjline' hate er!

To summarize his asserti01 brie~y, Itt us ~gree that f is ~ Tu ing-computable function, and Tis< Turirg m chin< that omputes i if

when the number x i~ writ en ir una y no ation on ' blank tape, and Tis s arted in st~ te q0 on th rightmos I of x, then, when th{ machine tops, the number f(x will appear in unary notatic non the tape.

Then Turing shows how to cor struct a si gle, fixed machine L with the property that for each and eve y n ring nachine T, therr:. is a string of symbols dT such that

if the number xis wri ten ir unary notation on a blan~ tape, followed by tht strir g dT and U is star ed ir q0 on the leftmost symbol o dT, then when tht machine stops the nmpber (x) \\ill apbear < n the tape,

where f(x) is the number tha wo ld h ve b en compt ted if the machine Thad been started with c nly x on it tape

This is what we are to provt in this chapter. Befo e doirg so let us see what it says. It says that no m< tter how C< mplep<. the tructure o a T ring machine T, its behavior is with'n the read ofth fixe machine tJ. It does not matter that T may be very ~1Uch large than U in its n mbe of s ates, or that T may use more and dif eren1 syml ols tl an U All U net: ds is o be provided With a Certain String of sym~OlS t T, an b it C n CO fnpUt the arne furk:tion that T does. What is T? ( bvio sly i is some way in whic the machine Tis described to U.

The existence of the machine U is a hief supp rt of the hesis that Turing's notion of computabi ity is an accept:~ble echniral C< unte part of our intuitive notion of effect ve ccmputabilitv. Fer on the one hand it is certainly hard to imagine an obj ctior that Turing m :~chin ~s an too powerful. All they do is read, write, and movt . Or the other hanc , no

' 1

SEC. 7.1 UNIVERSAL TURING MACHII"ES 137

cne has eve beer able to think of any hing a universal Turing machine canno do that one could r asonably sk o a well-defined instructionobeying pro ess. As w will ee later, t is id a has stood the test of time. P II other formu ations that seemed otherwise satisfactory have been shown equi,alent to th noti)n of Turing con putability.

Th universal mach ne qt ickly leads to some striking theorems bearing on wh t apbears to b the ultimate j tility of attempting to obtain efji ctive crite1ia for ejfec ivene s itsdf-t at is to recognize which descript ons of pro esses do in fact escripe efl~ctivt: processes. We shall discuss s pme c f the' e rest Its in the n xt chapter.

Fo tuna ely, the dt mon>tratic n th<t there exist universal machines, and even th ir C< nstruction is q ite S raigr tforward, and ill Uch of the "'ork t as ali eady ~een ~one n chapter ( . Th following sections complete t e co I struc ion.

7.1 T E UNIVERSAL M CHINE AS AN Ill TERPI ETIVE COM PUTEI

Th manner o ope ation of a universal uring machine is familiar to n any omp ter p ogramme sin t 1e fo m of "interpretive" programming s stems. The id a is this. Let f(x) be 1 uring-computable; then, by definit on, t ere i som Tur ng machin T \\ hich can compute its values. 1 hat i , for each valw of -', rep esen ed o the tape of T as a certain s ring ·x of symbols, will eventual!\ halt with an appropriate string s (x) re naini 1g on its t< pe. t is not important just what form is chosen f r sx, provi~ed tl at Wt agre througho t the discussion to keep it fixed.

No~ if he re~der f,vere ~iven a de cription of T and also of sx, he cpuld race ut th~ beh~vior of T with nput s x and find for himself what "'ill be the c rresf ondi g value of 1r(x). This is what the reader has done, presun ably, for spme f the mac ines ~escr bed in the previous chapter. 1- e required pres~mab y, an exte nal s orag medium-paper and pencil

SOm timt, and a pre ISe unders andi g Of ll.OW to interpret the descripfon of each machine.

Th uni' ersal mac! ine ' ill bt giver jus the necessary materials: a d serif tion, on it tap , of lr an< of . x; some working space; and the built-i1 cap< city o int rpret corr ctly he n les of operation as given in t e description ofT. It beh· v1or Nill b very simple. U will simulate the b havi or of Ton step at a 1m e. It w II be old by a marker Mat what point c n its tape r begins, <1 nd tr en it will eep a complete account of what 7 's tar:e loo s lik at each n orne t. It will remember what state T is supposed o be n, an j it can see what Twould read on the "simulated" t< pe. , hen 'J will simp y loo at t e description of T to see what T is next

138 UNIVERSAL TURING MACH

supposed to do, and do it! Th in a table of quintuples, to fin move, and what new state to into which is infinite only to the left, and These restrictions are inessentia , but

To make the situation more conc)ret<e, machine U has its tape divided ·

Pseudo- tape of T (single-ended)

The infinite region to the machine T. Somewhere in this $erm-llntinite where the reading head ofT is clurre:n~lly tc>¢arect.j

The second region contains the there is a space in which we just about to write, depending is a region, which will be description of T.

The structure of a Turing diagram, i.e., by the set of quin its states, inputs, and outputs. realized simply by setting down down on U's tape in a very str quintuples are represented in used for punctuation.

SEC 7.2

' I

TURING MACHINES 139

binary numbering to desigthat the universal machine

machine, hence it is limited trary, the machines to be

ay require arbitrarily large different symbol for each T machines will be 2-symbol ) Our problem is solved as binary number, we will use tes. In Fig. 7.2-1, k was

only single binary digits for D;j·

we need to represent T's CUI!rreJnt lstat.e1~;vnllbl,ol pair. For this we will use

the region of the tape called

on T's imitated tape.) over the leftmost X as

description-

four parts: . 6.1-5 is put into operation. te-symbol pair that matches

cQJildlttipn" area. On the path to the d 1 's to A's and B's. After to A's and B's, it runs back

the tape looks like Fig. 7.2-3. -6 is put into operation. It to the right until it has gone some O's and 1 's. These O's

140 UNIVERSAL TURING MACH

and I 's describe the new state eventually be printed in locat Next, the machine copies the machine-condition region; it Now the tape looks like Fig. 7.

(3) In its third phase, the m There it erases M and prints UemiJ<f!r·antly) ( = A or B) which it has until tained in the choice between Fig. 6.1-6. Then the machine B's on the tape to O's and I 's, that now represents D ij. Fin a ly, leftmost X. It erases the Sij w its place. (Sis a special letter done by the machinery of Fig Fig. 7.2-6.

From Fig. (L) M 6.1-6 ~

A

region

Write direction symbol

Fig. .2-5.

ig, 7

''

141

machine T is about to be it encounters for the first

direction Dij that T should in place of the A or B, and

wt~etlher Dij was A or B. Then ~·~·~., .. ~·y•u """'"rnPr it is 0 or I, and prints

to the right until it reaches ~ernerlllbereljl symbol, using A for 0 and

M ~Start B~g6.1-5 Wnte Locate new quintuple

symbol

Fig. 7.2-7. At this point, the all over again.

operations? The net effect particular state-symbol pair

~lfituple [Q;, S;, Qij, Sij, D;j] and it printed the new state

the machine carried out the ared to start again on the

142 UNIVERSAL TURING MACHIN S SEC. 7.2

Halt

ffix

/ /

/

( I

I I I

I I I I I I I I I _I I I I I I I I I

_j

I

' l 0 0 y I A

0 0 y 0 B

B

I A

0 A

0 B

A

0 B

I B

I B

I A

TURING MACHINES 143

Jete diagram is shown in 7.3 we will trace it through achine, presenting the tape

pie, a 2-state, 2-symbol The diagram of

1~1 L 0

,---c q1

Fig. 7.3-1

mla,chm~ proceeds to "count"- that 100, 10 I, II 0, Ill, 1000, etc., one

ilntere:ste:d in T itself but only full initial tape to describe

machine being simulated Q 0 , and sees there the sym-

0 0 01 I 0 0 0

0 0 OJio 0 0 0

0 0 l1 0 0 0

0 01 0 0 0 0

0 I Oo 0 0 0

0 o:::::::Il 0 0 0

0 I 01 0 0 0

0 rr=o 0 0 0

0 l1 0 0 0

0 l1 0 0 0 0

01 0 0 I 0 0 0

144 UNIVERSAL TURING MACHIN S SEC. 7.4 s 'C. 7. UNIV ERSAL TURING MACHINES 145

bol I = S 1• The reader will no te th< t we have mark ed ce rtain poin s in seve n stat s-b ut as here ader Nill Se e, th2 t machine, or any like it, would the diagram (Fig. 7.2-9) of U Nith he S)' mbol & ~ ~ et . B low be u nsuit· ble fc r exp osito YPUI pose~ are the tape configurations that occur at th ese pc ints c fthe first c ycle. ( )ur u niver al T uring mac' me as tl e property of simulating the

beh< vror Jf an Tur ng m achin e wit any input tape. There is no need start OOOMOOOYOI ? 0 0 0 0 I 0 I I I 0 X I 0 I I X I I I 0 0 y

to ir terpr et th( result na row! , in of any particular set of arith-erms ill OOOMOOOYOI ~ A A ; A 8 A 8 I I 0 X I 0 I I X I I I 0 0 y

met" atior s; the Uni' ersal mach ine c<: n simulate any Turing machine cope ill OOO.d_,000Y88 .\ A A I A 8 A 8 8 8 A X I 0 I I .\ I I I 0 0 y

acti ity v hate Acce pting Turi pg's thesis, conclude that the er. we & OOOAOOO Y I I X _.j J I A 8 A 8 8 8 A X I 0 I I X I I I 0 0 y

univ ersal mach ine c n sin ulate ~ffect ve process of symbol-manipu-<- any & OOOAOOO Y I S x 0 0 ( 0 I 0 I I I 0 X I 0 I I X I I I 0 0 y

a then atica I or ing else; it is a completely general <- Ia tic n, be it m ~nyth & 0 0 .H,I 0 0 0 Y I A X 0 0 ( 0 I 0 I I I 0 X I 0 I I X I I I 0 0 y

inst1 uctio ~-obe ~ing 1 ~ech< nism !i1 OOMIOOO Y I A ,\ 0 0 ( 0 I 0 I I I 0 X I 0 I I X I I I 0 0 y <--

PRO BLE~ 7.4-1. Sh w he w to get a ound the restriction that T, the Each time U reaches stage & , its mach ne t< pe h s th con lfigun tion mac ine b ing si lnulat1 d, is r mited to at pe infinite in only one direction. shown to the left in Table 7.3- . To the r ight i wh< t T's tape looks like

PRO BLE~ ~.Co at the same stage. The subsc ipt 0 p T's tape in die ates he Ia ~t syr f1bol 7.4- ~pan d to hem chine T that it is simulating, the

scanned, and is I if the scannin stat Q~. nd 0 if th( ning tate univ rsal r achir e WO ks ve y slo ly-~ o slowly, indeed, that one would was scan hard find i [1 it a y pn ctical application (especially in view of y exp ct to was Q0 •

nd of <~ how low r gular Turin gmac hines re to begin with). And the relative speed The terminal, right-hand Y nark thee the d scrir tion egior. If of U npare d toT deere the I ngth of the active part of T's tape as co ases a

the location phase does not yie dan quir tuple begi ning with Q;Sj the grow s, bee use L has o tra el all the '.1 ay down this length and back in process will encounter this Y a n.d ha lt. A stan ard vay t J rna ~e TL ring 1 each simul tion ycle. We c n rec esign U to prevent this further decline. machines stop is to assign no c uintu pie tc som e Q;5. j• an d u ecog izes Sket h hov this c an be done. this convention by halting on tt is Y. 1- int: I eplac e the s vmbo 'M't y a co py of the whole instruction region.

I Note that t he rna hine U, anc even he ne w machine just constructed, makes

PROBLEM 7.3-1. Why oug t we rovid a 0 ( r a 1) after his Y as eft cient use of theta peas c oes T

PRO BLE!\ 7.4-3. Co nstruc t a Tu ing rr achine which, when started with a 7.4 REMARKS blan tap< , pri ts its own descr iption (as a sequence of quintuples).

fr Che ter Y . Lee 1963] calls machi nes w hich can print their own descrip-

This construction of a unive rsall uring ine i reasc nab! stra ght- tion~ "intr pspec ive," althot gh I ~auld reserve this appellation for rna-mac

s whi h prir t wha they hink s thei own description. 1: chin forward in the way that differe rt pa ts of the I roces s-fi1 ding the c uin-[ o not tthe olutio n unle ss des iJerate! onsu

tuple, changing state, reading, prin ing, ~nd 1 povin g-a e sq a rate d in parts of the state diagram. At thee pens of r ~ore tates and/ br the use PRO BLE!\ 7.4- fl. Skc tch ot tar~< onstr ction of U that uses unary rather of a larger alphabet, we cou d h< ve rr a de he operat on ellen 1 ~ore than binar strin s tor prese [lt T's states in the encoding of the quintuples. straightforward, but we wished to ta e the opp brtun ty to illust ate 2 few more tricks and techniques tha t can be w ed in Turi g ffi< chine buil< in g. Some of these are hidden; the eade will disco ~er, f br ex mpl( , tha the symbols M and S are superftuc us ar d co ld be rep! cedI y X' thro ~gh-out by just changing the letters n the diag am. f one is in erest din con-structing ·a very small universal mach ine, o ne ca n. do r nuch bette ; but then it is necessary sometimes to us the same state for ever a I diff rent pur-poses, and the machines becom fill( h me re di ficult tour derst and. One also may gain by making chang es m hew y th mac hine escri ption are represented on the tape. In ch apter 14 w will com ine ~ uch echni ques to produce a very small machir e-o n.e wr ich u es fo ur sy nbol~ and only

l

8 LIMITATIONS EFFECTIVE SOME PROBL INSTRUCTION

The universal machine can d do, although more slowly. It characterization of just what form as we have for finite-state II[HicrHI[les.

this is a much more mysterious evidence that it must remain so the things that we can prove

8.1 THE HALTING PROBLEM

When a Turing machine, may be a very, very long time comes to a halt. For many m "computation" may go on procedure which would enable determine whether the process can be no effective procedure w argument is the commitment computations include all eff if there is any decision pn::>ct:dtH~ that the procedure is effective) which can carry out the proced such a procedure.

tThe term decision procedure, means all, that will enable us to solve the

SE . 8.1 NSOL V ABLE PROBLEMS 147

ine gives a slight simplification descriptions of T and t on

it is the case that T stops for t, e difficult case that T does some way of looking at U

ich it will never stop," we on "Does machine T halt

!em: "Does the particu-

m" for the tapes of one t there can be no effective

first, is based on finding qo1mrnHatiop can take if it is ultimately

tly feasible. For T uses a nu¢tJer Q of states while t has

Surely we should be able h that, if the machine has will never stop. Then all to compute this function

can begin to imitate the number of cycles. If the

N), then U can stop and This would be an ef-

aq~u1me:I11t except in the innocent asN)! This bound certainly shortly) that no machine of the obvious fact that But the bleak fact is that

In fact, since this argu-

mac~ines with S symbols and Q states, length N. So there are only a hence only a finite number of of the longest of this finite

lty in finding the largest of a

. So far as the decision pro' Q, N) such that g(S, Q, N) ~

148 UNSOLVABLE PROBI.EMS

ment is otherwise correct, the cedure for this problem will serv

The result of the simple but cjelica~e the most important conclusion until perfectly understood.

Let us suppose, as a hypothe have a machine D which will computation will ever halt, tape t. Then D has the form of

[Q]:

If D can solve the halting then it can certainly do it for tape t is a description dr of T i not concern ourselves with the lr1IIPctl"''n

ested in such introverted calcu the notion of a man contemplat

The two items of informati tions ofT and of the tape tto construct a new machine E T, but behaves otherwise like D to use initial tapes of the form

Let E be made of D, together w symbols. Then k can start with

UNSOLVABLE PROBLEMS 149

hypothetical machine £, halt exits, one printing "No" if (dr, dr) never We now make a trifling

if T eventually halts, given dr

if T never halts, given dr

ain a new machine£*; the the "Yes" exit. (See

nrJ,nP•rt.l. that it halts if T applied to the killer: what happens if E *

appl to dE* does not halt, and ude that such a machine

st in the first place! 1

__,____~ (x= any symbol)

ose readers who have not argument before. It

which forces us to discard ~JI,.cc.~c1'a notion whose absurdity

related to the argument numbers. In fact the un-

demonstrated by a form of present form because it

diagonal method. We will the next chapter.

150 UNSOLVABLE PROBLEMS

8.3 SOME RELATED UNSOLVABLI DECISION

PROBLEMS

SEC. ~.33

Several other impossibility esult are easil ded ced rom the basic halting problem result.

8.3. 1 The halting problem

It follows immediately that we can't I ave machine to ar swer the general question, "Does machir e T I alt f< r taJ=e t?" because tl is w< uld . include the solution of the prot !em, "Does T t alt f< r d1~" dis~.;ussd in the previous section.

8.3.2 The printing problem

Consider the question, "D es machine T ver rint the s mbo S 0

when started on tape t?" This also annct be 1ecid d, for all T, t), by any machine. For we may take each mach ne T (whid doe not rdin rily use the symbol S 0 and alter tha1 macl ine s that befo e each of ts hating exits it will first print S 0 • The1 the 'prin ing ~ robleln" f< r the new machine is the same as the hal tin~ prot !em fpr th old mach ne. ~ince any machine that does use S 0 can rst te cor verte~ to ne t at d esn't (by changing the name of S 0 to that of soPJ.e u1 used symbol), a[ld th n alt~red as above, solution of the printir g pro~lem woul ~ give solu ion tp all alting problems, and we know tha that cann t be one.

8.3.3 The blank-tape halting problem

Is it possible to build a mad ine "hich .viii d cide, fore ch rr achine T, "Does T halt if started on a blan tap ?" I ther were a machine to wive the problem of 8.3.1 above, the that mac! ine .., ould solve the t lank-tape problem as a special case. But the t nsohabilitv of he p oblem of .3.1 does not immediately imply tr1e ur solvability of t e pDblerr of .3.3 since the latter task might seen eas er because of its apparent y smaller domain of cases. However, we an s ow t at the problem are quiv lent by showing that for each machine-ta e pair ( T, ) the e is a corr sponding blank-tape halting problem for a cer ain o her n achir e wh ch w may call MT,t·

The machine M 1 .1 is canst ucte< dire tly f om he d scrip ion pf T and t by adding a string of ne' stat s to he d agrarf1 for T. L~t us suppose that the computation T, reqt ires hat be ~ tarte1 (in its i itial state) at the indicated position fa U pe:

SEC. 8.3 4 UNSOLVABLE PROBLEMS 151

T~e nef\' machine M 1 ,, will pegin sam< where on a blank tape with the st ing c f stat s

Star --(~0 r:l--@1~r2-···~ rm~'r<FJ--x~ rm+2-.., . . .

where 'X' is ~orne letter not ther vise cccurr'ng on the input tape t. We see tha M 1, mus be equiva ent, tarti g wi h a blank tape, to the mac ine 1 start' ng w th ta )e t, because M 1 , sim )ly writes a copy of t on the tape, positions itself cc rrect y, and then becomes identical with T from nat tine on

It f< II ow that if we ould decide the haiti g of the blank-tape computations i.e. olve he p oblern of .3.3, this .vould lead to the ability to d cide bou the nachines A 1,1 a1 d he ce al out all computations ( T, t) o the probl m o 8.3. . Sipce t is is impossible, the problem of 8.3.3 rr ust a so be unso vable.

8 3.4 he u1 iform halting pro lem

An Dther in ten sting prob em a ong his li [I.e is this: Can we decide for a [I arbitrary machine T "Do~s T alt f r eve y input tape?" It turns out tl at th s pre blem inclu~es tl e pn blem of 8 3.3 as a special case and is tl erefcre unsolvaple. ~e s etch the reduc1ion. Let T be an arbitrary Turing mad ine which, as us a!, is to be start din state Q 0 . We construct arwthe machine 8 by addir g son e sta es and quintuples to T. T 8 starts ir state q~. 2hoo eA and B to be two ymbols not used by T. For each s ate q ofT we ad 'oin the qu'ntup es

(q;, 4, q!, 0, L (q;, ~. q!' 0, R)

{q;, <I> q;,A,R I {q!', <I>, q;, B, L I

"here 'J [ anc q !' a e tw< new state add d for each q; and <I> denotes every s mba (thu q! r places the scan1ed s mba, whatever it may be, by A and th n mcves r'ght tc q;). We a so add thn e new states

lqf <1>. q' , o, 1 I {q!, <1>, Qo, B, Ll

and st: rt th1 mac~ine ip q~. Nm we !aim that the problem of whether 1 will IJalt or a bl~nk t pe is equi' alent to th problem of whether T 8 will halt or ever tape hence the probl~m of 8.3.4 is unsolvable.

FROBI EM .3.-1. Veri y this ast st teme t

152 UNSOLVABLE PROBLEMS SEC . 8.4 s c. 8.6 UNSOLVABLE PROBLEMS 153 \

PROBLEM 8.3-2. Show th t the unsol abilit v oft he pre blem of sec tion ]em of his c 1arac er oc urs i 11 sect on H . 6, T 1eorem 5 . Unsolvable prob-8.3.4 implies the unsolvabili y of he pr ~blem of se tion .3.1. (We just ]ems o this some what arro ant < hara ter a re studied in the theory of showed the converse.) " reati ve" a 1d "p oduc tive" sets- see F oger [1966].

8.3.5 A related problem: Infinite! prin ed to es

All the machines up to this poin (an all hat ' ill fc llow) are on- 85 CC)NSEC UENC ES FC R ALC 70RIT liMS ND

sidered to start in a given initial state with only a fini e ins ripti Jn on the CC)MPU ER P OGR MS: HE D EBUG :71NG

tape. For a moment, conside1 mac hines whic h are give 11 inti nitely m- P~ OBLE ~

scribed tapes. Does there exist a Tur ing rr achir e D' hich can< ecide the following question: "Given aT uring mad ine 1 , is th 11 inte rna] tate Th uns Jlvab lity ( f the halt ng p oble n can be similarly demon-ere a Q and some infinitely inscribed t ape f< r wh· ch T will n ot ha t wh n sta rted s rated for nyc mpu atior syst m (r ther than just Turing machines)

on that tape in state Q?" We h t dis us sec and will ot fu rther dis- \\ hich an s uitab y rna nipul te d ta ar d int erpret them as instructions. ve n cuss machines with non-finite in tial t< pein Orffi< tion, or un speci ed in itial I part icula , it is impo sible to de vise< unif orm procedure, or computer

states, but the reader may be inter sted in tl IS ea sily s a ted but ery p ogra m, wl ich c n Joe kat~ ny cc mpu er pn gram and decide whether or

difficult problem.3 not th t pr grarr will ever term in ate. Th's means that computation sc ienti ts ca 11not aspir to volvt a c mple tely foolproof "debugging"

PROBLEM 8.3-3. If the ini ial st te is specif ed, n e pro plem f wh ther p ogra fn. ( his c bserv ation hold on I) for rograms in computers with there is a non-halting, infinite ly ins< ribed ape i unso vable. Shov this. e senti lly u hlimi ed se onda ry stc rage, since otherwise the computer is a

fihite-s ate n achi• e an ~ the the haiti g pr pblem is in fact solvable, at least in prin iple.) Thi mea ns ah o.tha t we annot aspire to obtain a set

8.4 THE CREATIVE CHARACTER OF Tl E 0 rule s for decid ing v hen ny a leged "alg prithm" is foolproof in the UNSOLVABILITY ARGUMENl sc nse t at it f,vill tt rmin te fo all i itial ituat ·ons. Of course, we can find

r les w hich lvork orla ge cl sses fimi orta1 t problems. Let me point out another ntere sting featu e co rnmo to ~II o the The sew o ob ·ect t at C hurch's (or Turi ng's) thesis (see section 5.2)

problems above, though I desc ibe i only fort he ha ting probl m. 1--on- a lows too n uch, usual ly do so 01 the roun ds that the Turing machine sider the construction of E * fr bm E in 8 .. N bw it is pe feet! pos ible fc rmul ~tion of co mput ~bilit allo fvS CO mput tions whose lengths cannot that some machine £, while no able to d cide very quest ion a pout halt- b bou hded n ad' ance in an reas pnabl e wa'. The impossibility of com-ing, can decide some such ques ions. Tha is, w can upp< seth t wh n£ puting boun ~s (rr entio ned i sec ion 8 .I) tt at follows from the unsolv-says "Yes" then the ( T, d T) in ques ion v ill h It, a1 d th t wh ~n E says ability pfth halt ng p obler his o he of the o bstacles that seems to stand "No" then the ( T, dT) in questi bn wi I nev r hal , but that or so Pile in puts i1 the lvay o f find ·ng a form ulatio n of omp ~tability which is weaker yet E itself never halts and never ~ nnou nces dec sion. The re ce tainl do nbt COl pplet ly tri ial. exist many such machines (Pn blem :Cor struc one ) wh ch tt us re ~lize what we may call "partial dec lSI On proc dure " in whic all nnou 'rzced decisions are correct but the m;: chine will ot al ~ays omn it its If. ( iven 86 NPN-UI SOL \I ABILI Y OF INDI\ IOU A a machine E for such a partial 1 rocel ure, J he co rzstrw tion c if8.2 gives us a H LTINC PRO BLEM' particular problem which E cann bt sol e, na hlely he h· lting prob em o f £* for the taped E*. Furthermore, this e amp e is a tuall y exh bited -the re is We have show 11 the impo sibili y of an ef ective procedure to solve all no "non-constructive" step h re (< s tht re is in the C an tor diag onal Turing mad ine h altin~ prot !ems. But note that in no case did we show argument). The inadequacy of any iven alleg d dec ision proc edure can tt at th halt ng pr oblen for ny p rticu ar sit uation ( T 0 , t 0 ) is unsolvable, thus be demonstrated construct 'vely. i .. , th t the re is no ef ectiv pro edur e wh · ch will tell whether or not

Thus the problem of whid situ tions (T, 1 T) It ad tc halt ·ng is not ( r o, t 0 will ever l alt. ndee d, no such resul could be proven. For con-only effectively unsolvable, but is ass Jciate d wit 1 an e ffecti epro edur for sider tl e twc proc edur s, ""L ook t ( T ' t 0) and say 'Yes'," and "Look constructing counterexamples fc r anv prO I osed solut ion. A not her p rob- a ( T 0 , to) a 11d sa 'No '." 1 hese two rivial procedures may seem silly,

154 UNSOLVABLE PROBLEMS SE , 8.7

but each is certainly effective. And since ( T 0 , 0 ) de es 1n fact e ither halt or never halt, one of them must be corn ct! ([)f co rse t~e p10blen remains of finding which is the c orrec one, but t is ifl1por ant 1 o ob erve that this is not the question tha cone erns s her~.

Let us take a slightly di~ rent view of tfle m~tter. Fo a ! 1ven ( T 0 , t 0), it could well be that n one will ver f nd o t wh ther it hal s or not. It could conceivably be hat here ·s no way o fir d ou , in orne obscure sense. But it could no be t at sc meor e cotld pr ve that there IS

no way to find out. For that would l!::ad t< the lpllowing p rado Suppose that it had alleged y been pro~en t at there IS no v ay to find

out if (To, to) halts or not. Sur pose !so t at a great expe imen a! prpject were launched, involving the cc nstruction and e per a ion c f ( T 0 t 0). Sufficient funds are invested to p1 ovid{ for slo' , bu sure y ne er-er ding supply of tape. Now there 2re tvo ca es in facte-eith~r (7 ,t0 ) does eventually halt, or it never hal s. I the first ase, he e perir~enta approach will ultimately succee< , anc the ques ion v ill b sett ed. This would certainly contradict an proof th: t tht que: tion coulc neve r be settled! Hence it must happer that (T0 , t ) ne• er sV ps. n ot er wprds, the proof that there is no wa~ to find out Cfln bt use< to rove that (T0 , t 0 ) never halts. Hence, we wouh be a le to find cut. ( ontn dictic n!

We will return to this discu sion t the end < f Ch< pter

8.7 REDUCIBILITY OF ONE KIN I OF I NSOI VABL PROBLEM TO ANOTHER

In section 8.3.3 we showed the impos ibilitlv of c onstr~ctini a T ring machine which can decide wh ch T ring macl ines palt ' hen ~tarted on blank tapes. This problem, of ~ecid'ng w ich s tuati1 ns 0, bla1 k) le d to a halt, seems simpler than the orig·nal problem 8.3.1 of decic ing 'hich situations ( T, t) lead to a halt. But l!::t us ecall the s eps < f the proo . In 8.2 we showed that there is np rna hine whicfl can deci e all the ( T, t) situations, by showing that, if there were such a rna hine D, tt ere v ould also be a variant of it E * which must bo h hat and not flalt i[l the situation(£*, dp)-when given ts ov.n descriptipn. 'iVe then showed that for each pair ( T, t) there correspbnds an e sily const ucte rna hine M T,t which halts on a blank ta1 e if a)1d only if ( T, t) i self I alts.

Therefore the ability to decide the spe ial h lting problems < f the form ( T, blank), which include all t~e situatiops (k T, ,, I lank , wo1 ld gilve us the ability to decide all of the ( T, t) pal tin~ problem . We can ~ay tl at in a very simple sense the appare tly n ore c ifficu t pro~lem of th ( T, )'s is reducible to the apparently simpler I roblem of he ( 1, blar k)'s.

This notion of reducibility as b en st~died very caref~lly i the mod-

'·

\

SE . 8.8 (uNSOLVABLE PROBLEMS 155

er thee ry of comr utabi ity. t tun s out that several technically different nc tions of re ucib 'lity a e usc ful ir studlving he relations between differen kin s of unsolvability p oble Ins ar d, further, that one can define in nite ierarchies of more a1 d mere di ificult unsolvable problems, none of whic[l is *duciple tc its I redec essor . It has even been shown that th rear pairrs of u[lsolv ble r roblefr1s ntither pf which is reducible to the ottJer. J oger [ l9t 6] dis~usse thes mat ers ir detaiL

8.1 PR4I>BLEN S

PI OBL M 8.8-1. < onsider the class of all 2-symbol (0 = blank, I) mach nes 7. Ar~ the e decision proce~ures for the following questions? Th~t is, is thne an machine D, no necessarily 2-symbol, that always hats wit Yes r No, given T, t a d ans~ers t e question.

(I) [ oes T ever r; rint a I whe start~d on ape t? (2) [ oes T ever e ase a I whe start don ape t? Hint: Appl fin it -state machine th ory to problem (I). Problem (2) is

m1 ch, m~ch h: rder: f you solve it yot will 1 ot need to read the rest of the bopk! PI OBL M 8.8-2. I then a dec sion 1 roced(ure that will decide:

(I) Foran' T,is here: tape nwhchT1 illhalt? (2) For an T, wi I Tend witt a bla k tap for some input tape? (3) For an two Jpachil es T and T' do tl ey act the same, i.e., for every

taJ e t is he ult mate t>utcmpe the same'. (4) For an T, dces Ts arted pn t u e mme than N squares of tape?

NcJ>TES

1. The : rgum nt in more ~etail is: if D exi ts ther E exists, because it is easy to add a copyi[Ig me hanism like that< f Fig. 6.1-6. Similarly£* exists if E does, becat se we can a~d th two loop tates. Nov let us put dE* on E * 's tape. There are t' o po~ sibilit es:

(I £* 1 ventu1 lly hc.lts. lhat is to sa\, E* alts given dp. Then E takes its "'J es" e it, giv~n dE . But this r eans that v hen E finishes its computation on d 1•, it e entually en ers its uppe term nal s*te. Since E* is the same as E up to this point, hen E *, giv n dE even ually eaches its upper exit state and enter the ever- altinJ loop The ref on £* 'oes not halt, given dE*. This leave only he se ond pbssibi ity:

(2 E* ever alts, l!iven ~ E*. But b a sijnilar argument then £, given d £*, akes ts lover exi ; and so £* (which is i en tical to E up to this point) takes its lm er exi and oes h it.

Since there ar no 1 ther ossib lities, the e istence of E* itself is contradicte , hen e that of E, ence he ex stenc< of D.

156 UNSOLVABLE PROBLEMS SEC 8.8

2. The "Russell paradox" is simi Ia , whe e the xister ce of *is a nalog us, tc

C = The set of all sets , for lvhich ~does not b long i nc

That is, c is a member of C if ar don! if c ·s not p men ber o itsel . For ex-am pie, the set of all sets is a set, so it ·s not men ber o C. I ut th set o all books is not a book, so it is a me nber c fC. We ask, "Does C belong inC?" IfC i inC, then i does not b long n C. And

if Cis not inC, then it does bel or gin (! A It hough this r nay s<: em at first t o be a sort of frivolous pun, it turns o ut tot ea m st ser ·ous i dictrr ent o ordir ary, c ~ 1 HE co VIPl TAl ILE common-sense, ways of thinking abou sets a nd rei tions and n hal a cen tury of work on the foundations of lo :;ic, nc techr ically easy "'ay h s bee fOU[ d to .. F EAI . N' JME ER!,t handle the problem. There are some appa ently satisf ctory but echni ally quite difficult, ways to keep the r arado x out fmat hem a ical re asoni g.

3. The problem was posed by Buch i [196 ] and hown to be unsol able y Ho oper in his Ph.D. dissertation [ 1964].

9. RE If lEW OF Tl- E RE~ L NU ~BER SYSTI M

Mo t rea ers a e pre babl not am iii ar wi h the details of the modern

theory )f the real numb r>r sys tern. This sec tic n reviews some of the basic

de finiti ns. We a SUfi( as g iven he in Ieger 0, 1, 2, .... The rational mmber are efine d as uotie nts o inte 5ers, )r more basically as equiva-

le11ce c asses of o derec pair s (m, n); t\1 o pa rs (m.,n 1) and (m2,n2) are

in the s mec ass if and nly i mrn = 11i 2nr. The e are two hief vays )f def ning real r umbers, given the rationals.

One is hem thod due oDe dekin d; a eal n umber is defined by a "cut"

0 all t e rat ·onal into two Ia sse , sue 1 tha each member of one class

is less t han e very nemb er of the o her. Thu the real number 1r, whose

decima expa nsion begi s wit 11 3.1~ 1592 5 ... is "defined" by two classes,

o e of vhich cont ins ( mon othe rs)

and 3.1 md .14 a nd 3 141 nd 3.1415 and

.141 9 an 3.14 1592

a d the othe cont ains ( am on goth rs)

1 and 3.2 and .15 < nd 3 142 md 3.1416 and ?.141 0 an i 3.14 1593 etc.

E~ch o the 1 umb rs m thee amp! is a ratio ~a! number: 3.142 is equiva-

leflt to 142/ 1000 and !sot( 1571 /500.

tThis chaptt r is op ional. It assu mes th t the eader as been exposed to the elementary th ory o there· I num er sys em up to Ca tor's roof t at the reals are non-denumerable, et . Thi backg round s not< ssume in sul t chap I seque ers.

157

158 THE COMPUTABLE REAL NUME ERS SEC. .2 s c. 9 .. I TH E COl\ PUTABLE REAL NUMBERS 159

The same example shO\ s ho w a r eal n urn be can be d efine< by ~n p ROBI EM .2-2. Wee< uld al o deli ne a c pmputable number to be one for increasing or decreasing co rverg ~nt ir finite sequ nee pf ra ion a! s. A rd \\ hich t ere is a Tur ng m chine which , give non its initial tape, terminates it can be shown that, if he n als 2 re st itabl defi ned s eq uival nt \\ ith th nth 1 igit 0 that numb ~r. S~ow t ~at this definition is equivalent

t< that f9.2. classes of convergent seque ces c f rati pnals we~ et th1 sam stru ture as that defined by the cut met pod. In p< rticu ar on ~ mi~ ht de pne t e "r a! decimal numbers" (or the "real bin a ym mber ") il ten s of in fin ite 93 Jl E EXISTEN E 0 NOt -COt¥ PUTA aLE sequences of fractions with en on 1inat< rs th t gro wby powe s of 0 (or 2) R AL N UMBERS while the numerators grow ned' git at a tim e; thi~ may be sh ownt o defi ne the same structure of the re, I nun bers. (On has o pre vide or th equ v- We can how the e xister ceo non COlli utable real numbers in two alence, e.g., of 1.0000 ... w th 0. 999 . . . , b t oth erwis e the e is 110 di ffi- r ther differ nt w ays. Our rst c em or strati on is through appeal to the culty.) The definition by se< uenc sis t erne thod ofCa uchy. e isten e of unsol vable haltir g pre blem for uring machines.

9.2 THE (TURING)-COMPUT BLE I EAL t UMB RS 9 3.1 A par icula non-< ompu able eal n1 mber Ru

J Co side1 the I ehav or of our ~Jnive sal r achine U on some tape in-We define the computabl e real numl ers, i am ~Inner para lei tc that of s ribec with a fini e sec uenc< We pmpo sed o f the ten symbols 0, l, X, Y,

defining the (Cauchy) real umb seqt of d gits i nterp eted as I A, B, ~ f,N, R, an j S. Ne kr ow t at we ot decide, for all such tapes, rs, a ence cann

decimal fractions.t But we add one ~ ey re strict on. The t igits must be "' hethe rU"' ill evt ntua ly ha t,t ( his~ )II ow from section 8.3.3, because generated sequentially by a Turh g rna chine. Th t is, wen quire that in t is se of ta pes i clud s the class of al desc riptions of Turing machines order that the real number .aoa, 2 ••• t>e a ( ompz. table real umb1 r, th re s artin win blan tap< s.) must be a Turing machine \1 hich tarts with a bla k ta e an prin ts ou a No w let s thi k of these tape as n prese nting numbers. We identify tape of the form tl e syn bois p, l, , Y, ~. B, M, 1\ , R,' witt the symbols 0, l, 2, 3, 4, 5,

... 00 OXao ~a,x 2X. .000 ... 6 7, 8, 9. 7 hen t eac, tapt ther con espon ds a decimal number. Thus

We make the rule that, onct an X is pr 'nted, the rr achir emu t nev ~r me ve tl e tap

to the left of, or change, th ~t X. ltm ay m e an~ arne unt c f tap to he 000( MOO lJYOl rooo IXO I lOA 1001 XlliOOYOOO ... right for its computation, t ut th prin ting fan X is an ir evoc ble: n-

b com s the (deci nal) 1 umb r nouncement that a digit has been COlli uted and i in th e squ re tc the I ft of the x.t .000 0120 0012 )1110 2100 12111003

How is this different frc m th defi ition of a real umb er? t is d if-where ,ve m ke tt e con venti :m th t the first M will be a decimal point. ferent because in the defini ion o fa cc mput 1b/e r eal m mbe the de fin ing Then, also, ch dt cima ber t -viii correspond such a tape. to ea num ere sequence must be determin d by a fin te arr ount of in orm2 tion, i.e., he Hence we c2 n no rv talk oft e tar e co respc nding to the number n and state diagram of the associa edT ~ring mad in e. Ther is n sucl limi a-

tion for the ordinary real n ~mbe. T at tt is is ious c iff ere ~ce v ill v ce ve sa. 11 ser

We de fir putal le n mber Ru in terms of the be seen shortly. now e ou n01 -com (cpunt< bly i fin itt ) set pf Tu ring-• rach ire ta pes. Ru will begin with the

PROBLEM 9.2-1. ShO\ that any r< tiona numl er ca be s p defi ed b a d1 cima poin t and go in ~tot e rig t, its nth d igit is Turing machine which n1 ver p1 ints a ythin exce pt the ~igits nd th ~X's.

l if u a Its n the nth t ape; twe will treat only numbers be ween ( and I. Thi s simp! ifies no ation ~ecaus it all WS 0 if u ever a its n the nth tape. us to assume the decimal point is alway at the extren e left and de es not other ise af ect

the theory significantly. To how hatt he re2 I nun ber s o defi ned i~ non-computable, we derive twe need some such rule. Wi hout s ch a 1 rovis o one c uld m ver be sure t at a digit a L:Ontr dicti on fn m th sup posit ion th at the re is a Turing machine TR u is not going to be changed at son e late time. If th S UnCf rtainty alwa' s rem ined, ne

twe could hardly accept the process as n effe tive df finitio ofthf numb r, for pne w uld m ver ssum e some onven ion a bfJUt w here U starts o the tape, e.g., that it starts to the really know for sure anything abot t its v2 lue. ri ht of t~e seq ence. he d e ails a r•n't im portan .

' I

160 THE COMPUTABLE REAL 1\ UMBE S SEC. 9. EC. 9. Tl- E em PUT ABLE REAL NUMBERS 161

which can compute it. If the ewe e a 1 Rum achin , the n we rould buii< DE FIN IT ON

from it a somewhat more c om pi cated mac hine Mw hich can decid w say hat 2 n inf nite eque ce c , c2,. .. , en, ... of real numbers is whether U halts on an arbitr ry ta pe. ~ mce here can b e no mach ne A: rPpres nted by a urin g rna< hine T if, iven a representation of any inte-which can do this for all tap s (se sect on 8. ?), ar d sin eM is bu It in a

~ern, Twill print outt he di~ its of then urn be en, in the manner described straightforward way from T R V' it f ::>llow that TRu' ould note ist. i11 sect on 9.2.

If here were such a re~ resen ing 1 rae hi ne T for all the computable

PROBLEM 9.3-l. Sketd f\1 can be bu It fro in TRt . Ca rying out ne r a! n ~mbe s th n, b a c pnstr ctior sim' Jar to that of M from TRu how i~ 9.3. I, we coul< mak earr achir e T' ~hich , given m and n, would give full construction involves ompli ation s, but ·s quit instr uctive omec t s the mth ~igit nm of the li th co p!put ble r umber. In particular, then, \ e COl ldm ke a mach neT' 'whic h, gi en m would give us the mth digit

9.3.2 The set of computable-n al nu rbers is lmm of them th co ruput ble r urn be r. N ~xt w ~ could modify that machine countably infinite tp mal earr achir e T'" that ~oui< prin out he sequence

In the argument of 9.3 .I abo e, w obs rved that he t pes or tt e .000 \¥c""' en X .. Xl mmX .. · universal machine U could t e set into one to-or e cor respo ndenc e wi h

~avin (some of) the integers. Now he 01 mbe1 of su ch ta es is ount '(lbly i f"ifinit , 2 s a c bmpu table real pumb er. g go so far, we make one final

hence there is only a countabl infir ite se ofc mput nb/e- eat n mber s. B t change in c ur rr achin e: n e m2 chine T"" is like T"' except that for

it is well known that there 2 re an uncc untat le nu mber of re a! nu mber. each rr 'if c nm is I', th e ne\1 mac hine T"" prints '2' for it; if Cmn is not

Hence there must be some n on-ce able real r umb rs, a 11d in deed it ' ',the n T" 'prir ts 'l' fori . Nc w co sider the array of the digits of all mpu follows that "almost all" rea bers r mst t e nor -com putat le! the co nput ble n urn be s c1, 2' .. Cn, ... num

The argument of section 9.3.1 whil ~ slig tly n ore c om pi x, se ems 0 c1 = ~ ~ cl2 en C14 ...

give slightly more of a result. In a way i actu ally t ells u abo ut a ~ artie -

9 Jar non-computable number Un ortur ately but nevit ably, we c2 n't g t c2 ( 21 ( C23 c24 ... very close to that number, becau se th ver pro of th t it is no n-eon -

CJ = ( 31 C32 @ C34 ... putable tells us that we can' get~ Turi ng m achin , or ny e quiva ent, 0

8 give us its sequence of digits. Yet one< ould hard! y ask for a mor "co 1- C4 = ( 41 C42 C43 ... structive" proof of the exis ence of st ch a int< ngibl e obj ct a 11d st ll subscribe to anything resemt ling urin 's the sis.t

v here ~e h: ve ci cled he " iago a!" < igits Cmm· Note that the machine ] "" p ints' sequ nee ~hose mth ~igit i s new r equal to the corresponding

9.4 THE COM PUT ABLE NUME ERS, ~HILE Cfnm· he f2 ct th t eac ~ dia~ on a! entry has 1 een changed means that the COM PUT ABLE, CANNOT ~E EF ECTI\ ELY sequer ce ob taine bY! oing a! on~ the ew d iagonal must be different (in ENUMERATED! at leas one ~igit) from each nd e ery r pw se ~uence. Since we have given

an effe ~tive proce dure or w iting its di ~its, his sequence defines a com-An interesting property c f the CO ill' utab e rea nurr bers s tha , wh le ~ utabl nun ber. But his rr eans hat c ur fi pal Turing machine prints a

like the rational numbers tl ey fc rm a cour table or' enun erab le," s t, cpmpt table numl erth t did note ccur n the allegedly complete sequence they are unlike the rationals in tt at th ey ca nnot be "e ffecti ely e numc r- c f con puta ble 01 mbe1 s "re preseJ ted" by T This contradiction implies a ted." That is, they cannot be a! arn nged in a sequc nee ' hich can e t at n Tur ng m achin e can reprc sent he er tire set of computable num-"represented" by a Turing rr achir e. bers, h ence his se t can ot be effec ively enurr era ted.

Stu dents fami iar w th c, ntor' thea ry of infinite sets should note that, t That is, the argument really d ::les "d fine" ne an only one re l num ber. F or eit er a! thou h we hav emr: Joyed Can or's 'diag onal method," we have not

U halts, or it does not halt, on tt e Nth tape. This omple tely de term in es all he di~ its shown that here re an unco untab e nur 1ber f computable numbers! All of our non-computable number R M· ny rna them a icians and p iloso~ hers ~ el, no e-\I e ha\ e sho wn is that the c able numt ers cannot be "counted" by theless. that there is a serious qu estion as to wheth r one ough to b lieve hat R u's mpu

"existence" is really established. any Tu ring" vtachil e (or other effec ive p oces5 ). Their number is still only

162 THE COMPUTABLE REAL JSUMBE S SEC. 9.5.

that of the integers; they could be< ount d by an omniscient 'orac e" o an infinite computer. Cantor's argt ment for t1e un ountt bility of the rea numbers depends on applying tl e "d"agonalizafon method" t an enumeration whatever, without the restri tion to "effecti e" o "comput able" enumerations.

9.5 DESCRIPTIONS AND COMI UTABLE NlJMBERSt

Imagine for the moment hat n an has ev lved a per ectly unan bigu ous written language througt which he an comm nica e all his in uitiv notions. The expressions of this langua1 e are finit~ sequences of symbol from some (fixed) alphabet. A de criptwn of a real nun ber i an expres sion in this language which in son e clearly nder tood way determines, selects, generates, or allows us to reco nize that 1umb r and onlv tha number. We call describable ny n limbe whi< h has a description.

Warning: This intuitive ~otio is ! o va~ue t~at c ne n ay \1 onde whether it could have any useful p1ecisel~ defif!ed c unte part. At I resent this seems very unlikely, sine< no ope ha beer able to po"nt toward a wa of excluding logical paradoxes, wh le pn serv1 gall pr me st of the d~scriptive freedom of "natural" Ia guage. Lt t us c xplo e thi notipn, t ougt, as if it were well defined, and see w~at h ppens.

The set of describable re< I nun bers is su ely a countably infin te se , for any description is a finite tring of syf11bol . He~ce a! desc iptions caf"i be enumerated, e.g., by list in first the fi ~ite set oft xpressions of le gth , then those of length 2, etc. bf co~rse 1 ot a! strings wi I be descr ption s of numbers but this doesn't n atter. It f< llow! that some real umb rs a1e worse than non-computable som (in< eed, almo t all) are on-descril -able-there aren't enough c escri1 tions to gb aro~nd! The com1 utab e numbers are certainly descr babh; hen~e (a arg1 ed i1 9.3.1)), tl ey a e countable. Now let us go on with hiss ecuh tive tP.eory.

9.5.1 Some descriptions of nu[nbers do no describe computable numbers

Consider the non-comp table nun ber (R u descril ed i1 9.3.1. In English, it is described by:

The nth digit of R u f 1 if U t alts c n the nth t pe. l 0 if U ever halts on the nth ape.

tThis section is independent of what CJ!Iows and is proba ly not entire y sour d! It is interesting to speculate on what h2ppens if wet y to uild up an analysis for d scripti n along the lines of our analysis of effecti e con putatil n. Tt e subj ct of descri~ tion has been treated in many other ways?

SEC. 9 5.2 THE COMPUTABLE REAL NUMBERS 163

Henc~ R u s des ribable evt n the ugh ve ca 1't find all its values by an efective pro edur . Remark· We would expect no particular difficulty in

descr bing, in su Jer-E 11glish, what is a .... uring machine.

PROE LEM Let "be a r1y Tu ing n achine, and let R T be the describable r1umb r defi ed by

The nth d git of R T = {

01 f Tha Its on the nth tape.

f T never halts on the nth tape.

Prove that no effe tive {: rocedllre can decide, for every machine T, whether R r is c ompu able.

It folio Ns from thi prollem that no effe tive procedure can determine whicl desc ibab e nunbers are ce mpu able and which are not. In fact, in general it s not poss ble to dec"de which expressions describe numbers at all.

Q~ite f eque tly, < f cot rse, \ e ca1 sho ~ that some particular description sbecifi sac mpu able ~umber. Fpr ex; mple, the description of e as

""

~~ makes it p< ssiblc to sl ow tl at (tl e sequenc~ of digits of) e is computable.

PRO I LEM Sho ~ that e is c mputable! This requires an analysis to find but w en th repn sen tin~ mac~ine c n pri tan X.

~.5.2 Som~ descriptio s of cpmpu able r umbe s arer 't effe tive

F eque tly "e can desc1ibe a omputabl number in a manner which is omp etely ineffective in its If-\ hich does1 't give the slightest hint as to

l!ow o cor[lpute that numl er. Consi~er t e following non-effective decripfon o am mber N: 1 = l if th re are an infinite number of 5's in he de cima expansion of 1r, and Jlr = ( if not.

Nbw N is eit er 0 or it s !-there is no other possibility, and it must ureh be oply o e of those. In c ither case "t is a computable number, for is a1 d 0 ij. But neither tht authpr nor the reader knows which one it is,

nor does a yont else! I'm sure hat t e va t majority of mathematicians woulc bet, for v nou reasons, that N = l; but that isn't much help. It eems perft ctly I ossible to me that no one will ever find out the answer.

Perhaps tin e wil not tell. Tl is raises some intense y punting questions. Up to now each of our

'unsc lvabl pro !ems ' has been oncerned with some infinite set of deciions. We have never sho\\ n that the haltir g problem for any particular

164 THE COM PUT ABLE REAL l' UMBE S SEC. 9.5.

blank-tape Turing machine \ as u 1solv ble, but only tl at w cou d no hope to find any "uniform" roce ure o sol e all such prob ems. Som mathematicians accept these unif< rm unsolv bilit rest Its b t pn fer t believe that there can be no nalo ue f< r sin le prbblen s. Tjley s em tb feel that even if present metl ods f~il to solv~ a pro bien , we will fl!Wa) s be adding to our mathemafcal n sour< es ar d, througl sue adc ition , can eventually solve all parti ular probl ms. (It is clear that uch corrplete evolution of our methods cannot occur in accord with ny ef!ectile process, so such faith must depenc on some nexplaine< optimisti beli1 f

in invention.) Returning to the problem of the alue of A, it s cle r th~t th s

problem cannot be solved by brute comr utati tm of he d gits c f 1r t eno -mous precision. For at any time i1 the comrcutati< non will have only a finite number of 5's, and this pr< vide no tviderce ei her" ay. If ore finds no more 5's for a very long ime )ne rr ay bt come discouraged, y t one may be just about to stri e oil whil if ore get many 5's one can st ll draw no conclusion-the w II m y be abot t to un d y! o se tie tl e question will require some fbrm < f rna hem~ tical 'vroof+-a rr ore 'bstra t kind of prospecting.

A little earlier we mentioned that it set ms r ossib e to us tl at this question will never be solv< d. I seens also po sible, and this s qui e different, that it cannot be S( ttled. The first gloon y su! gesti m is hat " e will never happen to find a roof. The seco d is hat perhai s the e is r o proof-within the framewo k of any 'piau ible" exte sion of \\hat 1 e recognize today as valid mathema ical easm ing. A th"rd, e en more o~scure possibility, is that ore might prove that the ~uest on c~n't pe settled. To explore careful y what su h a esult coul me~ n, ar d what form it could take, would carry 1 s well bey nd \\hat can be disct ssed in this informal framework, and no me stems o un erstand it territly we II,

anyway. We will discuss instead spme 1 robl(ms wtwse olvat ility tatus is cit ar

yet sometimes misunderstoc d: (l) Fermat's Last Theor m ca not le pro ed un a/val /e. This is like the

situation in section 8.6. The question is: "f re there a yin egers x, y, z, and w, for which

with w larger than 2?" Thi que tion [night of cpurse nevt r be olve< -i.e., no one might ever find such a set of x,y,z, nd J.t. Bu it c<nnot be prov~d uns~lvable. ~or i ther wer sue a I roof, this woul ~ be inconsistent with the existenc of a x,y, z, w wluti< n; hence s ch a solut on could not exist; hence the ques ion \1 ould be a swer d "No," and this would solve the problem.

SEC. 9.5.2 lHE CC MPUTABLE REAL NUMBERS 165

(~) Th :rCn) June ion n ust be con put at /e, even though it may be imposst le to find out ho v to c mpute it. )efin the function:

1r(n) = {I i_f tl e dec mal < xpan ion c f 1r h s n 5's in a sequence. 0 If nbt.

Now we o servt that

(i) i (ii) i

7r( ) =

7r( ) =

th n 11 (n - I) = l th n 11 (n + I) = 0

(n > 0)

So the vah es of 1r(n) ave tither he fc rm

n = l, , 3, 4, ... , k, k l, k + 2, ...

1r(n) = I, , l, I, ... , l, D, 0,

for s me A, or t lse 1r n) = l for all n No one knows. Perhaps we will neve disc< vera way o fin out .vhat is, i it has a finite value. But 1r(n) is co nput ble ir any ase, or (i if th re is o k, then

(n) ~ I

whic cert inly s con puta t>le, a d (ii) if there is a k, then

{

I if n _.::; k 7r(n) =

0 if n > k

whic fun< tion ertai ly is ~lso cpmpt table The point is for the reader to note the <urious dis onnection bet\\een I nowing a function is "computa le" and knowin ho\'o to ccmpu* it!

( ) It i not 1 nowli whet~er 1r'(n) is computable. Define

7r'(n) = { ~ if t 1e number n occ rs w thin the decimal expansion of 1r.

ott erwise.

Then, for< xamJ le,

1r'(l~ I) = l, 11'(4159) = I, 7r'(26535) = I, etc.

~e know hese oecause we see tl emir kno ,vn parts of the decimal expanSIOn of 1r. But we c on't now that 1r'(n) is computable. To find out, I SUI pose, would req~ire c: maj< r ad1 a nee in the mathematical theory of transcend< ntal umb rs rather han n ac vance in the theory of computa le functio s. _C~rtair ly the kine of a gument, in the previous paragrap , abcut 1r(>1) sirrply doesn' wod here Most mathematicians would sure! suppose that 1r'(n) = I or a I n (and hence, incidentally, that 1r(n) = I for all n), bt t no ne k ows.

166 THE COMPUTABLE REAL NU MBER5 SEC. 9.5.3 s c. 9.E TH E COIV PUTABLE REAL NUMBERS 167

9.5.3 Mathematics and metama hem a tics ' the set of in egers who5 e (sm all est desc iptio ns are longer than ten words?

The conclusion of Section ~ I ut si1 ce th e que ted e pres ion b as its If ju t ten words we have a para-

.6 is 1 rob a Jiy so und. We c an no exp~ ct to cox. The roub e co !nes ' hen temp t carefully to formalize the we a find consistent informal argurr ents pro vi g th t the type of p rticu ar-

r otion of de scrip ion. Ther is n p tr01 ble ~ ith a straightforward repre-one-answer-problems consid ered there are ~nsol able. Bt t the re is

s~ntat· on of num pers, e.g., s str f dig its in the usual way. Then, to st ow tt is. \l Ire be ngs e

something peculiar about the a rgum nt w usee orne e.g., J( 0 is t re sm all est integ ~sent ble by less than three digits. r no repr aware of this peculiarity in t\ o irr port2 nt w ys. First, in c ther !lreas I ut we get nto t oubl ~ wh n the exp essio s are, as above, interpreted there are arguments of very m ch tl e san e ch racte r whi f::h se m ec ually c n the meta [nathe mati aile' el to alk a pout hem selves. Since we cannot reasonable, yet give us absurd esult ; this mak s us us pie ious pf all such

e~pect to be able oem peds ~ch s a tern nts i II a consistent logical system, arguments. This is why I sa id th t the arg men in < uesti n is only \ e ne~ d not be su rprist d at' para oxes 'like this: one expects them when probably sound.

ace tb t1e log ic its~ If is d efecf ve.

We see the peculiarity mon clea lywh en w~ try t rep! e infc rmal argument by a "mathematical y rig< rous' 'derr onstr ation e.g., one ike a proof of Euclid in which eacl step of tl e ar! umer t is ' 'auth Jrizec , by

9.6 P OBLE MS AI OUT COMF UTAB E NU MBER referring to some previously agre~ d-on "axi Jm." (A< tuall , Eu lid's proofs must be further justifiec by n feren es to "rult s of i oJere1 ce" ' hich

PROBJ EM .6-1. Prove that, if Xi a cc mputable number, then so are specify how new theorems rna v be ener ted l y usi ng ol d the or em and x 12, x 2 sin x axioms.) When we do this, ~e be orne awa1 e tha our "pro of" efers explicitly to the notion of pro< f itse f. B ~tin ~hat !lxion syst ~m? s the PROBJ EM .6.2. Defi e the comp 'rJtable complex numbers and sketch a system the proof talks about tl e san e sys em t at th !? pro 1 is onst1 ~cted p ooft at th y fon ran lgebr icall) close d field-that, if a0 , ••• , an are in? If not, the argument lose~ muc p of i s for e, fo eve1 if a probl ~m is c mpu able-c pmplt X, SO~ re the roots pfthe equation

unsolvable (that is, if a staterr ent i! neitl er pr ovab e nor disp ovab e) in ao + alz ... anz = 0

one system, there would be no hing urpr sing bout findi g thl t it c lin be settled in a larger system-e.~ ., one whi< h co tains that state nent san l-int: 5 how t at an stan ard c mpu ation method, e.g., Newton's method,

y elds a n effe tive rocec ure fc r fine ing t e decimal digits of the roots. additional axiom. I nore he di ficulti s tha arise forth ose ro ots that are rational, since they On the other hand, if the proo syst ms a re th sam e, so that state-a e corr putah le any way. ments can, in effect, talk abou t ther nselv s or bout their own proo s, we

must be prepared to find gra e difi culti s. 1- istor cally mos sue sys- PROBJ EM .6-3. De fin a ret !-com DUtab/ e function in the following way: terns of "metamathematics" -axic m sy stem5 con erne< wit the Jrem The fun ction (x) is real-c ompu able i there exists a Turing machine which, proving itself-have turned out to b e inc onsis ent, prod Jcing self g ven x = .a aza 3 • .. , tc gethe with an a bitrary integer n, in the form contradictions. Worse, it wa sho 'Ill b) Goc el th t if, withi 1 any such o an in finite initial ape system, one can prove that this will I ot h2 ppen r-i.e. the ~ ysten a sse ts it5 ... 0( 0 ... ( ynxc lXaz a3 X .. . XajX . .. own consistency-then surely it wi I in act b inc nsist ~nt. Henc ~om

ewiH \\ill al\\ ays te mina a tap ofth form cannot believe the most persw SIVe rgurr ent c fa m ather atic2 I syst min its own defense! Godel's me thods are e senti lly s imilar to th ose \1 e us~ I ... 00 D ... l f(x;n)" a 1Xa zX ... in chapter 8, but somewhat com licat1 d by the fiddit on a! macl iner) \\here n is bloc of~ Y's nd f (x; n) s the nth digit of f(x). (If this needed to talk about theorem• and proof . s ems t be tc o spe1 ialize a de pnitio , one might just say that f(x) is real-

This subject is quite com Jlicat d an d tee nica , and the eade whc c mpu table i there is an effec ive p ocedu re which finds (for each n) the wishes to pursue it is referred oRo gers [ 966]. n h dig it off (x), gi'-1 en the ex par sion c fx on a semi-infinite tape.)

The following well-known "par adox' ill us rates this ind )f difl cult' Pro e that any r al-co nputa le fur. ction ·s continuous in the ordinary f, o with descriptions very nicely. Co sider "the smal est i1 teger who e de s nse! -lint: ~ how t hatth discc ntinu us fu ction scription requires more than t£ d , Doe s this jescr iption de fin eam t 00 .. i X<! n WO s. = .333 ... integer? It ought to, since w kno w th2 t eve y set of ( ositi e) in eger (x) =

00 .. i X ~ I .333 ... must have a smallest member . And why houl j we ot be able to co 11side 3 =

168 THE COMPUTABLE REAL r UMBERS EC. 9.J

isn't real-computable (anc simi arly t hat a real-c mpu able f unctio n cant have a discontinuity at an poin ). Ho w can them achin deci e wh t is tt e first digit of /(.333 ... )?

NOTES

1. A good introduction to the tr eory c fin fin ite nu nbers and t o the eal n mbers, HE RE LAT ION S B ETWEEN is Courant and Robbins [196 ,Cha p. 5].

1 t J "PUR lNG M~ ,cH INE SAND 2. See, for example, Quine [ 196( ] and his fu ther eferer ces. or a nost i nteres -

I~EC URS NCl ing ambitious, and unconver tiona , if nc t enti ely cc mplet e, att< mpt o show IVE FU IONS how everyday intuitive ideas could be in orpo ated i n a fc rmal math< matic I system, see Freudenthal [ 196 ].

0.0 INTR< DUCl ION

In this nd tl e ne t cha pter e ex lore ~ number of formulations of he nc tion bfeff ctive CO ill) utab lity- and how that they all define the arne lass pf co ~put: tions or I rocec ures. Historically, many of these otio s arc se m bepen dent! . T he fa t th t they all turned out to be quiv lent· s one of 01 r rna ·or re ~sons for c onfidence in arguments using urin t;'S th eSIS. And ever if 1 urin~ corr putability should not agree

berfec tly w· th ev ryon 's int uitive idea a bot t what processes are effective, his rr ultip e dis over and equi a len em kes it certain that this par-icula clas of I roce~ ses h ~s an imp< rtant significance. Its study-the ranc n of I pathe rna tic call d tht ory o recu rsive functions-is one of the

inost elega nt a1 d se f-cor taine ~ m~ them atical developments of the went eth c ntur . It !so I rovid es a< ertai amount of practical insight

·nunc ersta nding then ~ture and I mita ions< f computer programs. n is ch pter devel bps t ne no tions of p1 imitive-recursive function and

1 enert l-rec1 rsive lrunct on ai d sh WS t ow t e latter notion is at least as omp ehen 1ve a tha of a Turi ng-m chin -computable function. The ~ath< rna tic al me thod for s ow in g thi is a urious one, called "arithme-izatic n." 1 rithr netiz< tion i s ven simi le, b1 t there is something artificial nd u natu ral al out i .W ile tl e log ic is< asy to follow, one feels that he wl ole t ing i a so t of c: xtenc ed jo Ke or pun, and that there ought to e an ore I ucid analy is th t yie Ids tl e sar n.e end result but gives more

in sigh into its n ture. Late r we will c utlin some more "natural" meth-ds, c ut fo n0\1 our main cone ern i to g et certain results within the rame work devel Jped liP to this p oint.

169

10.1 ARITHMETIZATION OF TU

We want first to show th machines within a rather show that one can build the co1(nput'* ideas about numbers rather th arithmetical technique to show how of effective computability yie on machines.

10.1. 1 Representation of the T uriha-rnal,ch

conditions in terms of qunHnor>IA<

Consider a Turing 111~'"11'11" tion; it will be in some state The tape will have a finite n otherwise blank. It is conveni nt to machines and to use '0' for the the tape will have the appeara

For example, if the tape is

then (s, m, n) = (0, 47, 15). only a finite number of b;'s represent each state q; by the I cc,rr,est)OJ1¢1Jmg sent the ·complete state of the by a quadruple of integers [q,

Each operation cycle of forming the numbers [q(

tThis representation is based on

RECURSIVE FUNCTIONS 171

Since the quadruples are HJi:lll.flllllt: lcond.~tic>ns, knowing the details of

the structure of the Turing rmat on is related to the Turing ay: , dij) and suppose for the moment

the right. Then the tape

o)

}

s* is the old c0 • The new b;'s have been moved up

the binary sum) and s ij fills

oved and the other digits

Co

right

172 TURING MACHINES AND RECURS! E FU 'ICTIO S SEC. 0.1.2

where we define H(n) to be the large t inte ger ir nj2, and P(n) o be ) if n is even, 1 if n is odd. That is,

P(O) = 0 H(O = 0

P(l) = I H(l = 0

P(2) = 0 H(2 = I

P(3) = I H(3 = I

P(4) = 0 H(4 = 2 etc.

and, in general, n = P(n) + 2 H(n . W at til is sh1 ws i that the ransformation involves only the simr lest rithr~etic idea -ad~itior (by unity), multiplication (by 2), a d tht quo ient 1 (n) and remain( er p n) of division (by 2).

If the direction d ij happens to be '0'- 'mo" e left"- th~n w~ need only exchange the roles of m and n ·n the abov tran forrr ation equa ions.

10.1.2 The Turing transformatic n

Now we will complete th rep esentation of the Tu ing nachine in terms of arithmetic operatiom. We can epre ent t e qu ntup e stn cture of a given two-symbol Turing mac ine t y red efinir g the thre fun tions (see section 6.0)

Q(q;, sj = q;j

R(q;,Sj = Sj

D(q;, Sj = dj

STA E

SYMBOL

DIR CTIO 'I

in terms of integer-valued furctior s of 'ntegt rs. only over the finite set of valut pair

(0, 0) (0, 1 )'

I, 0) I, I)' ... '

( k- 1' 0) (k-1,1)

hey peed be d fined

where k is the number of sta es of the wa-s mbo Tur ng machine. D and R take on only the two 'alues 0 an~ 1, lvhile Q can ass me alues between 0 and k - I. It will t e con~enie[lt als to h~ve tl e fun tion

D(q s) = I - D(q, )

such that 75 = I for "move It ft" a d 75 = 0 or "rnove right '-jut th~ opposite of D. We need also the di1ision by-2 unct ons,just d fined:

H(x) = qt otient of- I ~ ALF

" . -< X P(x) = remam era.

2 I PA ITY

s c. 10 1.3 TUR NG MACHINES AND RECURSIVE FUNCTIONS 173

Suppa e, finally,' e ha e the two< rithrr etic functions

1 +(x, v) = x +

J x(X, v) = X • y I MULTIPLY I

I' OW v ith t ese IVe car express ( ur transformation in a uniform way, \\ ithout any exter a! st tement ab tmt the vah e of d;/

q* = Q(q, s)

s* = P(m) · D(q,s) + P() · D(q,s)

m* = [2m + R q, s)] · D(t, s) -1 H(m) · 75(q, s)

n* = [2n + R('g, s)] 75(q s) + H(n) · D(q, s)

C bsen e the simJ: le-mipded trick used here. If the instruction is really "~ave righ ," then D(q, s = I and l5(q, s) = 0. In that case these e uati ns are exa tly tl e sane as hose of Trght· In the "move left" case, j st th app opriate ch nges are rr ade. Whi e this works, it has a certain I gical opacity (about whic we omplainec in the introduction of this chapte ). T e "p n" is that +' is used sa s )rt of logical "or" and 'x' is used a a lo~ ical ' and,' som wha as in Boo! ~an Algebra.

We hav exp essec the Turir g transformation entirely in terms of s mple aritl meti( fun tion . Ir deed (alt ough it may seem a little t diou ) we an write e ch ec uatic n explicitly as, for instance,

n *(q, , m,,) = 1 +cr P(P(2, r. ), R(q, s)) D (q, s)), P(H(m), D(2,s)))

Th s is v ritter out, not for cl< rity, put t< demonstrate that we really can express the t ansfc rmat on ec uations in terms of the given functions combi~ed b several ar plica ions t>f the "con position" operation.t

1 0.1.3 Red ction to the zero nd sL ccessc r func ions

Next we sho\'o that py th1 addi ion c f mo e equations we can eliminate r~any pf the basic ingn dient of tl e ab< ve sy tern! We do this by starting ' ith c rtain very imp! fun1 tions and< efini g the required new functions l y indl<ctim. In fact, usin1 indt ction, all fve need to start with is the 'zero unct'on"

O(x) = 0 I ZERO FUNCTION I

t Co enpositwn is the ope ration of sut stituti g fun tion names for variables in other unctio name . A rr ore fo mal treatmer t of this is gi en in Kleene [ 1952], but the details

· ren't i nportant here. Wha is imrortant is too serve ow we make new functions by using wmpmition operations on< ld, sirr pier, f nctions.

174 TURING MACHINES AND R CURS VE FU NCTIO NS SE . 10.2 SEC. I :u TU RING MACHI NES A NO RECURSIVE FUNCTIONS 175

and the "successor function' (for whi<h it is co veni nt tc hav twc ! other vari~ bles x2, .. ed a s par ameters, that is, as numbers . ,Xn egar notations): fixed hrou ghou the efini ion. (In e ch c se above there was either no

, I s UCCE FUNC uch Daran eter :>ron e.) Ir ead indu ctive definition of a function, there S( ) = + =X SOR ION appe~ r twc othe r fun tions whic h are as sur ned to have been already de-

Then we can define the additi< F+(J v the ions: fined: a bas efum tion ~ and anim uctiv step function x.t nfur ction , x) b quat i

1 ¢(0 x2, . . . ,Xn = ~ (x2, . . . , Xn)

~ { F•(o, x) ~ x I BASE <P(y' x2, . . ,Xn = )( (<P(y . , Xn),y, X2, ... , Xn) x2,.

F+(y', x) = (F+(y, ))' = S(f +(y, )) I INDU TION FOR ~ULA frhei npor ant t ing i that in e;; ch st p tht value of the right-hand side may I e cal ulat( din erms of al eady de fin d functional values. In any

Let us be sure to understand these equa ions bY se ing ~ OW t ey gi ~e th bartic ular i nstan e of his fc rm, t ~epa a met ~rs may be absent, either from value, say, of 3 + 6 = F+ (3, 6). v e fin that r/J itse f, or s exr: licit' ariab esin if; anc X·

Tl is int odu< tion orm' PR" is cal edpr "mitive-recursion. F+(3, 6) = (F+(2, 6 )' pr ~ (F+( , 6)) D F!Nil +(1,6 ))

ION = ((F+(l,( ))' )' ; (S(F

lfunct = (((F+(o, b))')' , (S(S F+(o 6)))) Any ·on th rt cm be a 'fine c. in terms of O(x),

; (S(S S(x) ,com 'JOSiti n,an dprin itive ecursion is called

= (((6)')')' (6))) a pri mitiw -recu sivej imcti£ n. = ((7)')' = (8)' = 9 .: (S(7 ) = ' c(8) = 9 w havt aim< st co mplet d tht proc ftha the functions q*, s*, m*, and 1

It can be seen that the value1 of a func ion s o den ned re dt term n'ed- * an all J rim it ·ve-re ursiv . 0 nly, v e ha e not yet accounted for the that the calculation must eve ntual y ter min a e-b caus oft he co nstant "fixed "fun ~tion Q,R,D,a ndD -that is, to show that they are primitive-decrease of the value of the in ducti iable ' ecurs ive fu nctio s. T J sho N hov this s dor e we show first how to define Jn va

We can define the multipl catio n fun< tion DX(y, x) sir niJar! fixe< fin it func tion < fone varia ble. ; uppc se, for example, that we want

{ FX(( , x) = 0 o den ne a uncti )n w x)so that

Fx(y , x) = F+( P(y x),x W(O) = 2, W( l) = 10, W(2) = 3, W(3) = 7

Of course, we can define a ny pa rticul ~r cm slant as a f ~ncti n: : nd tl at wt don 't can wha the pther valm s of W(x) are. Then we can < efine a sa is fact ory f ~ncti< n W by ( ~aste fully) defining enough other

C 4(x) = S( ~(S(~ (O(x )))) = 0"" = 4. luncti tms:

It is convenient to define t he fu ctior N(x ,whi his: '0'-d teet< r: \

{ Y(O = 3 i X(O = ID { W(O) = 2

{~ f\'(0) = l D} I ~ULL FUNC TION II Y(x' = 7 X(x' = } (x) W(x') = X(x). (y') = 0( ) = -hen we fin d tha t W( ) = ~(l) F= Y( D) = (j and that W(x) happens to

because then we can easily ot tain (x) a ndH x): lave v alue for II va ues o f X g eater than 3. Clearly, Y, X, and Ware

{: ~II pri [nitiVI -recu rsive funct ·ons. We leave to the reader to show that

{ P(O) = 0 (0) = 0 f~ncti< ns o two varia bies I 'ke ~ (x,y) and D(x,y) can be defined simi-P(y') = N(P(y )) y') = F+( f!(y) , P(y) I rly.

' twe give h rea vt ral fo m. cf> ·s the unctio being defined. >/; is the base func-'

ry gen

10.2 THE PRIMITIVE-RECURS!' E FU IICTIO NS ton, wt ich ap De a red only a sa co stant rasa single variable in the examples of 10.1.3.

he fun ction desc ibes s me cc mputa ion th at der ends on the induction variable y, already define value s of tt e new funct on ¢, and c n whatever parameters x 2, ... , Xn

All of our inductive de finiti< ns 1 the prev ·ous sec tic n ha ve the r ay be desire d. F r ins ance, in the de fin tion c f Fx(y, x) above >/;{x) = 0 and same general form. There is one i duct on v riabl , y,' nd p rhap some ~ (a,y, ) = F +(a,x ) = a X.

176 TURING MACHINES AND RE< URSIVE FUISCTIONS SEC. 10.2.1

We can conclude from all t is th t the Turi1 g tra'lsjormatiOI June ions

are primitive-recursive functions Warning. we aver ot yet ~hown, and !t is false, that any computation is primiti\ e rec~rsiv(; we will ! et to this

shortly.

PROBLEM. Sketch the cof!struc ion c f two varial /e fix d fu ction like

Q, R, D from primitive-recu sive dj;:finiti ns.

PROBLEM. Give primitiv(-recur ive d finiti< ns for f(y, ) = x "J(x) = x!

1 0.2.1 Turing machine compute ions c nd

primitive recursion

Although we have been abe to epre ent, y a et of four prim'tiverecursive functions, the effect of a singlt step of a Turi g m chin< , we have said nothing about the u timate effe t of onti ued pera ion c f the machine. We can handle this by in roducing new in teE er variable t to represent the number of step the macl ine I as t:: ken- the umb r of moments it has been in oper tion. Ne t we introduce four new (fivevariable) functions T4 , T" T"" Tn. he f netic n Tq( t. q, s m, n is tc give the value of q that will resu t if he m'(lchin T i staJ ted i1 con 1-iition

(q, s, m, n) and is run fort steps. To calculate this, we have tp iter te t t mes he tr· nsfo mati ns q , s*,

m*,n*. Thus,

Tq(O,q,s,m,n) = q

Tq(I,q,s,m,n) = q*(q,s,m,n

Tq(2,q,s,m,n) = q*(q*(q,s,n ,n),.*(q, ,m,l),m*(q,s,m,n) n*(q,s,m I

etc.

But, in fact, we can use the p imitive-re ursion sch me < irect y to defint Tq(t, q, s, m, n) for all values o t, an~ in g~nera

{Tq(O,q,s,m,n) q

Tq(l',q,s,m,n), Tq( ,q*( ,s,n,n),s*(q,s,m,n,

m*(q, s, *· n), r*(q, s, m, rz))

We can define similarly T., Tm. and Tn. t fo!tows t at the itented urin~ functions Tq. etc. are also p imiti e-rec rsive June ions. These definin~ equations have a simple intui ive n eani g. 1hey state hat t:1e ell ct o t + I operations, in initial ituat on (l, s, m n) is the same as t at o applying t such operations to ( q*, s*, 1 *, n )- tt at is of a I plying firs one operation and then t mor ope ation ! W at cc uld be sim ler?

EC. 10.3.1 TUF lNG !\> ACHI~ ES AI-D RECURSIVE FUNCTIONS 177

0.3 HE PROBLEM OF RECURSim WIT i SEVU AL V J RIAB ES

The propf in he pr vwu paragrapl is defective, because the recursive < efinition g ven f r Tq ~as not, in fact, an e ample of primitive recursion (PR) < s defined i~ section I P.2! he t oubl is that the parameters q, s, /fz, an< n do not s ay fi ed d~ring the c leu!:: tion. Nevertheless, it is pos~ ible tp define Tq usin! only legit mate form of PR, and it is the goal of t~is se tion o she w ho~ thi can e do e.

10.3.1 The inforn ation packing functions H, T,

Le us consider a imp! r form of the r roblem: Suppose we have a < efinition

T n', a, b) = T(n, < (a, b , T(a, b))

' here u an< T ar alr*dy-d efine< PR unct ons. Then I assert that T is ::I so P ~. but I car not ~nd :: ny di ect v ay tc force it into the form of the I R sc em a. The trout le is hat tre PF schc rna seems to provide no way ff,H twp different com utatipns t D interact; in this case the dependency < f T(n', a, b up01 botl u(a, ~)an T(a, b).

N c w this app~arance is eally false but t seems that to get around it requir s sorre so t of tnachitnery or d aling simultaneously with several sou ret s of i r1forn ation. Su pose for xam le, that we were able to find three R functio1s, C y,z), H{x) T{x), witt the properties that, for all J and ,

H(C y,z) = y

and

T(C y,z)) = z

'We wi I indt ed soon fir d jus such a tric of functions:

H = "'EAD

T = AIL

C = ONS RUC

hen e co ld re efim T as follm s:

T(n, a, b) = T* (n,C(a,b))

and define 1 * recursiv ly as

{ T*{C,x) = T(C,H(x ,T(x))

T*(n, x) = T*(n, C{c {H{x, T(x)), T{H{x), T(x))))

178 TURING MACHINES AND RE URSI E FU CTIOI s SEC. 0.3.2 s c. 1( .3.3 TUR *G M i

CHIN S AN j) RECURSIVE FUNCTIONS 179

n the surf a e, bu t the ~ctior will how I Sp ¢ i~ PR fter II-i e., bt cause it is This may not look any better o ext s iden tical with a PR function de-

that it is indeed PR. We first v rify hat 1 *has the r ight p oper ies: fiped a poth er way! I

T(n',a,b) = T*(n',C(a,b)) '~ P~OBI EM 0..3-1 . Prov ~by i duct ipn th t ¢(n, x) = if;(f(n, x)). Hint:

= T*(n, C(IT(H(C( ~.b)) , T(C(t , b))) T(H( (a,b ), T( < (a, b )))) I Lse ind ~ction onn, ass urn ing ea hhYI othes s true for all x. First prove that

= T*(n,C(IT(a,b), T(a, l ))) f n, u(.x )) = ( (f(n, x)).

= T(n, IT(a, b), T(a, b)) 'I

1 ).3.3 Defir ing th e info matic n-pac ing f nctio s Now we want to show that T as d fined abo\ e, is PR funct on. f we H, T, and (

think of In 0.3.1 we sl ), 7(1

owe< that if we coul find functions C(y, z), H(x), and C{IT(H(J ), T(x (x), (x))) T x) fo whi<h

as a single primitive-recursivt func ion f X, he fc llowi ng th eorer will H(C( [y, z)) = y apply. a d

T(C( lv, z)) = z

10.3.2 The ITn theorem.

I tl en w coul ~han ~lea funct on li e T( , q, s m, n) except with only two

Consider v riabl ~s T(t , a, b) in ad ditio1 tot. Befo re we construct H, T, and C, let us

{; O,x) = 1/;( x) n ten at, b w usi g th ~m rr ore ully, we r ally can handle the full T

n',x) = ¢( n,IT( )) p oble1 p! Fe r we an w ite

T t, q, .s ,m,n = 1 *(t, ( (C(q, s), C(m, n))) where 1/; and IT are PR. It is nc t obv ious t hat¢ is PR , fror n thi~ de fir ition

h rribl ' as t ism y see m, ar d the n all heir formation in (q, s, m, n) is because it does not fit as an in tanct ofth e PR schen a. B ut, le us lc ok a

its values: p eked into ne n mbe

cp(O, x) = ~t--(x), X = C(C q, s), C(m, rz >>

cp(!, x) = [t>(O, c (x)) 1/;( ( (x)), 0 , altt rnati ely,

c/>(2, x) = ~( l, ( (x)) 1/;( ( (IT(x) ), X = C(q, C(s, C(m, n)))

c/>(3, x) = ~ (IT( 0 (IT(x) ) ), Ir fact, all th ~info mati ~m ca n bee xtrac ed a~ ain simply by using

0 0 q = H(H( fx)} and clearly in general s = T(H( ))

c1> n, x) = 1/;( p-n(x) m= H(T( fx)) n = T(T( ) )

where ITn(x) means applying IT to x, n imes. No ~ we can ~efint ITn(x or , alte nativ ~ly,

recursively as q = H(x)

{f(C , x) =X I s = H(T(x))

I= lTG (n, x) m = H(T(T(x)))

f(l ', x) n = T(T(T(x)))

and clearly fis PR and w ich c prres ond o twc difft rent i nforn atior structures:

lf(n, J ) = ( n(x)

I q I s I I m I nl I I I Hence q s I m I n I c/>1 n,x) = if;( (n, x))

180 TURING MACHINES AND R E URSI E FU ~CTIO s EC. 1 0.3.3 EC. 1 P.3.3 TUI lNG ~ A CHI! ES AI D RECURSIVE FUNCTIONS 181

or tree structures: Now (ve we uld I "keto mak am bre d as tic distinction between even and

[A dd n ~mbe s; so wed( fine

~ 11 J( ) = W"Cx) . N(J (x)) T

lvhich hast re pn pert':

I I I \ J(x = - "f xis even T

" q s m q s m J(x = 0 f xis odd

So we can define the real T by he sc pema Neal ode neE x) = N(N x)) " hich t-vill b e 0 if x is 0, and 1 otherwise. Nowl onsi< er th seqt ence

T*(t , x) ' T*( t, a(x ) (x), E(J x)), E(J J(x))), E J(J(J(x)))), etc.

where a is a single monstrous buts ill p fun tion; then the ( n the prem see tl

assures us that T itself, that is, T(t, ~ ,s,m n) is prim tive-r ecurs ve. tis e sy to at if is d ivisib e by 2, n t ·mes, then E(J"(x)) will be I,

Now let us construct H, T, and . " e WI II ado bt a imp! enc ding < then ise it will t e zen . So, ~.!sing the a theo em if we define D(n, x) to be

scheme by defining {~ 0, x) = E x) C(y, z) = 2Y + 2 y+z I

n', x) = D (n, J( x))

If we think of C(y, z) as a bina y nu inber hen i will have the s "mple form o tha

1 9 0 .. 0 ~ I '-0 0 .. . OOJ 'J(n, .x ) = 1 (J"( ))

z y ¥e ha ve in D th e PR func ion ' hich tells whether or not 2" divides x. where y and z are now rep res nted by st mgs of zer OS. r OW I is e sy to \I ext ¥e de ne see that this C(y, z) is PR:

{: ~(x) ft- z

(0, X = C(y,z) = e p (y) + eJ p(y I)

(n', ) P(n', x) + G(n, x) = where o tha

{ex p (0) = I

exr (n') = 2. exp (rz) G(n, x) = E(x) + £( J(x)) +E J(J( (x))) + ... + E(J"(x))

So our real problem is to find PR d finiti bns f< r the H anc T for whicp is the pumb er of powe s of" , up t 0 2", hat d ividex. Finally, if we define

H( y + y+z+ ) = H(x = G (x, x)

T( y + IJy+z+l) = (ve ob ain t e fur ction we v ante< -be a use 2 can't divide x more than x

Now in fact H(x) is the numb er of times X IS venl divi ible ytw ); an< imes! t We have now only to ge T(x) We will leave it to the reader to

again with the aid of the a" th or err wev ill be able o pre ve it s PR. on fir m tha t if

DIVIDING BY 2 n X = 2Y. y+z+

We have already defined ( n sec ion 1 p.1) t efur ction H(x) tTh s last l bserv< tion is really ita!. \ e real y defin ed G(n, x) recursively to be n

H(x) X

if is e' L: p(i, x) - en 2 i= I

H(x) = X - I

if X sod< : nd tht s shm ed, in ident: lly, he w this "bou ded s mmation" can be done primitive-

2 t-- ecursi' ely, fo any ft nction.

182 TURING MACHINES AND R

then

(as we have shown), and

i.e., applying H to x, latter function is PR.

PROBLEM 10.3-2. actually simpler. We could see easy ways to use

Define the appropriate of C.

PROBLEM 10.3-3. that, on the whole,

Discuss this. In the Sol uti function C, for any E > 0,

so that in this sense one methods, used functions ·

y

PROBLEM 10.3-4. The efl.c:odiirjlg··de,;pding numbers and uses facts possible to avoid most of tive. Consider using for

which is much smaller (I) that C(y, z) has that there exist which

c. 1 .3.3 RECURSIVE FUNCTIONS 183

sqt1en1e1s have a critically important s uniquely the values of a

lar!~JW1Jent~>;l there can be no ambiguity. pply it himself, see Davis any Turing machine T, the

d Tn are defined for all values of B curiously, this very virtue

rer,tesent by primitive-recursive e computations of Turing

rer>reser1tation, in terms of primitive machine to halt. And this

rst g ance. the definitions a little more mpute a value T(x) if, when standard representation of

ith (a representation of) the we will choose for our

ine configuration t

we will arrange that the agree that any final con-

it to be simple, but the Turing pluJnct,ualtion so that it can recognize where rer•re~;en~atJion (0, 0, 0, x) because then the

signifi1Ca11t digit of x and would r~presc~nlt:1tion (0, 0, 0, 2 x), in effect, uses

moves right (counting the O's) the whole number x. Thus the

184 TURING MACHINES AND R CURSIVE FU CTIO S SE . 10.4

figuration of the form

(q = I, s = anything, m thin , n 2Y ( + 2 · any hing )

represents the function valu = y.t occurs, we say that the mach ne d es n t that T(x) is undefin~d.

Now we can easily arran imaginary Turing machine, framework of primitive-recu sive uncti

such put

function W(x) in 10.2 got de ned for a val es, w ethe we ante or not.)' What we need, prec sely, fort e represen ation ofT x), i able to find, given x, the value 1

for the smallest value oft for hich

Tq(t,O, ,0, X)=

neve etter,

it t to b

For this means to run the achi e un il it stat q 1 our altin state-and then read what is nits tape. It is usto ary to re rese t "the smallest value oft for which ... " b '1-1,' o th· t

K]

means "the least t for which P(t) K.' Us'ng th's no ation fine the function T(x) comp ted y Tu ing achi e T y th

T(x) = Tn(/-1 1 [ q(t,0,0,0,2x) 1], ,0,0 2x)

That is, first find out how many s eps i will take he Turing mac halt (i.e. to enter q 1), and t en c· lcula e th valu of afte tha iterations of the Turing transform· tion.

DEFINITION OF GENERAL REC RSIV

If a function ca be efine primitive recur ion, toget er minimization o erat r /-1, then it is recursive functi n (o mo e br efty function).

We have just shown functions chines .are general-recursive. We

utab e by Turi g m·nver e in he fo lowi g

chapters.

t That is, y is the number of zer s befo e the 10ne has to verify that the func ion E(X) =

{ ~(~') F+( (X), (X))

ead.

s c. 10.5 RECURSIVE FUNCTIONS 185

e d of our g

1 .5

itive recursive, then

f(n = (I) ¢(2) + ... + ¢(n)

. Pr ve als

inimi ation operator" J.l(l<x) is defined

mallet val e oft less tan x or w ich 1/;(t) = X(t), unless there is t, in hich case (x) 0." how that ¢(x) is primitive-recursive

i 1/;(t) nd X( ) are. Solut on in Kleen [ 195 , p. 228].

that LP( ) = ' the largest prime factor of X" rimit ve-re ursiv ?

In properties of the generalthe r suits are o the negative side-showing ction with such and such a property, or that clas of f ncti ns is effectively unsolvable.

owing tha while each effectively come cer ain I mitations, there are effective

s of exten ing uch lasse . In any ase, one obtains a beautiful, ly se f-co taine , ne structure of mathematics-a theory of the s be ween diff rent lass s of com utable functions. We have

s ace nly o sk tch f w kno n, but fortunately even the eleme tary esult are f gre tint rest.

186 TURING MACHINES AND R CURSIVE F NCTI

We must begin by pointin out hat t e de recursive function" is much ore ricky and said that a function is general recu set of equations. The troub e is looking at such a set of equa ions, that When we speak of a "functio " we mea each value (or n-tuple of val es, i resulting value." .But in ge era! yield defined values. For exa pie,

E(x)

, in 0.4, ubtl than it a ejin d by

, wit aria les)

do sn't !way

y[E(y) = x]

which satisfies the formal as ect f ou defi itio . N w fo x = 0 th value of F(x) is defined an is 0. Bu for , ou defi itio leads nowhere; the expression ,uy[ (y) = l] mea s: Fi st se if l; if this isn't so, see if E{l) = l; i this sn't s , see if£( ) = I; if 't so, etc., etc. The computation never t rmin tes ( ecau e E( ) is zero and no value is ever assigned o F( ).

This is a characteristic ph nom non, and ecause of i , we ust hesitate to assume that a sy tern f eq atio s re lly d fine recursive function. We nor ally requir aux liary evide ce f in the form of an inductive pro f that, fo arg men computation terminates with uni ue v lue.

Because of this we will t lk u ually of t e partial- ecursive fi nctio described by a set of equations inv lving zero, succ ssor, prim tive ecurs ion, and minimization. Whe we alk bout a pa tial-r curs·ve fu ctio (or partial function, for short) F(x) it is unde stoo that ther may ben value defined for some (or ev n any!) v lues f x. IfF x) ha pen to b defined for all values of x, th n we call t a t tal-re ursi e fun tion r, fo short, a total function. {In th liter ture 'tota -recu sive" is a yno ym o "general-recursive.") Of cou se a y tot 1-rec rsiv fun tion s als con sidered still to be a partial-re ursiv function s wei .

One might suggest that matter wou d be simplified, if we wou d just confine our attention to the total recu sive uncti ns. But e w·ll se , shortly, that this suggestion i high y im racti a!!

What does the term "recu sive' mea her ? It rec rrenc of the name of the thing-bein -defi ed i sin

{

F+(O x)

F+(y',x)

X

we find the name of F +, the thing bein -defi ed, i side an express on o the right-hand side of an equatio , so this is a r cursi e de niti n. It

to atisf the

P OBL M. ction , sot


for being a primitive-recursive

defi always define total-recursive function-values. Why is this?

The following re ursi ely:

of equ tions defi es a certain function A (x)

1 .6

c

{0, )

(x', 0)

(x', y')

(x)

y'

A(x l)

A(x a(x' y))

A(x x)

ot satisfy rmal conditions for either primifun tions but t ey d in fact give a unique value

uld, n fac , have given a much weaker arne y: " ny function defined by any

by Kleene [ 1936] that our

ann equations above really ou calculate A(2)? A(3)? A(4)?

Thes equa ions epres nt a s rt of simul aneous induction on two variab es at once. Fu ction defi ed in this way are known as "double

orne f the (including this one) are not is kn wn ( ee Peter [1951] or Robinson -recursive unctions that are not (n - 1 )

re ursiv . On he ot er ha d, it i also now that the n-recursive functions ar total Ind ed, in secti n 10 .. 1 ou proof of the u-theorem shows how on mig t star to pr ve th t the oubl -recu sive functions are total.

For sever I rea ons, e w be a le to think about all the partial-re ursi e fun tion as ar anged in a sequ nee r list

1 (x), f2(x , . . . f.( ), ...

e w uld like to have his li t arr nged in an orderly manner, n talk ab ut it in te ms f eff ctively defined operations. nt o a set of ob ·ects sa d scret , definite sequence is called

188 TURING MACHINES AND R CURS VE FUNCTIONS SE . 10.6

an enumeration of the set of bjec s. F r our pur oses it is imp rtan that the enumeration of the p rtial- ecur ive f nctio s be effective, t at is that we have an effective, com utab e wa to fi d w ich is the nth fu on the list, or at least to find enough for us to be sure that n en mer tion xists to know any details about its prop rties! ting it in detail, it is sufficie t to ive possible-that we could const uct ne if

Now the requirement tha the num to saying that given n and x, e are nab! tation of fn(x), that is, the c recursive function for the arg be undefined.) This in turn partial-recursive function t

which is universal in the sens efine exa tly has the same value. If we hav sue a fu ction U(n, x) w the index number and of fn( ) as he n h fu ctio or t e fu index n.

Why should we suppose t at th re ev n exi ts an effec ive e of the partial-recursive functi ns? After all, they a e a bizarre, disorderly variety o obj cts; he c nseq ence recursive definitions. We give wo r ason , one direc and ne i direct.

A direct construction inv lves obse ving that each parti 1-rec rsiv function has a definition that consi ts o a fin te se of e uati ns each in volving a finite number of sy bol desi nati g ze o, su recursion, and minimization, c mbi ed in an a rang men ber of commas, brackets, an par nthes s. he e les rangements aren't important; what is important is tion one always can set up s me ort f inti ite the composite objects, just a was don formed parenthesis sequenc s. I the things seem more complicate , the e is o e wa in hich they re a tuall simpler. That is: since most arti 1-rec rsive functions or ra her, efini tions) don't define genuine te min-ting omp tations at all, e ne d no be too concerned, in enum ratin the definition , tha all he s mbol strings generated be well-for ed. n fac , we can c edur that eventually produces any and II str" ngs f sy r th

tThis sentence makes sense onl if w identify the notio of (i) "effec ivenes " wit (ii) "definable in terms of general-re ursion." Th s is a hi los phical rather than · math -matical hypothesis: it will be reinf reed oon hen w sho that ii) is athe atically identical with (iii) "computable by a uring machi e."


ical m--that of staying inside a y us ng, s y, a binary-number scheme

(j st as in se tion 7 .2) or th unr mite n u bers of variable- and funcreq ired. Th n (as in section 9.3.1 which the

, th ugh he c apte as a whole is optional) a che e wil be a effe tive enumeration, provided

ription correctly. See also

u ivers I Tu ing achi e.

fi

ti ti

So ill talk fr ely f th nth part al-recursive function fn(x). not ssu e an thing! So e fu ctions fn(x) may not be der a ingle valu of . S me ifferently indexed functions may be t e sa e: f;( ) = 1(x) or all x. Indeed, some func

nfini ely o ten i the list ith different explicit definisure of is that if a function is partial-recur-

si east ne i We can ·!so e ectiv lye

o e ar ume t (w ich are p,(x), 2(x), ... ' n(x)

dex i the ration. itive-recursive functions of s) and refer to them as

that a set of functions is that we have a method

Desc ibe in orma ly bu in more detail, how one might constru t effe tive enume ation of (I) Th primitive-recursive functions, p; x). ( ) The artia -recu sive f s,f;( ).

step in he definition of a universal , x) = Pn( ), and show that it is total

the efinition of a universal partialhat s, try to anticipate the next

190 TURING MACHINES AND ~ECUF SIVE UNCT ONS ~ EC. lC .6 s c. lO.f> TUR NG M CHIN SAN D RECURSIVE FUNCTIONS 191

THEOREM 1 in few etail It ollov s (wi h so lne w rk) t jlat there is no effective pro-

Not all total-recursive fur. ctiom are 1 rim it ve-re ursiv . adure o tell whicJ ofth ~pan ialfu ction ~are otal functions.

RH ARK Proof Consider the fun< tion

ItS< ems bott er to ~orr abo tunc efine values for functions. Why F(x) = px(x) + 1 = V( f\• x) ft 1 n t do ~way with hem py so lne ru hless techr ique like the following? Let

It is clear (I) that this functi n is otal- ecun ive b caus PR funct ons ~ re I fn x) b a p artial recu sive unct ion. Defir e gn(X) = fn(X) when fn(x)

always defined but (2) that it ca nnot be p imiti ve-re< urSIV e, be a use it is defin ed, ar d Jet n(x) = 0 v henJ n(x) i not defined. Now let us confine

differs for at least one valu of it s arg urn en , nar nely ~7(n) Pn(l ), frc m o r att nti01 on I) to t e g'~ . Fa r the g's a re now defined everywhere!

each primitive function pn( )-C an to 's di gona arg men agai n! The The trc uble s tha not all th g's 0 co stru< ted will be computable, i.e.,

function is general recursi' e be< a use the ' 'univ rsal primi ive-r curs ·ve •' g< neral recu sive! In fa t the troul leis rofo nd, for:

function" V(n, x) is; hence, 0 IS TH ORE!\- 4

F(x) = S( /(x,-' )) Wh 'le SO! ne of he pt rtial runct ons a e no total because of some trivial THEOREM 2 I d feet 1 n the r defi riing quat1 ons, orne eire il: completably partial, because

There is a total-recursive June ion t1 at gr ows fi ster han ny p imiti e- tl.ey dl not agrel with any total recur ive j unction everywhere they are

recursive function! d 'fined

Proof Consider the tota func tion ~(x) ~efine d by Pro of L ~t F( ) be ~efim d to have value fx(x) + 1 wherever fx(x) is d finec , and to be und fined elsev here. No lv we cannot then argue that

S( 'b) ' 0 F(x) is a nev func ion c iff ere nt fro mall the} (x). F(x) could be the same

S(G ',b) S(a 'b) -t V(a b) a som fn fc r whi hfn( ) is u ndefi ed.t But 1 ow suppose that F(x) could

-< T( ~.a) F 0 b "co f11plet ed," py as ignir g va ues ' here it is undefined, to become

T(l ',a) S(a 'b) -t T(b a) s< me t ptal-r ecurs ve ft nction. ., hen the r ew total-recursive function v. ould ~ave orne posit onfn in th ~ enu fn.era ion (section 10.4) and, since

'- A (x) I= T( ,x) J, is a total recur ive f ~ncti n, fn n) is defi~: ed. Hence F(n) must have

In fact, bend fined in th e firs plac , an< F(n = fn (n), because when we "com-

x-l x-l p eted' F,w did 't ch nge I read y-defi ned v alues. But then by definition

L L I

A"(x) (i,j) F(n) = fn n) + I = n(n) i= 1 j~I

PROBLEM 10.6-4. Ve ify th at M x) ha the equir d pre perty that, for v. hich simp ossib e.

each n, Itf )II ow fron this that whi){ the parti:: )-recursive functions can be

e ectiv ely er umer a ted, the tc tal-n cursi e fm ctions can't! Specifically: M(x) > Pn (x}

for all sufficiently large .x . (In act, t is ho ds wh en eve X> n.) A- (x) i self TH ORE~ 5

therefore cannot be pr" mitiv< -recu sive! Wha is n )t pri mitive -recur sive Th re is CJn e.ff ective proc dure that, given any effective enumeration of about the above definiti n? (orne) total recu sive uncti ns, J rodul es a new total-recursive function

THEOREM 3 t at Wt s not in the given enu11 erati n.

There is no effective w~y to deci1 e, fo arb 'trary n, " he the fn(O is Pn of l et Tn x) be thee [lume atior . Th n

defined. I 1 (x) , Tx( ) + I

Proof If one could, th is wo l.lld s lve t 11e he lting prob em f r bl:: nk- tNot thus owC< ntor's ~iagon I argL ould ail! The partial-recursive functions ment tape Turing machines. To see tt is, as ume hat I l'(n, x is d fined in te ms C< n all b effect vely e1 urn era ted. Ir fact J (x) as define here is one of the /;'s, and /;(i) is

of a universal Turing mad ine, a nd w tJrk b ck fr pm th e end of IC .3, fil ing u define ~fort at fun tion!

192 TURING MACHINES AND ECUR SIVE I UNCT ONS s~c. 10 7 ' EC. JC .7 TU lNG I ACHI ES At D RECURSIVE FUNCTIONS 193

is a total-recursive function , for Tx(x is ' !way defi ned, ince all t e ' here thee are t xpre~ sions and D1 is stat ment (or equation) that may T-functions are total, and is diffe ent f om ach uncti on T (x), ~t le< st I e tru or f· lse.

when x has the value n. Th is ex~ ressic n me ~ns It is worth emphasizing hat he or m 5 has < posi tive c harac ter. It

says that so far as effective e atior co net rned, the' diagc naliz a- See i P1 is true; if so 1 he va ue of f is given by e 1 • ume s are

tion process," namely comJ uting with both the e ~ume at ion proc ~ss a d If PI is fa!~ e, the valw off is giv n by e2.

the processes being enumera ted, l as a curio usly reati e ch aractt r-o ne 1 his c ondit on a! expn ssion direc tly n place s the artificial multiplication gets something new. I ndee d ont can repe< t thi ove and ove' agai n, t ick. It a!> o dot s mo e; it has a so th pov er of the minimization oper-

getting infinitely many new func ions. Me st "I hi los ophe s," q uick to <tor. n fac t it c n gi\ e us here cursi e fur ction T(n) directly from the

leap at interpreting unsolval ility heon msa evid nee t hat m a chin es ha ve 1 urin! tran form at ion equa ions Jf 10. 1: limitations that men have not, have over! Jokec this curie us, I ositi e, T q, s, '11, n) = (if then n elst T(q*, s*, m*, n*)) q = aspect of the situation.

r ate hat t his el min a tes er tirel:y the paran eter t which carries the in-forma wna bout he le gth ( fthe com~ utati Jn, as well as eliminating the

10.7 CONDITIONAL EXPRES5 IONS; r inim izatio n op rator .n eM Cart hy fo rmalism is like the general

THE McCARTHY FORMJ LISM r curs ve (f leen ) sy tern, in b ing based on some basic functions, compc sitior , anc equ lity, but .vith the c onditional expression alone

The arithmetization of any p rae tic a! co mput ation proc edure by 1 he replac· ng b oth the p imiti ve-rec ursiv sch erne and the minimization

methods of section I 0.1 wot ld be aver v aw~ ward busi ess b ecaus e of he operat or.

obscure way in which the ' flow of co ntrol ' of he p oced re h s to be It s difi erent ho\\ ever, in t at it has explicit provisions for self-

built into the equations. Cons der he d vice in IC .1.2 Jy w ich he r ferer ce, se that inste ad o usm g tric ky ar ithmetic methods it can de-

direction function D(q, s) w as us d the re to contr ol wh ich tr ansfo rmati on s ribe com putat ons, enur nerati ons, univ rsal processes, etc. more

to apply. We used the expr ssion direct! . (\\ e wil not, howt ver, ive d etails of how it uses "quotation"

R(q, s))D q, s) -H( m)D( q, s) frthi.)

m* = (2m+ + We will ot de veloi= the vhole of M cCar hy's system here, but, to get

and the "branching" of the proce ss wa base don he fa ct th:: t we ould in t efta or of it, w will deve op th e Tu ing t ansformation in a simplified

effect compute both possib e res ults a nd th en m ultipl v one or t e ot er v rsior of it Ass met hat w hav avai able he notions of

by zero. This is both une stheti and ineff cient . Th is de ect-of pe or ZerJ process-description ability -is e quail a pi= a rent in t 1e co nstru tion of Sue

dequ< esso

Turing machine state diagr ms. Ther , too whil e the retic lly a te, Equ ality of nu mbers) the description of a compu ation proc edure is ve ry aw kwar d. T e wh ole Cor 1posi JOn bag of arithmetic tricks o this chaj: ter, vhile inter sting , are unsa is- Con ditio alE press IOn factory, both practically an d esth etical ly.

In the languages used for c igita com puter pro~ ramn ing, we f nd We defin e the ?rede essor func ion bv

is ion for selec ion e f dif erent putat ·on : much more adequate prm com p ed (n = pr ed2 ( '0) "branches." But practical omp uter I angu ges d o no lend them selve to p ed2 ( '1, m) = (if m' = nth nmf lse pred2 (n, m')) formal mathematical treatr 1ent- they are n ot de ignec to rr ake i eas~ to

V e wil prove theorems about tht edur~s th y de scrib I a I a per by I denc te pr d (n) by n . No te tha t pree (n) is not defined for n = 0; pro tl is wi I not r if th e fun ~pro McCarthy [ 1963], we find forr alisn that enha nces he p actic I asJ ect matte tion s use IJerly.

of the recursive-function pt, \', hile brese ving and impr ving its Ne t we ~efi nt the L seful funct ons:t once

mathematical clarity. tNot e that e can ot reg rd (if ~ then a else ~) as simply a function of three quantities McCarthy introduces" ondi ion a expr ssior s" of the f rm p,a,and b. Fo thisw auld rr ake us evalua e then all be orehand. But. often. the computa-

ti n oft will n ver te minat when p is tr ue; in his ca e we must evaluate a without even f I= (if o 1 the ~ e 1 e se e2 lc oking t b.

194 TURING MACHINES AND ~ ECUR~ lYE F NCTie NS SIC. 10. s c. lO 8 TUR NG M f\CHIN ES AN p RECURSIVE FUNCTIONS 195

P(x) = (if x = 0 the• 0 els (if X = l hen l else P(x- ) ) ) 1 p.s I ESCR PTiot OF OMP ~TATI PNS I SING

H(x) = (if x = 0 the• 0 els (if X = l hen C else fl(x- )')) liST -Sl RUCT ~RESt

X + y = (if X = 0 thei y els PX- H- y ') In his se ction weir trodt ce a orm<: I ism or representing functions of

xy = (if x = 0 the 0 els ~y + x-y " ists" of in egers In he n1 xt se tion we u e a more expressive system

This is enough for us to defi ne th fun tion com{ uted by a Turi1 g mf-tl at ca r des ribe "bran hing ' list~ , or I ists of lists, etc.

chine T, exactly as in section lO.l. : DE INITI )N

T(n) T* 0, 0, 0, 2") An /list sa se quen e of i ntege s

= T*(q, s, m, n) = (if l = l then else if D( q, s) = 0 tl en (x,, 2' .. . 'Xn)

T* Q(q, s), P( m), 2 ?1 + R(q, ), H( )) The II 'st of oint gers < > ' is cal led NI .

els T*( J(q,s , P(n ), H(, n), We de fin e thre e fun tion of I lists; if A i an integer and B is the list

2n + R( q, s)) ) <x ,, .. , 'Xn , the wee efine

which is both reasonably co mpac and rea so nably expr ssive . Of cour e C(A, B) = A,x ' ... ' Xn) CONSTRUCT

we have to define the state-t an sit ·on fL nctio :1s (as in l( .2), t ut fo this it H(B) = x, HEAD

is easy to use nested conditio nals I ke T(B) = (x2, . .. 'X > TAIL

R(x, y) = (if x = b the• (if y = 0 t hen 3 else ( fy = l If B is NIL u en H~ B) ar d T(l ) are unde pned. If B is a list of one integer then 2 e se .. )) else (if X = 11 hen(' f y = 0 <~)the H(B = X and 1 (B) = NIL.

then 5 e lse (if y = then l else ... )) etc.)) We can d edUC( the r elatio ns 1

and simply specify the funct' on pc int b poir t. H(C("' , B)) = A The above formulation I or T till i volv s the bin a y ar· thme izatic n

(C(f 'B)) = B of the Turing machine and i stap . We can: !so sl owd ·recti that gene1 a!

recursion is within the scop oft ~e pr sent orm: !ism ~Y ot servi g th t, sc that the f netic ns H find T can e US\ d to ~isassem ble an I list put to-

given any already defined ft netic nf(x , we can Dbtai1 the resul of a p- g\ ther flsing c. Now uppc se th ~t W( adjc in these functions and the

plying the minimization < per a or f.L [f(x = !I ] to it bv ev luati :lg cc ncep of a1 /lis to th e noti ons a read avai able (equality, composition,

J.L(O, N), where cc nditi Jnale xpres sion). The n we can c eal d irectly with the Turing rna-

J.L(X, N) = c fj(x) = N then else p,(x', N)) dine t< pes b y thi~ king of the tape as tw J /lis s, m and n. We then define the fou func tions, as "I st-pr Jcessi ng" o per at ons:

while primitive recursion is obtai ed b defi1 ition like

x)) ( *(q, ,m,1 ) = ')(q, )

cf>(y, x) = (if y = 0 th n If;( ) elst x(cf> y-, X ), y-, . *(q,. ) = if D( , s) I th en H(n) else H(m)) , m, 1

EXAMPLE n *(q, , m, 1 ) = ~ if D( , s) o I th en C(R(q, s), m)

We can define m/n = th e larg est x orw 1ich n ~ n x as: el e (if (m) o NIL then <o> else T(m))

mjn = divide m,n, 0, 0) f *( q, , m, 1 ) = ( if D( , s) o 0 th en C(R(q, s), n)

divide (m,. n, x, y) = (if y = n the ~ divi ~e(m n,x' 0) el e el e (if (n) o NIL then (0) else T(n))

(if m = nx y th ~n x e se di ide ( tn. n, 'y') ) T ese f uncti ns e press the unn quir tuple s in a straightforward way. F r ex: mple r*(, , s, f fl. n) "rea s" t e fir t element on list n, if the

PROBLEM 10.7-1. D~fine t ne fun tions m chin mm es rig ht, ot lherwi se the first Ierne nt of list m. The last clause (I) Prime(n) = (ifn is prir e the1 I elst 0). (2) gcd(m, n) = the reate tcom monc ivisor ofm: nd n. tThis ·ection is opti pnal. (3) ¢(n) = the num per of integ rs le~ s thar n wl ich h ve nc divi~ ors 1Thus the sy tern h' s as " rimiti e func ions" right a t the start, the C( y, z), H(x), and

in common with n. T( ) funct ions w ich w had t< const uct in sectio 10.3 1

196 TURING MACHINES AND ECUR lYE FUNCTI N

in the definitions of m* and n* is a d vice or m king the and n act as though they we e infinite, y adding n ex ra ze o w the machine reaches a point furth r ou tha it h s ev r rea hed For (0), which happens to b C(O, NIL), repr sents adjo"ning ne

square to a tape.

10.9 LISPt

enev r efor . blank

For the sake of complete ess e wil no desc ibe a little mor oft e full concept of list structure as develop d by Me arth . W wan to e able to handle, not just lists of in eger , but lists of lis s, lis s of lists f lists, etc. We can do this in n ele ant ann r, so hat t e ba ic fu ctio s H(x) and T(x) come out to be mor sym etri a!. To do this, e in rodu e a new basic notion-a binar co bin in ope atio "do "-and m ke t e notion of "list" a derived, s bsidi ry c ncep . W begin, th n, wi h only the following basic notions:

Atom: The enti y NIL is an atom l, 2, ... are ato s. Equality: We c· n tell if tw a to Symbolic ex pre sions · An a to expression. If and B are sym so is(A ·B).

Symbolic funct ·ons: If A and are pressions then

(A, ) (A B)

((A ·B)) A

((A ·B)) B

Then we can deduce the ide tity

C(H((A ·B)), ((A· B))) (A· B)

Lists will not be fundam ntal bjec s but are efine as ere! cert m kinds of symbolic expressions; n mely ( ) NIL s a list; if is n at m then (X) = (X · NIL) is a lis ; an rally if X Y, ... , Z, etc. · re atoms or lists then

t Optional. "LisP" (List Pr cessor deno es bot the a them tical f rmali m, par of which is described here, and the practical co puter progr mmin syste bas d on it, referenced in the notes. 1

EC. I .9 TU lNG ACHI ES A D RECURSIVE FUNCTIONS 197

ake a list that i a lis of lists: For example,

D, E))

, an (D, E) and whose symbolic xpression i

(( • ( ·NIL))· ( • (( • (E ·NiL))· NIL)))

0 e can verif tha the efini ions n 10.5.1 are consistent with these efinit"ons f H, T, C, a d lis s.

With th s rna hine y we can epre ent the Turing transformation as llow : Le t = he li t ( q, s, m, n), her m and n are themselves lists

r pres ntin the urin mac ine t pe h lves. Observe that

q = H( ) (T(t) (T(T t))) n = H(T(T(T(t))))

to i s four sublists. Define the

ta e (n) = (i n = NIL t en C( , NIL) else n)

make tap whe

((q.s,m,n)) (if (q,) = then

I st (Q q, s), H(n), C(R( , s), m), tape (T(n))) else I st (Q q, s), H(m) tape (T(m)), C(R(q, s), n)))

a d

*(t) (if (t) I th n H( (T(T(t))) else T*(T(t)))

t en T ((0, 0, 0, 2")) s the partial-re ursiv function computed by the T ring mac ine.

Sho that this s stem can be further reduced to one in whic the o ly primitiv s are

tom : Onl NIL

quali y: 0 ly "if X = IL th n ... else ... " uncti ns: (x), T x), C( , y)

a d

T at is, hat o e doe n't e en ne d the otio of integer or successor.

NOTE

l. In the LISP programming cons (A, B), car (A) and cdr this advanced computer list-p 1.5 Programmer's Manual [ 19 See also McCarthy [ 1963] fo Bobrow and Raphael [ 19 kind. The use of list-struc;tur~ in complicated computer-orc)li!;r.amm [1956]. The recognition that lsu]ppl¢rrtentjn:g produced an alternate form rlecursi~e furjct110n McCarthy, who also de,relcm~:d language.

1.0

eff,tktive computation which are digital computers than

uters have a number of atout by the machine is de

in meaningful blocks. to another as a function of

stored away in a memory basic step-an elementary

rne:ani!1gful unit. This combination complicated processes with

d-to-understand strings of :r"''"'"' "~"' of Turing machines nor

a satisfactory vehicle for ht really want to use for the system of the present

only in the computer pros developed here lie in a

for analysis and underfairly clearly how the sys-

exalrrime<I'J. The new machines present lved in effective processes,

re of what such processes

199

200 COMPUTER MODELS

can do. Of course, what seem nat ral a d in uitiv not seem so to someone else. ach of the yste the outstanding theorists, and ach has se ved i discoveries.

SEC 11.1

The first formulation ofTu ing- achi ter-like models appears in the pa er of Wan that would have been much m re d fficul pres isms. The development in t is c apter is a apte methods we have been using · nd i corp rates idea sug ested by and by John Cocke. One ca find mos of t e b sic r suits [1952] but without their formulation as co put r pr gram .

The reader unfamiliar wit co pute s is arne tha co pute s do not really work as shown here; in p rtic lar t ey d n't have s all umbers of "registers'' each of i finit cap city. Ins ead hey ave large numbers of registers of small c paci y!

11.1 PROGRAM MACHINES A D PR

If we are to use machines as ffecfve c mputabilit , we have somehow to lift the finit ness estri For Turi g m chine this was done by introducing the i finit me ape. But ther is a great deal of inconvenience about the urin -mac ine omp tati n pr cess. One should not have to pas thr ugh very hing rd i the course of writing down or r triev ng e ch ieee or old data. There is something repellent bout all t e art fices and encoding.

The modern digital computer as a diff rent large number of separately acce sible, fini e-ca memory units. Of course, any rea co pute me ory is act though the number of such re ister may be v ry Ia ge. might be to make our model ike t is, e cept ith registers. The trouble is that t is w uld ring the lines of a tape-like succes ion o the an infinite set of names or symbols or th regi handled directly by the finite-state art o the achi

We will suppose instead t at o r co put r ha of registers, but that each of hese can old rbitr rily that is, each register is infinite ·n ca acity. Th re wi I als part of the machine-the ce tral roce sor or pro ram 1ing nit. this part of the machine is fi ite, i can ot d al direct! with arbi raril large numbers. Therefore w will have to g·ve it some way to d al in directly with the full contents f the regis ers.

s c. 11 1 COMPUTER MODELS 201

Ou sol tion o thi is si pie and ather radical. The programming unit w'll be ble t ins ect a y re ister but c n discern only whether that r giste is e pty or n t-either hat i con ains 0 or that it contains a p Sltlv int ger, ut n thin else It ·s qui e surprising that even with t is re tricti n, a d wi h on y a nite urn er of such registers, we can s ill re lize any e ectiv proc ss- erfo m any effective computation. To

is, w will show that or any Turing machine there is an equivalent rna hine, as we will all it.

gam p ogra

Sy bol

[2]

D

'Halt"

1For recisio , we se the symb co tents the n mber tored n it. an way.

a'

H

Registers

Structure of a program

IS "built into" the

Meaning

e contents of register a to zero. Go o to next instruction.

Add I to the contents of register a. Go on to next instruction.

If co tents of a is not zero, decrease it by I and go on to next instruction. If co tents of a is zero, jump to the

register and the symbol a for its rom the context which is meant,

1Thes are ot, t en, " tored- rogra " c mpute s. It is generally recognized that the g eatest advan es in oder com uters ame t rough the notion that programs co ld be ept in he sa e me ory w th "d ta," a d that programs could operate on other pr grams, or on thems lves, a thou h the were ata. t is perhaps not so widely understo d tha one c n obt in the same heoret cal (t ough ot practical) power in machines wh se top level progra s cann t be s modi ed- s in th · s chapter.

202 COMPUTER MODELS

A program is a numbered sequ nee f inst uctio s. statement naming ( 1) an ope at ion, ( 2) regi ter, one or two other instructions W can explain th putting down a set of ope atio the programs.

Our first machine is cap ble four 11.1-1. The letter a in these exam les ould any other register, and then t e co resp ndin register so designated. We now desc ibe Suppose that a machine ha thr e registers a, program:

Instruc ion number

1 2 3 4 5 6 7

- ( )

- ( )

If the machine begins with instr ction num a certain number in a, the m chine will ventu /ly h It WI

ber in b (and with 0 in a). T see his, t ace he o erati in a and anything in b and w

Instruction number

1 set b to 0 2

go on 3 add 1 to b 4 add 1 to b. b is n ow2, a is l.)

5 set w to 0. 6 w is 0 so go ack o2 2 subtract 1 f om a 3 add 1 to b. 4 add 1 to b. 5 set w to 0. ( It wa 0 air eady, but no

harm is d ne.) 6 w is 0 so go to 2 2 NowaisO o go o7 7 Halt.

s c. 11.1

W,

that time here is th tw ce th t nu -

n, st rting with 2

Regi ter c nten s a b w

rn 0

0 1 2 2 0

1 2 0 0 2 0 0 3 0 0 4 0

0 4 0 0 4 0 0 4 0

EC. I .I

und, a wa process h

COMPUTER MODELS 203

d b was increased by 2. When b n w contains twice the original

e are usin I manner-it serves only as a evic for ettin the prog am t o an earlier instruction. We ould have simp y de ned spe ial o erati n for this, but we wanted to eep hen mber of di eren ope ations to a minimum. From now on, we

ill u e the instr ctio ~ e.g., 'go(n ,' for this purpose-understanding

hat i coul be r plac d by a w0, w- (n seq ence.

E iden ly th instructi n number -1, 2, 3, ... -have no intrinsic igni cane ; the serv only to in icat the r lative position of instructions n th seq ence W will omi the fro now on, and indicate the 'jum " or "tra sfer" of th pro ess y arr ws (just as we omitted state arne in t e dia ram of chapter 2-5). Ou program is now represented y th diag am

to us, for it computes the wo f

S art

exit

204 COMPUTER MODELS SE . 11.3

The first part of the progra puts H(a) in b, and puts P(a) n c. Note how c is alternately set to 0 a d I, · nd ow b is in rease eac time the program succeeds in subtract ng 2 from a. he I st t ree i stru tions simply transfer the contents o b in o a o th· t we end p with H a) in register a. We indicate by bo es th bas c rep titiv "lo f the program-the parts that really do the ork.

11 .2 PROGRAM FOR A TURIN

We now take an importa e sh w that, g·ven ny uring machine T, we can construct pro ram mac ine T which is equi alen to T. The machine M Twill ha e fo r reg sters nam d s, , n, nd z Th registers s, m, and n corresp nd t the num ers s, m, nd whi h, i chapter I 0, describe the tape of t e m · chine T. he transform these three quanti ies i pre isely the arne man er a were transformed by the Turing tr· nsfo mati n T*(q, s, m, n . T eac state q; of T there will corresp nd a bloc of progra , a d the even of entering state q; will be repr sente by M T e ecut ng t program. Each state q; is a soci· ted ith t o1

S;0 , d;o) and (q;, I, q;J, sii, d;1 . Figure 11.2-1 is the compl te pr gra correspo

Turing machine. We illustrat the case n wh ch d; hap ens o be "left' and dii happens to be "right.' T e diagram has co ment expl inin the function of each block of prog am. Whe the machine is star ed, a the top of the appropriate sta e pro ram one ust ave he p oper initia value of s, m, and n in registe s s, m, and n. T e z r giste is u ed o ly fo temporary storage during th cal ulati n of the tran for we assume wand z contain ze o at he st· rt.

11 .3 THE NOTIONS OF PROG AMMING LANGUAGES AND COM ILERS

Observe that the progra of ig. 11.2-1 is compo ed essenti· lly o copies of the programs of ou three pre ious examples. We seth m, i effect, as "subroutines"-m anin ful units f pr gra larg r th n th basic machine operations.

If we wish, we can be mo e for a! a out could define formally the express ons that

twe assume T is a two-symbo machine. t be c nsiste t wit the ormul· tion chapter 10.

EC. 11.3 COMPUTER MODELS 205

(As uming d;1 ="right")

Exit to q.

' 10

Exi

achine state q;.

a d si ilarl for he o her c mm nts. (The details of this could become v ry c mple .) Then e co ld w ite o r pr gram as follows, in a rather

ore e pres ive I· ngua e:

206 COMPUTER MODELS

Start: if s

(I) z +- 2n z+-z+so n +- z z +- H(m) s +- P(m)

m+-z Go to q;0

One can rather easily im velop) which could take this

to I else

gram of Fig. 11.2-1. Such a m translation from the more sop,hi:stH~a ation language of the basic a detailed exposition of what ming, there are many loose to show something of what

basic operations not really

CJ, E), and the like, and m

machines but are usually

will finally execute the resu

11 .4 A SIMPLE UNIVERSAL

If we choose T to be a u"''·'or·c<>J tion 11.2 gives us a program machine. Therefore, in so by a computer which has on!

If we begin with a certai

without the operation @].

11.4

to

COMPUTER MODELS 207

CJ and E). different operation type

!contents of a is not zero, tract I and go to the nth

on.

lconte:nts of a is zero, just on to the next instruction.

observe that with them

nes) which have the effect of [QJ

am computer need? We five registers s, m, n, w, and z. A m<lcjhine works will show that the

information it contains into

rewritten to eliminate the

could be eliminated at the

~"Jump to the nth instruc

universal program computer

208 COMPUTER MODELS

with a few simple operation quite a remarkable fact, and we simple systems which are cap Later on (in chapter 14) we really necessary; this involves s me

11.5 THE EQUIVALENCE OF PRCJGRAIM

WITH GENERAL-RECU '""'"T'·"'"''"

We will now show that a puted by a program machine u this, it is convenient to define nothing new is involved here routines) made up of the previo

"Copy" a-+

- Start

and:

0 "Jump unless equal"

tMost of today's widely available gerler:ll-tJur!'o:>e

'"index registers," and have also ope

Programmers of these machines will universal even without using their rna only in principle. since the registers o memory is needed for progra'l1

SEC I 1.5

We

COMPUTER MODELS 209

As an exercise one

irectly in terms of [] and

rimitive-recursion scheme. the values of two func-

s of program that, when will end up with the

wish to write a program register c/J, of the func-

rite a program to do this, nd y in registers x and y. of if;(x}, by making use of imilarly, let ® be a pro

the value of x(¢,u,x) in rs involved. Then the

210 COMPUTER MODELS

program

computes the value of ¢(y, through precisely the steps g

slightly awkward fashion to

0 need be involved here.

Observe that the co1mput2~tiom tbroc:eci:lds r,..,lr"""r·tll 0 toy directly, rather than h<>l.rln.>~<>·l-rlc

it unnecessary to keep a nulnlJer The operation a ~ b(n)

the machine to repeat the 1.t ~r2ttlcln cursion the number, y, of tion. But the same resou this, consider ¢(t) = 0,"

and suppose that ¢(t) is ori):nitiv~~

already have a program takes a number t from ~on;e<l,·~

Then, "the least t for which

and this is all we need to s function.t It follows from a

it can be computed by a

and EJ. tFor ~ 1 [</>(t) = k], simply

COMPUTER MODELS 211

concerning the relations bee know (from section 10.3)

unng-cc,miiDUitatlle, it is general-recursive. We dleJmo,nlstratdd that, if it is general-recursive, it is

[mJgr·lan1-c<i>rrtptiter has already been shown

1.7

In

it is program-computer if a function is program

llnP•-c:•imputable; this will close the t. This will drop out of

the reader should be able to diagram in Fig. 11.8-1 at

@], [],and B know that @], CJ, and EJ

at with @], [], and B we

sufficient set.

if X ,.0 Q

primitive-recursive functions "equality" operations. We

212 COMPUTER MODELS

also obtained the generai-rP~nh:ive turlctltlons. tinguish between these lesser terms of the computer operat to look, not just at the im;tnl¢tlortsl, gether, and it is there that we

The outstanding feature one knows in advance (by the primitive-recursive scheme) with the J.l operator, one doe done and the computation ter·tnimat~<i. program of 11.5 for primitive rec:ur:$10n. y "" 0) the computation en

Since the terminating co and since u begins at 0, we precisely y times. Since we it more directly by defining a

"Repeat" RPT a: [m, n]

It is understood that the n operation, so that it will

executing the \RPT\

range of a \RPT\

that instruction numbers

t Otherwise it would suffice to h (by problem 10.4-2) this won't do. cedure to tell whether the iteration

I But see the remark at the end

COMPUTER MODELS 213

lex.actllv what it should and leaves under the same conditions primitive-recursive function

the range of the ffi

write a program to realize

uting ¢ = ¢(0). The trick d then the program RPT ¢:

~klp~>ed, with no effect on u 0 the next two RPT's will

if¢ is not zero, the RPT ¢:

214 COMPUTER MODELS

[u0,u') program will end up of cp. Then t will be increased finally the whole process will Thus t will be increased until suit: Any general recursive fu,nrtion

puter using only operations @], to lie in its own range. In fact (necessarily) occurs at the end 0 to I times.

It should be mentioned t concerning what the RPT has could not be an instruction in reason is that the program cOlrrlPIUti~I! of how many RPT's remain to ticular amount of storage """"'c:" operations require infinite rP-~ri<ltPr.;; be treated differently from sidered. Under the tight res:tn~ti(Jn~ away with a finite register as~;oc:iat:etl to be greater than I, in a sel~-contflinir1g general.

PROBLEM 11.7-1. Many

G.~.EJ, (a, and

bases. Which combi11atiorjs Invent some other op,enlti<n)s

PROBLEM 11.7-2. Recu quickly. Consider the se<llue:ncel RPT and successor operatio

Po(a) = a'

Show that P0 (a) = a + P3(a) in ordinary function

Which is larger:

,048.

COMPUTER MODELS 215

ve-recursive. This can be than any primitive-recursive ments about programs for

? Consider the program for

that when n is larger than sense be growing faster than

be difficult, but it is worth does grow fast; the known decimal for F(5). For, while

operations,

of precisely what we have I. We have shown in sec-

216 COMPUTER MODELS

tion 10.4 that any Turing-m general-recursive function and can be computed by a program shows directly that Turing-m gram machines. In chapter 14 the diagram. Completion of computability discussed here despite their apparent differen,cds

1.8

a

PA Ill N TEMS

IPULATION

·ve procedure and identified that the notion of effective

of behavior, and sta~erne1~ts in the language.

rily on the machine part of such ram machines, and other

rn our attention toward the methods will be based on

P...!tnrP·""IiCln"'' or "enunciations" of a may seem to be, are in the

in some finite alphabet. ical system is ultimately, in

how some strings of symbols may

obvious but probably unto show the equivalence of with the very sharply re-

was able to reduce his broadest a family of astoundingly

ations. We can summarize .-.,1111011•,

1

in section 5 .2, of Turing's

symbols which or logical (or

219

220 THE

mathematical) sy of one of Post's "qa11011jcal

When we see how simple seem rash; but this and the tnllcn.vui'o

the equivalence of these with ability.

In studying Post's system tions," which specify how ones. These productions are of rules we imagined for mach nes not imperative statements at but are de;rm,isk,ive st2ttem~:nts. tion says how, from one sta such a form, one may derive a ical system, which is a set statements, does not even descj1nbe of a set of strings by (recursi The interesting thing is how needing the idea of a machi show that the notion of a of canonical system (under more general, idea of set c harder to see within the fra

12.0.1 PlanofPartlll

In this chapter we example. The formulation is ~rien,~e<j, what happens to the symbols does not appear explicitly. vert what are, to begin with, representations of the behav· ing that encompasses thew the loop and show that all ou Chapter 13 proves the beauti canonical systems in general Chapter 14 ties up a number lence of Post systems with Tu simple proof of the unsol problem," and also a state-(but not simplest) universal

SEC. 2.0.1

from the study of Euclid's with a set of "axioms"use we believe them to be Along with the axioms, we ce"; these tell us precisely

eorems"- from the axioms

he problem of formulating fectly clear what were the

tieth century, the logical er axiomatic systems in

overed, there had always axioms might be possibly soning process, so that it ical system.

foundations of mathematics discb·verd! that "common-sense reason

always a sufficiently reliable undations. The study of the

to results that were selfintuitive. A number of

ions led to genuinely parar~<tsonalble supposition that one could

in terms of common-sense The notion of effective pro-

ms: some of his "axioms" we concerning the interchange

inference concerning permissible the statements concerning equal

spe:cifjcally geometric content, and tried

222 THE SYMBOL-MANIPULA

cedure was old in mathem effective unsolvability, is new to mathematics; w their possibility ha~ been sth>pechl:d necessary to examme m formalize all the steps of tated in the early 1900's resulting theories- of the formal, and indeed became feet, a mathematical "metamathematics. "t

12.2 EFFECTIVE COMPUTA PREREQUISITE FOR

Before going into detail, sion will lead back to the t key point is this: We have to is really "all there"- that dubious "intuitive" steps. procedure to test whether an composed of deductive steps I nP•rrr>litt·,.rJ

Now "theorems" are obta· deduced theorems, and axio is an effective way to verify supported by correct applicatlim1s

To make this more preci A logistic system L is a

defined below. An alphabet is a finite set

assumed that all "strings" alphabet A= (a 1,a 2 , ••• ,a

An axiom is a finite strin terns with finite sets of axiom

A rule of inference is an ,.fif.,.rltivlp,Jv cd,miDU~a

of n + I strings. An R fu and 0 (for "false"). If R(s;

tNotably Post. See his autohiool·,mhirdl tThere are several "branc metlarrtatl~~nlatic~,

one. Others are concerned with desc~i)Jtic'n~, formal systems and their decidabil systems, models, etc.

12.2

vab fro

SYSTEMS OF POST 223

atively we may write

(R)

ference. Often, in a logistic me:dillltelv derivable from S 1, ... Sn"

can justify it. Since each I do, since there are only a

(Show that

ch a way that there is an · ate. This will be the

ve, since once has to axiom is only a finite

if s K is immediately deriv-a finite number of such

of R's. Each R test is, we go on to s K-l and do

t s be sK_ 1.). Clearly a process will confirm or

ure is effective, since it is

ber of restrictions. One We can do this, if we

an arbitrary string is an nd the notion of proof is


schema"- a rule which says an axiom." One might inference. (Again one could fective schema to determine ~h1etl1er ference at each step.) For ou quate. Shortly we shall show without any real loss of gen rule of inference of a kind time! At that point, we will a step-by-step process.

12.3

The proof-checking nrc\rf!''"

an alleged proof it gives pej:fec:tly cedure to decide whether a slightly different quei>tH>nj whether it is really a the<)re~--system, for it. The answer for which such procedures In any logistic system it is search through all possible nir<)OIS+-a theorem, this fact will be lconhrmed.l theorem is not really a tht~oi·tjno.,

To elaborate on this a I" ate all proofs-and thus all t)H:oretn use a procedure that ge alleged proof, and tests above. How does one ge11erlate a procedure that generates First generate all one-letter Next, generate all two-letter the preViOUS State (i.e., One-fJielJ[efl CTrlniCJC I

every letter of the alphabet. N ex gel11erat!! by taking each two-letter st letter of the alphabet. Clea way.

PROBLEM all finite strings in the separated by some speci subject to a constraint (

ION SYSTEMS OF POST 225

istinguish the strings the rnaworking on.

machine that

strings, one can modify After each step of the

tter string. Now let us cut here n - I ways to do this.) act f the main procedure, then

gs! In fact, we have done exactly one way. So this

(The original procedure at after each pair of strings pair into two parts, in all , depending on where the

this) all possible sequences a string is cut into two

I")ossiiblle cuts of the first part into ally generated string, this

In fact for an original ~eq1ue1nct~s of strings that can be

length n can be cut into There is a trivial proof.

t generates all finite seThere are many other

e we have given; ours has the unim-sec}U(:nc:¢ is generated exactly once.

rve as a Godel numbering case, as each sequence of test to see whether it is a

tht:ot·errJ-c~ndic~ate. If the string is really a ime, yield a proof of it! But if

terminate. Hence we have not have a "theoremhood ortly, that in general there

have decision procedures. formulation of Euclid-


ean geometry has this logic of deduction for sim decision procedure; in fact be made into a decision or<:ke:dute. tiona! logic with the quanti and "there exists an x such lations~is not, in general, O~<C!Olap of cases where the decision hr,,hl.l>m

Wang [ 1962] for some of th !ems are not decidable. T quite prepared to study Rogers [ 1966]. Of course, and presenting one, as in Euclidean geometry, does n method for proving tht!orerr~s of a decision problem usua cases of practical interest.

12.4 POST'S PRODUCTIONS. ~ANd<NI<:A~ FOR RULES OF INFEREN

In 12.2 we defined a ru whether a string s can be d¢1du<;e(j required the test to be effect to be effective. But we did n one might still make a rule of i upon some understanding o wh have some rule of inference strings as asserting things a strings might, for example, strings in chapter 4~the l"r,,~.,n~.

represent the "regular sets, used references to these unrll>rot,-,rl.rl

To avoid such dangerou rules of inference that con of symbols within string printed on a page~ and we ule o us, for the present, to direct our att.entlo·n domain of "syntax"~q exjpre:ssi1b analysed~ rather than the meanings of expressions.

To make it plausible t of success, we will paraph

e situation of an imaginary m~mllDUilate symbolic mathematical

rk with finite strings of axioms, and a finite set of

,JPT';r";lnn the validity of an alleged then, he will be confronted Is

roofs he has already verified

He must use one of the to apply each rule system-

will further suppose that certain set of talents:

string, and

ng certain and deleting

needed to verify proofs

and rearrange its parts a "production." Rather

tart with a few examples of on productions. Then,

fectly clear what is meant

is really necessary is the ability act accordingly. This parallels

hine to examine one square of its

228 THE SYMBOL-MANIPULATI

EXAMPLE J: THE EVEN

Alphabet: The ngle Axiom: The stri Production: If a is the string $11 rule of inference

It is evident that the theo

II,

EXAMPLE 2: THE

ExAMPLE 3: THE

The "palindromes" are forwards, like cabac or ab.c'Pt,cb.al. already a palindrome, it wi ain beginning and end. Also cl arly, we ing in this way out from t

PROBLEM 12.4-1. gives nothing else.)

PROBLEM 12.4-2. Th this is a set of strings th by a finite-state machine axioms but append the dr<Jdt1di

ngs

11 11 + 11

11 + 11

II II 11

$ $

SYSTEMS OF POST 229

rems are all true statements sertlteJnct:s like

the theorems will resemble

,I + $2 = $31 I + $2J = $3J

a theorem that consists of llnotlller string $ 2 , then an '= ', and

theorem that consists of the hen'=', then $ 3 , and finally ve the theorem that means

11 111 111 Ill

11 Ill 1111 11111

axiom by 1r I

by 11"2

by 11"2

axiom by 11"2

by 11"2

by 11"(

th••r~rl>m, in such a system.

230 THE SYMBOL-MANIPULATI N SYS EMS F PO T SE . 12.

PROBLEM 12.4-4. Desig a s stem whos theo ems statements involving bot addi ion nd ultipl catio difficult at this point but wi I bee sier a ter fu ther xamp es.

arit meti T is may b

EXAMPLE 5: WELL-FORME ST

In 4.2.2 we defined the s t of

namely the strings like

(), ( () ), ( () () ),

REN HESE

d st ings of p

()) (),

in which each left parenthesi has rna chin righ -han rna e. obtain all and only such strin s as hear ms b the s ste :

$ ($) $ $$ $1( $2-$1$2

For example, to derive the st ing ( ) ( ())):

() ( ())

(( )(()) ((())(( )) (( (()))

PROBLEM 12.4-5. Pro e ( () ) ( ( ( ())) in th s syst m.

a xi by by by by

PROBLEM 12.4-6. Con ider t e sa production:

abet and a iom ith t

e ca

Prove that this system g nerat s all and nly t e wei -form d pa enthe is strings. Note: a$ is allo ed to represent an empt , or· null" string, sot at

() ~ () ()is permitted, f r exa pie.

12.5 DEFINITIONS OF PROD CANONICAL SYSTEM

12.4, every production form

eac exa pie of f the gene a!

ANTECE ENT

go$Igi$2 ... $.

s c. 12.5

ith t

h; i II, a

xed tring; go and g n

he h's can be null. an " rbit ary" or " ariable" string, e nu I.

o be repla ed b a c rtain one of the

ake f rex mple the p oduc ion

f om e amp t e ass gnm

ARK

$1

e4 o

go gl g2

$2 $3

We

g3 ho hi

$!=$], $3 = $3,

$1 X $ 2 = $3$2

the form above by making

nul h2 ' = ,

nul h3 null '1 h4 null

o (or mor ) oft e $[ scan bet e same$;, as in the produchere $4 = $2 = $2. his reaks up, diagrammatically, as:

$1 X $2 = I I I I

go$I gi $2 g2

CO SEQUENT

I X $2 = $3 $2 I I I I I

hi $2 h2 $3h3$2h4

Post's mo t gene a! fo mulaf on a! owed ach production to have several a teced nts. his is discu sed i 13.2 and e prefer not to introduce this c mpli ation ere; i 13.2 we sh w th t the ore general form is equivalent,

a sen e, to t e spe ial si gle-a teced nt fo ms used here. in P st's ost g neral form latio s, he allowed two of the $'s in

e ant ceden to b the s me. his eant that the rule of inference would a ply o ly to a stri g (th orem in w ich t ere was an exact repetition of s me ( ariab e) su -strin in t o pi ces i the antecedent. We prefer to p ohibi ante eden s of his f rm, ot be ause we want to restrict the g neral ty of he sy terns, but b caus it w uld run counter to our intuitive P cture f wh t ought to e per itte as el mentary, unitary actions. The r cogni ion o the i en tit oft o arb 'traril long strings ought to have to b don by n ite ative proc ss; o herwi e it violates Turing's dictum ( ee 5.3 abo t what can e "se n at glan e" and what requires a multis age p ocess. 1

232 THE SYMBOL-MANIPULATI

DEFINITIONS

A production is a string-tlramsr~r·mi.nlg above, or (more generally) of

A canonical system is a

(1) an alphabet A (2) some axioms (s (3) some productions wlw~•el<:orts~;ant

12.6 CANONICAL SYSTEMS OF TURING MACHINES

Now we can show how ductions, can be arranged system seems permissive ra nothing that corresponds to an obvious mechanism that l:li<:tat~s there is no notion of time steps in a proof of a theorem proofs of a theorem, one sequence as a process-con tricks-mainly the use of we can, indeed, embed the formal system with only constructing a formal sy "simulates" the activity of a

EXAMPLE 6: PRODUCT!

Let (sJ.s 2 , ••• ,s,) be the alph certain Turing machine T. sequence of symbols

where n, is the length oft the complete state of the Tu (I) the current internal sta is located on the tape. We tape-representing string, b appropriate place, e.g., by uJiritinal

.. 'S nt

ine is in state q;; it is scanhas on it the letter seon the machine's tape.

tion by a set of Post pro-

t to proceed from any complete

to that represented by

reach (i,j, k) triple:

ml<tchmlt!-that is, when the symbol f the tape. But we can make the

r~rHesel11tatiqn by adjoining the produc-

machine comes to an end machine's computation as

svJm[JU1s. with provision for lengthe the following assertion.

containing one q; symbol will be precisely the

machine, if started in theorem of the system

eps of the proof will be exactly (We include the tape-extending

tion.) fa string contains only one to it (Verify this!), and the


result produceS a string that ret)re:seJiltS chine's computation because the system of productions.

PROBLEM 12.6-1. Why out of the first (right) Pr<>dtl¢tion:s1

PROBLEM 12.6-2. of 6.1.1 (p. 120).

PROBLEM 12.6-3. Sur>po:~e single production

xy is a theorem of the sy More precisely, describe can be derived, using only

EXAMPLE 7: A CANO

(in the form of unary (n + 1 )2

= n 2 + (2n + to the next by adding the corresoonding

Alphabet:

Axiom: 1

Production

This generates, in sequence, t

1P

111 p 1

11111P1111

1111111P111 1111

111111111P1 1111 11111111

In a sense, the square num used to separate two quan number to be used; on the stages (which is also the that one could use more puncl:ua.t01rs auxiliary quantities that one In the next example, we will

c. 12.

in at this point, how one could hine, of the kind described in

punctuation letter for each are really needed? The solu-

that there is no system of probers (in unary notation) which

alp,halbe~-th~t is, which has no extra punc-

the previous section. We se only theorems are the

und a system which generint·~)f]ma~icln but does not produce it

duction

have the "working

of the

the P above, are necesoduce sets of theorems of

236 THE SYMBOL-MANIPULAT

theoretical interest. Let us deal with this fact.

Suppose that we are inter~st:ed are expressed in a certain alj1h~1bt~t system.) Suppose that M' Some theorems of M' may additional letters.

DEFINITION

If the theorems of Mare rJre·cisclv letters of A, then we say Post canonical system, over A.

In example 7 of section 1 system whose theorems are the work, while the second Jlm)dllf:;tJton it converts a string with a to release the result-a squa (because it contains no P). I one in the extension alpha transformed and to protect rectly transformed. The ne such a computation.

PROBLEM 12.7-1. repeated strings, e.g., section 12.4, example 3, Can this be done here? original letters.

This example shows h plicated behavior. It is an e*ttens1f_} alphabet '1'. We have ad system otherwise generates

B. B. .B . B

B. .D .B

. . CD C . . D

etc. -!-

.DB. . C. . D. B.

. C .. D. B . c. D B.

CD B . C .. D B c. D B C .. DB . C. D.

D. B. C .. D. B . . C D. . B

D .. B

END

C. D B C. DB.

C. D B. .. C. D. B .

. CD. . B. C ... D . . B . . C .. D. B. .. C. D. . B C .. DB. .C. D. B. .. CD. B. .. D. B.

B. . B.-----.

. B. and I· ... ·I

.. B

5 is prime

by the

one can design canonical coimj:J•IIcat~:d process. We have already

new auxiliary letter for each e for each of the instructhe instructions of a pro

let the registers of the malmacl1lrte whose instructions are of

subtract


By section 11.4, this is a un simulate this program ""''""lllv follows:

Axiom: I 1R 1 11 ...

which is understood to mea given unary numbers in its re!dst:ed

Ij$1

If Ij is "If Rk r!

else go to Ir,"

This system is truly "monog~nic" never possible for more than the case of an addition instrjuc:tic>(l begins with the letter Ij. In are two productions beginn followed either by a '1' or by R k

apply. To release the final instruction and the answer i duction

which will release the unary proved form of this theo

PROBLEM 12.8-1.

PROBLEM 12.8-2. the set of strings (as an lexlten$ion) directly from the st~tte··tntns>iti•Jr the regular-expression

PROBLEM 12.8-3.

s th

TION SYSTEMS OF POST 239

1com~>l!t•~r J1ro~traJ!nming languages are based on Post productions. In particu

ricted use of occurrences of the COMIT, developed by Victor

lin:guistic analysis. Following this came ted languages embedded in the

[1966], and Teitelman

13 POST'S NORMA THEORE

FO M

13.0 INTRODUCTION

The theorem proved in Post's [ 1943] paper. I feel in mathematics: Any formal system with a single axiom an

Post's proof of the tht~ordm considerably simpler, beca things happen. Some clarity some is perhaps lost because To make up for this, we illustrate should help the reader see intl.Iitivellv

We will state and prove strength, then give some new in the proofs. In the subseq tween these and the results o

13.1 THE NORMAL-FORM TH SINGLE-ANTECEDENT PRiODUdTIOtd

THEOREM 13.1

Given a Post canonical s the form

240

.I

a

NORMAL-FORM THEREM 241

P* whose productions all

That is, those theorems of al alphabet A of P will be

) we can foresee the followthe apparent limitation of "tial letters of a theorem.

, but how can they check of g2 preceded by a copy of , we have to (1) determine that is, when a string con

... ,gn in that order, and n the discovered g;'s and

pqsitior:~s of the consequent forms ho,ht, ... ,hm. Further

nt form is satisfied in

of "rotating" strings so d to the front.

letters for the auxiliary will begin by supposing

vatttiJitlJtc and that theorems of P by the single letter T.

of P* for a particular

IPnJdtktitonl is actually

Taa

bb

a a

c

(1r-example)

(S)

242 POST'S NORMAL-FORM TH

(which does fit the antecedent The result should be

We begin by providing llV.lll'•"l

this antecedent form. We will

Alphabet: letters of A, an

Productions: Tab$ - $

This system of productions h the form Tab $1 cb$2b$3, string T3 • To see why this string. The following strings

so that the T symbol has r (only) the production T1e$

SEC. 3.1.1

de).

has

c b g b

c b g b c

AL-FORM THEOREM 243

Doomed strings, can never produce T2•

Doomed strings,

rtn,o,.,,,[,,. that, to produce T3

the , in order, and that each

svrrlb•:ll--1-re:aulres that the T symbol en-$ubst*ntg t form. Thus, for in-

"doomed" in that its T1

become upgraded to T2 •

rm, there is a route which form, production of

perfectly general method:

know how to get around

244 POST'S NORMAL-FORM THE

the apparent limitations of th rotation trick), and we know antecedent form. We still ha

consequent form.

REMARK

If the input string has the system will produce T3 by di~

will have three interpretation

Tabcbcb!!a,

$1 $3

These should eventually res

The only trouble with th antecedents, it destroys the information is needed to con tion, we will use the more introduce an array of new

and a new (and final) system

This system is like the nrf>vit"'IIIIO

mation instead of deleting it.

we obtain, following only t

SEC. 3.1.1 13 .. 2

b Az a

s:

(P

'n)

T's

a b

a b

a b a b A Tt

b b

a b

a a b

Az a

A2 a

A2 a

ban $2

AL-FORM THEOREM 245

Tz

jTz ala b AL

a b AL A~ T2

A2 a

Q AL

Q AL A~ A2 a

(S')

ke copies of the $ strings . That form itself will be

246 POST'S NORMAL-FORMAL T EORE f.t SEC. 3.1.2

,, ; I

SE~ . 13.1 3 POST'S r ORMAL-FORM THEOREM 247

set up by the production an ~eve tual 1~,

I Q$- $h v;1h 1 v;2 •• Jl;mh zy (P*- onti ued) Yaa ~ffil bb v.. ~a ~~ ~ cA !A}AiAJA~Z

I wl ere" e hav ma1 ked v here

in which the h's are the consta1 t stri gs o the< onse< uent and t he V s are I' he$ tring ~ hav been copied in. Now our

new letters indicating where t~e co rresp tmdin ~ $'s are t p go-1--that is, a W( rk is done exce pt fo rem ving the s affol ~ing-the A's and V's and

copy of$; is to be inserted at e ch o curn nee c fits ~. T he ne w lett rs Z y nd 2 used in th con true tion. Weu eat ick based on the following

and Y will be used later. Whe n this prod uctio is a jded 0 th< syst m of I ob serva ions: We an el min a e an A wh nit 1 eaches Z, for then its work

the last section and applied to he st ing 1 abefc bgbdt we c btain (usir g the IS jon e. Wee an el min a e a Jl once all tt e A's have passed it, for then its

h's of our original example) W< rk is done We will seth e Y t infc rm tl e V's that all the A's have pa ~sed t ~fOUl h. v e wi I not perm t y t p pas an A, but once all the A's

A!A}Aif' JA~a a~b1 V..aa ~czW" (S ") I

ha ~epa ~sed V, W" car com e up and c lim in ate the V. Thus the pro-du ction

Our trick will be to cause the A's t p "tr ckle' acre ss th stri1 g, w thou

changing their order. Whene ~er a f1 A~ passe a J; with the arne in de' Yx! ---+$ fxY (x ·n A)

(i), it will leave behind a copy of it sub5 cript letter x. T hus v hen II th< yv;~ -$ y (al I i = , n) (P*-continued) A's have passed all the V's th< re wi I be a of$; rext t fl Jl;!

' ... copy oeac ~~z~ ---+$ z xi A, i + 1,

The productions to do the trick ling a e: .. ,n

wi I des roy t he A son cont< ct wi hZ and he V's on contact with Y.

A~y$- $ A; (al xin 4, y i A) OJ ly wl en al the ~·sa d V' are ~one can come to stand just to the X let of 2

(a I i = n) , and we c elebn te th s con pleti Jn of the whole process with the

' ... fin al pr< ducti on of ours stem

which lets the A's pass over lc wer-e ase le tters,

I A~ Jj$ I-+ $ ~ A~ (if o;ej YZ -~ Tj (P*-completed)

so that the A's can skip over 1 on-n atchi rg v· , anc (P*- cont inued) ) A] plyin gall his tc OUT< xam1 le, st ing .S resul s finally in

\A~ V;$- $V;x i (all inA 'i = 1' ... n)\ 1 a a de be fa 'pdcc X

which leaves a copy of x to tt e rigt toft e v;. We~ )sow II ne< d p~ OBLI M 1 .1-1. Recor struct this I roof sing productions g$ --+ $h in

y~ -+$Y wh ich ne ther g nor h have lnore t ~an tv o lett~ rs.

z - sz p~ OBLI M 1 .1-2. Show that nly t ~o a1 xiliary letters are needed in pn ving t heore In 13.1 -1. In fact, c nly o e is r eeded(!) but this is very much ha der tc provt and r quire sa dif erent met he d for proving the theorem.

to allow strings to rotate aro nd. Applying these producti< ns t< our exam pies ring S" yi elds gre t 13. 1.3 ... omp eting he pr :>of

many routes and paths of g en era ted s rings but there is 01 ly 01 e fin I

result. Typical strings in the proc< ss, as the A 'smi rate o the right are: Ther e ren a in few loos end In he p oof. First, note that we ha en't eally prod ced: legit mate exter sion P* of P because P*, so far,

Y A!A}Ai" ~A~~ a~b b V.. at ~c2 do s not prod uce a flY st1 ings lvith ( nly I wer- ase letters. (Proof of this:

YA!A}Ait A~a 3 cA ~ bb V..t a~cl7 Th axic mso p C< ntain the pper case etter T. Every production with an uppe -case Jette in it ant< cede1 t alsc has one in its consequent. So,

YA!A}aal 3 dcb ~ib v.. aa~ flee A A~Z by indu< tion, every prod uced strin! has ~n UI per-case letter.) Now the

Yaa~dcbl A!V.. If a a A 1 Ai J; dec; ~A~~ -- str ngs \ e wa nt to "det ch" ~re tl ose t hat t egin with T, because any

I

248 POST'S NORMAL-FORM

string T$ is an assertion that tempting to introduce the pro T, but this won't work. The lower-case theorems, name because the present system In fact, we have to conclude avoid this. The cure: begin al that B = b 1 , b2, .. . , b, is a 2 , ••• , a, were really new of P by the productions

which do not allow the b's be converted to b's prematu and not to spurious pure b

PROBLEM. Prove this.

What if the original simply carry out the whole using entirely new sets of a mon. Then the P-prodnr-1·'"'''c o·p~rate in<iet:l~nder}tly, common T that allows any PttJrc•d~tctioi(l the operation of other P-iPr<J<Iitllc1:Iqns.

We need one more tht~or!;:m systems of Post, in their tensions. We have to ac~~<IHmt

whose antecedent concerns ample, in logic one often h and another theorem of the form says, in the language of prodluctwps,

should be a production of of saying that the pr<JdtlCtttqn section we will prove a mo

ductions, unless one introduces not erroneously broken. 1

we shall see later, because ems and machines.

of a Post system, in which a (theo.renris) to form a new theo

uction has the form

(7r)

and all the $'s are variThe meaning is that, if

tecedent conditions, then uent form, which may

argued that any operation l:l'c•cotul[llted for by some such pro-

this form can be replaced , incidentally, that this

is required. As you will forced upon us.

uPnP.rnJ multi-antecedent kind, we has only single-antecedent

¢a,rzm1{Co'll extension of P. (Then r-hnctrl''"t a normal extension P**

to be lower-case) and let of the new system P*

ioms now, anyway, since be no possibility of their

Q.:<Hre$1JOn~ing to each production of

250 POST'S NORMAL-FORM THE

the form 1r belonging to the lowing monstrously comr,licat~:d o·r<l>,ducu,on:

B$'t,B ..

What does this do? The ar:l1:ec{:qetat ahetmp~s if it contains, sandwiched bet~c~en B's, $ul)stt~in_gs forms

that is, to see if the proper i"r u:lredt.eh·ts somewhere among a string ofltht:or~mts

If this happens, the proper cp11secjuent adjoined to the string:

so that in the future the new I thc~o~em

axiom; that, after all, is what That is the plan, anyway.

monstrous production requi order-an undesirable restri is more serious. We have to correct in that the strings a~~agr1ejd

theorems and axioms. The it may run from the beJ~mm~tg end of a different P string. contains one or more B's · designed to make it easy production appends the cmt~aten1{!t1 form

(Observe also that the conse~~ut:n~ production cannot operate

251

erasing lower-case letters. If the inner X gets across

aRJperld~tge, and the system reverts thelon~ml appended to the theorem

aR~ecedc:nt components occur introduce a distinct proantecedents- this would more elegant solution is

for permuted productions?

mecpamism by which P strings (namely, B's and X's). To do

c~c:ckipg for B's, by adding the

stem that is a sort of analogue this by using the methods de-

inrw\rlation is that instead of applying we will apply it to the

uctions themselves as e which works with the

co1jnput~ wha that other system does. with descriptions of Post sys

"uu"'"-"='"-·'are needed.

252 POST'S NORMAL-FORM TH

THEOREM 13.3-1

There exists a certain sys Given any other canonical s AQfor U such that the system of Q in the following sense. and b be new letters, and e representing aj by j a's fo AQ,,that have only letters a theorems of Q.

To prove this, we have to to construct the axiom AQ.

By the theorems proved e has a normal extension P. <1> 1 , ••• , <~'s and productions P and let A, C, S, and T be with the axiom

Observe that this axiom is a k;o,mr>lle1te dlesc:nr1t can reconstruct all the axiom

Next, consider the pr<)dtlcjtJIOn

This production looks (in th "production" of the form form S$s$S and attempts accord with the "prod thing to do only (I) if it is and (2) if none of the strin tain any upper-case letters. theorem of the system P theorem- (or axiom-) list rer>lreser1ted following productions chec

The trick is this: lower-case letters

253

both strings will vanish, ·n also the production

with the new theo-se

an extra copy of it at the end rlelea~;¢d by the production

uction just so that the I the01ren1s of the system, just in

among the productions

·ons used above depend e)l,co<jr.ng mentioned in the

rs re-

the universal system to the number of productions to "do the work," two to

the pure

that has only one upper-case Can you do it with still only ably minimal, in some sense,

with only normal productters or productions.

universal Turing machine, ut the strings produced by U

254 POST'S NORMAL-FORM T EOR Ef.i SEC. I 3.3

as a function of the give a xi om. Consid r the pros pect o f deve loping a

theory of computability on th is ba is as comp red ith t e Tu ring r

recursive-function basis.

PROBLEM 13.3-5. We aver ot qu 'te pre ved e eryth ng eli imed ·n th ep-

rem 13.3-1. We have pro edit or an ~non palsy tern .N ow pro ~e it fpr

the original, general syste mQ. he pr esent ysterr , as d scrib\ d, wi I relea se

the theorems of P (which nclud thos ofQ but a! o son e oth rs). 0 co n-

plete the proof requires a light lv mor com licate d rete se sy s em.

PROBLEM 13.3-6. We have ever allow d an antec dent .vith dou le 1 4- v ERY sn .,PL E B ~SE s occurrence of a $;. As n pted i the emar s of ectior 12.5 Post allow d Fl ~R I ~Oft .PU trAB ILl~ ry this, but we feel that it is oten irely i 11 the s pirit o f the f nite-s ate a proal h.

In any case, show that f pr an syst\ m wi h thi5 mon a! pr ducti bn I

gene I

permitted, there are cano nical e xtensi ons of our n ore c nserv ative in d. In I

particular, begin by sho .ving hat w e can simul te th e effe t of he p o-I

duction $10$111 ~- ~ 10$ J

I

by an extension that do s not use < ouble $;'s i p its : ntece ent. Use he I 14.1 Ut IVER AL PI OGR~ MM CHI" ES )·

T method, as in section I .1.2 w TH T\ flo Rl GISTE RS

) In s ction 11.1 we intro uced "pr grarr machines" which could

NOTES COl npute any ecun ive ft nctio n by exect ting programs, made up of the

I tw< ope ation s bel )W, 0 n the cont nts < f re~ isters- n urn her-containing

I. For example, there is no ca onica syste m for he th orem of th pro1 ositio nat i eel s.

calculus, in its conventiona forrr , that does ot us at le st on exte sion et- [£] ¥\ddt ter. Or so I believe, but I aver ot be\ n abll to re onstr uct w at I t rink as j nity o the num erin register a, and go

one t inst uctic n. a proof of this.

2. See note I of chapter 12. ~D fthe num erin a is n ot zer o, then subtract 1 rom a and go tc the 1ext i nstruction, other-

wise ~ o to he nt 1 inst uctio n.

Wes owe , in I 1.2, t at th ese o erati pns w orking on just five registers we e enc ugh 1 o cor struc an ( quivc lent fan Turing machine, and we ren arke in I 1.4 n at th s cou ld be done with just two registers; we will

j I ~ nO\ pro e thi . Bu tour purp ose is not nerel to reduce the concept of pro gram mad ine t am 'nimu m, b 1t to also use this result to obtain a nur 1ber < f oth erwis obsc ure t eore ns.

Wen call ( rom 11.4) hatt ne op ratio n~ i.e., put zero in register a, \I can be si lnulat ~d if ~e ha egist r w ' ve a !read ~ containing zero; then we 'I

can also sew (n) a a~ ~i struc tion. cor o ur pu rpose s her it is more COn' enier t to assume that we have

J

___ _..- ' 255

256 VERY SIMPLE BASES FOR C OMPU ABILI Y SE . 14.

(£], Ia -(n)\, and \go(n)\ at the s art. hen

will serve as a 0, and we can as ume that' e ha e

a' ~o(n)

These are the only operatic ns th t app ar in the d agra 'l of 11.2, ~ o the are shown to be all we need to imult te an v Tur ng mt chim. .

To reduce the machine o Fig. 11.2- to one w th ju t twc regr ters r

and s, we will "simulate" a 12 rger umber of virtui 1 reg sters b~ us·_ng a elementary fact about arithm tic-namely thi t the prim -factonz~ 10n of an integer is unique. Suppc se, f~ r ex:: ~ple that we ;vant to sr n~l~t z

three registers x, y, and z. We w II bepn b) plac n~ tl ~ number 2 3 5 in register r and zero in regi~ ter s. The impc rtant thmg IS that fn m th~ single number 2x3y5z one car recover tl e nu rbers ~·_y, and simply bw determining how many time~ the umb~r ca be ~IVIdt d by 2, 3, and ~ respectively. For example, f r = 144(, the x = 5, Y = 2, and z = For our purposes, we have only t( show hov we (an o tain he e ect c f the operations x 1 and x-, y 1 ~ nd y , anc z 1 ar d z-.

INCREMENTING

Suppose we want to incn ment x, that is, add nity o x. This mears thatwewanttoreplace2xy5zby X+Iy5z = 2·rjy5z. But this is tte same as doubling the number in r! S"mila ly, ircrem ntin! Y. ar d z s trebling and quintupling (res Jectively) t .1e co tents of r. And thrs s dor e by the programs of Fig. 14.1-1. , he fist lo( pin_ ach program c unts r down while counting sup tw ce ( o thre or f ve tunes) s fas ; the secor d loop transfers the contents o s ba( k int r.

I I

ct~v go

(1) t (2}

(3)

F"g. 14.1 1

~ j

II

SH. 14.1 VERY SIMPLE BAS S FOR COMPUTABILITY 257

DECREME 'ITING

Subt actinl! uni y fn m x is a little mon tricky, smce we have to

de ermi e wh ther is ze o; if is zero, w want to leave it unchanged and

do a ~ · If is nc t zen, the we fvant o ch< nge YY5z into 2x-I Y5z

th; tis, c ivide the c nten s ofr by tv o. S milarly, decrementing y and z is (cc nditi nal c n no bein~ zm) eqL ivale t to ~ividing by three and five. W illus rate in Fig 14.1 2 a p ogra[n for the 5 case.

1

'~ /' r:.- --..... r r' r - ~ t -"'"r r - ~~

~· / l {if z wo. "'''

r - .>.!. s '

(~'~ ' go go

Fig. 4.1-2

The I op tc the eft d )es the div sion, by repeated subtraction. If the divi~ion comes ?ut exact- that is, has no remai der-then the lower loop copes th quotient back "nto r. If the di ision was inexact (i.e., if z was zen), tht looi to the ri~ht fi st re tores the remainder (which at that mo nent s sto ed ir the state of th rna hine -i.e., the location in the pro ram) and then mult plies the uotit nt b the divisor, putting the rest It ( wl ich i~ the c rigin~ I con ents) back into

his i all~ e ha've to srow, Or cJ arJy we C< n do the same for the jive regi ters rentipned ·n 11.2. \\ e simply p'-'t in r the number 2m3n5aTII w

and use tre sa re te hniq es gi~en j st above. To build a program machir e equivaler t to hat of Fig. 11.2 I, we take that diagram and replace eacl of i s ind"vidu' I ins ructi ns bw a cppy c f the equivalent program stru ture love have ju t dev lope . Th s proves:

HEO EM 14.1-1

1 or my Tu ing machi e T here exist a p ogram machine M T with just two rr>giste s the t behaves he st me as T in the sense described in secttons I .I at d 11.2) when st rted with ero tn one register and 2a3m5n

in thr> othu. This ma hine uses only the ope at ion~ [J and G.

258 VERY SIMPLE BASES FOR

REMARK

Now that we know there registers, we can improve there are just four kinds of lin!;tn]ctiionsl: "subtract (conditional) 1 "$1Jbt:ral~t (1;ot1diltiopa1 In such a machine, we can o the two registers, so that we with simple productions

for addition, and

for the conditional n.

14.2 UNIVERSAL PRC:>GRAJM-Iiii~1CHIII\IE

ONE REGISTER

We can get an even stro division operations (by con then we can do everything w 14.1-1 shows that we need on operations "multiply by 2 (or conditionally upon whether t tion. The proof of theorem I by 2 (or by 3) and division by precisely the same as thee~ x (or y). Furthermore, by thlc:orem but these four operations, another exponent level. Th its own proof ( !) we obtain:

THEOREM 14.2-1

For any Turing machine one register that behaves the 22a3m5n • 3 in its register. Th plication and (conditional)

PROBLEM 14.2-1. V Theorem 14.2-1.

PROBLEM 14.2-2.

S FOR COMPUTABILITY 259

. Show that in each case we reJ),lacing two of them with one that af

e, in the machine M Tv." ··an adjoin excnange (r, s). In M}

by 2. We can even reduce for machine M T it is sufficient to

and exchange;

better result than theorem owever, from the point of

theorem 14.1-1 because sic, finite actions. On the tents of a register by two

ause the amount of work in the register, beyond

be held against theorem reiJre:sel~te:d as a binary string. It

~ur·pri.tsJ·ing power from the information by selecting a

effectively compute such inlhrlm<~tinon from it. Thus, in the

quadruple (m, n, a, z) of 2m3n5a7z. There is nothing

f Turing machines (section th~!msel~es to represent arbitrary

(10.3) of enumeration of n even more complex en.3 we showed that a single letters from an alphabet.

po1nt~:d out by Godel [ 1931], a rary list structure (see

corn~sp>onlUe:nc:el, inductively:


For example,

PROBLEM 14.3-1. yield different *-numbers, the list structure from the*

that the list structure can

(I)

{ K* = K (I)

K* = 2a*3b*

{ K* 2K (2)

K*

{K* = 3K

(3) a* K* = 32 5

The ideal of a Godel n because while that subject i number and the foundations to be a theory of mathemati theoretic formalisms, as G theory (and sentences are no

MPU

3*

c*7d ...

and properly interpreting clorresp<)n<1I~g

makes it possible to in without paradoxes; and impossibility of a non-contr~tdJctcJiry consistency, using this tech one can use numbering sche effect, avoiding number-t rem. The methods of Smul was strongly influenced by bolic expressions rather tha I, for example, in the proof those of section 10.7, or tirely, are ultimately the cle

s 14.3

... ))

FOR COMPUTABILITY 261

Turing machine to a rnais to specify, with each

each tape (what to write, f symbols seen at the set of

instead of the quintuple sp(!Citication like the follow-

··iKK)

ulti-tape machine would iln.otrnJ-.ti~ns include:

g machines to do tasks of th~~o1~eti¢al purposes, and no one

It is much easier to machine for a complex

to different memory punctuation devices.

Turing machine using one for holding the

of the greater apparent putation range is the

artial-recursive function. conventional, one-tape,

of a universal Turing rtl!Jl~ior1s of a machine's tape for

rnt:se-ltne des¢1·ipti¢'n region-was finite, but e method won't work for

only two directions; but ning every Kth square of

e leave the construction


PROBLEM. machine.

Although K-tape machu11e$, machines, this conceivably what can be done with the surprise that:

THEOREM 14.4-1

Any computation that can by a machine with two semi-· read nor write on its tapes,

end.

In view we use the length from the rPklf~in ol

our representation of m and away from the ends, and mconditional on reaching an

Another consequence of

CoRoLLARY 14.4-1

Any Turing-machine whose tape is always entirely

For one can construct a Tlttriilglm~tcl:j to the given machine, which wu" "'~'"'

PROBLEM. corollary.

14.5 UNIVERSAL NON-ERASI TURING MACHINES

We can now demonst [ 1957], that for any Turing Ilmtchilne chine TN that never changes struct a two-symbol m:acl:unt<:l tape to. 1 's but can not c

Our proof will have two make an equivalent ma,cnme subject to the symbol-chtznf!iflf.!. rd'tri.rtiJm

are

ES FOR COMPUTABILITY 263

ing machine TN out of this tape-squares grouped into

I~e:nti(ic:ation:

in T/v correspond to I 's need ever be changed

two-symbol machine never erase a 1, once

T/v mentioned above. T/v will resentation for the content nt the state of T's tape of

AOOO ... 000 ...

f C's to the left. We have dulgramts that will perform the basic

C1·--+--+----+-To next instruction zero- branch instruction

264 VERY SIMPLE BASES FOR C MPUTABILITY SEC 14.5

operations of theorem 14.1-1, arne! , m', n', m-, an n-; the I tter conditional on whether m an n ar air ady ero. That is, e ha show that we can increment a d (conditi nally dec ement m and n numbers of B's and A's on N-s ape, with ut v olati g th sy changing restrictions. Now onsi er t e fo r sta e di gra s in Fig. 14.5-1. These do exactly what is w nted. For exa pie, pply the tatediagram for n- to the strings f r (m, n) = (2, I and (2, 0), star ing s mewhere to the right in each case:

(m, n) = (2, I)

.CCCBBAOOO

0 . CCCBBAOOO

0

. CCCBBAOOO.

0 . CCCBBAOOO.

0 .CCCCBAOOO

0 .CCCCBAOOO.

0 .CCCCBBOOO.

0 "to next instruction"

0

0 .

0 .

0 .

In each case the machine first runs o the left ntil i unte s the bloc of C's. (It will never move I ft of the r ght ost .) It then start to the right to perform the de ired oper tion. On! ope ation n- h subtlety at all; the machine i sup osed to "erase" an A (if t ere i it can only do this by changi g tha A t a hi her I tter B o C. t pr -pares for this by removing t e left ost B (if here is on ); w en it meets an A, it will change this to a B, t us re ucin the num er of A's · nd r -storing the number of B's. he r ader can erify that we h ve a so a -counted for the exceptional case n w ich t ere i no ; fi ally, in th case that there is no A (that i , if n = 0) B is unch nge but he machine takes a different exit from th stat -dia ram. Not also that in ea h ca e _

. 14.5

wil wor wil dot on

0

Fo

VERY SIMP E BASES FOR COMPUTABILITY 265

e block of C's, so that we input of another.

s "move left (by triplets)

in T!v by the network of


PROBLEM. The machine its computation, by chitngingl a machine similar in operati steps in the computation (th simulated) is retained. Co that, if (m~o n,) are the then T!t's tape at time t straightforward sense. For encoding

ml n I ... EEE ... EEED C

First make a state diagram string into

... EEE ... EEED mt C

operations m', n', m-, n-, tions. Compare this syste 13.3-1, which also keeps a

S FOR COMPUTABILITY 267

1921, Post [ 1965] studied a tl CIHi<~usly flr·nctr.J.ting problems of which the

¢x<1m11ne the first letter of S. If it ppend 00 to the result. If rs and append 1101. Perng, and repeat the process

ore letters. get

IIOIIIOIOOQQ lOIIIOIOOOOQQ

liOIOOOOOOllQl l 000000 I I 0 lllQl

QOOOIIOIIIOillQl 1---1-----<OIIOIIIOIIIOIQQ

ted itself (and hence will conppose that we start with

r example, (100)7, that is,

·nly give up without become repetitive?" In

"Is there an effective way ss will ever repeat when

this (00, 110 I) problem fa computer. Of course,

help from a computer

1

268 VERY SIMPLE BASES FOR OMPl TABIL TY SEC. 14. ~ EC. 14 .6 VEF Y SIM LE B SES FOR COMPUTABILITY 269

(unless it has a theory) except for c enca aid n stu ~ying ex an pies; but i DI FIN IT ON

the reader tries to study th e beh VIOr of H 0100 0010 PIOO 100 w thou i A IPost cano ical yster p (or other logical string-manipulation any

such aid, he will be sorry. swsterr ) is n onog nic i , for ~my s ring ~. th( re is at most one new string Post mentions the (00, 110 I) pre blem in p · ssing, in h ·s [19 3] P' per- ~ 'tha can e pre duce fron it (it one tep).

the one that announces the flOrm ~I- for fn th or err -an p say tha "th

little progress made in the sol tion ... 0 fsuch prob l~ms r ake hem andi ! Cl( arly< tag ~ ysten ism onog nic s ·nee, or any string, what happens

dates for unsolvability. " As it tur ns ou he v as ri ht. Whil the so iva t) it dt pend only on i s firs Jette ; twc diffe rent strings can be produced

bility of the (00, I 10 I) proble m is till u nsett led (sc me 1= artia I resu ts ar only if a st ing t as tv 0 dif eren first Jette s, which would be absurd.

discussed by Watanabe [ 1963 ]), it Is no w kn wn that s me I roble mso What ·s the mpo tance ofth e mo ogen ic pr< perty? It is that, if a string-

the same general character a e un' olvat le. I ven nore in ten sting is th r anip Jiatio n sys em i mo ogen c, th n it is like a machine in all im-

fact that there are systems o this lass hat < re un ivers< I in he se nse of portan t res1 ects, for i defi es a defin te pr bcess or sequence of things

theorem 14.1-1; namely, ther ~is a way to si In ulat an rbitr ~ry 1 uring- t at h< ppen 1-anc thes1 can be re~ ardec ash ~ppening in real time, rather

machine computation within a "ta " sys em. t an a me e the orem s abc ut a p un hang ing mathematical world or sDace.

DEFINITION In act,< ne c< n im gine ial n achir a spe e associated with a tag sys-A tag system is a Post nor fi1al c noni al sy tern hats tis fie s the cond- tc m. I See I ig. I .6-l. Th ·s rna chine is a little like a Turing machine

tions: If A = (a" ... ,an)ist real~ habe of tt e syst em, a rd e cept that

g;$ _, $h; (i = i,. .. 'n) (l) The e are two eads one for n ading and one for

are its productions, then wr tin g.

i (2) The tape begir s at a so urce, runs through the (I) All the ante cedert cor stant strin gs g; have wr ting I ead, has a n arb traril y ion g piece of "slack," the same length P. (2) The consequ ent s ring ; dep ends only l n the ,_.,_ first letter of the assoc a ted IT;.

r~ ~~ ~ .... ~ ) For example, in the (00, I 101) rob I< mjll' t mer tione ~. th\ re ar real y

j nte

' ad Read

eight productions, forming t e sys em"' ith p = 3: I head

000$- $ po 100$ -$ 101 \.--

001$- $ 00 101$ -$ 101 - \ l) ~:, 010$- $ 00 110$ -$ 101 I Ta1 e

011$- ~ 00 Ill$ -$ 101 sou ce

Slack sink

Because of the fact that the onse ~uent h; is deter lninec by t re fir t lett r I op

only of g;, and that the num er of letter sin tl e g's is a c onsta nt P, the t g j -

systems all have the characte r oft he (0 , II 0 I} pr< blem ; narr ely, t o oper-

ate a tag system, one has to: ~ -~ -

Read the first le ter a,. t--Erase P letters f om t ):le frc nt of the st in g. "- -~

t--

Append the ass ciate ~con eque nt str ngh; to th c ne-stc te t-t--end of the strinl . lnochine

It is very important to obs rve hat t re ve y de ~nitic n of a tag syst( m ......_

t---t---gives it a property not founc , gen rally inn Drmal oro her c anon· cal s S---~

terns; namely, a tag system i mon geni . Fi1 . 14.6-

I

270 VERY SIMPLE BASES FOR COMPUT BILIT

then runs through th< reac ing head, and nail) disappears forever into a "sin! ." I can nove only in one

direction. (3) The finite-state pa t of them chine mus be a )le to read a symbol, ad van e the tape P sq Jares and write the appropriate h; wit~ the write head. Bu othe wise, the machine has no int rna/ tates

SEC. 14.6

Note that the tag machine annct era e as mbo. It would do i no good to erase a symbol, since it can orzly re1d a svmbo at m Jst once!

The name "tag" comes from th children's arne- Post was ir terested in the decidability of the questi n: Does the reading head, w ich is adva cing at the constant rate of P squares per uni time ever atch up wi h the write head, which is advancing irr gular y. (We do not req ire the h/s o have the same lengths.) Note, in the ( :>0, 11 I) pr blem that he re d he2 d adv nces three units at each step, whil the ' rite t ead a vane s by wo or four nits. Statistically, one can see, the latter has the sam aver ge sp ed as he fo mer. Therefore, one would expec the s ring to van sh, or beco ne pe iodic. One would suppose this for mos initial strings, be ause f the hanc s are equal of getting longer or shorter' then i is al rost c rtain to ge shor ' frorr time to time. Each time the string gds shcrt, th re is a sign ifican char ce of repeating a previously writ en str'ng, and reJ eatin once mears rep ating forever, in a monogenic pro ess. Is ther an in tial st ing t at gr< ws fo ever, in spite of this statistical ocstacle No one knows. All he st ings I have studied (by computer) eithn bee me p riodi or v nish<: d, bu som only after many millions of itera ions!

THEOREM 14.6-1 (Cocke [1964])

For any Turing machine therr> exi ts a ag s1 stem T T t at b have like T. in the (m, n) sense of 11 2, wh~n gi~en an axior. that encot es T' tap1 as A a aa aa ... aa Bb b bb ... bl with m aa's ant n bb's. The ta!

system T T has deletion number P = 2.

COROLLARY 14.6-1

Computability with monog'r>nic r.orma syst• ms is equi alent to Cl mput ability with general-recursive uncti ns, 1 uring macAines, f!eneral em onica

systems, etc.

I I

.l

Proof We will construct sepa ate t g sy terns for each c f the state of the machine T. Then we will lin the e together (by i entifwing ertai letters of the different alphab ts) tc forrr a sir gle t g system that b~haves like the whole machine T. W will begin by cc nstructing a tag syste n that behaves like the right-hand si:le of Fig. 11.2-1

We have been accustome< to tl inkir g of Tur ng machine as < pera - 1

ing according to the scheme of F g. 14 6-2. It is equi' alent more cor- _ ___.- ;

I

SEC. 14. VER SIMPLE BA ES FOR COMPUTABILITY 271

venie~t here, and ctually fur dam ntall sim pier to think of th T . m chm as n ad ,, •~ ~· e unng e up o stat~s accurding to the scheme of Fig 14 6-3 L< oked at th s wa~ a Tt · "' h 11 h · · · · . 'J, nng "'ac ne w a e twice as many states but m some ~enst thes wer con ealec in it already, for it must have h~d a se ~~thp.nir o states to emember what it haj read while it was writing so , .. et Iug else.

Pl1 OBLI_M. C bviot sly st< tes in the n w sys em do not correspond exactly ~~r~~tes m the old s stem. The new s ates 2 re also quintuples, but of the

(S ate Wri e M '" I d ov, ea : ifO go to if 1 go to) (J; S; D; Q;o Qil

an ther is only o?< quin uple or ea h sta e, rather than two. Use this formulat on of Tunn mac ines to simi lify tr e development of section II.).

d ~ O\\' to real~ze uch 2 state by a ag S\' stem, we will exhibit a set of pro~ twn that will have t e effect of a mo e-rig t state. In such a case we

WI I want to cllangt: m and n s< that '

m- 2m+ S;

n - I (n) = { n,2

(n - l)/2 if n is odd.

f n is even,

->-5Bk!tf{~ Go to O,uiTLt:J-, ..

t, ~ Goto o,} -~EJ-- ... Fig. 14.6-3

272 VERY SIMPLE BASES FOR C< MPUTABILI1Y SEC 14.6

and we must prepare the system so that i will next go to state Q ;o i n is

even, and to Q ;1 if n is odd. Let us begin with the string

Ap(aa) Bb(lbY

where@ n means to repeat the ~ tring@ n t mes. Our first prodL ction will

be

l

A- Cc

a- cc~

or l

A---<

a -

Ccc

CCC(

depending on whether S; is SUI pose to b~ 0 o 1. _(lfhis ~epends ~n ly on what state we are in.) For br'evity, we 11 !II abbrevic te ta pro ~ctw'r~ by showing only the antecedent J rst I tter ~nd t~e corsequ'ent s nng: smce writing the a;x $ - $h; forn is ~ o rec undapt. n. ev ry c se 1, the number of letters deleted, is 2. Applyinl the prodt ctwn abo we le2 ds to Bb(bb)"Cc(cc)m'wherem' is ither 2m cr 2m+ 1 epending n wle~h~r or not the state required the machi11e to write 1 or 0 on the quar It is leaving. The key problem is ow t::> det rmin~ wheth~r is e'en o odd. The procedure for doing this i initi ted t y the prod ctior s

which result in the string

and

which lead to

i.t ., Bx$- $~

b x$ f---+ $s

~ s"D1 'fJo(dido)m'

Now the oddness or evenness of n is, at l~st, t< have an e ect; or

yields either

DIDo(dido)m'TITo(ti o)n-l 2 or Do d1do m'Tl ro(ti o)"12

depending on whether n was odd pr ev n, respect vely. Thi has a prefound effect, because in the fi st ca e the systt m wi I see only ubsc ipt ' ' _ _____.

I

I

s c. 14 7 VERY SIM LE B SES FOR COMPUTABILITY 273

svmbols from no on, while in th othu cas it will see ~::mly subscript '0' swmbols! H~nce, we can control v hat t appe~s now with two distinct sets c f pro uctic ns:

"'hich ields

TITo(tito)"'A ai(a 1a 1)m' or

r---' evenDo ~ aoA oao

do -+ aoao

'-'here fve ha~e wr tten' 'for n - 1)/2 < r n/2 as the case may be. Finally, r-- n ado- r----n even-T1- B1bl To-[+ B 0b 0

p oduce the desired fine I stri. gs

A 1a I(ala J)m' '11b J('J 1b 1) '~(n-I 12 o A cao(aoao)m'Bob 0 (b 0b 0)"'~n/2

Tht imp< rtant thin! abo t the e tw) pos ible final results is that they a e in t ntire y sep rate alphabets. This means that we can now write diffc rent 1 rodu tion to dt term new at will nex become of the string in the c· se th t n was ev< n anc in n e cas that n wa~ odd. Thus we are, in effect, a le to lay out th< stru ture pf a 1 rogr· m. '~hat should we do, in fact? \\ e wri east t of I rodu tiom like he o es al ove for each state Q; of the T~ring machine, using t ntirely diff rent ~lphabets for each. Then we link tt em;\ hene era exit tate :Jil is he st te Q , we make the output letters A I. a I. BI. a d b 1 )f Q; he s me a the nput letters A, a, B, and b of Q

1.

Similar y we iden ify tt e ou put etter A 0 , a0 , 8 0 and b0 of Q;0

with the inp t letters of what ver state i Q;0 . Thu , we can simulate the intercc nnec wns f sta es of an ar )itrar Tur ng machine by combining in one Ia ge ta~ syst m, a I tag prodt ctions des ribec above. This completes the proof o thee rem 14.6-1

1~ .7 U~SOL ABILI Y 01 POS 'S " ORRI SPONDENC PROBLEM"

In 1947 I ost s howd tha ther is r o eff ctive procedure to answer q L estio s of he fo lowir g kind:

TIE CORRES OND NCE PROB EM

G ivt n an alphc bet A and a fini e set of p< irs of words (g ;, h;) in the al habet A, · s tht re a seq U< nee 1 i 2 ... iN c f selections such that the

274 VERY SIMPLE BASES FOR CpMPU ABILI Y SEC. 14.7

strings

gi 1gi2 •. ·!iN ~ nd ~i 1 hi2 .• • hiiN

formed by concatenating~wr ting c own n ore er~< orrespond ng g' and

h's are identical? While Post's original proc f of the ursolvflbilit of such 1 roblc; ms i~

complicated, the result of 14. ~that m< nogenic nJrma syst ms can be universal--makes it very sin pie to pre ve; for we can shov tha an~ procerl u re that could effecti ely nswe r all corn spon dence que tion co q ually well be used tc tell whett er any tag, or < ther rrionc genic nurm. system will ever read a halting ymb Jl, and thi~ is eq uival nt tc telling .vhether any Turing rna chine com utation wi I hal .

Proof Let M be a mon< genic normal ysten with axiom £ anc productionsgi$-. $hi. Now suppose t' at th"s sys em happe s to ermi nate by eventually producing strir g Z ' hich does not begin .vith ~ ny o the g ;'s. Define G i and Hi as follows:

if gi = apaq ... ,, let Gibe I XapXaqX ... X , I

and

Now consider the correspondence ysten :

Go t

XAH 0

where X's are placed after ecch le ter o A a d be ore elCh letter cf Z to form A and Z. Y is a new let er.

AssERTION

This system will have a matchir g pai of i entiC'll str ngs i; and only if the monogenic normal system (g i$ __. $hi), when stc. rted vith /- , terminal s with the string z. In fact, if there is any sol Jtion to the cor espondenc e question, there is just one, a 11d th t solution is (for the G's) he sequence of antecedents and (for the h 's) the sequence of cc nseq ents enco ntered in producing Z from A.

If we can establish the tn th of the ~ ssertion, t en tt e um olvat ility f the general correspondenct foil ws, or cne c n ac apt ny urin~machine halting problem to que tion f wh ther he m~chin~ in questic n reaches a certain special st te with a blan tap~; tht n, in the normal system, we can reduce this to the ques ion of tern inatirg in a pa ticulflr _ ___-

.,,1

l

SEC. 14. VER SIMPLE BA ES FOR COMPUTABILITY 275

0 ':; ( st mg L. 0 , we can s mply equate thi to t~e unsolvable halting prob-le In for tag S\ stem .)

Wh is the ass rtion true~ Let us first no e what the X's and Y's are fo · Th X's are t< make sur that if a irzatch ·ng pair exists at all, it must be~in w th thf tran cripti'pn X.- oftle axi m A. This is assured by the fact th~t the only fvay a'rz H s ring wn stc rt wi han ~is by starting with XAH 0 •

~ d since all G str nl!! "ust start w than X, we; can be sure that any matchrn_ set' tarts ~it~ ¥A. ~imil rly, ~ ny matchi g pair of strings must end With at ansc 1pt10 of"- ~mJre p ecise y, mt st end with ZXY~because a 'J string can't end in v and an f strir g car end only with X or Y. It fo lows hat i ~her is ary sol tion at all to this correspondence problem, th n th solutiOn must be a strin~ which ca be resolved into the two fo ms:

G 0 Gip Gi ... C iNZ), Y

XAJ 0 Hi 1Hiz· .Hi~; Y

Bt tift is is he c~ se, tt en it folio :vs th t the sequence of Gi 's is exactly th seqL ence that .voulc be follow d b~ the original monogenic normal sy tern (gi, h)! We can see this inducti ely: :ve have already established th_ t the H s ring must begin with XA. Then the G string must begin WI h Go Why? B cause the syste n is n onogrmic! That means that the beginni g of axiorr A c n be matched c nly l y g 0 . Then g 0 determines ho -the strin~ to b added to A~ nd I t us remove Go from the front. Then th ~e is only ne G i 1 tha can mate 1 the beginning of the remaining str ng; t 11s corresp Jnds o the; g it that tt e no mal system would apply at the next step. Then Gi 1 etern ines Hi- the string that is to be added to what is eft a ter d letin Gi 1 from th~ font. Again, Gi

2 is determined,

be a_use the s stem is me nogenic a d it, too, must be precisely the produ twn antec dent the n Jrma system would u e at its second stage.

. Thu the seque ce of G;'s must beth same as that of the g;'s underlyi g the monogen c normal system (and hence must represent the steps on the c mpu ation of the still further underlying Turing-machine computat on). If th process tc; rmin tes, hen he Ia t G that was used will be fol owed in the strir g by Z~bv definition that which will remain after no more pr duct ons can be appli~d.

Ther~fore, if th strirgs m tch, hen hey nust both be the sequence mention d in he assertion.

So f3 r, the only mon pgeni nor !J1al s stems we have are the tag systen s. \l e co~ld, I owe\er, h ve u ed t eore!n 14.1-1 more directly to shew th< t mo ogen ·c norrrzal svstem are unive sal: Consider an arbitrary twc -regi ter rr achir e, an represent its st< te by a word of the form

lj Ill ... Ill Kj iii ... 111

276 VERY SIMPLE BASES FOR C DMPU ABILI Y SEC . 14.8 EC. 1,j .8.1 VE Y SIM PLE B ASES FOR COMPUTABILITY 277

Then, using the same method as " e use din 2.6, ve ca n rea ize tl e m- 4, 7) nach ne of this ectio n (de cribe din r v1insky [1961]). The reader is struction types of theorem 14. -1 as folio ,vs. ,ve1co ne to enter the c ompe ition (I be ieve t hat a certain one of the (3, 6)

Conditional subtract from first r egister: nachi nes II ight be ur 1vers 1, bu t can 't pr ve it)-although the reader houi< und rstan d cle rly t at th e que:: stion is an intensely tricky puzzle

IjKj$- $IrKr (i.e., go to Ir if egist rise mpty nd h s ess ntial y no serio s rna them a tical interest. To see how tricky 1$-$1 r things can I ecorr e, th read er ca refe r to r ny 1961 paper describing the

(sub ract and go to lj+ t) (6, 6) mach ine; i is n uch more com licat d than the machine of this /j1$---+$/j+l ~ ectio .

Kj$- $Kj+t

Conditional subtract from SeCOI dreg ister: 4.8.1 The four-s ffmbol seven -state

lj$- $Jt (rota e tot xam1 ~e sec ond r ~gister) unh ersal inachi e

Kjlf$---+ $Kf Ir (got Ir if regis er is mpt') ' Th ever noti pn of a un vers:: !Tm ing n achine entails the notion of Kf$- $Kr t escriJ tion; the II achir e to be sin ulatc:: d ha~ to be described, on the tape

ract I and j+l) r < f the univc:: rsal r ~achi ~e, in the f prm f SOl re code. So, also, must the Kjl$- $Kj+l (subt o to

i~itial tape br da a for the s mula ted II achir e be described. One way to If$- $/j+l ( 0 thi encc ding is to write, almc st lite rally the quintuples for the simu-

Add to first: I ted r 1achi 1e on the t:: pe of the u niver a! m:: chine; this is what we did for

lj$- $ Ij+ I I 1\j$- $Kj I tile rna chine of ch apter 7. 0 n the other hand there is no particular virtue i the ~uint pie f )rmu ation and one r night be able to get a simpler uni-

Add to second: versa! mach ine "' ith s mec ther repre enta ion. Nevertheless, we must

lj$- $ Ij+ I' K$- $Kj+ll not gc too far, or i one is p ~rmit ed a n arbitrary partial-recursive compu tatio to d o the en co ding nd is perrr itted to let the code depend

PROBLEM. Can you m ake 8 simi ar cc nstru( tion or th ree o moe on the initia data , then one< ould use a the< ode the result of the Turing-registers'? n achi e cor nputa tion "tself, and t his w ould urely be considered a cheat!

' (tWO ld gi ve us a (2, 0) m chine , sin e the answer could be written in unary Jn at ~pe,a nd n< com puter woul ~be r ecessary.) We have to make

14.8 "SMALL" UNIVERSAL Tl RING MAC ~INES some r l!le, e g., th ~t no hing ike ft II co rput< tion power may be spent on t e en< odin1 . In orma lly, t) is wi I bq uarar teed if the encodings for the

The existence of univers I rna chine S IS urpri ing noug h, ar d it s n a chili e strt. cture and} br thl data are IIane eparately. Then we can be startling to find that such rna chine can be qt ite si ~pie ·n str ~ctur . Ore s re th at th rna< hine was ot a plied to t e data during the encoding may ask just how small they an b ; but to ar swer this, pne n eds o ha e p oces . n is co nditic n, WI clai r, ju tifies what we do below. More some measure of size, or con plexi y, of a rna ~hine Sev eral r ~easu es c< n tc chni< ally, bne rr ight I equir , for exam pie, t at the encoding process be a be defended; Shannon [ 1956] sugg sts tl at on emig ~teo sider the I rodu t p imiti we-re< ursiv , sy rbol- rnani ulati pn OJ eration on the input; this, of the number of symbols a ~d th nun ber < f sta es, si nee, · s he shows, t< 0, W( uld g uarar tee tl at if hen: sultir g rna hine is universal, this is not this product has a certain ir varia ~ce. pne < an e~ chan e sta es ar d syr - d~e to some pow< r cor ceale ~in t peen odin ~ process. Davis [ 1956] dis-bois without greatly changi g th is pr duct Tc cou t tht nun ber f c sses his q uestic n. V e wil I pre~ ent fi st th encoding for our machine quintuples would be almost he sa me. a d the:: n its tate- ymb I tra sitio tabl .

In this section we desc ibe t he UI ivers I Tu ring mach ne w ith t e The:: four -sym ol, s even- tate mach ine VI ill work by simulating an smallest known state-symbc I pro duct. Thi mac hine s the mos rece nt a bitra y p = 2 ag s stem w kno w, b theorem 14.6-1, that if a entry in a sort of competitic n be inn in g wit 11 Ike o[IC 58] \\ hoe hibit d machin e can do t his, i mus t be mive sal. We know, by the proof of a six-sym bot, ten-state "( 6. 10)" rr achir e, W tan a be [I~ 60] (I ' 8), Mins y tf eorer n 14. -I' t at n prese nting the < uintt pies of an arbitrary Turing [1960] (6, 7), Watanable [19E I] (5, 8), 1\- insk [196 I] (6, 6), a d fin ally t he __ ~ rr achir e in t he fo m of a P = 2 ag sy stem is a tedious but trivial pro-

''

278 VERY SIMPLE BASES F )R CO MPUT BILIT EC. I .8.1 SEC. 14 .. I VER SIMP LE BA SES FOR COMPUTABILITY 279

cedure. Since one can se em dvar ce fo any such repr sent< tion how e~ actly their form tion neede d to ocate the { roduction corresponding to much work will be involv d, th e con versic n is ' prirr itive- recur ive c per- th at let er! ation. In fact, it involves only lv'ritin g dov n SIX teen rodu ction for ach The mad ine w ill US< only the h tters alreac y introduced: 0, I, y, and A. quintuple. It is un erstc od th ~t '0' is als o the blan sym bol on the remainder of the

Suppose, then, that tht: tags wsterr is: in rlnite tape. Tab e 14. -1 is the s ate-s mbo table for the machine; it is

Alp hal ur derst pod t at if ~10 ne ~ sta e is g ven, hem ~chine remains in its present et: a h a 2, ... 'am st te.

Produc tions a1 f- a11 a12 •• .a 1n 1 Tal Ie 14.8 I

a2 1--- a 2 a22 .. . a2n2 1 q q3 4 qs q6 q? . . ..

0 L 0 /I y L L yR y R 0 R am ____.am 1am2 • . . am m 0 L y } HALT y R/5 y L/3 A L/3 y R/6 where n; is the number of letter s in t he co nsequ ent o a;. Fore ach I tter I L/2 A J A L L/7 A R A R lR a; we will need a number 1 ;COn pute j as f llow : IL y [o, /6 I L/ L IR IR 0 R/2

Ill = 1 The mach ne stc rts in theft nbol i n S. q2 at st sy1 It seems useless to try N;+l = N;+ n; to ex pia in th mac hine, exce t by folio fving "t through an example, be-

so that N; = 1 + n 1 + n + .. ·+ni-l· Note hat t ~is is just pne r f!ore ca ~se it vari DUS ft nctio ns ar all r ~ixed up. Penerally, states q 1 and q2 than the number of letter in tl e pn duct" on cc nseq rents prect ding that re d the first ymb< I in ' , Joe te ar d rna rk th corresponding production of a;. We shall see the rea on fc r thi~ de fin ition hort y. P; and eras the first symb ol in S. State~ q3, q4, qs, and q 6 then

We will represent any tring (e.g., an a iom) a,a,. .. az pn U s tap ~by co by tht proc uctio r con eque rt at he en doU . (The copying works from a string of the form im ide tc out; his i~ why the p oduc ions ~ere ~ritten backwards.) When

tht end pf thl proc uctio n is c etect d (b1 q4 a f!d q 7 finding the 11), then .s = y 'Ay SA .. .AyN q? estOI es tht tape , rem oving the r nrki g of the production region and,

so that N;, used as the ler gth c fa st ring< f y's is us ed to repn sent ; to inc ident lly, ( rasin g anc ther ymb )I fro m S. Since two symbols were the machine. The A's are pace s. era sed f om S', an d the app opria te pr oduc ion is copied, we have a

The productions will b e rep1 esent ed as ollov s. D efine P= 2 tag proc ss. The probl min mak ing a "sm II" n achir e is to avoid use of new

P; = ll 0~ in;01. .. Q.! 0 N;2Q_ ONi! let ers fc r rna king All mar ing < f WO king places for this machine is {

do e by interc hang ng 0' and y's ar d I 's and .1 's. so that the representation of the cons quer t of c ; beg ns w th 11 and hen If q 3 meet a 0, then achir e hal s. It turn out that this can happen has representations of t he c nseq uent' let ers- In n verse ord ~r- on! w un er s1 ecial cone ition ; but thes cor ditions will come about, separated by 01 's. (We u e str ·ngs c f O's here inste< d of strin! s of lv's.) eve ntual y, if he sr ecial strin~ PH F 110 101 i used as a production and Now, finally, we can descr ibe th ewhc le of U's t pe; it is: thi! prod uctio r is n feren ed. oiL: ny of the I tters of the tag system are

sur pose< to c ~use ~hal , we assig to hem the production PH. The

l ••• oool fnl~- II··· p21R 111115 000 ... nUl rber Has~ igned to th ~halt symt ol Ph is 3. Cons ructi f1g an ~ foil pwin! an t: xamr le is tedious. Here is a simple

The secret of the encoding is th is: Th pair lla nd 01 are u ed a punc tua- ont:

tion marks; II marks t e be ginni 1g 0 a p oduc tion and )I marks AN E XAMP LE

spaces between letters in pro uctic n co sequ nt. Then are PXaC( y n; Wew ill co e the mac ine f r the simp e tag system punctuation marks in the tth pr duel! on P ;, and there is or e ext a II just al---; a2 to the left of S. Hence, tl ere a e ex ctly V;pu ctua ion n; arks bet we en S a2---; a 2a and the beginning of P;. ~ o the code YN; hose 1 to epre~ ent a ; con ai11~~ a3---; halt

j

280 VERY SIMPLE BASES FOR OMPL TABIL TY SEC. 14.8. EC. I .8.2 VE Y SIIV PLE B ASES FOR COMPUTABILITY 281

For this system, 4.8.2 Str cture of uni ersal machi nes

n1 = I N1 = I PI = 11 ( 0 emi htsu ppos<: , or h ope, hat t he pr operty that a Turing machine

2 N2= 2 p2 = 11 ( 000 )\ 00 s uni ersal shou ld im ply sc me 1 teres ing c onclusion about its state dia-n2 =

4 p3 = 11 ( 1 OJ (the halt p rodU( tion) I ram. But it see ms th ere is noth ing rr uch o say about this, in general,

N3 = e th( e uni ersal 1ines with structures so trivial that one I )ecau re ar mac

If we start with axiom a2a2 2· th is is ncod ed as yyA yyAy A a d th and awn o int<: restir g cor clusi ns. Supp ose, for example, we make a

tape is: traig tfon ard mach 'ne fc r a P=2 ag s stem. Let the axiom S be .\>Titte n out on th e tap(

1 11 01 01111 0000 01 0( 1111 Y yiAI YY l 0 11 yy A I fo 0 0 . o o! . . 5 ... 0 q, !

'--'-I nd st art th e mac hine t the begir ning of S .vith the state diagram shown

The machine marks symbols to th e left find' ng tv. o pu ctua ion g roup, n Fig 14.8 l.N oww can make such a tag machine for any Turing rna-and then goes to the right in .state q6, v ritin an"' at t e en .n e tape hine, by s ctior 14.6; so e ca 1 alsc do t for some universal Turing is then

,, nachi ne; h( nee t 1ere i am chine with this s tructure that is universal. I

I

I

111 01 01111 0000 01 0 )I AAj lA A y y yy 1 yy A ! Sto~ 0/ ®2- -011-1 ~012~01n1

~ \t ~~ ~ ~o-...

; L'o .!!.. 1--··· Note that two deletions hav beer mad e fro n the fron of th e tag strin !

~ ~ "-o ..._0- ... Om m

Now the production is copie j (bac kwar ds) at thee nd; t vo 0' , and A and o-......... ®2- ~Om2-@2--0mnm four more O's, forming yy 1 yyyy: ~

om1-l

..._

' . l 11 01 01!11 y y y y yA y v IAA I lA AI YY Y YY 1 yy 1 yy 1 yyyy

~ ig. 14. 8-1

On the next trip to the left, the rr achir e enc ount<: rs th 11, nean· ng th at T ere s imply does n't S( em t ) be ny s ructure required that is any

copying is to stop; the mach ne en ers q 7 and res to es th e tap to t e for m more comp licate d tha one needs tom ake a multiplication machine. Per-

haps his is not s Llrpris ing ir view ofth orerr s like theorem 14.2-1 and the

111 01 01111 0000 01 c o!11 )111 I lyy t llyy A diffict lty 0 excl 1ding full ecurs ion, .g., r ninimization, in any machine

00 0 00 0 YYYY hath as an y iter ative (loop ) abil ity. n an y case, the demonstration by

~ Shan on (I 956) hat, II OW( d enc ugh s ymbc Is, one can replace any Turing

mach ne b a h O-St2 te m chin sho ws th at the structure of the state

This is, in effect, like the tartir g sta te (q has the arne effec as ~ I) diagr m ca n be hid de n in t he de tails c fOp( ration and not clearly repre-

except that the string a 2a h s be en re place d, as it sl ou1d be, by ente in tt e top ology ofth ~ inte state conn ctions. az

a2a2a3. f>ROBLEM Cho sean two- ymbc l, two state machine and show that it is If you trace the operati )n th ough to t e en d, yo l wil see OW he

ot universa . Hir t: She w tha its h lting problem is decidable by describ-string next becomes a 3a 2a 3 . Foil ::>win~ that, after som curi ous s ruggl es,

roced ure th at dec ides v. hethe or n t it will stop on any given tape. ng a the symbol a 3 will cause th mac hine o ha It; it hrst vrites AA nd t his D. G. Bobr ::>w an d the au tho r did this f r all (2, 2) machines [ 1961, un-sequence eventually causes tate~ 3 toe ncou ter a zero. -•'

~ publis hed] b v a tee ious r ducti on to hirty- ::>dd cases (unpublishable).

L

15

Chapter 2

SOLUTI NS TO SELECT D RO LE S

2. 7-2. The output 0 indicates t at u to th pres nt, 0' and I 's h ve a! ays curred in pairs. Note th t the ower- ight- and s ate is "dea " in he se that the machine can ne er lea e it, . In t e no ation of ch ter 4, the output 0 co resp nds t clas of (00 vII)*.

2.7-3. If the machine has k st· tes, a zeros in ... Ok+ 11 ... , it woul have repe the' I' and continue the 'counting."

2.7-4. The outputs do not contain a y inf history that can affect I' ter ou puts.

2.7-5. The basic idea is to pro agate two k nds o three times as fast as th oth r. W en th

two equal parts (it is mer detail to accoun nd e en ch in lengths) and the process dupli a ted ·n eac half- hain, with ew " re w en ready" commands star ing a the enter A s ldier actua ly fir s at the moment he finds himsel an is Ia ted chain of len th I.

This solution takes n moments, for chai conjecture that there might b a so ution of th can be no faster solutio .) I fact it is n t har any small number f > 0, of he or er of (2 + difficult to see that the e is a exac \y 2n soluti n; th s was first hown by E. Goto and one was found by obert Baize , of arne ie In titute of Technology, using on\ 8 st tes f r eac sold.er, a d 8 inds f sig als flowing in either directi n-e uiva\ent to 64 in

SOLU IONS TO SELECTED PROBLEMS 2B3

A ublis ed s lutio , wit som furt er related theory and results, ppear in a aper y Pat ick C Fisc er [19 5].

C apte 3

3.2-1. he ke

3.2-4.

3 6-1. he ke

Chapte 4

(2) ( v II I *) *

the s nu bers produced by a binary counter aller numbers than the current count.

reader will be able to see the 20, use the net shown here.

t----•o

~--r-~-----------------.3

284 SOLUTIONS TO SELECTED ROBL MS

(3) ((0 v 1)000(0 v 1)0)*

Start-

(4) [0* 1(11)*0*01(11)* *0]*(11)*

4.3-2. R 11 (ae v abg v dg)(h v fg *

4.3-3. Rules: E v F = F v £, ( v F)*= ( *F) E* EF)*E = (FE)*

4.3-4. True, False, False, True, True,

4.5-4. E = (01 vOOl*Ov 11* )*

ER = (IOvOl*OOvOl* )*

The two states cannot be after the 000, so they cann

mu t los like hat i

(7 Ye: A machine ca

10"10"11 = 0101,001001,00010001 ...

4.5-6. (a*b*)*, (a*(bb)*(ba)*)*, (a ((bb) one operates from the out ide-in.

Chapter 6

6.1-1.

6.1-2. (I) Make a machine that multiplier of section 6.1.4

orks in ge era!, if

itsel ! (One can furth r

mber: then acts I ke th unar

• i

A

SOLU IONS TO SELECTED PROBLEMS 285

any ays t do his, so your solution won't be sum ends p to he left of B. Why do we use

top

)--f---+--~0-+--+----< L

8 'Subtr ct on "

( ) "Add o e"

Prime

Not prime

0

C PY Add one

286 SOLUTIONS TO

6.3-3. (I)

~0 \:7"

(2), (3) Just use the given

to make quintuples

and

where¢(i) = Oifq;isana~1:q>ta[ble

(4) 'Obviously, the only di belt\\j<:en and the machines above ¢onc1~rr1s can't go back to look at (5) No. Because of (2) (6) No, not unless you are reading chapter 8.

Chapter 7

7 .4-1. It is easy to describe the doubly-infinite tape as f all the information is "mla,ppe~, singly-infinite tape. To of the details in figs. 7.2-5 solving problem 7 .4-2, infinite tape immediately.

7.

9.2-1

SELECTED PROBLEMS 287

ny sequence of symbols from

;,,l,X, ...

consider the machine T rer>lre:>entlation of the machine T itself,

n+ I· The long sequence of

plus machinery for erasing begins by writing A 0 B and hen it needs more "blank"

ing away whatever might origi-

section 8.3.4 because, for vant whether the initial

given the length L of the of T, one can calculate the

go more than L · N steps sUtte··taJJ>econfi!mratlion, or erasing a l, or getting

infinite blank part of the if T has not written a l in

when their prototypes halt, T be a machine that halts

is equivalent to the halt-

in problem 8.8-1(1) and cyclic loop at that time.

ign a Turing machine Tp,q

recording its digits in the

288 SOLUTIONS TO SELECTED PROB EMS

first few states of Tp,q)· T en w< have to print the possi ly in nite c ecimal sequence for the remaind( r. Bu this s quen e is always ventually p riodi ,

e.g., like ¥- = 3.~r21 ~~ l~r, ...

so this cycle can be buil directly in o Tp, . In ident lly, c n yo use a finite-state argument to prove that he se~uenc of c igits pf a ation 1 number is periodic?

9.2-2. Given the machine Ta wh ch gi\ es an on the tape, we build a ne~ rna hine digit computation) the fo m

the nth d"git of a wh( n n is wriW n ~ wh se ta e has (at the end of each

... XaoX 1Xa2fy ... anX(~) Y

The new machine T~ re· ds th nurr ber b tweer X ar d Y, does -vhat a

would do, using methods like t ose o 8.3.~ modified a d ext nded to ke p the T~ calculation from overlar ping he alr~ady-] rinw material p ecedi g the last X. When finishec , the *pe should t ave tt e forn

Xao1Ya 1X .. XanXan+ 1X(n 1) Y

The machine will have h: d to r aintain a c py o n thr ughout the computation, but this is easy t arra ge. o show the other direct on of equivalence is easy: given a rr achin like T~, b ild a Ta tl at, g"ven n, allo fvS T~ to run until n X's ar prin ed; t en er se ev~rythipg on eithe side of

the number to the left of he Ia t X.

9.3-l. The key idea is simple but f uitful Be in b) supplying M wi h sore representation of the nt tape which it ca 11 inte pret s the integ r N. It puts a variant of them: chine T R u nto " uper ised perat on." It w: its until T R u has printed e actly n X' (as i pro !em 1 .2-2). It tt en st1 ps

simulating T Ru and inspects he d*it to the h ft of he Ia t X hat v as printed. This digit determines the ar swer.

9.6-3. The number k = .33333 ... is pbvio sly cc mput ble, sbj(k), if con putalle, must be defined. But there is no way to decide ts value at ny fir ite point of inspecting the input t pe. I or at any fi ite p int, t e rna hine will h ve seen nothing but 3's. Sp it m~st ch~ck th next input digit. It c n decide safely only when it first ncou ters a digit below 3 (in ~hich case (x) = 0) or one above 3 (in whicl case (x) ~ I). f the1 e is n such digit, and trat

is the case here, it can n ver h· It!

Chapter 10

10.3-3. If y and z are represen ed as inary numt ers, b t we rite, ·n bas1 three

f( ,z) = "y"2' z"

then we waste k of eac digit as well as ore whole digit. Th< n

f(y,z - 3. (yz)l+l/2

SOLU IONS 0 SELECTED PROBLEMS 289

In gen< raJ, b usin~ base , we get

;(y,z)- r()z)l+l r

< nd I/ can l e as s1jnall a we Iii e. Ore can bo a little better by using more

c ompli a ted odes nd can kee1

C(y, z) < lvz(l f)

( or an f anc large enoul h y ar d z). )ne c: n't do better than

C y, z) f-- y · z

l ecaus for ary Yo, Zo there are lvo. z pairs (y, z) for which

Y < o ar d z < zo

10 3-4. o fine y, given C( , z), find the large t n fo which

C(y, ) - ~ 2 ; I :;:,: 0

hen t is dif erenc will ~"lave the value of y; and there is a similar pro' edure for z.

10 4-1. he summat"on was dom in 10 3.2; tt e pro uct is similar.

10 4-2. Define

I 0 5-l.

I 0 6-l.

10 6-2.

l~efine

(x,y = I

1:--(x,y = 0

X k

1>( ) = = II _, J-1

if = y

if """ y

E(l/;(j), xU))

. his _is ~h_e desired~ func ion. Verify that the definition above can be put 1p prirr 1t1ve- ecurs1ve for[n, usi g problem 0.4-1.

(0) = I, A(l) =

216 2

, A(2 = 7, A(3) = 61, A(4) = 22 - 3.

I_ don't know any e pecia ly neat way~ to de these, though there is no part cular conc1 ptmil diffic lty. What I would do, if I had to describe < nume ation , is tc desc1ibe (I a pre cedure for enumerating, say, lexical raphi ally, all st ings f per[nittec sym ols and (2) a procedure for t~s~ing the st ings o see whett er th< y are well-formed sets of equations sat1sfy ng th definition of rimit ve (or general) recursion. These I rocedures ' auld take nly < few lines o describe, using one of the 1~oder. "str ng m nipul tion' com uter brogramming languages ment oned n the rote t chapter 12

ssuming ar effec ive p ocedt re fo1 evalt ating the definitions found in I 0.6-1, simp! eva! ate tt e nth ~efini ion fc und for the argument value x. If one ~oes i to de ails, ne w"ll hav to "'orry about functions of more than o e var· able.

290 SOLUTIONS TO SELECTED I ROBL MS SOL UTI ON S TO SELECTED PROBLEMS 291

10.6-3. If one remembers that the IL op rator is net ded c nly o ce, tl is pr< blem 2.4-1. Prov this by inc uctio . BA E: all palim romes of length I and 2 are in-isn't very different from IC .6-2. cludt d (as axiom s). A y pal ndror ne of length n > 2 has the form xPx

The function V(a, b) isn't rim it ve-rec ursive. wher e Pis a pali ndrorr e of I ngth ~ - 2. Use two induction hypotheses 10.6-4. /! foro dd an even length s.

10.7-1. (I) X, 2.4-2. lfN s the ' umbe r of st tes o them a chin , it must accept the palindrome

Prime (n) = P(n, 2) a a "'baN a

P(n, k) = (ifn = k hen 1 lse (i n = . (n/ k) the 0 but i n so < oing t hav ted a state during the first sequence l' t mu repe

else Pr(n, k + ))). of a' . If L is the lengt h oft e of s ates thus encountered, the same

•I e eye

mad inem ust als o acce pt (2) Define ',

Lba" a a aN

R(m, n) m there maine erwh nmi divid d by = m - n. - = whic is no a pal ndror ne. n '

Then 2.4-6. Any paren hesis 11est c n be egard ed as built up from the outside. Thus

gcd(m, n) (im> n then gcd(n m) el e wee< = n obt in the nest c fprot !em I .4-5 t y

(i R(n, m) = 0 then m els1 gcd ( R(n, m ), m)) __u_ ( ())

) ( 012 ) (3) </>(n) = 1/;(n, r - 1) ( ()()__(_ ) )

tf;(n, k) = (ifn I= I th n 0 el e (1/;(1 ,k- 1) + N(gcc (n, k) )) ( ( ) () (_i )_)) (() ()((_i ) ) ) )

10.9-1. Define integer equivalent to bt ( () ( ( ( () Q)))

0 1 2 ' Tog t, say ()() rom ( ), wei t$1b () ar d $2 be the null string. To prove

t t t ttc. the s ystem gene ates t ~ewe 1-forn ed st ings, use methods like those of NIL <NIL· NIL> <N L ·<I' IL • N L>>

,, secti bn 4.2 3. '~~

That is, we let NIL repr sent 0 anc , if n repre sents an in eger, we le 12.6-1. If the sk's ~ ere le tout, ~ewo uld ha ve bo h the productions C(NIL, n) represent its sue cesso . The f!, for ntege sx an dy, w can< efine

$1 ;Sj$ f-.. $1s j%$2 equal (x, y) I= (if-' = Nl L then and

(ify = Nl then NILe/ eC(N L, NIL) q;$ f-. Oq;$

elst (ify I= NIL then C (NIL, IL) Then for a f\'ith q els1 (T(x), (T(y) )) tring that b gins ·,two productions would apply to the equa same strin~ , and we cc uld g t two diffe1 ent proofs, eventually, of some

Then "equal (x, y) = NI "wil betn eon! ifxa dya eequ I inte ers. the01 em.

12.6-2. A sin pie se t ofpr oduct ons tl at wil suffic e here are: Chapter 12

,, $1~o0$2 ---+ $1 qo$2

12.3-1. The trick is to consider a, b, nd c to be the 0 1, 2 digits of a ernar $1 ~0 1$2 ---+ $1 q 1$2 number system and cons ruct mac ine li e tha of tl e solu tion c f prot-

$1 oB$2 H lem 6.3-1 but with ternar y add 'tion.

---+q

i $1 rlt0$2 ---+ $1 ql$2 12.3-2. Use exactly the same tric k as i 12.3 1, exc pt us 'ng ba se-4 n umbe s, with -~

the extra digit playing th role Jf stri arato , i.e., "comn a." ' $1 'ltl$2 ---+ $1 qo$2 g-sep

12.3-3. A string of length n can e cut n any or all of n 1 pi ces. '\t eac h place $1 ?tB$2 -q tH

there are two possibiliti s-di vided or no. So altoge ther t here ~ re 2n 1 wher the p ocess stops by pre ving ~ oHo qtH. Because of the character

possibilities. -------I ofthi prob em,\\ e don t need theta pe-su plying productions. '/

292 SOLUTIONS TO SELECTE PRO LEMS

12.6-3. The axiom must have ttJ.e for[n of' well- orme pan nthesi nest with x

for '(' andy for ')'.

12.6-5. It is possible to make a ysterr that 1 rodU< es on! sqm re nmpbers, e.g.:

Alpt abet: I

Axi< m: I

Pro< ucti01 : $ - $$$'

but this doesn't produc all tt e sqw re nUJ):lbers. Now any production on he a! habe with just ' ' can be p t in t~e

form kl kz kn P I $z .. · $n I

(Why?) Now either the k's ar all e~ual or not. If not, the1 then is so ~e k; r< k j. Then consid r any numt er x r roduc d by the sy tern, and any

number r < x:

x = (x - m - r + r ~ m f+ k ;( - rr - r) + k j + I

= ! ;X + (kj - k;)r I - k;m

If k; = 0, then this implies that th~ syst m co tains the a ithme ic se1 ies r(kj - k;) + (I - k;ltz) in r, an< the et of squa e nmhbers contains no arithmetic series. But f k; s not zero, then the s stem contains arbitrarily large pairs pf nmpbers ust ( j - ! ;) ap rt-a so irr possi !e. Hence all k's must bet qual. In that case

x = (x - m) + m -+ k(x m) I

and each production Pener tes a seque ce o num ers tl at gr w exponentially. No finite set of such equer ces c n form th< set f square numbers. (Show that hey g t too ar ap rt.)

12.7-1. 'Alp abet: 0, I

Auxiliary etter: f4 Axi m: A

Pro uctiors: $,- _.. $0A

$~-+!A

$~ -+ $

It cannot be done wi hout an auxilian lette . A ~etaih d proof is ery difficult, but to get an idea of wha happ ns, cc nside any r roduc ion

go$1~1···lk···$ngn- ... $ ···

Now consider any do ble string o the f rm

goglg2 ... kSgk+l· . . l!ngoglg2 .. . gkS[; k+l· · .gn

For any S, such a str ng m st be a the< rem. Wher this tring s givt n to the corresponding pr ductibn, on can et $j 0 ur lessj = k s that

Sgk+l···g.go I··· kS

SO UTIO~S TO SELECTED PROBLEMS 293

rep aces tach ? curre ce of $k _in ~e consequent. Because Sis arbitrary, the result won t be a c ouble stnng, liT gt neral, unless h occurs twice; and bee use t is is t ue fo each , one can slow that the consequent itself must be 'repe< ted." Ther one an u e subtle arguments about exponential gro~th o the st ings <sin U e solu ion t< 12.6-5 (p 292).

12.8- . Th folio lving c~noni a! system i an e tension for the regular expression (b c)abla v b< *)*:

Ax om: IYyz

Pn ducti ns: J $ -+ b$

J $ -+ c$

$ Y$' -+ $ab$'

$Z-+ $

$Z-+$WZ

$ W~'-+ $a$'

$W '-+ $bV$'

$V~'-+ $$'

$ V$' -+ $c V$'

On can~ ee in t~is th t Xi~ real! (b v c), Y is ab, Z is (a v be*)*, W is (a be* and 17 is c W at is he gereral principle here? Describe the

cons ruction indt ctivel .

12.8- . Fo exarr pie, h t M t:e described by qu druples (q;, sj, q;j, r;j), let qA be the set of tates egar< ed as ndica ing U at M, when started in q 0 , accepts the input tring and cpnsid< r a sy tern like

- 1 lphalet:Q ,Q1, .. ,Qn;sl, ... ,sm

1 xiom Q 0

I rodu<tions: Q;$ !-+ Q;1 $sj (alliandj)

Q;$f-+$ (allq;inqA)

Pre ve that this ysterr is a < anoni a! ex ens ion of the set of strings recogniz d by f\1, an~ adrr ire its elegarce a~ an alternative to the concept of fini e-stat mactJ.ine. Modi y the systerp so that recognition depends on the occur ence f cer ain o tput ymb< Is rather than on the occurrence of c rtain states

12.8-~. Th s problem is qui* complicat d, a! hough conceptually not very difficl lt. 0 e has to ke p, al< ng with tht axioms, a theorem list of all the strings gt neratt d up tp the resen time One must also have a procedure for detenpinin! when ver ary theprem n the theorem list can be matched to he an ecede t (left-hand part) pf any production; in that case one must thep rear ange he di covetFd pa ts to form the consequent of that produ tion 8 nd ad~ the esult o the theor~m list. One must make sure that

294 SOLUTIONS TO SELECTED P OBLE ~s SOLU IONS TO SELECTED PROBLEMS 295

each production is applied to evt ry tht orem (in fa t, if [nulti- ntec< dern C apte r 14

productions are considerec [see 3.2], o eac sub et of heart ms). One

must also test to find eve y wa' in " hich ach heore In rna ches each 1~.2-1. fthe umb< r in th e sing e regi ter R s 2'3 , then there is a correspondence

antecedent-for there may be me re th2 none corre t ana ysis ( ee 13 1)- etwet n the nstru tion s ts

and produce the correspon ding t ~eore rs. B caus< of al this, he T ring A~d It< r ~ Multiply R by 2 machine required will be v k:ry cc mplic ted. Wed o not go th rough this A~d It< '- Multiply R by 3 construction here because, ~e ha ved tt e the< in ch

s once e pro rems pter

13, the kinds of productic ns we have to co nsider will ecom e so r nuch St btrac I fro nr <-+ DivideR by 2

simpler that the correspc nding Turi ng-ma chine cons ructic n wi I be St btrac I fro nS <-+ DivideR by 3 quite trivial! For this rea son, he re der is en co ragec to a alyze this

problem carefully to see hat is inv< lved n the com licate d req ired 'lith t e usu a! cor ditior s for subtn ct an d divide. Then the proof of

procedure, but he should n :>t att< mpt t ) com plete his eft ort b\ fill in out heore m 14. 1-1 sh< WS th t we an w ite pr ograms, using the left-hand set

details of the Turing mach ·ne. he sa neeff )rt wi I ben uch t etter pent ifinst uction s, tha have the e ect, i we s trto 2m3n5a7ziiw, of the in-

on trying to understand, ar d per aps ir prov , the roofs in cha pter I tructi )ns

Add It )m,A dd It ) n, etc. ( S ubtrac t I fro m n, e c.

Chapter 13 I ut th n the corre pond ing p ogran s, usi ng the right-hand instructions

·t ill ha e the same ffect, if one starts ~ith} containing

13.1-1. Rather than adapt the cor struct ion in 13.1, it is j1 st as asy t prov e the 2 m3n5a z llw

more general theorem: An non ~a! ca panic; I syst m ha a no mal< xten- . 3

sian whose production co stant strin~ s g; a d h; t ave n p mo1 e tha1 two ·~ Yfore < etails are gi en in Minsl y [19( 1]. letters, with not more than a tot I oft ree ir any rodu tion. This prob- \ !em is fairly hard. My pr< of us s ma1 y ne\\ symt ols, i1 a hie rarch that

( 14 2-2. Define two n wins ructi< nsth; teach open te on two registers, rands.

gives a sort of binary repre sen tat ·an fo the c pnstar t strin gs of he or gina! / : A d 1 tc r,exc ~ange rand , got P Ij system. 1 : If > 0, subtr ct I f om r , exc ange r and s, go

to Ij; ott erwis exch nge r and s nd go to I k·

13.3-1. Append $4 to the right-hal d end of the an tee dent f 7r U· 1~wc nside r the f pur pr pgrarr s:

13.3-2. U uses only the upper-case letter A,C , S, a d T, nd lo wer-e sea a nd b. A A A B

The simulated alphabet c fQi ab, ( ab, a ab, .. . ' (a) b. So bb never A A B) A~

occurs. Then there will b no a mbigu ity if Ne re~ lace 1 , C, ~ , and T by '' (

A B s· B ------ B AbA,AbbA,AbbbA,and; bbbA and t he tes for f lsely )]aced uppe -case

letters still works. I B A B~h B

l db. b, aab

t t t 13.3-5. Construct P with only thr ee let ers, L a, ar Let a , aaal , etc.

represent letters of Q an< let l b, Lb b, Lb b, etc . serv as t e au iliary Ij Ij j lj h

letters of P. Finally, con truct U usi g LL b, LL Lb, L LLb, etc., as its If web gin w ith ev n nur nbers ·n bot h regi ters, then these programs are, auxiliary letters! Then on has )nly t cons truct relea se pre cedur e that ~or res~ on din gly, e actly like r ', s', - an d s- except with double-sized will not release any string ontai ning l in its interi r. 1 crem nts.

Tht same thing, essen ially, an bt done where the "normal" sequence 13.3-6. Take A and T to be new !etten : let 1 $ ass rt th; t $ is a the prem in the i then xt im tructi n for fj. C; n you see tl e difficulty with the rightmost

old system; then use r rogra [n, an< how o fix i by p ecedi1 g eve y program by "A, A, B, B"? T$ 10 211$ -+A $2A $ A $ 1( $211$ 3A 14. 3-2. ( I) No , beca se bo h 16 nd (4 have he sa [ne image.

A $ 1xA $2 A $31 -+A $,A$ A$v : ( ) Ye ; if a 1 urn be r is ev n, it i not< list; i it is odd, it is a list.

AAA $ 10$ 2 11hA -+1 $,0$2 ------( ) No , beca se bo h8a d (I) ave t e sarr e image.

I

s UG4 :aES TIO ~s 1 OR FURTHER STUDIES

'I ~ ) A ND DE~ ,cR PT4 )R- NDEXED B IBL lOG RAI HY

'

(

dNTRO >UCTI ON )

It s im~ ossib e to ive a full ccou nt of a mathematical theory-not

' < nly b ecaus e of he v~ st an ount of k 1owle dge accumulated in the past (as in the c se of COfflJ utabi ity)- but lso t ecause the process of assem-lling n ex ositi n, st a pin! and selec ing < ifferent versions and images leads o ne\1 vari tions and mode s. S< mec f these are inessential, others <rene w fan ilies of str LlCtur s tha t rais e nev problems in other areas of rna the matic al kn ow lee ge o igno ranee. To keep this book compact, it

I was nt cess a ry to inter upt t hese ines tumh , and often arbitrarily. The ollow ing n mark s con "ern ines < f tho ught that some readers may want

to pur sue f rther. T e in" tials at th rigt t of each topic are used to <ross- efere nee n e bib iogra phy.

,,· I LGEB RAIC HEOI Y OF MAC -liNES A

eginn ing m ainly with leene 's Re ular Ex pre sion Alge-ra, a new b anch of ab tract algebr a has devel Jped. It is OW 0 ten se en in them ore rr atherr a tical of th e computer

., <nd lir guisti jou nals. The few r eferen es w give here V ! ~howl OW tl e sta e-tran ition diagr msh ave b en studied

.{' y tre2 ting t he ini ut str ings < s eler 11ents of gr ups, semi-

! roups ,mon Jids, a nd ot er me dern 2 lgebr· ic ent ties. __ ,_/

297

298 READING SUGGESTIONS READING SUGGESTIONS 299

THEORY OF COMPUTATION c ARTIF ICIAL INTEL IGEN ~E I

There is very little theory toda to h lp or e pro ~e th t a The a uthor consi ers " hinki f"tg" t< be VI ithin the scope of

particular program computes a partie ~Jar f nctio f"t, or hat effect ve co rputa ion, a ndwi hes tc warn then ader against

two programs in the same or in iff ere rt Ian Puage com ute subtl defe tive a rgum nts tl at su gest hat t e difference

the same function. The fact tha the 1 roble In is r cursi ely betw( enmi ds an dmac ~ines an so ve the unso lvable. There

unsolvable in its most general fo m she uld n t be c onsid red is no evide nee fc r this In fact, here could 't be-how

a complete obstacle, and one ou ght to prOC( ed to a cia~ sifi- could you c ecide whett er a ! iven physi a!) m achine com-

cation of important solvable ases, as d )es A kerm ann putes a non comp1 table numb er? F igenb aum 1 nd Feldman [ 1954] for the predicate calculus. [1963 is a collec ion o sour epa ers ir the field of pro-

gram ning c ompu ers th at beh ave in ellige nt)y.

DECISION PROBLEMS ) PROG RAMI ~lNG LANG UAGE L

Some readers will want to purst e furt her st dy 0 recu sive Ther great man' ng Ia s born each olvat le but still f airly I

are a prog amm guag unsolvability and of the known rge year ~hich can b met n any issue of the JAC W, the Com-problem classes. We have just er ough ref ere nces f r the n to puler Journ 2/, the Com n. AC M, et. We have eferred here get started. Some of these poi t to ascim ting' arian s of only o tho se me st clo ely d scene ed fn m Po st Canonical the sort discussed in 14.7 (Wa g an his tuder ts tre a ted Syste rns an List-Proce sing~ ystem s. these problems as infinite two-d ·mens on a! ig-sa\ puz; les).

' INTEl MEDI I' TE II A CHI NEs M

FINITE AUTOMAT A F It is< bviot sly ve yimJ ortan to fi d the pries 1 hat apply to

This category includes just a fev. refer nces o top cs in nite class< s of c ompu ation lying betw en tl ose c f the simple

mathematics (e.g., switching th ory) hat b Faro the Ina in Finit Autc mata and tl e full class pf con putal le functions.

subjects of part I. Anu lnber < finte medi1 te cor cepts re sic wly tl king form-"pus -dow r auto mata, ' "lin ar be undec " anc "real-time" comr utatio ns, "c ounti g aut ornata ," an othc rs, many of

FORMAL GRAMMARS G whic rela1 e to ' ariou inter media e for mal g rammar (G) cone< pts. We a so in tude here c ertain in fin te "iterative

This field, emerging from early work s by ~hom ky 0 lin-array "and "groVI ing" nachi econ epts.

guistics, has become a signifi ant t ranch of m atherr a tics bearing both on programmin g Ian ~uage and theo y of NEUI OPH'I SIOLC GY N Automata between Finite and }ener I Rec ursive -the "In-

termediate Machines." Weh ave in ludec a fe\\ refer nces o par ers an d books that

>, migh stim late f urther stud of tl e ide of t e brain as a COIDI uter.

COMPLEXITY HIERARCHIES H

Within the computable functi1 ns, o tle ca find inter sting PROE ABILI TIC 1'1 ACHI NEs p

definitions of hierarchies of c< mple ity m asun s. Burn's [1964] theory, for example, class· fies f ~netic ns b the A V< ry irr porta fit pn ctical ques ion i tha of whether amounts of time or tape uset in t peir c pmpu ation: his "noi e" or othe phy ical r ali tie of I robat ilistic nature theory is interesting in the wa the arne bstra t stn cture --- -- can I e tole rated by th theo y. v e hm e refe renced a few arises from these two apparent! ~ diffe rent c ses. pape s tha appr< ach tl is qu stion from iff ere nt directions.

300 READING SUGGESTIONS

RECURSIVE FUNCTIONS AND I EGREES OF RELA IVE I NSOI V ABiliTY

This theory, only introduced i Cha ters , 10 nd 11, is already a substantial branch f rna hema ics. The [nost comprehensive treatment is Ro~ers [ 967], and ne should see also Davis [1958] and Klee~e [1952], f r other c01 nections with Mathematical Logi . D vis [ 965] ontains a splendid collection of reprints f imi ortan orig'nal s< urce

papers. The theory of Church [1936], f r example, gives pres ntation of effective computation ery d ifferer t (at first ~ ight) from any treated here.

TURING MACHINES

A number of references co nee n sli! htly d iff ere t formulations of Turing Machines. Fis her (1965b sort out lnany of these.

R

T

!IIBLIOC~RAPH'

f\cker[nann, Wilh lm (1954), olvab e cas s of he Decision P oblen, Nor h-Ho land.

~lum, Man el (1964), "A n achin -inde1 endent theory of rtcursiv June ions," Doct ral T esis, \1IT.

Bobrow, Daniel G. and Raph el, B. (1964), "A comparison o List- :>rocessing Computer Languag s," Comm. A CM 7, no.4 231- 40(April1~64).

~obrow, Daniel C. (19t6), "*ETEbR: < LISI interpreter f< r stri g mapipul2tion," The 'Progn mmin~ Language L SP: ts Op ratio1 and; pplict lions, MIT Press, Camb idge. July 1966.

Buchi J. R (196 ), "1 uring Mad ines nd tt e Entscheid ngsp oblen ," M, th. A1 nalen 148,201-21

~ullo< k, Tt eodor~ H. and korric ge, Adrian G. (1965), S ructu e ana Func ion iJ the '(Yervo s Sy terns of InVI rtebrt tes,". H. F reemi nand Co., 1 ew York.

Burks Arthur W and Wan~, Hac (1957), .. , he logic of a tomata,"J4CM , 193 218and27<-297.

Burks, Art ur " . (19 9), "romr uta tic n, Bt havior and S ructu e in Fixec and Gro ing (\uton at a," Self( rgani ing S stems Perg mon Press, 282-311.

Chomsky, 1' oam 1963) "Formal p operties of grammars," f. andb1 ok of Mathemati a! Ps cholof!y 2, John Wiley & Sons, 323- 18.

30

D

R,H

L

L

D

N

F, A

M

G,A

302 BIBLIOGRAPHY BIBLIOGRAPHY 303

Chomsky, Noam and Schutzenb erger, M. p . (196 3), " he G,A Go de , Kur t (19 I), " ]ber orma uner tschei dbare Satze D algebraic theory of context- ree I< ngua~ es," ~omp~ ter der Pri cipia Math matic a und verw ndter Systeme 1," Programming and Formal Syste ms, 1\ orth- Holla lld, Afonat hefte ur M a them rltik u 'ld Ph ysik 8, 173-198. 118-161. Guzm an, A :iolfo nd .rv clnto h, H (196 ), "C ONVERT," L

Church, Alonzo (1941), "The C alculi of L mbd2 -Con er- R (omm. ACM 9, no. 8, 60~ -615 ( Augu t 196(). sion," Annals of Mathematic Stud es 6, I rince on.

Haines, Le Jnard (196 ), "G 'ld re en era ion a ognition of G Copi, Irving M., Elgot, Calvin c. ad W ight, Jesse B. F fi rmal angua ges," Docto al Th esis, 1\ IT.

(1958), "Realization of even s by lc gical ets," IACJ.: 5, Harin g, Do aid ( 960), "The equen tial tr r1nsmi sion expres- F no. 2, 181-196 (April1958).

son fo flow grap s" echni al M emo. Electronic Courant, Richard and Robbins, H. (I< 41), ¥hat s Ma he- Svstem La be rator ,MP (Nov em be 1960

matics? Oxford Press. Hartn an is, J., L wis, P. M. and Stear s, R E. (1965), H,M Davis, Martin (1958), Computab ·tity a d Un olvab 'lity, vic- , T, 1: "Classi ticatio ns of omp tatio s by ime < nd memory

Graw-Hill. r quire nents "IF/ P Inte natio a/ C ngres . 1, 31-35,

Davis, Martin (1965), The Unde idabll , Rav en Pr ss, H ew- , T, 1: Spartar Books.

lett, N.Y. Hartn an is, . and Stear ns, R. E. (I 65), ' On t e computa- H,M

Synth I tiona! < ompl( xity c f algc rithm ," T! ans. " mer. Math.

Dertouzos, Michael (1965), Thr shold Logil: A sis Soc. (to bepu blishe :i). Approach. Research Monc graph no. 2, M IT Pr ess, Cambridge. Hebb, Don ld 0 (19~ 9), T: e Or ganizG tion tJ Behavior, N,l

" iley. Eggan, L. C. (1963), "Transitio grap hs anc the s ar-he ght F, I

of regular events," Mich. M th. J. 10,38 5-397 Henni e, Fre :ierick C. (I< 61), I erativ e Arrt ys of Logical Cir- M c its, M IT Pr ss, c mbri< ge.

Evey, J. (1963), "The theory an d app icatio s of J ushdo wn- ,.. Hoop

store machines," Doctoral The is, Ii eport NSF 10, r, Ph ilip ( 964), "The undec idabil ty of the Turing D

mach in imr 'lortal "ty p oblen " Docto al Thesis, Harvard University. H arvar Uni' ersity (Apri 1965.

Farber, D. J., Griswold, R. E. and P olons y, I. P. (19 64), L Hubel , Dav· dH. a ndW' esel, 1 . N.( 959), "Rec< ptive Fields N "SNOBOL: a string-mani ulati< n Ian guage " JA CM o Sing e Neu rons i 11 the ~at's triate Cort x," Journal 11, 21-30 (Jan. 1964). o Phys ology 148, 74-5~ I.

Feigenbaum, Edward and Felc man, Julia (19( 3), c om- L, I keno N. ( 958), "A 6 -symb ol 10 state Unive rsal Turing T pulers and Thought, McGra ¥-Hill 1963 .rv achin e," Pr oc. In t. of lee. ( ommz nicati ons, Tokyo.

Fischer, Patrick C. (1965), "Gen eratio n ofF rimes by a ne- , Kahr, Andr wS., \1oor ,Edw ard F. and\ Vang, Hao (1962), D dimensional real-time iterat ·ve ar ay," ACM 12, 88- "Entsc eidun gspro blem reduc d to the J EA case," 394 (July 1965). P oc. N r1tl. A< ad.S< ience, 48, 3, 365- 77 (lv arch 1962).

Fischer, Patrick C. (1965), "On Form !isms forT 11ring \1a- Kahr, Andr w S. 1963) "1m rovec redu tions of the Ent- D

chines," JACM 12, no. 4, 5 0-580 (Octo ber 19 65). S( heidu 11gspr< blem to su bclass es of AEA formulas," k athert a tical Theo y of 4utom ata, 5 7-70, Polytechnic

Freudenthal, Hans ( 1960), LI NC OS: t esign tfa Ia 'lguag for I,G P ess, B rook I n.

cosmic intercourse, North- H ollan . ""teen , Stet= hen C . (193 6), "( en era I recu sive unctions of R

Gilbert, E. N. (1954), "Lattice tt eoreti c PTOI erties of frc ntal F n tural numb rs," 11 Jath. 4nna!t n 112, 340- 53.

switching functions," Journ a/ of 'vi at h. and P hysics 33, --~ ""teen , Ste~ hen C . (19 2), b trodu tion o Me tamathema- R,D, T 57-67 (April 1954). ti s, Va Nos rand, Prine< ton .

. 1

304 BIBLIOGRAPHY

Kleene, Stephen C. (1956), "Re resen ation of e ents in nerve nets and finite automat ," A~ tomata Stu~ ies (Annals of Mathematics Studies. no. 34) Prine ton.

Krohn, Kenneth B. and Rhodes, J. L. (196 ), "A lgebr· ic theory of machines," Mathe malic I Th ory l'f Au ornata. Polytechnic Press, Brae klyn, 41-3 4.

Lee, Chester (1963), "The constmction of a self-d scribing Turing Machine," Mathema ica/ Jheory of A toma a, 155-164, Polytechnic Press, E rook! n.

de Leeuw, Karl, Moore, E. F., Sh· nnon C. E., and Shapi o, N. (1956), "Computability t y Probabili tic Machin " Automata Studies, Princeton, 183-212.

Lettvin, Jerome Y ., Maturana, H. R., M cCull xh, V . S. and Pitts, W. (1959), "What the Frog's Eye ells tr e Frog's Brain," Proc. IRE 47, 1940-1959. (reprinted in McCulloch [ 1965 ]).

Lottka, Alfred J. ( 1956), Element of A- a then atica/ Bio/olrry, Dover. (Original title: £/em nts of Physi a/ Bil logy.)

Markov, A. A. (1958), "On the inver ion complexity cf a system of functions," J A C l'v , 5, no. 4, 31-3 4 (0 tober 1958).

Mason, Samuel and Zimmermar, H. (1960), Elect onic 'ircuits, Signals and Systems, J hn Wiley.

McCarthy, John (1956), "The in ersiO of ft nctio s defined by Turing Machines," Az toma a Sndies, 177- 81,

Princeton.

McCarthy, John (1960), "Recu sive unctic ns of symtolic expressions," Comm. ACM 3, 13 -195 April1960)

McCarthy, John (1961), "A bas's for· mat! ematical th ory of computation," Proc. W< stern oint ~omr uter < onference(May 1961).

McCarthy, John, eta/. (1962), he L SP 1. Pro ramner's Manual, MIT Press, Cambr'dge.

McCarthy, John (1963), "A bass for mat emat cal th~ory of computation," (revisec and exter ded ersio of McCarthy [ 1961]) Comput1 r Pro'~>ranu ing a 'rid Fo mal Systems, Braffort and I irsd berg (Eds.), Nc rthHolland, Amsterdam, 33-70.

McCarthy, John and Painter, . (191 7), "( orrectness of a compiler for arithmetic expressi ns," Proce dings of a

F, A

F, A

T,R

P, T

N

F,~

c,

C, l

l

C, L, I

C,G __ -~

BIBLIOGRAPHY 305

ympo ium on A- a then atica/ Asp cts of Computer ~ cienc (Apr'! 196 ), Ar erica Ma hema ical Society,

967 (to be p .1blish d).

McC lloch War enS., and I itts, ,L-alter (1943 , "A logical c a leu its of he id as in manent in nervous activity,"

u//eti of r.tathe'rnatic£1 Biobhysic 5, 15-133 (reJrintec inM Cullcch[l965]).

McC lloch War en S. ( 195q), "A a the Tyche: of nervous r ets- he lucky re kane s," A- echanizatio of Thought

roces es 2, 611-6 4, H r Ma'esty'~ Stationery Office, ondof! (rep in ted in Mcrulloc h [ 19~ 5]).

McC lloch War en S. 1960 , "Th reliapility of biological systems," Se'f-Organizin SystPms, "64-281, Pergamon

ress.

McC lloch War enS. (1965 , Em odim nts o ·Mind, MIT ress, amb idge.

McNaught n, R bert and ~ amac a, H. (196 ), "Regular < xpres ions nd s*te gr phs or au oma*," Transacions o the I"f?_E PJOfessi na/ Group 1 n E/e tronic Com-

1 uters, EC-9 no. I 39-4 (March 191 0).

MeN aught )n, R. (196 ), "T e theory o automata, a surey," Advanres in Computer 2, 379-421, Academic ress.

Minsky, Marvin L. 0&56), "Som~ uni ersal elements for nite 3 utom · ta," / utom ta Stz dies, rince on.

Minsky, Marvin . (19 9), "Some n ethoc s of I- euristic Proramrr ing a dArt ficial ntelli ence, ' Pro1. Symposium n the Mech nizat on of nte//i ence. HM sO, London, -36.

Min ky, Iv arvin L. (1961), "Re ursiv~ unsplvability of ost's probl1 m of "tag" and bther opics in theory of urin! machines," Annasof!v.ath. 74,437 454.

Minsky, Marvin L. (1962), "Size a~d stmcture of universal urin1 mac! ines sing ag S stem ," Recursive Func

ion T eory, SymJ osia ir Pur Ma hema ics 5, A mer. ~ath. Soc.

Mim ky, Marvin L. (I< 63), ' Steps tow a d Ar ificial Intellience,' Con puter. and hougl t, Fei enba m and Feld

nan ( ds.), McGraw-H II. R print d from Proc. IRE Janucry 19( 1).

F,N,G

p

p

F, N, I, P,G

F, A

G,M,F,A

F

R, T,D

T

I

306 BIBLIOGRAPHY BIBLIOGRAPHY 307

Minsky, Marvin L. (1965), "Mat er, M ind a nd M odels,' I Ra in, lV ichae 0. (I 963), 'Prob abilistic aut ornata ," Infor- F, p

Proc. IFIP Congress, Vol. I, Sr artan Book> , 45-50. mati nand Cant of. 6, o.23 -245 Septe mber 963).

Minsky, Marvin L. and Papert, S. Per< eptr01 s: Th resholi c !

Ra Jin, M ichae 0.(1 963), 'Real time ompL tation ," Israel M Function Geometry, MIT Press, Cam bridg( (in p epar<J- J.M ath., 2 03-211.

tion). I

Ra I hevsk y, Nic holas (1938 , Ma he mat ·cal B ·ophy ics, Chi- N, F

Moore, Edward F. and Shannon, ClaU< e E. (1956 , "Re- p cago rev. ( d. Do ver (I 60).

liable circuits using less reliabl e rela s," J ourna of th Ra hevsk y, Ni holas (194( ), Ad vances and Appli at ion of F, N Franklin Institute 262, 191-208 291- 97. Mat ematl cal Bi logy, Univ. ofCh cago ress.

Moore, Edward F. (1956), "Geda ken- xperi ments on S(- F Ro Jinsor , Rap hael ~ rl. (19 48), ' Recu sion · nd de uble re- R quential Machines," Automat Stut ies C E. S annon ' cursi on,"/ m.M ath. S c. 54, 987-9 93. and J. McCarthy (Eds.), Prine( ton, I 29-15 .

-Iartle I

Ro 5ers, Jr. ( 1959) "Th( pres( nt th ory o f Turing T,R Moore, Edward F. (1964), Seque tial Vlachi es: elected F,} ,M mac ine cc mput bility," J . .s lAM '114 130.

Papers, Addison-Wesley. I

I Ro ers, l artie Jr. (I 966), he or ofRe cursiv Func ions and R, D, H, M, T Myhill, John (1960), "Linear bou ded utom ata," WADD M i E.ffe tive C om put abilit} , Me( raw- -1 ill (t J be ~ ublished

Technical Note 60-165, Wright Patte son A FB, C hio. in 19 67).

Newell, Allen, Shaw, J. C. and Si non, rl. A. (1956 , "Tt e L Ro enbla tt, Fr nk ( 962), "A c Jmpa is on f sev ral per- N logic theory machine," IRE Tr ansae ions o n Info maticn cept on m odels " Se f-Org ~nizin g Sys terns, Spartan theory /T-2, no. 3, 61-79. Boo s, Wa shing on.

Ott, G., and Feinstein, N.H. (196 1), "[ esign of se uenti I F Sd utzen berge , Ma co p (196 ), "( n co text- ree Ian- G,M machines from their regular ex pH ssion>," }A CM guaB es an d pus h-do\\ n aut om at

, In forma tion and no. 4, 585-600 (October 1961) Con ro/6, 46-2 4.

Papert, Seymour and McNaughtc n, R. (1966 ), "0 n top - F, A Sd utzen Jerger , Mar o P. ( 965), "Onf nite n onoic shaving A logical events," Theory of A utoma a, U iv. o Mich. only trivi I sub 5 roup ," In forma tion G nd C ntrol 8, Engrg. Summer Conferences. ' 190- 194.

l Peter, Rozsa (1951), "Rekursive Funk ionen " Ak ademiai R Sha nnon, Clau e E. ( 948), "Am them tical heory of com- F

Kiad6, Budapest. mun catior ," Be I Sys em 1 ech. ourna 27, 379-423,

ction ofth al com- R,D ! 623- J56.

Post, Emil L. (1943), "Formal redt gene binatorial decision problem," Am. ourna I of M ath. 65, Sha nnon Clau de E. (194 ), "S ynthe sis 0 two- terminal F 197-268. switc hing-e ircuit ," Be I Sys em 'T< ch. J urnal 28, 59-

Post, Emil L. (1946), "A variant c fa re ursiv ly un olvable D 98 (J nuar 1949.

problem," Bull. Amer. Math. oc.5 . 264 268. Sh· nnon Clau de E. (1956 ,"A unive sal T uring machine I

probl d D, G, R with two ntern I sta es," Au torr ata .S tudies (Annals

Post, Emil L. (1965), "Absolutely unsol able msa ofM ath. S udies 34), p incet n.

relatively undecidable propos tions -accc unto an an-ticipation," M. Davis, The Undec 'dable (m.s. unpub- Sh( pherd son, J C. (I 959), 'The educt on of two-v ay auto- F lished, 1941 ). mat< to o e-wa auto mata, ' IBJr J. 0 Rese arch and

Quine, Willard (1960), Word and ( bject, Wiley. Deve lopme 'It 3, n D. 2, I j8-20 (April 195' ).

Rabin, Michael 0. and Scott, D ana ( 959), "Fini te au o- F, D Sh( pherd son, J . C. a nd St urgis, H. E (196 ), "( omputa- R, T

mata and their decision probl ems," IBM Journ I of f. e- ility f rect rsive uncti ns," . Ass 1c. Cc mp. A 1ach. 10,

search and Development, 3, n . 2, I 14-12 (Ap il 195 ) . ------217-255.

"

308 BIBLIOGRAPHY

Smullyan, Raymond (1962), TJ eory 1 Fe rmal Syste (ns, R,D TA ~LE PF s PECI ~L S YMI3 OLS Princeton. I

Solomonoff, Ray J. (1964), "A fo malt eory pfind 11ctive m- I ference," Information and C£ ntrol , no. I, 1-~ 2; vol 7, $ A varia pie st ing o letters, 228 no.2,224-254,~arch-June '1964. B 2C7

Tarski, Alfred (1951), A Decisi nM thod orE ement pry ll ""'

2(8

Algebra and Geometry, Univ ofCa liforn a Pre s, Ber ~e- I - 2C8 ley. ' T e NI n, 19 < ) L ato

Teitelman, Warren (1966), "PI OT, a ste tow rd m an- I <x,, x2 ... ,x. > List, 19~

computer symbiosis," Ph.D. hesis ~IT <A. B> 1 <; 6 --o Inhibiti on,3

Turing, Alan ~. (1936), "On co mputa hie m mber , with an , R, 1: - E citat on,35 application to the Entschei< ungsi roble n," p oc. L on- v 0 '71 don Math. Soc., Ser. 2-42, 2 0-26' * M repec ted in Chapter 4, but used later eans

Turing, Alan M. (1950), "Comp llting VIa chi nery nd In tel- si nply to d sting uish between things, such as

ligence," Mind 59 (n.s. 236) ~33-4 0, als o The Worl of mach in es M and 1\ 1*, 71 Mathematics 4, Simon and S hust' r, 195 . 0 2( 1

von Neumann, John (1956), "I robab ilistic logic and the F, I - 1< 3, 20

synthesis of reliable organ 'sms f om t nrelia hie c1 m- + 1 4

ponents," Automata Studies Prine ton, 3-98. X 1' 4

Wang, Hao (1957), "A variant o Tu ing's theor of c R, 7r Denote s a I ost 1 roduction when not used to pm-

n:prese nt 3.1 ~159 .. ; 229 puting machines," JACM 4, no. I.

f.l,f.l( !x, N) T~emi ~imiz ~tion operator, 184 Watanabe, Shigeru (1960), "On mini mal L niven a! Tu ing a" 1 8

~achine," MCB Report, To kyo. 1> A recu sive f ~ncti n being defined, 178

Watanabe, Shigeru (1963), "Per odicit lv of I ost's systen of 1/J B se fu nctio ' 175

Tag," Math. theory of Au to mat , Po ytech ic P ess, ® ,,

Brooklyn, 83-99. A state hex a on a d a state transition arrow, 21, 12.

Yamada, H. (1962), "Real-time COm{: utatio nand recur sive ~" i ()-- AMcC ulloc 1-Pitt cell and an output fiber, 33 functions not real-time co nputa hie," IRE Trans. on Electronic Computers EC-11 '753- 60. ~ I I I I I I ~

Y ngve, H. (1962), "CO~ IT a nforrr ation Retri val L rn G A Turi g rna chine reading its tape, 117

an language," Comm. ACM 5, no. I, 19-28

I 5 Yngve, H. (1963), "CO~IT," ( omm. ACM 6, no . 3, 8 -84 L

(~arch 1963).

------3(9

l

IN DE " Ar ~D C: LOS )AR'

I ~ Bla k box 13 Bla k tapt '150, 262

i

a, a', a- ( rl. 201 Bol row, I aniel, 198, 239 Ac ept, s nonym for Re ogniz ' 131 Bo< lean a gebra, 173 Ac erma n, (fur ction), 186,2 4,215 226 Bra in, 32 Ad ~ition, (see a/ o Bina ry), 22 3, 30, 4ff., Bu< hi, J. I ., 156

7 Bul ock, 1 heodore, 65

I Ad dress, in men ory), 39 Bu ks, Ar hur W., 35 AI! orithrr , synm ymfor Effect ve pro cedure

05 c AI habet, 222 An log, 2 c, onstru ct function, 177, 180 AN '37 Cal '81

i An ecede t, the eft-ha d par of a 1 roduc Ca onical , mathematical term for describ-ton, 231 i gag< neral method for solving a prob-

Ar urn en , (of a unctio n), oft n calle~ I m tha does not take advantage of fea-ariabl -but ever i this t xt, 13 t~res th at may give simpler solutions in

Ar' thmeti '44ff. 1 articul ar cases; a standardized form, 53-Ar· thmeti ation, reduci ng pre blems in an 8

< ther a ea to robler s abo ~t arit metic Ca onica extension, 236ff. s e Red ucible, 169ff. Ca onica system, Post's abstract formal-

Ar ow, 21 i ms fo describing logical and mathemat-As ociati emem pry, 128 i al the ries, 220, 232 At m, us d here to mea ninde ompo able Ca tor, C eorg, 149, 161, 190,221

s mbol ·c objec t, 196 Ca ry, see Addition Au ornate n, usee in rna them a ical lit erature Ca chy, I 58

"\ c n theo y ofm achine to me an fini e stat< Ce l,(Mc ulloch-Pit'ts), 12, 33 c r in fin te but discre e mac ine. 1 sed in Ch nne!, 3ff. ron teet nical I teratu e to 5 uggest clock Ch rch, P lonzo, 108, Ill I ke pre ision < r rigid ty of b ehavio ,II Ch rch's hesis, see Turing's Thesis, 108

I Au iliary etter, 35ff. Co ke,Jo hn, 200, 270ff. I Ax om,!( 6,221 '222 Co [npiler , a computer program that trans-

I Ax om sc em a, 24 I tes fr< m one programming language to Ax omati systen '221 · nothe r; the second language is usually

I t~e basi language of the computer, 206 Co fnplete state, see Total state, 170

I Co [npone rt. technical meaning in theory of < iagran s: a part completely connected

Ba zer, R bert, 82 ithin 'tself but disconnected from the Ba e, (of inducti n), 74 r st of tne diagram, 24 Ba e, (of urn be systen ),sym nymj. r Co mposit on, combining functions so that

adix, 3 the val e of one is an argument of an-Bir ary, (c ounter or sea I r), se e Scaler 143 c ther, I 73 Bir ary, ( uring vtachir e), syn nymj. r Co mputa ility, short for Effective computa-

wo-sy mbol n achim '129 t ility, I 04 Bir ary n mber, see A dditio Mu tiplica Co mputa ion, the sequence of events in

--------tion, 2 2,30-3 I computing something, 119, 183

~11

a.

312 INDEX

Computer, usually refers to a real (a opposed to an abstract) digital com uter, 24, 29, 46, 51' 64, ll3ff., 153, 192, 99ff., 219

Conditional, a point in a computer pr gram at which the next sequence of instru tions is chosen according to the outco re of some test, l92ff., 20 l, 203

Consequent, the right-hand part of a pro-duction, 231

Constant (input), see Isolated machine Constant (function), 174 Converter, device to convert from se ial to

parallel (or vice-versa) pulse coding, 46ff., 123

Copi, Irving, 35, 97 Correspondence problem, 220, 274ff. Countable, see Enumerable, 160 Counter, 41 Cycle, see Loop

D

D, the Turing-machine tape-head rr oving function, 118

Dr, 136 dr(t), defined as l - D(t), 172 D(n,x), 181 Davis, Martin, 118, 222, 277 Decision, (procedure or machine), 78, l46ff.,

l52ff., 225 Decoder, 46ff. Dedekind, Richard, 157 Definition, 3ff., l32ff. Delay, 20, 22, 30, 35, 36 Derivable, (immediately), 222, 223 Dertouzos, Michael, 66 Description, l 04, 112, 162, 251 Diagonal, (method of Cantor), 149, 161,

190, 192 Digital, in machine theory means

finite-state properties as contraste continuous, infinitesimally adj properties, ll, 29

aving with

stable

Discrete, separate, non-continuous. n our context almost synonymous with I igital, 12

E

E, symbol for environment except when used in Kleene algebra (p. 71), as in EvF, 14

Ex, he eve funct on, 181 Effe tive (p ocedu e, con putati n), ex

pl ined a lengt in Ch p. 5, l 05, 13 219,221,222

Effe tively enume able, like E umer· blew'th the added requir ment hat th oneto one rr atchin can pe des ribed in an efective q.v.) n anner 160

Effie iency, 45 Elgc t, Calvin C., 5, 97 ELSE, Part >jCon itiona, 192 Enc der, 4 If. Enu nerabl , a countabl or enumera le in

fi ite set is on whos members c n be m · tched one-tJ-one, with he in egers. S me in nite s ts, lik the eal nu~bers o the pc ints o a lin , can't be co nted, [(0

Enumerati n, an· ctual ounting of an er umerable set, 149, l 7ff.

Env ronment, 14, 19, ll~ Equivalence class a class of th ngs that are

ec uivale t to o e a no her ac ordin to a relation "equiv' that has t e following p operti s: (i) If a eq iv b A ~D b quiv c tt en a e~ uiv c; ( i) if a quiv b then equiv a; and (i i) every a is quiv tp itsel . Disc ssed ir any rr odern ext on sets o algeb a, 16, 7

Euc id,22l,226 Exc tation, excita ory, 3 Ext nsion, short for c nonic I extension,

2 6ff.

F

F, I (q, s), the fu ction that d scribe state c anges, 15

F+ x,y), the addition f nctior, 13, 6, 19, l 3, 174

Fx x,y), he multipliC' tion f nctior, 173, l 4

fn(x), the nth par ial rec rsive functio , 188 Fac'litatio , 38 Far-out, 51, 59, ~6 Farper, 23 Feedback, 39, 55 Fei enbaum, Ed' ard, 7 Fei stein, 6 Fernat's last Theorem seep. 164 fc r defi

nition. 116, 16 Fib~r, 33

Fie, (of 'nform tion), 126 Finite, (number , a nu nber o e can actuall

ount p to, given er ough fme, l Finite automate n, synonym or Finite-sta e

machi e, ll Fif!ite-st te, ll Fi ing, 3 Fi cher, atrick C., 28 Fl p-ftop 40 Ft nction 132ff. 193

G, G(q, s , the f1 nction that *scribes a mahine's outpu , 17, l , 20

g; const nt str'ng of etters in productio ntecec ent, 2 0

G(n, x), I l G< te, 37 General r cursion, 169, 183, l 4, 210 215 Geometr , (Euc idean) 221 Gibert, E. N., 6 GO 203, 82 Gt del, K rt, 161, 182, 22 Gtdel nulnber, 25,25 Gcto, E., 29 Gr mma, 220 Guzman, Adolfc, 239

I

H, head f nction, 177, 80 H, symbcl for hi tory i Chap 2, me ns

'halt" fvhen a tape-s mbol, 15 h;, const< nt string of etters in pro~uctio

onseq ent, 2 0 h(1 ), the alffurction, 71, 17 , 180,203 h (t), histc ry of e ents up to m ment , 15 HAT, 119 183 Halting p oblem l46ff. Haring, C onald, 96 Hebb, Donald 0., 66 He oper, hi lip, 03 Ht bel, D vid, 6

IF ... THE ... ElSE, see Condi ional, 192 1-li t, 195 Inconsistent, 166

INDEX 313

Index register, 208 Inductio , (mathematical), 73-79, 173, 174,

179 In :luctive step function, 175 In nite, ee Enumerable, 160 In~nite n achine, 75, I 14 In ibitio , inhibitory, 33, 35, 58 In out, (c anne!), l3ff. In tant-oR, 86 In tructic n, 105, 199, 202ff. In tructic n-number, 202 In elligence, 2, 7, 55, 66, 192 In ern a! state, usually abbreviated to

'State, ' 16 In erpret r, a computer program that obeys

ules e pressed in a language different rom tl e computer's own "machine lanuage,' 112, 137

In erter, I IPl, a list structure programming language,

98 I soIa ted n achine, 23ff. Ite ation, 210, 212

J

J( ), spec al function, 181 Jur p, l9S If., 203 Ju taposi ion, synonym for Concatenation,

l

K

Ka r, An rew, 226 Kltene, S ephen C., 67, 79ff., 94, 97, 108,

Ill, II(, 118, 185, 187,200

L

La guage (programming), l96ff., 199, 20lff. Learning, ee Intelligence, 66 Le , Ches er, 145 Let (-mo ing), see Move, (tape) Let vin, Jc rome, 65 us , a list structure programming language,

a well< sa formulation of recursive functions, l' 6ff., 198

Lis , list-structure, 195, 196 Lo~ 2 (n), t e (possibly fractional) number of

times 2 must be multiplied by itself to y eld n, 4

314 INDEX

Logic, 219ff., 221 Logistic system, 222 Loop, synonym for Cycle; a circular losed

ordered sequence of things. We are most concerned with loops of states (p. 3ff.), but we are also concerned with lo< ps of parts (p. 39ff.) and loops of pr gram (p. 203ff.) See also Memory and I ecursion, 24, 39, 55, 204

Lottka, Alfred, 7

Me

McCarthy, John, 28, 66, 78, 192ff., 19! McCulloch, WarrenS., 32, 35, 38, 66 McCulloch-Pitts, refers to the paper o

Warren McCulloch and Walter Pitt , (1943),33-67,95,97

McNaughton, Robert, 97-99

M

m, the left-half tape number, 170 M, M*, MT, etc., symbols for vario s ma-

chines Machine, (concept of), Iff., 103, 220 Majority, 61 Markov, A. A., 65 Mason, Samuel, 96 Mathematics, 3 Memory, 15, 21, 24, 39, 66, 199ff. Metamathematics, 166, 219, 222 Minimization, technical definition on p. 184,

see also pp. 185, 193,210,213 Minsky, Marvin, 7, 28, 61, 66,276 Modus ponens, 248 Moment, means an instant of time; se

Event, 12 Monogenic, means generating just on

thing, 237, 269ff., 274ff. Monotonic, a device whose output n ver in

creases or never decreases when it inpu increases, 62

Moore, Edward F., 28, 65, 66, 226 Move, (right or left on tape); for Turing rna

chines, we always think of the tape as sta tionary and the machine as movir g, 117 118

Multiplication, 26ff., 46, 125 Myhill, John, 28

N

n, 170 N(x , then ll fun• tion, 1 4 Nan es, 20( NAN!>, 61 Neu a! net ork, sed h re to mean net-

w rk of interc nnect• d Me ullocl -Pitts ce Is, 33

Neu on, us din b ology o defir e the c lis of th nerv< us syst~m wh ch con~uct el ctroct emical waves along their urfact s and fil ers. I this ook, i some imes !neans " ~cCullpch-Pi ts neu on" -~escrit ed in lY cCullo h (19< 3) and define in Ct ap. 3. \\ e will princi1 ally u e the word "cell" in this Inter m aning, howe er. S e pp. 3 '39, 6 , 65

Nev ell, AI en, 19 NIL, the "z ro" lis -struc ure, I~ 5 Noi e, tecl nicalh descr bes a1 y phej:wme

n n or influen e that migh distu b the i< eal op ration of a sy tern, 29, 51

N 01 -deter jninate, used i mach· ne thepry to d note~ mach"ne wh se be avior is not e tirely pecified in tl e give1 descr ption. 1\ ust be distin uishe< from probal ilistic. ~ ot dis ussed in thi~ text, but see (for e ample Rabi ,(196 ).

No mal, ( ost sy tern), 40, 26 ff., 27 ff. No mal prpducti n, 240~. Nu 1 funct on, 17 Nu I sequ~nce, he "t ivial" abstr: ct se

q~ence f no s mbol: . It < oes n< t cons me a tnome1 t of ti!ne. \\ e use t only v hen it is mat emati ally rr ore aY kward t do wi hout i , 72, 9

Nu I sign: I, a moment at which m pulse ef!ters : McC~IIoch-Pitts ret. I does c ccupy full r omen of tin e, anc is unl"ke a ull se uencef--whic~ has to do v ith the expre sions r prese ting a event r ther t an the event tself, 49

c Op ration any c f the pasic comput tiona!

f nctiOI s avail ble in the "h rdwar "of a cpmput r, 201

OR, 38 Ou put, (c~annel , 13ff. ___--

p

Pn, he party fun tion, 171, 17 , 203 Pn ( ), the th primitive recurs ve fur ction,

1!9 Pali drom , a re\ ersible ex pre~ sion s1 ch as

":blewa Iere sawElba,"<28,23t Pap rt, SeJ mour, 6, 97 99 Par: dox, 1 9, 156 162, 166, 221,222 Par: llel, (c nvertc r), see onve ter, 49 Par: meter, an argument of a Junctio that

de es not chang durir g the ompu ation being dis ussed 175

Pan nthese , 74ff., 130, 1 8, 230 Pari y, me~ ns eve (-ness or odd( -nes~) of a

m mber, 20 Part refers to par of a lnachine cons dered

as a sim ler su -machine, H, 18, 1~, 32, 3

Part al (rec rsive unctio ), 186 Peri die, t chnically me ns ex ctly r peat-

in , 24 Pete , R6s a, 186 Pitt!, Walt r, see ~cCulloch-Pi ts (19< 3), 33 Poir sot, 7 Post Emil, see C nonic 1 syste~, 111, 118,

219ff., 2 1, 260, 267 Post~late, 21 PR, t~e prir itive ecursi' e sche(na, 17 , 177 Prec ecesso , n - 1 if n 2: 1 ( ero h s no

predeces or), g 3, 201 Prin e num~er, 23 , 256, 17s9ff. Prin itive ecursic n, 11 , 169, 174ff. 183,

2( 9ff. Prin ing, (1 nsolva~le problem), 152 Pro! abilist c, a machine whose beha ior is

no t enti1 ely de ermino d by "ts sta e bee~ use it includ~s ran~om v~riables not c< nsider d to b part < fits st te, 15

Pro ess, 22p,232,269 Pro essor, 000

Pro uctior, (Post , 220, 27, 23b, 232f . Pro ram·c mputc r, deli ed in 1.1; fo digi

ta I comr uter, j~e Co rputer 113ff, 199, 21 Iff., 2 I, 237, 255, 2 9ff.

Pro ram-n achine, syno ym f r Pn gramcompute

Pro ramm ng Ian uage, 103, I 2, 19!, 204, 2 9

Pro f, 166, 222 Puhe, 33

INDEX 315

Q

q, Q always denotes a state, 16, 119 q(t), Q(t), tate of a machine at moment t.

T ere is no consistent distinction in our me of ur per and lower-case q's.

q;. ';. give some numbering of the possible st tes of machine, the ith one.

Q;( ), 76 q', q'ij· the new state replacing Q or q;, 122 Qua~ruple 170 Quit, 33 Qui tuple, 119

R

r, Ji, usually denotes response or output si nal, 1

R(t , outp t (response) signal at moment t, 14

r;, J ;. given a numbering of a machine's p ssible putputs, the ith one

R~y 80 Ru, he uni ersal real number, 159 Rab n, Michael, 123 Rad x, the "base" of a positional number

sy tern, .g., ten for ordinary decimal m mbers two for the binary system most of en use~ in this book, 43

Rap ael, B rtram, 198 Ras evsky, Nicholas, 66 Rati nal n1 mber, 157, 158 Rea , (tape), 107, 117 Real numb r, 157ff. Rec• gnize, technically means to identify a

se uence of symbols as belonging to a ce tain sc t of sequences, 69, 131

Rec rsion heorem, 145 Rec rsive, definition), 73-79, 186 Rec rsive fpnction, 169ff., 184 Red cible, !54 Refr ctory, 61 Register, art of a computer that can

relnembe r a number, 200 Reg Jar ex ression, 72ff., 226 Reg Jar set (of sequences), 72ff., 226 Relalv, 65 Rele se, ( o string), 236, 238 Reli bility, 66 Rep esent, ~9 Rest iction 73

Ill

316 INDEX INDEX 317

Right (-moving), see Move, (tape), I 2-3 Succe sor, (f unctio ), has value n + I for U iversa Univers

Robinson, Abraham, 186 ment n, I 3, 201' mac hine, see a so I w arg Turing machi e, 112 , 251, 53

Rochester, Nathaniel, 123 Switc , 13, 3 , 48 U iversa Turing m chine, 132ft., 18 W ng, H 0, 35, 170, 200, 225, 262

Rogers, Hartley Jr., Ill, 153, 155, 166, 226 Symb I, 108 I 706ff., Symb lie, (e pressi n, fun tion), 196 76ff. w tanab , Shigeru, 268, 276

Rosenblatt, Frank, 66 U solvat RPT, the use of this operation was Synta , 226

le, s nonym for Unde cidabl . w ight, J esse, 25, 97 sug- Both erms are te chnica ly sh Jrt for w ite (on tape), 117 gested by John Cocke, 212ff. ecursi ely un solvab e, 113, 146ff., 163ff.

Rule, 132ff. T y

Rule of inference, 221, 222, 226 Russell's Paradox, 149, !56

T, tail functi n, 177 180 ' Yr gve, V ·ctor, 238

t, syr nbol for dis rete nomen of ime, s (ah ays ha s integ r valu ), 12, 4 V(n, x), I 89, 19( z

Tm, n• Tq T,. Turin~ tran forma ions Va lue, (o a fum tion), 32 s, S, denotes stimulus or input si nal. fun ctions, 176 VO Neur ann, ohn,6 , 63, 6o Zero fun tion, 173

Upper-case symbol is occasionally use d to Tags stem, 67ff. denote the input channel, 14 Tape, 107, I 7

s(t), S(t), input stimulus at moment t. ep- Tarsk i, Alfre d,225 resents successor function in Part II , 14 Teitel man, V arren 239

s;, S;, given some numbering of a mach ne's THEN part o Cond tiona!, 192 possible input signals, these represen the Theo em, 22 I, 223 ith one, 174, 201 Theo em-pr ving n achin '226

Scaler, a device that counts up to a ce tain Thres hold, 33 number and then resets and starts ove , 40 Thres hold lc gic,46 64

Scan, (tape), 117 Total (recu sive fu ction) , 186 Scott, Dana, 123 Tran fer, sy onym or Jurr p,203 Semantics, 226 Tran ition, neans hange of stat , 20 Serial, (converter), see Converter, 46 Tran lation, used ere to mean conve sion Set, 134, 220 fro m one o nota ion to anoth r, 206 Shannon, Claude E., 65, 66, 129, 276 Tree, 180 Signal, 13 Turin g, Ala M.,l )4, 109 , 118, 19,23 Simon, Herbert A., 198 Turin g-com utable , 135ff , 211 Simulate, one device simulates anothe r by Turin g mac ine, 1C7

reproducing all the important steps i the Turin g's Th sis, 10 '136, 145, I~ 6, 153, 169, other's operation, 137ff. 18 , 226

Smullyan, Raymond, Ill, 260 Turir g tran forma ion, tl e effec on th e (q, Soldier problem, 28ff. s, .,, n) descri tion y a tep 0 the Solomonoff, Ray, 66 Tu ingm chine, 172 Square, (of tape), 107, 108, 117 Two-symbo , (Turi ngma< hine), 29 Start, (fiber or pulse), in Chap. 4, plays

special role of fixing events relative to a particular moment in time, 47,68 u

State, synonym for Internal state, see also Total state, II, 16 U, us ally d notes unive rsal m chine 136

State, (of mind), 109 U(n, ), 188 189 State-transiti~n, means change of state, 20 Unar , (nurr ber), t he non -positi nal fi ger-State-transition diagram, 21 cou nting r umber systerr '43 Stored-program, 201 Unco untabl , not nume able ( .v.), I 60ff. . String, (of symbols), 219ff., 224, 225 Univ rsal se s of c mput r oper ations, 112 Sub-process, 123 Univ rsal se s of el ments 58ff. Subroutine, 204 Unde idable ,see U nsolva le, 11 Subset, 134 Unde fined, ( unctio n), 184 '186 -----

i

Documents

[Marvin Lee Minsky] Computation, Finite and Infini Copy