Critic SPL

Embed Size (px)

Text of Critic SPL

  • 8/8/2019 Critic SPL




    Department of System ScienceUnivers ity of California, Los AnB;eles, CA 900241. IntIDduction

    My purpose is to survey th e o rlg ln s o f th etheory of formal languages and automata through1964 and t o i nd ic at e some of th e main directionswhich th e :-;ttldy of th e sub4ect has takc::>n since tIlen.In th e discussion o f o ri gi ns , I shall concentrateon those I know best: the work on mathematicall inguistics and automatic translation of th e 50sand early 60s, which le d me in to the field . I hadhoped to trace th e developments since 1964 thIDughth e papers presented a t th e IEEE Symposia on th eFOill1dations of Computer Science ( ini t ia11v, Switchin g Circuit Theory and Logical Design; la te r ,Switching and Automata TI1eory) and th e companion,parvenu, AC11 Symposia on th e Theory of Computing.This good intention was impossible to s us ta in , part icular ly as I moved fIDm th e exuberant 60s intoth e grim 7Os. LDng gone are the days in which ab stracts o f o ne 's summer research and t he t he se s ofo ne 's b es t students automat ical ly appeared a t th enext Conference! St i l l I shall give pride of placeto such cont1'ibutions where possible.

    There (:il'e at least f ive (not altogether dist inct) sources foY' th e ideas developed in fonnallanguage theory. They Eire:elogic and recursive func t ion theo rys\vitching cil 'cuit trleory c]IKlemodeling of b i ol og i ca l systems.,

    developm::ntal systems andeITBthematical and computational l inguisticscomputer prograrmning and the of ALGOLand other Problem Qr)iented lrl.nguages

    Recursive func tion theo ry is ;:3urveyed elsewhel'C inthese Proceedings so I shal l mention relevant ideasonly in passing. The emphasis of this paper is onformal languages, so I shall descrilR only th e l'el at ed p a rt s of th e development of f ini te automatatheory, skimping 01' skipping purely machine 01 ' system oriented topics.Phrase structure gpammars wen=? originally clescribed by Chomsky [l959a] as a forrna.lization ofth e Irrrrnediate Constituent (IC) analysis used by l in-tWists ~ " ' 1 describing th p morpholog:/- aDd syntax' o fnatural languages. The work on :rn::lchine translation

    a t various inst i tut ions used varioustheories as bases for automatic syntactic analysist Th .lS paper was supported i l l part by th e NationalScience Foundation under Grants NSF MCS-78-04725and NSF MCS-79-01439.

    and appropriate programming techniques were devised.At th e same time, similar methods of syntax specification and programming techniques were used inth e defini t ion and implementation of Problem Oriente d Programming Languages. In th e early 60s, a l lo f th ese var ious threads were brought together asi t was recognized, and then formal ly proved, thata l l these models defined th e same class of languages,namely, th e family of context-free (CF) phrasestructure languages.

    TI1e rest of this paper is divided into threemain sections chronologically: p rio r to 1956; 1956to 1964; 1965 to th e present. The year 1956 sawt he publ ica ti on of th e t en ta t ive defin it ion ofphrase structure grammar [Chomsky,1956], though th eChomsky hierarchy of languages and ITBchines did notappear in pr in t unt i l 1959 [Chomsky,1959 J. Theyear 1964 i s a watershed: th e Chomsky hierarchy oflanguages, grarmnars and machines was completed.After tha t , formal language theory as a disciplinediverged from mathe:rn::ltical or computational l inguist ics . Most (though no t a l l ) of th e subsequent de velopments were inspired by ideas r el at ed t o computer rather than natural languages and to computin g and programning. Thus, fr om approximately 1965on , one can regard formal languaF\e t heory a s abranch (originally th e main one) 8f theoreticalcomputer science.

    Past 1964, time and space allow me to touchon only a few themes and mv references become avery spa rse subset of th e available l i tera tu re . Ishal l , no t unnaturally, dwell mos t on th e d i r e c t i o n ~ ;in which my oVJTl research went.. Before Ctlomsky: 1936-1955

    As evel'y computer science student should kr10v],th e Chomsky hierarchy in i t s standard formulationcon si st s o f foul' c la ss es o f languages, each classp1'operly conta in ing the next: recursively enumerab le, con tex t -sensi t ive , context- fr ee, r egular.These classes can be defined by placing an increasingly s t r i c t set of restrict ions on general rewritin g systems or equivalently by consider ing thelanguages accepted by nondeterministic machineswith an increasingly restricted type of data structure (o r Horking tape): 'I\.lring maChL1J.e, l inearbounded automaton, pushdown store automaton, f ini testate automaton. 1he formal proof of equivalenceof grammars and IIBchines was completed in 1964,bu t some of the definitions ar e much older.

    CH ]471-2/79/0000-0066$00.75 ( ~ ~ ) 1979 IEEE 66

  • 8/8/2019 Critic SPL


    anThe types of machines and grarrnnars used todef in e the c la ss o f r ecur si ve ly enumerable languageswere defined and studied in th e 30s and 40s byTuring [1936J and Post [1943,1947J, respectively.Many of th e notions I survey here are now presentedin quite different formulations from those in th eoriginal papers. By contrast , Turing's definitionof a "Turing TlBchine" , i ts jusTi fi_c:at-ion and th econstruction of the f i rs t univer'lsa1 Turing machine,are l i t t l e different from th e notation we us e now.

    The -halting problem is no t explicit ly discussed; onth e other hand, th e tabular description of machines(frequently with added documentation or comments!)is more convenient than some I have seen in recentbooks. "No attempt has yet been JIBde to show thatth e "computable" numbers include a ll nlIDlbers whichVJould naturally be regarded as computable. All (1rguments that can be given are bound to be , funrl amentally , appeals to in tuit ion and, for this reason,rather unsatisfactory ITk1thematically. The realquestion at issue is , 'What are t he possi bl e processes that can be carried ou t in computing a number?' "[Turing, 1936, p 249J.t

    The generation of sets by rewrlt lng systems,called production systems, ~ , v a s formalized by Post[1943J who observed that th e sets so generated arerecursively enumerable sets and v ic e ver sa . Theparticular type of production sys tem used by Chomsky was defined and termed "semi-Thue" by Post[1947J who g ive s th e natural mapping from Turingmachines (Post uses quadruples rather than Turing'squintuples) to semi-Thue systems. In 1936, Postgave, independently, an informal description of computat ion quite s imil ar t o Turing's and declaredthat proving enough formulations equivalent to recursiveness would change Church's thesis "not : ~ much to a definit ion or to an axiom bu t to a natu-pa l law" [Post, 1936,p 105 J.

    Thus, th e grammars (o r generat ing systems)and machines at th e "top" of th e Chomsky hierarchywere known, and known to be equivalent, by 1947.Another paper by Post from this periOd la te r provided th e key tool used in proving problems in forITBI language theory undecidable: th e CorrespondenceProblem [Post, 1946]. (It is nlITIored that thispaper was inspired by World War I I experience incryptography! )

    At th e "tJOttom" of the h iera rchy l th e f 2..rH-i ly of regular languages f i r s t defined ?y Kleene[lQSl and 1956J. Kleene's formulation lS based onth e paper of McCulloch and Pitts [1943J on th elogical analys is of nervous activity in which certain asslllTlptions regarding nerve nete:: were formulated. In Kleene's paper, f ini te automata arerepresented by nerve nets. Instead of words orstrings, th e paper discusses t ab le s o f input patterns tagged as ini t ia l or nonini t ia l . A table

    +IAccoroing to Professor I. J . Good, th e f i r s t "Tur-ing Tl12lchine" was a data-processor, nicknamed th eBombe, used by ULTAA during World War I I to crackth e Enigma machine [Lewin, ULTRA Goes to ~ ~ r : TheSecret 8 t o r y ~ Hutc11inson,1978,p 58J. SO Turingmachines ar e very practical!


    represents an . . . a l with an a t t ~ 1 i fi t is in i t i a l and, i f noninit ial , represents. . . . . . al with t he l oc ati on of an in timeunspecified. An event is represented by a seto f t ab le s. The concatenation EF of events E

    and F is the set of tables where X lS a tableIn r which is no t in i t i a l and Y is in F (so inClu'rent no-tat ion i s reLlIIv IT) . The class of regular events is defined as th e closure of th e unitevents ane1 the empty set under) union, concatenationand an operation we can call E':r th e correspondence between rep;ular events aJicl finite automata isestablished.

    The context-free phrase :=:;tructure grammar is( ~ n e attempt to fnrmcilize th e notion of IrrunedidtcConstitupnt erC) anal v s i c ~ , an approach to th e de; ~ c r i p t i o n o f th e morphology and syntax o f n at ur allan,P:uage advocated in the 40s ancI early 50s by!\rrerican linguists such as Bloch, Harris and Wells.Bloch explains, "I n a n a l y z i n , P ~ a given sentence, wef i rs t 5501(Jte th e irrunc'diate constl tuents of th esentence as a whole, th('n th e ( ~ o n s t i tuents of eachconstituent, and so nn to t he u lt imat e constituents- - a t c,-very s t ep choos ing our consti tuents in sucha way that th e total number o f d if fe re nt constructi on s w il l reJYBin as SITk'il l as possible" [Bloch,1946J. And again, "When a word contains three ormore morphemes,Lt is usually necessary to analyzej t into two and only two IMMEDIl\TE CONTITUilITS OJei ther or both of which may be susceptible of further anaJysis" [Bloch .-md Trager, 1942J.

    Various attempts were rrade a t formalizingthis p roc edu re. In "Frum 1"1orpheme to Utterance.,"Harris f i r s t apologizes fo r introducing "methodsas mathematical as th e one proposed here." Hismethod consists of f+t'st defining substitutionc la ss es o f morphemes , o r sequences of morphemes,\,ihich can appear in the same context or envir'on-ment in th e l a n ~ ; u d g E ) . ar e fonned; anequation Be =A me,3J1S -rha t a morpheme of class Bfollowed by arnorpheme of class C can be substituted fo r d mornheme of c la ss A. The utteranceis divided into m c ~ r p h e m e s vlhich ar e placed intomorpheme cL::Jsses. Then -the equations ar e used tomake repeated s u b ~ ~ t i ttJtion until (one hopes) th e~ t l h o l e uttercJ1ce is grouped into a sentence type;i .e . ., a morpheme class which corresponds to av,Jhole sentence rHarris ,1946J.

    i\nc)ther' attempt v J a ~ 3 made by \ ~ e l l s to "replace by a unified systeTI1c-=ttic theory th e heteroanc] incomplete methods hitherto offereddt:Jtennining immediate consti tuents" [\JJells,~ ~ 4 7 ~ p 31]. A sentence is ,Jivided into 2ICs(three or more are allowed by Wells only when noj iv is io n in tu Lwo rcs HB.k:es sense), each Ie i-l'l.tcICs and .so nn down. As ,3Jl example, th e sentence"the king of England opened Parliament" -- pleasenote th e date of Wells' paper! -- is diagrammed

    ++Bloch and Trager [1942J define a morpheme as al inguistic form which cannot be divided into smalle r mean in f2;fu1 parts; e.g. , "opened" contains twoITDrphemes, "open" and "-ed".

  • 8/8/2019 Critic SPL


    z v

    tonTn )osLnT n Tn)

    The Chomsky hierarchy, more or less in i t spresent form, appeared in "On Certain Formal Prop-

    Thus, th e categorial grammars of Bar-Hillelare perhaps th e f i rs t example of a mathematical system that is capable of generating formal languagesand which was inspired by work on machine translat ion. This became a stronger theme in th e late 50sand early 60s.

    3. The Formative Years: 1956-1964

    The 1956 paper d id stress the idea of g e ~ e r a -t ive grammar: not just diagramning utterances i l l alanguage bu t actually pruviding a mechanism fo rgenerating a l l and only th e sentences of the language.

    The phrase structure model appears in Chom-sky's"Three Models for th e Description of Lmguage"[1956J. The three models defined in that paper donot correspond a t a l l to t he c la sses of th e Chomskyhierarchy. They are theories fo r th e explanationof linguistic phenomena rather than p r e c i s ~ ~ t h e -matical models. These three models are: FlDlteState Markov Processes; Phrase Structure (a s a formalization of IC Analysis); Transformational Grammar. The finite state grammars defined hereactually generate only a proper subset of th e regular languages. Under phrase structure, Chomskydefines a [L: ,FJ grammar where F is a f ini te seto f r ule s of th e form uXv uyv, u ,X,v and yover a finite alphabet Vp, X is a single symbol,Y is nonempty and I is a finite set of in i t i a lstrings over Vp. A grammar gives rise to twodifferent languages. One, th e derived language, isth e set of a l l strings derivable from the axioms ofL ; i .e . , th e sentential form language. (So one ofthe ideas used in L-systems appears right here!)The other, th e terminal language, is the set of a llstrings which ar e in th e derived language and areterminated; i . e . , to which no further rules apply.Chomsky notes that "i n every interesting casethere will be a terminal vocabulary VT . everyterminal string is a string in VT and no symbolof VT is r ewri tt en in any of th e rules of F.In such a case, we can interpret th e terminalstrings as constituting th e language u n d e ~ anc:lysis(with VT as i t s vocabulary) and t he derlva tlonsof th e strings as py'Oviding their phrase structure"[Chomsky ,1956;p 117J. Chomsky shows that there ar eterminal languages which ar e no t derivable and thatth e families of derivable and of f in it e s ta te languages are incomparable. The distinction betwe:nwhat la ter became context-free and context-senSlt ive grammars is no t clear here; in fact, theexample of a language not terminal {VIfW I w in{a b}+}) is obviously context-sensltlve. Int r ~ s f o r m a t i o n a l grammar, transforrrations operateon th e derivations of a string produced by a phrasestructure gramnar. A trans format ion is built fromelementary transformations akin ~ pJ:rase structurerules a l l of which must be applled m parallel;this is similar to derivations in tabled contextsensitive L-systems.


    England ,


    of 1111Parliament

    cancels to (derives) v



    kingIII ed


    openth e

    -Iby ':"Jells: I

    which in tree notation would be:

    th e

    t The older among us may recognize t radi t ionalmethods of diagrarruning sentences, here and la ter ,bu t I do no t have a nice schola rly reference for i t!

    (ul ) (um)!WIJ [wnJThen th e b as ic rule of derivation is a cancellatiorlrule:

    In these formulat ions, IC analysis was meantas a method for describing th e structure of a stY' ingknown to be a sentence and no t a method for generating sentences no r distinguishing which strings.aresentences. Wells says, "The task of IC-analyslsi s th e task no t of descY'ibing what utterances occurbu t of describing, af te r these utterances have beengiven, what the i r constituents ~ [Wells, 1 9 4 7 ~p 100J. Harris recognized th at h ls formulae couldbe used in ei ther direction since he warned, "therear e fu rther l imita t ions o f sel ec ti on arrDng th e IYDr-phemes, so that no t a l l th e sequences provided byth e formulae occur" [Harris, 1946;p 178J.

    For example, i f John belongs to category n , poorn s 1to LnT ' and sleeps to ---cnT , one c an ana yze

    Poor John sleeps as

    In his formulation, Bar-Hillel was alreadyconcerned with "situations in which a completelymechanical procedure is required for discoveringt he syn tact ic structure of a g i v ~ n string . . S U c ~ asituation arises, for instance, lD connectlon wlthth e problem of mechanized t r a n s l a t ~ o n " [Bar-I?-llel,1953;p 47J. Bar-Hillel proposed hls categor&algrammars as a combination ~ f . t h e . m e t ~ o d ~ d e v ~ l ~ p e dby th e Pol is h logician KaSl f f i l r AJduk lewlcz [ Dlesvntaktische Konnexitat;' Studia p h i l o s o p h i c a ~ 1(1935) 1-27J with th e methods of IC analysis.Words and strings are assigned to categories whichcan be basic or operator. When an operator category forms ou t of lef t arguments of th e categoriesul ' ,urn and r ight arguments wl.' . . . ,wn ' astring belonging to th e category v, i t is said tobelong to th e category


  • 8/8/2019 Critic SPL


    perties of Grammars" [Chomsky ,1959aJ; th e definitions appear in a 1958 Quarterly Progress Report ofthe MIT Research laboratory of ElectrDnics [Chomsky,1958; th e QPR's ar an important source of in -formation on early WJrk on formal language theoryJ.Here the set of symbols i s formally divided intoVN (nonterminals) and VT (terminals); a rule musthave a t lea st one nonterminal on th e lef t hand side;th e terminal language is now th e set o f strin gsover VT generated from a designated in i t ia l symbol. The four types of gramrrars and languages aredefined here:Type 0 (no restriction)Type 1 uAv-+uyv , A in VN ' Y nonemptyType 2 A-+y , y nonemptyType 3 A-+aB or A-+a, A lD VN ' a in VTThe iden t if ica t ion of Type a languages with recursively enumerable sets is noted as well as the factthat Type 1 languages form a proper subclass of th efamily of recursive sets . I t is shown that{anbmanbmcc I n,m l} is Type 1 but n ot Type 2,using not a pumping leJllTIB bu t an argument on thecopying power of Type 2 grammars; i t i s noted thatthere are Type 2 languages which ar e no t Type 3 .Thus, there is a proper hierarchy of both gramrrarand language types.

    The connection between Type 3 languages andfinite state languages was noted. Indeed, Chomskyand Miller [1958J basically define a f inite stategrammar as a particular type of f inite state automaton and make no distinction between th e concepts.Their "finite state grarrrrn::lr" has a f inite set ofstates, a designated in i t ia l state So and, foreach state Si' a f inite number of transitions(arrows) to other states, each labeled either witha symbol or with the empty word. A word i s gener-ated by this grarrrrnar i f i t takes th e grarrrrrar fromSo back to So without passing through So enroute. Among other things, i t is shown that th egramrrar can be made W1arnbiguous (and indeed alrrDstdeterminist ic) in the sense that empty word transitions l ead only into So and, for each Si andsymbol a, there is a t most one arrow l abel ed a sleaving S i ; the proof is essentially th e same asthe construct ion of Rabin and Scott [1959]. I t i salso shown that the finite state languages form aBoolean algebra and a characterization akin to th ato f r egul ar expressions is provided.The family of finite state o r r eg ul ar l anguages was also studied from t he poin t of view

    derived from considering switching circuit theoryand logical design and inspired by Kleene' s paper[1951,1956J. Myhil l proved that a one le t ter setis regular if and only i f i t i s ultimately periodic and that a set is regular i f and only i f i tis th e union of some of th e equivalence c la ss es o fa congruence r el at io n o f f inite index [Myhill,1957J.Copi, Elgot and Wright [1958J cleaned up Kleene'sformulation of nets, events and regular expressions.McNaughton and YarrBda [19601 prDved t he so-ca ll edKleene-Myhill theorem using state graphs for automata and the standard regular expression language,


    including th e empty word. Ot t and Feinstein [1961Jconverted regular expressions (using th e now standard notation) into Improper State Diagrams (a typeof nondeterministic automata allowing t ransi t ionson empty input) and Improper State Diagrams intoProper State Diagrams (deterministic machines).The most quoted paper on finite state language is probably Rabin and Scott [1959J. Here,we have perhaps th e f irs t appearance of th e standard definition of nondeterministic f inite stateacceptor and th e classic conversion into determin

    i s t ic autorrata, a lt hough, a s noted above, th e ideaand the result already appear in Chomsky and Miller[1958J. Regular languages are charac te r ized interms of congruence relations of finite index(credited to Myhill [1957J) and in terms of rightinvariant equivalence relations of f inite index(credited to Nerode [1958J). Some of the closureproperties ar e established: r ev er sa l, t he Booleanoperations. Decision algorithms are provjded forth e emptiness, finiteness and equivalence problems.Two-way and twJ-tape deterministic finite autorrataar e defined and a reduction of two-way to one-waymachines given (i n th e sarre issue of th e journal,Shepherson [1959J gives a simpler construction).For languages accepted by deterministic two-tapeautomata, i t is shown that projections are regular,emptiness and finiteness are decidable, and thereis closure under complementation but not intersection or union. A noteworthy development is thef irs t (? ) use of th e Post Correspondence Problem,in this case to show th e W1decidability of th eemptiness of intersection problem fo r determin ist ictwo-tape automata and then the undecidabili ty ofemptiness for twJ-tape two-way machines.

    The theory of context-free languages was being developed during th e sarre period. I have no tbeen able to pinpoint th e in i t ia l appearance ofth e term "context-free" for th e Type 2 case; perhaps some reader can help. I t is not used byChomsky in 1958 or 1959, nor by Bar-Hillel, Perlesand Shamir [1961] nor Ginsburg and Rice [1962].I t is however used by Scheinberg [1960], who notesthat context-free graJIll1B.rS are a special case of asemi-Thue system and shows that th e family of context-free languages is closed under union bu t notintersection or complementation; he uses pumpingtype arguments to show that {anbnan I n l} is notcontext-free. The term "context-free" is alsoused by Chomsky in "On th e Notion 'Rule of Grammar'" [1961].

    Important parts of th e theory o f con text free languages were presented in 1961 by B3.r-Hillel , Perles and Shamir and by Parikh. BPScal l a Type 2 language a "simple phrase structure language" (SPL) . Among other things, thatpaper established that th e c la ss of SPLs is effectively closed W1der reversal, union, concatenation,star closure and SUbstitution, and that erasingand renaming rules can be effectively eliminated.The famous uvwxy-theorem or pumping lemma ( laterstrengthened by Ogden [1968J) appears. The rduct ion process for Type 2 gramnars appears and isused to show the decidability of emptiness andinfiniteness. The cross-product construction

  • 8/8/2019 Critic SPL


    is introduc ed to Sf'101r} t he c lo su re of SPL tinder in -tersection with FAL (Finite AutorncJ.ton Language =finite state language). The Post CorrespondenceProblem is used 3S th e key t oo l t o e st ab li sh th eundec idabl il i ty of a n u ~ e r of problems including:emptiness of intersection; t he uni ve rse problem;whether an SPL is an FAL; inclusion; equivalence,even equivalence to an FAL.

    In an NIT QPR [Parikh,196lJ, which did no tappear in j ou rn al form unti l 1966 bu t was s t i l lwidely inf luent ial , Parikh established two key l'esuIts: t he exi st ence of inherently ambiguous CFlanguages (the language {anbmakbr I n = k or m= r ,n,m,k,r I} cannot be generated by any unambiguousCF g r ~ ) ; th e .=ommutative map of a CF languagelS seffilllnear.

    \ ~ i l e th e th eo ry of context -f ree languageswas being developed, systems with th e power ofType 2 grammars were being used (knowingly or un knowingly) fo r th e specification and automatic syntactic analysis of both natural and a r ~ i f i c i a l( i . e . , programning) languages.

    Nany of the systems fo r automatic syntacticanalysis used explici t ly or implicitly th e conceptof a p u s h d o ~ m store. The origin of th e concept ofpushdown storage is not clear. t10st writers a t t r i -bute i t to Burks, Warren and v ~ i g h t [1954J who formalized the parenthesis-free notat ion of Lukasiewicz [1951J and used i t to describe a "simple andeasily mechanizable process of t r u t h - t a b l e ~ computat ~ o n " ; th e ide a of a pushdown store appears impliClt in th e description of their logical machine.I t was used more explici t ly by Newell and Shaw in"Programning th e Logic 'Theory Machine ll [1957J.The ir b as ic d ata structure is a l i s t vll10se elempntscan be l i s ts . ~ item in a l i s t is given bv a location word, half of which points to Cis th e addressof) th e element and th e other half of which pointsto t he locat ion word of th e nex t i tem on th e l i s t .To keep tr ac k o f th e available space, their systemuses an available-space l i s t which i s used as apushdown store. "vlhenever space is r eoui red forbuilding up a new l i s t , t h i s ~ is obtained by usingwords from th e front of the available-space l i s tand, whenever information is erased and th e wordsthat held i t become ava il ab le for use elsewhere,these words are added to th e front of th e availablespace l i s t " [Newell and Shaw, 1957 J.

    The term "last-in-first-out" storage was usedexplici t ly by Sarre lson and B3.uer [1960J who proposed th e use of LIFO storage as an aide in th etranslation of ALGOL (say) formulae into ffi3.chine instructions. They say "a l l incoming (source) information given by symbols which cannot immediately beevaluated is introduced, in th e sequence of f i r s ta.ppearance, into a special storage called 'symbolsce l la r ' , where a t any g iven t ime onlv th e elementintroduced l as t (the highest le ve l o f th e cel lar)is of immediate consequence and need be available.Each new symbol of source information is , in turn ,compared to th e momentarily highest cellar symbol.These two symbols in conjunction determine th eevaluat ion Cif .possible) of the h ighest cellar symbol by generatlon of an appropriate machine instruc-


    t ion a.1'1d the new state of the cell?.Y' by eliminatingth e o ld h ig he st symbol and/or introducing th e newsymbol as th e case may be . . . translation of formulap::ograrns into ~ a c h i n e operations, with th e exceptlon of recurslve address calculation, can be donesequentially, without s to ri ng o f th e formula program, as a pure i nput process. "The mechanical translation programs at Harvard and MIT used th e idea of a pushdown store.At MIT, Yngve in "A Model and a Hypothesis fo rLanguage Structure" [1961J starts by assuming aphrase structure (Type 2) grammar fo r English andconstructs a pushdown store t ransducer to generateEnglish sentences u sing the phr'ase structure grammar. He adds a depth hypothesis, limiting th e numbe r of symbols to be stored in th e pushdown storewhich reduces i t in power to a finite state t r a n s ~ducer. On the other hand, in order to handle discontinuous constituents, he adds rules of th e form(e.g.) A - ~ B . . . C which allow uDAv , say, tobecome uBDCv instead of uDBCv, etc.Rhodes a t th e National Bureau of Standards[Rhodes , 1961, th e published version of a 1959repor tJ or ig ina ted for th e automatic syntacticanalysis of Russian a rrethod la ter to be called"predict ive analysis" fThe idea is that each wordwhen analyzed may make predictions as to th e occur

    r ~ n c ~ of future words ir'.. th e sentence. IThese predlctlons are t re at ed i n a roughly LIFO fashion~ t h i s ~ no t explici t ly stated in th e Rhodes paper);J_teratlons are allowed i f t he ana ly si s cannot becompleted or i f there ar e alternative analyses.

    At Harvard, th e ideas of Burks, Warren and\Nright and of Rhodes were developed into th e Multiple-Path Predictive Analyzer for English and fo r;

  • 8/8/2019 Critic SPL


    i t to "LIFO" storage. "The Rhodes method of 'predictive' syntactic analysis is based on th e observation that in scanning through a Russian sentencefrom lef t to right i t is possible, on th e one hand,to make predictions about th e syntactic s t r ~ c t u r e sto be rret further to th e r ight and, on th e otherhand, to determine th e syntactic role of th e wondcUYTently being scanned by testing what previouslymade predictions i t ful f i l l s . The predictions ar estored in a linear array called th e 'predictionpool' which behaves approximately as a pushdownstore . . . . Predictive analysis yields a descriptionof the syntactic structure of a sentence in termsconsonant . . .with old-fashioned parsing, immediateconstituent theory . . . or phrase structure theory . . . .I t remains to analyze th e exact relation of th epredictive ID2thod to these theories" [Oett inger ,1961,p 105J.

    The predictive syntact ic analysis method wasessentially nondeterministic since different matchesof prediction and word could be made and resul t indifferent prediction strings. Thus th e methcxl wasimplemented by following; different paths at th esar.-e t i. rre, hence th e term "multiple-path syntacticanalyzer" [Kuno, Oettinger,1962]. I just cameacross a very old paper of mine, " It erat ion s forthe Syntactic Analyzer" [Greibach,196J], which discusses the problem and t r ies (not too successfully)t o r ec as t i t in terms of threading paths through amaze, and shows that in a few very special casesthe number of iterations would be linear in th elength of t he i nput . We were very much aware ofthe problem of exponential blow-up in th e numbero f i te rat ion s (o r paths), though we fel t that thisdid not happen i n " re al " natural languages; I dono t think we suspected that a polynomial parsingalgorithm was possible.

    A somewhat different model was used a t RAND,based on th e idea of tree diagrams of sentences[Hays,196lJ. In these dependency s y s t e m s ~ th enodes of a dependency tree correspond to terminalsymbols (word or word c la ss es ) o f th e sentencewhose structure i t r ep re sen ts , i n contrast to th ephrase structure t ree, where terminal symbols areassociated only with th e bottom node on each branch.One occurrence is independent, th e o ri gi n o f th etree . Every other occurrence except th e origindepends on one and only one other occurrence; everyoccurrence except th e origin depends, directly orindirectly, on the origin. A dependency t ree isdrawn in such a way that i t s nodes ar e arrangedhorizontally from le f t to rig ht in sentence wordorder. I t is also made to obey th e restriction ofprojectivity [Lecerf and Ihm,1960J so that no twobranches of the tree cross each other and vert icalprojection lines extending from each node onto ahorizontal line drawn below th e tree do no t intersect any of th e branches of th e tree. (A similaridea using "predicts" instead of "depends on" couldbe used fo r predictive analysis [Greibach,1962,1963aJ. )

    Immediate Constituent analysis was also used.Sakai [1962J proposed an algorithm fo r parsingsentences based on IC analysis with st r ic t binarydivision. 1m extensive IC grarrunar fo r English was


    developed a t RAND by Robinson [1962J fo r use witha pars ing algori thm attributed to Cocke [Hays,1962J.This algorithm consisted of generating for a givensentence, a ll two-word constitutes. All constit ute s of length n are generated using as constituents th e constitutes of sma ller l ength previouslyobta ined ( including individual words, constitutesof length 1), unti l a ll constitutes of the sentence length aI'e obtained. The program had th every important feature of processing a g iven phraseof a g iven type a t most once. Thus, Cocke's algorithm essentially showed that context-free languages can be parsed in polynomial t -J.; i ) Cndeed,OCn 3 )) although that was not immedLD' : , ' recognized.

    Independently, computer prograrrmers deviseda syntax for the spec ificat ion of th e l ~ f e r e n c elanguage for ArhOL. As Backus explained at th e1959 ICIP Conference, "There must exist a precisedescription of those sequences of symbols whichconstitute legal IAL prograITls . . . . fo r every legalprogram there must be a precise description of i ts'meaning', th e process or transformation which i tdescribes . . . .Heretofore there has existed no formaldescr'iption of a machine-independent language (otherthan that provided implicitly by a complete t ranslating program) which has met either of th e tworequirements above . . . . Only the descr ip tion of legalprograms has been completed, however . . . . In the de scription of IAL syntax which follows, we shal lneed some metalinguistic conventions for characterizing various strings of symbols" [Backus,1959J.The "metalinguistic convention" is not def ined formally bu t given by an example of formulae in whatis now called BNF ( foY ' Backus Naur Form), possiblevalues o f the v ar iab le defined and th e cormnent". . . th e formula above gives a recursive rule forth e formation of th e values of t he var iabl e . . . "

    The ATbOL 60 report used BNT to speci fy thesyntax of th e prDposed International AlgorithmicLanguage ( IAL) [Naur,1060]. The original syntaxof ALGOL turned ou t to be ambiguous, an embarrassing situation, and some of those involved wonderedabout th e possibility of automatically detectingsuch ambiguities. Cantor [1962] proved that thisis impossible -- th e ambiguity problem for Backussystems is undecidable. (His example of an ambiguous ALGOL statement is : "if B /\ C , then forI: =1 step 1 until N do i f D v E ,thenA[I]: =0 else K: =K+l;K: =K-l" , which under oneinterpretation will have th e effect of leaving KlITlchanged when 1CB /\ C) holds and under th e otherwill change K to K-l when 1 (B /\ C) holds.)Cantor used no t th e Post Correspondence Problembu t th e existence of an undecidable Post normalsystem.

    IrDns [1961J discussed "A Syntax DirectedCompiler fo r ALGOL 60", which was to be "a compiling system which essentially separates th e functio ns of defining th e language and translating i tinto another". His paper uses th e syntax of ALGOL60 and extends i t to allow specification of meaning(i n terms of the target language) as well as ofform.

  • 8/8/2019 Critic SPL


    Thus, th e same importai'l.t ideas emerged fo r .th e automatic translation of both natural and art l -f ic ia l languages:-separating syntax and semantics-using a generative grammar to specify the set

    of a l l and only legal sentences (programs)-analyzing th e syntax of th e sentence (program)

    and t hen u sing th e analysis to drive th et r a ' l ~ ~ c+ion (compilation)

    FurtherITDre., roughly similar methods were used tospecify t h2 syn tax and one key prograrrrrning technique - - th e pushdO'wn or LIFO store - - was used forat least part of th e analysis o r t ran slat ion [Oettinger,196l; Samelson and Bauer,1960J.Among th e systems for g e n e r a t i o ~ or,analysisproposed by linguists and computer SClentlsts wereth e following:

    -IC analysis-categorial grammar-Type 2 (CF) phrase structure grarnmaBNF (the syntax of ALGOL 60)-dependency systemspredictive analysispushdown store acceptor

    These various systems were gradually shownto have th e same power fo r language specification.This was no t unexpected; in a paper not pub li shedin f ul l u nt il 1964 , Gross [1962J hypothesized thata l l th e well-known models of language used in th efields of rrechanical translation and infor'TIBtionretr ieval would be th e same in power, bu t gave nofu l l proofs. In his 1959 paper, Chomsky had al-ready shown that Type 2 grarrtrIBrS are no more p o w ~ r -ful, in weak generat ive capaci ty, than IC analysls(i n th e sense of s tr ict binary division and no context) by showing that a ll CF languages could begenerated by ru les of th e kind A -+ b (b terminal;in IC analysis, this cO:rYsponds to "word b ,be-longs to class A") and A -+ Be (B,C nonterrrunals;this corresponds to "a constitute of class B followed by a constitute o f c la ss C can compose aconstitute o f c las s A"); this is th e ChomskyNormal Form.

    Bar-Hillel, Gaifman and Shamir [1960J provedthat "simple [ i .e . Type 2J phrase-st:r:ucture gramIIBrS as defilled by Chomsky, ar e equlvalent tovari;us types of cat ego ri al grammars, as discussedby Ajdukiewicz and Par-Hillel, which ar e therebyshown to be equiValent among themselves." In th eprocess, th e authors show that Type 2 gramrIB.rS canbe converted to a restr icted form of category system: vocabulary V, simple categories il l a set T,operator categories of th e forms [A \B] or[A\[A\BJJ for A,B,C in T, an assignmen t function f such that , for each a in V, f(a) is afinite nonempty set of simple and opera tor catego-


    r ies , canceling rules A[A\BJ to B andA[A\[B\CJJ to [B\CJ and an ini t ial category. Astring al . . . an o ~ e r V is g e n e ~ a t e d i f and onlyi f there are Ai i l l f(a i) ,Is l s n such thatAl . . An cancels to S. This. fonnulation o category system uses a slightly dlfferent notatlon fromthat originally proposed by Bar-Hillel [1953J.Gaifman [1961; revised and published in 1965J usedth e equivalence of categorial and Type 2 grarrrnarsto establish th e equivalence of those two systemswith a certain formulation of dependency systems.In his formulation, roughly speak ing, a filllctionf assigns words to categories in such a way thatsome word is assigned to each category and thereare a finite set of rules of the form X(Yl' . . . 'Yr..: Zl' . . . ,Zs) rreaning that i f X is in f(a) , thena as a word of category X can be expanded into(Y ,b ) . . . (Y ,b ) (X ,a) (Zl ,c l ) . . . CZs ,c s ) which can be1 1 r rseen to correspond to context-free ru les of th eform X -+Y . . .Y a Zl . . . Z or be diagrammed1 r s

    X in f(a) , b. in F(Y.), c. in f(Z.) , wherel l ] ]X Y Z ar e categories and a,b, ,c . are words, i' j l ](terminals). (Cf Greibach [1962,1963].)Ginsburg and Rice [1962J connected the syntaxof AlliOL 60 with Type 2 phrase structure grammarsby considering as AlliOL-like the l a n g u a ~ e ~ d e f ~ -able as one of th e coordinates of th e rm.nJlIB.l flxedpoint of an n-tuple standard function; this is th ebasis for th e power series approach to CF languages.

    This paper also def ine? the family of s e q ~ e n t i a llanguages, considered l ts closure propertles an?proved that i t properly i n c ~ u d e s the.class of f l-n it e s ta te and is properly lncluded In the class ofType 2 languages.The equivalence of nondeterministic pushdownstore acceptors and context-free grammars was in dependently established by Chomsky and Schutzenberger [Chomsky,1962; Chomsky and Schutzenberger ,1963] and Evey [1963J. In a 1962 QFR, Chomsky defined nondeterministic pushdown store automatawith e-rules; these machines accept i f , at the

    f i r s t return to th e ini t ial state, th e pushdownstore is empty and th e whole input has been processed. The paper shows that , fo r a pushdown storeautoma..ton M, there is a f ini te state transducer Tand a regular set R such that L(M) =T(K n R) ,where K is a one-sided cancellation language, andthus L(M) is context- f ree via the c losure of th efamily of CF languages under intersection.withregular sets [Bar-Hillel, Perles,and S h ~ r , 1 9 6 1 Jand under finite state transductlons [Glnsburg andRose,1963; draft available earlierJ. The paperalso characterizes CF languages as h(K n R) fo rh a horromorphism and claims that one can take asK th e two-sided cancellation language; however,I have never been able to understand that proof,

  • 8/8/2019 Critic SPL


    nor th e one in [Schutzenberger,1963J.In his thesis, Evey [1963J used a differentapproach. He defined a "hierarchy!! of s ix s ets ofmachines (with finite state control and up to twopushdown stores, possibly r es tr ic ted to countersor to 1 or 0 stores) and partitioned them into acceptors and transducers and into deterministic and

    nondeterministic. He showed that th ec la ss of languages accepted by nondeterministicacceptors of a given type is th e same as th e set ofoutput languages (ranges) of deterministic transducers of th e same type. Thus, th e class of languages accepted by nondeterministic pushdown storea ~ c e p t o r s (pdas) is th e class of images of setsL : ~ ' under pushdown store transduction. He showedthat th e latter is contained in th e family of context-free languages through consideration of special types of bracketed languages and what he called"pushdown store gramrrars". A pushdown store gram-I IB r is a Type 0 grammar such that there i s a f initeupper bound on the d is tance between the f i r s t nonterminal and th e f i rs t available nonterminal ( i . e . ,available for application of a rule) in any derivedstring. I t is easy to convert pdas to pushdownstore grammars; th e diff icult part is to show thatpushdown store generate CF languages(without of course appeallllg to th e equivalence ofCF graJI1l'IBrS and pdas!). Note that th e importantpart of th e proof of the equivalence between CFgraJI1l'IBrS and pdas is th e conversion of pdas togrammars ; i t was apparent to us a l l that a CF gram-mar could be parsed using a nondeterministic pdawith e-rules once th e definition was at hand.Evey also established that lef t- to-r ight or r ightto-lef t derivations suf fi ce for CF grammars andthat discontinuous rules o f the type advocated byYngve [1961J do no t increase th e power of a CFgramrrar; th e la t ter fact was also observed by Mat-thews [1963J.

    In my thesis [Greibach,1963aJ, I completedthis set of results by establishing that the methodof predictive analysis can handle a ll Type 2 languages. Rules in a predictive analysis grammar areconsidered to be of th e form (P,c) I Pl .Pk whereP and Pj are predictions and c is a syntacticword class. The argument pair (P,c) indicatesthat word class c may fu l f i l l a prediction P(thus initiating a syntactic structure corresponding to P) i f P is on to p of th e prediction pool(pushdown store); then P is removed from the topof th e pool and replaced by PI . .F\ , a string ofnew predictions that have to be fulf il led in theindicated order (PI f i r s t , I \ last ) . There werevarious exceptions permitted in th e actual Harvardsyntactic analysis program, bu t we need no t consider them here; rrost of them did no t affect thetheoretical power of the system. In this formulat ion, such a rule corresponds to a phrase structurerule P cPl Pk ,where P,PI ' . . ,Pk are nonterminal symbols and c is a terminal symbol.Thus, to prove that pred ict ive analysis could handl e a ll CF languages, i t suffices to show thatany Type 2 grar rnar can be converted to one withr ule s o f th e forms: P cP l . . Pk or P c ; i . e. ,the so-cal led Greibach Nornal Form. I also cons idered var ious types of predictive and dependency


    syst ems. Notice that a rule in GNF can be considered a special case of a rule in a dependencysystem (or, perhaps, Gaifman normal form!) whereth e one terminal symbol pe r rule can appear anywhere in th e right hand side of the rule; hence,Gaifrnan's theorem can be considered to be a corollary of mine. I did not realize until long afterwards that , using th e closure of th e family of CFlanguages under reversal, i t is no t hard to obtaina GNF grammar from a restr icted categorial systemand v ice ver sa . Incidentally, another restr ictedtype of Type 2 grammar, th e operator grammar (notwo nonterminals adjacent), proposed at this timeby Floyd [1963j in connect ion with tn e use of precedence grarrnnars to r parsing prograrrnning languages,can be shown to s uf fi ce f or CF languages, usingGNF [Greibach,1965J. In my thesis, I also showedambiguity undecidable in a very restric ted case:m-inimal linear graJnmars, meaning only one nonterminal X and ru les o f the form X uXv, X-+ u ., uand v strings of terminals. Since th e u su al formulation of th e Post Correspondence Problem didnot work, I proved a variant undecidable and usedthat fo r th e reduction [Greibach,1963bJ; this technique has been used by others.

    The top of th e hierarchy had already beenrecognized, as I have observed earlier . However,different formulations of Turing machines were ob tained. Counter machines were vaguely mentionedby Chomsky [1959J. Minsky [196lJ showed that in acertain sense a machine with two count er s can s imulate a Turing machine. A very elegant and intuit iv e d ir ec t proof (without th e encodings used byMinsky fo r input and output) was provided by Fischer in the f i r s t FOCS paper we cite [Fischer,1963;th e fu l l paper appeared in 1966J. He showed thattwo pushdown stores can simulate a Turing tape,two counters can simulate a pushdown store, andtwo counters can simulate any number of counters.Thus, a t this stage, we have a graJTU'TB.Y' and amachine for: Type 0, Type 2 and Type 3. We alsohave various other characterizations in each case(such as the regular expression characterizationfor regular languages). Only the Type 1 languagesremain to be characterized by machine. This wasth e las t case completed. The type of machine needed was fundamentally different from those in theother cases. What was required was no t a res t r ic-tion on th e access of the machine to i ts work tape"but on th e amount of tape allowed. We should notethat both time and tape restric t ions on Turingmachines were defined in the early 1960s.Yamada [196lJ f i r s t formalized the concept of

    realtime computation and estab lished the existenceof fill1ctions not realtime computable. Rabin [1964Jprovided what is now the standard definition of theclass of languages recognizable in realtime byk-tape or mu1titape deterministic Turing machinesand showed that two tapes are JIDre powerful thanone tape in realtime. In th e 1964 meeting of theSyrnposiwn on Switching Circuit Theory and LogicalDesign, Hartmmis and Stearns [1964J presented th eclassification of numbers, functions and sequencesby time compZexity on a multitape Turing machine;their hierarchy results can be applied to the class-

  • 8/8/2019 Critic SPL


    ification of formal languages by theirof recognit ion. In his thesis, Cole [ered computation in realtime by i terativef in it e s ta te machines.The classifaction of sets by space complexityof recognition was initiated by Myhill in a' widelycited but never published report [Myhill,1960J. He

    defined a deterministic linear bounded automaton(lbaJ as a two-way deterministic f inite state auto-maton with overprinting. He showed that rudimentary sets can be defined by d ete rm ini stic lba andposed th e question of whether this is a proper containment, a question which is s t i l l open today.The paper also gave an informal definition of machines with storage restric ted by an arbitraryfunction f such that fen) n everywhere.

    Landweber [1963J showed that th e class oflanguages generated by Type 1 phrase strUCTLrr'e languages (the class of context-sensitive, CS , languages) is not enlarged by allowing endmarkers, isclosed under the operat ion of intersection and in cludes th e class of languages accepted by determ inistic lbas.

    I recall that , when I was wr lt lng my thesisin 1963, i t was an lllteresting open question whether lbas corresponded to Type 1 gramnars as pdasto Type 2 gramrrers. With hindsight, i t i s hard tosee why this was a dif f icul t problem at a l l . Partof th e reason was that we were no t really used tonondeterministic machines - - to think nondeterminis t ical ly -- and Myhill 's paper defined only deterministic lbas. I t was known that deterministicand nondeterministic f inite automata had th e samepower [Chomsky and Miller,1958; Rabin and Scott,1959J and -- a s no ted , fo r example, in the thesisof Evey [1963J -- that deterministic and nondeterministic Turing acceptors had th e same power. I twas suspected and shortly proved mdependently byvarious people [Cole,1964; Haines,1965; Ginsburgand Greibach,1966, based on a 1964 draft] that de terministic pushdown automata d id not accept a l lCF languages and so were of lesser power than nondeterministic pdas. But lbas?

    Kuroda [1964J completed th e Chomsky hier'archy of languages and ma.chines when he defined non-detePministic lbas and showed that each contextsensitive (1\JPe 1) language can be accepted by anondeterministic lba. He also gave a proof thatth e language accepted by a nondeterministic Ibais context-sensitive. Further, he showed that th eclass of languages accepted by deterministic lbasis a Boolean algebra containing th e class o f context-free languages. He stated two important problems which rernain open today: Is th e class o fcontext-sensitive languages closed under complementation? and, Is th e clas s o f nonde te rmin is ti c l balanguages equal to th e c lass of deterministic lb alanguages? ( i .e . ; is NSPACE(n) = DSPACE(n)? th e"LBA. problem").

    Thus, th e Chomsky hierarchy was completed andt he s ta ge set fo r most of th e developments of formal


    language theory. A good survey of formal languagetheory in i ts relation to mathematical l inguistics,th e syn tact ic analysis of natural languages, methods fo r machine translation,and information re tr ieval , can be found in th e notes fo r th e 1964 HarvardSummer School Course on Language Data Processing[Oettinger,1964J. Although th e divergence was no trapid nor complete, that represen ts the last timewhen formal language theory could really be conside re d p a rt of mathematical l ingui st ic s o r , altern at iv el y, t he s tar t of th e disciplme as a pa rt oftheoret ical computer science.4. Directions after 1964

    From 1964 on, formal language theory developed as a branch of theoretical computer science.The take-off point was th e study of th e t;rammars,machines and languages of th e Chomsky hierarchy:restric t ions and extensions of context-free grammars and pushdown automata; restrictions on thecomputing abi l i ty and resources of Turing machines;abstractions of the proper ties of various classesof languages; th e need fo r bet ter algorithms forthe recogni tion of languages; consideration of th epract ical use of context-free grammars in parsingand compiling.Exactly which t op ic s a re classified under"formal language theory" and th e relative emphasisplaced on them is debatable: different books ont he sub ject have different selections. Time andspace do no t allow me to cover a ll possible topicsno r to cover anyone top ic at sufficient length.I claim an author's prerogative to dewll most onth e Subjects I know best and th e part of th e th eo ry to which I contributed.The directions that I shall consider brieflyare:

    -Restrictions on Context-Free Grammars and PDAs-Extensions of Context-Free Grammars and PDAs-Unifying Frameworks-Complexity Ouestions

    Those four topics ar e not disjoint and t he overl apand interaction between t he se a reas (and others)has been a fruitful source of techniques and results. Some directions I shall no t survey (althoughI have been connected with a few) include: algebraictheory of autorra.ta and languages; probabilistic languages; fuzzy languages; 2-dimensional languages;semantics (including schematology).5: Below th e Context-Free:Restrictions on Context-Free Grammars and PDAs

    Various types of res tr ic tions have beenplaced on context-free grammars and on pushdownstore autorra.ta (pda). Many of th e restrictions dono t affect th e power of t he devices . For example,we have seen t hat re st r ic ting the productions of acontext-free grarmnar to Chomsky Normal Form or toGreibach NOrITBl Form or to operator form s t i l l permits th e generation of a l l context-free languages.(That is , modulo th e empty word which I shall desisnate e .)

  • 8/8/2019 Critic SPL


    Chomsky's de fini ti on o f a pda allowed what Ihave called "e-rules": transit ions of the machinewhich do no t advance th e input tape. Eliminatinge-rules, thus restr ict ing th e pda to operate in"quasirealtime" , does no t limit th e power of a pda,even i f at th e same time th e number of states isrestricted to one [Greibach,1963aJ. Restricting apda to deterministic operat ion does make a difference, however. This is th e f i r s t restric t ion weshall consider' and, from a pr'actical viewpoint,th e most important. Restricting th e number of nonterminals in a rule to two makes no difference, aswe saw, bu t restr ict ing th e number of nonterminalsin each intermediate step of a derivation does; weshall consider restrictions of th at s or t and th ecorresponding types of pdas. The third res tr ict ion we shall consider is no t directly on th e gramrrBrS or ma.chines bu t on th e form of words in a language.Deterministic Context-Free Languages

    The basic properties of de te rmin ist ic pushdown store automata (dpda) ';Jere st"udied by Schutzenberger [1963J, ~ i n s b u r g and Greibach [1965J, andHaines [1965J under different formulations. Thelangpage accepted by a deterministic pushdown storeautomaton is called a cieterminist1:c context-freelanguage ( d c f l ) ~ every context-free languap;e (cfl)can be generated as th e range of a deterministicpushdoWfl store transducer [Evey, 1963 J . The p a l ~ n -drome or mirror image language {wx Iw in {a, b}~ ,x is the reversal of w} is a c f l which is no tdeterministic (Cole[1964J, Ginsburr and Greibach[1965J), so the family of deterministic contextfl'ee languages (DCFL) is a proper subfamily ofth e family of context-free languages (CFL).

    Fischer [1963J claimed that DCFL is l1ea s i ly " sho\VI1 to be closed tmder c o m p l e m e n t a t ~ i o n . How-ever, there is one diff icul ty. The standard defLYlitions of a dpda allow e ~ r u l e s and hence thepossib il i ty that th e rrBchine can " loop" and neverread a l l of i ts input tape. Eliminating e-r1ulesproduces th e c la ss o f realtime pda languageswhich Ginsburg and C;reibach [1965 J showed to be aproper subfamily of DCFL; their counterexamplecan be recognized by a realtime Turing machine(TI1) with one working tape, bu t Rosenberg [1967bJpr'Ovided an examp1e of a dcfl no t realtime definable by any multitape 1M. However, i t is possibleto clLTinate loopLYlg in deterroinistic pdas nnrlthus DCFL is closed under complementation [Schutzenberger,1963; Haines ,1965; Ginsburg and Greibach,1965 J. Ivlor significantly, this also shows thatdpdas can be to oper-ate in l inear time. An-other important fact is that a dpda can be effectivelv converted to an eqllivalent unambiguous context-free grammar ( i .e . , one left-to-right derivatio n fo r each word in the language) [Haines,1965;C;insburg and Greiffich, 1 ~ 1 6 5 J.

    These two properties -- lineae time recognltlon and unambiguity - - naturally directed attent ion to deterministic context-free languages bu ti t was th e introduction of LR(k) grammars b ~ Y ~ l u t h [1965J which established their i m p o r t ~ l c e forth e syntax of progrilllllning languages. The LR(k)


    restric t ion on context-free grammars is differentin natuy' from those we consider below. In a rightto- lef t derivation . . . U::9V . . . , th e nontermina1expanded in th e step u =?v and th e productionused in that step identify a handle of v. Roughlys p e a ~ i n g , a grammar is LR(k) ( i .e . , le f t- to-righttranslatable) i f th e handle of a r ight - to- le f tsentential form is always uniquely determined byth e string (o f terminals and nonterminals) to i t slef t and th e next k (terminal) symbols to i t sright. The language generated by an LR(k) grammaris a dcfl while every defl has some LR(l) gramffi3.r. For k fixed, i t is decidable whether a context-free grammar G is LR(k) bu t i t i s undecidable whether there is an integer k such that Gis LR(k) ; for the la t ter proof, Knuth definedand used another Valliant of the Post CorrespondenceProblem. Knuth [1965J discussed Pallsing proceduresfo r LR(k) gra.rrDTla.l"'s and the use of LR(k) grarrrrnarsfo r All;OL. Since that paper, various authors,such as DeRemer [1969,1971J have considered th e useof subclasses of LR(k) grarrrrnars for parsing prog r a m m i n ~ languages.

    A different grammatical characterization ofDCFL was given by Harrison and Havel [1972;1973J.They def ined t he class of (realtime) st r ic t deterministic grammars that generates p rec is e ly theclass of languages accepted by empty store by(realtime) dpdas which block on empty store. I tis decidable whether a grammar i s s tri ct determini s t ic . Addin,Q, an endrnarker to a dcfl makes i ts t r i c t deterministic so , for most purposes, th etheories of deterministic and st r ic t deterministiccontext-free languages coincide.Precedence languages were introduced by Floyd[1963J while Wirth and Weber [1966J proposed anextension in their discussion of Euler. The re lat ionship between precedence and deterministic context-free languages was ini t ia l ly no t quite clear.

    M. Fischer [1969aJ showed that th e class of Floydoperator precedence languages is properly containedin th e class of backwards deterministic Wirth-Weberprecedence, which in turn is properly contained inDCFL. However, every cf 1 can be generated bysome Wirth-VJeber pD2cedence grarrrrmr. On th e o th erhand, Graham [1970J observed that " if one takesth e definitions of extended precedence to be thosesUfSgested by Wirth and Weber in t he descr' ip tiveportions of their paper, rather than th e formalrlefin-it--jons presented , then a l l th e classes ofextended precedence languages which ar e nontrivialextensions of simple precedence are equivalent andare equivalent to th e c la ss o f deterministic languages and to th e class of bounded r ight contextlanguages. "Ginsburg and Greibach [1965J establishedmost of the basic closure and decision propertiesof DCFL. They showed that inclusion is Ufldecidable as are th e emptiness o f inte rsect ion (L n L' =v?) and th e identification problems (does a context-free grammar G generate a deterministiccontext-free language?). The equivalence problemand most of th e identification ( or subcl as s containment) problems remain open. Stearns [1967Jshowed that i t i s decidable whether a dcfl isregular.

  • 8/8/2019 Critic SPL


    The equivalence problem has been shown decidable fo r various subclasses of deterministic languages. A parenthesis grammar has rules of th eform A -+ (u) where u does no t contain n (T1 orTI )". McNaughton [1967J defined parenthesis grammars, showed that they generate deterministiccontext-free languages and that inclusion and equivalence ar e decidable. Knuth [1967J showed that i tis decidable whether an arbitrary context-free grammar generates a parenthesis language. Usually,equivalence i s shown decidable by showing inclusiondecidable. The f i r s t known exception is th e classof simple languages (generated by grammars in Nonnal Form such that a pair (Z ,b ) of anonterminal Z and terminal b appears in a t mostone rule Z -+ bu) for which Korenjak and Hopcroft[1966J showed equivalence decidable, while Friedman[1976J showed inclusion undecidable. Rosenkrantzand Stearns [1969J showed equivalence decidable forth e LL(k) grarrrrnars of Lewis and Stearns [1966J(which are stronger than simple gramnars bu t weakerthan LR(k) ). Interest in th e equivalence problemwas reawakened by Valiant [1974J who provided powerful new techniques and by Fri edman [1974J whoshowed th e problem to be logically equivalent toth e strong equivalence problem for monadic recursion schemes. Beeri [1975J improved Valiant'salgorithm for deciding equivalence of f ini te turndpdas . Recent1y i t has been shown tha t Valiant'salgorithm for nonsingular languages is a ls o v al idfor realtime s t r ic t deterministic languages andthat "L l =L2" is decidable for Ll real tin1es tr ict deterministic and L2 merely deterministic[Oyamaguchi,Honda and Inagaki,1979J; this may bet he s trongest result to date.Linear-like Languages: Nonterminal Restrictions

    Chomsky and Schutzenberger [1963J definedlinear context-free grammars by productions whoser ight hand side contain at most one nonterminal and,by adding arbi trary productions which could be usedonly ini t ia l ly , extended th e concept to metalinearg ramna rs. Gre ibach [1966J showed that , for a symbol c no t in L, lc L is linear i f and only i fL is regular i f and only i f ( l c ) ~ ' : is metalillear;thus, metalinear graJI1l1B.rS are stronger than l inearbu t weaker than context-free.Sentential forms of lillear graJ11I1BT'S can contain at most one nonterminal. Banerji [1963J generalized this concept to nonterminal bounded gram

    :rn:.:ITS where there is a uniform upper bound on th enumber of occurrences of nonterminals i ll sententialforms. Ginsburg and Spanier [1966bJ provided adifferent bu t equivalent grammatical characterization for th e languages generated by such grammarsand called them ultralineup. They a lso establ isheda JIBchine characterization via fin'tete-turn (alsocalled reversal-bounded) pdas: there i s a uniformupper bound on th e number of times a pda can turnfrom pushing ( increasing the store) to popping (decreas ing the store). This bound corresponds to th ebound on the number of occurrences of nonterminalsfor- th e equivalent grc3JTlffi3.r. A proper hierarchy ofclasses, a l l properly included in CFL, is definedby varying this bound. Thus, linear languages arethose accepted by single-turn pdas. Rosenberg


    Q967al provided different ITBchine realizations ofth e l inear context-free languages using two-tapef ini te state acceptors; this enabled him to proveth e unQecidability of the universe problem(" L = L ~ : " ) .Instead of restricting th e occurrence of nonterminals in a l l derivations, one can require thatthere i s a round k such that , for each word wgenerated by the grarrrrnar, there is a der ivat ion ofw in which no sentential form contains more thank occurrences of nonterminals. Ginsburg andSpanier D-968aJ called such grammars derivationbounded and proved that they generate th e closureof th e l inear context-free languages under substitution. Nivat [1967J and Yntema [1967J studiedthis very illteresting family under different formulations and proved that i t is indeed a propersubfamily of CFL. Walljasper [1974J defined afamily intermediate between ultralinear and derivation bounded by placing a uniform upper bound onth e occurrnces of nonterminals only on lef t - tor ight derivations.As mentioned, restr ict ing th e number ofstates in a nondeterministic pda does no t l imiti t s power (depending on t he p reci se definition ofacceptance). Two pushdown store symbols likewisesuffice to define a ll cfls . However, restrictingth e store to one symbol plus an endmarker yieldsth e family of one counter languages, which is aproper subfamily. Greibach [1967J showed that atwo-way infinite hierarchy of context-free languages could be obtained by combining th e finiteturn restriction with a generalization of th ecounter restriction (the pushdown store contentsmust belong to a fixed bounded regular set) .

    Bounded LanguagesA language L is bounded i f t he re a re words

    wI , . . . ,wn such that L wl*"'wn* The studyof bounded context-free languages was very frui t ful for they have elegant algebraic characterizations which allowed precise proofs of difficultresults on inherent ambiguity. A bounded languageL wl*"'wn* is context-free i f and only i f i tsParikh mapping (i . e ., the set { ( i I ' . . ,in) Ii 1 in'}) . f . . . fWI . . . wn il l L lS th e illlte unlon 0

    l inear sets each with stratified periods [Ginsburgand Spanier,1966a ] , i t is an unambiguous contextfree language i f and only i f i ts Parikh mapping isth e finite union o f d is jo in t linear sets each withindependent strat i f ied periods [Ginsburg and Ullian,1966J. Inclusion, equivalence and ambiguityare decidable for context-free grammars which generate bounded languages [Ginsburg and Spanier ,1964;Ginsburg and Ullian,1966J. Ginsburg and Ullian[1966J used t he cha ract er izat ion o f unambiguousbounded context-free languages to prove that i t isundecidable whether a context-free grammar generates an inherently ambiguous language (one fo rwhich there is no equivalent unambiguous contextfree grammar).

  • 8/8/2019 Critic SPL


    f CSL .

    6 . Between Context-Free and Contex-Sensitive:Extensions of Context-Free Grammars and PdasThe concepts of a pushdown store automatonand a context-free grammar have been extended invarious ways. Usually such systems yield a familylying between context-free and context-sensitive.When th e input tape access was changed - - e. g ., two

    way input heads - - this did not necessarily hold;however, I ta ke th e l iberty of surveying th e differen t cases together.Stack like

    A pushdown store automaton can compute onlyon th e top of i t s work tape, adding, removing orchanging symbols , and cannot visi t th e inter ior withou t erasing th e symbols above i t . A stack automaton-- introduced by Ginsburg, Greibach and Harrison[1966,1967J -- can operate either in pushdown modeon top of th e store or in reading mode in th e in terior of th e store; when i t vis i ts th e inter ior ofthe store, i t can read but not write; i . e . , i t canno t change th e contents of th e interior of th e store.A nonerasing stack automaton can push but not pop(erase) while in pushdown mode. A checking automaton [Greibach,1968aJ is a nonerasing stack automatonwhich once in reading mode can never return to pushdown I T D d e ~ essentially a l l i t can do is write a wordon i ts working tape and then move back and forthchecking i ts working tape against i t s i nput t ape.A nested stack automaton [Aho,1967J can act in pushdown mode, in reading mode or in stack creationmode. In stack creat ion mode, i t can create a newnested stack below the currently scanned symbol; inpushdown mode, i t operates on th e top of th e mainstack or of one of the nested interior stacks; inreading mode, i t can move i t s head within one ofth e s ta ck s without writing or ou t of the bottom bu tnever ou t of th e top of one of the nested stacks.I t is the last mentioned restric t ion that keeps th enested stack automaton from having th e power of aTuring machine.

    The hierarchy of languages accepted by th ecorresponding one-way nondeterministic devices canbe roughly diagrammed:CHECKING t NONERASING STACK\


    The families of context-free and of nonerasing stacklanguages are incomparable. The f amil ie s o t languages accepted by the one-way nondeterministic(deterministic) devices of a l l these types (checking, nonerasing stack, pushdown store, stack,nested stack) have most of th e same closure and de cis ion properties . An interesting exception is th esubstitution o p e r a t o ~ : th e family of (nonerasing)stack languages is not closed under substitutionwhile the other three a r e ~ the substitution closureof the family of stack languages is properly included in th e family o f n es te d s ta ck languages[Greibach,1969J. These observations strongly Ln-fluenced th e development of AFL theory and i t svariations.


    f i ~ l t i p l e Tapes and HeadsWe have already seen v ar ia ti on s i n th e accessto th e in pu t tap e. Rabin and Scott [1959J considered f ini te automata with two-way input ( i . e ., th ereading head can move lef t or right on i ts input)and with two input tapes. Rosenberg consideredf inite automata with mul ti pl e i nput t apes [1964J or

    with many reading heads on one input tape [1965J.Two interesting problems were lef t open by thesepapers: are k+ 1 heads better than k for a f i-nite state acceptor reading i t s input tape in onedirection only? is equivalence decidable fo r d ete rministic f ini te state acceptors with n 2 one-wayinput tapes? The f i r s t question was recently sett led by Yao and Rivest [1976J. The second questionremains open for n 3; i t is easy to see thatfor n =2 and other variations

  • 8/8/2019 Critic SPL


    Spanier [l968l:i) defined aontroz, sets on grarrmaP8: agramnar G generates with cOntrol set R the setof words w for which i t has 1eft-to-right derivations with associated production labe1 strings inR. Not surprisingly, ro s to f these devices ;do no taffect th e power of:'''gI\31IIMrs with Type 0, 1 or 3productions bu t do'"affeGt those with Type 2 (con-text-free) produCtions. Further, there are closerelationships 'arrong' 'these d i f f ~ t systems withor witheut 1eftlrost restric t ions arid, with or talith -out appearance cheCking (which allOWS 'Certain .specified productions to be skipPed whetr' their ap plication is ' i J r i p o s s i b l e ) ~ . A good survey apPearsin .SalonE.a "'[1973] . .

    Rewriting is regulated in a slightlY,differen t fashion in ' scattered 'Context \granm!rs' [GreibaChand Hopcroft,f969]. Rules ar e grOuped into sequeh"ceS .as fu :matrix . ' ~ . buT they' IID.1st J:>e appliedin ~ l e 1 and'in order' from left: to right ; .a nonerasing restriction is applied. ,The family o,fs c a t t ~ d ~ c c m t e x t l ~ g u a g e s has thepropertiea "of :the' fmly of ' context-sensi1;lve languages, whichis one':reason i t has. so>'far i ~ e n impossible to'prove ~ e b n t ~ t . If , a sequence' [Al 0+wl' .", . , A n ~ W n ] . : ' 'car;' be : a p ~ l i e d ' ? J 1 1 ~ when t h eA l are.a l l the nontermmals iri th e 'strmg, one ohtams' theabsoZ,utely paraZ,Z,-el graJ11!7laPB of Raj1ich [1971].These .generate an interesting sUbset of one-waychecking autonatbn languages: there is a uniformupPer bOtmd on', th e number oftimes"-the checkingautomaton can visi t anytape s q ~ . '

    The family. of nested stack a u t ~ t a ' qorresponds to tM:> quite diffenmt 'extensions of .contextfree gramnars: th e i n d e ~ e a ~ ' o f Ah6 [1967]and th e or macro g r c3 IIlIEr S of M. Fischer [1968],both inspired by Constructs ir i prog:remnrlng lan-'guages .. '. In . both cases, context-free productionsar e aUgTriented by allowing arguments. ' i n ' the caseof ',nacre ~ s ' e a c h " n o n t e ~ l haS ~ places Occupied by: ' ~ ~ i n g s ; ae A ( x i ~ : ,'.Xn'>'... v I'eplaces a str:J:ng A(ul ' ,Un) by . v' whe!'eeach 'Xi . in v fs ' ~ p l a c e d by':' u i . The 01 ' ( o U t ~side-in)' restriction rreans that the nIle ' is ' appli-.cable only i f A is outeIm)s1:; i ' -e., nOt :ih -an' argunent place of'any nbnterm:i.rli.U; this pr'Oduces 'theclass of . INDEXED or nested stack languages... 'lheIO ( i n s i d e ~ o u t ) restric t ion neans that the rule ' i s applicable only i f A is innerm:>st; i .e . ,the 'U iare' terminal strings. The families of 01 and 10languages are incomparable.

    As: we have seen', th e syntax of ALGOL 60 wasone of ."the inspirations' :for'the study of context..;"free langUage's. Double-level 'context-free gr'amnars,or W ~ g r a n u a r s , were, proposed for"the syntaxof .,AI.mL 6'8 One level ' cif the :'grc:nmer producesstrings bfcharacterswmch n a Y " ' ~ E e , t i s e ~ fu1:he, nonterminals of ' the serond level, thus poss ib ly pro-~ . ~ ~ j ~ ~ = i o g i ~ f ~ t e : r :ulations apectr" in [Sirifzoff',1967] and[Chastellierand C O ~ r a u e r , 1 9 6 9 1 . Since' W-grairJnars'pt'Oducea ll r .e . sets, various restric t ions on their"generative power have been considered (cf.,[Greibach,1974]).


    L-Systems'"IrsYstems are defined -by .restric t ions on th eapplication of productions bu t have generated sucha large body ,of ' l i terature that t h e Y ~ ' m u s t be considered to fOrm a , s ~ p a r a t e topic. , 'l'heywere introduced bytJndermeyer [1968J to capttlrecertain .features of .d e v e l ~ a l systen'B, 'and thus fonnthe'qnly p o s t ~ e h a n s k y . t o p i c we sha1.i examine notinspi.redprllnarily by cariPuter science concepts.As we have seen, t he b as ic ideas' 'Iof L-systemswere prefigured in Chomsky's 1959 paper. In .1 -systems, r ul es a re applied in parallel - - eVerysymbol in a string is simultaneously :rewritten - and one considers the derived 'langUClge' (no .distinction 'between tenninals and"noritermi.:rials) as well asperhaps the ' t e ~ l l ~ . . In fact , "terminal'language" !"eally divide'S into two ideas: e i t ~ th e

    adutt language o f strings whichcanilot be rewrit1:eninto other strings or the e ~ t e n d e d ~ ofstrings over a "partiCUlar designated vocabulary.I f the r ul es a re contextfree' rules, we have OLsystems (o r L-systems without interactIons). I nta bZ,edL-systems, productions ar e grouped into setscalled tables and" during a rewriting step ea9h symbo l must be rewritten using a Proouciion fran th esame table. In a detel'l1linistic system, only one '.'rule fran a table is applicable t o a symbol, whilein a prtopagating .system, .n i les arenonerasing.Systems ar e designed mnem:mically: 'E for extendeCllanguage, T for " t a b l e d ~ ' r n"for d e t e r m i n i s t i c ~ Pfor' propagating. ' F r o m a " f o n n a l l a n g u a g e " i e w p o ~ t ,th e family of ~ languages is perllaps'thenostinteresting, 'Since i t .has 'the mst attractive prop-erties. "

    R o z e n b e r ~ ~ Ibucet [1971] established th ebasic of j OL languages. Rozei1berg[1973a,b] considered TOL and EroL languages.Blattner [1973] showed that eqUivalende i s tmde";' ' ( J 7 5 (1943) 115-133.

    Post, E. L., For'TnaJ reductions of th e ,o;eneral combinatorial decision problem, American dOUl)nal o fMathematics., 65 (1943) 197-215.

    Zg46Bloch, B., Studies in colloquial Japarlese , I I ,Language., 22 (1946) 200-21t8 .Harris, Z., Fr'om morpheme 1.:0 utterance., Language.,22 (1946) 161-183.Post., E.L." A variant of a recursively unsolvableproblenl, BulZ.Am.Math.Soe . ., .52 (1946) 26 L+-268.

    Post., E.L. , ReCtlrsivc unsolvabili ty of a problemof 111ue, ~ I Symbo Z z : c ~ LO(Jic., 12 (1947) 1-11.\0e lls. , R .:=. , Immediate constituents, Language. , 23(1947) ()1-ll7.195]Harris 'I ~ . Me thods in 8tnu:tura l Lingui-st ics. ,IJniv.Chicago Press, 1951.}'=leene, S. c. Rep:t\?sentation of events in nerve

    n c t ~ ~ and 3utoTYl3.ta, ProJect Rand Res. Memn.R ~ L ? 0 4 _ , Santa Bonica, CA (Dec. IS ,1951); publishedin Automata S't;udieD (Sharmon, C.D. and i1cCapthy,lJ., eds) fJr"'incton Univ . P:cess, Princeton, NJ(195FJ) pp 3-42.

    Lukasie\tJicz, IT ., / ir isro+lp'8 Sylto[J'Z:stic from th eStandpo1>z t o f ModeP'r2 Porma l Logic., Oxford Univ.Press, 1951.1952Harris , Z ., Discourse analysis, Language., 28 (1952)1-30.1953Bar-Hillel, Y., A quasi-arithmetical not at ion fo rsyn tac tic descr ip t ion , Language., 29 (1953)47-58.

  • 8/8/2019 Critic SPL




    Rhodes, I . , A new approach to th e mechanical syntactic ana ly si s o f Russian, M e c h . 1 ~ a n s . .' 6(1961) 33-50; publ.version of 1959 NBS Report.

    Parikh, R. J . , Language-generating devices, MITRes.Lab.Electr . .' QPR # 60 (1961) 1 9 9 - 1 2 2 ~ publ ished as "On context-free languages", JACM.' 13(1966) 570-581.

    Sherry, M. E., Syntactic Analusis i.n Automatic Translat ion, Ph.D. thesis , Harvard, August 1960, asRept. No. NSF-5.' Math.Ling.Autom.Transl . .'HarvardComputation Laboratory, Cambridge, Mass.

    1961Bar-Hillel, Y., Perles, M. and Shamir, E., On formal properties of simple phrase sturucture grammars., Z.Phonetik.Sprach.Komm . .' 14 (1961) 143-179.

    Chomsky, N., On th e notion "Rule of Grammar 11 , inStYiucture of Language and Its Mathematical Aspects.' Proc.Sym.Appl.Math.' 12 (1961) 6-24, Amer.t1ath. Soc.

    Floyd" R.\'J., A note on mathematical induction onphrase structure grammars, Inform.Contr . .' 4(1961) 353-358.Hays, D.G., Groupins; and dependency t he or ie s, i nProc.Natl.Symp. Machine Translation (H.P. Edmundson, ed.) (1961) 258-66; based on RM'ID Memo.P-19190, Feb. 1960.Gaifman, H., Dependency systems and phrase structure systems, RM'ID MeITD P-2315 (1961) j publishedin Inform.ContYi . .' 8 (1965) 304-337.Greibach, S ., I te r at io n s for the syntactic analyzer, Rept. No. NSF-6.' Math.Ling.Auto.Trans . .'Earvard Comput.Lab., Cambridge, Mass. June 1961.Irons., E.T . ., A syntax directed compiler fo r ALGOL

    60., CACM.' 4 (1961) 51-54.Hinsky, ti . L . ., Recursive unsolvabili tv of Post 'sDrob1em of "Tag\! and o ther t op ic s in t he theory

    o f Turing machmes, Ann.Math. 74 (1961) 437-354.OettinBer, A . ~ . , Automatic syntactic analysis andth e D u s h d o ~ store, in Struc.Lang.Math.Aspects.'PYioc.Symp.Appl.Math . .' 12 (1961) 104-129, Amer.Math. Soc.Ott., G. and Feins te in, n .H., Design of sequenti almachines from the i r regular expressions, JACM,8 (1961) 585-600.

    Ilyhill , J , Linear bounded automata, Tv'ADC Tech. Note6-.165 (1960).lJaur, P. (ed.) , Report on the algorithmic languaGeALGOL 60 , CACMy 3 (1960) 299-314.Samelson, K. and Bauer, F.L., Sequential formulatranslation, CACM.' 3 (1960) 76-83.Scheinberg, S. ., Note on th e .Boolean prope rt ie s o fcontext-free languages, Inform.Contr. .'3 (1960)372-375.

    :L:.-,ljaur;hton, R. and Yamada, H., Pep:ular expressionsand state p:raphs for automata" IHE-P(;EC.' 9 (1960)39-47.

    and Unso l v a l n ~ l i ty.' r-1cGraw-

    Chomsky, A.r,J., Some properties of phrase structuregrammars., MIT Res.Lab. o f E Z e c t p o n i c s ~ .Prog.Rept.-, 49 (April 1958) 108-111.Chomsky, A.lT. and I '1i ller , G.A. , Finite state languac:;es, Information (5 C o n t r o l ~ 1 (1958) 91-112.Copi, I . 1'1., Elot, C. and Vlright, \J. B., Real iza tionof events by l og ica l ne ts , d.Association ComputingMachinery.' 5 (1958) 181-196.

    Backus, J.W. ., The syntax and semantics of the proposed international alGebraic language of th eZtJr10h A t : M - ( ' ~ t:onfpP?!lcP) D y ' ( ~ C - . Tnt? ('onf.Inform. Processing.' !JNESCO PaYil's" 195.0, pp 125-132.

    Chomsky, 1'J., On certain forma.l properties of V'am-mars, I n f o r m . C o n t r . ~ (1959) 137-167.

    Chomsky, N., A note on phrase-structure grammar's"Inform.Contr . .' 2 (1959) 393-395.Rabin, M. O. and Scott, D. '\ Finite automata andtheir decision Droblems, IBM J.Reseapch Development.' 3 (1959) 114-125.Shepherdson , J . C. ,The reduction of two-way autoInCltato one-way automata, IBM J.Res.& D e v . ~ 3 (1959)198-200.1960Bar-Hille1, Y., Gaifman, H. and Shamir., E., Oncategorial and phrase structure gramrrB.l's, Bu l ZRes.Council I s r a e l ~ 9 (1960) 155-166.Bossert, W., The implementation o f p re di ct iv e a na lysis , Rept. No. N S F - 4 ~ Math.Linguistics and Automatic Translation.' Harvard Comput.Lab. ,June 1960.

    VS lS elf anotation"t i o n ~ (1954) 53-57.


    Davis, H., ComputabiHill , NY, 1958.Nerode, A. , Lineae autoInCiton transformation., Ppoc.

    Am8 Y1iean !vJczth. Soc1:ety-, g (1958) 541-544.

    1.956Chomsky, N. , Three models fo r th e descr ip ti on o flanguage, 1956 Symp. TheoY'L! M I T ~September 1956; published IRE 1'ans. I!zfoY' -mation Theory, IT-2 (1956) 113-124.

    Shannon, C. E. and HCCill)thy" J. (eds.), AutomcItaS t u d i e s ~ Princeton Univ.Press, Princeton, NLT,1956.

    Chomsky, 1J., Si:l'uctures Houton G Co. ,'S-Gravenhat:e, 1957.Har ri s, Z ., Co-occurrence and transformation inl inguistic structure., L a n g u a g e ~ 33 (1957)283-340.t"1vhill" lJ., Finite automata and t he reDresenta tionof events., 1i,4DC Tech.Rept. 52-624 (1957) 112-137.1'Jewel1 'I A. 1 Emd Sha,,'1., lJ. C. ., Programming th e logictheory machine 1 Pl)OC. lies tern doint Computer Conf.

    Los A n g e t e s ~ e4-, F e b . 7 9 5 7 ~ pp . 230-240.1958


  • 8/8/2019 Critic SPL


    ., IsY'cze l rT. Math.

    n.p. On c o n t e x t - f r e e languaGesautoITBta, Infollm. C()'rztY'. 6 (196])


    Greibach, S . , Inverses of Phrase Structure Generat o r s ~ Ph.D. t h e s i s , Harvard Univ. , June 1 9 6 3 ~a l s o Rent. NSF-II , Math. L 1 ~ n q . A u t o m . Transl. -' Harvard Comput. Lab. , Cambrirl.fYe, Mass. Publ. as 'l "Theundecidabil i ty o f +-he anmig;uity problem f o r mini mal l i n e a r grarnrnars;l I n . f o r m . C o n t r . ~ 6 (1 96 3) 1 19 125: "ForJn3.1 parsin,a.: systems,n C A C M ~ 7 (1964)4 9 9 - 5 0 L ~ , (md "A new nOrrrB1-form theorem f o r contehi=-free phrase :=ctructtwe p;rammars," ~ T A C M - , .7,8(1965) L+2-52.

    B.3.r-I1il1el., Y. ( e d . ) , L ( J J L ( ! U O ~ ] e lInd I n f " o r m a t 1 : o n ~/\rldison q p ~ ; l t = ~ v . , 'l rt3.S,C;. 'l 1 9 t J L ~ .

    ro 1(=', S. n. Fir)u.! -1j1'irnc 7il7 I i;('YlOt1:VP/1pY'ays , 'I F / n ' z ' t e , - , S ~ / o t p 'l Ph.D. t h e s i s ,Hcll'varci Univ . , Au,gus t 1(jb4: (-]] so P,ept. no . BL-36 'lTheor-y o f 5'71,-i reJu'n(l, Harvard Comnut. Lab,Cambridge.Cinsburg, S. an d Spanier., ~ . I 1 . , Rotmded ALGOL-likelangua,q:es., Trans.Am.Math. Soc. 113 (1964) 333-:368.

    Kurcxia, S. Y., Classes o f languar-,es and l i n e a r bOlli1ded automata, Inform. Corztjl. 7 (196L) 207-223.r1oore, I . F . (ed.) , Mzchines: SelectedPapCl)S, Addison \vesley., ReadinG'l Mass., 1964.

    'l /\ . C. O. (eel.)., No Les fe)}) th e Harvar'dSummer- School Program on Language Data Processing,Harvard Comput. La.b. , A u p ; u ~ ; t 19b Ji.

    IIartmanis 'l lJ.

  • 8/8/2019 Critic SPL


    Hennie, sequences and off-IDleTilling rnachin.e computations., IEEE 8ynrp -' PP .168172 pub1. as POne-tape off - l ine Till'"'ing machinecomputations, Inform.Contr." 8 (1965) 553-578 .KasaJI1i, T. ., lm eff ic ient recognition and syntaxanalysis algorithm f o r con tex t- fr ee languages,Rept. No. 2-, Univ. Hau)aii Contr. # AF19(628)4379, 1965.

    Knuth, D.E. , On th e t ranslat ion of languages froml e f t to r ip;ht, Inform.Cont-r.-, 8 (1965) 607-639.leHis, F.M., Stearns., R. E. and Hartrnanis, J . ,

    ~ I e r n o r y bounds for recognition of cont ex t -f reeillld context-sensit ive l a ~ g u a g e s ~ IEEE Symp.-,pp.191-202.Rosenberg, A.L., On multi-head f ini te automata,IEEE S y n p . ~ pp.221-228; publ. in IBM J.Res.&Dev.-, 10 (1966) 388-394.Ruby, S.S. and F is ch er , P .C ., T r a n s l a L ~ i o n a ] methoj2,and computat iona l complexity, .IEEE Symp.-, pp.173-178.

    Stearns., R.S . ., Hartmanis., J . and Lewis , P.N., Hieparchies of memory l imited computations., IEEESymp.-, pp.l79-190.1966Ginsburg, S., The Mathematical Theory of ContextFree Languages-, McGraw-Hill., 1966.Ginsburg, S. ., Greibach, S.A. and Harrison, M.A.,One-way stack automata - Extended abstract , P-roc,IEEE Symp. Switching and Automata T h e o r y ~ BerkeZey-, CA-, Oct. 1 9 6 6 ~ p p . 4 7 - 5 2 ~ publ. as "One-waystack automata," ~ T A C M - , 1 4 (1967) 389-418 .Ginsburg, S. and Spanier, E.B"! Semigroups, Presburger formulas and languages" Pacific rI.Math. ,16 (1966) 285-29b.Ginsburg, S. and Spanier, I .H . , Finite-tt1rll pushdown automata, SIAM rJ. C o n t - r o Z ~ 4 (1966) 429-453.ellisburg, S. and Ul1ian, lJ. , Ambiguity ir t contextfree l a n ~ J ) l a g e s , JACM, 1.3 (1966) 62-89.Greibach, S., The unsolvabili ty of t he r ecogn it ionof l inear context-free languages., rJACM-, 13 (1966)582-S87.

    Korenjak., A.J-. and Hopcroft , ,J.E., Simple deter ministic languages, IEEE S y m p . ~ pp . 36-46.Lewis, P.M. and Stearns, R.E., Syntax directedtransduction, IEEE S y m p . ~ pp.2l-35; publ. in

    rIACM-, 15 (1968) 465-488.Wirth, N. and Weber, H., EUl.EF., a general iza t ion

    of PlGOL and i t s formal defini t ion 'I CACM-, .9(1966) 13-25 and 88-89.

    Younger, D.H., Context-free language processing intime n 3 _, IEEE Symp. -' pp. 7- 20 ; pub l. as "Recognit ion and parsing of cont ex t -f ree lc3Il.guages intime n 3 -' Inform. Contr. - .10 (1967) J89-208.196?Aha, A.V. , Indexed grarnrnar)s - 1m extens io n ofcontext-free grammars, Proe.IEEE Symp. S7.Jitch-ing and Automata T h e o r y ~ Austin T X O c t . 1 9 6 7 ~pp.21-31; publ . in JACM-, 15 (1968) 647-671; also asTfNested stack automata, If ~ T A C M ~ 16 (1969) 383-406 .


    Fischer, .C . and Meyer .. A.P., Rea.] time counter4"1chines - Prel iminary version, TEEE Symp. -' pp.~ . 48 -15 4; an e>"'Panded version publ . a.s: Fischer,P.C., Meyer', A.F.. and Rosenberg, A.L. 'I "CollilterlTBchines and counter languages," M a t h . S y s t . T h . ~ 2(1968) 265-283.

    f ischer, P.C., Meyer, A.R. and Rosenberg, A.L.,Turing machll1es with sever al r ead-wri te heads,IEEE Symp.-, pp.117-127; "Real-time simulat ion of multihead tape units.," J A C M ~ 19(1972) 590-607.Ginsburg, S. and Greibach, S ., Abst rac t f amil iesof languages, IEEE Symp . ., pp.128-245; publ . inStudies in Abstract Families o f Languages., (Ginsburg, S ., Greibach, S. and Hopcroft, J . , eds),Mem.Am.Math.Soc . ., No. 87 (1969) 1-32.C:;insburg, S ., Greibach, S.A. and Harrison, M.A.,Stack automata and compiling, JACM (1967) 172-201.(-;ray, J.N. , Harrison, M.A. and Ibarra, O.H., Twoway pushdown automata, Inform.Contr . ., 11 (1967)30-70.Greibach, S ., An inf in i te hierarchy of context - freelanguages, IEEE Symp.-, pp.32-366; publ . in JACM.,16 (1969) 91 -10 6.

    Hartmanis, J . , The complexity of ~ ~ d e c i d a b 1 e problems in automata theory, IEEE Symp.-, pp.112-116;publ. in J A C M ~ 16 (1969) 160-167.

    HopcYDft, LT. E. and Ullman, J .D., An approach to awlified theory of automata, IEEE Symp. -' pp.1401 4 7 ~ publ. in Bell S y s t . T e c h . J . ~ 46 (1967)1793-1829.

    Hopcroft , J . E. and Ullman, lJ. D. ., Nonerasing stackautomata, I ' : J . C o m p u t . S y s t . S c i . ~ 1 (1967) 166-186.Kasarni, T., A note on computing time for recog

    n i t ion ot l anguages genera ted by l inear grammars., I n f o l m . C ~ n t r . - , 10 (1967) 209-214.K n u t h ~ D.E., A charac ter iza t ion of parenthesislanguages, Inform.Contr.-, 11 (1967) 269-289.McNaughton, P'O' Parenthesis gramnars, (JACM-, 14(1967) 490-500.Nivat, M., Transductions des langages de Chomsky-,Ph.D. thes is , Grenoble Univ., 1967; pUbl. inAnn.Inst.Fourier-, Grenoble., 18 (1968) 339-455.Rogers, rI . ., Theory of Recursive Functions andEffective Computability, McGraw-Hill , 1967.Rosenberg, A. L., A machine real izat ion of th e1L"'1ear context-free languages., InfoYWl. Cont-r.

    10 (1967) 175-188.Eosenberg, A.L ., P ea l- time def in ab le languages,

    ~ ACM , 14 (1967) 645- 662 .Fosenkrantz, D.lT., Prograrrnned grarrrrnars - A ne""device for gener)ating formal languages, IEEE8ymp.-, pp.14-20; publ. as "Programmed grammarsand classes of formal languages , fT J A C M ~ 16(1969) 107-131.

    Sintzoff , M., Existence of Vru"'1 Wijngaarden'ssyntax for every' recursively erll.nnerable se t ,Ann.Soc.Sci. de B ~ ~ x e l l e s ~ 2 (1967) 115-118.Stearns, R.E. ., A regular i ty t es t for pushdownmachines, Inform.Contr.-, 11 (1967) 323-340.

  • 8/8/2019 Critic SPL


    1968Fischer, M.J., Grammars with macro-l ike productions 3 Proc.IEEE.Symp.Switching and AutomataTheorY3 SchenectadY3 NY3 Oct.19683 pp.131-142- Abstract of Ph.D. thesis, Harvard Univ., May1968; also Rept. No. NSF-223 Math. Ling. Autom.Trans l . Harvard Computation Laboratory.Ginsburg, S. and Hopcroft, J. E., Two-way balloonautomata and AFL, IEEE S y m p . ~ p p . 2 9 2 - 2 9 7 ~ J A C M ~ 17 (1970) 3-13.Ginsbu rg, S. and Spanier , E .H., terivation-boundedlanguages, IEEE Syrrrp. pp. 306-314 publ. in J C S S ~2 (1968) 228-250.Ginsburg, S. and Spani er , E .H ., Con tr ol sets ongrammars, Math.Syst.Th. , 2 (1968) 159-178 .Greibach, S., Checking automata and one-way stacklanguages, IEEE S y m p . ~ pp.287-291; publ. in

    J . C o m p u t . S y s t . S c i . ~ 3 (1969) 196-217 .Greibach, S., A note on undecidable properties offormal languages, Math.Syst.Th. 2 (1968) 1-6.Hopcroft, J. E. and Ullman , J .D., Decidable and undecidable questions about automata, J A C M ~ 15(1968) 317-324.

    Hopcroft, J. E. and Ul lman, J .D., Relations betweentime and tape complexi ties, J A C M ~ 15 (1968)414-427.Lindenmeyer, A., Mathematical models fo r cellularinteractions in development, I and I I , J.Theoretical B i o l o g y ~ 18 (1968) 280-315.Ogden, W., A helpful result for proving inherentambiguity, M a t h . S y s t . T h . ~ 2 (1968) 191 -194 .1969Aho, A.V. and Ullman, J.D., Translations on a context-free gramnar, Proc.ACM Symp. Theory of Com-

    p u t i n g ~ Mari