1 Formal Languages and Theory of Computation Wen-Hsiang Tsai 蔡文祥講座教授 2011/9 Taj Mahal in India

1

Formal Languages and Formal Languages and Theory of ComputationTheory of Computation

Wen-Hsiang TsaiWen-Hsiang Tsai蔡文祥講座教授蔡文祥講座教授

20120111/9/9

Taj Mahal in India

2

Chapter 0Chapter 0

IntroductionIntroduction

Opera House in Sidney, Australia

Chapter. 0 – p. 3

OutlineOutline

0.0 Reasons for Taking This Course0.0 Reasons for Taking This Course

0.1 General Concepts0.1 General Concepts

0.2 Problems Studied in Theory of Computation0.2 Problems Studied in Theory of Computation

0.3 Fields Related to Theory of Computation0.3 Fields Related to Theory of Computation

0.4 Applications of Theory of Computation0.4 Applications of Theory of Computation

0.5 Brief History of Theory of Computation0.5 Brief History of Theory of Computation

0.6 Textbooks and Course Grading0.6 Textbooks and Course Grading

Chapter. 0 – p. 4

0.0 Reasons for Taking This Course0.0 Reasons for Taking This Course

Philosophy of computer science (CS)Philosophy of computer science (CS) Indispensable for complete study of CSIndispensable for complete study of CS Broadening views in all fields of CSBroadening views in all fields of CS Enhancing problem solving capabilityEnhancing problem solving capability Basic training for undergraduate studentsBasic training for undergraduate students Background course for graduate studentsBackground course for graduate students Qualification exam topic for Ph. D. students.Qualification exam topic for Ph. D. students.

Chapter. 0 – p. 5


Course title ---Course title ---““Formal Languages and Theory of CompuFormal Languages and Theory of Compu

tation” (tation” ( 正規語言與計算理論正規語言與計算理論 )) Scope of course:Scope of course:

Formal languagesFormal languages Automata theoryAutomata theory ComputabilityComputability Computational complexityComputational complexity

Chapter. 0 – p. 6

0.1 General Concepts0.1 General Concepts Formal Languages ---Formal Languages ---

Definition --- Definition ---

Study of “Study of “strings of symbolsstrings of symbols,” including their,” including their

descriptions, properties,descriptions, properties,

generations, recognitions,generations, recognitions,

compiling, applications, …compiling, applications, …

Chapter. 0 – p. 7

0.1 General Concepts0.1 General Concepts Formal Languages ---Formal Languages ---

Concepts (1/2) ---Concepts (1/2) ---

Strings of symbols include those ofStrings of symbols include those of

natural languages,natural languages,

computer languages,computer languages,

discrete signals, …discrete signals, …

Chapter. 0 – p. 8


Formal Languages ---Formal Languages --- Concepts (2/2) --- Concepts (2/2) ---

Results of study of Results of study of automatic processing of automatic processing of

symbolic informationsymbolic information

Parallel to Parallel to numerical mathematical processinumerical mathematical processi

ngng

Similar to the role of Similar to the role of set theoryset theory ( (集合論集合論 )) in in

mathematicsmathematics

Chapter. 0 – p. 9

0.1 General Concepts0.1 General Concepts Automata Theory ---Automata Theory ---

A question --- A question ---

Do you know how a vending machine Do you know how a vending machine

works? Can you design one?works? Can you design one?

Vending machine room seen in Hokkaido, Japan 2004

Chapter. 0 – p. 10


Automata Theory ---Automata Theory ---

An example (1/3) --- An example (1/3) ---

How to design a vending machine?How to design a vending machine?

Use a Use a finite automatonfinite automaton!!




An example (2/3) --- An example (2/3) ---

Assumptions (for simplicity):Assumptions (for simplicity):

Only NT 5-dollar and 10-dollar coins are used.Only NT 5-dollar and 10-dollar coins are used.

Only drinks all of 20 dollars are sold.Only drinks all of 20 dollars are sold.



An example (3/3) ---An example (3/3) ---

Requiring “memory” called “states” for the design.Requiring “memory” called “states” for the design.

Notes: see next page. Notes: see next page.

$0startstart

$5 $5

$10 $10

$5

$15

$5

$10

$10

$10$10 $5

$20transition diagram



$10$10 $5 is returned as “output” (not shown)

$20 “Final state”


Notes for the last diagram:Notes for the last diagram:

$0startstart “Start state”





Study of dynamic behaviors of “discrete-Study of dynamic behaviors of “discrete-

parameter information systems” in form ofparameter information systems” in form of

“abstract computing devices” or “abstract computing devices” or

“machines.”“machines.”




Examples of discrete-parameter Examples of discrete-parameter

information systems ---information systems ---

digital systems, nerve systems, languagesdigital systems, nerve systems, languages

information transmission systemsinformation transmission systems

human-environment interactions, …human-environment interactions, …



Automata Theory ---Automata Theory --- Three major models of automata ---Three major models of automata ---

generatorgenerator --- with output and --- with output and withoutwithout input input

acceptoracceptor --- with input and --- with input and withoutwithout output output

transducertransducer --- --- bothboth with input and with output with input and with output

generator acceptor transducer

triggtrigger er signasignall

Yes or No



Examples ---Examples --- generator generator --- ---

““natural language” grammarnatural language” grammar (generating “sentences” spoken by people)(generating “sentences” spoken by people)

reception robotreception robot

(speaking organized words and sentences)(speaking organized words and sentences)

context-free grammarcontext-free grammar

(generating strings of symbols)(generating strings of symbols) ******

(*** abstract models studied in this course)(*** abstract models studied in this course)

Reception robot --- Expo 2005



Automata Theory ---Automata Theory --- Examples ---Examples ---

acceptor acceptor --- --- digital lockdigital lock (accepting digits)(accepting digits)

lexical analyzerlexical analyzer (recognizing computer language (recognizing computer language

keywords)keywords)

finite automatonfinite automaton

(accepting valid strings of symbols)(accepting valid strings of symbols) ******(*** abstract models studied in this course)(*** abstract models studied in this course)

Digital lock




Examples ---Examples --- transducer transducer --- ---

InterpreterInterpreter (translating natural languages)(translating natural languages)

CompilerCompiler (translating high-level languages into (translating high-level languages into

machine codes)machine codes)

Turing machineTuring machine

(transforming strings of symbols)(transforming strings of symbols) ******(*** abstract models studied in this course)(*** abstract models studied in this course)

“Interpreter”雙面翻譯 (a movie)



Computability ---Computability ---


Study of problem solving capabilities of Study of problem solving capabilities of

computational models.computational models.




Problem types based on Problem types based on resourcesresources --- ---

Impossible problemsImpossible problems

Possible-with-unlimited-resources-but-impoPossible-with-unlimited-resources-but-impo

ssible-with-limited-resourcesssible-with-limited-resources problems problems

Possible-with-limited-resources problemsPossible-with-limited-resources problems




Problem types based on Problem types based on timetime --- ---

Undecidable problemsUndecidable problems

Intractable Intractable problemsproblems

Tractable problemsTractable problems



Computability ---Computability --- Advantage ---Advantage ---

Studies of computability help us Studies of computability help us not to not to

waste timewaste time on “unsolvable problems” on “unsolvable problems”

already investigated before.already investigated before.



Computational complexity ---Computational complexity ---


Study of “Study of “efficiencyefficiency” of problem solving.” of problem solving.



Computational complexity ---Computational complexity --- Concepts (1/2) ---Concepts (1/2) ---

To unify comparison, an To unify comparison, an abstract modelabstract model is is

needed as the machine for executing needed as the machine for executing

problem solutions.problem solutions.

Usually the most famous “Usually the most famous “Turing machineTuring machine” ”

(an automaton) is used.(an automaton) is used.



Computational complexity ---Computational complexity ---

Concepts (2/2) ---Concepts (2/2) ---

Turing machine, though simple, has been Turing machine, though simple, has been

proved to be able to proved to be able to simulate any problem simulate any problem

solving steps (“algorithms”) solving steps (“algorithms”) designed by designed by

human beings! human beings!

Turing machine is the Turing machine is the foundationfoundation for for

development of modern computation theory!development of modern computation theory!


0.2 Problems Studied in Theory of 0.2 Problems Studied in Theory of ComputationComputation

Essential problems studied ---Essential problems studied ---

““What are the fundamental capabilities and What are the fundamental capabilities and

limitations of computers?”limitations of computers?”

What can a computer do at all? --- What can a computer do at all? ---

studied in the domain of studied in the domain of ComputabilityComputability!!

What can a computer do efficiently? --- studied What can a computer do efficiently? --- studied

in the domain of in the domain of Computational complexityComputational complexity!!


0.2 Problems Studied in Theory of 0.2 Problems Studied in Theory of ComputationComputation

A basic concept ---A basic concept --- A Turing machine conceptually is no A Turing machine conceptually is no

more than more than a pen and a paper with a pen and a paper with several reasoning rulesseveral reasoning rules!!

=？=？

Turing machine=

？

=？


0.3 Fields Related to Scope of 0.3 Fields Related to Scope of Theory of ComputationTheory of Computation

Fields Related theory

Compiling theory Formal languages

Switching circuit theory Automata theory

Algorithm analysis Computational complexity

Natural language processing Formal languages

Syntactic pattern recognition Formal languages

Programming languages Formal languages

Artificial intelligence Formal languages and automata theory

Neural networks Automata theory


0.4 Applications of Theory of 0.4 Applications of Theory of ComputationComputation

Applications ---Applications --- Text analysis ---Text analysis ---

text search (text search ( 文辭搜尋文辭搜尋 ))

text editingtext editing ( (文句編輯文句編輯 ))

Compiler design (Compiler design ( 編譯器設計編譯器設計 )) lexical analysis (lexical analysis ( 辭彙分析辭彙分析 ))

parser generation (parser generation ( 剖析器產生剖析器產生 ))



Applications ---Applications --- Language designLanguage design

programming language design (programming language design ( 程式語言設程式語言設計計 ))

document description language design (document description language design ( 公公文文描述語言設計描述語言設計 ))

e.g., HTML, XML, …e.g., HTML, XML, … picture language design (picture language design ( 圖畫語言設計圖畫語言設計 ))

e.g., SVG, VHML, …e.g., SVG, VHML, … special language design (special language design ( 特別語言設計特別語言設計 ) )



Applications ---Applications --- Digital system designDigital system design

computer design (computer design ( 計算機設計計算機設計 )) special digital system design (special digital system design ( 特殊數位系統特殊數位系統

設計設計 )) Protocol modeling and verification (Protocol modeling and verification ( 網路網路

協定模式化與驗証協定模式化與驗証 )) Expert system design(Expert system design( 專家系統設計專家系統設計 )) Cryptography(Cryptography( 密碼學密碼學 ) …) …



A specific application --- A specific application --- special digital special digital system designsystem design

Mainly based on automata theory --- Mainly based on automata theory --- using finite automata or “using finite automata or “sequential machinessequential machines” ”

(finite automata (finite automata both both with with input and input and with with outputoutput))

Some specific examples of digital Some specific examples of digital systems --- systems --- Elevators, vending machines, traffic signal Elevators, vending machines, traffic signal

control systems …control systems …


0.5 Brief History of Theory of 0.5 Brief History of Theory of Computation Computation

At beginning of 19th century, mathemAt beginning of 19th century, mathematicians believe: aticians believe:

Axiomatization of mathematicsAxiomatization of mathematics(( 數學公理化數學公理化 ))

That is,That is,

it is possible to decide the truth of any it is possible to decide the truth of any mathematical statement thru use of mathematical statement thru use of infinference rules erence rules andand axiom systems axiom systems..


0.5 Brief History of Theory of 0.5 Brief History of Theory of ComputationComputation

In 1931, Kurt GIn 1931, Kurt Göödel proposed the del proposed the incompleteness theorem incompleteness theorem which says:which says:

the truth of certain mathematical the truth of certain mathematical statements statements cannot cannot be proved by the use be proved by the use of mathematical axioms.of mathematical axioms.

That is, That is,

some mathematical problems some mathematical problems cannotcannot be be solved by algorithms.solved by algorithms.



From then, the ideal of From then, the ideal of axiomatization axiomatization

of mathematicsof mathematics was broken. was broken.

Then, mathematicians started to Then, mathematicians started to

think the scope of algorithm capabilities, think the scope of algorithm capabilities,

design theoretical computational machinesdesign theoretical computational machines, & , &

study their capability limitations.study their capability limitations.



Many theories were proposed, but the Many theories were proposed, but the

corresponding machines corresponding machines could not be could not be

manufacturedmanufactured at that time. at that time.



In 1936, In 1936, Turing Turing proposed the model proposed the model of of

universal algorithm machineuniversal algorithm machine,,

which later was calledwhich later was called

Turing machineTuring machine

mentioned before.mentioned before.



Some similar ideas were proposed by Some similar ideas were proposed by

other scholars, includingother scholars, including

Stephen C. Stephen C. KleeneKleene,,

Alonzo Alonzo ChurchChurch,,

Emil Emil PostPost, etc.,, etc.,

in different forms of models.in different forms of models.



In 1943, neural physiologistsIn 1943, neural physiologists

Warren S. Warren S. McCulloch McCulloch andand

Walter Walter PittsPitts

developed developed finite-state systems finite-state systems to to simulate neural networks in biological simulate neural networks in biological systems.systems.

They are pioneers of They are pioneers of automata theoryautomata theory..



In late 1940’sIn late 1940’s ，， Von Neumann Von Neumann

proposed the idea of proposed the idea of

stored programstored program

for computer models.for computer models.



In 1951, a In 1951, a real computer real computer following the idfollowing the idea of “stored program” was constructed:ea of “stored program” was constructed:

UNIVAC IUNIVAC I,,

the world's first commercially available cthe world's first commercially available c

omputeromputer (manufactured (manufactured by Eckert-Mauchby Eckert-Mauch

ly Computer Company) .ly Computer Company) .



In late 1950’s, linguist In late 1950’s, linguist N.N. Chomsky Chomsky proposed proposed aa mathematical model for mathematical model for grammars grammars of natural languages.of natural languages.

In 1956, he proposed further the In 1956, he proposed further the concept of concept of context-free grammarcontext-free grammar, , which may be used for defining which may be used for defining computer languages.computer languages.



In 1956, In 1956, Kleene Kleene proposed the concept proposed the concept of of finite automaton finite automaton for simulating neurafor simulating neural networks proposed by McCulloch and l networks proposed by McCulloch and Pitts.Pitts.

He also proposed He also proposed regular expressions regular expressions tto describe strings of symbols, and provo describe strings of symbols, and proved them ed them equivalent equivalent to finite automata.to finite automata.



In 1959 and 1960, In 1959 and 1960, John W. Backus & John W. Backus & Peter Naur Peter Naur proposed sequentially a spproposed sequentially a special expression for computer languagecial expression for computer language grammars, called later e grammars, called later Backus-Naur Backus-Naur formform (BNF) . (BNF) .

It was used to describe computer languaIt was used to describe computer language ge ALGOL-60ALGOL-60, leading to development of , leading to development of compiling theorycompiling theory..



In 1969, In 1969, Stephen A.Stephen A. Cook Cook found that found that computation problems may be computation problems may be separated into separated into tractabletractable and and intractable intractable ones.ones. Intractable problems are also called Intractable problems are also called NP-NP-

hardhard problems. problems. Such problems cannot be solved by Such problems cannot be solved by

computers except very computers except very small instances small instances (with only small-sized inputs).(with only small-sized inputs).



Textbook:Textbook: Hopcroft, Motwani, Ullman, Hopcroft, Motwani, Ullman, Introduction to AutoIntroduction to Auto

mata Theory, Languages, and Computationmata Theory, Languages, and Computation (3r (3rd Edition), Addison-Wesley, Boston, MA, USA, d Edition), Addison-Wesley, Boston, MA, USA, 20072007

Reference textbook:Reference textbook: 蔡文祥蔡文祥 , , 正規語言與計算理論正規語言與計算理論 (Formal Languag(Formal Languag

es & The Theory of Computation)(es & The Theory of Computation)( 尚未出版書尚未出版書籍；中文籍；中文 ) () ( 參考用參考用 ))



Course grading:Course grading: Homeworks 33.33%Homeworks 33.33%

mid-term exam 33.33%mid-term exam 33.33%

final exam 33.33%final exam 33.33%

Website for downloading course material aWebsite for downloading course material and quizzes :nd quizzes :

http://www.cis.nctu.edu.tw/~whtsai/http://www.cis.nctu.edu.tw/~whtsai/

Documents

1 Formal Languages and Theory of Computation Wen-Hsiang Tsai 蔡文祥 講座教授 2011/9 Taj Mahal in India

1 Formal Languages and Theory of Computation Wen-Hsiang Tsai 蔡文祥講座教授 2011/9 Taj Mahal in India