Theoretical Computer Science past, present and …math.sharif.ir/faculties/uploads/daneshgar/Lectures...Theoretical Computer Science A. Daneshgar Outline TCS History ToC-past ToC-present

TheoreticalComputerScience

A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Theoretical Computer Sciencepast, present and future

Amir DaneshgarSharif University of Technology

http://www.sharif.ir/~daneshgar

19 February 2018(30 Bahman 1396)

[email protected] / 44

http://www.sharif.ir/~daneshgar

mailto:[email protected]


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Outline

1 A concise history of (T)CS

2 ToC: older motivations

3 ToC: current motivations

4 ToC: impacts

Randomness

Expanders

5 TCS: speculations on the future

2 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Prehistory < 1900

Abacus (∼ 3000 B.C.): Probably existed inBabylonia (present-day Iraq).Antikythera mechanism (∼ 80 B.C.): Discoveredin 1901, within an ancient Greek shipwreck offthe island of Antikythera.Mechanical adding machine (∼ 1620− 1640):Wilhelm Schickard, Blaise Pascal, GottfriedWilhelm Leibniz.Difference Engine: Charles Babbage(1791− 1871).First program: Ada Augusta Byron, Countess ofLovelace (1815− 1852).

3 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

A mathematical boost 1900− 1940

David Hilbert (1900): addressed the InternationalCongress of Mathematicians with three mainquestions on computability.Kurt Gödel (1931): Answered two importantquestions on consistency and completeness.Alan Turing (1936): constructing a formal modelof a computer, the Turing machine and answeredHilbert’s Entscheidungsproblem by introducingthe halting problem.Contributions of Church, Turing, Post, Kleene, ...on the concept of computation and recursiontheory motivated by the concept of a proof of atrue mathematical statement.

4 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Digital computers 1940− 1950

Mark I electromechanical computer (1944): Thecalculations required for ballistics during World War II ledto this construction by Howard H. Aiken.Colossus: Built by British to help Alan Turing breaking thecode behind the German machine, the Enigma.ENIAC (1946): built at the Moore School at the Universityof Pennsylvania.EDVAC (1944): Mauchly, Eckert, and John von Neumann.Z3 (1941): the first operational, general-purpose,program-controlled calculator built by Konrad Zuse.Invention of the transistor (1947): By John Bardeen,Walter Brattain, and William Shockley.Invention of magnetic core memory (∼ 1949): By JayForrester.

5 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Compiler design 1950− 1960

Invention of the notion of a compiler (1951): By GraceMurray Hopper at Remington Rand.First FORTRAN compiler (1957): John Backus and others.LISP and ALGOL (1958): John McCarthy and Alan Perlis,John Backus, Peter Naur and others.Integrated circuits (1959): Jack Kilby (Texas Instruments)and Robert Noyce (Fairchild Semiconductor).

6 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

The CS discipline 1960− 1970

Computer science as a discipline (1962): The firstcomputer science department was formed at PurdueUniversity.The rise of automata theory and the theory of formallanguages (1960′s): Noam Chomsky, Michael Rabin andothers.

Theory of computation: A culmination of ideas comingfrom digital design, compiler design, recursion theory andcomplexity.

7 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

The rise of modern TCS 1970− 1980

Steve Cook’s seminal paper on NP-completeness (1971):Design of CRAY-1 (1976): Seymour Cray.Theory of computation: Digital circuit design+Compilerdesign+computability.

8 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Theory of computation: Origins (computability)

Al-Kharazmi(∼ 780-850 AD)

John von Neumann(1903 - 1957)

The fundamental question in early daysCan we provide a computational solution to any problem?

9 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Theory of computation: Origins (computability)

Hilbert (1900): Can every true statement be proven (in afinite axiomatic system)?Gödel’s incompleteness theorem (1931): Some truestatements are unprovable!Can every function be computed?Turing’s undecidability theorem (1936): Some functionsare not computable!

10 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Theory of computation: the coding trick

A problemtype P consists of the following data:Constants: .......Given input: .....Query: ...........?

An exampleConstants: 3.Given input: the integer n.Query: Is n divisible by 3?

Main question: How do you provide the data?

11 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Theory of computation: membership problem

The membership problemtypeConstants: Σ a finite set of alphabets and a subsetL ⊆ Σ∗.Given input: a word x ∈ Σ∗.Query: Is it true that x ∈ L?

Fact: Any yes-no problem as P can be reduced to amembership problem for some subset (i.e. a language) LP .

Main question: How do you provide the data?

12 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

A computational model (machine)

It is an abstract machine consisting of:

A hardware (i.e. a control unit).A memory.A mechanism to read the input.A mechanism to write the output.

working as a discrete dynamical system with local property.

13 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Theory of computation: early impacts

There was a controversy regarding the definition of computationduring the early days.

Definition through computational machines.Definition through constructional procedures(i.e. grammars).Definition through computable functions(i.e. recursion theory).

Church-Turing thesis (1934-1937)At the level of Turing machines all rational models ofcomputation are equivalent!

14 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Theory of computation: complexity measures

The fundamental questionsDo natural hard problems exist?What are the consequences of answers YES or NO to thisquestion?

15 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Theory of computation: complexity measures

Main Question: How much should we pay for a computation?

Typical cost functionsTimeUsed memoryeven more complex cost functions!

The variable: is the length of the input!

Main objective: Study the behaviour of the cost function?

16 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Theory of computation: P vs. NP

Acceptable (i.e. effective) cost functionsEarly days (1960’s): Polynomially bounded functions.Nowadays: low-degree polynomially bounded.Trend (big data): O(n log n) or less!

The complexity class P(1960’s: Cobham, Edmonds and Rabin)Consists of all decision problems that can be solved bypolynomial-time bounded deterministic decider algorithms.

It is good if we can effectively (i.e. fairly easily) solve a problem!

17 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Theory of computation: P vs. NP

FactThere exists a large number of fundamental decision problems(say more than 2000) for which any claim for a solution can beverified efficiently (i.e. in polynomial time), however, noefficient (i.e. polynomial time) solver is known for any one ofthese problems!

The complexity class NP(1970’s: Cook, Levin, Karp)Consists of all decision problems that can be solved bypolynomial-time bounded nondeterministic acceptor algorithms.

NP-Complete: The class of hardest problems in NP.

18 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Example for an NP-complete problem: HamiltonianCycle

1

2

3

4

5

a

bc

de

1

2

3

4

5

a

bc

de

19 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

An important question!

The fundamental questionsDo natural hard problems exist?

One has to formulate easy and hard!We are not Zeus: hence it is generally believed that theanswer is YES!It is astonishing that existence of hard problems is quiteimportant in modern technological applications!

A fundamental problem: Do hard problems exist?i.e., P ?

= NP .

20 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Theory of computation: 1 M$ Millennium Problems(Clay institute 2000)

Seven problems each gives you $1 million at least!

P vs. NP

Determine the answer to the question P ?= NP .

Riemann HypothesisThe prime number theorem determines the average distributionof the primes. The Riemann hypothesis tells us about thedeviation from the average. Formulated in Riemann’s 1859paper, it asserts that all the ’non-obvious’ zeros of the zetafunction are complex numbers with real part 1/2.

There are 5 more problems and 4 more unsolved ones!

21 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impactsRandomnessExpanders

TCS-future

Randomness and usefulness of hard problems

A couple of fundamental problems:

Can one produce (i.e. simulate) almost ideal random bits?Is randomness useful in computation? Can one userandomness to get easier solutions?Can one reproduce a large number of random bits using asmall number of ideal ones?What does Riemann Hypothesis say about the set ofnatural numbers?

22 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

On the concept of a “Proof"

How to make sure that a claim is true?A sound proof is a valid argument for the correctness of aclaim.

To make sure that a claim is true it is quite sufficient tohave/see/verify a proof of it!

However, to make sure that a claim is true it is alsosufficient to make sure that there exists a proof of it!!!!

23 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Example: the blind and the twins

Yellow Blue

Blue Yellow

Blue Yellow

Yellow Blue××

Dishonest Honest

24 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Example: BPP and secure communication

BPP is the randomized counterpart of P.

The class BPPA language L is in BPP if there exists a randomized algorithmA such that

x ∈ L implies that Pr(A(x) = accept) > 3/4.x 6∈ L implies that Pr(A(x) = reject) > 3/4.

A fundamental problem: Do hard problems exist?i.e., NP −BPP ?

= ∅.

25 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Example: BPP and secure communication

In secure communication:

It is assumed that everyone knows about the details ofalgorithms ENC and DEC except the security parameter κ(i.e. the key).The adversary problem: {p | ∃ κ ENC(κ, p) = c} ∈ NP .(oversimplified!)

NP −BPP = ∅ ⇒ there is no secure communication system!

Study of a possible converse is the subject of modern provablecryptography!

26 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Probabilistic algorithms

Question 1:Does randomness fundamentally help in computation? i.e. arethere problems with probabilistic polynomial-time algorithmicsolutions but no deterministic one?

Question 2:Does NP require strictly more than polynomial time?i.e. there exist natural hard problems!

At least one of the answers is NO!!

27 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Probabilistic algorithms

More on the story of random bits in computability!

On the existence of hard problemsAssuming factorization of integers has no efficient algorithmimplies P 6= NP .

[Blum, Micali, Yao, Nisan, Impagliazzo, Wigderson]Existence of hard problems (say P 6= NP or something similar!)implies the existence of Pseudo-random generators.

28 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Probabilistic proof system

A proof is an argument for a claim.Main question: Is it valid?

∃ a probabilistic verifier V (claim, arg) for the claim, suchthat

If the claim is true then V (claim, arg∗) = true for someargument arg∗.If the claim is false then V (claim, arg) = false for everyargument arg with probability more than 0.99.

29 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Probabilistic checkable proofs (PCP’s)

∃ a probabilistic verifier V (claim, arg) for the claim, suchthatthe verifier only reads at most 10 bits of the argument atrandom.

[Arora-Safra, Arora-Lund-Motwani-Sudan-Szegedy-Hastad]Every proof can be efficiently transformed into a PCP!

30 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Applications

Assuming the existence of natural hard problems:

Non-approximabilitySome NP-complete problems (e.g. MaxClique, MaxSat, ...) arenon-approximable!

Grading answer sheetsThere exists a randomize procedure using which you can gradethe answer sheets of your exam in which you only read at most10 random characters from each sheet and the maximumprobability of giving a wrong grade is less than 0.0001!

AuthenticationThere exists a randomize procedure using which you can proveto your bank on the Internet that you are YOURSELF withoutrevealing your electronic signature at all!31 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Graphs and Matrices

1

2

3

4

5

a

bc

de

1 2 3 4 5 a b c d e12345abcde

0 1 0 0 1 1 0 0 0 01 0 1 0 0 0 1 0 0 00 1 0 1 0 0 0 1 0 00 0 1 0 1 0 0 0 1 01 0 0 1 0 0 0 0 0 11 0 0 0 0 0 0 1 1 00 1 0 0 0 0 0 0 1 10 0 1 0 0 1 0 0 0 10 0 0 1 0 1 1 0 0 00 0 0 0 1 0 1 1 0 0

32 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Random bits

A fundamental problemDesign of pseudo-random generators, and extractors are amongthe most fundamental problems in ToC, Engineering andScience.

33 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Random regular graphs: main questions

0 dll 2

spectral gap

n

What can be said about the spectral gap?What can be said about other connectivity relatedparameters as chromatic number, expansion, Hamiltonicity.....Analysis of the extremal cases are usually quite challengingproblems.

34 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Primes and Zeta Functions: definitions

Consider a geometric space made of primes and theiramalgams.e.g. Q, number fields, function fields, Riemannianmanifolds, graphs, ...The connectivity of the space is naturally related tonumber of primes and how they are mixed together.Connectivity is a fundamental concept that can be studiedand measured in many different ways.A zeta (in general L) function is a mathematical conceptthat is supposed to present and reflect all these aspects ina reasonable way!

35 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Primes and Zeta Functions

Rational numbers

The Riemann zeta function ζ(s) =

∞∑n=1

1

ns=

∏p prime

(1− p−s)−1

is related to Hecke operators but possibility for relation to anatural diffusion is not fully understood yet.

Graphs

The Ihara zeta function ζ(u) =∏

[P ] prime

(1−u`([P ]))−1 is related

to the adjacency operator and this relation is fully understood.

Apply u := q−s to compare!

36 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Ramanujan Graphs

Riemann Hypothesis (RH) for QIf ζ(s) = 0 and 0 < Re(s) < 1 then Re(s) = 1/2.

Riemann Hypothesis (RH) for graphs (Ihara zeta func.)

If ζ(q−s)−1 = 0 and 0 < Re(s) < 1 then Re(s) = 1/2.

This is equivalent to the following:

Ramanujan graphsA (q + 1)-regular graph with adjacency matrix A satisfies RH iffit is Ramanujan, i.e. if

µdef= max{|λ| | λ ∈ Spec(A) & |λ| 6= q + 1}

then µ ≤ 2√q.

37 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Spectrum of a Ramanujan Graph

Note: Regularity is d = q + 1.

0 dll 2

2 1d -2 1d- -

n

Nontrivial eigenvalues are smallThe graph is sparse but highly connectedIt is a good sparse approximation of a complete graphAlon-Boppana 1986: We can not beat the bound 2

√d− 1

asymptoticallyThe bound 2

√d− 1 is the spectral radius of the infinite

d-regular tree (i.e. the universal cover!)38 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Expanders in TCS

0 dll 2

2 1d -2 1d- -

n

Expanders are sledgehammers of TCS! They are used in:

DerandomizationComplexity theoryError correcting codesCompressed sensingCommunication networksApproximate countingMeasure theoryNumber theory...39 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

Ramanujan graphs of arbitrary degree

A. W. Marcus, D. A. Spielman, N. Srivastava, 2013+Published in Annals of Mathematics (2015)There exist (arbitrarily large enough) bipartite regularRamanujan graphs of arbitrary degree.

The proof is based on the fundamental technique of interlacingfamilies of polynomials which is also used by the same authorsto prove Kadison-Singer Problem.

40 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present


TCS-future

A Royal Road to Mathematics

Propaganda!

Euclid of Alexandria (about 300 BC):There is no Royal Road to geometry.

Graph theory zooGraph theory is a Royal Road to the heart of modernmathematics that is free to be used by any curious scholar!

The road passes through Computer Science land of Oz!

41 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

Some hot topics

Big data and fast (O(n log n)) algorithms.Machine learning and foundations of AI.Provable cryptography.Quantum computation.Theoretical biology.Network theory and modeling.Image processing and computer graphics.Compressed sensingFoundations of mathematics and programming.

42 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future

A quotation!

In the prelude of “Récoltes et Semailles", AlexandreGrothendieck makes the following points on the search forrelevant geometric models for physics and on Riemann’s lectureon the foundations of geometry.It must be already fifteen or twenty years ago that, leafingthrough the modest volume constituting the complete works ofRiemann, I was struck by a remark of his “in passing".

... it could well be that the ultimate structure of space isdiscrete, while the continuous representations that we make ofit constitute perhaps a simplification (perhaps excessive, in thelong run ...) of a more complex reality; That for the humanmind, “the continuous" was easier to grasp than the“discontinuous", and that it serves us, therefore, as an“approximation" to apprehend the discontinuous.

43 / 44


A. Daneshgar

Outline

TCS History

ToC-past

ToC-present

ToC-impacts

TCS-future Thank you!Comments are Welcome

[email protected]

44 / 44

Documents

Theoretical Computer Science past, present and …math.sharif.ir/faculties/uploads/daneshgar/Lectures...Theoretical Computer Science A. Daneshgar Outline TCS History ToC-past ToC-present