Discrete MathematicsDiscrete Mathematics CS 2610cobweb.cs.uga.edu/~potter/dismath/Rev-422.pdf ·...

Discrete MathematicsDiscrete Mathematics CS 2610

1

Propositional Logic: Precedence

Logical Precedence

By convention…

L gOperator

r n

¬ 1∧ 2∨ 3→ 4↔ 5

Examples:Examples:Examples:Examples:¬ p p ∧ q → r is equivalent to ((¬ p) p) ∧ q) → r

2

p p ↔ q → r ∧ s is equivalent to p p ↔ (q → (r ∧ s))

Logic and Bit OperationsA bit is a binary digit: 0 or 1.

Bits are usually used to represent truth values.y pBy convention:

0 represents “false”; 1 represents “true”.

B d l l Bit operations correspond to logical operators, replacing false by 0 and true by 1

x y ¬ x x ∧ y x ∨ y x ⊕ y

0 0 1 0 0 0

0 1 1 0 1 1

1 0 0 0 1 1

3

1 1 0 1 1 0

Propositional Equivalences

A tautology is a proposition that is always true.Ex : p ∨ p p pppEx.: p ∨ ¬ p

FT

TF

p ∨ ¬ p¬ ppTT

A contradiction is a proposition that is always false.Ex.: p ∧ ¬ p p ∧ ¬ p¬ pp

FT

TF

p ppp

FF

A contingency is a proposition that is neither a tautology nor a contradiction.

Ep → ¬ p¬ pp

4

Ex.: p → ¬p FT

TF

FT

Propositional Logic: Logical EquivalenceIf p and q are propositions, then p is logically equivalent to q if their truth t bl th tables are the same.

“p is equivalent to q ” is denoted by p ≡ qp is equivalent to q. is denoted by p ≡ q

p q are logically equivalent if their p, q are logically equivalent if their biconditional p ↔ q is a tautology.

5

Propositional Logic: Logical Equivalences• Identity

p ∧ T ≡ pp ∨ F ≡ p

• Dominationp ∨ T ≡ Tp ∧ F ≡ F

• Idempotencep∨p≡ pp∧p≡ p

• Double negation

6

¬¬p≡ p

Propositional Logic: Logical Equivalences• Commutativity:

p ∨ q≡ q ∨ pp ∨ q q ∨ p p ∧ q≡ q ∧ p

• Associativity:

(p ∨ q) ∨ r ≡ p ∨ ( q ∨ r )

( ) ( )(p ∧ q) ∧ r ≡ p ∧ ( q ∧ r )

7

Propositional Logic: Logical Equivalences• Distributive:

p ∨ (q ∧ r) ≡ (p ∨ q) ∧ (p ∨ r)p ∨ (q ∧ r) ≡ (p ∨ q) ∧ (p ∨ r)

p ∧ (q ∨ r) ≡ (p ∧ q)∨ (p ∧ r)

D M ’• De Morgan’s:

¬(p ∧ q) ≡ ¬p ∨ ¬q (De Morgan’s I)

¬(p ∨ q) ≡ ¬p ∧ ¬q (De Morgan’s II)

8

Propositional Logic: Logical Equivalences• Excluded Middle:

p ∨ ¬p ≡ T

• Uniqueness: p ∧ ¬p ≡ F

• A useful LE involving →:p → q ≡ ¬p ∨ q

9

Propositional LogicUse known logical equivalences to prove that two propositions are logically equivalent

Example:

¬(¬ p ∧ ¬ q) ≡ p ∨ q¬(¬ p ∧ ¬ q) ≡ p ∨ q

W ill th LEWe will use the LE,

¬¬p ≡ p Double negation¬(p ∧ q) ≡ ¬p ∨ ¬q (De Morgan’s II)

10

Predicate LogicDefine:UGA(x) = “x is a UGA student.”Universe of Discourse – all peoplex is a variable that represents an arbitrary individual in the Universe of Discoursein the Universe of Discourse

A predicate P, or propositional function, is a function that maps objects of the universe of discourse to propositionsmaps objects of the universe of discourse to propositions

UGA(Daniel Boone) is a proposition.UGA( ) i t itiUGA(x) is not a proposition.

UGA(x) is like an English predicate template

11__________ is a UGA student

Predicate Logic: Universal QuantifierSuppose that P(x) is a predicate on some universe of discourse.

The universal quantification of P(x) (∀x P(x) ) is the proposition:

“P( ) i t f ll i th i f di ”“P(x) is true for all x in the universe of discourse.”

∀x P(x) reads “for all x, P(x) is True”∀x P(x) reads for all x, P(x) is True

∀x P(x) is TRUE means P(x) is true for all x in UD(x).

∀x P(x) is FALSE means there is an x in UD(x) for which P(x) is false.

12

Predicate Logic: Existential QuantifierSuppose P(x) is a predicate on some universe of discourse.

The existential quantification of P(x) is the proposition:

“There exists at least one x in the universe of discourse such that P(x) is true.”( )

∃ x P(x) reads “for some x, P(x)” or “There exists x, P(x) is True”

∃x P(x) is TRUE means there is an x in UD(x) for which P(x) is true.

∃x P(x) is FALSE means : for all x in UD(x) is P(x) false

13

Predicates - Quantifier negation

∀x P(x) means “P(x) is true for every x ”∀x P(x) means P(x) is true for every x.

What about ¬∀x P(x) ?( )

It is not the case that [“P(x) is true for every x.”]

“There exists an x for which P(x) is not true.”There exists an x for which P(x) is not true.

∃x ¬P(x)

Universal negation:¬∀x P(x) ≡ ∃x ¬P(x).

14

Proofs• A theorem is a statement that can be

proved to be true.proved to be true.

• A proof is a sequence of statements that form an argument.

15

Proofs: Modus Ponens

•I have a total score over 96.I have a total score over 96.

•If I have a total score over 96, then I get an A for the class.

∴ I get an A for this class

p

p → qTautology:

p q

∴ q(p ∧ (p → q)) → q

16

Proofs: Modus Tollens

•If the power supply fails then the lights go out•If the power supply fails then the lights go out.

•The lights are on.

∴ The power supply has not failed.

Tautology:¬q

p → q(¬q ∧ (p → q)) → ¬p

p q

∴ ¬p

17

Proofs: Addition

•I am a student•I am a student.

∴ I am a student or I am a visitor.

p Tautology:p

∴ p ∨ qp → (p ∨ q)

18

Proofs: Simplification

•I am a student and I am a soccer player•I am a student and I am a soccer player.

∴ I am a student.

p ∧ q Tautology:p q

∴ p(p ∧ q) → p

19

Proofs: Conjunction

•I am a student•I am a student.•I am a soccer player.

I am a st dent and I am a socce pla e∴ I am a student and I am a soccer player.

p

q Tautology:q

∴ p ∧ q((p) ∧ (q)) → p ∧ q

20

Proofs: Disjunctive Syllogism

I am a student or I am a soccer player.I am a not soccer player.

I am a student∴ I am a student.

p ∨ q

¬q Tautology:q

∴ p((p ∨ q) ∧ ¬q) → p

21

Proofs: Hypothetical Syllogism

If I t t t l 96 I ill t A i thIf I get a total score over 96, I will get an A in the course.If I get an A in the course, I will have a 4.0 semester average.

∴ ∴ If I get a total score over 96 then∴ I will have a 4 0 semester average∴ I will have a 4.0 semester average.

p → q T l

p q

q → r

∴

Tautology:

((p → q) ∧ (q → r)) → (p → r)

22

∴ p → r

Proofs: Resolution

I t ki CS1301 I t ki CS2610I am taking CS1301 or I am taking CS2610.I am not taking CS1301 or I am taking CS 1302.

∴ I am taking CS2610 or I am taking CS 1302.

p ∨ q T l

p q

¬ p ∨ r

∴

Tautology:

((p ∨ q ) ∧ (¬ p ∨ r)) → (q ∨ r)

23

∴ q ∨ r

Proofs: Proof by Cases

I h t k CS2610 I h t k CS1301I have taken CS2610 or I have taken CS1301.If I have taken CS2610 then I can register for CS2720If I have taken CS1301 then I can register for CS2720

∴ I can register for CS2720

p ∨ q

p → r T lp → r

q → r

∴

Tautology:

((p ∨ q ) ∧ (p → r) ∧ (q → r)) → r

24

∴ r

Fallacy of Affirming the Conclusion

•If you have the flu then you’ll have a sore throat•If you have the flu then you ll have a sore throat.

•You have a sore throat.

∴ You must have the flu.

Fallacy:q

p → q(q ∧ (p → q)) → p

p q

∴ p

25Abductive reasoning

Fallacy of Denying the Hypothesis

•If you have the flu then you’ll have a sore throat•If you have the flu then you ll have a sore throat.

•You do not have the flu.

∴ You do not have a sore throat.

Fallacy:¬p

p → q(¬p ∧ (p → q)) → ¬q

p q

∴ ¬q

26

Inference Rules for Quantified Statements

∀x P(x)∴ P(c)

Universal Instantiation(for an arbitrary object c from UoD)∴ P(c)

P(c)___

(for an arbitrary object c from UoD)

Universal Generalization

∃x P(x)

∴ ∀x P(x) (for any arbitrary element c from UoD)

Existential Instantiation∃x P(x)∴ P(c)

Existential Instantiation(for some specific object c from UoD)

P(c)__∴ ∃x P(x)

Existential Generalization(for some object c from UoD)

27

( j )

Proof: Valid argumentAn argument is valid if whenever all the premises are truethen the conclusion is true.

p1,…,pn: premises or hypotheses of the problemq: conclusion

An argument is valid if

p1 ∧ p2 ∧ … ∧ pn → q

is true when p p are trueis true when p1,…,pn are true.

Wh t h if i i f l ?

28

What happens if a premise is false?

ProofsStep 1: Translate the sentences into

logical expressionslogical expressions

Step 2: Use rules of inferences to build aStep 2: Use rules of inferences to build a proof

29

Direct proofsStart with premises and deduce the conclusion:conclusion:

Assume that the premises are trueApply rules of inferences and theoremspp y

30

Vacuous Proofs

p → q is vacuously true if p is falsep q y p

In this case, p → q is a vacuous proof

Ex. p: 0 > 1q: Mars is an asteroidq: Mars is an asteroid

What can we say about p → q ?

31

Trivial Proofsp → q is trivially true if q is true,

In this case, we have a trivial proof

Example:p

x>1 → 1=1x>1 → 1=1

32

Indirect ProofsTo prove p → q, we prove its contrapositive,

¬ q → ¬ p

Example:if n2 is even then n is evenis equivalent tois equivalent to …if n is odd then n2 is odd

W “If 2 i h i ” bWe can prove “If n2 is even then n is even” by proving “If n is odd then n2 is odd”

33

Proof By Contradiction: Reductio ad Absurdum

To prove p, we assume ¬ p and derive a contradiction.

Based on the tautologyBased on the tautology

( ¬ p → F ) → p

“if the negation of p implies a contradiction then p must be true”

Example:Example:“If I win $1,000,000, I will buy a sailboat.”“If I buy a sailboat, I will go sailing every summer.”“This summer I will take one vacation“This summer, I will take one vacation. “I plan to go biking this summer.”

P th t I h t t $1 000 000

34

Prove that I have not yet won $1,000,000.

Overview of last classA predicate P, or propositional function, is a function

that maps objects in the universe of discourse to propositions

Predicates can be quantified using the universal quantifier (“for all”) ∀ or the existential quantifierquantifier ( for all ) ∀ or the existential quantifier (“there exists”) ∃Quantified predicates can be negated as followsQ p g

¬∀x P(x) ≡ ∃x ¬P(x)¬∃x P(x) ≡ ∀x ¬P(x)

Quantified variables are called “bound”Variables that are not quantified are called “free”

35

Proof Techniques-Quantifiers: For all Proofs

∀ x P(x) : provide a proof, not just examples.examples.

Ex. “The product of any two odd integersEx. The product of any two odd integers is odd”

Proof:

36

Proof Techniques

Disproving ∀ x P(x)

Find an counterexample for ∀ x P(x)

a value k in the Universe of Discourse such that ¬ P(k)

Example: For every n positive number, 22n2 + 1 is prime.

37Find a counterexample:

Proof Techniques-Quantifiers: Existence Proofs

Two ways of proving ∃x P(x).

Existence Constructive Proof:Existence Constructive Proof:

Find a k in the UoD such that P(k) holds.

Existence Non-Constructive ProofExistence Non-Constructive Proof

Prove that ∃x P(x) is true without finding a k in the UoD such that P(k) holdsthe UoD such that P(k) holds

38

Proof Techniques-Quantifiers: Existence Proofs

∃x P(x) :Existence Constructive Proof:

Find a k in the UoD such that P(k) holds.( )

Example:

There is a rational number that lies strictly between 19 100 - 1 and 19 100

Proof:

39

Existential Proof: Non-ConstructiveProve that ∀n∈N, ∃p such that p is prime, and p > n.

Proof: (BWOC)Proof: (BWOC)Assume the opposite is true.Then ∃n, ∀p such that p is prime, p ≤ n.L t b ll th i bLet p1, p2, …, pk be all the prime numbers

between 2 and n.Consider the value r = p1 × p2 × … × pn + 1.p1 p2 pn Then r is not divisible by any prime number p ≤ n. Thus, either r is prime or r has prime factors greater than n!n!

40

SetsA set is an unordered collection of objects.

Examples: Order and repetition don’t

{ 1, 6, 7, 2, 9 }

{ a, d, e, 1, 2, 3}= {6, 7, 1, 2, 9}= {a, a, d, d, e, e, 1, 2, 3}

matter

The empty set, or the set containing no elements

{a, a, d, d, e, e, 1, 2, 3}

elements.∅ = {} Note: ∅ ≠ {∅}

Si l t i t S th t t i tl l t

41

Singleton is a set S that contains exactly one element

Universal SetUniversal Set is the set containing all the objects under consideration.the objects under consideration.It is denoted by U

42

Set Builder NotationSet Builder – characterize the elements in a set by stating the properties that the elements must have to belong to the set.

{ x | P (x) }

reads x that satisfy P(x), x such that P(x)reads x that satisfy P(x), x such that P(x)x belongs to a universal set U.

concise definition of a setconcise definition of a set

Examples:

{ | b }P = { x | x is prime number} U : Z+

M = { x | x is a mammal} U: All animalsQ+ = { x ∈ R | x = p/q, for some positive integers p, q }

43

Elements of setsx ∈ S means “x is an element of set S”x ∉ S means “x is not an element of set S

Example:3 ∈ S reads:3 ∈ S reads:

“3 is an element of the set S ”.

Which of the following is true:1. 3 ∈ R2 3 ∈ N2. -3 ∈ N

44

SubsetsA ⊆ B means “A is a subset of B” or, “B contains A”

“every element of A is also in B”or, ∀x ((x ∈ A) → (x ∈ B))

A ⊆ B means “A is a subset of B”B ⊇ A means “B is a superset of A”B ⊇ A means B is a superset of A

45

SubsetsA ⊆ B means “A is a subset of B”

For Every Set S,

i) ∅ S th t t i b t fi) ∅ ⊆ S, the empty set is a subset of every set

ii) S ⊆ S, every set is a subset of itself

46

Power SetsThe power set of S is the set of all subsets of S.

P(S) = { x | x ⊆ S }( ) { | ⊆ }

If S = {a} P(S) = ? {∅ {a}}If S = {a}, P(S) = ?If S = {a,b}, P(S) = ?If S = ∅ P(S)= ?

{∅, {a}}{∅, {a}, {b}, {a, b}}{∅}If S = ∅, P(S)= ? {∅}

Fact: if S is finite, |P(S)| = 2|S|.

47

n-TuplesAn ordered n-tuple, n ∈ Z+, is an ordered list

(a1, a2, …, an).

Its first element is a1.Its second element is a2, etc.Enclosed between parentheses (list not set).

Order and length matters:(1, 2) ≠ (2, 1) ≠ (2, 1, 1).(1, 2) ≠ (2, 1) ≠ (2, 1, 1).

48

Cartesian ProductThe Cartesian Product of two sets A and B is:

A x B = { (a, b) | a ∈ A ∧ b ∈ B}

Example:A= {a, b}, B= {1, 2}A {a, b}, B {1, 2} A × B = {(a,1), (a,2), (b,1), (b,2)}B × A = {(1,a), (1,b), (2,a), (2,b)}{( , ), ( , ), ( , ), ( , )}

Not commutative!

In general, A1 x A2 x … x An = {(a1, a2,…, an) | a1 ∈ A1, a2 ∈ A2, …, an ∈ An}

49

…, an ∈ An} |A1 x A2 x … x An| = |A1| x |A2| x … x |An|

Union OperatorThe union of two sets A and B is:

A B { | A B }A ∪ B = { x | x ∈ A v x ∈ B }

Example:A = {1,2,3}, B = {1,6}A {1,2,3}, B {1,6}A ∪ B = {1,2,3,6}

50

Intersection OperatorThe intersection of two sets A and B is:

A ∩ B = { x | x ∈ A ∧ x ∈ B}

Example:A = {1,2,3}, B = {1,6}A ∩ B = {1}

Two sets A, B are called disjoint iff their intersection is , jempty.

A ∩ B = ∅Example:Example: A = {1,2,3}, B = {9,10}, C = {2, 9}

A and B are disjoint sets, but A and C are not

51

j ,

Set Theory : Inclusion/Exclusion

What is the cardinality of A ∪ B ?t itwice

A BA∩B

Once

| | | | | | | |52

|A∪B| = |A| + |B| - |A ∩ B|

Set Complement

The complement of a set A is:

A = { x | x ∉ A}

Example:

x ∈ A ↔ x ∉ A

Example:

U = NA = {x∈N | x is odd }

∅ = U U = ∅

53

{ | }A = {x∈N | x is even }

Set DifferenceThe set difference, A - B, is:

A - B = { x | x ∈ A ∧ x ∉ B }

Example:

A = {2,3,4,5 }, B = {3,4,7,9 }

A- B = {2 5} It i t t ti !!A B = {2, 5}

B – A = {7,9}It is not commutative!!

54

Symmetric DifferenceThe symmetric difference, A ⊕ B, is:

A ⊕ B = { x | (x ∈ A ∧ x ∉ B) v (x ∈ B ∧ x ∉ A)}

(i.e., x is in one or the other, but not in both)

I it t ti ?Is it commutative ?

55

Set IdentitiesIdentity:

A ∪ ∅ = A , A ∩ U = A

Domination: A ∪ U = U , A ∩ ∅ = ∅

Idempotent: A ∪ A = A = A ∩ A

D bl l tDouble complement:

Commutative:

AA =)(Commutative:

A ∪ B = B ∪ A , A ∩ B = B ∩ A

Associative:

56

A ∪ (B ∪ C) = (A ∪ B) ∪ C A ∩ (B ∩ C) = (A ∩ B) ∩ C

Set IdentitiesAbsorption:

A ∪ (A ∩ B) = A

A ∩ (A ∪ B) = A

Complement: p

A ∪ A¯ = U

A ∩ A¯ = ∅

Distributive:

A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C)A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C)A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C)

57

De Morgan’s RulesDe Morgan’s I

(A U B) = A ∩ B

DeMorgan’s II(A ∩ B) = A U B

58

Proving Set IdentitiesHow would we prove set identities of the form

S1 = S2

Whe e S and S a e sets?Where S1 and S2 are sets?

1. Prove S1 ⊆ S2 and S2 ⊆ S1 separately.

Use previously proven set identities.

Use logical equivalences to prove equivalent set definitions.Use logical equivalences to prove equivalent set definitions.

2. Use a membership table.

59

Functions (Section 2.3)Let A and B be nonempty sets.

A f ti f f A t B i i t f tlA function f from A to B is an assignment of exactly one element of B to each element of A. We write f(a) = b if b is the unique element of B assigned by the q g yfunction f to the element a in A. If f is a function from A to B, we write f : A → B.

Functions are sometimes called mappings.

60

Proof Using Logical EquivalencesProve thatProof: First show (A U B) ⊆ A ∩ B then the reverse

(A U B) = A ∩ BProof: First show (A U B) ⊆ A ∩ B, then the reverse.

Let c ∈ (A U B)c ∈ {x | x ∈ A ∨ x ∈ B} (Def. of union)¬ (c ∈ A ∨ c ∈ B) (Def. of complement)¬ (c ∈ A) ∧ ¬(c ∈ B) (De Morgan’s rule)(c ∉ A) ∧ (c ∉ B) (Def of ∉)(c ∉ A) ∧ (c ∉ B) (Def. of ∉)(c ∈ A) ∧ (c ∈ B) (Def. of complement)c ∈ {x | x ∈ A ∧ x ∈ B} (Set builder notation)c ∈ A ∩ B (Def. of intersection)

By U.G., (A U B) ⊆ A ∩ B. Each step above is reversible therefore A ∩ B ⊆ (A U B)

61

reversible, therefore A ∩ B ⊆ (A U B).

Functions (Section 2.3)

Let A and B be nonempty sets.

A function f from A to B is an assignment of exactly one element of B to each element of A. We write f( ) b if b is th uniqu l m nt f B ssi n d b f(a) = b if b is the unique element of B assigned by the function f to the element a in A. If f is a function from A to B, we write f : A → B.

Functions are sometimes called mappings.

62

Example

A = {Mike, Mario, Kim, Joe, Jill}B = {John Smith Edward Jones Richard Boone}B {John Smith, Edward Jones, Richard Boone}

Let f:A → B where f(a) means father of a.

Mike Mario

John Smith

Ed d J

f

Mario Kim

Joe

Edward Jones

Richard Boone

Jill

A B

63Can grandmother of a be a function ?A (children) B (fathers)

Functions as Ordered PairsA function f :A→B can be represented as a set of ordered pairs (recall, a relation)

{(a,b) | a ∈A ∧ b = f(a)} ⊆ A x B

For every a ∈A, there is exactly one pair (a, f(a)).

64

Function TerminologyGiven a function f:A→B

A is the domain of f. B is the codomain of f.

If f(a)=b then b is the image of a under f.

a is the pre-image of b under f.In general, b may have more than 1 pre-image.

The range R of f (or image of f) is :R = {b | ∃a f(a)=b }. The set of all images of a’s.

For any set S ⊆ A, the image of S,f(S) = { b ∈ B | ∃a ∈ S, f(a) = b}

F s t T B th i im f T

65

For any set T ⊆ B, the inverse image of Tf−1(T) = { a ∈ A | f(a) ∈ T }

Example

Mike Mario

John SmithfMario

Kim

Joe Jill

Edward Jones

Richard Boone

Jill

A BDomain CodomainDomain Codomain

The image of Mike under f is John SmithMike is a pre-image of John Smith under fMike is a pre image of John Smith under f

R (f) = {John Smith, Richard Boone}f(Mike Mario Jill) = {John Smith Richard Boone}

66

f(Mike,Mario,Jill) {John Smith, Richard Boone}

f-1(Richard Boone) = {Joe, Jill}

Injective Functions (one-to-one)

A function f: A → B is one-to-one (injective, an injection) iff f(x) f(y) → x y for all x and y in injection) iff f(x) = f(y) → x = y for all x and y in the domain of f (∀x∀y(f(x) = f(y) → x = y))

Equivalently: ∀x∀y(x ≠ y → f(x) ≠ f(y))Equivalently: ∀x∀y(x ≠ y → f(x) ≠ f(y))

fA B

67Every b ∈ B has at most 1 pre-image

Surjective Functions (onto)

A function f: A → B is onto (surjective, an surjection)surjection)

iff ∀y∃x( f(x) = y) where y ∈ B, x ∈ A

ffA B

68Every b ∈ B has at least one pre-image

Bijective Functions

A function f: A → B is bijective iff it is one-to-one and onto (a one-to-one correspondence)and onto (a one to one correspondence)

f

The domain cardinality equals the codomain cardinalityA B

69

The domain cardinality equals the codomain cardinality

Function Composition

Given the functions g:A→B and f:B→C, the composition of f and g f ○g: A→C defined ascomposition of f and g, f ○g: A→C defined as

f ○g (a) = f ( g (a) )f ○g (h) ?

h 2 ℡

fgf ○g (h) ?

h

b3

5

℡

d

o

5

1

70

o 7

A B C

Function Composition

Properties

Associative: Given the functions g:A→B and f:B→Cand h:C→D then

h ○ (f ○g) ≡ (h ○ f ) ○ gh(f( ( ))) h(f( )) h(f( ( )))h(f(g(x))) ≡ h(f(x)) ○ g = h(f(g(x)))

but (f ○g) ≠ (g ○ f ) not Commutativebut (f ○g) ≠ (g ○ f ) not Commutative

71

Inverse Functions

Let f : A → B be a bijection, the inverse of f,

f -1:B → A

such that for any b ∈ B,

f -1(b) = a when f (a) = b

A Bf

72f-1

Inverse Functions

Let f: A → B be a bijection, and f-1:B → A be the inverse of f:inverse of f:

f-1 ○ f = IA = (f-1○f)(a) = f-1 (f(a)) = f-1 (b) = a A ( )( ) ( ( )) ( )f ○ f-1 = IB = (f○f-1)(b) = f(f-1 (b)) = f(a) = b

A Bf

73f-1

Floor and Ceiling Function

Definition: The floor function ⎣.⎦:R→Z, ⎣x⎦ is the largest integer which is less than or equal to xlargest integer which is less than or equal to x.

⎣x⎦ reads the floor of x

Definition: The ceiling function ⎡.⎤ :R→Z, ⎡x⎤ is the ll t i t hi h i t th l t smallest integer which is greater than or equal to

x.

⎡x⎤ reads the ceiling of x

74

Ceiling and Floor PropertiesLet n be an integer(1a) ⎣x⎦ = n if and only if n ≤ x < n+1(1a) ⎣x⎦ = n if and only if n ≤ x < n+1(1b) ⎡x⎤ = n if and only if n-1 < x ≤ n(1c) ⎣x⎦ = n if and only if x-1 < n ≤ x(1c) ⎣x⎦ = n if and only if x-1 < n ≤ x(1d) ⎡x⎤ = n if and only if x ≤ n < x+1(2) x-1 < ⎣x⎦ ≤ x ≤ ⎡x⎤ < x+1 (2) x 1 < ⎣x⎦ ≤ x ≤ ⎡x⎤ < x+1 (3a) ⎣-x⎦ = - ⎡x⎤

(3b) ⎡-x⎤ = - ⎣x⎦(3b) ⎡ x⎤ ⎣x⎦

(4a) ⎣x+n⎦ = ⎣x⎦+n (4b) ⎡x+n⎤ = ⎡x⎤+n

75

(4b) ⎡x n⎤ ⎡x⎤ n

Boolean Algebras (Chapter 11)

Boolean algebra provides the operations and the rules for working with the set {0 1}the rules for working with the set {0, 1}.

These are the rules that underlie electronic and optical circuits, and the methods we

ill di f d t l t VLSI d iwill discuss are fundamental to VLSI design.

76

Boolean Algebra

The minimal Boolean algebra is the algebra formed over the set of truth values {0 1} by using the over the set of truth values {0, 1} by using the operations functions +, ·, - (sum, product, and complement).

The minimal Boolean algebra is equivalent to The minimal Boolean algebra is equivalent to propositional logic where

O corresponds to Falsep1 corresponds to True• corresponds logical operator AND

s ds l i l t OR

77

+ corresponds logical operator OR- corresponds logical operator NOT

Equal Boolean Functions

Two Boolean functions F and G of degree n are equal iff for all (x1 x ) ∈ Bn F (x1 x ) = G (x1 x )equal iff for all (x1,..xn) ∈ B , F (x1,..xn) = G (x1,..xn)

Example: F(x,y,z) = x(y+z), G(x,y,z) = xy + zx

78

Boolean ExpressionsLet x1, …, xn be n different Boolean variables.

A Boolean expression is a string of one of the following f ( i d fi i i )forms (recursive definition):

0, 1, x1, …, or xn. are Boolean ExpressionsIf E1 and E2 are Boolean expressions then -E1, p(E1E2), or (E1+E2) are Boolean expressions.

Example:E1 = xE1 xE2 = yE3 = zE4 = E1 + E2= x + yE E E E5 = E1E2= x yE6 = -E3 = -zE7 = E6 + E4 = -z + x + yE8 = E6 E4 = -z ( x + y)

79Note: equivalent notation: -E = E for complement

Functions and ExpressionsA Boolean expression represents a Boolean function.Furthermore, every Boolean function (of a given degree)

can be represented by a Boolean expression with n can be represented by a Boolean expression with n variables.

F(x1,x2,x3)1000

x3x2x1

F(x1,x2,x3) = x1(x2+x3)+x1x2x3

1110

1010

0100

11100001

0101

80

0011

1111

Boolean Functions

Two Boolean expressions e1 and e2 that represent the exact same function F are called equivalentthe exact same function F are called equivalent

F(x1,x2,x3)x3x2x1

0100

1000

F(x1,x2,x3) = x1(x2+x3)+x1x2x3

1110

1010

0001

F(x1,x2,x3) = x1x2+x1x3+x1x2x3

0

0101

0011

81

1111

Boolean Identities

Double complement:x = x

Associative laws:x + (y + z) = (x + y) + zx x

Idempotent laws:x + x = x, x · x = x

Identity laws:

x (y z) (x y) zx · (y · z) = (x · y) · z

Distributive laws:x + y ·z = (x + y)·(x + z)Identity laws:

x + 0 = x, x · 1 = xDomination laws:x + 1 = 1 x · 0 = 0

x y z (x y) (x z)x · (y + z) = x ·y + x ·z

De Morgan’s laws:(x · y) = x + y (x + y) = x · yx 1 1, x 0 0

Commutative laws:x + y = y + x, x · y = y · x

(x y) x y, (x y) x yAbsorption laws:x + x ·y = x, x · (x + y) = x

82

the Unit Property: x + x = 1 and Zero Property: x ·x = 0

DNF: Disjunctive Normal Form

A literal is a Boolean variable or its complement.A minterm of Boolean variables x x is a Boolean A minterm of Boolean variables x1,…,xn is a Boolean product of n literals y1…yn, where yi is either the literal xi or its complement xi.

iExample:

minterms

x y z + x y z + x y z

Disjunctive Normal Form: sum of products

We have seen how to develop a DNF expression for a

83

p pfunction if we’re given the function’s “truth” table.

CNF: Conjunctive Normal Form

A literal is a Boolean variable or its complement.A maxterm of Boolean variables x x is a A maxterm of Boolean variables x1,…,xn is a Boolean sum of n literals y1…yn, where yi is either the literal xi or its complement xi.

Example:maxterms

(x +y + z) • (x + y + z) • (x + y +z)

84Conjuctive Normal Form: product of sums

Logic Gates: the basic elements of circuits

Electronic circuits consist of so-called gates connected by wires

xx x Inverter (NOT gate)

OR gateXX+Y

x

Y

AND gatex

y

xy

85

y

Multiway Logical Gates

Multiple Input AND, OR Gates

x1 x1+x2 +… + xnx2

xn

x1 x1•x2 •… • xnx2

xxn

86

Three Variable Karnaugh Maps

With the three variables x, y, z, we can let x and x be on the vertical side as beforebe on the vertical side as beforeThe table will now have 4 columns: yz, yz, yz, and yz

Order is important! Columns must be adjacent to each other

We also consider the first and last columns to be We also consider the first and last columns to be adjacent

Picture the table as a flattened cylinderA block of 2 cells cancels out 1 variableA block of 4 cells cancels out 2 variablesWh t if h bl k f 8 ll ?

87

What if we have a block of 8 cells?

3-Variable Example

xyz + xyz + xyz + xyz + xyz = z + xy

yz yz yz yz

1 1 1x

yz yz yz yz

1 1x

88

Analysis of AlgorithmsAnalyzing an algorithm

Time complexityp ySpace complexity

Time complexityTime complexityRunning time needed by an algorithm as a function of the size of the inputDenoted as T(N)Denoted as T(N)

W i t t d i i h f t th ti We are interested in measuring how fast the time complexity increases as the input size grows

Asymptotic Time Complexity of an Algorithm

89

y p p y g

Algorithm Complexity

Worst Case AnalysisL t b f ti t l bl f Largest number of operations to solve a problem of a specified size.Analyze the worst input case for each input size.y p pUpper bound of the running time for any input.

Most widely used.y

Average Case AnalysisAverage Case AnalysisAverage number of operations over all inputs of a given size

Sometimes it’s too complicated

90

Sometimes it s too complicated

Search Algorithms

Search Algorithm Problem:Find an element a in a list a a (not necessarily Find an element a in a list a1,…an (not necessarily ordered)

Linear Search Strategy: Examine the sequence one element after another until all the elements have been examined or the current element being examined is the element acurrent element being examined is the element a.

91

Sorting Algorithms

Problem: Given a sequence of numbers, sort the sequence in weakly increasing ordersequence in weakly increasing order.

Sorting Algorithms:g g

Input: A sequence of n numbers a1, a2, …, an

O t t Output: A re-ordering of the input sequence (a’1, a’2, …, a’n) such that a’1 ≤ a’2 ≤ … ≤ a’n

92

such that a 1 ≤ a 2 ≤ … ≤ a n

Sequences (Section 2.4)Def. :A sequence is a function from a subset of integers I to a

set S, (I ⊆ Z)f:I→S

Usually, the domain I is either a set of positive or non-negative consecutive integers {1,2,3…} or {0,1,2,3…}.

We will usually be using as the domain of I the sequence:We will usually be using as the domain of I the sequence:

I = {i ∈ Z | i > 0}

Notation:Notation:Let i ∈ I, the image f(i) is denoted as ai, where ai ∈ Sai is called a term of the sequence{a } represents the entire sequence{ai} represents the entire sequence

Note:If the domain I is finite the sequence is finite otherwise

93

If the domain I is finite, the sequence is finite, otherwise the sequence is infinite.

SequencesExamples:

Let the sequence {a } be defined asLet the sequence {ai} be defined as

ai = i + 3:Terms: a1, a2, a3, …1 2 3Sequence {ai}: { 4, 5, 6, 7, 8….}

ai = i2:ai iTerms: a1, a2, a3, …Sequence {ai}: { 1, 4, 9, 16, 25….}

ai = 1/i:Terms: a1, a2, a3, …Sequence {ai}: { 1 1/2 1/3 1/4 1/5….}

94

Sequence {ai}: { 1, 1/2, 1/3, 1/4, 1/5….}

SequencesDef.:An arithmetic progression is a sequence of the form

d 2d 3d a, a + d, a + 2d, a + 3d,… where a ∈ R is the initial term, and d ∈ R is the common difference,common d fference,

Observe that if I = {i where i >= 0 }, ai

= a + i*di

ai+1 = ai + d

Example:ExampleLet d = 3, {an} such that a=2, d=3{an} = {2, 5, 8, 11, 14,…}

95

SequencesD f A t i i i f th Def.: A geometric progression is a sequence of the form

a, ar, ar2, ar3,… a, ar, ar , ar ,… where a ∈ R is the initial term, and r ∈ R is the common ratio.

Observe that if I = {i | i >=0 }, ai

= ari

ai+1 = air, where a is the first termIt grows exponentially

96

Some Useful Sequences

n2 = 1 4 9 16 25 36n2 = 1, 4, 9, 16, 25, 36, …

n3 = 1, 8, 27, 64, 125, 216, …

n4 = 1, 16, 81, 256, 625, 1296, …

2n = 2, 4, 8, 16, 32, 64, …

3n = 3, 9, 27, 81, 243, 729, …

n! = 1, 2, 6, 24, 120, 720, …

97

SummationsLet {ai} be a sequence. We can create the following

summation of this sequencek

kjj

k

jii a...aa:a +++≡ +

=∑ 1

h i is called the index of summationi is called the index of summation

hh j ∈ Z+ is the lower bound (or limit)

hh k ∈ Z+, k ≥ j is the upper bound

(Also have ∏ for product.)

98

(Also have ∏ for product.)

SummationsExample

52∑i

3∑

=i

∑ +5

)1(k∑=

+1

)1(k

k

∑4

j∑=

−0

)2(j

j

( )∑=

+ −4

0

1 22j

jj

99

0j

CardinalityD f Th di lit f t i th b f Def.: The cardinality of a set is the number of

elements in the set.

Def.: Let A and B be two sets.A and B have the same cardinality iff there is a one-

to-one correspondence (bijection) between A and B

100

Countable Sets and Uncountable SetsD f S t A i t bl if it i fi it if it h th Def.: Set A is countable if it is finite or if it has the same cardinality as the set of positive integers. Otherwise it is uncountable.

ℵ ( l h) d h di li f i fi i

Examples:

0ℵ (aleph) denotes the cardinality of infinite countable sets

Examples:

Infinite Countable Sets: N, Z+, Z-, Z

Infinite Uncountable Sets: R, R+, R-

101

Countable Sets and Uncountable SetsH d d t t th t t i t bl ?How do you demonstrate that a set is countable ?

Suppose A is a set If there is a one-to-one and onto Suppose A is a set. If there is a one-to-one and onto function f : A Z+, then A is countable. Recall,

one-to-one means ∀x∀y(f(x) = f(y) → x = y)onto means ∀y∃x( f(x) = y)

102

Uncountable setsTh Th t f l b i t blTheorem: The set of real numbers is uncountable.

If a subset of a set is uncountable, then the set is uncountable.The cardinality of a subset is at least as large as the cardinality of the entire set.

It is enough to prove that there is a subset of R that is uncountable

Theorem: The open interval of real numbers[0,1) = {r ∈ R | 0 ≤ r < 1} is uncountable.

Proof by contradiction using the Cantor diagonalization argument (Cantor, 1879)

103

Uncountable Sets: RP f (BWOC) usin di n li ti n: Supp s R is Proof (BWOC) using diagonalization: Suppose R is countable (then any subset say [0,1) is also countable). So, we can list them: r1, r2, r3, … where

r1 = 0.d11d12d13d14… the dij are digits 0-9r2 = 0.d21d22d23d24…r3 = 0 d31d32d33d34r3 = 0.d31d32d33d34…r4 = 0.d41d42d43d44…etc.

Now let r = 0.d1d2d3d4… where di = 4 if dii ≠ 4di = 5 if dii = 4

But r is not equal to any of the items in the list so it’s But r is not equal to any of the items in the list so it s missing from the list so we can’t list them after all.r differs from ri in the ith position, for all i. So, our

ti th t ld li t th ll i i t

104

assumption that we could list them all is incorrect.

Order of Growth Terminology

O(1) C t tBest

O(1) ConstantO(log cn) Logarithmic (c ∈ Z+)O(logc n) Polylogarithmic (c ∈ Z+)O(n) LinearO(nc) Polynomial (c ∈ Z+)O(cn) Exponential (c ∈ Z+)O(n!) Factorial

Worst

105

Complexity of Problems

TractableA problem that can be solved with a deterministic A problem that can be solved with a deterministic polynomial (or better) worst-case time complexity.Also denoted as PExample:Example:

Search ProblemSorting problemFi d th m im mFind the maximum

106

Complexity of Problems

IntractableP bl th t t t t blProblems that are not tractable.Example:

l l lTraveling salesperson problem

Wide use of greedy algorithms to get an approximate solution.

F l d For example under certain circumstances you can get an approximation that is at most double the optimal solution.

107

most double the optimal solution.

Big-O Notation

Big-O notation is used to express the time complexity of an algorithmcomplexity of an algorithm

W h i i h We can assume that any operation requires the same amount of time.

The time complexity of an al orithm can be The time complexity of an algorithm can be described independently of the software and hardware used to implement the algorithm.

108

Big-O NotationDef.: Let f , g be functions with domain R≥0 or N and

codomain R. m

f(x) is O(g(x)) if there are constants C and k st

∀ k |f ( )| C | ( )|∀ x > k, |f (x )| ≤ C ⋅ |g (x )|f (x ) is asymptotically dominated by g (x )C|g(x)| is an upper bound of f(x)C|g(x)| is an upper bound of f(x).

C and k are called witnesses to

C|g(x)|

the relationship between f & g. |f(x)|

109k

Big-O PropertiesTransitivity:if f is O(g) and g is O(h) then f is O(h)

Sum Rule:If f1 is O(g1) and f2 is O(g2) then f1+f2 is O(max(|g1|,|g2|))

If f1 is O(g) and f2 is O(g) then f1+f2 is O(g)

Product RuleIf f1 is O(g1) andf2 is O(g2) then f1f2 is O(g1g2)

For all c > 0, O(cf), O(f + c),O(f − c) are O(f)

110

Big-Omega NotationDef.: Let f, g be functions with domain R≥0 or N and codomain R.

f(x) is Ω(g(x)) if there are positive constants C and k such that

∀x > k, C ⋅ |g (x )| ≤ |f (x )|

C ⋅ |g(x)| is a lower bound for |f(x)|

C|g(x)||f(x)|

C|g(x)|

111k

Big-Theta Notation

Def.:Let f , g be functions with domain R≥0 or N and codomain Rcodomain R.

f(x) is Θ(g(x)) if f(x) is O(g(x)) and f(x) is Ω(g(x)).

C2|g(x)|

|f(x)|

C1|g(x)|

|f(x)|

112

Big Summary

Upper Bound – Use Big-Oh

Lower Bound – Use Big-OmegaLower Bound – Use Big-Omega

Upper and Lower (or Order of Growth) –

Us Bi Th tUse Big-Theta

113

Number TheoryElementary number theory, concerned with numbers, usually integers and their properties or rational numbers numbers

mainly divisibility among integers Modular arithmetic

Some ApplicationsCryptographyCryptography

E-commercePayment systems…

Random number generationCoding theory

114

Hash functions (as opposed to stew functions ☺)

Number Theory - DivisionLet a, b and c be integers, st a≠0, we say that“a divides b” or a|b if there is an integer c where a divides b or a|b if there is an integer c where

b = a·c .a and c are said to divide b (or are factors)a and c are said to divide b (or are factors)

a | b ∧ c | b b is a multiple of both a and cb is a multiple of both a and c

Example:E mp5 | 30 and 5 | 55 but 5 | 27

115

Number Theory - DivisionTheorem 3.4.1: for all a, b, c ∈ Z:

1. a|0|2. (a|b ∧ a|c) → a | (b + c)3. a|b → a|bc for all integers c4. (a|b ∧ b|c) → a|c

P f (2) |b b d | Proof: (2) a|b means b = ap, and a|c means c = aqb + c = ap + aq = a(p + q)therefore a|(b + c) or (b + c) = ar where r = p+qtherefore, a|(b + c), or (b + c) = ar where r = p+q

Proof: (4) a|b means b = ap, and b|c means c = bqc = bq = apq

116

c q apqtherefore, a|c or c = ar where r = pq

The Division Algorithm

Division Algorithm Theorem: Let a be an integer, and d be a positive integer There are unique integers d be a positive integer. There are unique integers q, r with r ∈ {0,1,2,…,d-1} (ie, 0 ≤ r < d) satisfying

a = dq + ra dq r

d is the divisorq is the quotientq is the quotient

q = a div d

r is the remainderr = a mod d

117

Mod OperationLet a, b ∈ Z with b > 1.

a = q·b + r, where 0 ≤ r < b

Th d b d h i d f h Then a mod b denotes the remainder r from the division “algorithm” with dividend a and divisor b

109 mod 30 = ?

0 ≤ a mod b ≤ b – 1

118

Modular Arithmetic

Let a, b ∈ Z, m ∈ Z+

Then a is congruent to b modulo m iff m | (a −b) .

Notation: “a ≡ b (mod m)” reads a is congruent to b modulo m “a ≡ b (mod m)” reads a is not congruent to b modulo m.

Examples:Examples:5 ≡ 25 (mod 10)5 ≡ 25 (mod 3)

119

Modular ArithmeticTheorem 3.4.3: Let a, b ∈ Z, m ∈ Z+. Then

a ≡ b (mod m) iff a mod m = b mod mProof: (1) given a mod m = b mod m we have

a = ms + r or r = a – ms,b b b = mp + r or r = b – mp,a – ms = b – mpwhich means a – b = ms – mpwhich means a b = ms mp

= m(s – p)so m | (a – b) which means

a ≡ b (mod m)

120

Modular ArithmeticTheorem 3.4.3: Let a, b ∈ Z, m ∈ Z+. Then

a ≡ b (mod m) iff a mod m = b mod mProof: (2) given a ≡ b (mod m) we have m | (a – b)

let a = mqa + ra and b = mqb + rb

s |(( ) ( ))so, m|((mqa + ra) – (mqb + rb))or m|m(qa – qb) + (ra – rb)

recall 0 ≤ r < m and 0 ≤ rb < mrecall 0 ≤ ra < m and 0 ≤ rb < mtherefore (ra – rb) must be 0that is, the two remainders are the same,which is the same as saying

a mod m = b mod m

121

Modular Arithmetic

Theorem 3.4.4: Let a, b ∈ Z, m ∈ Z+. Then:a ≡ b (mod m) iff there exists a k ∈ Z sta ≡ b (mod m) iff there exists a k ∈ Z st

a = b + km.Proof: a = b + km meansProof: a = b + km means

a – b = km which meansm | (a – b) which is the same as sayingm | (a b) which is the same as sayinga ≡ b (mod m)(to complete the proof, reverse the steps)p p p

Examples:27 ≡ 12 (mod 5) 27 = 12 + 5k k = 3

122

105 ≡ -45 (mod 10) 105 = -45 + 10k k = 15

Modular ArithmeticTheorem 3.4.5: Let a, b, c, d ∈ Z, m ∈ Z+. Then if

a ≡ b (mod m) and c ≡ d (mod m), then:1. a + c ≡ b + d (mod m),2. a - c ≡ b - d (mod m),3 bd ( d )3. ac ≡ bd (mod m)

Proof: a = b + k1m and c = d + k2mProof: a = b + k1m and c = d + k2ma + c = b + d + k1m + k2m

or a + c = b + d + m(k1 + k2)( 1 2)which is

a + c ≡ b + d (mod m)

123

others are similar

Number Theory - PrimesA positive integer n > 1 is called prime if it is only

divisible by 1 and itself (i.e., only has 1 and itself y ( , yas its positive factors).

Example: 2 3 5 7 11 13 17 19 23 29 31 97Example: 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 97

A number n ≥ 2 which isn’t prime is called composite.p p

Example:All b 2 itAll even numbers > 2 are composite.

By convention, 1 is neither prime or composite.

124

By convent on, s ne ther pr me or compos te.

Number Theory - Primes

Fundamental Theorem of Arithmetic

Every positive integer greater than 1 has a uniquerepresentation as the product of a non-decreasing

f series of one or more primes

Examples:Examples:2 = 2 4 = 2·2 100 2 2 5 5100 = 2·2·5·5200 = 2·2·2·5·5999= 3·3·3·37

125

Number Theory – Prime Numbers

Theorem 3.5.3: There are infinitely many primes.

We proved earlier in the semester that for any integer x, there exists a prime number p such that g p pp > x.

L t ( ) | { | d i i } |Let Π(n) = | {p | p ≤ n and p is prime} |

126

Greatest Common Divisor

Let a,b be integers, a≠0, b≠0, not both zero. The greatest common divisor of a and b is the The greatest common divisor of a and b is the

biggest number d which divides both a and b.

Example: gcd(42,72)

Positive divisors of 42: 2,3,6,7,14,21,Positive divisors of 72: 2,3,4,6,8,9,12,24,36

gcd(42,72)=6

127

Least Common MultipleThe least common multiple of the positive integers aand b is the smallest positive integer that is divisibleby both a and b.

)lcm( ),max(),max(),max( 2211 nn bababa pppba = .),lcm( 21 npppba …=

Example: lcm(233572, 2433) = 243572

128

Modular ExponentiationFor large b, n and m, we can compute the modular

exponentiation using the following property:

a·b mod m = (a mod m) (b mod m) mod m

Therefore, bn (mod m) = (b mod m)n (mod m)

In fact, we can take (mod m) after each multiplication to keep all values lowkeep all values low.

129

AIC1

Slide 129

AIC1 Note: if a equiv b (mod m) and c equiv d (mod m) then ac equiv bd (mod m)

also if a equiv b (mod m) then a mod m = b mod m

cool!AI Center, 10/17/2006

Proving Properties of Infinite SetsGiven a predicate P(n), UD(n)={n > k, n ∈N }

To prove the proposition ∀n P(n)

We need to proof that the statement is true for all n > ktrue for all n > kIt is not enough to give some few examples:

Example:Claim: P(n): n2 + n + 41 is a prime number41, 43, 47, 53, 61, 71, 83, 97, 113, 131 are all prime

Have we proved that P(n) is true for all n > 0?

130

No Actually: P(41) = 1763 = 41*43 is not prime

Weak Mathematical Inductionl f k h l dPrinciple of Weak Mathematical Induction

1) [Base Case] P (m) is true for some m ∈ NUsually (but not always) the base case is proved for m = 0 or 1y y p

2) [Inductive Step] Inductive Hypothesis: Assume that P(n) is true, for an arbitrary n such that n ≥ mar trary n uch that n ≥ mProve

P(n) P(n+1)3)3) Then:

∀n ≥ m P(n) is true

Idea: If it’s true for n=1, then it’s true for n=2. If it’s true for n=2, then it’s true for n=3. If it’s true for n=3, then it’s true for n = 4 …

131[P (m) ∧ ∀ n ≥ m (P (n) → P(n + 1))] → ∀ n≥m P (n)

Strong InductionI f b th ti l i d ti th i d ti t In a proof by mathematical induction, the inductive step shows that if the inductive hypothesis P(k) is true, then P(k+1) is also true. In a proof by strong induction, the ( ) p y ginductive step shows that if P(j) is true for all positive integers not exceeding k, then P(k+1) is true.

For the inductive hypothesis we assume that P(j) is true for j = 1, 2, 3, …, k.j , , , ,

Yes, they are equivalent. But now we getto use P(1), P(2), … P(k) to prove P(k+1)not just P(k)!

132

Strong InductionPrinciple of Strong Induction1) [Base Case] show P (1) is true

2) [Inductive Step] assume P(j) for j = 1,2,…,kInductive Hypothesis: ProveInductive Hypothesis: Prove

P(1) ∧ P(2) ∧ … ∧ P(k) P(k+1)

133

Recursively Defined SequenceIn cu si l d fin d s qu nc :In a recursively defined sequence:1. Base or Initial Conditions

The first term(s) of the sequence are definedThe first term(s) of the sequence are defined2. Recursion or Recursive Step

The nth term is defined in terms of previous t sterms

The formula to express the nth term is called a recurrenceformula

Arithmetic Series:Base: a0=1, r=3R i 0

Recurrence Formula

Recursion: an=an-1+r, n > 0Geometric Series

Base: a0=3 r=2

134

Base: a0=3, r=2Recursion: an=an-1r, n > 0

Recursively Defined FunctionA function f(n) with domain N or a subset of N is defined

recursively, when f(n) is defined in terms of the previous functions of m < n

Basis: f(0) = 1Recursion:

Define f(n) from f defined on smaller termsDefine f(n) from f defined on smaller terms

ExampleLet f : N -> N defined recursively asyBasis: f(0) = 1Recursion: f(n + 1) = (n + 1) · f(n).

Wh t th lu s f th f ll in ?What are the values of the following?f(1)= 1 f(2)= 2 f(3)= 6 f(4)= 24

What does this function compute? n!

135

What does this function compute? n!

Recursively Defined SetA i fi it t S b d fi d i l b An infinite set S may be defined recursively, by giving:

Basis Step: A finite set of base elementsp f f mRecursive Step: a rule for forming new elements in the set from those already in the setEx l si R l : s ifi s th t th s t l t i s Exclusion Rule: specifies that the set only contains those elements specified in the basis step or those generated by the recursive step

Example:Let S be defined as follows

Basis Step: 1 ∈ SRecursive Step: if n ∈ S then 2n ∈ S

136

S = {2k | k ∈ N }

Set of StringsDef.:An alphabet Σ is a finite non-empty set of

symbols (e.g., Σ = {0, 1} )

Def.:A String over an alphabet Σ is a finite sequence of symbols from Σ (e.g., 11010 )y g

The set Σ* of strings over Σ can be defined as:

Basis Step: λ ∈ Σ* where λ is the empty string containing no symbolsno symbols

Recursive Step: if w ∈ Σ* and x ∈ Σ then wx ∈ Σ*

137

Is Σ* countable or uncountable ?

Recursive Definition on StringsConcatenation (combining two strings)

Basis Step: if w ∈ Σ* then w·λ = w, where λ is the p ,empty string containing no symbols.

R cu siv St p: if Σ* Σ* nd x Σ th nRecursive Step: if w1 ∈ Σ , w2 ∈ Σ and x ∈ Σ thenw1·(w2 x) ∈ Σ* (same as (w1 · w2) x ∈

Σ*)

Example:Σ={a, b}{a, }Let w1=aba, w2=a and x=b then abaab ∈ Σ*

138

Counting (now in chapter 5)The basic counting principles are the product rule andThe basic counting principles are the product rule and

sum rule.

Product Rule: Suppose that a procedure can be broken down into a sequence of two tasks. If there are nways to do the first task and for each of these ways ways to do the first task and for each of these ways of doing the first task, there are m ways to do the second task, then there are n·m ways to do the procedure.

Sum Rule: If a task can be done either in one of n ways Sum Rule: If a task can be done either in one of n ways or in one of m ways, where none of the set of n ways is the same as any of the set of m ways, then there are

139

n + m ways to do the task.

CountingThe Pigeonhole Principle: If k is a positive integer and The Pigeonhole Principle: If k is a positive integer and

k+1 or more objects are placed in k boxes, then there is at least one box containing two or more of the

( )objects. (prove BWOC)

Of 367 people at least two have the same birth dayOf 367 people, at least two have the same birth day.

For every integer n there is a multiple of n that has only For every integer n there is a multiple of n that has only 0s and 1s in its decimal expansion.

140

Counting

Part of combinatorics, the study of arrangements of objects (Sets sequences sebsets etc )of objects. (Sets, sequences, sebsets, etc.)

Counting relies on two important, but simple g p pprinciples: the Product Rule and Sum Rule

141

Counting

Note that sometimes we will not be able to make our subtasks completely distinct Some ways of our subtasks completely distinct. Some ways of solving a problem might fall into multiple subtasks.This leads to the Subtraction Principle.

Before introducing this principle, let’s consider the set versions of the Product and Sum Rulesthe set versions of the Product and Sum Rules.

If A and B are sets, then |A × B| = |A| ⋅ |B|If A and B are disjoint sets, j ,then |A ∪ B| = |A| + |B|

142

The Pigeonhole Principle

For k∈Z+, if k+1 or more objects are placed into k slots there is at least one slot containing two or slots, there is at least one slot containing two or more objects.

Generalized!!!!If N objects are placed into k slots, then there is at least one slot containing at least ⎡N/k⎤is at least one slot containing at least ⎡N/k⎤objects.

143

Permutations and Combinations

A permutation of a set of distinct objects is an ordered arrangement (list) of these objects.g ( ) jAn r-permutation of a set of distinct objects is an ordered arrangement of a subset of size r.

The number of r-permutations of a set with nelements is given by the product rule

P(n,r) = n ⋅ (n-1) ⋅ … ⋅ (n-r+1), orP(n,r) = n! / (n-r)!, for 0 ≤ r ≤ n

Example: How many ways to award medals in a race with 8 people?

144

Permutations and Combinations

An r-combination of a set of distinct objects is an unordered arrangement (subset) of size r.g ( )

The number of r-combinations of a set with nelements is given by elements is given by

C(n,r) = n! / [r! (n-r)!], for 0 ≤ r ≤ n

The binomial coefficient symbolism is also used. (More on that later!)Examples: Examples:

How many 5 card poker hands are there?How many bitstrings of length six contain exactly th 0’ ?

145

three 0’s?

ProbabilityWe can understand probability by considering sets

of outcomes:

We define a set S to be a sample space, a set of all ibl t f i tpossible outcomes of some experiment.

We define a set E ⊆ S, the set of all outcomes in which the event occurs.which the event occurs.

We further assume that all outcomes in S are equally likely.

Then the probability of the event occurring is:(E) |E| / |S|

146

p(E) = |E| / |S|

Probability

We use p(E) to denote the probability that an event occursevent occurs.

We use p(E) to denote the probability that an p( ) p yevent does not occur.

P(E) 1 (E)P(E) = 1 – p(E)

If a coin is flipped 5 times what is the probability of If a coin is flipped 5 times, what is the probability of at least one head coming up?

147

Probability

If E1 and E2 are two events in the same sample space thenspace, then

p(E1 ∪ E2) = p(E1) + p(E2) – p(E1 ∩ E2)

It’s just the subtraction principle again!

A number is selected at random from the set of positive integers less than or equal to 100.

What is the probability the number is divisible by What is the probability the number is divisible by either 2 or 5?

148

Probability Theory

When dealing with experiments for which there are multiple outcomes- x1 x2 x –we requireare multiple outcomes x1, x2, …, xn we require

0 ≤ p(xi) ≤ 1 for i = 1, 2, …, n and∑(i=1, n) p(xi) = 1

We can treat p as a function that maps elements from the sample space to real values in the range from the sample space to real values in the range [0,1]. We call such a function a probability distribution.

149

Probability TheoryUniform Probability Distribution:

p(xi) = 1/n, for i = 1, 2, …, np(xi) 1/n, for i 1, 2, …, n

All outcomes are equally probable.

150

Probability Theory

Note that sum and product rules apply when dealing with probabilities too!with probabilities too!

Sequences of events are products

Either/or requires sum rule and subtraction principle

Complementary rule works too!

151

Conditional Probability

The conditional probability of E given F is

P(E | F) = p(E ∩ F) / p(F)

This is the probability that E will/has occurred if we know that F has/will occur.

152

Independence

Two events, E and F, are independent iff

p(E1 ∩ E2) = p(E1) p(E2)

The two events don’t influence one another!

153

Repeated trials

If there are a number of trials being conducted, each of which has a probability of success of p each of which has a probability of success of p and a probability of failure of q = 1 – p, then the probability of exactly k successes in n ind p nd nt tri ls isindependent trials is

C(n k)pkqn-kC(n,k)p q

This is called the binomial distributionThis is called the binomial distribution.

154

Bayes’ Theorem

Consider the following problem:

There are two boxes holding red and green balls. Box 1 contains 2G, 7R.Box 2 contains 4G, 3R.

A ball is selected by choosing a box at random, then h i b l t d f th t bchoosing a bal at random from that box.

If a red ball is selected, what is the probability it cam from the first box? cam from the first box?

155

Bayes’ TheoremLet E be “a red ball is chosen”

So E is “a green ball is chosen”So E is a green ball is chosenLet F be “a ball is chosen from box 1”

So F is “a ball is chosen from box 2”

We want to know p(F|E).

156

Bayes’ Theorem

By conditional prob, p(F|E) = p(F∩E)/p(E).

We know p(E|F) = 7/9 and p(E|F) = 3/7We know p(F) = p(F) = 1/2We know p(F) p(F) 1/2

By conditional prob, p(E|F) = p(E∩F)/p(F)So, p(E∩F) = p(E|F)p(F) = (7/9)(1/2) = 7/18By the same logic, p(E∩F) = p(E|F)p(F) = 3/14Since p(E) = p(E∩F) + p(E∩F), p(E) = 38/63.

p(F|E) = p(F∩E)/p(E) = (7/18)(63/38) = 49/76 64 5%

157

p(F|E) = p(F∩E)/p(E) = (7/18)(63/38) = 49/76≈64.5%

Bayes’ Theorem

Given events E and F such that p(E) ≠ 0, p(F) ≠ 0,

p(F|E) = p(E|F)p(F) p(E|F)p(F) + p(E|F)p(F) p(E|F)p(F) p(E|F)p(F)

This is the equation resulting from the reasoning we just went through. It provides a means for calculating conditional probabilities in terms of other related conditional probabilitiesother, related conditional probabilities.

Why do this? Some conditional probabilities are easier than others to calculate directly.

158

Expected Values

We sometimes use the syntax X(s) to represent a random variable over some sample space Srandom variable over some sample space S.

For example, consider a random variable corresponding to the number of heads that come

h fl up when flipping a coin 2 times.

The sample space S is {HH HT TH TT}The sample space S is {HH, HT, TH, TT}

X(HH) = 2 X(HT) = 1 X(TH) = 1 X(TT) = 0X(HH) 2, X(HT) 1, X(TH) 1, X(TT) 0

The “s” in X(s) refers to an element of S.

159

Expected Values

There is a formal way to determine this calculation.

For a random variable X(s) over sample space S, the expected value of X isp

E(X) = ∑ p(s)X(s)s∈S

You might prefer to think of it this way…

s∈S

E(X) = ∑ p(X=r)rr∈X(s)

160

Variance

Expected value gives us an important piece of information regarding a distribution or random information regarding a distribution or random variable.

It’s like knowing the average grade for the class.

B t th l d ’t t ll h d tBut the class average doesn’t tell us how spread outthe classes scores were. For that we need another measure- a measure of spread.p

161

Variance

Variance is a measure of spread.

For a ranom variable X over a sample space S, the variance of X is given byg y

V(X) = ∑ (X(s) – E(X))2 p(s)s∈S

You may prefer the following form (I certainly do!):

s∈S

V(X) = E(X2) – E(X)2

162

Standard Deviation

Combined, variance and expected value can give a lot of information Many distributions such as the of information. Many distributions, such as the Normal distribution (bell curve), are defined in terms of these two parameters.

h d d d f d The standard deviation of X is sometimes used instead of variance. It has nice properties that you may learn about if you take a course in y m y f yprobability of statistics.

The standard deviation of X is given by σ(X) = V(X)½

163

Intro to Recurrence Relations

Earlier in the semester, we saw how we could define sequences recursively or functionallysequences recursively or functionally.

Specifically, we learned how to take functionally-p y ydefined sequences and transform them to recursively-defined sequences.

Example: an = 2n becomesa0 = 1a0 1an+1 = 2n+1 = 2⋅2n

= 2an, for n ≥ 1.

164

Intro to Recurrence Relations

Solving recurrence relations works in the opposite direction direction.

But there’s a catch… (Isn’t there always?)

A recursive definition of a sequence involves a recursive formula and a set of basis values.

Th f l it lf ith t th i iti l diti i The formula itself, without the initial conditions, is a recurrence relation.

We are going to be interested in solving relations both with, and without, initial conditions.

165

Intro to Recurrence Relations

Without initial conditions, a recurrence relation defines a set or family of sequencesdefines a set, or family, of sequences.

Consider an+1 = 2an.n 1 n

If a0 = 1, an = 2n.But if a0 = 3, an = 3⋅2n.

These two sequences are clearly similar. This is because a = 2a defines a family of sequences because an+1 = 2an defines a family of sequences, an = a0⋅2n, for n ≥ 1.

166

Intro to Recurrence Relations

A recurrence relation along with initial conditions specify a single sequence. Any such sequence is a p y g q y qsolution to the relation.

We can check solutions using substitutionWe can check solutions using substitution.

Consider the recurrence relation an = 2an-1 - an-2.n n 1 n 2Is an = 3n a solution for n ≥ 1? Try it out!

an = 2an-1 - an-2 = 2⋅3(n-1) – 3(n-2) 6 6 3 6= 6n – 6 -3n + 6

= 3n= an

167

an

Intro to Recurrence Relations

Finally, let’s see how we can apply recurrence relations and their solutions to a tough counting g gproblem.

How many bitstrings of length n do not contain How many bitstrings of length n do not contain consecutive 0’s?

The techniques we’ve studied so far can’t solve this without ridiculous amounts of effort!

One solution is 5-½( (1+5½)/2 )n+2 - 5-½( (1-5½)/2 )n+2One solution is 5 ( (1 5 )/2 ) 5 ( (1 5 )/2 ) .

We can find a more elegant and easier solution!!!

168