Snick snack CPSC 121: Models of Computation 2010 Winter Term 2 Revisiting Induction Steve Wolfman, based on work by Patrice Belleville and others 1

snick

snack

CPSC 121: Models of Computation2010 Winter Term 2

Revisiting Induction

Steve Wolfman, based on work by Patrice Belleville and others

1

Outline

• Prereqs and Learning Goals• Induction as a Formal Argument Form• Problems and Discussion

– Introductions– Odd Numbers– Horse Colours– CS Induction: Duplicate Detection, Binary

Search, MergeSort– More examples

• Next Lecture Notes

2

Learning Goals: Pre-Class

By the start of class, you should be able to:– Given a theorem to prove stated in terms of

its induction variable (i.e., usually, in terms of n), write out the skeleton of an inductive proof including: the basis step(s) that need to be proven, the inductive hypothesis, and the inductive step that needs to be proven.

3

Learning Goals: In-Class

By the end of this unit, you should be able to:– Formally prove properties of the non-negative

integers (or a subset like integers larger than 3) that have appropriate self-referential structure—including both equalities and inequalities—using either weak or strong induction as needed.

– Critique formal inductive proofs to determine whether they are valid and where the error(s) lie if they are invalid.

4

Outline





5

Induction: a Domino Effect

A proof by induction is like toppling a chain of dominoes.

Let’s say I tell you:– the first domino in the chain toppled– for every domino in the chain, if the domino

before it toppled, then the domino itself toppled

Did all the dominoes in the chain topple?

6

Formal (Weak) Induction

A formal induction proof follows the same pattern. To prove some property P(.) applies to all positive integers n, we prove:

The first domino topples.

If one domino topples…

Then the next one topples.

P(0) is true.

[For arbitrary k] If P(k) is true...

Then P(k+1) is also true.

Typically, we prove the second step by antecedent assumption/direct proof.

7

Form of an Induction Proof

Theorem: some property that depends on n

Basis step: show that a small case works

Inductive Hypothesis: Assume your property holds for n=k where k ≥ your basis step.

Inductive Step: Under this assumption, you need to prove that your property holds for n=k+1.

8

Side Note: Practical Induction

That’s how you formally write out the proof, but how do you figure out the proof?

Start at the inductive step!

9

Side Note: Practical Induction

Look at a “big” problem (of size n).

Figure out how to break it down into smaller pieces.

Assume those smaller pieces work. That will end up as your inductive hypothesis.

Figure out which problems cannot be broken down (usually small ones!). Those will end up as your basis step(s).

10

Outline





11

Worked Problem: How Many Introductions?

Problem: n people would like to introduce themselves to each other. How many introductions does it take?

For 2 people?

For 3 people?

For 4 people?

For 5 people?

…

For n people?Sound familiar?

Let’s prove it.12


Induction can feel very abstract. Let’s do the first few steps concretely…

For 1 person?

Given the number for 1, for 2?

Given the number for 2, for 3?13


And now the basis step plus the abstract connection between one step and the next…

For 1 person? 0 introductions.

Given the number for k (call it x), for k+1?a. x + 1b. x * kc. x + 2kd. x + 2(k + 1)e. None of these

14

My turn!

Formally…

Definition: When two people greet each other, we count that as two introductions.

To prove: It takes n(n-1) introductions for every pair of people in a group of n people to greet each other.

15

Proof

Basis step: A group of size 1 takes 0 intros. (Because there are no pairs.)

Note: we talked about induction to prove properties for the non-negative integers, but it can be more general.16

Proof (Continued)

Inductive step, to prove: If a group of size k takes k(k-1) intros, then a group of size k+1 takes (k+1)((k+1)-1) intros.

Assume size k takes k(k-1) intros (the Inductive Hypothesis)

...

17

Proof (Continued)

Inductive step, continued:

Imagine a group of size k that has completed introductions. That took k(k-1) intros by the IH.

Add one more to the group, and that person must greet all k other people, for a total of k(k-1) + 2k introductions.

k(k-1) + 2k = k[(k-1) + 2] = k(k+1) = (k+1)((k+1) – 1)

QED: if k takes k(k-1), (k+1) takes (k+1)((k+1)-1).

18

Proof (Completed)

Basis step: A group of size 1 takes 0 intros. (See proof above.)

Inductive step: If a group of size k takes k(k-1) intros, then a group of size k+1 takes (k+1)((k+1)-1) intros. (See proof above.)

QED.

From these two we could (but don’t have to) crank out a proof for any n that a group of size n requires n(n-1) introductions.

19

Cranking the Machine

Basis step: A group of size 1 takes 0 intros. (See proof above.)

Inductive step: If a group of size k takes k(k-1) intros, then a group of size k+1 takes (k+1)((k+1)-1) intros. (See proof above.)

Can you use these to prove the n=2 case?The n=5 case? a. Yes.The n=8675309 case? b. No.The n= case? c. It depends.

20

One More Time: the Induction Pattern

Establish the basis step– must lead to all the other cases– often P(1) or P(0), but not necessarily– often just one, but may be several

Prove the inductive step– always adds a case based on “smaller” cases– weak induction: if P(k) then P(k+1)– “strong” induction: if P(0), P(1), ..., P(k-2), and

P(k-1), then P(k+1).– generally: assume whatever you need!

QED, proven for all cases reachable by the inductive step from the basis step (usually all N)

21

Outline





22

Historical Problem: Sum of Odd Numbers

Problem: What is the sum of the first n odd numbers?

First, find the pattern. Then, prove it’s correct.

The first 1 odd number?

The first 2 odd numbers?

The first 3 odd numbers?

The first n odd numbers?Historical note: Francesco Maurolico made the first

recorded use of induction in 1575 to prove this theorem!23


Now, how do we prove it? We need...

Property: For all positive integers n... ?

Basis Step: Establish for n=?

Inductive Hypothesis: Assume... ?

Inductive Step: To prove... ?

24


Property: For all positive integers n, the sum of the first

n odd natural numbers is n2.

(We have figure this out on our own... Induction doesn’t help!)

Basis Step: Establish for n=?Inductive Hypothesis: Assume... ?Inductive Step: To prove... ?

25


Theorem: For all positive integers n, the sum of the first n odd natural numbers is n2.

Basis Step:

Establish for n=1. The sum of the “first 1 odd natural numbers” is 1, which equals 12.

Inductive Hypothesis: Assume... ?Inductive Step: To prove... ?

26


Theorem: For all positive integers n, the sum of the first n odd natural numbers is n2.

Basis Step for n = 1

Inductive Hypothesis: For some arbitrary k 1, assume the sum of the first k odd natural numbers is k2

Inductive Step: To prove... ?

27


Theorem: As above. Basis Step for n = 1 Inductive Hypothesis...Inductive Step: Prove that the sum of the first k+1

odd natural numbers is (k+1)2.The (k+1)th odd number is 2(k+1) - 1 = 2k + 1(k+1)2 = k2 + 2k + 1 = k2 + [(k+1)th odd number]By the IH, k2 = sum of the first k odd numbers; so,

(k+1)2 = sum of the first k+1 odd numbers.QED

28

Outline





29

Problem: Proof Critique

Theorem: All horses are the same colour.

See handout.

Problem: Critique the proof.

30


Theorem: All horses are the same colour.See handout.

Proof critique: Is the proof valid? a.Yes, because each step follows irrefutably

from the previous steps.b.Yes, because the premises are false.c.Yes, but not for the reasons listed here.d.No, because the conclusion is false.e.No, but not for the reasons listed here.

31


Theorem: All horses are the same colour.See handout.

Proof critique: Where is the error in the proof? a.The basis step.b.The inductive hypothesis.c.The inductive step.d.None of these, but there must be an error.e.None of these, and there may be no error.

32


Theorem: All horses are the same colour.

See handout.

Proof critique: Can the proof be fixed?

33

Outline





34

Induction and Computer Science

How important is induction? Is it just for proving things about N?

Induction forms the basis of…– proofs of correctness and efficiency for many

algorithms: e.g., How long does it take to detect duplicates in a list of n inputs? Does binary search work?

– “recursive” algorithms: e.g., “merge sort”– “recursive data structures”: e.g., How big can a “tree” of

height n be?35

Problem: Detecting Duplicates

Problem: How long does it take to detect duplicates in a list of n Inputs?

Let’s assume the amount of “time” taken is proportional to the number of comparisons we make between numbers.

We could compare each element to each other element and see if they’re the same.

Sound familiar? We already know how long this takes!

Note: this isn’t the fastest algorithm!36

Problem: Binary Search Works

Problem: Prove that binary search works.

Binary search is when we search a sorted list by checking the middle element.

If it’s what we’re looking for, we’re done.

Otherwise, we “throw out” the part of the list we don’t need (e.g., the left half if the element we looked at was too small) and start over on what remains.

37

Form of a Strong Induction Proof

Theorem: some property that depends on n

Basis step: show that one or more small cases (as needed) work

Inductive Hypothesis: Assume your property holds for any n = i where your basis step ≤ i < k and k is an integer > your basis step.

Inductive Step: Under this assumption, you need to prove that your property holds for n = k.

Hint: there’s really no difference. Just assume what you need in order to break the problem into smaller pieces.

38

Side Note: “Strong Induction”

We’ve used weak induction: basis step (n = a) plus “if it works for n then it works for n+1”.

In strong induction, we use: basis step plus “if it works for all a i < k then it works for k”.

Can we still build our individual proofs (for n = 1, n = 2, n = 3, ...) from that?

39

Sorting by “Merging”

Problem: sort a list of names.

Algorithm:1. If the list is of length 1, it’s sorted.

2. Otherwise:a) Divide the list in half (or as close as possible).

b) Sort each half using this algorithm.

c) Merge the sorted lists back together.

40

The Merge Step

Problem: given two lists of names in sorted order, merge them into a single sorted list.

Algorithm to merge lists a and b:1. Let c be an empty list.2. While a or b has elements remaining:

a) If one of a and b is empty, remove the first element from the other list and put it at the end of c.

b) Otherwise, compare the first elements of a and b, remove the smaller one, and put it at the end of c.

3. Return c.

41

Problem: Does Merging Work?

Problem: Prove that merging works.

Hint: the key to proving that a loop works is to use a “loop invariant”, some property that is true each time we start the loop. In this case, try focusing on what’s true of c on the nth time the loop starts.

Your property should be true on the 0th time the loop starts (when c is empty) and on the last time the loop starts (when c is, hopefully, the whole sorted list!).

42

Merging Invariant

Which of these is not an invariant we might prove for merging?

a.A is sorted

b.A and B are the same length

c.C contains i items (when we are about to execute the ith iteration of the loop)

d.Items in C are smaller than items in A and B

e.C is sorted

43

Merging Works: Basis Step

When we are about to start the “0th” iteration of the loop...

A is sorted?

B is sorted?

C is sorted?

C contains i = 0 items?

A+B contains n-i = n items?

Items in C are smaller than items in A, B?44

Merging Works: Inductive Hypothesis

ASSUME: When we are about to start the kth iteration of the loop...

A is sorted.

B is sorted.

C is sorted.

C contains i = k items.

A+B contains n-i = n-k items.

Items in C are smaller than items in A, B.45

Merging Works: Inductive Step

PROVE: When we are about to start the (k+1)th iteration of the loop...

A is sorted.

B is sorted.

C is sorted.

C contains i = k+1 items.

A+B contains n-i = n-k-1 items.

Items in C are smaller than items in A, B.46

Merging Works: Inductive Step

Which one is easiest to prove?

a.A is sorted.

b.C is sorted.

c.C contains i = k+1 items.

d.A+B contains n-i = n-k-1 items.

e.Items in C are smaller than items in A, B.

47

Problem: Merge Sort Works

Problem: Now that we know the merge step works, prove that merge sort works.

Algorithm:1. If the list is of length 1, it’s sorted.

2. Otherwise:a) Divide the list in half (or as close as possible).

b) Sort each half using this algorithm.

c) Merge the sorted lists back together.48


Basis Step: The algorithm does nothing to a list of length 1. Lists of length 1 are trivially sorted. Therefore, the algorithm is correct for n = 1.

49


Inductive Hypothesis: Assume the algorithm correctly sorts any list of length 1 i < k, for arbitrary k > 1.

Inductive Step: Prove that the algorithm correctly sorts a list of length k.

50


Inductive Hypothesis: Assume the algorithm correctly sorts any list of length 1 i < k, for arbitrary k > 1.

Inductive Step:

On a list of length k, the algorithm divides the list into two parts, one of length k/2, the other of length k/2.

k is at least 2; so, each half has at least one element (and is smaller than the original). So, the IH applies.

51


Inductive Step: Continued...

By the IH, the algorithm correctly sorts the two smaller lists. It then merges them. We’ve already proved that the merge step merges these into a single sorted list.

Therefore, the algorithm correctly sorts the list of length k.

QED

52

Outline





53

Problem:Prove that 2n < n!

Problem: prove that 2n < n!?

Note: is 2n < n!?54

“Rules” for Inequalities

To prove an inequality, do what you need in scratch work, then rewrite formally:

• Start from one side.• Work step-by-step to the other.• Never move “opposite” to your inequality (so,

to prove “<“, never make the quantity smaller).

• Strict inequalities: have at least one strict inequality step.

55

Outline





56

Learning Goals: In-Class

By the end of this unit, you should be able to:– Formally prove properties of the non-negative

integers (or a subset like integers larger than 3) that have appropriate self-referential structure—including both equalities and inequalities—using either weak or strong induction as needed.

– Critique formal inductive proofs to determine whether they are valid and where the error(s) lie if they are invalid.

57

snick

snack

Extra Slides

58


Theorem: All integers ≥ 2 are even.

Problem: Critique the proof. Is it valid? If not, why not? Can it be fixed, and how?

59

Problem:Sum of a Geometric Series

Problem: What is the sum of the first n terms of the form ai, if a is a real number between 0 and 1? (Note: we mean terms 0 through n.)

Note: we’re looking for

Can we use induction over the real numbers between 0 and 1?

1

11

a

an

60

Documents

Snick snack CPSC 121: Models of Computation 2010 Winter Term 2 Revisiting Induction Steve Wolfman, based on work by Patrice Belleville and others 1