Counting III: Pascal’s Triangle, Polynomials, and Vector Programs

Counting III: Pascal’s Triangle, Polynomials, and Vector

Programs

Great Theoretical Ideas In Computer Science

Steven Rudich

CS 15-251 Spring 2003

Lecture 11 Feb 14, 2004 Carnegie Mellon University

X1 X2 + + X3

1 + X1 + X11 + X + X22 + X + X 33 + … + X + … + Xn-1n-1 + X + Xnn ==

XXn+1 n+1 - 1- 1

XX - 1- 1

The Geometric Series

1 + X1 + X11 + X + X22 + X + X 33 + … + X + … + Xnn + ….. + ….. ==

11

1 - X1 - X

The Infinite Geometric Series

(X-1) ( 1 + X1 + X2 + X 3 + … + Xn + … )= X1 + X2 + X 3 + … + Xn + Xn+1

+ …. - 1 - X1 - X2 - X 3 - … - Xn-1 – Xn - Xn+1 - …

= 1

1 + X1 + X11 + X + X22 + X + X 33 + … + X + … + Xnn + ….. + ….. ==

11

1 - X1 - X

1 + x + ….. ____________________1-x | 1 -[1 – x] _____ x -[x – x2 ]

______ x2

-....

1 + X1 + X11 + X + X22 + X + X 33 + … + X + … + Xnn + ….. + ….. ==

11

1 - X1 - X

1 + aX1 + aX11 + a + a22XX22 + a + a33X X 33 + … + a + … + annXXnn + ….. =+ ….. =

11

1 - aX1 - aX

Geometric Series (Linear Form)

(1 + aX1 + a2X2 + … + anXn + …..) (1 + bX1 + b2X2 + … + bnXn

+ …..) =

11

(1 – aX)(1-bX)(1 – aX)(1-bX)

Geometric Series (Quadratic Form)


+ …..) =

1 + c1X1 + .. + ck Xk + …

Suppose we multiply this out to get a single, infinite

polynomial.

What is an expression for Cn?


+ …..) =

1 + c1X1 + .. + ck Xk + …

cn =

aa00bbnn + a + a11bbn-1n-1 +… a +… aiibbn-in-i… + a… + an-1n-1bb11 + +

aannbb00


+ …..) =

1 + c1X1 + .. + ck Xk + …

If a = b then

cn = (n+1)(an)


aannbb00

(a-b) (aa00bbnn + a + a11bbn-1n-1 +… a +… aiibbn-in-i… + a… + an-1n-1bb11 + a + annbb00)

= aa11bbnn +… a +… ai+1i+1bbn-in-i… + a… + annbb11 + a + an+1n+1bb00

- aa00bbn+1 n+1 – a – a11bbnn… a… ai+1i+1bbn-in-i… - a… - an-1n-1bb22 - a - annbb11

= - bn+1 + an+1

= an+1 – bn+1

aa00bbnn + a + a11bbn-1n-1 +… a +… aiibbn-in-i… + a… + an-1n-1bb11 + a + annbb00 = = aan+1 n+1 – – bbn+1n+1

aa - b- b


+ …..) =

1 + c1X1 + .. + ck Xk + …

if a b then

cn =


aannbb00

aan+1 n+1 – – bbn+1n+1

aa - b- b


+ …..) =

11

(1 – aX)(1-bX)(1 – aX)(1-bX)


aan+1 n+1 – – bbn+1n+1

aa - b- b

n=0..1Xn

=

=

n=0..1 Xnor (n+1)a(n+1)ann

when a=b

Previously, we saw that

Polynomials Count!

What is the coefficient of BA3N2 in the expansion of

(B + A + N)6?

The number of ways to

rearrange the letters in the

word BANANA.

1 X 1 X1 X 1 X

1 X 1 X

1 X

Choice tree for terms of (1+X)3

1 X X X2 X X2 X2 X3

Combine like terms to get 1 + 3X + 3X2 + X3

The Binomial Formula

(1 X) FHGIKJ

FHGIKJ

FHGIKJ

FHGIKJ

FHGIKJ

n k nn n

Xn

Xn

kX

n

nX

0 1 22 ... ...

binomial expression

Binomial Coefficients

0

(1 )n

n k

k

nx x

k


0

(1 )n

n k

k

nx x

k

One polynomial, two representations

“Product form” or“Generating form” “Additive form” or

“Expanded form”

0

0

(1 )n

n k

k

k

k

nx x

k

nx

k

“Closed form” or“Generating form”

“Power series” (“Taylor series”) expansion

Since 0 if n

k nk

Power Series Representation

By playing these two representations againsteach other we obtain a new representation ofa previous insight:

0

0

(1 )

2

nn k

k

nn

k

nx x

k

n

k

Let x=1.

The number ofThe number ofsubsets of an subsets of an nn-element set-element set

0

0

1

even odd

(1 )

0 ( 1)

2

nn k

k

nk

k

n nn

k k

nx x

k

n

k

n n

k k

Let x= -1.

Equivalently,

By varying x, we can discover new identities

0

0

1

even odd

(1 )

0 ( 1)

2

nn k

k

nk

k

n nn

k k

nx x

k

n

k

n n

k k

Let x= -1.

Equivalently,

The number of even-sized subsets of an n element set is the same as the number of odd-

sized subsets.

We could discover new identities by substituting in

different numbers for X. One cool idea is to try complex

roots of unity, however, the lecture is going in another

direction.

0

(1 )n

n k

k

nx x

k

Proofs that work by manipulating algebraic forms

are called “algebraic” arguments. Proofs that build a 1-1 onto correspondence are

called “combinatorial” arguments.

0

(1 )n

n k

k

nx x

k

Let Let OOnn be the set of binary strings of be the set of binary strings of length length nn with an with an oddodd number of ones. number of ones.

Let Let EEnn be the set of binary strings of be the set of binary strings of length length nn with an with an even even number of ones.number of ones.

We gave an We gave an algebraicalgebraic proof that proof that

OOn n = = E Enn

1

even odd

2n n

n

k k

n n

k k

A Combinatorial Proof

Let Let OOnn be the set of binary strings of length be the set of binary strings of length nn with an with an oddodd number of ones. number of ones.

Let Let EEnn be the set of binary strings of length be the set of binary strings of length nn with an with an even even number of ones.number of ones.

A A combinatorialcombinatorial proof must construct a proof must construct a one-one-to-one correspondence to-one correspondence betweenbetween OOn n and and EEn n

An attempt at a correspondenceLet fn be the function that takes an

n-bit string and flips all its bits.

...but do even n work? In f6 we have

110011 001100

101010 010101

Uh oh. Complementing maps evens to evens!

fn is clearly a one-to-one and onto function

for odd n. E.g. in f7 we have

0010011 1101100

1001101 0110010

A correspondence that works for all n

Let Let ffnn be the function that takes an be the function that takes an n-bit string and flips only n-bit string and flips only the first bitthe first bit..

For example,For example,

0010011 0010011 1010011 10100111001101 1001101 0001101 0001101

110011 110011 010011 010011101010 101010 001010 001010

The binomial coefficients have so many representations that

many fundamental mathematical identities

emerge…

0

(1 )n

n k

k

nx x

k


(1+X)1 =

(1+X)0 =

(1+X)2 =

(1+X)3 =

1

1 + 1X

1 + 2X + 1X2

1 + 3X + 3X2 + 1X3

(1+X)4 = 1 + 4X + 6X2 + 4X3 + 1X4

Pascal’s Triangle:kth row are the coefficients of (1+X)k

(1+X)1 =

(1+X)0 =

(1+X)2 =

(1+X)3 =

1

1 + 1X

1 + 2X + 1X2

1 + 3X + 3X2 + 1X3

(1+X)4 = 1 + 4X + 6X2 + 4X3 + 1X4

kth Row Of Pascal’s Triangle:

(1+X)1 =

(1+X)0 =

(1+X)2 =

(1+X)3 =

1

1 + 1X

1 + 2X + 1X2

1 + 3X + 3X2 + 1X3

(1+X)4 = 1 + 4X + 6X2 + 4X3 + 1X4

, , ,..., ,...0 1 2

n n n n n

k n

Inductive definition of kth entry of nth row:

Pascal(n,0) = Pacal (n,n) = 1; Pascal(n,k) = Pascal(n-1,k-1) +

Pascal(n,k)

(1+X)1 =

(1+X)0 =

(1+X)2 =

(1+X)3 =

1

1 + 1X

1 + 2X + 1X2

1 + 3X + 3X2 + 1X3

(1+X)4 = 1 + 4X + 6X2 + 4X3 + 1X4

0

0

1 1

0 1

2 2 2

1

1 1

1 2 1

1 3 3

0 1 2

3 3 3

0 31

3

1 2

“Pascal’s Triangle”

Al-Karaji, Baghdad 953-1029

Chu Shin-Chieh 1303The Precious Mirror of the Four Elements. . . Known in Europe by 1529 Blaise Pascal 1654

Pascal’s Triangle

11

1 11 1

1 2 11 2 1

1 3 3 11 3 3 1

1 4 6 4 11 4 6 4 1

1 5 10 10 5 11 5 10 10 5 1

1 6 15 20 15 6 11 6 15 20 15 6 1

“It is extraordinaryhow fertile in

properties thetriangle is.

Everyone cantry hishand.”

Summing The RowsSumming The Rows

11

1 1 ++ 1 1

1 1 ++ 2 2 ++ 1 1

1 1 ++ 3 3 ++ 3 3 ++ 1 1

1 1 ++ 4 4 ++ 6 6 ++ 4 4 ++ 1 1

1 1 ++ 5 5 ++ 10 10 ++ 10 10 ++ 5 5 ++ 1 1

1 1 ++ 6 6 ++ 15 15 ++ 20 20 ++ 15 15 ++ 6 6 ++ 1 1

=1=1

=2=2

=4=4

=8=8

=16=16

=32=32

=64=64

0

2n

n

k

n

k

11

1 11 1

1 2 11 2 1

1 3 3 11 3 3 1

1 4 6 4 11 4 6 4 1

1 5 10 10 5 11 5 10 10 5 1

1 6 15 20 15 6 11 6 15 20 15 6 1

6+20+6 1+15+15+1=

Summing on 1Summing on 1stst Avenue Avenue

11

1 11 1

1 2 11 2 1

1 3 3 11 3 3 1

1 4 6 4 11 4 6 4 1

1 5 10 10 5 11 5 10 10 5 1

1 6 15 20 15 6 11 6 15 20 15 6 1

1 ( 1)

1 2 2

n n

i k i k

i n n ni

11

1 11 1

1 2 11 2 1

1 3 3 11 3 3 1

1 4 6 4 11 4 6 4 1

1 5 10 10 5 11 5 10 10 5 1

1 6 15 20 15 6 11 6 15 20 15 6 1

Summing on kSumming on kthth Avenue Avenue

1

1

n

i k

i n

k k

11

1 11 1

1 2 11 2 1

1 3 3 11 3 3 1

1 4 6 4 11 4 6 4 1

1 5 10 10 5 11 5 10 10 5 1

1 6 15 20 15 6 11 6 15 20 15 6 1

=2

=5

=13

=3

=8

11

1 11 1

1 2 11 2 1

1 3 3 11 3 3 1

1 4 6 4 11 4 6 4 1

1 5 10 10 5 11 5 10 10 5 1

1 6 15 20 15 6 11 6 15 20 15 6 1

222

2222

Al-Karaji SquaresAl-Karaji Squares

11

1 1 1 1

1 1 2 2 ++ 2*2*11

1 1 3 3 ++ 2*2*3 3 1 1

1 1 4 4 ++ 2*2*6 4 6 4 1 1

1 1 5 5 ++ 2*2*10 10 10 10 5 5 1 1

1 1 6 6 ++ 2*2*15 15 20 20 15 15 6 6 1 1

=0=0

=1=1

=4=4

=9=9

=16=16

=25=25

=36=36

All these properties can be proved inductively and algebraically. We will give combinatorial proofs using

the Manhattan block walking representation of

binomial coefficients.

How many shortest routes from A to B?

B

A

10

5

Manhattan

k’th Avenue01

23

4

j’th Street 01

23

4

Manhattan

k’th Avenue01

23

4

Level n0

1

34

2

………

…………………… …………………………

…………………………………

………….

Manhattan

k’th Avenue01

23

4

Level n0

1

34

2

………

…………………… …………………………

…………………………………

………….

. . . . .. . . . .

. . . . .. . . . .

. . . . .

level nk’th Avenue1

11

11

1

1

11

11

32

34 46

156 65 510 10

1520

. . . . .. . . . .

. . . . .. . . . .

. . . . .


1

1

1

1

1

1

1

1

11

3

2

3

4 46

156 6

5 510 10

1520

1 1

1

n n n

k k k

. . . . .. . . . .

. . . . .. . . . .

. . . . .


1

1

1

1

1

1

1

1

11

3

2

3

4 46

156 6

5 510 10

1520

1

even

2n

k

n

k

. . . . .. . . . .

. . . . .. . . . .

. . . . .


1

1

1

1

1

1

1

1

11

3

2

3

4 46

156 6

5 510 10

1520

2

0

2n

k

n n

k n

0! 1

1 if 0

0 if 0 or

nk

k

nk k n

k

By convention:

(empty product = 1)

1

1

1

n

i

i n

k k

1

Corollary ( 1)

1 ( 1)

2 2

n

i

k

n n ni

. . . . .. . . . .

. . . . .. . . . .

. . . . .


1

1

1

1

1

1

1

1

11

3

2

3

4 46

156 6

5 510 10

1520

Application (Al-Karaji):

2 2 2 2 2

0

1

1 2 3

(1 0 1) (2 1 2) (3 2 3) ( ( 1) )

1 0 2 1 3 2 ( 1)

2 3 4 5 12

2 2 2 2 2 2

1 1 (2 1)( 1)2

3 2 6

n

i

n

i

i n

n n n

n n i

n n

n n n n n

Vector Programs

Let’s define a (parallel) programming language called VECTOR that operates on possibly infinite vectors of numbers. Each variable V! can be thought of as:

< * , * , * , * , *, *, . . . . . . . . . >

0 1 2 3 4 5 . . . . . . . . .

Vector Programs

Let k stand for a scalar constant<k> will stand for the vector <k,0,0,0,…>

<0> = <0,0,0,0,….><1> = <1,0,0,0,…>

V! + T! means to add the vectors position-wise.

<4,2,3,…> + <5,1,1,….> = <9,3,4,…>

Vector Programs

RIGHT(V!) means to shift every number in V! one position to the right and to place a 0 in position 0.

RIGHT( <1,2,3, …> ) = <0,1,2,3,. …>

Vector Programs

Example:

V! := <6>;V! := RIGHT(V!) + <42>;V! := RIGHT(V!) + <2>;V! := RIGHT(V!) + <13>;

VV!! = < 13, 2, 42, 6, 0, 0, 0, . . . > = < 13, 2, 42, 6, 0, 0, 0, . . . >

Stare

V! = <6,0,0,0,..>V! = <42,6,0,0,..>V! = <2,42,6,0,..> V!

= <13,2,42,6,.>

Vector Programs

Example:

V! := <1>;

Loop n times: V! := V! + RIGHT(V!);

VV!! = n = nthth row of Pascal’s triangle. row of Pascal’s triangle.

Stare

V! = <1,0,0,0,..>

V! = <1,1,0,0,..>V! = <1,2,1,0,..> V! = <1,3,3,1,.>

X1 X2 + + X3

Vector programs can be

implemented by polynomials!

Programs -----> Polynomials

The vector V! = < a0, a1, a2, . . . > will be represented by the polynomial:

Formal Power Series

The vector V! = < a0, a1, a2, . . . > will be represented by the formal power series:

V! = < a0, a1, a2, . . . >

<0> is represented by 0<k> is represented by k

RIGHT(V! ) is represented by (PV X)

V! + T! is represented by (PV + PT)

Vector Programs

Example:

V! := <1>;



PV := 1;

PV := PV + PV X;

Vector Programs

Example:

V! := <1>;



PV := 1;

PV := PV (1+ X);

Vector Programs

Example:

V! := <1>;



PV = (1+ X)n

What is the coefficient of Xk in the expansion of:

( 1 + X + X2 + X3 + X4

+ . . . . )n ?

Each path in the choice tree for the cross terms has n choices of exponent e1, e2, . . . , en ¸ 0. Each exponent can be any natural number.

Coefficient of Xk is the number of non-negative solutions to:

e1 + e2 + . . . + en = k


( 1 + X + X2 + X3 + X4

+ . . . . )n ?

n

n -1

1k

( 1 + X + X2 + X3 + X4

+ . . . . )n =

k

k 0

nX

n -1

11

1n

k

X


(a0 + a1X + a2X2 + a3X3 + …) ( 1 + X + X2 + X3

+ . . . )

= (a0 + a1X + a2X2 + a3X3 + …) / (1 – X) ?

a0 + a1 + a2 + .. + ak

(a0 + a1X + a2X2 + a3X3 + …) / (1

– X)

= k

k 0 i 0

a X

i k

i

=

PREFIXSUM(a0 + a1X + a2X2 + a3X3 + …)

Let’s add an instruction called PREFIXSUM to our

VECTOR language.

W! := PREFIXSUM(V!)

means that the ith position of W contains the sum of all the numbers in V from

positions 0 to i.

What does this program output?

V! := 1! ;Loop k times: V! := PREFIXSUM(V!) ;

k’th Avenue01

23

4

Can you see how PREFIXSUM can be represented by a

familiar polynomial expression?

W! := PREFIXSUM(V!)

is represented by

PW = PV / (1-X)

= PV (1+X+X2+X3+

….. )

Al-Karaji Program

Zero_Ave := PREFIXSUM(<1>);First_Ave := PREFIXSUM(Zero_Ave);Second_Ave :=PREFIXSUM(First_Ave);

Output:= First_Ave + 2*RIGHT(Second_Ave)

OUTPUTOUTPUT!! = = <1, 4, 9, 25, 36, 49, …. ><1, 4, 9, 25, 36, 49, …. >

Al-Karaji Program

Zero_Ave = 1/(1-X);First_Ave = 1/(1-X)2;Second_Ave = 1/(1-X)3;

Output = 1/(1-X)2 + 2X/(1-X)3

(1-X)/(1-X)(1-X)/(1-X)3 3 + 2X/(1-X)+ 2X/(1-X)3 3

= (1+X)/(1-X)= (1+X)/(1-X)33

(1+X)/(1-X)(1+X)/(1-X)33

Zero_Ave := PREFIXSUM(<1>);First_Ave := PREFIXSUM(Zero_Ave);Second_Ave :=PREFIXSUM(First_Ave);

Output:= RIGHT(Second_Ave) + Second_Ave

Second_Ave = <1, 3, 6, 10, 15,. Second_Ave = <1, 3, 6, 10, 15,.

RIGHT(Second_Ave) = <0, 1, 3, 6, 10,.RIGHT(Second_Ave) = <0, 1, 3, 6, 10,.Output = <1, 4, 9, 16, 25

(1+X)/(1-X)(1+X)/(1-X)3 3 outputs <1, 4, 9, ..>outputs <1, 4, 9, ..>

X(1+X)/(1-X)X(1+X)/(1-X)3 3 outputs <0, 1, 4, 9, ..>outputs <0, 1, 4, 9, ..>

The kThe kthth entry is k entry is k22

X(1+X)/(1-X)X(1+X)/(1-X)3 3 = = k k22XXkk

What does X(1+X)/(1-X)What does X(1+X)/(1-X)44 do?do?

X(1+X)/(1-X)X(1+X)/(1-X)44 expands to : expands to :

SSkk X Xkk

where Swhere Skk is the sum of the is the sum of the first k squaresfirst k squares

Aha! Thus, if there is an Aha! Thus, if there is an alternative interpretation of alternative interpretation of

the kthe kthth coefficient of coefficient of X(1+X)/(1-X)X(1+X)/(1-X)44

we would have a new way we would have a new way to get a formula for the sum to get a formula for the sum

of the first k squares.of the first k squares.

Using pirates and gold we found that:

k

k 0

nX

n -1

11

1n

k

X

k

k 0

X3

4

31

1

k

X

THUS:

Coefficient of Xk in PV = (X2+X)(1-X)-4

is the sum of the first k squares:

k

k 0

X3

4

31

1

k

X

Vector programs -> Polynomials-> Closed form expression

Expressions of the formExpressions of the form

Finite Polynomial / Finite Finite Polynomial / Finite PolynomialPolynomial

are called are called Rational Polynomial Rational Polynomial ExpressionsExpressions. .

Clearly, these expressions have Clearly, these expressions have some deeper interpretation as a some deeper interpretation as a

programming language.programming language.

What about this one?

X/(1-X-X2)

0 1 2 30 1 2 32

01i

ii

xF x F x F x F x F x

x x

The action of dividing one polynomial by another can

simulate a program to recursively compute Fibonacci

numbers.

Vector Program I/O

Example:

INPUT I!; /* not allowed to alter I! */

V! := I! + 1;Loop n times: V! := V! + RIGHT(V!) + I! ;

OUTPUT V!

Vector Recurrence Relations

Let P be a vector program that takes input.

A vector relation is any statement of the form:

V! = P( V! )

If there is a unique V! satisfying the relation, then V! is said to be defined by the relation V! = P( V! ).

Fibonacci Numbers

0 1

1 2 1

0, 1,

,n n n n

F F

F F F

Recurrence Relation Definition:

Vector Recurrence Relation Definition:

FF!! = RIGHT( F = RIGHT( F! ! + <1> ) + RIGHT( RIGHT( F+ <1> ) + RIGHT( RIGHT( F!! ) ) ) )

FF!! = RIGHT( F = RIGHT( F! ! + <+ <1>1> ) + RIGHT( RIGHT( F ) + RIGHT( RIGHT( F!! ) ) ) )

RIGHTRIGHT(F(F! ! +<+<1>1>) ) = = 00, a, a0 0 + + 11, a, a11, a, a22, a, a33,,

FF!! = = aa00, a, a1 1 , a, a22, a, a33, a, a44, . . ., . . .

RIGHTRIGHT( ( RIGHTRIGHT( F( F!! ) ) ) )

= = 00, , 00 , a , a00, a, a11, a, a22, a, a33, ., .

FF!! = RIGHT( F = RIGHT( F! ! + + 11 ) + RIGHT( RIGHT( F ) + RIGHT( RIGHT( F!! ) ) ) )

RIGHTRIGHT(F(F + + 11)) = = ((FF++11)) XX

FF = = aa00 + a + a1 1 X + aX + a2 2 XX22 + a + a3 3 XX33 + +

RIGHTRIGHT( ( RIGHTRIGHT( F ) )( F ) )

= F= F XX22

F = (F + 1) X + F XF = (F + 1) X + F X22

RIGHTRIGHT(F(F + 1) + 1) = ( = (F+1F+1)) XX

FF = = aa00 + a + a1 1 X + aX + a2 2 XX22 + a + a3 3 XX33 + . . . + . . .

RIGHTRIGHT( ( RIGHTRIGHT( F ) )( F ) )

= = F F XX22

F = F X + X + F XF = F X + X + F X22

Solve for F. F - FX - FX2 = XF(1-X-X2) = X

F = X/(1-X-X2)

What is the Power Series Expansion of x / (1-x-

x2) ?

Since the bottom is quadratic we can factor

it.

X / (1-X-X2) =

X/(1- X)(1 – (-)-1X)

where =

“The Golden Ratio”

XX

(1 – (1 – X)(1- (-X)(1- (--1-1X)X)

Linear factors on the bottom

n=0..∞= Xn?


+ …..) =

11

(1 – aX)(1-bX)(1 – aX)(1-bX)


aan+1 n+1 – – bbn+1n+1

aa - b- b

n=0..1Xn

=

=

11

(1 – (1 – X)(1- (-X)(1- (--1-1X)X)


n+1 n+1 – (-– (---1)1)n+1n+1

5 5 n=0.. ∞

= Xn

XX

(1 – (1 – X)(1- (-X)(1- (--1-1X)X)

Power Series Expansion of F

n+1 n+1 – (-– (---1)1)n+1n+1

55 n=0.. ∞

= Xn+1

0 1 2 30 1 2 32

01i

ii


x x

20

1 1

1 5

i

i i

i

xx

x x

0 1 2 30 1 2 32

01i

ii


x x

The ith Fibonacci number is:

0

1 1

5

i

i

i

References

Applied Combinatorics, by Alan Tucker

Generatingfunctionology, Herbert Wilf

Documents

Counting III: Pascal’s Triangle, Polynomials, and Vector Programs