Chapter 2 The Fundamentals: Algorithms, the Integers, and Matrices

• Algorithm: specifying a sequence of steps used to solve a problem.

• Computational complexity of an algorithm: what are the computer resources needed to use this algorithm to solve a problem of a specified size?

• Integers: properties of integers, division of integers, algorithms involving integers.

• Matrices: basic material about matrices, matrix arithmetic.

2.1 Algorithms

Definition 1. An algorithm is a finite sequence of precise instructions for performing a computation or for solving a problem.

Example 1 Describe an algorithm for finding the maximum (largest) value in a finite sequence of integers.Solution

1. Set the temporary maximum equal to the first integer in the sequence.

2. Compare the next integer in the sequence to the temporary maximum, and set the larger one to be temporary maximum.

3. Repeat the previous step if there are more integers in the sequence.

4. Stop when there are no integers left in the sequence. The temporary maximum at this point is the maximum in the sequence.

ii

n

aa

n i:

a

,...,a,aa

:max then max if

to2for

:max

integers) :(max Procudure

Sequence Finite ain Element Maximum theFinding 1. Algorithm

1

21

Instead of using a particular computer language, we use a form of pseudocode.

The properties of algorithms:

• Input

• Output

• Definiteness

• Correctness

• Finiteness

• Effectiveness

• Generality

Example 2 Describe an algorithm for finding an element x in a list of distinct elements .,...,, 21 naaa

found}not is if 0 isor , equals that term

ofsubscript theis {

0 else

then if

1

) and ( while

1

integers)distinct :,...,integer,:( Procedure

Algorithm.Search linear The 2 Algorithm

21

xx

location

locatiton:

ilocation:ni

i i:

axni

i:

a,aaxrchlinear sea

i

n

found}not is if 0 isor , equals that term

ofsubscript theis {

0 else

then if

end

else

1 then if

2

begin

)( while

1

integers) increasing:,...,integer,:( Procedure

Algorithm.Search Binary The 3 Algorithm

21

xx

location

location:

ilocation:ax

mj:

mi:ax

j)/(im:

ji

nj:

i:

a,aaxrchbinary sea

i

m

n

2.2 Complexity of Algorithms

Assume that both algorithms A and B solve the problem P.

Which one is better?

• Time complexity: the time required to solve a problem of a specified size.

• Space complexity: the computer memory required to solve a problem of a specified size.

The time complexity is expressed in terms of the number of operations used by the algorithm.

• Worst case analysis: the largest number of operations needed to solve the given problem using this algorithm.

• Average case analysis: the average number of operations used to solve the problem over all inputs.

ii

n

aa

n i:

a

,...,a,aa

:max then max if

to2for

:max

integers) :(max Procudure

Sequence Finite ain Element Maximum theFinding 1. Algorithm

1

21

1n

2(n-1)

Number of operations

3n-1

found}not is if 0 isor , equals that term

ofsubscript theis {

0 else

then if

1

) and ( while

1

integers)distinct :,...,integer,:( Procedure

Algorithm.Search linear The 2 Algorithm

21

xx

location

locatiton:

ilocation:ni

i i:

axni

i:

a,aaxrchlinear sea

i

n

1

2(n+1)

n

2

3n+5

Example 3 Analyze the time complexities of Algorithm 1 and 2

found}not is if 0 isor , equals that term

ofsubscript theis {

0 else

then if

end

else

1 then if

2

begin

)( while

1

integers) increasing:,...,integer,:( Procedure

Algorithm.Search Binary The 3 Algorithm

21

xx

location

location:

ilocation:ax

mj:

mi:ax

j)/(im:

ji

nj:

i:

a,aaxrchbinary sea

i

m

n

times.n)log(kk most at repeatsIt

.2nLet

2

k

1

1

2

2

11

Number of operations = 4 log n+42

For finding x in a list of n elements, we have two algorithms

Which use

Which one is better?

ly.respective ,rations) time(ope44log and 53 2 nn

• A constant factor does not influence the running time so much.

Therefore, 3n+5 grows faster than when n becomes larger. Usually, we say that 3n+5 is O(n) and say that is O( ).

4log4 2 n

4log4 2 n n2log

Orders of Growth

Running

Time

necessary operations

Operation

Per second

instant 1 second 11.5 days Never end

days

instant Instant 1 second Never end

days

Running time for a problem with size 610n

2n n2nlg n

6101210

593502

593402

Using silicon computer, no matter how fast CPU will be you can never solve the problem whose running time is exponential !!!

Asymptotic Notations: O-notation

Definition 2.1 A function f(n) is said to be O(g(n)) if there exist some constant . allfor )()(such that 0 and 0 0000 nnngcnfnc

)(Ng

)(0 Nfc

N0N

. then , )constant a is 0(c )(

)(lim If O(g(n))f(n)c

ng

nfn

Example 5

List the following function in O-notation in increasing order:

.2,!,,lg,,,lg 32 nnnnnnnn

Example 3 Prove 2n+1=O(n)

)(51210 Prove 4 Example 22 nOnn

n

nnnnn

n

n

20001.0

,1000lg100lg

,10001005n

functions? following theofoh -big theis What 6 Example

2

25

2.2 The Integers and Division

• We discuss the properties of integers which belongs to

the branch of Mathematics called number theory.

. dividenot does when write.We divides

that denotes notation The . of multiple a is and offactor a

is say that we divides When .such that cinteger an is thereif

divides say that we,0 with integers are and If 1. Definition

baa|bb

aa|babb

abaacb

baaba

a is a factor of b

b is a multiple of a

Example 1 Determine whether 3|7 and whether 3|12.

Solution: It follows that 3|7 and 3|12.

• Basic properties of divisibility of integers

Theorem 1. Let a, b, and c be integers. Then

1. If a|b and a|c, then a|(b+c).

2. If a|b, then a|bc for all integers c.

3. If a|b and b|c, then a|c.

1. There are s and t such that b=as and c=at. Therefore, b+c=a(s+t).

2. There is s such that b=as. Therefore, bc=a(sc)

3. There are s and t such that b=as and c=bt, therefore, c=a(st)

Proof

Definition 2. A positive integer p greater than 1 is called prime if the only positive factors of p are 1 and p. A positive integer that is greater than 1 and is not prime is called composite.

Example 2. Is 7 a prime?

• The primes less than 100 are 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89 and 97.

Theorem 2. Every positive integer can be written uniquely as the uniquely as the product of primes in order of increasing size.

22222222221024

37333999

641641

5522100

3 Example

.that

such ,0 with ,and integer unique are Then there

integer. positive a andinteger an be Let 3. Theorem

r dqa

dr rq

da

divisordividendquotient remainder

Example 4 What are the quotient and remainder when 101 is divided by 11?

Solution: We have .2911101

quotient remainder

Definition 3. Let a and b be integers. The largest integer d such that d|a and d|b is called the greatest common divisor of a and b, denoted by gcd(a,b).

Example 5 What is gcd(24,36) and gcd(17,22)?

Definition 4. The integers a and b are relatively prime if gcd(a,b)=1.

Definition 5. The least common multiple of the positive integers a and b is the smallest positive integer that is divisible by both a and b, denoted by lcm(a,b).

25434253

34253

732)32,73lcm(2

32432,73295256 :Solution

432).lcm(95256, Find 6 Example

Definition 6. Let a be an integer and m be a positive integer. We denoted by a mod m the remainder when a is divided by m.

Example 7 17 mod 5 = 2, -133 mod 9=2, and 2001 mod 101 = 82.

. modulo tocongruent is that indicate to

) (mod use We. divides if modulo tocongruent is then

integer, positive a is and integers are and If 7. Definition

mba

mb aa-bmmba

m ba

6)? (mod 14 24 and 6) (mod 517 Is 8 Example

2.4 Integers and Algorithms• The Euclidean Algorithm: Find the greatest common divisor of two positive integers

Lemma 1 Let a=bq+r, where a,b,q, and r are integers. Then gcd(a,b)=gcd(b,r).

nnnn-

nnn-

,nnnnnn

,

,

r,r,rr

,rr,rra,b

qrr

rr rqrr

rr rqrr

rr rqrr

brar

baba

)0gcd()gcd(

)gcd()gcd()gcd(

have we1 Lemma From

0

0

0

have We. and Let

, with and integers positiveGiven

1

2110

1

1112

233221

122110

10

} is )end{gcd(

mod

begin

0y

integers) positive :gcd( Procedure

AlgorithmEuclidean The 1 Algorithm

xa,b

ry:

yx:

yxr:

while

by:

ax:

a,b

} is )end{gcd(

mod

begin

0y

integers) positive :gcd( Procedure

AlgorithmEuclidean The 1 Algorithm

xa,b

ry:

yx:

yxr:

while

by:

ax:

a,b

Example 1

Find the greatest common divisor of 414 and 662 using the Euclidean Algorithm.

Solution:

41282

2282166

821166248

1661248414

2481414662

Hence, gcd(414,662 ） =2, since 2 is the last nonzero remainder.

• Representation of integers

. ab

aa ak

abababa n

nb

k

k,

kk

kk

0and ,

than less integers enonnegativ are ,integer, enonnegativ a is where

.

form in theuniquely expressed becan

integer positiveAny .1an greater thinteger positive a Let 1 Theorem

10

011

1

expansion?binary its as )(101011111

hasat integer th theofexpansion decimal theis What 2 Example

2

3511222222)(101011111

Solution23468

2

10234

16 )175627(1116016141610162(2AE0B)

3 Example

.(12345) ofexpansion 8 base theFind

4 Example

10

.)30071()12345( Therefore,

3083

03824

0248192

719281543

11543812345

Solution

810

})( is ofexpansion base end{the

1

mod

begin

0 while

0

n:q

integer) positive :nexpansion( b base Procedure

integer) positiveexpansion( Base ngConstructi Algorithm

011 bk-

k

aaanb

k k:

q/b q:

bq:a

q

k:

n: b

•　 Algorithm for integer operations

Addition of a and b

20121

20121

20121

01321

) (

) (

) (

sssssba

bbbbb

aaaaa

cc c cc

nnn

nn

nn

nnn-

Example 5

1 1 1 0

1 1 1 0

1 0 1 1

1 1 0 0 1

} is sum theofexpansion binary the{

end

2

2

begin

1 to0for

0:

ly}respective,) and

) are and of expansionsbinary {the

integers) positive :( Procedure

integers ofAddition 3 Algorithm

201

20121

20121

)ss(s

c:s

d c:

dcba: s

c)/b(a d:

n-j:

c

bbb(b

aaa(a ba

a,badd

n-n

n

jjj

jj

nn-

nn-

Multiplication of a and b

1 1 0

1 0 1 1

1 1 0

1 1 0

0 0 0

1 1 0

1 0 0 0 0 1 0

a

b

a shifts 0 place

a shifts 1 place

a shifts 3 places

} of value theis {

1 to0for

0

products} partial theare ,...,,{

end

0 else

places shift then 1 if

begin

1 to0for

ly}respective,) and

) are and of expansionsbinary {the

integers) positive :( Procedure

Integers. gMultiplyin 4 Algorithm

110

20121

20121

abp

cp p:

n- j:

p:

ccc

: c

ja:cb

n-j:

bbb(b

aaa(a ba

a,bMultiply

j

n

j

jj

nn-

nn-

2.6 Matrices

entries. same thehave thay if equal are matrices Two

. called is columns and rows ofnumber same h thematrix witA

matrix.n man as called is columns and rows h matrix witA

numbers. ofarray r rectangula a ismatrix A 1. Definition

square

nm

nnnjnn

inijii

nj

nj

aaaa

aaaa

aaaa

aaaa

A

21

21

222221

111211

i row

j column

i row j column

][ write toisnotation shorthandA ijaA

].[

matrices. be ][ and ][Let

2. Definition

ijij

ijij

baBA

n mbBaA

tj

k

titkjikjijiijij

ijij

babababaccAB

n kbBkmaA

12211 where],[

matrix. be ][ andmatrix an be ][Let

3. Definition

2 5 2

3- 1- 3

2- 4 4

2 1 1-

0 3- 1

1- 4 3

0 4 3

3- 2 2

1- 0 1

1 Example

3- 0 13

8- 1- 11

3- 3 4

2 1 1-

0 3- 1

1- 4 3

0 4 3

3- 2 2

1- 0 1

2 Example

. have We

11

12 and

12

11Let

3 Example

BAAB

.

B

A

• Algorithms for matrix multiplication

} and ofproduct theis ][cend{C

end

to1for

0

begin

to1for

begin

to1for

matrices) :tion(multiplicamatrix procedure

tionMultiplicaMatrix 1 Algorithm

ij BA

bac: c

kq:

:c

nj:

m i:

A,B

qjiqijij

ij

• Transposes and powers of matrices

.

I

j iδjiδInn

n

n

ijijijn

100

010

001

Hence,

.if 0 and if 1 where],[matrix the

is order ofmatrix identity The 4 Definition

.Then matrix. an beLet AAIAInmA mn

. and

be to ofpower th thedefine Wematrix. an beLet

times

0

r

rn AAAAIA

ArnnA

. where],[ s,other wordIn

. of columns and rows theinginterchangby

obtainedmatrix a is ,by denoted , of transposeThe

matrix. an be ][Let 5 Definition

jiijijt

t

ij

a bbA

A

AA

nmaA

• Zero-One Matrices

A matrix with entries that are either 0 or 1 is called a zero-one matrix.

].[ and ][

matrices. one-zero be ][ and ][Let 8 Definition

ijijijij

ijij

baBAbaBA

nmbBaA

0 1 0

0 0 0

00 1 1 10

01 10 01

and 0 1 1

1 1 1

00 1 1 10

01 10 01 have We

. 0 1 1

0 1 0B and

0 1 0

1 0 1 ALet

6 Example

BA

BA

.4 2 0

3 2 1AThen .

4 3

2 2

0 1

ALet

4 Example

t

.1 and 1

with and allfor if symmetric is Thus, . if

symmetric called is ][matrix squareA 6 Definition

njni

jiaaAAA

aA

jiijt

ij

symmetric. is

5 4 3

4 2 2

3 2 1

Matrix

5 Example

).()()( where

,matrix theis ][ matrix. one-zero

be ][ andmatrix one-zero an be ][Let

9. Definition

2211 kjikjijiij

ij

ijij

bababac

nmcBA

n kbBkmaA

.

.

0 1 1

0 0 0

0 1 1

1)(00)(1 1)(01)1( 0)0()11(

1)(10)(0 1)(11)0( 0)1()10(

1)(00)(1 1)(01)1( 0)0()11(

. 1 1 0

0 1 1B and

0 1

1 0

0 1

ALet

7 Example

BA .

} and ofproduct Boolean theis ][cend{C

end

)(

to1for

0

begin

to1for

begin

to1for

matrices) one-zero :product(Boolean procedure

ProductBoolean The 2 Algorithm

ij BA

bac: c

kq:

:c

nj:

m i:

A,B

qjiqijij

ij

. and be to

ofpower Boolean th the

define Wematrix. one-zero square a beLet

times

][]0[

r

rn AAAAIA

A r

A