Efficiency of Algorithms

Csci 107

Lecture 8

• Last time– Data cleanup algorithms and analysis (1), (n), (n2)

• Today– Binary search and analysis– Order of magnitude (lg n)– Sorting

• Selection sort

Searching• Problem: find a target in a list of values

• Sequential search– Best-case : (1) comparison

• target is found immediately

– Worst-case: (n) comparisons• Target is not found, or is the last element in the list

– Average-case: (n) comparisons• Target is found in the middle

• Can we do better? – No…unless we have the input list in sorted order

Searching a sorted list

• Problem: find a target in a sorted list– How can we exploit that the list is sorted, and come up

with an algorithm faster than sequential search in the worst case?

– How do we search in a phone book?

– Can we come up with an algorithm? • Check the middle value

• If smaller than target, go right

• Otherwise go left

Binary search

• Get values for list, A1, A2, ….An, n , target• Set start =1, set end = n• Set found = NO• Repeat until ??

– Set m = middle value between start and end– If target = m then

• Print target found at position m• Set found = YES

– If target < Am then • end = m-1

– Else • start = m+1

• If found = NO then print “Not found”• End

Efficiency of binary search

• What is the best case?– Found in the middle

• What is the worst case? – Initially the size of the list is n

– After the first iteration through the repeat loop, if not found, then either start = middle or end = middle ==> size of the list on which we search is n/2

– Every time in the repeat loop the size of the list is halved: n, n/2, n/4,….

– How many times can a number be halved before it reaches 1?

log2 x

• log2 x

– The number of times you can half a (positive) number x before it goes below 1

– Examples: • log2 16 = 4 [16/2=8, 8/2=4, 4/2=2, 2/2=1]

• log2 n = m <==> 2m = n• log2 8 = 3 <==> 23=8

log2 x

Increases very slowly

• log2 8 = 3

• log2 32 = 5

• log2 128 = 7

• log2 1024 = 10

• log2 1000000 = 20

• log2 1000000000 = 30

• …

Orders of magnitude

• Order of magnitude ( lg n)– Worst-case efficiency of binary search: ( lg n)

• Comparing order of magnitudes

(1) << (lg n) << (n) << (n2)

Comparing (lg n) and (n)

Does efficiency matter?

• Say n = 109 (1 billion elements)• 10 MHz computer ==> 1 instr takes 10-7 sec

– Seq search would take (n) = 109 x 10-7 sec = 100 sec

– Binary search would take (lg n) = lg 109 x 10-7 sec = 30 x10-7 sec = 3 microsec

Sorting

• Problem: sort a list of items into alphabetical or numerical order

• Why sorting? – Sorting is ubiquitous (very common)!!– Examples:

• Registrar: Sort students by name or by id or by department• Post Office: Sort mail by address• Bank: Sort transactions by time or customer name or accound

number …

• For simplicity, assume input is a list of n numbers• Ideas for sorting?

Selection Sort

• Idea: grow a sorted subsection of the list from the back to the front

5 7 2 1 6 4 8 3 |

5 7 2 1 6 4 3 | 8

5 2 1 6 4 3 | 7 8

5 2 1 3 4 | 6 7 8

…

|1 2 3 4 5 6 7 8

Selection Sort• Pseudocode (at a high level of abstraction)

– Get values for n and the list of n items

– Set marker for the unsorted section at the end of the list

– Repeat until unsorted section is empty• Select the largest number in the unsorted section of the list

• Exchange this number with the last number in unsorted section of list

• Move the marker of the unsorted section forward one position

– End

Selection Sort• Level of abstraction

– It is easier to start thinking of a problem at a high level of abstraction

• Algorithms as building blocks– We can build an algorithm from “parts” consisting of

previous algorithms– Selection sort:

• Select largest number in the unsorted section of the list• We have seen an algorithm to do this last time• Exchange 2 values

Selection Sort Analysis• Iteration 1:

– Find largest value in a list of n numbers : n-1 comparisons– Exchange values and move marker

• Iteration 2: – Find largest value in a list of n-1 numbers: n-2 comparisons– Exchange values and move marker

• Iteration 3: – Find largest value in a list of n-2 numbers: n-3 comparisons– Exchange values and move marker

• …• Iteration n:

– Find largest value in a list of 1 numbers: 0 comparisons– Exchange values and move marker

Total: (n-1) + (n-2) + …. + 2 + 1

Selection Sort

• Total work (nb of comparisons): – (n-1) + (n-2) + …. + 2 + 1 – This sum is equal to .5n2 -.5n (proved by Gauss)

=> order of magnitude is ( ? )

• Questions– best-case, worst-case ?– we ignored constants, and counted only comparisons.. Does this

make a difference?

• Space efficiency– extra space ?

Selection Sort

• In conclusion: Selection sort– Space efficiency:

• No extra space used (except for a few variables)

– Time efficiency• There is no best-case and worst-case• the amount of work is the same: (n2) irrespective of

the input

• Other sorting algorithms? Can we find more efficient sorting algorithms?

Exam 1

• Wednesday: Lab 4 (more efficiency, binary search, etc)– Due next Wednesday, but…

– Strongly encouraged to finish lab before Exam1

• Exam 1 (Monday febr 21st)– Material: Algorithms and efficiency

– Open books, notes, labs

– Practice problems handout

– Study group: this time only, Sunday night (instead of Monday)• Tom will email