Algorithm Analysis Page 1
Berner Fachhochschule - Technik und Informatik
Algorithms and Data Structures
Algorithm Analysis
Dr. Rolf Haenni
Fall 2008
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 2
Outline
Textbook Overview
Analysis of Algorithm
Pseudo-Code and Primitive Operations
Growth Rate and Big-Oh Notation
Big-Oh in Algorithm Analysis
Relatives of Big-Oh
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 3Textbook Overview
Outline
Textbook Overview
Analysis of Algorithm
Pseudo-Code and Primitive Operations
Growth Rate and Big-Oh Notation
Big-Oh in Algorithm Analysis
Relatives of Big-Oh
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 4Textbook Overview
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 5Textbook Overview
Algorithm Design I
Part I. Fundamental Tools
1. Algorithm Analysis
1.1 Methodologies for Analyzing Algorithms1.2 Asymptotic Notation1.3 A Quick Mathematical Review1.4 Case Studies in Algorithm Analysis1.5 Amortization1.6 Experimentation1.7 Exercises
2. Basic Data Structures
2.1 Stacks and Queues2.2 Vectors, Lists, and Sequences2.3 Trees2.4 Priority Queues and Heaps2.5 Dictionaries and Hash Tables
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 6Textbook Overview
Algorithm Design II
2.6 Java Example: Heap2.7 Exercises
3. Search Trees and Skip Lists
3.1 Ordered Dictionaries and Binary Search Trees3.2 AVL Trees3.3 Bounded-Depth Search Trees3.4 Splay Trees3.5 Skip Lists3.6 Java Example: AVL and Red-Black Trees3.7 Exercises
4. Sorting, Sets, and Selection
4.1 Merge-Sort4.2 The Set Abstract Data Type4.3 Quick-Sort4.4 A Lower Bound on Comparison-Based Sorting4.5 Bucket-Sort and Radix-Sort
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 7Textbook Overview
Algorithm Design III4.6 Comparison of Sorting Algorithms4.7 Selection4.8 Java Example: In-Place Quick-Sort4.9 Exercises
5. Fundamental Techniques
5.1 The Greedy Method5.2 Divide-and-Conquer5.3 Dynamic Programming5.4 Exercises
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 8Textbook Overview
Algorithm Design IV
Part II. Graph Algorithms
1. Graphs
1.1 The Graph Abstract Data Type1.2 Data Structures for Graphs1.3 Graph Traversal1.4 Directed Graphs1.5 Java Example: Depth-First Search1.6 Exercises
2. Weighted Graphs
2.1 Single-Source Shortest Paths2.2 All-Pairs Shortest Paths2.3 Minimum Spanning Trees2.4 Java Example: Dijkstra’s Algorithm2.5 Exercises
3. Network Flow and Matching
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 9Textbook Overview
Algorithm Design V
3.1 Flows and Cuts3.2 Maximum Flow3.3 Maximum Bipartite Matching3.4 Minimum-Cost Flow3.5 Java Example: Minimum-Cost Flow3.6 Exercises
Part III. Internet Algorithmics
1. Text Processing
1.1 Strings and Pattern Matching Algorithms1.2 Tries1.3 Text Compression1.4 Text Similarity Testing1.5 Exercises
2. Number Theory and Cryptography
2.1 Fundamental Algorithms Involving Numbers
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 10Textbook Overview
Algorithm Design VI2.2 Cryptographic Computations2.3 Information Security Algorithms and Protocols2.4 The Fast Fourier Transform2.5 Java Example: FFT2.6 Exercises
3. Network Algorithms
3.1 Complexity Measures and Models3.2 Fundamental Distributed Algorithms3.3 Broadcast and Unicast Routing3.4 Multicast Routing3.5 Exercises
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 11Textbook Overview
Algorithm Design VII
Part IV. Additional Topics
1. Computational Geometry
1.1 Range Trees1.2 Priority Search Trees1.3 Quadtrees and k-D Trees1.4 The Plane Sweep Technique1.5 Convex Hulls1.6 Java Example: Convex Hull1.7 Exercises
2. NP-Completeness
2.1 P and NP2.2 NP-Completeness2.3 Important NP-Complete Problems2.4 Approximation Algorithms2.5 Backtracking and Branch-and-Bound2.6 Exercises
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 12Textbook Overview
Algorithm Design VIII
3. Algorithmic Frameworks
3.1 External-Memory Algorithms
3.2 Parallel Algorithms
3.3 Online Algorithms
3.4 Exercises
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 13Analysis of Algorithm
Outline
Textbook Overview
Analysis of Algorithm
Pseudo-Code and Primitive Operations
Growth Rate and Big-Oh Notation
Big-Oh in Algorithm Analysis
Relatives of Big-Oh
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 14Analysis of Algorithm
Running Time of Algorithms
I An algorithm is a step-by-step procedure for solving a problemin a finite amount of time
I Most algorithms transform input objects into output objects
I The running time of an algorithm typically grows with theinput size
I The average case running time is often difficult to determineI We focus on the worst case running time
Ý Easier to analyseÝ Crucial to applications such as games, finance, robotics
I Sometimes, it is also worth to study the best case runningtime
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 15Analysis of Algorithm
Running Time: Example
0
80
100
Input Size
20
40
60
1000 2000 3000 4000 5000
Runn
ing
Tim
e
Best case
Average case
Worst case
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 16Analysis of Algorithm
Problem Classes I
I There are feasible problems, which can be solved by analgorithm efficiently
Ý Examples: sorting a list, draw a line between two points,decide whether n is prime, etc.
I There are computable problems, which can be solved by analgorithm, but not efficiently
Ý Example: finding the shortest solution for large sliding puzzles
I There are undecidable problems, which can not be solved byan algorithm
Ý Example: for a given a description of a program and an input,decide whether the program finishes running or will run forever(halting problem)
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 17Analysis of Algorithm
Problem Classes II
All Problems
ComputableProblems
FeasibleProblems
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 18Analysis of Algorithm
Experimental Studies
I There are two ways to analyse the running time of analgorithm:
Ý Experimental studiesÝ Theoretical analysis
I Experiments usually involve the following major steps:
Ý Write a program implementing the algorithmÝ Run the program with inputs of varying size and compositionÝ Use a method like System.currentTimeMillis() to get an
accurate measure of the actual running timeÝ Plot the results
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 19Analysis of Algorithm
Experimental Studies: Example
0
1000
5000
8000
10000
2000
3000
4000
6000
7000
9000
Input Size1000 2000 3000 4000 5000
Runn
ing
Tim
e (m
s)
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 20Analysis of Algorithm
Experimental Setup I
To setup an experiment, perform the following steps with care
I Choosing the question
Ý Estimate the asymptotic running time in the average caseÝ Test which of two algorithm is fasterÝ If an algorithm depends on parameters, find the best valuesÝ For approximation algorithms, evaluate the quality of the
approximation
I Deciding what to measure
Ý Running time (system dependent)Ý Number of memory referencesÝ Number of primitive (e.g. arithmetic) operations/comparisons
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 21Analysis of Algorithm
Experimental Setup II
I Generating test data
Ý Enough samples for statistically significant resultsÝ Samples of varying sizesÝ Representative of the kind of data expected in practice
I Coding the solution and performing the experiment
Ý Reproducible resultsÝ Keep note of the details of the computational environment
I Evaluating the test results statistically
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 22Analysis of Algorithm
The Power TestThe power test is a methodology to approximate the growth rateof a polynomial (feasible) algorithm
I Actual running time: t(n) ≈ b·nc , b =???, c =???I Samples: (xi , yi )
Ý xi = problem sizeÝ yi = t(xi ) = actual running time
I Transform xi into x ′i = log xi and yi into y ′
i = log yi
I Plot (x ′i , y
′i ) → Log-Log-Scale
I If possible, draw a line through samples → polynomial
I b = the line’s y -axis intercept, c = slope of the line
I If the growth of the points increases → super-polynomial
I If the plotted points converge to a constant → sublinear
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 23Analysis of Algorithm
The Power Test
0 5 100
1
5
8
10
Polynomialb=2, c=5/6
Sublinear
Super-Polynomial
t(n) = 2n 5/6
1 2 3=1000
4 6=1Mio.
7 8 9=1Mia.
2
1000 = 3
4
1Mio. = 6
7
1Mia. = 9
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 24Analysis of Algorithm
Limits of Experiments
I It is necessary to implement the algorithm, which may bedifficult
I Results may not be indicative of the running time on otherinputs not included in the experiment
I The running times may depend strongly on the chosen setting(hardware, software, programming language)
I In order to compare two algorithms, the same hardware andsoftware environments must be used
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 25Analysis of Algorithm
Theoretical Analysis
I Uses a high-level description of the algorithm instead of animplementation
I Describes the running time as a function of the input size n:
Ý T (n) = number of primitive operations
I Takes into account all possible inputsI Allows to evaluate the speed of an algorithm independent of
Ý hardwareÝ softwareÝ programming languageÝ actual implementation
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 26Pseudo-Code and Primitive Operations
Outline
Textbook Overview
Analysis of Algorithm
Pseudo-Code and Primitive Operations
Growth Rate and Big-Oh Notation
Big-Oh in Algorithm Analysis
Relatives of Big-Oh
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 27Pseudo-Code and Primitive Operations
Pseudo-Code
I Pseudo-code allows a high-level description of an algorithm
I More structured than English prose, but less detailed than aprogram
I Preferred notation for describing algorithms
I Hides program design issues and details of a particularlanguage
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 28Pseudo-Code and Primitive Operations
Pseudo-Code: Example
Find the maximum element of an array:
Algorithm arrayMax(A, n)Input array A of n integersOutput maximum element of AcurrentMax ← A[0]for i ← 1 to n − 1 doif A[i ] > currentMax then
currentMax ← A[i ]return currentMax
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 29Pseudo-Code and Primitive Operations
Details of Pseudo-Code I
I The rules of using pseudo-code are very flexible, but followingsome guidelines is recommended
I Method/function declaration
Ý Algorithm method or function(arg [, arg . . .])Input ...Output ...
I Control flow
Ý if ... then ... [else ...]Ý while ... do ...Ý repeat ... until ...Ý for ... do ...Ý Indentation replaces braces resp. begin ... end
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 30Pseudo-Code and Primitive Operations
Details of Pseudo-Code II
I Expressions
Ý Assignments: ← (like = in Java)Ý Equality testing: = (like == in Java)Ý Superscript like n2 and other mathematical notation is allowedÝ Array access: A[i − 1]
I Method/function call
Ý object.method(arg [, arg . . .])Ý function(arg [, arg . . .])
I Return value
Ý return expression
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 31Pseudo-Code and Primitive Operations
The Random Access Machine Model
I To study algorithms in pseudo-code analytically, we need atheoretical model of a computing machine
I A random access machine consists of
Ý a CPUÝ a bank of memory cells, each of which can hold an arbitrary
number (of any size) or character
I The size of the memory is unlimited
I Memory cells are numbered and accessing any cell in memorytakes 1 unit time (= random access)
I All primitive operations of the CPU take 1 unit time
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 32Pseudo-Code and Primitive Operations
Primitive Operations
I The primitive operations of a random access machine are theonce we allow in a pseudo-code algorithm
Ý Evaluating an expression (arithmetic operations, comparisons)Ý Assigning a value to a variableÝ Indexing into an arrayÝ Calling a method/functionÝ Returning from a method/function
I Largely independent from the programming language
I By inspecting the pseudo-code, we can determine themaximum (or minimum) number of primitive operationsexecuted by an algorithm, as a function of the input size
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 33Pseudo-Code and Primitive Operations
Primitive Operations: Example
Algorithm arrayMax(A, n)currentMax ← A[0]for i ← 1 to n − 1 doif A[i ] > currentMax then
currentMax ← A[i ]increment counter ireturn currentMax
# Operations22 + n2(n − 1)2(n − 1)2(n − 1)1
Hence, the total number of primitive operations of arrayMax isT (n) = 7n − 1.
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 34Growth Rate and Big-Oh Notation
Outline
Textbook Overview
Analysis of Algorithm
Pseudo-Code and Primitive Operations
Growth Rate and Big-Oh Notation
Big-Oh in Algorithm Analysis
Relatives of Big-Oh
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 35Growth Rate and Big-Oh Notation
Estimating Actual Running Times
I In reality, the time to execute a primitive operation differs
Ý Let a be the time taken by the fastest primitive operationÝ Let b be the time taken by the slowest primitive operation
I If t(n) denotes the actual running time of the algorithm on aconcrete machine for a problem of size n, we get
a·T (n) ≤ t(n) ≤ b·T (n)
I For arrayMax , this means that the actual running time
a·(7n − 1) ≤ t(n) ≤ b·(7n − 1)
is bounded by two linear functions
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 36Growth Rate and Big-Oh Notation
Growth Rates of Running Times
I Changing the hardware/ software environment
Ý Affects the constants a and bÝ Therefore, affects t(n) only by a constant factorÝ Does not alter the growth rate of t(n)
I Therefore, we can consider the growth rate of T (n) as beingthe characteristic measure for evaluating an algorithm’srunning time
I We will use the terms “number of primitive operations” and“running time” interchangeably and denote them by T (n)
I The growth rate of T (n) is an intrinsic property of thealgorithm which is independent of its implementation
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 37Growth Rate and Big-Oh Notation
Polynomial Growth Rates
I Most interesting algorithms have a polynomial growth rate
Ý Linear: T (n) ≈ nÝ Quadratic: T (n) ≈ n2
Ý Cubic: T (n) ≈ n3
I In a log-log chart, a polynomial growth rate turns out to be aline (see power test)
I The slope of the line corresponds to the growth rate
I The growth rate is not affected by constant factors orlower-order terms
I Examples
Ý 102n + 10000 is linearÝ 105n2 + 108n is quadratic
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 38Growth Rate and Big-Oh Notation
Polynomial Growth Rate
1
Input Size n101
Runn
ing
Tim
e T
(n)
102 103 104 105 106 107
102
104
106
108
1010
1012
linear
cubic quadratic
1
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 39Growth Rate and Big-Oh Notation
Constant Factors
1
Input Size n101
Runn
ing
Tim
e T
(n)
102 103 104 105 106 107
102
104
106
108
1010
1012
n
1
102n +105
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 40Growth Rate and Big-Oh Notation
Big-Oh Notation
I The so-called big-Oh notation is useful to better compare andclassify the growth rates of different functions T (n)
I Given functions f (n) and g(n), we say that f (n) is O(g(n)), ifthere is a constant c > 0 and an integer n0 ≥ 1 such that
f (n) ≤ c ·g(n), for n ≥ n0
I Example 1: 2n + 10 is O(n)
Ý 2n + 10 ≤ c ·nÝ n ≥ 10/(c − 2), i.e. pick c = 3 and n0 = 10
I Example 2: n2 is not O(n)
Ý n2 ≤ c ·nÝ n ≤ c , which can not be satisfied since c is a constant
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 41Growth Rate and Big-Oh Notation
Big-Oh Example 1
1
Input Size n
Runn
ing
Tim
e T
(n) n
1
2n +10
10 100 1000
10
100
1000
3n
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 42Growth Rate and Big-Oh Notation
Big-Oh Example 2
1
Input Size n
Runn
ing
Tim
e T
(n)
n
1
100n
10 100 1000
10
100
10000
10n
1000n2
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 43Growth Rate and Big-Oh Notation
Big-Oh and Growth Rate
I The big-Oh notation gives an upper bound on the growth rateof a function
I The statement “f (n) is O(g(n))” means that the growth rateof f (n) is no more than the growth rate of g(n)
I We can use the big-Oh notation to order functions accordingto their growth rate
f (n) is O(g(n)) g(n) is O(f (n))
f (n) g(n) Yes No
f (n) g(n) No Yes
f (n) ≡ g(n) Yes Yes
I In other words, O(f (n)) can be seen as an equivalence class offunctions with equal growth rate
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 44Growth Rate and Big-Oh Notation
Big-Oh Rules
I If f (n) is a polynomial of degree d , i.e.
f (n) = a0 + a1n + a2n2 + . . . adnd
then f (n) is O(nd)
Ý drop lower-order termsÝ drop constant factorsÝ e.g. 3 + 2n2 + 5n4 is O(n4)
I Use the smallest possible equivalence class
Ý say ”2n is O(n)” instead of ”2n is O(n2)”Ý say ”3n + 5 is O(n)” instead of ”3n + 5 is O(3n)”
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 45Big-Oh in Algorithm Analysis
Outline
Textbook Overview
Analysis of Algorithm
Pseudo-Code and Primitive Operations
Growth Rate and Big-Oh Notation
Big-Oh in Algorithm Analysis
Relatives of Big-Oh
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 46Big-Oh in Algorithm Analysis
Asymptotic Algorithm Analysis
I The asymptotic analysis of an algorithm determines therunning time in big-Oh notation
I To perform the asymptotic analysis
Ý Find the worst-case number of primitive operations T (n)Ý Express this function with big-Oh notation
I Example: arrayMax
Ý We determine that algorithm arrayMax executes at mostT (n) = 7n − 1 primitive operations
Ý We say that algorithm arrayMax “runs in O(n) time”
I Since constant factors and lower-order terms are eventuallydropped anyhow, we can disregard them when countingprimitive operations
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 47Big-Oh in Algorithm Analysis
Typical Functions in Algorithms
Constant: O(1)
Logarithmic: O(log n)
Linear: O(n)
Quadratic: O(n2)
Polynomial: O(nd), d > 0
Exponential: O(bn), b > 1
Others: O(√
n), O(n log n)
Growth rate order:1 < log n <
√n < n < n log n < n2 < n3 < · · · < 2n < 3n < · · ·
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 48Big-Oh in Algorithm Analysis
Growth Rate Comparison
n log n√
n n n log n n2 n3 2n
2 1 1.4 2 2 4 8 4
4 2 2 4 8 16 64 16
8 3 2.8 8 24 64 512 256
16 4 4 16 64 256 4,096 65,536
32 5 5.7 32 160 1,024 32,768 4.29× 109
64 6 8 64 384 4,096 262,144 1.84× 1019
128 7 11 128 896 16,384 2,097,152 3.40× 1038
256 8 16 256 2,048 65,536 16,777,216 1.15× 1077
512 9 23 512 4,608 262,144 1.34× 108 1.34× 10154
1,024 10 32 1,024 10,240 1,048,576 1.07× 109 1.79× 10308
Atoms in the universe: ca. 1080
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 49Big-Oh in Algorithm Analysis
The Importance of Asymptotics
1 operation/µs 256 ops./µs
Running Time Maximum Problem Size n New Size
400n 2, 500 150, 000 9, 000, 000 256n
20n log n 4, 096 166, 666 7, 826, 087 256n log n7+log n
2n2 707 5, 477 42, 426 16n
n4 31 88 244 4n
2n 19 25 31 n + 8
1 second 1 minute 1 hour
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 50Big-Oh in Algorithm Analysis
Math You Need to Review
I Logarithms/exponentials: a = bc ⇔ logb a = c
Ý a = blogb a
I Properties of logarithms
Ý logb ac = logb a + logb cÝ logb
ac = logb a− logb c
Ý logb ac = c logb a
I Properties of exponentials
Ý ba+c = babc
Ý ba−c = (ba)/(bc)Ý bac = (ba)c = (bc)a
I Base change: logb a = logc alogc b and bc = ac loga b
I Computer science: base b = 2, i.e. log2 a = log a
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 51Relatives of Big-Oh
Outline
Textbook Overview
Analysis of Algorithm
Pseudo-Code and Primitive Operations
Growth Rate and Big-Oh Notation
Big-Oh in Algorithm Analysis
Relatives of Big-Oh
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 52Relatives of Big-Oh
Relatives of Big-Oh I
I Big-Omega: we say that f (n) is Ω(g(n)), if there is aconstant c > 0 and an integer n0 ≥ 1 such that
f (n) ≥ c ·g(n), for n ≥ n0
I Little-Oh: we say that f (n) is o(g(n)), if there is a constantc > 0 and an integer n0 ≥ 1 such that
f (n) < c ·g(n), for n ≥ n0
I Little-Omega: we say that f (n) is ω(g(n)), if there is aconstant c > 0 and an integer n0 ≥ 1 such that
f (n) > c ·g(n), for n ≥ n0
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures
Algorithm Analysis Page 53Relatives of Big-Oh
Relatives of Big-Oh II
I Big-Theta: we say that f (n) is Θ(g(n)), if there are constantsc ′ > 0 and c ′′ > 0 and an integer n0 ≥ 1 such that
c ′·g(n) ≤ f (n) ≤ c ′′·g(n), for n ≥ n0
I The intuition behind the big-Oh relatives are
Ý Big-Oh: f (n) is asymptotically less or equal to g(n)Ý Little-Oh: f (n) is asymptotically strictly less than g(n)Ý Big-Omega: f (n) is asymptotically greater or equal to g(n)Ý Little-Omega: f (n) is asymptotically strictly greater than g(n)Ý Big-Theta: f (n) is asymptotically equal to g(n)
I The big-Theta notation is the most precise description of thegrowth rate, but the big-Oh notation is more commonly used
Berner Fachhochschule Rolf Haenni
Technik und Informatik Algorithms and Data Structures