View
253
Download
0
Category
Preview:
Citation preview
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 1/23
AaDS 2010/2011
Fundamental principles of
algorithms analysis Algorithms
and Data Structures
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 2/23
AaDS 2010/2011
Algorithm definitions
• A step-by-step problem-solving procedure, especially an established,recursive computational procedure for solving a problem in a finite number of steps. (The American Heritage dictionaries)
• An algorithm is a set of precise (i.e., unambiguous) rules that specify how tosolve some problem or perform some task (LINFO).
• Properties of algorithms generally include: input/output (i.e., receives input
and generates output), precision (each step is precisely stated),determinism (the intermediate results of each step of execution are uniqueand are determined only by inputs and results of the previous step),termination (ends after a finite number of instructions execute),correctness (the output generated is correct) and generality (eachalgorithm applies to a set of inputs). (LINFO)
• Informally, an algor i thm is any well-defined computational procedure that
takes some value, or set of values, as i nput and produces some value, or set of values, as output . An algorithm is thus a sequence of computationalsteps that transform the input into the output (Cormen at al.)
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 3/23
AaDS 2010/2011
Algorithms
• Are in every computer program. But also in anynon-computer procedure: – Instruction of tools
– Recipe (for food, for medicine)
– Technology – Calculation in science
• Can be expressed as: – Colloquial language
– Code in known programming language – Pseudocode
– Diagrams
– Mix of above
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 4/23
AaDS 2010/2011
Assumptions
• Problems – decidable , undecidable
– decision problems, computational (function) problems
• Algorithms
– correct : An algorithm is said to be correct if, for every inputinstance, it halts with the correct output. We say that a correctalgorithm solves the given computational problem.
– incorrect : An incorrect algorithm might not halt at all on someinput instances, or it might halt with an answer other than thedesired one. Contrary to what one might expect, incorrectalgorithms can sometimes be useful, if their error rate can becontrolled
– f in i te runt ime
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 5/23
AaDS 2010/2011
Fundamental notions
• elementary operation (step): – algebraic operation (addition, substraction, …)
– comparison (<, <=,…)
– coping, assigment (a=b)
• size of problem (n) – number of input data: – theory: number of bits
– real number: number of numbers to sort
– more useful number: size of matrix
– more than one number: for graph• complexity of problem/algorithm is represented
as a function of n, i.e. f (n)
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 6/23
AaDS 2010/2011
The fact
• The simplest algorithm for solving a
problem is often not the most efficient.
Therefore, when designing an algorithm,
do not settle for just any algorithm thatwork.
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 7/23 AaDS 2010/2011
Improving algorithm – an example
• How to calculate nk ? (k in R)
• Simple – from definition:
Power(n,k)Result=1;
While(k>0)
{
Result=Result*n;
k--; // k=k-1
}return Result;
k multiplications
n0=1
nk=n*n(k-1) for k>=1
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 8/23 AaDS 2010/2011
nk – better algorithm (1)
• n89=n(n44)2, n44=(n22)2, n22=(n11)2,
n11=n(n5)2, n5=n(n2)2, n2=n*n
n2k=(nk)2
n2k+1=n(nk)2
Power(n,k){
if(k==0) return 1;
if(k==1) return n;
result=Power(n,k/2);
if(k%2==0)
return result*result;else
return n*result*result;
}2*log(k) multiplications
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 9/23 AaDS 2010/2011
nk – better algorithm (2)
• n89=n64*n16*n8*n1=(((((n2)2)2)2)2)2)*(((n2)2)2)2
*((n2)2)2*n
101100222289 0346
nnnnnn
Power(n,k)
{ Result=1;
pow=n;
while(k>0)
{
if((k%2)==1)
Result*=pow;pow*=pow;
k/=2;
}
return Result;
}
2*log(k) multiplications
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 10/23
AaDS 2010/2011
Asymptotic Notation – Big Theta
For a given function g(n), we denote by Θ(g(n)) the set of functions f(n) having the property that there exists positive
constants c 1,c 2 , and n0 such that for all n>=n0 ,
c 1g(n) ≤ f(n) ≤ c 2 g(n)
nn0
f(n)
c1g(n)
c2g(n)
f ( x ) has o rder g ( x )
Θ determines
an equivalence
relation
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 11/23
Big Theta - example
• f(x)=x2+100x+1000
• f(x)= Θ (x 2 ) , because:
– for n=1000,c 1=1, c
2 =10 we have for x>=n:
• x2+100x+1000>x2
• x2+100x+1000<10x2
(1.000.000+100.000+1000<10*1.000.000)
AaDS 2010/2011
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 12/23
AaDS 2010/2011
Asymptotic Notation – Big Oh
For a given function g(n), we denote by O(g(n)) the set of functions f(n) having the property that there exists positive
constants c and n0 such that for all n>=n0
f(n) ≤ c∙g (n)
nn0
f(n)
cg(n)
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 13/23
Big Oh - example
• f(x)=x |sin(x)|
• f(x)= O(x) , because:
– for n=1,c=1, we have for x>=n:
• |sin(x)|<=1
• x|sin(x)|<=x
• f(x)=O(x2
)
AaDS 2010/2011
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 14/23
AaDS 2010/2011
Asymptotic Notation – Big Omega
For a given function g(n), we denote by (g(n)) the set of functions f(n) having the property that there exists
positive constants c and n0 such that for all n>=n0 ,
c∙g(n) ≤ f(n)
n
n0
f(n)
cg(n)
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 15/23
AaDS 2010/2011
Dependences
O n( )2
( )n2
( )n2
55 nn nlog
2 3n
6 32n6 2n n n nlog
n2
n n
4 32 n
n n!
Functions that havea smal ler o rder than
n2
Functions that have
a l arger o rder than
n2
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 16/23
AaDS 2010/2011
Complexity of problem/algorithm
• time complexity of a problem is the number of
steps that it takes to solve an instance of the
problem as a function of the size of the input ,
using the most efficient algorithm• space complexity of a problem is a related
concept, that measures the amount of space, or
memory required by the algorithm
• time complexity and space complexity can be
considered for a chosen algorithm.
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 17/23
AaDS 2010/2011
Hierarchy of orders – complexity classes
O( )1
O n(log )
O n( )
O n( )
O n n( log )
O n( )2
O n( )3
O n( )2
O nn( )2
O n( !)
O nn
( )
NP Problems
NP-complete problems
P Problems
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 18/23
AaDS 2010/2011
Relationships
• transitivity
• reflexivity
• symmetry
• transitive symmetry
))(()(implies))(()(i))(()(
))(()(implies))(()(i))(()(
))(()(implies))(()(i))(()(
nhn f nhn g n g n f
nhn f nhn g n g n f
nhn f nhn g n g n f
f n f n
f n f n
f n f n
( ) ( ( ))
( ) ( ( ))
( ) ( ( ))
))(()(if onlyandif ))(()( n f n g n g n f
))(()(if onlyandif ))(()( n f n g n g n f
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 19/23
AaDS 2010/2011
Relationships (cont.)
• Others property
nm = O(nk ), where m ≤ k
O(f (n))+O(g (n)) = O(|f (n)|+|g (n)|)
c ∙O(f (n)) = O(f (n)), where c is constant
O(O(f (n)) = O(f (n))
O(f (n))∙O(g (n)) = O(f (n)∙g (n))
O(f (n)∙g (n)) = f (n)∙O(g (n))
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 20/23
AaDS 2010/2011
Problems classification• P problems
– Computation complexity is at most
polynomial
• sorting
• matrix multiplication
• shortest path
• NP problems – Computation complexity is at least
exponential
• simplex method
• monk’s puzzle problem
• NP-complete problems• boolean satisfiability problem (SAT)
• Hamilton’s cycle
• set division
P
NP
NP-complete
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 21/23
AaDS 2010/2011
Complexity comparisonComplexity
function
Size n
10 20 30 40 50 60
n 0,00000001s 0,00000002s 0,00000003s 0,00000004s 0,00000005s 0,00000006s
n2 0,0000001s 0,0000004s 0,0000009s 0,0000016s 0,0000025s 0,0000036s
n5 0,0001s 0,0032s 0,0243s 0,1024s 0,3125s 0,7776s
n10
10s 2,8h 6,8 days 121,4
days
3,1 years 19,2
years
2n 0,0000001s 0,001s 1s 18,3 min 13 days 36,6
years
3n 0,000006s 3,5s 2,4 days 385
years
2*107
years
1,3*1012
years
n! 0,003s 77 years 8*1015
years
2*1032
years
9*1047
years
2*1065
years
age of universe = 13,7*109 years
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 22/23
AaDS 2010/2011
Complexity comparison (cont.)Complexity
function
Size n
10 100 1000 10000 100000 1000000
n 0,00000001s 0,0000001s 0,000001s 0,00001s 0,0001s 0,001s
nlog2n 0,00000003s 0,0000006s 0,00001s 0,0001s 0,0016s 0,02s
n2 0,0000001s 0,00001s 0,001s 0,1s 10s 16,6min
n3 0,000001s 0,001s 1s 16,6min 11,5 days 31,7
years
7/27/2019 AaDS - W01b Foundations.pdf
http://slidepdf.com/reader/full/aads-w01b-foundationspdf 23/23
AaDS 2010/2011
Complexity comparison (cont.)Complexity
function
Size of the biggest problem solved during 1h
By current computer By computer
100 times faster
By computer
1000 times faster
n N1 100N1 1000N1
n2
N2 10N2 31,6N2
n3
N3 4,64N3 10N3
n5
N4 2,5N4 3,98N4
2n
N5 N5+6,64 N5+9,97
3n N6 N6+4,19 N6+6,29
Recommended