CS242 Data Structures II

Algorithm Analysis1Joe Meehean

Efficiency of an AlgorithmAlgorithmset of simple instructions to solve a problemefficiency determined by resource usageResource UsageCPUmemorydisknetwork

Runtime ComplexityMeasures how CPU usage scales with problem sizeMeasured in basic operationsarithmetic operation (+,-,etc)assignment (a = 7)condition test (a == 0)write a single primitive

Runtime ComplexityConstant Timecomplexity groupnumber of operations independent of problem sizee.g., vector::size()takes the same amount of operations regardless of the vectors sizeone primitive write (return value)Runtime ComplexityLinear time# of operations directly proportional to problem sizee.g., sum all numbers in a listwe need tocreate a sum variableadd each of the numbers to the sumExamplesConstantget any single item from an array (e.g., array[7])add together two integersreturn a member variable of a classLinearfind the largest number in a listfind the average number in a listcopy all items from one array to another6Runtime ComplexityQuadratic time# of operations increase faster than the problem sizefind matching items between 2 arraysarrays are the same size

int matches = 0;for(int i = 0; i < array1_size; i++){ for(int j = 0; j < array2_size; j++){ if( array1[i] == array2[j] ){ matches++}}}ExampleAssume array1 and array2 have N items2 operations inside inner loopN times for inner loopN times outer loop2 * N * N = 2N2

int matches = 0;for(int i = 0; i < array1_size; i++){ for(int j = 0; j < array2_size; j++){ if( array1[i] == array2[j] ){ matches++}}}Big O NotationConstant-Time: O(1)Order 1Linear-Time: O(N)Order NQuadratic-Time: O(N2)Ignore constants or low order terms4n + 1 => O(n)4n + n2 => O(n2)Mathematic DefinitionRead about it in bookImportant if you dont understand Big O at the end of this lectureRuntime Complexity CasesDiscussed using three casesBest-caseleast number of operations possibleAverage-casemost likely number of operationsWorst-casemost operations that could occurusually care about the worst-case

Determining ComplexitySequence of statementssum each statements complexitysimple statements => O(1)if-then-elsewhat is the worst case?slowest of all possible pathsDetermining ComplexityLoopsmultiply statement block complexity by # of times loop executesif loop statement-block => O(1) and loop executes N-timesthen complexity => O(N)

Determining ComplexityIndependent nested loopsN & M are not dependent on each otherinner loop: O(M)outer loop: O(N)O(N*M), if N == M, then O(N2)for(int i = 0; i < N; i++){ for(int j = 0; j < M; j++){ //simple sequential statements }}Determining ComplexityDependent nested loops# of times inner loop executes dependent on outer loop

for(int i = 0; i < N; i++){ for(int j = i; j < N; j++){ //simple sequential statements }}Determining ComplexityInner loop executesi = 0Ni = 1N 1i = N 11(N + 1) * N / 2 => O(N2)for(int i = 0; i < N; i++){ for(int j = i; j < N; j++){ //simple sequential statements }}Determining ComplexityMethod call statementscount as complexity of the methods operationsWhat does all this mean?2 algorithms with same Big O may not run at same speedrecall we drop constants and low ordersf(N) = 4N + 5N2 => O(N2)g(N) = 2 + N2 => O(N2)g(N) is fasterBoth scale at the same rateif N doubles, the run time for g and f quadruplesWhat does all this mean?For sufficiently large N, andh(N) => O(N2)k(N) => O(N)k(N) will always be fasternot always true for small NWhat does all this mean?

What does all this mean?TimeBig OIf N DoublesConstantO(1)No changeLogarithmicO(logN)Increases by 1LinearO(N)DoublesQuadraticO(N2)QuadruplesWhat does all this mean?Sometimes the best or average case is more importantAll depends on the intended use of an algorithmIf you will only executethe best-case code in algo A and the worst-case in algo B, you must compare best A to worst BQuestions?23