An Introduction to An Introduction to Programming Programming

Concepts and OI-Concepts and OI-programmingprogramming

……from abstract from abstract theory to dirty theory to dirty

tricks…tricks…

Objectives TodayObjectives Today

Introduction to the concept of Introduction to the concept of “Algorithms”“Algorithms”

Introduction to common algorithms Introduction to common algorithms in OI competitionsin OI competitions

Introduction to graph theoryIntroduction to graph theory Introduction to complexityIntroduction to complexity ““Philosophy” of OI competitionsPhilosophy” of OI competitions OI-style programmingOI-style programming

What is an Algorithm?What is an Algorithm? From Wikipedia: An From Wikipedia: An algorithmalgorithm is a finite set of is a finite set of

well-defined instructions for accomplishing sowell-defined instructions for accomplishing some task which, given an initial state, will termime task which, given an initial state, will terminate in a corresponding recognizable end-state.nate in a corresponding recognizable end-state. (what does that mean?)(what does that mean?)

Usually, an algorithm solves a “problem”.Usually, an algorithm solves a “problem”. ExamplesExamples

Insertion sortInsertion sort Binary SearchBinary Search An algorithm does not have to be a computer progrAn algorithm does not have to be a computer progr

am! Think about other possible algorithms in real liam! Think about other possible algorithms in real lifefe

““Problem”sProblem”s Usually a set of well defined inputs Usually a set of well defined inputs

and corresponding outputsand corresponding outputs Example: the sorting problem:Example: the sorting problem:

Input: a list of numbersInput: a list of numbers Output: a sorted list of numbersOutput: a sorted list of numbers We can use a number of We can use a number of differentdifferent

sorting algorithms to solve the sorting sorting algorithms to solve the sorting problemproblem

Data StructuresData Structures Supplementary objects that help store Supplementary objects that help store

data in an algorithmdata in an algorithm Different data structures have different Different data structures have different

properties, and can store different types properties, and can store different types of data, and access them in different of data, and access them in different waysways

Selecting the right data structure can be Selecting the right data structure can be very important, as you will learn latervery important, as you will learn later

Examples: arrays, queues, stacks… more Examples: arrays, queues, stacks… more will be introduced laterwill be introduced later

Examples of algorithmsExamples of algorithms

Sorting algorithmsSorting algorithms Graph algorithms – Djikstra, Warshall-Graph algorithms – Djikstra, Warshall-

floyd, Bellman-Ford, Prims, Kruskalfloyd, Bellman-Ford, Prims, Kruskal Tree-Search algorithms – BFS, DFSTree-Search algorithms – BFS, DFS Linear Searching AlgorithmsLinear Searching Algorithms

Examples of Data Examples of Data StructuresStructures

Array – random accessArray – random access Queue – First in First OutQueue – First in First Out Stack – First in Last OutStack – First in Last Out Heap – extract min/max numberHeap – extract min/max number Binary Search Tree – Efficient insert, Binary Search Tree – Efficient insert,

search, delete, etc.search, delete, etc. Hash Table – fast lookupHash Table – fast lookup Other graph data structures Other graph data structures

discussed belowdiscussed below

Examples of Techniques in Examples of Techniques in Designing AlgorithmsDesigning Algorithms

RecursionRecursion Dynamic programmingDynamic programming GreedyGreedy Divide and conquerDivide and conquer Branch and boundBranch and bound (the above may have overlaps)(the above may have overlaps)

Using and Creating Using and Creating AlgorithmsAlgorithms

““It is science. You can derive them.”It is science. You can derive them.”“It is art. We have no way to teach you!”“It is art. We have no way to teach you!”– Alan Tam– Alan Tam

Why study algorithms?Why study algorithms? To solve problems that can be directly To solve problems that can be directly

solved by existing algorithmssolved by existing algorithms To solve problems that can be solved by To solve problems that can be solved by

combining algorithmscombining algorithms To get feelings and inspirations on how to To get feelings and inspirations on how to

design new algorithmsdesign new algorithms

Related IssuesRelated Issues

Proving correctness of algorithms (whProving correctness of algorithms (why? why not?)y? why not?)

Other methods: finding counter examOther methods: finding counter examples, “Unu’s Conjecture of Competitples, “Unu’s Conjecture of Competition”ion”

Questions?Questions?

GraphsGraphs

What is a graph?What is a graph?

Informally, a set of relationships betweeInformally, a set of relationships between thingsn things

A A graphgraph is defined as G=(V,E), where is defined as G=(V,E), where V is the set of vertices (singular: vertex)V is the set of vertices (singular: vertex) E is the set of edges that connect some of the E is the set of edges that connect some of the

verticesvertices A A pathpath is a sequence of vertices which ar is a sequence of vertices which ar

e connected by edge(s)e connected by edge(s)

ExampleExample

Map of AustraliaMap of Australia

WA

NT

SA

Q

NSW

V

T

Common Types of GraphsCommon Types of Graphs

Directed/Undirected GraphDirected/Undirected Graph Weighted/Unweighted GraphWeighted/Unweighted Graph ConnectivityConnectivity

WA

NT

SA

Q

NSW

V

T

TreesTrees

A few common definitions (equivalenA few common definitions (equivalent):t): Connected graph with no cyclesConnected graph with no cycles There is a unique path between any two vThere is a unique path between any two v

ertices ertices Connected graph with v – 1 edges (v = nuConnected graph with v – 1 edges (v = nu

m of vertices)m of vertices) Rooted/Unrooted TreesRooted/Unrooted Trees

Heap, Binary Search TreesHeap, Binary Search Trees

Representing a graphRepresenting a graph

Adjacency MatrixAdjacency Matrix Adjacency ListAdjacency List

ComplexityComplexity

What is complexity?What is complexity? We are not (yet!) concerned with the We are not (yet!) concerned with the

exact runtime or memory usedexact runtime or memory used We want to know how well an We want to know how well an

algorithm “scales up” (i.e. when algorithm “scales up” (i.e. when there is a large input). Why?there is a large input). Why?

Complexity (cont’d)Complexity (cont’d) Here’s why:Here’s why:

0

500

1000

1500

2000

2500

f(n) = 10n f(n) = 30n

f(n) = n^2 f(n) = n^3

Quasi-Formal Definition of Quasi-Formal Definition of Big-OBig-O

(you need not remember these)(you need not remember these)We sayWe say f(x) is in O(g(x))f(x) is in O(g(x))if and only ifif and only if there exist numbers xthere exist numbers x00 and M such t and M such that hat

|f(x)| ≤ M |g(x)| for x > x|f(x)| ≤ M |g(x)| for x > x00

Example 1 – Bubble sortExample 1 – Bubble sort

For i := 1 to n doFor i := 1 to n doFor j := i downto 2 doFor j := i downto 2 do

if a[j] > a[j-1] then swap(a[j], aif a[j] > a[j-1] then swap(a[j], a[j-1]);[j-1]);

Time Complexity? O(nTime Complexity? O(n22)) ““Swap Complexity”?Swap Complexity”? How about memory?How about memory?

Example 2 – Insertion Example 2 – Insertion SortSort

Quick introduction to insertion sort Quick introduction to insertion sort (you will learn more in the searching (you will learn more in the searching and sorting training):and sorting training): [] 4 3 1 5 2[] 4 3 1 5 2 [4] 3 1 5 2[4] 3 1 5 2 [3 4] 1 5 2[3 4] 1 5 2 [1 3 4] 5 2[1 3 4] 5 2 [1 3 4 5] 2[1 3 4 5] 2 [1 2 3 4 5][1 2 3 4 5]

Time Complexity = ?Time Complexity = ?

ApplicationsApplications Usually, the time complexity of the algorithm gives us Usually, the time complexity of the algorithm gives us

a rough estimation of the actual run time.a rough estimation of the actual run time. O(n) for very large NO(n) for very large N O(nO(n22) for n ~ 1000-3000) for n ~ 1000-3000 O(nO(n33) for n ~ 100-200) for n ~ 100-200 O(nO(n44) for n ~ 50) for n ~ 50 O(kO(knn) for O(n!) for very small n, usually < 20) for O(n!) for very small n, usually < 20 Keep in mindKeep in mind

The constant of the algorithms (including the implementatioThe constant of the algorithms (including the implementation)n)

Computers vary in speeds, so the time needed will be differentComputers vary in speeds, so the time needed will be different Therefore remember to test the program/computer before maTherefore remember to test the program/computer before ma

king assumptions!king assumptions!

ProblemProblem

I have implemented bubble sort for an I have implemented bubble sort for an Array A[N] and applied binary search oArray A[N] and applied binary search on it.n it. Time complexity of bubble sort?Time complexity of bubble sort?

O(NO(N22). No doubt.). No doubt. Time complexity of binary search?Time complexity of binary search?

O(O(lg lg N)N) Well, what is the time complexity of my alWell, what is the time complexity of my al

gorithm?gorithm?

PropertiesProperties

O(f) + O(g) = max(O(f), O(g))O(f) + O(g) = max(O(f), O(g)) O(f) * O(g) = O(fg)O(f) * O(g) = O(fg) So, what is the answer regarding to preSo, what is the answer regarding to pre

vious question?vious question?

Some other notations Some other notations (optional)(optional)

f(N) is f(N) is ΘΘ(g(N))(g(N)) iff f(N) is O(g(N)) and g(N) is O(f(N))iff f(N) is O(g(N)) and g(N) is O(f(N))

f(N) is o(g(N))f(N) is o(g(N)) For all C, there exists NFor all C, there exists N0 0 such thatsuch that

|f(N)| < C|g(N)| for all N > N|f(N)| < C|g(N)| for all N > N00 f(N) is f(N) is ΩΩ(g(N))(g(N))

iff g(N) is O(f(N))iff g(N) is O(f(N))

Again no need to remember themAgain no need to remember them

Computational Theory Computational Theory TopicsTopics

P (Polynomical)P (Polynomical) Can be solved in polynomical timeCan be solved in polynomical time

NP (Non-deterministic Polynomical)NP (Non-deterministic Polynomical) Can be checked in polynomial timeCan be checked in polynomial time NP does NOT stand for “not-polynomialNP does NOT stand for “not-polynomial

”!!”!! NP-CompleteNP-Complete

The “hardest” NP problemsThe “hardest” NP problems

““Philosophy” of OI Philosophy” of OI CompetitionsCompetitions

Objective of Competition…Objective of Competition… The winner is determined by:The winner is determined by:

Fastest Program?Fastest Program? Amount of time used in coding?Amount of time used in coding? Number of Tasks Solved?Number of Tasks Solved? Use of the most difficult algorithm?Use of the most difficult algorithm? Highest ScoreHighest Score

Therefore, during a competition, aim to Therefore, during a competition, aim to get highest score, at all costs – “All is fair get highest score, at all costs – “All is fair in love and war.”in love and war.”

ScoringScoring

A “black box” judging systemA “black box” judging system Test data is fed into the programTest data is fed into the program Output is checked for correctnessOutput is checked for correctness No source code is manually No source code is manually

inspectedinspected How to take advantage (without How to take advantage (without

cheating of course!) of the system?cheating of course!) of the system?

The OI Programming The OI Programming ProcessProcess

Reading the problemsReading the problems Choosing a problemChoosing a problem Reading the problemReading the problem ThinkingThinking CodingCoding TestingTesting Finalizing the programFinalizing the program

Reading the ProblemReading the Problem

Usually, a task consists ofUsually, a task consists of TitleTitle Problem DescriptionProblem Description ConstraintsConstraints Input/Output SpecificationInput/Output Specification Sample Input/OutputSample Input/Output ScoringScoring

Reading the ProblemReading the Problem

ConstraintsConstraints Range of variablesRange of variables Execution TimeExecution Time

NEVERNEVER make assumptions yourself make assumptions yourself Ask whenever you are not sureAsk whenever you are not sure (Do not be afraid to ask questions!)(Do not be afraid to ask questions!)

Read Read everyevery word carefully word carefully Make sure you understand before Make sure you understand before

going ongoing on

ThinkingThinking Classify the problemClassify the problem

Graph? Mathematics? Data Processing? Dynamic Graph? Mathematics? Data Processing? Dynamic Programming? etc….Programming? etc….

Some complicated problems may be a Some complicated problems may be a combination of the abovecombination of the above

Draw diagrams, use rough work, scribble…Draw diagrams, use rough work, scribble… Consider special cases (smallest, largest, etc)Consider special cases (smallest, largest, etc) Is the problem too simple?Is the problem too simple?

Usually the problem setters have something they Usually the problem setters have something they want to test the contestants, maybe an algorithm, want to test the contestants, maybe an algorithm, some specific observations, carefulness etc.some specific observations, carefulness etc.

Still no idea? Give up. Time is precious.Still no idea? Give up. Time is precious.

Designing the SolutionDesigning the Solution

Remember, before coding, you MUST have Remember, before coding, you MUST have an idea what you are doing. If you don’t know an idea what you are doing. If you don’t know what you are doing, do not begin coding.what you are doing, do not begin coding.

Some points to consider:Some points to consider: Execution time (Time complexity)Execution time (Time complexity) Memory usage (Space complexity)Memory usage (Space complexity) Difficulty in codingDifficulty in coding

Remember, during competition, use the Remember, during competition, use the algorithm that gains you most score, not the algorithm that gains you most score, not the fastest/hardest algorithm!fastest/hardest algorithm!

CodingCoding

Optimized for ease of coding, not for readingOptimized for ease of coding, not for reading Ignore all the “coding practices” outside, unless you Ignore all the “coding practices” outside, unless you

find them particularly useful in OI competitionsfind them particularly useful in OI competitions No Comments neededNo Comments needed Short variable namesShort variable names Use less functionsUse less functions NEVER use 16 bit integers (unless memory is NEVER use 16 bit integers (unless memory is

limited)limited) 16 bit integer may be slower! (PC’s are usually 32-16 bit integer may be slower! (PC’s are usually 32-

bit, even 64 bit architectures should be somewhat-bit, even 64 bit architectures should be somewhat-optimized for 32 bit)optimized for 32 bit)

Coding (2)Coding (2)

Use Use gotogoto, , breakbreak, etc in the , etc in the appropriate situationsappropriate situations Never mind what Djikstra has to say Never mind what Djikstra has to say

Avoid using floating point variables Avoid using floating point variables if possible (eg. real, double, etc)if possible (eg. real, double, etc)

Do not do small (aka useless) Do not do small (aka useless) “optimizations” to your code“optimizations” to your code

Save and compile frequentlySave and compile frequently See example program code…See example program code…

TestingTesting Sample Input/OutputSample Input/Output

“A problem has sample output for two “A problem has sample output for two reasons:reasons:1.1. To make you understand what the correct output To make you understand what the correct output

format isformat is2.2. To make you believe that your incorrect solution To make you believe that your incorrect solution

has solved the problem correctly has solved the problem correctly ”” Manual Test DataManual Test Data Generated Test Data (if time allows)Generated Test Data (if time allows) Boundary Cases (0, 1, other smallest cases)Boundary Cases (0, 1, other smallest cases) Large Cases (to check for TLE, overflows, etc)Large Cases (to check for TLE, overflows, etc) Tricky CasesTricky Cases

DebuggingDebugging

Debugging – find out the bug, and remDebugging – find out the bug, and remove itove it

Easiest method: writeln/printf/coutEasiest method: writeln/printf/cout It is so-called “Debug message”It is so-called “Debug message”

Use of debuggers:Use of debuggers: FreePascal IDE debuggerFreePascal IDE debugger gdb debuggergdb debugger

FinalizingFinalizing Check output formatCheck output format

Any trailing spaces? Missing end-of-lines? (for priAny trailing spaces? Missing end-of-lines? (for printf users, this is quite common)ntf users, this is quite common)

better test once more with sample outputbetter test once more with sample output Remember to clear those debug messagesRemember to clear those debug messages

Check I/O – filename? stdio?Check I/O – filename? stdio? Check exe/source file nameCheck exe/source file name Is the executable updated?Is the executable updated? Method of submission?Method of submission? Try to allocate ~5 mins at the end of competiTry to allocate ~5 mins at the end of competi

tion for finalizingtion for finalizing

Interactive TasksInteractive Tasks

Traditional TasksTraditional Tasks Give input in one goGive input in one go Give output in one goGive output in one go

Interactive TasksInteractive Tasks Your program is given some inputYour program is given some input Your program gives some outputYour program gives some output Your program is given some more inputYour program is given some more input Your program gives more outputYour program gives more output ……etcetc

ExampleExample ““Guess the number”Guess the number” Sample Run:Sample Run:

Judge: I have a number between 1 and 5, can you Judge: I have a number between 1 and 5, can you guess?guess?

Program: is it 1?Program: is it 1? J: Too smallJ: Too small P: 2?P: 2? J: Too smallJ: Too small P: 3?P: 3? J: Too smallJ: Too small P: 4?P: 4? J: CorrectJ: Correct P: 5?P: 5? J: Too bigJ: Too big P: Your number is 4!P: Your number is 4!

Open Test DataOpen Test Data Test data is knownTest data is known Usually quite difficult to solveUsually quite difficult to solve Some need time consuming algorithms, therefore you Some need time consuming algorithms, therefore you

are given a few hours (i.e. competition time) to run thare given a few hours (i.e. competition time) to run the programe program

Tricks:Tricks: ALWAYS look at all the test data firstALWAYS look at all the test data first Solve by hand, manuallySolve by hand, manually Solve partially by program, partially by handSolve partially by program, partially by hand Some with different programsSome with different programs Solve all with one program (sometimes impossible!)Solve all with one program (sometimes impossible!) Make good use of existing tools – you do not have to write all Make good use of existing tools – you do not have to write all

the programs if some are already available! (eg. sort, other lathe programs if some are already available! (eg. sort, other languages, etc)nguages, etc)

TricksTricks ““No solution”No solution” Solve for simple casesSolve for simple cases

50%50% Special cases (smallest, largest, etc)Special cases (smallest, largest, etc) Incorrect greedy algorithmsIncorrect greedy algorithms

Hard CodeHard Code Stupid Hardcode: begin writeln(random(100)); end.Stupid Hardcode: begin writeln(random(100)); end. Naïve hardcode: “if input is x, output hc(x)”Naïve hardcode: “if input is x, output hc(x)” More “intelligent” hardcode (sometimes not possible): prMore “intelligent” hardcode (sometimes not possible): pr

e-compute the values, and only save some of theme-compute the values, and only save some of them Brute forceBrute force Other Weird Tricks (not always useful…)Other Weird Tricks (not always useful…)

Do nothing (e.g.. Toggle, IODM)Do nothing (e.g.. Toggle, IODM)

Pitfalls / Common Pitfalls / Common MistakesMistakes

Misunderstanding the problemMisunderstanding the problem Not familiar with competition Not familiar with competition

environmentenvironment Output formatOutput format Using complex algorithms Using complex algorithms

unnecessarilyunnecessarily Choosing the hardest problem firstChoosing the hardest problem first

Advertisement (targeted Advertisement (targeted ad)ad)

NOI/IOI use Linux as competition environmNOI/IOI use Linux as competition environment exclusivelyent exclusively

We are thinking of providing Linux only envWe are thinking of providing Linux only environments for upcoming team formation tesironments for upcoming team formation test(s)t(s)

Linux, when used properly, can be more poLinux, when used properly, can be more powerful than Microsoft Windows werful than Microsoft Windows TMTM for contes for contests, because it has more powerful toolsts, because it has more powerful tools

Eg. Command Line tools, Powerful Editors Eg. Command Line tools, Powerful Editors (vim, emacs), etc.(vim, emacs), etc.

The EndThe End

Note: most of the contents are introductions oNote: most of the contents are introductions only. You may want to find more in-depth matenly. You may want to find more in-depth materialsrials Books – Introduction to AlgorithmsBooks – Introduction to Algorithms Online – Google, WikipediaOnline – Google, Wikipedia HKOI – Newsgroup, training websites of previous yHKOI – Newsgroup, training websites of previous y

ears, discuss with trainers/trainees.ears, discuss with trainers/trainees. Training – Many topics are further covered in later Training – Many topics are further covered in later

trainingstrainings Experience!Experience!

Any Questions?Any Questions?