TSP using Genetic Algorithms in MATLAB

Solving TSP using Genetic Algorithms

Mateusz Matyjaszczyk - 140293481

Abstract

In the Travelling Salesman Problem (TSP), we wish to visit n cities inthe shortest possible time (or distance), without ever visiting any city twice.This problem has a time complexity of O(n!) so an exhaustive search of everysolution is infeasible for large n. An alternative to this is the Simple GeneticAlgorithm (SGA) which mimics the behaviour of natural selection to find thebest route. This report introduces such an algorithm, and then applies it tothe TSP using MATLAB. An investigation is made into the behaviour of thealgorithm under various parameters.

1 Introduction

1.1 Search Problems and Travelling Salesman Problem (TSP)

Suppose we have a collection of items, with each item being slightly different thanthe other items in the collection. We wish to find an item in this collection withspecific characteristics. This is the simplest definition of a search problem. To findthis item, we could start by checking each item until we find the item we are lookingfor. This is called an exhaustive search. Although such an approach is successful ifthe collection in question is very small, it quickly breaks down for large collectionsas the time taken to find the item grows drastically.

One famous example of a search problem is the Travelling Salesman Problem(TSP). Suppose we are given a number of cities (nodes) and the distances betweenthese cities (edges) such that every city is connected with all the other cities. Wewish to compute the shortest distance in which we can travel to every city, withoutvisiting any city twice. There are n cities we wish to visit. Thus, when we choosea starting point we have n− 1 cities to choose from for the next destination. Then,there are n− 2 edges to choose from the next city and so on. Therefore, the numberof possible routes is

1

Solving TSP using Genetic Algorithms Mateusz Matyjaszczyk (140293481)

N = (n− 1)(n− 2)... = (n− 1)!,

where N is the number of possible routes. However, due to this being an undi-rected graph, reverse paths have exactly the same total distance. That is, the route{1, 2, 3, 4} is the same as the route {4, 3, 2, 1}. So, the number of unique routes isgiven by

N =(n− 1)!

2.

Consider a small number of nodes, say n = 6. Then, there are N = 5!2

= 1202

= 60different routes. We could easily use exhaustive search to find the value of every routeto obtain the shortest route. Next, consider a problem with n = 16 nodes. Then,there are N = 15!

2= 653, 837, 184, 000 unique routes. Suppose we have a computer

that can examine 100, 000 routes per second. To visit every route, it would take usover 75 days, even at such a high speed of calculations. The time increase is evenmore concerning for higher values of n.

In this project, it will be investigated how search problems with such large searchspaces can be tackled using Genetic Algorithms.

1.2 Overview of Genetic Algorithms

In the 1970s, Holland (1992) proposed an evolutionary algorithm to tackle searchproblems, called the Simple Genetic Algorithm (SGA). This algorithm imitates be-haviour seen in nature during natural selection. In short, the algorithm has thefollowing procedure:

1. Represent some random solutions as chromosomes. Evaluate the fitness of thechromosomes (how close the solutions are to the goal state).

2. Select fittest chromosomes, called the parent chromosomes.

3. Cross the parent chromosomes using some well defined process to obtain thechildren. Allow random mutations to occur.

4. Repeat the above steps until a given condition is met (e.g. a certain numberof repetitions, solution good enough, etc.)

Each of the above steps will now be discussed and some common techniques usedin Genetic Algorithms will be shown.

2


2 Simple Genetic Algorithm (SGA)

2.1 Representation

Before any computation can take place, the solutions to the search problem need tobe expressed as chromosomes, that are made up of genes. In such a representation,the user should easily move between solutions (states) by only performing a smallchange to the genes. For example, if the solution to a given search problem is aninteger, we can express it as a binary number. For example, the solution 105 couldbe written as:Note: In

a binaryrepre-senta-tion thebits/genesarecountedfrom theright,startingat 0

1 1 0 1 0 0 1

We can easily move between solutions by flipping one bit. So, by flipping the bit0, we could go to the state:

1 1 0 1 0 0 0

(Michalewicz and Fogel, 2004, p.35) propose an alternative representation for theTSP. Suppose we number our cities 1, . . . , n. We can express a route by a permutationof these values. Thus, for a problem with n = 7 cities, this could be:

4 7 6 1 5 2 1

Suppose we wish to find a similar solution. We could simply increment a singlegene by 1. For example changing bit 0 produces the following solution:

4 7 6 1 5 2 2

However, this solution is illegal since we visit the city labeled "2" twice whichviolates the problem description. A alternative would be to swap two consecutivebits/genes. For example, swapping the two right-most genes produces the followingsolution:

4 7 6 1 5 1 2

Typically, a genetic algorithm has a large number of chromosomes. Such a col-lection of chromosomes is called a population while a population at a given timeis called a generation. A large population is important for the later stages of thealgorithm, but it also increases the number of calculations, and thus decreases thespeed of the algorithm.

3


2.2 Fitness Function

We need a way of distinguishing between how well the solutions perform, which ingenetic algorithms is called the fitness of the chromosome. A fitness function isused to measure this. (Mitchell, 1996, p. 7-8) used a toy example of maximising afunction to illustrate the role of a fitness function. Suppose we have a quadraticfitness function

−x2 + 14x + 17, (1)

for which we wish to find the maximum. We have a binary representation with 5bits, similar to the representation seen in Section 2.1. Thus, we can evaluate thefitness using the above function for a small population of 4 chromosomes.

0 0 1 0 1 Value = 5 Fitness = 62

0 0 1 0 1 Value = 7 Fitness = 66

0 1 0 0 1 Value = 9 Fitness = 62

0 1 1 0 0 Value = 12 Fitness = 41

From this, we can see that the second chromosome is the fittest in this populationsince we wish to obtain the maximum. Next, consider an evaluation function for theTSP. Suppose we have the chromosome:

4 7 6 1 5 1 2

We are able to compute the distance between each consecutive city e.g. 4 and 7then 7 and 6 and so on. Thus, the fitness of a route in TSP can be simply calculatedas the sum of those distances.

2.3 Selection

2.3.1 Roulette-Wheel Selection

Once we are able to assess the fitness of a given population, we can select the fittestchromosomes to which we can later apply some operators. The main intuition behindthis step is that by choosing the fittest chromosomes as parents, we will also producefit children using crossover later on. This idea is known as the Buliding BlocksHypothesis.

4


Roulette-Wheel Selection (RWS) is one of the most basic algorithms for selec-tion. In this method, each individual chromosome has a probability to be selectedproportional its fitness. Jebardi and Madiafi (2013) state the following procedure forusing RWS:

• Let the chromosomes in the population be i = 1, . . . , n. Evaluate the fitness ofeach chromosome using the fitness function f(i).

• Calculate the total fitness, denoted by S = Σni=1f(i)

• For each chromosome, work out their proportional fitness using k(i) = f(i)S

• Choose each chromosome i with the probability k(i).

Consider the example population seen in Section 2.2. Such a population hadfour chromosomes {i1, ..., i4} with the corresponding fitness {62, 66, 62, 41}. Thetotal fitness of this population is S = 231. Thus, the probability of choosing eachchromosome is k = {0.27, 0.29, 0.27, 0.18} for each respective value of i. This canthen be expressed as intervals, such as 0, ..., 0.27, 0.27, ..., (0.27+0.29) and so on. Wecan then select a parent by generating a random number from the standard uniformdistribution and choosing the interval into which this random number falls into.

Jebardi and Madiafi (2013) suggest that this method might be unsuitable whena dominant chromosome is present in the population that is much fitter than otherchromosomes. In such a case, this chromosome will always be selected and this couldlead to premature convergence of the algorithm.

2.3.2 Tournament Selection

Tournament selection (TOS) is an alternative to RWS. This scheme involves choosinga number of chromosomes and then performing a "tournament". The method canbe summarised using the following procedure:

1. Choose k chromosomes for the tournament. Order the chromosomes in de-scending order. Let n = 0.

2. Let 0.5 ≤ p ≤ 0.9. Choose the most fit chromosome with probability p(1−p)n.

3. If no chromosome has been chosen, continue. Delete the most fit chromosomefrom the tournament. Increment n.

4. Run the above two steps until a chromosome has been chosen.

5


Selection pressure is the degree to which fitter chromosomes are favoured to bechosen. The more likely the fitter chromosomes are to be chosen, the higher theselection pressure. Since the convergence of the algorithm is largely dependent onselection pressure, Miller and Goldberg (1995) discuss that too little selection pres-sure could lead to the GA not converging, or taking unnecessarily long to converge.On the other hand, excessive selection pressure could lead to premature convergence,meaning that optimal solution is not found as the algorithm is stuck in a local op-tima. In addition, Miller and Goldberg (1995) suggest that for tournament selectionthe selection pressure can be altered by a proper choice of k (size of the tournament).Selection pressure can also be altered by the choice of p. When p is high, the fitterchromosomes are more likely to be chosen, which also increases selection pressure.

Jebardi and Madiafi (2013) has shown that GAs which implement TOS have abetter convergence rate than ones that use RWS. There exist many other selectionschemes such as truncation selection and rank selection. The choice of the scheme isoften determined by the problem to be solved and the selection pressure associated.

2.4 Crossover

2.4.1 One-point Crossover (1PX)

As outlined in Section 1.2, once some parent chromosomes have been selected, acrossover operation needs to applied to produce children (or offspring) chromosomes.The most basic type of such an operation is the one-point crossover (1PX).

Suppose we have two binary chromosomes. We choose a single crossover point,say between bit 1 and 2.Note: A

crossoverpoint isshownby adoublelinebetweencon-secutivebits/genes.

Parent 1 1 0 1 0 0

Parent 2 0 1 0 1 0

Let the three bits to the left of the crossover points stay the same. The bits tothe right of the crossover point are swapped between parents. That is, bits 0 and 1of parent 1 become the bits 0 and 1 of parent 2 and vice versa. Thus, we create thefollowing children:

Child 1 1 0 1 1 0

Child 2 0 1 0 1 0

6


(Sivanandam and Deepa, 2007, p. 51) suggest that using 1PX might not be ableto pass desired characteristics to the children. This can occur when the good infor-mation is stored at either end of the parent chromosomes. On the contrary, due tothe simplicity of 1PX, it also reduces the run time of the GA.

2.4.2 Two-point Crossover (2PX)

To overcome the problem of desired characteristics not being passed on from theparents to the children, an alternative could be to use a two-point crossover (2PX).

In such a crossover operator we define two crossover points. The children inheritthe head and the tail of the chromosome from the corresponding parent (parent 1to child 1 and parent 2 to child 2). The information between the crossover points isswapped between parents, that is child 1 obtains the information from parent 2 andvice versa.

Next, illustrate this using a simple example. Suppose we have the same pairof parent chromosomes as seen in Section 2.4.1 but the crossover points are nowbetween genes 0 and 1 and between genes 2 and 3:

Parent 1 1 0 1 0 0

Parent 2 0 1 0 1 0

Perform the crossover operation by copying the parents and then swapping thebits between the two crossover points defined above. Rename the new chromosomesas children.

Child 1 1 0 0 1 0

Child 2 0 1 1 0 0

Next, obtain the fitness of the children using (1). The fitness of the parent 1and parent 2 can be shown to be −103 and 57 respectively. Similarly, the fitnessof child 1 and child 2 is −55 and 41 respectively. Thus, performing the crossoveroperator creates children that are genetically different to their parents, althoughsome characteristics of the parents are preserved.

2.4.3 Partially-Matched Crossover (PMX)

Next, consider an example of a chromosome representation that is a permutation.Two parents have been selected and a 2PX operator is to be applied. The crossover

7


points are between genes 0 and 1 and between genes 2 and 3.

Parent 1 2 3 4 5 1

Parent 1 4 5 2 1 3

⇓

Child 1 2 3 2 1 1

Child 2 4 5 4 5 3

Performing such an operator creates children that are no longer permutations.A way of addressing this problem is to use the partially matched crossover (PMX).Here, we will consider PMX for parents with two crossover points but the techniquecan be applied to any number of crossover points. Consider the example above,where we obtained two children after performing 2PX operator. Note the violationsof the permutations:

Child 1 2 3 2 1 1 Violations: 1,2


It is desired to keep the value of the genes between the crossover points (theswapped parts between parents) to be the same as this was the outcome of theoperator that we applied. Thus, we can change bits 0,3 and 4. We adjust thechildren by inputting the violations of child 2 into child 1 and vice versa. So, 4 and5 will be used to correct child 1. Moving from the left, the first violation is 1 whichis located in bit 0 and 1. We cannot change bit 1 because it is between crossoverpoints. Thus, we change bit 0 with one of the violations from child 2. It does notmatter which violation we choose, but for consistance we will move from smallest tolargest value in the violation list. Thus, bit 0 becomes 4. The other violation of child1 is in bit 2 and bit 4. We cannot change bit 2 as it is between crossover points sowe change bit 4 to a value from the violations list for child 2. The only value left inthe list is 5 (4 was used for the first violation) so we change the value of bit 4. Weapply the same method to obtain a corrected child 2. For this example, the processcan be summarised as:

8



First violation is 1 located in bit 0 and 1. Correct bit 0 (outside crossover points)

Child 1 2 3 2 1 4

Second violation is 2 located in bit 2 and 4. Correct bit 4 (outside crossover points)

Child 1 5 3 2 1 4

This is now a correct permutation. Perform a similar correction for child 2. We go from

Child 2 4 5 4 5 3

to

Child 2 2 1 4 5 3

which is a correct permutation. Thus the corrected children are:

Child 1 5 3 2 1 4

Child 2 2 1 4 5 3

Various other crossover operators exist for permutations, with the most notableones being cycle crossover (CX) and order crossover (OX). Kumar and Kumar (2012)have shown that for TSP the PMX performs better than the other two crossoveroperators mentioned.

9


2.5 Mutation

Suppose we have a population of chromosomes:

1 0 1 0 1

1 0 0 1 0

0 0 1 1 1

1 0 1 1 0

Consider bit 3. The only value this gene contains in the whole population is 0.Thus, even if we perform a crossover operator, we might never be able to obtain1 as the value of this gene. Thus, we need a way of the chromosomes obtainingnew information that is not present in the original population. To achieve this, inhis original proposal of a genetic algorithm, Holland (1992) proposed a mutationoperator. This operator allows each gene to be altered with some small probability0 ≤ m ≤ 1.

For a binary representation, this can be done by generating a random numberu from the standard normal distribution and then flipping the bit if u < m. Toillustrate this, consider the example below with m = 0.2.

Chromosome 1 0 1 0 1

Generate 5 random numbers for i = 1, ..5

u = 0.76, 0.17, 0.74, 0.39, 0.66

Swap each bit if ui < m

Updated Chromosome 1 1 1 0 1

Thus, by performing the mutation operator we obtained a chromosome withinformation that was not available in the original population as bit 3 was flipped.(Sivanandam and Deepa, 2007, p. 56) suggested an alternative mutation operatorwhere the mutation operator is only applied if the mutation would improve thefitness of the chromosome. However, such an operator would increase the runningtime of the algorithm.

10


For a permutation representation, we cannot apply the above mutation operatoras it would create illegal chromosomes. Thus, an alternative would to swap twoconsecutive genes with probability p. This way, we will always mutate so that thechromosomes are legal solutions (permutations).

3 Applying SGA to TSP

3.1 Introduction

TSPLIB is an online library of sample TSP problems. Two of such problems will beselected and then the later proposed genetic algorithm will be applied in MATLAB.The two problems chosen are ulysses16 (n = 16) and ulysses22 (n = 22).

3.2 Representation and Evaluation Function

In section 2.1 we have suggested that a permutation is an appropriate representationfor the TSP. In the description of this problem, it states that we also always wishto start and finish in the same city. Thus, for ulysses16 we would always start andfinish at node numbered one. Between these two genes, we would have a permutationfrom 2 to 16. Thus, an example of a chromosome for the problem ulysses16 wouldbe:

1 14 13 12 7 6 15 5 11 9 10 16 3 2 4 8 1

TSPLIB also provides a distance matrix between these cities. We can calculatethe score of this chromosome by obtaining the distance between consecutive citiesand then summing those distances. For example, considering the example above, thedistance can be calculated as follows:

Count Current Node Next Node Distance1 1 14 4792 14 13 52...

......

...16 8 1 60

Total 6859

This is also the optimal route, as defined by the problem description on TSPLIB.The matrix of distances is saved in a .csv file in the directory of the program. This

11


allows the user to choose the distance matrix for which to calculate the route. Thus,for the above route we can use the command:

solution=[1,14,13,12,7,6,15,5,11,9,10,16,3,2,4,8,1];getScore(solution,’ulysses16dist.csv’)

ans =6859

Here, solution is a vector containing the chromosome and ulysses16dist.csv is thename of the file where the matrix of distances is located. This function returns thescore, which agrees with the score provided by TSPLIB for this route.

3.3 Initial population

Once we are able to represent routes and score them, we can generate a populationof chromosomes. This will be used in the alogirithm to obtain an initial populationthat we will then apply the genetic alogrithm to. Thus, if we wish to obtain 5 routesand their scores from the ulysses16 problem, we can use the code:

[population,scores]=initialPopulation(5,16,’ulysses16dist.csv’)

population =

1 9 5 12 7 8 11 13 3 6 ...1 12 4 13 7 15 9 5 3 14 ...1 8 15 10 3 14 5 6 7 11 ...1 4 10 13 6 14 8 15 11 7 ...1 12 10 2 13 8 6 9 15 5 ...

scores =14079 13075 10968 13273 12882

Thus, this function correctly generates legal chromosomes of the problem requested.The scores are also obtained.

3.4 Selection

In Section 2.3 we proposed two selection schemes: Roulette-Wheel Selection (RWS)and Tournament Selection (TOS). Out of these two, Jebardi and Madiafi (2013) have

12


shown that the TOS has a better convergence rate. Thus, for this particular geneticalgorithm we will use a TOS scheme.

The pseudocode outlined in 2.3.2 has been written as a function. To illustrate theworkings of this function, consider the ulysses16 problem with a population size of 10chromosomes. We wish to perform the selection amongst k = 5 (size of tournament)randomly chosen chromosomes from the population. Select chromosomes using thefunction defined in 2.3.2 with p = 0.5. Run the following code, using an initialpopulation as generated before.

A=initialPopulation(10,16,’ulysses16dist.csv’)tournament_Selection(A,0.5,5)

A =

1 9 8 7 13 6 15 12 14...1 3 5 7 6 13 12 15 11...1 5 12 7 14 2 3 6 10...1 4 2 6 10 7 9 8 14...1 3 2 10 8 11 9 6 14...1 14 15 7 11 5 8 10 3...1 6 14 15 7 4 9 2 13...1 3 14 11 8 6 13 4 5...1 9 2 13 15 4 12 8 6...1 9 7 8 15 12 5 3 2...

ans =

1 9 8 7 13 6 15 12 14...1 6 14 15 7 4 9 2 13...

Hence, the function works correctly by choosing some chromosomes from a givenpopulation.

3.5 Crossover

Once some chromosomes have been selected as parents, they will need to be crossedover to obtain children chromosomes. In section 2.4 some common crossover oper-ators have been defined. In section 2.4.3 it was mentioned that Partially Matched

13


Crossover (PMX) is a suitable choice for a permutation representation and that Ku-mar and Kumar (2012) have shown this operator to have better convergence ratesthan other common operators for permutation representations. Thus, this geneticalgorithm will use a PMX operator.

To perform PMX, we will need to define the crossover points. Since we wantour genetic algorithm to apply to problems with different number of nodes, it isunsuitable to keep the crossover points constant. For example, crossover points of5 and 12 might be suitable for ulysses16 since majority of the information withinchromosomes will be swapped between the parents. However, such crossover pointswill be unsuitable for larger problems such as berlin52 with n = 52 nodes since thegenes in the interval (12, 52) after this crossover point will not be altered. Thus, wewill define the crossover points to be 0.3 and 0.7 of the number of nodes (roundedto the nearest whole number).

Run the following code to check the crossover function.

A=initialPopulation(10,16,’ulysses16dist.csv’);selected=tournament_Selection(A,0.5,5)children=crossover2PMX(selected)

selected =

1 13 10 4 14 6 11 15 8 9 7...1 5 10 9 4 8 3 13 6 12 14...

children =

1 15 10 4 14 9 3 13 6 12 7...1 5 10 9 4 8 11 15 3 6 14...

We can see that the parents are significantly different than the children withthe information between the crossover points being swapped between the parents tocreate child chromosomes.Thus, the crossover operator works correctly.

3.6 Mutation

In Section 2.5 it was mentioned that we can perform mutation in a permutation rep-resentation by swapping two consecutive bits with a probability m. Such a function

14


was written and it will now be tested using a random population of two chromosomes.Let m = 0.1.

A=initialPopulation(2,16,’ulysses16dist.csv’)mutate(A,0.1,2,16)

A =1 5 2 3 12 8 9 7 6 4 15 10 11 14 13 11 14 9 11 8 2 5 10 7 (4 12) 15 6 13 3 1

ans=1 5 2 3 12 8 9 7 6 4 15 10 11 14 13 11 14 9 11 8 2 5 10 7 (12 4) 15 6 13 3 1

Here, we can see that in the bottom chromosomes 4 and 12 have swapped positions(shown in brackets). Thus, this function works correctly.

3.7 Termination criteria

Before running a genetic algorithm, we need to define a termination criteria. Thiswill allow the genetic algorithm to terminate when a desired solution is found.

One option is to consider the function to look for a certain solution. Thus, theprogram will run until a chromosome with a specific score is reached. If we are lookingfor the best solution to the ulysses22 problem, we will terminate the algorithm whenthe best score in the population is 6859. This approach is valid when we are lookingfor a specific solution that is ’good enough’ e.g. we want a solution that is lowerthan 7500 but we are not interested in the optimal one. However, if we wish to findan optimal route, we would usually not know the score of the optimum solution sowe cannot use this approach in such a case. This approach can also result in thealgorithm getting stuck in a local minimum. If we wish to run the ulysses16 problemuntil we find a solution lower than 8500 we run the following code:

main(50,30,0.1,9500,0.5,’ulysses16’)

Second consideration could be to terminate the algorithm after a solution hasnot improved for a certain q number of generations. For example, we are running analgorithm with q = 50. In iteration 70 we improved our solution. We keep runningthe solution and find that the solution in iteration 120 has the same score as in

15


iteration 70 so we terminate the algorithm. Such an approach could lead to prematuretermination if the algorithm cannot escape a local optima. This termination criteriacan be run using the command with q = 50:

main2(30,20,0.1,50,0.5,’ulysses16’)

The final option for a termination criteria is a combination of the two criteriaabove. We would run the algorithm while these criteria are satisfied: the score of thebest chromosome is greater than some threshold and the solution has improved inthe last q iterations. If the second condition is broken, the algorithm would restart.This would be repeated until we find the solution lower than the score we are lookingfor. Suppose we wish to find a solution with a score less than 8500 for the ulysses16problem. We also let q = 50, that is we restart the program if no improvement hasbeen made in 50 steps.

main3(30,20,0.1,50,0.5,’ulysses16’,8500)

3.8 Visualising the performance of the algorithm

In order to help determine how well a given run of the algorithm performs, somecharacteristics of each generation need to be captured. One such characteristic isthe fitness score of the fittest chromosome, which is the distance of the best solutionfound so far. The change of this over time can tell how fast the algorithm convergesand whether it is getting stuck in a local optima. This allows us to determine thebehaviour of the algorithm under certain parameters which in turn allows us tochoose appropriate choices of the parameters in order to find the optimum solution.

Another characteristic of the generation we may be interested in is the averagefitness score of all chromosomes in the population. This can tell us how diverse ourpopulation is. The more diverse a population is, the smaller the selection pressureand this results in the algorithm being less likely to get stuck in a local optima.

Given some termination criteria, we want to compare the fitness of the initialpopulation with the fitness of the final population. To do this, we may take the bestsolution in each of these generations and plot it. TSPLIB provides the positions ofthe nodes in the problem which can then be plotted as points in the right order tovisualise a path. When calling a problem, say ulysses16 the program will read thepositions from the file ulysses16pos.csv, provided such a file is located in the samedirectory. We will show the workings of such a function by generating two randomchromosomes from the ulysses16 problem and plotting the solutions. Note that herethe two solutions are randomly generated using the initial population function seen

16


earlier on, but in the main program these will be the best solutions in the first andfinal generations.

A=initialPopulation(2,16,’ulysses16dist.csv’)drawRoute(A(1,:),A(2,:),’ulysses16dist.csv’,’ulysses16pos.csv’)

The output of this command can be seen in Figure 1.

Figure 1: An example of visualising two random solutions.

4 Results

4.1 Investigating the effect of the size of the population

It will now be investigated how the size of the population effects the convergenceof the algorithm. To investigate this, the program obtains the value of the fittest

17


chromosome of each consecutive generation and upon terminating plots these val-ues against the number of the generation. In addition, the average fitness of eachgeneration is also shown as this can be used to comment on the diversity of thepopulation.

The algorithm was run with three different sizes of the population: 10, 25 and40. The program uses the second termination criteria (see Section 3.7) so it assumesthat the program is stuck in a local optima when an improved solution has not beenfound in the last 150 generations. The other parameters have been kept constant.The results of such a test are shown in Figure 2.

Firstly, consider the number of generations performed by each execution. Thenumber of generations increases as the population size increases: we have 191 gen-erations for population size of 10, 339 generations for a population size of 25 andfinally 751 generations for a populations of size 40. (Holland, 1992) states that thegenetic algorithm aims to avoid getting stuck in a local optima by keeping a largepopulation of solutions. The results obtained here correspond to this, as the numberof iterations before the run is stuck in a local optima increases as the population sizeincreases.

Also, compare the values of the minimum and the average. The difference betweenthese two values tells us how varied our population is. The closer the two are, themore uniform the population is. Similar populations are undesired as this increasesthe chance of the algorithm getting stuck in a local optima. In Figure 2 we canclearly see that the population is much more diverse for larger population sizes, dueto the bigger difference between the minimum and the average. It is worth notingthat the best solution found is closer to the optimal solution for higher populationsizes.

However, increasing the population size also causes the running time of the al-gorithm to increase. In Figure 2 the running time of each run of the algorithm hasbeen noted. Since the runs have a different number of generations, we need to workout the time taken to obtain each generation. Thus, we divide the running time bythe number of generations for each run. We obtain values of 0.055, 0.075 and 0.095for the population sizes of 15, 25 and 40 respectively. Thus, the running time pergeneration increases with the population size.

To conclude, we have shown how the population size can affect the convergenceof the genetic algorithm. Thus, it is important to keep the population diverse byhaving a large population size at each generation. However, we have also shown thatincreasing the population size increases the running time. Therefore, due to thistrade-off, the population size should be large enough to avoid local optima but notoverly large so that the computation time becomes infeasible.

18


4.2 Investigating the effect of mutation

We have applied a mutation operator to help keep genetic diversity in each gen-eration. It will now be investigated how different mutation rates can affect theconvergence of our algorithm. Results of such an experiment are shown in Figure 3.

When m = 0, there is no mutation present. Thus, if some information is notpresent in the initial generation, we will not be able to reach solutions with thosecharacteristics. Thus, an algorithm which does not use the mutation operator ismuch more likely to get stuck in a local optima, where every solution eventuallybecomes the same. This is the case in the graph for m = 0 in Figure 3 as the averageand minimum solutions are identical for generations > 25 which indicates that all thechromosomes in the population are exactly the same. Note that the last generationthe minimum solution has improved in is generation 9.

As we increase the mutation rate, we also increase the difference between theaverage and the minimum solution. Thus, mutation helps us to maintain a diversepopulation. In turn, this allows the algorithm to not converge prematurely.

The number of generations of each run is also noted. We can see that as weincrease m we also increase the number of generations. This corresponds with whatwe found above, that the higher mutation rates decrease the chance of getting stuckin a local optima.

The minimum solutions also seem to be lower for the higher mutation rates.However, very high mutation rates are also undesirable. In such cases, the genetic

algorithm behaves more like a random walk as good characteristics are less likely tobe preserved which could lead to the algorithm taking unnecessarily long to converge.

5 Optimal solution foundFor the ulysses16 an optimal solution was found and it can be seen in Figure 4. Thecommand used to obtain this result is

main3(30,20,0.1,150,0.5,’ulysses16’,6859).

As was found in Section 4.1, higher population sizes tend to keep the populationmore diverse which is a desired characteristic. However, an overly large populationsize can result in low running speed of the algorithm. Thus, a population size of 30was chosen, which allowed for a diverse population while keeping the computationreasonably fast.

We have also chosen the mutation parameter m = 0.1. This was investigatedin Section 4.2 and it was found that mutation can help in keeping the population

19


diverse. When m is too large, the algorithm behaves more like a random walk,meaning that the algorithm runs unnecessarily long. A value was chosen betweenthese two extremes.

We are also required to specify two parameters for tournament selection. Weneed to choose k (the size of the tournament) and p (probability of selecting eachchromosome). It is desired to keep k large as this allows for a larger variety ofchromosomes to be selected. Recall that 0.5 ≤ p ≤ 0.9. A small p leads to less fitchromosomes being more likely to be chosen while a large p means that the fittestparents are almost always chosen. Thus, a small p was chosen so that the algorithmis more likely to choose less fit parents which in turn could mean that the algorithmis less likely to get stuck in a local optima.

Finally the other two parameters is the solution to be found (in this case thisis 6859) and q (the number of generations after which we restart the algorithm ifa solution was not found). Finding the optimal solution using this command cantake some time and varies with each execution due to the aspects of randomness inselection and mutation.

Next, the algorithm was applied to ulysses22 problem. The following functionwas used to find a solution with a score of 7200 or less.

main3(20,20,0.1,150,0.5,’ulysses22’,7200).

The results can also be seen in Figure 5. The algorithm has been unable to findthe optimal solution of 7013 due to being stuck in a local optima. However, thesolution found with a score of 7153 is still a very good solution. When consideringthe graphed version of this solution (shown in the bottom figure) it can be seenthat this is close to what one would imagine the optimal solution to be. The onlydifference between this solution and the optimal one probably is a few swaps inthe very ’congested’ area in the middle. The solution found is superior to the bestchromosome in the initial population.

Finally, this example shows how genetic algorithms can be applied in practice.When usually faced with a large TSP problem, we will not know the optimal solution.Thus, we would not be able to tell if the found solution is the optimal one. We couldinstead search for a solution that is below a certain threshold (7200 here). Once sucha solution has been found, we would decrease the threshold and look for a solutionbelow this new threshold. We could continue this until the algorithm is not findingan optimal solution in some time.

20


6 ConclusionIn this project, we have outlined the workings of the Simple Genetic Algorithm. Wehave introduced a number of selection schemes (Rolette Wheel Selection, Tourna-ment Selection) and crossover operators (One-Point Crossover, Two-Point Crossover,Partially Matched Crossover) as well as the general approach to solving search prob-lems (Fitness function, representation etc.). These tools can be applied to solve arange of search problems. However, many other operators exist for the genetic al-gorithm which may be suited better for different problems. Michalewicz and Fogel(2004) mentions that there are as many genetic algorithms as there are problemsthat they are built to solve.

With the knowledge of the workings of the Genetic Algorithm, we have appliedthis to the Travelling Salesman Problem. We considered two problems: ulysses16with 16 nodes and ulysses22 with 22 nodes. We defined a permutation representationand then applied the knowledge of genetic algorithms to choose suitable operators forthis algorithm. We have chosen the algorithm to have a tournament selection schemeand a partially matched crossover. These choices have been based on research thathas shown these to be the most suitable choices of operators for TSP. We introduceda mutation operator that swaps two consecutive genes with some probability.

To help visualise the workings of the algorithm, we graph some characteristics ofthe population over the running of the algorithm. We also visualise the initial andoptimal solutions as graphs to show the difference between the starting solution andthe optimal one found. Finally, we defined three different termination criteria forthe algorithm.

We then investigated the effect of the size of the population and the mutationprobability. We found that higher population sizes help to maintain diversity ofthe population. However, there is a trade-off as the running time of the algorithmis extended. It was also shown that mutation has a similar trade-off where somemutation can help to keep a diverse population but a very high mutation rate couldlead to the algorithm taking unnecessarily long to converge.

Using these results we run the algorithm to find the best solution for the ulysses16problem. We knew the optimal solution so the algorithm was allowed to run untilwe found this specific solution. Such an approach is not useful in practice as weusually are not aware of the optimal solution. Thus, an approach where we look fora solution below a certain threshold was suggested. The best solution found for thisproblem was 7153, whilst it can be shown that the optimal solution for this problemis 7013.

This project can easily be extended to look at problems with a larger number

21


of nodes. There are many different selection schemes and crossover operators whichhave not been covered which can also be investigated. Finally, there are other meta-heuristic algorithms such as simulated annealing or tabu search that can be appliedto solve this problem.

ReferencesJ.H. Holland. Adaptation In Natural And Artificial Systems: An Introductory Analy-sis With Applications To Biology, Control, And Artificial Intelligence, 2nd Edition.A Bradford Book, 1992.

K. Jebardi and M. Madiafi. Selection methods for genetic algorithms.Int.J.Emerg.Sci., 3:333–344, 2013.

Bidhan K. Kumar, N. and R. Kumar. A comparative analysis of pmx, cx and oxcrossover operators for solving travelling salesman problem. IJLRST, 1:98–101,2012.

Z. Michalewicz and D.B. Fogel. How to Solve It: Modern Heuristics, 2nd Edition.Springer, 2004.

B.L. Miller and D.E. Goldberg. Genetic algorithms, tournament selection, and theeffects of noise. Comp. Sys., 9:193–212, 1995.

M. Mitchell. An Introduction To Genetic Algorithms. A Bradford Book, 1996.

S. N. Sivanandam and S. N. Deepa. Introduction to Genetic Algorithms. Springer,2007.

22


Figure 2: The graphs for average and minimum path for different settings of the sizeof the population. The other parameters have been kept at constant, meaningfulrates. The runtime of each execution is also shown.

(a) Population size = 10RUNTIME: 10.42 seconds

(b) Population size = 25RUNTIME: 25.47 seconds

(c) Population size = 40RUNTIME: 71.37 seconds

23


Figure 3: The graphs for average and minimum path for different settings of themutation rate. The other parameters have been kept at a constant, meaningfulrates.

(a) Mutation rate m = 0 (b) Mutation rate m = 0.01

(c) Mutation rate m = 0.05 (d) Mutation rate m = 0.10

24


Figure 4: The optimal solution found for the ulysses16 problem

(a) Behaviour of the algorithm over time

(b) Optimal route25


Figure 5: The best solution found for the ulysses22 problem

(a) Behaviour of the algorithm over time

(b) Optimal route26

Documents

TSP using Genetic Algorithms in MATLAB