A GRASP with path-relinking and restarts heuristic …celso/artigos/PCGMSTP.pdf2 R.G. Marzo and C.C. Ribeiro / Intl. Trans. in Op. Res. 27 (2020) 1–29 and j2V ‘. If k6= ‘, then

INTERNATIONALTRANSACTIONS

IN OPERATIONALRESEARCHIntl. Trans. in Op. Res. 27 (2020) 1–29

DOI: xx.xxxx/itor.xxxxx

A GRASP with path-relinking and restarts heuristic for theprize-collecting generalized minimum spanning tree problem

Ruslan G. Marzoa and Celso C. Ribeiroa,∗

aInstitute of Computing, Universidade Federal Fluminense,Niteroi, RJ 24210-346, Brazil.

E-mail: [email protected] [Ruslan]; [email protected] [Celso]

Received 20 12 2018, revised 30 08 2019, accepted 06 09 2019

Abstract

Given a graph with its vertex set partitioned into a set of groups, non-negative costs associated to its edges, andnon-negative prizes associated to its vertices, the prize-collecting generalized minimum spanning tree problemconsists in finding a subtree of this graph that spans exactly one vertex of each group and minimizes the sumof the costs of the edges of the tree less the prizes of the selected vertices. It is a generalization of the NP-hardgeneralized minimum spanning tree optimization problem. We propose a GRASP (Greedy Randomized AdaptiveSearch Procedure) heuristic for its approximate solution, incorporating path-relinking for search intensificationand a restart strategy for search diversification. The hybridization of the GRASP with path-relinking and restartsheuristic with a data mining strategy that is applied along with the GRASP iterations, after the elite set is modifiedand becomes stable, contributes to making the heuristic more robust. The computational experiments show that theheuristic developed in this work found very good solutions for test problems with up to 439 vertices. All input datafor the test instances and detailed numerical results are made available from Mendeley Data.

Keywords: prize-collecting generalized minimum spanning tree problem, generalized minimum spanning tree problem, GRASP,metaheuristics, randomized metaheuristics, path-relinking, restarts, time-to-target plots

1. Introduction

Let G = (V,E) be a graph defined by its vertex set V = {1, . . . , n} and its edge set E ⊆ V × V , with aprize pi ≥ 0 associated to each vertex i ∈ V and a cost ce ≥ 0 associated to each edge e = (i, j) ∈ E,with i, j ∈ V . Without loss of generality, we assume that graph G is complete. We also assume thatthe node set V can be partitioned in K disjoint sets or groups V1, . . . , VK , i.e.,

⋃Kk=1 Vk = V and

Vk ∩ V` = ∅, for any k, ` ∈ {1, . . . ,K} : k 6= `. Let e = (i, j) ∈ E be an edge with extremities i ∈ Vk

∗Author to whom all correspondence should be addressed (e-mail: [email protected]).

c© 2019 International Transactions in Operational Research c© 2019 International Federation of Operational Research Societies

2 R.G. Marzo and C.C. Ribeiro / Intl. Trans. in Op. Res. 27 (2020) 1–29

and j ∈ V`. If k 6= `, then e is said to be an intergroup edge, otherwise e is said to be an intragroupedge. The prize-collecting generalized minimum spanning tree problem (PC-GMSTP) consists in findinga subtree of G spanning exactly one vertex of each group V1, . . . , VK and minimizing the total cost ofthe edges in the tree less the prizes of the vertices selected in each group (Pop, 2004, 2007; Golden et al.,2008).

According to (Pop, 2007; Golden et al., 2008), the prize-collecting generalized minimum spanningtree problem was introduced by Stanojevic et al. (2004) in the context of the design of regional com-munication networks connecting local area networks (LAN). In this application, one unique gatewayneeds to be identified within each LAN, and the gateways have to be connected via a minimum spanningtree. Additionally, nodes within the same cluster are competing to be selected as gateway nodes, andeach node offers a certain compensation (or prize) if it is selected. The problem consists in minimizingthe total cost of the links used to connect the regional networks, discounting the prizes collected at thesites selected as the gateway locations. In the design of undersea cable networks connecting differentcontinents (Golden et al., 2008), not all countries or cities along the way can be directly connected tothe network, due to the high connection costs. Consequently, the designers of these undersea cable net-works usually choose one location from each of the specified set of regions that the network traverses.Given the potential monetary benefits associated with being a location that is directly connected to atranscontinental fiber-optic network with significant economic benefits, it is not uncommon for cities orcountries to compete against each other for selection as a location, offering monetary incentives in theform of tax credits and rebates to the builder or operator of the network. There are similar applicationsin marketing (Pop, 2007) and other areas.

If all prizes are equal to zero (or, if all prizes are simply equal for the vertices in each group of thevertex partition), then the prize-collecting generalized minimum spanning tree problem can be reducedto the generalized minimum spanning tree problem (GMSTP) (Feremans et al., 2004; Golden et al.,2005; Pop, 2004; Contreras-Bolton et al., 2016). Since the decision version of the latter was proved tobe NP-hard in Myung et al. (1995), PC-GMSTP is also NP-hard (Golden et al., 2008).

There are several integer programming formulations for the prize-collecting generalized minimumspanning tree problem, see e.g. (Pop, 2007; Golden et al., 2008). The formulation below is based onthe generalized subtour elimination formulation introduced by Myung et al. (1995) for the generalizedminimum spanning tree problem. Let zi = 1 if vertex i ∈ V is selected as a group representative vertex,zi = 0 otherwise. In addition, let xe = 1 if edge e belongs to the generalized minimum spanning tree,xe = 0 otherwise. A feasible solution for PC-GMSTP is any acyclic subgraph of G with K − 1 edgesthat spans exactly one vertex of each group. The optimization problem can be formulated as follows:


R.G. Marzo and C.C. Ribeiro / Intl. Trans. in Op. Res.27 (2020) 1–29 3

(PC-GMSTP) min∑e∈E

cexe −∑i∈V

pizi (1)

subject to: ∑i∈Vk

zi = 1 ∀k = 1, · · · ,K (2)

∑e∈E(S)

xe ≤∑

i∈S\{j}

zi ∀j ∈ S,∀S ⊂ V : 2 ≤ |S| ≤ |V | − 1 (3)

∑e∈E

xe = K − 1 (4)

xe, zi ∈ {0, 1} ∀e ∈ E,∀i ∈ V. (5)

In the above model, constraints (2) ensure that exactly one representative vertex is selected from eachgroup. Constraints (3) eliminate all subtours, where E(S) ⊆ E represents the subset of E formed by alledges with both extremities in a subset S ⊂ V with at least two and at most |V | − 1 vertices. Finally,constraints (4) guarantee that the spanned subgraph has exactly K − 1 edges. The objective function (1)subtracts from the total cost of the selected edges the second term corresponding to the sum of thecollected prizes.

Golden et al. (2008) proposed a heuristic for PC-GMSTP and adapted a local search and a geneticalgorithm originally developed for GMSTP. A genetic algorithm based solution approach was proposedin (Pop et al., 2017).

In this work, we propose a new GRASP heuristic with restarts for approximately solving the prize-collecting generalized minimum spanning tree problem. Section 2 presents the literature review coveringthe generalized minimum spanning tree problem and the prize-collecting generalized minimum spanningtree problem. Section 3 describes in detail the first, basic version of the GRASP heuristic for PC-GMSTPintegrating the construction and local search phases. Section 4 describes the hybridization of the basicGRASP heuristic with path-relinking, which is a major enhancement that adds to the original algorithma long-term memory mechanism using an elite set of diverse high-quality solutions. Section 5 completesthe algorithm, adding a restart strategy to diversify the search and to reduce the running times taken byGRASP with path-relinking. Computational experiments and numerical results are reported in Section 6,where we also show that the hybridization of the heuristic with a data mining strategy contributes tomake the former more robust. All test data for the 30 randomly created test instances are made availableat Mendeley Data, together with the best solutions found by the proposed algorithm. Concluding remarksare drawn in the last section.

2. Literature review

The generalized minimum spanning tree problem (GMSTP) is a generalization of the minimum spanningtree problem and was first introduced by Myung et al. (1995). Considering the same notation defined inthe previous section, it consists in finding a subtree of G spanning exactly one vertex of each group



Fig. 1. Schematic representation of a feasible solution of the generalized minimum spanning tree problem on a graph with 18vertices partitioned into K = 7 disjoint sets or groups (Pop, 2002).

V1, . . . , VK so that the sum of the costs of the tree edges is minimized. Figure 1 illustrates a feasiblesolution of the generalized minimum spanning tree problem on a graph with 18 vertices partitioned intoK = 7 disjoint sets.

The generalized minimum spanning tree problem was proved to be NP-hard by Myung et al. (1995).Pop (2004) proved that the problem remains NP-hard even when G is restricted to be a tree. The demon-stration is based on a polynomial transformation of the set covering problem into the generalized min-imum spanning tree problem. The same complexity results apply to the prize-collecting generalizedminimum spanning tree problem (Pop, 2007).

Myung et al. (1995) developed a branch-and-bound procedure for solving GMSTP, based on a lowerbound that approximates the linear relaxation of the dual of the multicommodity flow formulation ofGMSTP, that was used to solve problems in networks with up to 100 vertices and 4500 edges. They alsodeveloped a heuristic that computes a feasible solution of GMSTP using the dual solution. Problemswith up to 240 vertices were solved by Pop (2002). Dror et al. (2000) considered the variant of GMSTPin which at least one (instead of exactly one) vertex of each group should be selected.

Integer programming formulations of GMSTP were proposed by Feremans et al. (2002) and comparedin terms of their linear programming relaxations. Feremans et al. (2004) presented a polyhedral analysisand a branch-and-cut algorithm for GMSTP. This algorithm found optimal solutions for graphs with upto 200 vertices and random costs and for graphs with up to 160 vertices and costs satisfying the triangleinequality.

Pop (2004) proposed another mathematical formulation for GMSTP that was used in the developmentof an exact approach that may be seen as a special form of delayed row and column generation. This for-mulation was also discussed in (Golden et al., 2008). Golden et al. (2005) proposed two fast metaheuris-tics for GMSTP that obtained optimal solutions for most problems in a set of test instances (Golden et al.,2008). Pop et al. (2018) developed a two-level solution approach to GMSTP, decomposing the problem



in two subproblems that are solved separately: in the higher level (global problem), trees spanning theclusters are determined by a genetic algorithm using a diploid representation of the individuals, whilein the lower level (local subproblem) the best from among the above trees, spanning exactly one vertexfrom each cluster, is optimally determined by dynamic programming.

Another variant of GMSTP originally introduced in (Cockayne and Melzak, 1968) requires that atleast one (instead of exactly one) vertex of each group should be spanned and is more intimately relatedwith the Group Steiner Problem (GSP) (Duin et al., 2004), which is a generalization of the Steinerproblem in graphs (SPG). In this case, the disjoint sets V1, . . . , VK do not define a partition of the vertexset V and the subset V ′ = V −

⋃Kk=1 Vk establishes a set of Steiner vertices that may be used to connect

the selected representative vertices in each group, but do not necessarily have to be spanned (Goldenet al., 2008).

Duin et al. (2004) applied two heuristics, originally developed for the Steiner problem in graphs (Duinand Voß, 1999), and obtained good results for the above special case of GMSTP. For a set of problemswith random, Euclidean, and rectilinear distances in graphs with 100 to 400 vertices, algorithm Pilot-Drop obtained solutions within less than 0.3% of the optimal value on average, with a maximum gap of3.4% (Golden et al., 2008). A GRASP heuristic for GMSTP was proposed in Ferreira et al. (2012).

Duin et al. (2004) showed that an exact algorithm for the Steiner problem in graphs, similar to thatin (Duin, 1994), outperformed a genetic algorithm (Dror et al., 2000) in terms of solution quality andrunning times for the “at least one vertex” version of GMSTP. The genetic algorithm obtained solutionswithin less than 6.53% of the optimal value on average over 20 test problems, while the exact algorithmfound optimal solutions for all instances in small running times. The ant colony approach developedin Shyu et al. (2003) obtained results comparable with those obtained by the genetic algorithm. Haouariet al. (2005) developed an exact branch-and-bound algorithm for the“at least one vertex” version of GM-STP. Combined with a specialized pre-processing algorithm, branch-and-bound found optimal solutionsfor problems in graphs with up to 250 vertices, 1000 edges, and 25 groups in less than three hours ofrunning time (Golden et al., 2008).

To the best of our knowledge, Stanojevic et al. (2004) were the first to tackle the prize-collectinggeneralized minimum spanning tree problem (PC-GMSTP).

Golden et al. (2008) proposed three heuristics for PC-GMSTP and adapted a local search and a geneticalgorithm originally developed for GMSTP (Golden et al., 2005). They also proposed a branch-and-cutalgorithm, compared its performance with two variants of the exact procedure proposed in (Pop, 2004)for GMSTP, and showed that it obtains better bounds. The branch-and-cut algorithm obtained optimalsolutions for networks with random costs with up to 200 vertices and 40 groups in two hours of runningtime. In networks with Euclidean costs, the algorithm obtained optimal solutions for networks with upto 125 vertices in up to 2 hours of running time. The local search and the genetic algorithm found theoptimal solutions for all 296 test instances where the optimal value was known.

An exact algorithm and integer programming formulations for PC-GMSTP are presented in (Pop,2007). Optimal solutions are found for graphs with up to 240 vertices.



3. GRASP heuristic for PC-GMSTP

GRASP, which stands for a Greedy Randomized Adaptive Search Procedure, is a multi-start metaheuris-tic, in which each iteration consists of two main phases: construction and local search. The first phase isthe construction of a feasible solution, usually by a greedy randomized algorithm. Once a feasible solu-tion is obtained, its neighborhood is investigated until a local minimum is found during the local searchphase. The best overall solution is kept as the result. The reader is referred to Resende and Ribeiro(2016) for a complete account of GRASP, while surveys appeared in (Resende and Ribeiro, 2010, 2019).Annotated bibliographies of GRASP appeared in (Festa and Resende, 2009a,b). Recent applications ofGRASP are reported e.g. in (Bajwa et al., 2019; Climaco et al., 2019; Interian and Ribeiro, 2017).

The construction phase of a GRASP heuristic for PC-GMSTP is described in Section 3.1. Local searchstrategies are presented in Section 3.2. The first, basic version of the GRASP heuristic for PC-GMSTPintegrating the two phases is given in Section 3.3.

3.1. Construction phase

The greedy randomized heuristic implemented in the construction phase is a combination of the algo-rithms of Kruskal (1956) and Prim (1957) for the minimum spanning tree problem, see also (Cormenet al., 2009). Their adaptation as a heuristic for PC-GMSTP takes into account the fact that exactly onevertex should be selected from each group. It is also based on considering not only the edge costs, butalso the vertex prizes. Only intergroup edges should be considered, because intragroup edges are notallowed to be part of a solution, as far as only one vertex of each group is selected as its representative.

In order to consider the vertex prizes, we follow a strategy similar to that in (Golden et al., 2008).Initially, the cost of each edge is modified by the subtraction of the prizes of its two extremities. Thismodified edge cost will better reflect the change incurred in the total solution cost by the addition of thisedge. In the next steps of the algorithm, the edge costs are updated whenever a new vertex is added tothe tree that is being built. If a new vertex is inserted into the spanning tree, then its prize is added tothe cost of all edges incident to it, in such a way that the objective function value becomes correct. Thismodification of the edge costs ensures that the prizes of the vertices already in the tree are not consideredin the edge selection process (since that once a vertex is selected for the generalized spanning tree, it isonly the actual edge cost that makes a difference in the objective function).

We recall that Kruskal’s algorithm picks first a minimum cost edge to start the construction of aminimum spanning tree. In the case of the greedy randomized construction heuristic for PC-GMSTP, inthe first iteration an edge is randomly selected from a restricted candidate list RCLedges formed by thelowest cost edges. The next iterations use a Prim-like criterion to randomly select the next vertex thatwill be inserted in the generalized tree from a candidate list RCLvertices formed by the lowest insertioncost vertices that do not violate the PC-GMSTP constraints.

Each new iteration adds a new isolated vertex representing a new group V1, . . . , VK to the current solu-tion, without creating a cycle. The algorithm stops afterK−2 additional iterations, when the generalizedspanning tree contains exactly one vertex from each group.

Algorithm 1 presents the pseudo-code of the greedy randomized construction phase. ProcedureRANDOMIZED-CONSTRUCTION takes as inputs the graph G, the edge costs w, the node prizes p,



and a parameter α. During the execution of the algorithm, all vertices that are not spanned by the gener-alized spanning tree under construction are stored in a minimum priority queue V ′ based on the attributekey , such that for each vertex i, key(i) is the minimum modified cost of connecting vertex i to the treeby an edge. By definition, key(i) = ∞ if such an edge does not exist. The predecessor of a vertex i inthe tree is indicated by π(i). The priority queue V ′ becomes empty when the algorithm stops. In whatfollows, we denote by V −1(i) ∈ {1, . . . ,K} the index of the group to where a vertex i ∈ V belongs.

Algorithm 1 RANDOMIZED-CONSTRUCTION(G,w, p, α)1: for all i ∈ V do2: key(i)←∞; π(i)← 03: end for4: w(i, j)← w(i, j)− pi − pj , ∀(i, j) ∈ E5: wmin ← min{w(i, j) : (i, j) ∈ E}6: wmax ← max{w(i, j) : (i, j) ∈ E}7: threshold ← wmin + α · (wmax − wmin)8: RCLedges ← {(i, j) ∈ E : w(i, j) ≤ threshold}9: Randomly select the initial edge (a, b) from RCLedges

10: S ← {(a, b)}; f(S)← w(a, b)11: V ′ ← V \ (VV −1(a) ∪ VV −1(b))12: for ` = a, b do13: for all k ∈ V ′ : vertex k adjacent to vertex ` do14: w(`, k)← w(`, k) + p`15: if w(`, k) < key(k) then16: π(k)← `; key(k)← w(`, k)17: end if18: end for19: end for20: while V ′ 6= ∅ do21: cmin ← min{key(i) : i ∈ V ′}22: cmax ← max{key(i) : i ∈ V ′}23: threshold ← cmin + α · (cmax − cmin)24: RCLvertices ← {i ∈ V ′ : key(i) ≤ threshold}25: Randomly select a vertex a from RCLvertices

26: S ← S ∪ {(a, π(a))}; f(S)← f(S) + w(a, π(a))27: V ′ ← V ′ \ VV −1(a)

28: for all k ∈ V ′ : k adjacent to a do29: w(a, k)← w(a, k) + pa30: if w(a, k) < key(k) then31: π(k)← a; key(k)← w(a, k)32: end if33: end for34: end while35: return S, f(S)



The loop in lines 1–3 initializes the attributes of each vertex i ∈ V . In line 2, each vertex is setinitially with a very high value as its connection cost key(i) and a null value as its predecessor π(i),to indicate that it does not exist. For each edge (i, j), line 4 subtracts from w(i, j) the prizes pi andpj of its extremities. Lines 5 and 6 obtain the minimum and the maximum edge costs, respectively.Line 7 determines the threshold that will be used in the construction of the restricted candidate list. Therestricted candidate list RCLedges is built in line 8. A low cost edge (a, b) is randomly selected fromRCLedges in line 9. The edge set S of the generalized spanning tree is initialized with edge (a, b) andits cost f(S) with w(a, b) in line 10. The priority queue is initialized in line 11 with all vertices in V ,with the exception of those in groups VV −1(a) and VV −1(b) already spanned, to where vertices a and bbelong. For each extremity ` = a, b of the selected edge (a, b), the inner loop in lines 13–18 updatesthe attributes related to all vertices k remaining in the priority queue that are adjacent to extremity `.Line 14 adds the prize p` to the cost of the edge having vertex ` as one of its extremities and vertex kas the other, in order to compensate for the cost modification previously performed in line 4. Line 16updates the predecessor π(k) and the connection cost key(k) of vertex k. The main loop in lines 20–34 isrepeated for exactly K − 2 iterations, until exactly one vertex in each group is spanned and V ′ becomesempty. Lines 21 and 22 obtain the minimum and the maximum connection costs over all vertices inV ′, respectively. Line 23 determines the threshold that will be used in the construction of the restrictedcandidate list. The restricted candidate list RCLvertices is built in line 24. A low connection cost vertex ais randomly selected from RCLvertices in line 25. In line 26, the edge set S of the generalized spanningtree is updated with edge (a, π(a)) and its cost f(S) with w(a, π(a)). All vertices in group VV −1(a) areremoved from the priority queue V ′ in line 27, since vertex a is now spanned. The loop in lines 28–33updates the attributes related to all vertices k remaining in the priority queue that are adjacent to vertexa. Line 29 adds the prize pa to the cost of the edge having vertex a as one of its extremities and vertexk as the other, in order to compensate for the cost modification previously performed in line 4. Line 31updates the predecessor π(k) and the connection cost key(k) of vertex k. Line 35 returns the edge set Sand the total cost f(S) of the generalized spanning tree.

Example. We present an example of the application of procedure RANDOMIZED-CONSTRUCTIONof Algorithm 1 to the graph in Figure 2, with eight vertices labeled from A to H arranged in four groupsV1 = {A,C}, V2 = {B,D}, V3 = {E,G}, and V4 = {F,H}, with the intergroup edge costs indicatedin the adjacency matrix. Vertex prizes are indicated in red, while group indices are indicated in blue.

Figure 3 depicts the application of the heuristic with the candidate list parameter α = 0.1. Edges inblue in Figure 3(a) are those in the first restricted candidate list RCLedges built in line 8. Edge (G,H) inred in Figure 3(b) is randomly selected as the initial edge of the tree in line 9. Figure 3(c) correspondsto the first iteration of the main loop in lines 20–34. Edges (C,G) and (D,H) in blue connect thevertices in first restricted candidate list RCLvertices = {C,D} built in line 24, which are the closestto the partial tree under construction: at this time, π(C) = G and π(D) = H . Let us assume thatvertex D ∈ RCLvertices is randomly selected to enter in the tree in line 25 and edge (D,H) in red isadded to the tree in Figure 3(d). Now, Figure 3(e) corresponds to the second iteration of the main loopin lines 20–34. There will be only one vertex in the restricted candidate list RCLvertices = {A} builtin line 24, connected to the partial tree by edge (A,D) in blue. At this time, π(A) = D. Vertex A isnecessarily selected to enter the tree and so is edge (A,D) in red in Figure 3(f), which now correspondsto a generalized spanning tree of the original graph, with total cost 61 (the edge costs amount to 84,



Fig. 2. Graph and adjacency matrix with intergroup edge costs. Vertex prizes are indicated in red, while group indices areindicated in blue.

while the vertex prizes are 23). �

3.2. Local search

Local search methods start from any feasible solution and visit other (feasible or infeasible) solutions,until a feasible solution that cannot be further improved is found (Resende and Ribeiro, 2016). Lo-cal improvements are evaluated with respect to neighboring solutions that can be obtained by slightmodifications applied to a solution being visited. Since the solutions produced by a greedy randomizedconstruction are not necessarily optimal, local search almost always improves their quality.

Local search for PC-GMSTP is based on an 1-opt neighborhood defined as follows. Each solutionis implicitly characterized by K representative vertices, each of them representing the vertices in oneof the groups V1, . . . , VK . Associated to each solution there are potentially

∑Kk=1(|Vk| − 1) neighbors,

obtained by replacing exactly one of the selected vertices in the current solution by another one in thesame group. Once the subset of K representative vertices is selected, a complete solution is computedas a minimum spanning tree of the graph induced in G by the selected vertices.

We have developed two local search procedures for PC-GMSTP. The first local search procedure de-terministically visits the groups of the graph in the increasing order of their indices, as already appliedby Feremans (2001) (see (Golden et al., 2005)). Before moving to the next group, the algorithm exploresthe substitution of the representative of the current group being investigated by every other vertex inthis same group and updates the best neighbor solution whenever a better representative is found. Whilethe representatives of one group are explored and eventually changed, all others remain fixed. The lo-cal search procedure stops when no further improvement is possible. At any local search iteration, theprocedure moves from the current solution to the best neighbor.



Fig. 3. Application of procedure RANDOMIZED-CONSTRUCTION.



The pseudocode of Algorithm 2 describes a best-improving minimization local search strategy forPC-GMSTP starting from a feasible solution S, as described by Resende and Ribeiro (2016). A flagimprovement indicating whether or not a better solution was found is initialized with .TRUE. in line1. The loop in lines 2–19 is performed until a locally optimal solution is found, i.e., when no furtherimprovements are possible. The flag improvement is reset with .FALSE. in line 3 at each iteration. Thevalue fbest of the objective function for the best neighbor of the current solution S is initialized with alarge number in line 4. The two nested loops investigate the 1-opt neighborhood of the current solutionS: the outer loop in lines 5–14 examines all groups. Line 6 picks the representative vertex u of the groupVk being visited, while the inner loop 7–13 replaces vertex u by every other possible representative v inthe same group. Line 8 creates a neighbor S′ by replacing vertex u by v in the current solution S. If line9 detects that the new neighbor S′ improves the current best neightbor S, then lines 10 and 11 update thebest neighbor Sbest and its cost fbest , respectively. Lines 15 to 18 update the current solution S and theflag improvement in case an improving neighbor was found and a new iteration resumes. The locallyoptimal solution S is returned in line 20.

Algorithm 2 BEST-IMPROVING(G,w, p, S)1: improvement ← .TRUE.2: while improvement = .TRUE. do3: improvement ← .FALSE.4: fbest ←∞5: for k = 1, . . . ,K do6: Let u ∈ S : u ∈ Vk7: for all v ∈ Vk : v 6= u do8: S′ ← S ∪ {v} \ {u}9: if f(S′) < fbest then

10: Sbest ← S′

11: fbest ← f(S′)12: end if13: end for14: end for15: if fbest < f(S) then16: S ← Sbest17: improvement← .TRUE.18: end if19: end while20: return S

The second local search procedure is an adaptation of Algorithm 2 that visits the groups of the graphin a random order, following a similar proposal in (Golden et al., 2008).

Example. Figure 4 displays the application of procedure BEST-IMPROVING (Algorithm 2) to thesolution built in the example of Section 3.1. The initial solution in Figure 4(a) is the same generalizedspanning tree in red in Figure 3(f) obtained by procedure RANDOMIZED-CONSTRUCTION, formed



by vertices {A,D,G,H} and edges {(A,D), (D,H), (G,H)}, with total cost 61 (the edge costs amountto 84, while the vertex prizes are 23). The generalized spanning trees in blue in Figures 4(b)-(e) are itsneighbors, obtained by replacing the representatives of groups V1, V2, V3, and V4, in this order. The bestneighbor appears again in Figure 4(f), with edges now in red, with total cost 54. The next local searchiteration is illustrated by Figures 5(g)-(j), which shows the neighbors of the best neighbor found by thefirst iteration. In this case, the cost of every neighbor is greater than that of the solution in Figure 4(f).Therefore, the solution in red in Figure 4(f) is locally optimal and is returned by the best-improving localsearch algorithm. �

3.3. GRASP

Algorithm 3 illustrates the pseudo-code of the basic GRASP heuristic for PC-GMSTP. After initializingthe value of the incumbent in line 1, the GRASP iterations are carried out in the while loop in lines 2to 9. A solution S is built by algorithm RANDOMIZED-CONSTRUCTION in line 3. As shown before,a greedy randomized algorithm may not always generate a locally optimal solution. Therefore, in line 4,local search is applied to solution S provided by the semi-greedy algorithm. We use LOCAL-SEARCHto denote any of the local search methods considered in Section 3.2. If the objective function value f(S)of the local minimum produced in line 4 is better than the objective function value fbest of the incumbent,then the local minimum is made the incumbent in line 6 and its objective function value is recorded asfbest in line 7. The while loop is repeated until some stopping criterion is satisfied. The best solutionSbest found over all GRASP iterations is returned in line 10.

Algorithm 3 GRASP(G,w, p, α)1: fbest ←∞2: while stopping criterion not satisfied do3: S ← RANDOMIZED-CONSTRUCTION(G,w, p, α)4: S ← LOCAL-SEARCH(G,w, p, S)5: if f(S) < fbest then6: Sbest ← S7: fbest ← f(S)8: end if9: end while

10: return Sbest

4. Intensification by path-relinking

Path-relinking is a major enhancement that adds a long-term memory mechanism to GRASP heuristics.GRASP with path-relinking implements this long-term memory by using an elite set of diverse high-quality solutions previously identified during the search. Basically, at each iteration the path-relinkingoperator is applied to the solution found at the end of the local search phase and to a randomly-selected



Fig. 4. Application of procedure BEST-IMPROVING (Part I).



Fig. 5. Application of procedure BEST-IMPROVING (Part II).

solution from the elite set, returning new solutions that are similar to both of them, but possibly better.To implement GRASP with path-relinking, we make use of an elite set E to store a number of high-

quality diverse solutions found during the search. The elite set starts empty and is constrained to haveat most nE different solutions. Each solution produced at the end of the GRASP local search phase maybe relinked with one or more solutions of the elite set. Each solution resulting from path-relinking isconsidered as a candidate to be inserted in the elite set, according to algorithm UPDATE-ELITE-SETdescribed in Section 9.2 of (Resende and Ribeiro, 2016).

The pseudo-code of Algorithm 4 outlines the main steps of procedure GRASP+PR implementing aGRASP with path-relinking heuristic for PC-GMSTP. This algorithm variant relinks the locally optimalsolution produced in each GRASP iteration with a single, randomly chosen, solution from the elite set,following the backward path-relinking strategy (Resende and Ribeiro, 2005, 2010, 2016; Resende et al.,2010; Ribeiro and Resende, 2012). The output of the path-relinking operator is a candidate for inclusionin the elite set. Line 1 of the pseudo-code initializes the elite set E as empty. The loop in lines 2–16



repeats the iterations that make up the steps of GRASP with path-relinking, until some stopping criterionis met. Lines 3 and 4 correspond to the greedy randomized construction and local search phases of thebasic GRASP heuristic in Algorithm 3. Backward path-relinking is performed in lines 5–14 in case theelite set is not empty. First, a solution S′ is randomly selected from the elite set E in line 6. Line 7checks if the number |∆(S, S′)| of groups whose representative is different in S and S′ is greater thanthree, in which case path-relinking will be performed. This is so because Resende and Ribeiro (2016)(Chapter 8) showed that the application of path-relinking to solutions differing by three or fewer movescan be discarded, since in this case no better solution than both extremities can be found. Lines 8–12perform path-relinking from the best to the worst among S and S′, interchanging the roles of S and S′

as arguments if necessary. The solution S resulting from path-relinking is tested for inclusion in the eliteset E in line 15, which updates E by applying algorithm UPDATE-ELITE-SET. The algorithm returnsthe best-valued elite solution in line 17.

Algorithm 4 GRASP+PR(G,w, p, α, nE )1: E ← ∅2: while stopping criterion not satisfied do3: S ← RANDOMIZED-CONSTRUCTION(G,w, p, α)4: S ← LOCAL-SEARCH(G,w, p, S)5: if |E| > 0 then6: Randomly select a solution S′ from the elite set E7: if |∆(S, S′)| > 3 then8: if f(S) ≤ f(S′) then9: S ← BACKWARD-PR(G,w, p, S, S′)

10: else11: S ← BACKWARD-PR(G,w, p, S′, S)12: end if13: end if14: end if15: UPDATE-ELITE-SET(S, E , nE )16: end while17: return Sbest = argmin{f(S) : S ∈ E}

Algorithm 5 is an implementation of backward path-relinking for PC-GMSTP, in which one seeksa generalized spanning tree that improves both extremities of a path leading from an initial solutionSi to a guiding solution Sg. We assume that the initial solution Si is at least as good as (and possiblybetter than) the guiding solution Sg, hence the qualification of this strategy as backward. The currentsolution S, the incumbent Sbest , and the cost fbest of the latter are initialized in lines 1–3. A path fromthe initial solution Si to the guiding solution Sg is created in the loop going from line 4 to 10. In line 4,the algorithm checks if the restricted neighborhood N(S : Sg) ⊆ N(S) contains at least one feasiblesolution. In fact, N(S : Sg) does not have to be explicitly computed and stored: instead, its elementsmay be implicitly enumerated on-the-fly. There is one move in the restricted neighborhood for eachgroup k = 1, . . . ,K such that its representative is different in S and S′. In this case, let vertex u be therepresentative of Vk in S and v its representative in the guiding solution Sg. The corresponding solution



is obtained as S′ = S ∪ {v} \ {u}. As the algorithm traverses the path from S to Sg, the best restrictedneighbor of the current solution is selected at each iteration in line 5. Lines 6 to 9 update the best solutionSbest and its cost fbest if a new best-quality solution is found. Since the best solution found along thepath from Si to Sg may not be locally optimal, local search is applied to it in line 11. Finally, line 12returns the best solution Sbest found by backward path-relinking.

Algorithm 5 BACKWARD-PR(G,w, p, Si, Sg)1: S ← Si

2: Sbest ← S3: fbest ← f(S)4: while |N(S : Sg)| ≥ 1 do5: S ← argmin{f(S′) : S′ ∈ N(S : Sg)}6: if f(S) < fbest then7: Sbest ← S8: fbest ← f(S)9: end if

10: end while11: Sbest ← LOCAL-SEARCH(G,w, p, Sbest)12: return Sbest

Since it is more likely to find an improving solution in the restricted neighborhood of the better so-lution than in that of the worse, backward path-relinking usually performs better than forward path-relinking. Back-and-forward path-relinking does at least as well as either backward or forward path-relinking, but takes about twice as long to compute, since two (usually distinct) paths of the same lengthare traversed. Computational experiments on many problems (see e.g. (Resende and Ribeiro, 2016))confirmed these findings.

5. Diversification by restarts

Resende and Ribeiro (2011, 2016) have shown that restart strategies are able to reduce the running timetaken by a GRASP with path-relinking heuristic to reach a target solution value for many problems.

A restart strategy is typically defined as an infinite sequence of time intervals τ1, τ2, τ3, . . . whichdefine epochs τ1, τ1 + τ2, τ1 + τ2 + τ3, . . . when the algorithm is restarted from scratch. It can be shownthat the optimal restart strategy uses τ1 = τ2 = · · · = τ∗, where τ∗ is some (unknown) constant.

Implementing an optimal strategy may be difficult in practice. Runtimes can vary greatly for differentcombinations of algorithm, instance, and solution quality sought. Since usually one has no prior infor-mation about the runtime distribution of the stochastic search algorithm for the optimization problemunder consideration, one runs the risk of choosing a value of τ∗ that is either too small or too large. Avalue that is too small can cause the restart variant of the algorithm to take much longer to converge thana no-restart variant. On the other hand, a value that is too large may never lead to a restart, causing therestart-variant of the algorithm to take as long to converge as the no-restart variant.

Resende and Ribeiro (2011, 2016) showed that in GRASP with path-relinking, the number of it-



erations between improvements of the best solution tends to vary less than the runtimes for differentcombinations of instance and solution quality sought. Therefore, a simple and effective restart strategyfor GRASP with path-relinking is to keep track of the last iteration when the incumbent solution wasimproved and restart the GRASP with path-relinking heuristic if κ iterations have gone by without im-provement. Each restart consists in saving the incumbent and emptying out the elite set. We shall callsuch a strategy restart(κ).

Applications reported e.g. by Interian and Ribeiro (2017); Pessoa et al. (2013); Ribeiro and Santos(2019) illustrate the speedups in the average times to target solution value obtained by the hybridizationof GRASP with path-relinking heuristics with restart strategies. The pseudo-code in Figure 6 summa-rizes the steps of a GRASP with path-relinking heuristic using the restart(κ) strategy for a minimizationproblem. The algorithm keeps track of the iteration counter CurrentIter , as well as of the last iterationLastImprov when an improving solution was found, which are initialized in lines 2 and 3, respectively.Line 5 updates the iteration counter. Lines 6 to 17 are equivalent to the corresponding segment in Al-gorithm 4. Lines 18 to 22 update the best solution Sbest , its cost fbest , and the iteration LastImprov inwhich this new best-quality solution was found. If line 23 determines that more than κ iterations havegone by since the last improvement of the incumbent, then a restart is triggered, emptying out the eliteset in line 24 and resetting the iteration of last improvement to the current iteration in line 25. If restartis not triggered, then the solution S resulting from path-relinking is tested for inclusion in the elite set Ein line 27, which updates E by applying algorithm UPDATE-ELITE-SET. Finally, the algorithm returnsthe best overall solution Sbest in line 30.

6. Computational experiments

In this section, we evaluate the performance of the GRASP heuristic with path-relinking and restartsfor PC-GMSTP developed in the previous sections. All variants of the heuristic were implemented inC++ with the GNU GCC compiler version 5.4.0. All experiments were performed on a computer with a4-processor 2.90 GHz Intel Core i5-4460S CPU with 8 GB of RAM running the operating system LinuxUbuntu 16.04 LTS of 64 bits.

Since the test instances used in (Golden et al., 2008; Pop et al., 2017) could not be obtained in theliterature, we generated new instances similar to that in (Golden et al., 2008). The graphs and the distri-bution of their vertices in groups have been provided by the authors of (Fischetti et al., 1997). The prizeshave been randomly generated in the interval [0, 10], as for some instances in Golden et al. (2008). The30 randomly generated instances named #gXXXX#n as in (Fischetti et al., 1997) are shown in Table 1,where #g denotes the number of groups, #n denotes the number of vertices, and XXXX#n is the problemname in the TSPLIB (Reinelt, 1991).

6.1. Parameter tuning

We selected ten training instances, marked in boldface in Table 1, for tuning the three main parametersof heuristic GRASP+PR, corresponding to 36 different configuration settings:

• nine values for the quality parameter α of the restricted candidate list of the construction phase:



Algorithm 6 GRASP+PR+RESTARTS(G,w, p, α, nE , κ)1: E ← ∅; fbest ←∞2: CurrentIter ← 03: LastImprov ← 04: while stopping criterion not satisfied do5: CurrentIter ← CurrentIter + 16: S ← RANDOMIZED-CONSTRUCTION(G,w, p, α)7: S ← LOCAL-SEARCH(G,w, p, S)8: if |E| > 0 then9: Randomly select a solution S′ from the elite set E

10: if |∆(S, S′)| > 3 then11: if f(S) ≤ f(S′) then12: S ← BACKWARD-PR(G,w, p, S, S′)13: else14: S ← BACKWARD-PR(G,w, p, S′, S)15: end if16: end if17: end if18: if f(S) < fbest then19: Sbest ← S20: fbest ← f(S)21: LastImprov ← CurrentIter22: end if23: if CurrentIter − LastImprov > κ then24: E ← ∅25: LastImprov ← CurrentIter26: else27: UPDATE-ELITE-SET(S, E , nE )28: end if29: end while30: return Sbest

α = 0.1, 0.2, . . . , 0.9;• two local search strategies: deterministic or randomized; and• two values for the size nE of the elite set: nE = 10, 20.

First, Algorithm 4 (with 200 iterations as the stopping criterion) was run for each instance once foreach of the 36 parameter configuration settings. The average running time over these 36 runs was set asthe reference time for that instance. Next, each instance was run ten times for each of the 36 parameterconfigurations, each of them starting with a different seed and for the duration of its reference time. Thebest value for each instance was collected after 360 runs of that instance. Using the best known value foreach training instance, we computed for each parameter configuration the number of best values (at mostten) found for that instance. Also, for each combination of training instance and parameter configuration,



Table 1Randomly generated instances.

PC-GMSTP instances

11EIL51 21EIL101 39RAT19514ST70 21LIN105 40KROA200

16EIL76 22PR107 40KROB20016PR76 25PR124 45TS225

20KROA100 26BIER127 46PR22620KROB100 28PR136 53GIL26220KROC100 29PR144 53PR26420KROD100 30KROA150 60PR29920KROE100 30KROB150 64LIN318

20RAT99 31PR152 88PR439

we calculated the gap (%) between the average value of the ten runs and the best known value. Extendingthe calculations over the ten instances, we computed for each parameter configuration the number of bestvalues found (at most 100) and the average gap. For each triple of parameter values, Figure 6 displaysa point whose abscissa represents the average gap of a parameter configuration over the ten traininginstances and the ordinate corresponds to the number of runs in which this test version matched the bestknown value. The best versions are those with either the smallest abscissas or the largest ordinates, whenthe above two evaluation criteria and fixed running times for all versions are considered. We selected asthe best version that with α = 0.8, using the deterministic local search strategy, and with the size of theelite population nE = 10, for which the average gap from the best known value was 0.00757% and thenumber of matches of the best value was 90 (out of a possible maximum of 100) (point labeled 29 inFigure 6). These parameter settings will be used in the remainder of the computational experiments.

6.2. Experiments on test instances

GRASP with path-relinking with the parameter settings determined in the previous experiment wasapplied to the 20 remaining instances. Each instance was run ten times with different seeds for 200iterations each. Detailed results are presented in Table 2. For each instance in this table, we indicate,over the ten runs, the average number of solutions in the elite set, the average number of iterations inwhich path-relinking was performed, the average and maximum running time of the heuristic in seconds,the average and best solution values (best values marked in boldface are optimal values obtained byCPLEX), and the average iteration counter where the best solution was found.

We first notice that 13 instances in this table have been exactly solved in less than five hours of runningtime. We observe that for all 20 test instances, the best known solution value was reached in at least onerun. For each of the 17 smallest test instances, the best known solution value was obtained in all ten runsand, therefore, the average is equal to the best value.

Our findings and numerical results for the instances generated following similar patterns to thosein Golden et al. (2008) go in the same direction of the conclusions of its authors. Table 4 of Golden



89

90

91

0.007 0.008 0.009 0.01 0.011 0.012 0.013 0.014 0.015

11 1217 18

29 25 2734

9

8 5

20

2426

2228

21

1033

7

219

31

16

23 32 3

13

414

35 36 16

15

30

Best va

lue

s

Average gap from best value

configurations

Fig. 6. Comparative analysis of the 36 test versions of GRASP+PR with different settings for the parameter α, the local searchstrategy, and the size nE of the elite set. The best parameter values led to an average gap from the best known value of

0.00757 % and to 90 matches of the best value (point labeled with 29).

et al. (2008) presented numerical results for 130 instances that could be exactly solved in two hours ofrunning time, out of a total of 169 instances they generated. In our case, CPLEX found optimal solutionsfor eight out of ten training instances and for 13 out of the 20 test instances in Table 2, showing thata large number of optimal solutions could be obtained in both studies and that even a straightforwardcommercial solver shows a good performance for such test problems. The genetic algorithm and the localsearch heuristic algorithms proposed in Golden et al. (2008) found optimal solutions for all instanceswith up to 200 vertices in Table 4 of Golden et al. (2008) in less than 40 seconds of average runningtime for each instance type. Our GRASP+PR algorithm obtained the best known solutions in all ten runsfor all instances with at most 200 vertices in Table 2 in only 84.73 seconds on average (over the tenruns). However, we observe that this average time, as well as all other running times reported in Table 2,corresponds to the execution of 200 iterations, although the last column in this table indicates that muchfewer iterations are needed to reach the best solution for all instances. The fact that the genetic algorithmin Golden et al. (2008) finds a significant number of optimal solutions after only 15 generations usinga mutation rate of 50% (which privileges mutations over learning), as also does the straightforwardlocal search algorithm using initial solutions randomly generated, is another indication that a significantnumber of the test instances are relatively easy to be solved by both heuristics and exact approaches.

Regarding the sensitivity to prize ranges, we generated additional test problems on the same graphswith vertex prizes on the intervals [0, 100] and [0, 1000] and belonging to {0, 1, 10, 100, 1000}, as Goldenet al. (2008) also did. Experiments on these instances confirmed previous findings that, in the general



Table 2Results obtained with heuristic GRASP+PR on the 20 test instances (ten runs for each instance).

Instance Avg. |E| Avg. #PR Avg. time (s) Max time (s) Avg. value Best value Avg. it.

11EIL51 7.0 95.1 0.64 0.66 62.0 62 2.016PR76 6.9 141.5 2.30 2.35 47127.0 47127 4.620KROA100 2.9 52.2 6.75 7.03 7661.0 7661 1.620KROB100 10.0 137.6 5.99 6.18 8147.0 8147 4.320KROC100 9.0 165.0 6.07 6.20 7923.0 7923 9.220KROD100 2.4 52.5 6.49 6.72 7192.0 7192 1.220KROE100 9.7 162.3 5.99 6.17 8060.0 8060 9.921EIL101 9.9 138.0 5.46 5.65 78.0 78 2.722PR107 10.0 192.2 8.87 9.12 20360.0 20360 40.025PR124 9.6 151.3 13.71 14.08 29071.0 29071 4.228PR136 10.0 189.8 19.33 19.91 37376.0 37376 15.429PR144 3.9 115.7 24.15 24.87 39902.0 39902 3.939RAT195 10.0 197.7 73.79 76.96 511.0 511 12.240KROA200 9.2 195.3 82.90 86.00 11461.0 11461 21.040KROB200 9.8 195.0 84.73 88.33 11467.0 11467 75.646PR226 2.0 0.0 106.27 113.28 55254.0 55254 1.053PR264 10.0 194.0 264.52 276.78 21637.0 21637 8.060PR299 10.0 198.3 445.09 483.71 19999.4 19995 71.464LIN318 10.0 198.2 476.16 494.10 18142.5 18140 80.888PR439 10.0 198.8 1885.55 2048.42 51093.8 51093 77.2

case, the problems become easier as the vertex prizes increase and become larger with respect to theedges weights, in which case the relevance of the location subproblem predominates over the networkdesign subproblem. In addition, we also generated problems with vertex prizes on the interval [0, 5].Interestingly, these problems revealed to be harder than those with prizes in [0, 10], once again confirmingthe previous observation.

6.3. Experiments with restarts and long runs

We evaluate the performance of the restart strategy for PC-GMSTP for κ = 50, 100, 200. Instances60PR299 and 64LIN318 have been selected for the experiments with restarts, since they are among thelargest and showed the largest variability on the best solution found over the ten runs.

Figures 7 and 8 display typical time-to-target plots for test instances 60PR299 (with the target valueset to 19998) and 64LIN318 (with the target value set to 18145), respectively, for GRASP+PR withoutrestarts and with the restart(κ) strategy for κ = 50, 100, 200. These plots make the case for the use ofrestarts, showing that at least one of the restart strategies almost always found the target with the sameprobability systematically faster than the strategy without restarts.

As already observed e.g. by Resende and Ribeiro (2011) and Interian and Ribeiro (2017), the effect of



restart strategies can be mainly noticed in the longest runs. Considering the 200 runs for each of the twoinstances with time-to-target plots depicted in Figures 7 and 8, the long runs are those associated with thecolumns corresponding to the fourth quartile in Tables 3 and 4, respectively. Entries in the fourth quartilecorrespond to those in the heavy tails of the runtime distributions. The restart strategies in general donot affect too much the other quartiles of the distributions, which is a desirable characteristic. Comparedto the strategy without restarts, the restart(100) strategy was the one that lead to the most significantreductions in the overall average running time and in the average running time in the fourth quartile.

0

0.2

0.4

0.6

0.8

1

0 500 1000 1500 2000 2500

Cum

ula

tive

pro

babili

ty

Time to target (seconds)

No restartsRestart(50)

Restart(100)Restart(200)

Fig. 7. Time-to-target plots for instance 60PR299 with target value set to 19998 (0.015% greater than the best known value19995).

Table 3For instance 60PR299, 200 independent runs were executed for each strategy. Each run was made to stop when a solution asgood as the target solution value 19998 was found. For each quartile, the table gives the average running times in seconds overall runs in that quartile. The average running times over the 200 runs are also given for each strategy.

Strategy 1st 2nd 3rd 4th Average

No restarts 26.65 85.65 195.82 578.19 221.58Restart(50) 26.65 85.26 183.53 498.57 198.50Restart(100) 26.65 85.65 192.30 476.72 195.33Restart(200) 26.65 85.65 195.82 512.57 205.17

Next, we performed long runs for the three largest instances, comparing in Table 5 the results for



0

0.2

0.4

0.6

0.8

1

0 500 1000 1500 2000 2500

Cu

mu

lative

pro

bab

ility


No restartsRestart(50)

Restart(100)Restart(200)

Fig. 8. Time-to-target plots for instance 64LIN318 with target value set to 18145 (0.028% greater than the best known value18140).

Table 4For instance 64LIN318, 200 independent runs were executed for each strategy. Each run was made to stop when a solution asgood as the target solution value 18145 was found. For each quartile, the table gives the average running times in seconds overall runs in that quartile. The average running times over the 200 runs are also given for each strategy.

Strategy 1st 2nd 3rd 4th Average

No restarts 51.82 167.00 365.75 759.69 336.06Restart(50) 51.32 156.64 323.21 782.78 328.49Restart(100) 51.31 163.94 341.07 713.84 317.54Restart(200) 51.13 163.59 358.55 754.33 331.90

two variants of the heuristic. The strategy without restarts was made to stop after 1000 iterations orone hour of execution time, whatever occurs first, while restart(100) was made to stop after the samerunning time taken by the previous one. Each instance was run ten times with a different seed. For eachinstance and each strategy, the table gives the average running time, the average solution value, and thebest overall solution value, over the ten runs. For the same running time, strategy restart(100) obtained abetter average cost value on instance 60PR299.

We observe that although restarts were shown to be very helpful to speedup GRASP with path-relinking, in this work they were not capable to improve the best solutions found since, at least for thetest instances considered here, the algorithm without restarts was already able to find the best solutions



in relatively few iterations, as illustrated by the results in Table 2.

Table 5Long runs: no restarts and restart(100), ten runs for each instance. Stopping criterion: 1000 iterations or one hour for no restarts,same running time as no restarts for restart(100).

no restarts(1000 iterations or 60 minutes)

restart(100)(same time as no restarts)

Instance Average Average Best Average Average Besttime (s) value value time (s) value value

60PR299 2233.66 19995.5 19995 2234.96 19995.0 1999564LIN318 2354.90 18140.0 18140 2356.24 18140.0 1814088PR439 3605.30 51093.0 51093 3610.82 51093.0 51093

6.4. Integration with a data mining strategy

Ribeiro et al. (2006) proposed a hybridization of the GRASP metaheuristic that incorporates a datamining process. In this hybrid DM-GRASP (which stands for Data Mining GRASP) scheme, a datamining process extracts patterns from an elite set of solutions obtained after a significant number ofGRASP iterations. These patterns are then used to guide the GRASP construction phase in the followingiterations. The authors’ main goal was to give evidence that using a data mining technique could improvethe results obtained by GRASP. Computational experiments on the set packing problem showed thatemploying frequent patterns mined from an elite set of solutions lead GRASP to achieve better results.Besides, additional experiments showed that the data mining strategy accelerated the process of findinga good solution by the GRASP procedure.

The experiments performed by Plastino et al. (2014) to solve the server replication for reliable multi-cast problem showed that the Multi DM-GRASP strategy (MDM-GRASP), which explores the gradualevolution of the elite set of solutions, obtained better results than DM-GRASP in less computationaltime. Multiple and adaptive executions of the data mining process improved the results obtained by theDM-GRASP heuristic. Additional developments and analyses, regarding the hybridization of more so-phisticated GRASP approaches, showed that incorporating a data mining technique can also improvemore competitive GRASP strategies, such as reactive GRASP (Prais and Ribeiro, 2000).

These results motivated the hybridization of the GRASP+PR+RESTARTS heuristic with algorithmMDM-GRASP described in Section 6 of (Plastino et al., 2014). The main principle of this implementa-tion is to execute the mining process every time the elite set becomes stable, i.e., after a given number ofiterations have passed since the last change in the elite set occurred. The assumption is that mining morefrequently, and not only once, should make it possible to explore the gradual evolution of the elite setand allow the extraction of more refined patterns. Also, since the data mining process occurs only whenthe elite set is stable, the MDM-GRASP implementation may have a more self-adapted behavior to theproblem being solved.

Algorithm MDM-GRASP+PR+RESTARTS was applied with parameter values determined by com-putational tests on the set of training instances. The elite set is considered to become stable when it is



full (ten solutions) and after ten consecutive iterations without change. A set of representative vertices isconsidered a pattern if it appears in at least seven of the ten elite solutions (70% support). The ten largestpatterns found are inserted in the pattern set.

Table 6 shows comparative results for ten runs of each of the three variants GRASP+PR,GRASP+PR+RESTARTS, and MDM-GRASP+PR+RESTARTS on the three largest test instances. Eachrun of each algorithm used as stopping criterion the maximum time in Table 2 for the correspondinginstance. We observe that although the best solution values could not be improved, algorithm MDM-GRASP+PR+RESTARTS found the best known solution value in considerably more runs and, conse-quently, obtained better average solution values (see values in boldface).

Table 6Comparative results for GRASP+PR, GRASP+PR+RESTARTS, and MDM-GRASP+PR+RESTARTS: ten runs for each in-stance, running time is the same maximum time in Table 2 for the corresponding instance.

GRASP+PR GRASP+PR+RESTARTS MDM-GRASP+PR+RInstance Avg. value # best Avg. value # best Avg. value # best

60PR299 19999.4 3 19997.6 3 19996.4 964LIN318 18142.5 7 18142.5 7 18140.0 1088PR439 51093.8 9 51094.1 9 51093.0 10

Figure 9(a) illustrates the evolution of the cost of the solutions obtained at the end of the construction,local search, and path-relinking phases of algorithm MDM-GRASP+PR+RESTARTS along its 500 firstiterations for one run of instance 60PR299. Figure 9(b) gives a closer view of the solution values obtainedat the local search and path-relinking phases. In both figures, each vertical dashed line indicates a miningprocess, while each continuous vertical line indicates a restart. The figures show that local search reducesthe cost of the solutions obtained by the construction phase and that path-relinking further improves thesolutions obtained by local search. In fact, the best known solution value was reached four times by localsearch and 34 times by the path-relinking phase. We can also observe that there is a clear improvementin the quality of the solutions obtained by all algorithm phases after iteration 79, when data mining isfirst applied. Overall, solution quality continuously improves, mainly after the first restart and miningapplications.

The time-to-target plots in Figure 10 have been obtained from 200 runs of algorithms GRASP+PR,GRASP+PR+RESTARTS using the restart(100) strategy, and MDM-GRASP+PR+RESTARTS on in-stance 60PR299 with the same target value considered in Figure 7. They illustrate the typical situationobserved for most instances. Algorithm MDM-GRASP+PR+RESTARTS performs better than the othertwo variants, since its plot is the leftmost one, indicating a much higher probability that algorithm MDM-GRASP+PR+RESTARTS will find a target solution value faster than the others.

7. Concluding remarks

Given a graph with its vertex set partitioned into a set of groups, non-negative costs associated to itsedges, and non-negative prizes associated to its vertices, the prize-collecting generalized minimum span-ning tree problem consists in finding a subtree of this graph spanning exactly one vertex of each group



20000

21000

22000

23000

24000

25000

26000

1 50 100 150 200 250 300 350 400 450 500

Co

st

Iteration

ConstructionLocal search

Path-relinkingBest known (19995)

(a) Solution costs at the end of the construction, local search, and path-relinking phases.

19900

20000

20100

20200

20300

20400

20500

20600

20700

20800

1 50 100 150 200 250 300 350 400 450 500

Co

st

Iteration

Local searchPath-relinking

Best known (19995)

(b) Detailed solution costs at the end of the local search and path-relinking phases.

Fig. 9. 500 iterations of MDM-GRASP+PR+RESTARTS on instance 60PR299.

and minimizing the sum of the costs of the edges of the tree less the prizes of the selected vertices. It isa generalization of the NP-hard generalized minimum spanning tree optimization problem.

We proposed a GRASP heuristic for the approximate solution of the prize-collecting generalized mini-mum spanning tree problem, incorporating path-relinking for search intensification and a restart strategy



0

0.2

0.4

0.6

0.8

1

0 500 1000 1500 2000 2500

Cu

mu

lative

pro

ba

bili

ty


GRASP+PRGRASP+PR+RESTARTS

MDM-GRASP+PR+RESTARTS

Fig. 10. Time-to-target plots for instance 60PR299 with target value set to 19998 (0.015% greater than the best known value19995).

for search diversification. The computational experiments showed that the heuristic developed in thiswork performed very well, obtaining very good suboptimal solutions for test problems with up to 439vertices in reasonable computation times. The algorithm is very robust, systematically finding the samesolution in almost all runs.

Although restarts were shown to be very helpful to speed up GRASP with path-relinking, they werenot capable to improve the best solutions found. This was so because at least for the test instancesconsidered here, the algorithm without restarts was already able to find the best solutions in relativelyfew iterations.

In addition, we also noticed that the hybridization of the GRASP with path-relinking and restarts witha data mining strategy that is applied along the GRASP iterations, after the elite set is modified andbecomes stable, contributed to make the heuristic more robust.

We observed in Section 6.2 that the data for the test instances previously used in (Golden et al., 2008;Pop et al., 2017) could not be obtained, for which reason we generated new, similar instances for thecomputational experiments performed in this work. Comparative results with the findings of Goldenet al. (2008) indicated that the pattern they proposed for the generation of test instances seems to leadto relatively easy to be solved instances by both heuristics and exact approaches. Therefore, we madeavailable at Mendeley Data all input data and detailed numerical results for the instances generated inthis work (Marzo and Ribeiro, 2019) and we encourage other authors to contribute with better solutionsand more challenging instances for PC-GMSTP.



Acknowledgments

The authors are grateful to Paolo Toth and Albert Muritiba for sending the data needed to generatethe test instances described in Section 6, which are now available at Mendeley Data (Marzo and Ribeiro,2019). The authors are also thankful to an anonymous referee, who contributed to improve the readabilityof this work. Work of Ruslan G. Marzo was supported by a CAPES scholarship. Work of Celso C.Ribeiro was partially supported by CNPq research grants 303958/2015-4 and 425778/2016-9 and byFAPERJ research grant E-26/202.854/2017. This work was also partially sponsored by Coordenacao deAperfeicoamento de Pessoal de Nıvel Superior (CAPES), under Finance Code 001.

References

Bajwa, N., Melouk, S., Bryant, P., 2019. A hybrid heuristic approach to minimize number of tardy jobs in group technologysystems. International Transactions in Operational Research 26, 1847–1867.

Climaco, G., Rosseti, I., Simonetti, L., Guerine, M., 2019. Combining integer linear programming with a state-of-the-artheuristic for the 2-path network design problem. Interational Transactions in Operational Research 26, 615–641.

Cockayne, E., Melzak, Z., 1968. Steiner’s problem for set-terminals. Quarterly of Applied Mathematics 26, 213–218.Contreras-Bolton, C., Gatica, G., Barra, C., Parada, V., 2016. A multi-operator genetic algorithm for the generalized minimum

spanning tree problem. Expert Systems with Applications 50, 1–8.Cormen, T., Leiserson, C., Rivest, R., Stein, C., 2009. Introduction to Algorithms (3rd edn.). MIT Press, Cambridge.Dror, M., Haouari, M., Chaouachi, J., 2000. Generalized spanning trees. European Journal of Operational Research 120,

583–592.Duin, C., 1994. Steiner’s problem in graphs: Approximation, reduction, variation. Ph.D. thesis, Faculteit der Economische

Wetenschappen en Econometrie, Universiteit van Amsterdam, Amsterdam.Duin, C., Volgenant, A., Voß, S., 2004. Solving group Steiner problems as Steiner problems. European Journal of Operational

Research 154, 323–329.Duin, C., Voß, S., 1999. The Pilot method: A strategy for heuristic repetition with application to the Steiner problem in graphs.

Networks 34, 181–191.Feremans, C., 2001. Generalized spanning trees and extensions. Ph.D. thesis, Universite Libre de Bruxelles, Brussels.Feremans, C., Labbe, M., Laporte, G., 2002. A comparative analysis of several formulations for the generalized minimum

spanning tree problem. Networks 39, 29–34.Feremans, C., Labbe, M., Laporte, G., 2004. The generalized minimum spanning tree problem: Polyhedral analysis and branch-

and-cut algorithm. Networks 43, 71–86.Ferreira, C., Ochi, L., Parada, V., Uchoa, E., 2012. A GRASP-based approach to the generalized minimum spanning tree

problem. Expert Systems with Applications 39, 3526–3536.Festa, P., Resende, M., 2009a. An annotated bibliography of GRASP, Part I: Algorithms. International Transactions in Opera-

tional Research 16, 1–24.Festa, P., Resende, M., 2009b. An annotated bibliography of GRASP, Part II: Applications. International Transactions in

Operational Research 16. 131–172.Fischetti, M., Salazar Gonzalez, J., Toth, P., 1997. A Branch-and-Cut Algorithm for the Symmetric Generalized Traveling

Salesman Problem. Operations Research 45, 378–394.Golden, B., Raghavan, S., Stanojevic, D., 2008. The prize-collecting generalized minimum spanning tree problem. Journal of

Heuristics 14, 69–93.Golden, B., Raghavan, S., Stanojevic, D., 2005. Heuristic Search for the Generalized Minimum Spanning Tree Problem.

INFORMS Journal on Computing 17, 290–304.Haouari, M., Chaouachi, J., Dror, M., 2005. Solving the generalized minimum spanning tree problem by a branch-and-bound

algorithm. Journal of the Operational Research Society 56, 382–389.Interian, R., Ribeiro, C.C., 2017. A GRASP heuristic using path-relinking and restarts for the Steiner traveling salesman



problem. International Transactions in Operational Research 24, 1307–1323.Kruskal, J., 1956. On the Shortest Spanning Subtree of a Graph and the Traveling Salesman Problem. Proceedings of the

American Mathematical Society 7, 48–50.Marzo, R., Ribeiro, C., 2019. Test Instances for the Prize-Collecting Generalized MST Problem. http://dx.doi.org/10.17632/

vftc7kg6ty.2. Type: dataset.Myung, Y.S., Lee, C.H., Tcha, D.W., 1995. On the generalized minimum spanning tree problem. Networks 26, 231–241.Pessoa, L.S., Resende, M.G., Ribeiro, C.C., 2013. A hybrid lagrangean heuristic with GRASP and path-relinking for set k-

covering. Computers & Operations Research 40, 3132–3146.Plastino, A., Barbalho, H., Santos, L., Fuchshuber, R., Martins, S., 2014. Adaptive and multi-mining versions of the DM-

GRASP hybrid metaheuristic. Journal of Heuristics 20, 39–74.Pop, P., 2002. The generalized minimum spanning tree problem. Ph.D. thesis, University of Twente, Enschede.Pop, P., 2004. New Models of the Generalized Minimum Spanning Tree Problem. Journal of Mathematical Modelling and

Algorithms 3, 153–166.Pop, P., 2007. On the prize-collecting generalized minimum spanning tree problem. Annals of Operations Research 150,

193–204.Pop, P., Matei, O., Sabo, C., Petrovan, A., 2018. A two-level solution approach for solving the generalized minimum spanning

tree problem. European Journal of Operational Research 265, 478–487.Pop, P.C., Matei, O., Sitar, C.P., Danciulescu, D., 2017. A genetic algorithm based solution approach to solve the prize-collecting

generalized minimum spanning tree problem. In Dessouky, M., Cruz-Machado, V. and Grilo, A. (eds), Proceedings of the47th International Conference on Computers and Industrial Engineering, Lisbon, pp. 223.1–223.8.

Prais, M., Ribeiro, C.C., 2000. Reactive GRASP: An Application to a Matrix Decomposition Problem in TDMA TrafficAssignment. INFORMS Journal on Computing 12, 164–176.

Prim, R., 1957. Shortest connection networks and some generalizations. Bell Labs Technical Journal 36, 1389–1401.Reinelt, G., 1991. TSPLIB– A Traveling Salesman Problem Library. ORSA Journal on Computing 3, 376–384.Resende, M., Ribeiro, C., 2010. Greedy randomized adaptive search procedures: Advances and applications. In Gendreau, M.

and Potvin, J.Y. (eds), Handbook of metaheuristics. Springer, New York, (2nd edn.), pp. 293–319.Resende, M., Ribeiro, C., 2016. Optimization by GRASP: Greedy Randomized Adaptive Search Procedures. Springer, New

York.Resende, M., Ribeiro, C., 2019. Greedy randomized adaptive search procedures: Advances and applications. In Gendreau, M.

and Potvin, J.Y. (eds), Handbook of metaheuristics. Springer, New York, (3rd edn.), pp. 169–220.Resende, M., Ribeiro, C., Glover, F., Martı, R., 2010. Scatter search and path-relinking: Fundamentals, advances, and ap-

plications. In Gendreau, M. and Potvin, J.Y. (eds), Handbook of metaheuristics. Springer, New York, (2nd edn.), pp.87–107.

Resende, M.G.C., Ribeiro, C.C., 2005. GRASP with path-relinking: Recent advances and applications. In Ibaraki, T., Nonobe,K. and Yagiura, M. (eds), Metaheuristics: Progress as real problem solvers. Springer, New York, pp. 29–63.

Resende, M.G.C., Ribeiro, C.C., 2011. Restart strategies for GRASP with path-relinking heuristics. Optimization Letters 5,467–478.

Ribeiro, C.C., Resende, M.G.C., 2012. Path-relinking intensification methods for stochastic local search algorithms. Journalof Heuristics 18, 193–214.

Ribeiro, C.C., Santos, P.L.F., 2019. A heuristic for the minimum cost chromatic partition problem. RAIRO Operations ResearchTo appear.

Ribeiro, M., Plastino, A., Martins, S., 2006. Hybridization of GRASP metaheuristic with data mining techniques. Journal ofMathematical Modelling and Algorithms 5, 23–41.

Shyu, S., Yin, P., Lin, B., Haouari, M., 2003. Ant-Tree: An ant colony optimization approach to the generalized minimumspanning tree problem. Journal of Experimental & Theoretical Artificial Intelligence 15, 103–112.

Stanojevic, D., Golden, B., Raghavan, S., 2004. The Prize-Collecting Generalized Minimum Spanning Tree Problem. In 7thINFORMS Telecommunications Conference, Boca Raton.


Documents

A GRASP with path-relinking and restarts heuristic …celso/artigos/PCGMSTP.pdf2 R.G. Marzo and C.C. Ribeiro / Intl. Trans. in Op. Res. 27 (2020) 1–29 and j2V ‘. If k6= ‘, then