Racing Tracks Improvisation

Jiao Jian WangUniversity of Bonn

Bonn, GermanyEmail: wangjiaojian@uni-bonn.com

Olana MissuraUniversity of Bonn

Bonn, GermanyEmail: olana.missura@uni-bonn.com

Abstract—Procedural content generation is a popular tech-nique in the game development. One of its typical applicationsis generation of game levels. This paper presents a methodto generate tracks for racing games, by viewing racing trackgeneration as a discrete sequence prediction problem. To solve itwe combine two techniques from music improvisation. We showthat this method is capable of generating new racing tracks whichappear to be interesting.

I. INTRODUCTION

In recent years, procedural content generation (PCG) notonly has been used in some of the widely successful commer-cial games [1, 2], it has also received a lot of attention fromresearchers [3, 4, 5]. One of the possible application of PCG isgenerating tracks for racing games. Imagine a PCG algorithmthat does not simply generate new content, but learns fromthe given input tracks. This allows it to produce tracks thatposess similar structural properties. On the one hand, this canaugment the human designers’ creativity by allowing them toconcentrate on creating truly novel levels. On the other hand,it also increases the replayability of the game, since the tracksgenerator can keep generating new tracks for any single playerindividually in the post-release phase.

There have already been a few studies concerned with theracing tracks generation using search based approaches [8, 6,7]. Their success depends on the fitness function, which inturn depends on the player satisfaction. One way to estimatethis value is to construct or to learn a player model, anotheris to involve the players themselves in the selection process.Creating player models is a complicated learning task in itself,while including players in the process of evolving tracks relieson the assumption that a casual player would be interested insuch task.

In contrast to the above approaches, we present a con-structive method which is capable of generating racing tracksaccording to a player’s preferences in real-time and requiresfrom a player only selecting one or more tracks to use as aninput. At the development stage the generator would use as aninput tracks created by the game designers.

Our solution adopts methods from music improvisation[9, 10], turning racing track generation into a discrete sequenceprediction problem. Under the assumption that the interest-ingness of a track for a particular player is a result of itsstructural features, we first build a model from the tracks thatthe player selects. Using this model we construct new trackswhich, despite being different, still carry strong resemblancein structure to the input ones. We describe how it is done inSection II. To validate our approach, we have implemented a

-70

-60

-50

-40

-30

-20

-10

0

10

-10 -5 0 5 10 15 20 25 30

’1.dat’

(a) ‘rectangle’

0

20

40

60

80

100

120

140

-8-6-4-2 0 2 4 6

’2.dat’

(b) ‘zigzag’

Fig. 1: Example training patterns.

racing track generator for a popular racing game, Trackmania[11]. The setup and results are described in Section III.

II. ALGORITHM

A racing track can be viewed as a discrete sequencemade of different track blocks (straight lines, curves, etc.).More formally, we use an alphabet Σ = {s, r, l}, where srepresents a straight line, r and l represent a right and a leftturn respectively. Each of the three elements corresponds toa building block of the same length. The turns have a fixedvalue of 9 degrees. A single track S is a sequence of fixedlength N , constructed of the primitive elements, S ∈ ΣN .

Imagine that we have as training patterns two tracksfrom Fig 1, ‘rectangle’ and ‘zigzag’, their alphabet repre-sentations being ‘rrrrrrrrrss...rrrrrrrrrss..’ and ‘llllllllllllllllllllr-rrrrrrrrrrrrrrrrrrr...’ respectively. Two tracks have quite differentstructural features: ‘rectangle’ has long straight stretches withturns in one direction only, while ‘zigzag’ has alternating turnswith no straight runs in-between. Presented with these trainingexamples, the algorithm should generate new tracks that aresufficiently different but possess the same structural features.

Using a sequence representation, generating a new trackcan be viewed as predicting the next symbol in a new trackrepeatedly. In order to solve this discrete sequence predictiontask, we combined two algorithms. One of them is a losslesscompression algorithm named LZ-MS [12], used to extract thevariable-order Markov model [13] from the training patterns.The other one is an optimal string matching algorithm calledfactor oracle [14] that can be used for the transition rulemining. Together, these two algorithms create a charactermodel of the input sequences. From these models we generatenew sequences that maintain the same variable-order Markovmodel and transition rule as the input sequences.

978-1-4799-3547-5/14/1.00 ©2014 IEEE

Algorithm 1 LZ-MS, incremental parsing

Input: input sequence: s ∈∑∗, input shift: I , back shift: B

Output: phrase dictionary: d1: Initialize: d← ∅2: for i = 1 to I do3: let p be a empty phrase, p← ∅4: m← 05: for j = i to |s| do6: p← p + sj7: if p /∈ d then8: add p into d9: p← ∅

10: j ← max(m, j −B)11: m← j12: end if13: end for14: end for15: return d

A. Variable-order Markov model

Begleiter et al have shown that the discrete sequenceprediction problem can be solved via a lossless compressionalgorithm [13], for instance, LZ-MS [12], a variant of LZ78[15]. To this purpose, LZ-MS extracts from training patternsthe variable-order Markov model.

The LZ-MS algorithm first builds a phrase dictionary fromthe input sequence S ∈

∑∗ through incremental parsing (seeAlg 1). The phrase dictionary is used to construct a contexttree: Starting from the root labeled ∅, the context tree is grown,such that for every phrase in a dictionary there is a path fromthe root to a node in the tree. Afterwards, all the leaf nodesare transformed into internal nodes by adding children to themlabeled with the elements of Σ. The last step in constructingthe context tree is to weigh each leaf by 1 and each internalnode by the sum of weights of its offsprings. An example ofthe context tree for the input string ‘ssslsslr’ is shown in Fig.2.

To generate a track from a given context tree, the elementsof the track are drawn from the alphabet Σ with probabilitiesspecified by the tree. To calculate the probability P (c|S),where c ∈ Σ and S ∈ Σ∗, the tree is traversed along thepath corresponding to S and the probability is read off of thenext child corresponding to c. For example, given the tree inFigure 2, if we need to compute the P (l|ss), we first travelthe tree from the root and follow the path sequence ‘ss’, andstop at node ‘s’ which has a weight of 9. Its child node ‘l’ hasa weight of 7, therefore P (l|ss) = 7/9 = 0.78. If we reachthe leaf node before the end of the sequence S, we go back tothe root and continue. To learn from multiple training patterns,each one is processed sequentially and the context tree is builtfrom the resulting joined phrase dictionary.

Examples generated by LZ-MS from the ‘rectangle’ and‘zigzag’ training patterns are shown in Fig. 3, and Fig. 4respectively.

B. Factor Oracle

The variable-order Markov model alone is sufficient formost music applications. However, unlike music in which

Algorithm 2 Factor oracle

Input: input sequence: p ∈∑∗

Output: factor oracle1: add state 02: add suffix link s(0)← −13: for i = 1 to |p| do4: add pi into factor oracle using Alg. 35: end for6: return factor oracle

all possible transitions are valid, the racing tracks sometimesrequire certain transition rules to hold, which creates con-straints not represented in the context tree. Futhermore, theMarkov based approach relies on maintaining the resemblancebetween training patterns and generated patterns in orderto ensure interestingness. Intuitively, should the resemblanceexceed certain threshold, the human will perceive the generatedpatterns as nothing more than mere copies of training patterns.Therefore, it is also important to measure how different thegenerated patterns are from the training patterns.

We solved the above issue by merging the variable-orderMarkov model with a transition rule mining algorithm calledfactor oracle [14, 16], which is an automaton built upon a givensequence. It was invented for optimal string matching. Givenan input sequence S ∈ Σ∗, its factor oracle consists |S| + 1states. On each state i in factor oracle, the transitions t(i)point to the end positions of the first occurrence of factors ofS, and the suffix link s(i) points to the end position of the firstoccurrence of the maximal suffix of prefix(i) which occurs atleast twice in prefix(i). The pseudo code of incremental factororacle construction are shown in Alg. 2 and Alg. 3. An exampleof factor oracle constructed from sequence ‘ssslsslr’ is shownin Fig. 5.

It was shown in [16] that factor oracles can be usedfor music improvisation with an additional parameter p, themutation rate. The generation procedure starts from the state0 in the factor oracle, then travels along the arrows and copythe labels of the transitions it passes into the new track. Ateach state i, with probability p the process may jump through

Algorithm 3 Factor oracle, incremental

Input: factor oracle, symbol: piOutput: factor oracle

1: add state i2: add transition from state i− 1 to i by pi3: k ← s(i− 1)4: while k > −1 and there is no trasition from state k by pi

do5: add transition from state k to i by pi6: k ← s(k)7: end while8: if k == −1 then9: add suffix link s(i)← 0

10: else11: add suffix link s(i) ← where leads the transition from

k by pi12: end if13: return factor oracle

Root,19

l,5

l,1 r,1 s,3

l,1 r,1 s,1

r,3

l,1 r,1 s,1

s,11

l,1 r,1 s,9

l,7

l,1 r,3

l,1 r,1 s,1

s,3

l,1 r,1 s,1

r,1 s,1

Fig. 2: Context tree build from the sequence ‘ssslsslr’

-30

-25

-20

-15

-10

-5

0

5

-40 -30 -20 -10 0 10 20 30 40 50

’1_ov_0.dat’

-120

-100

-80

-60

-40

-20

0

-20-15-10 -5 0 5 10 15 20 25

’1_ov_1.dat’

-20

-10

0

10

20

30

40

50

60

70

80

90

-50 -40 -30 -20 -10 0 10

’1_ov_2.dat’

-40

-35

-30

-25

-20

-15

-10

-5

0

-20 -10 0 10 20 30 40 50

’1_ov_3.dat’

-30

-25

-20

-15

-10

-5

0

5

10

-20 -15 -10 -5 0 5 10 15 20 25 30

’1_ov_4.dat’

-80

-70

-60

-50

-40

-30

-20

-10

0

-40 -30 -20 -10 0 10 20 30

’1_ov_5.dat’

-70

-60

-50

-40

-30

-20

-10

0

-80 -70 -60 -50 -40 -30 -20 -10 0 10 20

’1_ov_6.dat’

-60

-50

-40

-30

-20

-10

0

0 10 20 30 40 50 60 70

’1_ov_7.dat’

-30

-20

-10

0

10

20

30

40

50

-40 -35 -30 -25 -20 -15 -10 -5 0 5 10

’1_ov_8.dat’

-40

-30

-20

-10

0

10

20

30

-10 0 10 20 30 40 50 60

’1_ov_9.dat’

Fig. 3: Examples generated from ‘rectangle’, using LZ-MS

-30

-25

-20

-15

-10

-5

0

5

10

15

-50 -40 -30 -20 -10 0 10

’2_ov_0.dat’

-90

-80

-70

-60

-50

-40

-30

-20

-10

0

10

-20 -10 0 10 20 30 40 50 60

’2_ov_1.dat’

-10

0

10

20

30

40

50

60

70

-35 -30 -25 -20 -15 -10 -5 0 5 10

’2_ov_2.dat’

-20

-15

-10

-5

0

5

10

15

20

25

-30 -25 -20 -15 -10 -5 0 5 10 15 20 25

’2_ov_3.dat’

-80

-70

-60

-50

-40

-30

-20

-10

0

10

-5 0 5 10 15 20 25 30 35

’2_ov_4.dat’

-15

-10

-5

0

5

10

15

20

-10 -5 0 5 10 15 20 25 30

’2_ov_5.dat’

-10

0

10

20

30

40

50

60

-10 0 10 20 30 40 50 60

’2_ov_6.dat’

-30

-25

-20

-15

-10

-5

0

5

-40 -30 -20 -10 0 10 20 30

’2_ov_7.dat’

-90

-80

-70

-60

-50

-40

-30

-20

-10

0

10

-5 0 5 10 15 20 25 30 35

’2_ov_8.dat’

-35

-30

-25

-20

-15

-10

-5

0

5

-70 -60 -50 -40 -30 -20 -10 0 10

’2_ov_9.dat’

Fig. 4: Examples generated from ‘zigzag’, using LZ-MS

the suffix link to state s(i), then from there pick a transitiontn ∈ t from s(i) randomly and jump through it to statetn(s(i)). If the mutation rate is set to 0, then a sequenceidentical to traning pattern will be generated. Therefore, thisparameter controls the similarity between the training patternand generated patterns.

Examples generated using factor oracle are illustrated inFig. 7, and Fig. 8.

Learning from multiple training patterns is also possible,but not without a small modification. Sharing the same starting

state, one state line is constructed for each training pattern.These state lines are not constructed sequentially, rather inturns, one symbol from every pattern at a time. An exampleof multi-patterns factor oracle is shown in Fig. 6.

C. Combining factor oracle and LZ-MS

While the factor oracle faithfully represents the transitionsfrom the input patterns, it does not have a probability model.To have both, we combine LZ-MS with the factor oracleand replace the mutation rate with the probabilities from thevariable-order Markov model.

0 1s 0 1 2s s 0 1 2 3s s s

0 1 2 3 4s

l

s

l

s

l

l

0 1 2 3 4 5s

l

s

l

s

l

l s

0 1 2 3 4 5 6s

l

s

l

s

l

l s s

0 1 2 3 4 5 6 7s

l

s

l

s

l

l s s l

0 1 2 3 4 5 6 7 8s

l

r

s

l

s

l

l s

r

s l r

Fig. 5: Factor oracle constructed from the sequence ‘ssslsslr’. Solid arrows are transitions, dashed arrows are suffix links.

The generation process starts with a factor oracle and a LZ-MS context tree constructed from the same training patterns,and traverses them simultaneously and correspondingly. Oneach factor oracle state i, the candidates transition list normallyconsists the transition to the next state i+ 1 as well as all thetransitions from suffix link s(i). However, the suffix link s(i)should be ingored if it leads back to the state 0, with the onlyexception being that the current state i is the last state. This isunder the assertion that only the tails of the training patternscan link back to their heads.

All the transitions in the candidates list are assignedprobabilities that are read off the LZ-MS context tree, namelythe current vertex and its children. Then one of the transitionsfrom the candidates list is chosen in accordance with theseprobabilities, the generator copies the resulting label into the

new track, and traverses the factor oracle and the context treeaccordingly. The pseudo code of this generation process isgiven in Alg. 4.

To quantitavely evaluate the resulting tracks we look attwo particular metrics: The mutation rate as the measure ofdiversity of new tracks, and the average log-loss as the measureof similarity to the training patterns.

The average mutation rates of the factor oracle & LZ-MSfor ‘rectangle’ and ‘zigzag’ training patterns are listed in TableI.

In order to test the similarity in probability models betweenthe training patterns and the generated patterns, we calculatethe average log-loss as proposed in [13]. Given a test sequenceStest = (c1 . . . cN ) ∈ ΣN , the average log-loss is defined as

0

1 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20 21 22

23 24 25 26 27 28 29 30 31 32 33 34

s

r

s

l

l

s

r

l

s

r

s s

r

l

s l s r r

s

r

l

s

l

s

l

l l

s

r

s r r

s

r s s

s r r

s

r

s

l

r

s

s rs

s l r l

Fig. 6: Factor oracle constructed from sequence ”ssssllsrrrss”, ”rllssslsrr” and ”ssrrrrsrslrl”

-25

-20

-15

-10

-5

0

5

10

15

20

0 10 20 30 40 50 60 70 80

’1_fo_0.dat’

-40

-35

-30

-25

-20

-15

-10

-5

0

5

10

-10 0 10 20 30 40 50 60

’1_fo_1.dat’

-30

-20

-10

0

10

20

30

40

-15 -10 -5 0 5 10 15 20 25 30

’1_fo_2.dat’

-20

-10

0

10

20

30

40

-10 0 10 20 30 40 50 60

’1_fo_3.dat’

-60

-50

-40

-30

-20

-10

0

-10 -5 0 5 10 15 20 25 30

’1_fo_4.dat’

-60

-50

-40

-30

-20

-10

0

10

-15 -10 -5 0 5 10 15 20 25

’1_fo_5.dat’

-40

-35

-30

-25

-20

-15

-10

-5

0

-50 -40 -30 -20 -10 0 10

’1_fo_6.dat’

-40

-35

-30

-25

-20

-15

-10

-5

0

-10 0 10 20 30 40 50 60 70

’1_fo_7.dat’

-25

-20

-15

-10

-5

0

5

10

15

20

25

-5 0 5 10 15 20 25 30 35 40 45

’1_fo_8.dat’

-35

-30

-25

-20

-15

-10

-5

0

5

-10 0 10 20 30 40 50 60

’1_fo_9.dat’

Fig. 7: Examples generated from ‘rectangle’, using factor oracle with 0.05 mutation rate

-40

-35

-30

-25

-20

-15

-10

-5

0

5

10

0 10 20 30 40 50 60

’2_fo_0.dat’

0

5

10

15

20

25

30

35

40

45

-15 -10 -5 0 5 10 15 20 25 30 35 40

’2_fo_1.dat’

0

10

20

30

40

50

60

70

80

-20 -10 0 10 20 30 40 50 60

’2_fo_2.dat’

0

5

10

15

20

25

30

35

40

45

-30 -25 -20 -15 -10 -5 0 5 10 15

’2_fo_3.dat’

0

10

20

30

40

50

60

70

80

90

-25 -20 -15 -10 -5 0 5 10 15

’2_fo_4.dat’

0

20

40

60

80

100

120

-30-25-20-15-10 -5 0 5 10

’2_fo_5.dat’

-80

-70

-60

-50

-40

-30

-20

-10

0

10

20

-20 -15 -10 -5 0 5 10 15 20 25 30

’2_fo_6.dat’

-40

-35

-30

-25

-20

-15

-10

-5

0

5

10

0 10 20 30 40 50 60 70 80 90

’2_fo_7.dat’

0

10

20

30

40

50

60

70

80

90

-20 -15 -10 -5 0 5 10 15

’2_fo_8.dat’

0

10

20

30

40

50

60

70

-50 -40 -30 -20 -10 0 10

’2_fo_9.dat’

Fig. 8: Examples generated from ‘zigzag’, using factor oracle with 0.05 mutation rate

Algorithm 4 Generator using factor oracle and LZ-MS

Input: a factor oracle: f , a LZ-MS context tree c.Output: an output sequence: o

1: let the current factor oracle state i← 02: let the current context tree vertex v ← root(c)3: while |o| < desired length do4: let the set of possible transitions a← ∅5: add the transition from fi to fi+1 into a6: if the suffix link s(i) > 0 or i = |f | then7: add all transitions t(s(i)) into a8: end if9: for every transition a ∈ |a| do

10: set probability p(a) according to the weights ofoffspring(v)

11: end for12: normalize the probabilities in a13: pick a transition a from a based on probabilities14: add the symbol of a to o15: move to next state i← a16: move to next vertex v ← v17: if v is leaf then18: v ← root(c)19: end if20: end while21: return o

TABLE I: Average mutation rate of factor oracle & LZ-MS

‘rectangle’ ‘zigzag’ both patterns0.52 0.52 0.55

`(P, Stest) = − 1

N

N∑i=1

logP (ci|c1 . . . ci−1) (1)

If the generated pattern maintains a probability modelsimilar to the training pattern, a small average log-loss isexpected. The average log-loss values of LZ-MS, factor oracleand the combined method on ‘rectangle’ and ‘zigzag’ trainingpatterns are listed in Table II.

From all above tables, we see that, compared to the factororacle, the combined method is capable of achieving highermutation rate and lower average log-loss at the same time.The tracks generated from the ‘rectangle’ and ‘zigzag’ patternstogether using the combined method are shown in Fig. 9. Whilebeing similar to training patterns in structural features, thegenerated examples show more diversity.

III. EMPIRICAL EVALUATION

To test the proposed method in a real-life racing game wechose TrackMania: Nations Forever [11]. The racing tracks inthis game are made of track blocks, which makes it a perfecttest bed for our approach. The screenshots of the racing tracksare in Fig. 10.

We implemented the proposed algorithm into a small utilityprogram, which reads racing track files from TrackMania, anduses them as training patterns to generate new racing track

TABLE II: Average log-loss

‘rectangle’ ‘zigzag’ both patternsLZ-MS 0.39 0.42 0.53

factor oracle (p=0.1) 0.36 0.36 0.45factor oracle (p=0.5) 0.66 0.65 0.91factor oracle (p=0.9) 0.85 0.90 1.24

factor oracle & LZ-MS 0.35 0.34 0.38

files. A player can pick the tracks that they are interested inas training patterns, and our program will generate new trackspossessing similar structural elements that, by our assumption,reflect the player’s interest.

To conform to the game constraints we implemented thefollowing heuristic to make sure there are no crossings in thegenerated tracks.

A. Crossing avoidance

One important issue in generating racing tracks is avoidingcrossings, as the game won’t allow track blocks to overlapeach other. Given a target track length N , we use the depth-first search to find a path of depth N , similar to the mazegeneration. On each tree node, if the generated track block willcause a crossing, then we omit this one and generate another.If all possible options on this node will cause crossing, thus adead end, then we fall back to previous node on upper level.

Since the complexity of generating each track block isconstant, the complexity of generate a track of length N isO(N) in the best case. For the worst case, in which only thelast path visited can reach depth N and all other paths end atdepth N − 1, the complexity is O(kN−1 − 1), where k is thenumber of options available on each tree node, or the size ofthe alphabet Σ.

There are two common methods in depth-first searchoptimization. One is to transfer the depth-first search into thebest-first search [17]. Unfortunately, we cannot use it since thismethod eliminates diversity and produces deterministic output.The other method is to know the dead end paths in advance,and to avoid visiting these paths. In the racing track generationtask, all racing tracks are likely to be used as training patternsmore than once. Therefore, for each training pattern, it is usefulto store the dead end paths encountered during generation forupcoming generations. Apparently, the total number of deadend paths for each training pattern can be very large, and itis impossible to store all of them. We restrict the algorithm tostore only the dead end paths which start shallow (close to theroot). From the empirical evidence, it appears that avoidingthese dead end paths is sufficient to provide a good boost tothe performance.

In practical terms, the solution is to introduce a boundarym on the number of fall backs. During the generation, wecount the number of fall backs. Each time the counter reachesthe boundary, we reduce the generate length by 1, and resetthe counter. Without this modification, the worst case fromabove will return a track of length N − 1 with complexityO(min(m, kN−1−1)). And the worst case in this modificationwill return a track of length n = argmaxn min(kn−1 −1,m(N −n)), with complexity O(min(kn−1−1,m(N −n)))

-40

-35

-30

-25

-20

-15

-10

-5

0

0 20 40 60 80 100 120

’multi_fo_lzms_0.dat’

-20

-10

0

10

20

30

40

50

0 10 20 30 40 50 60 70

’multi_fo_lzms_1.dat’

0

10

20

30

40

50

60

-30 -20 -10 0 10 20 30 40

’multi_fo_lzms_2.dat’

-20

-15

-10

-5

0

5

10

15

20

25

30

-80 -60 -40 -20 0 20 40

’multi_fo_lzms_3.dat’

0

10

20

30

40

50

60

70

80

90

-10 0 10 20 30 40 50 60 70 80 90 100

’multi_fo_lzms_4.dat’

-70

-60

-50

-40

-30

-20

-10

0

-60 -50 -40 -30 -20 -10 0 10 20 30

’multi_fo_lzms_5.dat’

-50

-45

-40

-35

-30

-25

-20

-15

-10

-5

0

-10 -5 0 5 10 15 20 25 30 35

’multi_fo_lzms_6.dat’

-40

-30

-20

-10

0

10

20

-30 -20 -10 0 10 20 30 40 50 60

’multi_fo_lzms_7.dat’

0

5

10

15

20

25

30

35

0 20 40 60 80 100 120

’multi_fo_lzms_8.dat’

-40

-30

-20

-10

0

10

20

30

40

0 10 20 30 40 50 60 70 80

’multi_fo_lzms_9.dat’

Fig. 9: Examples generated from ‘rectangle’ and ‘zigzag’ patterns together, using factor oracle and LZ-MS

Fig. 10: Screenshots of generated racing tracks in game TrackMania: Nations Forever

B. Results

The best way to find out whether or not the track generatoris capable of generating interesting tracks would be to test iton real players without reveal the experiment itself. Using thisapproach we can obtain the true opinions from the players, freeof unwanted biases such as selection bias and experimenter’sbias [18, 19, 20]. We created a web server and a home page forthe the racing track generator we implemented12, and postedit on several TrackMania player communities [21, 22, 23, 24,25, 26]. The players of TrackMania can download our trackgenerator and use it to generate new tracks based upon the onesthey select. Each time a player uses the generator to createa new track, the hash code of the player’s hardware ID isuploaded to our web server.

Intuitively, if a player finds the tracks generated by ourgenerator somewhat boring, she will likely turn for othersources for new tracks as there are many websites that providetracks for download [27, 28]. Therefore, we can safely assumethat a player would continue using our generator only if shefinds the tracks generated interesting, and the number of timesa player has used our generator reflects whether or not shelikes the generated tracks. Overall we have recorded 559 usesfrom 51 players, and 22 out of 51 players have used our trackgenerator no less than 10 times. The resulting usage stats aredisplayed in Figure 11.

1http://tmgen.co.nf/2http://sourceforge.net/projects/tmcg/

Fig. 11: Histogram of track generator uses

IV. CONCLUSIONS

We propose a method inspired by music improvisationtechniques, which is capable of generating in real time newracing tracks with the same structural features that a playerdisplays a preference for. To achieve that, we turn the racingtrack generation task into a discrete sequence prediciton prob-lem and solve it by building the variable-order Markov modelfrom traning patterns, which also preserves the transition rules.

A few questions yet need to be addressed. Due to thenon-deterministic output of the algorithm, it cannot generateclosed tracks, which is a requirement in a significant partof racing games. Additionally, our approach lacks the abilityto avoid overlapping naturally. In our future work we aimto ovecome above issues using the approaches applied inconstraint satisfaction problems for Markov processes [29].

Nevertheless, the empirical evidence from implementingthe proposed algorithm in a real racing game TrackManiashows that players reuse the generator without any externalmotivation, which we think is a positive feedback about theinterestingness of generated tracks.

REFERENCES

[1] Borderlands. Gearbox Software, 2009.[2] Diablo. Blizzard North, 1996.[3] N. Shaker, J. Togelius, and M. J. Nelson, Procedural Con-

tent Generation in Games: A Textbook and an Overviewof Current Research. Springer, 2014.

[4] M. Hendrikx, S. Meijer, J. Van Der Velden,and A. Iosup, “Procedural content generationfor games,” ACM Transactions on MultimediaComputing, Communications, and Applications, vol. 9,no. 1, pp. 1–22, Feb. 2013. [Online]. Available:http://dl.acm.org/citation.cfm?doid=2422956.2422957

[5] R. van der Linden, R. Lopes, and R. Bidarra,“Procedural Generation of Dungeons,” IEEETransactions on Computational Intelligence and AIin Games, vol. 6, no. 1, pp. 78–89, Mar. 2013.[Online]. Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6661386http://ieeexplore.ieee.org/xpls/abs\ all.jsp?arnumber=6661386

[6] D. Loiacono, L. Cardamone, and P.-L. Lanzi, “Automatictrack generation for high-end racing games using evolu-tionary computation,” Computational Intelligence and AIin Games, IEEE Transactions on, vol. 3, no. 3, pp. 245–259, Sept 2011.

[7] L. Cardamone, D. Loiacono, and P. L. Lanzi, “Interactiveevolution for the procedural generation of tracks ina high-end racing game,” in Proceedings of the13th annual conference on Genetic and evolutionarycomputation, ser. GECCO ’11. New York, NY,USA: ACM, 2011, pp. 395–402. [Online]. Available:http://doi.acm.org/10.1145/2001576.2001631

[8] J. Togelius, R. De Nardi, and S. Lucas, “Towards auto-matic personalised content creation for racing games,” inComputational Intelligence and Games, 2007. CIG 2007.IEEE Symposium on, 2007, pp. 252–259.

[9] P. Johnson-Lairda, “How jazz musicians improvise,” Mu-sic Perception, vol. 19, no. 3, pp. 415–442, 2002.

[10] B. Thom, “Unsupervised learning and interactivejazz/blues improvisation,” in Proceedings of theSeventeenth National Conference on ArtificialIntelligence and Twelfth Conference on InnovativeApplications of Artificial Intelligence. AAAIPress, 2000, pp. 652–657. [Online]. Available:http://dl.acm.org/citation.cfm?id=647288.721257

[11] TrackMania Nations Forever. Nadeo, 2007.[12] M. Nisenson, I. Yariv, R. El-Yaniv, and R. Meir, “Towards

behaviometric security systems: Learning to identify a

typist,” in Proceedings of the 7th European Conferenceon Principles and Practice of Knowledge Discovery inDatabases (ECML/PKDD), 2003, pp. 363–374.

[13] R. Begleiter, R. El-Yaniv, and G. Yona, “On predictionusing variable order markov models,” Journal of ArtificialIntelligence Research (JAIR), vol. 22, pp. 385–421, 2004.

[14] C. Allauzen, M. Crochemore, and M. Raffinot, “Factororacle: A new structure for pattern matching,” inProceedings of the 26th Conference on CurrentTrends in Theory and Practice of Informatics onTheory and Practice of Informatics, ser. SOFSEM’99. London, UK, UK: Springer-Verlag, 1999, pp.295–310. [Online]. Available: http://dl.acm.org/citation.cfm?id=647009.712672

[15] J. Ziv and A. Lempel, “Compression of individual se-quences via variable-rate coding,” IEEE Transactions onInformation Theory, vol. 24, no. 5, pp. 530–536, 1978.

[16] G. Assayag and S. Dubnov, “Using factor oraclesfor machine improvisation,” Soft Comput., vol. 8,no. 9, pp. 604–610, Sep. 2004. [Online]. Available:http://dx.doi.org/10.1007/s00500-004-0385-4

[17] J. Pearl, Heuristics: Intelligent Search Strategies forComputer Problem Solving. Boston, MA, USA:Addison-Wesley Longman Publishing Co., Inc., 1984.

[18] R. Lee, Unobtrusive Methods In Social Research(Understanding Social Research), 1st ed. OpenUniversity Press, 8 2000. [Online]. Available:http://amazon.com/o/ASIN/0335200516/

[19] C. Cortes, M. Mohri, M. Riley, and A. Rostamizadeh,“Sample selection bias correction theory,” in AlgorithmicLearning Theory, ser. Lecture Notes in ComputerScience, Y. Freund, L. Gyrfi, G. Turn, and T. Zeugmann,Eds. Springer Berlin Heidelberg, 2008, vol. 5254, pp.38–53. [Online]. Available: http://dx.doi.org/10.1007/978-3-540-87987-9 8

[20] B. Kantowitz, H. Roediger, and D. Elmes,Experimental Psychology, ser. International studentedition. Wadsworth, 2009. [Online]. Available:http://books.google.de/books?id=2-5VL8PHLsIC

[21] “Trackmania forum,” http://www.tm-forum.com.[22] “Trackmania files on filefront,”

http://trackmania.filefront.com.[23] “Trackmania exchange,” http://united.tm-exchange.com.[24] “Trackmania on 3dm games,” http://bbs.3dmgame.com.[25] “Trackmania on netshow,” http://game.ali213.net.[26] “Trackmania forum in french,”

http://forum.trackmaniaforever.com.[27] R. Hall and M. Lieberman, Microeconomics: Principles

and Applications: Principles and Applications, ser.Available Titles Aplia Series. South-Western, CengageLearning, 2009. [Online]. Available: http://books.google.com.br/books?id=XR9uvGmChp8C

[28] F. Hayek, The Road to Serfdom. University of ChicagoPress, 1944. [Online]. Available: http://books.google.com.br/books?id=1QyQwvLDLycC

[29] P. Roy and F. Pachet, “Enforcing meter in finite-length markov sequences.” in AAAI, M. desJardins andM. L. Littman, Eds. AAAI Press, 2013. [Online].Available: http://dblp.uni-trier.de/db/conf/aaai/aaai2013.html#RoyP13