Evolving Bot AI in Unreal (2)

8/7/2019 Evolving Bot AI in Unreal (2)

http://slidepdf.com/reader/full/evolving-bot-ai-in-unreal-2 1/33

A.M. Mora, R. Montoya, J.J. Merelo, P. Garca-Sanchez1, P.A. Castillo1,

J.L.J. Laredo, A. Martnez, andA.I.Esparcia3

Mati Bot



Presentation Outliney First Person Shooters (FPS)

y History of FPS

y

UnrealScript in a nutshelly 2 evolutionary approaches to evolve Unreal bots

y Genetic Algorithm

y Genetic Programming

y

The experimenty Results

y Future Work

y Bonus: Cloud computing and EA



The Problem-First Person Shootery First Person View.y Inputs:

y Can see the hands (which weapon is currently held).y Can see the enemies which are in front of the camera view.

y Outputs:y Direction of movement.y Jumpy Shoot/Hold firey Change weapon

y Goal:y

Kill everyone.y Get 1 frag for each kill.y Dont die.y Collect good weapons.y Collect good items.



History of FPSy Wolfenstein (ID software, 1987) First single player FPS

y Doom (ID software, 1988) First Multiplayer FPS

y Quake (ID software, 1992) First Autonomous botsy QuakeC-a program language for creating Quake Bot AI

y Showed simple AI(Fuzzy logic at best)

y Not suitable for GA

y Unreal Tournament (Epic Mega Games, 1999)y

UnrealScript-High level programming languageinfluenced by Java. Suitable for GA. [This paper is here]

y Unreal Tournament 2004 (Epic Mega Games, 2004)

y Unreal tournament 3 (EMG, 2009)



UnrealScript in a nutshelly Class declaration + inheritance:

y class MyClass extends MyParentClass;y Here are the built-in variable types supported in UnrealScript:

y byte: A single-byte value ranging from 0 to 255.y int: A 32-bit integer value.y bool: A booleanvalue: either true or false.y float: A 32-bit floating point number.y string: A string of characters. (see Unreal Strings)y constant: A variable that cannot be modified.y enumeration: A variable that can take on one of several predefined named integer values.

For example, the ELightType enumeration defined in the Actor script describes a dynamic

light and takes on a value like LT_None, LT_Pulse, LT_Strobe, and so on.y array<Type>: A variable length array of Type.y struct: Similar to C structures, UnrealScript structs let you create new variable types

that contain sub-variables. For example, two commonly-used Unreal structsarevector, which consists of an X, Y, and Z component; and rotator, which consists of a pitch, yaw, and roll component



UnrealScript in a nutshelly Supports repetition:

y While,For,DoWhile

y Conditionals:y If then else,case

y Polymorphism

y Built-in library (Math, String,Time,Vector,Debugging

and more)



Unreal Script Examples:y Declare an integer variable named "A".

y var int a;

y Declare a static array of 64 bytes named "Table".

y var byte Table[64];

y Declare a string variable named "PlayerName".

y var string PlayerName;

y Declare a float variable named "MaxTargetDist" and allow its value tobe modified from an UnrealEd property window.

y var() f loat MaxTargetDist;



Unreal Script Examples:

yFunction to compute the factorial of a number.y function int Factorial( int Number ) {

if( Number <= 0 )

return 1;

else

return Number * Factorial( Number - 1 ); }



Unreal Script-Bottom line

The Good:

1. Very similar to Java and C++ and very powerful.

2. Many games use the same language. (and this language is still being updated)The Bad:

1. Array length must be less than 60. Arrays must be one dimensional. [1999]

2. Cannot run faster than real-time.

The ugly:

1. There isnt a nice way to debug the code.

Link to the reference of UnrealScript if you want more info



Genetic Botsy 2 main approaches for the evolutionary process:

y Evolve the parameters of the conditions that trigger the

transition from the current state to a new statey Evolve the Finite State Machine of the bot. (the edges

between the states and the states themselves.



Genetic Algorithm-based Boty Firstly, 82 parameters were identified.

y The difficulty:

y

UnrealScript (as of 99) supports arrays of 60 f loats.y 82 parameters are too much to evolve in the game.

y Solution:

y Some parameters were not considered in the GA.

y Other parameters were represented as a function of anotherparameter which is evolved in the GA.

y In the end, 43 parameters (real numbers) wereevolved.







y Representation: 43 f loating points (normalized to [0,1]).

y Crossover: 2-Point Crossover

y

Mutation: simple gene mutation (change the value of arandom gene (or none) by adding or subtracting a randomquantity in [0,1])

y Selection: Ranked Based.

y

Scheme: Generational + Elitism

Genetic Algorithm-based Bot(cont.)



Fitness Function:



Genetic Programming-based Boty Use GP to evolve new bot behavior.

y First approach: Work with all possible input and

outputs. All states were considered.y Difficulty: UnrealScript cant handle it due to its

constraints.

y Solution:y Divide the search space of states into 2:

y Attacking, in which the bot decides between some possibleactions as: search, escape, move in a strategic way, attack andhow to do it (from distance, nearby, close, from a flank).

y Roaming, in which the bot searches for weapons and items



Search Space-GPy UnrealScript functions:

y Input functions (answer queries, ItemFound)

y Output functions (state alteration,Gotostate(Roaming,Camp)

y Considered only functions which are useful to decisionmaking. (HitWall, for instance, is used only for

animation).y Define rules on top of functions:

y IF <INPUT-F1> OR <INPUT-F2> THEN <OUTPUT-F>



Chromosome example:y IF X1 THEN Y7, IF X3 AND X5 THEN Y2, IF X8 THEN

Y1, and IF X9 THEN Y9

Rule Set Parameters



Evolution parameters:y Representation: Rule Tree as seen before + floating

points (normalized to [0,1])

y

Crossover: Node exchange(a valid one) + 2-PointCrossover

y Mutation: grows a random tree instead of a randomnode + simple gene mutation (like before)

y Selection: Ranked Based(like before)y Scheme: Generational + Elitism



Fitness Function



The Experimenty 8 players (bots-there was no technical difficulty for

letting humans to play) 1 of the 8 is the individual to

be tested and the rest of them came premade with thesystem.

y 1 hour for a generation (the game is played In real-time)

y

Each individual is playing until a threshold of defeatsis reached.



y The parameters for both methods:The Experiment



Results

y The GA-Bots behavior was more coherent than the GPGA-Bots.

y The GA-Bots behavior was fixed and therefore performed

better. only the conditions for changing a state could be change.y GPGA-Bots didnt perform as well because their behavior was

initially random and couldnt beat the Unreal bots.y Both of the best bots that were evolved from both

methods eventually outperform all of the Unreal premadebots by winning all matches.



Future Worky Find the best Evolution parameters.

y Implement this in a newer engine (like a new version

of Unreal)y Change the fitness function to be multi-objective

(Pareto front)

y Cooperative bots (team of bots vs another team)



EA and Cloud Computing



What is cloud computing?Hardware as Service

A Model for enabling convenient, on-demand networkaccess to a shared pool of configurable computingresources:

y Storage

y Servers

y Networks

y Applications

y Services



Amazon Web Servicesy Cloud computing vendor.

y Get access to as many computers as you like for a cheap

price.y Configure the image of each computer dynamically.

y Pay only for what you need.

y Provides storage as well. (and many other features)



Start a computer with service X

Start a computer with service Y

Start a computer with service Z

Communication channel

Access data



AWS as a tool for EA Study

first approach

Fitnessevaluator

Fitnessevaluator

Fitnessevaluator

Genetic

Algorithm

� Offers The ability torun a GA experimentfaster than running ona single PC.




trivial approach

Fitnessevaluator

Fitnessevaluator

Fitnessevaluator

GeneticAlgorithm

� Offers The ability to

run a GA experimentand close the clientapplication. Then youcan continue yourwork on the samecomputer and be more

effective.

configuration




Advanced approach

GeneticAlgorithm 1

ExperimentManager

SharedExperiment

data

GeneticAlgorithm 2

GeneticAlgorithm 3

�

Offers The ability to runmultiple GA experimentssimultaneously (forchecking what fitnessfunction is better or anew crossover operator

for instance)

configuration




More advanced approach

GeneticAlgorithm 1

ExperimentManager

Experimentdata

GeneticAlgorithm 2

GeneticAlgorithm 3

�

Uses the simplicity of thecloud to connect differentGA nodes and share thegenetic material(Migration) with eachother by a configuration

made in advance.

GA algorithmsgraph

configuration



Documents

Evolving Bot AI in Unreal (2)