Upload
waneta
View
44
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Dynamic Task Allocation in a turn based strategy game. Gilles Schtickzelle September 2012 ULB. Problem Statement. Creating an intelligent player for a turn-based strategy game. Working Framework: Many possible challenges to meet: Resource management Adversarial planning Spatial reasoning - PowerPoint PPT Presentation
Citation preview
Dynamic Task Allocation in a turn
based strategy game
Gilles SchtickzelleSeptember 2012
ULB
Problem Statement• Creating an intelligent player for a turn-based
strategy game.• Working Framework:
• Many possible challenges to meet:o Resource managemento Adversarial planningo Spatial reasoningo …
A game of FreeCol• Colonization of America• Establish settlements, grow and develop them• Victory: Declare independence & Beat the Royal
Expeditionary Force
Colony Management• Assigning tasks to units for optimal resources production
Division of labor in insect societies
• Ants and wasps colonies have efficient distributed task allocation mechanisms through stygmergy.
Bonabeau, E., Theraulaz, G., & Deneubourg, J.-L. (1996).
Response Threshold
• Ants have probabilistic response to stimuli:
• Varying threshold θ induces specializationo Reduces switching costso Increases individual efficiency
From insects to games• FreeCol
Colony
• Units
• Resources
• Expert units
• Ants/Wasps Colony
• Insects
• Tasks
• Specialization
Resources Dynamics• Surplus: Extra workers.
Shortage: Lose worker.
• Freedom. 50% required to win. Gives bonus or penalty to workers.
• required to make hammers
• Used to produce buildings or artileries
• required to make tools
• Used to produce buildings or artilleries
Allocation Mechanism• One stimulus Sr for each resource r =
• One set of dynamic thresholds θri per unit i
Stimuli and Thresholds
• Simple computation rules for each stimulus
• One set of dynamic thresholds θru per unit u
• Genetic Algorithm to find appropriate scale factors βr
𝜃𝑟𝑢=¿
Simple Scenario
AI goals
1. Reach the year 1776 with enough bells to be able to declare independence.
2. Have the best defense possible to resist the attack of the royal expeditionary force.
3. Allocate workers to1. minimize famine2. Keep the production modifier as high as possible
Results (Basic player)
Freedom % Size Famine Military
EXPERT 100% 14 0 24
MEAN (100 games) 91.57 ± 2.39 15.12 ± 0.85 0.18 ± 0.09 15.99 ± 0.62
1 34 67 1001331661992322652983313643974300
0.51
1.52
2.5
(B)Evolution of the production modifier (AI vs Expert)
PRODUCTION MODIFIER (AI)PRODUCTION MODIFIER (EXPERT)
1 34 67 1001331661992322652983313643974300
10
20
30
(C)Evolution of the number of artilleries (AI vs Expert)
ARTILLERIES (AI)ARTILLERIES (EXPERT)
Planning approach• Suboptimal allocation: building too early
• Two planning methods:
o Layered response threshold.
o Rule-based planning.
1 29 57 85 11314116919722525328130933736539342144905
1015202530
(C)Evolution of the number of artilleries (AI vs Expert)
ARTILLERIES (AI)ARTILLERIES (EXPERT)
Planning approach• Layered response threshold :
o Use two sets of scale factors: • Optimized for growth• Optimized for production
• Rule-based planning :
𝑃𝐺𝑅𝑂𝑊𝑇𝐻=𝑆𝐺𝑅𝑂𝑊𝑇𝐻
2
𝑆𝐺𝑅𝑂𝑊𝑇𝐻2 +Θ𝐺𝑅𝑂𝑊𝑇𝐻
2
𝑆𝐻𝐴𝑀𝑀𝐸𝑅𝑆={10 𝑖𝑓 (𝑆𝑡𝑜𝑟𝑎𝑔𝑒𝐻𝐴𝑀𝑀𝐸𝑅𝑆≤𝐵𝑢𝑖𝑙𝑑𝐻𝐴𝑀𝑀𝐸𝑅𝑆𝑜𝑟 𝑆𝑡𝑜𝑟𝑎𝑔𝑒𝐿𝑈𝑀𝐵𝐸𝑅 ≥15
𝑜𝑟 𝑆𝑖𝑧𝑒𝑐>6 )0 𝑖𝑓 𝑛𝑜𝑡
Planning Results (1)Layered AI Rule-based AI
1 23 45 67 89 11113315517719922124326528730933135337539741944102468
1012141618
Evolution of the colony size
SIZE (MULTI LAYER AI) SIZE (EXPERT)
1 23 45 67 89 1111331551771992212432652873093313533753974194410
5
10
15
20
25
30
Evolution of the number of artilleries
ARTILLERIES (MULTI LAYER AI) ARTILLERIES (EXPERT)
1 23 45 67 89 11113315517719922124326528730933135337539741944102468
1012141618
Evolution of the colony size
SIZE (RULE BASED) SIZE (EXPERT)
1 23 45 67 89 1111331551771992212432652873093313533753974194410
5
10
15
20
25
30
Evolution of the number of artilleries
ARTILLERIES (RULE BASED) ARTILLERIES (EXPERT)
Planning Results (2)
Military SoL % Size Famine
EXPERT 24 100 14 0
MEAN (BASIC) 15.99 ± 0.62 91.57 ± 2.39 15.12 ± 0.85 0.18 ± 0.09
MEAN (LAYERED) 17.49 ± 0.70 99.17 ± 0.65 14.53 ± 0.84 0.19 ± 0.10
MEAN (RULE BASED) 18.80 ± 0.54 98.22 ± 1.28 17.49 ± 1.03 0.16 ± 0.09
Statistics for 100 games with the simple scenario.
Modified Threshold rule
• Unit u produces resource r
• Unit u does not produces resource r
𝜃𝑢𝑟={0.1 𝑖𝑓 (𝑢𝑖𝑠 𝑒𝑥𝑝𝑒𝑟𝑡 𝑓𝑜𝑟 𝑟 )max ( 1, 𝜃𝑢𝑟−1 ) 𝑖𝑓 𝑛𝑜𝑡
𝜃𝑢𝑟={0.1 𝑖𝑓 (𝑢𝑖𝑠 𝑒𝑥𝑝𝑒𝑟𝑡 𝑓𝑜𝑟 𝑟 )min (10 , 𝜃𝑢𝑟 +1 ) 𝑖𝑓 𝑛𝑜𝑡
1 21 41 61 81 1011211411611812012212412612813013210
2
4
6
8
10
12
0
2
4
6
8
10
12
Evolution of thresholds and jobs for unit 5 (non expert)
FOOD BELLS NONE HAMMERS LUMBER TOOLS ORE jobs for unit 5
Thre
shol
ds
Jobs:1=FOOD2=BELLS3=NONE
4=HAMMERS5=LUMBER6=TOOLS
7=ORE
“State of the art” player
• Modified Threshold update rule + rule-based planning
1 24 47 70 93 116139162185208231254277300323346369392415438012345678
Evolution of unit's job over time (single layer AI player)
unit1 unit2 unit3 unit4 unit5 unit6 unit7 unit8 unit9 unit10
Jobs:1 = FOOD2 = BELLS3 = NONE
4 = HAMMERS5 = LUMBER6 = TOOLS
7 = ORE
1 24 47 70 93 116139162185208231254277300323346369392415438012345678
Evolution of unit's job over time (State-of-the-art AI player)
unit1 unit2 unit3 unit4 unit5 unit6 unit7 unit8 unit9 unit10
Jobs1 = FOOD2 = BELLS3 = NONE
4 = HAMMERS5 = LUMBER
6 = ORE7 = TOOLS
AI players comparison
Basic Layered rule-based State of the art0
5
10
15
20
25
30
0
5
10
15
20
25
30
AI players compared (number of military units)
Expert = 24
AI goal completion
1. Reach the year 1776 with enough bells to be able to declare independence.
2. Have the best defense possible to resist the attack of the royal expeditionary force.
3. Allocate workers to1. minimize famine2. Keep the production modifier has high as possible
Freedom % Size Famine Military
EXPERT 100% 14 0 24
MEAN (100 games) 100% ± 0.92 20.21 ± 0.94 0.04 ± 0.07 20.99 ± 0.55
Conclusions Human-level performances can emerge from
simple rules, without cheating. Easy to implement (compared to traditional rule-based only
AI). Easy to tune down performances (if playing against
non-expert). Hybrid system (with planning instructions)
improves on basic RTM− Tendency to chaos with large number of stimuli− Difficult to extend to other game aspects (combat,
spatial reasoning, diplomacy,…).