Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
CS 480: GAME AI MACHINE LEARNING IN GAMES
5/31/2012 Santiago Ontañón [email protected] https://www.cs.drexel.edu/~santi/teaching/2012/CS480/intro.html
Reminders • Check BBVista site for the course regularly • Also: https://www.cs.drexel.edu/~santi/teaching/2012/CS480/intro.html
• Project 4 due June 7th • Final exam, next week: June 5th
• Tactic & Strategy • Board Games • Learning
Outline • Student Presentations:
• “Game AI as Storytelling” • “Computational Approaches to Story-telling and Creativity” • “Procedural Level Design for Platform Games”
• Very Brief Introduction to Machine Learning • Machine Learning in Games • Course Summary
Outline • Student Presentations:
• “Game AI as Storytelling” • “Computational Approaches to Story-telling and Creativity” • “Procedural Level Design for Platform Games”
• Very Brief Introduction to Machine Learning • Machine Learning in Games • Course Summary
Machine Learning • Branch of AI that studies how to:
• Infer general knowledge from data: supervised learning • Infer behavior from data: learning from demonstration • Find hidden structure in data: unsupervised learning • Infer behavior from trial an error (data): reinforcement learning
• Underlying principle is inductive inference: • E.g. after seeing 100 times that objects fall down, we can infer by
induction the general law of gravity. • We cannot deduce gravity from observation, we can only induce it.
Supervised Learning • Given labeled (supervised) training data:
(x1,y1) (x2,y2) (x3,y3) … (xn,yn)
• Find the general function that maps x to y: • f(x) = y
Examples
Supervised Learning Example • Learning when to cross the street:
• X is defined as a list of features: (“cars coming?”, “pedestrian traffic light color”, “time”)
• Y defined as: cross, stay • Examples:
• x1 = (no, green, 3pm) y1 = cross • x2 = (no, red, 8am) y2 = don’t cross • x3 = (yes, red, 5pm) y3 = don’t cross • x4 = (yes, green, 9pm) y4 = don’t cross • x5 = (no, green, 10am) y5 = cross
• Outcome of machine learning: • f(x) = “cross if no cars coming and light color green”
Supervised Learning • Examples can be:
• Fixed-length vector of attributes: • Numerical • Categorical
• Structured representations (graphs, trees, logical clauses)
• Machine learning works better with: • Few representative attributes • Lots of data
Simplest Method: Nearest Neighbor • Examples are fixed vectors of real numbers • To predict f(x):
• f(x) = yi where (xi,yi) is the example with xi most similar to x (nearest neighbor)
Simplest Method: Nearest Neighbor • Examples are fixed vectors of real numbers • To predict f(x):
• f(x) = yi where (xi,yi) is the example with xi most similar to x (nearest neighbor)
x
Simplest Method: Nearest Neighbor • Examples are fixed vectors of real numbers • To predict f(x):
• f(x) = yi where (xi,yi) is the example with xi most similar to x (nearest neighbor)
x Nearest neighbor f(x) = gray
Machine Learning Methods: • K-Nearest Neighbor • Decision Trees (ID3, C4.5) • Linear Regression • Bayesian Models (Naïve Bayes) • Boosting (AdaBoost) • Kernel Methods (SVM) • Neural Networks (MLP) • Inductive Logic Programming • etc.
Supervised Learning Methods: • K-Nearest Neighbor • Decision Trees (ID3, C4.5) • Linear Regression • Bayesian Models (Naïve Bayes) • Boosting (AdaBoost) • Kernel Methods (SVM) • Neural Networks (MLP) • Inductive Logic Programming • etc.
Which one is the best?
Supervised Learning Methods: • K-Nearest Neighbor • Decision Trees (ID3, C4.5) • Linear Regression • Bayesian Models (Naïve Bayes) • Boosting (AdaBoost) • Kernel Methods (SVM) • Neural Networks (MLP) • Inductive Logic Programming • etc.
Which one is the best?
NONE! Each one has its own biases
Outline • Student Presentations:
• “Game AI as Storytelling” • “Computational Approaches to Story-telling and Creativity” • “Procedural Level Design for Platform Games”
• Very Brief Introduction to Machine Learning • Machine Learning in Games • Course Summary
Why Learning in Games?
Benefit for Games:
• Adaptive, believable Game AI • Find good player matches • Realistic player movement • Reduce development cost (e.g.
fine-tuning parameters)
Benefit for AI:
• Games are great test beds (perfect instrumentation and measurements)
• Reduced cost • Extremely challenging domains • Great way to motivate
students!
Uses of Learning in Games • Driving Games:
• Learn good driving behavior from observing humans • Fine-tune parameters of physics simulation, or of car handling
parameters
• RTS Games: • Automatically balance the game
• RPG/FPS Games: • Believable movements
• Others: • Specific game genres possible only with learning (training games) • Automatically adapt to player strategies • Learning player models
Example Usages: Black & White
Learning in Black & White
Desires
Opinions
Beliefs
Intention: Abstract
Plan
Specific Plan
Action List Belief-Desire-Intention Architecture
Learning in Black & White
Desires
Opinions
Beliefs
Intention: Abstract
Plan
Specific Plan
Action List Belief-Desire-Intention Architecture
Represented as hand-crafted perceptrons (single
layer neural networks)
Given the current situation, they activate more or less, triggering certain desires.
Example: hunger
The structure of the
perceptrons is hardcoded, but the parameters are
learned
Learning in Black & White
Desires
Opinions
Beliefs
Intention: Abstract
Plan
Specific Plan
Action List Belief-Desire-Intention Architecture
Represented as learned decision trees, one per
desire.
They capture towards which object the creature should
express each desire.
Example: hunger The creature will learn a
decision tree from player’s feedback of what can be
eaten
Learning in Black & White
Desires
Opinions
Beliefs
Intention: Abstract
Plan
Specific Plan
Action List Belief-Desire-Intention Architecture
Beliefs are just lists of properties of the objects in the game, used for learning
opinions.
Learning in Black & White
Desires
Opinions
Beliefs
Intention: Abstract
Plan
Specific Plan
Action List Belief-Desire-Intention Architecture
Which desire will be attempted, and towards
which object: e.g. “destroy town” or “eat human”
Learning in Black & White
Desires
Opinions
Beliefs
Intention: Abstract
Plan
Specific Plan
Action List Belief-Desire-Intention Architecture
Desire, and list of objects that will be used. E.g.
“Destroy town by throwing a stone”
Learning in Black & White
Desires
Opinions
Beliefs
Intention: Abstract
Plan
Specific Plan
Action List Belief-Desire-Intention Architecture
Specific list of actions to execute the plan
Learning in Black & White • BDI model to simulate character personality
• Learning is deployed at specific players to adapt the BDI model to player preferences
• Quite advanced, for commercial game standards!
Example Usages: Forza Motorsport
Example Usages: Forza Motorsport • Forza Motorsport lets you create “Drivatars”
• Drivatars use machine learning to “learn” to drive like the player.
• Supervised machine learning: • Input: the track just ahead of the car • Output: car trajectory
Track Segments
a4 a2 a3 a1 Segments
• Define a set of “track segments”. All the tracks are made up of these • For each segment, define the different ways in which it can be traversed
Learning from the Player • When the player plays:
1. For each segment the player traverses, Drivatar remembers the trajectory used (out of the ones it has stored), speed, and other parameters.
2. Drivatar only remembers the last data seen, and forgets old data (to adapt to newer driving styles of the player)
3. When playing the game, it selects the trajectories learned form the player for each track segment, and combines them into a minimal curvature line, which is the path taken by the AI
Minimal Curvature Line
Learning in Drivatars: Key Ideas • Learning to drive is very complex! Too Many parameters!
• Solution: simplify the problem!
• Simplifications: • Divide the track in segments • Identify a finite set of possible trajectories for each segment (this
defines a discrete and finite vocabulary for the machine learning method)
• The goal of AI in games is to make the game fun, not to create advanced AI! (at least from a game company perspective J)
Other Games Using Learning • Colin McRae Rally 2.0: Neural networks to learn how to
drive a rally car
Other Games Using Learning • Command & Conquer Renegade: learning new paths by
observing the player (e.g. player breaks a window and creates a new path)
Other Games Using Learning • Re-Volt: genetic algorithm tunes the parameters of the
steering behaviors for cars, to optimize lap time
Ways to Fake “Learning” in Games • Preprogram different levels of difficulty and change
between them at run time
• Rule-based system: have a large pool of rules, and add them bit by bit to the AI
• Change parameters during game play: • Increase the accuracy of shooting • Decrease reaction time of enemies • Etc.
Outline • Student Presentations:
• “Game AI as Storytelling” • “Computational Approaches to Story-telling and Creativity” • “Procedural Level Design for Platform Games”
• Very Brief Introduction to Machine Learning • Machine Learning in Games • Course Summary
Game AI • Intelligent behavior for characters in games
• The goal is not to make characters smart, but to make the game believable, engaging and fun
• There is a lot of design in game AI
• Commercial games use a collection of very simple, but highly customizable AI techniques (FSMs, Behavior Trees, Influence Maps, etc.)
Game AI
AI
World Interface
(perception)
Strategy
Decision Making
Movement
Game AI • Perception: interfacing the AI with the game
• Movement: jumping, firing, steering behaviors
• Pathfinding: quantization, navmeshes, A*, TBA*, LRTA*, D*-Lite
• Decision Making: FSMs, Decision Trees, Behavior Trees
• Tactics/Strategy: Rule-Based systems, waypoints, influence maps
• Board Games: Minimax, alpha-beta, Monte-Carlo search
• Learning in games: supervised learning, reinforcement learning
Game AI • Perception: interfacing the AI with the game
• Movement: jumping, firing, steering behaviors
• Pathfinding: quantization, navmeshes, A*, TBA*, LRTA*, D*-Lite
• Decision Making: FSMs, Decision Trees, Behavior Trees
• Tactics/Strategy: Rule-Based systems, waypoints, influence maps
• Board Games: Minimax, alpha-beta, Monte-Carlo search
• Learning in games: supervised learning, reinforcement learning
Projects 1, 2, 3, 4
Game AI • Perception: interfacing the AI with the game
• Movement: jumping, firing, steering behaviors
• Pathfinding: quantization, navmeshes, A*, TBA*, LRTA*, D*-Lite
• Decision Making: FSMs, Decision Trees, Behavior Trees
• Tactics/Strategy: Rule-Based systems, waypoints, influence maps
• Board Games: Minimax, alpha-beta, Monte-Carlo search
• Learning in games: supervised learning, reinforcement learning
Midterm
Game AI • Perception: interfacing the AI with the game
• Movement: jumping, firing, steering behaviors
• Pathfinding: quantization, navmeshes, A*, TBA*, LRTA*, D*-Lite
• Decision Making: FSMs, Decision Trees, Behavior Trees
• Tactics/Strategy: Rule-Based systems, waypoints, influence maps
• Board Games: Minimax, alpha-beta, Monte-Carlo search
• Learning in games: supervised learning, reinforcement learning
Final
Project 4 • Project 4 (and last): Rule-based Strategy for RTS Game (S3)
• Idea: • Create a perception layer that creates a simple knowledge base (logical
terms) • Create a simple unification algorithm with variable bindings • Define a set of actions the rule-based system can execute • Define a small set of rules (do not overdo it! J) • RETE is optional (extra credit) • See how well it plays and how easy is it to make the AI play well!
• Anyone wants to do a different project 4? Any ideas?
Next Week • Final Exam • Project 4 presentation