View
236
Download
1
Tags:
Embed Size (px)
Citation preview
1
GP End-Chess
Evolution of Chess Endgame Players
Ami Hauptman & Moshe Sipper
2
Outline Introduction
The Game of Chess – a solved problem? Important differences between human and
artificial chess players Chess Endgames - features & building
blocks GP problem definition Experiment and Results Future work
3
The game of Chess
First developed in India and Persia Considered an extremely complex game of
strategy and inventiveness Enormous search space
Roughly 50 possible moves at mid-game A typical game consists of a few dozen moves Estimated at 1043 in 40-move game (Shannon,
1950)
4
The game of Chess – AI history
First chess AI at 1958 – novice level Machine strength increasing linearly 1997 – defeat of former world champion,
Garry Kasparov, by IBM’s deep blue Last years – performance still increasing
Mainly Hardware Also Software
… The End ?
5
The game of Chess …NO! Deep(er) blue uses extreme brute-force,
traversing several millions of boards ps Very little generalization Virtually no human resemblance Deemed theoretically uninteresting
Chomsky: As interesting as a weight lifting competition between machine and man
Low A to I ratio; Low return
6
The game of Chess – Basic concepts
8x8 board Each player starts with 16 pieces, of 6
different types, and may only move 1 piece per turn
A piece can only move into an empty square or into one containing an opponent’s piece (a capture)
Win by capturing the opponent’s king
7
The game of Chess – pieces Pawn: may only move forward (or capture
diagonally) Bishop: diagonals Knight: L shaped moves. The only “unblockable”
piece Rook: Ranks & files Queen: Bishop & Rook combined King: 1 square in any direction. May not move
into attacked square
Values :
1 3 3/3.5 5 9 ∞
8
The game of Chess – example
White has over 30 possible moves
If black’s turn – can capture pawn at c3 and check (also fork)
9
The game of Chess – Check and Checkmate
“Checking” is attacking opponent’s king. Opponent must respond
“Mating” (Checkmate)
is when the opponent can’t avoid losing the king – and thus forfeiting the game
10
Human & Artificial Players - AI
AI uses search to assign a score to a board
Traverse the move tree from leaves - up
Select the best child using scores found
Only partial tree
Computer is O (Max) opponent is X
11
Human & Artificial Players – AI Search
Millions of boards (nodes) per second Little time for each board – less
knowledge Smart search algorithms –
pruning Alpha-beta variants (negascout etc.)
Still use heuristics at end – can’t see all tree
Most research revolves around search Human resemblance minimal – humans
use little search
12
Human & Artificial Players – Humans(1)
Humans use problem solving cognition Deeply knowledge based – extensive “Chess
theory” Massive use of pattern recognition Also use search but
Less deep Only develop “good” positions
More efficient – less nodes for “same” result Reminiscent of greedy search
Not only in chess
13
Human & Artificial Players – Grand Masters - Findings
Play against several opponents at the same level they play against a single opponent
GMs and novices: same level of performance when memorizing a random board; differ when memorizing real game positions
GM eye movements show they only scan “correct” parts of board
Strong Amateurs use the same meta-search as GMs - equally deep, same nodes, same speed; Differ in knowledge of domain (De Groot)
14
Endgames - example
White’s turn: mate in
5, with Qe6+
Features include: #moves for black king
minimal
Attacking, un-attacked
Checking
Officers same line\row
Black’s turn: draw
with: Rc1+, then Qg5
– fork & exchange
15
Endgames (2) - features
Few pieces remain (typically: king, 0-3
officers and sometimes pawns)
Fewer options, but more moves for each
piece
Trees still extremely large
16
Endgames - Building Blocks
Main goals Reduce search by “smart” features of the
board Use more game knowledge as humans do
Allow more complex features to be built by supplying basic ones (terminals) and building methods (functions)
Schemata evolution
17
Features Example - Fork
My piece is: Attacking 2 or more
pieces Protected or not
attacked Opponent pieces:
Unprotected OR protected but of
greater value Example: black must
exchange Q for R because of fork
18
Fork: Traditional AI search Only 3 legal moves for
black Find that one of white’s
next moves (out of 23 possible) captures black queen
Check all following moves for more piece exchanges
Sometimes, still check other moves (non capturing)
At end of search – compare remaining pieces
No fork “concept”
19
Features Example – Fork Feature Search (GP)
One of the features is isMyFork function – Checks all previously defined conditions
Also, use some smaller building blocks:
Is Opponent piece Attacked?
Is attacking piece protected?
Is opponent in check? Value of attacked piece
20
GP Problem Definition
Terminals & Functions
Fitness
Run parameters
Termination
We will see each element in the
following experiments
21
Endgame experiments conducted
KRKR – each player has 1 king and 1 rook
KRRKR – King with 2 Rook against King
and Rook
KRRKRR
KQKQ – Kings and Queens
KQRKQR – Combined
22
Basic program architecture
Generate all possible moves (depth=1)
Evaluate each board with GP individual
Select board with best score (or
stochastically decide between equal)
Perform best move
Repeat process with GP opponent until
game ends (or until only kings left)
23
KRKR Endgame Each player has 1 King, 1 Rook
“Toy” problem for chess endgames
Theoretical draw (experts never lose this)
Some interesting positions exist
24
KRKR Endgame - what needs to be learned
(1)
Avoid losing rook Don’t move to
attacked, unprotected squares
Vice versa - capture opponent’s rook if able
Black to move – white loses Rook
25
KRKR Endgame - what need to be learned (2) Avoid getting king
stuck in edges
Again, take advantage if opponent does this
Black to move – mate in 1
26
KRKR Endgames - Terminals Used in first runs: Is My Rook Attacked, Is Opp Rook
attacked Is {My, Opp} Rook Protected (two as
above) Is {My, Opp} Rook In Play Num Moves {My, Opp} king {My, Opp}-King’s proximity to edges Is Mate ERCs: ± {0.25, 0.5, 1} * MAX
MAX = 1000 (empirically)
27
KRKR Endgames - Functions Boolean
OR2, OR3, OR4
AND2, AND3, AND4
NOT
Arithmetic - +, -, *
Combined - <, =, >, IF
STGP
For now, no “chess” functions, only
terminals
28
KRKR Endgames - Fitness
Competitive, Random-2-ways Each individual plays against k randomly
selected opponents
Each game counts for both players
For each encounter Several games (typically 4) are played
Short games - ~5-8 moves per player
Each game starts at a random legal position
Safe start - no piece is attacked at the
beginning
29
KRKR Endgames – Fitness (2)
Scoring method: Victory: 1-2 points
Piece count advantage (theoretical win) – ¾
point
Draw: ½ point After advantage – 0 points
Loss: 0 points
30
KRKR Endgames – Parameters
Population size - 80
#Generations - 150..250
Operators: Reproduction 0.35
Crossover 0.5
Mutation 0.15 (including ERC mutation)
Termination – ~10-20 hours
31
KRKR Endgames – Results Every 10 generations, best individual
played against: Best of generation 0
An opponent performing random moves
Longer games: ~10-12 moves per player
50-150 games
Games were doubled – each player
staring from both positions
32
KRKR Endgames – Results
Bad results – no distinct improvement
Several reasons: Arithmetic operations problematic – we get
large numbers
Mate not distinct enough (traditionally
terminates the search)
Boolean functions not clear enough
Slow Runs due to large trees with
repeating functions
33
KRKR Endgames –Improvements
Boolean functions Divided to good and bad
Example: Is-My-King-In-Check changed to Is-
My-King-Not-In-Check
Mate changed to 1000*Mate
Added Not-My-Rook-Attacked-Unprotected
and Opp-Rook-Attacked-Unprotected
34
KRKR Endgames – Results - Improvements
Also consulted Chess
Experts – added
more:
Is-Opp-King-Behind-
Rook
Split to Opp-King-Prox-Rook
Opp-King-Behind-Rook
Is-Stalemate (only
kings left)Black moves and White loses Rook
35
KRKR Endgames – Results - Improvements
Arithmetic functions canceled Although Still using floats for terminals Also divide to good and bad:
NumNotMovesOppKing Theoretically justified – more “logical”
search in literature Empirically - need more logical rules, and
not : ( > (+ (#moves-k #moves-opp-k) 5.5)) Memoization – saves more than ½ the
time
36
KRKR Endgames – Final Results
Improvement Above 75% of games against random end in
advantage or mate
Still, too few mates, even when score for win
is increased – difficult to learn move sequence
Same against best of generation 0 (almost
random)
The main thing that was learned was avoiding
getting the rook captured
37
KRRKR Endgames Example (right)
Very good for white Black king
exposed 2 rooks close Next move –
captures rook (mate in 5)
38
KRRKR Endgames - goals One player has 2 rooks, the other – 1
Not theoretically drawn
We want one generalized individual for
all endgames and positions (Not one for
each endgame): Each player needs to play both advantage,
draw (KRKR) and disadvantage
Terminals need to be more general
39
KRRKR Endgames - changes Terminals - changed and added to cope
with changing state Material-Count (recall each rook = 5)
Num-My-Pieces-Not-Attacked, since now there
are more than 1
Is-My-King-Protecting-Piece and My-Officers-
Same-Line to allow more complex considerations
Functions If-Adv-Then-(left child)-Else-(right child)
Eventually divided to 3 trees
40
KRRKR Endgames - changes
Also added – comparing differences to
parent node Boolean Is-Material-Increase, which
compares to the parent node (board)
Material decrease is not needed since
considering only my move
Not-My-King-Moves-Decrease to further use
number of moves for king
41
KRRKR Endgames – Opponents
Random forsaken; Best-of-0 still used but less
Added new opponent – MASTER a program we wrote based on consultation with
experts, highest being International Master Boris Gutkin, ELO 2400 (only about 3000 of those…)
Used ~50 general positions and rules derived from them, together with scores for each
Defined a strategy (“Expert”) accordingly Tested evolved programs against it
Human competitive?
42
KRRKR Endgames – Fitness
Test were conducted by assigning each player both roles for each position
Fitness was refined – score effected by: Starting position (advantage or
disadvantage) End result – win, loss or draw Adv position ending in draw receives a
score of near zero Dis-adv ending in a draw will receive better
than 0.5
43
KRRKR Endgames – Results
Expert-defined performed extremely well
against Random and Best0
Evolved programs performed generally as
well as expert defined, sometimes better
Percent of favorable results in game outcomes
44
Main Experiment – KQRKQR Most complex
endgame we worked
with
Still theoretical draw
Highly position
dependant – “noisy”
Larger trees 2 officers
Queens
Easier to mate
45
KQRKQR Endgames - changes
Added – more “heavy” terminals (and
components) Boolean Is-Not-Mate-in-one, most time
consuming but necessary
Boolean Is-My-King-Not-Trapped Not all king’s moves lead closer to edges
Important but vague – usually happens with
complex terminals My-Officers-Same-Line
46
Simple TerminalsComplex TerminalsFunction SetNotMyKingInCheckEvaluateMaterialIf3(B,F,F)
IsOppKingInCheckIsMaterialIncreaseOr2(B,B)
MyKingDistEdgesIsMateOr3(B,B,B)
OppKingProximityToEdges
IsMateInOneAnd2(B,B)
NumMyPiecesNotAttacked
OppPieceCanBeCapturedAnd3(B,B,B)
NumOppPiecesAttackedMyPieceCannotBeCaptured
Smaller(B,B)
ValueMyPiecesAttackingIsOppKingStuckNot(B)
ValueOppPiecesAttackingIsMyKingNotStuck
IsMyQueenNotAttackedIsOppKingBehindPiece
IsOppQueenAttackedIsMyKingNotBehindPiece
IsMyForkIsOppPiecePinned
IsOppNotForkIsMyPieceNotPinned
NumMovesMyKing
NumNotMovesOppKing
MyKingProxRook
OppKingDistRook
MyPiecesSameLine
OppPiecesNotSameLine
IsOppKingProtectingPiece
IsMyKingProtectingPiece
Genome Summary
47
KQRKQR Endgames – New Opponent
CRAFTY, second in the 2004 Computer
Chess Championship (held at Bar-Ilan)
Uses brute force methods; State-of-the-
art search algorithms
Specializes in Blitz games (typically 3
minutes per game)
We limited to 5 secs per move, enough to
scan ~1.5 Million boards with pruning
48
KQRKQR Endgames – Our parameters
Used lookahead of 2 Typically ~5 secs per move
Simple Minimax search, but not Alpha-Beta
Played 5-6 moves per game
Never cancelled a game, even if it started
with mate-in-4 (which CRAFTY easily saw) Played each position 2 times, switching places
~100 games - reduce noises in starting
positions
49
Results: Master
50
Results: CRAFTY
51
Multiple Endgames Aim for general-purpose strategies
All endgames used during evolution
Results:
%Wins%Advs%Draws
Master6268
CRAFTY2472
52
Sample GP-EndchessTree 0: (If3 (Or2 (Not (Or2 (And2 OppPieceAttUnprotected NotMyKingInCheck) (Or2 NotMyPieceAttUnprotected 100*Increase))) (And2 (Or3 (And2 OppKingStuck NotMyPieceAttUnprotected) (And2 OppPieceAttUnprotected OppKingStuck) (And3 -1000*MateInOne OppKingInCheckPieceBehind NotMyKingStuck)) (Or2 (Not NotMyKingStuck) OppKingInCheck))) NumMyPiecesUNATT (If3 (< (If3 (Or2 NotMyPieceAttUnprotected NotMyKingInCheck) (If3 NotMyPieceAttUnprotected #NotMovesOppKing OppKingInCheckPieceBehind) (If3 OppKingStuck OppKingInCheckPieceBehind -1000*MateInOne)) (If3 (And2 100*Increase 1000*Mate?) (If3 (< NumMyPiecesUNATT (If3 NotMyPieceAttUnprotected -1000*MateInOne OppKingProxEdges)) (If3 (< MyKingDistEdges #NotMovesOppKing) (If3 -1000*MateInOne -1000*MateInOne NotMyPieceATT) (If3 100*Increase #MovesMyKing OppKingInCheckPieceBehind)) NumOppPiecesATT) (If3 NotMyKingStuck -100.0 OppKingProxEdges))) (If3 OppKingInCheck (If3 (Or2 NotMyPieceAttUnprotected NotMyKingInCheck) (If3 (< MyKingDistEdges #NotMovesOppKing) (If3 -1000*MateInOne -1000*MateInOne NotMyPieceATT) (If3 100*Increase #MovesMyKing OppKingInCheckPieceBehind)) NumOppPiecesATT) (If3 (And3 -1000*MateInOne NotMyPieceAttUnprotected 100*Increase) (If3 (< NumMyPiecesUNATT (If3 NotMyPieceAttUnprotected -1000*MateInOne OppKingProxEdges)) (If3 (< MyKingDistEdges #NotMovesOppKing) (If3 -1000*MateInOne -1000*MateInOne NotMyPieceATT) (If3 100*Increase #MovesMyKing OppKingInCheckPieceBehind)) NumOppPiecesATT) -1000*MateInOne)) (If3 (< (If3 100*Increase MyKingDistEdges 100*Increase) (If3 OppKingStuck OppKingInCheckPieceBehind -1000*MateInOne)) -100.0 (If3 (And2 NotMyPieceAttUnprotected -1000*MateInOne) (If3 (< NumMyPiecesUNATT (If3 NotMyPieceAttUnprotected -1000*MateInOne OppKingProxEdges)) (If3 (< MyKingDistEdges #NotMovesOppKing) (If3 -1000*MateInOne -1000*MateInOne NotMyPieceATT) (If3 100*Increase #MovesMyKing OppKingInCheckPieceBehind)) NumOppPiecesATT) (If3 OppPieceAttUnprotected NumMyPiecesUNATT MyFork)))))Tree 1: (If3 NotMyPieceAttUnprotected #NotMovesOppKing 1000*Mate?)Tree 2: (If3 1000*Mate? NumMyPiecesUNATT -1000*MateInOne)
53
Summary
Draw and better against Master-defined
Draw against a world class opponent On limited conditions (not a full game, time ,etc.)
Shows deep search may have an alternative
Fast, pattern-oriented search suggests
more human resemblance
Search and lookahead are still important
54
Future Work Add more pieces Improve evolution speed
Parallel nets Stronger board representations
Develop more cognitive models using evolution
Search scheme space as well as game space
Tackle beyond endgames Openings and mid-game
General game concept schemas (?)