Multi-Agent Systems: The Tao of Soccer A tutorial presented at SFU Surrey Yu Zhang March 8th, 2005

Multi-Agent Systems:Multi-Agent Systems:

The Tao of SoccerThe Tao of Soccer

A tutorial presented at SFU A tutorial presented at SFU SurreySurrey

Yu ZhangYu ZhangMarch 8th, 2005March 8th, 2005

OverviewOverview• Multi-Agent Systems.Multi-Agent Systems.

• Intelligent Agents, agent designsIntelligent Agents, agent designs• multi-agent systems.multi-agent systems.

• The Tao of Soccer.The Tao of Soccer.• BackgroundBackground• Client-Server ArchitectureClient-Server Architecture• GUI client and Java3D viewGUI client and Java3D view• TOS AI programming guideTOS AI programming guide• TOS DemoTOS Demo

• The EndThe End• Questions & AnswersQuestions & Answers

What is an agent?What is an agent?

• An agent is an entity An agent is an entity that perceives its that perceives its environment through environment through sensors and acts upon sensors and acts upon that environment that environment through effectors.[1]through effectors.[1]

•For each possible percept sequence, an ideal For each possible percept sequence, an ideal rational agent should do whatever action is expected rational agent should do whatever action is expected to maximize its performance measure, on the basis to maximize its performance measure, on the basis of the evidence provided by the percept sequence of the evidence provided by the percept sequence and whatever built-in knowledge the agent has.[1]and whatever built-in knowledge the agent has.[1]

Design Intelligent AgentsDesign Intelligent Agents• The job of AI is to design the agent program that The job of AI is to design the agent program that

implements the agent mapping from percepts to actions.[1]implements the agent mapping from percepts to actions.[1]

• In order to design an agent program, we need to know the In order to design an agent program, we need to know the possible percepts and actions, what goals or performance possible percepts and actions, what goals or performance measure that agent is supposed to achieve, and what sort measure that agent is supposed to achieve, and what sort of environment it will operate in.of environment it will operate in.

• We will consider three basic types of intelligent agents: We will consider three basic types of intelligent agents: Simple reflex agents, agents that keep track of the world Simple reflex agents, agents that keep track of the world and goal-based agents.and goal-based agents.

Agent Agent TypeType

PerceptsPercepts ActionsActions GoalsGoals EnvironmeEnvironmentnt

Virtual Virtual soccer soccer playerplayer

Position Position and speed and speed of other of other players players and the and the ballball

Move, Move, kick, kick, dribble dribble and talkand talk

Beat the Beat the other other team, win team, win the gamethe game

Virtual Virtual soccer soccer fieldfield

Simple reflex agentsSimple reflex agents

• A simple reflex agent[1] A simple reflex agent[1] stores a set of stores a set of condition-action rules. condition-action rules. For a TOS soccer player, For a TOS soccer player, we can have rules like:we can have rules like:– If have-ball and close-to-If have-ball and close-to-

opponent-goal then shootopponent-goal then shoot

• The simple reflex agent The simple reflex agent works well only if the works well only if the correct decision can be correct decision can be made on the basis of made on the basis of the current percept.the current percept.

• ProblemProblem: But certain : But certain states can not be states can not be determined only by the determined only by the current percept.current percept.– A soccer player is A soccer player is

dribbling the ball.dribbling the ball.

Agents that keep track of the Agents that keep track of the worldworld

• This type of agents This type of agents maintains some internal maintains some internal state information in state information in order to better order to better determine world states determine world states and conditions.[1]and conditions.[1]

• The agent program also The agent program also maintains two more maintains two more types of knowledge.types of knowledge.– How the world evolves How the world evolves

independently of the independently of the agent.agent.

– How’s the agent’s own How’s the agent’s own actions affect the world.actions affect the world.

• ProblemProblem: Reflex agents : Reflex agents are not flexible. The are not flexible. The rules has to be re-written rules has to be re-written to adapt to a new goal or to adapt to a new goal or a new environment.a new environment.

Goal-based agentsGoal-based agents• Goal-based agents[1] Goal-based agents[1]

store goal information store goal information and the information and the information about the results of about the results of possible actions in order possible actions in order to choose actions that to choose actions that achieve the goal.achieve the goal.

• Goal-based agents Goal-based agents reasons about the reasons about the future.future.

• How to set goals?How to set goals?– The designer manually set the goals for the agent.The designer manually set the goals for the agent.– The agent sets its own goals by defining a utility function.The agent sets its own goals by defining a utility function.

• Utility is a function that maps a state to a real number, Utility is a function that maps a state to a real number, which represents the associated degree of happiness.which represents the associated degree of happiness.

Layered agent architectureLayered agent architecture • A layered agent architecture is A layered agent architecture is

modeled as a hierarchy of modeled as a hierarchy of interactive sub agent modules.[3]interactive sub agent modules.[3]

• The higher layers are usually The higher layers are usually event-driven, goal-based.event-driven, goal-based.

• The lower layers are fixed-sample-The lower layers are fixed-sample-time driven, simple-reflex-based.time driven, simple-reflex-based.

• Each layer is a sub-agent with its Each layer is a sub-agent with its own inputs and outputs.own inputs and outputs.– Inputs: controls signals from the Inputs: controls signals from the

higher layer, the sensing signals higher layer, the sensing signals from the environment and the from the environment and the states from the lower layer.states from the lower layer.

– Outputs: controls signals to the Outputs: controls signals to the lower layer and the states to the lower layer and the states to the higher layer.higher layer.

• Control commands flow down and Control commands flow down and the sensing information flow up.the sensing information flow up.

Multi-agent systemsMulti-agent systems• A multi-agent system is a group of agents that interact to A multi-agent system is a group of agents that interact to

solve problems that are beyond the individual capabilities.[2]solve problems that are beyond the individual capabilities.[2]• Multi-agent system design is more complicated than a single Multi-agent system design is more complicated than a single

agent design. It presents many new cooperation issues.agent design. It presents many new cooperation issues.• A multi-agent system solves the big problem by “divide and A multi-agent system solves the big problem by “divide and

conquer”. This presents us…conquer”. This presents us…• Cooperation issue 1 - Cooperation issue 1 - Task allocationTask allocation: how to assign : how to assign

responsibility to a single agent?responsibility to a single agent?– Solution: each agent is assigned a role within the team with its Solution: each agent is assigned a role within the team with its

own behavior expectations and goals.own behavior expectations and goals.• Now agents may have conflicting knowledge, actions and Now agents may have conflicting knowledge, actions and

goals. This presents us…goals. This presents us…• Cooperation issue 2 – Cooperation issue 2 – Resolving ConflictsResolving Conflicts: How to resolve : How to resolve

conflicting knowledge, actions and goals among agents. conflicting knowledge, actions and goals among agents. – Solution: assign authority levels to roles, communication and Solution: assign authority levels to roles, communication and

negotiation among agents.negotiation among agents.• Now agents need to communicate, this presents us…Now agents need to communicate, this presents us…• Cooperation issue 3 – Cooperation issue 3 – CommunicationCommunication: How to understand : How to understand

each other?each other?– Solution: develop a language and a communication system that Solution: develop a language and a communication system that

can accept and interpret this language. can accept and interpret this language.

The The TaoTao of Soccer of Soccer

direction; way; direction; way; method; road; path; method; road; path; principle; truth; principle; truth; reason; skill; method; reason; skill; method; Tao (of Taoism); a Tao (of Taoism); a measure word; to measure word; to say; to speak; to talk say; to speak; to talk

:Tao:Tao

Background and HistoryBackground and History• 1992 – Prof. Alan Mackworth proposed 1992 – Prof. Alan Mackworth proposed

robotic soccer as a test bed for multi-robotic soccer as a test bed for multi-agent system research and started the agent system research and started the world's first soccer-playing robots world's first soccer-playing robots dynamitedynamite. .

• 1997 – The first 1997 – The first RoboCupRoboCup was held was held during the International Joint Conference during the International Joint Conference on Artificial Intelligence (IJCAI), in on Artificial Intelligence (IJCAI), in Nagoya, Japan. Nagoya, Japan. – Soccer Server is the official simulator for Soccer Server is the official simulator for

the RoboCup Simulation League.the RoboCup Simulation League. – Written in C++, complicated client/server Written in C++, complicated client/server

protocol, no human playing ability, only protocol, no human playing ability, only runs on UNIX type machines.runs on UNIX type machines.

• 2001 – The Tao of Soccer was created.2001 – The Tao of Soccer was created.– Written in pure Java, similar to RoboCup Written in pure Java, similar to RoboCup

but with simplified client/server protocol, but with simplified client/server protocol, and it can be played by human as well. and it can be played by human as well.

TOS: a soccer environment TOS: a soccer environment simulator simulator • For each simulation step, TOS For each simulation step, TOS

calculates and gives each agent calculates and gives each agent its percept, gets an action from its percept, gets an action from each agent, and then updates each agent, and then updates the environment.the environment.

• TOS Simulator has five main TOS Simulator has five main classes.classes.– SoccerWorld: represents the state SoccerWorld: represents the state

of TOS environment. A soccer of TOS environment. A soccer field, two teams of players, one field, two teams of players, one group of viewers and a ball.group of viewers and a ball.

– Host: a thread waits for Host: a thread waits for connection requests and actions connection requests and actions from agents.from agents.

– HeartOfWorld: a thread calculates HeartOfWorld: a thread calculates and gives each agent its percept, and gives each agent its percept, updates the SoccerWorld at every updates the SoccerWorld at every simulation step.simulation step.

– SoccerRules: simulates and SoccerRules: simulates and calculates the soccer rules, such calculates the soccer rules, such as off-side, goal scores.as off-side, goal scores.

– SoccerPhysics: simulates the SoccerPhysics: simulates the movement of player and the ball.movement of player and the ball.

Soccer Physics simulationSoccer Physics simulation• Soccer field is 2D rectangular, 100 x 65 meters. The center of Soccer field is 2D rectangular, 100 x 65 meters. The center of

the soccer field is set to (0,0), Y goes up, X goes right.the soccer field is set to (0,0), Y goes up, X goes right. • The players and the ball are treated as circles. Movements of The players and the ball are treated as circles. Movements of

these objects are simulated stepwise for every 50 milliseconds. these objects are simulated stepwise for every 50 milliseconds. (24steps/sec)(24steps/sec)

• At each simulation step, movement of each player is calculated At each simulation step, movement of each player is calculated in the following manner:in the following manner:– P1 = P0 + V0; P1 = P0 + V0; – V1 = V0 + A0; V1 = V0 + A0; – A1 = FORCE * K1 - V0 * K2;A1 = FORCE * K1 - V0 * K2;

• movement of the ball is calculated as:movement of the ball is calculated as:– P1 = P0 + V0; P1 = P0 + V0; – V1 = V0 + A0; V1 = V0 + A0; – If (kicked by a player) If (kicked by a player)

{ { A1 = KICKFORCE * K1;A1 = KICKFORCE * K1;V1 = 0;V1 = 0;

} } else A1 = -FRICTIONFACTOR * V0; else A1 = -FRICTIONFACTOR * V0;

• Collisions: When more than two players overlap, all of them are Collisions: When more than two players overlap, all of them are moved back until they do not overlap. Then their velocities are moved back until they do not overlap. Then their velocities are multiplied by -0.1.multiplied by -0.1.

Soccer Rules simulationSoccer Rules simulation• Once the server starts, it enters a 4-period match cycle Once the server starts, it enters a 4-period match cycle

repeatedly until it is turned off.repeatedly until it is turned off. – Pre Game: The referee is not activated. No score is recorded.Pre Game: The referee is not activated. No score is recorded.– First Half: The game starts. The referee is activated to enforce First Half: The game starts. The referee is activated to enforce

the soccer rule. the soccer rule. – Half Time: The referee is deactivated again. Half Time: The referee is deactivated again. – Second Half: The game restarts. The referee is reactivated to Second Half: The game restarts. The referee is reactivated to

enforce the soccer rule. enforce the soccer rule.

• A kick-off is a way of starting or restarting play at the start A kick-off is a way of starting or restarting play at the start of the match, or after a goal has been scored or at the start of the match, or after a goal has been scored or at the start of the second half of the match. When the kick-off mode is of the second half of the match. When the kick-off mode is on, the opponents of the team taking the kick-off are at on, the opponents of the team taking the kick-off are at least 9 meters from the ball until the ball is touched by an least 9 meters from the ball until the ball is touched by an opponent player. opponent player.

• Other implemented soccer rules are:Other implemented soccer rules are:– Goal kickGoal kick– Corner kickCorner kick– Throw inThrow in– Offside, the offside rule can be turned off by a command-line Offside, the offside rule can be turned off by a command-line

switchswitch

TOS: client-server architectureTOS: client-server architecture• TOS runs on internet TOS runs on internet

using the client-server using the client-server architecture.architecture.

• TOS simulator runs as TOS simulator runs as the server to provide the server to provide percepts to the agents percepts to the agents and receives actions and receives actions from them on the from them on the internet.internet.

• TOS agents run as the TOS agents run as the clients to receives clients to receives percepts from the percepts from the server and send actions server and send actions to the server on the to the server on the internet.internet.

• Percepts and actions Percepts and actions are encoded as LISP-like are encoded as LISP-like text messages, packed text messages, packed and transmitted as and transmitted as UDP/IP packets between UDP/IP packets between the server and client.the server and client.

• A client can be implemented in any A client can be implemented in any language.language.• A Java client can use built-in API A Java client can use built-in API soccer.common to communicate with the soccer.common to communicate with the server.server.• There are two types of TOS clients There are two types of TOS clients (agents), Player and Viewer.(agents), Player and Viewer.

Client to Server: ConnectClient to Server: Connect• First, a client has to join the game by sending a connection First, a client has to join the game by sending a connection

message to the server.message to the server.– (c ClientType SideType). For example (c p l)(c ClientType SideType). For example (c p l)– To do this in Java, we use API in soccer.commonTo do this in Java, we use API in soccer.common// step 0: setup network communication, usually only done once// step 0: setup network communication, usually only done onceTransceiver transceiver = new Transceiver( false );Transceiver transceiver = new Transceiver( false );

// step 1: fill the data// step 1: fill the dataConnectData connect = new ConnectData connect = new

ConnectData(ConnectData.PLAYER,ConnectData.LEFT);ConnectData(ConnectData.PLAYER,ConnectData.LEFT);// step 2: create the UDP packet by providing data and server // step 2: create the UDP packet by providing data and server

addressaddressPacket packet = new Packet( Packet.CONNECT, connect, address, Packet packet = new Packet( Packet.CONNECT, connect, address,

port);port);// step 3: send the packet// step 3: send the packettransceiver.send( packet );transceiver.send( packet );

• What if the connection message got lost during the What if the connection message got lost during the transmission? Or the team is full? transmission? Or the team is full? – Solution: the server sends back an acknowledge packet.Solution: the server sends back an acknowledge packet.

Server to Client: InitializeServer to Client: Initialize• The sever responds the connection request from the client by The sever responds the connection request from the client by

sending an initialization message back.sending an initialization message back.– (i ClientType Number)(i ClientType Number)– ClientType can be one of ClientType can be one of ll, , rr, , vv or or ff..

ll means left, means left, rr means right, means right, vv means means viewerviewer and and ff means means fullfull. In the case of . In the case of leftleft or or rightright, , NumberNumber is returned as the is returned as the player's player number. player's player number. fullfull means connection fails means connection fails because of the full team. because of the full team.

• To make the connection, the client can send a connection To make the connection, the client can send a connection packet every 60 seconds untilpacket every 60 seconds until– it receives an initialization packet from the server, which means it receives an initialization packet from the server, which means

a successful connection.a successful connection.– or it reaches the maximum number of tries, and fails the or it reaches the maximum number of tries, and fails the

connection.connection.

Server to Client: Player Server to Client: Player PerceptsPercepts• After the client-server connection is established, the server After the client-server connection is established, the server

begins to send visual information to the client. begins to send visual information to the client. • If the client is a player, it gets If the client is a player, it gets ssee packetee packet

– (s Time (s Time Side Id X Y Dir StatusSide Id X Y Dir Status BallX BallY BallControllerType BallX BallY BallControllerType BallControllerId PlayerInfo PlayerInfo ...); BallControllerId PlayerInfo PlayerInfo ...);

– PlayerInfo ::= (PlayerSide PlayerId X Y Dir) PlayerInfo ::= (PlayerSide PlayerId X Y Dir) • If the client is a viewer, it gets If the client is a viewer, it gets vview packetiew packet

– (v Time BallX BallY BallControllerType BallControllerId PlayerInfo (v Time BallX BallY BallControllerType BallControllerId PlayerInfo PlayerInfo ...);PlayerInfo ...);

• The difference between The difference between ssee and ee and vview packet is: view packet iew packet is: view packet does not have visual data for viewer himself.does not have visual data for viewer himself.

• We use soccer.common API to process visual data.We use soccer.common API to process visual data.

packet = transceiver.receive(); packet = transceiver.receive(); if(packet.packetType == Packet.SEE)if(packet.packetType == Packet.SEE)

{ { see = (SeeData) packet.data; see = (SeeData) packet.data;

distance2Ball = distance2Ball = see.player.position.distance( see.ball.position );see.player.position.distance( see.ball.position );

direction2Ball = direction2Ball = see.player.position.direction( see.ball.position );see.player.position.direction( see.ball.position );

}}

Server to Client: Player Server to Client: Player PerceptsPercepts• The server also sends referee information to the client The server also sends referee information to the client

whenever the situation on the field is changed. whenever the situation on the field is changed. – (r Time Period Mode LeftName LeftGoal RightName RightGoal)(r Time Period Mode LeftName LeftGoal RightName RightGoal) – Time ::= simulation step;Time ::= simulation step;– Period ::= 0 | 1 | 2 | 3; Period ::= 0 | 1 | 2 | 3; 00 means means preGamepreGame, , 11 means means firstHalffirstHalf, , 22 means means

halfTimehalfTime, , 33 means means secondHalf.secondHalf.– Mode ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11;Mode ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11;

00 means means beforeKickOffbeforeKickOff, , 11 means means kickOffLkickOffL, , 22 means means kickOffRkickOffR, , 33 means means throwInLthrowInL, , 44 means means throwInRthrowInR, , 55 means means cornerKickLcornerKickL, , 66 means means cornerKickRcornerKickR, , 77 means means goalKickLgoalKickL, , 88 means means goalKickRgoalKickR, , 99 means means offsideLoffsideL, , 1010 means means offsideRoffsideR, , 1111 means means playOn.playOn.

• We use soccer.common API to process referee data.We use soccer.common API to process referee data.

packet = transceiver.receive(); packet = transceiver.receive(); if(packet.packetType == Packet.REFEREE)if(packet.packetType == Packet.REFEREE)

{ { referee = (RefereeData)packet.data; referee = (RefereeData)packet.data;

gamemode = referee.mode;gamemode = referee.mode; gamePeriod = referee.period;gamePeriod = referee.period;

}}

Client to Server: Player ActionsClient to Server: Player Actions• A player can move around by sending the drive command to A player can move around by sending the drive command to

the server.the server.– (d Dir Force) (d Dir Force)

• -180 < Dir < 180-180 < Dir < 180

• -30 < Force < 100-30 < Force < 100

– To do this in Java, we use API in soccer.commonTo do this in Java, we use API in soccer.common

// step 0: setup network communication, usually only done once// step 0: setup network communication, usually only done once

Transceiver transceiver = new Transceiver( false );Transceiver transceiver = new Transceiver( false );

// step 1: fill the data// step 1: fill the data

DriveData driver = new DriveData( direction, force );DriveData driver = new DriveData( direction, force );

// step 2: create the UDP packet// step 2: create the UDP packet

Packet packet = new Packet( Packet.DRIVE, driver, address, port);Packet packet = new Packet( Packet.DRIVE, driver, address, port);

// step 3: send the packet// step 3: send the packet

transceiver.send( packet );transceiver.send( packet );

Client to Server: Player ActionsClient to Server: Player Actions• If the player is close enough to the ball, he can kick it by If the player is close enough to the ball, he can kick it by

sending the kick command to the server.sending the kick command to the server.– (k Dir Force) (k Dir Force)

• -180 < Dir < 180-180 < Dir < 180

• -30 < Force < 100-30 < Force < 100





KickData kicker = new KickData( direction, force );KickData kicker = new KickData( direction, force );


Packet packet = new Packet( Packet.KICK, kicker, address, port);Packet packet = new Packet( Packet.KICK, kicker, address, port);



Client to Server: Player ActionsClient to Server: Player Actions• A player can broadcast a message to everybody on the field A player can broadcast a message to everybody on the field

by sending the message to the server. by sending the message to the server. – (t Message) (t Message)

• The length of the Message <= 30The length of the Message <= 30

• At most one Message every 30 secondAt most one Message every 30 second





TalkData talker = new TalkData( “Attack! Attack! Attack!”);TalkData talker = new TalkData( “Attack! Attack! Attack!”);


Packet packet = new Packet( Packet.TALK, talker, address, port);Packet packet = new Packet( Packet.TALK, talker, address, port);



Server to Client: Player Server to Client: Player PerceptsPercepts• After the server receives a broadcast message from a client After the server receives a broadcast message from a client

, it sends audio information to all clients in the game., it sends audio information to all clients in the game.– (h Time Side Id Message)(h Time Side Id Message)

• We use soccer.common API to process audio data.We use soccer.common API to process audio data.

packet = transceiver.receive(); packet = transceiver.receive();

if(packet.packetType == Packet.HEAR)if(packet.packetType == Packet.HEAR)

{{ heard = (HearData) packet.data; heard = (HearData) packet.data;

String message = heard.message;String message = heard.message;

int speakerID = heard.id;int speakerID = heard.id;

int speakerSide = heard.side;int speakerSide = heard.side;

}}

SoccerMaster: a TOS GUI clientSoccerMaster: a TOS GUI client• SoccerMaster can be SoccerMaster can be

run in three modes.run in three modes.• In the VIEW mode, it In the VIEW mode, it

connects to the server connects to the server as a viewer client.as a viewer client.– The user observes The user observes

two AI teams play two AI teams play against each other.against each other.

• In the PLAY mode, it In the PLAY mode, it connects to the server connects to the server as a player client.as a player client.– The user plays the The user plays the

game himself.game himself.

• In the REPLAY mode, In the REPLAY mode, it reads a log file and it reads a log file and redisplays the match.redisplays the match.– The user reexamines The user reexamines

a recorded match.a recorded match.

• The user can hot-switch Java2D and The user can hot-switch Java2D and Java3D view any time, no application Java3D view any time, no application restart is needed.restart is needed.

TOS Java 3D ViewTOS Java 3D View• The java 3D view is The java 3D view is

implemented in package implemented in package soccer.client.view.j3dsoccer.client.view.j3d..

• FieldJ3DFieldJ3D is the main class for is the main class for displaying the 3D scene. In this displaying the 3D scene. In this class, we add content nodes to class, we add content nodes to the Java3D scene graph tree; the Java3D scene graph tree; such as soccer players, the ball, such as soccer players, the ball, the soccer field, the goal poles, the soccer field, the goal poles, etc.etc.

• RobotRobot contains the manually contains the manually coded player 3D model using coded player 3D model using spheres and cylinders.spheres and cylinders.

• MouseRotateXZMouseRotateXZ and and MouseZoomOnRightClickMouseZoomOnRightClick are are used to control the camera view. used to control the camera view.

VRML97Loader modLoader = new VRML97Loader();VRML97Loader modLoader = new VRML97Loader();URL modURL = getClass().getResource("/model/balloon.wrl");URL modURL = getClass().getResource("/model/balloon.wrl");Scene model = modLoader.load(modURL);Scene model = modLoader.load(modURL);BranchGroup balloon = model.getSceneGroup();BranchGroup balloon = model.getSceneGroup();scene.addChild(balloon);scene.addChild(balloon);

It’s very easy to add existing 3D It’s very easy to add existing 3D models to TOS 3D scene. For models to TOS 3D scene. For example, to load a balloon into example, to load a balloon into the 3D scene using the 3D scene using Xj3DXj3D loader loader library:library:

SFU Default AI Soccer TeamSFU Default AI Soccer Team• In the package In the package

edu.sfu.soccer.agentedu.sfu.soccer.agent, , there are 4 classes. there are 4 classes. AIPlayers, Formation, AIPlayers, Formation, Robot, and WorldModel. Robot, and WorldModel.

• AIPlayersAIPlayers is the main is the main class, it maintains one class, it maintains one or two teams of threads, or two teams of threads, each thread controls an each thread controls an independent TOS player.independent TOS player.

• RobotRobot contains the logic contains the logic for controlling the TOS for controlling the TOS player.player.

• WorldModelWorldModel contains contains the internal states, the internal states, actions and team actions and team formation.formation.

• FormationFormation contains the contains the team formation data.team formation data.• Each Robot is a Each Robot is a goal-based agentgoal-based agent, ,

uses three main methods, sense(), uses three main methods, sense(), plan() and execute(). plan() and execute().

AIPlayers: the main classAIPlayers: the main class• AIPlayers maintains one or two teams of Robots. Each team AIPlayers maintains one or two teams of Robots. Each team

can have 0-11 players. The size of each team is given at the can have 0-11 players. The size of each team is given at the command line. For example:command line. For example:– To control 11 players of the left team.To control 11 players of the left team.

java -cp soccer.jar edu.sfu.soccer.agent.AIPlayers -l 11 -r 0java -cp soccer.jar edu.sfu.soccer.agent.AIPlayers -l 11 -r 0– To control 11 players of the right team.To control 11 players of the right team.

java -cp soccer.jar edu.sfu.soccer.agent.AIPlayers -l 0 -r 11java -cp soccer.jar edu.sfu.soccer.agent.AIPlayers -l 0 -r 11

• Each player is run as a thread. Why do we use threads?Each player is run as a thread. Why do we use threads?– To soccer server, each separate UDP socket represents a unique To soccer server, each separate UDP socket represents a unique

soccer client. There’s no difference between a process client and soccer client. There’s no difference between a process client and a thread client.a thread client.

– A thread uses less computing resources than a process.A thread uses less computing resources than a process.– One Java process can control 22 AI players. It’s easier for us to One Java process can control 22 AI players. It’s easier for us to

use and to maintain the soccer AI.use and to maintain the soccer AI.

WorldModel and FormationWorldModel and Formation• WorldModelWorldModel is the memory bank for our soccer agent, it is the memory bank for our soccer agent, it

stores:stores:– How the world evolvesHow the world evolves..– 1, The static information about the soccer field, such as length 1, The static information about the soccer field, such as length

and width.and width.– 2, The static information about the soccer physics simulation, 2, The static information about the soccer physics simulation,

such as the maximum speed of players and the ball.such as the maximum speed of players and the ball.– Internal statesInternal states..– 1, The low level percepts such as the positions of moving 1, The low level percepts such as the positions of moving

objects on the field.objects on the field.– 2, The high level knowledge such as my relative distance and 2, The high level knowledge such as my relative distance and

direction to the ball, if I’m interested in getting the ball, if I’m direction to the ball, if I’m interested in getting the ball, if I’m stuck, etc.stuck, etc.

– Multi-agentMulti-agent. My current role in my team formation.. My current role in my team formation.– The high level actions I’m going to execute, such as shoot, The high level actions I’m going to execute, such as shoot,

move, chase, avoid stuck and avoid offside.move, chase, avoid stuck and avoid offside.• FormationFormation stores the formation information of a soccer stores the formation information of a soccer

team. A soccer team can have different style of formation, team. A soccer team can have different style of formation, such as “433”, “523” and “424”.such as “433”, “523” and “424”.

Robot: a goal-based soccer Robot: a goal-based soccer agentagent

• RobotRobot implements a implements a goal-based agentgoal-based agent.. – Sense(): receives UDP packets from the server about its current Sense(): receives UDP packets from the server about its current

percepts, updates its world model.percepts, updates its world model.– Plan(): determines the high level actions, such as shoot, dribble, Plan(): determines the high level actions, such as shoot, dribble,

move and chase.move and chase.– Execute(): translates the high level actions into a sequence of Execute(): translates the high level actions into a sequence of

low level actions that can be accepted by the soccer server, low level actions that can be accepted by the soccer server, such as kick and drive. such as kick and drive.

• Plan() method calls a list of decision-making methods to Plan() method calls a list of decision-making methods to determine the next high level action.determine the next high level action.– ShouldIScore(): determines if I should shoot the goal.ShouldIScore(): determines if I should shoot the goal.– ShouldIDribble(): determines if I should dribble.ShouldIDribble(): determines if I should dribble.– ShouldIPass(): determines if I should pass the ball to a ShouldIPass(): determines if I should pass the ball to a

teammate.teammate.– DetermineWhereToMove() & DeterminePlayerPos(): determines DetermineWhereToMove() & DeterminePlayerPos(): determines

where should I move when I don’t have the ballwhere should I move when I don’t have the ball

Create your own soccer Create your own soccer teamteam

• Approach 1: Make changes to the default SFU soccer team.Approach 1: Make changes to the default SFU soccer team.– Change the Change the FormationFormation class, experiment with different team formations. class, experiment with different team formations.– Improve the decision-making methods in the Improve the decision-making methods in the RobotRobot class. These are class. These are

ShouldIPass(), ShouldIScore(), ShouldIDribble(), ShouldIPass(), ShouldIScore(), ShouldIDribble(), DetermineWhereToMove() and DeterminePlayerPos(): .DetermineWhereToMove() and DeterminePlayerPos(): .

• Approach 2: Create your own AI team from scratch.Approach 2: Create your own AI team from scratch.– If you have a great idea that is hard to implement in the default SFU If you have a great idea that is hard to implement in the default SFU

team.team.– If you want to experiment with some other agent architecture. One If you want to experiment with some other agent architecture. One

example of such alternative is example of such alternative is Dynamo98Dynamo98 that uses that uses constraint netsconstraint nets robotic architecture.robotic architecture.

– Dynamo98Dynamo98 is in package com.graviton.dynamo98. You may also build is in package com.graviton.dynamo98. You may also build your team on this AI implementation as well.your team on this AI implementation as well.

• Approach 3: Add new AI methods into the existing AI Approach 3: Add new AI methods into the existing AI implementations. implementations. – Include a rule-based engine, such as a Prolog or Fuzzy logic, to handle Include a rule-based engine, such as a Prolog or Fuzzy logic, to handle

high level planning tasks, instead of hard-coding the decision-making high level planning tasks, instead of hard-coding the decision-making methods.methods.

– Add learning capabilities, such as neural networks, decision trees, Add learning capabilities, such as neural networks, decision trees, reinforcement learning and genetic algorithms.reinforcement learning and genetic algorithms.

TOS DEMO: Play TOS as a TOS DEMO: Play TOS as a gamegame• Start the TOS server. Double click soccerserver.batStart the TOS server. Double click soccerserver.bat

• Start the TOS GUI. Double click soccer.batStart the TOS GUI. Double click soccer.bat

• Play the game with the coach ability on.Play the game with the coach ability on.

• Move the player around with the left mouse click.Move the player around with the left mouse click.

• Open a DOS window, type Open a DOS window, type sfu_team.bat bothsfu_team.bat both to start AI. to start AI.

• Click space to chase the ball.Click space to chase the ball.

• When you have the ball, right click to kick the ball, left kick to When you have the ball, right click to kick the ball, left kick to dribble, and space to shoot.dribble, and space to shoot.

• Step forward the game, continue, step forward.Step forward the game, continue, step forward.

• Forward the game period, play again.Forward the game period, play again.

• Toggle the player ID display.Toggle the player ID display.

• Toggle music.Toggle music.

• Toggle 3D display.Toggle 3D display.

• Left click to rotate the 3D view, right click to zoom in/out the 3D Left click to rotate the 3D view, right click to zoom in/out the 3D view.view.

• Reset the top-down 3D view.Reset the top-down 3D view.

• Stop the game.Stop the game.

TOS DEMO: TOS as a TOS DEMO: TOS as a simulationsimulation• Start the TOS server. Double click soccerserver.batStart the TOS server. Double click soccerserver.bat

• Open a DOS window, type Open a DOS window, type dynamo98.bat bothdynamo98.bat both to start AI. to start AI.

• Start the TOS GUI. Double click soccer.batStart the TOS GUI. Double click soccer.bat

• View the game with the coach ability on.View the game with the coach ability on.

• Step forward the game, continue, step forward.Step forward the game, continue, step forward.

• Forward the game period, play again.Forward the game period, play again.

• Toggle the player ID display.Toggle the player ID display.

• Toggle music.Toggle music.


• Left click to rotate the 3D view, right click to zoom in/out the Left click to rotate the 3D view, right click to zoom in/out the 3D view.3D view.


TOS DEMO: TOS ReplayTOS DEMO: TOS Replay• Start the TOS GUI. Double click soccer.batStart the TOS GUI. Double click soccer.bat

• Load the log file.Load the log file.

• Play the log file.Play the log file.

• Fast forward, normal, fast forward.Fast forward, normal, fast forward.

• Pause, play, pause, forward, pause, playPause, play, pause, forward, pause, play

• Backward play, play, backward play, fast backward.Backward play, play, backward play, fast backward.


• Left click to rotate the 3D view, right click to zoom in/out the Left click to rotate the 3D view, right click to zoom in/out the 3D view.3D view.


The End and the beginning. The End and the beginning. How to get help and to help How to get help and to help others?others?

• Visit TOS website at Visit TOS website at http://soccer.sourceforge.net/soccer/http://soccer.sourceforge.net/soccer/• Join the mailing list atJoin the mailing list at

http://games.groups.yahoo.com/group/tao_of_soccer/http://games.groups.yahoo.com/group/tao_of_soccer/• Post your questions and answers to the TOS forums or to the Post your questions and answers to the TOS forums or to the

TOS mailing list.TOS mailing list.• Make your AI team publicly available on the TOS web site.Make your AI team publicly available on the TOS web site.• Join the TOS development.Join the TOS development.

– Find and fix bugs.Find and fix bugs.– Add new functions to the server and the GUI.Add new functions to the server and the GUI.– Improve the 3D view.Improve the 3D view.– Make 3D models.Make 3D models.– Improve the documentation.Improve the documentation.– Help to maintain the TOS website.Help to maintain the TOS website.

• Thank you and Good Luck!Thank you and Good Luck!

http://soccer.sourceforge.net/soccer/

http://games.groups.yahoo.com/group/tao_of_soccer/

ReferencesReferences1.1. Artificial Intelligence, A Modern Approach. 1995, By Stuart Artificial Intelligence, A Modern Approach. 1995, By Stuart

Russell and Peter NorvigRussell and Peter Norvig

2.2. Multiagent Systems. 1998, By Katia Sycara. AI Magazine Multiagent Systems. 1998, By Katia Sycara. AI Magazine 19(2)19(2)

3.3. A Constraint-Based Robotic Soccer Team. 2002, By Yu A Constraint-Based Robotic Soccer Team. 2002, By Yu Zhang and Alan Mackworth. Constraints, 7, 7-28.Zhang and Alan Mackworth. Constraints, 7, 7-28.

Questions & AnswersQuestions & Answers

Documents

Multi-Agent Systems: The Tao of Soccer A tutorial presented at SFU Surrey Yu Zhang March 8th, 2005