peer-to-peer and agent-based computingBasic Theory of Agency

(Cont’d)

2

• Imagine two environmental states e1,e2Env, such that– e1 e2 and– see (e1) = see (e2)

• Such environments are different, but as far as agents are concerned they are indistinguishable!– The environments are mapped to the same percept– The agent would receive the same perceptual

information from different environment states– Agents cannot tell them apart…

Indistinguishable environments (1)

• Environment states consisting of– Boolean variable temp set to true if the temperature of

the room is above 35o Celsius (otherwise temp is false)– Boolean variable smoke set to true if smoke has been

detected in the environment (otherwise smoke is false)• The full set of environment states is

{temp,smoke,temp,smoke,temp,smoke,temp,smoke}

• The see function of a thermostat agent is:

3

Indistinguishable environments (2)

e1 e2 e3 e4

see (e) =p1 if e =e1 or e =e2

p2 if e =e3 or e =e4

4

Agents with State (1)• Now, consider agents that record information about

environment state and history:

• Let I be the set of all internal states.• The perception function see is unchanged:

see : E Per• The action selection function is now a mapping

action : I Acfrom internal states to actions.

• An additional function next is introduced, which maps an internal state and percept to an internal state:

next : I Per I

5

Agents with State (2)

6

Agent Control Loop

1. Agent starts in some initial internal state i0

2. icur = i0

3. Agent observes the environment state e and generates a percept see (e )

4. Agent’s internal state is updated via next functioninext = next (icur ,see (e ))

5. Action selected by agent is action (next (icur ,see (e ))

This action is then performed6. Go to 3 with icur = inext

7

Tasks for Agents• We build agents to carry out tasks for us.• The task must be specified… • However, we want to tell agents what to do without

telling them how to do it.• So, the question is:

– how do we give an agent information about what states are better than others?

• Possibility: associate utilities with individual states– Agents pursue states that maximise utility.

• A task specification is a function

u : E

which associates a real number with every state.

8

Utility Functions over States (1)

9

Utility Functions over States (2)• But what is the value of a run? Is it

– The minimum utility of all states in a run?– The maximum utility of all states in a run?– The sum of utilities of all states in a run?– The average of utilities of all states in a run?

• Disadvantage: – It is difficult to specify a long term view when assigning

utilities to individual states!

• Another possibility – assign utilities to runs rather than individual states:

u : R • This takes an inherently long term view.• Other variations:

– Incorporate probabilities of different states emerging• Difficulties with utility-based approaches:

– Where do the numbers come from?– We don’t think in terms of utilities!– Hard to formulate tasks in these terms…

10

Utilities over Runs

• Simulated 2D grid environment with– Agents– Tiles– Obstacles and holes

• Agents can move up, down, left or right– If agent is next to a tile, agent can push tile.

• Holes have to be filled up with tiles by the agent to score points.

• Aim: score lots of points!!• Holes appear and disappear at random…• Utility function of a run r :

u (r ) =

11

Utility in the TILEWORLD

number of holes filled in rnumber of holes that appeared in rdef

• P (r |Ag,Env ) denotes probability that run r occurs when agent Ag is placed in environment Env :

P (r |Ag,Env ) = 1

• The optimal agent Agopt in an environment Env is the one that maximises expected utility :

Agopt = arg max u (r ) P (r |Ag,Env)(1)r R(Ag,Env)Ag AG

12

Expected Utility & Optimal Agents

rR(Ag,Env)

• Some agents cannot be implemented on some computers; – E.g. a function Ag : RE Ac may need more memory

than what is available to implement.• AGm denotes those agents that can be implemented

on machine (computer) m:AGm = {Ag | Ag AG and Ag can be implemented on m}

• We can replace equation (1) with the following, which defines the bounded optimal agent Agopt:

Agopt = arg max u (r ) P (r |Ag,Env) (2)

13

r R(Ag,Env) Ag AGm

Bounded Optimal Agents

• A special case of assigning utilities to histories is to assign 0 (false) or 1 (true) to a run– If a run is assigned 1, then the agent succeeds on that

run– Otherwise it fails.

• We call these predicate task specifications• We denote a predicate task specification as :

: R {0,1}

14

Predicate Task Specifications

• A task environment is a pair Env,, where Env is an environment, and

: R {0,1}is a predicate over runs.

• Let TE be the set of all task environments.• A task environment specifies:

– The properties of the system the agent will inhabit– The criteria by which an agent will be judged to have

either failed or succeeded

15

Task Environments (1)

• R (Ag, Env ) denotes the set of all runs of agent Ag in environment Env that satisfy :

R (Ag, Env ) = {r | rR (Ag, Env ) (r ) = 1}

• We then say that an agent Ag in task environment Env, is successful if

R (Ag, Env ) = R (Ag, Env )

16

Task Environments (2)

• Let P (r | Ag, Env ) denote the probability that run r occurs when agent Ag is placed in environment Env.

• The probability P (|Ag,Env ) that is satisfied by Ag in Env would be:

P (|Ag,Env) = P (r |Ag,Env )

17

The Probability of Success

rR (Ag,Env )

18

Achievement & Maintenance Tasks (1)• Most common tasks in agent-based computing are

– Achievement tasks: “achieve state of affairs ”– Maintenance tasks: “maintain state of affairs ”

19

Achievement & Maintenance Tasks (2)• An achievement task is specified by a set G of

“good” or “goal” states G E :– An agent succeeds if it is guaranteed to bring about at

least one of these states – We do not care which one – they are all considered

equally good• A maintenance goal is specified by a set B of “bad”

states B E :– An agent succeeds in a particular environment if it

manages to avoid all states in B, i.e., it never acts so that a state in B occurs.

20

Agent Synthesis• Agent synthesis is automatic programming:

– The goal is to have a program that will take a task environment and

– Automatically generate an agent that succeeds in this environment:

syn : TE (Ag {})

(Think of as being like null in Java)• Common agent synthesis mechanism: genetic

programming– E.g., in the evolution of simple predator and prey agents.

• A synthesis algorithm is sound – If it returns an agent, this agent succeeds in the task

environment passed as input– I.e., the algorithm satisfies the condition:

[syn (Env, ) = Ag] [R (Ag, Env ) = R (Ag, Env )]

• A synthesis algorithm is complete – if it is guaranteed to return an agent whenever there

exists an agent that will succeed in the task environment given as input;

– I.e., the algorithm satisfies the condition:

Ag AG .[R (Ag, Env ) = R (Ag, Env )] [syn (Env, ) ]

21

Agent Synthesis (2)

Multi-Agent Systems (CS5562) – Wamberto Vasconcelos 22

Suggested Reading• An Introduction to Multi-Agent Systems, M.

Wooldridge, John Wiley & Sons, 2002. Chapter 2.