May 14, 2008 1

Organization Design and Dynamic Resources

Huzaifa Zafar

Computer Science Department

University of Massachusetts, Amherst

May 14, 2008 2

Organization Design The organization of a multi-agent system is the collection of roles, relationships, and authority structures which govern its behavior - [Horling & Lesser 05]

Organization Design v/s Operational Design

Long Term v/s Short term

Used to guide Data Flow Resource Allocation Coordination Pattern … etc

May 14, 2008 3

Dynamic Resources Dynamic Resources are those resources where some characteristics of the resource changes over time

Example - Network Routing Cost of communication changes as network loads change

Paths in multi-hop communication changes as links fail

Environmental interference changes over time

Example 2 - Battery Power Consumption More usage of power implies faster battery consumption

Less available power implies an agent can take up less responsibility.

May 14, 2008 4

Outline How can we make better use of resource allocation given knowledge of the Organization design?

Network Routing eCQRouting Experimental Analysis

How can we redesign/adapt our organization to the changing resource?

Problem setup Challenges we face in solving this problem Example applications

May 14, 2008 5

Motivation

Application

Layer

Application

Layer

Network

Layer

Network

Layer

Agent A Agent B

Application

Message

Application

Message

Organization Knowledge, Message priority

Effect of message loss on performance

May 14, 2008 6

Introduction Objectives:

Significant number of network exploration messages required to support multi-hop communication

In turn reduces available bandwidth for application messages

Reduce this number in order to increase application level bandwidth

Further regulate the number of exploration messages based on:

Priority of messages Relationship between rate of message loss and performance

Use application level organizational estimates of direction and priority of communication in network level routing protocols

May 14, 2008 7

Routing At each time step do:

Each destination-agent sends out an exploration message

All other agents in the network receive this exploration message and use the corresponding time delay to predict cost of sending messages to the corresponding destination

Agents develop policies for sending messages based on costs

Policy dictates next hop when multi-hop routing Cost of sending exploration messages? eCQRouting: At each time step do:

Should I as the destination-agent send a message?

How much confidence do I as a source-agent have on the policies?

May 14, 2008 8

eCQRouting:Organizational Input

Direction and priority of communication Effect of message loss on performance Minimum path-confidence Exploration-decision frequency Learning rate (α ) - For Q-Learing

May 14, 2008 9

eCQRouting Step 1 Each agent has access to a weighted graph representing direction and priority of communication between agent roles in the network

No network-level topological information Use the graph to determine if an agent is a destination-agent {Cluster-Head and Regional-Agents}.

All agents are source-agents

May 14, 2008 10

Example NetworkSensor Agent

Regional Node

Cluster Head

Data Messages

Exploration Messages

Exploration messages are sent along with Data messages, causing interference and reduction in bandwidth

May 14, 2008 11

eCQRouting Step 2.1: source-agent

Uses time delay in receiving exploration messages along with Q-Learning to determine local policies

The policy of an agent determines the next best hop to a given destination

Confidence represents how well the Q-Value reflects the current state of the network

Confidence degrades with time in the absence of exploration messages

Calculated at source: The lower the confidence of an agent, the less its

Q-Values (and in turn policies) change with updates

Time delay in receiving exploration messages Current Confidence in Q-ValueLearning rate

May 14, 2008 12

eCQRouting Step 2.2:destination-agent

Exploration Objective: Determine the cost of sending a message from a source

Every cycle: Regulate this threshold depending on the organization (later in this talk)

Confidence has dropped below a threshold

A minimum path-confidence threshold is provided as input

Source agent communicates its confidence

Source-agents use exploration messages to estimate time required to sending application messages to the destination

May 14, 2008 13

Example Network

Benefits : Lower number of exploration messages

Exploration messages are of a smaller size

Q-Table is smaller

May 14, 2008 14

eCQRouting Step 2.3: Exploration based on message priority

More frequent exploration by high priority destinations (messages to the corresponding destination have high priority)

Destination agent changes threshold depending on message priority

Q-Values of application messages to high priority destinations are more accurate, with low priority messages less accurate.

May 14, 2008 15

eCQRouting Step 2.4:Exploration based on message loss

Source agents: Determine the rate of message loss to the destination Send message loss rate to the destination

Destination agents: Explore more frequently when current paths have significant application-level performance degradation

Agents tolerate high message-loss rates if the corresponding performance degradation is low

May 14, 2008 16

eCQRouting Step 2.4 : Exploration based on message loss

May 14, 2008 17

CNASCollaborative Network for Atmospheric Sensing

Power-Aware, Agent-Based nodes Hierarchical Organization Sensor Agents collect data Cluster Heads aggregate data and guide sensor agents

Cluster Heads send aggregated data to regional agents

May 14, 2008 18

CASA - Collaborative Adaptive Sensing of the Atmosphere

Considerably higher bandwidth requirement than CNAS

4 Roles; Radars, Feature Detectors, Feature Repositories and Optimizers

Roles higher in the hierarchy communicate with higher priority

May 14, 2008 19

Experiment - Bandwidth Increase

Networks range from 4 agents to 100 agents Agents are randomly placed such that density remains constant as network size increases

1 Cluster Head for every 3 Sensor Agents; placed randomly in the network

35% additional application bandwidth in the network of size 100 when compared to OLSR

May 14, 2008 20

Experiment - Robust Performance

Network of 160 agents 4 Optimizer agents; 4 Feature-Repository/Feature-Detector agents; Rest Radar agents

More robust performance degradation with message loss

Insignificant difference between the two threshold modification algorithms

May 14, 2008 21

Conclusions Reduce network exploration messages

Selected agents explore depending on organization knowledge

Each agent explores only if the confidence in Q-Value of the path is below a threshold

Regulate path-confidence threshold Priority of messages - high priority destinations explore more often

Effect of message loss on performance - Significant effect implies more exploration to find alternative paths

May 14, 2008 22

Future Work - Problem Setup

Resource - Network Routing Given - A basic organization

Question 1: How is this organization represented?

Wireless Networks Cost of sending messages fluctuate regularly

Adhoc Networks Agents enter and leave the network dynamically

Agent Failures Agents are unable to communicate with their neighbors

Emergent Organization?

May 14, 2008 23

Challenges Effect of change in organization on the network

Message interferences Changes in costs with changes in traffic Effects of mobility of nodes

Goodness of Organization How do we determine if one organization is better than another organization?

Cost of evaluating the organization Effect of time spent evaluating on the MAS

Reorganization/Adaptation costs Time spent in developing the new organization Cost of updating all agents with the new organization

May 14, 2008 24

Experimental Analysis Reorganizing CNAS

Re-ordering the leader agent priority lists Regional nodes

RoboRescue Fire Hazards

Organizing agents based on locations of fire hazards Predicting (or detecting) environmental changes

Communication Costs Reorganizing to reduce communication costs/limitations

May 14, 2008 25

Questions and Discussion