Algorithmic and Domain Centralization in Distributed Constraint Optimization Problems

John P. Davin

Carnegie Mellon University

June 27, 2005

Committee:

Manuela Veloso, Co-Chair

Pragnesh Jay Modi, Co-Chair

Scott Fahlman

Stephen F. Smith

Carnegie Mellon University

School of Computer Science

In partial fulfillment of the 5th year master's degree.

Full paper: http://www.cs.cmu.edu/~jdavin/thesis/

DCOP

• DCOP - Distributed Constraint Optimization Problem

– Provides a model for many multi-agent optimization problems (scheduling, sensor nets, military planning).

– More expressive than Distributed Constraint Satisfaction.

– Computationally challenging (NP Complete).

Centralization in DCOPs

• Centralization = aggregating information about the problem in a single agent.

resulting in a larger local search space.• We define two types of centralization:

– Algorithmic – a DCOP algorithm actively centralizes parts of the problem through communication. allows a centralized search procedure.

– Domain – the DCOP definition inherently has parts of the problem already centralized. eg., multiple variables per agent (ex. scheduling)

Motivation

• Current state of DCOP research: – Two existing DCOP algorithms – Adopt + OptAPO –

exhibit differing levels of centralization.– DCOP has been applied to several domains (eg,

meeting scheduling).– Only 1 variable per agent problems used in existing

work.• Still Needed:

– It is unclear exactly how Adopt + OptAPO are affected by centralization.

– Domains with multiple variables per agent have not been explored

Thesis Statement

• Questions: – How does algorithmic centralization affect

performance?– How can we take advantage of domain

centralization?

Spectrum of centralization

Outline

• Introduction

• Part 1: Algorithmic centralization in DCOPs

• Part 2: Domain centralization in DCOPs

• Conclusions

Part 1: Algorithmic Centralization in DCOP Algorithms

Evaluation Metrics

• How does algorithmic centralization affect performance?

• How do we measure performance?

Evaluation Metrics: Cycles

• Cycle = one unit of algorithm progress in which all agents process incoming messages, perform computation, and send outgoing messages.– Independent of machine speed, network

conditions, etc. – Used in prior work [Yokoo et al., Mailler et al.]

Evaluation Metrics: Constraint Checks

• Constraint check = the act of evaluating a constraint between N variables.– Provides a measure of computation.

• Concurrent constraint checks (CCC) = maximum constraint checks from the agents during a cycle.

Problems with previous measures

• Cycles do not measure computational time (they don’t reflect the length of a cycle).

• Constraint checks alone do not measure communication overhead.

We need a metric that combines both of the previously used metrics.

CBR: Cycle-Based Runtime

The length of a cycle is determined by communication and computation:

m

k

kcyclefortimecyclesmofruntimetotal0

)),(max(0

tkxccLcyclesmofruntimetotal i

m

kVxi

1.

tkxccLkcyclefortime iVxi

),(max

(t = time for one constraint check.)

2.

3.

),(max)(0

kxccmccc i

m

kVxi

Define ccc(m) as the total constraint checks:4.

tmcccmLmCBR )()(Cycle-Based Runtime: 5.

CBR parameters

tmcccmLmCBR )()(

•L =•L = communication latency (time to communicate in each cycle).

can be parameterized based on the system environment. Eg., L=1, 10, 100, 1000.

•We assume t=1, since constraint checks are usually faster than communication time.

L is defined in terms of t (L=10 indicates communication is 10 times slower than a constraint check).

Comparing Adopt and OptAPO

• Algorithmic centralization:

– Adopt is non-centralized. – OptAPO is partially centralized.

Prior work [Mailler & Lesser] has shown that OptAPO completes in fewer cycles than Adopt for graph coloring problems at density 2n and 3n.

But how do they compare when we use CBR to measure both communication time and computation?

DCOP Algorithms: Adopt

• ADOPT, [Jay Modi, et al.]– Variables are ordered in a priority tree

(or chain).

– Agents pass their current value down the tree to neighboring agents using VALUE messages.

– Agents send COST messages up to their parents. Cost messages inform the parent of the lower and upper bounds at the subtree.

– These costs are dependent on the values of the agent’s ancestor variables.

x1

x2

x4x3

VALUE messages

COST messages

Credit: Jay Modi

DCOP Algorithms: OptAPO

• cooperative mediation – an agent is dynamically appointed as mediator, and collects constraints for a subset of the problem.

• OptAPO agents attempt to minimize the number of constraints that are centralized.

• The mediator solves its subproblem using Branch & Bound centralized search [Freuder and Wallace].

x1

x5

x4x3

Values + constraints

OptAPO – Optimal Asynchronous Partial Overlay. [Mailler and Lesser]

x2

mediator

Results [Davin+Modi, ’05]

• Tested on graph coloring problems, |D|=3 (3-coloring).

• # Variables = 8, 12, 16, 20, with link density = 2n or 3n.

• 50 randomly generated problems for each size.

Cycles: CCC:

OptAPO takes fewer cycles, but more constraint checks.

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

8 12 16 20

Variables

CC

C

Adopt

OptAPO

0

1000

2000

3000

4000

5000

8 12 16 20

Variables

Cyc

les

Adopt

OptAPO

Density 2tmcccmLmCBR )()(

L=1

0

20000

40000

60000

80000

100000

8 12 16 20

# vars

Adopt

OptAPO

L=10

0

20000

40000

60000

80000

100000

8 12 16 20

# vars

Adopt

OptAPO

How do Adopt and OptAPO compare using CBR?

L=100

0

20000

40000

60000

80000

100000

8 12 16 20

# vars

Adopt

OptAPO

L=1000

0

50000

100000

150000

200000

250000

8 12 16 20 24

# vars

Adopt

OptAPO

L=10

0

3000000

6000000

9000000

8 12 16 20

# vars

Adopt

OptAPO

L=100

0

3000000

6000000

9000000

8 12 16 20

# vars

Adopt

OptAPO

L=1000

0

10000000

20000000

30000000

40000000

8 12 16 20 24

# vars

Adopt

OptAPO

How do Adopt and OptAPO compare using CBR?Density 3tmcccmLmCBR )()(

For L values < 1000, Adopt has a lower CBR than OptAPO. OptAPO’s high number of constraint checks outweigh its lower number of cycles.

L=1

0

3000000

6000000

9000000

8 12 16 20# vars

Adopt

OptAPO

How much centralization occurs in OptAPO?

OptAPO sometimes centralizes all of the problem structure.

How does the distribution of computation differ?

• We measure the distribution of computation during a cycle as:

This is the ratio of the maximum computing agent to the total computation during a cycle.

– A value of 1.0 indicates one agent did all the computation.

– Lower values indicate more evenly distributed load.

Agentsxi

iAgentsx

i

i

kxcc

kxcckload

),(

),(max)(

How does the distribution of computation differ?

• Load was measured during the execution of one representative graph coloring problem with 8 variables, density 2:

OptAPO has varying load, because one agent (the mediator) does all of the search within each cycle. Adopt’s load is evenly balanced.

Communication Tradeoffs of Centralization

Adopt has the lowest CBR at L=1,10,100, and crosses over between L=100 and 1000.

• OptAPO outperforms Branch & Bound at density 2 but not at density 3.

• How does centralization affect performance under a range of communication latencies?

Density=2

0

5 0 0 0 0

1 0 0 0 0 0

1 5 0 0 0 0

2 0 0 0 0 0

2 5 0 0 0 0

1 10 100 1000

L

CB

R

AdoptOptAPOCentralized

Density=3

0

1 0 0 0 0 0 0

2 0 0 0 0 0 0

3 0 0 0 0 0 0

4 0 0 0 0 0 0

5 0 0 0 0 0 0

6 0 0 0 0 0 0

7 0 0 0 0 0 0

8 0 0 0 0 0 0

9 0 0 0 0 0 0

1 10 100 1000

LC

BR

AdoptOptAPOCentralized

Part 2: Domain Centralization in DCOPs

How can we take advantage of domain centralization?

• Adopt treats variables within an agent as independent pseudo-agents. – Does not take advantage of

partially centralized domains.

x1

x2

x4x3

agent1

agent2

How can we take advantage of domain centralization?

• Instead, we could use centralized search to optimize the agent’s local variables. We call this AdoptMVA, for Multiple Variables per Agent.

– Potentially more efficient

• Adopt’s variable ordering heuristics do not apply to agent orderings

– Need agent ordering heuristics for AdoptMVA.

– Also need intra-agent heuristics for the local search.

x1

x2

x4

x3

agent1

agent2

Extending Adopt to AdoptMVA

• In Adopt, a context is a partial solution of the form {(xj,dj),(xk,dk)…}.

• We define a context where S is the set of all possible assignments to an agent’s local variables.

• We must modify Adopt’s cost function to include the cost of constraints between variables in s:

Ss

sdx textCurrentCondx sdx

jiijsdx

jiij

jj jj iiii

ddfddfs),( ),( ),(),(

),(),()(

Ss• Use Branch & Bound search to find the that minimizes the local cost.

Intra-agent cost Inter-agent cost

Results

• Setup: Randomly generated meeting scheduling problems: – Based on an 8-hour day (|D| = 8).– Number of attendees (A) per meeting is randomly

chosen from a geometric progression.– All meeting scheduling problems generated were fully

schedulable.• We compared using a lexicographic agent ordering for

both algorithms.

How does AdoptMVA perform vs Adopt?

High density meeting scheduling (4 meetings per agent):

Cycles: CBR:

[20 problems per datapoint.]

How does AdoptMVA perform vs Adopt?

Graph Coloring with 4 variables per agent, link density 2:

Cycles: CBR:

AdoptMVA uses fewer cycles than Adopt, and has a lower CBR at L=1000.

Adopt and AdoptMVA Variable ordering

Original Problem Adopt hierarchy AdoptMVA hierarchy

–AdoptMVA’s order has a reduced granularity from Adopt’s order. Adopt can interleave the variables of an agent, while AdoptMVA can only order agents.

Inter-agent ordering heuristics

• We tested several heuristics for ordering the agents: – Lexicographic– Inter-agents links – order by # of links to

other agents.– AdoptToMVA-Max – compute the Brelaz

ordering over variables, and convert it to an agent ordering using the maximum priority variable within each agent.

– AdoptToMVA-Min – AdoptToMVA-Max but using the minimum priority variables.

Comparison of agent orderings

Low Density meeting scheduling High Density meeting scheduling

•Intra-agent ordering = MVA-HigherVars

Graph Coloring with 4 variables per agent

Comparison of agent orderings

Low Density meeting scheduling High Density meeting scheduling

Ordering makes an order of magnitude difference in some cases.AdoptToMVA-Min was the best on 8 out of 11 cases, but with high variance.

•Intra-agent ordering = MVA-HigherVars

Intra-agent Branch & Bound Heuristics

• Best-first Value ordering heuristic: we put the best domain value first in the value ordering.

• Variable ordering heuristics: – Lexicographic– Random– Brelaz [Graph Coloring only] – order by number of

links to other variables within the agent.– MVA-AllVars – order by # of links to external

variables.– MVA-LowerVars – MVA-AllVars but only consider

lower priority variables.– MVA-HigherVars – MVA-AllVars but only consider

higher priority variables.

Comparison of intra-agent search heuristics•Goal: Reduce constraint checks used by Branch & Bound.•Metric: Average CCC per Cycle (TotalCCC / Total Cycles).

High density Meeting Scheduling

Cycles Avg CCC

Nearly all differences are statistically significant. Excepting MVA-AllVars vs MVA-HigherVarsMVA-HigherVars is the most computationally efficient heuristic. Low density Meeting Scheduling produced similar results.

Comparison of intra-agent search heuristics

Graph Coloring

Cycles Avg CCC

Confirms Brelaz is the most efficient heuristic for graph coloring.

How does Meeting Scheduling scale as agents and meeting size are increased?

Original Data Outliers removed

• Original data had several outliers (two standard deviations away from the mean) so they were removed for easier interpretation.

Meeting size has a large effect on solution difficulty.

(A = avg # of attendees (meeting size))

Thesis contributions

• Formalization of algorithmic vs. domain centralization.

• Empirical comparison of Adopt + OptAPO showing new results.

• CBR - a performance metric which accounts for both communication + computation in DCOP algorithms.

• AdoptMVA - a DCOP algorithm which takes advantage of domain centralization. We also contribute an efficient intra-agent search heuristic for meeting scheduling.

• Empirical analysis of the meeting scheduling problem.

• [Impact of Problem Centralization in DCO Algorithms, AAMAS ’05, Davin + Modi].

Future Work

• Improve AdoptMVA:

– Agent ordering heuristics – is there a heuristic which will work better than the ones tested?

– Intra-agent search heuristics – develop a heuristic that is both informed and randomly varied.

• Test DCOP algorithms in a fully distributed testbed.

The End

My future plans: AAMAS in July and MSN Search (Microsoft) in the Fall.

Contact: jdavin@cmu.edu