Gradual Relaxation Techniques with

Applications to Behavioral Synthesis

Gradual Relaxation Techniques with

Applications to Behavioral Synthesis

Zhiru Zhang, Zhiru Zhang, Yiping FanYiping Fan, , Miodrag Potkonjak, Jason Cong Miodrag Potkonjak, Jason Cong

Department of Computer Science

University of California, Los Angeles

Partially supported by NSF under reward CCR-0096383

OutlineOutline

Motivations & objectivesMotivations & objectives

Gradual relaxation techniquesGradual relaxation techniques

– Driver example: Driver example: Time-Constrained SchedulingTime-Constrained Scheduling

Other driver examplesOther driver examples

– Maximum Independent Set (MIS)Maximum Independent Set (MIS)

– Soft Real-Time SchedulingSoft Real-Time Scheduling

Experimental results Experimental results

ConclusionsConclusions

Motivations & ObjectivesMotivations & Objectives

Motivations:Motivations:– Many synthesis problems are Many synthesis problems are

computational intractablecomputational intractable SAT, scheduling, graph coloring, …SAT, scheduling, graph coloring, …

– Lack of systematical way to develop Lack of systematical way to develop effective heuristicseffective heuristics

Objectives:Objectives:– Development of a new general heuristic Development of a new general heuristic

paradigmparadigm Gradual RelaxationGradual Relaxation

– Applications to a wide range of synthesis Applications to a wide range of synthesis problemsproblems

Gradual Relaxation TechniquesGradual Relaxation Techniques

Most constrained principleMost constrained principle Minimal freedom reduction Minimal freedom reduction Negative thinkingNegative thinking Compounding variables Compounding variables Simultaneous step considerationSimultaneous step consideration CalibrationCalibration Probabilistic modeling Probabilistic modeling

Driver Example: Time-Constrained Scheduling (1)Driver Example: Time-Constrained Scheduling (1)

Problem: Time-constrained schedulingProblem: Time-constrained scheduling– Given: Given:

(1) A CDFG (1) A CDFG G(V, E)G(V, E) (2) A time constraint (2) A time constraint TT

– Objective:Objective: Schedule the operations of V into T cycles Schedule the operations of V into T cycles

so that the resource usage is minimized so that the resource usage is minimized and all precedence constraints in and all precedence constraints in GG are are satisfiedsatisfied

Driver Example: Time-Constrained Scheduling (2)Driver Example: Time-Constrained Scheduling (2) Related workRelated work

– M. C. McFarland, A. C. Parker, and R. M. C. McFarland, A. C. Parker, and R. Camposano, Proc. of IEEE, 1990Camposano, Proc. of IEEE, 1990

– G. D. Micheli, 1994G. D. Micheli, 1994– E. A. Lee and D. G. Messerschmitt, Proc. of E. A. Lee and D. G. Messerschmitt, Proc. of

IEEE, 1987, SDF schedulingIEEE, 1987, SDF scheduling Classical approach – Force-Directed Classical approach – Force-Directed

SchedulingScheduling– P. G. Paulin and J. P. Knight, DAC 1987P. G. Paulin and J. P. Knight, DAC 1987– Exploit schedule freedom (slack) to Exploit schedule freedom (slack) to

minimize the hardware resourcesminimize the hardware resources– Iterative approach: schedule one Iterative approach: schedule one

operation per iterationoperation per iteration

Driver Example: Time-Constrained Scheduling (3)Driver Example: Time-Constrained Scheduling (3) Determine ASAP & ALAP

Schedules

Determine Time Frame of each operation– Length of box : Possible

execution cycles– Width of box: Probability of

assignment– Uniform distribution, Area

assigned = 1

Create Distribution Graphs– Sum of probabilities of

each Op type– Indicates concurrency of

similar OpsDG(i) = Prob(Op, i) DG for Multiply

0 1 2 3 4

1

2

3

4

DG for Add, Sub, Comp

0 1 2 3 4

1

2

3

4

C-step 1

C-step 2

C-step 3

C-step 4

Time Frames

1/2

1/3

*

+ <

+*

*

-

*

*

-

*

* * * *

**

ASAP

+ <

+

-

-

ALAP

* *

*

*

*

*

+ <

+-

-

Principle:Principle:– First resolve the most constrained First resolve the most constrained

components components – Minimally impact the difficulty of still Minimally impact the difficulty of still

unresolved constraintsunresolved constraints Related work:Related work:

– General techniqueGeneral technique Bitner and Reigold, 1975; Brelaz, 1979, for graph Bitner and Reigold, 1975; Brelaz, 1979, for graph

coloringcoloring Pearl, 1984, intelligent searchPearl, 1984, intelligent search

– Slack based heuristicsSlack based heuristics Davis, Tindell, and Burns, 1993; Gldwasser, 2003Davis, Tindell, and Burns, 1993; Gldwasser, 2003

– Force-directed schedulingForce-directed scheduling Paulin and Knight, 1989Paulin and Knight, 1989

Most Constrained PrincipleMost Constrained Principle

Most Constrained Principle: Time-Constrained SchedulingMost Constrained Principle: Time-Constrained Scheduling Operation Operation Op,Op, at control step at control step ii, targeting control step , targeting control step tt

– Force(Op, i, t) = Force(Op, i, t) = DG(i) * x(Op, i, t)DG(i) * x(Op, i, t)– x(Op, i, t): the Prob change in i when Op is scheduled to tx(Op, i, t): the Prob change in i when Op is scheduled to t

The The self forceself force of operation of operation OpOp w.r.t control step w.r.t control step tt – Self Force(Op, t) = Self Force(Op, t) = iitime frametime frame Force(Op, i, t) Force(Op, i, t)

0 1 2 3 4

1

2

3

4

1/3

C-step 1

C-step 2

C-step 3

C-step 41/2

d* h

+i<

e+

c*

a*

j-

b*

f*

k-

g*

c*

0 1 2 3 4

1

2

3

4

C-step 1

C-step 2

C-step 3

C-step 4

1/3

d* h

+i<

e+

a*

j-

b*

f*

k-

g*

c*d*

Minimal Freedom Reduction / Negative Thinking (1)Minimal Freedom Reduction / Negative Thinking (1) Minimal Freedom Reduction Minimal Freedom Reduction – – key of a key of a

good heuristic: good heuristic: – To avoid the greedy behavior of optimizationTo avoid the greedy behavior of optimization– Make a small gradual atomic decision Make a small gradual atomic decision – Evaluate its individual impact before Evaluate its individual impact before

committing to large decisionscommitting to large decisions

Negative Thinking Negative Thinking – – way to realize way to realize Minimal Minimal Freedom ReductionFreedom Reduction– Traditional heuristics resolve a specific Traditional heuristics resolve a specific

component of the solutioncomponent of the solution

– Negative thinkingNegative thinking determines what will not be determines what will not be considered as a component of the solutionconsidered as a component of the solution

Minimal Freedom Reduction / Negative Thinking (2)Minimal Freedom Reduction / Negative Thinking (2)

Similar ideas:Similar ideas:– Improved Force-Directed scheduling:Improved Force-Directed scheduling:

W. F. J. Verhaegh, P. E. R. Lippens, E. H. L. Aarts, W. F. J. Verhaegh, P. E. R. Lippens, E. H. L. Aarts, J. H. M. Korst, J. L. van Meerbergen, and A. van J. H. M. Korst, J. L. van Meerbergen, and A. van der Werf, der Werf, IEEE Trans. on Computer-Aided Design IEEE Trans. on Computer-Aided Design of Integrated Circuits and Systems,of Integrated Circuits and Systems, 1995 1995

Gradually shrink operations’ time famesGradually shrink operations’ time fames

– Standard cell global routing: Standard cell global routing: J. Cong and Patrick H. Madden, ISPD, 1997J. Cong and Patrick H. Madden, ISPD, 1997 Iterative deletion method – from the complete Iterative deletion method – from the complete

routing graph, delete edges one by one to get an routing graph, delete edges one by one to get an optimum routing treeoptimum routing tree

Negative Thinking:Time-Constrained SchedulingNegative Thinking:Time-Constrained Scheduling

Traditional FDS:Traditional FDS:– Select Select minimum forceminimum force (Op, t)(Op, t), , scheduleschedule OpOp to to tt

Negative thinking FDS:Negative thinking FDS:– Select Select maximum forcemaximum force (Op, t)(Op, t), , removeremove tt from from Op’s Op’s time time

frameframe

DG for Multiply

0 1 2 3 4

1

2

3

4

DG for Add, Sub, Comp

0 1 2 3 4

1

2

3

4

C-step 1

C-step 2

C-step 3

C-step 4

Time Frames

1/2

1/3

d* h

+i<

e+

c*

a*

j-

b*

f*

k-

g*

d*

DG for Multiply

0 1 2 3 4

1

2

3

4

DG for Add, Sub, Comp

0 1 2 3 4

1

2

3

4

Time Frames

d* h

+

C-step 1

C-step 2

C-step 3

C-step 41/2

1/3

i<

e+

c*

a*

j-

b*

f*

k-

g*

Compounding Variables /Simultaneous Steps Consideration (1)

Compounding Variables /Simultaneous Steps Consideration (1) Compounding variablesCompounding variables

– For the problems where variables can be For the problems where variables can be assigned only to binary valuesassigned only to binary values

– Combine several variables togetherCombine several variables together Simultaneous steps considerationSimultaneous steps consideration

– Consider a small negative decision on a set of Consider a small negative decision on a set of variables simultaneouslyvariables simultaneously

Example: a SAT instanceExample: a SAT instance– Compound Compound xx11 and and xx22, there are 4 assignment , there are 4 assignment

optionsoptions– Evaluate their impacts to the maximum Evaluate their impacts to the maximum

constraintsconstraints– Negative thinking: remove one option, keep the Negative thinking: remove one option, keep the

other three other three promisingpromising options options

CalibrationCalibration

Heuristics conduct the optimization Heuristics conduct the optimization – Keep the options for Keep the options for importantimportant variables variables – Discard the options for Discard the options for unimportantunimportant variables variables

Example:Example:– In resource-minimization scheduling:In resource-minimization scheduling:

Multipliers are much more expensive than addersMultipliers are much more expensive than adders Preserve Preserve maximum slacks for maximum slacks for the multiplicationsthe multiplications Lower the priority to minimize required addersLower the priority to minimize required adders

C-step 1

C-step 2

C-step 3

C-step 41/2

1/3

*h+

<

+*

*

-

*

*

-

*d* h

+

C-step 1

C-step 2

C-step 3

C-step 41/2

1/3

<

+*

*

-

*

*

-

*

d*

Probabilistic ModelingProbabilistic Modeling

Options of every variable are non-uniformly Options of every variable are non-uniformly distributed distributed

Probabilistic modelingProbabilistic modeling– A non-uniform function of all constraints A non-uniform function of all constraints

imposed on a particular variableimposed on a particular variable

• probprob(1,1) = 0.6 (1,1) = 0.6 • probprob(1,2) = 0.3 (1,2) = 0.3 • probprob(1,3) = 0.1(1,3) = 0.1

3

1

2

c-step1

c-step2

c-step3

c-step4

c-step5

3

1

2

3

1

2

3

1

2

1

2

3

1

2

3

1

2

3

3

1

2

3

1

2

1

2

3

C-step 1

C-step 2

C-step 3

C-step 4

C-step 4

32

1

When is Gradual Relaxation Most Effective?When is Gradual Relaxation Most Effective?

Minimal freedom reduction / Negative thinkingMinimal freedom reduction / Negative thinking – A large number of variables have significant A large number of variables have significant

slack slack – Variables have complex interactions among a Variables have complex interactions among a

large number of constraintslarge number of constraints Compounding variables / simultaneous stepsCompounding variables / simultaneous steps

considerationconsideration– Each variable has a small set of potential Each variable has a small set of potential

valuesvalues CalibrationCalibration

– The final solution only involves relatively few The final solution only involves relatively few types of resourcestypes of resources

Probabilistic modelingProbabilistic modeling– Effective for large and complex instancesEffective for large and complex instances

Driver Example: Maximum Independent Set (1)Driver Example: Maximum Independent Set (1) Problem: Maximum Independent SetProblem: Maximum Independent Set

– Given: Given: G (V, E)G (V, E)– Objective: find a maximum-size Objective: find a maximum-size

independent set independent set V’V’ V, V, such that for such that for uu V’V’ and and vv V’V’, , (u, v) (u, v) E. E.

Related workRelated work– A popular generic NP-Complete problemA popular generic NP-Complete problem

M. R. Garey and D. S. Johnson, 1979M. R. Garey and D. S. Johnson, 1979– Useful for efficient graph coloringUseful for efficient graph coloring

D. Kirovski and M. Potkonjak, DAC 1998D. Kirovski and M. Potkonjak, DAC 1998

Driver Example: Maximum Independent Set (2)Driver Example: Maximum Independent Set (2)

Reasoning: Reasoning: – In practice, MIS size is much smaller than the total graph In practice, MIS size is much smaller than the total graph

sizesize A smaller decision:A smaller decision:

– To select a most constrained vertex not to be in the MISTo select a most constrained vertex not to be in the MIS– Simple heuristic: Simple heuristic:

hh11(v) = Number of Neighbors of v(v) = Number of Neighbors of v

– Look-forward heuristic: Look-forward heuristic: hh22 (v) = (v) = uuNeighbors (v) Neighbors (v) (1 / Number of Neighbors of u)(1 / Number of Neighbors of u)

Driver Example: Soft Real-Time Scheduling (1)Driver Example: Soft Real-Time Scheduling (1) Problem: Soft real-time schedulingProblem: Soft real-time scheduling

– Given: Given: (1) (1) A set of non-preemptive tasks A set of non-preemptive tasks ={={1 1 ,,2 2 ,,……nn} } and each taskand each task ii=(a=(ai i ,, d di i , , eeii) ) is characterized by an arrival timeis characterized by an arrival time a aii, , a a deadlinedeadline d dii and an execution time and an execution time eeii

(2)(2) A single processorA single processor P P (3)(3) A timing constraintA timing constraint T T

– Objective:Objective: Schedule a subset of tasks in Schedule a subset of tasks in on on

processor processor PP within the available time within the available time TT so so that the number of tasks scheduled is that the number of tasks scheduled is maximizedmaximized

Driver Example: Soft Real-Time Scheduling (2)Driver Example: Soft Real-Time Scheduling (2) Multimedia applicationsMultimedia applications

– B. Kao and H. Garcia-Molina, 1994B. Kao and H. Garcia-Molina, 1994– B. Adelberg, H. Garcia-Molina, and B. Kao, 1994;B. Adelberg, H. Garcia-Molina, and B. Kao, 1994;

Video and WWW serversVideo and WWW servers– M. Jones, D. Rosu, M.-C Rosu, 1997M. Jones, D. Rosu, M.-C Rosu, 1997

Formal definitionFormal definition– P. D’Argenio, J.-P Katoen, and E. Brinksma, 1999P. D’Argenio, J.-P Katoen, and E. Brinksma, 1999

CAD and embedded systemsCAD and embedded systems D. Ziegenbein, J. Uerpmann, and R. Ernst, ICCAD D. Ziegenbein, J. Uerpmann, and R. Ernst, ICCAD

20002000 D. Verkest, P. Yang, C. Wong, and P. Marchal, D. Verkest, P. Yang, C. Wong, and P. Marchal,

ICCAD 2001ICCAD 2001 K. Richter, D. Ziegenbein, M. Jersak, and R. Ernst, K. Richter, D. Ziegenbein, M. Jersak, and R. Ernst,

DAC 2002DAC 2002

Driver Example: Soft Real-Time Scheduling (3)Driver Example: Soft Real-Time Scheduling (3) Two phase heuristic:Two phase heuristic:

– Conflict minimizationConflict minimization Gradually shrink the time frame for every taskGradually shrink the time frame for every task

– LegalizationLegalization Probabilistic modeling:Probabilistic modeling:

– Trapezoid shape Trapezoid shape Task Probability DistributionTask Probability Distribution

si si+ei ci-ei ci

t

prob(i,t)

Driver Example: Soft Real-Time Scheduling (4)Driver Example: Soft Real-Time Scheduling (4) Objective:Objective:

– Minimize the Minimize the number of number of conflictsconflicts

Repeat until all Repeat until all tasks are lockedtasks are locked– Update Update distribution distribution

graphgraph– Compute forces for Compute forces for

every tasks at the every tasks at the start start and and cutoff cutoff time slotstime slots

– Select the Select the maximummaximum forceforce (T, t)(T, t), , removeremove time time slot slot tt from task from task TT’s ’s time frametime frame

Time Slot

Task.Prob

Time Slot

Task.Prob

Experimental Results: Maximum Independent SetExperimental Results: Maximum Independent Set Apply to DIMACS benchmark graphs for the Apply to DIMACS benchmark graphs for the

CliqueClique problem challenge problem challenge

Compare to a state-of-theCompare to a state-of-the--art iterative art iterative algorithmalgorithm– MIS algorithm used in D. Kirovski and M. MIS algorithm used in D. Kirovski and M.

Potkonjak, DAC 1998Potkonjak, DAC 1998– Similar quality Similar quality – Much faster: Much faster: 50X50X using using hh11, , 30X30X using using hh22

Look-forward heuristic outperforms the simple Look-forward heuristic outperforms the simple versionversion

Experimental Results:Time-Constrained Scheduling (1)Experimental Results:Time-Constrained Scheduling (1)

Scheduling results comparison under critical-path time constraintScheduling results comparison under critical-path time constraint

Node Cycles ALU ALU* MULT MULT *

FFT 27 7 3 3 2 2

WANG 48 7 5 5 8 8

MCM 94 8 11 10 9 9

HONDA 97 7 8 9 14 14

DIR 148 9 10 10 9 8

CHEM_PLANT 347 16 15 13 13 13

AIRCRAFT _S 422 20 15 12 16 12

11_u5ml 497 17 20 17 22 17

12_u5ml 547 19 21 17 16 16

FEIG_DCT 548 69 9 7 3 2

GSM 617 41 10 9 6 6

AIRCRAFT _L 2283 35 43 36 48 35

T otal - - 170 148 166 142

Improvement - - - 13% - 14%

Experimental Results:Time-Constrained Scheduling (2)Experimental Results:Time-Constrained Scheduling (2)

Node Cycles ALU ALU* MULT MULT *

FFT 27 10 2 2 1 1

WANG 48 10 4 4 8 3

MCM 94 12 8 7 6 6

HONDA 97 10 8 6 7 7

DIR 148 13 8 7 6 6

CHEM_PLANT 347 24 12 8 8 9

AIRCRAFT _S 422 30 10 8 16 9

11_u5ml 497 25 15 11 14 12

12_u5ml 547 28 15 11 12 11

FEIG_DCT 548 103 8 5 4 2

GSM 617 60 7 6 3 3

AIRCRAFT _L 2283 52 33 23 31 26

T otal - - 130 98 116 95

Improvement - - - 25% - 18%

Scheduling results comparison under time Scheduling results comparison under time constraint with 1.5x critical path lengthconstraint with 1.5x critical path length

Experimental Results:Soft Real-Time SchedulingExperimental Results:Soft Real-Time Scheduling

T T ask No. Period EDF. No. Busy Util.

p1_ts9 20 56 8 12 48 0.8571

p2_ts9 20 56 7 9 49 0.875

p3_ts9 20 56 5 6 53 0.9464

p4_ts9 20 56 4 6 47 0.8393

p1_ts14 25 145 7 10 119 0.8207

p2_ts14 25 145 7 12 128 0.8828

p3_ts14 25 145 10 18 115 0.7931

p4_ts14 25 145 6 9 124 0.8552

Average 22.5 100.5 6.75 10.25 85.38 0.8587

Soft real-time scheduling resultsSoft real-time scheduling results

ConclusionsConclusions

Development of gradual relaxation techniquesDevelopment of gradual relaxation techniques Most constrained principleMost constrained principle Minimal freedom reductionMinimal freedom reduction Negative thinkingNegative thinking Compounding variablesCompounding variables Simultaneous step considerationSimultaneous step consideration CalibrationCalibration Probabilistic modeling Probabilistic modeling

Applications to:Applications to:– Maximum independent setMaximum independent set– Time-constrained schedulingTime-constrained scheduling– Soft real-time schedulingSoft real-time scheduling