Soka University

Distributed Network Flow Optimization Algorithm with Tie-set Control based on Coloring for SDN

Genya IshigakiNorihiko Shinomiya Graduate School of Engineering,Soka University, Japan

Soka University

Introduction

2

Traffic Balancing and Network Flow Problems• Importance of Network stability • Efficiency in use of network equipments • Flow problem considering use rate of each link (Gallager, 1977)

Software Defined Networking (SDN) with multiple controllers• Separation of control plane (routing logic) from data plane

(forwarding function) • Manipulation of traffic in different network layers as “flow entity” • Management by distributed controllers for scalability

ControlPlane

Data Plane

Soka University

Introduction: Related Work

3

Traffic Balancing based on Equal-Cost Paths Distribution of network flows to shortest paths between same node pair

• Topological limitation to balancing of the loads • Significant complexity in obtaining paths and exchanging

control messages (J. Farkas, Z. Arató, 2009) • (With SDN) increase in shared info about global paths among

controllers • (With SDN) routing process requiring negotiation among

controllers

S t

ControlPlane

Data Plane

Soka University

Focus of this Study

4

Providing theoretical basis for traffic balancing for SDN with multiple controllers

Distributed network flow optimization algorithm• Realization of more drastic distribution of link loads • Flow balancing method without global path information

Problem 1. Balancing flow preserving flow mass balance law Deployment of loop structures called tie-set as partitioning unit

Problem 2. Consistency on flow updateUse of graph coloring on a meta-graph that represents adjacency relationships between the loops

Soka University

Fundamental Tie-setset of edges in a cycle composed of 1 co-tree edge and some tree edges

Tie-set Graphmeta-graph whose vertices are tie-sets, and edges show adjacency between tie-sets

Definitions

5

tree co-tree Tie-set L1 L2

e1

e2

e3

e5

v1 v2

v3

v4

v5

e4

e6

L3 e7

Tie-set Graph

L2L1

L3

Soka University

Model and Attributes

6

Attributes on each Edge1. direction of the edge (fixed) 2. capacity (constant): 3. amount of current flow (variable): �ck fk ck .

ck 2 R+

v3

v2

v1

e2

e1

2 (5)

- 5 (10)

flow (capacity)

Negative current flow runs against the direction of the edge.

Weighted Graph modeling Network switches: vertices cables: edges

V = {vi}E = {ek}

ek

Soka University

1. Network Flow Optimization (NFO) Problem

7

Attemptbalancing of network flows based on the capacity of each edge

Utilization of an Edge

, where is capacity and is current flow.

Objective Function (Utilization of a Network)

fkckg(fk) =fkck

MinimizeX

E

g2(fk)

subject to flow mass balance lawincoming flow < outgoing flow (at source nodes) incoming flow > outgoing flow (at sink nodes) incoming flow = outgoing flow (otherwise)

incoming outgoing

vertex

Soka University

1. Algorithm for NFO Problem

8

Utilization of a Tie-set

Optimum Point of this Tie-set Utilization Function • Finding the minimum by differentiation • Obtaining value of update flow (tie-set flow ) for each tie-set

� =X

ek2L�

g2(fk)

x�

flow (capacity)

7 ( 8 ) 7 ( 7 )

1 ( 4 )6

0

66 v1

v2

v3

e1 e2

e3

Li

6 6

0

6v1

v2

v3

3.806 ( 8 ) 3.806 ( 7 )

2.194 ( 4 )e3

e1 e2

Li

xi = �3.194

Soka University

2. Flow Update Exclusion (FUE) Problem

9

• Regulation on simultaneous optimization in order to guarantee consistency in updated flow on each edge

• Tie-sets adjacent to each other cannot be optimized at the same round.

L1L2

L3

L4

e1

e2

e4

e6

e8

e9

e3 e5e7

v1 v2

v3 v4

v5 v6

L1 L2 e3Because and shares ,simultaneous update on the flow on induces conflict on its value.

e3

Soka University

2. Algorithm for FUE Problem

10

Distributed Tie-set Coloring Control (DisTiCC)• Vertex coloring on tie-set graph for grouping tie-sets• Flow balancing conducted by every group

L1L2

L3

L4

e1

e2

e4

e6

e8

e9

e3 e5e7

v1 v2

v3 v4

v5 v6

Tie-set Graph

L2L1

L3 L4

=

Soka University

Simulation

11

GraphBarabási Albert random graph Newman Watts Strogatz random graph (with 20, 100, 200, 400, 700, 1000 vertices)

Tree used for tie-set compositionDijkstra tree from randomly chosen root vertex

Edge CapacityRandomly assigned ( )

Initial FlowRandomly assigned ( ) to edges on some paths

! Data in paper are average of 30 times executions.

50 ck 100

fk 100

Soka University

Simulation: Comparison

12

S t

6

S t

33

S t

22

2

A. Flow Balancing• Tie-set based FB • multi-path based FB

Redirection (division) of s-t flows to other edge-disjoint paths from s to t

! edge-disjoint paths do not share any edges among them.

B. Message Complexity • Tie-set based + DisTiCC • Tie-set based + TADiC (Nakayama, 2013)

Application of conventional approach to choose local leader vertices by exchanging unique values for tie-set selection

Soka University

Simulation: Comparison

13

A. Flow Balancing• Tie-set based FB • multi-path based FB

Redirection (division) of s-t flows to other edge-disjoint paths from s to t

! edge-disjoint paths do not share any edges among them.

B. Message Complexity• Tie-set based + DisTiCC • Tie-set based + TADiC (Nakayama, 2013)

Application of conventional approach to choose local leader vertices by exchanging unique values for tie-set selection

L2L1

L3

3

22 5

L2L1

L3

2 < 3

2 < 5

2 < 3 < 5

L2L1

L3

Soka University

Simulation Results: A. Flow Balancing

14

Increment (sync)

Become stand-by state

Gather info. from switches in aaLi

Distributed Coloring

round r ← 0 (sync)

Set my color ID γi

γi = r mod ∆ + 1?

Calculate tie-set flow xi

Update edge flows with xi

fk

Confirm all adjacent controllers become stand-by state

Yes

No

r

!

!

!

∂

∂xλφ(xλ)

!

!

!

≤ ϵNo

Yes

Optimization

DisTiCC

Fig. 4. Flowchart of Processing in each Controller.

finish their operation at the round, they increments r in sync.Every tie-set continues the optimization phase until it satisfiesthe termination condition discussed in IV-C.

VI. SIMULATION AND EXPERIMENTS

In this section, simulation results are analyzed in termsof three evaluation criteria: the performance of congestioncontrol, message complexity and optimization accuracy of flowbalancing.

The performance in balancing the use rates of edges arecompared between tie-set based and path based congestioncontrol. The path based congestion control redirects flowsbased on multiple paths from source node s to sink node t.This algorithm first extracts an alternative s− t path for everypair of source and sink. When flow balancing starts, it dividesand redirects flows on a current s− t path to the prepared s− tpath(s) based on load-sharing ratios. The redirection of flowscontinues until extra s− t path cannot be found. Thus, theutilization rates of edges on those paths are approximatelyequalized after the balancing. Details of this flow controlmethod appear as principles of traffic engineering for practicalnetworks in [7].

As an algorithm to resolve the FUE problem, Tie-set Au-tonomous Distributed Control (TADiC) [8] is also adoptedfor comparison with DisTiCC. TADiC is a traditional simplemethod to select vertices that are not adjacent with eachother by identifier exchanges. When exploiting TADiC, eachcontroller broadcasts its unique id to adjacent controllers, andit compares the values received from other controllers and itsown id. The controllers whose id is the largest in the valuescan perform their optimization. TADiC sufficiently preservesthe consistency property, but its message complexity tends to

13

14

15

16

17

18

19

20

21

0 5 10 15 20 25 30

Net

wor

k Fl

ow F

unct

ion

Time [ms]

Path-basedTADiC

DisTiCC

Fig. 5. Transition of Network Flow Function ΨN from 0ms to 30ms.

0

5

10

15

20

25

0 2000 4000 6000 8000 10000

Net

wor

k Fl

ow F

unct

ion

Time [ms]

Path-basedTADiC

DisTiCC

Fig. 6. Transition of Network Flow Function ΨN from 0ms to 10000ms.

be expensive due to iteration of the value exchanges. Thus,the number of messages in control plane is measured as acomparison criteria.

In addition, the accuracy of optimization defined as op-timization rate is also analyzed for DisTiCC and TADiC.Because the NFO algorithm based on tie-sets adopts theapproximate termination condition discussed in section IV-C,the accuracy of approximation needs to be examined.

In our simulation, underlying graphs with different sizes arebuilt exploiting Barabasi Albert (BA) random graph model.Note that an algorithm to generate BA graphs is modifiedso that generated graphs guarantee bi-connectivity because 1-connected graphs cannot perform redirection of flows to otherpaths. Edge capacity ck for edge ek is randomly decided be-tween 50 and 100, and random initial flow ξk (≤ 100) is givento ek on some arbitrarily selected paths. The approximationrange ε is set as 10−4. Additionally, a set of fundamental tie-sets LB is obtained from each graph based on Breadth FirstSearch Tree with a randomly chosen root vertex.

A. Edge Flow Balancing Performance

In order to compare the capability in smoothing edge loadfactors, tie-set based NFO algorithms using either DisTiCCor TADiC and path based flow balancing algorithm areconducted. Figure 5 and 6 describe the transitions of ΨNduring flow balancing in a network with 100 nodes. In thebeginning of balancing executions depicted in Figure 5, NFOwith TADiC demonstrates fast optimization, and path based

Path-based Tie-set based + TADiC

Tie-set based + DisTiCC

X

ek2E

✓fkck

◆2

Network Flow Function represents

the sum of utilization of all edges.

Path-based• Fast convergence • Low ability to scatter

flows (no more edge disjoint paths are found.)

Tie-set based• Slow convergence • Effective behavior of

flow balancing

Soka University

Simulation Results: B. Message Complexity

15

TABLE IAVERAGE NUMBER OF MESSAGES BETWEEN CONTROLLERS

# of nodes TADiC DisTiCC20 1309.0 206.9100 83541.2 1161.9200 172185.2 1398.5400 94680.4 1292.7700 95498.0 1394.51000 120153.5 1752.8

balancing finishes flow redirection in 5ms though NFO withDisTiCC gradually updates network flows on edges.

Because the path based algorithm terminates scattering ofnetwork flows when all s− t paths are assigned the dividedflows, termination speed is faster than tie-set based algorithms.However, the path based algorithm is not likely to realizesmoothing of edge use rates in a whole network because itsbalancing are equalization of load factors only among pathssharing a same source and sink. Hence, its final value ΨN ismuch larger than tie-set based algorithms.

In contrast, the NFO algorithms exploiting tie-sets as bal-ancing units demonstrate reduction of ΨN . This is becausethese algorithms focus on each edge in tie-sets instead ofglobal paths and redirects flows so that use rates of the edgesare equalized for all edges in the network.

B. Message ComplexityTable I describes the number of messages during executions

of the tie-set based NFO algorithms. Simulation results indi-cate that the NFO with DisTiCC redirects network flows withmuch smaller number of messages than NFO with TADiC onany graphs in our simulations. Because DisTiCC eliminatescommunications between controllers by exploiting comparisonof γi and r, it reduces communication complexity of the NFOalgorithm.

C. Optimization RateIn order to evaluate the accuracy of flow optimization,

optimization rate Γ is defined as

Γ =Optimized Network Flow Function Ψ∗

NInitial Network Flow Function Ψ0

N. (16)

Network flow function ΨN converges to 0 when the op-timum tie-set flow xλ is determined in all tie-sets withouterror values, so an algorithm marking smaller Γ with a samethreshold ε is said as more accurately balancing edge loads.

Figure 7 depicts optimization rate Γ in graphs of differentsize with the error bars illustrating 95% confidence interval.In graphs with less than 700 nodes, Γ of TADiC and Dis-TiCC show approximately the same values. However, DisTiCCperforms more thorough flow balancing in larger graphs with700 and 1000 nodes. This is because balancing of edge flowsis uniformly conducted over all tie-sets by DisTiCC whichdeploys color classes that cover a whole network. On the otherhand, TADiC determines a group of tie-sets to be optimizedat that round each time, so the balancing is apt to focus onsome tie-sets that hold large evaluation values.

0.4

0.45

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

20 100 200 400 700 1000

Opt

imiza

tion

Rate

Number of Nodes

TADiCDisTiCC

Fig. 7. Optimization Rate Γ = Ψ∗N / Ψ0

N .

VII. CONCLUSION AND FUTURE WORKS

This paper has proposed an distributed algorithm to smooththe edge load factors for congestion control and a methodto solve the value conflicts by parallel flow updates by somecontrollers for SDN with distributed controllers. The formaldefinition of congestion control and valid flow update underdistributed manner are introduced as Network Flow Opti-mization (NFO) problem and Flow Update Exclusion (FUE)problem respectively. Our algorithm solves the problems basedon flow updates exploiting loop structures called tie-sets andcoloring of a tie-set graph. Simulation results demonstratethat our algorithm performs more efficient flow balancingthan the path based algorithm. Additionally, the utilizationof coloring reduces the communication complexity amongdistributed controllers.

Because this paper provides an algorithm for flow optimiza-tion, implementation and evaluation under more practical SDNprotocols would be the next target.

ACKNOWLEDGMENT

The authors would like to thank Prof. L. F. Bic and Prof.M. Dillencourt at University of California, Irvine for theirinvaluable comments and suggestions for this paper.

This work was supported by JSPS KAKENHI Grant Num-ber 26330120.

REFERENCES

[1] R. Gallager, “A minimum delay routing algorithm using distributedcomputation,” IEEE Trans. Commun., vol. 25, pp. 73–85, Jan 1977.

[2] F. Paganini and E. Mallada, “A unified approach to congestion controland node-based multipath routing,” Networking, IEEE/ACM Transactionson, vol. 17, pp. 1413–1426, Oct 2009.

[3] S. Hassas Yeganeh and Y. Ganjali, “Kandoo: A framework for efficientand scalable offloading of control applications,” in Proceedings of theFirst Workshop on Hot Topics in Software Defined Networks, HotSDN’12, (New York, NY, USA), pp. 19–24, ACM, 2012.

[4] T. Koide, H. Kubo, and H. Watanabe, “A study on the tie-set graph theoryand network flow optimization problems: Research articles,” Int. J. CircuitTheory Appl., vol. 32, pp. 447–470, Nov. 2004.

[5] M. Luby, “Removing randomness in parallel computation without aprocessor penalty,” in J. Comput. System Sci., pp. 162–173, 1988.

[6] N. A. Lynch, Distributed Algorithms. San Francisco, CA, USA: MorganKaufmann Publishers Inc., 1996.

[7] E. D. Osborne and A. Simha, Traffic engineering with MPLS. Cisco Press,2002.

[8] K. Nakayama, K. Benson, L. Bic, and M. Dillencourt, “Complete au-tomation of future grid for optimal real-time distribution of renewables,”in Smart Grid Comm., 2012 IEEE Third Int. Conf. on, pp. 418–423, 2012.

Tie-set based + TADiCMessage exchange to decide which tie-sets conduct optimization at each round

Tie-set based + DisTiCC• Many messages sent in first coloring phase • only round synchronization messages are propagated at

each round

Soka University

Conclusion

16

Focus of this ResearchProviding theoretical basis for traffic balancing for SDN with multiple controllers (distributed network flow optimization algorithm)

Problem 1. Network Flow Optimization (NFO) problem • Deployment of tie-set structures called tie-set as partitioning unit • Preservation of flow mass balance law • Optimization based on differentiation of quadratic function

Problem 2. Flow Update Exclusion (FUE) problemUse of graph coloring on a tie-set graph

Simulation Results• High balancing performance of tie-set based algorithm in

contrast with path based one • Reduction of control messages by coloring scheme