Consistent Resource Scheduling and QoS management

Applied Research Center for Computer Networks

Moscow State University

Ruslan SmelyanskiyARCCN Director, Professor at Moscow State University

Consistent Resource Scheduling and QoS management

15.04.2023

Content

Brief ARCCN introduction and main

research direction overview

Two research problems examples:

The Consistent Resource Scheduling in DC

FDM-TCP: Flow DeMultiplexing TCP for QoS management

Huawei Shenzhen R.Smelyanskiy 2

15.04.2023

ARCCN

Research

Ideas

Education

Adoption

Cooperation

Development

Integration

Applied Research Center for Computer Networks (ARCCN) is a Russian

non-profit organization initially funded by Skolkovo intended to:

world-class competency center for Computer Networking in Russia;

collaboration between Russian and international research, scientific, educational and commercial experts and institutions;

demonstrate innovations in Networking for National Industry;

promote commercialization of a new networking technologies and services.

ARCCN is the only R&D competency center for SDN&NFV

in Russia today

Partners: Skolkovo, The Ministry of Education and Science of Russia, Rostelecom, Rostech, Sberbank, EMC, Intel, etc.


15.04.2023

Project Ecosystem

7%

20%

12%54%

7%Ph.D.

Assos. Prof

Postgraduate

Engineeres

Vendor's cer-tificated

ARCCN Team


15.04.2023

Key R&D directions:

Distributed SDN Controller and Applications for it;

Safety and Security in SDN;

QoS management in SDN;

NFV platform and VN functions;

Architecture for Open Flow-switch;

WAN operation prototyping and analysis;

Self Organizing Cloud for Carriers (Cloud Conductor);

Data Center Resource Management;

Forwarding policy validation and troubleshooting automation;

Software for integration with legacy transport on the Internet.


Consistent Resource Scheduling in DC

Huawei Shenzhen R.Smelyanskiy 615.04.2023

Scheduler determines a physical

resource for each element of a tenant

Data center resources model

Physical resource model: ,where

• Р - set of computational nodes: – vh(p) - number of CPU cores on node p– qh(p) - amount of RAM on node p

• М - set of data storages: – uh(m) – size of data storage m– type(m) - m data storage type

• К - set of network switches: – - bandwidth of the switch k

• L - set of network channels: – rh(l) – bandwidth of the channel l

),( LKMPH

( )h k


Tenant model

Tenant model:

• W - set of virtual machines: – v(w) – requested number of cores for VM w– q(w) – requested amount of RAM for VM w

• S - set of storages: – u(s) – size of storage s– type(s) – s storage type

• E - set of virtual channels:

– r(e) – bandwidth of the virtual channel e

Tenant types:• Loosely coupled:

• Tightly coupled:

),( ESWG

)]),}{,}[({ 11 ESW Kjj

Nii

)}]}{,}{{(),}{,}[({ 1'

1'

11Kjj

Nii

Kjj

Nii SSWWESW


Tenant Mapping on DC Resources

where

Mapping A is correct if the following constraints are fulfilled:

a)

b)

c)

d)

)()(),()( pqhwqpvhwvpp WwWw

: { , , { , }},A G H W P S M E K L

)()( lrherlEe

)()( kherkEe

),()( muhsumSs

)()(: mtypestypeSs m

, , ; , , , W S E G P M K L H


Replication and migration

Replication: • Duplicates storage m to m’• Channel to support consistency between m and m’:

• VM connects to m’

Migration: • Moves virtual machine or storage to another physical resource

': ii AAR

( ', , , , , , ); ; ; ',1 1 1

m l k k l m k K l L m m Mn n i i


': ii AAT

15.04.2023

Tenant mapping problem

The problem:

• For every tenant Gi from set of tenants Z = {Gi} and

given DC resources Hres define correct mapping Ai

that set A = {Ai} covers the maximum number of

tenants from Z


Residual graph

Renew Z –

Residual graph Hres:

• The following functions are recalculated: vh(p), qh(p), rh(l), τh(k), uh(m)

pWw

res wvpvhpvh )()()( ( ) ( ) ( )p

resw W

qh p qh p q w

( ) ( ) ( )

l

rese E

rh l rh l r e

( ) ( ) ( )

k

rese E

h k h k r e

( ) ( ) ( )

m

ress S

uh m uh m u s


Scheduling Round

The scheduling algorithm runs at the begging of the scheduling rounds

On a scheduling round:– Add new tenants to Z– Delete the tenants with TTL=0– Recalculate Residual graph as a new H


Scheduling algorithms

• A1: greedy and limited exhaustive search strategies– maps tenants as a whole (minimal common subgraph isomorphism algorithm)– suitable when critical resource is a data center physical network

• A2: greedy and limited exhaustive search strategies– constructs mappings element by element (bin-packing algorithm)– suitable when critical resources are computational nodes or data storages

• A3: Ant colony algorithm:– universal, but greater computational complexity


Initial data for test

Test # DC model (graph H) Tenants models (graph G)

Test 1450 computational nodes with 21000MB RAM and 16 cores

per node

450 data storages with 21000GB disk space per

storage

1350 virtual machines and 1350 storages total in 135 tenants.

Test 1 – loosely coupled tenantsTest 2 – tightly coupled tenants,

network load = 70%Test 2

Test 3500 computational nodes with 10000MB RAM and 16 cores

per node

500 data storages with 10000GB disk space per

storage

600 virtual machines and 600 storages per 100 tenants.

Test 3 – loosely coupled tenantsTest 4 – tightly coupled tenants,

network load = 70%Test 4

Fattree topology with 5 TOR switches, 10 aggregation switches, 40 edge switches

Huawei Shenzhen R.Smelyanskiy1615.04.2023

Comparison with OpenStack scheduling algorithms

Test #

Open stack CRM

FF RF

Test 1 50% 87.5% 100%

Test 2 0% 1% 100%

Test 3 100% 70% 100%

Test 4 0% 0% 100%

- FF – First Fit- RF – Random Fit- CRM – Consistent Resource Mapping


Conclusions

Open Stack algorithms are:– Ineffective for some cases of loosely coupled tenants– Not applicable for tightly coupled tenants

Consistent Resource Mapping algorithms:– Effective as for loosely as for toughly coupled tenants– Suitable for IaaS with SLAs


FDM-SDN: SDN with Flow DeМultiplexing TCP

is a new way to manage the quality of service

Hosts do affect connection quality!

• TCP congestion avoidance algorithms:– Goal: get connection with maximal bandwidth– Strategy: cut-and-try to detect the maximum currently available

amount of resources to utilize all of them

– Primary heuristic: AIMD (pessimistic)

– Recovering action: decries CWND

– Parameter: congestion window size (CWND), timeout

– Primary modes: slow start and congestion avoidance

– Triggering criteria: duplicate ACKs&timeouts, threshold

• There is no way to control routes intersections in traditional networks


FDM-SDN provides a new leverage

• Flow demultiplexing along several non-xing routes:– Goal: get connection with required bandwidth– Strategy: use cut-and-try to detect the amount of

provided resources and utilize all of them on several routes

– Primary heuristic: decries the number of routes (optimistic)

– Parameter: cumulative bandwidth of all routes– Primary mode: keep busy all routes– Triggering criteria: bandwidth deficiency– Recovering action: open new non-xing route


Flow DeMultiplexing Protocol

Standard socket API

Activity Monitor (Bandwidth Scarcity Detection)

TCP subflow (extra options)

TCP subflow

(extra options)

TCP subflow (extra options)

Application Layer

Transport Layer

Network Layer

FDM TCP (Packet Scheduling & Reordering)

Subflow Manager (Split Degree Adjustment)


Routing FDMP flows with SDN

Host A Host B

SDN Controller

Has MP_CAPABLE option?Install new FDMP connection!

SYNMP CAP

Key A

Set up new FDMP connection


Host A Host B

SDN Controller

Has MP_CAPABLE option?Complete partial FDMP connection!

SYN, ACKMP CAP

Key B

Routing FDMP flows with SDNSet up new FDMP connection


Host A Host B

SDN Controller

Has MP_JOIN option?Install new FDMP subflow for a

known connection!

SYNMP JOINToken B

Routing FDMP flows with SDNSet up new FDMP subflow


Host A Host B

SDN Controller

Subflow is not active any more!Remove the path!

Routing FDMP flows with SDN FDMP subflow manipulation

Actually, we store metadata and allow some subflows to resume.

We use flow eviction to remove this data.


Actually, we store metadata and allow some subflows to resume.

We use flow eviction to remove this data.

Host A Host B

SDN Controller

Get FDMP packet of a expired subflow!Either reroute the remembered subflow,

or force hosts to close it

RSTRST

Close violet subflow! Reschedule the packet to

red subflow!Close violet subflow!

Routing FDMP flows with SDN FDMP subflow manipulation


h2h1lower = 50 Mbps

middle = 50 Mbps

upper = 50 Mbps

Experiments

FDMP in a hammock

TCP MP TCP FDMP

49 Mbps 48 Mbps 81 Mbps

Required h1 to h2 bandwidth: 80 Mbps


upper = 50 Mbps

h2h1

h4h3

lower = 50 Mbps

middle = 50 Mbps

Adjustment to congestion

TCP MP TCP FDMP

49 Mbps 48 Mbps 81 Mbps

Required h1 to h2 bandwidth: 80 Mbps


Conclusion

1. In context of bandwidth, the party of FDMP & our routing application outperform both single-flow TCP and MP TCP without regards to ECMP

2. FDMP improves efficiency of infrastructure:It can increase utilization under the same load and allows network to process more traffic

3. Under a heavy load FDMP will support faster communication and/or transmit more data within the same period of time


Challenges we are working on

Use TCP-FDM to reduce delay and packet loss:– Which packet scheduling algorithm to use?– What routing algorithms are appropriate?

Enforce resource allocation & QoS policies:– How to avoid flow competition on congestion?– Do we need to control request frequency?– Is it efficient enough to rate-limit subflows?– How to combine TCP-FDM and Diff Serv model?

Develop FDMP as VNF within network:– How to integrate TCP-FDM routing application?– How to build an efficient TCP to TCP-FDM proxy?


15.04.2023 Huawei Shenzhen R.Smelyanskiy 35

http://arccn.ru/

+7 (495) 984 27 64

[email protected]

@ArccnNews

http://arccn.ru/