A Technique for improving the scheduling of network communicating processes in MOSIX

Rengakrishnan SubramanianMasters Report, Final Defense

Guidance by Prof. Dan Andresen

Agenda

n MOSIXn Network communicating processesn Breaking it downn Timingn Implementationn Testn Results n Conclusion

MOSIX - purpose

n Software tool for cluster computing

n Multiple servers work as if single to achieve high performance

n Automatic work distribution

n Load balancing

n Adaptive management (processes v/sresources)

MOSIX – tools

n Create a process (or more processes)

n Distribute (and redistribute)

n Algorithms respond to variation

n Works on Linux x86 platforms

n Kernel patch

n System Admin tools

Agenda

n MOSIX

n Network communicating processesn Breaking it downn Timingn Implementationn Testn Results n Conclusion

Network Processes

n Preemptive process migrationn Interaction with environment after

migrationn User context, System / UHN contextn Remote migrates, deputy stays at

UHNn Deputy has kernel resources (sockets

too!)

Network Processes - origin

Cluster of nodes

Node aProcess A & B

Node b

Node aProcess A

Node bProcess BCluster of

nodes

Network Process - migrated

Node b

Node aProcess AProcess B

B to A

B to A

Cluster of nodes

Network Process - migrated

Node bProcess B

Node cNode a

Process A

Node bNode c

Process B

Node aProcess A

Agenda

n MOSIXn Network communicating processes

n Breaking it downn Timingn Implementationn Testn Results n Conclusion

Breaking down – Microscopic User Space

Kernel / Socket Layer

TCP

Firewall

Lower Layers

MOSIX

User Space

Kernel / Socket Layer

TCP

Firewall

Lower Layers

MOSIX

User Space

Kernel / Socket Layer

TCP

Firewall

Lower Layers

MOSIX

Agenda

n MOSIXn Network communicating processesn Breaking it down

n Timingn Implementation n Testn Results n Conclusion

Timing - picture

T1

User Space

Kernel / Socket Layer

TCP/ IP

Firewall

Lower Layers

MOSIX

T2

T4

T5

T6

T7T3

T8

T911.94 µsec

8.59 µsec

24.22 µsec

2.9 µsec

Timing - equation

n The amount of time that can be saved if the

packets are redirected at the firewall layer =

(Time taken at the TCP layer, 24.22 µsec)

+ (Time taken at the socket layer, 2.9 µsec)

+ (Time taken by MOSIX to decide on the

fate of the packet, M µsec).

Timing – equation 2

n Recalculated Time = 2 X { (Time taken at the TCP layer, 24.22 µsec) + (Time taken at the socket layer, 2.9 µsec)} + (Time taken by MOSIX to decide on the fate of the packet, M µsec).

Timing - Inference

n Yes, time can be saved

n Yes, time can be saved at the firewall layer

n The packet can be identified

n The packet can be redirected

n Because, the firewall can do NAT

Agenda

n MOSIXn Network communicating processesn Breaking it downn Timing

n Implementation n Testn Results n Conclusion

Implementation - IPTables

n Built-in firewall tool in Linux kernel

n Successor of IPChains and IPFwadm

n MOSIX works on Linux x86

n Timing available for IPTables

Implementation - NAT

PRE-ROUTING

DNAT

POST-ROUTING

SNAT

[Routing Decision]

Local Process

IN

OUT

Implementation - picture

Node bFirewall

Node cServer

Node aClient

Implementation - rules

n # iptables -t nat -A PREROUTING -s $CLIENT -d $FIREWALL _SYSTEM -p tcp –dport $SERVER_PORT -i eth0 -j DNAT --to-destination $NEW_DESTINATION

n # iptables -t nat -A POSTROUTING -s $CLIENT -p tcp --dport $SERVER_PORT -o eth0 -j SNAT --to-source

$FIREWALL_SYSTEM

n # iptables -t nat -A PREROUTING -s $SERVER -d $FIREWALL_SYSTEM -p tcp --sport $SERVER_PORT -i eth0 -j DNAT --to-destination $CLIENT

Agenda

n MOSIXn Network communicating processesn Breaking it downn Timing

n Implementation

n Testn Results n Conclusion

Testing – what ?

n MOSIX communication through

redirection

n IPTables communication through

redirection

n Direct communication

Testing - Environment

n Pentium P4 CPUn 1.6 GHz Processor speedn Intel Ether express Network cardn 100 Mbps LANn Two Red Hat 7.2 Linux boxes with Kernel

2.4.19 n One Debian Linux box with Kernel 2.4.18n All nodes were connected on to the same

LAN switch

Testing – how?

n Server-Client communicating pairn Parameterizedn Buffer sizen Amount of data à time n Number of such communicating pairsn Port numbers n Prints out time taken for data transfer

Testing – MOSIX 1

Node bServer is created here. Server waits on a port number.

Node cNode aClient will reside hereClient is not yet created.

Testing – MOSIX 2

Node bServer is migrated manually using MOSIX admin tools to node cAll processes think server is still in Node b

Node cServer is now here.But, it goes to Node b for system calls. Node b is its UHN

Node aClient is created. Contacts server in Node b.Is unaware that server is in Node c

Testing - IPTables

Node bIPTables rules are written here.Forwards packets from Node a to Node c and vice-versa

Node aClient is created hereClient contacts Node b requesting for service

Node cServer is created here.It will get request from Node b (which is in reality from Node a). It will reply back to Node b

Testing - Direct

Node cServer is created here.It will get request from Node aIt will reply back to Node a

Node b

Node aClient is created hereClient contacts Node c requesting for service

Agenda

n MOSIXn Network communicating processesn Breaking it downn Timing

n Implementationn Test

n Resultsn Conclusion

Results – Execution time

Results - Bandwidth

Results – CPU Utilization

Results – Load Average

Results – Execution Graph

Execution Time Comparison

0

100

200

300

400

500

600

700

0 2 4 6 8 10 12 14 16

No of Connections

Tim

e (s

ecs)

mosix iptables direct

Results – Bandwidth Graph

Bandwidth Comparison

0

5

10

15

20

25

30

35

0 2 4 6 8 10 12 14 16

No of connections

Ban

dw

idth

(Mbps)

mosix iptables direct

Results – CPU Graph

%CPU Utlization Comparison

0

20

40

60

80

100

0 10 20 30 40 50 60

No of connections

%C

PU

Uti

lizat

ion

mosix iptables

Results – Load Graph

Load Average Comparison

0

1

2

3

4

5

0 10 20 30 40 50 60

No of Connections

Load

Ave

rage

mosix iptables

Agenda

n MOSIXn Network communicating processesn Breaking it downn Timing

n Implementationn Testn Results

n Conclusion

Conclusion - 1

n MOSIX takes 33% more time on

execution than direct communication

n IPTables takes only 4% more

n MOSIX takes 28% more time on

execution than IPTables

Conclusion - 2

n Bandwidth of MOSIX is 20% less than

IPTables

n MOSIX, on an average, takes 212% more CPU utilization

n MOSIX takes at least 138 times more load average than IPTables

Conclusion – 3

n Yes, better performance can be achieved

n Better Execution time

n Better Bandwidth utilization

n Better load average

n Better CPU utilization

Questions