Virtual Router in CloudStack 4.4Sheng Yang

Virtual Router in CloudStack 4.4Sheng Yang

Virtual Router(VR) in CloudStack

VPC VRPublic network

Web Tier: 10.1.0.1/24

App Tier: 10.1.1.1/24

DB Tier: 10.1.2.1/24

Web 1 Web 3Web 2

App 1 App 2

DB Master DB Slave

10.1.0.1/16

Site-to-site VPN

Remote Access VPN

Public Load balancing

VLAN 1001

VLAN 1000

VLAN 1002

Remote NetworkRouter

VR code flow – VirtualRouterElement

– Interface for NetworkOrchestrator– VirtualNetworkApplianceManager

– applyRules()– finalizeVirtualMachineProfile()– finalizeCommandsOnStart()

– In VR– /etc/init.d/cloud-early-config– /opt/cloud/bin/

Before 4.4 – Different hypervisor resource would implement different

way to execute commands– Because VR is not a “host” in CloudStack– Commands can be executed in different environment

– On the host, or in the VR– Commands can only be executed one by one

– It would take significant time to program large numbers of commands

New in 4.4 – Introduce the new VirtualRoutingResource

– Handle all the commands for VR– One instance for one hypervisor resource– All execution would be done inside VR

– A new VirtualRouterDeployer interface for hypervisor which need to programming VR

– PrepareCommand()– ExecuteInVR()– CopyFileToVR()– CleanupCommand()

New in 4.4 (cont.) – Introducing aggregation mechanism to accelerate

commands execution in VR– Involved when VR booting/recreating– Delay and merge all the commands execution at

VirtualRoutingResource level– One SSH for all initialization commands

AggregationControlCommand– Action

– Start, Finish, Cleanup– Delay execution initialization commands to a queue

after start– Return success immediately for commands during

the aggregated period– Merge and execute the commands when finish

– By generating and applying a single configuration file to VR

– Clean up the queue at last– In case things goes wrong in merge commands

AggregationControlCommand (cont.)– Only support commands return true or false

– No query command– Only used when boot up or recreate VR– One failure fails all the following commands

– No rollback available– Detail logs in /var/log/cloud.log

– Rotated

Performance measurement– Create a unit test for performance testing

– Implement VirtualRouterDeployer interface– Using VirtualRoutingResource– Test against pre-created VR– Execute a large numbers of commands

– Simulate different command execution flow for hypervisors

– Direct access to VR for VMware– One hop on host for XenServer

Performance result

1000 DHCP and VMData

1000 Port Forwarding Rules

0 200 400 600 800 1000 1200 1400

Aggregated

VMware Original

Xen Original

Time consumed in seconds, smaller is better

More in the plan– Further optimize VR configuration

– Current implementation is simply execute the command queue in the VR

– In the future the configuration file can be generated in mgmt server then replace in VR, which make VR update an almost O(1) action

– Better way to update software on VR– Debian repository may would be the choice

Thanks!Thanks!