MILESTONE IV

VMWebTeam Members:

Matthew Fusaro, Brendan Heckman, Ryan Mcgivern

“Measuring programming progress by lines of code is like measuring aircraft building

progress by weight.” – Bill Gates

LETS START WITH A QUOTE (OUR CONTRIBUTION TO THE ‘W’ IN CSE 4939W)

3000 lines of code to date!

This does not include libraries or generated

code. As usual, Mr. Gates forgot to mention anything about usefulness and

quality…

Backend Implementation (Matt and Brendan) Milestone

Project Component 1 2 3 4 5

Setup a VMWare ESX server I C - - -

Setup SVN I C - - -

Prep a server to run vmweb backend and host the Web Interface I C - - -

Build stubs, create packages from VMWare provided API and import these into our development environment

I C - - -

Establish a session with the ESX box over HTTPS using SOAP I C - - -

Create objects that will mimic the important entities in vmware (such as virtual machine, host, task) these will be represented as Object Classes in java.

I C - - -

Develop functions that will query the object structures on the ESX server and return the properties we are looking for.

I P PC

-

Take the returned "Inventory" and instantiate the mimicked objects with the data. These objects will be used to enter data into the MySQL database.

I I PC

-

Develop functions that will abstract the tedious SQL statements out of the functional code I I PC

-

Create a task tracking module to update progress on requested jobs from the front-end I I P P E

Make a scheduler that will take jobs destined for the ESX server and serialize them as to not do out of order execution of requested actions.

I I P C -

Time permitting; add ability to create new virtual machines. I I I I I

Decide upon utility functions which are most relevant and achievable during the scope if this project I P P C -

Implement utilities relevant to the Host machine I P P C -

Implement utilities relevant to the Resource Pool I P P C -

Implement utilities relevant to the Virtual Machines I P P C -

Front-end Implementation (Ryan) Milestone

Project Component 1 2 3 4 5

Initialize Project Skelton I P - - -

Install Perl I C - - -Install Catalyst Web-Application Framework I C - - -

Initialize Application Directory I P C - -Install Perl Module Dependencies I C - - -

Include JavaScript framework I I P C -Implement Authentication Module I C - - -Design Authentication DB Schema I C - - -

Create Authentication DB I C - - -Create Model for Authentication DB I C - - -

Implement UI for login I P C - -Enable model in authentication controller I P C - -

Implement Inventory Module I P C - -Create Model for Inventory DB I P C - -

Implement UI for Inventory I P C - -Implement Task Module I I P C -

Implement CRUD functions for Tasks I I I I EImplement UI for Adding Tasks I P P C -

LANGUAGES IN USE

• Communication with VMware® API.

• Database communication.

• General process logic.

Java

• Stores and retrieves jobs and inventory.

• Used by back-end Java and front-end perl.

SQL

• MVC to deliver service to the browser and to the user.

• Style sheets and JS for aesthetics.

Perl, HTML, CSS,

Javascript

LIBRARIES IN USEApache AXIS

VMware vSphere Web Services SDK for Java

Catalyst Web Framework

Connector/J MySQL JDBC

MooTools - A Compact Javascript Framework

• Holds jobs• Blocks when full• Notifies when a

position is readyJob

Queue

• Checks for jobs on interval

• Places pending jobs in queue

Job Checker

• Checks job queue • FIFO Execution• Spawns workersSchedule

r

EXECUTIONPROCESS

THREAD MAP

Main Thread

Spawn Scheduler

Spawn Job Checker

Scheduler Spawns Worker Threads

UTILITY FUNCTIONS

Perform various actions on what are called Managed Objects or Managed Entities.

Managed object/entities exist on the server side and are referred to by the client via Managed Object References (MOR).

For our purposes the MOR's will refer to either a virtual machine, a host machine, a resource pool, or a task.

VIRTUAL MACHINE UTILITY FUNCTIONS

Function Name Parameters Description

powerON() ManagedObjectReference vmMOR, String name

Powers on a given virtual machine

powerOff() ManagedObjectReference vmMOR, String name

Powers off a given virtual machine

restart() ManagedObjectReference vmMOR, String name

Restarts OS on a given virtual machine

reboot() ManagedObjectReference vmMOR Resets the power on a given virtual machine

suspend() ManagedObjectReference vmMOR, String name

Suspends a given virtual machine

rename() ManagedObjectReference vmMOR, String name

Allows for renaming of a virtual machine

delete() ManagedObjectReference Deletes a given virtual machine by deleting its managed object

unRegister() ManagedObjectReference vmMOR, String name

Removes a given virtual machine from the inventory without destroying its data

createSnapshot() ManagedObjectReference vmMOR, String nam, String description, boolean mem, boolean queice

Creates a snapshot/backup of a given virtual machine

deleteSnapshot() ManagedObjectReference vmMOR, Boolean

Deletes a snapshot/backup of a given virtual machine

deleteAllSnapshots() ManagedObjectReference vmMOR, String name

Deletes all snapshots/backups for a given virtual machine

mountTools() ManagedObjectReference vmMOR Mounts the Vmware tools disc on a vitual machine drive

HOST UTILITY FUNCTIONSFunction Name Parameters Description

powerUpFromStandby() ManagedObjectReference hostMOR

Wakes the host from standbuy

powerToStandby() ManagedObjectReference hostMOR, int timeout boolean evac

Puts the host into standby mode/sleep

reboot() ManagedObjectReference hostMOR

Reboots the host

enterMaintenanceMode() ManagedObjectReference hostMOR

Puts the host into maintenance mode

RESOURCE POOL UTILITYFunction Name Parameters Description

configure() String MOR,String parentMOR, String name,long cpu_lim, long cpu_reserved, long mem_lim, long mem_reserve,boolean cpu_expandable, boolean mem_expandable

Configures the options for a resource pool/

relocate() ManagedObjectReference rpMOR

Relocates a resource pool

INTERFACE - DBIX::CLASS REVIEW

ORM – Object-Relational Mapping Creates objects associated with tables in a

relational database Database Model

Creates result class file for each table Corresponds to row-level data in table

Enable this model in the appropriate controllers Allows the flow control to query data as

necessary Layer of abstraction – remove DB dependence

INTERFACE - RESULT SETS

$c->stash(vms => [$c->model('InventoryDB::VirtualMachine')->all]);

my $host = $c->req->arguments->[0];$c->stash(host => [$c-

>model('InventoryDB::Host')->search({ host_pk => $host })]);

$c->stash(template => 'HostView.tt');

INTERFACE - RESTFUL URI

Suppose VMWeb::Controller::Inventory With function sub showHost :Local /inventory/showHost/x/y/z /inventory/showHost

Calls the relevant function x,y,z are arguments stored in context ($c)

SOME SCREENSHOTS

AUTH MODULE

DASHBOARD

INVENTORY MENU

HOST VIEW

VM VIEW

INTEGRATION PATH

Implement job contract inside worker thread.

Have scheduler's worker threads talk to utility functions

Front-end perl code pulling result sets from inventory

Front-end perl code placing jobs into job db

Front-end perl code monitoring job status

End-to-End testing (entire stack functionality)

DEPENDENCIES

I must provide data in the database so Ryan can test his result sets

Brendan must provide a job definition contract so I can parse the job from the database and have the worker thread call his functions properly.

Brendan must also send task objects back so the worker threads can monitor completion status and progress.

I must provide Ryan with a table that has currently running and recently executed jobs. I must also clear this table so the interface doesn’t have to. (A read-only set of tables dramatically reduces the chances of SQL injection)

RISKS

Database concurrency. We are still working on a way to remove entities that no longer exist on the ESX server without truncating the whole table and refilling it.

Foreign Key dependencies MUST BE SPOT ON and update correctly.

A NOTE ON SCALING

Why are we making this so complicated? While we are writing this for only one ESX

host, this framework will take about 1 to 2 weeks of coding to manage a datacenter of hosts and a large amount of VMs on the order of 500-1000 depending on hardware.

Who would use this? Cloud providers, universities, and large

datacenters Role based access is a highly desired

feature.

OKAY, WE’RE DONE.

Questions or

Comments?