Virtualization for developers

The benefit of a standardized virtual stack for web development and the enterprise.

LinuxFest NorthWest April 27, 2013. Presented to Vancouver Linux Users Group (VANLUG), January 21 2013.

Table of contents. Intro Virtualization Virtual Private Servers Virtualization for Devs Prehistory What it solves

An aquarium as an analogy Stupid business practices

The future Q&A

IntroTl;dr; virtualization for web development is really smart, but not always used. Here is why you use it.

Look at some of the technical aspects. The social benefits are even greater than

the technical. Some of you may already know about

virtualization Consider the greater picture as I unpack all

this for a mixed audience. So you can sell others on it. Let’s not just

preach to the converted at LFNW 2013.

About moi About 12 years web dev. The old Vancouver PHP Users Association. Organized Several Conferences with

team. I don’t work for Oracle or other vendor. Dev and Project Management work. In my spare time: Blogger, biker, train in

Brazilian Jiu-jitsu.

A little credit to:

David Turnerhttp://www.turneris.com/

Who knew how to do a lot of this already… for helping me sort out some issues.

My turn to buy beer next time…

What’s in it for me? How can I profit from this? Standardize team skills. IT policy with a high degree of similarity

between development and production environments, which means fewer problems down the road.

Fast & flexible team creation. Skills & tools refactored.

Virtualization: The absolute basics Virtualization is the means to run an operating

system within a host operating system. A form of an emulator. Emulators have always been around. Very useful for web development, although its

not what we usually think of when we are talking about virtualizations.

“Hypervisor” In computing, a hypervisor or virtual

machine manager (VMM) is a piece of computer software, firmware or hardware that creates and runs virtual machines. Source: http://en.wikipedia.org/wiki/Hypervisor

For Virtual Private Servers. For Desktop environments.

Virtual Private Servers -for web hosting solutions Something between the palace of

dedicated hosting and the sewer of shared hosting.

The performance of a dedicated server on the budget of a shared host.

Good hosting companies provide VPS service. IMHO, bad ones don’t.

VPS vs Shared host page generation performance

on an index.php page

Dreamhost.com shared host: ~2.0 seconds.VPS: Canadian Web Hosting co-located on Peer1:

0.00002 seconds. Or an improvement of 2*10-5 over shared hosting. Holy crap.

VPS HostingLooks Like this:

http://esx.starwindsoftware.com/images/content/VMware-iSCSI-SAN-diagram.gif

FYI: key vendorsAquemu for Linux

OK, so what? This is a talk for a Linux Users Group. Hypervisors run the LAMP stack. The LAMP stack runs your website. You can run a mirror of your web server locally with many advantages. The codebase is normalized across the team with version control. The whole stack can be normalized with virtualization. I realize that some people may already get this, or do it. But some don’t.

VMs on the Desktop (for Development)with it, I can do this:

Mkay… Use virtualisation as a working tool to

build websites and things like it to have performance and behavior tested to a high degree of resolution.

You are able to run the codebase on a real stack, and even possible a mirror of your production stack.

Way gooder. Even deploy to Amazon EC2.

The prehistory – what sucked. Manual install of Apache MySQL & PHP

on win & mac desktop XAMPP (http://www.apachefriends.org/) MAMP (for Mac) WAMP CYGWIN

Dinosaurs died out for a reason

Dev prehistory (rant) Scratch install: was always a chore. Still a chore for the L33t. Skill not very transferrable. Never done often enough to become

good at it. And if you are good at it, who cares?

XAMPP: cannot upgrade components, must upgrade the whole thing.

MAMP: port hassles (privileged ports, as well as other issues). Under 1024 is privileged.

A challenge to set up mail sending, https.

Skills not useful for anything else. Friends don’t let friends use CYGWIN.

Dev prehistory (rant)

But the real reason these dinosaurs will die out Because they are not examples of the

real LAMP stack. No access to improvements served up

by the vendors. Technologies offer new things that aren't

included. These kits are a bundle. You fall behind. It sucks to be you.

Because it is not the

real stack.

Because it is not the stack your code is using in

production.

Demo some stuff already! Oracle VirtualBox is FOSS. Also consider

other hypervisors. Installing Gnu-Linux ISO distros is pretty

much the same as installing ‘on the metal’. How to get the host and guest to talk to

each other like a desktop machine talks to a remote server is the freakin’ trick.

See my blog, http://superwebdeveloper.com

The kind of things VMs for web development can solve

Your website is middleware. It lives in the host stack like a fish lives in this aquarium.

Your fish cannot live very well outside this aquarium. Neither can your website live outside of the stack.

BTW, what is wrong with this picture?

You wouldn’t know if you didn’t see the fish in the aquarium.

You don’t put cichlids in with angel fish! Cichlids are aggressive fish that need brackish water. Besides, these fish cannot exist in the same Ph balance as the community environment fish (the tetras). And the plants you see would be eaten by the cichlids.

The same problem exists with code as with these fish out of an aquarium

Even common blog middleware like Drupal and WordPress push the boundaries of the default php.ini settings. To be competitive and feature driven, new middleware products require tweaks to the vanilla directives in PHP.ini, my.ini, Apache2.conf and other software. Suhosin PHP patch comes to mind as well.

Sometimes, in the business world of web development, the king has no clothes.

Sorry, I could not find an appropriate slide for that.

PHP outside of a web server is about as useful as a fish on a sidewalk.

They want the fish without the fishbowl. The whole product of the enterprise is

never held as a discrete object. You never own a product unless you can

separate it from everything else, including hosting, and can demonstrate it at will.

Stupid business practices

Stupid business practices

No comprehensive IT policies in place is too often the reality.

No effort to standardize working stack, beyond version control.

Team leaders & management express total indifference to how team members build their environments.

A total disconnect between what a dev does and a sysadmin does.

More stupid business practices Teams are made up of multiple roles, Devs,

SysAdmin, designers, JavaScripters, lead, PM, sales, management, Qa, executives, deployment, consultants, third parties, clients, interns, kitties, puppies, and sociopaths.

Different roles have different skill sets, and no effort is made to standardize skills across team.

Standardize skill sets - use of this skill set is a way to determine if a person is competent to be on the team.

More on that… The old way burns money on unproven

ideas. OR jobs that the client hasn’t paid for

just yet. Valuable SysAdmin skills, knowledge,

authority is ‘hoarded’. Devs aren’t given authority extending to

root.

It got me thinking… Standardized virtual dev environments

provides a way to easily support team members, because no two people in a chair are alike.

Hypervisors can definitely help homogenize the technology the dev team runs on.

Hypervisors can definitely support the cause of spreading Gnu-Linux and the skill set for using it.

Use hypervisors to spread Gnu-Linux like the Norwalk virus in Vancouver in January.

Using Hypervisors is an easy way to get into GNU-Linux

Lost productivity, lost opportunity

Sure, no uniform dev team practices are not a problem most of the time.

But it can cause confusion, poor communication, persistent bugs, and additional deployment issues - FRICTION.

Opportunity to use the software in different ways.

Improvements provided by vendors & community – patches!

Other wisdom

“A separation of concerns” Stefan Priebsch, thePHP.cc.

Refactoring - every coders mantra, from which Design Patterns sprung forth.

Other wisdom “A grouping of Concerns.” When you standardize skill requirements,

any team member can support another. You become TRULY AGILE, can create

teams in an ad-hoc fashion (even though ‘Agile’ was concocted up to sell things to people and create buy-in by giving team members the illusion of control).

Test web security offline functionally, with virtual cats and fishes. Create disaster scenarios.

Another reason why What if you wanted to do things that were just a

leeetle bit off of the beaten path? NGINX NoSQL MariaDB ApacheSOLR LightHTTPD You will be SOL, or something like that. Or you

need a budget. Or a lot of time. Or a really good excuse. Or beg your hosting company. Or a dev server.

But we have a dev server.

But we have a dev server.

You pay for using and maintaining the service one way or another.

Code in development is noisy in the logs. Logs that fill up the server hard drive.

Security issues also apply to dev, not just production servers.

Compare to $0 cost. This toolset is useful and available to

any size of project. A common skill set for setting up new

desktop machines for development.

But we have a dev server.

One developer can own several desktop machines, and needs to have common dev tools regardless of platform.

dd command can publish ISO. Virtual appliances are easy to share or

convert for use in other hypervisors. Share the VM with other team members over

.torrent protocol. Sharing Gnu-Linux via .torrent was why it

was made in the first place.

But we have a dev server.

Dream Big on teh internets Not every idea on the Net is going to turn a

buck. Business burns cash like rockets burn fuel. BUT teh internets haz free tools can be

obtained to develop ideas for little or zero cost.

Host any number of projects on virtual machines without incurring costs while the project is in incubation, or is dormant.

Rapid deployment.

Or quick and dirty, if you like. Make throwaway experiments. Use it for a single task. DOS utilities may be available, but working

in MS DOS is hideous. Shell is universal. Working in the real shell may suit your job. FTP without concern for bandwidth use. A job with ffmpeg, editing videos from the

cmd line, or other resources too big to manage any other way.

Do more with less Comprehensive IT

policy promotes sloth!

The Future

Bundling middleware with the stack for distribution.

What evolved in the wake of the dying dinosaurs.

Prebuilt stacks ready for deployment.

Some have already taken this really far.

Amazon EC2 deployments. Vagrant & Chef (Ruby). But I like to tinker under

the hood a little bit more. Automated scaling. Open Shift (Red Hat). Twitter.com uses Murder to deploy in seconds

what used to take hours.

http://www.opscode.com/http://vagrantup.com/

Not the most technical talk on virtualization out there. I did not intend it to be. I really wanted to discuss the social aspects.

This is your real take away. Now go make some money with it.

Not everyone understands the power in these tools.

Teams can assemble quickly. Big ideas can turn on a dime. Put the growth of this area in perspective,

see its potential for growth.

Things to get working next

Date should synchronise without having to reboot.

Employ a fake date in the system for testing year change behavior.

Refresh IP address without reboot. Serving with a fake secure cert (HTTPS). Try Vagrant, Bitnami, Turnkey Linux offerings. Try ESXI and other hypervisors. Convince a team of developers to use this

method.

Shameless plug at the end Twitter: @pbgswd Tech blog:

http://www.superwebdeveloper.com/ For relaxation and modern social skills:

http://martialartsnomad.com/ Thanks LFNW 2013 Supporting blog posts, talk is online.

Sloth says thank you.