Selenium and Grinder:A powerful duo for load testing

Eric PughPrinciple, OpenSource Connections

Epugh@opensourceconnections.com

CvilleJUGMarch 21st, 2006

What we’re going to learn!

• Just enough about Selenium to get started!• Load testing with Grinder• Whether WEBrick or lighttpd is faster for serving

pages!

What is Selenium?

• From Selenium’s web site:– “Selenium is a test tool for web applications. Selenium tests run

directly in a browser, just as real users do.”

Selenium can be run two ways: “TestRunner” and “Driven”

“TestRunner” Mode

• Selenium application consisting of HTML and JavaScript is deployed along side the application to be tested.

• Tests are written in HTML tables.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

TestRunner control panel

“Driven” Mode

• Browser is under the control of another process that specifies what commands to be run

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

“Driven” Mode

• Controlling process is typically a Java, .NET, Ruby or Python application

// Javapublic void testRugbyHome() { selenium.open("http://localhost:3000/sprint"); selenium.click("submit"); selenium.assertTitle("Rugby"); selenium.click("link=Name"); selenium.assertTitle("Rugby");}

# Rubydef testRugbyHome puts selenium.open "http://localhost:3000/sprint" puts selenium.clickAndWait "submit" puts selenium.assertTitle "Rugby" puts selenium.clickAndWait "link=Name" puts selenium.assertTitle "Rugby"end

# Pythonselenium.open('http://localhost:3000/sprint')print selenium.clickAndWait('submit')print selenium.assertTitle('Rugby')print selenium.clickAndWait('submit')print selenium.assertTitle('Rugby')

“Driven” Mode

• Pros:– Better if you need conditional logic or looping– “White Box” testing. Need in depth knowledge of webapp

structure– Test writers prefer programming in a “real” language

• Cons:– Significantly more infrastructure to setup– Test writers need programming skills– Tests that require conditional logic can become brittle quickly

FYI: “Driven” will be renamed “Remote Control” in the next version of Selenium

We’ll focus on “TestRunner” mode!

Intro to Selenese

• Selenese is a simple language for specifying commands instructing Selenium what to do

• Consists of three parts: command, target, and value laid out as columns in a an HTML table:

<table> <tr> <td>command</td><td>target</td><td>value</td> </tr></table>

Selenese Commands

• Commands are either “Actions” or “Checks”

• Actions are commands that tell Selenium to do something:– Open a web page

– Click a link

– Select an option

– Type some text

• Checks validate the applications:– Title of the page

– Verify that a checkbox is checked

– Ensure a table structure contains a specific string

Selenese Element Locators

• Element Locators specify an HTML element that a command applies to.

• Multiple strategies for finding an element:

id=idSelect the element with the specified @id attribute.

name=nameSelect the first element with the specified @name attribute.

identifier=idSelect the element with the specified @id attribute. If no match is found,

select the first element whose @name attribute is id.link=textPattern

Select the link (anchor) element which contains text matching the specified pattern.

Selenese Element Locator using DOM

• Selenium supports using JavaScript to traverse the HTML Document Object Model

• DOM locators must begin with “document.”

•dom=document.forms['myForm'].myDropdown•dom=document.images[56]

Selenese Element Locator using XPath

• XPath expressions are the most powerful way of selecting elements in Selenium.

• XPath locators must begin with “//”

•xpath=//img[@alt='The image alt text']•xpath=//table[@id='table1']//tr[4]/td[2]•xpath=//div[@id='workflowMenu']//a[@id='goToPreviousStatus']/

Tip: IE’s implementation of XPath is buggy, your mileage may vary…

Common Actions: open

• open(url)

• Opens both relative and absolute URL’s.

Tip: Selenium can only open URL’s that are on the same site due to “Cross Site Scripting” security policy in browsers. We’ll discuss how to get around this later.

Common Actions: click/clickAndWait

• clickAndWait(elementLocator)

• Click on a link, button, checkbox, or radio button.

• If the click action loads a new page then use clickAndWait to tell Selenium to pause until the new page loads

Common Actions: type

• type/typeAndWait(elementLocator,value)

• Sets the value of an input field, as though you typed it in.

Common Actions: select

• select/selectAndWait(elementLocator, optionSpecifier)

• Select an option from a drop down based on the optionSpecifier

Select Option Specifiers

• Selenium supports multiple ways of specifying options of an HTML select statement.

• If multiple options match then the first one is selected:

label=labelPatternmatches options based on their labels, i.e. the visible text.•label=regexp:^[Oo]ther

value=valuePatternmatches options based on their values.•value=other

id=idmatches options based on their ids.•id=option1

index=indexmatches an option based on its index (offset from zero).•index=2

Check Commands

• Used to verify the state of the application

• Can check the value of a form field, presence of some text, the structure of the HTML, or the URL of the page.

• All checks can be used in two modes: “assert” or “verify”

• Assert failures cause the test to be aborted

• Verify failures are recorded but the test continues

Tip: A good practice is to Assert that you are on the correct page and then Verify the contents of the page.

String Matching with Check Commands

• Selenium supports various pattern syntaxes for matching string values:

glob:patternMatch a string against a "glob" pattern. "Glob" is a kind of limited

regular-expression syntax typically used in command-line shells. In a glob pattern, "*" represents any sequence of characters, and "?" represents any single character. Glob patterns match against the entire string.regexp:regexp

Match a string using a regular-expression. The full power of JavaScript regular-expressions is available.exact:string

Match a string exactly, verbatim, without any of that fancy wildcard stuff.

Common Checks: assertTitle

• assertTitle(titlePattern)

• Verify the title of the current page

Common Checks: assertValue

• assertValue(elementLocator, valuePattern)

• Verify the value of an input field (or anything with a value parameter)

• For checkbox/radio elements the value will be “on” or “off” depending on if it is checked or not

Common Checks: assertAttribute

• assertAttribute(elementLocator@attributeName, valuePattern)

• Verify the value of an element attribute.

• Great for testing CSS style sheet tags!

Common Checks: assertTextPresent

• assertTextPresent(text)

• Verify that some text shows up somewhere in the HTML.

• The inverse is of course assertTextNotPresent(text)!

There are many more commands, but that’s enough to get started!

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

Isn’t there an easier way then writing the HTML by hand?

Selenium IDE

• Firefox Plugin, requires Firefox 1.5• Available from http://www.openqa.org/selenium-ide/

• More then just a recorder, it’s also a great editor for your tests.

Tip: Quickly build the skeleton of your test by using the recorder, then go back and add your verify checks manually.

Now how do we do load testing?

The Grinder

• Java based load testing framework. Requires J2SE 1.3.

• Licensed under a BSD-style open source license.

• Support for testing HTTP based services as well as others such as SMTP, FTP, and JDBC.

• Comes with a plugin to facilitate recording HTTP test scripts in Jython.

• “The Grinder 3” is the recommended version to use

• http://grinder.sourceforge.net/

FYI: the official name is “The Grinder” not “Grinder”!

What you need to know

• You don’t need to know either Java or Python!

• Basic shell/batch scripting to get tool started– Configuring path statement and environmental variables

The Grinder Design

• Goal: Minimize system resource requirements while maximizing the number of test contexts (“virtual users”)

• Multi threaded, multi process implementation. – Each test contexts runs in it’s own thread.

– Threads can be split over many separate processes depending on the capabilities of the agent machine

• Distributed– Very simple to set up multiple agents and coordinate and monitor their

activity

– Console app can centrally manage tests and deploy them to the agents. Console starts and stops the agents.

Generating Grinder script from Selenium Test

• 1) Setup Firefox to use as proxy “localhost:8001”• 2) Start up Grinder’s TCP Proxy• 3) Replay script using Selenium IDE

• Note: Make sure to turn off caching in Firefox so you get all the requests recorded!

• Make sure in Firefox’s Proxy page localhost is set to proxy!

• Use Proxy filter to control what gets recorded!

Getting Grinder Installed

• One controller and two agents

You’ve got Questions? I’ve got Answers!

Email me at epugh@opensourceconnections!