Automating Windows Application Testing
and
Problem Resolution With Nagios

Sam Lansing

Introduction

What I do at Nagios

Member of the Technical Support team at Nagios.

I work on documentation and quality assurance, implementation of automated problem solutions into the Nagios family of software.

Example projects:

AutoIT Timer Plugin, Program Timer, Host Service Restart Event Handler, Selenium integration, and Selenium Nagios XI interface checks.

What is this automation thing?

Automation, the asset

Eliminates the hassle of terminal usage when an issue occurs

Precise and efficient testing

Less 3 o'clock AM calls

As far as the eye can see!

Possibilities and limitationsAutonomous problem solutions

Usable for simple things such as service restarting, to intricate expansive uses such as mass distribution

Main limitation is user interface interference

Advisable to cron scripts as to not over lap actions

Solutions, what works?

AutoIT and SeleniumCurrently tested and implemented with maximum capability

Large user base and support team

Easily molded to your needs

Triggered ScriptsEvent Handler driven

Simple yet effective

Easily Editable

Event Handlers, what are they?

Triggered when a Host or Service changes stateRestarting a service

Entering helpdesk tickets

Logging database information

Intermediary between Nagios and a remote hostCan be used to run a script if state is

Most efficient way to tie in AutoIT scripts otherwise the automation scripts may overlap, or continue running.

Event Handler example

Using XI Windows Monitoring WizardsA monitored service goes into a critical state and the servicerestart.sh script is triggered

Servicerestart.sh sends a command directly to the Windows Host to net stop/start service name

On the next check the service is determined to be running and the check returns to an OK state

AutoIT What is it?

AutoIT is actually has numerous partsAutoIt Recorder: Will record mouse gestures, and keyboard use and output as AutoIt script

SciTE Editor: The provided program with which to create and edit AutoIt scripts

Aut2Exe: A converter which may be used to create executable files from AutoIt scripts

AutoItX: DLL and COM control, allowing you to add AutoIt features to your favorite scripting language.

AutoIT Purpose

Originally designed to configure massive amounts of PCs, and has over time become a very powerful scripting language supporting complex expressions, loops, user functions, and just about everything in between.Easy to learn syntax

Manipulation of processes and windows

Use standard windows controls

Creation of GUIs

Unicode support

Works with Windows User Account Control and much more

AutoIT Purpose Cont.

Small and stand-alone, no registry entries or .dll's required which makes it easy to use on servers.

Scripts can be compiled into stand-alone executables for security reasons.

AutoIT is now fully Unicode aware and supports both 32, and 64-bit versions for it's main components.

AutoIT gives you full window support, windows are referenced by title, text, size, position, class, and internal Win32 API handles.

The Rule of the Automated Monitoring Road

Maintain control

Automation is great but use caution when over automating

You could end up making more work for yourself cleaning up

Integration With Nagios

Check Page Load Time

Checks the complete time a web page takes to load from the firefox browser.

Sends the data via NRPE through NSClient++ to Nagios

Nagios outputs the data on the XI web UI in a clear format

Check Page Load Time

Example video of AutoIT Recorder

Check Page Load Time

The Good, The Batch, And The Ugly

When the cron comes up for Nagios XI to run this check a remote batch file is executed.

Check Page Load Time

AutoIT script in the SCITE editor.

Check Page Load Time

AutoIT script cont.

Check Page Load Time

AutoIT script cont.

Check Page Load Time

The Batch file dumps text from the notepad created file to Nagios

The Nagios Server receives this as a long text string

The string is converted into a friendly readable format

Check Page Load Time

Check Program Load Time

AutoIT Timer Plugin Exchange PageIntegrating autoIT With Nagios Documentation

Windows Service Restart Event Handler

Event Handlers, what are they?

Triggered when a Host or Service changes stateRestarting a service

Entering helpdesk tickets

Logging database information

Intermediary between Nagios and a remote hostCan be used to run a script if state is

Most efficient way to tie in AutoIT scripts otherwise the automation scripts may overlap, or continue running.

Service Restart Event Handler

The Nagios Service is checking the existence of a running Windows Service

The Service goes down and Nagios see's this on its routine check

The Service was set up with a event handler to restart the Service and it is triggered

The Windows Service is restarted and Nagios switches the Service back to an Ok status

Service Restart Event Handler

Yes...another batch file.

Simple yet effective, you don't want to get your hands tied if something goes wrong.

Injects a Administrator provided argument from Nagios instead of being hard coded due to security reasons.

Service Restart Event Handler

Sending the command via NRPE from the Nagios Terminal

Providing a argument for the Service we desire to be restarted

The nice thing about the simplicity of the batch file is it only does two things

It stops the Service if it is running, if not it starts the Service, a simple on and off switch

Service Restart Event Handler

Example of the Event Handler script

Since the check only looks at two things If the service is up or if it is down We will only trigger this if it is in a Critical or Downed State

Possible uses for Event Handler driven AutoIT scripts

Using check_windowsupdates to check for new updates. When a certain value is met, say... 5 security updates, the event handler is triggered which starts an AutoIT script to update Windows

Cron Windows to run a system defrag, then when it has been run an event handler triggers a script to open a performance graphing program to see the difference so it is ready for you Monday morning when you walk in with your coffee

Since AutoIT is not available for Linux distributions, you could use a terminal running on Windows to send commands to a Linux system, if, say NRPE goes down and you want to tail a log after you restart it

Service Restart Event Handler

Host Service Restart Event Handler

Event Handler Exchange Page

Service Restart Event Handler Documentation (Windows)Service Restart Event Handler Documentation (Linux) - Alex Griffin

Using Selenium for Web Based Testing and Resolutions

Selenium

Selenium automates browsers. What you do with that power is entirely up to you. Primarily it is for automating web applications for testing purposes, but is certainly not limited to just that. Boring web-based administration tasks can also be automated as well.

Selenium has the support of some of the largest browser vendors who have taken (or are taking) steps to make Selenium a native part of their browser. It is also the core technology in countless other browser automation tools, APIs and frameworks.

Support for over 10 different language packs such as Java, C#, Ruby, Python, and Perl.

Selenium also includes an easy to use web recorder, with the ability to edit the script as you go.

Selenium

The four core Selenium Projects:Selenium IDE: Firefox add-on for recording and playback of tests in Firefox 2+, also able to generate code to run tests with the RC

Selenium Remote Control (RC): Server system for controlling browsers locally or remotely including almost every programming language and framework for testing

WebDriver: Natively drive browsers remotely or locally

Grid: Mass deployment of Selenium tests across multiple servers

Check_Selenium

Check_selenium is a Nagios friendly check designed to pass selenium Perl scripts through to a remote system. In this way you can control web based automation and testing through Selenium via Nagios. The checks return all performance data in a way readable by Nagios Core, and Nagios XI.

Examples are in Perl

Check_Selenium

Example of Selenium IDE, User friendly, loads of under the hood options

Check_Selenium

This check periodically opens a remote Firefox browser and performs a functionality check of the Nagios.com website

It runs through each of the drop down menu items to make sure none of those link are broken

Check_Selenium

Results of our test on the Nagios Terminal, Selenium RC reports each test live on separate lines so you can see an issue as it occurs.

Check_Selenium

Output on the Nagios XI Web UI in a short, friendly manner

Output on the Nagios Terminal

Check_Selenium

Selenium Example Video

Check_Selenium

Error from example video

Check_Selenium

Perl Selenium check

Selenium Integration Document

Perl check_selenium Exchange Page

Other Options?

Besides the programs and methods shown here, what other options do I have?

WinAutomation by Softmotive, Ltd.Has the added benefit of everything combined into one UI, it does offer you more control over intricate things like mouse movement coordinates

Clean, clear UI with images to assist you in seeing what your script is actually doing

Combines the effectiveness of Selenium, AutoIT and Event Handlers into one, but is difficult to incorporate with Nagios because of this togetherness

Other Options Cont.

Automate by Network Automation, IncInteractive drag and drop work flow design much in the same fashion as WinAutomation

Has a tight non scripting attitude about their software, which could be good or bad.

Both of these solutions would be good local problem solvers to a Windows environment as you can trigger localized Event Handlers based on the software and a set of events.

Thank you!

Click to edit the outline text format

Second Outline Level

Third Outline Level

Fourth Outline Level

Fifth Outline Level

Sixth Outline Level

Seventh Outline Level

Eighth Outline Level

Ninth Outline Level

Click to edit the title text format

2012

Click to edit the title text format

Click to edit the outline text formatSecond Outline LevelThird Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline LevelSeventh Outline LevelEighth Outline LevelNinth Outline Level