1

Automate user’s actions with Selenium, AutoIT and Sikuli

Mateusz Radkiewicz

2

Let’s automate simple web application!

• Selenium WebDriver

What about more complex actions?

• Selenium Actions, Javascript Executor

Need to interact with OS interface?

• Java.awt.Robot

Come to grips with MS Windows.

• AutoIT

Sky is the limit!

• Sikuli

Agenda

3

Selenium WebDriver

Basic Selenium WebDriver interactions:

• Open a web page

• Hit „back” and „forward” buttons

• Refresh a page

• Get current URL

• Delete or add a cookie

• Get the source of a page

• Switch to another tab/window

• Close the tab

• Click a link, button, checkbox, radiobutton, image

• Enter text into textfield, textarea

• Get text from a label

• Get web page element’s attributes

• Check if element is displayed

• Get element’s location on the screen

Control a browser Interact with a page

4

Selenium WebDriver

Let’s automate simple web application!

5

Selenium WebDriver

Amazon.com – simple test case

Objectives:

• Go to amazon.com

• Search for „octocopter”

• Get number of search results

6

Selenium WebDriver

Demo

7

Selenium WebDriver

What about more complex actions?

8

Selenium WebDriver

Selenium Actions

Simulate more advanced actions using mouse and keyboard:

• Click right mouse button

• Double click

• Move mouse over element

• Drag and drop

• Press combination of keys

• Hold Ctrl key and click elements on the list

9

Selenium WebDriver

Selenium Actions - example

Objectives:

• Select „Ford” and „Porsche”

• Move them to the right container

10

Resize text area by modifying „rows” attribute

Scroll the page left by 20 pixels

Enter text with JavaScript

Enable the button

Selenium WebDriver

Javascript Executor – a little bit of magicA simple way to execute JavaScript code directly from Selenium test

11

Java.awt.Robot

Need to interact with OS interface?Robot lets you press the keys and click on the coordinates on a screen.

12

Java.awt.Robot

Robot - exampleObjectives:

• Open TED.com page

• Press Ctrl+F to enable search

• Search for „gems” word

13

AutoIT

Come to grips with MS Windows

14

AutoIT

Features• Simple scripting language for automating Windows GUI

• Interacts with all standard windows controls

• Simulates keyboard and mouse actions

• Suitable for general scripting and creating simple applications

• Scripts can be compiled into standalone *.exe

• Easy to learn

• Freeware

15

AutoIT

File upload with Selenium and AutoIT - exampleObjectives:

• Go to „https://careers.kainos.com/members/”

• Click „Upload a CV …” button

• Enter a path to local file and click „Open”

16

AutoIT

Step 1: Inspect the windowUse „AutoIT Window Info” tool to identify the window, field and button.

17

AutoIT

Step 2: Write the script in „SciTE” editor

18

AutoIT

Step 3: Compile the code

Step 4: Include AutoIT *.exe into Selenium test

• Open „SciTE” menu Tools > Compile

• Enter location of output *.exe file

• Click „Compile Script”

19

AutoIT

Demo

20

Sikuli

Sky is the limit!

21

Sikuli

Features• Lets you automate anything that is displayed on the screen

• Uses image recognition algorithms to identify GUI components

• Can reach anything that’s out of range for other tools

• No need to have access to application’s structure

• Cross-platform

• Freeware

• Provides own IDE (Jython)

• Provides libraries for development in Java

• Extremely easy to create automatic test, problematic to maintain it

22

Sikuli

File download with Selenium and Sikuli - exampleObjectives:

• Go to „https://code.google.com/p/altdrag”

• Click „Download installer”

• Click „Save File” button

• Click „Save” button in the next window

• Press Win+D to minimize all windows

• Verify downloaded on the desktop

• Delete above file

23

Sikuli

Step 1: Download and install Sikuli• http://www.sikuli.org

Step 2: Include Sikuli in your project• Add „sikuli-java.jar” library in your IDE

Step 3: Create screenshots

24

Sikuli

Step 4: Write the script

25

Sikuli

Demo

26

Sikuli

Sikuli IDE

27

Sikuli

Matching Preview

28

Sikuli

Let’s rock!Objectives:

• Run the script on Windows

• Switch to Ubuntu on VMware

• Open Firefox

• Navigate to Rio de Janeiroon Google Maps

• Collapse side panel

• Enable Google Street Viewon Astoria Palace

• Look around Copacapanauntil you find a girl on a bicycle

• Zoom into the girl

29

Sikuli

Sikuli IDE - demo

30

31