STC Conference 2013 - QAI Global Servicesconference.qaiglobalservices.com/stc2013/PDFs/... · STC Conference 2013 Getting to know Robot Framework – The Generic Test Automation Framework

STC Conference 2013

Getting to know Robot Framework – The Generic Test Automation Framework

1

STC Conference 2013

Getting to know Robot Framework

- The generic test automation framework

(Tutorial)

Author 1 : Aravindhan Ashokan

Author 2 : Siva Sankar S

Name of the organisation : Indium Software (India) Limited

Address of organisation VDS House

41, 2nd Floor

Cathedral Road

Chennai – 600 086.

STC Conference 2013


2

1.1 Abstract

Robot framework is an open source test automation framework for generic test automation needs, usually used for acceptance testing and acceptance test driven development (ATDD). It is keyword driven, easy to install, has a table like structure and most importantly English-like commands for executing the scripts. It is very easy for non-programmers or business people to write tests in the framework due to its ease of use. In this tutorial, the user will get to understand the structure of Robot framework code and also identify the libraries required for their automation need.

1.2 Take Away: The audience will:

1) Understand a. What is Robot framework? b. What is it used for? c. Where is it applied? d. Why to use Robot framework?

2) Understand the various types of testing that can be achieved through Robot framework and choose the library(ies) for their testing need. 3) Get to see and understand example code snippets for automating Inbuilt, Selenium, Swing, AutoIT, Database, SSH, Requests (API), Android and iOS libraries.

1.3 Introduction

Fig 1.3 Robot Framework Introductions

Based on the specific requirement of the users, there are several libraries available that can be used in the framework as per their testing need. The testing capabilities can be extended by the test libraries that are written either in Java or Python. The framework provides the capability to create new keywords using the existing ones or totally new ones as a part of the existing library. As a result of which we can bundle up multiple keywords (for achieving a purpose) into a new higher level keyword.

STC Conference 2013


3

1.3.1 Why Robot Framework?

Fig 1.3.1 Benefits of Robot Framework

Apart from the above mentioned points, it is an open source tool that can satisfy any of the QA Automation

need an organization might have. It is easy to learn and implement the automation tool with minimal training to the resources.

1.3.2 High-level architecture:

Fig 1.3.2 Robot Framework Architecture

1.4 Different test data tables / Structure of Robot framework code:

Table name Used for Aliases

Setting table 1) Importing test libraries, resource files and variable files 2) Defining metadata for test suites and test cases

Setting, Settings, Metadata

Variable Defining variables that can be used elsewhere in the Variable, Variables

STC Conference 2013


4

Table name Used for Aliases

table test data

Test case table

Creating test cases from available keywords Test Case, Test Cases

Keyword table

Creating user keywords from existing lower-level keywords

Keyword, Keywords, User Keyword, User Keywords

1.4.1 Setting table

The Setting table is used to import test libraries, resource files and variable files and to define metadata for

test suites and test cases. It can be included in test case files and resource files. Note that in a resource file, a Setting table can only include settings for importing libraries, resources, and variables.

Name Description

Library Used for taking test libraries into use.

Resource Used for taking resource files into use.

Variables Used for taking variable files into use.

Documentation Used for specifying a test suite or resource file documentation.

Metadata Used for setting free test suite metadata.

Suite Setup Used for specifying the suite setup.

Suite Teardown Used for specifying the suite teardown.

Suite Precondition A synonym for Suite Setup.

Suite Postcondition A synonym for Suite Teardown.

Force Tags Used for specifying forced values for tags when tagging test cases.

Default Tags Used for specifying default values for tags when tagging test cases.

Test Setup Used for specifying a default test setup.

Test Teardown Used for specifying a default test teardown.

Test Precondition A synonym for Test Setup.

Test Postcondition A synonym for Test Teardown.

Test Template Used for specifying a default template keyword for test cases.

Test Timeout Used for specifying a default test case timeout.

1.4.2 Test Case table The settings in the Test Case table are always specific to the test case for which they are defined. Some of

these settings override the default values defined in the Settings table.

Name Description

[Documentation] Used for specifying a test case documentation.

[Tags] Used for tagging test cases.

[Setup] Used for specifying a test setup.

STC Conference 2013


5

Name Description

[Teardown] Used for specifying a test teardown.

[Precondition] A synonym for [Setup].

[Postcondition] A synonym for [Teardown].

[Template] Used for specifying a template keyword.

[Timeout] Used for specifying a test case timeout.

1.4.3 Keyword table

Settings in the Keyword table are specific to the user keyword for which they are defined.

Settings available in the Keyword table

Name Description

[Documentation] Used for specifying a user keyword documentation.

[Arguments] Used for specifying user keyword arguments.

[Return] Used for specifying user keyword return values.

[Timeout] Used for specifying a user keyword timeout.

1.5 Data formats supported in Robot framework:

Robot framework supports different formats of input/scripts, which is more of a table like structure with rows, columns and cells. The user can pick the format that they are comfortable with.

1.5.1 HTML Format:

Setting Value Value Value

Library BulitIn

Variable Value Value Value

${VALUE} Hi

Test Case Action Argument Argument

Example1 [Documentation] Demo test

Log ${VALUE}

Verify Path /sample

Keyword Action Argument Argument

Verify Path [Arguments] ${folderPath}

Directory Should Exist ${folderPath}

STC Conference 2013


6

1.5.2 TSV format:

*Setting*

Library BuiltIn

*Variable*

${VALUE} Hi

*Test Case*

Example1 [Documentation] Demo test

Log ${VALUE}

Verify Path /sample

*Keywords*

Verify Path [Arguments] ${folderPath}

Directory Should Exist ${folderPath}

1.5.3 Plain text format:

***Settings*** Library BuiltIn *** Variables *** ${VALUE} Hi *** Test Cases *** Example1 [Documentation] Demo test Log ${VALUE} Verify Path /sample *** Keywords *** Verify Path [Arguments] ${folderPath} Directory Should Exist ${folderPath}

1.5.4 Pipe and space separated format:

| *Setting* | *Value* | | Library | BuiltIn | | *Variable* | *Value* | | ${VALUE} | Hi | | *Test Case* | *Action* | *Argument* |

STC Conference 2013


7

| Example1 | [Documentation] | Demo test | | | Log | ${VALUE} | | | My Keyword | /sample | | *Keyword* | | Verify Pat | [Arguments] | ${folderPath} | | Directory Should Exist | ${folderPath}

1.6 RIDE (Robot IDE)

Fig 1.6 Robot IDE

Robot IDE (RIDE in short) is also a python based tool, which can understand and support different table formats that Robot framework can utilize. Based on the libraries we select in the Settings table, it has the ability to display related keywords as shown in figure 1.6.

1.6.1 Sample robot test results and execution logs:

STC Conference 2013


8

Fig 1.6.1 Sample Test results and log

1.7 Robot Libraries:

1.7.1 Standard libraries Following are the standard test libraries available along with Robot Framework, there is no requirement to

explicitly add any of these libraries in our test.

• BuiltIn : Test generic keywords are provided in this library.

• Collections : A library to handle lists, arrays and dictionary objects available in Python.

• Dialogs : Dialogs library provides means for pausing the test execution and getting input from users.

• Operating System : Library to handle OS related functions.

• Process : Robot Framework test library for running or closing processes that are already running.

• Screenshot : Screenshot is Robot Framework's standard library that provides keywords to capture and store screenshots.

• String : String manipulation operations can be done using this standard library.

• Telnet : Telnet is Robot Framework's standard library that makes it possible to connect to Telnet servers and execute commands on the opened connections.

• XML : XML is a test library for verifying contents of XML files.

1.7.2 Sample Code:

Built-in/Collections/String Library:

*** Settings *** Library BuiltIn Library Collections

Library String *** Test Cases ***

Example @{words} = Create List Hi Hello world

:FOR ${word} IN @{words}

Run Keyword If '${word}' == 'Hello' Exit For Loop ${len}= Get Length ${words} Log ${len}

${count} = Get Count ${words} Hi Should Be True 5 < ${count} < 10 Length Should Be ${words} 3 Should Be Empty ${words}

Sleep 10 ${time} = Get Time Log ${time}

Repeat Keyword 5 times Log demo Wait Until Keyword Succeeds 2 min 5 sec Log demo

STC Conference 2013


9

Dialogue Library:

*** Settings *** Library Dialogs *** Test Cases ***

Example Execute Manual Step Test execution paused #Perform manual operations and assign pass or fail status

${value} = Get Value From User Enter the value 1 Log ${value} Pause Execution Test execution paused. Press OK to continue.

Operating System Library:

*** Settings *** Library OperatingSystem

*** Testcases *** Example

Create Directory ./demo Create File ./demo/sample1.txt Hello!! File Should Exist ./demo/sample1.txt

Copy File ./demo/sample1.txt ./demo/sample2.txt File Should Exist ./demo/sample2.txt ${content}= Get File ./demo/sample2.txt

${file_size}= Get File Size ./demo/sample2.txt

Telnet Library:

*** Settings *** Library OperatingSystem

Library Collections Library Telnet 120 Library String

Resource ${Test_Conn} *** Keywords ***

Open connection and log in Open Connection ${HOST} Login ${USERNAME}

${PASSWORD} Execute Commands

Telnet.Open Connection ${HOST} Telnet.Login ${USERNAME} ${PASSWORD}

${prompt} ${regexp} = Telnet.Set Prompt $ Telnet.Write ksh ${prompt} ${regexp} = Telnet.Set Prompt $

${out} = Telnet.Read Until Prompt Append To List ${final_log} Command->ksh

Append To List ${final_log} ${out} ${prompt}${regexp} = Telnet.Set Prompt Pick one.

Telnet.Write ${PROFILE} Table_Insert Telnet.Write ggsci

Telnet.Read Until GGSCI (${HOST}) 1> Telnet.Write info all Telnet.Read Until GGSCI (${HOST}) 2>

Telnet.Write ${OBEY_INSERT} Telnet.Read Until GGSCI (${HOST}) 3>

*** Test Cases *** Test Case 1 Local To Remote File Transfer

STC Conference 2013


10

Execute Commands Table_Insert

XML Library:

*** Settings *** Library XML

*** Test Cases *** Example ${root} = Parse XML ./demo.xml

Should Be Equal ${root.tag} demo ${first} = Get Element ${root} firstnode

Should Be Equal ${firstnode.text} sample

Dictionary Should Contain Key ${firstnode.attrib} id Element Text Should Be ${firstnode} sample Element Attribute Should Be ${firstnode} id 3

Element Attribute Should Be ${root} id 6 xpath=first Element Attribute Should Be ./demo.xml id 3 xpath=first

1.7.3 External libraries: The unlike internal libraries the external libraries are required to be installed in order to use those.

• SeleniumLibrary - A web testing library that uses popular Selenium tool internally.

• Selenium2Library - A drop-in-replacement for SeleniumLibrary using newer Selenium 2 WebDriver API.

• Selenium2Library Java port - Java implementation of Se2Lib. Compatible with Jython 2.5.

• watir-robot - A web testing library that uses popular Watir tool via the remote library interface.

• WatinLibrary - A web testing library that uses Watin tool (a .NET port of Watir) via the remote library interface.

• SwingLibrary - A Swing GUI testing library.

• EclipseLibrary - A library for testing Eclipse RCP applications using SWT widgets.

• AutoItLibrary - Windows GUI testing library that uses AutoIt freeware tool as a driver.

• DatabaseLibrary (Java) - A test library that provides common functionality for testing database contents. Implemented using Java so works only with Jython.

• DatabaseLibrary (Python) - Another library for database testing. Implemented with Python and works also on Jython.

• SSHLibrary - A test library that enables SSH and SFTP.

• HTTP test library using livetest

• HTTP test library using Requests

• SudsLibrary - library for testing SOAP-based web services

• AndroidLibrary - Library for Android testing that uses Calabash Android internally.

• IOSLibrary - Library for iOS testing that uses Calabash iOS Server internally.

• Rammbock - Generic network protocol test library.

• How-To: Sikuli and Robot Framework Integration - This is not really a library but these instructions explain how to integrate Sikuli tool with Robot Framework

1.7.4 Sample Code:

Selenium/Selenium2 Library:

*** Settings ***

Library SeleniumLibrary Suite Setup Login

*** Variables *** ${url}= https://demo.com ${browser}= gc

*** Keywords *** Login

Start Selenium Server

STC Conference 2013


11

Open Browser ${url} ${browser} Set Selenium Timeout 15

Set Selenium Implicit Wait 30 Input Text name demo1 Input Text pass test1 Click Element login

Wait Until Page Contains Element //demo1 60 Logout

Close Browser Stop Selenium Server

*** Testcases *** Example

Login

Swing Library:

*** Settings *** Library SwingLibrary

*** Test Cases *** Sample Test Case

Start Application demo.demoApplication

Select Window demo List Insert Into Text Field description testing Push Button add

Select From List demolist 1 ${item}= Get Selected Value From List emolist Should Be Equal ${item} testing

Push Button cancel Select From List demolist 1 Push Button cancel

${items}= Get List Item Count demoList Should Be Equal As Numbers ${items} 1

AutoIt Library:

*** Settings *** Library AutoItLibrary

*** Keywords ***

Calculate 6x5 Win Wait Calculator ${empty} 60 Control Click Calculator Button6

Control Click Calculator ButtonX Control Click Calculator Button5 ${sample}= Control Get Text Calculator Screen

Log ${sample} *** Testcases ***

Calculator Calculate 6x5

Database Library:

*** Settings *** Library DatabaseLibrary

*** Variables **** ${ConnString} Some Connection String

*** Test Cases *** Example Connect to Database ${ConnString}

Go To http://localhost/example/demo.html Input Text q testing

STC Conference 2013


12

Click Button Save @{results} Query select * from stringdetails

Log Many @{results} Check if exists in database select id from stringdetails where q = 'testing' Disconnect from Database

SSH Library:

*** Settings ***

Library OperatingSystem Library Collections Library SSHLibrary

Library Selenium2Library Library String Resource ${Test_Conn}

*** Variables *** @{final_log}= ${empty}

*** Keywords *** Open connection and login

Open Connection ${HOST} Login ${USERNAME} ${PASSWORD}

Local To Remote File Transfer ${SUITE_TEST_NAME}= Convert To String ${SUITE_NAME} @{temp} = Split String ${SUITE_TEST_NAME} .

${SUITE}= Get From List ${temp} 1 Replace String ${SUITE} ${SPACE} _ Log ${SUITE}

SSHLibrary.Open Connection ${HOST} SSHLibrary.Login ${USERNAME} ${PASSWORD} Put File /dirsql/*.* ${DEST_SQL}

Put File /dirprm/*.* ${DEST_PRM} SSHLibrary.Close Connection

*** Test Cases *** Test Case 1 Open Connection and login

Local To Remote File Transfer

Requests Library (API):

*** Settings *** Library String Library RequestsLibrary

*** Variables *** ${AUTH_HEADER}= Authorization:'s0dsa9fasdfjhbwl'

*** Keywords *** API GET Request [Arguments] ${exp_out} ${url}

${API_RESPONSE}= Get session_url ${url} headers=${AUTH_HEADER} ${status}= Run Keyword If '${exp_out}'=='Success' ... Validate API Response 200

... ELSE IF '${exp_out}'=='Error 404' ... Validate API Response 404 ... ELSE IF '${exp_out}'=='Error 503'

... Validate API Response 503 API POST Request [Arguments] ${exp_out} ${url}

# Keyword incomplete Validate API Response ${outcome}

#Additonal Validation keywords goes here *** Test Cases ***

Sample API Test case

STC Conference 2013


13

API GET Request 200 http://some_api_url.com?somequerystring=somevalue API GET Request 404 http://some_api_url.com?someotherquerystring=someothervalue

API GET Request 503 http://some_api_url.com?yetanotherquerystring=yetanothervalue

Android Library:

*** Settings *** Library AndroidLibrary

*** Test Cases ***

Example Start Emulator demo Wait For Device

Press Menu Button Start Testserver with apk demo.apk Wait Until Keyword Succeeds

... 1min ... 5sec ... Connect To Testserver

Capture Screenshot Touch Text Views

Capture Screenshot

Touch Text Controls Capture Screenshot Touch Text 1. Light Theme

Capture Screenshot Screen Should Contain App Capture Screenshot

Screen Should Not Contain testing Setup Set Text Test Set Text 1 test Capture Screenshot

Screen Should Contain test

iOS Library:

*** Settings *** Library IOSLibrary

*** Test Cases *** Example

Set Device URL localhost:4452 Start Simulator Wait Until Keyword Succeeds 1 minute 5 seconds Is Device Available

Swipe right Rotate left Setup Set Text Test Set Text 1 test

Capture Screenshot Screen Should Contain test

1.8 Conclusion: Robot framework is easy to use and yet powerful acceptance level automation framework. It is flexible enough to extent its functionalities through Java and python modules. Test engineers without any programming experience can write test cases in it. The test cases written in tables make the test, well-structured and easily readable. The logs and reports provided by it are good information resource. If we can identify the robot library for our specific needs, it is pretty easy to automate any requirement that we may have. Please contact us to know more on how to perform test automation with Robot Framework which secures web applications.

STC Conference 2013


14

1.9 Appendix: Case Study of a project where we implemented Robot framework

Client : A leader in Media Space

Tools Used for Automation: Robot Framework: � Functional Regression Automation :

Selenium Library Selenium 2 Library Database Library

� API Testing: Requests Library

� Continues Integration Testing: Jenkins / Batch/Shell/Bash Scripting

� Compatibility Testing OS : Mac / Linux / Windows Browsers : Chrome/ Firefox/ IE / Opera

� Over Riding Robot Implementation: Python Scripting

Business Requirements: � Database Testing, User Interface testing and

API Testing � Test the end to end functionality of the web

based Tracking application � Compatibility with OS and Browsers ensuring

its performance � Frequent execution of regression test suite due

to daily builds

Challenges Faced by the Client: � Compatibility testing on different Platforms /

Operating Systems / Browsers

Challenges Faced at Indium: � Setting up the Robot Framework for this

application

The Approach: � Understand application functional features and

architecture � Develop test cases for various modules using

XML tool � Design Pattern using Page Object Model

Indium’s Solutions:

• We suggested Robot Frame work because it has variety of libraries like Operating System Library, Telnet Library, Selenium Library, Selenium2 Library, Swing Library, Database Library, SSH Library, Http Request Library, and API Testing Library ensuring the efficiency of testing and it is the best fit for heterogeneous applications

Services Offered: � Continues Integration/Smoke Testing � Regression Automation � Compatibility Testing � Integration Testing � Target based execution on cloud (Browser

Stack)

Value � Decreased Regression cycle time � Early detection of Code Merge Issues � Drastically reduced testing cycle time � 55% Reduction in Manual cost

Some metrics:

Fig 1.6 Some of the metrics wrt the case study

STC Conference 2013


15

References:

1) Indium Software – Robot Framework Jumpstart kit

2) http://robotframework.googlecode.com/hg/doc/userguide/RobotFrameworkUserGuide.html

3) https://code.google.com/p/robotframework/wiki/TestLibraries

Author Biography:

Aravindhan Ashokan

Aravindhan Ashokan is a Senior Project Manager in automation working for Indium Software (India) limited, Chennai and also head the automation CoE at Indium software. He has previously worked for organisations like Cybage, HCL, Sutherland etc. and holds a double masters degree in MCA and MBA. He was also holding a CSTE certification till Dec 2009. He is a professional with 13+ years of experience in Software Testing, Quality Department, Process Operations, MIS, Customer Service and Team Management; inclusive of 7.6 years in Quality Assurance and related areas. Well versed in Manual and Automation testing with good exposure to frameworks implementation. He has worked for colossal clients like Google, MySpace, The Washington Post, Hearst (iCrossing), British Telecom, Federal Home Loan Bank (FHLB), University of Pittsburgh Medical Center (UPMC), Critical Mention, Amazon and Mcafee.

Siva Sankar S

Siva Sankar started his career with Indium Software as a Test Associate and currently has 1.6 years of experience in Robot Framework. He is one of the active members of Automation CoE at Indium Software and has contributed to various R&D activities for the Solutions team. He has implemented target based execution using Robot Framework in his current project. He completed B.Tech from Saveetha Engineering College affiliated to Anna University, Chennai.

Documents

STC Conference 2013 - QAI Global Servicesconference.qaiglobalservices.com/stc2013/PDFs/... · STC Conference 2013 Getting to know Robot Framework – The Generic Test Automation Framework