Containerized End-2-End Testing

+

, Tobias Schneck ConSol Software GmbH

Test Pyramid

RobustIndependentIsolatedFast

Characteristics of End-2-End Testing

Different types of testing:Regression testsFunctional approval tests

Parallel tests with GUIs are complexStateful tests: user logins, sessions, historySetup and cleanup of test data

Manual effort > effort for test automation

Advantages of Container Technology

Isolation of environmentsRepository for versioning and distributionReproducible application environment

Dockerfile, docker-compose.ymlOptimized for parallel execution and cloud systems

Less memory and CPU overhead (shared Linux kernel)Starting containers on-the-flyUnique command line interface (orchestration tools)

Virtual Machine vs. Container (e.g. Docker)

Containerized GUIs### start the docker container via x-forwarding docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix:rw rasch/inkscape

### start the docker container with VNC interface # connect via URL: http://localhost:6911/vnc_auto.html?password=vncpassword docker run -it -p 5911:5901 -p 6911:6901 consol/ubuntu-xfce-vnc docker run -it -p 5912:5901 -p 6912:6901 consol/centos-xfce-vnc docker run -it -p 5913:5901 -p 6913:6901 consol/ubuntu-icewm-vnc:dev docker run -it -p 5913:5901 -p 6914:6901 consol/centos-icewm-vnc:dev

What's provided by ?

CategoryWeb tests through HTML selectors

Restricted to browser content

Open Source & Java API

Headless execution

Test writing assistance(recorder, tag finder)

Automatable / Test result reporting(CI, DB, monitoring environment)

Monitoring Integration Nagios OMD Icinga Check_MK

Test Definition (Java Script)

Test Case Structure// testcase.js

/************************************* * Initialization *************************************/ _dynamicInclude($includeFolder); var testCase = new TestCase(60, 70); var env = new Environment(); var appNotepad = new Application( "gedit"); var region = new Region();

/****************************** * Description of the test case ******************************/ try {

//...

/************************************************ * Exception handling and shutdown of test case **********************************************/ } catch (e) { testCase.handleException(e); } finally { testCase.saveResult(); }

Call all Sahi Functions for Web Testing// testacase.js /************************ * Call Sahi Functions ***********************/

_navigateTo( "http://sahi.example.com/_s_/dyn/Driver_initialized" ); _highlight(_link( "SSL Manager" )); _isVisible(_link( "SSL Manager" )); _highlight(_link( "Logs")) _click(_link( "Logs"))

testCase.endOfStep( "Test Sahi landing page" , 5);

Fluent API for UI Testing// testacase.js

/*** open calculator app ***/ var calculatorApp = new Application( "galculator" ).open(); testCase.endOfStep( "Open Calculator" , 3);

/*** calculate 525 + 100 ***/ var calculatorRegion = calculatorApp.getRegion(); calculatorRegion.type( "525") .find("plus.png") .click() .type("100");

calcRegion.find( "result.png" ).click(); calcRegion.waitForImage( "625", 5);

testCase.endOfStep( "calculate 525 +100" , 20);

Custom Functions// e.g. excluded into a separate common.js

/********** * Combine click and highlight *********/function clickHighlight ($selector) { _highlight($selector); _click($selector); }

/*************** * Open PDF in native viewer **************/ var PDF_EDITOR_NAME = "masterpdfeditor3" ;

function openPdfFile(pdfFileLocation ) { return new Application(PDF_EDITOR_NAME + ' "' + pdfFileLocation + '"').open(); }

Test Definition (Java)public class FirstExampleTest extends AbstractSakuliTest {

private static final String CITRUS_URL = "http://www.citrusframework.org/" ; private Environment env; private Region screen;

@BeforeClass @Override public void initTC() throws Exception { super.initTC(); env = new Environment(); screen = new Region(); browser.open(); }

@Override protected TestCaseInitParameter getTestCaseInitParameter () throws Exception { return new TestCaseInitParameter( "test_citrus" ); }

@Test public void testMyApp() throws Exception { // ... testcode }}

For the Maven dependencies, take a look at the .Java DSL Documentaion

Test Definition (Java)public class FirstExampleTest extends AbstractSakuliTest {

// ... initializing code

@Test public void testCitrusHtmlContent () throws Exception { browser.navigateTo(CITRUS_URL); ElementStub heading1 = browser.paragraph( "Citrus Integration Testing" ); heading1.highlight(); assertTrue(heading1.isVisible());

ElementStub download = browser.link( "/Download v.*/" ); download.highlight(); assertTrue(download.isVisible()); download.click();

ElementStub downloadLink = browser.cell( "2.6.1"); downloadLink.highlight(); assertTrue(downloadLink.isVisible()); }

@Test public void testCitrusPictures () throws Exception { browser.navigateTo(CITRUS_URL); env.setSimilarity( 0.8); screen.find( "citrus_logo.png" ).highlight(); env.type(Key.END); screen.find( "consol_logo.png" ).highlight(); }}

Sakuli End-2-End Testing Container

Demo - Sakuli Container# start the docker container docker run -it -p 5911:5901 -p 6911:6901 consol/sakuli-ubuntu-xfce docker run -it -p 5912:5901 -p 6912:6901 consol/sakuli-centos-xfce docker run -it -p 5913:5901 -p 6913:6901 consol/sakuli-ubuntu-xfce-java docker run -it -p 5914:5901 -p 6914:6901 consol/sakuli-centos-xfce-java

# start in parallel via docker-compose # use docker-compos.yml from https://github.com/ConSol/sakuli/tree/master/docker docker-compose up

Bakery Demo Setup

Bakery Demo Setup

Bakery Demogit clone https://github.com/toschneck/sakuli-example-bakery-testing.git

# start jenkins jenkins/deploy_jenkins.sh # start OMD montioring omd-nagios/deploy_omd.sh

# start the build of the application images bakery-app/app-deployment-docker-compose/deploy_app.sh

#start tests sakuli-tests/execute_all.sh #start tests for monitoring sakuli-tests/execute_all_4_monitoring.sh

What's next? Cloud-ready Containers!

In development for release 02/2017:Security: user namespacesSource2ImageBuild and run configurations

What's next?

Headless execution - Linux: VNC & Docker Windows: ?Video recording of the test execution (error documentation)Web UI to handle Sakuli test suitesConnect 3rd-party test management tools (HP QC, TestRail, ...)Selenium as an alternative to SahiImplement Junit 5 test runner

Links

ConSol/sakuli

ConSol/sakuli-examples

toschneck/sakuli-example-bakery-testing

sakuli@consol.de @sakuli_e2e

Thank you Munich!

Tobias Schneck

ConSol Software GmbHFranziskanerstraße 38D-81669 MünchenTel: +49-89-45841-100Fax: +49-89-45841-111

tobias.schneck@consol. toschneck

info@consol.dewww.consol.de

ConSol