TEST MANAGEMENT WITH ISO 29119

TEST MANAGEMENT

ADOPTING SKILLS TO BUILD EFFICIENT TEST PROCESS

Yulia Zavertailo, Senior Test Manager

02.05.2023

About me

• 11 years of work experience in Quality

Assurance field.

• Itera’s employee since 2009, consultant since

2014.

• Worked in nearshore, offshore, in different

environments.

• One of key competences is establishment of

entire test organization (Itera), QMS (KLP) and

implementation of ISO quality standards.

• IT project manager/Test leader at Telenor.

/ 4

02.05.2023

Agenda

• A real IT story

• Introduction to ISO and ISO 29119

• Digging into ISO 29119-2:2013 test process

• Tips/toolkit

/ 5

02.05.2023

First of all…

/ 6

A STORY of…

NOT Glory

02.05.2023

Survey

/ 7

02.05.2023

Ruter.no

/ 8

02.05.2023

Gjensidige.no

/ 9

o Submit claim error was sent over to Gjensidige in August, fixed, though the same issue appeared again in December …

o In addition, following bugs were also discovered: o My claims to disappear from the list of my claimso Couldn’t open pdf report for a claim (eventually service

desk emailed it to me…)o Travel cards disappear from my mobile app

02.05.2023

Iphone 6. IOS 9. Lock screen keyboard

/ 10

02.05.2023

Skiforeningens mobile app

/ 11I …hate Sinober!

02.05.2023

Norwegian.no mob app

/ 12

02.05.2023

Lessons learnt

• It is not enough just “to test”/quality is still low: – > IT often fail to assure no bugs on key functions in production which cost

customers time, money and patience.

• IT solutions become insecure to use: – >many end users cant trust software features and content.

• Value of IT digitalization is doubtful: – Does it make sense to automate online sale of tickets if user still has to

trust paper version?

– Do our users receive relevant notifications AND at the right time?

/ 13

02.05.2023

Quality Assurance and Quality Control

• Quality assurance is: all the planned and systematic activities

implemented within the quality system that can be demonstrated

to provide confidence that a product or service will fulfill

requirements for quality.

• Quality control is: the operational techniques and activities used

to fulfill requirements for quality.

/ 14

.Good test process! And across the organization.

02.05.2023

ISO STANDARDS

/ 15

02.05.2023

What is ISO/general

• ISO is the world largest standards developing organization.

• Accepted and applied all around the world.

• NOT ONLY IT (Between 1947 and the present day, ISO has published more than 16 500 International

Standards, ranging from standards for activities such as agriculture and construction, through mechanical

engineering, to medical devices, to the newest information technology developments).

/ 16

02.05.2023

ISO standards in nutshell

make the development, manufacturing and supply of products

and services more efficient, safer and cleaner

facilitate trade between countries and make it fairer

provide governments with a technical base for health, safety and

environmental legislation, and conformity assessment

share technological advances and good management practice

disseminate innovation

safeguard consumers, and users in general, of products and

services

make life simpler by providing solutions to common problems./ 17

02.05.2023

Survey

• Automotive software?

• Aircraft construction?

• Telenor?

• Itera?

/ 18

Quality (and safety)

02.05.2023

ISO/IEC/IEE 29119

/ 19

02.05.2023

ISO 29119. Overview

• It is a serie of standards, not just 1 standard:

– ISO/IEC 29119-1: Concepts & Definitions

– ISO/IEC 29119-2: Test Processes

– ISO/IEC 29119-3: Test Documentation

– ISO/IEC 29119-4: Test Techniques

– ISO/IEC 29119-5: Keyword Driven Testing

• Revised every 5 years

• Last revision is done 2013/09

/ 20

02.05.2023

ISO 29119. Purpose

• The purpose of the ISO/IEC/IEEE 29119 series of software testing standards

is to define an internationally-agreed set of standards for software

testing that can be used by any organization when performing any form of

software testing.

• By implementing these standards, you will be adopting the only

internationally-recognised and agreed standards for software testing, which

will provide your organisation with a high-quality approach to testing

that can be communicated throughout the world.

• It is not an alternative to ISTQB process, since ISTQB is not a proven

standard.

/ 21

02.05.2023

Any kind of ISO in IT in Norway... Why we don’t use it?

My theories….

• Local companies, are happy in our traditional environment (with bugs).

• Small companies, don’t want to invest (same as above).

• Lack of ISO knowledge as such (easier with ISTQb…).

• Lazy to find out what is ISO/how to apply/use best practices. Run after juicy

modern IT stuff instead.

– Last but not least….Chip top, top bop, hop pop…and in production! Fast,

efficient, with bugs.

/ 22

02.05.2023

ISO/IEC 29119-2: TEST PROCESSES

/ 23

02.05.2023

Mulit-layer model

• Organizational test process: – Processes on organizational level. F.ex test policy, test

strategy for entire organization. • Test Management processes:

– processes related to test management of entire project/phase/test type within a project.

• Dynamic test processes: – Processes which go in particular phase of testing f.eks SIT,

UAT etc or certain type of testing (ex. Performance, security, etc)

/ 24

Quite often missing!

02.05.2023

Mulit-layer model

/ 25

Organizational test processes

Dynamic test processes

Test management processes

Test planning

Test design and implementation

Test environment

set-up

Test execution

Test monitoring and control Test compeltion

Test incident reporting

02.05.2023

Organizational test process

/ 26

Organizational test process(Applied to test policy)

Organizational test process(Applied to test strategy)

Test Management process(Applied to project test management)

Org

aniz

atio

nal t

est p

olic

yFeedback on O

rganizational test policy

Organizational test policy

Organizational test strategy

Feedack on Organizational

test policy

Feecback on Organizational test strategy

Test management processes

02.05.2023 / 27

Organizational test processes

Dynamic test processes

Test management processes

Test planning

Test design and implementation

Test environment

set-up

Test execution

Test monitoring and control Test compeltion

Test incident reporting

Test resultsTest plan

Test completion report

Organizational test policy and test strategy

Feedback on organizational test policy and test strategy

Test plan Test Measures

Test Status reports, test completion report

Test management processes

02.05.2023

Dynamic test processes

/ 28

02.05.2023 / 28

Dynamic test processes

Test design and implementation

Test environment

set-up

Test execution

Test incident reporting

Test planTest Measures

Control directives

Test management processes

Test specification

Test environment requirements

Test environment readiness report

Test results No issues

Issues

02.05.2023

Yulia’s Toolkit

• See Excel example for test planning.

/ 29

Microsoft Excel-regneark

02.05.2023

Summary

ISTQB sub-processes ISO 29119-2:2013• Test planning and control • Test analysis and design• Test implementation and execution• Evaluation exit criteria and reporting• Test closure activities

• Organizational test process• Test management processes:

– Test planning process– Test monitoring and control

process– Test completion process

• Dynamic test processes – Test design and implementation– Test environment setup and

maintenance– Test execution process– Test incident reporting process

/ 30

02.05.2023

Sources

• iso.org • Google• Life-experience/practice• Online courses at udemy (lots of)• Books…

/ 31

02.05.2023

THANK YOU

/ 32