Martin Pol - Questioning the Evolution of Testing: What's Next? - EuroSTAR 2013

Martin Pol

Questioning The Evolution Of Testing: What’s Next?

www.eurostarconferences.com

@esconfs#esconfs

Then Today

S

tru

ctu

rePioneering

Structuring

Optimizing

Unstructured

Undisciplined

Flexible

Structured

Disciplined

Inflexible

• Unknown, no buy-in, no budget, no process, no tools

• Fully integrated in the development process

– Generalists, joined effort

– Prototype, exploratory driven

• Only test pioneers

Pioneering

Then Today

S

tructu

re

• Struggling required to get involved

• Lacking experiences, references

• Only welcome in emergency situations

•

Pioneering

Then Today

S

tructu

re

Structuring

Then Today

S

tructu

re

• Structuring the process

• Publishing

• Functionality and non-functionality

• Test automation

• Test Process Improvement models

– TPI, TMM, TMMi, TOM, TIM, TSM, …

Structuring

Then Today

S

tructu

re

• Dedicated (trained) staff

• (Early) involvement, master test planning

– Risk based coverage

• Independent testing

• Test competence center, factory methods

• Recognized as a function

Structuring

Then Today

S

tructu

re

• Becoming a referee

• Process Improvement freaks

• Measure and metrics mania

– Spreadsheet management

• Complaining and blaming

• Tunnel vision

Structuring

Then Today

S

tructu

re

ProcessTheorySchedule

ManualIndependentDisciplineStructureStandards

Structuring

Then Today

S

tructu

re

Technology push Efficiency push

The IT Optimizing

Then Today

S

tructu

re

• Architecture

– SOA

– Virtualization

– Web technology

– Devices

• Cloud

*aaS

searching, recording, accounting, paying, writing,

reviewing, tracking, calculating, developing,

listening, analyzing, transmitting, learning,

controlling, purchasing, testing, alarming,

changing, updating, deleting, accessing, rejecting,

correcting, studying, booking, receiving, tracing,

protecting, deciding, managing, teaching,

facilitating, identifying, copying, removing,

demonstrating, checking, showing, selecting,

subscribing, unsubscribing, sharing, mailing,

communicating, reading, playing, working,

meeting, gambling, shopping, storing, cross

checking, retrieving, configuring, sketching,

saving, accelerating, enhancing, creating, growing,

checking in, checking out, finding out, reaching,

denying, talking, designing, making, verifying,

measuring

Technologypush

The IT

Email

Surf

Transfer

Develop and Test

Operate and Manage Store

• Architecture

• Cloud

• SoLoMo

The IT

Technologypush

Hi Martin. Good to see you back!Today: 3 pairs of socks, you pay 2

• Architecture

• Cloud

•

The IT

Technologypush

• Architecture

• Cloud

• SoLoMo

• Big Data

• Integration

Technologypush

The IT

Efficiency push• Outsourcing

• *aaS

• Package software

• Agile, Scrum

• Collaboration

The IT

• Outsourcing

• *aaS

• Package software

• Agile, Scrum

• Collaboration

• DevOps

• Continuous Delivery

Efficiency push

The IT

• Architecture

• Cloud, *aaS

• SoLoMo

• Big Data

• Integration

• Outsourcing

• Packages

• Agile, Scrum

• DevOps

• Cont. Delivery

The ITOptimizing

Then Today

S

tructu

re

Optimizing

Today Tomorrow

• Architecture

• Cloud, *aaS

• SoLoMo

• Big Data

• Integration

• Outsourcing

• Packages

• Agile, Scrum

• DevOps

• Cont. Delivery

Actual Requirements for our Community

Optimizing

Then Today

S

tructu

re

• Agile, Scrum

– Joint effort and success

– Focus on business value

– Social skills required

– Test management role

Testing

• Agile, Scrum

• Context Driven

– Problem solving, practice, heuristics, stories, skills

– People, cooperation, community

– Rapid Software Testing

Optimizing

Then Today

S

tructu

re

Testing

The Seven Basic Principles of the Context-Driven School1. The value of any practice depends on its context.

2. There are good practices in context, but there are no best

practices.

3. People, working together, are the most important part of any

project’s context.

4. Projects unfold over time in ways that are often not predictable.

5. The product is a solution. If the problem isn’t solved, the product

doesn’t work.

6. Good software testing is a challenging intellectual process.

7. Only through judgment and skill, exercised cooperatively

throughout the entire project, are we able to do the right things at

the right times to effectively test our products.

• Agile, Scrum

• Context Driven

• SDTM (Service Driven Test Management)

– From complaining and blaming towards enabling

– Tactful leadership

Optimizing

Then Today

S

tructu

re

Testing

• Agile, Scrum

• Context Driven

• SDTM

• TestOps

– Test in production

– Cloud testing (Iaas, Taas)

– Integration

Optimizing

Then Today

S

tructu

re

Testing

• Agile, Scrum

• Context Driven

• SDTM

• TestOps

• Automation 2.0

– Automation of testing

– Architecture

Optimizing

Then Today

S

tructu

re

TestwareTest scriptsTest casesTest data

Registration Reporting

Test AutomationSoftware – Platform – Infrastructure

Test Automation Process & Organization

ProjectsChanges

PatchesFixes

System Under TestSoftware – Service – Platform - Infrastructure

Testing

Testing

• Agile, Scrum

• Context Driven

• SDTM

• TestOps

• Automation 2.0

• Cloud Computing

– Testing in the Cloud (TaaS)

– Testing Cloud Services

Optimizing

Then Today

S

tructu

re

• Agile, Scrum

• Context Driven

• SDTM

• TestOps

• Automation 2.0

• Cloud Computing

• SoLoMo

– Hot

– Sexy

– Wild

Optimizing

Then Today

S

tructu

re

Testing

• Agile, Scrum

• Context Driven

• SDTM

• TestOps

• Automation 2.0

• Cloud Computing

• SoLoMo

– Technical skills

– Multi dimensional

– Continuous integration

Optimizing

Then Today

S

tructu

re

Testing

• Agile, Scrum

• Context Driven

• SDTM

• TestOps

• Automation 2.0

• Cloud Computing

• SoLoMo

• Outsourcing

– Scoping

– Agreements

– Management

controls, metrics, health checks

test deliverables

test progress test process

product quality

Optimizing

Then Today

S

tructu

re

Testing

Optimizing

Then Today

S

tructu

re

Testing

• Agile, Scrum

• Context Driven

• SDTM

• TestOps

• Automation 2.0

• Cloud Computing

• SoLoMo

• Outsourcing

•

Optimizing

Then Today

S

tructu

re

Testing

• Agile, Scrum

• Context Driven

• SDTM

• TestOps

• Automation 2.0

• Cloud Computing

• SoLoMo

• Outsourcing

• Crowd testing

• Changed role

TMapDevOpsCreativeTolerant

CMMI

Outsourcing

SaaS

BYODWaterfall

Telecom

Logistics

Reporting

CI

CD

SBTM

Virtua

lization

IT-general

Networks

Infrastructure

PRINCE2

IREB

DistributedBusiness/RQMS

Development

Operations

TechnicalAutomation

Cloud

SoLoMo

Security

ProgrammingAutomation

Testing

Development

SocialTeam player

Communication

CraftsmanshipTesting

Test management

CHANGED ROLE

ProcessTheorySchedule

ManualIndependentDisciplineStructureStandards

PeopleSkills

ChangeAutomatedCollaboration

SolutionFlexibilityCreative

towards

Optimizing

Then Today Tomorrow

S

tructu

re

Optimizing our contribution

Process

TheorySchedule

DisciplineStructure

Standards

People

Change

Collaboration

Flexibility

Creative

TMap

ISTQB

ISO/IEC 29119

CDT

RST

Exploratory Testing

SBTM

Agile Scrum

Skills

Explicit

Tacit

www.polteq.com

martin.pol@polteq.com

The economy continues to challengeJointly optimizing the craft remains top requirement