Test Environments, the Future Achilles’

HeelMichiel Vroon, Quaboo,

The Netherlands

Europe’s Premier Software Testing Event

World Forum Convention Centre, The Hague, Netherlands

WWW.QUALTECHCONFERENCES.COM

“The Future of Software Testing”

Test environment

Achilles heel of the future?

Michiel Vroon

Test environment

Sounds familiar?

• I thought we solved this defect before?

• This part worked fine some moment ago!

• Who knows the correct configuration?

• What do you mean, first production??!!

• This looks different from production!

• Do you know a user-id we can use?

• What is in the current release?

• I cannot reproduce the defect!

Some causes

• No dedicated competence

• Difficult to define requirements for test

• There are no processes, plan or strategy

• Design is not clear

• Test data is not maintained

• Too many people involved

• Responsibilities are vague

• Manageable versus flexible

• …

Some solutions

• Define single point of contact

• Set up processes

• Arrange test data

• Work with environment types

• Use master/copy

• Install shared environments

• Have an authorization strategy

Define single point of contact

• Test environment coordinator

– Responsible for setting up and maintaining environment

– Single gateway to all parties involved

– Combination of technical and testing skills

– Control function

– Communicator and project leader

– In bigger organizations:

dedicated department of

test environments

Set up processes - 1

• Define what to deliver when and by whom

• Connection with other processes: design, build,

test and implementation

• Simple process:

1. Specifying the infrastructure

2. Realising the infrastructure

3. Specifying the infrastructure intake

4. Intake of the infrastructure

5. Maintaining the infrastructure

6. Preserving the infrastructure

2 3

5

Start

4

1

6

End

Set up processes - 2

Maintaining the environment

– Configuration management

– Change management

– Release management

Wish Use

Dev Test Acc Prod

Configuration management

Change

Release

Arrange test data

• Define test data

– Production data (real or scrambled)

– Test data (regular system functions or separate)

• Naming test data

– Real names

– Functional names

– Test related names

• Maintain test data

– Cumulative

– Periodic restore

– Several versions

Work with environment types

• DTAP –Model, 4 types of environments

– Development

– Test

– Acceptance

– Production

• Not a technical solution!

• Each type has it’s own characteristics on:

– Maintenance and ownership

– Authorization

– Flexibility

Example

Local development

STenvironment

UATenvironment

Development Test Acceptance Production

Environment in project

Environment type acc. to DTAP

Central development

PATenvironment

Productionenvironment

Shadowenvironment

Use master/copy

• Principle for setting up environments

Master

CopyCopy

Copy

• What is the master?

• What is in the master?

• Who owns the master?

Install shared environments

• An end-to-end environment

• Consistent end-to-end test data

• Shared use by different projects

• Well defined user processes

• Always up and running

• Continuous maintenance

Have an authorization strategy

• How to deal with authorization in the

environment?

• Test users or real life users?

• Impersonal users (roles like “guest”) or

personal users?

The case!

ITO ProjectIntegrale Test Omgeving(Integral Test Environment)

The situation

• Dutch bank

• Department Environments

– Responsible for all test environments

Single point of contact

Department environments

Coordinator environments

Maintenance&

Exploitation

Outsourceparties

Projects

External partners

Deliveryunit

Unix

Local

Central

Test data

Processes

Current situation

• Availability environments

– Demand > supply

– Temporary solutions

• Maintainability environments

– Versions

• Test data environments

– End-to-end consistency

• Complexity

– Replacement old client systems (OLI > SIEBEL)

– Connections

The environments

OLI (25)

X (25)

Y (20)

Z (15)

A (10)

B (5)

Siebel (2)

Replacement old client systems

OLI (25)

X (25)

Y (20)

Z (15)

A (10)

B (5)

Siebel (25)

end-to-end shared environments

OLI (5)

X (5)

Y (5)

Z (5)

A (5)

B (5)

Siebel (5) Connected systemsConsistent test dataMaintainedShared useRules

Goals

• Reduction of systems

• Transparency

– You know what’s in the environment (data, software,

hardware, connections etc)

– It’s consistent

– All users are known

• Shorten setup time

– It’s already there

– Only adjustments of connections

Results

Quaboo

Ludwigstraat 39

4701 NE Roosendaal

The Netherlands

E. michiel@quaboo.eu

M. +31(0)6 21 489 775

I. www.quaboo.eu