BCS - SIGIST - Testing as a Service - Models

@jonathon_wright

#BCS_SIGIST www.sigist.org.uk

Better TaaS

5th June 2015

[email protected]

Testing as a Service - ModelsJonathon Wright, Director, Testing, Quality, Assurance

mailto:[email protected]

2

@jonathon_wright


Hitachi - a global industry leader.Society Changes, Hitachi Transforms It.

$93.4

Billion947Companies

320,000+Employees

$3.4

Billion

Hitachi ranks 78th on the 2014 FORTUNE Global 500®.

© Hitachi Europe Ltd. 2014. All rights reserved.

3

@jonathon_wright


Hitachi – Digital Business in Europe

Information & Telecommunication Solutions Distribution Solutions

Railway Solutions Smart Cities

Service Opportunities

IT Service, Data Centre Solutions, Consulting

System Solutions

Large-scale systems, Highly reliable systems

Finger Vein Authentication

Worlds FastestElevator (40mph)

Hardware, Software

Rolling stock + maintenance

Electrical components

Signaling / Train control systems

Nuclear Power Business

Mining Equipment

Platforms

Smart eco cities

Big Data Infrastructure

Energy Consumption

@jonathon_wright

#BCS_SIGIST

Water & Natural Resources

Floating train 500kmph

4

@jonathon_wright


What is BiModal? and how is FluidIT going to affect CoreIT?

4

“the creation of new business designs

that connect people, business and things

(physical objects that are active players

and contribute to business value)

to drive revenue and efficiency. ”Gartner ® 2014

@jonathon_wright

#BCS_SIGIST

5

@jonathon_wright


Digital Enterprise - BiModal

How is FluidIT going to affect CoreIT?

Fluid IT• Agile and fluid

• Innovation, brand,

profit measures

• Think continuous

• Think days, weeks

Core IT• Reliable,

compliant, secure

• Think price/performance

• Plan and approvals driven

• Long life-cycles

6

@jonathon_wright


Digital Enterprise is not just about “disrupters”

@jonathon_wright

#BCS_SIGIST

6

7

@jonathon_wright

#BCS_SIGIST www.sigist.org.uk*Gartner, 2014, ‘Building your Digital Enterprise’ Gartner event, Poland, 15th May 2014

Digital Transformation

Disrupters

Testing in the Digital Enterprise

8

@jonathon_wright

#BCS_SIGIST www.sigist.org.uk*Gartner, 2014, ‘Building your Digital Enterprise’ Gartner event, Poland, 15th May 2014

Testing in a ‘Digital Enterprise’ Transformation Age

@jonathon_wright

#BCS_SIGISTTesting in the Digital Enterprise

@jonathon_wright

#BCS_SIGIST

‘Digitalised’ businesses are able to better adapt and exploit opportunity

Digital Enterprise Blueprint + Roadmap

9

@jonathon_wright


Testing industry is changing forever

Intel© XDK IoT in 2015

Does ‘Testing as a Function’ still have a future?

10

@jonathon_wright


Hitachi – Smart Cities - Copenhagen

Effecting Copenhagen Cleantech Cluster

City

Admin

Mobility

"Green and Comfortable

Transportation"

"Lower energy consumption"

- City of bikes

- ITS

- Mobility Management

- Copenhagen Connecting

- Buildings

- Transport

- Procurement

- Street Lighting

Energy

"Carbon-neutral city"

- Smart meter by 2020

- Wind turbines

- Biomass in power plant

- Smart Cities

@jonathon_wright

#BCS_SIGIST

Carbon Neutrality

by 2025

http://www.google.co.jp/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=UFD4tEVJyHok1M&tbnid=Ljh4nnShCmApZM:&ved=0CAUQjRw&url=http://www.northjersey.com/news/opinions/soliman_062410.html&ei=UpWEUofCJcHykQXc64GgCw&psig=AFQjCNGo-mQJtCAi9ULGHtk-chhGaBOprQ&ust=1384507029443816

http://www.google.co.jp/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=UFD4tEVJyHok1M&tbnid=Ljh4nnShCmApZM:&ved=0CAUQjRw&url=http://www.northjersey.com/news/opinions/soliman_062410.html&ei=UpWEUofCJcHykQXc64GgCw&psig=AFQjCNGo-mQJtCAi9ULGHtk-chhGaBOprQ&ust=1384507029443816

11

@jonathon_wright


Testing as a Function

Legacy Methodologies

Software Development Lifecycle

Hardware Analysis

Testing

Process

MethodologyBusiness

Software Development Lifecycle (SDLC) Model

12

@jonathon_wright


Testing industry is not ready

Support for business intelligence powered by testing ‘small data’

✔

Hitachi Content Platform Solution

During deployment of

the Hitachi Content

Platform Solution

70TB of satellite

information was

ingested into the

solution platform with

a daily growth rate of

50GB/day.

Business

Intelligence

Platforms

Business

Intelligence

Infrastructure

* http://www.hds.com/assets/pdf/hitachi-storage-solutions-at-work-nasa.pdf

http://www.hds.com/assets/pdf/hitachi-storage-solutions-at-work-nasa.pdf

13

@jonathon_wright


Testing as a Activity

Future Methodologies

Solution Delivery Lifecycle

Application

Portfolio

Business

IntegrationEnterprise

Management

Solution Delivery Lifecycle integration (SDLCi) Model

14

@jonathon_wright


Testing industry needs new models

Global challenges are not suitable for ‘Testing as a Function’ silos

Complex eco-

infrastructures

intelligent

machine-to-

machine inter-

connections

empowered by

‘Big Data’

* http://www.libelium.com/libelium-smart-world-infographic-smart-cities-internet-of-things/

http://www.libelium.com/libelium-smart-world-infographic-smart-cities-internet-of-things/

15

@jonathon_wright


New Model for Testing*

Model Based Testing (MBT)

*Paul Gerrard, 2014, ‘New Model for Testing’ discussion paper published, 6th August 2014

New Model for Testing

16

@jonathon_wright

#BCS_SIGIST www.sigist.org.uk*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014

Testing as a Service - Models

@jonathon_wright

#BCS_SIGISTTesting as a Service - Models

https://leanpub.com/taas

https://leanpub.com/taas

17

@jonathon_wright


Business Delivery Management

Model Based Design (MBD)


Business Lifecycle Management

Problem

Business Abstraction Layer

Service Abstraction Layer - Solution Delivery Lifecycle integration (SDLi)

Business Delivery Management - ‘As A Service’ layer

Test Abstraction Layer – Test Solution Delivery

Business Delivery Management - ‘Testing as a Service’ layer

Portfolio Lifecycle Management

Solution Lifecycle Management

Application Lifecycle Management

Test Abstraction Layer – Test Solution Delivery

Solution

18

@jonathon_wright



• Business Lifecycle Management

– Business on a Page (BoaP) level abstraction or encapsulation representing the core

businesses differentiators such as agility, visibility and flexibility

• Portfolio Lifecycle Management

– number of solutions that are represented by a logical grouping as part of a Business Portfolio

or split by Business Domain or Workstream

• Solution Lifecycle Management

– covers logical groupings of applications that make up the solution landscape

• Application Lifecycle Management

– refers to a single application instance



19

@jonathon_wright






Business Solution Discovery Business Solution Delivery

Business

on a Page

Business Level

Answers

Business Level

Questions

Business Level

Stories

Business

Requirements

Business Process

Validation

Business Process

Modelling

Business Domain

Knowledge

Business Process

Intelligence

Business Process

Design

Business Process

Scenarios

Business Process

Tests

Business Process

Data

Business Process

Transactions

Business User

Stories

Business Level

Keywords

Business Non-

Functional (NFRs)

Business Epic

Themes

Business Definition Layer

Business

Acceptance Criteria

Business Definition

Language

Business

Acceptance Tests

Business Process

Reporting

Business Process

Behaviours

Business Process

Components

20

@jonathon_wright





Business Solution Discovery Business Solution Delivery

Business Definition Layer

Test Abstraction Layer

Test Solution Discovery Test Solution Delivery

Test Definition Layer

Problem

Business Delivery Management (‘As A Service’ Abstraction Layer)

Service Abstraction Layer - Solution Delivery Lifecycle integration (SDLi)

*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014


Solution

Logistics

21

@jonathon_wright


Why ‘As A Service’

• Testing as a Service (TaaS)

• Software as a Service (SaaS)

• Platform as a Service (PaaS)

• Infrastructure as a Service (IaaS)

• Hardware as a Service (HaaS)

Testing in the Wild - Autonomous Vehicles

@jonathon_wright

#BCS_SIGIST *Jonathon Wright, Top Gear Track, Test Results: 1:47.63, http://youtu.be/P8JnQV0nXrc, 31st October 2014

Performance Test Results: 1:47.63

http://youtu.be/P8JnQV0nXrc

22

@jonathon_wright


Problem

Business

Customer

SolutionProblem

Definition

Business

Intelligence

(CIO, COO, CTO)

Solution

Decommission

Social

Intelligence

Business

Innovation

Social

Innovation

‘As A Service’ - Model



Business Problem, Idea or Challenge (cost, benefit, time to market)

Business Solution Delivery Quality (cheap, good, fast)

‘As A Service’

CustomerEnd-Users

23

@jonathon_wright

#BCS_SIGIST www.sigist.org.uk*Jonathon Wright, 2011, ‘Testing as a Service’ presented at BCS SIGIST, London, 5th December 2011


Service Abstraction Layer - Solution Delivery Lifecycle integration (SDLCi)


Portfolio Lifecycle Management


Solution Lifecycle ManagementBusiness Delivery Management

AdviceProblem

Business Solution

Discovery

Business

Solution DeliveryContinuous Delivery Continuous Deploymentvs.

Service Layer (Lean)

24

@jonathon_wright


Model Based Design


Shift Left

Shift Up

Shift Down

Shift Right

Test

in

Dev

Business

Acceptance

Testing

Test

Ops

User

Acceptance

Testing

Social

Enterprise

Solution Delivery Lifecycle integration (SDLi)

Continuous, Integration, Build & Deployment (CIBD)

Business

Enterprise

(CIO, COO, CTO)

Testing as a Service - Model

Testing As A

Service

25

@jonathon_wright


Introduction to Model Based Design (MBD)


Blueprint(s) Pattern(s) Recipe(s)

www.eurostarconferences.com* Test Automation Patterns, testautomationpatterns.wikispaces.com & testing-store.com

@jonathon_wright

#BCS_SIGIST

http://testautomationpatterns.wikispaces.com/

http://www.testing-store.com/

26

@jonathon_wright


Model Based Design (MBD) – Blueprint(s)


We used drawing (visual 2D art)

to represent the foundation

of basic blueprints

GEN 1

GEN 1k

“Truly wonderful, the mind of a child is.”

This demonstrates that models created from

a single entity (transcendence) only represent

ontology of the domain of discourse and

exclude the quantitative intelligence of others.

“No matter their experience, masters can still

learn from their apprentices.“

“That’s No Moon!”

In Digital Enterprise

= Business on a Page

Enterprise Collaboration is the Evolution

(not revolution) of detailed expert Blueprints

27

@jonathon_wright


Model Based Design (MBD) – Blueprint(s) – Real World Example

SAFe™ 4.0

An example

Blueprint(s) could

be any process

template that

defines the

high level

flows

“The best aspects

of these proven

test approaches

demonstrate how

they have evolved

over the past

decade, and this

echoes some of

the changes

towards more lean

and agile business

methodologies.

They are in a

constant state of

evolution – just as

the underpinning

technology evolves

over time.”

28

@jonathon_wright


Model Based Design (MBD) – Pattern(s)


Death Star Interior / Inner Chasm & Corridors

Collaboration of individual applies unique

patterns that will refine, predict and

inform the model through exploration

(enquiring and challenging) each other

Quantum Multifractality

Applying constructive patterns

from two or more point sources

(interference) to render multi-

dimensional (3D) models

GEN 2k

GEN 1080Each iteration can increase the

representation of the model by

applying and interpreting data

29

@jonathon_wright


Model Based Design (MBD) – Pattern(s) – Real World Example

Test Infrastructure as a Service (TIaaS.net) – TestOps 1.0 (VM Role Dispenser)

* Test Automation as a Service – Architecture, TAaaS.net or TestOps.net

“The best aspects

of these proven

test approaches

demonstrate how

they have evolved

over the past

decade, and this

echoes some of

the changes

towards more lean

and agile business

methodologies.

They are in a

constant state of


the underpinning

technology evolves

over time.”

“Over the last five

years we have been

developing a azure

cloud based solution to

provide true

Test Automaton as a

Service using the

hybrid approach

specifically designed

with the Hybrid Cloud

(Private/Public) in

mind to utilise the

instant scalability,

flexibility and

availability of the

Cloud.”



30

@jonathon_wright


Model Based Design (MBD) – Recipe(s)


Early recipes date back to

1600BC, IFTTT recipes

(conditional statements) =

DestroyDeathStar1.0I. All wings report in

II. Lock S-foils in attack position

III. Switch deflectors ON

IV. Keep visual scanning

V. Start attack run

VI. Move into position

VII. Stay in attack formation

VIII. Stay on target

IX. Use the FORCE

GEN 4k

• Channels are the basic

building blocks

• Triggers are the "this" part

of a recipe

• Actions are the “that” part

of the recipe

• Recipes are the

“equations” made from

Triggers and Actions

• Ingredients are the basic

data model, node, quantum

materials (dot) particles

or even atom(s)

Enterprise Gamification

represents the success criteria

e.g.

“Great shot kid that

was one in a million.”

31

@jonathon_wright


Model Based Design (MBD) – Recipe(s) – Real World Example

Test Automation as a Service (TAaaS.net) – TestOps 1.0 (>5 Years)

Select the Business Process Tests Generated Recipe

“The best aspects

of these proven

test approaches

demonstrate how

they have evolved

over the past

decade, and this

echoes some of

the changes

towards more lean

and agile business

methodologies.

They are in a

constant state of


the underpinning

technology evolves

over time.”

“Based on

business process

modelling to

create business

level keywords

written in natural

language

validated against

content sensitive

validation to

create reusable,

jigsaw pieces

representing the

Solution Under

Test (SUT).”

Test Automation as a Service

* Jonathon Wright, 2010, STARWest Track, ”Test Automation as a Service”, http://www.TaaS.net, 2nd October 2012

@jonathon_wright

#BCS_SIGIST

http://www.taas.net/

32

@jonathon_wright


Introduction to Model Driven Delivery integration (MDDi)

Agile Engineering

Core IT Fluid IT

Agile Engineering

* Adam Mann, Google Innovation Officer, ‘Creating the inter-planetary Internet protocol’, Wired.com, 6th May 2013

Open Innovation

33

@jonathon_wright


Introduction to Model Driven Delivery (MDDi) - Charter(s)

Test First Delivery (TFD) First Day Delivery (FDDi)

TDD

* Jonathon Wright, 2014, Gartner, http://www.slideshare.net/mobile/Jonathon_Wright/addi-ha-apiytestinghybridcloud

Domain Driven Design (DDD) Domain Driven Delivery (DDDi)

Transcendence Driven Delivery (TDDi)

Test First Delivery (TFD)

http://www.slideshare.net/mobile/Jonathon_Wright/addi-ha-apiytestinghybridcloud

34

@jonathon_wright


Charter – First Day Delivery (FDDi)

Model Driven Delivery (MDDi) integration

*The Advanced Research Projects Agency Network (ARPA) Network, 1969, 2 & 4 Nodes the precursor to the Internet

Business Process Modelling

Blueprint

Business Process Scenarios

Patterns

Business Process Components

RecipesNode Maps Node Learning Node Strengthening

• Model the proposed solution • Learning (Turing machine)

adapting to simulate logic

algorithms

• Validate all possible states

Smart Learning Machines

35

@jonathon_wright


Charter – Transcendence Driven Delivery (TDDi)



Patterns Node (View)

• Each Node creation (LOC)

passes requirement, story,

behaviour, acceptance test

• Each Node generates a

testable view (UI, Service,

or Data layer) of the solution

• Models Nodes and Links


Recipes NodeJS (Controller)


Blueprint Node (Model)

* Visual Studio 2015 CP6 & NodeJS, https://www.visualstudio.com/en-us/news/vs2015-vs.asp http://nodejstools.codeplex.com/

https://www.visualstudio.com/en-us/news/vs2015-vs.asp

http://nodejstools.codeplex.com/

36

@jonathon_wright


Charter – Domain Driven Delivery (DDDi)



Blueprint


Patterns


RecipesNode Discovery Node ValidationNode Systems

• Model the current solution • Validate all possible states• Apply:• Systems Dynamics

(epistemic & systemic

entropy)

• Systems Thinking

• Learn:• Ontology – formal naming,

definition of node types,

properties and

interrelationships

• Domain – universal set of

nodes (domain of discourse)

• Context – nouns/verb

discovery to enable context

sensitive validation.

* Huw Price, 2015, Grid-Tools, http://www.agile-designer.com/resources/model-based-testing-using-agile-designer/

@jonathon_wright

#BCS_SIGIST

https://www.grid-tools.com/solutions/agile-designer/

37

@jonathon_wright



Advanced Modelling Simulation Technologies

Watershed management

Pipe-network Management

Prediction of water pollution

Pipe-network analysis


Models can be analysed, visualized, and optimized

Models can provide assurance through, analysis, simulation and prediction

* http://www.computerweekly.com/news/2240233919/Societys-values-moving-from-Mono-to-Koto-says-Hitachi

http://www.computerweekly.com/news/2240233919/Societys-values-moving-from-Mono-to-Koto-says-Hitachi

38

@jonathon_wright


• Test Platform as a Service (TPaaS)

– Social Enterprise Collaboration

– Actionable Business Insight

– Enterprise Gamification

– Global Testing Marketplace

– All-Channel Customer Experience• Test Infrastructure as a Service (TIaaS)

• Test Automation as a Service (TAaaS)

• Functional Testing as a Service (FTaaS)

• Performance Testing as a Service (PTaaS)

• Mobile Testing as a Service (MTaaS)

• Security Testing as a Service (STaaS)



Test Platform as a Service (TPaaS) Model

39

@jonathon_wright


Global Testing Marketplace

Instant Scalability, Flexibility and Availability

@jonathon_wright

#BCS_SIGIST *Jonathon Wright, 2011, ‘Testing as a Service’ presented at BCS SIGIST, London, 5th December 2011


G

L

O

B

A

L

M

A

R

K

E

T

P

L

A

C

E

40

@jonathon_wright



Improved Communication, Collaboration and Mobility

*Jonathon Wright, 2012, ‘Test Automation as a Service’ at Fusion, Sydney, 10th September 2012


Test©

41

@jonathon_wright



Become a Testing Rock Star!

*http://blog.utest.com/2014/11/04/utest-platform-preview-new-dashboard-for-testers-on-paid-projects , 4th Nov 2014


@jonathon_wright

#BCS_SIGIST

http://blog.utest.com/2014/11/04/utest-platform-preview-new-dashboard-for-testers-on-paid-projects

42

@jonathon_wright


Enterprise Gamification

Service Thinking (Service as a Experience) Business Model

G

A

M

I

F

I

C

AT

I

O

N

E

N

T

E

R

P

R

I

S

E

Shift Left

Shift Up

Shift Down

Shift Right

Test

in

Dev

Business

Acceptance

Testing

Test

Ops

User

Acceptance

Testing

Social

Enterprise


* http://enterprise-gamification.com/attachments/article/233/Improving%20Software%20Quality%20with%20Gamification%20-%20Full%20Paper.pdf

Business

Enterprise

(CIO, COO, CTO)

http://enterprise-gamification.com/attachments/article/233/Improving Software Quality with Gamification - Full Paper.pdf

43

@jonathon_wright


Enterprise Collaboration in the Crowd – Real World Example

Test Intelligence 2.0 – Global Testing Insight (UX, Exploratory & Testing in the Wild)

Request1

Run2

Test3

* Real Crowd Tester, Liam Warrilow, 24th Friday 2015 @15:13 (Ramp up to 100 iWatch testers in under 24hrs)

Enterprise CrowdTesting

Testing in the Wild

Microsoft© HQ Seattle

44

@jonathon_wright


Actionable Business Insight powered by ‘Small Data’

Funnel Virtualisation

BusinessInsight

Data

Data

Data

Data

Data

Data

Data

Data

Data

Data

Data

Data

DataData

Data

Data

DataData

* Llyr Jones, 2014, Grid-Tools.com, https://twitter.com/GridTools/status/487264150578790400 , 10th July 2014


https://twitter.com/GridTools/status/487264150578790400

45

@jonathon_wright


Social Enterprise Collaboration

• Delivering true business value – Not just driving process efficiency

– Across the SDLCi with cross-functional domain tools

– Providing instant actionable business insight

• Supporting individual, team, organizational, and enterprise collaboration

– Integration with the Global Testing Marketplace (GTM) platform

– Produce high-quality reusable business assets in the cloud

– Capture specialist technical and cross-domain knowledge (i.e. DIDO)

– Game mechanics to encourage collaboration (i.e. Enterprise Gamification)

– Produce design patterns that allow business rules to be codified in business-specific meta-languages (i.e. BPMNv2.2, xPDL, XAML)

• Managing software that gets deployed everywhere

– All-Channel Customer Experience

*Jonathon Wright, 2014, ‘Agile Portfolio Management in the Cloud’ presented at Unicom, London, 20th March 2014


Embracing Agile Portfolio Management in the Cloud

46

@jonathon_wright



• Test Infrastructure as a Service (TIaaS)

– H(API)y Testing in the Cloud

– Test First Delivery

– Building bridges in the Cloud

– Cloud 2.0 beyond legacy








Test Infrastructure as a Service (TIaaS) Model

47

@jonathon_wright


Building Bridges in the Cloud

“If you build it, they will test” (IYBITWT)

*Jonathon Wright, 2014, ‘Ha(API)y testing in the hybrid-Cloud & beyond’ presented at Gartner ADM, London, May 19th 2014


48

@jonathon_wright


Cloud 2.0

Moving beyond the legacy cloud


select <IaaS>

select <PaaS>

select <SaaS>

select <TaaS>

select <TaaaS>

Cloud Recipe for Solution Under Test

Cloud Recipe (IFTTT)

CloudSystem Automation

YourTestCloud.com

“Heterogeneous test cloud that are open, flexible and extensible”

Testing Cloud

If <Solution Under Test>

Then <Build Cloud 2.0>

Testing Cloud


49

@jonathon_wright


Test First Delivery

Service / Network Virtualisation as part of Test Driven Delivery integration (TDDi)



50

@jonathon_wright


Lean = Open

Open (Digital) Innovation results in Open (Digital) Businesses

Open Digital Platforms

Lean Testing

Enterprise

Practices

BlueprintsSmart

“Sharing intellectual property such as software code or research

rules, adhering to open standards rather than creating your own,

and giving customers the freedom to easily exit your platform”

*Jonathan Rosenberg & Eric Schmidt, 2014, ‘How Google Works’, ISBN: 9781455582341, 23rd September 2014

Open Digital Enterprises

51

@jonathon_wright


Open Innovation 1.0 - Real World Example

SDLCi - Grid-Tools connected intelligence to PerfectoMobile

*Llyr Wyn Jones, ‘End-to-end vs. interface testing: the mathematics’, 31st March 2015

@jonathon_wright

#BCS_SIGIST

52

@jonathon_wright


Ha(API)y Testing in the Cloud

• Build Bridges in the Cloud

– Use Cloud Maps to Learn, Discover and Model endpoint(s)

– Create Bridges (VNet to VNet) “If you build it, they will test” (IYBITWT)

– Embrace Test Infrastructure as a Service (TIaaS) in the Hybrid Cloud (TQA™ Lab)

• Avoid Legacy Cloud

– Heterogeneous Test Infrastructure as a Service (TIaaS) that are open, flexible and extensible

– Continuous Build, Integration & Delivery in YourTestCloud

– Embrace Community Test Clouds (Collaborate/Share Recipes)

• First Day Delivery

– Enable Day-zero Performance, Penetration & Security Testing as a Service

• Support for powerful Test Automation as a Service (TAaaS)

– UI Testing that is Scalable and Cost-Efficient

– API Testing that is Repeatable, Reliable and Fast

Unlock true delivery innovation



53

@jonathon_wright





– Test Automation in the Cloud

– AMMi© Assessment Model

– First Day Automation





Test Automation as a Service (TAaaS) Model



54

@jonathon_wright


First Day Automation: Test Automation Experiences Book

* Dorothy Graham, ’Experiences of Test Automation’, http://www.dorothygraham.co.uk/automationExperiences/, ISBN-10: 0321754069, 9th January 2012


http://www.dorothygraham.co.uk/automationExperiences/

55

@jonathon_wright

#BCS_SIGIST www.sigist.org.ukhttp://partners.telerik.com/docs/default-source/Case-Studies/telerik-test-automation-in-the-cloud_automation-development-servicesBB95938E9FB5.pdf


Test Automation in the Cloud: Case Study

http://partners.telerik.com/docs/default-source/Case-Studies/telerik-test-automation-in-the-cloud_automation-development-servicesBB95938E9FB5.pdf

56

@jonathon_wright


AMMi© Assessment Model

The digital journey toward technical delivery enlightenment

AMMi Phase Target Criteria

Level 0 Accelerating Automated Test Lifecycle Methodology (ATLM)

Level 1 Traditional Framework Driven (Gen 1-5)

Level 2 Managed Processes are planned, performed, measured, and controlled

Level 3 Sustaining Automation process defined and validated against international

standards (ISO-29119)

Level 4 Quantified Predictability of automation process performance

Level 5 Optimising Automation process variation and statistical predictability

* Source: AMMi®(http://www.AMMi.org.uk) + Test Automation Patterns (http://TestAutomationPatterns.wikispaces.com)


http://www.ammi.org.uk/

http://testautomationpatterns.wikispaces.com/

57

@jonathon_wright

#BCS_SIGIST www.sigist.org.ukFunctional Testing as a Service (FTaaS) Model




• Functional Testing as a Service

– Shift everywhere (UP, DOWN, LEFT, RIGHT)

– Continuous, Integration, Build & Delivery






58

@jonathon_wright


Shift Everywhere

Test in Business, Test in Dev, Test Ops, Test in the Wild

Functional Testing as a Service (FTaaS) Model

Test in Business Test in Dev Test in the Wild

59

@jonathon_wright


Functional Testing in UFT 12 and beyond (i.e. LeanFT)

* https://www.packtpub.com/application-development/advanced-qtp-115-test-engineers-cookbook

@jonathon_wright

#BCS_SIGISTFunctional Testing as a Service (FTaaS) Model

https://www.packtpub.com/application-development/advanced-qtp-115-test-engineers-cookbook

60

@jonathon_wright






• Performance Testing as a Service

– Performance Testing in the Cloud





Performance Testing as a Service (PTaaS) Model

61

@jonathon_wright


Performance Testing in the Cloud

PTaaS – Microsoft – Visual Studio Team Foundation Service


62

@jonathon_wright


Performance Testing in the Cloud

@jonathon_wright#BCS_SIGIST Performance Testing as a Service (PTaaS) Model

PTaaS – HP - StormRunner (TrueAPI/TrueClient)

63

@jonathon_wright

#BCS_SIGIST www.sigist.org.ukMobile Testing as a Service (MTaaS) Model






• Mobile Testing as a Service

– Mobile Testing in the Cloud

– Continuous, Mobile, Build & Delivery

– High Volume Mobile (HVMAT)




64

@jonathon_wright


Mobile Testing in the Cloud

@jonathon_wright#BCS_SIGIST

Real Mobile Devices (PerfectoMobile) Test Studio for iOS (Telerik)

* Telerik.com, http://www.telerik.com/mobile-testing, 4th November 2014

Mobile Testing as a Service (MTaaS) Model

http://www.perfectomobile.com/

http://www.perfectomobile.com/

http://www.telerik.com/automated-testing-tools/ios-testing.aspx

http://www.telerik.com/automated-testing-tools/ios-testing.aspx

http://www.telerik.com/mobile-testing

65

@jonathon_wright


Continuous, Mobile, Build & Delivery

MobileCloud™

@jonathon_wright#BCS_SIGIST

* PerfectoMobile.com, http://www.perfectomobile.com/resources/papers/shift-app-testing-left-paper, 4th November 2014


http://www.perfectomobile.com/resources/papers/shift-app-testing-left-paper

66

@jonathon_wright


High Volume Mobile Automated Testing (HVMAT)

On demand in the cloud featuring Network Virtualisation

*Jonathon Wright, 2013, ‘Test Automation in the Cloud’ for EuroSTAR, Gothenburg, Sweden, 23rd July 2013

Mobile Testing as a Service (MTaaS) Model

67

@jonathon_wright








• Security Testing as a Service

– Security Testing in the Cloud



Security Testing as a Service (STaaS) Model

68

@jonathon_wright


Security Testing in the Cloud

How to prevent epistemic/systemic failure in the cloud from geo-realistic DoS attacks? Application

Security

@jonathon_wright

#BCS_SIGIST

Infrastructure

Testing

Secure Coding

Mobile Security

Database Security

Information

Security

Our services identify

configuration and

deployment flaws which

may be introduced

when integrating mobile

solutions into an

enterprise environment.

We also perform

manual security testing

and analysis of smart

phone and tablet based

apps. Typically this will

encompass both the

client application and

server side services

supporting the app.

*John Denneny, MD, http://www.pentest.co.uk/whitepapers.html , 5th November 2014

Security Testing as a Service (STaaS) Model

Mobile Security

http://www.pentest.co.uk/whitepapers.html

69

@jonathon_wright


Testing as a Service – Models - Summary

BusinessAssurance

BusinessQuality

BusinessTesting

Business Assurance provides the visibility and

transparency throughout the solution delivery lifecycle• Digital Enterprise

• Business on a Page

• Business Delivery Management

• Business Intelligence (Actionable Business Insight)

• Business Innovation (Enterprise Gamification)

Business Quality allows enforceable quality across the

entire solution delivery lifecycle owned by everyone• Business Lifecycle Management

• Business Solution Discovery / Delivery

• Business Enterprise Collaboration

• Business Acceptance Testing

Business Testing

• Testing as a Service - Models

• Solution Delivery Lifecycle integration (SLDCi)

• Test First Delivery / First Day Testing

• Continious, Integration, Build & Delivery

• Shift Everywhere (UP/DOWN/LEFT/RIGHT)

• All Channel Customer Experience

70

@jonathon_wright


Hitachi ConsultingHitachi’s Global Management Consulting & IT Services Business.

$740

MillionFY13 Revenue

6,500experienced &

dedicated

professionals

Better Approach. Better Results. Better Experience.

$10 Billion

results


Technology

BCS - SIGIST - Testing as a Service - Models