View
545
Download
0
Category
Preview:
Citation preview
© 2017 - ALL RIGHTS RESERVED.
About me
Jonathon is a strategic thought leader and distinguished technology evangelist. He specialises in emerging technologies, innovation and automation, and has more than 15 years of international commercial experience within global organisations, and is currently Director of Digital Assurance at CA R&D based in Oxford in the UK.
Jonathon combines his practical experience and leadership with insights into real-world applications of the core principles and methodologies underpinning DevOps, Digital Assurance and Enterprise Digital (Smart Cities, IoT and AI). As a result, he is frequently in demand as a speaker at international conferences such as EuroSTAR, Gartner, HPE Discover, Oracle Digital Forum, STAREast, STARWest, and Unicom. Jonathon is the author of several award winning books on test automation, as well as numerous podcasts, training courses and webinars.
© 2017 - ALL RIGHTS RESERVED.
Roles Awards Thought Leadership Colleagues
Me At a Glance
© 2017 - ALL RIGHTS RESERVED.
Automation Trends 2016 – DevOps
EuroSTAR - TestOps in the Cloud – The Force Awakens(http://www.slideshare.net/Jonathon_Wright/eurostar-testops-in-the-cloud)
SoftTest (Ireland) - Digital DevOps (http://www.slideshare.net/Jonathon_Wright/softtest-ireland-digital-engineering-evolution-devops)
© 2017 - ALL RIGHTS RESERVED.
Automation Trends 2016 – Methylmercury.
BCS SIGIST – Test Automation in Hybrid Cloud(http://www.slideshare.net/Jonathon_Wright/bcs-sigist-test-automation-in-the-hybrid-cloud)
BCS SIGIST – Testing as a Service - Models(http://www.slideshare.net/Jonathon_Wright/bcs-sigist-trackjonathonwrighttestingasaservicemodels)
© 2017 - ALL RIGHTS RESERVED.
Automation Trends 2016 – Java as a Service
STAREast – Testing in the Digital Transformation Age(http://www.slideshare.net/Jonathon_Wright/stareast-testing-in-the-digital-transformation-age-54958268)
YOW! (Brisbane) – DooM – How to guide(http://www.slideshare.net/Jonathon_Wright/yow-nights-in-brisbane-doom-how-to-survive-the-digital-mothership)
© 2017 - ALL RIGHTS RESERVED.
Automation Trends 2016 – MicroContainerization
WinOps (Melbourne) – Azure Stack – Docker & MicroContainerization(http://www.slideshare.net/Jonathon_Wright/winops-workshop-tfs-2015-update-3-windows-2016-server-azure-stack)
Oracle – Enterprise of Things (EoT)(http://www.slideshare.net/Jonathon_Wright/oracle-digital-evolution-enterprise-of-things)
© 2017 - ALL RIGHTS RESERVED.
Automation Trends 2016 – Quantum Teleportation
STARWest – Think you can just test that API think again!(http://www.slideshare.net/Jonathon_Wright/starwest-think-you-can-just-test-that-api-think-again)
BCS SIGIST – Testing as a Service - Models(http://www.slideshare.net/Jonathon_Wright/bcs-sigist-test-automation-in-the-hybrid-cloud)
© 2017 - ALL RIGHTS RESERVED.
Automation Trends 2016 – Autonomics
Vivid (Sydney) Festivals of Ideas – Digital Assurance(http://www.slideshare.net/Jonathon_Wright/digital-mothership-digitalfirestormcom-digital-assurance-kjr-digital)
Vivid (Sydney) Festivals of Ideas – Digital(http://www.slideshare.net/Jonathon_Wright/vivid-sydney-festivals-of-ideas-beyond-the-buzzwords-kjr)
© 2017 - ALL RIGHTS RESERVED.
Are you ready for ‘Test Automation’?
Automation Maturity Model index (AMMi®)Assessment to provide a current automation maturity level and the target maturity level:
“Sometimes half the battle just starting companies on their
automation journey by taking them one step closer to becoming
ready for automation.”
Level 1 – Accelerating – Automated Test Lifecycle Methodology (ATLM)
Level 2 – Managed – Processes are planned, performed, and controlled
Level 3 – Sustaining – Automation process defined and validated against international standards (ISO-29119)
Level 4 – Quantified – Predictability of automation process performance
Level 5 – Optimizing – Automation process variation and statistical predictability
Source – (http://testautomationpatterns.wikispaces.com/AMMI)
The AMMi model is useful in determining the maturity provided by automation
© 2017 - ALL RIGHTS RESERVED.
Product Technology
Product Roadmap
Test Process Test Environments
Test Automation
Tools
Test Automation effectiveness
Test Automation
Planning
Test Automation
Development
Test Automation Maintenance
Organization Structure
Automation Maturity Model (AMMi)
© 2017 - ALL RIGHTS RESERVED.
First Day Automation: Test Automation Experiences
* Dorothy Graham, ’Experiences of Test Automation’, http://www.dorothygraham.co.uk/automationExperiences, ISBN-10: 0321754069, 9th Jan 2012
Which ‘Automation Tool’ (10 Years Ago)
“One of my previous roles at large investment banks. Their landscape consisted
of over 300+ solutions under test (SUT), representing 2,700+ applications under
test (AUT), across 50+ global locations with over 300 automation practitioners.”
© 2017 - ALL RIGHTS RESERVED.
https://www.automation.org.uk/downloads/cv.pdf
Which ‘Automation Tool’ (Today)
How can we move ‘Test Automation’ to the Cloud?
“Existing technologies of virtualisation, business process modelling, cloud based
test automation solutions and rapid and easy internet access allow for the
development of approaches that allow companies to order ‘Testing as a Service’
and pay only for what they use (PAYU).”
platform, technology, client, browser, version, language, test type agnostic
Executing Cloud Tests
containerization
PaaS / IaaS
natural language validated against content sensitive validation
Creating Cloud Tests service-oriented architectureBorn in the Cloud
Test Automation as a Service - Introduction
Source: The Big Picture of Test Automation: Test Trustworthiness – Alan Page, Microsoft (2012)
Test Automation as a Service - Summary
© 2017 - ALL RIGHTS RESERVED.
Test Automation in the Cloud: Case Study
http://partners.telerik.com/docs/default-source/Case-Studies/telerik-test-automation-in-the-cloud_automation-development-servicesBB95938E9FB5.pdf
© 2017 - ALL RIGHTS RESERVED.
https://dzone.com/articles/the-death-star-an-ambiguous-requirements-issue
Blogs #ExcuseFreeTesting – Death Star Requirements
Team - Confederacy of Independent Systems
Enterprise Architect – Galen Erso
Sprint 1 - Top-secret Imperial weapons
project
Sprint 2 – Raids on Jedha, Eadu and Scarif
Sprint 3 – Technical Data on Tantive IV
Sprint 4 – Battle of Toprawa
Sprint 5 – Raid on Polis Massa
Sprint 6 – Moff Kalast and the Bothan Spynet
Co
llab
ora
tio
n
“The Death Star
blueprints are not in
the main computer”
- Stormtrooper
Blueprint – Death Star
Data Storage – R2-D2
Sprint 7 – Delivery blueprints to Bail Organa
Sprint 8 – Delivery blueprints to Jan Dodonna
ARA - Release Management – General Jan Dodonna Showcases “Thermal Exhaust Port”
Pattern –Fleet of 22 X-wing, 8 Y-wing, 2 R-22 Spearhead Starfighters
Deliv
ery
Patt
ern
sR
eci
pes
Recipe - DestoryDeathStar1.0 - Continuous, Build and Delivery
Define
Design
Develop
(Plan)
Deploy
Blueprint – Stardust
Data Storage – HDD
(Scarif – Data Centre)
Digital AssuranceDigital across the Galaxy
© 2017 - ALL RIGHTS RESERVED.
Digital AssuranceDigital across the Galaxy
* AxisAgile.com.au, http://www.axisagile.com.au/resources/scrumtrooper-images/
© 2017 - ALL RIGHTS RESERVED.
Digital AssurancePocketbook – Official Launch
NEW exclusive Pocketbook authored by Paul Gerrard, eminent Test-Transformation consultant, thought leader, teacher and international conference speaker.
Order your FREE copy today from http://bit.ly/digital-assurance
© 2017 - ALL RIGHTS RESERVED.
Digital AssurancePocketbook – World Tour
TORONTOLOS ANGELESSILCON VALLEY
© 2017 - ALL RIGHTS RESERVED.Product Test Engineer
Gen Zero
(Linear)AUTSUT
Budget in pounds
MainframeLocations:2Teams:5Tool(s): 1
License
x1
History - Three Decades - Mid-Late 90s
© 2017 - ALL RIGHTS RESERVED.
TestDirector 1.511995
1995
1998 TestDirector 5
2000
Quality Centre 112010
HP Next Generation ALM (NGA)
- HP Application Lifecycle Management
- HP Agile Manager2016 - TODAY
2014
2004
2010
21 years of ALM (1995 to 2016)
History – Application Lifecycle
© 2017 - ALL RIGHTS RESERVED.Product Test Engineer
1st Gen
(Modular)AUTSUT
Locations:5Teams:10 Client ServerTool(s): 3
Licenses
10+
http://www.google.co.in/patents/US7526498
Budget in thousands
Automation Framework ArchitectCore Hours 1am to 10am (EST)
Graduate TestersCore Hours 9am to 5pm (EST)
Automation EngineersCore Hours 9am to 5pm (GMT)
History - Three Decades - Late 90s
© 2017 - ALL RIGHTS RESERVED.
Automation - Retro Edition (1994 to 2016)
Joe Colantonio - First Conference Graduate Automation Role
First IoT Device
Head of Automation
Y2K
Progression Computers
First Laptop (286SX)
First Automation Framework (1999)
© 2017 - ALL RIGHTS RESERVED.
History - Automation Pyramid
Automated tests can run fast and frequently, which is cost-effective for software products with a long maintenance life
Automated tests run faster and efficiently,due to reused modules within different tests
Automated regression tests ensure the continuous system stability and functionality after changes to the solution under test
Frequent repetition of the same or similar test cases is laborious and time consuming to perform manually
Optimization of Speed, Efficiency, Quality and the Decrease of Costs
UI
API
Unit
© 2017 - ALL RIGHTS RESERVED.Automation Consultant
2nd Gen
(Data)AUTSUT
Locations:5Teams:15 Client ServerTool(s): 5
Budget ¼ million
Licenses
x25
Automation Consultants (UK)Core Hours 9am to 5pm (GMT)
Development (2 week sprints)Core Hours 9am to 5pm (GMT+1)
History - Three Decades - Early 00s
History - FrAgile (UI)
Shift from physical (UI)
Identity Services
Connectivity (HTTPS)
DOM(HTML5)
BIG Data(Database)
External Services
Workflows(Logic)
• OCR / Image• Object Models• XPATH• Regular
Expressions• Descriptive
Programming• Fuzzy Logic• Social Intelligence• Artificial
Intelligence
* Jonathon Wright or Wilson Marr, Hybrid Pattern, https://en.wikipedia.org/wiki/Hybrid_testing
© 2017 - ALL RIGHTS RESERVED.
Performance Testing - JMeter
Stuart Moncrieff, http://www.myloadtest.com/tools/
© 2017 - ALL RIGHTS RESERVED.Automation Framework Architect
3rd Gen
(Keyword)SUT
Locations:20+Teams:100+Tool(s): 15+
Budget in millions
Web
AUT
Automation Consultants (UK)Core Hours 9am to 5pm (GMT)
Automation Consultants (NY)Core Hours 9am to 5pm (EST)
Automation Consultants 9am to 5pm (UTC+8)
Development Capability (US)Core Hours 9am to 5pm
Development Capability (EUR)Core Hours 9am to 5pm
Licenses
100+
History - Three Decades - Late 00s
© 2017 - ALL RIGHTS RESERVED.
History - Automation Pyramid (API)
Focus shift from physical (UI) to logical (API) / messaging layer / database layer
SOA(API)
AJAX
CORBA
AMF
HTTP
SOAP
REST
Database(s)
ODBC
Ole DB
ADO .NET
JDBC
Java
Native
Messaging
ESB
AMQP
MSMQ
MQ
JMS
Tibco
© 2017 - ALL RIGHTS RESERVED.Global Head of Automation
4th Gen
(Hybrid)
Locations:25+Teams:50+Tool(s): 20+
Budget in multi millions
AUTSUT
Cloud
Automation Consultants (NY)Core Hours 9am to 5pm (EST)
Automation Consultants (UK)Core Hours 9am to 5pm (GMT)
Automation Consultants 9am to 5pm (UTC+8)
Development Capability (EUR)Core Hours 9am to 5pm
Licenses
PAYU
History - Three Decades - Early 10s
© 2017 - ALL RIGHTS RESERVED.
Application
Services
Data
Model
Model
Test Data
Services (SOA)
Application
Overlaying all layers
Legacy Distributed and Mainframe - Layered approach
Continuous Deployment
Supporting Future Test Automation
WCF
AJAX1%GWT0%WPF1%
Infragistics0%Flex1%
Web 2.0 (ExtJS, GWT)1%
FIX2%
SWIFT3%
Java/Java Swing6%
SAP0%
XML Web Services including SOAP4%
JMS4%
IBM MQ4%
TIBCO RV1%
UNIX Shell5%
Oracle Forms1%Peoplesoft0%
Oracle UIX1%
PL/SQL/Database6%
Java API5%
C++ API1%
.Net API2%
Informatica0%MF COBAL0%
TCP/IP2%
File comparison5%
Database <-> File comparison2%
Database <-> Database comparison2%
Jasper2%
Kondor+ 0%DBCheque0%SSR1%E30%WEBLOGIC0%Documentum0%oracle GoldenGate0%db-Cris1%DB Fastcollect0%Adobe lifecycle0%iBatis1%Smart Client1%
JSP1%
SPRING1%HIBERNATE1%
SOLARIS1%
Oracle 10G1%JSTL0%MF COBOL0%
AIX1%
Pega 6.10%Jboss0%Linux1%Net 3.51%
4.0 WPF1%RMDS1%INFRAGISTICS1%JDBC1%RMI1%dbAnalytic0%COHERENCE CACHE0%CORBA API0%SunOS0%STARR HUI0%HP/UX0%TOMCAT0%APACHE0%
JMS1%
HADOOP0%ICMP0%PERL0%XML0%VB SCRIPTING0%TIBCORV0%FRAPI0%
J2EE1%
Tibco messaging1%
REST0%
MSSQL 20050%Sybase0%SQL 2014IIS0%Hessian0%
STRUTS 20%
SITE MESH0%
LOG4J0%
EHCACHE0%
Other36%
Sample – 2,738 applications under test (AUTs)
History – CI and B(UI)ilds
© 2017 - ALL RIGHTS RESERVED.Global Head of Automation
5th Gen
(TaaaS)
Locations:100sTeams:1k+Tool(s): 50+
Budget 78 million
Cloud
SUT AUT
Automation Consultants (NY)Core Hours 9am to 5pm (EST)
Automation Consultants 9am to 5pm (UTC+8)
wA
gile
Team
s (A
sia)
Co
re H
ou
rs 9
am t
o 5
pm
(U
TC+
8)
wAgile Teams (US)Core Hours 9am to 5pm (EST)
wAgile Teams (EUR)Core Hours 9am to 5pm (CET)
Automation Consultants (UK)Core Hours 9am to 5pm (GMT)
History - Three Decades - Late 10s
© 2017 - ALL RIGHTS RESERVED.
Automation
Applications
Environment
Release
Services
Data
Model
UX Design/Mock-ups
Model
Test Data
Microservices
Release Management
Environment Management
Applications Code
Automation
Continuous Delivery - Layered approach
Continuous Delivery
CRMWPF4%
JAVA16%
Forms
Win32
JS2%Web2.0
ExtJSHTML5
ASP
GWT4% VB6
MFC
SAP
Qt
UNIXSILVERLIGHT
2%SWIFT/FIX
FLEX5%
PEOPLESOFTC++2%C#2%J2EE
.NETAIR
jQuery2%
Other34%
History – Continuous Delivery
41%
33%
6%
20%Automation (UI)
No UI
Unsuitable
Unsupported
Sample – 200 solutions under test (SUTs)
https://youtu.be/ql8ANg9p-ao
© 2017 - ALL RIGHTS RESERVED.
Blogs #ExcuseFreeTesting – Legacy is your Legacy!
https://dzone.com/articles/legacy-is-our-legacy-1
© 2017 - ALL RIGHTS RESERVED.
Design QA/TESTDEV PRODUCTIONPRE-PROD
Specification
CI/Build Functional testing UAT Integration
testing Performance engineering
Deploy to pre-prod
Code commit SCM
Design Spec
Requirements
Business Users
TESTData
TESTStub
TESTData
User BA
Product Manager
CustomerUsers
CustomerExperience
CustomerExperience
CustomerExperience
CustomerExperience
Design
Business Value
Business Value
Business Value
Improvement
Innovation
Intelligence
Assessment
Insight
Learning
Maintenance
Operations
Delivery
Testing
Support
Cu
sto
me
r
Value-Driven
Value Insight
Learn Learn
Monitor Monitor TESTStub
REALUsers
TESTData
REALData
TESTStub
USERCase
Bu
sin
ess
DevOps: Shift-X
idea
© 2017 - ALL RIGHTS RESERVED.
TestOps BUILD then TEST vs. DevOps BUILD then Deploy?
De-coupling
Solutions (SOA)
Continuous
Build, Integration
& Delivery
Good Coding
PracticesConfigurability &
Maintainability
Ops Hub
Infra as Code
Extend QA
(Ops)
Agile
Engineering
(Shift Left)
Agile
Engineering
(Shift Right)
APMBlueprints Recipes PatternsARABlueprints Recipes PatternsArD SV
TestOps (Test-Dev) with Automated Testing DevOps (Dev-Test) with Automated Deployment
Ops
Monitoring
(Testing)
Monitoring
(Ops)
Modelling
(Testing)
Modelling
(Ops)
Dev Test
© 2017 - ALL RIGHTS RESERVED.
QualityGate™ Development
6-10 secs – Check-in / Analyse / Commit (GIT)
0-6 secs – Minor UX change (Dev to TestOps) 10-40 secs – Predictive Testing (ArD)
QualityGate™ Map
Confidence Level 74%
40-50 secs – Predictive Deploy (Automic)
50-60 secs – Predictive Release (RA)
6% chance to break build
66% Quality Gate Map (confidence rate)
88% Quality Gate Map (confidence rate)
© 2017 - ALL RIGHTS RESERVED.
QualityGate™ Map
Confidence Level 69%
40-50 secs – Predictive Testing (ArD)1-5 secs – Smart Analysis (MVC)
5-10 secs – Smart Build (RA) 30-40 secs – Smart Testing (VS 2017)
10-20 secs – Smart Asset Cube (TAaaS.net)
QualityGate™ Testing
© 2017 - ALL RIGHTS RESERVED.
All media links:Domino’s Pizza orders Hitachi Consulting to advise on DevOps pushHitachi to boost Domino’s Pizza delivery with DevOps strategyHitachi Consulting and Domino’s Pizza collaborate for a slice of DevOps actionDomino’s signs up to Hitachi Consulting’s DevOps expertiseDomino’s continues to deliver with a DevOps strategy
Jonathon Wright said: "With DevOps, there is a strong focus on continuous testing and continuous delivery. This means there is more emphasis on testing much earlier in the solution delivery lifecycle. "Domino's Pizza has embraced continuous integration and testing to help the software delivery team rapidly build, test, and deliver secure, high-quality applications."
DevOps: in the wild
© 2017 - ALL RIGHTS RESERVED.
“The physical nature of wearables
technology makes it vital to testing using
real devices…. helps to mitigate
syncing/connectivity issues.”
Wearables – First Day Automation
© 2017 - ALL RIGHTS RESERVED.
Release Automation
Infrastructure Provisioning
Configuration Management
Application Platforms / PaaS
vSphere
Version Control / Artifact
Repository
Planning & Requirements
Change Management
Test Automation
DatabasesCI/Build Automation
DevOps: Release Automation
© 2017 - ALL RIGHTS RESERVED.
Digital Experiences (DX) Book Release 2017
‘Why?’ are our Digital Experience so uniquely different? could it be related to our various levels of Digital Proficiency*?:
Digital Natives - Comfortable in engaging all Digital Channels (i.e. generation x, y & z). Digital Naïve – Belief that they are Digital Natives based on ownership of technology (i.e. millennials / Xbox generation). Digital Designer – Build Digital Technology around their lifestyles and living space. Digital Savvy – Self-taught and aware of the Digital Capabilities of technology. Digital Citizen – Someone born into Digital Lifestyle and consumes without questions. Digital Immigrates – Crossed the chasm into the Digital World, forced into engagement with Digital Channels. Digital Voyeurs – Recognized the shift to digital but not ready to be active yet themselves. Digital Holdouts – Resisting the shift to digital, ignoring the impact. Digital Disengaged – Understand Digital Technology but avoiding engagement on purpose, often for privacy reason. Digital Dangerous – Extremely overconfident in their Digital Capabilities to the point they can cause disruption or damage.
So ‘Why?’ do we develop a single purpose endpoint (i.e. UI) to support all Digital Personas?When even the same object instance (i.e. UI-matrix) viewed by the same categorisation of
Digital Persona could be interpreted completely differently.
* Disrupting Digital, R Wang, 2016, Constellation Research
“The typical mobile user touches his or her phone 2,617 time every day, over an average of 145mins duration.”
© 2017 - ALL RIGHTS RESERVED.
Internet of Things – Testing Smart Cities
* Oracle Digital Forum - Digital Evolution: Enterprise of Things http://social-innovation.hitachi.com/en/copenhagens-big-energy-plans/
© 2017 - ALL RIGHTS RESERVED.
Internet of Everything: Sensors
* http://www.fastcompany.com/3051578/elasticity/hitachi-says-it-can-predict-crimes-before-they-happen
© 2017 - ALL RIGHTS RESERVED.
Microconterization
88mb
1gb
16mb
128mb
Windows Nano Server stateless and stateful Service Fabric Microservices supports massively scalable self-healing platforms. Unikernels reduces the footprint of Microservices in Microcontainers with Docker running MirageOS or Alpine Linux.
© 2017 - ALL RIGHTS RESERVED.
Windows Server 2016 TP4 (19/11/2015) supports Windows Docker Enginenative Windows Containers accessible through the Docker client Kitematic.
Pull Docker images from the Windows Docker Engine using PowerShell v5 (DCS).
Docker on Windows
© 2017 - ALL RIGHTS RESERVED.
CO
GN
ITIV
E A
DA
PTI
VE
AD
OP
TIO
N
Manual
Continuous
EnterpriseDigital
Lean Engineering
Autonomics
BUSINESS VALUE
LEVEL 1 LEVEL 2 LEVEL 3 LEVEL 4 LEVEL 5
?
Optimized Organization
for DesignOps
Continuous Insight & Learning
Value StreamMapping
Consistency & Collaboration
Heroes & Heavy Lifting
DIG
ITA
L M
ATU
RIT
Y
• Insight-Driven (Predictive / Prescriptive)• NoOps organized (IoT-Ops/CX-Ops/Sec-Ops)• Self-healing end-to-end autonomic orchestration• Microcontainerization & Microservices enabled• Open Innovation & Connected Intelligence• Quantum Teleportation (Shift X)
Where are you?
Minimum Viable Quality
• Silos, manual handovers, waterfall process• One release/year• Monolithic apps• Long term project/resource planning• Error prone dev/test/release processes
Minimum Viable Product
Minimum Viable Delivery
Organization Change
User Story Mapping InsightBeliefs, Values,
CultureValue Behaviors Heuristics
Digital Automation
© 2017 - ALL RIGHTS RESERVED.
DIGITAL ASSURANCE
VALUE-DRIVEN DELIVERY
Cognitive Adaptive Adoption (CA-A)
1Cognitive Adaptive Learning
2Cognitive Adaptive Testing
3Cognitive Adaptive
Technology
4Cognitive AdaptiveDelivery
© 2017 - ALL RIGHTS RESERVED.
Continuous Adoption
CognitiveAdaptive Adoption
Continuous Testing
Continuous Improvement
Continuous Delivery
Continuous Intelligence
Continuous Learning
Continuous Innovation
idea
DEVELOPMENT OPERATIONS
‘Shift-Right’ Digital Ecology
‘Shift-Up’ Digital Assurance feat. Connected Intelligence
‘Shift-Left’ Digital Evolution feat. DesignOps
CUSTOMER EXPERIENCE TESTING
DesignDevTestOpsAgility across all 4 increases speed, reliability and efficiency
CognitiveAdaptive Testing
CognitiveAdaptive
Improvement
Cognitive Adaptive Delivery
CognitiveAdaptive
Intelligence
CognitiveAdaptive Learning
CognitiveAdaptive
Innovation
DESIGN
‘Shift-Down’ Digital Archaeology
© 2017 - ALL RIGHTS RESERVED.
Proactive over Reactive
Governance over
Ownership
Maturity over Mastery
Predictive over Continuous
Evolution over Revolution
Engineering over Agility
Intelligence over Things
© 2017 - ALL RIGHTS RESERVED.© 2016 CA. ALL RIGHTS RESERVED.
The Digital ArchaeologyArtificial Intelligence & Machine Learning
BRINGS TRUE ARTIFICIAL INTELLIGENCE (AI) TO LIFECYCLE VIRTUALIZATION Virtualize services without requiring any knowledge or
decoding of the service protocols
Applies a genome sequence alignment algorithm, discovers byte-level patterns in message protocols
Now virtualize a much wider range of protocols without requiring a new DPH
HIGH ACCURACY: 99.6 – 100%
Increased speed and accuracy with Entropy Weighting + Message Clustering
The more data a service observes the data, more intelligent it becomes
Perfect for performance testing where we deal with tons of data
© 2017 - ALL RIGHTS RESERVED.
‘Shift Right’ Becoming an Insight-Driven Business
The Digital Ecology
REAL TIME, BIG DATA ANALYTICS EMBEDDED THROUGHOUT OUR PORTFOLIO
AGILE MANAGEMENT AGILE OPERATIONS DEVELOPER PRODUCTS
CONTINUOUS DELIVERY MAINFRAME SECURITY
Release management dashboard API analyticsReal-time app performance
© 2017 - ALL RIGHTS RESERVED.
* CA’s Analytics Reference Architecture
Data Collectors
Dat
a A
ggre
gato
rAgents
…
Probes
Machine/Log File Data
Insight-Driven
Existing Platforms
Custom Platforms
New Platforms
Partner Platforms
Analytics Engine
Dat
a In
ject
ion
AP
I
Dat
a A
cces
s A
PI
Service
Layer
Dat
a Ex
trac
tio
nData Processing (Speed)
Data Processing (Batch)
Data Bus
Powered by Connected Intelligence technologies
Cognitive Adaptive Insight (iCA)
© 2017 - ALL RIGHTS RESERVED.
Powered by Connected Intelligence technologies
Connected Intelligence
Release Management
Environment Management
ApplicationManagement
Infrastructure Management
RepositoryManagement
Data Management
Test Data LearnProbe
Cognitive Adaptive Insight (iCA)
CA App Experience Analytics
© 2017 - ALL RIGHTS RESERVED.
T H E D I G I TA L EC O LO G Y
RUNDEPLOYTESTBUILDPLAN
INTAKE RequirementsUser StoriesRelease Plan
MODEL Import User Stories to Automatically Create, Visualize and Optimize Tests
Initiate Functional, Non-Functional and Security as Code
FUNCTIONAL Subset/Mask Test Data Create/Reserve Test Data Test Automation Library Ensure Mobile Experience
INSIGHT-DRIVEN
CONFIG/DEPLOY Provision Entire Stack Confirm Configurations Approve Changes Successfully Deploy Internal or External Cloud
INTEGRATIONRemove Constraints with Virtual Services Mobile, Web, App Server,
Middleware, Backend, MF 3rd Party Systems / API’s
CODEDevelop and Commit Code, Scan Code, Version Control, Continuous Integration. Complete Build and Initiate Release
AUTOMATION AUTOMATION AUTOMATION
Speed/Time-to-Market Quality/Availability
AUTOMATION
Cost/Financial Mix Risk/Compliance
Data Automation
CustomerExperience
NON-FUNCTIONAL Performance Engineering Simulate Backend Load Test Outlier Conditions Ensure Mobile Experience Security / Penetration
Common Goals “Big Picture” Issues
DESIGN SHIFT-LEFT SHIFT-RIGHT OPS
Node.Probe
Node.TestTDDBDD
ATDD
BPMNNimbus
Visio
Node.Data
Node.Learn
MEASURE/FEEDBACK Customer Experience Business Service View Application View Infrastructure View Dynamic Capacity Feedback loop
© 2017 - ALL RIGHTS RESERVED.
“The best aspects of these proven test approaches
demonstrate how they have evolved over the past
decade, and this echoes some of the changes towards
more lean and agile business methodologies. They are in
a constant state of evolution – just as the underpinning
technology evolves over time.”
My section in the ‘Experiences in Test Automation’ book
(Dorothy Graham & Mark Fewster) 2012.
Final Thoughts – The future of automation?
© 2017 - ALL RIGHTS RESERVED.
“Digital engineering breaks down the silos across the
entire organisation – allows everyone to get involved in
evolving digital capability: connected intelligence maps
the value steam mapping of business challenges and
goals. This is a new way of IT delivery that allows for
digital initiatives to be deployed at pace across the
organisation but in a (relatively) stable and
industrialized state – a willingness to ‘fail forward’.”
My chapter in Engineering the New Reality /
Evolving the New Reality, 23rd October 2015
.
© 2017 - ALL RIGHTS RESERVED.
Continuous Testing 101 Webcast Series
Conversations with… Erik Simmons
Software & Systems Professional nuCognitive
Episodes:Solution Thinking & Value-Driven Delivery
Ray ArellFounder and Principal Coach
nuCognitive
catechnologies.podbean.com
© 2017 - ALL RIGHTS RESERVED.
IdeasBeliefs, Values, Culture
Heuristics
Behaviors
Value
AgileLean
Solution ThinkingWebcast Series - Continuous Testing 101
Copyright © 2016 nuCognitive LLC. All Rights Reserved.
Methods and Skills
Ray Arell
© 2017 - ALL RIGHTS RESERVED.
Erik Simmons
Copyright © 2016 nuCognitive LLC. All Rights Reserved.
Value-Driven DeliveryWebcast Series - Continuous Testing 101
© 2017 - ALL RIGHTS RESERVED.
Test Automation in the Cloud: Case Study
http://partners.telerik.com/docs/default-source/Case-Studies/telerik-test-automation-in-the-cloud_automation-development-servicesBB95938E9FB5.pdf
© 2017 - ALL RIGHTS RESERVED.
BPM Export (Visio)
© 2017 - ALL RIGHTS RESERVED.
Business Process Modelling
© 2017 - ALL RIGHTS RESERVED.
Business Process Workflow
© 2017 - ALL RIGHTS RESERVED.
Business Process Testing
“verbs and nouns in the test specific language are written in natural language which using context sensitive validation.”
Logi
n.P
roce
ssAction Object Event
Enter Username Text
Enter Password Text
Press Login Button
Domain Specific Language
Act
ion
s
Ob
ject
s
Even
ts
Press | Login | Button
Nouns
Verbs
Test Specific Language (TSL)
“The test editor is designed to be low waste, it will only offer nouns and verbs based on discovered artefacts not programming objects.”
© 2017 - ALL RIGHTS RESERVED.
Business Process Data
ALTOVA XML Spy 2013 (R2 SP1)
© 2017 - ALL RIGHTS RESERVED.www.eurostarconferences.com
Reporting - Burndown
TFS Workbench
Monitor velocity & capacity
© 2017 - ALL RIGHTS RESERVED.
Continuous Deployment
© 2017 - ALL RIGHTS RESERVED.
Test Agent(s) – Telerik Art of Testing Framework
Recommended