Upload
ca-technologies
View
981
Download
0
Embed Size (px)
Citation preview
Fueling DevOps with a Testing Trifecta: How the New World of Testing is Driving Agile Development
DevOps: Continuous Delivery
@Scott_D_Edwards
#CAWorld
Scott Edwards
CA Technologies
DevTest Portfolio
Session Number
2 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
© 2015 CA. All rights reserved. All trademarks referenced herein belong to their respective companies.
The content provided in this CA World 2015 presentation is intended for informational purposes only and does not form any type of
warranty. The information provided by a CA partner and/or CA customer has not been reviewed for accuracy by CA.
For Informational Purposes Only
Terms of this Presentation
3 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Abstract
In this session, see how CA’s DevTest portfolio arm your testing teams with three testing solutions built for Agile teams:
1. Test automation to create test cases right from requirements
2. Generating synthetic test data to be used on demand
3. Eliminating testing constraints by virtualizing every environment you need access to
Make testing the accelerator, not the hurdle, to fast application delivery with high quality.
Scott Edwards
CA Technologies
Director of Product Marketing
4 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Agenda
TESTING IS BROKEN
CUSTOMER SATISFACTION AT THE HEART OF DIGITAL INITIATIVES
WHY QUALITY IS CRITIAL
THE NEED FOR AN AGILE TESTING TRIFECTA
HOW TO ACT NOW
1
2
3
4
5
5 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
What is More of a Threat to You?
External Breach Regular Software Update
6 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
outage was caused by a failure during a daily system refresh“
7 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
KNIGHT TRADING … a “software update accessed outdated code that suddenly made more than
$440 MILLIONin bad trades in
30 MINUTES
BNY MELLONproblem arose after bank and 3rd party software provider pushed out an upgrade…
Affecting over
1,200 MUTUAL FUNDS
8 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Any Plan that Defers Testing To The End of a Release …
is BROKEN!
In an Agile / Continuous
Delivery World …
© 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
9 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
For Embarking on Digital InitiativesFreeForm Study, “Exploiting the Software Advantage” 2015Top Drivers
Improved Customer Satisfaction
Better Workforce Productivity
Improved Customer Experience
Improved Competitive Differentiation
Better Agility/ Speed to Market
Opening up New Markets
49%
44%
41%
38%
37%
35%
TESTING CANNOT BE AN AFTERTHOUGHT!
© 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
10 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Gatepoint Research Sept 2015147 IT Executives/Directors/Managers responded
© 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
87% Of respondents rate loss of customer satisfaction as potentially the most critical issue stemming from flubbed application delivery
11 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
What is Your Delivery Experience? What is Your Customer Experience?
12 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Any Focus On Speed…Without QualityWill Result in
CONTINUOUS DELIVERY SUICIDE
13 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Ignite Continuous DeliveryEliminate Your Development and Testing Barriers
Focus Area #1
DEVELOPER SPEED
Focus Area #2
AGILE TESTING TRIFECTA
Focus Area #3
IGNITING DEVOPS
Director of Development
KEY LAUNCH ASPECTS Service
Virtualization on Demand in Azure Marketplace
SV for Developers (vapi.ca.com)
KEY LAUNCH ASPECTS1. Test automation 2. Test data
management 3. Test constraint
removal
KEY LAUNCH ASPECTS DevOps Simulation
Game Hero Kit Benefit/ROI
Calculators
© 2015 CA. ALL RIGHTS RESERVED. FOR INTERNAL USE ONLY.
Dir. Of QA / Testing The Influencer
14 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Agile Testing Trifecta … Critical for Continuous Delivery
1Test Automation
2
Test Data Management
3Test Constraint
Removal
TODAY’S TRADITIONAL TESTING METHODS INHIBIT DEVELOPMENT SPEED AND
IMPACT QUALITY
A new testing methodology,
combined with advanced
technology, is needed
© 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
15 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Test Automation Solutions from CA1
16 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
“WHY DO SOFTWARE PROJECTS FAIL?”
Changing or poorly documented requirements*
#1 Reason
48%
*Source: IDC White Paper, sponsored by Application Developers Alliance, Application Developers Alliance Developer Insights Report: A Global Survey of Today's Developer, August 2015.
“…software developers would do well to invest more time and effort into understanding the requirements of the systems they plan to build”
Application Developers Alliance in partnership with IDC
Al HilwaIDC Program Director, Application Development Software
© 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
17 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
CA Test Case OptimizerAutomatically Create Test Cases… Right from Requirements
REAL WORLD RESULTS
Cover 100% of requirements in the smallest number of tests
80% increase in test coverage
Reduce test cycles by 30% by automatically detecting any
redundant, outdated and duplicate test cases
Define & manage changing Software Requirements, mapping them to unambiguous visual flow charts
56%Build Quality into your Software from the Start
of software defects can be traced back to ambiguities in requirements1
Financial Services CompanyReduced Test cases from
326 to just 17!
1Test Automation
18 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
CA Application Test w/ Mobile Test AutomationComplete test coverage for Agile Practices … Mobile to Mainframe
Beneath the UI … lies
the complexity to be tackled
MORE THAN GUI TESTING! Invoke and verify the behavior of each layer that makes up an application
Automate Unit, Functional, Load, Regression & Performance Testing
KEY BENEFITS
Use for API, Rest API, Mobile, Swift, and SAP testing
Presentation layer test automation for Web & Mobile UIs
Reporting and collaboration capabilities that allow team
members to execute, monitor and analyze test results
Financial Service Company Run over 400 tests on a nightly basis
QA runs 460 tests whenever development releases a build
83% reduction in defects
1Test Automation
19 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Test Data Management Solutions from CA2
20 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Why Test Data
Anonymised live data may only cover around 30% of a bank’s test coverage requirements
£800 million worth of fines to businesses breaching financial principles in 2015 (Financial Conduct Authority)
Fines and regulations by the EU's General Data Protection Regulation starting in 2016
Businesses will have to notify customers – in plain language – what information about them is collected and how it is used
GDPR fines from the Information Commissioner’s Office (ICO) currently planned for 2 per cent of a business’s global revenue (capped at €100million)
FCA planning unlimited fines
21 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
CA Test Data ManagerGenerate synthetic test data for maximum coverage
REAL WORLD RESULTS Create millions of rows of high quality data in
just minutes
Reduce infrastructure costs by up to $50k per database
Reduce the time needed to generate synthetic test data by as much as 50%
1Bender RBT
63% of data breaches caused by human error in exposing sensitive records$5.4 Million = average cost of data breach incidents
2Test Data Mgmt.
Automatic creation of realistic test data sets required for testing
Designed to Eliminate the Risk of Data Breach
Large Credit Card Company 70% reduction in time and effort for getting access to
Test Data (Months to days)
80% reduction in time spent manually searching for the ‘right data’
70% reusability of Test Data Conditions for releases
22 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
New Test Data & Test Automation Integrations
TCO w/ Rally
TCO w/ HP ALM
TDM w/ Service Virtualization
TDM w/ Mainframe
Automatically convert user stories into the smallest set of test cases with 100% coverage, linked to the right data and expected results
Import existing requirements and test cases to an unambiguous, active flowchart, de-duplicate them, and reduce test cycles by up to 30%
Generate realistic virtual data which covers the full range of possible scenarios for effective service virtualization
High Performance Masking and Test Data Generation for IMS and DB2 using native utilities and operations, and automated data profiling.
2Test Data Mgmt.
© 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
23 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Test Constraint Removal Solutions from CA3
24 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Why Virtual Services?Today’s Hurdle to Innovation? Constraints across the SDLC
Incomplete Developmen
t
Data Constraints
System Constraints
CostConstraints
DevelopmentConstraints
80%Only
56%
79%
of teams experience delays in development and QA due to unavailable dependencies
of critical dependencies are unavailable when dev and test teams need them
of teams face prohibitive restrictions, time limits or access fees on needed 3rd party services
Source: voke Market Snapshot TM Report: Service Virtualization – January 2015
3Remove Testing
Constraints
25 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Service Virtualization
the Wind tunnel of Software
Accelerate Application
Delivery
Simulate Dependent Systems
Provide 24/7 Access
Increase Developer Productivity
1 2 3
Data Constraints
System Constraints
CostConstraints
DevelopmentConstraints
How Service Virtualization Eliminates Testing Constraints
CA ServiceVirtualization
3Remove Testing
Constraints
26 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
“Shift-left” Testing
Without SV
WITH SV
Dev1
Dev2
Dev3
System Test
Integration
Performance Test
UAT
SystemDev1
Dev2
Dev1
Dev3
Dev2
Dev3 Integration Performance UAT…
Uncertain delivery schedule – defects persist until UAT
Typical composite app today waits for whole assembly to begin
With CA Service Virtualization quality effort moved earlier in the lifecycle
Reduction in SDLCFaster Delivery
“Went from finding 40% of our defects in the 3rd and final stage of testing to often finding ZERO. Service Virtualization turned this final test phase into a dress rehearsal for production.”
- AutoTrader http://bit.ly/AutoTrader-SV
3Remove Testing
Constraints
27 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
CA Continuous Application InsightRegression & API Testing … Reinvented
The CAI Difference Provides system of record behind the UI or API
Identifies dependencies between application
servers, databases and 3rd-party services
Turbocharge Your Service Virtualization
Global Technology Company Automated 3500 regression tests
Large Insurance Company Automatically generated 300-600 test cases
a day … w/ 90% coverage
Deep Insight to Diagnose Right Issues
Automatic Creation of Test &
Virtual Services
End-to-EndTransaction Discovery
3Remove Testing
Constraints
28 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Test Automation from Requirements to Production
Development
Pre
-Pro
du
ctio
nQA/Test – Goal > 80% AutomationFunctional Regression Integration Build Mobile
CA Test Case OptimizerTest Less, Cover More
Test cases optimized Deduplicate manual tests Quantify costs and time for changes
CA Service VirtualizationUnconstrained Development and Test
Simulate missing data, functions, and services. Simulate APIs and third-party systems Turn contracts into assets
Requirements
CA Application Test with Mobile Any Test, Anywhere
API functional and non-functional testing Create the test once and then run across all targets Build level reporting for simplified analysis Support for Selenium and Appium
PerformanceLoad Testing
CA Test Data ManagerThe Right Data, in the Right Place at the Right Time
Eliminate manual data creation and masking Minimize security risks by not exposing PII Get data to testers and outsourcers fast
Shift Left
Unit Testing
CA Continuous Application InsightAutomate the Automation
Automatically generate 1000s of test cases Provides system of record for what is behind the UI or API Follow the transaction across all systems and get detailed error reporting Logging and production monitoring
Test cases created and validated against requirements.
Review mobile testing needs.
Mobile and load testing via cloud providers.
29 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Your Next Steps
30 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
New Benefits Calculators
31 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
DevOps Simulation Game Experience…
Experience it @ CA World • See the Demo in Theater 3 area
Experience ½ Day Simulation Game! • Santa Clara, CA - December 3• Boulder, CO - December 10• Plano, TX - January 12• Chicago, IL – January 19• Atlanta, GA – January 26• New York, NY – February 2
Register Today at http://bit.ly/DevOpsSimXP_RS
32 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Q & A
33 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Recommended Sessions
SESSION # TITLE DATE/TIME
DO3T21S
Case Study: Nordstrom Succeeds in E-Commerce
with Innovative Performance Testing to Reduce
Cycle Times and Improve Quality
DEVOPS THEATER 3
Thursday at 11:30 am
DO4T17S
Case Study: T-Mobile’s DevOps and Continuous
Delivery Journey – Building a Foundation for a
Future Built for Agility
DEVOPS THEATER 4
Thursday at 1:00 Pm
DO3T24S
Business Transformation: Accelerate DevTest with
Microsoft AZURE™ and Service Virtualization on
Demand
DEVOPS THEATER 3
Thursday at 3:00 pm
34 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
For More Information
To learn more, please visit:
http://cainc.to/Nv2VOe
CA World ’15