Improve speed without rushing software out the door

Challenge for Testers: not just to be a good tester but also be able to engineer the process and take advantage of advanced automation practices.

Testing needs to ‘’shift left”: Testing is starting to be done by developers more frequently.QA professionals are still doing manual work, but they’re trying to automate the process as well.

Our Maturity Model

Key areas in Continuous Testing

Risk Assessment.Defect Casual Analysis.Code Quality Control.Traceability.Test Optimization.Service Virtualization.

Key benefits

Control risks, quality and costsDetermine current gaps in maturityFocus on the areas that matter

Assessing Your Continuous Testing Capabilities:Testing in a Continuous Delivery world

abstracta.usP6.1

TESTING MATURITY ASSESSMENT

PEOPLE

PROCESSESTECHNOLOGY

Goals / Context

LEVELS (risks, quality, costs)

Basic

Areas

Efficient Continuous

Assessment Action Plan

Tools & Methodologies

THE THREE PILLARS OF TESTING MATURITY

abstracta.usP6.1

abstracta.us

FUNCTIONALTESTS

AUTOMATEDTESTS

PERFORMANCETESTS

SECURITYTESTS

USABILITYTESTS

SOURCECODE

INCIDENTS/BUGS

TESTMANAGEMENT

ENVIRONMENT/INFRASTRUCTURE

Source code versioning

- UI automation- Unit testing with minimal coverage

- Unit testing- Automated API tests

Usability testing

BASIC TESTING EFFICIENT TESTING CONTINUOUS TESTINGAware of risks, quality

and costs are measuredRisks, quality, and

costs are controlledRisks are reduced, quality and costs are optimized

MANDATORY RECOMMENDED OPTIONAL

- Basic test management- Test planning with development

Basic tests of access control policies

- Separate dev/testingenvironments

- Few devices/browsers arecovered

- Functionality inventory (backlog)- Traceability between test cases and

features- Test cases, checklists, exploratorytesting

Bug tracking

Unit, API, and UI automated tests running continuously

CONTINUOUS INTEGRATION /CONTINUOUS DELIVERY

OWASP Top 10 pen testing

- Test design techniques- Tests are prioritized

- Testing before development- Impact analysis

Defect causal analysis

User testing

- Test data management- Full device/browser coverage- Virtual servers

- Test environment management- Containers- Service virtualization

Traceability between features, issues, and code

Performance tests running continuously

Accessibility testing

- Long-term code coveragestrategy

- Agile management- Testing and devs in one

unified team

- Client-side performance tests- System performance testing

before go-live- Reactive production monitoring

Code quality control

- Unit performance tests(during dev)

- Proactive monitoring inproduction

Automated security checks

P6.1

abstracta.us

FUNCTIONALTESTS

PERFORMANCETESTS

USABILITYTESTS

SOURCECODE

INCIDENTS/BUGS

TESTMANAGEMENT

ENVIRONMENT/INFRASTRUCTURE

BASIC TESTING EFFICIENT TESTING CONTINUOUS TESTING

· Pieces of source code get lost

· Not clear what version eachclient has, which complicatesfixing the corresponding code

· Not clear what is in eachenvironment, everyone works inshared environment

· Not sure if testing with the latest version

· Bad communication betweendevelopment & testing

· No knowledge of the state of eachincident

· No knowledge of the version inwhich an incident was fixed

· No test cycles defined

· Testing is hard, not business focused,starts late

· Takes too long to update a test case

· No knowledge of which incidentseach test case corresponds to

· Unclear what needs to be tested or when

· No record of what to test or withwhat priority

· No evidence of test executions

· No information on the qualitystatus of each version

· Incidents already solved reappear

· Getting feedback after introducinga new change takes long

· Automated tests take a long time to run

· High cost of maintenance

· Uncertainty before go-live; no ideahow the system will perform

· No control over production systemsor other environments

· No clear methodology to run teststhat simulate the expected load

· Security breaches, uncontrolledrisks or uncertainty concerning howunprotected the users are

· Users find the system difficult to use

· No evidence that the application isusable

· Users are resistant to change duetheir lack of involvement inacceptance testing

· No evidence that the application isuser-friendly

· No evidence that the application isaccessible to all

· Security standards are not met · No knowledge of how a new changeaffects security

· Need to release frequent security patches

· Performance problems are difficultto solve and are detected very late

· Unable to anticipate problems thatoccur in production

· No knowledge of how a new changeaffects performance

· Testers are bored and unmotivated,always executing the same tests

· Regression tests are executedmanually and take a long time

· Testers make mistakes when doingcheckups

· Fear and uncertainty when releasinga new feature to production

· Uncertain about how well the testsare designed

· Not clear what to test first

· No knowledge of what coverage weshould have

· Not enough time to meet theexpected coverage

· Testing starts after development,focused on detecting and reporting,not prevention

· When something changes, noknowledge of which test cases needto be executed

· Gap between development andtesting team

· No shared vision

· No knowledge of how to avoidincidents

· No knowledge of where incidentscome from

· No knowledge of which feature isaffected by a certain bug and whatline of code it relates to

· No traceability within code versions

· High technical debt levels· Maintainability problems· Lack of documentation· Dead or duplicate code· Doesn't follow best practices in

design or architecture

· Data is overwritten betweendevelopers, testers or automated tests

· There are devices that have problems

· Cannot test on all devices

· Finding bugs and solvingissues takes too long

· Integration is complex and costly

· Difficult to set up a newenvironment for a demo, test orwhatever is necessary

AUTOMATEDTESTS

SECURITYTESTS

Pains SolvedP6.1

By using continuous delivery practices, HP LaserJet Firmware team could:

Reduce overall development costs by ~40%

Increase programs under development by ~140%

Reduce development costs per program by 78%

Increase resources driving innovation by 5x

Source: Thoughtworks - The Case for Continuous Delivery.

Why Continuous Delivery?Case Study

Ready to Take the Next Step?

Contact us at hello@abstracta.us or call us +1 415 745-3678

abstracta.usP6.1