SOFTWARE TESTING CONCEPTSWith a focus on PHPUnit,

PHP_CodeCoverage, and

xDebug

INTRODUCTION

Aaron Bernstein – Aspiring Engineer One of the lead software engineers in the

Workspace Team at GoDaddy Been working with GoDaddy for about a year and a half.

Specifically tasked on our FaxThruEmail product.

Micah Wood – Aspiring Software Engineer One of the software engineers in the Workspace

Team at GoDaddy Been working with GoDaddy for about four years.

Specifically tasked on our FaxThruEmail product.

Farhan Zaman – Software Engineer in Test One of the automation engineers in the Workspace

Team at GoDaddy Been working with GoDaddy for about three and a half.

Specifically tasked on our Email Infrastructure and AIR product.

Combined Experience Includes: Fifteen plus years of programming in a

multitude of languages…

We’ve all been working towards fully covering our applications with unit, functional, integration, and browser testing over time.

WHO You, should love testing!!

ROLES

Software Engineers

Throughout the development process, one should be

performing tests on code that they author.

Test Engineers

By the nature of the title, their primary function is to

test software.

Automation Engineers

Developing tools and procedures to test software

through automated methods.

Management

Ultimately responsible for the product integrity.

WHAT Anything you code!!

LEVELS AND TYPES OF SOFTWARE TESTING

LEVELS

• Unit

• System

• Acceptance

• Integration

TYPES

• A/B

• Acceptance

• Accessibility

• Alpha/Beta

• Compatibility

• Destructive

• Development

• Internationalization and localization

• Installation

• Functional/Non-Functional

• Regression

• Smoke and Sanity

• Software Performance

• Security

• Usability

TESTING METHODS

STATIC

Reviews, walkthroughs, or inspections are referred to as static testing.

Static testing can be omitted, and in practice often is.

DYNAMIC

Executing programmed code with a given set of test cases is referred to as dynamic testing.

Dynamic testing takes place when the program itself is used.

Dynamic testing may begin before the program is 100% complete in order to test particular sections of code and are applied to discrete functions or modules.

Typical techniques for this are either using stubs/drivers or execution from a debugger environment.

WHEN As much as feasibly possible!!

PARTS OF THE PROOFING PROCESS

VERIFICATION

The process of determining whether the products of a given phase of the software development process fulfill the requirements established during the previous phase.

VALIDATION

The process of evaluating software at the end of software development to ensure compliance with intended usage.

WHERE But of course, where it fits!!

CONTINUOUS INTEGRATION AND/OR TRIGGERS

• Repository• Polling

• Package Management• At build time

• Deployments• At deployment time

• SCM Hooks• Client-Side

• This section splits them into committing workflow hooks, e-mail workflow scripts, and the rest of the client-side scripts.

• Server-Side

• These scripts run before and after pushes to the server.

* all of which can be quantified and automated.

THROUGH CONTINUOUS INTEGRATION METHODS

WHY Even the best programmers, can

and do, make mistakes!!

COST OF NOT TESTING OR LATE TESTING

Testing is the most time consuming and expensive part of software development.

Not testing is even more expensive.

If we have too little testing effort early, the cost of testing increases exponentially.

Planning for testing after development is prohibitively expensive.

MAIN FUNCTIONS OF TESTING

• Improve quality

• Reduce cost

• Preserve customer satisfaction

HOW Bunch of sugar and spice!!

(Magic and Fairy Dust)

CODE QUALITY AND THE C.R.A.P. INDEX

The C.R.A.P. (Change Risk Analysis and Predictions) index is designed to analyze and predict the amount of effort, pain, and time required to maintain an existing body of code.

The other factor used in calculating the C.R.A.P index, is the number of logic “paths” found within the code. The more individual paths found, the harder the code is to

maintain and the higher the index will be.

Given a Java method m, C.R.A.P.(m) = comp(m)^2 * (1 – cov(m)/100)^3 + comp(m)

90% of everything is crap.

~Sturgeon’s Law (one of many variants)

…software metrics, in general, are just tools. No single metric can tell the whole story; it’s just one more data point.

Metrics are meant to be used by developers, not the other way around – the metric should work for you, you should not have to work for the metric.

Metrics should never be an end unto themselves.

Metrics are meant to help you think, not to do the thinking for you.

~Alberto Savoia

TOOLS PHPUnit, PHP_CodeCoverage,

and xDebug

PHPUNIT

Testing with PHPUnit is not a totally different activity from what you should already be doing. It is just a different way of doing it.

The difference is between testing:

• Checking that your program behaves as expected,

• Performing a battery of tests, runnable code-fragments that automatically test the correctness of parts (units) of the software.

These runnable code-fragments are called unit tests.

Current Requirements

PHPUnit 3.7 requires PHP 5.3.3 (or later), but PHP 5.5.1 (or later) is highly recommended.

PHP_CodeCoverage, the library that is used by PHPUnit to collect and process code coverage information, depends on Xdebug 2.0.5 (or later), but Xdebug 2.2.3 (or later) is highly recommended.

PHPUNIT TESTING Demonstration

THANK YOU! QUESTIONS?

RESOURCES

Software Testing

PHPUnit

xDebug

Clover PHP Plugin for Jenkins

Clover Documentation

Pardon My French, But This Code is C.R.A.P. (1) (2)

How to Read and Improve the C.R.A.P. Index of Your Code

CONTACT INFORMATION

Provide feel free to contact us at the following addresses.

m: abernstein@godaddy.comgd_github: abernsteintwitter: @bernstein_aaron

m: fzaman@godaddy.comgd_github: fzamantwitter: @farhan_zaman

m: mwood@godaddy.comgd_github: mwood