CSCE 181Test Automation at Schlumberger

Glen Accardo, HTC Test Automation LeadJohn Wanke, HQ Test Automation Engineer

Agenda

Introduction Overview of Schlumberger A footnote about your degree program What is automated testing, and why do it? Challenges in test automation

Glen Accardo

Started at Schlumberger in 2005 Tested drilling, real-time, and production applications Currently I manage the test automation function at

the Houston Technology Center

Glen Accardo

Studied at University of Southwestern Louisiana– Computer Science, Mathematics, English

Held the following positions before Schlumberger– Technical writer– Web master/application developer– Developer/Tester for database performance

monitoring/tuning, and backup/recovery http://www.linkedin.com/pub/glen-accardo/0/905/904

Agenda

Introduction Overview of Schlumberger A footnote about your degree program What is automated testing, and why do it? Challenges in test automation

What is Schlumberger? The leading provider of

technology and project management to the upstream oil and gas industry for over 80 years

Over 110,000 employees, 140 nationalities in 80 countries

25 research & engineering facilities worldwide-2010 invested $919 million in R&D

The only true multinational corporation in the composition of its workforce, management team and Board of Directors

Agenda

Introduction Overview of Schlumberger A footnote about your degree program What is automated testing, and why do it? Challenges in test automation

Anyone Recognize This Phrase?

…a sound preparation in science, mathematics, English, statistics and computing.

Program Overview

The computer science curriculum is designed to prepare students to enter the rapidly expanding computer field. It is based upon the Institute of Electrical and Electronic Engineer Computer Society and the Association for Computing Machinery recommendations for curricula and courses. The four-year undergraduate curriculum in computer science includes a sound preparation in science, mathematics, English, statistics and computing. Students majoring in computer science at Texas A&M will take a 12-hour area of concentration in another department. Electives are available in the areas of: algorithms, languages and computability, software systems, networking, computer systems and architecture, artificial intelligence and cognitive modeling, vision, graphics and robotics and computational science and engineering. Advanced undergraduates may take graduate courses in these areas.

Why is English in the List?

Nearly all jobs require “Excellent communications skills” Employers will probably see your writing before your

programming—laziness or ineptitude in one will be applied to the other

Not just for technical writers Not just for a job interview Clear writing will distinguish you from people with equal

technical skills

Agenda

Introduction Overview of Schlumberger A footnote about your degree program What is automated testing, and why do it? Challenges in test automation

What is Testing?

“Testing is an infinite process of comparing the invisible to the ambiguous in order to avoid the unthinkable

happening to the anonymous.”

- James Bach

Problem Number 1

Development Effort

Testing Effort

Incremental

Cumulative

Which Leads To:- Risks for customers- Risks for product team- Increased support cost

Debate the finish criteria:- Remove features- Extend deadlines- Add resources

Take short cuts:- Don’t test features- Don’t test configurations- Don’t do non-functional tests

Solution: Add Test Automation

Test Automation Effort

What if we eliminate repetitive tests?

Development Effort

We can refocus manual test efforts on important, new testing.

Manual Test Effort

What Must Automated Tests Do?

Mimic behavior of a user or application in an environment

Validate the results ofthis behavior

Maintain or improve upon standards of otherforms of testing.

What is Test Automation Not?

Not record and playback Not a cheap alternative

to other methods of testing

Example

The TestComplete version of “Hello World.”

Impact?

Application exists (is installed?) Application starts without error Application accepts input without error Application saves file without error Application exits without error

Basic Metrics

Metric ValueTime to Develop 15 minutes

Time to run 16 seconds

Time to run manually 2 minutes

Metric ValuePerformance Improvement 86%

Can run 7.5x as often

Cost Cost of computer for 16 seconds vs.Cost of person for 2 minutes

Expandability Can run simultaneously on any number of configurations

Repeatability Near perfect

Problem Number 2

Question: “Should add feature X?”Answer: “Hmmm.”

Question: “Should we support configuration X?”Answer: “Yes.”

Testing Effort Grows on Multiple Dimensions

Features

Configurations

Data Sets

Two Part Solution

All Pairs Testing http://en.wikipedia.org/wiki/All-pairs_testing

Run automated tests in parallel

WebCam of HTC Automation

Impact

Product Automated Tests Equivalent Manual TestsStudio 20 hours * 4 platforms 8 person weeks

Matira 5 hours * 3 platforms 4 person weeks

Tupai 3 hours * 9 platforms 5 person weeks

PIPESIM 40 hours * 5 platforms ?????

Seabed 4 platforms * 1 hour 1 person week

OFM TBD TBD

Merak TBD TBD

MEP TBD TBD

IAM TBD TBD

Problem Number 3

Agenda

Introduction Overview of Schlumberger A footnote about your degree program What is automated testing, and why do it? Challenges in test automation

John Wanke

Started at Schlumberger in 2010 Tested economics, asset management, and enterprise planning

applications Currently I lead the test automation effort for the Production

Solutions group (services-based products)

John Wanke Studied at Wichita State University

– Electrical Engineering, Mathematics Held the following positions before Schlumberger

– Lead Systems Analyst / Software Engineer– Lab Systems Administrator– Web master/application developer– I/T Manager– Software Quality Assurance Engineer/Developer for

message queueing administration and monitoring http://www.linkedin.com/in/johnwanke

Management, Marketing, and Machiavelli

Well-known

Established

Easily understood

Simple operation

Alien and foreign

Newcomer

Complex

Requires skill to use

Challenges of Test Automation:Manage ExpectationsMetric ValueTime to Develop 15 minutes

Time to run 16 seconds

Time to run manually 2 minutes

Reality MeaningDevelopment Skills Much more than “hello world”

I/T skills Computers don’t magically appear and operate

Application Knowledge Not all applications are == notepad

Automation tool skills The application doesn’t run itself, does it?

Testing skills What can be tested? How?

Collaboration Dev aware of how to test. Tester must show results.

Merak Enterprise Planning (MEP) Solution Overview

Collaborate

Provide Visualize

Generate

Calculate

Integrate

Synchronize

Quality Control Staging

area

Data Warehouse

Merak Cube

Asia B.U.

N. America B.U.

EuropeB.U.

OFMExcel DBVOLTS

External Systems

Merak Peep databases

Calc NodeCalc Node

Calc Node

Calc Node Calc

Node

Secure role-based web or

desktop access

Intelligent oil & gas dimensions

MEP Product Architecture

The Technical Challenge

“The challenge you face is gaining automated / deterministic control of your test environment.” -- Jonathan Lee

The Technical Challenge, Part 1:Known Test Platforms

Merak Enterprise Planning Test Environment

The Technical Challenge, Part 2:Known Testware

Test Automation Effort

Development Effort

Manual Test Effort

Software written to test software

Subject to same controls as any other software asset

– Versioned to be in sync with app

The Technical Challenge, Part 3:Find a Tester or Make One

Q & A