Less05 2 e_testermodule_4

Copyright © 2008, Oracle. All rights reserved.

Playback Results Analysis

Introduction to Oracle Application Testing Suite: e-Tester

Copyright © 2008, Oracle. All rights reserved.5 - 2

Playing back your script

e-Tester executes the recorded actions.

As the script plays back, the web pages will be displayed in the browser pane.

The web pages displayed should match those recorded.

Copyright © 2008, Oracle. All rights reserved.5 - 3

Resource Validation

Resource Validation is a functional test of all the link and image references in a script (in addition to the default content comparison tests).

The validation test makes individual page requests for all image and link references to verify that no broken links or images exist.

Copyright © 2008, Oracle. All rights reserved.5 - 4

Resource Validation

By default, Resource Validation runs automatically after the playback of a script.

To manually run Resource Validation:

• Select Run/Resource Validation…

• Click the Start button

Copyright © 2008, Oracle. All rights reserved.5 - 5

Resource Validation Status Codes

All page requests return a status code from either the web server or the Windows internet communication layer

• Status Code 200 indicates that the item has been located and returned successfully (Pass).

• Status Code of 404 indicates that the item can not be located (failure).

• Status Code of 12002 indicates that a timeout has occurred during the request of the resource (Failure).

Additional error status codes can be found in the help files.

Copyright © 2008, Oracle. All rights reserved.5 - 6

Show Resource Download Statistics Resource Validation

Download statistics are available when the Show Resource Download Statistics box is checked.

This includes the download times for each resource and the file size for each image in the status column.

Copyright © 2008, Oracle. All rights reserved.5 - 7

Resource Validation

Resource Validation Option Interface

Copyright © 2008, Oracle. All rights reserved.5 - 8

The Results Report

Results Report is generated immediately following the playback of a script.

It allows the User to quickly see:

• Which pages have differences

• How many tests have been executed

• How many tests have had warnings or failures

Copyright © 2008, Oracle. All rights reserved.5 - 9

Results Report

Overall Summary

Details of The LastIteration

Error Details

Copyright © 2008, Oracle. All rights reserved.5 - 10

Tests Summary

Tests Summary provides an overview of the test results with the percentage of requests that passed, failed, or returned a warning.

Click on the warnings and failure links, in the “Tests Summary” section, to view more details.

Copyright © 2008, Oracle. All rights reserved.5 - 11

Script Details

• Script details list the download times for each page, in addition to the result status.

• Click on a page link to navigate to that page.

Some application pages may require session data and may not display.

Page Link

Copyright © 2008, Oracle. All rights reserved.5 - 12

Iteration Details

The iteration details provide the results for each playback (iteration) of the script.

Copyright © 2008, Oracle. All rights reserved.5 - 13

HTML Differences

Click the Differences link to view the differences between the recorded and played back pages.

In example shown, the HTML links can be clicked to open the recorded and tested source HTML in the browser pane.

Copyright © 2008, Oracle. All rights reserved.5 - 14

Results Report Options

Select Options/Current Script/Results/Results Report

Report Details: Allows you to modify the information included in the report.

Report Location: Allows you to modify where to save the report to, and how the report should be run and displayed.

Copyright © 2008, Oracle. All rights reserved.5 - 16

The Results Log

The results log displays the results of the visual script playback in the results pane.

By default, the results log displays all details (pass, fail and warning pages).

Copyright © 2008, Oracle. All rights reserved.5 - 17

Result Log Columns

Action: indicates the page executed

Elapsed Time: Displays the page download time

Result: Indicates whether the page has passed, failed, or generated a warning

Summary: Provides details on warnings

Copyright © 2008, Oracle. All rights reserved.5 - 18

Results Log Options

Select “Options/Current Script/Results/Results Log”

Log File Name: specifies the name that will be used when saving the results log

Logging Mode: specifies how the results log file is generated (Overwrite, Append, Do Not Log)

Report: specifies what the results log will report (Failures Only, All Details)

Copyright © 2008, Oracle. All rights reserved.5 - 20

Analyzing Results Summary

You can analyze script playback with any one of the three analysis options:

• Results Report

• Results Log

• Visual Script

Copyright © 2008, Oracle. All rights reserved.5 - 21

Analysis with the Results Report

Analyze the Script Details, in the Results Report, to identify which pages resulted in failures or warnings.

• Click on the “Differences” link to view more details about what changed.

Copyright © 2008, Oracle. All rights reserved.5 - 22

Analysis with the Results Log

Analyze the “Results” column to identify which pages encountered failures or warnings.

Look in the “Summary” column to find the details of the failure/warning.

Copyright © 2008, Oracle. All rights reserved.5 - 23

Analyze The Visual Script

Look for red flags, in the visual script, to identify differences between the recorded and played back pages.

Copyright © 2008, Oracle. All rights reserved.5 - 24

Visual Script Flags

Red Flag: Indicates that there is a difference on that node, or a node below it.

Red and Yellow Flag: Indicates that the difference on that node has been ignored.

Copyright © 2008, Oracle. All rights reserved.5 - 25

Analyzing The Visual Script

To go directly to the next node with a difference:

• Press the F3 key

• Select “Results/Find Next Failure”

To go to the previous node with a difference:

• Press the SHIFT and F3 keys together

• Select “Results/Find Previous Failure”

Copyright © 2008, Oracle. All rights reserved.5 - 26

Analyzing Text Differences

Textual content comparison test differences can be further analyzed by:• Double-clicking on the

difference node.

Textual content comparison tests include:• HTML Tests• Java Script Tests• VBScript Tests

Using “Show Differences In HTML” can quickly identify the changes in the underlying code (HTML, JavaScript or VBScript).

Copyright © 2008, Oracle. All rights reserved.5 - 27

Analyze Text Differences

Textual differences are highlighted in bright Blue and Red.• Bright Blue shows the

Originally Recorded (Master) HTML.

• Red shows the Tested HTML.

Dark Blue font represents text that is ignored by e-Tester for the content comparison tests.

Copyright © 2008, Oracle. All rights reserved.

Error “Resolution”

Introduction to Oracle Application Testing Suite: e-Tester

Copyright © 2008, Oracle. All rights reserved.5 - 29

Think About Failures/Warnings

Failures and Warnings do not always indicate that there is a problem with the application.

Differences can be caused by:

• Submitting a form using dynamic data

• Dynamic content on a web page

• Testing against a new build of the application

Determine a course of action:

• Ignore the Warning/Failure

• Accept the Warning/Failure

Copyright © 2008, Oracle. All rights reserved.5 - 30

Ignore Failures

If script playback results in differences that are due to dynamic content on the web page, then the differences can be ignored.

To ignore a Test Difference• Right Click the Different

Element• Select Ignore This Failure

Ignoring is easy but lets not be so drastic…

Copyright © 2008, Oracle. All rights reserved.5 - 31

Default Content Tests

e-Tester has built-in tests that are run, by default, during each playback of a script.

• The tests include content comparison tests on each object collection in the visual script, and resource validation tests on the images and links collections.

Default tests can be turned on and off using the “Default Test Manager.”

• The “Default Test Manager” can be modified on a page level or globally.

Copyright © 2008, Oracle. All rights reserved.5 - 32

Page Content Tests Manager

To launch the Content Tests Manager on a page level:• Right-click on a page• Select Page Content Tests

Manager…

Copyright © 2008, Oracle. All rights reserved.5 - 33

Global Content Tests Manager

To launch the Content Tests Manager globally:

• Select “Options/New Scripts (Global)/Functional Tests/Content Tests”

Copyright © 2008, Oracle. All rights reserved.5 - 34

Accept Differences

If script playback results in differences that are due to a new build of the web page, the tested page can be accepted.

Accepting tested pages will replace the recorded pages with the pages that were received during playback. This action may update the baseline for further testing.

Copyright © 2008, Oracle. All rights reserved.5 - 35

Accept Differences

To accept the differences on a page:• Right-click the page with the

differences and select Accept Tested Page

To accept all differences in the script:• Select Results/Accept All

Tested Pages

Copyright © 2008, Oracle. All rights reserved.5 - 37

Disable Failing Default Tests

To disable all the failed Default Tests on a script

• Results/Disable Failing Default Tests

Copyright © 2008, Oracle. All rights reserved.5 - 38

Review 4

1. How would you handle a warning on a default test if you knew that the it was due to development having updated that page?

2. What types of warnings would you expect on pages that are highly dynamic? How would you handle those Warnings?

3. What default tests are performed on links and images? What is being tested for each?

4. Sophie rebased her script but noticed that she still gets red flags. Why might this occur?