SOASTA CloudTest Release Notes Build 5623cdn.soasta.com/productresource/download/SOASTA... · SOASTA CloudTest™ Release Notes Build 5623.175 Aug 10, 2012 Table of Contents Build

SOASTA CloudTest™ Release Notes Build 5623.175 Aug 10, 2012

Table of Contents

Build 5623.175......................................................................................................1 Bugs Fixed ........................................................................................................1

Build 5623.170......................................................................................................2 Bugs Fixed ........................................................................................................2

Build 5623.167......................................................................................................3 Bugs Fixed ........................................................................................................3

Build 5623.155......................................................................................................4 Bugs Fixed ........................................................................................................4

Build 5623.153......................................................................................................5 Bugs Fixed ........................................................................................................5

Build 5623.141......................................................................................................6 Bugs Fixed ........................................................................................................6

Build 5623.138......................................................................................................7 Bugs Fixed ........................................................................................................7

Build 5623.132......................................................................................................8 Bugs Fixed ........................................................................................................8

Build 5623.110....................................................................................................10 Test Creation...................................................................................................10

OpenGL App Support and AppInternal Value Accessor..............................10 Test Execution.................................................................................................11

MakeAppTouchTestable Preview mode......................................................11 URL Tracking Enhancements......................................................................12

Bugs Fixed ......................................................................................................13 Build 5623.77......................................................................................................16

Bugs Fixed ......................................................................................................16 Build 5623.74......................................................................................................18

Test Creation...................................................................................................18 Support for Client Side Certificates for SOAP..........................................18

Copyright 2012. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

Bugs Fixed ......................................................................................................20 Build 5623.70......................................................................................................21

Bugs Fixed ......................................................................................................21 Build 5623.60......................................................................................................23

Test Creation...................................................................................................23 Random String Generation ISSE.............................................................23 Image Comparison and Validation (verifyScreenshot) for Mobile ............25

Test Execution.................................................................................................27 Support for Client Side Certificates for HTTP ..........................................27 Random String Generation ISSE Type ....................................................27 String-type field........................................................................................28 String-length field.....................................................................................28

System Utilities Object ....................................................................................29 Bugs Fixed ......................................................................................................30

Build 5623.53......................................................................................................32 Bugs Fixed ......................................................................................................32

Build 5623.42......................................................................................................34 Bugs Fixed ......................................................................................................34

Build 5623.39......................................................................................................36 Bugs Fixed ......................................................................................................36

Build 5623.36......................................................................................................38 Test Creation...................................................................................................38

TouchTest™ Release..................................................................................38 Welcome Page, Find Out How ...................................................................39 Clip Editor, Info Windows ............................................................................40

Info Window Tabs ....................................................................................41 General.................................................................................................41 Inputs....................................................................................................42 Waits.....................................................................................................43


Defining a Wait Using the Info Window.................................................43 Outputs .................................................................................................44 Define an Output using the Info Window ..............................................44 Validations ............................................................................................45 Define a Validation using the Info Window ...........................................45 Property Sets........................................................................................46

Property Chooser and ISSE Editor..............................................................47 Property Chooser Tabs............................................................................48

Seed Data Object Tab ..........................................................................48 ISSE Editor ..............................................................................................50

Using Seed Data Structs to Iterate For Each...............................................51 Read Only Mode for Compositions and Clips..............................................57

Test Execution.................................................................................................58 Verified Servers...........................................................................................58

Bugs Fixed ......................................................................................................59 Build 5564.70......................................................................................................65

Test Creation...................................................................................................65 Support for Adobe HDS bootstrap files .......................................................65

Build 5554.60......................................................................................................66 Bugs Fixed ......................................................................................................66

Build 5554.56......................................................................................................67 Bugs Fixed ......................................................................................................67

Build 5554.55......................................................................................................68 Bugs Fixed ......................................................................................................68

Build 5554.53......................................................................................................69 Bugs Fixed ......................................................................................................69

Build 5554.47......................................................................................................70 Bugs Fixed ......................................................................................................70

Build 5554.44......................................................................................................71


Bugs Fixed ......................................................................................................71 Build 5554.41......................................................................................................72

Bugs Fixed ......................................................................................................72 Build 5554.36......................................................................................................73

Bugs Fixed ......................................................................................................73 Build 5554.19......................................................................................................75

Bugs Fixed ......................................................................................................75 Build 5554.10......................................................................................................76

Bugs Fixed ......................................................................................................76 Build 5554.3........................................................................................................78

Test Creation...................................................................................................78 Change in behavior for HTML parsing affects XPath ..................................78

Test Execution.................................................................................................79 Cloud Provider Account Support for Rackspace (CloudTest Pro) ...............79

Setup Rackspace (OpenStack) as a Cloud Provider ...............................79 Async Play Start Improvements ..................................................................82

Bugs Fixed ......................................................................................................83 Build 5483.23......................................................................................................87

Bugs Fixed ......................................................................................................87 Build 5483.3........................................................................................................91

Test Creation...................................................................................................91 Introducing TouchTest™ .............................................................................91 Welcome Page Improvements ...................................................................92 Double Click to Open ..................................................................................93

Double Click to Open in Result Details ....................................................93 Double Click to Open in Analysis Widgets...............................................96 Double Click to Open in Composition Editor ............................................98 Double Click to Open in Clip Editor..........................................................98

User-Friendly Display of Properties & ISSEs...............................................99


Clip Editor Improvements ..........................................................................102 Using the New Record Toolbar Button ..................................................102 Browser Recording in the New Clip Editor .............................................104

Record Browser (new window) ...........................................................105 Record Browser (inline frame) ............................................................106

Open in Composition Button Now a Drop-Down....................................108 Open in Test Composition ..................................................................108 Play in Test Composition ....................................................................109 Debug in Test Composition ................................................................109

List View Improvements ............................................................................110 List View Repeats ..................................................................................110

Breakpoints in List View Gutter...........................................................112 Renamed Column Headings...............................................................112

Composition Editor Improvements ............................................................114

Locations in a TouchTest™ ...................................................................115 Locations in an HTTP(S) Test................................................................116 Track Properties, General Tab...............................................................117

Default Track, General Tab Properties ...............................................118 Adding Multiple Locations or Distributions to a Track................................119

Adding Locations To a Single Distribution .............................................120 Adding Locations To Distinct Distributions.............................................121 Changes to Repeat Method Drop-Down................................................121

Using Repeats with Multiple Track Locations ............................................123 Seed Data Improvements..........................................................................125

Tying Seed Data to a Virtual User(s) .....................................................126 Server Allocation....................................................................................127

Separate Slice Distribution .................................................................127 Single Slice Distribution......................................................................128

Data Section ..........................................................................................128


Column and Row Count......................................................................128 Row Traversal Types..........................................................................129 EOF Action .........................................................................................129 Row Representation ...........................................................................130

Assignment of seed data values to individual properties .......................131 Seed Data Result Events.......................................................................131

Multiple Custom Properties.................................................................132 Mobile App Targets .......................................................................................133

Creating a Mobile App Target ................................................................133 Test Execution ..................................................................................................136

Tenant Recognition ...................................................................................136 Test Analytics....................................................................................................138

Result Details Improvements.....................................................................138 General ..................................................................................................141

Composition, Band, Track, and Clip Level..........................................141 Target Level........................................................................................141 Message / Action Level.......................................................................142 Waits and Validations .........................................................................142

Custom Properties .................................................................................142 Input.......................................................................................................143

Target Demographics .........................................................................145 Waterfall Chart ..........................................................................................146

Creating a Custom Waterfall Dashboard ............................................147 Miscellaneous................................................................................................148

Automatic Update Improvements for EC2 .................................................148 Jenkins Plug-in Error Message Improvements ..........................................148 Support Browsers......................................................................................148

Bugs Fixed ....................................................................................................149

Build 5623.175

Bugs Fixed 37493: Clicking on Embedded Script edit control for If-Then-Else enters drag & drop mode Drag-and-drop mode was enabled within the Embedded Script field in an If-Then-Else container without any apparent way to exit it. 36766: Error when sorting servers by tenant Sorting by the Tenant column in the Central, Server list resulted in this error. 36559: Invalid white space character (0x2) in text to output The Session Template Wizard didn't recursively search inside mult-level containers. 36179: WSDL Parsing After recording a .NET site with web services, this null pointer exception error would occur while using the Convert Recording to Clip wizard.


1

Build 5623.170

Bugs Fixed 37473: Add new dashboard to a result, UI is shifted too far down The Dashboard Wizard, which appears after a user clicks the Plus (+) icon to add a new dashboard was lower in display than it had been in releases prior to Veyron, thus forcing the user to constantly scroll down. 37338: java.lang.reflect.InvocationTargetException This error occurred during the Target creation process after a user entered HTTP request header. 37281: Conductor doesn't parse URL pasted from browser properly This error occurred after a user pasted a CloudTest URL from the browser into Conductor preferences. 34854: Update included report template in CloudTest This release includes an update report template.


2

Build 5623.167

Bugs Fixed 37362: Create Grid from Composition doesn't work with multiple locations The Grid Wizard failed to properly allocate the number of servers by location for the given composition when multiple locations were defined. 37360: SOAP fault trying to accept the End User License Agreement This fault occurred on a tenant that had a trailing space in its name. Trailing spaces are not trimmed from tenant names. 36910: Error Analysis shows a validation error as an "Unknown Error" A validation error that was verifying an address unexpectedly displayed as an “Unknown Error” when it appeared in the Error Analysis widget. 36888: Mobile monitors stop before result is finished The mobile monitor would stop before the test composition had completed in some cases. Monitors will now close faster when there is a persist going on when the monitor tries to close. May make a better fix later but this one is simple and 36792: Exception when creating new Seed Data object The issue was actually that while editing a seed data cell and then clicking the add column button, the click spawns an insert column server call, then an update cell call immediately after that (double-clicking is not the cause, but it would only make things worse). There is no server or client synchronization, so there was a chance (~25%) that the server calls would modify the seed data repository object at the same time and a race condition would occur. The solution was to implement a client side queue that forces there to only be one outstanding server call per seed data object at any time, then when that call comes back the client will pick the next call from the queue and run it.


3

Build 5623.155

Bugs Fixed Maestro: Don't show unloaded Instances in Player Status As of this release, the Central tab, Player Status list will no longer show unloaded server instances. Maestro: Fix a null pointer exception that sometimes showed up in Player Status displays In some cases, a null pointer exception would occur when the Player Status was in display.


4

Build 5623.153

Bugs Fixed 36787: WebUI test including built in actions that fail clips Pre-action “Waitforpagetoloads” were being added without the default timeout parameter while browser recording. 36615: XSD validation failure while trying to save app action An attempt to save a clip containing an app action with a custom property resulted in this error. 36559: Invalid white space character (0x2) in text to output Pasted unicode characters that cannot be expressed in XML as startMarker or endMarker would cause this error in the Session Template Wizard, SubString Parser. 36366: Had to reregister device with CloudTest Lite This device mismatch error would occur in some cases when using a MakeAppTouchTestable-enabled mobile app to test after updating a CloudTest Lite environment where the given Device Agent was previously registered.


5

Build 5623.141

Bugs Fixed 36366: Had to re-register device with CloudTest Lite A Device Mismatch error would occur after updating CloudTest Lite to a newer build because the Coordinator wouldn’t cache the Device Agent repository object. This fix also includes a fix to the Coordinator regression involved in this case. 36319: Session Template Wizard does not search messages inside Transactions The Session Template Wizard wouldn’t find correlations for pages or messages inside a Transaction container.


6

Build 5623.138

Bugs Fixed 36040: Coordinator prevents TouchTest Agent login The Coordinator unexpectedly prevented TouchTest Agent login in CloudTest Lite environments.

35601: Error handling for querying external APIs

This error would occur in some cases when viewing the CloudTest Welcome page if a problem occurred with an external API for a resource on the page (such as a video or CloudLink document). 34609: About page doesn't handle contact errors This error occurred in some cases when accessing the About page, Updates tab. A user-friendly error will now display if underlying errors occur. 36191: this.m_oResultsViewer.m_oWidgetLogic.canvas is undefined; JS line 296 This error occurred while loading the Result Details dashboard.


7

Build 5623.132

Bugs Fixed 36145: Error when converting a clip This convert to clip error would occur when converting a multipart message that had an ampersand in the header. 35963: Maximum Image upload size in CloudTest Lite The CloudTest Lite edition unexpectedly had a maximum 15kb for image uploads that prevented successful upload of mobile app and other icons. This maximum size has been reset to 5 mb as expected. 35932: New activity map does not maximize Pushing the 'Maximize' pop out icon in the upper right of the Activity Map widget expanded the border but unexpectedly failed to expand the map. 35929: iOS 6 Beta 3 causes version mismatch

A version mismatch error would post from TouchTest Agent running under iOS 6 Beta 3 although recording worked as expected. The TouchTest Driver now uses firmware data as part of the encryption/decryption key. 35587: Attempt to run compile-and-go script on a cleared scope; JS line 2294 An error occurred in the Clip Editor’s lower panel App Action Editor. An additional check has been added in ErrorReporting.handleCaughtScriptError. 35581: Title bar adds to name Clicking on the Split button and/or creating a filter to change the Location shown in the Avg Response Time chart would append the action to the name 35481: this.m_oWidgetLogic is undefined; JS line 678 This error would occur while loading the Result Details dashboard. 35263: Change Account Name to User Name when adding a new user The User Editor box, Account Name field has been renamed to User Name for this release in order to be consistent with the same field label in the Central > Users list, General tab, as well as in Account Setings.


8

34993: Unable to evaluate substitution specification The ISSE Editor created an invalid ISSE resulting in this error. 34854: Update included report template in CloudTest An update to the default CloudTest report template is included in this release. 32663: List view uses wrong default icon for Mobile Apps For iOS mobile apps that had no custom icon defined in CloudTest, the iOS default image was not shown in Clip Editor list view as expected. 31838: All edition checksums should be sent to the client during updates Updates now use conditional compilation to resolve which checksum to.


9

Build 5623.110

Test Creation

OpenGL App Support and AppInternal Value Accessor CloudTest® Mobile can now access the values in an OpenGL apps, as well as other apps that hide their internal states, and use thoe exposed internal values to validate, as well as to conditionally wait for internal application state changes. Extending your project source code to use the AppInternal Value Accessor requires the usual steps to make your app TouchTestable, along with some additional ones described in the video and tutorial that accompany this release. The techniques used to expose OpenGL app internal states are also applicable to any application internal state to which the developer has rights.

• View the ZeroToTest 4 – Internal App Values video (also available on the Welcome Page in Central) to get started.

• Additionally, we have provided the SOASTA TouchTest™ OpenGL Tutorial for a comprehensive developer-level introduction to this new feature set.


10

http://link.brightcove.com/services/player/bcpid996446970001?bctid=1732491286001

http://cdn.soasta.com/productresource/download/SOASTA_TouchTest_OpenGL_Tutorial.pdf

http://cdn.soasta.com/productresource/download/SOASTA_TouchTest_OpenGL_Tutorial.pdf

Test Execution

MakeAppTouchTestable Preview mode The MakeAppTouchTestable utility now includes an optional Preview mode parameter. By using this parameter, the Xcode developer can preview which files will be changed by the utility, as well detect errors in using the command. Typical errors indicated by Preview mode include username/password errors, incorrect Xcode project paths, etc. In order to use Preview mode, download the latest MakeAppTouchTestable utility from the CloudTest Welcome page, Downloads section. From Terminal, insert the Preview mode parameter, -previewmode into the usual MakeAppTouchTestable syntax (shown below): java -jar MakeAppTouchTestable.jar -project ~/Documents/MyApp/MyApp.xcodeproj -target MyApp -previewmode -url http://ctserver/concerto -username bob -password pass

Note: Refer to the TouchTest™ Developer Guide for a complete discussion of MakeAppTouchTestable and other developer issues.

For example, the following Console output from Preview mode: Utility will now run in preview mode.

Configuring Target in Xcode Project...

Adding URL Types to Info Plist file of Automation Target...

Performing Source code modifications to link TouchTestDriver with User Application...

Will configure following Main File: /Users/user1/Documents/Demo/iLabyrinth/iOS/main.m

Will configure following AppDelegate File: /Users/user1/Documents/Demo/iLabyrinth/iOS/Classes/UDAppDelegate.m


11

http://cdn.soasta.com/productresource/download/SOASTA_TouchTest_Developer_Guide.pdf

URL Tracking Enhancements Additional composition-level URL Tracking settings have been added in the Composition Editor, Composition properties, Play Settings tab.

The following URL Tracking options are now seen in the URL Tracking drop-down:

• Full URL for Known Assets – This setting keeps statistics for asset URLs, which are those that end with an extension (e.g. "xxx.png", "yyy.css" or with a "/"). Other URLs are grouped together using the name Non-Asset under the domain. Use this setting when Full URL Analysis is desired for Pages but there are some URLs that have SessionIDs or other unique identifiers at the end of the URL. This will make performance and memory usage better than the "Full URL Analysis" choice but less than the "Host Name Only" choice.

• Full URL Analysis - Statistics are tracked by URL for every URL that is sent. Query parameters are ignored.

• Protocol and Host Names Only - Statistics are tracked for the protocol (HTTP, HTTPS) and the domain only. This setting will help performance and memory usage on larger tests where metrics are not needed past the domain level.


12

Bugs Fixed 35376: Buttons are not playing back correctly The tap gesture in a given app was not recording an offset and on playback the tap was accordingly wrong. 35251: Signed Conductor for Mountain Lion downloads The SOASTA Conductor wouldn't auto-populate data for users when run under the Mac OS X 10.8 (Mountain Lion) version. 35246: outputElementValue should return UIProgressView status When used with a UIProgressView control, the "elementValue" output/wait/validation now returns the value of the "progress" property. 35245: outputElementValue should return UISwitch status When used with a UISlider control, the "elementValue" output/wait/validation now returns the value of the "isOn" property. 35222: Automatic unload of idle secondary Maestro instances not handled properly A recent change caused the Maestro notification mechanism for automatically unloaded server instances to fail to appear in cases where such unloaded servers existed, thus causing a composition to play on a single server without notifying the user. These notifications, which remind the user that Maestro instances are unloaded after 30 minutes idle, have been restored. 35026: Dynamic Resource Cache not working or missing items that should be cached The Page Resource Cache was looking up based upon the exact original text of the resource URL. In this case, it was causing problems because some of the links in this HTML contained a superfluous port 80, which would not match the URL as returned by the communication layers which stripped out the unnecessary port 80 on send. It is now caching based upon a more general key involving the separate protocol, host, port, and file portions parsed from the URL. 34866: appInternalValue in Result Details missing labels and values Labels and values for appInternalValue were missing in the Result Details’ widget’s detail sections.


13

34864: UI issues in Clip Element, Info Window A number of issues with the Info window were observed, including: 1) the window didn’t resize after changing the Wait tab’s Command drop-down; 2) labels would overlap in some cases, and; 3) the incorrect width was being given to labels that are drop-downs on load. 34849: Exception when -proxypassword is not passed in to MATT utility

An exception would occur in the MakeAppTouchTestable utility if all proxy settings were entered but –proxypassword was left blank. Setting a null property is not allowed and we receive an exception.34848: Proxy parameters are not recognized when passed to MATT utility Entering the supported MakeAppTouchTestable arguments for proxy resulted in this failure. 34625 : Don't try to retrieve waterfall data without a result selected. An error occurred in a Waterfall dashboard where no result was selected. 34572: Exception: unexpected token: ( near line 1, column 413 The error analysis had a "WHERE" in it that caused omission of the WHERE in the generated SQL. 34545: Uncaught TypeError: Cannot read property 'style' of null; JS line 1238 A toggle was expanded before its container was in place, resulting in this error. 34429: Multiple OpenGL screenshots crash the app Extensive use of screenshots in a mobile app test would inadvertently crash an OpenGL app. 34421: Null pointer exception while changing tenants This error occurred when a user whose privileges included tenant, but not user administrative rights, attempted to change tenants. Now, checking the User Editor dialog box, Tenant Administrator box will automatically also check the User Administrator box, if it is not already checked. 34404: Composition Tree keeps grayed out when running it again The Debugging dashboard, Composition Tree widget wouldn't refresh (to become active again) when re-playing a composition in debugging mode. 34403: Display better error message when files are read-only If an Xcode project’s main.m or AppDelegate file(s) were read-only, the MakeAppTouchTestable utility would throw this exception. Now, the utility will display a user-friendly message.


14

34145: Screenshots no longer sized with lower Result Details panel Lower panel screenshots in the Result Details widget unexpectedly failed to resize with the panel. 33937: java.lang.NullPointerException This null pointer exception occurred when a special character was encountered. 33794: Script error in Debugger An error occurred when the boolean data type for System Properties were called in the Property Inspector. 33530: Null pointer exception running MakeAppTouchTestable This null pointer exception, which could occur in some cases when using the utility more than once on the same project, was introduced in Build 5623.77 with the fix for Case 32779. 32997: A result with ID '0' does not exist in the Results Server database This error occurred in the dashboard when a dimension value was chosen while no result was selected. 32437: OpenGL-ES precision recording does not accurately capture some events Playback of a test clip that contained a sequence of long press pans would sometimes fail because of an app action without the touch duration data necessary for successful playback. 29903: OpenStack error when trying to tear down already-deleted servers This error would occur when tearing down a server that had already been deleted externally.


15

Build 5623.77

Bugs Fixed 34422: Access to raw transactional data used in Waterfall chart As of this release, new Waterfall-related times such as Offset and DNS Lookup Duration, have been added to the export format for Clip Element Details and Clip Element Aggregate. The Offset is relative to the start of the container that was also already in the export. 34223: Changing certificate of an existing SOAP target does not work correctly An attempt to change the client-side certificate for an existing SOAP with a WSDL URL target resulted in this error. 33963: JavaScript error while trying to save a WebUI clip After manually adding a generic browser action to an unsaved clip, clicking the Locator Tool to set the location, and then saving and naming the new clip, this error would result after clicking OK.

33955: Proxy option on the username/login screen This error would occur in some cases where a prior proxy configuration existed that was no longer communicative when a CloudTest Lite user attempted an upgrade using the About box mechanism provided. For example, the user might be on a different network or some other proxy details may have changed that were out of the user’s control. As of this release, CloudTest Lite will allow proxy settings to be updated from the Login page. In the event of this or a similar error, CloudTest will present an "Update proxy settings" link for the user to provide updated proxy information. 33934: Problem importing exported file This fix augments the prior fix reported with Build 5623.74 (Veyron 20).


16


17

Build 5623.74

Test Creation

Support for Client Side Certificates for SOAP This release includes support for Client Side Certification Authentication support for SOAP/WSDL targets. Refer to Target for instructions about creating different target types in CloudTest, including SOAP/WSDL targets. Client Side Certificates can be configured for SOAP targets during target creation using the Define the WSDL Target step of the Target wizard.

The certificate is uploaded when the target is created.


18

Client Side Certificates can be configured for SOAP targets in the Target Editor when the top-level service node is selected, via the new Authentication and Client Side Certficate sections.

To use this feature,

• Enter the HTTP authentication credentials (for the given server).

• Click Choose File and browse to specify the certificate file.

• Provide a certificate password for the certification account. After the target is configured, the certificate is persisted in the target XML. For example, when the Record button is clicked to create a recording that uses this target, the certificate file is persisted in the recorded XML along with the host list.


19

Bugs Fixed 34053: No open start element, when trying to write end element This error occurred while editing an App Action in the Clip Editor lower panel. 33976: readFromURL call in the Script is stuck and not timing out New optional parameters "connectionTimeout" and "readTimeout" have been added to the "readFromURL" method on $util (and $context) to allow specification of the timeouts in milliseconds. If the parameters are not present, timeouts of ten minutes are used. 33934: Problem importing exported file In some circumstances, exporting a mobile test would then fail on import. 33919: Custom properties not being shown in ISSE editor A custom property of type Seed Data: Repository, which referred to a Seed Data object whose name had the special character “&”, didn’t appear in the ISSE Editor as expected. 33654: Cannot Generate a Report - 120 sec timeout consistently An attempt to generate a report from a large test that included the Errors By Type chart would silently timeout when the maximum number of series was reached. Now, CloudTest will count the number of series to show an error stating when the “max number of series” has been exceeded. 29538: Double-clicking Delay icon should open Edit mode In prior releases, it was necessary to precisely click the number of the Delay clip element in Icon mode to invoke edit mode (i.e. to change the Delay interval) as expected. Now, any part of the icon can be clicked.


20

Build 5623.70

Bugs Fixed DNS Option NOT getting saved in Target Setting the DNS Options section in the Target Editor would unexpectedly fail to save those options. 33944: Errors in randomization calculationss for ISSE and $util random string generators There was a small error in the randomization process when generating random strings for the ISSE and Scripting $util random string generation.Under some circumstances this could cause duplicate "random" strings to be generated in multi-server Compositions. 33870: Default target content should not include optional verifyScreenshot parameter The default target unexpectedly included the optional verifyScreenshot parameters for tolerance and delay. 33860: Negative time being shown for min response time The minimum response time unexpectedly displayed as a negative number in the Fundamentals widget. 33602: Image Validations fails on a given app action Touch events are now sent on the touched view rather than the view returned by the prior view. 33597: Auto complete breaks image validations Auto correction and auto capitalization in UITextView would break image comparison and validation. Auto correction is now deactivated when verifyScreenshot is enabled. 33527: Fundamentals can continue to show growing elapsed time Due to its use of underlying calcuations, an older result displaying in the Fundamentals widget could in some rare cases continue to show the Elapsed Time as growing although the test was long since completed. Now, the


21

Fundamentals widget uses the composition Status Indicator to determine if the test is still running. 33490: A Soap fault has occurred. faultString: The XML body of this repository object does not match the expected schema This error would occur in the Action Editor while adding validations that used a script. 33191: outputSelectedSegment accessor for validating UISegmentedControls A new outputSelectedSegment has been added for use in getting the state of an iOS button in a UISegmentedControl.


22

Build 5623.60

Test Creation

Random String Generation ISSE The ISSE Editor includes an additional Random Value tab for this release.

Random strings permit the substitution of a given ISSE expression by a randomly generated string.

Use the String type drop-down to specify a random string type for the given context from among the following:

• Alphanumeric A random alphanumeric string is to be substituted, consisting of the characters A-Z, a-z, and 0-9.

• Alphabetic A random alphabetic string is to be substituted, consisting of the characters A-Z and a-z.

• Decimal digits A random string of decimal digits is to be substituted, consisting of the characters 0-9.


23

• Hex digits A random string of hex digits is to be substituted, consisting of the characters 0-9 and A-F.

Random strings permit the substitution of a given ISSE expression by a randomly generated string.

Once placed into a given clip element, the ISSE for the random value appears in the Clip Editor lower panel.


24

Image Comparison and Validation (verifyScreenshot) for Mobile As of this release, CloudTest supports image comparison and validation via the new verifyScreenshot validation.

.

• To enable Screenshot Validations, open the mobile target and select Target Info, then scan down to locate the Screenshot Validation. Check Add verify screenshot during recording to enable Screenshot Validation in all app actions based on the given target.

• Optionally, set a value other than the default 300 ms for the screenshot delay.

• Define the tolerance level for the given target or action by entering a tolerance level. Tolerance is the percentage of the screen allowed to be different while comparing the images for the validation to succeed. For example, when setting the tolerance level in a clip with small differences in times—setting tolerance to 0 will result in a failure

For example, when setting the tolerance level in a clip with small differences in times—setting tolerance to 0 will result in a failure


25

and setting it to 1 will result in success.


26

Test Execution

Support for Client Side Certificates for HTTP This release includes support for Client Side Certification Authentication support within HTTP targets. Client Side Certifications can be configured for HTTP targets in the Target Editor, via the new Authentication and Client Side Certficate sections.

To use this feature,

• Enter the HTTP Authentication credentials (for the given web server).

• Click Choose File and browse to specify the certificate file.

• Provide a certificate password for the certification account.

After the target is configured, the certificate is persisted in the recording’s XML Record button is clicked the certificate file is uploaded to the server and is persisted in the recorded XML along with the host list.

Random String Generation ISSE Type A new type of In Situ Substitute Expression type that permits the substitution of randomly generated strings is included in this release. Copyright 2012. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

27

This form of ISSE causes a random string value to be substituted in place of the ISSE and has the:

{%% random : string-type, string-length %%}

or {%% randomvalue : string-type, string-length %%}

Spaces are not significant. The “random” and “randomvalue” keywords may be in any mixture of upper and/or lower case.

String-type field This fields specifies the type of random string to be generated (the set of characters from which the string is to be generated). It must be one of the following values, in any combination of upper and/or lower case:

• “alphanumeric”

A random alphanumeric string is to be substituted, consisting of the characters A-Z, a-z, and 0-9.

• “alpha”

A random alphabetic string is to be substituted, consisting of the characters A-Z and a-z.

• “decimaldigits”

A random string of decimal digits is to be substituted, consisting of the characters 0-9.

• “hexdigits”

A random string of hex digits is to be substituted, consisting of the characters 0-9 and A-F.

String-length field This field specifies the length of the random string to be generated (the number of characters). It must be an integer value greater than zero. Examples

{%% random : alphanumeric, 100 %%}

{%% randomvalue: hexdigits,5 %%} {%% random, encoding=base64: alphanumeric,100 %%}

{%% RandomValue: DecimalDigits, 15 %%}


28

System Utilities Object A new “SystemUtilities” ($util) object has been added to the SOASTA CloudTest object model for this release. This object contains methods for generating random strings, and for encoding/decoding strings into and from Base64. Script authors should also note that the readFromURL method has been moved from $context to the new $util. For backwards compatibility, this method remains callable from $context. Refer to the following Script Reference API pages for additional details relevant to this release: http://cdn.soasta.com/productresource/download/jsDoc/latest/symbols/_global_.html

http://cdn.soasta.com/productresource/download/jsDoc/latest/symbols/SystemUtilities.html


29

http://cdn.soasta.com/productresource/download/jsDoc/latest/symbols/_global_.html

http://cdn.soasta.com/productresource/download/jsDoc/latest/symbols/SystemUtilities.html

Bugs Fixed TouchTest™ Jenkins CI Tutorial now available A new TouchTest™ Jenkins CI Tutorial has been added to the Welcome page, Documentation & Samples section as of this release. 33544: Info window extends and contracts when open while recording The issue was that if post-actions waits/validations happened in a certain order, then the Info window was being reset to its default height before resizing back to the correct height to fit the validaiton/waits. The fix was to use the old height only when we refresh the popup, default otherwise. 33542: Cannot call method 'deferredSwitchContent' of null This error occurred while clicking the Info (gear) icon while mobile app recording was underway. 33525: Import/Export: The XML document is not well-formed An attempt to export an older result that was created prior to the introduction of DNS statistics used in Waterfall charts would fail. This affected exports only. 33505: ISSE toggle icon seems to be broken Entering an ISSE into a field where one is shown, and then clicking the blue ISSE button, would expand the ISSE box to expert mode, however, the expected content wasn’t displayed. Additionally, entering an invalid ISSE syntax, such as {%% sss:sss %%}, would result in a script error once the toggle was clicked. 33196: Info Window, Validations tab reloads 3x per recorded action The Info Window, Validations tab would unexpectedly reload each time a post- action wait or validation came in after the initial action. This also fixes Case 33057. 33091: Chrome freezes during recording if Info Window is shown In some cases while recording with the Info window feature shown, Google Chrome would freeze. Now, the clip is only saved when the clip element’s Info window is dirty. 32873: Null error in Result Details This null error occurred when the Result Details widget incorrectly expected a validation to have an (ISSE) expression. Since validations do not require such an expression, the Result Details widget no longer expects them. 29894: Names are incorrectly lower-cased in validation failure text Conductor names were unexpectedly lower-cased in result text posted to the Result Details, Waits and Validations section. Copyright 2012. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

30

http://cdn.soasta.com/productresource/download/SOASTA_TouchTest_Jenkins_Tutorial.pdf

29479: Conductor un-install leaves previous settings behind The Conductor’s Windows uninstall routine will now offer the user a choice to remove all Conductor settings.


31

Build 5623.53

Bugs Fixed TouchTest™ Advanced Tutorial now available A new TouchTest™ Advanced Tutorial has been added to the Welcome page, Documentation & Samples section as of this release. 33252: Clip unexpectedly crashes In some cases, playback of a very short pan action would cause TouchTestDriver to crash. 33065: Cannot read property 'imageId' of null An error occurred in the Choose Device Agent and Mobile App box while clicking the Device Agent in the list. 33010 : Imported app objects don't show in Central right away An imported mobile app object didn’t appear in the Central > Library unless the view was refreshed. 32779: MakeAppTouchTestable utility corrupts project files The MakeAppTouchTestable utility would corrupt a given project if that project already had a folder named “include.” 32711: Strange behavior for Locator (Validations) in List Mode This error would occur when clicking the Locator button to close Locator mode in a browser action clip. 32710: Clip Editor > Validations > Locator button opens Info window Instead of opening the page in the clip editor in order to locate the element of the webpage to target, clicking the Locator button opened the Info window for a given the browser action. 32672: Clip Validation UI bug When creating a validation, hitting escape while the Command box is selected unexpectedly moves the box. After which, leaving the window and then returning to it results in an empty options list in the Command box.


32

http://cdn.soasta.com/productresource/download/SOASTA_TouchTest_Advanced_Tutorial.pdf

32664: Renaming mobile app in lower panel does not work An attempt to rename a mobile app in the Central > Mobile Apps list, General tab, Mobile App name field unexpectedly failed. 32650: Need to make fewer status polling requests for Rackspace Calls to the Rackspace “get server status” are now changed from 5 to 20 seconds (or 3 calls per minute) per CloudTest server. Additionally, parallel requests are no longer in use and the “allocation block size” setting is ignored for Rackspace as of this release. 32605: Report Generation Failing Report generation failed on dual instance CloudTest environments. 32436: OpenGL-ES simple app playback does not work correctly Playback of an OpenGL-ES simple app test clip didn’t accurately reflect what was recorded (i.e. the ‘pan’ of a square doesn’t move on playback). 32086: When creating a new Clip Custom Property manually, there is a character added in the value type Constant While creating a Custom Clip Property manually (i.e. using the green Plus icon in the Properties > Clip Custom Properties node of the Clip Editor lower panel) unexpectedly resulted in a leading character being inserted. 30639: WebUI: Element Locator does not open preview at selected action's location When using the Element Locator, the preview panel that opens did not open at the location of the currently selected action. Instead, it opened at the URL defined in the recording target. This resulted in an inability to find/locate an element on the page for which a given action corresponded.


33

Build 5623.42

Bugs Fixed 32950: NPE in Convert to Clip wizard An error occurred while matching targets in the convert to clip wizard. Additional null checking has been added to detect any further occurrences. 32949: Failure while preparing to send Message (NPE) An error occurred attempting to playback XML messages. Fixes a bug introduced in the change to support Adobe HDS recorded streams. The bug caused a null pointer error on all requests with Content-Type text/xml. 32943: ISSE Editor - adding new custom properties using Seed Data When adding a new custom property using the ISSE Editor, the event handlers didn't get attached, which prevented changes to the given property. 32791: Play in Composition launches the app and then backgrounds it right away When the TouchTest Agent called for a getTasks poll right after launching the app it received no tasks and thinks its safe to terminate all. Because of this, their session is terminated as soon as the app is launched. Now, the agent does not try to terminate sessions if it has launched the app within the last 5 seconds. 32788: Privacy Policy element records different locators depending on if you tap on the front or end of it Certain locators, for elements inside of a custom UIButton, did not work as expected. 32857: Struct/Array Issue with Veyron 14 This fix augments the prior fix in Build 5632.40. 32199: Delete Button Doesn't Work Using UITableView edit controls to delete a cell in the table did not actually delete the cell. Also, using a swipe gesture on a UITableViewCell didn't show the delete button on the cell.


34

32193: Seed Data Editor is not working A deep packet inspection service used as part of a firewall prevented a third-party library that looks like an external URL and chopped the file off after that point. This fix splits up the URL-looking string into parts in an effort to pass packet inspection.


35

Build 5623.39

Bugs Fixed 32857: Struct/Array Issue with Veyron 14 When a change to an array was saved in a custom property that referenced seed data, a struct was displayed in its place when the clip was reopened. 32830: Welcome Page links opened in place Some links in the Welcome page, Find Out How section didn’t open in a new target window or tab as expected. 32829: JavaScript error in Result Details after clicking an action This "onToggleScreenshotSize" JavaScript error would occur after clicking an item in the Result Details, Cover Flow after play had finished. 32790: When one validation fails, the rest of the Validations don't run After an validation error occurred due to an invalid Locator, the remaining validations would fail. 32787: Pressing Ctrl-A or Home on the viewHierarchy output causes IE8 error An attempt to select the output of viewHierarchy in the Result Details dashboard would result in this JavaScript error in Internet Explorer 8. 32786: Can't change scroll action to scrollToVisible An attempt to change an app action, scroll, to scrollToVisible, resulted in this error. 32676: No tooltip for top-level buttons in Composition Editor As of this release, Composition Editor toolbar buttons have updated tooltips. 32675: No tooltip for top-level buttons in Clip Editor As of this release, Clip Editor toolbar buttons have updated tooltips.


36

32193: Seed Data Editor is not working, trying to trick the deep packet inspection parser An HTTP 206 issue occurred while downloading a combined seed data file.


37

Build 5623.36

Test Creation

TouchTest™ Release This release formally concludes the Beta introducing CloudTest’s new TouchTest™ technology, which delivers for the first time, complete functional test automation for multi-touch, gesture-based mobile applications. TouchTest™ technology delivers fast, precision functional testing while increasing the stability of automated tests across releases. CloudTest® can launch mobile apps that are under test through a lightweight software agent, SOASTA TouchTest™ Agent. Devices can be dedicated to testing in the lab, used as part of an external test, or crowd-sourced as part of a high volume, globally distributed test. Using SOASTA TouchTest™ Driver, which is compiled into the app under test, support is provided for recording, playback and validations of user actions within any iOS 5.0 device including iPhone, iPad, and iPod Touch. There is no need to jailbreak the iOS device and the device can be untethered. Along with release notes, this release comes with a complete documentation suite that includes:

• The TouchTest Tutorial TouchTest™ presents complete end-user instructions for test creation

• The TouchTest™ Developer Guide presents complete developer integration steps.


38

http://cdn.soasta.com/productresource/download/SOASTA_TouchTest_Tutorial.pdf?v6


Welcome Page, Find Out How The CloudTest Welcome page (Central > Welcome), Find Out How section has a new look with direct links to the CloudLink forum, knowledge base, and print documentation (PDF).


39

Clip Editor, Info Windows The Clip Editor now displays a new Info icon in both Icon and List views for both browser actions and app actions. Click this icon in either view to inspect Info about the selected item. For Icon view, the Info window presents all of the relevant information from list view that was unavailable in Icon view in prior releases.

Clip elements in the Clip Editor now display a new surface icon that replaces the Scope icon that previously appeared on most clip element types (excluding delays, checkpoints, and containers. While TouchTest recording, the Info window, General tab displays for the last action as it is added to the test clip.


40

You can make on the fly edits, click through the Info tabs, or return later to do the same. Information presented in the Info window is available presented in the lower panel editor (whichever corresponds to the element’s type). List view also presents the Info window during record mode, or after clicking the Info icon.

TIP: To turn off the Info window during recording, choose the Etc. (…) menu, Hide/Show Info While Recording.

Info Window Tabs The following tabs are presented in the Info window.

General

The General tab has the following fields:

• Action – The current read-only element type.

• Name – The editable name of the element.

• Target – The read-only target name.

• Errors should – The drop-down failure actions for the selection: o fail the parent – An error here will also fail the parent clip or

container o be recorded only – An error here will be posted in results only

• Scope – The current scope of the item. Copyright 2012. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

41

• Description – An optional description.

Inputs

The Info, Inputs tab corresponds to the lower panel Inputs node. The items listed here will vary according to the type of action. This corresponds to the lower panel Inputs node (when the selection is open in the Action Editor).

• Locator – For each app or browser action, its Locator is listed. Locators

are unique characteristics that identify a specific element or object on a mobile device or WebUI. Locators come in many forms (but only those forms relevant to browser and app actions are shown above), including links, IDs such as those defined within CSS, and XPath expressions.

• Other Inputs – Any additional inputs for the selection are listed. For example, Segment (above).


42

Waits

The Info, Waits tab corresponds to the lower panel Waits node. If no waits are attached to the selection, then the form indicates so.

Waits are commands that tell CloudTest not to execute an action until a condition is met (pre-action waits), or to not continue processing the outputs, validations and property sets of the Action until a condition is met (post-action waits). Click the green Plus icon to add a new wait before or after the selected action.

Defining a Wait Using the Info Window To define a wait of either type (before/after):

1. Select a wait from the Command drop-down. 2. Enter a locator or select it from a new or existing property. 3. Specify a timeout action (or accept the default Record in Results Only).


43

Outputs

The Info, Outputs tab corresponds to the lower panel Outputs node. If no outputs are attached to the selection, then the form indicates so.

Outputs specify what is to be shown in Result Details for a given action. Typical outputs include “captureScreenshot” and “outputElementText.” A single action can have an unlimited number of outputs. Click the green Plus icon to add a new output before or after the selected action.

Define an Output using the Info Window 1. Select an output from the Command drop-down. 2. Where required, enter a locator or select it from a new or existing property. 3. Optionally, check Only if there is an error to limit the output to occasions

when an error occurs on the selection.


44

Validations

The Info, Validations tab corresponds to the lower panel Validations node. If no validations are attached to the selection, then the form indicates so.

Click the green Plus icon to add a new validation on the selected action.

Validations verify some content or event occurred as expected and have a corresponding Failure Action. App Action validations can range from simple true/false conditions to more complex conditions. A single App Action can have an unlimited number of validations. Any validation failures will be exposed in the Results Dashboard.

Define a Validation using the Info Window 1. Select a validation from the Command drop-down. 2. Where required, enter a locator or select it from a new or existing property. 3. Select a Failure Action from the drop-down:

• If the failure action is set to Record in Results Only, the validation outcome will be recorded in the result, and the test composition will continue whether the validation passes or fails.


45

• If the failure action is set to Fail the Clip, the test composition will stop if the validation fails.

• If the failure action is set to Honor the (Browser or App) Action setting, the validation inherits the parent setting from the action.

Property Sets

The Info, Property Sets tab corresponds to the lower panel Property Sets node. If no property sets are attached to the selection, then the form indicates so.

Click the green Plus icon to add a new Property Set on the selected action.

Property Sets give you the ability to take text or data from the app you are testing and store it in a custom property for use in a subsequent action or message.

CloudTest includes three property sets, all of which have relevance for refining and editing a selected (Browser or App) Action. o Custom Properties

Custom Properties are user-defined properties that are available to all clip elements, including actions. Custom properties can be thought of as backdoors that allow access to portions of the object model more easily.


46

o System Properties System Properties are available to all clip elements, including Actions. SOASTA CloudTest defines system properties. For example, a test clip has system properties such as name, repeat timing, label, and more.

o Global Properties Global properties are defined within the Central > Global Properties List and are “global” within the entire SOASTA CloudTest environment—and can be used across compositions.

Property Chooser and ISSE Editor This release introduces further improvements to the Clip Editor, Property Chooser by differentiating the Property Chooser from the ISSE Chooser. Additional tabs for Expressions and Seed Data appear in relevant contexts.

The Property icon no longer appears in the Clip Editor, List view. Click the blue oval to launch the Property Chooser instead.


47

This release also completes the replacement of the old style Property Chooser with the new style Property Chooser that was begun with Veyron 1. If the clicked oval is an ISSE, it will open in the lower panel first. From there, edit it in place by clicking the Expand icon. Otherwise, click the Chooser icon to open it in the ISSE Editor. When a clip element has For Each set as a repeat value, the Property tab, System, Modifier(s) column turns into a drop-down. Use this drop-down to select the seed data field to iterate.

For example, in the shot above, the Random Scroll transaction will iterate the Column Name.

Property Chooser Tabs The Property Chooser, Property tab was present in some parts of CloudTest as of Veyron 1, and is now active in all situations. When For Each is set on a clip element and the Property Chooser is invoked, an additional Seed Data Object tab appears. Use this tab to apply Seed Data to the current selection using the specified property.

Seed Data Object Tab The Property Chooser, Seed Data Object tab presents the same separate Allocation and Data sections introduced in lower panel Custom Properties tabs


48

as part of Veyron 1. By placing this tab in the Property Chooser, seed data can be quickly applied to the current selection.

To link a custom property using a seed data object, enter its name in the Property name field. If the custom property doesn’t yet exist, it can be created before exiting the Chooser via the Property tab, Manually Enter Property field. Select a Value type and, then specify the Seed Data object or URL to use.

• If you need to create a seed data repository object, click Central > Seed Data to do so before making a selection (using either this tab or the Custom Properties tab in the lower panel for the given element). Refer to Creating and Editing Seed Data for steps.

• For setting options in the Advanced section, refer to Setting Advanced Seed Data Options


49

http://cloudlink.soasta.com/t5/Knowledge-Base/Creating-and-Editing-Seed-Data/ba-p/672

http://cloudlink.soasta.com/t5/Knowledge-Base/Setting-Advanced-Seed-Data-Options/ba-p/1606

http://cloudlink.soasta.com/t5/Knowledge-Base/Setting-Advanced-Seed-Data-Options/ba-p/1606

ISSE Editor The ISSE Editor opens to the default Property tab, which shows the same view of objects as the Property Chooser.

Click the Expression tab to display a simple ISSE Editor.


50

The Seed Data Objects tab also appears in the ISSE Editor.

Using Seed Data Structs to Iterate For Each In the following TouchTest example, we will use For Each to iterate a previously defined Seed Data object using the row representation value Struct. The test composition consists of the following:

• One test clip recorded using the SOASTA Demo app, which is a periodic table of elements.

o The first five actions below have to do with logging in; actions 6-9 scrolled to the top of the view, tapped an element name, swiped right to return to the view, and once again scrolled to the top of the view.


51

For the example scenario, we will revise the recorded clip, by adding the following:

• Create three transaction containers: o Group the first five actions in a transaction, Login o Group actions 6-9 in a transaction, Random Scroll. o Group action 10 in a transaction, Logout.

• Create two custom properties in the clip: o Create a custom property, clipProp Table of Elements. This

custom property uses Seed Data: Repository to employ the seed data object, Randomized Table of Elements.

o In the Allocation section, specify that Server allocation is Separate slices of the seed data.

o In the Data section, specify row count 12, Use columns first 2, the row traversal type is Sequential the EOF Action is Wrap, and that the row representation is Struct.

• Create a second custom property, clipProp ScrollToTop. This property

will be used to return the view to the zero offset position after Back is played back and the view is shown.


52

The Randomized Table of Elements used in this clip has two columns, Element Name, and Offset from Top. This offset (x,y) uses coordinates where the first value is the top coordinate (0.000000), and the second is the difference in the actual position of the element. For example, Argon is 0.000000,174.000000.

The Random Scroll transaction container’s actions were revised so that after each series we return to content offset zero, for the element name we use forEach value, .


53

1. The view action (App Action 7, whose locator is classname=UITableView[0]) was recorded when the user scrolled up to the zero position.

For this action, we revised Locator to use the Location Strategy, kindof=, so that sub-classes of classname can be found, and the Content Offset using the ISSE Editor to: {%% expr : $context.currentTransaction.forEachValue[0] %%}

2. The tap action (App Action8, whose original value was text=ElementName)

was recorded when an element’s name was tapped.

For this action, we revised the Locator with a placeholder for the Element Name value.

The third action swipes right so that we return to the view.


54

The fourth action once again scrolls to the top of the view and has the zero content offset value.

For this action, we’ll replace the numeric content offset with the custom property, ScrollToTop.

Finally, we will add a repeat to the Random Scroll transaction, and then change the repeat value to ForEach. Click Save on the Clip Editor toolbar before proceeding with the Property Chooser steps.

Double-click the blue oval for [not specified] to invoke the Property Chooser. Leave the clip selected, select the Custom tab on the right, and click clipProp Table of Elements.


55

The revised actions in the Random Scroll transaction are shown below:

We also added a comment to explain what the clip achieves:

Playing this composition will repeat the Random Scroll transaction 12 times (0-11 in the array). Each transaction will employ a different element from the slice.


56

Read Only Mode for Compositions and Clips As of this release, attempting to open a composition or clip which has already been opened by another user will display a new Object in Use dialog box.

• Choose Create your own copy to display the Save dialog box

• Choose Open a Read Only copy to view the composition only

• Choose Close to opt out

Adobe HDS Display in Clip Editor

In the initial release of Adobe HDS support, the Clip Editor did not decode the data for display, although they did display in both the recording and in results for the test composition containing clips that included HDS data. This shortcoming is resolved in this release. Additionally, a decoder for Adobe F4M files has been added in this release that provides access to the segment/fragment data for recorded streams. The decoder looks for a base64-encoded string in the //manifest/bootstrapInfo element, and if found, it will create a //manifest/boxes element with all the segment/fragment data.


57

Test Execution

Verified Servers This release introduces support for Verified Servers. A server is “verified” when the Grid Manager servers finish launching, checking has passed, and default monitoring has started. When these conditions are met the server(s) of the grid will be marked ‘verified’ and only then Maestro will attempt to use them. Servers are marked to verify by default in the Servers list, Settings tab, Verified checkbox.

If the Settings tab, Verified checkbox is unchecked, then Maestro will ignore the server. The server can be manually toggled to the unverified state, for example, if the server was set up manually for use at a later time. Additionally, a new Server Instances list state, Running (Unverified), has been added to indicate a non-verified server.

The Grid Manager user interface is unchanged with this new feature, however, the background process responsible for spinning up the grid will automatically toggle the servers to the "Verified" state when the grid is ready.


58

Bugs Fixed

32635: Screenshots cut at the botoom The captureScreenshot output was cut off in th display in some cases. 32300: Null pointer/divide by 0 issue with new exports This null pointer was due to periods in the middle of the test where there were no messages being sent or no transactions/pages being started (in this case caused by a pause). 32295: Recording UIAlert with text input causes app to crash After recording a UIAlert using a “secure text input,” the mobile app would crash. 32289: TouchTestAgent can’t handle short passwords The TouchTest Agent would disconnect and result in this exception in some cases when a one-character password was used. 32227: Automatically wait for dismissed alerts to disappear If an AppAction causes a UIAlertView to be hidden (e.g. tapping the "OK" button for an alert), validations did not always work correctly, because they were executed before the alert had completely disappeared. As of this release, TouchTest will automatically wait for the alert to completely disappear before performing any validations or outputs. 32268: Range or index out of bounds This “Range or index out of bounds” exception when we were looking for Locator text. 32244: Error executing database query A higher level nested container did not yet exist in the database when an attempt to de-nest containers in the Collection Analysis widget occurred. 32221: Screenshots do not work on simulator Using the "captureScreenshot" output on an iOS simulator would result in this error.


59

32199: Delete Button doesn’t work Using UITableView edit controls to delete a cell in the table did not actually delete the cell. Also, using a swipe gesture on a UITableViewCell didn't show the delete button on the cell. 32194: "Honor the App Action setting" is confusing The presentation of “failure actions” at the message/action level, and “errors should” were inconsistent with respect to one another as well as with respect to other settings at the same level. 32193: Seed Data Editor not working After clicking the New Seed Data button, or double-clicking an existing Seed Data object, an empty (all-white) tab would appear in Central and the expected UI would never appear. 32185: Maximizing the Outputs pane did not grow the text After maximizing the Outputs pane to view a longer output such as ViewHierarchy, the contents didn’t expand to use the available space. 32184: Device is left in "testing" status when Composition is finished In some cases, the device agent was left in "testing" status when a composition completed. 32182: MakeAppTouchTestable is adding pre-processor macros to the project level Running the utility resulted in the TouchTestDriver macro being added to both the project and the specified target, instead of just the target as expected. This also resulted in build failure in the non-testable version of the project. 32176: Jenkins integration throwing stack trace even on successful tests The sCommand utility was using the wrong accessor for the PlayCommand. 32097: A result with ID 'ERROR: relation "containerdimension68" does not exist Double click on an item in the Result Details dashboard when that result was deleted resulted in an exception. 31937: waitForGestureToComplete should not time out if gesture is in progress In cases where a long-running pan or scrollToVisible was played back, the "waitForGestureToComplete" would time out after 10 seconds (configurable), even if the gesture was still in progress. Now, it waits as long as it takes, and only times out if, for some reason, we detect that no work is being done. 31930: iOS Need to record/playback heyboard hide This adds the record/playback of the keyboard hide key, which wasn’t supported in earlier releases.


60

31920: ScrollToVis Parameters not displayed correctly in Clip Editor The issue was encountered when editing the parameter value in the lower panel, then clicking outside of the lower panel, which wouldn't blur the input or update the parameter value in the list mode column. Also, the blur code would take the value from the input before it was synchronized with the invisible ISSE input field, so stale data would be used the first time that the input was blurred. 31917: Cannot click on Save button after it changes to Done When a 'type' action was played back we were automatically hiding the keyboard, the fix was not to hide it. After the fix, the Keyboard gets hidden when you tap else where/outside of an input text (same as when it got recorded) 31901: Unable to connect to the Coordinator Service Windows XP unexpectedly failed to return output when running tasklist.exe, resulting in an exception that eventually stopped the connection with Coordinator. We now log this behaviour and returning false instead so we could start the Conductor normally. 31890: Scope doesn't update in Info window The Scope icon in the Info window was unexpectedly non-responsive. 31831: No Result exists with Repository ID 1139 A result was deleted while an Analysis widget was displaying data from that result. 31810: Hover dialog app crash in Clip Editor This issue was due to a change in the number of parameters that the "scrollToVisible" app action takes from 2 to 3. There was no repository migration, so old clips with those app actions would throw this exception when trying to save. When you open and close the popup bubble, it saves your changes to the clip element, which in this case would also cause the error. 31734: Specify Mobile App Object name in MakeAppTouchTestable Users will now be able to specify the name of mobile app object they wish to create in the CloudTest repository. 31721: Changing clip repeats causes comp Properties to disappear The issue was actually that all custom properties on compositions or bands would be cleared if you load them in the lower panel then switch to some other composition element. This is a side effect of changes to how custom properties are saved, the fix is to put the correct expando on all custom properties so they are saved correctly. 31716: Device agent timed out errors This fix bumps up the version number, which should’ve been done as part of a prior checkin while making some incompatible changes (as part of mobile monitoring).


61

31696: Select ALL in the Collection Rate in Fundamentals doesn't work as expected Fundamental rates when picking All for the page or transaction names was actually looking for a page named "all". 31673: Cannot call method 'getAttribute' of undefined This was happening when the popup for an app action with no inputs like portrait was opened and then closed. The popup was trying to parse the empty inputs section to save the data there. The fix was to check for empty sections and not try to save them. 31643: Exception: ERROR: table "seeddatarow41633" does not exist A ‘table not found’ error occurred while trying to delete a folder from the Library view. The case of the missing seed data table is now silently handled. 31612: waitForNotElementEnabled passes when true Using waitForNotElementEnabled should’ve failed when true, but passed. 31611: waitForElementCount should pass but fails Although an output, ElementCount, was returned as 1, CloudTest did not successfully wait when using waitForElementCount. 31548: A result with ID '3393' does not exist in the Results Server database A results service fault occurred. 31521: Row spacing issue in UI for Clip Elements In some cases while displayng an ISSE in the lower panel the row text would break awkwardly. 31342: Exception when passing in URL Scheme A String Index out of Bounds exception occurred while passing in an URL Scheme. 31328: Exception: unexpected token: AND near line 1, column 345 [SELECT (NC.StartTime - MOD(NC.StartTime, 1000) + 1000), SU This error resulted during report generation where there were two filters with the second filter set to "All." 31155: b is null; JS line 611 After re-selecting in the Command drop-down of an existing validation, this error would occur when Play in Composition was clicked. 31143: Cannot call method 'updateRightAlignedElement' of null This error occurred while scrolling the Device Agent drop-down list for a track. Additional null checking has been added for when the device list is trying to update itself using surface data and not context menu data.


62

30809: this.ValidateName is not a function; JS line 1548 This error occurred after editing the name of a track. 30803: ISSE expander mode input blinks The expanded ISSE editor in the lower panel would blink when typing or selecting text. 30794: MakeAppTouchTestable overwrites original preprocessor macros Some preprocessor project settings could be overwritten by the MakeAppTouchTestable (MATT) utility. Now, the MATT will preserve pre-processor settings inherited from Project settings and will not over-write them. 30784: Struct should be selected by default for any Seed Data Repository, or URL with columns in the first row The Custom Properties, Seed Data Repository type’s Data section, Struct field is now selected by default for any repository- or URL-based seed data source with columns in the first row. 30719: waitForNotElementContentMode should fail but passes A negative comparison wasn’t handled as expected in this case. 30674: waitForElementNotMultipleTouchEnabled fails for element that is 'false' as expected A negative comparison wasn’t handled as expected in this case. 30569: TypeError: 'null' is not an object (evaluating 'b.getElementsByTagName'); JS line 611 This error occurred in the lower panel Action Editor while editing an app action. 30552: Recording in vertical and replaying in horizontal fails A TouchTest clip recorded in portrait orientation failed to playback when the device was in landscape. Now, this is handled. 30517: Update notifications should show the latest major release, not the first minor available The About box, Updates list has been reorganized to show major updates first. 30505: Can't use ScrollToVisible for DatePicker year The ScrollToVisible action now has a scrollDirection parameter so that it doesn't always have to start from the top. 30341: c is null; JS line 1239 An error occurred resizing the browser window to a larger size while the Result Details dashboard was displayed.


63

30250: Result details not showing up Under some circumstances, the Result Details navigation tree didn’t populate as expected. This fix enabled chase mode even if the tree is hidden. 30126: Error on new map This error occurred after adding an Activity Map widget to a new dashboard when there were no locations or targets with geographic information. 30110: selectSegment exception causes crash Exceptions during "selectSegment" were not handled as expected. Now, if a "selectSegment" action is requested on an element that is not a UISegmentedControl, then we ignore it. 30039: Session Template Package Wizard sets to null some parameters Using the Session Template Package Wizard to apply paramaters to two JSessions unexpectedly result in null values. 30015: Mobile application hung at startup An application run in the Appcelerator environment would unexpectedly hang at startup. 29895: Errors not shown to user when in the "Loaded" state The Maestro status text is now set to simply "Unloaded" for a Server Instance that is going from the Loaded to Unloaded state. 29856: Ellipses in Cover Flow An icon that had ellipses on load, unnecessarily also had them on refresh. 29826: Appcelerator clip gets stuck A switch set on the UINavigationBar[0] would cause the recorded classname not to work on playback. Now, the tap is recorded using the button title instead of the UINavigationBar classname. 29777: Problem with login clip Playback of a mobile clip with a login failed unexpectedly. 28450: Precision Pinch is not playing back correctly The pinch location and the location of the touches are now recorded at the beginning of the pinch gesture for use during playback.


64

Build 5564.70

Test Creation

Support for Adobe HDS bootstrap files This release introduces support Adobe HDS Live streaming. The binary .bootstrap message is now being auto-decoded into XML, making it data-accessible via JavaScript.


65

Build 5554.60

Bugs Fixed 30223: Exporting full request and response data When exporting a result from a General mode composition, additional event details are provided for any objects (composition, clips, collections, messages, etc.). For HTTP messages this includes the full request and the response.


66

Build 5554.56

Bugs Fixed 30825: Report Generation didn’t work with some server configurations Report generation didn't work with certain CloudTest server configurations.


67

Build 5554.55

Bugs Fixed 30578: HTTPS recording produces browser warnings The SOASTA Conductor will now automatically update the certificate if an older version of the HTTPS certificate exists in cases where Browser Driving/Recording options are enabled. If yes, the certificate will be updated it with latest. 30577: Double Quote Problem Cookies that used quoted values weren’t handled as expected. 26208: Error Invalid "Set-Cookie" header in response CloudTest was unable to parse cookies which contained the string 'expires' in the cookie value.


68

Build 5554.53

Bugs Fixed

30637: Locator Finder 'preview' does not close

Once activated while recording a Web UI, the Locator Finder couldn’t be

closed.


69

Build 5554.47

Bugs Fixed 30578: HTTPS recording produces browser warnings Users who have already installed SOASTA Conductor will need to update the certificate in their browser configuration. For Internet Explorer, and Google Chrome running on Windows, the instructions are as follows: 1. Start SOASTA Conductor. 2. Right-click the icon that appears in the system tray, and choose Configuration. 3. Click OK. 4. You will be prompted to import the SOASTA Conductor HTTPS certificate. 5. Click Yes. For Safari, and Google Chrome running on the Mac, the instructions are as follows: 1. Start SOASTA Conductor. 2. In the menu bar, click the "SOASTA Conductor" menu and choose Preferences. 3. Click OK. 4. You will be prompted to import the SOASTA Conductor HTTPS certificate. 5. Click Yes. For Firefox on all operating systems, see the "Importing Site Certificates for HTTPS" section of CloudLink.


70

http://cloudlink.soasta.com/t5/Knowledge-Base/Changing-SOASTA-Conductor/ba-p/494

Build 5554.44

Bugs Fixed 28993: Comments are not persisting in clip Entered comments didn’t persist after closing and reopening a clip.


71

Build 5554.41

Bugs Fixed 30083: Functional test "stops recording As noted in the prior write-up (published with Build 5554.36) for this case, Firefox versions 8+ have cross-origin frame lock-down enabled by default. This lock-down provides a useful way to avoid XSS injection attacks. What this means is that for certain sites (windows/frames), CloudTest was unable to get access to the frameElement which is necessary to record whats happening in the given frame/window. On sites where this is applicable, the following manual change will turn off cross-origin frame lockdown for the Firefox profile in use:

1. Open the "user.js" file in the Firefox Profile (Found in ~/Library/Application Support/Firefox/Profiles on a Mac machine).

2. Add the following two lines to user.js: user_pref("capability.policy.default.Window.QueryInterface", 'allAccess');

user_pref("capability.policy.default.Window.frameElement", 'allAccess');

3. Restart Firefox. Once Firefox has been restarted after making this change, recording of the given site works correctly.

Note: Since making this change disables the cross-origin checks completely across Firefox, they should be undone after recording is completed.


72

Build 5554.36

Bugs Fixed 30083: Functional test "stops recording" on site A problem occurred doing browser recording on cross-domain sites due to a “Cross-domain frame lock” in use in Firefox 8+ browsers. Due to this cross-domain frame lock, a cross-domain window would not get registered, and therefore, wouldn’t record as expected. As of this release, when trying to determine whether a new window should be registered for recording or not, CloudTest will try and determine if this window falls under the domain of the original recorder window (the one under which recording was started). In some cases where this doesn’t succeed, it will be necessary to click to stop recording, and then resume recording using the respective URL directly (by entering it when the “Enter starting URL for recording” box appears on resume). 30082: Functional recording giving "Invalid URL" for valid URLs A properly formed URL with parameters would produce this error in the Clip Editor, Enter starting URL for recording box. 29994: Duplicate property created in clip A clip that had custom properties whose names were equivalent once special characters were stripped became corrupt after being saved or opened in a composition. 29985: Unable to record login For the latest Mac OS X Java version, a problem could occur with the Origin header being in the restricted headers list that would result in recording failure. As of this release, Conductor will set the "sun.net.http.allowRestrictedHeaders" JVM system property to "true" to turn off the HttpURLConnection blacklist. 29779: Graph widgets when exporting CSV show time as 16:01:00 Results exported to the CSV format unexpectedly failed to post the correct time format when “Relative time” was used.


73


74

Build 5554.19

Bugs Fixed 29963: Re-starting environment with grid up results in error during result creation This "could not create result" grid error occurred in a grid on an Environment that had been brought down and then brought back up while the grid was up. 29882: Rackspace OpenStack starts two copies of JBoss An attempt to use Rackspace OpenStack unexpectedly resulted in a redundant start of JBoss. 29857: Deadlock if there is an error while stopping a Composition containing Mobile Apps If a Composition actively playing Mobile Apps was stopped, and an error occurred in the Mobile App processing during that stop, sometimes a deadlock would occur, causing the Composition to remain in the "Stopping" or "Aborting" state forever. 29783: Error executing database query With a deeply nested result that had filters applied, it was possible to get this exception in the Hierarchical Collection Analysis. 29669: Error Report Submission should be background processing Error report submission could take too long in some situations. As of this release, CloudTest wil pass error report submission into a background thread, thus permitting the user to continue without being held up. 27810: Tracks with 1 user ignore leading Start time In prior releases, CloudTest would ignore leading Start times when repeats were set to 1. As of this release, those lead times are utilized as set.


75

Build 5554.10

Bugs Fixed Activity Map Widget fixed The Activity Map widget has been upgraded to utilize the Maps API v3 for this release. 29846: Error executing database query An error occurred while trying to filter the Event Log to display error events when a given container element didn’t exist in the dimension. This fix also includes a new feature to export Events to CSV. 29821: Support for Rackspace OpenStack API fixed Launching Rackspace OpenStack servers from a grid would fail with the error "Server returned invalid JSON." 29806: Dashboard not showing correct result With the async start of tests, the dashboard, when switching from the Edit tab to the Play tab, would sometimes show the previous result instead of the new one because the new result had not been created yet. As of this release, the play tab is only switched to after we get a result ID. 29824: Uncaught ReferenceError: ConcertoLayout is not defined; JS line 101 This error would occur during an attempt to export chart data to. 29815: Error switching to the debug tab while a composition is playing This script error would occur clicking the Debug tab for the first time while a composition played. 29781: Grid wizard comes up disabled for existing Grids Opening an existing grid in the Grid Manager resulted in a disabled grid (i.e. the Deploy Instances button was disabled as well as the navigation arrows).


76

29774: Results Viewer not up to date when Composition ends In some cases, after a Composition completed, the Result Details would present an incomplete display of events. This fix takes care of the missing events and status, etc.


77

Build 5554.3

Test Creation

Change in behavior for HTML parsing affects XPath As part of a fix introduced for Build 5483.23, the underlying HTML parser used by CloudTest was upgraded to a new version that now imposes the TBODY tag wherever TABLE is detected. This tag is already commonly imposed by some browsers, and is technically a required HTML tag, so this is a familiar issue for CloudTest users doing browser tests whenever specifying XPath in Property Sets, Validations, of when using the Browser Action Editor’s Locator tool. SOASTA recommends that a given XPath or JS DOM be specified so that it'll work with or without a TBODY in the target XML. This can be accomplished in a number of ways. For example, by using an XPath that selects past the TBODY, such as: //td[@id='xxx']

Or, in cases where it’s desirable to select on the table (or parent), and on a child: //table[@foo='bar']//td[2]

As is the case whenever specifying XPath, if an XPath doesn't allow for a TBODY, it's the path that's wrong, and not the DOM.


78

Test Execution

Cloud Provider Account Support for Rackspace (CloudTest Pro) As of this release, CloudTest Pro users can create Cloud Provider Accounts for the Rackspace Cloud (OpenStack).

By entering valid cloud provider account information for Rackspace, locations are automatically created and can then easily be used to provision servers via the Grid Manager.

Setup Rackspace (OpenStack) as a Cloud Provider Use the following steps to enter your valid Rackspace credentials as a CloudTest Pro/Enterprise Cloud Provider Account.

1. Select Central > Cloud Provider Accounts and then click New to launch the Cloud Provider Account dialog box.

2. Enter a name for the account. This name is used in locations and also appears in the Cloud Provider Account drop-down in the Grid Manager.

3. Change the “Type” drop-down to “Rackspace Cloud (OpenStack)”. The dialog box changes to reflect the available Rackspace data centers (e.g. a tab for DFW1 appears).

4. Check the box for the first Rackspace cloud site this account uses and then enter the Rackspace Account Name and API key, and then enter the API key a second time in the Confirm API key field. Repeat this step if other Rackspace data centers will be used with this account.

5. To find your API key, a. Log in to the Rackspace Cloud management portal.


79

b. Click the Your Account sidebar link, and then API Access in the sub-menu that appears.

c. Click the "Show Key" button (or "Generate New Key" if you don't already have one).

d. Note your username, and copy your API Key. 6. You can optionally configure an access control list to restrict which

CloudTest users are able to launch Rackspace servers with this account. This step is not required if you are the only CloudTest user that will be launching servers. To configure an access control list:

a. Select the CloudTest user you want to grant access to in the drop-down at the bottom of the dialog.

b. Click the Add button. c. A new row will appear in the list above the drop-down. d. To allow the user to view the Cloud Provider Account object (but

not necessarily use it), check the Read box. Note that even with this box checked, the user will not be able to view secure information (such as the Rackspace shared secret).

e. To allow the user to modify, delete, or re-name the Cloud Provider Account, check the Write box.

f. To allow the user to launch new servers using this Cloud Provider Account, check the Use box.

g. Repeat steps a-f for each CloudTest user that you want to grant access to.


80

7. Click OK to complete configuration of the Cloud Provider Account. The

account information will be validated by issuing a call to Rackspace. If the credentials are invalid, a warning will be shown. You can choose to go back and correct the credentials, or continue.

8. After the Cloud Provider Account is saved, a new Location for each Rackspace data center will be automatically created, if it does not already exist.

This location must be specified during Grid Manager configuration of grids For more information about Locations, refer to Using Locations.


81

http://cloudlink.soasta.com/t5/Knowledge-Base/Using-Locations/ba-p/384

Async Play Start Improvements This release includes back-end Async Play Start improvements that will result in the following user experience improvements: (1) greater reliability and better reporting of problems during Composition start, especially with multi-server environments, and (2) more status information during play start (this will result in additional Status Indicator messages such as “Creating Result”) and (3) more status information during Composition termination.


82

Bugs Fixed 29756: Playing back again leaves Result Details in this state Result Details sometimes did not show any data on subsequent test runs depending on timing. 29677: Multiple text entries with the same PlaceHolder don't record proper locators Multiple text entries with the same PlaceHolder didn't record proper locators. 29579: Cannot get back to real time mode After making a selection in the Result Details, Navigation Tree, attempts to return to “real time” mode would prove unreliable. 29560: Cannot select clip elements from the tree while playing back In some cases, attempts to select a clip element in the Result Details, Navigation Tree would fail if the composition was playing. 29550: Support record and playback of logging in by hitting the Go/Done iOS keyboard button This case adds support for record and playback of logging in by hitting the Go/Done iOS keyboard button. 29537: iPod screen shot is squished Screenshots produced as a result of the Output, captureScreenshot, were incorrectlu constrained in some cases. 29536: Ability to simulate (and record) Enter key

A given mobile app with a Login screen didn’t use a “Submit” button, but relied upon the Enter key instead. This fix adds new action called "keyboardReturn" rather then recording a type with an ascii code such as \13. 29524: Unexpected token e An error occurred when Result Details was showing a result that had been deleted. A useful error message will now display in this circumstance. 29522: A result with ID 'ERROR: relation "eventfact2958" does not exist This error would occur if a result from a Draft Composition was open and the Cover Flow was in use and a second user closed the Composition Editor, inadvertently deleting the draft. A user-friendly error message now handles this circumstance.


83

29511: not able to use TouchTestIDs In some cases, valid TouchTest IDs on a given button were not captured as expected during recording. 29402: Invalid argument This error likely occurred when drawing cross-hairs in a given chart and the underlying action had coordinates of “NaN”. As of this release, CloudTest will not draw cross-hairs if there aren't valid coordinates. 29222: a.GetItemFlow().Size is not a function; JS line 25 While watching a result in “real time” mode, the user double-clicked a Target to open it in the editor, and this error resulted when the view was switched back to the Dashboard tab. 29009: Typed in zip code not playing back A numeric zip code entry interface in a mobile app didn’t play back as expected. 28854: Import doesn't work in Google Chrome An attempt to upload an import file in Google Chrome 18 failed due to some change in Chrome. 28803: attempt to run compile-and-go script on a cleared scope; JS line 891 This error occurred running a script on cleared scope. As of this release, this error is ignored. 28763: Cannot read property 'uniqueIDPrefix' of null Additional null checking code has been added to detect further occurrences of this script error in the Dashboard Edit panel. 28756: com.soasta.common.xml2.XmlUtilities.transformDocFromFile An error occurred while creating a new Mobile App object. 28699: Clip with nested mobile clips does not offer device choice The Location drop-down didn't display the available Device Agents when a clip contained nested clips. Nested mobile targets are now handled by overriding them via their Device Agent property in the Composition Editor, Property Sets tab for the given target. 28638: this.__el_crosshair is undefined; JS line 13 An error occurred drawing a crosshair on a chart, likely due to an unknown or undefined coordinate. 28621: Rackspace teardown fails if server was externally deleted A change in the Rackspace API wherein a response code changed from 500 to 404 resulted in this error. Both codes are now handled.


84

28615: The referenced monitorspec '276' does not exist An error occurred when a deleted or invalid monitor was still in use in a widget. As of this release, we don't allow invalid/deleted monitors to be used in widgets. 28570: SCommand fails with NullPointerException if credentials are invalid If the credentials for SCommand were invalid, then a NullPointerException resulted instead of an "Invalid credentials" error. This also affected the Ant tasks, which in turn affected Jenkins. 28559: ISSE insertion via Property Chooser always inserts at the end in IE8 ISSE insertion didn’t work as expected in Internet Explorer 8 and 9 versions. 28275: Should be able to get back to the Property Chooser dialog from the blue ISSE oval After using the Property Chooser dialog box to create and place an ISSE. However, double-clicking the resulting blue-bordered ISSE would not bring the Property Chooser back up with the selection displayed as expected. 28240: Report Error – CloudTest Lite An attempt to create a simple report failed in CloudTest Lite only. 27437: Dependencies tab needs to show full path The Dependencies tab for a given test clip didn’t show the full path to compositions in which it was in use, make it difficult if not impossible to determine the actual dependencies (for example, the composition might exist in multiple folders with the same name). The full path is now shown. 25726: EC2 server list setup fails if there is an extra "Multi-Server Environment" folder EC2 server list setup failed if there were additional system monitor folders. 22685: Executive summary doesn't show grey box that's in the original source docx A generated report didn’t correctly reflect all the layout specifications of its report template.


85


86

Build 5483.23

Bugs Fixed Charts Not Rendering in Internet Explorer 8 Some charts unexpectedly failed to render when shown in Build 5483.3 in Internet Explorer 8. 28771: Problem with MakeAppTouchTestable The Make App TouchTestable utility would fail in projects whose main file included no header files, and also if the AppDelegate file was named ApplicationDelegate.mm. This is now handled. 28643: Xpath fails and causes user to restart CloudTest Lite There were problems parsing some complex HTML responses when applying XPath against the HTML for Property Sets or Validations, that caused the XPath to not match when it should. 28642: Central > Mobile Apps list, Owner column empty The Central, Mobile Apps list, Owner column was not populated with any values as expected. 28636: Result import fails An attempt to import data exported from an build didn’t succeed as expected due to a missing column for results. 28619: Choose a Device Agent box pre-selects app but not device While prior selection was in effect for the mobile app a user had previously selected, the Device Agent was not also selected in the Choose a Device Agent and Mobile App box. 28618: Selenium script import fails if there are invalid XML characters in the Open command A selenium script import would fail if Open contains invalid XML. For example, in the form of URI=xxx.


87

28610: Tapping causes too many taps Tapping in a mobile app would capture multiple taps instead of the one expected tap. As of this release, only one tap per event (as opposed to per gesture recognizer) is recorded. 28582: Exception: java.lang.NullPointerException When de-nesting a container, we could find that an intermediate container is not yet in the Results Service. 28581: waitForElementEnabled mismatch between arguments & help text The right panel help text for the waitForElementEnabled action indicated support for a pattern argument, but the argument in question was not included in the Waits form. 28579: waitForElementPresent documented arguments The right panel help text for the waitForElementPresent action indicated support for a pattern argument, but the argument in question was not included in the Waits form. 28570: SCommand Fails with NullPointerException if credentials are invalid If the credentials for SCommand were invalid, then a NullPointerException would result instead of an "Invalid credentials" error. 28552: Missing help text The reference text was missing for all of the orientation actions. 28551: Inconsistent icons in Results The Widget Selection Panel, Dashboard Data section incorrectly showed a Success (green checkmark) icon and a Fail (red X) icon for two identical runs of the same mobile app test that had identical results. 28545: Unexpected token at Flipper.init The Error dialog didn't capture an underlying error for display as expected. 28541: Clip: 'TC Tag Creation #1' appAction6 doesn't happen TouchTest recording wouldn’t recover after a problem with identical locators on a page occurred. For example, where more than one feature on a page had ‘text=Hashtag’ as its Locator. As of this release, the ability to record locators with an index such as text=Hashtag[2] has been added. 28540: Nothing is recorded after clicking 'Feedback' The issue here is that we call the "type" action as a pending action. So, unless we get the notification that type action has been completed, all the subsequent recorded actions are sent to the pending actions queue. In the case of a given field in a mobile app, the “type” action never finished (or the type completion notification never comes through) and thats why no more actions are sent to


88

CloudTest. As of this release, hiding the keyboard now cancels the pending action so that recording can resume. 28524: Error executing database query This problem occurred due to invalid SQL being generated when applying a filter on the Fundamentals rate metrics. 28477: The device agent has timed out message should go away when you click on the record button again After a Device Agent timed out, thus producing a Clip Editor message as expected, clicking Record after resolving the agent issue wouldn’t result in the resumption of recording as expected. 28472: TouchTestDriver does not retry failed network requests Once a network request failed, the request was not retried as robustly as expected. As of this release, we now retry up to 5 times if there is a network error. 28469: The device agent has timed out In prior releases, TouchTest Agent would receive a server task to launch the mobile app on the device and then go into the background; however, before this occurred, one or two server tasks would be intercepted by TouchTest Agent that were intended for TouchTest Driver, and this connection timeout would result. 28452: Type not recorded in the mobile app After entering edit mode in a given mobile app, the subsequent “type” action was not captured as expected. As of this release, we are now listening for UITextViewTextDidBeginEditingNotification and for UITextViewTextDidEndEditingNotification. 28371: TouchTest Agent should reject devices w/ iOS earlier than 5.0 In earlier versions, TouchTest Agent didn’t clearly indicate when an unsupported iOS version was encountered. 28054: UIPageControl doesn't record and playback In earlier versions, TouchTest didn’t record or playback the UIPageControl used in some mobile apps.


89

27129: App action “type” in the User ID field fails In earlier versions, TouchTest recording didn’t handle cases where a “type” is a combination of clear, sendKeys and submit. So that, when the submit sent "\n" to the text field it would the text that was just added with sendKeys. This is now handled. 26967: Seed Data object not handling '&' correctly Entering an ampersand in a Seed Data row would incorrectly result in its hex code equivalent (&) being inserted into the test. 26636: Attempting to duplicate a Playlist yields internal error Using a right-mouse click in Central > Library to Duplicate a Playlist would result in this "Internal error". 26633: Can't open Playlists from Library view In prior release in Central > Library, Playlists appeared in the list on the right. However, neither right-mouse-Open nor double-click would open them. 26491: Reports created in Gran Cabrio cannot be opened in MS Word 2007 Reports generated from Gran Cabrio (5303.x) builds wouldn’t open in Microsoft Word 2007 as expected. 25862: Results Databases should allow teardown after abnormal termination In prior releases, if a database was externally terminated outside CloudTest, the Results Database dialog’s Tear Down Instances button would remain inactive. As of this release, the Tear Down Instances button will become active if an externally terminated database is detected. 25840: Volume list should be filtered by CPA In prior releases, the Results Database dialog box’s Volume list included every volume in the selected Location, regardless of which CPA owned it. As of this release, this list will only show those Volumes owned by the selected Cloud Provider Account.


90

Build 5483.3

Test Creation

Introducing TouchTest™ CloudTest’s new TouchTest™ technology delivers, for the first time, complete functional test automation for multi-touch, gesture-based mobile applications. TouchTest™ technology delivers fast, precision functional testing while increasing the stability of automated tests across releases. CloudTest® can launch mobile apps that are under test through a lightweight software agent, SOASTA TouchTest™ Agent. Devices can be dedicated to testing in the lab, used as part of an external test, or crowd-sourced as part of a high volume, globally distributed test. Using SOASTA TouchTest™ Driver, which is compiled into the app under test, support is provided for recording, playback and validations of user actions within any iOS 5.0 device including iPhone, iPad, and iPod Touch. There is no need to jailbreak the iOS device and the device can be untethered. Along with release notes, this release includes a complete TouchTest Tutorial TouchTest™ with complete end-user instructions for test creation, as well as a TouchTest™ Developer Guide where important developer integration steps are presented. CloudTest User Interface Improvements for TouchTest™ include:

• Welcome Page improvements that facilitate TouchTest™ integration by providing relevant steps as a wizard.

• Clip Editor Improvements that facilitate inline TouchTest™ recording and editing.

• Composition Editor Improvements that facilitate TouchTest™ playback.


91




Welcome Page Improvements The CloudTest Welcome page (Central > Welcome) has a new, lighter, and easier-to-read look and feel for this release. Additional wizard steps are available to TouchTest™. Also, new TouchTest™ items have been added to both the Downloads and Documentation & Samples sections.

The first page of the TouchTest wizard present three simple first steps to make a mobile app TouchTestable, to install that app on a device, and to run a device agent on that device using Safari browser.


92

The remaining three steps of the wizard register that device (via the Device Agent in Safari), and proceed right away to clip creation, and finally, compositon creation.

Double Click to Open This release adds “Double Click to Open” support for items in the Composition Editor and Clip Editor, Analysis widgets, and in the Result Details dashboard, where its addition will have a dramatic impact on test editing due to the ability to open any part of a test clip precisely at the selection point. This ability to open the clip at an exact location will make test editing far easier. Double-click support that was provided in the Composition Editor in prior releases is now more widely in effect throughout. For example, in CloudTest linked scripts, nested clips, and in result items in both the Navigation Tree and Cover Flow.

Double Click to Open in Result Details Double-clicking from within Result Details is comprehensive: every part of the test, including the composition itself, can be opened in a new tab.

Click a clip to open it in a new Clip Editor tab with no item selected (shown below) in the parent clip.


93

Double click any clip element to open it in its respective editor at the same location as displayed in Result Details.

For example, double-clicking a selected page (shown below) will open

the parent clip in the Clip Editor with the page selected (as with the page shown above and below).


94

Items in playing compositions can be selected and then double-clicked while the test is streaming.

Whenever an item within a container is opened the container is also expanded.


95

Double Click to Open in Analysis Widgets Double click any item in a composition hierarchy in an Analysis widget to open it in a new tab in its respective editor. Double-click to open is provided in all Analysis widgets whenever a result is selected, whether in a stand-alone dashboard or a dashboard displayed from within the Composition Editor tabs. All analysis widgets are supported, including Composition Analysis, Clip Analysis, Clip Element Analysis, as well as collection and property value analysis widgets. No action is taken when an item in a widget displays an item other than a composition element (i.e. a URL). In the Composition Analysis widget, items can be opened in their respective editors.

Click the Composition, Band, or any Track to open the composition in the Composition Editor.


96

Drill down and click any clip or clip element to open it in a new tab in the Clip Editor.

When you do so, the new Clip Editor tab opens on top and the clicked item, in this case the “knowledgebase” page (shown above and below) container, is selected.

When a script in an Analysis widget is clicked, the Script Editor opens (rather than opening the clip in the Clip Editor at the insertion point for that script).


97

Double Click to Open in Composition Editor Double-click actions were already supported in the Composition Editor for clips in tracks (double-clicking a clip in a track opens the Clip Editor). As has been the case for many releases of CloudTest, double-clicking a clip in the Composition Builder adds it to first or selected track.

Double Click to Open in Clip Editor Double click a Nested Clip to open it in a new tab was introduced when Nested Clips were introduced.


98

The new tab is on top after a double-click.

User-Friendly Display of Properties & ISSEs Properties, property paths, and In Situ Subsitution Expressions (ISSEs) are now presented as blue rounded boxes throughout CloudTest. They are displayed as “bubbles” also in the lower panel. There is a new, lower-panel Expert button that allows users to expand the bubble to toggle between full text and bubble view). This improves both their visibility in the workspace and their accessibility. Simply double-click or hover (single-click) to inspect or edit their data and info. Double-click any property box to open it in its appropriate editor.

Placed properties now appear as rounded blue boxes throughout the CloudTest user interface making them easier to find later. For example, the Clip Editor, List View, Parameter 2 column shown above has a property, username, placed into its Query String.

Property boxes also appear wherever Property entry fields exist, such as in If-Then-Else or Switch containers shown in the Clip Editor.


99

Property boxes are also displayed in the Composition Editor whenever a property field or value is in display. For example, whenever a Repeat By Property repeat method is specified for either a track or clip repeat. When Repeat By Property is first selected from the Repeat Method drop-down, it will display [not specified] until its value is specified via the Property Chooser.

For more complex ISSEs, user-friendly display means hiding the underlying ISSE syntax in favor of a clickable, and more readable format as shown in the Group 1 container below, where a user-friendly ISSE is deployed.


100

A closer view shows that the user-friendly version of this expression has the same rounded-blue border as the other items above.

To inspect the full expression syntax for any user-friendly ISSE, double click to inspect it in the lower panel.

Note that the Locator field above shows the concatenated expression. To show the full ISSE, toggle the Expert button to the right of the field where the ISSE has been entered.


101

Clip Editor Improvements As part of the introduction of TouchTest™, the Clip Editor now provides the ability to record mobile app actions inline, just as browser actions are recorded inline. Target creation for mobile apps and WebUIs now occurs on-the-fly, the underlying targets are accessible when creating or modifying system or custom properties.

Using the New Record Toolbar Button The new Record drop-down is now prominently part of the main Clip Editor toolbar. To begin recording, simply click the Record drop-down and choose the recording type (e.g. Record Mobile App).

As of this release, the lower panel tabs are all minimized by default in order to increase the usable workspace in the editor. When a new Untitled Test Clip is created, the Clip Editor launches with a Record prompt such as the one shown above.

1. Once ready, click the Record button and then select Record Mobile App.

TIP: Browser recording options, which appeared in all browsers in

prior releases, now appear only in the supported Firefox.


102

The Choose a Device Agent and Mobile App wizard appears. Choose the Device Agent and Mobile App from among those configured and click Record on the lower right.

As a result of selection above, a target is created. Target creation via a separate wizard is no longer required.

You can discover the name of this automatically created target by looking at the Clip Editor > Messages Actions lower panel, Included Targets field.


103

• To change either the Device Agent or Mobile App click the corresponding

Target Editor, Section Info, Settings field for the given item.

Browser Recording in the New Clip Editor When in Firefox, if the necessary Browser Recorder extension has yet to be installed, an additional drop-down command appears.

Otherwise, two new browser recording commands appear, Record Browser (new window) and Record Browser (inline frame). These commands replace the Pop in/Pop out interface used in prior releases. All recording is performed in either a new window or inline frame within a given session.


104

Record Browser (new window) 1. Choose Record Browser (new window) and then enter the starting URL.

If the URL entered is already in a target, then that target is used. Otherwise, a new WebUI target is created.

2. Click Record to proceed with your recording in a new window. When you do so, the target URL opens in a new Firefox window and the Record button indicates recording is underway.

3. Perform the planned browser actions in the new window. Actions are

added to the Clip Editor as you go.


105

4. Click the Record button (in the original window Clip Editor tab) to stop

recording.

Record Browser (inline frame) 1. Choose Record Browser (inline frame) and then enter the starting URL.

If the URL entered is already in a target, then that target is used. Otherwise, a new WebUI target is created.


106

2. Click Record to proceed with your recording inline in the Clip Editor workspace.

When you do so, the target URL opens inline and the Record button indicates recording is underway.


107

3. Perform the planned browser actions in the workspace. Actions are added to the Clip Editor as you go.

4. Click the Record button to stop recording.

Open in Composition Button Now a Drop-Down The Clip Editor toolbar Open in Composition button is now a full-fledged drop-down that includes three commands.

Open in Test Composition • With a test clip open in the Clip Editor, choose this command to place the

clip into Track 1 of a new, draft Untitled Composition. Copyright 2012. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

108

When you do so, CloudTest switches automatically to a new Composition Editor tab. The composition is not loaded and play doesn’t commence automatically.

Play in Test Composition • With a test clip open in the Clip Editor, choose this command to place this

clip into Track 1 of a new, draft Untitled Composition, which will be loaded and automatically played with the Play tab visible.

Debug in Test Composition With a test clip open in the Clip Editor, choose this command to place this clip into Track 1 of a new, draft Untitled Composition, which will be loaded and automatically played in “Debug mode” with the Debug tab visible.


109

List View Improvements The Clip Editor, List view improvements in this release include the display of repeats in rows instead of in a vertical column, as well as showing breakpoints in the gutter, and renamed column headings.

List View Repeats The Clip Editor, List view now displays repeats as a separate line-item in the list above the clip element that has the repeat. This replaces the Repeat column in prior releases. Use Repeat By to specify a specific number of repeats for a given item. The Repeat By will appear above the item to which it is attached.

List view shows user-friendly paths for properties whenever they occur in List view. Property fields are shown in containers, as well as when a property is specified for a repeat (shown below). Changing the Repeat Method drop-down shown above from Repeat By to Repeat By Property introduces the Property Chooser dialog box. When a repeat method that requires a property to be set is first selected, it will show as [not specified].


110

Use the Property Chooser to make a property selection from among existing properties or to define a new property.

View property detail info in popups via single-clicking in or hovering over the given item.

Additional repeat types include For Each (formerly Property: For Each)

and While (formerly Property: While).

The Continuous repeat setting found in prior releases has been renamed Infinite.


111

Breakpoints in List View Gutter The List view also now shows breakpoints in the gutter. In the prior release, these breakpoints were displayed in the column to the right of the clip element name.

Renamed Column Headings Column headings in the List view have been renamed for this release. The default column sort order is Name, Operation, Parameter 1, Parameter 2, [Scope], Type (for Containers), and finally, Target Name.

Right-click the column header to display a list of columns to show. Uncheck any item in the list to suppress its display in List view.

User preferences set here are honored for all clips until changed again. Columns can also be reordered using drag and drop. For example, the Type column, second-to-right by default, has been placed in the position beween Name and Operation in the screenshot below.


112


113

Composition Editor Improvements Composition Editor improvements include a new track distribution user interface that supports multiple track distribution. A distribution is a grouping of one or more locations within a given track that share repeat settings. In the Composition Editor, the Track drop-down has a new look and feel that also reflects new underlying functionality. Locations are now more identifiable, with sections more visible in the list, and are selected by radio button.

• Locations can now be grouped in a distribution that shares repeat settings,

or, configured so that each location can have its own distribution with its own repeat settings. Repeat settings can be enabled or disabled per distribution.

If lower-panel location settings are already in effect, the Locations drop-down indicates so. Advanced settings can be specified in the Track, Property, General tab in the lower panel.

Note: Right-click a track and select Edit Properties to easily access

its lower panel settings.


114

Locations in a TouchTest™ TouchTest Agents appear in the Locations drop-down in TouchTests.

For TouchTests, the listed locations are the device agents to specify for test playback. Select the Connected target device from the Locations list to proceed (if it is not Connected, use Safari to login from the mobile device). This selection can also be made at the clip level by clicking Select at Clip (shown above). Once Select at Clip is chosen, the clip(s) in the track will display the Location drop-down.

If multiple clips are in the given track, each displays the Location drop-down.


115

Locations in an HTTP(S) Test • When a track with only an HTTP(S) clip is selected, the Location drop-

down familiar from prior releases appears (in its new context).

• When clips with a Mobile target(s) & a Server target(s) are combined into

one track, the Location drop-down refelects that both devices and servers are available to be configured.


116

If Select at Clip is selected, then the Mobile Device can be selected at the clip level on the given track.

Track Properties, General Tab For this release, the Track properties General and Repeat tabs have been merged into one General tab to accommodate all track properties related to server allocation, server location, distribution, and repeats.

• Former General tab settings appear at the top of the newly combined tab: Track Name, Percent Allocation, and Errors should.

• In addition to the former General tab, Maestro and Copy Count fields, the

merged General tab includes all of the former Repeat tab settings.

• These merged settings appear in the new General tab, Maestro Settings section (below) and can now be applied to the entire track, if grouped, or be individually set per location.

• Each row in the Maestro Settings form has its own Add/Delete icons, whether within or between distributions.


117

As in prior releases, repeat settings are disabled by default and appear in the tab only when a given Enable Repeat box is checked.

Default Track, General Tab Properties When a new composition is created, the following default track properties are set:

• Track Name: Track 1

• Percent Allocation: inactive (this field is inactive until a clip is added to the composition)

• Errors should: fail the band (this is the default for all composition elements)

• Maestro Settings (section): o Location: Any Location o Copy Count: Inactive by default (make Copy Count active by

selecting a Dedicated Load Server via the Location drop-down) o Enable Repeat: repeat settings are disabled by default (check the

box to enable repeat settings for the given Location(s) or Clip(s).


118

Adding Multiple Locations or Distributions to a Track The track properties General tab offers two approaches to track distribution—Locations sharing repeat settings can be grouped into a distribution (represented visually as an inner (darker gray section, with inner icons) within the Maestro Settings form,

With Distributions, one track can be configured per functional unit of clips, and it can run on as many servers and repeat in as many different ways for different sets of servers as required. In addition to providing a quick way to propagate repeat settings within a test composition; distributions will improve overall performance in some tests where many tracks slow test load times and complicate the interpretation of test results.

Note: The Delete icon(s) are inactive if only one item is listed in the distribution or in the overall track.


119

Or, Locations can be added to a new distribution (lighter gray, outer icons) that has its own repeat settings (shown above).

Adding Locations To a Single Distribution Locations can be grouped together into one distribution by using the inner (leftmost) Plus icon. Use this in order to have repeat settings that apply to all the locations grouped in that distribution.

1. Click the green Plus icon in the inner form (darker gray) to add a new Location to the given section.

When you do so, the Location is added to the distribution (inner form). One Enable Repeat checkbox is provided to apply repeat settings to the locations grouped in this section.

2. Repeat to arrive at the desired number of Locations in this distribution. 3. Next, select a Location, or server, for each row.

If Enable Repeat is checked, those settings will apply to all locations in this distribution.


120

Adding Locations To Distinct Distributions In this setup, each distribution has its own repeat settings. This is accomplished by by using the outer (rightmost) Plus icon to add Locations to a new distribution—in order to have repeat settings that apply to each location(s) grouped in a distribution.

1. Click the green Plus icon in the outer form (light gray) to add a new Location section to the track. Like with the top section, any section can have one or more locations defined.

2. Repeat to arrive at the number of Locations to use. 3. Next, select a Location, or server, for each row. 4. If Enable Repeat is checked, those settings will apply individually to each

distribution.

Changes to Repeat Method Drop-Down Fields in the Repeat Method drop-down have been renamed for this release to make them more user-friendly. Formerly, the top two listed methods were Fixed and Property: Fixed both of which were used to set a fixed value for a given repeat, one of which called a property to do so.

• The Fixed repeat method has been renamed Repeat By for this release.

Repeat By gets an integer value in all its possible contexts (e.g. whether parallel or serial repeats are in use). This value can also be set on the track surface and such changes are reflected in the lower panel (as shown above).


121

• The Property: Fixed repeat method has been renamed Repeat by Property for this release. This method calls a property. If that property is not yet specified, this is shown.

For Sequenced Band compositions, only parallel is enabled, unless Renew Parallel Repeats is checked. Checking Renew Parallel Repeats activates some fields below including the Maximum Duration option, as well as Paced and Inter-repeat delay in the Advanced Options section.

Note: Properties are now displayed as blue highlighted objects with rounded corners rather than by their path as in prior releases. Use the Property Chooser as before to select any existing property or to create a custom property on-the-fly.

• The Property: prefix has been dropped from For Each and While, and the

Continuous repeat setting has been renamed Infinite. As in prior releases, these settings are reflected on the track surface.


122

Using Repeats with Multiple Track Locations Locations can be grouped together within a distribution by using the inner (leftmost) Plus icon—in order to have repeat settings that apply to all the locations in that distribution. Enable Repeats must be checked per distribution to be in effect.

Alternately, use the outer (rightmost) Plus icon to configure track locations each with its own repeat settings.

On the track surface, the Copy Count number shows an accurate Copy Count, but it turns into plain non-input text in any additional location (e.g. when there is more than one location in a distribution and one of them doesn’t use Dedicated Load Server).


123

As in prior releases, the Track surface displays information about its current configuration. When more than one distribution is configured in the Track lower panel, the track surface will show [various] as its Virtual Users state. Clicking Locations here will show “Specified in lower panel.”


124

Seed Data Improvements This release includes further refinement of the Seed Data support via the Custom Properties interface introduced in recent releases. Instead of one simple Advanced section to define the seed data property, as well as its slice, and its distribution; new, separate Allocation and Data sections are now provided with more sophisticated server allocation and row traversal types.

As a result of these new settings, more robust random distribution support is available in this release, alongside slicing improvements, and seed data structs (for both Seed Data: Repository and Seed Data: URL--when Column names are in the first row is checked). In prior releases, a single Advanced section consisted of the following fields:

• Maximum rows per server

• Row count

• Column count

• EOF Action: (Wrap, Null, Error, Stop Track and Drain, and Stop Composition)

• Use a Subset of the seed data: (Starting Row, Number of Rows) New Allocation and Data sections with settings to define seed data distribution replace the simpler Advanced section found in the prior release. All of the above settings are present in the new sections and are joined by new settings.


125

Tying Seed Data to a Virtual User(s) It is now possible to pair a given virtual user to a distinct slice of data via the new Property value is tied to the virtual user setting with a subordinate setting, New value per virtual user iteration.

• Property value is tied to the virtual user When checked, all Custom Properties within the same virtual user that reference the same seed data source will get the same single seed data value.

• New value per virtual user iteration When checked, then each iteration of the virtual user will get a new value. In the example above, the two Custom Properties in the two Messages will get the same value within each iteration of the virtual user, but each iteration will have a different value. Note: The top-level Save value for analytics setting was available in

prior releases. In Cloud Test, an instance of a Track is generally used to represent a “virtual user”. Parallel repeat renewal of the Track is generally used to represent “iterations” of a virtual user. If there are multiple references to the same seed data source from different Custom Properties within the same virtual user, each such Custom Property will get a different value from the seed data.


126

For example, if Custom Properties defined in two different Messages in the same Clip both refer to the same seed data source, each will get its own value from the seed data.

Server Allocation The Allocation section mixes a new Server allocation setting with the Maximum rows per server and Use a subset of the seed data settings present in the former Advanced section to arrive at a more powerful way to slice data. The “Server allocation” setting controls how the data is divided among the servers in a multi-server Composition.

• Separate slices of the seed data will be sent to each server Choose this to divide the data evenly among all of the servers, and send a unique slice of the data to each server.

Separate Slice Distribution To cause every instance of a property to get a random value, then set “Server allocation” to “Separate slices of the seed data will be sent to each server” in order to cause different sets (slices) of the data to be sent to each server. Each server then selects randomly from within its slice. For example, for a Composition that plays on 5 servers with the option Separate slices of the seed data will be sent to each server, if the “Maximum rows per server” is specified as 1,000, then if there are at least 5,000 rows in the source then 1,000 rows of seed data will be distributed to each server. However, if the source only contains 600 rows, then only 120 rows would be distributed to each server (600/5). If there are not enough rows of seed data to distribute at least one row to each server, the load will fail.

• The same single slice of seed data will be sent to all servers” is Choose this to single slice data that is then sent to all servers, and each server will be using the same seed data.


127

For example, with this setting all 5 servers would share the same single set of seed data (either 5,000 or 600 rows in the above examples).

Single Slice Distribution To cause every server to randomly select from the entire set of data, when it is not required that every value be used exactly once across all servers, then set “Server allocation” to “The same single slice of seed data will be sent to all servers” to cause the entire set of data to be sent to all servers. All of the servers will then be randomly selecting from the entire range of values.

Data Section The second new section brings together new row and column settings that replace the former row count and column count settings.

Column and Row Count Rows and columns are defined in a new way for this release. Row count is expressed as an integer, while column use is specified either as All or First x.

• As in the prior releases, Row count setting specifies the number of rows of seed data to distribute to each individual instance of the Custom Property. If the value is greater than one, then the Custom Property’s value will be an array value, one entry in the array for each row.

Note: If both “Column count” and “Row count” are greater than one, then the property value will be either a two-dimensional array, where the first index is the row and the second index is the column (if “Array” was chosen), or an array of structs (if “Struct” was chosen).


128

Row Traversal Types The “Row traversal type” setting specifies how each server will traverse its slice of the seed data.

• Sequential

The server traverses through it’s set of data sequentially. The “EOF Action” setting applies and indicates what to do if the end of the server’s set of data is reached.

• Random with replacement

Whenever a new value is needed to fill in a new property value, one is chosen randomly from the server’s set of data. The “EOF Action” setting does not apply, since each selection is just made randomly from the entire set of data for that server, and therefore the server can never “run out” of values.

• Random without replacement

Whenever a new value is needed, one is chosen at random and marked as used. Thus a unique value is chosen each time from the server’s set of data. Since in this case the server can run out of data, the “EOF Action” setting applies.

EOF Action As in the prior release, the EOF Actions specifies what to do if any server reaches the end of it’s slice of seed data during play of the Composition.

• Wrap

The server will start over at the beginning of its slice of seed data.

• Null

The values of the Custom Properties will be set to nulls. If according to the settings the property value is to be an array, it will be set to an array of nulls of the appropriate dimension and size.


129

• Error

The item containing the Custom Property will fail. Error handling will proceed according to the regular “Failure action” settings of the item, it’s parents, and the Composition.

• Stop Track and Drain

The current instance of the item’s parent Track on the current server (and everything in it) is stopped. No new repeats or parallel repeat renewals will occur for that Track on this server. Other instances of the Track on this server that already exist will continue to their normal end, but no new instances will be started on this server. In other words, if using the recommended approach of having each Track represent a “Virtual User”, and if using “Parallel Repeat Renewal” to cause each Virtual User to be automatically replaced when it ends, then this setting will cause no new Virtual Users to be created on this server when its seed data runs out.

• Stop Composition

The entire Composition (on all servers) will be stopped immediately (as if the “Stop” button had been pressed).

If multiple Custom Properties refer to the same seed data source, each such reference can have it’s own EOF Action, they do not all need to be the same.

Row Representation

The “Column count” setting specifies the number of columns to retrieve from the seed data. If the setting is one, the property is set to a single value. If the setting is greater than one, the property value type depends upon the option chosen for the “Row representation” setting.

• If “Array” is chosen the property is set to an array value, with one array member for each column in the seed data. If there are fewer columns in the seed data than specified in “Column count”, then the extra values in the array are set to null. The array will always be the size specified by “Column count”.

• If “Struct” is chosen, the property is set to a struct value, with one member for each column in the seed data. The “Struct” option can only be used when accessing seed data via URL and the “Column names are in the first


130

row” option is chosen, or when a Repository seed data object is being used.

Note that Struct is available with Seed Data: URL only when the Column names are in the first row box is also checked.

Assignment of seed data values to individual properties When the Composition is playing, any time that a new item starts playing, such as a Clip, Chain, or Message, if that item contains Custom Properties that reference seed data, the server on which the item is playing extracts the next available row from the slice of seed data that server has been allocated and sets the Custom Property’s value. Depending upon the settings, this may result in multiple rows being extracted for each Custom Property value.

Seed Data Result Events Various events are logged to the Result that indicate how the seed data was allocated and used. These events can be viewed in the “Result Details” and “Event Log” Dashboard widgets. All such events are logged at the “Composition” level and thus will be present even when the Composition is played in “Load” mode. When play starts, a “Seed Data distribution summary” event is logged that shows a consolidated summary of the slices distributed to each server involved in the Composition:

When play starts on each server involved in the Composition, a “Seed Data slices distributed” event is logged for the server that shows information about the slices distributed to that server:

When play completes on each server involved in the Composition, a “Seed Data usage” event is logged that shows how much data that server used from the slice, as well as whether it was necessary to wrap back to the start of the data (if allowed to do so by the Custom Property settings). In this example, the first slice wrapped twice as the slice size was a single row and a total of 3 rows were allocated:


131

Multiple Custom Properties If a Composition contains references to multiple seed data sources, each server will have a separate, independent slice for each such seed data source. If multiple Custom Properties refer to the same seed data source, they will all extract data from the same single slice (each will get different unique rows from that slice). If multiple Custom Properties refer to the same seed data source but specify different values for “Maximum rows per server”, the highest specified value will be used. If multiple Custom Properties refer to the same seed data source, all references must specify the same “Starting row” and “Number of rows” settings, if those settings are specified. No security (such as Basic Authentication or other user ID or passwords) is supported when accessing seed data via URL. However, Repository Seed Data objects can be encrypted with a password. At Composition load time, you will be prompted to enter any such necessary passwords. If multiple Custom Properties refer to the same seed data source, each such reference can have it’s own EOF Action, they do not all need to be the same.


132

Mobile App Targets This release introduces a new “Mobile/Native” target type to the Target Definition Wizard for use with TouchTest devices. In most cases, this target will be automatically created during Device Agent and Mobile App selection in the Clip Editor. However, mobile app targets can also be created and specified manually. Mobile app target creation can also be achieved via the Target Definition Wizard. Targets created manually can then be associated with the correct mobile device via its Device Agent and a Mobile App on that device.

Creating a Mobile App Target 1. Select Central > Targets and then click New. 2. Select the Native / Mobile App target type. 3. Click Next to advance to the Settings screen.


133

4. In the Settings screen, give the mobile target a name. 5. Optionally, select a Target Image appropriate for the mobile app.

6. Click Next to advance to the Finish screen and then click the Finish button

to complete target creation. When you do so the target is opened in a new Target Editor tab in CloudTest.

Mobile targets are considerably simpler than WebUI targets, since they lack Location Strategies and many of the other Ajax-related fields required for browser-based functional tests.


134

Use the remaining steps to ensure that your target is used for a given Device Agent / Mobile App combination.

7. Specify the Device Agent by name from among those configured. Note: Refer to the TouchTest Tutorial for a complete step-by-step

guide to registering your mobile device for use with the TouchTest Agent.

8. Specify the Mobile App to use from among those configured. Note: Mobile apps are configured by the Mobile Administrator. If

your mobile app doesn’t appear, discuss it with your Admin. Refer to the TouchTest Developer Guide for additional instructions.

9. Check the Terminate app at clip completion box to ensure that the mobile app is returned to its original state at the end of the test (or between clip plays in cases where the clip is used more than once in a given test composition).

10. Save the target after making any changes.


135



Test Execution

Tenant Recognition CloudTest now recognizes a tenant boundary with respect to server selection. Maestro will now only pick servers that belong to the tenant in which the given composition is about to run. This tenant boundary will eliminate some confusion for grid administrators, as well as during Location selection in compositions, as well as in Play Options. The Central > Servers list (and all other lists that show servers) now includes an additional Tenants column as part of the Maestro server’s new ability to distinguish between tenants.

Only eligible servers, based on tenancy, will appear in the CloudTest user interface. For example, such as in the visible list of Locations presented in the Composition Editor’s Play With Options dialog box (as well as Play Options displayed in Central via the main toolbar).


136


137

Test Analytics

Result Details Improvements This release introduces a new, more interactive version of the Result Details Dashboard with animated display during Play, numerous navigation improvements, better organization of information into sections that are interactive and can be minimized/maximized to improve readability, all of which we believe will result in faster analysis. As noted in the Double Click to Open section above, all items in the Navigation Tree and Cover Flow can now be double-clicked. In the case of clip elements, the parent clip will be opened to the selection, thus making it easier to use Result Details in the test editing process.

• The Result Details tree now launches expanded, with the latest container

open, and focus set on the last item (for a playing composition). The data display is updated to that last item as results become available.


138

The next item in the stream will first appear in the Navigation Tree (e.g. AppAction8) just prior to the focus changing to it. Then, the lower panel sections change to match the context of the newly selected item. Once play is completed, the last clip (where more than one exists) remains expanded. The Cover Flow has also been redesigned and is wider and dynamically sizes based on the buffered items shown.

• The Cover Flow also keeps the last item in focus, unless some user

interaction overrides it. Hovering over an item will pop it up to show the operation.

• Lower panel section display changes based upon the composition element selected.


139

At the Composition level selection show below, the General section, and Custom Properties appear. Additional sections will appear where applicable. Sections can be maximized and minimized easily by clicking the top-right section button.

For example, additional sections relevant to TouchTest™ AppAction’s such as those shown above will now appear in the Summary tab based on the context, including Waits and Validations, Custom Properties as well as for Inputs and Outputs. Click the Minimize/Maximize button to expand or collapse any section to use the available dashboard space.


140

General The General section display is context-sensitive and varies depending on the selected item. Error detail information for the given selection is displayed in the General, Error(s) section (where applicable).

In addition to posting error(s) where applicable, the General section displays the following summary information (for the given selection):

Composition, Band, Track, and Clip Level • Error(s) – At the composition level, error(s) are posted, if any exist.

• Status – At the composition level, status is displayed as: Playing, Completed, Stopped, Aborted, etc.

• Effective Duration – At the composition level, the effective duration of a test is posted.

• Avg. Response Time – At the composition level, the average response time of messages and/or actions is posted.

• Total Message Bytes – At the composition level, the total message bytes is posted. If the composition was run in Preview mode, this is zero for both.

• Effective Message Throughput – The throughput as a measure of message or action bandwifth is posted.

Target Level • Error(s) – At the target level, error(s) are posted, if any exist.

• Name – The name of the selected target.

• Avg. Response Time – At the target level, the average response time of messages and/or actions is posted.

• Total Message Bytes – At the targetn level, the total message bytes is posted.


141

Message / Action Level • Error(s) – At the message/action level, error(s) are posted, if any exist. • Operation – Where applicable, the type of operation is shown. • Name - The message/action name

Waits and Validations Both built-in and custom waits and validations are posted in the new Waits and Validations section.

When you click on a wait or validation, the appropriate wait or validation will be highlighted in the Input or Output box below.

Custom Properties Custom Properties for the item selected in either the Navigation Tree or Cover Flow are displayed in this section, which appears in context.


142

Input The Inputs of the given selection are displayed in this section. For example, for an HTTP message, the form will always have two fields – URL Values and Query String.

For a browser action or app action (as shown below), the Input form fields will vary.


143

Output The Output of the selected item(s) is displayed in this section. If custom outputs are defined for that action, they are displayed. A custom output specifies what is it to be shown in the Result Details widget for a given message or action. Typical outputs include captureScreenshot, outputElementText, and outputInnerHTML. A single message or action can have an unlimited number of outputs.


144

Target Demographics The Target Demographics section appears when a target item is selected in Result Details. This section provides target detail info that varies by target type. For HTTP(S) targets, the URL is posted:

For Browser Action targets, the Conductor, Browser, URL, Wait Timeout, Wait Interval, Mouse Speed, Allow Native Xpath setting, and the Fire Blur on Type setting are all posted:

For App Action targets, the iOS Device Agent’s name, the Mobile App name, as well as the terminate behaviors shown below are posted.


145

Waterfall Chart This release introduces a new system Waterfall Dashboard (shown below) that appears during dashboard selection.

The Waterfall Chart works in conjunction with the Clip Element Hierarchy chart and both must be present if they are to be used in a custom dashboard. The Select an item in the Clip Element Hierarchy list to display the load times for various paged clip elements on the right.


146

in the Widget Type > Basic section that shows incremental information for pages within a composition. This chart will display the load time based on the selected clip element those pages for the given columns: Domain, URL, Timeline, Sent, and Received.

Creating a Custom Waterfall Dashboard 1. To create a custom Waterfall Dashboard, open the Widget Selection

Panel, Widget Type, Basic section. 2. Select the Clip Element Hierarchy widget and drag it into the dashboard

workspace. 3. Select the Waterfall Chart and drag it into the dashboard workspace. This

chart is the final entry listed in this section.


147

Miscellaneous

Automatic Update Improvements for EC2 Additional Automatic Update improvements in this release will make the upgrade process smarter for Amazon EC2-launched instances. Instead of just modifying the current environment, updates will examine the EC2 environment's volume, and then update any other environments using that volume as well.

Jenkins Plug-in Error Message Improvements Jenkins and sCommand now include improved error messages than those presented in Gran Cabrio. Probably some algorithm like "find the first item that had an error, chronologically, and display all of the errors (text and details) for that item as multiline text

Support Browsers As of this release, Mozilla Firefox 3.6 is no longer supported. The updated browser list includes the following browser version:

• Internet Explorer 8 and 9

• Google Chrome 4+

• Firefox 4+

• Safari 4+


148

Bugs Fixed 28468: Unable to rename result Renaming a result by renaming it from the Results list in Central and using the method provided didn’t work as expected. 28319: Average Response Time is zero for clip level and higher The Average Response Time and other statistics weren’t being posted in browser recording compositions as expected. 26098: Invalid argument.; JS line 2370 An error occurred in Internet Explorer 8 due to an “invalid width.” 25944: Error Analysis Widget not showing all data The Error Analysis widget didn’t show the complete tree view of the errors while a test ran, and also didn’t permit tree navigation as expected. 25893: A Soap fault has occurred. faultString: For input string: "undefined" An error occurred invoking the Session Template Wizard after selecting a string to replace. 25715: Attempting to access Target cookies for a Clip without Targets that support cookies yields null pointer exception A Script attempting to access cookies via a WebUI target using a script, such as: $context.currentClip.targets[0].cookies = null; var cookies = $context.currentClip.targets[0].cookies; would be aborted with a "Null pointer" Java exception. 25685: Changing a Clip to Timed caused a new, empty Clip to be displayed Attempting to Change Clip Timing after adding an HTTP message to an unsaved clip incorrectly allowed timing to be changed; however, a new Clip Editor tab was unexepectedly opened as a result. 25681: No more waits can be added if last wait is removed If a wait in a noOP action was removed by the user, there would be no subsequent way to add any more waits via the user interface. 25679: Can't get target-level header definitions or property analytics to work


149

A problem with targets whose underlying XML had incorrect bindings would prevent property analytics and other features from working as expected. 25677: Clip Editor keeps closing my Clip with "deleted" message while debugging Opening a clip in a new tab from within a composition, and then adding the open clip to a new draft composition, would result in this error closing the Clip Editor tab. 25562: IP address didn’t show up in the IP address column The HTTP(S) recording wizard included an extraneous IP Address column that in some cases wasn’t populated at all. This column has been removed as of this release. 25258: Very difficult to view results when stopped in the debugger After stopping the Debugger, attempting to view the result in Result Details for the given item was impossible as the Result Details widget was constantly refreshing and closing up every expanded detail. 25199: WebUI Targets Didn’t Handle Preview Mode Properly When used with a WebUI target, Preview Mode didn’t work as expected. The WebUI test attempted to find a Conductor and to cause that Conductor to perform actions not in line with what is expected in a preview.


150