SOASTA Cloudtest™ Release Notes Build 4617.207

SOASTA CloudTest™ Release Notes Build 4617.207

July 11, 2011

Table of Contents

Build 4617.207...... 1 Bugs Fixed ...... 1 Build 4617.188...... 2 Bugs Fixed ...... 2 Build 4617.180...... 3 Bugs Fixed ...... 3 Build 4617.171...... 4 Performance Intelligence...... 4 Analytic Performance Improvements...... 4 Build 4617.167...... 5 Bugs Fixed ...... 5 Build 4617.148...... 6 Bugs Fixed ...... 6 Build 4617.142...... 7 Bugs Fixed ...... 7 Build 4617.128...... 8 Bugs Fixed ...... 8 Build 4617.118...... 9 Performance Intelligence...... 9 Exporting Large Result Sets...... 9 Bugs Fixed ...... 10 Build 4617.111...... 11 Bugs Fixed ...... 11 Build 4617.103...... 12 Bugs Fixed ...... 12 Build 4617.101...... 13 Test Creation...... 13 Improved Clipboard Support...... 13

Test Execution...... 15 Support for Proxies with User Name and Password...... 15 Bugs Fixed ...... 16 Build 4617.79...... 18 Test Creation...... 18 Change To The Default CSS Retrieval Check Box ...... 18 Miscellaneous...... 19 Maestro Performance and Status Improvements ...... 19 DNS Timeout Error Message ...... 19 Grid Manager > Automatically Delete Monitors Checkbox ...... 19 Bugs Fixed ...... 20 Build 4617.64...... 22 Miscellaneous...... 22 Bugs Fixed ...... 23 Build 4617.57...... 24 Test Execution...... 24 New Default Value for DNS Minimum TTL ...... 24 Bugs Fixed ...... 25 Build 4617.38...... 27 Miscellaneous...... 27 Change to Connection and Response Calculations ...... 27 Bugs Fixed ...... 28 Build 4617.36...... 29 Test Execution...... 29 Cloud Provider Account Support for IBM Development & Test Cloud ...... 29 Setup IBM as a Cloud Provider...... 29 Launching Test Servers Using IBM Development & Test Cloud ...... 31 Creating a Grid using IBM ...... 31 Bugs Fixed ...... 33

Build 4617.30...... 34 Bugs Fixed ...... 34 Build 4617.21...... 35 Bugs Fixed ...... 35 Build 4617.15...... 36 Bugs Fixed ...... 36 Build 4617.9...... 37 Bugs Fixed ...... 37 Build 4617.5...... 38 Test Execution...... 38 Support for Browser Driving using Firefox 4 ...... 38 Build 4617.3...... 40 Test Execution...... 40 Cloud Provider Account Support for GoGrid and Windows Azure (CloudTest Pro) ...... 40 Setup GoGrid as a Cloud Provider ...... 40 Setup Windows Azure as a Cloud Provider ...... 42 Launching Test Servers using GoGrid or Windows Azure ...... 45 Creating a Grid using GoGrid ...... 45 Creating a Grid using Windows Azure ...... 46 Miscellaneous...... 48 New User Administration Role...... 48 Performance Intelligence...... 49 New Widget Type > Charts Categories ...... 49 Enhancements ...... 51 Internet Explorer 9 Added to Supported Browsers...... 51 Cancel Button When Testing Monitor Connections (Case 16282)...... 51 Bugs Fixed ...... 52 Build 4567.18...... 54 Bugs Fixed ...... 54

Build 4567.7...... 55 Bugs Fixed ...... 55 Build 4567.5...... 56 Bugs Fixed ...... 56 Build 4567.2...... 57 Test Execution...... 57 Monitoring AIX Servers...... 57 Prerequisites...... 57 Creating an AIX Monitor ...... 57 View Analytics Using the Monitor Dashboard ...... 63 Adding the Monitor Summary Widget to a Dashboard...... 65 Adding a Monitor Container to a Dashboard...... 66 Enhancements ...... 70 Session Template Package Wizard, Clip Scanner Improvements...... 70 Build 4567.1...... 71 SOASTA CloudTest™...... 71 SOASTA CloudTest Pro™ ...... 73 Test Creation...... 74 Comments in the Clip Editor...... 74 Working with Comments in Icon View...... 74 Working with Comments in List View...... 76 Using the Rich Text Editor for Comments...... 77 Select Which Message to Retrieve the Value from in Session Template Package Wizard ...... 80 Do Not Retrieve Nested CSS Resources ...... 81 Composition Editor > Play Settings tab ...... 81 Test Execution...... 82 Setting up An Amazon EC2 Account...... 82 Cloud Provider Accounts...... 83 Create a Cloud Provider Account ...... 83

Setup Amazon EC2 as a Cloud Provider...... 84 Configuring Access Control ...... 84 Result Databases (CloudTest Pro)...... 86 Create a New Results Database...... 87 Deploying a Results Database...... 89 Composition Editor > Results Database Selection (CloudTest Pro)...... 90 Specifying a Results Database for a Monitor (CloudTest Pro) ...... 93 Recently Played, Results Database Column (CloudTest Pro)...... 93 Grid Manager (CloudTest Pro) ...... 94 Creating a New Grid ...... 94 Pause and Resume Ramp ...... 97 Renew Parallel Repeats...... 100 Renew Virtual Users on a Track ...... 100 Renew Parallel Repeats in Sequenced Clips in a Timed Band...... 101 Renew Parallel Repeats in Containers in a Manually Timed Clip ...... 102 Play with Options Dialog for Playlists ...... 103 Player Status – Servers List ...... 104 HttpHostOverride Setting Configurable at the Target Level...... 105 Miscellaneous...... 107 New playNumberBeforeRenewal Property ...... 107 New rampPause and rampResume Methods in Composition Object...... 107 Enhancements ...... 109 Scanning Huge Clip Runs Out of Memory (Case 16480) ...... 109 Increase Height of Clip Editor Properties Pane (Case 16396)...... 109 Spinner Drop-Down Should Be Multi-Line (15670) ...... 109 Server Drop-Down Should Become a Tree (15144) ...... 109 Known Issues...... 111 Stopping Results Database ...... 111 Bugs Fixed ...... 112

Build 4617.207

Bugs Fixed

19930: Exception: The security token is invalid Sometimes an "invalid security token" error would occur when opening a result. When this occurred then repeated attempts to open that result would still result in an error but other results could be opened. This also fixes case 19683.

Build 4617.188

Bugs Fixed

Debugging for “Service Unavailable” Error Additional debugging code has been added to detect an HTTP 503 Service Unavailable error of unknown origin that occurs in CloudTest.

Build 4617.180

Bugs Fixed

Retry 3 Times on "Input read failed" This error occurred in cases where the browser opened a connection to the server to POST data for an AJAX call but the data was never sent, and the connection was broken, resulting in a SocketExeption. 18638: Bandwidth Charts Not Correct with Dynamic Data Data related to the domain of a message URL was not always correct and affected the Bandwidth charts. The Clip Element Analysis and URL Analysis were already correct.

Build 4617.171

Performance Intelligence

Analytic Performance Improvements This release includes underlying improvements to decrease the performance impact of viewing dashboards and running monitors, which can be significant in some larger tests.

Build 4617.167

Bugs Fixed

19073: Dynamic Resource Incorrectly Tries to Use SSL If HTML Document Is SSL A clip element in a page that was set to retrieve resources dynamically and whose HTML Document was a Secure Sockets Layer (SSL) incorrectly tried to use SSL.

Build 4617.148

Bugs Fixed

19242: Single Quote Character in an Error Message Causes Problems for Error Analysis Widget A single quote contained in an error message caused problems in an error message in the Error Analysis widget. 19109: Clip Takes Over 5 Minutes to Scan A clip converted from a recording took inordinately long to scan using the Session Template Package Wizard due to backslashes in some "large" responses.

Build 4617.142

Bugs Fixed

18978: Unable to Retrieve Property Value for Use in Validation If a Message used validation against a Custom Property, and that Custom Property belonged to a container (such as a Chain or a Clip) of that Message, and that container repeated, sometimes either the wrong property value would be used (a value from a different repeat of the container) or the error "Item is not accessible due to scope" would occur.

Build 4617.128

Bugs Fixed

18983: Force Stop on Multi-Server Test Took Inordinately Long In some cases, using Force Stop in a multi-server test would take inordinately long because the abort request was not received by the underlying result writer as expected.

Build 4617.118

Performance Intelligence

Exporting Large Result Sets In prior releases with very large result sets the time to export was inordinately long in some cases. As of this release the default for the Central > Servers, Service: Results, Export.Result.ChunkSize.Excel setting has been changed to 200,000.

This change dramatically increases the batch size for exporting result data in Excel format, since it doesn't seem to cause any problems with memory usage and makes exports much faster.

Bugs Fixed

18707: Uncaught TypeError: Cannot call method 'removeChild' of null; JS line 615 This null error likely occurred when changing dashboards in the Play or Results tabs of the Composition Editor. 18696: Recording Is Empty Using packet-level recording always displayed a benign “Recording Is Empty” message. 18664: Search Facility from Central Broken The Welcome page, Find Out How, Search field made an obsolete call to the Google search engine.

Build 4617.111

Bugs Fixed

18657: Service Unavailable As of this release, additional retries are attempted when polling dashboard data. 18637: NullPointerException Adding the Bandwidth Widget to a Dashboard A null error occurred during dashboard creation. 18628: b is undefined; JS line 1723 This error occurred while working with a track in the Composition Editor. 18621: Need to Consolidate Monitor Data In cases where many test servers were launched, then it could cause a large number of incoming HTTP requests to the Results Service Writer on the database machine. That can cause other incoming HTTP requests, such as for getting dashboard data, to throw an error. 18367: A Soap fault has occurred. faultString: Exception: 0 This error occurred in the Central test clips list after a test clip was duplicated from an existing test clip.

Build 4617.103

Bugs Fixed

18625: Can’t Resize Columns The Resize icon no longer worked in Firefox 4.

Build 4617.101

Test Creation

Improved Clipboard Support As of this release, every window, tab, and frame that shows data, input, or output from a composition is fully selectable. This expanded support for Clipboard operations applies to fields in the Composition Editor, the Result Details widget, and in the Clip Editor. In prior releases, many CloudTest fields did not fully support selecting and copying text to the clipboard. In particular, copy and paste was unavailable in the Result Details widget’s Request output. All Request fields are now fully selectable.

In the Clip Editor, additional fields are now also selectable. For example, the Properties tab, Clip Properties node’s path (shown below).

Test Execution

Support for Proxies with User Name and Password This release introduces two new Maestro service settings for corporate proxies that require a user name and password. Both HTTPProxy.UserName and HTTPProxy.Password can be set via the Central > Servers, Settings tab when Maestro is selected in the Service list. Define the HTTP.Proxy.UserName to use when sending requests by entering a value.

Define the HTTPProxy.Password to use when sending requests by entering and confirming a password value. The password is masked when entered as shown below.

Like the HTTP.Proxy.Port setting in prior releases, thee two new settings are reliant on the definition of the HTTP.Proxy.Server setting. If no server is defined, then proxy, username, and password are not sent.

Bugs Fixed

18622: Monitor Service Unresponsive While Stopping a Grid Monitor In previous releases, monitoring as a whole could be blocked if a single monitor took a long time to start or stop. As of this release, a single monitor cannot prevent access to others, regardless of how long it takes. 18561: Filtering by Time at Dashboard Level Does Not Work Dashboard-level filters weren't being used in a widget if the dashboard filter wasn't also placed on the surface. 18547: Session Template Package Wizard Runs CPU to 100% Scanning a given test clip would hang mid-way and peg the CPU of the Main Instance in a given CloudTest environment. 18536: $ is not defined; JS line 1532 This error occurred while working with lower panel properties with the Fundamentals widget selected. 18535: AIX Monitoring Reports Memory Demographics Incorrectly The monitoring Demographics widget was incorrectly reporting “pagesize” as RAM. 18528: New Users Does Not Recognize New User Admin Privilege A user with the User Administrator privilege was unable to access the New User dialog box as expected. 18525: Clip Location Keeps As Root Folder If It Is Not Changed During clip conversion, if the location specified was some location other than “/” but was not manually changed by the user, then the clip was incorrectly placed in the root folder (as viewable in Central > Library). For example, this a non-root location is suggested by CloudTest if the recording being converted resided in a non-root folder. 18523: Clip Timing Type Doesn't Work An attempt to change a clip's timing type in Google Chrome, or to click anywhere inside the Clip Timing Type dialog box, resulted in this error. 18369: HTTPS Recording URL-encodes Query Strings In previous releases, the Conductor proxy would URL-encode any query string sent over HTTPS, during either a recording or a WebUI/AJax test. As of this release, the Conductor proxy leaves the original query string intact.

18367: A Soap fault has occurred. faultString: Exception: 0 This error occurred when duplicating a test clip. 18349: No Data Exported for Results - Zero Bytes Users of CloudTest appliances might not be able to export results depending on the makeup of the server list. 18347: Unable to Access Composition from Playlist Double-clicking to open a test composition from a playlist resulted in an empty tab stating that the composition didn’t exist. 18332: Performance of ISSE Expressions Compilation of ISSE expressions was taking a significant amount of time. Frequent use of them (such as multiple times in every message) would cause some performance issues as the number of virtual users or throughput rate increased. 18259: A result with ID '2893' does not exist in the Results Server database In some cases, deleted results were not conveyed in the underlying layer, resulting in this error. 18236: If a Test Is Aborted but Results Service Is Not Informed Then It Will Continue to Send Data If a test was aborted, then it was possible for some test servers to not be informed of the result, and also some data that is sent at around of the time of the abort to be lost. 18179: Threads Blocked in DNS Resolver Don't Unblock When Composition Stops Threads that were blocked in DNS didn’t resolve in a timely manner when the user stopped the test composition. 17759: Illegal Group Reference This error occurred while creating a new clip from a recording on a given message that contained "$" in its URL wasn’t escaped. This character is now escaped as expected. 16831: this.getClipElement(_cae6) is null; JS line 712 When a package was applied on a given clip and then that clip was duplicated, this script error would occur when messages in the duplicated clip were highlighted in the lower panel. As of this release, the clip element UID is unique per clip instead of per repository.

Build 4617.79

Test Creation

Change To The Default CSS Retrieval Check Box In previous releases, when converting a recording to a clip, if the user selected to have pages created and to have dynamic resource retrieval enabled, then the Do Not Retrieve Nested CSS Resources check box was not checked by default.

As of this release, this checkbox has been renamed to Retrieve Nested CSS Resources and is still unchecked by default under the same conditions. This changes the default behavior—CSS resources will not be retrieved.

Miscellaneous

Maestro Performance and Status Improvements This release includes improvements and fixes to status display and stop handling including: (1) The status text line now includes an indication of the state of inter-server communication (how many servers currently have active communication pending and how many are currently retrying due to failed communication). (2) Stop processing of multi-server Compositions was performing significantly more inter-server communication than was actually necessary. This did not directly cause any problems, as the redundant communications were handled correctly and ignored. However, if could sometimes exacerbate problems if there are widespread inter-server communication problems at the time, since it would generate more of an inter-server communication load than was actually necessary.

DNS Timeout Error Message As of this release, in the event of DNS timeouts, CloudTest will show the error message "DNS request timed out" instead of "java.net.SocketTimeoutException.”

Grid Manager > Automatically Delete Monitors Checkbox As of this release, the Automatically Delete Monitors checkbox is unchecked. CloudTest will retain the Grid’s default monitoring data by default.

Bugs Fixed

18210: Logout Causing Abort in Results Service Logging out of CloudTest while playing a composition inside the Composition Editor would cause the result to be deleted. 18163: Invalid argument.; JS line 650 This like timing or drawing error occurred in Internet Explorer 8. 18162: A result with ID '2468' does not exist in the Results Server database Deleting a result that has already been deleted threw an error. 18151: Session Template Package Wizard Problems with Backslash Escapes A problem occurred with session parameters in a Session Template Package when characters were unnecessarily escaped with a backslash. Such characters are stripped out as of this release. 18138: Exception: Current User/Tenant Does Not Have Permission to Read This error occurred when a user clicked the Central – Server Instances list. 18113: Fundamentals Error When Time Not Yet Set If the fundamentals are used to show the Start Time, End Time, or Elapsed Time and one of those times is not yet set, then an error could occur. 18101: Zoom Out Not Working Across All Widgets In a dashboard that consisted of multiple charts, if a chart was zoomed out, the other charts did not also zoom out as expected. 18100: Uncaught TypeError: Cannot call method 'checkKnownClipElements' of null; JS line 86 This error occurred when a user highlighted a parameter in a duplicated clip that had a Session Template Package applied. 18097: Uncaught TypeError: Cannot set property 'checked' of undefined; JS line 281 This error occurred while scanning a test clip in the Session Template Package Wizard. 18088: XPath Not Appearing in Google Chrome After selecting XPath as the retrieval method of a message in the Clip Editor lower panel, the Response Type was not displayed (as XML) in Google Chrome as expected.

18055: A Soap fault has occurred. faultString: The XML body of this repository object does not match the expected schema: cvc-complex- This error occurred when saving a test clip after double-clicking a clip element in a page, and then unchecking Static. 18028: i is undefined; JS line 26 This error occurred while scanning a test clip using the Session Template Package Wizard. 17284: Report Export: Missing Legend When Only One Item in Chart Charts that can have multiple series, but which only have data for one series, now display a legend in reports. 17806: "$ is not defined" This error occurred when popping-out an analysis widget during composition play.

Build 4617.64

Miscellaneous

Azure: Change Back to Regular HTTP Instead of HTTPS Due to unexpected latency between Test Servers in Windows Azure, HTTP is now used instead of HTTPS for inter-server communication.

Bugs Fixed

18112: Error Creating Cloud Provider Account If Location Name Is Already Taken An error occurred in cases where one or more automatically created locations for an IBM Cloud Provider Account existed.

Build 4617.57

Test Execution

New Default Value for DNS Minimum TTL The default value for "DNS Minimum TTL" has been changed to 1800 seconds (30 minutes). This mimics what web browsers do. For more information about the DNS Options section of the Target Editor, refer to Using DNS Lookups. For general information and targets and their types, refer to Target.

Bugs Fixed

Script Editor Controls Restored As of Build 4567.1, in order to provide support for Chrome and Internet Explorer 9, some Script Editor controls that appeared in prior versions were omitted. Those controls, including Search/Replace, Goto Line, and Undo/Redo are restored in this release. 18021: Result Export with Multiple Instances If a result was exported from an EC2 environment in which the database was on a separate server from the repository, then the exported result was not complete. This fixes most cases but does not cover the case where a remote results database does not have direct access to the repository. This will be fixed separately. 18018: Double Clicking A Server in Player Status - Servers Gets This Error Double-clicking any entry in the Central > Player Status - Servers node in order to display the Player Status Summary Info popup window threw this error. 18007: A Soap fault has occurred. faultString: The clip element '62e132f3- c0a9-a3b3-869a-2d96177628fc' does not exist This error occurred when a test clip was saved after a container (e.g. page) in the clip that had enough clip elements to require virtual scrolling was expanded and then that container was deleted. 18006: Failure Action Setting Doesn't Work Properly for Scripts If a Script's Failure Action was set to "record only", an error would still stop the Composition. 18004: Result Does Not Have a Running Results Service If a result had been deleted but a dashboard still had the result displayed, then an incorrect error about the results service not running would have displayed. A correct message about the result no longer existing will now display. 17994: Uncaught TypeError: Cannot call method 'SaveSheet' of null; JS line 515 A null error occurred while switching between SOASTA Central nodes. 17989: d is null; JS line 4798 This null error occurred while attempting to copy and paste a composition from one Library folder to another.

17957: EJSC.__months[mydate.getMonth()] is undefined; JS line 13 If a time filter is set for a chart and the "between" option is used and the start time and the end time are set to the same time, then an error would occur. 17953: Monitor Using Bundled Public Key Does Not Work When Started from a Composition In cases where a composition's properties specified the use of a monitor that in turn used the auto-generated public key, that monitor failed when the composition was played on a remote server. 17887: Placing Received Binary Data into a Custom Property via a PropSet Caused an Error Event in the Result If a received response contained binary data (for example it was an image), and that binary data was placed into a Custom Property via a PropSet, the following error event would appear in the Result: "Unexpected error while constructing event text for property set event." This would not cause any problems with the Composition play, nor cause the Composition to stop, but an extra error event that would appear. 17874: Composition "Failure Action" Setting of "Never halt on errors at the Clip level and above" Does Not Work Properly If the Composition-level "Failure Action" was set to "Never halt on errors at the Clip level and above", error handling during Composition play was not correct. For the most part, it erroneously behaved as if it was set to "Never halt on error, regardless of component settings". 17806: $ is not defined; JS line 2489 This error occurred when an analysis widget (e.g. Composition Analysis, Collection Analysis) was popped out while the composition was playing. 17408: Cannot Start Monitor from Composition When starting a monitor from a composition, the monitor would fail to start after the first time.

Build 4617.38

Miscellaneous

Change to Connection and Response Calculations As of this release, the underlying calculations that pertain to the "Time to Establish Connection” event in the Event Log, as well as to "Response Time", Time to First Byte (TTFB), and Time to Last Byte (TTLB) fields in one or more widgets, will no longer include the time spent waiting for a free connection from the underlying layer.

Bugs Fixed

17862: Uncaught TypeError: Cannot call method 'setTimeout' of null; JS line 281 This error likely occurred when attempting to set focus on the Find field when Find and Replace was loading. 17820: Error in Mapping A browser detection error occurred in the Activity Map widget when viewed in Firefox and Chrome browser versions. 17808: 'this.GetParent()' is null or not an object; JS line 396 This error likely occurred when the Composition Editor > Edit tab was clicked a second time before the composition had loaded. 17751: this.transforms is undefined; JS line 412 A window focus error occurred when opening a dialog box from a dashboard.

Build 4617.36

Test Execution

Cloud Provider Account Support for IBM Development & Test Cloud As of this release, CloudTest Pro users can create Cloud Provider Accounts for the IBM Development & Test Cloud.

By entering valid cloud provider account information for the given vendor, locations are automatically created and can then easily be used to provision servers via the Grid Manager.

Setup IBM as a Cloud Provider Use the following steps to enter your valid IBM credentials as a CloudTest Pro Cloud Provider Account. 1. Select Central > Cloud Provider Accounts and then click New to launch the Cloud Provider Account dialog box. 2. Enter a name for the account. This name is used in locations and also appears in the Cloud Provider Account drop-down in the Grid Manager. 3. Change the “Type” drop-down to “IBM Development & Test Cloud”.

4. Enter the IBM username. 5. Enter the password and enter it again in the confirmation field. 6. You can optionally configure an access control list to restrict which CloudTest users are able to launch IBM servers with this account. This step is not required if you are the only CloudTest user that will be launching servers. To configure an access control list: a. Select the CloudTest user you want to grant access to in the drop- down at the bottom of the dialog. b. Click the Add button. c. A new row will appear in the list above the drop-down. d. To allow the user to view the Cloud Provider Account object (but not necessarily use it), check the Read box. Note that even with this box checked, the user will not be able to view secure information. e. To allow the user to modify, delete, or re-name the Cloud Provider Account, check the Write box. f. To allow the user to launch new servers using this Cloud Provider Account, check the Use box. g. Repeat steps a-f for each CloudTest user that you want to grant access to. 7. Click OK to complete configuration of the Cloud Provider Account. The account information is validated by issuing a call to IBM. If the credentials

are invalid, a warning will be shown. You can choose to go back and correct the credentials, or continue.

8. After the Cloud Provider Account is saved, a new set of locations for IBM will be automatically created, if they do not already exist. These locations must be specified during Grid Manager configuration of grids (as described below). For more information about Locations, refer to Using Locations.

Launching Test Servers Using IBM Development & Test Cloud Once an IBM Cloud Provider Account has been created for use with CloudTest Pro, you are ready to create a grid that will provision servers via that cloud vendor.

Creating a Grid using IBM 1. Select Central > Grids and then click New. The Grid Manager appears with Step 1 General displayed. 2. Enter a Grid Name, and optionally, enter a description and a repository location for the new grid, and then click the right arrow. The Step 2 Server Instances page appears.

3. Change the “Location” drop-down to one of defined IBM locations. For example, IBM Colorado. 4. Specify the number of Test Server instances, and then optionally, define Results Server Instances. For most cases, the default “One for every 50 Test Server” instances” is acceptable for result servers. 5. By default, IBM servers will be launched using Silver instances. If you wish to use Gold instances instead, check the “Show Advanced Settings” box and change the Server Class drop-down from IBM Test Server (Silver) to IBM Test Server (Gold). Copper, Bronze, and Platinum instances are not supported at this time. 6. If you wish to combine the IBM servers with servers from other cloud providers (e.g. Amazon EC2 or Windows Azure), click the Add a Location box. Otherwise, click the right arrow icon to proceed. The Step 3 Summary and Deploy Tear Down page appears. Click the “Deploy Instances” button to begin launching servers.

Note: CloudTest Pro users may see an additional Grid Monitor Database field for those configurations where more than one Results Database is available. For more about using and launching grids, refer to Managing Grids.

Bugs Fixed

17853: Recording Directs to the Wrong Page If a recording included a URL with a query string, but no path (e.g. "http://www.foo.com/?query=keyword"), then the ? character would be converted to ?? in the Test Clip. 17819: Using Absolute Time VUs Area Chart Gets Cut Off A virtual user chart for an active test with the x axis in absolute time, was not showing points after the last change in virtual users. 17784: Filtering Count of Error Clips Causes Error If a user tried to split or filter by clip with the Count of Error Clips chart widget, an error would occur. 17740: Browser Add On Doesn’t Work with FireFox 4.0 In some cases, the Browser Recorder FireFox add-on didn’t work with the Firefox 4.0 release. 17462: BrowserDetect.isIE is not a function; JS line 5557 This error could occur in several situations, including when opening the "Mass Target Change" dialog or the "Targets and Operations" dialog. 17374: Time-filtered Collection Analysis Problems An error would occur if a user tried to use a time filter with the Clip Analysis, Collection Analysis, or Composition Analysis widgets. 17365: this.oldFrame.contentWindow.Global.oResultsViewer is undefined; JS line 361 This error likely occurred due to the widget changing before the Results Details had a chance to become initialized.

Build 4617.30

Bugs Fixed

17461: AIX Monitoring Not Working for CPU or Disk I/O System resource monitoring for CPU and Disk Input/Output didn't work as expected on AIX. 17456: "Use auto-generated public key" setting not being used in test connection Setting “Use auto-generated public key” as the authentication method for a System Resource monitor server group didn’t result in that private key being used when testing the connection (e.g. by pressing Test Connection). 17443: Charts Need to Increase Decimal Places to Show Different y Values If a Y axis value is in large units (eg. GB) but the differences in value between the smallest point and the largest is small, then the Y axis can show duplicate values. 17432: Redrawing Widget Error This error occurred while redrawing a widget after its properties were changed. 17428: Uncaught TypeError: Cannot read property '__chart' of null; JS line 1 Depending on timing it was possible for this chart property error to occur. 17371: this.__el is null; JS line 13 Depending on timing it was possible for a chart error to occur when switching charts. 16859: Requests Getting Put in Pages Out of Order During the creation of pages in a converted test clip, a logout message was placed into the middle of a page's resources, which resulted in some unexpected behavior during testing. As of this release, when looking for content types for paging, CloudTest will utilize the Content-Type of a redirected resource (e.g. a 302 redirect).

Build 4617.21

Bugs Fixed

17408: Cannot Start Monitor from Composition When starting a monitor from a composition, the monitor would fail to start after the first time. 17407: c is null; JS line 497 This error occurred while switching from the Conductors list to the Monitoring Server Group list in Central.

Build 4617.15

Bugs Fixed

GoGrid Server Failure Not Handled In cases where GoGrid servers failed, information from their API wasn’t informative. As of this release, CloudTest handles failed servers by permitting them to be torn down gracefully. 17358: Error Detail Not Showing Up Detailed error data such as that presented in Error Analysis was missing from test data.

Build 4617.9

Bugs Fixed

17330: subordinateWidgetLogics is undefined; JS line 4230 In some cases when working with widgets in a dashboard, this error could occur. Additional null checking has been added to track further occurrences of this error. 17322: "Minutes to ramp" Should Not Appear for a Serially-Repeating Item The Composition Editor’s lower panel clip properties, available either when Edit Properties is selected from a clip’s context menu or when a given clip is selected in the Properties list in the lower panel, erroneously showed the Minutes to Ramp field for a serially repeating clip. 17318: "Renew Parallel Repeats" Being Checked Automatically The Renew Parallel Repeats checkbox in track properties was automatically checked in some cases. For example, after opening track properties in the lower panel and then clicking the Virtual Users icon on the track surface. 17317: Improper Handling of Parallel Repeat Renewal for a Repeat Count of 1 If a parallel repeat had a repeat count of 1 (e.g. no parallelism) and the Parallel Repeat Renewal option was chosen, two repeats with a repeat index of zero would be created. This would cause exceptions and internal error messages when the Result was viewed in the Result Viewer. 17281: Monitor Does Not Have a Running Results Service If a results database is terminated while a chart widget is displaying data from that database, then an exception would occur. As of this release, an Error dialog box appears in this circumstance. 17234: java.lang.NullPointerException Applying an Individual Template A null pointer occurred while applying an individual template to a test clip. 17191: Auto-Report Times Out In some cases widgets did not load as expected during report creation, thus causing the report creation to time out.

Build 4617.5

Test Execution

Support for Browser Driving using Firefox 4 As of this release, support is provided for browser driving via FireFox 4 whenever a WebUI Target uses either "Any Browser" or "FireFox" in its Settings, Browser drop-down field.

Bugs Fixed

17304: Library View Isn't Loading In some cases, library view and other non-root nodes in the SOASTA Central list did not load. 17266: Session Template Fails to Replace Value in Clip Applying the session template value in this case failed to replace the actual value in the test clip as expected. 17230: Blind Apply Not Working for All Values in Clip Some actual values in a test clip were not replaced during apply of a session template.

Build 4617.3

Test Execution

Cloud Provider Account Support for GoGrid and Windows Azure (CloudTest Pro) As of this release, CloudTest Pro users can create Cloud Provider Accounts for both the GoGrid and Windows Azure cloud vendors.

By entering valid cloud provider account information for the given vendor, locations are automatically created and can then easily be used to provision servers via the Grid Manager.

Setup GoGrid as a Cloud Provider Use the following steps to enter your valid GoGrid credentials as a CloudTest Pro Cloud Provider Account. 9. Select Central > Cloud Provider Accounts and then click New to launch the Cloud Provider Account dialog box.

10. Enter a name for the account. This name is used in locations and also appears in the Cloud Provider Account drop-down in the Grid Manager. 11. Change the “Type” drop-down to “GoGrid”.

12. Enter the GoGrid API key. 13. To find your API key, log onto the GoGrid Management Portal (my.gogrid.com), click the “My Account” link in the upper-right corner, and then choose the “API Keys” tab. If the list is empty, click the “Add an api key” link on the left. See the GoGrid Customer Portal < API Keys Tab topic for more information. 14. Enter the Shared Secret. 15. You can optionally configure an access control list to restrict which CloudTest users are able to launch GoGrid servers with this account. This step is not required if you are the only CloudTest user that will be launching servers. To configure an access control list: a. Select the CloudTest user you want to grant access to in the drop- down at the bottom of the dialog. b. Click the Add button. c. A new row will appear in the list above the drop-down. d. To allow the user to view the Cloud Provider Account object (but not necessarily use it), check the Read box. Note that even with this box checked, the user will not be able to view secure information (such as the GoGrid shared secret). e. To allow the user to modify, delete, or re-name the Cloud Provider Account, check the Write box.

f. To allow the user to launch new servers using this Cloud Provider Account, check the Use box. g. Repeat steps a-f for each CloudTest user that you want to grant access to.

16. Click OK to complete configuration of the Cloud Provider Account. The account information will be validated by issuing a call to GoGrid. If the credentials are invalid, a warning will be shown. You can choose to go back and correct the credentials, or continue. 17. After the Cloud Provider Account is saved, a new Location for GoGrid will be automatically created, if it does not already exist. This location must be specified during Grid Manager configuration of grids (as described below). For more information about Locations, refer to Using Locations.

Setup Windows Azure as a Cloud Provider To use Windows Azure from CloudTest Pro, you will need the following: • The Windows Azure subscription ID. This is available in the Windows Azure Management Portal. • A Windows Azure management certificate for API access. This is not the same thing as a service certificate, which is used to enable HTTPS access to a Windows Azure hosted service. To learn more about creating a management certificate, see http://technet.microsoft.com/en- us/magazine/gg607453.aspx. • A dedicated hosted service and a storage account in each Region that you wish to use. These must be created in advance, using the Windows Azure Management Portal. Use the following steps to create a Windows Azure Cloud Provider Account.

1. Select Central > Cloud Provider Accounts and then click New to launch the Cloud Provider Account dialog box. 2. Enter a name for the account. This name will be populated to the Cloud Provider Account field in the Grid Manager (as well as the automatically created Location(s) that correspond to this Windows Azure account). 3. Change the “Type” drop-down to “Windows Azure”.

4. Enter the Windows Azure subscription ID. 5. Browse to locate the API Certificate file for upload. 6. If the API certificate is protected by a password, then enter the Certificate Password and then enter it a second time in the Confirm Password field. 7. For each Region that you wish to use: a. Check the box in the list on the left. b. On the right, enter the URL Prefix (a.k.a. “DNS Prefix”) of the dedicated hosted service to use for this Region, the dedicated storage account name, and the storage access key. 8. You can optionally configure an access control list to restrict which CloudTest users are able to launch Azure servers with this account. This step is not required if you are the only CloudTest user that will be launching servers.

9. To configure an access control list: a. Select the CloudTest user you want to grant access to in the drop- down at the bottom of the dialog. b. Click the Add button. c. A new row will appear in the list above the drop-down. d. To allow the user to view the Cloud Provider Account object (but not necessarily use it), check the Read box. Note that even with this box checked, the user will not be able to view secure information (such as storage access keys). e. To allow the user to modify, delete, or re-name the Cloud Provider Account, check the Write box. f. To allow the user to launch new servers using this Cloud Provider Account, check the Use box. g. Repeat steps a-f for each CloudTest user that you want to grant access to.

10. Click OK to complete configuration of the Cloud Provider Account. The account information will be validated by issuing calls to Windows Azure for each Region. If the credentials are invalid, a warning will be shown. You can choose to go back and correct the credentials, or continue. 11. After the Cloud Provider Account is saved, a new Location will be created for each Windows Azure region that you configured. This location must be

specified during Grid Manager configuration of grids (as described below). For more information about Locations, refer to Using Locations.

Launching Test Servers using GoGrid or Windows Azure Once a Cloud Provider Account has been created for use with CloudTest Pro, you are ready to create a grid that will provision servers via that cloud vendor.

Creating a Grid using GoGrid 7. Select Central > Grids and then click New. The Grid Manager appears with Step 1 General displayed. 8. Enter a Grid Name, and optionally, enter a description and a repository location for the new grid, and then click the right arrow. The Step 2 Server Instances page appears.

9. Change the “Location” drop-down to “GoGrid”. 10. Specify the number of Test Server instances, and then optionally, define Results Server Instances. For most cases, the default “One for every 50 Test Server” instances” is acceptable for result servers. 11. By default, GoGrid servers will be launched using 4 GB of RAM. If you wish to change this to 8 GB, check the “Show Advanced Settings” box and change the Server Class drop-down from “GoGrid Test Server (Large)” to “GoGrid Test Server (XL)”. 12. If you wish to combine the GoGrid servers with servers from other cloud providers (e.g. Amazon EC2 or Windows Azure), click the Add a Location box. Otherwise, click the right arrow icon to proceed.

The Step 3 Summary and Deploy Tear Down page appears. Click the “Deploy Instances” button to begin launching servers.

Creating a Grid using Windows Azure 1. Select Central > Grids and then click New. The Grid Manager appears with Step 1 General displayed. 2. Enter a Grid Name, and optionally, enter a description and a repository location for the new grid, and then click the right arrow. The Step 2 Server Instances page appears.

3. Select a Windows Azure location. Note : There may be more than one Windows Azure location IF more than one Cloud Provider Account exists for this vendor OR if multiple regions have been configured. For example, the selection above was an automatically-created location for the Windows Azure Asia East region, which was specified during Cloud Provider Account creation. 4. Specify the number of Test Server instances, and optionally, define Results Server Instances. For most cases, the default “One for every 50 Test Server” instances” is acceptable for result servers. 5. By default, Large servers will be launched. If you wish to use Extra-Large servers instead, check the “Show Advanced Settings” box and change the Server Class drop-down from “Windows Azure Test Server (Large)” to “Windows Azure Test Server (XL)”. 6. If you wish to combine the Windows Azure servers with servers from other cloud providers (e.g. Amazon EC2 or GoGrid), click the Add a Location box. Otherwise, click the right arrow icon to proceed. The Step 3 Summary and Deploy Tear Down page appears. Click the “Deploy Instances” button to begin launching servers.

Miscellaneous

New User Administration Role As of this release, an Administrator may grant another user the role of “User Administrator” via the User Editor dialog box. The User Administrator can create, update, rename, and delete users in the same tenant as themselves. To grant User Administrator rights, the Administrator will check the User Administrator box when creating or editing that user in Central.

Once granted rights, the User Administrator has the ability to manage other user accounts within a given tenant via Central > Users, and the User Editor. This can be done via the User Editor dialog box, or via the Users list lower panel settings.

Performance Intelligence

New Widget Type > Charts Categories As of this release, widgets that appear in the Widget Selection Panel under the Widget Types list, Charts category have been further divided into the following categories: Bandwidth, Counts, Durations, Errors, and Result Comparison.

Bandwidth The Bandwidth category includes: Bandwidth by Collection, Bandwidth Usage, Bandwidth Usage (received), Bandwidth Usage (sent), Bandwidth Usage per Domain, Bandwidth Usage Received per Domain, Bandwidth Usage Sent per Domain, Top N Messages by Avg Bytes Sent and Received, Top N Messages by Bytes Received, Top N Messages by Bytes Sent, Top N Messages by Bytes Sent and Received, and Top N URLs by Average Bytes Sent and Received.

Counts The Counts category includes: Bytes Received, Bytes Sent, Bytes Sent and Received, Clips Completed, Collections Completed, Collections in Progress, Collections Started, Count of Clip Starts, Count of Error Clips, Count of Simultaneous Clips, Messages by Pass Fail, Messages Sent by Domain, Receive Rate, Send Rate, Top N Collections by Bits Received, Virtual Users, and Virtual Users by Location.

Durations The Durations category includes: Average Clip Duration, Average Collection Completion Time, Average Min Max Response Times, Average Response Time, Average Total Time, Avg Collection Duration over Virtual Users, Collection Completion Time, Collection Time per Clip, Detail Response Time, Max Response Time, Max Clip Duration, Max Collection Duration, Minimum Clip Duration, Top N Clips by Average Duration, Top N Collections by Average Duration, Top N Messages by Average Response Time, Top N Messages by Average TTFB (Time to First Byte), Top N Messages by Average TTLB – TTFB (Time to Last Byte minus Time to First Byte), and Top N URLS by Average Response Time,

Errors The Errors category includes: Error Count, Errors by Domain, Errors by Type, Errors by Type over Time, Top N Collections by Error Count, Top N Domains by Error Count, Top N Errors, Top N Messages by Error Count, and Top N URLs by Error Count.

Result Comparison The Result Comparison category includes: Multi-Result All Response Time, Multi-Result Avg Response Time, Multi-Result Avg Response Time by Virtual Users, Multi-Result Avg Response Time by Virtual Users per Composition, Multi-Result Error Count by Virtual Users, Multi-Result Error Rate, Multi-Result Error Rate by Virtual Users, Multi-Result Failed Virtual Users by Virtual Users, Multi-Result Max Response Time, Multi-Result Min Response Time, d -Result Throughput by Virtual Users.

Enhancements

Internet Explorer 9 Added to Supported Browsers As of this release, Internet Explorer 9 has been added to the list of CloudTest supported browsers, while Internet Explorer 7 is no longer supported. Currently supported browsers include the following:

o Firefox 3.5 and above o Internet Explorer 8 and above o Safari 3 and above o Google Chrome 3 and above

Cancel Button When Testing Monitor Connections (Case 16282) In prior releases, in cases when a Monitor Server Group connection was tested (via the Test Connection button), if the test took an inordinately long time there was no means to Cancel other than to exit and begin again. As of this release, a Cancel button appears after Test Connection is clicked and the Spinner icon appears to indicate that the connection is being tested. Click Cancel at any time to stop testing the connection.

Bugs Fixed

17216: Exception When Saving a Results Database This error occurred in the Results Database Configuration dialog box when switching regions, but leaving the volume (from the previous region). 17164: Can't Export Result Data using SCommand This exception occurred in some cases when exporting a result, or monitor data, from SCommand, if the machine that ran SCommand was not also a CloudTest server. 17154: Virtual Scrolling and Auto Selecting TCP Messages in Recording Editor Broken When Show Packets was on, and an HTTP message in a packet-level recording was selected, the Recording Editor didn’t scroll to the relevant packets as expected. 17142: div.parentNode is null; JS line 1695 This null pointer error occurred while editing a message in the Message Editor. 17091: Uncaught TypeError: Cannot read property 'data' of null; JS line 7650 This error occurred when resizing the navigation panel of the Central list 17089: Uncaught Error: NOT_FOUND_ERR: DOM Exception 8; JS line 228 This error occurred in Chrome 10 after clicking in the pagination text-entry area at the top of the Recently Played list. 17087: Uncaught SyntaxError: Unexpected token ILLEGAL; JS line 196 Message names that included a single quote after any message name suffix (e.g. ".html'") would cause an error in the Results Details widget. 17042: oldCanvas.parentNode is null; JS line 1084 This timing error occurred during the deleting of a widget and its redrawing. 17041: A Soap fault has occurred. faultString: The XML body of this repository object does not match the expected schema: cvc-datatype This error occurred when entering a non-integer in the Clip Properties’ Display Width field in the Composition Editor. 17001: Parallel Repeat Renewal Is Not Handled Properly in Timed Clips The Renew parallel repeats setting sometimes assigned incorrect repeat indexes for items in a Timed Clip and in sometimes skipped indexes if the Timed Clip contained items in it before the one that repeats.

16908: Uncaught TypeError: Cannot read property 'm_iQueryOrder' of undefined; JS line 1866 A timing issue occurred with scrolling a recording when reloading in the Recording Editor. 16862: Can't Open Original Composition after Using Save As Using Save As to create a second composition from the current one, and then clicking the original composition’s name in the Compositions list returned the user to the tab showing the second composition. 16842: Long Property Names in Property Chooser Not Handled In some cases, long property names didn’t display as expected in the Property Chooser dialog box. 16825: com.soasta.common.xml2.XmlNode cannot be cast to org.apache.xerces.dom.ChildNode This error occurred when clicking OK after applying a session template via the Test Clips list, right-click command Apply a Session Template or Package command. 16816: Label for Analysis Export Is Wrong for Bytes Sent and Received Exports of analysis widgets using the Excel button on the widget, would mis-label bytes sent and bytes received as being in KB. However the values are given in raw bytes. 16815: Average Collection Duration Should Include Clip Name The Average Collection Duration, Maximum Collection Duration, and Bandwidth by Collection charts now include the clip name along with the collection name so that collections can be differentiated and more easily found. 16804: Message Names not showing up in Internet Explorer 8 Message names didn’t appear as expected in the Clip Editor while running in Internet Explorer 8. 16732: Can’t Tell Clip from Which An Error Originated In some cases it was difficult to ascertain the connection between an error and the clip from which it originated. As of this release, the Error Analysis widget now contains nodes containing the Clip name one level above the message name. 16683: d is null; JS line 3161 An invalid date filter caused a script error. 15197: Script Guide Example Missing Function for Cut and Paste The source example for the “Script 6: Generate Current Timestamp in “T” Format” in the SOASTA CloudTest Script Guide omitted the end function addZero(n).

Build 4567.18

Bugs Fixed

16843: “Open in Composition” Button Doesn’t Work As Expected Using the “Open in Composition” button on the Clip Editor toolbar would open the Composition Editor, but then immediately close it. 16821: Include Average TTFB and (TTLB-TTFB) in CSV and XML Exports The Average Time to Last Byte, and Average Time to Last Byte minus Time to First Byte metrics are included in the applicable result exports. 16809: _136e is null; JS line 2712 If the user clicked the "pop-out widget" button while there was no result selected, this script error would happen. 16743: Unable to See or Click Record Button The Record button was not visible within the Recording Editor if the screen height was too short (e.g. there was no scrollbar when there should have been). 16720: Include Assets That Have CSS Referrer in the Page with CSS A PNG file was not included in the same page as its referrer as expected if that referrer started with “www.”. 16697: Scrolling While Recording Finished This error occurred when scrolling in the Recording Editor as it finalized a recording.

Build 4567.7

Bugs Fixed

16726: Changing Subordinate Widget Type Doesn't Work Attempting to change a subordinate widget type for a selected, combined widget in a dashboard didn’t succeed—the widget redrew but the same combination remained. 16705: event is not defined; JS line 1718 This error occurred while adding a Custom Property to a new test clip. 16562: After Adding New Property Box Still In Edit Mode After entering a clip property and hitting return, the lower panel incorrectly remained in Edit mode.

Build 4567.5

Bugs Fixed

16677: java.lang.NullPointerException This error occurred while viewing a message detail in the Result Details dashboard.

Build 4567.2

Test Execution

Monitoring AIX Servers As of this release, CloudTest supports monitoring AIX (Advanced Interactive eXecutive) servers using the Choosing Monitoring Server Group or Monitor.

Prerequisites In order to monitor an AIX server, SOASTA CloudTest must be able to establish an SSH connection to the server. If you are not sure how to enable SSH support, consult your AIX documentation. If a firewall prevents SSH access, refer to Monitoring a Server Behind a Firewall

Creating an AIX Monitor

1. In the Central list, select Monitoring Server Groups.

2. Click the New Monitoring Server Group icon.

The Monitoring Server / Monitor wizard appears.

3. Select Monitor Server and then click Next. The Name and Connection step is shown.

4. Type a name for the Monitoring Server and optionally provide a description.

5. Finally, type the hostname or IP address of the server into the Host field and click Add. At least one host name must be added before clicking Next.

6. The Resources step is shown. Check "System Resources" and click Next.

The System Resources Type and Authentication step is shown.

7. Select AIX, then enter the SSH port number, the user name, and password.

8. Click the Test Connection button to test the connection.

9. If the connection is successful, you will see a pop-up containing some information about the AIX server. If the connection fails, you will see an error message explaining what went wrong. Click Next. 10. Review the information to be sure it is correct, and then check the "Create a new Monitor for this Monitor Server" box and click Finish.

11. Enter a name for the Monitor and an optional description, and then click Next.

Note: For CloudTest Pro users, an additional field appears to select from among the configured Results Databases.

12. The System Resource Selection step appears. This page is used to specify which statistics to monitor. For each statistic, you can choose a "sample rate" which determines how often that statistic will be collected.

13. Some of the statistics require an entry in the "Arguments" column. For example: • For the "Network MBits Received" and "Network MBits Sent" statistics, you can enter the name of the network interface you want to monitor (for

example, ni0 or hme1). If you do not enter a value, then the total of all network interfaces will be tracked (this excludes the loopback interface). • For the "Per Process CPU Percentage", "Per Process Memory Usage", "Per Process Thread Count", and "Per Process Virtual Memory" statistics, you must enter the name of a process (for example, httpd). In some cases, there may be multiple matching. When this happens, the total for all matching processes will be tracked. • The "Performance Counter" statistic is specific to Microsoft Windows and should be ignored. When you are finished, click Next. The Summary step appears.

14. Review the summary information to be sure it is correct, and then click Finish.

View Analytics Using the Monitor Dashboard 1. Once the Monitor is created, go to the Central > Server Resources > Monitors list to view it.

2. Click the View Analytics link for the new AIX Monitor (on the far right). 3. When you do so, the Analytic Dashboard opens for the selected monitor.

The configured system resources that were checked during monitor creation are displayed using the default Time Range “Last 5 minutes.” To change the Time, click the top left drop-down and select a new range.

Adding the Monitor Summary Widget to a Dashboard The Monitor Summary widget presents per monitor statistics for both aggregate, or group, and individual hosts in a test composition. For general instructions about adding dashboards to compositions, refer to Creating a New Dashboard. 1. From the Composition Editor > Play tab; open the dashboard to which the Monitor Summary widget will be added. 2. Click the Toolbar Widget Selection Panel button to toggle it on.

For new dashboards, the Widget Selection Panel appears automatically toggled on. 3. Expand the Monitors section if it’s not already expanded, and then select the AIX Monitor from the list.

4. Optionally, expand and select from the Aggregation Level list. The aggregation level filters the monitor data by level. For example, by all the members of a Monitoring Server Group, or by a host within such a group. When the aggregate level is changed, an existing Monitor Summary widget’s information will also change.

5. Expand the Widget Types > Monitors list, and then scroll to select Monitor Summary from the list of available monitor widgets.

6. Drag the Monitor Summary widget into the dashboard and drop it.

The Monitor Summary widget appears with columns for each of the resources specified in the System Resource Selection page of the Monitor wizard. These resources can be edited by opening the given monitor from Central > Monitors.

The screenshot below shows an AIX Monitor with three hosts, each of which has CPU Percentage, Memory Usage, Network MBits Received, Network MBits Sent, IO Kbytes Read, and IO Kbytes Written set to monitor. The rows in the widget present the Mininum (Min), Average (Avg), and Maximum (Max) values that pertain to each resource for the aggregate selection shown. Hosts are listed in the subsequent rows.

Adding a Monitor Container to a Dashboard Monitors can be added to CloudTest dashboards from within a test composition by clicking the Play tab and then the plus icon at the top of the dashboard.

If the Monitor Container will be the only widget in a dashboard, when creating a new dashboard opt to select the layout that spans all the columns in the dashboard (shown below).

The Monitor Container is a two-column widget in a container that permits defined monitors to be easily added to new or existing dashboards while maintaining a separate time range from the other widgets in a dashboard. Use the following steps to add the Monitor Container to a dashboard in the Composition Editor. 7. From within the Composition Editor > Play tab; open the dashboard where the Monitor Container widget will be added. 8. Click the Toolbar Widget Selection Panel button to toggle it on or off.

For new dashboards, the Widget Selection Panel appears automatically toggled on. Expand the Widget Type section if it’s not already expanded, and then expand Containers to select from Dashboard Container and Monitor Container—you can use containers to add a new or existing dashboard to an already existing dashboard or to add resource monitors to a dashboard.

QuickTime™ and a decompressor are needed to see this picture.

The Widget Type > Containers list is shown.

9. Drag the monitor container to the dashboard workspace to add it. 10. The Choose Monitor screen appears. Select a monitor from the list and then click the green right arrow.

11. The Specify Server and Group Filters screen appears. Select either the Monitor Server Group (Monitoring AIX, part 1 in the example below) or any of the configured hosts for the Monitor Server Group (for example, if more than one host will be in use select the group). Click the green right arrow when done.

The Monitor Container is added to the dashboard. Unlike when accessed from Monitors > View Analytics, the Monitor Container in a test composition is not started unless the test composition’s monitoring properties are set to select it. 12. To make this selection, click the Composition Editor > Edit tab. 13. Click the Properties tab in the lower panel and then click Composition. 14. Select the Monitoring tab and check the box for the monitor to start when this test composition plays.

Note that where more than one configured monitor appears in the Monitoring tab it may be selected. The Monitor dashboard shown below shows separate demographics for each of three configured monitors.

Enhancements

Session Template Package Wizard, Clip Scanner Improvements This release includes additional performance improvements when using the Session Template Package Wizard’s Scan Clip feature. These improvements are in addition to those included in Build 4567.1, and represent further speed gains that are especially notable for larger test clips. Additionally, the Clip Scanner now utilizes virtual scrolling similar to that recently deployed in the Recording and Clip Editors. While scanning continues, only those requests in the view are loaded into the list. Once user scrolling continues the additional information is added to the list.

Build 4567.1

SOASTA CloudTest™ SOASTA CloudTest is a dynamic web application that combines an innovative browser-based user interface with a distributed web services back-end that supports modular test creation, scalable test execution, and state-of-the-art performance intelligence dashboards. SOASTA CloudTest supports the ability for multiple server test engines to be "stacked" in order to scale out the SOASTA architecture for high volume load testing. Testing can occur behind the firewall and get executed from the cloud. CloudTest’s distributed web services include: SOASTA Repository retrieves and stores SOASTA CloudTest objects such as user scenario recordings, test clips, test compositions, play lists, users, targets, performance data, and test results. CloudTest Pro is a dynamic web application that combines a Web Browser UI with distributed web services to support test creation, test grid deployment, test execution and real-time performance analytics. CloudTest Pro Components include CloudTest Builder, CloudTest Pro Main Appliance, CloudTest Load Server, CloudTest Results Server, CloudTest Database Server, and CloudTest Conductor. Main is a physical or virtual software appliance that contains the CloudTest Web Application, the CloudTest Repository, CloudTest Results Server and CloudTest Database Server. The CloudTest Repository stores and retrieves SOASTA CloudTest Objects such as Test Compositions, Test Clips, Users, Targets, Scenario Recordings, and Cloud Account Credentials. The CloudTest Web Application includes CloudTest Builder, CloudTest Grid Wizard and CloudTest Analytics Dashboard. Builder is a web application that can be accessed from the following Web browsers: Internet Explorer 7.0 and later, Firefox 3.0 and later, Safari 4.0 and later and Chrome 8.0. CloudTest Builder includes the visual editors used to build performance, load, functional and Web UI tests for a modern web application. Grid Wizard is a fully automated capability to define, provision, detect, and correct cloud vendor errors, as well as to health check, and create cross-cloud vendor test server grids. Test Servers, which are not provided by SOASTA, may come from public and private clouds and from manually provisioned test servers. Maestro, the CloudTest Load Server, is a test engine, implemented as a massively multi-threaded service, which executes all or parts of a complex test composition. A single Maestro node is capable of sending and validating

responses to thousands of http messages per second. Multiple Maestros can be combined to each execute parts of a large load test that can scale out to tens and hundreds of thousands of virtual users. Maestros can be geographically distributed and single test compositions can run geographically distributed while still producing a single integrated set of test results and analytics. Analytics Dashboard means a display, which works in conjunction with the CloudTest Results Server to display interactive, real-time streaming analytic dashboards. The CloudTest Analytics Dashboard correlates multiple data streams from a distributed load test into a single test result synchronized and displayed on a synchronized timeline. Results Server means an in-memory, real-time analytic service (with persistence to disk) purpose-built to handle very large result sets and analytics from complex distributed load tests. A Results Server is included with CloudTest Pro Main Appliance. Additional Results Servers are deployed by CloudTest Grid Wizard to work in concert with the Load Servers to analyze the real time streams of data as the test runs. Database Server means a persistent data store for Repository’s metadata and the Results Server’s data. The Customer may deploy additional Database Servers to the Cloud, in order to optimize performance and bandwidth use. All of the components of the SOASTA CloudTest architecture are services-based and can be installed on different physical servers to maximize scale and performance. In addition, SOASTA CloudTest includes an installable multi-purpose software agent called Conductor that records HTTP(S) message traffic for test creation. Conductor can be used to records HTTP(S) message and/or browser actions for test creation, launch browser instances and plays UI/Ajax/Flex tests on client machines and monitor and collect performance metrics from SOASTA components and from the customer's infrastructure for inclusion in the analytics. With SOASTA CloudTest, you can test any Web application or service, no matter how complex. Testing ranges from functional/regression testing to load testing to performance testing as well as Web UI/Ajax testing—SOASTA CloudTest natively supports them all. CloudTest is browser independent, and supports a long list of industry protocols such as SOAP, REST, HTTP(S), HTML, Ajax, JSON, and more.

SOASTA CloudTest Pro™ SOASTA CloudTest Pro™ is a new appliance that delivers the world’s largest platform for testing the performance of Web and mobile applications. With SOASTA CloudTest Pro, testing teams have the full control to create, execute, and analyze performance tests native to SOASTA CloudTest, with the added flexibility to launch those tests on a single platform that integrates infrastructure inside a firewall with private cloud and public cloud. CloudTest Pro delivers all of SOASTA CloudTest’s existing features with the addition of the following items. Support for user-configured It will be necessary to create Amazon EC2 volumes using EC2 utilities, and then use the resulting volume ID(s) in CloudTest Pro’s Results Database dialog in order to create the cloud- based results database. After an instance is launched, use the Elastic Block Store > Volumes section of the AWS Management Console to setup volumes including a recommended size. Refer to the Using Amazon EBS-Backed AMIs and Instances chapter of the Amazon Elastic Cloud Compute Cloud User Guide. Once a volume ID has been acquired, refer to Create a New Results Database below. SOASTA recommends an initial volume size of 300GB to decrease the likelihood of running out of space. Disc utilization can be checked at frequent intervals using df -h from the command line (several times initially, and then minimally, once every 3-4 months). If utilization reaches 50%-75% of the volume size, additional space can be added (for example, 100GB-200GB additional for a 300GB volume). • Cloud Provider Accounts. See Cloud Provider Accounts in the Test Execution section. • Support for Multiple Result databases—CloudTest now includes the ability to configure and launch more than one Results Database so that results from Cloud-based tests can stay in the cloud. • Grid wizard support for complex multi-cloud and multi-cloud provider accounts deployments in the clouds.

Test Creation

Comments in the Clip Editor This release introduces the ability to add comments as new clip elements. Comments can now be added to test clips in both Icon and List views via the new Comment icon, which has been added to the clip element toolbar in the Clip Editor, as well as by selecting a clip element and then choosing Insert comment after. Like checkpoints, comments do not have lower panel properties. Comments cannot be placed in containers (e.g. pages, groups, and chains). To add a comment in either Icon or List view, first select the clip element you’d like the comment to appear after, and then click the Comment icon on the toolbar.

When you do so, the comment is placed after the selection in a new row and the comment is selected for editing.

Working with Comments in Icon View While in Icon view, a comment in edit mode will display a blue border that extends across the entire row. While in this mode, text can be entered as with any editor.

Enter the comment text by typing.

As in a text editor, pressing Return starts a new line in the comment.

When there is no selection, or another clip element in the test clip is selected, the blue border is replaced by a gray border around the comment.

When the comment is selected it can be dragged and dropped into a new location (shown below) just as with any other clip element.

When the comment is not selected or in edit mode, a gray border surrounds it.

• To return to edit mode, double-click the element. • To move a comment, single click to select it and drag it to the desired location.

Working with Comments in List View In List view, the comments also occupy their own row. As in Icon view, a blue border surrounds the entire row when in edit mode. The edit area indicated by white also spans across the row, stopping only at the blue border.

When the comment row is not in edit mode, the blue border fills in all but the area immediately surrounding the entered comment text. Unselected comment rows have no border.

Using the Rich Text Editor for Comments Whenever a comment is selected in either Icon or List view, the Etc. menu includes an additional command, Open Rich Text Editor.

The Rich Text Editor includes formatting controls similar to those found in most word processing applications.

Font Style – Bold, Italic, Underline, and Strikethrough buttons appear on the left of the toolbar. Superscript – Subscript and Superscript buttons appear in the second section fro the left. Numbers/Bullets – Insert/Remove Numbered List and Insert/Remove Bulleted List buttons appear in the third section from the left.

Alignment – Align Left, Center, Align Right and Justify buttons appear in the fourth section from the left. Hyperlinks – Link and Unlink buttons appear in the fifth section. Click the Link button to access the Link dialog box, which has three tabs. Enter the URL in the Link Info tab as shown below.

Click the Link dialog box, Target tab to select a target window from the drop down (e.g. a frame, a popup, new window, topmost window, same window, or parent window. Click the Link dialog box, Advanced tab to enter any additional attributes supported by the HTML anchor tag (e.g. id, name, language direction, etc.). Image – The Image button appears in the next section.

Click Image to access the Image Properties dialog box.

The Image Properties dialog box supports all the attributes supported by the IMG HTML tag. Font Name – Select a font name from the drop down. Select text in the editor before applying a font name. Font Size – Select a font size from the drop down. Select text in the editor before applying size. Font Color/Background – The Font Color and Font Background buttons appear second to last section from the right. Select text in the editor before applying a color or background Block Styles – Select a block style (e.g. Normal or Comments) from the drop- down in the last section.

Select Which Message to Retrieve the Value from in Session Template Package Wizard As of this release, the Session Template Package Wizard’s Scan Clip function returns results that can be expanded to show all prior messages in the scanned test clip that contain that same value. The user can then select the most relevant value from among those expanded. The original message is displayed as the parent row, with only one level of nesting for all the other messages.

To expand the row, click its corresponding arrow.

Once expanded, the children are all displayed in one level. The expanded row shows the same value selected that was already selected before expansion, in which case that value appears with its radio button selected. This level contains all of the other messages with this value. A radio button is displayed for each possible selection. The child row is where a given value originates. It allows the user to chose to extract not from the first response we find the value in, but from previous responses.

Do Not Retrieve Nested CSS Resources As of this release, the retrieval of CSS resources that are nested within first-level CSS resources can be turned off at the page, click, track, and composition levels by checking the new “Do not retrieve nested CSS resources” checkbox in the Resource Settings for the given level.

This feature permits the user to filter all of the assets retrieved from a CSS that was dynamically retrieved, but not to filter the CSS itself. In such cases, the CSS file and its contents are retrieved without further recursion of its named assets.

Composition Editor > Play Settings tab A new Play Settings tab has been added to the Composition Editor > Properties in the lower panel. The Play Settings tab appears when composition properties are selected in the Composition Editor, Properties list. Composition play mode and runtime settings that formerly appeared in the General tab are now located in the new Play Settings tab. CloudTest Pro users may see an additional Results Database drop-down in the Play Settings tab.

For Administrators, refer to Result Databases (CloudTest Pro) for more about how to setup CloudTest Pro for end-users.

Test Execution

Setting up An Amazon EC2 Account A prerequisite for using CloudTest Pro’s public cloud capabilities is to have an account with the Amazon Elastic Cloud Compute (EC2) Service. Once an EC2 account is established, use the AWS Management Console. Before proceeding with the Cloud Provider Accounts section below, ensure that the following information is available: • Creating an Elastic IP address As the Amazon EC2 documentation explains, “Elastic IP addresses are static IP addresses designed for dynamic cloud computing. An Elastic IP address is associated with your account, not a particular instance.” Refer to the Using Instance IP Addresses chapter of the Amazon Elastic Compute Cloud User Guide. • Opening certain ports in the EC2 firewall using the AWS Management Console. Basic port configuration can be done via the Launch Instances wizard in the AWS Management Console. CloudTest Pro only requires 80 and 443 be opened in the firewall. • Creating a volume (including a recommended size) It will be necessary to create Amazon EC2 volumes using EC2 utilities, and then use the resulting volume ID(s) in CloudTest Pro’s Results Database dialog in order to create the cloud-based results database. After an instance is launched, use the Elastic Block Store > Volumes section of the AWS Management Console to setup volumes including a recommended size. Refer to the Using Amazon EBS-Backed AMIs and Instances chapter of the Amazon Elastic Cloud Compute Cloud User Guide. Once a volume ID has been acquired, refer to Create a New Results Database below. SOASTA recommends an initial volume size of 300GB to decrease the likelihood of running out of space. Disc utilization can be checked at frequent intervals using df -h from the command line (several times initially, and then minimally, once every 3-4 months).

If utilization reaches 50%-75% of the volume size, additional space can be added (for example, 100GB-200GB additional for a 300GB volume).

Cloud Provider Accounts In CloudTest Pro, an additional Cloud Provider Accounts node appears in Central > Settings. Users with Cloud Provider Accounts can use this node to configure those accounts for use in testing (e.g. Amazon EC2 accounts) for use with CloudTest Pro. When Central > Settings > Cloud Provider Accounts is selected, the Cloud Provider Accounts list shows all currently configured cloud provider accounts.

Create a Cloud Provider Account The following instructions presume a valid cloud provider account. Refer to the instructions provided by the cloud vendor to obtain such an account. Once valid credentials exist create a corresponding cloud provider account in Central using the instructions below. 1. To create a new Cloud Provider Account, click New while Cloud Provider Accounts is selected in Central. 2. The New Cloud Provider Account dialog box appears. 3. Provide a Cloud Provider Account Name (this will appear in the Cloud Provider Accounts list in Central), as well as the necessary account configuration fields for the given cloud provider (shown in the following sections).

Setup Amazon EC2 as a Cloud Provider For Amazon EC2, the following fields appear: Name, Type (already selected), Access Key, Secret Key, and Confirm Secret Key.

Type – Specify the type (e.g. the cloud provider’s name) Key / I.D. – Enter a valid cloud provider key or I.D. Secret Key – Enter the secret key for the account Confirm Secret Key – Re-enter the secret key for the account

Configuring Access Control Once the account is configured, add users and/or tenants via the Security section of the New Cloud Provider Account dialog box.

Security – Configure permissions per User / Tenant. Each user (per tenant) can have Read, Write, and Propagate access control permissions.

• Read – The user/tenant can view the Cloud Provider Account but not edit or execute tests. • Write – The user/tenant can view and edit the Cloud Provider Account. • Use – The user/tenant can view, edit, and use the Cloud Provider Account to launch Grids and Results Databases. Add – Click Add to include users or tenants in the Security list and then configure permissions per user or tenant. Note: Once configured, the Cloud Provider Account name is specified prior to runtime via the Grid Manager > Server Instances page.

If the wrong credentials are used here deployments in the Grid Manager that use this cloud provider account will fail.

Result Databases (CloudTest Pro) Result Databases provide the ability to store results in the cloud. This precludes the need to write results “down” from a cloud-based grid to the appliance. Results can be exported for later appliance use, when necessary. In CloudTest Pro configurations, an additional Result Databases node now appears in Central > Server Resources. Result databases can be created, defined, and then deployed to mounted volumes in the cloud. The creation of a second results database in CloudTest Pro will result in additional fields for user selection in the Monitors dialog box, in the Composition Editor, and in Grid Manager. When Central > Server Resources > Result Databases is selected, the Result Databases list shows all currently configured CloudTest Result Databases—both local and in the cloud.

The following Result Database columns are listed: Name, Server, Location (if any), Volume (Local or the mounted volume specified in the Result Database Configuration dialog), Last Launched (the date this was result database was last deployed), Last Stopped, and Default (the default database gets a green check mark).

Create a New Results Database 1. To create a new CloudTest Results Database, click New while Result Databases is selected in Central. 2. The New Results Database dialog box appears with the General tab in display.

Define all of the following results database General fields: Name – Provide a Results Database name (this will appear in the Central > Result Databases list) Location – Provide a configured location of the correct cloud type for this results database. Use Central > Locations if such a location doesn’t yet exist. Description (optional) – Optionally describe the results database CloudType –Currently, only EC2 is supported for cloud-based result databases. Cloud Provider Account-- Select one of the EC2 cloud provider accounts configured in the Central > Cloud Provider Accounts node. Instance Size – Specify a main instance size from among the options shown below: Extra-Large (recommended default), High-CPU Extra-Large, High- Memory Double Extra-Large, High-Memory Extra-Large, High-Memory Quadruple Extra-Large, and Large. This instance size should correspond to the instance size configured on the given cloud provider. Region – Select an Amazon EC2 cloud region from the drop-down options. The current EC2 regions are: ap-southeast-1, ap-northeast-1, eu-west-1, us-east-1, us-west-1. Zone – Specify the EC2 cloud zone that corresponds to the selected region. Note that zones are different per regional selection.

Volume (required) – Specify the volume to use. For example, vol-684c3501. If a volume to specify has not been created, refer to Setting up An Amazon EC2 Account above. 3. Click the Advanced tab to review the following additional options. SOASTA recommends using the default for all Advanced tab settings. Only experienced users should modify these settings.

Pending Timeout—Specify the amount of time in seconds before a pending server request times out Allocation Max Retries—Specify the maximum number of times to retry allocating Allocation Retry Delay—Specify the delay in seconds between retries Check Server Max Retries—Specify the maximum retries to check a server Check Server Retry Delay—Specify the delay in seconds between checking a server Allow Extra-Large Servers—Check this box to allow extra-large servers 4. Click Save after making all the desired configuration selections. Once the result database is saved, an additional button appears.

5. Optionally, click Set as Default Results Database is this database will be the default for all users.

Deploying a Results Database Deployment of a results database will create that database in the cloud via the cloud provider account credentials, with the specified characteristics (e.g. server class, region/zone), and on the specified volume. 1. Create a new or open an existing results database. 2. Save the results database configuration.

3. When ready, click Deploy Instances to start the result database’s deployment process. The Status Log shows feedback as the deployment begins. Once Deploy Instances is clicked, the button becomes inactive and the Stop Deploying button becomes active. Click Stop Deploying to stop the deployment in progress.

Note: After the results database is created and then deployed, it is specified prior to runtime using the Grid Manager > Summary and Deploy / Tear Down page, Grid Monitor Database drop- down

Composition Editor > Results Database Selection (CloudTest Pro) CloudTest Pro end users will see additional options in the Play Results tab on systems where more than one Results Database has been configured by an Administrator. Result Database selection is done on the Composition Editor toolbar or in the Composition Editor > Play Settings tab and is composition wide.

Note: The Administrator determines the default result database for a CloudTest Pro configuration. For configurations where only one results database is configured, this toolbar drop-down, as

well as its corresponding lower panel property (shown below), doesn’t appear.

Where applicable, the Results Database drop-down will show the following options: • (Use Default) – The administrator set default will be used. • (Prompt_ - The user will be prompted at test runtime to select a results database. • Other named database(s) – The listed non-local database(s) will be used if a named database other than local is selected. For example, “Amazon EC2 (RDB1)” as shown below. • Local database – The local database will be used. The default behavior for result database selection is to prompt the user to select a Results database at runtime. However, the creator of the test can choose to Use Default, Prompt, or specify a database to use for the given test composition. Toolbar selection corresponds to the same setting found in the Composition properties > Play Settings tab > Results Database drop-down.

When Prompt is set, the following dialog box appears at runtime.

If the result name is also set to prompt, both options are set in this dialog box.

Note: If CloudTest Pro cannot contact the Results Database server when the test composition is played, an error notifies the user of this state.

Specifying a Results Database for a Monitor (CloudTest Pro) As of this release, when defining a monitor on a CloudTest Pro configuration where more than one results database exists, the Results Database field appears to permit the user to also specify which results database to use for the given monitor.

Monitors defined to use a given results database can be included in a test composition via the Composition Editor > Monitoring tab just as in prior releases.

Recently Played, Results Database Column (CloudTest Pro) As of this release in CloudTest Pro configurations where more than one Results Database exists, the Central > Recently Played list will display an additional new Results Database column, (on the far right). This column indicates the results database from which the given result originated.

Each CloudTest configuration has at least one results database, which is identified as the “Local database.” CloudTest Pro databases will display by the name assigned to them by their creator. Double-clicking a result in the Recently Played list will open that result regardless of the results database that wrote the given results.

Grid Manager (CloudTest Pro) Use the Grid Manager to manage elastic, cross-cloud computing resources, used in one or more load tests. Grids are the primary method of launching cloud-based servers in SOASTA CloudTest Pro. Grids are used to specify how many Test Server and Results Server instances to launch and from which locations, as well as how to recover from failures when allocating servers from the given cloud provider. Additionally, grids are used to Check the deployed Server instances to ensure that the cloud-service allocated Test Server and Results Server instances are running and operational. You can create one or more Grids for use in test compositions that deploy cloud resources and match those with SOASTA CloudTest server instances. Note: Although more than one grid can be created, CloudTest Pro doesn’t currently support deploying multiple grids at the same time. Prerequisites You must first define at least one Location before you can setup a Grid for managing cloud resources. The Location must be cloud-enabled. See Using Locations.

Creating a New Grid

1. To get started, select Central list > Grids and then click New. When you do so, the Grid Manager, General page appears. 2. Enter a Grid Name, an optional Description, and a Save Location (this is the repository folder where the grid settings will be stored), and then click the right arrow to go to Step 2. The Server Instances page appears. Use this page to define both Test Server and Result Server instances. Test Server instances are used to generate load, while Results Service instances are used to analyze and report on the test data in real-time. 3. Enter the number of Test Server instances to be created per Location, or click Select Composition(s) to automatically determine the number of Test Servers for you. 4. Define Result Server instances per Location either as a ratio to Test Server instances or as a Fixed Amount.

5. Check Show Advanced Settings. 6. Specify the Cloud Provider Account to use in this location via the Server Instances page > Show advanced settings > Cloud Provider Account drop-down (shown above). If the logged-on user has rights to only one cloud provider account, then no selection is necessary. If the logged-on user has multiple cloud provider accounts, this step is mandatory per location. Note: The Cloud Provider Account drop-down appears only if there is more than one account that the logged-on user has rights to see (according to the permissions defined in the Security section of the Central > Cloud Provider account dialog box). Grid deployment cannot succeed without a valid cloud provider account selection (per location if more than one location will be used). 7. With Show Advanced Settings still checked for the given location, perform any of the following optional tasks: • To specify a Pending Timeout for this location (e.g. different timeouts for different cloud providers). • Specify the time in seconds that the allocation request will timeout and be added to the retry queue For example, after an instance is requested, it stays in the "Pending" state until it's ready, and then changes to "Running". The time it takes to reach "Running" can be anywhere from 30 seconds

to 10 minutes. Use this setting to specify a cut-off in seconds. By default, any servers that don't reach "Running" after 5 minutes are thrown away, and are re-tried with new servers. • To specify server classes for Test Server or Results Service instances Configured Server Classes can be selected from the Server Class drop-down. See Server Classes. Depending on the Location selection, some additional options may appear. For example, for Amazon EC2 locations. Refer to Creating Grids for more information about configuring advanced fields in the Grid Manager. 8. Click the right arrow to go to the Summary and Deploy/Tear Down page appears.

Use the Summary and Deploy/Tear Down page to review grid settings before using Deploy Instances. For more about changing default settings on this page and its tabs, refer to Creating Grids, Step 9. CloudTest Pro users will see an additional Grid Manager > Summary and Deploy / Tear Down page, Results Database drop-down. This drop-down is used to

specify whether to write results from the deployed grid to a Local DB or to another configured results database. 9. Select the Results Database to use in a given deployment via the Results Database drop-down (shown above). Note: In cases where the specified results database, which will be used by the grid’s default monitoring mechanism, has been deleted—a notification is posted, the deployment is disabled, and prompt appears to specify a new database. 10. Click Deploy Instances when ready. A spinner will appear while the default monitor is starting and stops when that phase is complete. The green checkmark appears only when default monitoring is deployed. • The Grid Ready icon will display when the requested servers are allocated successfully. For example, 10 0f 10 allocated Test Servers were successfully checked and allocated below. • The Grid Ready, with failures icon will display whenever allocation and checking finished without getting all of the requested servers. For example, 2 of 10 requested servers were checked and allocated. • The Stopped icon will display whenever all of the requested servers failed to be allocated or the Stop Deploying button is clicked.

Pause and Resume Ramp Test composition ramps can now be paused and resumed on demand using the new Pause Ramp/Resume Ramp drop down on the Composition Editor toolbar. A “ramp” is in use when one or more tracks have been specified to repeat in parallel within a specified Start Interval (provided as milliseconds). Parallel repeats, or virtual users, on a track start at the Start Interval rather than all at once. Note: In CloudTest, each parallel repeat of a track represents one Virtual User. Pause Ramp When Pause Ramp is set, all tracks with a parallel repeat with an interval will stop adding new repeats during the pause. Therefore, the number of Virtual users will hold steady and not increase.

Resume Ramp When Resume Ramp is set, CloudTest resumes adding new Virtual Users at the specified rate.

Ramp Pause and Resume Ramp are independent from full pause and resume (by clicking Play). A full pause pauses everything. All users, even existing users, pause what they are doing, and all activity (sending, etc.) is paused. Ramp pausing and full pausing can be done at any time, and can overlap, partially or fully. A full pause is performed with the Pause button on the Player Control.

Changes to track properties on the track surface correspond to the track’s lower panel properties.

Note: In CloudTest, each parallel repeat of a track represents one Virtual User. Another multi-track composition with ramps on Track 1 is shown below.

The Virtual Users chart below shows a ramp beginning in a test, with more users added over time.

A paused ramp appears as a flat spot or plateau in the Virtual Users chart. The flat spot is where the pause was in effect—no new users were being added. Existing users were still active, however.

Renew Parallel Repeats A Renew Parallel Repeats checkbox has been added to the repeat settings for tracks, for sequenced clips in timed bands, as well as for containers in manually timed clips.

Renew Virtual Users on a Track The Track properties > General tab in the Composition Editor now includes a Renew parallel repeats checkbox. Check this option to replace each virtual user with a new virtual user (parallel repeat) as it ends. Using this checkbox causes the count of parallel repeats to remain the same.

The active tracks will always be the number in order to generate the specified load. Since each track repeat gets a new repeat number (index). For example, 100 Virtual Users have an index 0 to 99, but the 35th user ends, it is replaced by new user 100. Script authors should note that index computations might be affected if the track’s repeat index is in use in the track when this setting is applied.

Additionally, icons for track repeats have been updated for this release to better reflect the new option. - Serial - Parallel - Parallel with renew (shown on the track surface below) Once Renew parallel repeats is checked in the track’s lower panel properties, the Virtual Users icon (shown below) indicates that as shown below.

Charts that use the track repeat instance for calculations are also affected by this change. • In General Mode, the Result Details widget may show more track repeat instances than the number of repeats specified (the 101 from the above example). Once again, only 100 are active at a time. • The “Virtual User” count in the “Fundamentals” widget counts the total number of Virtual Users ever created, so it may show a higher number than the number you specified (101 in the above example). • The “Current Virtual User” count in the “Fundamentals” widget, and the Virtual User Chart widget will always show the number of Tracks that are actually active (100 in the above example).

Renew Parallel Repeats in Sequenced Clips in a Timed Band Renew parallel repeats can be set on sequenced test clips in a timed band by using the clip surface toggle option in timed clips. Using this checkbox causes the count of parallel repeats to remain the same. Choose this option to replace each clip with a new clip (parallel repeat) as it ends.

The icons for clip repeats have been updated for this release to better reflect the new option. - Serial - Parallel - Parallel with renew (on the clip surface shown above) Whenever the clip is set to Parallel Repeat with Renew Parallel Repeats using the surface toggle, the Clip properties > General tab in the Composition Editor lower panel Renew parallel repeats checkbox is also checked (and vice versa). Script authors should note that index computations might be affected if the clip’s repeat index is in use in the clip when this setting is applied.

If Fixed, serial is set, the Renew Parallel Repeats checkbox is inactive. This option doesn’t appear for non-fixed repeat types.

Renew Parallel Repeats in Containers in a Manually Timed Clip Renew parallel repeats can be set on containers (e.g. Pages, Groups, or Chains) in manually timed test clips.

The icons for container repeats have been updated for this release to better reflect the new option. - Serial - Parallel - Parallel with renew (on the container surface shown above) Repeats can be set via the clip surface for the given container by right clicking and selecting Add Repeat. Unlike renew parallel repeats in tracks or clips (in the Composition Editor), renew parallel repeats is set only using the Clip Editor lower panel container properties. To open these properties, select and double-click the container. When you do so, its properties tab opens in the lower panel display

Whenever the container is set to Parallel Repeat with Renew Parallel Repeats checked (as shown above), in the container’s properties > General tab the container’s repeat icon reflects that state. Script authors should note that index computations might be affected if the container’s repeat index is in use in the clip when this setting is applied. If Fixed, serial is set, the Renew Parallel Repeats checkbox is inactive. This option doesn’t appear for non-fixed repeat types.

Play with Options Dialog for Playlists As of this release, the Central > Playlists feature includes a “Play with Options” dialog box that allows playlist-level test server selection. This dialog box can be invoked from the Central > Playlist node by selecting and then right-clicking the playlist by name, or by selecting a given test composition in that playlist.

Once invoked, the Play with Options dialog box appears.

Player Status – Servers List The Player Status - Servers list now appears as a standard list like all the other Central flat lists. Like other Central lists, this list allows selection, column resizing, column sorting and has a unified look.

The Player Status - Servers list now includes a new right-click context command, Reveal in Server Instances. Select any item in the list and right-click to choose this command.

Once selected, the view changes to the Servers list with the item selected.

Individual server details can still be accessed using double-click, or right-click to choose Open, to access server detail information in the Player Status Server Info pop up. As in prior releases, click Refresh to update the information in this window.

HttpHostOverride Setting Configurable at the Target Level As of this release, the Target Editor > Advanced Settings section includes an additional field, HTTP “Host” Header Override, for HTTP targets. HTTP host

overrides are necessary for applications that only respond to requests that include the correct host name in the HTTP “Host” header. This is particularly relevant for applications in production environments that can be reached via an IP address but whose headers require a host name. This field is used in such cases to override the host header for all messages based on the given HTTP target.

• To use this field, open the given HTTP target in the Target Editor, navigate to the Advanced Settings section (shown above) enter the valid host name in the field as an URL and then save the target. In prior releases, HTTP host overrides were performed via placement of a script calling the HttpHostOverride system property in the relevant test clip. See How do I override the value of the HTTP “Host” header in requests? for more information about the script method of overriding a host name. Note that changing the value of the HttpHostOverride system property via this new method, or the deprecated script method, causes only the value in the HTTP “Host” header to be changed, and does not affect the actual location to which the requests are being sent.

The HTTP host override can also be set on the Target via the Composition Editor > Track (per track name) properties > HttpHostOverride field.

Miscellaneous

New playNumberBeforeRenewal Property As of this release, the playNumberBeforeRenewal property has been added to all objects that have the playNumber property, including: Band, Track, Clip, Chain, Group, Page, Script, Message, and BrowserAction. playNumberBeforeRenewal (read only – integer) If this item repeats in parallel with the “Renew parallel repeats” option enabled, this is an integer value that is the “playNumber” value of the original parallel repeat for this item if it has been renewed because a prior parallel repeat ended. If this is the original parallel repeat, the value of playNumberBeforeRenewal will be equal to the value of playNumber. If this item does not repeat in parallel, the value will be zero. For example, if parallel repeat number 5 of the item ends, but the Renew parallel repeats checkbox is enabled, the ending repeat will be replaced with a new, replacement repeat. The new repeat will have new repeatIndex and playNumber values (according to how many other repeats have already occurred). However, the playNumberBeforeRenewal value will still be 5 in this example. This property is always 0 for Composition, Checkpoint, Delay, and Target.

New rampPause and rampResume Methods in Composition Object As of this release, the Composition object includes methods to support rampPause and rampResume. void rampPause() Pauses the Composition’s “ramp up”, if any, as if the user had pressed the “pause ramp” button. Note that this merely starts the ramp pausing. The ramp will not actually be completely paused until all portions of the Composition on all servers have paused the ramp. This method may be called at any time, including when the ramp up is already paused or pausing, or when it is resuming. “Ramp up” is defined as Track parallel repeating with an “interval”. void rampResume() Resumes the Composition’s “ramp up”, if any, as if the user had pressed the “resume ramp” button. Note that this merely starts the

ramp resuming. The ramp will not actually be completely resumed until all portions of the Composition on all servers have resumed the ramp. This method may be called at any time, including when the ramp up is already resuming or is not paused. “Ramp up” is defined as Track parallel repeating with an “interval”.

Enhancements

Scanning Huge Clip Runs Out of Memory (Case 16480) Major performance improvements are included in the Session Template Package Wizard, Clip Scanner, particularly with respect to scanning large test clips.

Increase Height of Clip Editor Properties Pane (Case 16396) The default height of the Properties panel in both the Clip Editor and Composition Editor has been changed from 250 to 300 pixels.

Spinner Drop-Down Should Be Multi-Line (15670) In some cases in prior releases, long error messages were not displayed in a legible fashion in the Composition Editor > Spinner drop down. As of this release, long error messages are displayed as multi-line messages using 30% of the maximum width of the editor’s workspace with a maximum of 10 lines. Messages longer than fit the available display will be truncated.

Server Drop-Down Should Become a Tree (15144) In prior releases, when viewing the Composition Editor > Track > Server Location drop-down, similar server names were often indistinguishable from one another. For example, this was often the case with longer lists of grid-created server names that were unique only within their location. Such lists became difficult to read in the Composition Editor, where server name was organized only by server name (e.g. sans its location).

As of this release, server names are organized as a tree by location as well as server type. Improved Error Text for Azure 409 Errors (Case 13631) As of this release, Grid Manager journal text for Microsoft Azure 409 API errors has been improved. Error text now indicates: "The current status of the Azure deployment does not permit this change."

Known Issues

Stopping Results Database A Results Database is monitored like all servers in CloudTest. This monitor must be stopped before it will be possible to tear down the database instance. The monitor is listed in the Central > Monitors list under the name of the Results Database with the suffix “monitor.” To stop the monitor from Central, refer to Starting and Stopping a Monitor for instructions. To stop the monitor using scommand, refer to Starting and Stopping a Monitor from a Command Line.

Bugs Fixed

Some “Internal Error” Events during Composition Stop If the timing on the server happened to be just right when a Composition was stopping, some "Internal Error" events would get logged into the Result with the text "Maestro Service Internal Error (Recorder.1)", and some data could be missing for a handful of items in the Result due to this failure. 16518: Target Creation Wizard Doesn't Launch If Target Already Open After setting Page Resource Filtering to “Static Resources Only” in the Composition Editor, the setting wasn't selected the next time track or clip properties were opened. 16481: Server List Conflict after Reloading the Server List This error occurred when starting a monitor while a Grid was being deployed. 16477: Filtering of Static Assets Doesn't Work As Expected After setting Page Resource Filtering to “Static Resources Only” in the Clip Editor, the setting wasn't selected the next time clip properties were opened. 16388: this.m_oPageResources is null; JS line 1882 This error occurred while editing page resources in the Clip Editor. 16386: this.getClipElementXml is not a function; JS line 1499 This error occurred in the Message Editor while viewing message results. 16381: this.m_oRecording is null; JS line 1850 This timing issue could occur when reloading a filtered recording that triggers scrolling while the recording may be nulled out for a few milliseconds. 16353: Monitor Chart Scaling Monitor charts now properly re-scale the Y-axis appropriately for the points that are displayed. 16333: Exception: unexpected token: , near line 1, column 269 [SELECT CF.StartTime, CF.Duration, CF.Collection.Name, …] The detail charts for Collection Completion Time and Completion Time per Clip do not support splitting since they do not aggregate on anything. 16309: Tests That Don't Reach 'Stopped' Status Cause Dashboards to Run Forever If a test ends in a catastrophic way, such as taking down a grid while the test is running, then it could be left in a state of "Playing" or "Stopping". As of this

release, when that test is opened in a dashboard the status and end time of the test will be fixed. 16301: java.lang.NullPointerException This error occurred while selecting a single server from the "Aggregation Level" section in the dashboard tree. 16259: A Soap fault has occurred. faultString: The requested operation would result in an integrity constraint violation This error occurred on opening the Property Chooser while editing a message. 16196: Object doesn't support this property or method; JS line 849 This error occurred in the Composition Editor in Internet Explorer 7. 16194: Object doesn't support this property or method; JS line 844 This error occurred in the Composition Editor in Internet Explorer 7. 16189: Substring Parser Misses a Space When parsing a substring, the parser omitted one of two spaces that were in the response being parsed. 16180: Excel Sheet Wrong in Error Analysis When exporting to Excel from the Error Analysis widget, the column headings were not aligned correctly and there was an extra column. 16111: java.lang.NullPointerException A null pointer exception occurred in the Message Editor. 16110: MassTargetChangeDlg.oAvailableTargetsList is undefined; JS line 6 This error occurred while performing a Mass Target Change in the Clip Editor. 16068: Property Sets in Targets Not Handled Properly When Multiple Messages Used the Same Target If a Target contained "Property Set" definitions, and a single Clip contained multiple Messages using that same Target, and the Property Set did not specify a full path to the property, then the second and later Messages' Property references were resolved relative to the path of the first Message, rather than the current Message being played. 16067: (Untitled) This error occurred when setting the axis caption for an axis that had yet to be defined. The axis caption is no longer set in this case. 16059: java.lang.NullPointerException This null pointer exception occurred when a user without proper privileges attempted to adjust account activation status.

16047: Invalid Set-Cookie Header in Response - Cannot Parse Date A Set-Cookie header that used a comma caused an error. This variation of the Set-Cookie header is now handled. 16032: Invalid argument.; JS line 509 This mouse over error occurred in Internet Explorer 8. 16028: URL Analysis Widget Is Very Slow with Large Datasets The URL Analysis widget failed to initially load data in a timely manner in some larger tests. As of this release, the widget only gets the protocol (http/https) on initial load and gets the next level when that has been opened. 15991: The monitor spec '23,847' does not exist Attempting to start a monitor that no longer exists caused an error message. 15988: Composition Still Playing But Should Be Done If a Track in a Sequenced Band contained at least one Clip and the repeat count (Virtual Users) for that Track was set to zero, the Composition would never end. It could still be stopped manually with the Stop button, however. 15980: java.lang.NullPointerException An empty (null) list of IDs sent by the browser resulted in this error. As of this release, empty lists prompt no further action. 15922: A Soap fault has occurred. faultString: The XML body of this repository object does not match the expected schema: cvc-complex- This error occurred while trying to save a Session Template in the Session Template Wizard. 15886: When Using Substring, Option "one at random" doesn't work A page that contained a property was set to use the substring option “one at random,” but the first item was selected in each case, rather than an item from among those available. 15853: Pop up on charts still shows top even when it is bottom The pop-up hints on a "Top N" type chart would always say "Top 5" even if the chart showed Bottom. As of this release, the "Top" and "5" accurately reflect the chart settings. 15849: Virtual User Chart Stops When No Message Activity In a test with a long delay, the virtual user chart correctly showed no activity; however, it incorrectly failed to show updated time. 15840: JSONObject["x"] is not a number A null query value caused an error during correlation of results data. As of this release, if a query value is null it is skipped for correlations.

15833: Assigning Y-Axis to a Dataset An issue occurred in assigning a Y-axis to a dataset in some cases. Additional debug code has been added to resolve further occurrences of this issue. 15804: Widget Titles Not Maintained When Exporting As Image When a chart was exported as an image, the title did not reflect splits that had been applied to that chart. 15797: CSV Exports Don't Have Clock Time The CSV and XML Spreadsheet exports now contain a new column, which has the time in date and time format. 15793: CountCompleted Field Not in the Collection Aggregate CSV Export The Track, Clip, and Collection Aggregate exports to CSV and XML Spreadsheet now include the number of items completed. 15790: Exception: Exception: could not resolve property: Collection of: com.soasta.resultsservice.persistence.TimeAggregate [SELECT ( Changing the chart type (e.g. from bar to scatter, etc.) did not reset the filters for that widget as expected. 15762: this._el is null; JS line 1993 This error may have occurred as the user clicked "Save As", and then subsequently clicked again as the dialog closed, and just before the folder tree gets a chance to render. 15758: Password Change Doesn't Work When Password Contains a Pound Sign Entering a password that contained the special character "#" was accepted as a change, but didn't work when login was attempted. 15755: Status log choice is missing when play is "Starting" The Status Indicator in the top right of the Composition Editor didn’t include a link to the Status Log widget when in the Playing state, unlike all the other states. 15752: A Soap fault has occurred. faultString: The requested operation would result in an integrity constraint violation. This error occurred when copying and pasting messages between clips. 15721: The "Other" category should only group items that are smaller than 5%, not 10% The "Other" category in pie charts groups together items which are less than 5 percent. Previously it was 10 percent. 15715: More than one Server List entry matches hostname "ip-10-113-14- 46.ec2.internal", address "10.113.14.46" This error occurred when one or more server instances shared an IP address.

15710: The requested operation would result in an integrity constraint violation This error occurred due to a duplicate object name when saving monitoring servers. 15699: java.lang.NumberFormatException: For input string: "Infinity" If a chart was exported to SVG and the series names were long and the width small, then an error would occur. 15694: Auto Scroll Doesn't Work in Central Auto-scroll in the Central list didn’t work as expected in some cases. 15659: Web UI Test Fails on Internet Explorer 7 An error in the underlying test layer resulted in the failure of a WebUI test in Internet Explorer 7. 15653: Scrolling in Monitor Server Group Dialog with Multiple Servers When multiple servers were listed, the Monitor Server Group dialog box required both vertical and horizontal scrolling for the user to read. 15619: Pie Chart Legend Does Not Show in PNG Export An exported pie chart didn’t include the legend portion of the chart. 15601: A result with ID '3571' does not exist in the Results Server database. If a user deleted a result while the Results Details widget was displaying it, then an error could occur. 15489: Zooming On Widgets Doesn't Rescale Y Axis When zooming into a chart, if the lowest part of the zoom box was below the Y- axis, then the max Y value of the zoom box was being ignored on the zoom. 15417: Type mismatch; JS line 651 This error occurred when saving an imported composition and result. 15402: A Soap fault has occurred. faultString: The requested operation would result in an integrity constraint violation This error occurred in the Clip Editor after deleting clip elements and then clicking undo. For example, opening a test clip, deleting a message, opening a second message, and then clicking Undo on the toolbar reproduced this error. 14228: Chains Shown As Dependencies for a Script If a chain in a clip also contained a script, that chain showed up as a dependency of that script. 14034: WSDL Choice choosing first option in some cases Selected options were not set appropriately if the choice was implemented at the type level.