Sample Preproduction Tester Feedback Package Page 2 of 5

Security and Software Assessment Services (SSAS) Preproduction Tester’s Feedback Form Sample

Note: These pages are intended to serve as an example. Please adapt them to meet your needs.

Preproduction Test of ______(software name and release ID)

Thank you for participating in the preproduction testing of the ______software.

General Instructions 1. Please complete the Preproduction Test Tasks below. (Either refer the user to the written documentation, or record here the information on computer system requirements such as: memory, hard disk space, operating system and version, any other software applications and versions needed such as spreadsheet software. For Web applications, note the browser, memory needed, and the URL of test site.) 2. Please answer the questions on the attached form--either by hard copy, or by opening up space in an electronic copy of this document. 3. Return your feedback to (name and e-mail) by (date).

Preproduction Testing Objectives  Evaluate software technical content  Evaluate software ease of use  Evaluate user documentation draft  Identify errors  (add any other specific goals)

Preproduction Test Tasks (example tasks) 1. Install the software. (does not apply to Web applications) 2. Complete Part 1: Functional Evaluation. These activities evaluate the software’s basic ability to generate the expected results, in the way you need them. Besides the listed functions, please use the software for other tasks you feel are important. 3. Complete Part 2: Overall Comments and Usability Evaluation. This section asks you to rate various aspects of the software using a 5-point scale. 4. Complete Part 3: Specific Questions. These questions request your feedback on specific issues about the software. 5. Please list any cosmetic errors you encounter, such as typos, missing graphics, etc. Note the screen name that appears in the top title bar for each item. 6. Return the completed feedback form

Known Problems

June 24, 2011 Page 1 of 6 Sample Preproduction Tester Feedback Package Page 2 of 6

(If there are known problems or omissions in the software, please list them here to save work for the preproduction testers.)

Preproduction Tester Feedback Form for ______(software name & release ID)

Your name: Your organization: Phone: E-mail: ______Date:

Please complete the form below. A convenient way to do this is to have an electronic copy of this form open at the same time as the software, and then move back and forth between the two applications. Thank you for your participation.

Part 1: Functional Evaluation Please follow these steps:

a) Complete the tasks below, utilizing your own actual data b) Use both the largest and smallest data sets or values that you expect the software to handle c) Record the values or data set sizes you used in the Comments area d) Compare the results generated by the software to the results you expected

Function to be Tester Date Comments, Ideas and Issues Tested Completed Function 1, see User's Guide Page xxx Function 2, see User's Guide, Page xxxx Function 3, see User's Guide, Page xxx

Please also test other software functions you feel are important. Add these with your comments to the table above.

Part 2: Overall Comments and Usability Evaluation Please complete the questions below, using the 5-point rating scale indicated. Please enter any comments in the space provided.

Usability Questions Rating Scale Specific Comments on Rating 1 - Strongly Disagree 2 - Somewhat Disagree 3 - Neither Agree nor Disagree (No Opinion) 4 - Somewhat Agree 5 - Strongly Agree Overall This software is useful to me 1------2------3------4------5 now, or it will be in the near future System output and reports are 1------2------3------4------5 useful and meet my needs Software has the capabilities I 1------2------3------4------5 need (note any exceptions) General impression is good 1------2------3------4------5 (why?) Software was easy to apply to 1------2------3------4------5 my specific situation Data entry effort is 1------2------3------4------5 manageable

Technical Content Appropriate technical and 1------2------3------4------5 scientific basis is used Uses proper terminology 1------2------3------4------5 Performs calculations correctly 1------2------3------4------5 (e.g. algorithms, unit conversions) Toolbars, menus, commands 1------2------3------4------5 and options are appropriate Labels and terms are accurate 1------2------3------4------5 and easy to understand (if not, what would you prefer?) Data formats are useful (if not, 1------2------3------4------5 what would you prefer?) I entered my own data and 1------2------3------4------5 received the expected results Boundary values (largest and 1------2------3------4------5 smallest expected values) were handled correctly

Software Operation Trouble-free operation 1------2------3------4------5 Easy to navigate within the 1------2------3------4------5 software Consistent and logical flow in 1------2------3------4------5 using the software

Easy to find what you are 1------2------3------4------5 looking for Software works as expected 1------2------3------4------5 (uses standard user interface features) Software works well within its 1------2------3------4------5 family of software applications (if known) Files import and export to 1------2------3------4------5 other needed applications Prints properly to a printer 1------2------3------4------5

Documentation Clearly describes software 1------2------3------4------5 purpose Organization is clear and 1------2------3------4------5 logical Examples show how to use the 1------2------3------4------5 main features (please list any features needing more explanation or examples) Tables, graphs & figures 1------2------3------4------5 provide sufficient guidance through major software options Do error messages clearly 1------2------3------4------5 direct the user to a solution? On-line help: was it easy to 1------2------3------4------5 find what you wanted? Included necessary technical 1------2------3------4------5 support information

Appearance Colors, symbols, and graphics 1------2------3------4------5 are legible and pleasing Looks professional 1------2------3------4------5 Correct spelling & grammar 1------2------3------4------5 Application windows have 1------2------3------4------5 consistent look and feel

Additional Comments

Part 3: Specific Questions

The EPRI project manager and developer should list any specific questions about the software that they would like input on, such as:

 technical approach  specific calculations  order of screens and steps  compatibility of the software with existing workflows  organization of menu items  quality of written explanations  suitability of certain labels,  terms or abbreviations used  annoying or frustrating experiences  other specific questions

THANK YOU !

June 24, 2011 Page 6 of 6