Human Performance Regression Testing Amanda Swearngin, Myra B. Cohen Bonnie E. John, Rachel K. E. Bellamy Dept. of Computer Science & Eng. IBM T. J. Watson Research Center University of Nebraska-Lincoln, USA P.O. Box 704 Lincoln, NE 68588-0115 Yorktown Heights, NY 10598, USA faswearn,
[email protected] fbejohn,
[email protected] Abstract—As software systems evolve, new interface features common tasks, necessitating longer mouse movements and, in such as keyboard shortcuts and toolbars are introduced. While fact, decrease efficiency. In addition, many toolbars with small it is common to regression test the new features for functional icons may add screen clutter and may decrease a new user’s correctness, there has been less focus on systematic regression testing for usability, due to the effort and time involved in human ability to discover how to accomplish a task over a simpler studies. Cognitive modeling tools such as CogTool provide some UI design. help by computing predictions of user performance, but they still Usability testing has traditionally been empirical, bringing require manual effort to describe the user interface and tasks, end-users in to a testing facility, asking them to perform tasks limiting regression testing efforts. In recent work, we developed on the system (or prototype), and measuring such things as the CogTool-Helper to reduce the effort required to generate human performance models of existing systems. We build on this work time taken to perform the task, the percentage of end-users by providing task specific test case generation and present our who can complete the task in a fixed amount of time, and vision for human performance regression testing (HPRT) that the number and type of errors made by the end-users.