Software Testing
USABILITY TESTING CONTENTS I. Usability Testing 1. User Interface Testing: 2. What make a Good UI? a) Follow Standards or Guidelines b) Consistent c) Flexible d) Comfortable e) Correct f) Useful II. Testing for the Disabled: 1. Accessibility Testing 2. The law 3. Accessibility Features in a Software III. Web Site Testing: 1. Web Page Fundamentals 2. Black-Box Testing: 3. Gray-Box Testing 4. White-Box Testing 5. Configuration and Compatibility Testing 6. Usability Testing 7. Introducing Automation
Anuradha Bhatia
Software Testing
I. Usability Testing 1. Software is written to be used. 2. The product should always be tested with respect to design and tested with specifications. 3. So much time and effort is spent on the technology aspects of writing the code that the development team ignores the most important aspect of software— that someone will eventually use the stuff. 4. It really doesn’t matter whether the software is embedded in a microwave oven, a telephone switching station, or an Internet stock trading Web site. 5. Eventually the bits and bytes bubble up to where a live person will interact with it. 6. Usability is how appropriate, functional, and effective that interaction is with the use. 1. User Interface Testing: 1. The graphical means that are used to interact with a software program is called its user interface, or UI. 2. All software have user interface, which helps the user interact with application software in an easier way. 3. User interface tests the reliability of the product with the ease with which the user understands the product. 2. What Makes a Good UI? (Question: What makes a good user interface? 6 points explained = 6 marks)
1. A good user interface (UI) is a must for the first look about the product. 2. The user interface makes it convenient to use. 3. Many software companies spend large amounts of time and money researching the best way to design the user interfaces for their software. 4. They use special usability labs run by ergonomic specialists. 5. The labs are equipped with one-way mirrors and video cameras to record exactly how people use their software. 6. Everything the users (subjects) do from what keys they press, how they use the mouse, what mistakes they make, and what confuses them is analyzed to make improvements to the UI. a) Follows Standards or Guidelines 1. The single most important user interface trait is that software follows existing standards and guidelines—or has a really good reason not to. 2. If software is running on an existing platform such as Mac or Windows, the standards are set. 3. Apples are defined in the book Macintosh Human Interface Guidelines, published by Addison-Wesley, and Microsoft’s in the book Microsoft Windows User Experience, published by Microsoft Press. 4. Each book goes into meticulous detail about how software that runs on each platform should look and feel to the user. 5. Everything is defined from when to use check boxes instead of an option button.
Anuradha Bhatia
Software Testing
b) Consistent 1. Consistency within software and with other software is a key attribute. 2. Users develop habits and expect that if they do something a certain way in one program, another will do the same operation the same way. 3. In Notepad, Find is accessed through the Search menu or by pressing F3. 4. In WordPad, a very similar program, it’s accessed through the Edit menu or by pressing Ctrl+F. 5. The Wordpad and Notepad behave differentially the same scenario. c) Flexible 1. Users like choices—not too many, but enough to allow them to select what they want to do and how they want to do it. 2. The Windows Calculator has two views: Standard and Scientific. Users can decide which one they need for their task or the one they’re most comfortable using.
Figure 1
d) Comfortable 1. Software should be comfortable to use. 2. It shouldn’t get in the way or make it difficult for a user to do his work. 3. Researchers put in every effort to make software comfortable and easy to use. 4. It can be a difficult concept to quantify, but can look for a few things that will give a better idea of how to identify good and bad software comfort: 5. Appropriateness of the software should look and feel proper for what it’s doing and who it’s for. 6. Error handling a program should warn users before a critical operation and allow users to restore data lost because of a mistake. 7. People take the Undo/Redo feature for granted today, but it wasn’t long ago that these features didn’t exist. 8. Performance being fast isn’t always a good thing. If an operation is slow, it should at least give the user feedback on how much longer it will take and show that it’s still working and hasn’t frozen.
Anuradha Bhatia
Software Testing
e) Correct 1. The comfort trait is admittedly a bit fuzzy and often can be left to interpretation. Correctness, though, isn’t. 2. When you’re testing for correctness, you’re testing whether the UI does what it’s supposed to do. Figure 11.6 is an example of a UI that isn’t correct. f) Useful 1. The final trait of a good user interface is whether it’s useful. 2. Remember, you’re not concerned with whether the software itself is useful, just whether the particular feature is. 3. A popular term used in the software industry to describe unnecessary or gratuitous features is dancing bologna. 4. It doesn’t matter whether the dancing bologna is in a solitaire program or a heart monitor machine, it’s bad for the user and means extra testing for you. II. Testing for the Disabled: (Question: Explain the concept of testing for disabled = 4 Marks) 1. Accessibility Testing 1. A serious topic that falls under the area of usability testing is that of accessibility testing, or testing for the disabled.
Table 6.1 2. A 1994–1995 government Survey of Income and Program Participation (SIPP) used by the U.S. Census Bureau found that in 1994, about 54 million people in the country had some sort of disability. Table 6.1 shows the complete breakdown. 3. Although there are many types of disabilities, the following ones make using computers and software especially difficult: Visual impairments:- Color blindness, extreme near and far sightedness, tunnel vision, dim vision, blurry vision, and cataracts are examples of visual limitations. People with one or more of these would have their own unique difficulty in using software.
Anuradha Bhatia
Software Testing
Hearing impairments:- Someone may be partially or completely deaf, have problems hearing certain frequencies, or picking a specific sound out of background noise. Such a person may not be able to hear the sounds or voices that accompany an onscreen video, audible help, or system alerts. Motion impairments:- Disease or injury can cause a person to lose fine, gross, or total motor control of his hands or arms. It may be difficult or impossible for some people to properly use a keyboard or a mouse. For example, they may not be able to press more than one key at a time or may find it impossible to press a key only once. Accurately moving a mouse may not be possible. Cognitive and language:- Dyslexia and memory problems may make it difficult for someone to use complex user interfaces. 2. The law 1. Fortunately, developing software with a user interface that can be used by the disabled isn’t just a good idea, a guideline, or a standard—it’s the law. 2. In the United States, three laws apply to this area: a) The Americans with Disability Act states that businesses with 15 or more employees must make reasonable accommodations for employees, or potential employees, with disabilities. The ADA has recently been applied to commercial Internet Web sites, mandating that they be made accessible. b) Section 508 of the Rehabilitation Act is very similar to the ADA and applies to any organization that receives federal funding. c) Section 255 of the Telecommunications Act requires that all hardware and software that transfers information over the Internet, a network, or the phone lines be made so that it can be used by people with disabilities. If it’s not directly usable, it must be compatible. 3. Accessibility Features in a Software 1. Software can be made accessible in one of two ways. The easiest is to take advantage of support built into its platform or operating system. 2. Windows, Mac OS, Sun Java, and IBM OS/2 all support accessibility to some degree. Software only needs to adhere to the platform’s standards for communicating with the keyboard, mouse, sound card, and monitor to be accessibility enabled. 3. The various types of keys are a) Sticky Keys allows the Shift, Ctrl, or Alt keys to stay in effect until the next key is pressed. b) Filter Keys prevents brief, repeated (accidental) keystrokes from being recognized. c) Toggle Keys plays tones when the Caps Lock, Scroll Lock, or NumLock keyboard modes are enabled. d) Sound Sentry creates a visual warning whenever the system generates a sound.
Anuradha Bhatia
Software Testing
e) Show Sounds tells programs to display captions for any sounds or speech they make. f) These captions need to be programmed into your software. III. Web Site Testing: 1. Web Page Fundamentals 1. In the simplest terms, Internet Web pages are just documents of text, pictures, sounds, video, and hyperlinks. 2. Web site that demonstrates many of the possible Web page features. 3. A partial list of them includes a) Text of different sizes, fonts, and colors Graphics and photos. b) Hyperlinked text and graphics c) Varying advertisements d) Drop-down selection boxes e) Fields in which the users can enter data. 2. Black-Box Testing: (Question: Explain how black box testing is done for web site testing. 4 points – 1 mark each = 4 Marks)
1. The easiest place to start is by treating the Web page or the entire Web site as a black box. 2. Check for the text, graphics, hyperlink and their correctness in the implementation. a) Text Web page text should be treated just like documentation and tested “Testing the Documentation”. Check the audience level, the terminology, the content and subject matter, the accuracy—especially of information that can become outdated—and always, always check spelling. b) Hyperlinks Links can be tied to text or graphics. Each link should be checked to make sure that it jumps to the correct destination and opens in the correct window. If the specifications for the Web site are not there, need to test whether the jump worked correctly. c) Graphics Check for, all graphics load and display properly? If a graphic is missing or is incorrectly named, it won’t load and the Web page will display an error where the graphic was to be placed. If text and graphics are intermixed on the page, make sure that the text wraps properly around the graphics. Try resizing the browser’s window to see if strange wrapping occurs around the graphic. d) Forms Forms are the text boxes, list boxes, and other fields for entering or selecting information on a Web page. Figure 13.5 shows a simple example from Apple’s Web site. It’s a signup form for potential Mac developers. There are fields for entering your first name, middle initial, last name, and email address.
Anuradha Bhatia
Software Testing
3. Gray-Box Testing 1. With gray-box testing, it’s the line between black-box and white-box testing. Still test the software as a black-box, but supplement the work by taking a peek at what makes the software work. 2. Web pages lend themselves nicely to gray-box testing. Most Web pages are built with HTML (Hypertext Markup Language). 3. You had to embed markups, sometimes called field tags, in the text. For example, to create the bolded phrase This is bold text. enter something such as this into your word processor: [begin bold]This is bold text.[end bold] 4. To create the line in HTML you would enter This is bold text. 4. White-Box Testing 1. Dynamic content is graphics and text that changes based on certain conditions, for example, the time of day, the user’s preferences, or specific user actions. 2. It’s possible that the programming for the content is done in a simple scripting language such as JavaScript and is embedded within the HTML. This is known as client-side programming. 3. Apply gray-box testing techniques when you examine the script and view the HTML. 4. For efficiency, most dynamic content programming is located on the Web site’s server; it’s called server-side programming and would require you to have access to the Web server to view the code. 5. Database-Driven Web Pages, many e-commerce Web pages that show catalogs or inventories are database driven. 6. The HTML provides a simple layout for the Web content and then pictures, text descriptions, pricing information, and so on are pulled from a database on the Web site’s server and plugged into the pages. 7. Programmatically Created Web Pages are Web pages, especially ones with dynamic content, are programmatically generated, the HTML and possibly even 8. The programming is created by software. A Web page designer may type entries in a database and drag and drop elements in a layout program, press a button, and out comes the 9. Security Web site security issues are always in the news as hackers try new and different ways to gain access to a Web site’s internal data. Financial, medical, and other Web sites that contain personal data are especially at risk and require intimate knowledge of server technology to test them for proper security.
Anuradha Bhatia
Software Testing
5. Configuration and Compatibility Testing (Question: Explain Configuration and compatibility testing = 4 Marks)
1. Hardware Platform:- Is it a Mac, PC, a TV browsing device, a hand-held, or a wristwatch? Each hardware device has its own operating system, screen layout, communications software, and so on. Each can affect how the Web site appears onscreen. 2. Browser Software and Version:- There are many different Web browsers and browser versions. Some run on only one type of hardware platform, others run on multiple platforms. Some examples are Netscape Navigator 3.04 and 4.05, Internet Explorer 3.02, 4.01, and 5.0, Mosaic 3.0, Opera, and Emacs. Each browser and version supports a slightly different set of features. A Web site might look great under one browser but not display at all under another. Web designers can choose to design a site using the least common denominator of features so that it looks the same on all of them, or write specialized code to make the site work best on each one. How would this impact your testing? 3. Browser Plug:- In Many browsers can accept plug-ins or extensions to gain additional functionality. An example of this would be to play specific types of audio or video files. 4. Browser Options:- Most Web browsers allow for a great deal of customization. Figure 13.7 shows an example of this. You can select security options, choose how ALT text is handled; decide what plug-ins to enable, and so on. Each option has potential impact on how your Web site operates—and, hence, is a test scenario to consider. 5. Video Resolution and Color Depth:- Many platforms can display in various screen resolutions and colors. A PC running Windows, for example, can have screen dimensions of 640×480, 800×600, 1,024×768, 1280×1024, and up. Web site may look different, or even wrong, in one resolution, but not in another. Text and graphics can wrap differently, be cut off, or not appear at all. The number of colors that the platform supports can also impact the look of your site. There can be as few as 16 colors and as many as 224. Could your Web site be used on a system with only 16 colors? 6. Text Size:- Did you know that a user can change the size of the text used in the browser? Could your site be used with very small or very large text? What if it was being run on a small screen, in a low resolution, with large text? 7. Modem Speeds:- Enough can’t be said about performance. Someday everyone will have high-speed connections with Web site data delivered as fast as you can view it. Until then, you need to test that your Web site works well at a wide range of modem speeds. 6. Usability Testing 1. Usability and Web sites are sometimes mutually exclusive terms. 2. You’ve no doubt seen pages that are difficult to navigate, outdated, slow, or just plain ugly. Not surprisingly, these sites were probably never seen by a software tester.
Anuradha Bhatia
Software Testing
3. A programmer or someone with little or no design experience (or maybe too much design experience) created the pages and uploaded them for the world to see without considering how usable they were. 4. The top ten mistakes of web designing. (Question: Explain the top ten mistakes made in web development = 4 Marks)
a) Gratuitous Use of Bleeding-Edge Technology. b) Scrolling Text, Marquees, and Constantly Running Animations. c) Long Scrolling Pages. d) Non-Standard Link Colors. e) Outdated Information. f) Overly Long Download Times. g) Lack of Navigation Support. h) Orphan Pages. i) Complex Web Site Addresses (URLs). j) Using Frames. 7. Introducing Automation 1. The simple act of clicking all the hyperlinks to make sure that they’re valid could take a great deal of time. 2. Add in testing the basic functionality of the Web site’s features, doing configuration and compatibility testing, and figuring out a way to test performance and loading by simulating thousands or even millions of users, and you have a big job.
Anuradha Bhatia