Linköping University | Department of Computer Science Master thesis, 30 ECTS | Datateknik 2017 | LIU-IDA/LITH-EX-A--17/038--SE

Designing for usability of 3D configuration in E-commerce – Interactive design of 3D in web applications

Design för användbarhet i 3D-konfigurering inom e- handel

Alfred Axelsson

Supervisor : Anders Fröberg Examiner : Erik Berglund

Linköpings universitet SE–581 83 Linköping +46 13 28 10 00 , www.liu.se Upphovsrätt

Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare – under 25 år från publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns lösningar av teknisk och admin- istrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sam- manhang som är kränkande för upphovsmannenslitterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida http://www.ep.liu.se/.

Copyright

The publishers will keep this document online on the Internet – or its possible replacement – for a period of 25 years starting from the date of publication barring exceptional circum- stances. The online availability of the document implies permanent permission for anyone to read, to download, or to print out single copies for his/hers own use and to use it unchanged for non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional upon the con- sent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping Uni- versity Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its www home page: http://www.ep.liu.se/.

c Alfred Axelsson Abstract

Mass production of consumer products is both wasteful and limit the consumers’ power to influence the design of what they are buying. When shopping for a product, the customer must choose between a range of specific products with limited variations. What if the customer could make all the design choices, creating and buying the product according to his or her own needs? A 3D product generator holding a configurable model of a product was created to re- place static content in online stores and give creative power to customers. This work aimed at creating an effective 3D product generator by evaluating how users experience the de- sign of and interaction with it, and finding general design goals when introducing interac- tive 3D content in static 2D environments. A prototype of a 3D product generator in a generic online storefront was implemented in two iterations, improving on and evaluating the design through usability testing. The evaluation of the final prototype suggested that the interface was indeed effective in both design and interaction. This work concludes that user feedback is crucial to creating a successful user experience, which in turn is important when creating interfaces for product configuration. Acknowledgments

First, I would like to thank everyone at SkyMaker AB for supporting me, with special shout- outs to my mentor Jonathan Brandtberg for his help throughout the project and our discus- sions, and to Kristofer Skyttner for giving me this opportunity. Furthermore, I want to thank Erik Berglund and Anders Fröberg for their supervision and help in completing this thesis. I also want to thank everyone who helped me by participating in the usability tests. This could not have been done without your help, thank you. Finally, I want to thank my family and the friends who have supported me both during my work on this project, but also during the entirety of my time at Linköping University.

iv Contents

Abstract iii

Acknowledgments iv

Contents v

List of Figures vii

List of Tables viii

1 Introduction 1 1.1 Aim...... 1 1.2 Research questions ...... 1 1.3 Delimitations ...... 2 1.4 Project Context ...... 2

2 Theory 3 2.1 Web design practices ...... 3 2.1.1 Interface design ...... 4 2.1.2 Visual Hierarchy ...... 4 2.1.3 2D vs 3D ...... 5 2.2 Usability testing ...... 6 2.2.1 Planning before testing ...... 6 2.2.2 Collecting and analyzing the results ...... 8 2.3 Focus groups ...... 10 2.4 Confidence interval ...... 10

3 Method 12 3.1 Implementation ...... 13 3.1.1 Prototyping ...... 13 3.1.2 Iterating the design with usability testing ...... 13 3.1.3 Formative evaluation of the prototype interface ...... 14 3.2 Evaluation ...... 16 3.2.1 Recruiting participants ...... 16 3.2.2 Test session ...... 16 3.2.3 Data collection and analysis ...... 17 3.3 Motivation ...... 17

4 Results 18 4.1 Implementation ...... 18 4.1.1 Wireframes ...... 18 4.1.2 Initial prototype ...... 19 4.1.3 Formative usability test ...... 21

v 4.1.4 Final prototype ...... 22 4.2 Evaluation ...... 24

5 Discussion 27 5.1 Results ...... 27 5.1.1 Prototype ...... 27 5.1.2 Formative results ...... 28 5.1.3 Summative results ...... 28 5.2 Method ...... 28 5.2.1 An iterative process ...... 29 5.2.2 Usability testing ...... 29 5.2.3 Sources of information ...... 30 5.3 The work in a wider context ...... 31 5.3.1 Societal aspects ...... 31 5.3.2 Ethical aspects ...... 31

6 Conclusion 32 6.1 Future work ...... 33

Bibliography 34

A Appendix A 36 A.1 Wireframes ...... 36

B Appendix B 40 B.1 Test plan formative test ...... 40 B.2 Test plan summative test ...... 43

C Appendix C 45 C.1 Prototype 1 ...... 45

D Appendix D 48 D.1 Prototype 2 ...... 48 List of Figures

3.1 Overview of the method ...... 12

4.1 Final Wireframe ...... 19 4.2 Screenshots of the first prototype (detailed images can be found in appendix C) . . 20 4.3 Task times from the formative study ...... 21 4.4 Task success from the formative study ...... 22 4.5 Screenshots of the final prototype (detailed images can be found in appendix D) . 24 4.6 Average number of errors per task from summative study ...... 25 4.7 Percentage of users that made an error per task from summative study ...... 26

A.1 First wireframe ...... 37 A.2 Second wireframe ...... 38 A.3 Final Wireframe in more detail ...... 39

C.1 Screenshot of Prototype 1 (initial screen) ...... 46 C.2 Screenshot of Prototype 1 (modified screen) ...... 47

D.1 Screenshot of Prototype 2 (initial screen) ...... 49 D.2 Screenshot of Prototype 2 (modified screen) ...... 50

vii List of Tables

3.1 Characteristics of participants in formative study ...... 15 3.2 Characteristics of participants in validation test ...... 16

4.1 List of issues found during the formative tests ...... 23 4.2 Actual characteristics of participants in the summative test ...... 25

viii 1 Introduction

Millions of users interact with web applications every day. Websites are designed with a purpose and are adapted to the content, the functionality and hopefully, its users. User Ex- perience (UX) is the experience of this interaction. T. Tullis and B. Albert [19] believe that the user experience is defined by three characteristics. These can be summarized as the observ- able or measurable experience of a user interacting with an interface. Interface design, how interaction is performed and how the system responds are some aspects of creating the user experience. When shopping for a product online, the customer has to choose between a range of spe- cific products with various features and designs as well as different prices. Then each prod- uct might come with varied measurements and colors. But the choices are still limited to the range of products the producer chose to manufacture. What if the customer could make all these choices him- or herself using a 3D product generator? First choosing a base product, then designing it using the available parameters and finally adding extra features to create the exact product that he or she was looking for. To enable this, we must make sure that it is easy for the customer to understand how to interact with the product generator and utilize all its features.

1.1 Aim

This work focuses on evaluating how users experience the design of and interaction with a 3D product generator in an online store. The goal is to create an effective user interface for the generator by iterating the design with usability testing. The process aims at finding general design goals when creating usable interfaces and introducing interactive 3D content in static 2D environments.

1.2 Research questions

By implementing a design based on research and then iterate it with usability tests the design will be evaluated and used to answer the following research question.

1 1.3. Delimitations

1. How should the interaction with a 3D product generator in a web application be de- signed to achieve high effectiveness in terms of usability, allowing users to easily reach their goals and complete their tasks?

1.3 Delimitations

To limit the scope of this work, the following delimitations have been defined. This allow for a more precise focus and help the decision process during implementation.

• The implementation only concern the interface of a product page on a web store. The web store will not be implemented to have full functionality.

• The implementation does not have to be browser independent.

• Color choices are not among the important aspects of the design.

1.4 Project Context

This document concern a master thesis project in computer science at Linköping Uni- versity. The project was carried out in cooperation with the company SkyMaker AB (www.skymaker.se), who encouraged and supported the project. SkyMaker is a small com- pany situated in Linköping, Sweden. They develop 2D and 3D generators to help their cus- tomers configure their products.

2 2 Theory

The design of online stores has shown to influence sales.[20] Users’ emotions are influenced by their perceptions of the usability and aesthetics of the store, these emotions connect de- sign attributes to whether they approach or avoid the store.[15] As this work focuses on the design of and interaction with an e-commerce interface using 3D elements, and evaluating the usability of design, the related work is spread in different fields of research. The design of user interfaces, web design, the use of 3D in websites and most importantly, usability testing.

2.1 Web design practices

Eccher [7] details that studies have shown how users visiting a homepage only spend 10 to 20 seconds, and suggests that usability is the key to conveying a message and keeping the user at the site. There are three areas of web design that impact the usability of the web site: architecture, layout and navigation. The architecture of the site is the way pages and sections of the site are connected and how the user flow is designed. There are some things to consider when designing the architecture: Using a site map to present the architecture. Using a naming convention that users are famil- iar with. Reducing the number of clicks needed for users to reach content they are looking for. Not linking out of the sections unexpectedly which could confuse users. Breadcrumb tech- niques could help reduce confusion and give users a sense of location on the site. Avoiding a flat architecture where everything is cluttered and confusing for users, and instead designing one that is cascading where content is organized in subsections[7]. The layout of the site should enable the user to locate information easily. It mainly refers to how elements on the site is positioned and can be split into the two areas scrolling and positioning. Scrolling is widely debated and there are some pros and cons to it. Notably a page with scrolling can fit more content while some user might prefer clicking through content. Horizontal scrolling however, is never acceptable according to Eccher [7] because it contradicts usability standards and require more motion than scrolling vertically. Positioning content is about where on the page different items are placed. The menu which typically is positioned on top or to the left. The header which sometimes is combined with the menu and could contain the logo, functional links and ads.[7] The rest of the space can be filled with different items. A feature area, content, a sidebar (often to the right) and a footer. There can also be ads displayed in these elements.[14]

3 2.1. Web design practices

Navigation is key when creating effective and usable websites. It should be intuitive to help users find what they are looking for. The menus should be consistent throughout the pages of the site. They should stay in the same locations, their content should be static and only change if subcategories are added when a user navigates, and there should be a limited number of menus. Preferably no more than two.[7] When navigation is good it will go unno- ticed. It will help users while being used without thinking. If the navigation doesn’t work as expected users will start to notice it.[2] The structure of a menu could be either horizontal or vertical. With a horizontal structure the width of the screen need to be considered while there is more room for content to stretch the width of the screen. For a vertical structure the con- cerns are opposite where the horizontal room for content is smaller while it is easier to add many items. The design of menu items is also important. Using text is preferred over images to make it clear what the navigation does. However, when designing the items, their length must be considered so that all items fit inside the length of the menu.[7] The menu items also need to be different enough to not confuse users[2]. All navigation, links and menus, must look like actual navigation to make it clear that they are interactive[2].

2.1.1 Interface design Interface design is the design if the interactive elements that will connect users to the func- tionality. When realizing interaction with the functionality, the interface elements are selected and arranged in a way that is intuitive and easy to use. This process is part of interface de- sign because it enables users to do things.[9] The design of the interface should focus on the actions most users are likely to take and not give the same priority to the rarely used func- tionality. In a successful interface, users should notice the important aspects while not seeing the unimportant parts. Some of the standard interface elements are: buttons, checkboxes, input fields and lists.[9]

2.1.2 Visual Hierarchy The concept of hierarchy is among the most important aspects of design. The visual hierarchy within a design is the sequencing of its elements to define the most important element, the second most important elements, and so on.[14] One tool used to draw the attention of users is contrast. Contrast between design elements of the interface can make the essential items stand out and catch the eye of users. Users pay attention when things are different, which won’t happen with a featureless interface without contrast.[9] B. Miller and R. Black [14] tell us that information of pretty much any type can be broken down into three to four levels. More levels of importance make it difficult to contrast the differences between levels. A hierarchy is created through a design system and the system can be developed through different methods. Alignment, shape, size, scale, color, texture and depth can all be used to define a hierarchy between design elements.

Alignment In a system of alignment, the elements are logically grouped through their meaning or func- tion. The hierarchy is created through elements that break this system. These elements are given more visual value as they don’t fit into a cohesive unit with other elements. Part of or- ganizing design elements is the organization of space. Intentionally created space, so called white space, is essential when designing relationships between elements to define a system of hierarchy.[14]

4 2.1. Web design practices

Scale Using scale to have dominant elements in a design is critical when creating hierarchical se- quences. Scale is relative, which means elements of different sizes need to be included for comparison to get a dynamic feeling of scale. Elements can also stretch out of borders or outside of a page to enhance the sense of scale.[14]

Color Color has the instructive qualities to guide, direct and persuade users while also being able to set a mood or tone to the design, which appeals to users’ emotions. As color signifies a mean- ing for people it can be used as a powerful tool in design. Colors can be recognized quickly making it important when forming relationships between elements.[14] A company gener- ally have a color palette used in all material made by the company. The colors in a palette are chosen to work well together and not to compete for the visual attention of users.[9]

Images Images or iconography can be used to replace text e.g. descriptions, and enable users to more easily get information from the layout of a page. All images on a page should be intentionally put there to add something. Too many images might prevent good user experience as they add to the size of the web page.[14]

Texture Texture can be anything from shiny and smooth to rough and dull. It helps connect users to the page’s content through the sense of a tactile experience. Type, images and similar elements on a page combine to create an overall texture that may be intended or not. This is present in every design and can be perceived subconsciously by the users.[14]

Depth Creating the impression of depth can be done by overlapping design elements, adding gra- dient color and shadows or using three-dimensional design elements. Depth and dimension in a page makes it more realistic, add visual interest and draw users into the design.[14]

2.1.3 2D vs 3D In this work, photos are the normally the two-dimensional content used in web design that could be replaced with three-dimensional content. Photos can be used to illustrate products, demonstrate what products do and answer users’ questions at a glance. This makes it possible for users to understand what they are buying without reading a lot of text. In their research, J. Allen and J. Chudley [2] discovered that users want to see large photos to enable them to immerse themselves in the photos and examine details of the products. Content photos should be made as large as possible or enlargeable if space is limited.[2] Research has been conducted to examine how users’ online shopping experience is af- fected by the use of 3D environments in the online store. The 3D environment in this case was a virtual store where the user could walk around in a 3D store and interact with books. The study propose that the user experience is greatly influenced by the use of a 3D environ- ment instead of a 2D storefront. The 3D store appeared harder to use which suggest that one should be cautious when introducing new technology to online stores.[20]

5 2.2. Usability testing

2.2 Usability testing

Usability tests can be used to test almost anything from business processes to mobile appli- cations. They test the usability of the product and not the users themselves. Conclusions can be drawn by observing how test users interact with and use the product and what problems arise.[2] By performing usability tests, you gain access to understanding your users. What pleases them or frustrates them, what their goals and expectations are and their experience with your product.[4] With this information the design process of your product can focus on the users to support them in their goals and how they want to use the product. "We are testing the product, not you." is the central mantra in usability testing. Usability can be defined as "The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified con- text of use.", from the International Organization for Standardization, ISO (9241-11). This definition name the measures of usability: effectiveness, efficiency and satisfaction.[4] These measures can be specified further and described as follows[4, 16, 19]:

• Usefulness - Determined by the degree to which a user’s goals can be achieved from using the product. Usefulness estimates if users will use the product or not.

• Effectiveness - Concerns the ease of use for users to reach their goals and how com- pletely a task can be performed. Effectiveness can be quantified using error rate.

• Learnability - Involve how well a user learned to use the product after a fixed time of training. This is a part of effectiveness.

• Efficiency - Is the speed with which a task is performed or the effort required to com- plete it. For the most part efficiency is in some way measured in time.

• Satisfaction - A user is more inclined to use, and perform well, when using a product that grant satisfaction. The extent to which the use of the product was a good experi- ence. Concerns the user’s perceptions of the product. User satisfaction can be deter- mined by asking about their experience.

2.2.1 Planning before testing The first part of the test process is planning, regardless of the test method. There are two main parts of planning for a usability test: choosing the test method and writing the test plan. The test plan defines the details of the test and can be used as support during the test process. Below, two kinds of usability studies are described followed by an explanation of a test plan.

Formative studies Formative studies are usually small informal tests that reveal what users like and produce a list of what should be fixed during development[4]. The main goal when performing forma- tive studies is to improve the design concepts early and this is why they are usually carried out early before development of the design is finalized[16, 19]. The nature of these kinds of studies make them suitable for iterative development[19]. Design can be evaluated, im- proved upon and evaluated again in a reasonably short amount of time. If the developers have incorrect assumptions about their users and their users’ goals the product will most likely suffer usability problems in the end[16]. The earlier the formative tests begin, the more impact they will have on the design[19]. This also implies that less assumptions have to be made. When planning for a formative study it is important to identify what subgroup of the user population that is to be tested. By recruiting participant in the same group based on a user profile the results will be better. Different groups have different goals and needs which

6 2.2. Usability testing must be identified to interpret the test results. If the test will be small with around five test participants one subgroup would be enough. With a larger test, more subgroups could be used while reducing the number of participants in each group. The findings between tests with different subgroups will likely overlap.[4] During formative testing the product’s functionality does not need to be complete. In- stead a prototype or model can be used to represent the product’s basic design and func- tionality during testing. The prototype could be anything from a paper representation to a user interface.[16] To obtain useful results from small tests of this kind the participant are given specific tasks to perform[4, 16]. The tasks represent realistic scenarios the participants can solve to accomplish some goal. Conducting all tests with the same set of tasks makes it possible to see recurring problems and common usage patterns.[4] Participants can also review the product and answer questions under guidance either if it is too early in develop- ment for tasks or while performing tasks. Either way, formative testing usually requires the moderator to interact considerably with the participant[16]. Much of the test data collected is of cognitive nature and require the moderator to explore the participant’s thoughts. This is typically done by encouraging the participants to "think aloud".[4, 16]. This adds an addi- tional dimension where users’ thoughts, reactions and feelings are expressed, which makes it easier to understand their experience and how to make it better.[4] The most suitable testing technique depend on the stage of the development process and how refined the prototype is[16].

Summative studies Summative studies are larger compared to formative and usually performed later in the de- velopment process[4]. Some litterature separate summative tests and verification tests. J Rubin and D Chisnell [16] defines the summative test as a more mature version of the forma- tive test, performed early to midway through the development process. Their description of the verification test instead holds a definition similar to the one used for summative testing throughout this work. The objective or goal of summative studies is to evaluate how well the product compares to its predetermined objectives or some benchmark.[16, 19] Their focus is to evaluate against some set of criteria and not to find ways to improve the product, as is the case with formative studies. Summative testing can also be used to compare features of two or more products.[4] Summative tests are often conducted when development of a product is completed or close to. They require a large number of participants as the results are used to generate metrics. Success on tasks, failure on tasks, average time on task, completion rates, and error rates are some examples of measures that can be obtained.[4] Not all larger studies are done in late stages of development. In some cases, the validity of small studies are challenged. More problems need to be uncovered by conducting larger studies. Specifically, this is true for large, complex websites where the user base consist of a variety of users and in systems where substandard usability could result in personal risk or injury.[4]

The test plan The test plan is the foundation of the test process and the documentation to support deci- sions about what to test, characteristics of participants etc[4]. It will describe the process and communicate what is decided and what resources are required to conduct the test[16]. A test plan typically consists of the following parts.

• Purpose or goal of the test. Explains the reason for conducting the test and any issues it will focus on. Can be defined at a high level as later parts will go further into detail[16].

• Research questions further describes the issues that will be cleared during the test. These questions must be accurate and clear to direct the research and describe what

7 2.2. Usability testing

the test want to find out. The answer to the research questions must be measurable or observable[16].

• Characteristics of test participants are specified in a user profile to describe the partici- pants that best represent the end users or a subgroup of them[16]. These characteristics are used as criteria when recruiting test participants and should be as specific as pos- sible. It is also important to define how many test users that is required to get valid results.[19] When testing using more than one subgroup of users each subgroup is de- scribed with its own user profile[4].

• Methodology that describes how the research and test session will be performed. This part should also describe the test’s type and include what will be done and how much time each step will require.[4, 16] With a detailed description of the method the credi- bility of the results will be solidified[16].

• A task list of what participants will perform when testing. The tasks should be se- quences of interactions future users will perform to achieve their goals when using the product.[4, 16] For each task, there should be a short description, the materials needed and which state the product needs to be in, a criterion for successful completion and any benchmarks that will be collected.[16]

• Test environment and equipment required to perform the test. This details what is needed to simulate realistic conditions and scenarios that correlate to actual situations where the product is expected to be used.[4, 16]

• Evaluation methods that describes what data that will be collected and how it will be done. All data of interest, both quantitative measures and participants’ preferences should be listed[16]. Any questionnaire that will be used to gather data should also be included[4].

• Deliverables that explains how the results of the test will be delivered. Outlines what will be presented and dates for delivery.[4, 16] This part is particularly important in a large organization where different teams and management rely on the results of the test.

2.2.2 Collecting and analyzing the results There are two types of data that can be collected from usability tests. Performance data, the qualitative measurements and preference data, collected from observations done during tests and questionnaires. What has been collected depends on the goal of the performed test and the process of analyzing is also done differently depending on the type of data and research questions.

Preference data The data collected from observing participants and using a "think aloud" process is known as qualitative data or preference data as it describes what users feel when interacting with the product. These data are primarily collected during formative studies. If the design process is iterative with studies in each iteration, the easiest way to analyze these findings is by counting the number of issues. The number of unique problem, the frequency of issues per participant or frequency of issues per task can be used to compare between iterations and determine if the usability is improving or not. The frequency of participants that experience issues or issues experienced in different areas of design can also be used to compare, but is even more suited to find what areas need the most work to improve the usability.[19] Preference data can be collected in a multitude of ways, e.g. observations, questionnaires, participants’ comments and your interpretation of their behavior. By comparing data from separate sources and look for consistencies and inconsistencies the strength or the findings

8 2.2. Usability testing can be assessed. This process is called triangulation. Triangulation might help discover con- flicts between different data where something that seemed an issue when observing actually wasn’t as important to the user. Triangulation can also be used to compare between quanti- tative and qualitative data.[4]

Task success In the field of usability, task success is the most common metric. It can be calculated for any kind of task for any kind of product that is tested. Tasks are defined with a clear end state, a goal, which is used to determine success. Task success can be either binary or be defined by levels of success.[19] Binary success has two results: success or failure. An average success rate from binary success can be calculated per task or per participant.[19] If succeeding part of a task is valuable, levels of success can be identified and used to measure the success rate. These levels must also be defined before conducting tests as sub targets of achieving full success.[19]

Time on task The time a user requires to complete a task, time on task or task time, is used to measure the tested product’s efficiency. Time on task is defined as the time between when the user start performing a task until the task is successfully completed.[19] Time on task is commonly analyzed by task through calculating the average time spent on the task by all participants. The median or geometric mean could also be calculated instead of the mean for time on task depending on if the collected data seems to be biased or if there are potential outliers.[19]

Errors Error rate, the number of errors made when interacting with a product can be used to de- fine the product’s effectiveness[16]. An action that have a negative impact on success is an error. But errors are not typically defined as one thing, instead there are several types of er- rors. There are errors of omission, when you leave something out or fail to take an action required to reach a goal, and errors of commission, doing something unnecessary that does not contribute to reaching the goal.[19] Error data is most often organized by task. If a given task present a single error oppor- tunity the data is binary, no error or one error. Analyzing binary error data is often done by task. The percentage of participants who experienced an error on a task can be compiled for each task. It is also possible to analyze the error rate by type of user and view number of errors made by users from different user groups. This is typically good when comparing results from different user groups that tested different prototypes or products. If a given task present multiple error opportunities the data is analyzed differently. One way to analyze is to calculate the error rate for each task by dividing the total number of errors for a task by the total number of error opportunities. Because different tasks have varying amounts of opportunities for errors, directly comparing error frequencies against each other could prove misleading. Another way to compare different task would be to calculate the average number of errors made by participants for each task. This also gives an indication of how a typical user would perform using the product. Calculating averages also reduce the bias of extremes where some users might contribute most of the errors. In some cases, it might be important to consider how severe an error is. By applying a severity level to errors and weigh them accordingly would make it possible to make a fair comparison of error frequency or error rate.[19] Sometimes a test will see if the product performs to some certain level. With error rate this would be a threshold, a limit of how many errors is acceptable. The threshold might differ

9 2.3. Focus groups between user groups or different tasks and before evaluation this should be established. By simply comparing the to the threshold it is possible to determine if the product is as effective as expected or if it needs more work.[19]

2.3 Focus groups

Focus group studies are conducted as series of discussions. These are created to collect partic- ipants perceptions on some areas of interest of the evaluated product, service, topic, etc.[12]. The use of focus group discussions is a qualitative research method where a group of people, the focus group, participate in an interactive discussion that focus on specific issues in the areas of interest[11]. Focus groups can be used as an alternative to, or complement to, usability testing, when evaluating the user experience of a system.

2.4 Confidence interval

Confidence intervals are used frequently in research to estimate a range containing the un- known, true population parameter. A confidence interval is calculated for some sample data using some confidence level, typically 95% (other typical values include 99%, 90%, 80% and 85%). The confidence level determines how confident we are in the method of generating the confidence intervals. A calculated confidence interval either contain the true parameter or it does not.[18] There are several formulas to calculate confidence interval for the sample data. If the data is binomial, as is true when there’s only one error opportunity or when measuring comple- tion rates, the most commonly used method is the Wald method. The formula for the Wald method is seen in equation (2.1) where p is the sample proportion, n is the sample size and z α is the critical value of the normal distribution for the defined level of confidence. (1´ 2 )

p(1 ´ p) p ˘ z(1´ α ) (2.1) 2 c n Intervals created using this method are inaccurate at small sample sizes or when the results are all close to 0 or 1. To create good intervals a large sample size is required.[18, 17] Another formula is the "Exact" method. It was created to work for all sample sizes and to guarantee that the confidence interval provides at least 95% coverage. However, exact intervals suffer from being too conservative. Their intervals become larger and cover more than the defined confidence level. This is especially true with small sample data.[18, 17] A third method is the adjusted-Wald method. For 95% confidence intervals this method is really the same as the Wald method but requires the addition of two successes and two failures to the observed sample data.[18, 17] More precise this means adding two to the numerator and four to the denominator which is derived from the critical value of the normal distribution, which is 1.96 for 95% intervals[18]. This is done in equation (2.2) where x is the number of completions and n is the sample size. By inserting p = padj into equation (2.1) the confidence interval is calculated. 2 x + z x + 1.92 p = 2 = (2.2) adj n + z2 n + 3.84 The adjusted-Wald method requires more work to calculate compared to the normal Wald method and is not used as much. However, research has shown that this method create good intervals for most completion rates, even close to 0 and 1.[18, 17, 1] At small sample sizes the method also have improved accuracy compared to the standard Wald method and the exact method. While, for larger sizes, adding two successes and two failures doesn’t do much to help performance, it also does nothing to harm the process, either.[18]

10 2.4. Confidence interval

When creating confidence interval for sample data that is continuous or of some rating scale, the t-distribution is recommended. The distribution works almost the same as the nor- mal except that it considers the size of sample data. It makes intervals wider when sample size get smaller to adjust to the estimate of the population. For larger sample sizes the con- fidence intervals using the t-distribution converges on those using the normal distribution. J. Sauro and J. Lewis [18] recommend using it for all sample sizes as it provides the best intervals regardless. s x¯ ˘ t ´ α ? (2.3) (1 2 ) n The formula for creating t-confidence intervals can be seen in equation (2.3) where x¯ is the sample mean, n is the sample size, s is the standard deviation and t α is the critical value (1´ 2 ) from the t-distribution for n ´ 1 degrees of freedom and the specified level of confidence.[18]

11 3 Method

The method used to create and implement the interaction with, and design of the interface as well as the evaluation of said interface is proposed in this chapter. During implementation, an iterative method was used to improve the design using usability tests. The initial design could then be enhanced and evaluated using a more extensive usability test. An overview of the method can be seen in figure 3.1.

Figure 3.1: Overview of the method

12 3.1. Implementation

3.1 Implementation

Implementation of the interface was carried out using established technology for web devel- opment: HTML5, JavaScript, CSS and the JavaScript graphics library three.js. Development was performed using the WebStorm IDE (Integrated Development Environment) from Jet- Brains (https://www.jetbrains.com/).

HTML5 The standard language used to create web pages is Hypertext Markup Language, HTML. HTML5 is the latest version of HTML.[5] HTML allow web developers to structure and present the contents of the web page.[21]

JavaScript (JS) JavaScript is a programming language commonly used to create functionality in the web browser. JavaScript is especially good for controlling the content of the document displayed on a web page. It can also be embedded in HTML as scripts. In the context of the web browser it is also called Client-side JavaScript as it is run in clients’ browser and not on a server.[8]

CSS Cascading Style Sheets (CSS) is a language used to define the visual style of elements defined in HTML. As HTML define the structure and presentation of a web page, CSS control the visual presentation of images, text and other elements. Size, fonts and colors are some of the design items CSS control.[13] three.js Three.js is a JavaScript library that enable developers to easily create 3D scenes for web pages. It provides an extensive set of functions that in turn uses WebGL to render the scenes.[6] WebGL is a graphics API (application programming interface) for advanced web-based 3D using JS and HTML5.[3]

3.1.1 Prototyping Initial focus in the project was the creation of a prototype interface. A prototype is an early sample or model of a product. It is built to be tested and improved upon until it’s in a state where the actual product can be developed from it.[10] In this project the initial prototype was created using wireframes. Wireframes are simple schematics of the page that depict the layout of its information, navigation elements and interface elements [9]. They establish the simple basics of the page’s visual design and are used early in the design process to convey the basic shapes and form of the web store. The wireframes were evaluated by asking for input from colleagues at SkyMaker and the resulting design was implemented using the technologies listed in chapter 3.1.

3.1.2 Iterating the design with usability testing The user interface design was improved in iterations using usability tests. A formative study was conducted on the prototype to improve its initial design, while the final design of the interface was later evaluated using a more extensive summative test. The tests helped deter- mine what is most important from the user’s point of view when interacting with the interface of a 3D web application. When conducting usability tests, design issues can be found that is

13 3.1. Implementation not obvious to designers or developers during development. These issues can then be reme- died and help enhance the design and usability further, resulting in an interface that is easy to use. The following usability metrics were collected during tests to determine if the prototype interface was usable and what parts needed to be re-designed to achieve higher usability:

• Task success - Users success rate when performing given tasks. Collected as binary success.

• Time on Task - How fast users could complete given tasks. Measured from starting a task until task success is achieved.

• Errors - How many mistakes users made when attempting to complete given tasks. Collected through counting errors of omission and errors of commission made.

• Self-reported issues - How users perceived the system. Collected by asking users what they thought about the product.

• Usability issues - Interpreted from how users behaved when solving tasks. Include confusion, misinterpretation and mistakes that did not help in completing tasks.

The learnability (how well a user learned to use the product) of the interface was deter- mined by measuring the task success and time on task while performing the tasks. The effectiveness (how completely tasks could be performed) was determined by measur- ing task success and error rate while the efficiency (how quickly a task could be completed) was determined simply from time on task. The users’ level of satisfaction was answered from the self-reported issues. These issues also showed if the users perceived the interface to be easy to use or not. Usability issues added proofing to the product by identifying issues that were not obvious from test statistics and the users’ answers.

3.1.3 Formative evaluation of the prototype interface The purpose of the formative study was to find design issues in the prototype design and to get an overview of what users might think of interacting with a 3D interface to generate a product. Planning was the first part of conducting the study and a test plan was written to detail the testing process. The different parts of a test plan are described in chapter 2.2.1. The test plan for this test is included as appendix B. To make sure the test plan was sound a test session was held not to test the prototype but to evaluate the procedures of the test. This helped improve the test plan and made the test sessions more structured.

Recruiting participants Recruitment of participants was based on their previous experience with online shopping. Participants was recruited among students at Linköping University. Characteristics and num- ber of participants can be seen in table 3.1.

Formative test session Each participant underwent a test session consisting of an introduction, a performance of tasks and a debriefing. The introduction prepared the participant before performing the test and explained what was expected of him or her. The moderator also asked what expectations the participant had toward the prototype. During the performance step the participant was encouraged to "think aloud" and the moderator both observed and interacted with the par- ticipant to create an understanding of how he or she perceived the interface, and the thought

14 3.1. Implementation

Minimum number of participants 4 Characteristics Experience with online shopping Frequent shoppers ( > 11 times / year) 1 Moderate shoppers ( 6 – 11 times /year) 2 Infrequent shoppers ( < 6 times / year) 1

Table 3.1: Characteristics of participants in formative study process involved in the participants’ decisions. The following tasks was given to the partici- pants during the test and performed in order. The description of each task and its success can be seen in the test plan, appendix B.

• Task 1 - Basic interaction with the 3D model.

• Task 2 - First time configuration using interactive components.

• Task 3 - Second time configuration using interactive components.

• Task 4 - Free exploration of the interface and configuration space.

During the debriefing uncertainties that remained were discussed and clarified to collect all possible issues of the interface. The participant also answered questions of the experience and could ask questions back.

Metrics There were several metrics collected during this test. Task success and time on task were collected as quantitative measurements. These metrics can be used to identify issues encoun- tered during tasks. Task success was defined for each task in the test plan and a stopwatch was used to measure time on task. Task success was analyzed per task as success rate with percent of users that succeeded. Confidence intervals for these statistics were calculated us- ing the adjusted-Wald method, described in chapter 2.4. Time on task was also analyzed by task, as the average time required to complete a task. The confidence interval of the results was calculated using the t-distribution. Self-reported issues and usability issues were collected as qualitative metrics. These can be used to get a picture of how users think when they interact with the product, what issues they experience, and how they approach the tasks. Self-reported issues can also help find design issues that cause a task failure or high time requirements when solving tasks. Two methods were used to collect issues.

• The "think aloud" process encouraged the participants to voice their preconceptions and expectations as well as their preferences when conducting the tasks. Thinking aloud helps gather both self-reported and usability issues.

• Asking questions before and after the study to gather the participants’ expectations and preconceptions as well as how they perceived the experience.

Triangulation was used to find consistencies among the issues and to compare with the quantitative findings. The result was presented as a list of design issues that impact the usability of the prototype. The issues were in order of severity based on how frequent they occurred and how it affected the user.

15 3.2. Evaluation

Minimum number of participants 12 Characteristics Experience with online shopping Frequent shoppers ( > 11 times / year) 4 Moderate shoppers ( 6 – 11 times /year) 4 Infrequent shoppers (< 6 times / year) 4 Age 18 - 24 4 25 - 30 4 >30 4 Gender Female 6 Male 6

Table 3.2: Characteristics of participants in validation test

3.2 Evaluation

When the design and implementation of the interface is completed it will be evaluated using a summative usability test. The purpose of the test is to find out if users can use the interface to complete their goals. This test will verify the effectiveness of the implementation and the results will be used to answer the research question.

How should the interaction with a 3D product generator in a web application be designed to achieve high usability in terms of effectiveness?

Planning for the summative test will be done using a test plan that will detail the process of the test. It will also ensure that tests are being conducted equally and that data collection is performed equal for all participants. The test plan for this test can be seen in appendix B.

3.2.1 Recruiting participants Participants will be recruited among students and through SkyMaker’s customer contacts. None of the users should have participated in the formative study detailed in chapter 3.1.3. The recruitment will be based on their previous experience with online shopping. Partici- pants will also be of varying age and gender. Characteristics and desired number of partici- pants can be seen in table 3.2.

3.2.2 Test session A test session will be structured into three parts: an introduction, a performance of tasks and a debriefing. During the introduction, the participant will be briefed on how the test works and why it is done to ensure that he or she will know what is expected during the test. The participant will then be left to complete the following tasks. The tasks is identical to those performed in the formative test sessions. Exact descriptions of these tasks and their definition of success can be found in the test plan, appendix B.

• Task 1 - Basic interaction with the 3D model.

• Task 2 - First time configuration using interactive components.

• Task 3 - Second time configuration using interactive components.

• Task 4 - Free exploration of the interface and configuration space.

16 3.3. Motivation

To ensure that the participant use the product in its finalized state without interference or support, the moderator simply observes. The post-test debriefing will be used to discuss any particular issues that might remain and to answer any of the participant’s questions.

3.2.3 Data collection and analysis The data collected during the test will consist of the number of errors made. This will be collected by counting all incorrect selections and all errors of omission. These data will give quantitative results of the interface’s effectiveness. The error rate will be calculated by task. Both as average number of errors made by all participants and as percentage of users that made an error. Confidence intervals will be calcu- lated to ascertain the reliability of the statistics. In the case of multiple errors the confidence interval will be calculated using the t-distribution while the adjusted-Wald method will be used to calculate for binary error rate.

3.3 Motivation

The decision to apply usability testing instead of focus groups were based on the fact that both qualitative and quantitative data would be collected. While focus groups are suitable for the collection of qualitative data they are not as useful when doing a quantitative evaluation. As the process was iterative, using both usability testing and another evaluation method was not considered, instead the same evaluation method was used in both iterations only collecting different types of data.

17 4 Results

In this chapter the results from each part of the project will be presented. The different results from the phases of implementation will be described first followed by the concluding results from the evaluation process. The results are presented chronologically and build upon each other. This is especially true for the implementation results.

4.1 Implementation

The results from the implementation phase include several items: the wireframes that was created as a base for the layout and design of the prototype, the initial prototype developed from these wireframes, the test results of the formative study and finally the improved pro- totype.

4.1.1 Wireframes This section describes the results of the wireframes’ creation process, detailing the final wire- frame, seen in figure 4.1. Initially, two wireframes with small differences were produced which were then combined and improved upon in a final version. The final wireframe was used to determine the basic layout of the interface. The final wireframe consisted of a header bar, a product pane, an information pane, and a footer. The header contained the site logo, a search bar and a shopping cart followed by a navigation bar with global navigation items for different product categories and account management. Below the header bar the product pane was placed containing breadcrumbs revealing the users’ location on the website, the 3D canvas for the product model and the interactive components linked to the 3D model. The grouping of the interactive components was outlined with sliders on top, other objects below them and a larger button at the bottom for adding the product to the shopping cart. The third pane contained a tab bar and a content area for displaying information linked to the different tabs. The footer at the bottom of the page would be holding site specific information and navigation to such information. The main changes from the first two wireframes was the restructured header bar and the removal of global navigation from the product pane to create better visual alignment. The early wireframes are included in appendix A.

18 4.1. Implementation

Figure 4.1: Final Wireframe

4.1.2 Initial prototype The following section details the elements of the first prototype interface. The functionality and graphic elements will be described to provide an understanding of how the interaction was designed. An overview of the prototype interface can be seen in figure 4.2 and a detailed version can be seen in appendix C. Following the first limitation in chapter 1.3, the parts of the prototype interface that does not concern the product lack full functionality. However, the interface was designed to look and feel as a fully functional web store product page. To create visual hierarchy neutral colors are used. Borders are set in light gray to avoid messy lines, text is in black and the background is white. Important parts, interactive component and active items are colored in a bright blue color to guide the users to notice and interact with them. The header bar was designed with navigation buttons, a search bar and a shopping cart button as in the wireframe. The cart and account buttons were given icons suited to their functions. None of the items in the header bar had any functionality and it was designed with different neutral colors compared to the rest of the page to be perceived as important without gaining visual hierarchy over the rest of the page. On top of the product pane there were breadcrumbs to give a sense of location and to give navigation back to what could be previously visited pages. These links also lacked function- ality as with the other navigation. Below the breadcrumbs the product name was presented in a large font, sitting atop the canvas of the 3D model. In the canvas, a 3D model of a table was shown standing on a wooden floor. The tabletop was a rectangular mesh with a textured surface. Each table legs consisted of a cylinder from the tabletop down to a rectangular foot. The legs were given a metal texture surface to resemble a metal material. Finally, the floor was a plane mesh with a wooden textured material. The scene in the canvas also contained three types of lights. Directional light from above the table model, a spotlight following the

19 4.1. Implementation

(a) Initial screen

(b) Modified model

Figure 4.2: Screenshots of the first prototype (detailed images can be found in appendix C) camera and ambient light to brighten the scene. The materials of the meshes were made more realistic through a color map and a normal map where the normal map was used to calculate how the light reflected in the material to give it a three-dimensional look while still being on a two-dimensional surface. In the bottom of the canvas an overlay was placed with images of arrows with the purpose to indicate interaction with the model. When a user was hovering over the canvas with the mouse the arrows became fully visible while they became transpar- ent when the mouse left the canvas. This design can be seen between figures 4.2a and 4.2b. To strengthen the indication of interaction further, the mouse cursor was changed from the standard arrow into a hand when hovering over the 3D canvas. Additionally the user could interact with the scene’s camera in two ways: rotating it by holding left mouse button and moving the mouse, and zooming in and out through scrolling the mouse wheel.

20 4.1. Implementation

To the right of the canvas, the last part of the product pane held the configuration com- ponents. Starting with sliders on top that stretched the width of the area. Maximum and minimum values were shown above the endpoints of the sliders and in the middle a title de- scribed what the slider did. Centered below each slider an input field displayed the slider’s current value which was connected to some parameter of the 3D model. This value could be changed either through the slider thump or through the input field. Below the sliders a bold font title told what part of the 3D model was concerned followed by drop down lists with material and color options for that part. By changing any parameter of the configuration the model was updated and rendered with the new parameter. As a special case, when the size of the tabletop was changed the size of the floor was also changed to stay larger than the table. Below the configuration components a large buy button was placed to be visible. Clicking it put the configured product in the shopping cart, updating the text of the shopping cart label. The information pane consisted of a menu bar with tabs that controlled the content of the content segment below. The bar could display product information and specifications, customer reviews and accessories. The menu had complete functionality while the content was placeholder text. The footer in the bottom of the page also lacked any functionality. It was designed with space for a company logo, social media links, and links to customer service.

4.1.3 Formative usability test The formative usability test was conducted as described in the test plan with no deviations. Actual characteristics of participants was identical to table 3.1 with four participants. The test plan can be found in appendix B. Results from this test include both the statistic analysis of participants’ time on task and task success followed by a list of design issues compiled from the qualitative data. Participant’s time on task gave an indication of how quickly users could perform different tasks using the prototype interface. As can be seen in figure 4.3, basic interaction with the 3D model for first time users (task 1) was fast while the first time interaction with the configu-

Geometric mean of task times with 95% c.i. 100

90

80

70 68,92

60

50

49,31 Task times times (s) Task 40

30 30,22

20 20,40

10

0 Task 1 Task 2 Task 3 Task 4

Figure 4.3: Task times from the formative study

21 4.1. Implementation ration components (task 2) required more time and gave more uncertain results. The second time participants interacted with the configuration (task 3) was considerably faster and could indicate they that learned quickly and the prototype having good learnability. When given the freedom to explore (task 4) the results show participants taking longer than when doing specified tasks which is logical. However, the confidence interval is narrow which indicate that users would require more or less the same amount of time to explore the interface. Task success from the test can be seen in figure 4.4. These results show the prototype enabling users to complete intended tasks. However, there were an issue understanding the zoom function in the 3D canvas which made task 1 harder to complete. All task success data is uncertain because of the sample size being small which results in wide confidence intervals. The list of design issues compiled from the qualitative data can be seen in table 4.1.

4.1.4 Final prototype In this section the final prototype interface is presented. Based on the issues outlined in table 4.1 changes were made to the first prototype described in section 4.1.2. These changes will be described to provide an understanding of how the functionality, design and interaction was improved. An overview of the final prototype can be seen in figure 4.5 and a detailed version can be seen in appendix D. The first change was due to issue one, lacking feedback on the buy button. Two anima- tions were added to the interface when a user click the button. The button itself is animated, changing color to white to appear pressed and changing text to tell the user that the item was added to the cart. The button then slowly transition back to its original appearance. Simulta- neously, the shopping cart in the header bar is updated and is animated in the same color as the original buy button to showcase that an update occurred to the cart, before transitioning back to its original appearance. These effects can be seen in figure 4.5b. Change number two was made to handle issue number two. A border was added to the 3D canvas to display what parts of the interface belonged to it. The border was designed as light gray, thin borders identical to the interface’s already existing borders. This indicated

Completion rate with 95% c.i. 100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0% Task 1 Task 2 Task 3 Task 4

Figure 4.4: Task success from the formative study

22 4.1. Implementation where the boundary was without cluttering the interface. An effect was added to the canvas to clearly display when the user hovered with the mouse cursor on top of it, changing the border color to the highlight color used in other parts of the interface. This effect can also be seen in figure 4.5b. To handle the third issue, three changes were made. First, the layout of the configuration items was changed. Items that configure the same part of the model was grouped vertically instead of horizontally and these groups were put next to each other. This clarified what items were connected. The second change was the increase in margin between sliders and groups of connected configuration elements. This improved the readability and made room for the third change. The last part of solving issue three was the addition of borders between sliders and between groups of connected configuration items. The borders were identical to the existing borders, thin and light gray. Their purpose was to define boundaries, clarifying what items belonged together while avoiding an increase of clutter in the interface. The next change dealt with issue four, the misunderstood arrows on top of the canvas. As a new change was made to show when the mouse cursor was hovering on top of the canvas, the arrows were removed to avoid creating confusion. When dealing with issue six, users not using the ability to zoom in the 3D canvas, a new canvas overlay was added. This overlay held a horizontal slider to indicate the ability to zoom and two icons to show what directions of the slider was zooming in and out. The slider moved when using the mouse wheel to zoom in and out, the buttons were given click events for zooming, and the slider thumb could also be used for zooming if preferred. This gave users more choices and a clear indication that zooming in the 3D canvas was possible. Issue seven was the use of standard HTML components that were perceived as out of place in the design. The input and drop-down components was redesigned with rounded corners to match the rounded design of the sliders and an effect was added when a compo- nent was interacted with. This effect gave the focused component a border in the highlight color which appeared to glow using a colored shadow. This can be seen in figure 4.5a where the effect is active on a focused drop-down list. The next issue was the floor appearing unnatural when it changed size. This was reme- died by setting its size to a fixed value that appeared as the model was placed in the middle of a room. The purpose of this change was to avoid creating confusion and the unnatural feeling users experienced during the test. To handle issue nine, the slider thumbs had their size increased enough to make them prominent without cluttering and shadowing other parts of the interface. The remaining issues were left more or less unattended. Issues connected to the sliders, four and ten, were mitigated by adding some explanatory text to the information tab. Issue

No. Issue Severity 1 No obvious feedback when pressing the buy button 9 2 Hard to know when the mouse cursor is inside the 3D canvas 8 3 Configuration items linked to one part does not seem to belong together 6 4 The arrow overlay purpose is not perceived as intended 6 5 Sliders with the same value are not aligned 6 6 User does not zoom 4 7 Standard HTML components does not fit in 4 8 The floor changing size is perceived as unnatural 3 9 Small slider thumbs 2 10 No explanation to maximum and minimum slider values 2 11 The camera position is locked 1 12 Input fields function is not clear 1

Table 4.1: List of issues found during the formative tests

23 4.2. Evaluation

(a) Initial screen

(b) Modified model

Figure 4.5: Screenshots of the final prototype (detailed images can be found in appendix D) eleven was not considered severe enough and issue twelve was dealt with indirectly when more margin and borders were added between sliders.

4.2 Evaluation

The evaluation process was conducted as a summative test of the prototype interface, follow- ing the description in the test plan found in appendix B. However, the process deviated from the test plan on the point characteristics of participants. The desired spread of participant characteristics and minimum number of participants can be seen in table 3.2. In table 4.2 be- low, the actual characteristics of the test participants can be seen. The deviation was based on the number of potential participants willing to help and the assumption that collecting

24 4.2. Evaluation

Number of participants 17 Characteristics Experience with online shopping Frequent shoppers ( > 11 times / year) 9 Moderate shoppers ( 6 – 11 times /year) 4 Infrequent shoppers (< 6 times / year) 4 Age 18 - 23 5 24 - 26 8 >26 4 Gender Female 6 Male 11

Table 4.2: Actual characteristics of participants in the summative test more data means better results. Results from the evaluation include the statistical analysis of the number of errors made by participants during the test. The results are presented in two parts. Both as average number of errors made per task and as percentage of users that made an error per task. Errors are shown as number of incorrect selections, number of errors of omission, and total number of errors. Figure 4.6 display the average number of errors made per task by participants of the sum- mative test. Blue bars represent incorrect selections, gray bars represent errors of omission, and orange bars represent the sum of all errors. 95% confidence intervals are specified for each type of data per task. This graph indicates that interaction with the model in the 3D canvas (task 1) was effective for most users and very rarely resulted in any kind of error. Participants seemed likely to experience at least one error when using the configuration com- ponents for the first time (task 2), while errors made from repeated interaction (task 3 and

Number of errors made per task with 95% c.i. Incorrect selections Errors of omission Total number of errors 1,9 1,8 1,7 1,6 1,5 1,4 1,3

1,2 1,24 1,1 1,12 1 0,9 0,8 0,82 0,7 0,71 0,6 0,59 0,5 0,53 0,4 0,3

0,2 0,24 0,1 0,18 0,12 0,12 0,06 0,06 0 Task1 Task2 Task3 Task4

Figure 4.6: Average number of errors per task from summative study

25 4.2. Evaluation

Percentage of users that made an error per task with 95% c.i. Incorrect selections Errors of omission Any error 100%

90%

80%

70% 70,6% 70,6%

60%

50% 52,9%

47,1% 40% 41,2%

35,3% 30%

20% 17,6%

10% 11,8% 11,8%

5,9% 5,9% 5,9% 0% Task1 Task2 Task3 Task4

Figure 4.7: Percentage of users that made an error per task from summative study task 4) show a decrease in numbers. The number of incorrect selections decreased quickly while users seemed to make roughly the same amount of errors of omission. However, errors of omission were rarely encountered at all. The next graph, figure 4.7 show the percentage of participants who made an incorrect selection in blue, an error of omission in gray, and any kind of error in orange. These results are also displayed per task with 95% confidence intervals. This graph show that a majority of participants made at least one error during their first configuration of the model (task 2). With repeated interaction, fewer participants made errors, specifically incorrect selections. Errors of omission was increased when configuring material and color of the model (task 3) but then decreased again. The results suggest that the prototype interface was indeed effective, allowing a large majority of users perform tasks without encountering any errors or at most a single error.

26 5 Discussion

The goal of this work was to create an effective user interface of a 3D product generator in an online store by iterating the design with usability testing, focusing on evaluating how users experience the design of and interaction with the interface. This discussion will be based on the goal and aim of this work as well as its replicability, reliability and validity. The results and the method will be discussed as well as the ethical and societal aspects.

5.1 Results

This work produced different kinds of results. The results come from the two iterations di- vided as design results and test results. During the process the results created include wire- frames, the first prototype and the formative test results which in turn resulted in the final prototype which through the summative test produced the final test results. There are some of these results that are major results: the final prototype, the formative test results and the summative test results.

5.1.1 Prototype The resulting prototype is first and foremost based on the wireframes. The concepts and layout created in the wireframes was based on one designer’s views of what an online store should look and feel like followed by discussion with colleagues. A larger design team could have created more diverse drafts before settling on which one to develop further. Interaction and discussion with others was limited throughout the process and the resulting design is the result of one person’s decisions. Additional opinions could potentially have resulted in a completely different design, but it could also be a similar solution to the one chosen in this work. When working with design, it is unlikely two projects will end in identical results even after following the same processes. Creative work is influenced by the individuals involved, the environment, and many other external factors. As such, the resulting prototype design is most likely unique for this work. The prototype can be considered a successful result of the design process. Not because it necessarily was an effective interface, but because it allowed for easy test sessions followed

27 5.2. Method by iterative improvement. Data could be collected from users without developing a working system and instead implementing the important aspects required for user interaction.

5.1.2 Formative results Any type of test result can be improved by testing with more participants. However, the results from the formative test, despite the small sample size, can be considered successful as they helped in improving the design. Task success and time on task does not give certain results in themselves because of the small sample size. They do however give some indications. Task times seem to display learn- ability, where users improve through similar tasks and repeated interaction. It also seems like the size and difficulty of the prototype and its interaction is in a good state at this point, because the interval for the exploratory task 4 is limited. Test participants performed the exploration in similar time. Limited exploration might also result in recurring customers as they are not intimidated by the size and complexity of the interface. On the other hand, a too simple interface could result in no recurring users as it is perceived as too simple or lacking possibilities. The issues collected from the formative study also provide some interesting information. To begin with, there were very few unique issues experienced by only one participant. There were a few but most of those also highlighted important issues that was considered critical enough to remedy. Most of the issues found were experienced by two or three of the partic- ipants. This indicate that the most critical issues are experienced by a majority of users and can be found through small usability tests.

5.1.3 Summative results Only one kind of metric was collected from the summative test, the number of errors users experienced. There were only a small number of errors experienced by participants during this step, both average number of errors per task and percentage of participants that experi- enced at least one error was low. This most likely show good results in terms of the interface’s effectiveness. But it also shows a continuing trend of high learnability. Both graphs displayed a decreasing number of errors through repeated interaction. From the summative test, the number of participants, or sample size, is expected to pro- vide credible results. There did not appear to be any significant differences between partici- pants with different characteristics, they instead seem to perform similar to each other. The participants can be seen to represent a subset of the population: young adults between ages 18 and 30 where gender and prior experience in online shopping does not matter consider- ably when using the prototype interface. The design, however, is not perfect. Participants did encounter errors the design could be improved further and made even more usable and effective.

5.2 Method

The method used in the process of this work was based on an iterative process where each iteration was divided into two parts: prototype design and usability testing. Even if the actual design of the prototypes is hard to replicate, the design process used is considered replicable in itself and all the steps involved is described in detail in chapter 3. This is also true for the usability testing where the process involved in both types of tests are described in the method chapter and each test have an associated test plan further detailing the test approach. The test plans can be found in appendix B. Replicating the method without prior experience in either design or usability testing is highly possible and anyone with a technical background should be able to create prototypes without issues.

28 5.2. Method

As mentioned above, creating a certain design concern not only implementing a method, following the process as it is described, step by step. It also involves creativity where the people involved, the environment, and other conditions affect the outcome. The design can vary considerably from the layout of wireframes to the color scheme to the visual feedback. However, there will most likely be some items in the design that will be present in more or less similar shapes. The 3D model will have to be presented in some way with some kind of interaction as that is the main purpose of a 3D product configurator over a 2D image or video. Configuration items of some sort and shape will also have to be included to allow the product configuration. Individual user feedback may differ in both performance and perception as different peo- ple have varying expectations, experience, and values. Even if the design itself may not be created identical and with varied individual feedback, the method should help achieve sim- ilar overall results from the usability tests. With continuous user feedback the biggest issues can be remedied and result in a prototype that achieve the same effectiveness. The sum- mative test was exhaustive enough, displaying reliable results with low variation between participants. The method includes two opportunities for performing measurements. When collecting data during the formative test the issues collected give an indication of what is expected from the system and what is perceived from using it. These issues directly measure what users consider to be good or bad design when interacting with the system and guide the design team when improving the system. The other two measures from the formative test, time on task and task success, give an indication of what kind of tasks are difficult, or impossible, to perform. This complement the issues and give additional support to designers when working to remedy the issues. The most important measurement collected in this work was the errors collected as quan- titative data from the summative test. User errors that occur when interacting with an inter- face measure the effectiveness of said interface. If a large amount of errors is encountered, the usability of the interface is not effective because it interrupt the user.

5.2.1 An iterative process An iterative process was chosen to align with the focus of this work. By iterating prototype design and usability testing the results was closely coupled with the users and their experi- ence when interacting with the interface. There are certainly other methods that can be used when designing software for usability concerns. The options will have their advantages and disadvantages and be fit for different kinds of problems. The chosen process was deemed suitable in this case and the choice was not motivated from a scientific comparison as this would have been outside of the scope of this project. Using an iterative process provided opportunities to evaluate and reflect on the progress, and respond to changes throughout the implementation phase. This made it easier to adapt to time requirements and was especially helpful when planning the second usability test. The process was divided into two iterations based on the time limit and the possibilities to perform user testing. By breaking up the time plan into shorter parts, additional iterations could have been possible. These iterations would have allowed for more user feedback. In term focusing more on the main issues in each iteration instead of implementing fixes to most issues after the first usability test. However, this process would have been less extensive and the changes between iterations would have been fewer.

5.2.2 Usability testing In performing usability testing the user feedback was directly helpful in improving the inter- face. There was no confusion in whether or not the interface is built for the users as could have been the case if the interface and its effectiveness was tested in other ways.

29 5.2. Method

By testing directly with participants and interacting with them, instead of doing remote usability tests, the issues found could be evaluated based on how they occurred and how they affected the participants. Some arguments for testing remotely would be the freedom perceived without someone watching every action, and a more natural environment where each user would normally perform tasks such as those being tested. Specifically, the sum- mative test could have been performed that way when no interaction took place during the actual test session. However, the environment created for both tests were focused on being quiet, comfortable, and natural to create conditions similar to a real scenario. There are certainly other ways to test software and collect rewarding results. The interface could have been tested by measuring technical aspects or using other types of usability tests. The configurable model used in the prototype was a simple table model which in a real application may not have be as interesting to use. There are certainly several online stores selling furniture and even bulkier consumer goods, but the participants could not see a real- life product of the table model as there were no sample product. This could have made participant perceive the prototype as either new and interesting, or as unrealistic and hard to grasp. Testing with this model also wasn’t very exhaustive in terms of configuration items. Having more parameters involved in a more advanced model could have given additional feedback on what users expect from the configuration. One of the most difficult decisions was the number of participants needed for the two usability tests. In the formative tests, more test participants would have greatly helped im- prove the quantitative metrics, giving better statistical results. However, these data were not as impactful as the qualitative issues. The number chosen was based on having little time and only focusing on finding the most important usability issues. Additional participants would certainly be recommended when looking to extensively find issues and composing a better severity rating of issues. Having more participants in the summative test could proba- bly have improved the collection of errors and given more confident data. However, the data collected is believed to be a good indicator of the effectiveness of the final prototype. As discussed in the previous section, additional iterations would have allowed more us- ability tests to collect user feedback. This could also help in improving the issue coverage of the prototype and enable developers to explore the design and test more solutions to the issues. It was also considered if tests should be performed with the same participants. This could have been helpful in determining if changes to a design solved old issues and if they intro- duced new issues or not. With more usability tests this would have been a possible process. In this work the argument against it was the fact that the summative did not focus on issues but rather on one metric. The summative test also required a larger test group where all par- ticipants should have the same conditions going into the test. A subset of the participants should not be in contact with the interface before conducting the test.

5.2.3 Sources of information There were three main research areas used as a basis for this work: web design and usabil- ity. The sources used were selected to provide information on web design using the chosen technologies and designing with focus on the user. The compiled information was built as a foundation to creating a usable web interface and evaluating how users experience its design and interaction, which was in line with the aim of this work. When looking for research into 3D on the web there were not a lot of typical guidelines. This is most likely because 3D content on the web is rather uncommon today and because there are other areas where research into 3D graphics can be found. However, one selected source of information was used because it explored the differences between 2D and 3D web design. Usability in software engineering is a broad research field, part of the even broader field of User Experience. A lot of the information is found digitally in blog posts but there are also much to be found in books and research articles. To limit the scope of this work, the in-

30 5.3. The work in a wider context formation gathered focused on usability testing and usability metrics to be able to answer the research question. The chosen sources were mainly books that describe all the general prin- ciples of usability testing where planning for and conducting tests were important aspects. The research was cited in other research material and appeared trusted by professionals in the field. This selection provided a solid base of information which was deemed sufficient for this work. Additionally, some research into statistical analysis and confidence intervals was found through the usability literature. Specifically, the research concerned the adjusted Wald method for calculating binomial confidence intervals from small data samples. The use of this research was based on the collection of data from small samples during the formative test.

5.3 The work in a wider context

This work focuses on introducing 3D configuration in online stores where users can design and buy the products based on their own perceptions and needs. This could have some potential implications for society. These implications are presented as societal and ethical aspects.

5.3.1 Societal aspects When users themselves can become the designers of the products they are buying, the need for mass producing changes into the need of mass customization. Companies normally create one design for a product and produce large quantities of that product to stock. With config- urable products, the need to store products is removed, and replaced with the need for new processes to manufacture unique products according to user needs. This could potentially reduce the overconsumption of today and benefit society. Additionally, this does not just move the creative power from specialists to the user, it also transfer responsibility. When the consumer is charged with the possibilities of exploring and finding new solutions he or she will also have to deal with the consequences if something is missing or does not work. This can be seen in other fields where the creative respon- sibilities have been moved to users. Some examples are websites created through sites like WordPress (http://wordpress.com) and games created using free game engines like Unity3D (https://unity3d.com/) and Unreal Engine (https://www.unrealengine.com). Introducing easily accessible customization could also have a negative impact on indi- vidual craftsmen, carpenters for instance, that create unique and customized products. If consumers prefer their own designs over someone else’s, individual craftsmen might lose business.

5.3.2 Ethical aspects This kind of web interface is more or less free from direct ethical consequences. One could ar- gue about online shopping safety, storing consumer information and other general concerns when dealing with online stores. This work does not focus on selling products and no such functionality has been implemented. Instead, the focus is on enabling creative freedom when buying products. One possible ethical consequence is connected to the societal aspect where mass produc- ing is replaced with production on demand. This could potentially reduce overconsumption when resources are only used to produce what customers want to buy. It could also remove the need of warehouses to store products before selling them, reducing the space needed by manufacturers. These potential effects would have a positive impact on nature.

31 6 Conclusion

This work was performed to investigate how to design the interface of a 3D product gener- ator, focusing on how users experience the design of and interaction with it. The goal was to create an effective interface for the generator with the help of user feedback from iterative usability testing. This was reflected in the research question:

How should the interaction with a 3D product generator in a web application be designed to achieve high usability in terms of effectiveness?

The work aimed at finding general design goals when creating usable interfaces and intro- ducing interactive 3D content in static 2D environments. When designing the presentation of and interaction with 3D content on web sites it was clear that user feedback is crucial to a successful user experience. Iteratively performing user testing, even with small test samples, improved the usability of the 3D interface and directly met user expectations. As a conclusion, some general elements could be interpreted from the results of this work and help answer the research question. These elements can help mitigate the disadvantages of 3D content in online stores, creating an effective interface.

• Keeping the possible exploration in the interface as low as possible or simplifying it can avoid a stressful or confusing user experience

– It could be seen in 4.3 that the time needed to explore the configuration space was limited. Participants of the first usability test also expressed the possible interac- tions to be simple, realistic and adequate for the given tasks.

• Using similar layout and design as expected from a standard online store to make users feel comfortable with something they are accustomed to

– Participants from the usability studies did not hesitate to approach their given tasks and seemed to understand the test environment and its purpose. Partici- pants from both the first and second usability study also commented on how they felt comfortable with the interface and understood what was expected from the interaction.

32 6.1. Future work

• Giving instant feedback on interaction can keep users interested

– Some of the most critical design changes made during the second iteration was concerned with the visual response from the interface, towards the user, when an interaction was made. After addressing these design flaws, results from the sum- mative test with few errors made imply that the design was successful in regards to visual feedback.

There is potential to using 3D configuration for consumer products and to succeed, the impression of such an interface must meet user expectations. While other areas may need evolving before complete consumer configuration is fully possible, e.g. new production tech- niques, this work emphasize the importance of creating a good user experience to succeed. There are already companies that offer different types and degrees of product configura- tion in their online stores. This work indicates that there are extensive opportunities to create configurable consumer products where the consumer can become the designer.

6.1 Future work

As the main focus of this work is studying how to design for effectiveness, future studies may want to do more exhaustive research into other usability aspects and their impact on the design. Users might see past an ineffective interface if it grants satisfaction, if it is efficient, and so on. Any combination of the usability aspects listed in 2.2. The product used in the prototypes of this work was a simple model of a table, but what if it was a mobile phone case or a porch? The potential products have their own implications and might require special kinds of interaction. This could be an interesting topic for further studies. One of the most important aspects of topics dealing with consumers is their views and expectations. This work does not study what potential consumers might feel when faced with the options and responsibilities of configurable products. There may be certain subsets of the population that appreciate the opportunity to take control over the design. There could also be certain products consumers are more likely to want to design and products that no one want to configure themselves. These topics are strong candidates where research can support the decision to work with configurable products.

33 Bibliography

[1] Alan Agresti and Brent A. Coull. “Approximate Is Better than ’Exact’ for Interval Es- timation of Binomial Proportions.” In: The American Statistician 2 (1998), p. 119. ISSN: 00031305. [2] Jesmond Allen and James Chudley. Smashing UX design. : foundations for designing online user experiences. Smashing magazine. Chichester, West Sussex, U.K. : John Wiley and Sons Ltd, 2012, 2012. ISBN: 9780470666852. [3] Andreas Anyuru. Professional WebGL programming : developing 3D graphics for the web. Wrox programmer to programmer. Chichester, U.K. : John Wiley & Sons, 2012., 2012. ISBN: 9781119968863. [4] Carol M. Barnum. Usability testing essentials.: ready, set– test!. Burlington, Mass. : Morgan Kaufmann, c2011, 2011. ISBN: 9780123750921. [5] Chris Coremans and Paul Deck. HTML : a beginner’s tutorial. [Brossard, Quebec, Canada] : Brainy Software, Inc., [2015], 2015. ISBN: 9781771970181. [6] Jos Dirksen and Suyog Gharat. Three.js essentials : create and animate beautiful 3D graphics with this fast-paced tutorial. Community Experience Distilled. Birmingham, England : Packt Publishing, 2014., 2014. ISBN: 9781783980864. [7] Clint Eccher. Professional web design. : techniques and templates, fifth edition. Boston, MA : Course Technology PTR, c2015, 2015. ISBN: 9781305257528. [8] David Flanagan. JavaScript.: the definitive guide. Sebastopol : O’Reilly, cop. 2006, 2006. ISBN: 0-596-10199-6. [9] Jesse James Garrett. The Elements of User Experience: User-Centered Design for the Web and Beyond. 2nd. Thousand Oaks, CA, USA: New Riders Publishing, 2010. ISBN: 9780321683687. [10] Angela Harmon. “Prototype.” In: Salem Press Encyclopedia (2016). [11] Monique M. Hennink. Focus group discussions. Understanding qualitative research. New York : Oxford University Press, 2014., 2014. ISBN: 9780190256111. [12] Richard A. Krueger and Mary Anne Casey. Focus groups : a practical guide for applied re- search. Thousand Oaks, Calif. : Sage Publications, cop. 2015, 2015. ISBN: 9781483365244. [13] Mike McGrath. CSS3. In easy steps. Leamington Spa, Warwickshire, U.K. : In Easy Steps, c2013, 2013. ISBN: 9781840785418.

34 Bibliography

[14] Brian G. Miller and Roger Black. Above the fold. : understanding the principles of successful Web site design. Cincinnati, Ohio : HOW Books, c2011, 2011. ISBN: 9781440308420. [15] Talya Porat and Noam Tractinsky. “It’s a Pleasure Buying Here: The Effects of Web- Store Design on Consumers’ Emotions and Attitudes.” In: Human-Computer Interaction 27.3 (2012), pp. 235–276. ISSN: 07370024. [16] Jeffrey Rubin and Dana Chisnell. Handbook of usability testing.: how to plan, design, and conduct effective tests. Indianapolis, Ind. : Wiley ; Chichester : John Wiley [distributor], c2008, 2008. ISBN: 0470386088. [17] Jeff Sauro and James R. Lewis. “Estimating completion rates from small samples using binomial confidence intervals: Comparisons and recommendations”. In: In Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting. Society, 2005, pp. 2100– 2104. [18] Jeff Sauro and James R. Lewis. Quantifying the User Experience : Practical Statistics for User Research. Waltham, MA, USA : Morgan Kaufmann is an imprint of Elsevier, 2016., 2016. ISBN: 128023082. [19] T. Tullis and B. Albert. Measuring the User Experience: Collecting, Analyzing, and Pre- senting Usability Metrics: Second Edition. Bentley University: Elsevier Inc., 2013. ISBN: 9780124157811. [20] Lucian L. Visinescu, Anna Sidorova, Mary C. Jones, and Victor R. Prybutok. “The influ- ence of website dimensionality on customer experiences, perceptions and behavioral intentions: An exploration of 2D vs. 3D web design.” In: Information And Management 52 (2015), pp. 1–17. ISSN: 0378-7206. [21] Paul Whitehead and James H. Russell. HTML. : your visual blueprint for designing Web pages with HTML, CSS, and XHTML. Visual read less, learn more. Hoboken, N.J. : Wiley, c2005., 2005. ISBN: 076458331X.

35 A Appendix A

A.1 Wireframes

This appendix present the wireframes produced in iteration 1 during the implementation phase.

36 A.1. Wireframes

Figure A.1: First wireframe

37 A.1. Wireframes

Figure A.2: Second wireframe

38 A.1. Wireframes

Figure A.3: Final Wireframe in more detail

39 B Appendix B

B.1 Test plan formative test

Test plan – formative study

Purpose The study is performed to create an overview of user’s perceptions of the 3D product generator and to evaluate the interaction with the 3D environment to find design issues.

Research question • What are the prototype’s major design issues? • What are the major usability issues with the prototype interface? Characteristics of participants

Minimum number of participants 4 Characteristics Experience with online shopping Frequent shoppers ( > 11 times / year) 1 Moderate shoppers ( 6 – 11 times /year) 2 Infrequent shoppers ( < 6 times / year) 1

Methodology The test session will be split in three parts – an introduction, a performance of tasks and a debriefing.

40 B.1. Test plan formative test

The introduction will be 4 minutes. First, the moderator will explain what is expected of the participant during the test and why the test is done. Here, the user will be urged to think aloud and describe what he or she think during each step when interacting with the prototype. The participant will be asked to answer what his or her expectations are. A scenario will then be described, outlining the scope of the prototype and put the participant in the role as a user. The user will be given 7 minutes to perform tasks (listed below under “Task list”). During this time, the moderator will observe the user, take notes from what he or she describes when thinking aloud and collect the quantitative metrics. After this, there will be a 4-minute debriefing where the user is asked about his/her perceptions of the system and to clarify if the moderator need more details on something. Task list • Interact with the model, look at the tabletop surface and inspect the table legs. o Success: The participant used the mouse to drag the model to see all parts of the model and to zoom using the mouse scroll wheel. • Create a table with a black metal tabletop and light gray table legs. Add it to the cart. o Success: A correct product was added to the cart. • Create a 2m wide, 90cm tall table. Add it to the cart. o Success: A correct product was added to the cart. • Create a tall square table with your favorite material and add it to the cart. o Success: A correct product was added to the cart. Test environment and equipment The testing environment will be a room that can be kept relatively quiet to correlate to a comfortable “homely” environment where people don’t disturb the participant. It could be a study room or classroom at the university, or a quiet room at the office, depending on where the participants feel most comfortable. It should have at least one table and two chairs. There should be enough room for both the participant and the moderator to sit by the same table and have the equipment set up. The equipment needed is a laptop and a computer mouse. The laptop used for the tests is a Lenovo ThinkPad with a 12.5” screen at solution 1366x768. The moderator should also use some system for taking notes.

Evaluation methods The following metrics will be collected:

41 B.1. Test plan formative test

• Task success – collected by noting if participants achieve the defined suc- cess of a task, or not. • Time on task – measured using a stopwatch. Time required to success- fully complete a task. • Self-reported issues – collected from the user by using a “think-aloud” process and by asking about the users’ perceptions of the prototype at the end of the test sessions. • Usability issues – gathered from observing the participants during the test. Negative reactions and faulty paths taken to solve tasks will be noted and triangulated with the other metrics to look for usability issues. Deliverables Task success will be compiled in a bar graph as success rate with percentage of successful participants. Time on Task will also be compiled in a bar graph, as average time required to complete each task. The issues collected will be compiled in a list in order of severity. The severity will be decided based on risk, occurrence and consequence.

42 B.2. Test plan summative test

B.2 Test plan summative test

Purpose This study will determine the effectiveness the interface of the 3D product generator prototype by measuring user errors.

Research question • How usable is the prototype interface, in terms of effectiveness? • What parts are the most usable and what could be better? Characteristics of participants

Minimum number of participants 12 Characteristics Experience with online shopping Frequent shoppers ( > 11 times / year) 4 Moderate shoppers ( 6 – 11 times /year) 4 Infrequent shoppers (< 6 times / year) 4 Age 18 - 24 4 25 - 30 4 >30 4 Gender Female 6 Male 6

Methodology The test session will be split in three parts – an introduction, a performance of tasks and a debriefing. The introduction will be 4 minutes. First, the moderator will explain what is expected of the participant during the test and why the test is done. A scenario will then be described, outlining the scope of the prototype and put the participant in the role as a user. The user will be given 7 minutes to perform tasks (listed below under “Task list”). During this time, the moderator will observe the user and collect the quantitative metrics by recording the screen. The user will be performing the test as if the prototype was a finalized product and will not be getting any help or feedback during the process of completing the tasks. After this, there will be a 4-minute debriefing where the user is asked about his/her perceptions of the system and to clarify if the moderator need more details on something. There is also time for user questions if he or she have any.

43 B.2. Test plan summative test

Task list • Interact with the model, look at the tabletop surface and inspect the table legs. o Success: The participant used the mouse to drag the model to see all parts of the model and to zoom using the mouse scroll wheel. • Create a table with a black metal tabletop and light gray table legs. Add it to the cart. o Success: A correct product was added to the cart. • Create a 130cm wide, 90cm tall table. Add it to the cart. o Success: A correct product was added to the cart. • Create a tall square table with your favorite material and add it to the cart. o Success: A correct product was added to the cart. Test environment and equipment The testing environment will be a room that can be kept relatively quiet to correlate to a comfortable “homely” environment where people don’t disturb the participant. It could be a study room or classroom at the university, or a quiet room at the office, depending on where the participants feel most comfortable. It should have at least one table and two chairs. There should be enough room for both the participant and the moderator to sit by the same table and have the equipment set up. The equipment needed is a laptop and a computer mouse. The laptop used for the tests is a Lenovo ThinkPad with a 12.5” screen at solution 1366x768. The moderator should also use some system for taking notes.

Evaluation methods The following metrics will be collected: • Errors – collected by counting incorrect selections and errors of omission. Deliverables The number of incorrect selections, the number of errors of omission and the total number of errors will be compiled in two bar graphs. In one as average number of errors made per task and in the second as percentage of users per task that made any such error.

44 C Appendix C

C.1 Prototype 1

45 C.1. Prototype 1

Figure C.1: Screenshot of Prototype 1 (initial screen)

46 C.1. Prototype 1

Figure C.2: Screenshot of Prototype 1 (modified screen)

47 D Appendix D

D.1 Prototype 2

48 D.1. Prototype 2

Figure D.1: Screenshot of Prototype 2 (initial screen)

49 D.1. Prototype 2

Figure D.2: Screenshot of Prototype 2 (modified screen)

50