EXAMENSARBETE INOM DATALOGI OCH DATATEKNIK, AVANCERAD NIVÅ, 30 HP STOCKHOLM, SVERIGE 2020
Interaction Principles of 3D World Editors in MobilePhones with Focus on the User Experience
VINCENT ERIK WONG
KTH SKOLAN FÖR ELEKTROTEKNIK OCH DATAVETENSKAP Sammanfattning 3D v¨arldsredigerare anv¨ands omfattande i mobilapplikationer f¨or ma- nipulering av virtuella v¨arldar, men det finns ingen generell designram- verk f¨or de b¨asta interaktionprinciperna f¨or v¨arldsredigerare i mobiltele- foner. Genom att studera interaktionsprinciper, koncept, och funktioner som anv¨ands i konsumentbaserade v¨arldsredigerare, denna studies fokus ligger p˚autveckling och design av en prototyp, sedan evalueras inter- aktionsprinciperna, uppdelade i f¨oljande anv¨andaromr˚aden: urval, place- ring, manipulering. Resultatet fr˚an en heuristisk evaluering som st¨odjs av anv¨andartester visar p˚aatt det finns en majoritet som f¨oredrar f¨oljande kombination av interaktionsprinciper f¨or ett b¨attre anv¨andarupplevelse: en organiserad och simplifierad butiksgr¨ansnitt f¨or urval, rutn¨atssystem ¨over det friasystemet f¨or placering av virtuella objekt, och en kombina- tion av snabb- och l˚angsam tryck f¨or manipulering av virtuella objekt i 3D v¨arldsredigerar-prototypen f¨or mobiltelefoner skapad f¨or denna studie.
1 Interaction Principles of 3D World Editors in Mobile Phones with Focus on the User Experience
Vincent Erik Wong [email protected] EECS School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Stockholm, Sweden Abstract to overload the user interface, which is not an optimal solu- 3D world editors are widely used in mobile applications for tion in terms of UX especially for small touch-based screen manipulation of the virtual world, however, there are no devices (i.e. iPhones for this research). frameworks of design regarding the best practice interac- Broadly, the use-case scenario for 3D world editors is a tion principles for 3D world editors on mobile phones. By simplied version of the modeling softwares (e.g., AutoCAD, looking at methods, concepts, and functions used in general Maya, and Blender). world editors are not designed for users consumer world editors, this research focused on the devel- to invent something novel, but rather they are designed to opment and design of a prototype, which was then used to manipulate already existing items that are provided within evaluate the interactions principles divided in the following the software - to customize and decorate the virtual world use cases: selection, placement and manipulation. The results in an eortless and seamless way. from a heuristic evaluation backed up by user evaluations One main line of thought regarding 3D world editors: in- proved that there is in fact a majority that prefers the follow- corporate the virtual world with a grid-system.Agrid-system ing combination of interaction principles in terms of user in the context of 3D world editors acts as a guideline to assist experience: an organized and simplied shop interface for users by snapping virtual objects into the correct position selection, grid-system over the free-system for placement of in the virtual world. Grid-systems provide users a smaller virtual objects, and a combination of tap and hold to manip- window of error for placing objects as compared to when ulate virtual objects for the 3D world editor prototype for a free-system is utilized. On the other hand, a free-system mobile phones created in this research. provides more precision and less restriction for users. Both techniques certainly have their advantages and disadvan- tages. 1 Introduction For small screen devices, like the iPhone, simplifying inter- Virtual world editors, also known as world editors, are often actions and minimizing the degree of manipulation for the implemented both within the video game industry (e.g., in 3D world editor is very important. Many applications that the game genres of real-time strategy and real-life simula- use a 3D world editor employ a grid-system; however, there tion) and outside of the game world (e.g., interior-design ap- is not an agreed-upon standardized framework for 3D world plications). The purpose of using world editors is to provide editors. The purpose of this research is to compile a frame- the user with a simplied sense of control over customization work of interaction principles in regards to the user-journey in the virtual world. The simpler the interaction is - while and use cases for 3D world editor on mobile phones with maximizing degrees of customization - the better the user focus on the overall UX. Interaction principles includes the experience (UX) and the success rate are [1]. use cases: selection, placement and manipulation of virtual 1 2 3D modeling softwares such as Autodesk Maya and Blender object. require great levels of expertise that cannot be developed in This research is supported by a Swedish company called a short amount of time. These three software tools are most Friend Factory AB that is working to release a social-media commonly used to create 3D world scenarios and 3D models application called Frever. The Frever app is primarily devel- of the virtual world. The task of creating and modifying the oped towards Apple iPhone devices target towards teenagers, 3D virtual world can then be quite tedious, even the simplest which is why this research focuses on 3D world editors on tasks of translation and rotation can be hard to comprehend mobile phones. The resultant 3D world editor prototype, de- as they require immense background knowledge e.g., tech- veloped from this research, will become the foundation for nical jargon, key-combinations for dierent functions, and the Frever app’s world editor. mapping from 2D to 3D (from a 2D computer display to the 3D virtual world). Most of these modeling softwares tend
1https://www.autodesk.com/products/maya/ 2https://www.blender.org/ 2 Research Question to the navigation and orientation, selection, and manipula- What are the best practice interaction principles for 3D world tion of virtual objects within the virtual world, respectively. editors on mobile phones from a user experience perspec- Specically, the manipulation use case can be divided into tive? the subcategories of translation and rotation [15]. The nal use case system control, refers to the human-computer in- 3 Related Work teraction at the core of UX research. These four use cases makes up the user journey for world editors. While Mine’s This research has two main areas of focus: UX and develop- research focused on 3D world editors in virtual reality, the ment. The UX phase studies user interaction approaches of use case and user journey for 3D world editors on mobile 3D world editors on mobile phones. The development phase phones would essentially the same as it belongs to the same covers the research and principles of 3D world editors. involves the same concept and idea of manipulating virtual 3.1 User Experience world. UX research comes from the eld of Human-Computer In- 3.1.2 Heuristic Evaluation teraction where the focus moves beyond a traditional em- Nielsen and Molich introduced Heuristic Evaluation (HE) phasis on usability [11]. By examining usability, research as a method for identifying issues regarding the design and has focused primarily on eciency, satisfaction, error, mem- user experiences of UI designs [23, 26]. Nielsen later devel- orability, and learnability [24]. However, UX research also oped a set of ten heuristics [25] as a guideline of evaluation. studies users’ behavioral perceptions (e.g., visual, touch, and Evaluators of HE are often referred to as experts and typi- smell) and positive emotions (e.g., joy, fun, and pride) [12]. cally kept between three to ve [23, 26]. Thus, the ecacy The aforementioned UX principles of focus has previously of the HE is heavily dependent on the expertise of the cho- been researched on 3D world editors [1, 5, 13–15, 20, 22]. sen evaluators [19]. Huy and VanThanh created another set Ali et. al. studied UX in UI for The Sims 3 [1] where the of heuristics for the evaluation of mobile apps from three study was divided in three parts to measure dierent aspects dierent viewpoints: developer, user and service/content of the game and background information - namely, players provider’s viewpoint [5]. These heuristics were designed demographic information, game engagement, and UX on for the evaluation of mobile phones, as mobile phones are the UI (the latter two elements employed a questionnaire limited by smaller screens compared to other devices and using ve-point likert-scale). Towards the end of the study, often in conjunction with touch based interactions as men- overall thoughts and opinions of users were noted via a semi- tioned by Gomoz et. al. [29]. Gomez et. al. similar to Huy structured interview. In this format, participants are asked a and VanThanh created another set of heuristics for mobile series of questions that were prepared prior to the interview UI. [7]. 3.1.3 Empirical Evaluation 3.1.1 Dening User Journey Unlike heuristic evaluation, empirical evaluation focuses At the early stages of application creation, it is paramount on actual end users rather than experts [7]. While there that developers have a well dened user journey. Mapping are certain nancial advantages to conducting a heuristic out the user journey gives developers both a clear overview evaluation with a small sample (user) population, it is best of the use case of the app as well as a script to follow for that empirical evaluation utilizes a more representative, large development, according to Endmann and Keßner [8]. sample size. Merrick and Maher argued that virtual worlds, often seen Bowman et. al. utilized the empirical evaluation method in simulation games e.g. The Sims3, or virtual world applica- of testbed to study the dierence between a set of nine inter- tions, e.g. Active Worlds 4 have an open-ended user journey action principles for selection and manipulation of virtual [21]. This refers to the notion that the user journey of 3D objects in virtual reality 3D world editors [5]. The testbed world editors does not follow a predetermined storyline, but was divided into two tasks (selection and manipulation) and rather allows for the user to manipulate app elements in- both tasks involved three within-subject variables [7]. Quan- denitely. In open-ended simulation games, there is not a titative data were the focus of this research to evaluate the singular, predened way to complete the simulation. There- usability of the dierent interaction principles, where the fore focus should be shifted towards the user journey of the task of selection and manipulation were timed respectively use cases. [5]. The current study has divided interactions of 3D world Hrimech et al. conducted a similar research as Bowman et. editors into four use cases [22]: navigation, selection, manip- al. [5], but only for three interaction principles when it comes ulation, and system control. The rst three interactions refer to selection and manipulation of virtual objects in virtual 3https://www.ea.com/sv-se/games/the-sims reality 3D world editor [15]. Unlike Bowman et. al, this study 4https://www.activeworlds.com/ focused more on the UX rather than usability. Instead of 3 collecting quantitative data in terms of time, Hrimech et. al. knowledge in a eld that most people do not have. Even the focused on qualitative data in the form of statement scores of core interaction of translating objects in world editors can a seven-point likert-scale. A combination of both qualitative be very complex. Houde demonstrates this point in her 1992 and quantitative data types is a good practice and broadly work [14] by reducing the degrees of freedom of manipu- used within UX research [2]. lating virtual objects relative to their position for computer devices. Houde utilized resting planes and gravity constraints 3.2 Development of Interaction Principles to prevent objects that are supposed to be stationary on the 3.2.1 Drag and Drop ground from oating mid-air by eliminating one axis. Bukowski and Séquin [6] took Houde’s technique a step In 1984, Apple introduced the drag and drop interaction [27] further by introducing the concept of object association - the to ease movement, copy, and deletion of les for their per- ability to stack objects onto one another (e.g., placing a plant sonal Macintosh computers. Later, the drag and drop method on a table). Goesele and Stuerzlinger [10] advanced Bukowski has become incorporated in 3D world editors to ease user and Sequin’s concept by dening constraints through den- interactions e.g., translation of virtual objects. From a user ing areas. The rst dening area is called the oer area - the experience perspective, there are many elements that must surface area in which other objects can be placed on. Of- be considered to implement the drag and drop feature - most tentimes, the oer area is a at surface with normals facing of which deal with visual feedback. Bill Scott listed fteen in- upwards. The second dening area is called the binding area- teraction moments that he found most important to consider the area where objects are bound to oer areas. Binding areas for UX design, e.g., mouse hover, mouse down, drag initiated, are typically at the bottom of an object. drag leaves original location [27]. These fteen interaction Holm et al [13] used object association to develop their moments benet users’ ability to interact with the virtual concept of the hierarchy tree. The hierarchy tree resembles world in a seamless and intuitive way, e.g., when drag leaves the inheritance tree of C++ where the distance of the branch the original location, the virtual object is updated according denes the compatibility of an object to snap onto other to the position of the mouse. objects (e.g., a vase in between a table and a bed would snap to the table even if the bed is closer in distance because the 3.2.2 Snap-dragging table is closer in class hierarchy). In 1986, Bier introduced the snap-dragging function for com- puters to simplify the task of drawing, translating, rotating, 3.2.4 Collision Detection and scaling 2D gures (i.e., lines and shapes) by snapping In 3D virtual worlds, collision detection plays an important to aligned lines, shapes, and vertices [4]. The snap-dragging role for perceived realism of users [16]. Specically, the pur- function can be viewed as an extension of a grid-system or pose of collision detection is to keep two or more virtual as an alternative constraint-based system. Bier then applied objects from overlapping one another. Collision detection the same concept of snap-dragging to 3D environments [3]. can be divided into two phases: the broad phase followed As described by Bier, the snap-dragging function is a com- by the narrow phase [17]. The broad phase reduces the num- bination of three interactive techniques: gravity, alignment ber of computations by collision culling - a calculation of object, and interactive transformation. The gravity technique the expected movement of moving objects. In the narrow snaps to points, curves, and surfaces of alignment objects. phase, the computation of collision for virtual objects is done Alignment objects act as a guideline to dene the reference in proximity to one another. Most commonly, axis-aligned points that objects can snap to. Finally, interactive transfor- bounding boxes (AABB) are used to simplify the computa- mation tracks motion and automatically snaps the object to tions of collision detection. A bounding box in 3D, as the specied vertice points. name suggests, is a volume in the form of a box encapsu- The intention with the snap-dragging function in 3D was lating the triangular mesh of the virtual object. The more to provide users with simplied interactions to create and intricate, precise, and power-consuming method is the com- modify 3D virtual worlds. Specically, snap-dragging min- putation of overlapping triangular meshes. Static bounding imizes an overload of information for users by decreasing boxes are preferred and continuously updating the bounding complex UI design and interactions. The snap-dragging func- box increases the computation time [16]. Unity’s built-in col- tion has been widely used in 3D world editors [9, 13, 20, 28]. lision detection utilizes both the mesh and AABB techniques. The AABB technique in Unity handles 3D primitive shapes 3.2.3 Constraints such as, boxes, spheres, and capsules. Mesh collider detects The manipulation of 3D environments using 2D screens and collision between 3D meshes [9]. input devices, such as a mouse and keyboard, can be quite tedious. This manipulation becomes even more complicated 3.2.5 Visual Indication when the input device is touch-based with a small screen i.e., A visual indicator is typically implemented in a 3D world smartphones. 3D world editing requires technical skills and editor as visual feedback for the user to signify selection 4 when the user selects (clicks) on a virtual object in the virtual to identify the most common interaction principles (see Ta- world. These visual indications are implemented dierently ble 2) and the most common user journey regarding the between software. In some cases, visual indicators look like world editors (see Figure 2). The apps were discovered by a bounding box that encapsulates the virtual object. This analyzing the top rated apps and the respectively suggested was also implemented by Goesele and Stuerzlinger [10]. An similar apps in dierent categories e.g. social, simulation, and alternative visual indication, used by Kovalčík et. al., in 3D interior design. The following table represents the apps dis- world editor is the change of the shader color upon selection covered and analyzed divided in three columns: 2D, 2.5D and of the virtual object [20]. 3D. 2D - apps in two dimensions. 2.5D - apps with semi three dimensional virtual objects. 3D - apps in three dimensions 4 Method & Implementation with adjustable camera view. In this section, both the method and implementation will be presented together in chronological order, following the Mobile apps whole research process from the beginning to the end. Phase 2D 2.5D 3D I - background market analysis on the most used interactions principles for 3D world editors on mobile phones and the Design Home Designer City Home Street most common user journey observed from the set of analyzed Boo Family Guy Minecraft apps (see Figure 2). The user journey and three frameworks (see Figure 3,4 and 5 respectively) were then designed accord- Home Design Forge of Empires My Cafe ingly and evaluated by ve experts following the principles Vlogger Go Viral Hay Day Planner 5D of heuristic evaluation. Phase II - focuses on the development of the 3D world editor prototype implemented with the ana- Zepeto Highrise Roblox lyzed interaction principles following the user journey. Only Megapolis Room Planner two frameworks were developed as one of the frameworks was eliminated in the process of the heuristic evaluation. Rollercoaster Ty- Swedish Home Phase III - covers the end user evaluation of the 3D world coon Touch Planner editor prototype. SimCity BuildIt The Sims Mobile Simpson Tapped Virtual Families 2 out Table 1. The 23 apps analyzed, presented in alphabetical order divided in sections of 2D, 2.5D and 3D.
4.1.2 User Journey The four use case interactions by Mine [22] were used as a foundation to nd the most common user journey, by ob- serving the 23 apps. The use case interaction turned out to be selection, placement and manipulation, these use case interactions could then be grouped together and form the following repetitive three step user journey.
Figure 1. Overview of the implementation process
4.1 Phase I. Background & Concept Design 4.1.1 Market Analysis A market analysis was conducted to nd the most commonly used interaction principles amongst mobile apps embedded with the component of world editor. A total of 23 applications Figure 2. Step by step, most common user journey for world (see Table 1) found on the Apple App Store were analyzed editors across 23 apps. 5 By identifying the most common user journey, a basic framework for 3D world editors could be designed. In line with the steps of the repetitive interaction cycle, users can select, place, and manipulate virtual objects in the virtual world. Specically, they can select by using a shop with a ltering method of categories, e.g, chair, bed, and table; place virtual objects in the 3D virtual world; and, manipulate a virtual object by rotating or removing the object from the scene. Three frameworks were designed to examine the eects of a grid, free, and predened system on the user experience. Figure 5. Framework III, eliminated as it scored worst out Specically, framework I was based on the most used inter- of the three. actions principles - namely, the ltered shop, grid-system and hold-manipulation interaction principles. Beyond the selec- tion step of step I (i.e., the shop), framework II followed the The following table represents the observed: interactions second most used interaction principles - namely, the free- from the 23 apps, the total number of apps implemented system and tap-manipulation interaction principles. Lastly, with the interaction, and a short description per interaction framework III was designed after the most unique observed principle. interaction - the predened interaction where each type of virtual object can only be placed at a predened area. Interaction Total Denition Based on the observed user journey and analysis on the Principles most used and most unique interaction principles, the follow- ing three frameworks became the product for the heuristic Shop 18 Categorized shop with ltra- evaluations: tion Inventory 7 Inventory list, usually fol- lowing the stack principle, rst in last out Predened 5 Designated areas for specic room furniture types Grid-system 14 A grid that assist the users with the action of placement Free-system 9 Full control over placement Collision detec- 14 Prevent virtual objects from tion intersecting one another Figure 3. Framework I, grid-system and hold to edit virtual objects. Tap- 10 Tap on virtual object to initi- manipulation ate manipulation mode Hold- 13 Hold on virtual object to ini- manipulation tiate manipulation mode Drag and drop 18 Translate by dragging Tap and drop 5 Translate by tapping Scaling 3 Size manipulation Color 5 Color manipulation Remove 22 Remove virtual object Rotation 15 Rotation of virtual object Table 2. Observed interactions from the analysis of 23 apps. Figure 4. Framework II, free-system, and tap-manipulation Divided in three sections for better distinction. virtual objects.
6 4.1.3 Heuristics Evaluation The interviews followed a semi-structured interview style Since the designed frameworks (Figures 3, 4, and 5) are more [7], where the issues were rst presented to the evalua- conceptual and the set of ten heuristics developed by Nielsen tors followed by a presentation of the ndings (see table were designed towards evaluating UI designs on desktop 2). Throughout the entire interview, it was paramount that [25], there were certain changes made for the applied use experts maintained an open dialogue with the principal in- of 3D world editor (e.g., ease of use and functionality). The vestigator. The interview concluded once the experts rated following set of heuristics used in this research was inspired each framework across each heuristic category. Specically, by Nielsen’s set of ten heuristics [25] and the works of Huy these ratings were given on a ve point scale. and Vanthanh [18]: After the heuristic evaluations, it was safe to eliminate the framework with the lowest score, framework III, from the study. The focus was then shifted towards frameworks I Heuristic Description and II for development and user testing. Frameworks I and Visibility of sys- The framework should keep users II mainly dier in their step II (placement). Thus, the two tem status informed about what is going on, frameworks could be compared from the standpoint of a grid- through appropriate feedback. system versus a free-system design. The grid-system is when the editor presents a grid upon selection to assist the user User control The framework provides the users interactions of e.g., translation and rotation in increments. and freedom with suitable controls and ways to Alternatively, a free-system design gives the user full control customize their own home/room. in translation to the decimal, which provides a more accurate Error preven- The framework prevents errors placing. tion from occuring. Error such as e.g. The following is a summary of the average score from the stacking objects on each other. heuristics evaluation by the ve experts. Flexibility and The framework entails exibility Heuristic I II III eciency of and eciency in room design. use Visibility of system status 4 3 4 Customization/ The framework entails opportunity User control and freedom 4 4 1 Personalization for customization/personalization. Error prevention 5 2 5 Ease of use The framework entails an easy in- teraction. Flexibility and eciency of use 3 3 2 Functionality The framework entails functional- Customization/Personalization 4 5 2 ities suitable for the intended out- Ease of use 5 3 5 come. Functionality 4 4 3 Non-limitation Limitations from the user’s perspec- tive. Non-limitation 4 3 1 Continuity Will keep the user’s to continuously Continuity 5 4 3 re-design their room. Accumulated score 8 4 3 Table 3. Description of the heuristics, the rst ve heuristics Table 4. Description of the heuristics, the rst ve heuristics come from Nielsen’s original ten heuristics, the other four come from Nielsen’s original ten heuristics, the other four from Huy and Vanthanh research. The description of the from Huy and Vanthanh research. The description of the heuristic has been adapted to t the purpose of this research. heuristic has been adapted to t the purpose of this research.
With the set of heuristics and three frameworks nalized, The experts were asked to express their opinions regard- ve evaluators were invited for individual interviews to va- ing the frameworks divided in the three steps. When it came lidify, suggest, and rank the three frameworks. Four out of to step I (selection), the general consensus was that a well cat- the ve interviews were held through online video calls with egorized UI with ltering option would be benecial for large Google Hangouts, whereas one was conducted in-person. sets of virtual objects, which was also observed for the apps Additionally, the evaluators came from various backgrounds with an inventory list that was stacked, as nding specic with occupations ranging from UX Designer to Game Art virtual objects then became tedious. For step II (placement) Director. the experts expressed that the free-system would yield the 7 feeling of customization while the grid-system gives clear Lastly, all virtual objects were assigned a representative guidelines and is more suited for the intended audience. An- tag(second level category) and Layer (wall, oor, furniture, other noteworthy opinion was that both frameworks have frame). Layers in Unity are used to assist with culling for the fairly simple interactions, and the threshold to learn is mini- Physics engine, which is used in collision detection. mum (i.e., regardless of which system is used for placement of virtual objects, there will not be a huge dierence, rather a 4.2.2 Interactions subjective opinion). The grid-system will benet users who The principle of the snap-dragging function by Bier [3, 4] was yearn for guidelines and less precision, while the free-system implemented for Framework I (grid-system), where virtual will benet users who prefer more control over the 3D world objects snap along the axes (depending on the type of virtual editor. When it comes to accessing the manipulation mode object, furniture, or frame) when dragged. The drag and drop for virtual objects, the experts’ opinions were divided, some interaction principle [27] was implemented in both frame- argued that Tap is better than hold to initiate manipulation works as a result of the original market analysis. The drag mode, as it is a quicker interaction, while others argued that and drop interaction principle was implemented to avoid er- the interaction of holding down prevents error, which can ror of misplacement when translating virtual objects, which occur if a user accidentally taps on a virtual object. can occur with the tap and drop. Lastly, the three frameworks were also ranked against one Upon selection of virtual objects, a change of color was another by the experts, from one as the most favorable frame- implemented as done by Kovalčík et. al [20], the color chosen work to three as the least favorable framework. Framework I to dierentiate a selected virtual object to an arbitrary virtual was ranked as rst choice by three experts, followed closely object was a shade of purple as no virtual object had a similar by Framework II, and Framework III was ranked the worst color scheme. by all ve experts. 4.2.3 Camera 4.2 Phase II. Development of the 3D World Editor For this particular study the interaction of the camera was Prototype neglected to solely focus on virtual object selection and ma- The 3D world editor was developed in Unity (v. 2019.3.0f3) nipulation, however work has been made. Camera interac- targeted towards iPhone X (iOS v. 13.3.1. resolution: 2436-by- tion is described as the interaction principle of navigation by 1125 pixel). Apart from Unity’s built-in Renderer, which was Mine(see chapter 3.1.1). With Unity built-in camera, it is easy substituted for Universal Rendering Pipeline (v.7.1.6, Frever to change camera view, for this prototype, perspective was app standard), all other standard libraries and packages were used. A script was added to the camera so that rotation and unchanged. To build from Unity to iPhone, Xcode (v.11.2.1) zoom was handled by simple touch interactions. To zoom - was used. pinch movements. To rotate - left/right swipes. The camera was constantly looking towards the middle and there was no 4.2.1 Assets way to move it otherwise, so when rotating, it spun around 3D assets in terms of 3D model, shaders, and material were the middle. bought in the Unity asset store to eliminate time spent on designing 3D models and to focus on the user interaction. To 4.2.4 Constraints reduce the time it took to design 3D models, the amount of The concept of a resting plane, introduced by Houde [14], furniture, i.e., virtual objects, were restricted to four pieces was implemented where furniture was bound to the oor of furniture per second level category e.g., bed, sofa, cab- by reducing translation in one axis. Hanging items, such as inet, table, chair, frames, table props, standing lamp, and frames, were exposed to all three axes but constrained to be plants. First level categories included furniture, wallpaper, attached to walls (depending on the rotation of the wall in and ooring. 3D space, one axis would be turned o). Furthermore, the un- As virtual objects were bought online, a one-by-one clean- derlying concept of object association, coined by Bukowski ing of the asset was done to remove any unwanted and un- and Sequin [6], was also exposed in the 3D world editor, giv- necessary scripts and components. All types of furniture had ing users the opportunity to stack smaller objects on tables. to be assigned an empty parent GameObject to x the error The action of stacking objects was restricted to only allow of pivot positioning (i.e., keeping dierent assets packages placement of the smaller objects in the middle of the table furniture to the same standard). Most importantly, the addi- regardless of the framework. tion and adjustment of the ObjectInfo-script and a BoxCollider component was done to handle everything regarding the vir- 4.2.5 Collision Detection tual object: collision detection, translation, rotation, removal, Unity’s built-in collision detection methods include colli- and change of shader color. The BoxCollider component is sion between meshes and the AABB method. Therefore, no essential for collision detection in Unity (explained further further implementations were developed as computational under collision detection). complexity was not a necessity or focus of this research. 8 Unity’s OverlapBox function, which is a part of the AABB agree. This online questionnaire activity was conducted to method, was utilized and implemented for the purpose of gain qualitative numerical data of statements to further in- this research. The function was called on each time the se- vestigate in-depth the UX per interaction principle. After all lected virtual object was translated, where it checked for any the tasks were completed, a short semi-structured interview other overlapping virtual object but itself. If the function was conducted to acquire qualitative data that would inform detected an overlapping, the selected virtual object changed the results of quantitative data. shader color to red. If there was no collision detected, the color switched to green until the done or cancel button was pressed. After either of these buttons were pressed, the vir- 4.3.3 Pilot Study tual object changed back to its original color scheme. Before the study began, a pilot study was conducted to gauge the approximate time to complete the user study. The pilot 4.2.6 Shop UI study also acted as an opportunity to nd what was working The UI for the shop was created with Unity’s canvas system. and what needed to be xed. At this stage of development, The UI components consisted of button, text, and panel, with some task descriptions and questionnaire text had to be some panels employing Unity’s ScrollRect script for scrol- rephrased for clarity. Additionally, one task was removed lable panels. Part of this research is aimed at designing an from the task compilation as it made the overall test too extension prototype of the Frever app; therefore, the UI for long (originally 50 reduced to 30 minutes), allowing for the the shop followed the design principles of the Frever app. possibility of user fatigue that impacted the results of the Coincidentally, the design principles used in the Frever app pilot study. correlated with the market analysis of categorized shops seen in Table 2. For nal result see Figure 6b. 4.3.4 User’s Thoughts & Observations 4.3 Phase III. User Study Just as the heuristic evaluation, the users’ commentary from 4.3.1 Background the interviews were logged, analyzed and compiled. Com- After development of all the aforementioned interactions, pared to the heuristic evaluation where the opinions were on methods, and functions, a user study was designed to test a conceptual level, the users commentary were based on the frameworks I and II. The tests were held individually, just actual prototype of the 3D world editor, which means that like the heuristic evaluation workshops, either through on- the users opinion were subjective to the prototype. Notable line calls with screen sharing of the phone or in-person commentary from two users regarding step I - was to make (three online, seven in-person). All user studies were screen use of generic thumbnails when ltering of category, as they recorded. believed that the size of the category buttons were too small and a thumbnail would yearn a quicker overview of the con- 4.3.2 Tasks tent of the category. For step II, the overall consensus was There were eight tasks in total, four per framework. The similar to what the experts expressed - the grid-system made rst task of each framework was identical; however, the it easier to place virtual objects, but the free-system provides remaining three tasks diered depending on the framework more precision and the perception of freedom to place as en- being tested. The order of framework - and corresponding visioned. Two users expressed that they would rather prefer set of tasks - presentation was counterbalanced across the a smaller grid-system if the grid-system were used, as that ten users to minimize any bias from order of presentation would provide more precision similar to the free-system, but [7]. Specically, ve users received framework I before II with guidelines and the snap function made easier for plan- while the other ve received framework II before I. ning and interaction. Overall, 8/10 of the users expressed that (1) Task 1 evaluated how users found specic virtual ob- they would prefer the grid-system over free-system. Step jects using the UI-based shop. (2) Task 2 examined the place- III - the general opinion was once again divided, half of the ment of virtual objects. (3) Task 3 investigated users’ abili- users preferred hold-manipulation over tap-manipulation ties to initiate manipulation mode within the virtual world, as it prevents the users from accidentally initiating manip- and the manipulations of rotation, translation and deletion. ulation for the wrong object. The other group thinks that Lastly, (4) task 4 presented users with a pre-furnished room hold-manipulation made it more tedious to make changes, where they were asked to manipulate items within the en- this could also be due to the duration of the hold (0.5 seconds) vironment without guidance. After the completion of each The most notable commentary from a user was that their task, users were asked to ll out an online questionnaire rst intuition is to tap, while holding would the interaction about their experience completing the task. Specically, the principle to initiate a more advanced manipulation mode (e.g. questionnaire allowed users to input their opinions on a ve- tap - to translate virtual objects. Hold - to initiate rotation point Likert-scale ranging from strongly disagree to strongly and removal of virtual objects). 9 4.4 Final Results 4.4.1 UI The following gures (Figure 6a and 6b) depict the nal UI design for the Frever app and user study. In gure 6a, the next button takes the user to the next tasks while the i button shows the task description. In Figure 6b, we see dierent elements of the shop UI: (1) opens the shop; (2) the scrollable horizontal furniture ltering; (3) the scrollable horizontal panel of virtual objects; (4) the expansion button that expands the furniture panel; and, (5) the scrollable horizontal panel for rst level categories (i.e., furniture, wallpaper, and ooring).
(a) Grid-system (b) Free-system
Figure 7. (a) and (b) showcase the nal manipulation mode UI mode. The slider is for rotation of the virtual object.
4.4.2 Quantitative Data By timing the completion per task, the following two graphs (Figure 8 and 9) respectively two tables (table 5 and 6) were given. This analysis was done to further investigate if there was a signicant dierence between the two frameworks or if it was due to familiarity i.e. iterations when it comes to the time spent to complete a task, another possibility could be up to chance. The graphs represent the average time for the ten users to complete each task. Figure 8 - between frameworks. Figure 9 - between iterations. The two tables demonstrate the results of calculating the signicant dierence using the (a) Task description. (b) Shop UI statistical model, T-test. Table 5 - between frameworks. Table 6 - between iterations. Figure 6. (a) and (b) demonstration of how the app was designed for the user studies.
Figure 7 demonstrates the interaction between grid respec- tively the free-system. In gure 7a a white grid is displayed over the oor to assist the users in planning and placement of virtual objects. Figure 7b shows the UI of free-system, without a grid for planning assistance. The UI panel at the bottom presents the actions available in manipulation mode: Figure 8. Average time of completion rotation, and removal. The Cancel button on the left is used to cancel or undo any changes done in manipulation mode. Statistical T-test calculations between the time of comple- Lastly the Done button is used to approve any manipulation tion for the two frameworks proves that there is no signif- made to the selected green (visual indication[20]) virtual icant dierence between the two frameworks (see Table 5 object. below) as the results proved to be up to chance. 10 opportunity to buy virtual objects. Additionally, it was bene- Task 1 Task 2 Task 3 Task 4 cial to employ Frever app’s shop UI because it would cut P 0.824 0.459 0.239 0.795 time drastically on the development end. The majority of the Table 5. T-Test: if the P value is underneath 0.05, there is a 23 apps had a ltering function to lter in-between dierent signicant dierence between frameworks. categories and a thumbnail of the virtual objects. Looking at the nal user study, it is safe to conclude that the shop was intuitive and easy to follow; however, the users suggested having a representative thumbnail of categories, instead of a tiny text tag bar above the shop panel. This would have provided a quicker overview of what the shop had to oer. Apart from that, all users found what they were supposed to look for and used the ltering function without instructions.
5.2 Placement Regarding step II, the placement interaction, the three frame- works provided distinct interactions in terms of placing vir- tual objects - specically, they oered a grid, free, and pre- dened system, respectively. By examining the interaction principle analysis table (Table 2), one could assume that the predened system would have a better ranking amongst Figure 9. Average time to complete the tasks where the the ve experts. However, the experts from the heuristic order of framework was not considered. evaluation saw Frameworks I and II as two very similar sys- tems with Framework I providing greater simplicity to the The table below proves that there is a signicant dierence placement of virtual objects. The end users expressed similar in terms of completing a task regardless of the framework. thoughts, believing that a grid-system was more intuitive Task 4, was the only task with an open ended description and easier to manipulate than a free-system. In short, a grid- for the users to decide whether they wanted to spend more system gave an instant visualization of where a virtual object or less time. would land whereas a free-system compromised such ease for greater precision. One end user suggested the capability Task 1 Task 2 Task 3 Task 4 to switch modes, between a grid- and free-system, in order to give users precision or ease on their own terms. P 0.001 0.000 0.000 0.152 Table 6. T-Test: if the p value is underneath 0.05, there is a 5.3 Manipulation signicant dierence between rst and second iteration. For tap versus hold interactions, there were mixed reviews amongst experts and end users as both actions work in simi- lar ways. Based on observation and verbal suggestions made by both experts and end users, it can be determined that a 5 Discussion combination of both tap and hold interactions would be the The initial question this research aimed to answer is: What most appropriate option. Specically, by a combination of are the best practice interaction principles for 3D world tap and hold, one is referring to tap to translate, and hold to editors on mobile phones from a user experience perspective? enter more advanced manipulation options such as rotation In the following, one can determine the extent that there are and deletion. This conclusion is based on observations of certain best practice interaction principles that should be users accidentally entering the manipulation mode for the used on 3D world editors for mobile phones. Additionally, wrong virtual object. limitations of the research and areas of further research will be discussed. 5.4 Quantitative result When it comes to eciency or usability, there was no sig- 5.1 Selection nicant dierence between the frameworks. However, by Firstly, the shop UI was developed according to the analyzed examining Figure 9 and Table 6, one can see a signicant dif- most used interaction principle for selection of virtual objects ference in time of completion between the rst time testing ( implemented in 18 out of the 23 apps, see Table 2), coinci- the app and the second time, regardless of the frameworks. dentally, the interaction principle for the shop UI was similar This further proves that the 3D world editor prototype devel- to the Frever apps shop UI. Most of the 23 apps designed a oped in this research has an easy UX design, resulting in bet- virtual object selection section as a shop where users had the ter user performance after practice, regardless of framework. 11 Thus, neither framework contributes to greater eciency of from a between-subject model with a bigger user population 3D world editors, but familiarity of the app did. that will allow for more isolated tasks, i.e., there were no isolated tasks for the rotation interaction for dierent sized 5.5 3D World editors virtual objects as some occupy more grid area than others. World editors in general are very constrained in terms of interaction independent of the input device. Traditionally for 7 Future Work computers, the UI tends to be quite complex and cluttered, For future work, implementing a smaller grid-system (smaller as mentioned by Ali et. al. [1]. A big research interest in 3D than one unit) would be of interest. The smaller the grid, the world editors is unfortunately in virtual reality (as seen in more similar to the free-system the grid-system will become. chapter 3). What distinguishes these two input devices from There must be a threshold for how small or big the grid size mobile phones is that everything from the virtual world to has to be to make a signicant or insignicant dierence in the input device is within the palm of one’s hand. terms of UX. There are disadvantages when it comes to mobile phones, The interactions analyzed in this research were based as the gate to the virtual world and input device on the same upon the market analysis, other interaction principles utiliz- surface. While 3D world editors on computers can clutter the ing e.g. augmented reality (AR) technology or accelerometer UI with e.g. buttons and information, mobile phones tend to may change the outcome and be a better practice to imple- simplify as much as possible, as the screen is smaller. This ment for mobile phones. leads to simpler UI design by reducing the amount of text For evaluation of the prototype, this research solely fo- and button, which this research tried to do by designing cused on the device choice of iPhone X and teenagers from everything to the bottom of the screen. Sweden. In the future a broader evaluation in terms of de- Virtual reality on the other hand allows the user to fully vices of dierent phone resolution, size and operative sys- immerse and physically translate in the virtual world, again, tems as well as users of dierent ages and cultures will give a mobile phones have to simplify interaction principles, and broader understanding if the choice of interaction principles make things work with minimum eort on a small screen, is universal or solely based on device, age and culture. therefore an example in this research is to lock the camera position to the middle of the room to avoid unwanted trans- 8 Conclusion lation in 3d space by constraining the 3D world editor as As there are interaction principles in 3D world editors not much as possible. covered in this research, and the number of subjects were minimal. No general conclusion can be drawn for all 3D 6 Method Critique world editors on mobile phones. However, for this particular The most ecient collision detection method in Unity is research and the interaction principles observed, it is save the function called OnCollisionEnter; however, a problem to conclude that a combination of a well organized catego- arose when designing this prototype because Unity does rized shop for selection of virtual objects, a grid-system (with not call the functions if the virtual object is not moving in smaller increments) for the placement of virtual objects, for space by Unity’s Physics engine. Therefore, a function called the use case of manipulation: drag and drop for translation OverlapBox was used instead. Using OverlapBox resulted in a and hold-manipulation (less than 0.5 seconds with visual dierent problem, however, as this function was called after feedback) for initiating manipulation mode (rotation and each translation and it had to re-calculate the bounding box deletion) of virtual object; would be the best practice inter- for each function call. This resulted in a minor fps drop as action principles for this prototype of 3D world editor on more computational power was used on the phone; however, mobile phones. it was not once mentioned by the users, only noticed by the author (developer) himself. Acknowledgments The grid-system was implemented so that virtual objects Thanks to the whole team at Friend Factory AB. For my time were translated in increments by one unit (one Unity unit at KTH, I want to thank Max Turpeinen, Mario Romero, and is supposedly equivalent to one meter in real life), which specically Björn Thuresson. Last but not least, thank you resulted in a limitation of placement. Erica M. Williams. Due to Covid-19, readjustments in terms of test users had to be taken. Therefore, only a small number of users were invited to the study as well as the study had to be conducted References under a within-subject experimental model. It minimized the [1] Nazlena Mohamad Ali, Siti Zahidah Abdullah, Juhana Salim, and Hy- owon Lee. 2013. Exploring user experience in game interface: a case opportunity to have more thorough and isolated tests. This study of The Sims 3. The Computer Games Journal 2, 1 (2013), 6–18. was specically observed as the users became confused when [2] Javier A Bargas-Avila and Kasper Hornbæk. 2011. Old wine in new the same tasks resurfaced again. The study would benet bottles or novel challenges: a critical analysis of empirical studies of 12 user experience. In Proceedings of the SIGCHI conference on human Electrical & Electronics Engineering (EEESYM). IEEE, 589–591. factors in computing systems. 2689–2698. [17] Philip M Hubbard. 1993. Interactive collision detection. In Proceedings [3] Eric A Bier. 1990. Snap-dragging in three dimensions. ACM SIGGRAPH of 1993 IEEE Research Properties in Virtual Reality Symposium. IEEE, Computer Graphics 24, 2 (1990), 193–204. 24–31. [4] Eric A Bier and Maureen C Stone. 1986. Snap-dragging. ACM SIG- [18] Ngu Phuc Huy and Do Vanthanh. 2012. Evaluation of mobile app GRAPH Computer Graphics 20, 4 (1986), 233–240. paradigms. In Proceedings of the 10th International Conference on Ad- [5] Doug A Bowman, Donald B Johnson, and Larry F Hodges. 2001. vances in Mobile Computing & Multimedia. 25–30. Testbed evaluation of virtual environment interaction techniques. Pres- [19] Laurie Kantner and Stephanie Rosenbaum. 1997. Usability studies of ence: Teleoperators & Virtual Environments 10, 1 (2001), 75–95. WWW sites: Heuristic evaluation vs. laboratory testing. In Proceedings [6] Richard W Bukowski and Carlo H Séquin. 1995. Object associations: a of the 15th annual international conference on Computer documentation. simple and practical approach to virtual 3D manipulation. In Proceed- 153–160. ings of the 1995 symposium on Interactive 3D graphics. 131–. [20] Vít Kovalčík, Jan Flasar, and Jiří Sochor. 2007. Extensible approach to [7] Dana Chisnell and Jerey Rubin. 2008. Handbook of usability testing: the virtual worlds editing. In Proceedings of the 5th international confer- How to plan, design, and conduct eective tests. Computer Bookshops ence on Computer graphics, virtual reality, visualisation and interaction Limited (2008). in Africa. 31–37. [8] Anja Endmann and Daniela Keßner. 2016. User Journey Mapping–A [21] Kathryn Elizabeth Merrick and Mary Lou Maher. 2007. Motivated rein- Method in User Experience Design. i-com 15, 1 (2016), 105–110. forcement learning for adaptive characters in open-ended simulation [9] Will Goldstone. 2011. Unity 3. x game development essentials. Packt games. In Proceedings of the international conference on Advances in Publishing Ltd, Chapter 1, 16–17. computer entertainment technology. 127–134. [10] Michael Gosele, Wolfgang Stuerzlinger, et al. 1999. Semantic con- [22] Mark Mine. 1995. ISAAC: A virtual environment tool for the interactive straints for scene manipulation. In in Proceedings Spring Conference in construction of virtual worlds. UNC Chapel Hill Computer Science Computer Graphics’ 99 (Budmerice, Slovak Republic. Citeseer. Technical Report TR95-020 (1995). [11] Marc Hassenzahl. 2018. The thing and I: understanding the relationship [23] Rolf Molich and Jakob Nielsen. 1990. Improving a human-computer between user and product. In Funology 2. Springer, 301–313. dialogue. Commun. ACM 33, 3 (1990), 338–348. [12] Marc Hassenzahl and Noam Tractinsky. 2006. User experience-a re- [24] Jakob Nielsen. 1994. Usability engineering. Morgan Kaufmann, Chap- search agenda. Behaviour & information technology 25, 2 (2006), 91–97. ter 2. [13] Roland Holm, Erwin Stauder, Roland Wagner, Markus Priglinger, and [25] Jakob Nielsen. 1994. Usability inspection methods. In Conference Jens Volkert. 2002. A combined immersive and desktop authoring companion on Human factors in computing systems. 413–414. tool for virtual environments. In Proceedings IEEE Virtual Reality 2002. [26] Jakob Nielsen and Rolf Molich. 1990. Heuristic evaluation of user IEEE, 93–100. interfaces. In Proceedings of the SIGCHI conference on Human factors in [14] Stephanie Houde. 1992. Iterative design of an interface for easy 3-D computing systems. 249–256. direct manipulation. In Proceedings of the SIGCHI conference on Human [27] Bill Scott and Theresa Neil. 2009. Designing web interfaces: Principles factors in computing systems. 135–142. and patterns for rich interactions. " O’Reilly Media, Inc.". [15] Hamid Hrimech, Leila Alem, and Frederic Merienne. 2011. How 3D [28] Graham Smith and Wolfgang Stuerzlinger. 2001. Integration of Con- interaction metaphors aect user experience in collaborative virtual straints into a VR Environment. In VRIC’01: Proc. of the Virtual Reality environment. Advances in Human-Computer Interaction 2011 (2011). Int’l Conf. 103–110. [16] Rui Huang. 2012. Optimizing collision detection in 3D games with [29] Rosa Yáñez Gómez, Daniel Cascado Caballero, and José-Luis Sevillano. model attribute and Bounding Boxes. In 2012 IEEE Symposium on 2014. Heuristic evaluation on mobile interfaces: A new checklist. The Scientic World Journal 2014 (2014).
13 TRITA -EECS-EX-2020:494
www.kth.se