An Investigation of Display Shapes and Projections for Supporting Spatial Visualization Using a Virtual Overhead Map

by

Catherine Solis

A thesis submitted in conformity with the requirements for the degree of Master of Applied Science Graduate Department of Mechanical & Industrial Engineering University of Toronto

c Copyright 2020 by Catherine Solis Abstract

An Investigation of Display Shapes and Projections for Supporting Spatial Visualization Using a Virtual Overhead Map

Catherine Solis Master of Applied Science Graduate Department of Mechanical & Industrial Engineering University of Toronto 2020

A novel map display paradigm named “SkyMap” has been introduced to reduce the cognitive effort associated with using map displays for wayfinding and navigation activities. Proposed benefits include its overhead position, large scale, and alignment with the mapped environment below. This thesis investigates the substantiation of these benefits by comparing a conventional heads-down display to flat and domed SkyMap implementations through a spatial visualization task. A within-subjects study was conducted in a virtual reality simulation of an urban environment, in which participants indicated on a map display the perceived location of a landmark seen in their environment. The results showed that accuracy at this task was greater with a flat SkyMap, and domes with stereographic and equidistant projections, than with a heads-down map. These findings confirm the proposed benefits of SkyMap, yield important design implications, and inform future research.

ii Acknowledgements

Firstly, I’d like to thank my supervisor Paul Milgram for his patience and guidance over these past two and a half years. I genuinely marvel at his capacity to consistently challenge me to improve as a scholar and yet simultaneously show nothing but the utmost confidence in my abilities. He often said that I remind him of himself; one can only hope to live up to such a standard. I would also like to thank the other members of my committee, Greg Jamieson and Justin Hollands, with whom I was privileged to collaborate throughout the entire process of this thesis. Their support and feedback undoubtedly elevated the quality of my work. I was also very fortunate to work with a group of talented and thoughtful graduate students and post-docs, and I’m grateful for the time and energy they spent with me in discussion about the research. I would also like to send my gratitude to Mindy Thuna at the Engineering Library, who let me pick her brain on multiple occasions about effective research strategies. Though graduate school can be a challenging and often isolating experience, my colleagues in the lab were a source of support, empathy, and friendship that I hope to carry with me well into the future. Joey, Adam, and Fahimeh: I can’t thank you enough for allowing me to bring my whole self into the office every day. I would like to thank my family, both from birth and chosen, for supporting me throughout this degree. My mother not only provided me the opportunity to pursue further studies, through her sacrifice and perseverance, but also modelled for me the strength and confidence I needed to face the difficulties set before me. Finally, I must thank my loving partner and best friend Charlie White, unwavering source of kindness and support. Having them by my side throughout this whole journey inspired me to grow as a person, and take so much more from this experience than I could have possibly imagined. It is a joy and a blessing to share my days with you, and my life is enormously, authentically better with you here.

iii Contents

1 Introduction and Motivation 1 1.1 Map-aided Wayfinding and Navigation ...... 1 1.2 SkyMap: A New Display Paradigm ...... 2

2 Background & Problem Scoping 4 2.1 Defining the Research Space ...... 4 2.2 Investigating a Dome-shaped SkyMap ...... 5 2.2.1 Task-optimized distorted displays ...... 7 2.3 Traditional Cartographic Projections ...... 8 2.3.1 Classification of projections ...... 8 2.3.2 Azimuthal cartographic projections ...... 9 2.4 Review of Literature Addressing Spatial Cognition and Map Use ...... 11 2.4.1 Cognitive processes associated with map-aided wayfinding ...... 12 2.5 Research Questions ...... 13

3 Experimental Method 14 3.1 Experimental Design ...... 14 3.1.1 The experimental task ...... 14 3.1.2 Map conditions ...... 17 3.1.3 Measurement of potentially mediating factors ...... 18 3.1.4 Non-performance measures ...... 20 3.2 VR Implementation of Experimental Environment ...... 20 3.2.1 Software ...... 20 3.2.2 Trial configurations and counterbalancing ...... 21 3.2.3 Hardware & physical setup ...... 22 3.3 Experimental Procedure ...... 23 3.3.1 Participant recruitment & screening ...... 23 3.3.2 Introduction and presentation of experimental task ...... 23 3.3.3 Experiment debrief ...... 25 3.4 Experimental Variables ...... 26 3.5 Experimental Hypotheses ...... 28 3.5.1 Hypothesis 1: SkyMap vs. heads-down display ...... 28 3.5.2 Hypothesis 2: Relative performance of dome projections ...... 28 3.5.3 Hypotheses about influence of trial configuration ...... 29

iv 4 Results 30 4.1 Overview ...... 30 4.2 Results by Trial Block ...... 32 4.2.1 Effect of map condition on Mean Absolute Error ...... 32 4.2.2 Effect of map condition on mean visual angle error ...... 35 4.2.3 Effect of map condition on mean response time ...... 37 4.2.4 Effect of map condition on mean trial confidence ...... 38 4.2.5 Effect of map condition on mental workload ...... 39 4.3 Analysis of Performance by Trial Configuration ...... 40 4.3.1 Significance testing of AE & VAE data ...... 43 4.3.2 Directional error ...... 44 4.4 Individual Differences ...... 45 4.5 Feedback & Qualitative Observations ...... 46

5 Discussion 47 5.1 Substantiation of SkyMap Advantages ...... 47 5.2 Effects of Trial Configuration on Map Condition Differentiation ...... 48 5.2.1 Interpreting the meaning of visual angle error ...... 48 5.3 Study Limitations ...... 51 5.3.1 Limitations & potential extensions of data analysis ...... 51 5.4 Considerations for SkyMap Implementation ...... 52

6 Conclusion 53 6.1 Summary of Key Findings ...... 53 6.2 Implications for SkyMap Design and Future Research ...... 54

References 55

Appendices

Appendix A: Details of Implementation of Inverse Polar Azimuthal Projections ...... 60 Appendix B: Experiment Forms and Questionnaires ...... 64 Appendix C: Supplementary Information & R Output from Data Analyses ...... 75

v List of Tables

2.1 Initial SkyMap shape considerations ...... 6 2.2 The five principal azimuthal projections and their preserved spatial properties ...... 10

3.1 Screenshots of all map conditions ...... 18 3.2 The 16 layouts defining the relative position of the green tower to the two reference towers . 22 3.3 Independent & predictor variables (by trial, trial block, and individual) ...... 26 3.4 Dependent variables (by trial, trial block, and individual) ...... 27

4.1 Tukey’s HSD values from post hoc analysis of MAE model ...... 33 4.2 Tukey’s HSD values from post hoc analysis of MVE model ...... 36 4.3 Correlation table comparing individual differences to overall task performance ...... 45

vi List of Figures

2.1 Three common developable map surfaces and their corresponding projection types ...... 8 2.2 Illustration of lines of longitude & circles of latitude for different azimuthal projections . . . . 9

3.1 Screenshot of example experimental task with the flat SkyMap condition ...... 14 3.2 Photograph of fun-house mirror ...... 15 3.3 Screenshots of the dome mesh and map texture ...... 21 3.4 Finite State Machine diagram of the experimental task procedure ...... 25

4.1 Histogram of Absolute Error data for all trials (excluding outliers) ...... 31 4.2 Distribution of Mean Absolute Error values by map condition ...... 32 4.3 Distribution of Mean Visual Angle Error values by map condition ...... 35 4.4 Distribution of mean response times by map condition ...... 37 4.5 Distribution of mean confidence ratings by map condition ...... 38 4.6 Distribution of mental workload ratings by map condition ...... 39 4.7 Absolute Error distribution by trial layout and distance ...... 40 4.8 Visual Angle Error distribution by trial layout and distance ...... 41 4.9 Absolute Error means by trial layout, distance and map condition ...... 42 4.10 Visual Angle Error means by trial layout, distance and map condition ...... 42 4.11 Density contours showing horizontal and vertical visual angle deviations by trial layout, dis- tance and map condition ...... 44

vii Chapter 1

Introduction and Motivation

1.1 Map-aided Wayfinding and Navigation

Visual representations of environments have been used for millennia for the description and understanding of spatial information. Maps, more specifically, have existed in different forms throughout many civilizations (Tufte, 1983). Though the display technology has since become more sophisticated, its aims have remained largely unchanged: to retain and share knowledge about an environment, specifically the locations of the elements it contains. One task facilitated by maps that is of particular interest in this thesis is that of wayfinding. Wayfinding can be described as the activity of moving through an environment towards a specific target or location (destination). I make a distinction here between wayfinding and navigation, which both fit the above description, by further specifying that wayfinding involves some sort of cognitive representation of a space that is accessed in order to make decisions about what route to take to arrive at the intended destination (Taylor, Bruny´e,& Taylor, 2008). This is distinct from navigation in that navigation does not rely on an existing representation of a space, but instead involves the use of cues, in the environment and/or provided by an aid, that indicate what route to follow. As the technology around maps and other wayfinding aids has advanced remarkably over recent decades, maps have become a tool for everyday use. The modern road map, whose visual style persists today, proliferated in the U.S. during the early 20th century with the rise of the automobile, as petrol stations gave away billions of them to help drivers navigate the roads (Yorke & Margolies, 1996). Around the same time, the Michelin Group tire company were giving away free guidebooks with maps to encourage driving (and tire purchasing) in France and neighbouring countries (Mayyasi, 2016). The development of consumer- accessible GPS technologies in the 1990s (Longley, Goodchild, Maguire, & Rhind, 2005) provided the ability to continuously identify your own location on a geographic representation. Navigational support systems built around this technology have introduced several other features, such as the identification of intended destinations’ location, the displaying of possible routes, and information to support route selection, all leading up to the current state of turn-by-turn navigation with naturalistic, conversational voice guidance. Though the use of maps reduces reliance on an existing cognitive spatial representation for wayfinding, and thus makes possible navigation in unfamiliar environments, it introduces new cognitive requirements for the process of relating information in the real-world environment to the corresponding map depictions. Navigating with maps requires interpreting where you are, locating where you want to go, deciding how

1 Chapter 1. Introduction and Motivation 2 to get there, realizing your planned trajectory, and determining while doing so whether you are going the right way. Montello et al. (2004) referred to the real-world environment and its map representation as, respectively, direct and indirect experiences of a space, and several studies have proven that relating the information from these two sources can be mentally effortful (Liben, 2009; Ishikawa & Kastens, 2005). Some of the cognitive expenditures associated with aided wayfinding are discussed in detail in Section 2.4.1. The cognitive effort associated with map reading has also been conceptualized in literature as map read- ability, which comprises the map reader’s ability to understand the information on a hierarchical range from sensory perceptibility to intelligibility (Garmiz, Manichev, & Mironov, 1988). Readability was acknowledged by Garmiz et al., and by Harrie, Stigmar, and Djordjevic (2015), to not be attributable to specific prop- erties of graphical elements, but instead reliant on several aspects of the map representation. Thus, while improving map readability would enhance wayfinding support, the method by which it can be done is not straightforward. The argument for enhanced readability of maps is further strengthened by the proliferation of GPS navigation technology, which further reduces the amount of cognitive resources dedicated to reading map information. The route selection and following activities involved with paper map use require interpreting (and internalizing, at least temporarily) the map’s spatial relationships with respect to the position of the navigator, the destination, and intermediate cues along the selected route. With a GPS device, on the other hand, interface designs may facilitate or even prevent these interpretations. This argument is supported by findings in multiple studies that have shown that in-vehicle GPS users glance more often at a turn-by-turn display than a map display (Brooks, Nowakowski, & Green, 1999) and exhibit other forms of disengagement from the environment (Leshed, Velden, Rieger, Kot, & Sengers, 2008). Furthermore, map-based navigation strategies have led to better recall of spatial information relative to turn-by-turn navigation (Dey, Karahalios, & Fu, 2018; Khan & Rahman, 2018). Though M¨unzer,Zimmer, and Baus (2012) identified this same trend, they acknowledged that appropriate visualization design of the display could still support internalization of spatial information, even in the presence of navigation aids. Since performance from an internalized representation of a space results in more efficient navigation than with the use of maps (Taylor et al., 2008), facilitating the development of a reliable and accurate mental representation should be of importance to map designers, regardless of whether additional navigation support is provided. In summary, improvement of map readability stands to benefit the proficiency not only of navigation “in the moment”, but also of the capacity to develop an accurate, reliable mental representation (for potential wayfinding). Herein lies an opportunity to introduce a design concept that allows for navigation activity in GPS-type aids, while addressing the cognitive effort associated with traditional, static map displays to better support wayfinding.

1.2 SkyMap: A New Display Paradigm

To address some of the cognitive expenditures associated with the use of traditional and current map displays, designers at Uncharted Software, Inc.1 have proposed a novel display paradigm, named SkyMap, for use with augmented reality2 (AR) technologies. SkyMap proposes a conventional bird’s-eye view map of an

1https://uncharted.software/ 2To be explicit with the terminology, and consistent with Milgram and Kishino (1994), SkyMap was intended by its creators to be realized as a virtual image, displayed digitally and superimposed on a direct view of an environment. Chapter 1. Introduction and Motivation 3 environment displayed above the observer in the environment, with a scale and orientation that places the map elements appear above (but not necessarily directly above) their real-world counterparts (Kapler, King, & Segura, 2019). The resulting display is intended to evoke the illusion of a larger-than-life mirror, reflecting the layout of the terrain below. There are three primary benefits proposed for SkyMap. The first is the displaying of the map in the “upper field of view”, henceforth defined as the area in the field of view above environmental features that are in front of the observer, often an information-barren area. (This is intended to correspond to the visual space commonly occupied by sky in outdoor environments, hence the name.) This allows for reduction of the information access effort associated with conventional heads-down displays, i.e., interfaces located in front of the observer below the forward field of view (FFoV) (Wickens, Hollands, Banbury, & Parasuraman, 2015; Smith, Gabbard, Burnett, & Doutcheva, 2017). As a corollary, the immediate space around the observer (and the information within it) remains unoccluded, avoiding the visual clutter introduced by other AR-based navigation aids that superimpose graphics on environmental features (Wickens et al., 2015). The second proposed benefit arises from the large scale implied by SkyMap, which can support visual discernibility of map elements and affords a higher level of detail for information representation. The third proposed benefit comes with the consistent alignment between the display and the real environment, as with other track-up maps, which alleviates the observer’s need to orient the map display (physically and/or mentally) with their view of the environment. The combination of these benefits should improve users’ wayfinding performance by offering enhanced map readability, thereby reducing cognitive effort in map reading and potentially supporting more accurate spatial inferences. At the time of this writing, four separate studies (excluding the present one) have been carried out to investigate whether the aforementioned benefits of SkyMap appear to be realizable. All of those studies have used an implementation of SkyMap in virtual reality (VR) and have involved various navigation/spatial un- derstanding tasks. Their results address SkyMap’s effectiveness in facilitating route following and supporting decision-making related to route selection. While those studies have provided valuable insight into SkyMap’s utility, they do not directly address the more fundamental questions of whether the benefits described above are realized. This is because their experimental tasks involved other aspects of wayfinding, such as locomo- tion, spatial memory, and comprehension of complex distance relationships. The research space of which subtasks of wayfinding/navigation are supported has yet to be fully explored. In this thesis, I investigate SkyMap’s effectiveness relative to a conventional, heads-down display for an experimental task that assesses, in isolation, the integration of direct and indirect views of the same space. I outline a design space to encompass all implementation decisions that fall under the SkyMap concept, with a focus on dome-shaped implementations. Well-established methods from the field of cartography are drawn upon for the selection of SkyMap projection approaches worth assessing. I describe the design of a spatial visualization experiment that compares these map conditions, and discuss the implications of my results for the design of a SkyMap that better improves spatial understanding, and by extension, better supports wayfinding. Chapter 2

Background & Problem Scoping

In order to corroborate SkyMap’s claims of improved wayfinding support, the design details of its imple- mentation should be optimized to highlight the benefits proposed above. In this chapter, I present a set of relevant variables that define the research space introduced by the SkyMap concept (derived from the broader design space for maps in general), and outline the process of scoping this space down to what can be reasonably and most promisingly investigated within one experiment.

2.1 Defining the Research Space

I present the basic characteristics that comprise a research space for SkyMap optimality: map scale (com- prising both size and shape, as elaborated below), level of detail, and visualization of self-location. Fundamental literature in cartography outlines some important considerations for map design (Jones, 2010), which fall into the following three general categories:

1) Map scale, i.e., how much of a geographical area to include in a given space;

2) Map projection, or shape – although the relevance of this depends on the size of the visualized geo- graphical area, map designers are reminded that the Earth’s curvature necessitates some adaptation for flat maps (Slocum, McMaster, Kessler, & Howard, 1999) (this is discussed further in Section 2.3); and

3) Map details, encompassing which environmental features are visualized and how, as well as the level of generalization, or how information is presented in aggregate to suitably fit within the visual space of the map.

The discussion of map scale, the first category above, becomes more involved with SkyMap, as the constraints of a flat surface (paper, screen, etc.) are removed in a mixed reality setting. Thus, the shape of the map itself must be decided (as described in the next section), as well as where it is displayed in an observer’s FFoV (though the SkyMap concept holds this fixed, to a degree). The second category, map projection choice, is not of major impact for maps at the scale range of interest for SkyMap’s potential applications (i.e., for street-level navigation).1 Though the majority of detailed map design decisions relate

1It should be clarified that the projections of interest discussed later in this thesis are not the same as the projection choice specified here, which here places the information from the Earth’s curved surface onto a flat surface.

4 Chapter 2. Background & Problem Scoping 5 to the third category, and are thus given more attention in discussions of map design in the literature (Tufte, 1983)2, I omit further consideration of these details, as they are neither unique nor novel to the SkyMap concept. An essential element of maps used specifically for navigation within an environment is the indication of the navigator’s location relative to the environment, or a “you are here” marker, henceforth referred to as a self-marker (McKenzie & Klippel, 2016). While not absolutely necessary, the inclusion of this one feature has been found to significantly reduce the cognitive effort associated with map-aided navigation, more specifically the sub-task of relating the information in a map to that in the navigator’s direct view (Levine, 1982; Montello, 2010). As mentioned in Chapter 1, the indication of self-location was a pivotal feature of GPS technology. To summarize, the design of any map involves the choice of several variables about what information is displayed and how. I argue that the consideration of SkyMap, specifically, warrants investigation of three basic decisions: map shape/size, map scale, and method of displaying a self-marker. Since the first decision is the one in which SkyMap’s novelty is most implicated, the following sections elaborate on how it can be investigated methodically.

2.2 Investigating a Dome-shaped SkyMap

As a computer-rendered map display, the lack of physical constraints on a SkyMap implementation makes for a vast design space. SkyMap may take on any shape that can be defined mathematically or computationally, not even necessitating a physical counterpart (or any compliance at all to the laws of optical physics). As such, herein lies the most “interesting” implementation issue brought about by this display concept. A natural starting place to address this design issue is to consider the paradigm of a flat map, made familiar by the dominance of paper- and screen-displayed maps. The SkyMap inventors, themselves, evoke the idea of a flat, horizontal SkyMap by calling to mind the idea of a giant mirror in the sky (Kapler et al., 2019)3. However, a problem that becomes evident through prototyping is the consequence of linear perspective: a flat, horizontal SkyMap display diminishes in the visual field as quickly as does the direct view of the environment, following an inverse square law. At the same time, having the map “horizontal” (see figure in Table 2.1, first row) allows for the observer to see the sky in the space underneath the plane of the map, undermining the efficiency of SkyMap’s use of the upper field of view for displaying information. These shortcomings can be addressed with tilting the flat surface (Table 2.1, second row), however other drawbacks arise as a result. Attempts to balance benefits and detriments bring about consideration of other physical shapes for SkyMap. Table 2.1 describes a few of these basic shapes in detail and describes the advantages and disad- vantages of each. This set is by no means exhaustive, but rather stems from the logical process of identifying drawbacks of one shape and addressing them comparatively together with other basic shapes. It is essential to note that, once we depart from shapes that have corresponding physical manipulations (ex. scaling, rotation) of a flat map, the question arises of how a flat map image might and should “map

2This design subspace is addressed in depth in cartographic literature; Park and Park (2019) provide a review of map details specific to the heads-up format. 3It can be argued that mirrors are commonly thought of as flat, considering the prevalence of flat vs. non-flat mirrors in daily life. Chapter 2. Background & Problem Scoping 6

Sketch of SkyMap Shape Advantages Disadvantages

Flat, horizontal map The primary benefit of this With the self-marker directly shape is the planar align- above the observer, it is out ment, i.e., map features ap- of the FFoV. pearing to lie directly above With a flat image the visual their real-life counterparts. angle diminishes quickly, and This would be a true simula- the sky space is not maxi- tion of a flat, horizontal mir- mally utilized. ror in the sky.

Flat, tilted map Improves over the horizontal Introduction of a tilt disrupts map in that the visual angle the planar alignment, to a of the map image diminishes greater extent with increas- more slowly. ing distance from the tilt With appropriate selection of axis. the tilt axis, the self-marker may be brought down into FFoV, and more sky space can be covered.

Dome-shaped map Consumes full field of view, Applying a flat map image on maximizing use of sky space. to a dome necessarily intro- A spherical dome, with the duces distortion, discussed observer at the centre, has further in Section 2.3. all points along the surface at the same distance from them, meaning the displayed map image has a constant visual angle size at all points.

Table 2.1: Initial SkyMap shape considerations Chapter 2. Background & Problem Scoping 7

onto” these shapes, as is the case for the dome-shaped SkyMap proposed in Table 2.1 (bottom row). Thus, selecting the dome-shaped SkyMap for further investigation, with its advantages presented above, also creates a need to further consider the “mapping”, henceforth referred to as the projection, of the flat map onto the dome surface. To be explicit, the term “projection” is defined in this thesis as a transformation, or set of transformations, which specify where each point on an original image (the flat map, unless otherwise specified) will be presented on a given shape, the projected surface. A projection may or may not be “true”, which is the case when projection rays diverge from a single point, line or surface, passing through all points in both the original image and projected surface (Snyder, 1987). It is equally plausible to consider the projection paradigm for all surfaces (those in Table 2.1 and beyond); however in the case of flat SkyMap surfaces it is more straightforward to think of them as a manipulation of the traditional flat map. Certainly one could consider projections onto a flat SkyMap surface that involve non-uniform manipulations (such as a fisheye effect, for example), but those are not considered further due to having the same drawbacks mentioned for flat, horizontal maps in Table 2.1 (top row), while also diminishing planar alignment. In fact, even with constraining further investigation to a dome-shaped projection surface, the design space is still quite vast, adding only the restriction that the SkyMap occupy the full upper field of view (i.e., occluding the sky in all directions). This vast space is explored in Section 2.3.

2.2.1 Task-optimized distorted displays

An easy argument to make against a dome-shaped SkyMap would be that it is an unfamiliar shape for a map to take on, and that it disrupts the correspondence between a flat image and an (often) approximately flat terrain. However, there is the precedent of “distortion-oriented displays”; Leung and Apperley (1994) provide a comprehensive review and taxonomy of its implementation approaches. These have been used to visualize physical spaces and distributed networks with non-uniform magnification, yielding performance improvements with certain tasks (Ragbar, 2017; Martin-Gonzalez, Heining, & Navab, 2009; Hollands, Carey, Matthews, & McCann, 1989). Potential factors contributing to the observed benefits include the more even distribution of important details that might otherwise be too small or cluttered in a spatially congruent display, and a reduction of effort required for an operator to locate and perceive information in the area(s) of greatest importance. This reasoning is also familiar in information visualization literature, where the principle is known as “focus plus context” (Card, Mackinlay, & Shneiderman, 1999). Chapter 2. Background & Problem Scoping 8

2.3 Traditional Cartographic Projections

It would not make sense to address the question, “How do we visualize a flat map on a domed surface?” without acknowledging that there is an immense body of research that has addressed the inverse problem, brought forward by our approximately spherical Earth: “How do we visualize spherical geographic informa- tion on a flat surface?” The answer to the latter is the concern of the field of cartography. I will provide an overview of cartographic projections in this section, with due attention given to the more relevant subtopics. A complete description of the most common traditional cartographic projections can be found in the manual by Snyder (1987). The concept of cartographic projections is as old as some of the most foundational ideas in cartography, with the earliest known published projection maps dating back to 150 C.E. (Berggren, Jones, & Ptolemy, 2000). Many of today’s well-known projections were developed during the 16th through 19th centuries, with several modified projections from the last century (Snyder, 1987). Projections are rendered necessary by the mathematical truth (a consequence of Gauss’ Theorema Egregium, published in 1828) that information on a spherical surface cannot be visualized on a flat surface without distortion.4

2.3.1 Classification of projections

There are three types of common developable surfaces, or surfaces onto which a sphere can be projected to produce a flat image: a cylinder, a cone, and a plane (Figure 2.1).5,6 Cylindrical, conical and planar (or azimuthal) projections each provide their own benefits and suitable applications in cartography, and maps of each projection type are still relevant today in various fields (Snyder, 1987). Projections that fall outside of these three categories also exist, and are in use (ex. pseudocylindrical, sinusoidal), but are not considered here.

Figure 2.1: Three common developable map surfaces (top row), and their corresponding projection types (bottom row): (from left) cylindrical, conic and azimuthal (Credit for source figures: c Penn State Univer- sity, is licensed under CC BY-NC-SA 4.0. Both figures adapted from original.)

4The rigorous mathematical expression of this can be found in cartographic literature that addresses differential geometry, such as Grafarend and Krumm (2006). 5In fact, the cylinder and plane are extreme cases of the cone: treating the “height” of the cone as the distance from the centre of the circular base to the point, a cylinder can be considered a cone of infinite height, and the plane seen as a cone of zero height. 6Other, more complex developable surfaces do exist, however the math associated with “unrolling” them onto a plane is less straightforward, and thus they are not considered here. Chapter 2. Background & Problem Scoping 9

Within each of the three categories, the projection must be further specified, defining the point/great circle(s) that lie tangent to the sphere, and (for true projections) the location and nature of the source of the projection rays. The culmination of these design decisions results in map projections which may preserve some properties of the spatial information (e.g., area, shape, distance relationships) at the expense of others. These properties are identified and exemplified for azimuthal projections in Table 2.2. While research in cartography and of geographic information systems have done the work of identifying the projections/map characteristics that are best suited for certain applications and map use tasks (Robinson & Snyder, 1997), the novelty of the map shape and relative observer location introduced by SkyMap renders it difficult to directly apply those findings. Though hypotheses can be made analytically, an experimental investigation to determine the optimal map projection(s) for SkyMap is appropriate.

2.3.2 Azimuthal cartographic projections

For the consideration of a SkyMap implementation that utilizes an inverse transformation of traditional cartographic projections (projecting a flat image onto a hemisphere), I argue that polar azimuthal projections (Figure 2.1, right) are the most sensible scoping choice. This is because of an important property possessed by this set of projections: they preserve directionality. This is visualized in Figure 2.1 (right), and best articulated in practical terms by Snyder (1987, pg. 141):

Azimuthal projections are characterized by the fact that the direction, or azimuth, from the center of the projection to every other point on the map is shown correctly. In addition, on the spherical forms, all great circles passing through the center of the projection are shown as straight lines. Therefore, the shortest route from this center to any other point is shown as a straight line.

Distinct from the conical and cylindrical forms, azimuthal projections allow for the projection-related distor- tion to be a function of only one of the two polar coordinates: in the radial direction, away from the centre. Thus, the distortion of the map display is uniform in all directions at any particular radial distance. By scoping to polar azimuthal projections, all the potential shape parameters simplify elegantly into one vari- able: the spacing between parallels of latitude, as a function of distance from the centre. Example spacings for azimuthal projections are illustrated in Figure 2.2.

Figure 2.2: Illustration of lines of longitude and circles of latitude for different azimuthal projections: (from left) orthographic (spacing between parallels of latitude decreases towards the equator), stereographic (spac- ing increases towards the equator), and equidistant (spacing is consistent). Chapter 2. Background & Problem Scoping 10

Since the preservation of different spatial properties can be achieved with any of the three developable surfaces identified above, this choice of projection type does not limit the experimental scope with respect to spatial property preservation. Considering that SkyMap is intended for wayfinding, and to be easily integrated with an observer’s direct view of the environment which has them at the centre, the set of polar azimuthal projections serve as the best candidate for further investigation. Table 2.2 illustrates the five principal polar azimuthal projections as outlined by Snyder (1987), which con- sist of both true and mathematically-defined projections that retain specific spatial properties, as described below. Further details on the mathematical construction of these projections can be found in Grafarend and Krumm (2006).

Projection Projection Rays Circles of Spatial Property Latitude Preserved

Alignment on the horizontal plane: every point (x, y, z) on the spher- Orthographic ical surface has its corresponding point on the plane at (x, y).

Conformality, i.e., angle preser- vation: an angle measured on Conformal the sphere surface would have the (Stereographic) same value when measured on the projection surface. As a result, the shape of features appears the same.

Arcs on the sphere surface are Gnomonic drawn as straight lines on the pro- jection surface.

Not a true Equal-area Preservation of (relative) areas projection

Preservation of (relative) distance Not a true Equidistant relationships from the centre to all projection points on the projection surface

Table 2.2: The five principal azimuthal projections and the spatial properties they preserve. (Image source: WikiMedia, licensed under CC BY-SA 3.0. All figures cropped from original.) Chapter 2. Background & Problem Scoping 11

2.4 Review of Literature Addressing Spatial Cognition and Map Use

In order to determine whether the benefits proposed by SkyMap can be realized, I identify a fundamental cognitive process involved in map-aided wayfinding activity for which SkyMap may offer better cognitive support, and by extension improve wayfinding performance. Furthermore, the selected process should be one for which we can anticipate a meaningful influence of the particular dome projection, the SkyMap design parameter of focus. An experimental task can then be designed to necessitate this process. Though the field of human spatial cognition has been researched for over half a century (Montello, Fabrikant, & Davies, 2018), the implications for map design have not been addressed to the same extent. One of the primary topics in this area addresses individual differences in wayfinding ability (e.g., Wolbers & Hegarty, 2010), and in particular the effect of sex and socio-cultural factors (e.g., Lloyd & Bunch, 2008). In the field of neuroscience, spatial cognition has been further specified for the purpose of identifying specific brain areas and functions/dysfunctions (e.g., Tr´es& Brucki, 2014; Aguirre & D’Esposito, 1999); however, with few recent exceptions (Conson et al., 2018; Qiu et al., 2019), that work is related more to internalized spatial representations than to map reading. The literature associated with cognition’s involvement specific to map use is dispersed throughout several different academic fields, often with research aims only tangential to the ones directly investigated here. Studies in cognitive science have acknowledged map interpretation as part of the greater context of map- aided navigation, but have tended to focus on high-level decision-making (e.g., Golledge, 2003) or navigation strategies (e.g., Zhong & Kozhevnikov, 2016). Other studies in this field, as well as in human factors, have investigated the influence of map displays on the development of internal spatial representations (e.g., Thorndyke & Hayes-Roth, 1982; Williams & Wickens, 1991). In recent decades, research in spatial cognition and computer science has more directly investigated the influence of geovisual interface elements on spatial understanding. Two likely contributing factors to this are Slocum et al.’s proposed research agenda presented to the International Cartographic Association (2001), which directly named topics in this area, and the advancement and proliferation of geographic information systems (GIS) software. Montello et al. (2018) provide an overview of notable recent work. Though map- reading task analyses have been published, they have focused more on the use of GIS software interfaces (e.g., Lloyd & Bunch, 2003; Tahir, McArdle, & Bertolotto, 2012), or have been formulated for the purposes of education in the geosciences (e.g., Kastens & Ishikawa, 2006; Liben & Titus, 2012), rather than within the context of using a map to navigate the environment. Research in the field of human factors, meanwhile, has tended to focus on design decisions related to specific interfaces (particularly for mixed reality, e.g., Allen, Mcdonald, & Singer, 1997). It has also acknowledged the map-reading cognitive process of mental rotation by investigating the effect of map ori- entation/alignment on wayfinding performance (e.g., Aretz & Wickens, 1992; McKenzie & Klippel, 2016); however that particular design element is not of interest in this thesis, as the rotational alignment of the map is treated as a given feature of SkyMap. In summary, while there is a breadth of literature on human spatial cognition and how it may inform the design of map interfaces, few publications have defined a set of individual cognitive processes associated with relating a map display to its environment in a navigation context. Chapter 2. Background & Problem Scoping 12

2.4.1 Cognitive processes associated with map-aided wayfinding

One of the papers that endeavours to define map-related cognitive processes is by Lobben (2004). She pro- vides a thorough overview of studies in this space, and how the focus of the research has evolved over the past several decades. Furthermore, she offers a set of subtasks within map-aided navigation and identifies their associated cognitive processes. Among the processes identified, two deal directly with the map/environment interaction, which is the subspace most relevant to SkyMap’s proposed benefits: self-location and visualiza- tion. Self-location, introduced previously in Section 2.1, involves the observer relating environmental cues to their corresponding map representation(s) in order to determine their location on the map. Visualization is defined by Lobben as the process of comparing information encoded from the map display with visual stimuli directly perceived in the environment, and mentally “placing” information from one view onto the other. This general definition of visualization would, I argue, make the process of self-location a specific instance of visualization.7 Based on the research aims specified thus far, the selected experimental task should require some sort of visualization process: the “placement” of information from one view onto the other. In accordance with the cognitive processes deemed directly relevant, I devised an experimental task which involves identifying the location of an element seen in the environment in terms of its location on the associated map display. Not only does this task require the “placement” of information from one view onto the other, which is the visualization process described above, but such a situation is relevant to the benefits SkyMap proposes to real applications. To illustrate, imagine a soldier using SkyMap to navigate. One of their duties might be to locate any threats they see in the environment, and share this information with offsite decision-makers to ensure safe and successful operations. This task necessarily requires the soldier to “place” the threat they see in front of them onto a map display in order to communicate its location.8 If SkyMap indeed improves the spatial fidelity associated with visualization processes (relative to what can be achieved with traditional map displays), then it provides the real benefit of improving the spatial understanding shared with all communicants. The specific implementation of the visualization task is addressed in Section 3.1.1, where justification is provided for more detailed design decisions.

7Lobben distinguishes the two processes based on their directionality (i.e., that self-location is an “environment-to-map” process, whereas visualization is a “map-to-environment” process); however, I group them together under a non-directional definition of visualization with the argument that the navigation settings involved with SkyMap do not necessarily dictate whether information of interest will be mapped or viewed directly. 8Theoretically, the same goal could be achieved with the soldier reporting the location with respect to their direct view of the environment; however, this information then has to be processed by other parties to make it of use to anyone who may not share this view. Chapter 2. Background & Problem Scoping 13

2.5 Research Questions

The scoping of the overall research space brought forth by the introduction of SkyMap, and the identification of a task that stands to benefit most by SkyMap’s offerings, has led to the following two research questions:

Research Question #1: How effective is SkyMap in helping one to estimate the location of an element in their surroundings, relative to traditional map displays? Research Question #2: How does the type of projection on the dome affect one’s capacity to estimate the location of an element in their surroundings with respect to the SkyMap display?

The overall hypothesis argued in this chapter is that the improved integration of an observer’s direct and indirect views will result in a more accurate spatial understanding of their surroundings, specifically the spatial relationships between features therein. Addressing Research Question #1, this leads to the hypothesis that SkyMap will be more effective than traditional map displays at helping an observer estimate the location of an element in their surroundings. I predicted that this would hold true despite the legacy of traditional map displays, and the familiarity it brings. The second research question, however, should be regarded as more exploratory in nature. As such, I refrained from making a making a hypothesis about how performance would change as a function of the different projection methods to be tested. This is due to the novelty of SkyMap and of the application of task-optimized distorted displays to spatial information at this scale. Furthermore, as highlighted in Table 2.2, each candidate projection preserves some spatial property of the map information, and the relevance of each property may depend on the strategy employed for the experimental task (of which there may be many). Nonetheless, one speculation that can be made is that the conformal projection may have a performance advantage over other projection types, as preservation of local shape may evoke familiarity with respect to the original map image. The two research questions are revisited at the end of the next chapter, after the experimental task is de- fined, at which point further speculation can be made, and the hypotheses expressed in terms of experimental variables. Chapter 3

Experimental Method

For this experiment, the type of map projection used on a dome-shaped SkyMap was investigated for its effect on performance of a spatial visualization task in comparison with performance using a traditional flat, track-up map, as well as a flat SkyMap. This chapter describes the major experimental design decisions and their underlying justification, the implementation of the task environment in virtual reality, and the experimental procedure.

3.1 Experimental Design

3.1.1 The experimental task: the three-tower, two-marker problem

The experimental task had subjects located in an urban environment, with three towers in their field of view. Two of the three towers had their locations indicated on a map display, and the subject was tasked with identifying the location of the third tower on this map display. This is depicted in Figure 3.1. A single response for this task was (correctly) anticipated to be produced on the scale of tens of seconds, so the

Figure 3.1: Screenshot of example experimental task with the flat SkyMap condition. The subject was tasked with marking the location of the green tower on the map using the pointer (magenta).

14 Chapter 3. Experimental Method 15

experiment was designed to consist of batteries of several trials in a row (further details are explained in Section 3.2.2). The implementation details of the trials were carefully chosen to control the availability of visual cues that subjects could use to complete the task. The most important visual cue included (as indicated by the title of this section) was the two additional towers, with their locations marked on the map. This was done to provide unambiguous information that related the appearance of spatial elements in the environment to those on the map. Though it may seem that the urban environment and its map should have sufficiently provided this information, the uniformity of the space (in colours and features) combined with the expanded viewing range afforded by the map (relative to the limited view of the buildings, due to occlusion) made information integration with these cues alone difficult. This observation also manifested in pilot studies. A thought experiment that illustrates the necessity of these additional towers for the SkyMap dome projections is one I call the “fun-house mirror” problem. A fun-house mirror, typically found at fairs and carnivals, distorts one’s image (often the face or body of the observer as they stand in front of it) for enter- tainment. Because a person is familiar with their own appear- ance and proportions, the existence of distortion and the types of transformations being made to their image are apparent. This is depicted in Figure 3.2. If, however, we imagine that an abstract sculpture were placed in front of such a mirror, there would not be enough information for someone to gain knowledge about Figure 3.2: Photograph of fun-house mir- the shape of the original sculpture based on the reflected image ror, with a boy observing a distorted im- alone, without viewing the sculpture directly. I argue that the age of his body. (Source: WikiMedia, li- same information deficiency exists in this experimental task for censed under CC BY-SA 3.0.) a dome-shaped SkyMap: the subject has available to them a map representation of an environment, much broader than the space they can view directly, but it is distorted because of the shape of the display sur- face. Because the subject has no previous knowledge of the environment (they are screened not to), and the dome introduces image distortions that are not commonly encountered elsewhere, they do not have enough information to gain knowledge about the true spatial configuration of the space beyond the limited region they can view directly. To address this potential deficiency of information, two reference points were provided in the form of two towers with their locations marked on the map, coloured brightly to eliminate any ambiguity.1 The intention was that subjects would be able to use the information provided by these towers to gain insight into how spatial relationships on the approximately flat terrain around them would appear on the curved dome surface. For this experiment’s fully within-subjects design, it was presumed that adding the two towers would provide the added benefit of mitigating transfer of inferred spatial understanding between one map condition and the next. As mentioned in Section 2.4.1, this task simulated the realistic scenario of indicating a feature or event on a map that is viewed in the environment. To this end, the additional towers simulated mapped landmarks that an observer might reference in order to communicate the location of something they see. By including brightly coloured towers that were taller than all other buildings (which were much more difficult to distinguish), the

1Tower colours used were selected to be distinct even for those with colour vision deficiency (Harrower & Brewer, 2003). Chapter 3. Experimental Method 16 potential for landmark referencing as a strategy was controlled and consistent across all trials. In order to confirm that the designed task was of adequate difficulty and to finalize details about the pro- tocol, pilot testing was conducted on three graduate student colleagues from the human factors department. They completed 2-6 trial blocks while thinking aloud and provided additional feedback after VR immersion. Their comments indicated that the two additional towers were necessary for performing the task. While the task presented to the paid participants was almost identical to the one presented to pilot testers, the latter informed details about the experiment timing and questionnaire procedures (described in subsequent sections). In addition to the two tower-marker pairs, the other elements of the experimental environment were designed to control cue availability. The design decisions, which apply to all map conditions, are outlined below. Location-specific visual cues: To preserve some ecological validity in the task, an existing urban environment was selected. Consistent with other studies in this SkyMap project (Reiner, Hollands, Jamieson, & Boustila, 2019), a region in South London, United Kingdom was selected for trial locations. It provided the benefit of being populated with several buildings similar in height and appearance, limiting the field of view and providing a complex navigation environment. Uniformity in the urban environment was desired, based on the assumption that existence of distinctive environmental features would lend themselves to providing anchors, and potentially support learning of the space across trials. This would, however, undermine the aims of this experiment: to understand the influence of the map projection on immediate spatial understanding, i.e., readability. Furthermore, an urban environment was sought that did not have a rectilinear street configuration, which would have also allowed spatial relations to be estimated without the use of the map (and thus, without performing the cognitive task of spatial visualization). Map design-based visual cues: As mentioned in Section 2.1, the design of the features on the map was not of interest in this particular investigation, so a simple map design was chosen that only distinguishes the same types of terrain that are also perceivable in the urban environment (i.e., roads, buildings, green spaces). The colour palette was the default one from the Mapbox “Streets” style2, similar to widely used maps available online (such as Google Maps) and on paper. Map shape-based visual cues: Though the initial intention for the experimental task was for subjects to rely solely on the two tower-marker pairs for information about the distortions between the environment and the map display, pilot testing uncovered that this was still insufficient in some cases. To address this, a faint square grid pattern was added to the projected maps (see figures in Table 3.1), providing further information about the projection shape. Because of the irregularity of the street layout, this information was not redundant, and was not found to make the task trivial.

2Information about Mapbox Streets: https://www.mapbox.com/maps/streets/. A preview of map style used in actual experiment, based on Streets v7, is linked here. Chapter 3. Experimental Method 17

3.1.2 Map conditions

As indicated by the research questions (Section 2.5), I aimed to compare SkyMap to traditional, heads-down maps, and analyze how different polar azimuthal dome projections would affect performance on the task. To these ends, a total of six map conditions were included in the experiment, as described below. See Table 3.1 for screenshots of all conditions. Heads-down, track-up map: This condition served as the control condition, from which performance of the SkyMap conditions would be compared. A track-up map was chosen, which remained positioned directly in front of the observer and rotated about the self-marker to maintain alignment with the observer’s view of the environment. This was chosen over a North-up map because all SkyMap conditions also remained in alignment, so requiring participants to perform mental rotation with the heads-down map would not allow for a fair comparison of the map characteristics of interest. The map was tilted with the top edge further away from the observer, 40◦ from upright. Flat SkyMap: Despite the disadvantages outlined in Table 2.1 above, the inclusion of a flat SkyMap condition allowed for assessing, in isolation, the specific benefits the SkyMap proposed by being large- scale and located in the upper field of view of the observer, without the potential confounds introduced by applying the curvature of dome projection geometry. If the flat SkyMap condition were to outperform its domed counterparts, this would provide more insight into the results of earlier experiments (i.e., Reiner et al., 2019), which utilized an offset dome projection as its only SkyMap condition. Azimuthal projection domes (orthographic, conformal and equidistant): Of the five polar azimuthal projections described in Table 2.2, the orthographic, conformal (stereographic) and equidistant projections were selected as experimental conditions. The gnomonic projection was omitted as it is effectively equivalent to a flat SkyMap (see Figure A.1, Appendix A); the equal-area projection was found in pilot testing to be nearly indistinguishable from the conformal projection, lacking justification for the additional time and effort that would be required to include it. Offset SkyMap: In all of the other SkyMap conditions, the self-marker was located at the top of the dome, in the direct centre of the SkyMap, directly above the observer. This offset condition, designed by developers at Uncharted Software, shifts the self-marker in the observer’s forward direction so that it is visible at the top of their field of view when looking directly forward. However, this change introduces direction-dependence of the map projection, which results in a map whose visual distortion changes with the observer’s head rotation. This had been reported in earlier experiments to have a disorienting effect, which was confirmed in pilot testing of this experiment. This condition was nevertheless included in the experiment, as a means of evaluating whether these drawbacks might be outweighed by the advantage of being able to locate oneself on the SkyMap without the effort of having to look straight upwards. Table 3.1 shows a screenshot of each of the map conditions as they appear at the beginning of all trial blocks (as explained in Section 3.3.2). Implementation of the conditions is discussed further in Section 3.2.1 below, and in Appendix A. Chapter 3. Experimental Method 18

Heads-Down Map Flat SkyMap Offset Dome (head elevation: 16◦) (head elevation: 25◦) (head elevation: 15◦)

Orthographic Dome Conformal Dome Equidistant Dome (head elevation: 35◦) (head elevation: 25◦) (head elevation: 30◦)

Table 3.1: Screenshots of all map conditions, as they appear at the beginning of each block. Head elevation, i.e., the angle at which the head was tilted upwards from the forward direction at the time the screenshot was taken, is as indicated.

3.1.3 Measurement of potentially mediating factors

In an effort to ensure that what was being tested was actually the influence of map condition on performance, I identified other factors related to individual differences that could be tested and potentially accounted for during analysis, with regards to the possibility of their masking the experimental effects in the fully within- subjects design. Spatial ability: There is extensive literature attempting to identify the constructs that fall under the umbrella of spatial ability, and how they would be involved in specific subtasks of wayfinding and navigation (Linn & Petersen, 1985; Wolbers & Hegarty, 2010). Ekstrom, French, and Harman (1979) identified the following three abilities: spatial visualization, spatial orientation and spatial memory. Spatial visualization, the skill corresponding to the task of the same name described in Section 2.4.1 above, was certainly expected to correlate with task performance. Spatial orientation, defined by Ekstrom et al. (1979) as “the ability to Chapter 3. Experimental Method 19 perceive positions and configurations of objects in space”, can be measured by mental rotation tests that correlate with a variety of spatial and navigation tasks (Maeda & Yoon, 2013; Hegarty, Montello, Richardson, Ishikawa, & Lovelace, 2006). Testing of the third category, spatial memory, was deemed less relevant to the present context, as the heads-up nature of the SkyMap display is intended to reduce the need to retain spatial information between looking at, and away from, the map. Further motivation to test for both spatial visualization and mental rotation abilities was introduced by results from Long, Gomer, Wong, and Pagano (2011), who compared the two abilities with respect to performance at teleoperation tasks involving bird’s-eye and direct line of sight views. They found, for the direct line of sight condition, that task performance was related to both constructs tested, but for the bird’s eye view condition, only the mental rotation scores were related. Furthermore, Pak, Rogers, and Fisk (2006) conducted a study involving the same two spatial abilities and found correlation with a search task that was conducted either through a text-based hierarchy or network map. They found that, while mental rotation was a predictor of task performance in both conditions, spatial visualization was a significant predictor only for the map-based conditions. Thus, it was decided that the testing of these two categories of spatial ability might yield insights into the nature of the cognitive activity involved with using SkyMap. Action video game experience: Several studies have shown that action video game experience can improve performance at spatial tasks (Bediou et al. (2018) provide a recent meta-analysis of this work), specifically when the gaming has been frequent within the past six months (Green & Bavelier, 2007). Ad- ditionally, since this experiment involved hardware and software designed for video gaming, familiarity with the input devices and with analogous interfaces may have potentially offered another advantage. To that end, participants were also asked to report general video game experience.3 VR experience: Participants were asked about the extent of their prior use of VR, because having VR experience was found to mediate the effect of SkyMap use on navigation-related tasks in a previous experiment with a similar setup (Reiner et al., 2019). It is also reasonable to hypothesize that those not distracted by the novelty of VR headset use may be able to better focus on the task, potentially influencing performance. Though longitudinal studies focusing on VR use over time are sparse, another SkyMap study is planned to investigate navigation performance with repeated uses of the VR-based SkyMap, potentially substantiating this hypothesis.3 Stereoacuity: Since the VR headset used in this experiment displayed stereoscopic imagery, a measure of stereoacuity was used to measure participants’ ability to combine the left- and right-eye images into a single 3-dimensional percept. However, that literature has not confirmed a correlation between stereoacuity and performance at spatial tasks in virtual environments at this scale (O’Connor & Tidbury, 2018; Tidbury, Brooks, O’Connor, & Wuerger, 2016), so this data was not included in the analysis. Other questionnaires administered were the Santa Barbara Sense of Direction scale, a measure of self- reported spatial abilities related to navigation (Hegarty, Richardson, Montello, Lovelace, & Subbiah, 2002), and the Navigation Strategy Questionnaire, which was designed to measure propensity to navigate using a map-like mental representation (Zhong & Kozhevnikov, 2016). While these measures have not been explicitly reported to relate directly to spatial visualization ability, they may provide insight into individual differences in performance at the experimental task. Since they were also used in the other SkyMap studies, including them here allows for future meta-analysis.

3See Appendix B for detailed questions. Chapter 3. Experimental Method 20

3.1.4 Non-performance measures

Aside from metrics of accuracy and speed for each trial (specified in Table 3.3 below), data was also taken on the participants’ head elevation (i.e., up/downward tilt angle, using the VR headset’s tracking capabilities), their confidence in each trial response on a six-point scale, and their subjective workload relative to each trial block measured with the NASA-TLX (Hart & Staveland, 1988). At the end of the experiment, participants also took the Simulator Sickness Questionnaire (Kennedy, Lane, Berbaum, & Lilienthal, 1993). This data was measured to provide supplemental insights about task performance, and potentially help to account for anomalous data.

3.2 VR Implementation of Experimental Environment

3.2.1 Software

Virtual urban environment & map display

All the data from the experimental task was collected in a VR environment developed using the Unity 3D game engine4 (version 2018.3.7f1) and its C# scripting application programming interface (API). The original Unity code, from which this experiment setup was derived, was developed by post-doctoral researchers Alexis Morris and Sabah Boustila, in collaboration with developers at Uncharted Software. It was developed in Unity version 2017.3.0f3. The virtual environment consisted of two main elements: a city and a map. The life-size 3D model of a city (South London, UK, as explained in Section 3.1.1 above) was provided by the WRLD5 API for Unity (version 0.7.1019) and added terrain, road, and building objects to the environment. The map used for all map conditions was added using the Mapbox SDK for Unity6 (version 2.0.0). To generate all map conditions, the Mapbox map was rendered with the same scale and alignment as the WRLD city, and its image was captured by an overhead virtual orthographic camera to generate a render texture (see Figure 3.3b). This texture was then applied to different flat and dome-shaped meshes, as per the map conditions. The details of how the render texture was applied to the dome-shaped mesh to generate different projections is described in Appendix A. To add to the sense of immersion (and to mask background noise from the experiment room and neigh- bouring offices), a soundtrack of ambient city sounds7 was played through the computer speakers at a low but audible volume while the trials were being conducted.

Implementation of map conditions

The application of the map’s render texture to a flat mesh (for the heads-down and flat SkyMap conditions) was straightforward, as the render texture was an image of a flat map, which only needed to be scaled and reflected to correctly display the desired image on a corresponding flat mesh. Note the flat maps, like all

4https://unity3d.com/unity 5https://www.wrld3d.com/3d-maps/ 6https://www.mapbox.com/unity/ 7[Ambience Hub]. (2014, November 28). City Sounds - Ambience - 1 Hour - As real as it gets [Video File]. Retrieved from https://youtu.be/ElU8g7xi6ws Chapter 3. Experimental Method 21

objects in the environment, were viewed from a linear perspective; that is, the areas of the map that were farther away from the viewer appeared smaller. The flat map was chosen to be circular in shape to maintain consistency in map coverage with the dome-shaped conditions. Implementation of the domed four map conditions, however, warrants further explanation. Firstly, a dome-shaped mesh was defined, which was a set of coordinates (vertices) in 3D space that established the dome’s presence in the virtual environment (shown in Figure 3.3a). What varied between the domed SkyMap conditions was not this mesh, but the definition of the texture coordinates, which associated points on a texture (in this case, the map render texture, described above) to vertices on the sphere. For each of the vertices of the dome mesh (Figure 3.3a), a point on the map texture/image (Figure 3.3b) needed to be associated with it. The location and distribution of the texture coordinate points was defined by the math of the projection geometries, described in detail in Appendix A.

(a) (b)

Figure 3.3: Screenshots of (a) the dome mesh which served as a map surface, with vertices and edges visualized, and (b) a sample of the map texture that was rendered on to the dome mesh, based on locations of texture coordinate points (note: purple points are for illustrative purpose only, and are not reflective of actual quantity and position).

3.2.2 Trial configurations and counterbalancing

A total of 144 unique trial stimuli were generated. This was done by first selecting 48 unique locations in the South London area, based on areas of minimal elevation change. To systematically vary the configuration of the towers (i.e., their positions relative to each other), a set of 16 layout configurations was devised, based on the location of the green tower relative to the other two in the observer’s horizontal (azimuthal) and forward directions. These configurations are depicted in Table 3.2. A greater amount of centre configurations were shown to account for the possibility that having the green tower on either side would make the task overly difficult, which would necessitate isolation of centre-configured trials for analysis. The 48 unique locations contain three sets of these 16 tower configurations. Towers were placed such that they lay on streets and intersections, to avoid overlap with building objects. A set of three distances between the observer and the centroid of the three towers (“near”, “mid”, and “far”) was also introduced into the experiment, at approximately 100 m, 300 m and 500 m away, respectively. The curvature of the dome, and the changing of the local scale of projections as a function of distance from the dome centre, led to the consideration that the dome projections might perform differently at different tower distances (as addressed further in Section 3.5). The distance between the three towers themselves remained fairly constant – they always lay within a circle with a 200 m radius (M = 94.9 m, SD = 41.8). This was to reflect the realistic scenario in which an observer would reference the nearest available cues when Chapter 3. Experimental Method 22

Centre configurations (8)

(2) a = b, a 6= b (2) a = b, a 6= b (2) a = b, a 6= b (2) a = b, a 6= b

Left configurations (4) Right configurations (4)

Table 3.2: The 16 layouts defining the relative position of the green tower to the two reference towers (dashed circles). Centre configurations consist of four different layouts, with each either having or not having the green tower be approximately equidistant to the other two towers. identifying the location of a target. The 48 tower configuration sets, with three observer distances defined, comprised the 144 total trial stimuli. These 144 stimuli were then divided into six sets of 24 trials, labelled A through F, each having a balance of tower layouts (12 with the green tower in the centre, and six each with it on the left and right), and distances (eight each of mid, near, and far tower centroids), with none of the 48 tower locations repeated within a block.8 In order to counterbalance the order of the six map conditions as well as the assignment of stimulus sets A-F to these conditions (thereby also balancing the order of stimulus set presentation), I used the method proposed by Zeelenberg and Pecher (2015). It systematically identifies a set of 12 block-condition order configurations (a pair of Latin squares) that balance the sequence effects at both immediate and remote scales.8

3.2.3 Hardware & physical setup

The VR headset used for the experiment was an Oculus Rift CV19 with Oculus Touch controllers. The Oculus Rift, first released in 2016, uses OLED technology to generate a display resolution of 1080 × 1200 px per eye, creating a combined resolution of 2160 × 1200 px that spans a horizontal field of view of approximately 110◦. It has a refresh rate of 90 Hz, and supports interpupillary distances between 58 and 72 mm.10 The Rift is reported to weigh 470 g, though it was slightly lower for this experiment as the headphone attachments were removed. The experiment was run using an Alienware 13 R3 laptop with an NVIDIA GeForce GTX 1060 graphics card. The virtual environment simulation was run directly from the Unity player. The Oculus VR integration with Unity11 incorporates other immersion-supporting features, such as placing the player camera (i.e., the “eyes” which render to the headset) at the appropriate height for the player, and rendering a 3D model of the Touch controllers in the virtual environment (as can be seen in Figure 3.1) that mimics the motion of

8 The analysis in Chapter 4 included verification that the division of the stimuli, as well as the presentation order of conditions and stimulus sets, did not have an undue influence on the results. 9https://www.oculus.com/rift/ 10Source for all Oculus Rift CV1 specifications: VentureBeat (Horwitz, 2019) 11https://developer.oculus.com/unity/ Chapter 3. Experimental Method 23 the physical controllers. Participants were seated in a high-back, swivelling and tilting chair throughout the duration of the VR immersion; they were free to lean back and rotate in the chair, but were asked to not move the chair on its wheels. To track head and Touch controller movement, two Oculus sensors were used, placed in front of the participants in the left and right corners of the experiment area, approximately six feet above the ground.

3.3 Experimental Procedure

A fully within-subjects design was employed for this experiment, with each participant seeing all map conditions. The process whereby participants were recruited, screened and taken through the experiment was approved by the University of Toronto’s Research Ethics Board.

3.3.1 Participant recruitment & screening

Participants were recruited through departmental and field-specific mailing lists, and poster and Kijiji ad- vertisement. All recruitment ads stated the basic eligibility criteria and compensation information (defined below), and directed readers to a screening form. Eligibility for participation was confirmed by reported normal or corrected-to-normal binocular vision, and no known vision problems. The screening questionnaire also included an optional inventory to tabu- late prior experiences of motion sickness (adapted from the revised MSSQ by Golding (1998)), providing the warning that high scores indicate a greater susceptibility to experiencing sickness in the experiment. Participants provided acknowledgement of this as part of confirming eligibility. A total of 38 adults (19 men and 19 women) took part in the experiment; they were between the ages of 18 and 38 inclusively (M = 27.1 years, SD = 5.2) and were composed of 18 self-identified students (undergraduate and postgraduate) and 20 non-students. Participants were paid at a rate of $15/hour, rounded to the nearest 20 minutes. To incentivize reasonable effort for the task, a draw for an additional $25 was held after study completion, with participants receiving a number of ballots proportional to their performance (defined for this purpose as accuracy × speed).

3.3.2 Introduction and presentation of experimental task

Before starting the experiment, participants read and signed a physical copy of the informed consent form (shown in Appendix B), and completed a web-based pre-experiment questionnaire to gather information about any prior VR and/or action video game experience, appended with the Navigation Strategy Question- naire12 and the Santa Barbara Sense of Direction Questionnaire (Hegarty et al., 2002) (Appendix B.2). Their spatial abilities were assessed using the VZ-2 Paper Folding Test, a test of spatial visualization ability, and the S-2 Cube Comparisons Test (Ekstrom, French, & Harman, 1976), a test of mental rotation ability, both administered on paper. Tests are described in further detail in Appendix B.2.5. Participants’ stereoacuity was tested using the Bausch & Lomb Ortho-rater, an instrument that displays high-precision stereo images and tests for levels of stereoacuity in the range of 80 to 15 arcseconds (Epstein & Tredici, 1973). The last

12The shortened version of this questionnaire (Brunec et al., 2018) was used in this experiment; the original questionnaire was developed and validated by Zhong and Kozhevnikov (2016). Chapter 3. Experimental Method 24 step before entering the virtual environment was the viewing of a narrated slideshow, explaining the details of the three-tower, two-marker task, the Touch controller inputs, and the organization of the experiment into blocks of trials. They were provided with the controller for their preferred hand while going through the slideshow, to familiarize themselves with the buttons that were to be used in the experiment. Participants were then fitted with the Oculus Rift headset and introduced to the virtual environment. For each block, the first trial was counted as a practice only, in which they could take their time to familiarize themselves with the map display for that block. The practice trial was the same for each block, with the towers at a moderate distance, and the green tower in the middle of a roughly collinear configuration (see Table 3.1). After this trial, the block would proceed with the trial block as prescribed by the counterbalancing method, with the trials themselves presented in a randomized order. Participants were reminded of the $25 draw, and that their odds of winning were improved by quick and accurate responses on the non-practice trials. They were also invited to think aloud if they wished, and were notified that the experimenter would take written notes of their comments. The trial algorithm is outlined diagrammatically in Figure 3.4. For each trial, the headset image faded from black into an urban environment, with the three towers, and a map display with two markers, as described in Section 3.1.1. Participants used the Oculus Touch controller to indicate their responses: a virtual ray was emitted from the controller which allowed them to use it as a pointer (see Figure 3.1), and a cursor would appear where the ray intersected the map surface. Using the index trigger, they would place a marker at the cursor’s current location. An overlay screen would then appear (with the trial environment still in view), which asked the participant to adjust a rating scale to match their confidence in their answer. Alternatively, they had the option at this juncture to ask the experimenter to cancel their response and re-place the marker. Confirmation of the confidence rating (pressing X or A on the left or right Touch controller, respectively) would end the trial, having the headset image fade to black as the participant and towers were “teleported” to new locations. Each trial, including the first practice one, followed the same format of marker placement and confidence rating. No feedback was provided at any time regarding participants’ accuracy (though further explanation was provided, in the uncommon case that the participant demonstrated misunderstanding of the task). During the experiment, data was recorded about trial duration, angle of upward/downward head tilt (via head tracking), the indicated response (via multiple characteristic metrics), and the subjects’ self-reported confidence. Table 3.4 outlines all dependent variables measured. Each participant who completed the experiment underwent six blocks of approximately13 20 non-practice trials. Only 20 of the total possible 24 trials in each block were presented (randomly chosen within the set) in order to reduce the total experiment time. Participants were offered rest time between each of the blocks (in addition to the instruction given at the start, that they could take a break or stop at any time). Total experiment time ranged between 80 and 200 minutes, inclusive of all pre- and post-VR activity.

13Since the designation of the final trial of the block was indicated manually by the experimenter, a small amount of trial blocks exceeded 20 trials due to human error. Nonetheless, over 95% of blocks contained either 20 or 21 responses. Chapter 3. Experimental Method 25

Start N = 0

Trial loading Black screen complete Loading trial (city, map & towers) Fade from black

Show canvas:

Pointer enabled Waiting for marker placement P asks to cancel P places marker response Marker is Show canvas: deleted

Waiting for rating confirmation P confirms confidence rating

N < 20 N = 20

Fade to black Show canvas: Show canvas: N += 1 Fade to black

Figure 3.4: Finite State Machine diagram of the experimental task procedure, showing program logic and participant (P) actions and how the process repeated 20 times (as counted by N ) for each block. Blue text describes trial logic; green text describes UI logic.

3.3.3 Experiment debrief

After each trial block, participants rated their mental workload with each map condition via the NASA- TLX questionnaire (Hart & Staveland, 1988), administered on either a desktop computer (located in the experiment room) or a tablet. After the final block they completed the relative weighting portion of the NASA-TLX as it applied to the overall experimental task (throughout all blocks). They also reported any symptoms of motion sickness through the Simulator Sickness Questionnaire (SSQ) (Kennedy et al., 1993). Lastly, they participated in a short debriefing interview in which they discussed strategies they used for the task, and were prompted to comment on the interface and its usability. Written notes were taken of these interviews. Chapter 3. Experimental Method 26

3.4 Experimental Variables

Tables 3.3 and 3.4 list, respectively, the independent/predictor and dependent variables measured or analyzed in this experiment.

Independent & predictor variables Variable Level Range & description By trial Trial ID categorical [1:48] identifying the tower configuration Trial index ordinal [0:20] denoting the first, second, . . . , 20th trial seen (0 = practice trial) Tower configurationa categorical {left, centre, right}b Centroid distance interval {near (∼100 m), mid (∼300 m), far (∼500 m)} a Note: each trial ID maps uniquely to one tower configuration. b collapsed into three levels from the 16 unique configurations (Table 3.2) By trial block Block ID categorical [A-F] identifying the stimulus set Block index ordinal [1:6] denoting the first, second, . . . , sixth block Map condition categorical {Heads-Down, Flat SkyMap, Orthographic Dome, Conformal Dome, Equidistant Dome, Offset Dome} By individual Participant ID (PID) categorical [03:40] Sex categorical (cis-)male, (cis-)femalec Age ratio [18:38] VR experience ordinal {“never”, “tried” (1-2 times), “some” (3-5), “significant” (6-10), “regularly” (10+)} Action video game experience ordinal {<1, [1:7], 8+} hours/week in past six months Paper folding (spatial visuali- interval [0:20] zation) test score (VZ-2)d Cube comparisons (mental interval [-42:42] rotation) test score (S-2)d Sense of Direction interval [7:105] with higher scores denoting a better self-rated Questionnaire score (SoD)e sense of direction Navigation Strategy interval [-14:14] with more negative scores related to stronger Questionnaire score (NSQ)f “landmark-oriented” navigation strategies, and more positive scores related to more “map-oriented” strategies c A third option was provided in the pre-questionnaire, which no respondents selected. d Ekstrom, French, & Harman, 1976 e Hegarty et al., 2002 f Brunec et al., 2018

Table 3.3: Independent variables set by experiment design (by trial or by trial block) and predictor variables measured from participants (by individual) Chapter 3. Experimental Method 27

Dependent variables Variable Level Range & description By trial Absolute Error (AE) ratio Absolute geographic distance [m] of placed marker from actual marker location Visual Angle Error (VAE) ratio Absolute distance in visual angle [deg] between the placed marker and actual marker location Response time ratio Time [s] between start of trial and placement of markerg Confidence ordinal [0:5] with 0 = “not confident at all” and 5 = “extremely confident” Head elevation ratio Rotation of head about the x-axis [deg] (i.e., elevation angle), sampled at 2 Hz throughout trial g In the event that a participant’s response was cancelled, the timer would resume until another marker was placed. By trial block Meanh absolute error (MAE) ratio Absolute Error [m] (see above) averaged over all trials in block Meanh visual angle error ratio Visual Angle Error [deg] (see above) averaged over all trials (MVAE) in block Meanh response time ratio Response time [s] averaged over all trials in block Meanh confidence ordinal Confidence rating [0:5] averaged over all trials in block Workload scorei interval [0:100]j, with higher ratings indicative of higher subjective mental workload h Aside from mean, other descriptive statistics can be gained from aggregating data from all trials in block (standard deviation, IQR, etc.) iHart & Staveland, 1988 j scaled from range produced by orginal marking scheme ([15, 105]) By individualk SSQ scorel interval from 0, with higher scores indicative of severity of symptoms k Block-aggregated data can be further aggregated to determine scores by individual; however, this was not of main importance in the analyses. l Kennedy et al., 1993

Table 3.4: Dependent variables, measures of experiment outcomes (by trial, trial block, and individual) Chapter 3. Experimental Method 28

3.5 Experimental Hypotheses

With the experimental design and variables established, the hypotheses formulated in Section 2.5 can be further specified to correspond to the dependent variables.

3.5.1 Hypothesis 1: SkyMap vs. heads-down display

To address Research Question #1 (How effective is SkyMap in helping one to estimate the location of an element in their surroundings, relative to traditional map displays?), the benefits of a larger scale and apparent real-world alignment led to the hypothesis that SkyMap would allow for more accurate spatial knowledge to be inferred, compared to traditional map use. However, the novelty of dome curvature being applied to a flat map image might undermine any benefits that would otherwise be gained from those two properties. As such, the flat SkyMap should demonstrate the benefit of those properties, without being compromised by curvature-based misunderstanding.

Hypothesis 1: Absolute error will be lower in the flat SkyMap condition than in the heads-down condition.

3.5.2 Hypothesis 2: Relative performance of dome projections

As mentioned in Section 2.5, Research Question #2 (How does the type of projection on the dome affect one’s capacity to estimate the location of an element in their surroundings with respect to the SkyMap display?) was more exploratory in nature. Nevertheless, I proposed that projections whose distortions are least exaggerated (specifically the conformal projection, in contrast to the orthographic projection) would result in greater accuracy.

Hypothesis 2a: Absolute error will be lower in the conformal and equidistant dome projection conditions than in the orthographic dome projection condition.

The equidistant and conformal projections were noted by researchers and pilot testers to be visually similar14, thus it was speculated that they would result in comparable performance. Notwithstanding, both conditions were included in the study because their respective preservation of different spatial properties had the potential to produce observable differences in a larger data set. While the conformal projection evokes visual similarity to the original flat map by preserving shapes of map features (relative angle), the equidistant projection preserves relative egocentric distance relationships (recall Table 2.2), which is a component of the accuracy measure being taken in the experiment. The offset dome condition, on the other hand, was found by the same group to be disorienting, due to the nonuniform distortion of the map display. The offset also introduces a more complex spatial visualization task relative to the other displays, as the green tower’s marker location must be mentally “placed” not just above the tower, but at a particular distance forward relative to the observer. (Furthermore, this distance changes with the tower’s distance from the observer, as well as the orientation of the observer’s head.) These disadvantages are speculated to greatly outweigh the advantage of having the self-marker in the FFoV for this task, and thus I hypothesize that the accuracy scores yielded by this condition will be surpassed not only by the flat SkyMap condition, but also by the heads-down condition, which has the advantage of familiarity.

14This is supported by their comparable scale factors (see Table A.3, Appendix A). Chapter 3. Experimental Method 29

Hypothesis 2b: Absolute error will be lower for the flat SkyMap and heads-down conditions than they will for the offset dome condition.

In summary, the following relationships are hypothesized for the accuracy scores of the different map conditions (> means “more accurate”):

• flat SkyMap > heads-down map > offset dome (Hypotheses 1, 2b) • conformal dome projection ≈ equidistant dome projection > orthographic dome projection (Hypothesis 2a)

3.5.3 Hypotheses about influence of trial configuration

Through the experimental design process, a potential was identified for different characteristics of the three- tower, two-marker task having an influence on accuracy scores. As such, the trial-level variables of tower centroid distance, and relative green tower position (layout) (Table 3.3) were defined, and their effects are hypothesized below.

Hypothesis 3: Absolute error will be lower for trials in which the green tower is in the centre, than for those in which it is on the left or right.

Hypothesis 3 followed from the reasoning that the interpolation of the green marker location (i.e., its place- ment in between the two reference towers) would be less challenging, and would involve less uncertainty, than the corresponding extrapolation. While I expect this hypothesis to be true for all the map conditions, I anticipate the extent of this tower layout effect to vary with different map conditions (i.e., that there will be an interaction). However, to predict how map condition affects layout is complex, for the reasons described below, so no further speculation is presented here. The distance variable was included and was expected to have an influence on accuracy due to several factors that change with distance, including the size of the map in the observer’s FoV, the head elevation required to view reference markers, and the proximity (in various directions) of the map surface to the towers. However, since these and other factors might have manifested as different advantages and disadvantages trading off with one another, I refrained from formulating a specific hypothesis about their combined effect. Chapter 4

Results

4.1 Overview

Thirty-six complete data sets were acquired for analysis. A total of 38 people participated in the experiment; one participant voluntarily withdrew from the experiment after three blocks due to reported symptoms of simulator sickness. Another participant’s data was omitted due to observed disengagement with the task (including obvious haphazard, guessed responses), corroborated by an average error score more than seven standard deviations above the participant mean.1 Each of the 12 condition-presentation configurations (as described in Section 3.2.2) were shown to three different participants. Trial block length ranged between 18 and 24 non-practice trials. With 36 complete sets of data, and one set with three completed blocks, a total of 219 cases (unless otherwise noted) were included in the block- aggregated data analysis. A small number (three in total) of single trial responses were omitted based on noted observations of task misunderstanding (for example, if a participant needed clarification beyond the first practice trial). A total of 4400 responses were considered in analysis of individual trials, with outliers excluded as indicated in specific analyses below. Two measures of accuracy were considered. The first, Absolute Error (AE), specifies the distance in metres between the geographical locations of the response marker and the correct position. Figure 4.1 below shows the distribution of absolute error for 99.1% of individual trials. (The omitted 38 of the 4400 responses had an absolute error > 315 m, lying over three standard deviations above the mean). The figure shows that the majority (75.1%) of all responses were accurate to within 100 m of the correct location, with 43.4% within 50 m. See Appendix C for full descriptive statistics. While the absolute error value is relevant to the real-life tasks simulated by this experiment, it provides the notable drawback of being sensitive to the effective scale of the map, defined here as the combined effect of the cartographic scale of the map projected onto its respective surface (flat surface or dome), and the distance and orientation of this map surface relative to the observer. Effective scale not only affects the discernible level of detail of the map, due to the resolution limitations of the Oculus Rift, but also changes the influence of hand unsteadiness at the time of recording a response. To address this, the response data from each trial was also recorded in viewport coordinates, which define the location of a point in the virtual

1measured using mean absolute error averaged over all trial blocks

30 Chapter 4. Results 31

Figure 4.1: Histogram of Absolute Error data for all trials (excluding outliers) world with respect to the 2D image, or viewport, that is rendered to the headset2. This measurement, when combined with the known rendered field of view of the camera viewport, allowed for derivation of visual angle measurements, where the Visual Angle Error (VAE) was then defined as the visual angle, in degrees, between the locations of the response and correct markers at the time the response was given. VAE measurements are reported by block in section 4.2.2 below.

2More information: https://docs.unity3d.com/ScriptReference/Camera.WorldToViewportPoint.html Chapter 4. Results 32

4.2 Results by Trial Block

For this analysis, the trial data was aggregated by block (N = 219). Significance for all statistical tests in this thesis was set at the α = .05 level. For all measures in this section, the original data is plotted and discussed briefly, followed by significance testing with transformed data if necessary.

4.2.1 Effect of map condition on Mean Absolute Error

Figure 4.2 below shows a box plot of the medians and distributions of Mean Absolute Error (MAE) values by map condition, with box bounds representing interquartile range (IQR). Plotted points are values that lay further than 1.5 × IQR away from the upper and lower IQR values, and plot whiskers extend to all points within those limits. Four outliers (over three standard deviations from the mean) exceeded the plot range and their values are annotated with their numerical values. Note that all subsequent box plots follow this format, and all plots show the data untransformed.

Figure 4.2: Distribution of Mean Absolute Error values by map condition, N = 219. For this and all subsequent box plots, box bounds indicate the interquartile ranges (IQR) and are bisected at the median. Plotted points are values that lie further than 1.5 × IQR away from the upper and lower IQR values, and whiskers extend to all points within those limits. Outliers, whose values exceed the plot range, are annotated with their numerical values.

Figure 4.2 shows that the sample population performed best with the flat SkyMap condition (M = 56.2 m, SD = 21.3), and worst with the offset dome (M = 109.3 m, SD = 72.0).3 Another notable finding evident from this figure is that the offset and orthographic dome conditions’ MAE values were more widely distributed, with respective ranges of 331 m and 256 m. See Appendix C for full descriptive statistics of MAE data by map condition.

3Several pairwise comparisons were found to be statistically significant (as described in the next section), and are thus not annotated on the graph to avoid visual clutter. Subsequent box plots, however, do identify significant pairwise contrasts. Chapter 4. Results 33

MAE significance testing

For linear modelling, data were transformed using a fifth root transformation. This was found through trial and error to be the most parsimonious transformation for which the data exhibits homogeneity of variance (as reported below) and reduced skew. Though the transformed data yielded a non-normal Shapiro-Wilk test statistic (W = 0.95, p < .05), attributable to significant positive skewness (0.88, SE = 0.16) and kurtosis (2.12, SE = 0.33), the Shapiro-Wilk test is sensitive to large sample sizes (Field, Miles, & Field, 2012), and this transformation brought the data closest to normality. These test statistics were not expected to change the outcome of the analyses being run. Normality may be visually inspected in Figure C.1 (Appendix C). The transformed data met the assumption of homogeneity of variance across map conditions via Levene’s test, F (5, 213) = 1.79, p = .12. The variance ratio of the transformed data was 2.40, which is below the

Hartley’s critical Fmax value for six treatment levels with 36 cases/group at α = .05 (Field et al., 2012). Multilevel linear modelling using a maximum likelihood fit was executed in R (v 3.6.1), using the lme() function from the nlme package (v 3.1-141). Participant ID was treated as a random effect for the intercept. Orthogonal contrasts were constructed to show anticipated differences between specific groups. Finally, Tukey’s honest significant difference (HSD) post hoc tests were conducted on the model using the glht() function from the multcomp package (v 1.4-10). The model showed that map condition was a significant predictor of performance, χ2(5) = 90.78, p < .0001.4 Despite selecting contrasts that were anticipated to be meaningful, their results are com-

Heads-Down Flat SkyMap Orthographic Conformal Equidistant 0.165*** Flat SkyMap d = 0.86 r = .4 -0.028 -0.19*** Orthographic d = -0.23 d = -0.79 r = .12 r = -.37 0.13*** -0.03 0.16*** Conformal d = 0.66 d = -0.19 d = 0.67 r = .31 r = -.1 r = .32 0.096* -0.07 0.12* -0.04 Equidistant d = 0.44 d = -0.37 d = 0.53 d = -0.19 r = .21 r = -.18 r = .26 r = -.09 -0.12** -0.29*** -0.095* -.26*** -0.22*** Offset Dome d = -0.53 d = -1 d = -0.28 d = -0.9 d = -0.78 r = -.26 r = -.25 r = -.14 r = -.41 r = -.36

A positive value indicates that the [column name] condition had higher error values than the [row name] condition. Significance codes: *** = p < .001, ** = p < .01, * = p < .05

Table 4.1: Tukey’s HSD values from post hoc analysis of MAE model, with two measures of effect size (Cohen’s d and Pearson’s r). Note that difference values are shown for the transformed data.

4The χ2 statistic denotes the result of the likelihood ratio test, which compares the goodness-of-fit of a model with and without the predictor of interest. Chapter 4. Results 34

plex to interpret given the large amount of conditions and are thus not reported here. A complete summary of model statistics is found in Appendix C. Table 4.1 shows differences between individual map conditions, calculated from Tukey’s HSD post hoc tests, and effect sizes (Cohen’s d and Pearson’s r). See Appendix C for full output from these tests. Relative performance, in terms of MAE, of all of the conditions can be summed up as below (with MAE scores increasing/accuracy decreasing from left to right):

Flat SkyMap Heads-Down Conformal > > Offset Dome Orthographic Equidistant

All > relationships above are significant, p < .05. To check whether a learning effect (or other order effect) influenced performance measures, the variable of block index was added to the model as a predictor. This variable identified each trial block’s position in the chronological order of presentation to the participant. The inclusion of block index as a predictor did improve the model for MAE scores, but it did not affect the statistical significance of the comparisons identified above. This analysis is shown in Appendix C, Section C.1.1. To confirm that the division of the 144 unique trial stimuli into six sets (as described in Section 3.2.2) did not influence performance measures, the variable of block ID (comprising labels A through F) was also added to the model as a predictor; however, it did not improve the model fit for MAE scores. This analysis is shown in Appendix C, Section C.1.2. Chapter 4. Results 35

4.2.2 Effect of map condition on mean visual angle error

Figure 4.3 shows a box plot of the distributions of mean visual angle error (MVAE) values by map condition. Four outliers exceeded the plot range and their values are annotated. Significance between groups (as tested below, see Table 4.2) is shown in pink.

Mean Visual Angle Error by Map Condition 50.8 42.0 33.4 32.0

20

15

10

5

- * Mean Visual Angle Error [deg] Error Angle Visual Mean -

. ***

. .**.

. *** . -*- 0

HeadsDown Flat SkyMap DomeOrtho DomeConf DomeEqD DomeOffset

Figure 4.3: Distribution of mean visual angle error values by map condition. Significance codes for pairwise contrasts (in pink): *** = p < .001, ** = p < .01, * = p < .05

Despite the reasonable assumption that the heads-down condition would have had smaller visual angle errors, due to the smaller visual angle coverage of the map’s entirety, we see from Figure 4.3 that these errors were comparable to those on the larger map displays. Furthermore, the taller box plot for this condition suggests lower precision in responses. See Appendix C for full descriptive statistics of MVAE data by map condition.

MVAE significance testing

The distribution of the untransformed MVAE data had comparable skew and kurtosis values to the untrans- formed MAE data, so this set was also transformed using a fifth root transformation. The data were, again, found to yield a significant Shapiro-Wilk test statistic (W = 0.82, p < .05), and positive values for skew (2.03, SE = 0.18) and kurtosis (5.39, SE = 0.36). Figure C.2 (Appendix C) shows the comparison of the transformed data to a normal distribution. The transformed data met the assumption of homogeneity of variance across map conditions via Levene’s test, F (5,213) = 1.175, p = .32. Multilevel linear modelling was performed, using the same methods for the MAE modelling above (Section 4.2.1). This model shows that map condition was a significant predictor of MVAE, χ2(5) = 31.88, p < .0001. The planned contrasts for this analysis did not reflect the identified relationships in the data, and thus are not reported here. Complete model output can be found in Appendix C. The relative performance of the map conditions can best be identified by the Tukey post hoc tests, whose results and effect sizes are summarized in Table 4.2. See Appendix C for full output from post hoc tests. Results showed that the offset dome condition yielded the lowest MVAE values, and this was significant Chapter 4. Results 36

Heads-Down Flat SkyMap Orthographic Conformal Equidistant -0.002 Flat SkyMap d = 0 r = 0 -0.083*** -0.082** Orthographic d = -0.54 d = -0.53 r = -.26 r = -.26 -0.035 -0.033 0.049 Conformal d = -0.31 d = -0.3 d = 0.36 r = -.16 r = -.15 r = .18 -0.045 -0.043 0.039 -0.010 Equidistant d = -0.34 d = -0.33 d = 0.2 d = -.14 r = -.17 r = -.16 r = .1 r = -.07 0.024 0.026 0.108*** 0.059* 0.069* Offset Dome d = 0.26 d = 0.34 d = 0.72 d = 0.69 d = 0.54 r = .18 r = .17 r = .34 r = .33 r = .26

A positive value indicates that the [column name] condition had higher error values than the [row name] condition. Significance codes: *** = p < .001, ** = p < .01, * = p < .05

Table 4.2: Tukey’s HSD values from post hoc analysis of MVE model, with two measures of effect size (Cohen’s d and Pearson’s r). Note that difference values are shown for the transformed data. when compared to the other three dome projections. Implications of this, and of the other significant contrasts, are discussed in the next chapter. The MVAE data was also investigated for order effects and for stimulus set effects; however the model was not improved by adding block index nor block ID as a predictor. Chapter 4. Results 37

4.2.3 Effect of map condition on mean response time

Figure 4.4 shows the distribution of mean response time (the average time over all ∼ 20trials within each block) for all trial blocks. Four outliers exceeded the plot range and their values are annotated. Significance between groups (tested as described below) is shown in pink. Mean Response Time by Map Condition 60 84.9 69.0 57.6 63.8

40

20

. **

Mean Trial Response Time [s] Time Response Trial Mean .

. *** . . ** .

. **

0 . .***.

HeadsDown Flat SkyMap DomeOrtho DomeConf DomeEqD DomeOffset

Figure 4.4: Distribution of mean response times by map condition. Significance codes (in pink): *** = p < .001, ** = p < .01

Response time significance testing

For linear modelling, times were log-transformed. The transformed data were normally distributed, W = 0.99, p = .37, with non-significant skew and kurtosis values. The transformed data met the as- sumption of homogeneity of variance across map conditions, F (5, 213) = 0.51, p = .77. Multilevel linear modelling was performed using the same methods above. The model output showed that map condition was a significant predictor of mean response time, χ2(5) = 31.31, p < .0001. Post hoc Tukey contrasts revealed that participants took significantly longer using the offset dome SkyMap than each of the flat SkyMap5 (difference = 0.14, p < .001, Cohen’s d = 0.63, Pearson’s r = .30), heads-down (difference = 0.10, p = .007, d = 0.25, r = .22), conformal dome (differ- ence = 0.12, p < .001, d = 0.55, r = .26) and equidistant dome (difference = 0.11, p = .002, d = 0.51, r = .25) conditions. They also took significantly longer using the orthographic dome than the flat SkyMap (difference = 0.10, p = .007, d = 0.42, r = .2). No other contrasts in the analysis were significant. See Appendix C for detailed output from the linear model and post hoc tests. It should be noted that these differences (with respect to the untransformed data) are on the scale of 1-2 seconds, and thus may not have a substantial practical impact.

5Note that difference values reported here are for the transformed data. Chapter 4. Results 38

4.2.4 Effect of map condition on mean trial confidence

Figure 4.5 below shows the distribution of the 219 block-aggregated response confidence ratings for the trials, which were rated on a scale from 0 to 5. Significance between groups (tested as described below) is shown in pink.

Mean Confidence by Map Condition .***. 5 *

4

3

2

Mean Confidence Rating Confidence Mean 1

. ** 0 . *

HeadsDown Flat SkyMap DomeOrtho DomeConf DomeEqD DomeOffset

Figure 4.5: Distribution of mean confidence ratings by map condition. Significance codes (in pink): *** = p < .001, ** = p < .01, * = p < .05

Confidence rating significance testing

The confidence rating scale was assumed to be interval for the purpose of fitting a linear model. This may be justified by the fact that, while the endpoints of the scale were given labels (“not confident at all” and “extremely confident”), the midpoints were visualized as an empty bar that would fill in with colour at fixed, evenly spaced increments of 20%.6 (This can be seen in Figure 3.4.) To meet the other assumptions for linear modelling, confidence ratings were reverse-scored (subtracted from the maximum value of 5) and a square root transformation was applied. The transformed data were normally distributed, W = 0.99, p = .08, with negative but non-significant skew and kurtosis values. The transformed data met the assumption of homogeneity of variance across map conditions, F (5, 213) = 0.12, p = .99 and had an acceptable variance ratio of 1.28. Multilevel linear modelling was performed using the same methods above. The model output showed that map condition was a significant predictor of confidence ratings, χ2(5) = 22.48, p = .0004. Post hoc Tukey contrasts revealed that participants were significantly less confident in their responses in the offset dome condition than they were in each of the heads-down7 (difference = 0.18, p < .001, Cohen’s d = 0.39, Pearson’s r = .19), flat SkyMap (difference = 0.16, p = .001, d = 0.40, r = .20), conformal dome (difference = 0.13, p = .02, d = 0.31, r = .15), and equidistant dome (difference = 0.13, p = .02, d = 0.30, r = .15) conditions. No other contrasts in the analysis were significant. See Appendix C

6The limitations of this method are discussed in Section 5.3.1. 7Note that difference values reported here are for the transformed data. Chapter 4. Results 39

for detailed output from the linear model and post hoc tests. It should be noted that these differences (with respect to the untransformed data) are on the scale of hundredths of a rating unit, and thus do not make a substantial practical impact.

4.2.5 Effect of map condition on mental workload

Subjective mental workload scores were calculated using the standard method prescribed by the pencil-and- paper package for the NASA-TLX (Hart & Staveland, 1988), scaled here to a possible range between 0 and 100 inclusively. Due to malfunctions in the saving process, some workload scores were lost (22 of the 216 total cases). However, all subjective weightings were recorded successfully, so all 36 participants who completed the study are represented in this data set. Figure 4.6 below shows distribution of the ratings for each map condition. Significance between groups (tested as described below) is shown in pink.

Subjective Workload Score by Map Condition

100 .**.

75

50

25

Subjective Workload Score Workload Subjective

. *** . 0 HeadsDown Flat SkyMap DomeOrtho DomeConf DomeEqD DomeOffset

Figure 4.6: Distribution of mental workload ratings by map condition. Significance codes (in pink): *** = p < .001, ** = p < .01

From Figure 4.6, it is evident that participants reported a lower workload with the heads-down (M = 48.6, SD = 19.9) and flat SkyMap (M = 47.4, SD = 18.5) conditions.

Workload significance testing

The same linear methods from above were applied. The data, as is, met assumptions of normality (W = 0.99, p = .2) and homogeneity of variance as per Levene’s test, F (5, 188) = 0.18, p = 0.97. It should be noted that the questionnaire was designed to generate ratings on an equal interval scale, so use of this parametric test is appropriate. Map condition was shown to be a significant predictor of workload, χ2(5) = 20.67, p < .0001. Post hoc Tukey contrasts revealed that participants experienced significantly higher workload in the offset dome condition than in each of the flat SkyMap (difference = 13.33, p < .001, d = 0.67, r = 0.32) and heads-down (difference = 13.01, p = .001, d = 0.58, r = 0.28) conditions. No other contrasts in the analysis were significant. See Appendix C for detailed output from the linear model and post hoc tests. Chapter 4. Results 40

4.3 Analysis of Performance by Trial Configuration

Since the aggregation of trial data by blocks masked the potential influence of the tower centroid distance and green tower (relative) position, linear modelling of the individual trials was conducted to see if any insights might be gained. Figure 4.7 below shows the histogram of Absolute Errors (AE) for 99.1% of all trials (similar to Figure 4.1), but separated into subplots based on trial layout (with the green tower either centred, or off-centre8) and tower centroid distance (towers ∼100 m, ∼300 m or ∼500 m from the participant, coded respectively as near, mid and far levels).

near mid far N = 713 N = 718 N = 727

100 centre

50

0

Count N = 718 N = 730 N = 716

100 off-centre

50

0 0 100 200 300 0 100 200 300 0 100 200 300 Absolute Error [m]

Figure 4.7: Absolute Error distribution by trial layout and distance (note: outliers are omitted)

Upon visual inspection, it can be noted that the trials with the green tower centred and near the partic- ipant (top-left plot of Figure 4.7) were the most skewed towards zero (i.e., more smaller errors). Trials with the tower off-centre did not appear to yield different AE values at different distances.

8Since differences were not expected to be seen between the tower being on the left or the right, those two tower configurations were collapsed into one variable for all the off-centre cases. Chapter 4. Results 41

Figure 4.8 below shows the VAE data, grouped by distance and layout.

near mid far 200 N = 715 N = 724 N = 732

150 centre

100

50

0 200 Count N = 722 N = 741 N = 723

150 off-centre

100

50

0 0 10 20 30 40 0 10 20 30 40 0 10 20 30 40 Visual Angle Error [deg]

Figure 4.8: Visual Angle Error distribution by trial layout and distance

The distributions for VAE data show that, for trials with the green tower centred (top row), the distribu- tions were more left-skewed when the towers were further away from the participants. The off-centre trials showed this trend to a lesser extent. The implications of this are discussed in the next chapter. Figures 4.9 and 4.10 below show interaction diagrams of distance and layout for each of the six map conditions, for the AE and VAE data respectively. The error values were either lower or comparable when the tower was centred, for all map conditions. At near distances, the Flat SkyMap outperformed the other map conditions, but at further distances the conformal dome showed comparable accuracy. It is also interesting to note that for the orthographic and equidistant dome projections, errors between the centred and off-centred tower layouts were less differentiated at the mid distance than at either of the near or far distances. Compared to Figure 4.9, Figure 4.10 shows more distinct differences in accuracy between centred and off-centred trials, showing that the green tower’s relative position affected participants’ capacity to correctly estimate its location. It also reveals that the wide distribution seen in the data for the heads-down condition (Figure 4.3) is actually comprised of three sets of precise data at three different distances. Visual angle error generally decreased with distance across all map conditions. The factors to which this may be attributable are complex and interplay with one another; these factors are discussed in detail in Section 5.2. Chapter 4. Results 42

Figure 4.9: Absolute Error means by trial layout, distance and map condition. Points represent means; error bars represent 95% CI. Points/error bars horizontally offset for visual clarity.

Figure 4.10: Visual Angle Error means by trial layout, distance and map condition. Points represent means; error bars represent 95% CI. Points/error bars horizontally offset for visual clarity. Chapter 4. Results 43

4.3.1 Significance testing of AE & VAE data

In order to further explore the effects seen in Figures 4.9 and 4.10 in a way that is clear to interpret for each map condition, and to establish statistical significance, the AE and VAE data for each map condition’s data were modelled separately. A fifth root transformation was applied to all the AE data, as in Section 4.2.1. However, this transformation, when applied to the VAE data, yielded values that violated homogeneity of variance, so a cube root transformation was applied instead to VAE values. For all models, multilevel modelling was used in the same way as in the previous sections. The models declared participant ID within block index as random effects for the intercept. To check for within-block learning effects, trial index (the variable identifying the position of the trial in the presentation order) was also introduced as a variable to see whether it improved the model for any map conditions. Models were built up sequentially to determine which main effects or interaction effects were significant, and only those terms were included in the models reported. A summary of each model, with relevant statistics identified, can be found in Section C.2.1 (Appendix C). Complete model output is also included in Appendix C. Below is an overview of the significant predictors of AE and VAE. Note that, because of the separate analyses by map condition, effects that are reported within map conditions may be considered interaction effects between map condition and the other predictor(s).

Predictors of absolute error (AE)

• Trial layout was a significant predictor of AE for the heads-down, flat SkyMap, and offset dome conditions. • Tower centroid distance was a significant predictor of AE for the heads-down and flat SkyMap condi- tions. • There was a significant interaction effect between layout and distance for the orthographic and equidis- tant dome projection conditions.

Predictors of visual angle error (VAE)

• Trial layout was a significant predictor of VAE for the conformal, equidistant, and offset dome condi- tions. • Tower centroid distance was a significant predictor of VAE for all map conditions except the flat SkyMap. • There was a significant interaction effect between layout and distance for the flat SkyMap condition.

Influence of trial index on error scores

• Trial index was a significant predictor of AE for the heads-down, flat SkyMap, and orthographic dome conditions, with AE scores being lower (accuracy increasing) for later trials within the block. • For the AE scores in the conformal dome condition, there was a significant interaction effect between trial index and layout, with AE scores being lower for later trials, but only for off-centre trials. • For the VE scores, trial index was a significant predictor for the heads-down condition. Chapter 4. Results 44

4.3.2 Directional error

Though the absolute error scores are meaningful to analyze for practical application, they do not provide information about whether responses were biased in a particular direction. To address this, the visual angle error was separated into its horizontal and vertical components (showing deviation in the azimuthal and elevation angles, respectively), and its values given a positive or negative sign based on the relative location of the response marker from the correct location. This data is shown in a density plot, by map condition, tower distance and tower layout, in Figure 4.11.

Heads-Down Flat SkyMap Orthographic Conformal Equidistant Offset

5 near 0

-5

5 mid 0

-5

Vertical VA deviation [deg] deviation VA Vertical 5 far 0

-5

-5 0 5 -5 0 5 -5 0 5 -5 0 5 -5 0 5 -5 0 5 Horizontal VA deviation [deg]

Green tower position: centre off-centre

Figure 4.11: Density contours showing horizontal and vertical visual angle deviations by trial layout, distance and map condition. Note that for each plot, the positive vertical direction approaches the self-marker (closer to the observer), and the negative vertical direction approaches the horizon (farther from the observer).

Figure 4.11 provides further insight about the nature of the errors that were made across different map conditions and trial configurations. Firstly, based on the proposed benefit of SkyMap’s azimuthal alignment, it would have been reasonable to anticipate that response deviation would occur more in the vertical direction than in the horizontal (azimuthal); this was indeed the case for centred trials in the flat SkyMap and non- offset dome conditions (orthographic, conformal and equidistant). We also see that, for the map conditions whose VAE scores had layout as a significant predictor (i.e., the conformal, equidistant and offset domes), the increase in absolute VAE is attributable to deviation in the horizontal direction, primarily at the near distances. The error distributions in the figure are, for the most part, centred around the correct response (the crosshairs at (0, 0)), with exceptions for the orthographic and offset domes. For those conditions, the dis- tributions are centred slightly higher, indicating that participants tended to underestimate distances. These same two conditions also have the only error distributions that are wider than they are tall, contradicting Chapter 4. Results 45

the reasoning that SkyMap alignment would mitigate azimuthal error, and this occurred for off-centre trials at the nearest distance level. It should be noted that, among all conditions, the smallest effective scale of the map was seen in those conditions, in the area closest to the participant. Furthermore, the offset dome showed the most azimuthal deviation at the nearest distances, which is consistent with the fact that the orientation dependence (which arose as a consequence of the forward shift) was highest around the area of the participant’s self-marker. Lastly, despite the similarity in absolute VAE for the conformal and equidistant domes, Figure 4.11 shows that the directions of the errors being made were quite different between these conditions, particularly when looking at the near distances, and the off-centre trial layout.

4.4 Individual Differences

As an initial exploration into the individual differences, a correlation table is presented below of the different non-experimental measures taken, compared with participants’ overall mean MAE scores. Measures in Table 4.3 include age, scores on visualization (VZ-2) and mental rotation (S-2) tests, action video game (AVG) and VR headset experience, Navigation Strategy Questionnaire scores (NSQ), and the Santa Barbara sense of direction (SoD) scale. The only measures that correlated with task error were the scores on the two spatial tests (r = -.52, p = .0009 with the VZ-2 score, and r = -.46, p = .004 with the S-2 score), which were also correlated with each other (r = .52, p = .0009). Self-reported sense of direction (SoD) scores had a strong correlation with the NSQ scores (r = .61, p < .0001) which were more positive for self-reported “map thinkers”. There was also a tendency for older participants to have more map-like thinking (r = .41, p = .013). The measures of VR experience were not significantly correlated with any other measures. A t-test also revealed that there was a significant sex difference in AE scores, t(29.397) = 2.81, p = .009, with males’ overall AE scores (M = 63.00, SD = 21.2) lower than females’ (M = 90.19, SD = 36.10). Data from the post-experiment simulator sickness questionnaire, scored as instructed in the original publication (Kennedy et al., 1993), had no correlations with any of the measures above.

Mean MAE VZ-2 score S-2 score AVG exp VR exp NSQ SoD VZ-2 score -0.52*** 1 S-2 score -0.46** 0.52*** 1 AVG exp -0.17 0.06 0.45** 1 VR exp 0 -0.1 -0.13 0.05 1 NSQ 0.05 0.14 -0.16 0.12 -0.05 1 SoD -0.06 -0.03 -0.09 0.13 0.16 0.61*** 1 Age -0.01 -0.27 -0.25 0.07 -0.02 0.41* 0.42 Significance codes: *** = p < .001, ** = p < .01, * = p < .05

Table 4.3: Correlation table comparing overall task performance (mean MAE) to data from pre-experiment questionnaires. Chapter 4. Results 46

4.5 Feedback & Qualitative Observations

During the debrief interviews (detailed in Appendix B.4), participants (Ps) were asked about their preferred interfaces and their strategies. Fifty-four percent (20 of 37) preferred the traditional heads-down map, and eight preferred the flat SkyMap. The remaining conditions were preferred by one to three participants each. Reported reasons for preferring the heads-down map included the fact that the entire map was visible and unoccluded (mentioned by three participants), but more often noted was that looking up for the other conditions was straining or not preferable (11 Ps). Two comments were made about the SkyMap being “larger”, with that serving as a benefit. Though the equidistant and conformal projections were only preferred by two participants each, they evoked comments about being “intuitive”, “clearer” and allowing judgement “by feel”. When asked which map conditions were easier to understand (as opposed to preferred), the conformal and equidistant projections were mentioned more often, four and five times respectively. Interestingly, the orthographic projection was noted in comments to be the easiest to understand by some (three Ps), and yet challenging by others (two Ps). The use of the two reference towers and markers was also substantiated by interview comments, where 20 of the participants mentioned it as part of their strategy. Some of these comments cited relating the green tower to the two towers, while other participants considered the distance the towers were away from them. Geometric relationships were also involved: there were five mentions of the “triangle” or shape made by the three towers, and four reported imagining lines either from themselves to the towers or between towers. At least two participants were observed extending their arms out straight (even though they could not see their arms while wearing the VR headset). Seven noted making use of environmental features (i.e., roads, other buildings). Chapter 5

Discussion

5.1 Substantiation of SkyMap Advantages

Research Question #1 sought to determine whether SkyMap’s proposed advantages are substantiated. Recall that the proposed advantages were:

(a) the utilization of the forward field of view, (b) the larger scale afforded by this larger overhead space, and (c) the seeming “alignment” of the environmental features and their symbolic map counterparts.

These advantages led to the alternative hypothesis (#1, section 3.5) that the flat SkyMap would outperform the heads-down map. The absolute error data indicate a performance improvement: from Table 4.1, we see that the flat SkyMap, and the dome SkyMaps with the equidistant and conformal projections, all had lower MAE scores than the traditional heads-down map, significant at the α = .05 level. These maps all possess advantage (b), with the tradeoffs between the extent to which they exhibit advantages (a) and (c), based on their particular projections and shapes (as described in Table 2.1). The flat SkyMap had significantly lower MAE scores than the other five conditions (in aggregate), with a medium effect size (r = 0.4)1. Though this map does not fully utilize the “sky” space, thus not exhibiting advantage (a) to its maximum potential, it effectively functions as a flat mirror in the sky, preserving alignment (advantage (c)). This is supported by observations from participants, who observed that the image is “just projected up” (P10), or that the map was “directly above the real life thing” (P16). P28 made the analogy that this condition was the “floor projected onto the roof”. The MVE scores were not significantly different in the contrast between the heads-down map and all non-offset SkyMap conditions (reported in Appendix C). However, the averaging of trial data over blocks masked many effects, which are discussed in the next section. A dome projection, as discussed in Table 2.1, was proposed as an improvement over the flat SkyMap, due to the utilization of the full sky-space, maximizing advantage (a), but at the cost of negatively impacting apparent alignment (advantage (c)). Tukey’s HSD contrasts (Tables 4.1 and 4.2) indicated that the equidis- tant and conformal domes did not yield significantly different MAE or MVE scores over the flat SkyMap

1This result emerges from one of the planned contrasts for the MAE model, as reported in Appendix C.

47 Chapter 5. Discussion 48 condition. Figures 4.9 and 4.10 illustrate that this holds true across tower configurations and distances. These results indicate that the maximizing of the sky-space utilization (advantage (a)) at the cost of planar alignment (advantage (c)) did not result in improved accuracy. Despite the lack of familiarity with the overhead map, absolute accuracy at locating objects (using both the flat overhead and certain domed SkyMaps) was improved over the traditional, heads-down map, and Hypothesis 1 may be accepted.

5.2 Effects of Trial Configuration on Map Condition Differentia- tion

Research Question #2 addressed the differences in performance that the dome projections might exhibit. While this question was posed as a more exploratory one, it was speculated that the conformal projection might yield the best accuracy because of its minimizing of noticeable distortion (by preserving shape), and that the equidistant projection might perform comparably, as its distortion relationships are not very different from the conformal map (see Table A.3, Appendix A). Those speculations were substantiated in the data, as those conditions yielded similar performance (as assessed by MAE data) that was better than the other two dome conditions (Table 4.1). Recall that these two projections also resulted in accuracy that did not differ significantly from the best-performing map condition, the flat SkyMap. Detailed insights about the different map conditions unearthed by the unaggregated data are discussed below. Looking at the trials’ AE scores by distance and layout (Figure 4.9), it is evident that the flat SkyMap condition performed better than the heads-down at the near and mid distances, and comparably at the far ones. The conformal and equidistant dome projections also have lower AE values than the heads-down map for centred trials, and comparable AE values for off-centre trials. For the conditions in which layout was a significant predictor of AE (the heads-down, flat SkyMap and offset dome conditions), Hypothesis 3 may be accepted: AE values were lower for trials in which the green tower was centred, compared to when it was off-centre. The AE values generally follow from the analysis for MAE scores, but don’t provide much more differentiation.

5.2.1 Interpreting the meaning of visual angle error

The VAE scores by trial, in contrast to the AE scores, reveal important insights. Firstly, they show that VAE was mediated by distance, with answers that were more accurate at further distances. For all map conditions except the flat SkyMap, this relationship was statistically significant. To understand potential contributors to this result, recall that VAE was intended to control for map conditions’ differing effective scales, defined earlier as the consequence of both the scale of the map image as it appears on its surface, and of the location of this surface relative to its observer. Theoretically, by controlling for effective scale, VAE should function as a measure that is more reflective of where participants thought the marker should be. When considering the two flat map conditions (the heads-down map and the flat SkyMap), an additional mediating factor should be considered: the psychomotor aspect of the experimental task. Participants were required to use an Oculus Touch controller to position their marker, which appeared on the map surface, and press a trigger button to record their response. Since the dome conditions maintained the map surface at the same distance from the participant in all directions, it had a consistent influence on all trials. For the Chapter 5. Discussion 49

flat maps, however, trials with the towers farther away meant that the region of interest on the map surface was also farther away (since both flat maps were tilted away from the participant), so small movements of the controller were more influential. One limitation to the meaningfulness of the VAE measure, however, is that the trend of smaller VAE values at farther distances follows logically from the visual angle subtended by the stimuli, i.e., the towers themselves. The general vicinity around the towers – essentially, the region containing a reasonable estimate to the green tower’s location – is visually smaller when the towers are farther away. As a result, the VAE range for a reasonable estimate is also smaller, yet not necessarily indicative of participants being more accurate in their location perceptions. Thus, it is necessary to consider VAE scores alongside AE scores to make reasonable conjectures about relative map condition performance. For the heads-down map condition, VAE decreased with distance. The fact that the map’s effective scale also decreases with distance (because of the tilt of the flat map surface) suggests that the responses were consistently within a comparable geographical size. This is supported by Figure 4.10 (top-left), and the presence of only a small effect of distance for AE scores (r = .09), which indicates that AE does not change drastically with distance. This slight increase of AE with distance may be attributable to the psychomotor aspect of the task, as elaborated above. In a similar way, the flat SkyMap’s VAE values also decreased with trial distance, with values comparable to the heads-down condition (when averaged over layout type; see Figure 4.10). Though the much larger scale of the flat SkyMap allowed for greater absolute accuracy (Figure 4.9), the location and orientation of the map surface caused both a faster diminishing of effective scale, and a greater sensitivity to motor-induced response deviations. These combined to result in AE values that increased incrementally with distance, with a significant yet small effect (r = .12). Additionally, the significant effect of trial layout (r = .21) for this map condition shows that the deviation in the azimuthal direction (i.e., off-centredness) resulted in worse accuracy relative to the heads-down map, despite its larger scale. The conformal and equidistant projections, which both out-performed the heads-down map in absolute accuracy, had larger VAE values at near distances, but comparable VAE to each other at mid and far distances (Figure 4.10). Again, the larger map scale afforded by the dome-shaped SkyMap resulted in better absolute accuracy at the farther two distances relative to the heads-down map. Based on participant observation, it is reasonable to posit that the higher VAE at near distances for the conformal, equidistant, and orthographic projected domes is attributable to the need for participants to look further upwards to place the marker, losing sight of the extents of the environment (and the distance relationships visible therein) and thereby counteracting the presumed advantages of map scale. The orthographic dome’s VAE values had comparable distance and layout dependencies as the two projections discussed above, however its AE values were significantly higher. One possible explanation for this is the fact that, for this projection, there was a farther visual separation between the towers and their markers, i.e., they appeared “higher up” on the dome surface. This would make it more difficult to perform the “projecting upwards” mental process that some participants reported using for other conditions, and heighten the complicating effect of looking up which was discussed above. Recall that there is also the proposed idea that the distortions introduced by this projection are more visually noticeable than in the others, thus potentially suppressing the tendency to make spatial inferences in the way one would with an undistorted bird’s-eye view map. The wider confidence intervals on the VAE data also manifest increased imprecision. It is also interesting to note that despite the significantly larger AE values observed with the offset dome Chapter 5. Discussion 50

relative to the heads-down map, its VAE values for centred trials had the lowest values among all conditions. One potential explanation is that the effective scale for the offset dome is very small in the space in front of the observer, because the forward shift “squishes” the map image to consume half the visual angle in the forward direction (i.e., the self-marker is 45◦ above the horizon rather than 90◦ as in the other SkyMap conditions), and that greatly reduced the visual angle size of a reasonable location estimate. As with the heads-down map, this scaling factor converted small VAE into relatively large AE, but even more so in the offset dome case, likely due to the farther distance of the map surface from the participant. The detriment induced by the directional dependence of distortion in the offset projection is evidenced by the considerable effect (r = .19) of layout on VAE: the arc length (visual angle distance) from participants’ self-marker to the green tower would change with their head orientation. Combined with the small scale of this projection, this resulted in the highest AE values among all conditions. Though the practical implications are minor, the offset dome also yielded significantly slower responses (Section 4.2.3) and lower confidence (Section 4.2.4) ratings relative to all other conditions except the orthographic, and significantly higher subjective mental workload than each of the heads-down and flat SkyMap conditions (Section 4.2.5). To summarize, the conformal and equidistant dome conditions, with their large scale and more uniform distortion (along the azimuthal and elevation dimensions), allowed for greater accuracy on a spatial visu- alization task that exceeded that of a traditional, heads-down map. The orthographic projection with its non-uniform distortion had worse accuracy, despite also having a larger scale. Thus, Hypothesis 2a may be accepted. Lastly, disruptions of spatial relationships introduced by the offset resulted in comparatively poorer performance: Hypothesis 2b was supported. Chapter 5. Discussion 51

5.3 Study Limitations

Despite the promising results presented above, the shortcomings and limitations of this study should be acknowledged in order to better understand the SkyMap paradigm and its utility. Firstly, the experimental setup lacks validity in a few respects, including visual realism. The virtual environment produced by WRLD 3D was very basic, uniform and detail-sparse on the street level (see Figure 3.1), making the artificial (and unrealistic) visually distinct landmarks even more salient. This leaves the influence of environmental distractors effectively unexplored. In a real setting, these distractors may require more mental demand for the task of matching a landmark to its map representation. The use of different software for the environment and the map also led to some inconsistencies between the two that were noticeable by participants during one or two trials; specifically, one participant made a comment when they observed that buildings they saw in front of them were not visualized on the map. Involvement of some randomization in the trial block presentation led to some unequal group sizes in the trial configuration variables (as reported in Figures 4.7 and 4.8). The randomization was introduced by the fact that only the first 20 trials in a randomly-shuffled 24-trial block were presented to participants, which was done to reduce total experiment time. Though the experiment allowed for a very large amount of observations, minimizing the impact of imbalances, an improvement on the design would ensure that all variables had equal exposure for all participants. A major limitation of this experiment was that it took place in VR rather than augmented reality (AR), the intended implementation format of SkyMap, where the technology may not integrate map images with the environment as smoothly as in a fully virtual space. An improvement on this experiment would be to either conduct it in a real outdoor urban environment, or utilize a 3D model that is richer in detail, and perfectly matches the map data. The Mapbox API is a viable candidate to provide these features, especially since it is already used for the map. With all factors considered, nonetheless, the use of VR and its complete control of the presented environ- ment eliminated the complications associated with a real-world setting, which would make the experiment more difficult to run and also complicate interpretation. Furthermore, the variables explored in this study and differentiated in the analysis do not rely heavily on ecological validity, rendering this work still insightful and beneficial.

5.3.1 Limitations & potential extensions of data analysis

While the analyses in this thesis addressed the research questions of interest here, it also has particular limitations, as well as opportunities for further exploration. For one, the questionnaire data (specifically the confidence rating) was analyzed with linear modelling, which is known to require that data be measured at the interval level (Spector, 1992). However, the robustness of these analyses to this assumption violation has been argued (see Willits, Theodori, and Luloff (2016) for a summary). Though the linear model showed that map condition had little practical impact on participants’ reported confidence, non-parametric statistics could be used to robustly confirm this. The conducted analyses also involved minimal outlier removal. Though outliers should not be removed without reasonable cause, overly influential cases may have weakened the conclusions presented. To poten- tially support removal of outliers, data such as the questionnaire responses and observational notes could be revisited to more stringently identify a lack of reasonable effort at the experimental task. Other analyses that can be done with the collected data to yield further insights include an analysis of Chapter 5. Discussion 52 the glance data, and quantitative analysis of the directional error scores.

5.4 Considerations for SkyMap Implementation

Now that the proposed advantages of the SkyMap concept over traditional maps have been addressed, and differentiation made among dome projections, considerations for further design refinement may be suggested from the data. Directions for further investigation are also offered, based on the specific nature of this task and its limited scope in relation to more general wayfinding and navigation activity. Firstly, it should be recalled that a flat SkyMap performed just as well as two of the dome projections on the metrics assessed here, despite not maximally utilizing overhead space. The proposed incremental benefit of consuming the entire overhead view (recall Table 2.1) was not manifested in this task. However, Figure 4.9 showed that AE increased with distance for the flat SkyMap, but not for the two viable dome candidates (the equidistant and conformal projections). Thus, though the distance range explored here did not show absolute distinctions between domed and flat SkyMaps, distance range of interest should be further considered in other designs. It should also be noted that the SkyMap conditions, even those that outperformed the heads-down map, showed an effect of trial layout. In realistic settings, this would mean that locations of landmarks that can be interpolated (i.e., that are located between features with known, marked locations) will be more accurately estimated than those that must be extrapolated. Thus, the sparseness of environmental cues should be considered when implementing SkyMap, as the traditional heads-down map did not exhibit such a dependence. Lastly, it should be acknowledged that the effort to include the self-marker in the forward field of view (via the offset dome projection condition) is one based on the proven importance of the self-marker in the literature (Levine, 1982; Montello, 2010). This was corroborated by reports from the debrief interview that most participants were referencing the self-marker for many, if not all of the trials. Due to the inclusion of several aligned SkyMap conditions in this study, reports of neck fatigue were frequent. Unfortunately, applying an offset to the top of the dome resulted in a detriment in several of the performance metrics analyzed. This supports the argument that alignment of the map to the environment is one of the instrumental contributors to SkyMap’s shown advantages. The experimental task performed in this study was meant to simulate situations where precise self-location was not a priority, so the importance of this information in specific applications should be assessed when making decisions about SkyMap use. It should also be noted that, like in this experiment, users of SkyMap might make the effort to view their self-marker even when it is not of primary interest. This issue should therefore be addressed by exploring design solutions that allow for low-effort self-location visualization without disrupting the alignment between SkyMap and the environment. Chapter 6

Conclusion

6.1 Summary of Key Findings

The investigation presented here sought to determine whether the benefits of the SkyMap concept intro- duced by Kapler et al. (U.S. Patent No. 9047699B2) would be realized for a spatial visualization task. The use of the overhead space, the larger map scale afforded, and the appearance of map alignment to the environment, allowed participants to more accurately indicate the location of an unmarked element in their environment. We may conclude that, under certain task conditions, the benefits of SkyMap were substanti- ated. Though these advantages were exhibited by a flat SkyMap that simulates a horizontal “mirror in the sky”, dome-shaped implementations were also investigated for a potentially enhanced benefit. However, the dome conditions performed either comparably (in the case of the conformal and equidistant projections) or worse than the flat SkyMap condition, showing no extra accuracy support. Furthermore, the dome condition that least resembled a traditional flat map (the orthographic projected dome) performed worse than other dome conditions, and also worse than the flat SkyMap. Lastly, despite the importance of looking at one’s self-location (as described in the literature and also manifested in participant feedback in this experiment), the implementation of an offset that disrupts the map alignment to the environment resulted in the worst performance in this experimental task, despite facilitating self-location viewing.

53 Chapter 6. Conclusion 54

6.2 Implications for SkyMap Design and Future Research

Beyond serving as a proof-of-concept for SkyMap’s advantages, this investigation also contributes potential design implications for wider use of SkyMap, and proposes research directions to further confirm its capacity to support navigation. Firstly, the finding that a flat SkyMap was as effective as a dome to support spatial visualization accuracy may be useful for considering what additional information can be provided in the visual space of the SkyMap user. For example, a flat SkyMap could be used when other information might be of interest to show in the unoccluded sky-space, whereas a dome SkyMap could be used to effectively visualize an annotated or more detail-rich map display without hurting spatial visualization accuracy. Secondly, the reliance of SkyMap effectiveness at this task on the location of surrounding cues (discussed earlier as trial layout) provides some indication of the types of environments and maps in which SkyMap may benefit performance. Further design considerations were mentioned that may better support extrapolation of spatial information. Finally, the implementation of a self-marker in a way that does not undermine the benefits of SkyMap and that does not impose an unacceptable mental workload remains as a design challenge for further exploration. References

Aguirre, G. K., & D’Esposito, M. (1999). Topographical disorientation: A synthesis and taxonomy. Brain, 122 (9), 1613–1628. doi: doi:10.1093/brain/122.9.1613 Allen, R. C., Mcdonald, D. P., & Singer, M. J. (1997). Landmark direction and distance estimation in large scale virtual environments. In Proceedings of the human factors and ergonomics society (Vol. 2, pp. 1213–1217). doi: doi:10.1177/1071181397041002109 ArcGIS. (n.d.). Grids and graticules. Retrieved 2019-11-30, from https://pro.arcgis.com/en/pro-app/help/ layouts/grids-and-graticules.htm Aretz, A., & Wickens, C. (1992). The Mental Rotation of Map Displays. Human Performance, 5 (4), 303–328. doi: doi:10.1207/s15327043hup0504 3 Bediou, B., Adams, D. M., Mayer, R. E., Tipton, E., Green, C. S., & Bavelier, D. (2018). Meta-analysis of action video game impact on perceptual, attentional, and cognitive skills. Psychological bulletin, 144 (1), 77. doi: doi:10.1037/bul0000130 Berggren, J. L., Jones, A., & Ptolemy. (2000). Ptolemy’s Geography : an annotated translation of the theoretical chapters. Princeton University Press. Brooks, A., Nowakowski, C., & Green, P. (1999). Turn-by-turn displays versus electronic maps: an on- the-road comparison of driver glance behavior (Tech. Rep.). Ann Arbor: The University of Michigan Transportation Research Institute (UMTRI). Retrieved from https://deepblue.lib.umich.edu/ Brunec, I. K., Bellana, B., Ozubko, J. D., Man, V., Robin, J., Liu, Z.-X., . . . Moscovitch, M. (2018). Multiple scales of representation along the hippocampal anteroposterior axis in humans. Current Biology, 28 (13), 2129–2135.e6. doi: doi:10.1016/J.CUB.2018.05.016 Card, S. K., Mackinlay, J. D., & Shneiderman, B. (1999). Readings in information visualization : using vision to think. Morgan Kaufmann Publishers. Conson, M., Bianchini, F., Quarantelli, M., Boccia, M., Salzano, S., Di Vita, A., & Guariglia, C. (2018). Selective map-following navigation deficit: A new case of developmental topographi- cal disorientation. Journal of Clinical and Experimental Neuropsychology, 40 (9), 940–950. doi: doi:10.1080/13803395.2018.1451493 Dey, S., Karahalios, K., & Fu, W.-T. (2018). Getting there and beyond: Incidental learning of spatial knowledge with turn-by-turn directions and location updates in navigation interfaces. In Proceedings of the symposium on spatial user interaction (p. 100–110). New York, NY, USA: Association for Computing Machinery. doi: doi:10.1145/3267782.3267783 Ekstrom, R. B., Dermen, D., & Harman, H. H. (1976). Manual for kit of factor-referenced cognitive tests (Vol. 102). Educational Testing Service. Retrieved from https://www.ets.org/research/policy research reports/

55 References 56

Ekstrom, R. B., French, J. W., & Harman, H. H. (1976). Kit of factor-referenced cognitive tests. Educational Testing Service. Retrieved from https://www.ets.org/research/policy research reports/ Ekstrom, R. B., French, J. W., & Harman, H. H. (1979). Cognitive factors: Their identification and replication. Multivariate Behavioral Research Monographs, 79 (2), 3–84. Retrieved from https://eric .ed.gov/?id=EJ220217 Epstein, D. L., & Tredici, T. J. (1973, nov). Microtropia (monofixation syndrome) in flying personnel. American Journal of Ophthalmology, 76 (5), 832–841. doi: doi:10.1016/0002-9394(73)90586-2 Field, A., Miles, J., & Field, Z. (2012). Discovering statistics using R. Sage publications. Garmiz, I. V., Manichev, S. A., & Mironov, Y. A. (1988). The readability of the map: Theoretical principles. Mapping Sciences and Remote Sensing, 25 (4), 276-283. doi: doi:10.1080/07493878.1988.10641728 Golding, J. F. (1998). Motion sickness susceptibility questionnaire revised and its relationship to other forms of sickness. Brain Research Bulletin, 47 (5), 507–516. doi: doi:https://doi.org/10.1016/S0361- 9230(98)00091-4 Golledge, R. (2003). Reflections on recent cognitive behavioural research with an emphasis on research in the United States of America. Australian Geographical Studies, 41 (2), 117–130. doi: doi:10.1111/1467- 8470.00199 Grafarend, E. W., & Krumm, F. W. (2006). Map Projections: Cartographic Information Systems. Springer. Green, C., & Bavelier, D. (2007). Action-video-game experience alters the spatial resolution of vision. Psychological Science, 18 (1), 88–94. doi: doi:10.1111/j.1467-9280.2007.01853.x Harrie, L., Stigmar, H., & Djordjevic, M. (2015). Analytical estimation of map readability. ISPRS Interna- tional Journal of Geo-Information, 4 (2), 418–446. doi: doi:10.3390/ijgi4020418 Harrower, M., & Brewer, C. A. (2003, jun). ColorBrewer.org: An online tool for selecting colour schemes for maps. The Cartographic Journal, 40 (1), 27–37. doi: doi:10.1179/000870403235002042 Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In (pp. 139–183). doi: doi:10.1016/S0166-4115(08)62386-9 Hegarty, M., Montello, D. R., Richardson, A. E., Ishikawa, T., & Lovelace, K. (2006). Spatial abilities at different scales: Individual differences in aptitude-test performance and spatial-layout learning. Intelligence, 34 (2), 151–176. doi: doi:10.1016/j.intell.2005.09.005 Hegarty, M., Richardson, A. E., Montello, D. R., Lovelace, K., & Subbiah, I. (2002). Development of a self- report measure of environmental spatial ability. Intelligence, 30 (5), 425–447. doi: doi:10.1016/S0160- 2896(02)00116-2 Hollands, J., Carey, T., Matthews, M., & McCann, C. (1989). Presenting a graphical network: a comparison of performance using fisheye and scrolling views. In G. Salvendy & H. Smith (Eds.), Designing and using human-computer interfaces and knowledge-based systems (pp. 313–320). Elsevier. Horwitz, J. (2019). Oculus Rift S versus Oculus Rift: the spec comparison chart. Retrieved 2019-12-01, from https://venturebeat.com/2019/03/20/oculus-rift-s-versus-oculus-rift-the-spec-comparison-chart/ Ishikawa, T., & Kastens, K. A. (2005). Why Some Students Have Trouble with Maps and Other Spatial Rep- resentations. Journal of Geoscience Education, 53 (2), 184–197. doi: doi:10.5408/1089-9995-53.2.184 Jones, C. E. (2010). Cartographic Theory and Principles. In M. Haklay (Ed.), Interacting with geospatial technologies (pp. 37–65). Wiley. Jung, T. (n.d.). Tissot Indicatrix. Retrieved 2019-12-01, from https://map-projections.net/tissot.php Kapler, T., King, R., & Segura, D. (2019). “SkyMap”: World-scale immersive spatial display. In Symposium on spatial user interaction. Association for Computing Machinery. doi: doi:10.1145/3357251.3358749 References 57

Kapler, T., Wright, W., & Senior, R. (2013). US Patent No. US 9047699B2. U.S. Patent and Trademark Office. Kastens, K., & Ishikawa, T. (2006). Spatial thinking in the geosciences and cognitive sciences: A cross- disciplinary look at the intersection of the two fields. Special Paper of the Geological Society of America, 413 (413), 53–76. doi: doi:10.1130/2006.2413(05) Kennedy, R. S., Lane, N. E., Berbaum, K. S., & Lilienthal, M. G. (1993). Simulator sickness question- naire: An enhanced method for quantifying simulator sickness. The International Journal of Aviation Psychology, 3 (3), 203–220. doi: doi:10.1207/s15327108ijap0303 3 Khan, N., & Rahman, A. U. (2018, dec). Rethinking the mini-map: A navigational aid to support spatial learning in urban game environments. International Journal of Human–Computer Interaction, 34 (12), 1135–1147. doi: doi:10.1080/10447318.2017.1418804 Leshed, G., Velden, T., Rieger, O., Kot, B., & Sengers, P. (2008). In-car GPS navigation: Engage- ment with and disengagement from the environment. In Proceedings of the SIGCHI conference on human factors in computing systems (p. 1675–1684). Association for Computing Machinery. doi: doi:10.1145/1357054.1357316 Leung, Y. K., & Apperley, M. D. (1994, jun). A review and taxonomy of distortion-oriented pre- sentation techniques. ACM Transactions on Computer-Human Interaction, 1 (2), 126–160. doi: doi:10.1145/180171.180173 Levine, M. (1982). You-are-here maps: Psychological Considerations. Environment and Behavior, 14 (2), 221–237. doi: doi:10.1177/0013916584142006 Liben, L. S. (2009, dec). The Road to Understanding Maps. Current Directions in Psychological Science, 18 (6), 310–315. doi: doi:10.1111/j.1467-8721.2009.01658.x Liben, L. S., & Titus, S. J. (2012). The importance of spatial thinking for geoscience education: In- sights from the crossroads of geoscience and cognitive science. In Earth and Mind II: A Synthe- sis of Research on Thinking and Learning in the Geosciences. Geological Society of America. doi: doi:10.1130/2012.2486(10) Linn, M. C., & Petersen, A. C. (1985). Emergence and characterization of sex differences in spatial ability: A meta-analysis. Child Development, 56 (6), 1479–1498. doi: doi:10.2307/1130467 Lloyd, R., & Bunch, R. (2008). Explaining map-reading performance efficiency: Gender, memory, and geographic information. Cartography and Geographic Information Science, 35 (3), 171–202. doi: doi:10.1559/152304008784864677 Lloyd, R., & Bunch, R. L. (2003). Technology and map-learning: Users, methods, and symbols. Annals of the Association of American Geographers, 93 (4), 828–850. doi: doi:10.1111/j.1467-8306.2003.09304004.x Lobben, A. K. (2004). Tasks, strategies, and cognitive processes associated with navigational map reading: A review perspective. The Professional Geographer, 56 (2), 270-281. Retrieved from https://www.tandfonline.com/doi/abs/10.1111/j.0033-0124.2004.05602010.x doi: doi:10.1111/j.0033- 0124.2004.05602010.x Long, L. O., Gomer, J. A., Wong, J. T., & Pagano, C. C. (2011). Visual spatial abilities in uninhabited ground vehicle task performance during teleoperation and direct line of sight. Presence: Teleoperators and Virtual Environments, 20 (5), 466–479. doi: doi:10.1162/PRES a 00066 Longley, P. A., Goodchild, M. F., Maguire, D. J., & Rhind, D. W. (2005). Geographical information systems and science (2nd ed.). John Wiley & Sons. Maeda, Y., & Yoon, S. Y. (2013). A meta-analysis on gender differences in mental rotation ability measured References 58

by the purdue spatial visualization tests: Visualization of rotations (PSVT:R). Educational Psychology Review, 25 (1), 69–94. doi: doi:10.1007/s10648-012-9215-x Martin-Gonzalez, A., Heining, S.-M., & Navab, N. (2009). Head-mounted virtual loupe with sight-based activation for surgical applications. In 2009 8th IEEE international symposium on mixed and augmented reality (pp. 207–208). IEEE. doi: doi:10.1109/ISMAR.2009.5336459 Mayyasi, A. (2016). Why Does a Tire Company Publish the Michelin Guide? Retrieved 2019-11-28, from https://priceonomics.com/why-does-a-tire-company-publish-the-michelin-guide/ McKenzie, G., & Klippel, A. (2016). The interaction of landmarks and map alignment in you-are-here maps. The Cartographic Journal, 53 (1), 43–54. doi: doi:10.1179/1743277414Y.0000000101 Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE Trans. Information Systems, vol. E77-D, no. 12 , 1321-1329. Retrieved from https://search.ieice.org/bin/summary.php ?id=e77-d 12 1321 Montello, D. R. (2010). You are where? The function and frustration of you-are-here (YAH) maps. Spatial Cognition and Computation, 10 (2-3), 94–104. doi: doi:10.1080/13875860903585323 Montello, D. R., Fabrikant, S. I., & Davies, C. (2018). Cognitive perspectives on cartography and other geographic information visualizations. In D. Montello (Ed.), Handbook of behavioral and cognitive geography (pp. 177–196). Edward Elgar Publishing. Montello, D. R., Hegarty, M., Richardson, A. E., & Waller, D. (2004). Spatial Memory of Real Environments, Virtual Environments, and Maps. In Human Spatial Memory: Remembering Where (pp. 272–307). Lawrence Erlbaum Associates. M¨unzer,S., Zimmer, H. D., & Baus, J. (2012). Navigation assistance: A trade-off between wayfinding support and configural learning support. Journal of experimental psychology: applied, 18 (1), 18–37. doi: doi:https://doi.org/10.1037/a0026553 O’Connor, A. R., & Tidbury, L. P. (2018). Stereopsis: are we assessing it in enough depth? Clinical & experimental optometry, 101 (4), 485–494. doi: doi:10.1111/cxo.12655 Pak, R., Rogers, W. A., & Fisk, A. D. (2006). Spatial ability subfactors and their influences on a computer- based information search task. Human Factors: The Journal of the Human Factors and Ergonomics Society, 48 (1), 154–165. doi: doi:10.1518/001872006776412180 Park, J., & Park, W. (2019). A review on the interface design of automotive head-up displays for communi- cating safety-related information. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 63 (1), 2016-2017. doi: doi:10.1177/1071181319631099 Qiu, Y., Wu, Y., Liu, R., Wang, J., Huang, H., & Huang, R. (2019). Representation of human spatial naviga- tion responding to input spatial information and output navigational strategies: An ALE meta-analysis. Neuroscience and Biobehavioral Reviews, 103 , 60–72. doi: doi:10.1016/j.neubiorev.2019.06.012 Ragbar, M. (2017). Evaluation of a novel display concept for minimally invasive surgery (Master’s thesis, University of Toronto). Retrieved from http://hdl.handle.net/1807/80181 Reiner, A. J., Hollands, J. G., Jamieson, G. A., & Boustila, S. (2019). A mirror in the sky: Assessment for an augmented reality method for depicting navigational information. Manuscript submitted for publication. Robinson, A., & Snyder, J. P. (Eds.). (1997). Mapping the Projection to the Need. Bethesda: Amer- ican Congress on Surveying and Mapping. Retrieved from https://courseware.e-education.psu.edu/ projection/ Slocum, T. A., Blok, C., Jiang, B., Koussoulakou, A., Montello, D. R., Fuhrmann, S., & Hedley, N. R. References 59

(2001). Cognitive and usability issues in geovisualization. Cartography and Geographic Information Science, 28 (1), 61–75. doi: doi:10.1559/152304001782173998 Slocum, T. A., McMaster, R., Kessler, F., & Howard, H. (1999). Thematic cartography and visualization. Prentice Hall. Smith, M., Gabbard, J. L., Burnett, G., & Doutcheva, N. (2017, apr). The effects of augmented reality head-up displays on drivers’ eye scan patterns, performance, and perceptions. International Journal of Mobile Human Computer Interaction, 9 (2), 1–17. doi: doi:10.4018/IJMHCI.2017040101 Snyder, J. P. (1987). Map projections–a working manual (Vol. 1395). US Government Printing Office. Spector, P. E. (1992). Summated rating scale construction: An introduction. Sage Publications. Tahir, A., McArdle, G., & Bertolotto, M. (2012). Identifying specific spatial tasks through clustering and geovisual analysis. In 2012 20th international conference on geoinformatics (pp. 1–6). IEEE. doi: doi:10.1109/Geoinformatics.2012.6270301 Taylor, H. A., Bruny´e, T. T., & Taylor, S. T. (2008, oct). Spatial Mental Representation: Implica- tions for Navigation System Design. Reviews of Human Factors and Ergonomics, 4 (1), 1–40. doi: doi:10.1518/155723408X342835 Thorndyke, P. W., & Hayes-Roth, B. (1982). Differences in spatial knowledge acquired from maps and navigation. Cognitive Psychology, 14 (4), 560–589. doi: doi:10.1016/0010-0285(82)90019-6 Tidbury, L., Brooks, K., O’Connor, A., & Wuerger, S. (2016). A systematic comparison of static and dynamic cues for depth perception. Investigative Ophthalmology and Visual Science, 57 (8), 3545–3553. doi: doi:10.1167/iovs.15-18104 Tr´es,E. S., & Brucki, S. M. D. (2014). Visuospatial processing: A review from basic to current con- cepts. Dementia & neuropsychologia, 8 (2), 175–181. doi: doi:http://dx.doi.org/10.1590/S1980- 57642014DN82000014 Tufte, E. R. (1983). The visual display of quantitative information. Graphics Press. Wickens, C., Hollands, J. G., Banbury, S., & Parasuraman, R. (2015). Engineering Psychology and Human Performance (4th ed.). Psychology Press. doi: doi:10.4324/9781315665177 Williams, H. P., & Wickens, C. (1991). Electronic map interpretation in a dual-task context. In Proceedings of the human factors society (Vol. 1, pp. 92–96). doi: doi:10.1518/107118191786755805 Willits, F., Theodori, G., & Luloff, A. (2016). Another look at likert scales. Journal of Rural Social Sciences, 31 (3). Wolbers, T., & Hegarty, M. (2010). What determines our navigational abilities? Trends in Cognitive Sciences, 14 (3), 138–146. doi: doi:10.1016/j.tics.2010.01.001 Yorke, D. A., & Margolies, J. (1996). Hitting the road: The art of the american road map. Chronicle Books. Zeelenberg, R., & Pecher, D. (2015). A method for simultaneously counterbalancing condition order and assignment of stimulus materials to conditions. Behavior Research Methods, 47 (1), 127–133. doi: doi:10.3758/s13428-014-0476-9 Zhong, J. Y., & Kozhevnikov, M. (2016). Relating allocentric and egocentric survey-based representations to the self-reported use of a navigation strategy of egocentric spatial updating. Journal of Environmental Psychology, 46 , 154–175. doi: doi:https://doi.org/10.1016/j.jenvp.2016.04.007 Appendix A

Details of Implementation of Inverse Polar Azimuthal Projections

This appendix provides the mathematical details of the dome projection implementations. The concepts are introduced and described at a high level in Section 3.2.1.

A.1 Dome Mesh Parametrization

The dome mesh is comprised of vertices, whose locations were defined as a set of concentric circles of decreasing radius (from the full dome radius, R, at height 0, to radius 0 — i.e. a point — at full dome height). These vertices are defined using polar coordinates, such that they are equally spaced along the azimuth (θ) and elevation (ϕ) dimensions. This mesh is shown in Figure 3.3a. Each of the mesh vertices’ (x, z, y)1 coordinates are parametrized using the formulae for transformation of spherical coordinates: x = R sin ϕ cos θ z = R sin ϕ sin θ y = R cos ϕ

1Note: in Unity, x, y, and z are defined relative to the observer as the right-, upward-, and forward-pointing directions, respectively.

60 Appendix A. Details of Implementation of Inverse Polar Azimuthal Projections 61

A.2 Defining the “Inverse” Cartographic Projections

To differentiate the inverse projections, it is useful to first define graticules, which are “lines showing parallels of latitude and meridians of longitude for the Earth” (ArcGIS, n.d.). Graticules are drawn on flat maps such that one can visualize intervals which would be equally spaced over the spherical surface. The geometry of a specific projection describes how the graticules (and, of course, the map image) are shaped and distributed over the flat surface. It then follows that, by using the equations of a particular projection to define the texture coordinates (in a sense, defining the graticule lines on the flat map), the geometric properties of the flat image that are preserved by the projection will be preserved on the dome mesh. For the case of polar azimuthal projections, the graticule lines take on a specific shape: meridians of longitude appear as straight lines through the centre, and parallels of latitude appear as concentric circles, ranging in radius r from r = R to r = 0 (R being the full dome radius) (See Snyder quote, Section 2.3.2). Let the texture coordinates, which are implemented as Cartesian coordinates (u, v), be instead expressed in 2-dimensional polar coordinates (r, α). Note that when limiting the scope to polar azimuthal projec- tions, the azimuthal coordinate θ on the sphere = the polar coordinate α on the render texture. From the simplifications explained above, the equations for the graticules can be expressed as follows: " # " # " # " # u r cos α f(ϕ) cos α cos α = = = f(ϕ) v r sin α f(ϕ) sin α sin α where f(ϕ) defines how the radii r of the concentric circles of latitude decrease from R to 0. Table A.1 shows the definition of f(ϕ) required for each projection2.

Projection f(ϕ) Orthographic R cos ϕ Conformal 2Rsin(π/2 − ϕ/2) Equidistant R(π/2 − ϕ)

Table A.1: Functions governing projection geometry for domed SkyMap conditions

A.2.1 Implementation of the Offset Dome Projection

The offset dome condition is constructed by applying a displacement to the texture coordinates for the equidistant projection. Points were shifted such that the centre of the map (that would have otherwise been at the top of the dome) is moved forward by (half of the map radius) × (the proportion of the height of where the point originally would have been on the dome). As such, the centre point of the map (the self-marker) is maximally shifted, and then projected to be halfway between the dome’s peak and its horizon (which is at a 45◦ elevation, since the base projection is equidistant). All other points experience a smaller forward shift, with points at the dome base (height = 0) not shifted at all.

2Note that scaling factors had to be applied to the stereographic and equidistant projections in order to maintain alignment at the horizon line, one of the constraints of SkyMap. Appendix A. Details of Implementation of Inverse Polar Azimuthal Projections 62

A.3 Visualization and Quantification of Projection-Induced Dis- tortion

Table A.2 below shows how an image of equally-spaced grid lines and circles appear when shown on the different dome projections used in the experiment.3 The reference image and all domes shown have a radius of 600 m.

Reference image: a (100 m)2 flat grid with perfect circles 60 m in diameter Conformal Projection Orthographic Equidistant (Stereographic)

External view

Internal view

Circles closer to the hori- Circle shape preserved over Circles appear to all have Description zon appear as if they were the dome surface* the same height stretched tall

* Note: true appearance of dome from observer’s perspective cannot be properly visualized via a 2D image, which may explain discrepancy from description.

Table A.2: Visualization of distortion induced by the different map conditions

3These visualizations are commonly used in cartography, where they are know as distortion ellipses, or Tissot’s indicatrix (Jung, n.d.). Appendix A. Details of Implementation of Inverse Polar Azimuthal Projections 63

Table A.3 below shows the scaling values along the meridians of longitude (height, from the observer’s view) and parallels of latitude (width, from the observer’s view), h and k respectively, for each of the dome projections at different elevation angles.

Orthographic Conformal Equidistant ϕ h k h k h k pole → 90◦ 1 1 2 2 1.571 1.571 80◦ 1.015 1 1.985 1.985 1.571 1.563 70◦ 1.064 1 1.940 1.940 1.571 1.539 60◦ 1.155 1 1.866 1.866 1.571 1.5 50◦ 1.305 1 1.766 1.766 1.571 1.446 40◦ 1.556 1 1.643 1.643 1.571 1.379 30◦ 2 1 1.5 1.5 1.571 1.299 20◦ 2.924 1 1.342 1.342 1.571 1.208 10◦ 5.759 1 1.174 1.174 1.571 1.108 horizon → 0◦ ∞ 1 1 1 1.571 1

Table A.3: Vertical and horizontal scaling factors h and k for the three dome projections used

A.3.1 Figures showing the gnomonic projection’s visual equivalence to a flat SkyMap

Figure A.1 below shows how the image of equally-spaced grid lines and circles from Table A.2 appear when shown on the gnomonic projection. Recall that the gnomonic projection’s notable property is that it visualizes arcs on a sphere surface as straight lines (Snyder, 1987), so it takes the straight grid lines and projects them onto the dome in a way that makes them appear as straight lines again.

(b) (a)

Figure A.1: Images of the dome with the gnomonic projection as seen (a) from a distance, and (b) from the centre. Note that this projection is not defined for the entirety of the hemisphere, so the image is “stretched” over that area down to the horizon line. Appendix B

Experiment Forms and Questionnaires

64 Appendix B. Experiment Forms and Questionnaires 65

B.1 Informed Consent Form

INFORMED CONSENT FORM Principal Investigator: Catherine Solis ​ Faculty supervisor: Prof. Paul Milgram ​ Affiliation: Department of Mechanical and Industrial Engineering, University of Toronto ​ Please read all information on this form and provide acknowledgement and consent in section VII. ​ Thank you for considering participating in this study. The purpose of this study is to investigate how people use a novel map display to estimate spatial relationships in a Virtual Reality (VR) environment. This research is sponsored by the Natural Sciences and Engineering Research Council (NSERC) in cooperation with the Department of National Defence, and Uncharted Software Inc.

I. Procedure Participation in this study will take approximately two hours. If you agree to participate and sign this form, you will complete a pre-experiment questionnaire and two tests of spatial abilities. You will then watch a presentation introducing and explaining the experimental procedure in detail, and then be fitted with the VR headset. In the virtual environment you will view a series of outdoor urban environments, and you will also see a map display. You will be asked to indicate on the map display the location of a landmark using a VR hand controller. Once you confirm your answer, you will be teleported to a new outdoor environment with a new set of landmarks for the next trial. You will perform the experiment in blocks, consisting of several trials each. For each block there will be a different map display, and there will be a practice trial at the beginning of each block. After each trial block, you will be asked to remove the VR headset to complete a brief questionnaire about your experience. We ask that you provide honest answers and opinions. After the three trials, you will be asked some questions about the experiment, and the experimenter will take written notes. Please feel free at any time to ask any questions you may have about the experimental procedure. Your decision to participate in this study is completely voluntary. You are free to withdraw at any time without any consequences. You are not required to answer any questions you do not feel comfortable answering.

II. Risks and Benefits Some users of VR headsets may experience “simulator sickness”. Symptoms of simulator sickness are similar to motion sickness and may include discomfort, eye strain, dizziness, headaches, or nausea. You may also feel discomfort and/or fatigue due to wearing the headset for the duration of the experiment. If you experience any discomfort, fatigue or uneasiness at any time, please inform the experimenter. You may take a break at any time, and we will also offer breaks between trial blocks. You are also free to stop the experiment at any time. Beyond financial compensation (described in section V below), you will have the opportunity to experience a virtual environment, and novel map displays. We hope you find this an interesting experience. Your participation in this study may contribute to our understanding of how different map displays can support spatial awareness. Our industry sponsor, Uncharted Software, may also benefit from this research by better understanding users’ ability to comprehend information from map displays.

1 of 3

Appendix B. Experiment Forms and Questionnaires 66

III. Extent of Privacy and Confidentiality Your privacy and identity will be protected in this study. In all data files, your name and personal identification will be removed and replaced with a code in order to preserve confidentiality. Paper tests will be scanned following your participation, after which all paper records will be securely shredded. All digital data will be held in password-protected files for seven years. Only the investigators and future researchers within the investigators’ lab will have access to the data files. If you choose to withdraw from participation, you may also request to have your data destroyed. Data collected may be used in research journals, conferences, or other scholarly activities. In any publication, information will be provided in such a way that you cannot be personally identified.

IV. Acknowledgement of Confidentiality of Technology Some of the technology that you will experience in this experiment has been developed at significant expense by our Canadian industrial partner, who clearly wishes to retain its competitive advantage. In addition, this project is but one part of a multi-year effort, such that revealing details of the present experiment may impact the results of future experiments. Consequently, you are asked to acknowledge our preference that details of the technology encountered in this experiment not be disseminated, either via email, social media, or other means. ​ ​

V. Compensation You will be paid at a rate of $15.00 per hour, rounded to the nearest 20 minutes. If you withdraw from the study, you will still be provided this compensation. Should you withdraw due to simulator sickness, you will be compensated the greater of $20 or $15/hour of study time rounded to the next 20 minutes. If you complete the study, you will also be entered into a draw for $25. You will have a minimum of one entry into the draw, but may receive more entries based on your overall performance on the experimental task. The winner of the draw will be contacted after the study is complete.

VI. Approval of Research The research study you are participating in has been approved by the Research Ethics Board at the University of Toronto. It may be reviewed for quality assurance to make sure that the required laws and guidelines are followed. In that case, representatives of the Human Research Ethics Program (HREP) may access study-related data and/or consent materials as part of the review. All information accessed by the HREP will be held to the same level of confidentiality that has been stated by the research team. If you have any questions about your rights as a participant, please contact the Ethics Review Office at [email protected] or (416) 946-3273.

Catherine Solis is undertaking this study in fulfilment of her MASc degree requirements. If you have any additional questions afterwards, about this study or about any publications arising from this work, please contact Ms. Solis by email ([email protected]).

2 of 3

Appendix B. Experiment Forms and Questionnaires 67

VII. Participant’s Acknowledgement and Consent

I acknowledge the following (please check all below): ☐ I am at least 18 years of age, and no older than 35; ​ ☐ I will be asked to participate in an experiment requiring me to wear a virtual reality headset and interact with a ​ virtual environment; ☐ I can expect the experiment to last approximately two hours; ​ ☐ I will be compensated for my participation at a rate of $15/hour, and entered into a draw to win $25 if I complete ​ the study; ☐ I am free to withdraw from the study at any time, upon which the experiment will end at once, and at my request ​ all respective data collected insofar will be destroyed; ☐ I have completed the screening questionnaire with accurate information, including indication that I do not have a ​ propensity to motion sickness; ☐ I understand the existence of risks associated with using VR indicated on the previous page; ​ ☐ All data collected in this study will be kept secure through the methods described above; ​ ☐ All published information using the data I provide will exclude any personally identifying information; ​ ☐ The experimenter prefers that details of the technology encountered in this experiment not be shared in any way; ​ ​ ​ ☐ I am free to ask questions about the process at any time. I can ask questions in person at the time of the ​ experiment, or afterwards by contacting Catherine Solis (information provided above)

By signing below, I acknowledge the above and give my voluntary consent.

Participant’s Printed Name: ______

Participant’s Signature: ______

To be completed by experimenter:

Date: ______

Participant ID:______

Researcher’s initials: ______

3 of 3

Appendix B. Experiment Forms and Questionnaires 68

B.2 Pre-Experiment Questionnaires

B.2.1 Demographics Questions Appendix B. Experiment Forms and Questionnaires 69

B.2.2 General Video Game/VR Experience Questions Appendix B. Experiment Forms and Questionnaires 70

B.2.3 Action Video Game Experience Questions

B.2.4 Sense of Direction and Navigational Strategies Questionnaires

The final two sections of the pre-experiment questionnaire consisted of the Santa Barbara Sense of Direction Scale (Hegarty et al., 2002), and the Navigational Strategies Questionnaire (Brunec et al., 2018), with questions in the online form presented, and eventually scored, as prescribed in the original publications. Appendix B. Experiment Forms and Questionnaires 71

B.2.5 Spatial Tests

Two spatial tests were administered on paper from the Kit of Factor-Referenced Cognitive Tests (1976 edition) (Ekstrom, French, & Harman, 1976): VZ-2, the Paper Folding test, and S-2, the Cube Comparisons test. The Paper Folding test is a multiple-choice, 20-item test. Each question consists of a set of diagrams, depicting a square piece of paper being folded and then hole-punched. The respondent must select the correct image of what that paper would look like unfolded (i.e. where the holes would be). The Cube Comparisons test is a binary-response, 42-item test. Each question consists of two images of cubes with symbols printed on their faces (and they are to assume that symbols on the cube faces are unique). The respondent must determine whether the two images could be of the same cube, just rotated, or whether the orientations of the symbols mean that the cubes are necessarily distinct. Scoring was done as prescribed in the manual (Ekstrom, Dermen, & Harman, 1976), with the VZ-2 score 1 tabulated as (# correct) - (# incorrect), and the S-2 score tabulated as (# correct) - 4 (# incorrect).

B.3 Post-Trial Questionnaire: NASA-TLX

The NASA Task Load Index (Hart & Staveland, 1988) was administered as an online form, with the ratings done between each block immediately after completion. The form was modified from the original 20-point scale presented in the original questionnaire to instead have a seven-point scale (see below). The ratings questionnaire was administered six times in total. Appendix B. Experiment Forms and Questionnaires 72 Appendix B. Experiment Forms and Questionnaires 73

After providing ratings for the final block, the relative workload comparisons were administered and participants were asked to have it reflect the experimental task over all blocks.

B.4 Post-Experiment Questionnaire: SSQ

A section was appended to the NASA-TLX online form (see previous section) that included the simulator sickness questionnaire (Kennedy et al., 1993), with questions presented, and eventually scored, as prescribed in the original publication. Appendix B. Experiment Forms and Questionnaires 74

B.5 Experiment Debrief

A semi-structured interview followed the final questionnaire (see previous section). The questions that provided the template for interviews are shown below.

1. Which display did you prefer: the “mini” map, the flat SkyMap or one of the dome SkyMaps? Why?

2. Which display did you find easiest to understand? Why?

3. What was your strategy for the task?

4. Did your strategy differ with the different map displays? If so, how?

5. Did you use the red self-marker to help you? For all map displays? What was it helpful for?

6. Would you use SkyMap (flat and/or dome) for navigating if it were commercially available? Why or why not?

7. Do you have thoughts or feedback on any of the displays, or on the experiment, that you haven’t already mentioned? Appendix C

Supplementary Information & R Output from Data Analyses

C.1 Analyses of block-aggregated data (Section 4.2.1)

Descriptive Statistics of all AE data

nbr.val nbr.null nbr.na min max range sum 4.400000e+03 0.000000e+00 0.000000e+00 8.488628e-01 8.497486e+02 8.488997e+02 3.387203e+05 median mean SE.mean CI.mean.0.95 var std.dev coef.var 5.796558e+01 7.698188e+01 1.093390e+00 2.143596e+00 5.260211e+03 7.252732e+01 9.421350e-01 skewness skew.2SE kurtosis kurt.2SE normtest.W normtest.p 2.780580e+00 3.766206e+01 1.251075e+01 8.474624e+01 7.607495e-01 2.441262e-62

Descriptive statistics of MAE data, grouped by map condition

AvgData2$Map.condition: HeadsDown median mean SE.mean CI.mean.0.95 var std.dev coef.var 7.180829e+01 7.964418e+01 5.351405e+00 1.086393e+01 1.030951e+03 3.210843e+01 4.031485e-01 skewness skew.2SE kurtosis kurt.2SE normtest.W normtest.p 1.713311e+00 2.182317e+00 3.094293e+00 2.014314e+00 8.254566e-01 5.441506e-05 ------AvgData2$Map.condition: Flat SkyMap median mean SE.mean CI.mean.0.95 var std.dev coef.var 57.0355996 56.1763468 3.4953264 7.0888505 452.0403501 21.2612406 0.3784732 skewness skew.2SE kurtosis kurt.2SE normtest.W normtest.p 0.2995365 0.3864095 -0.5625384 -0.3707160 0.9815205 0.7832033 ------

75 Appendix C. Supplementary Information & R Output from Data Analyses 76

AvgData2$Map.condition: DomeOrtho median mean SE.mean CI.mean.0.95 var std.dev coef.var 6.969590e+01 9.061181e+01 9.763826e+00 1.982162e+01 3.431963e+03 5.858296e+01 6.465268e-01 skewness skew.2SE kurtosis kurt.2SE normtest.W normtest.p 1.751169e+00 2.230538e+00 2.455467e+00 1.598453e+00 7.708636e-01 4.517325e-06 ------AvgData2$Map.condition: DomeConf median mean SE.mean CI.mean.0.95 var std.dev coef.var 54.2747954 60.6679193 4.1178099 8.3513055 627.3852503 25.0476596 0.4128650 skewness skew.2SE kurtosis kurt.2SE normtest.W normtest.p 0.6172150 0.7962227 -0.2537001 -0.1671898 0.9526555 0.1175256 ------AvgData2$Map.condition: DomeEqD median mean SE.mean CI.mean.0.95 var std.dev coef.var 5.731990e+01 6.595379e+01 5.105798e+00 1.036532e+01 9.384902e+02 3.063479e+01 4.644887e-01 skewness skew.2SE kurtosis kurt.2SE normtest.W normtest.p 1.721112e+00 2.192254e+00 2.913460e+00 1.896596e+00 8.009958e-01 1.704590e-05 ------AvgData2$Map.condition: DomeOffset median mean SE.mean CI.mean.0.95 var std.dev coef.var 9.120239e+01 1.093034e+02 1.183141e+01 2.399521e+01 5.179341e+03 7.196764e+01 6.584209e-01 skewness skew.2SE kurtosis kurt.2SE normtest.W normtest.p 2.428010e+00 3.132194e+00 6.063785e+00 3.996069e+00 6.962650e-01 1.881744e-07

Plots comparing transformed MAE data to the normal distribution

(a) (b)

Figure C.1: (a) Histogram and (b) quantile-quantile plot of transformed MAE data Appendix C. Supplementary Information & R Output from Data Analyses 77

Summary output from multilevel model to predict MAE by map condition

Linear mixed-effects model fit by maximum likelihood Data: AvgData2 AIC BIC logLik -149.4308 -118.9291 83.71538

Random effects: Formula: ~1 | PID (Intercept) StdDev: 0.1564668

Formula: ~1 | Map.condition %in% PID (Intercept) Residual StdDev: 0.1254392 0.05664809

Fixed effects: delta5rt ~ Map.condition Value Std.Error DF t-value p-value (Intercept) 2.3357034 0.027753100 177 84.16009 0.0000 Map.conditioncontrastTU -0.0081677 0.004245570 177 -1.92382 0.0560 Map.conditioncontrastFlat 0.0290970 0.005141304 177 5.65945 0.0000 Map.conditioncontrastUnch -0.0476760 0.006646751 177 -7.17283 0.0000 Map.conditioncontrastOrtho -0.0476096 0.009480281 177 -5.02196 0.0000 Map.conditioncontrastConfEqd 0.0190975 0.016365691 177 1.16692 0.2448 Correlation: (Intr) Mp.cTU Mp.cnF Mp.cnU Mp.cnO Map.conditioncontrastTU -0.004 Map.conditioncontrastFlat 0.003 0.000 Map.conditioncontrastUnch 0.004 0.001 0.005 Map.conditioncontrastOrtho -0.003 0.000 -0.004 -0.005 Map.conditioncontrastConfEqd 0.005 0.001 0.007 0.009 0.006

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -0.94794740 -0.24293593 -0.01547054 0.20863198 1.57777831

Number of Observations: 219 Number of Groups: PID Map.condition %in% PID 37 219 Appendix C. Supplementary Information & R Output from Data Analyses 78

Output from Tukey’s HSD post hoc tests for MAE model

Simultaneous Tests for General Linear Hypotheses

Multiple Comparisons of Means: Tukey Contrasts

Fit: lme.formula(fixed = delta5rt ~ Map.condition, data = AvgData2, random = ~1 | PID/Map.condition, method = "ML")

Linear Hypotheses: Estimate Std. Error z value Pr(>|z|) DomeEqd - DomeConf == 0 0.03819 0.03228 1.183 0.84506 DomeOrtho - DomeConf == 0 0.16193 0.03228 5.016 < 0.001 *** FlatMitS - DomeConf == 0 -0.03110 0.03200 -0.972 0.92690 TrackUp - DomeConf == 0 0.13429 0.03228 4.160 < 0.001 *** Uncharted - DomeConf == 0 0.25741 0.03200 8.044 < 0.001 *** DomeOrtho - DomeEqd == 0 0.12373 0.03244 3.814 0.00188 ** FlatMitS - DomeEqd == 0 -0.06930 0.03228 -2.147 0.26318 TrackUp - DomeEqd == 0 0.09610 0.03244 2.962 0.03612 * Uncharted - DomeEqd == 0 0.21922 0.03228 6.791 < 0.001 *** FlatMitS - DomeOrtho == 0 -0.19303 0.03228 -5.980 < 0.001 *** TrackUp - DomeOrtho == 0 -0.02763 0.03244 -0.852 0.95760 Uncharted - DomeOrtho == 0 0.09548 0.03228 2.958 0.03668 * TrackUp - FlatMitS == 0 0.16539 0.03228 5.124 < 0.001 *** Uncharted - FlatMitS == 0 0.28851 0.03200 9.016 < 0.001 *** Uncharted - TrackUp == 0 0.12312 0.03228 3.814 0.00191 ** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 (Adjusted p values reported -- single-step method) Appendix C. Supplementary Information & R Output from Data Analyses 79

C.1.1 Order Effects in MAE Data

To check whether order effects influenced performance measures, the variable of block index (indicating that the block was the first one seen, or the second, etc.) was added to the model to see whether it improved. Table C.1 below shows the goodness-of-fit metrics for the models for the MAE data.

Model Formula df AIC BIC logLik L. ratio 0 MAE ∼ 1 4 -68.6 -55.1 38.32 1 MAE ∼ ... + Map condition 9 -149.4 -118.9 83.72 90.78*** 2 MAE ∼ ... + block index 10 -154.1 -120.2 87.04 6.65*** 3 MAE ∼ ... + map condition:block index 15 -152.1 -101.3 91.06 8.04 (n.s.) ***: p < .001

Table C.1: Goodness-of-fit metrics of models which include block order effects

The inclusion of block index as a predictor did improve the model for MAE scores, but adding the interaction effect did not improve it (thus reported statistics are for Formula 2 in Table C.1, with no interaction term). The model showed that accuracy increased in later blocks, b = -0.014, t(176) = -2.56, p = .01, r = .19. Nonetheless, the inclusion of the block index term did not affect the statistical significance of the comparisons of MAE scores between map conditions.

C.1.2 Checking for stimulus set effects in MAE data

To check whether the stimulus set (the subset of the 144 unique trials) influenced performance measures, the variable of block ID (with labels A through F) was added to the model above to see whether it improved. Table C.2 shows the goodness-of-fit metrics for the models for the MAE data.

Model Formula df AIC BIC logLik L. ratio 2 MAE ∼ Map condition + block index 10 -154.1 -120.2 87.04 4 MAE ∼ ... + block ID 15 -154.3 -103.4 92.13 10.17 (n.s.)

Table C.2: Goodness-of-fit metrics of model with addition of stimulus set effects

The inclusion of block ID as a predictor did not improve the model for MAE scores. Further modelling with all interaction terms (not reported here) indicated that inclusion of further interaction terms would not improve the model either. Thus, it is reasonable to conclude that the six stimulus sets did not differ in a way that would influence experiment outcomes. Appendix C. Supplementary Information & R Output from Data Analyses 80

Descriptive statistics of MVAE data, grouped by map condition

AvgData2$Map.condition: HeadsDown nbr.val nbr.null nbr.na min max range sum 36.0000000 0.0000000 0.0000000 3.2525068 14.0458923 10.7933855 339.0577323 median mean SE.mean CI.mean.0.95 var std.dev coef.var 9.8793237 9.4182703 0.5050828 1.0253725 9.1839097 3.0304966 0.3217679 skewness skew.2SE kurtosis kurt.2SE normtest.W normtest.p -0.3570739 -0.4548203 -1.0187994 -0.6632151 0.9533388 0.1331996 ------AvgData2$Map.condition: Flat SkyMap nbr.val nbr.null nbr.na min max range sum 3.700000e+01 0.000000e+00 0.000000e+00 4.144264e+00 2.111728e+01 1.697301e+01 3.488176e+02 median mean SE.mean CI.mean.0.95 var std.dev coef.var 8.532584e+00 9.427503e+00 5.464280e-01 1.108207e+00 1.104759e+01 3.323792e+00 3.525633e-01 skewness skew.2SE kurtosis kurt.2SE normtest.W normtest.p 1.662715e+00 2.144944e+00 3.198099e+00 2.107565e+00 8.305655e-01 5.716018e-05 ------AvgData2$Map.condition: DomeOrtho nbr.val nbr.null nbr.na min max range sum 3.600000e+01 0.000000e+00 0.000000e+00 7.333877e+00 5.078242e+01 4.344855e+01 4.553489e+02 median mean SE.mean CI.mean.0.95 var std.dev coef.var 1.028406e+01 1.264858e+01 1.324104e+00 2.688074e+00 6.311703e+01 7.944623e+00 6.281039e-01 skewness skew.2SE kurtosis kurt.2SE normtest.W normtest.p 3.532805e+00 4.499885e+00 1.308087e+01 8.515344e+00 5.165812e-01 9.855633e-10 ------AvgData2$Map.condition: DomeConf nbr.val nbr.null nbr.na min max range sum 3.700000e+01 0.000000e+00 0.000000e+00 4.390527e+00 2.324181e+01 1.885129e+01 3.861736e+02 median mean SE.mean CI.mean.0.95 var std.dev coef.var 9.771493e+00 1.043712e+01 5.626826e-01 1.141173e+00 1.171463e+01 3.422664e+00 3.279318e-01 skewness skew.2SE kurtosis kurt.2SE normtest.W normtest.p 1.407609e+00 1.815851e+00 3.371025e+00 2.221524e+00 8.932680e-01 1.926076e-03 ------AvgData2$Map.condition: DomeEqD nbr.val nbr.null nbr.na min max range sum 3.600000e+01 0.000000e+00 0.000000e+00 4.934940e+00 4.197431e+01 3.703937e+01 4.024610e+02 median mean SE.mean CI.mean.0.95 var std.dev coef.var 9.856936e+00 1.117947e+01 1.122362e+00 2.278516e+00 4.534908e+01 6.734172e+00 6.023695e-01 skewness skew.2SE kurtosis kurt.2SE normtest.W normtest.p 3.410077e+00 4.343561e+00 1.171320e+01 7.625025e+00 5.231622e-01 1.175361e-09 ------AvgData2$Map.condition: DomeOffset nbr.val nbr.null nbr.na min max range sum 37.00000000 0.00000000 0.00000000 3.46484622 13.84995683 10.38511060 314.47518752 median mean SE.mean CI.mean.0.95 var std.dev coef.var 8.07351374 8.49932939 0.33602573 0.68149177 4.17779185 2.04396474 0.24048541 skewness skew.2SE kurtosis kurt.2SE normtest.W normtest.p 0.63815795 0.82323963 1.03525672 0.68224003 0.92370511 0.01437642 Appendix C. Supplementary Information & R Output from Data Analyses 81

Plots comparing transformed MVAE data to the normal distribution

Figure C.2: Histogram and quantile-quantile plot of transformed MVAE data

Summary output from multilevel model to predict MVAE by map condition

Linear mixed-effects model fit by maximum likelihood Data: AvgData2 AIC BIC logLik -370.8754 -340.3738 194.4377

Random effects: Formula: ~1 | PID (Intercept) StdDev: 0.06417868

Formula: ~1 | Map.condition %in% PID (Intercept) Residual StdDev: 0.08171309 0.0336533

Fixed effects: deltaVA5rt ~ Map.condition Value Std.Error DF t-value p-value (Intercept) 1.5751233 0.012305970 177 127.99668 0.0000 Map.conditioncontrastTU 0.0046832 0.002725252 177 1.71843 0.0875 Map.conditioncontrastFlat 0.0065603 0.003300487 177 1.98767 0.0484 Map.conditioncontrastUnch 0.0197166 0.004266442 177 4.62131 0.0000 Map.conditioncontrastOrtho -0.0145492 0.006086103 177 -2.39057 0.0179 Map.conditioncontrastConfEqd 0.0050936 0.010503450 177 0.48494 0.6283 Correlation: (Intr) Mp.cTU Mp.cnF Mp.cnU Mp.cnO Map.conditioncontrastTU -0.005 Map.conditioncontrastFlat 0.004 0.001 Map.conditioncontrastUnch 0.005 0.001 0.005 Map.conditioncontrastOrtho -0.004 -0.001 -0.004 -0.005 Map.conditioncontrastConfEqd 0.006 0.001 0.006 0.008 0.006 Appendix C. Supplementary Information & R Output from Data Analyses 82

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -1.837366242 -0.198238445 -0.006019088 0.162364309 1.567833178

Number of Observations: 219 Number of Groups: PID Map.condition %in% PID 37 219

Output from Tukey’s HSD post hoc tests for MVAE model

Simultaneous Tests for General Linear Hypotheses

Multiple Comparisons of Means: Tukey Contrasts

Fit: lme.formula(fixed = deltaVA5rt ~ Map.condition, data = AvgData2, random = ~1 | PID/Map.condition, method = "ML")

Linear Hypotheses: Estimate Std. Error z value Pr(>|z|) Flat SkyMap - HeadsDown == 0 0.001858 0.020717 0.090 1.00000 DomeOrtho - HeadsDown == 0 0.083474 0.020829 4.008 < 0.001 *** DomeConf - HeadsDown == 0 0.034733 0.020717 1.677 0.54742 DomeEqD - HeadsDown == 0 0.044920 0.020829 2.157 0.25846 DomeOffset - HeadsDown == 0 -0.024490 0.020717 -1.182 0.84557 DomeOrtho - Flat SkyMap == 0 0.081616 0.020717 3.940 0.00121 ** DomeConf - Flat SkyMap == 0 0.032875 0.020546 1.600 0.59849 DomeEqD - Flat SkyMap == 0 0.043062 0.020717 2.079 0.29841 DomeOffset - Flat SkyMap == 0 -0.026348 0.020546 -1.282 0.79483 DomeConf - DomeOrtho == 0 -0.048741 0.020717 -2.353 0.17315 DomeEqD - DomeOrtho == 0 -0.038554 0.020829 -1.851 0.43293 DomeOffset - DomeOrtho == 0 -0.107965 0.020717 -5.211 < 0.001 *** DomeEqD - DomeConf == 0 0.010187 0.020717 0.492 0.99647 DomeOffset - DomeConf == 0 -0.059223 0.020546 -2.882 0.04544 * DomeOffset - DomeEqD == 0 -0.069411 0.020717 -3.350 0.01044 * --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 (Adjusted p values reported -- single-step method) Appendix C. Supplementary Information & R Output from Data Analyses 83

Output (summary and post hoc Tukey contrasts) from linear modelling of mean trial duration

Linear mixed-effects model fit by maximum likelihood Data: AvgData2 AIC BIC logLik -169.2971 -138.7955 93.64856

Random effects: Formula: ~1 | PID (Intercept) StdDev: 0.1873332

Formula: ~1 | Map.condition %in% PID (Intercept) Residual StdDev: 0.1136605 0.05501502

Fixed effects: timeTransformed ~ Map.condition Value Std.Error DF t-value p-value (Intercept) 1.2264221 0.03241855 177 37.83087 0.0000 Map.conditioncontrastVAOffset -0.0171478 0.00385312 177 -4.45036 0.0000 Map.conditioncontrastVAHD 0.0000277 0.00476909 177 0.00582 0.9954 Map.conditioncontrastVAFlat 0.0129004 0.00609873 177 2.11527 0.0358 Map.conditioncontrastVAOrtho -0.0256441 0.00869814 177 -2.94822 0.0036 Map.conditioncontrastVAConfEqd 0.0046822 0.01501717 177 0.31179 0.7556 Correlation: (Intr) Mp.cndtncntrstVAOf M.VAHD Mp.VAF Mp.cndtncntrstVAOr Map.conditioncontrastVAOffset 0.003 Map.conditioncontrastVAHD -0.002 -0.004 Map.conditioncontrastVAFlat 0.003 0.005 0.001 Map.conditioncontrastVAOrtho -0.002 -0.004 -0.001 -0.005 Map.conditioncontrastVAConfEqd 0.004 0.007 0.002 0.009 0.006

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -0.90189924 -0.25416609 -0.01056045 0.20048409 1.48673685

Number of Observations: 219 Number of Groups: PID Map.condition %in% PID 37 219

Simultaneous Tests for General Linear Hypotheses

Multiple Comparisons of Means: Tukey Contrasts

Fit: lme.formula(fixed = timeTransformed ~ Map.condition, data = AvgData2, random = ~1 | PID/Map.condition, method = "ML") Appendix C. Supplementary Information & R Output from Data Analyses 84

Linear Hypotheses: Estimate Std. Error z value Pr(>|z|) Flat SkyMap - HeadsDown == 0 -0.038563 0.029620 -1.302 0.78417 DomeOrtho - HeadsDown == 0 0.064327 0.029763 2.161 0.25606 DomeConf - HeadsDown == 0 -0.017287 0.029620 -0.584 0.99213 DomeEqD - HeadsDown == 0 -0.007923 0.029763 -0.266 0.99982 DomeOffset - HeadsDown == 0 0.102998 0.029620 3.477 0.00681 ** DomeOrtho - Flat SkyMap == 0 0.102890 0.029620 3.474 0.00682 ** DomeConf - Flat SkyMap == 0 0.021276 0.029358 0.725 0.97898 DomeEqD - Flat SkyMap == 0 0.030640 0.029620 1.034 0.90644 DomeOffset - Flat SkyMap == 0 0.141560 0.029358 4.822 < 0.001 *** DomeConf - DomeOrtho == 0 -0.081614 0.029620 -2.755 0.06499 . DomeEqD - DomeOrtho == 0 -0.072250 0.029763 -2.427 0.14679 DomeOffset - DomeOrtho == 0 0.038670 0.029620 1.306 0.78212 DomeEqD - DomeConf == 0 0.009364 0.029620 0.316 0.99958 DomeOffset - DomeConf == 0 0.120285 0.029358 4.097 < 0.001 *** DomeOffset - DomeEqD == 0 0.110920 0.029620 3.745 0.00245 ** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 (Adjusted p values reported -- single-step method)

Output (summary and post hoc Tukey contrasts) from linear modelling of mean trial confidence rating

Linear mixed-effects model fit by maximum likelihood Data: AvgData AIC BIC logLik 11.36473 41.86638 3.317633

Random effects: Formula: ~1 | PID (Intercept) StdDev: 0.3762977

Formula: ~1 | Map.condition %in% PID (Intercept) Residual StdDev: 0.1591831 0.08545029

Fixed effects: confTransformed ~ Map.condition Value Std.Error DF t-value p-value (Intercept) 1.2945671 0.06395352 177 20.242312 0.0000 Map.conditioncontrastTU 0.0125193 0.00557391 177 2.246060 0.0259 Map.conditioncontrastFlat 0.0157561 0.00674952 177 2.334406 0.0207 Map.conditioncontrastUnch -0.0284566 0.00872658 177 -3.260905 0.0013 Map.conditioncontrastOrtho -0.0177611 0.01244546 177 -1.427112 0.1553 Map.conditioncontrastConfEqd 0.0001313 0.02148880 177 0.006109 0.9951 Correlation: (Intr) Mp.cTU Mp.cnF Mp.cnU Mp.cnO Map.conditioncontrastTU -0.002 Map.conditioncontrastFlat 0.002 0.000 Map.conditioncontrastUnch 0.002 0.000 0.006 Map.conditioncontrastOrtho -0.002 0.000 -0.004 -0.005 Map.conditioncontrastConfEqd -0.003 0.000 -0.007 -0.009 -0.005 Appendix C. Supplementary Information & R Output from Data Analyses 85

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -1.395369726 -0.234478468 0.003608038 0.237912116 1.213392004

Number of Observations: 219 Number of Groups: PID Map.condition %in% PID 37 219

Simultaneous Tests for General Linear Hypotheses

Multiple Comparisons of Means: Tukey Contrasts

Fit: lme.formula(fixed = confTransformed ~ Map.condition, data = AvgData, random = ~1 | PID/Map.condition, method = "ML")

Linear Hypotheses: Estimate Std. Error z value Pr(>|z|) Flat SkyMap - HeadsDown == 0 0.0120916 0.0423848 0.285 0.9997 DomeOrtho - HeadsDown == 0 0.0979378 0.0425839 2.300 0.1939 DomeConf - HeadsDown == 0 0.0447858 0.0423848 1.057 0.8984 DomeEqD - HeadsDown == 0 0.0445233 0.0425839 1.046 0.9025 DomeOffset - HeadsDown == 0 0.1762419 0.0423848 4.158 <0.001 *** DomeOrtho - Flat SkyMap == 0 0.0858461 0.0423848 2.025 0.3276 DomeConf - Flat SkyMap == 0 0.0326942 0.0420045 0.778 0.9712 DomeEqD - Flat SkyMap == 0 0.0324317 0.0423848 0.765 0.9733 DomeOffset - Flat SkyMap == 0 0.1641502 0.0420045 3.908 0.0013 ** DomeConf - DomeOrtho == 0 -0.0531519 0.0423848 -1.254 0.8099 DomeEqD - DomeOrtho == 0 -0.0534145 0.0425839 -1.254 0.8097 DomeOffset - DomeOrtho == 0 0.0783041 0.0423848 1.847 0.4350 DomeEqD - DomeConf == 0 -0.0002625 0.0423848 -0.006 1.0000 DomeOffset - DomeConf == 0 0.1314560 0.0420045 3.130 0.0216 * DomeOffset - DomeEqD == 0 0.1317186 0.0423848 3.108 0.0233 * --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 (Adjusted p values reported -- single-step method)

Output (summary and post hoc Tukey contrasts) from linear modelling of mental workload (NASA-TLX) ratings

Linear mixed-effects model fit by maximum likelihood Data: MWLratings AIC BIC logLik 1635.856 1665.266 -808.9279

Random effects: Formula: ~1 | PID (Intercept) StdDev: 13.46084 Appendix C. Supplementary Information & R Output from Data Analyses 86

Formula: ~1 | Map.condition %in% PID (Intercept) Residual StdDev: 12.06335 5.312911

Fixed effects: MWLpercent ~ Map.condition Value Std.Error DF t-value p-value (Intercept) 47.17383 3.352890 153 14.069601 0.0000 Map.conditionFlat SkyMap -0.32588 3.342328 153 -0.097501 0.9225 Map.conditionDomeOrtho 6.86559 3.378329 153 2.032243 0.0439 Map.conditionDomeConf 5.08898 3.378928 153 1.506093 0.1341 Map.conditionDomeEqD 4.76605 3.411315 153 1.397129 0.1644 Map.conditionDomeOffset 13.00709 3.401870 153 3.823512 0.0002 Correlation: (Intr) Mp.FSM Mp.cndtnDmOr Mp.cDC Mp.DED Map.conditionFlat SkyMap -0.526 Map.conditionDomeOrtho -0.536 0.523 Map.conditionDomeConf -0.513 0.515 0.510 Map.conditionDomeEqD -0.509 0.510 0.508 0.505 Map.conditionDomeOffset -0.518 0.519 0.514 0.506 0.501

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -1.30869184 -0.29764443 0.05218563 0.25607892 0.81879853

Number of Observations: 194 Number of Groups: PID Map.condition %in% PID 36 194

Simultaneous Tests for General Linear Hypotheses

Multiple Comparisons of Means: Tukey Contrasts

Fit: lme.formula(fixed = MWLpercent ~ Map.condition, data = MWLratings, random = ~1 | PID/Map.condition, method = "ML")

Linear Hypotheses: Estimate Std. Error z value Pr(>|z|) Flat SkyMap - HeadsDown == 0 -0.3259 3.2902 -0.099 1.00000 DomeOrtho - HeadsDown == 0 6.8656 3.3257 2.064 0.30599 DomeConf - HeadsDown == 0 5.0890 3.3263 1.530 0.64476 DomeEqD - HeadsDown == 0 4.7660 3.3581 1.419 0.71529 DomeOffset - HeadsDown == 0 13.0071 3.3489 3.884 0.00147 ** DomeOrtho - Flat SkyMap == 0 7.1915 3.2310 2.226 0.22564 DomeConf - Flat SkyMap == 0 5.4149 3.2590 1.662 0.55740 DomeEqD - Flat SkyMap == 0 5.0919 3.2910 1.547 0.63350 DomeOffset - Flat SkyMap == 0 13.3330 3.2548 4.096 < 0.001 *** DomeConf - DomeOrtho == 0 -1.7766 3.2941 -0.539 0.99455 DomeEqD - DomeOrtho == 0 -2.0995 3.3160 -0.633 0.98856 DomeOffset - DomeOrtho == 0 6.1415 3.2894 1.867 0.42250 DomeEqD - DomeConf == 0 -0.3229 3.3272 -0.097 1.00000 DomeOffset - DomeConf == 0 7.9181 3.3175 2.387 0.16074 DomeOffset - DomeEqD == 0 8.2410 3.3497 2.460 0.13616 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 (Adjusted p values reported -- single-step method) Appendix C. Supplementary Information & R Output from Data Analyses 87

C.2 Analyses of individual trial data (Section 4.3.1)

Plots of transformed AE data

(a) (b)

Figure C.3: (a) histograms and (b) quantile-quantile plots of transformed AE data by map condition

Plots of transformed VAE data

(a) (b)

Figure C.4: (a) histograms and (b) quantile-quantile plots of transformed VAE data by map condition Appendix C. Supplementary Information & R Output from Data Analyses 88

C.2.1 Summary of significant predictors from linear models of AE/VAE for each map condition

Heads-down map

Formula: AE ∼ layout + distance + trial.index Formula: VAE ∼ distance + trial index

There was a significant effect of layout, There was a significant effect of distance, F (1, 683) = 4.79, p = .03, with centred trials F (2, 683) = 79.80, p < .0001, with near tri- having significantly lower AE values, b = .051, als having significantly higher error scores than t(682) = 2.12, p = .03, r = .08. the other distances, b = -0.075, t(683) = -9.30, There was a significant effect of distance, p < .0001, r = .34, and mid distance trials having F (2, 682) = 5.11, p = .006, with near trials higher error scores than farther ones, b = -0.12, having significantly lower AE values, b = .021, t(683) = -8.82, p < .0001, r = .32. t(682) = 2.44, p = .02, r = .09. There was a significant effect of trial index, There was a significant effect of trial index, F (1, 683) = 6.77, p = .001, with VAE increasing F (1, 682) = 19.90, p < .0001, with AE decreas- with later trials in a block, b = .0052, t(683) = 2.60, ing with later trials in a block, b = -.00093, p = .0095, r = .01. t(682) = -4.46, p < .0001, r = .17. Appendix C. Supplementary Information & R Output from Data Analyses 89

Flat SkyMap

Formula: AE ∼ layout + distance + trial index Formula: VAE ∼ layout * distance

There was a significant effect of layout, There was a significant interaction effect between F (1, 703) = 32.46, p < .0001, with centred layout and distance, F (2, 702) = 4.81, p = .009. trials having significantly lower AE values, b = .14, Within the interaction term, there was a significant t(703) = 5.79, p < .0001, r = .21. contrast between the mid and far distances, and cen- There was a significant effect of distance, tre vs. off-centre layout, b = -.13, t(702) = -3.02, F (2, 703) = 5.67, p = .004, with near trials p = .003, r = .11. Looking at Figure 4.10 (or the fig- being significantly lower, b = .028, t(703) = 3.21, ure above), this means that the change in error going p = .001, r = .12. from mid to far distances was significantly larger for There was a significant effect of trial index, off-centre trials than it was for centred ones. F (1, 703) = 4.68, p = .03, with AE decreasing with later trials in a block, b = -.0048, t(705) = -2.16, p = .03, r =.08. Appendix C. Supplementary Information & R Output from Data Analyses 90

Orthographic dome

Formula: AE ∼ (layout * distance) + trial index Formula: VAE ∼ distance

There was a significant interaction effect between For the orthographic dome projection, there was a layout and distance, F (2, 680) = 3.09, p = .046. main effect of distance, F (2, 684) = 44.07, p < .0001. There was a significant effect of trial index, Contrasts revealed that VAE scores were higher in F (1, 680) = 14.74, p = .0001, with AE decreasing the near distance than in the others, b = -0.095, with later trials in a block, b = -.0077, t(705) = - t(684) = -8.34, p < .0001, r = .30, and also at the 3.94, p = .0001, r =.15. mid distance relative to the far, b = -.09, t(684) = - Within the interaction term, there was a significant 4.42, p < .0001, r = .17. contrast between the near and other distances, and centre vs. off-centre layout, b = -.04, t(680) = -2.49, p = .013, r = .095. This means that the effect of centredness on AE was different in near trials than it was for the other distances. Appendix C. Supplementary Information & R Output from Data Analyses 91

Conformal dome

Formula: AE ∼ layout * trial index Formula: VAE ∼ layout + distance

There was a significant interaction effect between There was a main effect of layout, F (1, 705) = 27.06, layout and trial index, F (1, 705) = 6.035, p = .01, p < .0001, with VAE scores significantly lower for with trial index having a greater effect on off-centred centred trials, b = .19, t(705) = 5.24, p < .0001, trials, b = -.011, t(705) = -2.45, p = .014, r = .09. r = .19. There was a main effect of distance, F (2, 705) = 23.96, p < .0001, with VAE scores significantly higher for near trials than for the other distances, b = -0.09, t(705) = -6.90, p < .0001, r = .25. Appendix C. Supplementary Information & R Output from Data Analyses 92

Equidistant Dome

Formula: AE ∼ layout * distance Formula: VAE ∼ layout + distance

There was a significant interaction effect between There was a main effect of layout, layout and distance, F (2,685) = 4.67, p = .001, with F (1, 687) = 15.315, p = .0001, with VAE a significant contrast between the near and other scores significantly lower for centred trials, b = .14, distances, and centred vs. off-centre layout, b = - t(687) = 3.90, p = .0001, r = .15. 0.05, t(685) = -2.92, p = .004, r = .11. There was a main effect of distance, F (2, 687) = 20.59, p < .0001, with VAE scores significantly higher for near trials than for the other distances, b = -0.07, t(687) = -5.96, p < .0001, r = .22. Appendix C. Supplementary Information & R Output from Data Analyses 93

Offset Dome

Formula: AE ∼ layout Formula: VAE ∼ layout + distance

There was a main effect of layout, F (1, 703) = 26.64, There was a main effect of layout, F (1, 701) = 107.7, p < .0001, with VAE scores significantly lower for p < .0001, with VAE scores significantly lower for centred trials, b = .13, t(703) = 5.16, p < .0001, centred trials, b = .305, t(701) = 10.24, p < .0001, r = .19. r = .36. There was a main effect of distance, F (2, 701) = 3.51, p = .03, with VAE scores decreasing with increasing distance; however, the planned contrasts were not significant. Appendix C. Supplementary Information & R Output from Data Analyses 94

C.2.2 Summary output from multilevel models to predict AE & VAE for each map condition

Heads-down map (AE model & VAE model)

Linear mixed-effects model fit by maximum likelihood Data: FullData[FullData$map_f == "HeadsDown", ] AIC BIC logLik 497.3947 534.0509 -240.6973

Random effects: Formula: ~1 | Block.Index (Intercept) StdDev: 0.04813523

Formula: ~1 | PID %in% Block.Index (Intercept) Residual StdDev: 0.1428121 0.3240161

Fixed effects: delta5rt ~ centred + distance + trial.index Value Std.Error DF t-value p-value (Intercept) 2.3642569 0.04178339 682 56.58365 0.0000 centredoff-centre 0.0514002 0.02426628 682 2.11817 0.0345 distancecontrastCentre 0.0209743 0.00860797 682 2.43661 0.0151 distancecontrastLeftRight 0.0229254 0.01487297 682 1.54141 0.1237 trial.index -0.0093435 0.00209452 682 -4.46093 0.0000 Correlation: (Intr) cntrd- dstncC dstnLR centredoff-centre -0.295 distancecontrastCentre -0.013 -0.021 distancecontrastLeftRight -0.041 0.004 0.001 trial.index -0.530 0.005 0.033 0.074

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -3.8924500 -0.6129293 0.0914009 0.6501702 2.6198412

Number of Observations: 722 Number of Groups: Block.Index PID %in% Block.Index 6 36

Linear mixed-effects model fit by maximum likelihood Data: FullData4[FullData4$Map.condition == "TrackUp", ] AIC BIC logLik 453.1921 485.2663 -219.5961

Random effects: Formula: ~1 | Block.Index (Intercept) StdDev: 4.301783e-05 Appendix C. Supplementary Information & R Output from Data Analyses 95

Formula: ~1 | PID %in% Block.Index (Intercept) Residual StdDev: 0.2554197 0.3066151

Fixed effects: deltaVA3rt ~ dist_f + trial.index Value Std.Error DF t-value p-value (Intercept) 1.9770162 0.04888088 683 40.44560 0.0000 dist_f1 -0.0745677 0.00814259 683 -9.15774 0.0000 dist_f2 -0.1236715 0.01407021 683 -8.78960 0.0000 trial.index 0.0051524 0.00198070 683 2.60128 0.0095 Correlation: (Intr) dst_f1 dst_f2 dist_f1 -0.015 dist_f2 0.032 -0.001 trial.index -0.427 0.033 -0.074

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -3.5848952 -0.5467716 0.1046033 0.6687484 2.4279062

Number of Observations: 722 Number of Groups: Block.Index PID %in% Block.Index 6 36

Flat SkyMap (AE model & VAE model)

Linear mixed-effects model fit by maximum likelihood Data: FullData[FullData$map_f == "Flat SkyMap", ] AIC BIC logLik 584.3338 621.2301 -284.1669

Random effects: Formula: ~1 | Block.Index (Intercept) StdDev: 2.230601e-05

Formula: ~1 | PID %in% Block.Index (Intercept) Residual StdDev: 0.1600901 0.339869

Fixed effects: delta5rt ~ centred + distance + trial.index Value Std.Error DF t-value p-value (Intercept) 2.0865678 0.03886221 703 53.69144 0.0000 centredoff-centre 0.1453578 0.02509699 703 5.79184 0.0000 distancecontrastCentre 0.0282699 0.00881271 703 3.20786 0.0014 distancecontrastLeftRight -0.0175631 0.01548332 703 -1.13432 0.2570 trial.index -0.0047609 0.00220074 703 -2.16332 0.0309 Correlation: (Intr) cntrd- dstncC dstnLR centredoff-centre -0.310 distancecontrastCentre 0.015 0.020 distancecontrastLeftRight 0.010 0.011 0.003 trial.index -0.575 -0.022 -0.027 -0.021 Appendix C. Supplementary Information & R Output from Data Analyses 96

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -3.2893086 -0.6294673 0.0962336 0.6888361 2.7876615

Number of Observations: 744 Number of Groups: Block.Index PID %in% Block.Index 6 37

Linear mixed-effects model fit by maximum likelihood Data: FullData4[FullData4$Map.condition == "FlatMitS", ] AIC BIC logLik 999.9182 1041.427 -490.9591

Random effects: Formula: ~1 | Block.Index (Intercept) StdDev: 6.358569e-05

Formula: ~1 | PID %in% Block.Index (Intercept) Residual StdDev: 0.1852742 0.4512104

Fixed effects: deltaVA3rt ~ centred + dist_f + centred:dist_f Value Std.Error DF t-value p-value (Intercept) 1.8723863 0.03858601 702 48.52501 0.0000 centredoff-centre 0.2217530 0.03333616 702 6.65202 0.0000 dist_f1 -0.0832420 0.01676041 702 -4.96659 0.0000 dist_f2 -0.0064480 0.02893660 702 -0.22283 0.8237 centredoff-centre:dist_f1 0.0164843 0.02366011 702 0.69671 0.4862 centredoff-centre:dist_f2 -0.1254255 0.04153375 702 -3.01984 0.0026 Correlation: (Intr) cntrd- dst_f1 dst_f2 cn-:_1 centredoff-centre -0.431 dist_f1 0.001 -0.001 dist_f2 0.003 -0.001 0.008 centredoff-centre:dist_f1 -0.001 0.021 -0.716 -0.007 centredoff-centre:dist_f2 -0.001 -0.009 -0.008 -0.704 0.002

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -3.4274493 -0.6385519 0.1072719 0.7190698 2.8663743

Number of Observations: 744 Number of Groups: Block.Index PID %in% Block.Index 6 37 Appendix C. Supplementary Information & R Output from Data Analyses 97

Orthographic Dome SkyMap (AE model & VAE model)

Linear mixed-effects model fit by maximum likelihood Data: FullData[FullData$map_f == "DomeOrtho", ] AIC BIC logLik 425.8977 471.7179 -202.9488

Random effects: Formula: ~1 | Block.Index (Intercept) StdDev: 2.434888e-05

Formula: ~1 | PID %in% Block.Index (Intercept) Residual StdDev: 0.2516875 0.2995044

Fixed effects: delta5rt ~ centred + distance + trial.index + centred:distance Value Std.Error DF t-value p-value (Intercept) 2.3495207 0.04950782 680 47.45757 0.0000 centredoff-centre 0.1215509 0.02248020 680 5.40702 0.0000 distancecontrastCentre 0.0295510 0.01137161 680 2.59866 0.0096 distancecontrastLeftRight 0.0163069 0.01947973 680 0.83712 0.4028 trial.index -0.0076859 0.00195007 680 -3.94133 0.0001 centredoff-centre:distancecontrastCentre -0.0400144 0.01609714 680 -2.48581 0.0132 centredoff-centre:distancecontrastLeftRight 0.0029506 0.02789024 680 0.10579 0.9158 Correlation: (Intr) cntrd- dstncC dstnLR trl.nd cnt-:C centredoff-centre -0.228 distancecontrastCentre -0.001 0.009 distancecontrastLeftRight -0.029 0.015 -0.009 trial.index -0.415 0.003 -0.009 0.054 centredoff-centre:distancecontrastCentre -0.017 0.000 -0.713 0.008 0.047 centredoff-centre:distancecontrastLeftRight 0.046 -0.020 0.002 -0.708 -0.101 -0.010

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -2.91143075 -0.63186128 0.01977304 0.67770485 2.92251393

Number of Observations: 722 Number of Groups: Block.Index PID %in% Block.Index 6 36

Linear mixed-effects model fit by maximum likelihood Data: FullData4[FullData4$Map.condition == "DomeOrtho", ] AIC BIC logLik 937.7691 965.2612 -462.8845

Random effects: Formula: ~1 | Block.Index (Intercept) StdDev: 0.0001079076

Formula: ~1 | PID %in% Block.Index (Intercept) Residual StdDev: 0.3298735 0.4311659 Appendix C. Supplementary Information & R Output from Data Analyses 98

Fixed effects: deltaVA3rt ~ dist_f Value Std.Error DF t-value p-value (Intercept) 2.1871564 0.05739332 684 38.10821 0 dist_f1 -0.0953819 0.01143386 684 -8.34206 0 dist_f2 -0.0872214 0.01974993 684 -4.41629 0 Correlation: (Intr) dst_f1 dist_f1 -0.001 dist_f2 0.005 0.013

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -3.51997205 -0.66871722 0.04869478 0.68350980 2.97914490

Number of Observations: 722 Number of Groups: Block.Index PID %in% Block.Index 6 36

Conformal Dome SkyMap (AE model & VAE model)

Linear mixed-effects model fit by maximum likelihood Data: FullData[FullData$map_f == "DomeConf", ] AIC BIC logLik 591.0591 623.3527 -288.5295

Random effects: Formula: ~1 | Block.Index (Intercept) StdDev: 0.0146411

Formula: ~1 | PID %in% Block.Index (Intercept) Residual StdDev: 0.1689547 0.3409347

Fixed effects: delta5rt ~ centred + trial.index + centred:trial.index Value Std.Error DF t-value p-value (Intercept) 2.0662921 0.04703196 705 43.93379 0.0000 centredoff-centre 0.1677661 0.05265481 705 3.18615 0.0015 trial.index 0.0052301 0.00324908 705 1.60973 0.1079 centredoff-centre:trial.index -0.0109272 0.00444810 705 -2.45660 0.0143 Correlation: (Intr) cntrd- trl.nd centredoff-centre -0.573 trial.index -0.699 0.633 centredoff-centre:trial.index 0.519 -0.878 -0.741

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -3.20786767 -0.64017958 0.03970859 0.68922750 3.17339059 Appendix C. Supplementary Information & R Output from Data Analyses 99

Number of Observations: 745 Number of Groups: Block.Index PID %in% Block.Index 6 37

Linear mixed-effects model fit by maximum likelihood Data: FullData4[FullData4$Map.condition == "DomeConf", ] AIC BIC logLik 1126.719 1159.013 -556.3596

Random effects: Formula: ~1 | Block.Index (Intercept) StdDev: 0.0002014207

Formula: ~1 | PID %in% Block.Index (Intercept) Residual StdDev: 0.1829359 0.4940914

Fixed effects: deltaVA3rt ~ centred + dist_f Value Std.Error DF t-value p-value (Intercept) 1.9494133 0.03978160 705 49.00288 0.0000 centredoff-centre 0.1909384 0.03645016 705 5.23834 0.0000 dist_f1 -0.0899494 0.01302895 705 -6.90381 0.0000 dist_f2 -0.0101087 0.02206010 705 -0.45823 0.6469 Correlation: (Intr) cntrd- dst_f1 centredoff-centre -0.466 dist_f1 -0.011 -0.005 dist_f2 -0.003 -0.004 -0.008

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -3.07372397 -0.63864218 0.07041937 0.64612165 3.15249156

Number of Observations: 745 Number of Groups: Block.Index PID %in% Block.Index 6 37 Appendix C. Supplementary Information & R Output from Data Analyses 100

Equidistant Dome SkyMap (AE model & VAE model)

Linear mixed-effects model fit by maximum likelihood Data: FullData[FullData$map_f == "DomeEqD", ] AIC BIC logLik 545.3611 586.6491 -263.6806

Random effects: Formula: ~1 | Block.Index (Intercept) StdDev: 1.964734e-05

Formula: ~1 | PID %in% Block.Index (Intercept) Residual StdDev: 0.1764972 0.3319064

Fixed effects: delta5rt ~ centred + distance + centred:distance Value Std.Error DF t-value p-value (Intercept) 2.1300045 0.03440818 685 61.90401 0.0000 centredoff-centre 0.1211323 0.02485518 685 4.87352 0.0000 distancecontrastCentre 0.0322170 0.01240706 685 2.59666 0.0096 distancecontrastLeftRight 0.0132245 0.02192170 685 0.60326 0.5465 centredoff-centre:distancecontrastCentre -0.0515446 0.01764774 685 -2.92075 0.0036 centredoff-centre:distancecontrastLeftRight -0.0275660 0.03076394 685 -0.89605 0.3705 Correlation: (Intr) cntrd- dstncC dstnLR cnt-:C centredoff-centre -0.365 distancecontrastCentre 0.018 -0.026 distancecontrastLeftRight 0.007 -0.010 0.010 centredoff-centre:distancecontrastCentre -0.013 0.016 -0.709 -0.006 centredoff-centre:distancecontrastLeftRight -0.005 -0.004 -0.007 -0.715 -0.002

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -4.04051164 -0.64497814 0.03336884 0.73911695 2.85725881

Number of Observations: 726 Number of Groups: Block.Index PID %in% Block.Index 6 36

Linear mixed-effects model fit by maximum likelihood Data: FullData4[FullData4$Map.condition == "DomeEqd", ] AIC BIC logLik 1024.922 1057.035 -505.461

Random effects: Formula: ~1 | Block.Index (Intercept) StdDev: 0.06809152

Formula: ~1 | PID %in% Block.Index (Intercept) Residual StdDev: 0.2924511 0.4589839 Appendix C. Supplementary Information & R Output from Data Analyses 101

Fixed effects: deltaVA3rt ~ centred + dist_f Value Std.Error DF t-value p-value (Intercept) 2.0182615 0.06131495 687 32.91630 0.0000 centredoff-centre 0.1372653 0.03432666 687 3.99880 0.0001 dist_f1 -0.0719972 0.01208139 687 -5.95935 0.0000 dist_f2 -0.0503685 0.02116277 687 -2.38005 0.0176 Correlation: (Intr) cntrd- dst_f1 centredoff-centre -0.283 dist_f1 0.010 -0.022 dist_f2 -0.004 0.018 0.000

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -3.02509975 -0.66308288 0.04003981 0.77093073 3.03711969

Number of Observations: 726 Number of Groups: Block.Index PID %in% Block.Index 6 36

Offset Dome SkyMap (AE model & VAE model)

Linear mixed-effects model fit by maximum likelihood Data: FullData[FullData$map_f == "DomeOffset", ] AIC BIC logLik 594.0636 617.1036 -292.0318

Random effects: Formula: ~1 | Block.Index (Intercept) StdDev: 3.671099e-05

Formula: ~1 | PID %in% Block.Index (Intercept) Residual StdDev: 0.2392927 0.3379737

Fixed effects: delta5rt ~ centred Value Std.Error DF t-value p-value (Intercept) 2.3511024 0.04311858 703 54.52644 0 centredoff-centre 0.1288525 0.02496574 703 5.16117 0 Correlation: (Intr) centredoff-centre -0.287

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -3.71035992 -0.56197707 0.08185505 0.62497052 2.84464886

Number of Observations: 741 Number of Groups: Block.Index PID %in% Block.Index 6 37 Appendix C. Supplementary Information & R Output from Data Analyses 102

Linear mixed-effects model fit by maximum likelihood Data: FullData4[FullData4$Map.condition == "Uncharted", ] AIC BIC logLik 809.3145 841.5706 -397.6573

Random effects: Formula: ~1 | Block.Index (Intercept) StdDev: 0.006671901

Formula: ~1 | PID %in% Block.Index (Intercept) Residual StdDev: 0.1233933 0.4030398

Fixed effects: deltaVA3rt ~ centred + dist_f Value Std.Error DF t-value p-value (Intercept) 1.7886497 0.02931632 701 61.01208 0.000 centredoff-centre 0.3053445 0.02981539 701 10.24117 0.000 dist_f1 -0.0205423 0.01050816 701 -1.95489 0.051 dist_f2 -0.0330777 0.01829450 701 -1.80807 0.071 Correlation: (Intr) cntrd- dst_f1 centredoff-centre -0.503 dist_f1 -0.008 0.022 dist_f2 -0.017 0.044 0.009

Standardized Within-Group Residuals: Min Q1 Med Q3 Max -3.25199159 -0.69411479 0.07047059 0.63643350 3.72230923

Number of Observations: 741 Number of Groups: Block.Index PID %in% Block.Index 6 37