Iowa State University Capstones, Theses and Graduate Theses and Dissertations Dissertations

2021

Marketing in , a qualitative exploration of how people recall the details of advertisements and what factors drive individual willingness to act across interaction technologies.

Adam M. Mazurick Iowa State University

Follow this and additional works at: https://lib.dr.iastate.edu/etd

Recommended Citation Mazurick, Adam M., "Marketing in extended reality, a qualitative exploration of how people recall the details of advertisements and what factors drive individual willingness to act across interaction technologies." (2021). Graduate Theses and Dissertations. 18553. https://lib.dr.iastate.edu/etd/18553

This Thesis is brought to you for free and open access by the Iowa State University Capstones, Theses and Dissertations at Iowa State University Digital Repository. It has been accepted for inclusion in Graduate Theses and Dissertations by an authorized administrator of Iowa State University Digital Repository. For more information, please contact [email protected]. Marketing in extended reality, a qualitative exploration of how people recall the details of advertisements and what factors drive individual willingness to act across interaction technologies

by

Adam Mazurick

A thesis submitted to the graduate faculty

in partial fulfillment of the requirements for the degree of

MASTER OF SCIENCE

Major: Human Computer Interaction

Program of Study Committee: James Oliver, Major Professor Michael Dorneich Rafael Radkowski

The student author, whose presentation of the scholarship herein was approved by the program of study committee, is solely responsible for the content of this thesis. The Graduate College will ensure this thesis is globally accessible and will not permit alterations after a degree is conferred.

Iowa State University

Ames, Iowa

2021

Copyright © Adam Mazurick, 2021. All rights reserved. ii

DEDICATION

This thesis is dedicated to my Wife, Julia, who believed in my research interests and supported me as I completed my graduate education and conducted this study. The frank reality is that my educational pursuits and goals could not have been achieved without my Wife's support. iii

TABLE OF CONTENTS

ACKNOWLEDGMENTS…………………………………………………………………….….iv

ABSTRACT………………………………………….………………………………………..…..v

CHAPTER 1. MOTIVATION………..…………………………………………………..……… 1 Definition of terms………………………………………………………………………. 1 Technology and social trends……………………………………………………………. 2 Literature review…………………………………………………………..…………….. 4 Research questions …………………………………………………………………….. 11

CHAPTER 2. EXTENDED REALITY HMD DEVELOPMENT……….……………….……. 13 Early hardware prototype………….………………………………………..………….. 13 Early software prototype……………………………………………………………….. 17 and support in one app…………………………………. 18 Final hardware prototype………………………………………………………………. 22

CHAPTER 3. STUDY DESIGN…………………………….…………………………………. 25 Purpose…………………………………………………………………………..…….. 25 Methods …………………………………………………………………………….…. 25 Sampling and inclusion criteria……………………………………………………..…. 26 Participant recruitment…………………………………………………………………. 27 Data collection procedures..……………………………………………………………. 27 Commercial descriptions……………………………………………………………….. 29 Analysis plan…………………………………………………………………………… 35

CHAPTER 4. RESULTS………….…………………………………………………………… 36 Perceptual Exercise Results……………………………………………………………. 36 Thematic Analysis Results………………………………………………………………40 AEIOU Observations……………………………………………………………………51

CHAPTER 5. CONCLUSION………………………………………………………………….52

REFERENCES………………………………………………………………………….………56

APPENDIX A. IRB APPROVAL…………………………………………………………….…60

APPENDIX B. QUESTIONNAIRE…………………………………………………………….61 iv

ACKNOWLEDGMENTS

I had to write this thesis under unusual circumstances associated with a pandemic. At times it was an exceptionally arduous and solitary experience conducting remotely moderated research, manufacturing hardware and engineering software. I want to thank my Dog Winston for being my constant companion and emotional support aid during twelve long months of research, iterative design and research execution.

I would like to thank an individual who made time to mail me drawings about advanced display technologies he purportedly worked on, a Mr. Bob Lazar. Though controversial and highly debated, Mr. Lazar’s drawings and schematics inspired some of the later discussions concerning affective computing and how mixed-reality commercials were visualized, displayed, and presented to users.

I would like to thank my committee chair, Dr. James Oliver, and my committee members,

Dr. Michael Dorneich, and Rafael Radkowski, for their guidance and support throughout the course of this research.

In addition, I would also like to thank my friends, colleagues, the department faculty and staff for making my time at Iowa State University a wonderful experience. I want to also offer my appreciation to those who were willing to participate in my surveys and observations, without whom, this thesis would not have been possible. v

ABSTRACT

Advertisers currently face a significant challenge marketing to consumers in 2021. It is well established that people in 2021 consume nearly five times as much information as in 1986.

To manage this increase of information, consumers have developed complex coping strategies that involve increased multi-tasking among digital applications like Gmail, LinkedIn, Outlook

Calendar and Facebook. These very coping strategies that involve multi-tasking are making consumers less efficient at focusing their attention. Whatsmore, consumers are dividing their attention across more interaction technologies. This combination of societal trends presents distinct challenges to advertisers who want consumers to recall their brands, services and products. In an age where consumers own multiple connected devices, how consumers recall ads across interaction technologies and what drives them to act in response to those advertisements are largely unexplored.

In order to explore how people recall the details of advertisements and what factors drive individual willingness to act in response to advertisements, a qualitative study was conducted in which twelve subjects experienced eight commercial variants across popular interaction technologies.

After experiencing commercials across Head-Mounted Virtual Reality, Head-Mounted

Mixed Reality, Smart Phones, SmartTVs, Hearables, Smart Watches and Smart Speakers, consumers were able to recall the brands and details of commercials more frequently on Head-

Mounted Mixed Reality and Wearable interaction technologies. Moreover, willingness-to-act was vi highest on Mobile AR, Mixed Reality and Hearable interaction technologies. On this basis,

Mobile AR, head-mounted Mixed Reality, Wearable and Hearable interaction technologies should be taken into account when developing advertising communications. 1

CHAPTER 1. MOTIVATION

This thesis engages in a discussion of multi-sensory interaction technologies, so it is essential to define the terms used.

Virtual Reality (VR):

The definition of Virtual Reality (VR) is a computer-generated digital environment that can be experienced and interacted with as if that environment were real. An ideal VR system enables users to see, hear, touch, smell and move within a virtual environment as if it was real

(Jerald, 2016). Although many technologies have been developed or proposed to acutate human sensory perception of these simulated stimuli, truly immersive VR has yet to be realized. To date, the VR technologies with the highest sensory fidelity are those related to visual and auditory systems.

Augmented Reality (AR)

Augmented Reality is characterized by technologies that enable the natural visual perception of the physical world to be augmented with computer-generated virtual objects. As first defined by Milgram, et al. (1994) augmented reality falls within the broader classification of

Mixed Reality (MR) which represents a spectrum spanning from the real environment to the ideal virtual environment. 2

Head Mounted Display (HMD):

A Head Mounted Display (HMD) is a visual display device that is more or less rigidly attached to the user’s head. HMDs can be categorized as a non-see-through (i.e., all imagery is computer generated for VR) or see-through (either via video or optical) for AR (Jerald, 2016).

Although the term HMD is typically associated with VR applications, many manufacturers are preparing AR-enabled “Smart Glasses.”

Technology and social trends

The amount of information people consume has skyrocketed in recent years. Researchers at the University of Southern California have found that the average person in 1986 was exposed to approximately forty newspapers worth of information daily and by 2006, that number had more than quadrupled to 174 newspapers’ worth of information daily (Parr, 2016).

Researchers indicate that people have adopted multiple technologies as a coping strategy to manage this increase of information. For example, in order to manage busy email inboxes, people have adopted applications like Google Gmail. To stay in touch with friends and family, people use social network applications like Facebook that make it easy to stay up-to-date on all the important updates and life events from their familial and social contacts through a single scrollable feed. 3

Research suggests that people are multitasking more in order to consume and cope with this information (Parr, 2016). In addition, numerous studies have found multitasking makes it harder to pay attention and regain focus.

The number of interaction technologies used by consumers is growing. For example, consumer adoption of wearable devices like Apple Watches and mixed reality headsets like

Oculus Quest 2 has increased significantly year over year (Lang, 2021).

Consumers are spending their attention on more than one device.

In summary, attention in 2021 is a scarce commodity. The coping strategies people are using to manage the volume of information they are being confronted with are making them less efficient at focusing their attention, and consumers are dividing their attention across more interaction technologies.

This combination of societal trends presents distinct challenges to advertisers. Until relatively recently, the vast majority of advertising spend was focused on broadcast, print and desktop computers. With an average of 10 connected devices in American homes, advertisers ask the question, on what devices will their advertising dollars go furthest to engage and attract consumers, build brand awareness, and increase revenue generation through direct-to-consumer sales (Vailshery, 2021)? 4

Literature review

Marketing research has established that most consumer decisions are memory-based

(Klemm, 2014). Since the goal of our research is to understand how consumers remember the details of advertisements, it makes sense that our discussion begins with a current exploration of the literature on the subject of memory.

It is generally agreed that there are three types of memory or memory function: sensory buffers, short-term memory (STM) or working memory, and long-term memory (LTM) (Dix &

Lauesen, 2007). The sensory memories act as temporary buffers for stimuli received through the senses (Dix & Lauesen, 2007). STM acts as a 'scratch-pad for temporary recall of information

(Dix & Lauesen, 2007). It is used to store information that is only required fleetingly (Dix &

Lauesen, 2007). STM also has a limited capacity. If STM is our working memory or 'scratch-pad,

LTM is our main resource (Dix & Lauesen, 2007). Here we store factual information, experiential knowledge, procedural rules of behaviour – in fact, everything that we 'know' (Dix

& Lauesen, 2007).

There is a lack of consensus among researchers about the exact differences between long- term, short-term, and working memory (Cowan, 2008). Some of this confusion occurs as a result of differing definitions of the different types of memory (Cowan, 2008). For Cowan, the distinction between long-term and short-term memory depends on whether it can be demonstrated that there are properties specific to short-term memory; the main candidates 5 include temporal decay and a chunk capacity limit (Cowan, 2008). Cowan thinks there are two possible ways in which these stores may differ: in duration and capacity (Cowan, 2008).

Current literature suggests that the distinction between short and long-term memory is a significant one (Norris, 2017). Arguments that suggest STM are nothing more than activated

LTM are challenged in current cognitive psychological, and perceptual literature (Norris, 2017).

Recent research involving the analysis of neuroimaging data demonstrates that people can have

STM stores that contain multiple complex representational structures. Moreover, the research suggests the LTM stores have never encountered these structures and are entirely independent of

STM stores altogether (Norris, 2017).

The model of memory presented above and the ensuing discussion are important to our study because it provides us with an abstract and simplified way of understanding how memory works. It is, however, important to note that the model of memory presented above is just that - a model. It treats memory types as three distinct buckets in which information is processed and stored in localized containers within the brain, and this model is just a simplification of how human cognitive and perceptual processes like memory are believed to work. Recent research suggests that human memory is not quite as localized as our models might suggest. In fact, recent research suggests that memory functions are more likely distributed than localized in nature

(Christophel, Klink, Spitzer, Roelfsema, & Haynes, 2017). By examining a series of studies and electrophysiology data, the authors discovered numerous instances where the brain modularizes activities into distributed regions of the brain and not local areas of the brain (Christophel, Klink, 6

Spitzer, Roelfsema, & Haynes, 2017). The researchers suggest that working memory is better characterized as a distributed network that gradually transforms sensory information towards an appropriate behavioural response across a temporal delay (Christophel, Klink, Spitzer,

Roelfsema, & Haynes, 2017).

The above discussion explores traditional and recent perspectives of how human memory works, and most of the discussions are narrowly focused on memory alone. As interaction technologies like VR and Quest 2 surpass sales expectations and become more ubiquitous, interest in immersive advertising has increased (Lang, 2021).

It is important to note that immersive advertising presents distinct challenges for advertising communication. First, there is an upfront investment in platform-specific virtual environment design where advertisements need to be uniquely adapted to each environment (Qin

& Lei, 2019). Second, there are currently distinct hardware limitations associated with VR, such as a small field of view and picture delay (Qin & Lei, 2019). Third, there is no commonly accepted industry standard in immersive technology (Qin & Lei, 2019).

Despite the challenges that immersive advertising presents to advertisers, research in

2017 began to confirm what some researchers had already suspected, namely that consumers may recall advertisements more effectively in VR. In a qualitative study that explored advertising perception on immersive virtual reality devices, participants were asked to tour a city in Virtual Reality and look at various advertisements scattered throughout the city's virtual 7 landscape (Borba & Zuffo, 2017). Statistical analysis of a survey administered to participants after exposure that measured recall and sense of immersion found that advertisements were more effectively recalled in an HMD than in other conditions that involved a Desktop condition and a

Cave Automatic Virtual Environment (Borba & Zuffo, 2017). The preceding study and its findings are highly relevant to any exploration of advertisement effectiveness across interaction technologies because the data may suggest that increased immersion produces higher recall rates for advertisement communications.

Virtual reality experiences involve varying levels of mobility from users. Some VR experiences are enjoyed standing up, and others are enjoyed sitting down. Some VR experiences simulate walking using different interaction designs, and with each approach comes a different level of cognitive load on the user. A study that recently explored the effect of three different approaches to walking in Virtual Reality found that interaction approaches that required less physical mobility from virtual reality users produced higher recall rates and lower trial times to complete tasks (Lai & McMahan, 2020). These findings are relevant to the current study because they may suggest that reducing mobility involved in immersive ads may improve recall rates because there are fewer cognitive resources being expended during periods of reduced mobility.

Today, when advertisements are designed for digital products and use-cases, Human-

Computer Interaction (HCI) professionals will use a heuristic analysis method during the human- centered design and evaluation stages. A question that is sure to arise for product teams executing various advertising-related initiatives might be, are heuristics and non-user evaluation methods 8 still relevant to an immersive design context? Recent research recently explored the verification of virtual reality heuristics and usability evaluation in the context of the user-centered design methodology. The research objective was to verify and improve two heuristic evaluations, one proposed by Sutcliff and Gault and the other by Rusu, both applied to VR environments using a

Google Cardboard (Oliveira, Simoes, & Correia, 2017). Experts with immersive design domain knowledge were given both of the heuristic guidelines and asked to use them to evaluate two different video games. Sutcliff and Gault's heuristics were able to help evaluators find sixty errors, whereas Rusu's heuristics found twenty-seven errors. The analysis still corroborates the importance of constant evaluations in the product development cycle, specifically software. This study's results are relevant to the current study because the heuristic analysis can be used to inform the design of any supporting interactions for immersive interaction technologies.

Thus far, this discussion has explored memory, advertising and Virtual Reality, but what about Mixed Reality? Mixed Reality combines real-world and digital elements, where users interact with and manipulate both physical and virtual items and environments, using next- generation sensing and imaging technologies (Intel, 2021). Mixed Reality allows you to see and immerse yourself in the world around you even as you interact with a virtual environment using your own hands—all without ever removing your headset (Intel, 2021). It provides the ability to have one foot (or hand) in the real world. The other is in an imaginary place, breaking down basic concepts between real and imaginary, offering an experience that can change the way you game and work today (Intel, 2021). Recent patents suggest that Apple has been working on a compact pair of smart glasses that use a novel implementation of near-eye displays in a headset 9 that is tethered to compatible next-generation iPhone models for use in Mixed Reality applications (Purcher, 2021). In 2021, unit sales of Augmented Reality (AR) glasses among the leading brands worldwide are expected to amount to 410,000 units, rising to 3.9 million units by

2024 (Alsop, 2021). It is reasonable to assume that smart glasses as an interaction technology will gain adoption by people, and consequently, interest in Mixed Reality advertising is forecasted to increase (Alsop, 2021).

What value might Mixed Reality offer advertisers? One of the most promising applications of Mixed Reality may be storytelling. In 2012 researchers brought Alice-in- wonderland's story to life using a complex orchestration of room-scale environments and interactive large-format display technologies (Nakevska, Hu, Langereis, & Rauterberg, 2012).

While this research involves a Mixed Reality implementation that does not resemble modern compact implementations of Mixed Reality, it demonstrates that Mixed Reality is an effective storytelling aid (Nakevska, Hu, Langereis, & Rauterberg, 2012).

TV advertising spending worldwide from 2000-2022 amounted to 869 million U.S. dollars, while in the Asia Pacific, ad expenditures on TV surpassed 56 billion dollars in the same period. North America remained the region with the highest TV ad spend that year (Guttmann,

2020). Given the level of advertising spending associated with familiar and traditional interaction technologies like TV advertising that is consumed on older Televisions and SmartTVs, it is somewhat unsurprising that researchers have explored using Mixed Reality to increase audience engagement while watching Television. In 2016 researchers wanted to explore how 3D scene 10 analysis, photometric analysis and camera pose tracking can be used to create engaging applications that interact with users as the user watches video content in mixed reality (Baillard et al., 2016). The researchers create a test application that demonstrates the combination of 3D scene analysis, photometric analysis and camera pose tracking to create a parallel scene in

Augmented Reality that is overlaid on to the physical viewing environment in the background.

At first glance, this research does not seem significant, but one of the most novel contributions the paper makes is the application of this technology to compact glasses Apple is suspected of releasing in the near future. This research is significant for this exploration of advertising across interaction technologies because it is the first instance of a potential use case for Smart Glasses while watching traditional SmartTVs. This study points to a future use case where compact

Smart Glasses can be used to watch existing Televisions and monitors in the room and provide interactive elements in parallel scenes.

While Smart Glasses and Mixed Reality HMDs seem poised to push the envelope for consumers' entertainment value, are there any other outcomes that Mixed Reality HMDs might enable for advertisers? Researchers have found that HMDs afford superior spatial awareness by leveraging our vestibular and proprioceptive senses as compared to traditional desktop displays

(Krokos, Plaisant, & Varshney, 2018). A recent study exposed forty participants to a virtual environment that contained faces of people and numbers. This was a between-subjects study in which participants were assigned to either a desktop condition or a Virtual Reality condition.

Subjects experienced the virtual environment on a Desktop Computer or on a Head-Mounted

Virtual Reality condition. Participants were asked to recall details about the faces and numbers in 11 the virtual environments that they were exposed to. The researchers found that the HMD condition provides a statistically significant superior memory recall ability compared to the desktop condition. The researchers concluded that this is a first step in using virtual environments for creating more memorable experiences that enhance productivity through better recall of large amounts of information organized using the idea of virtual memory palaces.

Research Questions

Should advertisers focus on sending contextually relevant advertisements to wearables on people's wrists? Should advertisers position their brands in virtual reality games and experiences available on Facebook's Quest 2? Questions like the above are tough questions to answer for two reasons. First, there is a lack of available research on how consumers interact with brands across the full spectrum of current interaction technologies consumers use in 2021. Second, the rate of technological development in 2021 is rapid; devices come into existence and become ubiquitous faster than researchers can develop studies that evaluate their effects.

A good example of a device type being released and suddenly gaining adoption with little documented research is the hearable device category. Hearables are defined as smart audio equipment worn in or over the ear; they provide an entirely new experience compared with traditional headsets (O'Dea, 2020). In addition to simply playing audio, hearables can perform functions such as controlling other devices, making and receiving phone calls, monitoring fitness indicators, and connecting with digital assistants, including Alexa and Google Assistant (O'Dea, 12

2020). In 2017, hearables were a nascent and relatively unknown consumer segment of products with shipments below 20 million, but by 2019, shipments of the devices surged from 48.6 million in 2018 to 170.5 million units in 2019 (O'Dea, 2020). A search of the term 'Hearables' in

IEEE only returns a handful of peer-reviewed results. None of the search results discuss research that is relevant for helping advertisers prioritize advertising spend by interaction technology.

What should innovative organizations and advertisers do to maximize the results of their advertising dollars across devices? Business questions that relate to revenue generation are perhaps overly ambitious and even a little premature, given that fundamental questions about how people experience advertisements and respond to them across devices remain unanswered.

Questions such as how do people recall the details of advertisements across device technologies? What factors contribute to a person's willingness to act in response to ads when experienced across devices remain largely unexplored. The purpose of this thesis is to explore the questions mentioned above.

This thesis describes a qualitative study involving twelve subjects from North America and their exposure to eight advertisements experienced across seven popular interaction technologies: Head-Mounted Virtual Reality, Head-Mounted Mixed Reality, Smart Phones,

SmartTVs, Hearables, Smart Watches and Smart Speakers. 13

CHAPTER 2. EXTENDED REALITY HMD DEVELOPMENT

Early hardware prototype

This thesis aims to explore how people remember the details of advertisements across interaction technologies and how willing they are to act in response to those advertisements.

According to a recent survey, in the U.S, the average American has access to more than ten connected devices (Vailshery, 2021). From a marketing research perspective, a distinct limitation of current research is the limited number of interaction technologies typically involved in a single experiment or study.

Given the expected ubiquity and commercial promise forecasted for Smart Glasses, it seems as though a study of how consumers experience advertisements across devices that does not include Mixed Reality Smart Glasses would be incomplete by both today and tomorrow's standards of technology-use.

Involving compact Mixed Reality Smart Glasses in any study at this time presents distinct challenges to HCI and Engineering Researchers. Apple Glasses or an Apple Mixed

Reality Headset is not available at the time of this writing.

Existing solutions are expensive. The cost of a Microsoft Hololens that is the product that most closely resembles what Patents from Apple suggest Smart Glasses will look like retails for

$3,500 (Microsoft, 2021) 14

Even if this study had the financial resources to send a Microsoft Hololens to each subject, there would be concerns about construct validity. Suppose this study was to test a

Hololens and evaluate it with consumers. In that case, that's not the same as evaluating a compact, affordable and lightweight mixed reality HMD or pair of Smart Glasses.

While using a solution like Google's Cardboard may seem like a more cost-effective research option to explore VR advertisement delivery, Google announced it was discontinuing its work in VR and support for its headsets (by: Tom Nardi et al., 2021). Moreover, the currently available Google SDK lacks the necessary functionality to support Mixed Reality and produces errors when compiled in Apple's Interactive Development Environment Apple.

Additionally, the VR design specification last provided from Google does not expose the rear-facing cameras of modern smartphones necessary to perform scene understanding and mixed-reality tasks (Google, 2020). Google Cardboard does not accommodate a rear-facing camera because it was not intended to support AR as well as VR. For this research, we wanted one Head-Mounted Display to accommodate both VR and AR tasks.

Lastly, as shown in Figure 1, the Google Cardboard Viewer design and manufacturing guidelines do not support mounting the device to a person's head. 15

Figure 1. 2019 Google Cardboard

How does one evaluate the use of a product that is not commercially available, is expensive to make, with the necessary software to enable the desired features? This research aimed to compartmentalize these problems of hardware availability, cost, and software development to answer this question.

In 1962 Victor Papanek, a designer and educator, was approached by the United States

Military to design a device that could deliver a radio signal to people living in remote parts of the world where they were likely to have no electricity or any money for batteries. As shown in

Figure 2, Papanek produced a design that departed from the design sensibilities that characterize mid-century architecture, industrial design, and graphic design. Papanek produced a radio made from recycled juice cans, a couple of nails, wax and copper wire - materials that were abundant in the third world. To his contemporaries, the radio was unapologetically ugly and a massive 16 departure from the work of designers in his era, but the radio was truly a great achievement in design — it was sustainable and met the needs of those people who would use the device.

Figure 2. Victor Papanek Radio

In a tradition inspired by Papanek, this study explored using available low-cost materials to build the first prototype. In 2020, had produced an inexpensive VR cardboard viewer similar to Google’s viewer called Labo for the gaming console that retails for approximately $20 USD (Fisher, 2020). The product also lacked a

Head-mounted Display Strap, but the device was made from a thicker stock of cardboard that was easier to manipulate and cut.

The first prototype was a modified Nintendo Labo cardboard VR viewer with a 3.5” x

1.8” section of the headset removed to expose the rear-facing cameras of a modern iPhone12 that would be positioned inside. Please see Figure 3. 17

Figure 3. Early Prototype Made From a Nintendo Labo

Early software prototype

The prototype software was an iOS application written using the Swift programming language. The software was developed using a combination of Apple's content and augmented reality technologies and frameworks, namely Combine, SceneKit, ARKit, and Reality Kit. The integrated development used to create the prototype application was Xcode 12.4, and the prototype was tested on an iPhone 12 Pro. 18

Mixed Reality and Virtual Reality support in one app

Simply making a hardware prototype that exposes rear-facing cameras on newer generation iPhones is not enough to enable an experience that supports both Virtual and Mixed

Reality. Achieving an outcome that supports Mixed Reality and Virtual Reality on the same headset requires prototype development of both hardware and software. The Mixed Reality prototype that uses the iPhone's rear-facing camera is described first, with the three steps outlined below. Based on the Mixed Reality prototype, enabling Virtual Reality was comparatively easier because a simple room model is integrated into the Augmented Reality scene to block out and occlude the camera stream that was being used as the scene background.

The initial Mixed Reality prototype that uses the iPhone’s rear facing camera is created via the three steps outlined below.

I. Create one ARSCNView to use twice

ARSCNView is a class made by Apple that provides a way for developers to create augmented reality experiences that blend virtual 3D content with a camera view of the real world

(Apple, 2020). When executed, the method ARSession object does three things.

First, the view automatically renders the live video feed from the device camera as the scene background (Apple, 2020). Second, the world coordinate system of the view's SceneKit scene is directly correlated to the AR world coordinate system established by the session 19 configuration (Apple, 2020). Third, the view automatically moves its SceneKit camera to match the real-world movement of the device (Apple, 2020).

II. Update the other eye every frame

In order to create a stereoscopic display, a function needed to be created that would make use of the camera feed obtained from ARSCNview for the other eye. As shown in Figure 4, a second function called 'getSecondaryPointOfView' was written, and it does four things.

First, the function creates a new secondary point of view by duplicating the primary point-of-view obtained from the camera stream in ARSCNView. Second, it updates this view every second. Third, it applies calculations to determine an adjusted position for the other eye

Note that the variable “mag” corresponds to the distance between a user’s eyes (in meters), typically referred to as the Interpupillary Distance (IPD). Finally, it returns an updated point of view object for use in the other eye. To determine the appropriate camera image scale values for each eye's field-of-view, values from prior iOS experiments were used (Wang, 2017).

Figure 4. The Secondary Point of View Function 20

III. Position eye views horizontally beside each other.

As shown in Figure 5, the returned point of view object is then used in an HStack object that renders both a primary view and a secondary view that is updated at 60 frames per second.

The repeated view is positioned horizontally next to the primary point of view with a secondary eye position offset value added only to the repeated view's position value as shown in Figure 6.

Figure 5. The SwiftUI Layout That Holds Each Rendered Eye View

Figure 6. The HStack SwiftUI Layout That Holds Each Rendered Eye View 21

As shown in Figure 7, the only technical way Virtual Reality and Mixed Reality conditions differed was that the Virtual Reality condition had a Virtual Environment design in which a primitive cube textured to look like a room blocked the background camera feed in

Mixed Reality. A room was firmly anchored to the point of view node that positions an entire scene. If the three walls and floor were removed, the application would display the same background camera stream that enables the Mixed Reality experience.

Figure 7. The Virtual Reality Scene in Xcode That Contains Three Walls Around The Advertisement Creating the Illusion of VR in Mixed Reality. 22

Final hardware prototype

This study's next step was to determine how multiple headsets could be manufactured and shipped to users for evaluation with a functioning hardware and software prototype.

Using multiple variants of the Nintendo LABO would not be ideal as the headset's exterior shell was quite large. The LABO's exterior hardly resembled the compact form one might expect from Apple in the near future. The goal was to create a compact and lightweight design that could be shipped to users. The headset would also have to be easy to assemble so they could be manufactured in China, assembled in Canada and shipped across North America for evaluation.

This research referenced the Google Cardboard Manufacturing guidelines that are provided to help users make their own homemade headsets (Google, 2019). As shown in Figure

8, these guidelines offer rough schematics and assembly drawings that are useful for manufacturing Google Cardboard Headsets.

Google's manufacturing guidelines indicated that 34 mm diameter aspherical singlet lenses should be used, and a cardboard, plastic or foam enclosure would work best. 23

Figure 8. Google’s Cardboard Manufacturing Blueprint (Discontinued)

By working with a manufacturing company in Jiangsu's province in China, a modified, lighter weight and more compact headset was designed and manufactured with a headband to support a head-mounted experience. Schematics for the HMD designed in this research are shown in Figure 9. The design also features two small 1" magnets to help lock the headset.

Figure 9. Adam Mazurick’s Mixed Reality/Virtual Reality HDM Blueprint 24

Figures 10-11 show the final manufactured and assembled HMD developed to facilitate this research. All the parts that were required for each headset are shown in Figure 11. The fully assembled prototype HMD is shown holding an iPhone 12 Pro with the software running in

Figure 11.

Figure 10. Individual Parts Required to Make the Prototype HMD

Figure 11. IPhone 12 Pro Shown In Adam’s Prototype HMD 25

CHAPTER 3. STUDY DESIGN

Purpose

This study had two goals: (a) to explore how people recall advertisements' details across devices and (b) to explore what factors contribute to a person's willingness to act in response to advertisements when experienced across devices. This study was conducted under the approval of IRB ID: 21-124, please see Appendix "A."

Methods

This study relied on three methods to explore the research questions it presents. The first method is remotely moderated semi-structured interviews facilitated over Zoom video conferencing software. The second method is a qualitative research methodology known as

'experience prototyping'. Experience Prototyping is defined as a form of prototyping that enables design team members, users and clients to gain a first-hand appreciation of existing or future conditions through active engagement with prototypes (Buchenau & Suri, 2000). The third method is a contextual research activity that involves a perceptual memory exercise in which participants are asked to recall specific details of commercials and rate their willingness to act in response to commercials experienced across interaction devices using the Questionnaire in

Appendix "B." 26

Sampling & Inclusion Criteria

It has previously been recommended that qualitative studies require a minimum sample size of at least 12 to reach data saturation (Clarke & Braun, 2014). Accordingly, a sample of 12 was considered enough for the qualitative analysis and scale of this study.

Participants needed to have normal (or corrected) vision and no hearing loss in order to complete study procedures. A question in our Screener assesses hearing loss. Participants had to speak English as their primary language. Moreover, participants cannot have any prior photosensitive seizure disorder. Participants were evenly balanced across gender and were distributed across three age ranges with balanced quotas for each age segment.

Age inclusion criteria:

Age 18 - 29 n = 4 ( 50/50 Male and Female )

Age 30 - 44 n = 4 ( 50/50 Male and Female )

Age 45 - 60 n = 4 ( 50/50 Male and Female )

Participants had to be citizens of either Canada or the United States and were required to own all of the following technology devices. 27

• iPhone 11+ onward.

• Apple Watch Series 4 and above.

• Bluetooth wireless headphones.

• AppleTV or a SmartTV that enables casting from your phone.

Participant Recruitment

Participants were recruited using LinkedIn Navigator by way of direct messages that asked them to visit a Google Form that was used to screen participants according to the desired study inclusion criteria. Once successful participants indicated their willingness to participate and meet the criteria, they received an invitation and follow-up by email to schedule the interview.

Data Collection Procedures

In advance, participants were mailed one of the mixed reality prototypes manufactured according to the manufacturing blueprint made by Adam Mazurick shown in Figure 10. The headset was pre-assembled to mitigate confusion and ensure consistency of evaluation.

Participants were sent a test application using Apple's Test Pilot service used to Beta test applications. The application contained links to 8 commercials that are accessible from within the same single iPhone application. For example, the audible commercial for the voice and 28 headphone product categories will be a .wav file. The VR and MR categories' commercial includes a virtual environment in which virtual environment branded messaging is strategically placed.

Each commercial contained four parts:

1. 1 of 8 commercial variants

2. Product details

3. A feature, advantage or benefit (FAB)

4. Call to action (CTA)

The experience was designed to be enjoyed from a seated and stationary position. No physical walking or navigation was required from users. Commercial variants were carefully designed to appear before the participant's eyes upon loading. The order of the commercials and their assignment to subjects were randomized using a swift randomization function. This is part of a counterbalancing effort to mitigate carry-over effects.

Prior to the study, the HMD assembly and installation of the software on the subject's phones was checked to ensure it was working correctly. The study's scheduled duration was 70 minutes, and the entire session was recorded using the Zoom recording feature.

29

Subjects were asked to deploy the application, follow the onscreen instructions and experience all eight commercial variant types. The tasks that users were asked to complete consist of selecting play and watching the commercial.

Commercial descriptions:

As shown in Figure 12, the Virtual Reality interaction technology commercial was accessed via the prototype HMD and presented a Redbull Can as the product. The copy was

'Improve Focus,' and the call-to-action was 'Order now.'

Figure 12. Virtual Reality Commercial

As shown in Figure 13, the Mixed Reality interaction technology commercial was accessed via the prototype HMD and presented a Nike Shoe as the product. The copy was, '

Nike, Air Zoom Pegasus, Lighter and Faster,' and the call to action was 'Order Now.’ 30

Figure 13. Mixed Reality Commercial

The SmartSpeaker interaction technology commercial was accessed via the iPhone via a prerecorded .wav audio file where the product was Bank of America mortgages. The commercial read aloud:

"Get the right mortgage to finance your new home with a Bank of America, low interest mortgage, a no costs, no obligation prequalification request that takes about five minutes, get access to experienced lending specialists to help you every step of the way, easily manage the entire process online through home loan navigator. Get more with Bank of America. Home loans apply today. " 31

The call to action was to apply today. It is important to note that Smart Speaker use was simulated using the subject’s iPhone. The iPhone was kept stationary to simulate interaction with a Smart Speaker.

As shown in Figure 14, the Wearable interaction technology commercial was accessed via the Apple watch and presents Advil as the product. The copy was, "Got a headache? Reduce headaches and flu symptoms easily with extra-strength Advil. Two pills are all it takes." The call- to-action was 'Try Free.’

Figure 14. Wearable Commercial 32

As shown in Figure 15, the SmartTV interaction technology commercial contained Tesla as the product, and the copy was, "Gas, where we're going, we don't need gas." The call-to-action was, 'Book a test drive.’ During the SmartTV commercial variant, participants were asked to use

AirPlay to cast video from the phone to either AppleTVs or SmartTVs. Users were asked to deploy the TV experience, and they were guided through the procedure to screen share if necessary.

Figure 15. SmartTV Commercial

The Hearable interaction technology commercial was accessed via the Bluetooth wireless headphones and presented a prerecorded .wav audio commercial where the product was AT&T.

The commercial read aloud: 33

"AT&T's five G network is fast, reliable and secure, and today it's nationwide.”

The call-to-action was, "Learn more about Five G phones."

As shown in Figure 16, the Mobile Phone interaction technology commercial contained a

Coca-Cola glass bottle as the product. The copy was 'Refreshing Coca-Cola 25% less sugar,' and the call-to-action was 'Try Free.'

Figure 16. Mobile Phone Commercial

As shown in Figure 17, the Augmented Reality interaction technology commercial was accessed via the hand-held iPhone itself (no HMD) and presented a Playstation 5 Controller as 34 the product. The copy was, "Starwars coming to PS5." The call-to-action was "tap the controller.”

Figure 17. Augmented Reality Commercial

Each commercial variant type was designed to be experienced in under three minutes. As the user completed each commercial variant type, technology-use, interactions, any coping strategies or workarounds the user encountered were documented according to the Activities 35

Environment Interaction Observation Users (AEIOU) observation framework as described below.

At the end of the experience, participants were asked to complete a perceptual memory exercise and questionnaire. Please see Appendix A for the questionnaire used to collect subject responses. Commercials were delivered to participants in the following randomized order shown in Figure 18.

Figure 18. Randomization Table

Analysis Plan

Content analysis and thematic analysis was performed using a qualitative data analysis tool called Quirkos (Quirkos Inc., 2019). The number of correctly recalled brands, commercial details, calls-to-action, and self-reported willingness-to-act in response to each commercial was recorded into a table and analyzed. The number of for the correctly identified brands, commercial details, calls-to-action and ratings were tabulated. Important observations of subjects were organized according to the AEIOU observational framework (Hanington & Martin, 2012). 36

CHAPTER 4. RESULTS

Perceptual Experiment Results

Brand Recall

Figure 19. Brand Recall by Interaction Technology

Brands were recalled most frequently on Mixed Reality, Mobile Phone and Wearable interaction technologies (Figure 19). By contrast, brands were recalled the least frequently on

Smart Speakers, SmartTVs and Hearable devices. 37

Commercial Detail

Figure 20. Commercial Detail Recall by Interaction Technology

Commercial details were recalled most frequently on Mixed Reality, Mobile Phone and

Smart Speaker interaction technologies (Figure 20). Commercial details were recalled the least frequently on Voice speakers, Hearable and Smart TV interaction technologies. 38

Calls-to-Action

Calls-to-Action were recalled most frequently on Smart Watch, SmartTV, Mobile Phone and Hearable interaction technologies (Figure 21). Calls-to-Action were not recalled at all on

Smart Speakers and were recalled the least frequently on Hearable, Smart TV and Mobile Phone interaction technologies.

Figure 21. Calls-to-Action Recall by Interaction Technology 39

Willingness-to-Act

The mean Willingness-to-Act in response to the commercials was highest on Mobile AR,

Mixed Reality and Hearable interaction technologies (Figure 22). The mean Willingness-to-Act in response to the commercials was the lowest on Smart TV, Smart Speaker and Mobile Phone interaction technologies.

Figure 22. Willingness-to-Act by Interaction Technology 40

Thematic Analysis Results

This study applied a qualitative content analysis that involved transcribing videos into text data and then encoding discrete text response data.

There were over 50 sub-themes that were identified across the entire subject dataset; however, the following 15 themes were the most frequently discussed and coded themes with the highest level of conversation volume.

1. My memory

2. Tune out

3. Purchase evaluation

4. Action

5. Contextual relevance

6. Sustainability

7. Ad Design

8. Trust

9. Personalization

10.Virtual environment

11.Irritants 41

As shown in Figure 23, there is a thematic visualization produced using Quirkos qualitative data analysis software. Quirkos helps organize and code our entire dataset of response data from 12 subjects. The software makes managing large qualitative datasets more manageable.

The visualization provided by the Quirkos software arranges the themes according to the number of quotes per theme in descending order from largest to smallest. This study used a grounded theory-based approach where repeating themes were found by carefully reviewing the data. We coded the emergent themes with keywords and phrases, and then we grouped the codes into concepts hierarchically. Next, we relied on relationship identification to organize the concepts.

Figure 23. Theme cluster visualization of Quirks from Quirkos 42

Bad memory

‘Bad memory’ as a theme in this research encompasses any participant response data that involves perceptions of poor memory individual memory performance.

Subjects expressed considerable surprise with how quickly they forgot the brands and details of the commercials.

“I don't recall. I'm surprised that I don't recall, to be honest with you, because that was like a pretty interesting environment to view it in. OK, so the details that I do remember right is that obviously it's not see through. Right. So you don't see anything around the background aside from the object actually advertised like pretty neutral. That's about it.”

“Oh, my God. I was supposed to remember them all! Boy, you were talking about…

What was it? Oh, my God.” When the subject said, ‘I was suppose to remember them all!', this utterance was used not as a question but as a statement. When the statement was expressed, there was a sense of disbelief in how quickly the subject could forget the details of the commercial.

"Oh, my gosh, I feel like this is like two minutes ago.” 43

Tune out

‘Tune out’ as a theme in this research encompasses any participant response data that suggest subjects intentionally or unintentionally ignored parts of commercials or entire commercials.

Six of the twelve subjects reported unintentionally tuning-out and ignoring audible advertisements entirely.

“I guess for both of the audio ones, like I feel like they kind of remind me of like radio ads and, you know, like for me a radio is just, you know, radio ads or something just in between music for me growing up, you know, being in the car. So I just kind of tune it out, I guess if and when I'm supposed to be listening in my brain, honestly, I just was like, no, like, this is just background noise. And I wouldn't really call for most services. I'd rather click on a button. That would be the easiest thing for me.”

“You know when you’re driving to work, and you’re listening to a song on the radio but its about to end…and you’re kinda all like…you know some dumb commercial sh*t is about to come on-I automatically tuned it out-I guess. I’m sorry.”

“I don't know, because the visual when I see something I remember is way better than when I'm just hearing it.” 44

Purchase evaluation

‘Purchase evaluation’ as a theme in this research encompasses any participant response data that discusses how specific commercial types enable or do not enable helping subjects make buying decisions.

Eight subjects shared perceptions that Mixed Reality commercials enabled them to make better purchasing decisions.

“I mean, it was right in front of me. I could see the stitching in the shoe and every detail I would have to go to the store to see…it was a total game-changer.”

“Yes. But also, I feel like it was I don't know if you're looking for a shoe, then it's there it is. There's a shoe. It made it easy to order it.”

“I like that you could see this, you can kind of get the details of it in a three-dimensional way”. 45

Subjects perceived the advertisements in Mixed Reality as more in-focus and attention grabbing.

“Like it it just it was more in focus for me. So, you know, like I remember that had something to do with going.”

“They'll remember, they'll remember it was a very interesting advertising, so I don't remember what it has to do, but it was definitely attention grabbing.”

“I don’t know, it sorta stood out against my desk-like it Popped. It wasn’t real, but it was.”

Action

‘Action’ as a theme in this research encompasses any participant response data that discusses taking actions or responding to the commercials' requests.

Subjects indicated that Advertisements in VR, Augmented Reality and Mixed Reality were more engaging.

“I would rate it at five, rather engaging. Interesting. Just to see where that ad goes further.

I would click.” 46

Contextual relevance

‘Contextual relevance’ as a theme in this research encompasses any participant response data that discussed the context in which the commercials were played.

Participants perceived Wearable advertisements as being more urgent than other advertisements.

“It reminded me of an Amber alert-it was like that. When a kid goes missing my watch goes crazy and I guess because I got it on my watch, I felt like it was more important…I felt like it was urgent like an Amber alert.”

Subjects expressed an expectation that advertisements be more contextually relevant and proactive.

“It would be cool if the ads knew I was in my living-room or outside and then somehow changed to fit in the world better.”

“With your technology, can you scan what’s in my living-room and then suggest stuff I would like to buy? It would be nice if it somehow knew I was out of laundry detergent and offered me a coupon or something. ” 47

Sustainability

‘Sustainability’ as a theme in this research encompasses any participant response data that discussed perceptions of the Head Mounted Display’s cardboard design.

Participants perceived the use of recycled cardboard used in the construction of the headset positively.

“Oh, I think it's cool. It's like a sustainability thing as well.”

“I like the headset! I’m always worried about dropping my phone and Apple Care but with this headset you don’t worry about dropping it.”

Ad Design

‘Ad design’ as a theme in this research encompasses any participant response data that contained perceptions of the advertisement’s design quality or merits.

Participants demonstrated reading comprehension challenges with longer copy in mixed reality. 48

“I found it hard to read things in the Mixed Reality condition. I was more focused on the product and the text was at times in the way.”

“Sometimes the text floating in space-like it - was hard to read. It was white and -like, it, somehow I guess matched parts of the background.”

Trust

‘Trust’ as a theme in this research encompasses any participant response data that discusses perceptions of trusting or not trusting specific advertisements and the actions they ask consumers to take.

Participants perceived voice advertisements on both Voice Speakers and Hearables as less trust-worthy.

“I don’t trust the voice advertisements, I guess that’s because I think someone is listening to me.”

“I wouldn’t click the CTA because it’s a voice ad and I guess-like I didn’t know long it would potentially go on for.” 49

“I don’t man, I guess I feel like the f*cking CIA and trump can buy your data and with this, I feel they could literally sit in your ear and like monitor your thoughts.”

Personalization

‘Personalization’ as a theme in this research encompasses any participant response data that suggested that participants might want Ads that were uniquely tailored to them as individuals.

Participants expressed a desire for greater personalization in the advertisements that were presented to them.

“I like the Nike ad, but like I don’t wear running shoes that much. It would be cool if your tech knew what was in my closet and what wasn’t and surfaced ads with stuff I needed.”

“So-could you show me purses and stuff with this? Like it could do that right? I would use this for that all the time. wow…” 50

Virtual environment

‘Virtual Environment’ as a theme in this research encompasses any participant response data that discusses perceptions of commercials' virtual environments.

Participants perceived virtual advertisements as being more interesting to explore.

“I like that in the Mixed Reality and VR I am not being shown an ad, I am exploring an ad and there’s a difference there.”

“I guess you like feel-like you wanna look everywhere.”

Irritants

‘Irritants’ as a theme in this research encompasses any participant response data that notes frustrations, workarounds or negative perceptions expressed by the subjects while interacting with commercials

Female participants identified the strap that held the HMD in place to be irritating.

“It kept getting tangled in my hair…at times it hurt.”

“I wish you made a lady-hair-friendly strap-but I don’t even know how you’d do that.” 51

AEIOU Results

The current study employed an observational framework called AEIOU during the course of evaluating the prototype with subjects. AEIOU is an organizational framework reminding the researcher to attend to, document and code information under guiding taxonomy of Activities,

Environments, Interactions, Objects and Users (Hanington, 2012).

One activity that nearly half of all the subjects engaged in was untangling hair to fit the head-mounted display in place. Subjects would consistently struggle with the strap and tried different approaches to getting it in place. Two of the female subjects developed a strategy of attaching the band on the left side of the headset, holding the headset in place and pulling the strap around their heads.

Another activity that eight subjects experienced was difficulty understanding the connected status of devices. When participants experienced the hearable or watch commercials, there were moments of confusion for participants when they expressed uncertainty about whether or not their audio accessories were connected. 52

CHAPTER 5. CONCLUSION

The results of this research may corroborate and support prior research findings that suggest head-mounted displays help people encode information into short-term memory more effectively (Krokos, Plaisant, & Varshney, 2018). Further research with respect to validating consumer memory models across interaction technologies is a defined area of opportunity for future research. This present study's unique contribution is that it explored how consumers remember information on a head-mounted mixed reality headset. This is the first such exploration of how mixed reality might impact short-term working memory. This particular study may represent an interesting starting point for future research exploring relationships and variables related to head-mounted mixed reality devices and human memory.

This study's findings reveal that the subjects were most willing-to-act to advertisements and select specific calls-to-action across wearable interaction technologies such as Hearables and

Smart Watches. When asked why participants were more likely to act in response to wearable ads, subjects expressed a sentiment that indicated they perceived advertisements delivered via

SmartWatch to be more urgent in nature. One subject analogized receiving advertisements on a

SmartWatch as being similar to Amber Alerts. Why subjects perceive wearable advertisement communications as more urgent is unknown and constitutes an area of further research for both marketers and researchers.

Another finding of this research is that when advertisements were played auditorily on voice and hearable interaction technologies, consumers were more likely to tune them out and 53 ignore them altogether. When asked why people tuned out the auditory advertisements, a number of users made references to their experience listening to radios; they equated advertisements to noise. What specific auditory features and conditions cause this phenomenon are not explored and present a new frontier for further research.

During this qualitative study, participants expressed a desire that advertisements in mixed reality and virtual reality be more contextual and personalized. Based on the participant responses, advertising communication personalization represents a distinct area of opportunity for future research; however, this expectation communicated by subjects presents an entirely new set of challenges to advertisers. Understanding human emotionality or broader dimensions of context such as who and what is in the room may necessitate advances in hardware. For example, to gather emotional data new methods of extracting neurophysiological data from people in real- time is needed to enable advertisers to understand consumer emotional data. Advancements in hearable technologies may enable physiological in-ear sensing that obtains EEG data without the need for cumbersome BCI interfaces (Goverdovsky et al., 2017). In the near future, exploring how hearables can be used for physiological in-ear sensing to personalize advertising communication is a profoundly exciting and largely unexplored research area.

It is important to note that the same commercial was not experienced across all interaction technologies. For example, the Advil advertisement was experienced on an Apple

Watch but not on any other interaction technologies. The fact that each commercial was not experienced across all interaction technologies during the perceptual memory exercise could be 54 considered a confounding variable. One could suggest that people's recall and willingness to act are due to the commercials' content and not the interaction technologies through which the advertisement is experienced.

While such a potential confound is possible, one should also consider what would be required to address the presence of such a confound and the limitations inherent to those solutions as well.

To mitigate the potential existence of the potentially confounding variable, one would have to resort to a limited number of options. First, the prototype could be modified to feature only one brand like Advil across all eight interaction technologies. Second, one could alter the exercise to use a between-subjects design in which different participants are assigned to different conditions corresponding to a variable.

The first alternative is still vulnerable to the potential for confounds as it is possible that a study could get extremely enthusiastic Advil subjects who are more predisposed to react to the

Advil Brand content and not the interaction technology that branded content is consumed on. The potential for confounds still exists and are no less diminished.

The second alternative is a between-subjects design; however, given the phenomenon, this study is investigating this approach would present distinct disadvantages concerning practicality. For the study discussed in this thesis using a between-subjects research design 55 alternative, one might need eight groups, each with enough subjects for a control group and a test group. This alternative would increase the scale, complexity and resources required to execute such a study to impractical limits.

It is important to underline that this thesis was a qualitative study. The methods this thesis used were qualitative research methods that do not rely on using probabilities or significance testing to draw conclusions about what is likely to be true. This thesis aimed to explore participant perceptions of advertisements and the work structure associated with experiencing advertisements across various interaction technologies. This thesis's position is that the qualitative findings it presents are not diminished due to the potential of a confounding variable described above.

This research may indicate that head mounted displays that involve a one-strap design approach to securing displays to a person’s head may present accessibility challenges to individuals with longer hair, specifically Women. A future area of research might be exploring new types of strap designs that enable greater accessibility across gender.

Perhaps the most salient finding of this research study is that advertising strategies that involve a narrow focus on a limited number of consumer interaction technologies may be less effective at realizing advertising outcomes than a cross-channel approach that leverages advertising communication across multiple interaction technologies. 56

REFERENCES

Alleyne, R. (2011, February 11). Welcome to the information age – 174 newspapers a day. Retrieved April 10, 2021, from https://www.telegraph.co.uk/news/science/science-news/ 8316534/Welcome-to-the-information-age-174-newspapers-a-day.html

Alsop, T. (2021, March 18). AR glasses unit sales worldwide 2024. Retrieved April 11, 2021, from https://www.statista.com/statistics/610496/smart-ar-glasses-shipments-worldwide/

Apple. (2021). AR Kit Documentation. Retrieved April 18, 2021, from https:// developer.apple.com/documentation/arkit/arscnview

Apple. (2021). TestFlight. Retrieved April 12, 2021, from https://developer.apple.com/testfight/

Audio transcription software: Speech to text to magic. (n.d.). Retrieved April 12, 2021, from https://trint.com/

Baillard, C., Alleaume, V., Fradet, M., Jouet, P., Laurent, A., Luo, T., . . . Servant, F. (2016). Mixed reality extended tv. 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). doi:10.1109/ismar-adjunct.2016.0107

Barba, E., & Marroquin, R. Z. (2017). A primer on spatial scale and its application to mixed reality. 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). doi:10.1109/ismar.2017.27

Borba, E. Z., & Zuffo, M. K. (2017). Advertising perception with immersive virtual reality devices. 2017 IEEE Virtual Reality (VR). doi:10.1109/vr.2017.7892331

Buchenau, M., & Suri, J. F. (2000). Experience prototyping. Proceedings of the Conference on Designing Interactive Systems Processes, Practices, Methods, and Techniques - DIS '00. doi:10.1145/347642.347802

Christophel, T. B., Klink, P. C., Spitzer, B., Roelfsema, P. R., & Haynes, J. (2017). The distributed nature of working memory. Trends in Cognitive Sciences, 21(2), 111-124. doi:10.1016/j.tics.2016.12.007

Clarke, V., & Braun, V. (2014). Thematic analysis. Encyclopedia of Critical Psychology, 1947-1952. doi:10.1007/978-1-4614-5583-7_311

Cowan, N. (2008). Chapter 20 What are the differences between LONG-TERM, short-term, and working memory? Progress in Brain Research, 323-338. doi:10.1016/ s0079-6123(07)00020-9

Dix, A., & Lauesen, S. (2007). Human-computer interaction. Third edition. Harlow: Prentice Hall. 57

Fisher, C. (2020, June 10). A few NINTENDO Labo Kits drop to $20 each on Best Buy. Retrieved April 12, 2021, from https://www.engadget.com/nintendo-labo-vr-kit-best-buy- sale-090032613.html

Gabana, Daniel, Tokarchuk, Laurissa, Hannon, Emily, & Gunes, Hatice. (2017). Effects of valence and arousal on working memory performance in virtual reality gaming. 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII), 36-41.

Gallegos, J. (2020). Nonlinear design in VR: Dof, PoV, PoI. Retrieved April 12, 2021, from https://learningsolutionsmag.com/articles/nonlinear-design-in-vr-dof-pov-poi

Google. (n.d.). Manufacture cardboard. Retrieved April 01, 2021, from https://arvr.google.com/ cardboard/manufacturers/

Goverdovsky, V., Von Rosenberg, W., Nakamura, T., Looney, D., Sharp, D. J., Papavassiliou, C., . . . Mandic, D. P. (2017). Hearables: Multimodal physiological in-ear sensing. Scientific Reports, 7(1). doi:10.1038/s41598-017-06925-2

Guttmann, A. (2020, September 28). TV ad spend worldwide by REGION 2000-2022. Retrieved April 11, 2021, from https://www.statista.com/statistics/268666/tv-advertising-spending- worldwide-by-region/

Harding, S. (2018, December 20). What is fov? A basic definition. Retrieved April 12, 2021, from https://www.tomshardware.com/reviews/fov-field-of-view-definition,5740.html

Intel. (2021). Virtual reality vs. augmented reality vs. mixed reality. Retrieved April 11, 2021, from https://www.intel.ca/content/www/ca/en/tech-tips-and-tricks/virtual-reality-vs- augmented-reality.html

Jerald, J. (2016). The VR book human-centered design for virtual reality. In The VR book human- centered design for virtual reality (pp. 9-10). Place of publication unknown: ACM/M&C.

Klemm, W. (2014, February 21). How advertisers get you to remember ads. Retrieved April 11, 2021, from https://www.psychologytoday.com/us/blog/memory-medic/201402/how- advertisers-get-you-remember-ads

Krokos, E., Plaisant, C., & Varshney, A. (2018). Virtual memory palaces: Immersion aids recall. Virtual Reality, 23(1), 1-15. doi:10.1007/s10055-018-0346-3 58

Lai, C., & McMahan, R. P. (2020). The cognitive load and usability of Three Walking metaphors for Consumer virtual reality. 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). doi:10.1109/ismar50242.2020.00091

Lang, B. (2021, March 31). Facebook reality labs vp: Quest 2 has outsold all oculus headsets combined. Retrieved April 11, 2021, from https://www.roadtovr.com/oculus-quest-2- sales-bosworth-bloomberg/? fbclid=IwAR2B42n0Rv06D0vk9PGAXv_0jE7pxzjgyNdv2zZaXgIQ8EqFywovlDmxBpI

Microsoft. (n.d.). HoloLens 2: Find specs and features - Microsoft Hololens 2. Retrieved April 12, 2021, from https://www.microsoft.com/en-us/p/holoLens-2/91pnzzznzwcp/? activetab=pivot%3Aoverviewtab

Milgram, P., Takemura, H., Utsumi, A. and Kishino, F. (1994) Augmented Reality: A class of displays on the reality- continuum, Telemanipulator and Technologies, SPIE, Vol. 2351, pp. 282-292

Nakevska, M., Hu, J., Langereis, G., & Rauterberg, M. (2012). Alice's adventures in an Immersive mixed reality environment. 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). doi:10.1109/ismar.2012.6402585

Nardi, T. (2021). Google calls it quits With vr, But Cardboard lives on. Retrieved April 19, 2021, from https://hackaday.com/2021/03/30/google-calls-it-quits-with-vr-but-cardboard-lives- on/

Norris, D. (2017). Short-term memory and long-term memory are still different. Psychological Bulletin, 143(9), 992-1009. doi:10.1037/bul0000108

O'Dea, S. (2020, June 19). Topic: Hearables. Retrieved April 13, 2021, from https:// www.statista.com/topics/6209/hearables/

Oliveira, E., Simoes, F. P., & Correia, W. F. (2017). Heuristics evaluation and improvements for low-cost virtual reality. 2017 19th Symposium on Virtual and Augmented Reality (SVR). doi:10.1109/svr.2017.31

Papanek, V. J. (2019). Design for the real world. London: Thames & Hudson.

Parr, B. (2016). Captivology: The science of capturing people's attention. New York: HarperOne. 59

Purcher, J. (2021, April 06). Apple has won a patent for allowing future Apple glasses to be able to READ invisible markers placed on real-world objects. Retrieved April 11, 2021, from https://www.patentlyapple.com/patently-apple/2021/04/apple-has-won-a-patent-for- allowing-future-apple-glasses-to-be-able-to-read-invisible-markers-placed-real-world- objects.html

Willemsen, P., Jaros, W., McGregor, C., Downs, E., Berndt, M., & Passofaro, A. (2018). Memory task performance across augmented and virtual reality. 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). doi:10.1109/vr.2018.8446457

Qin, H., & Lei, J. (2019). The application of virtual reality technology in advertising communication. 2019 International Conference on Virtual Reality and Intelligent Systems (ICVRIS). doi:10.1109/icvris.2019.00027

Quirkos Inc. (n.d.). Simple qualitative data analysis software. Retrieved April 12, 2021, from https://www.quirkos.com/

Vailshery, L. (2021, January 22). Average number of connected devices in Households worldwide 2020. Retrieved April 12, 2021, from https://www.statista.com/statistics/ 1107307/average-number-connected-devices-households-worldwide/

Wang, H. (2017). Hanleyweng - overview. Retrieved February 18, 2021, from https:// github.com/hanleyweng 60

APPENDIX A. IRB APPROVAL MEMO 61

APPENDIX B. Questionnaire

Q1. What was the product in commercial 1? Q2. What were the details of the commercial? Q3. What did the advertisement ask you to do at the end? Q4. On a scale of 1-5 where 1 is not likely and 5 is very likely, how likely are you to do what this specific ad asked you to do? Q5. Can you expand on your provided rating. Why do you feel that way?

Q6. What was the product in commercial 2? Q7. What were the details of the commercial? Q8. What did the advertisement ask you to do at the end? Q9.On a scale of 1-5 where 1 is not likely and 5 is very likely, how likely are you to do what this specific ad asked you to do? Q10. Can you expand on your provided rating. Why do you feel that way?

Q11. What was the product in commercial 3? Q12. What were the details of the commercial? Q13. What did the advertisement ask you to do at the end? Q14. On a scale of 1-5 where 1 is not likely and 5 is very likely, how likely are you to do what this specific ad asked you to do? Q15. Can you expand on your provided rating. Why do you feel that way?

Q16. What was the product in commercial 4? Q17. What were the details of the commercial? Q18. What did the advertisement ask you to do at the end? Q19. On a scale of 1-5 where 1 is not likely and 5 is very likely, how likely are you to do what this specific ad asked you to do? Q20. Can you expand on your provided rating. Why do you feel that way?

Q21. What was the product in commercial 5? Q22. What were the details of the commercial? Q23. What did the advertisement ask you to do at the end? Q24. On a scale of 1-5 where 1 is not likely and 5 is very likely, how likely are you to do what this specific ad asked you to do? Q25. Can you expand on your provided rating. Why do you feel that way?

Q26. What was the product in commercial 6? Q27. What were the details of the commercial? Q28. What did the advertisement ask you to do at the end? 62

Q29. On a scale of 1-5 where 1 is not likely and 5 is very likely, how likely are you to do what this specific ad asked you to do? Q30. Can you expand on your provided rating. Why do you feel that way?

Q31. What was the product in commercial 7? Q32. What were the details of the commercial? Q33. What did the advertisement ask you to do at the end? Q34. On a scale of 1-5 where 1 is not likely and 5 is very likely, how likely are you to do what this specific ad asked you to do? Q35. Can you expand on your provided rating. Why do you feel that way?

Q36. What was the product in commercial 8? Q37. What were the details of the commercial? Q38. What did the advertisement ask you to do at the end? Q39. On a scale of 1-5 where 1 is not likely and 5 is very likely, how likely are you to do what this specific ad asked you to do? Q40. Can you expand on your provided rating. Why do you feel that way?