
2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) Alpaca: AR Graphics Extensions for Web Applications Tanner Hobson* Jeremiah Duncan* Mohammad Raji* Aidong Lu† Jian Huang* ABSTRACT an AR device (if present) simultaneously, which extends the spaces In this work, we propose a framework to simplify the creation of in which a user can explore the content in the web application. Augmented Reality (AR) extensions for web applications, with- We intend Alpaca to be a personalized bridging framework that out modifying the original web applications. We implemented the connects live web and AR contexts on a user’s laptop and AR device, framework in an open source package called Alpaca. AR extensions respectively. At present, the visual channel of AR is the sole focus developed using Alpaca appear as a web-browser extension, and of this work. We design that the Alpaca Server, which implements automatically bridge the Document Object Model (DOM) of the web cross-device synchronization, runs on the user’s laptop rather than as with the SceneGraph model of AR. To transform the web application a publicly accessible service. By running the server locally, we can into a multi-device, mixed-space web application, we designed a minimize the cross-device latencies to improve the user experience restrictive and minimized interface for cross-device event handling. of interacting with a mixed-space application. We demonstrate our approach to develop mixed-space applications For AR developers, the Alpaca framework provides an abstracted, using three examples. These applications are, respectively, for ex- easy-to-use interface for developers who are not experts of DOM ploring Google Books, exploring biodiversity distribution hosted (Document Object Model) to effectively instrument and manage by the National Park Service of the United States, and exploring changes to DOM elements. The resulting multi-device capability YouTube’s recommendation engine. The first two cases show how a aids AR developers in significantly lowering their barrier of entry 3rd-party developer can create AR extensions without making any into the widely varied ecosystem of web applications. modifications to the original web applications. The last case serves For web developers, the Alpaca framework provides a simplified as an example of how to create AR extensions when a developer cre- interface and runtime, where code using the web graphics API (e.g. ates a web application from scratch. Alpaca works on the iPhone X, THREE.js) can be automatically bridged to run on an AR device and the Google Pixel, and the Microsoft HoloLens. remain synchronized with the web application. Without needing to program on the AR device, user interactions with the scene graph Index Terms: Computing methodologies—Computer graphics— can auto-update the corresponding web DOM elements in real-time. Graphics systems and interfaces—Mixed / augmented real- For end users, Alpaca appears as a browser extension. When in- ity; Computer systems organization—Architectures—Distributed voked, the Alpaca Server runs as a daemon process and transparently architectures—Client-server architectures; manages interactions between the DOM in the browser and the scene graph in the AR runtime. Users get a synchronized multi-device 1INTRODUCTION experience and can interact with Alpaca-enhanced applications both Since inception, the web has led to a rich and diverse ecosystem for on their laptops and their AR devices, such as iPhone X. developing applications that impact our daily lives. This ecosystem Our design of Alpaca have been driven by three types of appli- lives in our 2D screens. With the advent of AR, researchers have cation needs: scene-heavy applications have large complex scene attempted marrying AR with web in custom-built applications to graph models; asset-heavy applications require Alpaca Server to leverage the rich interactions and explorability of AR. manage many assets concurrently at runtime; update-heavy appli- Rewriting web applications with AR in mind can be costly. There cations incur a sustained high volume of synchronizations. are few overlaps between the web development and the AR develop- Our design of Alpaca has been inspired by works such as MapRe- ment worlds. The near ubiquity of web browsers and the availability duce [14] and Tapestry [41, 42] that provide a minimal decoupled of a standard executing environment contrasts sharply with the niche interface between the framework’s components. Through this work, and disparate ecosystems that AR applications must conform to. In we have found that it is possible to use a common infrastructure essence, adding AR to an existing application has until now meant a to transparently abstract away all the cross-device communication costly rewrite and rearchitecture. and synchronization functionalities that are required in event-driven In this work, we have developed a bridging framework that simpli- programming. Consequently, the AR device becomes an extension fies extending web applications into AR without a complete rearchi- of the web application and becomes application agnostic, enabling tecture. Our framework is called Alpaca and has two main areas reuse. of improvement. In terms of content, it enables immersive explo- Our framework is composed of three parts that work in tandem: ration of natively 3D information using an AR device, rather than a browser extension library, a state management server, and an AR being constrained to the 2D-screen-space. In terms of architec- device runtime. Each part is in charge of mapping from one type of ture, Alpaca provides a minimal bridging interface that reduces the data model to another. The browser extension maps the DOM to a complexity and overhead of mixed-space applications. JSON representation of the 3D scene to be used on the AR device. While some web applications may have AR counterparts, mixed- The server manages this JSON scene structure and synchronizes space web applications developed with Alpaca run on a desktop and this scene to the AR device. The AR runtime then takes this scene and manages the rendering and interaction with these objects. AR- *These authors are with the Department of Electrical Engineering and initiated events are handled by taking the opposite route. Computer Science, University of Tennessee, Knoxville, TN 37996. E-mail: We demonstrate the efficacy of Alpaca as a reusable infrastructure {thobson2,jdunca51,mahmadza}@vols.utk.edu and [email protected] by building three types of applications: asset-heavy, update-heavy, †This author is with the Department of Computer Science, University of and scene-heavy applications. These correspond to the Google North Carolina at Charlotte, NC, 28262. E-mail: [email protected] Books, YouTube, and a GeoMap based application. Our results include performance, memory footprint metrics of Alpaca, and a demo video (in supplemental materials) that show how a user can use the interoperable capabilities to effectively mix the 2D desktop space together with the 3D physical space made available through the AR 2642-5254/20/$31.00 ©2020 IEEE 174 DOI 10.1109/VR46266.2020.00-68 devices. In a way, the resulting mixed-space is a new extension of screen moves the camera which alters the view of the scene that was the typical workspace of a user, which deserves much future research shown. by the computer graphics and the VR communities in general. While AR could increase the perception, cognition, and sensory In the remainder, we cover related work in Section 2, and our loads of users [39], studies on using these senses in different applica- system design in Section 3. We describe the process to develop tions have shown mixed results [16,21,46,49], however. In addition, Alpaca applications in Section 4, and then show demonstration we may not need to use all of the senses together [16]. applications in Section 5. We conclude in Section 6. Guidance towards areas of interest in an application is an impor- tant aspect of AR applications and one that can build on the existing 2RELATED WORK notion of spatial locality. This area has been explored in AR [17], 2.1 Use Scenario and many of the insights apply to Alpaca. In desktop applications, The distributed graphics applications in this work aim to help users guidance is handled through the familiar UI/UX of scrolling and use data-intensive web apps from desktop and AR simultaneously. panning through a 2D canvas. Our design philosophy is to help developers offer both forms of guidance in their applications. As web applications have become popular and widespread, many of the most useful web applications often have unique, valuable, and large datasets behind them. While the amount of data and 2.3 Porting Web to AR functionality of web applications grow at compounding rates, today’s Porting web content or the web ecosystem onto AR devices is not a users are also becoming more mobile than before. Oftentimes, a user new topic. Most research so far considers a scenario where an AR could be working away from their office or lab settings, where they device is the sole device of the user. Few have considered building cannot make use of large displays that have been proven as extremely a distributed graphics application where a user has an integrated valuable for tasks that require high density of information [4, 5]. workspace that spans desktop screen-space and AR physical-space We hypothesize that distributed graphics applications can help 2D simultaneously. This difference in scope is the primary difference screen displays of PCs and 3D physical space seen through personal between this work and existing literature. AR devices become an integrated virtual workspace. To that end, Showing web content in AR. In this regard, different approaches while porting web to AR is not a new endeavor, building distributed have been developed for smartphone-based vs head-mounted AR. graphics applications that enable web applications to stride desktop On smartphones, due to the prevalence of general-purpose mobile and AR devices simultaneously is a new endeavor.
Details
-
File Typepdf
-
Upload Time-
-
Content LanguagesEnglish
-
Upload UserAnonymous/Not logged-in
-
File Pages10 Page
-
File Size-