DOCSLIB.ORG

Exploratory Programming of Distributed Augmented Environments

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Exploratory Programming of Distributed Augmented Environments Exploratory Programming of Distributed Augmented Environments Blair MacIntyre Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Graduate School of Arts and Sciences COLUMBIA UNIVERSITY 1999 © 1999 Blair MacIntyre All Rights Reserved Abstract Exploratory Programming of Distributed Augmented Environments Blair MacIntyre Augmented reality is a form of virtual reality that uses see-through displays to enhance the world with computer-generated material. When combined with more traditional displays, a powerful augmented environment emerges in which two and three dimensional informa- tion can be presented to a user simultaneously on a combination of displays. Prototyping these environments is challenging both because they are highly distributed, interactive sys- tems, and because of the exploratory nature of building systems for new interaction para- digms. We have developed a testbed for exploratory programming of distributed augmented envi- ronments, called Coterie. A single programming model is used for both single and multi- process programs by building applications as groups of threads communicating via shared objects. The distributed programming model is distributed object memory (DOM), an object-based approach to distributed shared memory. Coterie’s DOM presents the pro- grammer with both client-server and replicated distributed objects. Both interpreted (Repo) and compiled (Modula-3) languages present the application pro- grammer with similar DOM programming models. Modula-3’s replicated objects are implemented using Shared Objects, an object replication package that is tightly integrated with the Modula-3 object system and designed to be flexible and easy-to-use. Repo is implemented using the Shared Object package, and presents the programmer with an interpreted language that supports both client-server and replicated objects uniformly across its entire type system. The final important component of Coterie is Repo-3D, a high-level, distributed graphics library, built using the Shared Objects package and tightly integrated with Repo. By making all graphical objects extensible and transparently distrib- utable, programmers can use Repo-3D scene graphs as the basis for their application data structures, allowing complex distributed graphical applications to be created in a straight- forward manner. Numerous stand-alone and distributed augmented environment systems have been devel- oped using Coterie, and demonstrate its usefulness. These include an architectural anat- omy system for viewing the support structures inside the walls of a building, a construction assistant for space frame buildings, a maintenance and repair task for tele- phone crossboxes, an augmented reality tour guide, and a number of interface concepts for the National Tele-Immersion Initiative. Table of Contents CHAPTER 1 Introduction . 1 1.1 Exploratory Programming of Distributed Augmented Environments . 3 1.2 Research Contributions . 6 1.2.1 Shared Objects: A Distributed Shared Object Memory. 6 1.2.2 Repo: A Distributed Interpreted Language . 8 1.2.3 Repo-3D: A Distributed 3D Graphics Library. 9 1.2.4 Coterie: Exploratory Programming of AE Systems. 11 1.2.5 Prototype Augmented Reality Applications . 11 CHAPTER 2 An Overview of Coterie . 13 2.1 Previous Work: Augmented Reality . 13 2.1.1 KARMA. 14 2.1.2 Windows on the World. 15 2.1.3 Architectural Anatomy. 16 2.2 Motivation . 17 2.3 Requirements for the Testbed . 19 2.4 Related Research Areas . 22 2.4.1 Virtual Environment Systems . 22 2.4.2 Distributed Groupware. 25 2.5 Implementation Overview . 26 2.5.1 Virtual Environments: Tracker Support . 28 2.6 Initial Prototypes . 30 2.6.1 Architectural Anatomy. 30 2.6.2 Telephone Crossbox Maintenance . 31 2.6.3 Spaceframe Construction. 32 2.6.4 Automated Tour Guide . 33 CHAPTER 3 Shared Objects. 38 3.1 Distributed Shared Memory. 39 3.2 Related Work . 42 3.3 Shared Object Package Design . 47 3.3.1 Goal: Tight Integration. 47 3.3.2 Model: Totally Ordered, Write-Update Objects . 48 3.3.3 Event Driven Control Flow: Callback Objects . 53 3.4 Implementation . 54 3.4.1 Object Definition and Runtime Code Generation . 55 i 3.4.1.1 Example Object Definition . 56 3.4.1.2 Callback Object Usage . 61 3.4.1.3 Passing State Between Processes . 63 3.4.1.4 Additional Tracker Examples . 65 3.4.2 The Shared Object Runtime. 66 3.4.2.1 Thread Management . 68 3.4.2.2 Exception and Return Value Handling . 71 3.4.3 Restrictions . 72 3.5 Performance and Usability. 73 3.5.1 Shared Object Performance . 73 3.5.2 Shared Object Usability . 79 3.6 Discussion . 82 CHAPTER 4 Repo. 84 4.1 Related Work . 86 4.2 An Overview of Obliq and Repo . 87 4.3 Distributed Semantics . 88 4.4 Replication Syntax. 92 4.4.1 Declarations . 92 4.4.2 Cloning Data . 95 4.4.3 User-defined Picklers . 97 4.5 The Replication Module . 98 4.6 Examples . 99 4.6.1 Simple Tracker Report Distribution. 99 4.6.2 Asynchronous Change Notification . 101 4.6.3 Multi-person Spaceframe Construction . 102 4.6.4 Distributed Mutexes. 105 4.6.5 Hierarchical Object Directories . 108 4.7 Implementation . 111 4.8 Usability of Repo . 115 CHAPTER 5 Repo-3D . 118 5.1 Related Work . 121 5.2 Obliq-3D: An Overview. 123 5.3 Design Of Repo-3D . 126 5.3.1 Conversion to Shared Objects . 126 5.3.1.1 Graphical Objects . 127 5.3.1.2 Properties . 128 5.3.1.3 Animation Handles . 129 5.3.1.4 Input Callbacks . 131 5.3.1.5 Change Notification . 131 ii 5.3.2 Local Variations. 133 5.3.3 Extensibility . 135 5.4 Examples . 137 5.4.1 A Tutorial Example . 138 5.4.2 Yet Another Tracker Example . 138 5.4.3 A Truncated Pyramid Object . 140 5.4.4 An Animation Examiner . 141 5.5 Implementation . 144 5.6 Performance . 153 5.7 Discussion . 155 CHAPTER 6 Coterie Examples. ..
Load more

Recommended publications
  • Exploratory Programming for the Arts and Humanities
    Exploratory Programming for the Arts and Humanities Exploratory Programming for the Arts and Humanities Second Edition Nick Montfort The MIT Press Cambridge, Massachusetts London, England © 2021 Nick Montfort The open access edition of this work was made possible by generous funding from the MIT Libraries. This work is subject to a Creative Commons CC-BY-NC-SA license. Subject to such license, all rights are reserved. This book was set in ITC Stone Serif Std and ITC Stone Sans Std by New Best-set Typesetters Ltd. Library of Congress Cataloging-in-Publication Data Names: Montfort, Nick, author. Title: Exploratory programming for the arts and humanities / Nick Montfort. Description: Second edition. | Cambridge, Massachusetts : The MIT Press, [2021] | Includes bibliographical references and index. | Summary: "Exploratory Programming for the Arts and Humanities offers a course on programming without prerequisites. It covers both the essentials of computing and the main areas in which computing applies to the arts and humanities"—Provided by publisher. Identifiers: LCCN 2019059151 | ISBN 9780262044608 (hardcover) Subjects: LCSH: Computer programming. | Digital humanities. Classification: LCC QA76.6 .M664 2021 | DDC 005.1—dc23 LC record available at https://lccn.loc.gov/2019059151 10 9 8 7 6 5 4 3 2 1 [Contents] [List of Figures] xv [Acknowledgments] xvii [1] Introduction 1 [1.1] 1 [1.2] Exploration versus Exploitation 4 [1.3] A Justification for Learning to Program? 6 [1.4] Creative Computing and Programming as Inquiry 7 [1.5] Programming Breakthroughs
    [Show full text]
  • Part I: Julia for Matlab Users
    Julia Part I Julia for Matlab Users Prof. Matthew Roughan [email protected] http://www.maths.adelaide.edu.au/matthew.roughan/ UoA Oct 31, 2017 M.Roughan (UoA) Julia Part I Oct 31, 2017 1 / 33 I write to find out what I think about something. Neil Gaiman, The View From the Cheap Seats M.Roughan (UoA) Julia Part I Oct 31, 2017 2 / 33 Section 1 Get Started M.Roughan (UoA) Julia Part I Oct 31, 2017 3 / 33 The reason I feel like we can do this is because (I hope) you all know some Matlab, and Julia is syntactically and operationally very much like Matlab I syntax is very similar 1 I REPL is similar F tab completion, and up arrows work F ? = help F ; = shell escape to OS I JIT compiler I Use cases are similar 1REPL = Read-Evaluate-Print Loop; old-school name is the shell, or CLI. M.Roughan (UoA) Julia Part I Oct 31, 2017 4 / 33 So have a go You should have installed Julia before the workshop Start it up I start up varies depending on IDE, and OS I I am using simplest case (for me): the CLI, on a Mac I it’s all very Unix-y Type some calculations a = 3 b = a + 2 c = a + bˆ2 Create a script, e.g., “test.jl”, and “include” it include("test.jl") I its a little more cumbersome than Matlab M.Roughan (UoA) Julia Part I Oct 31, 2017 5 / 33 Section 2 Julia Isn’t Matlab (or Octave) M.Roughan (UoA) Julia Part I Oct 31, 2017 6 / 33 Julia may look a lot like Matlab but under the hood its very different and there are a lot of changes that affect you otherwise why would we bother? M.Roughan (UoA) Julia Part I Oct 31, 2017 7 / 33 Why Julia? Big
    [Show full text]
  • Exploratory Data Science Using a Literate Programming Tool Mary Beth Kery1 Marissa Radensky2 Mahima Arya1 Bonnie E
    The Story in the Notebook: Exploratory Data Science using a Literate Programming Tool Mary Beth Kery1 Marissa Radensky2 Mahima Arya1 Bonnie E. John3 Brad A. Myers1 1Human-Computer Interaction Institute 2Amherst College 3Bloomberg L. P. Carnegie Mellon University Amherst, MA New York City, NY Pittsburgh, PA [email protected] [email protected] mkery, mahimaa, bam @cs.cmu.edu ABSTRACT engineers, many of whom never receive formal training in Literate programming tools are used by millions of software engineering [30]. programmers today, and are intended to facilitate presenting data analyses in the form of a narrative. We interviewed 21 As even more technical novices engage with code and data data scientists to study coding behaviors in a literate manipulations, it is key to have end-user programming programming environment and how data scientists kept track tools that address barriers to doing effective data science. For of variants they explored. For participants who tried to keep instance, programming with data often requires heavy a detailed history of their experimentation, both informal and exploration with different ways to manipulate the data formal versioning attempts led to problems, such as reduced [14,23]. Currently even experts struggle to keep track of the notebook readability. During iteration, participants actively experimentation they do, leading to lost work, confusion curated their notebooks into narratives, although primarily over how a result was achieved, and difficulties effectively through cell structure rather than markdown explanations. ideating [11]. Literate programming has recently arisen as a Next, we surveyed 45 data scientists and asked them to promising direction to address some of these problems [17].
    [Show full text]
  • Java Makes Scripting Languages Irrelevant?
    tools of the trade Editor: Diomidis Spinellis ■ Athens University of Economics and Business ■ [email protected] Java Makes Scripting Languages Irrelevant? Diomidis Spinellis Simplicity does not precede complexity, but follows it. — Alan J. Perlis n computing, we often solve a complex came up with Application Basic as its general- problem by adding another level of indi- purpose scripting language for all its office pro- rection. For example, in Unix file systems, ductivity applications. an index node, or inode, data structure lets Those early developments acquainted pro- I the operating system allocate files concur- grammers with the notion of customizing ap- rently and sparsely while still providing an plications through scripting and opened the efficient random-access capability. When we road for powerful, general-purpose scripting want to customize large, complex systems or languages such as Perl, Python, and Ruby (see express fluid, rapidly changing John Ousterhout’s article, “Scripting: Higher- requirements, we frequently Level Programming for the 21st Century,” in add a scripting layer on top of Computer’s March 1998 issue). Scripting lan- the corresponding system. Back guages glued to applications serve an impor- in 1962 to 1963, Dan Murphy tant purpose: they greatly ease the application’s did this when he developed his configuration and customization and support TECO (text editor and correc- end-user programming by offering a safe and tor) editor on the Digital Equip- friendly development environment. Gone are ment Corp.’s PDP-1; its com- the intricacies of C’s memory management, the mand language also doubled as convoluted string manipulation, and the com- an arcane (to put it politely) plexity of the application’s internal data struc- macro language.
    [Show full text]
  • Data-Driven Tool Construction in Exploratory Programming Environments
    Data-driven Tool Construction in Exploratory Programming Environments by Marcel Taeumel A thesis submitted to the Digital Engineering Faculty University of Potsdam in partial fulfllment of the requirements for the degree of Doctor Rerum Naturalium (Dr. rer. nat.) Advisor Prof. Dr. Robert Hirschfeld Software Architecture Group Hasso Plattner Institute University of Potsdam, Germany February 2, 2020 This work is licensed under a Creative Commons License: Attribution – NonCommercial – NoDerivatives 4.0 International. This does not apply to quoted content from other authors. To view a copy of this license visit https://creativecommons.org/licenses/by-nc-nd/4.0/ Published online at the Institutional Repository of the University of Potsdam: https://doi.org/10.25932/publishup-44428 https://nbn-resolving.org/urn:nbn:de:kobv:517-opus4-444289 Abstract This work presents a new design for programming environments that promote the exploration of domain-specifc software artifacts and the construction of graphical tools for such program comprehension tasks. In complex software projects, tool building is essential because domain- or task-specifc tools can support decision making by representing concerns concisely with low cognitive efort. In contrast, generic tools can only support anticipated scenarios, which usually align with programming language concepts or well-known project domains. However, the creation and modifcation of interactive tools is expensive because the glue that connects data to graphics is hard to fnd, change, and test. Even if valuable data is available in a common format and even if promising visualizations could be populated, programmers have to invest many resources to make changes in the programming environment.
    [Show full text]
  • Exploration and Explanation in Computational Notebooks Adam Rule, Aurélien Tabard, James Hollan
    Exploration and Explanation in Computational Notebooks Adam Rule, Aurélien Tabard, James Hollan To cite this version: Adam Rule, Aurélien Tabard, James Hollan. Exploration and Explanation in Computational Note- books. ACM CHI Conference on Human Factors in Computing Systems, Apr 2018, Montréal, Canada. pp.1-12, 10.1145/3173574.3173606. hal-01676633 HAL Id: hal-01676633 https://hal.archives-ouvertes.fr/hal-01676633 Submitted on 5 Jan 2018 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Exploration and Explanation in Computational Notebooks Adam Rule Aurélien Tabard James D. Hollan Design Lab, UC San Diego Univ Lyon, CNRS, Design Lab, UC San Diego La Jolla, CA Université Lyon 1, LIRIS, UMR5205, La Jolla, CA [email protected] F-69622, Villeurbanne, France [email protected] [email protected] tion. Analysts struggle to track which of the many versions ABSTRACT Computational notebooks combine code, visualizations, and of their code produced a particular result [11, 17]. Explora- text in a single document. Researchers, data analysts, and tion often leads to dead-ends, prompting analysts to view even journalists are rapidly adopting this new medium. We code as being “throw-away” and see little point in annotat- present three studies of how they are using notebooks to ing it [17].
    [Show full text]
  • Adaptive Optimization for Self: Reconciling High Performance with Exploratory Programming
    ADAPTIVE OPTIMIZATION FOR SELF: RECONCILING HIGH PERFORMANCE WITH EXPLORATORY PROGRAMMING A DISSERTATION SUBMITTED TO THE DEPARTMENT OF COMPUTER SCIENCE AND THE COMMITTEE ON GRADUATE STUDIES IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY Urs Hölzle August 1994 © Copyright by Urs Hölzle 1994 All rights reserved I certify that I have read this dissertation and that in my opinion it is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of Philosophy. _______________________________________ David M. Ungar (Principal Advisor) I certify that I have read this dissertation and that in my opinion it is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of Philosophy. _______________________________________ John L. Hennessy (Co-Advisor) I certify that I have read this dissertation and that in my opinion it is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of Philosophy. _______________________________________ L. Peter Deutsch I certify that I have read this dissertation and that in my opinion it is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of Philosophy. _______________________________________ John C. Mitchell Approved for the University Committee on Graduate Studies: _______________________________________ iii iv Abstract Object-oriented programming languages confer many benefits, including abstraction, which lets the programmer hide the details of an object’s implementation from the object’s clients. Unfortunately, crossing abstraction boundaries often incurs a substantial run-time overhead in the form of frequent procedure calls. Thus, pervasive use of abstrac- tion, while desirable from a design standpoint, may be impractical when it leads to inefficient programs.
    [Show full text]
  • Software Engineering in the Twenty-First Century Michael R
    AI Magazine Volume 13 Number 3 (1992) (© AAAI) Articles Software Engineering in the Twenty-First Century Michael R. Lowry There is substantial evidence that AI technology can meet the require- ments of the large potential market that will exist for knowledge-based software engineering at the turn of the century. In this article, which forms the conclusion to the AAAI Press book Automating Software Design, edited by Michael Lowry and Robert McCartney, Michael Lowry discusses the future of software engineering, and how knowledge-based software engineering (KBSE) progress will lead to system development environments. Specifically, Lowry examines how KBSE techniques pro- mote additive programming methods and how they can be developed and introduced in an evolutionary way. By the year 2000, there will be a large poten- encoded form and automated. KBSE will revo- tial market and a fertile environment for lutionize software design, just as computer- knowledge-based software engineering (KBSE). aided design has revolutionized hardware In the coming decade, hardware improve- design, and desktop publishing has revolu- ments will stimulate demand for large and tionized publication design. sophisticated application software, and stan- This article draws on the chapters in dardization of software interfaces and operat- Automating Software Design and other sources ing systems will intensify competition to present one vision of the future evolution among software developers. In this environ- of KBSE. After an executive summary and a ment, developers who can rapidly create brief history of software engineering, the role robust and error-free software will flourish. of AI technology is examined in mainstream Developers who deliver buggy software years software engineering today.
    [Show full text]
  • Exploring Exploratory Programming
    2017 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) Exploring Exploratory Programming Mary Beth Kery Brad A. Myers Human-Computer Interaction Institute Human-Computer Interaction Institute Carnegie Mellon University Carnegie Mellon University Pittsburgh, PA 15213 Pittsburgh, PA 15213 [email protected] [email protected] Abstract— In open-ended tasks where a program’s behavior be difficult or even impossible to simulate in “low-fi” cannot be specified in advance, exploratory programming is a key prototyping mediums like paper or a whiteboard. Exploratory practice in which programmers actively experiment with differ- programming fills the crucial role of the medium when ent possibilities using code. Exploratory programming is highly prototyping must occur in code. This practice is key for relevant today to a variety of professional and end-user pro- situations where key attributes of exploration: “flexibility, grammer domains, including prototyping, learning through play, discovery, and innovation” [1] are needed to understand how digital art, and data science. However, prior research has largely the program should behave. lacked clarity on what exploratory programming is, and what behaviors are characteristic of this practice. Drawing on this data At first glance, exploratory programming may seem diffi- and prior literature, we provide an organized description of what cult to separate from normal programming. Typical program- exploratory programming has meant historically and a frame- ming requires some experimentation and some creative prob- work of four dimensions for studying exploratory programming lem-solving to reach a goal [4][5]. However, the practice of tasks: (1) applications, (2) required code quality, (3) ease or diffi- designing the goal at the same time as experimenting in code is culty of exploration, and (4) the exploratory process.
    [Show full text]
  • Easy-To-Learn 3D Scripting for Novices
    Chapter 1 – Alice: Easy-to-Learn 3D Scripting for Novices A Dissertation presented to the Faculty of the School of Engineering and Applied Science at the University of Virginia In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy (Computer Science) by Matthew J Conway December 1997 DRAFT COPY - LAST MODIFIED 02/02/98 10:21 AM 1 Chapter 1 – Alice: Easy-to-Learn 3D Scripting for Novices © 1997 Matthew Conway All Rights Reserved DRAFT COPY - LAST MODIFIED 02/02/98 10:21 AM 2 Chapter 1 – Approval Sheet This dissertation is submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Computer Science) Author This dissertation has been read and approved by the Examining Committee: Dissertation Advisor Committee Chairman Committee Member Committee Member Committee Member Committee Member Accepted for the School of Engineering and Applied Science: Dean Richard W. Miksad School of Engineering and Applied Science DRAFT COPY - LAST MODIFIED 02/02/98 10:21 AM 3 Chapter 1 – Table Of Contents Chapter 1 Acknowledgements ................................................................................................................7 Chapter 2 The Three Stages of Media................................................................................................ 10 Chapter 3 Executive Overview............................................................................................................ 20 Chapter 4 Alice System Overview......................................................................................................
    [Show full text]
  • Alice: Easy-To-Learn 3D Scripting for Novices
    Alice: Easy-to-Learn 3D Scripting for Novices A Dissertation presented to the Faculty of the School of Engineering and Applied Science at the University of Virginia In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy (Computer Science) by Matthew J Conway May, 1998 Alice: Easy-to-Learn 3D Scripting for Novices © 1998 Matthew Conway All Rights Reserved Table Of Contents Chapter 1 Acknowledgements................................................................................................................7 Chapter 2 The Three Stages of Media.................................................................................................10 Chapter 3 Overview ..........................................................................................................................20 Chapter 4 Working with Alice ..............................................................................................................24 Chapter 5 System Description ..............................................................................................................48 Chapter 6 The Alice Main Loop...........................................................................................................71 Chapter 7 Empirical Evidence..............................................................................................................91 Chapter 8 Traditional APIs for 3D................................................................................................... 109 Chapter 9 Objects, Parts and Children............................................................................................
    [Show full text]
  • Babylonian-Style Programming Design and Implementation of a General-Purpose Editor Integrating Live Examples Into Source Code
    Babylonian-style Programming Design and Implementation of a General-purpose Editor Integrating Live Examples Into Source Code David Raucha, Patrick Reina, Stefan Ramsona, Jens Linckea, and Robert Hirschfelda a Hasso Plattner Institute, University of Potsdam, Germany Abstract When working on a program, developers traditionally have to simulate the behavior of the abstract code in their heads until they can execute the application. Live programming aims to support the development and comprehension of programs by providing more immediate feedback on program behavior, but the divide between code and behavior often remains. The goal of example-based live programming is to remove this gap by allowing programmers to explore the actual behavior of their code during development. This is achieved by defining live examples for parts of the application. The idea of live examples has been already addressed in other tools and environments. However, most of those solutions are limited to specific domains and are suitable only for small programs. Thus, we aimto extend the application of example-based live programming to more complex programs potentially spanning multiple modules. We investigate existing solutions to derive a set of requirements for an integration of live examples into source code. Based on these requirements we propose a new approach to live examples and present a proto- type in its support. We reproduce, discuss, and extend scenarios from related work to show the practicality of our approach in the context of larger, more complicated, and with that also more realistic scenarios. Also, we measure and evaluate the system response time of our prototypical implementation. Our first results show that example-based live programming can provide more insights into therun-time behavior of parameterized code for non-trivial programs.
    [Show full text]