Reproductions Supplied by EDRS Are the Best That Can Be Made from the Original Document

Home , Mirror site, Museums and the Web, TOWeb

DOCUMENT RESUME

ED 482 045 IR 058 778

AUTHOR Bearman, David, Ed.; Trant, Jennifer, Ed. TITLE Museums and the Web 2002: Selected Papers from an International Conference (6th, Boston, Massachusetts, April 17-20, 2002). INSTITUTION Archives and Museum Informatics, Pittsburgh, PA. ISBN ISBN-1-885626-25-1 PUB DATE 2002-04-00 NOTE 252p.; Accompanying CD-ROM is not available from ERIC. For selected individual papers, see IR 058 779-800. AVAILABLE FROM Archives & Museum Informatics, 2008 Murray Ave., Suite D, Pittsburgh, PA 15217. Tel: 412-422-8530; Fax: 412-422-8594; e-mail: [email protected]; Web site: http://www.archimuse.com/. For full text:

http://www.archimuse.com/mw2002/ . PUB TYPE Collected Works Proceedings (021) Non-Print Media (100) EDRS PRICE EDRS Price MF01/PC11 Plus Postage. DESCRIPTORS *Information Technology; Multimedia Materials; *Museums; Technology Uses in Education; User Needs (Information); Users (Information); *Web Sites; *World Wide Web IDENTIFIERS Electronic Resources; *Virtual Museums; Web Site Design

ABSTRACT

This proceedings contains the following selected papers from the Museums and the Web 2002 international conference: "The Electronic Guidebook: Using Portable Devices and a Wireless Web-Based Network To Extend the Museum Experience" (Robert Semper, Mirjana Spasojevic); "Eavesdropping on Electronic Guidebooks: Observing Learning Resources in Shared Listening Environments" (Allison Woodruff, Paul M. Aoki, Rebecca E. Grinter, Amy Hurst, Margaret H. Szymanski, James D. Thornton).; "Can You See Me? Exploring Co- Visiting Between Physical and Virtual Visitors" (Areti Galani, Matthew Chalmers); "The Museum Wearable: Real-Time Sensor-Driven Understanding of Visitors' Interests for Personalized Visually-Adgmented Museum Experiences" (Flavia Sparacino); "Now That We've Found the 'Hidden Web,' What Can We Do With It? The Illinois Open Archives Initiative Metadata Harvesting Experience" (Timothy W. Cole, Joanne Kaczmarek, Paul F. Marty, Christopher J. Prom, Beth Sandore, Sarah Shreeves); "Combining the CIDOC CRM and MPEG-7 To Describe Multimedia in Museums" (Jane Hunter); "Today's Authoring Tools for Tomorrow's Semantic Web" (Andy Dingley, Paul Shabajee); "The Virtual Ramp to the Equivalent Experience in the Virtual Museum: Accessibility to Museums on the Web" (Liddy Nevile, Charles McCathieNevile); "Adding Value to Large Multimedia Collections through Annotation Technologies and Tools: Serving Communities of Interest" (Paul Shabajee, Libby Miller); "Content Management for a Content-Rich Website" (Nik Honeysett); "Here and There: Managing Multiply-Purposed Digital Assets on the Duyfken Web Site" (Marjolein Towler, Valerie Hobbs, Diarmuid Pigott); "Pyramid Power: A Train-the-Trainer Model To Increase Teacher Usage of the ArtsConnectEd On-Line Resource" (Scott Sayre, Kris Wetterlund); "Digital Primary Source Materials in the Classroom" (Nuala Bennett, Brenda Trofanenko); "Statistics, Structures & Satisfied Customers: Using Web Log Data To Improve Site Performance" (Darren Peacock); "How Do You Like To Learn? Comparing User Preferences and Visit Length of Educational Web Sites" (David T. Schaller, Steven Allison-Bunnell, Minda Borun, Margaret B.

Reproductions supplied by EDRS are the best that can be made from the original document. Chambers); "Evaluating The Featuresof Museum Websites: (The Bologna Report)" (Nicoletta Di Blas, CarolinaOrsini, Maria Pia Guermandi, Paolo Milekic); Panlini); "Towards TangibleVirtualities: Tangialities" (Slavko Environments for "Making It Realtime: Exploringthe Use of Optimized Realtime Carl Historical Simulation and Education"(Chris Calef, Turlif Vilbrandt, Vilbrandt, Janet Goodwin, JamesGoodwin); "Networked Multi-Sensory Experiences: Beyond Browsers on theWeb and in the Museum" (FabianWagmister, Web Jeff Burke); "SystematicallySpeaking: How Do Natural History Museum Vigh); and Sites Represent Science?" (RoyHawkey); "Hacking Culture" (Pia "Storytelling and the Web in SouthAfrican Museums" (Katherine J. Goodnow, CD-ROM Yngvar Natland). Authorbiographies are included. An accompanying their includes: a list of all the speakersat the conference and links to the abstracts, biographies, and papers(where available); an overview of Museums and the Web 2002 conference programand links to abstracts and paper biographies; and the results of theBest of the Web 2002 conference(requires Internet access). Most paperscontain references. (MES)

Reproductions supplied by EDRS are the best that canbe made from the ori inal document. Museums and the Web 2002

Selected Papers from an International Conference

U.S. DEPARTMENT OF EDUCATION PERMISSION TO REPRODUCE AND Office of Educational Research and Improvement DISSEMINATE THIS MATERIAL HAS EDUCATIONAL RESOURCES INFORMATION BEEN GRANTED BY CENTER (ERIC) /*This document has been reproducedas received from the person or organization originating it. D. Bearman 0 Minor changes have been made to improve reproduction quality.

TO THE EDUCATIONAL RESOURCES Points of view or opinions stated in this INFORMATION CENTER (ERIC) document do not necessarily represent official OERI position or policy.

edited by David Bearman and JenniferTrant

8EST rs0_' Museums and the www.archimuse.com/ Web 2002 mw2002/

3 Archives & Museum Informatics Consulting, Publishing and Training for Cultural Heritage Professionals

Informatics: The interdisaplinary study of informationcontent, representation, technology and applications and the methods and strategies by which information is used in organizations, networks, cultures and societies.

Archives & Museum Informatics organizes conferences, workshops and seminars, publishes journals andmono- graphs, and consults for cultural heritage organizations worldwide. For over 10 years,our educational goal has been to provide specialists in archives,museum and cultural and heritage informatics with timely and challenging opportunities for professional exchange and training. Our consulting services emphasizeinter- institutional collaboration,strategic planning andstan- dards-based solutions.

To be kept informed of coming events, conferences and new publications contact us at:

Archives & Museum Infortnatics 2008 Murray Avenue, Suite D Pittsburgh, Pennsylvania 15217 USA phone: +1 412 422 8530 fax: +1 412 422 8594 email: [email protected] http://www.archimuse.com

Ask to be put on our mailing list.

4 Museums and the www.archimuse.com/ Web 2002 mw2002/

Selected Papers from an International Conference

edited by David Bearman and Jennifer Trant Museums and the Web 2002 Selected papers from an international conference

edited by David Bearman and Jennifer Trant Pittsburgh, PA:Archives & Museum Informatics ISBN: I -885626-25- I

©Archives & Museum Informatics, 2002

Printed in the United States of America 6 Contents

Introduction

Cyberspace in Our Space David Bearman and Jennifer Trant, Archives & Museum Informatics, USA 3 The Edge of the Web

The Electronic Guidebook: Using Portable Devices and a Wireless Web-based Network to Extend the Museum Experience Robert Semper, Exploratorium, and Mirjana Spasojevic,

Hewlett-Packard Labs, USA II Eavesdropping on Electronic Guidebooks: Observing Learning Resources in Shared Listening Environments Allison Woodruff, Paul M.Aoki, Rebecca E. Grinter, Amy Hurst, Margaret H. Szymanski, and James D.Thornton, Palo Alto Research Center, USA 2 I Can You See Me? Exploring Co-Visiting Between Physical and Virtual Visitors Areti Galani and Matthew Chalmers, University of Glasgow, Department of Computing Science, UK 3 I The Museum Wearable: real-time sensor-driven understanding of visitors' interests for personalized visually-augmented museum experiences Flavia Sparacino, MIT Media Lab, USA 41 Foundations

Now That We've Found the 'Hidden Web,' What Can We Do With it?The Illinois Open Archives Initiative Metadata Harvesting Experience Timothy W. Cole, Joanne Kaczmarek, Paul F. Marty, Christopher J. Prom, Beth Sandore, and Sarah Shreeves, University of Illinois at Urbana- Champaign, USA 63 Combining the C1DOC CRM and MPEG-7 to Describe Multimedia in Museums Jane Hunter, DSTC Pty Ltd, University of Queensland,Australia 73 Today's Authoring Tools for Tomorrow's Semantic Web Andy Dingley, Codesmiths, Bristol and Paul Shabajee, Graduate School of Education & Institute for Learning and Research Technology, University of Bristol,UK 85 7

The Virtual Ramp to the Equivalent Experience in the Virtual Museum:Accessibility to Museumson the Web Liddy Nevi le, Motile Research,Australia and Charles McCathieNevile, W3C, France 93 Adding Value to Large Multimedia CollectionsThrough Annota- tion Technologies and Tools: Serving Communities of Interest Paul Shabajee, Graduate School of Education and the Institute for Learning and Research Technology (ILRT), University of Bristol, Libby Miller, Institute for Learning and Research Technology (ILRT), University of Bristol, and Andy Ding ley, Codesmiths, UK 101 PGP "Pretty Good Practice" Content Management for a Content-Rich Website Nik Honeysett, J. Paul Getty Trust, Los Angeles, USA 115 Here and There: Managing Multiply-Purposed Digital Assets on the Duyiken Web site Marjolein Towler, CONSULTAS Pty Ltd.,Valerie Hobbs and Diarmuid Pigott, Murdoch University, Australia 123 Pyramid Power:A Train-the-Trainer Model to IncreaseTeacher Usage of the ArtsConnectEd On-line Resource Scott Sayre,The Minneapolis Institute of Arts, and Kris Wetterlund, Independent Educational Write r, Arts Co n nectEd Lead Trainer 137 Digital Primary Source Materials in the Classroom Nuala Bennett, and Brenda Trofanenko, University of Illinois Urbana-Champaign, USA 149 Statistics, Structures & Satisfied Customers: Using Web Log Data to Improve Site Performance Darren Peacock, National Museum of Australia,Australia 157 How Do You Like To Learn? Comparing User Preferences and Visit Length of Educational Web Sites David T. Schaller and Steven Allison-Bunnell, Educational Web Adventures, Minda Borun and Margaret B. Chambers, Museum Solutions, USA 167 EvaluatingThe Features of Museum Web Sites (The Bologna Report) Nicoletta Di Bias, HOC-DEI, Politecnico di Milano, Maria Pia Guermandi, IBC, Istituto Beni Culturali, Emilia Romagna, Carolina Orsini, Universita di Bologna, Paolo Paolini, Politecnico di Milano, Italy 179

vi @Archives & Museum Informatics, 2002 Museums and the Web 2002

Interpenetration

TowardsTangible Virtualities:Tangialities Slavko Milekic, M.D., PhD, The University of the Arts, USA 189 Making It Rea ltime: Exploring the Use of Optimized Rea ltime Environments for Historical Simulation and Education Chris Calef,TurlifVilbrandt, Mythworks, USA/Japan, Carl Vilbrandt, University of Aizu, Japan, Janet Goodwin,Aizu History Project, USA/Japan, James Goodwin, University of California, USA 197 Networked Multi-sensory Experiences: Beyond Browsers on the Web and in the Museum Fabian Wagmister and Jeff Burke, HyperMedia Studio, School of Theater, Film and Television, University of California,Los Angeles, USA 205 Repositioning the Museum

Systematically Speaking: How Do Natural History Museum Web Sites Represent Science? Roy Hawkey,The Natural History Museum, London, UK 219 Hacking Culture PiaVigh, Director CultureNet Denmark, Denmark 23 I Storytelling and the Web in South African Museums Katherine J. Goodnow and Yngvar Natland, International Museums Programme, Univ_ersity of Bergen, Norway, in collaboration with Iziko Museums of Cape Town, South Africa 239 About the Authors Authors' Biographies 249

©Archives & Museum Informatics, 2002 vii Acknowledgements

Our sincere thanks go to all the authors of papers for Museums and the Web 2002, who once again proved that it is possible to write your paper before the conference. This rigor helps to document our activities, and contributes to our growing understanding of the intersection of Museums and the Web.

The Museums and the Web site at http://www.archimuse.com/mw2002 is maintained by Adam Brin of Archives & Museum Informatics. Please visit it to find many more papers, live links to sites described, and full descriptions of the demonstrations and workshops presented at MW2002 and all previous Museums and the Web conferences.

This volume was copy-edited by Sheila Trant.

The Museums and theWeb conference was managed by Helen Goodman of Event Resources, Pittsburgh, PA.

10 Introduction

11 Cyberspace in Our Space

Introduction Since the Museums and the Web Conference began in 1997, it has examined the confluence between museums, as social institutions with a certain historical tra- dition and mission, and the "Web", a set of rapidly evolving technologies that combined image, text and multimedia hyperlinking with Internet addressing. Al- ready it is clear that these forces affect each other. Museums have not gone "on" the Web simply to recreate themselves in cyberspace or to become merged with other virtual information stores. Instead, they are beginning to appropriate the technologies of theWeb. Museum professionals have engaged the Web as a venue for the exploration of issues they face daily, and increasingly they have found in the Web tools that are of use within the museum.The Web is a becoming part of our physical space just as the museum is re-creating itself in cyberspace; the dialogue between the two is reshaping our sense of museums and museolou

This volume of papers presented at the Museums and the Web 2002 Conference explores the many faces of museums on the Web and the new face of the Web in the museum.The authors probe the many roles of the museum as a social institution, as a series of programs executed in physical space and time, as a collection of objects and knowledge, or as a node in the Web.

Insofar as it proves possible in the coming decades to deploy emerging Web standards to enable museum-like mediation between objects and experience, interpretations of meaning and sources of knowledge, museums will be significant players in the semantic Web. But as the papers in this volume demonstrate, the path to interoperability will first require syntactic agreements and adherence to structural standards for metadata harvesting declarations.These foundations for interoperability are the precondition for broadly available toolsets and newly developing conventions of interaction. The access such standards enable for the handicapped,for scholars,for the general public and for two-way communication between creators and participants is being explored. But the underlying assumptions that Web standards will in fact support semantic interoperability and that museums will achieve a mature respect for standards have yet to be proven. Both assumptions are at this point tenuous: museums have been more willing to "adapt" standards than to "adopt" them, and efforts to create structures for universal meaning, or even methods to find generalizable meaning in local perspectives, have so far proven elusive.

Whether knowledge of our heritage is ultimately integrated into a "world wide" Web of knowledge, or not, museums have embraced the Web and are mastering the management methods, pedagogical techniques, and assessment strategies needed to support effective institutional programming. Leading museums have introduced structures, technologies and business relations from the world of commercial organizations into museum "content management", and have professionalized theirWeb operations.Where home-grown programs once reigned, professionally planned, inter-institutional, technologically sophisticated Web-based programs raise a challenge. Most importantly, a rigor in evaluation rarely seen in the assessment of non-technologically mediated museum programming is being

1 ()4.. Museums and the Web 2002:Proceedings Archives & Museum Informatics, 2002, p. 3 BEST COPY AVALABLE Bearman &Trant, Cyberspace in Our Space

introduced, and indeed, expected, in Web programs. Improved methodologies for evaluation in the initial planning, pre-testing and post-implementation phases are being implemented regularly. Museum management is coming to expect an explanation of the relative effectiveness of different means of interpretation. In a relatively short time, Web-based feedback mechanisms have helped museums tine-tune the instruments of their communication using sophisticated evaluation tools.

Museums' successes derive increasingly from integration of theWeb into themuseum. Not only is this affecting work processes such as content management, education and evaluation; but on-site exhibitions and interpretation are also beginningto benefit from theWeb-based delivery of context for objects on exhibition.TheWebas a potential threat to the on-site, physical, real world museum is confronted, by moving the virtual exhibitionlocated somewhere else, in cyberspace,and most often approached from somewhere other than the museum homepagecloser to the physical space; in- gallery interactives have begun to bridge the gap between the virtual exhibition a separate venue requiring distinctive content, and competing for funds and staff en- ergy and ongoing museum exhibition programming, with all the strengths of the museum's identity and physical presence. Web-driven handhelds married to wireless communications facilitate interpenetration of the virtual and the real.This marriage of physical and conceptual space is inspiring experiments, especially by university-based researchers, and generating a new social and intellectual construct:mixed real4While it is too early to know how people will ultimately integrate mixed reality into their everyday life, museums and academic researchers working withmuseums are modeling the options.

Museums have adopted the Web in as many different ways as any other social institutions in our society.Already there are some signs that the museum as an institution may be significantly changed by the encounter. Just as commerce is not fundamentally affected by B2B applications (which make commercial transactions more effective) but might be fundamentally altered by reverting to on-demand custom fabrication (as with Dell Computers) or the social exchange of free information (as with Amazon),so also museums could be fundamentally altered if other types of institutions usurp their role as custodians and gatekeepers of art, impinge on their position as collectors and inter- preters culture, or offer methods that make possible the aggregation of information from many sources. Or the Web could provide tools for museums to reconnect with their communities, reasserting their relevance by empowering their visitors, virtual and physical.

The Edge of the Web This year, the edge of the Web is a wedge that penetrates museum galleries; wireless computing and handheld devices have moved the Web from cyberspace intomuseum space. The first section of the Museums and the Web 2002 Proceedings reports on several projects exploring the potential for innovative interpretation ofmuseum objects and ideas, delivering networked digital objects to people using handheld devices in wireless networked spaces and watching as those people communicate with each other, with the "smart space" around them and with linked knowledgesources.

Robert Semper from the Exploratorium and Mirjana Spasojevic of Hewlett-Packard Laboratories introduce the state of the art in the "electronic guidebook",a metaphor

1 3

4 ©Archives & Museum Informatics, 2002 Museums and the Web 2002

and a physical, portable device that uses a wirelessWeb-based network to "extend the museum experience". As these devices are becoming smaller, more affordable and more capable, museums will soon be able to count on them to facilitate delivery of complex information to visitors.What is not yet clearand is the subject of this and other researchis what kind of information should be delivered,to whom and at what time.We still have much to learn about how this can be done in a way that attracts attention to the artefact or idea we are communicating, and enables communication among visitors rather than inhibits it.There have been several attempts to implement full-blown hand-held guides within galleries (including those at SFMOMA reported by Peter Samis at the ichim 2001 meeting in Milan, Italy); Semper and Spasojevic step back to examine requirements for success in a systematic fashion, considering infrastructure, interface and content development.Two problems are identified in this research: I ) the potential for interference with the social experience of visiting, as the device gets in the way of communication between visitors, and 2) actual physical interference, as holding the device impairs visitors' ability to interact with the exhibit.

At PARC, a group of researchers specifically addressed the social issues identified by Semper and Spasojevic, exploring the relative advantages and disadvantages for social interaction of different methods of delivering audio in a gallery tour environment, including one where visitors shared similar aural experiences.Their conclusions point both to new strategies for technological mediation and to the need for further studies of computer-human interface options and effects.At the University of Glasgow, Galani and Chalmers conducted a systematic analysis of the social context of the museum visit, exploring how it can be affected by technology.They studied ways in which different visitors to an exhibit actually interact together once they are there, with each other, the gallery, the Web and the virtual environment.Their results similarly point to technologies that could substantially overcome the barriers to social interaction in a Web-facilitated visit, and could possibly integrate Web and physical visits.

The second barrier is physical. In her exploration of the past and potential of the "Museum Wearable", Flavia Sparacino of the MIT Media Lab examines ways in which wearable computers could augment the gallery environment, enhance visitors' experiences, and in the future even enable visitors to use their hands freely.

Foundations

While museums may have been conceived as independent sources of interpreted knowledge, theWeb has pointed out the weaknesses of this isolation for the construction of complex understanding. It takes little advantage of the inter-relationships between and among disparately located museum objects.Together museums' collective knowledge can only be identified, navigated, explored, and integrated if its structure is explicitly declared. Interoperable knowledge models could drive tools to enable access across barriers of differing abilities, disciplinary perspectives, and documentation traditions. But the levels of standardization that museums will need to adopt, minimally to move data successfully from place to place (without carrying much meaning), to say nothing of the inherent complexity of the project of moving information created in one intellectual perspective into meaning in another context, are daunting.There is much scope for research in this area, and the papers in the "Foundations" section touch on significant areas of standards definition and application, in metadata harvesting, data modeling, the semantic web, accessibility, and annotation.

is 14 ©Archives & Museum Informatics, 2002 5 Bearman &Trant, Cyberspace in Our Space

The metadata harvesting experiments conducted by the multi-departmental research team at the University of Illinois begin to peel back the layers of standards infrastructure required to easily integrate information from disparate technical environments, media formats, and disciplinary perspectives, and to deliver it tousers with different capabilities, knowledge and tools. In the "Illinois Open Archives Initiative Metadata Harvesting Experience", supporting the OAI server and harvesting technologyper se is not even seen as a fundamental barrier most museums couldn't overcome.Attention is focused on the next level of the problem that of standardizing the content of metadata, which is shown to be quite difficult even when a very simplecommon de- nominator, the Dublin Core, is all that is attempted.

Jane Hunter tries to bridge a much more complex set of metadata models for knowledge representation in mapping the CIDOC Conceptual Reference Model (CRM) designed to describe museum data to MPEG-7 designed to describe multimedia, in order to describe multimedia museum content. Hunteruses the Resource Descrip- tion Framework (RDF) in order to express the enhancements neededto the CIDOC Model to incorporate critical MPEG-7 concepts.Andy Ding ley and Paul Shabajeepur- sue the theme of content integration as they envision how today's authoring tools, including XML and RDF, can be used to construct the "semantic Web" envisioned by the W3C. Using the ARKive project as their test-bed, theyare prototyping the interoperable knowledge environment of the future. Liddy Nevile and Charles McCathieNevile take the dialog a step further as they introducea series of W3C standards designed not simply to enable users of theWeb tomove across data sources, but also to enable access in different modalities and with different enhanced interfaces for users with disabilities.

An alternative approach to building a semantic network is explored by the teamat the University of Bristol in their examination of the use of annotation to enable the collaborative indexing of a common data store according to the perspectives of various communities of interest. Using a relatively low technology approach in place ofauto- mated tools and highly sophisticated mechanisms for translating between schemas, they propose a cooperative, social means to bridge intellectual perspectives anden- able cross-disciplinary access.

PGP (Pretty Good Practice) After less than a decade, it is evident that museum new media practice for the Web has introduced new rigor throughout the process of managing content, delivering learning experiences and evaluating results. Indeed, we now have"pretty good practice"if not "best practice" (something we will probably never achieve and mightnot recognize if we did). Like pretty good privacy, it's the best we've got.

Two papers in this section address the need that museums, and other organizations with lots of data, have to manage their "assets" using sophisticated tools that make possible the maintenance of lively, timely, and rapidly changingWeb sites.The methods introduced by the Getty, to integrate data from its museum andmany research cen- ters, and by the Duyfken project,to bring content from numerous sources together on an hourly basis, are indicative of the types of content management practices that are becoming necessary everywhere. 15

6 ©Archives & Museum Informatics, 2002 Museums and theWeb 2002

The purpose, of course, is to create sites that have the depth to be attractive and the range of digital objects to appeal to a variety of different visitorswhat is referred to elsewhere in this volume as having "a low threshold and a high ceiling" (Nevi le and McCathieNevile). Pretty good practice at this point is to develop programs that bring educators in the classroom and in the museum together to develop strategies for enabling and improving use. In "Pyramid Power", Scott Sayre and Kris Wetterlund report methods of teacher training used to introduce teachers to museum resources and explore how they can make good use of them.The power of their model comes in the 'train the trainer' cascade. In their exploration of issues surrounding the integration of digital primary resources in the classroom, Bennett and Trofanenko describe a collaboration among teachers and a number of cultural heritage organizations.

In the end, the question is whether the investments we are making, in teaching tools and in rich museum Web sites, really pay off. Darren Peacock proposes a framework and suggests quantitative measures from Web log data to assess user satisfaction with the Web site of the National Museum of Australia.A group of commercial designers and museum educators turn to "Comparing User Preferences and Visit Length of Edu- cational Web Sites" in a qualitative study that bridges learning theory and interactive media development. And a group of Italian academic software engineers apply their design-based metrics to the assessment of some well-known museum sites with results that are informative for anyone planning an evaluation or designing a site.

Interpenetration As the edge of the Web insinuates itself into museum space, university researchers are developing new tools and techniques to wrap cultural content in digital context and present it in an engaging and intuitive manner. The results challenge the boundary between the virtual and the real, and ask us to consider how we might best engage with culture, digitally.

Physician and experimental psychologist Slavko Milekic is defining a new domain Tangialitythat bridges Reality andVirtuality Here museum data and museum visitors may soon come together through haptic interfaces.At the University ofAizu, researchers are exploring the use of tools and metaphors developed for games to make visual- izations of historical simulated environments for museum interpretation.And at UCLA, in the Hypermedia Studio of the School of Theater, Film and Television, installation artists are constructing environments where performers, in real and virtual spaces, interact with audiences that are vital participants. In each of these mixed reality environments, real people, digital input and data interact in an environment, respond, and adapt.They suggest that the interpenetrated world of mixed-reality is, increasingly, the world that museum programming and the use of the Web will exploit.

Repositioning the Museum So, will this change the museum? Has it changed the museum? Communications and media theory predict that the introduction of new media could fundamentally alter the museum message. Indeed, after a relatively few years in the life of a fairly conserva- tive type of institution, it is becoming apparent that theWeb can affect the fundamental nature of museums.The Web is not just a brochure to advertise the museum (as it was in the first couple of years), or an alternative venue in cyberspace in which to imple-

ment museum programs (as it has been for the past couple of years) .The final section of this book offers tantalizing hints about the impact the Webmay have on the nature of the museum as a social institution, its mission and purpose. If,as PiaVigh suggests vis vis net art in "Hacking Culture", the medium is the message and themessage can only be conveyed in a new social context beyond the museum, then the Web will have proved a serious challenge to museums. But if, as Roy Hawkey suggests, the medium makes possible the communication of a message that had been subjugatedto a minor position in traditional museums where it failed to fit into the medium of exhibition display, then theWeb could liberate the museum to realize science education ina way that was previously constrained. Or perhaps, as in the culturally sensitive exploration of potentialities of Web interactions with museum missions elaborated by Goodnow and Natland, the museum can realize a new mission by grappling with precisely this complex interaction between what a museum representsas an institution and what museology might enable.

And remember...

This volume is entitled Selected Papers from Museums and the Web 2002. Weare delighted that it reflects the rich discourse that characterizes the conference and this community, but as the choice of which conference papers to print becomes evermore difficult, we remind readers that four times as many papers from the conferenceare included on the CD-ROM that accompanies this book. And even the CD doesnot fully represent the meeting.To appreciate the entire range of creative discussion that accompanies this annual encounter between the Museum and the Web, it would be necessary not just to review all the papers, but also to experience the 14 pre-conference workshops, 50 demonstrations, 35 exhibits and 15 mini-workshops,as well as the performances, Crit Rooms and Usability lab- interactive eventswhich are not documented on-line or on the CD. Ultimately, a conference is a social event:the experience cannot be fully represented in any publication. Nevertheless, we hope this volume stimulates more research and experimentation, and yetmore publication about Museums and the Web.

David Bearman and Jennifer Trant Archives & Museum Informatics http://www.archimuse.com

8 ©Archives & MUseum Informatics, 2002 The Edge of the Web

13 The Electronic Guidebook: Using Portable Devices and aWireless Web-based Network to Extend the Museum Experience

Robert Semper, Exploratorium, and Mirjana Spasojevic, Hewlett-Packard Labs, USA

Abstract

Recent advances in wireless network technologies create the potential to significantly enhance the experience of a visit to a museum. On the exhibit floor, visitors carrying wirelessly connected portable devices can be given opportunities for exploration, sharing, explanations, context, background, analytical tools, and suggestions for related experiences. When these devices are part of a Web-based network, they can help extend the museum visit: in advance, through activities that orient visitors; and afterward, through opportunities to reflect and explore related ideas. The Electronic Guidebook project is a study of visitors equipped with such technologies, conducted by the Exploratorium in partnership with researchers at Hewlett-Packard Labs and the Concord Consortium.The project is investigating how a Web-based computing infrastructure can provide museum visitors with an augmented museum experience so that they can better plan their visit, get the most out of it while they are in the museum, and be able to refer backto their visit once they have returned to their home or classroom. The goals are to understand what technological infrastructure supports this extended museum experience, and to obtain preliminary data on how different aspects of the technologies, and the content delivered through them, affects engagement with the exhibits and pre- or post-visit learning activities. Keywords: handheld computers, wireless networks, electronic guides, mobile computing devices

Background The project created a test-bed and tested a net- learning experience with exhibits would include ways work using a variety of handheld computers and of capturing the visiting experience for later reflec- radio-frequency identification tags to link visitors tion, being able to access additional material that with exhibit-related content delivered by a Web- provides a context for the exhibit, and extending based server.Visitors in the study were able to ac- the interaction with the exhibit beyond simple ma- cess Web-based content, including text, images, nipulation. If museums are to fulfill their potential video, and audio, during a visit. In addition,they were role as multi-dimensional educational institutions, able to construct a record of their visit by book- they need additional mediation techniques that sup- marking exhibit content, creating images, notations, port these needs. And these techniques need to and other artifacts, and to access that record on a augment the simple exhibit experience without de- personalWeb page in the museum or following their stroying the informal ambiance that is the hallmark visit. of museums. In effect, what is needed are tools that can provide the kind of contextual support with Rationale exhibits that is offered by the familiar travel guides but defined in a new interactive way. The exhibit environment in museums excels in providing the public with direct experiences with phe- Guides are an important part of our travel experi- nomena, ideas, and objects. But while the exhibits ence. We use them in advance to plan our trips. provide compelling opportunities to stimulate in- They provide us with an historical context for the quiry and exploration, they are not, by themselves, country or city we are going to see.We take them completely successful in supporting conceptual along to provide quick access to essential informa- learning, inquiry-skill-building, analytic experiences, tion like addresses and transit information.We use or follow-up activities at home or school.An ideal them as in-depth guides to tours of historical land-

BEST COPY AVAILABLE Museums and theJgeb002: Proceedings Archives & Museum Informatics, 2002, p.II Semper and Spasojevic,The Electronic Guidebook

Before Visit Durin,g Visit After Visit Orientation Information Reflection Registration Context Analysis Background Experimentation Extension

Wireless Computer at Computer at handheld device home, school or home, school or in the exbibit museum museum sPaee

Network server with exhibit-related content and personalized Web pages

Fig. 2:The Museum Experience Extended through a Wireless Web-based World

orient visitors, and afterward through opportunities to continue reflection and explore related ideas. Fig.Visitors using the Electronic Guidebook in the Exploratorium By adding personalized technology to the museum experience in these ways, we can greatly enhance marks.They help us translate words and exchange the visit's educational value. currency during our visit.We personalize them with notes, hotel brochures and postcards from the places we have stayed.We refer to them after the Research Project trip to remind us of our visit, and they form the The Electronic Guidebook project (http:// basis of talking about our trip with our friends.The www.exploratorium.edu/guideboold) is a National best guidebooks become dog-eared from use by Science Foundation funded activity designed as a the end of a trip and become keepsake reference proof of concept research study to explore the points for whenever we "revisit" the journey. In potential of portable computing devices and a wire- short, they serve as an ongoing tool for enhancing less Web-based network. Headquartered at the our direct visit experience with a place and culture. Exploratorium (http://www.exploratorium.edu),the project is a partnership of the Exploratorium, Con- Emerging computer and network technologies have cord Consortium, an educational research and de- the potential to significantly enhance the experi- velopment organization, and HP Labs, the research ence of a visit to a museum, giving visitors the op- and development laboratory of Hewlett-Packard portunity to transcend the traditional limits of time Company. The Exploratorium worked with the and physical space as they engage in a museum's Concord Consortium to identify the technical is- offerings. Electronic guides, enhanced via interac- sues and potential uses of handheld devices in the tive technology, can not only serve as reference in- museum setting, and the museum formed a part- formation, but also provide expert guidance, dy- nership with HP Labs to develop and test a Web- namic advice, recommendations for further inquiry, based, wireless network deployable in the and other learner supports no paper-based travel Exploratorium' physical space.The aim of the project guide can provide. On the exhibit floor, additional is to investigate how a mobile computing infrastruc- information, guidance, or feedback at the right time ture can enable museum visitors to create their own can turn a simple interaction with an exhibit into a "guide" to the Exploratorium, using a personalized, meaningful learning experience. Inexpensive por- interactive system to better plan their visit, get the table devices can give visitors opportunities for most out of it while they are in the museum,and be exploration, sharing, explanations, context, back- able to refer back to their visit once they have re- ground, analytical tools, and suggestions for related turned to their home or classroom. experiences. The Electronic Guidebook project focused on three The visit can be extended through sustaining learn- °simultaneous strands of investigation: ing opportunities: in advance through activities that 20 BEST COPY AVAILABLE 12 ©Archives & Muteum Informatics, 2002 Museums and theWeb 2002

I. information technology infrastructure (networked components delivering the information); 2. human computer interface issues (form factors, interfaces, and usability issues for various audiences); and 3. content development (design,formatting,and topi- cal content of deliverable information).

The project was designed as a proof of concept study to explore potential avenues for future research and development. It was not envisioned to support the implementation of a fully functional Pi-station: beacon (Infrared), barcode, RFID system, but rather to point the way for future de- plus processing, 110 and networking velopments.A key idea was to create the test-bed within the existing museum context and use off- Fig 3: Setting:The Augmented the-shelf technology to the extent possible so as Interactive Museum) to be able to explore this technology in a realistic setting. Often technology projects are tested in ide- of sensing technologies such as infrared receivers alized conditions without the distractions and the and barcode readers.Those readers are integrated messiness that occurs in the real world. Our goal with their handheld device, which is wirelessly net- was to study the interaction of all of the elements, worked. Users access electronic services by using technologs functionality and content, within an au- handheld devices to pick up URLs from barcodes thentic situation. or infrared 'beacons' attached on or near the objects of interest.Those services are provided using The project worked with an existing set of museum wireless Web technology: Web browsers on the exhibits that are part of the "MatterWorld" exhibit handheld devices and Web servers in the environ- area.These exhibits that addressed topics in sound, ment.The services can be adapted for users based mechanics and heat are highly interactive, distrib- on their context, e.g. their identity, location, device uted in space,and individual in design. Because these capabilities, personal interests and preferences. exhibits have been in the museum for many years, there exists a rich set of supplemental materials The Electronic Guidebook project test-bed con- (text, images, teaching activities, exhibit developer sists of a wireless handheld carried by the visitor, a interviews, etc.) already developed.While this ma- "pi station" (point-of-information station) at each terial could provide a rich context for the visitor, it exhibit, and a content server connected to the is usually not readily accessible because it is scattered World Wide Web on the Internet and accessible in files, offices, and the brains of staff members. both inside the museum and from home or school. The visitor located in the museum space was identified using the handheld by a short range IR trans- TheTest-bed mission, an RFID sensor (http://www.aimglobal.org/ Initial developmental studies which tested different techologies/rfid/), or a barcode swipe. Data trans- devices, networks and servers were pursued in 2000 mission to the handheld was provided using a radio by Concord Consortium and the Exploratorium. LAN system in the exhibit space that provided con- For the testbed we augmented the museum envi- nectivity to a Web server which is connected to ronment with technologies developed by the the public Internet. Cooltown research program at HP Labs (http:// cooltown.hp.com) [Kindberg (2001); Kindberg Each pi station is a tall stand with a base that can (2000)1. In the Cooltown scenario, all physical enti- contain a laptop and a panel at eye level that can ties (people, places and things) have a 'Web pres- contain an IR beacon (http://cooltown.hp.com/ ence' [Debaty (2000); Kindberg (2002)]. Nomadic beacon_full.htm), RFID reader, barcode, and/or cam- users navigate from the physical to the virtual world era.At each pi station,the visitor can"sense" the URL by picking up links toWeb resources using a variety of the main page for the corresponding exhibit.The

©Archives &Muse2 iformatics,2002 13 BEST COPY AVAILABLE Semper and Spasojevic,The Electronic Guidebook

cuphr.Oioot 1.10 At)f

Arileles c.d.* loccefioot

Fig. 4: Handheld devices L to R HP jornada 548, HP fornada 690, Hitachi ePlate)

page is then retrieved from the Web server and displayed with a browser on the handheld.The pi- station was designed to help test different interaction modalities and to facilitate working with different exhibits for different tests. In future system design, some of this functionality might be built into the exhibit itself.

Battery 129, in the Mann Headland. is an excellent place to hear echoes Pam Three types of hand-helds have been used for the suns in the tunnels over there all the tune studies: HP Jornada 690's and 720's (a Hand-held (Mk rn *onmovip) 0 PC with a physical keyboard), HP Jornada 540's (a Pocket PC with on-screen keyboard), and Hitachi ePlates (touch-screen tablets).The hand-helds are 4BACK connected to the network using 802.1 I b cards which provide wireless connectivity at II mbps.A Fig. 5: Echo Tube Pages single 802.1 I b base station is connected to the Exploratorium's network near the exhibits. walk up to a particular exhibit, the guidebook provides interaction, information, and ideas about the The Jornadas have IR communication ports and exhibit.When approaching the Exploratorium's Echo Esquirt software that allows them to collect URLs Tube exhibit, for example, visitors see a small Web from Cooltown IR beacons.When the visitor points page on their handheld guidebook, suggesting ways the Jornada at an IR beacon, the beacon transmits of interacting with the tube, such as clapping and to the Jornada the URL of the on-line content as- counting to measure the speed of sound.The guide- sociated with that beacon.A barcode reader is at- book also provides visitors with intriguing questions tached to the Hitachi ePlate.The raw ID provided to consider (such as,"What does a dog bark in the by the barcode is mapped to a URL, which is then tube sound like?") and a way for them to enter their treated like the URL's provided by beacons. own questions and observations (which other visitors can then access). In addition, visitors can learn Information and contextual material for each ex- more about the science of echoes (with deeper hibit are displayed on the hand-helds using a modi- explanations than available on the exhibit label), real- fied Web browser.When the visitor picks up a bea- world connections (for example, recommended con or barcode for an exhibit, the main page for locations in the San Francisco area for hearing ech- that exhibit is automatically displayed.The browser's oes), and the history and evolution of the exhibit user interface has been simplified so that the only itself (including stories from the exhibit develop- browser buttons are forward, back, home, and a ers). The system allows visitors to save items of bookmark, which we added to allow visitors to interest to a personalized Web page that they can record visited pages in their personal scrapbook. access after their visit at home or school.

Using the System All exhibits have a main page whose URL is stored in the pi station, as well as 3-4 sub-pages.When the Visitors carry the electronic guidebooks with them main page gets downloaded by the browser (after as they walk through the museum floor. As they picking up a URL through a beacon or a barcode), 9 O. 14 @Archives & Museum Informatics, 2002 BEST COPYAVAILABLE Museums and theWeb 2002 visitors can click on any part of the image to get to What are the issues surrounding the use of these the "nuggets" page which provide links to several physical devices during a visit to an exhibit envi- areas for further exploration of content related to ronment? How can we integrate the devices into the exhibits. the complex social activity of a museum visit without fragmenting a group? How can they be com- In addition to the individual exhibit pages, visitors fortably handled during a prolonged visit? can access the Electronic Guidebook home page. The home page contains a list of exhibits and a map What is an appropriate software, hardware and of exhibit locations. It also gives visitors access to a exhibit graphics interface design for the experi- personal scrapbook page on the server where they ence at the exhibit, at home before or after a can record URLs of interesting material which can visit and in the Learning Studio as part of the be reviewed later on theWWW from any browser. museum visit? What are the important accessibility issues? For example, how many visitors have Web access? For those who do not have Web Research Studies access at home, in what ways can we provide them The research team that is studying the use of the with closure to their experience? test-bed is made up of museum educators, scientists, educational technology researchers, learning What are the key hardware and software design researchers and human-computer interface re- issues (screen size, speed of use, amount of ma- searchers. The project started with a set of ques- terial) that form a threshold of feasibility for this tions that the test-bed was designed to help an- idea?What are the specific design elements (text, swer. graphics, etc) which are important for the experience? What are the specific mediation compo- Technology Development nents that this technology can support at exhibits (capturing information, providing context, cre- What is the feasibility and effectiveness of vari- ating representations, taking data)? ous wireless receiving and transmitting solutions for an exhibit space including distributed and/or Educational Potential centralized infrared and radio transceivers? What is the effectiveness of the Electronic What is the appropriate distributed computing GuideBook system for capturing experience, pro- power relationship between the handheld device, viding context and extended interaction? How the exhibit computer, the transceiver system and can we create resources that stimulate explora- the network servers? tion and reflection rather than having the handheld just seem like an electronic textbook? What is the optimal integration of the dynamic user database in the Web environment? What kinds of learning can we facilitate: Cogni- tive? Affective? Attitudinal? Skill-based? What are What are the appropriate handheld device data the key issues that might motivate the use of these sensors for the museum exhibits and the design devices? How can we create resources that stimu- issues surrounding the interface between these late exploration and reflection? sensors and the exhibits themselves? Interface Design What is the value of the different project components to the overall educational experience (i.e. What is a feasible system for providing handheld pre visit, at the exhibit, in the museum's class- devices to the public? How can we distribute room spaces, in the home after the visit)? these devices equitably and to a diverse cross- section of our visitors? Will the devices be used The initial work has been focused on developing by individuals or by multiple visitors in a group? the test-bed and initiating the study with a series of How will security be handled? What are the lo- research questions.These studies have focused on gistics of carrying them on the exhibit floor? the following ideas:

©Archives & Museum Informatics, 2002 I 5 Semper and Spasojevic,The Electronic Guidebook

What are the basic affordances? For example, is female).The adults ranged in age from about 25 to the device easy to carry?Are screen graphics easy about 50.All users were fluent speakers of English to interpret? Is the wireless connectivity reliable? with no major physical disabilities. Is the 1-2 meter range for reading a Coo !town beacon more convenient for users than theI - The evaluation approach selected for the tests was I Ocm range for scanning a barcode? participant observation with a project member shadowing a visitor or a small group of visitors.Visi- What is the visitors' attention to artifacts? That tors were directed up to the general area of the is, how much attention do the users pay to each instrumented exhibits.They interacted with those exhibit or piece of on-line content, and what is exhibits and sometimes nearby non-instrumented the quality of this attention? For example, did the exhibits. The project members observed the visi- users look at a particular page? If so, did they tors' actions and reactions, and helped them out glance at it or read some text on it, for example? when they had major problems. Semi-structured Were the users mainly paying attention to the interviews were held afterwards.The tests were done exhibit, paying attention to the content on their when the museum was open to other visitors. handheld device, or fiddling with the device itself (e.g. trying to solve some problem with the in- For these initial tests, we used HP Jornada 690's, terface)? except for one subject who used an Hitachi ePlate. 1n-depth content has been developed for six exhib- What are the visitors' paths through physical and its, including one pair of exhibits that are themati- virtual space? What is the sequence of points in cally related to one another. Most of the pages have physical space (exhibits) and virtual space (VVeb heavily graphical content, with a few having mixed pages) through which the users pass as they visit text/graphics content and parallel audio content.The the museum? Of particular interest are trajecto- content is formatted for the Handheld PC display ries that correspond to high-level user events such size.A stand was placed next to each exhibit, with a as following a virtual link to get more informa- barcode and IR beacon containing the appropriate tion on a topic, deciding to walk over to another URL for each exhibit's content. Content was de- exhibit, shifting attention back and forth between signed primarily for an adult audience. the physical and virtual at a single exhibit. Lessons Learned The next set of studies (currently underway) are designed to help us better understand how differ- The overall reaction from the visitors was positive. ent types of content (including different multime- They enjoyed reading the on-line content. Some dia formats, types of suggested activities, and the adults felt that the content would not be interest- ability for visitors to add their own observations ing to children. However, children who used the and ideas) affect users' experience at the system did actually spend a fair amount of time read- Exploratorium and in what ways mobile electronic ing the on-line content and trying out possibilities resources support or hinder their experience in suggested by it.Visitors frequently commented that the museum context. Also planned are studies of the hand-helds were really fun, novel devices. On the "out of museum" use of the network to ex- the one hand,this helped keep them interested. On plore its utility before and after a visit. the other hand, the "wow" factor of using the device distracted them from paying attention to both Overview of Studies the exhibits and the on-line content. To date, we have carried out several rounds of in- Above all, visitors liked the idea of being able to formal visitor studies.The goal of these tests was bookmark information to look at later. Both teach- to get a general sense of how visitors reacted to ers and children thought this feature would allow the system and to uncover major issues.The users the children to play more during their museum visit, for these initial tests were 12 local school teachers completing related homework assignments after the (9 female, 3 male), 14 other adults (8 male, 6 fe- visit. male), and 8 young people, ages 10-17, (5 male, 2 24

I 6 ©Archives & Museum Informatics, 2002 Museums and the Web 2002

Aside from technical problems discussed below,the playing. Headphones can be used, but the Jornada main negative comments were that using the hand- 690 happens to lack a headphone jack (a problem held tended to distract people from playing with fixed in the Jornada 720).Therefore,the visitors had the exhibits. The issues here are both mechanical to be prompted, by on-screen graphics, to hold the and cognitive. Holding the handheld prevents people hand-held up to their ears. from freely using their hands to manipulate the exhibits. Moreover, reading the content is itself dis- Robustness issues are a concern for use in an inter- tracting from interacting with the exhibit. active science museum. Robust devices are essential if on-line content is to be seamlessly integrated Using hand-helds in a museum setting may also dis- into everyday activities.The hand-helds struck most rupt normal social interactions between members visitors as too fragile to withstand being dropped of social groups, because each visitor can get lost in or bumped. Many exhibits at the Exploratorium in- the world of the device and pay less attention to volve materials like sand and water,which handhelds the rest of his group. For example, we observed were not designed to withstand. two boys moving around the museum floor together, talking about the exhibits and working things out The wireless network performance was adequate together.Yet when we gave them each a handheld, for downloading Web pages and short audio files. one spent more time reading the information on However, network performance can be disrupted the device while the other was still interested in by obstacles such as elevator shafts. It is essential seeing as much as he could. On some occasions, to test network performance in the place where it when they needed two people to make an exhibit will be used and to check carefully for possible work, they had to spend time trying to find each "holes" where performance is bad. Multiple wire- other again. We believe that these issues can be less base-stations may be required in such a case. overcome by designing a system to specifically support visitors' conversation patterns, such as the Picking up Beacons and Barcodes system tested at an historic house [Aoki (2000); Woodruff (200 I)] where the electronic guidebook We found that visitors were able to sense beacons became a third party in the visitor conversation. or barcodes with the handheld devices with little trouble. Other Cooltown demos have used a beep to indicate successful beacon or barcode pickup. Handheld design and features Since this sound cannot be heard in the Hand-helds do not routinely come with straps. Car- Exploratorium, visual feedback must be provided rying cases are only available for certain models. instead. This represented a serious problem in the Exploratorium because visitors must have one or Likewise, we found that a brief hands-on introduc- both hands free to operate the exhibits.When early tion is required: sending visitors off with only ver- trials made it apparent that a strap was necessary, bal instruction isn't sufficient. However, all visitors we had to resort to tying a strap around the Jornada became fluent with the devices quickly. The one 690's hinge. However, even with a strap attached to person who used a barcode reader behaved simi- it, a Hitachi ePlate was too heavy to use for a pro- larly to those using beacons. longed time during the visit. The visitors never accidentally picked up an IR bea- Our original plan involved comparing audio rendi- con for a different exhibit. However, some visitors tions of content to parallel graphic versions, as in a accidentally picked up an exhibit's beacon repeat- recent handheld project at an historic house (Aoki edly, because they happened to hold the Jornada so 2000 and Woodruff 200 )This study found that visi- that its IR port pointed at the beacon while they tors preferred audio content to visual content, pre- were browsing the on-line content.We have since sumably because it left their eyes free to look at added features to the browser to minimize dupli- the exhibits. However, the Exploratorium is ex- cate pickups by ignoring them if they occur very tremely noisy.When the Jornada is held in the hand, soon after original pickup, and by querying the visi- sound from its speakers is essentially inaudible.Some tor about whether to redisplay the exhibit top-level visitors were not even aware that any sound was page if the delay is longer.

@Archives &Museunaarmatics,2002 17 Semper and Spasojevic,The Electronic Guidebook

Graphical User Interface However, other visitors may wish to follow a tour or see a group of thematically related exhibits. Or In these studies, we gave visitors only extremely the on-line content may suggest another exhibit brief instructions on how to use the browser inter- closely related to the one they are at. It is unclear face.Visitors familiar with hand-helds and browsers how to best direct visitors from one exhibit to an- had no trouble using the interface. However, some other exhibit. Some of our test visitors had diffi- visitors inexperienced with hand-helds had signifi- culty locating pi stations on the same floor and only cant problems successfully selecting objects on the a modest distance away, even when marked by bright touch screen, e.g. they dragged the stylus along the orange flags. The on-line map wasn't used much - surface rather than tapping.The visitors also weren't probably because it was not prominent enough in familiar with standard browser buttons such as " the on-line interface or was too difficult to relate back". However, these visitors did use the system to the 3D environment. successfully after brief hands-on instruction. For the last two trials, we added a practice exhibit to the check-out process to alleviate this problem. Future Research Plans We plan to continue our research with the test- Content Design and Navigation bed in the future with a specific aim of conducting studies with different audiences (general public, Visitors were navigating the content using a stylus school field trips,Explainers (floor interpretive staff), and clicking on images or text hyperlinks presented teachers, members and staff.) An important feature on the Web pages. When the top-level navigation of our next study will be to examine the potential page for an exhibit consisted of one large picture use of the system in before-and-after visit situa- (e.g. Fig. 6), some visitors had trouble figuring out tions. Our aim is to continue to explore issues of that they could click on part of it or which parts of using wireless networks and portable computers it were click-able. Some visitors expressed a desire to expand and extend the museum visit experience. to be able to see how much content there was for each exhibit and which parts of it they had/hadn't seen so far. Conclusion An additional part of the Electronic Guidebook Many visitors to the Exploratorium seem to mi- project is a conscious effort to stimulate discus- grate from one exhibit to the next interesting-look- sions within the museum community and between ing one. In such cases, the visitors only need to be museums and industry about the potential of using able to locate a pi station once they have already these systems in a museum setting.To initiate these approached the chosen exhibit.This is not difficult discussions, the Exploratorium sponsored the first so long as the pi station has a reasonably distinctive Electronic Guidebook Forum in the fall of 2001 .This appearance. two-day forum brought together 39 researchers and

Hwn r,n,j Plote

l'att.ivj -"0"Motiot: AhGlio Tu7:c - tip!ruliars Ltcunj lEackbeard Ifet;iels

Fig. 6: Electronic Page

BEST COPY AVAILABLE 18 @Archives & Museum Informatics, 2002 Museums and theWeb 2002 developers from industry, academia, and the mu- References seum world for discussion of the latest findings on the application of handheld computers and wire- Aoki, R M. (2000) and Woodruff,A. Improving less networks in museum exhibitions. The forum Electronic Guidebook Interfaces Using a discussions centered on interrelated aspects of elec- Task-Oriented Design Approach. Proc. 3rd tronic guidebook projects in museums and on ACM Conf on Designing Interactive Systems, emerging questions from the field. The format in- New York, NY,Aug. 2000, 319-325. cluded full group discussion of these topics, as well as discussion in small groups on lessons learned Debaty, P. (2000) and Caswell, D. Uniform Web and recommended next steps.The goal of the fo- Presence Architecture for People, Places, and rum was to identify key issues in the museum field Things. HPL Technical Report HPL-2000-67, that will inform further work on wireless handheld to appear in the IEEE Personal Communica- devices and stimulate research and implementation. tion magazine on Pervasive computing. An in-depth report on the forum is available at http://www.exploratorium.edu/guidebook/. Kindberg,T. (2002). Implementing physical hyperlinks using ubiquitous identifier resolu- Acknowledgements tion. HP Laboratories technical report HPL- 2001-95r. I .To appear in Proceedings 1I th The authors would like to sincerely acknowledge International World Wide Web Conference, the hard work and support provided by the many Honolulu, May 2002. people who have contributed to this project.They include: Kindberg,T. (2001) and Barton. J.A Web-based Nomadic Computing System. Computer KatherinaAudley, Stephen Bannasch, Doug Conaway, Networks, Elsevier, vol 35, no. 4, March 2001, Philippe Debaty, Margaret Fleck, Marcos Frid, 443-456. Caroline Gattein, Joshua Gutwill-Wise, Ron Hipschmann, Sherry Hsi, Tim Kindberg, Eamonn Kindberg,T. (2000) et al. People, Places,Things: O'Brien-Strain, Mike Petrich, Rakhi Rajani, Natalie Web Presence for the Real World. Proc. 3rd Rusk, Larry Shaw, RobertTinker, QuanTran,Adrian Annual Wireless and Mobile Computer Van Allen, Noel Wanner, Karen Wilkinson. Systems and Applications, Monterey CA, USA, Dec. 2000, I 9. This material is based upon work supported by the National Science Foundation under Grant No. ISE- Woodruff,A. (2001),Aoki, RM., Hurst,A., and 990 1985. Szymanski, M.H. Electronic Guidebooks and Visitor Attention. Proc. 6th Intl Cultural Heritage Informatics Meeting (ICHIM '01), Milan, Italy, Sep. 2001.

2 7

@Archives & Museum Informatics, 2002 19 Eavesdropping on Electronic Guidebooks: Observing Learning Resources in Shared Listening Environments

Allison Woodruff, Paul M.Aoki, Rebecca E. Grinter, Amy Hurst, Margaret H. Szymanski, and James D.Thornton, Palo Alto Research Center, USA

Abstract

We describe an electronic guidebook. Sotto Voce, that enables visitors to share audio information by eavesdropping on each other's guidebook activity.We have conducted three studies of visitors using electronic guidebooks in a historic house:one study with open air audio played through speakers and two studies with eavesdropped audio.An analysis of visitor interaction in these studies suggests that eavesdropped audio provides more social and interactive learning resources than open air audio played through speakers.

Introduction Previous research suggests that users of electronic Our design is guided by the following principle: we guidebooks prefer open air audio delivered through want to support visitor interaction with three main speakers to audio delivered through a headset (see, entities that make demands on their attention.These e.g., Kirk,2001;Woodruff,Aoki, Hurst, & Szymanski, entities are the information source, the visitor's 200 I ).The well-known visitor desire for social in- companions, and the physical environment"the teraction (Hood, 1983) is a key reason for this pref- guidebook, the friend, and the room" (VVoodruff, erence: when visitors use open air audio, they can Aoki et al., 2001). As we add capabilities that en- listen to content together and discuss it, whereas hances visitor interaction with one entity, we must headsets often isolate visitors into experiential be careful that we do not compromise visitor inter- "bubbles" (Martin, 2000). However, open air audio action with the others (e.g., we do not want to im- is problematic when many visitors are present in prove visitor-visitor interaction at the expense of the same location, as has been confirmed by infor- visitor-room interaction.) mal experiments conducted by commercial audio guide vendors (L. Mann, Antenna Audio, personal To understand the impact of the eavesdropping communication). mechanism on the overall visitor experience, we conducted two studies of visitors using the system We describe an alternative mechanism for sharing to tour a historic house. We applied qualitative audio.This mechanism, which we call eavesdropping, methods to the resulting data, including an analysis preserves the social interaction enabled by open of visitor interviews and an applied conversation air audio while avoiding the audio "clutter" that analytic study of recorded audiovisual observations. open air audio necessarily entails. In our system, Because the eavesdropping was an optional feature visitors independently select objects in their guide- that visitors could turn on or off at will, we ob- books and listen to the audio content through one- served several categories of use, e.g., pairs of visi- ear headsets; these headsets allow them to hear tors who did not use eavesdropping, pairs of visi- each other speak and interact conversationally. Fur- tors who used eavesdropping intermittently, and ther, wireless networking enables visitors to op- pairs who engaged in continuous mutual eavesdrop- tionally listen to their companion's guidebook in ping. addition to their own.The intimate, often directed, nature of the resulting shared audio context has In this paper, we focus on the visitors who engaged led us to call the system Sotto Voce. in mutual eavesdropping, which is the category that 23 Museums and the Web 2002:Proceedings BEST COPY AVAILABLE Archives & Museum Informatics, 2002, p. 21 Woodruff et al, Eavesdropping on Electronic Guidebooks most closely approximates open air audio.We com- the audio content. The design is the same as that pare the typical behavior of these mutual eaves- used in Study 2, reported in (Aoki et al., 2002), but droppers to that of visitors in a previous study who we briefly discuss it here to provide context. Over- used open air audio to create a shared listening all, visitors have a positive response to the guide- experience (Woodruff,Aoki et al., 200I;Woodruff, book and report that it is easy to use (Aoki et al., Szymanski,Aoki, & Hurst, 200 I ). (The three studies 2002;Woodruff,Aoki et al., 2001). are summarized in Table 1 .) Most of the discussion is based on analysis of the observational data.We Guidebook device. We implemented the device observe that mutual eavesdroppers had a different using the Compaq iPAQ' 3650 handheld computer, activity structure and were more mobile than visi- which includes a color LCD touchscreen display. tors who used open air audio.As a result of these With an IEEE 802.1 I b wireless local-area network changes, mutual eavesdroppers had increased re- (WLAN) card, the device measuresI 63mm x sources for engaging in interactive learning:they had 83mm x 34mm (6.4" x 3.3" x 1.3") and weighs richer and more extensive social interaction, and 368g (13 oz.). they had more resources for physically exploring their environment. For example, visitors had more To support eavesdropping, paired devices commu- substantive discussion in response to guidebook nicate over the WLAN using Internet protocols descriptions, and they were more likely to discuss (UDP/IP).The audio content is the same on all de- objects not described in the guidebook. Given the vices, so the devices send and receive control mes- importance of social learning in the museum envi- sages ("start playing clip 10," "stop playing clip 8") ronment (Falk & Dierking, 2000), the preliminary rather than waveform audio. Since our goal is to evidence presented here is encouraging and sug- enhance co-present interaction,the device does not gests further avenues for work along these lines. support remote voice communication.

The remainder of the paper is organized as follows. User interface. This part of the system is very First, we discuss the design of SottoVoce. Next, we similar to that used in previous studies, and its de- describe the method employed in our user study. sign rationale is more thoroughly described else- We then turn to findings. These are divided into where (Woodruff,Aoki et al., 2001). Individual visi- the impact of the design on visitor behavior and tors obtain information about objects in their envi- the implications of these behavioral changes for visi- ronment using a visual interface.This helps visitors tors' learning resources. After discussing related maintain the flow of their visual task (looking at the work, we summarize our findings and describe fu- room and its contents), which tends to reduce de- ture directions. mands on user attention.The interface resembles a set ofWeb browser imagemaps; at a given time,the Prototype Design visitor sees a single photographic imagemap that depicts one wall of a room in the historic house In this section, we discuss the design and imple- (FigureI , center).Visitors change the viewing per- mentation of the guidebook device, key aspects of spective (i.e., display a different imagemap) by press- its user interface, the design goals for the audio ing a hardware button. When visitors tap on an environment, the eavesdropping mechanism, the imagemap target, the guidebook plays an audio clip audio delivery mechanism, and the construction of that describes that object. Many, but not all, of the

Table I. Summary of studies conducted. Audio sharing Participants Previous Study mechanism Recruited Public papers (Woodruff, Aoki et al., 2001; Woodruff, I Open air 14 Szymanski et al., 2001) 2 12 (Aoki et al., 2002) Eavesdropping 3 47 -- Table I 29 22 ©Archives & Museum Informatics, 2002 Museums and the Web 2002

listening to their own clips and A's clip ends first,A i.esdroa will then hear the remainder of B's clip as if it had Ca * Queet "started in the middle:' To control a device's eaves- loud dropping volume (i.e., the volume at which A hears B's clips), the interface includes three option buttons:"Off," "Quiet" and "Loud" (Figure I , top left). "Loud" is the same as the volume for personal clips.

In abstract terms, eavesdropping provides a relatively simple audio space model (Mackay, I 999).We did consider other options, such as a telephony- like connection model in which visitors would in- Figure I. Electronic guidebook and headset. dependently initiate and terminate audio sharing sessions with their companions. We also consid- objects visible on the screen are targets; to help ered email-like asynchronous models in which visi- visitors identify targets, the guidebook displays tap tors would send and receive audio clips at their tips (Aoki, Hurst, &Woodruff, 2001)transient tar- convenience. We rejected more complex abstrac- get outlines that appear when the user taps and tions that involved multiple actions (send/receive, fails to "hit" a target (Figure I, bottom left).A dem- connect/accept/reject, etc.) because we believed that onstration of the visual interface is available online the necessary interface gestures would distract visi- (http://www.parc.com/guidebooks/). tors from their experience with the environment and their companions. In the audio space model, Audio design goals. Results from StudyI sug- sharing requires no gestures of its own.To "receive," gested several design criteria. Visitors want to be a visitor merely sets the eavesdropping volume.To able to share audio descriptions and converse. At "send:' a visitor simply selects an object; playing a the same time,visitors want to retain personal con- description has the side effect of sharing it, if the trol over the selection of descriptions. Further, the companion chooses to eavesdrop.The audio space design needs to facilitate the ability of visitors to model has the further advantage that it supports explore their physical environment, and the design simultaneous listening, which enhances social inter- needs to be sufficiently lightweight that it makes action by creating the feeling that the content is minimal demands on the users' attention. Finally, part of a shared conversation (Woodruff,Szymanski the design needs to be feasible in public environ- et al., 2001). ments with many visitors.These criteria ruled out a number of options like open air audio (which is not Audio delivery.Visitors hear descriptions through feasible for large numbers of visitors) or splitters headsets.We conducted a small study (n=8) to iden- that allow two visitors to listen to audio from a tify headsets that would allow visitors to converse single device (which restrict visitor movement and and that visitors would readily accept (Grinter & do not allow visitors individual control over the Woodruff, 2002). Based on this study, we chose audio content to which they are listening).The eaves- commercial single-ear telephone headsets, locally dropping model described below is an alternative modified by the removal of the boom microphone that meets all of the criteria. (FigureI , right). This configuration leaves one ear available to hear sounds from the external environ- Eavesdropping. In concrete terms, paired visitors ment, and visitors find the over-the-head design share audio content as follows.When visitor A se- desirable because it is familiar and gives them the lects an object on her device, she always hears her sense that the headset is securely attached. own audio clip. If A is not currently playing an audio clip, but her companion B is, then B's audio clip can Audio content. The prototype contains descrip- be heard on A's device. In other words, audio clips tions of 51 objects in three rooms of the house. In are never mixed, and A's device always plays a per- most regards, the descriptions are recorded along sonal clip (selected byA) in preference to an eaves- principles described in (Woodruff, Szymanski et al., dropped clip (selected by B).Audio playback on the 2001). The audio clips vary in length between 5.5 paired devices is synchronized; if A and B are both and 27 seconds, with the exception of one story

@Archives &Museumln9ormatics,2002 23 Woodruff et al, Eavesdropping on Electronic Guidebooks

that runs for 59 seconds. The clip length is much dren test SottoVoce in the first and second studies, shorter than conventional audio tour clips, which visitors from the ages of approximately 5-17 are often run to 180 seconds, and is intended to facili- quite rare at Filoli unless they are visiting with a tate conversation by providing frequent opportu- school group.) nities for visitors to take a conversational turn. Procedure. Visitors to the house were recruited Since we use single-ear headsets, both personal and at the entrance to the Library, the first room dis- eavesdropped audio content are necessarily pre- cussed in the guidebook.After signing consent forms, sented in the same ear.We distinguish the two types visitors were fitted with a wireless microphone, of content using two mechanisms. First, we apply a given guidebooks, and trained in their use. Next, small amount of reverberation to the eavesdropped they visited the three rooms for which the guide- audio.A single earphone cannot effectively deliver book had content. When they finished using the spatialized audio (Blauert, 1997), but can support guidebooks, they participated in a semi-structured other sound effects; we chose reverberation after interview. conducting user tests (n=6) involving scenario-based tasks using the guidebook. Second,the default eaves- The visitors' conversation and comments during the dropping volume ("Quiet"),which is most frequently interview were recorded using the wireless micro- used by visitors, is softer than the personal volume. phones; the visitors were videotaped by fixed cameras while using the guidebooks (all visitors to the Method house were notified that videotaping was in progress); and the visitors' use of the guidebooks We have conducted three major user studies at was logged by the device. Filoli, a Georgian Revival historic house located in Woodside, California (http://www.filoli.org/). Study Visitors typically spent about 15 minutes using the I used an earlier version of Sotto Voce that sup- electronic guidebooks. Their participation in the ported open air audio, whereas Studies 2/3 used study took approximately 30-45 minutes; no time the current version of Sotto Voce that supports limits were imposed during any portion of the pro- eavesdropping as described in the design section of cedure. this paper. Study 1 and Study 2 involved previously recruited participants on days the house was closed Analysis.We analyzed the data in several ways. For to the general public, whereas Study 3 involved 47 example, we transcribed and analyzed the interview visitors recruited on-site on days the house was data to examine the visitors' attitudes and feelings open to the general public. (Again, these studies about the technology and their experience. The are summarized in Table I .) majority of the findings presented in this paper are based on another method we used, conversation Because the participants and procedures for Study analysis (Sacks, I 984). 1 and Study 2 have been reported previously, below we report only the participants and procedure Conversation analysis is a sociological method used for Study 3.We then discuss our analytic methods, to examine naturally occurring social interaction which were the same in all studies. to reveal organized patterns.

Participants. In Study 3, we observed 20 pairs, one To find such patterns, conversation analysts study group of three, and one group of four using the collections of interactive encounters and identify guidebooks. These pairs and larger groups were sequences of actions that were recurrently made by comprised of visitors who had come to Filoli to- the participants. Actions in our context might in- gether, e.g., mother/daughter or friend/friend pairs. clude making a verbal utterance, pointing at an ob- The majority of visitors had not previously used a ject, or selecting a description. handheld device.The visitors covered a wide range of ages: the youngest visitors were in the "18-29" To this end,we create a composite video of visitors age range, and seven visitors who used the guide- and their guidebook screens and audio (re-created book were "over 70." (While we had several chil- from the guidebook activity logs). We then tran-

.31 24 ©Archives & Museum Informatics, 2002 Museums and theWeb 2002

scribe the actions taken by visitors, including dia- open air audio did afford the opportunity to par- logue, and look for recurring patterns to identify ticipate in shared responses to the "story," moti- visitors' systematic practices. vating the visitors to begin setup for another sequence. From Open Air To Eavesdropping: By contrast, participation in mutual eavesdropping Changes InVisitor Behavior created an ongoing assumption that the couple In this section,we compare the behavior of the pairs would continue in the shared activity.This supposi- who chose to use mutual eavesdropping in Studies tion of continuing shared activity meant that setup 2/3 to that of similarly engaged pairs who used open tended to be cursory. Further, while open air audio air audio in Study I. was primarily conducive to follow-up discussions that related directly to descriptions, mutually eaves- Specifically, we discuss the structure of the visitors' dropped audio was conducive to many diverse types interactions and their physical mobil4 The effect of follow-up sequences such as discussion of ob- of these aspects can be identified in the visitors' jects not described in the guidebook. learning-related behavior, which is the subject of the following section. The change in activity structure had at least two beneficial effects. Changed Activity Structure First, by reducing the effort needed to choose Visitor activity was structured very differently with and listen to descriptions, mutual eavesdropping eavesdropped audio than with open air audio.The freed visitors to direct more attention to mean- new structure had a lower coordination cost, de- ingful interactions with their environment and manding less attention. The decreased attention their companions (i.e., away from the guidebook burden was reflected in the visitors' interactions. and routine coordination). In other words, the reduction in low-quality coordination talk meant In all of the studies, a single overall structure per- that visitors had more time to investigate the vaded the interactions. Specifically, they exhibited room and its contents and that a higher propor- the sequential, multi-phase organization known as tion of talk tended to focus on topics of sub- storytelling in the conversation analytic literature stance. (Sacks, 1974); as part of this organization, visitors created a conversational role for the audio descrip- Second, since the new activity structure sup- tions, i.e., they treated the guidebook like a "third ported more diverse types of sequences, visitors party" taking an extended conversational turn (Aoki were more likely to pursue new topics or inves- et al., 2002;Woodruff, Szymanski et al., 2001). Paired tigate objects not described in the guidebook. visitors entered a state of engagement at the beginning of a given storytelling sequence; levels of Increased Mobility engagement generally rose and then fell over the course of a given sequence; and visitors then had Visitors in Studies 2/3 were noticeably more mo- the options of dis-engaging (resulting in indepen- bile during periods of engagement. In Study 1, the dent activity), remaining engaged in shared activity, open air audio was played at a low volume, so any or maintaining a nascent engagement in expecta- movement that changed the relative position of the tion of subsequent re-engagement (Szymanski, visitors could cause significant sound attenuation 1999). due to distance or blockage (e.g., due to interposed obstacleseven changes in body orientation could With open air audio, visitor interactions tended to cause the audio to be blocked).As a result, couples focus on choosing individual objects and coordi- tended to remain close together and stationary nating with their companions to listen to the de- while sharing audio descriptions. See Figure 2a, in scriptions.This setup, repeated for each sequence, which a grandmother is bending over to listen to focused more attention on coordination activity the audio description that her granddaughter is play- than seems necessary or desirable. However, the ing of the portrait over the fireplace. Note how 32 @Archives & Museum Informatics, 2002 25 Woodruff et al, Eavesdropping on Electronic Guidebooks

FigureI. Comparison of visitor mobility patterns.

(a) Visitors standing close together when (b) Visitors standing far apart when using using open air audio. mutually eavesdropped audio. this position prevents her from examining the paint- the changed activity structure and increased mo- ing while she listens. In Studies 2/3, visitors were bility during engagement improved the learning less constrained. Because movement could not at- environment. Here, we discuss two learning-related tenuate the audio information, visitors could sepa- resources that were enhanced by the guidebook: rate from each other physically while listening to the nature of the visitors' social interaction and their descriptions and remaining engaged. See Figure 2b, opportunities for exploring the room and its con- in which both visitors are listening to a description tents. of the marble staircase.While both visitors are examining the staircase, they have each chosen differ- Our analyses are based upon a collection of tran- ent vantage points. However, this positioning does scribed excerpts from which the following extracts not compromise their social connection: when the have been derived. These extracts are meant to audio description reveals that only the first four exemplify and highlight specific behaviors rather than steps are actually solid marble,the male visitor looks to illustrate the organization of the visitors' inter- to his companion and she laughs, even though they actions (space limitations preclude the use of rep- are not standing together. resentative excerpts of this kind).

The increased mobility resulting from use of mu- Table 2 summarizes the notation used in this sec- tual eavesdropping took many forms.We observed tion. For clarity of presentation, the extracts have several common behaviors that rarely, if ever, oc- been simplified to use conventional capitalization curred with open air audio. For example, visitors and punctuation (e.g., commas and periods). would often walk together while a description was playing, e.g.,to approach the object being described. Note that the discussion in this section is limited In other cases, a single visitor would walk closer to to the increased availability of learning resources. A the object currently being described while their claim of increased learning would require a different companion remained stationary. In still other cases, type of study, e.g., one that measured the visitors' a visitor would investigate a different object from knowledge before and after their visit. Such a study the one currently being described and then rejoin is beyond the scope of this paper. the companion. Table 2. Summary of transcription notation. From Open Air To Eavesdropping: x: Visitor X is speaking. ((comment or action)) X-PDA: Visitor X's guidebook is speaking. Increased Resources For Learning **whisper** Speech at reduced volume.

The study observations provide evidence that both emphasis Emphasis in speech. of the factors described in the previous section (n) A conversational pause of n seconds. (.) 3 indicates a "micropause."

26 @Archives & Museum Informatics, 2002 Museums and the Web 2002

Depth and Length of Social Interaction By contrast, consider an interaction in which J and L (Extract II, Study 2) were mutually eavesdropping. When using mutual eavesdropping, visitors re- sponded more fully to audio descriptions.Visitors L-PDA: ... All of the architectural fea- were also more likely to discuss features of the tures of this room, including the walnut pan- object not mentioned in the description or to dis- elling, are modelled on an 18th century Brit- cuss objects that were not described in the guide- ish library. In the original library, each of the book at all. outlined panels would have contained framed pictures. Each of these phenomena represents a way in which L: Really. visitors collaboratively built on the shared audio J: Yeah. descriptions, working together to construct mutual L: That's a lot of pictures. ((points at wall learning resources that broaden, deepen or expand and sweeps arm across walls)) their discussion of the room's contents.The impor- J: That's a lot of pictures. ((nods "yes"))(.) tance of such social learning, particularly (but not J: That would've been very cluttered. limited to) conversation, has been widely supported in the visitor studies literature (see, e.g., (Falk & Extract II. Dierking, 2000; Russell, 1994)). Social interaction around artifacts affords the "opportunity for the This interaction is richer in many respects than that visitor to make connections with familiar concepts shown in the previous extract. J and L both speak, and objects" (Hein, 1995); adding resources for in- taking more turns to discuss the description thanV teraction adds more such opportunities.The remain- and W. By making a statement about the number of der of this subsection gives some examples, linking pictures, L reinforces for J a quantitative observa- the behavioral changes of the previous section to tion that is not made in the description itself. By the construction of learning resources. gesturing at the many empty wall panels, L adds physical, spatial, and visual elements to the experi- Characterizations. With mutual eavesdropping, ence, linking both visitors to a vision of the "origi- response to an audio description was likely to be nal" library that overlays their actual surroundings. more reflective and include a physical focus relative J agrees with the quantitative statement, pauses, and to the object being described.With open air audio, then responds by saying it would have been "clut- visitors would often have a very minimal response, tered," a qualitative assessment that indicates that and even the more substantive responses were gen- she has in fact visualized the room as it might have erally limited to reactions to the description that been. had just occurred. This increased reflection on descriptions was evi- The following extracts are representative of this denced in many ways. Visitors worked together effect. Consider V and W (Extract 1, Study 1) who more to understand descriptions, e.g., a visitor were listening with open air audio. would sometimes express confusion about a description and the companion would help explain it. V-PDA: Many of the top shelves con- Further, with mutual eavesdropping, visitors more tain false books. They are lighter than nor- frequently branched off into sequences that were mal books, so they reduce the stress on the not directly related to the description of the con- bookcases. Many are made of greeting cards, tent. For example, they might point out a specific clothing, fabric, et cetera. physical feature of the object that was not men- W: Eh hah, that's a riot. (('N looks atV and tioned in the description, or discuss some way that smiles)) (0.2) They're just for looks. it related to their own life. Extract I. Additionally, visitors showed more evidence of es- While V and W do share a response, the substance tablishing complex relationships between objects. is limited to a single paraphrase of the audio de- One visitor pointed at a series of paintings on dif- scription, analogous to "text echo" of exhibit labels ferent walls, saying,"Okay, so that's his wife, and that's (McManus, 1989) (though possibly more affective, his mother, right?" Or consider T's comments (Ex- because of the audio delivery). 3 I

@Archives & Museum Informatics, 2002 27 Woodruff et al, Eavesdropping on Electronic Guidebooks

tract III, from Study 2) when he first enters a par- the description ended gave them more opportuni- ticular room. His statements indicate that he has ties to observe and discuss its specific features. constructed a category of "secret cabinets" that occur in this house and that he is alert to instances Expanded Resources for Physical of this category as he moves from room to room. Exploration T: Ah, more secret cabinets. With mutually eavesdropped audio, the examina- (0.4) tion of objects was more frequently occasioned by T: I like that a lot about this house. ((walks their presence in the room rather than their pres- into the bar closet)) ence in the guidebook. Once visitors began to ex- Extract III. amine an object, they might discuss it or play a description of it if one were available. Interactions displaying this kind of orientationi.e., at the granularity of a thematic collection rather This implicit shift in emphasis from the guidebook than a single objectalmost never occurred with to the room as the impetus for exploration is im- open air audio. portant because it shifts the visitor's role. It is broadly (though perhaps not universally) accepted Moreover, mutually eavesdropping visitors often that learning is enhanced by enabling visitors to discussed objects that were not described in the navigate the museum without leading them through guidebook, unlike open air audio visitors. The fol- it (Falk & Dierking, 2000). However, even "free lowing sequence, in which J teaches L about a plant, choice" navigation can be constrained by, e.g., which occurred immediately after they finished their re- objects have descriptive content associated with sponse to a description: them.Visitor behavior indicates that use of mutual eavesdropping increased the guidebook's utility as J: Okay, your- your test for the day, what's a reference (an adjunct to the room) as opposed that one? ((points to plant)) to an inventory (a directed guide to the room). (0.2) J: The plant. Characterizations. In the study using open air (0.4) ((L leans in to look)) audio, examination of objects often began with ob- L: Morning glory. Eh heh heh heh, I don't jects contained in the guidebook and proceeded by know, what is it? spatial locality.That is, visitors tended to switch the J: I think it's a mandevilla vine, but I'm visual interface to a given wall and then look at the not sure. objects in the guidebook that interested them on L: God, I can't believe you know that. that wall. Object choice was often based on targets Extract IV. seen in the visual interface or on short-term memory of such targets. Reasons. Both of the behavioral changes resulting from use of mutually eavesdropped audio had im- In the eavesdropping studies, the next object to pact on social interaction.The primary factor was examine was less frequently chosen based on avail- the new activity structure, which allowed more ability in the guidebook. (In many of these cases, we space for reflection and for visitors to initiate new know that the examination was prompted by the conversational sequences that were not structured room rather than the guidebook because the ob- around the audio descriptions. Increased mobility jects were not described in the guidebook In the constituted a secondary factor.Visitors would of- other cases, the visitors spoke their thoughts aloud ten start descriptions while they were far away from which was entirely self-prompted since none of objects.As mentioned above, visitors were unlikely the studies involved a speak-aloud protocol.) In- to walk toward the object while the description stead,visitors would encounter objects in their field was playing with open air audio. However, with of view, e.g., objects that were near an object they eavesdropped audio, they were more likely to ap- had just examined, or they would deliberately ex- proach the object; being close to the object when amine sequences of objects they perceived as being related. .35

28 ©Archives & Museum Informatics, 2002 Museums and theWeb 2002

For example, in Extract111,T walks into a new room, books for many years (Screven, 1975). Related work notices the bar closet and actually walks into it. AF in HC1 has focused on aspects such as location- ter this, his companion D finds the description in aware computing (Abowd et al., 1997), and only the guidebook and plays it. Note that because the recently have significant user studies been reported sound does not attenuate, the visitors can listen to (e.g., (Cheverst, Davies, Mitchell, Friday, & Efstratiou, the description together while T stands inside the 2000)).The HC1 studies focus on system design and tiny closet and D stands outside. evaluation; here, we focus on the effects of our system on visitor interaction. Reasons.While the same resources were available with open air audio, they were used much more Media and interaction.There is an extremely rich frequently in the mutual eavesdropped case due to literature on collaborative multimedia environments; the changed activity structure and the increased of particular interest are media spaces (Mackay, mobility in the room.Specifically,the mutually eaves- 1999). Many of these systems have been evaluated, dropped audio was more conducive to sequences but most apply either ethnographic techniques or that were not directly responsive to guidebook quantitative methods to studies of installed work- content; visitors were generally more open to ex- place systems. In this study, we apply conversation ternal triggers with the new activity structure.Visi- analytic techniques to the study of a mobile, lei- tors acted in a manner more consistent with "Let's sure-activity system that provides shared access to see what's here in the room" than with "Let's see application content. what's here in the guidebook?' Further, visitors had more attention to give to the room due to the re- Conclusions duced attentional demands and wandered more in the room due to increased mobility, so they were In this paper, we have described an eavesdropping more likely to encounter and investigate objects. mechanism that allows visitors to listen to each other's guidebooks. Our findings show that mutual Related Work use of this eavesdropping mechanism can lead to increased learning resources as compared with the Our work draws together three main areas of re- use of speakers in open air: couples using mutual search. Space limitations preclude an extended dis- eavesdropping in Studies 2/3 had more substantive cussion; additional references are contained in (Aoki interactions and exhibited an increased awareness et al., 2002;Woodruff,Aoki et al., 2001; Woodruff, of the room and its contents when compared to Szymanski et al., 200 I ). those using open air audio in Study I.

Interaction in museum settings. The impor- New work is addressing some of the open issues tance of social interaction to museum visitors is from this study. We are preparing a discussion of well known (e.g., (Hood, 1983)).There are two types the ways in which the visitors creatively used our of studies of particular interest. McManus observed eavesdropping mechanism for tasks other than en- visitor usage of text labels; she noted that visitors hancing their social interaction, e.g., for monitoring were inclined to treat exhibit labels as conversation their children.We are also planning an experiment to which they had been party (McManus, I 989).A using bone conduction headsets that can provide number of studies of museum visitors have been binaural audio without occluding the ears. conducted using methods derived from conversation analysis (see, e.g., (Falk & Dierking, 2000), Ch. Acknowledgements 6, and (vom Lehn, Heath, & Hindmarsh, 200 I ).These studies focus on talk, interaction and learning in We are deeply grateful to Tom Rogers and Anne conventional environments; here, we have focused Taylor of Filoli Center for their assistance with this on the effects of electronic guidebooks on social project.Amy Hurst performed this work during an interaction and learning resources. internship from the College of Computing, Geor- gia Institute of Technology Electronic guidebooks. The cultural heritage community has formally studied electronic guide-

3 6' ©Archives & Museum Informatics, 2002 29 Woodruff et al, Eavesdroppingon Electronic Guidebooks

References Mackay,W. E. (1999). Media Spaces: Environments for Multimedia Interaction. In M. Beaudouin- Abowd, G. D.,Atkeson, C. G., Hong, J., Long, S., Lafon (Ed.), Computer-Supported Cooperative Kooper, R., & Pinkerton, M. (1997). Work Chichester, UK:Wiley & Sons, 55-82. Cyberguide:A Mobile Context-Aware Tour Guide.Wireless Networks, 3(5), 421-433. Martin, D. (2000).Audio Guides. Museum Practice, 5(1), 71-81. Aoki, P M., Grinter, R. E., Hurst,A., Szymanski, M. H.,Thornton, J. D., &Woodruff,A. (2002). McManus, P. M. ( I 989). Oh,Yes They Do! How Sotto Voce: Exploring the Interaction Be- Visitors Read Labels and Interact with Exhibit tween Conversation and Mobile Audio Text. Curator, 32(3), I 74- I 89. Spaces. In Proc.ACM SIGCHI Conferenceon Human Factors in Computing Systems. New Russell,T. (1994).The EnquiringVisitor: Usable York:ACM Press, to appear. Learning Theory for Museum Contexts. Journal of Education in Museums, 15, 19-21. Aoki, P. M., Hurst,A., & Woodruff,A. (2001).Tap

Tips: Lightweight Discovery of Touchscreen Sacks, H. ( I 974).An Analysis of the Course ofa Targets. In Extended Abstracts,ACM SIGCHI Joke's Telling in Conversation. In R. Bauman & Conference on Human Factors in Computing J. Sherzer (Eds.), Explorations in the Ethnog- Systems. New York:ACM Press, 237-238. raphy of Speaking. Cambridge: Cambridge Univ. Press, 337-353. Blauertd. (1997). Spatial Hearing (rev. ed.). Cambridge, MA: MIT Press. Sacks, H. (1984). Notes on Methodology. In J. M. Atkinson & J. Heritage (Eds.), Structures of Cheverst, K., Davies, N., Mitchell, K., Friday,A., & Social Action. Cambridge: Cambridge Univ. Efstratiou, C. (2000). Developing a Context- Press, 21-27. Aware Electronic Tourist Guide: Some Issues and Experiences. In Proc.ACM SIGCHI Screven, C. G. (1975).The Effectiveness of Conference on Human Factors in Computing Guidance Devices on Visitor Learning. Systems. New York:ACM Press, 17-24. Curator, 18(3), 219-243.

Falk, J. H., & Dierking, L D. (2000). Learning From Szymanski, M. H. (1999). Re-engaging and Dis- Museums.Walnut Creek, CA: Altamira Press. engaging Talk in Activity. Language in Society, 28(1), I -23. Grinter, R. E., & Woodruff,A. (2002). Ears and Hair: What Headsets Will People Wear? In vom Lehn, D., Heath, C., & Hindmarsh, J. (2001). Extended Abstracts,ACM SIGCHI Conf. on Exhibiting Interaction: Conduct and Collabo- Human Factors in Computing Systems. New ration in Museums and Galleries. Symbolic York:ACM Press, to appear. Interaction, 24(2), 189-216.

Hein, G. E. (1995).The Constructivist Museum. Woodruff,A.,Aoki, P. M., Hurst,A., & Szymanski, M. Journal of Education in Museums, 16, 21-23. H. (2001). Electronic Guidebooks and Visitor Attention. In D. Bearman & J.Trant (Eds.), Hood, M. G. (1983). Staying Away:Why People Cultural Heritage Informatics 2001 (Proc. 6th Choose Not to Visit Museums. Museum ICH IM). Philadelphia:A&MI, 437-454. News, 61(4), 50-57. Woodruff,A., Szymanski, M. H.,Aoki, P. M., & Kirk, J. (2001). MUSEpad. Presentation at the Hurst,A. (200 I ). The Conversational Role of Electronic Guidebook Forum, San Francisco, Electronic Guidebooks. In G. D.Abowd, S. CA, October 2001. Shafer & B. Brumitt (Eds.), Ubiquitous Computing (Proc. 3rd International Confer- 3 7 ence). Berlin: Springer-Verlag, I 87-208.

30 ©Archives & Museum Informatics, 2002 Can You See Me? Exploring Co-Visiting Between Physical and Virtual Visitors

Areti Galani and Matthew Chalmers, University of Glasgow, Department of Computing Science, UK

http://www.dcs.gla.ac.uldequatorkity.hani

Abstract

We explore issues of social context and interaction between digital and physical museum visitors, using as a focus of discussion the City project, itself set within a larger interdisciplinary project called Equator.We look at collaborative environments that span different media, in particular handheld mobile devices,Web-based hypermedia and 3D virtual environments.We discuss two main strands in our research:the methods and results of two pilot visitor studies in two cultural institutions in Glasgowthe Lighthouse and the House for an Art Loverand the development of our prototype system which establishes three-sided collaboration between physical,Web and virtual environment visitors. We then present preliminary results and issues arising from our on-going system development and user trials.We conclude with future plans for further system evaluation and deployment. Keywords:Visitor studies, social context virtual reality, wearable computers, hypermedia.

Introduction In their definition of the interactive museum expe- This paper further examines social context in mu- rience, Falk and Dierking (1992) described three seums, and its technological support.We are espe- key elements that influence the way visitors expe- cially interested in social context among new audi- rience museums:the physical context,the personal ences who visit museums via the Web and 3D vir- context and the social context. Physical context tual reality applications.While studies of traditional mainly covers the physical layout of the space and museum audiences have shown the importance of has been extensively studied by designers (Com- social context, new media as used in museums do munications DesignTeam,1976) and more recently not often support social interaction. In studying both by space syntax theoreticians (Psarra, Grajewski & traditional museums and new technologies, we aim O'Neil, 2002). Personal context covers the prior to address issues such as how to integrate visits to knowledge of the visitors, their personal aims and the virtual museum with visits to the 'physical' mu- expectations, and their current state of mind. Per- seum, especially when a visitor may visit both.This sonal context and its influence on the learning ex- is partly a response to the way that the number of perience have also been studied by means of evalu- digital visitors is steadily growing and, in some cases, ation teams and learning theories such as outstripping the number of visitors to the corre- constructivism and Gardner's multiple intelligences sponding physical museums (Lord, 1999). As a re- theory. Last, but not least, social context covers sult, many wish to find ways to encourage the geo- the social interaction during the museum visit be- graphically distant digital visitor to become a physi- tween the visitors and their immediate compan- cal visitor, and to encourage physical visitors to ions, as well as other visitors and museum staff. maintain a relationship with the museum after they Several aspects of the social context in a physical walk out its door. museum setting have been examined, focusing mainly on school groups and families (Diamond, Our project, City, investigates ways to support and 1986; McManus, 1987; Falk & Dierking, 1992). More enrich context in museums, cultural institutions and recently, social scientists in the SHAPE project also the cityin particular, social context. Set within a examined the social cues in the interaction with research consortium called Equator (www.equator. displays (Vom Lehn, Heath & Hindmarsh, 200 I ). ac.uk), our work is on context that involves some-

3 3 Museums and theWeb 2002:Proceedings Archives & Museum Informatics, 2002, p. 3 I Galani and Chalmers, Can You See Me?

thing richer and more complex than a collection of way covisitors move around the space is also in- isolated media and disjointed pieces of information. fluential. Other visitors who happen to be in the Computer scientists as well as museum profession- museum at the same time may also have an effect, alsfor different reasonstend to focuson the as may those who visited at some time in the past. obvious differences between traditional and digital media, and treat each one independently. Here,a This latter influence,asynchronous social awareness, broader viewpoint takes account of their similari- is a key feature of the museum experience. Tradi- ties and interdependencies. tionally afforded comment books, visitors can write about what they did or felt during their museum We initially discuss our approach to understanding experience.Any later visitor can read the comments; and working with social context, pointing out prior i.e. the communication is not generally personal- work in both traditional and digital media.We then ized or directed towards any one particular reader. present two parallel lines of work that share a theme: Recently entire exhibitions and displays have re- the architect, designer and artist, Charles Rennie volved around this notion of social context. For Mackintosh (1868- I 928).We report on studies of example, in London, visitors to the Science Museum social interaction in existing museums and exhibi- (www.sciencemuseum.org.uk) can read other visi- tions, and on a system infrastructure and proto- tors' opinions on controversial issues, and in the type to bridge digital and physical visits to an exhi- interactive games of the Wellcome Wing can com- bition room and its collection of artifacts.Thispro- pare their results with those of previous visitors. totype supports interaction between people visit- Visitors also have the opportunity to create a Web ing or exploring the room, even though they use page with the highlights of their visit, and access it quite different media: wearable computers, later online with their friends and family.This raises hypermedia and virtual environments. Thus a visi- the issue of whether and how to structure, curate tor using any one of these styles can interact with or select from such contributed 'collections' as they other visitors using other media. We offer some grow in size and value. initial observations on the use of our prototype system before outlining our ongoing and planned Web sites may offer various forms of social aware- work ness.These may be synchronous, as in chat rooms, or asynchronous, as in mailing lists and recommen- Social context dations. Awareness much like that of comment books is afforded by, for example, the Amazon.com We approach the issue of social context by consid- bookstore.One can leave comments about specific ering interaction between visitors.We initially use books, CDs and so forth, and can read comments a deliberately nafve categorization of context, di- from earlier visitors. However, dynamic and person- viding it in terms of time, space and medium. First, alized information can be cheaply offered when interaction may be synchronous or asynchronous. compared to traditional media. For example, each Secondly, it may occur locally, within the museum, Amazon visitor's profile of movement around and or remotely, with at least one visitor being physi- purchases from the site is dynamically combined cally located beyond its walls.Thirdly, it may involve with others in creating recommendations for the digital media such as Web sites and virtual environ- individual, i.e. selections from the many items sold ments, or traditional 'physical' media such as the by Amazon, based on the match between a visitor's museum building itself, displays and, of course, the profile and those of earlier visitors. Each movement artefacts of the museum collection. and purchase leads to different personalized recommendations.The Hippie (Oppermann & Specht, One experiences synchronous social awareness 1999) and the SottoVoce projects (Aoki &Woodruff, when visiting a museum with some friends or as a 2000) have already explored many of the issues in- member of a larger group: one's covisitors.The de- volved in the personalized delivery of content in a sign of the exhibition space, through its layout, dis- museum environment but, as far as we know, plays, artefacts and supporting materials, clearly in- recommender systems have not yet been applied fluences the visitor's movement and activity, but the in museums. 39

32 ©Archives & Museum Informatics, 2002 Museums and theWeb 2002

Synchronous social interaction is also a key feature boundaries between visitors who are local and re- in virtual environments. The vast computer game mote, and between digital and traditional media.As industry is centred on such technology, and is now described in a later section, ongoing and future work exploring the possibilities of collaborative game play- is directed towards applying the results of our stud- ing on-line. Furthermore, chat channels such as Ac- ies to our systems, especially with regard to more tive Worlds (www.activeworlds.com) combine di- explicit support for the roles visitors take with re- rected textual communication with graphical rep- gard to each other, and towards asynchronous in- resentation of users as avatars. Museums and re- teraction. lated institutions have explored the same medium, but for purposes such as the presentation of Studies artefacts too fragile or numerous to be put on display and of reconstructed archaeological sites, and In collaboration with two institutions in Glasgow, in supporting geographically distant visitors. Such a The Lighthouse (www.thelighthouse.co.uk) and the medium supports social interaction between visi- House for an Art Lover (www.houseforanart tors, as in the Virtual Leonardo project in the Na- lover.co.uk), we are carrying out two sets of visitor tional Museum of Science and Technology in Milan studies of qualitative character. Our methodology (www.museoscienza.org), and the Van Gogh Mu- is influenced by recent qualitative museum visitor seum inAmsterdam (www.vangoghmuseum.n1).Web studies as well as by ethnographic and anthropo- users share a common representation of the mu- logical methods used in social sciences and recently seum and are afforded basic resources for social in computing science. Our aim is to better under- interaction: mutual visibility and audibility. lmmersive stand the relationship between the visitor and the VR technology, such as headmounted displays and social context and environment of the visit.These roomsized (and roomshaped) projection surfaces, studies, unlike the majority of the visitor studies is used in a number of cultural institutions, e.g. the conducted in the museum sector, do not look at Foundation of the Hellenic World (www.fhw.gr). interactions around specific displays or temporary Relatively little support is given in such systems for exhibitions. Instead, both studies are situated in asynchronous awareness, however. permanent exhibitions.

Most research on museums and the Web has, we We believe that qualitative methods, such as natu- suggest, tended to treat local and remote visitors ralistic ethnography, in the exploration of yisitors' in isolation from each other, and to treat traditional interaction are necessary to achieve a better un- and digital media similarly.To our knowledge, there derstanding of 'how visitors see things, and what has been little or no work that bridges between meanings they give to their experiences, rather than local and remote, and between traditional and digi- simply to enumerate frequencies for pre-formed tal media. However, those who visit the digital mu- categories' (MacDonald, 1993). Furthermore, our seum may visit the traditional museum, and vice corpus of observations can be used to address ei- versa. Previously seen digital information may influ- ther of two related issues: the generation and de- ence a visitor's interpretation of traditional displays livery of content, and social interaction during the and artefacts, and vice versa. New technologies let visit.The latter will be the main focus of the follow- visitors to the traditional museum interact with digi- ing discussion. tal visitors, and combine traditional and digital media, in the same experience. The two institutions were chosen because they share the same topic, Mackintosh, but they explore In order to deepen our understanding of social different ways of engaging the visitor.The Lighthouse context,we have undertaken a pair of ongoing stud- has developed an interpretation centre, often sim- ies of social awareness and context in traditional ply called the Mack Room, with a number of origi- museums and cultural institutions. These are de- nal objects intermixed with 23 workstations and scribed in the next section.We then describe a par- displays that convey a substantial amount of digital allel stream of activity within our project, building a information. The House for an Art Lover is a re- technological infrastructure for covisiting and syn- cently constructed house, but built, decorated and chronous social context that crosses or blurs the furnished according to Mackintosh's entry to a 1901

4 0 © Archives & Museum Informatics, 2002 33 Galani and Chalmers, Can You See Me?

Figure 2. The visitor on the right, who has been in the House for an Art Lover several times before, points out and describes to her covisitor a less obvious feature of the ceiling lights' design

Figure I.An interactive display in the Mack In our studies we concentrated on 'casual visitors' Room is operated by a visitor with more (Falk & Dierking, 1992) in groups or as singletons, background knowledge of the exhibition rather than the more structured visits of school than his covisitor. groups and families. We focused on interpersonal interaction, and also observed how the information delivery media available in the gallery, such as design exhibition. It is widely perceived as a his- audioguides, touch-screens, videos, labels and so toric house attraction,and a visitor is offered a leaflet forth, influenced this interaction. and an audioguide that describe the construction project and Mackintosh's work We categorized the interaction between co-visitors into three main styles. In the first, covisitors In the Lighthouse we watched visitors and in eight are tightly connected, staying together during their cases manually recorded their movements on a map. visit, and interacting with the same display at the In the House for an Art Lover we conducted par- same time. In the second style, co-visitors are loosely ticipant observation of visitors for six days within a connected: they interact with different displays, but period of two months. In both locations, we used in the same area and thus stay relatively close to photographs and notes to add to our record of each other.Thirdly, covisitors are independent navi- visitor activite also obtained some other sources gators,following their own individual routes for the of information regarding visitor experience. We main part of the visit and meeting with each other were offered copies of the Mack Room designer's only occasionally. architectural drawings, showing the expected flow of archetypal visitors around the exhibits in the Mack In tightly connected interaction, covisitors stay Room, along with notes on the expected experi- close to each other as they actively and ences gained by following such paths. In the House collaboratively interact with the displays, discussing for an Art Lover, we were given access to the re- artefacts and their descriptions. An important in- sults of a marketing survey of visitors. on this style of interaction is the background

34 ©Archives & Museum Informatics, 2002 Museums and the Web 2002

Figure 3.Although close by each other, these Figure 4.This couple is walking round the two covisitors in the House for an Art Lover Mack Room together, but the woman on the are attending to different sources of right interacts with a touch screen while her information: an audioguide and a companion, on the left, looks over her leaflet respectively shoulder at a video display. knowledge of visitors and their history of previous close enough to maintain an awareness of each other. visits to the gallery. The most experienced usually They usually exchange brief comments about what leads the interaction by operating the touch screens they have seen and point things out to each other. (Fig. I ), or by pointing out details not obvious from They may be very close to each other but engaged the labels/commentaries (Fig. 2). Different covisi- in quite different information, as in the leaflet reader tors may undertake the leading role at different and audioguide listener in Figure 3, and may attend times during a visit. to different displays at any given moment, as in Fig- ure 4. The pace of the visit is an essential factor in this style. People usually share their interests in the dis- Visitors using audioguides usually experience this plays and also share a pace of reading the available style of interaction.The audioguide often works as material. They also decide together where to go a resource for discussion despite the fact the de- next.This style of interaction occurs quite often in vice itself is not designed to support explicit inter- galleries where information is not easily available action between visitors.This style of co-visiting also or accessed,so people tend to move around together seems to favour gestural communication: a person and help each other to interpret the exhibits. may want to share an experience but not to interrupt the companion's experience, so favours hand In the second style of interaction, covisitors are gestures or eyebrow movements rather than ver- loosely connected.These visitors do not consistently bal communication. interact with the same objects and displays, but stay

4 ')

©Archives & Museum Informatics, 2002 35 Galani and Chalmers, Can You See Me?

Independent navigation occurs in covisitors who Loosely connected covisitors are also aware of visit displays separately from each other. Each per- each other's location and possibly orientation at son has an individual pace, resulting sometimes in any given moment.They use verbal or gestural com- people waiting for the covisitors outside the gal- munication, but they do not necessarily share con- feries, or 'slow' visitors cutting their visits short in tent synchronously. Finally, independent navigators order to catch up with others. Repeat visitors to are only peripherally aware of their companions' the gallery often follow this style, and in some cases locations. Such a visitor will usually inform covisi- they leave the immediate space of the gallery and tors in a room when about to leave it, and indicate proceed to another room. Repeat visitors also tend a destination, if their mutual awareness extends to not take the audioguide at all or to consult it beyond a single room.They do not constantly share very infrequently compared to less experienced co other resources but are potential users of all of visitors. them during meetings.

This style of co-visiting includes many meeting points between the members of the group: sometimes The Current System accidental, sometimes deliberate. For example, two In parallel with our studies, we have developed a covisitors may meet accidentally when their in- system to explore social context that bridges or dependent paths cross in a display that is of inter- blurs the boundaries between visitors who are lo- est to both. In order to deliberately meet, one visi- cal and remote, and between digital and physical. tor may change navigation in the space, moving very The system involves three types of visitors: some- quickly between points with good visibility in the one walking around the Mackintosh Room with a gallery in order to find the covisitor. Regardless of wearable computer, someone remote from the the way people meet, they discuss what they have Mack Room but moving throughWeb pages related seen in their individual tours, or suggest displays to to the room,and another remote visitor using aVR each otherin some cases almost dragging their model of the Mack Room. companions to show them a specific display. They also discuss the logistics of the visit; for example, A set of scenarios (Chalmers, 2001; Galani, 2001) where to go next, when they will have lunch, and so was used to further explore the concept of- co- forth. visiting in museums and the city, and as a focus for system design. In our current scenario a visitor to It is important to point out visitors do not rigidly Glasgow, Vee, visits the Mack Room in The Light- conform to the same archetypal style of interac- house.While browsing the displays, she invites her tion during a visit. In all observed cases, covisitors friend Anna, who is some distance away, to join her experienced their immediate social context through on the visit. Anna joins in by accessing an on-line direct and close interaction at some times, and pe- 3D model of the room.A third friend, Dub, is then ripheral awareness at other times. Each co-visitor invited; he joins them by accessing the Web site for takes advantage of a dynamically changing set of the room.The scenario introduces three different resources. We identified mutual audibility, mutual visitors who are remote from each other, and who visibility and shared content as essential resources have access to different technologies.Such scenarios in co-visiting.These three do not influence each of have proved an effective way for a diverse team of the interaction styles described above to the same researchers to communicate with each other, al- extent. lowing us to shift our focus between studies, design, implementation and evaluation while maintain- Closely connected covisitors use all three of the ing a common context. resources extensively. They have concrete knowledge of their companions' locations and orienta- Vee, or the visitor in the role ofVee, has a handheld tions, which are often similar to their own. They computer, an HP Jornada, which supports wireless talk almost constantly with each other, and share network communications. She also has a jacket content whether in the form of touch screens,video Bristol University's Cyber-Jacket, shown in Figure 5 screens or labels.They can share and explore inter- that has within or on it several devices: an ultra- pretations of content seen or heard previously. sound detector to register position within the room

36 ©Archives & Muse4n11nformatics, 2002 Museums and theWeb 2002

(Randell & Muller, 2001), an electronic compass to show orientation, and a battery. Cables within the jacket connect these parts to each other and to the handheld.The recorded position is updated every few seconds and sent off to another 'server' computer (hidden behind the reception desk of the Mack Room).The handheld computer can be used to browse pages about the Mack Room's displays and artefacts based on the exhibition's catalogue and delivered from the Web server on the local network A number of those pages generally describe thematic zones within the room. We support automatic triggering of those pages: ifVee walks into a thematic zone within the room, a Web page for that zone is automatically displayed. Figure 5. Bristol University's Cyberfacket is Dub has a normalWeb browser running on a laptop worn by the `physical' visitor in the Mack computer.We currently set this laptop inThe Light- Room (on the right of the picture), and has house but away from the Mack Room, and connect location and orientation sensors for a handheld computer. it to the local network. He can browse pages about the Mack Room, and by tracking the zone associated with each page displayed, we imply a 'position' used by this software is the visitor's location and in the room for him. the size of the display. Linky models associations between artefacts, zones and hypermedia content Anna uses aVR model of the Mack Room as well as and, when delivering a Web page for such a loca- aWeb browser. Unlike the Web visitor,Anna's position, it can use these associations to add description is her position within the virtual environment. tions of other things to see and places to move to. We initially experimented with the room-sized The server computer also runs software, called immersive VR display (ReaCTOR) at University EQUIP (www.equator.ac.uk/technology/equip/ College London (UCL), but current experimenta- index.htm), that collects the positions of the vari- tion happens within The Lighthouse, where we use ous visitors, relates them to zones, and triggers any another networked laptop showing simpler 'desk- automatic display of Web pages, maps and avatars. top VR' graphics. EQUIP also handles audio streams between visitors. Each visitor has a microphone and headphones, and can talk with the other two.We initially used mo- Experimenting with the System bile telephones to support conversation, but now have audio connections within our system. Each visi- Our approach to system development is to carry tor can see personal location as well as locations of out short tests or trials, where minor system re- the others.Vee and Dub each have a map showing finements can be made, in between each major de- the Mack Room set within their Web page and velopmental step.The first of these was carried out showing visitors' ongoing positions as dots, as in in two rooms in UCL when we had established an Figure 6.Anna has her covisitors shown as avatars, initial but rough communication between the wear- as in Figure 7. able system and the VR system, i.e. between Vee and Anna.We have just begun a second trial in the A server computer assists the visitors' personal Lighthouse, with the system supporting threeway systems.This runs aWeb server based on the Linky communication between Vee,Anna and Dub. software (www.equator.ecs.soton.ac.uk/technology/ leaky/leaky.shtml) developed by the University of In the UCL trial, we invited colleagues external to Southampton to deliver hypermedia tailored to the the City project to take on the roles of Vee and context of the visitor.At the moment, the 'context' Anna, so that we might gain initial observations of

4 4 ©Archives & Museuin Informatics, 2002 37 Galani and Chalmers, Can You See Me?

VEE

Figure 7.The view of the VR visitor,Anna, shows the Mack Room and avatars of her covisitors.

referring to. Participants' engagement in the shared experience was shown by their use of vocabulary and body movements used in their everyday physical interactions, and by their awareness of each other's relative positions.Two instances of the latter involved the participant Phil (as Vee) who felt that he was in the way of Bill (Anna), and stepped Figure 6. The web visitor, Dub, sees a map of the Mack Room showing the positions back to free the view: of himself and his covisitors, and the outlines of thematic zones. P: Can you read this poster on the wall? B: Where? Which one are you looking at? interaction between a visitor in a virtual environ- That one... ment and a visitor with a handheld device in a physi- P: Oh, I'm sorry, I'm in the way mate. Can cal environment.We videotaped the room where you read it now? Can you read the Willow Vee was, and the VR as seen by Anna. We useda Tearooms? smaller version of the Mack Room and a number of objects related to Mackintosh, such as posters and [...] postcards.This trial took place in UCL, rather than B: ...interferes with the wall. I believe I am the Lighthouse, to let us experiment with an too close to the physical wall. immersive VR display and to use UCL:s infrastructure for recording activity in a virtual environment. P: Oh yeah I'll go out of the way. There One of the participants also wrote a lengthy feed- you go. Don't go through the wall. back document after the event (Stenton, 200 I). B:That's the Willow Tearooms again.

In general, the participants felt that being in differ- P: Ah, the Willow Tearooms... Sorry, you ent geographical spaces did not make mutual aware- must be standing on my foot then ness problematic.They extensively used the audio B: I believe I'm sitting inside you. Yes. Its' connection during the trial, especially when there rather... were issues of shared reference to resolve. With limited facilities for sharing body orientation, they P: Oh... inside me what a thought! often spoke to each other to confirm or question The video footage revealed that in the first of these exactly what they were facing and what they were cases, Phil was not quite in Bill's way. However, Phil's 4 5 38 ©Archives & Museum Informatics, 2002 Museums and theWeb 2002 concern and motion to avoid blocking Bill's view and between collections and the city. Those asso- indicated a sense of shared experience and pres- ciations will form a resource for their later visits, ence,As Phil later wrote: but also for the visits of others.

Clearly our dialogue was one of discovering Secondly, our current prototype system is being the shared experience rather than sharing tailored and adapted to suit the Mack Room and to the visit.A defining moment for me was the take account of the results of our studies and ex- request from my virtual partner,"Get out of ploratory evaluations. Project members are now the way; I can't see the poster?' At this point using the initial prototype along with other depart- it felt like we were in the same room rather mental colleagues, Lighthouse staff and a few mem- than sharing information about two identi- bers of the public. We consider it particularly im- cal rooms in different places (Stenton). portant to have the sociologist and museum studies experts in our project use the system them- With regard to the participants' use of the handheld selves. However, we are not yet undertaking formal and the VR equipment respectively, we observed a or sustained evaluation or observation. Instead, we separate set of issues. One of the common visitor make minor changes to the system from day to day gestures in a traditional museum setting is to point in the light of these informal short-term trials.This at something. In the case of the VR, this is tracked has led to work on reducing the delay between a and displayed;i.e. the avatar has a head and a hand. 'wearable' visitor moving into a thematic zone and We observed, however, that when using the wear- the automatic Web page display, refining the posi- able and handheld, the visitor used his free hand to tion tracking, and changing the design of the pages point at things.This gesture went unnoticed by the shown on the handheld so as to better combine system and consequently by his companion. On the map and content. other hand, the digital visitor had great difficulty reading textual information due to limited resolu- In the course of these early and small design itera- tion in the virtual environment. Barbieri and Paolini tions, we improve our understanding of the Room, (2000) examined the same problem with regard to its visitors and our technology.While we have stud- desktopVR in theVirtual Leonardo project,and they ied the Mack Room and its visitors before, and our used 2D graphics in the delivery of textual infor- system before, the combination is new. We have mation.We use the same solution in the desktop many ideas that may be implemented in the near VR version of our system, although we are also future, such as more explicit support for the visi- considering the use of a second handheld device to tors acting as guides, graphical representation of display textual information to the VR visitor,Anna. visitors' paths through the room, and recommen- dation of people, places and artifacts. Still,we wish Field trials inside The Lighthouse are, at the time of to leave space for new ideas and priorities to grow writing, still in progress.The next section outlines from our ongoing evaluation and observation. In this and other ongoing work. pausing before a major redesign, we wish to secure a balance between the studies of visitors, studies of Ongoing and Future work visitors using our system, technological interests in citywide use and adaptive information, and theo- So far we have discussed two parallel strands of retical work on notions of social context, interpre- our work.The studies and system development have tation and media. raised our awareness of social context issues, and of the possibilities and the limits of our technology. Conclusion While the studies continue, the system work has subdivided into two strands. First, our technologi- Social context is an essential factor of the museum cal infrastructure is being refined and expanded to experience. Social interaction with companion(s) support future systems where we support visitors and other members of the public directly influences moving between buildings and exhibitions across a museum experience. In the light of new technolo- the city, and where visitors can create associations gies that support remote access to museum set- within a museum collection, between collections, tings, we can see that the social context of the

A r- @Archives & Musegaformatics, 2002 39 Galani and Chalmers, Can You See Me?

museum visit can extend beyond a physical room Diamond, J. (1986).The behaviour of Family or building, and include computer-mediated inter- Groups in Science Museums. Curator, 29(2). actions.Similarly, museums can enrich the digital site 139-154. by better integrating it with the physical site. The City project aims at both understanding andsup- Falk, H. J. & L. D. Dierking, (1992). The museum porting social interaction in cultural institutions; experience.Washinghton:Whalesback Books. interaction that may be synchronous or asynchronous, and that may involve multiple media such as Galani,A. (2001). Vee in the Mackintosh Room. mobile computers, hypermedia and virtual environ- Consulted February II, 2002. (http:// ments. Our system supports interaction between www.dcs.gla.ac.uk/equator/200 I /05/ physical and digital visitors, as well as between visi- scenario.html). tors and their environments, in a dynamic and contextual way. Our aim is a canvas of interaction that Lord, M. (1999). Editorial. Museum International, spans different places, people and times, with a fo- October 1999,Vol. 51 No 4. 3-3. cus on the creation and delivery of cultural information that is contextually appropriate, useful and MacDonald, S. (1993).The enigma of the visitor engaging. sphinx. In S. Bicknell & G. Farmelo (Eds.) Museum Visitor Studies in the 90s. London: Acknowledgements Science Museum, 77-85.

Our thanks go to Ian Mac Coll and Barry Brown McManus, RM. (1987). It's the Company you (Glasgow), Cliff Randell (Bristol), Anthony Steed Keep... The Social Determination of Learn- (UCL), Chris Greenhalgh, Ian Taylor &Tom Rodden ing-related Behaviour in a Science Museum. (Nottingham), Dave Millard (Southampton), Dorothy International Journal of Museum Management McKay (The House for an Art Lover), Lynn Bennett and Curatorship, 6. 263-270. and Stuart MacDonald (The Lighthouse).We grate- fully acknowledge the EPSRC's funding, and the do- Oppermann, R. & M. Specht, (1999).A Nomadic nation of equipment within Hewlett Packard's Art Information System for Adaptive Exhibition & Science programme. Guidance. In D. Bearman & J.Trant (Eds.) Cultural Heritage Informatics:selected papers References from ichim99.Archives and Museum Informat- ics. 103-109. Aoki, R M. & A.Woodruff, (2000). Improving Electronic Guidebook Interfaces Using a Psarra, S. & T. Grajewski & M. O'Neil, (2002). Task-Oriented Design Approach. In Designing Rethinking museum designthe effect of Interactive Systems 2000.ACM. 319-325. layout on visitors patterns of movement and use. Museum Practice, March 2002. Barbieri,T. & P Paolini, (2000). Cooperative Visits for Museum WWW Sites aYear Later: Randell, C. & H. Muller, (200 I ). Low Cost Indoor Evaluating the Effect. In D. Bearman & J.Trant Positioning System. In G.D.Abowd, B. Brumitt (Eds.) Museums and the Web 2000.Archives & & S. Shafer (Eds.) Ubicomp 2001: Ubiquitous Museum Informatics. Computing. Springer. 42-48.

Chalmers, M. (200 I). Tales in the City:Adaptive Stenton, P (200I).A user's view of the Equator Information in the Physical City. Consulted Mackintosh Room Demo. Consulted Febru-

February 11, 2002. (http://www.dcs.gla.ac.uk/ ary 1I, 2002. (http://www.dcs.gla.ac.uk/ equator/2001/01/vee.html). equator/200 I /06/stenton/stenton.html).

Communications Design Team (1976). Spatial Vom Lehn, D., C. Heath & J. Hindmarsh, (2001). considerations. In Communications Design Exhibiting Interaction: Conduct of Collabora- Team (Eds.) Communicating with the Museum .4'1' tion in Museums and Galleries. Symbolic Visitors. Royal Ontario Museum. Interaction,Vol. 24 No 2. 189-216.

40 @Archives & Museum Informatics, 2002 The Museum Wearable: real-time sensor-driven understanding of visitors' interests for personalized visually-augmented museum experiences

Flavia Sparacino, MIT Media Lab, USA

http://www.media.mitedulflavialprojects.html

Abstract

This paper describes the museum wearable: a wearable computer which orchestrates an audiovisual narration as a function of the visitors' interests gathered from their physical path in the museum and length of stops.The wearable consists of a lightweight and small computer that people carry inside a shoulder pack. It offers an audiovisual augmentation of the surrounding environment using a small, lightweight eye-piece display (often called private-eye) attached to conventional headphones. Using custom built infrared location sensors distributed in the museum space, and statistical mathematical modeling, the museum wearable builds a progressively refined user model and uses it to deliver a personalized audiovisual narration to the visitor.This device will enrich and personalize the museum visit as a visual and auditory storyteller that is able to adapt its story to the audience's interests and guide the public through the path of the exhibit. Keywords:wearable computer, handheld, personalized experience, Bayesian networks, location aware mobile computing

I. Introduction

In the last decade museums have been drawn into displayed objects. Each of these storytelling aids has the orbit of the leisure industry and compete with advantages and disadvantages. Catalogues are usu- other popular entertainment venues, such as cin- ally attractive and well done, yet they are often too emas or the theater, to attract families, tourists, cumbersome to carry around during the visit as a children, students, specialists, or passersby in search means to offer guidance and explanations. Guided of alternative and instructive entertaining experi- tours take away from visitors the choice of what ences. Some people may go to the museum for mere they wish to see and for how long. They can be curiosity, whereas others may be driven by the de- highly disruptive for the surrounding visitors, and sire for a cultural experience.The museum visit can their effectiveness strictly depends on the knowl- be an occasion for a social outing, or become an edge, competence, and communicative skills of the opportunity to meet new friends. While it is not guide.Audio tours are a first step to help augment possible to design an exhibit for all these catego- the visitor's knowledge.Yet when they are button ries of visitors, it is desirable to offer exhibit de- activated, as opposed to having a location identifi- signers ways to attract as many as possible amongst cation system,they can be distracting for the visitor. the variety of individual visitors or visitor catego- The information conveyed is also limited by the only- ries.Technology can help offer the museum public audio medium: it is not possible to compare the opportunities to personalize their visits according artwork described with previous relevant produc- to their desires and expectations. tion of the author, nor to show other relevant images. Interactive kiosks are more frequently found Traditional storytelling aids for museums have been today in museum galleries. Yet they are physically signs and text labels, spread across the exhibit space; distant from the work they describe, thus do not exhibit catalogues, typically sold at the museum support the opportunity for the visitors to see, com- store; guided tours, offered to groups or individu- pare, and verify the information received against the als; audio tours; and more recently video or multi- actual object.The author's experience suggests that media kiosks with background information on the when extensiveWeb sites are made available through

4 3 BEST COPY AVAILABLE Museums and theWeb 2002:Proceedings Archives & Museum Informatics, 2002, p. 41 Sparacino,The Museum Wearable

Reality Computer Augmentation""" Augmented Reality

video ft ptctu re John Smith. 1034. augmented reality ntrt art. Rutherford ,o1k0,11.11, glasses and 5..q...1 on July 2. 1991. headphones

power unit Front view Side view r utrnhlenr.

carrying backpack infrared location Wearable IR receiver Computer body motion small, lightweight sensor augmented reality computer glasses and headphones Figure l.The Museum Wearable: explanation of concept and application interactive kiosks placed along the museum galler- can be personalized to be able to dynamically cre- ies, these may absorb lengthy amounts of the visi- ate and update paths through a large database of tors' museum time, thereby detracting from, rather content and deliver to the users in real time during than enriching, the objects on display. Finally, panels the visit all the information they desire. If the visi- and labels with text placed along the visitors' path tors spend a lot of time looking at a Monet, the can interrupt the pace of the experience as they system needs to infer that the users likes Monet, require a shift of attention from observing and con- and update the paths through the content to take templating to reading and understanding (Klein, that into account.This research proposes the mu- 1986). seum wearable as an effective way to turn this scenario into reality. Indeed, when we walk through a museum there are so many different stories we could be told. Some of This document illustrates the hardware, authoring these are biographical about the author of an art- techniques, and software created for the construc- work; some are historical and allow us to compre- tion of the museum wearable. It speculates on its hend the style or origin of the work; and some are ability to facilitate a new style of exhibit design, and specific about the artwork itself, in relation to other postponesits, as future work, the assessment at the artistic movements. Museums usually have largeWeb exhibit's site, of the museum wearable's contribu- sites with multiple links to text, photographs, and tion to the public's experience. movie clips to describe their exhibits.Yet it would take hour's for a visitor to explore all the informa- 2.The Museum Wearable tion in a kiosk, to view theVHS cassette tape associated with the exhibit, and read the accompanying Wearable computers have risen to the attention of catalogue. Most people do not have the time to technological and scientific investigation (Starner, devote nor motivation to assimilate this type of in- 1997) and offer an opportunity to "augment" for formation, and therefore the visit to a museum is visitors their perception/memory/experience of the often remembered as a collage of first impressions exhibit in a personalized way. The museum wear- produced by the prominent feature of the exhibits, able is a wearable computer which orchestrates an and the learning opportunity is missed. How can audiovisual narration as a function of the visitors' we tailor content to the visitors in a museum, dur- interests gathered from their physical path in the ing their visits, to enrich both the learning and en- museum and length of stops. It offers a new type of tertainment experience ?We want a system which entertaining and informative museum experience,

42 © Archives &MultaInformatics, 2002 Museums and the Web 2002

ok_ vader. ,1-41114,

Figures 2, 3. Camera "wearing" the head mounted display: shows how the user's brain assembles the real world's image seen by the unencumbered eye with the display's image seen by the other eye, into a fused augmented reality image.

more similar to mobile immersive cinema than to tions of commercially available infrared location the traditional museum experience [figure I]. identification systems such as short range and nar- row cone emission.The location system is made by The museum wearable is made by a lightweight CPU a network of small infrared devices, which transmit hosted inside a small shoulder pack and a small, light- a location identification code to the receiver worn weight private-eye display.The display is a commer- by the user and attached to the display glasses.The cial monocular,VGA-resolution, color, clip-on screen transmitters have the size of a 9V battery, and are attached to a pair of sturdy headphones. When placed inside the museum, next to the regular mu- wearing the display, after a few seconds of adapta- seum lights.They are built around a microcontroller, tion, the user's brain assembles the real world's and their signal can be reliably detected at least as image, seen by the unencumbered eye, with the far as 30 feet away within a large cone range of display's image seen by the other eye, into a fused approximately ten to thirty degrees, according to augmented reality image [figures 2, 3]. the area that needs to be covered, and up toI 00 feet along a straight line.The emitter location iden- The wearable relies on a custom-designed long- tification tags have been embedded inside standard range infrared location-identification sensor to light fixtures to allow the exhibit designer to easily gather information on where and how long the visi- place them in the museum, next to the regular tor stops in the museum galleries.A custom system museum lights, and using the same power rack as had to be built for this project to overcome limita- the regular museum spotlights [figures 4, 5, 6, 7].

AN1b7..

Figures 4, 5, 6, 7. Location sensor: emitter tags embedded inside light fixtures

rf; ©Archives & Museaphtlliformatics, 2002 43 Sparacino,The Museum Wearable

USB port which is connected to the infrared receiver with a USB to serial converter, and a VGA and headphone output, which are connected to the video/audio inputs of the head-mounted display of the museum wearable. It also supports one type II card, which can be used to host a PCMCIA card for wireless communication over the Internet or a PCMCIA video acquisition card.All these features, combined with a battery life of 2.5-5.5 hours with the standard lightweight battery, large enough for a single museum visit, make the picturebook an ideal Figure 8. PAQ pocket PC 3670 and Figure 9. choice for the selected application [diagram I]. Sony picturebook with removed LCD An alternative to the picturebook is the smaller In view of having a museum wearable which can handheld IPAQ pocket PC 3670 [figures 8, 9].The later be expanded to include other sensors, and iPAQ 3670 features 64 MB of SDRAM and a 206- process information not just from the infrared lo- MHz Intel StrongARM SA- I 110 32-bit RISC Pro- cation sensor, but for example also from a small cessor. It has USB or serial connectivity that would camera processing images in real time, a commer- interface with the infrared receiver of the museum cially available processing unit has been selected for wearable and it is only 5.11" x 3.28" x 0.62" this project.The processor of choice is a small-sized (HxWxD) in size.The iPAQ 3670 is a desirable so- laptop computer: the SONY picturebook PCG- lution for the deployment of several museum C I VPK, selected for its combined size, weight, com- wearables which need to work only with the lo- puting power, multimedia capabilities, and long-last- cation sensorin a museum.The iPAQ solution is ing batteries. Given that the images generated by cost effective because its price, with the necessary the laptop are viewed uniquely through the head accessories, is about half the price of the SONY mounted display, the LCD screen has been removed picturebook, and its size is also smaller. from the picturebook, to reduce weight and size. The picturebook features a CrusoeTM processor The size and weight of both the wearable's proces- TM5600 clocked at 667 MHz, and without the LCD sor and augmented reality display are critical for a weighs only approximately one lb, and has a size of museum application.The display cannot have a heavy 0.5" X 9.8" X 6.0" (H x W x D). The picturebook and power hungry powering unit which requires has aI 5GB capacity hard drive, which allows the frequent battery changes. Glasses also need to eas- programmer to store on the local hard drive sev- ily fit various people's head sizes, with annexed hair eral hours (8-10) of MPEG-compressed VGA reso- style, which is not an easy task.The wearable would lution video (640x480) (approximately one hour of be handed out to between ten and one hundred MPEG-compressed 640x480 video per one GB of people a day, and therefore needs to be of robust available space on the internal hard drive). It also assembly and easy to wear. Two different design has 128 MB SDRAM, which allows the computer to solutions were implemented after a thorough it- play smoothly audio and video clips, as well as pro- erative design process which included consider- cess images in real time when the computer is con- ations about video resolution, power consumption, nected to a camera.The external ports include one purchase availability and foremost, ergonomics, with

HARDWARE

IR EMITTER IR SERIAL PENTIUM II I VGA k I HEAD MOUNT DISPLAY TAG RECEIVER 400 MHz

Diagram I:The museum wearable hardware architecture 51 44 ©Archives & Museum Informatics, 2002 Museums and theWeb 2002

Figures 10,1 1.The common fashion and high fashion private eye display

the inclusion of adaptability with sensors, weight, HMD's powering unit: hosted inside the contain- and size. The common fashion display features an ing shoulder pack augmented reality display, which joins together a Infrared receiver:the sensor is located on top of lightweight VGA resolution color display from the the headphones and the receiver circuit is located Micro Optical corporation, and a commercial high inside the containing shoulder pack. quality sturdy set of headphones [figureI 0]. The The museum wearable plays an interactive audiovi- high fashion display is a provocative stylish mount, sual documentary about the displayed artwork on mainly intended for visitors with a strong sense of the private-eye display. Each mini-documentary is aesthetics, and suitable for use in wearable fashion made of small segments which vary in size from shows, to promote a non-nerdy and highly fashion- twenty seconds to one and half minutes. A video able wear of augmented reality displays.With this server, written in C++ and DirectX-8, plays these design the MicroOptical augmented reality display assembled clips and receivesTCP/IP messages from is rigidly mounted to a pair of Oakley "over the another program containing the information mea- top" glasses, as illustrated in figure II . sured by the location ID sensors.This server-client architecture allows the programmer to easily add Summary of elements of the museum wearable other client programs to the application, to com- hardware: municate to the server information from other possible sources such as sensors or cameras placed containing shoulder pack along the museum aisles to measure how crowded computer (CPU):SONY picturebook from which the galleries are or how often a certain object has the display has been removed to reduce weight been visited.The client program reads IR data from Head Mounted Display (HMD):VGA resolution the serial port, and the server program does infer- MicroOptical clip-on mounted on sturdy head- ence, content selection, and content display using phones with a custom mount DirectX for full screen play back of the MPEG compressed clips [diagram 2]. 52

MAIN THREAD SOF1WARE SECONDARY THREAD

SERVER USES SENSOR/TAG INFO C MODULE PROGRAM TO UPDATE PROBABILITIES WHICH READS TCP/IP COMMUNICATESIN THE BAYESIAN NETWORK IRTAG SERIAL/TAG CLIENT PROGRAM INFO TO THE CONTENT SELECTION AND MAIN THREAD -r&PLAYOUT USING DIRECT-X

I CLIENT PROGRAM 1 CLIENT PROGRAM 2j

WIRELESS LAN

FIXED FIXED COMPUTER 1 COMPUTER 2

SENSORS ATTACHED TO FIXED COMPUTERS AND PLACED ALONG THE VISITOR'S PATH

Diagram 2:The museum wearable software architecture

3.Visitors' interests and visitor types groups.While this colorful classification well depicts To identify visitor preferences, museums seek to the variety of public that museums need to attract, identify target groups: individuals who share com- entice and educate, it is too sophisticated to model mon traits such as culture, ethnic or social affilia- mathematically, at least initially. tion, educational level, and leisure preferences. Cu- rators and designers need to assess the basic knowl- More usefully for this research, Dean generalizes edge and expectations of these groups to be able museum visitors in three broad and much simpler to reach, communicate with, and stimulate curios- categories (Dean, 1994, pp. 25-26).The first category ity in all of their visitors. Eleanor Hooper-Greenhill includes what he calls the "casual visitors": people identifies target groups which include families, school who move through a gallery quickly and who do parties, other organized educational groups, leisure not become heavily involved in what they see. Ca- learners, tourists, the elderly, and people with visual visitors use some of their leisure time in muse- sual, auditory, mobility or learning disabilities ums but do not have a strong stimulus or motiva- (Hooper-Greenhill, 1999, p. 86). She then suggests a tion to deepen their knowledge about the objects partition of museum resources, to target, attract on display.The second group, the "cursory visitors," and entertain these different groups. During a per- show instead a more genuine interest in the mu- sonal interview, Beryl Rosenthal, director of exhi- seum experience and their collections. According bitions at the MIT Museum, described a more so- to Dean, these visitors respond strongly to specific phisticated visitor type classification. She identified objects that stimulate their curiosity and wander stroller moms, accompanied by children three years through the gallery in search of further such stimu- old or younger; window shoppers: families who lus for a closer exploration of the targeted objects. cruise through the museum in search of an alterna- They do not read every label nor absorb all avail- tive leisure experience; button pushers, typically able information, but will occasionally read and adolescents; school groups; the date crowd; and the spend time in selected areas or with selected ob- PhDs, who want to know (and criticize) everything jects of interest they encounter in the galleries.The in the museum. Young visitors, children 5-14 also third group is a minority of visitors who thoroughly represent a separate group of visitors with differ- examine exhibitions with much more detail and ent learning needs and curiosities than the other attention.They are learners who will spend an abun-

53 46 ©Archives & Museum Informatics, 2002 Museums and theWeb 2002 dance of time in galleries, read the text and labels, and closely examine the objects. Dean attributes 4. Sto(ry)chastics: Sensor-driven differences between "people who rush", "people Understanding of Visitors' Interests who stroll", and "people who study" to different with Bayesian Networks prior experiences and educational level.Yet he states In order to deliver a dynamically changing and per- that it is important for museums to be equipped to sonalized content presentation with the museum communicate with and interest all visitors by scal- wearable, a new content authoring technique had ing and designing an exhibit so that it offers enter- to be designed and implemented.This called for an tainment to the "stroller" as well as an opportunity alternative method than the traditional complex to deepen knowledge for the "learners". centralized interactive entertainment systems which simply read sensor inputs and map them to actions Serrell (1996) also divides visitors into three types: on the screen. Interactive storytelling with such one- the transient, the sampler and the methodical view- to-one mappings leads to complicated control pro- ers. She notes that currently museum evaluators grams which have to do an accounting of all the are using terms like "streakers, studiers, browsers, available content, where it is located on the display, grazers and discoverers" to characterize museum and what needs to happen when/if/unless. These visitors' styles of looking at exhibits. But she con- systems rigidly define the interaction modality with cludes that this type of categorization is not useful the public, as a consequence of their internal archi- for summative evaluation, suggesting that it is a sub- tecture, and lead to presentations which have little jective method of classification, and that it is not depth of content, are hard to modify, ad hoc, and fruitful to try to create exhibitions that serve these prone to error. The main problem with such con- different styles of visiting. Instead she suggests that tent authoring approaches is that they acquire high a more objective means of classification be found, complexity when drawing content from a large da- such as average time spent in the exhibition space. tabase, and that once built, they are hard to modify or to expand upon. In addition, when they are sen- In accordance with the simplified museum visitor sor-driven they become depended on the noisy typology suggested by Dean and Serrell, the mu- sensor measurements, which can lead to errors and seum wearable identifies three main visitor types. misinterpretation of the user input (Sparacino,I999). To offer a more intuitive understanding of these Rather than directly mapping inputs to outputs, the types they have been renamed: the busy, selective, system should be able to"understand the user" and and greedy visitor type.The greedy type wants to to produce am output based on the interpretation know and see as much as possible, and does not of the user's intention in context. have a time constraint; the busy type just wants to get an overview of the principal items in the ex- To overcome the limitations of one-to-one map- hibit, and see little of everything; and the selective ping systems, the mathematical modeling approach type wants to see and know in depth only about a developed to author content for the museum wear- few preferred items.The identification of other visi- able is a real-time sensor-driven stochastic model- tor types or subtypes has been postponed to fu- ing of story and user-story interaction. It has there- ture improvements and developments of this re- fore been called sto(ry)chastics.Sto(ry)chastics uses search. graphical probabilistic modeling of story fragments and participant input,gathered from sensors, to tell The visitor type estimationisobtained a story to the user, as a function of estimated inten- probabilistically with a Bayesian network using as tions and desires during interaction.With this ap- input the information provided by the location iden- proach, the coarse and noisy sensor inputs path tification sensors on where and how long the visi- and length of stops given by the location sensors tor stops, as if the system were an invisible story- are coupled to digital media outputs via a user teller following the visitor in the galleries and try- model, estimated probabilistically by a Bayesian net- ing to guess preferences based on observation of work. external behavior. A Bayesian network is a graphical model which encodes probabilistic relationships amongst variables

51 © Archives & Museum Informatics, 2002 47 Sparacino, The Museum Wearable

together to build a model.This allows the digital architect, or the engineer, to communicate on the same ground (the graph of the model) as the curator and therefore to be able to encapsulate the curator's domain knowledge in the network, together with the sensor data.

Sto(ry)chastics uses a Bayesian network to estimate the user's preferences taking the location identification sensor data as the input or observations of the network. The user model is progressively refined as the visitor progresses along the museum galleries: the model is more accurate as it gathers more observations about the user. Figure 12 shows the Bayesian network for visitor estimation, limited Figure I 2.Topology of the Bayesian network to three museum objects (so that the figure can fit for visitor time estimation, limited to three in the document), selected from a variety of pos- objects/time slices. sible models designed and evaluated for this research. Figures I 3 and 14 show state values for the of interest (Pearl, 1988; Jordan, 1999). Such graphs network after the visitor has made a long stop at not only provide an attractive means for modeling the first object, followed by another long stop at and communicating complex structures, but also the second object. form the basis for efficient algorithms, both for propagating evidence and for learning about param- The sto(ry)chastics approach has several advantages. eters. Bayesian networks encode qualitative influ- It is: ences between variables in addition to the numeri- cal parameters of the probability distribution. As I .Flexible: it is possible to easily test many differ- such they provide an ideal form for combining prior ent scenarios by simply changing the parameters knowledge and data. By using graphs, not only does of the system. it become easy to encode the probability indepen- dence relations amongst variables of the network, 2.Reconfigurable: it is also quite easy to add or but it is also easy to communicate and explain what remove nodes and/or edges from the network the network attempts to model (Smyth, 1997). without having to "start all over again" and specify Graphs are easy for humans to read, and they help again all the parameters of the network from focus attention, for example in a group working scratch.This is a considerable and important ad-

El =.:. .M 3181 neutral 3116 neutral 34.25 neutral ----/' 33.70 neuttei I 45.57 iterestkg 38.23 interesting 41.86 ittesesting 36.74 interesting 21.52 boxing 28.61 boring nes Wiwi 2156 boring

El El

El A 11.39 busy LSO busy 58.23 greedy 77.80 greedy 30.38 selective 2030 selective

Figures 13, 14. Values of the network nodes after the visitor has madea long stop at the first object followed by another long stop ot the second object.

Jrz 48 ©Archives & Museum Informatics, 2002 BEST COPY AVAILABLE Museums and theWeb 2002

vantage with respect to hard coded or heuristic 5. Context-sensitive: for any system to be robust approaches to user modeling and content selec- and to provide relevant information to its user, it tion. Only the parameters of the new nodes and is important to model the context of interaction the nodes corresponding to the new links need together with the other system parameters. For to be given.The system is extensible story-wise example sto(ry)chastics could provide an expla- and sensor-wise.These two properties:flexibility nation of a visitor's change of behavior at the and ease of model reconfiguration, allow the sys- museum. If suddenly a greedy type starts making tem engineer, the content designer, and the ex- short stops, the system, before concluding that hibit curator to work together and easily and that visitor is actually a selective or busy type, cheaply try out various solutions and possibili- could test if the current time is near closing time ties until they converge on a model which satis- for the museum galleries, or if, by use of other fies all the requirements and constraints for their room sensors, there is a great crowd in the gal- project. A network can also rapidly be leries where the visitor is making short stops. reconfigured for another exhibit. Coming up with the right conclusions, given this type of external information, means that the sys- 3. Robust: Probabilistic modeling allows the system is context-sensitive. tem to achieve robustness with respect to the coarse and noisy sensor data. Sto(ry)chastics has therefore implications both for the human author (designer/curator) who is given 4. Adaptive: sto(ry)chastics is adaptive in two ways: a flexible modeling tool to organize, select, and de- it adapts both to individual users and to the en- liver the story material, as well as the audience, who semble of visitors of a particular exhibit. For in- receives personalized content only when and where dividuals, even if the visitor exhibits an initial it is appropriate. "greedy" behavior, it can later adapt to the visitor's change of behavior. It is important to notice that, 5. Experimentation: the Museum reasonably and appropriately, the system "changes its mind" about the user type with some inertia: Wearable at the MIT Museum's i.e. it will initially lower the probability for a greedy Robots and Beyond Exhibit type until other types gain probability. The ongoing robotics exhibit at the MIT Museum Sto(ry)chastics can also adapt to the collective provided an excellent platform for experimenta- body of its users. If a count of busy/greedy/selection and testing with the museum wearable [figures tive visitors is kept for the exhibit, these num- 15,16 ].This exhibit, called Robots and Beyond, and bers can later become priors of the correspond- curated by Janis Sacco and Beryl Rosenthal,features ing nodes of the network, thereby causing the landmarks of MIT's contribution to the field of ro- entire exhibit to adapt to the collective body of botics and Artificial Intelligence.The exhibit is orga- its users through time.This feature can be seen nized in five sections: Introduction, Sensing, Moving, as "collective intelligence" for an exhibit which Socializing, and Reasoning and Learning, each includ- can adapt not just to the individual visitors but ing robots, a video station, and posters with text and also to the set of its visitors. photographs which narrate the history of robotics

. 4 MS_

Figures 15 and 16.Visitor with the Museum Wearable at MIT Museum's Robots and Beyond Exhibit

©Archives & Museum InSlatics, 2002 49

BEST COPY AVAILABLE Sparacino,The Museum Wearable

Itrinr601'leertZt61 "thb'P.6 1ST ..eitgezzai) 0 NOW

CylCIOR PlAtI Figure I 7. Example of annotations of visitor's path and stop duration at MIT Museum's Robots and Beyond exhibit

at MIT.There is also a large general purpose video tracking team had a map and a stop watch. Their station with large benches for people to have a task was to draw on the map the path of individual seated stop and watch a PBS documentary featur- visitors, and annotate the locations at which visi- ing robotics research from various academic insti- tors stopped, the object they were observing, and tutions in the country. for how long they stopped. In addition to the tracking information, the team of evaluators was asked In order to set the initial values of the parameters to assign a label to the overall behavior of the visitor, of the Bayesian network, experimental data was according to the three visitor categories earlier de- gathered on the visitors' behavior at the Robots scribed:"busy","greedy", or "selective" [figure 17]. and Beyond exhibit.According to the VSA (Visitor Studies Association, http://museum.cl.msu.edu/vsa), A subset of twelve representative objects of the timing and tracking observations of visitors are of- Robots and Beyond exhibit were selected to evalu- ten used to provide an objective and quantitative ate this research, to shorten editing time. The ge- account of how visitors behave and react to exhibi- ography of the exhibit needed to be reflected into tion components. This type of observational data the topology of the network, as shown in figure 18. suggests the range of visitor behaviors occurring in Additional objects/nodes of the modeling network an exhibition, and indicates which components at- can be added later for an actual large scale installa- tract, as well as hold, visitors' attention ( in the case tion and further revisions of this research. of a complete exhibit evaluation this data is usually accompanied by interviews with visitors, before and The visitor tracking data is used to learn the pa- after the visit). rameters of the Bayesian network.The model can later be refined; that is, the parameters can be fine- During the course of several days a team of col- tuned, as more visitors experience the exhibit with laborators tracked and made annotations about the the museum wearable.The network has been tested visitors at the MIT Museum. Each member of the and validated on this observed visitor tracking data 57 50 @Archives & Museum Informatics, 2002 BESTCOPYAVAILABLr Museums and the Web 2002

tItn0 CLIss:rsatan 77'

VISITOR TYPE inpSusan-

trcea c-a

Otwei Fer171 a StrAVIEIMED

OUT

ibetv,!'srARr 'St3tIon

SyStan X g) ortut)t.21 PtArxs

4,1> 1I Untroo

V-Isky Arm

ricltrotom aro _

Figure 18.The geography of the exhibit needs to be reflected into the topology of the network by parameter learning using the Expectation Maxi- Biography of author: anecdotes, important mization (EM) algorithm, and by performance analy- people in artist's life (answers: who) sis of the model with the learned parameters, with History of the artwork: previous relevant work a recognition rate of 0.987 (Sparacino, 2001). of the artist Context: historical, what is happening in the The system works in two steps. The first is user world at the time of creation type estimation as described above.The next step Process: particular techniques used or invented is to assemble a mini-story for the visitor, relative to create the artwork (answers: how) to the nearest object [figures 19, 20, 21]. Most of Principle: philosophy or school of thought the the audio-visual material available for use by the author believes in when creating the artwork (an- museum wearable tends to fall under a set of char- swers: why) acterizing topics, which typically define art and sci- Form and Function: relevant style, form and ence documentaries.This same approach to docu- function which contribute to explain the artwork mentary as a composition of segments belonging Relationships: how is the artwork related to to different themes has been developed by Houbart other artwork on display in her work which edits a documentary based on Impact: the critics' and the public's reaction to the viewer's theme preferences, as an offline pro- the artwork cess (Houbart, 1994). The difference between Houbard's work and what the museum wearable This project required a great amount of editing to does is that the museum wearable performs edit- be done by hand (non automatically) in order to ing in real time, using sensor input and Bayesian segment the two hours of video material available network modeling to figure out the user's prefer- for the Robots and Beyond Exhibit at the MIT mu- ences (type).After an overview of the audio-visual seum in the smallest possible complete segments. material available at MIT's Robots and Beyond ex- After this phase,all the component video clips were hibit, the following content labels, or bins, were iden- given a name, their length in seconds was recorded tified to classify the component video clips: into the system, and they were also classified according to the list of bins described above.The clas- Description of the artwork: what it is, when sification was done probabilistically; that is, each clip it was created (answers: when, where, what) has been assigned a probability (a value between

4:1

Figures 19, 20.Visitor wearing the museum wearable and receiving an audiovisual story about the displayed artwork (picture in picture). zero and one) of belonging to a story category.The passed on to the visitor, and that the story is a"good sum of such probabilities for each clip needs to be story", in that it respects basic cause-effect rela- one.The result of the clip classification procedure, tionships and makes sense to humans. Therefore for a subset of available clips, is shown in table I. the Bayesian network described in the previous paragraph needs to be extended with additional To perform content selection,"conditioned" on the nodes for content selection [figures 22, 23]. The knowledge of the visitor type, the system needs to additional "good story" node, encodes, as prior be given a list of available clips and the criteria for probabilities, the curator's preferences about how selection.There are two competing criteria: one is the story for each object should be told. given by the total length of the edited story for each object, and the other is given by the ordering A study of how content is distributed geographi- of the selected clips.The order of story segments cally along the exhibit, both in two and three di- guarantees that the curator's message is correctly mensions. was also performed.The purpose of this COg - Greedy Type La

It- Greedy lYpe !it 111111%1 It - Busy lYpe

Kismet: Selective Type

Kismet: Busy Type

Figure 21. Storyboards from various video clips shown on the museum wearable's display at MIT Museum's Robots and Beyond Exhibit 5 9 52 ©Archives & Museum Informatics, 2002 Museums and theWeb 2002

linleskie Metro Casten 0531Ature Coghisbory 021 090 043 031 0.7 0.2 0.3 0 0 0.1 0 0 0 0.8 0 0 0 0 0 0.1 0.2 0 0 0.1 0 0 0.15 0 0 0 0.4 0.1 1 0.2 0 0.2 0 o 0.2 0.1 0 0 o o 0 o o o o 1 1 1 1

CogIntio CleidealgaOaxhiteation Dexintso Docellrhiese 041 114 034 072 096 0.5 04 0.3 0.8 0.4 0 0 0 0 0 0 0 0 0.2 0 0 0.2 0 0 0 0.2 0.3 0 0 0.3 0 0.3 0.1 0.4 0 0.4 0.5 0 0 0 0 0 0 0 0 0 o 0.5 o 1 s 1 1 1

6A111992.1110 TIMM Mixt IthicyrithiehloRierlevelopKiereceeeneor MOM 1.11947114 IN MEM 026 067 066 160 066 Dasalption SIC 0.6 0 0.1 0.2 0.2 norm 115r 0 0 0.2 0 0 Canned CrX 0 0 0 0 0 likoraphy BED 0.2 0.9 0 0 0 Prow= PRC 0 0 0.1 0.9 0.13 Prindple 0 0 0.2 0 0.3 Fano Functlan PAP 0 0 0.4 0.2 0.1 RelatIoneldpe Rn 0 0.1 0 0 0 LflZNoi DIP 0 0 0 0 0 Tbtal P 1 1 1 9 1

r, rrr Irlaradd Log30blard laglartbyIngflandego logliangany 113151Th DI WC! 153 055 031 043 .4 tliatery KST 0 0 0 Contort C7X 0 1Cgrap6er BZO 0 0.2 Proms PRC 0.6 0.5 0.3 Principle PBC 0 0.1 Pane Mimetic./ PAP 0 11614316641,1F4 In Impace WIP 0 0 1

Table I. Selected segments cut from the video documentation available for the MIT Museum's Robots and Beyond ExhibitAll segments have been assigned a set of probabilities which express their relevance with respect to nine relevant story themes or categories.

25.1 2g 3344872 noutd 0342241 re.trol 3227580 ireotrog 0.3221379 3327535 tang 5320379 baitg

iii 0233491 ckm

rem 1 a 455302 41.4 02E0235 1.52 2g 0E55177 1.9 mom gady 0344135 tedine

Q7 shod 13 long

2<_1 03 DSC good 01 HST gm] (103 CTX -good o1-.7 klocyrd--dro (110 010 good 115672 04 mug kiodovetO au FRC- pod 6a0724 esg togfid 55E03 Way kitacotenzon POIC pod 524724 go mete. 07 1028 (iv Irk.. 309 FAF - good 724724 am coglum 24 18212 *03 REL -good 519329 042 kicenotioro cum BO good

Figures 22, 23. Extension of Bayesian network to perform content selection. 60 @ Archives & Museum Informatics, 2002 53 BEST COPY AVAILABLE Sparacino, The Museum Wearable

04117 80 oemberems.

10-.71461 poZit:n 'WC tK 113T Cereal clx OC...vvC7 PCIC twcput sutsbe :P,PZ-7 p4^C 1 ()FaroAr.,.emat FAR '" 0 II Racana."4..., 1,21. CY.* C L-1 le? ---I OUT ENTRARTO '3mx-mAsr

LAtgiltA4 3 8,1SYava RD= (.>

hoirt4AusEum

FLO4111OR

Figure 24.Two dimensional representation of content distribution for MIT Museum's Robots and Beyond Exhibit. study was to visualize the different stories for dif- (Healey, 1999). If for example the GSR sensor mea- ferent visitors edited by the museum wearable as sures a train of peaks when the wearable is playing paths through the hyperspace of content in the a segment with biographical information about the exhibit.The 2D study shows colored pie charts in portrayed artist, the system can infer, with a certain the vicinity of the twelve tracked objects at the probability, that the visitor has a strong interest in museum. Each pie chart represents the content avail- this topic i.e. biography. It will then update the visi- able for the corresponding object.The size of the tor interest profile with the gathered visitor pref- pie chart is proportional to the amount of content erences.The probabilistic framework offered by the available for that object The size of the colored Bayesian network approach is particularly relevant slices of the chart represents the contribution of for this type of sensor. For example,the sensor could each story bin to the content available for the ob- measure excitability for other reasons than that a ject [figure 24].The 3D content map provides a vi- compelling video segment being shown, such as sualization of how the content bins contribute to meeting a friend, or recalling something that hap- create a storyscape specific to this exhibit. It con- pened earlier during the day.These "false positive" tains color coded vertical columns (a color for each data points would be modeled as "noise" intrinsic content bin) whose height is proportional to the in the GSR measurements.The additional GSR sen- amount of content that each bin contributes to for sor has been tested in a computer-based simulated the corresponding object [figures 25, 26]. environment In this case, the decision node for content selection also needs to take into account For content personalization the system should be the visitor's preferences, which compete with the able to infer an interest profile of the visitors, in curator's ordering preferences to assemble the best addition to their type as they wander along the matching audiovisual story for each object. exhibit gallery.An interest profile in the context of this research means a rating of preference for the story themes given in table 1.A GSR (Galvanic Skin 6. Potential Impact of the Museum Response) sensor can potentially give very useful Wearable on Exhibit Design information to the museum wearable. This sensor Potential changes and improvements that the mu- responds to skin conductivity and is often used in seum wearable can produce for the space design of the medical and psychological field as an aid to the MIT Museum's Robots and Beyond exhibit be- monitor an individual's level of excitement or stress. came obvious in the course of this project 61 54 ©Archives & Museum Informatics, 2002 BESTCOPYAVAILAB!r Museums and theWeb 2002

Figures 25, 26. Three dimensional representation of content distribution for MIT Museum's Robots and Beyond Exhibit.

At the current exhibit the posters and labels oc- I . There would be no more need to have so many cupy half of the available exhibit wall space,and while posters and text labels, as the corresponding in- they certainly provide useful information, they re- formation could be provided in a more appealing quire long stops for reading, take useful space away audio visual form, in a video documentary style from other interesting objects which could be dis- by the museum wearable. The space now made played in their stead, and are not nearly as compel- available by eliminating the large posters could ling and entertaining as a human narrator (a mu- be used to display more robots, which are the seum guide) or a video documentary about the dis- true protagonists of the exhibit. Typically most played artwork.The tracking data and our observa- exhibits have to discard many interesting objects tion of museum visitors also revealed that people as there is not enough physical space available in do not spend sufficient time to read all of what is the museum galleries for all objects. Therefore described in the posters to absorb the correspond- making more space available is a clear advantage ing information.A great deal of the space occupied provided to the exhibit designer and the curator. by the posters and text labels is therefore wasted, Figures 27, 28 show how the posters at the en- as most people don't take advantage of informa- trance of the MIT Museum's Robots and Beyond tion provided in a textual form. exhibit can be replaced by more objects to be seen and appreciated by the public.These images The video stations, located in each section of the are extracted from a three dimensional Alias exhibit, complete the narration about the artwork Wavefront's Maya 3 animation realized to visual- by showing the robots in motion and by featuring ize the wearables' potential impact on the exhibit's interviews with their creators.While the video sta- space [figure 29]. tions provide compelling narrative segments, they are not always located next to the object described, 2. Visitors would be better informed, as the infor- and therefore the visitor needs to spend some time mation currently provided by the posters is locating the described objects in the surrounding mostly neglected by the public. The same infor- space in order to associate the object to the corre- mation would instead become part of the overall sponding narrative segment.The video stations de- narration provided by the wearable, and it would tract attention from the actual objects on display, be better absorbed and appreciated by the pub- and are so much the center of attention in the ex- lic. hibit that the displayed objects seem to be more of a decoration around the video stations than the 3. The video kiosks would no longer be necessary actual exhibit. because the same material would be presented by the museum wearable. The robots would be The potential improvements to the exhibit layout again the center of attention for visitors, as the offered by the museum wearable are summarized wearables allows both the real world and the as follows: augmented audiovisual information to be seen at 62 ©Archives & Museum Informatics, 2002 55 BEST COPY AVAILABLE Sparacino,The Museum Wearable

Figures 27,28. Potential impact of the museum wearable on the current exhibit layout:the posters in the Roots section are replaced by new objects.

the same time as part of the wearer's real sur- museum wearable, it does not attempt to perform round view. This would again make more space either user modeling or a more sophisticated form available for additional objects to be displayed. of content selection and authoring. It does one-to- one associations between triggering objects and The fact that the museum wearable presents audio recording or play back of clips. Besides general train- visual material together with the corresponding ing, is used specifically in the museum context to object, rather than separately in space and time,and allow a visitor to record salient moments of the within the same field of view of the visitor, thanks explanation by a human guide to later replay them to the private-eye display, is also of great impor- in the context of an independent visit to a museum, tance. While no studies have been conducted yet without a guide.The museum wearable, in contrast, on the quality and effectiveness of the learning ex- focuses on estimating the visitor's type and interest perience offered by the museum wearable, there is profile to deliver a flexible user-tailored narrative reasonable hope that synchronous and local infor- experience from audio/video clips that have been mation provided while actually looking at the ob- prerecorded.These clips or animations would usu- ject described by the wearable can make a longer ally be part of the museum's digital media collec- and more effective impression on the visitor.With tion.As opposed to DYPERS, it does not have the this device curators may be able to present a larger ability to record new content to be played out at a variety of more connected material in an engaging later time. Its purpose is to create for the visitor a manner within the limited physical space available path-driven personalized and immersive cinematic for the exhibit. experience which takes into account the overall trajectory of the visitor in the museum,the amount 7. Related Work of time that visitor stays to look at and explore the objects on display, to select a personalized story Oliver (http://www.media.mit.edu/nuria/dypers/ for the visitor, out of several possible digital stories dypers.html; Schiele, 1999) developed a wearable that can be narrated. computer with a visual input as a visual memory aid for a variety of tasks, including medical, training, Hällerer and Feiner (1 997) have built a university or education.This system records small chunks of campus information system, worn as a wearable video of a curator describing a work of art, and computer.This device is endowed with a variety of associates them with triggering objects.When the sensors for head tracking and image registration. objects are seen again at a later moment, the video Both the size of the wearable, mounted on a large is played back. The museum wearable differs from and heavy backpack, as well as the size of the dis- the previous application in many ways. DYPERS is a play, are inappropriate use in a museum visit. personal annotation device, and as opposed to the 6 3 56 @Archives & Museum Informatics, 2002 Museums and theWeb 2002

Fi r., iii11.6a71.111.0.4';a

trf

Figure 29. 3D model: start frame of animation of a visitor at the exhibit.

Various groups are working to augment the mu- urns (Broadbent and Marti,I 997). As participants seum visit with mobile devices that are not wear- to HIPS, Oppermann and Specht (1999) describe able computers but handhelds, which do not have a an adaptive system which requires the user's inter- private-eye or head-mounted display but rely on vention to personalize the presentation.This how- the handheld's screen for visual communication.One ever can be quite disruptive of the visitor's museum of the main drawbacks of such devices is that the experience, in a way similar to those multi-path DVD visitor is obliged to toggle his/her attention between movies which stop at all turning key point and ask the objects on display and the handheld's screen, the viewer to choose an option before continuing. alternatively looking ahead towards the objects and A system such as the museum wearable, capable of then down to the screen. The private eye of the inferring the user's preferences with a mathemati- museum wearable instead allows the visitor to have cal model able to use the sensors' information as a true augmented reality experience by presenting cues and the curators' knowledge of their public as the viewer with a fused image which mixes together a guideline can be indeed less disruptive. the real world and the computer augmentation either as a picture-in-picture effect or as two super- The MUSEpad project (Kirk, 2001) is developing an imposed layers of information. original mobile device to enable visitors with disabilities to customize and optimize their learning Amongst the handheld based projects, Spaspjevic and leisure experiences in museums.Their feasibil- and Kindberg (2001) describe the electronic guide- ity study examines user personalization to allow book, currently under development at the San Fran- for adaptation to the needs of different users, cisco Exploratorium Science Museum. This device whether by providing video-only, audio-only or uses a combination of infrared and RFID location magnified images for visitors with poor vision. sensors to give visitors the ability to either view or bookmarkWeb pages which provide additional de- Aoki and Woodruff (2000) describe an electronic scription information on the objects of the exhibit. guidebook prototype that facilitates social interac- This portable device is used mainly to record the tion during the museum visit. Using a touch sensi- visitors' path through the exhibit,typically as a group, tive screen on the handheld and a photograph-based so that later the visit can be discussed and com- interface that the visitors can click on, the handheld mented upon in a classroom setting. offers a text or audio description of the selected object, according to the user's preferences.Visitors The European HIPS project proposes a user-cen- are then provided a mechanism to hear each other's tered approach to information delivery in muse- audio selection. Such research on mobile augmen- 6 4 ©Archives & Museum Informatics, 2002 57 Sparacino,The Museum Wearable

Figures 30, 3Early version of the museum wearable at the SIGGRAPH 99 Millennium Motel. tation devices which can be used collectively as a exhibit as that information can now be tailored to group is quite important for the future of this field. each individual visitor; they can personalize the audiovisual explanations provided to the public based The author showed an early prototype of the mu- on the visitor's type and exploration strategy. seum wearable called "Wearable City of News", as a demonstration for the SIGGRAPH 99 Millennium The museum wearable provides more than a simple Motel, where hundreds of users tested the system associative coupling between inputs and outputs. for seven full days (http://www.siggraph.org/s99/ The sensor inputs, coming from the long range in- conference/etech/projects.html). The system fea- doors infrared positioning system, are coupled to tured a jacket with an embedded computer and digital media outputs via a user model, and esti- head-mounted display to show visitors Web pages mated probabilistically by a Bayesian network.The and video previews of the Millennium Motel dem- ability to coordinate and present the visual mate- onstrations using a short range but low power in- rial as a function of the visitor's estimated type (i.e. frared location system [figures 30, 3 I]. busy, greedy, or selective types, or other appropriate types), seamlessly, appropriately, and in conjunc- 8. Discussion and Future Work tion with the path of the wearer inside the exhibit, is an important feature of this device. Bayesian net- The museum wearable fuses the audiovisual docu- works have the additional advantage that they al- mentary, which illustrates and extends an exhibit, low to us encapsulate our human knowledge about with the visitor's path inside that exhibit, using a the context of use of the museum wearable (a par- wearable computer. By having the public use this ticular exhibit, a trade show) in appropriate nodes device, curators and exhibit designers can accom- of the network. plish multiple goals simultaneously: they can have objects narrate their own story; they do not needs With respect to the traditional museum audio tour, special rooms to show audiovisual explanations the museum wearable introduces the following in- about the exhibit as with the wearable the narra- novations: it does not constrain the visitor to fol- tive is unfolded by the visitor's path in the museum; low any predefined sequential path in the museum, they can show more artwork than what is physi- or to continuously press buttons, but it relies in its cally on display, including video, images, audio, and sensing system to find the visitor's location and re- text about other important objects for the exhibit spond; consequently, it adds a layer of visual aug- (these usually do not see the light because of the mentation, not just auditory; through Bayesian net- physical limitations of the available space); they do work based user modeling it provides personalized not need to disseminate panels with textual expla- content to each visitor, as a function of the esti- nation or video monitors along the aisles of the mated visitor type. 65 58 ©Archives & Museum Informatics, 2002 Museums and theWeb 2002

An experimentation phase at the museum should students, it would be useful, for learning purposes, follow the current research.A procedure must be to make each visitor profile available to the users set to establish if and how the museum wearable at the end of the exhibit, and have the system re- does actually enhance learning and entertainment group the visitors according to matching profiles, at an exhibit, or how the content shown does actu- for further discussion. This same capability could ally match the visitor's preferences. also be made available to visitors in the museum's cafeteria at the end of their tour, to play The research here presented can also be expanded matchmaking among those who wish to be involved. in various ways. One direction of work is to find Alternatively the visitor's profile, path, and length ways to get to know the visitors better so as to of stay can be used to create a Web-based exhibit target content presentation more accurately to- catalogue whose URL can be sent to the visitor as wards their level of knowledge or competence. a personalized basis for further learning. Asking visitors to fill out lengthy questionnaires upon entering the exhibit may not be practical. It is in- Acknowledgments stead desirable to experiment with additional sensors, such as the GSR, a motion sensor (accelerom- A group of dynamic MIT undergraduates contrib- eter), or a tiny video camera to obtain a more ac- uted to the museum wearable at different stages of curate estimate of the visitors' interest profiles and development.The author would like to thankTracie levels of attention. More visitor tracking data could Lee, Aman Loomba, Manuel Martinez, Eric Hilton, be gathered at the museum site, to eventually infer Chin Yan Wong, Audrey Roy, Anjali D'Oza, and Sa- more visitor types than the ones described in this rah Mendelowitz for the many hours of work they document, and compare them with the more so- dedicated to this project. My deepest thanks also phisticated visitor typologies discussed in the mu- go to Janet Pickering, Beryl Rosenthal, and Janis seum literature. Sacco from the MIT Museum, to help make this project a reality. Neil Gerschenfeld,Alex Pentland, The museum wearable can also become a very use- Glorianna Davenport, Kent Larson,Walter Bender, ful tool to gather visitor tracking data in the mu- and Ron McNeil have also supported this project. seum. For example, rather than coming up with a set of visitor types from the museum literature,one References could adopt the opposite approach of "inferring" the visitor types from a statistical analysis of the Aoki, P.M. (2001) and Woodruff,A.The Conversa- tracking data (path and stop duration) gathered by tional Role of Electronic Guidebooks. In: visitors with the museum wearable. This informa- Proc. UBICOMP 2001, Springer-Verlag. tion would help the curator, exhibit designer, and the modeler of the interactive museum experience Broadbent, J. (1997), Marti, P. Location aware to refine their knowledge about visitor types for a mobile interactive guides: usability issues. In: specific exhibit. Similarly, by analyzing the posterior Proc. of International Cultural Heritage values of the object nodes, the curator and the ex- Informatics Meeting (ICHIM '97), Paris, hibit designer could see which objects are the most France. interesting or boring for the visitors, and change the exhibit layout accordingly. Dean, D. (1994). Museum Exhibition:Theory and Practice. London, Routledge. An important extension to the museum wearable would allow it to support visitors who want to come Healey, J. (1999), Seger, J., and Picard R. Quantify- to the museum as a group and have the freedom to ing Driver Stress: Developing a System for comment and discuss the artwork amongst them- Collecting and Processing Bio-Metric Signals selves.A simple modification to the current proto- in Natural Situations. In: Proceedings of the type would be to add a small microphone capable Rocky Mountain Bio-Engineering Symposium, of detecting when the visitor is talking, and then April 16-18 1999. automatically pause the narration. If for example the group of visitors is composed of high school 68 @Archives & Museum Informatics, 2002 59 Sparacino,The Museum Wearable

Hällerer,T. (1999), Feiner, S., Pavlik, J. Situated Schiele, B.(1999), Oliver, N.Jebara,T. and Pentland, Documentaries: Embedding Multimedia A. DyPERS: Dynamic Personal Enhanced Presentations in the Real World, In: Proc. Reality System. In Proceedings of Intl. Third Int. Symp. on Wearable Computers Conference on Vision Systems (ICVS 99). (1SWC '99), San Francisco, CA, October 18- Gran Canaria. Spain. 19, 1999, pp. 79-86. Serrell, B. (1996).The question of visitor styles. In: Hooper-Greenhill E (1999). Museums and their S. Bitgood (Ed.).Visitor Studies:Theory, visitors, London, Routledge. Research, and Practice,Vol. 7.1. Jacksonville AL:Visitor Studies Association, pp. 48-53. Houbart, G. ( I 994). Viewpoints on Demand:Tailoring the Presentation of Opinions in Video. MIT Smyth, P. (1997). Belief Networks, Hidden Markov Masters Thesis. Models, and Markov Random Fields:A Unifying View. Pattern Recognition Letters, Jordan, M.1. (1999), editor. Learning in Graphical vol. 18, number I 1-13, pp. 1261-1268. Models.The MIT Press. Sparacino F. (1999), Davenport G., Pentland A., Kirk J. (2001).Accessibility and New Technology in Media Actors: Characters in Search of an the Museum. In: D. Bearman and J.Trant (eds.) Author. In: Proc. of IEEE Multimedia Systems Museums and the Web 0 I Proceedings, '99, International Conference on Multimedia Seattle, USA. Computing and Systems (IEEE ICMCS'99), Centro Affari, Firenze, Italy 7-11 June 1999. Klein, L (1986). Exhibits: Planning and Design. Madison Square Press, NewYork, pp70-71. Sparacino, F. (2001). Sto(ry)chastics: a Bayesian network architecture for combined user model- Oppermann, R. (1999), Specht, M.A Nomadic ing sensor fusion, and computational storytelling Information System for Adaptive Exhibition for interactive spaces. MIT PhD Thesis. Guidance. In: D. Bearman and J.Trant (eds.): Proceedings of the International Conference Spaspjevic, M. (2001) and Kindberg,T.A Study of on Hypermedia and Interactivity in Museums an Augmented Museum Experience. Hewlett (1CHIM 99),Washington, September 23 25, Packard internal technical report. HPL-2001- pp. 103 109. 178.

Pearl, J. (1988). Probabilistic Reasoning in Intelligent Starner,T. (1995), and Pentland,A.Visual Recogni- Systems: Networks of Plausible Inference. tion of American Sign Language Using Hidden Morgan Kaufmann, San Mateo, CA. Markov Models. In: Proc. of International Workshop on Automatic Face and Gesture Recognition (1WAFGR 95). Zurich, Switzer- land.

6 7

60 @Archives & Museum Informatics, 2002 Foundations

6 3 Now That We've Found the 'Hidden Web,' What Can We Do With It? The Illinois Open Archives Initiative Metadata Harvesting Experience

Timothy W. Cole, Joanne Kaczmarek, Paul F. Marty, Christopher). Prom, Beth Sandore, and Sarah Shreeves, University of Illinois at Urbana-Champaign, USA

Abstract

The Open Archives Initiative (0AI) Protocol for Metadata Harvesting (PMH) is designed to facilitate discovery of the "hidden web" of scholarly information such as that contained in databases, finding aids, and XML documents. OAI- PMH supports standardized exchange of metadata describing items in disparate collections such as those held by museums and libraries. This paper describes recent work done by the University of Illinois Library, recipient of one of seven 0AI-related grants from the AndrewW. Mellon Foundation. An overview is given of the process used to export metadata records describing holdings of the Spurlock Museum at the University of Illinois. These metadata records were initially created to help track artifacts as they were procured, stored, and displayed and now are used also to support end-user searching via the Spurlock Museum Website. Spurlock metadata records were mapped to Dublin Core (DC) and then harvested into the Illinois project's metadata repository. The details of the processes used to transform the Spurlock records into OAI compliant metadata and the lessons learned during this process are illustrative of the work necessary to make museum collections available using OAI-PMH. Assuming institutions like Spurlock make metadata available, what then can be done with these information resources? We discuss the OAI-based search and discovery services being developed by the University of Illinois. Issues such as need for normalization of metadata, importance of presenting search results in context, and difficulties caused by institution-to-institution variations in metadata authoring practices are discussed.

Keywords: Open Archives Initiative (0AI), metadato harvesting Dublin Core, cultural heritage, interoperability, heterogeneous collections

Introduction Although the Open Archives Initiative (0AI) origi- the feasibility of using OAI-PMH to build services nally focused on the exchange of metadata describ- to reveal and make more accessible collections of ing e-prints in the scientific community (Van de cultural heritage material. Sompel & Lagoze, 2000), the OAI-Protocol for Metadata Harvesting (0Al-PMH) holds much prom- While a focus of the Illinois research is on building ise for similar exchanges of metadata describing and testing software tools designed to harvest the collections of museums and other cultural heri- metadata provided by OAI-compliant metadata pro- tage institutions (Perkins, 200 I). Materials in these viders, we also are exploring what can be done with types of collections often are not well indexed (or 0AI-harvested metadata. What is the fidelity and not indexed at all) by commercial Web search en- usefulness of the indexes and search services that gines. Metadata describing such holdings is hidden can be built atop a repository of metadata harvested in databases, finding aids, and XML documents, or using the OAI-PMH? The utility of such a metadata otherwise is not readily available to Web search repository will depend on many factors, including systems, which typically understand little more than the development of more uniform metadata simple HTML. As a result, these materials remain authoring practices across various communities of hard to find and out of reach for many researchers. metadata providers. Metadata is currently used pri- With an interest in making these materials more marily for a variety of local purposes, and the visible to scholars and other researchers, the Uni- schemas to which collections conform have been versity of Illinois at Urbana-Champaign, through an adapted and tailored to meet local needs. Less at- Andrew W. Mellon Foundation grant, is exploring tention has been paid to using metadata to support

!_) BESTCOPYAVAILABLE Museums andthe9eb2002: Proceedings Archives & Museum Informatics, 2002, p. 63 Cole et al, Now That We've Found the 'Hidden Web' universal interoperability. With the advent of OAI metadata providers . Fourteen institutions have more universally standard schemas for the support provided the Illinois OAI-PMH project team with of interoperability is gaining ground. snapshots of relevant metadata, in a few cases providing an entire database,which has then been made Even assuming broader and more consistent appli- available from Illinois servers in a manner compli- cation of metadata schemas designed to support ant with OAI-PMH. interoperability, there remains considerable work to be done in learning how best to utilize aggrega- While all of the metadata harvested from the regis- tions of metadata describing heterogeneous infor- tered OAI metadata provider sites was in the re- mation resources. There is a need to normalize quired Simple Dublin Core format (DC), metadata metadata so as to enable more consistent retrieval from the fourteen unregistered sites were given to of results from cross-collection searches. Challenges the project in a variety of formats. These included also exist for presenting search results in ways that finding aids formatted in encoded archival descrip- provide appropriate contextual information for the tion (EAD), MARC format metadata describing bib- records retrieved. We will discuss more detailed liographic information, and local metadata schemas examples of these and other issues throughout the stored in HTML files, XML files, or proprietary da- paper. We will outline our efforts to overcome these tabase structures. We subsequently mapped these issues in building a useful cross-collection reposi- records into DC, prior to making them available in tory of cultural heritage materials. We will also dis- accord with the OAI-PMH. cuss our work in progress, our plans for future development, and our reasons for believing that the Cross-Collection Repository Issues OAI protocol has great potential for increasing access to and exposure of hidden resources via the To enhance cross-collection searching, local Web. metadata schemas need to be mapped to standard schemas. Even with greater adoption of standard Illinois Project Harvest ExperienceTo metadata schemas like DC,there remain wide variations in the use of authoring conventions and the Date depth of descriptive information included when As of February, 2002, the Illinois OAI-PMH project creating metadata to describe cultural heritage col- has harvested metadata from twenty-five different lections. institutions or consortiums. The resources described range from museum artifacts such as pot- OAI-PMH Basics tery and clothing, to archival manuscript and personal paper collections,to digitized photographs and The Open Archives Initiative Protocol for Metadata monographs. The sets of harvested metadata range Harvesting (0Al-PMH) is a new and still evolving in size from the largest at 900,000 records to the protocol designed to allow institutions to share smallest at 23 records. Some sites include metadata metadata easily.It is hoped that the standard can not relevant to cultural heritage mixed with more help increase the discoverability of resources by appropriate metadata. Our OAI Harvester soft- scholarly researchers. The Protocol underwent one ware excluded such non-relevant metadata from minor revision during its first full year of experi- our indices. In total, we have inspected over two mental implementation. Version 2.0 is scheduled million individual records, resulting in an index of for release in May 2002. The fundamental pieces of approximately 770,800 records relevant to the sub- the protocol are its adherence to well-formed XML, ject domain of interest. The institutions providing its use of the HTTP standards for data transmis- metadata are diverse, including academic libraries, sion, and its requirement that metadata records museums, historical societies, public libraries, the shared via the OAI-PMH be made available in the Library of Congress, and special consortia such as DC metadata schema (optionally records may be the Colorado Digitization Project and the Alliance made available in additional metadata schemas as and Lincoln Trail Library systems in Illinois. Eleven well) (Lagoze and Van de Sompel, 200 I ). contributing institutions are registered OAI 70 64 ©Archives & Museum Informatics, 2002 Museums and the Web 2002

LIBRARY OF AMERICAN CIMI DEMO SPURLOCK CONGRESS MUSEUM OF REPOSITORY MUSEUM NATURAL HISTORY Sample 79,629 records 2,004 records 197,233 records 46,612 records Analyzed

% of Average % of Average % of Average % of Average records times records times used records times records times containing used per containing per containing used per containing used per element record element record element record element record Title 100% 1.00 100% 1.00 71% 1.07 100% 1.00 Type 100% 2.34 100% 1.00 100% 3.50 100% 3.00 Creator 97% 1.17 100% 1.00 43% 1.03 16% 1.00 Publisher 73% 1.00 100% 1.00 100% 1.37 100% 1.00 Description 23% 2.39 4% 1.00 94% 1.13 93% 3.44

Source Not Used - Not used - 25% 1.00 2% 1.00 Relation Not Used - 100% 1.00 96% 1.45 23% 1.03 Format Not Used - 100% 1.00 22% 2.54 70% 1.00 Date 72% 1.00 100% 1.00 64% 1.10 70% 1.00

Subject 66% 1.61 48% 1.00 95% 3.21 100% 1.00 Identifier 100% 1.00 100% 1.00 100% 1.43 100% 2.00 Language 56% 1.00 Not Used - 60% 1.00 Not Used - Coverage 40% 1.07 100% 1.00 60% 3.70 99% 3.23 Rights 25% 1.00 100% 1.00 81% 1.65 100% 1.00 ContributorNot Used - 100% 1.00 44% 2.19 50% 1.00

Table I - Dublin Core elements used and number of times repeated

Metadata Authoring Practices One of the common challenges in setting up an OAI- PMH metadata provider service is ensuring that the Harvesting done so far shows that metadata metadata to be exported is mapped properly into authoring practices differ both in the selection of DC. There are fifteen DC elements (version I. I): DC elements used and the depth of information Title, Creator, Subject, Description, Publisher, Con- provided even within specific communities. As pre- tributor, Date,Type, Format, Identifier, Source, Lan- viously noted, DC elements are both optional and guage, Relation, Coverage, and Rights. Definitions repeatable. These allow a significant degree of varia- and recommended usage of these elements can be tion in the interpretation of which elements to use found at the Dublin Core Metadata InitiativeWebsite and how to use them when metadata are created . Any or all of these in DC and when mapped to DC. We examined elements may be used, and all are repeatable. Be- records from the Library of Congress American cause much of the existing metadata about museum Memory Project, the American Museum of Natural and special collections was developed with limited History, the CIMI Demonstration Repository, and resources, using local schemas and with local needs the Spurlock Museum to compare the use of DC in mind, the process of mapping to DC can be time elements (See Table I ). Within each set of sample and resource consuming and potentially frustrat- records, we calculated the percent of records that ing. There is, however, a growing community of DC contain each of the DC elements and the average users and freely available tools to assist with this number of times an element is used in records that process. contain it. The usage variations are due in part to

BESTCOPYAVAILABLE

©Archives & Museum Informatics, 2002 65 Cole et al, Now ThatWe've Found the 'Hidden Web' the intrinsic differences in the collections described preserve richness of the local metadata schema,the by the metadata. However, some variations are Spurlock metadata had to be mapped first to Quali- clearly due to different decisions made by each in- fied Dublin Core (DCQ) and then to simple DC stitution when determining how to map their (Dublin Core Qualifiers, 2000). metadata into DC. These variations pose a challenge when attempting to build cross-collection The decisions made about mapping the metadata search services and determining what depth of de- were more time consuming than the time spent tail should be shown in record display. The varia- writing the scripts and manipulating the data. Here tions can also influence how a search engine's rank- is an example of some of the decisions that were ing algorithm orders the result set. made for the cultural materials provided by Spurlock: Spur lock Museum Metadata Mapping Subject Museums began to use personal computers for One instance of the DC Subject element was in- tracking information about their collections using a cluded in each record. The decision was made to variety of proprietary databases. The use of pro- concatenate three fields from the original metadata prietary software not specifically designed with the records that could be considered equivalent to DC museum community in mind can lead to inconsis- Subject. Colons were used to distinguish each of tencies in the way collections are catalogued and the three original strings that were concatenated tracked. The descriptive elements within a locally to create the single DC Subject element. If there created metadata format and the degree of com- were not three values to concatenate, then the DC pleteness when classifying and cataloging the items Subject filed would have two colons next to each can vary widely from one organization to another. other or a lone colon at the beginning or end of This may have the effect of diluting the potential the element. for discoverability as searching aggregated repositories requires some degree of predictable struc- Date ture and normalization of metadata terms or con- The Date element was qualified as DCQ cepts. At the same time, one should not overlook date.created. the value of descriptive, locally created metadata applied to artifacts by professionals familiar with Coverage the collection and the museum. This is particularly The Coverage element was qualified by appending valuable for collections used primarily by a local Spurlock metadata field names to the DC element community of users. At the University of Illinois as DCQ coverage.spatial, coverage.temporal, or Spur lock Museum, FileMakerPro software is used coverage.cultural. to track the procurement, storage, stage of processing, and display of artifacts.It is also used to Description maintain descriptive information about the materi- Key words (Materials, Manufactural Process, Munsell als. The FileMakerPro database interacts with aWeb Color ID) were added to the Description field val- server to provide public access to a portion of the ues in the original metadata to clarify what was collection . This meant by the content of the Description element. database provides direct online access to 45,000 of over 210,000 records describing cultural heritage Table 2 gives the complete list of mappings from and natural science artifacts. The Illinois OAI-PMH local Spurlock-specific metadata schema to DC team was provided with metadata for all 210,000 metadata schema used to export the metadata via records. the 0AI- PMH. In addition to the mapping given below, 3 fixed DC Type values ("culturar "physical As a first step to making these metadata records object," and "original") were included in each available via OAI, Spur lock's metadata were ex- Spurlock metadata record exported via 0AI. Fig- tracted from the FileMakerPro database using a ure 1 shows how a typical Spurlock metadata record simple extraction script.It was necessary to map looks when retrieved by an end-user using the the locally created and customized metadata for- Spurlock Museum website. Figure 2 shows this same mat to a schema more closely related to DC. To metadata record exported via 0AI. 79 66 © Archives & Museum Informatics, 2002 Museums and theWeb 2002

Another challenge is due to the very flexible na- SPURLOCK DATABASE DC ,

FIELD ELEMENT I ture of the EAD standard designed to encourage Object Name Title participation by a large number of institutions. It is Artist/Maker Creator not surprising that the EAD records analyzed for Classification Subject the Illinois OAI-PMH project reveal many differences Measurements Format in tag structures and encoding patterns between Culture Coverage institutions. Despite these differences, the records Geographic Coverage seem to have enough consistency to make a gen- Time Period Coverage eral cross mapping between EAD and OAI-DC Date Date possible. Although the EAD Application Guidelines Materials Description include two recommended mappings to DC, one Manufacturing Process Description for the finding aid itself (i.e.the electronic resource) Munsell Color ID Description and another for the resources described in the find- Visual Description Description ing aid (i.e. the actual manuscripts or archives), nei- Published Description Description ther mapping will generate records adequate for Scholarly Notes Description use in an OAI repository. While researchers are Reproduction interested in discovering the existence of actual (if it is a reproduction) Description Reproduction Remarks archives or manuscripts, metadata about the find- (if it is a reproduction) Source ing aid itself is also necessary in order to point ef- Bibliography Relation fectively to the source records. For this reason, a Comperanda Relation flexible approach in mapping an EAD finding aid into Credit Line/Donor Name an OAI record is needed. (if publicly assessable) Contributor Spur lock Museum, University of We are currently testing a schema that produces Illinois at Urbana-Champaign Rights many DC metadata records from a single EAD file. Spur lock Museum, University of We first produce a record describing the entire Illinois at Urbana-Champaign Publisher collection of materials in the finding aid. This top- Accession Number Identifier level record becomes a base record for the finding URL to Spur lock's Database Identifier aid and must be as complete, accurate, and concise Table 2 - Spur lock cultural material as possible. The base record includes a link to the metadata fields mapped to Dublin Core source EAD file as well as references to related parts of the collection. These related parts are described in other individual records we produce for Item-Level vs. Collection-Level Metadata each component level found within the EAD file's One of the more challenging aspects of implement- description of subordinate components. Although ing the OAI protocol is mapping from metadata it is not possible to link these records to each other schemas designed to describe collections of mate- adequately using simple DC, a sophisticated use of rials (e.g., an EAD Finding Aid record) to the DC qualified DC fields can produce linked records. We metadata schema. Finding aids may describe as many hope this method will provide functional and easily as several thousand items or folders in an archive searchable records. while DC has typically been used to describe individual items (e.g., books, photographs, letters, per- Enhancing Discoverability of sonal journals, audio files).Each EAD record includes metadata describing the entire collection and Resources a "description of subordinate components" which Metadata authoring practices as discussed above lists the separate series, sub-series, folders and items play important roles in determining the value of a found in the collection. Some EAD files reach hun- cross-collection repository focusing on cultural dreds of kilobytes, or even several megabytes, in heritage materials.Providing context for the size. The challenge is to allow the richness of such metadata, normalization techniques, and develop- a large file to be exposed and made searchable along- ing the search engine and interface also need to be side other records that describe a single item or a examined and explored for ways to enhance the much smaller collection. discoverability of the collected metadata. 73 ©Archives & Museum Informatics, 2002 67 BEST COPYAVAILABLE Cole et al, Now That We've Found the 'Hidden Web'

TNE WILLIAU R. AND CLARICE V. Ahlmolo Del

Pliplantol ow oal:spurlock:cultural:59738 Spurlock Museum IdarmibMi 2001-11-15 AT THE UNIVERSITY OP ILLINOIS Itmed., as.

Chicago Public Library and Identify whichif anyvocabularies were used others. All the metadata has been entered into a by data providers; database in a format that meets the specific needs Determine whether there is any controlled vo- of the TDC project and does not conform to DC. cabulary that the project team could successfully However, the display of the records is very clean apply to all of the data providers, or, if not, create and consistent and maps well into DC for the pur- such a vocabulary specific to our repository; poses of the Illinois OAI-PMH project. With a link Apply our vocabulary to the metadata to aug- directly to the Teaching with Digital Content re- ment the 'native' vocabulary; source provided from a URL that has been mapped Build mechanisms into the search interface that into the DC Identifier field, the context is easily would take advantage of the normalization; and maintained by providing more background informa- Gauge the success of the normalization for re- tion about the record when the user clicks on this source discovery. link. These goals translate into a five-step process as We plan to explore methods for providing context follows. internally within the repository as well, linking back to the owning institutions. Providing easy access I. Extract and analyze the element values to related records may reveal overlooked informa- (content) tion and insights. The collocation of works by the same author and under the same subject headings We organized the element values by metadata pro- is a given in databases using a standardized metadata vider. Each unique value was listed, along with the format and data entry. In an aggregated database, number of times it appeared within each individual this collocation task is not trivial, but is potentially metadata set. For example, we discovered that only even more illuminating. The context could be eleven of the twenty-five metadata providers used thought of in different ways, such as: the Type element and that approximately 1440 different Type values appear in the entire metadata a specific archaeological dig; set. Some providers only use one value, and others a specific time period or era (e.g. Civil War); use over 800. (See Table 3) The number of values a single donor; seems to be dependent on whether the institutions a particular geographic area (e.g.South America); are using a specific or general vocabulary. A large or number of records in the CIMI metadata use very a specific genre of art or literature (e.g. Art specific types such as "Physical ObjectTOYS." Nouveau). Metadata Normalization 2. Determine how each element is interpreted and what controlled vocabulary, if any, is In order to provide context internally and to en- used. hance discoverability of metadata records in a cross- collection repository, some normalization of the We discovered that metadata providers used ele- metadata is desirable. We believe that normaliza- ments in a variety of ways. For example, the DC tion can be done on several of the DC elements, Date element was used for the date the digital item including Type, Format, Coverage, and Date. We was created, the date the physical item was created hope to provide some degree of normalization on or published, and the date an item was added to a the Subject element but have not yet developed collection. Because OAI-PMH requires use of simple our strategy for this more complex normalization. DC, the qualifiers that may have explained these values further were not present. Similarly the vari- Effective normalization requires us to: ety and specificity or generality of controlled vocabularies also influenced the next steps in the nor- Understand how the element was interpreted malization process. by metadata providers and which elements in

75 @Archives & Museum Informatics, 2002 69 Cole et al, Now That We've Found the 'Hidden Web'

NUMBER VOCABULARY USED FOR TYPE OF ELEMENT AND SPECIFICITY OF METADATA PROVIDER VALUES VOCABULARY Alliance Library System 431 LC Subject Headings and ALA's Guidelines on Subject Access to Individual Works of Fiction, Drama, etc. (specific)

American Museum of Natural I DCT I(general) History

Celebration of Women I DCT I (general) Writers CIMI Demonstration 886 CIMI modifications to Dublin Core Type Repository Vocabulary (DCT I) and local vocabularies (general and specific)

Formations I Local vocabulary (general) Library of Congress American 87 LC Thesaurus of Graphic Materials ll Memory (specific)

Open Video Project, University I Local vocabulary (general) of North Carolina Chapel Hill

Perseus Digital Library I DCT I(general) Spurlock Museum 4 CIMI modifications to DCT I (general)

University of Michigan, Digital I Local vocabulary (general) Library Production Service University of Tennessee 25 Local vocabulary (hierarchical) Special Collections

Table 3 - Number of Values and Vocabulary Used for Type Element

3. Determine focus and vocabulary for After new records are harvested, the system com- normalization pares the values of specific elements with the mapping already done and adds appropriate terms from Each DC element was examined to determine our our vocabulary where applicable. If a term that has focus for normalization for that particular element. not already been mapped appears, it is flagged and While the Date element suggests a need to choose looked at by human eyes. between options like 'date created' or 'date contributed' and the particular format the value is dis- 5. Providing services based on the played in, the Type element required us to focus on normalization process the range and specificity of vocabulary used. For the Type element we determined resource discov- The last step is to provide services so that users ery may be enhanced by adding slightly more gen- can take advantage of the normalization process. eral terms into the record alongside the 'native' The ability to limit searches or sort results by type vocabulary (e.g.: 'physical object' added to 'toy'). of resource or to search for specific date ranges or Following the guidelines from the DC Type Work- geographic areas is each made possible by the nor- ing Group, we developed our own vocabulary, pull- malization process. ing terms from existing vocabularies where applicable,that provided the level of specificity that would End-User Search Interface Design benefit our users the most (Apps, 200 I). Strategies We began our interface design following guidelines 4. Normalizing the data for scenario-based design (Carroll, 2000). We iden- Once the vocabulary is developed, the values in the tified likely users of the system as two distinctly metadata are mapped to the vocabulary terms. The separate groups - scholarly researchers and K- I 2 actual normalization of the data is an automated teachers and students. We have prioritized the process. Following the mapping, values from the scholarly researcher as our primary user group for vocabulary are added to each record as additional. the preliminary interface design phase of the project.

URIC Cultural Heritage Repository pert searches." These expert searches are intended to provide examples of search strategies used by vY I *** .obetul experts from specific user communities. We will Shnple Search In All Collections also include in our development user search features that focus on the K- I 2 teacher and student Look for: I In user profiles as prioritized by our original scenario- use for exact phrase. based design decisions. This user group has particular needs that center on curriculum development and the use of on-line resources for achieving Figure 3 - Simple search screen comes. Under consideration is an enhancement option that would gather user input in the form of As time allows, we also intend to focus on K- I 2 annotations or other types of notes one might be teachers and students. In both cases, the need for inclined to make about particular sets of retrieved a simple design seems important. This is in line records and add these to the index in a way that with generally accepted "best practices" in inter- provides links and displays relationships between face design and is a logical choice given the poten- records that are related according to the research- tial for complex and/or inconsistent record displays ers using the system. due to the heterogeneity of the materials and institutions represented in the repository. We identi- Conclusion fied several types of common tasks our users might want to do with a repository of cultural heritage The Illinois OAI Metadata Harvesting Project rep- materials and sketched out scenarios that supported resents an attempt to enable the integrated search- these tasks. Based on these scenarios and working ing of diverse types of cultural heritage informa- from the interface built into our search engine (de- tion. In so doing, scholars may discover new links veloped by the University of Michigan's Digital Li- across materials that are physically dispersed and brary Extension Service), we designed the first it- that may potentially address common themes across eration of the end-user search interface following disciplines that have not been previously formally general usability heuristics as outlined by Nielsen recognized. Providing cross-collection repositories (2000). Figure 3 shows a simple, preliminary search that preserve the context of the items represented interface developed as much for diagnostic purposes by disparate metadata and offer easily navigable as for the end-user. search interfaces of this metadata requires several essential components: Work in Progress / Future Plans Metadata authoring practices must be compli- As of February 2002, we have begun usability test- ant with community accepted standard schema(s); ing. The feedback provided has already highlighted Techniques for displaying item-level vs. collection- obvious flaws both in the actual interface as well as level records need to be developed; back-end indexing methods. We will continue us- Normalization processes should be applied to ability testing with new iterations of the interface. the metadata prior to indexing; and Subjects will include students enrolled at the Uni- 'Best practices' for interface design should be versity of Illinois Graduate School of Library and adhered to for end-user search tools. Information Science. We will also participate in a The value of a usable search system that represents survey of potential users to be distributed jointly texts, manuscripts, images, digital objects, and arti- by University of Illinois and the University of Michi- facts simultaneously is difficult to quantify at this gan. This survey is intended for various scholarly stage of our investigation. However,we believe that researchers on both campuses and will provide feed- the exploration of techniques for providing simul- back to be used in the future design and enhance- taneous access to such a range of materials is es- ment of the system. Current plans for enhancing sential if we are to assist today's scholars in achiev- search functionality include adding options to coming their full potential as researchers. bine and refine searches and access to stored "ex- BEST COPY AVAILABLE 77 © Archives & Museum Informatics, 2002 7 1 Cole et al, NowThat We've Found the 'Hidden Web'

Acknowledgement Lagoze, C. &Van de Sompel, H. (2001). The open archives initiative: building a low-barrier The authors would like to acknowledge the gener- interoperability framework Proceedings of the ous support of the Mellon Foundation and the will- first ACM/1EEE-CS joint conference on Digital ingness of numerous institutions to provide libraries 2001. Retrieved February 8, 2002 metadata allowing us to explore the potential of from ACM Digital Library. http://doi.acm.org/ OAI-PMH for cultural heritage materials. 10.1145/379437.379449.

References Nielsen, J. (2000). Designing web usability: the practice of simplicity. Indianapolis, IN: New Apps,Ann, editor (2001). Guidance for Domains Riders Publishing. and Organizations DevelopingVocabularies for Use with Dublin Core: Outline. Working Open Archives Initiative Protocol for Metadata Draft. 28 Nov 2001. Accessed February 4, Harvesting. Version 1.1. 2001-07-02. http:// 2002, from http://epub.mimas.ac.uk/DC/ www.openarchives.org/OAII typeguide.html. openarchivesprotocol.htm.

Carroll, J.M. (2000). Five reasons for scenario- Open Archives Initiative Registered Data Provid- based design. Interacting with Computers 13: ers. http://www.openarchives.org/Registere 43-60. BrowseSites.pl

Dublin Core Metadata Initiative. http:// Spurlock Museum at the University of Illinois. www.dublincore.org/. http://www.spurlock.uiuc.edu/.

Dublin Core Element Set,Version 1.1: Reference University of Illinois at Urbana-Champaign OA1 Description. 1999-07-02. http:// Metadata Harvesting Project. http:// www.dublincore.org/documents/dces/. oai.grainger.uiuc.edu/.

Dublin Core Qualifiers. 2000-07-11. http:// Van de Sompel, H & Lagoze, C. (2000). The Santa www.dublincore.org/documents/dcmes- Fe Convention of the Open Archives qualifiers/. Initiative. D-Lib Magazine 6(2). http:// www.dlib.org/dlib/february00/vandesompel- Encoded Archival Document Application Guide- oai/O2vandesompel-oai.html lines forVersion 1.0. 1999. http:// Icweb.loc.gov/ead/ag/aghome.html.

Jane Hunter, DSTC Pty Ltd, University of Queensland,Australia

Abstract

This paper describes a proposal for an interoperable metadata model, based on international standards, which has been designed to enable the description, exchange and sharing of multimedia resources both within and between cultural institutions. Domain-specific ontologies have been developed by two different ISOWorking Groups to standardize the semantics associated with the description of museum objects (CIDOC Conceptual Reference Model) and the description of multimedia content (MPEG-7) - but no single ontology or metadata model exists for describing museum multimedia content.This paper describes an approach which combines the domain-specific aspects of MPEG-7 and CIDOC-CRM models into a single ontology for describing and managing multimedia in museums.The result is an extensible model which could lead to a common search interface and the open exchange, sharing and integration of heterogeneous multimedia resources distributed across cultural institutions. Keywords: Multimedia, Metadata, Interoperability, MPEG-7, CIDOC-CRM

I. Introduction

Multimedia provides museums with a rich paradigm tics associated with resources. It provides the un- for capturing, communicating and preserving cul- derlying foundation upon which digital asset man- tural information. It offers new capabilities for struc- agement systems rely to provide fast, precise ac- turing, interpreting and communicating knowledge, cess to relevant resources across networks and and the significance of artifacts within museum col- between organisations.The metadata associated with lections through the use of digital video, audio, im- multimedia objects is infinitely more complex than ages, graphics and animation. Making collections simple metadata for resource discovery of simple available in digital form, both in-house and through atomic textual documents and the problems and networks, provides museums with a tremendous costs asssociated with generating such metadata are opportunity to meet their educational mandate. correspondingly magnified. When linked together over networks, museum multimedia databases become even more valuable as Metadata standards enable interoperability between cross-cultural resources for educational and re- systems and organizations so that information can search purposes. be exchanged and shared. Standardized metadata models have been developed to describe museum In addition, the potential to re-use multimedia con- objects (CIDOC Conceptual Reference Model) tent to create new intellectual property, has fur- (CIDOC Conceptual) and to describe multimedia ther accelerated the growth in the size and num- content (M PEG-7) (ICS) but no standards currently ber of institutional multimedia databases. Existing exist for specifically describing museum multimedia multimedia objects are being combined and reused content. Hence the key goal of this project is to to generate complex, interactive multimedia, analyze and evaluate each of these existing stan- hypermedia, virtual reality displays and participa- dards and to determine a way to merge the two tory exhibitions.This has led to a demand for sys- ontologies to generate a standardized model for tems and tools which can satisfy the more sophis- describing museum multimedia content. Such a ticated requirements for storing, managing, search- model, which is capable of supporting the exchange ing, accessing, retrieving, sharing and tracking com- of information between existing collection manage- plex multimedia resources. ment systems (for physical artefacts) and emerging digital asset management systems would enable Metadata is the value-added information which knowledge and resources to be shared, re-used and documents the administrative, descriptive, preser- exchanged to a much greater extent than is cur- vation, technical and usage history and characteris- rently possible both within and between museums. 79 Museums and the Web 2002:Proceedings BEST COPY AVAILABLE Archives & Museum Informatics, 2002, p. 73 Hunter, Combining the CIDOC CRM and MPEG-7

Hence in the remainder of this paper we describe or historical significance lies with the medium or both the CIDOC/CRM and MPEG-7 metadata recording technology e.g.,the earliest photographs, models.We then analyze them both to determine film, video and audio recordings. In some cases, the the overlaps, intersections and differences. Based multimedia object is valuable because of the per- on this analysis we hope to be able to determine son who captured or recorded it (e.g., home movie how the models can best be merged to combine collection of John E Kennedy) or because it is part the two domain-specific vocabularies of MPEG-7 of a larger collection or bequest or it is exemplary and CIDOC/CRM, without introducing semantic of a particular genre, era or technique. In many cases, inconsistencies or redundancies.The final outcome both the semantic content and the context, agents, is in essence, a single, machine-understandable, ex- technique and medium are all of importance and tensible ontology designed to support the descrip- need to be recorded. tion and management of multimedia resources within museums. Often multimedia content has been generated for preservation and dissemination purposes.The mul- 2.The Nature of Multimedia in timedia resource may be a digital surrogate of the Museums original culturally significant museum artifact which is too valuable or fragile to be handled or is inac- 2.1 Types of Multimedia in Museums cessible for reasons of location. In many situations, the multimedia resources are created as an alter- Audiovisual or "multimedia" content within muse- native visual representation (image, model) of a ums is highly diverse and varies widely in origin, physical museum artifact or as a replacement for genre, purpose, media type, format, quality, age, con- earlier analog and digital formats which are becom- text and the reason for its cultural significance or ing obsolescent. retention within a museum or collection. Multime- dia in museums can include everything from disin- Hence in addition to the typical bibliographic infor- tegrating maps on paper to full feature films on DVD. mation,the metadata for multimedia resources may Within the scope of this paper we are referring to: need to describe detailed formatting information, images, audio, video, multimedia, graphics and ani- structural or segmentation information (temporal, mation - in both analog and digital form. TableI spatial and spatio-temporal segments), semantic in- below provides an overview of the typical consti- formation (description of the objects/people/places/ tution of multimedia collections within a museum. events which are recorded) and the event history and rights information.A detailed description of the In most cases, it is the semantic content, which is metadata requirements for multimedia in museums depicted or recorded on the multimedia resource is provided in the next section. which is of value i.e., the resource records an event, place, person, object or concept which is of cul- 2.2 The Metadata Requirements for tural, historical, geographic or educational impor- Multimedia in Museums tance. For some multimedia resources, the cultural The metadata associated with multimedia resources photographs, prints, maps, Images manuscripts, documents, drawings, can be classified into a number of different catego- paintings, movie stills, posters ries: songs, music, plays, interviews, oral Audio histories, radio programs, speeches, Bibliographic metadata - this includes infor- lectures, performances, language mation about the resource's creation/production recordings (date, place) and the individuals or organizations full feature films, documentaries, news Video/Film clips, anthropological/expedition involved (e.g., producer, director, and cast) and footage, home movies, animation the resource's classification information (e.g.,title, Graphics 3D models, simulated walk-throughs of abstract, subject, and genre). buildings, archeological sites, VRML Multimediapresentations, slide shows, SMIL files, Formatting metadata - this includes informa- Quick Time VR tion about the format, encoding, storage and system requirements associated with the resource. Table Overview of Museum Multimedia Types 8 0

image audio video text multimedia format format (Quicktime, format (text/ms format (image/tiff) format (text/html) (audio/aiff) MPEG I) word) filesize (bytes) filesize (bytes) filesize (bytes) filesize (bytes) filesize (bytes) version (v 4.0) version (v2.5) version (vI.1) version = 97 version (v3.0) resolution samplingrate compression software (MS Front Page dimensions (640x480) (600dpi) (44. IkHz) (zip) 2.0) dimensions samplesize (16 aspectratio (4:3) characterset storagetype (HD server) (1024x768) bit) (Unicode) aspectratio (4:3) duration duration (32min I 2sec) template template (Program.xs1) (04:45:56.34) (summary.dtd) colourdepth (8-bitcompression compression (mp2) bandwidth requirements grayscale, 24-bit (MPEG2/Layer 3) colour) colourpalette encapsulation encoding (mp2) system requirements (OS, (CMYK, RGB, (Real Audio G2) software, GrayScale) hardware,peripherals) framerate (25fps)tracks ( mono, sound (Yes/No) stereo) colourLUT storagetype storagetype (DVD) (base64) (Phillips DAT) orientation colour (Colour or B/W) (Portrait, Landscape) compression special Effects (CCIT4) (ChromaKey ) storagetype (CD- delivery and presentation ROM, jazz, hard requirements drive) (bandwidth, operating system, hardware, ) scanner (make, software, peripherals model, serial #) camera details and settings (make, model, serial#, aperture, focallength, filter) Table 2:Typical Format Metadata for Different Media Types

Table 2 shows the formatting metadata typically Events and rights metadata - this is infor- recorded for different media types. mation describing the life history of the resource. It includes everything from acquisition and relo- Structural metadata - this provides informa- cation events to the reformatting, editing, repack- tion about the structural decomposition of the aging and distribution events to the metadata at- multimedia resource into spatial, temporal or tribution events to the usage, copyright agree- spatio-temporal segments (scenes, shots, frames, ments, and permission events. image regions) and the relationships between these segments. A number of projects have developed or are developing metadata models for multimedia in museums Content metadata - this provides indexes to (Gabriel, 2001 ;The TOKEN 2000) or for historical the actual content which is recorded or depicted.- audiovisual collections (The ECH0).These projects within the multimedia resource. Content are either developing their own application-specific metadata can vary from natural language descrip- data models and vocabularies, or choosing one of tions of the people, objects, places or events which the existing standards (MPEG-7 or CIDOC/CRM). are depicted to the low level audio or visual fea- None have considered the approach of merging tures such as colour histograms or volume. ontologies from the museum domain and the multimedia domain into a single ontology. 81 © Archives & Museum Informatics, 2002 75 Hunter, Combining the CIDOC CRM and MPEG-7

Cover TitleNO: 53508 Title: PAILTRIN SPENCER CCEIECTION: UNRESPRICTED FCCTAGE Years: 1901 to 1912 Countryof Origins Australia Medium: Film Class: Documentary Sub-Class: Aust.Aboriginal & TOrres Strait Islander Ethnographic Cinamatographer/Director ofphotography: BaldwinSpencer Copyright contact: MUseum Victoria Production Company: National Film and Sound Archive, Australia Smeary: 41 minute compilation of threesequencesof unrestricted footage originally filmed by Baldain Spencer between 1901 and 1912. Contents: Seguence/. Visiting or Avenging dance of Aranda men, Alice Springs, showing a large group of men In close formation running up and down, carrying spears. (00:00:00 - 00:12:45) Sequence 2. Decorated Aranda women and girls dancing In a line with men seated In the foreground. (00:12:46 - 00:27:20) Sequence 3. Burialoeremniesand dances of Bathurst and Melville Islands showing PWomeni burial poles and scenes men and boys participating the Pukumani burial ceremony.(00:27:21 - 00:41:05) General notes: All requests for access other than on-site audition for research purposes, to be directed to the Collections Menager, Ethnohistoric Materials, Indigenous Collections Dept, Museum Victoria. SUbjects: Aboriginal peoples (Australian), Aboriginal rites and ceremonies, 1900s, Baldwin Spencer Item 1:Digital Cepy, 53508.mpg, MPBG-1, Silent, Bladk & White, 00:41:05, Copied from film in 1999 by MESA Example

2.3 A Typical Example

Consider an example typical of multimedia content trates the class hierarchy for theCIDOCCRM, as held by museums or archives: a film owned by the generated by the SIS knowledge base (ICS). The Museum ofVictoria which contains unedited foot- detailed specification of theCIDOCCRM Version age ofAustralian Aboriginal tribal ceremonies filmed 3.2 which includes detailed descriptions of the class by anthropologist Baldwin Spencer between 1901 and property definitions, hierarchies and relation- and 1912. Below is the catalogue item from ships, is available from (Crofts, 2002). Screen Sound Australia's online catalogue. In 1999, the original film was copied to digital format (MPEG- In order to understand and evaluate theCIDOC I) by the National Film and Sound Archive for the CRM's ability to describe multimedia resources,we purpose of preservation. apply it to the example in Section 2.3.

In Sections 3.2 and 4.2 below we compare the abili- 3.2 A CIDOC CRM Description of the ties of the CIDOC CRM and MPEG-7 to describe Example museum multimedia, by describing this example using their domain-specific vocabularies. Together with an analysis of the class and property hierarchies provided by theCIDOCCRM, this ex- ercise reveals that theC1DOCCRM's strengths lie 3.The CIDOC/CRM in its ability to describe: 3.1 Overview of the CIDOC CRM The"CIDOCobject-oriented Conceptual Refer- Identification information; ence Model" (CRM), was developed by the ICOM/ Acquisition and ownership information; CIDOCDocumentation Standards Group to provide an 'ontology' for cultural heritage information. Its primary role is to serve as a basis for mediation of cultural heritage information and thereby provide the semantic 'glue' needed to enable wide area information exchange and the integration of heter- ogenous resources between cultural institutions.

The CIDOC CRM is presented as an object-oriented extensible data model, expressed in RDF

Schema (RDF Schema Spec 2000). FigureI illus-

8 2 Figure I -The CIDOCICRM Class Hierarchy

. formation; E22: Man-Made Object Physical attributes and features - dimensions, Object ID 53508 E42: Object Identifier marks, visual items, material, sections, physical lo- < has_title>BALDWIN SPENCER COLLECTION : UNRESTRICTED FOOTAGE cation etc.; E35: Titieu/In_class> Historical events - CIDOC CRM supports rich Documentary Film E55: Type semantic descriptions of concepts or events - Baldwin Spencer both real world events,as well as the events which E21: Person Central Australia, E53: Place occur in the life cycle of a resource, and those cfalls_withIn> Australia E53: Place events which are depicted in the visual information objects. 1901-1912 E52: Time Spanu/In_class> However the CIDOC CRM is limited in its ability Museum of Victoria uhas_type> Museum to describe digital objects and particularly digital multimedia or audiovisual content. The following --cwat-prodticed=by>4Nationatftitnand:Sound,Archive._ - E21: Production Company requirements are inadequately supported: ffim E55: Type Formatting attributes (encoding, storage, system Aboriginal rites and ceremonies E55: Type requirements) for digital images, audio, video, text and multimedia such as those shown in Table I 41 minute compilation of three sequences of unrestricted footage originally filmed by Baldwin Spencer between 1901 and 1912. are not currently supported; Description E55: Type < /has_note> It is possible to define sections of physical ob- uhas_note>All requests for access other than on-site audition for research purposes, to be directed to the Collections Manager, Ethnoifistoric Materials,Indigenous Collections Dept, Museum Victoria. jects using spatial measurements or coordinates and the temporal location of Events or Periods uname>duration uvalue>00:41:05 using the Date and TimeSpan entities - but not minutesqunit> temporal, spatial or spatio-temporal locations Segment 1 within non-physical digital media; uhas_typesvideoSegment 00:00:00 00:12:45 Physical features can be described but not visual udepicts_concept>VIsiting or Avenging dance of Aranda men, Alice Springs, or audio features such as colour histograms, re- showing a large group of men in close formation running up and down, carrying spears gions, shape, texture, volume etc.; Segment 2 Hierarchical or sequential summaries of audiovi- uhas_type>videoSegment sual content which specify keyframes, scene 00:12:46 00:27:20 changes or key videoclips, are not supported Decorated Aranda women and girls dancing in a line with men seated in the foreground. qdepicts_concept> The CIDOC CRM provides the is_documented_in < /is_composed_of> property to record the relationship of a culturally Segment 3 vIdeoSegment significant physical or real-world artefact or event 00:27:21 00:41:05 to its visual/audio/audiovisual recording, which is uclepicts_concept>BurIal ceremonies and dances of Bathurst and Melville classed as a Document. However there is no explicit showing Pukumani burial poles and scenes of men and boys participating Islands In the Pukumanl burial ceremony. support for the different media types in CIDOC/ CRM. Because the CIDOC CRM is designed to pro- 53508.mpg E31: Documente/in_dass> vide an extensible underlying framework, it may be National Film and Sound Archive Canberra possible to improve support for multimedia descrip- E53: Place dalls_within> Australia tions, through the addition of MPEG-7 multimedia- E53: Place specific sub-classes and sub-properties to existing 1999 CIDOC/CRM superclasses and super-properties. E52: Time Span We investigate this approach in Section 5. Example 2 83 BEST COPY AVAILABLE ©Archives & Museum Informatics, 2002 77 Hunter, Combining the CIDOC CRM and MPEG-7

4. M PEG-7 - the Multimedia Content datatyping constraints required by MPEG-7. How- Description Interface ever semantic interoperability is necessary to enable systems to exchange data (e.g., metadata de- 4.1 An Overview of MPEG-7 scriptions), to understand the precise meaning of that data and to translate or integrate data across The Moving Pictures Expert Group (MPEG),a work- systems or from different metadata vocabularies. ing group of ISO/IEC, is expected to shortly re- Hence it was recognized that there was a need to lease the final standard for MPEG-7 (MPEG-7 Over- formally define the semantics of MPEG-7 terms; and view), the "Multimedia Content Description Inter- to express these definitions in a machine under- face",a standard for describing multimedia content. standable, interoperable language. RDF Schema The goal of this standard is to provide a rich set of (RDF Schema Spec 2000) was the obvious choice standardized tools to enable both humans and ma- due to its ability to express semantics and semantic chines to generate and understand audiovisual de- relationships through class and property hierarchies scriptions which can be used to enable fast efficient and its endorsement by the W3C's Semantic Web retrieval from digital archives (pull applications) as Activity (W3C). Consequently the Adhoc Group well as filtering of streamed audiovisual broadcasts for MPEG-7 Semantic Interoperability was estab- on the Internet (push applications). MPEG-7 can lished and an MPEG-7 ontology was developed and describe audiovisual information regardless of stor- expressed in RDF Schema and DAML+OIL exten- age, coding, display, transmission, medium, or tech- sions (Hunter Adding, Hunter An ROF 2001). The nology. It addresses a wide variety of media types extensions provided by DAML+OIL (DAML+01L) including still pictures, graphics, 3D models, audio, were necessary to satisfy certain requirements such speech, video, and combinations of these (e.g., mul- as the need for multiple ranges and sub-class specific timedia presentations). constraints. The basic class hierarchy of MPEG-7 content and segments is shown in Figure 2 e.g.,the MPEG-7 provides: MPEG-7 class VideoSegment is a subclass of both Video a core set of Descriptors (Ds) that can be used and Segment. to describe the various features of multimedia content; The relationships of the Segment types to other segment types and multimedia entities are dependent pre-defined structures of Descriptors and their on the allowed types of decomposition. Multimedia relationships, called Description Schemes (DSO. resources can be segmented or decomposed into sub- segments through 4 types of decomposition: Initially MPEG-7 definitions (description schemes and descriptors) were expressed solely in XML Spatial Decomposition - e.g., spatial regions Schema [9-1 I]. XML Schema proved ideal for ex- within an image; pressing the syntax, structural, cardinality and Temporal Decomposition - e.g., temporal Image video segments within a video; Text Spatiotemporal Decomposition - e.g., mov-

Image 14 Still Region StiIIRegion3D ing regions within a video; or by MediaSource Decomposition - e.g., the dif- Video Video Segment ferent tracks within an audio file or the different media objects within a SMIL presentation Multimedia H Audio Audio Segment Content Segment Associated with each of the subclasses in Figure 2 AudioVisual 14-1Audimifiuol Segment are various properties which define permitted re- HMultimedia AudioVisual Moving Multimedia lationships between the segment classes corre- Segment Region Region sponding to specific structural or organizational

Video description schemes and the permitted audio, vi- Text sual and audiovisual attributes associated with dif- Figure 2 - MPEG-7 Multimedia Segment Class ferent types of multimedia segments. Hierarchy 84

Type Feature Descriptors Video Still Moving Audio Feature VisualColor Dominant Color SegmentRegionRegionSegment Scalable Color Time X - X X Color Layout Shape - X X - ColorStructure Color X X X - GoFGoPColor (extension of Texture - X - - ColorStructure) Motion X - X - Texture HomogeneousTexture Audio X - - X TextureBrowsing Table 4 - Relationships between Segment EdgeHistogram Types and Audio and Visual Descriptors Shape RegionShape ContourShape In addition to the basic Multimedia and Segment Shape3D entities and the visual and audio descriptors, MPEG- 7 provides standardized Description Schemes which Motion Camera Motion combine the base classes and properties above, into MotionTrajectory pre-defined structured relationships (1SO...-5, 2001. Parametric Motion Figure 3 provides an overview of the organization MotionActivity of MPEG-7 Multimedia DSs into the following ar- AudioSilence Silence eas: Basic Elements, Content Description, Content Timbre lnstrumentTimbre Management, Content Organization, Navigation and HarmoniclnstrumentTimbre Access, and User Interaction. PercussivelnstrumentTimbre These different MPEG-7 DSs enable descriptions Speech Phoneme of multimedia content which cover: Articulation Language Information describing the creation and produc- Musical Melodic Contour tion processes of the content (director,title, short Structure feature movie); Rhythm Information related to the usage of the content Reverberation, Pitch, Contour, SoundEffects Noise (copyright pointers, usage history, broadcast schedule); Table 3 - MPEG-7 Visual and Audio Features Media information of the storage features of the and their Corresponding Descriptors content (storage format, encoding); Structural information on spatial, temporal or The visual and audio features which may be associ- spatio-temporal components of the content ated with multimedia and segment classes are listed (scene cuts, segmentation in regions, region mo- in Table 3. Associated with each of the visual and tion tracking); audio features is a choice of descriptors, also illustrated in Table 3. Precise details of the structure Co llsetisan .) Models Ma and semantics of these descriptors are provided in Ontatrargarinvion Ibanana, ISO/IEC 15938-3 FCD Multimedia Content De- NasIgarland Um Creation& &tam ' scription Interface - Part 3 Visualand ISO/IEC Psdastion %arsons 15938-3 FCD Multimedia Content Description In- Media sap 11= ))Canenfermapnws User terface - Part 4 Audio (ISO, 2001). Views j tirts.= \Manna efescrOlim 1 s =Os= Sono= C=3 Only particular visual and audio descriptors are applicable to each segment type.Table 4 illustrates Buie .107Mnb Scher= Bore Lis= .4 =die ) Bo= the association of visual and audio descriptors to Toe= =tones localisation nob . different segment types.The MPEG-7 RDF Schema (Hunter,Adding; Hunter,An ROF) specifies the con- Figure 3 - Overview of MPEG-7 Multimedia straints on these property-to-entity relationships. DSs (ISOIIEC) 85 ©Archives & Museum Informatics, 2002 79 BEST COPY AVAILABLE Hunter, Combining the CIDOC CRM and MPEG-7

Information about low level features in the con- MVegl cal4em-ormagegurge07,.d.m2cor mlosocsi,http://4ww.w3.cam/2001/XMISc4ers-1nstance. tent (colors, textures, sound timbres, melody de- calm olgog7="urnowegoseg7 dichecas2001" molosomlw.httpt/hom.w3.org/M4L/1998/naremmte. scription); xsiocheralocationw'orrorcegogeg7tschenat2001 .4me97-2001.xser> Costription ssiltypew*ContectEntitylypa"> Conceptual, semantic information of the reality <1411t41e0iaCcoten4 zsi.typew"Videdlyre",