<<

FORUM A Conceptual Framework for Mixed Environments: Designing Novel Learning Activities for Young Children

Abstract or acting in physical , when interacting with digital , provides greater embodiment for the user, How do we conceptualize and design mixed reality envi- where embodiment refers to immanent presence, compared ronments (MREs)? Here we describe a first pass at a con- with interacting with more abstract representations such as Downloaded from http://direct.mit.edu/pvar/article-pdf/11/6/677/1623826/105474602321050776.pdf by guest on 28 September 2021 ceptual framework and use it to inform the design of differ- interface metaphors that conventional computer-user in- ent kinds of activities for children to experiment with. Our teractions provide (Dourish, 2001). In other words, the aim was to investigate how different MRE setups affected kinds of interactions experienced in mixed reality environ- children’s exploratory behavior and their understanding of ments fit more naturally with the way we act and interact them. The familiar activity of color mixing was used: differ- with the everyday world. They do so by capitalizing on our ent setups were provided, where paint or light colors could familiarity with the everyday physical world, especially our be mixed, by using either physical tools, digital tools, or a innate and well-learned repertoire of physical actions (such combination of these. The findings of our study showed as grasping, pushing, and lifting). that novel mixes of physical and digital “transforms” engen- Such claims about the of physical embodiment dered much exploration and reflection. in computer-mediated interactions are usually supported by recourse to philosophical and phenomenological writings, typically Heidegger, Merleau-Ponty, and Gib- 1 Introduction son. To begin to understand how different kinds of mixed might produce different user experiences Recent advances in the design of interactive technol- that differ from other kinds of computer-mediated in- ogies have allowed the possibility of designing mixed real- teractions requires us also to begin systematically exam- ity environments (MREs). Drascic and Milgram (1996) ining them from a theoretical and empirical perspective. write “between the extremes of real life and Virtual Reality From a theoretical point of view, we can consider a po- lies the spectrum of Mixed Reality, in which views of the tential distinction as that between real world are combined in some proportion with views of (i) the “real” world in which spaces and artifacts are a virtual environment” (p. 123). A number of claims have acted on by conventional physical actions and been made about the benefits of different kinds of mixed where the user’s understanding is, therefore, in realities (also sometimes called augmented reality), includ- terms of general causal models of the world, and ing enriching the user experience (Camarata, Yi-Luen Do, Gross, & Johnson, 2002; Schnadelbach et al., 2002), en- hancing learning (Underkoffler & Ishii, 1998), and im- proving collaborative working and planning (Fjeld et al., Yvonne Rogers 2002). However, little is known as to why and how aug- [email protected] menting real life with digital representations produce such Mike Scaife effects. One main thesis that has been proposed is that Silvia Gabrielli manipulating familiar physical artifacts (such as toy bricks) Hilary Smith Eric Harris Interact Lab-COGS Presence, Vol. 11, No. 6, December 2002, 677–686 University of Sussex © 2002 by the Massachusetts Institute of Technology Brighton, BN1 9QH, UK

Rogers et al. 677 678 PRESENCE: VOLUME 11, NUMBER 6

(ii) the “virtual,” in which a different, and as yet little (Gislen & Harvard, 2000) and KidStory (Alborzi et al., understood, of causal models operate and ac- 2000). These have all centered on the familiar activity of tion is arbitrarily coupled to the properties of the storytelling, but mediated and enacted out through perceived world. quite different means compared with traditional aural and verbal methods. Children are encouraged to con- However, we now also have the possibility of extend- struct and manipulate aspects of the physical environ- ing the ontological profusion of worlds and objects to ment to create and listen to theirs and other’s stories. include environments with pervasive computing proper- This includes getting children to record and replay sto- ties, building artifacts that have embedded digital intelli- ries through moving toy animals around a mat, creating

gence. To some extent, such objects have properties of Downloaded from http://direct.mit.edu/pvar/article-pdf/11/6/677/1623826/105474602321050776.pdf by guest on 28 September 2021 stories in conjunction with using video projections, and both the former two. This raises the question of how manipulating 3-D structures by changing their color, people will deal, not only with the virtual spaces that texture, and form. KidsRoom (Bobick et al., 1999) was Drascic and Milgram (1996) describe, but with mixed also designed as an interactive narrative playspace, using reality environments that combine real, virtual, and images, music, narration, light, and sound effects, and ubiquitous forms. in which children were guided through a reactive adven- Minimally, we need a terminology/taxonomy that ture story. allows us to describe acting in and on these forms, and The notion of tangibles is increasingly gaining cur- we offer a beginning set here. But we also need to link rency as a way of describing physical artifacts embedded this to appropriate empirical work, to see how far these with computational power or closely coupled with digi- have utility and whether they can be useful for tal responses. Within the context of learning, they are design of MREs. often designed to enable the occurrence of unexpected The area we have chosen to empirically investigate events, novel reactions, novel activities, and novel com- our conceptualization of mixed reality environments is and learning. Our focus is on examining how new binations of activities or events, which in turn facilitate forms of physical/digital embodiment might encourage children to question them and to reflect on their experi- children to explore more and reflect on what they are ences (Hoyles & Noss, 1999; Stanton et al., 2001; Rog- doing. Promotion of reflection in children is a general ers et al., 2002). learning goal, as it is well known to stimulate awareness Much of this line of has so far been ad hoc and enhance learning (Piaget & Inhelder, 1967). A in its design, focusing on enabling children to appro- number of researchers have started to experiment with priate technologies in creative ways, rather than on manipulative materials and physical artifacts to provide understanding the of mixed realities per se. A novel playing and learning activities, with this goal in general approach has been to provide a bricolage of mind (Colella, Borovoy, & Resnick, 1998; Druin & tangibles and activity spaces. In contrast, our goal Perlin, 1994; Resnick et al., 1998; Stanton et al., 2001). here is to begin to understand better what the effects Most well known is the work of the MIT lab, where a are of particular kinds of mixed realities on children’s main emphasis has been on creating a variety of physical exploratory behavior and reflection. In particular, we toys and learning tools, embedded with computational are interested in how a very familiar embodied activity and communication capabilities, aimed at enhancing can be “dressed up” in the context of different mixed interactivity and to engage children in new ways of realities, creating “deviations” from what children thinking (Cassell & Ryokai, 2001; Kolomyjec, Cassell, usually expect to happen and to see how they react to Kafai, & Williamson, 1997). them. To this end, we have developed a conceptual Specific projects that have developed physical and vir- framework of mixed realities, which we categorize tual spaces to support children’s , include next in terms of the set of four possible combinations StoryMat (Ryokai & Cassell, 1999), Video Sandbox of the physical and the digital. Rogers et al. 679

2 Conceptualizing Mixed Reality Spaces Table 1. Transform Type and Children’s Existing Level of in Terms of Transforms Familiarity with Them

Level of Previous research into conceptualizing mixed real- Transform type familiarity ities has focused on moving from one physical place to another virtual one (and versa) via boundaries that Physical action 3 Physical effect (PPt) Highly transparently connect separate, nonoverlayed physical familiar and virtual spaces. For example, Koleva, Schnadelbach, Physical action 3 Digital effect (PDt) Unfamiliar Benford, & Greenhalgh (2000) and Koleva et al. Digital action 3 Digital effect (DDt) Familiar

(2001) describe the properties of a “traversable” inter- Digital action 3 Physical effect (DPt) Highly Downloaded from http://direct.mit.edu/pvar/article-pdf/11/6/677/1623826/105474602321050776.pdf by guest on 28 September 2021 face as something that “gives the of joining unfamiliar physical and virtual worlds together and that users can physically cross from one to the other” (p. 233). A number of properties have been suggested that can af- tions of actions and effects, varying along physical and fect the success of these kinds of traversal , digital dimensions (see table 1). namely permeability (how information passes through a These include a physical action with a physical ef- boundary), situation (the boundary’s spatial proper- fect (such as making a mark on paper with a pencil) ties) and dynamics (its temporal properties). In addi- and a physical action with a digital effect (such as tion to the of crossing boundaries, we propose moving a wand causing an animation to appear). The the basic construct of transforms as a way of concep- conceptual spaces are also characterized in terms of tualizing a mixed reality . By this we mean how familiar the transforms are to the children. The changes in the state of the world. In everyday life, rationale for using familiar/unfamiliar as an organiz- people routinely encounter and represent transforms ing here is that it should reflect experience between states of the world—for example, in percep- and, consequently, as our previous research has tion (seeing an object disappear and then reappear or shown, that unfamiliarity provokes reflection by the changing one’s viewpoint), in action (when the pur- child (Rogers et al., 2002). pose of a gesture changes), and in cognition (as when we re-represent and reinterpret the state of the world). Dealing with transforms involves some im- 3 Using to plicit or explicit of what causes changes of per- Design an Experimental Setting ceptual/cognitive states; that is, some sort of causal link is usually involved. Hence, transforms are a con- To investigate the four conceptual distinctions, we stant feature of ongoing and cognition. designed a mixed reality environment for young chil- Here, we propose the additional term transform type dren as part of the Equator project (www.equator. to identify the different kinds of forms involved, viz ac.uk). Our goal, within this research initiative, is to the real, virtual, and digitally enhanced trio identified develop novel mixed reality spaces that enable children in our definition of mixed reality environments. For to experiment and play across different media and repre- our purposes, however, we will use the term physical, sentations while engaged in simple or more complex rather than real, in this paper to allude to actions/ activities. In particular, we wish to demonstrate some- activities/effects that do not involve virtual/digitally thing of the unique properties of the digital to enhance enhanced artifacts. For the latter, we shall use digital the physical by offering certain kinds of external cogni- as a cover-all term. tive support (Scaife & Rogers, 2001). The chief objec- Our framework divides the conceptual space into four tive in the design of the present MRE was to get chil- kinds of transform types, coupling different combina- dren to both experience and reflect upon their 680 PRESENCE: VOLUME 11, NUMBER 6

interactions, allowing hypotheses about their conceptual- portant to realise that we are referring here to the mech- izations of these environments. At the heart of our design anism that potentiates the transform. Thus, a “digital was the need to exemplify a variety of transforms within a action” (like using a painting program on a display specific domain, based on the combinatorial screen), inevitably involves some degree of physical ac- possibilities of real, virtual, and ubiquitous forms. tion on the part of the user, but it is the (digital) mech- To this end, we developed an activity space called the anism that allows this that is crucial here. The four Chromarium, an environment in which color may be transform types developed for the Chromarium study contained, observed, and experimented on in a variety are described in subsection 3.1 through 3.4. of ways. The core activity the children were required to engage in was discovering and experimenting with the Downloaded from http://direct.mit.edu/pvar/article-pdf/11/6/677/1623826/105474602321050776.pdf by guest on 28 September 2021 3.1 Condition 1: Physical-to-Physical mixing of colors. This is a very familiar physical activity Transform (PPt: Action and Effect of that children normally enjoy using jars of paint, and Same Kind) which we wished to extend through providing novel transforms that they were unfamiliar with. To achieve In this condition, we used the highly familiar ac- this, we considered how color mixing changes when tivity of mixing paints as a baseline. In addition, we using different media (such as light) in conjunction with asked the children to mix lights using a set of flashlights designing the various transforms (such as digital to that had different-colored filters and were shone under- physical). Although children are very familiar with the neath an acrylic surface (see figure 1). Mixing colors effects of mixing colored paints, they are unfamiliar with with paint involved using a paintbrush and selecting wet the different effects of mixing lights. For instance, mix- paints from different pots and combining them on a ing all three additive primary colors using light produces palette. The main constraint with this medium (that the white, whereas mixing equivalent primary paint colors, children are highly familiar with) is that it is easy to add subtractive primaries, produces a brown-black color. colors but not to remove them. In contrast, this con- For each kind of transform type, we asked the chil- straint does not occur when mixing lights, as it is easy to dren to guess what would happen if they mixed differ- both add and remove colors by moving the flashlights ent colors and then subsequently got them to experi- toward and away from the surface. We were interested ment with the different setups. Sample questions in whether the children discovered this in their experi- included topics such as menting and if they reflected upon this.

● which secondary colors are obtained from mixing two primary ones (“What color do you make when 3.2 Condition 2: Physical-to-Digital mixing red and yellow?”), Transform (PDt: Action in Physical ● what happens when you mix two secondary colors with Digitally Based Effect) together (“What color do you make when mixing In this condition, we used RFID tags to enable green and purple?”), and physical actions to trigger virtual effects. We built two ● what happens when all the primary paint colors are colored blocks, with a different color displayed on each mixed together (“What color do you make when of their six faces. Each face of the block was also embed- mixing red, yellow, and blue?”). ded with a hidden RF tag, so that it had a unique iden- In all cases, the of the transform—namely tifier. When a face of the block was placed on the RF color mixing—remains constant (although the particular tag reader, an animation mirroring the color of the form of this obviously varies). What we have done in identified face was triggered and projected onto an adja- our design here is to alter the transform type, following cent vertical display (see figure 2). When two blocks our conceptual framework. Four types were set up and were placed together on the tag reader, the two surfaces labeled according to the mechanisms involved. It is im- were read at the same , triggering an animation of Rogers et al. 681 Downloaded from http://direct.mit.edu/pvar/article-pdf/11/6/677/1623826/105474602321050776.pdf by guest on 28 September 2021

Figure 1. Physical-to-Physical transform (PPt): using flashlights to Figure 2. Physical-to-Digital transform (PDt): using colored building mix light colors. blocks to mix digital colors. the color that would result if the two were mixed (for added and subtracted when mixing either digital paint instance, a red and yellow face would show an orange or light. animation). As with condition 1, the children were asked to guess and subsequently experiment with what 3.4 Condition 4: Digital-to-Physical colors might result when combining the surfaces of the Transform (DPt: Digital Action with blocks together. Physical Effect)

This was the most difficult condition to provide a 3.3 Condition 3: Digital-to-Digital plausible transform. In the end, we used the Mimio Transform (DDt: Action and Effect of setup to enable a digitally potentiated action to trigger a Same Kind) physical effect. A two-colored (such as blue and white) digital windmill was displayed on the desk surface. Mov- For this condition, we provided an interactive hor- ing the arms of the digital windmill triggered the spin- izontal surface (Mimio) that enabled digitally potenti- ning of a physical windmill that was placed nearby and ated actions to trigger digital effects. Two software tools had corresponding sets of colored arms (see figure 4). were used that supported mixing light or paint in a digi- The effect was to produce a spinning color in the physi- tal space, respectively (Mixing Colours and Computer cal windmill (for instance, light blue-silver when blue Crayons). Both involved dragging colored disks (repre- and white arms were selected). senting paint or light) to overlap. Mimio input devices, It should be pointed out that the four conditions disguised as either a paintbrush or a flashlight, were were not designed to be functionally equivalent, but as used for selecting the colors. The “paintbrush” caused different kinds of mixed realities with the aim of helping the digital disks to move when the brush hairs were us to understand how different combinations of the pressed against them (see figure 3). The “flashlight” was physical and the digital affect children’s behavior. In held slightly away from the surface, and a button on its particular, we were interested in how children under- side was pressed to activate it. Hence, the setup for the stand and reflect upon novel transforms (that is, DPt DDt condition mirrored the two kinds of activities of and PDt) compared with more familiar ones (DDt and the PPt condition. A main difference between the two PPt). Accordingly, we analyzed our quali- conditions, however, was that colors could be both tatively. 682 PRESENCE: VOLUME 11, NUMBER 6 Downloaded from http://direct.mit.edu/pvar/article-pdf/11/6/677/1623826/105474602321050776.pdf by guest on 28 September 2021

Figure 3. Digital-to-digital transform (DDt): using digital paint and Figure 4. Digital-to-physical transform (DPt): using digital colors to light tools to mix digital colors. mix physical colors (DPt).

4 Method ferent conditions and what kinds of accounts were given by the children when talking about the underlying cause Ten pairs of children, aged between five and six, and effects of the various transforms. were asked to take part in the Chromarium study. They were told that they would be mixing colors in fun and 5.1 Children’s Explorations with the sometimes unusual ways. They took part in all four con- Transforms ditions, starting with the familiar activity of mixing physical paints. Pairs of children were asked to collabo- As we had predicted, the children experimented rate because this is generally considered to be a more and reflected more with the less familiar transform types effective way of getting children to talk openly and re- (PDt, DPt, and DDt). The most experimenting was flect upon their experiences. To elicit their understand- done in the PDt condition and involved using the col- ing of the transforms, we prompted them to talk about ored blocks to mix colors. For example, several of the their experience in the different conditions by asking pairs of children tried to discover if the tag reader sur- open-ended questions such as “What do you think will face was able to identify or capture other objects besides happen if...?” or “How do you think this is work- the colored blocks. Two girls, for example, touched the ing...?” To elicit further from the chil- tag reader area with their faces, expecting them to be dren, we also made use of counterfactual questions, scanned as digital images and projected onto the screen, such as asking the children to say what would happen if just like the blocks. Their assumption was that the tag they tried to mix other materials (such as paper or fab- reader acted like a scanning device and could read any ric). The children were recorded on videotape during kind of object. the study. All the children tried experimenting with the various physical properties of the blocks to see what other ef- fects they might produce. For example, several pairs of 5 Findings the children placed the colored blocks in towers to see whether different colors would emerge with different The children’s interactions and verbal reports were combinations of colored blocks. The way the RF tags transcribed from the video data. These were analysed in had been engineered in the blocks meant, however, that terms of what kinds of exploration took place in the dif- only two surfaces could be read and mixed at any one Rogers et al. 683

time. One pair of children also expected digital mixing mixing the same colored paints (for example, mixing to happen when moving the blocks together, away from red and green made a yellow color as opposed to the table surface. Another pair tried to put the blocks brown). When questioned about this, however, the chil- against the digital image projected onto the wall to see dren were unable to explain why there was a difference if any effect was produced. They also tried to see if they between mixing lights and paints, nor did they try to could select a block’s face by orienting it towards the experiment further with other combinations that might area projected on the surface. Three of the pairs of chil- produce different effects. Only one child was able to dren pressed the blocks down hard on the table surface, come up with an account, and did so by resorting to the trying to amplify the mixing (such as making it darker). use of (Facilitator: “Did you know you can mix

We also found that the children experimented the colors like that with [the flashlight]?” Child: “No, but I Downloaded from http://direct.mit.edu/pvar/article-pdf/11/6/677/1623826/105474602321050776.pdf by guest on 28 September 2021 most with the transforms, where the coupling between do know what you can do with sun and rain. I know the cause and effect could be rapidly observed and re- how you make a rainbow. Sun reflects on rain.”). versed. This was most noticeable for the colored blocks, To try to get the children to reflect and more with which mixing new colors could be done rapidly about the differences when mixing colors in the differ- simply by turning the blocks over. Indeed, all pairs of ent conditions, we tried a different tact, which was to children placed most, if not all (with several repeats) of ask counterfactual questions, to which the children were the possible permutations of the six different sides of the much more forthcoming with their answers. For exam- two colored blocks on the RF tag reader. Turning the ple, when asked “Why is material not like paint?”, all of blocks over to recombine different surfaces and then the children were able to respond with answers like “be- watching the immediate effect as a digital representation cause it’s not gooey!” or “it is not runny.” This implies on the screen was very satisfying and perpetuated fur- that they were aware of a key property of paint that is ther explorations. It also proved to be a highly collabo- what enables color to be mixed, but that other media or rative activity. Typically, both children would grab a materials lack and thus prevents them from being mixed block and turn it, or sometimes one would take both in the same fashion. When it came to understanding while the other made suggestions as to what faces to mix. how the hard surfaces of the colored blocks could be In contrast, in the DDt condition, the children combined to make a new color, the children were able tended to explore color mixing using the software tools to make an exception to this “rule.” The “mixing” was by themselves and chose not to talk with each other or done through the transform, producing a digital repre- make suggestions as to what each should do. A conse- sentation of the resultant color. Hence, it did not con- quence was that they did not explore nearly as many flict with the of their earlier reasoning because combinations, even though there was the same scope they understood a different cause-effect model to be for experimenting with and reversing actions as in the underlying it. In , the children’s understanding of PDt condition. the involved in this condition were largely technocentric. When asked to explain how the digital color mixing worked, they said things such as “The ef- 5.2 Children’s Accounts of the fect is coming from the computer’s screen over there, Transforms and it arrives here by means of electricity” (six-year-old The very familiar activity of mixing colors using girl). Another child (a five-year-old boy) pointed first at paints elicited unremarkable responses; the children the table under which the tag reader was concealed, knew beforehand the outcome of mixing two or three then to the projected image, then to the computer, and colors, and this was confirmed by their actions. When said “. . . connect, connect, connect...wire...itis provided with the flashlights, they were bemused at first connected under here [table] and goes all the way up to by the fact that the result of shining the lights together there [PC behind him]!”. under the acrylic surface produced a different color from Similarly, in the DPt condition, the children often 684 PRESENCE: VOLUME 11, NUMBER 6

resorted to technocentric explanations of what caused and different ways of augmenting the physical, we chose their actions in the digital environment to mix colors in to conceptualize mixed reality here in terms of how ac- the physical windmill. One pair said “this [digital wind- tivities can be transformed through different ways of mill] is making that go round and round, because we combining the digital and physical. We provided a sim- are using this [the Mimio pen and the digital wind- ple classification of transform types that enabled us to mill]...andthey are connected with wires!” They also design different kinds of mixed realities, varying in their looked under the table to discover which device was degree of novelty and type of underlying cause and ef- causing an effect and how the different pieces of tech- fect. nology were connected to each other. We’ve since used the framework to develop other

The action of pushing was also considered an impor- novel learning environments for children (Price et al., Downloaded from http://direct.mit.edu/pvar/article-pdf/11/6/677/1623826/105474602321050776.pdf by guest on 28 September 2021 tant part of the : one pair of children ex- 2002). It has provided us with a structure to inform the plained that the physical windmill made a silver green- design of different kinds of activity spaces that combine blue color because they were pressing on the blue and and chain familiar actions with unfamiliar effects. In so white arms of the digital windmill. Another child said to doing, it has allowed us to be systematic in combining the digital with the physical and vice versa, based on our the other, “this is making it go round and round,” to understanding of familiar physical-physical transforms. which the other said “it’s because we’re pressing the The most interesting transform that was found in our computer.” Interestingly, similar to the children’s rea- empirical study of color mixing was where the children soning about the pushing down of the colored blocks, manipulated physical artifacts (colored blocks) to create one pair thought that the faster they moved the digital digital results. The combination of reversibility and im- windmill around the paler the color would become in mediate feedback enabled by this transform was central. the corresponding physical windmill, as its arms would In addition, the affordances of the physical blocks in- likewise move faster. (This was not actually possible to vited “embodied” actions. Namely, the physical blocks do, although the perceptual effect of the physical wind- were “ready at hand” for manipulating and sharing: mill starting up and slowing down did convey this.) they could easily be picked up by both children simulta- neously or in close succession and a range of highly visi- ble and natural actions performed on them, like swap- 6 Discussion ping, turning, combining, and placing them on the surface or on top of each other. In contrast, even A main objective of the work reported here was to though the digital-digital transform provided the same develop a conceptualization of mixed realities that could scope for reversibility and immediate feedback, it was provide a means of systematically investigating and in- discovered that the selection of the digital disks of color forming the design of novel user experiences. In partic- using the Mimio pen or the flashlight did not support ular, we wanted to begin understanding how people anything like the same kind of ready-at-handedness. dealt with mixed realities that combine different physical Instead, the degrees of freedom of what actions could and digital forms, especially those that capitalize on fa- be performed on the disks was heavily constrained (only miliar actions. Do they treat the mixed reality experi- dragging allowed), and just one child could perform an ences differently from everyday experiences, and, if so, action at a time while the other remained essentially an are there differences across the possible set of physical onlooker. Moreover, to switch roles required one child and digital combinations? To begin answering this ques- to consciously and deliberately hand over the input de- tion, we argued, needed both theoretical and empirical vice to the other child—a gesture that is often socially investigation. difficult for children to do, since it requires them to Whereas others have focused on conceptualizing the hand over “control,” which often can be problematic, properties of traversals across physical and virtual spaces either through the child forgetting about the other or Rogers et al. 685

not wanting to for various (such as being sub- Bobick, A. F., Intille, S. S., Davis, J. W., Baird, F., Pinhanez, missive or showing failure). Studies of collaborative C. S., Campbell, L. W., Ivanov, Y. A., Schutte, A., & Wil- learning, in which children have to share the same son, A. (1999). The KidsRoom: A perceptually-based inter- mouse when using a piece of software, point to how active and immersive story environment. Presence: Teleopera- tors and Virtual Environments, 8(4), 369–393. awkward and difficult such change-overs can be (Stan- Camarata, K., Yi-Luen Do, E., Gross, M. D., & Johnson, ton et al., 2001). B. R. (2002). Navigational blocks: Tangible navigation of In terms of designing novel transforms, our study has digital information. CHI 2002 Proc., 752–753. shown that the most challenging transforms to create Cassell, J., & Ryokai, K. (2001). Making space for voice: are digital-to-physical ones as it was hard to provide a Technologies to support children’s fantasy and storytelling. range of plausible effects. Nevertheless, what we did Personal Technologies, 5(3), 203–224. Downloaded from http://direct.mit.edu/pvar/article-pdf/11/6/677/1623826/105474602321050776.pdf by guest on 28 September 2021 create was very intriguing to the children. We are cur- Colella, V., Borovoy, R., & Resnick, M. (1998). Participatory rently exploring more diverse digital-physical combina- simulations: Using computational objects to learn about tions. A crucial feature is to ensure that there is immedi- dynamic systems. In CHI ’98,9–10. ate feedback with as little delay as possible between the Dourish, P. (2001). Where the action is: The foundations of performance of an action and the perception of its effect embodied interaction. Cambridge: The MIT Press. Drascic, D., & Milgram, P. (1996). Perceptual issues in aug- (as happens with everyday physical-to-physical trans- mented reality. In M. T. Bolas, S. S. Fisher, & J. O. Merritt, forms). We are also looking at how to design counterin- (Eds.), SPIE Stereoscopic Displays and Virtual Reality Sys- tuitive and more complex types of cause and effects to tems III, 2653, 123–134. see if they can facilitate more elaborate forms of explora- Druin, A., & Perlin, K. (1994). Immersive environments: A tion and reflection in mixed reality environments. Fi- physical approach to the computer interface. Proc. of Hu- nally, one of the claims we would make about the bene- man Factors in Computing Systems (CHI ’94), 325–326. fits of designing novel mixed realities in the context of Fjeld, M., Lauche, K., Bichsel, M., Voorhorst, F., Krueger, play and learning is that juxtaposing the “unexpected” H., & Rauterberg, M. (2002). Physical and virtual tools: Ac- with the highly familiar promotes “richer” experiences, tivity theory applied to the design of groupware. In B. A. prolonged interest, and more reflection. Nardi & D. F. Redmiles (Eds.), A Special Issue of CSCW: Ac- tivity Theory and the Practice of Design, 11(1–2), 153–180. Gislen, Y., & Harvard, A. (2000). Designing amplified play- space for children. [Available on-line at: http://narrativity. Acknowledgments interactiveinstitute.se/narrativetoys/html/Publications/ sandpaper.htm]. The authors would like to thank Jonathan Cohen for use of Hoyles, C., & Noss, R. (1999). Playing with (and without) his Macromedia Director serial Xtra for communicating with words. Proc. of the 7th European Logo Conference Eurologo Tag-it, Ontario Science Centre for kindly permitting our use ’99, 18–29. of their Mixing Colours and Computer Crayons in this Koleva, B. N., Schnadelbach, H. M., Benford, S. D., & work (www.ontariosciencecentre.org), NIQ inc for EZI/O, Greenhalgh, C. M. (2000). Traversable interfaces between and Virtual Ink Corporation for the Mimio device. real and virtual worlds. Proc. ACM Conference on Human Factors in Computing Systems (CHI 2000), 233–240. Koleva, B., Taylor, I., Benford, S., Fraser, M., Greenhalgh, C., References Schnadelbach, H. M, vom Lehn, D., Health, C., Row-Fatt, J., & Adams, M. (2001). Orchestrating a mixed reality per- Alborzi, H., Druin, A., Montemayor, J., Sherman, L., Taxen, formance. CHI 2001 Proc, 38–45. G., Best, J., Hammer, J., Kruskal, A., Lal, A., Plaisant Kolomyjec, B., Cassell, J., Kafai, B. Y., & Williamson, M. Schwenn, T., Sumida, L., Wagner, R., & Hendler, J. (1997). The implications of a theory of play for the design (2000). Designing StoryRooms: Interactive storytelling of computer toys. Proc. of the 24th annual Conference on spaces for children. DIS2000 Proc., 95–104. Computer Graphics & Interactive Techniques, 431–433. 686 PRESENCE: VOLUME 11, NUMBER 6

Piaget, J., & Inhelder, B. (1967). The child’s conception of Scaife, M., & Rogers, Y. (2001). Traversing between the digi- space. New York: W.W. Norton & Company. tal and the physical: What does it mean? Equator Workshop Price, S., Rogers, Y., Scaife, M., Stanton, D., & Neale, H. Proc. (Apr. 24, 2001, Porto, Portugal), 7–9. (2002). Using “tangibles” to promote novel forms of playful Schnadelbach, H., Koleva, B., Flintham, M., Fraser, M., learning. To appear in Behaviour, Information and Technology. Izadi, S., Chandler, P., Foster, M., Benford, S., Green- Resnick, M., Martin, F., Berg, R., Borovoy, R., Colella, V., halgh, C., & Rodden, T. (2002). The Augerscope: Kramer, K., & Silverman, B. (1998). Digital manipulatives: A mixed reality interface for outdoors. CHI 2002 Proc., New toys to think with. CHI ’98 Proc., 281–287. 9–16. Rogers, Y., Scaife, M., Harris, E., Phelps, T., Price, S., Smith, Stanton, D., Bayon, V., Neale, H., Ghali, A., Benford, S., H., Muller, H., Randell, C., Moss, A., Taylor, I., Stanton, Cobb, S., Ingram, R., Wilson, J., Pridmore, T., & D., O’Malley, C., Corke, G., & Gabrielli, S. (2002). Things O’Malley, C. (2001). Classroom collaboration in the design Downloaded from http://direct.mit.edu/pvar/article-pdf/11/6/677/1623826/105474602321050776.pdf by guest on 28 September 2021 aren’t what they seem to be: Innovation through technol- of tangible interfaces for storytelling. CHI 2001 Proc., 482– ogy inspiration. Proc. of DIS 2002, 373–378. 489. Ryokai, K., & Cassell, J. (1999). StoryMat: A playspace for col- Underkoffler, J., & Ishii, H. (1998). Illuminating light: An laborative storytelling. Proc. of the ACM SIGCHI Conference optical design tool with a luminous-tangible interface. CHI on Human Factors in Computing Systems (CHI ’99), 272–273. ’98 Proc., 542–549.