From:MAICS-2000 Proceedings. Copyright © 2000, AAAI (www.aaai.org). All rights reserved. VEHanoi: Performance differences in Virtual vs Real Environments Tyler Waite School of Library and Information Science Indiana University

ABSTRACT off the side of a virtual cliff withoutthe fear This research used a real and a virtual Tower of death. The finality of actions, whichonce of Hanoi puzzle to examine performance performed can never be undone, does not differences betweenvirtual and real apply in this world of light and illusion. environments. The puzzle was solved twice in the real environmentand twice in the Commentingon the differences between the virtual environmentby each of the typical computerscreen interface and the participants. A one-way ANOVAwas experience of a VE, Johnson (1994) states performedfor each of the four trials to "Virtual reality gives the user an comparethe performance difference environmentto work in, rather than a screen between the two environments. to look at." Researchers in chemistry Comparisons were made on number of (Burdea, Richard & Coiffet 1996), movesto solve the puzzle, time to movean meteorology, astronomy, network object, and wand selection method. The administration (Crnz-Neira et al 1993, see analysis showeda significant difference in also Larijani 1994), and air traffic control favor of real world performanceof the task (Heim, 1993), not to mention psychology with respect to time to movean object. The (Cruz-Neiraet al 1993), just to namea few, paper concludes with a report on participant have all been exploiting the potential commentsand an exploration of the possible application of virtual reality technologyto sources of this difference. develop new and exciting environments to workin. Keywords CAVE,virtual environment, Tower of The fact that VEsare creatable provides Hanoi, spatial cognition scientists with a powerful newtool with which to explore humanbehavior and INTRODUCTION cognition. With VE’s the cognitive scientist Quotingthe wordsof researcher J. Findlay, can develop a variety of highly controlled speaking about virtual environmentsin scenarios that can be called up in an general, "These environments may provide instance. The researcher can quickly switch the necessarytools to assist researchers in the participant in or out of different test disclosing the mysteries of human scenarios with a few key-strokes. The perception." (Stanney 1998). Virtual whole environment can be altered or only environments (VE) present the researcher one characteristic of one specific object with a three dimensionalworld that is programmableand under their control. VE technology encompasses a variety of Within this computerized world, a person equipment. There are head mounted can free their mindof concern about displays, monocular displays, augmented incurring serious phYSicalpain as a reality displays, haptic (tactile) interfaces, consequenceof their actions. They can step desktop 3D worlds, and CAVE Robotic ,and Simulated Agents 13 environments to name a few. This study CAVEenvironment also permits the user to used a CAVEenvironment. The CAVEat view their ownbody, and the bodies of other Indiana University is comprisedof four whoare in the environment with them projection surfaces; three 8 foot by 8 foot (Leigh, 1996), while they are immersed screens, and the floor (see fig. 1). The three the virtual world, which maylessens wall screens use rear projection, while the feelings of claustrophobia or feelings of floor image is produced from an overhead disembodiment. mirror projecting the imageon the floor. The screens are arranged in front of and to either side of the person inside the CAVE. The stereoscopic images of the CAVEare produced using shutter glasses which blend the imageson the projection surfaces into a virtual world whose contents when approachedappears to exist not on the screens but in the open space between the projection surfaces. The glasses also have a head-tracking device connected to them so that the proper perspective and environment view can be presented Figure 1: Drawing of CAVEfrom http://www.cica.indiana.edu/ For this experimentthe participants interacted with the environmentusing a Whydevelop a Virtual Tower of Hanoi? CAVEwand. The wand is roughly the size Accordingto Fillbrant (1987)"In the of a typical television remotecontrol. The psychology of thinking, the Towerof Hanoi base of the wandis cylindrical to permit is one of the most extensively studied easy grasping and holding. The top of the problems. Nevertheless the central question wandhas three buttons and a mini joystick of what and howparticipants learn during for movingaround in the virtual their solving efforts muststill be considered environments. as not being answeredsufficiently." The Tower of Hanoi (TOH)paradigm and it’s What advantages does a CAVEenvironment isomorphs have found significant offer over the more common,and often performance differences between: right cheaper, head mountedvirtual environment brain vs left brain cognitive styles (HMVE)?One of the problems of HMVEs (Albaile,1996), medicated vs unmedicated is they have been reported to induce motion ADHDchildren (Aman 1998), normal sickness. In the CAVEenvironment, while heating vs hearing impaired young adults motionsickness is still possible, these (Luckner 1992), normal vs obsessive effects are reportedly extremely low (Cruz- compulsiveindividuals (Matraix-Coles et al Neira, Sanden, and DeFanti 1993). This is (1999). It has also been used to study the due in part to the fact that CAVEimages do effects of Parkinson’s disease (Saint-Cyr not need to be updated for every turn of the 1988). Fromthis sampleof research it is head or movementof the users body. It can clear that the TOHpuzzle is a very useful remainstatic until the user activates an tool in evaluating a widevariety of topics. object or indicates that a changeof Thus the flexibility of a virtual TOHmight perspective or location is desired. The prove itself a useful tool for cognitive

14 MAICS-2000 research, transfer of training issues.

Howmight TOHbe related to usability and Zhang & Norman (1994) using a TOH user evaluation tests? Stanney (1998) isomorphsof coffee cups filled with coffee commentingon a review of HCI studies instead of rings were able to eliminate states that "The HCIstudies indicated that stacking errors since participants in the individuals whoscore low on spatial experimentsinstinctively knewnot to place memorytests generally have longer mean the smaller coffee cup on top of a larger execution times and morefirst-try-errors." coffee cup. In VEsthe external Since TOHcan be viewed as a measure of representations and the inherent properties spatial reasoningability (Albaile, 1996), of objects can vary significantly fromtheir comparinga users performanceto a baseline real world models. Since the objects are not measure of TOHperformance, might be a solid they can easily pass through other useful addition to a user assessmentbattery objects and the user unless this quality is of tests similar to the Virtual Environment programmedinto the object. Since there is Performance Assessment Battery no gravity objects will not drop to the (VEPAB)(Lampton,1994). groundunless there is a subroutine built into the code that performs this action. Theywill TOHhas been defined as a transformational not make a sound unless this is programmed problem. A transformational problem, is into their routine, and the soundthat is described by Greeno(1978) as, requiring assigned to them is not boundby any natural "an initial state and goal state, and constraints such as their inherent material permissible operators to transform the initial composition. And if VE objects were state into the goal state." Fromthis programmedto act as if there was some definition it wouldseem that researching gravitational force it could be of an infinite howpeople solve transformational problems degree of strength and could comefrom a such as the TOHmight provide insights into variety of directions. VEobjects also cannot howpeople movefrom an initial state to a be felt with out proper equipmentand goal state whennavigating and manipulating programs. Howmight these factors impact modemgraphical user interfaces, and virtual performance on the task? environments. Fromthis introduction it is clear that the Sweller (1983) discovered that participants potential questions that could be explored whohad been trained to solve the 5-ring with VE’s are many. This initial study was TOHproblem ran into difficulty when asked designed specifically to evaluate whether to then solve a 4-ring TOHpuzzle. At first there are significant performancedifferences this result seemsstrange since the 4-ring between the VEand the real world on the solution is simpler in that it requires of TOHtask. The dependant variables that lower minimumnumber of moves to reach were examinedin the study are, numberof the solution. However,if participants try to movesto solve the puzzle, and time to move apply the steps of the solution to the 5-disk a ring. While no significant performance TOHpuzzle to the 4-ring TOHpuzzle they differences are expected, VEperformance end up placing the 4-ring stack on the maybe slower due to the unfamiliar setting middle peg instead of on the goal peg where and interface. The behavioral differences it needs to be. Thus TOHcan be employed outlined in the previous paragraph mayalso to evaluate both negative and positive be a source of performancedifference.

. Robotic and Simulated Agents 15 Since objects can be selected with the wand Three of the real rings were borrowedfrom using two methods we looked at whether the a Fischer Price "Rock-A-Stack"ring- click-hold (place wandon object, push and stacking toy. The largest ring and the three- hold button to pick up object, moveobject, pronged base of the puzzle came from a release button to drop object) methodfor Discovery Toys ring toss game. The virtual object selection, or the click-release (place tings attempted to matchthe color to the wandon object, push and release button to appropriate size of the real rings. However, pick up object, moveobject, push and during the developmentof the virtual tings it release button to drop object) methodhad was discovered that the virtual tings needed any impact on performance. It is believed to be larger than the size of the real rings that the click-hold methodis a morenatural due to trouble picking up the rings with the methodsince the user holds downthe button VEwand if ring size had been matched. similar to howa person has to hold on to an object in order not to drop it. Participants in the VEhad no tactile feedbackso a visual cue, (ring turned red) If no significant differences are found was given to indicate whenan object had between the two environments then the been touched. An auditory cue was also virtual TOHcan be viewed as a valid tool provided when the object was picked up (a for studying howvarious object and sliding sound) and released (a dropping environmental manipulations impact task kerplunk type sound). The VE also lacked performance.If significant differences are poles to place the rings on but instead had found between the two environments then flat black target ring markson the virtual further research should try to establish the tabletop to indicate the three valid placement source of the performancedifferences. locations. The VErings also remained suspended in the place where they were METHOD released, there was no virtual gravity acting Twelvegraduate students at Indiana on the objects. Clearly these are potentially University were used in this study. Their significant differences betweenthe two average age was 31.25. All but one of the environments. Howeversince these factors participants were tight-handed. The may be commonin many virtual participants passed tests for color blindness environmentsit is worth studying whether and stereoscopic vision. Only 3 subjects their effect on performanceis significant. reported that they were familiar with the Towerof Hanoi puzzle. Seven of the In an attempt to control for learning effects, subjects reported that they had experienced a six of the participants performedthe real virtual environmentbefore, howeverit was worldtask first and the virtual task second. generally not a Cave environment (3D The other six participants performedthe games, full motion simulator, HMD,gloves, tasks in the reverse order; virtual first, real one subject had seen a CAVEtour but second. The task was performed with 4 hadn’t used the technology). Noneof the disks. The object of the puzzle is to move participants received any form of monetary the stack of tings fromthe far left position to compensationfor their participation. the far right position. Movementof the Interest in the CAVEand virtual rings must follow the rules that only one environmentswas a sufficient motivating ring can be movedat a time, and a ring can factor. only be placed on top of a larger ring or an

16 MAICS-2000 empty pole position. Since in the VE there was nothing to prevent pulling rings from the bottom of the stack a further rule was added that tings must be selected from the top of a stack. Also in order to test whether wand use affected performance, half of the 3 * subjects were instructed to use the click-hold ~5 wand method and half were instructed to use ::--.Z the click-release method. Participants were 2 also questioned about their CAVE 1.s expectations and experience. ;

RESULTS its o A one-way analysis of variance was I 2 3 4t performed to compare the performance on TrIM the task between the two environments for each of the four trials. The influence of the Figure 1: Timeto moverings in secondsby two wand methods on performance in the environment. VE proved non-significant. While the number of moves to solve the puzzle varied widely (15-113) there was no significant difference detected between the % % environments. However, the time to move a % % ring was significantly different between two % environments across all four trials, with \ performance in the CAVEbeing \

significantly slower. (Trial 1: F = 26.557, ~° p = 0.000, df= 1; Trial 2: F=16.057, I .’\ p=0.002, df =1; Trial 3: F=15.072, p=0.003,

df= 1; Trial 4: F = 10.684, p:0.008, df= 1). °’’.°. ~.

MNOVA:SS DF MS F P

Trial 1:22.411 1 22.411 26.557 0.000 ; 2 3 4 Error 8.439 10 0.844 Teal Trial2:15.402 1 15.402 16.057 0.002 Error 9.592 10 0.959 Trial 3:3.873 1 3.873 15.072 0.003 Figure 2: Timeto moveobjects by group.It is Error 2.569 10 0.257 interestingto note that whilethe real worldtrials appearto havehad a beneficialeffect on the virtual Trial 4:9.668 1 9.668 10.684 0.008 environmentthe virtual environmentshows no Error 9.049 10 0.905 beneficialtransfer to the real environment. Table1. Analysisof variancebetween the real and virtual environmentson time to movean object for trials 1-4. Theresults indicate that performancewas significantly slowerin the virtual environment.

Robotic and Simulated Agents 17 attemptedto select one of the rings in the pair. Three subjects were noted using movesto adjust ring positions. The line wherethe floor meets the front wail was visible through the table. A couple subjects broke the rules and attemptedto select tings

m-° from the middle of the stack instead of from -~.°oo°’- the top. (Whenthis occurred participant wereinstructed to restore the ring to its former position then continue.)

After experiencing the CAVEthe ~ 3 4 ww participants were asked to describe their experience. One commentedthat they felt that their mindwas going faster than the Figure 3: Number of Moves. Analysis indicated no significant difference between number of moves to system was able to render. They also solve puzzled. Still the increase in number of moves missed having tactile feedback from the in the real environment is quite unexpected. objects. However,one participant stated Howeverit should be noted that there is a single that they had used the auditory cues as a outlier of 113 moveson trial 4 for the real substitute for the lack of tactile feedback. environment. Somefelt that the visuals were rougher than they expected, the objects seemedfuzzy, and COMMENTS AND OBSERVATIONS Participants were asked what they expected not as realistic. Others weresurprised at the intensity and realism of the simulation. the CAVEenvironment would be like. A Several commentedthat the wand was few expected that it wouldbe similar to the sometimesdifficult to use in selecting the Holodeckof the television showStar Trek, objects, suggesting that gloves maybe more or the movie "the Matrix". Others had their natural. Several complainedabout the lack doubts, stating that they expectedit of gravity and one participant felt the lack of wouldn’tquite live up to reality. They gravity caused them to adjust disk positions~ expected that the objects wouldbe "boxy", in order to match their conceptual model of and the motion "choppy". Somefelt that howthe rings should look, and another was performing tasks in the CAVEwould be annoyedthat the rings did not sit on top of moredifficult since they wouldnot be able each other. One participant commentedthat to perform them exactly the same as they they felt the lack of gravity interfered with wouldin the real world. their ability to rememberhow they had solved the puzzle in the real world. Others Whileobserving the participants in the commentedthat they disliked the fact that CAVEseveral recurring problems were they could break normal physical laws such noted. Six of the participants had trouble as putting tings throughthe table. selecting the largest ring. Four subjects put a ring through the virtual table. Four Even with these commentsabout the CAVE subjects had notable difficulty selecting the most subjects felt that the task seemed rings. Four subjects were observed basically the samein both environments. accidentally embeddingrings inside each Strangely even with all the complaints listed other. Occasionallya pair of adjacent rings aboveseveral participants stated that they would highlight whenthe participant 18 MAICS-2000 felt the task was easier in the CAVE!One Subjects performing the task in the REoften stated that the VEpuzzle seemedmore like a dropped the rings once the rings were over game while the RE puzzle seemed more like the target pole, howeverthe VEobjects did a test. Perhapspart of this is becausethe not have gravity to makethem fall. This video camerawas visible to the participants lack of gravity in the VEmay have caused while they were performing the REtask. It the participants to take moretime in placing was noted by one of the subject that the lack the objects. As mentioned before the VE of objects surrounding the rings in the VE had no poles. In the REthe poles kept the madeit easier to concentrate. rings in a neat stack and were utilized with gravity to directing the ring to its proper Whilethree of the click-release subjects position. In the VEthe participant needed to reported no trouble using that wandmethod, pay closer attention to whereexactly they two felt it was counter intuitive, not mouse placed the ring over the target location. The like, and hard to remember.Participants virtual objects were not solid objects and as using the click-hold methodstated that they such could be put through the table or into felt it was similar to the mouse,more another object, whichcould create natural, intuitive, and morelike holding difficulties whenattempting to movethe objects in real world. rings.

Even the REtask was not free of troubles. As mentionedbefore the virtual rings were Duringtesting it was discovered that the larger than the real rings in order to facilitate smallest ring could fit within the hole of the picking with the wand. However, increased largest ring, which occasionally madethe size of the rings should have aided selection smallest ring difficult to pick up. Also, of the rings and shouldn’t have necessarily someparticipants mentionedthat the green slowed movementof the objects. The and blue rings seemedto be moresimilar in increase in size did not place the target ring size, and thus harder to distinguish between, positions outside of the comfortable reach of than the yellow and orange rings. It was the participants. also observed that the poles did on occasion interfere with.ring placement.If the ring was It is interesting to note that there is a not placed correctly over the pole whenthe decreasein significance across the trials. ring was droppedit could bounceoff the Thus it maybe possible that the difference is pole causing the participant to break their due in part to the lack of familiarity in using fluid movementto reclaim the poorly the wand to moveobjects in a CAVE dropped ring. environment. Future research maywant to comparethe performancedifferences of a DISCUSSION more natural interface device such as pinch Giventhe dissimilarities of the gloves, or haptic feedbackgloves, in environmentsand the observed difficulties, comparison to the wand device. One thing it is not surprising that there were that is certain is that visual similarity of performance differences between the two objects is not sufficient to ensure similar environments.It is not clear what the exact interaction behaviors betweenreal virtual source is of this difference. However,it is environments. Other elements such as quite likely due to those factors mentioned gravity, object solidity, and tactile feedback earlier in the paper. without a doubt play a significant role in howwe interpret and react to the world

Robotic and Simulated Agents 19 around us and should be modeled if a Heim, M. (1993). The Essence ofVR. . realistic virtual world is the goal. HeimTheMetphysics of Virtual Reality (pp. 109-138). Oxford: Oxford University Press. Reference List Lampton,D. R., Knerr, B. W., Goldberg, S. L., o Albaile, M. A. (1996). Inferred Hemispheric . Style and Problem Solving Bliss, J. P., Moshell,J. M., & Blau, Performance. Perceptual and Motor B. S. (1994). The virtual environment performance assessment battery Skills, 83, 427-43. (VEPAB): Development and Aman,C. J., Roberts, R. J. Jr., & Pennington, evaluation. Presence: Teleoperators . and Virtual Environments,3(2), 145- B. F. (1998). A Neuropsychological 157. Examination of the Underlying Deficit in Attention Deficit 10. Larijani, Hyperactivity Disorder: Frontal Lobe L. C. (1994). Applications for Versus Right Parietal Lobe Theories. Science & Engineering. In L. C. LarijaniThe Virtual Reality Primer DevelopmentalPsychology, 34(5), (pp. 165-178). NewYork: McGraw 956-969. Hill, Inc.

Burdea, G., Richard, P., & Coiffet, P. (1996). Leigh, J., & Johnson, A. In Proceedingsof the . MultimodalVirtual Reality: Input- 11. Output Devices, SystemIntegration, International Conferenceon and HumanFactors. International Multimedia Computing and Systems Journal of HumanComputer (pp. 20-23). Los Alamitos,Calif.: IEEE Computer Society Press. Interaction, 8(1), 5-24. 12. Luckner, J. (1992). Problem Solving: Cruz-Neira, C., Leigh, J., Papka, M., Barnes, . C., Cohen, S. D., Engelman,R., Comparison of Hearing Impaired and Hudson,R., Roy,T., Siegel, L., HearingIndividuals. Journal of the American Deafness and Vasilakis, C., DeFanti, T. A., & Rehabilitation Association, 25(4), 21- Sandin, D. J. (1993). Scientists 27. Wonderland: A Report on Visualization Applications in the CAVEVirtual Reality Environment. 13. Stanney, K. M., Mourant, R. R., & Kennedy, In IEEE Symposiumon Research R. (1998). HumanFactors Issues Frontiers in Virtual Reality (pp. 59- Virtual Environments: A Review of 66). Los Alamitos, California: IEEE the Literature. Presence, 7(4), 327- ComputerSociety Press. 351.

14. Zhang, J., & Norman,D. A. (1994). o Cruz-Neira, C., Sandin, D. J., &DeFanti, T. A. (1993). SurroundScreen Proj ection Representations in Distributed Based Virtual Reality: The Design Cognitive Tasks. Cognitive Science, and Implementation of the CAVE.In 18, 87-12. SigGraph93 (pp. 135-142).

Greeno, J. G. (1978). Natures of problem . solving abilities. In W. K. Estes (Ed.), I would like to thank Dr. Katy Handbookof learning and cognitive Boerner for programming the processes(Vol. 5). Hillsdale, N. J.: environment. And Dr. Andrew Erlbaum. Dillon for advising me on the project. ° Heim, M. (1993). The Erotic Ontology Cyberspace. M. HeimTheMetphysics of Virtual Reality (pp. 83-108). Oxford: Oxford University Press.

20 MAICS-2000