<<

FORUM ShortPapers TheRealityofExperience:Gibson©sWay

Abstract about the appropriate ‘‘rulers’’ for describing that real- ity.This is true for behavioral sciences as well as for This paper considers some Žrst thatmight provide physical sciences. The question ofthe measure for ‘‘real- abasis for an objective science ofexperience (presence or ity’’ is also afundamentalstarting point for theories and immersion). Dimensions thatare considered include classical measures ofvirtual environments. What are the appro- Newtonian measures ofthe distal stimulus, changes in neural priate rulers for measuring the experiences in virtual en- mechanisms reecting theproximal stimulus, vironments and for comparingthose experiences against theoretic measures ofthe statistical properties ofevents, and the experiences ofreality? As the title ofthis article indi- functional properties related tointentions and abilities. Gib- cates,I have abias. Ibelieve that the assumptions under- son’s ecological framework is suggested as apromising func- lying Gibson’s (1979) ecological approachprovide a tional approach for deŽning thereality ofexperience in rela- promising basis for framingquestions of‘ ‘presence’’ and tionto theproblem ofdesigning virtual environments. This ‘‘immersion’’ with regard to virtual environments. approach emphasizes thetightcoordination between per- However, the goal ofthis article is not toconvince read- ceptionand action and Žxes themeasurement coordinate ers ofthis position, but simply to heighten to system relative tothecapacity for action. the assumptions and existential commitmentsthat shape our theoretical and experimental perspectives on experi- ence. 1Introduction

Inthe early stages ofthis [relativity] theory,its name 2Newton©sWay led to the erroneous impression that this approachin science is based on the philosophic view of‘relativ- Newton chose space and as the absolutes for ism’— the that all is relative only to his program.Thus, space and time were considered as the ‘knower’— i.e.,that there is no objective knowl- dimensions ofacontainer whose was indepen- edge to talkabout. Of course, Einstein never had this dent ofthe objects contained within. This container view in —his approachwas just the opposite, then becamethe basis for an ‘‘objective’’ description of where one focuses onthe invariant (objective) law of an or . Itbecamethe measure ofreality . . To avoid confusion, Einstein tried to rename Fechner (1860/1966) extended Newton’s programinto his theory‘invariententheorie’(theor yofinvariants), the field ofpsychology ,arguing that ‘‘it is only the physi- implying afocus ofthis theoryon absoluteness rather cal that is immediately open to measurement,whereas than relativeness. However, he eventually rejected the the measurement ofthe physical canbe obtained only as namechange because offurther confusion hethought dependent on the psychical’’ (p.8). Thus, Fechner boot- it mightentail. (Sachs,1993, p. 4) strapped his programon the sameexistential commit- Any science begins with some assumptions about the fundamentalnature of‘ ‘’’ —whether there is an absolute, objective, reality independent ofthe observer orwhether the factsof reality are only relative to an ob- John M.Flachand server. These assumptions guide our or guesses John G.Holden PsychologyDepartment WrightState University Presence, Vol. 7, No. 1, February 1998, 90–95 Dayton,OH 45435 r 1998by theMassachusetts Institute of Technology j[email protected]

90 PRESENCE:VOLUME7,NUMBER1 Flachand Holden 91

mentas Newton. For the mostpart, experimental psy- insights are not pertinent atthe scale ofhumanbehavior; chology remains committedto Newton’s basis in which toargue that Einstein’s insights have implications for three-dimensional space and time (e.g.,chronometric atomicand cosmic scales ofreality ,but not for reality at analysis) are the absolutes against which to measure be- the ecological scale. For psychology this argumentleads havior and experience. naturally to aview ofthe environment as objects distrib- uted within acontainer with dimensions ofspace and time.The container and the objects within are consid- 3Einstein©sWay ered to have an absolute existence independent fromthe animal studied. For those working in the Fechne- Einstein, along with manyothers (e.g.,Galileo, rian , Newtonian measures becomethe touch- Huygens,Leibniz) questioned the decision to use space stone for reality.Classical measures ofspace and time and time as absolutes. Einstein argued that there was no becomethe defining properties ofastimulus. They de- ‘‘objective’’ test (i.e.,) that could distinguish fine the ‘‘real’’ object. Humanexperience is measured the difference between uniform motion and absolute against this reality.The implication for virtual reality sys- rest. The implication was that space and time were rela- tems is that fidelity is defined in relation to the corre- tive to areference frameand thus could not be the basis spondence between the simulated world and the ‘‘real’’ for bootstrapping ascience committedto realism. Space world asmeasured using the ruler and clockof classical and time could not bethe measure ofreality .These mea- physics. Thus, we have constructs of‘‘real time’’ and sures were not observer independent. The relative na- ‘‘real distance’’ against which we can comparethe ‘‘vir- ture ofspace and time creates adilemma—either relin- tual ’’ and ‘‘virtual distances.’’ quish the commitmentto realism, to ,and maintain space and time asthe basis for measurement, or,find anew basis. Einstein refused to give upthe com- 5Helmholtz©sWay mitmentto realism and chose instead to realign his basis for reality.This commitmentled tothe ofthe Helmholtz (e.g.,1962) chose tobootstrap the speed oflight as the absolute upon which to bootstrap psychological programon biology.Henoted the differ- measures ofreality . ence between the ‘‘distal’’ stimulus, that could bede- Einstein’s choice, obviously,has had revolutionary fined in classic Newtonian terms and the ‘‘proximal’’ implications for our view ofthe physical world. Because stimulus, which could bedefined in terms ofchanges in the speed oflight has dimensions ofspace and time, sensorymechanisms.With the emphasis on Mu¨ller’s these dimensions which in Newton’s programwere or- ‘‘law ofspecific nerve energies,’’ Helmholtz chose bio- thogonal are now viewed as dependent. The speed of logical mechanisms as the basis for objectively defining light provided amappingfrom space to time and vice the stimuli ofexperience. Thus, for those who follow the versa. Sucha mappingwas inconceivable within the Helmhotzian tradition, the structure ofnerves, eyes, Newtonian program.But what are the implications for a ears, muscles, brains, and soonbecomefundamental science ofcognition and behavior? What are the implica- touchstones for understanding the experience ofreality . tions for ascience ofexperience? What are the implica- With this approach,the ‘‘acid’’ test ofatheoryis the tions for technologies such as VRwhose objective is to ability tobe‘‘objectified’’ or instantiated by an underly- shape experiences? ing biological mechanism.This approachleads naturally toconstructs such as ‘‘visual space’’ or ‘‘auditoryspace’’ 4Fechner© sWay where the experience ofspace and time are specific to the biological mechanisms that mediate the experience. Oneresponse is to persevere with the commit- The implication ofthis approachfor virtual reality is that ments ofNewton and Fechner—to argue that Einstein’s fidelity is defined relative to the ability tosimulate the 92 PRESENCE:VOLUME7,NUMBER1

biological mechanisms—the proximal stimulus. Thus, binocular and binaural inputs mightbe considered es- sential to ahigh-fidelity experience ofspace.

6Broadbent©sWay

Broadbent (1971), Miller (1956), and others chose to emphasize principles ofcontrol and communi- cation as the grounds for bootstrapping studies ofexpe- rience. Inthis programthe properties ofcommunication channels and control systems provided the basis against which to measure reality.This foundation allowed the Figure 1. Alternative views of aclosed-loop information processing abstraction ofinvariant principles using information system.The traditional representation (A)treats stimulus andresponse theory,control theory, and signal detection theory.For as distinctand external tothe coordination. The alternative those who have followed Broadbent’s lead, stimuli are representation (B)illustrates Dewey’s intuition thatthe stimulus and defined in terms oftheir statistical properties (e.g.,bits response are coupled within the coordination. per second or probability density functions). With this approach,emphasis was shifted fromthe classical New- tonian dimensions ofspace and time and the biological ton/Fechner’s view ofthe information processing sys- dimensions ofneural activity to dimensions that re- temas contained within an environment. Stimuli and flected the observer’s expectations. The probability or responses are seen asdistinct and are represented asex- likelihood ofan event becamea defining feature against ternal tothe coordination. They are ofthe environment, which to measure the experience ofthat event. This ap- not intrinsic tothe coordination. The representation in proach suggests that information processing rate,sensi- Part Bis identical, in terms ofcontrol theory.However, tivity,bias, and stability mightprovide the best measures the impression created by this representation better cap- offidelity when evaluating virtual reality systems. tures Dewey’s intuitions. Inthis representation, stimuli and responses are intimately coupled and are atthe heart ofthe coordination. This perspective requires ashift 7Dewey©sWay fromproperties ofobjects in the environment (in terms ofspace time orstatistical properties) or fromstatic The reflex arc idea, ascommonlyemployed, is defec- properties ofbiological processing mechanisms to dy- tive in that it assumes sensory stimulus and motorre- namicproperties ofthe relationship that arises fromin- sponse as distinct psychical existences, while in reality teraction with an environment. Dewey’s perspective rep- they are always inside aco-ordination and have their resents afunctionalist approachwhere action and the significance purely in the part they play in maintaining purposes ofthe action play asignificant role in defining and reconstituting the co-ordination. (Dewey,1896/ the stimuli ofexperience. The reality ofexperience is 1972, p. 99) defined relative to the and action capabilities This quote fromDewey reflects an important concern ofthe ‘‘actor.’’ Note that in this functionalist perspective about the way the offeedback has been opera- the humanis no longer considered a‘‘passive’’ observer tionalized within psychology.Figure 1attempts to illus- responding to stimuli, but an active and creative force trate this concern. Part Aofthe figure illustrates the shaping the stimuli ofexperience. The implication of conventional representation offeedback for an informa- Dewey’s intuitions for virtual reality are that the focus tion processing system.This representation retains New- shifts to the coordination between and ac- Flachand Holden 93

tion. The measure offidelity is the degree to which the beginning to be attemptedin what is loosely called sys- simulation captures the richness ofnatural couplings tems theory’’ (1979, p.2). On the other hand,Gibson between perception and action. was clearly dissatisfied with traditional operationaliza- tions ofsystems theory in the study ofhumanperfor- mance.For example,he argued that ‘‘Shannon’s concept 8Gibson©sWay ofinformation applies to telephone hookupsand radio broadcasting in elegant ways but not,I think,to the Gibson shares Fechner’s goal to develop apsycho- firsthand perception ofbeing in-the-world, to what a physics to account for the relation between baby gets when first it opens its eyes. The information and areal physical environment. However, unlike Fech- for perception, unhappily,cannot be defined and mea- ner,Gibson does not accept the Newtonian programas sured as Claude Shannon’s information canbe’ ’ (1979, an appropriate basis for describing the objects ofpercep- p.243). Also, when people responded to Gibson’s con- tion. Heargued that ‘‘the concept ofspace has nothing cept ofa perceptual system as ‘‘nothing but acase of to do with experience. ...Spaceis amyth,a ghost,a feedback’’ hewas discouraged and wrote ‘‘people did fiction for geometers’’ (1979, p.3). He continues, ‘‘The not understand’’ (1979, p.244). Gibson chose to focus ofthree dimensions with three axes for Cartesian onthe problem ofcoordination as the fundamental coordinates was agreat convenience for mathematics. .. problem rather than on the problem ofcommunication but an abstraction that had verylittle to do with actual emphasized in the information processing program.This perception’’ (1979, p.148). With regard to time,Gib- focus is consistent with the functional approachto cog- son wrote, ‘‘The flow ofabstract emptytime, however nitive systems represented by Dewey’s position. useful this concept maybe to the physicist, has no reality There is anotable similarity between Gibson’s way for an animal.W eperceive not time but processes, and Einstein’s way.The speed oflight is afundamental changes,sequences’ ’ (1979, p.12). constraint onaction.This is the touchstone upon which Gibson shared Helmholtz’s appreciation for the con- Einstein bootstrapped his approachto reality.Ina similar straints that biological mechanisms place on perception. way,Gibson has chosen the constraints on action asthe However, whereas Helmholtz focused on the micro- fundamentalbasis for the reality ofexperience. The structure ofsensory and neural processes, Gibson fo- speed oflight bridges space and time in Einstein’s pro- cuses onthe macrostructure ofaction. Gibson was more gram,constraints on action bridges animal and environ- concerned about the biological constraints on action and mentin Gibson’s program.The construct ofaffordance how they shape and limit the functional couplings be- reflects this new fundamentalbasis for reality in the rela- tween animals and environments than upon the biologi- tionship between actor and environment. Itpromises an cal constraints on communication within anervous sys- ‘‘objective’’ scientific programfor approaching . tem.He argued that ‘‘information is not specific to the Inthis programmeaning is not asubjective interpreta- banksof photoreceptors, mechanoreceptors, and che- tion, which is the product ofinformation processing. moreceptors that lie within the organs’’ (1979, p. Rather, meaning canbe objectively specified and mea- 243). sured in terms ofthe constraints on action.Constructs Gibson shares Broadbent’s intuition that the invariant such as Pi numbers,in which dimensionless ratios be- properties ofhuman performance mightrest with tween environmental objects and effectors serveas an higher-order abstract properties ofperception-action objective basis for measurement,reflect this approachto systems as mightbe described bythe theoryofdynamic meaning(e.g., Warren, 1995;Shaw ,Flascher, and Kadar, systems. However, hehad serious misgivings about 1995). Itis not the absolute size ofan object oran abso- whether this systems approachcould be based on statis- lute distance that defines the reality ofan event. Rather tical notions ofinformation. On the one hand,he wrote, it is the size relative to hand size or distance relative to a ‘‘What psychology needs is the kindof thinking that is modeof locomotion. Note while this approachmeasures 94 PRESENCE:VOLUME7,NUMBER1

objects and events ‘‘relative’’ toanactor,it is noless ob- information on how to behave in acoherent, coopera- jective than other physical approaches to measurements. tive fashion.At this level, semantics maycome in. (p. The ‘‘invariants’’ ofreality are considered tobeobjective 23) actor-dependent properties ofevents, not observer-inde- The implication ofGibson’ s approachfor virtual real- pendent properties. ity is afocus onthe coupling between perception and Einstein and Gibson’s programare similar in another action asthe focal point ofdesign. From this perspective respect. Whereas the Newtonian programdescribed the it is the dynamic interplay between visual, acoustic, and world in terms ofinteractions between particles, Einstein tactile feedbackand the actions oflooking around and chose to describe the world in terms offields. Field de- manipulating objects that determines the fidelity ofa scriptions give preeminence toabstract as the simulation. Smets(1995) characterizes this perspective fundamentalbasis ofcausality.Inother words, con- and the implications for design for telepresence in terms straints becomefundamental. This shift fromparticle ofa shift froma focus on properties ofan imageto con- descriptions to field descriptions is clearly reflected in cern for properties offlow,fromfocus on motion paral- Gibson’s constructs such as ‘‘safe field oftravel’ ’ and lax to focus on movement parallax. From this perspec- ‘‘optical flow field.’’ Kugler and Turvey’s (1987) model tive, action takesprecedence. The experience depends oftermite nest building and Kirlik’s (1995) model of more onwhat canbe ‘‘done’’ than on the ofvi- skill in ahelicopter simulation are important examples sual or acoustic images.In this perspective the experi- that illustrate the power offield descriptions. AsFeyn- ence ofspace is determined by the degrees offreedom man(1965) has noted,equations based onNewton’s on action,rather than by the resolution ordimensional- laws (particles), fields, or minimumprinciples are math- ity ofthe displays. This does not meanthat the display of ematically equivalent. However, they are ‘‘completely information is irrelevant, any more than Einstein’s unequivalent when you are trying toguess new laws’’ (p. choices madeNewton’ s calculus irrelevant. Rather, it is a 53). Physics has discovered that field theories have led to question ofprecedence. For Gibson’s way,action takes better guesses about laws of‘‘.’’ Inasimilar vein, precedence. Itfixes the coordinate system against which the ecological approach topsychology is betting that invariant properties ofperception and action can bedis- field theories will lead to better guesses about the laws of covered. ‘‘what matters.’’ For Gibson’s way the reality ofexperience is grounded The emphasis on constraints as fundamentalis also in action.Thus, in the design ofexperiences in virtual reflected in the movementof ecological psychology to- environments the constraints on action takeprecedence ward nonlinear dynamics as alanguage for describing over the constraints on perception. This approachpre- the coordination (e.g.,Beek, 1989; Haken, 1988; Kelso, dicts that the experience ofspace will depend more on 1995;Thelen, 1995). The phase space representations the modeof locomotion than onthe visual and acoustic are images ofthe dynamic constraints oncoordination. images.The reality ofa surface will be in its implications Inthis respect, the following quote fromHaken (1988) for action (e.g.,does it impede locomotion) rather than reflects ashift fromShannon’ s notion ofinformation as in its appearance (e.g.,does it looklike awall). Inthis constraints on transmission rate,to atheoryofinforma- approach,the reality ofexperience is defined relative to tion that more directly relates to the constraints oncoor- functionality,rather than toappearances. dination.

The concept ofinformation is arather subtle one ... information is linked not only with channel capacity 9 Summary or with orders given froma central controller to indi- vidual parts ofa system—it can acquire also the role of As noted in the introduction, aprimarygoal ofthis ‘‘medium’’ towhose existence the individual parts ofa article is to challenge those interested in designing vir- system contribute and fromwhich they obtain specific tual environments toconsider possible foundations for Flachand Holden 95

understanding and measuring the reality ofexperience Fechner,G. (1966). Elementsof psychophysics. (H. E. Adler (i.e.,presence or immersion). These assumptions repre- [Trans.];D. H.Howes &E.G.Boring [Eds.].New York: sent ‘‘first principles’’ for ascience ofexperience. As Holt, Rinehart &Winston. (Originally published in 1860 as such,they are not true orfalse.They provide abasis for Elementeder psychophysik .) our rules ofmeasurement,a touchstone for objectivity. Feynman,R. (1965). The character of physicallaw . Cambridge, But these first principles cannot themselves be proven or MA:MIT Press. falsified. These assumptions either provide productive Gibson, J.J.(1979). The ecological approachto visual percep- platforms for exploration or not. tion. Boston: Houghton Mifflin. Asecondarygoal ofthis correspondence is to present Haken,H. (1988). Information andself-organization. Berlin: Gibson’s ecological perspective for consideration. There Springer-Verlag. is no question that the Fechnerian, Helmhotzian, and Helmholtz, H.von (1962). Treatise on physiological optics. information processing have stimulated im- New York:Dover Press. Kelso, S.(1995). Dynamicpatterns. Cambridge,MA: MIT portant experimental programs and have produced sig- Press. nificant insights with regard tounderstanding Kirlik, A.(1995). Requirements forpsychological models to performance.These are well-accepted perspectives, but support design: Toward ecological taskanalysis. In J.Flach, creativity often depends onanability toshift perspec- P.Hancock,J. Caird,& K.Vicente (Eds.). Globalperspec- tives and to create alternative representations. Alterna- tives on the ecology of human-machinesystems (p.68– 120). tive representations are not right or wrong, despite the Mahwah, NJ:Erlbaum. rancor and territoriality ofacademic debates that imply Kugler, P.N.,&Turvey,M. T.(1987). Information, natural otherwise. They are simply more or less productive with law,and the self-assemblyof rhythmic movement. Mahwah, respect toparticular goals ofunderstanding and applica- NJ:Erlbaum. tion. Itis unlikely that any single perspective orrepre- Miller, G.A.(1956). The magicnumber sevenplus orminus sentation will provide acomplete understanding or will two: Somelimits on our capacityto processinformation. be appropriate for everyapplication. Webelieve that the Psychological Review, 63, 81–97. first principles outlined byGibson for an ecological ap- Sachs,M. (1993). Relativity inour time: From physicsto hu- proach tohumanperformance provide an important manrelations. London: Taylor& Francis. alternative representation that mightenrich our under- Shaw,R.,Flascher, O., &Kadar,E. (1995). Dimensionless standing ofhuman performance. We further believe that invariants forintentional systems:Measuring the fit of ve- this perspective mightmake a particularly productive hicular activities to environmental layout. In J.Flach,P . platformfrom which to launch explorations into applica- Hancock,J. Caird,& K.Vicente (Eds.). Globalperspectives tions ofvirtual reality. on the ecology of human-machinesystems (p.293– 358). Mahwah, NJ:Erlbaum. Smets,G. (1995). Designing fortelepresence: The Delft vir- References tual window system.In P.Hancock,J. Flach,J. Caird,& K. Vicente (Eds.). Localapplications of the ecological approach to Beek,P .(1989). Jugglingdynamics. Amsterdam:Free Univer- human-machinesystems (p.182– 207). Mahwah, NJ: sity Press. Erlbaum. Broadbent, D.E.(1971). Decision andstress. London: Aca- Thelen, E.(1995). Motor development. AmericanPsychologist, demic Press. 50(2), 79–95. Dewey,J.(1972). The reflex arcconcept in psychology.In Warren,W. (1995). Constructing anechoniche. InJ.Flach,P . John Dewey: The early works, 1882–1898. Carbondale,IL: Hancock,J. Caird,& K.Vicente (Eds.) Globalperspectives on Southern Illinois University Press.(Original workpublished the ecology of human-machinesystems (p.210– 237). 1896.) Mahwah, NJ:Erlbaum.