HUMAN COMPUTER INTERACTION (5A)

INTERACTIVE CHAIR 423 SEUNG WOOK KIM, HSIN-HSIEN CHIU

CALLIGRAPHIC BRUSH 433 SHENG KAI TANG, WEN YEN TANG

AN AR-BASED NAVIGATION INTERFACE 441 MEILIN KUO

SMART TOPOLOGY 451 CHIEH-JEN LIN

NOTES ON DESIGNING MULTI-DISPLAY SPACES 461 SHUO-TAO CHIANG, SHENG-FEN CHIEN

INTERACTIVE CHAIR

A Body-Centered Approach to the Design of 3D User Interface

SEUNG WOOK KIM, HSIN-HSIEN CHIU University of California at Berkeley 370 Wurster Hall, Berkeley, CA 94720-1800, USA [email protected]; [email protected]

Abstract. InteractiveChair is proposed as a natural and affordable medium that enables direct control of locomotion in 3D virtual space by using dynamic sitting postures. InteractiveChair detects subtle movements of a chair occupant in real time by measuring relative position and orientation between the user’s body and the chair. While standard user interfaces require users to consciously relate the device manipulation by hand to the action in the virtual space, InteractiveChair directly leverages the correlation between our body and spatial conception to control the avatar’s locomotion. This study determines typical user behaviours on the chair and measures them through sensors that are exquisitely integrated with a typical swivel chair. The study then presents results from a user study to measure the qualitative factors of user experience in controlling virtual locomotion with InteractiveChair.

Keywords. interactive chair, HCI, 3D interface, virtual environment

1. Introduction

The virtual world is becoming an extension of architectural space. However, interacting with 3D content by using standard user interfaces (e.g., keyboard/ mouse, gamepad) is not an easy task for casual users. While the ability to skillfully use them often becomes the major objective to be achieved in 3D games, they do not provide an easy-to-learn and natural method to interactive with 3D virtual worlds, where social interaction and collaboration are more emphasized than competition between users. Particularly, the control of first- person locomotion requires rapid and accurate manipulation of joystick, arrow 424 S. W. KIM, H. H. CHIU keys or mouse by hand, but none of them reflects the way we move our body naturally in the real world. Many of user activities in the virtual world can greatly benefit from a body-centered input device. In this study, InteractiveChair is proposed as a natural and affordable medium that enables direct control of locomotion by using dynamic sitting postures. Like other furniture, chair has been an essential interface between occupants and built environments, and it can successfully continue to serve this role in the virtual world. InteractiveChair detects subtle movements of a chair occupant in real time by measuring relative position and orientation between the user’s body and the chair. Typical movements that InteractiveChair detects include forward and backward leaning, as well as horizontal swiveling (yawing) of the chair. The main goal of this project is to determine typical user behaviours on the chair and measure them through sensors that are integrated with a standard swivel chair.

2. Related Work

There have been many works to develop 3D user interfaces based on our bodily motions. The most direct and natural technique for locomotion in 3D space is to physically walk through it. The HiBall tracking system (Welch et al, 2001) optically tracks a wide area by using a scalable tracking grid on the ceiling. However, this method is spatially limited since the size of the environment must be less than tracking range. The space also needs to be free of obstacles. Alternatively, walking simulation has been used to implement interfaces for locomotion in 3D space. Noma and Miyasato (1998) used a common treadmill to provide a walking motion and feel without translating the users’ body. While this approach removes the limitation on the size of the environment, it suffers from high latency for sudden or sharp turns. More recently, CyberCarpet (CyberWalk Project, 2008) proposed an omni-directional treadmill that allows the user to perform locomotive motion in any direction (Templeman, 1999). The Virtusphere system (2008) also took a similar approach with a 2.6m sphere made of ABS plastic. However, none of these approaches are free from the problem of user fatigue. Researchers empirically realized that users have much lower tolerance for physical motion in a virtual environment than they do in the real world. Relevant approaches have been made to implement the virtual locomotion control without simulating physical walking necessarily. One of the latest studies introduced the use of balance board (Wii Fit by Nintendo Inc.). Bergweiler et al. (Advanced Tangible Interface Lab, 2008) suggested that leaning our body on this board could naturally control the locomotion in the virtual world, while raising INTERACTIVE CHAIR 425 questions regarding user stability as well as unnatural quick turns. 3DV system (2008) also proposed the similar idea, except using a 3D depth camera instead of any sensing board on the bottom, to detect the user’s leaning motion in standing position. Gaze-directed steering has been also considered as a common technique to solve the user fatigue problem (Kessler et al., 2000). It allows the user to move in the direction toward which he/she is looking. The gaze direction is usually obtained from the orientation of a head tracker. TrackIR (2008) implements this idea by using IR camera and retro-reflective markers. Lee (Wii Remote Projects, 2008) also proposed a head-tracking 3D interface by using a commercial game controller Wii Remote. This gaze-directed steering (i.e., head tracking) is easy to understand and control, however its usage limits our natural body movement by rigidly coupling our gaze and body in the virtual space. Since our head movement originally serves many other perceptual functions, it is not natural to couple the gaze direction with the locomotive direction. In the mean time, Tan et al. (1999; 2001; 2002) developed SensingChair that supports multimodal interaction through the user’s torso movement. The main idea of this prototype is to develop a real-time system that feels its occupant through a layer of ‘artificial skin’. Two pressure sensor sheets, made possible with commercial pressure distribution measurement system, were surface- mounted on the seat and the back rest of an office chair to capture various sitting postures. Since the analysis of pressure distribution in this system required heavy computation process for pattern recognition, it was only suitable for static applications such as the force deployment control of vehicle airbag, or the maintenance of healthy postures of users. Tracking transitional postures in real time was not technically feasible.

3. Design and Implementation of InteractiveChair

In contrast to earlier work, InteractiveChair can provide following benefits. First, it does not require users to consciously relate the device manipulation by hand to the action in the virtual space, because it directly leverages the correlation between our body and spatial conception to control the avatar’s locomotion. Secondly, the use of torso for locomotive motions in 3D space can release both hands free to manipulate other devices, so that more intellectual actions (e.g., chatting, gesturing, grabbing or pointing) can be performed concurrently. Thirdly, a user can control the locomotive speed dynamically by leaning forward or backward with a different amount of degrees. Finally, the small amount of torso motion required for InteractiveChair does not quickly increase the fatigue of users, compared to prior 3D locomotion techniques that are based on full-body movement of the user. 426 S. W. KIM, H. H. CHIU

3.1 SPATIAL CORRELATION BETWEEN SITTING POSTURES AND LOCOMOTION IN 3D

Relative position and orientation between torso and thigh determine most of sitting postures. Especially, torso movements generate varied forms of sitting postures, reflecting the occupant’s bodily intention. This allows InteractiveChair to leverage the spatial correlation between dynamic sitting postures and locomotive motions. This project particularly focuses on two types of sitting behaviors – leaning forward/backward and swiveling actions.

3.1.1 Leaning behavior and locomotion

In typical office settings, a person sitting on a chair often leans over the desk to pay special attention on the content of screen or document. To the contrary, the occupant leans against the backrest of chair to relax or relieve attention. In case of standing or walking behaviors, this forward-leaning motion usually initiates, therefore is followed by, the action of “approaching” or “running”. In the same analogy, the back-leaning motion (including the transition from forward-leaning position to upright position) initiates “backing off” or “stopping” action. The proposed system leverages this spatial conception that correlates our sitting posture with locomotion (shown in Figure 1). It provides the user with a direct traveling control in 3D space, where the avatar being controlled would stand still when the user’s torso is in upright position; otherwise the avatar (and the associated virtual camera) should move corresponding to the current leaning direction of user. The system also uses the degree of torso inclination to control the locomotive speed in the current leaning direction. It allows the user to accelerate or decelerate the action of virtual avatar in real time.

Figure 1. Actions of leaning forward / backward are closely related to locomotion INTERACTIVE CHAIR 427

3.1.2 Swiveling and orientation

Torso movements on a swivel chair also involve turning motions. Originally, the swiveling function of a chair is to give a wide angle of spatial access to the user being seated. For the avatar control in 3D space, this swiveling motion on the chair can be translated as either 1) the rotation of entire body orientation (together with the gaze), or 2) the change of gaze direction. (a) Control of body orientation coupled with gaze As the user turns the swivel chair, the camera view in the 3D space follows the swiveling direction. For example, turning the chair toward right will rotate the virtual camera clockwise (from the top view). This is in line with the convention of FPS games, in which the camera rotates toward the direction of mouse movement. In the real world analogy, the swiveling motion on the chair can be understood as the action of turning direction while walking. In this case, the gaze of 3D avatar remains coupled with the body orientation in the virtual world (Figure 2-i).

(i) Body orientation control (ii) Gaze control

Figure 2. Different translation of swiveling motion

(b) Control of gaze decoupled from body orientation Unlike other gaze-directed steering techniques such as the head tracker, the proposed system enables an independent gaze control of 3D avatar while keeping its orientation in place. A swivel chair can leverage the relative movement between head and torso orientations, like we can naturally turn our heads to look around while walking in a constant direction. Using the spatial correlation between the physical body and the virtual world, the system keeps virtual and 428 S. W. KIM, H. H. CHIU physical gazes in line, while the torso orientation directs the avatar’s locomotion (Figure 2-ii). In this project, switching between two modes is simply triggered by a key press, however additional bodily cues can be considered for more natural interaction (e.g., putting elbows on the front-end of the desk or the armrests of the chair can change the direction of gaze only).

3.2 SENSING TRANSITIONAL SITTING POSTURES

To detect transitional sitting postures, two types of sensors were mounted on a swivel chair (shown in Figure 3 and 4). Detecting leaning motion: Distance between the user’s back and the backrest of the chair was measured by using an ultrasonic (40 kHz) range finder (Ping by Parallax Inc.). Ultrasound has been used to image the human body for 50 years, and medically it poses no known risks to human health (Nicholas, 2003). This sensor was installed above the backrest by using a height-adjustable mount made of foam board. Measuring swivel motion: A standard single-turn potentiometer (10K Ohm) was used to detect the swivel angle of the chair as rotated by the user. In order to increase the sensitivity of sensor by 5 times, the chair shaft was enhanced with a cylinder-shaped bucket (diameter: 10 inches), so it could engage with a 2-inch wheel that is fixed to the potentiometer head.

Figure 3. Prototype InteractiveChair Figure 4. Sensor systems and controller INTERACTIVE CHAIR 429

4. Evaluation

The basic goal of user study was to evaluate if InteractiveChair can 1) reduce the difficulty in learning 3D locomotion interface, 2) deliver the sense of spatial orientation (Bowman et al., 1999; Darken et al., 1999), and 3) increase the feeling of virtual presence (Lessiter et al., 2001; Slater, 1999). To measure these qualitative factors of user experience, a post-questionnaire survey was conducted to compare InteractiveChair with two typical interfaces: keyboard + mouse combination and gamepad. Quantitative user performance was not considered because InteractiveChair has an additional control of locomotive speed (ranging from 0.2 to 1.8 in each direction), which may influence the test results in favor of the proposed system over the other interfaces that only allow a constant locomotive speed (0.8). (Shown in Figure 5)

Figure 5. InteractiveChair (left: forward control / right: yaw control)

4.1 PROCEDURE

20 participants were recruited among adults (age>30). Each participant was instructed to travel along a given route (Figure 6) with the interface selected among 3 choices (mouse+keyboard, gamepad or InteractiveChair). In the second experiment, the participant was instructed to repeat the travel along the reverse route, with the second interface.

Figure 6. Traveling route in the user test 430 S. W. KIM, H. H. CHIU

4.2 RESULTS

The questionnaire measured interface easiness (3 items), spatial orientation (3 items), and virtual presence (3 items) between two interface systems: keyboard/mouse vs. InteractiveChair, or gamepad vs. InteractiveChair. It showed that the sense of virtual presence was stronger with InteractiveChair ( t(9) = 3.57, p < .01 ) (Figure 7).

Figure 7. Comparison between Keyboard+Mouse and InteractiveChair

The comparison between gamepad and InteractiveChair showed results that are similar to the first comparison. The feeling of virtual presence was still higher with InteractiveChair ( t(10) = 4.58, p < .01 ) (Figure 8).

Figure 8. Comparison between Gamepad and InteractiveChair INTERACTIVE CHAIR 431

5. Discussion

InteractiveChair currently allows 2DOF (forward/backward shifting and yawing), whereas standard game interfaces usually provide 3DOF for locomotion (forward/backward shifting, left/right shifting and yawing) and another 1DOF for viewing (pitching). InteractiveChair can achieve additional 1DOF (left/right shifting) in many different ways. An IR sensor and reflective markers can be used as a method to obtain additional DOF locomotion. Depth sensor can be also considered as a robust alternative, although it may not be inexpensive solutions necessarily. A webcam can also measure the swiveling (yawing) movement by tracking the direction of optical flow (GestureTek, 2008; Paragios et al., 2005; Wang et al., 2006), but it could result in potential drifting errors. Alternatively, a gyroscope sensor or electric compass can be used for tracking swiveling motion.

6. Future Work

While InteractiveChair can benefit from the increased DOF as mentioned above, the overall experience of 3D user interaction can be enhanced by expanding the concept of body-centered interaction. Most of all, a head tracking method can be integrated with InteractiveChair. Haptic feedback is another element to increase the degree of engagement with the virtual space.

7. Conclusion

This project introduced a body-centered approach to 3D interaction design which preserves the user’s spatial conception and motor intention lost in standard 3D game interfaces. This study examines the correlations between human sitting posture and locomotion behavior, and applies them in the design of InteractiveChair. By integrating inexpensive sensors with a standard swivel chair, the proposed system provides a natural method to interact with 3D virtual space. First, this system shows that the torso leaning motion on the chair can dynamically change the locomotion speed of avatar. The correlation between sitting posture and spatial conception supports the natural transition from body movement to avatar locomotion. Secondly, the use of swiveling function of chair immediately delivers the body intention to the control of avatar orientation, which can be translated either as the whole body orientation or as the gaze direction. In the user study, participants evaluated that both the keyboard-mouse configuration and the gamepad were relatively difficult to use for 3D game application, while InteractiveChair helped them learning and controlling 3D locomotion more easily. For overall qualitative user experience, there was measurable difference in 3D locomotion behavior between groups using standard user interfaces and groups using the InteractiveChair. 432 S. W. KIM, H. H. CHIU

References

3DV Systems: 2008, http://www.3dvsystems.com/ Advanced Tangible Interface Lab: 2008, http://www.dfki.de/iui/advanti/lab/index.html Bowman, D., Davis, E. T., Hodge, L. F. and Badre: 1999, A.N. Maintaining Spatial Orientation during Travel in an Immersive Virtual Environment. Presence: Teleoperators and Virtual Environments, 8(6): 618-631 CyberWalk Project: 2008, http://www.cyberwalk-project.org/ Darken, R.P., Allard, T. and Achille, L.B.: 1999, Spatial Orientation and Wayfinding in Large- Scale Virtual Space II. Presence: Teleoperators and Virtual Environments, 8(6):3-6 GestureTek: 2008, http://www.gesturetekmobile.com/ Kessler, G., Bowman, D. and Hodges, L.: 2000, The Simple Virtual Environment Library: An Extensible Framework for Building VE Applications. Presence: Teleoperators and Virtual Environments, 9(2): 187-208. Lessiter, J., Freeman, J., Keogh, E. and Davidoff, J.: 2001, A Cross-Media Presence Questionnaire. Presence: Teleoperators and Virtual Environments, 10(3): 282-297 Nicholas H.: 2003, AAPM/RSNA Physics Tutorial for Residents: Topics in US: B-mode US: Basic Concepts and New Technology, RadioGraphics, Vol.23, 1019 - 1033. Noma, H. and Miyasato T.: 1998, Design for Locomotion Interface in a Large Scale Virtual Environment-ATLAS: ATR Locomotion Interface for Active Self Motion. ASME-DSC , 64: 111-118. Paragios, N., Chen., Y., and Faugeras., O.: 2005, Mathematical Models in Computer Vision. Springer, Chapter 15, 239-258. Slater, M.: 1999, Measuring Presence: A Response to the Witmer and Singer Presence Questionnaire. Presence: Teleoperators and Virtual Environments, 8(5): 560-565 Tan, H. Z., Ebert, D. S., Slivovsky, L. A., Svakhinem, N, and Leimgruber III, J.: 2002, sensingChair. In SIGGRAPH 2002 Conference Abstracts and Applications, 135 Tan, H. Z., Slivovsky, L. Z. and Pentland A.: 2001, A Sensing Chair Using Pressure Distribution Sensors. Transactions on Mechatronics. IEEE/ASME, Vol. 6, No. 3. Tan, H. Z.: 1999, A sensing chair, In Proceedings of the Eighth (8th) International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, American Society of Mechanical Engineers Dynamic Systems and Control Division, 67: 313-317. Templeman, J.: 1999, Virtual Locomotion: Walking in Place through Virtual Environments. Presence: Teleoperators and Virtual Environments, 8(6): 598-517 TrackIR: 2008, http://www.naturalpoint.com/trackir/ Virtusphere: 2008, http://www.virtusphere.com/ Wang, J., Zhai, S. and Canny, J.: 2006, Camera Phone Based Motion Sensing: Interaction Techniques, Applications, and Performance Study. In Proceedings of UIST, 101-110. Welch, G., Vicci, L., Brumback, S., Keller, K., and Colucci, D.: 2001, High-Performance Wide- Area Optical Tracking: The HiBall Tracking System. Presence: Teleoperators and Virtual Environments, 10(1):1-21 Wii Remote Projects: 2008, http://www.cs.cmu.edu/~johnny/projects/wii/