2021 IEEE Virtual Reality and 3D User Interfaces (VR)

Larger Step Faster Speed: Investigating Gesture-Amplitude-based Locomotion in Place with Different Virtual Walking Speed in Virtual Reality

Pingchuan Ke* Kening Zhu† School of Creative Media School of Creative Media and City University of Hong Kong Shenzhen Research Institute City University of Hong Kong

ABSTRACT walking travel in VR, how to control the virtual walking speed to In this paper, we present a series of user studies to investigate the reach the far-away target quickly and precisely remains to be solved. technique of gesture-amplitude-based walking-speed control for loco- For the limb-based gestures (i.e., arm swinging and foot stepping), motion in place (LIP) in virtual reality (VR). Our 1st study suggested existing works usually utilized the swinging/stepping frequency [54] that compared to tapping and goose-stepping, the gesture of marching or the amplitude of the foot/arm movement [6] to control the walking in place was significantly preferred by users across three different speed. Torso-leaning-based approach is another LIP option where the virtual walking speed (i.e., 1×, 3×, and 10×) while sitting and stand- angle of torso leaning could be adopted for speed control [22]. While ing, and it yielded larger motion difference across the three speed the leaning posture may provide a similar vestibular experience of levels. With the tracker data recorded in the 1st study, we trained a real-world walking [20], it is often used in specific scenarios with Support-Vector-Machine classification model for LIP speed control continuous movement, such as flying [20, 28, 60], dragon riding [42], based on users’ leg/foot gestures in marching in place. The overall and so on, rather than step-by-step walking. In addition, research accuracy for classifying three speed levels was above 90% for sitting showed that leaning-based locomotion may suffer from low accuracy and standing. With the classification model, we then compared the and speed [15]. Considering the step-by-step walking in place in VR, marching-in-place speed-control technique with the controller-based existing research shows that compared with the step-frequency-based teleportation approach on a target-reaching task where users were speed-control method, the step-amplitude-based approach could sitting and standing. We found no significant difference between the improve the effectiveness and the precision in the short-distance two conditions in terms of target-reaching accuracy. More impor- travel, and offer better speed control in the medium- and the long- tantly, the technique of marching in place yielded significantly higher range travel [6]. While both step-frequency- and -amplitude-based user ratings in terms of naturalness, realness, and engagement than speed-control approaches have been proposed for LIP in VR, the the controller-based teleportation did. maximum speed of these techniques was usually around 2 times of normal walking speed. Higher virtual walking speed could Index Terms: H.5 [INFORMATION INTERFACES AND PRE- potentially facilitate the large virtual world. At present, there is few SENTATION (e.g., HCI)]: Multimedia Information Systems— research on the method of switching across a wide range of the virtual Artificial, augmented, and virtual realities walking speed (e.g., from 1× to 10× or even higher) for LIP in VR. In this paper, we focus on the foot/leg-gesture-based LIP 1 INTRODUCTION techniques leveraging the footstep amplitude to control wide range Spatial locomotion/navigation is one of the essential tasks in immer- of virtual walking speed, from a normal walking speed to a much sive virtual reality (VR) [35]. With the explosion of immersive VR in higher speed level. In Study 1, we investigated how users may the consumer market, a number of hardware and software solutions perform their foot/leg-based gestures/movements to match the virtual have been developed to improve the efficiency and the experience of walking speed while sitting and standing. This was mainly to find spatial locomotion in VR. Locomotion control can rely on either using out the foot/leg gesture that is preferred by the users for matching physical peripheral devices (e.g. handheld controllers or joysticks) or with different walking speeds in VR (i.e., 1×, 3×, and 10× of the gesture-based interaction using the motion of body parts (e.g., physi- normal walking speed). The results showed that marching in place cal walking, and walking in place - WIP). WIP is a popular locomotion (MIP) received the highest ratings from the participants, and it technique that enables users to perform virtual locomotion by perform- yielded more distinguishable motion features for gesture-based gain ing a stepping-like gesture without physically moving around [34, 44, control. With the tracker data recorded in Study 1, we trained two 48]. Compared to using external handheld controllers/joysticks, WIP Support-Vector-Machine models for three-level speed classification allows for better spatial awareness [23] while approximating the body in standing and sitting respectively, to support the automatic speed motion of real-world walking [40], thus offering the proprioceptive switching based on the footstep amplitude. At the current stage, we feedback to reduce the motion sickness [18]. Unlike unrestricted real- focused on the discrete speed classification/selection instead of the world physical walking, which may require a large physical space to continuous speed estimation, as previous research showed that the match the VR space or enable walking redirection [38], the WIP tech- discrete speed selection in VR locomotion could potentially reduce nique allows users to perform the VR locomotion in small physical ar- the motion sickness [10,51]. Lastly, we conducted the user study to eas. Researchers also proposed other locomotion-in-place techniques, compare the effectiveness of virtual target reaching using three tech- such as leaning, which require even less body movement [25, 28, 53]. niques: controller-based teleport (Tele), MIP with controller-based In addition, researchers proposed the VR locomotion technique of speed control (cMIP), MIP with gesture-based speed control (gMIP) arm swinging without the need of foot motion [27, 31, 37]. However, supported by the classification model. The results showed that gMIP the requirement of arm/hand movement for LIP in VR may affect the achieved a comparable target-reaching accuracy as Tele did, and was simultaneous hand-based tasks, such as object manipulation [13, 52]. rated to be significantly more natural, real, and engaging. While a user is performing the gestures/postures for locomotion The paper makes contributions in three folds: in place (LIP), one of the challenges could be the ability to naturally control the virtual walking speed. For instance, in a long-distance • We investigated the users’ behavior and preference towards *[email protected] gesture-amplitude-based speed control for LIP in VR. †Corresponding author: [email protected] • We experimented and developed the data-driven gesture-amplitude- based speed-control models for LIP in VR. • We evaluated the effectiveness of gesture-amplitude-based LIP speed control for a target-reaching task in VR.

2642-5254/21/$31.00 ©2021 IEEE 429 DOI 10.1109/VR50410.2021.00067 2 RELATED WORK Translational Gain [55]. Bolte et al. proposed the Jumper Metaphor The presented research is highly inspired by the existing works on which combines natural direct walking with teleport through locomotion in place (LIP) and walking-speed control in VR. large-scale virtual environments [3]. By predicting the user’s walking direction and target location, the Jumper Metaphor directly translate 2.1 Locomotion in Place in VR the user to the target in a high speed with a fast and blurred animation of movement. Wilson et al. found that in an interactive room-scale Researchers have proposed various types of LIP techniques, virtual environment, when using the large virtual walking speed, including button trigger in the physical controllers (e.g., teleport and positional accuracy diminishes at speed gains beyond 2× [56]. joystick), stepping in place (e.g., marching and tapping in place), arm Abtahi et al. investigated the speed-gain-visualization methods on swinging in place, body leaning, and so on. Buttussi and Chittaro the gain levels of 1×, 3×, and 10× for real walking in VR [1]. provide a thorough literature review on various types on locomotion Enlarging the translational speed may induce motion sickness techniques in VR [7]. The common movement trigger was pressing while using VR. To address this issue, Interrante et al. proposed Seven the direction buttons in the handheld controllers or joysticks [41]. League Boots [17], which amplifies movements along the users’ However, this may introduce serious motion sickness in VR due to walking direction, without modifying the sideways movements that the mismatching between the still body in real and the moving visual naturally occur when walking. Interrante et al. have also shown that content in VR. To address this problem, the locomotion technique users maintain better spatial awareness when using Seven-League of Point&Teleport was proposed to reduce the visual motion and Boots, compared to flying with a magic wand [16]. Moreover, they support point&click-based instant locomotion in VR [4]. To support have found that users overwhelmingly prefer Seven-League Boots hand-free locomotion in VR, foot-based teleportation was also to 2D Translational Gain [17]. Later, Bhandari et al. introduced proposed and studied [52]. However, existing research showed that Legomotion to allow users switch between real walking and walking the Point&Teleport technique still may not provide as much presence in place to enable navigation at scale in VR [2]. as body-motion-based locomotion techniques [8]. In certain contexts, While above-mentioned existing work focused on different the Point&Teleport technique may also negatively affect the time presentation/visualization techniques for different speed levels, they spent on target reaching, especially with obstacles on the way [5]. mostly studied the user experience and performance under one fixed To increase the naturalness of locomotion in place, researchers speed level at a time. As in the actual VR interaction users may want have proposed and experimented gesture/posture-based LIP tech- to control the virtual walking speed by themselves, there is a need niques, such as arm swinging [37], foot stepping [11,39,44,48–50,54], to facilitate the smooth speed-switching/-controlling process in real foot cyclying while sitting [12], and body leaning for both sitting and walking and locomotion in place. In addition, while the techniques standing [28]. These gesture-based LIP methods aimed to mimick of enlarging translational speed were usually applied for real walking parts of the limb or torso motion and the vestibular experience of in VR [29], the potential of applying large virtual walking speed to real-world walking, such as left and right limbs (e.g., arm and leg) locomotion in place is still unexplored. In the presented studies, we moving alternatively, and body leaning forward. Experiments showed aim to fill these gaps by investigating the foot/leg-motion-based LIP that foot-motion-based LIP could yield better spatial awareness techniques for speed control. than the arm-swinging method [57]. In addition, the arm-swinging locomotion technique may not be suitable for the scenario involving 3 SELECTED LOCOMOTION TECHNIQUES simultaneous hand-based interaction (e.g., grasping/manipulating In this paper, we focus on the foot/leg-gesture-based locomotion in virtual objects) while users are walking [13, 52]. Researchers also place. By reviewing the existing related literature, we selected three proposed the method of torso leaning [28] for locomotion in place. As types of foot/leg gestures. the user lean his/her body forward, he/she will move forward along the head direction, and the speed could be controlled by the angle of 3.1 Marching leaning [22]. Compared to limb-based gestures, body-leaning could Marching (Fig. 3(a)&(b)) was a common foot/leg gesture adopted reduce the physical workload [28], but may lead to longer response for locomotion in place [11, 48, 54]. With this gesture, the user time for the users [15]. alternately lifts each leg as if climbing a flight of stairs or marching In this presented research, we focused on the foot/leg-gesture- on the spot. Marching could provide the proprioceptive experience based LIP techniques in both standing and sitting. Specifically, we which is similar to one induced by real walking [43]. focused on the gestures of marching, tapping, and goose stepping in place, and studied their effects on the user behavior, performance, 3.2 Tapping and preference under different levels of virtual walking speed. The foot-tapping gesture (Fig. 3(c)&(d)) was proposed by Nilsson et al. [30]. It is a gesture where movement is generated by tapping each 2.2 Walking Speed in VR heel against the ground. The initial swing is now replaced by the user Besides locomotion in place, researchers also investigated applying lifting one heel off the ground without breaking contact with the toes different levels of virtual walking speed in VR to allow users to nav- and the terminal swing corresponds to lowering the heel again. Com- igate in a VR space which is larger than the physical space. Most pared to the marching gesture, tapping requires less physical move- existing research on limb-gesture-based locomotion in place calcu- ment. Nilsson et al. showed that the tapping gesture was significantly lated the moving speed in VR linearly from the amplitude of the limb less tiring, and yielded sigificantly less drift than marching [30]. movement [22]. This approach usually results in a limited range of vir- tual walking speed (i.e., between 1 to 1.75 times of the normal walking 3.3 Goose-Stepping speed). For higher virtual walking speed for walking in place, Nilsson The third leg gesture resembles the walking movement when the user et al. showed that the gesture of walking in place in a normal speed can goose-stepping in place (Fig. 3(e)&(f)). The goose step is a special afford a range of translational gain between 1.5 and 2.5 while main- step performed on formal parades and other ceremonies. While goose taining the naturalness of walking in VR [32]. In addition, their exper- stepping, the user swings his/her legs off the ground while keeping iments revealed that by increasing the step frequency, the range of nat- each leg straight. Goose-stepping may involve more leg muscle and ural visual speed gain can be expanded (e.g., between 2.50 and 3.00) larger motion than the other two selected gestures (i.e., normal walk- for walking in place [33]. In this paper, we further push the boundary ing/marching and tapping) [59]. Existing research on hand-gesture- of speed gain for LIP in VR, to investigate how users may match their based human-computer interaction showed that users preferred a LIP gestures with a wide range of walking speed for LIP in VR. larger range of hand motion for stabler and more accurate control [26, The virtual walk speed which is much higher than the normal 45, 46]. To this end, we hypothesized that goose-stepping with larger walking speed received more research attention for real walking foot/leg motion may provide a larger range of movement that can be in VR. As one of the early research in large virtual walking speed, reliably mapped to different levels of virtual walking speed, compared Williams et al. have studied speed gains up to 10× using 2D to the other two types of gestures (i.e., marching and tapping).

430 4 STUDY 1: UNDERSTANDING USER FOOT/LEG BEHAV- sign the consent form voluntarily. The facilitator then explained IORSFOR DIFFERENT VIRTUAL WALKING SPEED the flow of the study, introduced the three locomotion gestures, and In the first user experiment, we investigated how users may presented the think-aloud protocol to encourage the participant to perform and perceive the selected foot/leg-based LIP gestures verbally describe his/her experience during the study. for matching with different visual speed in VR. This was to The study for each participant consists of six blocks corresponding determine the potential technique which is preferred by the users to six combinations of body pose (sitting or standing) and foot/leg and can be feasibly implemented for LIP speed control in VR. We gesture (marching, tapping, or goose-stepping). In each block, the performed a comparative study using a within-subjects design facilitator first demonstrated the specific leg/foot gesture, and guided including three selected foot/leg gestures and two body postures the participant to practice the gesture until the participant felt comfort- (i.e., standing and sitting), resulting in 3×2 = 6 conditions. That able doing so. The participant was then asked to perform the leg/foot is, as shown in Fig. 3, Standing-Marching (StM), Standing-Tapping gesture to match the visual speed in the virtual world. The speed- (StT), Standing-GooseStepping (StG), Sitting-Marching (SiM), matching task was similar to the ones in the previous WIP-related Sitting-Tapping (SiT), and Sitting-GooseStepping (SiG). The order works [32,33]. While the participants in the previous works needed to of the conditions was counter-balanced in Latin square. match the speed of the treadmill with the visual speed, our participants were asked to perform the LIP gestures for speed matching. We have 4.1 Participants selected three types of gains: 1×, 3×, and 10×. The walking speed in VR of 1× was calculated as HeightO fUser/0.45 [30]. Following Twelve participants (5 females and 7 males) were recruited from the previous work on natural walking in VR [1], the participant first a local university. The average age was 24.8 years old (SD = 4.30). matched the gain of 1×, then 3×, and lastly 10×. For each speed The average height was 172.3cm (SD = 2.36). Seven were in the level, the participant was instructed to take his/her time to explore area of art and design, and five were from computer science. Three and match his/her gesture. Once the participant reported that he/she participants reported that they have heard of VR but never tried it was satisfied with the matching between his/her gesture and the gain, before, and the rest self-reported that they have played VR previously. the participant was asked to perform the gesture to walk in the virtual 4.2 Apparatus and Instruments world for 10s. During the 10s walking, the system recorded the 3D positions of the trackers at their highest positions. The questionnaire Fig. 1 shows the set up of the study environment. The experiment was items of general discomfort, fatigue, dizziness, and nausea from the conducted in a 5m×5m space in a local research lab. We developed Simulation Sickness Questionnaire (SSQ) [19] was used before and a VR prototype using Unity3D 2019 and HTC Vive Pro bundles. after each block. Disorientation was not included, because the partic- The prototype showed the virtual scenario of a town (Fig. 2). The ipant’s disorientation score might not contribute much to his/her SS participants wore one HTC Vive tracker right above each of their score since he/she were not asked to make any turns. In the end of each ankles. We adopted the step-detection mechanism in the previous block, the participant also filled in the post-block questionnaire con- WIP works [6, 30], by triggering the forward movement and the taining 10 items, shown in Table 1, in a Likert scale from 1 (strongly stepping sound at the maximum height of the tracker. disagree) to 7 (strongly agree). All the questionnaire items were translated to the participant’s native language, to smoothen the rating process. The study session for each participant took around 1.5 hours. 4.4 Results The goal of Study 1 was to select the potentially suitable gesture for locomotion in place while standing and sitting respectively. Therefore, we conducted the statistical analysis on the questionnaire responses and the tracker data for standing and sitting separately. We defined the suitable gesture as the gesture 1) that received significantly better questionnaire ratings (i.e., user preference), and 2) in which the tracker data were more distinguishable across the three gain levels (i.e., detectability for gain control). 4.4.1 Questionnaire Responses Table 1 summarizes the descriptive results of the questionnaire Figure 1: Setup of Study 1. response in Study 1. For the standing pose, Friedman Tests showed that the type of LIP gesture significantly affected the participants’ rating on the naturalness (χ2(2) = 12.57, p < 0.005), the realness (χ2(2) = 12.50, p < 0.005), the engagement (χ2(2) = 9.05, p < 0.05), the likeness (χ2(2) = 8.36, p < 0.05), the physical demand (χ2(2) = 8.21, p < 0.05), the comfort (χ2(2) = 7.80, p < 0.05), and the control- lability (χ2(2) = 7.58, p < 0.05). Overall the marching gesture was rated higher than the other two gestures. Post-hoc pairwise Wilcoxon Signed Ranks Test showed that StM was rated significantly more natural than the other two (StT: Z = 2.16, p < 0.05; StG: Z = 3.08, p < 0.005). Marching was also rated significantly more real than the other two (StT: Z = 2.15, p < 0.05; StG: Z = 2.68, p < 0.01). In addition, StM was rated significantly higher than StG in terms of engagement (Z = 2.54, p < 0.05), likeness (Z = 2.69, p < 0.01), the comfort (Z = 2.68, Figure 2: The virtual-world prototype used in Study 1. p < 0.01), and the controllability (Z = 2.32, p < 0.05). In terms of physical demand, StM received significantly lower rating than StG (Z 4.3 Procedure = 2.39, p < 0.05), while there was no significantly difference between There were one facilitator and one participant in each study session. StM and StT in these aspects. While marching was significantly more Upon the arrival of a participant, the experiment facilitator introduced preferred by the participants for standing, Friedman Tests showed the purpose of the study, and guided the participant to fill the pre-study that for sitting, the type of LIP gesture only had a significant effect questionnaire for his/her anonymous biographic information, and on the participants’ rating of engagement (χ2(2) = 7.53, p < 0.05).

431 Questionnaire Item StG Mean StT Mean StM Mean SiG Mean SiT Mean SiM Mean I like using this technique for walking in VR. 3.29 (1.54) 4.43 (1.91) 4.71 (1.33) 4.79 (1.53) 4.71 (1.73) 4.83 (1.59) It is comfortable to use this technique for walking in VR. 3.14 (1.75) 4.43 (1.95) 4.64 (1.28) 4.43 (1.7) 4.93 (1.82) 4.96 (1.66) It is engaging to using this technique for walking in VR. 3.21 (1.63) 4.36 (1.74) 4.86 (1.35) 3.79 (1.65) 4.36 (1.85) 4.79 (1.89) It is natural to use this gesture for walking in VR. 3.29 (1.49) 4.57 (1.79) 5.64 (0.93) 4.14 (1.91) 4.36 (1.34) 4.64 (1.75) The VR experience is consistent with my real-world experience. 3.43 (1.5) 4.07 (1.38) 5.07 (1.33) 4.07 (1.76) 4.36 (1.45) 4.41 (1.64) The walking speed in VR has an influence on my gesture. 5.43 (1.02) 4.79 (1.48) 5.64 (0.93) 5.50 (0.94) 5.21 (1.37) 5.43 (1.34) It is easy to match my gesture with the walking speed in VR. 4.79 (1.35) 5.21 (1.54) 5.21 (1.28) 5.57 (1.17) 5.07 (1.35) 5.57 (1.49) It is easy to control the walking movement in VR using this gesture. 4.36 (1.45) 5.00 (1.11) 5.64 (1.15) 5.29 (1.33) 4.93 (1.18) 5.00 (1.44) It is mentally demanding to use this technique. 2.21 (1.05) 2.00 (0.88) 1.86 (0.77) 2.36 (1.39) 2.71 (2.20) 2.14 (1.03) It is physically demanding to use this technique. 4.93 (2.56) 3.07 (2.46) 3.43 (1.6) 4.00 (2.11) 3.50 (1.70) 3.93 (2.40)

Table 1: Average questionnaire responses in Study 1. The numbers within the brackets are the standard deviations. The cells highlighted in green receive highest ratings within the poses of standing and sitting. StG StM StT SiG SiM SiT Before After Before After Before After Before After Before After Before After General Discomfort 0.06 0.13 0.00 0.13 0.00 0.06 0.00 0.13 0.00 0.06 0.10 0.14 Fatigue 0.24 0.33 0.16 0.25 0.11 0.19 0.22 0.19 0.17 0.31 0.35 0.21 Dizzy 0.06 0.13 0.05 0.19 0.00 0.13 0.06 0.06 0.00 0.06 0.30 0.07 Nausea 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

Table 2: Average ratings on the selected sickness symptoms in Study 1.

2 Post-hoc pairwise Wilcoxon Signed Ranks Test showed that SiM In addition, the speed level revealed the largest effect size (ηp) on was rated significantly more engaging than SiG (Z = 2.13, p < 0.05), the tracker data for StM, among the three LIP gestures in standing, while there was no significantly difference between SiM and SiT. which indicates that the tracker positions were more varied among For the change in the selected motion-sickness symptoms before the three gains in StM than the other two gestures in standing. This and after each block (Table 2), there was no significant effect of any further implies that it could be potentially easier to distinguish the independent factor. gesture amplitude of StM for LIP gain control. For sitting, the questionnaire responses revealed less difference 4.4.2 Tracker Data among the three LIP gestures than those in standing. The significant Besides from the subjective questionnaire responses, we also difference was only found on the ratings of engagement between SiM examined the tracker data recorded while the participants performing and SiG. There was no significant difference between SiM and SiT the three different gestures in sitting and standing. Taking the speed for the ratings of all questionnaire items, with SiM slightly higher level (1 or 3 or 10) as the independent variables, we performed the in likeness, comfort, engagement, naturalness, and realness. Looking repeated-measures ANOVA(RMANOVA)on the trackers’ position at the tracker data while sitting, the speed level had the largest effect data on X, Y, and Z axis for each LIP gesture in either standing or size on the Y and the Z axis for SiM, indicating the potentially more sitting. The X, Y, and Z axis represent the directions of left/right, reliable gesture-based gain control. While we hypothesized that the up/down, and forward/backward respectively, as shown in the goose-stepping gesture may yield a more distinguishable tracker data coordinate system in Fig. 1. The position data in each axis were due to its range of motion, it was not reflected in the actual data. This normalized across the three gain levels before the statistical analysis. could be due to the higher physical demand of using the goose step In general, the tracker data on Y and Z axis increased in magnitude which placed difficulties in maintaining the motion amplitude. from 1× to 3×, and from 3× to 10×. This implies that the To this end, we selected marching with both standing and sitting participants’ movement amplitude increased as the gain increased. poses (i.e., StM and SiM highlighted in green in Table 3), for further Fig. 3 shows the examples of representative gesture amplitudes investigation on the effectiveness of gesture-based LIP speed control. performed by the participants to match different gain levels across different poses and gestures. The RMANOVA results showed that 5 STUDY 2: GESTURE-AMPLITUDE-BASED GAIN CLASSIFI- the gain level significantly influenced the tracker data on Y and Z axis CATION FOR LIP WITH SUPERVISED MACHINE LEARNING in all six conditions. Table 3 shows detailed results of RMANOVA Study 1 showed that the participants’ gesture amplitudes on the Y and the post-hoc pairwise comparisons. and the Z axis were significantly affected by the visual translational speed in VR. Therefore, we further explored the feasibility of speed control or gesture-amplitude classification using machine-learning techniques. Each speed level was regarded as a class, and the speed control across three levels (i.e., 1×, 3×, and 10×) can be considered as a 3-class classification problem for StM and SiM individually. The Y-axis and the Z-axis data of the two trackers (i.e., left and right) for StM and SiM in Study 1 were used for the training, the validation, and the testing of the machine-learning models. Study 1 recorded 684 data entries for StM, with 228 for 1×, 228 for 3×, and 228 for 10×. 588 data entries (i.e., 196 1×, 196 3×, and 196 10×) were recorded for SiM. Noted that the unequal sample sizes of StM and SiM could be due to the difference between the action duration in standing and sitting, as we observed that the gestural action generally took shorter Figure 3: Examples of LIP gestures and amplitudes for different gain time to perform and thus generated less tracker data. For each pose levels. (i.e., standing or sitting), the data were normalized across the three gain levels within each participant. Fig. 4(a) and (b) illustrate the data sets for StM and SiM respectively. With these two data sets, two 4.5 Summary of Study 1 machine-learning experiments were done for StM and SiM separately. According to the questionnaire responses, StM was significantly First, we determined the optimal classification method and more preferred by the participants than the other two for standing. the features to be used. We considered four commonly used

432 Y Z X RMANOVA Pairwise Comparison RMANOVA Pairwise Comparison RMANOVA F(2,22) = 51.92, p < 0.0005, F(2,22) = 26.37, p < 0.0005, StM 2 10× > 3×, 10× > 1×, 3× > 1× 2 10× > 3×, 10× > 1×, 3× > 1× ηp = 0.825 ηp = 0.706 F(2,22) = 13.51, p < 0.0005, F(2,22) = 25.27, p < 0.0005, StT 2 10× > 3×, 10× > 1×, 3× > 1× 2 10× > 3×, 10× > 1×, 3× > 1× ηp = 0.551 ηp = 0.697 F(2,22) = 6.84, p < 0.005, F(2,22) = 5.07, p < 0.05, StG 2 10× ∼ 3×, 10× > 1×, 3× > 1× 2 10× ∼ 3×, 10× > 1×, 3× > 1× ηp = 0.383 ηp = 0.315 F(2,22) = 12.50, p < 0.0005, F(2,22) = 72.54, p < 0.0005, No significant effect of the gain level SiM 2 10× > 3×, 10× > 1×, 3× ∼ 1× 2 10× > 3×, 10× > 1×, 3× > 1× ηp = 0.532 ηp = 0.868 F(2,22) = 12.48, p < 0.0005, F(2,22) = 28.27, p < 0.0005, SiT 2 10× > 3×, 10× > 1×, 3× ∼ 1× 2 10× > 3×, 10× > 1×, 3× ∼ 1× ηp = 0.530 ηp = 0.720 F(2,22) = 12.32, p < 0.0005, F(2,22) = 39.42, p < 0.0005, SiG 2 10× > 3×, 10× > 1×, 3× ∼ 1× 2 10× > 3×, 10× > 1×, 3× > 1× ηp = 0.528 ηp = 0.782

Table 3: Details of RMANOVA results on the tracker data. The > in the “Pairwise Comparison” indicates the significant different with p < 0.05, and the ∼ indicates non-significant difference.

The two prediction models were exported for Fig. 5(a) and (b) show the confusion matrices of the SVM models for classifying the three gain levels in StM and SiM respectively.

6 STUDY 3: INVESTIGATING THE EFFECTIVENESS OF GESTURE-AMPLITUDE-BASED GAIN CONTROLFOR LIP With the SVM models trained for three-level gain classification, we conducted the user study to investigate the effectiveness of using gesture-based gain control for LIP in VR. More specifically, we Figure 4: Data-set visualization for (a) StM and (b) SiM. compared the user performance and experience of using 1) controller- based teleportation (denoted as Tele), 2) controller-based gain control 1x 3x 10x 1x 3x 10x with marching in place (denoted as cMIP) (C), and 3) gesture-based gain control with marching in place (denoted as gMIP), for the task 1x 100% 1x 93.98% 6.02% of target reaching in VR. Tele was chosen as the baseline condition as it is the common LIP technique supported by many VR applications.

3x 1.02% 92.86% 6.12% 3x 4% 86.67% 9.33% 6.1 Participants Twelve participants (3 females and 9 males) were recruited from a local university. The average age was 25.34 years old (SD = 3.99). 10x 4.44% 95.56% 10x 2.44% 6.10% 91.46% The average height was 170.9cm (SD = 5.37). Six were in the area of art and design, and the rest were from computer science. Two (a) StM (b) SiM participants reported that they have heard of VR but never tried it before, seven played with VR a few times, and the rest self-reported Figure 5: The confusion matrices of SVM on gesture-amplitude- that they have played with VR a lot. based speed classification: (a) StM, (b) SiM. Rows represent the ground-truth labels, and columns are the predicted labels. 6.2 Apparatus and Instruments machine-learning methods, including support-vector machine Study 3 was conducted in the same location as Study 1, and used a (SVM) [36], k-nearest neighbors (KNN) [9], logistic regression similar VR prototype. Among the selected LIP technique, Tele, as (LR) [58], and random forest (RF) [47]. The Y and the Z coordinates shown in Fig. 6-left, was based on the 6-DOF tracking of the Vive were used as two individual features, resulting in three possible handheld controller and on the touchpad being pressed as a button. feature sets: A) Y only, B) Z only, and C) Y and Z. The performance When users pressed and held the button, a parabola was generated of each classification method may vary with different training from the controller with the position highlighted on the ground. features used. To obtain the optimal feature set for each method, Users could tilt the controller up and down to continuously adjust we compared the performance of each feature set on all the four the position on the ground with the parabola, and release the button methods. For the model training and testing, we adopted the strategy to be teleported to the indicated position. In our study, we adopted of leave-one-user-out cross validation due to its lower bias compared the maximum teleportation distance in the virtual world as 10 times to K-fold cross validation (K<12) [21,24]. That is, in each iteration of the participant’s normal walking speed which was calculated in of training and testing, we used 11 users’ data for training, and 1 the same way as Study 1, which matched our highest gain level of 10. user’s data for testing. The average testing accuracy was calculated For cMIP and gMIP, the VR prototype adopted the same walking after twelve training iterations. Validating the four models with the mechanism in Study 1 which triggered walking at the peak of the three feature sets, it was found that for StM, feature set C (i.e. Y and tracker positions. We only adjusted the virtual moving speed in the Z) gave the best performance for SVM (97.29%) and KNN (96.74%), forward direction, to minimize the motion sickness. In cMIP (Fig. 6- while feature set B (i.e. Z only) yielded the best performance for middle), the participant was asked to perform marching in place with LR (93.90%) and RF (95.89%). For SiM, the best performance of either standing or sitting. He/she can select the gain level by interact- all the four experimented models (SVM: 90.59%; KNN: 85.72%; ing with the touchpad on the Vive controller as follow: 1× - not press- LR: 80.97%; RF: 88.93%) was found with feature set B (i.e. Z only). ing the touchpad, 3× - pressing and holding the bottom part of the As the SVM model achieved the highest cross-validation accuracy touchpad, and 10× - pressing and holding the top part of the touchpad. (StM: 97.29%; SiM: 90.59%) among the four models, it was chosen For gMIP (Fig. 6-right), similar to cMIP, the participant performed for training the final prediction model for gesture-based gain control. marching in place. Instead of using the controller for gain control, For each gesture, the optimal data set (i.e., Y and Z for StM, and Z the participant could select the speed level with the amplitude of only for SiM) was randomly divided into two sets: 70% for training, leg movement while marching. The VR prototype transmitted the and 30% for validation. The final SVM models for StM and SiM amplitudes of the trackers in real time to a Python server through achieved the validation accuracy of 96.14% and 90.70% respectively. local network. The Python server first normalized the received

433 Figure 6: The three LIP techniques for comparison in Study 3: left - Tele, middle - cMIP, right - gMIP. Figure 7: The three types target distance in Study 3: left - short distance, middle - medium distance, right - long distance. tracker data based on a pre-calculated scale, and then predicted the gain level. The predicted gain level was sent back to the VR prototype for corresponding visual rendering of the user movement. Our VR experience, perceived workload. The experiment for one experiments showed that the computational time for classifying one participant lasted for around 1.5 hours. data entry was around 15ms, which ensured the real-time prediction. 6.5 Results 6.3 Study Design To compare the effectiveness of the three LIP techniques for target reaching, we performed the repeated measures ANOVA on the offset We adopted a within-subject design with three independent factors: distance between the participant-confirmed position and the target the body pose (i.e., standing and sitting), the LIP technique (i.e., Tele, (i.e. accuracy), the time to reach the target (i.e. trial-completion time), cMIP, and gMIP), and the initial distance of the target (i.e., short, and the real-world positional drift. We performed the non-parametric medium, and long). In each trial, the participant was asked to reach the statistics on the questionnaire responses. We also analysed the target as fast and as accurate as possible, using the specific LIP tech- locomotion strategy adopted by the participants for target reaching. nique in the specific body pose. When arriving at the virtual target, the participant needed to stop and press the trigger button on the Vive con- 6.5.1 Accuracy troller in his or her dominant hand to confirm, and the current target To measure the accuracy during the target-reaching task, the exper- disappeared with a new one being spawned. The measured dependent imental system calculated and recorded the target offset which is the variables include: 1) time to reach the target, 2) the distance between Euclidean distance between the target center and the participant’s posi- the user-confirmed position and the target in VR, 3) the amount of real- tion when the trigger was pressed as confirmation. The repeated mea- world positional drift which is calculated by the distance from the start- sures ANOVA showed a significant effect of pose (F(1,11) = 12.86, p ing position at the end of each trial, and 4) participants’ subjective rat- < 0.005, η2 = 0.539) with the standing pose yielding significantly less ings towards the post questionnaire regarding the usability, the motion p sickness, the naturalness and the realness. To reduce the learning ef- offset than the sitting pose. On the other hand, there was no significant fect on the same target distance, we randomized the initial distance of effect of LIP technique or initial target distance. In addition, there the target in each trial within a selected range. That is, the target with was no interaction effect among pose, LIP technique, and initial target short distance was spawned randomly between 5m to 15m towards distance. Fig. 8 shows the descriptive results of the target offsets. the participant’s current position in the virtual world, and medium distance was within the range of 25m and 75m, and long distance was randomized between 100m and 300m. Fig. 7 demonstrates the examples of the targets at the short, the medium, and the long distance. The combination of the body pose and the LIP technique resulted in six experiment blocks which were presented to the participant in a Latin-square-based counterbalanced order. In each block, each range of target distance was repeated for 15 times with a random value within the range each time, and presented in a random order. In total, each participant did 2 types of body pose × 3 LIP techniques × 3 ranges of distance × 15 repetitions = 270 trials.

6.4 Procedure Similar to Study 1, there were one facilitator and one participant in each study session in Study 3. We followed the similar procedure of “Introduction - PreQuestionnaire - Experiment Blocks” in Study 1. Each experiment blocks contained a training session and a testing session. During the training session, the facilitator demonstrated the Figure 8: Descriptive results of target offset in Study 3. particular LIP technique, and guided the participant to practice the technique with the randomly spawned targets. The participant was 6.5.2 Trial-Completion Time allowed to practice as much as he/she wanted until he/she reported The trial-completion time was calculated by measuring the duration to be familiar with the presented technique. For gMIP, the participant from the target being spawned and the participant pressing the trigger. was instructed to go through a matching process between his/her The repeated measures ANOVA showed that there was a significant marching gestures and the three gain levels. Each gain level lasted < 2 for 30 seconds. This was to collect the tracker data of the participant effect of LIP technique (F(2,22) = 50.73, p 0.0005, ηp = 0.822) 2 matching his/her marching gesture to the gain levels, and calculate and distance (F(2,22) = 10.48, p < 0.005, ηp = 0.488), but not pose. the data-normalization scale for gain prediction. After the training Post-hoc pairwise Bonferroni tests revealed that Tele was significantly session, the participant took off the VR headset, and took a 3-minute faster than the other two (p < 0.005), and gMIP was significantly compulsory break. The facilitator then helped the participant to faster than cMIP (p < 0.05). As expected, the long-distance target put on the headset and hold the controller on his/her right hand. took significantly longer time than those with medium and short dis- Upon finishing each block, the participant filled the post-block tance (p < 0.05), but there was no significant difference between the questionnaire to rate the usability of the particular technique, his/her trial-completion time for the targets in medium and short distances.

434 Figure 10: Descriptive results of real-world positional drift in Study 3.

explicit report of prediction error from the participants. This implies a considerable usability of leg-gesture-based gain control for LIP. Figure 9: Descriptive results of trial-completion time in Study 3. Post-hoc pairwise Wilcoxon Signed Ranks Test showed that, both gMIP (Z = 3.75, p < 0.005) and cMIP (Z = 2.96, p < 0.005) In addition, the repeated measures ANOVA showed an interaction were rated significantly more real than Tele, and gMIP was rated effect between distance and LIP technique (F(4,44) = 3.40, p < 0.05, significantly more real than cMIP (Z = 2.85, p < 0.05). gMIP was η2 = 0.236). Post-hoc pairwise Bonferroni tests suggested that the rated to provide a more consistent experience across different senses p than cMIP (Z = 2.82, p < 0.05) and Tele (Z = 3.72, p < 0.005). In significant time efficiency of Tele over the other two techniques was addition, gMIP was rated t o be more natural (Z = 2.77, p < 0.05) and found for the targets in all ranges of distances (p < 0.005), while more preferred (Z = 2.54, p < 0.05) than Tele. In terms of perceived the significant difference of trial-completion time between gMIP workload, gMIP was significantly less mentally demanding than and cMIP was shown for the targets in long and medium distances cMIP (Z = 3.55, p < 0.005) and Tele (Z = 2.78, p < 0.05). However, (gMIP < cMIP: p < 0.05) but not in short distances. Fig. 9 shows gMIP yielded significantly higher physical workload than the other the descriptive results of trial-completion time. two (Tele: Z = 3.95, p < 0.005; Tele: Z = 3.18, p < 0.005), as it required motion-amplitude-based gain control. 6.5.3 Real-world Positional Drift As Tele didn’t require the participants to make large body movement, we considered the real-world positional drift for Tele as 0. We mainly compared the real-world positional drift of gMIP and cMIP. 6.5.5 Locomotion Strategy Additionally, we observed certain amount of drifting in the sitting condition due to the usage of the office chair with wheels. Fig. 10 The LIP techniques may affect users’ behaviors and strategies to ad- shows the descriptive results of the real-world positional drift of gMIP just the size (i.e. movement distance) in each step for target reaching. and cMIP across different poses and ranges of initial target distance. Overall, the size of each step had a medium positive correlation with The repeated measures ANOVAshowed a significant effect of pose the real-time distance to the target in each step, which was statistically 2 significant (Pearson Correlation: r = 0.464, p < 0.0005, n = 29533). (F(1,11) = 15.67, p < 0.005, ηp = 0.587), LIP technique (F(1,11) 2 That is, when a new target was spawned, the participant tended to use = 4.84, p < 0.05, ηp = 0.335), and initial target distance (F(2,22) = large steps, either longer teleportation distance or higher speed level, 2 43.56, p < 0.0005, ηp = 0.798). There was no interaction effect among to get close to the target as fast as possible. As he/she moved close to these three factors. Post-hoc pairwise Bonferroni tests suggested that the target, the participant switched to smaller steps to reach the target standing yielded significantly more drift than sitting did (1.01m vs for the accuracy. The similar medium positive correlation between 0.532m, p < 0.005). gMIP resulted in significantly less drift than the step size and the real-time distance to target was also found in cMIP (0.60m vs 0.94m, p < 0.05). The range of long target distance the pose of standing and sitting separately (Standing: r = 0.493, p led to significantly more drift (Mean = 1.26m, SD = 0.10) than both the < 0.0005, n = 14932; Sitting: r = 0.435, p < 0.0005, n = 14599). ranges of medium (Mean = 0.59m, SD = 0.04, p < 0.0005) and short Looking at each LIP technique individually, both gMIP and cMIP (Mean = 0.46m, SD = 0.09, p < 0.0005) distance, while there was no yielded a strong and statistically significant positive correlation significant difference between medium and short target distance. between the step size and the real-time distance to target (gMIP: r = 0.503, p < 0.0005, n = 10809; cMIP: r = 0.519, p < 0.0005, 6.5.4 Questionnaire Responses n = 12449). On the other hand, there was a small but statistically Table 4 summarizes the descriptive results of the questionnaire correlation between the step size and the real-time distance to target response for standing and sitting respectively. Wilcoxon Signed for Tele (r = 0.208, p < 0.0005, n = 6271). Noted that the total number Ranks Tests showed no significant difference between standing and of recorded step for Tele (n = 6273) was much smaller than those of sitting in all questionnaire items. Friedman Tests suggested that the gMIP (n = 10811) and cMIP (n = 12451), echoing with the significant type of LIP technique significantly affected the participants’ rating shorter trial-completion time with Tele over the other two. During the experiments, we observed that while using Tele, most participants on the realness (χ2(2) = 12.12, p < 0.05), the consistency among 2 2 tended to reach the maximum teleport distance in each step, and different senses (χ (2) = 10.34, p < 0.05), the naturalness (χ (2) switched to a smaller distance only in the final step. Five participants = 9.49, p < 0.05), the likeness (χ2(2) = 7.86, p < 0.05), the mental commented that the visual feedback of the highlighted position in demand (χ2(2) = 20.16, p < 0.005), and the physical demand (χ2(2) Tele made them tend to use the maximum teleport distance, to reach = 22.24, p < 0.005). While Tele is a LIP technique that has been the target quickly. This may explain why the correlation between widely adopted in the consumer market [14], we didn’t observe the step size and the real-time distance to target was smaller in Tele. any significant difference among the experimented techniques in On the other hand, six participants reported that the visual feedback terms of controllability and correctness of output. For gMIP which in Tele may weaken the realness and the naturalness in VR, making relied on the SVM model for gain prediction, we didn’t receive any it less consistent with the real-world experience.

435 Standing Sitting Tele cMIP gMIP Tele cMIP gMIP I like using this technique for walking in VR. 4.25 (0.45) 5.50 (0.29) 6.00 (0.25) 4.25 (0.52) 5.58 (0.29) 5.58 (0.51) It is comfortable to use this technique for walking in VR. 5.25 (0.39) 5.42 (0.26) 5.75 (0.18) 5.5 (0.45) 5.67 (0.31) 5.42 (0.37) It is engaging to using this technique for walking in VR. 3.58 (0.50) 5.25 (0.41) 6.08 (0.23) 3.67 (0.56) 5.67 (0.28) 5.58 (0.43) It is natural to use this gesture for walking in VR. 3.58 (0.47) 5.08 (0.29) 6.25 (0.22) 3.58 (0.51) 5.33 (0.36) 5.67 (0.33) The information coming from your various senses are consistent. 2.50 (0.31) 5.08 (0.29) 6.00 (0.25) 3.00 (0.44) 5.00 (0.33) 5.33 (0.50) The VR experience is consistent with my real-world experience. 2.42 (0.31) 4.92 (0.26) 5.83 (0.17) 2.58 (0.38) 4.50 (0.29) 5.83 (0.37) It is easy to control the walking movement in VR using this gesture. 4.83 (0.41) 5.33 (0.26) 5.83 (0.21) 4.83 (0.53) 5.50 (0.26) 5.83 (0.34) I can easily anticipate what would happen next based on my actions. 5.83 (0.34) 5.33 (0.22) 6.00 (0.17) 5.5 (0.39) 5.42 (0.34) 5.42 (0.28) The system can respond correctly to my actions. 6.08 (0.29) 5.50 (0.19) 5.92 (0.19) 6.25 (0.25) 5.92 (0.26) 6.25 (0.25) It is mentally demanding to use this technique. 3.17 (0.60) 3.17 (0.59) 2.42 (0.40) 3.17 (0.65) 3.50 (0.68) 2.58 (0.48) It is physically demanding to use this technique. 1.83 (0.24) 4.25 (0.49) 5.25 (0.48) 1.75 (0.22) 3.83 (0.63) 5.58 (0.57)

Table 4: Average questionnaire responses in Study 3. The numbers within the brackets are the standard deviations. The cells highlighted in green receive highest ratings within the poses of standing and sitting.

7 DISCUSSION commenting about their experience on this process of data pre- 7.1 Gesture-based Gain Control vs Teleportation recording, it remains unknown how the design of the data-recording process can affect the overall user experience and performance in Our Study 1 showed that marching in place (MIP) was generally the actual testing session. Many existing VR applications provide preferred by the participants, and it yielded more distinguishable the training session to familiarize the users before the actual usage, motion features for gesture-based gain control. Thus, MIP was so the data-pre-recording process can be integrated with training. selected for the development of the machine-learning model on There needs in-depth study on maintaining the balance between the gain classification and the experiments on target-reaching efficiency. efficiency of data pre-recording and the user experience. Study 3 showed that gMIP achieved a comparable target-reaching Thirdly, we mainly investigated the LIP gestures with legs and accuracy compared to controller-based teleportation (Tele), and took feet. Previous research on LIP showed that the gesture of arm swing significantly shorter target-reaching time than cMIP. As commented could be less obtrusive and physically demanding compared to leg by three participants, it took them quite a bit of time to get familiar movements, and maintain the sense of presence in VR [37]. However, with the mapping between the controller operation and the gain level. the scenario with simultaneous walking and hand tasks may suffer In addition, they needed to think about the controller operation for from the hand-motion-based locomotion technique [52]. Therefore, gain switching during the target-reaching task, and this was also we plan to investigate the gesture of arm-swing for the gain-matching reflected by cMIP’s higher rating on the mental demand compared task, and investigate its LIP performance with and without concurrent to gMIP. Yet it was intuitive for them to perform larger motion for hand tasks (e.g., object selection and manipulation). faster speed. On the other hand, we observed significantly shorter Fourthly, three discrete gain levels were studied in the presented task-completion time in Tele than the two MIP techniques. This research. Existing works on WIP calculated the visual speed in VR as result echoed with the previous works that teleport yielded faster a linear function of the leg/foot amplitude [6, 30]. Although previous target-reaching speed than continuous movement [7]. research showed that the discrete speed selection in VR locomotion In addition, gMIP was rated significantly more real than the other may lead to lower motion sickness compared to the continuous two counterparts. Four participants commented that it is similar to speed changing [51], the continuous speed may increase the travel walking in the real world where people walk faster with bigger steps. accuracy. To this end, we will further explore the regression model However, the requirement of increasing the motion amplitude yielded for gesture-based continuous speed estimation. higher physical workload for gMIP than the other two. While the high Last but not the least, we mainly focused on LIP with motion-based physical workload may induce fatigue, five participants told that the speed control in our studies. However, the results may not be large physical movement with gMIP could sometimes compensate directly transferable for the real-walking-based locomotion in VR, the motion sickness in VR (though this was not explicitly reflected as users may perform different gestures to match the speed level in by the questionnaire ratings). Thus, it is suggested that teleport is real walking. Similar experimental protocols could be adopted for more recommended for the tasks with the priority of time, and the investigating the gain-matching gestures for real walking in the future. gesture-based LIP speed control could be suitable for the application with the demands of naturalness, realness, and engagement. 8 CONCLUSION 7.2 Limitations & Future Work In this paper, we present a series of studies investigating gesture-based While our studies suggested the feasiblity of motion-amplitude-based LIP techniques for controlling the virtual walking speed in virtual gain control for LIP in VR, we also identified a few limitations and reality. Our first study selected the gesture of marching in place possibilities for future improvement. Firstly, Study 3 showed that according to the user preference and its distinguishable motion Tele was rated significantly less real than the others. This could be due features for gain control. With the motion data recorded in the to the VR scenario which mimicked a real-world town. Bozgeyikli et first study, we experimented the data-driven techniques for gain al. showed that there was no significant difference between teleport classification, and achieved an overall accuracy above 90% with and WIP in terms of presence with teleport being slightly higher [4]. SVM. Our last user study further showed the effectiveness of In their studies, a fairy-like virtual world was used, and their users using marching in place with gesture-based gain control for target commented they “felt more like a video game because [they] wasn’t reaching in VR, with its comparable accuracy towards teleportation actually walking to move in VR.” To this end, we hypothesise that and significantly higher user ratings on VR experience. With the the experimented VR scene may be a confounding factor on users’ gesture-based speed-control mechanism, we aim to contribute an perception towards the LIP techniques. effective and engaging LIP technique for immersive VR. Secondly, while our experiments showed that the SVM model could provide reliable gain prediction based on the user’s leg motion, ACKNOWLEDGMENTS the system required the new user to perform the LIP walking with This research was partially supported by the YoungScientists Scheme the three gain levels for at least 1.5 minutes, 30 seconds for each of the National Natural Science Foundation of China (Project No. gain, for generating the personal normalization scale. The explicit 61907037), the Applied Research Grant (Project No. 9667189), and pre-calibration process was also adopted in some previous works on the Centre for Applied Computing and Interactive Media (ACIM) WIP [30, 32]. Although there was no participant in Study 3 explicitly in School of Creative Media, City University of Hong Kong.

436 REFERENCES Proceedings of the 3rd ACM Symposium on Spatial User Interaction, pp. 103–112, 2015. [1] P. Abtahi, M. Gonzalez-Franco, E. Ofek, and A. Steed. I’m a [21] P. A. Lachenbruch and M. R. Mickey. Estimation of error rates in giant: Walking in large virtual environments at high speed gains. discriminant analysis. Technometrics, 10(1):1–11, 1968. In Proceedings of the 2019 CHI Conference on Human Factors in [22] E. Langbehn, T. Eichler, S. Ghose, K. von Luck, G. Bruder, and Computing Systems, pp. 1–13, 2019. F. Steinicke. Evaluation of an omnidirectional walking-in-place user [2] J. Bhandari, S. Tregillus, and E. Folmer. Legomotion: scalable interface with virtual locomotion speed scaled by forward leaning angle. walking-based virtual locomotion. In Proceedings of the 23rd ACM In Proceedings of the GI Workshop on Virtual and Augmented Reality symposium on virtual reality software and technology, pp. 1–8, 2017. (GI VR/AR), pp. 149–160, 2015. [3] B. Bolte, F. Steinicke, and G. Bruder. The jumper metaphor: an effective [23] W. B. Lathrop and M. K. Kaiser. Perceived orientation in physical and navigation technique for immersive display setups. In Proceedings of virtual environments: changes in perceived orientation as a function Virtual Reality International Conference, vol. 1, pp. 2–1, 2011. of idiothetic information available. Presence: Teleoperators & Virtual [4] E. Bozgeyikli, A. Raij, S. Katkoori, and R. Dubey. Point & teleport Environments, 11(1):19–32, 2002. locomotion technique for virtual reality. In Proceedings of the 2016 [24] A. Luntz. On estimation of characters obtained in statistical procedure Annual Symposium on Computer-Human Interaction in Play, pp. of recognition. Technicheskaya Kibernetica, 3, 1969. 205–216, 2016. [25] M. Marchal, J. Pettre,´ and A. Lecuyer.´ Joyman: A human-scale joystick [5] E. Bozgeyikli, A. Raij, S. Katkoori, and R. Dubey. Locomotion in for navigating in virtual worlds. In 2011 IEEE Symposium on 3D User virtual reality for room scale tracked areas. International Journal of Interfaces (3DUI), pp. 19–26. IEEE, 2011. Human-Computer Studies, 122:38–49, 2019. [26] N. Marquardt, R. Jota, S. Greenberg, and J. A. Jorge. The continuous [6] L. Bruno, J. Pereira, and J. Jorge. A new approach to walking in place. interaction space: interaction techniques unifying touch and gesture on In IFIP Conference on Human-Computer Interaction, pp. 370–387. and above a digital surface. In IFIP Conference on Human-Computer Springer, 2013. Interaction, pp. 461–476. Springer, 2011. [7] F. Buttussi and L. Chittaro. Locomotion in place in virtual reality: [27] M. McCullough, H. Xu, J. Michelson, M. Jackoski, W. Pease, W. Cobb, A comparative evaluation of joystick, teleport, and leaning. IEEE W. Kalescky, J. Ladd, and B. Williams. Myo arm: swinging to explore transactions on visualization and computer graphics, 2019. a ve. In Proceedings of the ACM SIGGRAPH Symposium on Applied [8] N. Coomer, S. Bullard, W. Clinton, and B. Williams-Sanders. Evaluating Perception, pp. 107–113, 2015. the effects of four vr locomotion methods: joystick, arm-cycling, [28] T. Nguyen-Vo, B. E. Riecke, W. Stuerzlinger, D.-M. Pham, and point-tugging, and teleporting. In Proceedings of the 15th ACM E. Kruijff. Naviboard and navichair: Limited translation combined symposium on applied perception, pp. 1–8, 2018. with full rotation for efficient virtual locomotion. IEEE transactions [9] S. A. Dudani. The distance-weighted k-nearest-neighbor rule. IEEE on visualization and computer graphics, 2019. Transactions on Systems, Man, and Cybernetics, SMC-6(4):325–327, [29] N. C. Nilsson. Walking Without Moving: An exploration of factors 1976. influencing the perceived naturalness of Walking-in-Place tech- [10] Y. Farmani and R. J. Teather. Evaluating discrete viewpoint control to niques for locomotion in virtual environments. PhD thesis, Aalborg reduce cybersickness in virtual reality. Virtual Reality, pp. 1–20, 2020. Universitetsforlag, 2015. [11] J. Feasel, M. C. Whitton, and J. D. Wendt. Llcm-wip: Low-latency, [30] N. C. Nilsson, S. Serafin, M. H. Laursen, K. S. Pedersen, E. Sikstrom,¨ continuous-motion walking-in-place. In 2008 IEEE Symposium on 3D and R. Nordahl. Tapping-in-place: Increasing the naturalness of immer- User Interfaces, pp. 97–104. IEEE, 2008. sive walking-in-place locomotion through novel gestural input. In 2013 [12] J. P. Freiwald, O. Ariza, O. Janeh, and F. Steinicke. Walking by cycling: IEEE Symposium on 3D User Interfaces (3DUI), pp. 31–38. IEEE, 2013. A novel in-place locomotion user interface for seated virtual reality [31] N. C. Nilsson, S. Serafin, and R. Nordahl. The perceived naturalness experiences. In Proceedings of the 2020 CHI Conference on Human of virtual locomotion methods devoid of explicit leg movements. In Factors in Computing Systems, pp. 1–12, 2020. Proceedings of Motion on Games, pp. 155–164. 2013. [13] N. N. Griffin, J. Liu, and E. Folmer. Evaluation of handsbusy vs hands- [32] N. C. Nilsson, S. Serafin, and R. Nordahl. Establishing the range of free virtual locomotion. In Proceedings of the 2018 Annual Symposium perceptually natural visual walking speeds for virtual walking-in-place on Computer-Human Interaction in Play, pp. 211–219, 2018. locomotion. IEEE transactions on visualization and computer graphics, [14] P.-H. Han, C.-E. Hsieh, Y.-S. Chen, J.-C. Hsiao, K.-C. Lee, S.-F. 20(4):569–578, 2014. Ko, K.-W. Chen, C.-H. Chou, and Y.-P. Hung. Aoes: Enhancing [33] N. C. Nilsson, S. Serafin, and R. Nordahl. The influence of step teleportation experience in immersive environment with mid-air haptics. frequency on the range of perceptually natural visual walking speeds In ACM SIGGRAPH 2017 Emerging Technologies, SIGGRAPH ’17. during walking-in-place and treadmill locomotion. In Proceedings of Association for Computing Machinery, New York, NY, USA, 2017. doi: the 20th ACM symposium on virtual reality software and technology, 10.1145/3084822.3084823 pp. 187–190, 2014. [15] A. Harris, K. Nguyen, P. T. Wilson, M. Jackoski, and B. Williams. [34] N. C. Nilsson, S. Serafin, and R. Nordahl. Walking in place through Human joystick: Wii-leaning to translate in large virtual environments. virtual worlds. In International Conference on Human-Computer In Proceedings of the 13th ACM SIGGRAPH International Conference Interaction, pp. 37–48. Springer, 2016. on Virtual-Reality Continuum and its Applications in Industry, pp. [35] N. C. Nilsson, S. Serafin, F. Steinicke, and R. Nordahl. Natural 231–234, 2014. walking in virtual reality: A review. Computers in Entertainment (CIE), [16] V. Interrante, L. Anderson, B. Ries, E. O’Rourke, and L. Gray. 16(2):1–22, 2018. Experimental investigations into the feasibility of using augmented [36] W. S. Noble. What is a support vector machine? Nature Biotechnology, walking to facilitate the intuitive exploration of large scale immersive 24(12):1565–1567, 2006. doi: 10.1038/nbt1206-1565 virtual environments. In Proceedings of the 4th symposium on Applied [37] Y. S. Pai and K. Kunze. Armswing: Using arm swings for accessible perception in graphics and visualization, pp. 144–144, 2007. and immersive navigation in ar/vr spaces. In Proceedings of the 16th [17] V. Interrante, B. Ries, and L. Anderson. Seven league boots: A new International Conference on Mobile and Ubiquitous Multimedia, pp. metaphor for augmented locomotion through moderately large scale 189–198, 2017. immersive virtual environments. In 2007 IEEE Symposium on 3D User [38] S. Razzaque, Z. Kohn, and M. C. Whitton. Redirected walking. Citeseer, interfaces. IEEE, 2007. 2005. [18] B. K. Jaeger and R. R. Mourant. Comparison of simulator sickness [39] S. Razzaque, D. Swapp, M. Slater, M. C. Whitton, and A. Steed. using static and dynamic walking simulators. In Proceedings of the Redirected walking in place. In EGVE, vol. 2, pp. 123–130, 2002. Human Factors and Ergonomics Society Annual Meeting, vol. 45, pp. [40] B. E. Riecke, B. Bodenheimer, T. P. McNamara, B. Williams, P. Peng, 1896–1900. SAGE Publications Sage CA: Los Angeles, CA, 2001. and D. Feuereissen. Do we need to walk for effective virtual reality [19] R. S. Kennedy, N. E. Lane, K. S. Berbaum, and M. G. Lilienthal. navigation? physical rotations alone may suffice. In International Simulator sickness questionnaire: An enhanced method for quantifying Conference on Spatial Cognition, pp. 234–247. Springer, 2010. simulator sickness. The international journal of aviation psychology, [41] B. Sarupuri, S. Hoermann, F. Steinicke, and R. W. Lindeman. Trig- 3(3):203–220, 1993. gerwalking: a biomechanically-inspired locomotion user interface for [20] E. Kruijff, B. Riecke, C. Trekowski, and A. Kitson. Upper body leaning efficient realistic virtual walking. In Proceedings of the 5th Symposium can affect forward self-motion perception in virtual environments. In

437 on Spatial User Interaction, pp. 138–147, 2017. [42] B. Schulte, C. P. Bimberg, J. Hertel, F. Neu, P. Heidenreich, G. Bruder, and P. Lubos. Evaluation of two leaning-based control methods in a dragon riding interface. [43] M. Slater, M. Usoh, and A. Steed. Steps and ladders in virtual reality. In Virtual Reality Software And Technology, pp. 45–54. World Scientific, 1994. [44] M. Slater, M. Usoh, and A. Steed. Taking steps: the influence of a walking technique on presence in virtual reality. ACM Transactions on Computer-Human Interaction (TOCHI), 2(3):201–219, 1995. [45] M. Spindler, M. Martsch, and R. Dachselt. Going beyond the surface: studying multi-layer interaction above the tabletop. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1277–1286, 2012. [46] S. Subramanian, D. Aliakseyeu, and A. Lucero. Multi-layer interaction for digital tables. In Proceedings of the 19th annual ACM symposium on User interface software and technology, pp. 269–272, 2006. [47] V. Svetnik, A. Liaw, C. Tong, J. C. Culberson, R. P. Sheridan, and B. P. Feuston. Random Forest: A Classification and Regression Tool for Compound Classification and QSAR Modeling. Journal of Chemical Information and Computer Sciences, 43(6):1947–1958, nov 2003. doi: 10.1021/ci034160g [48] J. N. Templeman, P. S. Denbrook, and L. E. Sibert. Virtual locomotion: Walking in place through virtual environments. Presence, 8(6):598–617, 1999. [49] S. Tregillus and E. Folmer. Vr-step: Walking-in-place using inertial sensing for hands free navigation in mobile vr environments. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 1250–1255, 2016. [50] M. Usoh, K. Arthur, M. C. Whitton, R. Bastos, A. Steed, M. Slater, and F. P. Brooks Jr. Walking¿ walking-in-place¿ flying, in virtual environments. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques, pp. 359–364, 1999. [51] A. Von Kapri, T. Rick, and S. Feiner. Comparing steering-based travel techniques for search tasks in a cave. In 2011 IEEE Virtual Reality Conference, pp. 91–94. IEEE, 2011. [52] J. von Willich, M. Schmitz, F. Muller,¨ D. Schmitt, and M. Muhlh¨ auser.¨ Podoportation: Foot-based locomotion in virtual reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–14, 2020. [53] C. Wang, O. A. Dogaru, P. L. Strandholt, N. C. Nilsson, R. Nordahl, and S. Serafin. Step aside: an initial exploration of gestural input for lateral movement during walking-in-place locomotion. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology, pp. 1–5, 2018. [54] J. D. Wendt, M. C. Whitton, and F. P. Brooks. Gud wip: Gait- understanding-driven walking-in-place. In 2010 IEEE Virtual Reality Conference (VR), pp. 51–58. IEEE, 2010. [55] B. Williams, G. Narasimham, T. P. McNamara, T. H. Carr, J. J. Rieser, and B. Bodenheimer. Updating orientation in large virtual environments using scaled translational gain. In Proceedings of the 3rd symposium on Applied perception in graphics and visualization, pp. 21–28, 2006. [56] G. Wilson, M. McGill, M. Jamieson, J. R. Williamson, and S. A. Brewster. Object manipulation in virtual reality under increasing levels of translational gain. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–13, 2018. [57] P. T. Wilson, W. Kalescky, A. MacLaughlin, and B. Williams. Vr locomotion: walking¿ walking in place¿ arm swinging. In Proceedings of the 15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry-Volume 1, pp. 243–249, 2016. [58] R. E. Wright. Logistic regression. In Reading and understanding mul- tivariate statistics., pp. 217–244. American Psychological Association, Washington, DC, US, 1995. [59] C.-C. Wu, S.-W. Chou, and W.-H. Hong. Effectiveness of goosestep training or its modification on treating patellar malalignment syndrome: clinical, radiographic, and electromyographic studies. Archives of orthopaedic and trauma surgery, 129(3):333–341, 2009. [60] Y. Zhang, B. E. Riecke, T. Schiphorst, and C. Neustaedter. Perch to fly: Embodied virtual reality flying locomotion with a flexible perching stance. In Proceedings of the 2019 on Designing Interactive Systems Conference, DIS ’19, p. 253–264. Association for Computing Machinery, New York, NY,USA, 2019. doi: 10.1145/3322276.3322357

438