ADATA-DRIVENMOTIONPLANNING FRAMEWORKFORINTERACTIVEWALK PATHSIMULATIONWITHINTHE AUTOMOTIVEINDUSTRY PHILIPPAGETHEN from Gräfelfing DOCTORALTHESIS A thesis submitted in fulfillment of the requirements for the degree of Doctor rerum naturalium (Dr. rer. nat.) Human-Computer-Interaction Group Institute of Media Informatics Faculty of Engineering, Computer Science and Psychology 2020 acting dean: Prof. Dr. Maurits Ortmanns, Ulm University referees: Prof. Dr. Enrico Rukzio, Ulm University Prof. Dr. Martin Manns, University of Siegen day of defense: October 23th 2020 Philipp Agethen: A Data-Driven Motion Planning Framework for Interactive Walk Path Simulation Within the Automotive Industry, Doctoral dissertation. ©2020 This document was typeset in LATEX using the typographical look-and-feel classicthesis developed by André Miede. classicthesis is available for both LATEX and LYX: https://bitbucket.org/amiede/classicthesis/ DEDICATEDTOMYWIFEHANNA COPYRIGHT [citation] For the sake of clarity, literally used or only marginally modified text passages, being previously published by the author, are highlighted in the margin notes. An example can be found on the left side. This particularly applies for following manuscripts: [6] Philipp Agethen, Felix Gaisbauer, and Enrico Rukzio. “A Proba- bilistic Steering Parameter Model for Deterministic Motion Planning Algorithms.” In: Computer Graphics Forum 38.1 (2019), pp. 549–563. doi: 10.1111/cgf.13591. With permission of Wiley. [4] Philipp Agethen et al. “Towards Realistic Walk Path Simulation of Single Subjects: Presenting a Probabilistic Motion Planning Algo- rithm.” In: Proceedings of the 11th Annual International ACM SIGGRAPH Conference on Motion, Interaction, and Games. MIG ’18. Limassol, Cyprus: ACM, 2018, pp. 1–10. isbn: 978-1-4503-6015-9. doi: 10.1145/3274247. 3274504. With permission of ACM. [7] Philipp Agethen et al. “Counterbalancing Virtual Reality In- duced Temporal Disparities of Human Locomotion for the Manufac- turing Industry.” In: Proceedings of the 11th Annual International ACM SIGGRAPH Conference on Motion, Interaction, and Games. MIG ’18. Li- massol, Cyprus: ACM, 2018, pp. 1–5. isbn: 978-1-4503-6015-9. doi: 10.1145/3274247.3274517. With permission of ACM. [11] Philipp Agethen et al. “Behavior Analysis of Human Loco- motion in the Real World and Virtual Reality for the Manufacturing Industry.” In: ACM Transactions on Applied Perception 15.3 (July 2018), pp. 1–19. issn: 1544-3558. doi: 10.1145/3230648. With permission of ACM. [5] Philipp Agethen et al. “Interactive Simulation for Walk Path Planning within the Automotive Industry.” In: Procedia CIRP 72 (2018). 51st CIRP Conference on Manufacturing Systems, pp. 285 –290. issn: 2212-8271. doi: 10.1016/j.procir.2018.03.223. With permission of Elsevier. [9] Philipp Agethen et al. “Presenting a Novel Motion Capture-based Approach for Walk Path Segmentation and Drift Analysis in Manual Assembly.” In: Procedia CIRP 52 (2016). The Sixth International Con- ference on Changeable, Agile, Reconfigurable and Virtual Production (CARV2016), pp. 286 –291. issn: 2212-8271. doi: 10.1016/j.procir. 2016.07.048. With permission of Elsevier. v [10] Philipp Agethen et al. “Using Marker-less Motion Capture Systems for Walk Path Analysis in Paced Assembly Flow Lines.” In: Procedia CIRP 54 (2016). 6th CIRP Conference on Learning Factories, pp. 152 –157. issn: 2212-8271. doi: 10.1016/j.procir.2016.04.125. With permission of Elsevier. Furthermore, following icons are used throughout the thesis: Icon made by Freepik from https://www.flaticon.com/ free-icon/image_739249 Icon made by Freepik from https://www.flaticon.com/ free-icon/capture_1719890 Icon made by Freepik from https://www.flaticon.com/ free-icon/vr-glasses_2036269 Icon made by Freepik from https://www.flaticon.com/ free-icon/settings_2099058 Icon made by Eucalyp from https://www.flaticon.com/ free-icon/robot_2083532 Icon made by Good Ware from https://www.flaticon.com/ free-icon/book_864685 ABSTRACT Walking is one of the most ordinarily used motion and ubiquitous in our everyday lives. Owing to its evident significance, locomotion is subject to a wide-ranging field of research, spanning various disci- plines, applications and level of detail. An area, which in this context has recently attracted particular attention, is the synthesis of human motion and motion planning. This includes the generation of center of mass trajectories in a bird’s eye view. Even though use-cases en- compassing walk paths differ in their respective scope, the ability to realistically predict real-world observations has shown to be deci- sive. For instance, it is of significant importance for the simulation of evacuation scenarios to accurately model pedestrian dynamics and local interactions. A high prediction quality is equally important when planning walking trajectories of assembly operators. Especially for the latter, however, related work reveals limitations in terms of mo- tion realism. This is accompanied by a lack of validation approaches, ensuring the validity of artificial walk paths. In the automotive industry, these shortcomings are confronted with an increasing demand for simulation methods, which is caused by disruptive changes. Particularly the significant extension of product portfolios in combination with extensive individualization options induce additional efforts for production planning departments. This paradigm shift also affects the process of walk path planning, whose methods are becoming increasingly inaccurate and inefficient. To cope with both challenges, this thesis proposes a framework, which allows to realistically plan and validate walk paths. The so- called interactive walk path simulation concept can be regarded as sym- biosis of immersive virtual reality and motion planning in form of a feedback-loop. As the user can easily investigate and reenact synthetic motions in true-to-scale, on the one hand side, invalid simulation out- comes can reliably be detected - and corrected. On the other hand side, motion planning algorithms enable the examination of different process-variants and the consideration of the probabilistic nature of human motion. In essence, it is possible to reinforce the individual advantages and compensate the drawbacks of both groups. To implement the generic concept for the use-case of automotive production planning, the situation in assembly lines is grasped in a first step. To this matter, a marker-less motion capture system is pre- sented, which facilitates the investigation of walk paths in industrial environments. Two conducted on-site case-studies reveal that espe- cially the consideration of operator-drift and probabilistic nature of the human locomotor system is essential. Incorporating these findings, vii two data-driven motion planning algorithms are presented, constituting the backbone of this thesis. By utilizing probabilistic motion models, which take into account the intrinsic variability, both approaches en- able the generation of statistically distributed motion corridors. Even though the algorithms are nearly exclusively generating suitable re- sults, it is nevertheless crucial to review the synthesized walk paths in virtual reality with respect to their validity. In this context, effects need to be explicitly taken into account, which are induced by the use of head-mounted displays. Therefore, several experiments are conducted, analyzing the behavioral disparity between human locomotion, being performed without any equipment and in virtual reality. Based on the obtained knowledge, compensation strategies are eventually pre- sented, which counterbalance the use of the immersive visualization technique in the feedback loop. The framework and all of its introduced components are evaluated through multiple case studies, systematically assessing their applica- bility to overcome the identified drawbacks. Unifying all concepts, algorithms, technical and conceptual explorations, as well as qualita- tive and empirical research, it can be concluded, that the presented interactive walk path simulation framework significantly increases resem- blance of artificial center of mass trajectories to reality. Throughout this thesis, special care has been given to generalizabil- ity and to a generic design, so that contributions can be applied to a wide range of domains beyond process engineering. PUBLICATIONS 1. Philipp Agethen, Felix Gaisbauer, Philipp Froehlich, Martin Manns, and Enrico Rukzio. “Towards Realistic Walk Path Sim- ulation in Automotive Assembly Lines: A Probabilistic Ap- proach.” In: Procedia CIRP 67 (2018). 11th CIRP Conference on Intelligent Computation in Manufacturing Engineering, 19-21 July 2017, Gulf of Naples, Italy, pp. 464 –469. issn: 2212-8271. doi: 10.1016/j.procir.2017.12.243. 2. Philipp Agethen, Felix Gaisbauer, Martin Manns, Max Link, and Enrico Rukzio. “Towards Realistic Walk Path Simulation of Single Subjects: Presenting a Probabilistic Motion Planning Algorithm.” In: Proceedings of the 11th Annual International ACM SIGGRAPH Conference on Motion, Interaction, and Games. MIG ’18. Limassol, Cyprus: ACM, 2018, pp. 1–10. isbn: 978-1-4503- 6015-9. doi: 10.1145/3274247.3274504. 3. Philipp Agethen, Felix Gaisbauer, Michael Otto, and Enrico Rukzio. “Interactive Simulation for Walk Path Planning within the Automotive Industry.” In: Procedia CIRP 72 (2018). 51st CIRP Conference on Manufacturing Systems, pp. 285 –290. issn: 2212-8271. doi: 10.1016/j.procir.2018.03.223. 4. Philipp Agethen, Felix Gaisbauer, and Enrico Rukzio. “A Prob- abilistic Steering Parameter