A 3D stateroom editor Andreas Alexander Gansen Master’s Thesis Autumn 2016 A 3D stateroom editor Andreas Alexander Gansen Abstract Today’s software systems reach easily hundreds of thousands of lines of code, and such systems do frequently benefit from the use of state machines, which help in managing system complexity by guaranteeing completeness and consistency. To visualize such state machines, statecharts have been introduced, which also offer a formalism for or- thogonal and hierarchical states. Many developers use state machines, but even with statecharts as a tool, it is a major challenge to keep an overview of the machine and its effects. Gaining an overview as a newcomer to a project is an even larger challenge. In this paper, we argue that a 3D statechart can alleviate these challenges somewhat, and present an editor for state machines that are given in SCXML, an XML notation for statecharts. This editor allows the user to explore the state machine by navigating freely in a 3D environment and make changes to the machine. The editor is a prototype and incomplete. It is an attempt to reflect the idea of having statecharts presented in 3D space. Contents I Introduction1 1 Motivation2 1.1 Mnemonic strategy............................3 1.2 Problem Definition / Statement......................4 1.3 Limitations................................4 1.4 Main Contributions............................4 1.5 Outline..................................5 1.6 Acknowledgement............................6 II Theoretical Foundation7 2 Introduction to SCXML8 2.1 Basic Primitives..............................8 2.2 Automaton................................9 2.3 Expanding to SCXML.......................... 11 2.4 SCXML notation............................. 15 2.4.1 State............................... 15 2.4.2 Parallel.............................. 16 2.4.3 Final............................... 16 2.4.4 Pseudo states........................... 16 2.4.5 Transition............................. 17 2.4.6 The root tag <scxml>....................... 18 2.4.7 Event descriptors......................... 18 2.5 SCXML interpretation.......................... 19 2.5.1 Transition selection process................... 19 2.5.2 Data Model............................ 20 2.5.3 Executable Content........................ 21 2.5.4 Communication.......................... 21 2.5.5 Extending SCXML........................ 22 2.5.6 Configuration........................... 23 2.6 Final Note................................. 23 3 Mathematical background 24 i MY TITLE ii 3.1 Coordinate Systems............................ 24 3.2 Quaternions................................ 32 3.3 Billboarding................................ 37 3.4 Algorithms................................ 39 III Blender 41 4 Introduction to Blender 42 5 Blender Game Engine 43 5.1 Logic Editor................................ 44 5.2 Billboarding................................ 50 6 Trial and error: Stateroom editor as game 51 6.1 HUD - Heads up display......................... 51 6.1.1 Movement............................ 53 6.2 Modeling Atomic State.......................... 54 6.3 Modeling Transitions........................... 55 6.4 Unsolved problems - Final analysis.................... 57 IV OpenGL 59 7 Introduction to OpenGL 60 7.1 Rendering pipeline overview....................... 61 7.2 Prerequisites................................ 63 7.2.1 Binding concept......................... 63 7.2.2 Framebuffer............................ 63 7.2.3 Textures.............................. 64 7.2.4 Primitives............................. 66 7.3 Draw call................................. 66 8 Rendering pipeline 68 8.1 Viewport setting.............................. 68 8.2 General Process.............................. 69 8.3 Buffer Objects............................... 69 8.3.1 Uniform buffer object...................... 70 8.3.2 Buffer Texture.......................... 71 8.3.3 Vertex buffer object........................ 71 8.4 Vertex Array Object............................ 72 8.5 Shaders.................................. 73 8.5.1 Vertex Shader........................... 73 8.5.2 Geometry Shader......................... 74 8.5.3 Fragment Shader......................... 75 8.6 Per-Sample Operations.......................... 76 MY TITLE iii 8.6.1 Depth Test............................ 76 8.6.2 Blending............................. 76 8.6.3 Transform feedback....................... 77 9 Design 78 9.0.1 Floating and stationary states................... 78 9.0.2 Anchors.............................. 78 9.0.3 Label............................... 79 9.0.4 Menu............................... 79 9.0.5 Tooltip.............................. 80 9.0.6 Stationary states......................... 80 9.0.7 Floating states.......................... 83 9.0.8 Transitions............................ 86 9.0.9 User Navigation......................... 86 10 Implementation in C++ 89 10.1 Introduction................................ 89 10.2 Navigation and Interaction........................ 90 10.2.1 Frustum determination...................... 90 10.2.2 Object picking.......................... 91 10.2.3 Main and debug menu...................... 93 10.3 Architecture................................ 94 10.3.1 Qt plugin............................. 95 10.3.2 stateroom editor plugin...................... 97 10.4 Modeling SCXML Elements....................... 98 10.4.1 Stationary States......................... 98 10.4.2 Floating States.......................... 101 10.4.3 Transitions............................ 101 10.5 Modules.................................. 103 10.5.1 scxmlParser............................ 103 10.5.2 text_3d.............................. 106 10.5.3 helper_gl............................. 110 10.5.4 scxmlInterpreterManager..................... 110 10.5.5 Serializing to SCXML...................... 111 10.5.6 Shared uniform buffer...................... 111 11 Simulator 113 11.1 Harel’s watch as blender game...................... 113 11.1.1 Communication server...................... 116 11.2 Communication client........................... 116 11.3 Debugger................................. 117 12 Evaluation and Discussion 119 13 Conclusion 120 MY TITLE iv 13.1 Summary................................. 120 List of Figures 2.1 Basic primitives of a state machine....................9 2.2 State diagram presenting a Moore and a Mealy machine......... 10 2.3 Compound state example......................... 11 2.4 History states of type shallow and deep.................. 12 2.5 A parallel state.............................. 13 2.6 Two transitions.............................. 13 2.7 Nested compound states......................... 14 2.8 Initial tag................................. 15 2.9 Transition types (internal/external).................... 18 3.1 Right-handed coordinate system...................... 24 3.2 Overview of coordinate systems and matrix transformation........ 28 3.3 Perspective vs. orthographic projection.................. 29 3.4 Frustum.................................. 29 3.5 Field of view............................... 30 3.6 Complex system mapped into the Cartesian system........... 34 3.7 Assembled quaternion rotation...................... 36 3.8 World oriented billboarding........................ 38 3.9 The effect of billboarding on different axes............... 38 5.1 Controller column from BGE....................... 45 5.2 Handling mouse click with logic bricks................. 45 5.3 Billboard Actuator............................ 50 5.4 Several states billboarded around two axis................. 50 6.1 Empty stateroom............................. 51 6.2 Stateroom under development...................... 52 6.3 Logic spawn button............................ 52 6.4 Vertices for atomic state.......................... 54 6.5 Atomic state model............................. 55 6.6 Transition in BGE with bones...................... 56 6.7 Stateroom in BGE (editing states).................... 58 7.1 OpenGL pipeline............................. 61 7.2 OpenGL primitives............................ 66 v MY TITLE vi 8.1 Vertex attributes.............................. 72 8.2 Overview Per-Sample Operation..................... 76 9.1 Main menu................................. 80 9.2 Final state icon............................... 80 9.3 Yakindu hierarichal state......................... 81 9.4 Atomic state mock-up........................... 81 9.5 Dialog to edit source tags......................... 82 9.6 History/Initial state icon......................... 82 9.7 Menu compound state........................... 83 9.8 Mock-ups for compound states...................... 83 9.9 Mockup for floating labels........................ 85 9.10 Mockup to fold a state.......................... 85 9.11 A folded composite state.......................... 85 9.12 Internal/External transition........................ 86 9.13 State chart for user input......................... 87 10.1 Mouse hover over Pie menu....................... 93 10.2 Overview of used/developed libraries................... 94 10.3 Architecture Qt plugin........................... 96 10.4 Core plugin architecture.......................... 97 10.5 Atmic state as superellipse........................ 98 10.6 State drawn elliptic as triangle fan..................... 99 10.7 Atomic state 3-parted........................... 100 10.8 Catmull rom atomic state......................... 100 10.9 Compound state.............................. 101 10.10Line segments with gaps........................