Integrating Virtual Reality, Motion Simulation and a 4D GIS

Integrating Virtual Reality, Motion Simulation and A 4D GIS

JUERGEN ROSSMANN, ARNO BUECKEN, MARTIN HOPPEN, MARC PRIGGEMEYER

Rossmann, J., Buecken, A., Hoppen, M., & Priggemeyer, M. (2016). Integrating Virtual Reality, Motion Simulation and a 4D GIS. Research In Urbanism Series, 4(1), 25-42. doi:10.7480/rius.4.854 KEYWORDS Virtual Forrest Virtual reality; Motion simulation; 4D GIS; Geodesign; Abstract Geodesign requires the visualization of concepts and ideas within a context to is understandable a way that place in respective of the of geo-information people with differentdifferent have backgroundsinvolved roles the of – planners,All place. geographers, the of architects,inhabitants or but users the also requirements and need different information to fulfil their tasks structure of a software In this contribution, we present the process. geodesign within the as well as component, and a VR system combining a GIS, a simulation system interfaces to differentinteraction devices (like a GPS receiver, a spacemouse, multi-screen projection systems or devices for haptic feedback). This enables immersive as well as context, of the place in its geographical simulations plan’s knowledge of a of the regardless are understandable that presentations need to convert frequently All this happens without the symbolic language. by the different roles. tools that are commonly used between the software

RIUS 4: GEO-DESIGN 26 INTEGRATING VIRTUAL REALITY, MOTION SIMULATION AND A 4D GIS 27 - - - - - CREATING A GIS FROM A 3D SIMULATION SYSTEM 3D SIMULATION A GIS FROM A CREATING INTRODUCTION Most require software environments multiple components independent sim- and information approach a forest The presented was developed for We will describe the synchronization computers, the visuali- of multiple We will describe the synchronization The next section as follows: illus- of this paper The rest is structured 2. In this In this contribution, we present an approach of a integrated 4D geo fully 1. According Steinitz to Geodesign (2012) “is and applica the development for editing and displaying 3D geo-data. GIS provide While standard a 2D top view on the scenery and in some cases a 2.5D or even a rather limited 3D view, additional software is needed to display the same scenery on large scale The disadvantage of multiple software displays or in virtual reality systems. data editing impressive visualization tool chain from to the products in the is the frequent need to convert and exchange data. user interface ap that required a number of algorithms and ulation system graphic information and virtual reality (VR) system. While data management virtual reality (VR) system. graphic and information is on OGC based like (Open Geospatial the Geography standards Consortium) Markup Language (GML), it supports of multiple the synchronization clients views – even for a seven screen allows rendering screens for multiple and panoramic projection or a vis- system CAVE Besides VR-style environment. ualization techniques, it is also possible to the data for simulation use or to system. even feel it with a motion simulator versatile motion simulator, which is use of a highly the and zation component based on an industrial robot. Several aspects like the used washout filter and a robot with a still limited of use enable the physiological that the foundations in a large world are introduced. workspace to display poses and forces next the In by GIS functionality. system trates, how we extend a 3D simulation section, VR capabilities. we describe the simulation system’s Subsequently, the motion simulation approach is The contribution with presented. ends a conclusion of the presented work. proaches, which were implemented in already VEROSIM, an existing 3D sim- tion of design-related processes intended to change geographical to change geographical areas intended processes tion of design-related study are applied they in which between He states that collaboration realized”. and There challenging. be can process geodesign a in involved roles different the part of a is an essential results an intuitive steps and fore of the visualisation geodesign This goes process. successful with together geo the evolution of 3D, 2D to in recent which happened from (GIS) graphic information systems dimension. years, By now, some approaches time even consider as a fourth Visualization simple data moved ahead from of the corresponding maps to- cities. landscapes and However, it cases, three-dimensional most in is wards display on a single computer. limited to a single - - - Figure 1. The architecture of the database synchronization approach. Interface Flexible Database In Figure 1, an exemplary synchronization scenario is shown. Two sim- Two is shown. scenario 1, an exemplary synchronization Figure In For accessing (geo-)databases from a simulation system, a flexible yet 2.1 ulation system. Originally, Originally, was developed of simulation for the this system system. ulation robotic of applications a variety to has since but work cells, enhanced been from the fields ofindustrial automation, space roboticsGIS, of starting with the latter, instead Thus, for a support standard for stand and environment. ard geo-data like for geo-databases and interfaces modelling SupportGIS Java VEROSIM. to the existing were added PostgreSQL/PostGIS (SGJ) or native ulation clients with their respective ulation clients with their respective and #2) are connected to databases (#1 ing schema synchronization, the simulation system adopts the schema from ing system schema synchronization, the simulation “speak start-up so both systems the system the central database once during schema can be Subsequently, data conforming to this same language”. - repli For example, database, cated to the simulation on-demand. driving a virtual trees, data (e.g. surrounding model, spatially nearby a large forest car through tile data) is loaded into the simulation database while objects are unloaded database can be are left behind. Thus, the simulation when they seen as an “intelligent”, real-time capable approach cache for the central database. The also allows modifying replicated data. Here, change notifications are used to (Hoppen, betweendatabases the updates Waspe, Rast, & Ross synchronize mann, 2014). Thus, synchronizing multiple when simulation clients to the same central database, it be cannot only used for data management, but also simulation model. as a communication hub for the shared efficient concept was developed (Hoppen, Rossmann,runtime internal a simulation system’s Its basic2010). idea is to synchronize Schluse, & Waspe, and functional level. Us central database on schema, data database with the

RIUS 4: GEO-DESIGN 28 INTEGRATING VIRTUAL REALITY, MOTION SIMULATION AND A 4D GIS 29 - - Temporal Data Management Client Synchronization Additionally, basic Additionally, basic for editing required GIS functionality the stored data Note that all these mechanisms are of the actual data model independent 2.2 2.3 a central database containing a simple ‘Tree’ object object with a The state. ‘felled’ a containing database central a simple ‘Tree’ object and is replicated to ‘true’ the state changes #1 Client clients. to both it syncs previous where the to the central database value is with versioned a timestamp tend. Subsequently, a notification sent to client #2 allows it to func data synchronization, and besides schema change. Finally, adopt the tional synchronization tional is between semantics synchronization used to translate the systems. matrix of a CityGML Gröger, & (Kolbe, For example, transformation the Plümer, 2005) implicit geometry is translated to a data structure known to engine. for the presented work, Altogether, render the simulation system’s Active VEROSIM using the concept was realized database synchronization this (VSD) and SGJ. Simulation Database son, 1998) as the central geodatabase. When changing data in a temporal database, its previous state does not get but is lost, preserved in terms of historic versions that are still accessible by the user. As geo-data represents the state of real at world phenomena one or more points in time, a temporal database allows capturing this inherent time dependence. Different interpretations of time database be applied. A transaction time, so-called time dimensions, may automatically associates committed timestamps with In contrast, any change. in a valid time database, the user (or some process) assigns timestamps that represent the point in real time a change has taken place or will take place. Both concepts be combined, can even yielding a bi-temporal Using database. concept, database synchronization a temporal snapshot the aforementioned purpose, database. For that runtime simulation system’s is replicated to the changing On system. simulation the within time reference a specifies user the altering accordingly. When gets updated time, this reference the snapshot database, synchronized simulation system’s within the replicated data the 1 shows within the central, temporal database. Figure changes are versioned an example, where a from a change simulation client is replicated to the central temporal database. Here, the previous value (“false”) becomes a historic version before the new value is adopted and a notification is emitted. was implemented, e.g., for measurement, vector or raster editing, editing, or raster vector or gradient measurement, e.g., for was implemented, or- In visualization, with an integrated GIS. 3D simulation system yielding a and reproduce the changes of the 3D model (or any other forest der to capture dimension (Hoppen, & Schluse, Rossmann, time as a fourth model), we added Weitzig, & Dyre 2012). This is realized by using a temporal database (Jensen, and can be transferred to other (geo-)data than trees and forests. Thus, the (geo-)data to other than trees and forests. and can be transferred - Figure 2. Example scenario for database synchronization. VIRTUAL REALITY There are multiple are multiple virtual reality applications with geo-infor- There deal that Furthermore, our approach of multiple in- the synchronization allows our Furthermore, 3. very same approach can also be used to monitor the change in urban areas, approachsame in urban change the to monitor very used also be can CityGML using the e.g. like formats common also other but data model, IFC are others Classes), STL Foundation many and (STereoLithography) (Industry supported. mation. Examples are landscape visualization, architecture as well as simula- Examplesare landscape visualization, mation. architecture cases, it is required to export most or other vehicles. In tion of cars, aircrafts like DXFconvert it to some 3D format (AutoCAD Drawing the geo-data and Exchange Format) or IGES (Initial Graphics Exchange Specification) in order however, the presented system, the inte- In to use it in a simulation system. 2D or- of VEROSIM is activated, view from the is changed grated 3D renderer dependent clients either by a simple property synchronization mechanism, clients either by a simple property synchronization dependent a fully-fledged distributed simulation protocol, or by using a central, active above. This allows to either as presented geo-database (Hoppen al., 2014) et link multiple or multi-projector displays for CAVE environments panoramic for several projections, views same scenery or to generate independent of the client each connected The approaches can even be combined so that users. example using combined an 2 shows Figure screens. use multiple can in turn model is synchronization approaches: the simulation managed by a central geo-database replicated and to two clients (jeep on TV, helicopter on mul ti-projector projection).panoramic itself distributes The projection system for rendering the three stereo images. systems the simulation to six slave

RIUS 4: GEO-DESIGN 30 INTEGRATING VIRTUAL REALITY, MOTION SIMULATION AND A 4D GIS 31 - - Figure 3. A virtual city guide. The software system also offers physical simulation of objects. This way, thogonal to a to a thogonal 3D perspective projection – if and required multiple – - comput ers are with together linked approach synchronization the above. described allowing be adjusted, can individually each computer, to view On the frustum and right the for images the render computers two (where views stereo define combination as a well as screens) views (using adjacent eye), panoramic left of both. The renderer supports different lighting situations, change of day time, different weather scenarios and even the photorealistic visualization 3 gives an example with objects. a Figure virtual city guide. The of natural computer. On the of hardware with the scales renderer of the performance PC with a graphic games it board designed for computer a standard - is possi ble to visualize with eighty environments million vertices at forty frames per It is can simply be projected on a 3D ground. 2D geography features second. also possible to for this information – like use metaphors an auto-generated representing property boundaries. feature fence for a surface ufacturer’s ufacturer’s on-board computer. The simulation can be configured in a way it is possible to use the geo-data for simulation purposes. The objects can be simulation purposes. geo-data for it is possible to use the controlled by the user with several different interaction devices like a wire- data gloves, a spacemouse, tracking system, six DOF (degrees of freedom) less seat with the man like a harvester joysticks or even a dedicated hardware Figure 4. The capsule of the motion simulator. MOTION PLATFORM MOTION Although providing a deep visual immersion, it is still difficult, e.g., to es- to e.g., difficult, still is it immersion, visual deep a providing Although are limited in their all Stewart/Gough hexapods Due to their mechanics, 4. timate the real inclination of a hill or the effort needed to follow a steep road. steep a follow to needed effort the or hill a of inclination real the timate This impression, however, can easily be achieved by moving the user according approaches the hill’s slope. There are several to so-called motion for this cueing. utilize Conventional approaches mostly a Stewart/Gough platform, a hexapod hydraulic that allows moving the top plate in six This dimensions. a seat with only carry installation, which can a small is scalable from system deck flight a moves that solution a towards helmet, data a and person single a applications. for professional multi-user training or a ship’s bridge with these Thus, steep inclinations cannot be simulated movement. rotatory devices. A more advanced approach is a motion an in- simulator based on against dustrial robot. While the main disadvantages the hexapod are a low- er maximum a payload and larger space requirement, the motion simulator benefits from the versatile movements of the industrial robot. With this- de possible. vice, even an overhead situation becomes that the results or performance logs are also geo-coded. Therefore, it is pos- it Therefore, geo-coded. also logs are performance or results the that - envi corresponding in the simulation of the results discuss and to study sible notices or geo-coded logs are track-marks of geo-coded Examples ronment. a small sign. be displayed as which can

RIUS 4: GEO-DESIGN 32 INTEGRATING VIRTUAL REALITY, MOTION SIMULATION AND A 4D GIS 33 Figure 5. The motion simulator system. Our system features a KUKA KR-500 TÜV robot, a six-axis industrial robot industrial six-axis a robot, TÜV KR-500 KUKA a features system Our For work the in described this contribution, we to use a decided ro- with a maximum payload of 500 kg. This is sufficient for the capsule including capsule the for sufficient is This kg. 500 of payload maximum a with robot The 5). (Figure kg 120 to up with passenger a and electronics installed all operates in a work cell with a diameter of approximately requires a 10 m and height clearance of about 7 m. The capsule can be accessed in a height of 2.4 safety, the user’s ensure m by using a staircase To and a retractable platform. bot-based motion simulator with with simulator motion bot-based capsule, a small which is equipped with a 180 cm wide hemispherical screen in of a front high resolution 3D projector with a fisheye lens above the user as well as a stereosound system (Figure interaction, 4). For user capsule the which can two touchscreens, provides be two information, or additional used to three-axis instruments display The capsule is equipped control. with an throttle a joysticks, and two pedals opaque textile cover keeps that out visual pro- impressions from the outside viding a better immersion. - - - Figure 6. Components involved in motion cueing. Motion Cues The benefit of this six-degrees-of-freedom robot is the precise mapping precise the is robot six-degrees-of-freedom this of benefit The The vestibular system responds differently to translational and rotation- and translational to differently responds system vestibular The By computing the appropriate robot motion in real-time, the driver’s 4.1 the capsule is equipped equipped is the capsule a bidirectional with link, audio and downlink a video a in smoke its is hardware-restricted The robot detector. to movements avoid capsule as between the as well ground the capsule and between the collisions includes an acceleration the robot control limit Furthermore, and the robot. applied the forces that are to control impacts The the user. to ac- are limited 2004), EN 13814 (DIN, cording feels accelerations but still the user to DIN up to 2 g. of simulated movements and accelerations onto a pose in the real world. This world. accelerations onto a pose in the real and movements of simulated seat can be positioned oriented accordingway, an attached and to the user’s situation within the simulation. For example, in the simulation of a wood a forest, passing through a driver navigates 6, left), when (Figure harvester through rough terrain and evading obstacles, it is invaluable for a realistic con- In tactile element. with a 3D visual information simulation to extend the trast to screens, the hemispherical smaller projection provides for the possibility of strong peripheral visual cues. These motion cues are caused subcon- sciously by motion observed in visually the peripheral of view.field Different researchers have already put huge efforts into studying these effects. Their motion feedback, strong provide a very cues already these conclusion is that sys vestibular the through observed motion by amplified be still can they but tem (Telban, 2005). al motion. Due to the robot’s construction, there is a strong influence on the ro- motion perception to extended and, therefore, due semi-circular canals tations is intensified bydesign. Nevertheless, since thresholds for the hu been proposed (Zacharias, jerking 1978), strong man motion perception have movements can be utilized to convey translational accelerations by stimulat ing the otolithic organs. motion and the visual feedback can be for a holistic synchronized driving

RIUS 4: GEO-DESIGN 34 INTEGRATING VIRTUAL REALITY, MOTION SIMULATION AND A 4D GIS 35 - - Distributed Simulation The washout-filteringstep is essential to provide measures for the lim- This motion has to be by an actuator, performed e.g., a six-axis robot like The simulation comprises different computers to carry out specific tasks 4.2 impression impression (Telban, 2005). This can be achieved by estimating motion cues by a vehicle caused By calculating to a passenger. induced and accelerations of can subset motion a filtering, washout subsequent a with frame passenger the system vestibular a passenger’s stimulate to be used can that be estimated passenger perceives the the Additionally, motion cues. relevant causing the of view. her peripheral field cues due to visual motion real Since a vehicle in the (Grant, & Reid, 1997). robot the ited workspace of pas- while the regards to its physical behaviour, with world can move freely, limited due to the is rather in the simulator motion envelope senger seat’s of the robot, a mapping constraints to be applied. motion has of the of Models inner ear can be applied the human the perceived to preserve Since motion. canals as well as the otolithic the semi-circular organs can be as modelled (Zacharias, actual motion are per- 1978), only parts of the damped systems ceived Thus, it until the others. “washout” masks is possible, e.g., to stop ac- celerating the capsule in one direction after a short period of time and then to motion induction pristine position further move backwards to the for slowly without the passenger noticing. simulator, carrying a capsule providing motion For this a passenger seat. our has robot control system robot to move, to the manufacturer’s an interface to be provided. In general, such an interface needs to meet real-time hard constraints specified by the manufacturer, which are specific to a particular Communication robot control system. protocols, simulation, planning and execution are separated onto different machines constituting a distributed from are detached software This way, time critical parts of the simulation. the non-time-critical parts and executed on dedicated computers providing to meet the specified constraints. enough computing resources el is composed of different parts with different masses. A wheel suspension is attached to provide realistic as a damped mass-spring system in- modelled vehicle (Jung, Rast, Kaigom, the and & Ross terrain teraction between rough In the examplemann, 2011). of the wood harvester, the vehicle also has a As this provides a huge level of inter- crane to grab and work on tree trunks. action between vehicle it and environment, also provides a huge potential for a realistic motion feedback. (Figure 6). Computer (A) either runs a Windows or a Linux operating system. Computer (A) either runs 6). (Figure and the vehicle’s dynamics for VEROSIM to simulate a It provides a platform environment it with. interacts For a realistic simulation, the vehicle’s mod - 𝑝𝑝

𝑣𝑣

𝑎𝑎 𝑃𝑃𝑃𝑃 Figure 7. Linear control model for accelerated motion. − + The first approach utilizes a simple linear model describing accelerat- The motion The feedback - ve is (B) by transforming computer on calculated cope with the inverse kinematics path planning, and simulation the To a target pose exceeds the robot limits, is calculated that When steps have profiles velocity compute to implemented were approaches different Two 𝑐𝑐𝑐𝑐𝑐𝑐 𝑝𝑝 ed motion (Figure 7). Accelerations (a) are integrated twiceAccelerations 7). (with the velocity ed motion (Figure to provide (v) as intermediary) positions (p). Since positions are controlled a sta- by accelerations, a closed-loop linear controller is to necessary ensure (propor- PD is implemented with the The controller behaviour. ble system comparing desired position algorithm the tional-derivative) control (p_cmd) actual position is parametrized with the to provide (p). The controller small locities locities and accelerations into poses new robot trajectories. and the runs It QNX operatingwith its system pre-emptive 1992). (Hildebrand, scheduler con- to meet the real-time of the whole setup component This is a mandatory straints imposed by the Kuka Robot (KRC2). Control A sampling frequency of QNX motion. Thus, the robot attained to achieve to be 1/12kHz has smooth system clock runs with a 1 ms resolution to provide a fine granularity for the VEROSIM task scheduler. This its environment. robot and physical of the a model VEROSIM uses system workspace available motion, considering the to evaluate the is used model These constraints are imposed and robot constraints. by limits for the axis’ positions, velocities By exceeding and accelerations. any of these limits, the to signal a fault message is issued an error is stopped and robots movement state. Because simulation this also prevents the from continuing, the robot to, to be strictly adhered prevent a to un passenger from constraints have comfortable accelerations. perceived minimize robot in a way to wrongly to be taken to move the the motion cues. One examined approach was to move the robot to its farthest axes that continue moving those only then possible Cartesian position and limits to provide a smooth stop. are sufficiently far away from their for the robot to follow. When a velocity profile is calculated, it is used to in- terpolate new target positions axes. for the robot’s These positions are set by the KRC2. the internal position controllers of

RIUS 4: GEO-DESIGN 36 INTEGRATING VIRTUAL REALITY, MOTION SIMULATION AND A 4D GIS 37 - - 𝛼𝛼 𝑑𝑑 + HP HP Tilt Figure 8. Basic washout filter principle. 𝜔𝜔 𝑎𝑎 Simulation To provide target positions provide target described approaches, the previously for a To The second approach The second parameterizes cubic hermite splines to describe Either way, resulting poses of robot the simulated are sampled with a fre quency of 250Hz and subsequently used as target values for the internal posi internal the for values target as used subsequently and 250Hz of quency accordingly. the KRC that move the physical robot tion controllers of washout filter was implemented being the basis for the whole motion cuing process. To this point, a classical washout filter design (Krämer, 2004) has been utilized on computer (B), executed with a frequency of 250 Hz. A basic layout of this filter is depicted in Figure 8. Three main parts can be identified that operate in close coordination. For all of them, the vehicle simulation has to provide accelerations (a,ω) that are separated into a translational and a ro- is utilized component by the tilt coor- tational component. The translational dination (TILT, Figure 8) as well as the high pass filter (HP, Figure 8 top) for low removes filter pass high translational the Since motion. translational the be frequency components, which would for long acceleration phases in useful vehicles, these low tilt the frequency accelerations coordination transforms earth’s This way, the is applied seat. passenger into an orientation that to the accelerations that would otherwise be force is used to display gravitational lost. system response times while being while times response system Subsequently, stable with overshoot. little limits. robot acceleration fit the to are truncated values the acceleration a velocity profile. When a new target position isreceived, it is taken as the spline’s finalpoint setting the velocity to zero.Accordingly, the robot’s- cur rent position velocity and is spline’s starting used for the point. This allows coping scenarios. First, with two position no new target received is until the spline’s final point is reached. Thus, the robot is stopped and waits for new Second, a commands. new target position is received while the robot is still position and velocity is used to a calculate moving. new Since pro- current the file, the current one can be replaced by the newly computed velocity profile behaviour. without causing jerking Synchronization Robot Control These changes in orientation will ultimately induce rotational motion motion rotational induce will ultimately in orientation changes These The filtered accelerations (d,α) can be used to control the robot motion. Different computer systems are combined to provide a distributed simu- distributed a provide to combined are systems computer Different (A) simulates a vehicle The VEROSIM instance on computer whose seat’s critical is a non-time part of the simulation, As this synchronization For its KRC2 robots, KUKA provides a control panel that can be used for 4.3 4.4 cues if performed too rapidly. Since thresholds for the perception the of these for Since thresholds too rapidly. cues if performed changes exist, the effectscan be compensated. As with translational accelerations, rotational accelerations are also high-pass-filtered (HP, Figure 8 cause would phases that long acceleration to remove unacceptably bottom) com- frequency low However, limits. predefined its outside move to robot the coordi- tilt the to counterpart no is there as differently processed are ponents motion cueing. are lost and cannot be utilized in nation. Thus, they position,to its pristine controller a linear capsule to return guarantee the To is parameterized to keep this motion accelerations for The controller is used. cues. The the induction of false motion to avoid below perceptual thresholds pristine position has to be chosen carefully to provide for a maximum flexi- bility in accelera- the consecutive lengthy movements. The aforementioned exhaust the robot’s workspace tion phases might reducing otherwise possible motion cues. lation for our motion simulator. All computers have a specific task to perform to task specific a have computers All simulator. motion our for lation and, therefore, to have synchronize specific properties. As mentioned above, VEROSIM provides a simple property mechanism (Hoppen synchronization can be used to replicate et al., 2014). It individual properties of the simulation model between VEROSIM instances on different computers. Every time a property is modified, the new value and its timestamp are sent to all other connected instances. frame is used as an input frame for the washout filter. As the washoutfilter computer (B), instance on is executed on the QNX VEROSIM the seat’s frame has to be between synchronized both instances. Hence, computer (A) acts as a to computer of the seat’s frame are sent server for computer (B) and changes (B) where the appropriate and rotational accelerations for the translational processing. components are calculated for further frames are not sampled at fixed rates, but rather as soon as changes occur. with an Therefore, as the vehicle simulation on computer (A) is carried out update frequency of 25 Hz to 30 Hz, new samples arrive at the least every 40 ms.

RIUS 4: GEO-DESIGN 38 INTEGRATING VIRTUAL REALITY, MOTION SIMULATION AND A 4D GIS 39 - - 𝑃𝑃𝑃𝑃𝑃𝑃 𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶 𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴 𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴 𝐾𝐾𝐾𝐾𝐾𝐾퐾 𝐾𝐾𝐾𝐾𝐾𝐾𝐾𝐾 𝐸𝐸𝐸𝐸𝐸𝐸 𝐸𝐸퐸𝐸𝐸𝐸𝐸 𝐸𝐸 𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶 𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶 𝐶𝐶 . 𝑝𝑝�𝑝𝑝𝑝𝑝 ) 𝐵𝐵 ( Position control integration with the Robot Sensor Interface. Figure 9. Position control integration with the Robot Sensor 𝐶𝐶 𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉 𝐶𝐶𝐶𝐶𝐶𝐶 𝑉𝑉𝑉𝑉 For applications,such (RSI) provides the Robot Sensor Interface KUKA On the QNX computer, is VEROSIM process and is executed as a real-time For our motion 9 simulator, a block (right) diagram as depicted For in our Figure 𝐶𝐶𝐶𝐶𝐶𝐶 𝐶𝐶 𝑄𝑄𝑄𝑄𝑄𝑄 robot robot It is control. a graphical interface user to variables, display and states It can information. be also system writing used for KRL (Kuka Robot Language) it Furthermore, hardware. real-time robot’s the to be executed on programs allows to moving directly robot the with a simple Even though this keystroke. enables the user to easily manipulate the robot’s position, it is not sufficient to large implement sensor applications or applications on external dependent path planning. package. technology It is a KRL application-programming interface compris- ing function blocks that can be connected into block diagrams implementing complex KRC2, algorithms. On the the RSI programming is carried out using function calls. connect them by KRL to instantiate blocks and manually code When the RSI definition is finished, functiona call can pass control over the robot position from the integrated panel control to by en- an external system tering the so-called sensor driven mode. This enables the RSI function mode interface. causes an override of the user blocks and, therefore, ler and makes the robot move. The robot’s current position is also transmit makes the robot move. The robot’s current ler and 9 left). ted back (QNX VEROSIM) via to the external system Ethernet (Figure motion feedback algo- the for Thus, here, a closed-loop feedback controller rithms can be implemented on Computer (B) as well. connected to the KUKA KRC2. As mentioned above, its primary task is to exe- for profiles velocity calculate to and kinematics robot the of simulation a cute to channel a data opensVEROSIM QNX Thus, the physical robot to follow. the replicates block diagram and RSI in the function block running Ethernet the is implemented. A function block is instantiated for Ethernet connectivity A function block is implemented. is instantiated for Ethernet axis (Ethernet), control (Axis position (Axis Ctrl) and This way, Pos). sensor new positions via to be be can appliedcommanded Ethernet to the robot’s axes. The axis control block passes positions to the internal position control - Motion simulator responds to left-hand curve of simulated jeep. Motion simulator responds to left-hand curve of Figure 11. An impression of the motion simulator’s dynamic behaviour. Figure 10. CONCLUSIONS A combination of these techniques yields a simulation system that pro- that system a simulation techniques yields of these A combination Altogether, the presented combination with of a 3D simulation system 5. the simulated robot’s motion by commanding positions sampled from the the positions commanding by motion from robot’s sampled simulated the velocity profile. Since theRSI Ethernetblock demands a 12ms timeslot, new a 1/12 kHz frequency. with are sampled asynchronously positions vides holistic driving driving impressions. A passenger jeep a simulated (Fig simulat- see and feel accelerations 10) can vehicle through by steering a ure ed terrain in a realistic For example, manner. while driving a left-hand long capsule is moved likewise motion simulator the to induce the realistic curve 11 gives an impression of the motion simulator’s driving impression. Figure dynamic behaviour. VR, GIS and motion simulation functionalities provides a fully integrated motion simulation functionalities provides a fully VR, GIS and required tool chain the usually GIS approach.reduces virtual reality and It export import and eliminates the need to permanently to a single software,

RIUS 4: GEO-DESIGN 40 INTEGRATING VIRTUAL REALITY, MOTION SIMULATION AND A 4D GIS 41 - - The haptic motion simulator feedback an industrial-robot-based from The motion feedback induced to the passenger’s perceptual organs is With the presented approach, GIS, VR visualization, presented With the hap- simulation and Applications with a robot-based motion sim- for this integrated solution data between between data the possibility and adds multiple systems to interact with the be- It like geo-data system. in a VR displayed when GIS even a common in live maps GIS, even to generate object to change comes possible data in the - in the virtual re explore to immediately the results and the geo-data from can simulation results georeferenced On the other hand, ality environment. a Virtual ideas of which is one of the tools, GIS available all with be evaluated For example, & Rast, 2010). Jung, Testbed (Rossmann, of a performance logs can harvester be combined with 2D or 3D maps and displayed in the GIS or VR view. impressions of the to the geo-data the even more information about adds or of any simulated objectMovements of the camera user. within geo-da- the consider which filters washout using by movements robot to converted are ta cal- These physiologicalperception limits of the movement the of a human. loop inner control passed to the robots - in a re are then culated movements with al-time the process. visual Together impacts of the visual-range-filling hemispherical projection, this haptic feedback provides a holistic impression for the user. motion planning carried out on the re- directly depending on the path and al-time system. A huge variety of different parameters and perceptual limits can be exploited to increase the immersion of a passenger into the simulated environment, holding the capabilities for future research. A washout filter implementation specifically applied to the robot axes in favour of the Car- tesian filter implementation will lead to a more efficient workspace utiliza tion. Hence, specific feedback channels can be prioritized (e.g. lateral motion) lateral (e.g. prioritized be can channels feedback specific Hence, tion. leading motion to an enhanced perception, reducing while simultaneously the risk of motion sickness. tic feedback are merged together, delivering added value for each these of fields. In geodesign processes, this combination provides a foundationinformation for exchange between the different roles. The VR system allows an therefore concepts and intuitive of the planned immersive visualisation and for every role without the ne- provides access to the displayed information special drawings or the symbolic language of understand and cessity to learn other roles. city visualizations including range from or a Stewart/Gough platform ulator - including cars or helicopters to simula presentations and virtual city tours tions like wood and Virtual Testbeds in the presented harvester multiple areas of engineering. With the 4D support, visualisations change of larger land scapes become possible. - REFERENCES Research Research project Virtual Forest: This project is co-financed by the- Euro ACKNOWLEDGEMENTS chinery and structures – Safety; German version EN 13814:2004. Beuth-Verlag. chinery and structures – 34(2), 145-151. (pp. 113-126). Other Kernel Architectures Inter- 5th The (Eds.), Nagel C. & König, G. Kolbe, T. In Sources. Data Geo Object-Oriented And Systems Germany: Shaker Verlag. 3D Geoinformation ISPRS 2010 (pp. 51-56). Aachen, national Conference on (pp. 1-12). Proceedings of the 2012 Winter Simulation Conference ing for 3D Simulation Applications. International Journal of Computer Theory and Engineering (IJCTE), 6(3), 247-253. glossary of temporal database concepts—february 1998 version. In Temporal Databases: Research Springer. and Practice (pp. 367-405). Berlin-Heidelberg, Germany: on Versatile Rigid Multi-Body Dynamics Simulation. In ASME 2011 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (pp. 1481-1490). American Society of Mechanical Engineers. formation for disaster management (pp. 883-899). Berlin-Heidelberg: Springer. formation for disaster management (pp. 883-899). plattform. Shaker Publishing. cations in the woods and on the moon. In Intelligent Robots and Systems (IROS), 2010 IEEE/RSJ Inter- national Conference on (pp. 4952-4957). IEEE. ear and Nonlinear Approaches. NASA/CR-2005-213747. bridge MA Control Systems Dept. Grant, P. R., & Reid, L. D. (1997). Motion washout filter tuning: Rules and requirements. Journal of Aircraft, Aircraft, of Journal requirements. and Rules tuning: filter washout Motion (1997). D. L. Reid, & R., P. Grant, and Microkernels on Workshop USENIX In QNX. of Overview Architectural An April). (1992, D. Hildebrand, Hoppen, M., Rossmann, J., Schluse, M., & Waspe, R. (2010). A New Method For Interfacing 3D Simulation Hoppen, M., Schluse, M., Rossmann, J., & Weitzig, B. (2012). Database-Driven Distributed 3D Simulation. Hoppen, M., Waspe, R., Rast, M., & Rossmann, J. (2014). Distributed Information Processing and Render- Jensen, C. S., Dyreson, C. E., Böhlen, M., Clifford, J., Elmasri, R., Gadia, S. K., et al. (1998). The consensus Based Development Application VR Fast January). (2011, J. Rossmann, & G., E. Kaigom, M., Rast, J., T. Jung, Kolbe, T. H., Gröger, G., & Plümer, L. (2005). CityGML: Interoperable access to 3D city models. In Geo-in DIN – German Institute for Standardization (2004). DIN EN 13814: Fairground and amusement park ma- Krämer, M. A. (2004). Universelle Fahrzeugsteuerung als integrativer Bestandteil einer VR-Simulations- Rossmann, J., Jung, T. J., & Rast, M. (2010, October). Developing virtual testbeds for mobile robotic appli- Press. Esri CA: Redlands, Design. by Geography Changing – Geodesign for Framework A (2012). C. Steinitz, Lin- Algorithm Development: Human-Centered Motion Cueing Telban Robert, J., & Cardullo, F. M. (2005). Cam- inc. Newman and Beranek Bolt analysis. pilot-vehicle for models cue Motion (1978). L. G. Zacharias, pean Union and the federal state of North Rhine-Westphalia, European Re- pean Union state and the federal Rhine-Westphalia, of North European future. our – Investing in (ERDF). Europe Fund gional Development

RIUS 4: GEO-DESIGN 42