Eurographics Workshop on Visual Computing for Biology and Medicine (2019) B. Kozlíková, L. Linsen, and P.-P. Vázquez (Editors)
Medical Animations: A Survey and a Research Agenda
Bernhard Preim 1 and Monique Meuschke1
1University of Magdeburg & Research Campus STIMULATE, Germany
Abstract Animation is a potentially powerful instrument to convey complex information with movements, smooth transitions between different states that employ the strong human capabilities to perceive and interpret motion. Animation is a natural choice to display time-dependent data where the dynamic nature of the data is mapped to a kind of video (temporal animation). Clipping planes may be smoothly translated and object transparency adapted to control visibility and further support emphasis of spatial relations, e.g. around a tumor. Animation, however, may also be employed for static data, e.g. to move a camera along a predefined path to convey complex anatomical structures. Virtual endoscopy, where the virtual camera is moved inside an air-filled or fluid-filled structure is a prominent example for these non-temporal animations. Animations, however, are complex visualizations that may depict a larger number of changes in a short period of time. Thus, they need to be assessed in their capability to actually convey information. In this paper, we give a survey of temporal and non-temporal animated visualizations focussed on medical applications and discuss the research potential that arises. To be employed more widely, cognitive limitations, e.g. change blindness, need to be considered. The reduction of complexity in temporal animations is an essential topic to enable the detection and interpretation of changes. Emphasis techniques may guide the user’s attention and improve the perception of essential features. Finally, interaction beyond the typical video recorder functionality is considered. Although our focus is medicine, the discussion of a research agenda is partially based on cartography, where animation is widely used.
Categories and Subject Descriptors (according to ACM CCS): Animation
1. Introduction multiple coordinated views for interactive data exploration. Synchro- nized visualizations of internal and external 3D views are typical Computer-generated animations have a long tradition. Powerful for virtual endoscopy [JLS∗97]. animation tools were developed already in the 1980s, primarily motivated by the needs of the film industry. Computer-generated Animations may be used for surgical planning and training animations are created based on metaphors from traditional films, [ÇK00,MBP06], and for anatomy education [HH95,PRS96]. Also like a story board and key frames that are interpolated to achieve the use of animations for patient education [KKSP08,McG10] and smooth transitions. Animation provides enormous flexibility. A vir- forensic use cases [Fis08, VOH17] was extensively studied. While tual camera may be rotated in a fixed distance around a pathology forensic use cases require a high degree of realism (referred to as to assess its morphology (orbiting). The virtual camera may display scientific animations), patient education benefits from plausible, a region around a pathology, zooming in particularly interesting abstracted visualizations where also aesthetic aspects are consid- regions, e.g. to assess an infiltration, or it may zoom out and move to ered. In contrast, animations for medical education are driven more another interesting viewpoint. The camera may also be moved inside by learning-theoretic considerations, e.g. motivational aspects and structures, e.g. in bronchoscopy or colonoscopy [LJK95]. Animated cognitive load theory. displays provide motion parallax, an essential depth cue that sup- ports the interpretation of volume rendered images and maximum Like static medical visualizations may benefit from carefully intensity projections that are otherwise difficult to interpret [SGS95]. chosen default values, the parameters that guide an animation may be re-used in similar cases, e.g. to report cases with a similar pathology Animation may also be used to smoothly change parameters of a in a reproducible manner [IHT∗02, MP10b]. In addition to these transfer function and thus to display or hide structures, to change col- animations, where no dynamic process is visualized, animation is ors or textures for emphasis of focus objects and to adjust clipping a natural choice to display dynamic medical image data, such as planes or more complex resection geometries. Virtually every param- simulated or measured blood flow [dHJEV16], motion of the heart eter of a static display may be smoothly changed to a different state. wall [JAES87] and other functional processes. “Animation let us While most animations are restricted to one viewport, multiple syn- observe how an object changes its shape, size, and position . . . over chronized animations may provide different perspectives—similar to time” [AWM10].
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda
Animated visualizations may be completely predefined with no or Other areas of visual perception, such as color and contrast per- only basic interaction. At the other end of the spectrum, highly cus- ception, shape and depth perception as well as flow perception are tomized animations may be created on demand where the user has also relevant for animation design. For the sake of brevity, we do full control over all parameters. Thus, the purpose ranges from com- not discuss these other areas. As a starting point for dealing with munication of known facts with predefined, often carefully prepared the role of perception in visualization, we recommend the work of animations to exploratory data analysis [DMKR92]. COLIN WARE [War12]. Since a large number of medical animations has been created with general purpose software instead of dedicated Medical visualization comprises both direct volume rendering systems for medical use, we mention major tools in Sect. 2.3. controlled by a transfer function and (polygonal) surface rendering. These two families of visualization techniques also occur in medical animation. However, surface-based rendering is clearly prevailing. 2.1. Cognitive and Perceptual Basics This is due to the large flexibility to adjust polygonal models, e.g. by smoothing or simplifying them [McG10]. Another argument for The design of animations has to consider a number of aspects re- the use of surface models is that medical animations often contain lated to perception, cognition, motivation and learning theory. The additional elements, such as surgical instruments or crime weapons effectiveness of animations may depend on [RCL09] in forensic animations that are represented as surface model. • the actual content to be displayed, So far, there is no survey article on animation in medicine. Ani- • the specific processes that are shown, mation in other areas, however, was discussed in a number of survey • learner characteristics, such as domain knowledge and spatial articles. The most recent of these articles relate to character ani- abilities as well as mation, in particular to muscle modelling [CRPPD17], hand and • the context in which the display of animations is embedded, e.g. finger modeling [WWS∗15], and more general to physics-based verbal instructions. approaches to character animation [GP12]. There is a slight overlap Research in this area is not comprehensive and in particular medi- to the topics discussed in these articles with respect to biomechanics. cal animations are not investigated well. Thus, our discussion also However, the specific examples of animation using principles from considers non-medical animations. The amount to which experi- biomechanics are different: We consider examples in forensics that ences from other areas, such as cartography and mechanics, can be were not part of previous surveys. Other surveys focussed on the generalized towards medical use cases is currently not clear. application of animation to photorealistically generated images, e.g. ∗ ∗ Wald et al. [WMG 09]. The survey of Kreiser et al. [KMM 18] Motion perception is the rather low-level process of identifying describes projection-based medical visualization techniques. It is and assessing movements. An essential aspect is the ability to detect briefly mentioned, since flattened visualizations may be animated changes at all. In a series of experiments, it was demonstrated that similar to animated maps in cartography. However, Kreiser et al. do users may miss even movements of larger characters known as not discuss any animated technique. change blindness [SL97]. Change blindness has a number of reasons, including saccadic eye movements that occur when a new region Organization. In this paper, we give an overview of animation in an image is fixated. For a period of up to 200 ms, humans are design (Sect.2) and discuss how these methods can be applied blind during these eye movements [Gre15]. Not only that humans to medical visualization. Patient-specific animations are useful for do not reliably detect changes, they are also not aware that they diagnosis and treatment planning. However, the effort to generate have not fully understood a video sequence. Thus, even users who animations must be strongly reduced in clinical medicine. Therefore, miss essential changes report to be certain in their decision—a we discuss strategies to re-use animations in Sect.3. In Section4 we phenomenon called change blindness blindness [LMDIS00]. For discuss temporal medical visualizations with a focus on animated animation design, emphasis techniques, such as arrows and the use blood flow. Non-temporal animations are discussed in Sect.5. In of colors that grab attention, may be considered. Change blindness Section6, we discuss application areas, including medical education, also depends on the speed of presentation. Neither very slow nor virtual endoscopy, and forensics. A crucial issue is the interaction to fast movements are beneficial to support the detection of changes. steer animated displays. This issue is described in Section7. Finally, we discuss a research agenda (Sect.8). The ideas for future work Correct interpretation of an animation not only requires that users in this area are largely inspired by cartographic animation—an area detect changes, they also should interpret the character of a change, that is more mature and involves systematic research on cognitive e.g. whether a quantitive value has increased or decreased and to limitations in the interpretation of animations. what extent the value changed compared to the original state. Thus, perception researchers use change detection (CD) levels, where
2. General Principles for Animation Design • CD 1 relates to the detection of a change, • CD 2 to the direction of change, and In this section, we provide a high-level discussion of perceptual • CD 3 to the amount of change [GB09]. and cognitive issues involved in the interpretation of animations, including motion perception, change blindness, and cognitive load Evaluations in cartography animation reveal that humans make (Sect. 2.1). Since medical animations often aim at educational pur- errors when they should answer CD level 2 and CD level 3 questions. poses, we also briefly discuss learning theories. As a second ingre- Thus, users may notice changes despite having not enough time to dient in animation design, we discuss principles from film making figure out what exactly changed. Perceptual research indicates that that were applied to computer-generated animations (Sect. 2.2). the problems are more severe when it is necessary to pay attention
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda to different regions in an image. This split screen attention situation 2.2. Theories and Principles for Animation Design reduces the human change detection ability [AW05]. Animation design involves unique aspects that go beyond model- ing for interactive use. A geometric model with fine details may Cognitive load theory. Cognitive load theory considers the lim- be appropriate for exploration but when moved with moderate or ited resources of the human brain to process information. Short-term even high speed, the details may be unnoticed. Since the viewing memory is limited and as a consequence information overload may directions are determined by an animation author, parts of a geo- interfere with learning [Swe04]. Researchers investigating carto- metric model may always be hidden or appear too dark, based on graphic animation therefore suggest to restrict both the complexity the lighting specification which the viewer typically also cannot and the duration of animations [Har07]. Animations should be rather influence [HH95]. Temporal coherence is an issue: Visualization short, typically below one minute. Also based on cognitive load techniques that involve stochastic sampling or slight inaccuracies theory, Ruiz et al. [RCL09] discussed evidence that breaking an may lead to distracting flicker if tiny elements show up and disappear animation into smaller chunks reduces cognitive load and is ben- suddenly. As a consequence, animations must be iteratively devel- eficial for learning. They recommend that the information should oped, carefully checked and refined to reduce the risk of unwanted remain available at the end. Cognitive load is further distinguished effects as much as possible. in intrinsic cognitive load that comprises the information that need Hand-drawn character animation provides an essential basis for to be processed, and extrinsic cognitive load that comprises other animation design. Lasseter [Las87] mentioned eleven principles information that consumes cognitive resources without any effect based on an in-depth analysis of literature in this area. While some on learning. Obviously, extrinsic cognitive load, e.g. anything that principles apply primarily to facial expressions and other peculiari- may distract, should be minimized [Swe04]. ties of character animation, the following principles are also relevant for (non-temporal) medical animations. Learning theory. We will see that a considerable number of med- Animations should be plausible with respect to mechanical laws ical animations serve educational purposes. The use of animation which means that animated objects should be guided by gravity and in educational settings is driven by the attempt to “increase student inertia. Viewers will use such knowledge to interpret an animation, interest in the subject material” [Fis08]. Fisk continues by saying e.g. an object that can be accelerated very fast must be light, whereas “Today’s students have been raised in an environment rich in videos heavy objects make slower movements. Objects are stretched and and visual stimulation. As a result, animated demonstrations of squeezed during motion, at least if they are not completely rigid. scientific topics may be more familiar and engaging for younger viewers” [Fis08]. While there is broad consensus about this motiva- Anticipation. The principles described by Lasseter [Las87] fur- tional aspect, the evidence that animations indeed improve learning ther explain that the actual movement shown in an animation re- and are more efficient than learning with other media is controver- quires preparation, as we discussed in Sect. 2.1. In the language of sial. Obviously, animations do not automatically have a substantial film, this is referred to as anticipation. Lasseter gives the example educational value. Excessive details and animations that are primar- of a boy kicking a ball: The foot must be pulled back before the ball ily decorating may be distracting. Instead, animations should be can be kicked. In a similar way, also a step of a surgical intervention consequently designed and evaluated with specific educational goals can be prepared. Otherwise, viewers tend to miss the essence of a in mind, where the dynamic character is likely beneficial. movement. “Anticipation is a device to catch the audience’s eye, to prepare them for the next movement and lead them to expect it. . . . Anticipation is also used to direct the attention . . . to the right Fisk [Fis08] discusses investigations by other authors that indicate part of the screen at the right moment.” [Las87]. the importance of an adequate preparation. Basic introductory infor- mation should be given to students prior to observing an animation. Staging. After the anticipation, the core of a movement takes Also during the animation, “narration should be used to explain the place. Lasseter argues to show just one movement at a time to events occurring in the animation” [Fis08]. Audio recordings or text support focussed attention on this movement. If an animation should embedded in the animation can be used. While audio recordings convey different movements, these should be serialized [Las87, have the advantage that spoken text can be better perceived concur- Fis08]. In a “busy” animation with many moving objects, Lasseter rently to an animation, displayed text requires visual attention to be argues, that the eye tends to focus on relatively still regions—missing split between observing the animation and reading text. the dynamics completely [Las87]. According to the staging principle, the moving object should further be emphasized and thus differ from other objects. The aspect of guiding attention in an animation was Ruiz et al. [RCL09] summarize research on animations that are emphasized by many authors, also related to medical animation, e.g. effective for learning. They conclude that movements similar to Fisk [Fis08]. In essence, with careful anticipation and staging, the those, a learner is expected to perform later, are highly effective. As author of an animation can effectively guide the viewer to certain a consequence, the actual handling of instruments and the conduct objects in a certain sequence—exactly what is desirable to realize a of steps in surgery can be learned better based on animations. Less communicative intent, e.g. for an educational purpose. effective—according to this theory—are animations that explain, for example, how a machine or the heart works. Ruiz et al. [RCL09] Arcs. Another principle, briefly referred to as arcs, relates to argue that even the best animations may not ideally serve their typical camera movements: cameras typically move along arcs and (learning) purpose if they are not carefully embedded in an overall only rarely along straight lines. The camera orientation changes learning strategy, comprising more active types of learning. only slightly. Traditional animation is based on keyframes and thus
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda
3D computer animation systems often follow this principle as well. and may choose among different strategies to create smooth anima- Catmull [Cat72] presented the first such system. In this paper, also tions based on an interpolation scheme. These interpolation-based the first scripting language, the Motion Picture Language, was intro- techniques are often summarized as “Tweening” [Sak06]. The com- duced to define animations. It supports accelerated and decelerated plex tools aim at efficient support for professional users instead of movements as well as concurrent or overlapping processes. To re- an easy-to-use lean interface for casual users. We found a number alize camera movements along arcs, animation systems employ of animations created by physicians for educating students or pa- spline-based interpolation techniques [KB84]. tients. Cutting et al. [COH∗02] used Maya from Alias Wavefront for creating animations to support surgical training (cleft lip surgery). Animations have a number of essential characteristics: Herman [Her02] used Cinema4D to create animations for patient ed- • Duration. Typically, animations for presentation purposes are ucation. Lim et al. [LBR04] used Truespace to provide educational rather short, mostly below a minute. material for regional anesthesia training examples. In Sect.6, we • Speed. The speed determines how much time users have available discuss these and other examples. to detect and interpret changes. The preferred speed may vary considerably and should be adjustable. 2.4. Scientific and Non-Scientific Animations • Temporal Scale. When dynamic data is displayed, a certain world time interval is mapped to a certain display time interval. Medical animations can be used to convey physiological properties, • Interpolation. The specific interpolation method used for the e.g. the pulsatile blood flow and the resulting vessel wall movement. transition between states or positions, e.g. linear or cubic. The Also biomechanical properties, e.g. the range of motion of the shoul- interpolation determines the smoothness of camera movements. der, can be effectively communicated with an animation. This leads to a further useful discrimination of animations. There are three types of motion in an animation [Zet13, Sto17]: • Scientific animations are based on the laws of physics, such as • Primary motion. Movements of the actual actors, e.g. a car, or energy conservation and kinematic laws. Scientific animations a cat escaping from a dog. In medical applications, growth pro- aim at a high degree of realism and correctness. cesses, or changes due to a contrast agent that diffuses over time • Non-scientific animations in contrast aim at a plausible depiction are examples for primary motions. of movements without any guarantee that the behavior is correctly • Secondary motion. Movements of the camera, e.g. pan, tilt, and shown. Non-scientific medical animations may be used for patient dolly, and movements of the lense elements, e.g. zooming. Also education or more generally for educational processes. adaptations of illumination settings or transparency in a computer- Scientific animations are primarily used in medical research and in generated animation belong to secondary motions. In medical selected diagnostic processess, such as rupture risk assessment of animations, also transfer function parameters may be animated. an aneurysm based on an unsteady simulation of blood flow. • Tertiary motion. Switches from one shot to another, i.e. transitions, such as fade, dissolve and wipe. 3. Re-use of Medical Animations Primary motions are the core of temporal animations, whereas sec- ondary motions occur primarily in non-temporal animations. Ter- Medical animations created for anatomy education, patient educa- tiary motions are relevant, e.g. for longer non-temporal animations tion or surgical training may be unique, i.e. exactly one animation to provide smooth transitions between strongly different shots. is generated and frequently used (as it is). They are often based on data of a healthy and “normal” person, e.g. the Visible Human As a final classification, based on Parent [Par12] and Stoll- dataset, abstracting from any anatomical and pathological variations fuss [Sto17], we mention of a patient. Re-use in the sense of adapting the animation to a simi- lar situation is of minor importance in these educational situations. • Artistic animation “in which the animator has the prime responsi- However, in routine diagnosis and therapy planning, only patient- bility for crafting the motion” [Par12]. specific animations are meaningful. It is essential that they can be • Data-driven animation, “where live motion is digitized and then created in a cost-effective manner, e.g. without a large amount of mapped onto graphic objects” [Par12]. user input. The idea of re-using an animation is conceptually similar • Procedural animation, “in which there is a computational model to example-based animations [WL08]; a concept that was used, for . . . used to control the motion. Usually, this is in the form of setting example, for clothing animation [WHRO10]. Although similar, the conditions for some type of physical . . . simulation” [Par12]. concepts are not identical. Wang and Lee [WL08] use different ani- We will see example for each of these categories. mations to compose a new animation. We consider re-use, as defined by Muehler et al. [MP10b], as the transfer of a single animation to similar datasets. 2.3. Tools for Animation Design In addition to effectiveness, standardized approaches to anima- Animations are often created with professional general purpose ani- tion generation also ensure reproducibility, e.g. animations are less mation software. In the articles discussed in this paper, 3D studio dependent on the actual user. There are considerable efforts in all max [MZL10], Cinema4D [Her02], Maya [COH∗02] and Caligari areas of radiology to standardize documentation, e.g. with respect to TrueSpace [LBR04] are mentioned. These animation tools are in- protocols and sequences as well as terminology. The incorporation tended for digital artists and animators. They provide support for of 3D visualization and 3D animation in (standardized) documen- keyframe animations, e.g. the author defines a set of keyframes tation obviously requires also to standardize the use of algorithms
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda and parameters of (dynamic) visualizations [HNN∗03], including al. [RWIB∗07] who provide a GPU-based solution for the effec- transfer function specifications, viewing directions and clipping tive computation of the video sequences. Thus, the computation of planes. Ideally, the expert knowledge of a physician is only neces- transfer function parameters, the direct volume rendering and video sary to create a first or very few animations and further animations encoding are carefully distributed to achieve optimal load balancing. are automatically adapted based on these examples. We found two publications driven by this goal related to the use case of cerebral 3.2. Re-use of Animations for Surgery Planning aneurysm diagnostics [IHT∗02] and planning surgery in case of neurovascular compression syndromes [HNN∗03]. Surgery planning has similar requirements than diagnosis: A precise understanding of the pathology and the spatial relations around it is A simple type of animation frequently used for diagnosis and re- essential for surgery planning as well. Additional requirements relate porting is “a rotating cineloop to convey the 3D structure” [SSN∗98], to the discussion of different surgical strategies, e.g. the selection of which is primarily applied to maximum intensity projection (MIP) an implant, the selection of an access path or the choice of a more images [SGS95] and local MIP [SSN∗98]. In these loops, the virtual or less radical intervention. camera is moved horizontally around the center of the dataset, thus displaying the relevant vascular structures from a large number of Muehler et al. [MP10b] discussed the use of animations for viewpoints. Most vascular structures appear unoccluded in one of surgery planning (oncologic neck and liver surgery planning). They these views. The precise coordinates of the viewpoint depend on the emphasized that animations may summarize a longer individual bounding box of the dataset. In this section, we discuss attempts to planning process for collaborative discussions, e.g. in a tumor board. create animations that are adapted to other cases as automatically as This requires an easy generation of animations and the re-use of one possible. We start with a special example from neuroradiology, the animation for similar cases. For tumor surgery planning, access plan- diagnosis of cerebral aneurysms (Sect. 3.1) and go on with examples ning and the assessment of infiltrations are typical examples. Access from surgery planning (Sect. 3.2). planning can be supported by an animation that moves the camera gradually from outside closer to the tumor, eventually combined with fading out occluding structures. The assessment of potential 3.1. Re-use of Animations for Cerebral Aneurysm Diagnostics infiltrations requires to study the tumor and nearby structures, such For diagnosis, it is essential that a pathology is detected, that its as vascular structures. A careful investigation from many different shape can be recognized and assessed in detail and that the shape viewpoints is necessary, i.e. an animation where the infiltration of all can be quantitatively assessed, e.g. its size or volume is determined. potential risk structures is displayed sequentially. For each structure, Based on such information radiologists report on the existence of a rotation around the tumor from a fixed distance (orbiting) in an a pathology and the stage or severity of a disease. 3D visualiza- appropriate speed is needed. If such an animation is defined once, tions and animations may be beneficial in case of pathologies with a e.g. for a neck tumor, it can be adapted to many similar cases. complex shape that are partially occluded or otherwise difficult to as- The system described by Muehler et al. [MP10b] provides sub- sess, e.g. vascular pathologies. Vascular pathologies include plaques, stantial support for animation generation. Automatically generated stenosis, aneurysms and arterio-venous malformations (AVMs). We and manually selected viewpoints can be employed to define a describe the (re)use of animations for diagnosis aneurysms that may smooth camera path. So-called key states can be defined and stored occur in the whole arterial system, e.g. abdominal aortic or cerebral for later re-use with a different dataset. aneurysms. Slice-based animations. Muehler et al. [MP10b] also consid- Cerebral aneurysms are dilations of a cerebral artery that are ered slice-based visualizations and provided means to animate them. relevant due to the involved risk of rupture. Iserhardt-Bauer et Thus, the system navigates the user within the slices and emphasizes al. [IHT∗02] created standardized animations to support a systematic important pre-segmented structures with either colored contours or search for such aneurysms based on locations where most aneurysms a certain fill style. The slice-based animations may be structured occur. A posterior overview and different lateral views are chosen to in meaningful parts, e.g. assessment of lymph nodes of a particu- analyze selected cerebral arteries. CT angiography datasets are em- lar region. The re-use requires that the same structures indeed are ployed and subvolumes are extracted in a standardized way. Clipping assigned the same labels. planes are inserted at specific landmarks and the transfer functions Planning neurovascular compression syndrom surgery. The are adapted to a new case in a clearly defined manner (the basic neurovascular compression syndrome is characterized by vascu- strategy for this adaptation is a histogram analysis involving also lar structures that touch the intracranial nerve which may lead to derived data, such as gradient magnitude). Instead of one large video considerable pain that requires neurosurgical treatment. Vega et sequence, five smaller sequences are generated. Each consists of al. [HNN∗03] use the techniques developed in the same group (re- circular 360 degrees flight around a subvolume. The videos are call [IHT∗02]) to create standardized videos for surgery planning. rendered on a graphics server and distributed via web clients. In The videos were used for rehearsal in the operating room where an evaluation, it could be shown that 18 out of 19 aneurysms (in they could be watched via a laptop. The animation generation could patients) were found by analyzing the automatically generated video be controlled by parameters, such as the temporal resolution, the sequences. The aneurysm that was missed was occluded by bony number of steps and the radius of the sphere on which the virtual structures—a situation where even expert radiologists may miss an camera rotates [HNN∗03] (see Fig.1). In the terminology of diBiase aneurysm. et al. [DMKR92], this represents an on-demand computation of an The original approach was later refined by Roessler et animation that is primarily used to explore the data.
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda
Figure 2: Animated pathlines represent simulated blood flow over time. In the scheme, one pathline enters the aneurysm whereas the second remains in the parent vessel. The part of the pathline that corresponds to the current time is emphasized by color. A Gaussian was used to modify the color saturation with a peak at the current Figure 1: Animation generation: The small spheres on the right time point (From: [LGP14]). indicate the camera path. The size of the sphere, the number of steps and the temporal resolution can be adjusted. The left view serves to test the current specification (From: [HNN∗03]). is therefore necessary as a pre-processing step [POM∗09]. We will not discuss motion correction here. Instead, we assume that these 4. Temporal Medical Animations problems are either not severe and can be ignored or they are already According to the terminology used in cartography [KEM97], we solved with appropriate methods. discriminate temporal and non-temporal animations. A temporal Animated Display of Blood Flow Data. Lawonn et al. [LGP14] animation (in cartography) depicts spatio-temporal patterns, such animated pathlines, representing simulated blood flow in cerebral as the development of crimes in different regions (crime mapping), aneurysms, i.e. dilatations of the cerebral arteries (recall Sect. 3.1). population growth, migration patterns or the spread of an infec- The animation conveys differences in speed which is essential for tious disease. Thus, the depicted changes indicate how values in risk analysis. High-speed flow that enters the aneurysm and hits the different regions change over time. A non-temporal animation (in wall is related to an increased risk. The display of the pathlines is cartography) presents static data focussing on different regions, typ- adjusted such that regions of the pathline that are close to the vessel ically combined in a smooth movement. As an example, to study wall are emphasized by appropriate colors (see Fig.2). This is large scale phenomena, a globe is slightly rotated to display geo- valuable since the spatial relation between the pathline and the wall referenced data in different parts of the world. Thus, geometric of curved vessels may be difficult to perceive, e.g. due to occlusion. transformations, such as rotation, zooming, and translation, as well Whether these animations correctly convey all relevant information as emphasis techniques, such as modifications of colors, are typi- or are too difficult and cause change blindness, was not tested. cal examples for non-temporal animations. In the terminology of Parent [Par12], temporal animations are primarily data-driven, rep- Adaptive Temporal Scale. Temporal animations are typically resenting primary motion captured in dynamic medical image data. characterized by a constant temporal scale, i.e. a factor that trans- We also consider procedural animations, e.g. animations resulting lates world time to display time. This is usually beneficial, in par- from biomedical simulation. For the visualization, the simulated ticular if users are accustomed to such animations, e.g. physicians character is often not directly addressed. The generated animations that routinely analyze dynamic medical image data. However, a strongly depend on assumptions and parameters guiding the simula- constant temporal scale implies that some parts of an animation are tion. Thus, a procedural animation may also be employed to study less interesting, e.g. because the rate of change is low or the changes the plausibility of the underlying simulation model. are not essential for the diagnostic task. On the other hand, other parts of the same animation may be considered too fast to recognize and interpret changes. When analyzing cardiac blood flow data, for 4.1. Temporal Animations of Raw Medical Image Data example, physicians look carefully when vortical behavior is visible Temporal animations in medicine primarily relate to dynamic med- and wait when no such behavior occurs. Based on these observa- ∗ ical image data, such as MR perfusion, 4D PC-MRI data, and 4D tions, Koehler et al. [KPG 16] described the choice of an adaptive angiography data. Animation is a also natural choice to present the temporal scale based on an appropriate “interestingness” measure. results of unsteady simulations, e.g. in hemodynamics, biomechan- The system is applied to 4D PC MRI data and the interestingness ics or biophysics. measure is based on vorticity, e.g. more time is spent on temporal intervals with a large amount of vortical behavior, whereas intervals Compared to cartographical animations, one major challenge in where the flow is laminar are displayed faster. The underlying algo- the animation of dynamic medical image data is due to the fact that rithm is inspired by histogram equalization, i.e. it aims at a constant a living patient is imaged. Thus, breathing, pulsatile blood flow, amount of feature visibility over the whole animation. muscle relaxation and other physiological processes hamper the interpretation of four-dimensional data (3D+time). As an example, The feedback from physicians was positive in general. The physi- if the contrast enhancement in the female breast is analyzed to cians emphasized that it is necessary to have an option to disable characterize a suspicious lesion, the lesion is moved primarily due to this adaptive behavior. Thus, the adaptive temporal scale may not re- breathing and therefore a voxel with coordinates x,y,z at time ti often place the constant scale, but it gives a valuable additional perspective. does not correspond to a voxel with the same spatial coordinates at Another aspect of the physician’s feedback was that the adaptive time ti+1. Motion correction, actually a type of image registration, character needs to be communicated by an appropriate legend. In
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda particular, because constant scale animations are so widespread, scenarios of a crime [MZL10]. An essential basis for the animation users need to be made aware if a different scale is employed. of biomechanical movements is the H-Anim ISO/IEC standard that comprises a model of the human anatomy consisting of joints and This strategy may be applied to a wide range of dynamic image segments [JB08].† The movements resulting from using H-Anim data and interestingness measures that may be interactively adjusted. are based on Newtonian’s laws of motion (scientific animations). As an example, also other flow features or the maximum speed may They are employed, e.g. for forensic investigations [MZL10]. Based influence the local temporal scale. on H-Anim, data from a CT or MRI scanner can be employed to Dynamic Contrast-enhanced Perfusion Data. Perfusion data extract a skeleton and match it with the abstract data incorporated represent the contrast enhancement over time. They are acquired, in the standard. Then, the model may be animated displaying cor- e.g., to analyze the perfusion of the brain after an ischemic stroke or rect movements. The H-Anim standard incorporates a hierarchy of to analyze the perfusion of the heart muscle after an infarction. In joints, e.g. when the arm is moved, the hand is moved as well. The these examples, the temporal resolution is rather high and therefore joints are modeled at four “levels of articulation” [MZL10]. The an animation can be generated that represents sufficient temporal coarsest model consists of an 18-joint skeleton; more detailed levels continuity. The animations cycle through the time frames and are comprise 71, 94 and 144 joints respectively. H-Anim also provides valuable “to assess image noise and artifacts, but especially for the a Web3D-capable motion viewer that may export animated films. assessment of enhancement patterns” [POM∗09]. The use of ani- H-Anim is used in particular to move the whole human body—not mated functional image data for quality control was also emphasized just the skeleton. Thus, skin vertices for example are also moved in by Choyke et al. [CDK03]. a realistic manner when associated joints are moved. Estimating and Visualizing Heart Motion. In the diagnosis of the coronary heart disease, animation is also used to study the ven- 4.3. Temporal Animations of Map-Based Medical Data tricular wall motion. The most abnormal behavior is represented by In medical visualization, projection-based (or map-based) tech- akinetic regions that do not contribute to the wall motion, typically niques are often used to reveal the distribution of parameters at since this tissue is ischemic after an infarction. Heart motion is usu- a glance, i.e. without occlusion problems inherent to 3D visualiza- ally assessed with 2D echocardiography where physicians analyze tions [KMM∗18] For dynamic data, e.g if the results of an unsteady the wall motion in a few selected slices and mentally combine their flow simulation are projected, map animations arise. Value changes, impressions to a model of the complex movement and its pathologi- e.g. pressure or blood flow velocity, are typically color-coded lead- cal variations. Already in 1987, Jilin et al. [JAES87] described how ing to dynamic heat maps. Like other temporal medical animations, the information extracted from individual slices may be combined the temporal scale needs to be carefully chosen. Also other aspects, in a smooth 3D animation. Jilin et al. suggested to manually contour e.g. the use of temporal smoothing to avoid short-term fluctuations, four slices representing two longitudinal and two cross-sectional require careful consideration. While in similar cartography anima- slices (both at end-systolic and end-diastolic state) and to interpolate tions considerable experience exists how to reduce map complex- between them with splines. ity [TM18], we found no medical map-based visualizations taking The visual inspection may be enhanced by a quantitative analysis map complexity into account. of the velocity magnitude and direction. Suehling et al. [SAJ∗03] in- Spatial epidemiology is an area that is at the intersection between troduced optical flow methods to fit velocity distributions to echocar- medicine and geosciences. In spatial epidemiology, the incidence of diography data and superimposed either velocity magnitude (color- diseases, health disorders, such as fatty liver, and potential risks, are coded) or velocity direction (using arrows) on the animated display. depicted on maps (typically with data aggregated over administrative Since then substantial research was carried out to improve motion units, so-called choropleth maps). Such maps reveal disease clusters estimation. However, no recent publication discusses visualization and enable an in-depth analysis [CRN∗08]. Typically, the underly- techniques to display the results in appropriate animations. ing data is time-dependent and thus temporal visualizations, such as change maps or animations are generated. General issues of ani- 4.2. Temporal Animation in Functional Anatomy mated choropleth maps are discussed by MacEachren et al. [MD91]. MacEachren et al. [MBHP98] show medical examples of map ani- Functional anatomy comprises an understanding of the relation mations. Animations in their system are generated on the fly after between anatomical structures and their function. This includes the user specified a temporal interval that should be shown. More- the movement of joints (biomechanics), the pumping of the heart over, attributes, such as the incidence of heart diseases among man muscle and the pumping of the Eustachian tube [HH95]. Computer- and the unemployment rate, may be chosen. Such bi-variate map generated animations to convey these functions are typically based animations require to discretize the values or even to binarize them, on an explicit modeling of these movements. Thus, in contrast to e.g. regions with elevated numbers and numbers below a threshold measured or simulated blood flow, there is no predefined and precise are distinguished—leading to four different value combinations. temporal scale. However, we consider these animations also as ∗ temporal, since a dynamic process is displayed and the modeling Iqbal et al. [IHN 16] focussed on cancer prevention analyzed as- typically aims at realistic and physically-based movements. sociations between chronic diseases, such as diabetes, and common Functional anatomy is not only essential within anatomy edu- cation but also in areas, such as forensics, where possible recon- † http://www.web3d.org/working-groups/ structions of body posture may be important to understand possible humanoid-animation-h-anim
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda
Table 1: Major temporal animation systems in medicine (ordered chronologically).
Data Application and Major technique Key Publications 2D Echocardiography Diagnosis of heart motion, interpolation [JAES87] Video-captured data Anatomy education, Eustachian tube [HH95] 2D Echocardiography Diagnosis of heart motion, motion estimation [SAJ∗03] Functional image data Quality control [CDK03] Spatial health statistics Spatial edidemiology, map animation [CRN∗08] Modelling with H-Anim Forensic investigation [MZL10] Simulated Bloodflow Aneurysm Diagnosis, animated pathlines [LGP14, MEB∗17] Measured Bloodflow Cardiac diagnosis, adaptive temporal scale [KPG∗16]
types of cancer, e.g. for breast and colon cancer. They displayed 5.1. Viewpoint Selection these associations for the entire population (based on 782 million patient visits in Taiwan) and for persons of different age groups. For non-temporal animations a camera path is valuable to show a The animation in their CAMA (Cancer Association Map Animation) complex anatomy virtually from different perspectives. The defini- system conveys how the prevalence of chronic diseases and cancer tion of a camera path can be supported by algorithmic components. change depending on age. Inspired by Hans Roslings dynamic bub- The overall task breaks down in the selection of good viewpoints and ble charts, they employ circular glyphs that change their position the combination of these viewpoints to a smooth movement. Finally, in the scatterplot to represent the dynamic character. The glyphs the speed of the camera movement may be adapted, e.g. close to a represent the co-morbidities of chronic diseases and cancer. key viewpoint the camera may slow down, pause a few seconds and Temporal Re-expression. An idea from cartographic animation speed up again. The selection of good viewpoints can be guided by that is potentially useful for medical animation is to change the geometric considerations. Some target structures should be visible presentation order to convey relations and trends not easily recog- at least to a large extent and their projection should not be too small. nizeable in the original sequence of data. Kraak et al. [KEM97] Viewpoint entropy, a measure from information theory, turned out to suggest to display periodic data aggregated over the months of the be a good quality criterion for selecting good viewpoints [VFSH01]. year, such that firstly all January measurements are shown, then all In essence, a minimum set of viewpoints is selected that conveys February measurements and so on. For epidemiology data, such the maximum amount of information. Since viewpoint entropy is animations would reveal the variability of the incidence of influenca highly sensitive to the triangulation, Sbert et al. [SPFG05] suggest within a season, whereas a conventional display would emphasize another measure based on the Kullback-Leibler divergence. the seasonal differences. Data that is dependent on the heart beat may be re-expressed such that all peak systolic data are displayed Viola et al. [VFSG06] and Muehler et al. [MNTP07] sampled a sequentially and all peak diastolic data as well. Table1 summarizes sphere around the scene of the relevant anatomy to determine good temporal animation systems. viewpoints. Viola et al. [VFSG06] determine one characteristic viewpoint for each object in a pre-processing step.
5. Non-Temporal Medical Animation Muehler et al. [MNTP07] also integrated preferences of surgeons, A wide variety of animation techniques have been developed that e.g. views that are familiar to them. Furthermore, a stability cri- use animation to convey the complex spatial relations within the terion was incorporated: A viewpoint is only selected if possible human anatomy. In the terminology of Parent [Par12], these anima- viewpoints in the neighborhood are appropriate also. Thus, the cam- tions are artistic, they are neither data-driven nor procedural (guided era stays for a while in a beneficial region, e.g. to see a tumor and by a simulation). Most of the non-temporal medical animations are its surrounding. This stability criterion also supports the interac- based on surface models derived from segmented medical image tive exploration, i.e. if the animation is interrupted and the user data. But also animated volume renderings have been employed, e.g. initiates incremental changes of the camera, the target structures for virtual endoscopy or for presenting different possible variants remain visible. Viewpoint selection algorithms are also available for of the anatomy as a result of uncertainty characterization. In this volume-rendered images, e.g. [ZAM11]. There is a large variety of section, we discuss algorithms that are essential for medical ani- viewpoint selection methods, see the survey article by Bonaventura mations without details about specific applications. We start with et al. [BFS∗18]. the computation of good viewpoints (Sect. 5.1) and continue with camera path planning (Sect. 5.2), which comes in two flavors: the Once viewpoint candidates are determined, a selection can be computation of a path to show a patient model from outside and the computed taking into account that the resulting set should be as computation of a path for a fly-through, e.g. inside the bronchial tree. diverse as possible. Thus, it is ensured that a target structure is really We discuss strategies to annotate (medical) animations (Sect. 5.3) seen from significantly different perspectives. Viewpoints are also and we discuss scripting languages that were developed to translate characterized by the distance to the interesting structures. Often, it high-level instructions in the low-level commands for a graphics is desirable that the camera maintains a fixed distance—a type of library (Sect. 5.4). navigation known as orbiting.
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda
5.2. Camera Path Planning in the data. Thus, while labels may be placed at regions where they initially do not occlude important parts of the data, they may later Finally, the selected viewpoints need to be ordered, e.g. a meaningful occupy salient regions, e.g. where a vortex arises. Goetzelmann sequence is established. This sequence of viewpoints is connected, et al. [GHS07] suggest the use of video processing techniques to typically along a path computed as a spline, following the arc prin- identify calm regions with little changes that may serve for label ciple stated by Lasseter (recall [Las87] and Sect. 2.2). There are placement. Their paper employs examples from engineering, e.g., many variants, how viewpoints may be connected to a camera path. labeled animations that explain complex engines. The strategies can Muehler and Preim [MP10a] discuss the following criteria: be translated to 3D models of a patient anatomy. • The path should be as short as possible. • A path should be chosen where relevant information is displayed during the whole camera movement. 5.4. Scripting Languages • Unfamiliar or uncomfortable viewpoints should be avoided. Modern visualization toolkits, such as Amira [SWH∗05], contain Obviously, trade-offs are necessary. The shortest geodesic path simple and general scripting facilities to generate camera movements between a set of points on the surrounding sphere does not neces- and object transformations. These, however, are not sufficient, e.g. to sarily reveal relevant information. It may also contain unfamiliar effectively explain the anatomy in a particular region or to support viewpoints. Thus, a weighted combination of the criteria may serve a tumor board discussion on the resectability of a tumor. In the as objective function for an optimization process. Camera path plan- following, we describe more powerful mechanisms for animation ning may be combined with adjustments of transparency to ensure generation, including scripting languages. the visibility of essential anatomical structures. As an example, if a The generation of animations based on a scripting language was pathology should be shown, occluding objects are rendered semi- first described by Catmull [Cat72] and later by Zeltzer [Zel91]. transparently as long as they actually occlude the target object. Viola Zeltzer [Zel91] refers to the level on which an animation is specified et al. [VFSG06] discuss primarily how two viewpoints v and v — 1 2 as the task level where visualization goals are formally described. An each serving to emphasize one structure—may be connected also example for a task may be ”show object
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda
Figure 3: Two screenshots from an animation for anatomy educa- Figure 4: The two instructions (on top of the images) specify an an- tion: The initial view of the foot anatomy is rotated; a pen points at imation. The numbers in brackets specify the temporal interval. The the origin of a muscle and a ligament that partially occluded the “sceneIntroduction” creates a horizontal rotation. The “viewDis- muscle was made transparent. A textual explanation is displayed. tance” statement creates an animation that zooms to the two spec- In the further course, arrows follow the different branches of the ified objects and integrates the display of their minimum distance muscles guided by its centerline (From: [PRS96]). (From: [MBP06]).
Preim et al. [PRS96] adapted these strategies to generate ani- an interval, i.e. an instruction starts and ends at an absolute time mations for anatomy education. In addition to the objects of the (in seconds). Thus, the execution of instructions may overlap, e.g. scene, also meta-graphical symbols, such as arrows could be used a second instruction specifies a movement that starts two seconds to explain anatomical structures. Moreover, textual labels and multi- later and ends at the same time as a first instruction. Figure4 gives line explanation were available and could be integrated in a smooth an example for an instruction and screenshots from the generated animation. For this purpose, continuous zooming techniques were animation. The terms used in the scripting language are related to employed to provide the space to include the textual components. therapeutic questions. Also the scripting language of Muehler et At the task level, authors could specify which objects or groups al. supports primary motions, e.g. motions of clipping planes and of objects should be explained. Their IllustratorControl language secondary, but no tertiary motions. provides commands to further specify how objects should be ex- plained, e.g. whether an arrow should be used to show an object (see Discussion. Although individual scripting languages provide con- Fig.3). The basic idea of this language was to generate expressive siderable support specialized on a number of medically relevant animations even with very short descriptions, but to enable more tasks, they are not comprehensive. As an example, no medical ani- fine-grained control optionally. In particular, authors can specify mation system provides support for tertiary movements, e.g. typical with how much detail an object is explained. The author does not transitions between scenes. need to specify timings—the inbuilt rules compute how much time is needed for certain changes based on observations of what can be interpreted. However, this also represents a limitation, since authors 5.5. Navigation-Assisted Animation Design may want to deliberately deviate from the default animation speed. The IllustratorControl language supports primary motions, e.g. mo- Liao et al. [LHM14] introduced a completely new approach to tions of arrows and other pointing devices and secondary motions, animation design. Instead of keyframing or scripting languages, but no tertiary motions. they exploit characteristics of the exploration process of domain scientists to create animations. This is surprising first, because an The interpretation of the script controls an Open Inventor program, exploration process with a considerable amount of trial-and-error where primarily the support for interpolation was employed. In differs strongly from a presentation situation that summarizes the addition to camera movements and transformations of objects, also findings. However, the trial-and-error phases are removed and it the appearance could be adapted, e.g. in a smooth transition. As an turns out that users created animations with a similar quality like example, the transparency of an object could be increased to enable with a keyframe-based system in a shorter period of time. The use the recognition of an occluded object and to fade out an outer object, case considered there was the analysis of a time-varying hurricane e.g. an organ. dataset. Muehler et al. [MBP06] also provide a scripting language for gen- erating medical animations with a focus on intervention planning. 6. Selected Applications They also incorporate movements of clipping planes, the presen- tation of slice-based visualizations and simple animated volume In this section, we give an overview of applications where animation renderings based on changes of 1D transfer functions. Clipping was employed to support medical education, diagnosis, treatment planes may be selective, e.g. they may be applied to a subset of planning and also forensic use cases. We comment, if possible, on all anatomic structures. Like in other systems, camera positions the specific animation design and evaluations that assess the value of are specified relative to objects instead of using absolute coordi- animation, typically by comparing with other modes of information nates. The time in this scripting language is explicitly specified as presentation.
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda
6.1. Medical Education Animations were developed and evaluated in a variety of medical education applications, including anatomy education, embryology (neonatal development), surgery training, histology, cellular pro- cesses and regional anesthesia teaching. The survey article by Ruiz et al. [RCL09] lists 13 papers using animation for medical education. Animations for educational purposes should be designed with the findings of learning theory on the reception of animation in mind (recall Sect. 2.1). Anatomy Education. The use of 3D visualization for anatomy education was widely discussed including commercial systems [PS18]. However, only very few systems provide animations or explicitly discuss their use for anatomy education. Pioneering work was carried out by Habbal and Harris [HH95]. They argue that the potential of animation is particularly high for displaying functional Figure 5: An exploded view of the skeletal anatomy of the foot was anatomy (recall Sect. 4.2). In their pilot study they employed draw- created in an animation. The exploded views are embedded in a ings of the human heart, imported them in a professional tool (3D comprehensive anatomy education system (From: [RPDS00]). Studio Max) and used keyframing to create animations that depict the complex relation between the valves and the ventricles. They included movements of incompetent heart valves to illustrate this pathological process. The whole animation design fulfils their phi- sizes the possibility to display objects from unusual viewpoints that losophy: “A successful animation requires three mutually dependent are hardly possible, e.g. when dissecting a cadaver. components: good artistic modeling, high quality images and a story Surgical Training. Dynamic presentations play an essential role to tell” [HH95]. Most of the paper deals with strategies to cope with in surgery training. However, intraoperative video is the presenta- the hardware limitations at that time. tion mode. As an example, WebSurg (https://www.websurg. com Based on the scripting language described in Sect. 5.4, Preim et ) provides thousands of videos for explaining a large variety of al. [PRS96] discuss the process of generating animations to convey surgeries. The videos are structured according to major steps of the the spatial relations between bones, muscles and ligaments, using surgery and contain textual explanations in the style of PowerPoint examples from the foot and knee anatomy. The animations contain slides. In addition, computer animations often show slice-based vi- rotations to show particular structures, the display of instructional sualizations and related annotations that make the user familiar with text, e.g. the innervation and origin of a muscle, and emphasis by the pathology and the surrounding anatomy. changing colors. Inspired by the way lecturers explain elongated Cutting et al. [COH∗02] discuss the use of 3D animations for structures, such as muscles, pointing devices are used to illustrate the surgery training and provide evidence that 3D animations are valu- course of such structures (recall Fig.3). The underlying geometric able for surgery training. They deal with cleft lip and palate surgery models consisting of a few dozens of labeled anatomical structures and developed an animation based on polygonal models of skin, were created by digital artists and acquired from a commercial bone, cartilage and muscles. The models were smoothed, and sim- vendor. Thus, the models are not patient-specific—an aspect that plified and imported in Maya (Alias/Wavefront, Toronto, Canada). was considered essential for educational aspects [McG10]. The animations show the action of a scalpel. We briefly discussed the use of animated exploded views for The major problem is that the use of a scalpel changes the geome- technical illustration (recall Sect. 5.4). The same principles have try radically: Incisions alter the topology of the underlying geometric been employed in an anatomy education system to reveal spatial models. Maya and other commercial animation systems do not pro- relations [RPDS00], see Fig.5. Exploded views may be achieved by vide support for adequately treating such topology changes. As a slightly scaling down anatomical structures, thereby increasing the consequence, Cutting et al. prepared many small animations (with- gap between them. Animating the transition between the original out topology changes) and combined them in a final step [COH∗02]. state and the exploded view reduces the mental effort to understand Since the anatomic model contains different layers, transparency the relation between both. is used to display the skin and inner layers, e.g. the nasal cartilage complex, simultaneously. Movements of the skin are propagated Vernon et al. [VP02] discuss teaching aids, including 3D models to the (linked) inner layer representing cartilage and muscles. The and (only briefly) animation for medical teaching. The paper pro- animation is actually a hybrid between temporal and non-temporal vides recommendations, largely focussed on commercial tools and animations. The temporal component is a simulation of the pump their use. Keyframe animation and motion capture to support biome- mechanism of the Eustachian tubes, whereas camera movements chanical animations are emphasized. Compared to the large number and zooming operations represent the non-temporal component. of systems that provide interactive exploration of 3D models for anatomy teaching (recall [PS18]), the use of animation in this area Endoscopic Ultrasonography. Endoscopic ultrasonography is is limited. Fisk [Fis08] discusses the value of animations for various an essential imaging technique that requires considerable experi- medical disciplines. With respect to anatomy education, he empha- ence to be used effectively. Burmester et al. [BLH∗04] developed
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda a training system that supports the understanding of this imaging comprises major steps of the intervention and the risk for associ- technique. The system was based on the VOXELMAN, a compre- ated complications. In the following, we describe three animation hensive anatomy education system, that employs the Visible Human systems for patient education. dataset. Ultrasonography images were simulated to create appropri- Hermann [Her02] compared the quality of patient education with ate animations. The basic functions to control a video, e.g. playing 3D animation with a conventional approach based on textual infor- forward and backward, were supported. mation only. The regional anatomy around the thyroid gland and Regional Anesthesia Teaching. “Regional anesthesia requires basic steps of surgery are shown in an animation created with Cin- clinical skills of patient positioning, surface marking, and needle ema4D. The specific complications that may arise related to anatom- manipulation” [LBR04]. Lim et al. observed that training opportu- ical structures, e.g. nerves, in close proximity are also displayed. nities largely depend on suitable patients and decreased over time. The control group had 10 minutes time to read a textual explanation In their specific example, they aim at a training of the interscalene carefully adapted to patients. The animation group (both groups brachial plexus block, a regional anesthesia that is indicated for arm comprise 40 patients each) reported significantly better subjective and shoulder operations. understanding and more trust. The patients considered an abstract graphical display also as superior to a video sequence showing the Lim et al. prepared a number of small animations, lasting between actual surgery. The description, however, is not comprehensive. No 2s and 8s, showing individual steps of the procedure. TrueSpace details about the specific design of the animation, e.g. duration, from Caligary was used (the system is no longer available). A neck speed, camera paths, are given. The screenshots indicate that the model was prepared and used for selecting good viewpoints to visualizations are rather simple and clear with high contrast. The demonstrate surface landmarks and the needle movement. Similar anatomical structures are strongly smoothed and thus easy to per- to Cutting et al., Lim et al. [LBR04] emphasize the importance of ceive. The animated anatomy is not patient-specific. It was prepared transparency to convey the spatial relations. They also emphasize once and re-used often to justify the expense of around 20.000 Euro. the role of incorrect handling of the needle and their consequences as part of the generated animations. All animations were integrated We already described the SinusEndoscopy system (re- in a PowerPoint presentation to convey the overall process. call [KKSP08]). This system was also used for patient education. Thus, the surgeon explained the surgery and associated risks based The evaluation was carried out with 24 users ranging in their on the specific data of that patient. The virtual camera moved along experience from those with no experience in this area to experts. a precomputed path leading to animations that last about 40 sec- The large majority (21 users) stated that the animations enhanced onds. The animations were presented by the operating surgeon on their anatomical and technical understanding. Users also had to set a large 40 inch monitor to explain the surgical strategy to the pa- surface markers for needle placement. Those with little to moderate tient. The evaluation revealed that the large majority of 127 patients experience improved their performance significantly after the pre- found these animations helpful. Additional comments reveal that sentation with the integrated animations, leading Lim et al. [LBR04] patients gained trust in the procedure. The medical implications are to the conjecture that “less advanced trainees would probably benefit described in a medical journal paper [SLF∗09]. more from didactic teaching. Therefore, we chose a less interactive approach.” The realistic evaluation is a strength of the paper. Unfor- McGhee [McG10] describes an approach to generate animations tunately, little details about the animation design are given. Thus, it that emphasize aesthetics, appeal and artistic proficiency to engage is not clear which decisions lead to this favorable outcome. learners or patients. As an example, material properties and shaders are carefully tweaked to achieve highly reflective vessel visualiza- Discussion. 3D animations are particularly useful for training tions (derived from clinical MR angiography). They contain blood procedures where they are used to explain individual steps or the cells and indicate blood flow during the cardiac cycle (see Fig.6). composition of a procedure based on these steps. Carefully designed McGhee also emphasizes the “narrative”, the story that is conveyed animations use different viewpoints that provide an unobstructed by means of dynamic images to “engage students and improve their view to the relevant anatomical structure and instruments, including understanding of disease . . . and treatment” [McG10]. McGhee also viewpoints that are impossible to realize when preparing a real video. provides a discussion of further evidence that pictorial content is ben- Transparency is another powerful mechanism in a 3D animation to eficial for patient education. In conclusion, animation is beneficial convey spatial relations, e.g. within a transparently rendered organ. for patient education in case of planned surgical interventions. Moreover, the visual display in an educational 3D animation is clearer and easier to interpret compared to an intraoperative video. For interventions, such as anesthesia teaching, a video could not 6.3. Virtual Endoscopy show interesting internal details. Cutting et al. [COH∗02] argue Virtual endoscopy, where a camera is moved inside the human body, that animation supports an early stage of learning, whereas more is probably the most important area for generating animations in experienced learners benefit more from active types of learning, e.g. medicine. Virtual endoscopy is an umbrella term: solving tasks in a virtual reality simulation system. • Virtual angioscopy relates to virtual endoscopy in the arteries, • virtual bronchoscopy to the inspection of the bronchial tree, and 6.2. Patient Education • virtual colonoscopy addresses the search for polyps in the colon. Complex spatial information needs to be conveyed in the process of An animated “fly-through” is generated to inspect the walls of patient education before surgery or interventions. The information tubular structures. Due to the complex shape of structures, such as
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda
Synchronized animation. In virtual endoscopy it is typically not sufficient to provide the relevant information. A virtual camera that moves inside a complex structure, such as the bronchial tree, reveals the local detail of the bronchial wall surface but does not convey where the camera is in relation to anatomical landmarks, i.e. the global context is missing. Virtual endoscopy systems therefore typi- cally generate synchronized animations that display the endoscopic view and a 3D view from outside indicating the current camera posi- tion and orientation. An alternative to this type of synchronization is the display of 2D slices along with the endoscopic view, where the currently displayed 2D slice contains a cross-hair cursor to indicate the current camera position. A split screen display where external 3D views and 2D slice views are synchronized with the virtual en- doscopy view was already suggested by Jolesz et al. [JLS∗97]. Even the three orthogonal slicing directions (axial, sagittal, coronal) are often displayed simultaneously with the endoscopic view to support Figure 6: The human aorta and some of its side branches extracted the interpretation of the current camera position and orientation in from MRI data and combined with a particle simulation with blood context (see Fig.7). On the other hand, this split-screen situation flows (From: [McG10]). raises interpretation problems of its own (recall Sect.2). Surgery planning. Virtual endoscopy is also useful for plan- ning endoscopic surgery [ÇK00, KKSP08]. The different target the colon, a keyframe animation based on user-selected key view- user groups lead to slightly different requirements related to the ∗ points is tedious [JLS 97]. Instead, the generation of a “fly-through” interactive exploration of the data. Surgeons may use an animated requires little user input, i.e. the user selects a start point and an fly-through to define a position where they cut in the tissue. Thus, endpoint in the target structure, e.g. the colon. they do not only want to interrupt an animation but also to mark a Camera path planning. Based on an automatic segmentation position or draw a line. Radiologists, on the other hand, may also and centerline determination, a path is created that connects the annotate an endoscopic view when they see a suspicious region. start and endpoint. All points along this path are inside the target Cakmak et al. [ÇK00] created textured geometric models from the structure, approximately in the middle. For strongly curved self- Visible Human dataset and assigned mechanical properties to realisti- intersecting structures, this is not easy to achieve. A focus of path cally deform medical tissue when forces, such as instruments, are ap- computation is to avoid collision with the wall of the target structure. plied. Mass spring models are employed for modeling the dynamic The centerline may be post-processed to better serve as a camera changes. Such functional animations may convey complications, path. It is often smoothed to avoid sudden changes of the camera such as bleedings, and the smoke when tissue is ablated [BGG98]. direction. However, collision avoidance always has higher priority. The centerline may contain many branches caused by bulges in the Krueger et al. [KKSP08] developed a virtual endoscopy system target structure. for surgery planning in the nasal cavity and patient education (recall Fig.7). The risk is highly patient-specific, e.g. only in some cases All these processes may be performed with default parameters surgery involves areas close to the optic nerve. Since there are no in a predefined manner and as a result an animation is created that tubular structures that may guide the virtual camera, the camera path smoothly connects the start and endpoint to enable the physician to is manually defined, e.g. with selecting points in the slices. The sur- inspect the wall of the colon or the bronchial tree. The sensitivity in geon used the generated animations for rehearsal. The information the detection of polyps along the wall may be improved considerably conveyed is highly welcome. On the other hand, the effort to prepare if the physician watches a second animation that displays the path such patient-specific animations is considerable and thus the system from the end to the start point. Some folds are better visible if the was no longer used after the study. camera looks at them from the opposite direction. Path planning for virtual endoscopy was first discussed by Lorensen et al. [LJK95]. We did not find any publications that describe a perception-based They adapted concepts from robot motion planning to derive a study in which information was actually extracted from such syn- path where the camera does not collide with anatomical structures. chronized animations. Obviously, the cognitive load of interpreting Hong et al. [HKW∗95] and Rubin et al. [RBA∗96] are other early synchronized animations is increased compared to the single endo- examples for the generation of offline animations serving as fly- scopic animation. through endoscopy. Discussion. Stollfuss [Sto17] discusses animated fly-throughs in Hong et al. [HMK∗97] described the first comprehensive virtual detail including the influence of varying parameters that control colonoscopy system including a careful discussion of useful inter- illumination, texture and other aspects that influence the perceived actions. The user is guided from the start to the target point and realism. From a didactic perspective he emphasizes that animated glides in the colon center—a navigation inspired by the submarine endoscopy is based on data and many decisions of an author. We metaphor. Bartz [Bar05] gives a survey on the first decade of virtual discussed already McGhee’s work to characterize the potential of endoscopy developments. aesthetically pleasing and informative animations for patient ed-
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda
of the optimization procedure, only the top 10 % of viewpoints are further processed. Moreover, a clustering algorithm is employed to group the remaining start points and a representative per clus- ter is used to determine an optimal viewpoint. Thus, views from different perspectives are generated around an aneurysm showing possible rupture-prone areas. Finally, a camera path gliding around the target structure at a fixed distance (orbiting) is used to create an animation. The evaluation indicates that the generated animation is considered useful for the assessment of the rupture risk. Manually selected views from medical experts showing interesting surface regions were quite similar to the automatically selected views with respect to camera position and view direction. However, manual exploration is required in addition, e.g. to explore small structures, such as blebs. The interesting situation in this work is that temporal data is analyzed and camera flight, a typical technique for non-temporal animations, is used to focus the analysis to relevant regions and temporal intervals. Thus, the analysis of 3D + time data may also benefit from path planning. Figure 7: SinusEndoscopy: A system that provides a fly-through the nasal cavities as preparation for surgery. The current camera 6.5. Forensics position is indicated in three orthogonal slices (left), whereas the large right view presents the endoscopic view (From: [KKSP08]). Animated visualizations in forensics are generated in case of shoot- ings or other severe crimes. They may integrate different viewpoints, representing, e.g. the position of witnesses. Animations integrate raw data, representing the patient anatomy, gun, or knifes, other ucation (recall [McG10]). Their focus was on enhanced virtual aspects of the crime scene and reconstructed data, e.g. the bullet endoscopy applied to the vascular system. path [MZL10]. The major use cases for such animations are to dis- cuss possible reconstructions with respect to plausibility and likeli- 6.4. Animation for the Diagnosis of Medical Blood Flow Data hood and to demonstrate the results to the judges in the court room.‡ “Physically accurate forensic animation may shed light on investi- In the field of medical visualization with respect to support the gating what exactly happened at a specific crime scene, causes and diagnosis and treatment planning of vascular diseases, the visual effects that embrace the issue who is at fault or guilty.” [MZL10]. exploration of patient-specific blood flow data plays a crucial role. To assess the severity of a disease and find appropriate treatment, This development is largely based on the increasing use of virtual morphological and hemodynamic aspects have to be analyzed simul- autopsy, i.e. post-mortem CT and MRI data [TBB∗05]. Since a taneously to understand their interplay. This is a challenging task cadaver does not move and radiation is not so strongly limited, the due to the complexity of the time-dependent data. anatomy can be displayed in high spatial resolution and with a good signal-to-noise ratio. The data acquisition may be carried out as We discussed cerebral aneurysms already in Sect. 3.1. Here, we a whole body scan or as a scan of different body parts that are discuss how animations may be used to enhance this diagnostic later combined [VOH17]. Due to the large size of the data, efficient process by incorporating simulated flow data. For this purpose, rendering is challenging [LWP∗06]. Figure8 shows an example of Meuschke et al. [MEB∗17] developed an automatic selection of a reconstructed crime based on post-mortem data. optimal viewpoints that are connected to a camera animation for the visual exploration of time-dependent blood flow data in cerebral Animations in forensics fall in this category. The whole animation aneurysms. With this system, suspicious surface, regions can be generation process aims at faithfulness and credibility. Aesthetic analyzed according to morphological and flow attributes without a considerations are not essential. Biomechanical animations are em- time-consuming manual exploration. ployed to discuss possible postures of a victim during a crime. The H- Anim standard is employed for this purpose, where a moderate level The selection of viewpoints is modeled as an optimization prob- of detail (level 2 with 71 joints) is considered appropriate [MZL10]. lem. It consists of three major steps. First, a target function is for- mulated that should be optimized. Second, start points for searching appropriate views are chosen that serve as input for the optimization 6.6. Animated Display of Uncertainty for Diagnosis problem. In the last step, the resulting viewpoints are connected to a In this subsection, we focus on a selected aspect of medical volume camera path. The target function considers visibility of the aneurysm rendering. The display of medical image data involves considerable surface part and an interestingness measure derived by two user- selected attributes, e.g. low wall thickness and high wall shear stress. For start point selection, an ellipsoid around the whole vessel sur- ‡ Ma et al. [MZL10] report that computer-generated animations are admitted face is sampled discretely. To reduce the number of necessary runs in the courtroom in the UK if correctness is carefully analyzed.
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda
Figure 8: Reconstruction of a crime scene. The cadaver dataset was imaged with CT and the puncture of the lung with a knife is clearly visible (From: [PCH∗05]). Figure 10: An articulated shape model of the knee with two parts (From: [ALvTZ19]).
6.7. Animated Display of Shape Spaces Many applications in medical imaging benefit from statistical shape and appearance analysis. Based on a larger number of ground truth segmentations, often a principal component analysis is employed to represent the major modes of variation. Such models are primarily employed for image segmentation, but they also capture knowl- Figure 9: Slightly different transfer functions strongly effect the edge relevant for anatomy education (understanding the variability appearance of a vessel. In a short animation, the possible visu- of anatomy) and for diagnosis (better understanding of the fuzzy alizations are integrated and support an effective interpretation boundary between healthy and pathological changes in particular for (From: [LLPY07]). elderly patients). Ambellan et al. [ALvTZ19] discuss the idea to use animation as an educational visualization to convey the shape space. Instead of showing many instances as small multiples along the 1st, 2nd and 3rd mode of variation, shapes are interpolated (morphing) from the lowest to the highest value along an axis. uncertainty due to the noisy character of the images and further arti- facts. Thus, a volume rendering with a certain transfer function (TF) Based on the interpolation, a seamless transition can be shown leads to one possible display of the anatomical situation. Slightly in an animation. While numerous talks on statistical shape models modified transfer functions lead to a different but also possible inter- contain such videos, this is hardly discussed in terms of animation pretation. While small changes in the transfer function parameters design. The temporal scale needs to be chosen carefully, taking the may lead to similar small changes in the visualization, which indi- amount of changes into account. Color may be used to emphasize cates stability towards these parameters, small changes may also strong deviations from the mean shape. Not only are the geometry lead to substantial diagnostically relevant changes. This holds in of shapes different, but also the appearance, the “texture”, e.g. of particular for the display of vascular structures in contrast-enhanced bones, differs, as an aspect of normal differences or diseases, such CT data. Radiology textbooks inform physicians that the diagnosis as osteoporosis. Animations may also be used to convey these dif- of a stenosis should not be based on one setting of a transfer function. ferences. Some anatomical structures exhibit a complex topology, Instead, different parameters should be employed as well. e.g. the hip or the knee. In these cases, shape models are derived for the individual parts and also the relative motion between the parts Lundstroem et al. [LLPY07] use animations to convey different may be captured—leading to an articulated model (see Fig. 10). An- TF settings. The border of a vascular structure is considered as an imations may be used to convey both motions of joints (functional instance of a probability density distribution (PDF). Based on an anatomy) and variability of the individual parts. appropriate sampling of the PDF, different instances are created, sorted according to similarity and combined in a smooth animation. Table2 summarizes temporal animation systems. It is striking A stenosis—based on this animation—is only diagnosed if it appears that most systems are based on some kind of CT data which is in all frames of the animation. A sensitivity lens is used to restrict probably due to the standardized intensity scale. Another interesting the animation to a (small) region, e.g. the region where a static aspect is that there is only one recent publications with a focus on display suggests a stenosis (see Fig.9). Thus, the foveatic vision non-temporal medical animation. is focussed on this area and reliably detects major changes if they would occur. Animation were used for visualizing uncertainty before 7. Interactive Animations in geovisualization [ESG97, Fis93]. Later, Akiba et al. [AWM10] discussed a transfer function overview template where the opacity Interaction is essential to control animations independent of the TF is animated to reveal all structures in a volume dataset. field of application. We employ cartography to discuss interactive
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda
Table 2: Major non-temporal animation systems in medicine (ordered chronologically).
Data Application and Major technique Key Publications Geometric models of the foot anatomy Anatomy education, scripting language [PRS96] Abdominal CT Virtual colonoscopy, path planning [HMK∗97] Abdominal CT Virtual colonoscopy, combined 2D and 3D views [JLS∗97] Abdominal CT endoscopic surgical training tool, effects such as bleeding [BGG98] Clinical CT, MRI and Visible Human Virtual endoscopy, surgery training [ÇK00] Abdominal CT Surgery training, palate surgery [COH∗02] Geometric models Regional anesthesia training, transparency adaptation [LBR04] Neck and abdominal CT Surgery planning and education, scripting language [MBP06] CT angiography Diagnosis of vascular pathologies, uncertainty lenses [LLPY07] Head CT Virtual endoscopy, surgery planning and patient education, realism [KKSP08] Collection of clinical data and segmentation Variational anatomy, shape spaces [ALvTZ19]
animation, since it is integrated and discussed in many cartography on motion perception and human cognition. In the following we publications. In cartography, it is well-known that viewers are frus- discuss the large potential for future research in medical animations. trated if they cannot control animation speed [MG94]. Ogao and Perception- and Cognition-Based Medical Animations. The Kraak [OK02] define the following interaction tasks for temporal major bottleneck for a widespread use of medical animations is cartographic animations: probably not technological. Powerful software tools and comput- • Adjust spatial and temporal resolution, ers may be used to generate and use medical animations, even if • Adjust spatial and temporal scale, large medical volume data are involved. Instead, substantial research • Enable multiple linked (dynamic) views, activities are needed to design animations that directly support es- • Query temporal data, and sential tasks in medical education, surgery planning, forensics and • Use mouse-over, e.g. show labels, measures. patient education. The systems described in this paper focus on algorithmical details without any discussion of perceptual limita- While these interaction tasks are relevant for map-based (2D) ani- tions. Moreover, no evaluation was carried out to investigate what mations, additional interaction tasks are essential for 3D animations. was actually seen in these animations. Task-based experiments are The user may stop an animation displayed in a 3D viewer and then often analyzed based on questions, such as what users have seen, interactively explore the data by zooming and rotation. Once the whether a certain object was observed or how long something takes. user is finished, a short animation may be generated to return to For cartography animation, Fabrikant et al. argue that eye-tracking the initial state, before the animation is continued. The animations experiments should be carried out to better understand the viewing generated for anatomy education (recall [PRS96]) can be interrupted pattern [FG05]. Medical animations for educational purposes have in this manner. Such a feature is even more important for virtual very rarely been designed and evaluated with learning theoretic argu- endoscopy, e.g. virtual colonoscopy (recall Sect. 6.3). The automati- ments in mind. Ruiz et al. concluded that “few studies have explored cally generated animation leads the physician to a particular point the effectiveness of animations in medical education” [RCL09]. This along the centerline where she may notice a suspicious lesion. Then situation has not changed significantly. she should be able to explore this lesion, measure its extent, annotate it for a report and then “ask” the system to continue the fly-through. User Studies for Authoring Systems. There is an obvious lack of user studies related to the authors’ perception of an animation sys- The discussion of interactive control also comprises input devices. tem. We discussed keyframing, scripting languages, and navigation- The joystick is a natural choice to control presentation speed [Sto17] assisted authoring processes. Comparative studies are needed to but also to steer the camera, e.g. in virtual endoscopy. We found no understand which animation processes are more efficient, satisfying research comparing input devices for steering medical animations and provide the desired precision and control to realize the intent of in particular in endoscopy. an author. Semi-Transparent Medical Animations. Several authors, in 8. Research Agenda particular physicians creating animations for educational purposes manually (recall [COH∗02]) emphasize the unique advantage of In this section we focus on general aspects of medical animation displaying movements involving different layers with appropri- and do not discuss in which particular applications animation gen- ate use of transparency. The generation of perceptually effective eration may be promising. The interactive use of 3D visualizations semi-transparent animations, however is challenging. It involves the for medical education and therapy planning is investigated quite afore-mentioned aspects of motion perception and the topic of trans- well, including perception-based studies that characterize the shape parency perception [SA02]. Designing and testing such animations and depth perception performance of users associated with these is an open research challenge. visualization techniques. In contrast, only few studies were carried out to compare different animation designs and their consequences Integrating Measures and Labels. Therapy planning benefits
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda from 2D and 3D visualizations combined with labels and measures, day interval between two examinations or a 500 day interval. Ideally, e.g. the volume or extent of a tumor or the minimum distance to an animation that interpolates between the examinations, conveys a risk structure. Animations that show a pathology with their sur- the uncertainty involved, that is obviously larger for longer intervals. rounding context enriched with this textual information may further Non-Temporal Animating of Volume Data. Volume render- support such processes. The textual information may be shown tem- ing is essential for medical diagnosis and treatment planning. porarily when the related visual information is visible well or even Therefore, it would be natural to develop (temporal and non- highlighted. Thus, the integration of textual elements benefits from temporal) animations using the means of volume rendering, e.g. a synchronization with the graphical elements of an animation. to enhance salient structures. While outside of medicine a num- Combining Abstraction and Animation Generation. Illustra- ber of essential developments were carried out to animate volume tive visualization provides us with a broad range of techniques data and in particular to control highlighting of different portions that abstract geometric models, e.g. to feature lines [LGP14, VI18]. [HZM13,RLH14,SRBG10,WS07], this area is under-represented in While abstraction is useful for the interactive exploration of medical medicine. The only examples we identified is the work of Iserhardt- surface models, it is probably even more important for animation. Bauer et al. [IHT∗02] on aneurysm diagnosis and from Lundstroem Animations are at a greater risk of producing cognitive overload and et al. [LLPY07] on using animation for uncertainty animation. abstraction may allow to restrict the visual complexity to a level users can cope with. Thus, further research is needed to integrate Integrating Animations in Storytelling Processes. Storytelling abstraction in animation generation workflows and to assess the for broader audiences becomes increasingly important also for med- perceptual and cognitive effectiveness of such abstracted anima- ical applications such as health and patient education. Animation tions. Animations may also provide smooth transitions between may be a crucial part of storytelling systems that may integrate different levels of abstraction, i.e. a morphing process supports the interactive exploration and predefined video sequences to be inter- understanding of relations between different levels. rupted for exploration. Pioneering work in this area was realized by Wohlfahrt and Hauser [WH07] who employed interactive volume Temporal generalization, such as temporal suppression and tem- visualization with animations. Similar concepts may be particularly poral smoothing, from cartographic animations may provide a useful for a wide range of educational applications in medicine. good source of inspiration. For readers interested in cartogra- phy animation, we refer to Maceachren et al. [MBHP98], Har- 9. Concluding Remarks rower [Har07, HF08] and Monmonier [Mon96]. Techniques from Animation is a useful presentation mode for medical image data and cartography animation are particularly relevant if medical projection derived models of the human anatomy. The dynamics involved in an- ∗ techniques (see the survey from Kreiser et al. [KMM 18]) are used imation may naturally display processes, such as growth processes to depict time-dependent data. and joint movements, and enable a deeper understanding compared Combination of Animated and Static Visualizations. Ani- to a sequence of static displays. On the other hand, animation de- mated visualizations may give a good overview of spatio-temporal sign has to consider perceptual and cognitive limitations, related developments and support a goal-directed in-depth analysis. Since to motion perception, change blindness and cognitive load theory. animations also have disadvantages, related to change blindness and According to these theories, animations should be rather short and limited capability to interpret detected changes, it may be combined exhibit a low or moderate complexity. Animations may be carefully with static displays that can be observed more carefully. Static visu- integrated in overall workflows related to medical education and alizations may depict the state of dynamic data at selected temporal therapy planning. In educational settings, animations are particu- positions or compare the state at two selected positions. Such sub- larly useful at early stages to make the learner familiar with certain traction images are used for example in the analysis of perfusion processes. In therapy planning, animations may provide an efficient data, to detect regions where the contrast agent arrival is delayed. overview of the relevant patient anatomy, whereas the in-depth ex- Moreover, timeline-based visualization is a typical static variant of ploration of 3D visualizations and 2D slice-based visualizations dynamic data display. It is also used for perfusion data to compare support the actual decision process. Animations are not restricted to contrast agent enhancement in different regions, e.g. potentially a presentation that is only passively observed: Animations may be pathologic and likely healthy regions. generated on demand, they may be interrupted and even a sequence of observing a video and interactively exploring a region is possible. The crucial aspect is that static visualizations and animations are linked with each other, e.g. when an animation is displayed it Non-temporal animations typically employ different camera po- should be obvious where the current timepoint is in relation to a sitions. The computation of candidates for a good camera position timeline-based visualization. and the computation of a path that combines the selected candidates is an essential support for creating non-temporal animations. Creating Animations to Convey Longitudinal Changes. Se- vere diseases, such as cancer or aortic dissections, require moni- A number of publications discuss the use of high-end animation toring of patients over time to assess growth, shrinkage or other software, such as Cinema 4D, for creating a single (expensive) ani- changes of pathologies. Animations may summarize such changes— mation. Since these software systems were not designed with use an idea that is described by Ambellan et al. [ALvTZ19]. Animations cases in medical education in mind, they miss important features for that convey longitudinal changes are also potentially relevant in such use cases. The retraction of tissue for example cannot be plau- developmental biology, where the stages of embryonal development sibly represented with the deformers provided by Maya [COH∗02]. are analyzed. The special situation with these longitudinal data is Thus, substantial additional effort is necessary to simulate such a that they do not exhibit a regular sampling, i.e. there may be a 200 process. These animations are fine-tuned for a particular purpose
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda and can hardly be adapted to new requirements, e.g. to display the MACEACHREN A.M.: Geovisual analytics to enhance spatial scan statis- use of another surgical instrument. Thus, widespread use of anima- tic interpretation: an analysis of us cervical cancer mortality. International tions requires a streamlined generation process, including re-usable journal of health geographics 7, 1 (2008), 57.7,8 components like the templates introduced by Akiba et al. [AWM10]. [CRPPD17] CRUZ RUIZ A.,PONTONNIER C.,PRONOST N.,DUMONT In contrast, a number of computer scientists developed animation G.: Muscle-Based Control for Character Animation. Comput. Graph. Forum 36, 6 (2017), 122–147.2 systems for a wide range of visualization tasks. Often, they are based on scripting languages. To illustrate the difference to the man- [dHJEV16] DE HOON N.H.,JALBA A.C.,EISEMANN E.,VILANOVA A.: Temporal Interpolation of 4D PC-MRI Blood-flow Measurements ually created animations: While the selection and combination of Using Bidirectional Physics-based Fluid Simulation. In Proc. VCBM viewpoints in non-temporal animations is a tedious process in man- (2016), pp. 59–68.1 ually created animations, it is supported by viewpoint computation [DMKR92] DIBIASE D.,MACEACHREN A.M.,KRYGIER J.B., techniques in more general animation systems. REEVES C.: Animation and the role of map design in scientific visual- ization. Cartography and geographic information systems 19, 4 (1992), Acknowledgements. We want to thank Dr. Gabriel Mistelbauer for 201–14.2,5 comments on the manuscript. This study was partially funded by the [ESG97] EHLSCHLAEGER C.R.,SHORTRIDGE A.M.,GOODCHILD Federal Ministry of Education and Research in Germany within the M. F.: Visualizing spatial data uncertainty using animation. Computers Forschungscampus STIMULATE (Grant Number 13GW0095A). & Geosciences 23, 4 (1997), 387–95. 15 [FG05] FABRIKANT S.I.,GOLDSBERRY K.: Thematic relevance and per- ceptual salience of dynamic geovisualization displays. In Proc. ICA/ACI References International Cartographic Conference (2005). 16 [ALvTZ19] AMBELLAN F., LAMECKER H., VON TYCOWICZ C.,ZA- [Fis93] FISHER P. F.: Visualizing uncertainty in soil maps by animation. CHOW S.: Statistical Shape Models - Understanding and Mastering Cartographica: The International Journal for Geographic Information Variation in Anatomy. Technical report, Zuse-Institute Berlin, 2019. 15, and Geovisualization 30, 2-3 (1993), 20–27. 15 16, 17 [Fis08] FISK G.: Using animation in forensic pathology and science [AW05] ALEXANDER A.L.,WICKENS C.D.: Flightpath tracking, education. Laboratory Medicine 39, 10 (2008), 587–92.1,3, 11 change detection and visual scanning in an integrated hazard display. In [GB09] GOLDSBERRY K.,BATTERSBY S.: Issues of change detection in Proc. of the Human Factors and Ergonomics Society Annual Meeting animated choropleth maps. Cartographica: The International Journal for (2005), vol. 49, pp. 68–72.3 Geographic Information and Geovisualization 44, 3 (2009), 201–215.2 KIBA ANG A [AWM10] A H.,W C.,M K.-L.: AniViz: A template-based [GHS07] GÖTZELMANN T., HARTMANN K.,STROTHOTTE T.: Annota- animation tool for volume visualization. IEEE CG&A 30, 5 (2010), 61–71. tion of Animated 3D Objects. In Proc. of Simulation und Visualisierung 1, 15, 18 (2007), pp. 209–22.9 [Bar05] BARTZ D.: Virtual endoscopy in research and clinical practice. [GP12] GEIJTENBEEK T., PRONOST N.: Interactive character animation Comput. Graph. Forum 24, 1 (2005), 111–26. 13 using simulated physics: A state-of-the-art review. Comput. Graph. Forum [BFS∗18] BONAVENTURA X.,FEIXAS M.,SBERT M.,CHUANG L., 31, 8 (2012), 2492–2515.2 WALLRAVEN C.: A survey of viewpoint selection methods for polygonal [Gre15] GREGORY R.L.: Eye and brain: The psychology of seeing, models. Entropy 20, 5 (2018), 370.8 vol. 38. Princeton university press, 2015.2 [BGG98] BAUR C.,GUZZONI D.,GEORG O.: VIRGY: A Virtual Reality [Har07] HARROWER M.: The cognitive limits of animated maps. Car- and Force Feedback-Based Endoscopic Surgery Simulator. In Proc. of tographica: The International Journal for Geographic Information and Medicine Meets Virtual Reality (1998), pp. 110–116. 13, 16 Geovisualization 42, 4 (2007), 349–57.3, 17 ∗ [BLH 04] BURMESTER E.,LEINEWEBER T., HACKER S.,TIEDE U., [Her02] HERMANN M.: Dreidimensionale Computeranimation–neues HÜTTEROTH T., HÖHNE K.: Eus meets voxel-man: three-dimensional Medium zur Unterstützung des Aufklärungsgesprächs vor Operationen anatomic animation of linear-array endoscopic ultrasound images. En- Akzeptanz und Bewertung der Patienten anhand einer prospektiv ran- doscopy 36, 8 (2004), 726–30. 11 domisierten Studie–Bild versus Text. Der Chirurg 73, 5 (2002), 500–7.4, [But94] BUTZ A.: BETTY: Planning and Generating Animations for the 12 Visualization of Movements and Spatial Relations. In Proc. of Advanced [HF08] HARROWER M.,FABRIKANT S.: The role of map animation for Visual Interfaces (1994), pp. 53–58.9 geographic visualization. Geographic visualization: concepts, tools and [Cat72] CATMULL E.: A system for computer generated movies. In Proc. applications (2008), 49–65. 17 of the ACM annual conference (1972), pp. 422–31.4,9 [HH95] HABBAL O.,HARRIS P.: Teaching of human anatomy: a role [CDK03] CHOYKE P. L., DWYER A.J.,KNOPP M. V.: Functional tumor for computer animation. Journal of Audiovisual Media in Medicine 18, 2 imaging with dynamic contrast-enhanced magnetic resonance imaging. (1995), 69–73.1,3,7,8, 11 Journal of Magnetic Resonance Imaging: An Official Journal of the In- [HKW∗95] HONG L.,KAUFMAN A.,WEI Y.-C., VISWAMBHARAN ternational Society for Magnetic Resonance in Medicine 17, 5 (2003), A.,WAX M.,LIANG Z.: 3d virtual colonoscopy. In Proc. of IEEE 509–20.7,8 Biomedical Visualization (1995), pp. 26–32. 13 [ÇK00] ÇAKMAK H.K.,KÜHNAPFEL U.: Animation and simulation [HMK∗97] HONG L.,MURAKI S.,KAUFMAN A.E.,BARTZ D.,HE techniques for vr-training systems in endoscopic surgery. In Proc. of T.: Virtual voyage: interactive navigation in the human colon. In Proc. of Computer Animation and Simulation. Springer, 2000, pp. 173–85.1, 13, ACM SIGGRAPH (1997), pp. 27–34. 13, 16 16 [HNN∗03] HIGUERA F. V., NARAGHI R.,NIMSKY C.,FAHLBUSCH [COH∗02] CUTTING C.,OLIKER A.,HARING J.,DAYAN J.,SMITH R.,GREINER G.,HASTREITER P.: Standardized 3D documentation for D.: Use of three-dimensional computer graphic animation to illustrate neurosurgery. Computer Aided Surgery 8, 6 (2003), 274–82.5,6 cleft lip and palate surgery. Computer Aided Surgery 7, 6 (2002), 326–31. [HZM13] HSU W.-H., ZHANG Y., MA K.-L.: A multi-criteria approach 4, 11, 12, 16, 17 to camera motion design for volume data animation. IEEE Trans. Vis. [CRN∗08] CHEN J.,ROTH R.E.,NAITO A. T., LENGERICH E.J., Comput. Graph. 19, 12 (2013), 2792–2801. 17
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda
[IHN∗16] IQBAL U.,HSU C.-K.,NGUYEN P. A. A., CLINCIU D.L., D.J.: Change blindness blindness: The metacognitive error of overesti- LU R.,SYED-ABDUL S.,YANG H.-C.,WANG Y.-C., HUANG C.- mating change-detection ability. Visual Cognition 7, 1-3 (2000), 397–412. Y., HUANG C.-W., CHANG Y.-C., HSU M.-H.,JIAN W.-S., LI Y.- 2 C.J.: Cancer-disease associations: A visualization and animation through [LWP∗06] LJUNG P., WINSKOG C.,PERSSON A.,LUNDSTROM C., medical big data. Computer Methods and Programs in Biomedicine 127, YNNERMAN A.: Full body virtual autopsies using a state-of-the-art Supplement C (2016), 44–51.7 volume rendering pipeline. IEEE Trans. Vis. Comput. Graph. 12, 5 [IHT∗02] ISERHARDT-BAUER S.,HASTREITER P., TOMANDL B., (2006), 869–76. 14 KÖSTNER N.,SCHEMPERSHOFE M.,NISSEN U.,ERTL T.: Standard- [MBHP98] MACEACHREN A.M.,BOSCOE F. P., HAUG D.,PICKLE ized Analysis of Intracranial Aneurysms Using Digital Video Sequences. L. W.: Geographic visualization: Designing manipulable maps for ex- In Proc. MICCAI (2002), pp. 411–18.1,5, 17 ploring temporally varying georeferenced statistics. In Proc. of IEEE [JAES87] JILIN L.,AFFELD K.,ENGELHORN M. W., SCHARTL M.: Symposium on Information Visualization (1998), pp. 87–94.7, 17 Animated 3d-model of the human heart based on echocardiograms. In [MBP06] MÜHLER K.,BADE R.,PREIM B.: Adaptive Script Based Proc. of Aachener Symposium für Signaltheorie (1987), pp. 155–58.1,7, Animations for Intervention Planning. In Proc. MICCAI (2006), pp. 478– 8 85.1, 10, 16 [JB08] JUNG Y., BEHR J.: Extending H-Anim and X3D for advanced [McG10] MCGHEE J.: 3-D visualization and animation technologies in animation control. In Proc. of the ACM Symposium on 3D web technology anatomical imaging. Journal of anatomy 216, 2 (2010), 264–70.1,2, 11, (2008), pp. 57–65.7 12, 13, 14 [JLS∗97] JOLESZ F. A., LORENSEN W. E., SHINMOTO H.,ATSUMI H., [MD91] MACEACHREN A.M.,DIBIASE D.: Animated maps of aggre- NAKAJIMA S.,KAVANAUGH P., ETAL.: Interactive virtual endoscopy. gate data: Conceptual and practical problems. Cartography and Geo- American journal of roentgenology 169, 5 (1997), 1229–35.1, 13, 16 graphic Information Systems 18, 4 (1991), 221–29.7
[KB84] KOCHANEK D.H.,BARTELS R.H.: Interpolating splines with [MD08] MAASS S.,DÖLLNER J.: Seamless Integration of Labels into local tension, continuity, and bias control. In Proc. of ACM SIGGRAPH Interactive Virtual 3D Environments Using Parameterized Hulls. In Proc. (1984), pp. 33–41.4 of Eurographics Workshop on Computational Aesthetics (2008), pp. 33– 40.9 [KEM97] KRAAK M.-J.,EDSALL R.,MACEACHREN A.M.: Carto- ∗ graphic animation and legends for temporal maps: Exploration and or [MEB 17] MEUSCHKE M.,ENGELKE W., BEUING O.,PREIM B., interaction. In Proc. of Cartographic Conference (1997), vol. 1, pp. 253– LAWONN K.: Automatic viewpoint selection for exploration of time- 61.6,8 dependent cerebral aneurysm data. In Proc. of Bildverarbeitung für die Medizin (2017), pp. 352–57.8, 14 [KF93] KARP P., FEINER S.: Automated presentation planning of ani- mation using task decomposition with heuristic reasoning. In Proc. of [MG94] MONMONIER M.,GLUCK M.: Focus groups for design improve- Graphics Interface (1993), pp. 118–25.9 ment in dynamic cartography. Cartography and Geographic Information Systems 21, 1 (1994), 37–47. 16 [KKSP08] KRÜGER A.,KUBISCH C.,STRAUSS G.,PREIM B.: Sinus [MNTP07] MÜHLER K.,NEUGEBAUER M.,TIETJEN C.,PREIM B.: Endoscopy - Application of Advanced GPU Volume Rendering for Virtual Viewpoint selection for intervention planning. In Proc. EuroVis (2007), Endoscopy. IEEE Trans. Vis. Comput. Graph. 14, 6 (2008), 1491–98.1, pp. 267–74.8 12, 13, 14, 16 [Mon96] MONMONIER M.: Temporal generalization for dynamic maps. [KMM∗18] KREISER J.,MEUSCHKE M.,MISTELBAUER G.,PREIM Cartography and Geographic Information Systems 23, 2 (1996), 96–98. B.,ROPINSKI T.: A survey of flattening-based medical visualization 17 techniques. Comput. Graph. Forum 37, 3 (2018), 597–624.2,7, 17 ∗ [MP09] MÜHLER K.,PREIM B.: Automatic textual annotation for surgi- [KPG 16] KÖHLER B.,PREIM U.,GROTHOFF M.,GUTBERLET M., cal planning. In Proc. VMV (2009), pp. 277–84.9 PREIM B.: Adaptive Animations of Vortex Flow Extracted from Cardiac 4D PC-MRI Data. In Proc. of Bildverarbeitung für die Medizin (2016), [MP10a] MÜHLER K.,PREIM B.: Günstige Kamerapfade für medizinis- pp. 194–99.6,8 che Animationen. In Proc. of Bildverarbeitung für die Medizin (2010), pp. 46–50.9 [Las87] LASSETER J.: Principles of traditional animation applied to 3d computer animation. In Proc. of ACM SIGGRAPH (1987), pp. 35–44.3, [MP10b] MÜHLER K.,PREIM B.: Reusable Visualizations and Anima- 9 tions for Surgery Planning. Comput. Graph. Forum 29, 3 (2010), 1103–12. 1,4,5 [LBR04] LIM M.,BURT G.,RUTTER S.: Use of three-dimensional [MZL10] MA M.,ZHENG H.,LALLIE H.: Virtual reality and 3d anima- animation for regional anaesthesia teaching: application to interscalene tion in forensic visualization. Journal of forensic sciences 55, 5 (2010), brachial plexus blockade. British journal of anaesthesia 94, 3 (2004), 1227–31.4,7,8, 14 372–77.4, 12, 16 [OK02] OGAO P. J., KRAAK M.-J.: Defining visualization operations for [LGP14] LAWONN K.,GASTEIGER R.,PREIM B.: Adaptive Surface temporal cartographic animation design. International journal of applied Comput. Graph. Visualization of Vessels with Animated Blood Flow. earth observation and geoinformation 4, 1 (2002), 23–31. 16 Forum 33, 8 (2014), 16–27.6,8, 17 [OP14] OELTZE-JAFRA S.,PREIM B.: Survey of Labeling Techniques [LHM14] LIAO I.,HSU W., MA K.: Storytelling via navigation: A novel in Medical Visualizations. In Proc. VCBM (2014), pp. 199–208.9 approach to animation for scientific visualization. In Proc. of Smart Graphics (2014), pp. 1–14. 10 [Par12] PARENT R.: Computer animation: algorithms and techniques. Newnes, 2012.4,6,8 [LJK95] LORENSEN W., JOLESZ F., KIKINIS R.: The exploration of ∗ cross-sectional data with a virtual endoscope. Interactive Technology and [PCH 05] PREIM B.,CORDES J.,HEINRICHS T., KRAUSE D.,JACHAU the New Medical Paradigm for Health Care (1995), 221–30.1, 13 K.: Quantitative Bildanalyse und Visualisierung für die Analyse von post-mortem Datensätzen. In Proc. of Bildverarbeitung für die Medizin [LLPY07] LUNDSTRÖM C.,LJUNG P., PERSSON A.,YNNERMAN A.: (2005), pp. 6–10. 15 Uncertainty visualization in medical volume rendering using probabilistic [POM∗09] PREIM B.,OELTZE S.,MLEJNEK M.,GROLLER E.,HEN- animation. IEEE Trans. Vis. Comput. Graph. 13, 6 (2007), 1648–55. 15, NEMUTH A.,BEHRENS S.: Survey of the visual exploration and analysis 16, 17 of perfusion data. IEEE Trans. Vis. Comput. Graph. 15, 2 (2009), 205–20. [LMDIS00] LEVIN D. T., MOMEN N.,DRIVDAHL IVS.B.,SIMONS 6,7
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association. Bernhard Preim & Monique Meuschke / Medical Animations: A Survey and a Research Agenda
[PRS96] PREIM B.,RITTER A.,STROTHOTTE T.: Illustrating Anatomic scientific documentation, reconstruction and animation in forensic: in- Models - A Semi-Interactive Approach. In Proc. of Visualization in dividual and real 3D data based geo-metric approach including optical Biomedical Computing (1996), pp. 23–32.1, 10, 11, 16 body/object surface and radiological CT/MRI scanning. Journal of foren- sic science 50, 2 (2005), JFS2004290–15. 14 [PS18] PREIM B.,SAALFELD P.: A survey of virtual human anatomy education systems. Computers & Graphics 71 (2018), 132–53. 11 [TM18] TRAUN C.,MAYRHOFER C.: Complexity reduction in choro- pleth map animations by autocorrelation weighted generalization of time- [RBA∗96] RUBIN G.D.,BEAULIEU C. F., ARGIRO V., RINGL H., series data. Cartography and Geographic Information Science 45, 3 NORBASH A.M.,FELLER J. F., DAKE M.D.,JEFFREY R.B.,NAPEL (2018), 221–37.7 S.: Perspective volume rendering of CT and MR images: applications for endoscopic imaging. Radiology 199, 2 (1996), 321–30. 13 [VFSG06] VIOLA I.,FEIXAS M.,SBERT M.,GRÖLLER E.: Importance- Driven Focus of Attention. IEEE Trans. Vis. Comput. Graph. 12, 5 (2006), [RCL09] RUIZ J.G.,COOK D.A.,LEVINSON A.J.: Computer anima- 933–40.8,9 tions in medical education: a critical literature review. Medical education 43, 9 (2009), 838–46.2,3, 11, 16 [VFSH01] VÁZQUEZ P.-P., FEIXAS M.,SBERT M.,HEIDRICH W.: Viewpoint selection using viewpoint entropy. In Proc. VMV (2001), [RLH14] RADEVA N.,LEVY L.,HAHN J.: Generalized temporal focus+ vol. 1, pp. 273–80.8 context framework for improved medical data exploration. Journal of digital imaging 27, 2 (2014), 207–19. 17 [VI18] VIOLA I.,ISENBERG T.: Pondering the concept of abstraction in (illustrative) visualization. IEEE Trans. Vis. Comput. Graph. 24, 9 (2018), [RPDS00] RITTER F., PREIM B.,DEUSSEN O.,STROTHOTTE T.: Using 2573–88. 17 a 3d puzzle as a metaphor for learning spatial relations. In Proc. of Graphics Interface (2000), pp. 171–78. 11 [VOH17] VILLA C.,OLSEN K.,HANSEN S.: Virtual animation of victim-specific 3D models obtained from CT scans for forensic recon- [RWIB∗07] RÖSSLER F., WOLFF T., ISERHARDT-BAUER S., structions: living and dead subjects. Forensic science international 278 TOMANDL B.,HASTREITER P., ERTL T.: Distributed video generation (2017), e27–e33.1, 14 on a gpu-cluster for the web-based analysis of medical image data. In Proc. of SPIE Medical Imaging: Visualization and Image-Guided [VP02] VERNON T., PECKHAM D.: The benefits of 3d modelling and Procedures (2007), vol. 6509, International Society for Optics and animation in medical teaching. Journal of Audiovisual media in Medicine Photonics, p. 650903.5 25, 4 (2002), 142–48. 11 [SA02] SINGH M.,ANDERSON B.L.: Toward a perceptual theory of [War12] WARE C.: Information visualization: perception for design. transparency. Psychological review 109, 3 (2002), 492. 16 Elsevier, 2012.2 [SAJ∗03] SUHLING M.,ARIGOVINDAN M.,JANSEN C.,HUNZIKER [WH07]W OHLFART M.,HAUSER H.: Story Telling for Presentation in P., UNSER M.A.: Myocardial motion analysis and visualization from Volume Visualization. In Proc. EuroVis (2007), pp. 91–98. 17 echocardiograms. In Proc. of SPIE Medical Imaging: Image Processing [WHRO10] WANG H.,HECHT F., RAMAMOORTHI R.,O’BRIEN J. F.: (2003), vol. 5032, pp. 306–14.7,8 Example-based wrinkle synthesis for clothing animation. ACM Trans. [Sak06] SAKCHAICHAROENKUL T.: MCFI-based animation tweening Graph. 29, 4 (2010), 107.4 algorithm for 2d parametric motion flow/optical flow. Machine Graphics [WL08] WANG Y.-S., LEE T.-Y.: Example-driven animation synthesis. & Vision International Journal 15, 1 (2006), 29–49.4 The Visual Computer 24, 7-9 (2008), 765–73.4 [SGS95] SAKAS G.,GRIMM M.,SAVOPOULOS A.: Optimized maxi- [WMG∗09] WALD I.,MARK W. R., GÜNTHER J.,BOULOS S.,IZE T., mum intensity projection (mip). In Proc. of EG Workshop on Rendering HUNT W., PARKER S.G.,SHIRLEY P.: State of the art in ray tracing Techniques (1995), pp. 51–63.1,5 animated scenes. Comput. Graph. Forum 28, 6 (2009), 1691–1722.2 [SL97] SIMONS D.J.,LEVIN D. T.: Change blindness. Trends in [WS07] WOODRING J.,SHEN H.-W.: Incorporating highlighting an- cognitive sciences 1, 7 (1997), 261–67.2 imations into static visualizations. In Proc. of Visualization and Data [SLF∗09] STRAUSS G.,LIMPERT E.,FISCHER M.,HOFER M.,KU- Analysis (2007), vol. 6495, p. 649503. 17 BISCH C.,KRÜGER A.,DIETZ A.: Virtuelle Echtzeit-Endoskopie der [WWS∗15] WHEATLAND N.,WANG Y., SONG H.,NEFF M.,ZORDAN Nase und Nasennebenhöhlen. HNO 57, 8 (2009), 789–96. 12 V., JÖRG S.: State of the art in hand and finger modeling and animation. [SPFG05] SBERT M.,PLEMENOS D.,FEIXAS M.,GARCÍA F. G.: View- Comput. Graph. Forum 34, 2 (2015), 735–60.2 point Quality: Measures and Applications. In Proc. of EG Workshop on [ZAM11] ZHENG Z.,AHMED N.,MUELLER K.: iView: A feature Computational Aesthetics (2005), pp. 185–92.8 clustering framework for suggesting informative views in volume visu- [SRBG10] SIKACHEV P., RAUTEK P., BRUCKNER S.,GRÖLLER E.: alization. IEEE Trans. Vis. Comput. Graph. 17, 12 (2011), 1959–68. Dynamic Focus+ Context for Volume Rendering. In Proc. VMV (2010), 8 pp. 331–38. 17 [Zel91] ZELTZER D.: Making them move. Morgan Kaufmann Publishers [SSN∗98] SATO Y., SHIRAGA N.,NAKAJIMA S.,TAMURA S.,KIKINIS Inc., San Francisco, CA, USA, 1991, ch. Task-level Graphical Simulation: R.: Local maximum intensity projection (lmip: A new rendering method Abstraction, Representation, and Control, pp. 3–33.9 for vascular visualization. Journal of computer assisted tomography 22, [Zet13] ZETTL H.: Sight, sound, motion: Applied media aesthetics. Cen- 6 (1998), 912–17.5 gage Learning, 2013.4 [Sto17] STOLLFUSS S.: Animierte anatomie. In In Bewegung setzen... Springer, 2017, pp. 149–68.4, 13, 16 [Swe04] SWELLER J.: Instructional design consequences of an analogy between evolution by natural selection and human cognitive architecture. Instructional science 32, 1-2 (2004), 9–31.3 [SWH∗05] STALLING D.,WESTERHOFF M.,HEGE H.-C., ETAL.: Amira: A highly interactive system for visual data analysis. In The visualization handbook, vol. 38. Elsevier Butterworth-Heinemann, UK, 2005, pp. 749–67.9 [TBB∗05] THALI M.J.,BRAUN M.,BUCK U.,AGHAYEV E.,JACK- OWSKI C.,VOCK P., SONNENSCHEIN M.,DIRNHOFER R.: VIRTOPSY:
c 2019 The Author(s) Eurographics Proceedings c 2019 The Eurographics Association.