A Survey of Visualization Methods for Special Relativity

Daniel Weiskopf1

1 Visualization Research Center (VISUS), Universität Stuttgart [email protected]

Abstract This paper provides a survey of approaches for special relativistic visualization. Visualization techniques are classiﬁed into three categories: Minkowski spacetime diagrams, depictions of spatial slices at a constant time, and virtual camera methods that simulate image generation in a relativistic scenario. The paper covers the historical outline from early hand-drawn visualizations to state-of-the-art computer-based visualization methods. This paper also provides a concise presentation of the mathematics of special relativity, making use of the geometric nature of spacetime and relating it to geometric concepts such vectors and linear transformations.

1998 ACM Subject Classiﬁcation I.3.5 Computational Geometry and Object Modeling, J.2 Physical Sciences and Engineering

Keywords and phrases Special Relativity, Minkowski, Spacetime, Virtual Camera

Digital Object Identiﬁer 10.4230/DFU.SciViz.2010.289

1 Introduction

Einstein’s special theory of relativity has been attracting a lot of attention from the general public and physicists alike. In 2005, the 100th anniversary of the publication of special relativity [7] was the reason for numerous exhibitions, popular-science publications, or TV shows on modern physics in general and Einstein and his work in particular. In special relativistic physics, properties of space, time, and light are dramatically different from those of our familiar environment governed by classical physics. We do not experience relativistic effects and thus do not have an intuitive understanding for those effects because we, in our daily live, do not travel at velocities close to the speed of light. Special relativity is usually described in terms of mathematical models such as spacetime and Lorentz transformations. Since special relativity has a strong geometric component, visualization can play a crucial role in making those geometric aspects visible without relying on symbolic notation. This paper gives an overview of different approaches for visualizing various aspects of special relativity to a range of different audiences. One application is the support of visual communication for a general public, for example, by means of illustrations in popular-science publications, exhibitions, or TV shows. Another audience are students because visualization can be used to improve the learning experience in high- school or university courses. For example, depictions help to motivate, interactive computer experiments allow for exploration and active participation, and visual explanations can enrich a symbolic description of mathematical ideas. A third group of people are experts in physics and relativity. Although they do not need visualization to learn and understand the mathematics of special relativity, visualization may engage them in a different way of thinking. Edwin F. Taylor, a renowned teacher of special relativity [33], expressed his experience with special relativistic visualization as follows [32,

© D. Weiskopf; licensed under Creative Commons License NC-ND Scientiﬁc Visualization: Advanced Concepts. Editor: Hans Hagen; pp. 289–302 Dagstuhl Publishing Schloss Dagstuhl – Leibniz Center for Informatics (Germany) 290 A Survey of Visualization Methods for Special Relativity

Section 6]: “[...] I have come to think about relativity quite differently [...] My current view of the subject is much more visual, more fluid, more process oriented, covering a wider range of phenomena [...] In short, both my professional life and my view of physics have been transformed [by relativistic visualization].” These different types of audiences have a quite varying background knowledge. Therefore, different visualization approaches may be employed for different audiences and purposes. This paper distinguishes three classes of basic approaches (see Section 2): A direct visualization of spacetime by Minkowski diagrams, a visualization of a subset of spacetime for a fixed time, and the simulation of images as taken by a fast moving camera. Technical aspects and algorithms for these different approaches are discussed in Sections 5–7. All three approaches have in common that they may benefit from interactive computer implementations that facilitate trial-and-error explorations through the user. This paper has several goals. First, it provides a survey of state-of-the-art computer-based methods for special relativistic visualization. The methods are roughly structured along the aforementioned classification of approaches. Because several alternative algorithms are available for the camera metaphor, these algorithms are further classified in subcategories (see Section 7). In addition, this paper provides a concise presentation of special relativity in Section 3, making use of the geometric nature of spacetime and relating it to geometric concepts such vectors and linear transformations. This paper also provides a historical outline of the development of special relativistic visualization (Section 4).

2 Types of Visualization Approaches

This paper adopts a geometric point of view on special relativity. Key elements are the concepts of spacetime and Lorentz transformations, which relate different coordinate systems in spacetime. Section 3 explains these concepts on a mathematical level. Spacetime is the combination of 3D space and a single temporal dimension, leading to a joint 4D description. Physical experiments, even if they are only virtual, are represented in form of 4D spacetime coordinates. For example, a point-like object that moves through space and time leaves a trace in spacetime—a so-called worldline. Similarly, light rays can be represented as lines through spacetime. Therefore, spacetime and traces therein are sufficient to describe the physical scenarios that are relevant for this paper. Lorentz transformations represent changes between coordinate systems—transformations between different frames of reference. Due to the Lorentz transformation, observers in different frames of reference typically provide different coordinate descriptions for the very same physical object. In other words, both spatial and temporal positions are dependent on the reference frame—space and time are not addressed by absolute coordinates, but they are relative. The structure of spacetime and the Lorentz transformation can be derived from two postulates: the principle of relativity (i.e., physical laws are valid and unchanged in any inertial reference frame) and the invariance of the speed of light (i.e., the speed of light in vacuo has a finite and constant value, regardless of the reference frame). This derivation can be found in textbooks, such as [22]. The three visualization approaches discussed in this paper can be related to spacetime in the following ways. All approaches have in common a reduction of dimensionality of 4D spacetime. (a) Minkowki diagrams. Minkowski diagrams are spacetime diagrams. They depict spacetime by graphically representing both temporal and spatial dimensions in a single image. The dimensionality of the spatial domain is reduced to either one or two (by taking a slice D. Weiskopf 291 through space), which leads to a total number of two or three dimensions for the spacetime diagram. In this way, graphical representations of the 2D or 3D diagram are feasible in an image. The advantage of Minkowski diagrams is their direct visualization of spacetime itself—Minkowski diagrams are the visual pendant to the mathematical geometry of special relativity. Figures 1 and 2 show typical examples of Minkowski diagrams. (b) Spatial slices. Another way of reducing dimensionality is to construct a spatial slice of constant time. This slicing corresponds physically to a simultaneous measurement of positions in 3D space. Time and simultaneity depend on the frame of reference, i.e., a spatial slice is always defined with respect to a reference frame. Spatial slices are a natural metaphor because they model measurements in 3D space for a “frozen” time. Figure 3 provides an example of several spatial slices taken at different times. (c) Virtual camera model. The virtual camera model simulates a physical experiment: what kind of image would a camera produce in a special relativistic setting? This approach simulates what we would see and, therefore, is the special relativistic analog of standard image synthesis by rendering non-relativistic scenes. Figure 4 illustrates the virtual camera view for high-speed travel toward the Brandenburg Gate. In a non-relativistic setting, where the speed of light is assumed to be infinite, approaches (b) and (c) are identical. Special relativity, however, requires us to make a clear distinction between seeing and measuring. Measurements are made at sample points simultaneously with respect to the reference frame of the observer. In contrast, seeing is based on the photons that arrive simultaneously at the camera of the observer. These photons are usually not emitted simultaneously (with respect to the observer’s reference frame) due to the finite speed of light. Following [41], approaches (a) and (b) can be regarded exocentric visualization, which present an outside view, whereas approach (c) can be considered an egocentric visualization, which is produced from the perspective of the user.

3 Elements of Special Relativity

This section provides a brief introduction to the mathematics of special relativity, discussing the concepts of spacetime, Lorentz transformation, four-vectors, and the Minkowski metric. More details on these concepts can be found in textbooks like [21, 22], or in a visualization- orientated presentation of special relativity [40]. Spacetime consists of three spatial dimensions and one temporal dimension. Analogously to a vector in Euclidean 3D space, a vector in spacetime can be described by four components

xµ = (t, x, y, z) = (x0, x1, x2, x3) , µ = 0, 1, 2, 3 .

The Greek indices {1, 2, 3} refer to three spatial components and the index 0 refers to the temporal dimension. To simplify the notation in this paper, natural units are used, in which the speed of light c = 1. The geometry of spacetime is described by the Minkowski metric

1 0 0 0  0 −1 0 0  ηµν =   , (1) 0 0 −1 0  0 0 0 −1 which is used to compute distances and inner products in spacetime. The inner product between two vectors, aµ and bµ, is

µ µ µ ν a · b = ηµν a b .

C h a p t e r 1 9 292 A Survey of Visualization Methods for Special Relativity

The Einsteinian sum convention is applied: terms with duplicate indices are implicitly summed, e.g., the indices µ and ν are implicitly summed over all entries 0 ... 3. The inner product is used to deﬁne the squared length of a spacetime vector—a so-called four-vector. For example, the position vector xµ has squared length

µ µ µ ν 2 2 2 2 x · x = ηµν x x = t − x − y − z .

The Lorentz transformation is a linear and homogeneous transformation of spacetime vectors

µ0 µ ν x = Λ ν x , (2)

µ where Λ ν is the matrix representation of the transformation. A Lorentz transformation is defined as a transformation that does not change the inner product or the squared length of four-vectors—it does not affect the geometry of spacetime as described by the Minkowski metric. The collection of Lorentz transformations forms the Lorentz group. A Lorentz transformation connects two inertial frames of reference and leaves the speed of light invariant. Lorentz transformations can be interpreted as a combination of spatial rotations of the two reference frames and a so-called Lorentz boost. The Lorentz boost is a velocity transformation between two reference frames that move at different relative speeds, given by (cf. [21, p.69])

  γ −β γ nx −β γ ny −β γ nz 2 µ  −β γ nx (γ − 1) nx + 1 (γ − 1) nx ny (γ − 1) nx nz  Λ ν =  2  . (3)  −β γ ny (γ − 1) nx ny (γ − 1) ny + 1 (γ − 1) ny nz  2 −β γ nz (γ − 1) nx nz (γ − 1) ny nz (γ − 1) nz + 1

Here, ~n = (n1, n2, n3) is the normalized direction of motion, β is the velocity relative to the speed of light, and γ = 1/p1 − β2. By including translations between reference frames, the Lorentz group is extended to the Poincaré group. The Lorentz group transforms four-vectors in spacetime, while the Poincaré group transforms points in spacetime. A spacetime point is usually called an event. For comparison, an analogous distinction has to be made between points and vectors in Euclidean 3D space—linear transformations are applied to vectors while affine transformations are applied to points. Events and four-vectors are tightly connected: the difference between two spacetime events is described by a four-vector that connects both events. In general, the squared length of a four-vector is independent of the frame of reference because it is not changed by Lorentz transformations. Therefore, a vector can be characterized by its length. A vector is lightlike when it has vanishing length. Lightlike difference vectors are important for image synthesis because they connect light emission and absorbtion events. Accordingly, the propagation of a photon can be described by its lightlike four-wavevector, which combines circular frequency and 3D wavevector:

kµ = (ω,~k) . (4)

The 3D wavevector ~k points into the light direction and has length k = 2πλ−1, where λ is the wavelength. The circular frequency ω is related to the frequency ν of the photon by ω = 2πν. Wavelength and frequency are related by λ = ν−1. By applying a Lorentz transformation to the four-wavevector, the aberration of light and the Doppler effect can be immediately computed. The relativistic aberration of light D. Weiskopf 293 describes the change of light direction caused by the Lorentz transformation. The aberration can be expressed as cos θ − β cos θ0 = , φ0 = φ , (5) 1 − β cos θ where directions are given by spherical coordinates (θ, φ) and (θ0, φ0). The Doppler effect accounts for the transformation of wavelength from one inertial frame of reference to another and is described by ω ω0 = , (6) D with the Doppler factor 1 D = . (7) γ (1 − β cos θ) Expressed in terms of wavelength, the Doppler effect is

λ0 = D λ . (8)

Finally, the wavelength-dependent radiance Lλ is transformed according to 1 L0 (λ0, θ0, φ0) = L (λ, θ, φ) . (9) λ D5 λ A derivation of this relation is described by Weiskopf et al. [43] in the context of special relativistic visualization. The transformation of radiance leads to the so-called searchlight eﬀect because light that is incident from the direction of motion is increased in radiance.

4 Historic Remarks and Recent Developments

This section discusses the timeline of developments in special relativistic visualization. The presentation is in approximately chronological order, from early work on special relativity to recent improvements on computer-based visualization. Although this overview of the literature covers papers from both physics and computer science, the focus is on a survey of special relativistic visualization based on modern computer graphics. Einstein’s original article on special relativity [7] was published in 1905. However, the contemporary understanding of special relativity as a description of spacetime associated with a certain metric goes back to Hermann Minkowski, as laid out in his 1908 lecture on “Raum und Zeit” (“space and time”) [20]. The accompanying four-vector formalism may be attributed to Arnold Sommerfeld and his publications [29, 30] from 1910. These developments form the basis for a geometric point of view on special relativity that is widely adopted in most contemporary presentations and textbooks. Moreover, this geometric interpretation is vital for the extension to general relativity. More historical details on the early developments in special relativity can be found in an article by Walter [36]. A direct visualization of the spacetime geometry is typically based on a Minkowski diagram, which is a most popular type of visualization in textbooks. Minkowski introduced his diagrams in his lecture on space and time [20]. The second fundamental visualization approach—using spatial slices—was also used early on. For example, such a visualization appears in the movie “Die Grundlagen der Einsteinschen Relativitäts-Theorie”, which is an early popular-science documentary film and probably the first animated visualization of special relativity. This film, with a length of over

C h a p t e r 1 9 294 A Survey of Visualization Methods for Special Relativity

two hours, had its premiere in 1922. Unfortunately, no copy of the original movie is known to have survived until today [37]. However, an abridged and modified American version that was released in 1923 as a 20-minute film with the title “The Einstein Theory of Relativity” is available, including an accompanying booklet [27]. The third visualization approach—the virtual camera model—took much longer before it was developed. Remarkably, the issue of image generation in the context in special relativity was ignored for a long time, or wrong interpretations were given. For example, Gamow equates the Lorentz contraction and the visual appearance of moving bodies in his book “Mr Tompkins in Wonderland” [8], neglecting the difference between seeing and simultaneously measuring. Apart from a previously disregarded article by Lampa [18] in 1924 about the invisibility of the Lorentz contraction, it was only in 1959 that the first correct solutions to the issue of image generation within special relativity were presented by Terrell [34] and Penrose [23]. Later, more detailed descriptions of the geometrical appearance of fast moving objects were discussed in the physics literature, e.g., by Weisskopf [45], Boas [3], Scott and Viner [26], Scott and van Driel [25], Hollenbach [11], Hickey [10], Suffern [31], Burke and Strode [4], Sheldon [28], Terrell [35], and Kraus [17]. In the 1980s, computers were gradually used for advanced special relativistic visualization. The earliest published example known to the author is a software by Taylor [32], which was used for teaching courses at MIT starting 1986. This software included both Minkowski diagrams and the virtual camera model, and rendering was based on line graphics. Besides this published work, there might have been other implementations of that era that may be overlooked today because their results were not disseminated publicly. In 1989, Hsiung and Dunn [12] were the first to publish an advanced rendering technique for the virtual camera approach. Their rendering technique is based on an extension of standard 3D ray tracing. Within the following year, Hsiung and co-workers extended their work to include the visualization of the Doppler effect [14], relativistic time dilation [13], and the time-buffer method [15]. In 1991, Gekelman et al. [9] described a rendering method related to the time-buffer. Later developments included extended illumination models by Chang et al. [5] and Betts [2], which were subsequently improved by Weiskopf and co-workers [43, 44]. Other papers addressed the issue of acceleration in the context of the virtual camera approach [9, 44, 16, 24, 39]. As alternative rendering methods, texture-based rendering was introduced by Weiskopf [38] and image-based rendering by Weiskopf et al. [42]. Weiskopf [40] described these techniques in more detail and proposed a special relativistic version of the radiosity method. Li et al. [19] extended special relativistic ray tracing to include reflection and transmission phenomena. Finally, Weiskopf et al. [41] reported on their experiences with relativistic visualization for popular-science and educational presentations.

5 Minkowski Diagrams

A Minkowski diagram is the standard way of depicting the spacetime of special relativity. Figure 1 shows a typical example. Such a diagram is a direct visualization of the mathematical concepts described in Section 3: spacetime events are visualized as points (dots), four-vectors are shown as connecting lines, the wordlines of objects as lines, and reference frames are indicated by their respective coordinate axes. The light rays absorbed by the observer form a light cone that is also illustrated by a line (or a collection of lines). The popularity of Minkowski diagrams in textbooks and other scientiﬁc presentations of special relativity is rooted in the fact that those diagrams provide a geometric visualization of spacetime and its important constituents. In other words, a Minkowski diagram goes D. Weiskopf 295

S s t T w

o L X

E x e

Figure 1 1+1D Minkowski diagram show- Figure 2 2+1D Minkowski diagram with ing light emission from a static object and light several worldtubes and a backward light cone absorption by a moving camera. originating from a virtual camera. hand-in-hand with the mathematical description from Section 3. The superposition of inertial reference frames using a Minkowski diagram allows us to visualize the Lorentz transformation of an event. Figure 1 illustrates the Lorentz transformation of the event E to frame S0 by projection lines (light solid lines). Unfortunately, there are a few issues connected with Minkowski diagrams. One problem is the reduction of dimensionality: typically, only the temporal dimension and one spatial dimension are shown in a diagram. Sometimes a second spatial dimension is included. However, Minkowski diagrams never show all four dimensions of spacetime but only a subspace thereof. Therefore, these diagrams cannot provide a faithful representation of the complete physical scenario. Another issue is the interpretation of angles and inner products in a Minkowski diagram. Since the Minkowski metric (Eq. (1)) is different from the Euclidean metric, inner products in a Minkowski diagram cannot be inferred intuitively from our experience with Euclidean space, in which the diagram is rendered. For example, the inner product of the t and x axes of any reference frame vanishes even though these axes do not seem to be perpendicular in the rendered diagrams (see the coordinate system S0 in Figure 1). However, this interpretation issue is more of a problem in popular-science presentations than in scientific publications due to the different background knowledge of the readers. Therefore, Minkowski diagrams primarily address physicists and researchers as an audience. Minkowski diagrams are usually not based on complex scene descriptions or other external data because they rather target the visualization of simple spacetime relationships such as a lightlike connection between emission and absorption events. Therefore, Minkowski diagrams are typically generated without specific computer support. For example, they may be hand-drawn illustrations or produced with a generic vector graphics program. The educational software by Taylor [32] is one of the few examples where spacetime diagrams are computer-generated. Another example is the system by Diepstraten et al. [6] that automatically generates Minkowski diagrams from a 3D scene description. This system, as another advanced feature, allows for 2+1D diagrams with two spatial axes. 2+1D diagrams

C h a p t e r 1 9 296 A Survey of Visualization Methods for Special Relativity

v v v o o o

O O O

Figure 3 Spatial slices at three diﬀerent Figure 4 Virtual camera view for traveling times, taken from the motion of a cube-shaped at β = 0.99 toward the Brandenburg Gate. object.

have a couple of advantages compared to traditional 1+1D diagrams: extended objects can be shown instead of point particles; the visibility properties between objects can be visualized; objects may move in various directions of motion; angles become apparent in two spatial dimensions; ﬁnally, the relativistic aberration of light can only be seen in more than one spatial dimension. Figure 2 shows a 2+1D Minkowski diagram with several worldtubes (i.e., the 2D analog of wordlines) and a backward light cone originating from the observer. The intersections of the backward light cone with the worldtubes indicate the emission of light that is registered by the virtual camera.

6 Spatial Slices

Spatial slices can be constructed for a fixed time, which corresponds to a simultaneous measurement of positions in 3D space. Due to the relativity of time, a spatial slice is always associated with a reference frame—this visualization approach intrinsically depends on coordinate systems. Therefore, one may have to face apparent paradoxes if this frame dependency is not carefully observed. On the other hand, spatial slices are quite intuitive in the sense that they show simultaneous measurements. Spatial slicing is typically applied to present an “outside” perspective by an omniscient viewer in order to explain relativistic phenomena. Because such phenomena are usually related to some aspects of light propagation, typical visualizations include not only depictions of scene objects but also representations of light rays. Figure 3 shows a 2D example of several spatial slices taken at different times. Here, both a moving box-shaped scene object and a light ray are shown. Spatial slicing can be implemented by using standard 2D or 3D computer graphics that is based on the assumption of infinite speed of light. Instantaneous light transport is one way of realizing a simultaneous measurement. The only extension, as compared to standard computer graphics, is the need for a Lorentz transformation of scene objects when the reference frame is changed. While the Lorentz transformation could be computed according to the matrix–vector multiplication from Eq. (2), there is a simpler solution for uniformly moving objects: these objects are reduced in length with a factor 1/γ, which is called Lorentz contraction. Therefore, the relativistic effects can be included by a simple scaling along the direction of motion, which can even be done “manually” in any 2D or 3D of-the-shelf graphics tool (e.g. Adobe Illustrator or Maya). The main application of spatial slices is for physics education and popular-science presentations. Examples can be found on the web page [W3]. D. Weiskopf 297

7 Virtual Camera Model

The virtual camera model is based on a physical experiment: what kind of image would a camera produce in a special relativistic setting? This approach is the special relativistic analog of standard image synthesis. It is also related to spatial slicing from the previous section. The main difference is that the finite speed of light is taken into account for the virtual camera model. The virtual camera model is typically applied to a fast moving camera: how would the world be perceived in a “relativistic flight simulator”? Because this scenario is conceptually simple, it is appropriate for popular-science and educational illustrations [41].

7.1 Special Relativistic Polygon Rendering The idea of special relativistic polygon rendering is to transform the 3D geometry of a static scene to the apparent shape of the scene objects as seen by a relativistic observer. This approach works in object space, transforming the original 3D geometry to another 3D geometry. The scene objects emit light (either directly or indirectly through light reﬂection) and, thus, can be related to light emission events. Conversely, the camera is related to a light absorption event. Relativistic polygon rendering relies on the relationship between light emission and absorption events, and the Poincaré transformation of these events in order to allow for moving cameras or objects. The Minkowski diagram in Figure 1 illustrates these transformations for a single point-like scene object. The line {(t, xe)|t} denotes the worldline of the object in its rest frame S. The intersection of the worldline of the scene object with the backward light cone originating from the camera event O determines the emission event E, i.e., the connection between E and O is lightlike. The time coordinate of E in the frame S is determined by

0 0 p 1 1 2 2 2 2 3 3 2 xo − xe = (xe − xo) + (xe − xo) + (xe − xo) , (10)

µ µ where xe denotes the coordinates of E and xo the coordinates of O. The Poincaré transformation allows us to transfer the coordinates of the emission event from the scene frame to the camera frame. The spatial coordinates of that event can be used to render the object as seen by the fast moving camera. To render a more complex scene, the transformation of events is applied to all vertices of the tessellated scene. Please note that, due to the nonlinear nature of Eq. (10), the combined transformation of vertex locations is nonlinear. Relativistic polygon rendering fits to the GPU rendering pipeline: vertex coordinates are modified at the first stage of the rendering pipeline, whereas the other stages of the pipeline remain unaffected. The transformation of vertex coordinates can be done either by CPU processing or in a vertex program on the GPU. The object-space approach to special relativistic rendering is used in several papers with sometimes slight variations of the same computational theme [5, 9, 15, 24, 39]. Object-space relativistic rendering has several advantages: it is easy to implement, it is fast due to its direct support by graphics hardware, and it allows for scene objects that move at different speeds. The main disadvantage is caused by the nonlinear transformations of vertex coordinates. The linear connections between vertices through straight edges may lead to artifacts that are most prominent for large, nearby triangles. This problem can be overcome by a view-dependent re-tessellation to refine large triangles. The transformation of vertex positions only accounts for the apparent geometry as seen by the moving camera. The relativistic effects on illumination can be incorporated by modifying the color and intensity according to the Doppler and searchlight effects (see Eqs. (8) and (9)).

C h a p t e r 1 9 298 A Survey of Visualization Methods for Special Relativity

Figure 5 Polygon rendering of relativistic radiosity: non-relativistic view (left), aberration eﬀects for β = 0.6 (center), and all relativistic eﬀects included (right).

Typically, relativistic polygon rendering considers only direct illumination. However, global illumination is feasible as well. For example, special relativistic radiosity rendering allows for global illumination with diffuse reflections [40]. Like non-relativistic radiosity, the first rendering stage is view-independent and computes the radiosity solution for a static scene. The second stage uses relativistic polygon rendering to construct the image for a relativistic observer. Figure 5 shows an example of relativistic radiosity.

7.2 Image-Based Special Relativistic Rendering Image-based special relativistic rendering [42] computes images without the need for a 3D scene representation. It uses the plenoptic function, which describes the radiance field depending on light direction, spacetime position, and wavelength [1]. The plenoptic function is first acquired for a static camera in the rest frame of the scene, and then transformed to the frame of a moving camera. Afterwards, non-relativistic rendering methods are applied to generate the final image. The Lorentz transformation of the plenoptic function is governed by the following relativistic effects: the aberration of light changes the direction according to Eq. (5), the Doppler effect modifies the wavelength according to Eq. (8), and the searchlight effect alters the radiance according to Eq. (9). Often, the plenoptic function is considered only for a single camera location, leading to a panorama image taken from that position. Here, the relativistic aberration just leads to a nonlinear warping of the panorama. The Doppler and searchlight effects additionally change the color and brightness of the panorama. However, all these operations can be realized by simple image manipulations. Image-based rendering can be applied to real-world images [42] or to textures generated on-the-fly by non-relativistic rendering of 3D scenes [38, 41]. Figure 4 shows an example of texture-based rendering where only the geometric effects due to the aberration of light are visualized. Image-based rendering has the following advantages. First, it is well supported by the graphics pipeline of GPUs—non-relativistic rendering may be used to generate the panorama and only one additional rendering step is required for the image transformation. Second, no view-dependent re-tessellation of the scene is needed because the nonlinear transformations work on a per-pixel basis for the panorama image. Image-based rendering is per-pixel accurate, both for the geometric and illumination effects. A disadvantage of image-based rendering is that the panorama needs to be acquired at high resolution to ensure a sufficient sampling rate after the nonlinear Lorentz transformation. Another shortcoming is the lack of support for objects that move at differing speeds. D. Weiskopf 299

7.3 Special Relativistic Ray Tracing

A third class of rendering approaches is based on ray tracing. For scenarios with a single static scene and a moving observer, non-relativistic ray tracing needs only a slight modification to incorporate relativistic effects: the primary ray directions are transformed from the moving camera frame to the frame of the static scene [12]; afterwards, non-relativistic ray tracing is performed. The ray direction and wavelength can be transformed by applying the Lorentz transformation to the corresponding wavevector (see Eq. (4)). The radiance is transformed according to Eq. (9). Full 4D special relativistic ray tracing is needed to include more advanced effects such as accelerating scene objects and shadowing, reflection, or transmission of light [19, 41]. 4D ray tracing represents light rays, scene objects, and the intersection between rays and objects in 4D spacetime. In this way, animated and accelerated objects can be modeled. To compute the local illumination at an intersection event, the light information needs to be transferred into the rest frame of the object that is hit by the ray. This transformation is accomplished by Eqs. (5), (8), and (9). One advantage of ray tracing is high image quality and the support for reflection and transmission. Full 4D ray tracing additionally extends the range of effects to include accelerated objects and advanced interreflection between objects that move relative to each other. Therefore, 4D ray tracing is the visualization method with the widest range of supported relativistic effects. The main disadvantage of ray tracing are high computational costs, which typically make interactivity impossible.

7.4 Acceleration

The accelerated motion of a point-like object through spacetime can be computed by solving a corresponding equation of motion (an ordinary diﬀerential equation). In this way, accelerating cameras can be modeled [24, 39, 44]. The image generation for an accelerating camera is identical to the relativistic rendering for a co-moving, non-accelerating camera (i.e., a camera that moves with the same instantaneous velocity). Therefore, any of the aforementioned rendering techniques can be employed. The acceleration of extended scene objects is more complex and subject to issues of correct physical modeling to ensure a consistent and physically plausible motion [16, 41]. Here, the rendering is usually accomplished by 4D ray tracing.

8 Open Issues

Most of the aforementioned virtual camera techniques facilitate rendering at interactive rates so that the efficiency of relativistic rendering can be considered a solved problem. A largely unsolved issue, however, is the photorealistic computation of wavelength-dependent radiance in a scene, which is the prerequisite for realistic rendering of the Doppler and searchlight effects. The main challenge is the acquisition or modeling of wavelength-dependent properties of scene objects and light sources (even beyond the visible spectrum because of the Doppler shift). Although there exist some interaction metaphors for the virtual camera approach [39, 41], the development of appropriate user interfaces is a goal of on-going research. Similarly, the computer support for generating Minkowski diagrams and spatial slicing could be greatly improved by specific interaction models and design interfaces.

C h a p t e r 1 9 300 A Survey of Visualization Methods for Special Relativity

Web Links

A wealth of information on special relativistic visualization can be found on the web. Examples are: a list of special relativistic ﬂight simulators [W2], a timeline for the development of computer-based relativistic visualization [W1], and didactics material for teaching relativity [W3].

Web Links W1 D.V. Black. Visualization of non-intuitive physical phenomena. http://www.hypervisualization.com [accessed Feb 20, 2006]. W2 A. Hamilton. Guide to special relativistic ﬂight simulators. http://casa.colorado.edu/~ajsh/sr/srfs.html [accessed Feb 20, 2006]. W3 U. Kraus, C. Zahn. Space time travel. http://www.spacetimetravel.org [accessed Feb 20, 2006].

References 1 E. H. Adelson and J. R. Bergen. The plenoptic function and the elements of early vi- sion. In Michael Landy and J. Anthony Movshon, editors, Computational Models of Visual Processing, pages 3–20, Cambridge, 1991. MIT Press. 2 Christopher Betts. Fast rendering of relativistic objects. The Journal of Visualization and Computer Animation, 9(1):17–31, 1998. 3 Mary L. Boas. Apparent shape of large objects at relativistic speeds. American Journal of Physics, 29(5):283–286, May 1961. 4 John Robert Burke and Frank J. Strode. Classroom exercises with the Terrell effect. Amer- ican Journal of Physics, 59(10):912–915, 1991. 5 Meng-Chou Chang, Feipei Lai, and Wei-Chao Chen. Image shading taking into account relativistic effects. ACM Transactions on Graphics, 15(4):265–300, October 1996. 6 J. Diepstraten, D. Weiskopf, and T. Ertl. Automatic generation and non-photorealistic rendering of 2+1D Minkowski diagrams. In Proc. WSCG 2002, pages 139–146, 2002. 7 Albert Einstein. Zur Elektrodynamik bewegter Körper. Annalen der Physik, 17:891–921, 1905. 8 George Gamow. Mr Tompkins in Wonderland. University Press, Cambridge, 1939. 9 Walter Gekelman, James Maggs, and Lingyu Xu. Real-time relativity. Computers in Physics, 5(4):372–385, July/August 1991. 10 F. R. Hickey. Two-dimensional appearance of a relativistic cube. American Journal of Physics, 47(8):711–714, 1979. 11 D. Hollenbach. Appearance of a rapidly moving sphere: A problem for undergraduates. American Journal of Physics, 44(1):91–93, 1975. 12 Ping-Kang Hsiung and Robert H. P. Dunn. Visualizing relativistic effects in spacetime. In Proc. Supercomputing ’89, pages 597–606, 1989. 13 Ping-Kang Hsiung, Robert H. Thibadeau, Christopher B. Cox, and Robert H. P. Dunn. Time dilation visualization in relativity. In Proc. Supercomputing ’90, pages 835–844, 1990. 14 Ping-Kang Hsiung, Robert H. Thibadeau, Christopher B. Cox, Robert H. P. Dunn, Michael Wu, and Paul Andrew Olbrich. Wide-band relativistic Doppler effect visualization. In Proc. IEEE Visualization, pages 83–92, 1990. 15 Ping-Kang Hsiung, Robert H. Thibadeau, and Michael Wu. T-buffer: Fast visualization of relativistic effects in spacetime. Computer Graphics, 24(2):83–88, 1990. D. Weiskopf 301

16 U. Kraus, H. Ruder, D. Weiskopf, and C. Zahn. Was Einstein noch nicht sehen konnte. Schnelle Computer visualisieren relativistische Effekte. Physik Journal, 1(7/8):77–83, July 2002. 17 Ute Kraus. Brightness and color of rapidly moving objects: The visual appearance of a large sphere revisited. American Journal of Physics, 68(1):56–60, 2000. 18 Anton Lampa. Wie erscheint nach der Relativitätstheorie ein bewegter Stab einem ruhen- den Beobachter? Zeitschrift für Physik, 27:138–148, 1924. 19 Jiang Li, Heung-Yeung Shum, and Qunsheng Peng. An improved spacetime ray tracing system for the visualization of relativistic effects. In Eurographics 2001 Short Presentations, 2001. 20 Hermann Minkowski. Raum und Zeit. Physikalische Zeitschrift, 10:104–111, 1909. 21 Charles W. Misner, Kip S. Thorne, and John Archibald Wheeler. Gravitation. Freeman, New York, 1973. 22 C. Møller. The Theory of Relativity. Clarendon Press, Oxford, second edition, 1972. 23 Roger Penrose. The apparent shape of a relativistically moving sphere. Proceedings of the Cambridge Philosophical Society, 55:137–139, 1959. 24 René T. Rau, Daniel Weiskopf, and Hanns Ruder. Special relativity in virtual reality. In Hans-Christian Hege and Konrad Polthier, editors, Mathematical Visualization, pages 269–279. Springer Verlag, Heidelberg, 1998. 25 G. D. Scott and H. J. van Driel. Geometrical appearances at relativistic speeds. American Journal of Physics, 38(8):971–977, August 1970. 26 G. D. Scott and R. R. Viner. The geomentrical appearence of large objects moving at relativistic speeds. American Journal of Physics, 33(7):534–536, July 1965. 27 Garrett P. Serviss. The Einstein Theory of Relativity. E. M. Fadman, New York, 1923. 28 Eric Sheldon. The twists and turns of the Terrell effect. American Journal of Physics, 56(3):199–200, 1988. 29 Arnold Sommerfeld. Zur Relativitätstheorie I: Vierdimensionale Vektoralgebra. Annalen der Physik, 32:749–776, 1910. 30 Arnold Sommerfeld. Zur Relativitätstheorie II: Vierdimensionale Vektoranalysis. Annalen der Physik, 33:649–689, 1910. 31 K. G. Suffern. The apparent shape of a rapidly moving sphere. American Journal of Physics, 56(8):729–733, 1988. 32 Edwin F. Taylor. Space-time software: Computer graphics utilities in special relativity. American Journal of Physics, 57:508–514, 1989. 33 Edwin F. Taylor and John Archibald Wheeler. Spacetime Physics. Freeman, New York, second edition, 1992. 34 James Terrell. Invisibility of the Lorentz contraction. Physical Review, 116(4):1041–1045, November 1959. 35 James Terrell. The Terrell effect. American Journal of Physics, 57(1):9–10, 1989. 36 Scott Walter. Minkowski, mathematicians, and the mathematical theory of relativity. In H. Goenner, J. Renn, J. Ritter, and T. Sauer, editors, The Expanding Worlds of General Relativity, pages 45–86. Birkhäuser, Boston, Basel, 1999. 37 Milena Wazeck and Thomas Bührke. Der verschollene Film. Bild der Wissenschaft,, pages 47–49, February 2005. 38 Daniel Weiskopf. Fast visualization of special relativistic effects on geometry and illumination. In Proc. EG/IEEE TCVG Symposium on Visualization, pages 219–228, 2000. 39 Daniel Weiskopf. An immersive virtual environment for special relativity. In Proc. WSCG 2000, pages 337–344, 2000. 40 Daniel Weiskopf. Visualization of Four-Dimensional Spacetimes. PhD thesis, Universität Tübingen, 2001. http://w210.ub.uni-tuebingen.de/dbt/volltexte/2001/240.

C h a p t e r 1 9 302 A Survey of Visualization Methods for Special Relativity

41 Daniel Weiskopf, Marc Borchers, Thomas Ertl, Martin Falk, Oliver Fechtig, Regine Frank, Frank Grave, Andreas King, Ute Kraus, Thomas Müller, Hans-Peter Nollert, Isabel Rica Mendez, Hanns Ruder, Tobias Schafhitzel, Sonja Schär, Corvin Zahn, and Michael Zat- loukal. Explanatory and illustrative visualization of special and general relativity. IEEE Transactions on Visualization and Computer Graphics, 12(4):522–534, 2006. 42 Daniel Weiskopf, Daniel Kobras, and Hanns Ruder. Real-world relativity: Image-based special relativistic visualization. In Proc. IEEE Visualization, pages 303–310, 2000. 43 Daniel Weiskopf, Ute Kraus, and Hanns Ruder. Searchlight and Doppler eﬀects in the visualization of special relativity: A corrected derivation of the transformation of radiance. ACM Transactions on Graphics, 18(3):278–292, July 1999. 44 Daniel Weiskopf, Ute Kraus, and Hanns Ruder. Illumination and acceleration in the visualization of special relativity: A comment on fast rendering of relativistic objects. The Journal of Visualization and Computer Animation, 11(4):185–195, 2000. 45 Victor F. Weisskopf. The visual appearance of rapidly moving objects. Physics Today, 13(9):24–27, 1960.