picture from Visual Sphere: a new approach to image rasterization

Jakub Maximilian Fober∗ January 7, 2020

S 1 Introduction

→ A This paper presents new perspective model. One based on a Gaussian Sphere, able to reproduce G image of any shape, perspective geometry and an- gle of view (AOV). A model capable of combin- ing pictures from sources previously incompati- ble, like fish-eye camera view and wide-angle lens h picture. Based on this model, new rasterization → B method is presented, one able to render real-time → C curvilinear image directly from third-dimension. This work is divided into nine sections, grad- ually unfolding the topic, from philosophical Figure 1: Projection of ABC triangle onto image standpoint, to technical specifics. In following grid G of a visual unit sphere S. Projected edge section, the history of perspective picture is in- CA is formed by the great circle h. troduced, establishing grounds for later discus- sion. Sections three and four overview percep- tion of perspective picture, its geometry and sen- Abstract sation of motion. Following sections five and six present Universal Perspective model. Fifth There is great demand for perspective projection section relates to mathematics of perspective; model able to produce computer-generated (CG) present equations of transformation and projec- image up to 360° of view, with lens-distortions, tion of 2D/3D data and lens distortions. Sixth directly from three-dimensional space, to pixel section presents rasterization process for 3D sur- data. Currently there is no practical direct faces. Section number seven refers to generat- method for rasterization of real-time graphics in ing barycentric coordinates of a triangle in the curvilinear perspective. Every real-time perspec- Universal Perspective model. Section eight in- tive imagery incorporates Pinhole Camera model corporates Universal Perspective model in mea- as a base, some with additional layers of distor- surement and simulation of real optical systems, tion on top. Also to note, knowledge about rela- with variable no-parallax point. Section nine fol- tionship between motion and perspective has not lowed by references, conclude all information and been properly formulated, leaving a void in that present direction for further studies. All support field of image science. graphical material is presented on last five pages. This paper aims at solving those issues. Study involves exploring history of perspective picture, redefining abstract theorem of image (as recorded in common-knowledge), establishing rules of per- spective image and presenting new, universal model for perspective projection and rasteriza- tion in CG graphics.

Keywords: perspective, non-linear projec- Figure 2: Rasterized polygon with texture (Paolo tions, spherical perspective, graphics hardware, th mathematics of art Uccello, 15 century) from a perspective , where Ωd = 270°, k = 0.32, l = 62%, s = 86%. ∗ [email protected] & http://maxfober.space/

1 2 Previous work and history of mixed spherical and cylindrical projection in of the topic anamorphic lens, with perspective preservation. Definition. Conservation of perspective - lines Current image abstract theorem was established th converging at the optical-axis vanishing-point re- in 15 century, in a book De Pictura, by main straight (also see perspective picture defini- L. B. Alberti. Based on invention of F. tion on the next page). Brunelleschi, Alberti defined geometrical and theoretical rules for designing perspective CG image technology did not follow film indus- projections.2 These rules are currently used in try progress in that field. Still based on Alberti CG polygon-based graphics. theorem, became incompati- Major theoretical statement that laid founda- ble with film, generating great costs, when two 20 tion for image projection technologies and current had to be joined together. Which part took lens understanding of image nature can be found in aberration rotoscopy, where geometry correction Alberti abstract definition of image. He would has to be performed manually at each frame. describe a painting to be like a window in a wall.1 Currently in computer-games industry, CG im- agery is practically unable to produce realistic, “First of all, on the surface on which I curvilinear simulation of visual space (VS), or am going to paint, I draw a rectangle of even simulate anamorphic lens geometry, due to whatever size I want, which I regard as limits of linear perspective and resource costs an open window through which the sub- of overcoming those. Some hybrid solutions for 1 ject to be painted is seen.” real-time graphics where proposed,11,12 that com- bine rasterization with ray-tracing, or tessella- But in times of its discovery, as for now, linear tion. Such approach allows for a semi-practical perspective introduced itself with several issues. and limited production of real-time pictures in a When there is a need for a wide-angle view, one non-linear perspective. close to the human visual field, geometrical dis- tortions appear to dominate visual aspect of the picture. Those issues were noticed by Renais- 3 On visual space geometry sance artists, like L. Da Vinci. He put to test the Alberti theorem and produced paintings of and image perception accurate-perspective.6 In his Treatise on Paint- Perspective picture visible inside the visual space ing, Da Vinci notes that picture conforms to the gives some sense of immersion (e.g. picture, film, idea of a window only when viewed from one spe- computer game) even without visual illusion.8 It cific point in space.7 Stating that seen otherwise, is perceived as a visual symbol of an abstract objects appear distorted, especially in the periph- point of view. Picture immersion does not break, ery. Picture then, viewed from a different point as long as object’s appearance do not exhibit too ceases to be like a window in a wall and becomes much deformation. a visual symbol of an abstract point of view.a Representing point of view invokes separation Some 18th century late Baroque and Neo- from the surrounding. To enhance this effect, en- classical artists, when encountered those is- vironment stimuli is being reduced, like in case sues, introduced derivative projections. Like of movie theater. To uphold the immersion lights G. P. Pannini with later re-discovered Panini are turned off and silence is expected. Or in case Projection,17 or R. Barker, who established of horror-game session played at night, to sepa- the term Panorama.25 It was a new type of rate from safe-space of home. perspective, in a form of cylindrical projection, where abstract window frame became horizon- Remark. On the opposite side, picture which is tally curved, reducing deformation artifacts in a an integral part of the surrounding, can be cate- 23 wide, panoramic depictions of architecture. gorized under the Trompe l’œil technique. Since most of the time picture presents point Invention of motion picture followed by the of view (e.g. film, video game, visualiza- rise of film industry, resulted in demand for a tion), it’s wise to consider subject’s properties new image geometry. Previously still, now pic- of vision when designing picture’s perspective. tures had to be pleasing to the eye, in motion. But instead of producing mechanical simula- 1950s brought anamorphic cinematography to the tion, perspective should symbolize total sensory 2 wider audience. Lenses like CinemaScope and experience. later Panavision14 became a standard in film Theorem. To create immersive visual symbol of production. Figure 3 on page 11 shows example a visual space, it is necessary to use the curvilin- aEffect also referred to Zeeman Paradox.8 ear perspective instead of linear.

2 Proof. Geometry of human visual space contra- panini, to symbolize binocular vision; two dicts linear perspective principle, as visual field spherical projections combined into extends beyond linear perspective angle of view. one panoramic image.b Linear perspective, based on a tangent of an an- gle, exhibits limit of 179. (9) 8° of view. While anamorphic, as cylindrical projections, like visual field extends horizontally up to 220° for Panini, tend to elongate propor- 13 binocular vision.  tions vertically; there is a need for correction. At narrow AOV both types of perspective are suitable for immersive picture. In such case dif- Visual sphere as a whole image ferences are negligible. Sub-figure 4a on page 11 presents these differences in comparison to five Common idea of an image is limited to a finite, major perspective projections. Same differences two-dimensional plane. Which is subjective, due seem exaggerated at higher AOV values (see Sub- to constrains of human visual field and up-front figure 4b). placement of eyes. One can construct a rectan- Corollary. Practical limit for immersive pic- gular frame, which at certain distance from the ture in linear perspective is between 60° and 110° eyes will cover full visual field (VF). In case of some animals (e.g. horse, rabbit), visual space AOV. Wider-angles exhibit deformations known 3,16 as the Leonardo Paradox,8 which are then domi- confines much wider VF. With only few blind nant in the image perception and break picture’s spots, spanning to almost 360°. Such field cannot immersion.  be enclosed by a single rectangular frame. Thus image nature is not of a frame. Another model To show wider-angle picture it is necessary to has to be chosen instead. One able to cover full use curvilinear projection. But there is a ten- Ω = 360° is a sphere. dency to see the world not through sight but the understanding. We understand that the wall is Remark. Cylindrical projections cannot cover full 360 flat, therefore we see it that way. Picture pro- ° of view, in all directions. They are a hy- jected into the eye is just a visual symbol of a brid between frame and spherical model. When Ω < 180 physical wall and has its own physical proper- vertically-oriented, full v °. ties (e.g. perspective and shape). Therefore its All three-dimensional space around given ob- visual representation is curvilinear, where curva- servation point, can be projected onto a sphere, ture symbolizes wider field of view. with given observation point as origin. Even doe sphere itself is a 3D object, its surface (as well as Exercise. Reader can validate the curvilinear image nature)19 is two-dimensional. nature of human visual space,4,10 by following A. Therefore creating perspective picture, is mat- Radley experiment: ter of representing portion of the visual sphere on “Also when you have a moment, get a a flat surface, a fundamental topic in . 30 cm ruler (. . . ), and whilst looking forward bring it close to the bottom of Definition. Let us define perspective picture as your nose, and notice how its shape at the azimuthal projection, where lines converg- the outer edges curves upwards and for- ing at the optical axis remain wards. It may take you a few minutes to straight. Such that conservation of perspective be able to see this effect, because you are occurs (see definition on the previous page). so accustomed to not noticing it ! But Remark. Each projection of sphere on a flat sur- once you do you will be amazed to see face is a compromise and can preserve only some your curved field of view as it really is properties (e.g. shape, area, distance or direc- for the first time”.18 tion), which in case of perspective picture re- lates to some symbolic information about physi- Visual space symbolic picture cal space. Problem. Which curvilinear perspective would be best for a visual symbol of visual space? Physical space properties preserved in az- imuthal projections Proposition. A model based on the anamorphic lens geometry; a mix between fish-eye, panini and Gnomonic (rectilinear) projects all great circles anamorphic projection. as straight lines, thus preserving di- rections. For , straight- fish-eye, as it can represent wider AOV, than line in object-space remain straight. linear perspective (e.g. π) and con- forms to the curvilinear nature of VS. bEffect also referred as stereopsis.

3 It does not preserve proportions, an- Example. Most commonly found in very cheap gles nor area or distances (see fig- lenses, like the peephole door viewer. Thanks to ure 5a). Extreme distortion occur illuminance preservation, it’s commonly used in away from the center, in form of a ra- sky photography.22 dial stretch (see Leonardo Paradox)8. Ω ∈ (0, π) AOV . 4 Image geometry and sensa- Example. Most common perspective type in painting, 3D graphics and architectural visual- tion of motion ization. Sometimes it is used to overempha- Image perspective affects the way motion picture size building’s appearance by leveraging Leonardo is perceived. It can enhance certain features, like Paradox.8 Wide AOV combined with lowered op- proportions and shapes, movement or spatial sen- tical center creates an effect of acute corners, giv- sation. It can also guide viewer’s attention to a ing extraordinary look. This technique may con- specific region of image (e.g. center or periphery). fuse the public, as symbolic picture experience Knowledge about these properties is essential for won’t match building’s visual-space appearance. conscious image design. Stereographic (conformal) preserves angles (at line intersection point). There is no Attention focusing perceivable radial compression, thus smaller figures retain their shape. Remark. In a film design, there are several tech- It does not preserve distances (non- niques to focus viewer’s attention on a specific isometric), nor angular surface area. portion of the picture, like motion, color, light For 3D projection, most important and composition. Attention focusing though com- factor is the conservation of propor- position and motion is related to image perspec- tions (see figure 5b). AOV Ω ∈ tive, as its geometry can compress and stretch (0, 2π). the image. In composition, rule of thirds states that viewer’s attention focuses on four corners Example. In a picture with stereographic pro- of a rectangle produced by division of the image jection, actor’s face keeps its shape and propor- into three, equal-size rows and columns. In mo- tions, even at wide AOV. This projection also tion, attention generally drives towards objects gives best spatial sensation (where visual cues are approaching the camera or those growing in scale. available). Good use case is navigation through Attention also focuses on objects entering the pic- tight spaces and obstacles. ture frame. Same rules apply loosely in reverse, Equidistant preserves angular distance from the as attention suspense. center point (see figure 5c). For 3D Filmmakers tend to frame the image so that projection, angular speed of motion region of interest lays in accordance to the rule is preserved. Radial compression re- of thirds. In case of computer games, region of in- mains low-to-moderate at extreme Ω terest is usually located at the center, thus viewer angles. AOV Ω ∈ (0, 2π]. must overcome the principle of thirds and some properties of linear perspective in order to switch Example. This projection is recommenced for attention on that region. In order to focus on target aiming or radar map navigation, where all the center, games usually incorporate some non- targets are projected onto a Gaussian Sphere. diegetic elements, like crosshair. Such approach 5 Equisolid preserves angular area. Gives good may lower immersiveness of symbolic picture. sensation of distance (see figure 5d). Radial compression is moderate up to Attention focusing motion of perspective π. Near the maximum Ω, compres- Radial stretching and compression are the main sion is high. AOV Ω ∈ (0, 2π]. attention focusing factors of perspective projec- Example. When there are no spatial ques, this is tion. They give subconscious sensation of move- best projection to put emphasis on a distance to ment towards camera, and can amplify figures the viewer.12 Good use case is flight simulation, screen-relative speed of motion. where only sky and other aircraft are in-view. Gnomonic (rectilinear), due to extreme radial Orthographic-azimuthal preserves planar illumi- stretch, attention drives towards periph- nance. It is a of ery. When approaching image bounds fig- a visual hemisphere. Has extreme ures grow in scale and speed (see figure 5a). radial compression, especially near π This combined with motion-sensitive periph- (see figure 5e). AOV Ω ∈ (0, π]. eral vision adds to the effect. At wider AOV

4 amplified motion breaks immersion of sym- Universal perspective 3D→2D coordinates bolic picture. transformation

Stereographic also draws attention towards pe- vˆ = (ˆvx, vˆy, vˆz)  q  riphery. Figures grow in scale near bounds, 2 2 2 θ = arccos vˆz ÷ vˆx + lvˆy +v ˆz but immersion does not break, as propor-  Ω  tions are preserved, even at wide AOV (see tan (kθ) ÷ tan k , 0 < k ≤ 1  2 figure 5b). Ω R = θ ÷ 2 , k = 0  Ω  sin (kθ) ÷ sin k 2 , 0 > k ≥ −1 Equidistant drives attention towards the cen- ter, as figures in periphery are radially com- " ~ #     pressed (see figure 5c). This projection pre- fx vˆx R 1 ~ = q fy vˆy 2 2 l (1 − s) + s serves screen-relative, radial speed of mo- vˆx + lvˆy tion, making it uniform and representative across the picture. 3D coordinates are represented by a normalized 3 vector vˆ ∈ [−1, 1] , where view origin is at po- Equisolid sition (0, 0, 0). Transformed picture coordinates also drives attention towards the cen- ~ 2 ter, as radial compression is even greater (see are represented by vector f ∈ [−1, 1] , where im- figure 5d). Figure speed of motion in screen- age center is at position (0, 0). space slightly declines towards periphery. Angle θ is between vector vˆ and the Z axis. R is the normalized distance between projected vector f~ and the image center, where f~ ← vˆ. Orthographic has extreme radial compression Angle Ω is equal to AOV, where Ωmax ∈ [ [π, 2π]. that breaks immersion of symbolic picture Scalar k represents various projection types: (see figure 5e). When in motion, image seems to be imposed on an artificial sphere. Gnomonic (rectilinear) k = 1 Stereographic...... k = 0.5 k = 0 Gnomonic and Orthographic projections are the Equidistant...... k = −0.5 two extremes of azimuthal spectrum. They are Equisolid ...... k = −1 both least suited for an immersive picture. Orthographic ...... Cylindrical perspective type, while symbolizing l ∈ (0, 1] binocular vision, also gives visual cues for vertical Scalar is the spherical projection factor, l ≈ 0 axis orientation. Such cue is undesirable in case of where represents cylindrical projection and l = 1 camera roll motion, or when the view is pointing spherical projection. s ∈ [4/5, 1] up or down, as image vertical axis will not align Scalar describes anamorphic correction s = 1 l = 1 with depicted space orientation. In such case per- of non-spherical image. For or there spective geometry should transition from panini is no anamorphic correction. to spherical projection (see figure 6 on page 12). Universal reverse 2D→3D coordinates transformation

~  ~ ~  5 Perspective picture trans- f = fx, fy q formations ~2 ~2 R = fx + lfy  Ω   Below are presented algorithms for producing arctan tan k 2 R ÷ k, 0 < k ≤ 1  Ω custom perspective picture, from 3D and 2D θ = 2 R, k = 0 data. For a proper transformation, 2D coordi-  Ω   arcsin sin k 2 R ÷ k, 0 > k ≥ −1 nates must be normalized for a given AOV type (e.g. vertical, horizontal or diagonal).    ~   sin(θ)    vˆx fx /R 1 sin(θ) 1 Example. i  vˆy  =  f~   /R    For a pixel in a picture of aspect- y l(1−s)+s ratio 16:9 and AOV measured horizontally, co- vˆz 1 cos (θ) 1 ordinates (ix, iy) must be centered and horizon- ~ tally normalized, so that ix ∈ [−1, 1] and iy ∈ Picture coordinates are represented by vector f ∈  9 9  2 − 16 , 16 . [−1, 1] , where image center is at position (0, 0).

5 Transformed 3D coordinates are represented by points get stretched into lines, like in equidistant 3 normalized vector vˆ ∈ [−1, 1] . This transforma- projection, where at Ω = 360° point opposite to tion is a reverse of the Universal 3D→2D trans- the camera direction forms a ring around picture form on the previous page. bounds. Both transforms produce perspective picture In proposed visual sphere model, every per- (see definition on page 3). Such formula allows spective picture has its own spherical coordinates, for a smooth adjustment of image geometry in ac- at each pixel. Thus single point can occupy mul- cordance to visible content. Base projection type tiple places, conforming to the principles of curvi- is adjusted by the k component. It manipulates linear perspective. Such format of perspective image perception. Cylindrical projection, is ad- allows for stitching and layering images of any justed by the l component. Low l values should perspective geometry. represent view at level (see figure 6a). For roll Remark. Visual sphere image can be recon- motion, recommended value l is 100% (see fig- structed from six visual-pyramid pictures,c each ure 6b). Anamorphic correction of non-spherical covering Ω = 90°, with three mutually perpendic- image, driven by the s component, depends on ular and three adjacent camera view directions. the subject in view. Purpose of the s scalar is to adjust figure’s proportions. Points in spherical projection model are no longer Combination of 3D and 2D transformation can transformed into screen space. Rather lines are be used to map between two different projections, calculated and combined to form a polygon im- for example Stereographic ↔ Equidistant, using age (see figure 7a on page 13). This process in- two separate projection components, ki and ko volves rotating perspective vector map data. The for input and output image. goal is to align one of the axis with a great cir- cle of the polygon edge (see figure 7b). One way Lens distortion of perspective picture to rotate one axis component is to calculate dot product between the perspective map vector and Creating perspective picture of a real optical sys- unit vector perpendicular to the two edge points. tem may require additional deformation of the vector data. Most commonly used is the Brown- 24 Rasterization of the ABC polygon Conrady lens distortion model. Lens distortion ˆ vector is added to the frame coordinates f~ vector. triangle using vector G from per- spective map r = f~ · f~ Projected polygon geometry is always part of a " ~ # " ~ #   ~0 fx fx k1r great circle. The goal of the algorithm is to ras- f = ~ + ~ fy fy k1r terize polygon shape formed by those spherical " # lines. Rasterization process involves determin- f~  k r2   q r  + x 2 + 1 ing orientation of the great-circle. Then rotating ~ k r2 q r fy 2 2 perspective map, so that the vertical axis aligns " #! " #  p  f~ f~ with this great-circle. Next, the step-function is 1 x x ˆ0 + · performed on the Gx component of the rotated p2 f~ f~ y y vector Gˆ0. Full polygon picture is defined by in- tersection of three such operations. Where r is the dot product of two f~ vectors. k1 and k2 are the radial distortion coefficients. Rotation of Gˆx axis component to a great q1 and q2 are the prism aberration coefficients. circle of two triangle points A~ and B~ p1 and p2 are the misalignment coefficients.

ˆ ~ ~ 6 From visual pyramid, to vi- X = A × B Gˆ0 = Gˆ · Xˆ sual sphere x 3 Where Xˆ ∈ [−1, 1] is a normalized cross product Visual pyramid of Alberti theorem is by defini- of triangle vectors A~ and B~ , where {A,~ B,~ C~ } ∈ tion limited to acute angles, which is restricting 3. Rotated perspective map Gˆ0 component is in terms of projections it can describe. This prop- R x derived from a dot product of the rotated Xˆ axis erty makes stitching or layering multiple pictures vector and perspective map Gˆ vector. At position defined in such space a problematic task. Gˆ0 = 0 lays the great circle crossing points Aˆ and In standard perspective model 3D point posi- x Bˆ. tion is transformed into 2D screen coordinates. But in a case of some curvilinear projections, cProcess known as cube mapping

6 beyond the outline (due to half-pixel offset in the ! pstep(x) function). It can be corrected by a miter Gˆ0 1 pstep(Gˆ0 ) = x + ∩ [0, 1] mask. There are many ways to form such mask, x ˆ0 |∆Gx| 2 one is to define the smallest circle over projected ABC triangle. Furthermore rasterization matrix Function pstep(x) ∈ [0, 1] is a variation of a sim- can be extended to 4×3, which product is four- ple step(x) function, but with anti-aliasing. It component mask vector. rasterizes hemisphere on which the opposite point ˆ ˆ0 C lays. |∆Gx| is equivalent to the fwidth(x) func- tion, which returns pixel width of x. If back-  ~ ~   ˆ  A × B facing polygons are to be rendered, the Gˆ0 must Gx ~ ~ x ~ ˆ  B × C  −Gˆ0 T =  Gy    be flipped ( x) when polygon is facing back.  C~ × A~  Gˆ z Sˆ N~ = (A~ − B~ ) × (C~ − B~ ) ~ ~ ~ t = pstep(T1) ∧ pstep(T2) ∧ pstep(T3)  1, if x > 0 ~  ∧ pstep(T4 − S) sign(x) = 0, if x = 0  ~ 4 −1, if x < 0 Where T ∈ [0, 1] represents source for triangle ~ ~ ~ ~ masks of each edge (T1, T2, T3) and miter T4. Vec- f = sign(N~ · A~) tor Sˆ is the smallest-circle center vector and S is If the triangle is back-facing, sign of a dot product a step function threshold point, equal to cosine ˆ between triangle normal N~ and one of the triangle of an angle between S and circle edge. To de- points will be equal to −1. If it’s front-facing termine smallest circle center and step threshold, ˆ0 C. Ericson 9 the result will be 1. Therefore multiplying Gx based on solution, following algo- component by f will properly render back-facing rithm can be defined. polygons. ˆ0 ˆ ˆ Rasterization of triangle with matrix B = B − A Cˆ0 = Cˆ − Aˆ Since triangle edge rasterization is performed d = (Bˆ0 · Bˆ0)(Cˆ0 · Cˆ0) − (Bˆ0 · Cˆ0)2 three times, full triangle rasterization can be rep-  0 0 0 0 0 0 0 0  resented as a 3 × 3 matrix multiplication. (Bˆ ·Bˆ )(Cˆ ·Cˆ )−(Bˆ ·Cˆ )(Cˆ ·Cˆ ) ~b = 2d  (Bˆ0·Bˆ0)(Cˆ0·Cˆ0)−(Bˆ0·Cˆ0)(Bˆ0·Bˆ0)  2d  Gˆ   A~ × B~  x ~ ~ ˆ ~ ~ s = b1 T =  Gy   B × C  ˆ ~ ~ t = ~b Gz C × A 2  ˆ ˆ ~ ~ ~ A+C , if s ≤ 0 t = pstep(T1) ∧ pstep(T2) ∧ pstep(T3)  2  Aˆ+Bˆ  , else if t ≤ 0 n S~ = 2 P Bˆ+Cˆ [0, 1] ∩ ti  2 , else if s + t ≥ 1 i  Aˆ + sBˆ0 + tCˆ0, otherwise ABC t Rasterising triangle image of perspec- S = |S~| tive vector map Gˆ is equivalent to the intersec- tion of pstep(x) function of each component of ~ 2 3 Where b ∈ R is the two-component barycentric T~ ∈ [−1, 1] vector. T~ vector is derived from ˆ position of the circumcenter. If d = 0, all pro- multiplication of perspective vector map G vec- 9 jected A,ˆ B,ˆ Cˆ points lay in line, therefore trian- tor and a rasterization matrix. Each row of the gle is degenerate and miter mask can be omitted. rasterization matrix represents great-circle tan- Vector S~ represents center of the smallest circle. gent vector, which is a normalized cross product Threshold for pstep(x) function is denoted by S between points forming the triangle edge. When as cosine of the angle between smallest circle edge combining multiple triangle masks it’s important and the center vector, which is equivalent to the to sum each t mask, otherwise gaps in-between length of S~. polygons will occur.

Miter of the anti-aliased triangle outline Wire-frame AB line rasterization In special case, when projected polygon edges Following algorithm will produce wire-frame im- meet at very narrow angle, its corers will extend age of projected line segment.

7 ! ~ ~ |Gˆ0 | N · A lpstep(Gˆ0 ) = 1 − x ∩ [0, 1] u = x ˆ0 N~ · Gˆ ∆Gx ≡ |Gˆ → ABC| Aˆ + Bˆ Lˆ = 2 Distance u represents multiplier of the visual ˆ m = pstep(Gˆ · Lˆ − Lˆ · Aˆ) sphere vector G, to intersection point on the ABC triangle plane. Since Gˆ is a unit vector, l = m ∧ lpstep(Gˆ0 ) x value u can be exported as depth. Here, vector 0 A~ Where Lˆ is the AB line-middle vector, lpstep(Gˆ ) can be replaced by any point on the triangle x A~ B~ C~ function rasterizes great-circle line-image, cross- plane (e.g , and ). ing points Aˆ and Bˆ. Radial mask m is formed by   a pstep(x) function of dot product between per- ~b  |(B~ − uGˆ) × (C~ − uGˆ)|  ˆ ˆ 1 1 spective map vector G and line-middle vector L,  ~b  = |(C~ − uGˆ) × (A~ − uGˆ)| ˆ  2    ~ minus dot product of L and one of the line points ~ ~ ˆ ~ ˆ |N| b3 |(A − uG) × (B − uG)| (here Aˆ). Radial mask m combined with a great AB l circle image forms line segment image . Barycentric vector ~b is a proportion of surface area. From vector ~b, various vertex properties Simple-particle procedural rasterization can be interpolated (e.g. depth, normal direction A B Following algorithm will produce approximate and UV coordinates), given each vertex , and C gradient image of a procedural spherical particle has associated value. for given position and radius. ~ ~ ~ ~ frag(b, Ap,Bp,Cp) = b1Ap + b2Bp + b3Cp scr(x) = x(2 − x) Interpolated triangle property p is a product of a ! (~b, A ,B ,C ) Gˆ · Pˆ − cos θ function frag p p p . Which is equivalent parl(G,ˆ P~ , r) = scr ∩ [0, 1] ~ 1 − cos θ to the dot-product of barycentric vector b and p value associated to each triangle vertex. p cos θ = 1 − sin2 θ s r2 ≡ 1 − 8 No-parallax point mapping P~ · P~ Real optical systems exhibit phenomenon known 15 Where P~ ∈ R3 is the particle position, r is repre- as the floating no-parallax point , where each senting particle radius and α is the angle between portion of the picture represent different projec- P~ and a ray from observation point O, tangent tion origin. To simulate such perspective, vec- tor length can encode z position offset, such that to the particle surface, so that ]P rO = 90°. G~ = gGˆ, g = |G~ | and Gˆ = G~ . In spherical lens, z offset can be described as 7 Fragment data interpola- a product of function P (θ). Offset value can be tion from barycentric coor- approximated by an optical measurement of the parallax alignment (see Figure 9 on page 15). dinates To measure origin offset, first static NPP pic- ture must be produced. If camera lens does not Rendering realistic polygon graphics involves produce such image, it can be derived from a se- shading and texture mapping. Values of normal, quence of images, each taken at different z posi- depth and UV coordinates associated to each ver- tion (see Subfigure 9b). Offset map value is then tex are interpolated across polygon surface using equivalent to the source image z position. barycentric coordinates of the fragment point. To reproduce picture with a floating NPP, each 3D point must be transformed prior to projection, accordingly to perspective map position and as- N~ = (A~ − B~ ) × (C~ − B~ ) sociated offset value. Normal vector N~ of the triangle plane ABC is derived from cross product of two triangle edges, 9 Conclusion and future work where N~ ∈ R3. Length of this vector is equal to the area of a parallelogram formed by those two I proved that visual sphere model is superior to edges. the visual pyramid of linear perspective. Later

8 I described design rules for perspective picture 6. Da Vinci, L. Annunciation oil and tempera which give symbolic meaning to geometric at- on panel. Painting. Uffizi Gallery, Florence, tributes of perspective projection. Also presented Italy, 1475-1480. http://en.wikipedia. mathematical equations for producing universal org / wiki / De _ pictura # /media / File : perspective vector , combined with the algo- Leonardo _ Da _ Vinci_ - _Annunciazione . rithm for rendering 3D polygon graphics directly jpeg. from three-dimensional space. This new solution 7. Da Vinci, L. A Picture is to be viewed from fits well into current graphical pipeline, replacing one Point only in A treatise on painting only low-level rasterization processes. trans. by Rigaud, J. F., 52 (Project Guten- Presented visual sphere model unites all type berg, Salt Lake City, USA, 2014). http:// of perspective projections under one technical so- gutenberg.org/ebooks/46915. lution, making perspective geometry a fluid con- struct, as perspective maps can easily be com- 8. Dixon, R. Perspective Drawings in Matho- bined and interpolated. graphics 82–83 (Dover Publications, Great Picture geometry can now be designed to Britain, 1987). smoothly adapt to the visual story, giving new di- 9. Ericson, C. Minimum bounding circle mension of control over mental perception of im- (sphere) for a triangle (tetrahedron) http: age. Presented concepts and equations may also / / realtimecollisiondetection . net / find its use in other fields, not imagery-related. blog/?p=20 (2020). This study did not fully explore the process of 10. Erkelens, C. J. The Perspective Struc- rendering floating NPP images. Further studies ture of Visual Space. i-Perception 6, will include research over calibration and simu- 204166951561367. http : / / journals . lation of real optical systems with floating NPP. sagepub . com / doi / pdf / 10 . 1177 / Also performance tests, comparison to current so- 2041669515613672 (2015). lutions should be performed in future research. Psychological evaluation of perspective geometry 11. Gascuel, J.-D., Holzschuch, N., Fournier, G. magnitude of influence on depicted space percep- & Péroche, B. Fast Non-linear Projections tion, performed on a large sample data, could be Using Graphics Hardware in ACM Sympo- an interesting field of study. sium on Interactive 3D Graphics and Games (ACM, Redwood City, California, 2008), 107–114. http : / / maverick . inria . fr / References Publications / 2008 / GHFP08 / i3d2007 . pdf. 1. Alberti, L. B. On Painting in. On paint- 12. Glaeser, G. & Gröller, E. Fast generation ing and on sculpture trans. by Grayson, of curved perspectives for ultra-wide-angle C., 55 (Phaidon, London, England, 1967). lenses in VR applications. The Visual Com- http://as.vanderbilt.edu/koepnick/ puter 15, 365–376 (1999). Modernism _ f05 / materials / images / alberti_window.htm. 13. Hueck, A. Von den Gränzen des Sehvermö- gens ger, 84. http://hans-strasburger. 2. Argan, G. C. & Robb, N. A. The Architec- userweb.mwn.de/materials/hueck_1840_ ture of Brunelleschi and the Origins of Per- ocr.pdf (Müller’s Archiv, Tartu, 1840). spective Theory in the Fifteenth Century. Journal of the Warburg and Courtauld In- 14. Konigsberg, I. The complete film dictio- stitutes 9, 96 (1946). nary 12. http : / / archive . org / details / completefilmdict00koni 3. Bagley, L. H. & Lavach, D. in Varga, (Nal M. Textbook of Rabbit Medicine E-Book Penguin INC, New York, USA, 1987). chap. 9.1 Ocular anatomy and physiology 15. Littlefield, R. Theory of the No- (Butterworth-Heinemann, 2013). Parallax Point in Panorama Pho- 4. Baldwin, J., Burleigh, A. & Pepperell, R. tography. http : / / janrik . net / Comparing Artistic and Geometrical Per- PanoPostings / NoParallaxPoint / spective Depictions of Space in the Visual TheoryOfTheNoParallaxPoint . pdf Field. i-Perception 5, 536–547. http : / / (2006). journals . sagepub . com / doi / pdf / 10 . 16. Miller, P. E. & Murphy, C. J. Equine Vision 1068/i0668 (2014). in Equine Ophthalmology (ed Gilger, B. C.) 5. Casamassina, M. King Kong’s Immersive chap. Visual perspective and field of view Style IGN. http://ign.com/articles/ (Elsevier Health Sciences, St. Louis, USA, 2005 / 07 / 18 / king - kongs - immersive - 2010). style (2019).

9 17. Pannini, G. P. Interior of St. Peter’s oil on canvas. Painting. National Gallery of Art, Washington DC, USA, 1754. http:// web.archive.org/web/20130327080328/ vedutismo.net/Pannini/. 18. Radley, A. Lookable User Interfaces and 3D in Cipolla-Ficarra, F. V. Handbook of Re- search on Interactive Information Quality in Expanding Social Network Communica- tions 46 (IGI Global, 2015). http://books. google.com/books?id=9OyfBwAAQBAJ. 19. Rybczyński, Z. a Treatise on the Visual Image (ed Oszmiańska, H.) chap. 1.2.2. http : / / books . google . com / books ? id = mUHNAQAACAAJ (Art Stations Founda- tion, Poznań, Poland, 2009). 20. Rydzak, J. Zbig Rybczyński, Oscar-Winning Filmmaker interview. Arizona, USA, 2014. http : / / youtube . com / watch ? v = 071 _ XgZ4H2I. 21. ShareGrid, I. Todd AO High-Speed Anamor- phic 35mm T1.4 at T2.0_log from The Ul- timate Anamorphic Lens Test. https : / / learn.sharegrid.com/sharegrid-lens- test-todd-ao-anamorphic (2020). 22. Thoby, M. About the various projections of the photographic objective lenses. Ortho- graphic fisheye projection http://michel. thoby.free.fr/Fisheye_history_short/ Projections/Various_lens_projection. html (2019). 23. Wade, N. J. & Hughes, P. Fooling the Eyes: Trompe L'Oeil and Reverse Perspec- tive. Perception 28, 1115–1119. http : / / wexler.free.fr/library/files/wade% 20(1999 ) %20fooling % 20the % 20eyes . %20trompe%20l’oeil%20and%20reverse% 20perspective.pdf (1999). 24. Wang, J., Shi, F., Zhang, J. & Liu, Y. A new calibration model of camera lens distortion. Pattern Recognition 41, 607–615. http:// dia . fi . upm . es / ~lbaumela / Vision13 / PapersCalibracion / wang - PR208 . pdf (2008). 25. Wikipedia, c. Robert Barker (painter). Bi- ography Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/w/index.php? title=Robert_Barker_(painter)&oldid= 907715733#Biography (2019).

10 Figure 3: Still from Todd–AO High-Speed Anamorphic lens (35mm T1.4) with visible curvilinear perspective. This type of lens was featured in films like Conan the Barbarian, Dune and Mad Max.21 © 2017 ShareGrid, Inc.

R R Orthographic Orthographic Equisolid Equisolid Equidistant Equidistant Stereographic Stereographic Gnomonic Gnomonic

 

(a) Linear-scale graph plotting ray of angle θ ∈ [0°, Ω], as (b) Linear-scale graph plotting ray of angle θ ∈ [0°, Ω], as the horizontal axis, and screen-position radius R ∈ [0, 1], the horizontal axis, and screen-position radius R ∈ [0, 1], as the vertical axis, where Ω = 40° (equivalent of R = 1). as the vertical axis, where Ω = 170° (equivalent of R = 1).

Figure 4: comparison of radial compression in five major azimuthal projections, across two different AOV values (represented by Ω); narrow (4a) and wide (4b).

11 I.II. III. II.I. III.

(a) Rectilinear (Gnomonic) projection. (b) .

II.I. III. II.I. III.

(c) Equidistant projection. (d) Equisolid projection.

II.

I. III. II.I. III. r 1 2 o

(e) Orthographic-azimuthal projection. (f) Examples model of equal-size discs (I, II, III) at equal distance r, facing observation point o, where Ωh = 170°, θ1 = 30° and θ2 = 60°.

Figure 5: Example of motion in perspective picture in various azimuthal projections, where Subfigure 5f presents viewed elements layout.

(a) Example of image geometry for pitch and yaw motion (b) Example of image geometry for roll motion (arrows), (arrows), where Ωh = 120°, k = 0, l = 10% and s = 95%. where Ωh = 120°, k = 0, l = 100% and s = 95%.

Figure 6: Examples of image geometry for given type of view motion.

12 S

A→

G

A^ O b

^ B^ C

h B→

→ C

(a) Projection of polygon triangle ABC onto an image grid G imposed over the visual sphere S, where projected triangle line b belongs to a great circle h.

GY

S Y^ X^ G

GZ GX Z^ O

A^ h C^

⊥ (b) Rotation of image grid Gˆx component, to unit vector Xˆ, where Xˆ = h and X⊥Y ⊥Z, so that {AC, Y,ˆ Zˆ} ∈ h. ˆ ~ ~ ˆ0 ˆ ˆ Required components can be calculated from cross and dot product; X = A × C and Gx = X · G.

Figure 7: Projection of triangle ABC onto visual unit sphere S, where projection origin O is at the sphere center. Edge of the projected triangle is always produced by an arc of a great circle, here AC ∈ h. Image grid G represents visual sphere vector map, where each pixel color is a spherical unit vector.

13 → (a) Rectilinear perspective vector map, where Ωd = 140°, (b) Rasterized triangle in rectilinear perspective using vec- k = 1, l = 1. tor map, where Ωd = 140°, k = 1, l = 1.

→ (c) Equirectangular projection vector map of whole sphere, (d) Rasterized triangle in equirectangular projection, where Ωh = 360° and Ωv = 180°. where Ωh = 360° and Ωv = 180°.

→ (e) Mustache style lens distortion vector map, where Ωd = (f) Mustache style lens distortion triangle rasterization, 131°, k = 0.32, l = 62%, s = 86%, k1 = −0.6, k2 = 0.4. where Ωd = 131°, k = 0.32, l = 62%, s = 86%, k1 = −0.6, k2 = 0.4.

→ (g) Curvilinear perspective vector map, where Ωd = 270°, (h) Rasterized triangle in curvilinear perspective, where k = 0.32, l = 62%, s = 86%. Ωd = 270°, k = 0.32, l = 62%, s = 86%.

→ (i) Perspective vector map of five-screen horizontal array (j) Rasterized triangle of five-screen horizontal array, in rectilinear projection, where single screen Ωh,v = 60°, where single screen Ωh,v = 60°, k = 1 and l = 1, giving a k = 1 and l = 1, giving a total 5Ωh = 300°. total 5Ωh = 300°.

Figure 8: Examples of polygon triangle rasterization directly from three-dimensional space to the image, 3 3 using visual sphere vector map G, where Gn ∈ [0, 1] ← [−1, 1] . [

14 1. N 2.

m 3.

C z

S z

(a) Model of no-parallax point (NPP) calibration rig. (b) Example of a variable camera position z encoded in an Which measures misalignment of markers (m) between image sequence (1, 2, 3). Element 1 presents alignment of northern (N) and southern (S) hemisphere as seen through the peripheral markers, while element 3 presents alignment camera (C). of the side markers, element number 2 presents position in- between.

m m m m m m m 1 m m m 5 1 2 3 4 5 2 3 4

I. II.

m m m m 1 m m m 5 1 m m m 5 2 3 4 2 3 4

III. IV. (c) of the NPP alignment using calibration rig (9a). Example I presents misalignment of the camera in all three axes. Examples II, III and IV present alignment in X, Y axes. In example II peripheral marker pairs m1, m5 present alignment near horizontal AOV, while pairs m2, m4 are misaligned due to floating NPP. In example III markers m2 and m4 are aligned, while peripheral markers m1, m5 are not. Position z in example III is less than II. Example IV presents “slit-scan” composite of variable z position, where all markers are aligned.

Figure 9: Floating no-parallax point calibration process.

© 2020 Jakub Maksymilian Fober (the Author). The Author provides this document (the Work) under the Creative Commons CC BY-ND 3.0 license available online. To view a copy of this license, visit http://creativecommons.org/licenses/by-nd/3.0/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA. The Author further grants permission for reuse of images and text from the first page of the Work, provided that the reuse is for the purpose of promoting and/or summarizing the Work in scholarly venues and that any reuse is accompanied by a scientific citation to the Work.

15