Efficient Computation of Multidimensional Data Structures on CPU Platform

Home , Iterated function system, Pi (art project)

Eotv¨ os¨ Lor and´ University Faculty of Informatics Department of Programming Languages and Compilers

Eﬃcient computation of multidimensional data structures on CPU platform

Supervisors: Departments of Computer Algebra and Prog. Lang. & Compilers Author:

Attila Kovács Zalán Végvári Associate professor Software Information Technology BSc.

Melinda T´oth Assistant lecturer ELTE-IK, PNYF

Budapest, 2018. T´emabejelent˝o

1 Contents

1 Introduction 1 1.1 Fractals ...... 1 1.1.1 Whatisafractal?...... 1 1.1.2 Calculatingfractals...... 2 1.1.3 TheEscape-TimeAlgorithm...... 3 1.2 Generalization...... 3 1.2.1 ExtendingTheEscape-TimeAlgorithm...... 3 1.2.2 Extendingthespace ...... 4 1.3 Auniversaltool...... 5 1.4 Scripting...... 5 1.5 Userinterface ...... 6

2 User documentation 8 2.1 Thesolution...... 8 2.2 NViewer...... 8 2.3 Graphicalengine ...... 9 2.3.1 Thetransformationmatrix...... 9 2.3.2 Thefunctions ...... 10 2.3.3 Thelayers...... 11 2.4 Goals...... 13 2.5 Thelanguage ...... 13 2.5.1 Modules ...... 14 2.5.2 Modes ...... 14 2.5.3 Types ...... 15 2.5.4 Parametersandstyles ...... 16

1 2.5.5 Queries ...... 16 2.5.6 Definitions...... 17 2.5.7 Nesteddefinitions...... 17 2.5.8 Patterns ...... 18 2.5.9 Branching ...... 19 2.6 Standardlibrary ...... 19 2.7 Toolusage...... 23 2.7.1 Configuration ...... 24 2.7.2 Sessions ...... 24 2.7.3 Scripts...... 25 2.7.4 Camera ...... 25 2.7.5 Stylesandparameters ...... 26 2.7.6 Scenes ...... 26 2.7.7 Exportingdata ...... 27

3 Technical documentation 28 3.1 Problemdescription...... 28 3.2 Enginestructure ...... 29 3.3 Scriptstructure ...... 30 3.3.1 Typesystem...... 30 3.3.2 Functionstructure ...... 31 3.3.3 Informationﬂow ...... 31 3.4 Importingtheengine ...... 32 3.5 Importingthescriptparser...... 32 3.6 Extendingthescriptlanguage ...... 33 3.7 Optimizations...... 33 3.7.1 Functionstructure ...... 34 3.7.2 Constants ...... 34 3.7.3 Memorymanagement...... 34 3.8 Comparism ...... 35 3.8.1 C++...... 35 3.8.2 Haskell...... 35 3.9 Examples ...... 36

2 4 Summary 41 4.1 English...... 41 4.2 Magyar ...... 42

5 Results & testing 43 5.1 Testingplan...... 43 5.2 Scripts...... 44

Bibliography 48

6 Appendix 49

3 Abstract

NViewer is a tool for visualizing N-dimensional shapes, more speciﬁcally fractals. This is not a tool to speed up something that was already known, it aims to uncover novel options for us. This tool allows us to be able to look into higher dimensions with an easy to handle camera technique. Further it allows us to discover structures unimaginable for us. NViewer provides us an easy way to transfer mathematical objects from paper to screen, in form of a scripting language. Additionally we are also providing a new type of graphical engine to render the results. Chapter 1

Introduction

First, I intend to explain why this project was created and the problems it solves. The main objective is to create a tool that is easy to use not only for speciﬁc problems, but in more general cases.

1.1 Fractals

Before we proceed any further, I need to clarify a number of concepts.

1.1.1 What is a fractal?

A fractal is a geometrical or physical structure having an irregular or fragmented shape at all scales of measurement between a greatest and smallest scale such that certain mathematical or physical properties of the structure, as the perimeter of a curve or the ﬂow rate in a porous medium, behave as if the dimensions of the structure (fractal dimensions) are greater than the spatial dimensions. [1]

When somebody asks me what fractals are and I am to answer in a simple way, I would answer that these object represent a form of art created by mathematics and logic.

1 Figure 1.1: The mandelbrot set

1.1.2 Calculating fractals

While fractals can be created in many diﬀerent ways, they appear even in Nature. Plant development often shows fractal-like structures. Typical examples include plants such as broccoli, pinecone and basically most leaves. [4]

In this project I will discuss only those, generated using computers. Some basic methods can be implemented in comenius logo in a recursive graphical representation pattern. A more complex way is to choose a set of points and run them across an iterated function system, also referred to as IFS [6]. In this case the set of points will be transformed in each step Figure 1.2: Broccoli which, in some cases, will approximate a set, which can be, to some extent, a fractal. As shown below, the nviewer project is capable of visualizing these types of fractals in an ineﬃcient way, however my primary objective was to work with The Escape-Time Algorithm and some other similar algorithms.

2 1.1.3 The Escape-Time Algorithm

The Escape-Time Algorithm is a widely used algorithm to visualize fractals. The basic idea is to use an iteration function and test all points in the observed space to test if the iteration is convergent or not. If so, the point belongs to the fractal. Although this task is not entirely possible using a computer, approximations can be conducted. The most fa- mous application of this technique is the Mandelbrot Set [2]. Figure 1.3: Mandel- The convergence is tested after each iteration, as follows: if brot using low itera- the length of the current vector is greater than a speciﬁc tion settings value, it is not convergent and the iteration is interrupted. If this does not happen in a speciﬁc number of iterations, the original point (where the iteration stared), is part of the fractal at the current approximately level. For the points that are not part of the fractal, the time when they ”escape” is used for colouring, which both shows how the iterations approximates the fractal and provides an aestethical image.

1.2 Generalization

Ok. Now a tool is available which is able to render 3D fractals, which is itself already useful, further, t many other objects can be created using the same approach.

1.2.1 Extending The Escape-Time Algorithm

For proper functioning, the Escape-Time Algorithm requires the availability of the following features: per-pixel data management, graphical engine to render the results, mathematical core to perform the necessary calculations at an acceptable speed. Once these features are available, it becomes easy to extend some of the features. The Escape-Time Algorithm provides escape-time to the points of the space, so that these can be coloured, where the colouring is carried out as a result

3 of a built-in process. Therefore, the graphical engine will not use this algorithm, but it will use more general functions that assign colours to points. This way, the colouring methods for the Escape-Time Algorithm can ban modified manually or it allows the implementation of various algorithms. At this point, we are able to create a function that colours each point in the space for rendering. It allows us to render any shape or object that can be expressed in such a function. Though this is not enough, we cannot yet apply light or any other effect to the surface. This sounds unimportant for the first time, because it seems to be only a special effect. When working in 3D, lighting is one of the most important tasks, because it allows us to visualize the surface of the object. Figure 1.4 shows a part of the mandelbox with and without lighting. If no lighting is present, fog effect had to be used with a special colouring to make a surface patterns visible. Without these effects, only the outline of the mandelbox is visible. The second image show what it looks like, when lights are applied. There is a slight green fog that makes the distances more visible. So far this can be done using the graphical engine, but it might not satisfy all our needs, because lighting is applied in the same way to all points of the surface. This issue is continued (section 2.3.3).

(a) Without lighting (b) With lighting

Figure 1.4: Mandelbox with and without lighting

1.2.2 Extending the space

A lot of 2D fractals have been recognized. They are easy to generate, easy to examine and at the same time they are beautiful. Additionally, three and more dimensional

4 fractals are known. Of course, besides fractals, a 4D model of the black holes has also been described [3]. This creates a need for a graphical engine that is capable of rendering N-dimensional objects. Usually, tools that operate in N-dimension, simply conduct a 2D or 3D projection of the original object. Is that enough? Projections cover some details, because only the surface will be visible.

1.3 A universal tool

Most tools are designed to serve one purpose only. That is not wrong at all, because specialized tools are usually more optimized, but they have a great disadvantage: they have to be re-created for every problem. A more general tool could be used for the same purposes without the need to re-create it every time. It would be less optimized, but having a ready-to-use tool saves time, work and energy. In conclusion, we need a tool, that is (1) capable of importing functions for space colouring, shader functions, (2) capable of rendering the results in N-dimension and easy to use. This tool is expected to possess an easy-to-use user interface, where the objects can be surveyed; a graphical engine to support all necessary operations in the N-dimensional space and a system to import functions that will visualize the object.

1.4 Scripting

When I was designing this software, I considered a couple of options here. Extending a C++ class would be the most simple method for me to implement. In this case the user would have to create a child class of an existing class, that I created in this project and either build that in the code or add as a library. This is unfortunately time consuming for the users and requires programming knowledge, but this software is not designed only for programmers. Use other simple languages as scripting language is also a considerable method. With the right choice of the scripting language, it could provide the users a nice interface for entering functions. The problem here is that it would be extremely slow. An other language would be used which is designed for a much more general

5 purpose which makes it slower for this purpose and it has to be able to communicate with the graphical engine. For this reason I created a new scripting language , later referred to as NViewerScript. More on this in the next section.

1.5 User interface

There is a need for two versions: a desktop version and the server version. To begin with the simple one, the server version is required to operate on server machines only for the purpose of exporting high quality images. It needs no graphical interface and it only needs to be able to open ﬁles exported from the desktop version and process those to create the desired images. The desktop version, on the other hand, needs to be an easy-to-use tool with a graphical interface, that supports many features including, most importantly displaying fractals, exporting data to the server version and manage the camera conﬁgurations which produced a nice image. Figure 1.5 show the user interface: The rendered image will be displayed in the Display widget. The messages, errors, warnings will be logged and shown in the Log widget. The memory usage of the script parser and the script itself is displayed in the Mem- ory usage widget. The Camera settings widget is a simple interface for moving the camera. There are more advanced options as well, that can be accessed through the menu.

6 Figure 1.5: User interface: 1 - Display; 2 - Log; 3 - Memory usage; 4 - Camera settings

7 Chapter 2

User documentation

This chapter will cover the basics of the NViewerScript, the user interface and the ideas behind the solution.

2.1 The solution

In this chapter I am going to write about the solutions to the problems described in 1.

2.2 NViewer

NViewer is a universal tool for displaying N-dimensional objects, that are represented by a colouring function and optionally a shader function. It allows the user to create these functions in a functional scripting language (discussed in chapter 2.3.3) and import them to an easy-to-use interface, where they are also displayed. The camera, filming the object, can be translated, rotated, scaled in many different ways to achieve the desired results. The object will be displayed in real time, if the quality settings and a computer configuration allow it. The images can be exported later in higher quality, allowed by a different server. Although, it is similar to other 3D fractal display softwares, two key differences are known: the scripting language and the camera.

8 2.3 Graphical engine

The camera is usually a point in the 3D space with a rotation, field of view, near and far clipping. In this case, that camera is represented by an (N+1)-dimensional transformation matrix, the field of view, near and far clipping. For this special type of camera, imagine a normal camera in 3D, originated at (0 , 0, 0), facing (0 , 0, 1), the field of view, near and far clipping are parameters, set by the user. Between the near and far clipping, there are a some planes perpendic- ular to the camera’s direction. The number of these planes is called the Depth . On each plane, there is a grid of points with Width columns and Height rows. We are going to work with the points of these grids. As mentioned earlier, the camera has a transformation matrix. This matrix will be used to transform each of the selected points into the N-dimensional space. The colouring functions can now determine the colour of these points and the shader functions can shade the surface points. The graphical engine renders an image of the points that are visible from the perspective of the camera (some points are transparent, and some are covered).

2.3.1 The transformation matrix

The transformation matrix is ( N + 1) × (N +1) dimensional, where the last column stores the position of the camera in the N-dimensional space with a 1 in the end, the last row consists of zeros. The remaining part is for rotation and scaling. At the beginning, this matrix is an identity matrix. When translating the camera the last column is changed, when rotating or scaling the N × N submatrix is modiﬁed accordingly. There is also an extra option to specify a matrix manually.

 x11 x12 x13 ... x 1N p1   x21 x22 x23 ... x 2N p2     ......   ......      xN1 xN2 xN3 ... x NN pN       0 0 0 . . . 0 1 

9 Translation: The camera can be translated by adding the translation vector, v, to the last coloumn.  x11 x12 x13 ... x 1N p1 + v1   x21 x22 x23 ... x 2N p2 + v2     ......   ......      xN1 xN2 xN3 ... x NN pN + vN       0 0 0 . . . 0 1 

Rotation: The camera can be rotated around two diﬀerent axises ( a, b) with an angle ( α) by creating a rotation matrix and multiplying it with the N × N submatrix of the transformation matrix. The order of the multiplication depends on the rotation method: local or global. The rotation matrix: ∀i, j ∈ [1 ..N ] :

sin (α) if ( i = a ∧ j = a) ∨ (i = b ∧ j = b)  cos (α) if i = b ∧ j = a    Rij = −cos (α) if i = a ∧ j = b (2.1)

1 if i =6 a ∧ i =6 b ∧ j =6 a ∧ j =6 b   0 otherwise   2.3.2 The functions

Colouring function: As indicated by its name, it is used to specify the colour of the points. The colour is an rgba code, where a is the transparency. 0 transparency means that the points is not visible at all. If a = 1 tha point is called solid, it covers everything behind it. The points where 0 < a < 1 will create a layer of fog as the pictures indicate:

Shader function: This function retrieves the position, normal vector of the surface and the previously determined colour of the points that form the visible part of the surface. The surface contains only solid points, transparent points are ignored. This function is to apply the lights or any other eﬀects that only aﬀect the

10 (a) Transparent (b) Solid

Figure 2.1: Solid vs. transparent surface. A default shader function built in the engine is also available. If there is no shader manually added, the default will be used. There are two built-in parameters that specify the behaviour of the default shader: directional light (expressed in its direction) and ambient light. The purpose of the main shader is to highlight the surface of the objects.

2.3.3 The layers

Before transforming the points from the 3D space, they are positioned on planes. The set of points inside one plane is called a layer. The basic idea of rendering is to calculate the colour of all the points, fold the layers starting with the background colour as initial value then the most distant layer (from the camera) ﬁnishing with the nearest layer. The sum of the layers will be image that is displayed.

c1.rgb + c2.rgb := lerp (c1.rgb, c 2.rgb, c 2 ∗ d/ ((1 − c2.a ) + c2.a ∗ d)) (2.2)

c1.a + c2.a := 1 (2.3) where c1 is the colour of the distant point, c2 is the colour of the near point and d is the distance between them. Unfortunately, a problem arises here: this would not be very fast this way, because solid points cover everything behind them, and the hidden points are still calculated. The best option would be to start with the nearest layer, and once reached a solid point, skip every point behind it. Then start the layer addition from the last layer, where there was a visible point and only work with the visible points henceforth.

11 The problem is that this requires a lot of memory. In case of 1024 × 1024 image with 4096 depth, it would require 16 Gib of RAM. For this reason, not all layers are calculated early. The user can decide how many layers are acceptable and these layers will be equally distributed in the visible space. This is still a powerful tool for optimizing the calculation, since neighbour layers are not likely to cover much diﬀerent points. Lighting: The most important thing to calculate is the normal vector of the surface, which is also not an easy task. When lighting is enabled, extra layers are calculate, new layers added before the near clipping and beyond the far clipping and all the layers are widened. This way each point, which was originally in one of the layers, will have surrounding points in every direction. The number of the extra layers and points can be changed in the system settings, which aﬀects light quality and memory usage.

Figure 2.2: The surface

Figure 2.2 shows how the surface normal is calculated for the red point. The black (and the red) points represent the solid points, the white points represent the transparent points. The blue arrow indicates the direction of the camera. The points on the top of the image are close to the camera (always). The points on the left have a lower X coordinate in the original plane (only this section, there is an other section for the Y axis). First we have to calculate steepness of both the X-axis

12 and the Y-axis section. For this purpose a really simple algorithm is used to speed up the process: proceed in both directions and ﬁnd the two nearest edges. The line crossing the two edges will be the surface.

2.4 Goals

The primary goals of this language are expected to be simple to use and eﬃcient for the targeted tasks. My choice was a functional language, because it is easy to learn and use and most fractals/shapes, that are considered in this project, can be expressed as functions, which are easy to implement as NViewerScript scripts. The language is designed for mathematicians primarily who are not necessary skilled in programming languages, but a functional language does not need programming skills. The speed is also a very important property. The colouring function will be evaluated numerous times, for a HD image, it can be more than 2 ×10 9. This requires a special tool, since most functional language are not designed for this purpose.

2.5 The language

The script is consists of the following parts:

1. Module name (optional for the main script): module package.ﬁlename;

2. Intervall of supported dimension number (subintervall of [3..]): dimensions [a..b];

3. Styles and parameters: style Real s = 1.0;

4. Definitions: let main (v) = #00ff00ff ;

The order of these parts does not matter and deﬁnitions can be nested to make helper functions. The signature assigned to the deﬁnitions must be unique, which

13 consists of the deﬁnition’s name and its parameters. Each part is closed with a ( ;);

2.5.1 Modules

Modules are designed to allow code reuse. A script can be written into multiple files. One file will be the main script, that is required by the graphical engine, the colouring functions is defined in this file. The other files will be the modules or packages. They need to move to standard locations, known by the program. The main script can import them using their module name, which consists of the path from the root of a standard location to the file, the filename without the extension, separated by ( .). When importing, the content of the module will be copied to the beginning of the script, unless it is already imported.

2.5.2 Modes

A further feature is available, called preview mode that allows to have two versions of the same script: preview and export. The aim here is to provide a version that is fast to generate, which is good for the discovery part, and on other version which will produce high quality images. Figure ?? shows an example of the preview and export mode. The preview mode is indicated with #preview and closed by #end or #export . The code between will be ignored when exporting high quality images. The export mode begins with #export and is closed by #end or #preview , it is ignored in the preview images.

1 module mode ; 2 3 dimensions [3..3]; 4 5 #preview 6 //Fast code for prview 7 let main (v) = #ff0000ff; 8 # export 9 //High quality for exporting images 10 let main (v) = #00ff00ff; 11 # end

Figure 2.3: Example for the preview and export modes

14 2.5.3 Types

The language uses static typing, which means that the type of every expression is evaluated during the compilation process. Bool: This is the logical type. The possible values are: True, False. Int: This type represents the integers. It is based on the C++ integer implementation. Real: This type represents the real numbers. The implementation can be configured manually when installing the program. It can be based on one of the C++ floating point types: float, double, long double. Complex: This type represents the complex numbers. The implementation is based on the std::complex type in C++. Colour: Colours are represented as 4 real numbers - RGBA - each in the range of 0 , 1. The alpha is the transparency, where 1 is the solid and 0 is the invisible. Vector: Vector is an array with N elements, where N is the dimension number and each element is a Real . Matrix: Matrices are represented by an array of N Vectors , where N is the dimension count. Tuple: Tuple is a recursive type. It can be constructed from many other types. The typename comes from the subtypes, separated by ( ,) closed in brackets. For example: (Real, Colour, (Vector, Matrix)) Tuples are implemented as arrays of pointers in C++. List: There is no list in general as in Python or other dynamic typed languages, list are always based on a type. Their typename is the basic type name closed in square brackets. For example: [Real] or [(Real, Vector)] Lists are implemented as linked lists in C++.

15 2.5.4 Parameters and styles

Parameters and styles are constant values that can be defined from the editor. The difference between a style and a parameter is the way they are stored. Parameters are stored session-wide and styles can be saved into style configuration files, thus enabling multiple style settings for one session. There is no difference in the way they behave in the script. They can be defined with the parameter and the style keywords, followed by the type, the name and the initial value. The type here is required, there is no auto type detection, because they can be defined in the editor and not only depend on the script itself.

1 module example.param; 2 3 dimensions [3..3]; 4 5 parameter Real size = 1.0; 6 style Colour col = #ff00ffff; 7 8 let main (v) = 9 | if ( abs (v[0]) < size) col 10 | #00000000;

Figure 2.4: Example for parameters and styles

2.5.5 Queries

Queries are similar to const functions, but their value is stored in the script, rather it is determined by the camera and session. Queries are handled in a diﬀerent way to optimize the calculations. Expressions that rely only on constant values and query values can not be evaluated while compiling, but they can be evaluated before rendering one image, and later used as constant values.

• Session.dim - Dimension number

• Session.light - Direction of light

• Session.ambient - Level of ambient light

• Camera.pos - Position of camera

16 • Camera.dir - Direction of camera

• Camera.transform - Transformation matrix of the camera

2.5.6 Deﬁnitions

Deﬁnitions are basically functions. They always begin with a let keyword, followed by their type, name, parameters in brackets and their values. In case of constant functions, the parameters are missing. The type of a function is formed by the input type, followed by an arrow and the value type. In case of a constant function, only the value type is present. The type of the function will be auto completed, if not given or not completely given. The typename that begins with a capital letters is a ﬁx type (either a built-in type, or an error), type names beginning with lowercase letter are incomplete types. If there is not type given, the script parser will generate an incomplete type from the parameter list. Functions with incomplete types are just templates, that will be in- stantiated when they are used and the type of the instance will be completed based on the parameters given as input.

1 module example.def; 2 3 dimensions [3..3] 4 5 let a->a f (x) = x; 6 7 let f2 (y) = 3; 8 9 let parent (x) = 10 let helper (y) = x+y; 11 helper (2); 12 13 let Vector -> Colour main (v) = #00ff00ff;

Figure 2.5: Example for deﬁnitions

2.5.7 Nested deﬁnitions

Definitions can be nested for two reasons: optimisation, simple structure. When one expression is used multiple times in a definition, it can be defined as a function on its own within the original definition. During the evaluation process, these nested

17 definitions, in or helper functions, will be evaluated first (only if they are used), and later their value will be used as a constant, this speeds up the process. The helper function is positioned after the ( =) and follows the normal definition syntax. The helper functions will inherit the non-inherited parameters of the parent function. An example can be seen in Figure 2.5.

2.5.8 Patterns

Patterns work in a unique way in this language, they are based on tuples and the other types are handled in general. The parameters of the functions are entered as a patterns. The syntax is the following:

• S → P; S,S; (S)

• P → a valid name for a parameter

A pattern matches a value if structure of the pattern is a substructure of the value’s structure. In other words, the tree generated from the pattern (where the nodes are the tuples and theirs leaves are its content) is a subgraph of the tree generated from the value’s type. Each parameter will refer to its corresponding subtree in the value’s structure. For example:

1 module example.pattern; 2 3 dimensions [3..3] 4 5 let f (x(y,z)) = z; 6 7 let g = f((1,2));

Figure 2.6: Example for patterns

In Figure 2.6:

• x = (1 , 2)

• y = 1

• z = 2

18 2.5.9 Branching

Functions can be divided into branches by: logical statements; certain values; in- tervals; head-and-tail pattern for the list type. Each branch begins with a ( |). The branch with no condition is the default branch (there can be no more than one). If it does not exists, the parser generates a default branch which throws an exception when used to avoid undeﬁned behaviour. The branches are tested in the order, they are entered and the ﬁrst match will give the value of the function.

Figure 2.7: Example for branches

2.6 Standard library

There are two types of standard functions: those that comes with the language as a feature, and the functions found in libraries. Standard libraries is a feature, planned to be implemented later in the future. This would mean having standard script files that define functions and constants. There would be a file for each field of science that is considered useful for the nviewer project. There are a lot of functions that come with the language. These functions are written in C++ and are optimised well. Using these functions will likely improve the results compared to creating these functions in NViewerScript. Operators, such as + , −, ∗, /, ˆ, =, <, >, < =, > =, ! = , !, or, and, xor are defined in this section. The other functions are: Constructors:

• Real(int)

• Real.pi

19 • Real.e

• Complex(Int)

• Complex(Real)

• Complex(Real,Int)

• Complex(Int,Real)

• Complex(Real,Real)

• Colour(Real,Real,Real,Real)

• Colour.red

• Colour.green

• Colour.blue

• Colour.white

• Colour.black

• Colour.none

• Vector.zero

• Vector.x

• Vector.y

• Vector.z

• Matrix.zero

• Matrix.identity

Int:

• xi(Bool)

• inc(Int)

• pow2(Int)

• round(Real)

20 • ﬂoor(Real)

• ceiling(Real)

• truncate(Real)

• factorial(Int)

• div(Int,Int)

• mod(Int,Int)

• abs(Int)

• exp(Int)

Real:

• imag(Complex)

• real(Complex)

• magnitude(Vector)

• sqrmagnitude(Vector)

• sin(Real)

• cos(Real)

• tan(Real)

• asin(Real)

• acos(Real)

• atan(Real)

• pow2(Real)

• sqrt(Real)

• log2(Real)

• log10(Real)

• log(Real,Real)

21 • abs(Real)

• abs(Complex)

• degree(Real)

• radian(Real)

• dot(Vector,Vector)

• arg(Complex)

• norm(Complex)

• exp(Real)

• sinh(Real)

• cosh(Real)

• tanh(Real)

• asinh(Real)

• acosh(Real)

• atanh(Real)

• clamp01(Real)

• clamp(Real,Real,Real)

• root(Real,Int,Int)

• lerp(Real,Real,Real)

Complex:

• pow2(Complex)

• conj(Complex)

• proj(Complex)

• polar(Real,Real)

• exp(Complex)

22 • log(Complex)

• log10(Complex)

• sqrt(Complex)

• sin(Complex)

• cos(Complex)

• tan(Complex)

• asin(Complex)

• acos(Complex)

• atan(Complex)

• sinh(Complex)

• cosh(Complex)

• tanh(Complex)

• asinh(Complex)

• acosh(Complex)

• atanh(Complex)

• lerp(Complex,Complex,Real)

Colour:

• lerp(Colour,Colour,Real)

Vector:

• lerp(Vector,Vector,Real)

2.7 Tool usage

This chapter will cover the proper usage of this tool for a user. You will read about the structure of the user interface and the options you have.

23 2.7.1 Conﬁguration

The software has a global configuration, which is mostly responsible the hardware depen- dant options. The data is stored in an XML file, globally used by each session. This file stores the preview quality settings, the memory limits, the allowed number of threads used for the calculations, keyboard options, limits for the scripts and the recently used files to open them on startup. This file is loaded at first and is required for any other feature to work. This file is automatically managed by the software, including creating and autosaving. The user can modify these settings either through the user interface or from the configuration file directly. The possible options and their description is showed in Figure 2.8.

(a) System (b) Input

Figure 2.8: Conﬁguration

2.7.2 Sessions

The session play the role of the projects. A session can have one main script file, any number of scene and style collections (discussed in section 2.7.5), but only one can be open at a time. Sessions can be managed through the menu, where they can be created and opened. When creating a new session, the dimension number has to be specified and that will be permanent in that session. Each session has its own configuration file, but it is handled automatically. It stores what files belong to the session and, of course, the dimension number.

24 2.7.3 Scripts

Scripts are text files with .nvw extension that can be opened from the nviewer. The main script - which is selected by the user - is required to have a main (v) :: Vector → Colour function, which is the colouring function and it can have a shader ((v, n, c)) :: (Vector → Vector → Colour) → Colour function which is, as its name indicates, the shader function. There is a default shader function and for this reason it is not required. The main script can import other script files, which are called packages. The packages do not have a main function (it is not a rule, but if they do, the main script will be useless). The packages are located at standard script locations that the nviewer can find and are referenced by their module names.

2.7.4 Camera

The camera can be moved in three diﬀerent ways:

• Using the panels for simple movements

• Keyboard to do a bit more advanced movements

• Though a menu for full control

The panel shows the camera’s position and direction. The focus determines the speed and the camera can be ﬁxed to a point, that focus point. The focus is the distance to the focus point, which is located in front of the camera. Scale is the distance between the camera and the far clipping plane. Moving is done along the camera’s direction though the panel. If you need advanced or accurate camera settings, you can enter the

• Position

• Rotation

• Direction

• Scale

• Transformation matrix of the camera. The keyboard is the most important for discovering purposes. There are two modes, fixed focus point motions and normal motions. The difference is that, the fixed focus point

25 motions keep the focus point in its place and move around it, while the normal mode uses linear movements. The camera can be:

• Move forward/backward

• Move along an axis in the local space

• Rotate around the focus point in the direction of an axis in the local space

• Rotate regardless of the focus point around two selected axises

(a) System (b) Input

Figure 2.9: Camera controls

2.7.5 Styles and parameters

Styles and parameters are both configuration of the script through options defined in the script itself (and some extra). Both can be entered as valid NViewerScript expressions which are then evaluated and if their type matches the expected type, used as constants. The difference is that the parameters are saved in the session configuration file, which means that they are globally set and there is only one version of them. Parameters are designed for properties that change the shape of the fractal, like the iteration limit. This is not a rule, even the background could be a parameter. Styles are stored in style collection files. These files store multiple stylesheets. A stylesheet is a configuration of the parameters marked as styles (styles are special parameters). Stylesheets can be loaded at any time from the currently open collection file. Styles are usually used for properties that do no affect the shape, only the appearance, mostly colours. The parameter and style settings can be accessed from the menu.

2.7.6 Scenes

Scenes are camera configurations which includes the transformation matrix, the focus and the scale of the camera. Scenes can be captured in the editor from the menu and later loaded by their name. Scenes are stored in scene collection files. These files are vulnerable

26 to changing the dimension number. This is cannot be done through the editor, but it is possible through the session configuration file. Scene collection files are not only required to store the manually captured scenes, but the last visited scene is also stored here (used at startup), which means that the software only remember the last scene, if there is a scene collection file open.

2.7.7 Exporting data

After creating a scene- and a style collection, it is possible to make a list of scene-style pairs. These pairs can be exported as an export data file, that can later be opened on the server side. The export data file is a zip file, containing the:

• scene collection ﬁle

• style collection ﬁle

• scene-style pair list

• scripts in use (main script and the imported modules)

• session conﬁguration

The server version will extract this ﬁle and export the selected scene with the corresponding styles to images with the selected quality. The images are not rendered by the editor, because it usually required a powerful machine.

27 Chapter 3

Technical documentation

3.1 Problem description

My task was to create an interpreter for the language described in the previous chapters and a rendering engine that can render the desired images. The script interpreter targets the highest possible level of optimization. The interpreter does not compile the code, rather it creates a memory structure which enables robust execution of the script. As it is described later in more detail, the this memory structure is represented as a tree graph, whose nodes are the functions in the script, and these nodes are also represented as graphs. This graph is traversed to evaluate the function described in a script ﬁle. The rendering engine takes a function as an input, which can be represented as a script instance (CustomFunctionsClass), but does not necessarily use an NViewerScript. My goal was to create an algorithm that minimizes the number of function evaluation necessary to render the image. The simple version would be to render any transparent points that are not occluded my solid points, and the solid points only on the surface of the object being rendered. The problem is that the surface normal cannot be estimated in some cases. The rendering engine is not required to produce a photo realistic image, local illumination is suﬃcient.

28 Figure 3.1: Use case diagram

3.2 Engine structure

The engine consists of three main parts:

• Function handler

• Graphical engine

• User interface

The core of the system is the function handler . It does not depend on any other parts of the project, thus it can be used outside. It aims at function evaluation in general. On the top level of this module there is an abstract class which is responsible for function representations. It can be specialized by creating child classes. The module allows us to create functions at this level in C++, though it is not accessible from outside. The most important specialization is the script parser, where the behaviour is determined by an NViewer script. The structure of this class will be discussed in details in section 3.3 The graphical engine relies on the function handler. It is constructed using a function, among some other parameters, which will be rendered to a texture. The main part of this module is the camera class. It is a templated class, which requires a ﬂoating point type. When building the project, it can be set to either ﬂoat, double or long double. The camera supports multi-threading: the screen is divided into columns, each thread of which is responsible for working with one of them. The threads are synchronised after each layer of the image. The graphical engine can be used without the current user interface. The

29 user interface is responsible for the communication between the user and the graphical engine. This is on top of the top layer of the project and it is written using Qt to support window handling and graphics.

Figure 3.2: Basic class overview

3.3 Script structure

This section will discuss how the script language works in details.

3.3.1 Type system

All values are passed between functions as (void*) pointers. Each type has its own constructor class, which are based on an abstract class, called CustomType. Instances of this type can be created from a string that contains a valid type. These constructors are responsible for allocating memory for the value and creating its structure by linking its parts together, if necessary. Some types are recursive, such as the list and tuple, which allocate memory using the base constructor(s) then link them together to form an array or a linked list. All these types have memory allocation method and a copy method.

30 3.3.2 Function structure

Functions (in this subsection we are discussing C++ functions, not mathematical functions) are created recursively: First a generator function is created that will build and maintain a valid structure while compiling the script and which is deleted afterwards. This function receives a string, an expression (possibly invalid). In the following steps, it tries to find a pattern or value or any known expression. If there is none, it will throw an exception and give an error string describing the problem. If a value or expression is found, the a function is created that will return the expected result (const function, identity function). In other cases we consider parametered functions. The parameters are analyzed and compiled recursively, then the corresponding function is created, that will later evaluate these parameters. This method creates a tree structure of the functions (some parameters refer to other function, possibly creating circles, but these references do not belong to this structure). On the top of these trees, there is a special function, that is responsible for the evaluation of helper functions and for storing the input to allow recursion. There is a tree for each nviewer script definition which are located in a graph as the nodes, where the edges are the references. Some of these nodes do not depend on the initial input (the position of the point that is rendered), these nodes are evaluated during compilation and replaced by const functions. The remaining graph allows fast information flow.

3.3.3 Information ﬂow

Information moves through the system in packets. When a function has multiple parameters, they are located in one of the packages as one parameter. The main node is called ﬁrst with the position of the rendered point as a vector of ﬂoats. Nodes can return:

• a constant value

• the input packet

• a manipulated version of the values of some other functions or the input packet

In the last case, these functions are the parameters and they all get the initial packet, the only exceptions are the references. References point to other nodes, new packages are created, depending on what a speciﬁc node needs. In ideal case, the original package can be passed, or some part of it. When this is not the case, new items have to be packed together, which requires time.

31 3.4 Importing the engine

The engine, after importing, will provide a camera class to work with. A camera, an instance of this class, allows you to set the used function, the transformation, near and far clipping, the focus and the scale and it will render the image to a texture.

1 #include 2 #include "exporter/Helper/exporter.h" 3 #include "common/Math/include/info.h" 4 5 using namespace std ; 6 7 int main ( int argc , char *argv []) 8 { 9 Log log("log.txt", &std::cout); 10 PythonManager pythonManager(argv[0]); 11 12 try { 13 Exporter e(argc, argv); 14 return e.run (); 15 } catch (ERROR_CODE e) { 16 LOG<

Figure 3.3: Importing the engine for exporting purposes

Figure 3.3 is a simpliﬁed version of the exporters main function. For more details check the ﬁles in exporter/Helper/ in the project directory.

3.5 Importing the script parser

The script parser can be used outside the project, since it is the main part of the core module. function.h contains the deﬁnition of the abstract function class. The script parser can be used through this class very easily. The script parser can be accessed via a static method that creates a function based on a given ﬁlename or module name.

32 1 void Session::loadScriptInPreview(string filename, INotifiable *script, INotifiable *runtime) 2 { 3 data.script = filename; 4 clearScript(); 5 int numThreads = readInt(mcfg->getApp()->getAttribute("threads",DEFAULT_THREADS)); 6 LOG<::getFunction (filename, &ss, dim, 1.0, vram, true , numThreads); 10 func->setVRamNotifiers(script,runtime); 11 func->setStyle(&data.style); 12 func->set(&data.param); 13 cam = new Camera (func, mcfg); 14 file->save(data); 15 setD (); 16 }

Figure 3.4: Instantiating a function

The code in ﬁgure 3.4 is used in the Session (part of the graphical engine) class. It loads a script in preview mode for real-time displaying.

3.6 Extending the script language

The scrip language has some default functions that are implemented in C++ which are well optimized. This set of these functions can be extended. There is a function data file in the standard library of the project. This file contains the function definitions in a text file, which will be used to generate a C++ hpp file. New functions can be added here, by writing the name of the function (to call it from the script), the type and the definition in C++ (with some templates). There is a shell script to generate the hpp file. The project needs a rebuild to apply changes.

• Standard functions: gen/std/scriptfunctions.fdata

• Generator script: ./conﬁgure.sh

3.7 Optimizations

The ﬁrst version of the interpreter was not eﬃcient enough to be used in any real case, only for low quality images.

33 3.7.1 Function structure

The evaluation system is based on a graph that has a tree structure with some special edges that can create circles for recursion. A function inside one deﬁnition creates a tree, that can be evaluated on its own. These trees are connected via various types of edges that can deal with recursion. Compiling an NViewer script means building this graph. This graph is designed to create fast information ﬂow allowing thousands of millions of evaluations in minutes.

3.7.2 Constants

Functions allocate memory for the value they were going to return, according to the original theory. This was true for the constant values as well and this can be extremely slow. My idea was to reserve a space in the memory in advance for the value, when it is possible: constant values, queries, expressions that are evaluated multiple times, copying values in some manner. In these cases there is no need for memory allocation. Abolishing these factors resulted on sixfold speed.

3.7.3 Memory management

The basic idea was to allocate memory for the value that is created in a process and delete it when it is no longer needed. This is necessary to support all the types but still be efficient. There are only two problems with it: allocation, deallocation. Allocation is done for every single integer, which means a problem when billions of points are calculated. Deallocation is a requisite after allocation to avoid memory leakage. My solution was to create a virtual memory ( vram ), which is basically a huge fragment of the memory that is allocated when the script is loaded and deallocated when the script is closed. The size of the virtual memory can be configured manually. When allocating space for a value, the needed amount of bytes are taken from the vram and next time the remaining space will be used the same way. This means that there is on way to free space except when the last bytes are freed, but it is possible to clear the entire vram. The vram is cleared before processing one pixel and if the configuration is done properly regarding the quality settings, there will be enough space for the calculations. This optimization sped up the process to double calculation rate.

34 3.8 Comparism

When talking about new programming languages, it is important to compare it to others. I picked C++ and Haskell, because they are also, in some ways, good for this task, though they have some disadvantages.

3.8.1 C++

The language is written in C++, so it cannot be any faster than a well-written C++ code, though a C++ code can be very complex and long where errors are likely to occur. If a simple code in NViewerScript ranges around 10 lines, it is estimated to be around 100 lines in C++. This is not true for every possible code, but it is still a problem. An NViewer script can be written in much less time, almost no time needed for debugging.

3.8.2 Haskell

The structure of Haskell is not designed to evaluate the same function over 2 billion times and therefor it is not very eﬃcient. The same code in Haskell can be around 64 times slower than the nviewer script.

35 3.9 Examples

1 module performance.test1; 2 3 dimensions [3..3]; 4 5 parameter Real size = 1.0; 6 style Real transp = 0.9; 7 8 let main (v) = 9 let x2 = v[0]; 10 let y2 = v[1] - 0.65; 11 let z2 = v[2] - 0.65; 12 | if ( abs (v[0]) + abs (v[1]) + abs (v[2]-2) < size) Colour (1.0,1.0,1.0,transp) 13 | #ff000000;

Figure 3.5

Figure 3.5 demonstrates an octahedron used for lighting, performance and scripting language tests.

36 1 module fractals.mandelbox; 2 dimensions [3..3]; 3 4 style Int iterations = 10; 5 style Real limit = 1000.0; 6 style Colour inside1 = #ff0000ff; 7 style Colour inside2 = #00ffffff; 8 style Colour fcol = #00ff0000; 9 style Colour tcol = #0000ffa0; 10 style Real scale = 2.0; 11 style Real colouring = 1.0; 12 13 let transform (x) = 14 | if (x > 1) 2 - x 15 | if (x < (-1)) (-2) - x 16 | x; 17 18 let transformV (v) = {transform(v[0]), transform(v[1]), transform(v[2])}; 19 let transformV2 (v) = 20 let mgn = sqm(v); 21 | if (mgn < 0.25) 4*v 22 | if (mgn < 1) (1/mgn) * v 23 | v; 24 25 let iterate (z) = scale * transformV2(transformV(z)); 26 let sqm (v) = v[0]^2 + v[1]^2 + v[2]^2; 27 28 let level ((v,c,n)) = 29 | (*,*,0) -> 0 30 | if (sqm(v) <= limit) level(( iterate (v)+c,c,n-1))+1 31 | 0; 32 33 let getColour ((x,v)) = 34 | if (x >= iterations) lerp (inside1,inside2, 0.5 * ( sin (sqm(v)*colouring) + 1)) 35 | lerp (fcol,tcol,x/iterations); 36 37 let main (v) = getColour((level((v,v,iterations)),v));

Figure 3.6: Mandelbox

Figure 3.6 shows a close-up image of the mandelbox set with lighting and a soft red fog.

37 1 module special.heart; 2 dimensions [3..3]; 3 4 parameter Real size = 1.0; 5 parameter Real zx = 3.0; 6 parameter Real dx = 1.0; 7 8 style Real size2 = 1.5; 9 style Colour body = #ff0000ff; 10 style Colour from = #00000000; 11 style Colour to = #00000000; 12 13 let size_2 = size^2; 14 let size2_2 = size2^2; 15 let d_2 = size2_2 - size_2; 16 17 let sphere ((x,y,z)) = 18 let x2 = x^2; 19 let y2 = y^2; 20 let z2 = z^2; 21 | if (x2 + y2 + z2 < size_2) True 22 | False ; 23 24 let D ((x,y,z)) = x^2 + y^2 + z^2; 25 26 let main (v) = 27 let z = v[2]; 28 let d = v[0]^2 + v[2]^2; 29 let dd = D((v[0], v[1]-d*dx, v[2]*zx)); 30 | if (dd < size_2) body 31 | if (dd < size2_2) lerp (from,to,(dd-size_2) / d_2) 32 | #00000000;

Figure 3.7: heart

Figure 3.7 shows a sphere that has been scaled and manipulated to resemble a heart.

38 1 module test.fourdim; 2 3 dimensions [4..4]; 4 5 style Colour col = #ff0000ff; 6 parameter Real inner_size = 0.9; 7 8 let size = 1.0; 9 10 let main (v) = 11 | if ( abs (v[0]) > v[3] or abs (v[1]) > v[3] or abs (v[2]) > v[3]) #00000000 12 | if ( abs (v[0]) > abs (v[3])*inner_size and abs (v[1]) > abs (v[3])*inner_size) col 13 | if ( abs (v[0]) > abs (v[3])*inner_size and abs (v[2]) > abs (v[3])*inner_size) col 14 | if ( abs (v[2]) > abs (v[3])*inner_size and abs (v[1]) > abs (v[3])*inner_size) col 15 | #00000000;

(a) Normal section (b) rotated in 4D

Figure 3.8: 4D pyramid

Figure 3.8 is the skeleton of a 4-dimensional pyramid. The construction is done by deﬁning the skeleton of a cube where the size is speciﬁed by the 4th dimension, thus creating a pyramid.

39 1 module test.test3; 2 3 style Real transparency = 0.7; 4 style Real blue = 1.0; 5 6 let patternfunc ((x,y,z)) = 7 | ([0.0;1.0], [0.0;1.0], [0.0;1.0]) -> Colour ( sin (x), sin (y),blue,transparency) 8 | Colour (0.0,0.0,0.0,0.0); 9 10 let main (v) = patternfunc((v[0],v[1],v[2]));

Figure 3.9: Fog cube

Figure 3.9 is the ﬁrst object deﬁned using NViewerScript, a transparent cube.

40 Chapter 4

Summary

4.1 English

In this document I have written some words about fractals and described the problems that I wanted to solve; I made a demonstration about the scripting language that I have developed for this project, including some examples; I created a guide on using the interface of this tool; I described how the engine and the scripting language work in details. The scripting language is described in two parts, the ﬁrst provides information about the language itself, the second is about the interpreter. To sum up the progress done in this project, I have created

1. a graphical engine that works well with high dimensions and is able to render function to the screen, that takes a point in the space and provides a colour as result, which includes the transparency.

2. a function scripting language to allow easy and quick implementation of functions. This language comes with a lot of optimizations and therefor is much faster than other functional languages for this special task.

3. a user interface made in Qt. The target platform is Ubuntu linux, but it should operate on all platforms, supported by Qt. The user interface has on other version, designed for server computers, which does not have a graphical interface. This version does not rely on Qt.

These together make up a universal tool for rendering N-dimensional object, mostly fractals that is easy to handle. It can provide an excellent overview of the fractal in question when performing researches. It is also useful when working with shape equations and

41 complicated forms, described by mathematical functions. The most diﬃcult part was to optimize both the script interpreter and the graphical engine as well. Most of the code is built in a way to support parallel programming, but this feature is only implemented on the level of the CPU.

4.2 Magyar

Ebben a dokumentumban beszéltem a fraktálokrólés jellemeztem azokat a problémákat, amelyeket szerettem volna megoldani; Bemutattam a scripting nyelvet, amit ehhez a pro- jekthez fejlesztettem ki, néhány mintával együtt; Kész´ıtettem egy le´ırást a felhasználói felület kezeléséhez; Részletesen le´ırtam, hogy a motor és a nyelv hogyan m˝uködnek. A nyelv két részben van taglalva, az els˝orésze a nyelvi elemekr˝olszól, a második pedig a feldolgozóprogramról. Osszegzésképpen:¨ kész´ıtettem

1. egy grafikai motort ami m˝uködik magas dimenziószámok esetén és képes függvényeket megjelen´ıteni, amelyek egy tér egyes pontjaihoz sz´ıneket rendelnek.

2. egy funkcionális scripting nyelvet ami lehet˝ovéteszi a korábban eml´ıtett függvények le´ırását. A nyelvi elemeket kioptimalizáltam, ´ıgy másfunkcionális nyelvekhez képest hatékonyabban képesek végrehajtani a feladatban megfogalmazott speciális m˝uveleteket.

3. egy felhasználóifelületet amit Qt környezetben´ırtam meg. A célplatform Ubuntu linux, de a programnak m˝uködnie kellene másplatformokon is, amit a Qt támogat. A felhasználóifelületnek van egy konzolos verziója, amit szervergépekhez terveztem. Ez a verziónem ép´ıt a Qt-ra, ´ıgy anélkül is lehet ford´ıtani, futtatni.

A felsorolt elemek együttvéve alkotnak egy eszközt, mely képes N-dimenziósalakzatokat, többnyire fraktálokat, megjelen´ıteni. Ez az eszközhasználható fraktáltémájúkutatások esetén az ábrákgenerálására. Továbbáakkor is hasznos lehet, ha komplex alakzategyen- leteket kell ábrázolni. A feladat legnehezebb részét a grafikai motor és a script feldolgozóoptimalizálása jelen- tette. A kódtöbbsége úgy van meg´ırva, hogy támogassa a párhuzamos programozást, viszont csak CPU platformon m˝uködik.

42 Chapter 5

Results & testing

5.1 Testing plan

Camera controls