Symbolical Index Reduction and Completion Rules for Importing Tensor Index Notation Into Program- Ming Languages

Symbolical Index Reduction and Completion Rules for Importing Tensor Index Notation into Program- ming Languages

Satoshi Egi

Mathematics Subject Classiﬁcation (2010). Primary 53-04; Secondary 68-04. Keywords. Tensor index notation; diﬀerential forms; scalar parameters; tensor parameters.

Abstract. In mathematics, many notations have been invented for the concise representation of mathematical formulae. Tensor index notation is one of such notations and has been playing a crucial role in describing formulae in mathematical physics. This paper shows a programming language that can deal with symbolical tensor indices by introducing a set of tensor index rules that is compatible with two types of parameters, i.e., scalar and tensor parameters. When a tensor parameter obtains a tensor as an argument, the function treats the tensor argument as a whole. In contrast, when a scalar parameter obtains a tensor as an argument, the function is applied to each component of the tensor. On a language with scalar and tensor parameters, we can design a set of index reduction rules that allows users to use tensor index notation for arbitrary user- defined functions without requiring additional description. Furthermore, we can also design index completion rules that allow users to define the operators concisely for differential forms such as the wedge product, exterior derivative, and Hodge star operator. In our proposal, all these tensor operators are user-defined functions and can be passed as arguments of high-order functions.

1. Introduction From the latter half of the twentieth century, after the first implementation of a compiler for high-level programming languages, a large amount of notations have been invented in the field of programming arXiv:1804.03140v2 [cs.PL] 10 Feb 2021 languages. Lexical scoping [10], high-order functions [28, 36], and pattern matching [13, 16, 17] are features specific to programming invented for describing algorithms. At the same time, researchers evolve programming languages by importing successful mathematical notations. For example, deci- mal number system, function modularization, and infix notation for basic arithmetic operators have been imported in most programming languages [9]. The importation of mathematical notations into programming is not always easy. This is because the semantics of some mathematical notations are vague and complex to implement them as a part of programming languages. This paper discusses a method for importing tensor index notation invented by Ricci and Levi- Civita [32] for dealing with high-order tensors into programming languages. Tensor calculus often appears in the various fields of computer science. Tensor calculus is an important application of symbolic computation [25]. Tensor calculus is heavily used in computational physics [20] and computer visions [19]. Tensor calculus also appears in machine learning to handle multidimensional data [21]. The importation of tensor index notation makes programming in these fields easy. The current major method for dealing with tensors is using a special syntax for describing loops for generating multi-dimensional arrays. The Table [6] expression from the Wolfram language is such a 2 S. Egi

syntax construct. Xij + Yij is represented as follows with the Table expression. The following program assumes that all dimensions corresponding to each index of the tensor are a constant M. Table[X[[i,j]] + Y[[i,j]],{i,M},{j,M}]

For contracting tensors, we use the Sum [5] expression inside Table. XikYkj is represented as follows. Table[Sum[X[[i,k]] * Y[[k,j]], {k,M}], {i,M},{j,M}] This method has the advantage that we can use an arbitrary function defined for scalar values for tensor operations. The following Wolfram program represents ∂Xij/∂xk. D is the differential function in the Wolfram language. Table[D[X[[i,j]],x[[k]]],{i,M},{j,M},{k,M}] Due to this advantage, the Wolfram language has been used by mathematicians in actual research. [27, 26] However, in this method, we cannot modularize tensor operators such as tensor multiplication by functions. Due to this restriction, we cannot syntactically distinguish applications of different tensor operators, such as tensor multiplication, wedge product, and Lie derivative, in programs. This is because we need to represent these tensor operators combining Table and Sum every time when we use them. Modularization by functions is also important for combining index notation with high- order functions. If we can pass tensor operators to high-order functions, we can represent a formula like “Xi1 Xi2 ...Xin ” (the number of tensors multiplied depends on the parameter n) by passing the operator for tensor multiplication to the fold function [12]. There are other existing works that take the same approach with the Wolfram language. NumPy’s einsum operator [4], Diderot’s EIN operator [23], and tensor comprehensions [39] are such work. Some of them provide a syntactic construct whose appearance is similar to mathematical formulae. However, they have the same restriction on function modularization. This restriction comes from the requirement that users need to specify the indices of the result tensors (e.g., “ij” of Aij = XikYkj) for determining whether to contract a pair of identical symbolic indices. Maxima [2, 38, 35] takes a different approach from Wolfram. In Maxima, we describe formulae of tensor calculus using several special operators prepared for tensors, such as + and ·, that support tensor index notation. + is a function that sums the components of two tensors given as arguments. · is a function for tensor multiplication. It takes the tensor product of the two tensors given as arguments and takes the sum of the trace if there are pairs of a superscript and a subscript with the same index variable. Using this method enables index notation to be directly represented in a program as the mathematical expressions. However, in this method, index notation can be used only by functions that are specially prepared to use it. One of the reasons is that index rules differ for each operator. For example, + and · work in the different way for the same arguments such as Ai + Bi and Ai · Bi. Ai + Bi returns a vector, but Ai · Bi returns a scalar. Ahalander’s work [8] that implements index notation on C++ takes the same approach. Array-oriented programming languages, such as APL and J, take a completely different approach. They do not use tensor index notation for representing tensor calculus. Instead, they invented a new notion, function rank [11]. Function rank specifies how to map the operators to the components of tensors. When the specified function rank is 0 for an argument matrix, the operator is mapped to each scalar component of the matrix (Ai + Bjk). When the specified function rank is 1 for an argument matrix, the operator is mapped to the rows of the matrix by regarding the matrix as a vector of vectors (Aj + Bij). When the specified function rank is 2 for an argument matrix, the operator is applied to the matrix directly (Ai + Bij). J> (2 $ 1 2) +"1 0 (2 2 $ 10 20 30 40) 11 12 21 22

31 32 Symbolical Index Reduction and Completion Rules for Importing Tensor Index Notation 3

41 42 J> (2 $ 1 2) +"1 1 (2 2 $ 10 20 30 40) 11 22 31 42 J> (2 $ 1 2) +"1 2 (2 2 $ 10 20 30 40) 11 21 32 42

A similar idea to the function rank is also imported into various programming languages and frame- works, including Wolfram [1] and NumPy [7]. However, the function rank has a limitation that it does not allow to represent an expression that requires transposition of an argument tensor: e.g., Aij + Bji (this expression requires the transposition of the matrix B). This paper shows that a combination of a set of symbolical index reduction and completion rules with scalar and tensor parameters, which is a simplified notion of function rank, enables us to use tensor index notation for arbitrary functions defined just for scalar values without requiring any additional descriptions. In our method, we do not distinguish tensor operators that handle symbolical tensor indices with the other user-defined functions, therefore we can pass these tensor operators as arguments of high-order functions.

2. Language Design for Importing Tensor Index Notation This section presents a new method for importing tensor index notation into programming languages. Briefly, it is achieved by introducing two types of parameters, scalar parameters and tensor parameters, and simple index reduction rules. First, we introduce scalar and tensor parameters. Second, we introduce a set of index reduction rules that is compatible with them. The combination of scalar and tensor parameters and the proposed index reduction rules enables us to apply user-defined functions to tensors using tensor index notation. Third, we introudce index completion rules for omitted tensor indices. By designing the index completion rules for omitted indices properly, we become able to concisely define the operators even for the differential forms [34], such as the wedge product, exterior derivative, and Hodge star operator. The method proposed in this paper has already been implemented in the Egison programming language [15]. Egison has a similar syntax to the Haskell programming language.

2.1. Scalar and Tensor Parameters Scalar and tensor parameters are a similar notion to the function rank [11]. When a scalar parameter obtains a tensor as an argument, the function is applied to each component of the tensor. In contrast, when a tensor parameter obtains a tensor as an argument, the function treats the tensor argument as a whole. We call a function that takes only scalar parameters scalar function and a function that takes only tensor parameters tensor function. For example, “+”, “-”, “*”, and “/” should be defined as scalar functions; a function for multiplying tensors and a function for matrix determinant should be defined as tensor functions. The difference between function rank and these two types of parameters is that function rank allows users to control the level of mapping, whereas scalar and tensor parameters only allow users to specify whether we map the function to each component of the tensor or not. Instead, the proposed tensor index reduction rules combined with scalar and tensor parameters allow users to control the level of mapping. Figure 1a shows the definition of the min function as an example of a scalar function. The min function takes two numbers as the arguments and returns the smaller one. “$” is prepended to the beginning of the parameters of the min function. It means the parameters of the min function are scalar parameters. When a scalar parameter obtains a tensor as an argument, the function is applied to each component of the tensor like tensor product as Figures 1b and 1c. As Figure 1c, if the indices of the tensors given as arguments are identical, the result matrix is reduced to the vector that consists of the diagonal components. This reduction is defined in the proposed tensor index reduction rules that will 4 S. Egi def min $x $y := if x < y then x else y

(a) Deﬁnition of the min function 1 10 min(1, 10) min(1, 20) min(1, 30) 1 1 1 min(2 , 20 ) = min(2, 10) min(2, 20) min(2, 30) = 2 2 2 3 30 min(3, 10) min(3, 20) min(3, 30) 3 3 3 i j ij ij (b) Application of the min function to the vectors with diﬀerent indices 1 10 min(1, 10) min(1, 20) min(1, 30) min(1, 10) 1 min(2 , 20 ) = min(2, 10) min(2, 20) min(2, 30) = min(2, 20) = 2 3 30 min(3, 10) min(3, 20) min(3, 30) min(3, 30) 3 i i ii i i (c) Application of the min function to the vectors with identical indices

Figure 1. Deﬁnition and application of min function def (.) %t1 %t2 := contract (+) (t1 * t2)

(a) Deﬁnition of the “.” function 1i 10 10 20 30i 2 · 20 = contract(+, 20 40 60 ) = 10 + 40 + 90 = 140 3 30 30 60 90 i i 1 10 10 10 2 · 20 = contract(+, 40 ) = 40 3 30 90 90 i i i i 1 10 10 20 30 10 20 30 2 · 20 = contract(+, 20 40 60 ) = 20 40 60 3 30 30 60 90 30 60 90 i j ij ij

(b) Application of the “.” function

Figure 2. Deﬁnition and application of “.” function

be explained in the next section. Thus the min function can handle tensors even though it is defined without considering tensors. Figure 2a shows the definition of the “.” function as an example of a tensor function. “.” is a function for multiplying tensors. “%” is prepended to the beginning of the parameters of the “. ” function. It means the parameters of the “.” function are tensor parameters. As with ordinary functions, when a tensor is provided to a tensor parameter, the function treats the tensor argument as a whole maintaining its indices. “.” is defined as an infix operator. For defining an infix operator, we enclose the name of a function with parenthesis. In Figure 2a, “+” and “*” are scalar functions for addition and multiplication, respectively. contract is a primitive function to contract a tensor that has pairs of a superscript and subscript with identical symbols. Figure 2b shows the example for calculating the inner product of two vectors using the “.” function. We can use the “.” function for any kind of tensor multiplication such as tensor product and matrix multiplication as well as inner product. Let us show another example in differential geometry. When the mathematical expression in Fugure 3a is expressed in a standard way in the Wolfram language, it becomes a program such as the one shown in Figure 3b. The same expression can be expressed in our system as shown in Figure 3c. Our system supports two types of indices, both superscripts and subscripts. A subscript is represented by “_”. A superscript is represented by “~”. A double loop consisting of the Table and Sum expressions Symbolical Index Reduction and Completion Rules for Importing Tensor Index Notation 5

∂Γi ∂Γi Ri = jl − jk + Γm Γi − Γm Γi jkl ∂xk ∂xl jl mk jk ml (a) Formula of Riemann curvature tensor R=Table[D[Γ[[i,j,l]],x[[k]]] - D[Γ[[i,j,k]],x[[l]]] +Sum[Γ[[m,j,l]] Γ[[i,m,k]] - Γ[[m,j,k]] Γ[[i,m,l]], {m,M}], {i,M},{j,M},{k,M},{l,M}]

(b) Wolfram program that represents the formula in Figure 3a def R~i_j_k_l := withSymbols [m] ∂/∂ Γ~i_j_l x~k - ∂/∂ Γ~i_j_k x~l + Γ~m_j_l . Γ~i_m_k - Γ~m_j_k . Γ~i_m_l

Figure 3. Representations of Riemann curvature tensor formula appears in the program in the Wolfram language, whereas the program in our system is flat, similar to the mathematical expression. This is achieved by using tensor index notation in the program. In particular, the reason that the loop structure by the Sum expression in the Wolfram language does not m i m i appear in our expression to express Γ jkΓ ml − Γ jlΓ mk is that the “.” function can handle Einstein summation notation. It is emphisized that the program (∂/∂ Γ~i_j_k x~l) in this example expresses i ∂Γ jk ∂xl in the first term on the right-hand side. In the Wolfram language, the differential function D is applied to each tensor component, but our differential function ∂/∂ is applied directly to the tensors. The differential function ∂/∂ is defined in a program as a scalar function like the min function. When a tensor is provided as an argument to a scalar function, the function is applied automatically to each component of the tensor. Therefore, when defining a scalar function, it is sufficient to consider only a scalar as its argument. That is, in the definition of the ∂/∂ function, the programmers need write the program only for the case in which the argument is a scalar value. Despite that, the program ∂/∂ Γ~i_j_k x~l returns a fourth-order tensor. Thus, we can import tensor index notation including Einstein notation into programming if we clearly distinguish between tensor functions such as “.” and scalar functions such as “+” and ∂/∂. 2.2. Reduction Rules for Tensors with Indices This section presents a whole set of index reduction rules that are compatible with the scalar and tensor parameters. Let us consider the reduction rules only for a single tensor, and that is enough to define the set of the index reduction rules. This is because the reduction rules are applied only after a scalar function is applied to tensor arguments as seen in Fig. 1. 2.2.1. Multiple Identical Symbolical Indices. Tensors are reduced only when they have identical symbols as their indices. First, let us discuss the cases that tensors have identical superscripts or subscripts. When multiple indices of the same symbol appear, our system converts it to the tensor composed of diagonal components for these indices. After this conversion, the leftmost index symbol remains. For example, the indices “_i_j_i” convert to “_i_j”. In our system, unbound variables are treated as symbols. These symbols can be used as indices of tensors. In our system, a tensor is expressed by enclosing its components with “[|” and “|]”. We express a higher-order tensor by nesting this notation, as we do for an n-dimensional array. In this paper, we show the evaluation result of a program in the comment that follows the program. “--” is the inline comment delimiter of our language. [|[|11,12,13|],[|21,22,23|],[|31,32,33|]|]_i_j-- [|[|11,12,13|],[|21,22,23|],[|31,32,33|]|]_i_j 6 S. Egi

E({A, xs}) = if e(xs) = [] then {A, xs}) elsif e(xs) = [{k,j}, ... ] & p(k,xs) = p(j,xs) then E({diag(k, j, A), remove(j, xs)) elsif e(xs) = [{k,j}, ... ] & p(k,xs) != p(j,xs) then E({diag(k, j, A), update(k, 0, remove(j, xs)))

Figure 4. Pseudo code of index reduction

[|[|11,12,13|],[|21,22,23|],[|31,32,33|]|]_i_i-- [|11,22,33|]_i

[|[|[|1,2|],[|3,4|]|],[|[|5,6|],[|7,8|]|]|]_i_j_i-- [|[|1,3|],[|6,8|]|]_i_j

When three or more subscripts of the same symbol appear, our system converts it to the tensor composed of diagonal components for all these indices.

[|[|[|1,2|],[|3,4|]|],[|[|5,6|],[|7,8|]|]|]_i_i_i-- [|1,8|]_i

Superscripts and subscripts behave symmetrically. When only superscripts are used, they behave in exactly the same manner as when only subscripts are used.

[|[|[|1,2|],[|3,4|]|],[|[|5,6|],[|7,8|]|]|]~i~j~i-- [|[|1,3|],[|6,8|]|]~i~j

2.2.2. Superscripts and Subscripts with Identical Symbols. Next, let us consider a case in which the same symbols are used for a superscript and a subscript. In this case, some existing work [8] automatically contracts the tensor using “+”. In contrast, our system just converts it to the tensor composed of diagonal components, as in the above case. However, in this case, the summarized indices become a supersubscript, which is represented by “~_”.

[|[|11,12,13|],[|21,22,23|],[|31,32,33|]|]~i_i-- [|11,22,33|]~_i

Even when three or more indices of the same symbol appear that contain both superscripts and subscripts, our system converts it to the tensor composed of diagonal components for all these indices.

[|[|[|1,2|],[|3,4|]|],[|[|5,6|],[|7,8|]|]|]~i~i_i-- [|1,8|]~_i

The reason not to contract it immediately is to parameterize an operator for contraction. The components of supersubscripts can be contracted by using the contract expression. The contract expression receives a function to be used for contraction as the ﬁrst argument, and a target tensor as the second argument. This feature allows us to implement a tensor multiplication function. contract (+) [|11,22,33|]~_i-- 66

In the above program, the “+” function is passed to contract as a prefix operator. When an infix operator is enclosed with parenthesis, it becomes an prefix operator.

2.2.3. Pseudo Code for Index Reduction. Figure 4 shows the pseudo-code of index reduction explained in the above. E(A,xs) is a function for reducing a tensor with indices, where A is an array that consists of tensor components and xs is a list of indices appended to A. For example, E(A,xs) works as follows with the tensor whose indices are “~i_j_i”. We use 1, -1, and 0 to represent a superscript, subscript, and supersubscript, respectively.

E({[|[|[|1,2|],[|3,4|]|] ,[|[|5,6|],[|7,8|]|]|] ,[{i,1}, {j,-1}, {i,-1}]}) = {[|[|1,3|],[|6,8|]|], [{i,0}, {j,-1}]} Symbolical Index Reduction and Completion Rules for Importing Tensor Index Notation 7

Let us explain the helper functions used in Figure 4: e(xs) is a function for ﬁnding pairs of identical indices from xs; diag(k, j, A) is a function for creating the tensor that consists of diagonal components of A for the k-th and j-th order; remove(k, xs) is a function for removing the k-th element from xs; p(k , xs) is a function for obtaining the value of the k-th element of the assoc list xs; update(k, p, xs) is a function for updating the value of the k-th element of the assoc list xs to p. These functions work as follows. e([{i, 1}, {j, -1}, {i, -1}]) = [{1,3}] diag(1, 2, [|[|11,12|],[|21,22|]|]) = [|11,22|] p(2, [{i, 1}, {j, -1}]) = -1 remove(2, [{i, 1}, {j, -1}]) = [{i, 1}] update(2, 0, [{i, 1}, {j, -1}]) = [{i, 1}, {j, 0}]

2.3. Implementation of Scalar and Tensor Parameters Let us explain how to implement scalar and tensor parameters. The implementation of tensor parameters is the same with the ordinary parameters because the function treats the argument tensor as it is. In contrast, a function with scalar parameters is converted to a function only with tensor parameters by using the tensorMap function as follows.

\ $x $y -> ... -- =>\%x%y-> -- tensorMap(\%x-> tensorMap(\%y-> ...)y)) --x

As the name implies, the tensorMap function applies the function of the first argument to each component of the tensor provided as the second argument. When the result of applying the function of the first argument to each component of the tensor provided as the second argument is a tensor with indices, it moves those indices to the end of the tensor that is the result of evaluating the tensorMap function. Let us review the min function defined in Figure 1a as an example. This min function can handle tensors as arguments as follows. min [|1,2,3|]_i [|10,20,30|]_j-- [|[|1,1,1|],[|2,2,2|],[|3,3,3|]|]_i_j min [|1,2,3|]_i [|10,20,30|]_i-- [|1,2,3|]_i

Note that the tensor indices of the ﬁrst evaluated result are “_i_j”. If the tensorMap function simply applies the function to each component of the tensor, the result of this program is [|[|1 1 1|]_j [|2 2 2|]_j [|3 3 3|]_j|]_i. However, as explained above, if the results of applying the function to each component of the tensor are tensors with indices, it moves those indices to the end of the tensor that is the result of evaluating the tensorMap function. This is the reason that the indices of the evaluated result are “_i_j”. This mechanism enables us to directly apply scalar functions to tensor arguments using index notation as the above example. Next, let us review the “.” function deﬁned in Figure 2a as an example of a tensor function. When a tensor with indices is given as an argument of a tensor function, it is passed to the tensor function maintaining its indices. It allows us to directly apply tensor functions to tensor arguments using index notation as in the following example.

[|1,2,3|]~i . [|10,20,30|]_i-- 140 [|1,2,3|]_i . [|10,20,30|]_j-- [|[|10,20,30|],[|20,40,60|],[|30,60,90|]|]_i_j [|1,2,3|]_i . [|10,20,30|]_i-- [|10,40,90|]_i

Tensor parameters are rarely used compared with scalar paramters because a tensor parameter is used only when deﬁning a function that contracts tensors. 8 S. Egi

2.4. Inverted Scalar Arguments The ∂/∂ function in Figure 3c is a scalar function. However, ∂/∂ is not a normal scalar function. ∂/∂ is a scalar function that inverts indices of the tensor given as its second argument. For example, the program (∂/∂ Γ~i_j_k x~l) returns the fourth-order tensor with the indices “~i_j_k_l”. To deﬁne scalar functions such as ∂/∂, we use inverted scalar parameters. Inverted scalar parameters are represented by “*$”. A program that uses inverted scalar parameters is transformed as follows. The flipIndices function is a primitive function for inverting the indices of a tensor provided as an argument upside down. Supersubscripts remain as supersubscripts even if they are inverted. def ∂/∂ $f *$x := ... -- => def ∂/∂ %f%x := tensorMap(\%f-> tensorMap(\%x-> ...)(flipIndicesx)) --f

In the following example, we apply ∂/∂ to tensors. ∂/∂ [|(r * (sin θ)),(r * (cos θ))|]_i [|r,θ|]_j -- [|[|(sin θ),(r*(cos θ))|],[|(cos θ),(-1*r*(sin θ))|]|]_i~j ∂/∂ [|(r * (sin θ)),(r * (cos θ))|]_i [|r,θ|]_i -- [|(sin θ),(-1*r*(sin θ))|]~_i

2.5. The withSymbols Expression The withSymbols expression is syntax for generating new local symbols as the Module [3] expression in the Wolfram language. One-character symbols that are often used as indices of tensors such as i, j, and k are often used in another part of a program. Generating local symbols using withSymbols expressions enables us to avoid variable conﬂicts for such symbols. The withSymbols expression takes a list of symbols as its ﬁrst argument. These symbols are valid only in the expression given in the second argument of the withSymbols expression. withSymbols [i] contract (+) ([|1,2,3|]~i * [|10,20,30|]_i)-- 60

It acts in a special way when the evaluation result of the body of the withSymbols expression contains the symbols generated by the withSymbols expressions. In that case, the result tensor is transposed to shift those symbols backward and remove them. In the following evaluation result, the matrix is transposed because j is shifted backward before it removed. This mechanism is useful to handle diﬀerential forms that will be discussed in the next section. withSymbols [j] [|[|1,2|],[|3,4|]|]_j_i)-- [|[|1,3|],[|2,4|]|]_i

2.6. Index Completion Rules for Tensors with Omitted Indices By designing the index completion rules for omitted indices properly, we can extend the proposed method explained so far to express a calculation handling the differential forms [34]. There are several prior studies [22, 37] for programming differential forms. These studies prepare special data structures for differential forms and the operators for differential forms, such as exterior derivative and Hodge star operator, are implemented on these data structures. On the other hand, our approach represents a differential form just as a multi-dimentional array as we do for tensors. This approach allows us to define the operators for differential forms using tensor index notations and the operators defined for tensors. 2.6.1. Differential Forms. In mathematics, we sometimes denote a p-form by appending only (n−p) indices to an n-th order tensor. For example, a third order tensor ω~i_j denotes a matrix-valued 1- form. We import this notation for p-forms into programming by designing the proper index completion rules that are suitable for this notation. For general scalar functions, it is natural to complement the same indices to each argument tensor as follows. Let A and B in the following examples be scalar-valued 2-forms. A + B-- => A_t1_t2+ B_t1_t2 Symbolical Index Reduction and Completion Rules for Importing Tensor Index Notation 9

In most of the cases, if we complement indices as above, we can represent the operation for differential forms. However, this completion is not suitable for functions specially defined for differential forms such as the wedge product and exterior derivative. In the case of the wedge product, we would like to append the different indices to each argument as follows. A ∧ B-- => A_t1_t2 ∧ B_t3_t4

We introduce the “!” operator for this purpose. If the “!” operator is prepended to function application, the omitted indices are complemented by the latter method. With this method, the function for calculating the wedge product is deﬁned in one line as follows. def (∧) %A %B := A !. B

We can also deﬁne the exterior derivative in one line as follows. The flip function is a function for swapping the arguments of a two-argument function. It is used to transpose the result. def d %A := !(flip ∂/∂) params A Next, we can deﬁne Hodge star operator. def hodge %A := let k := dfOrder A in withSymbols [i, j] (sqrt (abs (det g_#_#))) * (foldl (.) (( N k)_(i_1)..._(i_N) . A..._(j_1)..._(j_k)) (map (\t -> g~(i_t)~(j_t)) [1..k])) It is a direct translation of the following mathematical formula for Hodge star operator [33].

p i1j1 ikjk ik+1 in ∗ A = det |g| · i1...in · Aj1...jk · g ··· g · e ∧ ... ∧ e (1) In the above program, dfOrder is a function that returns p when it obtains an p-form; det is a function for calculating the determinant of the argument matrix; is the Levi-Civita symbols; “#” used in the indices as g_#_# represents a dummy symbol. All instances of “#” are treated as different symbols. In a program that deals with high-order tensors, the number of symbols used for indices increases. A dummy symbol suppresses that. This mechanism makes it easier to distinguish indices, thereby also improves the readability of the program. Thus, we can concisely define the operators for differential forms by controlling the completion of omitted indices. We can see the sample programs that use the functions defined above in Egison Mathematics Notebook[14].

3. Demonstration This section presents a program for calculating the Riemann curvature tensor [18, 34, 30], in which the fourth-order tensor is necessary to be calculated. Figure 5a is a program for calculating the Riemann curvature tensor of S2 using the formula of curvature form. Our language has an interface for Jupyter Notebook [24], which renders a evaluation result as mathematical formula. In our system, when binding a tensor to a variable, we can specify the type of indices in the variable name. For example, we can bind different tensors to $g__, $g~~, Γ___, and Γ~__. This feature is also implemented in Maxima [2] and simplifies variable names. In Figure 5a, some of the tensors are bound to a variable with symbolical indices such as Γ_i_j_k. It is automatically desugared as follows. This syntactic sugar renders a program closer to the mathematical expression. transpose is a function for transposing the tensor in the second argument as specified in the first argument. def Γ_i_j_k := ... -- => def Γ___ := withSymbols[i,j,k] --(transpose[i,j,k] ...)

The antisymmetrize function is used in the definition of the curvature form Ω~i_j. It is a function for normalizing a differential form to an anti-symmetric tensor. It is defined in our system as a tensor function. The evenAndOddPermutations function returns a tuple of even and odd permutations of the given 10 S. Egi size. The subrefs function appends the elements of the second argument collection as the subscripts to the first argument tensor. def antisymmetrize %X := let p := dfOrder X (es, os) := even-and-odd-permutations p in withSymbols [i] (sum (map (\ σ -> subrefs X (map (\t -> i_(σ t)) [1..p]) es) - sum (map (\ σ -> subrefs X (map (\t -> i_(σ t)) [1..p]) os)) / (fact p)

Figure 5b shows the mathematical formulae of Christoﬀel symbols and curvature form. These formulae appear in the 4th, 5th, and 8th programs in Figure 5a.

(a) A program for calculating Riemann curvature tensor of S2 using the formula of curvature form 1 ∂g ∂g ∂g Γ = ( ij + ik − kj )Γi = gimΓ Ωi = dωi + ωi ∧ ωk ijk 2 ∂xk ∂xj ∂xi jk mjk j j k j

(b) Mathematical formulae used in the above program

Figure 5. Program for diﬀerential geometry in our language Symbolical Index Reduction and Completion Rules for Importing Tensor Index Notation 11

4. Conclusion This paper showed that we can import tensor index notation into a programming language by introducing a set of symbolical index reduction rules that is compatible with scalar and tensor parameters. The proposed method allows users to define functions that handle symbolic tensor indices without additional descriptions for handling these symbolic tensor indices. We demonstrated the proposed method by showing our proof-of-concept programming language Egison. The proposed method can be implemented also in the other programming languages. It is an interesting research topic to think about how to incorporate the proposed method into existing programming languages. For example, we think it is possible to introduce the concept of scalar and tensor parameters using a static type system [31]. In a language with a static type system, whether the parameter of a function is a scalar or tensor parameter can be specified when specifying the type of the function. In particular, it is of substantial significance to incorporate the proposed method into programming languages such as Formura [29] and Diderot [23] that have a compiler to generate code for executing tensor calculation efficiently. For example, incorporating the proposed method into For- mura would enable us to describe physical simulation using not only the Cartesian coordinate system but also more general coordinate systems such as the polar and spherical coordinate systems in simple programs. This paper focuses on the importation of tensor index notation into programming. However, there are still many notations in mathematics that are useful but not yet introduced into programming. Importation of these notations is not only useful but also leads us to deepen our understanding on mathematical notations.

Acknowledgements The author thanks Hiroki Fukagawa for motivating the author to import the notation for diﬀerential forms into programming languages. The author thanks Yutaka Shikano for intensive discussions on the content of the paper. The author thanks Momoko Hattori and Mayuko Kori for developing a user- friendly interface of the proposed system. The author thanks Hiromi Hirano, Hidehiko Masuhara, and Michal J. Gajda for helpful feedback on the earlier versions of the paper.

References [1] Listable - Wolfram Language Documentation. http://reference.wolfram.com/language/ref/Listable.html, 2020. [2] Maxima - a Computer Algebra System. http://maxima.sourceforge.net/, 2020. [3] Module - Wolfram Language Documentation. http://reference.wolfram.com/language/ref/Module.html, 2020. [4] NumPy. http://www.numpy.org/, 2020. [5] Sum - Wolfram Language Documentation. http://reference.wolfram.com/language/ref/Sum.html, 2020. [6] Table - Wolfram Language Documentation. http://reference.wolfram.com/language/ref/Table.html, 2020. [7] Universal functions (ufunc) — NumPy v1.16 Manual. https://docs.scipy.org/doc/numpy/reference/ ufuncs.html, 2020. [8]K. Ahlander.˚ Einstein summation for multidimensional arrays. Computers & Mathematics with Applica- tions, 44(8-9):1007–1017, 2002. [9] J. W. Backus. The history of FORTRAN i, II and III. IEEE Annals of the History of Computing, 1(1):21– 37, 1979. [10] J. W. Backus, F. L. Bauer, J. Green, C. Katz, J. McCarthy, P. Naur, A. Perlis, H. Rutishauser, K. Samel- son, B. Vauquois, et al. Report on the algorithmic language ALGOL 60. Numerische Mathematik, 2(1):106– 136, 1960. [11] R. Bernecky. An introduction to function rank. ACM SIGAPL APL Quote Quad, 18(2):39–43, 1987. 12 S. Egi

[12] R. S. Bird and P. Wadler. Introduction to functional programming. Prentice Hall International series in computer science. Prentice Hall, 1988. [13] R. M. Burstall. Proving properties of programs by structural induction. The Computer Journal, 12(1):41– 48, 1969. [14] S. Egi. Egison mathematics notebook. http://www.egison.org/math/, 2017. [15] S. Egi. The Egison Programming Language. http://www.egison.org/, 2020. [16] S. Egi and Y. Nishiwaki. Non-linear pattern matching with backtracking for non-free data types. In Asian Symposium on Programming Languages and Systems (APLAS 2018), pages 3–23. Springer, 2018. [17] S. Egi and Y. Nishiwaki. Functional programming in pattern-match-oriented programming style. In The Art, Science, and Engineering of Programming, 2020. [18] D. A. Fleisch. A student’s guide to vectors and tensors. Cambridge University Press, 2011. [19] R. Hartley and A. Zisserman. Multiple view geometry in computer vision. Cambridge university press, 2003. [20] T. J. R. Hughes. The Finite Element Method: Linear Static and Dynamic Finite Element Analysis. Dover Civil and Mechanical Engineering. Dover, 2000. [21] Y. Ji, Q. Wang, X. Li, and J. Liu. A Survey on Tensor Techniques and Applications in Machine Learning. IEEE Access, 7:162950–162990, 2019. [22] J. Karczmarczuk. Functional coding of differential forms. In Scottish Workshop on Functional Program- ming, 1999. [23] G. Kindlmann et al. Diderot: a domain-specific language for portable parallel scientific visualization and image analysis. IEEE transactions on visualization and computer graphics, 22(1):867–876, 2016. [24] T. Kluyver, B. Ragan-Kelley, F. Pérez,B. E. Granger, M. Bussonnier, J. Frederic, K. Kelley, J. B. Ham- rick, J. Grout, S. Corlay, et al. Jupyter notebooks - a publishing format for reproducible computational workflows. In ELPUB, pages 87–90, 2016. [25] A. V. Korolkova, D. S. Kulyabov, and L. A. Sevastyanov. Tensor computations in computer algebra systems. Programming and Computer Software, 39(3):135–142, 2013. [26] Y. Maeda, S. Rosenberg, and F. Torres-Ardila. Computation of the Wodzicki-Chern-Simons form in local coordinates. Computations for S1 actions on S2 × S3. http://math.bu.edu/people/sr/articles/ ComputationsChernSimonsS2xS3 July 1 2010.pdf, 2010. [27] Y. Maeda, S. Rosenberg, and F. Torres-Ardila. The geometry of loop spaces II: Characteristic classes. Advances in Mathematics, 287:485–518, 2016. [28] J. McCarthy. History of LISP. In History of Programming Languages Conference (HOPL-I), pages 173– 185, 1978. [29] T. Muranushi et al. Automatic generation of efficient codes from mathematical descriptions of stencil computation. In Proceedings of the 5th International Workshop on Functional High-Performance Computing, pages 17–22. ACM, 2016. [30] Y. Ollivier. A visual introduction to Riemannian curvatures and some discrete generalizations. Anal- ysis and Geometry of Metric Measure Spaces: Lecture Notes of the 50th Séminaire de Mathématiques Supérieures (SMS), Montréal, pages 197–219, 2011. [31] B. C. Pierce. Types and Programming Languages. MIT Press, 2002. [32] M. Ricci and T. Levi-Civita. Méthodes de calcul différentiel absolu et leurs applications. Mathematische Annalen, 54(1-2):125–201, 1900. [33] M. Ru. Hodge theory on Riemannian manifolds. https://www.math.uh.edu/∼minru/Riemann08/ hodgetheory.pdf, 2008. [34] B. F. Schutz. Geometrical methods of mathematical physics. Cambridge university press, 1980. [35] E. Solomonik and T. Hoefler. Sparse tensor algebra as a parallel programming model. Preprint at arXiv:1512.00066, 2015. [36] G. L. Steele and R. P. Gabriel. The evolution of Lisp. In History of Programming Languages Conference (HOPL-II), pages 231–270, 1993. [37] G. J. Sussman and J. Wisdom. Functional differential geometry. MIT Press, 2013. [38] V. Toth. Tensor manipulation in GPL Maxima. Preprint at cs/0503073, 2005. Symbolical Index Reduction and Completion Rules for Importing Tensor Index Notation 13

[39] N. Vasilache, O. Zinenko, T. Theodoridis, P. Goyal, Z. DeVito, W. S. Moses, S. Verdoolaege, A. Adams, and A. Cohen. Tensor comprehensions: Framework-agnostic high-performance machine learning abstrac- tions. Preprint at arXiv:1802.04730, 2018.

Satoshi Egi (S. Egi) Rakuten Institute of Technology, Rakuten, Inc., Setagayaku, Tokyo 158-0094, Japan e-mail: [email protected]