See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/277587946

Towards Green Multi-frontal Solver for Adaptive

Article in Procedia Computer Science · June 2015 DOI: 10.1016/j.procs.2015.05.240

CITATION READS 1 29

6 authors, including:

Konrad Jopek Pawel Gepner AGH University of Science and Technology in … Intel

13 PUBLICATIONS 37 CITATIONS 42 PUBLICATIONS 288 CITATIONS

SEE PROFILE SEE PROFILE

Jacek Kitowski Maciej Paszynski AGH University of Science and Technology in … AGH University of Science and Technology in …

228 PUBLICATIONS 851 CITATIONS 178 PUBLICATIONS 1,282 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Solver performance analysis and optimization View project

Linear computational cost and memory usage hybrid solvers for problems of electromagnetic waves propagation over the human head model View project

All content following this page was uploaded by Maciej Paszynski on 02 June 2015.

The user has requested enhancement of the downloaded file. Procedia Computer Science

Volume 51, 2015, Pages 984–993

ICCS 2015 International Conference On Computational Science

Towards green multi-frontal solver for adaptive finite element method

H. AbbouEisha1,M.Moshkov1,K.Jopek2,P.Gepner3, J. Kitowski2,andM. Paszy´nski2

1 King Abdullah University of Science and Technology, Thuwal, Saudi Arabia 2 AGH University of Science and Technology, Krakow, Poland 3 Intel Corporation, Swindon, United Kingdom

Abstract In this paper we present the optimization of the energy consumption for the multi-frontal solver algorithm executed over two dimensional grids with point singularities. The multi-frontal solver algorithm is controlled by so-called elimination tree, defining the order of elimination of rows from particular frontal matrices, as well as order of memory transfers for Schur complement matrices. For a given mesh there are many possible elimination trees resulting in different number of floating point operations (FLOPs) of the solver or different amount of data trans- ferred via memory transfers. In this paper we utilize the dynamic programming optimization procedure and we compare elimination trees optimized with respect to FLOPs with elimination trees optimized with respect to energy consumption. Keywords: multi-frontal direct solver, adaptive finite element method, green computing

1 Introduction

The problem of optimization of energy consumption during large scale scientific computations is of the great importance to the scientific community. There are some works addressing the optimization of the computational environments to reduce the energy consumed during the scientific computations [20, 27]. Some recent papers address also the energy consumption for different implementations of the computational algorithms. In particular it is emphasized that memory access is more energy consumming than FLOPs [16, 30]. In this paper we focus on the optimization of energy consumption during the Finite Element Method (FEM) computations. FEM is a widely used approach [22, 35, 15, 19] to find approximate solutions for partial differential equations (PDEs). Those PDEs are specified along with boundary conditions and the domain of the solution. First, a mesh of rectangular or triangular elements is used to cover the specified domain and approximates the solution over it. Second, the PDEs are transformed into so-called weak form and discretized using basis of polynomial functions spread over finite elements vertices, edges and interiors. Afterwards, we specify a linear combination of a set

984 Selection and peer-review under responsibility of the Scientific Programme Committee of ICCS 2015 c The Authors. Published by Elsevier B.V. doi: 10.1016/j.procs.2015.05.240 Towards green multi-frontal solver for adaptive finite element method H. AbbouEisha, et. al. of basis functions to approximate the solution of the discretized PDEs. Finally, we can use a solver that finds the coefficients of the linear combination of the basis functions to obtain an approximate solution of the input PDE. As the result of the discretization of the PDE over the computational mesh, a global system of linear equations is generated. The system is spread over several frontal matrices, one frontal matrix per one finite element over the mesh. The rows and columns in these matrices correspond to basis functions associated with element vertices, edges and interiors. A frontal matrix over a single finite element contains rows and columns corresponding to basis functions associated with this element vertices, edges and interior. The global matrix can be obtained by merging these frontal matrices according to common mesh nodes. However the solver algorithm presented in this paper works with the global matrix distributed over particular finite elements and performs partial factorizations followed by merging of pairs of frontal matrices. The pattern how frontal matrices are factorized and merged is prescribed in the elimination tree, with leaves of the tree associated with frontal matrices generated over particular mesh nodes. The goal of the solution process is to solve the considered PDE in its discretized form over the geometry represented by the mesh. The computational mesh can be further refined, by breaking selected finite elements into smaller one. In the case of two dimensional regular meshes, where initial elements cover blocks with different parameters of the PDE, we can distinguish point, edge, and mixed singularities. In these cases mesh refinements are performed towards one or several points, one or several lines located at the border of elements, or towards mixture of such objects. In this paper we focus on multi-frontal direct solver algorithm [17, 18], the state-of-the-art solver for obtaining the exact solution of a linear systems generated by FEM. The multi- frontal solver algorithm constructs many frontal matrices, usually associated with particular finite elements. It eliminates fully assembled rows in these matrices, and merges resulting sub-matrices (called Schur complements) into new frontal matrices. This process is repeated recursively until all the rows are fully assembled and can be eliminated. Elimination trees define the order in which the solver manage the process of eliminations. Number of floating point operations (FLOPs), memory usage, memory transfers and the energy consumption of the solver depend essentially on the structure of the elimination tree. The problems of construction of the optimal elimination trees, with respect to either FLOPs, memory usage, memory transfers or energy consumption, or a combination of the above factors, are very complicated. There are some problems related to the optimization of the elimination trees which are NP complete [32, 33]. There are several heuristic algorithms utilized for construction of the elimination tree optimal with respect to the number of FLOPs. The most important heuristic algorithms are available with MUMPS multi-frontal direct solver [5, 6, 7, 28]. These are the nested-dissections [23], implemented in METIS library [24], minimum degree algorithms [21], approximate minimum degree [3, 4, 14], quasi-approximate minimum degree [12, 13] or PORD algorithm [31]. There are several variations of these algorithms like approximate minimum degree AMD etc. [3, 4]. It is well known fact that the nested-dissections algorithm provides elimination tree delivering optimal number of FLOPs for uniform grids [26]. In our previous work [1, 2] we proposed a dynamic programming algorithm finding alterna- tive elimination trees for adapted non-uniform grids, refined towards point or edge singularities. We considered the optimization with respect to FLOPs number. In this paper we perform an automatic search for alternative elimination trees providing optimal energy consumption of the multi-frontal solver algorithm for the class of non-uniform refined meshes. We compare our optimal elimination trees to the one delivering optimal FLOPs number.

985 Towards green multi-frontal solver for adaptive finite element method H. AbbouEisha, et. al.

2 Finite Element Method, Multi-Frontal Direct Solvers and Elimination Trees

In this section we introduce the state-of-the-art multi-frontal solver algorithm on the simple example of four finite element mesh with second order polynomial basis functions spread over finite elements.

Figure 1: Exemplary two finite element mesh and basis functions spread over the mesh

The FEM [15] starts with two dimensional boundary-value elliptic PDE transformed into so-called weak (variational) formulation of the form (1): Find u ∈ V such that b (u, v)=l (v) ∀v ∈ V (1) where b (u, v)andl (v) are some problem dependent bilinear and linear functionals, and  2 2 V = {v : v + ∇v dx < ∞,tr(v)=0onΓD} (2) Ω is the functional Sobolev space over an open set Ω called the domain, and ΓD is the part of boundary of Ω where Dirichlet boundary conditions are defined. For a given domain Ω the hp-FEM consists in constructing a finite dimensional subspace V ⊂ V {ei } V hp with a finite dimensional polynomial basis hp i=1,...,Nhp . The subspace hp is constructed by partitioning the domain Ω into so-called finite elements. In this paper we restrict our considerations to rectangular elements in two dimensions or hexahedral elements in three dimensions. The basis functions are constructed by gluing together the so-called shape functions constructed over vertices, edges, and interiors of finite elements in two dimensions, or over vertices, edges, faces, and interiors of finite elements in three dimensions. Figure 1 presents an exemplary two-dimensional mesh consisting of rectangular finite ele- ments with vertices, edges and interiors, as well as shape functions defined over vertices, edges and interiors of rectangular finite elements of the mesh. We introduce four shape functions over the four vertices of the two dimensional rectangular element, pi − 1 shape functions over each of the four edges of the element, and (ph −1)×(pv −1) shape functions over an element interior [15]. These basis functions are utilized to approximate a solution of the weak form of the PDE being solved.

986 Towards green multi-frontal solver for adaptive finite element method H. AbbouEisha, et. al.

In the example presented in Figure 1 there are first order polynomial basis functions associ- ated with element vertices, second order polynomials associated with element edges, and second order polynomials in both directions, associated with element interiors. The Figure presents only selected basis functions, associated to vertices number 1, 3, 5, 11, 13 and 15, edges 2, 4, 8, 12, 14 and 18 as well as all interiors. For more details we refer to [15].

2.1 Multi Frontal Solver Algorithm and Elimination Trees In this section we present an overview of the multi-frontal solver algorithm executed over the exemplary mesh presented in the previous section. Among several possible elimination trees we select one exemplary presented in Figure 2. It has to be emphasized, that this is only one possible elimination tree for the mesh from Figure 1, and this elimination tree has been selected here just for the ilustration of the solver algorithm and cost function definition. The root of the elimination tree is the entire mesh, the son nodes contains recursive partitions of the mesh, and the leaves contain particular finite elements. Figure 3 presents the processing of the first branch of the multi-frontal solver algorithm associated with the top two elements, followed by processing of the first branch of the multi- frontal solver algorithm associated with the bottom two elements, followed by the solution of the top interface problem. In this case the solver starts with generating element frontal matrices for both elements (panels (a), (b)). The rows and columns in these matrices contain basis functions defined over the mesh. The matrix entries corresponds to integrals of the multiplications of the basis functions, which are PDEs dependent. The solver localizes all the rows of the frontal matrices that are fully assembled. In this case, the interior node is fully assembled, since the 7 9 basis function associated with the interior nodes 7 and 9 (compare functions ehp and ehp in Figure 1) have support defined only inside the corresponding single element. Additionally, the rows corresponding to boundary basis functions related with nodes 1, 2, 6, and 4, 5, 10 are fully assembled in the first or second frontal matrix, respectively. This is because their supports are also located only inside the corresponding single element. However, we must keep all the nodes located on the interface with the two bottom elements. All these rows can be eliminated from the matrix (panels (a), (b)), and what remains are the rows associated with the basis functions 3, 8 located on the common edge and 11, 12, 13, 14, 15 located on the interface with the bottom two elements. The multi-frontal solver merges the resulting sub-matrices (panel (c) reorders them to eliminate first two rows (panel (d)). The same procedure is repeated for the bottom two elements (panels (e)-(h)). Finally, the interface problem associated to nodes 11, 12, 13, 14 and 15 is formulated and solved (panels (i) and (j)).

3 Cost functions

We can defne several cost functions measuring performance of the multi-frontal solver algorithm based on the elimination tree. The first cost function is the one measuring the FLOPs, as reported in [2]. The functions estimates the number of FLOPs during particular partial forward eliminations, performed in steps (a), (b), (e), (f) and (j) as presented in Figure 3. The meaning of these steps have been described in section 2.1. They correspond to processing of the elimination tree presented in Figure 2. The number of FLOPs is estimated by using the formula (3)

a(6b2 − 6ab +6b +2a2 − 3a +1) FLOPs(a, b)= , (3) 6

987 Towards green multi-frontal solver for adaptive finite element method H. AbbouEisha, et. al.

Figure 2: Multi-frontal solver algorithm processing the first branch of the elimination tree relatedtothetwofirstelements.

Figure 3: Multi-frontal solver algorithm processing the first branch of the elimination tree related to the top first elements (panels (a)-(d)), followed by processing the second branch of the elimination tree related to the bottom first elements (panels (e)-(h)), followed by the solution of the common interface problem (panels (i)-(j)). where a denotes the number of rows to be eliminated, and b the local size of the frontal matrix, compare [1]. In our example in steps (a), (b), (e) and (f) b = 9 and a = 4, while is step (i) a = b =5 In this paper we propose a new cost function, measuring the memory transfers necessary to form a new frontal matrices out of the obtained Schur complements. The number of memory

988 Towards green multi-frontal solver for adaptive finite element method H. AbbouEisha, et. al.

Table 1: FLOPs and memory transfers on a sample four element domain for multi-frontal solver

Step abFLOPs(a, b)StepabMEM(a, b) 149374 14 9 25 249374 24 9 25 327133 32 7 25 449374 44 9 25 549374 54 9 25 627133 62 7 25 75570 75 5 0 TOTAL FLOPs 1832 TOTAL memory transfers 150

transfers is given by (4) 2 MEM(a, b)=(b − a) . (4) This is because the size of the Schur complement is equal to the size of the local system minus the size of the rows that can be eliminated in the local system. The Schur complements are transfered then to the parent level where they are merged into a new local system. Using these formula, we can directly compute the costs for the sample 2D domain presented in the previous sections, compare Table 1. The FLOPs considered above concerns only the factorization step, they do not consider the backward/forward substitutions (the so called solution phase). This is because the factorization step is the most expensive one and the cost of the solution phase can be neglected when we optimize the elimination tree for the multi-frontal solver algorithm.

4 Optimization algorithm

We utilize the same optimization algorithm as the one presented in [1, 2], however with the cost function updated to reflect both FLOPs and memory transfers. Additionally, we assume that energy consumption for the memory transfers is around 100 time higher than for FLOPs. It is of course a simplified assumption, and it assumes order of magnitude and does not include specific architecture design. However 96 % of processors is based on Intel, include the Xeon. The differences of sizes of L1, L2 and L3 caches is not taken into account. The architecture of memory is also not taken into accout, however for both DDR3, and IMC for Xeon the architectures are similar. Taking all that into account our estimation is correct. For intel processor we have around 10-20 pJ per FLOP, and 1000 pJ per Memory Access (DRAM Access). For a Xeon phy we have L1 Hit=39pJ, L2/L3 Hit=389 pJ, and DRAM Access=1320 pJ. Based on these data we can assume that FLOP/Memory Access ratio is around 100. In our simple example the energy consumption cost is equal ENERGY = 1832+150=∗100 16832.

5 Numerical results

The presented optimization algorithm has been executed on two dimensional computational grids with point and edge singularities. The computations has been performed on 1 core of

989 Towards green multi-frontal solver for adaptive finite element method H. AbbouEisha, et. al.

Intel(R) Xeon(R) CPU E7- 4860 @ 2.27GHz, with DDR3, 1066 MHz according to [36]. We compare the execution times for the elimination trees optimized with respect to FLOPs to the trees optimized with respect to energy consumption. The elimination trees obtained from our optimization procedures have been submitted to our GALOIS solver [29], and the resulting execution times has been measured. The comparison is presented in Figure 4 for the mesh with edge and point singularity. We have also performed the curve fitting algorithm to estimate the exponent factor of the trends. We can draw the following conclusions from the presented experiments:   1.6831 • The energy optimal elimination tree for mesh with edge singularity results in O N execution time, while the FLOPs optimal elimination tree results in O N 0.963 execution time. In other words the price to pay for the green solver is the lost of linear computational cost of the solver.

• The energy optimal elimination tree for mesh with point singularity is practically identical with the mesh optimizing the execution time.

Figure 4: Execution time of FLOPs vs ENERGY optimal solvers for 2D mesh with edge sin- gularity (left panel) and point singularity (right panel). Both plots uses log-log scales on both axes.

Let us compare the elimination trees for the case of edge singularity. The tree optimizing FLOPs, following the result presented in [1] is presented on left panel in Figure 5. The new tree optimizing the energy is presented on right panel in Figure 5. The tree optimizing energy prefers longer slices of elements. Let us also compare the elimination trees for the case of point singularity. The tree opti- mizing FLOPs, following the result presented in [1] is presented on left panel in Figure 6. The new tree optimizing the energy is presented on right in Figure 6. The trees for point singularity are practically identical, and they differ only by order of cutting the levels. Note that the opti- mization of FLOPs does not necessary leads to faster execution times, since memory transfers also take up to two orders of magnitude more time as the FLOPs.

6 Conclusions

In this paper we compared elimination trees controlling the execution of the multi-frontal di- rect solver algorithm over refined grids, optimized with respect to the number of floating point operations (FLOPs) as well as energy consumption. As a criterion for energy consumption

990 Towards green multi-frontal solver for adaptive finite element method H. AbbouEisha, et. al.

Figure 5: Left panel: Elimination tree optimizing the FLOPs for mesh with edge singularity. Right panel: Elimination tree optimizing the energy for mesh with edge singularity

Figure 6: Left panel: Elimination tree optimizing the FLOPs for mesh with point singularity. Right panel: Elimination tree optimizing the energy for mesh with point singularity. cost we assumed that FLOPs results in 100 less energy consumption than memory transfers. The execution time of both FLOPs optimized and energy optimized solvers have been com- pared on two dimensional girds with point and edge singularity. We conclude that optimizing the solver wih respect to energy consumptions results in slight increase of the execution time. The optimization of the energy consumption over model refined 2D grids may be generalized to more complex refined grids by decomposition of the grid into subgrids with either point or edge singularity, and building standard nested-dissection tree on top of it. Additionally, for non-linear or non-stationary problems the energy optimization may be performed in every iteration. Future work may include the extension of the optimization algorithm to different

991 Towards green multi-frontal solver for adaptive finite element method H. AbbouEisha, et. al. grids, including 3D grids with point, edge, face and mixed singularities. We may also consider different implementations of the solver algorithm, including different memory transfers, and test them on grids with singularities as well as on the uniform grids, and for transient problems [10]. Another topic is the optimization of energy during numerical integration [34, 11, 25].

Acknowledgement The work presented in this paper has been supported by Polish National Science Center grant no. DEC-2012/07/B/ST6/01229 and by King Abdullah University of Science and Technology (KAUST). The work of Konrad Jopek has been supported by Dean’s grant.

References

[1] AbouEisha, H. and Gurgul, P. and Paszy´nska, A. and Paszy´nski, M. and Ku´znik,K. and Moshkov, M. An automatic way of finding robust elimination trees for a multi-frontal sparse solver for radical 2D hierarchical meshes, Lecture Notes in Computer Science 8365 (2013) 531-540 [2] AbouEisha, H. and Moshkov, M. and Calo, V. and Paszy´nski, M. and Goik, D. and Jopek, K. Dynamic programming algorithm for generation of optimal elimination trees for multi-frontal direct solver over h-refined grids, Procedia Computer Science 29 (2014) 947-959 [3] Amestoy, P. R. and Davis, T. A. and Du, I. S. An Approximate Minimum Degree Ordering Algo- rithm, SIAM Journal of Matrix Analysis & Application, 17, 4 (1996) 886-905. [4] Amestoy P.R., Dollar H.S., Reid J.K., Scott J.A. An approximate minimum degree algorithm for matrices with dense rows. Technical Report RAL-TR-2007-020, Rutherford Appleton Laboratory, (2007) [5] Amestoy P.R. and Duff, I.S. Multifrontal parallel distributed symmetric and unsymmetric solvers, Computer Methods in Applied Mechanics and Engineering, (2000) 184, 501-520. [6] Amestoy, P. R. and Duff, I. S. and Koster, J. and L’Excellent, J.Y., A fully asynchronous multi- frontal solver using distributed dynamic scheduling, SIAM Journal of Matrix Analysis and Applica- tions, (2001) 1(23), 15-41. [7] Amestoy, P. R. and Guermouche, A. and L’Excellent, J.-Y. and Pralet, S., Hybrid scheduling for the parallel solution of linear systems, Computer Methods in Applied Mechanics and Engineering, (2001) 2(32), 136-156. [8] Bana´s, K. and Plaszewski, P. and Maciol, P., Numerical integration on GPUs for higher order finite elements, Computers and mathematics with applications (2014) 67(6) 1319-1344. [9] Bana´s, K. and Plaszewski, P. and Maciol, P., Numerical integration on GPUs for higher order finite elements, Computers and mathematics with applications (2014) 67(6) 1319-1344 [10] Bana´s, K. and Demkowicz, L., Entropy controlled adaptive finite element simulations for com- pressible gas flow, Journal of computational physics (1996) 126(1) 181-201. [11] Bana´s, K. and Plaszewski, P. and Maciol, P., Numerical integration on GPUs for higher order finite elements, Computers and mathematics with applications (2014) 67(6) 1319-1344 [12] Carmen J. L. An AMD preprocessor for matrices with some dense rows and columns. http://www.netlib.org/linalg/amd/amdpre.ps. [13] Davis T.A. Direct Methods for Sparse Linear Systems, Fundamentals of Algorithms, Volume 2, SIAM: Philadelphia, PA, (2006) [14] Flake, G.W. and Tarjan, R.E. and Tsioutsiouliklis, K. Graph clustering and minimum cut trees, Internet Mathematics 1 (2003), 385-408. [15] Demkowicz, L. Computing with hp-Adaptive Finite Elements, Vol. I. One and Two Dimensional Elliptic and Maxwell Problems, Chapman and Hall/Crc Applied Mathematics and Nonlinear Science

992 Towards green multi-frontal solver for adaptive finite element method H. AbbouEisha, et. al.

(2006) [16] Demmel, J. Lower Bounds on Algorithm Energy Consumption: Current Work and Future Di- rections, SIAM Conference on Computational Science & Engineering, Challenges of Energy-aware Scientific Computing, February 25- March 1, The Westin Boston Waterfront, Boston, Masachusetts, USA (2013) [17] Duff, I. S. and Reid, J. K. The multifrontal solution of indefinite sparse symmetric linear systems, ACM Transactions on Mathematical Software (1983) 9, 302-325. [18] Duff, I. S. and Reid, J. K. The multifrontal solution of unsymmetric sets of linear systems, SIAM Journal on Scientific and Statistical Computing, (1984) 5, 633-641. [19] Gurgul, P. and Sieniek, M. and Paszy´nski,M. and Madej, L. and Collier N., Two-dimensional hp- adaptive algorithm for continuous approximations of material data using space projection, Computer Science, 14(1), (2013) 97-112 [20] Gienger, M. Towards Energy aware Scheduling between Federated Data Centres, Proceedins of e-Challenges, 29-30 October, Belfast (2014) [21] Heggernes, P. and Eisenstat, S.C. and Kumfert, G. and Pothen, A. The Computational Complexity of the Minimum Degree Algorithm, ICASE Report No. 2001-42 (2001). [22] Hughes, T. J. R. The Finite Element Method. Linear Statics and Dynamics Finite Element Method Analysis, Dover (2000). [23] Karypis, G. and Kumar, V. A fast and high quality multilevel scheme for partitioning irregular graphs, Siam Journal of Scientific Computing (1998) 20(1) 359-392. [24] Karypis, G. and Kumar, V. METIS - Unstructured Graph Partitioning and Ordering System, Version 2.0, Technical Report, (1995) [25] Kruzel, F. and Bana´s,K., Vectorized OpenCL implementation of numerical integration for higher order finite elements, Computers and mathematics with applications (2013) 66(10) 2030-2044. [26] Liu, J.W.H. The multifrontal method for sparse matrix solution: theory and practice, SIAM Review 34 (1992), 82-109. [27] Luszczek, P. Energy and Power Consumption Trends im HPC, ICL Lunch, Sept. 20, Knoxville, TN (2013). [28] MUlti-frontal Massively Parallel Sparse direct solver (MUMPS) http://mumps.enseeiht.fr/ [29] Paszy´nska, A. and Paszy´nski, M. and Jopek, K. and Wo´zniak,M. and Goik, D. and Gurgul, P. and AbouEisha, H. and Moshkov, M. and Calo, V. and Lenharth, A. and Nguyen, D. and Pin- gali, K. Quasi-Optimal Elimination Trees for 2D Grids with singularities, accepted to Scientiffic Programming (2014). [30] Tiwari, A., and Laurenzano, M. A. and Carrington, L. Snavely A. Modeling Power and Energy Usage of HPC Kernels, Parallel and Distributed Processing Symposium Workshops & PhD Forum (IPDPSW), IEEE 26th International, 21-25 May (2012) 990 - 998. [31] Schulze, J. Towards a tighter coupling of bottom-up and top-down sparse matrix ordering methods, BIT, 41, 4 (2001) 800. [32] Yannakakis, M. Computing the minimum fill-in is NP-complete, SIAM Journal on Algebraic Dis- crete Methods (1981) 2, 77–79. [33] Luce, R. and Esmond, Ng. On the minimum FLOPs problem in the sparse Cholesky factorization, SIAM Journal of Matrix Analysis & Applications 35-1 (2014) 1-21. [34] Wo´zniak, M., Fast GPU integration algorithm for isogeometric finite element method solvers using task dependency graphs, Journal of Computational Science, (2015) in press. corrected proofs, doi:10.1016/j.jocs.2015.02.007 [35] Zienkiewicz, O. C. and Taylor, R. L. and Ziu, J. Z., The Finite Element Method: Its Basis and Fundamentals, Elsevier, 7th edition (2013). [36] http://ark.intel.com/products/53571/Intel-Xeon-Processor-E7-4860-24M-Cache-2 26-GHz-6 40- GTs-Intel-QPI

993

View publication stats