Nonlinear Preconditioning: How to Use a Nonlinear Schwarz Method to Precondition Newton’S Method

NONLINEAR PRECONDITIONING: HOW TO USE A NONLINEAR SCHWARZ METHOD TO PRECONDITION NEWTON’S METHOD

V. DOLEAN∗, M.J. GANDER† , W. KHERIJI‡ , F. KWOK§ , AND R. MASSON¶

Abstract. For linear problems, domain decomposition methods can be used directly as iterative solvers, but also as preconditioners for Krylov methods. In practice, Krylov acceleration is almost always used, since the Krylov method finds a much better residual polynomial than the stationary iteration, and thus converges much faster. We show in this paper that also for non-linear problems, domain decomposition methods can either be used directly as iterative solvers, or one can use them as preconditioners for Newton’s method. For the concrete case of the parallel Schwarz method, we show that we obtain a preconditioner we call RASPEN (Restricted Additive Schwarz Preconditioned Exact Newton) which is similar to ASPIN (Additive Schwarz Preconditioned Inexact Newton), but with all components directly defined by the iterative method. This has the advantage that RASPEN already converges when used as an iterative solver, in contrast to ASPIN, and we thus get a substantially better preconditioner for Newton’s method. The iterative construction also allows us to naturally define a coarse correction using the multigrid full approximation scheme, which leads to a convergent two level non-linear iterative domain decomposition method and a two level RASPEN non-linear preconditioner. We illustrate our findings with numerical results on the Forchheimer equation and a non-linear diffusion problem.

Key words. Non-Linear Preconditioning, Two-Level Non-Linear Schwarz Methods, Precondi- tioning Newton’s Method

AMS subject classiﬁcations. 65M55, 65F10, 65N22

1. Introduction. Non-linear partial differential equations are usually solved after discretization by Newton’s method or variants thereof. While Newton’s method converges well from an initial guess close to the solution, its convergence behaviour can be erratic and the method can lose all its effectiveness if the initial guess is too far from the solution. Instead of using Newton, one can use a domain decomposition iteration, applied directly to the non-linear partial differential equations, and one obtains then much smaller subdomain problems, which are often easier to solve by Newton’s method than the global problem. The first analysis of an extension of the classical alternating Schwarz method to non-linear monotone problems can be found in [27], where a convergence proof is given at the continuous level for a minimization formulation of the problem. A two-level parallel additive Schwarz method for non-linear problems was proposed and analyzed in [12], where the authors prove that the non- linear iteration converges locally at the same rate as the linear iteration applied to the linearized equations about the fixed point, and also a global convergence result is given in the case of a minimization formulation under certain conditions. In [28], the classical alternating Schwarz method is studied at the continuous level, when applied

arXiv:1605.04419v1 [math.NA] 14 May 2016 to a Poisson equation whose right hand side can depend non-linearly on the function and its gradient. The analysis is based on ﬁxed point arguments; in addition, the

∗Department of Maths and Stats, University of Strathclyde, Glasgow, United Kingdom [email protected] †Section de Mathématiques, Université de Genève, CP 64, 1211 Genève, Switzerland, [email protected] ‡Laboratoire J.-A. Dieudonné,UniversitéNice Sophia Antipolis and project Coffee Inria Sophia Antipolis Méditerranée,France, [email protected] §Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong, felix [email protected] ¶Laboratoire J.-A. Dieudonné,UniversitéNice Sophia Antipolis and project Coffee Inria Sophia Antipolis Méditerranée,France, [email protected] 1 2 author also analyzes linearized variants of the iteration in which the non-linear terms are relaxed to the previous iteration. A continuation of this study can be found in [29], where techniques of super- and sub-solutions are used. Results for more general subspace decomposition methods for linear and non-linear problems can be found in [34, 32]. More recently, there have also been studies of so-called Schwarz waveform relaxation methods applied directly to non-linear problems: see [19, 21, 11], where also the techniques of super- and sub-solutions are used to analyze convergence, and [24, 4] for optimized variants.

Another way of using domain decomposition methods to solve non-linear problems is to apply them within the Newton iteration in order to solve the linearized problems in parallel. This leads to the Newton-Krylov-Schwarz methods [7, 6], see also [5]. We are however interested in a different way of using Newton’s method here. For linear problems, subdomain iterations are usually not used by themselves; instead, the equation at the fixed point is solved by a Krylov method, which greatly reduces the number of iterations needed for convergence. This can also be done for non- linear problems: suppose we want to solve F (u) = 0 using the fixed point iteration un+1 = G(un). To accelerate convergence, we can use Newton’s method to solve F(u) := G(u) − u = 0 instead. We first show in Section 2 how this can be done for a classical parallel Schwarz method applied to a non-linear partial differential equation, both with and without coarse grid, which leads to a non-linear preconditioner we call RASPEN. With our approach, one can obtain in a systematic fashion nonlinear preconditioners for Newton’s method from any domain decomposition method. A different non-linear preconditioner called ASPIN was invented about a decade ago in [8], see also the earlier conference publication [9]. Here, the authors did not think of an iterative method, but directly tried to design a non-linear two level preconditioner for Newton’s method. This is in the same spirit as some domain decomposition methods for linear problems that were directly designed to be a preconditioner; the most famous example is the additive Schwarz preconditioner [13], which does not lead to a convergent stationary iterative method without a relaxation parameter, but is very suitable as a preconditioner, see [20] for a detailed discussion. It is however difficult to design all components of such a preconditioner, in particular also the coarse correction, without the help of an iterative method in the background. We discuss in Section 3 the various differences between ASPIN and RASPEN. Our comparison shows three main advantages of RASPEN: first, the one-level preconditioner came from a convergent underlying iterative method, while ASPIN is not convergent when used as an iterative solver without relaxation; thus, we have the same advantage as in the linear case, see [14, 20]. Second, the coarse grid correction in RASPEN is based on the full approximation scheme (FAS), whereas in ASPIN, a different, ad hoc construction based on a precomputed coarse solution is used, which is only good close to the fixed point. And finally, we show that the underlying iterative method in RASPEN already provides the components needed to use the exact Jacobian, instead of an approximate one in ASPIN. These three advantages, all due to the fact that RASPEN is based on a convergent non-linear domain decomposition iteration, lead to substantially lower iteration numbers when RASPEN is used as a preconditioner for Newton’s method compared to ASPIN. We illustrate our results in Section 4 with an extensive numerical study of these methods for the Forchheimer equation and a non-linear diffusion problem. 3

2. Main Ideas for a Simple Problem. To explain the main ideas, we start with a one dimensional non-linear model problem

L(u) = f, in Ω := (0,L), u(0) = 0, (2.1) u(L) = 0,

2 where for example L(u) = −∂x((1 + u )∂xu). One can apply a classical parallel Schwarz method to solve such problems. Using for example the two subdomains Ω1 := (0, β) and Ω2 := (α, L), α < β, the classical parallel Schwarz method is

n L(u1 ) = f, in Ω1 := (0, β), n u1 (0) = 0, n n−1 u1 (β) = u2 (β), n (2.2) L(u2 ) = f, in Ω2 := (α, L), n n−1 u2 (α) = u1 (α), n u2 (L) = 0. This method only gives a sequence of approximate solutions per subdomain, and it is convenient to introduce a global approximate solution, which can be done by glueing the approximate solutions together. A simple way to do so is to select values from one of the subdomain solutions by resorting to a non-overlapping decomposition,

n α+β n u1 (x) if 0 ≤ x < 2 , u (x) := n α+β (2.3) u2 (x) if 2 ≤ x ≤ L,

˜T which induces two extension operators Pei (often called Ri in the context of RAS); n n n we can write u = Pe1u1 + Pe2u2 . Like in the case of linear problems, where one usually accelerates the Schwarz method, which is a ﬁxed point iteration, using a Krylov method, we can accelerate the non-linear ﬁxed point iteration (2.2) using Newton’s method. To do so, we introduce two solution operators for the non-linear subdomain problems in (2.2),

n n−1 n n−1 u1 = G1(u ), u2 = G2(u ), (2.4) with which the classical parallel Schwarz method (2.2) can now be written in compact form, even for many subdomains i = 1, ··· ,I, as

I n X n−1 n−1 u = PeiGi(u ) =: G1(u ). (2.5) i=1 As shown in the introduction, this ﬁxed point iteration can be used as a preconditioner for Newton’s method, which means to apply Newton’s method to the non-linear equation

I ˜ X F1(u) := G1(u) − u = PeiGi(u) − u = 0, (2.6) i=1 because it is this equation that holds at the ﬁxed point of iteration (2.5). We call this method one level RASPEN (Restricted Additive Schwarz Preconditioned Exact Newton). We show in Figure 2.1 as an example the residual of the nonlinear RAS 4

2 0

0 −2

−2 −4

−4 −6

−6 −8 −8 −10 −10

−12 −12

−14 −14

−16 −16

−18 −18 00.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 00.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Fig. 2.1: Illustration of the residual when RAS is used as a nonlinear solver (left), or as a preconditioner for Newton’s method (right).

iterations and using RASPEN as a preconditioner for Newton when solving the Forch- heimer equation with 8 subdomains from the numerical section. We observe that the residual of the non-linear RAS method is concentrated at the interfaces, since it must be zero inside the subdomains by construction. Thus, when Newton’s method is used to solve (2.6), it only needs to concentrate on reducing the residual on a small number of interface variables. This explains the fast convergence of RASPEN shown on the right of Figure 2.1, despite the slow convergence of the underlying RAS iteration. Suppose we also want to include a coarse grid correction step in the Schwarz iteration (2.2), or equivalently in (2.5). Since the problem is non-linear, we need to use the Full Approximation Scheme (FAS) from multigrid to do so, see for example [3, 26]: given an approximate solution un−1, we compute the correction c by solving the non-linear coarse problem

c n−1 c n−1 n−1 L (R0u + c) = L (R0u ) + R˜0(f − L(u )), (2.7)

c where L is a coarse approximation of the non-linear problem (2.1) and R0 is a re- n−1 striction operator. This correction c := C0(u ) is then added to the iterate to get the new corrected value

n−1 n−1 n−1 unew = u + P0C0(u ), (2.8) where P0 is a suitable prolongation operator. Introducing this new approximation from (2.8) at step n − 1 into the subdomain iteration formula (2.5), we obtain the method with integrated coarse correction

I n X n−1 n−1 n−1 u = PeiGi(u + P0C0(u )) =: G2(u ). (2.9) i=1 This stationary ﬁxed point iteration can also be accelerated using Newton’s method: we can use Newton to solve the non-linear equation

I ˜ X F2(u) := G2(u) − u = PeiGi(u + P0C0(u)) − u = 0. (2.10) i=1 We call this method two level FAS-RASPEN. 5

We have written the coarse step as a correction, but not the subdomain steps. This can however also be done, by simply rewriting (2.5) to add and subtract the previous iterate, I I n n−1 X n−1 n−1 n−1 X n−1 u = u + Pei (Gi(u ) − Riu ) = u + PeiCi(u ), (2.11) i=1 | {z } i=1 n−1 =:Ci(u ) P ˜ where we have assumed that i PiRi = IV , the identity on the vector space, see Assumption 1 in the next section. Together with the coarse grid correction (2.8), this iteration then becomes I n n−1 n−1 X n−1 n−1 u = u + P0C0(u ) + PeiCi(u + P0C0(u )), (2.12) i=1 which can be accelerated by solving with Newton the equation

I ˜ X F2(u) := P0C0(u) + PeiCi(u + P0C0(u)) = 0. (2.13) i=1

This is equivalent to F˜2(u) = 0 from (2.10), only written in correction form. 3. Definition of RASPEN and Comparison with ASPIN. We now define formally the one- and two-level versions of the RASPEN method and compare it with the respective ASPIN methods. We consider a non-linear function F : V → V 0, where V is a Hilbert space, and the non-linear problem of finding u ∈ V such that F (u) = 0. (3.1)

Let Vi, i = 1,...,I be Hilbert spaces, which would generally be subspaces of V . We consider for all i = 1,...,I the linear restriction and prolongation operators Ri : V → Vi, Pi : Vi → V , as well as the “restricted” prolongation Pei : Vi → V .

Assumption 1. We assume that Ri and Pi satisfy for i = 1,...,I

RiPi = IVi , the identity on Vi, I X and that Ri and Pei satisfy PeiRi = IV . i=1 These are all the assumptions we need in what follows, but it is helpful to think of the restriction operators Ri as classical selection matrices which pick unknowns T corresponding to the subdomains Ωi, of the prolongations Pi as Ri , and of the Pei as extensions based on a non-overlapping decomposition. 3.1. One- and two-level RASPEN. We can now formulate precisely the RASPEN method from the previous section: we deﬁne the local inverse Gi : V → Vi to be solutions of

RiF (PiGi(u) + (I − PiRi)u) = 0. (3.2) In the usual PDE framework, this corresponds to solving locally on the subdomain i the PDE problem on Vi with Dirichlet boundary condition given by u outside of the subdomain i, see (2.4). Then, one level RASPEN solves the non-linear equation

I ˜ X F1(u) = PeiGi(u) − u = 0, (3.3) i=1 6 using Newton’s method, see (2.6). The preconditioned nonlinear function (3.3) corresponds to the ﬁxed point iteration I n X n−1 u = PeiGi(u ), (3.4) i=1 see (2.5). Equivalently, the RASPEN equation (3.3) can be written in correction form as I I ˜ X X F1(u) = Pei(Gi(u) − Riu) =: PeiCi(u), (3.5) i=1 i=1 where we deﬁne the corrections Ci(u) := Gi(u)−Riu. This way, the subdomain solves (3.2) can be written in terms of Ci(u) as

RiF (u + PiCi(u)) = 0. (3.6) In the special case where F (u) = Au − b is aﬃne, (3.6) reduces to

−1 RiA(u + PiCi(u)) − Rib = 0 =⇒ Ci(u) = Ai Ri(b − Au), where Ai = RiAPi is the subdomain matrix. This implies

I ˜ X −1 F1(u) = PeiAi Ri(b − Au), i=1 and we immediately see that the Jacobian is the matrix A preconditioned by the PI −1 restricted additive Schwarz (RAS) preconditioner i=1 PeiAi Ri. Thus, if a Krylov method is used to solve the outer system, our method is equivalent to the Krylov- accelerated one-level RAS method in the linear case. To deﬁne the two-level variant, we introduce a coarse space V0 and the linear re- 0 striction and prolongation operators R0 : V → V0, P0 : V0 → V . Let F0 : V0 → V0 be the coarse non-linear function, which could be deﬁned by using a coarse discretization of the underlying problem, or using a Galerkin approach we use here, namely

F0(u0) = Re0F (P0(u)). (3.7)

0 0 Here, Re0 : V → V0 is a projection operator that plays the same role as R0, but in the residual space. In two-level FAS-RASPEN, we use the well established non-linear coarse correction C0(u) from the full approximation scheme already shown in (2.7), which in the rigorous context of this section is deﬁned by

F0(C0(u) + R0u) = F0(R0u) − Re0F (u). (3.8) This coarse correction is used in a multiplicative fashion in RASPEN, i.e. we solve with Newton the preconditioned non-linear system

n X Fe2(u) = P0C0(u) + PeiCi(u + P0C0(u)) = 0. (3.9) i=1 This corresponds to the non-linear two-level ﬁxed point iteration

n n+1 n n X n n u = u + P0C0(u ) + PeiCi(u + P0C0(u )), i=1 7

n n with C0(u ) defined in (3.8) and Ci(u ) defined in (3.6). This iteration is convergent, as we can see in the next section in Figure 3.1 in the right column. In the special case of an affine residual function F (u) = Au − b, a simple calculation shows that

I ! −1 X −1 −1 Fe2(u) = P0A0 Re0 + PeiAi Ri(IV − P0A0 Re0) (b − Au), i=1 where we assumed that the coarse function F0 = A0u0 − b0 is also linear. Thus, in the linear case, two-level RASPEN corresponds to preconditioning by a two-level RAS preconditioner, where the coarse grid correction is applied multiplicatively. 3.2. Comparison of one-level variants. In order to compare RASPEN with the existing ASPIN method, we recall the precise definition of one-level ASPIN from [8], which gives a different reformulation F1(u) = 0 of the original equation (3.1) to be solved. In ASPIN, one also defines for u ∈ V and for all i = 1, ··· ,I the corrections as in (3.6), i.e., we define Ci(u) ∈ Vi such that

RiF (u + PiCi(u)) = 0, where PiCi(u) are called Ti in [8]. Then, the one-level ASPIN preconditioned function is deﬁned by

I X F1(u) = PiCi(u), (3.10) i=1 and the preconditioned system F1(u) = 0 is solved using a Newton algorithm with an inexact Jacobian, see Section 3.4. The ASPIN preconditioner also has a corresponding ﬁxed point iteration: adding and subtracting PiRiu in the deﬁnition (3.6) of the corrections Ci, we obtain

RiF (u + PiCi(u)) = RiF (Pi(Riu + Ci(u)) + u − PiRiu) = 0, which implies, by comparing with (3.2) and assuming existence and uniqueness of the solution to the local problems, that

Gi(u) = Riu + Ci(u).

We therefore obtain for one-level ASPIN

I I I X X X F1(u) = PiCi(u) = PiGi(u) − PiRiu, (3.11) i=1 i=1 i=1 which corresponds to the non-linear ﬁxed point iteration

I I I n n−1 X n−1 n−1 X n−1 X n−1 u = u + PiCi(u ) = u − PiRiu + PiGi(u ). (3.12) i=1 i=1 i=1 This iteration is not convergent in the overlap, already in the linear case, see [14, 20], and needs a relaxation parameter to yield convergence, see for example [12] for the non-linear case. This can be seen directly from (3.12): if an overlapping region belongs to K subdomains, then the current iterate un is subtracted K times there, 8

6 6 Newton Newton 5 One Level AS 5 One Level RAS Two level AS Two level RAS One Level ASPIN One Level RASPEN 4 Two level ASPIN 4 Two level FA−RASPEN 3 3

2 2

1 1

0 0

−1 −1

−2 −2

−3 −3

−4 −4

−5 −5

−6 −6

−7 −7

−8 −8

−9 −9

−10 −10 02 4 6810 12 14 16 18 20 22 24 02 4 6810 12 14 16 18 20 22 24 Iteration n Iteration n 6 6 Newton Newton 5 One Level AS 5 One Level RAS Two level AS Two level RAS One Level ASPIN 4 One Level RASPEN Two level ASPIN 4 Two level FA−RASPEN 3 3

2 2

1 1

0 0

−1 −1

−2 −2

−3 −3

−4 −4

−5 −5

−6 −6

−7 −7

−8 −8

−9 −9

−10 −10 020 40 60 80 100 120 140 160 180 020 40 60 80 100 120 140 160 180 Linear subdomain solves Linear subdomain solves Fig. 3.1: Error as function of non-linear iteration numbers in the top row, and as number of subdomain solves in the bottom row, for ASPIN (left), and RASPEN (right).

and then the sum of the K respective subdomain solutions are added to the result. This redundancy is avoided in our formulation (3.4). The only interest in using an additive correction in the overlap is that in the linear case, the preconditioner remains symmetric for a symmetric problem. We show in Figure 3.1 a numerical comparison of the two methods, together with Newton’s method applied directly to the non-linear problem, for the first example of the Forchheimer equation from Section 4.1 on a domain of unit size with 8 subdomains, overlap 3h, with h = 1/100. In these comparisons, we use ASPIN first as a fixed-point iterative solver (labelled AS for Additive Schwarz), and then as a preconditioner. We do the same for our new nonlinear iterative method, which in the figures are labelled RAS for Restricted Additive Schwarz. We see from this numerical experiment that ASPIN as an iterative solver (AS) does not converge, whereas RASPEN used as an iterative solver (RAS) does, both with and without coarse grid. Also note that two- level RAS is faster than Newton directly applied to the non-linear problem for small 9 iteration counts, before the superlinear convergence of Newton kicks in. The fact that RASPEN is based on a convergent iteration, but not ASPIN, has an important influence also on the Newton iterations when the methods are used as preconditioners: the ASPIN preconditioner requires more Newton iterations to converge than RASPEN does. At first sight, it might be surprising that in RASPEN, the number of Newton iterations with and without coarse grid is almost the same, while ASPIN needs more iterations without coarse grid. In contrast to the linear case with Krylov acceleration, it is not the number of Newton iterations that depends on the number of subdomains, but the number of linear inner iterations within Newton, which grows when no coarse grid is present. We show this in the second row of Figure 3.1, where now the error is plotted as a function of the maximum number of linear subdomain solves used in each Newton step, see Subsection 4.1.1. With this more realistic measure of work, we see that both RASPEN and ASPIN converge substantially better with a coarse grid, but RASPEN needs much fewer subdomain solves than ASPIN does.

3.3. Comparison of two-level variants. We now compare two-level FAS- RASPEN with the two-level ASPIN method of [30]. Recall that the two-level FAS- RASPEN consists of applying Newton’s method to (3.9),

n X Fe2(u) = P0C0(u) + PeiCi(u + P0C0(u)) = 0, i=1 where the corrections C0(u) and Ci(u) are deﬁned in (3.8) and (3.6) respectively. ∗ Unlike FAS-RASPEN, two-level ASPIN requires the solution u0 ∈ V0 to the coarse ∗ problem, i.e., F0(u0) = 0, which can be computed in a preprocessing step. A In two-level ASPIN, the coarse correction C0 : V → V0 is deﬁned by

A ∗ F0(C0 (u) + u0) = −Re0F (u), (3.13) and the associated two-level ASPIN function uses the coarse correction in an additive fashion, i.e. Newton’s method is used to solve

I A X F2(u) = P0C0 (u) + PiCi(u) = 0, (3.14) i=1

A n n with C0 (u ) defined in (3.13) and Ci(u ) defined in (3.6). This is in contrast to two-level FAS-RASPEN, where the coarse correction C0(u) is computed from the well established full approximation scheme, and is applied multiplicatively in (3.9). The fixed point iteration corresponding to (3.14) is

I n+1 n A n X n u = u + P0C0 (u ) + PiCi(u ). i=1

Just like its one-level counterpart, two-level ASPIN is not convergent as a ﬁxed-point iteration without a relaxation parameter, see Figure 3.1 in the left column. Moreover, because the coarse correction is applied additively, the overlap between the coarse space and subdomains leads to slower convergence in the Newton solver, which does not happen with FAS-RASPEN. 10

3.4. Computation of Jacobian matrices. When solving (3.5), (3.9), (3.11) and (3.14) using Newton’s method, one needs to repeatedly solve linear systems in- volving Jacobians of the above functions. If one uses a Krylov method such as GMRES to solve these linear systems, like we do in this paper, then it suffices to have a pro- cedure for multiplying the Jacobian with an arbitrary vector. In this section, we derive the Jacobian matrices for both one-level and two-level RASPEN in detail. We compare these expressions with ASPIN, which approximates the exact Jacobian with an inexact one in an attempt to reduce the computational cost, even though this can potentially slow down the convergence of Newton’s method. Finally, we show that this approximation is not necessary in RASPEN: in fact, all the components involved in building the exact Jacobian have already been computed elsewhere in the algorithm, so there is little additional cost in using the exact Jacobian compared with the approximate one. 3.4.1. Computation of the one-level Jacobian matrices. We now show how to compute the Jacobian matrices of ASPIN and RASPEN. To simplify notation, we define dF u(i) := P G (u) + (I − P R )u and J(v) := (v) (3.15) i i i i du By differentiating (3.2), we obtain dG i (u) = −(R J(u(i))P )−1R J(u(i)) + R . (3.16) du i i i i We deduce for the Jacobian of RASPEN from (3.3) ˜ I I dF1 X dGi X (i) −1 (i) (u) = Pei (u) − I = − Pei(RiJ(u )Pi) RiJ(u ), (3.17) du du i=1 i=1 since the identity cancels. Similarly, we obtain for the Jacobian of ASPEN (Additive Schwarz Preconditioned Exact Newton) in (3.11)

I I I dF1 X dGi X X (u) = P (u) − P R = − P (R J(u(i))P )−1R J(u(i)), (3.18) du i du i i i i i i i=1 i=1 i=1 PI since now the terms i=1 PiRi cancel. In ASPIN, this exact Jacobian is replaced by the inexact Jacobian

inexact I ! dF1 X (u) = − P (R J(u)P )−1R J(u). du i i i i i=1 We see that this is equivalent to preconditioning the Jacobian J(u) of F (u) by the additive Schwarz preconditioner, up to the minus sign. This can be convenient if one has already a code for this, as it was noted in [8]. The exact Jacobian is however also easily accessible, since the Newton solver for the non-linear subdomain system RiF (PiGi(u) + (I − PiRi)u) = 0 already computes and factorizes the local Jacobian (i) matrix RiJ(u )Pi. Therefore, the only missing ingredient for computing the exact (i) (i) Jacobian of F1 is the matrices RiJ(u ), which only diﬀer from RiJ(u )Pi by a few additional columns, corresponding in the usual PDEs framework to the derivative with respect to the Dirichlet boundary conditions. In contrast, the computation of the inexact ASPIN Jacobian requires one to recompute the entire Jacobian of F (u) after the subdomain non-linear solves. 11

3.4.2. Computation of the two-level Jacobian matrices. We now compare the Jacobians for the two-level variants. For RASPEN, we need to diﬀerentiate F˜2 with respect to u, where F˜2 is deﬁned in (3.9): n X Fe2(u) = P0C0(u) + PeiCi(u + P0C0(u)). i=1

dC0 dCi To do so, we need du and du for i = 1,...,I. The former can be obtained by diﬀerentiating (3.8): 0 dC0 0 0 F (R0u + C0(u)) R0 + = F (R0u)R0 − Re0F (u). 0 du 0 Thus, we have

dC0 ˆ−1 = −R0 + J (J0R0 − Re0J(u)), (3.19) du 0 0 ˆ 0 where J0 = F0(R0u) and J0 = F0(R0u+C0(u)). Note that the two Jacobian matrices are evaluated at diﬀerent arguments, so no cancellation is possible in (3.19) except in special cases (e.g., if F0 is an aﬃne function). Nonetheless, they are readily available: Jˆ0 is simply the Jacobian for the non-linear coarse solve, so it would have already been calculated and factorized by Newton’s method. J0 would also have been calculated during the coarse Newton iteration if R0u is used as the initial guess. dCi We also need du from the subdomain solves. From the relation Gi(u) = Riu + Ci(u), we deduce immediately from (3.16) that dC dG i = i − R = −(R J(u(i))P )−1R J(u(i)), (3.20) du du i i i i (i) where u = u + PiCi(u). Thus, the Jacobian for the two-level RASPEN function is dFe2 dC0 X dC0 = P − P˜ (R J(v(i))P )−1R J(v(i)) I + P , (3.21) du 0 du i i i i 0 du i

dC0 (i) where du is given by (3.19) and v = u + P0C0(u) + PiCi(u + P0C0(u)). For completeness, we compute the Jacobian for two-level ASPIN. First, we obtain A dC0 du by diﬀerentiating (3.13), which gives A dC0 ˆ−1 = −J Re0J(u), (3.22) du 0 ˆ 0 A ∗ where J0 = F0(C0 (u) + u0). In addition, two-level ASPIN uses as approximation for (3.20) dC i ≈ −(R J(u)P )−1R J(u). (3.23) du i i i Thus, the inexact Jacobian for the two-level ASPIN function is

dF2 ˆ−1 X −1 ≈ −P0J Re0J(u) − Pi(RiJ(u)Pi) RiJ(u). (3.24) du 0 i

Comparing (3.21) with (3.24), we see two major differences. First, dC0/du only ˆ simplifies to −Re0J(u) if J0 = J0, i.e., if F0 is affine. Second, (3.21) resembles a two- stage multiplicative preconditioner, whereas (3.24) is of the additive type. This is due to the fact that the coarse correction in two-level RASPEN is applied multiplicatively, whereas two-level ASPIN uses an additive correction. 12

4. Numerical experiments. In this section, we compare the new non-linear preconditioner RASPEN to ASPIN for the Forchheimer model, which generalizes the linear Darcy model in porous media flow [18, 33, 10], and for a 2D non-linear diffusion problem that appears in [1]. 4.1. Forchheimer model and discretization. Let us consider the Forch- ∞ heimer parameter β > 0, the permeability λ ∈ L (Ω) such√ that 0 < λmin ≤ λ(x) ≤ −1+ 1+4β|g| λmax for all x ∈ Ω, and the function q(g) = sgn(g) 2β . The Forchheimer model on the interval Ω = (0,L) is defined by the equation

 (q(−λ(x)u(x)0))0 = f(x) in Ω,  D u(0) = u0 , (4.1)  D u(L) = uL .

Note that at the limit when β → 0+, we recover the linear Darcy equation. We consider a 1D mesh deﬁned by the M + 1 points

0 = x 1 < ··· < x 1 < ··· < x 1 = L. 2 K+ 2 M+ 2

The cells are defined by K = (x 1 , x 1 ) for K ∈ M = {1, ··· ,M} and their K− 2 K+ 2 xK− 1 +xK+ 1 2 2 center by xK = 2 . The Forchheimer model (4.1) is discretized using a Two Point Flux Approximation (TPFA) finite volume scheme. We define the TPFA transmissibilities by  1 T 1 = for K = 1, ··· ,M − 1  K+ |x −x | |x −x |  2 K+ 1 K K+1 K+ 1  2 + 2 λK λK+1  λ1 λM  T 1 = ,TM+ 1 = ,  2 |x1 − x 1 | 2 |x 1 − xM | 2 M+ 2

x 1 R K+ 1 with λK = 2 λ(x)dx. Then, the M cell unknowns uK , K ∈ M, |xK+ 1 −xK− 1 | xK− 1 2 2 2 are the solution of the set of M conservation equations  q(T 1 (uK − uK+1)) + q(T 1 (uK − uK−1)) = fK ,K = 2, ··· ,M − 1  K+ 2 K− 2  D q(T 3 (u1 − u2)) + q(T 1 (u1 − u )) = f1, 2 2 0 D  q(T 1 (uM − u )) + q(T 1 (uM − uM−1)) = fM , M+ 2 L M− 2

x R K+ 1 with fK = 2 f(x)dx. In the following numerical tests we will consider a uniform xK− 1 2 L mesh of cell size denoted by h = M . 4.1.1. One level variants. We start from a non-overlapping decomposition of the set of cells

Mfi, i = 1, ··· ,I, S such that M = i=1,··· ,I Mfi and Mfi ∩ Mfj = ∅ for all i 6= j. The overlapping decomposition Mi, i = 1, ··· ,I of the set of cells is obtained by adding k layers of cells to each Mfi to generate overlap with the two neighbouring subdomains Mfi−1 (if i > 1) and Mfi+1 (if i < I) in the simple case of our one dimensional domain. 13

1.1 60 1 permeability right hand side Initial guess 0.8 Exact solution 50 1 0.6

0.4 40 0.9 0.2

30 0 0.8 −0.2 20 −0.4 0.7 −0.6 10 0.6 −0.8

0 −1

0.5 −1.2 −10 −1.4 0.4 −1.6 −20 −1.8 0.3 −30 −2

−2.2 0.2 −40 −2.4

−2.6 0.1 −50 −2.8

0 −60 −3 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

Fig. 4.1: Permeability ﬁeld (left), source term (middle), initial guess and solution (right).

#M #Mi In the ASPIN framework, we set V = R , and Vi = R , i = 1, ··· ,I. The restriction operators are then deﬁned by

(Riv)K = vK for K ∈ Mi, and the prolongation operators are ( (P v ) = v for K ∈ M , (P v ) = v for K ∈ M , i i K K i and ei i K K fi (Pivi)K = 0 for K 6∈ Mi, (Peivi)K = 0 for K 6∈ Mfi.

The coarse grid is obtained by the agglomeration of the cells in each Mfi defining a coarse mesh of (0,L). I Finally, we set V0 = R . In the finite volume framework, we define for all v ∈ V 1 X (R0v)i = vK for all i = 1, ··· ,I, #Mfi K∈Mfi X (Re0v)i = vK for all i = 1, ··· ,I.

K∈Mfi

In our case of a uniform mesh, R0 corresponds to the mean value in the coarse cell i for cellwise constant functions on M, whereas Re0 corresponds to the aggregate flux over the coarse cell Mfi. For v0 ∈ V0, its prolongation v = P0v0 ∈ V is obtained by the piecewise linear interpolation ϕ(x) on (0, x1), (x1, x2), ··· , (xI ,L) where the xi are the centers of the coarse cells, and ϕ(xi) = (v0)i, i = 1, ··· ,I, ϕ(0) = 0, ϕ(L) = 0. Then, v = P0v0 is defined by vK = ϕ(xK ) for all K ∈ M. The coarse grid operator F0 is defined by F0(v0) = Re0F (P0v0) for all v0 ∈ V0. We use for the numerical tests the domain Ω = (0, 3/2) with the boundary conditions u(0) = 0 and u(3/2) = 1, and different values of β. As a first challenging test, we choose the highly variable permeability field λ and the oscillating right hand side shown in Figure 4.1. We measure the relative l1 norms of the error obtained at each Newton iteration as a function of the parallel linear solves LSn needed in the subdomains per Newton iteration, which is a realistic measure for the cost of the method. Each Newton iteration requires two major steps: 14

1. the evaluation of the fixed point function F, which means solving a non-linear problem in each subdomain. This is done using Newton in an inner iteration on each subdomain, and thus requires at each inner iteration a linear subdomain solve performed in parallel by all subdomains (we have used a sparse direct solver for the linear subdomain solves in our experiments, but one can also use an iterative method if good preconditioners are available). We denote the maximum number of inner iterations needed by the subdomains at the in outer iteration j by lsj , and it is the maximum which is relevant, because if other subdomains finish earlier, they still have to wait for the last one to finish. 2. the Jacobian matrix needs to be inverted, which we do by GMRES, and each GMRES iteration will also need a linear subdomain solve per subdomain. We denote the number of linear solves needed by GMRES at the outer Newton G iteration step j by lsj . Hence, the number of linear subdomain solves for the outer Newton iteration j to in G complete is lsj + lsj , and the total number of linear subdomain solves after n outer Pn in G Newton iterations is LSn := j=1 lsj + lsj . In all the numerical tests, we stop the linear GMRES iterations when the relative residual falls below 10−8, and the tolerances for the inner and outer Newton iterations are also set to 10−8. Adaptive tolerances could certainly lead to more savings [15, 16], but our purpose here is to compare the non-linear preconditioners in a fixed setting. The initial guess we use in all our experiments is shown in Figure 4.1 on the right, together with the solution. We show in Figure 4.2 how the convergence depends on the overlap and the number of subdomains for one level ASPIN and RASPEN with Forchheimer model parameter β = 1. In the top row on the left of Figure 4.2, we see that for ASPIN the number of linear iterations increases much more rapidly when decreasing the overlap than for RASPEN on the right for a fixed mesh size h = 0.003 and number of subdmains equal 20. In the bottom row of Figure 4.2, we see that the convergence of both one level ASPIN and RASPEN depends on the number of subdomains, but RASPEN seems to be less sensitive than ASPIN. 4.1.2. Two level variants. In Figure 4.3, we show the dependence of two-level ASPIN and two-level FAS-RASPEN on a decreasing size of the overlap, as we did for the one-level variants in the top row of Figure 4.2. We see that the addition of the coarse level improves the performance, for RASPEN when the overlap is large, and in all cases for ASPIN. In Figure 4.4, we present a study of the influence of the number of subdomains on the convergence for two-level ASPIN and two-level FAS-RASPEN with different values of the Forchheimer parameter β = 1, 0.1, 0.01 which governs the non-linearity of the model (the model becomes linear for β = 0). An interesting observation is that for β = 1, the convergence of both two-level ASPIN and two-level FAS-RASPEN depends on the number of subdomains in an irregular fashion: increasing the number of subdomains sometimes increases iteration counts, and then decreases them again. We will study this effect further below, but note already from Figure 4.4 that this dependence disappears for two-level FAS-RAPSEN as the the nonlinearity diminishes (i.e., as β decreases), and is weakened for two-level ASPIN. We finally show in Table 4.1 the number of outer Newton iterations (PIN iter for ASPIN and PEN iter for RASPEN) and the total number of linear iterations (LSn iter) for various numbers of subdomains and various overlap sizes obtained with ASPIN, RASPEN, two-level ASPIN and two-level FAS-RASPEN. We see that the 15

20 subdomains, h=0.003 20 subdomains, h=0.003 4 4 h h 3h 3h 3 3 9h 9h 15h 15h 2 2

1 1

0 0

−1 −1

−2 −2

−3 −3

−4 −4

−5 −5

−6 −6

−7 −7

−8 −8

−9 −9

−10 −10 0100 200 300 400 500600 700 800 900 0100 200 300 400 500600 700 800 900

15 cells per subdomain, overlap 3h 15 cells per subdomain, overlap 3h 4 4 10 subdomains 10 subdomains 3 20 subdomains 3 20 subdomains 40 subdomains 40 subdomains 2 2

1 1

0 0

−1 −1

−2 −2

−3 −3

−4 −4

−5 −5

−6 −6

−7 −7

−8 −8

−9 −9

−10 −10

−11 −11

−12 −12 0100 200 300 400 500600 700 800 900 0100 200 300 400 500600 700 800 900

Fig. 4.2: Error obtained with one-level ASPIN (left) and one-level RASPEN (right): in the top row obtained with 20 subdomains, h = 0.003, and decreasing size of overlap 15h, 9h, 3h, h; in the bottom row obtained with different number of subdomains 10, 20 and 40, overlap 3h, and a fixed number of cells per subdomain. The Forchheimer problem is defined by the permeability, source term, solution and initial guess of Figure 4.1.

20 subdomains, h=0.003 20 subdomains, h=0.003 4 4 h h 3h 3h 3 3 9h 9h 15h 15h 2 2

1 1

0 0

−1 −1

−2 −2

−3 −3

−4 −4

−5 −5

−6 −6

−7 −7

−8 −8

−9 −9

−10 −10 0100 200 300 400 500600 700 800 900 0100 200 300 400 500600 700 800 900

Fig. 4.3: Error obtained with two-level ASPIN (left) and two-level FAS RASPEN (right) obtained with 20 subdomains, h = 0.003, and decreasing overlap 15h, 9h, 3h, h. The Forchheimer problem is deﬁned by the permeability, source term, solution and initial guess of Figure 4.1. 16

2 3 10 subdomains 10 subdomains 20 subdomains 20 subdomains 1 30 subdomains 2 30 subdomains 40 subdomains 40 subdomains 50 subdomains 50 subdomains 0 1

0 −1

−1 −2 −2 −3 −3 −4 −4 −5 −5 −6 −6

−7 −7

−8 −8

−9 −9

−10 −10 020 40 60 80 100 120 140 160 180 200 220 240 260 020 40 60 80 100 120 140 160 180 200 220 240 260

2 3 10 subdomains 10 subdomains 20 subdomains 20 subdomains 1 30 subdomains 2 30 subdomains 40 subdomains 40 subdomains 50 subdomains 50 subdomains 0 1

0 −1

−1 −2 −2 −3 −3 −4 −4 −5 −5 −6 −6

−7 −7

−8 −8

−9 −9

−10 −10 020 40 60 80 100 120 140 160 180 200 220 240 260 020 40 60 80 100 120 140 160 180 200 220 240 260

2 3 10 subdomains 10 subdomains 20 subdomains 20 subdomains 1 30 subdomains 2 30 subdomains 40 subdomains 40 subdomains 50 subdomains 50 subdomains 0 1

0 −1

−1 −2 −2 −3 −3 −4 −4 −5 −5 −6 −6

−7 −7

−8 −8

−9 −9

−10 −10 020 40 60 80 100 120 140 160 180 200 220 240 260 020 40 60 80 100 120 140 160 180 200 220 240 260

Fig. 4.4: Error obtained with two-level ASPIN (left) and two-level FAS RASPEN (right) and diﬀerent number of subdomains 10, 20, 30, 40, 50. From top to bottom with decreasing Forchheimer parameter β = 1, 0.1, 0.01. The Forchheimer problem is deﬁned by the permeability, source term, solution and initial guess of Figure 4.1. 17

ASPIN Number of subdomains 10 20 40 type of iteration PIN iter. LS iter. PIN iter. LS iter. PIN iter. LS iter. Overlap size n n n h 8 184 15 663 - - 3h 7 156 14 631 11 883 5h 6 130 11 479 10 744 RASPEN Number of subdomains 10 20 40 type of iteration PEN iter. LS iter. PEN iter. LS iter. PEN iter. LS iter. Overlap size n n n h 7 150 9 369 9 701 3h 7 145 8 324 9 691 5h 6 126 7 274 9 659 Two-level ASPIN Number of subdomains 10 20 40 type of iteration PIN iter. LS iter. PIN iter. LS iter. PIN iter. LS iter. Overlap size n n n h 7 184 9 316 8 285 3h 6 141 9 246 7 183 5h 6 135 8 199 7 164 Two-level FAS-RASPEN Number of subdomains 10 20 40 type of iteration PEN iter. LS iter. PEN iter. LS iter. PEN iter. LS iter. Overlap size n n n h 7 134 9 272 8 258 3h 7 133 8 220 6 136 5h 6 112 8 211 6 116

Table 4.1: Comparison in terms of non-linear and linear iterations of the diﬀerent algorithms for the Forchheimer problem deﬁned by the permeability, source term, solution and initial guess of Figure 4.1.

coarse grid considerably improves the convergence of both RASPEN and ASPIN. Also, in all cases, RASPEN needs substantially fewer linear iterations than ASPIN. We now return to the irregular number of iterations observed in Figure 4.4 for the Forchheimer parameter β = 1, i.e when the non-linearity is strong. We claim that this irregular dependence is due to strong variations in the initial guesses used by RASPEN and ASPIN at subdomain interfaces, which is in turn caused by the highly variable contrast and oscillating source term we used, leading to an oscillatory solution, see Figure 4.1. In other words, we expect the irregularity to disappear when the solution is non-oscillatory. To test this, we now present numerical results with the less variable permeability function λ(x) = cos(x) and source term f(x) = cos(x) as well, which leads to a smooth solution. Starting with a zero initial guess, we show in Figure 4.5 the results obtained for Forchheimer parameter β = 1, corresponding to the ﬁrst row of Figure 4.4. We clearly see that the irregular behavior has now disappeared for both two- level ASPIN and RASPEN, but two-level ASPIN still shows some dependence of the iteration numbers as the number of subdomains increases. We show in Table 4.2 the 18

2 2 10 subdomains 10 subdomains 20 subdomains 20 subdomains 1 1 30 subdomains 30 subdomains 40 subdomains 40 subdomains 50 subdomains 50 subdomains 0 0

−1 −1

−2 −2

−3 −3

−4 −4

−5 −5

−6 −6

−7 −7

−8 −8

−9 −9

−10 −10

−11 −11

−12 −12 0 20 40 60 80 100 120 140 160 180 200 220 240 260 0 20 40 60 80 100 120 140 160 180 200 220 240 260

Fig. 4.5: Error obtained with two-level ASPIN (left) and two-level FAS RASPEN (right) with overlap 3h, and diﬀerent number of subdomains 10, 20, 30, 40, 50 for the smooth Forchheimer example.

ASPIN Number of subdomains 10 20 40 type of iteration PIN iter. LS iter. PIN iter. LS iter. PIN iter. LS iter. Overlap size n n n h 5 118 5 228 6 520 3h 5 118 5 227 6 516 5h 5 117 5 222 6 480 RASPEN Number of subdomains 10 20 40 type of iteration PEN iter. LS iter. PEN iter. LS iter. PEN iter. LS iter. Overlap size n n n h 4 92 4 172 4 340 3h 4 87 4 172 4 331 5h 4 88 4 168 4 313 Two level ASPIN Number of subdomains 10 20 40 type of iteration PIN iter. LS iter. PIN iter. LS iter. PIN iter. LS iter. Overlap size n n n h 5 140 5 240 5 280 3h 5 130 6 170 6 200 5h 5 115 7 149 6 147 Two level FAS RASPEN Number of subdomains 10 20 40 type of iteration PEN iter. LS iter. PEN iter. LS iter. PEN iter. LS iter. Overlap size n n n h 4 77 3 87 4 131 3h 3 60 3 67 4 90 5h 3 55 3 57 3 57

Table 4.2: Comparison in terms of non-linear and linear iterations of the diﬀerent algorithms for the smooth Forchheimer example. 19

Number of 1-Level 2-Level G in min G in min subdomains n lsn lsn lsn LSn lsn lsn lsn LSn 10 1 19 (20) 4 (4) 3 (3) 15 (20) 7 (4) 3 (3) 2 19 (20) 3 (6) 3( 3) 87 (118) 16 (21) 3 (6) 2 (3) 60 (130) 3 19 (20) 2 (4) 2 (2) 17 (22) 2 (3) 1 (2) 4 19 (20) 2 (2) 1 (2) - (24) - (3) - (1) 5 - (21) - (1) - (1) - (25) - (2) - (1) 20 1 40 (41) 5 (5) 3 (3) 15 (22) 8 (5) 3 (3) 2 40 (41) 3 (7) 2 (2) 172 (227) 18 (23) 3 (6) 2 (3) 67 (170) 3 40 (41) 2 (5) 1 (2) 21 (24) 2 (5) 1 (2) 4 40 (41) 2 (3) 1 (1) - (24) - (3) - (1) 5 - (41) - (2) - (1) - (24) - (2) - (1) 6 - (-) - (-) - (-) - (31) - (1) - (1) 40 1 78 (80) 5 (5) 3 (3) 14 (22) 9 (5) 3 (3) 2 81 (81) 3 (6) 2 (2) 331 (516) 17 (22) 3 (7) 1 (2) 90 (200) 3 79 (82) 2 (6) 1 (2) 20 (24) 2 (6) 1 (2) 4 81 (82) 2 (5) 1 (1) 24 (24) 1(5) 0 (1) 5 - (82) - (3) - (1) - (23) - (3) - (1) 6 - (82) - (2) - (1) - (25) - (2) - (1) 7 - (-) - (-) - (-) - (31) - (1) - (0)

Table 4.3: Numerical results with one- and two-level RASPEN and ASPIN for the 1D non-linear smooth Forchheimer problem. ‘-’ indicates that the method has converged.

complete results for this smoother example, and we see that the irregular convergence behavior of the two-level methods is no longer present. We finally give in Table 4.3 a detailed account of the linear subdomain solves needed for each outer Newton iteration n for the case of an overlap of 3h. There, we use the format itRASPEN(itASPIN), where itRASPEN is the iteration count for RASPEN and itASPIN is the iteration count for G ASPIN. We show in the first column the linear subdomain solves lsn required for the inversion of the Jacobian matrix using GMRES, see item 2 in Subsection 4.1.1, and in in the next column the maximum number of iterations lsn needed to evaluate the nonlinear fixed point function F, see item 1 in Subsection 4.1.1. In the next column, min we show for completeness also the smallest number of inner iterations lsn any of the subdomains needed, to illustrate how balanced the work is in this example. The last column then contains the total number of linear iterations LSn, see Subsection 4.1.1. These results show that the main gain of RASPEN is a reduced number of Newton iterations, i.e. it is a better non-linear preconditioner than ASPIN, and also a reduced number of inner iterations for the non-linear subdomain solves, i.e. the preconditioner is less expensive. This leads to the substantial savings observed in the last columns, and in Table 4.2. 4.2. A non-linear Poisson problem. We now test the non-linear preconditioners on the two dimensional non-linear diffusion problem (see [1])  −∇ · ((1 + u2)∇u) = f, Ω = [0, 1]2,  u = 1, x = 1, (4.2) ∂u  = 0, otherwise. ∂n The isovalues of the exact solution are shown in Figure 4.6. To calculate this solution, we use a discretization with P1 finite elements on a uniform triangular mesh. All 20

IsoValue 0.997804 1.0011 1.00329 1.00549 1.00768 1.00988 1.01208 1.01427 1.01647 1.01866 1.02086 1.02305 1.02525 1.02745 1.02964 1.03184 1.03403 1.03623 1.03842 1.04391

Fig. 4.6: Exact solution of the non-linear Poisson problem (4.2)

1-Level 2-Level G in min G in min N × N n lsn lsn lsn LSn lsn lsn lsn LSn 2 × 2 1 15(20) 4(4) 3(3) 13(23) 4(4) 3(3) 2 17(23) 3(3) 3(3) 59(78) 15(26) 3(3) 3(3) 54(86) 3 18(26) 2(2) 2(2) 17(28) 2(2) 2(2) 4 × 4 1 32(37) 3(3) 3(3) 18(33) 3(3) 3(3) 2 35(41) 3(3) 2(2) 113(132) 22(39) 3(3) 2(2) 74(126) 3 38(46) 2(2) 2(2) 26(46) 2(2) 2(2) 8 × 8 1 62(71) 3(3) 2(2) 18(35) 3(3) 3(2) 2 67(77) 3(3) 2(2) 211(240) 23(44) 3(3) 2(2) 77(139) 3 74(84) 2(2) 1(2) 28(53) 2(2) 2(1) 16 × 16 1 125(141) 3(3) 2(2) 18(35) 3(3) 3(2) 2 136(155) 2(2) 2(2) 418(471) 23(44) 2(2) 2(2) 75(140) 3 150(167) 2(2) 1(1) 27(54) 2(2) 2(1)

Table 4.4: Numerical results with one- and two-level RASPEN and ASPIN for the non-linear diﬀusion problem.

calculations have been performed using FreeFEM++, a C++ based domain-specific language for the numerical solution of PDEs using finite element methods [25]. We consider a decomposition of the domain into N × N subdomains with an overlap of one mesh size h, and we keep the number of degrees of freedom per subdomain fixed in our experiments. We show in Table 4.4 a detailed account of the number of linear subdomain solves needed for RASPEN and ASPIN at each outer Newton iteration n, using the same notation as in Table 4.3 (Newton converged in three iterations for all examples to a tolerance of 10−8). We see from these experiments that RASPEN, which is a non-linear preconditioner based on a convergent underlying fixed point iteration, clearly outperforms ASPIN, which would not be convergent as a basic fixed point iteration. 4.3. A problem with discontinuous coefficients. We now test the non-linear preconditioners on the two dimensional Forchheimer problem, which can be written 21

Fig. 4.7: Left: Fine discretization of the domain for N = 4. The red and black inclusions correspond to low-permeability regions. Middle: Coarse grid used for two- level methods. Right: Exact solution for the 2D Forchheimer problem.

β = 0.1 β = 1 2 × 2 4 × 4 8 × 8 2 × 2 4 × 4 8 × 8 Newton 19 19 19 38 44 48 ASPIN 6 div. div. 6 div. div. ASPIN2 5 6 7 6 7 9 RASPEN 5 4 4 5 5 5 RASPEN2 4 4 4 5 5 6

Table 4.5: Number of non-linear iterations required for convergence by various algorithms for the 2D Forchheimer problem, as a function of problem size. Divergence of the method is indicated by ‘div’.

as  −∇ · q = 0, Ω = [0, 1]2,   q + β|q|q = Λ(x)∇u, (4.3) u = 0 on Γ , u = 1 on Γ ,  d0 d1  q · n = 0 on ∂Ω \ (Γd0 ∪ γd1), where the Dirichlet boundaries Γd0 and Γd1 are located at the bottom left and top right corners of the domain, namely,

Γd0 = {(x, y) ∈ ∂Ω: x + y < 0.2}, Γd1 = {(x, y) ∈ ∂Ω: x + y > 1.8}.

The permeability Λ(x) is equal to 1000 everywhere except at the two inclusions shown in red and black in the left panel of Figure 4.7, where it is equal to 1. The mesh in the above figure is used to discretize the problem, using P1 finite elements; the exact solution is shown in the right panel. We consider a decomposition of the domain into N ×N subdomains with an overlap of one mesh size h, and we keep the number of degrees of freedom per subdomain fixed in our experiments. For the two-level methods, the coarse function F0 consists of a P1 discretization of the problem over the coarse grid shown in the middle panel of Figure 4.7. To measure the difficulty of this problem, we run our non-linear algorithms (standard Newton, one and two-level ASPIN, one and two-level RASPEN) on this problem 22

1-Level 2-Level G in min G in min N × N β n lsn lsn lsn LSn lsn lsn lsn LSn 2 × 2 0.1 1 22(29) 5(5) 5(5) 82(106) 10(20) 6(5) 5(5) 47(102) 2 24(32) 4(4) 4(4) 12(21) 3(4) 3(4) 3 25(33) 2(3) 2(2) 14(22) 2(3) 2(3) 4 - (-) - (-) - (-) - (25) - (2) - (2) 0.2 1 22(28) 4(4) 3(3) 53(69) 9(19) 4(4) 3(3) 29(49) 2 24(34) 3(3) 3(3) 14(23) 2(3) 2(3) 0.5 1 22(28) 4(4) 4(4) 53(69) 9(19) 4(4) 3(3) 29(49) 2 24(34) 3(3) 3(3) 14(23) 2(3) 2(3) 1.0 1 22(28) 4(4) 4(4) 53(69) 10(21) 4(4) 3(3) 30(51) 2 24(34) 3(3) 2(2) 14(23) 2(3) 2(2) 4 × 4 0.1 1 41(53) 5(5) 4(4) 145(179) 11(21) 6(6) 4(4) 52(111) 2 45(56) 4(4) 3(3) 14(23) 3(4) 3(3) 3 48(58) 2(3) 2(2) 16(24) 2(4) 2(3) 4 - (-) - (-) - (-) - (26) - (3) - (2) 0.2 1 41(52) 4(4) 3(3) 94(118) 11(21) 4(4) 3(3) 33(54) 2 47(59) 2(3) 2(2) 16(26) 2(3) 2(2) 0.5 1 41(51) 4(4) 3(3) 94(116) 11(21) 4(4) 3(3) 33(54) 2 47(58) 2(3) 2(2) 16(26) 2(3) 2(2) 1.0 1 41(51) 4(4) 3(3) 94(116) 11(21) 4(3) 3(3) 34(53) 2 47(58) 2(3) 2(2) 17(26) 2(3) 2(2) 8 × 8 0.1 1 86(104) 5(5) 3(3) 468(573) 16(24) 6(5) 3(3) 73(160) 2 92(111) 3(4) 2(2) 21(27) 4(4) 3(3) 3 95(115) 3(3) 2(2) 24(26) 2(4) 2(2) 4 90(116) 2(2) 1(1) - (30) - (3) - (2) 5 90(111) 2(2) 1(1) - (35) - (2) - (1) 0.2 1 84(103) 4(4) 3(3) 373(457) 16(24) 4(4) 3(3) 46(62) 2 93(115) 3(3) 2(2) 24(31) 2(3) 2(2) 3 94(117) 2(2) 1(1) - (-) - (-) - (-) 4 91(111) 2(2) 1(1) - (-) - (-) - (-) 0.5 1 84(104) 4(4) 3(3) 374(461) 16(25) 4(4) 3(3) 46(63) 2 94(115) 3(3) 2(2) 24(31) 2(3) 2(2) 3 94(119) 2(2) 1(1) - (-) - (-) - (-) 4 91(112) 2(2) 1(1) - (-) - (-) - (-) 1.0 1 84(104) 4(4) 2(2) 375(461) 16(25) 4(4) 3(2) 47(64) 2 95(115) 3(3) 2(2) 25(32) 2(3) 2(2) 3 95(119) 2(2) 1(1) - (-) - (-) - (-) 4 91(112) 2(2) 1(1) - (-) - (-) - (-)

Table 4.6: Numerical results with one- and two-level RASPEN and ASPIN for the 2D Forchheimer problem. ‘-’ indicates that the method has converged.

for β = 0.1 and β = 1. We show in Table 4.5 the number of iterations required for the convergence of each algorithm. We see that between the discontinuous permeability and the non-linearity introduced by β, standard Newton requires many iterations to converge, and one-level ASPIN diverges for the larger problems. On the other hand, one and two-level RASPEN (and two-level ASPIN, to a lesser extent) converge in a small number of non-linear iterations. Next, we compare the one and two-level variants of ASPIN and RASPEN in terms of the total amount of computational work. To deal with the convergence problem in 23 one-level ASPIN, we adopt the continuation approach, where we solve the problem for a sequence of β (0, 0.1, 0.2, 0.5 and 1.0), using the solution for the previous β as the initial guess for the next one. Table 4.6 shows a detailed account for each outer Newton iteration n of the linear subdomain solves needed for both RASPEN and ASPIN using the same notation as in Table 4.3. We omit the data for β = 0, as the problem becomes linear in that case. We see again from these experiments that the RASPEN-based preconditioners can handle non-linearly diﬃcult problems, requiring fewer non-linear iterations and linear solves than their ASPIN counterparts.

5. Conclusion. We have shown that just as one can accelerate stationary iterative methods for linear systems using a Krylov method, one can also accelerate fixed point iterations for non-linear problems using Newton’s method. This leads to a guiding principle for constructing non-linear preconditioners, which we illustrated with the systematic construction of RASPEN. While this design principle leads to good non-linear (and linear) preconditioners, see for example [22, 23] for a similar approach for non-linear evolution problems, it is by no means the only approach possible; in the linear case, for instance, the additive Schwarz preconditioner [13], as well as the highly effective and robust FETI preconditioner [17] and its variants, do not correspond to a convergent iteration. Indeed, clustering the spectrum into a few clusters is sometimes better than having a small spectral radius, see for example the results for the HSS preconditioner in [2]. Thus, it is still an open question whether there are other properties that a preconditioner should have that would make it more effective, even if it is associated with a divergent iteration. For non-linear preconditioning, maybe it is possible to greatly increase the basin of attraction of the non-linearly preconditioned Newton method, or to improve its preasymptotic convergence, before quadratic convergence sets in. It also remains to carefully compare RASPEN with linear preconditioning inside Newton’s method; promising results for ASPIN can be found already in [31].

Acknowledgments. We would like to thank TOTAL for partially supporting this work.

REFERENCES

[1] G. W. Anders Logg, Kent-Andre Mardal, editor. Automated Solution of Differential Equa- tions by the Finite Element Method: The FEniCS Book, volume 84 of Lecture Notes in Computational Science and Engineering. Springer, 2012. [2] M. Benzi, M. J. Gander, and G. H. Golub. Optimization of the Hermitian and skew-Hermitian splitting iteration for saddle-point problems. BIT Numerical Mathematics, 43(5):881–900, 2003. [3] W. L. Briggs, S. F. McCormick, et al. A multigrid tutorial. SIAM, 2000. [4] F. Caetano, M. J. Gander, L. Halpern, and J. Szeftel. Schwarz waveform relaxation algorithms for semilinear reaction-diffusion equations. NHM, 5(3):487–505, 2010. [5] X.-C. Cai and M. Dryja. Domain decomposition methods for monotone nonlinear elliptic problems. Contemporary mathematics, 180:21–27, 1994. [6] X.-C. Cai, W. D. Gropp, D. E. Keyes, R. G. Melvin, and D. P. Young. Parallel Newton–Krylov– Schwarz algorithms for the transonic full potential equation. SIAM Journal on Scientific Computing, 19(1):246–265, 1998. [7] X.-C. Cai, W. D. Gropp, D. E. Keyes, and M. D. Tidriri. Newton-Krylov-Schwarz methods in CFD. In Numerical methods for the Navier-Stokes equations, volume 47, pages 17–30. Vieweg+Teubner, 1994. [8] X.-C. Cai and D. E. Keyes. Nonlinearly preconditioned inexact Newton algorithms. SIAM Journal on Scientific Computing, 24(1):183–200, 2002. [9] X.-C. Cai, D. E. Keyes, and D. P. Young. A nonlinear additive Schwarz preconditioned inexact 24

Newton method for shocked duct flow. In Proceedings of the 13th International Conference on Domain Decomposition Methods, pages 343–350. DDM.org, 2001. [10] Z. Chen, G. Huan, and Y. Ma. Computational Methods for Multiphase flows in porous media. SIAM, 2006. [11] S. Descombes, V. Dolean, and M. J. Gander. Schwarz waveform relaxation methods for systems of semi-linear reaction-diffusion equations. In Domain Decomposition Methods in Science and Engineering XIX, pages 423–430. Springer, 2011. [12] M. Dryja and W. Hackbusch. On the nonlinear domain decomposition method. BIT Numerical Mathematics, 37(2):296–311, 1997. [13] M. Dryja and O. B. Widlund. An additive variant of the Schwarz alternating method for the case of many subregions. Technical Report 339, also Ultracomputer Note 131, Department of Computer Science, Courant Institute, 1987. [14] E. Efstathiou and M. J. Gander. Why Restricted Additive Schwarz converges faster than Additive Schwarz. BIT Numerical Mathematics, 43(5):945–959, 2003. [15] L. El Alaoui, A. Ern, and M. Vohral´ık. Guaranteed and robust a posteriori error estimates and balancing discretization and linearization errors for monotone nonlinear problems. Computer Methods in Applied Mechanics and Engineering, 200(37):2782–2795, 2011. [16] A. Ern and M. Vohral´ık.Adaptive inexact Newton methods with a posteriori stopping criteria for nonlinear diffusion PDEs. SIAM Journal on Scientific Computing, 35(4):A1761–A1791, 2013. [17] C. Farhat and F. Roux. A method of finite element tearing and interconnecting and its parallel solution algorithm. Internatl. J. Numer. Methods Engrg., 32:1205–1227, 1991. [18] P. Forchheimer. Wasserbewegung durch Boden. Zeitschrift des Vereines Deutscher Ingenieuer, 45:1782–1788, 1901. [19] M. J. Gander. A waveform relaxation algorithm with overlapping splitting for reaction diffusion equations. Numerical Linear Algebra with Applications, 6:125–145, 1998. [20] M. J. Gander. Schwarz methods over the course of time. Electronic Transactions on Numerical Analysis, 31:228–255, 2008. [21] M. J. Gander and C. Rohde. Overlapping Schwarz waveform relaxation for convection domi- nated nonlinear conservation laws. SIAM J. Sci. Comp., 27(2):415–439, 2005. [22] F. Haeberlein, L. Halpern, and A. Michel. Newton-Schwarz optimised waveform relaxation Krylov accelerators for nonlinear reactive transport. In Domain Decomposition Methods in Science and Engineering XX, pages 387–394. Springer, 2013. [23] F. Haeberlein, L. Halpern, and A. Michel. Schwarz waveform relaxation and Krylov accelerators for reactive transport. 2015. submitted. [24] L. Halpern and J. Szeftel. Nonlinear nonoverlapping Schwarz waveform relaxation for semilinear wave propagation. Mathematics of Computation, 78(266):865–889, 2009. [25] F. Hecht. New development in FreeFem++. Journal of Numerical Mathematics, 20(3-4):251– 266, 2012. [26] V. E. Henson. Multigrid methods nonlinear problems: an overview. In Electronic Imaging 2003, pages 36–48. International Society for Optics and Photonics, 2003. [27] P.-L. Lions. On the Schwarz alternating method. I. In R. Glowinski, G. H. Golub, G. A. Meu- rant, and J. Périaux,editors, First International Symposium on Domain Decomposition Methods for Partial Differential Equations, pages 1–42, Philadelphia, PA, 1988. SIAM. [28] S.-H. Lui. On Schwarz alternating methods for nonlinear elliptic PDEs. SIAM Journal on Scientific Computing, 21(4):1506–1523, 1999. [29] S.-H. Lui. On linear monotone iteration and Schwarz methods for nonlinear elliptic PDEs. Numerische Mathematik, 93(1):109–129, 2002. [30] L. Marcinkowski and X.-C. Cai. Parallel performance of some two-level ASPIN algorithms. In Domain Decomposition Methods in Science and Engineering, pages 639–646. Springer, 2005. [31] J. O. Skogestad, E. Keilegavlen, and J. M. Nordbotten. Domain decomposition strategies for nonlinear flow problems in porous media. Journal of Computational Physics, 234:439–451, 2013. [32] X.-C. Tai and M. Espedal. Rate of convergence of some space decomposition methods for linear and nonlinear problems. SIAM journal on numerical analysis, 35(4):1558–1570, 1998. [33] J. C. Ward. Turbulent flow in porous media. J. Hydr. Div. ASCE, 90:1–12, 1964. [34] J. Xu. Two-grid discretization techniques for linear and nonlinear PDEs. SIAM Journal on Numerical Analysis, 33(5):1759–1777, 1996.