arXiv:2004.13112v2 [math.OC] 14 Jun 2020 guidance[ azine a hnabscotmlcnrlsle.BcueDD a bas head of as was mathematics 1 computational th DIDO page in theory, winner Because control big optimal a (PS) as solver. emerged pseudospectral control new then optimal a basic on a then was re pcrlalgorithm[ spectral free, first[ more one added event historic h upt fDD nld h olcino eesr cond the in necessary are of defin conditions Principle[ collection sary problem Pontryagin’s the applying the include by just DIDO generated is of DIDO outputs to problem The input the The writing resembles (“pen-to-paper”). closely optima an format of This format DIDO’s problem. describe first provide In we to problems. philosophy, control this principle: optimal founding solving its for approach to malist’s true remained Thi from has 2001. DIDO since DIDO of same the usage virtually the remained has changes, perspective internal current its Despite as to refer simply g we a evo that to gradually approach and discretization new quietly PS entirely it patched solver, that programming code nonlinear simple a indeed was DIDO of demonstration flight first a also was It plishments. Control ic t neto,cra1960[ circa u inception, remained its largely have since fundamentals whose Principle gin’s solution. t computed test the of “independently” ity to Principle Pontryagin’s apply may Fig. (see media lblyotmlmnue htsvdNS $1,000,000[ NASA saved that maneuver optimal globally a n oue ntrekypromneelements: performance work key that three algorithms on of focused suite and a actually is algorithm DIDO’s u simple solvers[ its programming beyond nonlinear well generic moved DIDO in Me implemented claims. algorithm disingenuous of degrees varying with terialized * .Seta ceeainbsdo h rnilso discret of principles the on based acceleration Spectral 2. .Vrfibeacrc ftecmue ouini em ffe of terms in solution computed the of accuracy Verifiable 3. .Eteerbsns otepitta ges sntrequi not is “guess” a that point the to robustness Extreme 1. uhro h textbook, the of Author h nu-uptfaue fDD r nacrac ihPon with accordance in are DIDO of features input-output The nNvme ,20,teItrainlSaeSainexecut Station Space International the 2006, 5, November On i oe tr nteueo Scnrlter o attitude for theory control PS of use the on story cover a did olgaePbihr,SnFacso A 2015. CA, Francisco, San Publishers, Collegiate , ettneig(e Theorem (see tunneling gent and iiyadotmlt tests[ optimality and bility rmauser[ a from 4 IMNews SIAM .I xliigtewdsra ulct ntetechnical the in publicity widespread the exploiting In ]. 1 nacmnst h DIDO the to Enhancements nw oe ae nnieP iceiain ma- discretizations PS naive on based codes “new” ) 7 n22,DIDO 2020, In o h aetvrino IOi aal fecpn h trap the escaping methods. of capable exam optimization is trajectory challenge DIDO A of version paper. latest this the how in internal described the are of algorithms description and A t techniques. solvers programming programming nian nonlinear generic of use la simple the their Over problems. pr traveling-salesman high-performan solving continuous-time designing from in roots: sensitivity-analysis using aerospace linear maneuver its optimal beyond globally well a plications executed had it announced olt lutaetevrosnacso otygnsPrin Pontryagin’s of nuances various the illustrate to tool ]; [ 3 .Introduction I. 7 .Subsequently, ]. , outputs 8 rmro otygnsPicpeo Optimal on Principle Pontryagin’s on Primer A .Ta s hl h 01vrino DIDO of 2001-version the while is, That ]. 10 9 .Apnie n eciethe describe B and A Appendixes ]. © fDD adntisipt! user a inputs!) its not (and DIDO of ]. 2 und2!Tesfwr akg mre n20 sabsc us basic, a as 2001 in emerged package software The 20! turned omn fNS’ ra accom- great NASA’s of many to ] 1 nApni fti paper); this of A Appendix in 5 EECnrlSsesMag- Systems Control IEEE , 6 oamliaee guess- multifaceted a to ] 9 .Bcueteneces- the Because ]. IOsalgorithm DIDO’s nhl,the anwhile, showcasing sbecause is s eoptimal- he otry A93940 CA Monterey, vdt an to lved nchanged concept e nunison in 1 © control l yhand by cotan- e ie on lined .This ]. user’s a which , mini- a eneric itions ition. eof se trya- .M Ross M. I. asi- red ed ed . eatmbls otrcnl,i a enue osolve to used been has it recently, Most automobiles. ce 1 il u ucl oet rmnnei 07atrNASA after 2007 in prominence to rose quickly but ciple © ttodcds IOsagrtm aeeovdfrom evolved have algorithms DIDO’s decades, two st utfctdeggmn ffs pcrlHamilto- spectral fast of engagement multifaceted a o iui fcdn n iln knobs[ dialing o and focus ene coding to wasteful of user expend minutia a than for rather important more “problem-of-problems” as is the problem it hand- because iterative a is of solve This amount must user. least the software with good possible DID as a ciently Principle. that Pontryagin’s fu philosophy its of the to toolbox understanding on DIDO clear the use a to employ needed is to is ity necessary that k apparatus is only no workings The however, tively. inner DIDO; its of of details edge algorithmic and mathematical as hwhwDD a saetetapnso lclmnmm an solution. minimum” viable a “local present a to of side trappings other the the on escape emerge can DIDO how show Sec. inlclarity. tional rvln aemnproblem[ salesman traveling h ai otnostm piiainvralsaepack are variables called structure optimization DIDO continuous-time basic The besi unu oto ouhrn ai,non- rapid, ushering to control quantum in oblems problem ig fa“oa iiu”ta nnr ayother many ensnare that minimum” “local a of pings pia oto Toolbox Control Optimal nacmnst IOta en t mathematics its define that DIDO to enhancements b l rbe rmrbtc sicue oshowcase to included is robotics from problem ple z}|{ ( b a * G lhuhtentto sdhr sfil self-explanatory fairly is here used notation the Although h w-vn rbe a evee sa eeetr”conti “elementary” an as viewed be may problem two-event The eei two-event generic A IO ic hn h olo a rw nap- in grown has toolbox the then, Since DIDO. V I IOsFra n tutrsfra for Structures and Format DIDO’s II. ) fti ae ywyo hleg rbe rmrbtc to robotics from problem challenge a of way by paper this of                                            X x primal primal primal ( = ⊂ ujc to Subject Minimize e h K L R L x N ≤ L 1 ≤ . . . x states time controls rfinl MATLAB er-friendly x , . . . , ≤ e J h ( [ ( K primal x x x 0 a ( 11 ( [ , · x pia oto rbe a eformulated be may problem control optimal N E t eei Problem Generic ) x x + ) , ˙ , ( x , ( 12 f · u ) = x Z u ) t , codn to: according , ( , t 0 ( · u 0 13 0 t f t , ) f U t , u ) t , x ( ( ]. t, , · F x f ) f 0 ( = ⊂ t , t , t , , ( ( p 9 t x p 0 R ) 0 f .Teepit r mlfidin amplified are points These ]. ) ←− ←− ←− ) r u ( t , , t , , N t ≤ u 1 ≤ p ) f teaching- f u u , . . . , , ( := ] , , h t u e p p ) U U ( t, , ] ) t ≤ ) t, , p N K ) u p U ) ) dt x t u e [ see , ) ( (

              

· · uigb the by tuning ) ) dynamics events path functional preamble sbased is O cost g nthe on rgy 9 nuous-time di the in ed solving n o addi- for ] lcapac- ll nowl- effec- (1b) effi- (1a) (1c) d 2

Additional optimization variables packed in primal are:

primal.initial.states x0 (2a) ←− primal.initial.time t0 (2b) ←− primal.final.states xf (2c) ←− primal.final.time tf (2d) ←− primal.parameters p (2e) ←− The names of the five functions that define Problem (G) are stipulated in the structure problem according to:

problem.cost [E(x0, xf ,t0,tf , p), ← F (x(t), u(t), t, p)] (3a) problem.dynamics f(x(t), u(t), t, p) (3b) ← problem.events e(x0, xf ,t0,tf , p) (3c) ← problem.path h(x(t), u(t), t, p) (optional) ← (3d)

problem.functionals K[x( ), u( ),t0,tf , p] (optional) ← · · (3e) Figure 1. Sample media coverage of NASA’s successful flight The bounds on the constraint functions of Problem (G) are also asso- operations using PS optimal control theory as implemented ciated with the structure problem but under its fieldname of bounds in DIDO. (i.e. problem.bounds) given byc,

bounds.events [eL, eU ] (4a) ← Although proving Pontryagin’s Principle is quite hard, its ap- bounds.path [hL, hU ] (optional) (4b) plication to any problem is quite easy! See [9] for details on the ← L U “HAMVET” process. The first step in this process is to construct the bounds.functionals [K , K ] (optional) (4c) f ← Pontryagin Hamiltonian. For Problem (G) this is given by , Although it is technically unnecessary, a preamble is highly recom- λ x u p x u p λT f x u p (6) mended and included in all the example problems that come with the H( , , , t, ) := F ( , , t, )+ ( , , t, ) d full toolbox. The preamble is just a simple way to organize all the That is, constructing (6) is as simple as taking a dot product. Forming primal variables in terms of symbols and notation that have a useful the Lagrangian of the Hamiltonian requires taking another dot prod- meaning to the user. In addition, the field constants is included in both uct given by, primal and problem so that problem-specific constants can be man- aged with ease. Thus, problem.constants is where a user defines the H(µ, λ, x, u, t, p) := H(λ, x, u, t, p)+ µT h(x, u, t, p) (7) data that can be accessed anywhere in the user-specific files by way of primal.constants. where, µ is a path covector (multiplier) that satisfies the complemen- tarity conditions, denoted by µ h, and given by, A “search space” must be provided to DIDO using the field search † under problem. The search space for states and controls are specified L 0 if hi(x, u, t, p)= hi by ≤ L U = 0 if hi < hi(x, u, t, p) < hi µ h µi LL UU  U search.states [x , x ] (5a) † ⇔  0 if hi(x, u, t, p)= hi ←  ≥ L U LL UU unrestricted if hi = hi search.controls [u , u ] (5b) (8) ←  LL UU Constructing the adjoint equation now requires some calculus of gen- where, (x, u) and (x, u) are nonactive bounds on the state and erating gradients: control variables; hence, they must be nonbinding constraints. ∂H The inputs to DIDO are simply two structures given by problem λ˙ = (9) and algorithm. The algorithm input provides some minor ways to test − ∂x certain algorithmic options. The reason the algorithm structure is not The adjoint covector (costate) satisfies the tranversality conditions extensive is precisely this: To maintain the DIDO philosophy of not given by, wasting time in fiddling with various knobs that only serves to distract ∂E ∂E a user from solving the problem. That is, it is not good-enough for λ(t0)= λ(tf )= (10) DIDO to run efficiently; it must also provide a user a quick and easy − ∂x0 ∂xf approach in taking a problem from concept-to-codee. where, is the Endpoint Lagrangian constructed by taking yet an- Before taking a concept to code fast, it is critically important to E other dot product given by, perform a mathematical analysis of the problem; i.e., an investigation of the necessary conditions. This investigation is necessary (pun in- E(ν, x0, x ,t0,t , p) := E(x0, x ,t0,t , p) tended!) not only to understand DIDO’s outputs but also to eliminate f f f f T common conceptual errors that can easily be detected by a quick rote + ν e(x0, xf ,t0,tf , p) (11) application of Pontryagin’s Principle. and ν is the endpoint covector that satisfies the complementarity con- cBecause some DIDO clones still employ the old DIDO format (e.g., dition ν e (see (8)). The values of the Hamiltonian at the endpoints “bounds.lower” and “bounds.upper”), the user-specific codes must be remapped also satisfy† transversality conditions known as the Hamiltonian value to run on such imitation software. conditions given by, dDIDOLite, the free version of DIDO, has limited capability. eWarning: The quick and easy concept-to-code philosophy does not mean a ∂E ∂E [@t0]= [@tf ]= (and ν e) (12) user should code a problem first and ask questions later. DIDO is user-friendly, H ∂t0 −H ∂tf † but it’s not for dummies! Nothing beats analysis before coding; see Sec. 4.1.4 of [9] for a simple “counter” example and Sec. V of this paper for the recom- fWe ignore the functional constraints (K[·]) in the rest of this paper for mended process. simplicity of presentation. 3 where, is the lower Hamiltonian[9, 14]. The Hamiltonian mini- adding two new variables. The computational cost for adding these mizationH condition, which is frequently and famously stated as two variables is quite cheap; see Sec. VI.C for details. The linear components are handled separately and efficiently through the use of Minimize H(λ, x, u, t, p) N N Birkhoff basis functions[15, 16, 17, 18, 19]. DIDO then (HMC) u (13) ∈ Subject to u U focuses on solving the nonlinear equations with the linear equations  ∈ tagging along for feasibility. Of all the equations, special attention is reduces to the function-parameterized nonlinear optimization problem paid to the the Hamiltonian minimization condition; see (14). This given by, is because the gradient of the cost function is damped by a “step size” when associated with the gradient of the Hamiltonian[20]. Thus, unlike Minimize H(λ, x, u, t, p) nonlinear programming methods that treat all variables the same way, (HMC for (G)) u Subject to hL h(x, u, t, p) hU DIDO’s Hamiltonian programming method treats different variables ( ≤ ≤ (14) differently. This is because DIDO generates processes that are cen- The necessary conditions for the subproblem stated in (14) are given tered around a Hamiltonian while a nonlinear programming algorithm by the Karush-Kuhn-Tucker (KKT) conditions: is based on equations generated by a Lagrangian. Because a Hamilto- nian is not a Lagrangian, DIDO treats a state variable differently than ∂H a control variable. Similarly, a dynamics function is handled differ- = 0 µ h (15) ently than a path-constraint function. These differences in treatments ∂u † of different functions and constraints are in sharp contrast to nonlin- Similarly, the necessary condition for selecting an optimal parameter p ear programming methods where all variables and functions are treated is given by, identically.h The mathematical details of this computational theory are ∂E tf ∂H described in Appendix A. + dt = 0 (16) ∂p ∂p In addition to treating the optimal control variables differently, Zt0   DIDO also treats the Birkhoff pseudospectral problem differently for Finally, the Hamiltonian evolution equation, different values of N and the “state” of the iteration. As noted in Sec. I, DIDO’s algorithm is actually a suite of algorithms. There are three ma- d ∂H H = (17) jor algorithmic components to DIDO’s main algorithm. A schematic dt ∂t that explains the three components is shown in Fig. 2. The basic idea behind the three-component formula is to address the three different completes the HAMVET process defined in [9]. i All of these necessary conditions can be easily checked by running objectives of DIDO, namely, to be guess-free, fast and accurate. The DIDO using the single line command,

[cost, primal, dual]= dido problem, algorithm (18) where cost is the (scalar) value of J,primal is as defined
in (1) and stabilization (2), and dual is the output structure of DIDO that packs the entirety of component the dual variables in the following format: accuracy dual.Hamiltonian [@t] (19a) ←− H component dual.dyanmics λ( ) (19b) ←− · dual.events ν (19c) search path ←− dual.path µ( ) (19d) ←− · acceleration component Note that the necessary conditions are not supplied to DIDO; rather, the code does all of the hard work in generating the dual information. It is the task of the user to check the optimality of the computed solu- Figure 2. A schematic for the concept of the three main algo- tion, and potentially discard the answer should the candidate optimal rithmic components of DIDO. solution fail the test. functionality of the three major components of DIDO are defined as III. Overview of DIDO’s Algorithm follows: From Secs. I and II, it is clear that DIDO interacts with a user in 1. The stabilization component: The task of this component of much the same way as a direct method but generates outputs similar the algorithm is to drive an “arbitrary” point to an “acceptable” to an indirect method. In this regard, DIDO is both a direct and an starting point for the acceleration component. indirect method. Alternatively, DIDO is neither a direct nor an indi- rect method! To better understand the last two statements and DIDO’s 2. The acceleration component: The task of this suite of algorithms unique suite of algorithms, recall from Sec. II that the totality of vari- is to rapidly guide the sequence of functional iterates to a capture ables in an optimal control problem are, zone of the accurate component. 3. The accurate component: The task of this component is to gen- x( ), u( ),t0,tf , p, λ( ), µ( ), and ν · · · · erate and refine a solution that satisfies the precision requested Internally, DIDO introduces two more function variables v( ) and ω( ) by the user. g · · by rewriting the dynamics and adjoint equations as, DIDO’s main algorithm ties these three components together by moni- toring their progress (i.e., “state” of iteration) and triggering automatic x˙ = v; v = ∂λH (20a) switching between components based on various criteria. Details of all λ˙ = ω; ω = ∂xH (20b) three components of DIDO are described in Appendix B. − The quantities v and ω are called virtual and co-virtual variables re- hAlthough nonlinear programming algorithms may treat linear and nonlinear spectively. Thus, all nonlinear differential equations are split into lin- constraints differently, this concept is agnostic to the differences betweens state ear differentials and nonlinear algebraic components at the price of and control variables. iThe word fast is used here in the sense of true-fast; i.e. fast that is agnostic gThis rewriting is done after a domain transformation; see Appendix A for to the specifics of computer implementations. A more precise meaning of fast details. is defined in Appendix B. 4

IV. Best Practices in Using DIDO where, x := (x,y,θ) is the state of a differential-drive robot, u = Despite the multitude of internal algorithms that comprise DIDO’s (ωr,ωl) is the control and c is a constant[25]. Obstacles the robot must algorithm, the only concept that is needed to use DIDO effectively is avoid during navigation are modeled as circular path constraints[23] of Pontryagin’s Principle[9]. That is, the details of DIDO’s algorithms radius Ri centered at (xi,yi), i = 1,...,Nobs. The robot itself has a are quite unnecessary to use DIDO effectively. Best practices in pro- “bubble” of radius R0; hence, ri := Ri + R0. ducing a good DIDO application code in the least amount of time are: Although it is very tempting to take the posed problem and code it up in DIDO, this is highly not recommended as noted in Sec. II. 1. Using Pontryagin’s Principle to generate the necessary condi- This is because DIDO is not an application software; rather, it is a tions for a given problem as outlined in Sec. II. mathematical toolbox for solving a generic optimal control problem (see Problem (G) posed in Sec. I). Thus, a user is creating an “app” 2. Determining if the totality of necessary conditions are free of using DIDO as the “operating system.” Consequently, an analysis of concept/computational problems; for example, square roots, di- the posed optimal control problem is strongly recommended before vision by zero etc. If yes, then reformulating the problem as coding. This advice is in sharp contrast to the brute-force advocacy of discussed in [9], Chs. 1 and 3. “nonlinear programming methods for optimal control.”

3. Identifying the subset of the necessary conditions that can be easily checked. Examples are constant/linear costate predic- A. Step 1: Pre-Code Analysis tions, switching conditions etc. A pre-code analysis is essentially an analysis of the posed problem us- ing Pontryagin’s Principle[9]. In following the “HAMVET” procedure 4. Scaling and balancing the equations. A detailed process for scal- enunciated in Sec. II, we first construct the Pontryagin Hamiltonian, ing and balancing is described in [21]. λx cos θ + λy sin θ H(λ, x, u) := (ωr + ωl)+ cλθ (ωr ωl) (21) If a problem is not well-balanced, DIDO will execute several subalgo- 2 − rithms to generate an extremal solution. This will effectively bypass its main acceleration component. Tell-tale signs of imbalanced equations where λ := (λx,λy,λθ) is an adjoint covector (costate). The La- are if the costates and/or Hamiltonian values are very large or very grangian of the Hamiltonian is given by, small; e.g., 10±9.

Once DIDO generates a candidate solution, it is critical for a user Nobs to verify and validate (V&V) the computed solution. Best practices for 2 2 H(µ, λ, x, u) := H(λ, x, u)+ µi(x xi) +(y yi) − − post computation are: i=1 X 1. Independently testing the feasibility of the computed solution. + µrωr + µlωl (22) If the solution is not feasible, it is not optimal! where, µ := (µ1,...,µNobs , µr, µl) satisfies the complementarity 2. Testing the optimality of the computed solution using the dual conditions, outputs generated by DIDO. 2 2 2 0 if (x xi) +(y yi) = ri µi ≤ − 2 − 2 2 (23a) DIDO also contains a number of other computational components such = 0 if (x xi) +(y yi) >ri as the DIDO Doctor ToolkitTM to assist the user in diagnosing and help-  − − L ing avoid certain common mistakes in coding and problem formulation. 0 if ωr,l = ω ≤ L U Additional features continue to be added, particularly when new theo- µr,l = 0 if ω <ωr,l <ω (23b) retical advancements are made and/or new improvements in computa-  U  0 if ωr,l = ω tional procedures are possible. ≥ The adjoint equations are now given by,

V. Solving A Challenge Problem from Robotics Nobs ˙ ∂H One of the earliest applications of DIDO that went beyond its λx := = 2µi(x xi) (24a) − ∂x − aerospace boundaries were problems in robotics[22]. A motion plan- i=1 X ning problem with obstacle avoidance can easily be framed as an opti- Nobs ∂H mal control problem with path constraints[23, 24]. A particular time- λ˙ y := = 2µi(y yi) (24b) − ∂y − optimal motion planning problem for a differential-drive robot can be i=1 framed as: X ∂H λx sin θ + λy cos θ λ˙ θ := = − (ωr + ωl) (24c) X = R3 U = R2 − ∂θ 2 preamble x =(x,y,θ) u =(ωr,ωl) ) Hence, from (24) and (23a), we get,

Minimize J[x( ), u( ),t0,tf ] ˙ ˙ · · (λx, λy) =(0, 0) if µi = 0 i = 1,...,Nobs (25) cost ∀ := tf t0  − )  Equation (25) is thus a special necessary condition for Problem (R).   cos θ Any claim of an optimal solution to Problem (R) must be supported by  Subject to x˙ = (ωr + ωl)  2 (25) for mathematical legitimacy. Hence, before writing the first line    sin θ dynamics of code, we expect DIDO to generate constant co-position trajectories problem  y˙ = (ωr + ωl)    t (λx,λy) whenever the robot is sufficiently far from all obstacles.  2  7→ (R)  ˙ Furthermore, over regions where µi = 0, we expect t (λx,λy) to  θ = c(ωr ωl) 6 7→  −  be different. More importantly, because t µ is allowed to be a Dirac z}|{  0 0 0  7→ x(0) = (x ,y ,θ )  delta function (see [9] for details) the co-position trajectories may jump f f f events  x(tf )=(x ,y ,θ ) instantaneously. In addition, because µi(t) 0, the jumps (if any) in   ≤  2 2 2 the co-position trajectories are always positive (upwards) if x(t) > xi  (x xi) +(y yi) ri 0  for λx(t) and y(t) > yi for λy(t). Hence, before any code is writ-  − − − ≥  i = 1,...,Nobs path ten, we expect DIDO to generate constant co-position trajectories with    L U  possible up-jumps (or down-jumps). Note that these are all necessary  ω ωr,l ω   ≤ ≤ conditions for optimality. Stated differently, any candidate solution      5 that cannot satisfy these conditions is certainly not optimal. Conse- quently, when optimality claims are made on computed solutions with- 10 out a demonstration of such necessary conditions, it is questionable, at the very least. Next, from the Hamiltonian minimization condition, we get, 8

λx cos θ + λy sin θ 6 + cλθ + µr = 0 (26a)

2 y 2 1 λx cos θ + λy sin θ cλθ + µl = 0 (26b) 4 2 −

2 If µr(t) = 0 or µl(t) = 0 over a nonzero time interval, then (26) must be analyzed for the existence of singular arcs. If singular arcs are absent, then, from (23b) we get the condition that t (ωr,ωl) must 0 be “bang-bang” and in accordance with the “law:” 7→ -6 -4 -2 0 2 4 6 x L ω if µr,l(t) 0 ωr,l(t)= U ≤ (27) ω if µr,l(t) 0 Figure 3. “Trick” boundary conditions and obstacles set up for Prob-  ≥ lem (R). Finally, combining the Hamiltonian value condition with the Hamiltonian evolution equation, we get the condition, C. Step 3: Generating DIDO’s Guess-Free Solution [@t]= 1 (28) Because DIDO is guess-free, it requires no initialization. Thus, one H − only defines the problem as stated and runs DIDO. It is clear from Fig. 4 that DIDO was able to provide the robot a viable path to nav- TM B. Step 2: Scaling/Balancing and Setting up the “Problem of igate around the obstacles. This is DIDO’s Superior To Intuition Problems” capability. At this point, one can simply declare success relative to the The second critical preparatory step in setting up a problem for DIDO is scaling and balancing the equations[21]. This aspect of the the problem set up is discussed extensively in [21] and the reader is strongly encour- aged to read and understand the principles and practices presented in this article. At this point, we simply note that it is often advantageous to “double-scale” a problem. That is, the first scaling is done “by-hand” to reduce the number of design parameters (constants) associated with a problem[9]. Frequently, canonical scaling achieves this task[21]. The second scaling is done numerically to balance the equations[21]. Once a problem is scaled to its lowest number of constants, it is worthwhile to set up a “problem of problems”[9] even if the intent is to solve a single problem. To better explain this statement, observe that although Problem (R) is a specific optimal control problem in the sense of its specificity with respect to Problem (G) defined in Sec. I, one can still create a menu of Problem (R′s) by changing the num- ber of obstacles Nobs, the location of the obstacles (xi,yi), the size of the obstacles Ri, the size of the robot R0, the exact specification of the boundary conditions (x0, xf ) etc. This is the problem of problems. Figure 4. DIDO’s solution to Problem R obtained via its default The pre-code analysis and the necessary conditions derived in Sec. V.A ( ) guess-free mode that illustrates its Superior To IntuitionTM capabil- are agnostic to these changes in the data. Furthermore, once a DIDO- ity. application-specific code is set up, it is easy to change such data and generate a large volume of plots and results. This volume of data can be reduced if the problem is first scaled by hand to reduce the number posed challenge and “collect the reward.” Nonetheless, in noting that of constants down to its lowest level[9]. In any case, the new challenge DIDO is purportedly capable of solving optimal control problems, it is generated by DIDO-runs is a process to keep track of all these varia- convenient to use Problem (R) to illustrate how the “path” shown in tions. This problem is outside the scope of this paper. The problem that Fig. 4 may have actually solved the stated problem. In effect, there are is in scope is a challenge problem, posed by D. Robinson[26]. Robin- (at least) two steps in this verification and validation process (known son’s challenge comprises a clever choice of boundary conditions and widely as V&V): the number, location and size of the obstacles. The challenge problem is illustrated in Fig. 3. 1. Using a process that is independent of the optimization, show As shown in Fig. 3, two obstacles are placed in a figure-8 configu- that the computed solution is feasible; and ration and the boundary conditions for the robot are on opposite sides of the composite obstacle. The idea is to “trick” a user in setting up 2. Show that the necessary conditions (as many as possiblej) are a guess in the straight line path shown in blue. In using this guess, satisfied. an optimal control solver would get trapped by straight line solutions and would claim to generate an optimal solution by “jumping” over In engineering applications, Step 1 is quite critical for safety, reliability the single intersection point of the figure-8 configuration. Attempts at and success of operations. Furthermore, it is also a critical feature of “mesh refinement” would simply redistribute the mesh so that the dis- any numerical analysis procedure to “independently” check the validity crete solution would continue to jump across the single point obstacle of a solution. in the middle. According to Robinson, all optimal control codes failed to solve this problem with such an initialization. jSee, for example, (25), (27) and (28). 6

D. Step 4: Producing an Independent Feasibility Test x y θ To demonstrate the independent feasibility of a candidate solution to Target 10.0000 0.0000 0.0000 an optimal control problem, we simply take the computed control tra- ♯ DIDO jectory t u and propagate the initial conditions through the ODE, 10.0000 0.0000 0.0000 7→ (optimization) ♯ 0 x˙ = f(x, u (t)), x(t0)= x (29) Propagated 09.993 0.02876 -0.07314 Let t x♯ be the propagated trajectory; i.e., one obtained by solving (true) the initial7→ value problem given by (29). Then, x♯( ) is the truth solution that is used to test if other constraints (e.g. terminal· conditions, path Table 1. True accuracy vs optimization errors constraints etc.) are indeed satisfied. In performing this test for Problem (R), we first need the DIDO- computed control trajectory. This is shown in Fig. 5. It is clear that both cluding DIDO) must be viewed with a significant amount of skepti- cism because they do not reflect reality. Despite this apparently obvi- 1.5 ous statement, outrageously small errors that are not grounded in basic science or computational mathematics are frequently reported. For in- r 1 stance, in [27], a minimum time of 324.9750302 seconds is reported l (pg. 146). The purported accuracy of this number is ten significant dig- 0.5 its. Furthermore, a clock to measure this number must have a range of a few hundred seconds and an accuracy of greater than one microsec- u 0 ond! To better understand the meaning of “small” errors (reasonable vs ridiculous), recall that a scalar control function u( ) is computationally -0.5 N · N represented as a vector, U := (u0, u1,...,uN ). Let U⋆ be the opti- mal value of U N for a fixed N. Then, a Taylor-series expansion of the -1 cost function U N J N around U N may be written as, 7→ ⋆ -1.5 N N N N N N N N 0 5 10 15 J (U )= J (U )+ ∂J (U ) (U U ) ⋆ ⋆ · − ⋆ t 2 + (1/2)(U N U N ) ∂ J N (U N ) (U N U N ) − ⋆ · ⋆ · − ⋆ + higher order terms (30) Figure 5. Candidate DIDO control trajectory to Problem (R). Setting ∂J N (U N ) = 0 in (30), we get, controls take on “bang-bang” values with no apparent singular arcs. ⋆ 2 Remark 1 It is abundantly clear from Fig. 5 that DIDO is easily able N N N N 2 N N N N 2 J (U ) J (U⋆ ) = U ∂ J (U⋆ ) U U U⋆ to represent discontinuous controls. Consequently, the frequent lamen- − · · − tation in the literature that PS methods cannot represent discontinuities + higher order terms (31) b b in controls is quite false. where U is a unit vector. In a perfectly conditioned system, the Hessian Using t (ω ,ω ) shown in Fig. 5 to propagate the initial conditions r l term in (31) is unity; hence, in this best-case scenario we havel, through7→ the “dynamics” of the differential drive robot generates the so- lution shown in Fig. 6. The independent propagator used to generate b k r N N N N N N Fig. 6 was ode45 in MATLAB . Also shown in Fig. 6 are the prop- U U J (U ) J (U⋆ ) (32) k − ⋆ k∼O −   q

10 Let ǫM be the machine precision. Then from (32), we can write, X 12.43 x (ode) Y 9.993 N N −8 y (ode) U U⋆ > (√ǫM ) 10 (33) 5 k − k O ∼ −16 position where, the numerical value in (33) is obtained by setting ǫM 10 X 12.43 (double-precision, floating-point arithmetic). ∼ Y 0.02876 N th 0 The vector U⋆ is only an “N degree” approximation to the 0 2 4 6 8 10 12 optimal function . The “best” convergence rate in approxima- t u( ) tion of functions is· exponential[29]. This record is held by PS 3 approximations[29, 30, 31] because they have near-exponential or “spectral” rates of convergence (assuming no segmentation, no knotted 2 (ode) concatenation of low-order polynomials for mesh-refinement[6, 32], perfect implementation etc.). In contrast, Runge-Kutta rates of ap- attitude 1 proximation are typically[33] only (4) or (5) (fourth-order or fifth- X 12.43 O O Y -0.07314 order). Assuming convergence to within a digit (very optimistically!) 0 0 2 4 6 8 10 12 we get, t N −7 U (t) u(t) > 10 (√ǫM ) 10 (34) − ×O ∼

N Figure 6. Propagated state trajectory using controls from Fig. 5. where, we have abused notation for clarity in using U (t) to imply an interpolated function. In practice, the error may be worse than 10−7 (seven significant digits) if the derivatives of the data functions in a agated values of the terminal states. The errors in the terminal values given problem (gradient/Jacobian) are computed by finite differencing. of the propagated states are the “true” errors. They are significantly Of course, none of these errors take modeling errors into account! An higher than those obtained by the optimization process as indicated in independent feasibility test is therefore a quick and simple approach Table 1. The error values generated by any optimization process (in- for estimating the true feasibility errors of an optimized solution. Any kDifferent propagators can generate different results; this aspect of solving (29) is not taken into account in the ensuing analysis. lEquation (32) is well-known in optimization; see, for example [28]. 7 errors reported without such a test must be viewed with great suspi- 1.5 cion!m Given the arguments of the preceding paragraphs, it is apparent that l 1 a “very small” error that qualifies for the term “very accurate” control l may be written as, 0.5 U N (t) u(t) 10−6 (35) k − k∼ 0 Consequently, achieving an accuracy for u(t) beyond six significant digits in a generic problem is highly unlikely. Furthermore, recall that -0.5 the estimate given by (35) is based on perfect/optimistic intermediate steps! Therefore, practical errors may be worse than 10−6. Regardless, -1 to achieve high accuracy, the problem must be well conditioned. To achieve better conditioning, the problem must be properly scaled and -1.5 0 5 10 15 balanced regardless of which optimization software is used. See[21] t for details. Providing DIDO a well-scaled and balanced problem also triggers the best of its true-fast suite of algorithms. Figure 8. Verification of (27) for the (ωl(·), µl(·)) pair generated by Remark 2 The optimistic error estimate given by (34) or (35) does DIDO. not mean greater accuracy is not possible. The main point of (35) is to treat “accuracy” claims of computed solutions with great caution. It is also important to note that the practical limit given by (35) does the candidate solution shown in Fig. 4 that if this trajectory is optimal, not preclude the possibility of achieving highly precise solutions. See, we require µ2(t) = 0 over the entire time interval [t0,t ]. That this is for example, [34] where accuracies of the order of −7 radians were f 10 indeed true is shown in Fig. 9. Also shown in Fig. 9 is the path covector achieved in pointing the Kepler space telescope. function µ1( ) generated by DIDO. Note first that µ1(t) 0 t as re- · ≤ ∀ E. Step 5: Testing for Optimality 0.5 This step closes the loop with Step 1. In the pre-code analysis step, we noted (see (25)) that the co-position trajectories must be constants with potential jumps. Recall that (see (18)) DIDO automatically generates 0 all the duals as part of its outputs. Thus, the dual variables that are of interest in testing (25) is contained in dual.dynamics. A plot of x the costate trajectories generated by DIDO is shown in Fig. 7. It is y -0.5 1 0.5 2 -1

0 -1.5 x 0 5 10 15 y t

-0.5 Figure 9. DIDO predicts µ1(t) to be the sum of two impulse func- tions centered at the jump points of the co-position trajectories t 7→ (λx, λy). -1

0 5 10 15 quired by (23a). This function is zero everywhere except for two “sharp t peaks” that are located exactly where the co-position trajectories jump. These sharp peaks are approximations to atomic measures[9]. As noted in [9], page 141, we can write µ( ) as the sum of two functions: one Figure 7. DIDO-generated costate trajectories for Problem (R). in L∞ and another as a finite sum· of impulse functions. From this concept, it follows that apparent from this figure that the co-position trajectories t (λx,λy) 7→ are indeed constants with jumps over two spots. If tc is the time when µ1(t)= ηj δD(t tj ) (36) a jump occurs, then it follows that λy(t) must jump upwards because − j y(tc) > yi where tc is the point in time when the robot “bubble” just X touches the obstacle bubble. In the case of the x-coordinates, we have where δD is the Dirac delta function centered at tj and ηj is finite, the x(tc) < xi at the first touch of the bubbles; see Fig. 4. Hence, λx(t) value of the impulse[9]. From (24), (36) and the signs of (x(tj ) x1) must jump downward, which it indeed does in Fig. 7. Subsequently, − and (y(tj) y1) (see Fig. 4) it follows that λx(t) must jump downwards at the second touch of the bubbles x(tc) > xi; hence, λx(t) jumps − upwards. first and upwards afterwards while λy(t) must jump upwards at both Because DIDO also outputs µ( ) in dual.path, it is a simple mat- touch points. This is exactly the solution generated by DIDO in Fig. 9. · A useful teaching aspect of Fig. 9 is its ability to explain difficult ter to test (27). The path covector trajectory µl( ) is plotted in Fig. 8 · theoretical aspects of measure theory[14] that are of practical value over the graph of ωl( ). It is clear from this plot that µl( ) is indeed a · · in understanding the fundamentals of optimal control theory. switching function for ωl( ). The same holds for ωr and µr. This plot is omitted for brevity. · Yet another verification of the optimality of the computed trajec- Because there are two obstacles in the challenge problem (modeled tory is shown in Fig. 10. According to (28), the lower Hamiltonian must be a constant and equal to 1. DIDO’s computation of the Hamil- as two inequalities), dual.path also contains µ1( ) and µ2( ) that sat- − isfy the complementarity conditions given by (23a·). It is apparent· from tonian evolution equation is not perfect as apparent from Fig. 10. There are several reasons for this[5]. With regards to Fig. 10 specifically, one mIt is possible for the errors given by (32) (and hence (34)) to be smaller in of them is, at least, in part because the computations of the Dirac delta certain special cases. functions in DIDO not perfect as evident from Fig. 9. 8

0 B. Comparison With Other Methods and Software It is forgone conclusion that every author of a code/method/software will claim superiority. In maintaining greater objectivity, we simply -0.5 identify scientific facts regarding DIDO, PS theory and its implemen- tations.

-1 1. PSvsRK;PSvsPS H[@t] DIDO is based on PS discretizations. The convergence of a PS dis- -1.5 cretization is spectral; i.e., almost exponentially fast[29, 30, 31]. Con- vergence of Runge-Kutta (RK) discretizations are typically (4). Con- sequently, it is mathematically impossible for a fourth-orOder method to converge faster than a higher-order method, particularly an “ex- -2 0 5 10 15 ponentially fast” method. Claims to the contrary are based on a com- t bination of one or more of the following:

Figure 10. DIDO-computed evolution of the lower Hamiltonian t 7→ i) Not all PS discretizations are convergent just as not all RK meth- H[@t] is on average a constant and equal to −1. ods are convergent[33]. Unfortunately, even a convergent PS method may be inappropriate for a given problem[35] because of mathematical technicalities associated with inner products and boundary conditions[36]. By using an inappropriate PS discretiza- VI. Miscellaneous tion over a set of numerical examples, it possible to show that it performs poorly or similarly to a convergent RK method[37]. DIDO’s algorithm as implemented in DIDO is not perfect! In recognition of this distinction, it should not be surprise to any user that ii) The “exponential convergence rate” of a PS discretization is based DIDO has bugs. It should also not be a surprise to any reader that on the order of the interpolating polynomial: higher the order, the DIDO and/or PS methods get frequently compared to other software, better the approximation[29]. Consequently, an RK method can be methods etc. In this section we address such miscellaneous items. easily made to outperform a sufficiently low-order PS discretiza- tion that is based on highly-segmented mesh refinements; see [18] for details. A. Known Bugs In general, DIDO will run smoothly if the coded problem is free of iii) In a correct PS discretization, the control is not necessarily repre- singularities, well posed, scaled and balanced. Both beginners and ad- sented by a polynomial[35]; this fact has been known since at least vanced users can avoid costly mistakes in following the best practices 2006[38]. It is possible to generate a bad PS discretization by set- as outlined in Sec. IV, together with using the DIDO Doctor Toolkit,TM ting the control trajectory to be a polynomial. In this case, even an and a preamble in coding the DIDO files. Even after exercising such Euler method can be made to outperform a PS discretization. care, a user may inadvertently make a mistake that is not detected by the DIDO Doctor Toolkit. In this case, DIDO may exit or quit with iv) Low order PS discretizations are claimed to outperform high-order cryptic error messages. Examples of these are: PS methods based on the assumption that a high-order method ex- hibits a Gibbs phenomenon. It is apparent from Fig. 5 that there is no Gibbs phenomenon in a properly implemented PS discretiza- 1. Grid Interpolant Error tion.

This problem occurs when a user allows the possibility of tf t0 = 0 in the form of the bounds on the initial and final time. The− simple 2. Implementation vs Theory; Theory vs Implementation remedy is to ensure that the bounds are set up so that tf t0 > 0. PS discretizations are deceptively simple. Its proper implementa- − tion requires a significant amount of computational finesse[17, 19, 2. Growthcheck Error 29, 39, 40, 41]. For instance, in Lagrange PS methods (see Fig. 13 in Appendix A), it is very easy to compute Gaussian grid points[39, This is frequently triggered because of an incorrect problem formula- 40] poorly, or generate differentiation matrices with large round-off tion that the DIDO Doctor Toolkit is unable to detect. Best remedy is errors[41], or or implement closure conditions[36, 42] incorrectly. In to check the coded problem formulation (and ensure that it is the same doing so, it is easy to show poor numerical performance of theoretically as one conceived). Example: if “x” is coded as “y.” convergent PS discretizations. Furthermore it is easy to show large run times of fast PS discretizations by simply patching it with generic nonlinear programming solvers. It is also easy to claim poor computa- 3. SOL Error tional performance of PS theory by using DIDO clones or other poor implementations of PS techniques[37]. It is also easy to show poor This is not a bug; rather, it is simply as an error message where DIDO performance of DIDO by improper scaling or balancing or other inap- has gone wrong so badly that it is unable to provide any useful feedback propriate use of the toolbox. to a user. The best remedy for a user is to start over or to go back toa To maintain scientific integrity, it is important to compare and con- prior version of his/her files to start over. trast performance of trajectory optimization methods in a manner that is agnostic to its implementation and/or computational hardware. For in- 4. Hamiltonian Error stance, in static optimization, it is a scientific fact that a Newton method converges faster than a gradient method irrespective of its implemen- This is neither a bug nor an error but is listed here for information. Con- tation on a computer[28]. This is because computational speed is de- sider the case where the lower Hamiltonian is expected to be a constant. fined precisely in terms of the order of convergence and not in terms The computed Hamiltonian may undergo small jumps and oscillations of computational time. Furthermore, each Newton iteration is expen- around the expected constant. These jumps and oscillations are ampli- sive relative to a gradient method because it involves the computation fied for minimum time problems and/or when one or more of the path of a Hessian and solving a system of linear equations[28]. This fact re- covector functions contain Dirac-delta functions as evident in Fig. 9. mains unchanged no matter the implementation or the computer hard- When these are absent, the lower Hamiltonian computed by DIDO is ware. Similar to its static optimization counterpart, it is indeed possible reasonably accurate; see [9] for additional details on impulsive path to perform computer-agnostic analysis of dynamic optimization meth- covector functions and for the meaning of the various Hamiltonians. ods; see [20] for an example of a mathematical procedure for such an 9 analysis. This topic in trajectory optimization is mostly uncharted also served as discovery tool for other NASA firsts; see the Feature and remains a wide open area of research. Nonetheless, there is no Article in IEEE Spectrum[48]. shortage of papers that use “alternative facts” and/or compare selective performance of codes as part of a self-fulfilling prophecy. iv) The 2008 version of DIDO was the first guess-free, general- purpose optimal control solver[7]. To the best of the author’s knowledge, DIDO remains the only such software well after a 3. DIDO T PS decade! It is quite possible for an algorithm to perform better than theory (e.g. v) The first general-purpose optimal control solver to be embed- simplex vs ellipsoid algorithm) or for a code to perform worse than the- ded on a microprocessor was the 2011 version of DIDO’s algo- ory. The latter case is also true if a theory is implemented incorrectly, rithm. See [35] for details on the computer architecture, embed- poorly or imperfectly. Consequently, using a code to draw conclusions ding methodology, proof-of-concept and run-time performance about a theory is not only unscientific, it may also be quite misleading. improvements achieved by embedding. DIDO is not immune to this fact. Furthermore, because PS discretiza- tion is only an element of DIDO’s algorithm, it is misleading to draw C. Computer Memory Requirements conclusions about PS theory using DIDO. This fact is strengthened by the point noted earlier that even DIDO’s algorithm as implemented in From (49) in Appendix A, it follows that the memory requirements for the variables is (N), where (N + 1) is the number of points DIDO is not perfect! In fact, even the implementation is not complete! O Despite all these caveats, it is still very easy to refute some of the claims that constitute the grid (in time). This estimate can be refined further to (nN) where n is the number of continuous-time variables. For made in the literature by simply running DIDO for the purportedly hard O instance, for Problem (G), n (4Nx + Nu + Nh). Assuming 8 bytes problems. For example, the claims made in [37] were easily refuted in ∼ [43] by simply using DIDO. per variable, the memory requirements for variables can be written as (nN 10) bytes with 8 replaced by 10. Thus, for example, for N = O1, 000,× the memory estimate can be written as 10n KB or 10 KB per 4. DIDO vs DIDO’s algorithm continuous variable. Clearly, the number of variables is not a serious In principle, DIDO’s algorithm can generate “near exact” solutions;n limiting factor for DIDO’s algorithm. This is part of the reason why however, its precision is machine limited by the fundamental “square- it was argued in [49] that the production of solutions over a million root equation” given by (32). In practice, the precision is further lim- grid points was more connected to numerical science than computer ited than the square-root formula because of other machine and ap- technology. proximation errors (see (35)). In addition, DIDO’s algorithm, as im- Carrying out an analysis similar to that of the previous paragraph, plemented in DIDO, the toolbox, is practically “tuned” to generate the it is straightforward to show that the number of constraints is also best performance for “industrial strength” problems. This is because not a significant limiting factor to execute DIDO’s algorithm. By the the toolbox, is designed to work across the “space of all optimal con- same token, the gradients of the nonlinear functions (see (49)) are also not memory-intensive because Hamiltonian programming naturally ex- trol problems.” Consequently, it is not too hard to design an academic λ,D,N problem that shows a “poor performance” of DIDO, the toolbox, even ploits the separability property inherent in (P ). This conclusion carries over to Problem (G) as well (see also [19]). Thus, the dominant though DIDO’s algorithm may be capable of significantly superior per- ⋆ formance. sources of memory consumption are the matrices A and A introduced in (47) and (48) respectively (see Appendix A). In choosing discretiza- tions that satisfy Theorem 1, the memory requirements for the linear 5. First Facts: DIDO vs Other Software 2 2 system given in (49) is 2N variables, and hence 2NxN for solving As the old adage (popularized by John F. Kennedy) goes, success has Problem (G). In using this number and the analysis of the previous many fathers but failure is an orphan. Invariably, every author will paragraph, it follows that the memory cost for a grid size of 250 points claim something “first” or something “better” on a successful idea. For- is about 1 MB per state variable (and independent of the number of tunately, something first is easily verifiable. The following milestones other variables). For 1, 000 grid points, the memory requirement is document some of DIDO’s firsts: about 10 MB per state variable. Because its memory requirements are relatively low and conver- i) The first PS optimal control solver was DIDO (this includes the gence is fast, it follows that DIDO’s algorithms can be easily embed- pre-2001 version[44]). All optimal control solvers prior to DIDO ded on a low end microprocessor. Specific details on a generic com- were based on RK-type methods[45]. This fact is evident in the puter architecture for embedded optimal control are described in [35]. highly-regarded 1998 survey paper[45] where there is no mention As noted earlier, it is important to note the critical difference between of PS methods;o see also the 2001 book[27] for an absence of any DIDO’s algorithms and DIDO, the optimal control toolbox. The lat- discussion on PS optimal control theory. ter is not designed for embedding while the former is embed-ready. A proof-of-concept in embedding was shown in [35]. Consequently, it is ii) The 2001 version of DIDO was the first object-oriented, general- technically feasible to generate real-time optimal controls by coordi- purpose optimal control software[9, 35, 46]. Prior to DIDO 2001, nating the Nyquist frequency requirements of digitized signals to the users had to provide files and variables in a convoluted fash- Lipschitz frequency production of optimal controls[9]. The mathemat- ion to nonlinear programming solvers; see, for example, Ap- ical foundations of this theory along with some experimental results are pendix A.3 of [27]. After it attained widespread publicity in 2006- described in [50] and [51]; see also [9], Sec. 1.5 and research articles 07 (see Fig. 1), DIDO’s format and structures (see Sec. II) were on the Bellman pseudospectral method[52, 53]. rapidly cloned[9]. Subsequently, the object-oriented methodology of DIDO was adopted as a computer-programming standard with varying modifications. VII. Conclusions iii) The first flight-proven general-purpose optimal control solver was DIDO’s solution to the “space station problem” that created the 2006 version of DIDO[1, 2, 3]. While other trajectory codes headlines in 2007 is now widely used by other codes to benchmark have been in the work-flow at NASA[47], they have all been spe- their performance. The 2001–2007 versions of DIDO have been ex- cial purpose tools. DIDO is the first general-purpose optimal con- tensively cited as a model for new software. The early versions of the trol solver to be inserted in flight operations. Incidentally, it has software were indeed based on the admittedly naive approach of patch- ing nonlinear programming solvers to pseudospectral discretizations. nThe statement follows from the fact that DIDO’s algorithm is grounded on Later versions of the optimal control toolbox execute a suite of funda- the convergence property of the Stone-Weierstrass theorem and approximations mentally different and multifaceted algorithms that are focused on be- in Sobolev spaces; see [35]. ing guess-free, true-fast and verifiably accurate. Despite major internal oBecause survey papers typically look backwards, they have the unfortunate changes, all versions of DIDO have remained true to its founding prin- side-effect of being obsolete the moment they get published. ciple: to facilitate a fast and quick process in taking a concept to code 10 using nothing but Pontryagin’s Principle as a central analysis tool. In [9] I. M. Ross, A Primer on Pontryagin’s Principle in Optimal Con- this context, DIDO as a toolbox is more about a minimalist’s approach trol, Second Edition, Collegiate Publishers, San Francisco, CA, to solving optimal control problems rather than pseudospectral theory 2015. or its implementation. This is the main reason why the best way to use DIDO is via Pontryagin’s Principle!p [10] L. S. Pontryagin, V. G. Boltayanskii, R. V. Gamkrelidze and E. Because DIDO does not ask a user to supply the necessary con- F. Mishchenko, The Mathematical Theory of Optimal Processes, ditions of optimality, it is frequently confused for a “direct” method. Wiley, 1962 (Translated from Russian). Ironically, it is also confused with an “indirect” method because DIDO also provides costates and other multipliers associated with Pontrya- [11] I. M. Ross, R. J. Proulx and M. Karpenko, “An optimal con- gin’s Principle. From the details provided in this paper, it is clear trol theory for the Traveling Salesman Problem and its variants,” that DIDO does not fit within the traditional stovepipes of direct and arXiv preprint (2020) arXiv:2005.03186. indirect methods. DIDO achieves its true-fast computational speed [12] I. M. Ross, R. J. Proulx and M. Karpenko, “Autonomous UAV without requiring a user to provide a “guess,” or analytical gradi- Sensor Planning, Scheduling and Maneuvering: An Obstacle ent/Jacobian/Hessian information or demanding a high-performance Engagement Technique,” 2019 American Control Conference, computer; rather, the “formula” behind DIDO’s robustness, speed and Philadelphia, PA, USA, 2019, pp. 65–70. accuracy is based on a Hamiltonian-centric approach to dynamic op- timization whose fundamentals are agnostic to pseudospectral dis- [13] I. M. Ross, M. Karpenko and R. J. Proulx, “A Nonsmooth Calcu- cretizations. This is likely the reason why DIDO is able to solve lus for Solving Some Graph-Theoretic Control Problems,” IFAC- a wide variety of “hard” differential-algebraic problems that other PapersOnLine 49-18, 2016, pp. 462–467. codes/methods have admitted difficulties. [14] F. Clarke, Functional Analysis, Calculus of Variations and Opti- mal Control, Springer-Verlag, London, 2013; Ch. 22. Endnotes [15] G. G. Lorentz and K. L. Zeller, “Birkhoff Interpolation,” SIAM DIDO© is a copyrighted code registered with the United States Journal of Numerical Analysis, Vol. 8, No. 1, pp. 43-48, 1971. Copyright Office. Under Title 17 U.S.C. §107, fair use of DIDO’s for- mat and DIDO’s structures is permitted (with proper citation of this [16] W. F. Finden, “An Error Term and Uniqueness for Hermite- article) but limited to research publications, news articles, scholarship, Birkhoff Interpolation Involving Only Function Values and/or nonprofit or academic use. First Derivative Values,” Journal of Computational and Applied Mathematics, Vol. 212, No. 1, pp. 1-15, 2008.

References [17] L.-L Wang, M. D. Samson and X. Zhao, “A Well-Conditioned [1] N. Bedrossian, S. Bhatt, M. Lammers and L. Nguyen, “Zero Collocation Method Using a Pseudospectral Integration Matrix,” Propellant Maneuver: Flight Results for 180◦ ISS Rotation,” SIAM Journal of Scientific Computaton, Vol. 36, No. 3, pp. A907- NASA CP Report 2007-214158; 20th International Symposium A929, 2014. on Space Flight Dynamics, September 24-28, 2007, Annapolis, MD. [18] N. Koeppen, I. M. Ross, L. C. Wilcox and R. J. Proulx, “Fast Mesh Refinement in Pseudospectral Optimal Control,” Journal of [2] N. Bedrossian, S. Bhatt, M. Lammers, L. Nguyen and Y. Zhang, Guidance, Control and Dynamcis, (42)4, 2019, 711–722. “First Ever Flight Demonstration of Zero Propellant Maneuver [19] I. M. Ross and R. J. Proulx, “Further Results on Fast Birkhoff Attitude Control Concept,” Proceedings of the AIAA Guidance, Pseudospectral Optimal Control Programming,” J. Guid. Control Navigation, and Control Conference, 2007, AIAA 2007-6734. Dyn. 42/9 (2019), 2086–2092.

[3] W. Kang and N. Bedrossian, “Pseudospectral Optimal Control [20] I. M. Ross, “A Direct Shooting Method is Equivalent to An Indi- Theory Makes Debut Flight: Saves NASA $1M in in under 3 rect Method,” arXiv preprint (2020) arXiv:2003.02418. hrs,” SIAM News, page 1, Vol. 40, No. 7, 2007. [21] I. M. Ross, Q. Gong, M. Karpenko and R. J. Proulx, “Scaling [4] N. Bedrossian, S. Bhatt, W. Kang and I. M. Ross, “Zero Pro- and Balancing for High-Performance Computation of Optimal pellant Maneuver Guidance,” IEEE Control Systems Magazine, Controls,” Journal of Guidance, Control and Dynamics, Vol. 41, Vol. 29, Issue 5, October 2009, pp. 53-73. No. 10, 2018, pp. 2086–2097.

[5] I. M. Ross and F. Fahroo, “Legendre Pseudospectral Approxima- [22] L. R. Lewis and I. M. Ross, “A Pseudospectral Method for Real- tions of Optimal Control Problems,” Lecture Notes in Control and Time Motion Planning and Obstacle Avoidance,” Platform In- Information Sciences, Vol. 295, SpringerVerlag, New York, 2003, novations and System Integration for Unmanned Air, Land and pp. 327342. Sea Vehicles (AVT-SCI Joint Symposium) (pp. 10-1–10-22). Meet- ing Proceedings RTO-MP-AVT-146, Paper 10. Neuilly-sur-Seine, [6] I. M. Ross and F. Fahroo, “Pseudospectral Knotting Methods for France: RTO. Solving Optimal Control Problems,” Journal of Guidance, Con- trol and Dynamics, Vol. 27, No. 3, pp. 397-405, 2004. [23] L. R. Lewis, I. M. Ross and Q. Gong, “Pseudospectral motion planning techniques for autonomous obstacle avoidance,” Pro- [7] I. M. Ross and Q. Gong, “Guess-Free Trajectory Optimization,” ceedings of the 46th IEEE Conference on Decision and Control, AIAA/AAS Astrodynamics Specialist Conference and Exhibit, 18- New Orleans, LA, pp. 5997-6002 (2007). 21 August 2008, Honolulu, Hawaii. AIAA 2008-6273. [24] K. Bollino, L. R. Lewis, P. Sekhavat and I. M. Ross, “Pseu- [8] Q. Gong, F. Fahroo and I. M. Ross, “Spectral Algorithm for Pseu- dospectral Optimal Control: A Clear Road for Autonomous In- dospectral Methods in Optimal Control,” Journal of Guidance, telligent Path Planning,” Proc. of the AIAA Infotech@Aerospace Control, and Dynamics, vol. 31 no. 3, pp. 460-471, 2008. 2007 Conference, CA, May 2007.

pDIDO’s philosophy of analysis before coding, as exemplified in Sec. V, [25] K. M. Lynch and F. C. Park, Modern Robotics: Mechanics, Plan- is in sharp contrast to the traditional “brute-force” direct method. The widely- ning, and Control, Cambridge University Press, Cambridge, MA, advertised advantage of traditional direct methods is that a user apparently need 2019. not know/understand any fundamentals of optimal control theory to create/run codes for solving optimal control problems. [26] D. Robinson, Private communication. 11

[27] J. T. Betts, Practical Methods for Optimal Control Using Nonlin- [45] J.T. Betts, “Survey of numerical methods for trajectory optimiza- ear Programming, SIAM, Philadelphia, PA, 2001. tion,” J. Guid. Control Dyn. 21 (1998), 193–207.

[28] P. E. Gill, W. Murray and M. H. Wright, Practical Optimization, [46] I. M. Ross and F. Fahroo, “A Pseudospectral Transformation of Academic Press, London, 1981. the Covectors of Optimal Control Systems,” Proceedings of the First IFAC Symposium on System Structure and Control, Prague, [29] L. N. Trefethen, Approximation Theory and Approximation Prac- Czech Republic, 29-31 August 2001. tice, SIAM, Philadelphia, PA, 2013. [47] R. A. Lugo et al, “Launch Vehicle Ascent Trajectory Using [30] W. Kang, I. M. Ross and Q. Gong, “Pseudospectral Optimal The Program to Optimize Simulated Trajectories II (POST2),” Control and its Convergence Theorems,” Analysis and Design of AAS/AIAA Spaceflight Mechanics Meeting, San Antonio, TX, Nonlinear Control Systems, Springer-Verlag, Berlin Heidelberg, 2017, AAS 17-274. 2008, pp. 109–126. [48] N. Bedrossian, M. Karpenko and S. Bhatt, “Overclock My Satel- [31] W. Kang, “Rate of Convergence for a Legendre Pseudospectral lite: Sophisticated Algorithms Boost Performance on the Cheap,” Optimal Control of Feedback Linearizable Systems,” Journal of IEEE Spectrum, Vol. 49, No. 11, 2012, pp. 48–54. Control Theory and Applications, Vol. 8, No. 4, pp. 391–405, 2010. [49] I. M. Ross, M. Karpenko and R. J. Proulx, “The Million Point Computational Optimal Control Challenge,” SIAM Conference on [32] Q. Gong and I. M. Ross, “Autonomous Pseudospectral Knotting Control and Its Applications, MS24, Pittsburgh, PA, July 2017. Methods for Space Mission Optimization,” Advances in the As- tronatuical Sciences, Vol. 124, 2006, AAS 06-151, pp. 779–794. [50] I. M. Ross, P. Sekhavat, A. Fleming and Q. Gong, “Optimal Feed- back Control: Foundations, Examples, and Experimental Results [33] E. Hairer, S. P. Nørsett and G. Wanner, Solving Ordinary Dif- for a New Approach,” Journal of Guidance, Control and Dynam- ferential Equations I: Nonstiff Problems, Springer-Verlag Berlin ics, Vol. 31, No. 2, March-April 2008. Heidelberg, 1993. [51] I. M. Ross, Q. Gong, F. Fahroo and W. Kang, “Practical Stabi- [34] M. Karpenko, I. M. Ross, E. Stoneking, K. Lebsock and C. J. lization Through Real-Time Optimal Control,” 2006 American Dennehy, “A Micro-Slew Concept for Precision Pointing of the Control Conference, Inst. of Electrical and Electronics Engineers, Kepler Spacecraft,” AAS/AIAA Astrodynamics Specialist Confer- Piscataway, NJ, 1416 June 2006. ence, August 9-13, 2015, Vail, CO. Paper number: AAS-15-628. [52] I. M. Ross, Q. Gong and P. Sekhavat, “The Bellman Pseudospec- [35] I. M. Ross and M. Karpenko, “A Review of Pseudospectral Opti- tral Method,” AIAA/AAS Astrodynamics Specialist Conference mal Control: From Theory to Flight,” Annual Reviews in Control, and Exhibit, Honolulu, Hawaii, AIAA-2008-6448, August 18-21, Vol.36, No.2, pp.182–197, 2012. 2008.

[36] F. Fahroo and I. M. Ross, “Advances in Pseudospectral Methods [53] I. M. Ross, Q. Gong and P. Sekhavat, “Low-Thrust, High- for Optimal Control,” AIAA Guidance, Navigation, and Control Accuracy Trajectory Optimization,” Journal of Guidance, Con- Conference, AIAA Paper 2008–7309, Honolulu, Hawaii, August trol and Dynamics, Vol. 30, No. 4, pp. 921–933, 2007. 2008. [54] I. M. Ross, F. Fahroo and J. Strizzi, “Adaptive Grids for Tra- [37] S. L. Campbell and J. T. Betts, “Comments on Direct Tran- jectory Optimization by Pseudospectral Methods,” AAS/AIAA scription Solution of DAE Constrained Optimal Control Prob- Spaceflight Mechanics Conference, Paper No. AAS 03-142, lems with Two Discretization Approaches,” Numer Algor (2016) Ponce, Puerto Rico, 9-13 February 2003. 73:807-838. [55] Q. Gong, I. M. Ross and F. Fahroo, “Pseudospectral Optimal [38] Q. Gong, W. Kang and I. M. Ross, “A Pseudospectral Method for Control On Arbitrary Grids,” AAS Astrodynamics Specialist Con- the Optimal Control of Constrained Feedback Linearizable Sys- ference, AAS-09-405, 2009. tems,” IEEE Transactions on Automatic Control, Vol. 51, No. 7, July 2006, pp. 1115-1129. [56] Q. Gong, I. M. Ross and F. Fahroo, “Spectral and Pseudospectral Optimal Control Over Arbitrary Grids,” Journal of Optimization [39] I. Bogaert, “Iteration-free computation of Gauss-Legendre Theory and Applications, vol. 169, no. 3, pp. 759-783, 2016. quadrature nodes and weights,” SIAM J. Sci. Comput., 36 (2014), pp. A1008–A1026. [57] F. Stenger, “A ‘Sinc-Galerkin’ Method of Solution of Boundary Value Problems,” Mathematics of Computation, 33/145, pp. 85– [40] A. Glaser, X. Liu, and V. Rokhlin, “A Fast Algorithm for the Cal- 109, 1979. culation of the Roots of Special Functions,” SIAM J. Sci. Comput., 29 (2007), pp. 1420–1438. [58] W. Chen, Z. Shao and L. T. Biegler, “A Bilevel NLP Sensitivity- based Decomposition for Dynamic Optimization with Moving Fi- [41] R. Baltensperger and M. R. Trummer, “Spectral Differencing nite Elements,” AIChE J., 2014, 60, 966–979. with a Twist,” SIAM J. Sci. Comput., 24(5), 14651487, 2003. [59] W. Chen and L. T. Biegler, “Nested Direct Transcription Op- [42] I. M. Ross and F. Fahroo, “Discrete Verification of Necessary timization for Singular Optimal Control Problems,” AIChE J., Conditions for Switched Nonlinear Optimal Control Systems,” 2016, 62, 3611–3627. Proceedings of the American Control Conference, June 2004, Boston, MA. [60] L. Ma, Z. Shao, W.Chen and Z. Song, “Trajectory optimization for lunar soft landing with a Hamiltonian-based adaptive mesh [43] H. Marsh, M. Karpenko and Q. Gong, “A Pseudospectral Ap- refinement strategy,” Advances in Engineering Software Volume proach to High Index DAE Optimal Control Problems,” arXiv 100, October 2016, pp. 266–276. preprint 2018, arXiv:1811.12582. [61] Q. Gong, I. M. Ross and F. Fahroo, “A Chebyshev pseudospec- [44] F. Fahroo and I. M. Ross, “Costate estimation by a Legendre tral method for nonlinear constrained optimal control problems,” pseudospectral method,” Proceedings of the 1998 AIAA GNC Proceedings of the 48h IEEE Conference on Decision and Con- conference, Boston, MA, August 1998, Paper 98-4222. trol (CDC), Shanghai, 2009, pp. 5057–5062. 12

[62] E. Polak, Optimization: Algorithms and Consistent Approxima- A. Introduction to the Cotangent Barrier tions, Springer-Verlag, Heidelberg, 1997. Using the equations and the process described in Sec. II in apply- ing Pontraygin’s Principle to Problem (P ), it is straightforward to [63] Q. Gong, I. M. Ross, W. Kang and F. Fahroo, “On the Pseu- show that the totality of conditions results in the following nonlinear dospectral Covector Mapping Theorem for Nonlinear Optimal differential-algebraic equations: Control,” Proceedings of the 45th IEEE Conference on Decision and Control, San Diego, CA, 2006, pp. 2679–2686. x˙(t)= f(x(t), u(t)) λ (P ) λ˙ (t)= ∂xH(λ(t),x(t), u(t)) (41)  − [64] I. M. Ross, “An Optimal Control Theory for Nonlinear Optimiza-  0= ∂uH(λ(t),x(t), u(t)) tion,” J. Comp. and Appl. Math., 354 (2019) 39–51. Using the concept of the virtual control variable for both the state and [65] I. M. Ross, “An Optimal Control Theory for Accelerated Opti- the costate equations, the DIDO-form of (P λ) is given by, mization,” arXiv preprint (2019) arXiv:1902.09004. x˙(t)= v(t) [66] K. Yosida, Functional Analysis, Sixth Edition, Springer-Verlag, λ˙ (t)= ω(t) λ,D − New York, 1980. (P )  0= f(x(t), u(t)) v(t) (42)  −  0= ∂xH(λ(t),x(t), u(t)) ω(t) 0= ∂ H(λ(t),x(t), u(t)) − [67] E. J. McShane, “Partial Orderings and Moore-Smith Limits,” The u  American Mathematical Monthly, Vol. 59, No. 1, 1952, pp. 1–11.  where, ω is called the co-virtual control variable. DIDO is based on the principle that (42) is computationally easier than (41) to the ex- tent that all the differential components are concentrated in the exceed- Appendix A: DIDO’s Computational Theory ingly simple linear equations. To appreciate the simple sophistication of this principle, consider now an application of Pontryagin’s Principle A vexing question that has plagued computational methods for op- to Problem (P D); this generates, timal controls is the treatment of differential constraints. DIDO ad- dresses this question by introducing a virtual control variable v to rewrite the differential constraint as, x˙(t)= v(t) 0= f(x(t), u(t)) v(t) D,λ − (P )  λ˙ (t)= ∂ H(µ(t),x(t), u(t)) (43) dx  x := v (37a)  −λ(t)= µ(t) dτ  0= ∂uH(µ(t),x(t), u(t)) dΓ  f(x, u, Γ(τ; t0,tf )) = v (37b)  dτ    Comparing (43)to(42), it is clear that, where, Γ : R R is an invertible nonlinear transformation[9, 35, 54], (P D,λ) (P λ,D) (44) → 6≡ −1 t = Γ(τ; t0,tf ) τ =Γ (t; t0,tf ) (38) That is, (44) is a statement of the noncommutativity of the “λ” and “D” ⇔ operations. This noncommutativity can be written more elaborately as, and τ is transformed time over a fixed horizon [τ 0,τ f ]. Because the λ D (P ) D λ (P ) (45) virtual control variable affects only the dynamical constraints, the rest ◦ 6≡ ◦ of the problem formulation remains unchanged. Thus the age-old ques-

tion of the “best” way to discretize a nonlinear differential equation is Figure 11 encapsulates this notion in terms of an “insurmountable” relegated to answering the same question for the embarrassingly simple cotangent barrier. It depicts the fact that an application of Pontryagin’s linear equation given by (37a). D , P P D Remark 3 The transformation given by (37) is only internal to DIDO.

xxxxxxxxxxxxxxxxxxxxxxxxxxxx As far as the user is concerned a virtual control variable does not exist. xxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxx

cotangent xxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxx barrier xxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxx As simple as it is, a surprisingly significant amount of sophistication is xxxxxxxxxxxxxxxxxxxxxxxxxxxx necessary to implement (37a) efficiently. To explain the details of this D, P computational theory, we employ the special problem defined by,

tf Minimize J[x( ), u( )] := F (x(t), u(t)) dt (P ) · · t0 (39)  Z D  Subject to x˙ (t)= f(x(t), u(t)) P P D  where, t0,tf ,x(t0) and x(tf ) are all fixed at some appropriate values. Furthermore, because the time interval [tf t0] is fixed, there is no Figure 11. An illustration of the cotangent barrier indicating need to perform a “τ”-transformation to generate− the virtual control the impossibility of moving “up” (i.e. in λ) from (P D,λ) to variable. Consequently, using (37), we construct the DIDO-form of (P λ,D). (P ) as, Principle to the DIDO-form of the primal problem does not generate a Minimize J[x( ), u( ), v( )] := · t · · DIDO-form of the primal-dual problem. The noncommutativity of the f two operations seems to suggest that there was no value in using the D  F (x(t), u(t)) dt (P )  t0 (40) DIDO form of the primal problem. Despite this apparently discourag-  Z  Subject to x˙(t)= v(t) ing result, we now show that it is possible to tunnel through the barrier, 0= f(x(t), u(t)) v(t) albeit discretely (pun intended!).  −   13

B. Discrete Tunneling Through the Cotangent Barrier Definition 1 DIDO’s generalized equation is defined by (P λ,D,N ). Let XN , U N , and V N be (N + 1)-dimensional vectors that represent Similar to λ,D , Problem D may be discretized to generate the discretized state, control and virtual control variables respectively over (P ) (P ) following problem, an arbitrary grid πN : D,N N N N N Minimize J [X , U ,V ] X := (x0,x1,...,xN ) (46a) := qT F (XN , U N ) N  U := (u0, u1,...,uN ) (46b) (P D,N )  N (50)  X N N  Subject to A = C V := (v0, v1,...,vN ) (46c)  V N   N N N From linearity arguments, it follows that a broad class of generic dis-  f(X , U ) V = 0  − cretizations of x˙ = v may be written as,  where, q = (q0, q1,...,qN ) is an (N + 1) 1 vector of positive N × X N N N quadrature weights associated with the specifics of the discretization A AxX + AvV = C (47) N V N ≡ given by A (and inclusive of π ).   Remark 4 From (49) and (50), it follows that DIDO’s entire approach where, A := [Ax Av] comprises two (N + 1) (N + 1) discretiza- ∗ | N × to discretization is isolated in the matrix pair A, A . Part of DIDO’s tion matrices that depend on N, π , and a choice of the discretization computational efficiency is achieved by exploiting{ this} linear system. method. The choice of the discretization method also determines CN , N an (N + 1) 1 matrix. The dependence of Ax and Av on N and π Lemma 1 Let Q be the positive definite diagonal matrix defined by × is suppressed for notational convenience. Following the same rationale diag (q0, q1,...,qN ). Define, as in the production of (47), a generic discretization of λ˙ = ω may − † −1 T † −1 T be written as, Ax := Q Ax Q and Av := Q Av Q (51) N Then, under appropriate technical conditions at the boundary points, ∗ Λ ∗ N ∗ N ∗N N N+1 N N+1 A N AλΛ + AωΩ = C (48) there exist multipliers Ψ R and Ψ R for Prob- Ω ≡ A d   lem (P D,N ) such that its dual∈ feasibility conditions∈ can be written as, N N+1 N N+1 where Λ R and Ω R are defined analogous to (46), † N N ∗N ∗ ∈ ∈ A Ψ Ψ = C (52) A is a matrix (that may not necessarily be equal to A) that depends on v A − d N and πN , and C∗N is a matrix similar to CN . Collecting all relevant N N N † N ∂xH Ψd , X , U + AxΨA = 0 (53) equations, it follows that a generic discretization of (P λ,D) may be written as,  N N N ∂uH Ψd , X , U = 0 (54) XN   A = CN N N V N where ΨA and Ψd are given by,    N N −1 N N −1 N  ∗ Λ ∗N ΨA := Q ΨA Ψd := Q Ψd (55)  A N = C λ,D,N  Ω (P )    (49)  N N N N Proof. This lemma can be proved by an application of the multiplier  ∂λH(Λ , X , U ) V = 0  − theory. Details are omitted for brevity.  N N N N ∂xH(Λ , X , U ) Ω = 0  − It follows from Lemma 1 that if A† (and hence A) can be chosen  N N N  ∂uH(Λ , X , U ) = 0 such that its x- and v-components constitute A∗ of (49), then, it will   be possible to commute the noncommutative operations inherent in the where, H is reused as an overloaded operator to take in discretized vectors as inputs. cotangent barrier, except that this would have been achieved in the dis- DIDO is based on the ansatz that (49) is a well-structured, grid- cretized space. This idea is captured in Fig. 12 and the statement of its tunnel theorem agnostic root-finding problem that can be solved efficiently for any possibility is enunciated as the . given N and πN . The basis for this ansatz is as follows: N , D , D, N 1. The A-A∗ system is agnostic to the problem definition and data P P

xxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxx

functions; hence, it can be handled separately and “indepen- xxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxx

cotangent xxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxx cotangent

xxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxx dently” over the “space of all optimal control problems.” xxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxx

barrier xxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxx tunneling

xxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxx 2. The A-A∗ system is always a linear (affine) equation even if D, D, , N Problem (P ) contains inequalities. P P 3. The nonlinear elements of (49) are based on the Hamiltonian of Problem (P ). The Hamiltonian is constructed by a simple dot- product operation (see Sec. II). In its discretized (overloaded) D D, N form, H has the property that the Hamiltonian and its gradients P P at i are not connected to their values at (i + 1). This separable N programming[28] property of the Hamiltonian and its gradients N over π can be harnessed for computational efficiency. Figure 12. Discrete tunneling allows (P D,λ,N ) to move through the cotangent barrier to connect with (P λ,D,N ). 4. Solving the nonlinear elements of (49) involves a sequential lin- earization as an inner-loop. Augmenting the A-A∗ linear sys- tem to this inner loop solves the entire problem. The iterations ∗ for solving the augmented linear system can be tailored to the Theorem 1 (A Tunnel Theorem) Suppose A and A are discretiza- special structure of the sequentially linearized Hamiltonian sys- tion matrix pairs that satisfy the conditions, tem and the specific method of discretization captured in the N Ax = I (56a) A-A∗ system. − ∗ N Aλ = I (56b) λ,D,N − All these features of (P ) are exploited in the production of ∗ † DIDO’s algorithm as detailed in Appendix B. Aω = Av (56c) 14 then, the necessary conditions for Problem (P D,N ) are equivalent to Appendix B: DIDO’s Suite of Algorithms λ,D,N (P ) under the transformation, As noted in Sec. I, DIDO’s algorithm is actually a suite of algo- rithms. Several portions of this suite can be found in various publi- N N Ψd = Λ (57a) cations scattered across different journals and conference proceedings. N N In this section, we collect and categorize this algorithm-suite to explain ΨA = Ω (57b) how specific algorithmic options are triggered by the particulars of the user-defined problem and inputs to DIDO. Proof. The proof of this theorem follows from Lemma 1 and by D,N a substitution of (56) and (57) in the the definitions of (P ) and C. Introduction and Definitions (P λ,D,N ).  Each main algorithm in DIDO’s suite is true-fast, spectral and Hamiltonian. That is, each of DIDO’s main algorithm solves the Definition 2 (P D,N ) is called a DIDO-Hamiltonian programming DIDO-Hamiltonian programming problem (see Definition 2) using a problem if A and A∗ are chosen in accordance with Theorem 1. modified[18, 19] spectral algorithm[8, 56] that is true-fast. True-fast is different from computationally fast in the sense that the former must See [20] for a first-principles introduction to the notion of Hamiltonian be agnostic to the details of its computer implementations whereas the programming. A DIDO-Hamiltonian programming problem is simply latter can be can be achieved via a variety of simple and obvious ways. an adaptation of this terminology to Problem (P D,N ). For instance, any optimization software, including a mediocre one, can be made computationally fast by using one or more of the following:

Theorem 2 All Lagrange PS discretizations over any grid (including 1. A good/excellent “guess;” Gauss, Radau, and Lobatto) fail to satisfy the conditions of Theorem 1. 2. A high-performance computer; Proof. The proof of this theorem is fairly straightforward. It follows from the fact that all Lagrange PS discretizations[17, 18, 35] over any 3. A compiled or embedded code; and, grid[36, 55, 56] are based on a differentiation matrix, D. Hence all Lagrange PS discretizations of x˙ = v are given by, 4. Analytic gradient/Jacobian/Hessian information. A truly fast algorithm should be independent of these obvious and other DXN = V N V N DXN = 0 (58) ⇒ − run-time code improvements. For an algorithm to be true-fast, it must N satisfy the following properties (see Fig. 14): This implies Ax = D and Av = I which violates the conditions − of Theorem 1.  1. Converge from an arbitrary starting point;

Theorem 3 A collection of Birkhoff PS discretizations of optimal con- 2. Converge to an optimal/extremal solution;r trol problems satisfy the conditions of Theorem 1. 3. Take the fewest number of iterations towards a given tolerance Proof. The proof of this theorem follows from the fact that a Birkhoff criterion; and/or PS discretization[18, 19] may be written in a form that generates N 4. Use the fewest number of operations per iteration. Ax = I ; see [19] for further details. −  Theorems 2 and 3 imply that there are two broad categories of PS discretizations for optimal control problems, namely Lagrange and Birkhoff. Within these two main methods of discretizations, it is pos- sible to generate a very large number of variations based on the choice q arbitrary of basis functions and grid selections. A process to achieve at least starting eighteen variations of PS discretizations based on classical orthog- points convergence onal polynomials is shown in Fig. 13. Despite this apparently large (accumulation point)

Birkhoff J Gauss-Lobatto speed (a & b) a Legendre c Gauss-Radau o (PS) b Chebyshev Gauss Figure 14. Schematic for defining an algorithm and its funda- Lagrange i mental properties. Pseudospectral

Figure 13. Schematic for generating at least 18 variants of PS Thus, a true-fast algorithm can be easily made computationally fast, but discretizations; figure adapted from [19]. its implementation may not necessarily be computationally fast. Like- wise, a purportedly computationally fast algorithm may not necessarily be true-fast. DIDO achieves true-fast speed by calling several true- variety of choices, Theorems 2 and 3 reveal that there are no essential fast algorithms from its toolbox. The specifics of a given problem and differences between PS discretizations based on different grid points. user inputs initiate automatic triggers to generate a coordinated DIDO- All that matters for optimal control applications is whether they are iteration that defines the flow of the main algorithm. To describe this based on Lagrange or Birkhoff interpolants. Nonetheless, when ad- algorithm suite, we use with the following definitions from [20]: ditional technical factors are taken into account, such as the inherent structure of the resulting inner-product space[36], the choice of a grid Definition 3 (Inner Algorithm ) Let (P N ) denote any discretiza- can have a deleterious effect on convergence; see [35, 36] for details. tion of (P ). An inner algorithm Afor Problem (P ) is defined as finding NA N N N See also [9], Sec. 4.4.4. a sequence of vector pairs (X0 , U0 ), (X1 , U1 ),... by the iterative map, qThe basis functions need not be polynomials[19, 29, 57]. Nonpolynomial N N N N (Xi+1, Ui+1)= (Xi , Ui ), i N “designer” basis functions can also be generated via the nonlinear domain trans- A ∈ formation Γ used in (37b); see [9, 29, 35, 54] for details. rAn extremal solution is one that satisfies Pontryagin’s Principle. 15

Note that N is fixed in the definition of the inner algorithm. Definition 6 (DIDO-Hamiltonian Algorithm λ,D) A Hamiltonian algorithm λ,D is said to be DIDO-HamiltonianA if it generates an ad- Definition 4 (Convergence of an Inner Algorithm) An inner algo- ditional sequenceA of vector pairs (V N ΩN ), (V N , ΩN ),... for a fixed rithm is said to converge if 0 0 1 1 A N such that lim XN := XN , lim U N := U N N N N N n ∞ n ∞ lim Vn := V∞ , lim Ωn := Ω∞ n→∞ n→∞ n→∞ n→∞ N is an accumulation point that solves Problem (P ). is an accumulation point that solves Problem (P λ,D,N ). Definition 5 (Hamiltonian Algorithm λ ) Let (P λ,N ) denote any λ,D D,λ A Remark 6 Because (P ) (P ) (see (44)), we have discretization of (P λ). A (convergent) inner algorithm λ is said λ,D,N D,λ,N 6≡ A (P ) (P ) unless the discretization offers a cotangent to be Hamiltonian if it generates an additional sequence of vectors 6≡ λ,D,N D,λ,N N N tunnel; see Theorem 1. Even if (P ) (P ), it follows from Λ0 , Λ1 ,... for a fixed N such that ≡ Lemma 2 that δλ,D,N = δD,λ,N , where δλ,D,N and δD,λ,N denote the N N 6 λ,D,N D,λ,N lim Λn := Λ∞ constraint tolerances on (P ) and (P ) respectively. The n→∞ DIDO-Hamiltonian algorithm fixes this discrepancy. λ,N is an accumulation point that solves . N (P ) A solution to Problem (P ) does not imply a solution to Problem (P ). N Remark 5 Note that the Hamiltonian algorithm is focused on solving In fact, an optimal solution to Problem (P ) may not even be a fea- (P λ,N ) not (P N,λ). In general, (P λ,N ) = (P N,λ) for finite N even sible solution to Problem (P ). This point is often lost when nonlinear if they are equal in the limit as N . 6 programming solvers are patched to solve discretized optimal control →∞ problems. To clarify this point, we use the following definitions from Lemma 2 Suppose (P λ,N ) = (P N,λ) for finite N. If (P N,λ) is [20]: solved to a practical tolerance of δN,λ > 0, then, the resulting solution Definition 7 (Convergence of a Discretization) Let (XN , U N ) be a solves (P λ,N ) to a tolerance of δλ,N > 0 such that ∞ ∞ solution to Problem (P N ). A discretization is said to converge if λ,N N,λ δ > δ (59) N ∞ N ∞ lim X∞ := X∞ , lim U∞ := U∞ N→∞ N→∞ Proof. This lemma is proved in [20] for a shooting method and Euler discretization. Its extension to the general case as given by (59) is an accumulation point that solves Problem (P ). follows from the assumption that (P λ,N )=(P N,λ) for finite N.  Remark 7 In Definition 7 and subsequent ones to follow, we have From (59) it follows that δλ,N = δN,λ. A Hamiltonian algorithm fixes taken some mathematical liberties with respect to the precise notion of this discrepancy. 6 a metric for convergence. It is straightforward to add such precision; As explained in [20], Problem (P N ) is better categorized as a however, it comes at a significant cost of new mathematical machinery. Hamiltonian programming problem rather than as a nonlinear program- Consequently, we choose not to include such additional mathematical ming problem. This is, in part, because a Hamiltonian is not a La- nomenclature in order to support the accessibility of the proposed con- grangian and Problem (P N ) contains “hidden” information that is not cepts to a broader audience. N,λ λ,N accessible via (P ). A Hamiltonian algorithm aims to solve (P ) Definition 8 (Dynamic Optimization Algorithm ) Let be an al- N,λ B B while a nonlinear programming algorithm attempts to solve (P ). gorithm that generates a sequence of integer pairs (N0,m0), Even when these two problems are theoretically equivalent to one an- (N1,m1) such that the sequence other, a nonlinear programming algorithm does not solve (P λ,N ) to the · · · N,λ N0 N0 N1 N1 same accuracy as (P ) as shown in Lemma 2; hence, a Hamiltonian (Xm0 , Um0 ), (Xm1 , Um1 ),... algorithm is needed to replace or augment generic nonlinear program- ∞ ∞ converges to an accumulation point (X∞ , U∞ ) given by Definition 7. ming algorithms. These ideas are depicted in Fig. 15. Specific details Then, is called a dynamic optimization algorithm for solving Prob- lem (PB). From Definitions 7 and 8, it follows that a generic dynamic optimiza- nonlinear tion algorithm involves the production of a double infinite sequence of programming vectors[20]. From Definition 7 it follows that the production of this problems double infinite sequence may be greatly facilitated by the use of an in- ner algorithm; see Definition 3. From Definition 8, it follows that if an inner algorithm is used to construct , then it need not even be exe- Hamiltonian convex cuted to completion! This idea is part ofB DIDO’s process for generating optimization programming a true-fast optimization algorithm. problems Although the idea of not taking the inner algorithm to completion was motivated by Definition 8, the definition itself is driven by the theory of consistent approximations[8, 30, 38, 62, 63]. In fact, mk < for k < is actually a theoretical requirement for consistency. Consistency∞ requires∞ that δλ,N > ǫN > 0 (see Lemma 2) where ǫN must be sufficiently large for finite N and must have the property ǫN 0 as N . The rate at which ǫN goes to zero cannot be faster→ Figure 15. A depiction of the differences between nonlinear than the→ order ∞ of convergence; for example, ǫN 0 cannot be faster and Hamiltonian programming. than linear for Euler approximations. For PS approximation→ s, we may choose ǫN 0 as N at the spectral or near-exponential rates. → →∞ in augmenting a nonlinear programming algorithm with a Hamiltonian Definition 9 (Hamiltonian Dynamic Optimization Algorithm λ) component may be found in [58, 59, 60]. Alternative implementations A dynamic optimization algorithm λ is said to be HamiltonianB if it that achieve similar effects are discussed in [42, 55, 56, 61]. Note B also that in Hamiltonian programming, the separable-programming- generates an additional sequence of vectors D,N type property inherent in (P ) is naturally exploited. In the multi- N0 N1 Λm , Λm ,... variable case, this property lends itself to casting the discretized prob- 0 1 ∞ ∞ ∞ lem in terms of a compact matrix-vector programming problem rather such that they converge to an accumulation point (X∞ , U∞ , Λ∞) that than a generic a nonlinear programming problem[19]. solves (P λ). 16

Definition 10 (DIDO-Hamiltonian Dynamic Optimization Algorithm) functions provides sufficient conditions for guidability. The specifics of A Hamiltonian dynamic optimization algorithm λ,D is said to be a Lyapunov-stable algorithm rely on the local convexity of the contin- DIDO-Hamiltonian if it generates an additionalB sequence of vector uous search trajectory. Because convexity cannot be guaranteed along pairs the search path (see Fig. 16), a suite of algorithms is needed to sup- N0 N0 N0 N1 (Vm0 , Ωm0 ), (Vm0 , Ωm1 ),... port different but simple objectives. DIDO is based on the idea that ∞ ∞ ∞ ∞ ∞ the three components of stability, acceleration and accuracy achieve that converges to an accumulation point (X∞ , U∞ ,V∞ , Λ∞, Ω∞) λ,D the requirements of a true-fast algorithm. DIDO’s main algorithm that solves (P ). ties these three components together by monitoring their progress and triggering automatic switching between components based on various It is clear that Definition 10 relies on Definitions 2–9. From these criteria as described in the subsections that follow. definitions it follows that DIDO-Hamiltonian dynamic optimization al- gorithm generates a double-infinite sequence of tuples, E. Details of the Three Components of DIDO N (X, U, V, Λ, Ω)N0 , (X, U, V, Λ, Ω)N1 ,..., (X, U, V, Λ, Ω) f , Each of the three components of DIDO contain several subcomponents. m0 m1 mf Many of these subcomponents are described in detail across several ∞ ..., (X, U, V, Λ, Ω)∞ (60) published articles. Consequently, we lean heavily on these articles in describing the details of DIDO’s algorithms. whose limit point purportedly converges to a solution of P λ,D. 1. The Stabilization Component D. Overview of The Three Major Algorithmic Components of DIDO There are three main elements to the stabilization component of DIDO: The production of each element of the sequence denoted in (60) is a) The Guess-Free Algorithm; determined by suite of algorithms that call upon different inner-loop DIDO-Hamiltonian algorithms (see Definition 6). The specific calls b) Backtracking and Restart; and on the inner loop are based upon the performance of the outer loop (see Definition 10). DIDO uses different inner loops to support its c) Moore-Smith Sequencing. three performance goals of robustness, speed and accuracy as stated in Secs. I and III. Consequently, the three major algorithmic components The details of these three elements are as follows: of DIDO’s suite of algorithms comprise (see Fig. 16): a) The Guess-Free Algorithm 1. A stabilization component: The task of this component of the N algorithm is to drive an “arbitrary” tuple (X, U, V, Λ, Ω)m to an In its nominal mode, DIDO is guess-free; i.e., a user does not provide “acceptable” starting point for the acceleration component. any guess. Thus, one of the elements of the stabilization component is to generate the first point of the sequence denoted in (60). Because the 2. An acceleration component: The task of this suite of algorithms guess-free element itself requires a starting point, this phase of the al- is to rapidly guide the sequence of (60) to a capture zone of the gorithm runs through a sequence of prioritized starting points and each accurate component. starting point is used as an input for an elastic programming algorithm described in [7] and [8]. The second priority point is executed only if 3. An accurate component: The task of this component is to gen- the first priority point terminates to an infeasible “solution.” If all pri- erate a solution that satisfies the precision requested by the user. ority points terminate to infeasibility, then DIDO will execute a final attempt at feasibility by minimizing a weighted sum of infeasibilities. If this phase is not successful, then DIDO will present this solution to the user but tag it as infeasible. If, at any point, a feasible solution is found, the guess-free element of the algorithm terminates with a suc- N0 cess flag. This point is denoted as (X, U, V, Λ, Ω)m , the first point of stabilization 0 the DIDO sequence. This is the Superior To IntuitionTM concept first component announced in 2008; see [7] and [8]. accuracy component b) Backtracking and Restarts The second element of the stabilization component is triggered if the search path monitor algorithm detects instability (i.e., divergence) in the accelera- acceleration tion component. The task of this element of the stabilization compo- s component nent is to use a “backtracked” point to generate an alternative starting point for the acceleration component by reusing some of the elements of the guess-free algorithm. The production of this alternative point Figure 16. A schematic for the concept of the three main algo- is performed through a priority suite of subalgorithms that comprise rithmic components of DIDO. Same as Fig. 2, repeated here restarts and elastic programming[7, 8]. for ready reference. c) Moore-Smith Sequencing The three-component concept is based on the overarching theory intro- The third element of the stabilization component is triggered if the duced in [64]. According to this theory, an algorithm may be viewed monitor algorithm detects rapid destabilization in the acceleration com- as a discretization of a controlled dynamical system where the control ponent that cannot be handled by backtracking alone. In this situation, variable is a search vector or its rate of change[65]. In the case of static a backtracked point is used as a starting point to generate sequential optimization, the latter case generates accelerated optimization meth- perturbations in Sobolev space[35, 30] by generating continuous-time ods that include Polyak’s heavy ball method, Nesterov’s accelerated candidate solutions via interpolation. The perturbed discretized vec- gradient method and the conjugate gradient algorithm[65]. In apply- tor is used as starting point to generate a new sequence. If this new ing this theory for dynamic optimization, the DIDO-sequence given by sequence continues to exhibit instability, the entire process is repeated (60) may be viewed as a discretization of a continuous primal-dual dy- namical system. The convergence theory proposed in [64, 65] rests on sBecause algorithmic backtracking in dynamic optimization can be compu- the guidability (i.e., a weaker form of controllability) of this primal- tationally expensive, DIDO saves a small number of prior iterates that can be dual system. New minimum principles[64, 65] based on Lyapunov instantaneously recalled for rapid restarts. 17 until stability is achieved or the number of Sobolev perturbations ex- 3. The Accuracy Component ceeds a predetermined ceiling. In the latter case, DIDO returns the last iterate as an infeasible solution. The perturbations are also stopped Technically, this component of DIDO is the simplest; however, its com- if the projected variations exceed the bounds specified in the search puter implementation requires a significant amount of skill in practical space; see (5) in Sec. II. Because this sequence generation does not and algorithmic matters. This is because the accuracy component of fit within the standard Arzel`a-Ascoli theorem, we denote it as Moore- DIDO’s algorithmic suite is handed a near-final solution; hence, its Smith[66, 67]. “only” technical task is to deliver the final solution. This point sug- gests that an implementation of the accuracy component can be a sim- ple quasi-Newton algorithm (or its accelerated version as presented λ,D,Nk in [65]) for solving (P ) for the last few values of Nk. While 2. The Acceleration Component the preceding arguments are technically correct, the main implemen- tation challenge here is in ensuring that the momentum gained from the acceleration component is not lost in achieving the targeted accu- The nominal phase of the acceleration component is to rapidly generate racy requested by the user. In achieving its stated goals, the iterations the DIDO sequence (Cf. (60)) subsequent to the first point handed to of this algorithm are largely focused on generating the targeted accu- it by the stabilization component. DIDO’s Hamiltonian algorithm (see racy of satisfying the Pontryagin necessary conditions. As noted ear- Definition 6) produces this sequence. This algorithm uses Lemma 2 lier in Sec. III, this component of DIDO tightens the tolerances on the and a tentative value of λ,D,Nk for each ; see Re- δ Nk,k = 1, 2,... Hamiltonian minimization condition that may not have been met at the mark 6. The values of λ,D,Nk for are revised based on the δ k > 2 termination of the accelerated algorithm. This procedure is done se- solution generated by the previous sequence and the predicted rate of quentially to maintain dual feasibility that is consistent with the value convergence. The predicted rate of convergence is given by formulas of N = N . The mathematical details of this process are described in derived in [8, 30]. Although the equations derived in [8, 30] are for f [35, 55, 56, 61]. Lagrange PS methods, it is not too difficult to show that they are valid for Birkhoff PS methods as well because of the equivalence conditions derived in [18]. F. An Overview of the Nominal, True-Fast, Spectral Algorithmic Flow The theory for the fast spectral algorithm presented in [8] is founded on an applications of the Arzel`a-Ascoli theorem[66] to PS Because the nominal mode of DIDO is guess-free; i.e., a user does discretization[30]. Because the guarantees provided by Arzel`a-Ascoli not provide a guess, the first element of the stabilization component is invoked to generate the first element of the DIDO sequence given theorem are only for a subsequence, an implementation of this pow- N0 erful result is not straightforward[35, 30]. Hence, we augment this by (X, U, V, Λ, Ω)m0 ; see (60). The acceleration component of DIDO result using the theory of optimization presented in [64]. According rapidly iterates this point to completion or near-completion denoted by Na . If this point satisfies all the termination criterion, to this theory, the DIDO sequence denoted in (60) may be viewed as (X, U, V, Λ, Ω)ma Na a discretization of a controllable continuous “search trajectory;” see the main algorithm terminates; otherwise, (X, U, V, Λ, Ω)ma is handed Fig. 16. The dynamics of this search trajectory is given by a primal- Nf off to the accurate component which iterates it to (X, U, V, Λ, Ω)mf . dual controllable system that can be guided to a candidate optimal solu- N Because (V, Ω) f are internal variables, DIDO discards these virtual tion. Minimum principles based on Lyapunov functions provides suffi- mf primal and dual variables. The remainder of the variables that support cient conditions for a successful guidance algorithm[64, 65]. The Lya- a solution to λ,Nf are passed to the user through the use of primal punov function forms a merit function; and, under appropriate techni- P and dual structures described in Sec. II. For λ,Nf these variables are cal conditions[64, 65], the search trajectory can be designed to drive P passed according to, an arbitrary point to a stable point. The stable point is defined in terms of the necessary conditions for optimality. For optimal control, N primal.states = X f (61a) these necessary conditions are given by Pontryagin’s Principle[9]. A first-principles’ application of these ideas to analyze a direct shooting primal.controls = U Nf (61b) method method is described in [20]. dual.dynamics = ΛNf (61c) A guarantee of convergence is achieved in terms of local convexity of a Riemannian metric space that can be defined using a regularized It is clear from the entirety of the preceding discussions that most Hessian;t see [65] for theoretical details. Because any point in an iter- of DIDO’s algorithms are completely agnostic to the specifics of a ation can get trapped in a nonconvex region, subalgorithms are needed PS discretization. Consequently, they can be adapted to almost any to detect and escape the trap regions. Detection of trap points can be method of discretization! made by monitoring the decrease in the Lyapunov function. Escape from suspected trap points is made by a call to the stabilization com- ponent of DIDO described previously. An acceleration of the DIDO sequence, denoted by (60), is achieved through a combination of PS discretizations[5, 35, 18, 19, 56, 61]; see also Fig. 13. As shown by Theorem 3, the family of Birkhoff PS discretizations offers a special capability of tunneling through the cotangent barrier. Consequently, an accelerated spectral sequencing is achieved by coordinating the progress of the outer loop with the inner- loop sequence as stipulated in Definition 6. Perhaps the most important aspect of a Birkhoff PS discretization is that it offers a “flat” condition number of the resulting linear system[18, 17]; i.e., an (1) growth with respect to N. In contrast, the condition number ofO a Lagrange PS discretization, be it Gauss, Gauss-Radau or Gauss-Lobatto, grows as (N 2) no matter the choice of the orthogonal polynomial[18, 19]. BecauseO condition number affects both accuracy and computational speed, its combination with its cotangent tunneling capability makes a Birkhoff PS discretization a doubly potent method to generate accel- erated algorithms.

tBecause the “exact” Hessian may not be positive definite, a “correction” can better ensure progress towards the Lyapunov-stable point[64].