Rochester Institute of Technology RIT Scholar Works Theses 11-1-2006 Streamlining of the state-dependent Riccati equation controller algorithm for an embedded implementation Sergey Katsev Follow this and additional works at: https://scholarworks.rit.edu/theses Recommended Citation Katsev, Sergey, "Streamlining of the state-dependent Riccati equation controller algorithm for an embedded implementation" (2006). Thesis. Rochester Institute of Technology. Accessed from This Thesis is brought to you for free and open access by RIT Scholar Works. It has been accepted for inclusion in Theses by an authorized administrator of RIT Scholar Works. For more information, please contact [email protected]. Streamlining of the State-Dependent Riccati Equation Controller Algorithm for an Embedded Implementation by Sergey Katsev A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Science in Computer Engineering Supervised by Associate Professor of Computer Engineering Dr. Juan Cockburn Department of Computer Engineering Kate Gleason College of Engineering Rochester Institute of Technology Rochester, New York November 2006 Approved By: Dr. Juan Cockburn Associate Professor of Computer Engineering Primary Adviser Dr. Marcin Lukowiak Assistant Professor of Computer Engineering Dr. Roy S. Czernikowski Professor of Computer Engineering Thesis Release Permission Form Rochester Institute of Technology Kate Gleason College of Engineering Title: Streamlining of State-Dependent Riccati Equation Controllers for an Embedded Implementation I, Sergey Katsev, hereby grant permission to the Wallace Memorial Library reproduce my thesis in whole or part. Sergey Katsev Date Dedication To my grandparents and my mom, who gave up what they had in the Ukraine so that I could make a life for myself in the United States. iii Acknowledgments First, I would like to thank my advisor, Dr. Juan C. Cockburn for being patient and men- toring me through this work. Also I would like to thank my committee, Dr. Roy S. Cz- ernikowski and Dr. Marcin Lukowiak. Many thanks go to the other people and organiza- tions that helped me with technical aspects, in particular David Bishop and other members of the IEEE VHDL-200X Fast Track group and the authors of the LAPACK and SLICOT libraries. I would also like to thank Eric Nelson and John Schnyder for general advice and proofreading. Finally, I would like to thank Dr. Andreas Savakis for giving me the final push I needed. iv Abstract In many practical control problems the dynamics of the plant to be controlled are nonlin- ear. However, in most cases the controller design is based on a linear approximation of the dynamics. One of the reasons for this is that, in general, nonlinear control design methods are difficult to apply to practical problems. The State Dependent Riccati Equation (SDRE) control approach is a relatively new practical approach to nonlinear control that has the sim- plicity of the classical Linear Quadratic control method. This approach has been recently applied to control experimental autonomous air vehicles with relative success. To make the SDRE approach practical in applications where the computational resources are lim- ited and where the dynamic models are more complex it would be necessary to re-examine and streamline this control algorithm. The main objective of this work is to identify im- provements that can be made to the implementation of the SDRE algorithm to improve its performance. This is accomplished by analyzing the structure of the algorithm and the underlying functions used to implement it. At the core of the SDRE algorithm is the solution, in real time, of an Algebraic Riccati Equation. The impact of the selection of a suitable algorithm to solve the Riccati Equation is analyzed. Three different algorithms were studied. Experimental results indicate that the Kleinman algorithm performs better than two other algorithms based on Newton’s method. This work also demonstrates that appropriately setting a maximum number of iterations for the Kleinman approach can improve the overall system performance without degrading accuracy significantly. Finally, a software implementation of the SDRE algorithm was developed and bench- marked to study the potential performance improvements of a hardware implementation. The test plant was an inverted pendulum simulation based on experimental hardware. Bot- tlenecks in the software implementation were identified and a possible hardware design to remove one such bottleneck was developed. v Contents Dedication ...................................... iii Acknowledgments ................................. iv Abstract ....................................... v Glossary ....................................... xi 1 Introduction ................................... 1 1.1 Thesis outline . 5 2 Supporting Work ................................ 7 2.1 Background . 7 2.2 SDRE methodology . 8 2.2.1 Linear Quadratic Regulator [1] . 9 2.2.2 SDRE controller design . 9 2.2.3 SDC parametrization . 12 2.2.4 Selection of Q and R matrices . 13 2.3 The Continuous-Time Algebraic Riccati Equation . 14 2.4 Riccati Equation solution algorithms . 16 2.4.1 Newton algorithm . 17 2.4.2 Newton’s method with Exact Line Search [7] . 18 2.4.3 Kleinman iteration [25, 26] . 19 2.4.4 Schur Decomposition . 20 2.5 Previous SDRE implementations . 21 3 Iterative Method Comparison ......................... 24 3.1 Accuracy analysis . 25 3.2 Speed analysis . 27 3.3 Kleinman maximum iterations . 35 vi 3.3.1 Effect on accuracy . 35 3.3.2 Effect on performance . 37 4 Application: Inverted Pendulum ........................ 42 4.1 Inverted Pendulum SDRE derivation . 42 4.2 Matlab simulation . 46 4.3 Selection of the Q and R matrices . 49 5 SDRE Software Implementation ........................ 56 5.1 Software algorithm design . 56 6 Hardware Co-design .............................. 62 6.1 Design considerations . 64 6.2 dhseqr Hardware components . 67 6.2.1 Validating the design . 72 6.3 Matrix multiplication . 74 6.3.1 Design . 74 6.3.2 Functional results . 75 6.3.3 Synthesis results . 80 6.3.4 Timing results . 81 7 Conclusions ................................... 83 7.1 Future work . 84 A Controllability, stabilizability, observability, detectability [20] ........ 90 A.1 Controllability and stabilizability . 90 A.2 Observability and detectability . 91 A.3 Example . 92 B Software Documentation ............................ 94 B.1 Using runSDRE . 94 B.2 Software functions . 96 B.2.1 I/O functions . 96 B.2.2 Matrix functions . 98 B.2.3 Matrix algebra functions . 99 B.2.4 Debugging functions . 100 B.2.5 Library functions . 101 vii List of Figures 2.1 SDRE Algorithm Diagram . 12 2.2 Pseudo-Code for Newton Algorithm [7] . 17 2.3 Pseudo-Code for Exact Line Search Algorithm . 18 2.4 Pseudo-Code for Kleinman Algorithm . 19 2.5 Pseudo-code for Schur Algorithm [27] . 23 3.1 Iterative method accuracy . 26 3.2 Matlab code for one Kleinman Algorithm iteration . 28 3.3 Matlab code for one Newton Algorithm iteration . 28 3.4 Average iterations to converge to a solution . 30 3.5 Total Newton and Kleinman iterations . 31 3.6 Kleinman and Newton Speed Results with Trend Line . 34 3.7 Effect of Kleinman iterations on number of total algorithm invocations . 36 3.8 Effect of Kleinman iterations on result accuracy . 37 3.9 Kleinman Max Iterations vs. Total Time . 39 3.10 Sample Application Maximum Kleinman Iteration Complexity . 41 4.1 Pendulum Parameters Block Diagram . 43 4.2 Quanser Consulting Inc. Linear Inverted Pendulum [33] . 44 4.3 Inverted Pendulum diagram, including sensor and actuator locations . 45 4.4 Inverted Pendulum Simulink Model . 46 4.5 Inverted Pendulum Models sub-block . 47 4.6 Simulation: Pendulum Angle . 49 4.7 Simulation: Cart Position . 50 4.8 Simulation: Pendulum Control . 51 4.9 Simulation: Pendulum angle at steady state with pulse disturbance . 52 4.10 Simulation: Cart position at steady state with pulse disturbance . 53 4.11 Simulation: Pendulum control action at steady state with pulse disturbance . 54 4.12 Q & R matrix analysis: Angle under-penalization . 54 4.13 Q & R matrix analysis: Angle over-penalization – Pendulum Angle . 55 viii 4.14 Q & R matrix analysis: Angle over-penalization – Cart Position . 55 5.1 High Level Software Data Flow . 57 5.2 Kleinman Algorithm Call Graph . 58 5.3 Schur Algorithm Call Graph . 59 5.4 High Level Software Call Graph . 60 6.1 System Block Diagram . 63 6.2 O(N 2) vs. O(N 3) complexity for low N . 64 6.3 Kleinman algorithm slow functions . 65 6.4 Systolic Array Structure for QR decomposition[16] . 68 6.5 Matlab code for QR iteration . 69 6.6 Hardware Structure for Eigenvalue Module[24] . 71 6.7 Testbench Simulation for QR Iteration Design . 73 6.8 Systolic Array Structure for Matrix Multiplication[24] . 75 6.9 Multiplication Array Testbench Functional Simulation . 76 6.10 Inverted Pendulum Maximum Value . 78 6.11 Software Matrix Multiplication vs. Systolic Array Complexity . 82 B.1 Sample inverted pendulum states input file . 95 B.2 Sample state matrices input file . 96 B.3 Example execution of runSDRE program . 97 ix List of Tables 2.1 CARE solution algorithms summary . 21 2.2 Execution Time Comparisons for Schur and Kleinman Methods . 22 2.3 Previous Work Summary . 23 3.1 Effect of Step Size (t) on Exact Line Search Convergence . 27 3.2 Iterative Method Accuracy Results . 29 3.3 Iterative Method Timing Results (seconds) . 32 3.4 Kleinman iterations timing test . 38 3.5 Maximum Iterations Computation using Equation 3.2 . 40 4.1 Quanser Inverted Pendulum parameters . 46 6.1 Function Execution Time . 66 6.2 Inverted Pendulum Maximum Value . 77 6.3 Multiplication Unit Fraction Precision Results . 80 6.4 Matrix Multiplication Synthesis Results . 81 x Glossary A adaptive control A control technique which involves modifying the control law used to compensate for the fact that the system parameters may be varying or uncertain [17]. affine This term refers to a function with a constant slope. A function is affine if it is equal to a constant plus a linear function of variables [31].