Czech Technical University in Prague Faculty of Electrical Engineering Department of Control Engineering Bachelor’s thesis Numerical optimization of industrial processes Lenka Caletková Supervisor: Ing. Jan Šulc Study Programme: Cybernetics and Robotics Field of Study: Systems and control Date: May 2013 iv Acknowledgements I would like to show my greatest appreciation to my supervisor Ing. Jan Šulc, who was always willing to help me with this project - even in his extra time - during the last two semesters. Without his encouragement and guidance this project would not have materialized. Many thanks belong to all my friends, who supported me in my tough times. Last but not least, I want to thank my supportive sister Jana and my great parents, who enabled me to study at the CTU, always stood by my side and gave me all their love. Abstract This thesis handles the conception of numerical optimizations and presents Python libraries, which are able to solve optimization problems. It compares their usability, stability and computing possibilities. All these aspects were tested on two specific industrial processes - control of the tunnel ventilation and heating systems control in buildings. A functional code in Python programming language for performance optimization of tunnel ventilation represents the outcome of this thesis. Abstrakt Tato bakalářská práce se věnuje pojmu numerické optimalizace a prezentuje knihovny pro- gramovacího jazyka Python, které je možné použít pro řešení optimalizačních úloh. Zároveň porovnává jejich použitelnost, stabilitu a výpočetní schopnosti. Všechny tyto aspekty byly testovány na dvou konkrétních průmyslových procesech - řízení ventilace v tunelech a řízení tepelných systémů v budovách. Výstup této práce představuje funkční kód v jazyce Python, který má za úkol optimalizovat výkon při ventilaci tunelu. vi Contents 1 Introduction1 1.1 Outline of the thesis................................2 2 Numerical optimization3 2.1 Sub-fields of numerical optimization.......................3 2.1.1 Convex optimization............................3 2.1.1.1 Least-squares problems.....................4 2.1.1.2 Linear programming.......................4 2.1.1.3 Semidefinite programming....................4 2.1.1.4 Quadratic programming.....................4 2.1.2 Other optimization problems.......................5 2.1.2.1 Non-linear programming.....................5 2.2 Practical examples of using optimization.....................5 2.2.1 Embedded optimization..........................5 2.2.2 Portfolio optimization...........................5 2.2.3 Data fitting.................................5 2.2.4 Model predictive control (MPC).....................6 3 Python libraries7 3.1 Licenses.......................................7 3.1.1 GPL.....................................7 3.1.2 BSD, CPL and MIT licenses........................8 3.2 Python libraries for optimization problems....................9 3.2.1 SciPy and its package Optimize......................9 3.2.2 CVXOPT.................................. 11 3.2.2.1 Examples of using CVXOPT.................. 12 3.2.3 CVXPY................................... 12 3.2.4 APMonitor................................. 13 3.2.4.1 Examples of using APMonitor................. 14 3.2.5 OpenOpt.................................. 15 4 Testing of code for special industrial processes 16 4.1 Industrial processes................................. 16 4.1.1 Tunnel ventilation control......................... 16 4.1.2 Control of building heating system.................... 18 ix CONTENTS x 4.2 Comparison..................................... 19 4.2.1 Ventilation control of the tunnel Blanka................. 19 4.2.1.1 MATLAB and fmincon..................... 20 4.2.1.2 SciPy............................... 20 4.2.1.3 APMonitor............................ 21 4.2.1.4 OpenOpt............................. 21 4.2.2 Heating system control of CTU’s building................ 22 4.2.2.1 MATLAB and quadprog..................... 22 4.2.2.2 CVXOPT............................. 22 4.2.2.3 OpenOpt............................. 22 5 Conclusion 24 A Implementation 28 A.1 Quadratic programming.............................. 29 A.1.1 CVXOPT.................................. 29 A.1.2 OpenOpt.................................. 30 A.2 Non-linear programming (NLP).......................... 31 A.2.1 OpenOpt.................................. 33 A.3 SciPy........................................ 34 A.4 APMonitor..................................... 36 Chapter 1 Introduction In the last century, the term “mathematical optimization” was brought to light thanks to works of great mathematicians like Fermat, Lagrange, Newton or Gauss. They defined the important basics, which created a path for the nowadays known definition of numerical optimization. This field had been developed during decades, but the ’boom’ of this concept has took place in the last 20 years, when the computer-time came. It was almost impossible to solve even simple optimization problems by ourselves, but with computers, there was a chance to improve and use this strong concept. It is spreading out through many fields - like economics, chemistry, energetics and many others. Numerical optimization is in these days an important part of control systems, because we can achieve many savings through this ‘tool’. At our faculty we are using optimization for ventilation control of complex road tunnels, chemical processes, or heating of buildings [24, 10]. To show the potential of this method, let us show you an example. Our control department has been using optimization, accurately Model predictive control, for heating system of our faculty building and the savings has been between 15 and 30 percent of energy in comparison to the previous heating control system [10]. The aim of this thesis is to find and test possible Python libraries, that can be utilized for numerical optimization of some processes. The numerical computing environment MAT- LAB is usually used for this purpose. It is very popular in academic domain and verified in practise and can be described as a stable and robust software, but it is really expensive in commercial sphere. Therefore small companies can not afford this software, so Python and some of its libraries offer a great chance to create software tools also for such clients. This thesis should find Python libraries, which could be a serious competition for MATLAB. 1 CHAPTER 1. INTRODUCTION 2 1.1 Outline of the thesis • Chapter 1- Introduction presents the subject and aims of this thesis. • Chapter 2- Numerical Optimization explains this concept, divides it into the sub-fields and shows us some practical examples. • Chapter 3- Python libraries lists all the used software and describes their possi- bilities to solve different types of optimization. This part includes few sample codes, which demonstrate the differences among the libraries. • Chapter 4- Testing of code for special industrial processes describes the im- plementation of this thesis. That means comparison of used software for two concrete industrial processes - Heating control of the CTU1 building in Prague and the Venti- lation control of the tunnel Blanka. • Chapter 5- Conclusion describes the advantages and disadvantages of tried software, chooses the best possible Python library for tunnel ventilation control and presents possible future improvements. 1The abbreviation ‘CTU’ presents the Czech Technical University in Prague. Chapter 2 Numerical optimization Before starting writing a code, we should first understand, what numerical or mathematical optimization means. The word ‘optimal’ is often used, but not always correctly. To describe something as optimal, we first have to define the conditions, for which we can consider this solution is truly optimal. Mathematical optimization or just optimization has two parts - objective function (or cost function) and constraints, which define the set, in which we are looking for the optimal solution. In general first we have to define an optimization problem and then we seek the best solver for this problem. Let us define the mathematical optimization problem. For optimization variable x = (x1; :::; xn) minimize f0(x) (2.1) subject to fi(x) ≤ bi; i = 1; :::; m: n n The f0 : R −! R is the objective function and fi : R −! R are constraint functions and the constants bi represent the bounds of the constraints [5, pp. 1]. Our demanding optimal solution x∗ has to fulfil for any real z, which satisfies the con- ∗ ∗ straints f0(z) ≥ f0(x ). This vector x is global extreme and “the smallest objective value among all vectors that satisfy the constraints”[5]. 2.1 Sub-fields of numerical optimization Although the optimization can be divided into many sub-fields, we will have a closer look only at few of them - especially on that sub-fields, that will used for our industrial processes. 2.1.1 Convex optimization This sub-field studies the cases, when the objective function and the constraint set are convex (concave) for minimization (maximization). And this case guarantees, that we are able to find global minimum (maximum) of this function. Therefore a lot of mathematicians and engineers try to describe an optimization problem as a convex or concave, because it ensures that they will find the sought optimum. Mr. Boyd and Mr. Vandenberghe wrote an important script [5] about this part of numerical optimization. 3 CHAPTER 2. NUMERICAL OPTIMIZATION 4 2.1.1.1 Least-squares problems This sub-field is one of the simplest convex optimization problems, because it does not contain any constraints and