Visualizing Optimization Trajectories
Total Page:16
File Type:pdf, Size:1020Kb
Visualization Research Center University of Stuttgart Allmandring 19 D–70569 Stuttgart Masterarbeit Visualizing Optimization Trajectories David Hägele Course of Study: Informatik Examiner: Prof. Dr. Daniel Weiskopf Supervisor: Dr. Antoine Lhuillier, M.Sc. Moataz Abdelaal, M.Sc. Rafael Garcia Commenced: May 27, 2019 Completed: November 27, 2019 Abstract Nonlinear constraint optimization has many applications in technical, scientific as well as economic fields. Understanding solver behavior can help to improve solvers, choose appropriate hyperparameters, and formulate better performing nonlinear programs. This thesis proposes a visual analytics tool for analyzing constraint optimization problems. The optimization process is depicted by a set of two-dimensional trajectories, representing the trace of intermediate solutions during the optimization process. This allows us to obtain an overview of the evolution of the optimization process. To support detailed analysis, sup- plemental views are added to show the constraints violations and areas of feasible solution. Furthermore, different interaction techniques are implemented to facilitate the exploration process. To showcase the usefulness of the approach, findings from an exemplary analysis based on optimization logs of robot motion planning are presented. Kurzfassung Nichtlineare Optimierung unter Nebenbedingungen hat vielerlei Anwendungen in technis- chen, wissenschaftlichen und auch ökonomischen Bereichen. Das Verstehen von Löserver- halten kann dabei helfen Löser zu verbessern, angemessene Hyperparameter zu wählen, und performantere nichtlineare Programme zu formulieren. Diese Masterarbeit schlägt ein Visual Analytics Werkzeug zur Analyse von Opti- mierungsproblemen vor. Der Optimierungsvorgang wird durch eine Menge von zwei- dimensionalen Trajektorien dargestellt, die den Verlauf der zwischenzeitigen Lösungen während des Prozesses repräsentieren. Das erlaubt es uns einen Überblick über die zeitliche Entwicklung des Optimierungsvorgangs zu gewinnen. Um eine detaillierte Analyse zu ermöglichen, werden ergänzende Ansichten zur Darstellung der Nebenbedingungsverlet- zungen und Bereiche zulässiger Lösungen hinzugefügt. Außerdem werden verschiedene Interaktionstechniken implementiert, um die Exploration zu erleichtern. Um die Nüt- zlichkeit des Ansatzes zu zeigen, werden die Ergebnisse aus einer beispielhaften Analyse eines Optimierungslogs zur Bewegungsplanung eines Roboters präsentiert. 3 Contents 1 Introduction 11 1.1 Motivation..................................... 11 1.2 Goals........................................ 12 1.3 Outline...................................... 13 2 Related Work 15 2.1 Optimization Visualization............................ 15 2.2 Visualization of Evolution............................ 17 3 Constraint Optimization 21 3.1 Optimization Problem.............................. 21 3.2 Nonlinear Programming............................. 22 4 Visualizing Optimization Trajectories 27 4.1 Visual Representation.............................. 27 4.2 Robot Path Trajectories.............................. 27 4.3 Dimensionality Reduction............................ 29 4.4 Constraints Visualization............................. 32 4.5 Explore Through Interaction........................... 34 5 Implementation 39 5.1 Architecture.................................... 39 5.2 Dataset Description................................ 40 5.3 Dimensionality Reduction Considerations................... 42 5.4 Used Technologies................................ 42 6 Evaluation 45 6.1 Visualization Tool Overview........................... 45 6.2 Use Case Scenarios................................ 47 7 Conclusion 55 7.1 Summary..................................... 55 7.2 Results....................................... 55 7.3 Future Work.................................... 56 Bibliography 59 5 List of Figures 3.1 Nonlinear program examples.......................... 22 4.1 Optimization Trajectory............................. 28 4.2 Per Configuration Trajectories.......................... 29 4.3 Dedicated Time Axis & Linesearch Highlighting................ 31 4.4 Dimensionality Changing Trajectories...................... 31 4.5 Per Configuration Trajectory Violations..................... 32 4.6 Constraint-Violations-View............................ 33 4.7 Constraint Curve Plots.............................. 33 4.8 Zooming in on a view.............................. 34 4.9 Supplementary GUI for Filtering......................... 35 4.10 Selected Per Configuration Trajectory...................... 36 4.11 Selected Per Configuration Trajectory...................... 37 4.12 Constraint-Violations of Selected Groups.................... 38 5.1 MVC Pattern.................................... 39 6.1 Visualization Tool Overview........................... 45 6.2 Video Sequence of Robot Movement...................... 47 6.3 Analysis of Robot Path Sections......................... 48 6.4 Analysis of Robot Path Velocities......................... 49 6.5 Analysis of Initial and Final Path........................ 49 6.6 Analysis of Path Evolution............................ 50 6.7 Analysis of Constraint Evolution......................... 51 6.8 Analysis of Constraint Violations on Trajectories................ 52 6.9 Analysis of Constraint Violations........................ 53 7 List of Listings 3.1 Pseudo code implementation of the augmented Lagrangian method..... 24 3.2 Pseudo code implementation of Newton’s method............... 25 5.1 Example KOMO log file content......................... 44 9 1 Introduction 1.1 Motivation Mathematical optimization plays an important role in today’s world where complex systems emerge in many different technical, economical and scientific fields. The subdiscipline constrained optimization also known as linear-/nonlinear programming, that deals with problems of finding optima within a constrained set of possibilities, is particularly challeng- ing. It has many applications such as utility maximization and expenditure minimization in microeconomics, route planning for drones [ZD15] or motion planning for autonomously moving robots [Tou15], solving inverse imaging problems such as tomographic reconstruc- tion [ABF11] or optimizing signal transmitting systems for minimal transmission power [YL07]. Constrained optimization bears many challenges starting with the formulation of the optimization problem that includes the objective function, as well as equality and inequality constraint functions. Functions can be linear or nonlinear, differentiable or discrete, and the optimization problem is classified accordingly, which leads to different possibilities for solving. Nonlinear programs (NLP) can be especially hard to solve due to possible local optima, disconnected feasible regions and hard to determine feasible starting points [Chi06]. Moreover, quite some effort has been put into determining whether an NLP is feasible at all, as it is not trivial [AA95; NTY99]. Apart from these general difficulties in constrained optimization, the choice of algorithm for solving may change the result, due to its individual behavior. Choosing an appropriate algorithm depends on the problem and whether it fulfills certain requirements the algorithm imposes. For iterative solvers different initializations can yield different results, or may even fail to produce a feasible solution in one case but not the other. Last but not least, the choice of parameters to tune an individual implementation of an algorithm can have a major effect on the optimization’s outcome. Exemplary optimization problems from text books may be easy to understand and are great for building intuition on the subject, but real world problems quickly become very complex for several reasons, such as a large number of constraints, which creates a large number of complicated regions. High dimensional problems are quite common and harder to grasp, especially when constraints depend on several of the dimensions. Also, nonlinear objectives or constraints make it difficult to envision a problem geometrically. Time dependent systems allow for further complication when introducing constraints that have to be met only at specific times or are subject to sets of subsequent time steps. 11 1 Introduction Ultimately a solver may generate an unsatisfactory solution or generate solutions under intolerable circumstances. For example an infeasible solution or a solution that is only locally optimal is generated. Maybe the algorithm takes too long to converge or does not converge at all, or the algorithm only succeeds for specific initializations. Diagnosis of the causes for such behavior is hard due to the sheer amount of variables that come into play during an optimization and due to the complexity of the optimization problem itself. Due to the fuzziness of such diagnostic tasks and the impossibility to algorithmically tackle them, it seems reasonable to leverage visualization in order to get a better understanding of the optimization problem and the corresponding algorithm behavior. The knowledge of a domain expert paired with a visual analytics tool has been proven to work well in these kind of scenarios in the past and should be able to bring light into the dark. 1.2 Goals Since constraint optimization solvers often employ iterative methods to refine their solution and work their way to the optimum step by step, there is an inherent notion of trajectories that an optimization algorithm leaves behind. These trajectories promise to be insightful with