Deterministic Optimization and Design Jay R
Total Page:16
File Type:pdf, Size:1020Kb
Deterministic Optimization and Design Jay R. Lund UC Davis Fall 2017 Deterministic Optimization and Design Class Notes for Civil Engineering 153 Department of Civil and Environmental Engineering University of California, Davis Jay R. Lund February 2016 © 2017 Jay R. Lund "... engineers who can do only the best of what has already been done will be replaced by a computer system." Daniel C. Drucker Engineering Education, Vol. 81, No. 5, July/August 1991, p. 480. "Good management is the art of making problems so interesting and their solutions so constructive that everyone wants to get to work and deal with them." Paul Hawken Acknowledgments: These notes have improved steadily over the years from corrections and suggestions by numerous students, TAs, and substitute instructors. Particularly useful improvements have come from Ken Kirby, Orit Kalman, Min Yip, Stacy Tanaka, David Watkins, and Rui Hui, with many others contributing as well. 1 Deterministic Optimization and Design Jay R. Lund UC Davis Fall 2017 Table of Contents Some Thoughts on Optimization ................................................................................................ 3 Introduction/Overview ................................................................................................................ 5 Design of A Bridge Over A Gorge ............................................................................................. 7 Systems Analysis ...................................................................................................................... 10 Steps to Solving a Problem Using Optimization ...................................................................... 12 Unconstrained Optimization Using Calculus ............................................................................ 16 Analytical vs. Search methods .................................................................................................. 21 Lagrange Multipliers ................................................................................................................. 24 Alternative Method for Inequality Constraints ......................................................................... 31 Mathematical Programming ...................................................................................................... 33 Karush-Kuhn-Tucker (KKT) Conditions for an Optimal Solution ........................................... 36 Linear Programming ................................................................................................................. 37 The Simplex Method: A Graphical Interpretation ................................................................... 41 The Simplex Tableau ................................................................................................................ 42 The Simplex Method - An Algorithm ....................................................................................... 44 Some Simplex Details ............................................................................................................... 48 Implementing the Big-M Method ............................................................................................. 52 Sensitivity Analysis in Linear Programming ............................................................................ 54 Duality....................................................................................................................................... 58 Example optimization: California Hydropower ........................................................................ 61 Transportation Problems ........................................................................................................... 61 Scheduling Problems ................................................................................................................ 63 LP Solution of Non-Linear Objectives with Linear Constraints .............................................. 65 Integer Programming and Mixed Integer-Linear Programming ............................................... 67 Summary of the BIP Branch and Bound Algorithm ................................................................. 69 More Non-Linear Objective Functions with Linear Constraints Using Integer-Linear Programming............................................................................................................................. 72 Computational Aspects ............................................................................................................. 73 Dynamic Programming (DP) .................................................................................................... 74 More DP Applications .............................................................................................................. 81 Non-Linear Programming ......................................................................................................... 84 Genetic Algorithms/Evolutionary Algorithms .......................................................................... 93 The Final Days ........................................................................................................................ 100 Appendices .............................................................................................................................. 101 PROBLEM FORMULATION: SOME GUIDELINES ......................................................... 103 INTERPRETATION OF RESULTS: SOME GUIDELINES ................................................ 104 INTRODUCTION TO SOLVING LINEAR PROGRAMS USING LINDO ........................ 105 THE SIMPLEX ALGORITHM .............................................................................................. 106 Selected Bibliography ............................................................................................................. 107 2 Deterministic Optimization and Design Jay R. Lund UC Davis Fall 2017 Some Thoughts on Optimization "All models are wrong, but some are useful." - G.E.P. Box (1979), Robustness in Statistics. "Operations research is the art of giving bad answers to problems to which otherwise worse answers are given." - T.L. Saaty (1959), Mathematical Methods of Operations Research, p. 3. "What would life be without arithmetic, but a scene of horrors." - Revd. Sydney Smith, 1835 “The purpose of mathematical programming is insight, not numbers” – A.M. Geoffrion 1978 "Decision analysis separates a large-scale problem into its sub-parts, each of which is simpler to manipulate and diagnose. After the separate elements are carefully examined, the results are synthesized to give insights into the original problem." - Harvey Wagner (1975), Principles of Operations Research. "The adoption of operations research calls for an act of faith in the potential benefits of a systematic approach to decision-making." - Harvey Wagner (1975), Principles of Operations Research. "Even when quantitative analysis is of central importance for a managerial decision process, an operations- research-oriented system never supplies all the information required for action, no matter how sophisticated the system's design. Furthermore, a truly successful implementation of an operations research system must apply behavioral as well as mathematical science, because the resultant system must interact with human beings. And finally, the very process of constructing an operations research system involves the exercise of judgement in addition to the logical manipulation of symbols and data." - Harvey Wagner (1975), Principles of Operations Research. "Le mieux est l'ennemi du bien." The best is the enemy of the good. - Voltaire “A great craftsman does not put aside the plumb-line for the benefit of a clumsy carpenter.” Mencius (China, c. 300 B.C., Book VII, Part A, 41) "Life must be lived forwards, but it can only be understood backwards" - Soren Kierkegaard "A man's gotta know his limitations." - Clint Eastwood "A problem well put is a problem half solved." - Anon. “The bottom line for mathematicians is that the architecture has to be right. In all the mathematics that I did, the essential point was to find the right architecture. It's like building a bridge. Once the main lines of the structure are right, then the details miraculously fit. The problem is the overall design.” Freeman Dyson, "Freeman Dyson: Mathematician, Physicist, and Writer" interview with Donald J. Albers, The College Mathematics Journal, vol 25, no. 1, January 1994. “An expert problem solver must be endowed with two incompatible qualities, a restless imagination and a patient pertinacity.” Howard W. Eves in Mathematical Circles, Boston: Prindle, Weber and Schmidt, 1969. “The errors of definitions multiply themselves according as the reckoning proceeds; and lead men into absurdities, which at last they see but cannot avoid, without reckoning anew from the beginning.” Thomas Hobbes, in J. R. Newman (ed.) The World of Mathematics, New York: Simon and Schuster, 1956. 3 Deterministic Optimization and Design Jay R. Lund UC Davis Fall 2017 “In order to translate a sentence from English into French two things are necessary. First, we must understand thoroughly the English sentence. Second, we must be familiar with the forms of expression peculiar to the French language. The situation is very similar when we attempt to express in mathematical symbols a condition proposed in words. First, we must understand thoroughly the condition. Second, we must be familiar with the forms of mathematical expression. George Polyá, How to