Paul E. Fishback Linear and Nonlinear Programming with Maple: An Interactive, Applications-Based Approach For Barb, Andy, and Margaret Contents List of Figures xiii List of Tables xv Foreword xix I Linear Programming 1 1 AnIntroductiontoLinearProgramming 3 1.1 The Basic Linear Programming Problem Formulation . 3 1.1.1 A Prototype Example: The Blending Problem . 4 1.1.2 Maple’s LPSolve Command................ 7 1.1.3 TheMatrixInequalityFormofanLP. 8 Exercises ............................... 10 1.2 Linear Programming: A Graphical Perspective in R2 ..... 14 Exercises ............................... 17 1.3 BasicFeasibleSolutions . 20 Exercises ............................... 25 2 The Simplex Algorithm 29 2.1 TheSimplexAlgorithm . 29 2.1.1 AnOverviewoftheAlgorithm . 29 2.1.2 AStep-By-StepAnalysisoftheProcess . 30 2.1.3 Solving Minimization Problems . 33 2.1.4 A Step-by-Step Maple Implementation of the Simplex Algorithm .......................... 34 Exercises ............................... 38 2.2 Alternative Optimal/Unbounded Solutions and Degeneracy . 40 2.2.1 Alternative Optimal Solutions . 40 2.2.2 UnboundedSolutions . 41 2.2.3 Degeneracy ......................... 43 Exercises ............................... 45 2.3 ExcessandArtificialVariables:TheBigMMethod . 48 Exercises ............................... 53 2.4 A Partitioned Matrix View of the Simplex Method . 55 2.4.1 PartitionedMatrices . 55 vii viii 2.4.2 PartitionedMatriceswithMaple . 56 2.4.3 The Simplex Algorithm as Partitioned Matrix Multipli- cation............................. 57 Exercises ............................... 62 2.5 TheRevisedSimplexAlgorithm . 64 2.5.1 Notation........................... 64 2.5.2 Observations about the Simplex Algorithm . 65 2.5.3 AnOutlineoftheMethod . 65 2.5.4 Application to the FuelPro LP............... 66 Exercises ............................... 69 2.6 Moving beyond the Simplex Method: An Interior Point Algorithm .............................. 71 2.6.1 The Origin of the Interior Point Algorithm . 71 2.6.2 TheProjectedGradient. 71 2.6.3 AffineScaling........................ 74 2.6.4 SummaryoftheMethod. 77 2.6.5 Application of the Method to the FuelPro LP ...... 78 2.6.6 A Maple Implementation of the Interior Point Algorithm .......................... 79 Exercises ............................... 81 3 StandardApplicationsofLinearProgramming 85 3.1 TheDietProblem .......................... 85 3.1.1 EatingforCheaponaVeryLimitedMenu . 85 3.1.2 The Problem Formulation and Solution, with Help from Maple ............................ 86 Exercises ............................... 89 3.2 Transportation and Transshipment Problems . 90 3.2.1 ACoalDistributionProblem . 90 3.2.2 The Integrality of the Transportation Problem Solution 91 3.2.3 Coal Distribution with Transshipment . 93 Exercises ............................... 95 3.3 BasicNetworkModels . 97 3.3.1 The Minimum Cost Network Flow Problem Formula- tion.............................. 97 3.3.2 Formulating and Solving the Minimum Cost Network FlowProblemwithMaple. 99 3.3.3 TheShortestPathProblem . 100 3.3.4 MaximumFlowProblems . 102 Exercises ............................... 104 4 DualityandSensitivityAnalysis 107 4.1 Duality ................................ 107 4.1.1 TheDualofanLP ..................... 107 4.1.2 WeakandStrongDuality . 109 ix 4.1.3 An Economic Interpretation of Duality . 114 4.1.4 AFinalNoteontheDualofanArbitraryLP . 115 4.1.5 TheZero-SumMatrixGame. 116 Exercises ............................... 120 4.2 SensitivityAnalysis . 124 4.2.1 Sensitivity to an Objective Coefficient .......... 125 4.2.2 Sensitivity to Constraint Bounds . 129 4.2.3 Sensitivity to Entries in the Coefficient Matrix A .... 134 4.2.4 Performing Sensitivity Analysis with Maple . 137 Exercises ............................... 139 4.3 TheDualSimplexMethod . 142 4.3.1 OverviewoftheMethod. 142 4.3.2 ASimpleExample . 143 Exercises ............................... 147 5 Integer Linear Programming 149 5.1 An Introduction to Integer Linear Programming and the BranchandBoundMethod . 149 5.1.1 ASimpleExample . 149 5.1.2 TheRelaxationofanILP. 150 5.1.3 TheBranchandBoundMethod. 151 5.1.4 Practicing the Branch and Bound Method with Maple . 158 5.1.5 Binary and Mixed Integer Linear Programming . 159 5.1.6 SolvingILPsDirectlywithMaple. 160 5.1.7 An Application of Integer Linear Programming: The TravelingSalespersonProblem . 161 Exercises ............................... 166 5.2 TheCuttingPlaneAlgorithm . 172 5.2.1 Motivation.......................... 172 5.2.2 TheAlgorithm ....................... 172 5.2.3 A Step-by-Step Maple Implementation of the Cutting PlaneAlgorithm .... .... .... ... .... ... 176 5.2.4 Comparison with the Branch and Bound Method . 179 Exercises ............................... 179 II NonlinearProgramming 181 6 AlgebraicMethodsforUnconstrainedProblems 183 6.1 NonlinearProgramming:AnOverview . 183 6.1.1 The General Nonlinear Programming Model . 183 6.1.2 Plotting Feasible Regions and Solving NLPs with Maple 184 6.1.3 APrototypeNLPExample . 187 Exercises ............................... 189 6.2 Differentiability and a Necessary First-Order Condition . 192 6.2.1 Differentiability. 192 x 6.2.2 Necessary Conditions for Local Maxima or Minima . 194 Exercises ............................... 197 6.3 Convexity and a Sufficient First-Order Condition . 199 6.3.1 Convexity .......................... 199 6.3.2 TestingforConvexity . 201 6.3.3 Convexity and The Global Optimal Solutions Theorem . 203 6.3.4 Solving the Unconstrained NLP for Differentiable, ConvexFunctions . 205 6.3.5 MultipleLinearRegression . 206 Exercises ............................... 209 6.4 Sufficient Conditions for Local and Global Optimal Solutions 212 6.4.1 QuadraticForms . 212 6.4.2 Positive Definite Quadratic Forms . 214 6.4.3 Second-order Differentiability and the Hessian Matrix 216 6.4.4 Using Maple To Classify Critical Points for the UnconstrainedNLP . 224 6.4.5 TheZero-SumMatrixGame,Revisited . 225 Exercises ............................... 227 7 NumericToolsforUnconstrainedNLPs 231 7.1 TheSteepestDescentMethod . 231 7.1.1 MethodDerivation . 231 7.1.2 A Maple Implementation of the Steepest Descent Method............................ 235 7.1.3 ASufficient Condition for Convergence . 237 7.1.4 TheRateofConvergence . 240 Exercises ............................... 242 7.2 Newton’sMethod ......................... 244 7.2.1 Shortcomings of the Steepest Descent Method . 244 7.2.2 MethodDerivation . 244 7.2.3 A Maple Implementation of Newton’s Method . 247 7.2.4 Convergence Issues and Comparison with the Steepest DescentMethod. 248 Exercises ............................... 253 7.3 TheLevenberg-MarquardtAlgorithm . 255 7.3.1 Interpolating between the Steepest Descent and NewtonMethods. ... .... .... .... ... ... 255 7.3.2 TheLevenbergMethod . 255 7.3.3 TheLevenberg-MarquardtAlgorithm . 257 7.3.4 A Maple Implementation of the Levenberg-Marquardt Algorithm .......................... 258 7.3.5 NonlinearRegression . 261 7.3.6 Maple’s Global Optimization Toolbox ........ 263 Exercises ............................... 263 xi 8 MethodsforConstrainedNonlinearProblems 267 8.1 The Lagrangian Function and Lagrange Multipliers . 267 8.1.1 SomeConvenientNotation . 268 8.1.2 The Karush-Kuhn-Tucker Theorem . 269 8.1.3 InterpretingtheMultiplier. 273 Exercises ............................... 275 8.2 ConvexNLPs ............................ 279 8.2.1 SolvingConvexNLPs . 279 Exercises ............................... 282 8.3 SaddlePointCriteria . 285 8.3.1 TheRestrictedLagrangian. 285 8.3.2 SaddlePointOptimalityCriteria . 287 Exercises ............................... 289 8.4 QuadraticProgramming . 292 8.4.1 Problems with Equality-type Constraints Only . 292 8.4.2 InequalityConstraints . 297 8.4.3 Maple’s QPSolve Command................ 298 8.4.4 TheBimatrixGame. 300 Exercises ............................... 305 8.5 Sequential Quadratic Programming . 309 8.5.1 Method Derivation for Equality-type Constraints . 309 8.5.2 TheConvergenceIssue. 314 8.5.3 Inequality-TypeConstraints . 315 8.5.4 A Maple Implementation of the Sequential Quadratic Programming Technique . 318 8.5.5 AnImprovedVersionoftheSQPT . 320 Exercises ............................... 323 A Projects 327 A.1 ExcavatingandLevelingaLargeLandTract . 328 A.2 TheJuiceLogisticsModel . 332 A.3 WorkSchedulingwithOvertime . 336 A.4 Diagnosing Breast Cancer with a Linear Classifier . 338 A.5 TheMarkowitzPortfolioModel . 342 A.6 AGameTheoryModelofaPredator-PreyHabitat . 345 B ImportantResultsfromLinearAlgebra 347 B.1 LinearIndependence . 347 B.2 TheInvertibleMatrixTheorem . 347 B.3 TransposeProperties . 348 B.4 PositiveDefiniteMatrices . 348 B.5 Cramer’sRule............................ 349 B.6 TheRank-NullityTheorem . 349 B.7 TheSpectralTheorem . 349 B.8 MatrixNorms ............................ 350 xii C Getting Started with Maple 351 C.1 TheWorksheetStructure . 351 C.2 Arithmetic Calculations and Built-In Operations . ... 353 C.3 ExpressionsandFunctions . 354 C.4 Arrays,Lists,Sequences,andSums . 357 C.5 Matrix Algebra and the LinearAlgebra Package . 359 C.6 PlotStructureswithMaple . 363 D Summary of Maple Commands 371 Bibliography 391 Index 395 List of Figures 1.1 Feasible region for FuelPro LP.................... 15 1.2 SamplecontourdiagramproducedbyMaple.. 16 1.3 LP with alternative optimal solutions. 17 1.4 Feasible region for FuelPro LP.................... 20 1.5 Feasible region for FuelPro LP.................... 23 2.1 FeasibleregionforLP2.4. 40 2.2 FeasibleregionforadegenerateLP2.5. 44 2.3 FeasibleregionforExercise5. 47 2.4 Feasible region