The R-to-MOSEK Optimization Interface
Henrik Alsing Friberg
MOSEK ApS, Fruebjergvej 3, Box 16, DK 2100 Copenhagen, Blog: http://themosekblog.blogspot.com
ISMP, Berlin 2012 http://www.mosek.com Introduction
2 / 48 What is R?
Introduction The R Project What is R? What is MOSEK? What is Rmosek? A fast open-source programming language for Installation technical computing and graphics. Simple models
Advanced models Better formulations Highlights: Data handling and visualization ■ One million users – Intel Capital, 2009 High performance
Summary ■ The Comprehensive R Archive Network (3895 packages and counting)
■ Direct interfaces to C and Fortran (as well as C++, Perl, Java, Python, Matlab, Excel etc.)
■ Revolution Analytics
■ RStudio 3 / 48 What is MOSEK?
Introduction MOSEK What is R? What is MOSEK? What is Rmosek? A high performance software library designed to Installation solve large-scale convex optimization problems. Simple models
Advanced models Better formulations Highlights: Data handling and visualization ■ Linear constraints (LP) High performance Summary ■ Second-order cone constraints (SOCP)
■ Semidefinite constraints (SDP)
■ Mixed-integer variables (MIP)
■ Quadratic terms (QP and QCQP)
■ Separable convex operators – log(xi), exp(xi), ... 4 / 48 What is Rmosek?
Introduction Rmosek What is R? What is MOSEK? What is Rmosek? A simple interface from R to the optimizers of Installation MOSEK. Simple models
Advanced models Better formulations Why? Data handling and visualization ■ Some financial customers find R better than MATLAB. High performance
Summary ■ Replace unofficial implementations.
5 / 48 What is Rmosek?
Introduction Rmosek What is R? What is MOSEK? What is Rmosek? A simple interface from R to the optimizers of Installation MOSEK. Simple models
Advanced models Better formulations Why? Data handling and visualization ■ Some financial customers find R better than MATLAB. High performance
Summary ■ Replace unofficial implementations.
Goals:
■ Open-source
■ Highly effective
■ Integrate with what R users normally do 5 / 48 Installation
Introduction A oneliner for Windows, MacOS and Linux: What is R? What is MOSEK? install.packages("Rmosek", type="source") What is Rmosek? Installation
Simple models
Advanced models
Better formulations Data handling and visualization
High performance
Summary
6 / 48 Installation
Introduction A oneliner for Windows, MacOS and Linux: What is R? What is MOSEK? install.packages("Rmosek", type="source") What is Rmosek? Installation
Simple models Package overview: Advanced models library(help="Rmosek") Better formulations Data handling and visualization Major functions:
High performance ■ mosek(prob) Summary ■ mosek write(prob, file) ■ mosek read(file)
The problem is an R-variable.. Create as you like!
6 / 48 Simple models
7 / 48 Linear optimization
Introduction
Simple models 3x1 + 1x2 + 5x3 + 1x4 Linear optimization maximize 1 2 3 Advanced models subject to 3x + 1x + 2x = 30,
Better formulations 2x1 + 1x2 + 3x3 + 1x4 ≥ 15, Data handling and 2x2 + 3x4 ≤ 25, visualization
High performance 0 ≤ x1 ≤ ∞, Summary 0 ≤ x2 ≤ 10, 0 ≤ x3 ≤ ∞, 0 ≤ x4 ≤ ∞.
8 / 48 Linear optimization
Introduction lo1 <- list() Simple models Linear optimization Advanced models ## Objective Better formulations lo1$sense <- "max" Data handling and visualization lo1$c <- c(3, 1, 5, 1)
High performance Summary ## Constraint matrix lo1$A <- spMatrix(nrow = 3, ncol = 4, i = c(1, 2, 1, 2, 3, 1, 2, 2, 3), j = c(1, 1, 2, 2, 2, 3, 3, 4, 4), x = c(3, 2, 1, 1, 2, 2, 3, 1, 3) )
9 / 48 Linear optimization
Introduction ## Constraint bounds Simple models Linear optimization lo1$bc <- cbind(c(30, 30), Advanced models c(15, Inf), Better formulations c(-Inf, 25)) Data handling and visualization
High performance ## Variable bounds Summary lo1$bx <- cbind(c(0, Inf), c(0, 10), c(0, Inf), c(0, Inf))
10 / 48 Advanced models
11 / 48 Second-order cone and semidefinite optimization
2 1 0 Introduction x1 x1 Simple models minimize 1 2 1 • +
Advanced models 0 1 2 Second-order cone 1 0 0 and semidefinite optimization subject to 0 1 0 • x1 + x1 = 1.0 , Better formulations 0 0 1 Data handling and visualization 1 1 1 High performance 1 1 1 • x1 + x2 + x3 = 0.5 , Summary 1 1 1 3 2 2 (x1,x2,x3) ∈ R , x1 ≥ x + x , x1 0 . p 2 3
12 / 48 Second-order cone and semidefinite optimization
2 1 0 Introduction x1 x1 Simple models minimize 1 2 1 • +
Advanced models 0 1 2 Second-order cone 1 0 0 and semidefinite optimization subject to 0 1 0 • x1 + x1 = 1.0 , Better formulations 0 0 1 Data handling and visualization 1 1 1 High performance 1 1 1 • x1 + x2 + x3 = 0.5 , Summary 1 1 1 3 2 2 (x1,x2,x3) ∈ R , x1 ≥ x + x , x1 0 . p 2 3
sdo1 <- list() sdo1$sense <- "minimize" sdo1$c <- c(1,0,0)
13 / 48 Second-order cone and semidefinite optimization
2 1 0 Introduction x1 x1 Simple models minimize 1 2 1 • +
Advanced models 0 1 2 Second-order cone 1 0 0 and semidefinite optimization subject to 0 1 0 • x1 + x1 = 1.0 , Better formulations 0 0 1 Data handling and visualization 1 1 1 High performance 1 1 1 • x1 + x2 + x3 = 0.5 , Summary 1 1 1 3 2 2 (x1,x2,x3) ∈ R , x1 ≥ x + x , x1 0 . p 2 3
sdo1$A <- spMatrix(nrow=2, ncol=3, i = c(1,2,2), j = c(1,2,3), x = c(1,1,1) )
14 / 48 Second-order cone and semidefinite optimization
2 1 0 Introduction x1 x1 Simple models minimize 1 2 1 • +
Advanced models 0 1 2 Second-order cone 1 0 0 and semidefinite optimization subject to 0 1 0 • x1 + x1 = 1.0 , Better formulations 0 0 1 Data handling and visualization 1 1 1 High performance 1 1 1 • x1 + x2 + x3 = 0.5 , Summary 1 1 1 3 2 2 (x1,x2,x3) ∈ R , x1 ≥ x + x , x1 0 . p 2 3
sdo1$bc <- cbind(c(1.0, 1.0), c(0.5, 0.5))
15 / 48 Second-order cone and semidefinite optimization
2 1 0 Introduction x1 x1 Simple models minimize 1 2 1 • +
Advanced models 0 1 2 Second-order cone 1 0 0 and semidefinite optimization subject to 0 1 0 • x1 + x1 = 1.0 , Better formulations 0 0 1 Data handling and visualization 1 1 1 High performance 1 1 1 • x1 + x2 + x3 = 0.5 , Summary 1 1 1 3 2 2 (x1, x2, x3) ∈ R , x1 ≥ x + x , x1 0 . p 2 3
sdo1$bx <- cbind(c(-Inf, Inf), c(-Inf, Inf), c(-Inf, Inf))
16 / 48 Second-order cone and semidefinite optimization
2 1 0 Introduction x1 x1 Simple models minimize 1 2 1 • +
Advanced models 0 1 2 Second-order cone 1 0 0 and semidefinite optimization subject to 0 1 0 • x1 + x1 = 1.0 , Better formulations 0 0 1 Data handling and visualization 1 1 1 High performance 1 1 1 • x1 + x2 + x3 = 0.5 , Summary 1 1 1 3 2 2 (x1,x2,x3) ∈ R , x1 ≥ x + x , x1 0 . p 2 3
sdo1$cones <- cbind( list("quad", c(1,2,3)) )
17 / 48 Second-order cone and semidefinite optimization
2 1 0 Introduction x1 x1 Simple models minimize 1 2 1 • +
Advanced models 0 1 2 Second-order cone 1 0 0 and semidefinite optimization subject to 0 1 0 • x1 + x1 = 1.0 , Better formulations 0 0 1 Data handling and visualization 1 1 1 High performance 1 1 1 • x1 + x2 + x3 = 0.5 , Summary 1 1 1 3 2 2 (x1,x2,x3) ∈ R , x1 ≥ x + x , x1 0 . p 2 3
sdo1$bardim <- c(3)
18 / 48 Second-order cone and semidefinite optimization
2 1 0 Introduction x1 x1 Simple models minimize 1 2 1 • +
Advanced models 0 1 2 Second-order cone 1 0 0 and semidefinite optimization subject to 0 1 0 • x1 + x1 = 1.0 , Better formulations 0 0 1 Data handling and visualization 1 1 1 High performance 1 1 1 • x1 + x2 + x3 = 0.5 , Summary 1 1 1 3 2 2 (x1,x2,x3) ∈ R , x1 ≥ x + x , x1 0 . p 2 3
# Block triplet format (SDPA) sdo1$barc$j <- c(1,1,1,1,1) sdo1$barc$k <- c(1,2,2,3,3) sdo1$barc$l <- c(1,1,2,2,3) sdo1$barc$v <- c(2,1,2,1,2) 19 / 48 Second-order cone and semidefinite optimization
2 1 0 Introduction x1 x1 Simple models minimize 1 2 1 • +
Advanced models 0 1 2 Second-order cone 1 0 0 and semidefinite optimization subject to 0 1 0 • x1 + x1 = 1.0 , Better formulations 0 0 1 Data handling and visualization 1 1 1 High performance 1 1 1 • x1 + x2 + x3 = 0.5 , Summary 1 1 1 3 2 2 (x1,x2,x3) ∈ R , x1 ≥ x + x , x1 0 . p 2 3
# Block triplet format (SDPA) sdo1$barA$i <- c(1,1,1, 2,2,2,2,2,2) sdo1$barA$j <- c(1,1,1, 1,1,1,1,1,1) sdo1$barA$k <- c(1,2,3, 1,2,3,2,3,3) sdo1$barA$l <- c(1,2,3, 1,1,1,2,2,3) sdo1$barA$v <- c(1,1,1, 1,1,1,1,1,1) 20 / 48 Better formulations
21 / 48 General quadratic formulation
Introduction 1 T T T minimize 2 x Qx + c x + e v + k Simple models subject to −v ≤ x ≤ v, Advanced models
Better formulations General quadratic formulation Separable quadratic formulation Conic quadratic formulation rr <- mosek_read("probQP.opf") Data handling and qp <- rr$prob visualization
High performance
Summary rqp <- mosek(qp)
22 / 48 General quadratic formulation
Introduction 1 T T T minimize 2 x Qx + c x + e v + k Simple models subject to −v ≤ x ≤ v, Advanced models
Better formulations General quadratic formulation Separable quadratic formulation Conic quadratic formulation ITE PFEAS DFEAS POBJ TIME Data handling and 8 1.3e-14 3.1e-08 4.213156702e+03 74.84 visualization 9 1.2e-14 2.5e-09 4.213153627e+03 82.54 High performance
Summary 10 1.1e-14 1.3e-10 4.213153292e+03 90.21
23 / 48 Separable quadratic formulation
T Introduction Cholesky Factorization: Q = A A Simple models
Advanced models 1 T T T Better formulations minimize 2 w w + c x + e v + k General quadratic formulation subject to −v ≤ x ≤ v, Separable quadratic Ax w, formulation = Conic quadratic formulation Data handling and visualization
High performance
Summary
24 / 48 Separable quadratic formulation
T Introduction Cholesky Factorization: Q = A A Simple models
Advanced models 1 T T T Better formulations minimize 2 w w + c x + e v + k General quadratic formulation subject to −v ≤ x ≤ v, Separable quadratic Ax w, formulation = Conic quadratic formulation Data handling and visualization
High performance
Summary Q <- sparseMatrix(qp$qobj$i, qp$qobj$j, x = qp$qobj$v, dims = c(nx,nx), symmetric = TRUE) A <- chol(Q)
25 / 48 Separable quadratic formulation
T Introduction Cholesky Factorization: Q = A A Simple models
Advanced models 1 T T T Better formulations minimize 2 w w + c x + e v + k General quadratic formulation subject to −v ≤ x ≤ v, Separable quadratic Ax w, formulation = Conic quadratic formulation Data handling and visualization
High performance
Summary sqp <- qp
sqp$qobj <- list(i = 1:nx, j = 1:nx, v = rep(1,nx))
26 / 48 Separable quadratic formulation
T Introduction Cholesky Factorization: Q = A A Simple models
Advanced models 1 T T T Better formulations minimize 2 w w + c x + e v + k General quadratic formulation subject to −v ≤ x ≤ v, Separable quadratic Ax w, formulation = Conic quadratic formulation Data handling and visualization
High performance
Summary eye <- Diagonal(nx) zeros <- Matrix(0, nrow=nx, ncol=nx)
sqp$A <- rBind( sqp$A, cBind(-eye, A, zeros) )
27 / 48 Separable quadratic formulation
T Introduction Cholesky Factorization: Q = A A Simple models
Advanced models 1 T T T Better formulations minimize 2 w w + c x + e v + k General quadratic formulation subject to −v ≤ x ≤ v, Separable quadratic Ax w, formulation = Conic quadratic formulation Data handling and visualization
High performance
Summary ITE PFEAS DFEAS POBJ TIME 17 1.1e-06 1.5e-08 4.213156615e+03 29.76 18 1.0e-06 1.5e-08 4.213156490e+03 31.36 19 8.0e-06 4.8e-10 4.213153446e+03 32.99
28 / 48 Conic quadratic formulation
Introduction 1 T T minimize 2 p + c x + e v + k Simple models −v ≤ x ≤ v, Advanced models subject to Ax = w, Better formulations 2 General quadratic 2qp ≥ kwk2 formulation 1 Separable quadratic q = 2 formulation Conic quadratic formulation Data handling and visualization
High performance
Summary
29 / 48 Conic quadratic formulation
Introduction 1 T T minimize 2 p + c x + e v + k Simple models −v ≤ x ≤ v, Advanced models subject to Ax = w, Better formulations 2 General quadratic 2qp ≥ kwk2 formulation 1 Separable quadratic q = 2 formulation Conic quadratic formulation Data handling and visualization
High performance cqp$qobj <- NULL Summary
30 / 48 Conic quadratic formulation
Introduction 1 T T minimize 2 p + c x + e v + k Simple models −v ≤ x ≤ v, Advanced models subject to Ax = w, Better formulations 2 General quadratic 2qp ≥ kwk2 formulation 1 Separable quadratic q = 2 formulation Conic quadratic formulation Data handling and visualization
High performance cqp$c <- c(0.0, 0.5, cqp$c) Summary
cqp$bx <- cBind( c(0.5, 0.5), c(0.0, Inf), cqp$bx )
31 / 48 Conic quadratic formulation
Introduction 1 T T minimize 2 p + c x + e v + k Simple models −v ≤ x ≤ v, Advanced models subject to Ax = w, Better formulations 2 General quadratic 2qp ≥ kwk2 formulation 1 Separable quadratic q = 2 formulation Conic quadratic formulation Data handling and visualization
High performance cqp$cones <- cBind( Summary list("rquad", 1:(2+nw)) )
32 / 48 Conic quadratic formulation
Introduction 1 T T minimize 2 p + c x + e v + k Simple models −v ≤ x ≤ v, Advanced models subject to Ax = w, Better formulations 2 General quadratic 2qp ≥ kwk2 formulation 1 Separable quadratic q = 2 formulation Conic quadratic formulation Data handling and visualization
High performance ITE PFEAS DFEAS POBJ TIME Summary 10 8.7e-07 1.7e-06 4.213161986e+03 6.67 11 1.3e-07 2.5e-07 4.213154532e+03 7.17 12 1.2e-07 2.5e-07 4.213154518e+03 7.66
33 / 48 Data handling and visualization
34 / 48 Problem inspection
Introduction r <- mosek read("network.mps") Simple models problem <- r$prob Advanced models Better formulations pdf("picture.pdf") Data handling and visualization print(image(problem$A)) Problem inspection Math made easy dev.off() The efficient frontier
High performance
Summary
35 / 48 Math made easy
Introduction Normalize a data-matrix: Simple models (each column of X is a time series) Advanced models Better formulations N <- nrow(X) Data handling and visualization mu r <- matrix(1,nrow=N) %*% colMeans(X) Problem inspection Xbar <- 1/sqrt(N-1) (X - mu r) Math made easy * The efficient frontier
High performance
Summary
36 / 48 Math made easy
Introduction Find the QR factorization: Simple models Advanced models factor <- qr(Xbar) Better formulations Q <- qr.Q(factor) Data handling and visualization R <- qr.R(factor) Problem inspection Math made easy The efficient frontier
High performance
Summary
37 / 48 Math made easy
Introduction Save results and read them back in: Simple models Advanced models R <- as.matrix( Better formulations read.csv("data/qr-r.csv", header=FALSE) Data handling and visualization ) Problem inspection Math made easy The efficient frontier
High performance
Summary
38 / 48 The efficient frontier
Introduction T minimize r (w0 + x) − λkR(w0 + x)k Simple models T subject to e x = 0. Advanced models
Better formulations Data handling and visualization Problem inspection Math made easy The efficient frontier
High performance
Summary
39 / 48 The efficient frontier
Introduction T minimize r (w0 + x) − λkR(w0 + x)k Simple models T subject to e x = 0. Advanced models
Better formulations Data handling and visualization plot(RISK, RET, 'l', Problem inspection Math made easy xlim = c(0, 0.1), The efficient frontier ylim = c(tlow, thigh))
High performance
Summary points(SHARPE RISK, SHARPE RET)
lines(x = c(0,1), y = rf*sum(w0) + SHARPE RATIO*c(0,1))
39 / 48 The efficient frontier
Introduction T minimize r (w0 + x) − λkR(w0 + x)k Simple models T subject to e x = 0. Advanced models
Better formulations Data handling and visualization Problem inspection Math made easy The efficient frontier
High performance
Summary
40 / 48 High performance
41 / 48 Column generation
Introduction Multiple knapsack problem: Simple models Advanced models ■ One pool of (weight, value)-items Better formulations Data handling and ■ N knapsacks of limited capacity visualization High performance ■ Maximize profit Column generation Solving subproblems Performance
Summary
42 / 48 Column generation
Introduction Multiple knapsack problem: Simple models Advanced models ■ One pool of (weight, value)-items Better formulations Data handling and ■ N knapsacks of limited capacity visualization High performance ■ Maximize profit Column generation Solving subproblems Performance One subproblem per knapsack (MILP): Summary ■ Select items for the individual knapsack
■ Maximize reduced-cost-profit
One master (LP):
■ Select item-disjoint solutions for each knapsack
■ Maximize profit 42 / 48 Solving subproblems
Introduction TASK: Update and solve subproblems in parallel. Simple models
Advanced models Better formulations >> require("doMC") Data handling and visualization Loading required package: doMC High performance Loading required package: foreach Column generation Solving Loading required package: iterators subproblems Performance Loading required package: codetools
Summary Loading required package: multicore
>> registerDoMC(cores = 16)
43 / 48 Solving subproblems
Introduction mosek clean() Simple models Advanced models SPsols <- Better formulations foreach(i = 1:nsack) %dopar% { Data handling and visualization prob <- updateObj(SP[[i]], High performance itemProfits, Column generation Solving itemDuals, subproblems sackDuals[i], Performance
Summary MPfeasibility) sol <- getSolution(prob)
if (isImproving(prob, sol)) { return(sol) } else { return(NULL) } } 44 / 48 Performance
Introduction 200 knapsacks of: Simple models Advanced models ■ capacity = 1, 2, ..., 200 Better formulations Data handling and visualization 200 items of:
High performance Column generation ■ weight = 1, 2, ..., 200 Solving subproblems ■ Performance profit = 200, 199, ..., 1
Summary
45 / 48 Performance
Introduction 200 knapsacks of: Simple models Advanced models ■ capacity = 1, 2, ..., 200 Better formulations Data handling and visualization 200 items of:
High performance Column generation ■ weight = 1, 2, ..., 200 Solving subproblems ■ Performance profit = 200, 199, ..., 1
Summary Optimality in 449 seconds:
■ 37600 subproblems solved (17117 columns added)
■ 187 master problems solved
45 / 48 Performance
Introduction 200 knapsacks of: Simple models Advanced models ■ capacity = 1, 2, ..., 200 Better formulations Data handling and visualization 200 items of:
High performance Column generation ■ weight = 1, 2, ..., 200 Solving subproblems ■ Performance profit = 200, 199, ..., 1
Summary Optimality in 449 seconds:
■ 37600 subproblems solved (17117 columns added)
■ 187 master problems solved
Average speed: 84 problems per second. 45 / 48 Summary
46 / 48 More information
Introduction The R project Simple models ■ r-project.org Advanced models Better formulations Home of MOSEK ApS Data handling and visualization ■ mosek.com
High performance Summary Need help? MOSEK Google Group! More information ■ Conclusion groups.google.com/group/mosek
The Rmosek introduction page ■ cran.r-project.org/web/packages/Rmosek/ index.html
The Rmosek development site ■ rmosek.r-forge.r-project.org
47 / 48 Conclusion
Introduction Goals: Simple models
Advanced models ! Open-source Better formulations
Data handling and ! Highly effective visualization
High performance ! Integrate with what R users normally do Summary More information Conclusion
48 / 48 Conclusion
Introduction Goals: Simple models
Advanced models ! Open-source Better formulations
Data handling and ! Highly effective visualization
High performance ! Integrate with what R users normally do Summary More information Conclusion Thank you!
48 / 48