Decision-Making Under Severe Uncertainty: from Worst-Case Analysis to Robust Optimization

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Decision-making under severe uncertainty: from worst-case analysis to robust optimization

Moshe Sniedovich

School of Mathematics and Statistics The University of Melbourne www.moshe-online.com

AMSI Optimise 18 June 18-22, 2018 The University of Melbourne, Melbourne, Australia 1/36 A

version can be found in the chapter From statistical decision theory to robust optimization: a maximin perspective on robust decision-making (Sniedovich 2016).

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Warning: Mathematical content

This is a

presentation.

2/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Warning: Mathematical content

This is a

presentation. A

version can be found in the chapter From statistical decision theory to robust optimization: a maximin perspective on robust decision-making (Sniedovich 2016). 2/36 Decision making under severe uncertainty Worst-case analysis (WCA) Wald’s maximin paradigm Robust optimization (RO) Relationship between WCA, RO and Wald’s paradigm Role of constraints in RO, WCA and Wald’s paradigm Local vs global WCA Conservatism issue Summary and conclusions

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Programme

3/36 Worst-case analysis (WCA) Wald’s maximin paradigm Robust optimization (RO) Relationship between WCA, RO and Wald’s paradigm Role of constraints in RO, WCA and Wald’s paradigm Local vs global WCA Conservatism issue Summary and conclusions

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Programme

Decision making under severe uncertainty

3/36 Wald’s maximin paradigm Robust optimization (RO) Relationship between WCA, RO and Wald’s paradigm Role of constraints in RO, WCA and Wald’s paradigm Local vs global WCA Conservatism issue Summary and conclusions

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Programme

Decision making under severe uncertainty Worst-case analysis (WCA)

3/36 Robust optimization (RO) Relationship between WCA, RO and Wald’s paradigm Role of constraints in RO, WCA and Wald’s paradigm Local vs global WCA Conservatism issue Summary and conclusions

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Programme

Decision making under severe uncertainty Worst-case analysis (WCA) Wald’s maximin paradigm

3/36 Relationship between WCA, RO and Wald’s paradigm Role of constraints in RO, WCA and Wald’s paradigm Local vs global WCA Conservatism issue Summary and conclusions

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Programme

Decision making under severe uncertainty Worst-case analysis (WCA) Wald’s maximin paradigm Robust optimization (RO)

3/36 Role of constraints in RO, WCA and Wald’s paradigm Local vs global WCA Conservatism issue Summary and conclusions

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Programme

Decision making under severe uncertainty Worst-case analysis (WCA) Wald’s maximin paradigm Robust optimization (RO) Relationship between WCA, RO and Wald’s paradigm

3/36 Local vs global WCA Conservatism issue Summary and conclusions

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Programme

Decision making under severe uncertainty Worst-case analysis (WCA) Wald’s maximin paradigm Robust optimization (RO) Relationship between WCA, RO and Wald’s paradigm Role of constraints in RO, WCA and Wald’s paradigm

3/36 Conservatism issue Summary and conclusions

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Programme

3/36 Summary and conclusions

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Programme

3/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Programme

3/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Programme

No need to take notes. A copy of the presentation will be available online next week and/or by request.

4/36 Conceptual framework Decision apace: D (set of all available decisions, d ∈ D) Uncertainty space: U (set of all values of the uncertainty parameter under consideration, u ∈ U )

Objective functions: O1, O2,..., Ok.

Constraints: con1, con2,..., conm.

Task Find the best decision d ∈ D given that the ‘true’ value of the uncertainty parameter u ∈ U is unknown.

Diﬃculty The task is ill-deﬁned (except for trivial problems). Note that the decision is made before the ‘true’ value of u is revealed.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Decision-making Under Severe Uncertainty

5/36 Decision apace: D (set of all available decisions, d ∈ D) Uncertainty space: U (set of all values of the uncertainty parameter under consideration, u ∈ U )

Objective functions: O1, O2,..., Ok.

Constraints: con1, con2,..., conm.

Task Find the best decision d ∈ D given that the ‘true’ value of the uncertainty parameter u ∈ U is unknown.

Diﬃculty The task is ill-deﬁned (except for trivial problems). Note that the decision is made before the ‘true’ value of u is revealed.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Decision-making Under Severe Uncertainty

Conceptual framework

5/36 Task Find the best decision d ∈ D given that the ‘true’ value of the uncertainty parameter u ∈ U is unknown.

Diﬃculty The task is ill-deﬁned (except for trivial problems). Note that the decision is made before the ‘true’ value of u is revealed.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Decision-making Under Severe Uncertainty

Conceptual framework Decision apace: D (set of all available decisions, d ∈ D) Uncertainty space: U (set of all values of the uncertainty parameter under consideration, u ∈ U )

Objective functions: O1, O2,..., Ok.

Constraints: con1, con2,..., conm.

5/36 Find the best decision d ∈ D given that the ‘true’ value of the uncertainty parameter u ∈ U is unknown.

Diﬃculty The task is ill-deﬁned (except for trivial problems). Note that the decision is made before the ‘true’ value of u is revealed.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Decision-making Under Severe Uncertainty

Conceptual framework Decision apace: D (set of all available decisions, d ∈ D) Uncertainty space: U (set of all values of the uncertainty parameter under consideration, u ∈ U )

Objective functions: O1, O2,..., Ok.

Constraints: con1, con2,..., conm.

Task

5/36 Diﬃculty The task is ill-deﬁned (except for trivial problems). Note that the decision is made before the ‘true’ value of u is revealed.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Decision-making Under Severe Uncertainty

Conceptual framework Decision apace: D (set of all available decisions, d ∈ D) Uncertainty space: U (set of all values of the uncertainty parameter under consideration, u ∈ U )

Objective functions: O1, O2,..., Ok.

Constraints: con1, con2,..., conm.

Task Find the best decision d ∈ D given that the ‘true’ value of the uncertainty parameter u ∈ U is unknown.

5/36 The task is ill-deﬁned (except for trivial problems). Note that the decision is made before the ‘true’ value of u is revealed.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Decision-making Under Severe Uncertainty

Conceptual framework Decision apace: D (set of all available decisions, d ∈ D) Uncertainty space: U (set of all values of the uncertainty parameter under consideration, u ∈ U )

Objective functions: O1, O2,..., Ok.

Constraints: con1, con2,..., conm.

Task Find the best decision d ∈ D given that the ‘true’ value of the uncertainty parameter u ∈ U is unknown.

Diﬃculty

5/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Decision-making Under Severe Uncertainty

Conceptual framework Decision apace: D (set of all available decisions, d ∈ D) Uncertainty space: U (set of all values of the uncertainty parameter under consideration, u ∈ U )

Objective functions: O1, O2,..., Ok.

Constraints: con1, con2,..., conm.

Task Find the best decision d ∈ D given that the ‘true’ value of the uncertainty parameter u ∈ U is unknown.

Diﬃculty The task is ill-deﬁned (except for trivial problems). Note that the decision is made before the ‘true’ value of u is revealed. 5/36 Conventional Linear Programming max {cT x : Ax ≤ b, x ≥ 0} x vs ‘Uncertain’ Linear Programming max {cT (u)x : A(u)x ≤ b(u), x ≥ 0} x

If u is unknown then the ‘uncertain’ LP problem is ill-deﬁned. Diﬃculty Under uncertainty, the set of optimal solutions may depend on the assumed value of the uncertainty parameter.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Decision-making Under Severe Uncertainty

Example: linear programming problems

6/36 vs ‘Uncertain’ Linear Programming max {cT (u)x : A(u)x ≤ b(u), x ≥ 0} x

If u is unknown then the ‘uncertain’ LP problem is ill-deﬁned. Diﬃculty Under uncertainty, the set of optimal solutions may depend on the assumed value of the uncertainty parameter.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Decision-making Under Severe Uncertainty

Example: linear programming problems Conventional Linear Programming max {cT x : Ax ≤ b, x ≥ 0} x

6/36 ‘Uncertain’ Linear Programming max {cT (u)x : A(u)x ≤ b(u), x ≥ 0} x

If u is unknown then the ‘uncertain’ LP problem is ill-deﬁned. Diﬃculty Under uncertainty, the set of optimal solutions may depend on the assumed value of the uncertainty parameter.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Decision-making Under Severe Uncertainty

Example: linear programming problems Conventional Linear Programming max {cT x : Ax ≤ b, x ≥ 0} x vs

6/36 If u is unknown then the ‘uncertain’ LP problem is ill-deﬁned. Diﬃculty Under uncertainty, the set of optimal solutions may depend on the assumed value of the uncertainty parameter.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Decision-making Under Severe Uncertainty

Example: linear programming problems Conventional Linear Programming max {cT x : Ax ≤ b, x ≥ 0} x vs ‘Uncertain’ Linear Programming max {cT (u)x : A(u)x ≤ b(u), x ≥ 0} x

6/36 Diﬃculty Under uncertainty, the set of optimal solutions may depend on the assumed value of the uncertainty parameter.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Decision-making Under Severe Uncertainty

Example: linear programming problems Conventional Linear Programming max {cT x : Ax ≤ b, x ≥ 0} x vs ‘Uncertain’ Linear Programming max {cT (u)x : A(u)x ≤ b(u), x ≥ 0} x

If u is unknown then the ‘uncertain’ LP problem is ill-deﬁned.

6/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Decision-making Under Severe Uncertainty

Example: linear programming problems Conventional Linear Programming max {cT x : Ax ≤ b, x ≥ 0} x vs ‘Uncertain’ Linear Programming max {cT (u)x : A(u)x ≤ b(u), x ≥ 0} x

If u is unknown then the ‘uncertain’ LP problem is ill-deﬁned. Diﬃculty Under uncertainty, the set of optimal solutions may depend on the assumed value of the uncertainty parameter.

6/36 Issue Easy case Diﬃcult case ∩ X ∗(u) 6= {} ∩ X ∗(u) = {} u∈U u∈U Rare Common

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Decision-making Under Severe Uncertainty

More generally:

Certainty vs Uncertainty opt f (x) opt f (x, u) x∈X x∈X(u) X ∗ = arg opt f (x) X ∗(u) = arg opt f (x, u) , u ∈ U x∈X x∈X(u)

7/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Decision-making Under Severe Uncertainty

More generally:

Certainty vs Uncertainty opt f (x) opt f (x, u) x∈X x∈X(u) X ∗ = arg opt f (x) X ∗(u) = arg opt f (x, u) , u ∈ U x∈X x∈X(u)

Issue Easy case Diﬃcult case ∩ X ∗(u) 6= {} ∩ X ∗(u) = {} u∈U u∈U Rare Common

7/36 Probability theory Fuzzy set theory Uncertainty theory (Lin 2007, 2009) Worst-case analysis (WCA) ···

There are many ways to quantify uncertainty, e.g.

In this discussion we deal exclusively with WCA.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Quantiﬁcation of Uncertainty

8/36 Probability theory Fuzzy set theory Uncertainty theory (Lin 2007, 2009) Worst-case analysis (WCA) ···

In this discussion we deal exclusively with WCA.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Quantiﬁcation of Uncertainty

There are many ways to quantify uncertainty, e.g.

8/36 Fuzzy set theory Uncertainty theory (Lin 2007, 2009) Worst-case analysis (WCA) ··· In this discussion we deal exclusively with WCA.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Quantiﬁcation of Uncertainty

There are many ways to quantify uncertainty, e.g. Probability theory

8/36 Uncertainty theory (Lin 2007, 2009) Worst-case analysis (WCA) ··· In this discussion we deal exclusively with WCA.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Quantiﬁcation of Uncertainty

There are many ways to quantify uncertainty, e.g. Probability theory Fuzzy set theory

8/36 Worst-case analysis (WCA) ··· In this discussion we deal exclusively with WCA.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Quantiﬁcation of Uncertainty

There are many ways to quantify uncertainty, e.g. Probability theory Fuzzy set theory Uncertainty theory (Lin 2007, 2009)

8/36 ··· In this discussion we deal exclusively with WCA.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Quantiﬁcation of Uncertainty

There are many ways to quantify uncertainty, e.g. Probability theory Fuzzy set theory Uncertainty theory (Lin 2007, 2009) Worst-case analysis (WCA)

8/36 In this discussion we deal exclusively with WCA.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Quantiﬁcation of Uncertainty

There are many ways to quantify uncertainty, e.g. Probability theory Fuzzy set theory Uncertainty theory (Lin 2007, 2009) Worst-case analysis (WCA) ···

8/36 In this discussion we deal exclusively with WCA.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Quantiﬁcation of Uncertainty

There are many ways to quantify uncertainty, e.g. Probability theory Fuzzy set theory Uncertainty theory (Lin 2007, 2009) Worst-case analysis (WCA) ···

8/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Quantiﬁcation of Uncertainty

There are many ways to quantify uncertainty, e.g. Probability theory Fuzzy set theory Uncertainty theory (Lin 2007, 2009) Worst-case analysis (WCA) ··· In this discussion we deal exclusively with WCA.

8/36 TheSuperProgrammer

http://www.thesuperprogrammer.com/2018/02/ds-algorithm-analysis-best-worst-and.html

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

9/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

TheSuperProgrammer

http://www.thesuperprogrammer.com/2018/02/ds-algorithm-analysis-best-worst-and.html 9/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

5.4 Worst-case scenario/best-case scenario Page last updated: 2004 It can be useful to ask young people to consider ’worst-case’ and ’best-case’ scenarios in response to change. Some questions that you could use in this strategy are:

’If you were to stop using heroin, what do you imagine would be the worst things that could possibly happen?’

’If you were to keep using heroin, what do you imagine would be the best things that could possibly happen?’

These questions could be varied to include the ’worst-case’ and ’best-case’ imaginings around change and staying the same. Australian Government, Department of Health http://www.health.gov.au/internet/publications/publishing.nsf/Content/ drugtreat-pubs-front9-fa-toc~drugtreat-pubs-front9-fa-secb~drugtreat-pubs-front9-fa-secb-5~drugtreat-pubs-front9-fa-secb-5-4 10/36 Hope for the best but plan for the worst! If in doubt, assume the worst! The gods to-day stand friendly, that we may, Lovers of peace, lead on our days to age! But, since the aﬀairs of men rests still incertain, Let’s reason with the worst that may befall. William Shakespeare (1564-1616) Julius Caesar, Act 5, Scene 1

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

A well established, intuitive, approach to uncertainty and variability:

11/36 If in doubt, assume the worst! The gods to-day stand friendly, that we may, Lovers of peace, lead on our days to age! But, since the aﬀairs of men rests still incertain, Let’s reason with the worst that may befall. William Shakespeare (1564-1616) Julius Caesar, Act 5, Scene 1

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

A well established, intuitive, approach to uncertainty and variability: Hope for the best but plan for the worst!

11/36 The gods to-day stand friendly, that we may, Lovers of peace, lead on our days to age! But, since the aﬀairs of men rests still incertain, Let’s reason with the worst that may befall. William Shakespeare (1564-1616) Julius Caesar, Act 5, Scene 1

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

A well established, intuitive, approach to uncertainty and variability: Hope for the best but plan for the worst! If in doubt, assume the worst!

11/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

A well established, intuitive, approach to uncertainty and variability: Hope for the best but plan for the worst! If in doubt, assume the worst! The gods to-day stand friendly, that we may, Lovers of peace, lead on our days to age! But, since the aﬀairs of men rests still incertain, Let’s reason with the worst that may befall. William Shakespeare (1564-1616) Julius Caesar, Act 5, Scene 1

11/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis Case 1    Case 2  .  . Best case (Optimization)  Case n-1   Case n  Case 1    Case 2  .  . Worst case (Pessimization)  Case n-1   Case n  Optimization Pessimization min max max min

opt pes 12/36 1 For any given x, A(u)x ≤ b(u) is better than A(u0)x ≤/ b(u0) 2 In Optimization Theory, constraint satisfaction (feasibility) has precedence over optimality, namely optimality implies feasibility. Hence, in a worst-case analysis of LP problems, we require A(u)x ≤ b(u) , ∀u ∈ U

E.g. what is the worst value of u with respect to a system of uncertain linear constraints?

A(u)x ≤ b(u)

Observations

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

How about constraints?

13/36 1 For any given x, A(u)x ≤ b(u) is better than A(u0)x ≤/ b(u0) 2 In Optimization Theory, constraint satisfaction (feasibility) has precedence over optimality, namely optimality implies feasibility. Hence, in a worst-case analysis of LP problems, we require A(u)x ≤ b(u) , ∀u ∈ U

Observations

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

How about constraints? E.g. what is the worst value of u with respect to a system of uncertain linear constraints?

A(u)x ≤ b(u)

13/36 2 In Optimization Theory, constraint satisfaction (feasibility) has precedence over optimality, namely optimality implies feasibility. Hence, in a worst-case analysis of LP problems, we require A(u)x ≤ b(u) , ∀u ∈ U

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

How about constraints? E.g. what is the worst value of u with respect to a system of uncertain linear constraints?

A(u)x ≤ b(u)

Observations 1 For any given x, A(u)x ≤ b(u) is better than A(u0)x ≤/ b(u0)

13/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

How about constraints? E.g. what is the worst value of u with respect to a system of uncertain linear constraints?

A(u)x ≤ b(u)

Observations 1 For any given x, A(u)x ≤ b(u) is better than A(u0)x ≤/ b(u0) 2 In Optimization Theory, constraint satisfaction (feasibility) has precedence over optimality, namely optimality implies feasibility. Hence, in a worst-case analysis of LP problems, we require A(u)x ≤ b(u) , ∀u ∈ U

13/36 Constraints Certainty Worst case (wrt u) constraints(x) constraints(x, u), ∀u ∈ U

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

More generally,

14/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

More generally, Constraints Certainty Worst case (wrt u) constraints(x) constraints(x, u), ∀u ∈ U

14/36 In the context of the following uncertain LP problem, what is the worst, or a worst, value of u ∈ U with respect to a given x? z∗ = max {cT (u)x : A(u)x ≤ b(u), x ≥ 0} x

Solution

 T arg min c (u)x , A(u)x ≤ b(u), ∀u ∈ U U (x) := u∈U Inadmissible , otherwise

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

Quiz

15/36 Solution

 T arg min c (u)x , A(u)x ≤ b(u), ∀u ∈ U U (x) := u∈U Inadmissible , otherwise

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

Quiz In the context of the following uncertain LP problem, what is the worst, or a worst, value of u ∈ U with respect to a given x? z∗ = max {cT (u)x : A(u)x ≤ b(u), x ≥ 0} x

15/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

Quiz In the context of the following uncertain LP problem, what is the worst, or a worst, value of u ∈ U with respect to a given x? z∗ = max {cT (u)x : A(u)x ≤ b(u), x ≥ 0} x

Solution

 T arg min c (u)x , A(u)x ≤ b(u), ∀u ∈ U U (x) := u∈U Inadmissible , otherwise

15/36 Worst-case analysis of decision x ∈ X

z∗(x) := pes {f (x, u): constraints(x, u) , ∀u ∈ U } u∈U U (x) := arg pes {f (x, u): constraints(x, u) , ∀u ∈ U } u∈U

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

More generally, Uncertain optimization problem

opt {f (x, u): constraints on x and u } x∈X

16/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

More generally, Uncertain optimization problem

opt {f (x, u): constraints on x and u } x∈X

Worst-case analysis of decision x ∈ X

z∗(x) := pes {f (x, u): constraints(x, u) , ∀u ∈ U } u∈U U (x) := arg pes {f (x, u): constraints(x, u) , ∀u ∈ U } u∈U

16/36 Global The worst-case analysis is conducted over the entire uncertainty space U . Local The worst-case analysis is conducted over a (relatively) small neighborhood in U .

Uncertainty space, U

u˜

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

Global vs Local worst-case analysis

17/36 Local The worst-case analysis is conducted over a (relatively) small neighborhood in U . Uncertainty space, U

u˜

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

Global vs Local worst-case analysis Global The worst-case analysis is conducted over the entire uncertainty space U .

17/36 Uncertainty space, U

u˜

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

Global vs Local worst-case analysis Global The worst-case analysis is conducted over the entire uncertainty space U . Local The worst-case analysis is conducted over a (relatively) small neighborhood in U .

17/36 Uncertainty space, U

u˜

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

17/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Worst-case analysis

u˜

17/36 Abraham Wald (1902-1950)

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

18/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Abraham Wald (1902-1950) 18/36 A game between DM and Nature (uncertainty) DM plays ﬁrst Nature reacts to the decision made by DM DM is awarded an outcome whose value depends on the decisions made by DM and Nature DM strives to obtain the best outcome Nature is malevolent

How do we treat (severely) uncertain parameters in the context of decision-making processes?

Conceptual framework

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Fundamental question

19/36 A game between DM and Nature (uncertainty) DM plays ﬁrst Nature reacts to the decision made by DM DM is awarded an outcome whose value depends on the decisions made by DM and Nature DM strives to obtain the best outcome Nature is malevolent

Conceptual framework

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Fundamental question How do we treat (severely) uncertain parameters in the context of decision-making processes?

19/36 DM plays ﬁrst Nature reacts to the decision made by DM DM is awarded an outcome whose value depends on the decisions made by DM and Nature DM strives to obtain the best outcome Nature is malevolent

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Fundamental question How do we treat (severely) uncertain parameters in the context of decision-making processes?

Conceptual framework A game between DM and Nature (uncertainty)

19/36 Nature reacts to the decision made by DM DM is awarded an outcome whose value depends on the decisions made by DM and Nature DM strives to obtain the best outcome Nature is malevolent

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Fundamental question How do we treat (severely) uncertain parameters in the context of decision-making processes?

Conceptual framework A game between DM and Nature (uncertainty) DM plays ﬁrst

19/36 DM is awarded an outcome whose value depends on the decisions made by DM and Nature DM strives to obtain the best outcome Nature is malevolent

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Fundamental question How do we treat (severely) uncertain parameters in the context of decision-making processes?

Conceptual framework A game between DM and Nature (uncertainty) DM plays ﬁrst Nature reacts to the decision made by DM

19/36 DM strives to obtain the best outcome Nature is malevolent

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Fundamental question How do we treat (severely) uncertain parameters in the context of decision-making processes?

Conceptual framework A game between DM and Nature (uncertainty) DM plays ﬁrst Nature reacts to the decision made by DM DM is awarded an outcome whose value depends on the decisions made by DM and Nature

19/36 Nature is malevolent

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Fundamental question How do we treat (severely) uncertain parameters in the context of decision-making processes?

19/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Fundamental question How do we treat (severely) uncertain parameters in the context of decision-making processes?

19/36 DM is optimizing the outcome wrt the decision variable. Nature is pessimizing the outcome wrt the uncertainty parameter.

In plain language, Wald’s Maximin Decision Rule Rank decisions according to their worst-case performance. Hence, select a decision whose worst-case performance is at least as good as the worst-case performance of any other decision.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Consequences

20/36 Nature is pessimizing the outcome wrt the uncertainty parameter.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Consequences DM is optimizing the outcome wrt the decision variable.

20/36 In plain language, Wald’s Maximin Decision Rule Rank decisions according to their worst-case performance. Hence, select a decision whose worst-case performance is at least as good as the worst-case performance of any other decision.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Consequences DM is optimizing the outcome wrt the decision variable. Nature is pessimizing the outcome wrt the uncertainty parameter.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Consequences DM is optimizing the outcome wrt the decision variable. Nature is pessimizing the outcome wrt the uncertainty parameter.

20/36 Wald’s Maximin Decision Rule Rank decisions according to their worst-case performance. Hence, select a decision whose worst-case performance is at least as good as the worst-case performance of any other decision.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Consequences DM is optimizing the outcome wrt the decision variable. Nature is pessimizing the outcome wrt the uncertainty parameter.

In plain language,

20/36 Rank decisions according to their worst-case performance. Hence, select a decision whose worst-case performance is at least as good as the worst-case performance of any other decision.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Consequences DM is optimizing the outcome wrt the decision variable. Nature is pessimizing the outcome wrt the uncertainty parameter.

In plain language, Wald’s Maximin Decision Rule

20/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Consequences DM is optimizing the outcome wrt the decision variable. Nature is pessimizing the outcome wrt the uncertainty parameter.

20/36 Rawls (1971, p. 152) The maximin rule tells us to rank alternatives by their worst possible outcomes: we are to adopt the alternative the worst outcome of which is superior to the worst outcome of the others.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

There are many formulations for Wald’s Maximin Decision Rule, e.g.

21/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

There are many formulations for Wald’s Maximin Decision Rule, e.g. Rawls (1971, p. 152) The maximin rule tells us to rank alternatives by their worst possible outcomes: we are to adopt the alternative the worst outcome of which is superior to the worst outcome of the others.

21/36 z∗ := opt f (x) x∈X

Example 1: worst-case analysis of decision x z∗(x) := pes f (x, u) u∈U

Example 1: maximin counterpart problem z◦ := opt pes f (x, u) x∈X u∈U Textbook example, z◦ := max min f (x, u) x∈X u∈U

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Example 1: unconstrained optimization

22/36 Example 1: worst-case analysis of decision x z∗(x) := pes f (x, u) u∈U

Example 1: maximin counterpart problem z◦ := opt pes f (x, u) x∈X u∈U Textbook example, z◦ := max min f (x, u) x∈X u∈U

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Example 1: unconstrained optimization z∗ := opt f (x) x∈X

22/36 z∗(x) := pes f (x, u) u∈U

Example 1: maximin counterpart problem z◦ := opt pes f (x, u) x∈X u∈U Textbook example, z◦ := max min f (x, u) x∈X u∈U

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Example 1: unconstrained optimization z∗ := opt f (x) x∈X

Example 1: worst-case analysis of decision x

22/36 Example 1: maximin counterpart problem z◦ := opt pes f (x, u) x∈X u∈U Textbook example, z◦ := max min f (x, u) x∈X u∈U

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Example 1: unconstrained optimization z∗ := opt f (x) x∈X

Example 1: worst-case analysis of decision x z∗(x) := pes f (x, u) u∈U

22/36 Textbook example, z◦ := max min f (x, u) x∈X u∈U

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Example 1: unconstrained optimization z∗ := opt f (x) x∈X

Example 1: worst-case analysis of decision x z∗(x) := pes f (x, u) u∈U

Example 1: maximin counterpart problem z◦ := opt pes f (x, u) x∈X u∈U

22/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Example 1: unconstrained optimization z∗ := opt f (x) x∈X

Example 1: worst-case analysis of decision x z∗(x) := pes f (x, u) u∈U

Example 1: maximin counterpart problem z◦ := opt pes f (x, u) x∈X u∈U Textbook example, z◦ := max min f (x, u) x∈X u∈U

22/36 Example 2: worst-case analysis of decision x z∗(x) := pes {f (x, u): constraints(x, u), ∀u ∈ U } u∈U

Example 2: maximin counterpart problem z◦ := opt pes{f (x, u): constraints(x, u), ∀u ∈ U } x∈X u∈U Textbook example, z◦ := max min{f (x, u): constraints(x, u), ∀u ∈ U } x∈X u∈U

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Example 2: constrained optimization z∗ := opt {f (x): constraints on x} x∈X

23/36 z∗(x) := pes {f (x, u): constraints(x, u), ∀u ∈ U } u∈U

Example 2: maximin counterpart problem z◦ := opt pes{f (x, u): constraints(x, u), ∀u ∈ U } x∈X u∈U Textbook example, z◦ := max min{f (x, u): constraints(x, u), ∀u ∈ U } x∈X u∈U

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Example 2: constrained optimization z∗ := opt {f (x): constraints on x} x∈X

Example 2: worst-case analysis of decision x

23/36 Example 2: maximin counterpart problem z◦ := opt pes{f (x, u): constraints(x, u), ∀u ∈ U } x∈X u∈U Textbook example, z◦ := max min{f (x, u): constraints(x, u), ∀u ∈ U } x∈X u∈U

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Example 2: constrained optimization z∗ := opt {f (x): constraints on x} x∈X

Example 2: worst-case analysis of decision x z∗(x) := pes {f (x, u): constraints(x, u), ∀u ∈ U } u∈U

23/36 Textbook example, z◦ := max min{f (x, u): constraints(x, u), ∀u ∈ U } x∈X u∈U

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Example 2: constrained optimization z∗ := opt {f (x): constraints on x} x∈X

Example 2: worst-case analysis of decision x z∗(x) := pes {f (x, u): constraints(x, u), ∀u ∈ U } u∈U

Example 2: maximin counterpart problem z◦ := opt pes{f (x, u): constraints(x, u), ∀u ∈ U } x∈X u∈U

23/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Example 2: constrained optimization z∗ := opt {f (x): constraints on x} x∈X

Example 2: worst-case analysis of decision x z∗(x) := pes {f (x, u): constraints(x, u), ∀u ∈ U } u∈U

Example 2: maximin counterpart problem z◦ := opt pes{f (x, u): constraints(x, u), ∀u ∈ U } x∈X u∈U Textbook example, z◦ := max min{f (x, u): constraints(x, u), ∀u ∈ U } x∈X u∈U

23/36 Example 3: worst-case analysis of decision x z∗(x) := min {cT (u)x : A(u)x ≤ b(u), ∀u ∈ U } u∈U

Example 3: maximin counterpart problem

z◦ : = max min {cT (u)x : A(u)x ≤ b(u), ∀u ∈ U } x≥0 u∈U = max max {t : cT (u)x ≥ t, A(u)x ≤ b(u), ∀u ∈ U } x≥0 t∈R = max {t : cT (u)x ≥ t, A(u)x ≤ b(u), ∀u ∈ U } x≥0,t∈R

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm Example 3: LP problem z∗ := max {cT x : Ax ≤ b, x ≥ 0} x

24/36 z∗(x) := min {cT (u)x : A(u)x ≤ b(u), ∀u ∈ U } u∈U

Example 3: maximin counterpart problem

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm Example 3: LP problem z∗ := max {cT x : Ax ≤ b, x ≥ 0} x

Example 3: worst-case analysis of decision x

24/36 Example 3: maximin counterpart problem

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm Example 3: LP problem z∗ := max {cT x : Ax ≤ b, x ≥ 0} x

Example 3: worst-case analysis of decision x z∗(x) := min {cT (u)x : A(u)x ≤ b(u), ∀u ∈ U } u∈U

24/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm Example 3: LP problem z∗ := max {cT x : Ax ≤ b, x ≥ 0} x

Example 3: worst-case analysis of decision x z∗(x) := min {cT (u)x : A(u)x ≤ b(u), ∀u ∈ U } u∈U

Example 3: maximin counterpart problem

24/36 max {f (x): constraints(x, u), ∀u ∈ U } x∈X E.g. max {cT x : A(u) ≤ b(u), ∀u ∈ U } x≥0 That is, the maximin paradigm allows the objective function to be independent of the uncertainty parameter u.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Oﬀ the record comment Despite many comments made in the peer review literature to the contrary, the following is deﬁnitely a glut kosher Maximin model:

25/36 E.g. max {cT x : A(u) ≤ b(u), ∀u ∈ U } x≥0 That is, the maximin paradigm allows the objective function to be independent of the uncertainty parameter u.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

25/36 That is, the maximin paradigm allows the objective function to be independent of the uncertainty parameter u.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

25/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Wald’s maximin paradigm

Oﬀ the record comment Despite many comments made in the peer review literature to the contrary, the following is deﬁnitely a glut kosher Maximin model: max {f (x): constraints(x, u), ∀u ∈ U } x∈X E.g. max {cT x : A(u) ≤ b(u), ∀u ∈ U } x≥0 That is, the maximin paradigm allows the objective function to be independent of the uncertainty parameter u.

25/36 The logical progression of the “robust counterpart” approach used in Robust Optimization is as follows (e.g. Ben-Tal and Nemirovski 1998): Certainty problem min {f (x): F(x) ∈ K} x∈X where K ⊂ Rm is a convex cone. Uncertain problem for decision x max {f (x, u): F(x, u) ∈ K, ∀u ∈ U } u∈U where U ⊂ RM . Robust counterpart problem min max {f (x, u): F(x, u) ∈ K, ∀u ∈ U } x∈X u∈U

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Robust optimization

26/36 Uncertain problem for decision x max {f (x, u): F(x, u) ∈ K, ∀u ∈ U } u∈U where U ⊂ RM . Robust counterpart problem min max {f (x, u): F(x, u) ∈ K, ∀u ∈ U } x∈X u∈U

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Robust optimization

26/36 Robust counterpart problem min max {f (x, u): F(x, u) ∈ K, ∀u ∈ U } x∈X u∈U

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Robust optimization

26/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Robust optimization

The logical progression of the “robust counterpart” approach used in Robust Optimization is as follows (e.g. Ben-Tal and Nemirovski 1998): Certainty problem min {f (x): F(x) ∈ K} x∈X where K ⊂ Rm is a convex cone. Uncertain problem for decision x max {f (x, u): F(x, u) ∈ K, ∀u ∈ U } u∈U where U ⊂ RM . Robust counterpart problem min max {f (x, u): F(x, u) ∈ K, ∀u ∈ U } x∈X u∈U 26/36 The emphasis in Robust Optimization is on Mathematical Programming problems. The “robust counterpart” approach used in Robust Optimization is an application of Wald’s maximin paradigm to Mathematical Programming problems. The “robust counterpart” approach used in Robust Optimization has been used (under diﬀerent titles) in other disciplines well before the birth of Robust Optimization in the mid 1990s.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Robust optimization

Observations Robust counterpart problems are maximin or minimax problems.

27/36 The “robust counterpart” approach used in Robust Optimization is an application of Wald’s maximin paradigm to Mathematical Programming problems. The “robust counterpart” approach used in Robust Optimization has been used (under diﬀerent titles) in other disciplines well before the birth of Robust Optimization in the mid 1990s.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Robust optimization

Observations Robust counterpart problems are maximin or minimax problems. The emphasis in Robust Optimization is on Mathematical Programming problems.

27/36 The “robust counterpart” approach used in Robust Optimization has been used (under diﬀerent titles) in other disciplines well before the birth of Robust Optimization in the mid 1990s.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Robust optimization

27/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Robust optimization

Observations Robust counterpart problems are maximin or minimax problems. The emphasis in Robust Optimization is on Mathematical Programming problems. The “robust counterpart” approach used in Robust Optimization is an application of Wald’s maximin paradigm to Mathematical Programming problems. The “robust counterpart” approach used in Robust Optimization has been used (under diﬀerent titles) in other disciplines well before the birth of Robust Optimization in the mid 1990s.

27/36 to the best of our knowledge, the only previous example related to this question is due to Soyster [16] (see below).The issue of hard uncertain constraints, however, is not a novelty for Control Theory, where it is a well-studied subject forming the area of Robust Control (see, e.g., [17] and references therein).

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Robust optimization

28/36 The issue of hard uncertain constraints, however, is not a novelty for Control Theory, where it is a well-studied subject forming the area of Robust Control (see, e.g., [17] and references therein).

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Robust optimization

28/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Robust optimization

A maximin perspective In the article Robust solutions of uncertain linear programs (Ben-Tal and Nemirovski, 1999, p. 2) we read ( color added): Dealing with uncertain hard constraints is perhaps a novelty in Mathematical Programming; to the best of our knowledge, the only previous example related to this question is due to Soyster [16] (see below).The issue of hard uncertain constraints, however, is not a novelty for Control Theory, where it is a well-studied subject forming the area of Robust Control (see, e.g., [17] and references therein).

28/36 Our life experience is that worst-case outcomes are often associated with rare (extreme) events whose likelihood of occurrence is very (extremely?) small. Practically speaking then, Wald’s maximin paradigm advocates a paranoiac approach to uncertainty. What then is the justiﬁcation for the use of Wald’s maximin paradigm, Robust counterpart approach, and other decision theories based on a worst-case approach to uncertainty?

Diﬃculty There is no guarantee that a decision based on Wald’s maximin paradigm performs well (relative to other decisions) under “normal” scenarios, rather then the worst-case scenario.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Conservatism

29/36 Practically speaking then, Wald’s maximin paradigm advocates a paranoiac approach to uncertainty. What then is the justiﬁcation for the use of Wald’s maximin paradigm, Robust counterpart approach, and other decision theories based on a worst-case approach to uncertainty?

Diﬃculty There is no guarantee that a decision based on Wald’s maximin paradigm performs well (relative to other decisions) under “normal” scenarios, rather then the worst-case scenario.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Conservatism

Our life experience is that worst-case outcomes are often associated with rare (extreme) events whose likelihood of occurrence is very (extremely?) small.

29/36 What then is the justiﬁcation for the use of Wald’s maximin paradigm, Robust counterpart approach, and other decision theories based on a worst-case approach to uncertainty?

Diﬃculty There is no guarantee that a decision based on Wald’s maximin paradigm performs well (relative to other decisions) under “normal” scenarios, rather then the worst-case scenario.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Conservatism

29/36 Diﬃculty There is no guarantee that a decision based on Wald’s maximin paradigm performs well (relative to other decisions) under “normal” scenarios, rather then the worst-case scenario.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Conservatism

Our life experience is that worst-case outcomes are often associated with rare (extreme) events whose likelihood of occurrence is very (extremely?) small. Practically speaking then, Wald’s maximin paradigm advocates a paranoiac approach to uncertainty. What then is the justiﬁcation for the use of Wald’s maximin paradigm, Robust counterpart approach, and other decision theories based on a worst-case approach to uncertainty?

29/36 Diﬃculty There is no guarantee that a decision based on Wald’s maximin paradigm performs well (relative to other decisions) under “normal” scenarios, rather then the worst-case scenario.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Conservatism

29/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Conservatism

Diﬃculty There is no guarantee that a decision based on Wald’s maximin paradigm performs well (relative to other decisions) under “normal” scenarios, rather then the worst-case scenario.

29/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Conservatism

Abraham Wald John von Neumann (1902-1950) (1903-1957) Statistical Decision Theory Two-Player Zero-sum Games

30/36 A tale of two Maximin paradigms Wald von Neumann Player 1 plays ﬁrst Both players play simultaneously Player 2 is Nature Player 2 is an Adversary Stability is not required Stability is required

Observation To justify the worst-case approach adopted by Wald, it is necessary to assume that Nature is an adversary that reacts to given decisions made by Player 1.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Conservatism

Conceptually, on the agenda is a game involving two players, call them Player 1 and Player 2.

31/36 Player 2 is Nature Player 2 is an Adversary Stability is not required Stability is required

Observation To justify the worst-case approach adopted by Wald, it is necessary to assume that Nature is an adversary that reacts to given decisions made by Player 1.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Conservatism

Conceptually, on the agenda is a game involving two players, call them Player 1 and Player 2.

A tale of two Maximin paradigms Wald von Neumann Player 1 plays ﬁrst Both players play simultaneously

31/36 Stability is not required Stability is required

Observation To justify the worst-case approach adopted by Wald, it is necessary to assume that Nature is an adversary that reacts to given decisions made by Player 1.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Conservatism

Conceptually, on the agenda is a game involving two players, call them Player 1 and Player 2.

A tale of two Maximin paradigms Wald von Neumann Player 1 plays ﬁrst Both players play simultaneously Player 2 is Nature Player 2 is an Adversary

31/36 Observation To justify the worst-case approach adopted by Wald, it is necessary to assume that Nature is an adversary that reacts to given decisions made by Player 1.

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Conservatism

Conceptually, on the agenda is a game involving two players, call them Player 1 and Player 2.

A tale of two Maximin paradigms Wald von Neumann Player 1 plays ﬁrst Both players play simultaneously Player 2 is Nature Player 2 is an Adversary Stability is not required Stability is required

31/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Conservatism

Conceptually, on the agenda is a game involving two players, call them Player 1 and Player 2.

A tale of two Maximin paradigms Wald von Neumann Player 1 plays ﬁrst Both players play simultaneously Player 2 is Nature Player 2 is an Adversary Stability is not required Stability is required

Observation To justify the worst-case approach adopted by Wald, it is necessary to assume that Nature is an adversary that reacts to given decisions made by Player 1.

31/36 (Wald’s) Maximin solution

$ u1 u2 u3 u4 ··· u1000 min max d1 12222 2 11 d2 5000 5000 5000 5000 50001 − 1 −

Optimal solution: d1. Does it make sense?!

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Conservatism

Example What is the maximin solution to the ‘uncertain’ problem deﬁned by the following decision table?

$ u1 u2 u3 u4 ··· u1000 d1 1 2 2 2 2 2 d2 5000 5000 5000 5000 5000 1 −

32/36 Optimal solution: d1. Does it make sense?!

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Conservatism

Example What is the maximin solution to the ‘uncertain’ problem deﬁned by the following decision table?

$ u1 u2 u3 u4 ··· u1000 d1 1 2 2 2 2 2 d2 5000 5000 5000 5000 5000 1 −

(Wald’s) Maximin solution

$ u1 u2 u3 u4 ··· u1000 min max d1 12222 2 11 d2 5000 5000 5000 5000 50001 − 1 −

32/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Conservatism

Example What is the maximin solution to the ‘uncertain’ problem deﬁned by the following decision table?

$ u1 u2 u3 u4 ··· u1000 d1 1 2 2 2 2 2 d2 5000 5000 5000 5000 5000 1 −

(Wald’s) Maximin solution

$ u1 u2 u3 u4 ··· u1000 min max d1 12222 2 11 d2 5000 5000 5000 5000 50001 − 1 −

Optimal solution: d1. Does it make sense?! 32/36 Wald’s maximin paradigm is based on an ad hoc approach to uncertainty/variability. It is a valid approach in cases where a worst-case approach to uncertainty is valid. Its use must therefore be justiﬁed on a case-by-case basis. The “robust counterpart” method of Robust Optimization is a (rather late) application of Wald’s paradigm. It is time to the reinvent the ﬁeld of Maximin Programming z∗ := max min {f (x, u): constraints(x, u), ∀u ∈ U } x∈X u∈U Viva le/la Maximin!

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Summary and conclusions

33/36 It is a valid approach in cases where a worst-case approach to uncertainty is valid. Its use must therefore be justiﬁed on a case-by-case basis. The “robust counterpart” method of Robust Optimization is a (rather late) application of Wald’s paradigm. It is time to the reinvent the ﬁeld of Maximin Programming z∗ := max min {f (x, u): constraints(x, u), ∀u ∈ U } x∈X u∈U Viva le/la Maximin!

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Summary and conclusions

Wald’s maximin paradigm is based on an ad hoc approach to uncertainty/variability.

33/36 Its use must therefore be justiﬁed on a case-by-case basis. The “robust counterpart” method of Robust Optimization is a (rather late) application of Wald’s paradigm. It is time to the reinvent the ﬁeld of Maximin Programming z∗ := max min {f (x, u): constraints(x, u), ∀u ∈ U } x∈X u∈U Viva le/la Maximin!

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Summary and conclusions

Wald’s maximin paradigm is based on an ad hoc approach to uncertainty/variability. It is a valid approach in cases where a worst-case approach to uncertainty is valid.

33/36 The “robust counterpart” method of Robust Optimization is a (rather late) application of Wald’s paradigm. It is time to the reinvent the ﬁeld of Maximin Programming z∗ := max min {f (x, u): constraints(x, u), ∀u ∈ U } x∈X u∈U Viva le/la Maximin!

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Summary and conclusions

33/36 It is time to the reinvent the ﬁeld of Maximin Programming z∗ := max min {f (x, u): constraints(x, u), ∀u ∈ U } x∈X u∈U Viva le/la Maximin!

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Summary and conclusions

33/36 Viva le/la Maximin!

Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Summary and conclusions

Wald’s maximin paradigm is based on an ad hoc approach to uncertainty/variability. It is a valid approach in cases where a worst-case approach to uncertainty is valid. Its use must therefore be justiﬁed on a case-by-case basis. The “robust counterpart” method of Robust Optimization is a (rather late) application of Wald’s paradigm. It is time to the reinvent the ﬁeld of Maximin Programming z∗ := max min {f (x, u): constraints(x, u), ∀u ∈ U } x∈X u∈U

33/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Summary and conclusions

33/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Bibliography

Ben-Tal, A., and Nemirovski, A. (1997) Robust truss topology design via semideﬁnite programming. SIAM Journal of Optimization, 7(4):991-1016. Ben-Tal, A., and Nemirovski, A. (1998). Robust convex optimization. Mathematics of Operations research, 23(4):769-805. Ben-Tal, A., and Nemirovski, A. (1999). Robust solutions of uncertain linear programs. Operations Research Letters, 25:1-13. Ben-Tal, A., El Ghaoui, L. and Nemirovski, A. (2009). Robust Optimization. Princeton University Press.

34/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Bibliography

Rawls, J. (1971). Theory of Justice. Belknap Press, Cambridge, MA. Sniedovich, M. (2012a) Fooled by local robustness. Risk Analysis, 32(10):1630-1637. Sniedovich, M. (2016). From statistical decision theory to robust optimization: a maximin perspective on robust decision-making. In Doumpos, M., Zopounidis, C., and Grigoroudis, E. (eds.) Robustness Analysis in Decision Aiding, Optimization, and Analytics, pp. 59-87. Springer, New York. Wald, A. (1939). Contributions to the theory of statistical estimation and testing hypotheses. Annals of Mathematical Statistics, 10(4):299-326.

35/36 Introduction DMUSU Uncertainty WCA Wald RO Conservatism Summary

Bibliography

Wald, A. (1945). Statistical decision functions which minimize the maximum risk. The Annals of Mathematics, 46(2):265-280. Wald, A. (1950). Statistical Decision Functions. John Wiley, NY. von Neumann, J. (1928). Zur theories der gesellschaftsspiele, Mathematische Annalen,100:295-320. von Neumann, J., Morgenstern, O. (1944). Theory of Games and Economic Behavior. Princeton University Press, Princeton. Quotes from the third edition, 1953; sixth printing, 1955.

36/36