Two-Stage Stochastic Optimization for the Strategic Bidding of a Generation Company Considering Wind Power Uncertainty
Total Page：16
File Type：pdf, Size：1020Kb
Load more
Recommended publications
-
Stochastic Optimization from Distributed, Streaming Data in Rate-Limited Networks Matthew Nokleby, Member, IEEE, and Waheed U
1 Stochastic Optimization from Distributed, Streaming Data in Rate-limited Networks Matthew Nokleby, Member, IEEE, and Waheed U. Bajwa, Senior Member, IEEE T Abstract—Motivated by machine learning applications in T training samples fξ(t) 2 Υgt=1 drawn independently from networks of sensors, internet-of-things (IoT) devices, and au- the distribution P (ξ), which is supported on a subset of Υ. tonomous agents, we propose techniques for distributed stochastic There are, in particular, two main categorizations of training convex learning from high-rate data streams. The setup involves a network of nodes—each one of which has a stream of data data that, in turn, determine the types of methods that can be arriving at a constant rate—that solve a stochastic convex used to find approximate solutions to the SO problem. These optimization problem by collaborating with each other over rate- are (i) batch training data and (ii) streaming training data. limited communication links. To this end, we present and analyze In the case of batch training data, where all T samples two algorithms—termed distributed stochastic approximation fξ(t)g are pre-stored and simultaneously available, a common mirror descent (D-SAMD) and accelerated distributed stochastic approximation mirror descent (AD-SAMD)—that are based on strategy is sample average approximation (SAA) (also referred two stochastic variants of mirror descent and in which nodes to as empirical risk minimization (ERM)), in which one collaborate via approximate averaging of the local, noisy subgra- minimizes the empirical average of the “risk” function φ(·; ·) dients using distributed consensus. Our main contributions are in lieu of the true expectation. -
Metaheuristics1
METAHEURISTICS1 Kenneth Sörensen University of Antwerp, Belgium Fred Glover University of Colorado and OptTek Systems, Inc., USA 1 Definition A metaheuristic is a high-level problem-independent algorithmic framework that provides a set of guidelines or strategies to develop heuristic optimization algorithms (Sörensen and Glover, To appear). Notable examples of metaheuristics include genetic/evolutionary algorithms, tabu search, simulated annealing, and ant colony optimization, although many more exist. A problem-specific implementation of a heuristic optimization algorithm according to the guidelines expressed in a metaheuristic framework is also referred to as a metaheuristic. The term was coined by Glover (1986) and combines the Greek prefix meta- (metá, beyond in the sense of high-level) with heuristic (from the Greek heuriskein or euriskein, to search). Metaheuristic algorithms, i.e., optimization methods designed according to the strategies laid out in a metaheuristic framework, are — as the name suggests — always heuristic in nature. This fact distinguishes them from exact methods, that do come with a proof that the optimal solution will be found in a finite (although often prohibitively large) amount of time. Metaheuristics are therefore developed specifically to find a solution that is “good enough” in a computing time that is “small enough”. As a result, they are not subject to combinatorial explosion – the phenomenon where the computing time required to find the optimal solution of NP- hard problems increases as an exponential function of the problem size. Metaheuristics have been demonstrated by the scientific community to be a viable, and often superior, alternative to more traditional (exact) methods of mixed- integer optimization such as branch and bound and dynamic programming. -
MIE 1612: Stochastic Programming and Robust Optimization
MIE 1612: Stochastic Programming and Robust Optimization Fall 2019 Syllabus Instructor: Prof. Merve Bodur Office: BA8106 Office hour: Wednesday 2-3pm (or by appointment) E-mail: [email protected] P.S. Please include course code (MIE 1612) in your email subject TA: Maryam Daryalal Office hour: Monday 2:30-3:30pm, in BA8119 (or by appointment) E-mail: [email protected] P.S. Please include course code (MIE 1612) in your email subject Lectures: Monday 4-6pm (GB119) and Wednesday 1-2pm (GB303) Suggested texts: Lectures on Stochastic Programming { Modeling and Theory, SIAM, Shapiro, Dentcheva, and Ruszczy´nski,2009. Introduction to Stochastic Programming, Springer-Verlag, Birge and Louveaux, 2011. Stochastic Programming, Wiley, Kall and Wallace, 1994. Stochastic Programming, Handbooks in OR and MS, Elsevier, Ruszczy´nskiand Shapiro, 2003. Robust Optimization, Princeton University Press, Ben-Tal, El Ghaoui, and Nemirovski, 2009. Course web pages: Quercus, Piazza Official course description: Stochastic programming and robust optimization are optimization tools deal- ing with a class of models and algorithms in which data is affected by uncertainty, i.e., some of the input data are not perfectly known at the time the decisions are made. Topics include modeling uncertainty in optimization problems, two-stage and multistage stochastic programs with recourse, chance constrained pro- grams, computational solution methods, approximation and sampling methods, and applications. Knowledge of linear programming, probability and statistics are required, while programming ability and knowledge of integer programming are helpful. Prerequisite: MIE262, APS1005 or equivalent, and MIE231, APS106S or equivalent. 1 Overview The aim of stochastic programming and robust optimization is to find optimal decisions in problems which involve uncertain data. -
Extending Algebraic Modelling Languages for Stochastic Programming
Extending algebraic modelling languages for Stochastic Programming Christian Valente, Gautam Mitra, Mustapha Sadki CARISMA, Brunel University and OptiRisk Systems Uxbridge, Middlesex, UK Robert Fourer Northwestern University, Evanston, IL Abstract Algebraic modelling languages (AML) have gained wide acceptance and use in Mathematical Programming by researchers and practitioners. At a basic level, stochastic programming models can be defined using these languages by constructing their deterministic equivalent. Unfortunately, this leads to very large model data instances. We propose a direct approach in which the random values of the model coefficients and the stage structure of the decision variables and constraints are "overlaid" on the underlying deterministic (core) model of the SP problems. This leads not only to a natural definition of the SP model, the resulting generated instance is also a compact representation of the otherwise large problem data. The proposed constructs enable the formulation of two stage and multistage scenario based recourse problems. The design is presented as a stochastic extension of the AMPL language which we call SAMPL; this in turn is embedded in an environment called SPInE (Stochastic Programming Integrated Environment) which facilitates modelling and investigation of SP problems. Table of Contents 1 Introduction.............................................................................................................2 1.1 Background .................................................................................................................2 -
Stochastic Optimization
Stochastic Optimization Marc Schoenauer DataScience @ LHC Workshop 2015 Content 1 Setting the scene 2 Continuous Optimization 3 Discrete/Combinatorial Optimization 4 Non-Parametric Representations 5 Conclusion Content 1 Setting the scene 2 Continuous Optimization 3 Discrete/Combinatorial Optimization 4 Non-Parametric Representations 5 Conclusion Stochastic Optimization Hypotheses Search Space Ω with some topological structure Objective function F assume some weak regularity Hill-Climbing Randomly draw x0 2 Ω and compute F(x0) Initialisation Until(happy) y = Best neighbor(xt) neighbor structure on Ω Compute F(y) If F(y) F (xt) then xt+1 = y accept if improvement else xt+1 = xt Comments Find closest local optimum defined by neighborhood structure Iterate from different x0's Until(very happy) Stochastic Optimization Hypotheses Search Space Ω with some topological structure Objective function F assume some weak regularity Stochastic (Local) Search Randomly draw x0 2 Ω and compute F(x0) Initialisation Until(happy) y = Random neighbor(xt) neighbor structure on Ω Compute F(y) If F(y) F (xt) then xt+1 = y accept if improvement else xt+1 = xt Comments Find one close local optimum defined by neighborhood structure Iterate, leaving current optimum IteratedLocalSearch Stochastic Optimization Hypotheses Search Space Ω with some topological structure Objective function F assume some weak regularity Stochastic (Local) Search { alternative viewpoint Randomly draw x0 2 Ω and compute F(x0) Initialisation Until(happy) y = Move(xt) stochastic variation on -
Optimization of Conditional Value-At-Risk
Implemented in Portfolio Safeguard by AORDA.com Optimization of conditional value-at-risk R. Tyrrell Rockafellar Department of Applied Mathematics, University of Washington, 408 L Guggenheim Hall, Box 352420, Seattle, Washington 98195-2420, USA Stanislav Uryasev Department of Industrial and Systems Engineering, University of Florida, PO Box 116595, 303 Weil Hall, Gainesville, Florida 32611-6595, USA A new approach to optimizing or hedging a portfolio of ®nancial instruments to reduce risk is presented and tested on applications. It focuses on minimizing conditional value-at-risk CVaR) rather than minimizing value-at-risk VaR),but portfolios with low CVaR necessarily have low VaR as well. CVaR,also called mean excess loss,mean shortfall,or tail VaR,is in any case considered to be a more consistent measure of risk than VaR. Central to the new approach is a technique for portfolio optimization which calculates VaR and optimizes CVaR simultaneously. This technique is suitable for use by investment companies,brokerage ®rms,mutual funds, and any business that evaluates risk. It can be combined with analytical or scenario- based methods to optimize portfolios with large numbers of instruments,in which case the calculations often come down to linear programming or nonsmooth programming. The methodology can also be applied to the optimization of percentiles in contexts outside of ®nance. 1. INTRODUCTION This paper introduces a new approach to optimizing a portfolio so as to reduce the risk of high losses. Value-at-risk VaR) has a role in the approach, but the emphasis is on conditional value-at-risk CVaR), which is also known as mean excess loss, mean shortfall, or tail VaR. -
Adaptive Stochastic Optimization
Industrial and Systems Engineering Adaptive Stochastic Optimization Frank E. Curtis1 and Katya Scheinberg2 1Department of Industrial and Systems Engineering, Lehigh University 2Department of Operations Research and Information Engineering, Cornell University ISE Technical Report 20T-001 Adaptive Stochastic Optimization The purpose of this article is to summarize re- Frank E. Curtis Katya Scheinberg cent work and motivate continued research on ◦ the design and analysis of adaptive stochastic op- January 18, 2020 timization methods. In particular, we present an analytical framework—new for the context Abstract. Optimization lies at the heart of of adaptive deterministic optimization—that sets machine learning and signal processing. Contem- the stage for establishing convergence rate guar- porary approaches based on the stochastic gradi- antees for adaptive stochastic optimization tech- ent method are non-adaptive in the sense that niques. With this framework in hand, we remark their implementation employs prescribed param- on important open questions related to how it eter values that need to be tuned for each appli- can be extended further for the design of new cation. This article summarizes recent research methods. We also discuss challenges and oppor- and motivates future work on adaptive stochastic tunities for their use in real-world systems. optimization methods, which have the potential to offer significant computational savings when 2 Background training large-scale systems. Many problems in ML and SP are formulated as optimization problems. For example, given a 1 Introduction data vector y Rm from an unknown distribu- tion, one often∈ desires to have a vector of model The successes of stochastic optimization algo- parameters x Rn such that a composite objec- ∈ rithms for solving problems arising machine tive function f : Rn R is minimized, as in learning (ML) and signal processing (SP) are → now widely recognized. -
Mean-Risk Objectives in Stochastic Programming
Shabbir Ahmed Mean-risk objectives in stochastic programming April 20, 2004 Abstract. Traditional stochastic programming is risk neutral in the sense that it is concerned with the optimization of an expectation criterion. A common approach to addressing risk in decision making problems is to consider a weighted mean-risk objective, where some dispersion statistic is used as a measure of risk. We investigate the computational suitability of various mean-risk objective functions in addressing risk in stochastic programming models. We prove that the classical mean-variance criterion leads to computational intractability even in the simplest stochastic programs. On the other hand, a number of alternative mean-risk functions are shown to be computationally tractable using slight variants of existing stochastic program- ming decomposition algorithms. We propose a parametric cutting plane algorithm to generate the entire mean-risk efficient frontier for a particular mean-risk objective. Key words. Stochastic programming, mean-risk objectives, computational complexity, cut- ting plane algorithms. 1. Introduction This paper is concerned with stochastic programming problems of the form minf E[f(x; !)] : x 2 Xg; (1) where x 2 Rn is a vector of decision variables; X ⊂ Rn is a non-empty set of feasible decisions; (Ω; F; P ) is a probability space with elements !; and f : Rn × Ω 7! R is a cost function such that f(·; !) is convex for all ! 2 Ω, and f(x; ·) is F-measurable and P -integrable for all x 2 Rn. Of particular interest are instances of (1) corresponding to two-stage stochastic linear programs (cf. [1, 13]), where X is a polyhedron and f(x; !) = cT x + Q(x; ξ(!)) (2) with Q(x; ξ) = minfqT y : W y = h + T x; y ≥ 0g: (3) Here ξ = (q; W; h; T ) represents a particular realization of the random data ξ(!) for the linear program in (3). -
Conservative Stochastic Optimization with Expectation Constraints
1 Conservative Stochastic Optimization with Expectation Constraints Zeeshan Akhtar,? Amrit Singh Bedi,y and Ketan Rajawat? Abstract This paper considers stochastic convex optimization problems where the objective and constraint functions involve expectations with respect to the data indices or environmental variables, in addition to deterministic convex constraints on the domain of the variables. Although the setting is generic and arises in different machine learning applications, online and efficient approaches for solving such problems have not been widely studied. Since the underlying data distribution is unknown a priori, a closed-form solution is generally not available, and classical deterministic optimization paradigms are not applicable. Existing approaches towards solving these problems make use of stochastic gradients of the objective and constraints that arrive sequentially over iterations. State-of-the-art approaches, such as those using the saddle point framework, are able to ensure that the optimality gap as well as the constraint violation − 1 decay as O T 2 where T is the number of stochastic gradients. The domain constraints are assumed simple and handled via projection at every iteration. In this work, we propose a novel conservative − 1 stochastic optimization algorithm (CSOA) that achieves zero average constraint violation and O T 2 optimality gap. Further, we also consider the scenario where carrying out a projection step onto the convex domain constraints at every iteration is not viable. Traditionally, the projection operation can be avoided by considering the conditional gradient or Frank-Wolfe (FW) variant of the algorithm. The state-of-the-art − 1 stochastic FW variants achieve an optimality gap of O T 3 after T iterations, though these algorithms have not been applied to problems with functional expectation constraints. -
Monte Carlo Methods for Sampling-Based Stochastic Optimization
Monte Carlo methods for sampling-based Stochastic Optimization Monte Carlo methods for sampling-based Stochastic Optimization Gersende FORT LTCI CNRS & Telecom ParisTech Paris, France Joint works with B. Jourdain, T. Leli`evre,G. Stoltz from ENPC and E. Kuhn from INRA. A. Schreck and E. Moulines from Telecom ParisTech P. Priouret from Paris VI Monte Carlo methods for sampling-based Stochastic Optimization Introduction Simulated Annealing Simulated Annealing (1/2) Let U denote the objective function one wants to minimize. U(x) min U(x) () max exp(−U(x)) () max exp − 8T > 0 x2X x2X x2X T In order to sample from πT? where U(x) π (x) = exp − T T sample successively from a sequence of tempered distributions πT1 , πT2 , ··· with T1 > T2 > ··· > T?. Monte Carlo methods for sampling-based Stochastic Optimization Introduction Simulated Annealing Simulated Annealing (1/2) Let U denote the objective function one wants to minimize. U(x) min U(x) () max exp(−U(x)) () max exp − 8T > 0 x2X x2X x2X T In order to sample from πT? where U(x) π (x) = exp − T T sample successively from a sequence of tempered distributions πT1 , πT2 , ··· with T1 > T2 > ··· > T?. or sample successively from a sequence of (nt-iterated) kernels (PTt (x; ·))t such that πT PT = πT . Monte Carlo methods for sampling-based Stochastic Optimization Introduction Simulated Annealing Simulated Annealing (2/2) Under conditions on X, on the cooling schedule (Tt)t, on the kernels (Pt)t, on the dominating measure and the set of minima, ··· Kirkpatrick, Gelatt and Vecchi. Optimization via Simulated Annealing. Science (1983) Geman and Geman. -
Stochastic Optimization,” in Handbook of Computational Statistics: Concepts and Methods (2Nd Ed.) (J
Citation: Spall, J. C. (2012), “Stochastic Optimization,” in Handbook of Computational Statistics: Concepts and Methods (2nd ed.) (J. Gentle, W. Härdle, and Y. Mori, eds.), Springer−Verlag, Heidelberg, Chapter 7, pp. 173–201. dx.doi.org/10.1007/978-3-642-21551-3_7 STOCHASTIC OPTIMIZATION James C. Spall The Johns Hopkins University Applied Physics Laboratory 11100 Johns Hopkins Road Laurel, Maryland 20723-6099 U.S.A. [email protected] Stochastic optimization algorithms have been growing rapidly in popularity over the last decade or two, with a number of methods now becoming “industry standard” approaches for solving challenging optimization problems. This chapter provides a synopsis of some of the critical issues associated with stochastic optimization and a gives a summary of several popular algorithms. Much more complete discussions are available in the indicated references. To help constrain the scope of this article, we restrict our attention to methods using only measurements of the criterion (loss function). Hence, we do not cover the many stochastic methods using information such as gradients of the loss function. Section 1 discusses some general issues in stochastic optimization. Section 2 discusses random search methods, which are simple and surprisingly powerful in many applications. Section 3 discusses stochastic approximation, which is a foundational approach in stochastic optimization. Section 4 discusses a popular method that is based on connections to natural evolution—genetic algorithms. Finally, Section 5 offers some concluding remarks. 1 Introduction 1.1 General Background Stochastic optimization plays a significant role in the analysis, design, and operation of modern systems. Methods for stochastic optimization provide a means of coping with inherent system noise and coping with models or systems that are highly nonlinear, high dimensional, or otherwise inappropriate for classical deterministic methods of optimization. -
Contextual Covariance Matrix Adaptation Evolutionary Strategies
Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17) Contextual Covariance Matrix Adaptation Evolutionary Strategies Abbas Abdolmaleki1;2;3, Bob Price5, Nuno Lau1, Luis Paulo Reis2;3, Gerhard Neumann4∗ 1: IEETA, DETI, University of Aveiro 2: DSI, University of Minho 3: LIACC, University of Porto, Portugal 4: CLAS, TU Darmstadt, Germany 5: PARC, A Xerox Company, USA Abstract the ball to its team mates which are positioned on differ- ent locations on the field. Here, the soccer robot should Many stochastic search algorithms are designed to learn to kick the ball to any given target location, which is optimize a fixed objective function to learn a task, specified by the context vector, on the field. In such cases, i.e., if the objective function changes slightly, for learning for every possible context is clearly inefficient or example, due to a change in the situation or con- even infeasible. Therefore our goal is to generalize learned text of the task, relearning is required to adapt to tasks from similar contexts to a new context. To do so, we the new context. For instance, if we want to learn learn a context-dependent policy for a continuous range of a kicking movement for a soccer robot, we have to contexts without restarting the learning process. In this pa- relearn the movement for different ball locations. per, we consider stochastic search algorithms for contextual Such relearning is undesired as it is highly inef- learning. Stochastic search algorithms [Hansen et al., 2003; ficient and many applications require a fast adap- Sun et al., 2009; Ruckstieߨ et al., 2008] optimize an objec- tation to a new context/situation.