Lecture #8: Monte Carlo method
ENGG304: Uncertainty, Reliability and Risk
Edoardo Patelli Institute for Risk and Uncertainty
E: [email protected] W: www.liv.ac.uk/risk-and-uncertainty T: +44 01517944079 A MEMBER OF THE RUSSELL GROUP
Edoardo Patelli University of Liverpool 18 March 2019 1 / 81 Lecture Outline 1 Introduction 2 Monte Carlo method Random Number Generator Sampling Methods Buffon’s experiment Monte Carlo integration Probability of failure 3 Summary 4 Computer based class Some useful slides Assignments
Edoardo Patelli University of Liverpool 18 March 2019 2 / 81 Introduction Programme ENGG304
1 28/01/2019 Introduction 2 08/02/2019 Human error 3 11/02/2019 Qualitative risk assessment: Safety analysis 4 18/02/2019 Qualitative risk assessment: Event Tree and Fault Tree 5 25/02/2019 Tutorial I 6 04/03/2019 Model of random phenomena 7 11/03/2019 Structural reliability 8 18/03/2019 Monte Carlo simulation I + Hands-on session 9 25/03/2019 Tutorial II
10 01/04/2019 Monte Carlo simulation II + Tutorial III (Hands-on session)
Edoardo Patelli University of Liverpool 18 March 2019 3 / 81 Introduction Summary Lecture #7 Safety Margin Fundamental problem Performance function defined as “Capacity” - “Demand” Safety margin: M = C − D = g(x) 2 For normal and independent random variables: M ∼ N(µM , σM ) q 2 2 µM = µC − µD σM = σC + σD
Reliability index β
β = µM /σM represents the number of standard deviations by which the mean value of the safety margin M exceeds zero
Edoardo Patelli University of Liverpool 18 March 2019 4 / 81 Introduction Summary Lecture #7 Safety Margin Fundamental problem Performance function defined as “Capacity” - “Demand” Safety margin: M = C − D = g(x) 2 For normal and independent random variables: M ∼ N(µM , σM ) q 2 2 µM = µC − µD σM = σC + σD
Reliability index β
β = µM /σM represents the number of standard deviations by which the mean value of the safety margin M exceeds zero
Edoardo Patelli University of Liverpool 18 March 2019 4 / 81 Introduction Summary Lecture #7 Reliability index Geometrical Interpretation/1
Edoardo Patelli University of Liverpool 18 March 2019 5 / 81 Monte Carlo method Introduction Lecture Outline 1 Introduction 2 Monte Carlo method Random Number Generator Sampling Methods Buffon’s experiment Monte Carlo integration Probability of failure 3 Summary 4 Computer based class Some useful slides Assignments
Edoardo Patelli University of Liverpool 18 March 2019 6 / 81 Monte Carlo method Introduction
What is the Monte Carlo method?
Edoardo Patelli University of Liverpool 18 March 2019 7 / 81 Monte Carlo method Introduction Monte Carlo method
Computation technique based on random numbers Numerical experiment by generating a random sequence of number with prescribed probability distribution. Collecting quantity of interest
Edoardo Patelli University of Liverpool 18 March 2019 8 / 81 Monte Carlo method Introduction Monte Carlo method How did Monte Carlo simulation get its name? Name derived from the Principality of Monaco
Edoardo Patelli University of Liverpool 18 March 2019 9 / 81 Monte Carlo method Introduction Monte Carlo method How did Monte Carlo simulation get its name? Name derived from the Principality of Monaco
Edoardo Patelli University of Liverpool 18 March 2019 9 / 81 Monte Carlo method Introduction Monte Carlo method Origins 1777 Comte de Buffon - earliest documented use of random sampling
Edoardo Patelli University of Liverpool 18 March 2019 10 / 81 Monte Carlo method Introduction Monte Carlo method Origins
1777 Comte de Buffon - earliest documented use of random sampling 2 ∗ L P(needle intersects the grid) = πt
Edoardo Patelli University of Liverpool 18 March 2019 10 / 81 Monte Carlo method Introduction Monte Carlo method Origins
1777 Comte de Buffon - earliest documented use of random sampling 2 ∗ L P(needle intersects the grid) = πt
Edoardo Patelli University of Liverpool 18 March 2019 10 / 81 Monte Carlo method Introduction Monte Carlo method Origins
1777 Comte de Buffon - earliest documented use of random sampling 2 ∗ L P(needle intersects the grid) = πt
Edoardo Patelli University of Liverpool 18 March 2019 10 / 81 Monte Carlo method Introduction Monte Carlo method Origins 1777 Comte de Buffon - earliest documented use of random sampling 2 ∗ L P(needle intersects the grid) = πt 1786 Laplace suggested to estimate π by random sampling
Edoardo Patelli University of Liverpool 18 March 2019 10 / 81 Monte Carlo method Introduction Monte Carlo method Origins 1777 Comte de Buffon - earliest documented use of random sampling 2 ∗ L P(needle intersects the grid) = πt 1786 Laplace suggested to estimate π by random sampling http://mste.illinois.edu/reese/ buffon/bufjava.html Edoardo Patelli University of Liverpool 18 March 2019 10 / 81 Monte Carlo method Introduction Monte Carlo method Modern time 1930s, Enrico Fermi first experimented with the Monte Carlo Fermiac tools (transport of neutrons)
Edoardo Patelli University of Liverpool 18 March 2019 11 / 81 Monte Carlo method Introduction Monte Carlo method Manhattan Project 1942-1946 Calculation of the explosive yield of atomic bomb Stanislaw Ulam suggested to used ENIAC for “method of statistic trials” Metropolis suggested the name “Monte Carlo” Von Neumann developed the first computer code (1947)
Edoardo Patelli University of Liverpool 18 March 2019 12 / 81 Monte Carlo method Introduction
Is it really used?
Edoardo Patelli University of Liverpool 18 March 2019 13 / 81 Monte Carlo method Examples Monte Carlo applications
Solution of integrals, differential equation, complex systems etc... Simulation random events Cryptography, Decision-Making Games
Edoardo Patelli University of Liverpool 18 March 2019 14 / 81 Monte Carlo method Examples Monte Carlo applications
Solution of integrals, differential equation, complex systems etc... Simulation random events Cryptography, Decision-Making Games
Edoardo Patelli University of Liverpool 18 March 2019 14 / 81 Monte Carlo method Examples Monte Carlo applications
Solution of integrals, differential equation, complex systems etc... Simulation random events Cryptography, Decision-Making Games
Edoardo Patelli University of Liverpool 18 March 2019 14 / 81 Monte Carlo method Examples Monte Carlo applications
Solution of integrals, differential equation, complex systems etc... Simulation random events Cryptography, Decision-Making Games
Edoardo Patelli University of Liverpool 18 March 2019 14 / 81 Monte Carlo method Main components Monte Carlo method Main components
Random number generator Probability distribution functions (describing the model) Sampling rules (how to sample from PDFs) Error estimator Variance reduction technique Computational resources
Edoardo Patelli University of Liverpool 18 March 2019 15 / 81 Monte Carlo method Random Number Generator Lecture Outline 1 Introduction 2 Monte Carlo method Random Number Generator Sampling Methods Buffon’s experiment Monte Carlo integration Probability of failure 3 Summary 4 Computer based class Some useful slides Assignments
Edoardo Patelli University of Liverpool 18 March 2019 16 / 81 Monte Carlo method Random Number Generator Random Number Generator Definition A tool or method able to generate an (automatic) sequence (of number) in which each term is unpredictable
Edoardo Patelli University of Liverpool 18 March 2019 17 / 81 Monte Carlo method Random Number Generator Type of Random Number Generators Definitions True random Each element has equal probability of being chosen from the set Pseudo-random A finite set that display qualities of random numbers
Edoardo Patelli University of Liverpool 18 March 2019 18 / 81 Monte Carlo method Random Number Generator Type of Random Number Generators True Random Number Generator vs Pseudo Random Number Generator
Pseudo-Random True Random Number Number Generators Generators Characteristic Excellent efficiency Poor efficiency Deterministic Non-deterministic Periodic Aperiodic Applications
Edoardo Patelli University of Liverpool 18 March 2019 19 / 81 Monte Carlo method Random Number Generator Type of Random Number Generators True Random Number Generator vs Pseudo Random Number Generator
Pseudo-Random True Random Number Number Generators Generators Characteristic Excellent efficiency Poor efficiency Deterministic Non-deterministic Periodic Aperiodic Applications Simulation and Mod- elling, Video Games, Security
Edoardo Patelli University of Liverpool 18 March 2019 19 / 81 Monte Carlo method Random Number Generator Type of Random Number Generators True Random Number Generator vs Pseudo Random Number Generator
Pseudo-Random True Random Number Number Generators Generators Characteristic Excellent efficiency Poor efficiency Deterministic Non-deterministic Periodic Aperiodic Applications Simulation and Mod- Lottery and Draws, elling, Video Games, Random Sampling, Security Games and Gambling
Edoardo Patelli University of Liverpool 18 March 2019 19 / 81 Monte Carlo method Random Number Generator Pseudo-Random Number Generator
Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin (John Von Neumann, 1951)
Edoardo Patelli University of Liverpool 18 March 2019 20 / 81 Monte Carlo method Random Number Generator Pseudo-Random Number Generator Characteristics Approximate the PDF Statistically independent Should pass a battery of statistical tests Be aperiodic Difficult to prove whether a sequence of numbers is random
Edoardo Patelli University of Liverpool 18 March 2019 21 / 81 Monte Carlo method Random Number Generator Pseudo-Random Number Generator Characteristics Approximate the PDF Statistically independent Should pass a battery of statistical tests Be aperiodic Difficult to prove whether a sequence of numbers is random
Edoardo Patelli University of Liverpool 18 March 2019 21 / 81 Monte Carlo method Random Number Generator Pseudo-Random Number Generator Visual test Left: Linear Congruential Generator Right: Mersenne twister
Edoardo Patelli University of Liverpool 18 March 2019 22 / 81 Monte Carlo method Random Number Generator Pseudo-Random Number Generator Linear Congruential Generator
Rn+1 ≡ (aRn + c) (mod m) where: m, 0 < m (modulo) a, 0 < a < m (multiplier) c, 0 ≤ c < m (increment)
R0, 0 ≤ R0 < m (seed / start value) The modulo operation (mod ) finds the remainder after division of one number (aRn + c) by another (m)
Edoardo Patelli University of Liverpool 18 March 2019 23 / 81 Monte Carlo method Random Number Generator Pseudo-Random Number Generator Linear Congruential Generator (Example)
Rn+1 ≡ (5Rn + 5) (mod 7) where:
a = 5; c = 5, m = 7, R0 = 5 < m Sequence: { 5, Period < m
Edoardo Patelli University of Liverpool 18 March 2019 24 / 81 Monte Carlo method Random Number Generator Pseudo-Random Number Generator Linear Congruential Generator (Example)
Rn+1 ≡ (5Rn + 5) (mod 7) where:
a = 5; c = 5, m = 7, R0 = 5 < m Sequence: { 5, 2, Period < m
Edoardo Patelli University of Liverpool 18 March 2019 24 / 81 Monte Carlo method Random Number Generator Pseudo-Random Number Generator Linear Congruential Generator (Example)
Rn+1 ≡ (5Rn + 5) (mod 7) where:
a = 5; c = 5, m = 7, R0 = 5 < m Sequence: { 5, 2, 1, Period < m
Edoardo Patelli University of Liverpool 18 March 2019 24 / 81 Monte Carlo method Random Number Generator Pseudo-Random Number Generator Linear Congruential Generator (Example)
Rn+1 ≡ (5Rn + 5) (mod 7) where:
a = 5; c = 5, m = 7, R0 = 5 < m Sequence: { 5, 2, 1, 3, Period < m
Edoardo Patelli University of Liverpool 18 March 2019 24 / 81 Monte Carlo method Random Number Generator Pseudo-Random Number Generator Linear Congruential Generator (Example)
Rn+1 ≡ (5Rn + 5) (mod 7) where:
a = 5; c = 5, m = 7, R0 = 5 < m Sequence: { 5, 2, 1, 3, 6, Period < m
Edoardo Patelli University of Liverpool 18 March 2019 24 / 81 Monte Carlo method Random Number Generator Pseudo-Random Number Generator Linear Congruential Generator (Example)
Rn+1 ≡ (5Rn + 5) (mod 7) where:
a = 5; c = 5, m = 7, R0 = 5 < m Sequence: { 5, 2, 1, 3, 6, 0, Period < m
Edoardo Patelli University of Liverpool 18 March 2019 24 / 81 Monte Carlo method Random Number Generator Pseudo-Random Number Generator Linear Congruential Generator (Example)
Rn+1 ≡ (5Rn + 5) (mod 7) where:
a = 5; c = 5, m = 7, R0 = 5 < m Sequence: { 5, 2, 1, 3, 6, 0, 5,... } Period < m Tool to sample numbers from a uniform distribution u ∼ U(0, m)
Edoardo Patelli University of Liverpool 18 March 2019 24 / 81 Monte Carlo method Random Number Generator Random Number in Matlab
R = rand(m,n); m-by-n matrix uniformly distributed R = randn(m,n); m-by-n matrix normally distributed R = exprnd(mu,m,n); m-by-n matrix exponentially distributed with mean parameter mu
Edoardo Patelli University of Liverpool 18 March 2019 25 / 81 Monte Carlo method Random Number Generator Random Number in Matlab
R = rand(m,n); m-by-n matrix uniformly distributed R = randn(m,n); m-by-n matrix normally distributed R = exprnd(mu,m,n); m-by-n matrix exponentially distributed with mean parameter mu
Edoardo Patelli University of Liverpool 18 March 2019 25 / 81 Monte Carlo method Sampling Methods Lecture Outline 1 Introduction 2 Monte Carlo method Random Number Generator Sampling Methods Buffon’s experiment Monte Carlo integration Probability of failure 3 Summary 4 Computer based class Some useful slides Assignments
Edoardo Patelli University of Liverpool 18 March 2019 26 / 81 Monte Carlo method Sampling Methods
How can we sample from a given distribution? All the random number generators produce samples from U [0, 1)
Edoardo Patelli University of Liverpool 18 March 2019 27 / 81 Monte Carlo method Sampling Methods Random variables PDM/PDF f (x): probability of a possible outcome of a random experiment CDF F(x): Probability that a possible outcomes will be found at a value less than or equal to x
Edoardo Patelli University of Liverpool 18 March 2019 28 / 81 Monte Carlo method Sampling Methods Inverse transform method Allows to sample from any cumulative distribution function (CDF). Require inversion of the CDF
Sampling procedure: 1 generate u ∼ U(0, 1) 2 identify F(xˆ) = u
−1 xˆ ← FX (u ∼ U(0, 1))
Edoardo Patelli University of Liverpool 18 March 2019 29 / 81 Monte Carlo method Sampling Methods Inverse transform method: Uniform distribution x realisation of a random variable X; u ∼ U(0, 1); x−a 1 FX (x) = b−a; f (x) = b−a; for a ≤ x ≤ b −1 x = FX (u) = a + (b − a)u
Edoardo Patelli University of Liverpool 18 March 2019 30 / 81 Monte Carlo method Sampling Methods Inverse transform method: Exponential dist. x realisation of a random variable X; u ∼ U(0, 1); −λx −λt FX (x) = 1 − e ; fX (x) = λe ; for 0 ≤ x ≤ ∞ −1 1 x = FX (u) = −λlog(1 − u)
Edoardo Patelli University of Liverpool 18 March 2019 31 / 81 Monte Carlo method Sampling Methods Inverse transform method: Discrete distribution X is a random variable with discrete values xk , k = 0, 1,... with probability masses: fk = Pr{X = xk } ≥ 0, k = 0, 1,... Pk FX (x) = i=0 fi = Fk−1 + fk , k = 0, 1,...
Edoardo Patelli University of Liverpool 18 March 2019 32 / 81 Tips: Ordering the sequence of x : xk+1 ≤ xk
Monte Carlo method Sampling Methods Inverse transform method: Discrete distribution
fk = Pr{X = xk } ≥ 0, k = 0, 1,... and Pk FX (x) = i=0 fi = Fk−1 + fk , k = 0, 1,... Sampling algorithm: 1 generate u ∼ U[0, 1]
2 find min(k)|Fk > u 3 X = xk
Edoardo Patelli University of Liverpool 18 March 2019 33 / 81 Tips: Ordering the sequence of x : xk+1 ≤ xk
Monte Carlo method Sampling Methods Inverse transform method: Discrete distribution
fk = Pr{X = xk } ≥ 0, k = 0, 1,... and Pk FX (x) = i=0 fi = Fk−1 + fk , k = 0, 1,... Sampling algorithm: 1 generate u ∼ U[0, 1]
2 find min(k)|Fk > u 3 X = xk
Edoardo Patelli University of Liverpool 18 March 2019 33 / 81 Monte Carlo method Sampling Methods Inverse transform method: Discrete distribution
fk = Pr{X = xk } ≥ 0, k = 0, 1,... and Pk FX (x) = i=0 fi = Fk−1 + fk , k = 0, 1,... Sampling algorithm: 1 generate u ∼ U[0, 1]
2 find min(k)|Fk > u 3 X = xk
Tips: Ordering the sequence of x : xk+1 ≤ xk
Edoardo Patelli University of Liverpool 18 March 2019 33 / 81 Monte Carlo method Sampling Methods Rejection Sampling Allows to sample from any PDF
Bounding the f (x) with a function g(x) (e.g. constant)
Procedure:
1 generate u1 ∼ g(x) 2 generate u2 ∼ U(0, g(u1)) 3 if u2 > f (u1) Reject 4 if u2 ≤ f (u1) Accept
Edoardo Patelli University of Liverpool 18 March 2019 34 / 81 Monte Carlo method Sampling Methods Rejection Sampling Allows to sample from any PDF Bounding the f (x) with a function g(x) (e.g. constant)
Procedure:
1 generate u1 ∼ g(x) 2 generate u2 ∼ U(0, g(u1)) 3 if u2 > f (u1) Reject
4 if u2 ≤ f (u1) Accept
Edoardo Patelli University of Liverpool 18 March 2019 34 / 81 Monte Carlo method Sampling Methods Rejection Sampling Allows to sample from any PDF Bounding the f (x) with a function g(x) (e.g. constant)
Procedure:
1 generate u1 ∼ g(x) 2 generate u2 ∼ U(0, g(u1)) 3 if u2 > f (u1) Reject
4 if u2 ≤ f (u1) Accept
Edoardo Patelli University of Liverpool 18 March 2019 34 / 81 Monte Carlo method Sampling Methods Rejection Sampling Allows to sample from any PDF Bounding the f (x) with a function g(x) (e.g. constant)
Procedure:
1 generate u1 ∼ g(x) 2 generate u2 ∼ U(0, g(u1)) 3 if u2 > f (u1) Reject
4 if u2 ≤ f (u1) Accept
Edoardo Patelli University of Liverpool 18 March 2019 34 / 81 Monte Carlo method Sampling Methods Rejection Sampling Allows to sample from any PDF Bounding the f (x) with a function g(x) (e.g. constant)
Procedure:
1 generate u1 ∼ g(x) 2 generate u2 ∼ U(0, g(u1)) 3 if u2 > f (u1) Reject
4 if u2 ≤ f (u1) Accept
Edoardo Patelli University of Liverpool 18 March 2019 34 / 81 Monte Carlo method Sampling Methods Sampling from a normal distribution Based on Central Limit Theorem For large enough samples, the sum of Identical Independent Distributed (IID) samples is close to the normal distribution Sum of IID samples U(0, 1) P 1 mean: µS = n µXn = 2n variance: σ2 = P σ2 = 1 n. S n Xn 12 2 If n = 12: µS = 6 and σS = 1.
Edoardo Patelli University of Liverpool 18 March 2019 35 / 81 Monte Carlo method Sampling Methods Sampling form a normal distribution Algorithm
1 Sample 12 values uj ∼ U[0, 1], n = 1, ··· , 12
2 P12 zi = n=1 un − 6
Then zi approximate a standard normal distribution. 2 P µz = 12/2 − 6; σz = n 1/12 = 1
Edoardo Patelli University of Liverpool 18 March 2019 36 / 81 Monte Carlo method Buffon’s experiment Lecture Outline 1 Introduction 2 Monte Carlo method Random Number Generator Sampling Methods Buffon’s experiment Monte Carlo integration Probability of failure 3 Summary 4 Computer based class Some useful slides Assignments
Edoardo Patelli University of Liverpool 18 March 2019 37 / 81 Monte Carlo method Buffon’s experiment Buffon’s experiment example Strategy Sample random position of N needles Check if the needle intersect the parallel line
Counts how many needles (Ns) intersect the lines Estimate the probability of intersecting the line as N Pˆ = s N
Edoardo Patelli University of Liverpool 18 March 2019 38 / 81 Monte Carlo method Buffon’s experiment Buffon’s experiment example Algorithm
1 Using 2 random numbers u1, u2: u1 ∼ U[0, 1) and u2 ∼ U[0, 1)
2 Calculate d = u1 ∗ t 3 Calculate φ = u2 ∗ π/2 4 if d ≤ Lcos(φ) the needle intercepts a line (update counter Ns = Ns + 1) 5 Repeat procedure N times 6 Estimate probability intersection ˆ Ns P = N
Edoardo Patelli University of Liverpool 18 March 2019 39 / 81 Monte Carlo method Buffon’s experiment Buffon’s experiment example Algorithm
1 Using 2 random numbers u1, u2: u1 ∼ U[0, 1) and u2 ∼ U[0, 1)
2 Calculate d = u1 ∗ t 3 Calculate φ = u2 ∗ π/2 4 if d ≤ Lcos(φ) the needle intercepts a line (update counter Ns = Ns + 1) 5 Repeat procedure N times 6 Estimate probability intersection ˆ Ns P = N
Edoardo Patelli University of Liverpool 18 March 2019 39 / 81 Monte Carlo method Buffon’s experiment Buffon’s experiment example Algorithm
1 Using 2 random numbers u1, u2: u1 ∼ U[0, 1) and u2 ∼ U[0, 1)
2 Calculate d = u1 ∗ t 3 Calculate φ = u2 ∗ π/2 4 if d ≤ Lcos(φ) the needle intercepts a line (update counter Ns = Ns + 1) 5 Repeat procedure N times 6 Estimate probability intersection ˆ Ns P = N
Edoardo Patelli University of Liverpool 18 March 2019 39 / 81 Monte Carlo method Buffon’s experiment Buffon’s experiment example Algorithm
1 Using 2 random numbers u1, u2: u1 ∼ U[0, 1) and u2 ∼ U[0, 1)
2 Calculate d = u1 ∗ t 3 Calculate φ = u2 ∗ π/2 4 if d ≤ Lcos(φ) the needle intercepts a line (update counter Ns = Ns + 1) 5 Repeat procedure N times 6 Estimate probability intersection ˆ Ns P = N
Edoardo Patelli University of Liverpool 18 March 2019 39 / 81 Monte Carlo method Buffon’s experiment Buffon’s experiment example Algorithm
1 Using 2 random numbers u1, u2: u1 ∼ U[0, 1) and u2 ∼ U[0, 1)
2 Calculate d = u1 ∗ t 3 Calculate φ = u2 ∗ π/2 4 if d ≤ Lcos(φ) the needle intercepts a line (update counter Ns = Ns + 1) 5 Repeat procedure N times 6 Estimate probability intersection ˆ Ns P = N
Edoardo Patelli University of Liverpool 18 March 2019 39 / 81 Monte Carlo method Buffon’s experiment Buffon’s experiment example Algorithm
1 Using 2 random numbers u1, u2: u1 ∼ U[0, 1) and u2 ∼ U[0, 1)
2 Calculate d = u1 ∗ t 3 Calculate φ = u2 ∗ π/2 4 if d ≤ Lcos(φ) the needle intercepts a line (update counter Ns = Ns + 1) 5 Repeat procedure N times 6 Estimate probability intersection ˆ Ns P = N
Edoardo Patelli University of Liverpool 18 March 2019 39 / 81 Monte Carlo method Buffon’s experiment Buffon’s experiment example Matlab code % percent makes the rest of the line a comment throws = 10000; t=1; L=1; % Sample 2 vectors of 10000 pseudorandom in [0,1) U=rand(2,throws); % compute d and phi (vectors of 10000 random numbers) d=U(1,:)*t; phi=0.5*pi*U(2,:); % hits is a vector of zeros where false and ones where true hits= d<=0.5*cos(phi); % Compute the estimator Prob=sum(hits)/ throws; % Estimate pi piEst=2*L / (t*Prob)
Edoardo Patelli University of Liverpool 18 March 2019 40 / 81 Monte Carlo method Monte Carlo integration Lecture Outline 1 Introduction 2 Monte Carlo method Random Number Generator Sampling Methods Buffon’s experiment Monte Carlo integration Probability of failure 3 Summary 4 Computer based class Some useful slides Assignments
Edoardo Patelli University of Liverpool 18 March 2019 41 / 81 Monte Carlo method Monte Carlo integration Evaluation of Definite Integrals
Z G = g(x)f (x)dx
x can be seen as a random variable f (x) has characteristic of a probability density function g(x) is also a random variable E[g(x)] = R g(x)f (x)dx = G Var[g(x)] = E[g2(x)] − G2
Edoardo Patelli University of Liverpool 18 March 2019 42 / 81 Monte Carlo method Monte Carlo integration Evaluation of Definite Integrals
Z G = g(x)f (x)dx
x can be seen as a random variable f (x) has characteristic of a probability density function g(x) is also a random variable E[g(x)] = R g(x)f (x)dx = G Var[g(x)] = E[g2(x)] − G2
Edoardo Patelli University of Liverpool 18 March 2019 42 / 81 Monte Carlo method Monte Carlo integration Monte Carlo darts method Z G = g(x)f (x)dx
1 Generate sample N points (xi) from f (x)
2 Evaluate function g(xi) (prize) 3 Computed expected prize
Edoardo Patelli University of Liverpool 18 March 2019 43 / 81 Monte Carlo method Monte Carlo integration Advantages of Monte Carlo method Definite Integrals Z Z Z G = g(x)dx = ··· g(x1,..., xn)dx1 ... dxn F F Easy to be implemented Applicable to any integral (or system of equations) Provide the exact solution for N → ∞ Simulation often gives better physical visibility than a set of equations Analytical models usually involve simplifying assumptions to make the model analytically tractable
Edoardo Patelli University of Liverpool 18 March 2019 44 / 81 Monte Carlo method Estimation of π Estimation area of a circle ZZ 2 2 2 C = f (x1, x2)dx1dx2 = π ∗ r ; F : x1 + x2 ≤ r F The integral can be rewritten as: ZZ C = H(x1, x2) · f (x1, x2)dx1dx2
( 2 2 1 if x ∈ C; x1 + x2 ≤ r H(x1, x2) = 0 otherwise We can use the dart game to estimate the area of the circle (f (x1, x2) = uniform distribution )
Edoardo Patelli University of Liverpool 18 March 2019 45 / 81 Monte Carlo method Estimation of π Estimation of π Strategy
Area Circle: C = π ∗ r 2 Area of the square: S = (2 ∗ r)2 Ratio of the areas: π ∗ r 2 π = 4 ∗ r 2 4
Edoardo Patelli University of Liverpool 18 March 2019 46 / 81 Monte Carlo method Estimation of π Estimation of π Variance of the estimator
N total samples n of N will fall in C (reds) N − n will not fall in C (blues) C ≈ n/N
n is random variable that follows a binomial distribution with p = C/S
Edoardo Patelli University of Liverpool 18 March 2019 47 / 81 Monte Carlo method Estimation of π Estimation of π Variance of the estimator (cont.)
n ' Np = N C and Cˆ = n S S N
Statistical error of Cˆ (Coefficient of Variation): r σ σ pNp(1 − p) 1 N − n c = n = ' √ Cˆ n Np N n Estimation of C with precision of %1 requires N ≈ 3 × 104 samples
Edoardo Patelli University of Liverpool 18 March 2019 48 / 81 Monte Carlo method Estimation of π Estimation of π Variance of the estimator (cont.)
n ' Np = N C and Cˆ = n S S N
Statistical error of Cˆ (Coefficient of Variation): r σ σ pNp(1 − p) 1 N − n c = n = ' √ Cˆ n Np N n Estimation of C with precision of %1 requires N ≈ 3 × 104 samples
Edoardo Patelli University of Liverpool 18 March 2019 48 / 81 Monte Carlo method Probability of failure Lecture Outline 1 Introduction 2 Monte Carlo method Random Number Generator Sampling Methods Buffon’s experiment Monte Carlo integration Probability of failure 3 Summary 4 Computer based class Some useful slides Assignments
Edoardo Patelli University of Liverpool 18 March 2019 49 / 81 Monte Carlo method Probability of failure Definite Integrals Failure criteria
Failure: exceedance of the yield stress g(θ) = σmax(θ) − σ(θ) θ: structural and load uncertainty vector
Failure (F): σ(θ) ≥ σmax (θ) Safe (S): σ(θ) < σmax (θ)
Edoardo Patelli University of Liverpool 18 March 2019 50 / 81 Monte Carlo method Probability of failure Failure quantification
Bi-dimensional case
Z Z fX(x) dx = IF (x) fX(x) dx F where: 0 ⇐⇒ X ∈ S (X) = IF 1 ⇐⇒ X ∈ F
Edoardo Patelli University of Liverpool 18 March 2019 51 / 81 Monte Carlo method Probability of failure Failure quantification Monte Carlo simulation Evaluation (”dart” game): f (x)dx probability to hit a point IF (x) the prize Estimate: direct Monte Carlo simulation
N Z 1 X P = (x) f (x) dx ≈ (X(k)) f IF X N IF k=1
to meet specified accuracy: N ∝ 1 Pf
Edoardo Patelli University of Liverpool 18 March 2019 52 / 81 Monte Carlo method Probability of failure Failure quantification Monte Carlo simulation Evaluation (”dart” game): f (x)dx probability to hit a point IF (x) the prize Estimate: direct Monte Carlo simulation
N Z 1 X P = (x) f (x) dx ≈ (X(k)) f IF X N IF k=1
to meet specified accuracy: N ∝ 1 Pf
Edoardo Patelli University of Liverpool 18 March 2019 52 / 81 Summary Lecture #8 Lecture Outline 1 Introduction 2 Monte Carlo method Random Number Generator Sampling Methods Buffon’s experiment Monte Carlo integration Probability of failure 3 Summary 4 Computer based class Some useful slides Assignments
Edoardo Patelli University of Liverpool 18 March 2019 53 / 81 Summary Lecture #8 Monte Carlo method Powerful tool for solving deterministic and stochastic problems Extremely useful for solving problems that are no approachable by normal means Cover a variety of different fields and applications.
Edoardo Patelli University of Liverpool 18 March 2019 54 / 81 Summary Lecture #8 Monte Carlo method Main components
Random number generator Probability distribution functions Sampling rules (how to sample from PDFs) Error estimator Variance reduction technique Computational tools
Edoardo Patelli University of Liverpool 18 March 2019 55 / 81 Summary What next? Lecture #9: Tutorial II 25/03/2019
Tutorial class Review and consolidate concepts. Solve exercises supervised by a tutor.
Random Variables, moments PDF/CDF Structural reliability FOSM method
Edoardo Patelli University of Liverpool 18 March 2019 56 / 81 Computer based class Some useful slides Lecture Outline 1 Introduction 2 Monte Carlo method Random Number Generator Sampling Methods Buffon’s experiment Monte Carlo integration Probability of failure 3 Summary 4 Computer based class Some useful slides Assignments
Edoardo Patelli University of Liverpool 18 March 2019 57 / 81 Computer based class Some useful slides Lecture Outline 1 Introduction 2 Monte Carlo method Random Number Generator Sampling Methods Buffon’s experiment Monte Carlo integration Probability of failure 3 Summary 4 Computer based class Some useful slides Assignments
Edoardo Patelli University of Liverpool 18 March 2019 58 / 81 Computer based class Some useful slides Lecture #8: Computer class 18/03/2019 Interactive Matlab session Room: ENG-HHTC Sample from distributions Compute values of π (compute definite integral) Reliability analysis of a simple supported beam Computing waiting time at a bus stop
Edoardo Patelli University of Liverpool 18 March 2019 59 / 81 Computer based class Some useful slides Monte Carlo method
Tool for solving deterministic and stochastic problems Extremely useful for solving problems that are no approachable by normal means Cover a variety of different fields and applications. Main components Probability distribution functions (describing the model) Random number generator Sampling rules (how to sample from PDFs)
Edoardo Patelli University of Liverpool 18 March 2019 60 / 81 Computer based class Some useful slides Useful resources
Available on Vital Learning resources → Lectures → Lecture #8 (Sampling rules) Learning resources → Part 3 Monte Carlo
Edoardo Patelli University of Liverpool 18 March 2019 61 / 81 Computer based class Some useful slides MATLAB program window
Ribbon Bar Current folder tree view Command line and call output display Workspace variables
Edoardo Patelli University of Liverpool 18 March 2019 62 / 81 Computer based class Some useful slides MATLAB command window
Enter commands and run programs interactively Call operations Assign variables
Edoardo Patelli University of Liverpool 18 March 2019 63 / 81 Computer based class Some useful slides MATLAB variables
Numerical: any number type Cell: matrix of “anything” Logical: only true or false created with {} Character and strings: text accessed in the same way written between ’’ as matrix String concatenation with Structure: save any data in fields []: [’This is a’ ’ string’] access fields with . Convert number to string: myStruct.field1 = 1.2; num2str myStruct.field2 = ’a’
Edoardo Patelli University of Liverpool 18 March 2019 64 / 81 Computer based class Some useful slides MATLAB Editor
View and edit MATLAB program files (scripts and functions) Provide code analysis and debugging Scripts Collection of MATLAB statements Executing a script is equivalent to input commands in CLI All the variables created in the script are added to the workspace Functions Programs that accept input arguments and return outputs Called with the necessary inputs from CLI Only the function outputs are added to the workspace
Edoardo Patelli University of Liverpool 18 March 2019 65 / 81 Computer based class Some useful slides Script example
Edoardo Patelli University of Liverpool 18 March 2019 66 / 81 Computer based class Some useful slides Function example
Edoardo Patelli University of Liverpool 18 March 2019 67 / 81 Computer based class Some useful slides Matrices MATLAB means MATrix LABoratory: everything is a matrix! Matrix creation: A = [1,2,3,4; 5 6 7 8; 9 10 11 12] , separates elements on the same row, and concatenate horizontally matrices with the same number of rows ; terminate the current row and start a new one, concatenate vertically matrices with the same number of columns Matrix access: () A(1,2) access number in first row, second column A(:,1:2) return sub-matrix of all the original matrix rows, and first and second column A([1 3 ,:)] return sub-matrix of all first and third rows, and all columns Some matrix initialization functions: ones(m,n) create a matrix of m rows and n columns full of ones zeros(m,n) create a matrix of m rows and n columns full of zeros
Edoardo Patelli University of Liverpool 18 March 2019 68 / 81 Computer based class Some useful slides Matlab operators and controls +, − Addition and subtraction. A + B adds A and B. A − B subtracts B from A. .∗ Array multiplication. A. ∗ B element-by-element product of the arrays A and B. .ˆ Array power. A.ˆ B matrix with elements A(i, j) to the B(i, j) power. ./ Array division. A./B matrix with elements A(i,j)/B(i,j) * Matrix multiplication. A*B linear algebraic product of A and B. ˆ Matrix power. / Matrix right division. \ Matrix left division. X = A\B solution to the equation AX = B.
Edoardo Patelli University of Liverpool 18 March 2019 69 / 81 Computer based class Some useful slides Matlab operators and controls Relational operators: <, <=, >, >=, ==, ˜= Return matrix of logical Example M = (A > B), M(i, j) = true if A(i, j) > B(i, j) Logical operators: &, | (matrix operation), &&, || (short-circuit scalar operation), ˜ Short circuit: if the beginning of an expression already allows to give a final answer, return the answer without executing all the commands Example: (b ~= 0) && (a/b > 2)
Edoardo Patelli University of Liverpool 18 March 2019 70 / 81 Computer based class Some useful slides Matlab controls Conditional statements
Conditional statements: control program flow by selecting which blocks of code to execute based on the value of test expressions if scalar-logical-expression statements end
Edoardo Patelli University of Liverpool 18 March 2019 71 / 81 Computer based class Some useful slides Matlab controls Loops control
for loop for index = start value:increment:end value statements end loop concludes where all the possible values of index have been evaluated
Edoardo Patelli University of Liverpool 18 March 2019 72 / 81 Computer based class Some useful slides Matlab controls Loops control
while loop while scalar-logical-expression statements end loop concludes when the logical expression become false loop can be always stopped with break statement
Edoardo Patelli University of Liverpool 18 March 2019 73 / 81 Computer based class Assignments Lecture Outline 1 Introduction 2 Monte Carlo method Random Number Generator Sampling Methods Buffon’s experiment Monte Carlo integration Probability of failure 3 Summary 4 Computer based class Some useful slides Assignments
Edoardo Patelli University of Liverpool 18 March 2019 74 / 81 Computer based class Sampling from a distribution Assignment #1: Sampling Sampling from a normal distribution Write a code to generate samples X from Gaussian Distribution ∼ N (4, 2) using only random numbers from Uniform distribution. Compare with the samples generated using Matlab build-in function
Edoardo Patelli University of Liverpool 18 March 2019 75 / 81 Computer based class Sampling from a distribution Assignment #1: Sampling Useful tips
1 Sample 12 values ui ∼ U[0, 1], i = 1, ··· , 12 P 2 zj = ui − 6 (then zi approximate a standard normal distribution)
3 Transform the samples zj (see slide 44 Lecture 4). X − µ Z = ∼ N (0, 1) σ 4 Matlab build-in functions rand ∼ U[0, 1] 5 Matlab build-in functions randn ∼ N [0, 1]
Edoardo Patelli University of Liverpool 18 March 2019 76 / 81 Computer based class Monte Carlo integration Assignment #2: Monte Carlo integration Estimation of π Write a code to estimate the value of π Useful tips Area Circle: π ∗ r 2 Area of the square: 2 ∗ r 2 π ∗ r 2 π Ratio of the areas: = 4 ∗ r 2 4 Tool: dart game
Edoardo Patelli University of Liverpool 18 March 2019 77 / 81 Computer based class Monte Carlo integration Assignment #3: Monte Carlo integration Evaluation of Definite Integral Evaluate via Monte Carlo the integral
Z 1 π sin(π/2x)1 2 G = cos x dx = = = 0.6366 0 2 π/2 0 π Suggestion: use dart method π Setting f (x) = 1 for x ∈ [0, 1] and g(x) = cos 2 x Sample Xi and collect the prize g(Xi ) 1 PN Average prize GN = N i=1 g(xi )
Edoardo Patelli University of Liverpool 18 March 2019 78 / 81 Computer based class Reliability analysis Assignment #4: Cantilever Beam Reliability Analysis Compute probability exceedance maximum displacement
F, B, H random variables E, ρ constants F
L, wmax parameters Displacement:
ρgBHL4 FL3 BH3 H w = + ; I = L 8EI 3EI 12 B Failure: w > wmax Edoardo Patelli University of Liverpool 18 March 2019 79 / 81 Computer based class Reliability analysis Assignment #4: Cantilever Beam
Parameter Value Generate realisations of the ρ 600 input parameters using the inverse transform method E 109 B ∼ U(0.10, 0.12) Evaluate the model (i.e. H ∼ U(0.20, 0.30) compute w) F ∼ EXP(1/4000) Count number of times the L 2.5 beam exceeds wmax wmax 0.005 Estimate the probability of failure
Edoardo Patelli University of Liverpool 18 March 2019 80 / 81 Computer based class Time dependent problem Assignment #5: Bus waiting time A passenger arrives at a bus-stop at some arbitrary point in time. Tip: Sample time between buses from uniform distribution The number of bus arrivals in the time interval [0, t] follows the Poisson distribution with mean λt. Tip: Sample time between buses from exponential distribution The mean interval between the buses is 10 minutes Compute the average waiting time until the next bus
Edoardo Patelli University of Liverpool 18 March 2019 81 / 81