DOCSLIB.ORG

Lecture Notes 4 Expectation • Definition and Properties

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Lecture Notes 4 Expectation • Definition and Properties Lecture Notes 4 Expectation • Definition and Properties • Covariance and Correlation • Linear MSE Estimation • Sum of RVs • Conditional Expectation • Iterated Expectation • Nonlinear MSE Estimation • Sum of Random Number of RVs Corresponding pages from B&T: 81-92, 94-98, 104-115, 160-163, 171-174, 179, 225-233, 236-247. EE 178/278A: Expectation Page 4–1 Definition • We already introduced the notion of expectation (mean) of a r.v. • We generalize this definition and discuss it in more depth • Let X ∈X be a discrete r.v. with pmf pX(x) and g(x) be a function of x. The expectation or expected value of g(X) is defined as E(g(X)) = g(x)pX(x) x∈X • For a continuous r.v. X ∼ fX(x), the expected value of g(X) is defined as ∞ E(g(X)) = g(x)fX(x) dx −∞ • Examples: ◦ g(X)= c, a constant, then E(g(X)) = c ◦ g(X)= X, E(X)= x xpX(x) is the mean of X k k ◦ g(X)= X , E(X ) is the kth moment of X ◦ g(X)=(X − E(X))2, E (X − E(X))2 is the variance of X EE 178/278A: Expectation Page 4–2 • Expectation is linear, i.e., for any constants a and b E[ag1(X)+ bg2(X)] = a E(g1(X)) + b E(g2(X)) Examples: ◦ E(aX + b)= a E(X)+ b ◦ Var(aX + b)= a2Var(X) Proof: From the definition Var(aX + b) = E ((aX + b) − E(aX + b))2 = E (aX + b − a E(X) − b)2 = E a2(X − E(X))2 = a2E (X − E(X))2 = a2Var( X) EE 178/278A: Expectation Page 4–3 Fundamental Theorem of Expectation • Theorem: Let X ∼ pX(x) and Y = g(X) ∼ pY (y), then E(Y )= ypY (y)= g(x)pX(x)=E(g(X)) y∈Y x∈X • The same formula holds for fY (y) using integrals instead of sums • Conclusion: E(Y ) can be found using either fX(x) or fY (y). It is often much easier to use fX(x) than to first find fY (y) then find E(Y ) • Proof: We prove the theorem for discrete r.v.s. Consider E(Y )= ypY (y) y = y pX(x) y {x: g(x)=y} = ypX(x) y {x: g(x)=y} = g(x)pX(x)= g(x)pX(x) y x {x: g(x)=y} EE 178/278A: Expectation Page 4–4 Expectation Involving Two RVs • Let (X,Y ) ∼ fX,Y (x, y) and let g(x, y) be a function of x and y. The expectation of g(X,Y ) is defined as ∞ ∞ E(g(X,Y )) = g(x, y)fX,Y (x, y) dxdy −∞ −∞ The function g(X,Y ) may be X, Y , X2, X + Y , etc. • The correlation of X and Y is defined as E(XY ) • The covariance of X and Y is defined as Cov(X,Y )=E[(X − E(X))(Y − E(Y ))] =E[XY − X E(Y ) − Y E(X)+E(X) E(Y )] = E(XY ) − E(X) E(Y ) Note that if X = Y , then Cov(X,Y ) = Var(X) EE 178/278A: Expectation Page 4–5 • Example: Let 2 for x, y ≥ 0, x + y ≤ 1 f(x, y)= 0 otherwise Find E(X), Var(X), and Cov(X,Y ) Solution: The mean is ∞ ∞ E(X)= xf(x, y) dy dx −∞ −∞ 1 1−x 1 1 = 2xdydx = 2 (1 − x)x dx = 3 0 0 0 To find the variance, we first find the second moment 1 1−x 1 1 E(X2)= 2x2 dy dx = 2 (1 − x)x2 dx = 6 0 0 0 Thus, 1 1 1 Var(X)=E(X2) − (E(X))2 = − = 6 9 18 EE 178/278A: Expectation Page 4–6 The covariance of X and Y is 1 1−x Cov(X,Y ) = 2 xy dy dx − E(X) E(Y ) 0 0 1 1 1 1 1 = x(1 − x)2 dx − = − = − 9 12 9 36 0 EE 178/278A: Expectation Page 4–7 Independence and Uncorrelation • Let X and Y be independent r.v.s and g(X) and h(Y ) be functions of X and Y , respectively, then E(g(X)h(Y )) = E(g(X))E(h(Y )) Proof: Let’s assume that X ∼ fX(x) and Y ∼ fY (y), then ∞ ∞ E(g(X)h(Y )) = g(x)h(y)fX,Y (x, y) dx dy −∞ −∞ ∞ ∞ = g(x)h(y)fX(x)fY (y) dx dy −∞ −∞ ∞ ∞ = g(x)fX(x) dx h(y)fY (y) dy −∞ −∞ = E(g(X))E(h(Y )) • X and Y are said to be uncorrelated if Cov(X,Y ) = 0, or equivalently E(XY )=E(X) E(Y ) EE 178/278A: Expectation Page 4–8 • From our independence result, if X and Y are independent then they are uncorrelated To show this, set g(X)=(X − E(X)) and h(Y )=(Y − E(Y )), then Cov(X,Y ) = E[(X − E(X))(Y − E(Y ))] = E(X − E(X))E(Y − E(Y ))=0 • However, if X and Y are uncorrelated they are not necessarily independent • Example: Let X,Y ∈ {−2, −1, 1, 2} such that pX,Y (1, 1) = 2/5, pX,Y (−1, −1) = 2/5 pX,Y (−2, 2) = 1/10, pX,Y (2, −2) = 1/10, pX,Y (x, y) = 0, otherwise Are X and Y independent? Are they uncorrelated? EE 178/278A: Expectation Page 4–9 Solution: y 1/10 2 2/5 1 −2 −11 2 x 2/5 −1 1/10 −2 Clearly X and Y are not independent, since if you know the outcome of one, you completely know the outcome of the other. Let’s check their covariance 2 2 2 2 E(X)= − − + = 0, also 5 5 10 10 E(Y ) = 0, and 2 2 4 4 E(XY )= + − − = 0 5 5 10 10 Thus, Cov(X,Y ) = 0, and X and Y are uncorrelated! EE 178/278A: Expectation Page 4–10 The Correlation Coefficient • The correlation coefficient of X and Y is defined as Cov(X,Y ) ρX,Y = Var(X)Var(Y ) • Fact: |ρX,Y |≤ 1. To show this consider 2 X − E(X) Y − E(Y ) E ± ≥ 0 σ σ X Y E (X − E(X))2 E (Y − E(Y ))2 E (X − E(X))(Y − E(Y )) 2 + 2 ± 2 ≥ 0 σX σY σXσY 1 + 1 ± 2ρX,Y ≥ 0 =⇒ −2 ≤ 2ρX,Y ≤ 2 =⇒ |ρX,Y |≤ 1 • From the proof, ρX,Y = ±1 iff (X − E(X))/σX = ±(Y − E(Y ))/σY (equality with probability 1), i.e., iff X − E(X) is a linear function of Y − E(Y ) • In general ρX,Y is a measure of how closely (X − E(X)) can be approximated or estimated by a linear function of (Y − E(Y )) EE 178/278A: Expectation Page 4–11 Application: Linear MSE Estimation • Consider the following signal processing problem: Noisy Y X ˆ Channel Estimator X aX + b • Here X is a signal (music, speech, image) and Y is a noisy observation of X (output of a noisy communication channel or a noisy circuit). Assume we know the means, variances and covariance of X and Y • Observing Y , we wish to find a linear estimate of X of the form Xˆ = aY + b, which minimizes the mean square error MSE = E (X − Xˆ)2 • We denote the best such estimate as the minimum mean square estimate (MMSE) EE 178/278A: Expectation Page 4–12 • The MMSE linear estimate of X given Y is given by ˆ Cov(X,Y ) X = 2 (Y − E(Y ))+E(X) σY Y − E(Y ) = ρ σ + E(X) X,Y X σ Y and its MSE is given by 2 2 Cov (X,Y ) 2 2 MSE = σX − 2 = (1 − ρX,Y )σX σY • Properties of MMSE linear estimate: – E(Xˆ)=E(X), i.e., estimate is unbiased – If ρX,Y = 0, i.e., X and Y are uncorrelated, then Xˆ = E(X) (ignore the observation Y ) – If ρX,Y = ±1, i.e., X − E(X) and Y − E(Y ) are linearly dependent, then the linear estimate is perfect EE 178/278A: Expectation Page 4–13 Proof 2 • We first show that mina E (X − b) = Var(X) and is achieved for b = E(X), i.e., in the absence of any observations, the mean of X is its minimum MSE estimate, and the minimum MSE is Var(X) To show this consider E (X − c)2 = E [(X − E(X)) + (E(X) − b)]2 = E (X − E(X))2 + (E(X) − b)2+ 2(E( X) − b) E(X − E(X)) = E (X − E(X))2 + (E(X) − b)2 ≥ E (X − E(X))2 , with equality iff b = E(X) • Now, back to our problem. Suppose a has already been chosen. What should b be to minimize E (X − aY − b)2 ? From the above result, we should choose EE 178/278A: Expectation Page 4–14 b = E(X − aY )=E(X) − a E(Y ) So, we want to choose a to minimize E ((X − aY ) − E(X − aY ))2 , which is the same as 2 2 2 2 E ((X − E(X)) − a(Y − E(Y ))) = σX + a σY − 2aCov(X,Y ) This is a quadratic function of a. It is minimized when its derivative equals 0, which gives Cov(X,Y ) ρX,Y σXσY ρX,Y σX a = 2 = 2 = σY σY σY The mean square error is given by 2 2 2 2 2 2 ρX,Y σX 2 ρX,Y σX σX + a σY − 2aCov(X,Y )= σX + 2 σY − 2 × ρX,Y σXσY σY σY 2 2 = (1 − ρX,Y )σX EE 178/278A: Expectation Page 4–15 Mean and Variance of Sum of RVs • Let X1,X2,...,Xn be r.v.s, then by linearity of expectation, the expected value of their sum Y is n n E(Y ) = E Xi = E(Xi) i=1 i=1 Example: Mean of Binomial r.v.
Load more

Recommended publications
  • Laws of Total Expectation and Total Variance
    Laws of Total Expectation and Total Variance Definition of conditional density. Assume and arbitrary random variable X with density fX . Take an event A with P (A) > 0. Then the conditional density fXjA is defined as follows: 8 < f(x) 2 P (A) x A fXjA(x) = : 0 x2 = A Note that the support of fXjA is supported only in A. Definition of conditional expectation conditioned on an event. Z Z 1 E(h(X)jA) = h(x) fXjA(x) dx = h(x) fX (x) dx A P (A) A Example. For the random variable X with density function 8 < 1 3 64 x 0 < x < 4 f(x) = : 0 otherwise calculate E(X2 j X ≥ 1). Solution. Step 1. Z h i 4 1 1 1 x=4 255 P (X ≥ 1) = x3 dx = x4 = 1 64 256 4 x=1 256 Step 2. Z Z ( ) 1 256 4 1 E(X2 j X ≥ 1) = x2 f(x) dx = x2 x3 dx P (X ≥ 1) fx≥1g 255 1 64 Z ( ) ( )( ) h i 256 4 1 256 1 1 x=4 8192 = x5 dx = x6 = 255 1 64 255 64 6 x=1 765 Definition of conditional variance conditioned on an event. Var(XjA) = E(X2jA) − E(XjA)2 1 Example. For the previous example , calculate the conditional variance Var(XjX ≥ 1) Solution. We already calculated E(X2 j X ≥ 1). We only need to calculate E(X j X ≥ 1). Z Z Z ( ) 1 256 4 1 E(X j X ≥ 1) = x f(x) dx = x x3 dx P (X ≥ 1) fx≥1g 255 1 64 Z ( ) ( )( ) h i 256 4 1 256 1 1 x=4 4096 = x4 dx = x5 = 255 1 64 255 64 5 x=1 1275 Finally: ( ) 8192 4096 2 630784 Var(XjX ≥ 1) = E(X2jX ≥ 1) − E(XjX ≥ 1)2 = − = 765 1275 1625625 Definition of conditional expectation conditioned on a random variable.
    [Show full text]
  • Probability Cheatsheet V2.0 Thinking Conditionally Law of Total Probability (LOTP)
    Probability Cheatsheet v2.0 Thinking Conditionally Law of Total Probability (LOTP) Let B1;B2;B3; :::Bn be a partition of the sample space (i.e., they are Compiled by William Chen (http://wzchen.com) and Joe Blitzstein, Independence disjoint and their union is the entire sample space). with contributions from Sebastian Chiu, Yuan Jiang, Yuqi Hou, and Independent Events A and B are independent if knowing whether P (A) = P (AjB )P (B ) + P (AjB )P (B ) + ··· + P (AjB )P (B ) Jessy Hwang. Material based on Joe Blitzstein's (@stat110) lectures 1 1 2 2 n n A occurred gives no information about whether B occurred. More (http://stat110.net) and Blitzstein/Hwang's Introduction to P (A) = P (A \ B1) + P (A \ B2) + ··· + P (A \ Bn) formally, A and B (which have nonzero probability) are independent if Probability textbook (http://bit.ly/introprobability). Licensed and only if one of the following equivalent statements holds: For LOTP with extra conditioning, just add in another event C! under CC BY-NC-SA 4.0. Please share comments, suggestions, and errors at http://github.com/wzchen/probability_cheatsheet. P (A \ B) = P (A)P (B) P (AjC) = P (AjB1;C)P (B1jC) + ··· + P (AjBn;C)P (BnjC) P (AjB) = P (A) P (AjC) = P (A \ B1jC) + P (A \ B2jC) + ··· + P (A \ BnjC) P (BjA) = P (B) Last Updated September 4, 2015 Special case of LOTP with B and Bc as partition: Conditional Independence A and B are conditionally independent P (A) = P (AjB)P (B) + P (AjBc)P (Bc) given C if P (A \ BjC) = P (AjC)P (BjC).
    [Show full text]
  • Probability Cheatsheet
    Probability Cheatsheet v1.1.1 Simpson's Paradox Expected Value, Linearity, and Symmetry P (A j B; C) < P (A j Bc;C) and P (A j B; Cc) < P (A j Bc;Cc) Expected Value (aka mean, expectation, or average) can be thought Compiled by William Chen (http://wzchen.com) with contributions yet still, P (A j B) > P (A j Bc) of as the \weighted average" of the possible outcomes of our random from Sebastian Chiu, Yuan Jiang, Yuqi Hou, and Jessy Hwang. variable. Mathematically, if x1; x2; x3;::: are all of the possible values Material based off of Joe Blitzstein's (@stat110) lectures Bayes' Rule and Law of Total Probability that X can take, the expected value of X can be calculated as follows: P (http://stat110.net) and Blitzstein/Hwang's Intro to Probability E(X) = xiP (X = xi) textbook (http://bit.ly/introprobability). Licensed under CC i Law of Total Probability with partitioning set B1; B2; B3; :::Bn and BY-NC-SA 4.0. Please share comments, suggestions, and errors at with extra conditioning (just add C!) Note that for any X and Y , a and b scaling coefficients and c is our http://github.com/wzchen/probability_cheatsheet. constant, the following property of Linearity of Expectation holds: P (A) = P (AjB1)P (B1) + P (AjB2)P (B2) + :::P (AjBn)P (Bn) Last Updated March 20, 2015 P (A) = P (A \ B1) + P (A \ B2) + :::P (A \ Bn) E(aX + bY + c) = aE(X) + bE(Y ) + c P (AjC) = P (AjB1; C)P (B1jC) + :::P (AjBn; C)P (BnjC) If two Random Variables have the same distribution, even when they are dependent by the property of Symmetry their expected values P (AjC) = P (A \ B jC) + P (A \ B jC) + :::P (A \ B jC) Counting 1 2 n are equal.
    [Show full text]
  • Conditional Expectation and Prediction Conditional Frequency Functions and Pdfs Have Properties of Ordinary Frequency and Density Functions
    Conditional expectation and prediction Conditional frequency functions and pdfs have properties of ordinary frequency and density functions. Hence, associated with a conditional distribution is a conditional mean. Y and X are discrete random variables, the conditional frequency function of Y given x is pY|X(y|x). Conditional expectation of Y given X=x is Continuous case: Conditional expectation of a function: Consider a Poisson process on [0, 1] with mean λ, and let N be the # of points in [0, 1]. For p < 1, let X be the number of points in [0, p]. Find the conditional distribution and conditional mean of X given N = n. Make a guess! Consider a Poisson process on [0, 1] with mean λ, and let N be the # of points in [0, 1]. For p < 1, let X be the number of points in [0, p]. Find the conditional distribution and conditional mean of X given N = n. We first find the joint distribution: P(X = x, N = n), which is the probability of x events in [0, p] and n−x events in [p, 1]. From the assumption of a Poisson process, the counts in the two intervals are independent Poisson random variables with parameters pλ and (1−p)λ (why?), so N has Poisson marginal distribution, so the conditional frequency function of X is Binomial distribution, Conditional expectation is np. Conditional expectation of Y given X=x is a function of X, and hence also a random variable, E(Y|X). In the last example, E(X|N=n)=np, and E(X|N)=Np is a function of N, a random variable that generally has an expectation Taken w.r.t.
    [Show full text]
  • Chapter 3: Expectation and Variance
    44 Chapter 3: Expectation and Variance In the previous chapter we looked at probability, with three major themes: 1. Conditional probability: P(A | B). 2. First-step analysis for calculating eventual probabilities in a stochastic process. 3. Calculating probabilities for continuous and discrete random variables. In this chapter, we look at the same themes for expectation and variance. The expectation of a random variable is the long-term average of the ran- dom variable. Imagine observing many thousands of independent random values from the random variable of interest. Take the average of these random values. The expectation is the value of this average as the sample size tends to infinity. We will repeat the three themes of the previous chapter, but in a different order. 1. Calculating expectations for continuous and discrete random variables. 2. Conditional expectation: the expectation of a random variable X, condi- tional on the value taken by another random variable Y . If the value of Y affects the value of X (i.e. X and Y are dependent), the conditional expectation of X given the value of Y will be different from the overall expectation of X. 3. First-step analysis for calculating the expected amount of time needed to reach a particular state in a process (e.g. the expected number of shots before we win a game of tennis). We will also study similar themes for variance. 45 3.1 Expectation The mean, expected value, or expectation of a random variable X is writ- ten as E(X) or µX . If we observe N random values of X, then the mean of the N values will be approximately equal to E(X) for large N.
    [Show full text]
  • Probabilities, Random Variables and Distributions A
    Probabilities, Random Variables and Distributions A Contents A.1 EventsandProbabilities................................ 318 A.1.1 Conditional Probabilities and Independence . ............. 318 A.1.2 Bayes’Theorem............................... 319 A.2 Random Variables . ................................. 319 A.2.1 Discrete Random Variables ......................... 319 A.2.2 Continuous Random Variables ....................... 320 A.2.3 TheChangeofVariablesFormula...................... 321 A.2.4 MultivariateNormalDistributions..................... 323 A.3 Expectation,VarianceandCovariance........................ 324 A.3.1 Expectation................................. 324 A.3.2 Variance................................... 325 A.3.3 Moments................................... 325 A.3.4 Conditional Expectation and Variance ................... 325 A.3.5 Covariance.................................. 326 A.3.6 Correlation.................................. 327 A.3.7 Jensen’sInequality............................. 328 A.3.8 Kullback–LeiblerDiscrepancyandInformationInequality......... 329 A.4 Convergence of Random Variables . 329 A.4.1 Modes of Convergence . 329 A.4.2 Continuous Mapping and Slutsky’s Theorem . 330 A.4.3 LawofLargeNumbers........................... 330 A.4.4 CentralLimitTheorem........................... 331 A.4.5 DeltaMethod................................ 331 A.5 ProbabilityDistributions............................... 332 A.5.1 UnivariateDiscreteDistributions...................... 333 A.5.2 Univariate Continuous Distributions . 335
    [Show full text]
  • (Introduction to Probability at an Advanced Level) - All Lecture Notes
    Fall 2018 Statistics 201A (Introduction to Probability at an advanced level) - All Lecture Notes Aditya Guntuboyina August 15, 2020 Contents 0.1 Sample spaces, Events, Probability.................................5 0.2 Conditional Probability and Independence.............................6 0.3 Random Variables..........................................7 1 Random Variables, Expectation and Variance8 1.1 Expectations of Random Variables.................................9 1.2 Variance................................................ 10 2 Independence of Random Variables 11 3 Common Distributions 11 3.1 Ber(p) Distribution......................................... 11 3.2 Bin(n; p) Distribution........................................ 11 3.3 Poisson Distribution......................................... 12 4 Covariance, Correlation and Regression 14 5 Correlation and Regression 16 6 Back to Common Distributions 16 6.1 Geometric Distribution........................................ 16 6.2 Negative Binomial Distribution................................... 17 7 Continuous Distributions 17 7.1 Normal or Gaussian Distribution.................................. 17 1 7.2 Uniform Distribution......................................... 18 7.3 The Exponential Density...................................... 18 7.4 The Gamma Density......................................... 18 8 Variable Transformations 19 9 Distribution Functions and the Quantile Transform 20 10 Joint Densities 22 11 Joint Densities under Transformations 23 11.1 Detour to Convolutions......................................
    [Show full text]
  • Lecture 15: Expected Running Time
    Lecture 15: Expected running time Plan/outline Last class, we saw algorithms that used randomization in order to achieve interesting speed-ups. Today, we consider algorithms in which the running time is a random variable, and study the notion of the "expected" running time. Las Vegas algorithms Most of the algorithms we saw in the last class had the following behavior: they run in time , where is the input size and is some parameter, and they have a failure probability (i.e., probability of returning an incorrect answer ) of . These algorithms are often called Monte Carlo algorithms (although we refrain from doing this, as it invokes di!erent meanings in applied domains). Another kind of algorithms, known as Las Vegas algorithms, have the following behavior: their output is always correct, but the running time is a random variable (and in principle, the run time can be unbounded). Sometimes, we can convert a Monte Carlo algorithm into a Las Vegas one. For instance, consider the toy problem we considered last time (where an array is promised to have indices for which and the goal was to "nd one such index). Now consider the following modi"cation of the algorithm: procedure find_Vegas(A): pick random index i in [0, ..., N-1] while (A[i] != 0): pick another random index i in [0, ..., N-1] end while return i Whenever the procedure terminates, we get an that satis"es . But in principle, the algorithm can go on for ever. Another example of a Las Vegas algorithm is quick sort, which is a simple procedure for sorting an array.
    [Show full text]
  • Missed Expectations? 1 Linearity of Expectation
    Massachusetts Institute of Technology Course Notes, Week 14 6.042J/18.062J, Fall ’05: Mathematics for Computer Science December 2 Prof. Albert R. Meyer and Prof. Ronitt Rubinfeld revised December 8, 2005, 61 minutes Missed Expectations? In the previous notes, we saw that the average value of a random quantity is captured by the mathematical concept of the expectation of a random variable, and we calculated expectations for several kinds of random variables. Now we will see two things that make expectations so useful. First, they are often very easy to calculate due to the fact that they obey linearity. Second, once you know what the expectation is, you can also get some type of bound on the probability that you are far from the expectation —that is, you can show that really weird things are not that likely to happen. How good a bound you can get depends on what you know about your distribution, but don’t worry, even if you know next to nothing, you can still say something relatively interesting. 1 Linearity of Expectation 1.1 Expectation of a Sum Expected values obey a simple, very helpful rule called Linearity of Expectation. Its sim­ plest form says that the expected value of a sum of random variables is the sum of the expected values of the variables. Theorem 1.1. For any random variables R1and R2, E [R1+R2] =E [R1] +E [R2] . Proof. Let T::=R1+R2. The proof follows straightforwardly by rearranging terms from the definition of E [T ]. E [R1+R2] ::=E [T ] � ::= T (s) · Pr {s} s∈S � = (R1(s) +R2(s))· Pr {s} (Def.
    [Show full text]
  • On the Method of Moments for Estimation in Latent Variable Models Anastasia Podosinnikova
    On the Method of Moments for Estimation in Latent Variable Models Anastasia Podosinnikova To cite this version: Anastasia Podosinnikova. On the Method of Moments for Estimation in Latent Variable Models. Machine Learning [cs.LG]. Ecole Normale Superieure de Paris - ENS Paris, 2016. English. tel- 01489260v1 HAL Id: tel-01489260 https://tel.archives-ouvertes.fr/tel-01489260v1 Submitted on 14 Mar 2017 (v1), last revised 5 Apr 2018 (v2) HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. THÈSE DE DOCTORAT de l’Université de recherche Paris Sciences Lettres – PSL Research University préparée à l’École normale supérieure Sur la méthode des mo- École doctorale n°386 ments pour d’estimation Spécialité: Informatique des modèles à variable latentes Soutenue le 01.12.2016 On the method of moments for es- timation in latent variable models par Anastasia Podosinnikova Composition du Jury : Mme Animashree ANANDKUMAR University of California Irvine Rapporteur M Samuel KASKI Aalto University Rapporteur M Francis BACH École normale supérieure Directeur de thèse M Simon LACOSTE-JULIEN École normale supérieure Directeur de thèse M Alexandre D’ASPREMONT École normale supérieure Membre du Jury M Pierre COMON CNRS Membre du Jury M Rémi GRIBONVAL INRIA Membre du Jury Abstract Latent variable models are powerful probabilistic tools for extracting useful latent structure from otherwise unstructured data and have proved useful in numerous ap- plications such as natural language processing and computer vision.
    [Show full text]
  • Advanced Probability
    Advanced Probability Alan Sola Department of Pure Mathematics and Mathematical Statistics University of Cambridge [email protected] Michaelmas 2014 Contents 1 Conditional expectation 4 1.1 Basic objects: probability measures, σ-algebras, and random variables . .4 1.2 Conditional expectation . .8 1.3 Integration with respect to product measures and Fubini's theorem . 12 2 Martingales in discrete time 15 2.1 Basic definitions . 15 2.2 Stopping times and the optional stopping theorem . 16 2.3 The martingale convergence theorem . 20 2.4 Doob's inequalities . 22 2.5 Lp-convergence for martingales . 24 2.6 Some applications of martingale techniques . 26 3 Stochastic processes in continuous time 31 3.1 Basic definitions . 31 3.2 Canonical processes and sample paths . 36 3.3 Martingale regularization theorem . 38 3.4 Convergence and Doob's inequalities in continuous time . 40 3.5 Kolmogorov's continuity criterion . 42 3.6 The Poisson process . 44 1 4 Weak convergence 46 4.1 Basic definitions . 46 4.2 Characterizations of weak convergence . 49 4.3 Tightness . 51 4.4 Characteristic functions and L´evy'stheorem . 53 5 Brownian motion 56 5.1 Basic definitions . 56 5.2 Construction of Brownian motion . 59 5.3 Regularity and roughness of Brownian paths . 61 5.4 Blumenthal's 01-law and martingales for Brownian motion . 62 5.5 Strong Markov property and reflection principle . 67 5.6 Recurrence, transience, and connections with partial differential equations . 70 5.7 Donsker's invariance principle . 76 6 Poisson random measures and L´evyprocesses 81 6.1 Basic definitions .
    [Show full text]
  • Overdispersed Models for Claim Count Distribution
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by DSpace at Tartu University Library TARTU UNIVERSITY FACULTY OF MATHEMATICS AND COMPUTER SCIENCE Institute of Mathematical Statistics Frazier Carsten Overdispersed Models for Claim Count Distribution Master’s Thesis Supervisor: Meelis K¨a¨arik, Ph.D TARTU 2013 Contents 1 Introduction 3 2 Classical Collective Risk Model 4 2.1 Properties ................................ 4 2.2 CompoundPoissonModel . 8 3 Compound Poisson Model with Different Insurance Periods 11 4 Overdispersed Models 14 4.1 Introduction............................... 14 4.2 CausesofOverdispersion. 15 4.3 Overdispersion in the Natural Exponential Family . .... 17 5 Handling Overdispersion in a More General Framework 22 5.1 MixedPoissonModel.......................... 23 5.2 NegativeBinomialModel. 24 6 Practical Applications of the Overdispersed Poisson Model 28 Kokkuv˜ote (eesti keeles) 38 References 40 Appendices 41 A Proofs 41 B Program codes 44 2 1 Introduction Constructing models to predict future loss events is a fundamental duty of actu- aries. However, large amounts of information are needed to derive such a model. When considering many similar data points (e.g., similar insurance policies or in- dividual claims), it is reasonable to create a collective risk model, which deals with all of these policies/claims together, rather than treating each one separately. By forming a collective risk model, it is possible to assess the expected activity of each individual policy. This information can then be used to calculate premiums (see, e.g., Gray & Pitts, 2012). There are several classical models that are commonly used to model the number of claims in a given time period.
    [Show full text]