Review of Probability Wilks, Chapter 2 Ji-Sun Kang and Eugenia Kalnay AOSC630 Spring, 2008, updated in 2017 Definition § Event: set, class, or group of possible uncertain outcomes q A compound event can be decomposed into two or more (sub) events q An elementary event cannot be decomposed § Sample space (event space), S q The set of all possible elementary events § MECE (Mutually Exclusive & Collectively Exhaustive) q Mutually Exclusive: no more than one of the events can occur q Collectively Exhaustive: at least one of the events will occur ! A set of MECE events completely fills a sample space Probability Axioms § P(A) ≥ 0 § P(S) = 1 § If (E1 ∩ E2)=0, i.e., if E1 and E2 exclusive, then P(E1 ∪ E2)=P(E1)+P(E2) E1 E2 Probability § Probability ~ Frequency Recall Threat Score (TS) #E = yes q P(E) = lim P(F = yes ∩ Ob = yes) n→∞ total _ n = P(F = yes ∪ Ob = yes) § If E 2 ⊆ E 1 , then P(E1) ≥ P(E2 ) = § P (E1 ∪ E2 ) = P(E1) + P(E2 ) − P(E1 ∩ E2 ) Venn Diagrams E E E1 2 E1E 1 ∪ E2 2 E1 ∩ E2 Conditional Probability § Probability of E1 given that E2 has happened P(E ∩ E ) P(E E ) = 1 2 1 2 P(E ) 2 P(E1 ∩ E2 ) = P(E1 E2 )P(E2 ) ; Multiplicative Law § Independent Event q The occurrence or nonoccurrence of one does not affect the probability of the other P ( E1 ∩ E 2 ) = P ( E1 ) P ( E 2 ) P(E1 ∩ E2 ) i.e. P(E1 E2 ) = = P(E1) P(E2 ) Exercise § From the Penn State station data for Jan. 1980, compute the probability of precipitation, of T>32F, conditional probability of precipitation if T>32F, and conditional probability of precipitation tomorrow if it is raining today § Prove graphically the DeMorgan Laws: P (A∪B)c = P Ac ∩Bc ; P (A∩B)c = P Ac ∪Bc ! { } { } { } { } Total Probability § MECE events, {Ei}, i=1, …, I I I P(A) = ∑ P(A ∩ Ei ) = ∑ P(AEi )P(Ei ) i=1 i=1 S A ∩ E2 A E A ∩ E3 A ∩ E4 ∩ 5 E1 E2 E3 E4 E5 Bayes’ Theorem § Bayes’ theorem is used to “invert” conditional probabilities q If P(E1|E2) is known, Bayes’ Theorem may be used to compute P(E2|E1). Multiplicative law P(Ei ∩ A) P(A Ei )P(Ei ) P(A Ei )P(Ei ) P(Ei A) = = = P(A) P(A) I ∑ P(A Ei )P(E j ) Law of total probability j=1 q Combines prior information with new information Example of Bayesian Reasoning § Relationship between precipitation over SE US and El Nino q Precipitation Events: E1(above), E2(normal), E3(below) are MECE q El Nino Event: A q Prior information (from past statistics) ü P(E1)=P(E2)=P(E3)=33% ü P(A|E 1)=40%; P(A|E 2)=20%; P(A|E 3)=0% P(A ∩ E ) P(A| E ) = 1 = =0.40 1 P(E ) A 1 P(A ∩ E2 ) P(A| E2 ) = = =0.20 P(E2 ) E1 33% E 2 33% E 3 33% Example of Bayesian Reasoning § Total probability of A 3 P(A) = ∑ P(A| Ei )P(Ei ) i=1 = P(A| E1)P(E1) + P(A| E2 )P(E2 ) + P(A| E3 )P(E3 ) = 0.4*0.33 + 0.2*0.33 + 0*0.33 = 0.20 § NEW information: El Nino is happening! Probability of above normal precipitation? P(A ∩ E ) P(E | A) = 1 1 P(A) A 0.4*0.33 = = 0.66 = 0.2 E1 33% E 2 33% E 3 33% Probability Density Function when we have two observations, T1 and T2 With Gaussian errors, the best estimate of T is the weighted average of T1 and T2: The weights are proportional to the error variance of the other obs.: 2 2 σ 2T1 +σ 1 T2 T = 2 2 σ 1 +σ 2 2 Precision=inverse of σ Precision of T= Precision of T1 + Precision of T2: 1 1 1 2 = 2 + 2 σ T σ 1 σ 2 If we have a forecast Tf (prior info.) and then an observation To The optimal analysis Ta is the best estimate of the truth (it has minimum errors): 2 2 σ oTf +σ f To Ta = 2 2 σ f +σ o The analysis Ta is more accurate than both the forecast Tf and the observation To ! 1 1 1 2 = 2 + 2 σ a σ f σ o These are the formulas used in data assimilation! Now let’s use a Bayesian approach for Data Assimilation Bayes theorem P(T | To ) = P(T | Tf )P(To | T ) / P(To ) “The posterior probability of the true temperature T given the prior information Tf , and after receiving the new observation To , is given by the prior probability of T (based on the forecast Tf ) multiplied by the likelihood of T given the observation To, normalized by the total probability of obtaining a measurement To.” The likelihood of T given the observation To is the same as the probability of observing To given a true temperature T (Edwards, 1984). This formula can be briefly read as: “posterior = prior . likelihood /normalization” The Bayesian approach for Data Assim is very general (not just Gaussians) If we assume Gaussianity, the Bayes theorem leads to the Variational approach: P(T | To ) = P(T | Tf )P(To | T ) / P(To ) = 2 2 2 2 −(T −T f ) /2σ f −(To −T ) /2σ o e e 2πσ o 2 2 = −(To −Tcli ) /2σ o 2πσ f 2πσ o e 2 2 2 2 const *e−[(T −T f ) /2σ f +(To −T ) /2σ o ] Since we want to maximize the probability of T, and Tcli, the climatological temperature probability distribution does not depend on T, the maximization can be written as the minimization of the exponent: The Bayesian approach for Data Assim is very general (not just Gaussians) From Bayes theorem, we minimize -the exponent, a cost function J that measures the squared distance between the optimal temperature T that we are seeking and the prior forecast, and with the new observation, both normalized by their error variances: 2 2 2 2 J = (T − Tf ) / 2σ f + (To − T ) / 2σ o Although the variational formulation looks very different from the classical formulation shown before, for Gaussian errors, both give the same solution. However, the Bayesian approach can be used with any probability distributions and allows the implementation of efficient “particle filters” (e.g., Penny and Miyoshi, 2016, Poterjoy, 2016). .