A Statistical Technique for Evaluating Hurricane Modification Experiments
Total Page:16
File Type:pdf, Size:1020Kb
QC NOAA Technical Memorandum ERL WMPO-29 U.S. DEPARTMENT OF COMMERCE NATIONAL OCEANIC AND ATMOSPHERIC ADMINISTRATION Environmental Research Laboratories A STATISTICAL TECHNIQUE FOR EVALUATING HURRICANE MODIFICATION EXPERIMENTS Richard W. Knight Glenn W. Brier Weather Modification Program Office Boulder, Colo. May 1976 QC 'U. to wS -wo.^ NOAA Technical Memorandum ERL WMPO-29 A STATISTICAL TECHNIQUE FOR EVALUATING HURRICANE MODIFICATION EXPERIMENTS Richard W. Knight Glenn W. Brier National Hurricane and Experimental Meteorology Laboratory Coral Gables, Florida Weather Modification Program Office Boulder, Colo. May 1976 LIBRARY JUL 20 1976 N.QAA. U* S. Dept, of Commerce ATMOsp^ UNITED STATES NATIONAL OCEANIC AND Environmental Research DEPARTMENT OF COMMERCE ATMOSPHERIC ADMINISTRATION Laboratories Elliot L. Richardson, Secretary Robert M White, Administrator Wilmot N Hess Director Of °C lb 29/6 CONTENTS Page ILLUSTRATIONS .. iv TABLES.. v ABSTRACT 1 1. INTRODUCTION 1 2. GENERAL DESCRIPTION 2 3. EXAMPLE OF THE B-SCORE COMPUTATION FROM A HYPOTHETICAL CASE 6 4. DATA REQUIREMENTS FOR THE COMPUTATION OF THE B-SCORE 9 5. SUMMARY AND CONCLUSIONS 9 6. REFERENCES 11 iii ILLUSTRATIONS Figure Page Three forecast functions making up a forecast period 1. ℓ and having a monitoring increment period of length Δ. 3 The total monitoring period T, the forecast period 2. ℓ, and the monitoring increment number T1 to T99+m. 5 3. Pressure, temperature, and dew point observed at a hypothetical station during a month. 7 TABLES Table Page 1. Example of matrix of observed and forecast events arranged for comparison and computation of scores 4 2. Array of indices used for B-Score computation 5 3. Change criteria and forecast function parameters used with cold frontal passages at a hypothetical 6 station 4. B-Score parameters for two hypothetical experiments 8 Y A STATISTICAL TECHNIQUE FOR EVALUATING HURRICANE MODIFICATION EXPERIMENTS Richard W. Knight and Glenn W. Brier1 A statistical technique is developed for evaluating the non- randomized Project STORMFURY hurricane seeding experiments. Modern principles of design and analysis of comparative experiments use (1) replication, from which a quantitative estimate can be made of the experimental "error" or the variability of the response to a treatment and (2) randomization_, a process of allocating treatments to the experimental material by tossing a coin (or equivalent procedure), which may make it possible to attribute whatever effects are observed to the treatment only. Together, these two principles enable one to assess the validity of the results in terms of a probability statement. However, in the STORMFURY Project it is planned to seed nearly all experimental units, leaving essentially no controls. With the concept of randomization in time, it is possible to develop an evaluation technique to quantitatively determine whether there is an association between a treatment and the event following the treatment. A hypothetical example is presented to facilitate the understanding of the technique; the elements that are most likely to identify seeding-induced changes in a storm environment are discussed briefly. 1. INTRODUCTION The seeding hypothesis used in Project STORMFURY is based on definite cause and effect relationships that, in turn, are based on theoretical and observational studies. Basically, the hypothesis calls for seeding clouds at radii greater than that of the eyewall to enhance convection outside of the eyewall region. If convection can be successfully stimulated, part of the low-level inflow originally maintaining the eyewall convection will be directed into convection at a radius greater than that of the eyewall, and thus the transport of angular momentum and water vapor to the old eyewall will be reduced. As the region of major vertical mass transport is shifted to a greater radius, the maximum winds will diminish due to conservation of momentum principles. The requirements, therefore, to objectively evaluate the results of a hurricane seeding are the following: 1) Observe the entire sequence of changes that occurs in the meteorolog ical elements to test whether this sequence corresponds to the hypothesized chain of events, in space and time, and 2) Determine the statistical significance of this correspondence. 1 Glenn Brier is a Research Associate in the Department of Atmospheric Sciences, Colorado State University, Fort Collins, Colorado 80523 It is well known that the natural time and space variability of meteorological elements measured in hurricanes is very high, a fact estab lished both by land-based observations and those obtained from aircraft, (e.g., peak wind gusts may exceed 1-minute sustained wind speeds by 20 to 25 percent (Atkinson, 1974). This, of course, causes serious problems with testing the results of seeding when the classical designs for comparative experiments are used. For example, even if dramatic changes in a particu lar element were to occur after a seeding, it would take data from many trials to show that indeed there must be a seeding-induced change. The process of collecting a sufficient number of treated and control cases might take years, which is an unacceptable delay. An alternative technique, therefore, was developed by Brier (1975) for evaluation of the results of Project STORMFURY seeding experiments. The proposed statistical technique is based on a quantitative description of the hypothesized sequence and timing of events expected to occur after a seeding treatment. It becomes obvious that the likelihood of some complex sequence occurring by chance will be considerably lower than that for a single-element event. Therefore, the technique should have the capability of providing a quantitative answer to the question: how well did the hypothetical sequence of events predict what actually happened after the seeding when compared to the variations and events during the rest of the monitoring period? Essentially, the remainder of this monitoring period acts as a control. The description of the development of the technique (hereinafter called the B-Score or B), is covered in the following section. 2. GENERAL DESCRIPTION The objective is to compute a score or index to measure the association between the hypothesized and observed sequence of events A, B, C, ..., in a monitoring period of length T, with the forecast sequence F(A), F(B), F(C),... restricted to a period of length & when & << T. The events A, B, C,... are those defined by some physical hypothesis to follow a seeding treatment (S) when monitoring is taking place. An example would be the time required for a 1°C temperature drop in the eye of the storm. The forecasts of the events A, B, C are designated as F(A), F(B), F(C),... and specified by the hypothesis to occur after the treatment S during the interval between to and t^. This forecast time length & is defined by the time span encompassed by the distribution curves (assumed here to be Gaussian) of the forecast functions out to + 3a (fig.1). The forecast functions F(A), F(B), F(C)... are represented by normal distributions with maxima at the expected times for events A, B, C... to occur, based on the physical hypothesis. Actually, any appropriate distribution can be used for the forecast functions, and a change will be made if warranted by future experience. The standard deviations and maximum ordinates may all be different. The forecast functions are "standardized" or adjusted so that the maximum ordinates sum to unity. This provides for an index I, which has a maximum value of unity in case of perfect agreement between the forecast times ta, t^, tc and observed events at T^ and Tg, Tq respectively. The minimum value of I is zero and occurs, for example, if events A, B, C... occur outside the interval t^- tQ. Indices intermediate between zero and unity can be obtained 2 Figure 1. Three forecast functions making up a forecast period Z having a monitoring increment period of length A. The increment A is arbitrary, but should be small enough to represent the continuity in F(A) , F(B), and F(C). (Actual values of F(A), F(B), F(C), used by the computer program, are taken from the curves at points marked by •.) and the problem is to devise a score to determine whether a particular index Is (corresponding to the beginning of the sequence of events caused by seeding), is unusual with respect to the general concept of randomization in time. The following procedure is used. The hypothesized means (t^ , , tc ...) and standard deviations aa> crfc>> ac•**) of the forecast functions F(A), F(B), F(C),... respectively are specified based on previous experience and/or results of numerical hurricane models. The forecast functions all have zero value outside of the interval Z. The next step is to subtract the interval Z from the total monitoring period T, leaving a period T-£ = L (fig.2). The interval L is next divided into 99 parts so that if T^ is the beginning of the monitoring period, then T2 = Tj + A t3 = Ti + 2A T100 = T + 99A • • T = T100 + where m is an integer such that mA = Z. (This usually necessitates a slight adjustment in Z and T.) Next, the functions F(A), F(B), F(C) ... are 3 Table 1. Example* of matrix of observed and forecast events arranged for comparison and computation of scores (forecast function values are taken from points on the curves marked in fig. 1) Observed events Forecast sequence Product values Forecast Col 1 Monitor increment increment +Col 2 number A B c number F(A) F(B) F(C) AxF(A) Bx(B) CxF(C) +Col i 0 0 0 1 .00 .00 .00 0 0 0 0 2 0 0 0 2 . A2 .01 .00 0 0 0 0 3 1 1 0 3 .13 .15 .00 .13 .15 0 .28 A 0 0 1 A .00 .31 .03 0 0 .03 .03 5 0 0 0 5 .00 .09 .08 0 0 0 0 6 0 0 0 6 .00 .00 • 1A.