MAY 2013 K I M E T A L . 873
Real-Time Detection and Filtering of Chaff Clutter from Single-Polarization Doppler Radar Data
YONG HYUN KIM AND SUNGSHIN KIM Department of Electronics and Electrical Engineering, Pusan National University, Busan, South Korea
HYE-YOUNG HAN,BOK-HAENG HEO, AND CHEOL-HWAN YOU Korea Meteorological Administration, Seoul, South Korea
(Manuscript received 1 August 2012, in final form 26 November 2012)
ABSTRACT
In countries with frequent aerial military exercises, chaff particles that are routinely spread by military aircraft represent significant noise sources for ground-based weather radar observation. In this study, a cost- effective procedure is proposed for identifying and removing chaff echoes from single-polarization Doppler radar readings in order to enhance the reliability of observed meteorological data. The proposed quality control procedure is based on three steps: 1) spatial and temporal clustering of decomposed radar image elements, 2) extraction of the clusters’ static and time-evolution characteristics, and 3) real-time identification and removal (or censoring) of target echoes from radar data. Simulation experiments based on this procedure were conducted on site-specific ground-echo-removed weather radar data provided by the Korea Meteoro- logical Administration (KMA), from which three-dimensional (3D) reflectivity echoes covering hundreds of thousands of square kilometers of South Korean territory within an altitude range of 0.25–10 km were re- trieved. The algorithm identified and removed chaff clutter from the South Korean data with a novel decision support system at an 81% accuracy level under typical cases in which chaff and weather clusters were isolated from one another with no overlapping areas.
1. Introduction a. Problem posed by chaff echoes and the proposed approach The network of ground-based weather radars is an essential tool for real-time monitoring of rapidly de- Chaff is a radar countermeasure commonly com- veloping weather events, and in assessing the near-term posed of metallized glass fibers or other lightweight potential threat level posed by these events. Thus, the strips of reflecting material that are released by aircraft ability to accurately forecast the course of weather to distract radar-guided missiles from their targets. The events critically depends on the reliability of measure- chaff bands are very shallow and tend to show a distinct ments coming from these sources. However, weather vertical tilt from the vertical viewpoint after a given radar readings are subject to nonweather noise, such as amount of time has elapsed from the chaff’s initial re- normal ground (NG), anomalous propagation (AP), lease. In a country such as South Korea where very chaff, contrail, and hazardous clutter sources (fire, ash frequent aerial military exercises are held in a relatively plumes). These sources are often confused with weather confined air space, chaff clutters with a sizeable volume echoes, and significantly compromise the reliability of and coverage area (several hundred square kilometers) radar data. In particular, chaff clutter represents one of were observed on 231 out of 365 days in 2010 (see the main noise sources among nonweather noise echoes Fig. 1). This represents 63.3% of the total observation in countries with frequent aerial military exercises. days (Han et al. 2011). These dispersed nonweather agents easily extend to several tens of kilometers over a large area. They make it very difficult to obtain reli- Corresponding author address: Sungshin Kim, Department of Electronics and Electrical Engineering, Pusan National University, able weather condition estimates in an affected area in Jangjeon-dong, Geumjeong-gu, Busan 609-735, South Korea. the hours following the exercise—chaff remains air- E-mail: [email protected] borne for roughly 8–15 h.
DOI: 10.1175/JTECH-D-12-00158.1
Ó 2013 American Meteorological Society Unauthenticated | Downloaded 10/08/21 04:20 AM UTC 874 JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY VOLUME 30
FIG. 1. KMA’s 2010 record of monthly variation of the number of days that chaff echoes are observed. (bottom) Number of days that chaff echoes were observed, and (top) their percentage of the total number of days in each month (readapted from Han et al. 2011).
As a result, cheap and robust methods for identifying and removing chaff echoes from radar data are being actively investigated by national weather forecasting FIG. 2. The network of ground-based radars used by the KMA. agencies in a number of countries in order to improve the quality of meteorological data. However, difficulty arises from the fact that with a current nationwide network (such as average intensity, storm size, velocity variance, of single-polarization-based C- and S-band Doppler radars storm shape, and orientation) to be tracked over time installed in the majority of affected weather observation in addition to the storm center’s location. Johnson et al. centers, residual chaff clutter is not easily distinguished (1998) used an enhanced Weather Surveillance Radar- from weather echoes because of their overlapping reflec- 1988 Doppler (WSR-88D) centroid tracking algorithm tivity range. called Storm Cell Identification and Tracking (SCIT) to In an attempt to solve this problem, analyses of time track and identify storm cells (isolated, clustered, or series 3D radar constant altitude plan position indicator line storms) at various levels of maximum threshold (CAPPI) data are being conducted in this study to iden- reflectivity. tify key discriminating characteristics that can differen- c. Previous works on chaff characterization tiate chaff echoes from weather echoes to a reasonable degree of accuracy. Previous works on chaff characterization are pri- marily concerned with monostatic and bistatic radar b. Previous works on tracking amorphous cross sections (RCSs). For monostatic RCSs, Marcus weather objects (2004) developed a chaff model based on aerodynamic There are several algorithms described in the current principles that determines density and orientation dis- literature that are designed to automatically track de- tributions of reflective fibers within chaff clutters. For formable weather structures (usually storm cells). They bistatic RCSs, Guo and Uberall€ (1992) proposed a include a spatial relaxation-labeling algorithm (Barnard variational method of computing radar-scattering cross and Thompson 1980) that uses disparity analysis to de- sections for a bistatic-scattering cross section for chaff termine the correspondence between a set of feature dipoles. points selected from a stereogram, and a temporal re- The terminal chaff fall velocity was calculated by es- laxation algorithm (Zhang 1991) that tracks Euclidean timating a chaff drag coefficient as described by Jiusto storm centers over time and handles the merging and and Eadie (1963). Vasiloff and Struthwolf (1997) briefly splittingofstormsbyallowingsinglestormstobematched detailed the time-evolution characteristics of chaff echoes to several storms in previous or subsequent image frames. and their dissipation pattern within convective cloud cells, Krezeski et al. (1994) improved Zhang’s tracking algo- resulting in inaccurate estimation of the amount of pre- rithm by adding the concept of pseudostorms and property cipitation for observed cloud structures. Through the coherence, which allows multiple features of a storm analysis of the chaff’s polarimetric properties, Zrni cand
Unauthenticated | Downloaded 10/08/21 04:20 AM UTC MAY 2013 K I M E T A L . 875
FIG. 3. Proposed CEDR QC algorithm.
Ryzhkov (2004) concluded that polarimetric radars could 2. System overview provide a simple and effective way to detect chaff based a. Radar data used on experimental evidence obtained with circularly polar- ized radars. The Korea Meteorological Administration (KMA) primarily collects weather information from 11 ground- d. Previous works on identification of various types based radar observation centers with a combined coverage of radar clutters area large enough to cover the entire South Korean pen- In the past several years, various techniques have been insula, as illustrated in Fig. 2. Three of these centers— proposed on radar data quality control (QC). Many of Baengyeongdo, Youngjongdo, and Myeonbongsan—use these studies were primarily concerned with the detec- single-polarization C-band Doppler radars. Eight other tion of NG and AP clutter (Pratte et al. 1993; Pamment centers use single-polarization S-band Doppler radars. and Conway 1998; Grecu and Krajewski 1999; Kessinger The observation range for each site varies between 200 and et al. 1999; Steiner and Smith 2002; Berenguer et al. 2006; 256 km. Radars at each site feature 10–16 elevation angles Cho et al. 2006; Lakshmanan et al. 2007; Hubbert et al. for each sweep and generate 3D radar volume data in 2009), while others were concerned with the detection of universal format (UF) at 10-min intervals. The radar vol- less routinely occurring hazardous clutter, such as fire ume data used within UF data structures for the QC pro- plumes (Rogers and Brown 1977; Melnikov et al. 2008) cedure are ground echoes-removed (CZ) reflectivity data and volcanic ash clutter (Marzano et al. 2006). Many of that are the by-product of Gaussian model adaptive pro- these detection techniques adopt in some way fuzzy in- cessing (GMAP) and infinite impulse response (IIR) fil- ference (FI) or neural network (NN)-based clutter dis- tering (Siggia and Passarelli 2004). From these volume crimination schemes at the range gate level of one or data, 3D CAPPI data with 1-km horizontal and 0.25-km more radar moments (reflectivity, velocity, spectrum vertical resolution are constructed for an altitude range of width, etc.). In addition, many of these studies apply 0.25–10 km. Our study features two test simulation cases a more holistic and crisp approach of filtering data using dated 3 and 15 June 2010 at the Oseongsan (KSN) site radar moment thresholding during the preprocessing and that were chosen among 116 cases representing different postprocessing QC stages. sample dates, times, and sites between 2010 and 2011.
Unauthenticated | Downloaded 10/08/21 04:20 AM UTC 876 JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY VOLUME 30
FIG. 4. 2D example of NS clustering procedure with hypersphere (dotted square) scan radius of 1 grid distance resulting in three clusters.
Since weather events are monitored in real time at procedures (these procedures apply to all time series 10-min intervals, the QC procedure for all 11 sites must images or frames within a user-defined time period) be- be completed within the short time frame of 10 min or fore being subjected to the image element tracking and less. The algorithm thus takes into account this time re- inferring procedures designed to detect and remove chaff striction and limited computational resources that impose signatures. significant constraints on the choice of QC model and As a first step, the algorithm performs a coordinate representing procedures. system transformation of the original site-specific reflec- tivity volume data from polar to Euclidean coordinates b. Proposed chaff echo detection and removal before subjecting these data to an image decomposition (CEDR) QC algorithm (data partitioning) procedure. This transformation is In the proposed QC algorithm, illustrated in Fig. 3, the achieved by retrieving interpolated 2D CAPPI layers original time series radar data undergo a series of from original CZ reflectivity volume data for regularly transformations, sorting, and image element regrouping spaced altitudes. The layers are then stacked one upon
Unauthenticated | Downloaded 10/08/21 04:20 AM UTC MAY 2013 K I M E T A L . 877
TABLE 1. Simulation environment and parameters for NS TABLE 2. Simulation time measurement of NS clustering. clustering. Total Average Environment settings clustering clustering and parameters Description and values Radar sample date time (s) time (s) CPU Intel Core i5 M480 2.67 GHz KSN 3 Jun 2010 (1000–1300 KST 7.79 0.41 RAM 3072 MB at 10-min intervals for a total of OS Linux Ubuntu 10.10 19 inference time frames) Radar polarization Horizontal KSN 15 Jun 2010 (1000–1300 KST 6.27 0.33 type at 10-min intervals for a total Test radar site KSN (Oseongsan) of 19 inference time frames) Size of LS R 3 u 3 f: 980 3 360 3 15 (total: 5 292 000 bytes) Size of LE X 3 Y 3 Z: 480 3 480 3 41 (total: 9 446 400 bytes) the temporal clustering procedure forms an isocluster L 3 3 3 3 Resolution of E X Y Z:1km 1km 0.25 km subspace where member elements are 2-tuple cluster Zth 0dBZ 3 identifiers (t, k) for clusters in different time frames that Sth 190 grid data points or 47.5 km tk 2 grid distance are determined to be similar by criteria of cluster size and cluster centroid location (t denotes a time frame index and k denotes cluster ID). The establishment of the corre- spondence between clusters in subsequent 3D radar im- the other, thereby resulting in 3D Euclidean reflectivity ages enables the retrieval of clusters’ time-evolution
(corrected) volume data (LE). characteristic profiles, provided that the number of asso- Subsequently, a preliminary data sorting procedure is ciated (linked) clusters between image frames is above applied to all individual data points constituting this newly a threshold minimum (Rth). Otherwise, the inference sys- created volume data. The procedure categorizes these data tem for target cluster identification is not applied to these into two reflectivity range groups: below-threshold and disjoint normal clusters. above-threshold reflectivity groups. Threshold reflectivity Retrievable dynamic characteristic profiles include
Zth is chosen between a 230- and 5-dBZ value range a cluster’s time-dependent altitude (the z component of (the default is 0 dBZ); typical chaff clutters exhibit re- the cluster centroid), its size expansion (the volume or flectivity within a 10- and 35-dBZ value range. This step projection area) profile, and the mean reflectivity pro- can dramatically speed up the overall tracking procedure file. The static characteristic profiles include the long- by limiting the clustering procedure only to data points ness factor (the degree to which a cluster looks long and in the selected group. The output of this sorting pro- thin) for a fixed time frame. cedure is a set of 3-tuple data points (x, y, z)thathas Regardless of whether a cluster being probed is a chaff reflectivity values above a threshold minimum. This cluster, it is then inferred based on these characteristic sparse set (B) and the 3D Euclidean reflectivity volume profiles. We note that among these retrievable profiles, data (LE) are then passed as inputs to the spatial clus- a cluster’s time-dependent altitude profile, from which a tering procedure. particle’s altitude-averaged fall velocity can be calcu- The spatial clustering procedure decomposes the sparse lated, is of particular importance. This is because a clus- set B (sparse in the sense that B is selected from larger ter’s structure can be directly inferred from this profile input dataset) into multiple subsetsorpartitionsbyadis- because of the correlation that exists between the size of tance proximity criteria. Spatial clustering used in this particles making up the cluster and the particles’ time- model does not require specifying the number of clusters averaged fall velocity (Mitchell 1996; Jiusto and Eadie a priori. Resulting partitions from this procedure form 1963; Matrosov et al. 2002). individual cluster sets containing spatially connected or This means that by observing a cluster’s vertical move- neighboring data points. Once the spatial clustering pro- ment profile alone, one can determine with reasonable cedure that is repeated for all time series images within confidence whether a cluster is chaff. Chaff clusters exhibit a specified inference time period is completed, in order gradually decreasing altitude levels, whereas most typical to further reduce the time taken by the overall tracking weather-related clusters maintain uniform or a somewhat algorithm, the size of each cluster (the cardinality of the irregularly fluctuating altitude level, provided that there is cluster set) formed from the previous step is computed. no rapid storm surge situation or other situations that re- Clusters only in selected size categories are chosen for the sult in significant vertical air movement. Once chaff clut- temporal clustering procedure, which associates identical or ters have been identified using the proposed inference similar clusters between time series images. The output of method, their coordinates undergo a transformation
Unauthenticated | Downloaded 10/08/21 04:20 AM UTC 878 JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY VOLUME 30
FIG. 5. 2D and 3D radar image decomposition for KSN 1130 KST 3 Jun 2010 data: (left) top view and (right) side angle view. (a) Full echoes. (b) Below-threshold reflectivity (,0dBZ) echoes removed. (c) Below-threshold re- flectivity (,0dBZ) echoes and below-threshold size (,190 grid points or 47.5 km3) clusters removed.
Unauthenticated | Downloaded 10/08/21 04:20 AM UTC MAY 2013 K I M E T A L . 879
FIG. 6. 2D top view of radar image decomposition for KSN 1230 KST 15 Jun 2010 data. (a) Full echoes. (b) Below- threshold reflectivity (,0dBZ) echoes removed. (c) Below-threshold reflectivity (,0dBZ) echoes and below threshold size (,190 grid points or 47.5 km3) clusters removed. (d) Target normal cluster being probed (later identified as chaff cluster).
procedure (back to polar coordinates) and are subse- 3. Spatial clustering of radar echo image elements quently removed from the original L reflectivity volume S a. Description of neighborhood-scan (NS) data to generate chaff-removed L data (L0 ). S S clustering method The proposed method has a limitation in its ability to detect chaff echoes at an early stage of their development Various clustering techniques are used in remote sens- because there are not enough data points from which key ing applications and pattern recognition problems to per- dynamic profiles can be extrapolated. The method also form image decomposition prior to a feature extraction has a limitation of when chaff clutter overlaps with other procedure. The clustering procedures essentially partition weather echoes, since the mixed presence of chaff clutter a finite number of objects into a finite number of groups, cannot be resolved or discriminated with CZ reflectivity so that any two objects belonging to the same group are information alone, as previously stated. Hence, chaff more similar than those belonging to different groups. This clutters have the best chance of being detected in clear- section describes the NS image data clustering method that sky conditions with no interfering amorphous weather immediately follows the step (in Fig. 3) that filters out the structures. above-threshold reflectivity data.
Unauthenticated | Downloaded 10/08/21 04:20 AM UTC 880 JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY VOLUME 30
FIG. 7. Example of EL clustering procedure with T 5 6 resulting in six isocluster sets. Here, T is the total number of inference time frames and NC is the number of the similarity comparisons performed at each step. Total number of similarity comparison performed in this example is 49. Starting from the last time frame t6, each normal cluster in this frame is compared to normal clusters in the im- mediately preceding time frame t5. Clusters with minimum dissimilarity are then linked together as a result of this comparison. Sub- sequently, starting from linked cluster in t5, the same comparison procedure is repeated with clusters in the immediately preceding time frame until a cluster is no longer able to find a matching cluster from the preceding time frame. Procedure then goes back with the remaining clusters in t6, and the whole procedure is repeated until there are no more clusters left with unassigned isocluster membership in all inference time frames.
The NS clustering method shown in Fig. 4 is loosely if it is insufficiently close to any of the existing clusters. based on the single-pass method or the leader method However, the proposed method diverges from the single- (Karkk€ ainen€ and Franti€ 2007), in that all elements (data pass method in its implementation and time complexity. points) are treated as individual clusters at the start and The single-pass and NS methods are of O(n2)andO(Mn) that at each iteration step, each element (data point) is time complexity, respectively, where M is the number of compared to clusters formed thus far, and is either added data points enclosed by a local neighborhood volume to the closest cluster or is used to start (lead) a new cluster space (simply termed hypersphere)tobescannedandn is
Unauthenticated | Downloaded 10/08/21 04:20 AM UTC MAY 2013 K I M E T A L . 881
FIG. 9. ANFIS for the determination of the degree of similarity between two temporally separated clusters. The zeroth layer of this NN represents two fuzzy input variables: d and h. First layer repre- sents linguistic values associated with each of these two input vari- ables (IF-part). Second and third layers represent rules and norm (AND-logic with attribution of weights for each rule). Fourth layer represents the THEN-part of the rules where each rule makes their own contribution to the overall output. Fifth and sixth layers repre- sent weight normalization of the output and the final output itself.
requirement imposed by the QC system (50 s or less for each incoming radar site data), the algorithm was run
FIG. 8. Two input fuzzy variables, d and h, in the antecedent with for 19 successive time frames with the simulation envi- their MFs to represent different degrees of the similarity between ronment settings and parameters shown in Table 1. The two temporally separated clusters. average clustering time per time frame was then calcu- lated by taking the ratio of the total clustering time the total number of input data points for which location taken by all frames to the number of frames. The result similarity comparisons are performed. is shown in Table 2. In NS clustering, each element (data point) in the input The overall data partition or decomposition of a 3D sparse set B, which points to a specific volume element in radar image that results from reflectivity range and size range filtering is illustrated in Figs. 5 and 6. LE, is checked once (with negligible processing time) for its cluster membership. If no cluster membership is pre- viously assigned to the element, then the element is used 4. Temporal clustering of radar echo image to start a new cluster. For each element successively elements added to the new cluster set, a neighborhood volume scan a. Description of evolution-linkage (EL) is performed to detect and to add more neighboring data clustering method points having reflectivity above a threshold minimum (i.e., .0dBZ), provided that these neighboring data points The EL clustering procedure is in essence an object- have not already been added to the cluster set (added or tracking procedure that associates similar clusters between flagged members are prevented from being added again). images to form an isocluster set, as defined in section 2b. Once no additional members are detected and added The tracking procedure is fundamentally based on the as- to the current cluster set, the process moves to the next sumption that it is substantially unlikely that a tracked ob- cluster and the same procedure is repeated until all ele- ject will undergo large displacement within the relatively ments (data points) in the input sparse set B are processed. smalltimewindowdefinedbytheimageframerate.The The key feature of this algorithm is that the neigh- EL clustering procedure, an example of which is shown borhood volume scan for the proximity check of above- threshold reflectivity data elements is limited to data points within the hypersphere and to data points that TABLE 3. Simulation time measurement of EL clustering. have not yet been flagged (assigned a membership). Total These key features dramatically improve the speed of clustering the clustering procedure. Radar sample date time (s) b. Simulation result of NS clustering KSN 3 Jun 2010 (1000–1300 KST at 10-min intervals 0.17 for a total of 19 inference time frames) To evaluate the performance of the NS clustering KSN 15 Jun 2010 (1000–1300 KST at 10-min intervals 0.23 method and to determine if it meets the execution time for a total of 19 inference time frames)
Unauthenticated | Downloaded 10/08/21 04:20 AM UTC 882 JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY VOLUME 30
FIG. 10. Time series images of selected target cluster for KSN 3 Jun 2010: (left) top view and (right) side angle view for (a) 1010, (b) 1040, (c) 1110, and (d) 1210 KST.
Unauthenticated | Downloaded 10/08/21 04:20 AM UTC MAY 2013 K I M E T A L . 883 in Fig. 7, uses the distance that separates clusters’ centroids TABLE 4. Notation used throughout section 5a for vertical and clusters’ size difference (either volume or the planar movement profile. projection area difference) as thebasecriterionforasimi- Symbol Description larity comparison. Other arbitrary features such as a mor- t Time (s) phological feature or mean reflectivity can be used as 21 yc Chaff fall velocity (m s ) 21 comparison criteria. However, for the simultaneous moni- yT Terminal chaff fall velocity (m s ) toring of many clusters at optimum frame rates, these fea- mc Chaff particle mass (kg) tures could be redundant. The EL clustering procedure has g Acceleration of a body within the earth’s 22 2 2 1 2 gravitational field (g 5 9.8 m s ) O[(T 1)(q q)/2] or simply O(q ) time complexity 23 rair Air density (kg m ) (considering the worst-case scenario, where the cluster- 23 rc Chaff density (kg m ) 2 tracking process goes all the way to the first frame), where fc Chaff cross-sectional area (m ) q is the average count of normal clusters within each time kc Chaff drag coefficient frame and T is the number of inference time frames. gR Reynolds number dc Chaff dipole diameter (m) The algorithm essentially makes linkage chains connect- 21 mair Dynamic viscosity of air (s kg m ) ing similar clusters (represented by dot-connected cells in P Pressure (Pa) Fig. 7) in adjacent time frames until there are no more T Temperature (K) 2 2 similarclusterslefttobeconnected (the clusters are tracked R Ideal gas constant (R 5 8.314 47 J mol 1 K 1) 5 21 as long as they do not undergo large displacement, size mair Molar mass of dry air (mair 0.028 964 4 kg mol ) 5 21 expansion, or shrinkage). Linkages are formed starting z Temperature lapse rate (z 0.0065 K m ) P Sea level standard atmospheric pressure from the last time frame and continuing in reverse time 0 (P0 5 101.325 kPa) order. At each linkage junction, a selected cluster at T0 Sea level standard temperature (T0 5 288.15 K) a particular time frame is compared with all clusters in ha Altitude (km) the immediately preceding time frame and the one that ap p/21 most closely matches is linked. This process is repeated until every cluster within the inference time period is The size difference between two clusters is de- assigned an isocluster membership. No two clusters termined by h 5 1 2 minfV1, V2g/maxfV1, V2g,where within a single time frame will have the same isocluster V1, V2 is the cluster’s respective volume (or planar membership. projection area). These two variables constitute fuzzy variables with their membership functions (MFs), as b. Similarity measure shown in Fig. 8.
Given two clusters C1,C2 in two adjacent time frames Based on these two fuzzy variables, the following two- and their respective centroids p1 and p2, the distance that input single-output Takagi–Sugeno–Kang (TSK) fuzzy separates these two centroids is given by d 5 kp1 2 p2k. model with four rules is constructed: