ERAD 2012 - THE SEVENTH EUROPEAN CONFERENCE ON RADAR IN METEOROLOGY AND HYDROLOGY

Multi-PRI and SMPRF software update for weather radars

Jörn Sierwald Eigenor Corporation, Lompolontie 1, 99600 Sodankylä, , [email protected] (Dated: 28 May 2012)

Jörn Sierwald

1 Introduction The Eigenor WnD software is designed for Multi-PRI1. We have modified the C-band radar at the University of , Finland to transmit a pattern of three pulse intervals by programming the Sigmet RVP900™ processor. The existing processor retains all its standard capabilities and the Multi-PRI measurements use the IRIS task scheduling, antenna control and calibration so that WnD measurements can be scheduled alongside the normal tasks of the radar. The WnD software offers the standard products of normal dual-polarization radar but with the benefit of the extended velocity range of Multi-PRI. WnD estimates the autocorrelation function for each volume and calculate velocity distributions to separate precipitation from other objects such as birds or sea clutter.

2 Radar setup 2.1 Kumpula radar We have used the dual-pol C-band Radar built by Vaisala which is located on the campus of the in Kumpula. It is jointly operated by the University of Helsinki and the Finnish Meteorological Institute. The first Multi-PRI measurement (see chapter 3.5) has been done in May 2011 using a Sigmet RVP8 processor. The system has then been upgraded to a RVP900 processor. In the end of March 2012 the Eigenor WnD measurements were integrated into the IRIS scheduling system. In order to send Multi-PRI patterns, we have modified the software of the processor. The RVP900 is able to generate I/Q time series data which is then sent to another computer for processing. The data processing capabilities of the RVP900 after I/Q generation are not used while running in WnD mode. Apart from the modified sending and the transfer of time series data the system is not modified. Antenna control, calibration and scheduling work as usual. The radar retains all its normal capabilities after installing the Eigenor WnD software. WnD measurements and the standard tasks supplied by Vaisala can be mixed in the scheduler. 2.2 Multi-PRI timing Multi-PRI transmissions allow a radar to measure higher unambiguous velocitiesa. The timing for the transmissions must be precise, as WnD requires that the intervals between pulses have a large common divisor which itself must be a multiple of the bin length. This requirement is caused by the SMPRF system which resolves range ambiguities. For example, the Kumpula radar was supposed to use a range resolution of one microsecond (just under 150 meters), a factor of 250 and an interval ratio of 7:8:10. The intervals thus are 1×250×7, 1×250×8 and 1×250×10 microseconds, in other words 1750 µs, 2000 µs and 2500 µs. As the RVP900 did not allow a configuration in terms of microseconds, a resolution of 150 m is used instead. Because 150 m are slightly longer than one µs the actual timings used are 1751.2 µs, 2001.38 µs and 2501.73 µs. This document uses the rounded values for better readability. The resulting unambiguous velocity is the velocity of an ordinary Doppler radar using a PRI of 250 µs. For the Kumpula installation with a frequency of 5610 MHz the unambiguous velocity range is from -53.5 m/s to +53.5 m/s. Using the ordinary PRF of 580 Hz the radar has an unambiguous velocity range from -7.7 m/s to +7.7 m/s.

3 Software implementation details 3.1 Data input and output The angular resolution of the radar is usually the same as the width of the main lobe of the antenna, in this case 1 degree. The software handles eight “beams” (one beam being the data received during one degree of a revolution of the antenna) in parallel using vectorization. For dual polarization radars it handles four beams in parallel. In a real-time setup the software will wait for enough data to start processing. When real-time processing is not a requirement, WnD reads data for entire revolutions of the antenna at once. Data is to be stored as HDF5, pending a specification for our non-standard products.

1 Multi-PRI – Sending pulses using more than one fixed interval between pulses. For instance intervals of 1000 µs, 1200 µs and 1400 µs and then starting over ERAD 2012 - THE SEVENTH EUROPEAN CONFERENCE ON RADAR IN METEOROLOGY AND HYDROLOGY

3.2 Dual polarization WnD support single and dual polarization radars. For the pulses, WnD require a short repeating pattern. A pattern of n times a short PRI followed by m times a long PRI is not supported. The trivial case of only one PRI is supported. For dual polarization, the radar is expected to send the same output signal over both feeds (single transmitter). 3.3 Calibration Calibration is handled by the processor software. Calibration data is saved with the time series data. 3.4 Ground clutter removal As the first step of processing WnD perform a polynomial fit. The data for one degree of rotation at each range is fitted with a polynomial of a low degree. This is then subtracted from the original signal. The result is a high-pass filter, filtering out ground clutter which changes slowly as the radar turns. As the filter cannot distinguish between ground clutter and useful signal such as rain at speed zero, both the filtered signal and the original are passed on to further processing. Later stages then choose which signal is used. 3.5 SMPRF and data selection The published SMPRF methodb,c,d resolves range ambiguities by solving a set of linear equations. Eigenor has tested this method and has implemented an alternative which selectively discards data which is likely to contain ambiguities.

Ranges for 250 µs×[7;8;10]

Pulse 0 262,5 112,5

Pulse 1 112,5 187,5 75

Pulse 2 75 300

0 50 100 150 200 250 300 350 400 Range in km

Mixed with previous echo Unambiguous Mixed with next echo

Figure 1. Usable range for Eigenor WnD standard timing. Figure 1 shows the overlapping echoes of the standard timing. WnD assumes that there are no echoes from beyond 375 km. The red and blue areas are derived from the same I/Q samples, but are different after the random phase processing (WnD sends pulses with random phases). The data selection method works as follows: The green “Unambiguous” range marks data that is free of ambiguities. The blue “Mixed with previous echo” range marks data that has echoes from short ranges mixed with echoes from high ranges. As the echo from beyond 260 km is very likely to be small compared to the low range part WnD use the data from the blue areas in calculations. The red “Mixed with next echo” areas however are discarded. 3.6 ACF estimation WnD calculates an estimate of the autocorrelation function for each volume. The data points of the ACF are not equally spaced. For our standard timing the data points are at 0, 1750, 2000, 2500, 3750, 4250, 4500, 6250, 8000, 8250, 8750, 10000, 10500, 10750, 12500, 14250, 14500, 15000, 16250, 16750, 17000 and 18750 microseconds. ACFs are calculated for both polarizations and across polarizations, with and without ground clutter. 3.7 ACF inversion Calculating a velocity distribution from the ACFs is not straightforward because the data points are not equally spaced. Furthermore, calculating a high-resolution velocity distribution is counterproductive. When calculating a distribution with a range of ±50 m/s and a resolution of 0.1 m/s the vast majority of the distribution will be noise or nothing at all. Precipitation will be spread out over many bins of the histogram. This is pleasant looking to the human eye but classification algorithms will have to combine the data again to have just one object of precipitation. The algorithm chooses a number of model functions and tries to find a linear combination that match the estimated ACF. An example: Choose a uniform grid of 1000 speeds from -50 m/s to +50 m/s. Each speed has a corresponding ACF, which is of the form . Using the method of simulated annealing, calculate a likely linear combination of these model functions which matches the ACF we have measured. The result is a histogram of speeds that can be displayed easily. Rather than trying to work with the histogram of 1000 data points Eigenor has built the data reduction into the simulated annealing. In addition to a fine grid of speeds like the one explained above WnD introduces a range of model functions that ERAD 2012 - THE SEVENTH EUROPEAN CONFERENCE ON RADAR IN METEOROLOGY AND HYDROLOGY have a “width” in the velocity domain. The algorithm will favor model functions with a large width over combinations of functions with a small width. The algorithm will also favor solutions that have a low number of elements. 3.8 ACF inversion using entire beams For performance reasons the Eigenor WnD software processes packages of four beams, each having the width of the main lobe of the antenna. The objects found during ACF inversion can extend over many volumes. As a rule of thumb, objects with a large width in the velocity domain such as rain have a high chance to be present in an adjacent volume. Birds, on the other hand, are unlikely to extend over several volumes. For this reason the ACF inversion of the volumes are not done one-at-a-time but in a pattern that allows detection of objects than span several volumes. Objects with a relatively large width already present in an adjacent volume are favored during the simulated annealing. Considering data from several volumes at once enhances the quality of object detectione. 3.9 Calculation of standard products WnD calculates a set of standard products without making use the velocity distribution. These are calculated using established and published methods which make the measurement comparable with other software. Reflectivity is calculated by subtracting the background noise level from the power measured, both with and without the ground clutter removed. The signal to noise ratio is calculated from the reflectivity and the background noise level. Velocity and Gaussian width are calculated using a stripped down version of the simulated annealing which does just one Gaussian fit. Differential reflectivity Zdr is calculated from the ACFs of the two polarizations at time zero. Copolar correlation ρco uses the cross-polarization correlation function at time zero. 3.10 Calculation of phase shift

The calculation of specific differential phase Κdp does not use the raw signal but the velocity distribution. The distribution is split so that only parts with large width are used. If there are no objects with a large width, the part with the largest width is chosen. Dropping ground clutter, birds and other objects such as ships from the calculation improves the quality of the Κdp calculation. If there are several objects of large width in the distribution the program will create several Κdp figures. This feature will be released in Q3/2012 with the next version of Eigenor WnD. 3.11 Differential reflectivity

If there are several objects in the velocity distribution WnD calculates the Zdr and ρco for each one. 3.12 Classification WnD does not perform classification by itself. Its output however is designed to feed a classification system. Eigenor’s own classification module is used for internal testing but is not released. 3.13 Portability The Eigenor WnD software requires a stream of I/Q samples and metadata such as antenna position as input. The radar must be able to send Multi-PRI patterns with pulses having random phases. The phase of each pulse must be available as metadata or as part of the I/Q data. The WnD software is not limited to a certain receiver but it is designed to process generic I/Q input data. It is expected to work in other frequency bands and with flexible Multi-PRI timings. Eigenor has used it with data from the Luosto/Finland Gematronik C-band radar. It will be used in the MMEA 3.2 triple band weather radar currently under development and the Ridgeline Instrument X-band radar.

4 Results We have successfully handled the integration of Eigenor WnD in daily operation on a common weather radar system. We have also acquired time series data for clear weather, rain, snow, graupel, a snow storm, sea clutter, birds, ships and ground clutter. 4.1 Single polarization operation The ground clutter measured consists of houses and coasts of islands. There are no mountains nearby. The ground clutter removal by polynomial fitting was found to produce artifacts that depend on the degree of the polynomial and the speed of rotation. A degree of 5 and a speed of 10 degrees per second will add a signal of 30 Hz which will show up as a speeds of - 0.8 m/s and +0.8 m/s. The speed at which the artifacts appear is predictable and the effects can be suppressed in both time domain and in the velocity distribution. As the signal without polynomial fitting is also processed the power at +/- 0.8 m/s can be compared to the original signal to tell artifact from normal signal. Overall, our ground clutter removal works as effectively as Vaisala’s system for single-PRI. In both systems areas with very high ground clutter signals will be marked as invalid even in rain. Reflectivity calculation works as expected. The data beyond 260 km is of questionable value, we have not found useful signal content during our measurements during the winter and spring of 2012. We’re looking forward to measuring thunderstorms later this year. ERAD 2012 - THE SEVENTH EUROPEAN CONFERENCE ON RADAR IN METEOROLOGY AND HYDROLOGY

The results of Gaussian speed and width calculation are different from standard single-PRF mode. Rather than computing a mean velocity, our system displays the dominant peak. The Gaussian width differs from single-PRF result as well, as the value is computed from the absolute value of the entire ACF. The single polarization processing is mature and ready for use. 4.2 Effectiveness of SMPRF With existing C-Band radar installations the SMPRF method is rarely useful. Using an ordinary PRF of 580 Hz the maximum unambiguous range is 260 km. At maximum range and using an elevation of 0.5 degrees this measures altitudes from 4 km to 8 km at once and measuring even higher ranges makes limited sense. The output power of the radars is also adequate. In other words, existing C-band radars do not have a range ambiguity problem to begin with. The problem is rather the low unambiguous velocity of 7 to 8 m/s. This is solved by Multi-PRI, dual-PRF or dual-PRT. When using the 1750/2000/2500 µs timing, Eigenor WnD extends the unambiguous range for existing C-band radars to 375 km but it cannot calculate velocity distributions beyond 260 km. The products at very high ranges are limited to reflectivity on both polarizations. This restriction applies to both SMPRF range unfolding and the simple data selection method. The reason is the ground clutter of the city of Helsinki which is far more powerful than the signal from 300 km away. The blue areas from Figure 1 can be used even though they may contain overlapping echoes. In the unusual case of an object at 290 km with high reflectivity the object will show up at range 27.5 km in the echo “Pulse 1”. Since the three measurements of pulses 0, 1 and 2 at 27.5 km are added up using the variance as a weight, the relatively high variance of pulse 1 will prevent the object from falsifying the reflectivity measurement. The red “Mixed with next echo” areas however are unusable. They contain ground clutter and high power echoes from close ranges. The data is discarded. The reflectivity at very high ranges is calculated using only the echo of pulse 2. For radars other than powerful C-band weather radars the SMPRF range unfolding can be useful. An X-band magnetron radar with limited power is more likely to have range ambiguity problems and range unfolding can be used here. Solid state radars using pulse compression on the other hand have a problem with SMPRF, as the receiver is blanked during the long transmission periods. High duty cycles cause large areas of invalid data and this has to be countered by a longer Multi-PRI pattern. If the invalid areas do not overlap there is only one echo unavailable at any given range, therefore the pattern needs one more pulse to compensate. Generally, the Multi-PRI should not be used to lower the PRI if ground clutter is expected. If the designed range of a weather radar is 50 km, the smallest PRI should be no less than 333 µs. WnD will then filter out echoes from beyond 50 km. 4.3 Autocorrelation functions The useful length of the ACF depends on the wavelength of the radar and the objects measured. For a C-band weather radar a length of 20 ms is sufficient. Solid objects such as ships will return signals that are coherent for longer but the emphasis is on precipitation. 50 Autocorrelation, mean velocity -6.4 m/s

40

30

20

10 Abs Real 0 Imag 0 µs 2500 µs 5000 µs 7500 µs 10000 µs 12500 µs 15000 µs 17500 µs 20000 µs -10

-20

-30

-40

Figure 2. Example autocorrelation Figure 2 shows an example of a measured ACF. Notice the gap between 0 and 1750 µs, where the imaginary part should have sine form, but no data is available. The spacing of the data points is best visible on the plot of the absolute value. ERAD 2012 - THE SEVENTH EUROPEAN CONFERENCE ON RADAR IN METEOROLOGY AND HYDROLOGY

Possible range ambiguities are avoided by the random phase processing, except for the value at time zero which is handled by data selection or by the SMPRF method “zerolag inversion”b. 4.4 Velocity distributions The simplest set for unit testing and display is a set of 256 model functions representing a uniform grid of velocities. The results are encouraging and we have collected data of various signal mixtures, such as bird plus rain, rain plus sea clutter, ships plus rain, layers of precipitation and more. The results show sidebands, for instance at ±12.8 m/s but these do not affect processing as this type of velocity distribution is mostly for display and debugging. See Chapter 4.6 for a sample. Full velocity distributions cannot be calculated with low signal to noise ratios. While the stripped-down version can determine the dominant speed at low ratios, for instance down to -4 dB in SNR when integrating over 100 measurements, the distribution looks very noisy (not surprising when a large portion of the signal is actually noise). In such cases the algorithm will restrict itself to only one Gaussian fit. The sum of the widths of the objects in the velocity distribution has an upper limit. The base system using single-PRF has a velocity range of ±7.7 m/s, an interval with a total width of 15 m/s. Even though the unambiguous velocity range of the Kumpula installation covers 107 m/s using WnD, the velocity distribution cannot be calculated when the sum of widths approaches 15 m/s. This is an inherent limitation of Multi-PRI setups. A snowstorm over Helsinki on 2012-02-28 managed to exceed the limit in some volumes with layers moving at different velocities. A practical limit seems to be 10 m/s, beyond this the distribution shows white noise. The minimum PRI has to be lowered in order to measure volumes having a very wide distribution of speeds. When using the data reduction, the typical output of the ACF inversion is a combination of one to three areas of rain, zero to two areas of ground clutter, sea clutter and not more than four objects of narrow width such as ships or birds. The level of detail in the velocity distribution can be increased by using a more complex Multi-PRI pattern and a slower rotational speed of the radar but radar operators generally favor higher scanning speeds. 4.5 Dual polarization operation Differential reflectivity works as expected. Development for calculating phase products (as explained in Chapter 3.10) is not finished at the time of writing but will be ready in Q3/2012. 4.6 Sample view

Figure 3. Velocity PPI Figure 3 shows a PPI plot of velocity. Elevation is 0.5 degrees. The black line in the main screen is about 110 km away from the radar which is located in Helsinki. East of Helsinki is a large area with a pattern of yellow and orange. In this area the radar measures two layers at once. The layers move at different speeds, so one speed or the other is displayed as dominant, resulting in a pattern. The display on the left side shows the velocity distribution along the black line on the main screen. The background color is an indication of reflectivity along the black line. The two distinct speeds are clearly ERAD 2012 - THE SEVENTH EUROPEAN CONFERENCE ON RADAR IN METEOROLOGY AND HYDROLOGY distinguishable, one layer at about 10 m/s the other at 15 m/s. Dual polarization products can be calculated for both layers. This is particularly useful for the lower one of the two layers as the radar antenna is already at minimum elevation.

4.7 Benefits Eigenor WnD offers algorithms and methods otherwise only found in universities - fully optimized for real-time operation with ordinary commercial weather radars. Compared to standard signal processing WnD offers  Higher unambiguous velocities compared to base installation  Ground clutter removal for Multi-PRI  The calculation of velocity distributions allow separation of objects otherwise not possible  Separate parameters for each object found per volume  All parameters are derived from the same measurement. There is no need for separate tasks to measure velocities and reflectivity.

Acknowledgment Eigenor would like to thank the Finnish Meteorological Institute, Helsinki University, Sodankylä Geophysical Observatory and Vaisala Corporation for their support. Eigenor WnD is partially based on scientific achievements of the MMEA 3.4 program (http://www.greennetfinland.fi/en/index.php/Main_Page) a Cho J. Y. N., 2005: Multi-PRI Signal Processing for the Terminal Doppler Weather Radar. Part II: Range–Velocity Ambiguity Mitigation. Journal of atmospheric and oceanic technology, 22, 1507-1519 b Pirttilä J., Lehtinen M. S., Huuskonen A., Markkanen M., 2005: A Proposed Solution to the Range–Doppler Dilemma of Weather Radar Measurements by Using the SMPRF Codes, Practical Results, and a Comparison with Operational Measurements. Journal of Applied Meteorology, 44, 1375-1390 c Lehtinen, M., 1999: Method and system for measuring radar reflectivity and doppler shift by means of a pulse radar. World Intellectual Property Organization International Patent Publication Number WO 99/49332. d Ruzanski E., Hubbert J. C., Chandrasekar V., 2007: Evaluation of the Simultaneous Multiple Pulse Repetition Frequency Algorithm for Weather Radar. Journal of atmospheric and oceanic technology, 25, 1166-1181 e Moisseev D. N., Chandrasekar V., 2008: Polarimetric Spectral Filter for Adaptive Clutter and Noise Suppression. Journal of atmospheric and oceanic technology, 26, 215-228