remote sensing

Article Automatic Detection of Lightning Whistlers Observed by the Plasma Wave Experiment Onboard the Arase Satellite Using the OpenCV Library

Umar Ali Ahmad 1,2 , Yoshiya Kasahara 1,* , Shoya Matsuda 3 , Mitsunori Ozaki 1 and Yoshitaka Goto 1 1 Graduate School of Natural Science and Technology, Kanazawa University, Kakuma-machi, Kanazawa 920-1192, Japan 2 School of Electrical Engineering, Telkom University, Jl. Telekomunikasi No.1 Dayeuhkolot, Kab. Bandung 40257, Indonesia 3 Institute of Space and Astronautical Science, Japan Aerospace Exploration Agency (JAXA), Sagamihara, Kanagawa 252-5210, Japan * Correspondence: [email protected]



Received: 6 June 2019; Accepted: 27 July 2019; Published: 30 July 2019


Abstract: The automatic detection of shapes or patterns represented by signals captured from spacecraft data is essential to revealing interesting phenomena. A signal processing approach is generally used to extract useful information from observation data. In this paper, we propose an image analysis approach to process image datasets produced via plasma wave observations by the Arase satellite. The dataset consists of 31,380 PNG files generated from the dynamic power spectra of magnetic wave field data gathered from a one-year observation period from March 2017 to March 2018. We implemented an automatic detection system using image analysis to classify the various types of lightning whistlers according to the Arase whistler map. We successfully detected a large number of whistler traces induced by lightning strikes and recorded their corresponding times and frequencies. The various shapes of the lightning whistlers indicate different very-low-frequency propagations and provide important clues concerning the geospace electron density profile.

Keywords: whistler mode waves; automatic detection; lightning whistler; satellite; plasmasphere; image analysis

1. Introduction Geospace is the outer region of space surrounding the near Earth from 70 km above the surface to approximately 10 RE (RE is the radius of the Earth). Within the geospace, there is a special type of electromagnetic wave called a lightning whistler. Eckersley [1] noted that whistlers are observed after sferics, which are broadband electromagnetic emissions induced by lightning strikes. Storey [2] demonstrated that lightning strikes emit electromagnetic waves with different frequencies and that some portion of these waves leaks into the magnetosphere. Carpenter [3] introduced the concept of the plasmapause, or the magnetospheric plasma knee, which suggested a sudden change in the electron density decrease with increasing magnetic activity. This knee exists at all times in the magnetosphere. When a whistler wave, which is nominally below 30 kHz, propagates through the magnetosphere, it propagates with different group velocities depending on the frequency, that is, signals at lower frequencies arrive later those at higher frequencies. The study of whistlers has been enhanced over the years with the advantages of spacecraft data and computer modeling and has progressed significantly. Previous studies have shown that the existence of whistler waves plays an important role in energetic electron acceleration and loss processes

Remote Sens. 2019, 11, 1785; doi:10.3390/rs11151785 www.mdpi.com/journal/remotesensing Remote Sens. 2019, 11, 1785 2 of 15 in the radiation belt [4–10]. Such waves are also useful for derivations of the spatial distribution of the geospace electron density and lightning discharges because we can estimate these properties using the propagation characteristics of such a signal from its source point to observation point. In recent years, the number of studies of whistlers originated from lightning has increased gradually from both ground-station and satellite observations. Santolik et al. [11] analyzed data measured by the DEMETER satellite in the night-side region, which are closely related to the lightning activity detected by the METEORAGE lightning detection network. Fiser et al. [12] developed software for the automatic detection of fractional-hop whistlers in the very-low-frequency (VLF) spectrograms recorded by the ICE (Instrument Champ Electrique; Electric Field Instrument) experiment onboard the DEMETER satellite and compared their result to the EUCLID (European Cooperation for Lightning Detection, www.euclid.org) network. They studied the intensity distribution of lightning whistlers versus the location of the lightning and derived the mean whistler amplitude with the horizontal and vertical scales compared to the location of the lightning sources. Zheng et al. [13] analyzed the whistler waves observed by the Van Allen Probes (RBSP) compared to global lightning data from the World Wide Lightning Location Network (WWLLN) and successfully predicted that approximately 22.6% of the whistlers observed by the satellite correspond to possible source lightning in the actual WWLLN data. Zahlava et al. [14] analyzed the measurements performed by DEMETER and RBSP to investigate the longitudinal dependence of the whistler mode waves in the inner magnetosphere of the Earth; their result indicated a significant longitudinal dependence on the night side inside the plasmasphere. Bayupati et al. [15] analyzed the dispersion of lightning whistlers observed by the AKEBONO satellite along its trajectory and discussed the relationship between propagation times of lightning whistlers and the electron density profile along their propagation path. Their work showed that analyzing the dispersion trends of lightning whistlers is a powerful method to determine the global electron density profile in the plasmasphere. Oike et al. [16] analyzed the spatial distribution and temporal variation of the occurrence frequency of lightning whistlers detected by the AKEBONO satellite compared to the lightning activities derived from ground-based observations. Their work demonstrated that the occurrence of lightning whistlers strongly correlates with lightning activity as well as the electron density distribution around the Earth, especially in the ionosphere. The result from AKEBONO revealed that lightning whistlers are primarily observed only in the L-shell region below three, where the L-shell, or the L-value, is a set of magnetic field lines in a dipole model that crosses the magnetic equator of the Earth at several RE equal to the L-value. Due to the limitation of the altitude coverage of AKEBONO; however, the spatial distribution of lightning whistlers in higher altitude regions has not been thoroughly examined. In addition, note that the detection method introduced by Bayupati et al. [15] and Oike et al. [16] can only cover the most popular type of whistlers because they only took into account a spectral shape that has the ideal dispersion of frequency and time for a lightning whistler. However, there are various types of lightning whistlers. Helliwell [17] examined the various types of lightning whistler spectra observed by ground-based observatories and classified them into nine types. The classification results were summarized in an atlas of whistler spectra [17]. In previous studies, the propagation characteristics of lightning whistlers have been intensively studied and, accordingly, lightning whistlers are recognized as important phenomena for monitoring the geospace environment. For example, they could be used as a powerful tool to remotely sense the electron density profile, if we can identify the source location of lightning flashes using the World Wide Lightning Location Network (WWLLN) and then determine the propagation time of the lightning whistlers from the source point to the observation point as a function of their frequency, because the propagation time can theoretically be derived as a function of the electron density along the propagation path [15]. The continuous measurement of lightning whistlers using spacecraft is very helpful for monitoring the spatial and temporal variation of the geospace electron density profile. In this paper, we developed a new detection application and classification method for lightning whistlers. An application was written in the C#.NET language using the OpenCV library for image processing (EMGU CV) and was applied to data obtained by the Arase satellite. Using the principles of Remote Sens. 2019, 11, 1785 3 of 15 image processing, pattern matching, and classification, we investigate the dispersion, cutoff frequency, and duration of the lightning whistlers. We define and classify the lightning whistlers according to our Arase whistler map. Compared to previous satellites such as DEMETER and AKEBONO, the altitude of Arase is much higher. The high-resolution waveform data measured by Arase provide wider area coverage in the inner magnetosphere than previous observations and allow the detection of various types of lightning whistlers.

2. Observations The ERG (exploration of energization and radiation in geospace) satellite, called Arase, was launched to explore the acceleration and loss mechanisms of relativistic electrons around the Earth during geospace storms [18]. Since Arase has an elliptical orbit with an initial apogee and perigee of ~32,000 km and 460 km, respectively, with an orbital inclination of 31◦, it covers a wide altitude range over a latitudinal region from the geomagnetic equator to the mid-latitude region. The PWE (plasma wave experiment) [19] instrument was developed to measure DC electric field and plasma waves, covered the frequency range for electric field from DC to 10 MHz, and a few Hz to 100 kHz for magnetic field. The PWE has several instruments such as a WPT (wire probe antenna), MSC (magnetic search coil), EFD (electric field detector), WFC (waveform capture), OFA (onboard frequency analyzer), and HFA (high frequency analyzer). WFC is a waveform receiver; which is one of the sub-systems of the PWE [20] that measures two components of the electric field and three components of the magnetic field. WFC has two modes: The “chorus mode” covers a frequency range below 20 kHz with a sampling rate at 65,536 samples/s and the “EMIC (electromagnetic ion cyclotron wave) mode” covers a frequency range below a few hundred Hertz with a sampling rate of 1024 samples/s. In this paper, we analyze the magnetic field data using the chorus mode. We investigated lightning whistlers measured by WFC using the datasets of the magnetic field component. To generate a plot showing the variation of the frequency with time, we calculated the absolute value using the three components of the magnetic field waveforms. q B = B2 + B2 + B2 (1) | | α β γ where Bγ is a spin axis component of a magnetic field vector and Bα and Bβ are spin-plane components [21]. Whistlers are prominent in VLF electromagnetic energy bursts produced by frequent lightning discharges. Figure1 shows a spectrogram, or the dynamic power spectra, of a whistler observed by the AKEBONO satellite [15]. Dynamic power spectra or spectrograms are visual representations of the spectrum generated by performing a band-pass filter and a Fourier transform. Whistler observations using spectrograms can help us understand the phenomena known as dispersion. Whistler dispersion depends on the refractive index, which is a function of the frequency, and therefore the signal propagates at different velocities at different frequencies. The propagation of whistlers along the geomagnetic field lines from the southern to northern hemispheres, and vice versa, is shown in Figure2. The dispersion of a whistler indicates a gap in the propagation delay in the frequency domain. Lightning whistlers can propagate thousands of kilometers from the sources of the lightning strikes in the plasmasphere of the Earth, and their spectral properties primarily depend on the plasma environment (electron density and ambient magnetic field) along their propagation paths. These data are relevant to estimations of the electron density profile. Remote Sens. 2019, 11, x FOR PEER REVIEW 3 of 15

97 processing (EMGU CV) and was applied to data obtained by the Arase satellite. Using the principles 98 of image processing, pattern matching, and classification, we investigate the dispersion, cutoff 99 frequency, and duration of the lightning whistlers. We define and classify the lightning whistlers 100 according to our Arase whistler map. Compared to previous satellites such as DEMETER and 101 AKEBONO, the altitude of Arase is much higher. The high-resolution waveform data measured by 102 Arase provide wider area coverage in the inner magnetosphere than previous observations and allow 103 the detection of various types of lightning whistlers.

104 2. Observations 105 The ERG (exploration of energization and radiation in geospace) satellite, called Arase, was 106 launched to explore the acceleration and loss mechanisms of relativistic electrons around the Earth 107 during geospace storms [18]. Since Arase has an elliptical orbit with an initial apogee and perigee of 108 ~32,000 km and 460 km, respectively, with an orbital inclination of 31°, it covers a wide altitude range 109 over a latitudinal region from the geomagnetic equator to the mid-latitude region. 110 The PWE (plasma wave experiment) [19] instrument was developed to measure DC electric field 111 and plasma waves, covered the frequency range for electric field from DC to 10 MHz, and a few Hz 112 to 100 kHz for magnetic field. The PWE has several instruments such as a WPT (wire probe antenna), 113 MSC (magnetic search coil), EFD (electric field detector), WFC (waveform capture), OFA (onboard 114 frequency analyzer), and HFA (high frequency analyzer). 115 WFC is a waveform receiver; which is one of the sub-systems of the PWE [20] that measures two 116 components of the electric field and three components of the magnetic field. WFC has two modes: 117 The “chorus mode” covers a frequency range below 20 kHz with a sampling rate at 65,536 samples/s 118 and the “EMIC (electromagnetic ion cyclotron wave) mode” covers a frequency range below a few 119 hundred Hertz with a sampling rate of 1,024 samples/s. 120 In this paper, we analyze the magnetic field data using the chorus mode. We investigated 121 lightning whistlers measured by WFC using the datasets of the magnetic field component. To 122 generate a plot showing the variation of the frequency with time, we calculated the absolute value 123 using the three components of the magnetic field waveforms.

|𝐵| = 𝐵𝛼 +𝐵𝛽+𝐵𝛾 (1)

124 where Bγ is a spin axis component of a magnetic field vector and Bα and Bβ are spin-plane components 125 [21]. 126 Whistlers are prominent in VLF electromagnetic energy bursts produced by frequent lightning 127 discharges. Figure 1 shows a spectrogram, or the dynamic power spectra, of a whistler observed by Remote Sens. 2019, 11, 1785 4 of 15 128 the AKEBONO satellite [15]. Dynamic power spectra or spectrograms are visual representations of 129 the spectrum generated by performing a band-pass filter and a Fourier transform.

Remote Sens. 2019, 11, x FOR PEER REVIEW 4 of 15

134 Whistler observations using spectrograms can help us understand the phenomena known as 135 dispersion. Whistler dispersion depends on the refractive index, which is a function of the frequency, 136 and therefore the signal propagates at different velocities at different frequencies. The propagation 137 of whistlers along the geomagnetic field lines from the southern to northern hemispheres, and vice 138 versa, is shown in Figure 2. The dispersion of a whistler indicates a gap in the propagation delay in 139 the frequency domain. Lightning whistlers can propagate thousands of kilometers from the sources 140130 of the lightning strikes in the plasmasphere of the Earth, and their spectral properties primarily 141131 dependFigureFigure on 1. 1.theAn An exampleplasma example ofenvironmentof aa frequency–timefrequency–time (electron spectrum density of of a alightning lightningand ambient whistler whistler magnetic observed observed onfield) on21 21June along June 1997, 1997,their 142132 propagationfromfrom AKEBONO, AKEBONO, paths. These with with data the the marked markedare relevant lightninglightning to estimations whistler detected detected of the usingelectron using the the density1/1√/√f conversionf conversion profile. technique technique 133 (image(image source source [15 [15]).]).

143 Figure 2. Propagation path of a lightning whistler. 144 Figure 2. Propagation path of a lightning whistler. 2.1. Dispersion of a Lightning Whistler 145 2.1. TheDispersion relationship of a Lightning between Whistler the frequency and time of a lightning whistler is theoretically explained 146 by Eckersley’sThe relationship law [22], between which proves the frequency a simple relationshipand time of between a lightning the arrivalwhistler time is (theoreticallyt) of a lightning 147 whistlerexplained at aby frequency Eckersley’s (f )law as follows:[22], which proves a simple relationship between the arrival time (t) of 148 a lightning whistler at a frequency (f) as follows:t = D / √ f + t0 (2)

where D is a constant value called dispersiont of= D lightning /√f + t0 whistler; t is the arrival time at a frequency(2) f ; 149 andwheret0 is D the is a time constant when value the called lightning dispersion strikes. of Bayupati lightning [whistler;15] used t the is the simple arrival principle time at a of frequency Eckersley’s 150 lawf; and on t waveform0 is the time data when from the AKEBONO lightning strikes. to detect Bayupati lightning [15] used whistlers. the simple Theconventional principle of Eckersley’s technique for 151 thelaw automatic on waveform detection data from of lightning AKEBONO whistlers to detect is lig tohtning identify whistlers. straight The lines conventional in a diagram technique in which for the 152 dynamicthe automatic power detection spectra of are lightning converted whistlers into a diagramis to identify with straightt on the lines horizontal in a diagram axis andin which 1/√f theon the 153 verticaldynamic axis. power The spectra result of are this converted technique into is showna diagram in Figure with t1 on, where the horizontalt1 and t2 axisare theand arrival 1/√f on times the of 154 thevertical detected axis. lightning The result whistler of this technique at frequencies is shown of f1 inand Figuref2, respectively.1, where t1 and The t2 dispersionare the arrivalD oftimes lightning of 155 whistlerthe detected is determined lightning whistler by detecting at frequencies a straight of line f1 and and f2 deriving, respectively. its gradient The dispersion from Equation D of lightning (3). 156 whistler is determined by detecting a straight line and deriving its gradient from Equation (3). t t D = 2 − 1 (3) 𝑡1 −𝑡 1 𝐷= f2 f1 1√ − 1√ − (3) It can be seen that the spectrum of lightning√𝑓 whistler√𝑓 appears as a straight line, which is well marked by thick curves as shown in Figure1. Equations (2) and (3); however, only explain the simple 157 It can be seen that the spectrum of lightning whistler appears as a straight line, which is well relationship between frequency and time for a subset of lightning whistlers; we cannot detect other 158 marked by thick curves as shown in Figure 1. Equation (2) and (3); however, only explain the simple types of lightning whistlers. The other types of whistlers shown in the whistler atlas compiled by 159 relationship between frequency and time for a subset of lightning whistlers; we cannot detect other 160 types of lightning whistlers. The other types of whistlers shown in the whistler atlas compiled by 161 Helliwell do not satisfy this equation. Accordingly, it is necessary to develop a new method to detect 162 various types of lightning whistlers.

Remote Sens. 2019, 11, 1785 5 of 15

Helliwell do not satisfy this equation. Accordingly, it is necessary to develop a new method to detect various types of lightning whistlers.

2.2. Arase Whistler Map We proposed a definition of the type of lightning whistler observed by the Arase satellite and named it the “Arase whistler map”. This whistler map is partially adapted from the original definition of the whistler atlasRemote definedSens. 2019, 11 by, x FOR Helliwell PEER REVIEW [17]. The principal themes of the classification definition5 of 15 used Remote Sens. 2019, 11, x FOR PEER REVIEW 5 of 15 Remote Sens. 2019, 11, x FOR PEER REVIEW 5 of 15 in Helliwell’s whistlerRemote Sens. atlas2019, 11, were x FOR PEER based REVIEW on ground observation result. For example, the di5 ffoference 15 163 2.2.Remote Arase Sens. Whistler 2019, 11, x Map FOR PEER REVIEW 5 of 15 between163 one-hop 2.2. Arase and Whistler two-hop Map whistler is the amount of relative dispersion, which depends on 163 2.2. Arase Whistler Map 164163 2.2. AraseWe proposed Whistler Map a definition of the type of lightning whistler observed by the Arase satellite and the length164163 of propagation2.2. AraseWe proposed Whistler path. Mapa definition It was possibleof the type toof lightning distinguish whistler one-hop observed and by the two-hop Arase satellite whistlers and on 165164 namedWe it proposed the “Arase a definition whistler ofmap”. the type This of whistlerlightning map whistler is partially observed adapted by the Arasefrom satellitethe original and ground-observations165164 namedWe becauseit proposed the “Arase the a definition observation whistler ofmap”. the point type This of is whistlerlightning fixed. Suchmap whistler is a classificationpartially observed adapted by the is; Arase however,from satellitethe original not and highly 166165 definitionnamed it ofthe the “Arase whistler whistler atlas definedmap”. Thisby Helliwell whistler [17]. map The is principalpartially themesadapted of from the classificationthe original 166165 definitionnamed it ofthe the “Arase whistler whistler atlas definedmap”. Thisby Helliwell whistler [17]. map The is principalpartially themesadapted of from the classificationthe original relevant for167166 spacecraftdefinition observations usedof the in whistler Helliwell’s because atlas whistler defined the atlas by observation Helliwell were based [17]. point on The ground isprincipal always observation themes changing. result.of the Forclassification Particularly example, for 167166 definition usedof the in whistler Helliwell’s atlas whistler defined atlas by Helliwell were based [17]. on The ground principal observation themes result.of the Forclassification example, the case of168167 very thedefinition elliptical difference used orbit between in Helliwell’s such one-hop as the whistler and Arase two-hop atlas satellite, were whistler based dispersion ison the ground amount of observation lightning of relative result. whistlerdispersion, For example, drasticallywhich 168167 thedefinition difference used between in Helliwell’s one-hop whistler and two-hop atlas were whistler based ison the ground amount observation of relative result. dispersion, For example, which 169168 dependsthe difference on the between length one-hopof propagation and two-hop path. Itwhistler was possible is the amountto distinguish of relative one-hop dispersion, and two-hop which changes and169168 it isdependsthe impossible difference on the between tolength distinguish one-hopof propagation and one-hop two-hop path. and It whistler was two-hop possible is the unless amountto distinguish the of relative location one-hop dispersion, of and wave two-hop which source is 170169 whistlersdepends onon theground-observations length of propagation because path. the It observationwas possible point to distinguish is fixed. Such one-hop a classification and two-hop is; identified.170169 We classifiedwhistlersdepends onon the theground-observations lightninglength of propagation whistlers because path. based the It observationwas on possible the duration point to distinguish is fixed. of the Such one-hop whistler a classification and trace two-hop instead, is; 171170 however,whistlers onnot ground-observationshighly relevant for spacecraft because the obse observationrvations because point isthe fixed. observation Such a classificationpoint is always is; 171170 however,whistlers onnot ground-observationshighly relevant for spacecraft because the obse observationrvations because point isthe fixed. observation Such a classificationpoint is always is; because172171 the lengthchanging.however, of durationnotParticularly highly relevantreflects for the casefor the spacecraft of length very elliptical obse ofrvations propagation orbit suchbecause as paththethe Araseobservation from satellite, the point wavedispersion is always source of to 172171 changing.however, notParticularly highly relevant for the casefor spacecraft of very elliptical observations orbit suchbecause as thethe Araseobservation satellite, point dispersion is always of the observation173172 lightningchanging. point. whistler InParticularly addition, drastically for wethe changes case classified of andvery it theellipticalis impossible multiple orbit to such distinguish lightning as the Araseone-hop whistlers satellite, and two-hop according dispersion unless of to the 173172 lightningchanging. whistler Particularly drastically for the changes case of and very it ellipticalis impossible orbit to such distinguish as the Araseone-hop satellite, and two-hop dispersion unless of 174173 thelightning location whistler of wave drastically source is changes identified. and We it is classi impossiblefied the to lightning distinguish whistlers one-hop based and ontwo-hop the duration unless combination174173 ofthelightning the location durations whistler of waveof drastically source the traces, is changes identified. which and We it suggests is classi impossiblefied the the to lightning existence distinguish whistlers of one-hop multiple based and ontwo-hop paths. the duration unless 175174 ofthe the location whistler of wavetrace sourceinstead, is because identified. the We length classi offied duration the lightning reflects whistlersthe length based of propagation on the duration path In this175174 paper, ofthe the location we whistler develop of wavetrace an sourceinstead, improved is because identified. detection the We length classi of method,fied duration the lightning basedreflects whistlers onthe patternlength based of propagation or on shape the duration detection path 176175 fromof the the whistler wave tracesource instead, to the because observation the length point. of In duration addition, reflects we classified the length the of multiple propagation lightning path 176175 fromof the the whistler wave tracesource instead, to the because observation the length point. of In duration addition, reflects we classified the length the of multiple propagation lightning path using image177176 processingwhistlersfrom the accordingwave and source image to the to combinationthe analysis observation approaches, of the point. durati Inons addition, via of the standard traces, we classified which patterns suggests the multiple by the measuring existence lightning of the 177176 whistlersfrom the accordingwave source to the to combinationthe observation of the point. durati Inons addition, of the traces, we classified which suggests the multiple the existence lightning of pixel values,178177 linemultiplewhistlers groupings, paths. according and to the detected combination line of inthe comparisondurations of the to traces, the which shapes suggests in the the reference existence ofatlas. 178177 multiplewhistlers paths. according to the combination of the durations of the traces, which suggests the existence of 179178 multipleIn this paths. paper, we develop an improved detection method, based on pattern or shape detection The entire179178 process multiple runsIn this paths. concurrently, paper, we develop reading an improved all the detection images method, at the based same on time, pattern detecting or shape detection the whistler 180179 usingIn image this paper, processing we develop and image an improvedanalysis approaches, detection method, via standard based patterns on pattern by measuring or shape detection the pixel lines, and180179 giving using theIn image results. this paper, processing As we we develop and can image see an in improvedanalysis Table 1approaches, ,detection the Arase method, via whistler standard based patterns map on pattern shows by measuring or the shape spectral detection the pixel forms 181180 values,using image line groupings, processing and and detected image analysis line in comparisonapproaches, to via the standard shapes inpatterns the reference by measuring atlas. The the entire pixel 181180 values,using image line groupings, processing and and detected image analysis line in comparisonapproaches, to via the standard shapes inpatterns the reference by measuring atlas. The the entire pixel of the lightning182181 processvalues, whistlers lineruns groupings, concurrently, and distinguishes and readingdetected all line each the in images typecomparison ofat whistler.the to same the shapes time, We detecting in group the reference the whistler spectral atlas. Thelines, forms entire and into 182181 processvalues, lineruns groupings, concurrently, and readingdetected all line the in images comparison at the to same the shapes time, detecting in the reference the whistler atlas. Thelines, entire and single-trace183182 whistlers, givingprocess the whichruns results. concurrently, include As we can nose, reading see short,in allTable the middle, 1,images the Arase at and the whistler longsame time, whistlers, map detecting shows andthe the spectral multi-trace whistler forms lines, of whistlers, andthe 183182 givingprocess the runs results. concurrently, As we can reading see in allTable the 1,images the Arase at the whistler same time, map detecting shows the the spectral whistler forms lines, of andthe 184183 lightninggiving the whistlers results. As and we distinguishes can see in Table each 1, type the ofArase whistler. whistler We mapgroup shows the spectral the spectral forms forms into single- of the which include184183 thelightninggiving multiple-traces the whistlers results. As and we and distinguishes can multiple-combinations see in Table each 1, type the ofArase whistler. whistler types. We mapgroup This shows the grouping spectral the spectral forms helps forms into us single- formulateof the 185184 tracelightning whistlers, whistlers which and includedistinguishes nose, eachshort, type middle, of whistler. and long We whistlers, group the andspectral multi-trace forms into whistlers, single- the classification185184 tracelightning and whistlers, detection whistlers which and algorithm includedistinguishes nose, described short,each type middle, in of the whistler. and next long section.We whistlers, group the andspectral multi-trace forms into whistlers, single- 186185 whichtrace whistlers, include whichthe multiple-traces include nose, andshort, multiple-combinations middle, and long whistlers, types. andThis multi-trace grouping whistlers,helps us 186185 whichtrace whistlers, include whichthe multiple-traces include nose, andshort, multiple-combinations middle, and long whistlers, types. andThis multi-trace grouping whistlers,helps us 187186 formulatewhich include the classification the multiple-traces and detection and algorithmmultiple-combinations described in the types. next section.This grouping helps us 187186 formulatewhich include the classification the multiple-traces and detection and algorithmmultiple-combinations described in the types. next section.This grouping helps us Table187186 1. Araseformulatewhich whistler include the map:classification the Typemultiple-traces and and principaldetection and algorithm definitionsmultiple-combinations described of lightning in the types. next whistlers section.This partiallygrouping adaptedhelps us 187 formulate the classification and detection algorithm described in the next section. 188187 formulateTable the 1. Arase classification whistler and map: detection Type and algorithm principal described definitions in of the lightning next section. whistlers partially from188 the originalTable whistler 1. Arase atlas whistler by Helliwell map: Type [17 and]. principal definitions of lightning whistlers partially 189188 adaptedTable 1. Arasefrom the whistler original map: whistler Type andatlas principal by Helliwell definitions [17]. of lightning whistlers partially 189188 adaptedTable 1. Arasefrom the whistler original map: whistler Type andatlas principal by Helliwell definitions [17]. of lightning whistlers partially No189 TypeNo of Whistler adapted Type offrom Whistler the original whistler atlas Definition by DefinitionHelliwell [17]. Spectral FormForm 189 No adapted Type offrom Whistler the original whistler atlas by DefinitionHelliwell [17]. Spectral Form Single Trace SingleNo Trace Type of Whistler Definition Spectral Form SingleNo Trace Type of Whistler Definition Spectral Form SingleNo Trace Type of Whistler Definition Spectral Form Single Trace A whistler whose frequency-time curve exhibits Single Trace A whistler whose frequency-time curve exhibits A whistlerA whosewhistler frequency-time whose frequency-time curve exhibits curve both risingexhibits and 1 Nose1 Nose bothA whistler rising whoseand falling frequency-time branches. The curve delay exhibits is at a 1 Nose falling branches.bothA whistler rising The delaywhoseand falling is atfrequency-time a minimum branches. at The the curve nosedelay exhibits frequency is at a 1 Nose minimumboth rising at and the falling nose frequency branches. The delay is at a 1 Nose minimumboth rising at and the falling nose frequency branches. The delay is at a 1 Nose minimumboth rising at and the falling nose frequency branches. The delay is at a minimum at the nose frequency Short minimum at the nose frequency Short2 Short A whistler that has a duration of less than 1 s. 2 2 WhistlerShort A whistlerA thatwhistler has a durationthat has ofa duration less than 1of s. less than 1 s. Whistler2 WhistlerShort A whistler that has a duration of less than 1 s. 2 WhistlerShort A whistler that has a duration of less than 1 s. 2 Whistler A whistler that has a duration of less than 1 s. Whistler Middle 3 Middle A whistler that has a duration between 1 s and 2 s. Middle3 Middle A whistler that has a duration between 1 s and 2 s. 3 3 WhislerMiddleA whistlerA thatwhistler has a durationthat has betweena duration 1 s andbetween 2 s. 1 s and 2 s. Whisler3 WhislerMiddle A whistler that has a duration between 1 s and 2 s. 3 Whisler A whistler that has a duration between 1 s and 2 s. Whisler Long 4 Long A whistler that has a duration of more than 2 s. 4 WhistlerLong A whistler that has a duration of more than 2 s. Long4 WhistlerLong A whistler that has a duration of more than 2 s. 4 4 WhistlerLong A whistlerA thatwhistler has a durationthat has ofa duration more than of 2 s.more than 2 s. Whistler4 Whistler A whistler that has a duration of more than 2 s. Multi TraceWhistler Multi Trace Multi Trace Multi Trace Multi Trace Multi Trace A multiple whistlers consisting of multiple basic 5 Multiple-Traces A multiple whistlers consisting of multiple basic 5 Multiple-Traces middleA multiple whistlers whistlers with consisting the same oflength multiple of duration. basic 5 Multiple-TracesA multiple middleA whistlersmultiple whistlers consisting whistlers with of consisting multiplethe same basic oflength multiple middle of duration. whistlers basic 5 Multiple-Traces5 Multiple-Traces middleA multiple whistlers whistlers with consisting the same oflength multiple of duration. basic 5 Multiple-Traceswith the samemiddle length whistlers of duration. with the same length of duration. Amiddle multiple whistlers whistlers with consisting the same lengthof multiple of duration. basic A multiple whistlers consisting of multiple basic Multiple- A multiple whistlers consisting of multiple basic 6 Multiple- middleA multiple whistlers whistlers with consisting the different of multiple length basic of 6 CombinationsMultiple- middleA multiple whistlers whistlers with consisting the different of multiple length basic of 6 CombinationsMultiple- durations.middle whistlers with the different length of 6 CombinationsMultiple-A multiple durations.middle whistlers whistlers consisting with of multiple the basicdifferent middle length whistlers of 6 Multiple- Combinations6 Combinations middle whistlers with the different length of 190 Combinationswith the didurations.fferent length of durations. 190 durations. 190 durations. 190 190

Remote Sens. 2019, 11, 1785 6 of 15

2.3. TypicalRemote Shape Sens. 2019 of,Lightning 11, x FOR PEER Whistlers REVIEW Observed by Arase 6 of 15

191 There2.3. Typical are several Shape examplesof Lightning ofWhistlers lightning Observed whistlers by Arase that were detected in the one-year dataset from March 2017 to March 2018 observed by WFC onboard Arase. In this section, we introduce the typical 192 There are several examples of lightning whistlers that were detected in the one-year dataset from shapes193 March of lightning 2017 to whistlersMarch 2018 observed observed by WFC the Arase onboard satellite Arase. andIn this compare section, we them introduce to the the Arase typical whistler map;194 nose,shapes short, of lightning middle, whistlers long, observed multiple-traces, by the Aras ande satellite multiple-combinations and compare them to whistlerthe Arase measuredwhistler by Arase195 /WFCmap; arenose, shown short, inmiddle, Figure long,3. multiple-traces, and multiple-combinations whistler measured by 196 Arase/WFC are shown in Figure 3.

197 198 199 ( a)

200

201

202

203 (b)

204

205

206

207 (c)

208

209

210

211 (d)

212

213

214

215 (e)

216

217

218

219 (f)

220

221

Figure 3. (a) Nose whistler at 10:44:27 UT on 27 September 2017; (b) short whistler at 14:24:08 UT on 2

October 2017; (c) middle whistler at 02:24:18 UT on 14 September 2017; (d) long whistler at 02:26:22 UT on November 18, 2017; (e) multiple-traces at 08:39:45 UT on 7 September 2017; (f) multiple-combinations at 16:37:18 UT on 9 October 2017, observed by the PWE/WFC (plasma wave experiment/waveform capture). Remote Sens. 2019, 11, 1785 7 of 15

Since we did not detect the source location or the direction path of the lightning whistler, we classified the type based on only the shape or pattern representation of the lightning whistler inside the spectrogram. In the examples, as shown in Figure3, the spectra were generated with the same period of (8 s) of data, where the X-axis indicates time and the Y-axis shows the frequency of the spectral observation. In the case of a single-trace whistler, we distinguished the type via its duration, except for the nose whistler because it has a unique shape. Meanwhile, for the case of short, middle, and long whistlers, we defined the type by its duration, as described in Table1. In the case of multiple-traces and multiple-combinations whistlers, we checked the duration of each dispersion. When the whistler traces had the same duration, we called it a multiple-traces, whereas we called it a multiple-combinations when there were different durations. Since we only saw the whistler shape based on frequency and time points of detected whistlers, we compared the time delays of the whistlers relative to their pixel locations.

3. Flow of the Detection Process

3.1. Pre-Processing To generate dynamic power spectra of the observed waveforms, we implemented a short-time Fourier transform analysis with a 62.5 ms Hann window that included 4096 samples with a 94% overlap (shifting the window by 256 samples). The contour unit is an arbitrary unit/Hz (dB). To obtain a better detection result, we need a clear fine structure of the dynamic spectra of a lightning whistler with a high signal-to-noise ratio. The detailed performance of the magnetic search coil magnetic sensor can be found in Ozaki et al. [21]. We applied a combination of filter operations using Gaussian filter. This technique eliminates the noisy pattern from the original spectra. To obtain the smoothing effectively, Gaussian smooth (also known as Gaussian blur) was implemented using a standard deviation sigma value of σ = 1. ( ) 1 x2 + y2 Gσ = exp (4) 2πσ2 − 2σ2 To generate a clear fine structure from the edge detector, we applied a Laplacian operation to compute the second derivative of the image resulting from the Gaussian blur. Figure4 shows an example of the dynamic spectra of a magnetic field observed at 02:24 UT on 14 September 2017. The upper panel shows the dynamic spectra processed from the WFC data, and the lower panel shows the edge spectra after using Gaussian blur and Laplacian filtering to remove noise. As can be seen in the upper panel, the clear fine structure of the lightning whistler is characterized by a discrete tone that decreases in frequency with increasing time. The bottom panel shows the clear result of a whistler trace in the dynamic spectra after implementing the Gaussian blur and edge detection via the Laplacian filter. We then performed the above-mentioned pre-processing on the result of the dynamic power spectra and saved it to an image file (PNG format) that fit our monitor size resolution of 1920 1017 pixels. × The saved file has the same information as shown in Figure4. This file was then used as input for our detection system. The total number of spectral image data files produced during the observations is shown in Table2. Remote Sens. 2019, 11, 1785 8 of 15 Remote Sens. 2019, 11, x FOR PEER REVIEW 8 of 15

Remote Sens. 2019, 11, x FOR PEER REVIEW 8 of 15

263

264 FigureFigure 4. 4.( top(top)) The The original original middle middle whistler whistler dynamic dynamic spectra spectra observed observed at at 02:24:17 02:24:17 UT UT on on September September 14, 265 263 2017.14, 2017. (bottom (bottom)) Edge Edge spectra spectra after after using using a Gaussian a Gaussian blur blur with with a mask a mask (1,0.1) (1,0.1) and and Laplacian Laplacian filtering filtering to remove noise. 266 264 to removeFigure 4.noise. (top) The original middle whistler dynamic spectra observed at 02:24:17 UT on September 265 14, 2017. (bottom)Table Edge 2. Total spectra numbers after using of the a Gaussian spectra image blur with file a as mask a function (1,0.1) and of month.Laplacian filtering 267 266 to remove noise.Table 2. Total numbers of the spectra image file as a function of month. Year 2017 2018 Year 2017 2018 267 Month MarTable Apr 2. MayTotal numbers Jun of Jul the spectra Aug image Sep file Octas a function Nov of Decmonth. Jan Feb Mar Month Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar TotalYear 2017 2018 Total 2102 649 2088 3209 2319 2205 5027 3378 3021 1696 2153 2163 1370 MonthFiles 2102 Mar 649Apr 2088May 3209Jun 2319 Jul Aug2205 Sep5027 Oct3378 Nov3021 Dec1696 Jan2153 Feb 2163 Mar 1370 Files Total 2102 649 2088 3209 2319 2205 5027 3378 3021 1696 2153 2163 1370 3.2. OverviewFiles of the Detection System 268 4.3. Overview of the Detection System Figure5 shows the overall flow chart of our proposed whistler detection method. The quality of 269 268 4.3.Figure Overview 5 shows of the the Detection overall System flow chart of our proposed whistler detection method. The quality of the input image of the candidate area directly affects the accuracy of the target detection task. We have 270 269 the inputFigure image 5 shows of the the candidate overall flow area chart directly of our affe propctsosed the whistler accuracy detection of the method.target detection The quality task. of We two different spectra (original and pre-processed), as shown in Figure4. To extract information from the 271 270 havethe two input different image spectraof the candidate (original area and directly pre-processe affectsd), the as accuracy shown inof Figurethe target 4. Todetection extract task. information We input image, our detection system only uses the segmented input image from the pre-processed spectra. 272 271 fromhave the two input different image, spectra our (originaldetection and system pre-processe only usesd), as the shown segmented in Figure input4. To extract image information from the pre- This operation not only accelerates the speed of target detection but also improves the detection 273 272 processedfrom the spectra. input image,This operation our detection not only system accelerate only usess the the speed segmented of target input detection image butfrom also the improves pre- performance, because the information has a clear fine structure with the unwanted information 274 273 the processeddetection spectra. performance, This operation because not the only informatio acceleratesn the has speed a clear of target fine detectionstructure but with also the improves unwanted 275 274eliminated informationthe detection (e.g., eliminated theperformance, attached (e.g., noise).thebecause attached the informatio noise). n has a clear fine structure with the unwanted 275 information eliminated (e.g., the attached noise).

276 276 277 FigureFigure 5. The 5. The overall overall flow flow chart chart of of the the whistlerwhistler detectio detectionn system. system. The The process process from from left to left right to rightshows shows 277 278 segmentation,Figuresegmentation, 5. The extractionoverall extraction flow of chartof the the candidate ofcandidate the whistler area,area, thedetectiothe classification classificationn system. rule, rule,The and process and the the detection from detection left result. to result. right shows 278 segmentation, extraction of the candidate area, the classification rule, and the detection result.

Remote Sens. 2019, 11, 1785 9 of 15

Remote Sens. 2019, 11, x FOR PEER REVIEW 9 of 15 The segmentation target area indicates an area with an important detection where a lightning 279 whistlerThe is segmentation located. We define target a area partitioned indicates area an inside area with this area,an important then split detection it into grid where lines, a as lightning shown in 280 Figurewhistler6. Each is located. grid line We represents define a partitioned information area about inside the this time area, and then frequency. split it Theinto X-axisgrid lines, represents as shown the 281 timein Figure of the 6. observation Each grid line with represents 100-ms increments. information Theabout Y-axis the time represents and frequency. the frequency The X-axis on a logarithmicrepresents 282 scale.the time Then, ofwe the calculate observation the average with 100-ms of each increm pixel pointents. forThe every Y-axis represented represents grid the point.frequency We limited on a 283 thelogarithmic detection scale. frequency Then, we range calculate from the 1 kHz average to 20 of kHz. each Thepixel segmentation point for every result represented of the grid image point. grid 284 ofWe the limited target the area detection was produced frequency using range thefrom OpenCV 1 kHz to function20 kHz. The and segmentation the neighboring result pixel of the relation image 285 (N8).grid of Groups the target of pixels area was of the produced binary imageusing the with OpenCV similar function features (colors)and the neighboring are obtained, pixel in which relation the 286 whistler(N8). Groups region of is pixels 1 (non-white) of the binary and image the non-whistler with similar regionfeatures is (colors) 0 (white). are Subsequently,obtained, in which we usedthe 287 Bresenham’swhistler region line is algorithm 1 (non-white) [23] for and every the stepnon-whistler after segmentation, region is 0 from(white). line Subsequently, grouping, line we detector, used 288 andBresenham’s line counter. line Bresenham’s algorithm [23] algorithm for every is step a widely after usedsegmentation, high-speed from algorithm line grouping, that rasterizes line detector, straight 289 linesand/ circlesline counter./ellipses Bresenham’s using only addition algorithm and is subtraction a widely used operations high-speed of pure algorithm integers; descriptionthat rasterizes and 290 thestraight concept lines/circles/ellipses of this algorithm isusing available only in addition Reference and [23 ].subtraction operations of pure integers; 291 description and the concept of this algorithm is available in Reference [23].

292

293 FigureFigure 6. 6.The The segmentationsegmentation ofof an input image of a 4-s dura durationtion observation observation into into a a grid grid line line to to extract extract 294 thethe corresponding corresponding information. information. The process of line grouping starts from a neighboring pixel that is connected with the Bresenham 295 The process of line grouping starts from a neighboring pixel that is connected with the function; then the pixels are grouped as a line-group. Further, the area of the adjacent neighboring 296 Bresenham function; then the pixels are grouped as a line-group. Further, the area of the adjacent block is scanned with triangular scanning to locate the boundary. The checked block is eliminated 297 neighboring block is scanned with triangular scanning to locate the boundary. The checked block is 298 andeliminated then another and then block another is moved block to is thatmoved is already to that is marked already as marked a line-group. as a line-group. After the After line-group the line- is 299 connected,group is connected, we add labeling we add forlabeling each startingfor each andstarti endingng and point ending of thepoint line-group of the line-group associated associated with the 300 pixelwith positionthe pixel of position the grid of corresponding the grid corresponding to time on to the time X-axis on the and X-axis frequency and frequency on the Y-axis. on the Examples Y-axis. 301 ofExamples detected of lines detected are shown lines inare Figure shown7. in Then, Figure the 7. detected Then, the lines detected are analyzed lines are one analyzed by one one as whistler by one 302 traces:as whistler Single traces: lines Single that represent lines that a represent whistler. Therea whis aretler. two There types are oftwo whistler types of traces: whistler Single traces: traces Single and 303 multi-traces.traces and multi-traces. The system The marks system each marks starting each and starti endingng and point ending (x, y) ofpoint the whistler(x, y) of the trace, whistler compares trace, its 304 locationcompares based its location on the pixel based grid, on andthe pixel stores grid, the correspondingand stores the corresponding information. The information. X-axis (time) The indicates X-axis 305 the(time) duration indicates of the the lightning duration whistler,of the lightning and the whistler, Y-axis indicates and the itsY-axis frequency. indicates its frequency. According to the Arase whistler map, there is one type of whistler that is distinguished by a different shape than the others, the nose whistler. The others have the shape of a normal whistler and are either single-trace or multi-trace whistlers. The duration of lightning whistlers as well as their combination is important when deciding the type of lightning whistler propagating in geospace. We developed a classification rule and principle, in which the nose whistler is checked for before the other types are detected. The detection process starts with line-group counting. If no line-group exists, the event is identified as no-whistler. If a line-group exists, the next step is to determine if it complies with the nose whistler check criteria. If yes, then we mark it as a nose whistler. If we fail to detect a nose shape, then we check the number of line traces in the line-group process, we categorize it 306

307 Figure 7. The result of the detected line and grouped as line-group.

Remote Sens. 2019, 11, x FOR PEER REVIEW 9 of 15

279 The segmentation target area indicates an area with an important detection where a lightning 280 whistler is located. We define a partitioned area inside this area, then split it into grid lines, as shown 281 in Figure 6. Each grid line represents information about the time and frequency. The X-axis represents 282 the time of the observation with 100-ms increments. The Y-axis represents the frequency on a 283 logarithmic scale. Then, we calculate the average of each pixel point for every represented grid point. 284 We limited the detection frequency range from 1 kHz to 20 kHz. The segmentation result of the image 285 grid of the target area was produced using the OpenCV function and the neighboring pixel relation 286 (N8). Groups of pixels of the binary image with similar features (colors) are obtained, in which the 287 whistler region is 1 (non-white) and the non-whistler region is 0 (white). Subsequently, we used 288 Bresenham’s line algorithm [23] for every step after segmentation, from line grouping, line detector, 289 and line counter. Bresenham’s algorithm is a widely used high-speed algorithm that rasterizes 290 straight lines/circles/ellipses using only addition and subtraction operations of pure integers; 291 description and the concept of this algorithm is available in Reference [23].

292

293 Figure 6. The segmentation of an input image of a 4-s duration observation into a grid line to extract 294 the corresponding information.

295 The process of line grouping starts from a neighboring pixel that is connected with the 296 Bresenham function; then the pixels are grouped as a line-group. Further, the area of the adjacent 297 neighboring block is scanned with triangular scanning to locate the boundary. The checked block is 298 eliminated and then another block is moved to that is already marked as a line-group. After the line- 299 group is connected, we add labeling for each starting and ending point of the line-group associated 300 with the pixel position of the grid corresponding to time on the X-axis and frequency on the Y-axis. 301 RemoteExamples Sens. 2019of detected, 11, 1785 lines are shown in Figure 7. Then, the detected lines are analyzed one by10 one of 15 302 as whistler traces: Single lines that represent a whistler. There are two types of whistler traces: Single 303 astraces a single-trace and multi-traces. or multi-trace The system whistler, marks and each then starti weng check and ending the duration point (x, of eachy) of tracethe whistler line detected. trace, 304 comparesRemote Sens. its 2019 location, 11, x FOR based PEER on REVIEW the pixel grid, and stores the corresponding information. The X-axis10 of 15 The event will then be classified based on the rules that can be summarized as shown in Figure8. 305 (time) indicates the duration of the lightning whistler, and the Y-axis indicates its frequency. 308 According to the Arase whistler map, there is one type of whistler that is distinguished by a 309 different shape than the others, the nose whistler. The others have the shape of a normal whistler and 310 are either single-trace or multi-trace whistlers. The duration of lightning whistlers as well as their 311 combination is important when deciding the type of lightning whistler propagating in geospace. 312 We developed a classification rule and principle, in which the nose whistler is checked for before 313 the other types are detected. The detection process starts with line-group counting. If no line-group 314 exists, the event is identified as no-whistler. If a line-group exists, the next step is to determine if it 315 complies with the nose whistler check criteria. If yes, then we mark it as a nose whistler. If we fail to 316 detect a nose shape, then we check the number of line traces in the line-group process, we categorize 306317 it as a single-trace or multi-trace whistler, and then we check the duration of each trace line detected. 318 The event will then be classified based on the rules that can be summarized as shown in Figure 8. 307 Figure 7. TheThe result result of of the the detected detected lin linee and and grouped grouped as as line-group. line-group.

319 Figure 8. Decision tree for the all-whistler classifier. 320 Figure 8. Decision tree for the all-whistler classifier. To successfully detect a nose whistler, we define a criteria-matching region as shown in Figure9. 321 To successfully detect a nose whistler, we define a criteria-matching region as shown in Figure The explanation of this region is as follows. The black line is an example of a nose whistler existing in 322 9. The explanation of this region is as follows. The black line is an example of a nose whistler existing a diagram of the dynamic spectra. It has a starting point, which is marked as XsYs and an ending point 323 in a diagram of the dynamic spectra. It has a starting point, which is marked as XsYs and an ending marked XeYe. As a reference of the Cartesian axis, the orange line represents the Y-axis (frequency) and 324 point marked XeYe. As a reference of the Cartesian axis, the orange line represents the Y-axis the purple line represents the X-axis (time). The green region is located to the left of the Y-axis, and the 325 (frequency) and the purple line represents the X-axis (time). The green region is located to the left of 326 the Y-axis, and the blue region is located to the right of the Y-axis. These two regions will help us 327 determine whether the component of the targeted whistler trace/whistler line belongs to the criteria 328 matching a nose whistler.

Remote Sens. 2019, 11, 1785 11 of 15

blue region is located to the right of the Y-axis. These two regions will help us determine whether the Remote Sens. 2019, 11, x FOR PEER REVIEW 11 of 15 component of the targeted whistler trace/whistler line belongs to the criteria matching a nose whistler.

329 Figure 9. Nose-whistler detection criteria-matching region. 330 Figure 9. Nose-whistler detection criteria-matching region. We developed an algorithm to check for nose whistlers first; this algorithm is the first step before 331 we canWe successfully developed an detect algorithm all other to typescheck of for lightning nose whistlers whistler. first; The this pseudo algorithm code andis the detailed first step detection before 332 weof acan nose successfully whistler are detect described all other in Algorithm types of 1. lightning whistler. The pseudo code and detailed 333 detection of a nose whistler are described in Algorithm 1. 334 Algorithm 1 The Nose-whistler detection algorithm 335 Algorithm1 For each1: The line-group Nose-whistler in the image detection algorithm 336 1 For 2each Find line-group the maximum in frequency the image and time on the X-axis and Y-axis (XsYs) 3 var Ys = line_group.Min(s => s.point.Y); 337 2 Find4 varthe Xs maximum= line_group.Where(s frequency=> s.point.Yand time== onYs).Min(s the X-axis=> s.point.X); and Y-axis (XsYs) 338 3 var 5Ys Find = line_group.Min(s the lowest time point => on s.point.Y); the X-axis 339 4 var 6Xsvar = left_xline_group.Where(s= line_group.Min(s =>=> s.point.Ys.point.X); == Ys).Min(s => s.point.X); 7 Find the lowest frequency point on the Y-axis 340 5 Find8 varthe left_y lowest= line_group.Where(s time point on=> thes.point.X X-axis== left_x).Max(s => s.point.Y); 341 6 var 9left_x Compare = line_group.Min(s the pixel location region=> s.point.X); of the starting time 10 if (left_x >=Xs) 342 7 Find the lowest frequency point on the Y-axis 11 return false; 343 8 var 12left_y Compare = line_group.Where(s the pixel location region => s.point.X of the whistler == left_x).Max(s trace in the green => s.point.Y); region 344 9 Compare13 if ((left_x the <=pixelXs -location (1 * line_group.neighborInterval) region of the starting && left_y time - Ys > 14 line_group.neighborInterval + 1) == false) 345 10 if (left_x15 return >=Xs) false; 346 11 return16 Find false; the lowest frequency point of the start point of the whistler trace on the Y-axis 347 12 Compare17 var bottom_y the pixel= line_group.Max(s location region=> s.point.Y); of the whistler trace in the green region 18 Find the lowest frequency point of the last point of the whistler trace on the Y-axis 348 13 if ((left_x <= Xs - (1 * line_group.neighborInterval) && left_y - Ys > 19 var bottom_x_fr_bottom_y = line_group.Where(s => s.point.Y == bottom_y).Max(s => s.point.X); 349 14 line_group.neighborInterval20 Compare the pixel location + region 1) == offalse) thewhistler trace in the blue region 350 15 return21 if (bottom_x_fr_bottom_yfalse; >= Xs + (1 * line_group.neighborInterval)) == false) 22 return false; 351 16 Find23 Detectionthe lowest result frequency point of the start point of the whistler trace on the Y-axis 352 17 var24 bottom_ywhistler.type = line_group.Max(s= WhistlerModelType.Nose; => s.point.Y); 353 18 Find25 returnthe lowest true; frequency point of the last point of the whistler trace on the Y-axis

354 194. Experimentalvar bottom_x_fr_bottom_y Result and Discussion = line_group.Where(s => s.point.Y == bottom_y).Max(s => 355 s.point.X); To evaluate the performance of the proposed method, we tested a set of real images of dynamic 356 20spectra Compare produced the pixel by WFC. location The entireregion experiment of the whistler was performed trace in the on blue a Core-i7-4810 region MQ 2.8 GHz 357 21 if (bottom_x_fr_bottom_y >= Xs + (1 * line_group.neighborInterval)) == false) 358 22 return false; 359 23 Detection result 360 24 whistler.type = WhistlerModelType.Nose;

Remote Sens. 2019, 11, x FOR PEER REVIEW 12 of 15

Remote Sens. 2019, 11, x FOR PEER REVIEW 12 of 15 361 25 return true; 361 25 return true; 362 5. Experimental Result and Discussion

363362 5.Remote ExperimentalTo Sens. evaluate2019, 11 ,Resultthe 1785 performance and Discussion of the proposed method, we tested a set of real images of dynamic12 of 15 363364 spectraTo producedevaluate the by performanceWFC. The entire of the experiment proposed methwas performedod, we tested on aa setCore-i7-4810 of real images MQ of2.8 dynamic GHz (8 364365 spectraCPUs) with produced 16GB ofby RAM. WFC. Before The entire processing experiment the enti wasre dataset,performed we ontested a Core-i7-4810 the detection MQ algorithm 2.8 GHz for (8 (8 CPUs) with 16 GB of RAM. Before processing the entire dataset, we tested the detection algorithm 365366 CPUs)each type with of 16GB lightning of RAM. whistler, Before as processing can be seen the in enti Figurere dataset, 10 and we Figure tested 11. the The detection detected algorithm whistler for is for each type of lightning whistler, as can be seen in Figures 10 and 11. The detected whistler is traced 366367 eachtraced type by ofa box,lightning and thewhistler, result asof can each be detection seen in Figure is stored 10 and in aFigure text file 11. withThe detectedeach characteristic, whistler is by a box, and the result of each detection is stored in a text file with each characteristic, including 367368 tracedincluding by type,a box, start and time, the resultend time, of each start detection frequenc isy, storedend frequency, in a text andfile withtime consumedeach characteristic, for each type, start time, end time, start frequency, end frequency, and time consumed for each detection 368369 includingdetection process.type, start Some time, whistlers end time, were start not frequenc detectedy, due end to frequency, the density and between time consumed pixels. We for rely each on process. Some whistlers were not detected due to the density between pixels. We rely on the concept 369370 detectionthe concept process. of the Some neighboring whistlers pixel, were so not if detectedthe filtering due into the densitypre-processing between results pixels. in We a weaklyrely on of the neighboring pixel, so if the filtering in the pre-processing results in a weakly connected or 370371 theconnected concept or of un-connected the neighboring line, thispixel, will so affectif the ou filteringr detection in the system. pre-processing The event willresults not in be a detected weakly un-connected line, this will affect our detection system. The event will not be detected as a line because 371372 connectedas a line because or un-connected it is not a single line, thisconnected will affect line. ou Tor preventdetection this system. and improve The event the will detection not be accuracy, detected it is not a single connected line. To prevent this and improve the detection accuracy, it is necessary to 372373 asit isa linenecessary because to it developis not a single an intermediate connected line. proce Toss prevent between this spectral and improve filtering the and detection detection. accuracy, This develop an intermediate process between spectral filtering and detection. This process should make 373374 itprocess is necessary should to make develop adjustments an intermediate to maintain proce thess betweenpixel quality, spectral prevent filtering pixel and loss, detection. and refine This adjustments to maintain the pixel quality, prevent pixel loss, and refine connected lines to improve the 374375 processconnected should lines tomake improve adjustments the detection to maintain process andthe increasepixel quality, the accuracy. prevent pixel loss, and refine detection process and increase the accuracy. 375 connected lines to improve the detection process and increase the accuracy.

376

376377 Figure 10. The detected nose, short, middle, and long whistler types. Figure 10. The detected nose, short, middle, and long whistler types. 377 Figure 10. The detected nose, short, middle, and long whistler types.

378 378 Figure 11. The detected multiple-traces and multiple-combinations whistler types.

For comparison with the algorithm result, we used “manual inspection” that we made a year before we developed this application, that is, we conducted a direct inspection to identify all the Remote Sens. 2019, 11, 1785 13 of 15 available image spectra. In this observation, we visually inspected, one by one, every image spectra and investigated the detected lightning whistlers via the spectrogram. Since this type of observation depends solely on the experience of the investigator, the manual inspection took a long time and we checked every detail of each spectrogram and carefully marked the event and type using the principles defined in the Arase whistler map. In our detection policy for automatic detection, we prioritize highest accuracy of false-detection. False-detection means that shape of lightning whistler trace is inappropriately detected and thus differently classified because of the background noise. To decrease the false-detection rate, a higher threshold level is necessary, but it causes an increase in mis-detection rate. Here mis-detection means that weak lightning whistler cannot be detected due to faint and patchy line that fails line grouping by the Bresenham’s line algorithm. Using optimized parameters, we performed bulk processing of detection for one-year data. To use our proposed algorithm, we loaded all the images as input into the application, and the application processed each detection concurrently with the available CPU and memory. It took 30 h to completely detect all the whistlers in the available dataset. Some whistlers were not detected due to our detection policy. The result of the detection system as output from the application is shown in Table3. There are 711 image files in which we could detect lightning whistlers, while 1165 of line groups were identified among the 711 image files. We couldn’t detect any long whistlers when we apply the optimized parameter. In fact, we can detect a long whistler as shown in Figure 10, if we optimize the parameter only for this event, but it causes an increase of false detection rate for the other events due to contaminated noise. In future work, we need to solve this problem of how to reduce the noise or add some denoising technique in order to get better pre-processed spectra.

Table 3. The comparison detection result using the defined algorithm and manual inspections.

Automatic Lightning Whistler Detection: Output Result Total number of 1-Year 31,380 image file observation image spectra Spectra with whistler event 949 image file from manual inspection Spectra with whistler event 711 image file from automatic detection Mis-detection rate 25.07% Detection configuration: 1 kHz to 20 kHz frequency threshold Number of line-group detected 1165 Whistler type: single-trace Whistler type: multi-traces Number of detected whistler Nose Short Middle Long Multiple-traces Multiple-combinations 58 147 238 - 340 382

We assume that the manual inspections are more reliable than the automatic detection because the algorithm still has a limited capability and further work needs to be done to improve it, such as selecting different types of filtering to improve the quality of the input image in the pre-processing and to increase the detection accuracy. By detecting whistler using duration approached, our classification system has decreased the number of false classifications. False classification mean that the system classified the wrong type of lightning whistler within the same criteria. It most likely happens when the false line detected as lightning whistler. In Table3, we show that we could not classify the long whistler. Actually, we observed the long whistler in manual inspection, but all events to be categorized into long-whistler were mis-detected due to our amplitude threshold level. Remote Sens. 2019, 11, 1785 14 of 15

5. Conclusions In this paper, we presented the detection methodology for various types of lightning whistlers observed by WFC onboard the Arase satellite. The lightning whistler data were captured by the WFC magnetic field component during a one-year observation period from March 2017 to March 2018. Each footprint of the magnetic field during the observation period was converted into dynamic spectra and stored in an image file. We collected the image files and developed a detection system for the lightning whistlers captured inside the images. Using image processing and image analysis approaches, we developed a methodology and classification system to automatically detect lightning whistlers that is robust, fast, and informative. The detection results were very accurate, and six types of lightning whistler were successfully classified. The various types of lightning whistler detected by WFC onboard the Arase satellite will correspond to study of propagation characteristic of the signal of whistler waves depending on the region.

Author Contributions: U.A.A. developed the method for lightning whistler detection and wrote the whole of this paper. Y.K. is a principal investigator of the PWE and a supervisor of this research project. S.M. is a Co-I of the PWE responsible for the development of the onboard software and data management. M.O. is a Co-I of the PWE responsible for the development of magnetic search coil (MSC). Y.G. is a sub-supervisor of this research project responsible for the signal processing technique. All authors read and approved the final manuscript. Funding: We sincerely thanks to the Indonesia Endowment Fund for Education (LPDP) Ministry of Finance Indonesia for financial support and Doctoral Research Fund (LPDP No. 20160822018945), Ministry of Research, Technology and Higher Education Indonesia, and Telkom University for supporting fund. This research was partially supported by a Grant-in-Aid for Scientific Research from the Japan Society for the Promotion of Science (#16H04056, #16H01172 and #18H04441). Acknowledgments: PWE/WFC-spectrum (Level 2 provisional v04) data stored in the ERG Science Center were used to generate the frequency-time spectrograms of magnetic field components. The ERG Science Center is operated by ISEE/Nagoya University and ISAS/Japan Aerospace and Exploration Agency (JAXA). We thank to M Alfattah for his assistance on the machine learning and classification best practices. Conflicts of Interest: The authors declare no conflict of interest.

References

1. Eckersley, T.L. Musical Atmospherics. Nature 1935, 135, 104–105. [CrossRef] 2. Storey, L.R.O. An investigation of whistling atmospherics. Phil. Trans. Roy. Soc. 1953, 246, 113–141. [CrossRef] 3. Carpenter, D.L. Whistler evidence of a ‘knee’ in the magnetospheric ionization density profile. J. Geophys. Res. 1963, 68, 1675–1682. [CrossRef] 4. Abel, B.; Thorne, R.M. Electron scattering loss in Earth’s inner magnetosphere: 1. Dominant physical processes. J. Geophys. Res. 1998, 103, 2385–2396. [CrossRef] 5. Rostoker, G.; Skone, S.; Baker, D.N. On the origin of relativistic electons in the magnetosphere associated with some geomagnetic storms. Geophys. Res. Lett. 1998, 25, 3701. [CrossRef] 6. Elkington, S.R.; Hudson, M.K.; Chan, A.A. Acceleration of relativistic electrons via drift-resonant interaction with toroidal-mode Pc-5 ULF oscillations. Geophys. Res. Lett. 1999, 26, 3273. [CrossRef] 7. Lauben, D.S.; Inan, U.S.; Bell, T.F. Precipitation of radiation belt electrons induced by obliquely propagating lightning-generated whistlers. J. Geophys. Res. 2001, 106, 29745–29770. [CrossRef] 8. Rodger, C.J.; McCormick, R.J.; Clilverd, M.A. Testing the importance of precipitation loss mechanisms in the inner radiation belt. Geophys. Res. Lett. 2004, 31, L10803. [CrossRef] 9. Millan, R.; Thorne, R. Review of radiation belt relativistic electron losses. J. Atmos. Sol. Terr. Phys. 2007, 69, 362–377. [CrossRef] 10. Meredith, N.P.; Horne, R.B.; Glauert, S.A.; Baker, D.N.; Kanekal, S.G.; Albert, J.M. Relativistic electron loss timescales in the slot region. J. Geophys. Res. 2009, 114, A03222. [CrossRef] 11. Santolík, O.; Parrot, M.; Inan, U.; Burešová, D.; Gurnett, D.; Chum, J. Propagation of unducted whistlers from their source lightning: A case study. J. Geophys. Res. 2009, 114, A03212. [CrossRef] 12. Fiser, J.; Chum, J.; Diendorfer, G.; Parrot, M.; Santolik, O. Whistler intensities above thunderstorms. Ann. Geophys. 2010, 28, 37–46. [CrossRef] Remote Sens. 2019, 11, 1785 15 of 15

13. Zheng, H.; Holzworth, R.H.; Brundell, J.B.; Jacobson, A.R.; Wygant, J.R.; Hospodarsky, G.B.; Mozer, F.S.; Bonnell, J. A statistical study of whistler waves observed by Van Allen Probes (RBSP) and lightning detected by WWLLN. J. Geophys. Res. Space Phys. 2016, 121, 2067–2079. [CrossRef] 14. Záhlava, J.; Nˇemec,F.; Santolík, O.; Kolmašová, I.; Hospodarsky, G.B.; Parrot, M.; Kurth, W.S.; Bortnik, J.; Kletzing, C. Longitudinal dependence of whistler mode electromagnetic waves in the Earth’s inner magnetosphere. J. Geophys. Res. Space Phys. 2018, 123, 6562–6575. [CrossRef] 15. Bayupati, I.P.A.; Kasahara, Y.; Goto, Y. Study of Dispersion of Lightning Whistler observed by Akebono Satellite in the Earth’s Plasmasphere. IEICE Trans. Commun. 2012, E95-B, 3472–3479. [CrossRef] 16. Oike, Y.; Kasahara, Y.; Goto, Y. Spatial distribution and temporal variations of occurrence frequency of lightning whistlers observed by VLF/WBA onboard Akebono. Radio Sci. 2014, 49, 753–764. [CrossRef] 17. Helliwell, R.A. Whistlers and Related Ionospheric Phenomena; Stanford Univ. Press: Stanford, CA, USA, 1965. 18. Miyoshi, Y.; Shinohara, I.; Takashima, T.; Asamura, K.; Higashio, N.; Mitani, T.; Kasahara, S.; Yokota, S.; Kazama, Y.; Wang, S.Y.; et al. Geospace exploration project ERG. Earth Planets Space 2018, 70, 101. [CrossRef] 19. Kasahara, Y.; Kasaba, Y.; Kojima, H.; Yagitani, S.; Ishisaka, K.; Kumamoto, A.; Tsuchiya, F.; Ozaki, M.; Matsuda, S.; Imachi, T.; et al. The plasma wave experiment (PWE) on board the Arase (ERG) satellite. Earth Planets Space 2018, 70, 86. [CrossRef] 20. Matsuda, S.; Kasahara, Y.; Kojima, H.; Kasaba, Y.; Yagitani,S.; Ozaki, M.; Imachi, T.; Ishisaka, K.; Kumamoto, A.; Tsuchiya, F.; et al. Onboard software of plasma wave experiment aboard Arase: Instrument management and signal processing of waveform capture/onboard frequency analyzer. Earth Planets Space 2018, 70, 75. [CrossRef] 21. Ozaki, M.; Yagitani, S.; Kasahara, Y.; Kojima, H.; Kasaba, Y.; Kumamoto, A.; Tsuchiya, F.; Matsuda, S.; Matsuoka, A.; Sasaki, T.; et al. Magnetic search coil (MSC) of plasma wave experiment (PWE) aboard the Arase (ERG) satellite. Earth Planets Space 2018, 70, 76. [CrossRef] 22. Gurnett, D.A.; Kurth, W.S.; Cairns, I.H.; Granroth, L.J. Whistler in Neptune’s Magnetosphere: Evidence of Atmospheric Lightning. J. Geophys. Res. 1990, 95, 20967–20976. [CrossRef] 23. Bresenham, J.E. A linear algorithm for incremental digital display of circular arcs. Commun. ACM 1977, 20, 100–106. [CrossRef]

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).