Identification of Patterns in Cosmic-Ray Arrival Directions

Identiﬁcation of Patterns in Cosmic-Ray Arrival Directions using Dynamic Graph Convolutional Neural Networks

T. Bister, M. Erdmann, J. Glombitza, N. Langner, J. Schulte, M. Wirtz RWTH Aachen University, III. Physikalisches Institut A, Otto-Blumenthal-Str., 52056 Aachen, Germany

Abstract We present a new approach for the identification of ultra-high energy cosmic rays from sources using dynamic graph convolutional neural networks. These networks are designed to handle sparsely arranged objects and to exploit their short- and long-range correlations. Our method searches for patterns in the arrival directions of cosmic rays, which are expected to result from coherent deflections in cosmic magnetic fields. The network discriminates astrophysical scenarios with source signatures from those with only isotropically distributed cosmic rays and allows for the identification of cosmic rays that belong to a deflection pattern. We use simulated astrophysical scenarios where the source density is the only free parameter to show how density limits can be derived. We apply this method to a public data set from the AGASA Observatory. Keywords: ultra-high energy cosmic rays, sources, magnetic fields, neural networks

1. Introduction of particles with energies above 39 EeV with star- burst galaxies exhibits an increasing significance, The quest for sources of ultra-high energy cosmic currently at the 4.5σ level [11, 12]. rays has entered a new phase. On the one hand, For nuclei with charge Z, deflections in cosmic the experimental results of recent years have sharp- magnetic fields of several degrees per charge unit ened the boundary conditions for this search; on the are estimated [13, 14, 15, 16, 17, 18, 19, 20], with other hand, current developments in data analysis the strongest deflection expected in the magnetic technologies are opening up new perspectives. field of our Galaxy [21, 22, 23, 24, 25]. The dif- The measurements relevant to source searches ferent rigidities of the nuclei may lead to patterns include first the sharp drop in the energy spec- in the arrival directions which can be used to dis- trum above 50 EeV [1, 2, 3, 4]. Second, measure- tinguish cosmic rays of a single source or of a con- ments of the slant depth of air showers in the at- catenated source cluster from isotropically arriving mosphere indicate a mixed composition of protons particles [26, 27]. and heavier nuclei [5, 6, 7]. Both these observa- In the field of data analysis, new technologies tions increase the probability of astrophysical sce- around machine learning are the driving force of narios in which the sources are some megaparsecs much lot of the progress. Deep-learning meth- away and accelerate nuclei to a maximum rigidity ods [28] in particular have gained considerable at- (=energy/charge) [8]. The third relevant observa- tention in astroparticle physics [29, 30, 31, 32, 33, tion is the discovery of a large-scale dipole signal 34, 35, 36, 37, 38, 39]. arXiv:2003.13038v1 [astro-ph.HE] 29 Mar 2020 above 8 EeV energy which shows - with a signifi- Recently, we introduced a fit method which con- cance of more than five standard deviations - that tracts patterns in the arrival directions induced by the arrival directions of cosmic rays are not en- the Galactic magnetic field to determine most prob- tirely isotropic [9, 10]. Also, there is an indication able extragalactic source directions [40]. In this of an intermediate-scale anisotropy: the correlation fit, several thousand parameters are determined si- multaneously (source directions, particle charges), Email address: [email protected] which was realized by using the backpropagation (M. Erdmann) method developed for neural network training [41].

Preprint submitted to Astroparticle Physics March 31, 2020 In this paper we present a new method for source Later, in an advanced simulation, we use an as- identification that directly uses pattern recognition trophysical scenario that depends only on one free in cosmic-ray arrival directions. A similar approach parameter, the density of sources. Here, all cos- using convolutional neural networks was developed mic rays originate from these sources, with only a in parallel [39]. Here, we use the concept of so-called few closer sources contributing several cosmic rays dynamic graph convolutional neural networks [42] which form a pattern in the arrival distributions. which have already been successfully adapted and This results in a few signal patterns distributed applied to challenges in particle physics [43, 44]. across the whole sky, each with particles from a In contrast to our fit method [40], here we do common source, while most particles appear to have not use an explicit model of the Galactic magnetic isotropic arrival directions. field, but instead train the network with typical Finally, we present the AGASA Observatory and deflection patterns resulting from these field mod- its public data set of ultra-high energy cosmic rays. els [45, 46, 47, 48, 49, 50]. The scope of our method is to search for the exis- 2.1. Simplified cosmic-ray scenario with a single tence of patterns in cosmic-ray arrival directions: if source such patterns are detected by the network, we clas- In order to keep the results unambigous and sify which individual particles originate from a com- clean, this first simulation setup is simplified yet mon source and which can be attributed to isotropic fairly in agreement with the measurements de- arrival directions. For this classification we analyze scribed in section 1. variables autonomously formed by the network. During the training process we generate up to If no pattern is found, we derive a lower limit 300.000 simulation sets on-the-fly with the following on the density of cosmic-ray sources using astro- general setup. The predominant part of all NCRs physical simulations with the source density as their events, called background, is positioned isotropi- only free parameter. We exploit public data of the cally on the sky. We then add a certain fraction AGASA Observatory to demonstrate the applica- fS = NS/NCRs of NS signal cosmic rays which tion to real cosmic-ray measurements. mimic a pattern arising from deflections in the This work is structured as follows. First, we Galactic magnetic field (GMF). Current models of present astrophysical benchmark simulations used the GMF consist of a superposition of a regular for training and performance estimation as well as field part which deflects particles coherently and the public dataset of the AGASA Observatory. We a turbulent field part which results in blurring and then introduce the functionality of dynamic graph widening of the incoming particle beam. The mag- convolutional neural networks. We describe the nitude of the magnetic field influence depends on search for the existence of anisotropic patterns, the the cosmic-ray rigidity subsequent identification of cosmic rays and the de- termination of source density limits. Finally, we E R = , (1) apply our methods to the data measured by the Z AGASA Observatory. We complete the work with our conclusions. the ratio of its energy and charge. To prevent over- training of the network which could learn specific patterns in fixed directions of the sky we do not 2. Cosmic rays: simulations and measure- use one fixed field parametrization. Instead, we ments use a random value for the orientation of the sig- In this section we first present the simulated data nal pattern and adopt the field magnitude of both we use for training the network and evaluating its regular and turbulent fields from [46, 48] in the fol- performance. lowing way: we account for the turbulent field by For an in-depth understanding of the network, we a Fisher [51] distribution with a rigidity-dependent start with simplified simulations in which a single width of source generates a few cosmic rays that exhibit typ- T σturb(R) = rad , (2) ical signal patterns in the arrival directions due to R/ EV the deflection in cosmic magnetic fields. As background we further add isotropically arriving cosmic where we choose T position-dependently based on rays. typical scattering obtained in [46, 48]. During the 2 training we fix T = 2.8 which corresponds to half 60◦ the maximum of the expected turbulent deflection in [46, 48] and more than double the mean value. 30◦ It was tested that training with a fixed value T for 90◦ 0◦ -90◦ all training sets is beneficial for the network per- 0◦ formance even when evaluating with varying values of T . -30◦ The coherent deflection angle δcoh is described by -60 D ◦ δcoh(R) = rad , (3) R/ EV 19.6 19.8 20.0 20.2 20.4 log10(Energy / eV) with a varying deflection power D which is determined position-dependently based on typical deflec- Figure 1: Example of simulated arrival directions used for tions from [46]. For the training it is taken from network training: the pattern induced by a magnetic field with a coherent deflection power D = 7.2 is situated at a Gaussian distribution based on deflections again ◦ ◦ D Galactic longitude l ≈ 16 and Galactic latitude b ≈ 45 . from [46]. Additionally, a lower cut on ensures The source of this signal pattern is indicated by the star that the coherent deflection mostly remains larger symbol. It has a signal fraction of fS = 5.5%, thus NS = 55 than the turbulent one. signal cosmic rays are contained in this arrival pattern. The The cosmic-ray energies follow the parameterized remaining 945 events are isotropically distributed. measured energy spectrum given in [52] which contains a broken power law with a smooth transition function above the ankle. The energy threshold of fixed number of signal cosmic rays. In the follow- 40 EeV is oriented on the data used for the recent ing we extend this simplified approach to constrain first indication of intermediate-scale anisotropy in actual physics quantities using the network. cosmic-ray arrival directions [11]. At these high A common assumption is to consider a uniformly energies typical analyses of modern observatories distributed source population with source density contain around (1000) events which our simu- ρ = N /V , thus exhibiting an average num- O S sources lation accounts for with a total event number of ber of Nsources source candidates in any given spa- NCRs = 1000. tial volume V . Each of the sources accelerates nu- A pure helium composition is chosen for the net- clei of nuclear fractions fA with a simple power-law work training to enable the learning of comprehen- energy spectrum of spectral index γ and maximum sive energy orderings. Also, having a mono-atomic rigidity cut-off Rcut: composition ensures that the network does not ar- dNA −γ tificially learn ratios between elemental fractions fA E fcut(E,ZARcut) (4) which could differ between simulations and data. dE ∝ · · Nevertheless, using helium instead of protons ac- These nuclei are emitted isotropically and they counts for the measured heavier component of cos- propagate through the universe before arriving on mic rays at the highest energies. An example sim- Earth. More details about the source modeling, e.g. ulation is shown in Fig. 1. the cut-off function fcut, can be found in [8]. To evaluate the network performance we also sim- Under the conditions that each source i emits the ulate a mixed composition scenario of the same kind same number of nuclei with the same energy spec- using 15% hydrogen, 45% helium and 40% carbon, trum and the extragalactic magnetic field is weak nitrogen and oxygen (in equal parts) nuclei. compared to the Galactic one, the arriving particle flux f per source above an energy threshold at the 2.2. A general astrophysical scenario originating edge of our Galaxy depends on only two aspects. from a source population The first one is the geometrical effect in a three- dimensional propagation scenario where the flux fi −2 In the previous section we introduced a simple of a source i at distance di decreases with fi di astrophysical simulation consisting of two parts, an while the source number per distance shell increases∝ 2 isotropic background and an additional magnetic- by Nsources di . Thus, the geometrical effect alone field induced arrival pattern with an arbitrarily would result∝ in a constant flux per distance bin. 3 The second important effect is interactions dur- 60◦ ing cosmic-ray propagation, e.g. with photon fields, where particles lose energy and the composition 30◦ changes. Additionally, nuclear decay and cosmo- 90◦ 0◦ -90◦ logical effects can attenuate particles. When com- 0◦ bined, these processes cause a suppression of particles from faraway sources. Thus, overall this scenario will again result in some prominent sources -30◦ and a practically isotropic background of farther -60 sources of which only very few particles arrive on ◦ Earth. 19.6 19.8 20.0 The parameters describing the sources within log10(Energy / eV) this general universe setup have been evaluated be- Figure 2: Example of simulated arrival directions at a source fore, using a combined fit of energy spectrum and −3 3 composition of one-dimensional CRPropa3 [53] sim- density of 10 /Mpc . For this rather small source density an anisotropic distribution of cosmic rays is obvious. Note ulations to cosmic-ray data measured by the Pierre that the energy spectrum and composition of arriving cosmic Auger Observatory [8]. In summary, the deter- rays is in accordance with measurements [8]. mined source parameters were: a spectral index of γ = 0.87, a rigidity cut-off at log10(Rcut/V) = 18.62, and a composition containing 88 percent ni- 2.3. Dataset of the AGASA Observatory trogen and 12 percent silicon (for details see [8]). When arriving on Earth, the composition was al- The Akeno Giant Air Shower Array (AGASA) tered due to propagation effects. consisted of 111 surface detectors deployed over an We now combine these one-dimensional propaga- area of about 100 km2 and additionally 27 muon tion results with the described three-dimensional detectors. AGASA was situated at 138◦ 300 E and simulation setup including both geometrical and 35◦ 470 N. This location determines the shape of interaction effects. The only free parameter is the exposure of the observatory which we calculate the source density ρS which regulates the number according to [56]. The observatory was put into and strength of the patterns induced by nearby operation in 1990 [57, 58] and was dismantled in sources. For a small source density there are only 2007 [59]. few and therefore very prominent nearby sources; The public event list for cosmic rays with energies this results in rather obvious anisotropic arrival above E = 40 EeV and zenith angles up to θ = 45◦ patterns within an isotropic background. For high contains 57 events in total [60, 61]. source densities we therefore expect predominantly isotropic arrival directions. Having these extragalactic arrival directions we 3. Graph Convolutional Neural Networks can treat deflections by the GMF analogously as in section 2.1. Therefore, the magnitude of the co- In the following we will first present an overview herent and of the turbulent deflection respectively of the deep learning techniques necessary for this again follow [46, 48]. Different to the first simula- analysis and then show the architecture of the dy- tion this time cosmic rays originate from more than namic graph convolutional neural network used. one source and thus a related deflection model has to be applied on the sphere as a whole. This is 3.1. Convolutional neural networks ensured by using a HEALPix map [54, 55] of the direction of the coherent deflection in [46, 48]. Traditional convolutional neural networks [62] During training, σturb(R) is again chosen inde- are most successful e.g. in computer vision and pendently of the cosmic-ray direction as described pattern recognition. The dense, discrete and reg- in Eq. (2) keeping T fixed. Additionally, a random ular collection of image pixels allows for scanning individual rotation is applied to the direction and the image with pixelated filters of a constant ex- coherent deflection maps, resulting in different de- tent. Usually, K filters (kernels), covering only kn flections in every generated sample. An example neighboring pixels, i.e. a small part of the overall arrival directions distribution is shown in Fig. 2. image, are used. The convolution of a single filter 4 with index a applied to pixel i of a single channel explicit cosmic ray can still be identified in the image reads: deeper network layer allowing for node classifica- kn tion and high interpretability. a X a fi = θj xij (5) j=1 3.3. Dynamic graph convolutional neural networks a with the result fi stored in a feature map, which in turn is scanned by further filters after adding a bias The special feature of dynamic graph convolu- and applying an activation function. In addition tional neural networks (DGCNNs) is that in each to Cartesian grids, spherical grids have also been layer the original graph is projected onto another used in convolutional networks [63, 39] based on graph with arbitrary dimension in coordinate space the healpix pixelization [55]. as illustrated in Fig. 3. Each particle can still be Usually, filters for deeper layers receive informa- followed through the network, however, its nearest tion from more distant pixels due to the increas- neighbors have changed. The high-dimensional co- ing receptive field of view, so that in a figurative ordinates which for each transition define the near- sense short-range correlations can be examined in est neighbors and thus the graph are derived from the first layers and long-range correlations in the the properties (features) ~xc of the particles, e.g. ar- deeper layers. All K filters indexed a with their rival direction, energy, shower depth, etc. respective parameters are trained on the basis of a task formulated in the objective function. coordinate x

4 8 3.2. Graph convolutional networks 6 7 2 A problem for applications of CNNs in astropar- 1 5 3 ticle physics is that the arrival directions of cosmic neighbor (x, y) features xc rays are continuously distributed. If one wants to 6 4 position them artificially on a grid corresponding kernel network 1 2 5 1 8 to the experimental directional resolution, the pixel 7 3 occupancy is extremely sparse on the one hand and, xc' on the other hand, in a few pixels several particles 6 kernel network 2 8 4 may be found which requires an algorithm to ag- 3 2 gregate the information. Both aspects are rather 5 7 1 x " unfavorable for the actual application as this leads c to loss of information or major computational costs. Furthermore, most convolution algorithms are de- Figure 3: Principle of a dynamic graph convolutional neu- signed for Euclidean manifolds and therefore cannot ral network with the example of 8 cosmic rays with arrival coordinates (x, y) and properties ~xc (features). The convo- handle the symmetry of spherical data. lution operation is performed by a neural network on the The concept of graph convolutional networks coordinates and features, using the 2 nearest neighbors of (GCN) [64, 65] solves the unnatural requirement each particle. The result of the operation is placed in a of particles placed on a regular grid. In GCNs, new high-dimensional space, thereby changing the neighborhood of the cosmic rays. In this way, the arrival patterns each particle can be treated individually with its ar- of the cosmic rays, even if they are distributed over the en- rival direction. All particles together form a point tire sphere, can be jointly characterized immediately in the cloud and, when using a specific neighborhood as- following network layer. signment, a graph. Here, the exact alignment and position of the particles has to be considered in the convolutional operation and in the structure of the For the convolution using the kernel, the kn near- graph. In this astroparticle-physics application, the est neighbors of a particle i are considered. Particle graph is constructed in a spherical shape as the cos- i has M-dimensional properties ~xi,c which will be mic rays arrive almost uniformly from all directions related to the properties of the kn neighboring par- onto Earth. The particles in immediate proximity ticles. The convolutional operation is implemented to each other in the coordinate space are then re- following [42] using a neural network which depends garded as the environment for a convolution. In on all values of ~xi,c, and the differences ~xi,c ~xij ,c contrast to standard convolutional networks, the between the M properties of the particle i and− those 5 of its neighbor ij: coordinates. As input features we use the arrival direction together with the energy of the cosmic ray, hΘ(~xi,c, ~xi ,c) = NNΘ(~xi,c, ~xi,c ~xi ,c), (6) thus four dimensions (x, y, z, E). j − j While the normalized directional vector (x, y, z) where Θ are trainable parameters of the kernel net- has reasonable numerical values by definition, the work NNΘ. The same kernel network is then ap- strong variation in the values of the cosmic-ray en- plied to all kn neighbors. The final result of the ergies is reduced prior to the network input. By 0 3 convolution for particle i can be obtained by an using a transformation of the energy E = E0/E 3 aggregation over the neighborhood, e.g. by calcu- with E0 = 175, 600 EeV , the energy distribu- lating the average value: tion becomes more uniform with most values at (1)/EeV2. kn O 0 1 X As the neighborhood we consider the kn = 16 ~xi,c = hΘ(~xi,c, ~xij ,c). (7) kn nearest particles in coordinate space or in the com- j=1 bined coordinate and feature space, respectively. The output dimension of the kernel network then When varying this number we found no improve- defines the new dimension of the coordinate space ment for the tasks studied in this work. In the first EdgeConv layer, the kn nearest neigh- for the resulting new graph consisting of the NCRs cosmic rays, illustrated in Fig. 3. The original N- bors are calculated using the initial coordinates of dimensional coordinate space (x, y) is extended, re- the cosmic rays (input C in Fig. 4). The convo- sulting in a new graph of K feature dimensions, de- lutions are applied to their features (input F in 0 Fig. 4). Each EdgeConv layer performs one convo- noted by ~xc in the middle plane of Fig. 3. This is the decisive difference to a classic graph convolutional lution as described in Eq. (7) using a kernel network network: the clustered particles are not necessarily with three fully connected layers of the same dimension K. Batch normalization with momentum next to each other in their original spatial coordi- −5 nates. Thus, new neighborhoods resulting from the of 0.9 and = 10 as well as ReLU activation are similar properties of the particles are investigated applied after each layer. Finally, the average over in the forthcoming convolutional layer. all neighbors is taken to obtain a permutational in- K Also, the following convolutional layer is realized variant layer. The dimension of the convolution by a network (cf. Fig. 3 lower level). As the rear- kernels is adjusted according to the specific task ranged particles inherit new neighbors, both short- studied below. These successive transformations and long-range correlations are already analyzed at can be understood as a continuous and non-linear this step. If a binary classifier is trained using the filter applied to each cosmic-ray neighborhood. objective function, the parameters of the two neigh- We use this network either as a graph convolu- boring networks will be adjusted such that two sep- tional neural network (GCN) by limiting the coor- arate groups of particles become visible in the high- dinates given to all subsequent EdgeConv layers to dimensional space denoted by ~x00 in the figure. the cosmic-ray arrival directions, or alternatively c as a dynamic graph convolutional neural network (DGCNN) by extending the coordinates with the 3.4. Network architecture feature space obtained in the previous EdgeConv Our implementation1 of the network architecture layer. Only in the latter case do we allow new neigh- is inspired by [43]. In this reference, Eq. (7) is re- borhoods to be formed. Depending on the task to ferred to as EdgeConv layer [42]. In Fig. 4, we show be solved, we find advantages for either of the net- at the top the input with the N-dimensional coor- work architectures. dinate space and the M-dimensional feature space. After the convolutions, global average pooling is Instead of expressing the cosmic-ray arrival di- used to gather the global information contained in rection by zenith and azimuth, we use the three- all cosmic rays. Using two fully-connected layers dimensional normalized vector (x, y, z) of the ar- and a final softmax activation, a binary output is rival direction to avoid discontinuities in spherical achieved which we call: signal : xsig isotropy : (1 xsig) (8) − 1code available at https://git.rwth-aachen.de/ Thus, this final output of the network can be inter- niklas.langner/edgeconv_keras preted as a probability xsig that the corresponding 6 set of cosmic rays contains an arrival pattern which originated from a source.

4. Searching for cosmic-ray sources

We evaluate the performance of the network in three challenges. In the ﬁrst challenge, the goal is Coordinates Features to recognize whether a signal pattern of a source 푁-dimensional 푀-dimensional is present within the arrival directions and energies of 1000 cosmic rays. In an extension, the sec- BatchNorm ond challenge is to classify each particle as either belonging to a source or to an isotropic contribu- C F tion. In the third task, the network should rec- EdgeConv

K = X, 푘푛 = 16, ReLU ognize whether there are signal patterns of several sources distributed across the sphere.

Concatenation (optional) 4.1. Expected sensitivity for the signal pattern of a single source C F EdgeConv In the first task we use the GCN (Fig. 4) with the , , ReLU K = 2X 푘푛 = 16 arrival directions of the cosmic rays as fixed coordinates in all convolutional layers. This is meaning- Concatenation (optional) ful as there is at most one coherent signal pattern on the sphere of arrival directions. An experiment C F with the DGCNN exhibited no improvement be- EdgeConv cause there was no need to merge signal patterns K = 4X, 푘푛 = 16, ReLU scattered across the sphere. We use K = 16 convolution kernels in the first Global Average EdgeConv layer and increase this number in the fol- Pooling lowing layers as shown in Fig. 4. After the global average pooling, a fully-connected layer with 256 Dropout=0.1 Fully connected nodes is used. These hyperparameters were opti- dim = 256, ReLU mized by a scan. The categorical cross entropy was used as the objective function, and the Adam opti- Fully connected mizer [66] was used for training. The learning rate −3 −5 dim = 2, Softmax was scheduled to drop from 10 to 10 during the first two thirds of the training, following the falling range of the cosine function (between 0 and π). The Figure 4: Architecture of the applied neural network. The batch size was set to 60 using 30 sets of 1000 ar- kn = 16 nearest neighbors of each particle are determined from the coordinate space C. The convolutional operations rival directions with a signal pattern of 55 helium on the features F are performed three times in EdgeConv nuclei from a source and 30 sets of 1000 isotrop- blocks. Optionally, neighbored particles are fixed through ically distributed arrival directions. The compar- all calculations of the network or varied by concatenation of the spatial coordinates with the features obtained with the atively high signal fraction during training proved convolutional operations. to be advantageous, since the fully trained network is then able to detect anisotropic arrival patterns even with much smaller signal fractions. Training experiments on different signal fractions or with a mixed composition provided no improvement. The expected sensitivity of the network predic- tion was evaluated on 1000 signal sets each with a total of 1000 cosmic rays, of which a varying num- 7 ber NS originate from a source and 10 sets with 7 isotropic arrival directions. We show both the eval- 100 uation with helium nuclei and the evaluation with Helium 1 a mixed composition. Keep in mind that, in both 10− Mixed charges cases, the network is trained with helium nuclei 2 10− only. 3σ 3 10− The final value xsig of the softmax function for -value each of the signal sets was compared to the val- p 10 4 ues of the isotropic sets. We define the p-value as − 5 the relative number of isotropic sets whose function Median 10− value xsig exceeds that of the signal set. 10 6 In Fig. 5 we show the median of these p-values for − 5σ all 1000 signal sets as a function of the number of 7 10− simulated cosmic rays belonging to the signal pat- 0 10 20 30 tern. NS As expected, the mono-atomic case is easier as all cosmic rays of a pattern are ordered according to Figure 5: Probability that isotropic cosmic-ray arrival di- their energy and the charges have no additional in- rections result in a network response at least as high as a fluence on the deflection angle. Here, for 13 source signal enriched simulation as a function of the number NS of source cosmic rays. The total number of events in one sim- nuclei in 1000 cosmic rays a signal pattern is de- ulated dataset is 1000. Shown is the median p-value from tected with more than 3σ (standard deviations). 1000 simulated evaluations for a helium composition (solid The threshold of 5σ is reached with 18 helium nu- red symbols) and for a mixed composition (open blue sym- clei from one source. bols). The network was trained on signal scenarios with NS = 55 source nuclei and scenarios of solely isotropic ar- For a mixed composition, the minimum number rival directions, both containing helium nuclei only. of particles from one source increases to 21 (3σ) or 31 (5σ), respectively. These numbers are remark- able considering that the network never actually dealt with mixed charges during training. If the sented as a vector in a 64-dimensional feature space network were to exploit only the helium nuclei that resulting in a total dimensionality of (64 NCRs). × it was trained on with a contribution of 45% to The pooling layer takes the mean over all cosmic the mixed-composition scenario, one would expect rays resulting in 64 output values. These encode the that 40 cosmic rays would be needed for an effect of information about whether or not there is a signal 5 standard deviations. Thus, to achieve the same pattern in the arrival directions of the cosmic rays, effect with fewer cosmic rays, the network also ex- which is then processed by the subsequent fully con- ploits information carried by the other nuclei. nected layers to provide the final binary output. In- stead, we now use the complete information of (64 4.2. Identification of cosmic rays from a source NCRs) values before the pooling layer to identify × In the second task, each individual cosmic ray cosmic rays originating from the source or from the will be examined to establish whether it belongs to isotropic arrival distribution. a signal pattern and thus originates from a common In Fig. 6a we show the median values for each of source, or whether it is a particle of the isotropic the 64 dimensions for a single set of 1000 cosmic arrival distribution. rays. The 15 cosmic rays from the source are de- For this classification we analyze the au- noted by the dark-red marked distribution and the tonomously formed variables of the network be- median values of isotropically arriving particles are fore the pooling layer as described in the following. shown as a distribution marked in gray. The or- Changing the network’s output layer to perform a der of the 64 variables is arbitrary and shown here binary classification of each of the 1000 cosmic rays for visualization purposes in a way that the median yielded only slightly better results. For the ben- value for the isotropic distribution increases. The efit of an insight into the functioning of the origi- colored areas around the median values denote the nal network, however, here we decide for cosmic-ray 68% quantile of the values of all contributing cosmic identification using this approach. rays. After the final EdgeConv layer with 64 kernels For the particles from isotropic arrival distribu- (K = 4 16, cf. Fig. 4), each cosmic ray is repre- tions the median values are close to zero in all di- × 8 mensions, and even invisible in the figure for dimensions 0 to 29 owing to their small values. For 10 the particles of a signal pattern the information ap- pears to be encoded in the dimensions 0 to 29 with 8 median values around 9. In the last 34 dimensions, however, the median values of signal cosmic rays 6 Source pattern are close to zero. Thus, the network allocates ap- Isotropic background proximately half of the available feature dimensions 4 for cosmic rays originating from the source, while the other half is used for isotropic cosmic rays. This Feature coordinates 2 indicates that the network effectively performs clustering in the high-dimensional feature space which separates the signal cosmic rays from the isotropic 0 ones. 0 10 20 30 40 50 60 Feature dimensions after final EdgeConv layer For an additional visualization of this process we use the t-SNE algorithm. This algorithm em- (a) beds high-dimensional data in a reduced number of dimensions by keeping the implicit structure, i.e. Source pattern neighborhoods and connections, in place. For fur- Isotropic background ther information refer to [67]. The three-dimensional representation of the 64- dimensional point cloud after the final Edge-Conv layer is shown in Fig. 6b. One can see clear clustering of the cosmic rays from the source pattern, while some isotropic cosmic rays are also clustered near this region. To distinguish between cosmic rays that most likely originated in a source and isotropic ones we make use of the network’s clustering of source cosmic rays in the first dimensions of the feature space. Subsequently, for each cosmic ray we take the median of the first 27 dimensions (preserving a buffer (b) zone) of its vector in the feature space. We iden- Figure 6: Visualization of the network for separating sig- tify cosmic rays with a median value greater than 3 nal cosmic rays (red) from isotropically arriving cosmic rays as coming from a source and below 3 as belonging (gray). For a single simulated data set with 1000 cosmic to the isotropically arriving distribution. We have rays, 15 of which originate from a common source the me- verified that this threshold works well even if the dian values seen in (a) (solid lines) are calculated for each of the 64 dimensions of the final EdgeConv layer. The shaded total number of cosmic rays varies. areas represent the 68% quantile. (b) A three-dimensional To assess the quality of the identification, we cal- representation of the final EdgeConv layer’s output is cre- culate the efficiency of assigning a signal particle as ated using the t-SNE technique [67]. belonging to the signal pattern by: # correctly identified signal cosmic rays = (9) # signal cosmic rays The simulated underlying signal pattern contain- We also calculate the purity with which a particle ing 15 particles is identified and all signal par- identified as signal actually belongs to the signal ticles were labeled correctly with an efficiency of pattern of a source as: 100%. Some surrounding isotropic particles were # correctly identified signal cosmic rays incorrectly classified in this example with a median ρ = # identified signal cosmic rays greater than 3, which reduces the purity to 65.2%. (10) To assess whether or not this is a representative An example of classified arrival directions for example we examine 500 sets with varying num- a pure helium simulation is shown in Fig. 7. bers of signal cosmic rays. In Fig. 8 we show the 9 60◦ 1.0

30◦ 0.8

90◦ 0◦ -90◦ 0◦ 0.6

-30◦ Source pattern

Efficiency 0.4 Isotropic background -60◦ 0.2 0 1 2 3 4 5 6 7 8 9 10 Mixed charges Median of the first 27 feature dimensions Helium 0.0 Figure 7: Classified arrival directions: the color scale in- 0 10 20 30 40 50 dicates the median of the first 27 feature dimensions. A NS cosmic ray is identified as signal with a median of the 27 feature values greater than 3 and is otherwise attributed to (a) isotropy. The signal pattern (denoted by filled circles) is correctly identified. 1.0

0.8 efficiency and purity as a function of the number of signal particles of a source separately for the sce- 0.6 nario with pure helium and for the mixed composition. Purity 0.4 For the helium scenario, 18 signal particles from one source were required to achieve a significance 0.2 of 5 standard deviations (cf. Fig. 5). If the pat- Mixed charges Helium tern is detected, in median all 18 cosmic rays are 0.0 identified correctly with an efficiency of 100 percent 0 10 20 30 40 50 as shown in Fig. 8a. Nevertheless there are cases NS where fewer or even no cosmic rays with median feature dimension values above 3 are detected re- (b) sulting in cases with a low efficiency. This is shown Figure 8: Median efficiency (a) and purity (b) of 500 simu- by the shaded region representing the 68% inter- lated sets as a function of the number of signal cosmic rays val in Fig. 8a. The median purity of 60% shown in NS coming from a single source. The remaining (1000 − NS) Fig. 8b is still rather low at NS = 18 and exhibits cosmic rays are distributed isotropically. Both the pure he- a large spread. This means that, in addition to the lium (solid red) and the mixed composition (blue) are shown. Transparent bands represent the 68% quantile. (in median) 18 correctly identified source particles, (in median) 12 of the isotropically arriving particles are incorrectly identified as signals. In the mixed composition case, 31 identified source particles for 5 standard deviations have an efficiency and purity For this analysis we use the dynamic graph con- of around 60% 70%. volutional network (DGCNN) with the extension − of the coordinate space from the second network layer onward by considering the arrival directions of 4.3. Search for multiple cosmic-ray sources the particles as well as the features resulting from In the third challenge, we again aim to identify both the particles and their neighborhoods. Thus, signal patterns in the arrival directions and ener- the network has the possibility to change neighbor- gies of cosmic rays. Due to the general astrophys- hoods in deeper layers in such a way that neighbor- ical scenario used (see section 2.2), there may be hood properties can be exploited for the respective signal patterns from several different sources, each challenge. Here, we find that the dynamic graph contributing different numbers of cosmic rays. network performs better than the graph network 10 with fixed neighborhoods. Since signal patterns can 100 appear anywhere on the arrival sphere of the par- 1 ticles, the dynamic graph network can effectively 10− analyze the signal patterns of different localizations 2 10− on the sphere at an early stage by forming suitable 3σ 3 neighborhoods. Optimized by scanning we decide -value 10− p to increase the kernel size from K = 64 in the first 4 10− EdgeConv layer to K = 256 in the last one. Exposure 5 NCRs = 57 Median 10− As training data we use simulated astrophysical Exposure −3 −3 scenarios with a source density of 10 Mpc . At 10 6 NCRs = 1000 − 5σ this rather small source density the arrival direc- No exposure 7 NCRs = 1000 tions exhibit stronger anisotropies (cf. Fig. 2) which 10− 4 3 2 1 0 enables a stable training process. It was verified in 10− 10− 10− 10− 10 −3 −3 1 a scan over the source density that 10 Mpc is Source Density / Mpc3 a reasonable choice. As isotropic comparison we also use the same sce- Figure 9: Probability of the cosmic-ray arrival directions with multiple signal patterns to originate from isotropic scenarios, but randomize the arrival directions. In this narios as a function of the source density. The signal patterns way we ensure that the energy distribution of the were simulated using the general astrophysical scenario with cosmic rays remains unchanged. the source density as the only parameter (section 2.2). We follow the previously outlined calculation of the p-value: first, we calculate the network predic- tion xsig of 1000 simulated astrophysical scenarios 5. Source density limit from AGASA data with signal patterns from different sources. We then determine the relative number of isotropic scenarios In this section we apply the dynamic graph that have a larger value xsig than each of the signal convolutional network to the 57 ultra-high energy scenarios. events of the AGASA Observatory (cf. section 2.3). In Fig. 9 we show the median p-value, determined To begin with we train the network using simulated from the 1000 signal scenarios, as a function of the datasets with 57 cosmic rays within the exposure of source density. Under the conditions of this gen- the observatory. For the training we use the gener- eral astrophysical scenario, a set of 1000 arriving alized astrophysical scenario (cf. section 2.2) with cosmic rays with signal patterns can be identified source density as the only free parameter where we ρ −4 −3 with 3 standard deviations at a source density of determined S = 10 Mpc to be a good choice 4.0 10−2 Mpc−3. The limit of 5 standard deviations based on a scan on simulations. To account for is reached· at a source density of 1.5 10−2 Mpc−3. the difference between the energy spectrum of the · AGASA data and the simulated one, we replace the If data of an observatory which can observe only cosmic-ray energies before applying the deflection a part of the sphere are used, the sensitivity de- in the simulation: for this, 57 random energies are creases accordingly. Here, we use the geometric ex- sampled from a power law spectrum with γ = 3.4, posure of the AGASA Observatory by way of exam- which results in a median energy close to the− me- ple while keeping NCRs at 1000. The blue symbols dian of the AGASA measurements. The energies in Fig. 9 denote the corresponding reduced exclu- are replaced such that the order of the energy val- sion limits where the 5σ confidence limit is reached ues remains the same. for 5 10−3 Mpc−3. · For the measured 57 events, we determine the p- Additionally, we explore the sensitivity of the value as presented in section 4.1. For that we gener- network on a dataset with only NCRs = 57 cosmic ate isotropic sets of 57 cosmic rays by randomizing rays following the AGASA exposure. The corre- the arrival directions of the measured events. This sponding p-values are shown in gray in Fig. 9. One procedure again ensures that the energy distribu- can see that the sensitivity is considerably reduced tion of the cosmic rays remains unchanged. as expected. The 3σ confidence limit is only reached The network response xsig from the AGASA for a source density of 6 10−5 Mpc−3 and the 5σ dataset lies fairly within the isotropic expectation at 8 10−6 Mpc−3. · and is exceeded by p = 65% of the isotropic sim- · 11 ulations. Thus, in the relatively few events of the For a generalized astrophysical scenario, which AGASA Observatory the network finds no signs of depends only on the source density and gen- signal patterns in the arrival directions. Therefore, erates multiple source patterns distributed over we calculate a lower limit for the source density the sphere, dynamic graph convolutional networks for the generalized astrophysical scenario. To this prove to be advantageous. In this case, the neigh- end, we invert the question and ask what minimum borhoods of the particles in deeper network layers source density ρS the generalized astrophysical sce- can be optimized for the convolutional operation in nario needs to have in order to satisfy the p-value such a way that the properties of signal patterns of the data. For this purpose, we investigate the can be studied together despite very different spa- distribution of the p-values for 1000 simulated data tial arrival directions. sets at each source density ρS of the generalized As a measure for the sensitivity of such networks, astrophysical scenario. we expect that the existence of signal patterns can We obtain a lower limit of 3 10−4 Mpc−3 for be proven with 5 standard deviations for 1000 cos- the density of cosmic-ray sources· at 95% confidence mic rays at an upper limit for the source density of level. 1.5 10−2 Mpc−3. · According to Fig. 9, we expect that a larger num- As an example for data measured by an obser- ber of cosmic rays, such as those measured by mod- vatory we analyzed the public data of the AGASA ern observatories, will significantly improve the de- experiment with energies above 40 EeV. A signal termination of the source density. In addition, sig- pattern could not be detected here with 57 cosmic nal patterns could be detected by the higher data rays only. However, from these data we give a lower statistics. limit on the source density of the generalized astro- 95 −4 −3 physical scenario of ρS = 3 10 Mpc at 95% confidence level. · 6. Conclusion We also show that we could greatly improve such an exclusion limit for the source density when us- In this work we investigated graph networks for ing data from current observatories with more than the identification of source patterns in the cosmic- 1000 cosmic rays. Furthermore, patterns may be ray arrival directions and energies. These source found and thus sources of ultra-high energy cosmic patterns originate from cosmic nuclei that are de- rays identified. flected in the coherent magnetic field of our Galaxy and allow cosmic-ray sources to be identified. Acknowledgments Graph convolutional networks are particularly well suited to investigate cosmic rays with their We wish to thank Yannik Rath for his valuable individual arrival directions without digitizing the comments on the manuscript. This work is sup- directions on a Cartesian or spherical grid. The ported by the Ministry of Innovation, Science and convolutional operation is defined by the particles’ Research of the State of North Rhine-Westphalia, nearest neighbors, whose properties are character- and by the Federal Ministry of Education and Re- ized in a K-dimensional space. search (BMBF). When classifying a data set as either isotropic background or containing a signal pattern, our in- References vestigations show that the particles of a signal pattern are bundled in the K-dimensional property [1] J. Abraham, et al., Observation of the suppression of the flux of cosmic rays above 4 × 1019eV, Phys. Rev. space separately from the isotropically arriving par- Lett. 101 (2008) 061101 (2008). arXiv:0806.4302, doi: ticles. 10.1103/PhysRevLett.101.061101. The network thus identifies data sets with a sig- [2] J. Abraham, et al., Measurement of the energy spec- 18 nal pattern of 18 helium nuclei in 982 isotropically trum of cosmic rays above 10 eV using the Pierre Auger Observatory, Phys. Lett. B685 (2010) 239–246 distributed particles with a confidence of 5 standard (2010). arXiv:1002.1975, doi:10.1016/j.physletb. deviations against isotropic data sets. Additionally, 2010.02.013. in median the network correctly identifies all 18 he- [3] T. Abu-Zayyad, et al., The Cosmic Ray Energy Spec- lium nuclei from the signal pattern and incorrectly trum Observed with the Surface Detector of the Tele- scope Array Experiment, Astrophys. J. 768 (2013) identifies 12 particles that actually belong to the L1 (2013). arXiv:1205.5067, doi:10.1088/2041-8205/ isotropic background. 768/1/L1. 12 [4] A. Aab, et al., Measurement of the cosmic ray spectrum Galactic Nuclei and Fermi γ-ray Sources?, The As- above 4 × 1018 eV using inclined events detected with trophysical Journal 719 (1) (2010) 459–468 (jul 2010). the Pierre Auger Observatory, JCAP 1508 (2015) 049 doi:10.1088/0004-637x/719/1/459. (2015). arXiv:1503.07786, doi:10.1088/1475-7516/ [19] G. Golup, D. Harari, S. Mollerach, E. Roulet, Search- 2015/08/049. ing for signals of magnetic lensing in ultra-high energy [5] A. Aab, et al., Depth of Maximum of Air-Shower Pro- cosmic rays, JCAP 1107 (2011) 006 (2011). arXiv: files at the Pierre Auger Observatory: Measurements 1104.1103, doi:10.1088/1475-7516/2011/07/006. at Energies above 1017.8 eV, Phys. Rev. D90 (12) [20] G. Giacinti, M. Kachelriess, D. V. Semikoz, G. Sigl, (2014) 122005 (2014). arXiv:1409.4809, doi:10.1103/ Ultrahigh Energy Nuclei in the Turbulent Galac- PhysRevD.90.122005. tic Magnetic Field, Astropart. Phys. 35 (2011) [6] A. Aab, et al., Depth of maximum of air-shower pro- 192–200 (2011). arXiv:1104.1141, doi:10.1016/j. files at the Pierre Auger Observatory. II. Composition astropartphys.2011.07.006. implications, Phys. Rev. D90 (2014) 122006 (2014). [21] K. Dolag, D. Grasso, V. Springel, I. Tkachev, Map- arXiv:1409.5083, doi:10.1103/PhysRevD.90.122006. ping deflections of ultrahigh energy cosmic rays in con- [7] A. Aab, et al., Inferences on mass composition and strained simulations of extragalactic magnetic fields, tests of hadronic interactions from 0.3 to 100 EeV using JETP Lett. 79 (2004) 583–587, [Pisma Zh. Eksp. Teor. the water-Cherenkov detectors of the Pierre Auger Ob- Fiz.79,719(2004)] (2004). arXiv:astro-ph/0310902, servatory,, Phys. Rev. D96 (12) (2017) 122003 (2017). doi:10.1134/1.1790011. arXiv:1710.07249, doi:10.1103/PhysRevD.96.122003. [22] D. Paoletti, F. Finelli, CMB Constraints on a Stochastic [8] A. Aab, et al., Combined fit of spectrum and Background of Primordial Magnetic Fields, Phys. Rev. composition data as measured by the Pierre D83 (2011) 123533 (2011). arXiv:1005.0148, doi:10. Auger Observatory, JCAP 1704 (04) (2017) 038, 1103/PhysRevD.83.123533. [Erratum: JCAP1803,no.03,E02(2018)] (2017). [23] R. Durrer, A. Neronov, Cosmological Magnetic Fields: arXiv:1612.07155, doi:10.1088/1475-7516/2018/ Their Generation, Evolution and Observation, Astron. 03/E02,10.1088/1475-7516/2017/04/038. Astrophys. Rev. 21 (2013) 62 (2013). arXiv:1303.7121, [9] A. Aab, et al., Observation of a Large-scale Anisotropy doi:10.1007/s00159-013-0062-7. in the Arrival Directions of Cosmic Rays above 8×1018 [24] S. Hackstein, F. Vazza, M. Brüggen,G. Sigl, A. Dun- eV, Science 357 (6537) (2017) 1266–1270 (2017). arXiv: dovic, Propagation of ultrahigh energy cosmic rays in 1709.07321, doi:10.1126/science.aan4338. extragalactic magnetic fields: a view from cosmologi- [10] A. Aab, et al., Large-scale cosmic-ray anisotropies cal simulations, Mon. Not. Roy. Astron. Soc. 462 (4) above 4 EeV measured by the Pierre Auger Observa- (2016) 3660–3671 (2016). arXiv:1607.08872, doi:10. tory, Astrophys. J. 868 (1) (2018) 4 (2018). arXiv: 1093/mnras/stw1903. 1808.03579, doi:10.3847/1538-4357/aae689. [25] J. D. Bray, A. M. M. Scaife, An upper limit on [11] A. Aab, et al., An Indication of anisotropy in arrival di- the strength of the extragalactic magnetic field from rections of ultra-high-energy cosmic rays through com- ultra-high-energy cosmic-ray anisotropy, Astrophys. J. parison to the flux pattern of extragalactic gamma- 861 (1) (2018) 3 (2018). arXiv:1805.07995, doi:10. ray sources, Astrophys. J. 853 (2) (2018) L29 (2018). 3847/1538-4357/aac777. arXiv:1801.06160, doi:10.3847/2041-8213/aaa66d. [26] P. Abreu, et al., Search for signatures of magnetically- [12] L. Caccianiga, Anisotropies of the highest energy induced alignment in the arrival directions measured cosmic-ray events recorded by the Pierre Auger Obser- by the Pierre Auger Observatory, Astropart. Phys. vatory in 15 years of operation, PoS ICRC2019 (2019) 35 (2012) 354–361 (2012). arXiv:1111.2472, doi:10. 206 (2019). arXiv:1909.09073. 1016/j.astropartphys.2011.10.004. [13] T. Stanev, Ultrahigh-energy cosmic rays and the large [27] A. Aab, et al., Search for patterns by combining scale structure of the galactic magnetic field, Astrophys. cosmic-ray energy and arrival directions at the Pierre J. 479 (1997) 290 (1997). arXiv:astro-ph/9607086, Auger Observatory, Eur. Phys. J. C75 (6) (2015) doi:10.1086/303866. 269 (2015). arXiv:1410.0515, doi:10.1140/epjc/ [14] D. Harari, S. Mollerach, E. Roulet, Signatures of galac- s10052-015-3471-0. tic magnetic lensing upon ultrahigh-energy cosmic rays, [28] I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, JHEP 02 (2000) 035 (2000). arXiv:astro-ph/0001084, The MIT Press, 2016 (2016). doi:10.1088/1126-6708/2000/02/035. [29] M. Erdmann, J. Glombitza, D. Walz, A Deep Learning- [15] D. Harari, S. Mollerach, E. Roulet, F. Sanchez, Lensing based Reconstruction of Cosmic Ray-induced Air Show- of ultrahigh-energy cosmic rays in turbulent magnetic ers, Astropart. Phys. 97 (2018) 46–53 (2018). arXiv: fields, JHEP 03 (2002) 045 (2002). arXiv:astro-ph/ 1708.00647, doi:10.1016/j.astropartphys.2017.10. 0202362, doi:10.1088/1126-6708/2002/03/045. 006. [16] G. Golup, D. Harari, S. Mollerach, E. Roulet, Source [30] M. Erdmann, L. Geiger, J. Glombitza, D. Schmidt, position reconstruction and constraints on the galactic Generating and refining particle detector simulations magnetic field from ultra-high energy cosmic rays, As- using the Wasserstein distance in adversarial networks, tropart. Phys. 32 (2009) 269–277 (2009). arXiv:0902. Comput. Softw. Big Sci. 2 (1) (2018) 4 (2018). arXiv: 1742, doi:10.1016/j.astropartphys.2009.09.003. 1802.03325, doi:10.1007/s41781-018-0008-x. [17] G. Giacinti, M. Kachelriess, D. V. Semikoz, G. Sigl, [31] I. Shilon, M. Kraus, M. Büchele, K. Egberts, T. Fischer, Ultrahigh Energy Nuclei in the Galactic Magnetic Field, T. L. Holch, T. Lohse, U. Schwanke, C. Steppa, S. Funk, JCAP 1008 (2010) 036 (2010). arXiv:1006.5416, doi: Application of Deep Learning methods to analysis of 10.1088/1475-7516/2010/08/036. Imaging Atmospheric Cherenkov Telescopes data, As- [18] Y.-Y. Jiang, L. G. Hou, J. L. Han, X. H. Sun, W. Wang, tropart. Phys. 105 (2019) 44–53 (2019). arXiv:1803. Do Ultrahigh Energy Cosmic Rays Come from Active 10698, doi:10.1016/j.astropartphys.2018.10.003.

13 [32] S. Delaquis, et al., Deep Neural Networks for En- doi:10.1093/mnras/stt1731. ergy and Position Reconstruction in EXO-200, JINST [48] R. Jansson, G. R. Farrar, The Galactic Magnetic Field, 13 (08) (2018) P08023 (2018). arXiv:1804.09641, doi: Astrophys. J. 761 (2012) L11 (2012). arXiv:1210.7820, 10.1088/1748-0221/13/08/P08023. doi:10.1088/2041-8205/761/1/L11. [33] F. Führer,T. Charnock, A. Zilles, M. Tueros, Towards [49] M. C. Beck, A. M. Beck, R. Beck, K. Dolag, A. W. online triggering for the radio detection of air show- Strong, P. Nielaba, New constraints on modelling the ers using deep neural networks, EPJ Web Conf. 216 random magnetic field of the MW, JCAP 1605 (05) (2019) 03004 (2019). arXiv:1809.01934, doi:10.1051/ (2016) 056 (2016). arXiv:1409.5120, doi:10.1088/ epjconf/201921603004. 1475-7516/2016/05/056. [34] A. Guillen, A. Bueno, J. M. Carceller, J. C. Martinez- [50] J. L. Han, R. N. Manchester, W. van Straten, P. Demor- Velazquez, G. Rubio, C. J. Todero Peixoto, P. Sanchez- est, Pulsar rotation measures and large-scale magnetic Lucas, Deep learning techniques applied to the physics field reversals in the galactic disk, The Astrophysical of extensive air showers, Astropart. Phys. 111 (2019) Journal Supplement Series 234 (1) (2018) 11 (jan 2018). 12–22 (2019). arXiv:1807.09024, doi:10.1016/j. doi:10.3847/1538-4365/aa9c45. astropartphys.2019.03.001. [51] R. Fisher, Dispersion on a Sphere, Proc. R. Soc. Lond. [35] M. Erdmann, F. Schlüter, R. Smida, Classification A. 217 (1130) (1953) 295–305 (May 1953). doi:10. and Recovery of Radio Signals from Cosmic Ray In- 1098/rspa.1953.0064. duced Air Showers with Deep Learning, JINST 14 (04) [52] F. Fenu, The cosmic ray energy spectrum mea- (2019) P04005 (2019). arXiv:1901.04079, doi:10. sured using the Pierre Auger Observatory (2017) 9– 1088/1748-0221/14/04/P04005. 16[PoSICRC2017,486(2018)] (2017). doi:10.22323/1. [36] G. Anton, et al., Search for Neutrinoless Double-Beta 301.0486. Decay with the Complete EXO-200 Dataset, Phys. Rev. [53] R. Alves Batista, A. Dundovic, M. Erdmann, K.-H. Lett. 123 (16) (2019) 161802 (2019). arXiv:1906.02723, Kampert, D. Kuempel, G. Müller, G. Sigl, A. van doi:10.1103/PhysRevLett.123.161802. Vliet, D. Walz, T. Winchen, CRPropa 3 - a Pub- [37] P. Bezyazeekov, et al., Advanced Signal Reconstruction lic Astrophysical Simulation Framework for Propagat- in Tunka-Rex with Matched Filtering and Deep Learn- ing Extraterrestrial Ultra-High Energy Particles, JCAP ing (2019). arXiv:1906.10947. 1605 (05) (2016) 038 (2016). arXiv:1603.07142, doi: [38] M. Kronmueller, T. Glauch, Application of Deep Neural 10.1088/1475-7516/2016/05/038. Networks to Event Type Classification in IceCube, PoS [54] A. Zonca, L. Singer, D. Lenz, M. Reinecke, C. Rosset, ICRC2019 (2020) 937 (2020). arXiv:1908.08763, doi: E. Hivon, K. Gorski, healpy: equal area pixelization and 10.22323/1.358.0937. spherical harmonics transforms for data on the sphere in [39] O. Kalashev, M. Pshirkov, M. Zotov, Identifying nearby python, Journal of Open Source Software 4 (35) (2019) sources of ultra-high-energy cosmic rays with deep 1298 (Mar. 2019). doi:10.21105/joss.01298. learning (2019). arXiv:1912.00625. [55] K. M. Górski,E. Hivon, A. J. Banday, B. D. Wandelt, [40] M. Erdmann, L. Geiger, D. Schmidt, M. Urban, F. K. Hansen, M. Reinecke, M. Bartelmann, HEALPix: M. Wirtz, Origins of Extragalactic Cosmic Ray Nu- A Framework for High-Resolution Discretization and clei by Contracting Alignment Patterns induced in Fast Analysis of Data Distributed on the Sphere, The the Galactic Magnetic Field, Astropart. Phys. 108 Astrophysical Journal 622 (2005) 759–771 (Apr. 2005). (2019) 74–83 (2019). arXiv:1807.08734, doi:10.1016/ arXiv:arXiv:astro-ph/0409513, doi:10.1086/427976. j.astropartphys.2018.11.004. [56] P. Sommers, Cosmic ray anisotropy analysis with a [41] M. Abadi, et al., Tensorflow: Large-scale machine full-sky observatory, Astropart. Phys. 14 (2001) 271– learning on heterogeneous distributed systems, CoRR 286 (2001). arXiv:astro-ph/0004016, doi:10.1016/ abs/1603.04467 (2016). arXiv:1603.04467. S0927-6505(00)00130-4. [42] Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bron- [57] N. Chiba, et al., Akeno giant air shower array (AGASA) stein, J. M. Solomon, Dynamic graph CNN for learning covering 100 km2 area, Nucl. Instrum. Meth. A311 on point clouds, CoRR abs/1801.07829 (2018). arXiv: (1992) 338–349 (1992). doi:10.1016/0168-9002(92) 1801.07829. 90882-5. [43] H. Qu, L. Gouskos, ParticleNet: Jet Tagging via Parti- [58] H. Ohoka, S. Yoshida, M. Takeda, Further develop- cle Clouds (2019). arXiv:1902.08570. ment of data acquisition system of AGASA, Nucl. In- [44] G. Kasieczka, T. Plehn, et al., The Machine Learning strum. Meth. A385 (1997) 268–276 (1997). doi:10. Landscape of Top Taggers, SciPost Phys. 7 (2019) 14 1016/S0168-9002(96)01027-3. (2019). doi:10.21468/SciPostPhys.7.1.014. [59] Institute for Cosmic Ray Research, University of Tokyo, [45] M. S. Pshirkov, P. G. Tinyakov, P. P. Kronberg, K. J. Scientific Activities: Report to the Review Comitee, Newton-McGee, Deriving global structure of the Galac- unpublished (2007). tic Magnetic Field from Faraday Rotation Measures [60] M. Takeda, et al., Small-Scale Anisotropy of Cosmic of extragalactic sources, Astrophys. J. 738 (2011) 192 Rays above 1019 eV Observed with the Akeno Giant (2011). arXiv:1103.0814, doi:10.1088/0004-637X/ Air Shower Array, The Astrophysical Journal 522 (1) 738/2/192. (1999) 225–237 (Sep 1999). doi:10.1086/307646. [46] R. Jansson, G. R. Farrar, A New Model of the Galac- [61] N. Hayashida, et al., Updated AGASA event list above tic Magnetic Field, Astrophys. J. 757 (2012) 14 (2012). 4 x 1019 eV, Astron.J. 120 (2000) 2190 (2000). arXiv: arXiv:1204.3662, doi:10.1088/0004-637X/757/1/14. astro-ph/0008102. [47] M. S. Pshirkov, P. G. Tinyakov, F. R. Urban, Mapping [62] Y. LeCun, B. J., Convolutional networks for images, UHECRs deflections through the turbulent galactic speech, and time-series, in: M. A. Arbib (Ed.), The magnetic field with the latest RM data, Mon. Not. Roy. Handbook of Brain Theory and Neural Networks, MIT Astron. Soc. 436 (2013) 2326 (2013). arXiv:1304.3217, Press, 1995, Ch. 10, p. 14 (1995).

14 [63] N. Perraudin, M. Defferrard, T. Kacprzak, R. Sgier, DeepSphere: Efficient spherical Convolutional Neu- ral Network with HEALPix sampling for cosmological applications, Astron. Comput. 27 (2019) 130– 146 (2019). arXiv:1810.12186, doi:10.1016/j.ascom. 2019.03.004. [64] W. L. Hamilton, R. Ying, J. Leskovec, Representation learning on graphs: Methods and applications, CoRR abs/1709.05584 (2017). arXiv:1709.05584. [65] T. N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, arXiv preprint arXiv:1609.02907 (2016). [66] D. P. Kingma, J. Ba, Adam: A Method for Stochastic Optimization (2014). arXiv:1412.6980. [67] L. van der Maaten, G. Hinton, Visualizing high- dimensional data using t-sne, Journal of Machine Learning Research 9 (Nov) (2008) 2579–2605 (2008).