December 3rd 2019 Souriau Symplectic Structures of Lie Group Machine Learning on Statistical Drone Doppler & Kinematic Signatures Frédéric BARBARESCO Key Technology Domain Processing, Control & Cognition KTD PCC « SENSING » Segment Leader ENS Ulm 1942 KTD PCC Representative for Thales Land & Air Systems www.thalesgroup.com OPEN Drone Detection, Tracking and Recognizing ▌ Drone Recognition The illegal use of drones requires development of systems capable of detecting, tracking and recognizing them in a non-collaborative manner, and this with sufficient anticipation to be able to engage interception means adapted to the threat. The small size of autonomous aircraft makes it difficult to detect them at long range with enough early warning with conventional techniques, and seems more suitable for observation by radar systems. However, the radio frequency detection of this type of object poses other difficulties to solve because of their slow speed which can make them confused with other mobile echoes such as those of land vehicles, birds and movements of vegetation agitated by atmospheric turbulences. It is therefore necessary to design robust classification methods of its echoes to ensure their discrimination with respect to criteria characterizing their movements (micro-movements of their moving parts and body kinematic movements). Applications of Geometric and Structure Preserving Methods OPEN 2 Cambridge University – Newton Institute, 03/12/19 Drone Recognition on Radar Doppler Signature of their moving parts ▌ Drone Radar Micro-Doppler Signature The first idea is to listen to the Doppler signature of the radar echo coming from the drone, which signs the radial velocity variations of the reflectance parts of the moving elements, like the blades. Depending on the speed of the drone, the number of moving elements and their speed of rotation, and the roll & pitch attitude characterizing the angle of observation, the Doppler signature of the drone will be modified. Other factors may also vary this signature as the payload that will vary the blades rotation speeds, or as the wind according to which the drone will change the engine speeds of each blade and the attitude of the drone. The size of the radar radio frequency sensor analysis box, which depends on the beam width (related to the size of the antenna) and the distance resolution (linked to the bandwidth), can also be adapted in the cases of drones close to each other as in coordinated flights or in swarms, which will mix the Doppler signatures of several objects, sometimes with echoes from the ground at very low altitude. Applications of Geometric and Structure Preserving Methods OPEN 3 Cambridge University – Newton Institute, 03/12/19 Drone Recognition on Kinematics Signature of their body trajectory ▌ Drone Radar Kinematics Signature To improve the classification performance of drones when the blade Doppler signature is more difficult to characterize (blade fairing, carbon blades, ...), we consider, in addition to Doppler signatures, the drone kinematic, characterizing its - speed / acceleration / jerk - curvature/torsion of its trajectory Kinematics Data are provided through Invariant Extended Kalman Filter (IEKF) Radar Tracker based on local Frenet-Seret model. Applications of Geometric and Structure Preserving Methods OPEN 4 Cambridge University – Newton Institute, 03/12/19 Existing THALES AI Technologies for Drone Recognition www.thalesgroup.com OPEN Drone Recognition by Radar Micro-Doppler Signature ▌ Drone Radar Micro-Doppler Deep Learning Use of Radar simulator to learn on Hybrid data (simulated and real data) Micro-Doppler time/frequency spectrum ▌ Complex-valued CNN Fourier transform is a convolution by the Fourier atoms. We can learn a Fourier-like complex filter bank. ▌ HPD neural networks Covariance matrix has HPD (Hermitian Positive Defnite) structure Statistical analysis of manifold-valued data : Information Geometry Applications of Geometric and Structure Preserving Methods OPEN 6 Cambridge University – Newton Institute, 03/12/19 Fully CNN, SPDNet/HPDNet Architecture and Complex ConvNet (THALES/Sorbonne University PhD of Daniel Brooks) ▌ Adaptation for SPD/HPD matrix ▌ Adaptation for Complex convolution & Fully Convolution Network (time axis) Applications of Geometric and Structure Preserving Methods OPEN 7 Cambridge University – Newton Institute, 03/12/19 Drone Recognition by Radar Micro-Doppler Signature: Results ▌ Validation on NATO Database 10 classes (7 drones and birds) – → 10 classes Applications of Geometric and Structure Preserving Methods OPEN 8 Cambridge University – Newton Institute, 03/12/19 References Daniel A. Brooks, Olivier Schwander, Frédéric Barbaresco, Jean-Yves Schneider, Matthieu Cord, A Hermitian Positive Definite neural network for micro-Doppler complex covariance processing, International Radar Conference, Toulon, Septembre 2019 Daniel A. Brooks, Olivier Schwander, Frédéric Barbaresco, Jean-Yves Schneider,, Matthieu Cord, Complex-valued neural networks for fully-temporal micro-Doppler classification, International Radar Symposium (IRS), Ulm, Juin 2019 D. A. Brooks, O. Schwander, F. Barbaresco, J. Schneider, and M. Cord. Exploring Complex Time-series Representations for Riemannian Machine Learning of Radar Data. In ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3672–3676, May 2019 D. Brooks, F/ Barbaresco, Y. Ziani, J.Y. Schneider, C. Adnet, IA & réseaux de neurones profonds pour la reconnaissance Radar de drones sur critères Micro- Doppler et Cinématique, CYBERWEEK, CE&SAR « IA et Défense » conference, Rennes, November 2019 Applications of Geometric and Structure Preserving Methods OPEN 9 Cambridge University – Newton Institute, 03/12/19 Drone Recognition by Radar Kinematics Signature ▌ Drone Radar Kinematics XGBoost Learning Drone trajectories and Kinematics are simulated by auto-pilot : ENAC Paparazzi UAV Birds trajectories and kinematics are characterized by birds on GPS: MOVEBANK Statistics features extraction (ordered statistics, L-moments, quantiles, …) from time series of drone : speed / acceleation / jerk, 2D horizontal speed module, 3D speed module, 2D horizontal curvature, 3D curvature, torsion logarithm Applications of Geometric and Structure Preserving Methods Python time series statistics: lmoments, tsfreshOPEN 10 Cambridge University – Newton Institute, 03/12/19 Gradient Boosting: XGBOOST ▌ Gradient Boosting Construct classifiers iteratively, each new one focusing on the errors made by the previous ones. The final prediction is the classifiers majority vote. Great performance even in the high dimension setting ; can be parallelized. better results than Random Forests ▌ XGBoost XGBoost (Chen and Guestrin, 2016) Several hyperparameters to tune : Number of trees, Learning rate, examples proportion to build each tree, variables proportion to build each tree, Maximum depth of a tree, Minimum number of Applicationsexamples of Geometric andin Structure a region Preserving Methods to make a new split OPEN 11 Cambridge University – Newton Institute, 03/12/19 XGBOOST Learning on Drones/Birds Kinematics Applications of Geometric and Structure Preserving Methods OPEN 12 Cambridge University – Newton Institute, 03/12/19 References ▌ Random Forest and XGBoost Gérard Biau and Benoît Cadre. “Optimization by gradient boosting”. In: arXiv preprint arXiv:1707.05023 (2017) Gérard Gérard Biau and Erwan Scornet. “A random forest guided tour”. In: Test 25.2 Biau, SCAI (2016), pp. 197–227 Gérard Biau Erwan Scornet, Johannes Welbl, Neural Random Forests , arxiv arXiv:1604.07143 Gérard Biau, Erwan Scornet, A Random Forest Guided Tour, arXiv:1511.05741 Erwan Scornet, On the asymptotics of random forests, arXiv:1409.2090 Erwan Scornet X/CMAP ▌ Recognition on Kinematic Data R. Ginoulhac, F.Barbaresco & al, Target Classifcation Based On Kinematic Data From AIS/ADS-B, Using Statistical Features Extraction and Boosting, IRS, Ulm, Juin 2019 R. Ginoulhac, F. Barbaresco & al, Coastal Radar Target Recognition Based On Kinematic Data (AIS) with Machine Learning, International Radar Applications of Geometric and Structure Preserving Methods Conference, Toulon, Sept. 2019 OPEN 13 Cambridge University – Newton Institute, 03/12/19 Motivations for Lie Group Machine Learning ENS Ulm 1942 www.thalesgroup.com OPEN Lie Group Machine Learning for Drone Recognition ▌ Drone Recognition on Micro-Doppler by SU(1,1) Lie Group Machine Learning Verblunsky/Trench Theorem: all Toeplitz Hermitian Positive Definite Covariance matrices of stationary Radar Time series could be coded and parameterized in a product space with a real positive axis (for signal power) and a Poincaré polydisk (for Doppler Spectrum shape). Poincaré Unit Disk is an homogeneous space where SU(1,1) Lie Group acts transitively. Each data in Poincaré unit disk of this polydisk could be then coded by SU(1,1) matrix Lie group element. Micro-Doppler Analysis can be achieved by SU(1,1) Lie Group Machine Learning. ▌ Drone Recognition on Kinematics by SE(3) Lie Group Machine Learning Trajectories could be coded by SE(3) Lie group time series provided through Invariant Extended Kalman Filter (IEKF) Radar Tracker based on local Frenet-Seret model. Drone kinematics will be then coded by time series of SE(3) matrix Lie Groups characterizing local rotation/translation of Frenet frame along the drone trajectory. Applications of Geometric and Structure Preserving Methods OPEN 15 Cambridge University – Newton Institute, 03/12/19 Drone Recognition by Lie