Aided Inertial Estimation of Wing Shape Leandro R. Lustosa, Ilya Kolmanovsky, Carlos E. S. Cesnik, Fabio Vetrano

To cite this version:

Leandro R. Lustosa, Ilya Kolmanovsky, Carlos E. S. Cesnik, Fabio Vetrano. Aided Inertial Estimation of Wing Shape. Journal of Guidance, Control, and Dynamics, American Institute of Aeronautics and Astronautics, 2021, 44 (2), pp.210-219. ￿10.2514/1.G005368￿. ￿hal-03203912￿

HAL Id: hal-03203912 https://hal.archives-ouvertes.fr/hal-03203912 Submitted on 21 Apr 2021

HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés.

Open Archive Toulouse Archive Ouverte (OATAO )

OATAO is an open access repository that collects the work of some Toulouse researchers and makes it freely available over the web where possible.

This is an author's version published in: https://oatao.univ-toulouse.fr/27437

Official URL : https://doi.org/10.2514/1.G005368

To cite this version :

Lustosa, Leandro R. and Kolmanovsky, Ilya and Cesnik, Carlos E. S. and Vetrano, Fabio Aided Inertial Estimation of Wing Shape. (2021) Journal of Guidance, Control, and Dynamics, 44 (2). 210-219. ISSN 0731-5090

Any correspondence concerning this service should be sent to the repository administrator: [email protected]

Aided Inertial Estimation of Wing Shape

Leandro R. Lustosa,∗ Ilya Kolmanovsky,† and Carlos E. S. Cesnik‡ University of Michigan, Ann Arbor, Michigan 48109-2140 and Fabio Vetrano§ Airbus Operations S.A.S., 31060 Toulouse, France https://doi.org/10.2514/1.G005368 Advanced large-wing-span aircraft result in more structural flexibility and the potential for instability or poor handling qualities. These shortcomings call for stability augmentation systems that entail active structural control. Consequently, the in-flight estimation of wing shape is beneficial for the control of very flexible aircraft. This paper proposes a new methodology for estimating flexible structural states based on extended Kalman filtering by exploiting ideas employed in aided inertial navigation systems. High-bandwidth-rate gyro angular velocities at different wing stations are integrated to provide a short-term standalone inertial shape estimation solution, and additional low-bandwidth aiding sensors are then employed to bound diverging estimation errors. The proposed filter implementation does not require a flight dynamics model of the aircraft, facilitates the often tedious Kalman filtering tuning process, and allows for accurate estimation under large and nonlinear wing deflections. To illustrate the approach, the technique is verified by means of simulations using sighting devices as aiding sensors, and an observability study is conducted. In contrast to previous work in the literature based on stereo vision, a sensor configuration that provides fully observable state estimation is found using only one camera and multiple rate gyros for Kalman filtering update and prediction phases, respectively.

0 Nomenclature s ∈ R = arbitrary wing arc length ∈ R ∈ R3×3 = direction cosine matrix sj = arc length of jth rate gyro D ∈ R H ∈ R3×3 = angular velocity to Euler angles rate trans- t = time instant wg ∈ R3 = jth rate gyro noise formation j 1×3 x ∈ R3N3M Hθ ∈ R = angular velocity to pitch rate transformation EKF = extended Kalman filter state 1×3 3 Hϕ ∈ R = angular velocity to roll rate transformation x^s ∈ R = local wing x axis 1×3 3 Hψ ∈ R = angular velocity to yaw rate transformation x^b ∈ R = body x axis ∈ Z y ∈ R3 = local wing y axis K = number of cameras ^s ∈ Z y ∈ R3 M = number of rate gyros ^b = body y axis ∈ Z z ∈ R3 N = number of mode shapes ^s = local wing z axis 3 Pb ∈ R = rigid-body angular velocity (x axis) z^b ∈ R = body z axis Γ ∈ R P^b ∈ R = rigid-body rate gyro output (x axis) = wing dihedral angle P ∈ R = jth station angular velocity (x axis) δθ ∈ RN = twist modal amplitudes error j δθ ∈ R P^ ∈ R = jth rate gyro output (x axis) i = ith twist modal amplitude error j δϕ ∈ RN = anhedral modal amplitudes error pk∕l ∈ R3 = position of marker k with respect to camera l δϕ ∈ R = ith anhedral modal amplitude error Q ∈ R = rigid-body angular velocity (y axis) i b δψ ∈ RN = sweep modal amplitudes error ^ Qb ∈ R = rigid-body rate gyro output (y axis) δψ i ∈ R = ith sweep modal amplitude error Qj ∈ R = jth station angular velocity (y axis) 3 εj ∈ R = jth rate gyro drift ^ ∈ R = jth rate gyro output (y axis) × Qj Θ ∈ RM N = modal to Euler pitch rates transformation ∈ R Rb = rigid-body angular velocity (z axis) θ ∈ R = local twist angle N R^b ∈ R = rigid-body rate gyro output (z axis) θ ∈ R = twist modal amplitudes ∈ R Rj = jth station angular velocity (z axis) θi ∈ R = ith twist modal amplitude θ~ ∈ R R^j ∈ R = jth rate gyro output (z axis) i = ith twist mode shape r ∈ R3 = displacement vector Λ ∈ R = wing sweep angle 3 Φ ∈ RM×N = modal to Euler roll rates transformation rf ∈ R = displacement vector in body frame ∈ R ϕ ∈ R = local anhedral angle s = wing arc length ϕ ∈ RN = anhedral modal amplitudes φ ∈ R3N = angular to modal rates transformation ~ ϕi ∈ R = ith anhedral mode shape ϕi ∈ R = ith anhedral modal amplitude Ψ ∈ RM×N = modal to Euler yaw rates transformation ψ ∈ R = local sweep angle ψ ∈ RN = sweep modal amplitudes ψ i ∈ R = ith sweep modal amplitude ψ~ i ∈ R = ith sweep mode shape 3 *Post-Doctoral Research Fellow, Department of Aerospace Engineering. ωb ∈ R = rigid-body ground truth velocity †Professor, Department of Aerospace Engineering. ω^ ∈ R3 = rigid-body rate gyro output ‡ b Clarence L. “Kelly” Johnson Professor, Department of Aerospace 3 ωj ∈ R = jth station ground truth velocity Engineering. ω ∈ R3 §Loads and Aeroelastics Engineer. ^ j = jth rate gyro output I. Introduction All computer vision algorithms mentioned above are run offline HE resulting increased structural flexibility in high-aspect-ratio after flight testing. A different approach in Ref. [12] pursued in-flight T wing designs progressively couples aeroelastics with flight estimation from active LED markers detection by analog electro-optic dynamics and generally gives rise to instability and/or poor handling receivers and allowed for continuous data sampling and processing. qualities. Stable and safe piloting calls for stability augmentation However, accurate calibration of analog systems is challenging and systems (SASs) that simultaneously account for elastic and tradi- prone to noise; thus, engineering practice usually favors digital sys- tional flight dynamics degrees of freedom. Consequently, in-flight tems. Nevertheless, after an extensive and complex calibration pro- estimation of wing shape is beneficial for the control of very flexible cedure, this system successfully provided real-time continuous-time aircraft (VFA). This paper presents a novel technique for wing shape data for a 0.44-scale model of a 17,000-lb fighter airplane. estimation in the presence of large wing deflections based on In-flight wing shape estimators for use in SASs must satisfy hard approaches from the field of inertial navigation. real-time computation constraints and high-bandwidth requirements, A state estimator is herein called a flight dynamics model-based thus prohibiting employment of standalone computer vision systems. A practical alternative often pursued in navigation systems is to resort estimator if its design requires a mathematical dynamic model of the to inertial sensors for high-bandwidth data while exploiting low- aircraft relating actuator values (e.g., ailerons deflection, flap con- bandwidth aiding sensors (e.g., global navigation satellite system figuration) to system state trajectory (e.g., velocity, attitude, struc- receivers, cameras) for bounding divergent estimation errors. In tural modal amplitudes). Such estimators are vehicle dependent, Refs. [13,14], extended Kalman filters (EKFs) were implemented, and their application to modified aircraft designs entails lengthy re- which combined data from flight dynamics models and inertial identification/tuning campaigns, e.g., wind tunnel testing, numeri- sensors to obtain accurate results for bending; unfortunately, in cal simulations, and flight testing. Additionally, even when a model Ref. [13] the estimator failed to characterize twist satisfactorily. is available, it is often not precise or observable enough to support Additionally, the techniques in Refs. [13,14] rely on an EKF formu- state observer-based controller design. To address this challenge, lation that precludes simple tuning due to lack of information on this paper proposes a model-free approach to wing shape estimation the statistics of the flight dynamics modeling errors. Finally, the using inertial sensors. The technique avoids strain gauges and fiber approach in Ref. [13] assumed small wing deflections, thus hindering – Bragg gratings (FBG) [1] sensing due to their structural strain its application to VFA. displacement modeling requirements, although their use as com- To the best of the authors’ knowledge, this paper is the first attempt plementary aiding sensors is still feasible. to use the aided inertial navigation framework to estimate wing shape An alternative flight dynamics model-free approach is to employ without resorting to flight dynamics models, and its main contribu- sighting devices and stereo-vision tracking of visual references on the tion is a novel extended Kalman filter-based technique for wing wing. Whereas by using existing rivets or joint lines (natural candidates shape estimation based on distributed rate gyros and optional aux- for markers) Tagai et al. [2] failed to achieve satisfactory estimation iliary sensors. Because of the use of Kalman filtering, there is great accuracy on a Beechcraft Type 65 Queen Air, experiments with flexibility in the choice of aiding sensors type, precision, sampling artificial markers yielded subpixel precision.Alternatively, Meyer et al. rates, and number. Herein a case study with sighting devices illus- [3] employed the image pattern correlation technique (IPCT) to exploit trates the technique and its properties. The resulting real-time esti- surfaces covered with random patterns (i.e., texture) to estimate wing, mator 1) does not require a flight dynamics model of the aircraft, slat, and flap deformations on an Airbus A320. Rudnik and Schwetzler 2) has a straightforward tuning procedure, 3) has an adequate band- [4] exploited the same experimental apparatus to study low-speed high width for VFA control purposes, and 4) allows for large and nonlinear lift and significantly enhance the accuracy and reliability of the pre- wing deflections. diction of maximum lift for commercial aircraft. The remainder of the paper is organized as follows: Sec. II pro- Similarly, Kirmse [5] employed IPCT to estimate the wing shape poses a standalone rate gyro algorithm for wing shape estimation, and of a VUT100 Cobra airplane. During the flight, aircraft vibration Sec. III illustrates the proposed formulation of an extended Kalman caused significant camera misalignment that ground vibration tests filter to incorporate additional sensor data and enhance estimation failed to reproduce. To correct for misalignment during postflight precision. Section IV illustrates the approach using cameras as aiding analysis, Kirmse [5] proposed a technique for recalibration of extrin- devices. Section V investigates the sensor placement impact on filter sic parameters (e.g., camera position and orientation) based on visual observability, and Sec. VI summarizes the main findings of the paper. data. Camera misalignment also posed a challenge in Ref. [6] in the study of a high-altitude long-endurance vehicle. In the latter, intrinsic parameters remained practically constant during flight. II. Standalone Rate Gyro Wing Shape Estimation In computer vision, outdoor lighting poses additional problems. In A. Algorithm Derivation fact, Kurita et al. [7] devised the flight plan for experiments to This section introduces a technique for estimating wing shape using optimize lighting conditions to study static load deformations on a an array of rate gyros. Consider an aircraft equipped with M rate gyros Cessna Citation Sovereign. Interestingly, because the cameras were distributed along its right wing (see Fig. 1). Consider additionally, the installed inside the cabin, the intrinsic optical parameters were a aircraft inertial navigation system (INS) and, accordingly, its associ- function of cabin pressure due to window displacement (and there- ated global aircraft frame fx^b; y^b; z^bg. The global aircraft frame will fore compound lens reconfiguration) caused by fuselage deforma- be considered as the rigid-body reference towhich flexible frames will tion. Pang et al. [8] additionally warn of difficulties due to optimal be defined. The structural deflection is modeled by defining a defor- shutter speed values being dependent on lighting conditions, and mation reference line (DRL) fixed to the wing structure (see Fig. 1). light reflection from the wing, which was significantly reduced by Furthermore, the DRL is defined such that it contains all M rate gyros. application of a light dusting of nonreflective flat white paint for The DRL shape is parameterized by Euler angle functions ϕs; t, accurate optical tracking of high-intensity light-emitting diodes θs; t, and ψs; t, where s ∈ R denotes the arc length along the (LEDs). A similar approach using LEDs and videogrammetry was reference line (s 0 at the wing root). In this work, the angles ϕs; t, previously developed for the measurement of wing twist and deflec- θs; t, and ψs; t are called local anhedral, local twist, and local tion as a function of time of an F/A-18 research aircraft at NASA’s sweep angles, respectively, and they model the orientation of the local Dryden Flight Research Center [9]. However, the LED housings, wing frame fx^s; y^s; z^sg with respect to the body-fixed global especially outboard toward the trailing edge of the wing, were barely aircraft frame fx^b; y^b; z^bg. This formulation is similar to the Frenet– distinguishable from the background and called for manual selection Serret trajectory-based frames approach [15] and neglects extension of target image locations. Graves and Burner [10] used retro-reflec- effects. Therefore, to increase method accuracy, the DRL should be tive tape targets to ensure high-contrast imagery. This allowed for located as close as possible to the wing neutral axis, i.e., the axis along automated target detection employing blob analysis. Liu et al. [11] which there are no longitudinal stresses or strains. provide a comprehensive and systematic summary of photogram- Figure 2 further illustrates the proposed deformation description metric techniques for aerospace applications. and axes conventions. If all three local angles are zero at s s0, i.e., Fig. 1 Rate gyros placement and deformation reference line s definition.

( , ) ( , ) ( , ) ( , ) ( , )=0 ( , )

( , ) ( , ) ( , ) ( , )=0 ( , ) ( , )

( , ) ( , ) ( , ) ( , ) ( , ) ( , ) ( , ) ( , ) ( , ) ( , ) a) b) c) Fig. 2 Deformation examples.

ϕs0;tθs0;tψs0;t0, then the wing section axes at s The jth rate gyro yields angular velocity measurements such that s0 align with the aircraft body axes, as Fig. 2b suggests. Therefore, 0 1 0 1 zero local angles do not necessarily correspond to undeformed wing P^j P shape. Additionally, spatially constant angle functions model sweep B C B j C B ^ C @ A ε wg and dihedral angles (Λ and Γ, respectively), as Fig. 2a suggests. This @ Qj A Qj j j (4) y R paper defines the DRL as always tangent to the local ^s. Figures 2b R^j j and 2c illustrate orientation and right-handed sign conventions where z ψ −y θ −x ϕ the ^ ( ) ^ ( ) ^ ( ) rotation order is assumed from body to the ε wg ’ local wing section. This rotation order introduces a singularity in where j and j are, respectively, jth rate gyro s drift and noise. T ^ T θ 90°, which is unlikely to happen even in VFA. On the other hand, Additionally, ωj Pj;Qj;Rj and ω^ j P^j; Qj; R^j denote an alternative rotation order such as z^ (ψ)−x^ (ϕ)−y^ (θ) introduces a ground truth and measured angular velocities described in the jth ϕ singularity in 90°, which may occur in VFA. rate gyro local frame fx^s; y^s; z^sg, respectively. Similarly, ωb The jth rate gyro is installed at s sj, and it is considered aligned P ;Q ;R T and ω^ P^ ; Q^ ; R^ T denote ground truth and x y z b b b b b b b with f ^sj; ^sj; ^sjg without loss of generality. The corresponding measured angular velocities expressed in the global rate gyro frame angle functions are described as a superposition of N basis modes, i.e., fx^b; y^b; z^bg, respectively. Angular velocities are related to the deriv- XN atives of the Euler angles according to ϕ ϕ~ ϕ sj;t isj it for j 1;:::;M (1) 0 1 i1 ϕsj;t ∂ B C XN B θ C @ sj;t A θs ;t θ~ s θ t for j 1;:::;M (2) ∂t j i j i ψ i1 sj;t 2 3 XN 1 sin ϕs ;t tan θs ;t cos ϕs ;t tan θs ;t ψ ψ ψ 6 j j j j 7 sj;t ~ isj it for j 1;:::;M (3) 6 7 6 0 cos ϕs ;t − sin ϕs ;t 7 i1 4 j j 5 ϕ ϕ ϕ ϕ~ ϕ 0 sin sj;t cos sj;t where the symbols , i, and i denote deformation angle, mode cos θsj;t cos θsj;t shapes, and modal amplitudes, respectively. Furthermore, a column |{z} Hϕs ;t;θs ;t;ψs ;t vector of modal amplitudes is denoted by a bold symbol, e.g., 0 1j j j ϕ ϕ ;:::;ϕ T . The basis modes fϕ~ ; θ~ ; ψ~ : i 1;:::;Ng; are − b 1 N i i i B Pjt Dj tPbt C chosen to minimize N by accounting for most recurrent shapes B C × B − b C occurring during flight, for instance, by proper orthogonal decom- @ Qjt Dj tQbt A (5) position of finite element method simulations to varying loads or R t − DbtR t eigenmodes of a representative mass condition [16]. j j b b x y z x y z where Dj t is the direction cosine matrix from f ^bt; ^bt; ^btg to f ^sj;t; ^sj;t; ^sj;tg, given by 2 3 cos θ cos ψ cos θ sin ψ − sin θ b 4 − ϕ ψ ϕ θ ψ ϕ ψ ϕ θ ψ ϕ θ 5 Dj t cos sin sin sin cos cos cos sin sin sin sin cos (6) sin ϕ sin ψ cos ϕ sin θ cos ψ− sin ϕ cos ψ cos ϕ sin θ sin ψ cos ϕ cos θ

0 1 2 30 1 ϕ ϕ θ θ ψ ψ and sj;t, sj;t and sj;t, as a shorthand HT s ;tω − Dbω θ~ θ~ θ_ B θ 1 1 j b C 6 1s1 ··· Ns1 7B 1 C notation. B . C 6 . . . 7B . C T T . . . . . The first, second, and third rows of H are denoted by Hϕ, Hθ , and @ . A 4 . . . 5@ . A T T b Hψ , respectively, such that H s ;tω − D ω θ~ s ··· θ~ s θ_ θ M M j b |{z}1 M N M N Θ H ϕ θ ψ ϕ sj;t; sj;t; sj;t (17) h i ϕ θ ϕ θ T 0 1 2 30 1 1 sin sj;t tan sj;t cos sj;t tan sj;t (7) T b Hψ s ;tω − D ω ψ ψ ψ B 1 1 j b C 6 ~ 1s1 ··· ~ Ns1 7B _ 1 C B . C 6 . . . 7B . C h i @ . A 4 . . . . 5@ . A T ϕ − ϕ T b Hθ ϕsj;t; θsj;t; ψsj;t 0 cos sj;t sin sj;t Hψ s ;tω − D ω ψ~ s ··· ψ~ s ψ_ M M j b |{z}1 M N M N (8) Ψ (18) h i sin ϕsj;t cos ϕsj;t T Hψ ϕs ;t; θs ;t; ψs ;t 0 (9) Φ Θ Ψ j j j cos θsj;t cos θsj;t Note that , , and depend only on the choice of basis functions and sensor placement. Assuming M>Nand invertibility of ΦT Φ, ΘT Θ, and ΨT Ψ, the least-squares solutions of Eqs. (16–18) are To avoid clutter, the compound function notation Hϕsj;t, given by Hθsj;t, and Hψ sj;t will be used for the left-hand sides of Eqs. (7–9). Furthermore, note that 0 1 0 1 ϕ_ HT s ;tω − Dbω B 1 C B ϕ 1 1 j b C XN B . C T −1 T B . C ∂ @ . A Φ Φ Φ @ . A (19) ϕs ;t ϕ~ s ϕ_ t for j 1;:::;M (10) . . ∂ j i j i ϕ_ T ω − bω t i1 N HϕsM;t M Dj b 0 1 0 1 XN T ω − bω ∂ θ_ Hθ s1;t 1 Dj b θs ;t θ~ s θ_ t for j 1;:::;M (11) B 1 C B C ∂t j i j i B . C ΘT Θ −1ΘT B . C i1 @ . A @ . A (20) θ_ T ω − bω N Hθ sM;t M Dj b ∂ XN ψ ψ ψ 0 1 0 1 sj;t ~ isj _ it for j 1;:::;M (12) T b ∂t ψ Hψ s ;tω − D ω i1 B _ 1 C B 1 1 j b C B . C ΨT Ψ −1ΨT B . C @ . A @ . A (21) ω and thus, using Eq. (5) and the definition of j, one obtains ψ T ω − bω _ N Hψ sM;t M Dj b

XN HT ω − bω ϕ~ ϕ_ Integrating Eqs. (19–21) in time yields the standalone rate gyro ϕ j Dj b isj it for j 1;:::;M (13) i1 wing shape estimation (RG-WSE) algorithm. All estimated outputs from the RG-WSE algorithm are represented in this work with a hat on its corresponding symbol. For instance, ϕ^ t is the RG-WSE XN 1 estimate of ϕ t. Additionally, in state-space representation, HT ω − Dbω θ~ s θ_ t for j 1;:::;M (14) 1 θ j j b i j i Eqs. (19–21) can be recast as x_ φx; u, where x ϕ; θ; ψ i1 u ω ω ω ’ and 1;:::; M; b. Figure 3 illustrates the algorithm s input and output variables. Inputs are angular velocity measurements at M XN multiple wing locations and at the global aircraft frame, and outputs HT ω − bω ψ ψ ψ j Dj b ~ isj _ it for j 1;:::;M (15) are modal amplitudes in a truncated N-dimensional space of defor- i1 mation modes, i.e., ϕ^ , θ^, and ψ^ ∈ RN. which can be recast as the following systems of linear algebraic equations: 0 1 2 30 1 HT s ;tω − Dbω ϕ~ ϕ~ ϕ_ B ϕ 1 1 j b C 6 1s1 ··· Ns1 7B 1 C B . C 6 . . . 7B . C @ . A 4 . . . . 5@ . A HT s ;tω − Dbω ϕ~ s ··· ϕ~ s ϕ_ ϕ M M j b |{z}1 M N M N Φ (16) Fig. 3 Schematic of RG-WSE algorithm. B. RG-WSE Angles to Displacement Formulation Transformation III. RG-WSE Error Model for Extended Kalman Although the RG-WSE algorithm uses an angle-based parameter- Filtering ization of wing deformation, displacement-based coordinates are For fusing sensor output data with extended Kalman filtering, this often desired instead. This section presents how to transform between work uses a linearized model for RG-WSE errors. To best conform these descriptions. with RG-WSE, the state vector to be estimated is composed of modal Figure 4 illustrates both descriptions at the same point s sj.At amplitude errors and rate gyro drifts. Accordingly, any given time t, the displacement formulation rs; t maps a given point s units of distance away from the wing root along the wing x δϕ δϕ δθ δθ δψ δψ εT εT εT T shape. At a fixed instant of time t, note that EKF 1 ··· N 1 ··· N 1 ··· N 1 ··· M b (26) ∂r s; ty^s; t (22) δϕ δθ δψ ∂s where i, i,and i are defined as the difference between the ground truth and the RG-WSE-computed values. Therefore, r since s; t is parameterized by the arc length. In the body-fixed ^ δϕi ϕi − ϕi for i 1;:::;N frame fx^b; y^b; z^bg, integrating Eq. (22) yields δθ θ − θ^ i ;:::;N Z i i i for 1 s T T δψ ψ − ψ^ for i 1;:::;N (27) rfs; t Dϕs; t; θs; t; ψs; t 010 ds (23) i i i 0 or,inmatrixnotation,δϕ ϕ − ϕ^ , δθ θ − θ^,andδψ ψ − ψ^ . where rf is the displacement vector described in the body frame, and Assuming a constant rate gyro bias model (i.e., ε_j 0), linearizing Dϕ; θ; ψ is the direction cosine matrix given by Eq. (27) with respect to the EKF state and noise yields

2 3 cos θ cos ψ cos θ sin ψ − sin θ Dϕ; θ; ψ4 − cos ϕ sin ψ sin ϕ sin θ cos ψcos ϕ cos ψ sin ϕ sin θ sin ψ sin ϕ cos θ 5 (24) sin ϕ sin ψ cos ϕ sin θ cos ψ− sin ϕ cos ψ cos ϕ sin θ sin ψ cos ϕ cos θ

! " # where ϕs; t, θts; , and ψs; t are obtained according to Sec. II.A. φx;u − φx^;u^ ∂φ ∂φ ∂φ ∂φ ∂φ x ≈ ∂ϕ ∂θ ∂ψ ∂u x wg _EKF EKF 0 |{z}∂u C. RG-WSE-Based Wing Coordinates 3M×1 |{z}0000 G In previous sections, the wing deformation was kinetically A described as an elastica with torsion and two bending degrees of (28) freedom related to the local angles attached to points on the wing. All rate gyros were considered installed on the DRL. However, addi- where wg aggregates all the sensor noise vectors, i.e., wg tional sensors might be installed in stations disjoint from the DRL. In wg wg wg x^ u^ this case, an extrapolation model for positioning points not belonging 1 ;:::; M ; b , and all the Jacobians are evaluated at ; , to the deformation line is required. obtained here numerically by means of first-order finite differences. Each point on the wing can be described by the sum of a reference Similarly, a discrete-time model is obtained by point r s0;t in the deformation line in s0, and a Δp vector orthogo- f x ≈ x x Δ nal to the DRL. EKFtk EKFtk−1 _EKF t This work assumes that cross sections normal to the DRL are rigid g AΔt Ix t − G ∂w (29) under wing deformation. This implies that Δp is always written as |{z} EKF k 1 |{z}EKF Δx; 0; Δz in the fx^s0; y^s0; z^s0g basis, where Δx and Δz are Fk Gk constant in time. Therefore, every point on the wing at any given time can have its position described as the triple s0; Δx; Δz; these are where ∂wg is the discrete-time white noise equivalent of wg.The hereafter called wing coordinates, and relate to global frame position statistics of ∂wg should conform to rate gyro manufacturer specifi- pf according to cations. The prediction step of the EKF is then written as 0 1 x^ − F x^ − − Δx kjk 1 k k 1jk 1 0 0 0 0 T @ A T T pf rfs ;tD ϕs ;t; θs ;t; ψs ;t 0 (25) Pkjk−1 FkPk−1jk−1Fk GkQkGk (30) Δz where Qk, Pkjk,andPkjk−1 are the covariance of the rate gyros noise, the a posteriori error covariance matrix, and the a priori error covari- ance, respectively. This formulation provides a simple recipe for EKF tuning because Qk is directly related to rate gyro manufacturer spec- ifications. However, artificial process noise inflation might be neces- sary to account for linearization errors and aliasing in RG-WSE. Figure 5 illustrates the proposed aided inertial wing shape estima- tor architecture. Additional aiding sensors (e.g., cameras, strain gauges, accelerometers) could be added to the architecture by means of adequate modeling of their EKF observation equations. The next section illustrates an application using sighting devices as aiding sensors. Periodical correction of RG-WSE must be performed with Fig. 4 Angle-based and displacement-based descriptions. the acquired EKF estimates to decrease the effect of linearization f rCl where Dl and f denote the direction cosine matrix from body frame to camera Cl frame, and camera position with respect to wing root in body frame, respectively (see Fig. 6). All cameras are assumed fixed f in body frame and thus Dk is constant. Hence the camera measure- ment zk∕l ∈ R2 in pixels is given by

fpk∕l zk∕l Π (32) 100pk∕l

where f ∈ R is the focal length in pixels and Π is defined by 010 Π (33) 001

Given these assumptions, zk∕l depends exclusively on modal k∕l Fig. 5 Aided inertial wing shape estimator overall architecture. amplitudes ϕi, θi, and ψ i, i 1;:::;N. Therefore, z linearization yields

∂zk∕l ∂zk∕l ∂z k∕l zk∕l ϕ θ ψ − zk∕l ϕ^ θ^ ψ δϕ δθ δψ errors. Additionally, the estimated rate gyro drifts are calibrated |{z} ; ; ; ; ^ ∂ϕ ∂θ ∂ψ during flight by subtracting in software their biases based on Eq. (4). ≜Δzk∕l (34) IV. Camera-Aided Rate Gyro Wing Shape Estimation Consequently, an appropriate model for an observation equation is It will now be shown that integration in time of rate gyro drift and – given by noise in Eqs. (19 21) yields unbounded RG-WSE estimation errors. Therefore, additional sensors are required to prevent estimation errors ∂zk∕l ∂zk∕l ∂z k∕l ∂z k∕l ∂z k∕l ∂z k∕l Δzk∕l x from diverging in time. Auxiliary devices can be incorporated in the ∂ϕ ∂θ ∂ψ ∂ε ··· ∂ε ∂ε EKF RG-WSE framework by means of EKF observation equations. This |{z}1 M b work proposes an implementation of EKF update phase that employs Hk∕l δϕ δθ δψ ∕ mappings from modal amplitude errors (i.e., , ,and )to wk l (35) auxiliary sensors output. This section illustrates this for a sighting CAM device case study wherevisual markers are 1) placed at knownpositions wk∕l ∈ R2 on the wing, and are 2) tracked by cameras rigidly installed close to the where CAM is additive white Gaussian noise with statistics wing root. For a given camera, the auxiliary sensor output is defined as depending on camera quality, tracking algorithm performance, and – ∂zk∕l∕∂ε the difference in pixels between a measured marker position and the marker camera positioning, j0 for all j 1;:::;M, ∂zk∕l∕∂ε predicted marker position based on RG-WSE outputs ϕ^ , θ^,andψ^ ,and and b0. With a linear sensor model in hand, the EKF incorporates measurements through the classical Kalman filter camera–marker positioning. Multiple markers, Mk, k 1;:::;K, formulas [17]: tracked by multiple cameras, Cl, l 1;:::;L, are considered. y Δzk∕l − k∕lx A. EKF Observation Equations k H ^kjk−1 k∕l k∕l T The proposed rate gyro and sighting device data fusion technique Sk H Pkjk−1H Rk is based on tracking visual markers Mk, each with a priori known k∕l T −1 Kk Pkjk−1H Sk wing location sk;xk;zk. Recall from Sec. II.C that points fixed to x x y the wing are modeled assuming constant wing coordinates during ^kjk ^kjk−1 Kk k deformation. For each marker M , its position with respect to a k − k∕l k∕l 3 Pk I KkH Pkjk−1 (36) camera Cl, denoted by p ∈ R , is described in the Cl camera k coordinate frame using a pinhole camera model as wk∕l wk∕l T where Rk E CAM CAM is the measurement noise covariance, pk∕l −DfrCl Dfr s ;t and the mean of the measurement noise is assumed to be zero. In l f l f k ∕ 0 1 practice, artificial R inflation with respect to wk l might be neces- x k CAM B k C sary to account for additional sources of uncertainties, e.g., camera f ϕ θ ψ T B C Dl D sk;t; sk;t; sk;t @ 0 A (31) vibration. Rate gyro-based EKF prediction updates occur at a higher rate than zk EKF observation updates from the camera measurements. Thus

a) Camera axis definition and symbols b) Camera view Fig. 6 Camera-related symbols and frame definitions. between two updates from the camera measurements per Eq. (36) leading to ground truth values given by there will be multiple prediction updates from the gyro measurements ! ! ! per Eq. (30). This allows for heterogeneous sensor configurations ϕ1t Γ θ1t 0 with respect to sampling time, and favors nonuniform sampling. ϕt ; θt ; ϕ Γ ω θ In this work, camera tracking algorithms are not considered; 2t 0 sin t 2t 0 ! instead, it is assumed that image processing already took place with ψ Λ ∕ 1t precision represented by wk l . Examples of target-tracking image ψt (39) CAM ψ processing algorithms in the context of wing deformation are given in 2t 0 Refs. [8,10]. However, Pang et al. [8] build upon a stereo vision setup to update an EKF with estimated 3D positions of markers. The Ten rate gyros were installed in the wing according to the position- present camera-aided rate gyro wing shape estimation (CRG-WSE) ing defined in Fig. 7b. In the camera-aided case, additional visual approach eliminates the need for stereo vision because it uses the 2D trackers and sighting devices are installed as also depicted in Fig. 7b. projections of markers in the camera plane directly. As a conse- Their biases and noise characteristics are described in Table 2, and are quence, any number of cameras can be employed independently, purposely deteriorated with respect to the performance of low-cost rendering camera frame synchronization unimportant. inertial measurement units (IMUs) readily available in the market (e.g., the Xsens MTi 100-series) to reduce simulation time. B. Standalone and Aided Methods Comparison Figure 8 illustrates the simulation results for both standalone and camera-aided experiments. Modal amplitude and bias estimation By means of computer simulation, two experiments were con- σ ducted to illustrate the degrees of observability of standalone and errorsp are plotted with their respectivep 2 EKF covariances. More- δϕ2 δθ2 δψ2 δϕ2 δθ2 δψ2 aided observer algorithms. In both numerical experiments, right wing over, 1 1 1 and 2 2 2 are plotted sepa- shape estimation of an aircraft is pursued while performing a har- rately due to their distinct orders of magnitude. monic motion given by Note that standalone estimation has its errors diverging with time; thus, its use imposes an upper bound on the duration of missions. On ϕs; tΓ sΓ sinωt the other hand, the camera-aided estimator yields bounded errors. 0 However, for the given sensor imperfection statistics, rate gyro drift is θs; t0 negligibly estimated because its initial covariance is only marginally ψs; tΛ greater than the steady-state filter covariance. ω t0 (37) b V. Observability Analysis Γ Γ Λ Λ This section studies the impact of rate gyros and visual markers where 0 < t < tsim, 0 < s < b. Additionally, , 0, , and 0 are scalar constants given in Table 1. The overall trajectory of the wing number and placement on the proposed camera-aided estimator due to this deformation profile is illustrated in Fig. 7a. For algorithm precision. For that purpose, the numerical experiment in Sec. IV.B implementation, polynomial shape basis and N 2 are assumed, is repeated with different sensor configurations. Since N 2 was thus matching the number of modes in ground truth, and, accordingly, assumed, a minimum of two rate gyros is required for RG-WSE implementation, as discussed in Sec. II.A. A varying quantity from 2 ϕ~ sθ~ sψ~ s1 to 10 rate gyros in even numbers is examined assuming equally 1 1 1 spaced placement along the semispan b (see Fig. 9a). For each rate ϕ~ θ~ ψ 2s 2s ~ 2ss (38) gyro configuration, the visual markers quantity is also varied from 2 up to 10 in even numbers. Their placement starts at the wingtip (s b) and proceeds inward with 2 m spacing (see Fig. 9b). Addi- Table 1 Wing shape movement description in tionally, odd- and even-numbered markers are placed toward the simulation leading and trailing edges, respectively, but always 2 m away from the DRL. ΓΓΛ ω 0 tsim b Although a universal figure of merit for measuring observability is −0.09 rad 0.03 rad 0.36 rad 20 s 17 m π rad/s nonexistent [18], observability is evaluated herein through end-of- simulation EKF Pkjk covariances comparison. In this sense, smaller

o o o o o x o o x x o o x xo o: Rate-gyro (×10) x: Visual Marker (×5)

a) Deformation trajectory b) Sensor positioning Fig. 7 Simulation wing trajectory and sensor positioning.

Table 2 Sensor bias and noise statistics in simulation ε ε wg ωk∕l b j Rate gyro noise density ( j ) Camera noise standard deviation ( CAM) 0.1 ⋅ 1; 1; 1 deg ∕s 0.2 ⋅ 1; 1; 1 deg ∕s 0.01 deg ∕s∕Hz 0.01 m (assuming f 1) Fig. 8 Comparison of estimated error and predicted EKF covariance for standalone estimation (on the left) and aided estimation (on the right).

a) Positioning of M rate gyros, M {2, 4, . . . , 10}

b) Positioning of K markers, K {2, 4, . . . , 10}, = 2 m Fig. 9 Sensor positioning patterns for observability analysis experiments. covariances are associated with more observable, thus more precise, configurations. Figure 10 displays end-of-simulation EKF standard deviations for each proposed configuration. Because EKF covariances vary slightly across simulation runs, due to different EKF Jacobians values occurring due to RG-WSE random errors, Fig. 10 displays the 2σ t ˆ 100 s root mean square (RMS) values of Monte Carlo simulations with 10 Fig. 10 EKF values at the end of the simulation ( ) for different instrumentation quantities. independent runs for each configuration. Each independent simulation ε wg randomly samples different rategyros drift j and noise j values. As expected, observability increases with the number of sensors and in-flight wing deformations. The number of rategyros M should match markers. However, as the number of markers increases, the impact N, and there is little gain in adding additional units. Subsequently, the on system accuracy of increasing the number of rate gyros gets number of markers K is chosen to attain the desired performance or diminished. Additionally, modal error standard deviations decrease should be as large as the visual tracking algorithm allows without log-linearly with the number of visual markers. violating robustness and real-time requirements. Note that a system The comments above suggest some design guidelines for choosing with no camera updates would have estimation errors diverging, sensor quantity numbers. First, the designer determines N by deciding whereas a system with no rate gyros would have low bandwidth as how many (and which) basis modes best represent the bulk of the measurements are available only at camera sampling instants. Fig. 11 Shortfall in observability of δθ1 and δθ2 due to aligned visual markers.

Although RG-WSE imposes rate gyros installation on the DRL, Acknowledgment visual markers are allowed to be anywhere on the wing. However, This material is based upon work supported by Airbus in the frame some practical considerations must be respected. For example, of the Airbus-Michigan Center for Aero-Servo-Elasticity of Very Fig. 11 shows the outcome of a numerical experiment where five Flexible Aircraft (CASE-VFA). equally spaced visual markers are positioned on the reference deformation line (instead of closer to the leading and trailing edges as before). The noticeably degraded observability in θ, by a factor of References 10 in precision with respect to the other components (and previous [1] Ma, Z., and Chen, X., “Fiber Bragg Gratings Sensors for Aircraft Wing experiments in Sec. IV.B), reinforces the intuitive result that a line Shape Measurement: Recent Applications and Technical Analysis,” cannot fully define a rigid-body orientation. Therefore, rectilinear Sensors (Basel), Vol. 19, No. 1, 2019, p. 55. placement of markers should be avoided if wing twist estimation is https://doi.org/10.3390/s19010055 “ pursued. [2] Tagai, R., Nakakita, K., Kurita, M., and Nakajima, T., Wing Deforma- tion Measurement as Bases for In-Flight Aerodynamics,” AIAA Flight Testing Conference, AIAA Paper 2016-3355, 2016. VI. Conclusions https://doi.org/10.2514/6.2016-3355 [3] Meyer, R., Kirmse, T., and Boden, F., “Optical In-Flight Wing Defor- This paper proposed a novel technique for high-bandwidth real- mation Measurements with the Image Pattern Correlation Technique,” time estimation of wing shape using an array of rate gyros, and lays New Results in Numerical and Experimental Fluid Mechanics IX, edited the groundwork for tight EKF interfacing with additional sensors. by A. Dillmann, G. Heller, E. Krämer, H. P. Kreplin, W. Nitsche, and Tight EKF coupling eliminates suboptimal assumptions that often U. Rist, Vol. 124, Notes on Numerical Fluid Mechanics and Multidis- reduce precision and make tuning impracticable in reality. To further ciplinary Design, Springer, Cham, Switzerland, 2014, pp. 545–553. facilitate tuning, the proposed technique avoids resorting to flight https://doi.org/10.1007/978-3-319-03158-3_55 [4] Rudnik, R., and Schwetzler, D., “High Lift INflight VAlidation dynamics models because their uncertainties are difficult to charac- (HINVA)—Overview About the 1st Flight Test Campaign,” 32nd AIAA terize. Furthermore, design guidelines are provided for tuning and Applied Aerodynamics Conference, AIAA Paper 2016-0041, 2016. deciding on sensor number and placement. https://doi.org/10.2514/6.2016-0041 The proposed technique inherits the benefits of Kalman filtering, [5] Kirmse, T., “Recalibration of a Stereoscopic Camera System for In- e.g., multirate asynchronous aiding sensors integration, low compu- Flight Wing Deformation Measurements,” Measurement Science and tational cost, and fault detection, isolation, and recovery capabilities. Technology, Vol. 27, No. 5, 2016, Paper 054001. In particular, this work shows that tight optical system integration https://doi.org/10.1088/0957-0233/27/5/054001 yields a fully observable state estimation using a single camera. This [6] Sanches, A. C., Silvestre, F. J., Morales, M. A. V., and Hesse, H., “ contrasts with previous research, where loosely coupled integration Designing a Vision-Based System to Measure In-Flight Wing Defor- mation,” Proceedings of the 31st Congress of the International Council called for stereo vision approaches (thus two sighting devices or of the Aeronautical Sciences (ICAS), International Council of the Aero- more). Finally, much room remains for future research. This includes nautical Sciences, Bonn, 2018, pp. 1–7. the analysis of the case when gyros are not installed on the DRL, and [7] Kurita, M., Koike, S., Nakakita, K., and Masui, K., “In-Flight Wing impact on the accuracy at higher frequencies when the camera may Deformation Measurement,” 51st AIAA Aerospace Sciences Meet- fail to capture all the information. ing Including the New Horizons Forum and Aerospace Exposition, Aerospace Sciences Meetings, AIAA Paper 2013-0967, 2013. 1982, pp. 1088–1094. https://doi.org/10.2514/6.2013-967 https://doi.org/10.2514/3.44816 [8] Pang, Z. Y., Cesnik, C. E. S., and Atkins, E. M., “In-Flight Wing [13] Kotikalpudi, A., Danowsky, B. P., Schmidt, D. K., Regan, C. D., and Deformation Measurement System for Small Unmanned Aerial Gupta, A., “Real-Time Shape Estimation for a Small Flexible Flying- Vehicles,” 55th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Wing Aircraft,” AIAA SciTech 2019 Forum, AIAA Paper 2019-1818, Dynamics, and Materials Conference, AIAA Paper 2014-0330, 2014. 2019. https://doi.org/10.2514/6.2014-0330 https://doi.org/10.2514/6.2019-1818 [9] Burner, A. W., Lokos, W. A., and Barrows, D. A., “In-Flight Aeroelastic [14] Pang, Z. Y., and Cesnik, C. E. S., “Strain State Estimation of Very Measurement Technique Development,” Optical Diagnostics for Flu- Flexible Unmanned Aerial Vehicle,” AIAA SciTech 2016 Forum, AIAA ids, Solids, and Combustion II, Proceedings of SPIE, Vol. 5191, Soc. of Paper 2016-1955, 2016. Photo-Optical Instrumentation Engineers, Bellingham, WA, 2003, https://doi.org/10.2514/6.2016-1955 pp. 186–199. [15] Kreyszig, E., Differential Geometry, Dover, New York,1991, pp. 40–43, https://doi.org/10.1117/12.504157 Chap. 2. [10] Graves, S. S., and Burner, A. W., “Development of an Intelligent Video- [16] Medeiros, R. R., Cesnik, C. E. S., and Coetzee, E. B., “Computational grammetric Wind Tunnel Measurement System,” Optical Diagnostics Aeroelasticity Using Modal-Based Structural Nonlinear Analysis,” for Fluids, Solids, and Combustion, Proceedings of SPIE, Vol. 4448, AIAA Journal, Vol. 58, No. 1, 2020, pp. 362–371. Soc. of Photo-Optical Instrumentation Engineers, Bellingham, WA, https://doi.org/10.2514/1.J058593 2001, pp. 120–131. [17] Crassidis, J. L., and Junkins, J. L., Optimal Estimation of Dynamic https://doi.org/10.1117/12.449368 Systems, CRC Press, Boca Raton, FL, 2011, Chap. 3. [11] Liu, T., Burner, A. W., Jones, T. W., and Barrows, D. A., “Photogram- https://doi.org/10.1201/b11154 metric Techniques for Aerospace Applications,” Progress in Aerospace [18] Krener, A. J., and Ide, K., “Measures of Unobservability,” Proceedings Sciences, Vol. 54, Oct. 2012, pp. 1–58. of the 48th IEEE Conference on Decision and Control, IEEE Publ., https://doi.org/10.1016/j.paerosci.2012.03.002 Piscataway, NJ, 2009, pp. 6401–6406. [12] DeAngelis, V. M., “In-Flight Deflection Measurement of the HiMAT https://doi.org/10.1109/CDC.2009.5400067 Aeroelastically Tailored Wing,” Journal of Aircraft, Vol. 19, No. 12,