PARAMETER ESTIMATION FOR DYNAMIC SOARING WITH UNMANNED AERIAL VEHICLES

Duoneng Liu*,Xianzhong Gao** * China Aerodynamics Research and Development Center **National University of Defense Technology, China

Keywords: wind field estimation, unmanned aerial vehicles, dynamic soaring

Abstract constraint for the development of UAVs [4, 5]. However, albatrosses can fly long distances Unmanned Aerial Vehicles (UAVs) can even around the world almost without flapping significantly make use of wind shear to extract their wings, which means that they are flying energy by the flight technique named dynamic nearly at no mechanical cost [6]. After long time soaring. Wind shear in the field is the necessary observations, people gradually find that condition for dynamic soaring and has a albatrosses are particularly adept at exploiting a significant impact on the performance of special maneuver when they are flying in wind dynamic soaring. Thus, it is important for the shear. This maneuver is named as dynamic on-board auto-pilot of UAV to obtain the soaring, which is defined as a flying technique estimates of wind shear in real-time. In order to used to gain kinetic energy without effort by achieve this aim, a method to estimate the wind repeatedly crossing the wind shear [7]. field is proposed from a new perspective in this Like albatrosses, UAVs may be paper: the parameters of wind field can be programmed to perform dynamic soaring treated as unknown parameters in the flight autonomously to extract energy from wind shear. model of UAVs, and then, the problem to If the knowledge of wind field is assumed to be estimate the wind field can be transformed to already known, then the energy extraction estimate the unknown parameters in the flight process can be cast as a trajectory optimization model. For the problem to estimate the unknown problem [8], such as the works of Deittert et al. parameters in the nonlinear flight model [9, 10] and Sachs et al. [11, 12]. However, this exhibiting Gaussian behavior, a particle filter is assumption will not be available during flight. developed, and evaluated through its Moreover, there are currently no sensors that application on dynamic soaring. The results can be carried on a small UAVs to measure the show that the proposed particle filter framework 3D wind field ahead of the UAVs [13], thus it is can estimate the unknown parameters in wind important for the on-board auto-pilot of UAVs field more accurately, reliably and robustly than to be able to map or estimate the wind field in that of extended Kalman filter. real-time using only on-board measurements. In order to achieve this goal, many scholars 1 Introduction have paid great efforts to the on-line estimation method of the wind field. Lawrance et al. [14-16] In recent years, Unmanned Aerial Vehicles have described a method for wind field (UAVs) represent one of the most interesting estimation based on Gaussian Process technologies in aeronautics [1]. Meanwhile, the Regression. Langelaan et al. [8, 13] seek to major handicap associated with UAVs is the develop a method for wind field estimation that limited on-board energy capacity [2, 3], and the uses known structure to simplify estimation, and energy supplement has already been the crucial

1 DUONENG LIU, XIANZHONG GAO

the parameter estimation can be implemented by linear Kalman filter. Based on their pioneering works, in this paper, an estimator based on the flight model of UAVs is proposed to estimate the wind field in dynamic soaring. The parameter of wind field is treated as the unknown parameter in the model. The particle filter (PF) framework that estimates the unknown parameter of wind field with both constant and time-varying parameter is evaluated through its application in dynamic soaring, and the results of the extended Kalman filter (EKF) are presented for comparison. Fig. 1. Forces Acting on UAVs. are positions, μ is bank angle, L is force, D is 2 Models for Wind Estimation drag force, T is the thrust.  mV T D mgsin  mVW cos sin  (2)  2.1 Wind Shear Model mVcos  L sin mVW cos  (3)  In order to describe the wind shear above the mV  Lcos mg cos  mVW sin  sin  (4) water surface and at ridges, the exponential hV  sin (5) model, which has been adopted by Sachs et al.  [12, 17], is simply formulated as x VVhcos sin W  (6)

p yV  cos cos  (7) Vh W  VhHRR /  (1) The lift and drag force are expressed as where h is the altitude, Vw is the speed of wind follows which is the function of altitude, HR is the 1 2 L  SCVWL (8) reference height, VR is the wind speed at 2 reference height, p is the unknown parameter, 1 DSCV  2 (9) which denotes the reference value that is used to 2 WD indicate the strength of wind shear and to take the properties of the surface into account. The drag coefficient CD depends on the lift coefficient CL, yielding 2.2 Flight Model of UAVs 2 CCD DL0 KC (10) The same as Deittert’s model [10], a 6 degree of where C is the zero-lift drag coefficient, K is freedom (DOF) model of UAVs is adopted in D0 the induced drag factor. As described in [10], a this paper, and it is supposed that a linear wind fictitious force F appears in equations (2)-(7) shear is always blowing along x-axis. The DYN since the wind relative frame is not inertial: definitions of the forces and angles used in this model are shown in Fig.1 where V represents  XY FmVDYN W the projection of velocity of aircraft on XY- (11) p1 plane, Ψ is the azimuth measured clockwise mpVHRR// hH R V sin from Y-axis and γ is flight path pitch angle. The 6 DOF equations of motion for UAVs It can be seen clearly from equation (11) are given as follows, where the speed of an that the magnitude of FDYN is completely aircraft is modeled in a wind relative reference decided by the parameters and states of UAVs, frame and the position of aircraft is modeled in as well as the parameters of wind field. From an earth fixed frame, which is defined as the this viewpoint, the parameters of wind field can inertial reference frame. V is airspeed, x and y be treated as the unknown parameters in the

2 WIND SHEAR PARAMETER ESTIMATION FOR DYNAMIC SOARING WITH UAVS

flight model of UAVs, consequently, the EExx ˆˆ,  xxxxˆ T P(17) problem to estimate the wind field can be 0 0 0000 0 transformed to estimate the unknown The relevant measurements of aircraft are parameters in the flight model of UAVs. the local airspeed V measured by the airspeed meter and the position vector by the GPS in T 3 Augmented Particle Filter inertial frame, [x y h] . Considered here is the discrete measurement of the form, the measurement equation can be expressed as 3.1 Augmented System Formulation zhxnhx  ,   VxyhT (18) It can be known from equations (2)-(7) that the kkkkkkkk dynamic equation of aircraft is affected by the The measurement noise n is assumed to be wind shear. As such, it is possible to estimate a white, zero mean Gaussian random process the parameter of wind shear through the sensor that is uncorrected with the disturbance input: measurements of aircraft, such as airspeed meter, TT inertial navigation system (INS), Global EEnnnRwnkkkkk 0,   , Et 0 (19) Position System (GPS) and so on. In this work, the nonlinear dynamic equations (2)-(7) can be where Rk is the spectral density matrix of described by the ordinary differential equation measurement noise. For the given dynamic equation (13) and xfxu ttttt ,,  w  (12) measurement equation (18), the problem to estimate the wind shear can be formulated as the where the vector f is a nonlinear function of the problem of sequentially estimating the states xk state x, control input u and time t. and unknown parameter p in equation (1) when The discrete form of equation (12) can be the new observation zk is obtained. expressed as 3.2 Overview of Particle Filtering Algorithm xxkk1111Tk fxu kk,,1  w k(13) The PF estimates the state of the aircraft using where ΔT is the discrete time, and then the state available observations expressed in equation is defined as (18). It approximates a posteriori probability i T density function (PDF) by a set of particles, xk , xkkkkkkkkVhxyp  (14) i and their associated weights, ωk ≥0 and N i where the parameter pk has been augmented as ω 1, in a discrete summation form [19]:  i1 k states to the dynamic system, and then the procedure can be reduced to a pure states N ˆ ii pxzkk1: ω k xx k k (20) estimation problem [18]. i1 The control input is defined as

T where N is the number of particles. The weights ukLkk C   (15) ω are defined to be as follows:

iii The disturbance input w is a white, zero- ppzxkk  xx kk1 ωωii (21) mean Gaussian random process, i.e. kk1 ii qxxkk1 , z k T EwwwQkkj 0, Ekj  (16) where q xxii, z is a proposal distribution where E denotes the expected value of the  kk1 k function, and Q is the spectral density matrix, δ called as the importance density. The ii is the Dirac delta function. qxxkk1 , z k plays an important role in the The expected values of the initial state and performance of PF, and ideally it should be its covariance are assumed known:

3 DUONENG LIU, XIANZHONG GAO

N equivalent with the true posterior xi from the priori distribution p x , and  0i1  0  distribution pˆ xzkk1:  . However, it is not set ωi = 1/N, k = 1. known in general, and this will be further 0 discussed below.  Step2. Importance sampling: The value of each particle is estimated by With these particles xi and associated k the propagation of system model alone, i.e., it is weights, the estimated state vector xˆ k can be estimated before the observation is considered calculated as follows N xxuQii~,, f (28) NN  kkkki1  11  1 ˆˆiii xxxzkkkkkkp1: ω x (22) ii11  Step3. Weighting: The weight of each particle is evaluated by As discussed above, the optimal equation (26) when the new measurement is importance density function that minimizes the available, and normalize the weight as variance of importance weights is N ωωii / ω j (29) ii ˆ kk j1 k qpxxkk11:, z k  xz k k (23)  Step4. Re-sampling: This optimal importance density is not The estimation of each particle is corrected known in general, one popular suboptimal based on weighting information in equation (29) choice is the transitional prior [18]: i and re-sampling strategy. Then, set ω0 = 1/N. ii qpxxkk11, z k  xx kk (24)  Step5. Output: The estimation of state is calculated by If it is furthermore assumed that the equation (22), set k = k+1 and go back to Step 2. process noise is additive zero-mean Gaussian noise, then the PF can propagate the state estimate using the system model: 4 Wind Shear Parameter Estimation

ii xxuQkkkk~,, f 11  1 (25) 4.1 Dynamic Soaring Simulation Taking equation (24) into equation (21), The trajectory patterns of dynamic soaring are the new measurements can be combined with decided by the different terminal constraints. the propagated estimate of state in equation (25) Zhao [20] has classified the trajectories of to generate the updated estimate of state via the dynamic soaring into two patterns: loiter pattern weights: and traveling pattern. Taking the loiter pattern and its constrains as the example, the ωωii p zx i (26) kk1  kk trajectories of dynamic soaring can be N calculated according to the direct collocation xˆ  ωiix (27) kkk approach with the software program AMPL and i1 IPOPT [21, 22] and the the flight model’s In fact, the role of the weights ω in PF is parameters in equation (2)-(7) are given in similar to the gain in Kalman Filter. The weight Table 1. The trajectory is then tracked by a is defined in the way that it minimizes the UAV model using the LQR-based controller in estimation error covariance after the update, and the flight simulation performed in it can be used to combine the propagated Matlab/Simulink. By Introducing the process estimate with the new measurement. After noise and measurement noise into dynamic above discussing, the process of PF to estimate soaring flight, Monte Carlo simulation is the state in equation (12) can be summarized as performed to get simulative sensor  Step1. Initialization： measurements. Then the sensor measurements At time k = 0, generating initial particles are used for wind field estimation. 4 WIND SHEAR PARAMETER ESTIMATION FOR DYNAMIC SOARING WITH UAVS

Table 1. The Flight Model’s Parameters 80

Parameter Value Unit Explanation 70 m 81.7 kg Mass 2 60 Sw 4.2 m Wing area

K 0.045 - Lift induced drag factor 50 C 0.00873 - Parasitic drag coefficient D0 40 n max 5 - Max load factor o 30 μmax 75 Max bank angle

γmax 70 ° Max flight-path angle 20 Airspeed of aircraft V [m/s] CLmax 1.5 - Max lift coefficient 10 HR 10 m Reference altitude 0 5 10 15 20 25 Time [s] VR 0.637 m /s Wind speed at HR Fig. 2. The Measurement of the Airspeed with Constant Parameter. 4.2 Constant Unknown Parameter Estimation 250 In the simulated dynamic soaring process of this section, the true value of parameter p is assumed 200 to be a constant and equal to 1. The covariance 150 matrix of measurement noise in equation (18) is 100 set as follows: h [m] 50

T 2222 0 Enn  diag  nnnnVhxy  (30) -50 -400 100  diag 1 5 5 5 -200 0 0 -100 200 -200 y [m] The simulated measurements are shown in x [m] Figs. 2 and 3. The covariance matrix of process Fig. 3. The Measurement of the Position with Constant noise in equation (16) is set as follows: Parameter. T 22 22222 Eww diag  wwwwwwwVhxyp    In order to quantitatively analyze the (31) performance of PF and EKF, the estimated = diag 2 0.17 0.17 5 5 5 0.2 parameter’s mean value and σ bound of root During the estimation process the initial mean square error (RMSE) is defined as: n parameter in equation (1) is assumed unknown: 2 ˆ RMSEtptptjj   (33) Ep00000, Epp   Qp (32) j1 1 M The Qp0 in equation (32) is the covariance M  RMSE t (34) RMSEM  i of the initial parameter p0, and the Qp0 equals to i1 10 in the simulation, which indicates there are 1 M  RMSERMSE itM RMSE (35) less confidence to believe that the E(p0) in M 1 equation (32) is the true initial value of the i1 parameter. where n is the dimension of parameter p, here n To undertake a fair comparison, both the = 1; M is the times of realizations, here M = 40. EKF and PF are implemented in recursive form. The mean value of RMSE(t) in Fig. 6 The estimation results for parameter p in Fig.4 shows that, with an incorrect initial value of p0, and the relevant results of wind speed in Fig. 5 the PF can achieve significantly more accurate indicate that the PF converges to the actual estimates of p than the EKF. The 1σ bounds for value after about 2s. However, the slow the RMSE(t), based on 40 realizations, also convergence of EKF results in a poor indicate that the PF is more reliable and robust performance in estimating the wind field. than the EKF.

5 DUONENG LIU, XIANZHONG GAO

25 is normally determined by the state dimensions, The true value the non-linearity of the system, and the Estimated by EKF 20 properties of the unknown parameters [23]. In Estimated by PF general, the more particles, the better estimation 15 performance can be achieved such that a lower MRMSE with a tighter σ bound. However, the

10 more particles, the more computational load is required. Thus, it is important to keep a balance

5 between the performance and computational load for the application of PF. In order to do so, The estimatedThe wind speed instead to directly compare the computational 0 0 5 10 15 20 25 load of algorithms by their consumed CPU time, Time [s] the RMSE, the times of divergence based on 40 Fig. 4. The Estimated Wind Speed with Constant realizations, and the ratio of required CPU time Parameter. by EKF to that by PF with different number of 2 particles are summarized in Table 2, where the RMSE is defined as equation (36), and can be 1.5 used to indicate the estimation performance of 1 each filter.

1 T 0.5 The true value RMSE RMSEtdt (36) Estimated by EKF T 0 0 Estimated by PF The results in Table 2 are obtained on an -0.5 The parameter p Intel®Core™, i3-2100 [email protected] computer -1 running under Windows XP. They show that the RMSE of PF is inversely-proportional with the -1.5 0 5 10 15 20 25 number of particles and the CPU time is Time [s] proportional with the number of particles. When the number of particles is N = 200, the RMSE Fig. 5. The Estimated p Value with Constant Parameter. and CPU time of PF is almost equivalent to that of EKF. The RMSE of PF is greatly reduced 4 compared to EKF when N = 1000, however, the The mean value of RMSE with EKF ratio of PF’s CPU time to that of EKF will 3.5 The 1 bound of RMSE with EKF The mean value of RMSE with PF increase to 3.866. As the period of dynamic 3 The 1 bound of RMSE with PF soaring is about 20 seconds, and the 2.5 computational ability of on-board computer in UAVs is less than that of the personal computer, 2 how to reduce the computational load of PF and 1.5 make it suited for the application on the on- 1 board computer still need further research.

RMSE parameter of p 0.5 Table 2. The Estimation Performance

0 0 5 10 15 20 25 PF Time [s] Filters EKF N N N= N=100 Fig. 6. The Mean Value of RMSE Based on 40 =200 =500 1000 Realizations. RMSE 0.410 1.210 0.359 0.333 0.129 The other issue with the PF applied to the wind field estimation is to decide on the number CPU time 1 0.676 1.134 2.279 3.866 of particles. The appropriate number of particles

6 WIND SHEAR PARAMETER ESTIMATION FOR DYNAMIC SOARING WITH UAVS

4.3 Variable Unknown Parameter Estimation transformed to estimate the unknown During dynamic soaring process, the unknown parameters in the flight model. The application parameter p may vary over time, owing to the of particle filter (PF) for the on-line wind field specific correlation of the p at a particular estimation in dynamic soaring is then location and environment. The factors affecting introduced. The unknown parameter is treated the variation of parameter p have been studied as a state and incorporated into the flight model of UAVs to obtain robust estimates. The in [24, 25], the research results show that, the heating and cooling cycle of the air adjacent to proposed PF framework is evaluated on the the earth during the 24 hours of the day wind field with both constant and time-varying influences the parameter p significantly, on a parameters. The results show that the PF diurnal basis, wind speeds are higher during the framework can estimate the unknown daylight hours and drop to below-average parameters in wind field more accurately, values at night. reliably and robustly than that of Extended Thus, the following study considers an Kalman Filter. The implications of this study unknown, time-varying parameter p in wind are that particle filters are particularly attractive field. To simulate the time-varying effect, the for applications requiring on-line parameter actual value of p is assumed to follow a linear estimation. Further improvements of the decreasing trend, i.e.: importance density for PF and how to reduce the computational load to be suitable for the

pkkpTAk1  T (37) application on the on-board computer of UAVs are also under investigation. where kT is the time index. The initial value of p 2 is also set as shown in equation (32) whilst the 1.5 true values in equation (37) are p0 = 1 and A =

1/48. 1 The covariance matrix of measurement noise is the same as that in equation (30).The 0.5 covariance matrix of process and the initial parameter are the same as those in equation (31). 0 The parameter p parameter The The true value varying The estimation results for time- -0.5 Estimated by EKF parameter in Figs. 7 and 8 clearly indicate that the Estimated by PF superior results are attained for the PF. At the -1 0 5 10 15 20 25 beginning of estimation, the oscillations present Time [s] in the PF are mainly due to the large initial Fig. 7. The Estimated p Value with Variable covariance of parameter p0, which allows the Parameter. methodology to search over large regions for areas of high probability for the true value of the 12 The true value parameter. Although both the EKF and PF can Estimated by EKF obtain satisfactory estimates at the end of 10 Estimated by PF dynamic soaring, the PF follows the trend of 8 parameter p more quickly and precisely. 6

5 Conclusions 4 The method to estimate the wind field is 2 The estimated wind speed wind estimated The proposed from a new perspective in this paper. 0 The parameters of wind field are treated as the 0 5 10 15 20 25 unknown parameters in the flight model of Time [s] Unmanned Aerial Vehicles (UAVs), and then, Fig. 8. The Mean Value of RMSE Based on 40 the problem to estimate the wind field is Realizations.

7 DUONENG LIU, XIANZHONG GAO

References 2011 IEEE International Conference on Robotics and Automation, Shanghai, China, pp 2499-2505, 2011. [1] Filippis L D, Guglieri G, and Quagliotti F. A [16] Lawrance N R J. Autonomous soaring flight for minimum risk approach for path planning of UAVs, unmanned aerial vehicles. PhD Thesis, The Journal of Intelligent and Robotic Systems, Vol. 61, University of Sydney, Australia, 2011. pp 203-219, 2011. [17] Sachs G. Minimum shear wind strength required for [2] Langelaan J W. Biologically inspired flight dynamic soaring of albatrosses. Ibis, vol. 147, pp 1- techniques for small and micro unmanned aerial 10, 2005. vehicles. AIAA Guidance, Navigation and Controls [18] Olivier L E, Huang B and Craig I K. Dual particle Conference, Honolulu, Hawaii, AIAA Paper 2008- filters for state and parameter estimation with 6511, 2008. application to a run-of-mine ore mill. Journal of [3] Langelaan J W and Bramesfeld G. Gust energy Process Control, vol. 22, pp 710-717, 2012. extraction for mini- and micro- uninhabited aerial [19] Shao X, Huang B and Lee J M. Constrained bayesian vehicles. 46th AIAA Aerosciences Conference, Reno state estimation – a comparative study and a new Nevada, AIAA Paper 2008-0223, 2008. particle filter based approach. Journal of Process [4] Chakrabarty A. Flight path planning of UAV based Control, vol. 20, pp 143-157, 2010. on atmospheric energy harvesting. Master Thesis, [20] Zhao Y J. Optimal patterns of dynamic soaring. The Pennsylvania State University, USA, 2010. Optimal Control Appication and Methods, vol. 25, pp [5] Depenbusch N T. Atmospheric energy harvesting for 67-89, 2004. small uninhabited by gust soaring. Master Thesis, [21] Fourer R, Gay D M and Kernighan B W. AMPL: a The Pennsylvania State University, USA, 2011. Modeling Language for Mathematical Programming. [6] Sachs G, Traugott J, Nesterova A P, et al. Flying at Duxbury, MA: Thomson, 2002. no mechanical energy cost: disclosing the secret of [22] Wächter A and Biegler L T. On the implementation wandering albatrosses. PLoS ONE, vol. 7, p e41449, of a primal-dual interior point filter line search 2012. algorithm for large-scale nonlinear programming. [7] Richardson P L. How do albatrosses fly around the Mathematical Programming, Vol. 106, No. 1, pp 25- world without flapping their wings. Progress in 57, 2006. Oceanography, vol. 88, pp 46-58, 2011. [23] Chen T, Morris J and Martin E. Particle filters for [8] Langelaan J W, Alley N and Neidhoefer J. "ind field state and parameter estimation in batch processes. estimation for small unmanned aerial vehicles. AIAA Journal of Process Control, vol. 15, pp 665-673, Guidance, Navigation and Control Conference, 2005. Toronto, Canada, AIAA Paper 2010-8177, 2010. [24] Farrugia R N. The wind shear exponent in a [9] Deittert M. Dynamic soaring flight in turbulence. mediterranean island climate. Renewable Energy, vol. AIAA Guidance Navigation and Control Conference, 28, pp 647-653, 2003. Chicago Illinois, AIAA Paper 2009-6012, 2009. [25] Rehman S. Wind shear coefficients and their effect [10] Deittert M, Richards A, Toomer C A, and Pipe A. on energy production. Energy Conversion and Engineless unmanned aerial vehicle propulsion by Management, vol. 46, pp 2578-2591, 2005 dynamic soaring. Journal of Guidance, Control, and Dynamics, vol. 32, pp 1446-1457, 2009. [11] Sachs G and Casta O. Dynamic soaring in altitude region below jet streams. AIAA Guidance, Navigation, Contact Author Email Address and Control Conference and Exhibit, Keystone Contact author: Xianzhong Gao Colorado, AIAA Paper 2006-6602, 2006. Mailto: [email protected] [12] Sachs G and Casta O. Optimization of dynamic soaring at ridges. AIAA Atmospheric Flight Mechanics Conference and Exhibit, Austin Texas, Copyright Statement AIAA Paper 2003-5303, 2003. The authors confirm that they and their organization, hold [13] Langelaan J W, Spletzer J, Montella C and copyright on all of the original material included in this Grenestedt J. Wind field estimation for autonomous paper. The authors also confirm that they have obtained dynamic soaring. IEEE International Conference on permission, from the copyright holder of any third party Robotics and Automation, RiverCentre, Saint Paul, material included in this paper, to publish it as part of Minnesota , pp 16-22, 2012. their paper. The authors confirm that they give permission, [14] Lawrance N R J and Sukkarieh S. Autonomous or have obtained permission from the copyright holder of exploration of a wind field with a aircraft. this paper, for the publication and distribution of this Jouranl of Guidance, Control, and Dynamics, vol. 34, paper as part of the ICAS proceedings or as individual pp. 719-723, 2011. off-prints from the proceedings. [15] Lawrance N R J and Sukkarieh S. Path planning for autonomous soaring flight in dynamic wind fields. 8