Energy Optimization of a Hybrid Unmanned Aerial Vehicle (UAV)

Home , Electric aircraft, Unmanned aerial vehicle

Thesis

Presented in Partial Fulﬁllment of the Requirements for the Degree Master of Science in the Graduate School of The Ohio State University

Danielle Meyer, BS, BA

Graduate Program in Electrical and Computer Engineering

The Ohio State University

2018

Thesis Committee: Dr. Jiankang Wang, Advisor Dr. Mahesh Illindala

ABSTRACT

Unmanned Aerial Vehicles (UAV) have continued to receive attention from corpo- rations and governmental agencies due to their wide range of potential applications and hybrid nature. More Electric Aircraft (MEA) promise many beneﬁts (e.g., reduced weight, decreased fuel consumption, and high reliability) and their development continues to be the trend. Hybrid UAVs are an ideal prototype to implement con- cepts of aircraft electriﬁcation due to their small size and the DC nature of their power systems. However, papers addressing the energy optimization UAV electric power systems fail to consider the importance of high accuracy and computational speed. This thesis proposes an energy optimization method to enhance the energy durability of a

UAV through a novel approach integrating an optimization formulation and a detailed

UAV simulation model, with physical circuitry characteristics. This approach allows for increased computation efficiency while still capturing physical system constraints experienced during real world flight, which are complex and highly nonlinear due to aerial, thermal, and electrical dynamics. Optimization formulations created within this work are based on dynamic programming and moving-horizon model predictive control (MPC). The efficacy of this method is proven on a realistic UAV system.

Within the MPC formulation, various charge strategies are implemented and fuel consumption is calculated to provide insight into the trade-oﬀs inherent within the

UAV system, wherein battery discharging is required for high demand dash periods,

i but additional charge can only be supplied via increased output engine power. That is, minimal fuel consumption must be considered in light of the need for non-optimal output engine power to charge the battery such that a total mission can be completed.

Algorithmic considerations regarding horizon size for MPC and algorithmic enhancements, considering random loads and renewable generation capacity on-board the

UAV are presented. These results regarding enhanced algorithmic elements provide insight into the capability of the algorithm to function within a real-time environment and the beneﬁt of solar arrays to provide additional generation. Using MPC as the optimization technique of choice allows for the development of an algorithm capable of handling both missions with a deterministic load and within online implementations, as deterministic cases represent a downsized problem where algorithmic considerations can be studied and iterated to reach satisfactory online implementation. While this thesis approaches the problem from the perspective of UAV design, i.e., optimization for a deterministic load proﬁle, the algorithmic enhancements provided here represent initial steps towards online implementation.

ii Dedicated to my parents

iii ACKNOWLEDGMENTS

This thesis would not have been possible without the support of my advisor and mentor, Dr. Jiankang Wang, and my parents.

iv VITA

2009 - 2016 ...... BS, Electrical and Computer Engineer- ing, The Ohio State University 2009 - 2016 ...... BA, German, The Ohio State Univer- sity 2016 - present ...... MS, Graduate Research Associate, Department of Electrical and Com- puter Engineering, The Ohio State University

PUBLICATIONS

Research Publications

D. Meyer, J. Choi, J.K. Wang, ”Increasing EV public charging with distributed generation in the electric grid,” 2015 IEEE Transportation Electriﬁcation Conference and Expo (ITEC), Dearborn, MI, 2015, pp. 1-6.

D. Meyer, R. Alexander, J.K. Wang, ”A simple method for energy optimization to enhance durability of hybrid UAV power systems,” 2017 North American Power Symposium (NAPS), Morgantown, WV, 2017, pp. 1-6.

D. Mao, D. Meyer, J.K. Wang, ”Evaluating PEV’s impact on long-term cost of grid assets,” 2017 IEEE Power Energy Society Innovative Smart Grid Technologies Conference (ISGT), Arlington, VA, pp. 1-5.

In Press. D. Meyer, J.K. Wang, ”Integrating Ultra-Fast Charging Stations within the Power Grids of Smart Cities: A Review,” IET Smart Grid.

In Press. Ziran Gao, D. Meyer, J.K. Wang, ”Visualizing the Impact of PEV Charging on the Power Grid,” 2018 IEEE Transportation Electriﬁcation Conference & Expo (ITEC), Long Beach, CA, pp. 1-5.

v FIELDS OF STUDY

Major Field: Electrical and Computer Engineering

vi TABLE OF CONTENTS

Page

Abstract ...... i

Dedication ...... iii

Acknowledgments ...... iv

Vita...... v

List of Tables ...... ix

List of Figures ...... x

Dynamic Programming - Nomenclature ...... xii

Model Predictive Control - Nomenclature and Symbols ...... xiii

Chapters:

1. Introduction ...... 1

1.1 Prior Work ...... 2 1.2 Motivation ...... 4 1.3 Organization of Thesis ...... 6

2. Characterization of the UAV System ...... 8

2.1 UAV System Representation ...... 8 2.2 UAV Power Demand ...... 10 2.3 Energy Management Considerations ...... 12

vii 3. Dynamic Programming for UAV Energy Management ...... 13

3.1 Dynamic Programming ...... 13 3.2 Application of Dynamic Programming to UAV Energy Management 15 3.2.1 Algorithmic Implementation ...... 18 3.3 UAV System Model ...... 20 3.4 Results ...... 22 3.5 Conclusion ...... 24

4. Model Predictive Control for UAV Energy Management ...... 25

4.1 Model Predictive Control (MPC) ...... 25 4.2 Application of MPC to UAV Energy Management ...... 27 4.2.1 Algorithmic Development ...... 30 4.3 UAV System Model ...... 33 4.4 Results ...... 34 4.4.1 Energy Management for a 50cc Engine ...... 34 4.4.2 Energy Management for a 28cc Engine ...... 42 4.5 Conclusion ...... 54

5. Additional Algorithmic Considerations - MPC for UAV Energy Optimization 57

5.1 The Importance of Horizon Length within the MPC for UAV Energy Optimization ...... 58 5.2 MPC in the Presence of Load Uncertainty ...... 62 5.3 Integrating Renewable Generation ...... 71 5.4 Conclusion ...... 77

6. Conclusions and Future Work ...... 80

Bibliography ...... 85

viii LIST OF TABLES

Table Page

4.1 Fuel Consumption for Four Test Cases of the 50cc Engine ...... 35

4.2 Parameter, Horizon Length, and Saved Solution Values for Test Cases 1-4 (50cc Engine) ...... 38

4.3 Fuel Consumption Comparisons for MPC Output and UAV Model Sim- ulation (50cc Engine) ...... 39

4.4 Fuel Consumption for Two Cases of No Charging (28cc Engine) . . . 43

4.5 Fuel Consumption and Battery Size Comparisons for 24 and 36 Hour Missions ...... 48

4.6 Fuel Consumption for MPC Output and UAV Model Simulation (28cc Engine) ...... 51

4.7 Fuel Consumption of MPC Output and UAV Model Simulation in the Case of an Extended Mission (28cc Engine) ...... 54

5.1 Solution Speeds per Iteration for Various Horizon Lengths ...... 67

5.2 Double Junction Gallium Arsenide PV Product Characteristics [1] . . 73

ix LIST OF FIGURES

Figure Page

1.1 The proposed technique, wherein an optimization algorithm provides control decisions U to a realistic UAV model, which implements the decisions in a real-world scenario including non-linear dynamics. Per- formance is interpreted by operators, who provide algorithm alterations H...... 5

2.1 A simpliﬁed, block diagram representation of the UAV system under study...... 10

2.2 The predetermined load proﬁle developed based on a known mission. This proﬁle is used during the planning scope, i.e., determining the correct sizing of components to ensure minimum fuel consumption while completing the entire mission duration...... 11

3.1 Illustration of the possible paths the system can take under a ﬁxed

C-rate assumption, showing the transition from SOC1 to SOC2.... 18

3.2 UAV Simulink model developed to validate results of the algorithm according to dynamics within the operating system...... 21

3.3 A comparison of the DP Formulation simulation and the Simulink UAV model simulation...... 22

4.1 An illustration of the moving horizon of the MPC algorithm implemented. Predeﬁned values of saved solutions m = 5, 000 and horizon length h = 10, 000 are used. The algorithm starts at time zero and solves for solutions up to the black line. 5, 000 solutions are then saved, up to the green line. The horizon then shifts forward by 10, 000 and solves from the green line to the blue line. This process repeats until the entire mission is solved...... 31

x 4.2 The UAV Simulink model developed to validate results and for use within real-time implementation with the MPC formulation. This model considers the dynamics of the UAV...... 33

4.3 Energy Source Output results of the MPC algorithm for four cases of UAV operation: (i) no charging needed for the duration of the mission, (ii) aggressive charging behavior, (iii) a gradual charge pattern, and (iv) linear charging to meet future dash demand. Battery power is shown in blue, load power is shown in yellow, and engine power is shown in red...... 36

4.4 Battery SOC results of the MPC algorithm for four cases of UAV operation: (i) no charging needed for the duration of the mission, (ii) aggressive charging behavior, (iii) a more gradual charge pattern, and (iv) linear charging to meet future dash demand...... 37

4.5 A comparison of algorithmic and UAV model results for two cases. The ﬁrst case does not charge the battery, with battery power and SOC results shown in (a) and (b), respectively. The second case implements aggressive charging, with battery power shown in (c) and SOC shown in (d)...... 40

4.6 A comparison of algorithmic and UAV model results for two cases of charging. The ﬁrst case follows a gradual charge strategy, with battery power and SOC results shown in (a) and (b), respectively. The second case implements linear charging, with battery power shown in (c) and SOC shown in (d)...... 41

4.7 Energy source output power (ﬁrst row) and SOC (second row) results of the MPC algorithm for cases of no charging where (i) engine power is optimal and (ii) battery discharge equals zero during cruise periods. 44

4.8 Energy source output power (a) and SOC (b) results of the MPC algorithm for a case of gradual charging using the 28cc engine. This strategy results in fuel consumption of 33, 985...... 45

xi 4.9 A comparison of energy source output and SOC for two cases of linear charging. The ﬁrst case uses a d parameter comparable to the cost of engine power with energy source output results in (a) and SOC in (c). The second case, with energy source results in (b) and SOC in (d), uses a d value of a and a c value also equal to a, thus almost equally weighting all variables in the objective...... 46

4.10 An extended 36 hour mission proﬁle. Tests were conducted to determine additional fuel requirements for a longer mission...... 47

4.11 A comparison of energy source output and SOC for two cases of no charging for an extended 36 hour mission. The ﬁrst case allows discharge during cruise periods with energy source output results in (a) and SOC in (c). The second case, with energy source results in (b) and SOC in (d), does not allow discharge during cruise periods. . . . 49

4.12 A comparison of energy source output and SOC for two cases of no charging for an extended 36 hour mission. The ﬁrst case implements gradual charging with energy source output results in (a) and SOC in (c). The second case, with energy source results in (b) and SOC in (d), implements linear charging...... 50

4.13 A comparison of algorithmic and UAV model results for two cases of no charging. The ﬁrst case allows for battery discharge during cruise periods, with battery power and SOC results shown in (a) and (b), respectively. The second case does not allow discharge during cruise periods, with battery power shown in (c) and SOC shown in (d). . . 53

4.14 A comparison of algorithmic and UAV model results for two cases charging. The ﬁrst case implements aggressive charging, with battery power and SOC results shown in (a) and (b), respectively. The second case uses a linear charge strategy, with battery power shown in (c) and SOC shown in (d)...... 55

5.1 A comparison of energy source output in the face of varied horizon sizes: (a) 6, 625, (b) 10, 000, (c) 15, 000, and (d) 20, 000. With each increase in horizon length, the initial start of charge behavior before the second dash period occurs earlier...... 61

xii 5.2 Three possible load proﬁles that can be chosen by the algorithm. The proﬁle in (a) moves dash periods backwards by 600 seconds, (b) moves dash periods forwards by 600 periods, and (c) changes the length of dash periods present in the sample mission...... 64

5.3 Performance of the MPC algorithm in the face of uncertain load pro- ﬁles. Energy source output and SOC in the case of a minimum feasible horizon are shown in (a) and (b), respectively. In the case of a maximum feasible horizon, energy source output in shown in (c) and SOC is shown in (d)...... 69

5.4 MPC performance in the face of a more intensive load proﬁle. An unexpected, quick dash period occurs after the third dash period, but the MPC algorithm was able to increase engine power output to meet this unexpected demand...... 70

5.5 Four possible load proﬁles (Power (kW) v. Time (sec)) tested to determine algorithmic performance in the face of drastic load changes. Note that dash periods can be greatly extended or occur very frequently, depending on random selection...... 71

5.6 MPC output for the case of drastic load uncertainty. The UAV completes its mission, but presents undesirable trends in battery and engine power usage as it accounts for possible load demand...... 72

5.7 Initial tests of solar generation lead to oscillations in battery and engine power (a) and no noteworthy additional charging to the battery (b). . 75

5.8 Tests of 500W capable solar cells produces oscillations in battery power (a), but provides ancillary charge to the battery, increasing SOC (b) without increasing engine power...... 76

xiii Dynamic Programming - Nomenclature

k Time Index in Seconds

P ek Engine Power D Bk Battery Discharge Power C Bk Battery Charge Power

Loadk Load u1, u2 Binary Battery Control

SOCk Battery State-of-Charge

Ek Battery Energy E Battery Energy Lower Bound E Battery Energy Upper Bound P e Engine Power Lower Bound P e Engine Power Upper Bound

D Bk Discharging Battery Power Upper Bound C Bk Charging Battery Power Upper Bound γ Dynamic Programming Cost Coeﬃcient

xiv Model Predictive Control - Nomenclature and Symbols

k Time Index in Seconds

P ek Engine Power

P bk Battery Power

Ek Battery Energy

Fk Available Fuel

Loadk Load F Lower Bound on Available Fuel P e Engine Power Lower Bound P e Engine Power Upper Bound P b Battery Power Lower Bound P b Battery Power Upper Bound E Battery Energy Lower Bound E Battery Energy Upper Bound h Length of Horizon m Number of Saved Solutions a, b, c, d Model Predictive Control Cost Coeﬃcients

REk Renewable Energy Generated, A Uniformly Dis- tributed Random Variable

xv CHAPTER 1

Introduction

The past several decades have seen a rise in environmental concern across various industries, from waste management to manufacturing. The transportation sector, in particular, has been consistently making progress towards the goal of reducing fuel consumption, making transport a more environmentally friendly aspect of everyday life. Automobiles, trains, ships, and airplanes have begun the transition towards models that obtain power from either gas energy sources paired with electric energy sources (hybrids) or electric energy sources alone (pure electric). Though visions of hybrid and all-electric vehicles were introduced into the academic sphere as early as the 1970s, recent advancements have brought these developments to the focus of global research communities and industries [2]. The aviation industry has made signiﬁcant progress towards creating hybrid, or more-electric, aircraft. Boeing, an industry leader, introduced the state-of-the-art hybrid aircraft, the 787 Dreamliner.

This model removes almost 20 miles of wiring via the transition to a more-electric power system, i.e., one without pneumatic systems, reducing aircraft weight and increasing fuel eﬃciency [3]. This work will deﬁne those aircraft utilizing electric energy for all non-propulsive systems as more electric aircraft (MEA).

1 With regards to smaller aircraft, unmanned aerial vehicles (UAVs) have become a high interest topic for researchers, especially due to their wide range of applications, e.g., product delivery and homeland security. In fact, UAV market value was estimated at 13.22 billion USD in 2016, with a projected rise to 28.27 billion by 2022 [4].

UAVs can manifest in hybrid or all-electric forms, but larger models require a non- electric source of propulsion due to their weight. A transition from MEA to fully electric aircraft can provide further environmental benefits, e.g, an even lower fuel requirement, reduced dependence on fossil fuels, and increased reliability. However, in order to reach this goal, work must be done to optimize electric power systems such that conventional combustion engines are removed in favor of reliable electric alternatives. Hybrid UAVs provide an excellent prototype for investigations of this transition. Specifically, they are much smaller in size than a traditional commercial airliner and they utilize a purely DC power system. Both of these qualities lend themselves to familiar power system optimization techniques. Optimizing energy management schemes for multiple energy sources is an essential step towards further improvements in the field of aircraft electrification. Focusing on smaller-scale UAVs gives invaluable insight, as techniques can be derived and scaled for larger models.

1.1 Prior Work

Optimization schemes for hyrbrid electric vehicles (HEV), such as trucks, buses, and cars, have been investigated using a variety of techniques. These techniques, however, cannot be directly applied to the problem of energy management for UAVs due to fundamental diﬀerences in operation [5]. Techniques must be altered to account for the vast diﬀerence in size between a UAV and HEV, UAV operating system

2 environment and requirements, and the necessity of very fast decision making introduced by critical loads that operate on a magnitude of seconds within UAVs. For example, UAVs must be able to account for very quick bursts of increased power demand caused by the use of aerial photography equipment. Additionally, HEV can utilize braking to generate some of their battery charge. UAVs, on the other hand, do not brake and must utilize diﬀerent schemes for charging their battery in a way that does not aﬀect overall system performance.

A few recent works have proposed optimization methods for energy management among hybrid sources for UAVs in operation and design phase [6, 7]. Nevertheless, most works overlook two critical requirements of UAV Electric Power Systems (EPS).

First, energy optimization decisions need to be made in a very short time. Under critical conditions, such as in the presence of disturbances caused by failure or internal fault, slow decisions may not only greatly deteriorate energy performance, but also result in catastrophic consequences. For example, genetic algorithms for UAV power optimization [8] require large amounts of time to evolve toward the optimal, especially in the absence of population control. In the case of a 24 hour mission proﬁle and in the presence of energy optimization decisions made each second, the amount of potential candidates increases exponentially. Second, energy optimization methods need to accurately account for the physical constraints of the UAV EPS and any exogenous disturbance. Simpler formulations require less computation time and can be used by simplifying the structure of an operating system. Linear [9] and convex programming, for example, are well-established optimization techniques that are often applied to HEV systems [10] and microgrids, which are very similar to the UAV

EPS due to their size and DC nature. These methods, however, can ignore critically

3 important nonlinearities present during real-world ﬂight, which can greatly reduce the eﬃcacy of the optimization technique. Nevertheless, including these in an optimization formulation greatly increases the dimension, complexity, and computational time.

1.2 Motivation

This work seeks to improve upon past works related to energy optimization for hybrid UAVs. Though a hybrid UAV is used within this research, the methods can be adapted for larger aircraft or for alternative transportation sources, e.g., automobiles.

Past works in this field have used either (i) very complex formulations considering the non-linearities present in real-world flight situations, sacrificing computational efficiency and potential for real-time implementation, or (ii) formulations considering only power system critical operational constraints, e.g., power balance and output power bounds, sacrificing a consideration of realistic operational conditions. This work attempts to address these pitfalls using a formulation considering power system critical components, paired with operational testing in the form of a realistic UAV model. The structure of this technique is shown in Fig. 1.1.

The connection between the algorithmic output and the UAV model allows for algorithmic improvement based on real system needs. An open-loop conﬁguration can be used if users want to interpret the results of both components and provide improvements. For example, if the algorithm is failing to address an aspect of UAV operation that impacts realistic performance, the algorithm can be altered. Closed- loop implementation, however, more closely mimics the optimization of energy that

4 Figure 1.1: The proposed technique, wherein an optimization algorithm provides control decisions U to a realistic UAV model, which implements the decisions in a real-world scenario including non-linear dynamics. Performance is interpreted by operators, who provide algorithm alterations H.

occurs during a mission, wherein real system outputs are fed directly to the algorithm, allowing for real-time improvement via more accurate system state values.

This approach allows for the use of simple, but major constraints in the optimization algorithm, reducing overall computation time and complexity, paired with the implementation of these decisions in a realistic model that applies the complex, highly non-linear dynamics of the system.

The optimization algorithm itself must consider future energy requirements, indicating that the minimization of the overall trajectory is of interest. Dynamic programming, tree searches, and model predictive control can all be applied to problems of this type. All of these methods become computationally expensive in the face of aerial dynamics and other complex constraints, which are required for reliable results.

Thus, the solution structure used within this work allows for the use of trajectory minimization without sacriﬁcing computational speed, as the complexities within the

5 system can be handled via the UAV model. Though validation models are commonly carried out within optimal control research [11], few have applied these techniques to the simulation and analysis of UAV systems [12]. In addition, those pairing validation and algorithmic development tend to use each component in a disconnected way, failing to capitalize on the key beneﬁt of using them simultaneously. This work

fills this gap by conducting UAV system analysis for a two problems, each with a dif- ferent algorithm. First, a known path problem with a fixed-horizon mission profile is solved, i.e., a planning problem. This method can help designers to size UAV system components. Second, an unknown path problem, i.e., optimization in the face of an unknown load profile, provides initial insight into the applicability of this method for real-time implementation.

1.3 Organization of Thesis

In Chapter 2, the UAV under study is characterized, operational parameters and total UAV power demand are introduced, and a discussion of energy management considerations for the UAV problem serve as the groundwork for the optimization methodologies implemented. In Chapter 3, dynamic programming formulations and a related algorithm are used to solve a planning problem for the UAV. In particular, given a pre-defined mission profile and pre-set mission duration, the question of how the UAV should charge and discharge the battery is addressed. In essence, this problem answers the question of how large a battery should be to provide support for the UAV throughout the entirety of a pre-defined mission. The results of this approach are tested within a UAV model to assess performance.

6 Chapter 4 presents an alternative method for trajectory optimization, model predictive control. This approach utilizes a moving horizon to break the entire mission proﬁle into computationally eﬃcient sizes. Though overall optimality is not explicitly guaranteed, the convex nature of each smaller trajectory sub-problem guarantees global optimality for each sub-problem. This algorithmic approach is tested on a variety of test cases for two sizes of engine, a 50cc and 28cc model. Algorithmi- cally obtained results are then applied to a UAV model and compared to determine algorithmic performance within a more realistic system.

Chapter 5 introduces a variety of additional algorithmic considerations. Specif- ically, the importance of MPC horizon size is discussed and results are shown to illustrate horizon importance. The effect of randomness within the load profile is also tested. The introduction of random load profiles more closely follows a real-time scenario and introduces feasibility problems. These concerns are addressed via a systematic testing of appropriate horizon sizes to guarantee feasibility for the entirety of a mission. Finally, the performance of the MPC algorithm in the face of stochastic generation, introduced by renewable generation (solar panels), is introduced and initial results are discussed.

Finally, Chapter 6 concludes this work. Concluding remarks will focus on overall trends in algorithmic performance and results for various case studies. Future work will be introduced to build on the work detailed within this thesis.

7 CHAPTER 2

Characterization of the UAV System

This chapter outlines the structure of the UAV under study and its power requirements throughout a 24 hour mission. The structure of the system influences the constraints and bounds that appear in the optimization formulations developed in Chapters 3 and 4. The structure of the system introduces key trade-offs between energy efficiency and battery usage, discussed at the end of this chapter.

2.1 UAV System Representation

The hybrid UAV under study can be considered, in a simplified form, as consisting of two power sources, an internal combustion engine (ICE) and a battery, which provide power to a main 50V DC propulsion bus. The size of the ICE is specified by the UAV designers and its parameters, i.e., fuel map and output power bounds, are easily integrated into the developed optimization frameworks. The same is true of the battery, as designers can connect cells in parallel to increase overall capacity. Charge and discharge rates for the battery are specified within the optimization frameworks and discussed within Chapters 3 and 4, as are the ICE models being used.

The main propulsion bus is responsible for providing power to the rest of the components in the system, including the motor (the propulsion source) and the loads.

8 It is common practice in power system analysis to aggregate loads during study, as this aggregation can simplify computation procedures [13]. When under normal operating conditions, the energy losses are negligible, further indicating that an aggregation of loads is a reasonable consideration [13]. Thus, in order to simplify computation efforts and focus on high level energy management, this work aggregates all loads within the optimization framework. It is important to keep in mind, however, that these loads consist of payloads and flight critical loads, i.e., those systems required for UAV flight.

In extended analysis, these two types of loads should be distinguished, as ﬂight critical systems would be considered uninterruptible and can not be shut down to improve power system performance. This consideration may not be of critical interest for a small scale model UAV, but for larger aviation systems the interruptability of loads is of interest in the event of a power system fault.

A secondary 28V battery is also included within the system model and is used in the event of a fault in one of the main power sources that results in insufficient available energy for powering loads, specifically flight critical loads. The effects of this battery are not considered within this work, as it is utilized during fault scenarios, which are not investigated.

The simpliﬁed view of the UAV system, displayed in Fig. 2.1, also lends itself to viewing the purely DC system of the UAV as a type of DC micro-grid, which presents more diverse optimization and modeling opportunities. Further, the methods developed here and outlined in Fig. 1.1, could be applied to other DC systems and micro-grids.

9 Figure 2.1: A simpliﬁed, block diagram representation of the UAV system under study.

2.2 UAV Power Demand

The energy usage of the UAV is studied first with respect to a predetermined load profile, which is specified based on a known mission that is representative of a real-world UAV mission. In the planning scope, this predetermined mission profile is critical, as it allows for the components to be sized to achieve minimal fuel consumption and completion of the mission. Without knowing an expected mission profile, it would be difficult to size components. The predetermined mission profile is shown below, in Fig. 2.2.

The UAV experiences two types of ﬂight, a “cruise” period, in which total load demand is 1.8kW , and a “dash” period with a total load demand of 5kW . There

10 UAV Load Profile for a Known Mission 6

5.5

4.5

3.5 Power (kW) 3

2.5

1.5 0 1 2 3 4 5 6 7 8 9 Time (sec) ×104

Figure 2.2: The predetermined load proﬁle developed based on a known mission. This proﬁle is used during the planning scope, i.e., determining the correct sizing of components to ensure minimum fuel consumption while completing the entire mission duration.

are no limitations as to what can contribute to total load demand, so long as the

UAV’s two power sources are capable of supplying the power required. It is of note that the UAV spends most of its mission in the cruise period. Depending on the engine being used, the ICE could be capable of supplying the entire demand during these periods. The remainder of the mission, the dash periods, require the use of both the ICE and the battery to meet power demand, as the ICE alone is insuﬃcient.

Occasionally, there are periods where load demand is higher than 1.8kW and lower than 5.0kW , which corresponds to a payload with lower power requirements being used. Additionally, the takeoﬀ period, right at the start of the mission, requires

5.5kW of power, necessitating ICE and battery use.

11 2.3 Energy Management Considerations

In observing the system structure and load demands experienced during a mission, the trade-oﬀ between decreased fuel consumption, i.e., lower output engine power, and battery usage becomes apparent. That is, lowering the output of the engine to a level below the required cruise level of 1.8kW necessitates the discharging of the battery to meet the full demand. Though this is a reasonable means of operation, the system must ensure that there is enough battery capacity available to meet the power requirements of upcoming dash periods. During these dash periods, increasing engine output far above its optimal level is undesirable, thus increased battery power will be required during these periods to meet total demand. Eﬃcient engine operation during a dash period means the engine should not ramp up to a high output power level requiring increased fuel consumption if the battery can supplement total output power.

The engine, however, is the sole means of charge for the battery. This means that the engine must output more power than is required by loads in order to charge the battery. Inherent in this higher output power, is a higher fuel consumption. The optimization formulations developed must therefore take into account the balance between fuel consumption and a valid system state, i.e., all load demand is met.

Awareness of this trade-oﬀ is at the crux of all formulations developed. An interesting scenario presents itself in the form of renewable energy. UAVs can possess solar cell arrays capable of supplementing power available to the system. In this conﬁguration, considered in Chapter 5, the solar cells could provide a means of charge for the battery without requiring a higher output engine power.

12 CHAPTER 3

Dynamic Programming for UAV Energy Management

This chapter focuses on the application of dynamic programming (DP) and tree searching (TS) to the problem of UAV energy management optimization. Within this chapter, DP is ﬁrst introduced and its basic properties are discussed. Next, DP will be applied to the formulation of the UAV problem and used, in conjunction with the methodologies of TS, to develop an algorithm ensuring the UAV charges at a rate that allows it to complete a full mission. The UAV system model used to validate the results is introduced and results are provided and discussed. The formulations and results in this chapter have been previously published [14].

3.1 Dynamic Programming

Dynamic programming, ﬁrst introduced by Richard Bellman [15], is an optimization procedure, wherein a complex problem is broken down into simpler sub-problems.

The solutions to these sub-problems are stored and can be looked up again later if the same sub-problem is presented within a sequence of decisions. DP is applied to multi-stage problems, i.e., those problems where the decisions in one stage aﬀect the later stages. A central result of DP is the Bellman equation, named after its creator.

The Bellman equation provided in (3.1), restates the problem in a recursive form and

13 serves as a necessary condition for optimality. All problems to which DP is applied must be transformed into an equation of this type, often the most diﬃcult step in creating a DP formulation.

V (x0) = maxu0 [f(x0, u0) + βV (x1)], (3.1)

where a0 and x1 are subject to system constraints. V (x) represents the value function, which describes the best possible value of the objective function, as a function of state x. Value functions are used to ﬁnd the optimal action as a function of the state, i.e., the policy function. This policy function deﬁnes the optimal policy that should be taken for the optimization problem under study.

DP is both an optimization method as well as a computer programming method, but in the context of this research the optimization applications are of most interest.

In this research, DP refers to the simplification of a complex decision, i.e., minimizing fuel consumption over a 24 hour period, by breaking it down into a sequence of decisions. Value functions V1, ...VN are used to represent the state of the system from time k = 1, ..., N. The value of the final value function, calculated first, is determined by the state x at the final time and all other value functions are calculated backwards, using (3.1). It is of note that though DP can capitalize on problem structure to yield faster computation times and robust results, it does suffer from the curse of dimensionality [16].

DP has been applied to problems of many types, from optimal switching of power converters [17] to reinforcement learning [18]. Notably, DP methods have been applied to optimize energy optimization for electric vehicles. For example, the trajectory with minimal energy consumption for a desired route can be found using DP [19].

14 DP methods have also been applied to problems of energy management strategies seeking to optimize battery life [20] and home energy management [21]. As discussed previously, the UAV operating system is similar to a DC microgrid and microgrid energy management strategies have also leveraged DP strategies with good results [22].

With a breadth of research on the energy management of other forms of electriﬁed transport and microgrids demonstrating desirable results using DP approaches, this approach is investigated here to determine its performance within the UAV problem.

3.2 Application of Dynamic Programming to UAV Energy Management

With the goal of achieving minimum fuel consumption, while also ensuring that the totality of mission load demand is met, i.e., the entire ﬂight is completed, dynamic programming methods are leveraged to create an optimization formulation. The system is modeled via mixed-integer linear equations and constraints based on the system description outlined within Chapter 2. The overall formulation is created with an eye towards design purposes. That is, the formulation seeks to determine the proper sizes of components to ensure a mission can be completed. The dynamic programming system is created in a way that depicts the relationship between current and past states as

xk+1 = fk(xk, uk), k = 0, 1, ..., N, (3.2)

where xk denotes the previous time step and summarizes all past information necessary for future operation, up to the end of the time-horizon N. The control decision at time k, uk, inﬂuences the output of the current state xk+1 via a function f.

15 As discussed previously, the formulation must consider the entirety of the UAV

mission, indicating that it must be additive over time with regards to fuel consump-

tion. That is, the ﬁnal cost is the cost of the last state plus the costs of all previous

decisions. Thus, the objective function of the overall minimization problem can be

formulated as

N−1 X min f(xN ) + min f(xk) , (3.3) u∈U u∈U k=0

where f(xk) is a mixed-integer linear program (MILP). The optimization program

occurring at each time step is formulated as shown below.

min γP ek (3.4a) P e,u1,u2 D C s.t. P ek + u2Bk − u1Bk = Loadk (3.4b)

C D E ≤ SOCk−1 + ∆k(u1Bk − u2Bk ) ≤ E (3.4c)

P e ≤ P ek ≤ P e (3.4d)

C C 0 ≤ u1Bk ≤ Bk (3.4e)

D D 0 ≤ u2Bk ≤ Bk (3.4f)

u1, u2 ∈ {0, 1} (3.4g)

In the above MILP, P ek is the output engine power of the UAV, in kW, and γ

is a parameter relating output engine power to fuel consumption. Battery charge

C D and discharge powers at time k are given by Bk and Bk , respectively. Ek denotes the battery energy at time k and is calculated based on the previous state’s state-of- charge SOCk−1 and the battery discharge or charge power times a time constant ∆k,

16 equal to one second. u1 and u2 are battery charge and discharge control variables,

respectively. Note that (3.4e), (3.4f), and (3.4d) convey equipment constraints in the

form of operational bounds. Note that simple nomenclature descriptions are provided

in the Dynamic Programming Nomenclature section at the beginning of this thesis.

C D Within the implementation portion of this formulation, Bk and Bk are not considered as continuous variables in order to limit the search space. The battery must

be able to provide at least 1 kW of power during dash periods, based on the sam-

ple proﬁle and engine operation bounds; thus, ﬁxing charge and discharge C-Rates

C D results in a possible value for Bk and Bk of 0kW or 1kW . The constraint (3.4c) ensures that battery SOC is kept within allowable bounds, which are designed for

equipment safety, e.g., a lower bound of 25%. The power balance constraint, (3.4b)

ensures that the system makes decisions that meet total load demand and provides

an analytic representation of the trade-oﬀ between decreased fuel consumption and

increased battery use, discussed within Chapter 2. Speciﬁcally, at all times, the sys-

tem must discharge to meet demand, which requires a suﬃcient amount of available

power. This, in turn, requires that the engine output more power to charge the bat-

tery, resulting in higher costs earlier in the mission, else the system fails to complete a

mission due to an inability to meet demand. The outputs of this formulation are the

engine power and control values, which determine whether the battery is charging or

discharging at each time instant k. These outputs are fed into the UAV simulation platform, discussed later in this chapter.

17 3.2.1 Algorithmic Implementation

The inﬂuence of previous decisions on current feasibility and overall cost indicate that the entire mission must be considered during optimization implementation. If a static optimization formulation is implemented at each time step, the system will make decisions that lower the cost of the objective function with no regard for future feasibility requirements, leading to overall mission failure. The developed algorithm ensures that the entire mission is considered during decision making procedures.

The system can, at any time, be considered to have three possible paths under a

ﬁxed C-Rate assumption: charge 1kW , discharge 1kW , or do nothing. A graphical representation of these paths is shown in Fig. 3.1. If the battery charge/discharge is instead a continuous variable, i.e., it can charge or discharge any value between an upper and lower bound, the paths become inﬁnite.

Figure 3.1: Illustration of the possible paths the system can take under a ﬁxed C-rate assumption, showing the transition from SOC1 to SOC2.

18 Using Fig. 3.1, it is easy to see that the right-most path represents the most cost-eﬃcient path, as it results in the lowest total engine power. However, this path may be infeasible. The middle path results in the engine providing all required power throughout the mission. This path is also infeasible, as battery power is required during dash periods. Finally, the left-most path represents the least cost-eﬀective path with charging occuring at every time step through the mission, resulting in higher SOC and fuel consumption. Note that this tree becomes incredibly large in the face of a mission lasting for 86, 401 seconds, or 24 hours.

There are various methods that can be used to solve (3.4a)-(3.4g). One possible strategy involves the testing of all possible paths, from start to ﬁnish, and determining the optimal solution. However, testing all paths within the UAV problem is unacceptable, as the number of paths increases exponentially with ﬂight duration. To achieve low computation time, an algorithm is developed to function as a hybrid DP and tree search algorithm. This algorithm capitalizes on critical system and decision tree characteristics.

Inherent within available decisions is the lowest cost path, consisting of pure discharge; however, this is not possible if total load demand is equal to the lower bound of the engine or if there is no battery power available. If this option is not available, the next lowest cost option is to “do nothing”. The highest cost option, charging, is only implemented when the other options lead to infeasibilities, i.e., the system must charge to meet demand. Thus, an algorithm reaching inﬁnite cost (infeasibility) must proceed backward in time until charging is a feasible option and then charge to meet future demand. This algorithm is provided below. Results of this procedure are provided at the end of this chapter.

19 Algorithm 1 Optimal Energy Use Initialize SOC, Load Proﬁle j = 1 while j ≤ Total Mission Time do Calculate engine power, SOC, cost if cost(j) = ∞ and strategy(j) = discharge then j ← j − 1 strategy(j) ←do nothing else if cost(j) = ∞ and strategy(j) =do nothing then j ← j − 1 strategy(j) ← charge else if cost(j) = ∞ and strategy(j) = charge then while strategy(j) = charge do j = j − 1 end while strategy(j) ← charge end if end while

3.3 UAV System Model

The Simulink model used to validate the algorithmic outputs of the formulation used within this chapter can be viewed in Fig. 3.2. A detailed description of Simulink components used to create this model are not discussed in depth here, as the focus of this work is to discuss research related to the optimization formulations and their algorithmic implementations.

In essence, this model was created to account for the formulation’s consideration of limited constraints, i.e., those considering only operational bounds and power balance. The physical system, however, consists of highly nonlinear behaviors due to aerial, electrical, and thermal dynamics. Though these could be included within the optimization formulation, computation requirements and speeds become extensive and a formulation would be created such that optimality could not be guaranteed.

20 Figure 3.2: UAV Simulink model developed to validate results of the algorithm according to dynamics within the operating system.

The use of the UAV model provides a method for considering non-linear dynamics outside of the optimization formulation itself. The model in Fig. 3.2 consists of a

56cm3 engine and a 48V Lithium-Ion battery, with an auxillary battery for use in case of faults. Battery output voltages are converted to proper bus voltages and charging circuits are modeled using ideal current sources. The battery model used here is created according to the SimPowerSystems battery model [23] and sized according to the MEA battery model and power requirements conveyed through the sample load proﬁle [24]. It is of note that loads in this model are not aggregated, allowing for a consideration of variable characteristics, and are modeled as constant power loads.

The main controller provides the system with outputs derived by the optimization formulation.

21 Figure 3.3: A comparison of the DP Formulation simulation and the Simulink UAV model simulation.

3.4 Results

After running the algorithm to determine the required UAV model inputs of engine power and battery control decisions, the UAV model was run to determine the performance of the algorithm within a system containing non-linearities. The algorithm determines the desired system response at a time resolution of 1 second, for the entire mission proﬁle. The results of the algorithm are shown with a blue curve within Fig. 3.3.

Charge behavior can be viewed by referencing the engine power plot, provided as the middle plot in Fig. 3.3. As the plot shows, engine power increases directly before each increase in load power, which accounts for the charging of the battery

22 needed to meet the demand of an upcoming dash period. Thus, optimal decisions are to discharge only when necessary to meet demand and to charge only often enough that demand is met and the SOC remains above the predeﬁned lower bound of

25%. Therefore, the UAV stores and charges just enough to meet dash period load requirements. This results in minimal excess fuel consumption caused by unnecessary charge behavior.

Due to the fact that the charging and discharging rates were modeled as constant, the SOC plot shows linear charging patterns. The height of the battery SOC plot is directly related to the length of the upcoming dash period, as a longer dash period requires more available battery power. The battery was sized according to the largest dash period of 1, 800 seconds, beginning at time 31, 500. The spike in engine power at 30, 000 seconds occurs due to an increase in load demand. Battery power was not used at that time, as the load demand was less than the maximum engine output.

The orange curves in Fig. 3.3 correspond to the outputs of the UAV model.

Compared to the results from the algorithm, model performance indicates that implementation in a system with realistic energy sources, i.e., those with dynamics, leads to results that closely follow the predicted results from the algorithm. The minor difference in values of supplied engine power are present to account for additional losses in the system. Transients in the engine power plot are a result of the non-idealistic nature of the engine and battery models, for example, voltage variability. Transients in the UAV model last for 2 seconds and the engine ramps up to meet additional power requirements. This model assumes that the engine can response to any change in load demand instantaneously. Finally, the SOC plot results closely resemble each other for both output curves. The minimum value of SOC for the algorithm is 25%, but

23 the UAV model outputs a minimum value of 24.1%, due to the non-ideal parameters

within the SimPowerSystems battery model.

3.5 Conclusion

In this chapter, the concept of dynamic programming was applied to the problem

of UAV energy optimization and paired with tree search-based algorithm. This algo-

rithm seeks to obtain the minimum fuel consumption by implementing decisions that

capitalize on inherent system characteristics related to lowest cost paths based on a

ﬁxed C-rate. The outputs of this algorithm, when run through a realistic UAV model

considering dynamics of power sources, behave in an acceptable manner. The outputs

of both systems closely follow each other with the exception of engine transients.

The ﬁxed C-rate assumption, however, severely limits possible system decisions.

Further, engine ramp rates are necessary to ensure that engine decisions are, in

fact, physically feasible. This algorithmic implementation also only produces bat-

tery charging trend of only linear increases or decreases in SOC. This formulation and algorithm were improved to allow for varied charging patterns, a more realistic consideration. Fuel consumption is not explicitly tracked algorithmically, which could lead to the system choosing a path that is not actually feasible for a ﬁxed quantity of fuel. These downfalls are addressed in later chapters. Regardless, this formulation provides an initial step for investigation of the energy optimization problem and indicates valuable research directions that are the focus of the following chapters.

24 CHAPTER 4

Model Predictive Control for UAV Energy Management

Model predictive control (MPC) and its application to UAV energy optimization are discussed within this chapter. After introducing the main components of an MPC formulation, a UAV speciﬁc formulation is derived. This formulation is then used to create an algorithm, which is detailed here. A new UAV system model is developed and used within the remainder of the work discussed to test algorithmic outputs.

Results for various cases of the planning problem are presented and discussed.

4.1 Model Predictive Control (MPC)

Model predictive control is an optimization and control technique, in which an optimal control problem is solved at each time instant, i.e., the sampling instant.

MPC typically utilizes a moving horizon, over which optimization of a subproblem is conducted. Solutions of this optimization can be saved completely or in part and the time step at which the next optimization procedure occurs moves forward by the amount of solutions saved. This process is repeated until the entirety of the time period under study is solved for. Though the linear quadratic regulator (LQR) can be used for optimal control [25], MPC is implemented due to the presence of a receding time horizon and calculation of a new optimal solution within each time window under

25 study [26]. Thus, MPC allows for real-time optimization using constraints, evaluated numerically at each step. The procedure of horizon movement and saved solutions is detailed further within section 4.2.1 of this chapter.

MPC discretizes a continuous time system so that an optimal control problem can be solved [27]. Thus, MPC solves for an optimal state and control trajectory.

Though, it is of note that considering the entire time horizon as separate pieces, as in the problem of MPC with a moving horizon, does not allow for a statement of global optimiality. The best statement of optimality that can be guaranteed is for each subproblem, provided optimality conditions are met, e.g., convexity. State space representations of MPC are most common, due to the widespread application of MPC within control engineering and optimal plant control. Though the problem is presented in a diﬀerent context, the goal of optimizing a trajectory utilizing a moving horizon is very much the same.

MPC based approaches have been successfully applied to problems of energy management for hybrid vehicles. Due to the use of MPC for complex plant control, especially for plants with multiple inputs and strict system constraints, some have turned to these methods for the optimization of hybrid vehicles with forms of energy storage [28]. Speciﬁcally, determining the optimal energy split between a conventional combustion engine and electrical machines in a hybrid-electric automobile using a closed-loop MPC strategy yielded improvement over commercially available controllers [29].

26 4.2 Application of MPC to UAV Energy Management

Due to the requirement of trajectory minimization, an MPC formulation is created within this chapter. The system must consider the duration of the mission when making decisions, as future feasibility is directly related to past decisions. However, under a 24 hour mission proﬁle, the computation eﬀorts become extensive when considering the entirety of the mission at one time. An MPC formulation allows for the solution of a smaller horizon that moves in a predetermined increment, until the entire mission is solved. For example, a horizon of 10, 000 time steps can be solved and 1, 000 of those solutions can be saved. The formulation moves forward and then solves the horizon from k = 1, 000 to k = 11, 000 and again saves 1, 000 solutions, repeating until the entire mission is determined. This solution structure allows for the formulation to be implemented in a real-time manner and is commonly implemented in control research for real-time plant control. The MPC formulation developed for

UAV energy optimization is shown below.

27 h X min aP ek + b + c|P ek − P ek−1| + dP bk (4.1a) P ek,P bk i=0

s.t. P ek−1 − .5 ≤ P ek ≤ P ek−1 + .5 (4.1b)

Ek = Ek−1 − ∆kP bk−1 (4.1c)

Fk = Fk−1 − (aP ek−1 + b) (4.1d)

P ek + P bk = Loadk (4.1e)

Fk ≥ F (4.1f)

P e ≤ P ek ≤ P e (4.1g)

P b ≤ P bk ≤ P b (4.1h)

E ≤ Ek ≤ E (4.1i)

Note that simple nomenclature deﬁnitions are listed within the MPC Nomencla- ture and Symbols section at the beginning of this thesis. In the above formulation, system dynamics are represented in (4.1b) to (4.1d). Speciﬁcally, (4.1b) ensures that the ramp rate of the engine within each time interval of 1 second remains within operationally determined bounds of 500W/sec. The energy dynamics relating each time step are represented in (4.1c), where the battery energy at step k is equal to the energy at step k − 1 minus battery power at step k − 1. Battery power is not modeled in two separate charge and discharge components. Instead, a continuous variable that can range from the maximum charge rate, taken as a negative, to the maximum discharge rate is considered. For example, at a maximum charge rate of

10C and a maximum discharge rate of 20C, the battery power variable can range from −10 times the rated capacity to 20 times the rated capacity. This consideration

28 also leads to the removal of binary control variables due to processing of results af-

ter computation, discussed in detail within the algorithmic implementation section.

Finally, fuel dynamics are enforced by (4.1d), where the available fuel at k is equal to the previous amount of fuel minus the fuel consumed at the previous step. Power balance is assured at each time step by (4.1e) and the remaining constraints ensure that operational bounds on engine power and battery energy are respected during optimization.

Fuel consumption of the engine is modeled using a linear approximation of the fuel map for the 50cc engine and a convex quadratic function for the 28cc engine.

This linear approximation for the larger engine is shown in the formulation above, within (4.1d) and the objective function as aP ek +b. These are appropriately changed for test cases considering the smaller engine. In this case, the fuel map is a convex function and the objective function becomes

h X 2 min aP ek + bP ek + c|P ek − P ek−1| + dP bk. (4.2) P ek,P bk i=0 Other terms in the objective function are present to discourage oscillations in

engine output, achieved by a positive weight assigned to the absolute value of the

change in engine power between each time step, c|P ek − P ek−1|. Battery charging

is incentivized within this objective by a weighting factor d. If battery power is

negative, i.e., charging, the program can achieve a lower total cost to the objective

and simultaneously ensure the importance of fuel consumption is not ignored via

proper tuning. Finally, the length of the horizon is the value of h in the objective

summation.

This formulation provides a number of beneﬁts, all of which are related to its

property of convexity. The overall formulation, consisting of a quadratic or linear

29 cost function and a convex constraint set, allows for the use of convex solvers within the algorithm. Further, due to its convexity, we can be assured that the solution over each horizon is globally optimal.

4.2.1 Algorithmic Development

The selection of a horizon of optimization is a key choice, as it aﬀects how the solver views the necessity of charging behavior, both in terms of cost and in terms of feasibility. For example, a long horizon allows the solver to see future dash periods and begin charging at a cost-beneﬁcial rate, such that future demand can be met.

However, a shorter horizon may not “see” the future dash period and decide instead to avoid charging, which will lead to future infeasibilities. The selection of this horizon is discussed in detail within the following chapter. Here, the horizon is chosen according to desired performance outputs. A deeper discussion of horizon length is presented in Chapter 5.

After initializing the horizon length h, the length of saved data m must be chosen.

This feature implements a moving horizon, which is shown as a diagram within Fig.

4.1. Speciﬁcally, the solver determines the optimal solution over h and saves m solutions. In the next iteration the solver starts at the mth time step and again solves for the optimal solution over the horizon m + h. This process repeats until the entire mission proﬁle is determined. Here, the entire mission is known so there is no

“prediction” of load demand occurring. Studies within Chapter 5, however, introduce uncertainty within the load proﬁle, thus implementing an element of prediction.

After the length of horizon and amount of saved data have been deﬁned, the algorithm will determine the initial conditions on all variables for the horizon under

30 Figure 4.1: An illustration of the moving horizon of the MPC algorithm implemented. Predeﬁned values of saved solutions m = 5, 000 and horizon length h = 10, 000 are used. The algorithm starts at time zero and solves for solutions up to the black line. 5, 000 solutions are then saved, up to the green line. The horizon then shifts forward by 10, 000 and solves from the green line to the blue line. This process repeats until the entire mission is solved.

study. At the ﬁrst iteration, these values are equal to the initial SOC at mission start and zero for all other variables. During subsequent iterations, initial values are passed to the solver using the last value of the previous iteration, thereby ensuring continuity throughout all horizons. Finally, after all solutions are obtained, data post-processing occurs to generate control vectors for the UAV model.

31 Due to the convex nature of the optimization program, CVX is used to solve each

MPC subproblem. CVX is a MATLAB compatible package capable of specifying and

solving convex problems [30,31]. Though CVX can call upon four solvers, the Gurobi

solver was used for the results presented within this work. CVX is desirable as its user

interface allows for inputting the problem in a familiar notation, without the need to

create vast matrices, which is required for most built-in MATLAB solvers. CVX has

been shown to perform well with MPC problems [32], including those concerned with

unmanned aircraft [33].

After solving the energy optimization problem over the speciﬁed horizon, the

algorithm stores the required number of solutions m, if the solution is feasible. If the solution is infeasible, the length of horizon and/or saved data can be changed to acquire better performance. The objective parameters c and d, related to changes

in output power and charging behavior, can also be tuned to guide performance in

pursuit of desired proﬁles. This is discussed in detail within the results section of this

chapter. A full version of the algorithm is provided below.

Algorithm 2 MPC - Optimal Energy Use Initialize Length of Horizon h and Saved Data m, and Operational Bounds Current Time Step Initialized i = 1 while i ≤ Total Mission Length − m do Initialize SOC and Load Proﬁle for Time Under Study, = i + h Call CVX to solve MPC Formulation over Horizon h if Solution Feasible then Store P e, P b, F, E, from i to i + m else if Infeasible Solution then BREAK. Choose New h, m end if i ← i + m end while Post-processing of data to generate control inputs for UAV model

32 4.3 UAV System Model

The UAV model used with the MPC formulation diﬀers from the model presented within Chapter 3. Again, the detailed components implemented within the Simulink model are not discussed here, as it is not the focus of this work. Similar to the previous model, the UAV represented via Simulink was created to account for more complex dynamics that are not considered within the optimization formulation. This allows for a more holistic consideration of the UAV, without complicating the MPC algorithm and sacriﬁcing computational speed. Fig. 4.2 shows a diagram presenting an overview of the UAV model.

Figure 4.2: The UAV Simulink model developed to validate results and for use within real-time implementation with the MPC formulation. This model considers the dynamics of the UAV.

One aspect of interest when considering the performance of MPC outputs within this model is the presence of a tie-line connecting the engine and battery to the load.

The tie-line will induce some losses within the UAV that are not modeled in the

33 MPC formulation. Loads within the UAV are aggregated and modeled as constant current loads. Future iterations of this model will represent loads individually, as in the previous Simulink model.

The engine is modeled as constant current and its controls adjust to deliver the reference current, determined using engine power requirements. These requirements are provided to the model based on MPC outputs. The battery is modeled as constant voltage of 50V . A converter is not used within this model to convert the battery’s 74V output to the bus’s 50V level. Due to the fact that this is a high level model, a simple gain is used to convert battery power using a 98% eﬃciency. This, paired with the tie-line, cause slight diﬀerences between MPC outputs and UAV model performance, discussed in detail later in this chapter.

4.4 Results

The results in this section are split into two separate portions. The ﬁrst results section deals with energy optimization for a 50cc engine. These results are presented for four cases of charge behavior and compared to outputs generated by the UAV model outputs to determine the performance of the algorithm. Another set of test cases are conducted for a smaller, 28cc engine. These results are also compared to outputs generated by the UAV model, using algorithmic outputs as control inputs, similar to the DP case in Chapter 3.

4.4.1 Energy Management for a 50cc Engine

The fuel map for the 50cc engine can be approximated by a function of the form aP ek +b. This equation provides speciﬁc fuel consumption (SFC) and must, therefore,

3 be transformed as (aP ek + b)P ek to determine real fuel consumption in ccm . This

34 transformation allows for fuel consumption to be tracked and ensure that limits on

available fuel are not broken during mission solving procedures. Fuel consumption

values for each of the four test cases presented in this section are summarized in Table

4.1

Table 4.1: Fuel Consumption for Four Test Cases of the 50cc Engine

Case Number Fuel Consumption (ccm3) 1 28,241 2 32,141 3 31,763 4 29,576

The ﬁrst case studied for this engine includes a battery sized at 6.0384kW h.

This size of battery allows for the entire mission to be completed without charging, provided the engine operates with a lower bound of 1.8kW h. The rest of the cases utilize a battery sized at 3.0192kW h and require charging in order to complete the mission. For case 2, aggressive charging is pursued. For case 3, a more gradual charge pattern is obtained, though charging is still incentivized. Finally, case 4 results in charge behavior that serves as a way for the UAV to complete the mission. That is, linear charging occurs only enough to ensure available battery capacity is suﬃcient for required load demand. Fig. 4.3 illustrates the results of these test cases in terms of energy source output and Fig. 4.4 presents the results for each of these cases in terms of battery SOC.

35 Case One - Power Performance Case Two - Power Performance 6 6

4 4

2 2

Power (kW) 0 Power (kW) 0

-2 -2 0 5 10 0 5 10 Time (sec) ×104 Time (sec) ×104 Case Three - Power Performance Case Four - Power Performance 6 6

4 4

2 2

Power (kW) 0 Power (kW) 0

-2 -2 0 5 10 0 5 10 Time (sec) ×104 Time (sec) ×104

Figure 4.3: Energy Source Output results of the MPC algorithm for four cases of UAV operation: (i) no charging needed for the duration of the mission, (ii) aggressive charging behavior, (iii) a gradual charge pattern, and (iv) linear charging to meet future dash demand. Battery power is shown in blue, load power is shown in yellow, and engine power is shown in red.

Note that the output power plot in Fig. 4.3 can be used to observe charging

pattern in terms of the change in output power of the engine, which corresponds

directly to the change of input power to the battery, based on constraint (4.1e). The

aggressive charging pattern results in a quickly rising SOC that tapers oﬀ as capacity reaches the upper bound of 80%. The less aggressive, more gradual case of charging

(case 3) shows a slower approach to the 80% upper bound. Finally, in case 4, we

36 see no charging until additional capacity is required in order to ensure a successful

mission. The amount of additional capacity required is directly related to the height

of the SOC curve, i.e., more charge is needed for the ﬁnal dash period than the one before. It is of note that for the aggressive and gradual charging cases, the battery operates with an SOCinitial = SOCfinal incentive.

Case One - Battery SOC Case Two - Battery SOC 0.8 0.8

0.6 0.7

SOC (%) 0.4 SOC (%) 0.6

0.2 0.5 0 5 10 0 5 10 Time (sec) ×104 Time (sec) ×104 Case Three - Battery SOC Case Four - Battery SOC 0.8 0.8

0.7 0.6

0.6 0.4

SOC (%) 0.5 SOC (%) 0.2

0.4 0 0 5 10 0 5 10 Time (sec) ×104 Time (sec) ×104

Figure 4.4: Battery SOC results of the MPC algorithm for four cases of UAV operation: (i) no charging needed for the duration of the mission, (ii) aggressive charging behavior, (iii) a more gradual charge pattern, and (iv) linear charging to meet future dash demand.

37 Tuning parameter descriptions, horizon values, and saved solution sizes are provided in Table 4.2. It is of interest that between cases 2 and 3, only the value of the horizon has been changed. This change to horizon results in a diﬀerent charging pattern, as the MPC algorithm can limit increased costs over a longer horizon using a more gradual charge pattern. The eﬀect of horizon size will be further investigated within Chapter 5. Recall that parameter c discourages oscillations in engine power

P ek and parameter d incentivizes charging. Within this chapter, these parameter values are chosen such that desired performance is obtained. In future work, these will be adaptively chosen by the algorithm itself based on horizon length and the size of saved solutions.

Table 4.2: Parameter, Horizon Length, and Saved Solution Values for Test Cases 1-4 (50cc Engine)

Case Parameter, Horizon, and Saved Solution Values Number c d Horizon Length Saved Solutions 1 0 0 2,000 1,200 2 50 1,000 2,000 1,200 3 50 1,000 5,000 1,200 4 50 0 15,000 1,200

Algorithmic Performance within the UAV Model - 50cc Engine

The results above were input to the UAV model described in section 4.3 to analyze their performance within a system considering losses. Table 4.3 provides the results of total fuel consumption for all cases alongside the MPC results. Overall, the MPC

38 output underestimates fuel consumption by around 1, 500ccm3. This is due to the use of a linear approximation of the fuel map within the MPC algorithm. The UAV model uses the real concave fuel map function to calculate fuel consumption. In order to retain the convexity of the overall formulation, the linear approximation was created for use during the MPC algorithm. Though the results are not exact, the MPC formulation gives a good approximation of fuel use for planning purposes.

During real-time implementation, where fuel remaining functions as a constraint on operational decisions, the system can provide real-time updates to the amount of fuel used during the MPC simulation. This implementation strategy utilizes a closed loop approach to the strategy detailed in Chapter 1 and Fig. 1.1.

Table 4.3: Fuel Consumption Comparisons for MPC Output and UAV Model Simu- lation (50cc Engine)

Case Fuel Consumption (ccm3) Number MPC Output UAV Model Output 1 28,241 30,660 2 32,141 33,660 3 31,763 33,690 4 29,576 31,640

Trends in fuel use remain similar, with the no charging case using less fuel for both the algorithm and model outputs and the case of linear charging using the second lowest amount of fuel, as expected. It is of note, however, that while the MPC algorithm gives lower fuel consumption for a gradual case of charging, the UAV model actually consumes more fuel under this case than in the case of aggressive charging.

39 This indicates that it may be more beneﬁcial to quickly charge the battery of the real

UAV, rather than instituting gradual changes to engine power to charge the battery.

Engine power results are not presented, as the model directly implements MPC engine outputs.

4000 0.8 Algorithm Algorithm 3500 Simulink Simulink 0.7 3000

0.6 2500

2000 0.5

1500 State of Charge

Power Battery (W) 0.4 1000

0.3 500

0 0.2 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 Time(s) 104 Time(s) 104 (a) (b)

2500 0.85 Algorithm Algorithm 2000 Simulink Simulink 0.8

1500 0.75 1000

500 0.7

0 0.65 State of Charge

Power Battery (W) -500 0.6 -1000

0.55 -1500

-2000 0.5 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 Time(s) 104 Time(s) 104 (c) (d)

Figure 4.5: A comparison of algorithmic and UAV model results for two cases. The ﬁrst case does not charge the battery, with battery power and SOC results shown in (a) and (b), respectively. The second case implements aggressive charging, with battery power shown in (c) and SOC shown in (d).

Engine output values for each of the cases detailed in the section above were input within the UAV System Model, built using Simulink. Fig. 4.5 shows the performance

40 of the UAV model under output engine conditions speciﬁed by the MPC algorithm for Cases One and Two. Fig. 4.6 shows the performance of the model using outputs from the MPC for Cases Three and Four. Due to the fact that the engine power values determined by the MPC algorithm are directly provided to the UAV model, these values are the same for both and plots are not provided.

3500 0.85 Algorithm Algorithm 3000 Simulink 0.8 Simulink 2500 0.75 2000 0.7 1500

1000 0.65

500 0.6

0 State of Charge Power Battery (W) 0.55 -500 0.5 -1000 0.45 -1500

-2000 0.4 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 Time(s) 104 Time(s) 104 (a) (b)

3500 0.8 Algorithm Algorithm Simulink Simulink 3000 0.7

2500 0.6

2000 0.5 1500 0.4 1000 State of Charge Power Battery (W) 0.3 500

0 0.2

-500 0.1 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 Time(s) 104 Time(s) 104 (c) (d)

Figure 4.6: A comparison of algorithmic and UAV model results for two cases of charging. The ﬁrst case follows a gradual charge strategy, with battery power and SOC results shown in (a) and (b), respectively. The second case implements linear charging, with battery power shown in (c) and SOC shown in (d).

41 The slight differences between the algorithm and the model come from the non- idealities considered in the model. Some losses occur as the power flows through the tie-line and in the battery converter. Another source of discrepancy is the fact that the algorithm gives an input for every second, whereas the Simulink model runs in steps of 0.5s, always holding the input value from the algorithm until the next second. In some cases that causes the battery to be charged or discharged slightly more or less. This effect is visible within all battery power plots, with slight increases at change points of battery power values. This also manifests within the SOC plots, as the model outputs are slightly over the values of the MPC algorithm due to the slight increases in battery power implemented by the UAV model. Overall, however, the results indicate that the MPC algorithm does a sufficient job of estimating state values, even though losses and transients are neglected within the overall optimization formulation. Further, outputs for both the algorithm and UAV Model are very similar in all cases, indicating desirable performance.

4.4.2 Energy Management for a 28cc Engine

Within these test cases, the size of the engine is reduced to a 28cc engine and battery capacity varies based on the test case. Initially, a test case of no charging needed was investigated. The optimal engine operation point for the 28cc engine is

1.6284kW h, well below the lowest load demand value of 1.8kW h. Thus, there are two options for a case of no charging with regards to engine power. One such case allows for the engine to output its optimal power for the entire mission, requiring that the battery discharges for the entire mission. Such a test case requires a battery sized at approximately 13.5kW h. It is of note that this battery introduces more cost, in terms

42 of purchase price, and in terms of overall UAV weight. The second option requires the engine to output 1.8kW h during cruise periods and a much smaller battery, sized at

6.0384kW h. Fuel consumption for these two cases are presented in Table 4.4. Within both of these test cases, a horizon of 10, 000, saving 1, 200 solutions was used and parameters c and d are both set to 0.

Table 4.4: Fuel Consumption for Two Cases of No Charging (28cc Engine)

Case Description Fuel Consumption (ccm3) Battery Size (kW h) No Charging 24,300 13.5 No Charging/No Discharging 27,747 6.0384

Note that, as shown in Table 4.4, the fuel savings obtained through a battery almost double the size are 3, 447ccm3 of fuel. This may not be enough fuel savings to justify such a large battery, which contributes more weight to the overall UAV.

The addition of 37 battery cells to achieve the case of discharge during cruise periods results in about 2.96kg of added weight to the UAV. Though the fuel savings are of note, UAV designers must weigh the costs of a larger battery versus the cost of storing more fuel. Compared to the larger 50cc engine, which also implemented the strategy shown in Case Two, over 1, 000ccm3 less fuel is used, indicating that the smaller engine is more fuel eﬃcient. As expected, trends in the energy source output power plots, shown in the ﬁrst row of Fig. 4.7 are similar, with the engine operating at a lower point when allowed to discharge during cruise periods. This corresponds to the slight decrease of battery SOC that occurs during cruise periods, shown in

43 the lower left plot, while in the case of no discharging during cruise periods, battery power remains constant unless a dash period occurs.

Case One - Power Performance Case Two - Power Performance 6 6 Battery Battery Engine Engine 4 Load 4 Load

2 2 Power (kW) Power (kW)

0 0 0 2 4 6 8 10 0 2 4 6 8 10 Time (sec) ×104 Time (sec) ×104 Case One - Battery SOC Case Two - Battery SOC 0.8 0.8

0.6 0.6

SOC (%) 0.4 SOC (%) 0.4

0.2 0.2 0 2 4 6 8 10 0 2 4 6 8 10 Time (sec) ×104 Time (sec) ×104

Figure 4.7: Energy source output power (ﬁrst row) and SOC (second row) results of the MPC algorithm for cases of no charging where (i) engine power is optimal and (ii) battery discharge equals zero during cruise periods.

A gradual charge case was also implemented for the 28ccm3 engine. Within these tests, parameters c = 50 and d = 1, 000, the same conditions implemented for the larger engine. The horizon was also kept the same at 5, 000, but only 200 solutions were saved during each iteration. Results in Fig. 4.8 show this change in saved data results in a much more gradual change to output engine power and further indicate

44 the importance of the length of saved solutions and horizon. This strategy yields ﬁnal

fuel consumption of 33, 985. Though the engine is allowed to operate at its optimal output of 1.6284kW h, the MPC algorithm instead chooses to operate the engine at

1.8kW h, to avoid increasing costs due to discharging behavior, even when charging incentive d is decreased.

Energy Source Output and Load Profile Battery SOC for Duration of Trip 6 0.8 Engine Power Load Profile 5 Battery Power 0.7

4 0.6

3 0.5

2 SOC (%) Power (kW) 0.4 1

0.3 0

-1 0.2 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 Time (sec) ×104 Time (sec) ×104 (a) (b)

Figure 4.8: Energy source output power (a) and SOC (b) results of the MPC algorithm for a case of gradual charging using the 28cc engine. This strategy results in fuel consumption of 33, 985.

Finally, two cases of linear charging were run to determine algorithmic perfor-

mance under these conditions. The ﬁrst case, with energy source results shown in

Fig. 4.9(a) and SOC in Fig. 4.9(c), uses a d parameter that is equal in value to the value of a in the convex equation for fuel mapping aP e2 − bP e + c, more equally

weighting all costs. The second case, with energy source results in (b) and SOC re-

sults in (d), uses a d value equal to 0, i.e., charging is not incentivized. The ﬁrst case of

linear charging used 32, 424ccm3 of fuel, while the second case used less, 30, 799ccm3,

45 as expected due to less charging in the second case. Interestingly, though the 28cc

engine operates above its optimal engine power output, fuel consumption obtained

of the larger engine is only about 500ccm3 less. That is, though the smaller engine

operates above its optimal point, fuel consumption is still comparable to that of the

50cc engine output during case 4, detailed in section 4.4.1.

Energy Source Output and Load Profile Energy Source Output and Load Profile 6 6 Engine Power Engine Power Load Profile Load Profile 5 5 Battery Power Battery Power

4 4

3 3

2 2 Power (kW) Power (kW)

1 1

0 0

-1 -1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 Time (sec) ×104 Time (sec) ×104 (a) (b)

Battery SOC for Duration of Trip Battery SOC for Duration of Trip 0.8 0.8

0.7 0.7

0.6 0.6

0.5 0.5 SOC (%) SOC (%)

0.4 0.4

0.3 0.3

0.2 0.2 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 Time (sec) ×104 Time (sec) ×104 (c) (d)

Figure 4.9: A comparison of energy source output and SOC for two cases of linear charging. The ﬁrst case uses a d parameter comparable to the cost of engine power with energy source output results in (a) and SOC in (c). The second case, with energy source results in (b) and SOC in (d), uses a d value of a and a c value also equal to a, thus almost equally weighting all variables in the objective.

46 To investigate the eﬀect of longer missions on fuel consumption for the cases above, an extended mission of 36 hours was created and all cases above were repeated. This mission consists of an extra dash period and is shown in Fig. 4.10.

Extended Mission Profile 6

5.5

4.5

3.5

3 Power Demand (kWh)

2.5

1.5 0 2 4 6 8 10 12 14 Time (sec) ×104

Figure 4.10: An extended 36 hour mission proﬁle. Tests were conducted to determine additional fuel requirements for a longer mission.

Fuel consumption for normal and extended mission proﬁles are presented in Table

4.5 for ease of comparison. The linear proﬁle discussed for the extended mission was implemented using the strategy of setting both c and d equal to the a value in the convex fuel map of aP e2 − bP e + c, i.e., more equal weighting of cost coeﬃcients.

For both cases of no charging, one in which there is discharging occurring throughout cruise periods and one in which there is no discharging, the trends remain the same as in the case of a shorter 24 hour mission. Plots of these results are provided below, for easy visualization of MPC performance. Of note is the increase in battery size for the extended mission case, wherein the battery is allowed to discharge during cruise periods. This strategy requires a battery sized at around 20.5kW h. However,

47 Table 4.5: Fuel Consumption and Battery Size Comparisons for 24 and 36 Hour Missions

Case Fuel Consumption (ccm3) Description 24 Hour Mission 36 Hour Mission No Charging 24,300 36,450 No Charging/Discharging 27,747 41,620 Normal Charging 33,994 50,998 Linear Charging 30,799 47,018

as Fig. 4.11(c), shows, the battery could be reduced by a small amount and still allow

the mission to complete, as the ﬁnal SOC is slightly above the lower bound of 20%.

For the case of no charging and no discharging during cruise periods, the battery is increased to 8.65504kW h, which corresponds to using 13 additional battery cells than in the case of a 24 hour mission. Note that each cell weighs about 80 grams. These additional cells add 390 grams to the overall UAV weight. Comparing the cases of the 36 hour mission, the case with allowed discharge, using a 20.5 capacity battery,

results in 4.707kg added to UAV weight (58 additional cells), versus the case of no

discharge during cruise periods. This drastic increase in weight is not desirable, even

considering fuel savings of 5, 000ccm3.

Finally, cases of gradual and linear charging were tested for an extended mission.

Parameter, horizon, and saved solution values were set equal to those used during

the 24 hour mission. It was assumed that battery capacity would also stay the

same, 2.41536kW h, for each of these test cases. Thus, fuel consumption is the only

value that changes for each caes, with total fuel consumption results shown in Table

48 Energy Source Output and Load Profile Energy Source Output and Load Profile 6 6 Engine Power Engine Power Load Profile Load Profile 5 5 Battery Power Battery Power

4 4

3 3 2 Power (kW) Power (kW) 2 1

1 0

0 -1 0 2 4 6 8 10 12 14 0 2 4 6 8 10 12 14 Time (sec) ×104 Time (sec) ×104 (a) (b)

Battery SOC for Duration of Trip Battery SOC for Duration of Trip 0.8 0.8

0.7 0.7

0.6 0.6

0.5 0.5 SOC (%) SOC (%)

0.4 0.4

0.3 0.3

0.2 0.2 0 2 4 6 8 10 12 14 0 2 4 6 8 10 12 14 Time (sec) ×104 Time (sec) ×104 (c) (d)

Figure 4.11: A comparison of energy source output and SOC for two cases of no charging for an extended 36 hour mission. The ﬁrst case allows discharge during cruise periods with energy source output results in (a) and SOC in (c). The second case, with energy source results in (b) and SOC in (d), does not allow discharge during cruise periods.

4.5. Increasing mission length by 12 hours results in increased fuel consumption of

17, 004ccm3 for the case of gradual charging. This is expected, as the battery conducts two more rounds of charging to 80%. For the case of linear charging with all cost coeﬃcients equally weighted, fuel consumption increases by 16, 219ccm3. Results in

Fig. 4.12 are provided for visualization of charging strategies implemented for the

49 increased mission duration. As the results show, trends mimic those present for the 24 hour mission. This is expected, as the parameter weights are equal for both mission lengths. Even with an extended mission, the engine chooses to operate at its non- optimal value to meet load demand, thus removing the need for increased output later in a mission due to necessary charge behavior to meet load demand.

Energy Source Output and Load Profile Energy Source Output and Load Profile 6 6 Engine Power Engine Power Load Profile Load Profile 5 5 Battery Power Battery Power

4 4

3 3

2 2 Power (kW) Power (kW)

1 1

0 0

-1 -1 0 2 4 6 8 10 12 14 0 2 4 6 8 10 12 14 Time (sec) ×104 Time (sec) ×104 (a) (b)

Battery SOC for Duration of Trip Battery SOC for Duration of Trip 0.8 0.8

0.7 0.7

0.6 0.6

0.5 0.5 SOC (%) SOC (%)

0.4 0.4

0.3 0.3

0.2 0.2 0 2 4 6 8 10 12 14 0 2 4 6 8 10 12 14 Time (sec) ×104 Time (sec) ×104 (c) (d)

Figure 4.12: A comparison of energy source output and SOC for two cases of no charging for an extended 36 hour mission. The ﬁrst case implements gradual charging with energy source output results in (a) and SOC in (c). The second case, with energy source results in (b) and SOC in (d), implements linear charging.

50 Algorithmic Performance within the UAV Model - 28cc Engine

To test the efficacy of MPC outputs within the UAV model built within Simulink, updated with a fuel map corresponding to the 28cc engine, output engine power results were provided as inputs to the model. Battery power and SOC results for a provided engine profile were investigated to determine performance differences between algorithmic implementations and performance within a real-UAV system. Fuel consumption differences are summarized in Table 4.6. Results representative of trends found in all cases of implementation are provided in Fig. 4.13, which shows a comparison of algorithmic and UAV model results for battery power and SOC in the case where (i) engine power is optimal and (ii) battery discharge equals zero during dash period.

Table 4.6: Fuel Consumption for MPC Output and UAV Model Simulation (28cc Engine)

Case Fuel Consumption (ccm3) Description MPC Output UAV Model Output No Charging 24,300 36,170 No Charging/No Discharging 27,747 39,980 Aggressive Charging 33,685 43,320 Linear Charging 32,424 43,220 Linear Charging (Only to Meet Demand) 30,799 41,880

51 For the case of the 28cc engine, fuel consumption values vary between the UAV model and MPC algorithm within a range of 9, 000 and 12, 000ccm3. The MPC algorithm utilizes an approximated fuel map, mapping from output engine power to speciﬁc fuel consumption (SFC). This approximation is derived using engine speci-

ﬁcation sheets of SFC versus rotations per minute (RPM) and output power versus

RPM. The creation of this map can cause some differences between fuel consumption calculated by the algorithm and SFC calculated by the UAV model, which uses an approximation of the fuel map for SFC versus RPM. Thus, the UAV model uses a fuel map specifically provided by the manufacturer, while the MPC algorithm utilizes a fuel map derived from approximations of the specification sheets. For example, consider a case of 1.5kW h of output engine power. In this case, the UAV model will calculate fuel consumption based on a requirement of 7000RPM and find a value of 800ccm/kW h. The algorithm, however, will find a value of about 600ccm/kW h.

Future work will consider these diﬀerences and implement strategies to push MPC fuel consumption values closer towards those values found by the UAV model.

Performance of the MPC outputs were also tested within the UAV model for extended mission cases. Since two cases of no charging were presented in the case of a 24 hour mission, the two representative cases for an extended mission were chosen as the case of aggressive charging and linear charging. Trends in results for these cases are similar for both the 24 and 36 hour mission, thus these results give a complete view of UAV model performance in a variety charge strategies. Table 4.7 gives a comparison of fuel consumption for four charge strategies during an extended mission. Similar to the results in Table 4.6, fuel consumption is higher within the UAV model due to its use of the SFC versus RPM speciﬁcation sheet, discussed previously.

52 4000 0.8 Algorithm Algorithm 3500 Simulink Simulink 0.7 3000

0.6 2500

2000 0.5

1500 State of Charge

Power Battery (W) 0.4 1000

0.3 500

0 0.2 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 Time(s) 104 Time(s) 104 (a) (b)

0.8 3500 Algorithm Algorithm Simulink Simulink 3000 0.7

2500 0.6

2000 0.5 1500 State of Charge

Power Battery (W) 0.4 1000

500 0.3

0 0.2 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 4 Time(s) 10 Time(s) 104 (c) (d)

Figure 4.13: A comparison of algorithmic and UAV model results for two cases of no charging. The ﬁrst case allows for battery discharge during cruise periods, with battery power and SOC results shown in (a) and (b), respectively. The second case does not allow discharge during cruise periods, with battery power shown in (c) and SOC shown in (d).

As previously discussed, diﬀerences between UAV model and MPC performance are due to slight losses presented by the tie-line and battery converter. Further, the use of a 0.5 second time step causes the model to hold the input value from the algorithm until the next second, which sometimes causes the battery to be charged or discharged slightly more or less. This eﬀect is again visible within all battery power plots, with slight increases at change points of battery power values.

53 Table 4.7: Fuel Consumption of MPC Output and UAV Model Simulation in the Case of an Extended Mission (28cc Engine)

Case Fuel Consumption (ccm3) Description MPC Output UAV Model Output No Charging 36,450 54,250 No Charging/No Discharging 41,620 59,970 Aggressive Charging 50,998 64,770 Linear Charging 47,018 63,430

4.5 Conclusion

In this chapter, model predictive control was applied to the problem of UAV energy optimization. After creating a convex MPC formulation, CVX was used within

MATLAB to create an algorithm capable of implementing moving-horizon MPC.

This algorithm utilizes predetermined horizon lengths and saved solution sizes to solve for optimal energy usage within portions of the mission, repeating until the entire mission is completed. Due to the use of moving horizons, global optimally of the final solution cannot be guaranteed, though the convex properties of each individual problem guarantee global optimality for each subproblem. A UAV model was developed within Simulink and includes losses in the form of a tie-line and battery converter. The outputs of both systems, the algorithm and model, closely follow each other with the exception of slight differences in battery power, caused by the difference in time-step size between the systems. Fuel consumption values differ between the UAV model and MPC algorithm due to a difference in fuel maps. While the UAV model utilizes RPMs to derive SFC, the MPC algorithm utilizes a fuel

54 3500 0.8 Algorithm Algorithm 3000 Simulink Simulink 0.7 2500

2000 0.6

1500 0.5 1000 State of Charge

Power Battery (W) 500 0.4

0 0.3 -500

-1000 0.2 0 2 4 6 8 10 12 0 2 4 6 8 10 12 Time(s) 104 Time(s) 104 (a) (b)

3500 0.8 Algorithm Algorithm Simulink Simulink 3000 0.7

2500 0.6

2000 0.5 1500 0.4 1000 State of Charge Power Battery (W) 0.3 500

0 0.2

-500 0.1 0 2 4 6 8 10 12 0 2 4 6 8 10 12 Time(s) 104 Time(s) 104 (c) (d)

Figure 4.14: A comparison of algorithmic and UAV model results for two cases charging. The ﬁrst case implements aggressive charging, with battery power and SOC results shown in (a) and (b), respectively. The second case uses a linear charge strategy, with battery power shown in (c) and SOC shown in (d).

map of SFC versus Engine Power, derived based on engine speciﬁcation sheets. The

SFC versus RPM map, however, is explicitly provided by the manufacturer. Future research eﬀorts will work to push these two fuel consumption values closer together by considering the diﬀerences the two fuel maps and calculations performed by the

MPC algorithm.

55 The variety of tests cases studied within this chapter, from linear to aggressive charging, provide insights into the benefits of each charge strategy. Aggressive charging utilizes the most fuel, but provides a higher average state of charge, which may be beneficial in the face of an unknown load profile. The presence of load uncertainty is an essential step towards real-time implementation and is presented in the following chapter by building on the algorithm presented here. Further, this algorithm provides a basis for another UAV enhancement, renewable generation capabilities, also presented in the following chapter. Thus, this chapter presents the groundwork for additional research and provided key insights into fuel consumption trends, while validating the performance of the MPC formulation and algorithm.

56 CHAPTER 5

Additional Algorithmic Considerations - MPC for UAV Energy Optimization

Within this chapter, the MPC formulation for UAV energy optimization is further investigated with regards to the importance of proper horizon length selection. The algorithm is also further enhanced to consider uncertainty within the load profile. This formulation structure is intended to enhance the ability of the developed algorithm within real-time implementation. That is, the planning scope is no longer a key focal point as the work instead turns towards maximizing total flight time for a fixed quantity of fuel and fixed component sizes, e.g., engine and battery. Algorithmic enhancements to achieve this goal are discussed in depth and results are presented to demonstrate the ability of the algorithm to handle uncertainty within the load and generation. Generation uncertainty is induced by the presence of renewable generation, solar arrays, on-board the UAV. It should be noted that this chapter introduces initial results into all topics discussed and will be further enhanced within future work.

57 5.1 The Importance of Horizon Length within the MPC for UAV Energy Optimization

Before discussing the importance of horizon selection, it is beneﬁcial to review the objective function of the MPC algorithm, as it highly inﬂuences algorithmic decisions.

Within the MPC algorithm, decisions regarding charge behavior and engine power outputs are determined in a way that minimizes overall cost of the objective (4.2).

Since the objective is altered to ensure that charging occurs, i.e., the objective contains terms that negatively weight charge behavior leading to lowered overall cost, the algorithm will charge in a manner that balances the cost of fuel and charge behavior.

The addition of a term to penalize oscillations helps to avoid abrupt changes in engine power output. Without this term, the algorithm will choose to oscillate as it can immediately offset increased engine power output with a lowered value. Recall that in the case of gradual charging, decreases in parameters did not affect overall charge decisions. In that case, the c parameter was decreased from 10, 000 to less than 1 and no change to output occurred. Thus, though the relative weighting of each parameter can cause more or less aggressive charge behavior between various strategies, the horizon plays a critical role in determining performance for a specific strategy. That is, though weights can be changed to achieve more aggressive charging, when weights are kept the same and the horizon is instead varied, changes to engine output power and, thus, fuel consumption will occur.

To demonstrate the importance of horizon, all parameters are set to a weight comparable to a, the value attached to the squared term in the convex fuel map for engine power. a is approximately 0.19, c is set to 0.5, to discourage oscillatory engine behavior, and d is set to 0.15, very close to the value of a. Thus, the objective

58 becomes n X 2 min aP ek + bP ek + c|P ek − P ek−1| + dP bk. (5.1) P ek,P bk i=0 With all terms closely weighted, a saved data value of 200 is set for all cases presented here. This allows for an investigation of the importance of horizon selection alone, without compounding factors. Note, however, that with a higher weight on charging, results can be changed towards a more aggressive charging scheme. Thus, the results presented here represent the importance of horizon for one case of parameter values.

It is important to consider the way in which the MPC algorithm determines charge behavior. First, consider a horizon of 1, 000. In this case, the algorithm looks over

1, 000 load demand values and optimizes the value of the objective function, such that cost is lowered and the mission is feasible over this horizon. Thus, decisions are made with a consideration of only one 86th of the mission and the algorithm fails to appreciate the importance of charge behavior, as it is not considering possible high load demand in future periods. As these dash periods approach, the algorithm may not have enough time to charge to ensure there is adequate battery capacity to meet demand. In fact, in the case of a horizon of 1, 000, the algorithm fails to complete the entire mission, with infeasibilities presenting less than halfway into total mission duration. In the case of a known mission proﬁle, i.e., no uncertain load elements, the upper bound on feasibility for horizon length is determined solely by MATLAB’s capabilities, or the length of time at which MATLAB’s memory becomes insuﬃcient.

This level was found to be a horizon length of 70, 000. Though horizon lengths less than this level are feasible, time requirements are extensive, but this is not a limiting concern unless real-time implementation is considered. Note that a discussion of the

59 upper bound on horizon length when faced with load uncertainty is detailed and systematically determined in the next section.

Disregarding random load elements, the lower bound on feasibility for horizon length was determined to be 6, 625. Determination of this value was completed systematically. Prior tests indicated that a horizon of 7, 000 yielded feasible results under these parameter values. The halfway point between 1, 000, shown to be infeasible, and 7, 000, i.e., 4, 000, was tested. This point was also found to be infeasible, so the halfway point between 4, 000 and 7, 000, i.e., 5, 500, was tested. This method was continued until the final value of 6, 625 was found. This method of testing allows for a systematic determination of a minimal value for feasible horizon length in an effective and efficient manner, without extensive testing of all values from 1, 000 to 7, 000. Note that under parameter values that more highly incentivize charging, smaller horizon values will be feasible. After determining the minimal horizon, the effect of horizon was demonstrated via an optimization of the total mission at horizon lengths of 6, 625, 8, 000, 10, 000, and 15, 000. These results are shown in Fig. 5.1, below, in terms of energy source output power.

The results of these tests immediately show that the relative weighting of the parameters under this test case encourages a strategy of charging only when necessary to meet future demand. Further, the weighting of charge behavior causes an increase to objective function values in the case of discharging, which results in engine output running at the cruise demand of 1.8kW h. Comparing the case of a 6, 625 horizon (a) and the 8, 000 horizon (b), it is apparent that increasing the horizon size allows the algorithm to begin charging as soon as future load demands become available to the algorithm, resulting in earlier charge behavior. The minimal horizon, 6, 625 allows

60 Energy Source Output and Load Profile Energy Source Output and Load Profile 6 6 Engine Power Engine Power Load Profile Load Profile 5 5 Battery Power Battery Power

4 4

3 3

2 2 Power (kW) Power (kW)

1 1

0 0

-1 -1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 Time (sec) ×104 Time (sec) ×104 (a) (b)

Energy Source Output and Load Profile Energy Source Output and Load Profile 6 6 Engine Power Engine Power Load Profile Load Profile 5 5 Battery Power Battery Power

4 4

3 3

2 2 Power (kW) Power (kW)

1 1

0 0

-1 -1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 Time (sec) ×104 Time (sec) ×104 (c) (d)

Figure 5.1: A comparison of energy source output in the face of varied horizon sizes: (a) 6, 625, (b) 10, 000, (c) 15, 000, and (d) 20, 000. With each increase in horizon length, the initial start of charge behavior before the second dash period occurs earlier.

for the battery to be charged such that dash demand is met, but requires higher output engine power over a shorter amount of time. As the horizon increases, the engine can operate at a lower rate for a longer time period to ensure there is suﬃcient battery capacity available. Speciﬁcally, the initial time of charge behavior before the second dash period occurs at about 25, 000 seconds for the shortest horizon and then begins earlier, at 22, 000s, 17, 600s, and 12, 600s, with each horizon length increase.

61 These strategy changes also impact the main result of interest, fuel consumption. The initial case results in fuel consumption of 31, 502ccm3, while the ﬁnal case of a 20, 000 horizon provides the least fuel consumption at 30, 643ccm3, as the engine can operate at a lower output power.

Horizon length plays a critical role in the performance of the MPC algorithm.

When parameter values are determined such that the general strategy is achieved, e.g., aggressive charging, horizon length changes will eﬀect second by second decisions within the algorithm. A longer horizon will seek to balance the high cost of engine power output and negative cost of charging by stretching charge behavior out over a longer time period. A shorter horizon, however, will seek to charge quicker, as costs must be balanced over a shorter time period. This impacts fuel consumption, as higher output engine power leading to more rapid charging is often necessitated for short horizons due to short notice of dash periods. On the other hand, a lower output engine power over a longer period of time can occur when dash periods are known well ahead of time, leading to lowered fuel consumption. Thus, horizon length should be carefully considered during algorithmic testing, as desired results could be achieved at varied fuel consumption levels based on how far into the future the algorithm is permitted to look. The importance of horizon becomes much more apparent in the case of random load demands, presented in the next section.

5.2 MPC in the Presence of Load Uncertainty

During a real UAV flight, it may not be possible to provide the algorithm with an exact load profile. In all of the previous cases tested within this work, it has been assumed that the MPC algorithm has access to the load profile that will actually

62 occur. To reach the goal of real-time implementation, the performance of the algorithm in the presence of load uncertainty is of interest. For example, if the algorithm assumes it has tens of thousands of seconds to charge the battery, but an unexpected dash period occurs, the UAV may fail to meet its objective. Providing the MPC algorithm with uncertain load elements yields initial insights into the performance of the developed method within a real-time scenario.

To test the algorithm, random load profiles were created by moving dash periods and adjusting the duration of dash periods. Fig. 5.2 shows each of these load profiles and the sample mission profile, for comparison. Specifically, one load profile moves each dash period forward by 600 seconds and the other moves each dash period backwards by 600 seconds. Physically, this corresponds to a window of 10 minutes for each dash period to occur. The third possible profile rearranges the dash periods, such that the longest one occurs last instead of second and the shortest occurs second, instead of first. Though these profiles do not significantly differ from the sample mission, they provide a basis for work into random loads. A difference of even 10 minutes of charging can still impact feasibility. In the physical mission sense, these possible load profiles represent an uncertainty of 10 minutes regarding when dashes will occur and uncertainty about the order in which dash periods occur. Cases of load uncertainty considering more drastic changes to load, i.e, very long dashes or frequent dash periods, result in highly undesirable output power behavior and are unlikely to occur in real flight scenarios. Test results are provided for cases of drastic profiles are provided at the end of this section to illustrate algorithmic performance.

Algorithmically, MPC occurs in the same manner. A selection of load for each horizon must be made, however, to implement load uncertainty. A random integer

63 Possible Load Profile #1 Possible Load Profile #2 6 6 Sample Mission Sample Mission 5.5 Random Load Profile 5.5 Random Load Profile

5 5

4.5 4.5

4 4

3.5 3.5 Power (kWh) Power (kWh) 3 3

2.5 2.5

2 2

1.5 1.5 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 Time (sec) ×104 Time (sec) ×104 (a) (b)

Possible Load Profile #3 6 Sample Profile 5.5 Random Load Profile

4.5

3.5 Power (kWh) 3

2.5

1.5 0 1 2 3 4 5 6 7 8 9 Time (sec) ×104 (c)

Figure 5.2: Three possible load proﬁles that can be chosen by the algorithm. The proﬁle in (a) moves dash periods backwards by 600 seconds, (b) moves dash periods forwards by 600 periods, and (c) changes the length of dash periods present in the sample mission.

variable r is chosen in each iteration of the algorithm. If r is equal to 1, 2, or 3, the load implemented within the MPC solver is one of the new load proﬁles. If not, the sample mission proﬁle is used to create the solution. Within implementation, r can take on values from 1 to 4. A full version of the algorithm implemented for uncertain loads is provided below. Note that while the loads in this section are referred to as

64 random, there are only four speciﬁc load patterns that can occur. In the future, the algorithm will be further enhanced to replicate a wider variety of random load proﬁles that the mission may experience and, thus, provide more robust results.

Algorithm 3 MPC - Optimal Energy Use with Random Loads Initialize Length of Horizon h and Saved Data m, and Operational Bounds Current Time Step Initialized i = 1 while i ≤ Total Mission Length − m do Initialize SOC for Time Under Study, = i + h Determine Random Load if Random variabler = 1, 2, 3 then Load for Time Under Study 6= Sample Load else if Load for Time Under Study = Sample Load then end if Call CVX to solve MPC Formulation over Horizon h if Solution Feasible then Store P e, P b, F, E, from i to i + m else if Infeasible Solution then BREAK. Choose New h, m end if i ← i + m end while Post-processing of data to generate control inputs for UAV model

When determining horizon length and the size of saved solutions in the context of random elements with the load, it is beneﬁcial to consider the physical meaning of these variables. Saving too many solutions may lead to undesirable real ﬂight response, i.e., a lot of trust in the provided load data and dynamics of the model.

That is, with increasing numbers of saved solutions, the algorithm is increasing its trust that the values of load used during the solution will actually occur. Within real- time ﬂight, however, this may lead to undesirable response. Additionally, if the MPC algorithm saves too many solutions, its estimation of SOC or remaining fuel may be

65 incorrect. By the time these values are updated to reflect real system states within a real-time implementation, the difference may affect feasibility for the remainder of the mission. Saving too few solutions increases computation time, but can allow the algorithm to more appropriately address differences between expected values (SOC,

Load, etc.) and real-system values within real time implementation. However, when optimizing over a large horizon, charging behaviors can be stretched out over a longer time, easily leading to infeasibilties in the event of an unexpected dash demand.

With regards to computation times, it is important to note that saving 200 solutions requires 432 iterations, regardless of horizon size. Total computation length will be equal to the number of iterations times the solution speed per iteration for a given horizon. Table 5.1, below, provides experimentally obtained data for average solution time for a variety of horizon lengths. This is important to consider when determining horizon length, especially in the case of real-time implementation. For example, if a horizon of 60, 000 is used and 200 solutions are saved, total solution time becomes 432 iterations ×76.3 seconds, or 549 minutes of total solution time. In the planning scope this value is not of critical importance, but when trying to obtain real-time communication between the algorithm and UAV/UAV model this will likely become too diﬃcult to implement and a horizon with a quicker solution speed will become more desirable.

For the purposes of this initial determination of the algorithm’s ability to handle random load elements, saved solutions were set at 1, 200, indicating that the algorithm can trust about 20 minutes of load data and estimated SOC values before requiring updates from the UAV. Note that these tests do not occur within a real- time environment. The value of 800 saved solutions was determined by setting a

66 Table 5.1: Solution Speeds per Iteration for Various Horizon Lengths

Horizon Length Time Per Iteration (seconds) 60,000 76.3 50,000 68.8 40,000 27.3 20,000 22.3 15,000 15.8 10,000 11.6 5,000 2.8 2,000 3.73

horizon value and starting at 200 saved solutions. This was increased by 200 six

times until a feasible saved solution length was obtained. Future work can further

improve determination of this variable using feedback from the UAV model. Note

that the d parameter was increased from 0.15 to 0.17 to express the need for slightly more aggressive charging in the face of unknown demand. c remains at 0.05. The parameters indicate a mostly linear charging strategy, which may cause infeasiblities in the face of an unexpected dash period. These infeasibilities may not be present in the case of aggressive charging, which retains a higher average SOC. Addressing an increased need for aggressive charging and/or increased available battery capacity can also be addressed via tuning of parameters. In these test cases, however, the battery is slightly oversized at 4.41kW h to account for the worst case scenario, in which loads are selected such that a dash period is extended to a length that makes a 2.41kW h battery, used in the previous chapter, infeasible.

67 To determine the minimum and maximum horizon lengths for feasible solutions, the procedure detailed in section 5.1, above, was used again. In the case of non- random loads, the maximum horizon length is determined purely based on MATLAB’s storage limits. On the other hand, maximum horizon length in the face of uncertainty is limited by the algorithm’s ability to meet load demand that may unexpectedly arise.

For the minimum feasible horizon, tests using the previously described method yielded a value of 5, 125. Recall that the minimum feasible horizon without load uncertainty was found to be 6, 625. Though there is uncertainty in load, the minimum feasible horizon decreases in this case due to a higher d parameter, which facilitates charging, and a larger battery. The same process was repeated to determine the maximum feasible horizon, which was found to be 31, 125. Results are presented below for a minimum feasible horizon and maximum feasible horizon. Each case used the same parameter values and saved solution length of 1, 200.

In the results present in Fig. 5.3, the load profile shown is created based on the possible profile chosen during each optimization iteration. Since 1, 200 solutions are saved, the first 1, 200 values of the load in each iteration are stored as the load that actually occurs during this mission. In a real-time implementation, the UAV may not be able to accurately determine the next 20 minutes of load demand and this amount of saved data will thus need to be adjusted. As the results show, available SOC reaches the lower bound in the case of a minimum horizon and sees minimal changes to output engine power over the entirety of the mission, due to increased battery capacity. However, charging does still occur, as shown in Fig. 5.3(b). In the presence of maximum horizon, SOC does not reach the lower bound and an additional dash period in the start of the mission, shown in (c), is easily handled. It is important to

68 Energy Source Output and Load Profile Battery SOC for Duration of Trip 6 0.8 Engine Power Load Profile 5 Battery Power 0.7

4 0.6

3 0.5

2 SOC (%) Power (kW) 0.4 1

0.3 0

-1 0.2 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 Time (sec) ×104 Time (sec) ×104 (a) (b)

Energy Source Output and Load Profile Battery SOC for Duration of Trip 6 0.8 Engine Power Load Profile 5 Battery Power 0.7

4 0.6

3 0.5

2 SOC (%) Power (kW) 0.4 1

0.3 0

-1 0.2 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 Time (sec) ×104 Time (sec) ×104 (c) (d)

Figure 5.3: Performance of the MPC algorithm in the face of uncertain load proﬁles. Energy source output and SOC in the case of a minimum feasible horizon are shown in (a) and (b), respectively. In the case of a maximum feasible horizon, energy source output in shown in (c) and SOC is shown in (d).

note that it is possible for the algorithm to choose a more intensive proﬁle than what is represented in Fig. 5.3. One of these is provided in Fig. 5.4, where an unexpected dash period occurs after the third dash. In this case, the algorithm increases engine power output to meet future demand. Thus, even in the presence of a more demanding load proﬁle, the MPC algorithm is able to handle load requirements.

69 Energy Source Output and Load Profile 6 Engine Power Load Profile 5 Battery Power

2 Power (kW)

-1 0 1 2 3 4 5 6 7 8 9 Time (sec) ×104

Figure 5.4: MPC performance in the face of a more intensive load proﬁle. An unexpected, quick dash period occurs after the third dash period, but the MPC algorithm was able to increase engine power output to meet this unexpected demand.

To demonstrate the eﬀect of drastic load proﬁle uncertainty on algorithmic per-

formance, consider the four possible load proﬁles in Fig. 5.5. Dash periods can be

greatly extended or occur very frequently based on the random selection that occurs

within the algorithm. When testing MPC performance under such load proﬁles, re-

sults show that the UAV is able to complete the mission. As Fig. 5.6 shows, however,

rapid changes to battery output occur in the presence of such intense load proﬁle

possibilities. Within this test case, it was found that the highest feasible horizon

was 2, 000, as the algorithm required shorter horizons to accurately account for the possibility of a drastically extended dash period. Though the MPC algorithm can handle such cases of drastic load uncertainty, they were not a focus of study in this thesis, as they are unlikely to occur in real ﬂight scenarios.

70 Figure 5.5: Four possible load proﬁles (Power (kW) v. Time (sec)) tested to determine algorithmic performance in the face of drastic load changes. Note that dash periods can be greatly extended or occur very frequently, depending on random selection.

5.3 Integrating Renewable Generation

Solar powered flight is not a new concept [34] and with the rise in the use of UAVs, which are incredibly light weight and have much lower power requirements than larger aircraft, solar flight has become more popular. In the absence of purely solar flight

[35], in which solar power drives UAV propellers, solar panels are increasingly being

71 Figure 5.6: MPC output for the case of drastic load uncertainty. The UAV completes its mission, but presents undesirable trends in battery and engine power usage as it accounts for possible load demand.

placed on UAV wings to supplement their energy sources [36]. Recent developments in solar cell technology have enabled engineers to create ﬂexible, high power density solar panels that can form to curves on wing surfaces [37].

This section serves as the groundwork for future research focused on integrating solar panel capabilities within the MPC UAV energy optimization algorithm. This baseline research utilizes technical speciﬁcations from a solar panel that has been used by the Naval Research Laboratory to conduct research into UAV solar panel performance [38]. The double junction Gallium Arsenide PV product produced by

Alta Devices is a ﬂexible solar cell sheet that can be applied to UAV wings to provide/supplement overall UAV energy production [1]. Speciﬁcations of interest to the

MPC algorithm are provided in Table 5.2. Further electrical characteristics of these

72 cells will be considered when the UAV Simulink Model is extended to include solar

panels.

Table 5.2: Double Junction Gallium Arsenide PV Product Characteristics [1]

Electrical Characteristic Speciﬁcation at 25◦C Eﬃciency 29% Power per Cell 0.28W Power Density 290W/m2

Throughout this section, it will be assumed that these solar cells are compatible

with the altitude of ﬂight for the UAV under study. The formulation used for MPC

implementation remains the same, excluding the load balance constraint, where solar

generation will manifest. Thus, the new constraint is

P ek + P bk = Loadk − REk, (5.2)

where REk is represented by a random variable. It is ﬁrst assumed that the UAV under study is large enough to accommodate 0.5m2 of solar cells, thus allowing it to

generate 145W of power. The random variable REk is uniformly distributed from 0

to 145 within these initial tests. In the future, a more realistic distribution can be

created to account for solar variations experienced during ﬂight. For example, there

should be no solar output during overnight hours. Distribution of solar radiation at

high altitudes should be modeled as accurately as possible within the Simulink model

to obtain realistic results for real-time ﬂight. However, in the MPC study conducted

here, the main focus is to determine the eﬃcacy of the algorithm in the face of

73 variable generation and the uniform distribution gives insight into performance. It is

of note that the maximum total solar panel output of 145W is one 32nd of total load

demand during a cruise period. This output is unlikely to make a large impact on

total performance, but may help to smooth operation of the engine and/or battery

and provide additional battery charge.

The capability of solar generation within the MPC was ﬁrst tested using a horizon

of 10, 000, saving 1, 200 solutions. Parameter values were set at a level comparable

to the cost of engine power within the objective function, i.e., c and d are equal to

2 a in aP Ek − bP Ek + c. As the results show, equal weighting of all parameters leads to severe oscillations in battery power output, shown in Fig. 5.7(a), with input solar generation being immediately used and providing no additional battery charge of note, as shown in (b). Though there are oscillations, the battery absorbs changes to overall load induced by the renewable generation in (5.2). Thus, as the solar generation enters the UAV, the battery absorbs this extra power to allow steady engine power output. However, since the maximum solar output is so small compared to total load demand, no signiﬁcant increase to battery SOC occurs. For example, even in a case

of maximum solar output, total load demand still remains at 1.655, which is close

to the optimal engine power output value. The battery, therefore, does not charge a

signiﬁcant amount during this period. For any other solar output, the value of total

load demand is still above optimal engine output, causing the battery to discharge

to meet demand. Overall, no signiﬁcant increase in available battery capacity occurs

due to the baseline operating conditions of the UAV system. Overall, the engine can

operate at its optimal point, but must still increase output to charge the battery,

resulting in no net fuel consumption gain. Note that solar generation is not explicitly

74 plotted in the results to maintain clarity of the ﬁgures, but its value can be observed via the oscillations of battery power.

Energy Source Output and Load Profile Battery SOC for Duration of Trip 6 0.8 Engine Power Load Profile 5 Battery Power 0.7

4 0.6

3 0.5

2 SOC (%) Power (kW) 0.4 1

0.3 0

-1 0.2 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 Time (sec) ×104 Time (sec) ×104 (a) (b)

Figure 5.7: Initial tests of solar generation lead to oscillations in battery and engine power (a) and no noteworthy additional charging to the battery (b).

To guide the system towards utilizing solar generation to provide additional charge to the battery, without utilizing more input engine power, the solar array was increased in size to contribute more power to the overall system in a manner that facilitates charge and helps the engine perform at its optimal point. To accomplish this, maximum solar output was set to 500W , represented by a uniform random variable between 0 and 500. Within this test, c and d were both set equal to zero to fully characterize the eﬀect of larger solar cell capacity on-board the UAV. That is, charging was not incentivized. The results of this test case, shown in Fig. 5.8 show that, though there are oscillations in the battery power output, the desired goal of driving charge behavior without increasing engine output is obtained. It is of note

75 that within these results, the engine operates at its optimal point for most of the mission, with the battery oscillating and providing additional support to meet changes to total load demand. Thus, the oscillations in battery power occur due to the lowering of total load demand due to solar generation. For example, if the solar array outputs a maximum value of 500W , total load demand during a cruise period becomes

1.3kW , which is lower than optimal engine power output. Therefore, the battery will slightly charge to absorb the excess engine power being produced, eliminating a need for decreased engine power output. The renewable generation is used to facilitate charging, as shown in Fig. 5.8(b).

Energy Source Output and Load Profile Battery SOC for Duration of Trip 6 0.8 Engine Power Load Profile 5 Battery Power 0.7

4 0.6

3 0.5

2 SOC (%) Power (kW) 0.4 1

0.3 0

-1 0.2 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 Time (sec) ×104 Time (sec) ×104 (a) (b)

Figure 5.8: Tests of 500W capable solar cells produces oscillations in battery power (a), but provides ancillary charge to the battery, increasing SOC (b) without increasing engine power.

The engine does ramp up slightly to provide additional battery capacity directly before a dash period, but it does not reach the high output levels seen in the test cases shown in Chapter 4, where no renewable generation is used and engine output goes

76 above 2kW h to pre-charge the battery for a dash. During dash periods, engine power remains at or below the 1.8kW h level, which is much lower than outputs during cases without solar generation, where output can go above 2kW h. Finally, solar generation results in signiﬁcantly less fuel consumption, as only 24, 857ccm3 of fuel is used. The only fuel consumption level lower than this result is in the case of no charging and discharging during cruise periods, with the use of a heavy 13.5kW h battery. This case, however, utilizes the smallest battery tested, 2.41536kW h.

These results indicate that solar generation can be an invaluable resource for the

UAV, as it allows for charging to occur without increasing engine power and fuel consumption. However, the tests conducted here indicate that there is a minimum size at which solar generation becomes a valuable addition. A solar array sized at

145W does not provide enough input power to the UAV and results in only oscillations to the battery power, without increasing charge. Additional solar generation testing will involve determining optimal solar array sizes while considering weight additions.

Eﬀorts to reduce battery power oscillation will also be addressed.

5.4 Conclusion

Algorithmic enhancements can be added to the base MPC algorithm of Chapter

4 to replicate various additions to the UAV under study. At the core of all MPC implementations, however, is the length of horizon. As shown within this chapter, maximum horizon length is determined solely by MATLAB’s memory limits in the face of purely deterministic load proﬁles. When stochasticity is introduced, however, maximum horizon length is determined by the ability of the algorithm to meet potential unexpected increases to demand. Horizon length plays a critical role in the

77 performance of the MPC algorithm. Longer horizons seek to balance objective costs by stretching charge behavior out over a longer time period, but can lead to infeasibilities in uncertain load situations. Shorter horizons charge more rapidly, as costs must be balanced over a shorter time period, but often lead to increased fuel consumption.

Random load elements more faithfully replicate the operating conditions experienced by the MPC algorithm within real-world ﬂight scenarios. Initial tests into random loads within this chapter consisted of only four possible loads, but indicate that the algorithm can adapt to uncertain elements and still complete the mission.

Results show that a few unnecessary charge events occur due to a prediction of demand that does not manifest; however, overall performance allows for the UAV to complete its mission. Future enhancements will create a wider variety of potential loads and seek to minimize unnecessary charge behaviors, even in the presence of uncertainty. Parameter determination for random loads will also be investigated, with an eye towards intuitive determination of parameter values by the algorithm itself, without user input. That is, desired charge strategies can be used to adaptively determine parameter values without excessive tuning.

Finally, the MPC algorithm was shown to function well in the face of uncertain generation induced by solar arrays. Solar arrays are commonly paired with UAVs in practice and results show that they can lead to improved fuel consumption, as they provide charge without increasing output engine power. However, there is a minimum size of solar array that can provide supplemental power at a level that results in charge behavior. Future work will focus on determining this minimum solar array size and integrate more realistic distributions of solar generation.

78 Overall, initial explorations into enhanced considerations for the MPC algorithm indicate that it is able to handle base cases for both uncertain load elements and renewable generation. These additions allow for a more complex UAV representation and give insights into both real-time performance, in the case of load uncertainty, and the beneﬁts of renewable generation on-board a UAV.

79 CHAPTER 6

Conclusions and Future Work

Throughout the past several decades environmental concern has risen across various industries. The transportation sector, in particular, has been consistently making progress towards the goal of reducing fuel consumption, making transport a more environmentally friendly aspect of everyday life. Automobiles, trains, ships, and airplanes have begun the transition towards hybrid and all-electric models. Though some aviation industry leaders have made strides towards more electric aircraft, in which electric energy is used for all non-propulsive systems, more research is needed to facilitate the transition to all electric aircraft. Hybrid Unmanned Aerial Vehicles provide an excellent prototype for investigations into the feasibility of this transition.

UAVs are commonly used for homeland defense, hobby ﬂight, and product delivery.

Due to their much smaller size compared to a traditional commercial airliner and purely DC power system, UAVs provide a desirable platform for research into all electric aircraft. Both of these qualities lend themselves to familiar power system optimization techniques. Optimizing energy management schemes for multiple energy sources is an essential step towards further improvements in the ﬁeld of aircraft electriﬁcation. Focusing on smaller-scale UAVs gives invaluable insight, as techniques can be derived and scaled for larger models.

80 The work presented here seeks to improve upon past works related to energy optimization for hybrid UAVs. Past works have used either (i) very complex formulations considering the non-linearities present in real-world flight situations, sacrificing computational efficiency and potential for real-time implementation, or (ii) formulations considering only power system critical operational constraints, e.g., power balance and output power bounds, sacrificing a consideration of realistic operational conditions. This work, however, addresses these pitfalls using a formulation considering power system critical components, paired with operational testing in the form of a realistic UAV model. The connection between the algorithmic output and the UAV model allows for algorithmic improvement based on real system needs. An open-loop configuration can be used if users want to interpret the results of both components and provide improvements. Closed-loop implementation mimics real-time implementation, wherein real system outputs are fed directly to the algorithm, driving algorithmic variables closer to their true states.

Optimization formulations based on both dynamic programming and model predictive control can be applied to the problem of UAV energy optimization. Within this work, both formulations and their corresponding algorithms were shown to provide desirable results in the face of a deterministic load profile. In the case of a dynamic programming based formulation, fixed C-Rate battery specifications were implemented to capitalize on inherent system characteristics that influence the lowest cost, i.e., lowest fuel consumption, path the UAV can take with regards to energy optimization. This strategy, however, fails to capture possible system decisions and limits the scope of the algorithm. This downfall was addressed through the creation of an MPC formulation, along with an additional consideration of fuel consumption.

81 The MPC algorithm developed was capable of implementing various charge strategies, including cases of no charging, aggressive charging, gradual charging, and linear charging. Overall, the MPC formulation and its corresponding algorithm allow for a deeper look into UAV energy optimization. When implementing algorithmic decisions within a UAV model, both methods provided satisfactory results. Differences in time-steps between the algorithm and model cause differences in observed battery power. Further, due to the fuel maps used, fuel consumption differs between the algorithm and model.

Algorithmic enhancements were considered and provide baseline results regarding

MPC performance in the face of random loads and solar generation. Regardless of the UAV under study, horizon length is a key aspect of the MPC algorithm, as solutions depend on the length of future data available to the algorithm. Though longer horizons can help limit rapid charging, which leads to increased fuel consumption, it also increases computation times and can lead to infeasiblities when presented with load proﬁles that are uncertain. A systematic method was presented for determining minimal and/or maximal horizon lengths based on the presence of random loads, or lack thereof. Finally, the MPC formulation was proven to be capable of integrating renewable generation considerations and initial insights into the beneﬁt of these capabilities were discussed. Initial results show that solar arrays must be optimally sized, such that they provide enough additional generation that the battery can be charged without requiring additional engine power.

Future work will seek to improve upon the results presented here in a variety of ways. First, fuel consumption diﬀerences between the UAV model and MPC algorithm will be further investigated and improved. The MPC algorithm could be

82 altered to include fuel consumption values based on RPMs, as this fuel map more closely resembles the physical operation of the UAV’s engine. Improvements to the algorithm will also attempt to develop an intuitive means of determining parameter values, without requiring tuning. For example, the algorithm could be developed in a way that determines parameter values given a predetermined horizon length and description of desired charge strategy. More advanced techniques, such as machine learning or artiﬁcial intelligence, can be applied to make horizon size and parameter values adapt based on the variance of the uncertainty present within the system.

That is, based on the characteristics of the uncertainty, the algorithm can intuitively select optimal parameter sizes needed to obtain desirable performance. Horizon and saved solution lengths will be further investigated, especially in the case of real-time implementation. Certain horizon lengths may be too time expensive to implement within the UAV model. For example, the formulation may take too long to solve for long horizons, leaving the model without control inputs for a period of time.

Thus, the algorithm and model must work together such that the algorithm provides timely control inputs and the model does not implement controls too quickly for the algorithm. This will require extensive work to ensure desirable performance.

When considering random loads, the possible profiles created here represent sim- plistic possibilities. Future work will create more extensive random profiles and further push the algorithm towards robust solutions. Finally, solar generation capabilities will be improved by the integration of realistic solar radiation distributions. A more in-depth analysis of the minimal size of solar generation that can be beneficial to overall UAV performance will also be conducted.

83 The work presented here demonstrated the beneﬁts of utilizing a general optimization formulation in conjunction with a realistic UAV model. Computational speeds remain fast as complex dynamics are considered within a Simulink model. MPC was shown to be a valuable tool for considering the problem of UAV energy optimization and future work will further improve the presented algorithm so that more complex problems, e.g., load uncertainty, can be considered. The ultimate goal of real-time implementation will enable the algorithm to function in both the planning and operational scope, thus closing the loop between the UAV model and MPC algorithm.

84 BIBLIOGRAPHY

[1] Gallium Arsenide Solar Product, Alta Devices, 12 2017.

[2] R. Jones, “The more electric aircraft: the past and the future?” IET, 1999.

[3] Boeing. 787 electrical system. [Online]. Available: http://787updates. newairplane.com/787-Electrical-Systems/787-Electrical-System#

[4] MarketsandMarkets, “Unmanned aerial vehicle (uav) market, by application, class (mini, micro, nano, tactical, male, hale, ucav), subsystem (gcs, data link, software), energy source, material type, payload and region - global forecast to 2022,” MarketsandMarkets, Tech. Rep. AS 2802, 2016.

[5] H. Zhang, C. Saudemont, B. Robyns, and M. Petit, “Comparison of technical features between a more electric aircraft and a hybrid electric vehicle,” in Vehicle Power and Propulsion Conference, 2008. VPPC’08. IEEE. IEEE, 2008, pp. 1–6.

[6] J. Langelaan, “Long distance/duration trajectory optimization for small uavs,” in AIAA Guidance, Navigation and Control Conference and Exhibit, 2007, p. 6737.

[7] S. Hosseini, R. Dai, and M. Mesbahi, “Optimal path planning and power alloca- tion for a long endurance solar-powered uav,” in American Control Conference (ACC), 2013. IEEE, 2013, pp. 2588–2593.

[8] A. Aibinu, H. B. Salau, C. Akachukwu, and M. Nwohu, “Polygamy based genetic algorithm for unmanned aerial vehicle (uav) power optimization: A proposal,” in Electronics, Computer and Computation (ICECCO), 2014 11th International Conference on. IEEE, 2014, pp. 1–5.

[9] G. Steinmauer and L. Del Re, “Optimal control of dual power sources,” in Con- trol Applications, 2001.(CCA’01). Proceedings of the 2001 IEEE International Conference on. IEEE, 2001, pp. 422–427.

[10] T. N¨uesch, P. Elbert, M. Flankl, C. Onder, and L. Guzzella, “Convex optimization for the energy management of hybrid electric vehicles considering engine start and gearshift costs,” Energies, vol. 7, no. 2, pp. 834–856, 2014.

85 [11] X. Shen, J. Zhang, and T. Shen, “Real-time scenario-based stochastic optimal energy management strategy for hevs,” in Control Conference (ECC), 2016 Eu- ropean. IEEE, 2016, pp. 631–636.

[12] S. D. A. Fletcher, P. Norman, S. Galloway, P. Rakhra, G. Burt, and V. Lowe, “Modeling and simulation enabled uav electrical power system design,” SAE In- ternational Journal of Aerospace, vol. 4, no. 2011-01-2645, pp. 1074–1083, 2011.

[13] A. Collin, J. Acosta, B. Hayes, and S. Djokic, “Component-based aggregate load models for combined power ﬂow and harmonic analysis,” 2010.

[14] D. Meyer, R. Alexander, and J. Wang, “A simple method for energy optimization to enhance durability of hybrid uav power systems,” in 2017 North American Power Symposium (NAPS), Sept 2017, pp. 1–6.

[15] R. Bellman, Dynamic programming. Courier Corporation, 2013.

[16] W. B. Powell, Approximate Dynamic Programming: Solving the curses of dimensionality. John Wiley & Sons, 2007, vol. 703.

[17] A. Heydari, “Optimal switching of dc-dc power converters using approximate dynamic programming,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 3, pp. 586–596, March 2018.

[18] C. Santiago, J. C. Nascimento, and J. S. Marques, “Fast and accurate segmen- tation of the lv in mr volumes using a deformable model with dynamic programming,” in 2017 IEEE International Conference on Image Processing (ICIP), Sept 2017, pp. 1747–1751.

[19] S. Rohkmper, M. Hellwig, and W. Ritschel, “Energy optimization for electric vehicles using dynamic programming,” in 2017 International Conference on Re- search and Education in Mechatronics (REM), Sept 2017, pp. 1–5.

[20] L. Tang, G. Rizzoni, and M. Lukas, “Comparison of dynamic programming-based energy management strategies including battery life optimization,” in 2016 In- ternational Conference on Electrical Systems for Aircraft, Railway, Ship Propul- sion and Road Vehicles International Transportation Electriﬁcation Conference (ESARS-ITEC), Nov 2016, pp. 1–6.

[21] B. Jeddi, Y. Mishra, and G. Ledwich, “Dynamic programming based home energy management unit incorporating pvs and batteries,” in 2017 IEEE Power Energy Society General Meeting, July 2017, pp. 1–5.

[22] L. N. An and T. Quoc-Tuan, “Optimal energy management for grid connected microgrid by using dynamic programming method,” in 2015 IEEE Power Energy Society General Meeting, July 2015, pp. 1–5.

86 [23] O. Tremblay, L.-A. Dessaint, and A.-I. Dekkiche, “A generic battery model for the dynamic simulation of hybrid electric vehicles,” in Vehicle Power and Propul- sion Conference, 2007. VPPC 2007. IEEE. IEEE, 2007, pp. 284–289.

[24] S. N. Motapon, L.-A. Dessaint, and K. Al-Haddad, “A comparative study of energy management schemes for a fuel-cell hybrid emergency power system of more-electric aircraft,” IEEE transactions on industrial electronics, vol. 61, no. 3, pp. 1320–1334, 2014.

[25] K. Zhou, J. C. Doyle, K. Glover et al., Robust and optimal control. Prentice hall New Jersey, 1996, vol. 40.

[26] L. Wang, Model predictive control system design and implementation using MATLAB R . Springer Science & Business Media, 2009.

[27] M. Diehl, “Optimization algorithms for model predictive control,” Encyclopedia of Systems and Control, pp. 989–997, 2015.

[28] M. West, C. Bingham, and N. Schoﬁeld, “Predictive control for energy management in all/more electric vehicles with multiple energy storage units,” in Electric Machines and Drives Conference, 2003. IEMDC’03. IEEE International, vol. 1. IEEE, 2003, pp. 222–228.

[29] H. Borhan, A. Vahidi, A. M. Phillips, M. L. Kuang, I. V. Kolmanovsky, and S. Di Cairano, “Mpc-based energy management of a power-split hybrid electric vehicle,” IEEE Transactions on Control Systems Technology, vol. 20, no. 3, pp. 593–603, 2012.

[30] M. Grant and S. Boyd, “CVX: Matlab software for disciplined convex programming, version 2.1,” http://cvxr.com/cvx, Mar. 2014.

[31] ——, “Graph implementations for nonsmooth convex programs,” in Recent Ad- vances in Learning and Control, ser. Lecture Notes in Control and Information Sciences, V. Blondel, S. Boyd, and H. Kimura, Eds. Springer-Verlag Limited, 2008, pp. 95–110, http://stanford.edu/∼boyd/graph dcp.html.

[32] Y. Wang and S. Boyd, “Fast model predictive control using online optimization,” IEEE Transactions on control systems technology, vol. 18, no. 2, pp. 267–278, 2010.

[33] R. Singhal and P. Sujit, “3d trajectory tracking for a quadcopter using mpc on a 3d terrain,” in Unmanned Aircraft Systems (ICUAS), 2015 International Conference on. IEEE, 2015, pp. 1385–1390.

[34] R. Boucher, “History of solar ﬂight,” in 20th Joint Propulsion Conference, 1984, p. 1429.

87 [35] A. T. Klesh and P. T. Kabamba, “Solar-powered aircraft: Energy-optimal path planning and perpetual endurance,” Journal of guidance, control, and dynamics, vol. 32, no. 4, pp. 1320–1329, 2009.

[36] G. Kumar, S. Sepat, and S. Bansal, “Review paper of solar-powered uav,” Inter- national Journal of Scientiﬁc & Engineering Research, vol. 6, no. 2, pp. 41–44, 2015.

[37] C. K. Chin, “Extending the endurance, missions and capabilities of most uavs using advanced ﬂexible/ridged solar cells and new high power density batteries technology,” NAVAL POSTGRADUATE SCHOOL MONTEREY CA, Tech. Rep., 2011.

[38] A. Devices, “Alta devices excels in naval research lab solar uav shootout,” https://www.altadevices.com/ alta-devices-excels-naval-research-lab-solar-uav-shootout/, accessed: 2018- 02-28.