<<

IEEE TRANSACTIONS ON VEHICULAR , VOL. 58, NO. 9, NOVEMBER 2009 4741 Intelligent Control Based on Learning of Optimal Control Parameters and Prediction of Type and Traffic Congestion Jungme Park, Zhihang Chen, Leonidas Kiliaris, Ming L. Kuang, M. Abul Masrur, Senior Member, IEEE, Anthony M. Phillips, and Yi Lu Murphey, Senior Member, IEEE

Abstract—Previous research has shown that current driving efficiently control the flow through the vehicle system. conditions and driving style have a strong influence over a vehicle’s Our research focuses on the latter. Vehicle power management fuel consumption and emissions. This presents a methodol- has been an active research area in the past decade and has ogy for inferring road type and traffic congestion (RT&TC) levels from available onboard vehicle data and then using this informa- intensified recently by the emergence of hybrid electric vehicle tion for improved vehicle power management. A machine-learning . Most of the previous approaches were developed algorithm has been developed to learn the critical knowledge about based on mathematical models or knowledge derived from fuel efficiency on 11 facility-specific drive cycles representing dif- static vehicle operation data. The application of optimal control ferent road types and traffic congestion levels, as well as a neural theory to power distribution and management has been the learning algorithm for the training of a neural network to predict the RT&TC level. An online University of Michigan-Dearborn most popular approach, which includes linear programming [1], intelligent power controller (UMD_IPC) applies this knowledge to optimal control [2], and, particularly, dynamic programming real-time vehicle power control to achieve improved fuel efficiency. (DP) [3]–[5]. In general, these techniques do not offer an online UMD_IPC has been fully implemented in a conventional (non- solution because they assume that the future drive cycle is hybrid) vehicle model in the powertrain systems analysis toolkit entirely known. However, these results can be used as a bench- (PSAT) environment. Simulations conducted on the standard drive cycles provided by the PSAT show that the performance of the mark for the performance of online power control strategies. If UMD_IPC algorithm is very close to the offline controller that is only the present state of the vehicle is considered, optimization generated using a dynamic programming optimization approach. of the operating points of the individual components can still be Furthermore, UMD_IPC gives improved fuel consumption in a beneficial, but the benefits will be limited [6]–[8]. Interesting conventional vehicle, alternating neither the vehicle nor its components. techniques for deriving effective online control rules based on the results generated by offline DP and quadratic programming Index Terms—Fuel economy, machine learning, road type (QP) can be found in [3] and [9]. and traffic congestion (RT&TC) level prediction, vehicle power management. Recent research has shown that current driving conditions and the driver’s driving style have a strong influence over a vehicle’s fuel consumption and emissions [10], [11]. Driving I. INTRODUCTION patterns exhibited by a real-world driver are the product of USTOMER demand for improved fuel economy is chal- the instantaneous decisions of the driver to respond to the C lenging the automotive industry to produce affordable (physical) driving environment. Specifically, varying road type new that deliver better fuel efficiency without sacri- and traffic conditions, driving trends, driving styles, and vehicle ficing performance, safety, emissions, or reliability. To meet operating modes have had varying degrees of impact on vehicle this challenge, it is very important to optimize the fuel consumption. However, most of the existing vehicle power and the various devices and components of the vehicle system, control approaches do not incorporate knowledge about driving as well as the energy-management strategy that is used to patterns into their vehicle power-management strategies. The main contribution of this paper is an algorithm for optimization Manuscript received October 1, 2008; revised March 12, 2009 and May 14, of vehicle power management that utilizes inferred knowledge 2009. First published July 17, 2009; current version published November 11, of road type and traffic congestion (RT&TC). Only recently has 2009. This was supported in part by the State of Michigan through the 21st Jobs Fund under a grant and in part by the Institute of Advanced Vehicle the research community in vehicle power control begun to ex- Systems, University of Michigan-Dearborn, under Grant 06-1-p1-0727. The plore ways to incorporate knowledge about driving patterns into review of this paper was coordinated by Dr. M. S. Ahmed. online control strategies [12]–[15]. A comprehensive overview J. Park, Z. Chen, L. Kiliaris, and Y. L. Murphey are with the Department of Electrical and , University of Michigan-Dearborn, of intelligent system approaches for vehicle power management Dearborn, MI 48128 USA (e-mail: [email protected]). can be found in [16]. M. L. Kuang and A. M. Phillips are with the Ford Motor Company, Dearborn, MI 48120 USA. This paper presents our research on intelligent vehicle power M. A. Masrur is with the U.S. Army RDECOM-TARDEC, Warren, MI management using machine learning. Specifically, we will 49307 USA. present machine-learning algorithms for learning about the Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. optimal power control parameters for all 11 standard facility- Digital Object Identifier 10.1109/TVT.2009.2027710 specific (FS) drive cycles proposed in [17] and [18] and

0018-9545/$26.00 © 2009 IEEE 4742 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 58, NO. 9, NOVEMBER 2009

Fig. 1. Intelligent power control in a vehicle system. about predicting road types and traffic congestion, as well as tem for predicting roadway type and traffic-congestion level, an online University of Michigan-Dearborn intelligent power Section IV presents the intelligent online vehicle power- controller (UMD_IPC) that applies the knowledge obtained management system, namely, UMD_IPC, Section V presents through machine learning to online vehicle power control with the experiment results, and Section VI presents the conclusion. the online prediction of driving environment by a neural net- work. UMD_IPC has been fully implemented in a conventional II. OPTIMAL POWER CONTROL IN A CONVENTIONAL vehicle model built using the powertrain systems analysis - VEHICLE SYSTEM USING MACHINE LEARNING kit (PSAT) (http://www.transportation.anl.gov/software/PSAT/ Fig. 1 illustrates the interaction between the proposed in- index.html) simulation and tested on 11 drive cycles telligent power controller UMD_IPC and the major power provided by the PSAT library. PSAT is a high-fidelity sim- components in a conventional vehicle system. At any given time ulation developed by Argonne National , during a drive cycle, based on the current vehicle state, which is Argonne, IL, under the direction of and with contributions represented by the current vehicle speed, driver power demand, from Ford, General Motors, and Chrysler. PSAT is a “forward- electrical load and state of charge (SOC) of the battery, the looking” model that simulates vehicle fuel economy and per- UMD_IPC calls the neural network NN_RT&TC to predict the formance in a realistic manner—taking into account transient current RT&TC level and calculates the set point behavior and characteristics. It can simulate a to the battery controller and a resultant feedforward torque com- broad range of predefined vehicle configurations (conventional, pensation to the controller. The variable P , representing electric, fuel cell, series hybrid, parallel hybrid, and power split s the power actually to be charged (P > 0) or discharged (P < hybrid). s s 0) from the battery, is set by the UMD_IPC with the aim of In this research project, the PSAT software is used to build minimizing fuel consumption. The desired engine power P , a high-fidelity vehicle model; simulate drive cycles to gener- eng which is calculated based on the optimal value of P ,isusedto ate numerical data, such as fuel consumption and emissions s find the feedforward torque compensation through the engine and vehicle performance; and implement an intelligent power fuel-efficiency . The functional relationship between P controller UMD_IPC. Experiments will show that the online eng and P is shown as follows: performances of UMD_IPC are very close to the offline optimal s controller built based on DP. In comparison with the default Peng = Pd + Ge2m(Pe,ω) (1) controller used by the vehicle model in PSAT, our results Pe = Pl + Pb (2) showed a maximum of 3.95% fuel reduction in an urban drive P = η (P , SOC,T) (3) cycle. Furthermore, the implementation of UMD_IPC does not b in2out s require the change of any vehicle components. Although the where research results presented in this paper were generated based ω engine speed; on a conventional vehicle model, the proposed technology can Pd driver demanded power at the ; be extended to a hybrid vehicle system, which is the authors’ Pe electrical power from the alternator; ongoing effort. Ge2m(Pe,ω) mechanical power required by the alternator This paper is organized as follows. Section II presents the based on alternator efficiency map Φalt to machine-learning process of optimal power control in a con- produce a given electrical power Pe at a ventional vehicle, Section III presents a neural network sys- given speed; PARK et al.: VEHICLE POWER CONTROL BASED ON MACHINE LEARNING OF OPTIMAL CONTROL PARAMETERS 4743

where γ(Ps,t) is the fuel consumed as a function of Ps(t) at time t. The fuel-consumption function γ(Ps,t) is approximated as a convex quadratic function, i.e.,

2 γ(Ps,t)≈ϕ2(t)Ps(t) +ϕ1(t)Ps(t)+ϕ0(t),ϕ2 >0 (5)

where ϕi represents time-varying coefficients. The objective function then becomes N min J = min γ (Ps(t),t) Ps P s t=1 N   ≈ 2 min ϕ2(t)Ps (t)+ϕ1(t)Ps(t)+ϕ0(t) (6) Ps t=1

where P s contains the optimal values of Ps(t) for t = 1,...,N. To create a well-posed problem, the constraint that the energy in the battery at the end of the drive cycle Es(N) Fig. 2. Battery efficiency map Φbat. must match the energy at the beginning of the cycle Es(0), i.e., Es(N)=Es(0) is applied to the optimization. This constraint Pl electrical power required by the various ve- hicle electrical loads; can be written as N Ps actual power stored in and drawn out of the  − battery; Es(N) Es(0) = Ps(t)=0. (7) SOC battery state of charge; t=1 Pb power output at the battery controller, which By adjoining this constraint to our objective function us- is a function of the internal battery power ing a Lagrange multiplier, we obtain the following Lagrange Ps, SOC, and battery temperature T and is function: denoted as ηin2out(Ps, SOC,T). N   2 ηin2out(Ps, SOC,T) reflects power losses in the battery. In this L (Ps(1),...,Ps(N),λ)= ϕ2(t)Ps(t) + ϕ1(t)Ps(t) paper, ηin2out is derived by modeling the battery-efficiency map t=1 Φbat shown in Fig. 2. The battery-efficiency map contains the N battery charge/discharge curves generated by the battery model + ϕ0(t) − λ Ps(t). (8) in PSAT for SOC = 40%, 50%, and 60%. It appears that within t=1 the battery-efficiency range of 40%–60%, the battery charge By taking the partial derivatives of the Lagrange function L and discharge curves have very little variation. with respect to Ps(t), t =1,...,N, and λ, respectively, and The machine-learning algorithm, namely, learning minimum by setting the equations to 0, combined with (7), the optimal power consumption on FS drive cycles (LMPC_FSDC), is battery power setting at each time step t can be obtained as presented in Section II-B. LMPC_FSDC attempts to learn the follows: values of parameters that minimize the vehicle fuel consump- λ − ϕ1(t) tion function γ, which is empirically modeled as a quadratic Ps(t)= (9) 2ϕ2(t) function of Ps. Using this quadratic function, a fuel consump- tion cost index is defined and solved using a QP approach to where produce optimal values of Ps. The resultant optimal solution N is dependent on the drive cycles. The machine-learning algo- ϕ1(t) 2ϕ2(t) rithm LMPC_FSDC learns the optimal values of the empirical λ = t=1 . N parameters generated by the QP for a defined set of roadway 1 2ϕ2(t) types and congestion levels. The QP approach is based on t=1 the research presented in [9], which is briefly summarized in The above formula for calculating λ requires the knowledge Section II-A. of ϕ1(t) and ϕ2(t) over the entire drive cycle, which is not available in advance to the online controller during normal real- A. Vehicle Power-Optimization Model world driving. To solve this problem, we adopt the method proposed in [9] that uses a proportional–integral controller to The power-optimization problem is modeled as a multistep produce a value of λ online based on the measured energy level decision problem in a drive cycle with N steps that minimizes in the battery Es, i.e., a performance index J, i.e., λ(t)=λ0 + KP (Es(0) − Es(t − 1)) N N t−1 min J = min γ (Ps(t),t)= min γ (Ps(t),t) (4) + KI (Es(0) − Es(p)) . (10) P Ps Ps s t=1 t=1 p=1 4744 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 58, NO. 9, NOVEMBER 2009

TABLE I Then, the online optimal power controller UMD_IPC chooses STATISTICS OF THE 11 FS DRIVE CYCLES the optimal battery power settings based on RTi. The machine- learning algorithm LMPC_FSDC has been developed to learn the optimal power settings for all 11 FS drive cycles, i.e., RTi, i =1,...,11. Fig. 3 shows the major computational steps in LMPC_FSDC. The algorithm requires the use of a high-fidelity vehicle sys- tem modeling and simulation program F, such as PSAT or ADVISOR. Two major steps in the algorithm require the use of such a simulation program. First, we need the simulation program to build a vehicle model V of a particular interest. Second, we run the vehicle model V in the simulation program F to generate step-by-step system state data: Pd(t), Pl(t), and ω(t), t =1,...,N, every standard FS drive cycle, RTi, i = 1,...,11. The optimal parameters ϕ (t), ϕ (t), ϕ (t), λ , and K i 0 1 2 0 P The fuel matrix F_R is generated for all the Ps values and K are obtained by the machine-learning algorithm I within the specific upper and lower bounds of Ps, denoted described in Section II-B with the constraints of the up- as Ps_ min(t) ≤ Ps(t) ≤ Ps_ max(t), which are calculated as per and lower bounds of P (t), which are also discussed s follows. Let Peng_ max(ω(t)) be the maximum engine power in Section II-B. with engine speed ω(t) and Palt_ max(ω(t)) be the maximum mechanical alternator power with the given speed ω(t).Both P (ω(t)) and P (ω(t)) are defined by the vehicle B. Machine Learning About Optimal Power Settings eng_ max alt_ max model V. At each time t, the maximum electrical power at the We model the road environment of a driving trip as a se- alternator Pe_ max(ω(t)) can be calculated by quence of different roadway types, such as local, freeway, and arterial/collector, augmented with different traffic congestion Pe_ max (ω(t)) = Gm2e {min [Peng_ max (ω(t)) levels. Sierra Research Inc. has shown that fuel efficiency and − P (t),P (ω(t))] ,ω(t)} (11) emissions are connected to roadway types as well as traffic d alt_ max congestion levels. They developed a set of 11 standard drive where G (P ,ω) is a function that calculates the elec- cycles presented in [17] and [18], called facility-specific (FS) m2e alt trical power based on the alternator efficiency map Φ for cycles, to represent passenger and light truck operations alt a given mechanical power P and rotational speed ω.The over a broad range of facilities and congestion levels in urban alt min[P (ω(t)) − P (t),P (ω(t))] in (11) repre- areas. Table I shows the most recent definition of these road eng_ max d alt_ max sents the maximum mechanical power at time t. Based on the types [18], along with the labels that we assigned, where V avg engine and alternator constraint, the upper and lower bounds of is the average vehicle speed in meters per second, Vmax is the Ps are calculated by maximum vehicle speed in meters per second, and Amax is the maximum acceleration. The 11 drive cycles are divided into P (t)=η (P (ω(t)) − P (t)) the following four categories of roadway types: 1) freeway; s_ max 1 out2in e_ max l − 2) freeway ramp; 3) arterial; and 4) local. The two categories, Ps_ min 1(t)=ηout2in (0 Pl(t)) (12) freeway and arterial, are further divided into subcategories based on a qualitative measure called level of service (LOS) where ηout2in is a function that calculates the internal battery that describes operational conditions within a traffic stream power Ps, namely the power to be stored or drawn from based on speed and travel time, freedom to maneuver, traffic the battery for a given battery power Pb at the battery ter- interruptions, comfort, and . Six types of LOS are minal, by using the battery efficiency map Φbat shown in defined with labels, i.e., A through F, with LOS A representing Fig. 2. (Pe_ max(ω(t)) − Pl(t)) and (0 − Pl(t)) represent, re- the best operating conditions and LOS F the worst. Each LOS spectively, the maximum and minimum battery power Pb at the represents a range of operating conditions and the driver’s battery terminal at time t. perception of those conditions; however, safety is not included Since the boundary of Ps(t) is also constrained (or restricted) in the measures that establish service levels [18], [19]. In by current SOC, i.e., SOC(t), as shown in Fig. 14, the upper and this paper, we use this set of 11 FS cycles as the standard lower bounds of Ps are measure of roadway types and traffic-congestion levels. For the convenience of description, we label these 11 FS cycles as Ps_ max(t) = min (Ps_ max 1(t),Ps_ max 2(t)) RT1,...,RT11. The problem of optimal vehicle power management is formu- Ps_ min(t) = max (Ps_ min 1(t),Ps_ min 2(t)) (13) lated as follows. Assume that for any given drive cycle DC(t) (t ∈ [0,te], where te is the ending time of the drive cycle) at any where Ps_ max 2(t) and Ps_ min 2(t)) can be calculated based on given time t, the vehicle is operating according to one of the 11 SOC(t). A fuel-rate matrix F_R(Ps(t),t| ω(t),Pd(t),Pl(t)) is road types and traffic-congestion levels, i.e., RTi, i =1,...,11. generated for each time step t as a function of Ps(t), which is PARK et al.: VEHICLE POWER CONTROL BASED ON MACHINE LEARNING OF OPTIMAL CONTROL PARAMETERS 4745

Fig. 3. Computational steps of machine-learning algorithm LMFC_FSDC.

i i i i i the charge and discharge power within the system constraints control parameters ϕ1, ϕ2, λ , KP , and KI as follows: specified in (13) at time t for the given engine speed ω(t), N N required drivetrain power Pd(t), and electric load power Pl(t). 1 1 ∗ ϕi = ϕi (t) ϕi = ϕi (t) (14) The tth column of matrix F_R( ,t) is represented as a convex 1 N 1 2 N 2 t=1 t=1 quadratic cost function of Ps. By using a regression function, we can obtain the coefficients ϕ2(t), ϕ1(t), and ϕ0(t) such N i ϕ1(t) 2 ≈ ∗ i that ϕ2(t)Ps (t)+ϕ1(t)Ps(t)+ϕ0 F_R( ,t) with the 2ϕ2(t) λi = t=1 (15) best fit. N 1 Fig. 4 shows a few example of the actual fuel rates and 2ϕi (t) the convex quadratic cost functions calculated at various t=1 2 i i · · −2 i i · · −6 time steps associated with the vehicle model Ford Taurus KP = ϕ2 4 10 KI = ϕ2 6 10 . (16) in the Arterial AB drive cycle, i.e., RT8. Note the fuel rate has been multiplied with the chemical-energy contents of The machine-learning algorithm to the 11 standard FS drive fuel, i.e., Hf =44 kJ/g [20], to obtain a suitable scaling, cycles and the results are shown in Table II, which serves as and Ps values have been normalized as follows: {Ps(t) − the knowledge base for the online controller UMD_IPC. mean(Ps(·))}/σ(Ps(·)), where mean(Ps(·)) is the mean of Ps, and σ(Ps(·)) is the standard deviation of Ps. This illustrates III. PREDICTING ROADWAY TYPE AND that a quadratic function is a good choice to represent the TRAFFIC CONGESTION LEVEL fuel function F_R(Ps(t),t| ω(t)Pd(t),Pl(t)).Fig.5shows the coefficients {ϕ2(t),ϕ1(t),ϕ0(t)| t = 511,...,533} of the The problem of roadway type prediction is formulated as quadratic cost function of Ps(t) for the same drive cycle used follows. Let SP(t) be the speed profile of a driver on the road in Fig. 4. t =0, 1,...,tc, where tc is the current time instance, and let These coefficients obtained at all time steps for each drive R(t) be the roadway types that the driver needs to go through cycle RTi, i =1,...,11 are then used to calculate the power to complete his trip 0

TABLE II OPTIMAL PARAMETER SETTINGS GENERATED BY THE MACHINE-LEARNING ALGORITHM LMFC_FSDC, FOR 11 STANDARD FS DRIVE CYCLES

Fig. 4. Actual and calculated fuel rate for the arterial AB drive cycle RT8.

time step. To solve this problem, four different aspects of the roadway type predictor need to be determined as follows. 1) Select effective features that can be extracted from SP(t), tc − ΔZ

TABLE III FOURTEEN FEATURES SELECTED FOR ROADWAY-TYPE PREDICTION

to represent the 11 FS drive cycles. One important issue in a multiclass neural network classifier is the proper encoding of the classes in the output nodes of the neural network. We chose to use a “one-hotspot” method [24] described as follows. Since this is an 11-class classification problem, we need an 11-bit output layer. Each class is assigned a unique binary string (codeword) of length k. For example, class 1 is assigned a codeword of 00000000001, class 2 is assigned a codeword of 00000000010, class 3 is assigned of a codeword 00000000100, etc. The advantage of this encoding is that it gives enough tolerance among different classes. The neural network is trained using the well-known back- propagation algorithm for weight update. Based on the study results presented in the last section, we use ΔZ = 150 s and Δt =3s. The training and test data are generated from 11 Sierra data and 11 PSAT drive cycles as follows. The Fig. 6. Segments of a speed profile. feature vector x1,x2,...,x14 is generated as follows. For each drive cycle DC(t)(0≤ t ≤ te), DC segments are gen- more likely we will catch all the road type transitions. However, erated on the intervals s0 =[t0, ΔZ),...,sk =[kΔt, ΔZ + as we stated before, smaller time steps demand more computa- kΔt),...,ske =[te − ΔZ, te], where k ≥ 1. From the speed tional power. As a tradeoff, Δt =3and ΔZ = 150 were used function of each segment, we extract a vector of the 14 features in the experiments presented in this paper since they gave good specified in Table III. The feature vector extracted from every performances on both the training and test data. In Section V, speed signal segment is labeled by the roadway type of its next we also analyzed the fuel efficiency with three different time segment since we are training the prediction function. steps. There are totally 4399 segments generated from these 22 drive cycles. The separation of training and test data is through a random stratified sampling procedure. The resulting C. Training a Neural Network to Predict Road Types training data contain 3519 feature vectors, and the test data We developed a multilayered multiclass neural network, contain 880 feature vectors. The performance of the neural namely, NN_RT&TC, for the prediction of road types and network on the fivefold cross validation is 95% on the training traffic congestion levels. Fig. 8 shows the architecture of data and 94% on the test data. NN_RT&TC. The input layer has 14 nodes for the features When NN_RT&TC is used inside a vehicle to predict the specified in Table III, a hidden layer of 20 nodes, and 11 roadway type at time tc, the vector of the 14 features is extracted output nodes representing the 11 class labels {RT1,...,RT11} from the vehicle speed during the time interval [tc − 150 s, tc]. PARK et al.: VEHICLE POWER CONTROL BASED ON MACHINE LEARNING OF OPTIMAL CONTROL PARAMETERS 4749

Fig. 7. Prediction accuracies using various window sizes and time steps.

Fig. 9. Example of segmented, labeled, and predicted drive cycle LA92.

by the neural network NN_RT&TC are shown in blue. Notice Fig. 8. Architecture of NN_RT&TC. that there is a delay in the prediction for the first 150 s.

IV. UMD_IPC: AN INTELLIGENT ONLINE The output from NN_RT&TC is the roadway type that is used VEHICLE POWER CONTROLLER to produce the optimal power distribution during time interval [tc,tc +3 s]. We use 11 PSAT drive cycles as test data to The intelligent power controller UMD_IPC, which contains evaluate the UMD-IPC system. The PSAT drive cycles can be the neural network NN_RT&TC, has been fully implemented considered as composites of the 11 classes of roadway types in the PSAT simulation environment. Fig. 10 gives the major and traffic congestion levels. Fig. 9 shows an example of a computational steps of UMD_IPC at any given time t during a drive cycle LA92 that is segmented and labeled according to real-world drive cycle. The UMD_IPC has the knowledge base { i i i i i | the definition of the 11 standard FS RT&TC classes, as defined KB = ΔZ = 150 s, Δt =3s, ϕ1, ϕ2,λ ,KP , and KI i = in [18]. The x-axis indicates the time, and the y-axis indicates 1,...,11}, which is generated by the machine-learning algo- the speed in meters per second. The prediction results generated rithm presented in Section III. 4750 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 58, NO. 9, NOVEMBER 2009

Fig. 10. Computational flow of UMD_IPC: an intelligent vehicle power controller.

At any time t during a real-time drive cycle in a vehicle sys- The battery energy at the current time Es(t) is calculated tem, UMD_IPC is able to obtain the vehicle state V_state(t)= based on the current SOC(t) followed by the calculation of λ(t) {vs(t),Pd(t),Pl(t),ω(t), SOC(t)}. If the vehicle is at the using the following formula: start mode, i.e., t<ΔZ, UMD_IPC uses the default power r − − control. When t =ΔZ, UMD_IPC gets the current vehicle λ(t)=λ0 + KP (Es(0) Es(t 1)) state V_state(t) and calls the neural network NN_RT&TC t−1 to make the first prediction of the roadway type and traffic r − + KI (Es(0) Es(p)) . (17) congestion level. Based on the road type R(t) predicted by p=1 NN_RT&TC, UMD_IPC retrieves the optimal control parame- r r r r ters associated with the road type r = R(t), ϕ1, ϕ2, λ , KP , Fig. 11 shows the λ(t) values in the simulation of all 11 PSAT r r and KI . If this is the first prediction, λ is used as the initial drive cycles, which change with time and the road-prediction value, i.e., λ0. results. PARK et al.: VEHICLE POWER CONTROL BASED ON MACHINE LEARNING OF OPTIMAL CONTROL PARAMETERS 4751

Fig. 13. Palt_ max(ω): max mechanical alternator power with alternator speed.

Fig. 11. λ values for all 11 PSAT drive cycles. Fig. 14. Ps boundary at time t as a function of SOC(t) value.

The optimal engine power at time t is calculated using the formula

o o Peng(t)=Pd(t)+Ge2m (Pl + ηin2out (Ps (t)) ,ω) . (20)

o o Both Ps (t) and Peng(t) are sent to the vehicle system, and the UMD_IPC continues the process at time t +1.

V. E XPERIMENTS Fig. 12. Peng_ max(ω): max engine power with engine speed. UMD_IPC has been implemented in a conventional vehicle The instantaneous fuel-rate matrix at time t, i.e., model provided by the PSAT software, namely, a Ford Taurus F_R(P (t),t| P (t),P (t),ω(t)), is calculated through s d l with a 95-kW 1.9-L Spark Ignition engine, five- manual the following procedure based on the current constraints of , a 12–14-V 2-kW alternator, and a 66-A · h/12-V P (t), the engine power P , the engine speed ω(t), and the s eng lead acid battery. Since electrical loads in passenger vehicles, engine efficiency map Φ , which is provided by the vehicle eng usually, are no larger than 1000 W, a constant electrical load system as a function of engine power and engine speed. Pl = 1000 W is used in all the simulations. Figs. 12 and 13 For every Ps(t) such that Ps_ min(t)

Peng(t)=Pd(t)+Ge2m (Pl(t)+ηin2out (Ps(t)) ,ω) (18) constraints of Ps with various SOC values. The online controller UMD_IPC is applied to all 11 PSAT where Ge2m calculates the mechanical power based on the drive cycles. Figs. 15 and 16 show the detailed experiment alternator efficiency map Φalt for the given electrical power results generated from the three most interesting driving Pe(t)=Pl(t)+ηin2out(Ps(t)) at the given speed ω, and cycles, namely, UDDS, LA92, and UNIF01. UDDS, which is ηin2out calculates the corresponding battery power output at sometimes called FTP72, represents city driving conditions the terminal based on the battery efficiency map Φbat shown in an urban area with frequent stops. LA92, which is also in Fig. 2 for the given the internal battery power Ps. Then, called unified cycle, was constructed from segments of an F_R(Ps(t),t| Pd(t),Pl(t),ω(t)) = Φeng(Peng(t),ω(t)). actual driving recording in Los Angeles. It is a more aggressive The optimal power to be charged to or discharged from the driving cycle than the federal test procedure (FTP) as it has battery is obtained by searching through the fuel-rate matrix for higher speeds, higher acceleration rates, fewer stops per meter, the Ps that minimizes the following quantity: and less idle time (see Fig. 9). The UNIF01 cycle developed by Sierra Research for the California Air Resources Board is o { − } Ps (t) = arg min F_R (Ps(t),t) λ(t)Ps(t) (19) a modified form of the LA92. For the purpose of comparison, Ps(t) we applied offline DP controller to these three drive cycles in where Ps_ min(t) ≤ Ps(t) ≤ Ps_ max(t). the attempt of finding the optimal benchmark performances. 4752 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 58, NO. 9, NOVEMBER 2009

Fig. 15. SOC comparison on three driving cycles. The x-axis represents time measured in seconds, and the y-axis represents the SOC measured in percentages. (a) SOC compensation during driving cycle UDDS. (b) SOC compensation during driving cycle UNIF01. (c) SOC compensation during driving cycle LA92. It should be kept in mind that the DP controller is not applicable ferent power controllers, namely, the offline DP controller, the to online control [3], [9] since it requires full knowledge of the Ford Taurus controller provided by PSAT, and the UMD_IPC entire drive cycle to optimize the power management strategy controller. It can be observed that the SOC curves generated at each time. by the UMD_IPC from all three drive cycles behave quite In Fig. 15, we show that the battery SOC generated during similarly to the respective ones generated by the offline DP the simulation run of the three drive cycles using the three dif- controller, whereas the SOC curves generated by the Ford PARK et al.: VEHICLE POWER CONTROL BASED ON MACHINE LEARNING OF OPTIMAL CONTROL PARAMETERS 4753

Fig. 16. Comparisons of the battery power Ps generated by the three controllers DP, UMD_IPC, and Ford Taurus. (a) Battery power Ps generated during drive cycle UDDS.(b) Battery power Ps generated during drive cycle UNIF01.

Taurus controller are significantly different from the optimal consumption, and (b) presents the fuel saved. We use the curves. Fig. 16 presents battery power Ps dynamically gener- fuel consumed by the Ford Taurus in PSAT as the baseline ated by the three controllers for UDDS and UNIF01 cycles. The to measure the fuel saved by the DP and UMD_IPC con- battery power for DP and UMD_IPC controllers are discretized trollers. The UMD_IPC gave more than 2% savings on fuel by a step size of 50 W. These graphs clearly show that the consumption from six drive cycles, namely, UDDS, UNIF01, battery powers generated by the UMD_IPC controller are close LA92, 505UDDS, SC03, and TripEPA. In particular, for the to the optimal ones generated by DP. UDDS, UNIF01, and LA 92 drive cycles, UMD_IPC’s perfor- Fig. 17 presents the performance comparison with respect mances are very close to the optimal (DP) controller: For the to fuel consumption on all 11 PSAT drive cycles generated UDDS drive cycle, UMD_IPC saved 3.95% fuel, while the DP by the same three power controllers; (a) presents the fuel controller saved 4.05%; for the UNIF01 drive cycle, UMD_IPC 4754 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 58, NO. 9, NOVEMBER 2009

Fig. 17. Performance comparison on fuel consumption. (a) Total fuel consumption with Pl = 1000 W. (b) Fuel saving with Pl = 1000 W.

Fig. 18. Fuel efficiency comparisons on different time steps used by UMD_IPC. PARK et al.: VEHICLE POWER CONTROL BASED ON MACHINE LEARNING OF OPTIMAL CONTROL PARAMETERS 4755 saved 3.29% fuel, while the DP controller saved 3.47%; and drive and is therefore easy to implement in an existing for the LA 92 drive cycle, UMD_IPC saved 3.05% fuel, while conventional vehicle configuration. Currently, we are develop- the DP controller saved 3.15%. These results demonstrate that ing machine-learning technologies with applications to hybrid UMD_IPC is able to realize good fuel-economy improvements vehicle power-management systems. We anticipate that more over the existing conventional control strategy in all drive significant fuel reduction will be achieved in hybrid vehicle cycles, and on some drive cycles, it can give near-optimal power systems. performances. We also studied the fuel efficiency with three different time REFERENCES steps, i.e., Δt =1, 3, and 5 s, and the results are shown in [1] E. D. Tate and S. P. Boyd, “Finding ultimate limits of performance for Fig. 18. It appears that UMD_IPC gave similar performances hybrid electric vehicles,” presented at the Soc. Automotive Eng. Fu- for all three time steps within the precision of two digits. To ture Transp. Technol. Conf., Detroit, MI, 2000, Soc. Automotive Eng. analyze the computational efficiency of UMD_IPC, the follow- Paper-01-3099. [2] S. Delprat, J. Lauber, T. M. Guerra, and J. Rimaux, “Control of a parallel ing experiment is conducted using a computer that has a 3-GHz hybrid powertrain: Optimal control,” IEEE Trans. Veh. Technol., vol. 53, Intel Core 2 Duo processor and a 4-GB DDR3 random access no. 3, pp. 872–881, May 2004. memory. We applied UMD_IPC to the drive cycle UDDS [3] C.-C. Lin, H. Peng, J. W. Grizzle, and J.-M. Kang, “Power management with a time step of Δt =3 s for neural network prediction strategy for a parallel hybrid electric truck,” IEEE Trans. Control Syst. Technol., vol. 11, no. 6, pp. 839–849, Nov. 2003. and update the optimal battery power Ps every second. The [4] T. Hofman and R. van Druten, “Energy analysis of hybrid vehicle pow- entire simulation was completed in 300 s. Since UDDS has ertrains,” in Proc. IEEE Int. Symp. Veh. Power Propuls., Paris, France, 1370 s, the time takes to calculate the optimal power setting, Oct. 2004. [5] I. Arsie, M. Graziosi, C. Pianese, G. Rizzo, and M. Sorrentino, “Opti- which includes the time needed for neural network prediction mization of supervisory control strategy for parallel hybrid vehicle with of roadway types, is 0.22 s on the average. provisional load estimate,” in Proc. 7th Int. Symp. AVEC, Arnhem, The Netherlands, Aug. 2004. [6] V. H. Johnson, K. B. Wipke, and D. J. Rausen, “HEV control strategy for VI. CONCLUSION real-time optimization of fuel economy and emissions,” presented at the Soc. Automotive Eng. Future Transp. Technol. Conf., Detroit, MI, 2000, The authors have presented an intelligent vehicle power Paper-01-1543. controller, namely, the UMD_IPC, which has been devel- [7] G. Paganelli, G. Ercole, A. Brahma, Y. Guezennec, and G. Rizzoni, oped through machine learning of optimal control parame- “General supervisory control policy for the energy optimization of charge- sustaining hybrid electric vehicles,” Soc. Autom. Eng. Jpn. Rev., vol. 22, ters with respect to 11 FS road types and traffic conges- no. 4, pp. 511–518, Apr. 2001. tion levels. UMD_IPC contains a neural network, namely, [8] A. Sciarretta, L. Guzzella, and M. Back, “A real-time optimal control the NN_RT&TC, designed and trained for in-vehicle pre- strategy for parallel hybrid vehicles with on-board estimation of the con- trol parameters,” in Proc. IFAC Symp. Adv. Autom. Control, Salerno, Italy, diction of 11 different roadway types and traffic-congestion Apr. 19–23, 2004. levels. We have also presented a feature extraction algorithm [9] M. Koot, J. T. B. A. Kessels, B. de Jager, W. P. M. H. Heemels, to extract effective features from vehicle speed segments as P. P. J. van den Bosch, and M. Steinbuch, “Energy management strategies the input to NN_RT&TC and showed the importance of for vehicular electric power systems,” IEEE Trans. Veh. Technol., vol. 54, no. 3, pp. 771–782, May 2005. the two parameters, namely, ΔZ, which is the signal win- [10] E. Ericsson, “Variability in urban driving patterns,” Trans. Res., Part D, dow size, and Δt, which is the prediction step, with respect vol. 5, no. 5, pp. 337–354, Sep. 2000. to the accuracy of the prediction results of NN_RT&TC. [11] E. Ericsson, “Independent driving pattern factors and their influence on fuel-use and exhaust emission factors,” Trans. Res., Part D, vol. 6, no. 5, An offline machine-learning algorithm was proposed to pp. 325–341, Sep. 2001. generate a knowledge base that contains the optimal control [12] S.-I. Jeon, S.-T. Jo, Y.-I. Park, and J.-M. Lee, “Multi-mode driving control parameters for all 11 standard FS drive cycles. During an ofaparallelhybridelectricvehicleusingdrivingpatternrecognition,”Trans. ASME,J.Dyn.Syst.Meas.Control, vol. 124, no. 1, pp. 141–149, Mar. 2002. online control process, UMD_IPC, which is the proposed on- [13] I. Kolmanovsky, I. Siverguina, and B. Lygoe, “Optimization of powertrain line controller, applies the appropriate optimal control param- operating policy for feasibility assessment and calibration: Stochastic dy- eters in the knowledge base to the current road type predicted namic programming approach,” in Proc. Amer. Control Conf., Anchorage, by the neural network NN_RT&TC to generate optimal power AK, May 2002, vol. 2, pp. 1425–1430. [14] R. Langari and J.-S. Won, “Intelligent energy management agent for a par- to be charged to or discharged from the battery and the optimal allel hybrid vehicle—Part 1: System architecture and of the driving engine power. UMD_IPC has been fully implemented in a con- situation identification process,” IEEE Trans. Veh. Technol., vol. 54, no. 3, ventional vehicle model, Ford Taurus, in the PSAT simulation pp. 925–934, May 2005. [15] J.-S. Won and R. Langari, “Intelligent energy management agent for a environment and tested on the 11 drive cycles provided by the parallel hybrid vehicle—Part 2: Torque distribution, charge sustenance PSAT library. Our simulation results show that UMD_IPC gave strategies, and performance results,” IEEE Trans. Veh. Technol., vol. 54, a performance within less than 0.15% of the optimal DP, which no. 3, pp. 935–953, May 2005. [16] Y. L. Murphey, “Intelligent vehicle power management—An overview,” is an offline optimal controller, for eight drive cycles, and less in A Chapter in the Studies in Computational Intelligence (SCI), than 0.25% for the remaining drive cycles. The maximum fuel vol. 132. Berlin, Germany: Springer-Verlag, 2008, pp. 169–190. saved in comparison to the Ford Taurus controller is 3.95%. [17] T. R. Carlson and R. C. Austin, “Development of speed correction cycles,” UMD_IPC saved more than 2% fuels in six drive cycles and Sierra Res., Inc., Sacramento, CA, Rep. SR97-04-01, 1997. [18] “SCF improvement—Cycle development,” Sierra Res. Sacramento, CA, more than 1% fuel in eight drive cycles. In conclusion, the pro- Sierra Rep. No. SR2003-06-02, 2003. posed machine-learning technique, combined with roadway- [19] Highway Capacity Manual 2000, Trans. Res. Board, Washington, DC, type prediction, is an effective approach in real-time intelligent 2000. [20] J. T. B. A. Kessels, “Energy management for automotive power,” vehicle power management. Furthermore, the proposed vehicle Ph.D. dissertation, Mech. Eng., Techn. Univ. Eindhoven, Eindhoven, The power control technology does not require any changes to the Netherlands, 2007. 4756 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 58, NO. 9, NOVEMBER 2009

[21] F. Ferri, P. Pudil, M. Hatef, and J. Kittler, “Comparative study of tech- M. Abul Masrur (M’84–SM’93) received the Ph.D. niques for large scale feature selection,” in Pattern Recognition in Prac- degree in from Texas A&M tice IV, E. Gelsema and L. Kanal, Eds. Amsterdam, The Netherlands: University, College Station, in 1984. Elsevier, 1994, pp. 403–413. Between 1984 and 2001, he was with Ford Re- [22] Y. L. Murphey and H. Guo, “Automatic feature selection—A hybrid sta- search and then joined the U.S. Army tistical approach,” in Proc. Int. Conf. Pattern Recog., Barcelona, Spain, RDECOM-TARDEC, Warren, MI, where he has Sep. 3–8, 2000, pp. 382–385. been involved in various vehicular electric power- [23] J. A. Crossman, H. Guo, Y. L. Murphey, and J. Cardillo, “Automotive system architecture concepts, electric power man- signal fault diagnostics—Part 1: Signal fault analysis, feature extraction, agement, and inverter fault diagnostics. and quasi optimal signal selection,” IEEE Trans. Veh. Technol., vol. 52, Dr. Masrur was the recipient of the Best Auto- no. 4, pp. 1063–1075, Jul. 2003. motive Paper Award from the IEEE [24] G. Ou and Y. L. Murphey, “Multi-class pattern classification using neural Vehicular Technology Society for his transactions in 1998 and the networks,” Pattern Recognit., vol. 40, no. 1, pp. 4–18, Jan. 2007. 2006 Society of Automotive Environmental Excellence in Trans- portation Award. He is the current Chair of the Motor Subcommittee within the IEEE Power and Energy Society. He served as an Associate Editor for the IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY from 1999 Jungme Park received the B.S. degree in statistics to 2007. from Korea University, Seoul, Korea, in 1989 and the M.S. and Ph.D. degrees in computer science from the University of Alabama, Tuscaloosa, in 2001. Anthony M. Phillips received the B.A. degree She is currently a Research Scientist with the (magna cum laude) in physics from Gustavus Adol- Department of Electrical and Computer Engineering, phus College, St. Peter, MN, in 1990 and the M.S. University of Michigan, Dearborn. Her current re- and Ph.D. degrees in — search interests include computer vision, optimiza- control systems from the University of California, tion, and vehicle power management of conventional Berkeley, in 1993 and 1995, respectively. and hybrid electric vehicles. Upon completing his study, he joined the Ford Motor Company, Dearborn, MI as a Product Devel- opment . He was appointed as a Technical Expert when he joined the Research and Advanced Zhihang Chen received the Ph.D. degree in applied Engineering staff in 1998. In his current position as mathematics from Peking University, Beijing, China, a Senior Technical Leader, he has responsibility for Ford’s advanced vehicle in 2000. control system development for hybrid and fuel-cell electric vehicles. He is He is currently a Research Scientist with the Uni- a member of the Editorial Board of the International Journal of Alternative versity of Michigan, Dearborn. His research interests Propulsion. His research interests include vehicle energy management, distrib- include machine learning and intelligent systems, uted system control, and control system development and methods. He is with applications to vehicle power management. the holder of 29 U.S. and international in automotive controls. Dr. Phillips is a member of the Society of Automotive Engineers and the American Society of Mechanical Engineers.

Yi Lu Murphey (SM’97) received the Ph.D. de- Leonidas Kiliaris was born in Trenton, MI, in gree in computer engineering from the University of 1983. He received the B.Sc.Eng. and M.Sc.Eng. de- Michigan, Ann Arbor, in 1989. grees in electrical engineering from the University of She is currently a Professor and the Chair of the Michigan, Dearborn, in 2006 and 2009, respectively. Department of Electrical and Computer Engineering, He is currently conducting research with the Uni- University of Michigan, Dearborn. Her current re- versity of Michigan in power management of light- search interests include machine learning, computer and heavy-duty conventional and hybrid electric vision, and intelligent systems, with applications to vehicles. engineering diagnostics, vehicle power management, and robotic vision systems.

Ming L. Kuang received the B.S. degree in mechan- ical engineering from the South China University of Technology, Guangzhou, China, in 1982 and the M.S. degree in mechanical engineering from the University of California, Davis, in 1991. Since 1991, he has been with the Ford Motor Company, Dearborn, MI, in various engineering po- sitions. He became a Technical Expert in 2000 for the Escape Hybrid vehicle program and played a critical role in the development and implementation of the vehicle/powertrain control system, delivering the first Ford Escape Hybrid and Mercury Mariner Hybrid vehicles to pro- duction. He is currently a Technical Leader in vehicle controls in research and advanced engineering. His primary research interests include vehicle control architecture, vehicle control system development, and implementation methodologies, as well as advanced vehicle control algorithm development for hybrid and fuel-cell vehicles. He is the author or coauthor of 20 technical papers in various engineering journals and conferences. He is the holder of 36 U.S. and international patents. Mr. Kuang was the recipient of the 2005 Henry Ford Technology Award and the Society of Automotive Engineers 2007 Henry Ford II Distinguished Award for Excellence in Automotive Engineering.