Intelligent Train Automatic Stop Control (Itasc)

INTELLIGENT TRAIN AUTOMATIC STOP CONTROL (ITASC)

Ali Siahvashi BSc in Electrical Engineering

Shiraz University of Technology

MSc in Electrical Railway Engineering

Iran University of Science and Technology (IUST)

A Thesis Submitted in fulfilment of the requirements for the degree of Doctor of Philosophy in the

Department of Computing Faculty of Science and Engineering Macquarie University

Supervisor: Prof. Mehmet Orgun Associate Supervisor: Prof. Yang Wang 2020

Keywords

Algorithm, Jerk, iTASC, Stopping position, Station, Train, Train automation stop controller, Communication-based train control system, Railway control and signalling system, Rolling stock, Locomotive, Rolling stock brake system, Rolling stock brake distance, Service brake, Normal brake, Emergency brake, wagon brake, Precise stopping, Stopping errors, railway station, Platform screen door, Safety, Punctuality,

Automatic train operation, Automatic train protection, Automatic train supervision,

Consist, Artificial intelligence, Machine learning, Reinforcement learning, Q-learning,

Fuzzy control, Double q-learning, Fuzzy double q-learning.

Abstract

In a conventional train signalling system, stopping a train at stations is the responsibility of train drivers. Before each station, a signal known as Home Signal in railway terminology, warns the driver that the train is approaching a station.

However, due to different brake system characteristics and capabilities, different track profiles as well as different competency levels of drivers, it is a challenging task to stop a train precisely by just one braking action while maintaining a uniform quality of ride.

In addition to this, the use of platform screen doors (PSD) in railway stations can introduce various challenges for planners, track engineers, rolling stock manufacturers, brake engineers and PSD suppliers. Monitoring stopping spots, the braking rate, and real data are the initial requirements for any further development and evaluation for a sound and stable train control system.

In the last three decades, train automatic stop control (TASC) algorithms have been developed and applied to different metro and heavy haul rail corridors all over the globe. However, even the most developed controllers have relied entirely on station markers such as home signals, on-the-track sensors or Balises. Although, position uncertainty has been considered in several studies before, it has been largely ignored in TASC studies so the foremost shortcoming of previously developed TASC algorithms is that they had not considered position uncertainty. The second most important problem with these algorithms for TASC is the exclusion of the inherent time delay in braking systems in response to any control signal.

iii

Therefore, to consider those factors, a braking model for station stopping is developed in this thesis, which accounts for the time dependency of the train’s air brake system to improve the accuracy of the train’s stopping.

Finally, train position uncertainty, which is a missing concern in previous works, has been added to this thesis’s study.

Table of Contents

Keywords ...... i Abstract ...... iii Table of Contents ...... v List of Figures ...... vii List of Tables ...... ix List of Algorithms ...... x List of Abbreviations ...... xi Statement of Originality ...... xiv Acknowledgements ...... xv Chapter 1: Introduction ...... 1 1.1 Background and Motivation ...... 1 1.2 Research Design, Aims and Objectives of the Thesis ...... 6 1.3 Significance of this research ...... 8 1.4 Key Innovations of The Research ...... 9 1.5 Structure of the Thesis ...... 10 Chapter 2: Literature Review...... 15 2.1 Introduction ...... 15 2.2 The Background of the Study ...... 18 2.3 Train Automatic Stop Control (TASC) ...... 23 2.4 Rolling Stock Dynamic Behaviour ...... 30 2.5 Rolling Stock Braking Model ...... 30 2.6 Summary ...... 31 Chapter 3: Train Dynamic Modelling ...... 39 3.1 Introduction ...... 39 3.2 Train Modelling ...... 40 3.3 Train Active Force Modelling ...... 41 3.4 Train Brake Principles ...... 43 3.5 Train Brake Modelling ...... 54 3.6 TASC Modelling ...... 60 3.7 Brake Parameters ...... 65 3.8 Simulation Parameters ...... 66

3.9 Conclusion ...... 67 Chapter 4: TASC Benchmark ...... 70 4.1 Introduction ...... 70 4.2 TASC Benchmarck ...... 72 4.3 Simulation Data Set ...... 92 4.4 Simulation Results and Comparison ...... 93 4.5 Conclusion ...... 95 Chapter 5: iTASC ...... 99 5.1 Introduction ...... 99 5.2 Reinforcement Learning ...... 100 5.3 Q-Learning ...... 105 5.4 Fuzzy Double Q-Learning ...... 108 5.5 iTASC Closed Control Loop ...... 112 5.6 Simulation Results ...... 116 5.7 Conclusion ...... 120 Chapter 6: Conclusions ...... 124 6.1 Summary ...... 124 6.2 The Main Contribution ...... 126 6.3 Future Research Directions ...... 126 Bibliography ...... 131

List of Figures

Figure 1-1: TASC Module beneath a Tokyu 7000 EMU, Japan (Source: Wikipedia) ...... 4 Figure 1-2: Signalling system, ATO and TASC Relationship ...... 6 Figure 2-1: Train Automatic Stop Control [2]...... 15 Figure 2-2: ATO Speed Profile ...... 33 Figure 2-3: Speed Adjustment [62] ...... 33 Figure 3-1: Stationary Train Forces ...... 40 Figure 3-2: Train Forces in Uphill, Downhill and Tangent Track Profiles ...... 42 Figure 3-3: Wheel -Tread Brake System [103] ...... 45 Figure 3-4: Axle-Mounted Disc Brake System [103] ...... 45 Figure 3-5: Wheel-Mounted Disc Brake System [103] ...... 45 Figure 3-6: Locomotive Brake System [37] ...... 48 Figure 3-7: SIEMENS locomotive ER24PC, Braking Resistor Components; (1) Radial resistor sub assembly, (2) Blower, (3) Resistor high voltage connection, (4) Terminal box ...... 50 Figure 3-8: Faiveley Transport Parking Brake Scheme ...... 51 Figure 3-9: Automatic Air Brake Main Components [37]...... 53 Figure 3-10: Sliding Valve And Graduating Valve Positions: (A) Release, (B) Preliminary Quick Service, (C) Brake, and (D) Lapping [37] ...... 54 Figure 3-11: Train Speed Profile ...... 63 Figure 4-2: Example of SVMs Classification to Find a Large Margin Classifier in the given binary classification problem (Between red and blue spots) ...... 73 Figure 4-3: SVR uses the samples that are further away from the predicted value (blue circles with red outline) and ignores samples which are close to the predicted line (blue circles). The solid line is the prediction and dashed line is the margin...... 74 Figure 4-4: Loss function Based on Slack Concept ...... 77 Figure 4-5: SVR Technique for TASC, Station 1 ...... 80 Figure 4-6: SVR Technique for TASC, Station 1 ...... 81 Figure 4-7: SVR technique for TASC, station 2 ...... 82 Figure 4-8: SVR Technique for TASC, Station 3 ...... 83 Figure 4-9: Structure of a Feedforward Network with One Hidden Layer [105] ...... 84 Figure 4-10: MATLAB Neural Network Training Interface ...... 85

vii

Figure 4-11 Structure of the Neural Network with One Hidden Layer and Braking Distance as its Output ...... 86 Figure 4-12: ANN Technique for TASC, Station 1 ...... 87 Figure 4-13: ANN Technique for TASC, Station 2 ...... 88 Figure 4-14: ANN Technique for TASC, Station 3 ...... 88 Figure 4-15: ANN Technique for TASC, Station 4 ...... 89 Figure 4-16: Fuzzy System Block Diagram [99]...... 89 Figure 4-17: Fuzzy Function of Membership for Input and Output of Trains [5]...... 90 Figure 4-18: FC Technique for TASC, Station 1 ...... 91 Figure 4-19: FC Technique for TASC, Station 2 ...... 91 Figure 4-20: FC Technique for TASC, Station 3 ...... 92 Figure 4-21: FC Technique for TASC, Station 4 ...... 92 Figure 5-1: iTASC Relationship with AI and Rail System Disciplines ...... 100 Figure 5-2: Comparison Between Different Types of ML [96] ...... 101 Figure 5-3: RL Elements, Sutton and Barto [87] ...... 102 Figure 5-4 Fuzzy Memberships Functions ...... 109 Figure 5-5: CBTC iTASC Closed Control Loop ...... 114 Figure 5-6: iTASC Procedure and Flowchart ...... 115 Figure 5-7: iTASC Algorithm, Station 1 ...... 117 Figure 5-8: iTASC Algorithm, Station 2 ...... 118 Figure 5-9: iTASC Algorithm, Station 3 ...... 118 Figure 5-10: iTASC Algorithm, Station 4 ...... 119 Figure 5-11: The Mean Square Error Averaged on 20 Folds of the Best Three Models ...... 119

viii

List of Tables

Table 2-1: Summary of TASC Algorithms ...... 36 Table 3-1: Train Forces ...... 43 Table 3-2: RISSB’s Minimum Braking Distances ...... 48 Table 3-3: TASC Braking Model ...... 60 Table 4-1: Algorithms in the TASC Benchmark ...... 71 Table 4-2: The Results of the Single Model Trained on All Data ...... 94 Table 5-1 iTASC Simulation Parameters ...... 116 Table 5-2 Summary of TASC Algorithms in Three Stations ...... 120

List of Algorithms

Algorithm 4-1: A Typical Backpropagation Algorithm ...... 86 Algorithm 5-1: Fuzzy Double Q-learning Pseudocode for iTASC ...... 111

List of Abbreviations

AAB Automatic Air Brake ADP Approximate Dynamic Programming AI Artificial Intelligence AP Access Point ARTC Australian Rail Track Corporation ASM Average Speed Method ATC Automatic Train Control ATO Automatic Train Operation ATP Automatic Train Protection ATS Automatic Train Supervision BA Braking Attempt BCU Brake Control Unit BD Braking Distance CART Classification and Regression Tree CBTC Communication-Based Train Control CI Confidence Interval CC Cooperative Control DB Dynamic Brake DCS Data Communication System DP Dynamic Programming DTO Driverless Train Operation EB Emergency Brake ECP Electronically Controlled Pneumatic brake system EOA End of Movement Authority ETCS European Train Control System FC Fuzzy Control FIS Fuzzy Inference System FNN Fuzzy Neural Network GA Genetic Algorithm

HS High-Speed HVAC Heating, Ventilation, and Air Conditioning HOP Handover Point IEEE Institute of Electrical and Electronic Engineers ILC Iterative Learning Control iTASC Intelligent Train Automatic Stop Control LCU Locomotive Control Unit LR Linear Regression LRHC Linear Receding Horizon Controller MA Movement Authority MAS Multi-Agent System MRAC Model Reference Adaptive Control ML Machine Learning MOPSO Multi Objective Particle Swarm Optimisation MVB Multifunctional Bus NB Normal Brake NGTC Next Generation Train Control System NN Neural Network OPAL Contactless fare collection system for public transport services in the greater Sydney area of New South Wales, Australia PSD Platform Screen Door RA Rail Automation RISSB Rail Industry Safety and Standards Board RL Reinforcement Learning SB Service Brake SP Station Position SRP Station Reference Point SVR Support Vector Regression SVM Support Vector Machine TASC Train Automatic Stop Control TCMS Train Control and Management System

xii

TCN Train Communication Network TCS Train Control System TILC Terminal Iterative Learning Control TL Train Location WA Western Australia WB Wagon Brake WTB Wired Train Bus

xiii

Statement of Originality

This work has not previously been submitted for a degree or diploma in any university. To the best of my knowledge and belief, the thesis contains no material previously published or written by another person except where due reference is made in the thesis itself.

Signature:

Date: 22/12/2020

xiv

Acknowledgements

It is my pleasure to thank those who support me during the period of my PhD journey. My first and foremost gratitude goes toward my principal supervisor, Professor Mehmet Orgun for his admirable supervision, encouragement and guidance. I also wish to extend my sincere appreciation to my associate supervisors, Professor Yan Wang for his invaluable support and advice throughout my PhD. It was truly my honour to work under their supervision and to be a part of their research team.

Furthermore, I would like to convey my sincerest thanks to my Calibre group colleagues especially Mr. Ronald Pearson and my MQ colleagues and friends, especially Dr Mehdi Shafiei and Dr Peter Anderson for reviewing this work and their technical feedbacks and also Dr Eugene Quah and Mr. Kourosh Langarrizadeh for their support and encouragement.

I gratefully acknowledge Macquarie University (MQ) for providing my iMQRES scholarship, which has given me this opportunity to develop my research and learning skills. Also, I would like to thank the Office of HDR Training and Partnership especially Mrs Pauline Woo as well as Department of Computing staff for their support during my PhD period.

Special thanks to my lovely family, my dad, my sister and my brother, for their constant encouragement and support in whole my life.

xvi

Chapter 1: Introduction

1.1 BACKGROUND AND MOTIVATION

With the development of new rail transport systems all over the world and improvement in the already-in-use rail systems in major cities to meet the new requirements for a modern public transport system, new train control systems have been introduced over the last three decades. Among a vast number of factors which affect a rail system performance, safety, punctuality of service, efficiency in energy consumption and comfort of ride for passengers are of crucial importance in order to encourage the public sector to use public transport on a daily basis rather than using their own private means of transport.

To improve safety and to avoid accidents, rail transport systems need a system to control the movements of trains on rail lines, known as ‘railway control and signalling system’. A rail signalling system gives the driver, or an on-board computer system (in driverless trains), the required instructions for a safe, efficient, punctual and comfortable trip.

The main difference between road vehicles and rail vehicles, which are generally known as rolling stock, in terms of their braking system is that, rolling stock has a much longer braking distance compared to road vehicles. The reason for this is due to differences between the adhesion coefficient between rail and wheels of rolling stock. Rolling a steel wheel over a steel rail differs from moving a vehicle’s rubber tyre on an asphalt surface of a road. The contact surface between steel rail

Chapter 1: Introduction 1

and steel wheels has much less friction than that of rubber tyres on gravel or bitumen road surfaces; thus, rail vehicles roll more freely. This lack of friction is ideal for moving a massive train. In Australia, as an example, the combined length of heavy haul carriages can exceed 2 kilometres with weights as heavy as 35,000 tones [94].

Consequently, this makes it more difficult for a rolling stock control system to stop the train precisely. The long braking distance is one of the main reasons for the development of rail control and signalling systems.

The Automatic Train Operation (ATO) system in state-of-the-art rolling stock acts as a substitute train driver. This module automatically controls the train movements and executes all the tasks which are already assigned to an on-board train driver. ATO main functions are as follows:

1. Releasing parking brake

2. Throttling;

3. Smoothing train throttling notches;

4. Cruising;

5. Coasting;

6. Selecting braking mode;

7. Braking;

8. Smoothing train braking notches;

9. Optimising train speed profile;

10. Stopping in stations; and,

11. Opening train doors at stations (for passenger trains).

To achieve the required safety level, an Automatic Train Protection (ATP) system will constantly supervise the ATO and assure that the ATO will not exceed the maximum allowable speed for each track section and activate the brake system if there is other rolling stock within its braking distance. The combination of ATO and

ATP is an advanced train control system known as Automatic Train Control (ATC) system.

Although in recent years several driverless trains must have been proposed to avoid the likelihood of human error and to achieve a punctual, efficient and comfortable train ride, the challenges and the potential solutions with the ATO systems are yet to be investigated. Most of the current ATO systems rely on a considerable number of control and signalling devices to control the train on a track and in a station.

As part of an ATO system, in a driverless train, a control system namely Train

Automatic Stop Control (TASC) is responsible for stopping the train in stations precisely while considering passenger comfort, Figure 1-1.

Rolling stock stopping time at stations, which is known as dwell time, is the time which is needed for opening and closing the train’s and PSD doors as well as the time for passengers to alight and board. In the situation that rolling stock does not stop at the precise position, it needs time for extra movement either forward or backward which will be added to the total stopping time at the station.

Precise stopping in a station could improve the rail system’s punctuality of service by reducing the dwell time in stations and also TASC’s smooth braking rate

Chapter 1: Introduction 3

will improve passenger comfort during the deceleration period prior to reaching the exact stopping spot in stations.

In this thesis, a new TASC algorithm as well as a sophisticated method of rolling stock dynamic modelling and braking behaviour for TASC (for the speed of less than

20 km/h) has been developed.

Figure 1-1: TASC Module beneath a Tokyu 7000 EMU, Japan (Source: Wikipedia)

One of the main benefits of the TASC algorithm, as mentioned previously in this chapter, is its precise stopping ability in stations. Recently most of busy train lines around the world have started using Platform Screen Doors (PSD) which need more precise train stopping in stations. PSD is synchronised with rolling stock doors and prior to its opening control command, the signalling system will check if rolling stock doors are precisely aligned with PSD with an acceptable position error. Hence, it is expected that TASC could improve the stopping time of rolling stock at stations, which consequently affects the punctuality of overall rail services.

TASC will also improve the level of passenger comfort as it uses one single smooth braking rate to stop the train at the predefined position in each station and is closely align with the PSD. Therefore, passengers experience neither hard deceleration nor different braking rates during entering and stopping at stations.

The TASC procedure as one major part of any train control system, as well as its relationship with rail control and signalling systems, has been depicted in Figure 1-2.

In this picture it can be seen that an Automatic Train Control (ATC) system, as one of main part of a rail control and signalling system, consists of two subsystems;

Automatic Train Protection (ATP) and Automatic Train Operation (ATO) systems. The former is responsible for the safety issues of train movements while the latter is the train driver substitution.

ATO includes TASC for station stopping and as it can be seen in this Figure 1-2,

ATO can control the traction and braking force of the rolling stock the through Brake

Control Unit (BCU), Brake Callipers (CLP), Traction Control Unit (TCU) as well as rolling stock throttling via Traction Control Unit (TCU) and Traction Motors (TM). TASC, on the other hand, can only control the brake system similar to ATO.

ATP, will supervise speed profile limits during ATO and TASC operation periods and will apply Emergency Brake (EB) if either of the systems are unable to perform within the ATP speed limits.

Chapter 1: Introduction 5

Figure 1-2: Signalling system, ATO and TASC Relationship

1.2 RESEARCH DESIGN, AIMS AND OBJECTIVES OF THE THESIS

This research develops an original solution for a TASC algorithm based on the train dynamic model, braking system behaviour and track profiles. Different TASC algorithms based on different Artificial Intelligence (AI) techniques; Reinforcement

Learning (RL), Q-Learning, Artificial Neural Network (ANN), and Fuzzy control are considered and compared to formulate a sustainable solution and to improve the stopping error of trains in stations with the minimum required devices on-board and in stations.

As mentioned before, this work develops a sophisticated train model as well as a braking system behaviour model which are used to validate the performance of

TASC. In order to achieve the main objectives of this thesis, which is improving the stopping error of trains in stations, while assuring passenger comfort during the stopping procedure, the following research studies were conducted:

➢ Reviewing and analysis current TASC algorithms, their performance and

shortcomings (Section 2.3). To do so, the background of the study (Section

2.2) as well as other main parts of TASC systems such as rolling stock

dynamic behaviour and braking models have been identified and reviewed

(Section 2.4 and 2.5).

➢ Developing a train dynamic behaviour model while considering track

profiles (Section 3.3).

➢ Developing a sophisticated rolling stock braking model (Section 3.5).

➢ Extracting TASC-related parameters from trains and a brake model (Section

3.6)

➢ Developing different TASC algorithms, using different AI techniques,

catering for stopping trains at stations (Section 0, 4.2.2, 4.2.3).

➢ Gathering real data for simulation across different rail networks; including

Sydney Train, Tehran Metro and Iran National Railway lines.

➢ Comparing the performance of different algorithms by applying them to the

rolling stock and braking models (Section 4.3).

➢ Developing a novel control algorithm for the TASC problem known as iTASC

based on previous best practices (Chapter 5).

Chapter 1: Introduction 7

➢ Identifying future research directions based on the current study (Section

6.3).

1.3 SIGNIFICANCE OF THIS RESEARCH

The fast-growing rail transport system and tendency for high-speed trains

(HST), driverless trains, CBTC systems and European Train Control System (ETCS) have made this area interesting for both industry and academia. To improve passenger safety, new rail systems have recently begun using PSD in stations - a technique which calls for precise stopping of trains at stations.

From a practical perspective, most stopping algorithms are not applicable for cutting-edge railway control and signalling systems which mostly rely on wireless communication as the main means of transferring data between rolling stock and stations and have minimum wayside and on-track equipment. In this regard, the TASC algorithm can play an important role in developing new rail systems using data transmitted from Access Points (AP) in stations. Therefore, new TASC procedures are needed to provide, update and refine in order to obtain a highly accurate train stopping algorithm while considering passengers’ riding comfort.

As a normal characteristic of a rolling stock braking system, which is the mechanical interface between a TASC algorithm and the wheels of rolling stock,

Figure 1-2, its performance differs over time and depends on quite a vast number of factors. Hence, advanced TASC algorithms are required to update the rolling stock dynamic model in each time step to increase the accuracy of its Braking Distance (BD).

1.4 KEY INNOVATIONS OF THE RESEARCH

The main contribution of this thesis is to develop a new TASC algorithm with acceptable performance on a sophisticated rolling stock model. Additionally, this work develops new approaches for modelling a rolling stock and braking system to increase the accuracy of TASC. In order to fulfil the main objectives of current research work, the following innovative research developments are accomplished, and described in this thesis as:

1. Passenger comfort in a stopping procedure at stations depends on braking

rate and speed adjustment of a control system. Therefore, TASC has

developed a smooth speed profile at each stopping procedure.

2. The investigations in this thesis show that the real performance of TASC

algorithms is highly dependent on parameters of rolling stock, brake

systems in different sections of rail tracks. Therefore, the novelty of

Chapter 3 of this thesis describes the development of a sophisticated

model for both systems.

3. In order to achieve more practical results, the research which is described

in this thesis uses braking delay time. This means that, if the control system

works perfectly, the braking force is ready after a specific amount of time.

So, the TASC learning algorithm has taken this fact into account.

4. The new TASC algorithm considers undistributed train mass, as the number

of passengers in different wagons could vary, based on the design of rail

lines and stations.

Chapter 1: Introduction 9

1.5 STRUCTURE OF THE THESIS

This thesis is presented in six chapters.

Chapter 1. Introduction:

This chapter presents an overview of the thesis with the study’s background, a description of the problem, motivations for the research, and the significance and contributions of the work.

Chapter 2. Literature Review:

This comprehensive review focuses on four main areas of research as follows and six major issues of TASC algorithm that have remained unsolved and need special consideration addressed:

1. TASC algorithms;

2. Train dynamic modelling;

3. Rolling stock braking system behaviour; and,

4. Machine Learning approaches in rail control and signalling systems.

Chapter 3. Train Dynamic Modelling:

This chapter discusses the development of a rolling stock dynamic model and train baking system behaviour for the TASC algorithm. In the first part of this chapter an appropriate rolling stock model for TASC algorithms is provided, which contains:

• Rolling stock active forces;

• Rolling stock models in different operational mode; throttling, cruising, coasting and braking;

• Rolling stock braking distance and braking time formulae for the TASC;

• Track profile impacts on rolling stock model and train active forces.

In the second part of Chapter 3, rolling stock braking system behaviours are studied and formulated based on active parameters in TASC performance. Finally, a braking scenario for TASC application has been introduced and formulated.

Chapter 4. TASC Benchmark:

This chapter represents different Artificial Intelligence (AI) approaches to provide a sophisticated TASC module. At the first part of Chapter 4, different learning algorithms are provided as follow and all algorithms have been fed by the same data set to enable comparison of their performance in similar situations:

1- Linear Regression;

2- Support Vector Regression;

3- Artificial Neural Network;

4- Fuzzy Control

The main focuses of each learning algorithm are reducing the stopping error of a train in a station while improving the passengers’ comfort. Finally, in the second part of the chapter, the results of different algorithms have been compared.

Chapter 5. iTASC:

In this chapter, based on study which has been conducted and reported on in the previous chapter, a new TASC algorithm has been developed. As we have compared different AI techniques and combined them to develop this algorithm, the

Intelligent Train Automatic Stop Control (iTASC), is the name that has been chosen for this research, this chapter and for the new algorithm itself.

Chapter 1: Introduction 11

The iTASC algorithm is a combination of machine learning techniques and fuzzy control systems. Reinforcement Learning, Q-learning, Double Q-learning and fuzzy control systems have been used in its development and its performance has been shown by using models of rolling stock and braking systems which are provided in

Chapter 3 and has been compared by TASC benchmark results as presented in

Chapter 4.

Chapter 6. Conclusions and Future Works:

Finally, in this chapter, the contributions of the thesis are summarised, the conclusions from this research are drawn, and possible future work directions are highlighted.

Chapter 2: Literature Review

2.1 INTRODUCTION

The term ‘Train Automatic Stop Control’ (TASC) was first coined by Oshima,

Yasunobu and Sekino in 1983 [2], where the authors introduced a predictive fuzzy control method for the problem of train automatic stop control to address rail system safety, comfort of riding and precise stopping in railway stations as shown in Figure

2-1. The authors placed a transponder or, as they call it, a point signal, at the entrance of each station and by passing it, the train control system realises that the station stop position is at a predefined distance ahead and activates the train’s braking system accordingly. Fuzzy rules based on the initial speed of train while entering the statin, as we later shall call it, Handover Point (HOP), allocate the appropriate braking rate to the train.

Increasing a train’s speed to achieve more capacity increases the HOP speed and makes it more challenging for the TASC to control the train and stop it at the

Figure 2-1: Train Automatic Stop Control [2].

Chapter 2: Literature Review 15

precise location. Moreover, migration from conventional railway control and signalling system to the new railway signalling system introduces new challenges for precise stopping of trains. It involves the use of recently emerging technology known as Next Generation of Train Control (NGTC) system, where wireless communication is the main means of transferring data between onboard systems and wayside equipment as well as automatic and unattended train control systems; commonly known as a driverless train.

The main problems relating to the precise stopping of rolling stock at stations’ platforms are:

1- Rolling stock dynamic behaviour is not a linear function. It is based on

different factors such as rolling stock dynamic behaviour and its braking

systems, as well as track profiles for each station and different other

environmental factors such as wind, tunnel and wet rail effects.

Consequently, the new controlling methods should be based on an

algorithm for non-predictive environments.

2- Even if they have been modelled perfectly, rolling stock braking systems’

characteristics and responses change over time during their life cycle which

means they could not be expected to perform at the same braking rate,

every time.

3- Rolling stock position uncertainty in the Communication-Based Train Control

(CBTC) system makes the problem of precise stopping even more complex

issue as the stopping distance needs to be updated continuously depending

on a train’s current position.

As the number of research work in TASC is limited to a few studies which we have thoroughly reviewed them in section 2.3, we find it useful to add some backgrounds to our study which mainly comes from Automatic Train Operation

(ATO).

In our solution we have not consider journey time as in stopping a train in a station, where TASC is all about, precise stopping is not matter of computation time and the reason for it is, the speed in station while TASC take the control of train is less than 15-20 km/h and stopping course in a few seconds, so timing is negligible and can be safely ignored.

In this chapter, we also added two more sections to provide a review of two main parts in TASC algorithm which are Train and Brake model. So, the provided literature review in this chapter is addressing:

1- Study’s Background: In Section 2.2, a brief review of the Automatic Train

Operation (ATO) system and Rail Automation (RA) are provided.

2- Train Automatic Stop Controller (TASC): In Section 2.3, a complete review

of train stopping algorithms from their beginning until now is provided.

Recent research in both machine learning and control engineering

disciplines are provided in this section. Moreover, this section ends with a

study about utilisation of Reinforcement Learning (RL) algorithm for the

problem of precise stopping of a train in a station to show the effectiveness

of RL for rail system traffic control and especially for braking system control.

3- Rolling Stock Dynamic Behaviour: Trains, locomotives and wagons, as they

are generally known as rolling stock, play crucial roles in TASC algorithms.

Chapter 2: Literature Review 17

Hence, in Section 2.4, train modelling based on the results of several studies

has been reviewed.

4- Rolling Stock Braking Model: As the main control signal of TASC addresses

the train’s brake rates while approaching a station, this study has tried to

provide a comprehensive review of the rolling stock braking systems in

Section 2.5. To clarify the basic concepts and procedure of rolling stock

braking, several methods are reviewed and studied in this section.

2.2 THE BACKGROUND OF THE STUDY

Rail Automation and Automatic Train Operation (ATO) in which an automatic control system drives the train while having no driver or driver assistant on-board has gained momentum in urban rail systems across the globe and even in freight and heavy haul rail corridors as well. Rio Tinto introduced the first ever fully automated freight train in Western Australia (WA) and it had completed one million kilometres of autonomous operation by the end of 2018 [40].

IEEE 1698 [41] has raised the difficulty and complexity of developing a completely new generation of an efficient ATO system which addresses current issues of rail transport system as follows:

1. Safe movement;

2. Traffic management;

3. Ride comfort; and

4. Energy efficiency.

In a survey research work on Automatic Train Operation (ATO), Yin et al [21], in addition to other aspects of ATO systems which have already been mentioned above, have emphasised station stopping accuracy as one of the ATO main features. Huang and Her [42] have introduced a fuzzy control system for an ATO for entire movement of a rolling stock between two stations while considering the ride comfort and also station stopping requirements. In this work, a simple mathematic model of rolling stock propulsion systems had been used and the only considered resistance for train movement is David experimental formula. While the concept of fuzzy rules in this work is novel at the time, however the brake characteristics and station profile are completely ignored. Wang et al [43], in a survey work, have used the Driverless Train

Operation (DTO) terminology referring to the autonomous operation of a train instead of ATO and addressed the consistency of station stopping time, which is more or less similar to the problem of TASC for an ATO system.

Wang et al [44] used a machine learning approach to provide an intelligent algorithm for automatic operation of trains in heavy haul corridors. Gao et al [45] have introduced a new model of train for automatic operation of High-Speed (HS) trains. They have used second-order differential equation to capture rolling stock movement characteristics and state constraints which could not thoroughly include all the train behaviour. Train states including the current position of the train and real time speed but there is not any sign of brake modelling in this work.

SH Han et al [52] have developed a Genetic Algorithm (GA) and introduced an optimal train operating method from energy consuming perspective. However, similar to many other studies, the movement of train is simply considered just by

Chapter 2: Literature Review 19

Davids formula. ATO methods by Liang et al [46], Amrani et al [70] and Kang [71] have used GA and mainly have focussed on a train’s running curve by incorporating a penalty factor to the algorithm and Dominguez et al [47] have introduced eco-driving for an ATO system by using a Multi Objective Particle Swarm Optimization (MOPSO) which all these work suffering from exact modelling of a train dynamic behaviour at one hand and the brake systems on the other.

Sekine et al [48] used a Fuzzy Neural Network (FNN) for the ATO application, the algorithm is of two degrees of freedom while Cao, Ma and Zhang [49], Oshima,

Yasunobu and Sekino [53], Chang and Xu [54], Chang, Xu and Quek [55], Jing, Zixing and Limin [56], Sekine et al [57], Bing, Hairong and Yanxin [58] and Dong et al [63] have used a Fuzzy Control (FC) system for ATO which used a predictive algorithm. Su et al [50] have used a numerical algorithm for ATO system which optimises both the train’s timetable as well as the train speed profile. Lu et al [51] have provided a distance-based method for train speed control which either can assist a train driver or an ATO system. Although these research works have touched the train stopping as part of the train movement between two stations, the TASC algorithm is not among their priority lists.

The results of research work of Wang, Hou and Li [59] have developed an

Iterative Learning Control (ILC) solution for the ATO system. Gao et al [60] have introduced a robust adaptive control approach for the ATO system. Ganesan,

Ezhilarasi and Benni [64] have introduced a non-linear model of high-speed (HS) trains based on a longitudinal dynamic by considering track profile and train

formation. However, there is not any indicate of stopping precision measurement and braking characteristics of their model are too simple.

Yin et al [61] provided an intelligent algorithm for an ATO system using

Reinforcement Learning (RL) approaches by focusing of dynamic modeling of the train and similar to many other research studies, braking factor in the ATO as well as

TASC algorithm is underestimated.

The study work by Lin et al [65] has developed an energy optimized algorithm for an ATO. Similar to this work, Yazhemsky, Rashid and Sirouspour [66] have proposed an optimal controller for a train trip between two stations in terms of energy efficiency, time efficiency or a mixed control. For solving the issue of optimization of a train’s speed profile, Matsuura and Miyatake [67] have applied

Dynamic Programming (DP) while Albrecht, Binder and Gassel [68] introduced the concept of target point as well as window point for speed optimisation for an ATO system. Baranov et al [69] have used three different algorithms for Automatic Train

Control (ATC) system. The focus of mentioned works is optimization of train movements which potentially could contain the train stopping problems in stations and can be considered as the background of the TASC problem however critical items in TASC procedure are missing in them.

Research works by C. C. William [28], Calderaro et al [72], Calderaro et al [73],

Gupa, Mahajan and Garg [74], Liang et al [75], Gao et al [76], Cheng et al [77], Mo,

Yang, and Gao [78], Wang et al [79], Jia, Xu and Wang [80], Su, Tang and Roberts [81],

Bai et al [82], Yang et al [83] and Xi et al [84], J. Yin, D. Chen and L. Li [86] are trying to apply relatively new control algorithms to the problem of ATO and achieving an

Chapter 2: Literature Review 21

optimised energy consumption by providing an efficient speed profile for trains between two stations.

ATO as well as Intelligent Train Operation algorithms which is the subject of some research works such as in [28] and [86], are a method for entire train journey on a track, specifically from station to station considering timetabling and controlling the headway of the line to achieve the best performance in terms of applying traction and brake efforts However, TASC, just focuses on the last portion of the journey and only control the train brake system which eventually can be merged into those algorithms.

However, the models used in these works are equivalent electrical circuit of an electrical railway system by just considering train traction motors as the main part of the model which could not introduce the entire characteristic of a rolling stock dynamic modeling.

Calderaro et al and Calderaro et al have applied Dynamic Programming (DP), while research work in [74] to [76] and [83] have used Genetic Algorithm (GA). Cheng et al have provided an algorithm known as Classification and Regression Tree (CART).

Wang et al generally used Machine Learning (ML) technique and regression algorithm has been used in Xi et al study work. Finally, Cooperative Control (CC) has been used in both works by Su, Tang and Roberts [81] as well as Bai et al [82].

The research focus in Khadilkar’s work [88] is on providing timetable and scheduling information in the rail system and generates a methodology for handling large scale problems. Li et al [89] and Jianwei et al [92] have addressed the problem of energy saving in railway lines by using Monte Carlo approach. Yasunobu and

Matsubara [90] are using a fuzzy target for the problem of parking a car while the fuzzy target is achieved from a RL approach. Lin and Sheu [91] offered a solution for metro traffic using RL. The research work by Jiang et al [93] has used the RL for coordinating passengers in train stations during peak hours. Similar problem can be seen in these works as well; simple dynamic model of train and ignoring brake type of the rolling stock as well as braking time delay which heavily affects the stopping distance in the stations. To show the effectiveness of proposed algorithm, in Chapter

5, Section 5.6, the time delay for the selected rolling stock in the simulation has been set up to the maximum based on used braking system which is 2 seconds.

2.3 TRAIN AUTOMATIC STOP CONTROL (TASC)

Oshima, Yasunobu and Sekino [2], introduced a predictive fuzzy control method for the problem of train automatic stop control to address:

• rail system safety;

• comfort of riding; and

• precise stopping.

While their work was a steppingstone toward migrate from linear control in

TASC problem by introducing fuzzy control and endeavouring to imitate an experienced driver behaviour, this work was heavily involved in introducing the new idea of using fuzzy control in TASC problem.

The train model is too simple and the only brake characteristic that had been considered is train braking delay which is 0.2 second while the actual braking delay time even in current modern braking systems could reach to 4 second.

Chapter 2: Literature Review 23

Another issue with this research work is in the TASC control loop, they have provided control over both brake and traction systems which means TASC controller can activate both brake and traction systems. The major problem with this, is TASC controller takes control of a train movement in the last section of speed profile where before that train had experienced a coasting section. If it could active the traction while it is trying to stop the train it means that passenger will experience different jerk levels during stopping procedure. Several braking efforts not only affect the passenger comfort level, but from brake maintenance perspective could be costly.

To overcome the skidding and slipping problems related to axle generators and transponders in train correct positioning, Yoshimoto, Kataoka and Komaya [3] introduced a TASC which used range sensors. This method makes it possible for the train to get its position continuously while avoiding skidding and wheel diameter errors. That was a powerful method of controlling a train to achieve an automatic stop at stations. In this paper ATO has been introduced and the main functions of an

ATO system have been clarified. By using predictive fuzzy control this study addressed the following problems of TASC:

1. Passenger ride comfort;

2. Precise stopping; and

3. Running time.

The novelty of their work is clear for the time, however, it could be understood that this method need a plenty of transmitters installed on the track which is not applicable in new generation of the railway control and signalling systems i.e. CBTC

and ETCS which are trying to rely on wireless communication and limiting the field equipment.

In a research work by Siahvashi and Moaveni [5], the problem of precise stopping has been seen as part of the entire train control system, Automatic Train

Control (ATC), and is addressed by combining Cooperative Control (CC) systems and

Multi-Agent Systems (MAS). Although the concept of cooperative control in railway control and signalling system was introduced for the first time in this work however, this work is not mature in terms of modelling a train dynamic behaviour and braking characteristic from coasting phase of train movement up to a full stop in a station.

Hou et al [6] introduced a learning method known as Terminal Iterative

Learning Control (TILC), which uses the stopping errors in train position at stations to update the current braking rate and reach a better performance each time. To some extend this study is similar to the work by Chen et al [7], in which authors have used machine learning methods and data from Balises in stations to provide an online learning approach for the problem of stopping a train in a station, precisely. Train braking system adjust its braking rate after passing each Balise, at stations but simple train modelling by just relaying on Davis formula for train movement in [6] and the

Newton second law in [7] as well as ignoring the real braking delay which effectively play a role in stopping procedure in both works are their main shortcomings.

Ma and Zeng [8], have proposed three different data mining algorithms for the problem of stopping a train in a station. They have captured 15 affecting factors in a braking procedure and used these factors to analyse precise stopping at a station.

The main missing point in this research study is authors have not mentioned train and

Chapter 2: Literature Review 25

brake modelling as well as track profile in those 15 identified factors which potentially effective factors after considering these missing parts can be expanded twofold from a data mining point of view.

The approach in a study by Peng and Qingyua [9] to achieve a precise train stopping, is using predictive control. The algorithm considers both system delay time and real-time train parameters to obtain an optimum control signal while ignoring a plenty of crucial parameters for modelling the train and its braking system. The system delay time in this study is 0.4 which if we consider it as control system delay plus braking delay time it too small to satisfy both; the actual braking delay for just braking system in an actual scenario could reach up to 4 seconds. Moreover, the stopping position error still needs improvement to satisfy the actual industry need for the modern railway control and signalling system which using the synchronised platform screen doors (PSD) in the stations and the position error between train doors and PSD should not be more than 20 cm.

Bai et al [10] have developed a fuzzy neural network for the TASC algorithm.

The authors limited the scope of research work by mainly focused on stopping a freight train and considering changes in train pneumatic braking systems over time to achieve a precise stopping in one braking effort while the train model and especially locomotive dynamic characteristics largely have been ignored. They simply assumed the train as a mass point while even in that model have not consider the weight of the train.

Aum [11] has used a closed-loop control method for a TASC algorithm known as Linear Receding Horizon Controller (LRHC) which controls the braking rate of the

Korean metro which is using air brake system. However, the train model which has been used in this work just considers a few parameters of train; train mass, position and velocity. Additionally, similar to many other research works, Davis formula is introduced for the train dynamic behaviour.

Chen, Wang and Li [12], have utilised Support Vector Machine (SVM) for the

TASC algorithm of a High-Speed Train (HST). They have compared their study result to the Average Speed Method (ASM) and showed that their result exhibited 38.8% improvement in the positioning error. However, train and brake modelling as well as track profile characteristics especially for high speed line are vastly underestimated in their simulation.

Ha, Kim and Kang [13] have developed a precise stopping method for Korean metro system while considering the Platform Screen Door (PSD) in stations. They used a transmitter which is a laser light to provide the system with stop error at stations and an onboard receiver which wirelessly communicates with the platform modules. This work had addressed the problem of train stopping procedure in a station thoroughly however still has some shortcomings as follows:

1- This method needs to add a module in the platform where the rolling stock

doors might locate which performs as a receiver and could limit the

platform space and heavily prone to experience vandalism;

2- Train doors need extra modules to communicate with the receiver in the

platform;

Chapter 2: Literature Review 27

3- Reflection of light which is the main media of communication between

these two modules can easily be suffer from disturbance in rainy weather

in open stations;

4- Trian model as well as braking system is ignored in the algorithm; and

5- The solution is totally detached from the signalling system and train control

management system (TCMS) which means implementing of this method

will impose a huge interfacing management workload.

Yasunobu, Miyamoto and Ihara [14] mainly dealt with the problem of train automatic control focusing on some common items such as safety, passenger comfort, train velocity and station stopping errors. The authors addressed precise stopping at stations by using a fuzzy controller to develop a solution to this issue as part of automatic train operation (ATO) system while using a simple model for train and an immature braking system model.

The problem of stopping at stations, has been address by Aguiar et al [15] by using an advanced control strategy for train braking systems. In this study, authors have considered model errors, failure and unknown disturbance by focusing just on train control model without modelling of the brake system and its real behaviour during the actual stopping scenario. Their train model is similar to what had been used in [10]; considering the entire train and the resistance forces as a mass point.

The focus of a study by Oda, Niimi and Maki [16] was to develop an algorithm for automatic tuning of TASC devices parameters. The study did not use rolling stock dynamic characteristics and tested by utilising proportional control to achieve tuned parameters for TASC which in their best simulation result, they have achieved a

stopping accuracy of 35 cm which is not sufficient for the modern railway control and signalling system where the train doors need to be aligned with PSD.

Temple et al [17] evaluated a TASC algorithm and developed a software-only countermeasure to assure precise train stopping for a simply modelled train.

Results of Aguiar et al research [18] improved the accuracy of train positioning.

The authors considered just the wheel jamming faults, formulated the problem and then introduced a linear parameter-varying approach which had been applied to a simple model of a train which just considers train velocity, position and braking force.

The study by WU et al [19] evaluated a countermeasure to assure TASC achieves valid information from Balises or transponder on the track at stations. The research heavily relies on data from Balises which could have some error and in recently introduced railway signalling, train position is continuously being calibrated by a second source of positioning like wireless communication. They rely on the

Newton second law for the train dynamic behaviour and assumed the train as a mass point. Additionally, in their model, they put the total mass of the train when it passes a specific Balise which practically should be an axle weight.

Anuszczyk et al [39] developed a theory and mathematical modelling for braking curves and modelling of passenger trains’ braking systems and their relationship with automatic train stopping at underground metro line stations. This work suffering from lack of real data simulation and validation as well as underestimate the braking procedure by ignoring the braking type, braking weight and braking delay time in practical scenarios.

Chapter 2: Literature Review 29

2.4 ROLLING STOCK DYNAMIC BEHAVIOUR

Wang, Tang and He [20], have introduced an optimal control approach for heavy haul trains consisting of locomotives and wagons by using Approximate

Dynamic Programming (ADP). The train in their model used a pneumatically controlled braking system. They simply used the Newton’s laws for train dynamic behaviour and ignored braking models.

In a study by Yi [24], a mathematical model of diesel and electrical trains as well as their braking system modelling and train active forces have been provided which inspire us for our modelling in this work. Guastafierro et al [29] used Davis equation for modelling train movement and provide an optimised coasting curve which is ignoring braking behaviour.

Iwnicki [35], provides a longitudinal model for the Australian rail industry while considering a function of different factors including locomotive control input, brake input, track profile and the characteristics of rolling stock and bogie. A similar model can be found in a research study by Spiryagin et al [32] which we have used them in our modelling chapter.

2.5 ROLLING STOCK BRAKING MODEL

Barney, Haley and Nikandros [25] have provided a method of train braking distance calculation, considering the mass of train in the calculation and adding braking delay time. Their model is valid for the conventional signalling system where it is using different three aspects signals prior to each station but cannot be applied to the communication-based train control systems.

In Ahmad’s study [26] the Model Reference Adaptive Control (MRAC) has used to address the train dynamic brake control system to achieve an accurate estimation of a train’s braking distances which some of his ideas have been used in the modelling chapter of this work. Ning [27] dealt with a braking system model for the ETCS while considering track speed restrictions, braking delay time, track profile and deceleration ability of rolling stock. This model still needs improvement by adding some critical aspects of dynamic behaviour of the rolling stock itself.

In the study by SAGA et al on Shinkansen’s braking system in Japan [33], the authors reviewed the impact of rainy weather and wet rails on train braking distance which we have borrowed that idea from them in the modelling chapter. While

Hijikata and Nishimura [34] simulated train braking force based on the estimation of each car’s force, for a freight train, this approach also took into consideration the coupling devices between each wagon.

Wei et al [36] have introduced an integrated model of air brakes for heavy-haul trains considering a braking component’s model such as distribution valves. A similar approach can be found in the research work introduced by Wu et al [37] and Wei,

Ahmadian and Zhang [38] which we have applied them in our model.

2.6 SUMMARY

This chapter has reviewed studies regarding automatic train operation (ATO).

As it is interconnected to ATO, Train Automatic Stop Control (TASC) has been addressed in several articles regarding automation of train movement between stations and smooth stopping at a predefined position at stations. In ATO systems,

Chapter 2: Literature Review 31

the focus of most articles has been placed on an online, intelligent and efficient algorithms for train movement between stations while ignoring the speed adjustment when a train and its ATO system switch between different modes of operation; starting, accelerating, cruising, coasting, braking and finally stopping at the next station. These modes are represented in Figure 2-2.

In most studies, the train speed profile has been divided into four sections as can be seen in Figure 2-2. A train starting from a stationary status and will experience the following modes;

1- Accelerating: train will start throttling from a stationary mode until

meets the service speed;

2- Cruise controlling: by reaching to the service speed, this mode takes the

control of the train to establish a steady speed profile;

3- Coasting: while approaching the next station, by reason of energy

saving, the train control system will start a mode in which there is

neither traction force nor braking.

4- Braking: when a train reaches the vicinity of the station, the braking

mode will be activated. The first active brake system is dynamic brake

(DB) and after reaching to a predefined speed (normally between 15-20

km/h based on the performance of DB), air brakes will be activated.

In Figure 2-2, the changing train operation modes from each section to the next has been shown by a smooth line while the reality is, there is a speed adjustment between different modes of operation as shown in Figure 2-3. This speed adjustment

affects passengers’ comfort of ride and; therefore, the train’s control system and specifically TASC, need to take this fact into account when being designed.

Figure 2-2: ATO Speed Profile

Figure 2-3: Speed Adjustment [62]

The TASC algorithms currently introduced in research studies have the following shortcomings:

Chapter 2: Literature Review 33

1. In the braking mode, the current TASC algorithms have ignored the speed

adjustment which normally occurs when the train control system tries to

adjust the train braking rate for smooth stopping at stations.

2. Another missing link so far, is the ignorance of blending braking modes of

rolling stock prior to a station. A train braking system in a stopping procedure

uses a combination of dynamic brakes and air brakes, or, an Electronically

Controlled Pneumatic (ECP) brake system. Trains reduce their speed using

dynamic brakes prior to the station and, as long as they reach the speed of 20

km/h, the ATO system will deactivate the dynamic brake and, based on the

rolling stock design, automatic air brakes or ECP will take control of the train

until it reaches the stopping spot in the station. Consequently, blending

braking modes can be deployed in a proposed TASC algorithm.

3. For a braking model and calculation of braking distance (BD) in various

articles, Davis formula with fixed coefficients have been applied. This

equation ignores many other important parameters of track profile, rolling

stock and the braking system itself all of which play important roles in BD and

precise stopping of trains at stations. Identifying effective parameters for the

braking mode, especially for speeds of less than 20 km/h at which point TASC

takes the control of the train, could improve performance of the TASC

algorithm.

4. Another problem with braking models of many research studies is that the

braking force has been assumed to be available as long as the brake signal is

released from ATO or train control system. However, the reality is different

from this. There is always a braking delay time which differs from train to train

and there is a need to be aware of that while developing a rail system

involving rail signalling systems, platform screen doors, rolling stock as well as

the TASC algorithm.

5. By developing a Communication-Based Train Control (CBTC) system or

European Train Control System (ETCS), in most rail systems all over the word

in which their control and signalling systems approaches are based on a

bidirectional wireless communication between rolling stock and wayside

equipment, the train’s position uncertainty needs to be considered in TASC

algorithms. There are different standards such as IEEE 1474.1 [1], which

allows the suppliers of train control and signalling systems to have a position

uncertainty margin through their design and every TASC algorithm must be

aware of this limitation of wireless data communication.

6. The solution is totally detached from the railway control and signalling system

and train control and management system (TCMS) which means

implementing of this method will impose a huge interfacing management

workload.

7. Last but not least, for a passenger train, there might be variation in the mass

of train due to passing through different stations and boarding and alighting

different number of passengers at each station. The mass of train also could

be distributed unevenly across the train because of some busy station

entrances, while most article have assumed an evenly distributed mass of

train over different bogies and axles. What needs to be considered in a TASC

Chapter 2: Literature Review 35

algorithm, is the number of passengers in each car based on the weight of the

train. Similar consideration must be taken into account for freight trains as it

is quite possible to have empty wagons in a consist because of limitation of

loading and unloading in some rail yards as well as a train’s configuration and

shunting limitations.

Table 2-1 shows a summary of reviewed works based on identified critical items for TASC algorithm. The last raw indicates the scope of this research work where we placed out algorithm name “iTASC” to facilitate comparing it with previous work as well as our work shortcomings which we will highlight them in the last chapter of this book where we will introduce a plenty of research areas for future work.

Table 2-1: Summary of TASC Algorithms

Reference PC T/BM BD PU TP UM C/ES PT PA W/WE ES

[2]           

[3]           

[5]           

[6,7]           

[8]           

[9]           

[10,11,12]           

[13]           

[14,15,16,39]           

[17,18,19]           

[47,52,65]           

[66]           

iTASC           

Table Guide: PC: Passenger Comfort UM: Uneven Mass T/BM: Train/Brake Modelling C/ES: CBTC/ETCS Suitability BD: Brake Delay PT: Processing Time PU: Position Uncertainty PA: PSD Alignment TP: Track Profile W/WE: Wind/Wet rail Effects

Chapter 2: Literature Review 37

Chapter 3: Train Dynamic Modelling

3.1 INTRODUCTION

For a train precise stopping procedure of a train, it is essential to know the train’s braking characteristic and dynamic behaviour of the rolling stock which is a function of resultant force at any given time. Running a train needs a massive energy to overcome train resistance forces and consequently stopping moving trains in stations as well as holding them on a grade if the station track is not tangent, will require a great deal of energy. Normally, based on the design, trains usually use four main braking systems to accomplish this: friction brake, dynamic brake, electromagnetic brake and parking brake.

In this chapter, trains’ dynamic modelling and air braking systems modelling are introduced. Based on this study, sophisticated models for both systems have been developed. The research output of this chapter will be used in the following chapters to assess and compare the performance of proposed TASC algorithms.

Although this chapter will provide a sophisticated train modelling guideline, for the purpose of TASC simulation, we just focused on the last segment of train braking behaviour as stopping a train in stations using TASC algorithm means the brake procedure normally starts far from station, depend on the speed, train length, brake model etc. and the last portion of the task where train speed is 15-20 km/h is assigned to TASC.

Chapter 3: Train Dynamic Modelling 39

3.2 TRAIN MODELLING

Based on operational modes, a train experiences different forces during its journey or trip. Figure 3-1 shows a stationary train on a tangent track and, as can be seen in this illustration, the forces of a train’s weight relate directly to its mass and the force of the parking brake to ensure that a train remains stationary until its control system provides running force. The train’s weight will change if a track is non- tangent. This situation will be discussed in Section 3-3.

Figure 3-1: Stationary Train Forces

To move a train forward or backward, the parking brake needs to be release and a running force needs to overcome train weight and its wheels’ reaction with rails. This force is known as ‘starting resistance’.

As soon as a train starts moving it experiences different forces based on its speed and the track’s profile. The general rule of train movement is that at each moment the running force needs to overcome the resistance force to move a train forward.

3.3 TRAIN ACTIVE FORCE MODELLING

In most train models, a normal train trip from one station to other has three working modes: Traction mode, Coasting mode and Braking mode.

Traction deals with train running resistance and overcomes it to provide the train with tractive effort to move forward, thus acceleration in this mode is positive.

Train traction motors at this mode are powered to produce running force. Train resultant force 퐹푇 is

퐹푇 = 퐹푡 − 푅푇 (3-1)

Where, 푅푇 is total resistance force and 퐹푡 is tractive force.

At coasting operation, the force acting on the train is only a running resistance force 푅푇, so the resultant force 퐹푇 is

퐹푇 = − 푅푇 (3-2)

During braking mode, the active force on the train includes the total running resistance force 푅푇 and a braking force 퐹퐵, so 퐹푇 is

퐹푇 = −(푅푇 + 퐹퐵) (3-3)

To calculate braking distance and the time required to come to a stop, the model introduced by S. Yi [24] has been used as follows:

푉 5푑푉 5 (푉 −푉 ) ∆푡 = ∫ 2 = 2 1 (푚푖푛) (3-4) 푉1 퐹푡 퐹푡

푡 = ∑ ∆푡 (푚푖푛) (3-5)

Chapter 3: Train Dynamic Modelling 41

푉 83.33푉푑푉 41.7 (푉2−푉2) ∆푆 = ∫ 2 = 2 1 (푚) (3-6) 푉1 퐹푡 퐹푡

푆 = ∑ ∆푆 (푚) (3-7)

In (3-7), 푡 is braking time and 푆 is the braking distance and, for more accuracy,

ΔV=2 to 5 km/h has been selected. Based on the above formula for calculating S, the amount of 퐹푡 is needed and to reach to 퐹푡 according to (3-3) the braking force and resistance force are necessary. Section 3.4 will deal with this issue.

Ft = F1-F2

Ft = F1 - (F2 + mgsinΦ) F2 F1 Ft = F1 + mgsinΦ - F2 F1

F2 F2

mg F1 Φ

mg mg

Figure 3-2: Train Forces in Uphill, Downhill and Tangent Track Profiles

Based on Newton’s second law, the dynamic model of a train can be calculated by having its mass and active forces which apply to the train.

퐹 = 푚푎 (3-8)

Where

퐹 is the total force including tractive effort, resistant effort and braking effort can be seen in Figure 3-2. Table 3-1 summarises train forces where;

푚 is train’s mass; and

푎 is train’s acceleration

Considering internal forces between train wagons and cars, Yin et al [21]

introduces a dynamic model which has been modified to be used for TASC purposes

as follows:

푅 푚푖푣̇푖 =푢푖 + 푓푖−1-푓푖-퐹푖 i=1, 2, …, n (3-9)

Where

푚 is the mass of a train;

푢 is the traction or braking force;

푓 is the internal force; and

퐹 is the running resistance.

Table 3-1: Train Forces

Train mode Uphill slope forces Tangent track Downhill slope forces forces Stationary 퐹푡 = 퐹1 − (퐹2 + 푚푔푠푖푛휙) 퐹푡 = 퐹1 − 퐹2 퐹푡 = 퐹1 + 푚푔푠푖푛휙 − 퐹2 train Moving 퐹푡 > 퐹1 − (퐹2 + 푚푔푠푖푛휙) 퐹푡 > 퐹1 − 퐹2 퐹푡 > 퐹1 + 푚푔푠푖푛휙 − 퐹2 train

3.4 TRAIN BRAKE PRINCIPLES

Based on their design as well as their intended use, trains have different types

of braking systems. The following section gives a brief introduction to the most

popular braking systems and paves the path for modelling rolling stock braking

systems for the TASC algorithms in the following chapters.

Chapter 3: Train Dynamic Modelling 43

3.4.1 Friction or Air Brake Friction brakes or Air brakes are one of a train’s main braking systems. The basic braking mechanisms in this system have been introduced by Hasegawa and Uchida

[103] as follows:

1- Wheel-tread brakes;

2- Axle-mounted disc brakes; and

3- Wheel-mounted disc brakes.

Figures 3-3, 3-4 and 3-5 show these braking systems accordingly. All these mechanisms use an object such as a brake shoe or lining that applies friction to the disc. The applied pressure is adjusted to control the braking force.

In the wheel-tread brake, the brake shoe applies friction to the wheel tread, creating a sliding effect. High speed trains cannot use this type of brake, since doing so will damage the wheel tread. Instead, they normally use axle—or wheel mounted disc brakes.

Axle-mounted disc brakes are used on trailer bogies, because they have sufficient space to accommodate such a system. Wheel-mounted disc brakes are used on motor bogies that must accommodate the traction motor and have insufficient space for an axle-mounted brake.

Figure 3-3: Wheel -Tread Brake System [103]

Figure 3-4: Axle-Mounted Disc Brake System [103]

Figure 3-5: Wheel-Mounted Disc Brake System [103]

Chapter 3: Train Dynamic Modelling 45

3.4.2 Air Brake’s Main Components Train air brake systems have different components and subsystems based on their design, but the main components of brake systems are generally the same and could be listed as follow:

1. Train Clean Room

2. Compressor (used for a brake system as well as other pneumatic

equipment such as horn, windscreen wiper and sanding system)

3. Main Reservoir

4. Driver’s Brake Valve

5. Brake Pipe

6. Brake Cylinder

7. Distributer (or Triple Valves)

8. Auxiliary Reservoir

9. Brake Pads

Note 1: As a main part of locomotive air system, a Clean Room is a module which is a combination of filters and electrical motors to take outside air and filter it and provide required air for compressor and air-cooled components such as generator and traction motors.

Note 2: A compressor is the pump which draws air from the atmosphere through a locomotive’s clean room and compresses it to the main reservoir. By using a Driver’s Brake Valve, the mode of the braking system will be selected.

Note 3: Based on these components, there are different parameters that directly and indirectly affect the braking distance of trains. This will be reviewed in the proposed TASC braking model.

3.4.3 Independent Air Brake The independent brake system is the brakes on the locomotive itself. A good study in this field has been done by Wu et al [37]. The independent brake can be applied and released separately from the train’s brakes. The driver’s handle has different notches and makes it possible to vary brake application from light to full.

The independent or locomotive brake system is shown in Figure 3-6. In this system, the air compressor detects the pressure of the main reservoir and automatically starts or stops the feed of pressurised air to the main reservoir as required to maintain pressure. The regulating valve controls the openings of Chokes

휙5 and 휙3 according to the position of the driver’s handle and the pressure in the equalising reservoir. The equalising reservoir has a small volume (12-20 L based on the final design) and acts as a pilot for the pressure change of the brake pipe. The relay valve detects the pressure difference between the equalising reservoir and the brake pipe so as to change the openings of Chokes 휙2 and 휙4. When the equalising reservoir pressure is higher than the pipe pressure, Choke 휙2 is opened and the main reservoir charges the brake pipe until an equilibrium is reached between the equalising reservoir and the brake pipe.

Chapter 3: Train Dynamic Modelling 47

Figure 3-6: Locomotive Brake System [37]

A full-service application of the locomotive’s automatic brake is able to stop the locomotive on a level tangent track, with dry rails, within the following maximum distances, based on section 13 of the Rail Industry Safety and Standards Board’s

(RISSB) standards for locomotives which is shown in Table 3-2 [106], but the exact required braking distance usually will be defined during the detailed design phase:

Table 3-2: RISSB’s Minimum Braking Distances

Speed (km/h) Distance (m) 20 50 40 150 60 250 80 400

100 600

120 800

140 1050

3.4.4 Dynamic Brake (DB) In the preceding chapter, the braking procedure prior to stopping at a station has been briefly explained and it has been shown, in Figure 2-2, that the braking mode takes place after the coasting phase and that the braking procedure itself consists of two main parts: dynamic braking from about 50 to 60 km/h to 20 km/h, and air braking from 20 km/h to a full stop of the train.

To apply dynamic brakes, the train control system will set the throttle to idle and set the dynamic brake to ‘set up’. This will energise the traction motor fields, which turns the traction motors into generators. Energising the motor fields provides a great deal of turning resistance on the axles. The control system then adjusts the amount of braking force by progressing through the notches on the dynamic brake lever.

While the train is coasting, the current position of the train will be received from the CBTC system. Based on this position the train control system, normally using an eight-notch controller similar to the throttle, energises the traction-motor fields and this causes the motors to act as generators. The resistance of the motor field acts as a brake on the train, which in turn helps to slow the train’s speed. The electric current generated by the motors in the dynamic-braking mode is a waste product and is dissipated as heat in banks of resistors located in a train carbody Figure 3-7.

Chapter 3: Train Dynamic Modelling 49

Figure 3-7: SIEMENS locomotive ER24PC, Braking Resistor Components; (1) Radial resistor sub assembly, (2) Blower, (3) Resistor high voltage connection, (4) Terminal box

Dynamic braking alone is insufficient to stop a train, as its braking effect rapidly diminishes below 15 to 20 km/h. The most effective retarding range of the dynamic brake is between 20 to 75 km/h, based on the rolling stock design. Therefore, it is always used in conjunction with the regular air brake. This combined system is called ‘blended braking’.

Although blended braking combines both dynamic and air braking, the resulting braking force must be designed to be the same as that provided by the air brakes on their own. This is achieved by close cooperation of ATO and TASC systems through maximising the dynamic brake portion, and automatically regulating the air brake portion, as the main purpose of dynamic braking is to reduce the amount of air braking required. This conserves air and minimises the risks of over-heated wheels.

The rough estimation is that the dynamic braking provides between 50% and 70% of the braking force during blended braking.

3.4.5 Parking Brake (PB) The parking brake may employ a system that utilises stored energy such as springs, hydraulics, pneumatic or electrical energy, mechanical ratchets, screw mechanisms, chains, cables, levers and linkages, or any combination thereof. But most commonly used is an indirect spring-applied brake controlled by air supplied from the auxiliary reservoir of one of the bogies to the parking brake cylinders. The auxiliary reservoir will be verified in terms of volume size for parking brake application. A parking brake will hold a locomotive on a grade of 1 in 30 for an indefinite period, based on Australian Rail Track Corporation (ARTC) Engineering

Standard, WOS 01.400 and it will operate on the maximum number of axles but not on less than 40% of the total number of axles.

Figure 3-8: Faiveley Transport Parking Brake Scheme

The parking brake is applied or released when the double action (bistable) magnet valve 06.05/01 receives the proper electric pulse. The system does include

Chapter 3: Train Dynamic Modelling 51

an anti-compound system directly on the parking brake cylinders in order to avoid superposing service and parking brake efforts.

A pressure transducer, 06.04/01, informs the Locomotive Control Unit (LCU) about the parking brake pressure. The parking brake indicators will show a red flag outside the locomotive when the parking brake is applied and a green flag when the parking brake is released.

Choke 107 assures a correct monitoring of the parking brake’s pressure, including in case of hose rupture. The choke will be sized 3.0 mm initially and will be finally defined during commissioning. A power assist may be utilised when power is available, but the parking brake design will allow the parking brake to be applied or released manually (or automatically, if so designed), without power assistance.

3.4.6 Automatic Air Brake The automatic air brake is a fail-safe brake that is being used in rolling stock. In this system, compressed air (700 – 900 kPa) which is produced by the locomotive compressor unit is transmitted along the train through a brake pipe. The air pressure is lowered to 490 kPa by a pressure regulator and the air is fed via the brake valve, brake pipe, and control valves to the auxiliary air reservoirs. When the compressed air in brake pipes and auxiliary air reservoirs is at 490 kPa, the brakes are not activated. However, the activated brake valve cuts the flow of air from the pressure regulator, so the air pressure in the brake pipes falls. The fall in air pressure is detected by the control valve which then regulates the flow of compressed air from the auxiliary air reservoirs to the brake cylinders. The brake cylinders activate the braking mechanisms to slow down the rolling stock and stop them.

The automatic air brake system controls the air brakes on all wagons and wagon sets of a train. The train’s brake line runs the length of the train, with hoses connecting the locomotives and wagons. Once a train is assembled, and all brake hoses are connected, the train line is charged with air, usually to 90 pounds of pressure. A control valve on each car draws air from the train’s brake line for that wagon’s reservoir. Changing the level of air pressure in the pipe applies or releases pressure on the brake pads on the brake discs (Figure 3-10).

Locomotive Wagon 1 Wagon 2

Figure 3-9: Automatic Air Brake Main Components [37]

To apply the automatic brakes, the train driver moves the handle to a desirable notch and allows air to escape from the train’s brake line, reducing the pressure. The drop-in pressure triggers the control valve on each wagon to apply the brakes. The percentage of air reduced determines the amount of air transferred from each wagon’s reservoir to the brake cylinder, which regulates the amount of braking force.

To release the brakes, train control system closes the brake valve. The train’s brake line begins recharging, which signals the control valves to release them (by releasing air from the cylinders) throughout the train.

Chapter 3: Train Dynamic Modelling 53

3.4.7 Wagon Brake (WB) The wagon brake system utilises the well-known principle of a ‘triple valve’ or, in new designs of locomotives, a ‘distributor’. Brakes are applied when brake pipe pressure is lower than auxiliary reservoir pressure, while brakes are released when the pressure difference is reversed. These basic actions are achieved via the sliding valve and graduating valve as shown in Figure 3-10. The service piston divides the volume of a distributor into an upper chamber and a lower chamber which are, respectively, connected to brake pipes and auxiliary reservoirs. The service piston changes the positions of the graduating valve and the sliding valve according to different pressure situations in the upper and lower chambers. There are various ports, orifices, and connecting grooves in the graduating valve, sliding valve, and sliding valve seat. Therefore, different positions of valves can form different air passages.

Figure 3-10: Sliding Valve And Graduating Valve Positions: (A) Release, (B) Preliminary Quick Service, (C) Brake, and (D) Lapping [37]

3.5 TRAIN BRAKE MODELLING

Most train brake modelling has ignored some items and just focused on a few main factors for braking distance. Calculation of a precise stopping spot using the

TASC algorithm which aligns rolling stock doors with PSDs, while using wireless data communication in the context of CBTC or ETCS systems, requires a sophisticated model which considers all possible factors.

Based on a train’s active forces, brake components and the braking principles introduced in previous sections, a train’s braking model which is appropriate to be used in TASC simulation is introduced in this section. To model a braking system which is sophisticated enough to be used in TASC, the below-mentioned factors must be considered in the proposed model:

1. Train’s mass

2. Train’s mass distribution

3. Train’s axles

4. Passenger weights (w0-w4)

5. Train’s speed

6. Train’s aerodynamic/cross section

7. Wind effect

8. Track profile (gradient, curve) in station

9. Station wet/dry rail

10. Rail profile

11. Tunnel effect

12. Train’s braking delay time

13. Train’s disc characteristics

Chapter 3: Train Dynamic Modelling 55

14. Train’s dynamic brake characteristics

15. CBTC position uncertainty

3.5.1 Effect of the distribution of a train’s mass Most train braking models ignore a train’s mass distribution and assume that a train’s mass is evenly distributed along the train’s length while the reality is different.

Based on stations’ design through a certain route, the train’s cars can be loaded unevenly. Based on [26] this can be modelled as follows:

2 푚. 푎. 푠 + 0.5. 푚. 푈 + 푚. 푔. (ℎ1 − ℎ2) = 0 (3-10)

Where m is train mass, 푔 is gravity acceleration, a is train acceleration rate, U is

Kinetic energy of the train at the time which brake is applied, S is stopping distance and ℎ is height (reflecting uphill and downhill).

For a TASC model, based on an average car’s capacity in each train, if the number of passengers for each car is more than two third of that car’s capacity, then that car is assumed as occupied, less than two third and more than one third is half empty otherwise it is deemed to be empty:

Car X has 1/3 or less of its capacity occupied, it is assumed as empty;

Car X has 1/3 to 2/3 of its capacity occupied, it assumed as half empty; and

Car X has more than 2/3 of its capacity occupied, it assumed as full.

3.5.2 Air and wind effects Air resistance of a moving train can be calculated from Davis’s formula which shows the normal resistance of air against a moving train while the wind effect is calculated from (3-11).

푘1 (3-11) 퐴 = ( + 1000푘 ). 푣푔2 푟 푚 2

Where

퐴푟 is the air resistance or wind effect (Newton);

푚 is train’s weight (ton);

푣 is train’s speed;

푘1=1.61797 kg/m;

푘2=0.000032 1/m.

Wind effect is positive if the wind direction is the same as the direction of train movement and it is negative if it is at an angle to the train. Therefore, wind effect is constantly variable. We shall add the resultant force to flange force in our simulation based on 3-12.

퐹푟 = 퐹푓 ± 푊푟 (3-12)

Where

퐹푟 is the total flange force (Newton);

퐹푓 is flange force (Newton); and

푊푟 is wind effect (Newton).

Chapter 3: Train Dynamic Modelling 57

To calculate flange force, 퐹푓 (푁푒푤푡표푛), we have used the Röckl's formula

[Wikipedia: Curve Resistance] which is:

650⁄ (3-13) 퐹 = { 푅 − 55 푅 > 300 푚 푓 500 ⁄푅 − 30 푅 < 300 푚

Where

푅 is the curve radius in meter;

3.5.3 Dry and Wet Rail Effects The condition of rail in terms of being wet, oily or dry will affect the friction between rails and train wheels which will be captured in our model for TASC simulation by using (3-13) and (3-14).

45.6 (3-14) ɥ = 0.0624 + 퐷 푣 + 260

13.5 (3-15) ɥ = 0.0405 + 푊 푣 + 120

Where:

ɥ퐷 is the effect of dry rail;

ɥ푊 is the effect of wet rail;

푣 is the rolling stock speed.

3.5.4 Dynamic Brake Model Dynamic brakes at lower speeds (less than 35 km/h) which is the scope iTASC, could safely be assumed as a linear function of train speed as per (3-15) 푣 is the rolling stock speed:

Rd ≈ BD = 12. V (3-16)

3.5.5 Parking Brake Model As the parking brake will not affect our model, we have assumed that it has a constant force which is sufficient to hold a train on each platform section after TASC brings the train to a halt.

3.5.6 TASC Brake Model In previous sections, different factors of train movements and braking systems based on different operational modes have been explained and mathematically modelled. The focus in those sections was only on the last part of train movement prior to stopping at the station as the focus of TASC algorithm is on this area of train movements, but modelling procedures and overall assumptions are precise enough to be applied to other modes of train operation and also use ATO algorithms.

Table 3-3 summarises those effective factors in modelling of a TASC braking system based on a train’s different braking systems, components which have been mentioned so far, and the different effective forces which are being applied to the train as it approaches a station prior to its stopping at the predefined position:

Chapter 3: Train Dynamic Modelling 59

Table 3-3: TASC Braking Model

Tractive N/A to TASC Force 2 Running Basic 1- Journal Resistance 푅퐵 = 푎 + 푏푉 + 푐푉 1 (N) Resistance Resistance 2- Flange Resistance 3- Air Resistance Additional 1- Grade Resistance 푅퐺 = 1000. 푀. 푔. sin 푎 (N) Resistance 2- Curve Resistance 34.9푎 (N) 푅 = . 푔 3- Tunnel Resistance 퐶 퐿푐 4- Wind 푅푇 = 0.00013. 퐿푇푈 (N) 푘1 5- Dry/wet rail 푅 = ( + 1000푘5). 푣. 푔2 (N) 푊 푚 effects 45.6 (N) ɥ = 0.0624 + , ɥ = 퐷 푣+260 푊 13.5 0.0405 + 푣+120

Starting N/A to TASC

Train Acting Force Acting Train Resistance ′ " Brake Friction/Air 1- Disk Brake 퐵퐹 = 1000. 푓푐(∑ 푃푐 + ∑ 푃푐 ) (N) Force Brake 2- Shoe Brake Dynamic 1- Regenerative ≈ BD = 12000. V (N) Brake Brake 2- Hydraulic Brake Electromagne 1- Magnetic Rail N/A to TASC tic Brake Brake 2- Eddy Current Brake Parking Brake 1- Disk Brake BP = Cons. (N)

3.6 TASC MODELLING

Even when there is an experienced driver on-board, it is hard to achieve precise stopping in stations due to different factors which play active roles in rolling stock brake procedure. Some of these factors are, constant changes in mechanical parts of a brake system, wear and tear of braking systems, different braking pad shapes, track

1 Based on [27] RB= 0.75+9.02n/w+0.0305v+(2.22CA/1000W)v^2 (N/tonnes) Where W= total weight of the car (ton) A=cross section of the train (m^2) N= number of axles V=speed (m/s)

profiles like gradient and curvature, temperature or rail wetness on rainy days when station platforms and tracks are located in an open area.

Based on what has been discussed so far, three ultimate goals of TASC algorithms are as follows:

1- Precise train stopping in stations;

2- One braking effort for stopping the train; and,

3- Passenger comfort level during the stopping procedure.

Prior to arriving at the station, when a train is on the main lines, the train control system will use its dynamic brakes to reduce the train’s speed from service speed of the main line to less than 20 km/h. The brake control unit will monitor the train’s speed while applying the dynamic brake and will release the dynamic brake when the train speed reaches less than 20 km/h. After that, TASC will activate the train’s automatic brake to bring it to a halt at an exact spot in the station.

Considering station stopping procedures, the braking model for TASC applies to the dynamic behaviour of a train at a speed of less than 20 km/h, with no traction effort, and applying an automatic air brake.

Limited deceleration is allowed to be applied by the train braking system as it will directly affect passengers’ comfort levels in passenger trains. For freight trains it potentially could damage the freight, or even damage the train and rupture coupler yokes in heavy trains based on the Ma and Zeng study [8]. So the challenge of precise stopping in stations has two major restrictions. If the braking force is massive, it damages the smooth operation of a train and even could stop a train before the exact

Chapter 3: Train Dynamic Modelling 61

predefined spot. In this case, the train will need to release the brake, reactivate the traction force, move forward slightly and reapply the brakes. If the braking force is not strong enough, the train may overrun the station and have to move backward to reach the desirable point.

Figure 3-11 shows a train speed profile based on our initial simulation results.

In our model we consider five distinct sections in a typical train speed profile as can be seen. Train movement starts by releasing the parking brake at station n. Section B shows the acceleration of a train until meeting the track service speed. Section C is cruising mode and a train control system will keep the acceleration at zero. When approaching the station (n+1), traction force is released as can be seen in Section D, and the train will experience a mode that is called ‘coasting’ in rail system terminology, which means that no traction and no brake force are being applied to the rolling stock. By nearing the station (n+1) where the train is supposed to stop, the massive inertia of a train must be overcome, for it to stop at a precise position. To achieve this prior to reaching a station, in Section E, the train’s dynamic brake will be applied.

After slowing to a speed of 20 km/h, ‘handover’ will occur. At this Handover

Point (HOP), the train’s braking changes from dynamic braking to the air braking procedure and TASC will take the control of the train until it reaches a halt at the predefined point known as Station Reference Point (SRP). So, the TASC has three control variables in our model as follow:

1) Initial speed of the train which normally is 15 to 20 km/h;

2) Position of the train compared to the stopping spot; and

3) Braking force.

The first two variables are out of train automatic stopping control, but TASC must adapt the braking force based on those parameters as they have a direct effect on stopping/braking distance of a train in stations.

Figure 3-11: Train Speed Profile

Having the exact stopping position at each station, wayside equipment, Access

Point (AP) will send the stopping spot’s data to the train’s on-board computer. At any moment the train can recognise its current position based on a comparison between two positioning systems; on-board odometry system and AP’s data package. It has been assumed that the exact position of a train at any given time is the numerical average of these two independent positioning systems, as follows:

푃푂 + 푃퐴푝 (3-17) 푃 = 푇 2

Where:

푃푇 is Train position;

푃푂 is train position calculated by odometry system;

Chapter 3: Train Dynamic Modelling 63

푃퐴푝 is Train position which is recognised by AP.

So TASC will consider the difference between HOP and SRP as braking distance which is introduced in (3-7) and, based on this distance and other dynamic parameters of the train, will set up and constantly calculate the required braking force to reach the exact stopping position.

푆 = ∑ ∆푆 (푚) = 퐻푂푃 − 푆푅푃 (3-18)

Calculation in (3-18) is a continuous procedure from HOP to SRP for any given time until train reaches to the stopping point, SRP. TASC algorithm will replace the

HOP with the current position of train based on (3-17) and will continue to calculate the BD and recalculate the brake rate until BD is equal to zero. Equation (3-19) shows the procedure:

푆 = ∑ ∆푆 (푚) = 퐻푂푃 − 푆푅푃

= (푃푇) 푡 − 푆푅푃

(3-19) = (푃푇) 푡+1 − 푆푅푃

⁞

= (푃푇) 푡+푛 − 푆푅푃

Note: The brake blending between 0 km/h and 15 km/h will normally be tuned/modified during the commissioning phase by rolling stock manufacturers.

Braking delay time can be considered in the simulation by letting the propulsion system speed up, until a constant amount of time that brake system needs to build up. For example, if at the HOP the speed is 20 km/h and the braking delay is four seconds, we consider the current deceleration for four seconds and then the TASC calculated deceleration will be applied.

3.7 BRAKE PARAMETERS

1- Initiate speed

2- Dynamic/Static coefficient of friction

3- Braking force

4- Deceleration

5- Train’s weight

6- Track profile; gradient and curve

7- Train’s mass distribution

8- Wheel arrangement

9- Wheel diameter

10- Number of axles

11- Number of brake discs per train axle

12- Build up brake time (Train borne unit reaction, Time to disable

propulsion unit, Additional time to apply brake, braking time)

13- Types of on-board braking systems (Direct, Dynamic)

Chapter 3: Train Dynamic Modelling 65

14- Cylinder pressure

3.8 SIMULATION PARAMETERS

In this section the parameters related to modelling of the train and its braking system for the simulation part of following chapters has been introduced. Table 3-4 summarises the data in two different main categories; general data and brake specifications. This data comes from real train specifications and we assumed that train consists of two locomotives and 24 wagons.

More information with regard to different stations’ track profile and the exact position of each station’s SRP will be given during simulation in the following chapters.

Table 3-4: Simulation Parameters Train: 2 Locomotives Class24 and 24 wagons 1 General Data of Locomotive Train Passenger train Number of cars 8 Brake type friction pairing of disc-brakes Wheel arrangement CO-CO Number of bogies 2 per car Number of axles per wagon 3 Number of total axles 96 2 Brake Specification Calculation method (UIC 544-1) Brake mode P (P mode for single locomotive) Assessment trains running at speeds 15 of less than 100 km/h using brake Factor of Inertial Mass 1.1 Brake weight 98% Minimum line curve 190 m Line Gradient 2/1000 Brake delay time 4 s

3.9 CONCLUSION

This chapter has dealt with modelling of the rolling stock and their braking systems. For simulation purposes, having a sophisticated dynamic model of rolling stock is critical.

Based on the focus of this thesis, having a precise braking model of rolling stock, especially for the lower speed, less than 20 km/h will help to achieve better results.

This part of the research study has been introduced in two main sections; Train dynamic behaviour modelling and braking system modelling.

Simulation for TASC benchmark in chapter 4 as well as iTASC simulation will use the models which are developed in this chapter and simulation data which are gathered in section 3.8, table 3-4.

Chapter 3: Train Dynamic Modelling 67

Chapter 3: Train Dynamic Modelling 69

Chapter 4: TASC Benchmark

4.1 INTRODUCTION

To facilitate a performance review and validation of our new algorithm, iTASC, which will be introduced in Chapter 5, in this chapter we report on the development of a TASC benchmark. The benchmark consists of the available and already-applied

Artificial Intelligence (AI) techniques to Automatic Train Operation (ATO) systems and

Train Automatic Stop Control (TASC) algorithms which had been introduced in chapter 2, Section 2.3.

In chapter 1, section 1.1, figure 1.2 the relationship between railway control and signalling system, ATO and TASC has been explained and because of TASC research work rarity, we have expanded our benchmark to ATO system as well. Table

4-1 shows our main motivations and references for the benchmark and the AI technique which has been used in each of them. Based on what has been represented in this table, it could be seen that our benchmark is browning Fuzzy control system and Artificial Neural Networks (ANN) from previous TASC algorithms and SVM from

ATO algorithms.

To stablish a valid benchmark, we have also examined and experimented the performance of selected algorithms using our real data set and present a critical comparison and analysis based on results of the simulations. In chapter 5 we will compare the performance of our algorithm using the same data set with these algorithms as our benchmark.

Table 4-1: Algorithms in the TASC Benchmark

Algorithm Type Technique Used

Reference TASC ATO FC ANN SVM

[2] and [10]     

[5] and [14]     

[7]     

[12]     

[23] and      [48]

This chapter is organised as follows: Firstly, some common AI techniques for train automatic stop control (TASC) and train automatic operation (ATO)—including

Support Vector Regression, Artificial Neural Network and Fuzzy Control System— have been developed and are reported on. This is followed by the simulation results of their performance using the same data set which is being used in the chapter 5. In

Section 4.3, the comparison of mentioned methods is introduced. Section 4.4 provides a brief conclusion.

Chapter 4: TASC Benchmark 71

4.2 TASC BENCHMARCK

This section outlines the TASC benchmark based on different AI techniques. We have proposed three TASC algorithms and conducted some preliminary simulations while just considering the track profile in our models to identify performance of the developed TASC benchmark algorithms. This practice has helped us to clarify a comparison between the performance of iTASC or intelligent TASC which is developed in chapter 5, and other previously developed TASC algorithms of train stop control which we have named them TASC benchmark.

We have developed two machine learning (ML) algorithms. We have also included a fuzzy control system for Automatic Train Operation (ATO) in our benchmark which is basically developed based on our previous work in [5].

The data sets for all algorithms are the same and reported from real data from different railway lines. All the benchmark algorithms have been applied to the braking system’s model and train dynamic model which were introduced in Chapter 3, in

Sections 3.5 and 3.6 respectively.

4.2.1 Support Vector Regression (SVR) Support Vector Machine (SVM) and SVR have been widely used in classification and regression tasks. The theory behind these models is to program linear or nonlinear models which generalises well to stored data by ‘learning’ a hyper-plane in multi-dimensional space which maximises the margin of the samples from the boundary. Figure 4-2 shows the large margin of classification.

Figure 4-1: Example of SVMs Classification to Find a Large Margin Classifier in the given binary classification problem (Between red and blue spots)

SVR was introduced by Vapnik et al in 1997 with the idea of applying a large margin concept to regression problems [104]. The theory behind this model is very similar to that of SVM. The idea is to ignore samples with values that are close to the predicted values and focus on the margin samples. Figure 4-3 shows this concept where the blue circles are samples with predicted values close to the observed value and the blue circles with red outline are the samples with errors which are greater than epsilon.

Chapter 4: TASC Benchmark 73

Figure 4-2: SVR uses the samples that are further away from the predicted value (blue circles with red outline) and ignores samples which are close to the predicted line (blue circles). The solid line is the prediction and dashed line is the margin.

In the following, the training algorithm for SVR which is presented in [104] has

푑 been developed for the problem of TASC. Let {(푥1, 푦1), … , (푥푙, 푦푙)} ⊂ 푅 × 푅 be the training set where d denotes the dimension of the feature space and R denotes the type of label data which is real value. The aim of the training algorithm is to find the function f(x) which predicts all the samples with less than epsilon error and is as flat as possible. Here we assume f is a linear function:

푓(푥) = 〈휔, 푥〉 + 푏 with ω ∈ℝ푑, 푏 ∈ ℝ (4-2)

For a given input x, the output is calculated as the dot product of the weights

(w) and the input (x) plus the bias term (b). To make sure that f is flat, the weights should be as small as possible by minimising the Euclidean norm of the weights ||

w||2. In other words, the problem of the training can be formulated as an optimisation problem, as following:

1 (4-3) minimise ‖휔‖2 2

푦 − 〈휔, 푥 〉 − 푏 ≤ 휀 (4-4) subject to { 푖 푖 〈휔, 푥푖〉 + 푏 − 푦푖 ≤ 휀

The objective function is to minimise the norm of the weights subject to make predictions for all the data points with an error smaller than 휀. The above problem is a complex problem which has a feasible solution. In some cases when the samples are noisy, it is possible to have some errors above the acceptable threshold, 휀. This

∗ can be done by introducing slack variables 휉푖, 휉푖 into the optimisation problem. The

∗ idea is to allow some samples to have a higher error than 휀 like 휀 + 휉푖 표푟 휀 + 휉푖 but minimise these larger errors by adding them into the main objective function:

푙 1 (4-5) minimise ‖휔‖2 + 퐶 ∑(휉 + 휉∗) 2 푖 푖 푖=1

푦푖 − 〈휔, 푥푖〉 − 푏 ≤ 휀 + 휉푖 (4-6) ∗ subject to {〈휔, 푥푖〉 + 푏 − 푦푖 ≤ 휀 + 휉푖 ∗ 휉푖, 휉푖 ≥ 0

The trade-off between the flatness of 푓 and the amount of deviation from 휀 is determined by C>0. A loss function based on the slack variable could be defined as following, Figure 4-4 shows the loss function as a function of the slack variable:

Chapter 4: TASC Benchmark 75

0 if |휉| < 휀 (4-7) |휉| = { 휀 |휉| − 휀 otherwise

Similar to many optimisation problems, the dual form of the support vector regression can be obtained:

푙 1 (4-8) 퐿 = ‖휔‖2 + 퐶 ∑(휉 + 휉∗) 2 푖 푖 푖=1

푙

− ∑ 훼푖(휀 + 휉푖 − 푦푖 + 〈휔, 푥푖〉 + 푏) 푖=1

푙 푙 ∗ ∗ ∗ ∗ − ∑ 훼푖 (휀 + 휉푖 + 푦푖 − 〈휔, 푥푖〉 − 푏) − ∑(휂푖휉푖 + 휂푖 휉푖 ) 푖=1 푖=1

∗ ∗ Here L is the Lagrangian and 휂푖, 휂푖 , 훼푖, 훼푖 are Lagrangian multipliers. The condition of optimum solution is that the following derivatives are zero:

푙 휕퐿 (4-9) = ∑(훼∗ − 훼 ) = 0 휕푏 푖 푖 푖=1

푙 휕퐿 (4-10) = 휔 − ∑(훼∗ − 훼 )푥 = 0 휕휔 푖 푖 푖 푖=1

휕퐿 (∗) (∗) (4-11) (∗) = 퐶 − 훼푖 − 휂푖 = 0 휕휉푖

Figure 4-3: Loss function Based on Slack Concept

By substituting the above conditions in the optimisation problem, we obtain the dual optimisation problem:

(4-12) 1 Minimise {− ∑ (훼 − 훼∗)(훼 − 훼∗)〈푥 , 푥 〉 2 푖 푖 푗 푗 푖 푗 푖,푗=1

푙 푙 ∗ ∗ − 휀 ∑(훼푖 + 훼푖) + ∑ 푦푖(훼푖 − 훼푖 )} 푖=1 푖=1

푙 ∗ ∗ Subject to ∑(훼푖 − 훼푖 ) = 0 and 훼푖, 훼푖 ∈ [0, 퐶] 푖=1

The regression function can be rewritten as following:

푙 푙 ∗ ∗ 휔 = ∑(훼푖 − 훼푖 )푥푖 = 0 and therefore, 푓(푥) = ∑(훼푖 − 훼푖 )〈푥푖, 푥〉 + 푏 = 0 푖=1 푖=1

Chapter 4: TASC Benchmark 77

This equation shows that the weights can be calculated as a linear combination of the training samples. A similar approach can be used to obtain a kernel version of support vector regression:

훼푖(휀 + 휉푖 − 푦푖 + 〈휔, 푥푖〉 + 푏) = 0 (4-13)

∗ ∗ 훼푖 (휀 + 휉푖 + 푦푖 − 〈휔, 푥푖〉 − 푏) = 0 (4-14)

(퐶 − 훼푖)휉푖 = 0 (4-15)

∗ ∗ (퐶 − 훼푖 )휉푖 = 0 (4-16)

푏 = 푦푖 − 〈휔, 푥푖〉 − 휀 푓표푟 훼푖 ∈ (0, 퐶) (4-17)

∗ 푏 = 푦푖 − 〈휔, 푥푖〉 − 휀 푓표푟 훼푖 ∈ (0, 퐶) (4-18)

푙 (4-19) ∗ 휔 = ∑(훼푖 − 훼푖 )휑(푥푖) 푖=1

푙 (4-20) ∗ 푓(푥) = ∑(훼푖 − 훼푖 )푘(푥푖, 푥) + 푏 푖=1

푘(푥푖, 푥) = 휑(푥푖)휑(푥) (4-21)

푘(푥푖, 푥) is the kernel and can take different forms such as polynomial, Gaussian, or Radial Basis Function (RBF).

푙 1 (4-22) 푅 [푓] = ‖휔‖2 + 퐶 ∑ 퐿 (푦) 푟푒푔 2 휀 푖=1

0, 푓표푟 |푓(푥) − 푦| < 휀 (4-23) 퐿 (푦) = { 휀 |푓(푥) − 푦| − 휀, otherwise

The optimisation problem of the kernelised SVR is as following:

1 1 (4-24) Minimise 휏(휔, 휉(∗), ε) = ‖휔‖2 + 퐶 (푣휉 + ∑(휉 + 휉∗)) 2 푙 푖 푖

Subject to ((휔. 푥푖) + 푏) − 푦푖 ≤ 휀 + 휉푖 (4-25)

∗ 푦푖 − ((휔. 푥푖) + 푏) ≤ 휀 + 휉푖

(∗) 휉푖 ≥ 0, 휀 ≥ 0

푘(푥푖, 푥) = 휑(푥푖)휑(푥)

푘(푥푖, 푥) = 휑(푥푖)휑(푥) (4-26)

Where 휈 is the upper bound on the fraction of error points or the lower bound on the fraction of points inside the ε-insensitive tube. The dual formulation is given below:

Maximise W(훼(∗)) (4-27)

푙 푙 1 = ∑(훼 − 훼∗)푦 − ∑ (훼 − 훼∗)(훼∗ − 훼 )푘(푥 , 푥 ) 푖 푖 푖 2 푖 푖 푖 푖 푖 푗 푖=1 푖,푗=1

푙 (4-28) ∗ subject to ∑(훼푖 − 훼푖 ) = 0 푖=1

퐶 0 ≤ 훼(∗) ≤ , 푖 푙

푙 ∗ ∑(훼푖 + 훼푖 ) ≤ 퐶. 휈 푖=1

Chapter 4: TASC Benchmark 79

∗ Where 훼푖, 훼푖 are Lagrangian multipliers. The slack variables are also included in the above optimisation problem.

To simulate the SVR model with a Gaussian kernel and a polynomial kernel, first we standardised the data by performing mean removal and dividing the data by the standard deviation. Then the scale of the kernel was set to auto. The order of polynomial kernel was set to three.

Figure 4-4: SVR Technique for TASC, Station 1

Figure 4-5 shows the results of simulation for station 1, which as mentioned in section 4.2.1, is a levelled station without any curvature during station track section.

We again ignored other parameters here.

The results are promising in this method compare to those of RL simulation as the estimated data and observed data show a similar trend. But the strength of this method is apparently lying in higher speeds, as can be seen, for the speed of more

than 75 km/h, the learning trend is more satisfactory. However, in low speed which is the main focus of TASC algorithm there are some noticeable errors.

Developing a clear understanding of SVR algorithm in TASC issue we have conducted simulations for three different stations as can be seen in Figure 4-6, 4-7 and 4-8.

Figure 4-5: SVR Technique for TASC, Station 1

The position of the station in Figure 4-6 is located at 4250 m from a starting point at the beginning of the rail line. Using SVR algorithm, the train stops at the position of 4248.39 m. While it is not exact stopping, comparing to the stopping error of LR, SVR is showing noticeably better results.

This station is the same as station 1 in LR which as mentioned before, is levelled without any curvature. We still have not added different parameters in our model such as uneven distributed passengers or braking delay because while the braking distance is improved compared to the previous technique, there are still some issues

Chapter 4: TASC Benchmark 81

with the number of Braking Attempt (BA) to stop the train. The BA in station 1 which is a levelled station is 13 times, which means the TASC algorithm has changed the braking rate 13 times from HOP to SRP and this directly affect the passenger’s level of ride comfort.

Similar problem can be found in station 2 and station 3, Figure 4-7 and Figure

4-8 respectively. Where BA is 9 for station 2 and 10 for station 3.

푑푎 The jerk level which is the derivative of acceleration (퐽 = ) is considered as 푑푡 another factor which shows passengers’ level of comfort. Jerk in station 3 is -4 which is far from comfort standard level (normally recommended jerk level is less than 1

푚 ( ⁄푠3)).

Figure 4-6: SVR technique for TASC, station 2

Figure 4-7: SVR Technique for TASC, Station 3

4.2.2 Artificial Neural Network (ANN) Neural Network is a class of machine learning technique that were widely used in the 1980s. After two decades, thanks to progress made in computational power, neural networks are a trending technology in many artificial intelligence applications like object recognition, automatic control systems and recommender systems. The training of ANN is based on a well-known algorithm called ‘Backpropagation’. The

Backpropagation neural network is a multilayered, feedforward neural network and is by far the most extensively used algorithm [105]. It is also considered one of the simplest and most general methods used for supervised training of multilayered neural networks, Lu [105].

Backpropagation works by approximating the non-linear relationship between the input and the output by adjusting the weight values internally. It can further be

Chapter 4: TASC Benchmark 83

generalized for the input that is not included in the training patterns (predictive abilities).

Figure 4-8 shows the topology of the Backpropagation neural network that includes an input layer, one hidden layer and an output layer. It should be noted that

Backpropagation neural networks can have more than one hidden layer. The MATLAB training interface for ANN has been shown in Figure 4-9.

Figure 4-8: Structure of a Feedforward Network with One Hidden Layer [105]

Figure 4-9: MATLAB Neural Network Training Interface

In our simulation, to train the neural network, we have used a three-layered

Feedforward Neural network with five neurons in the hidden layer, two neurons in the input layer for speed and track slope and one neuron in the output layer, which is the braking distance, Figure 4-10. The Sigmoid function was used as the activation function in the hidden layer and we have used the Backpropagation algorithm for training the network, Algorithm 4-1. The objective function was the mean square error between the output of the network and the observed braking distances.

Chapter 4: TASC Benchmark 85

Bias

1 θ1

W1,3

x1 O1

Input Layer Hidden Layer Output Layer

Figure 4-10 Structure of the Neural Network with One Hidden Layer and Braking Distance as its Output

Algorithm 4-1: A Typical Backpropagation Algorithm

1. µ ← Training Data Set 2. £ ← Initialise all weights 3. Ѳ ← Initialise all biases 4. ω ← Number of layers in the Neural Network 5. Propagate forward 6. ɥ ← The error for all layers 7. Backpropagate error in each layer 8. Repeat until ɥ < Predefined value

The results can be seen in Figures 4-11 to 4-14. Figure 4-11 shows noticeable error in braking distance for low speed and simulation for three different stations in

Figure 4-12, 4-13 and 4-14 show noticeable errors in stopping distance. The stopping distances for station 1, 2 and 3 are 4247.218 m, 6223.606 m and 8548.850 m while

as per previous, the exact position of SRP for those three stations are 4250, 6250 and

8550 respectively.

Station one has the worst braking distance, while station 3 has the highest level

푚 of jerk which is 5 ( ⁄푠3) . Braking Attempts for ANN algorithm are 12, 5 and 9 for stations 1 to 3. The BA in station 1, which is a levelled station shows that the ANN algorithm was not able to find a proper braking rate and change the braking rate several times until reaches to the SRP.

Figure 4-11: ANN Technique for TASC, Station 1

Chapter 4: TASC Benchmark 87

Figure 4-12: ANN Technique for TASC, Station 2

Figure 4-13: ANN Technique for TASC, Station 3

Figure 4-14: ANN Technique for TASC, Station 4

4.2.3 Fuzzy Control A fuzzy control system or fuzzy logic is a powerful computational approach which could be applied to complex systems when there is not clear membership status for elements. In fuzzy control, set elements could partially belong to a set rather than crisp sets, for which the membership value of elements can only obtain two values; 0 where the element does not belong to the set, or 1 where the element is a member of the set. The basic block diagram of a fuzzy system has been shown in

Figure 4-18.

Figure 4-15: Fuzzy System Block Diagram [99].

Chapter 4: TASC Benchmark 89

To develop the fuzzy benchmark, we have used the fuzzy controller from our previous work [5], where an ATC system has been introduced using a fuzzy controller in each train, based on multi-agent systems (MASs) which are controlled through cooperative control. The if-then rules control the movements of trains between two stations. The controller model contains membership functions which are triangle and used Mamdani FIS. Instead of simple train dynamic model in [5], the rolling stock and braking systems models which are introduced in previous chapter have been used.

The result for the Fuzzy controller has been shown in Figures 4-20 to 4-23.

Figure 4-16: Fuzzy Function of Membership for Input and Output of Trains [5].

Simulation results for FC algorithm have been shown in Figures 4-17 to 4-20. As can be seen in these figures, the result in fuzzy control is the best among other already-reviewed algorithms.

Braking distance is in an acceptable range, the stopping error is about 5% and the jerk level is limited to -3. But the downside of FC algorithm is the number of BA which is the highest among other algorithms. As an example, BA in station 3, Figure

4-20 is 17 times which means TASC algorithm is struggling to find a suitable braking rate and is continuously changing the braking rates until train stops.

Figure 4-17: FC Technique for TASC, Station 1

Figure 4-18: FC Technique for TASC, Station 2

Chapter 4: TASC Benchmark 91

Figure 4-19: FC Technique for TASC, Station 3

Figure 4-20: FC Technique for TASC, Station 4

4.3 SIMULATION DATA SET

To validate our models which is introduced in next chapter; Chapter 5, as well as realising the performance of previously developed TASC algorithms, a diverse set

of data set from different lines and different rolling stock (20 sets) providers have been gathered as follow:

1- Tehran Metro – DC trainsets, DKZ3 (7 sets)

2- Sydney Metro – Waratah, A set (3 sets)

3- Siemens Locomotive, ER24PC (10 sets)

Braking characteristics coming from the rolling stock data sheets while for the braking distance we have used the test results before they start revenue operation. For data set 3, Siemens Locomotive, we also used some routing test results which the braking distance has been measured using Garmin eTrex 30 GPS in open area between Shahrood and Mashahd stations.

The track profile for the simulation gathered across three stations across T1 line in Sydney metro rail network. The track profile details coming from track layout which include the length of each station which is not important for the scope of this study and station curves and gradients.

4.4 SIMULATION RESULTS AND COMPARISON

Table 4-2 summarises the results of the single model trained and tested 20 times/fold using random portioning.

Each fold comes from a real rolling stock specification and their performance in real operation. We have used 70 per cent of the data set (which is coming from different sources; Sydney metro, Tehran metro and Iran National Railway) for training and 30 per cent of the data for testing.

Each row shows the result of the four models in each fold. The last two rows are the mean error and the confidence interval. The average across all the train types and confidence intervals are presented in the last two rows, respectively.

Chapter 4: TASC Benchmark 93

Based on performance results, the best model in terms of stopping error was the fuzzy control system with more accurate stopping distance, among others. The main problem with FC as the single control procedure for the TASC algorithm is the high number of braking attempt (BA) which indicates the weakness of the algorithm to find a suitable braking rate in low speed. This is the main motivation for us in

Chapter 5 to benefit from stopping accuracy of FC while trying to limit the BA in train low speed.

We also have conducted simulation for entire data set by combining different rolling stock parameters and braking distance to achieve a bigger data set in the hope of achieving better results but as the results was not so promising we have not reported them and just focused on the single mode simulation.

In single mode simulation, we have collected braking distance for 20 different rolling stock models with different dynamic behaviour as well as different braking systems.

Table 4-2: The Results of the Single Model Trained on All Data

Fold LR FC SVR Gk SVR Plk ANN 1 96.288 0.56672 0.62837 0.7342 0.7770

2 104.72 0.56192 0.67944 0.75641 0.7234

3 110.48 0.48906 0.57904 0.67757 0.6783

4 89.116 0.5352 0.58454 0.64807 0.6870

5 93.206 0.57234 0.6581 0.70947 0.7305

6 104.15 0.57335 0.70336 0.76804 0.7469

7 88.088 0.56004 0.62276 0.65109 0.7489

8 106.4 0.60001 0.64713 0.76039 0.8563

9 111.47 0.58509 0.71299 0.8009 0.7347

10 126.46 0.56328 0.75642 0.79136 0.7313

11 107.35 0.54143 0.6216 0.68269 0.7851

12 123.23 0.57737 0.67152 0.73286 0.8164

13 93.045 0.55468 0.63258 0.71209 0.6511

14 103.19 0.56096 0.66604 0.73262 0.7424

15 122.28 0.56667 0.7378 0.77099 0.9044

16 97.94 0.56454 0.71178 0.7612 0.7650

17 126.38 0.54752 0.66409 0.72452 0.7542

18 138.01 0.52617 0.58354 0.67584 0.6703

19 119.08 0.51804 0.63401 0.67859 0.6752

20 101.4 0.58228 0.64134 0.77142 0.6831

Mean 108.11 0.55733 0.65682 0.72702 0.7431

STD 5.9493 0.010923 0.021242 0.019688 0.0640

Single Mode 1 0.8 0.6 0.4 MSE 0.2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Fuzzy Control 0.6 0.6 0.5 0.5 0.6 0.6 0.6 0.6 0.6 0.6 0.5 0.6 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.6 SVR Gaussian kernel 0.6 0.7 0.6 0.6 0.7 0.7 0.6 0.6 0.7 0.8 0.6 0.7 0.6 0.7 0.7 0.7 0.7 0.6 0.6 0.6 SVR Polynomial kernel 0.7 0.8 0.7 0.6 0.7 0.8 0.7 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.8 0.8 0.7 0.7 0.7 0.8 Neural Network 0.8 0.7 0.7 0.7 0.7 0.7 0.7 0.9 0.7 0.7 0.8 0.8 0.7 0.7 0.9 0.8 0.8 0.7 0.7 0.7 Fold

Fuzzy Control SVR Gaussian kernel SVR Polynomial kernel Neural Network

4.5 CONCLUSION

This chapter has dealt with developing a benchmark for TASC algorithms. Five different TASC algorithms were introduced based on the best practices, and their performance were tested using models for rolling stock and braking systems which were proposed in Chapter 3.

Simulation results showed that, among introduced AI algorithms, Fuzzy controller and SVR with Gaussian kernel offer the best results; minimum stopping

Chapter 4: TASC Benchmark 95

error and the most comfortable level of ride. The stimulating fact about Fuzzy controller is that, the controller in [5] has been applied to the models as explained in

Chapter 3. As a powerful tool, the fuzzy controller inspired us to use it as a part of our TASC algorithm as shown in Chapter 5.

We have chosen four different stations over the Sydney Rail Network with different track profiles in their station platforms as well as in their previous tracks.

The same stations will be used in Chapter 6 to draw a valid comparison.

Chapter 4: TASC Benchmark 97

Chapter 5: iTASC

5.1 INTRODUCTION

In this chapter we introduce a new TASC algorithm, called iTASC, which has been developed using the RL technique.

In chapter 4, after introducing the TASC algorithms, simulation on real data has been done and the results of those simulations led us into using Fuzzy systems and proven models in Q-learning helped us to choose fuzzy q-learning model.

iTASC has been built upon the strength of double q-learning and the fuzzy control system and eliminates the weakness of overestimation of q-learning and model-based approach of the fuzzy control system. Figure 5-1 indicates the relationship between iTASC and other Artificial Intelligence (AI) techniques as well as rail system disciplines.

Chapter 5: iTASC 99

Figure 5-1: iTASC Relationship with AI and Rail System Disciplines

5.2 REINFORCEMENT LEARNING

Reinforcement learning (RL) is one of the well-known and popular machine learning (ML) techniques, which has successfully been applied to many complex engineering and non-engineering problems. RL is goal-oriented within which an agent or agents will find a policy and learn through their trial-and-error interactions in an environment without a teacher or supervisor. Those interactions and agent experiments bring feedback and delayed rewards or punishments for the agent through exploration or exploitation of states and actions. The agent aims to reach to the goal while accumulating maximum rewards. These two features of reinforcement learning highlight one of challenges in this learning approach, which is totally different from supervised and unsupervised learning, Figure 5-2.

100

One of the unique challenges in RL which is not seen in other ML techniques is keeping a balance between exploration and exploitation within uncertain learning environments. At each episode of learning, an agent aims to obtain the maximum rewards; but, as mentioned before, RL is a goal-oriented learning approach which means an agent must accumulate maximum rewards while trying to reach the goal.

If agent exploits at each episode it might stay in local maximum which mean infinite learning loop and never reaches to the final goal.

Another problem of exploiting all the time is that an agent might overlook a better choice of actions; therefore, an agent must explore new actions in the hope of gaining better rewards [87].

Figure 5-2: Comparison Between Different Types of ML [96]

Exploration-based policies make the agent tend to choose a random action during some small fractions of the time, in order to get new information which could lead to better policies and a better control in our TASC algorithm. Exploitation• based policies, instead, make the agent tending to choose actions which are supposed to be the best (in terms of accumulated reward) and so focusing on the already acquired information. A good way to follow for balancing exploration and exploitation is to

Chapter 5: iTASC 101

choose a behaviour policy as a mixture of both policies, as the following equation shown:

푛푡ℎ푎푐푡푖표푛 = 푟. 푛푒푥푝푙표푟푒 + (1 − 푟)푛푒푥푝푙표푖푡 (5-1)

Where 0 < 푟 < 1 is a parameter that can be opportunely adjusted to make the policy more exploration-like (r close to 1) or exploitation-like (r close to 0). In this second case there are, for instance, all the on-policy methods.

5.2.1 Reinforcement Learning Elements The main elements of RL are reward (or punishment) signal which is assigned to the agent based on its action taken; policy which is the way of agents’ behaviour in the environment; and value function which defines the best actions in long run.

Agent at each time takes an action and based on that action, its current state will change and based on the environment within which agent toke that action, will go to the new state and acquire a reward signal, Figure 5-3.

Figure 5-3: RL Elements, Sutton and Barto [87]

The agent here in TASC algorithm is TASC itself which interacts with the environment. The environment is The ATO and stations platform and even it can be extended to PSD but, as we have chosen the SRP, based on the station platform and

102

PSD, it is not necessary to add another parameter for PSD in our environment model.

To be more precise, our environment in iTASC are as follow:

1- Station gradient;

2- Station curve;

3- Station rail situation; wet, dry;

4- Number of passengers;

5- Passenger distribution through the train;

6- Train dynamic behaviour as per Section 3-3; and

7- Brake performance including all brake parameters in Section 3-4.

Based on previous chapters and complexity of TASC algorithm, the learning approach could not be supervised as based on different factors, ATO might handover the train to the TASC in different situations in terms of weight of train, gradient of a track, speed at handover point etc.

We have used all four sub-elements of a reinforcement learning system in our iTASC algorithm as follows:

1- Policy: the way of applying brake prior to stop

2- Reward signal: a scaler value, sometimes known as Reinforcement, which assign

to iTASC after each stopping procedure, based on the stopping positions and

braking rates; the more exact position of stopping with the smoother braking

rate, the higher reward the iTASC will get.

3- Value function: a prediction of iTASC reward signal over the long-run.

4- Model of environment: for efficient learning, prior information of station and

braking distance have been considered in the iTASC algorithm.

Chapter 5: iTASC 103

5.2.2 Markov Decision Processes (MDPs) MDPs formulate the process of sequential decision making when each decision influences the current outcomes as well as subsequent ones - in other words, delayed rewards. At each time, the decision making process is in state 푠 and a decision maker takes an action or makes a decision 푎. This action takes the decision maker to the next state 푠′ and the decision maker will get a reward based on the new state, old

′ state and the action; 푅푎(푠, 푠 ).

The MDPs are the mathematical description of reinforcement learning (RL) and what has been shown in Figure 5-3 was actually an MDP where:

• 푆 is a set of states;

• 퐴 is a set of actions;

′ ′ ′ • 푃푎(푠, 푠 ) = 푃푟{푠푡+1 = 푠 | 푠푡 = 푠, 푎푡 = 푎 }, 푠 and 푠 휖 푆 ; and

′ • 푅푎(푠, 푠 ) is the reward which is received after changing the state from

푠 to 푠′ as a result of the action 푎;

∞ 푘−푡−1 푘 • 퐺푡 = ∑푘=푡+1 훾 푅 is the total rewards which are achieved from

a sequence of decisions or actions.

The solution for each MDP is finding a policy for making the best decision at each state which is as follows:

′ ′ ′ (5-2) 푉(푠) = ∑ 푃휋 (푠) (푠, 푠 )(푅휋(푠)(푠, 푠 ) + 훾푉(푠 )) 푠′

(5-3) 휋(푠) = 푎푟푔푚푎푥 {∑ 푃 (푠′| 푠, 푎)(푅(푠′| 푠, 푎) + 훾푉(푠′))} 푠′

104

5.3 Q-LEARNING

Q-learning is one of the most popular and active branches of Reinforcement

Learning (RL) in many areas in which agents experience a non-predictive and dynamic environment. This method of learning has also been considered as a method of

Temporal-Difference (TD) learning which is a combination of Monte Carlo and

Dynamic Programming (DP) for solving finite Markovian decision processes, Sutton and Barto [87] and Watkins [95]. Watkins in [97] has introduced the Q-learning update function shown below:

푄푡+1 (푠푡, 푎푡) = 푄푡 (푠푡, 푎푡) + 훼푡 (푠푡, 푎푡)[ 푟푡 + 훾푚푎푥푄푡 (푠푡+1, 푎) − (5-4)

푄푡 (푠푡, 푎푡)]

Where

푄푡 (푠푡, 푎푡) indicates the value of the action 푎 when agent is in state 푠 at time 푡

훼푡 is learning rate of the agent which is 훼푡 (푠, 푎) 휖 [1,0]

′ 푠′ 푟푡 will be from 푅: 푆 × 퐴 × 푆 → ℝ where 퐸{푟푡| (푠, 푎, 푠 ) = (푠푡, 푎푡, 푠푡+1 } = 푅푠푎

푠푡 shows the agent state at 푡

푠푡+1 will be determined by a fixed state transition distribution 푃: 푆 × 퐴 × 푆 →

[0,1]

푠′ ′ 푃푠푎 is the probability of ending up at 푠 for agent while it is in 푠 and takes action 푎

푠′ and ∑ 푃푠푎 = 1. 푠′

What has been shown in (5-4) is the solution to the Bellman optimality equation shown below [87], [98]:

Chapter 5: iTASC 105

∗ 푠′ 푠′ ∗ ′ (5-5) ∀ 푠 , 푎: 푄 (푠, 푎) = ∑ 푃푠푎 (푅푠푎 + 훾 max 푄 (푠 , 푎)) 푠′

Since Q-Leaning is an off-policy TD method, while the improved policy is greedy and deterministic, the actions chosen for control are based on some other policy

(behaviour policy), which could be unrelated with the first one (e.g. an E-greedy policy, with a uniform distribution over the action space).

In order to overcome the problem of overstimulation of action value in Q- learning algorithm and improving the learning rate, the Double q-learning has been introduced by Hasselt in [98]. Based on this study Double q-learning shows better performance and not only overestimations, but it also in some cases shows underestimation.

In railway territory and in the iTASC algorithm, if we could not achieve to find the exact stopping position which is the ideal performance of the iTASC agent in train stopping at a station, the position is preferred to be underestimated which means stopping before the exact position rather than passing the target position. This is because railway operators prefer forward movement in stations if there is any error rather than backward throttling or turnback siding.

The only operation mode within which rolling stock backward traction is allowable is in the rail yards or maintenance depots where train configuring occurs or shunting movements is being handled.

Q-learning has been considered as an off-policy control method because the function of q-learning learns from outside of the current policy and also this is a value-

106

based algorithm which concentrates on the reward taken by agent according to the actions chosen rather than policy-based algorithm.

To run the Q-learning algorithm we used a 10 by 10 state space. The number of iterations was set to 10000. The learning rate was set to 0.1.

5.3.1 Double Q-Learning In double Q-learning [98], there are two Q functions and their update functions based on (5-4) use the value of the other as follows:

퐴 퐴 푄 푡+1 (푠푡, 푎푡) = 푄 푡 (푠푡, 푎푡) + 훼푡 (푠푡, 푎푡)[ 푟푡 + (5-6)

퐵 퐴 퐴 훾푚푎푥푄 푡 (푠푡+1, 푎 ) − 푄 푡 (푠푡, 푎푡)]

퐵 퐵 푄 푡+1 (푠푡, 푎푡) = 푄 푡 (푠푡, 푎푡) + 훼푡 (푠푡, 푎푡)[ 푟푡 + (5-7)

퐴 퐵 퐵 훾푚푎푥푄 푡 (푠푡+1, 푎 ) − 푄 푡 (푠푡, 푎푡)]

Where

푎퐴 = 푎푟푔푚푎푥 푄퐴 (푠, 푎);

푎퐵 = 푎푟푔푚푎푥 푄퐵 (푠, 푎); and

Similar to (5-4), 푄푡 (푠푡, 푎푡) indicates the value of the action 푎 when agent is in state 푠 at time 푡

훼푡 is learning rate of the agent which is 훼푡 (푠, 푎) 휖 [1,0]

′ 푠′ 푟푡 will be from 푅: 푆 × 퐴 × 푆 → ℝ where 퐸{푟푡| (푠, 푎, 푠 ) = (푠푡, 푎푡, 푠푡+1 } = 푅푠푎

푠푡 shows the agent state at 푡

푠푡+1 will be determined by a fixed state transition distribution 푃: 푆 × 퐴 × 푆 →

[0,1].

푠′ ′ 푃푠푎 is the probability of ending up at 푠 for agent while it is in 푠 and takes action 푎

푠′ and ∑ 푃푠푎 = 1. 푠′

Chapter 5: iTASC 107

As the main aim of the agent in the RL environment is maximising the reward function, for the problem of iTASC, we have placed the maximum reward on the exact position of rolling stock stopping point at the stations where all rolling stock doors are perfectly aligned with the corresponding PSD. For other situation which is different from the learning goal, the closer to the predefined position, the better reward will agent receive. Similar policy has been applied for the jerk which will be explained in simulation section, 5-6.

5.4 FUZZY DOUBLE Q-LEARNING

Fuzzy Q-learning has been introduced by Glorennce and Jouffe [100] within which Fuzzy sets have been combined with the popular Q-learning algorithm and in that algorithm, actions, Q-value and Q-function have been inferred by using a Fuzzy logic system. The idea of iTASC has been formed by studying this paper and combining the Fuzzy Q-learning with Double Q-learning [98].

In [101] Glorennce and Jouffe have introduced the action in Q-learning and Q- function which are based on Takagi-Sugeno FIS as follows:

푁 ∑푖=1 푎푖(푥) × 푎푖 (5-8) 푎(푥) = 푁 ∑푖=1 푎푖(푥)

푁 ∑푖=1 푎푖(푥) × 푞(푆푖, 푎푖) (5-9) 푄(푥, 푎) = 푁 ∑푖=1 푎푖(푥)

Where the number of rules is 푁 and

풊풇 푆푖 풕풉풆풏 푎푐푡푖표푛 = 푎푖, 푎푖 ∊ 퐴푖 (5-10)

108

풊풇 푆푖 풂풏풅 푎푐푡푖표푛 = 푎푖 풕풉풆풏 푞 − 푣푎푙푢푒 = 푞(푆푖, 푎푖) (5-11)

In (5-10) 퐴푖 is possible actions in state 푆푖 and the 푆푖 which is defined by 푥1 is 푆푖,1 and

푥2 is 푆푖,2 . . . and 푥푛 is 푆푖,푛.

풊풇 푥 푖푠 푆푖 풕풉풆풏 푎[푖, 1] 푤푖푡ℎ 푞[푖, 1] (5-12)

or 푎[푖, 2] 푤푖푡ℎ 푞[푖, 2]

⁞

or 푎[푖, 퐽] 푤푖푡ℎ 푞[푖, 퐽]

In (5-12) 푎[푖, 푗] is the 푗푡ℎ possible action and [푖, 푗] is its corresponding q-value. The

Membership function of our previous work [5] has been used and the figure 5-4 shows the range of those functions. Then number of rules are 7 to manage a smooth stopping procedure.

Figure 5-4 Fuzzy Memberships Functions

Algorithm 5-1 shows the algorithm for Fuzzy Double Q-learning algorithm which we have named as iTASC.

Chapter 5: iTASC 109

110

Algorithm 5-1: Fuzzy Double Q-learning Pseudocode for iTASC

1. 푆푡푒푝1: 퐼푛푖푡푖푎푙푖푠푖푛푔 푄퐴, 푄퐵, 푠 푤푖푡ℎ 푡푒푠푡 푑푎푡푎

2. 푆푡푒푝 2: 풓풆풑풆풂풕풆

3. 푆푡푒푝 3: 푇푎푘푒 푎푛 푎푐푡푖표푛 푎푖 푓표푟 푒푎푐ℎ 푟푢푙푒 푖

푁 ∑푖=1 푤푖(풙) × 푎푖 4. 푆푡푒푝 4: 퐷푒푓푖푛푒 푎(풙) = 푁 ∑푖=1 푤푖(풙)

5. 푆푡푒푝5: 퐶ℎ표표푠푒 푎, 푏푎푠푒푑 표푛 푄퐴, 푄퐵 표푏푠푒푟푣푒 푟, 푠′

6. 푆푡푒푝 6: 퐶ℎ표표푠푒 푒푖푡ℎ푒푟 푈푃퐷퐴푇퐸 퐴 표푟 푈푃퐷퐴푇퐸 퐵

7. 푆푡푒푝 7: 풊풇 푈푃퐷퐴푇퐸 퐴 풕풉풆풏

∗ ∗ 퐴 8. 퐷푒푓푖푛푒 푞[푖, 푖푎] 푤ℎ푖푐ℎ 푖푠 푡ℎ푒 푠푒푙푒푐푡푖표푛 표푓 푎푐푡푖표푛 푖푎 푡ℎ푎푡 푚푎푥푖푚푖푠푒푠 푡ℎ푒 푄 푣푎푙푢푒

∑푁 ( ) ∗ 퐴 푖=1 푤푖 풙 푎푖푞[푖, 푖푎] 9. 푄 (풙, 푎) = 푁 ∑푖=1 푤푖(풙)

∑푁 푤 (풙′)푎 푞[푖, 푖∗ ] [ ∗ ] ( ′) 푖=1 푖 푖 푏 퐴( ) 10. ∆푞 푖, 푖푎 = 휂(푅 푥, 푎, 푥 + 훾 ( 푁 ′ ) − 푄 풙, 푎 ) ∑푖=1 푤푖(풙 )

11. 푆푡푒푝 8: 풆풍풔풆 풊풇 푈푃퐷퐴푇퐸 퐵 풕풉풆풏

∗ 퐴퐵 12. 퐷푒푓푖푛푒 푞[푖, 푏] 푤ℎ푖푐ℎ 푖푠 푡ℎ푒 푠푒푙푒푐푡푖표푛 표푓 푎푐푡푖표푛 푖푎 푡ℎ푎푡 푚푎푥푖푚푖푠푒푠 푡ℎ푒 푄 푣푎푙푢푒

∑푁 ( ) ∗ 퐵 푖=1 푤푖 풙 푎푖푞[푖, 푖푏] 13. 푄 (풙, 푎) = 푁 ∑푖=1 푤푖(풙)

∑푁 푤 (풙′)푎 푞[푖, 푖∗ ] [ ∗ ] ( ′) 푖=1 푖 푖 푎 퐴( ) 14. ∆푞 푖, 푖푏 = 휂((푅 푥, 푎, 푥 + 훾 ( 푁 ′ ) − 푄 풙, 푎 ) ∑푖=1 푤푖(풙 )

15. 푆푡푒푝 9: 풆풏풅 풊풇

16. 푆푡푒푝 10: 퐶푎푙푐푢푙푎푡푒 푟푒푤푎푟푑

17. 푆푡푒푝 11: 푠 ← 푠′

18. 푆푡푒푝 12: 풖풏풕풊풍 푒푛푑

Chapter 5: iTASC 111

5.5 ITASC CLOSED CONTROL LOOP

iTASC uses the closed control loop which has been shown in Figure 5-4. In this diagram iTASC module which is one part of ATO system, takes the control of train in stations and its algorithm is what we have developed in Section 5-4, Fuzzy Double Q- learning algorithm.

The output of the iTASC is a control signal which regulates the train braking rate. The other modules in this control system are as follow:

1. The train module is based on introduced model in Section 3-4 of Chapter 3

and for the braking distance the formula 3-7 has been used.

2. The braking system is using the model that has been introduced in Sections 3-

3 of Chapter 3. Some of the parameters of braking model are could not be

applied to TASC for example magnetic rail brake or starting resistance (refer

to table 3-3).

3. The Running Resistance module represents the Basic and Additional

Resistance in table 3-3. These parameters are always negative except for

Gradient Resistance which could be either negative or positive. For this

reason, we extract the Track Profile Resistance and showed it in different

module with bot negative and positive output.

4. Operational Control Centre (OCC) manages the high-level issues including safe

and efficient rail service, passengers flow, trains traffic etc.

5. Automatic Train Supervision (ATS) regulates the rail line traffic and reschedule

trains based on OCC information.

112

6. Zone Controller (ZC) prevents confliction in train movements and works as a

local interlocking system. Moreover, it connects via 802.3 protocol to the

remote I/O from which platform screen door (PSD) will be controlled based

on updated train position and its doors relative position toward the PSD.

7. As one major part of CBTC system, Data Communication System (DCS), sends

and receives bidirectional information wirelessly. This information includes

Station Reference Point (SRP) location to the iTASC as well as train relative

location towards SRP.

8. For controlling the train movement, iTASC also needs train speed. Odometry

system get real speed of train and send it to the iTASC via Train

Communication Network (TCN) which consists of Multi Vehicle Bus (MVB) and

Wired Train Bus (WTB).

9. This figure also shows Traction module as part of train throttling system, but

the amount of traction force is zero for the application of iTASC where train is

supposed to stop.

10. Braking force and running resistance are always negative as they are

resistance to train movement, but track profile depends on the track gradient.

It could be either positive in downhill track sections or negative in uphill track

sections and obviously zero when rolling stock is moving on a levelled track.

Chapter 5: iTASC 113

Traction OCC Wireless Nil Wired 802.11 802.3 V0 X0 ATSATS ZC Station DCS TASC Braking System - Train Model 1/s 1/s

Wired 802.3 - Running Resistance RI/O PSD ± Track Profile Figure 5-5: CBTC iTASC Closed Control Loop

In Figure 5-5, the procedure of iTASC algorithm has been shown. As can be seen in this figure, as train nearing the station, an AP send the signal to the train and shows that the train is entering a station. We have named it as Station Position (SP) and it will be marked using an access point. On-board computer continuously calculates the train location (TL) and compares it with station Referencing Point (SRP).

As long as the ATO realises that train is reaching to the speed of less than 20 km/h, it releases the DB and will hand over the control of the train to iTASC.

iTASC will set the braking distance based on train location, initial speed at HOP and other active parameters like track profile. We have chosen the acceleration updating rate to 0.05 which means at each attempt iTASK just allowed to change the acceleration rate by 5% of its current value, this helps the algorithm to guarantee the riding comfort level.

The agent in this study is train itself or train controller, which is being trained based on its own performance and getting reward or punishment from the environment, which is train station. During training period we had used 70% of the

114

available real data, from each action, the agent gets more and more experience and finally reaches to a level of performance that we satisfied with its performance, and used the 30% of the data to compare our model performance with the real data of train braking distance that comes from an experienced train driver performance.

Step 1: Initialising

Start Step 2: Choosing a braking rate

NO YES NO Step 3: Applying braking delay SP > TL > SRP 0 < V < 20

time YES

Step 4: Applying passenger a -

weight for each wagon Deactivate Dynamic Brake

Step 5: Applying brake Get Train Location (TL) Step 6: Measuring the Braking

YES TL=SRP Distance (BD) NO

Calculate Braking Step 7: Considering train Distance (BD)

position uncertainty YES Keep Current BD=TL-SRP Deceleration Step 8: Comparing BD with SRF NO NO BD > TL-SRP a a + 0.05 Step 9: Picking the YES

corresponding reward from a a –

Algorithm 5.1 End

Step 10: Updating the Q values

Step 11: Jump to Step 2

Figure 5-6: iTASC Procedure and Flowchart

Chapter 5: iTASC 115

5.6 SIMULATION RESULTS

In this section, simulation results of the iTASC algorithm has been shown for 4 different stations and they have been compared with the top two TASC benchmark algorithms discussed in the previous chapter; FC and SVR GK. Table 5.1 shows the parameters that have been considered in simulation.

Table 5-1 iTASC Simulation Parameters

# Parameter Description 1 Station curve Different amounts of curvature for different stations (near the minimum allowable curve for the last station, 215 m) 2 Station Slope/gradient Positive and negative slopes have been considered 3 Uneven distributed load of Formula 3-10 pp 48 train throughout the train Random 3 empty cars out of 24 has been put in simulation 4 Wind effect 32 km/h which is the average of wind speed in NSW has been considered for the last station 5 The effect of wet, dirty and Consider the coefficient of wet rail in the last oily rail station 푚 6 Comfort of passenger Ideal deceleration target is set to 1 ⁄푠3 7 Number of braking is limited to 1 8 Position uncertainty 10 cm random uncertainty has been added to the train position 9 Brake delay time 4 seconds

Stations 1, 2 and 4 are the same as those three stations mentioned in the previous chapter for TASC benchmark. As the curvature and gradient of station 3 in this section is more than those of stations 1, 2 and 3 in the previous chapter; 215 m, and also this station is located in an open area where the wet rail and wind effects are applied, we have saved this challenging situation for our new algorithm to deal with.

116

Figure 5-7: iTASC Algorithm, Station 1

Station Reference Point (SRP) in station 1, Figure 5-6, is located at 4250m and iTASC has stopped the train at 4249.991m. From the figure it is obvious that there is

푚 just one BA and the level of jerk is limited to -1.49 ( ⁄푠3).

In station 2, Figure 5-7, the algorithm shows relatively the same trend; one BA

푚 and jerk level of -1.74 ( ⁄푠3). The SRP is located at 6225 and stopping position of the train is 6249.884.

Chapter 5: iTASC 117

Figure 5-8: iTASC Algorithm, Station 2

Figure 5-9: iTASC Algorithm, Station 3

The stopping position in stations 3 and 4, Figures 5-8 and 5-9, are 7499.659m and 8549.957m respectively. We have applied the wet rail and wind effect (32km/h) in the iTASC algorithm for station 3. The station curvature, 215 m, in this station is close to minimum allowable amount of station curvature (190 m, Table: 3-4).

To make the situation even more complex, we have randomly put 3 empty cars out of 8 cars in train during simulation to simulate uneven distribution of passengers.

As can be seen we have experienced one BA (which directly related to the passenger

푚 comfort) in both stations and the jerk level is limited to -1.4 ( ⁄푠3) which shows an independent robust performance of iTASC.

Note: If train jerk is more than 2, this is similar to stopping a car, while the driver slams on the brake. If the jerk limited to 1-1.5 as it is recommended, but having

118

different braking attempts, it will be similar to stopping a car going downhill while frequently applying the brake rather than one smooth attempt. So, both jerk and braking attempt should be controlled to achieve passenger comfort as has been done in this research work.

Figure 5-10: iTASC Algorithm, Station 4

Figure 5-11: The Mean Square Error Averaged on 20 Folds of the Best Three Models

Chapter 5: iTASC 119

Figure 5-10 has summarised the simulation results of iTASC and has compared them with the top two algorithms in the previous chapter as the TASC benchmark;

Fuzzy control and SVR GK. We have already mentioned those algorithms as the TASC benchmark and here they give a clear picture of iTASC performance level.

Table 5-2 Summary of TASC Algorithms in Three Stations

Position Curve Gradient iTASC FC SVR GK ANN

Station 1 4250 3958 Levelled 4249.991 4249.309 4248.395 4247.218 m Station 2 6250 370 m Uphill / 65 6249.884 6224.738 6223.781 6223.606 cm Station 3 8550 250 m Downhill 8549.957 8549.711 8549.318 8548.850 /60 cm Station 1 Jerk -1.49 -1.4 -1.3 -1.3 Station 2 Jerk -1.74 -1.8 -1.6 -1.8 Station 3 Jerk -1.48 -3 -4 -5 Station 1 BA 1 11 13 12 Station 2 BA 1 10 9 5 Station 3 BA 1 17 10 9

5.7 CONCLUSION

The precise train stopping in stations allows a faster alighting and boarding procedure in railway stations and consequently improves railway headway as well as platform safety by enabling PSD in stations. In addition to safety, using PSD in stations will improve the station energy consumption for HVAC systems.

A new TASC algorithm known as iTASC has been developed in this chapter to improve the stopping error of train in stations as well as passenger ride comfort level. iTASC takes into account different key points in stopping a train in station under CBTC constraints such as position uncertainty, lack of wayside equipment as well as some important factors which are inherent in rail systems such as unevenly distributed

120

passenger over the trains’ cars, Braking delay time in rolling stock and different track profiles in each station.

The performance of the iTASC algorithm has been investigated using models in

Chapter 3 and the previous chapter’s benchmark which 7 functions in fuzzy system combined with double q-learning, the scenarios in the simulations are for three different stations to indicate the performance in levelled, uphill and downhill station.

It also used real data from 20 rolling stock sets with effective station and brake parameters. With comparing data in table 5-2, the stopping error in iTASC is 4% which shows an average of 24% improvement compared to the Fuzzy control algorithm while considering different active parameters in our model and 35.99% improvement over that of SVR GK.

The method well addresses metro limitations. A metro normally operates in higher speed and more crowded stations where railway operators are more interested to have ATO and PSD. The suburban trains follow fewer limitations in this respect and obviously iTASC could be implemented in them as well.

iTASC would be a main subsystem of ATO and driverless train which will handle the last segment of driving a train into a station and stop it.

Although it is not the scope of this thesis to deal with cost modelling and economic justification, considering all other complex system currently used as well as safety and interface limitations around them, it is not only economic, but all big rail companies desperately looking for a solution to make it happen.

Chapter 5: iTASC 121

122

Chapter 5: iTASC 123

Chapter 6: Conclusions

This chapter first provides a summary of the contributions of this thesis and discusses several promising avenues of future research.

6.1 SUMMARY

In a railway system, especially in a new control and signalling system where all the critical messages, commands and control signals transfer wirelessly, and the mechanical wayside equipment has reached to the minimum level, stopping the trains at the exact position in stations is one of the critical aspects of an efficient and punctual rail service. In passenger trains, precise stopping facilitates installation of

Passenger Screen Door (PSD) in stations. At the same time, precise stopping is of crucial importance for the freight trains as recently the general tendency in logistic industry is also entering a totally automated era, where most of the labour-intensive tasks are being assigned to fully automated systems. As an example, the automatic stacking cranes in rail yards as well as seaports are handling almost all of loading and unloading tasks.

Precise stopping also plays an important role in rail service safety, and as previously mentioned, it is considered as one of the enabling technologies for using

PSD in train stations which has a huge impact on passenger safety. In addition to this, precise stopping by itself can reduce the number of train accidents in busy stations and maintenance yards where trains move close to each other.

124

This thesis has developed a Train Automatic Stop Control (TASC) approach using artificial intelligence techniques that we have named it as Intelligent Train Automatic

Stop Control (iTASC). An efficient stopping algorithm for the communication-based train control (CBTC) without relying on Balises’ telegrams has been developed by a combination of three powerful AI approaches; Reinforcement learning, Double Q- learning and Fuzzy control system.

Chapter 5 introduces a closed control loop and an algorithm for measuring the braking distance and stopping error which considers different important factors for controlling trains and the braking procedure such as rolling stock braking delay and passenger distribution status of the train. Moreover, in this chapter a sophisticated mathematical approach leading to the new algorithm of train automatic stop control, iTASC, has been proposed. Fuzzy double q-learning shows 23% improvement in performance on the TASC benchmark which has been introduced in Chapter 4.

The performance of the new algorithm has been shown by developing a sophisticated benchmark in Chapter 4 for comparison. The benchmark is developed based on different algorithms; Linear Regression technique, Support Vector

Regression technique, Artificial Neural Network and Fuzzy control system.

Additionally, a great effort has been made to gather relative real-world rolling stock, braking system and track profile data for simulation and assessment of iTASC performance.

The first part of Chapter 3 of this work, has dealt with train dynamic behaviour modelling. Different active parameters which play important roles in rolling stock dynamic behaviour as well as braking and stopping procedure have been considered

Chapter 6: Conclusion 125

and to the best of our knowledge this combination of sophisticated train and braking system models is the first of its kind in TASC-related algorithm and train dynamic behaviour simulations.

In the second part of Chapter 3, a step-by-step mathematical method of modelling of train braking systems has been introduced. Using the effective parameters of rolling stock air brake and dynamic brake system in the modelling, leads the model through a performance similar to an experienced train driver. This makes the model reliable for assessing the iTASC performance.

6.2 THE MAIN CONTRIBUTION

In this section we summarise the main contributions of this thesis as follows:

1- We have trained a sophisticated model for ATO and TASC simulation purposes

2- The detailed track profile of train stations has taken into account in stopping procedure

3- The algorithm shows a smooth stopping profile and guaranties the passenger comfort during brake period

4- The algorithm has introduced a strong capability of reinforcement learning in solving rail transport problems

5- The introduced algorithm fills the gaps between Double Q-learning and Fuzzy sets and paves the way for the future deep learning approaches in similar or other closely related application domains.

6.3 FUTURE RESEARCH DIRECTIONS

The recommendations below are identified during developing the new control algorithm and modelling phase of this study for future works.

126

6.3.1 Merging the iTASC algorithm with CBTC ATO As mentioned in the previous chapters, one of the main issues in driverless trains which use fully automated operation systems known as autonomous trains, is precise stopping in the stations which is the main focus of this study, but this algorithm must be applicable in a full operational scenario of trains and not just in stations. In this study we just focused on last section of train speed profile; stopping.

Therefore, next step in this area of research could potentially be designing a CBTC

ATO algorithm which embodies the iTASC. The CBTC ATO algorithm must contain the iTASC procedures, interface with it and control its activations and deactivations period.

6.3.2 Optimal Train Speed Profile For the iTASC algorithm, the efficiency of speed profile must be guaranteed from different perspective including, service punctuality, energy consumption, passenger comforts as well as precise stopping at the stations.

6.3.3 Deep Learning (DL) Approaches Deep learning (DL) techniques are usefully applied to lots of practical problem these days especially in the field of automatic train control (ATO). The developed models in Chapter 3 for train and braking system have shown a highly accurate performance. Hence, applying deep learning algorithm to the introduced models can be considered to investigate deep learning performance in TASC problem provided that there is a big data base.

6.3.4 Multi Agents System (MAS) Q-learning iTASC In this study and while we had applied reinforcement learning, we considered the train or its TASC system as a single intelligent agent while for the algorithm in 5-

Chapter 6: Conclusion 127

4, when applying to the entire rail line for different trains or even throughout the rail network, using multi-agents system or cooperative control system while applying Q- learning or Double Q-learning could potentially be an effective method of controlling the fleet.

6.3.5 Platform Screen Doors (PSD) Alignment One of the main reasons of considering the train precise stopping as a crucial characteristic of a modern railway control and signalling system is the need for utilising PSD in crowded stations. Currently, train stopping, and PSD are being controlled by different control systems which should consider several interfaces in their design, and they are interlocked. However, these two control regimes can be integrated into a single control algorithm which reduce the probability of conflicting control commands.

6.3.6 TASC Time and Energy Evaluation As can be seen in Table 2.1, TASC algorithms have been largely ignored in terms of their processing time and energy saving benefits. The former can be studied in the area of the train software configuration or train control and management system

(TCMS) and the later could be seen as part of the entire train movement energy consumption.

6.3.7 Next Generation of Train Control (NGTC) Systems The future of train control and signalling system is a sophisticated and unified signalling system which combines different available technologies; CBTC, ETCS, and

Internet of Smart Trains (IoST). The ATO system in NGTC will be more sophisticated and this new trend will open new research areas for train stopping procedure.

128

Chapter 6: Conclusion 129

Bibliography

[1] IEEE 1474.1™, "Standard for Communication-Based Train Control (CBTC), Performance and Functional Requirements". Available from: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1405808, 2004. [2] H. Oshima, S. Yasunobu and SH. Sekino, "Automatic Train Operation System Based on Predictive Fuzzy Control". International Workshop on Artificial Intelligence for Industrial Applications, pp. 485-489, 1988. [3] K. Yoshimoto, K. Kataoka and K. Komaya, "A Feasibility Study of Train Automatic Stop Control Using Range Sensor", IEEE Intelligent Transportation Systems Conference Proceedings, pp. 802-807, 2001. [4] S. Yasunobu, S. Miyamoto and H. Ihara, "A Fuzzy Control for Train Automatic Stop Control," society of instrument and control engineers, transactions on, vol. e-2, no. 1, pp. 1-9, 2002. [5] A. Siahvashi and B. Moaveni, "Automatic Train Control Based on the Multi- Agent Control of Cooperative Systems", The Journal of Mathematics and Computer Science, vol. 1, no.4, pp.247-257, 2010. [6] Z. Hou et al., " Terminal Iterative Learning Control Based Station Stop Control of a Train," International Journal of Control, vol. 84, pp. 1263-1274, 2011. [7] D. Chen et al., " Online Learning Algorithms for Train Automatic Stop Control Using Precise Location Data of Balises," IEEE Transactions on Intelligent Transportation Systems, vol. 14, no. 3, pp. 1526-1535, 2013. [8] L. Ma and X. Zeng, "Analysis on Train Stopping Accuracy based on Regression Algorithms," Journal of Software, vol.9, no. 5, 2014. [9] W. Peng and W. Qingyuan, "Research of the Automatic Train Control Based on Adaptive generalized Predictive Control," Proceedings of the 33rd Chinese Control Conference, pp. 3399-3404, 2014. [10] Y. Bai et al., "Station Stopping of the Freight Trains with Pneumatic Braking," Hindawi Publishing Corporation, pp. 1-7, 2014. [11] S. Aum, "Modeling and Precision Stop of Metropolitan Train with Receding Horizon Control," Master Thesis, 2015. [12] D. Chen, L. Wang and L. Li, "Position Computation Models for High-Speed Train Based on Support Vector Machine Approach," Applied Soft Computing, no. 30, pp. 758-766, 2017. [13] M. Ha, H. Kim and C. Kang, "A precision Stopping Measurement Device to Automatically Detect Position Errors of an Urban Train at Railway Stations," International Journal of Control, Automation and Systems, vol. 15, pp. 848- 856, 2017. [14] S. Yasunobu, S. Miyamoto and H. Ihara, "Fuzzy Control For Automatic Train Operation System," Control in transportation System, pp. 33-39, 1983. [15] B. Aguiar et al., "A Robust and Fault Tolerant Approach for Automatic Trian Stop Control System Design," IFAC Paper On Line, vol. 50, no. 1, pp. 8549- 8554, 2017.

Bibliography 131

[16] A. Oda, I. Niimi and K. Maki, "Development of Automatic Parameter Tuning for Train Automatic Stop Control Device," Electronicx and Communications in Japan, vol. 100, no. 11, pp. 629-634, 2017. [17] W. G. Temple et al., "On Train Automatic Stop Control Using Balises: Attack and Software-Only Countermeasure," 22nd IEEE Pacific Rim International Symposium, pp. 274-283, 2017. [18] B. Aguiar et al., "Improving Train Position Accuracy in Case of Wheel Jamming Faults," IFAC Papers On Line, vol. 51, no. 26, pp. 25-30, 2018. [19] Y. WU et al., "Position Manipulation Attacks to Balise-Based Train Automatic Stop Control," IEEE Transactions on Vehicular Technology, vol. 67, no. 6, pp. 5287-5301, 2018. [20] X. Wang, T. Tang and H. He, "Optimal Control of Heavy Haul Train Based on Approximate Dynamic Programming," Advances in Mechanical Engineering, Vol. 9, no. 4, pp. 1-15, 2017. [21] J. Yin et al., "Research and Development of Automatic Train Operation For Railway Transportation Systems: A Survey," Transportation Research Part C 85, 548-572, 2017. [22] EN 13803-1, European Standard, "Railway Application - Track - Track alignment design parameters - Track gauges 1435 mm and wider - Part 1: Plain line,", 2010. [23] D. Chen et al., " Position computation models for high-speed train based on support vector machine approach," Applied Soft Computing, vol. 30, pp. 758- 766, 2015. [24] S. Yi, "Principles of Railway Location and Design," Academic Press, 2018. [25] D. Barney, D. Haley and G. Nikandros, "Calculation Train Braking Distance," 6th Australian Workshop on Safety Critical Systems and Software, 2011. [26] H. A. Ahmad, "Dynamic Braking Control for Accurate Train Braking Distance Estimation Under Different Operating Conditions," PhD thesis, Virginia Polytechnic Institute and State University, 2013. [27] B. Ning, "Advanced Train Control Systems," WIT Press, 2010. [28] C. C. William, "Efficient Driving of CBTC ATO Operated Trains," PhD Thesis, Delft University of Technology, 2017. [29] A. Guastafierro et al., "A Method for Optimizing Coasting Phase in Railway Speed Profiles: an Application to an Italian Route," IEEE International Conference on Electrical Systems for Aircraft, Railway, Ship Propulsion and Road Vehicles & International Transportation Electrification Conference (ESARS-ITEC), pp. 1-7, 2016. [30] J. P. Powell and R. Palacin, "Passenger Stability Within Moving Railway Vehicles: Limits on Maximum Longitudinal Acceleration," Urban Rail Transit, pp. 95-103, 2015. [31] Transport for NSW, Train Patronage - Monthly Figures, Monthly Trips by Operators and by line. Available from: https://www.transport.nsw.gov.au/data-and-research/passenger- travel/train-patronage/train-patronage-monthly-figures. [32] M. Spiryagin et al., "Design and Simulation of Rail Vehicles," CRC Press, Taylor & Franics Group, 2014.

132

[33] Sh. SAGA et al., "An Evaluation Method of the Braking Performance Using Bogie Traction Force," QR of RTRI, vol. 56, no. 4, pp. 256-261, 2015. [34] D. Hijikata and T. Nishimura, "Braking Force Estimation of Each Car," QR of RTRI, vol. 57, no. 2, pp. 98-104, 2016. [35] S. Iwnicki, "Handbook of Railway Vehicle Dynamics 09 - Longitudinal Train Dynamics," CRC Press, Taylor & Francis, 2006. [36] W. Wei et al., "An air brake model for longitudinal train dynamics studies," International Journal of Vehicle Mechanics and Mobility, Vehicle System Dynamics, vol. 55, no. 4, pp. 517-533. [37] Q. Wu et al., "Railway Air Brake Model and Parallel Computing Scheme," Journal of Computational and Nonlinear Dynamics, vol. 12, pp. 051017-1-11, 2017. [38] W. Wei, M. Ahmadian and J. Zhang "Heavy Haul Train Simulation of Air Brake System and Longitudinal Dynamics," Te American Society of Mechanical Engineering, Joint Rail Conference, 2014. [39] J. Anuszczyk et al., "Generating and Modeling of Braking Curve and the Assessment of the Quality of Automatic Stopping of the Train," Transport Problems, vol. 13, no. 2, pp. 13-25 2018. [40] Railway Gazette, "Rio Tinto operates first fully-autonomous test train," Available from: https://www.railwaygazette.com/news/single- view/view/rio-tinto-operates-first-fully-autonomous-test-train.html, 2017. [41] IEEE 1698™, "Guide for the Calculation of Braking Distances for Rail Transit Vehicles" IEEE Vehicular Technology Society (VTS). Available from: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5332051, 2009. [42] Sh. Huang and Sh. Her, "Fuzzy Control of Automatic Train Operation System," International Journal of Modelling and Simulation, vol. 17, no. 2, pp. 143-150, 1997. [43] Y. Wang et al., "Survey on Driverless Train Operation for Urban Rail Transit Systems," Urban Rail Transit, vol. 2, pp. 106-113, 2016. [44] X. Wang et al., Intelligent operation of heavy haul train with data imbalance: A Machine learning method, Knowledge-Based Systems, no. 163, pp. 36-50, 2019. [45] Sh. Gao et al., "Automatic Train Operation of High-Speed Train with State Constraints: A Nested Proportion-like Control Approach," Chinese Automation Congress (CAC), pp. 2884-2888, 2018. [46] Y. Liang et al., "A Modified Genetic Algorithm for Multi-Objective Optimization on Running Curve of Automatic Train Operation System Using Penalty Function Methos," International Journal of Intelligent Transportation Systems Research, vol. 17, pp. 74-87, 2019. [47] M. Dominguez et al., "Multi objective particle swarm optimization algorithm for the design of efficient ATO speed profiles in metro lines," Engineering Application of Artificial Intelligence, vol. 29, pp. 43-53, 2014. [48] S. Sekine et al., "Application of fuzzy neural network control to automatic train operation and tuning of its control rules," IEEE Fuzzy Systems conference, 1995, pp. 1741-1746.

Bibliography 133

[49] Y. Cao, L. Ma and Y. Zhang, "Application of fuzzy predictive control technology in automatic train operation," Cluster Computing, Available from: https://doi.org/10.1007/s10586-018-2258-0, 2018. [50] Sh. Su et al., "A Subway Train Timetable Optimization Approach Based on Energy-Efficient Operation Strategy," IEEE Transaction on Intelligent Transportation Systems, vol. 14, no. 2, pp. 883-893, 2013. [51] Sh. Lu et al., "Single-Train Trajectory Optimization," IEEE Transaction on Intelligent Transportation Systems, vol. 14, no. 2, pp. 743-750, 2013. [52] SH Han et al., "An Optimal Automatic Train Operation (ATO) Control Using Genetic Algorithms (GA)," IEEE TENCON, pp. 360-362, 1999. [53] H. Oshima, S. Yasunobu and Sh. Sekino, "Automatic Train Operation System Based on Predictive Fuzzy Control." International Workshop on Artificial Intelligence for Industrial Applications, 1988, pp. 485-4897. [54] C.S. Chang and D.Y. Xu, "Differential evolution-based tuning of fuzzy automatic train operation for mass rapid transit system," IEE proceeding, Electric Power Application, vol. 147, no. 3, pp. 206-212, 2000. [55] C.S. Chang, D.Y. Xu and H.B. Quek, "Pareto-optimal set based multi-objective tuning of fuzzy automatic train operation for mass transit system," IEE proceeding, Electric Power Application, vol. 146, no. 5, pp. 577-583, 1999. [56] W. Jing, C. Zixing and J. Limin,"Direct Fuzzy Neural Control with Application to Automatic Train Operation," Control Theory and Application, vol. 15, no. 3, pp. 391-399, 1998. [57] S. Sekine et al., "Application of fuzzy neural network control to automatic train operation and tuning of its control rules," IEEE Fuzzy Systems conference, 1995, pp. 1741-1746. [58] G. Bing, D. Hairong and Zh. Yanxin, "Speed Adjustment Braking of Automatic Train Operation System based on Fuzzy-PID Switching Control," IEEE Computer Society, Sixth International Conference on Fuzzy Systems and Knowledge Discovery, pp. 577-580, 2009. [59] Y. Wang, Zh. Hou and X. Li, "A novel automatic train operation algorithm based on iterative learning control theory," IEEE International Conference on Service and Logistics, and Information, pp. 1766-1770, 2008. [60] Sh. Gao et al., "Approximation-Based Robust Adaptive Automatic Train Control: An Approach for Actuator Saturation," IEEE Transaction on Intelligent Transportation Systems, vol. 14, no. 4, pp. 1733-1742, 2013. [61] J. Yin et al., "Intelligent Train Operation Algorithms for Subway by Expert System and Reinforcement Learning," IEEE Transaction on Intelligent Transportation Systems, vol. 15, no. 6, pp. 2561-2571, 2014. [62] H. Dong et al., "Automatic Train Control System Development and Simulation for High-Speed Railways," IEEE Circuits and System Magazine, vol. 10, no. 2, pp. 6-18, 2010. [63] H. Dong et al., "Extended Fuzzy Logic Controller for High Speed Train," Neural Computing and Applications, vol. 22, no. 2, pp. 321-328, 2011. [64] M. Ganesan, D. Ezhilarasi and J. Benni, "Hybrid model reference adaptive second order sliding mode controller for automatic train operation," IET Control Theory & Applications, vol. 11, no. 8, pp. 1222-1233, 2017.

134

[65] C. Lin et al., "Study on Energy-Saving Optimization of Train Coasting Control Based on Multi-Population Genetic Algorithm," 3rd IEEE International Conference on Control, Automation and Robotics, pp. 627-632, 2017. [66] D. Yazhemsky, M. Rashid and Sh. Sirouspour, "An On-Line Optimal Controller for a Commuter Train," IEEE Transaction on Intelligent Transportation Systems, vol. 20, no. 3, pp. 1112-1125, 2019. [67] G. Matsuura and M. Miyatake, "Optimal train speed profiles by dynamic programming with parallel computing and the fine-tuning of mesh," WIT Transaction on The Built Environment, vol. 135, pp. 767-777, 2014. [68] T. Albrecht, A. Binder and Ch. Gassel, "Application of real-time speed control in rail-bound public transportation systems," IET Intelligent Transportation Systems, vol. 7, no. 3, pp. 305-314, 2013. [69] L. Baranov et al., "Automatic Train Control Algorithms with Regulation Restrictions Adaptive to System State Changes," International Journal of Engineering Research & Science, vol. 2, no. 4, pp. 84-96, 2016. [70] A. Amrani et al., "Train speed profiles optimization using a genetic algorithm based on a random-forest model to estimate energy consumption," Proceeding of 7th Transportation Research Arena TRA, 2018. [71] M. Kang, "A GA-base Algorithm for Creating an Energy-Optimum Train Speed Trajectory," Journal of International Council on electrical Engineering, vol. 1, no. 2, pp. 123-128, 2011. [72] V. Calderaro et al., "An algorithm to optimization speed profiles of the metro vehicles for minimizing energy consumption," IEEE International Symposium on Power Electronics, Electrical Drives, Automation and Motion, pp. 813-819, 2014. [73] V. Calderaro et al., "Deterministic vs heuristic algorithm for eco-driving application in metro network," IEEE International Conference on Electrical Systems for Aircraft, Railway , Ship Propulsion and Road Vehicles (ESARS), 2015. [74] S. Gupa, P. Mahajan and R. Garg, "Tractive Energy Optimization in Railway Electric Traction System," 1st IEEE International Conference on Power Electronics; Intelligent Control and Energy System (ICPEICES), pp. 1-5, 2016. [75] Y. Liang et al., "A Modified Genetic Algorithm for Multi-Objective Optimization on Running Curve of Automatic Train Operation System Using Penalty Function Method," International Journal of Intelligent Transportation Systems Research, vol. 17, pp. 74-87, 2019. [76] F. Gao et al., "Study on Multi-Objective Intelligent Speed Controller Model of Automatic Train Operation for High speed Train Based on Grey System Theory and Genetic Algorithm," 11th IEEE International Symposium on Computational Intelligence and Design, pp. 189-194, 2018. [77] R. Cheng et al., "Intelligent driving methods based on sparse LSSVM and ensemble CART algorithm for high-speed trains," Journal of Computers & Industrial Engineering, vol. 127, pp. 1203-1213, 2019. [78] P. Mo, L. Yang and Z. Gao, "Energy-Efficient Train Operation Strategy with Speed Profiles Selection for an Urban Metro Line," Transportation Research Record, pp. 1-13, 2019.

Bibliography 135

[79] X. Wang et al., "Intelligent operation of heavy haul with data imbalance: A machine learning method," Journal of Knowledge-Based Systems, vol. 163, pp. 36-50, 2019. [80] C. Jia, H. Xu and L. Wang, "Nonlinear Hybrid Multipoint Model of High-Speed Train with Traction/Braking Dynamic and Speed Estimation Law," Hindawi Mathematical Problems in Engineering, Available from: https://doi.org/10.1155/2019/1364657, 2019. [81] Sh. Su, T. Tang and C. Roberts, "A Cooperative Train Control Model for Energy Saving," IEEE Transaction on Intelligent Transportation Systems, vol. 16, no. 2, pp. 622-631, 2015. [82] Y. Bai et al., "Cooperative Control of Train to Minimize Net Energy Consumption," IEEE Transaction on Intelligent Transportation Systems (Early Access), pp. 1-15, 2019. [83] X. Yang et al.' "A Survey on Energy-Efficient Train Operation for Urban Rail Transit," IEEE Transaction on Intelligent Transportation Systems, vol. 17, no. 1, pp. 2-13, 2016. [84] G. Xi et al., " A Hierarchical Ensemble Learning Framework for Energy-Efficient Automatic Train Driving," Tsinghua Science and Technology, vol. 24, no. 2, pp. 226-237, 2019. [85] W. Lin and J. Sheu, "Optimization of Train Regulation and Energy Usage of Metro Lines Using an Adaptive-Optimal-Control Algorithm," IEEE Transactions on Automation Science and Engineering, vol. 8, no. 4, pp. 855-864, 2011. [86] J. Yin, D. Chen and L. Li, "Intelligent Train Operation Algorithms for Subway by Expert System and Reinforcement Learning," IEEE Transaction on Intelligent Transportation Systems, vol. 15, no. 6, pp. 2561-2571, 2014. [87] R. Sutton and A. Barto, "Reinforcement Learning; An Introduction," Second Edition, The MIT Press, 2018. [88] H. Khadilkar, "A Scalable Reinforcement Learning Algorithm for Scheduling Railway Lines," IEEE Transaction on Intelligent Transportation Systems, vol. 20, no. 2, pp. 827-736, 2019. [89] Y. Li et al., "Intelligent Trainer for Model-Based Deep Reinforcement Learning," Available from: arXiv:1805.09496v5, 2019. [90] S. Yasunobu and T. Matsubara, "Fuzzy Target Acquired by Reinforcement Learning for Parking Control," SICE Annual Conference, pp. 1242-1247, 2003. [91] W. Lin and J. Sheu, "Metro Traffic Regulation by Adaptive Optimal Control," IEEE Transaction on Intelligent Transportation Systems, vol. 12, no. 4, pp. 1064-1073, 2011. [92] QU. Jianwei et al., "Optimization for High-Speed Train Operation Considering Driving Strategies and Timetable," Proceedings of the 32nd Chinese Control Conference, pp. 8167-8171, 2013. [93] Z. Jiang et al., "Reinforcement Learning Approach for Coordinated Passenger inflow Control of Urban Rail Transit in Peak Hours," Journal of Transportation Research Part C, vol. 88, pp.1-16, 2018. [94] B. Collins, "Pilbara miners call on drivers not to dice with 35,000-tonne trains that take a kilometre to stop," Available from: https://www.abc.net.au/news/2015-08-12/pilbara-iron-ore-train-railway- safety/6688578, 2015.

136

[95] C. Watkins and P. Dayan, "Q-Learning: Technical Note," Kluwer Academic Publishers, Machine Learning, vol. 8, pp. 279-292, 1992. [96] H. Hasselt, A. Guez and D. Silver, "Deep Reinforcement Learning with Double Q-Learning," Proceedings of the thirtieth AAAI Conference on Artificial Intelligence, pp. 2094-2100, 2016. [97] C. Watkins, "Learning from Delayed Rewards," PhD thesis, King's College, 1989. [98] H. Hasselt, "Double Q-learning," Neural Information Processing Systems Conference, NIPS, 2010. [99] P. Kofinas, A.I. Dounis and G.A. Vouros " Fuzzy Q-Learning for multi-agent decentralized energy management in microgrids," Journal of Applied Energy, vol. 219, pp. 53-67, 2018. [100] P. Glorennce and L. Jouffe, "Fuzzy Q-Learning," Proceeding of 6th International Fuzzy Systems Conference, 1997. [101] H. Oshima, S. Yasunobu and S. Sekino, "Automatic Train Operation System Based on Predictive Fuzzy Control," IEEE International Workshop on Artificial Intelligence for Industrial Applications, pp. 485-489, 1988. [102] L. Ma and X. Zeng, "Analysis on Train Stopping Accuracy based on Regression Algorithms," Journal of Software, vol. 9, no. 5, pp. 1237-1244, 2014. [103] I. Hasegawa and S. Uchida, "Braking Systems," Railway Technology Today 7, Japan Railway & Transport Review 20, pp. 52-59, 1999. [104] V. Vapnik et al., "Support Vector Method for Function Approximation, Regression Estimation, and Signal Processing," Advanced in Neural Information Processing Systems (NIPS) 9, pp. 281-287, 1996. [105] W. Lu," Neural Network Model for Distortion Buckling Behaviour of Cold- Formed Steel Compression Members," Helsinki University of Technology Laboratory of Steel Structures Publications 16, Available from: https://research.aalto.fi/en/publications/neural-network-model- for%20distortional-buckling-%20behaviour-of-coldformed-steel- compression%20members%20(68babc32-7429-410e-95e5- 95595ac83742).html, 2000. [106] Rail Industry Safety and Standard Board (RISSB)," Section 13: LOCOMOTIVES ". pp. 52.

Bibliography 137