FEATURED ARTICLES Latest Developments for Safe and Reliable Railways

Development of Traff ic Management System for Tohoku Main Line and Using Virtualization

Hitachi adopted virtualization for a traff ic management system for the first time when it undertook the integration of the systems for the Tohoku Main Line and Senseki Line. Along with hardware consolidation, the systems were integrated in a way that satisfied the customer’s requirements, including the use of multiple guests on the virtualization platform to enable an earlier commencement of operation on the Senseki Line so as to minimize the risks associated with switching to the new system, and the provision of a training environment in a timely manner with respect to the development schedule. The realtime virtualization platform adopted for the project delivered the realtime control performance, high availability, and ease of maintenance demanded of a traff ic management system. Configuring the system on a virtualization platform also helps minimize life cycle costs by making it possible to reduce the amount of development work needed during future equipment upgrades.

Kazuho Nakajima Noriyuki Kikuchi Yoshitaka Ishikawa Sadao Niitsuma Takeshi Takebayashi Takuya Tsujikawa

runs from Station on the Tohoku Main Line to Station on the Senseki Line. As the 1. Introduction Senseki Line runs on direct current (DC) electric power whereas the Tohoku Main Line runs on alter- Th e Tohoku Main Line (from Kuroiso to Ishikoshi) nating current (AC), the service was provided using and the Senseki Line (from Aoba-dori to Rikuzen- HB-E210 series hybrid diesel rolling stock. As the Yamashita) each had their own control centers and traffi c management systems being independent of East Railway Company had established sepa- each other imposed restrictions that obstructed the rate traffi c management systems for the lines, with no adoption of fully automatic control, including the facility for the sharing of track. inability of the previous systems to share train number An interconnecting line between the Tohoku Main information, trains needed to halt on the connecting Line and Senseki Line was completed in May 2015 line and the control center had to use manual proce- and the new Senseki-Tohoku Line introduced that dures for some actions.

76. Figure 1 — System Scope

The Tohoku Main Line and Senseki Line each [Tohoku Main Line] had their own traff ic management system. Train Ban-etsu West Line Riku-East Line

services that operated under both systems com- menced in May 2015 with the opening of the Senseki-Tohoku Line. Natori Sendai Iwakiri Kogota Ishikoshi (Kuroiso) Iwanuma Koriyama Tsukinoki Fukushima Nagamachi Higashi-Sendai Shin-Shirakawa Asaka-Nagamori Ban-etsu Abukuma Joban Sendai Ishinomaki East Line Express Line Line Airport Line Suigun Access Line Tohoku Freight Line Line

Senseki-Tohoku Line [Connecting lines] commenced operation in May 2015

Senseki-Tohoku Line An interconnecting line was added linking the Tohoku Main Line between Shiogama and Matsushima stations and the Senseki Line between Matsushima-Kaigan and Takagimachi Tagajo stations. Using this new line, services commenced Sendai Aoba-dori Takagimachi on the Senseki-Tohoku Line between Sendai (Ishinomaki)

Station on the Tohoku Main Line and Ishinomaki Higashi-Shiogama Rikuzen-Yamashita Matsushima-Kaigan Station on the Senseki Line.

[Senseki Line] Senseki Freight Line

With both systems coming due for replacement, a adoption of the virtualization platform means a new traffi c management system that integrated opera- training environment can be provided that includes tion on the Tohoku Main Line and Senseki Line was PRC and closely approximates actual operation. To installed to achieve time savings from use of pas- minimize the switchover risks associated with the sage control and full automation of the connecting system integration, the project also included a staged line (see Figure 1). As the traffi c management system switchover to the new system, with guests intended needed to provide scope for expansion as well as sus- for future use being utilized to commence operation tainable long-term operation with high reliability, the on the Senseki Line fi rst. new system was implemented using information and control servers running on a realtime virtualization 2. 1 platform with enhanced realtime control performance System Configuration for programmed route control (PRC) so as to reduce As control center operation was to continue as before, life cycle cost, including the cost of future upgrades. the confi guration was made up of a single system with Th is article describes the features of the realtime two control centers. Th e main equipment for PRC virtualization platform and its use for the traffi c man- and centralized traffi c control (CTC) was located agement system. at the Tohoku Main Line control center, with the Senseki Line control center having control desks only. Th ese were connected via a network running between 2. System Overview the centers (see Figure 2). Compared to the standalone systems, system Th e information and control servers used for the integration reduced the number of devices required upgrade are able to host virtual machines (“guests”) for shared equipment such as the monitoring desks, for up to three systems. Th ese are the integrated sys- scheduling system, and train traffi c monitoring server tem for the Tohoku Main Line and Senseki Line, that are not associated with particular sections of track, the training system, and a system to be used for new thereby saving both energy and space. Consolidating sections of track in the future. all of the key equipment in the same place also simpli- Whereas the previous training system lacked PRC fi ed maintenance. and was only used for teaching console operation,

Hitachi Review Vol. 67, No. 7 840–841 77. Figure 2 — Block Diagram of Traff ic Management System for Tohoku Main Line Senseki Line Tohoku Main Line control center control center and Senseki Line The CTC, control desks, and PRC are connected to Control desk Scheduling Training Control desk system desk the control system LAN and the control desks and PRC are connected to the information system LAN. Each LAN has a redundant dual-network configuration.

Tohoku TTS Inter- Information Senseki Unused Training system LAN control- PRC OS OS center Control network system LAN Virtualization platform Physical server Monitoring Train traffic desk monitoring CTC server (Tohoku Main Line) (Senseki Line)

CTC network

Traffic status CTC station display unit equipment

PRC: programmed route control CTC: centralized traff ic control TTS: train traff ic simulator OS: operating system LAN: local-area network

2. 2 (1) Confi guration of system for early commencement System Configuration and Operation Using of operation on Senseki Line Virtualization Platform A network confi guration in which guest 2 operated Use of a virtualization platform meant that the risks independently was used and the Senseki Line control associated with switching over to the integrated sys- desk and Senseki Line CTC connected. Th is enabled tem could be minimized by confi guring the parts of monitoring test runs to be conducted for the Senseki the system intended for the Senseki Line on unused Line system in parallel with the integrated system. guests and bringing these into service before the rest Also, the training desk was temporarily installed at of the system (see Figure 3). the Senseki Line control center to allow preparatory

Figure 3 — Operation of Virtualization Guests The Senseki Line part of the system commenced operation using what will ultimately be unused guest 2 (intended for future use). This was done to minimize the risks during switchover, with it being possible to switch over to the integrated system by switching the network connections.

(1) Configuration of system for early (2) Early commencement of (3) Commencement of operation commencement of operation on Senseki Line operation on Senseki Line of integrated system Separation of networks for guests 1 and 2

Tohoku Main Line Senseki Line Tohoku Main Line Senseki Line Tohoku Main Line Senseki Line

Network separation Network PRC and connection PRC PRC connected Guest 1 Guest 2 Guest 3 Guest 1 Guest 2 Guest 3 Guest 1 Guest 2 Guest 3

Tohoku Senseki Training Tohoku Senseki Training Tohoku Unused Training Senseki equipment Senseki equipment Senseki equipment

Monitoring Monitoring Senseki Monitoring In use Tohoku In use Guest Tohoku test run test run training test run training unused training

OS OS OS OS OS OS OS OS OS

Virtualization platform Virtualization platform Virtualization platform Physical server Physical server Physical server

Network separation Network CTC CTC and connection CTC CTC CTC CTC connected

Tohoku Senseki Tohoku Senseki Tohoku Senseki Main Line Line Main Line Line Main Line Line

78. FEATURED ARTICLES

Figure 4 — Use of Virtualization Guests Previously unused guests can be used to conduct monitoring test runs for major Monitoring test runs can be Reduce cost of upgrade to future system enhancements. The amount of modification conducted for major enhancements work required for future upgrades is reduced because applications are able to be migrated Current New An image of the guest system can as objects. software software be copied as-is to the new hardware OS OS PRC Online desk Virtualization platform Section Section Training Section A B C A Unused Unused OS OS OS OS

PGW Virtualization platform Virtualization platform CTC Physical server Physical server CTC Test desk display data Virtualized PRC Virtualized PRC (current hardware) (future hardware)

PGW: parallel gateway

training to be conducted for control of the Senseki 3. Features of Realtime Line. Virtualization Platform (2) Early commencement of operation on Senseki Line 3. 1 Th e parts of the system intended for the Senseki Ensuring Realtime Control Performance when Line (guest 2) went into service before the rest of the Executing on Virtual Machines system, commencing operation in November 2017, Th e realtime control performance requirements of the with testing of the integrated system being conducted information and control servers include predictabil- in parallel. Th e training system was installed at the ity (that operations are executed in the deterministic Tohoku Main Line control center to provide an envi- order and with predictable results), timeliness (that ronment for traffi c control training. operations commence at the required time intervals), (3) Commencement of operation of integrated system and low latency (the time taken for the process asso- Operation of the integrated system in its fi nal form ciated with a requested operation to actually start commenced in August 2018 with the switchover of executing). A problem with the server virtualiza- the inter-control-center network and other associ- tion used for general-purpose information systems ated work. is that it is subject to variable latency due to delays in software execution by the virtual machines. Th is 2. 3 occurs due to the confl icts that arise when multiple Future Use of Virtualization Platform virtual machines want to execute simultaneously, and In addition to their use for future branch line addi- as a result of the server virtualization platform using tions or for other systems, unused guests can also be software on the virtual machines to emulate access to used for monitoring test runs during major system physical hardware. enhancements to reduce the cost and improve the Accordingly, the realtime virtualization platform quality of on-site testing (see Figure 4). software used for the traffi c management system has Furthermore, past system upgrades have required a a resource allocation mechanism that provides vir- lot of time and eff ort to be spent on the migration of tual machines with exclusive use of the processors, applications as a consequence of hardware and operat- disk, and network hardware they are using. It uses ing system life cycles. Developing and implementing this mechanism to eliminate confl icts between the the new system on a virtualization platform, in con- execution of virtual machines running simultane- trast, is intended to reduce the amount of develop- ously. Similarly, the latency of the software running ment work required when upgrading to new hardware. on the virtual machines is kept within a certain level

Hitachi Review Vol. 67, No. 7 842–843 79. Figure 5 — Ensuring Realtime Control Performance The diagram shows the possible benefits of the resource allocation mechanism of the realtime virtualization platform. While delays in the completion of execution by soft ware A due to reasons (1) and (2) occur in the timechart at the top-right, the execution delay for the soft ware A becomes constant when the resource allocation mechanism is used, as shown in the timechart at the bottom-right.

Execution flow chart Time

Execution delay Realtime virtualization servers Virtual Software for control applications machine 1 A (1) Delay due Virtual Software to conflict Virtual Virtual machine 2 B machine 1 machine 2 with other processing Server virtualization (2) Delay due to Realtime Software Software platform virtualization virtualization A B platform hardware processing Resource OS OS Resource allocation allocation mechanism adopted mechanism Execution delay made constant Virtual Software Hardware machine 1 A resources (1) No Virtual Software execution machine 2 B conflict

Partition hardware resources used by the Realtime virtualization (2) Can execute virtual machines to minimize delays due to platform in parallel execution conflicts, etc.

by executing this software on diff erent processor cores Th e way rapid switchover was achieved in the past to those used for the realtime virtualization platform was to assign a high priority to the monitoring of itself. Th is overcomes the problem of variable latency each system by the other so as to prevent misdetection when using server virtualization and, together with due to execution delays. Th is enables fast and reliable other measures for ensuring predictability and timeli- switchover when a fault is detected by using the com- ness, ensures that the virtual machines deliver realtime puting hardware’s reset mechanism to shut down the control performance (see Figure 5). faulty hardware. Th e realtime virtualization platform computing hardware also assigns a high priority to 3. 2 monitoring of each system by the other and uses the Ensuring High Availability of Virtual Machines reset mechanism for rapid switchover. While there is Th e method used to improve availability on past a potential when virtual machines monitor each other information and control systems was to use multiple for faults to be misdetected due to an accumulation of information and control servers (redundancy) so that execution delays and it is not possible to shorten the continuity of execution could be retained in the event monitoring times, the realtime virtualization platform of a fault by rapidly switching over executing software achieves faster fault detection because it is confi gured to diff erent computing hardware. Likewise, when to perform monitoring directly on the same comput- using virtualization, high availability is achieved by ing hardware as the virtual machines. It is equipped adopting a redundant confi guration for the computing with a mechanism whereby, when a fault occurs on hardware that hosts virtual information and control a virtual machine, it resets that virtual machine only servers and providing the capability for high-speed and switches over to the backup computing hardware switchover. so as to minimize the impact on execution by other

80. FEATURED ARTICLES

Figure 6 — Enhancements to Availability, Fault Analysis, and Maintenance Use of the reset mechanism enables high-speed switchover, the integrated trace mechanism facilitates analysis, and the resource allocation mechanism makes for easier maintenance.

(1) High availability Minimize impact on Redundant configuration other virtual machines Reset of individual Switchover of individual for individual virtual virtual machines virtual machines machines

Realtime virtualization servers Realtime virtualization servers for for control applications control applications

Realtime Virtual Virtual Realtime Virtual Virtual virtualization machine 1 machine 2 virtualization machine 1 machine 2 platform platform Active Active Standby system Virtual machine Virtual machine → Standby system reset mechanism system reset mechanism Active system system Fault

Integrated trace Software A Software B Integrated trace mechanism mechanism Software A Software B

Operation trace Operation trace Operation trace Operation trace Operation trace Operation trace

Virtual machine Virtual machine backup mechanism OS OS backup mechanism OS OS

Resource allocation Resource allocation mechanism mechanism

Hardware Hardware

Reset Reset mechanism mechanism

(2) Easier fault analysis (3) Ability to maintain virtual t1: Transmit (Virtual machine 2) machines individually t2: Transmit (Virtualization) Trace Time-series t3: Write (Virtual machine 1) Backup of individual collection display t4: Receive (Virtualization) virtual machines t5: Write (Virtualization)

virtual machines on the same computing hardware 3. 3 that do not have a fault. Th is provides a switchover Enabling Fast and Reliable Maintenance of mode whereby switchover is not performed for the Individual Virtual Machines other virtual machines that do not have a fault [see Th e fact that server virtualization involves running the Figure 6 (1)]. virtual machines and virtualization platform in paral- Operationally, a common maintenance practice for lel makes fault analysis more diffi cult, including when preventing interference with the continuity of on- execution delays occur. To deal with this, the realtime site system operation when information and control virtualization platform is equipped with an integrated servers need to be shut down for maintenance work, trace mechanism that can present operation trace data such as replacing software, is for the servers to be shut in time-series format. Th is provides a unifi ed overview down in turn, one at a time, with one of the multiple of the operation of each virtual machine and of the servers continuing to run at all times. To allow for this, realtime virtualization platform as well as the opera- the realtime virtualization platform has the ability to tion of other virtual machines, making it possible to manually shut down individual virtual machines. rapidly identify the source of execution delays [see Figure 6 (2)].

Hitachi Review Vol. 67, No. 7 844–845 81. As separate information and control servers are Authors Kazuho Nakajima used for each control and other function in an infor- Signal System Planning Division, Tohoku mation and control system, the job of backing up Construction Off ice, East Japan Railway Company. Current work and research: Design of traff ic software includes making frequent system backups management systems. of these servers. Because it is likely when running multiple virtual machines that software maintenance will be needed on these individually, the realtime vir- Noriyuki Kikuchi Sendai Electric Construction Division, Tohoku tualization platform includes the ability to perform Construction Off ice, East Japan Railway Company. Current work and research: Construction supervision system backups separately for each virtual machine of traff ic management systems. [see Figure 6 (3)]. As backups typically involve copying large amounts of data from disk to backup storage, this needs to be Yoshitaka Ishikawa Signal System Planning Division, Tohoku done in a way that does not interfere with the opera- Construction Off ice, East Japan Railway Company. tion of other currently executing virtual machines. Current work and research: Design of traff ic management systems. To achieve this, the resource allocation mechanism referred to earlier includes a mode that limits resource use by backup operations, including processor time Sadao Niitsuma Transportation Control Systems Engineering and disk access bandwidth. Th is enables maintenance Department, Information & Control Systems Division work to be performed rapidly and reliably on indi- 2, Control System Platform Division, Services & Platforms Business Unit, Hitachi, Ltd. Current vidual virtual machines. work and research: Development of railway traff ic management systems.

Takeshi Takebayashi 4. Conclusions Control System Platform Design Department, Control System Platform Development Division, Control System Platform Division, Services & Platforms Business Unit, Hitachi, Ltd. Current work and Th is article has presented a case study of Hitachi’s research: Development of middleware for control fi rst ever use of a virtualization platform for a traffi c systems.

management system on a conventional railway line, Takuya Tsujikawa and described how the platform achieves realtime per- Transport Management Systems and Solutions Department, Transportation Systems Division, formance, high availability, and ease-of-maintenance. Railway Systems Business Unit, Hitachi, Ltd. Current work and research: Management of railway traff ic In the future, Hitachi intends to continue adapting systems. to new technology, reducing life cycle costs, and devel- oping technology with the aim of supplying high- quality social infrastructure that is safe and secure.

Reference 1) K. Shimizu et al., “Information and Control Platforms for Globalization and Enhancement of Service Extensibility,” Hitachi Review, 63, pp. 519–528 (Sep. 2014).

82.